React Native SDK leads to unexpected `on('change', ...)` calls reverting previous flag evaluations

[tiagoalves]

Describe the bug

I'm using the React Native SDK and monitoring flag updates with the client.on('change', ...) event. This event is called more times than the respective identify() calls.

I'm aware I should be using useBoolVariation() but for internal reasons I can't use that.

To reproduce

I've created this repo to reproduce this: https://github.com/tiagoalves/launchdarkly-issue

Instructions are in the README. I've shared our Mobile Client id with your support team.

This is the typical behaviour I'm seeing:

• identify(A)
• on('change', ...) called for context A (as expected)
• identify(B)
• on('change', ...) called for context B (as expected)
• on('change', ...) called for context A (unexpected)
• on('change', ...) called for context B (unexpected)

Sometimes change is only called twice (as expected). Sometimes 3 times, 4 times or more.

I've also resorted to using a promise chain for the consecutive identify() calls but even that didn't help.

Expected behavior

I'd expect the change event to exactly match the identify() calls.

• identify(A)
• on('change', ...) called for context A
• identify(B)
• on('change', ...) called for context B

Logs

In the logs we can see when the identify promise chain is started (LdTest calling identify log) which leads to two consecutive identify calls. This leads to two expected change triggers (LdTest ON CHANGE logs). But then two more change are received reverting the correct flag evaluation.

 (NOBRIDGE) LOG  LdTest MiCA Banner value: {"reason": "LD client not initialized", "value": false}
 (NOBRIDGE) LOG  LdTest MiCA Banner value: {"reason": {"errorKind": "USER_NOT_SPECIFIED", "kind": "ERROR"}, "value": false}
 (NOBRIDGE) LOG  LdTest calling identify
 (NOBRIDGE) LOG  LdTest ON CHANGE {"deployment": {"key": "unknown", "kind": "deployment", "timestamp": -1}, "kind": "multi", ... "user": {"country": "UNDEFINED", "custom": {"country": "UNDEFINED", "deviceId": "7964fa47-f9c3-44a6-9541-3ea1eb055134", "state": "UNDEFINED"}, "key": "anonymous", "kind": "user"}}
 (NOBRIDGE) LOG  LdTest MiCA Banner value: {"reason": null, "value": true}
 (NOBRIDGE) LOG  LdTest ON CHANGE {"deployment": {"key": "unknown", "kind": "deployment", "timestamp": -1}, "kind": "multi", ... "user": {"country": "UNDEFINED", "custom": {"country": "UNDEFINED", "deviceId": "7964fa47-f9c3-44a6-9541-3ea1eb055134", "state": "UNDEFINED"}, "key": "anonymous", "kind": "user"}}
 (NOBRIDGE) LOG  LdTest MiCA Banner value: {"reason": null, "value": false}
 (NOBRIDGE) LOG  LdTest ON CHANGE {"account": {"accountId": "f127648c-1532-4f39-aafb-66576272aee7", ..., "user": {"country": "PRT", "custom": {"accountId": "f127648c-1532-4f39-aafb-66576272aee7", "accountName": "AccountName", "country": "PRT", "deviceId": "7964fa47-f9c3-44a6-9541-3ea1eb055134", "organizationId": "880802a8-234f-44b6-9bca-c27c2d4fe693", "organizationName": "OrgName", "state": "UNDEFINED"}, "email": "user@example.com", "isNew": false, "key": "f9b59c77-99bc-4fdd-9548-54508d3cf2d5", "kind": "user"}}
 (NOBRIDGE) LOG  LdTest MiCA Banner value: {"reason": null, "value": true}

SDK version

10.10.0

Language version, developer tools

• React Native: 0.76.9
• Expo: 52.0.46
• TypeScript: 5.3.3

OS/platform

macOS 15.4.1

Additional context

All of this works as expected using the regular JS SDK. Seems to just be a problem with the React Native SDK.

[kinyoklion]

@tiagoalves Hello, than you for the report. There are a few things going on here and I want to cover some fundamentals before getting into problems. The change events are not 1:1 with identify events.

A single identify could cause:
A change event from loading from cache.
A change event from getting the fresh flag values.
And then subsequently N change events as updates are received through streaming.

The change event will happen if there are any changes to the flag, not just the value. This is because there is data associated with analytic events that require flag re-evaluation. For example the flag could have the same treatment, but have been added to an experiment.

Currently it is important to not overlap identify calls, as it may result in some unexpected behavior.

This behavior though, is not expected:

on('change', ...) called for context B (as expected)
on('change', ...) called for context A (unexpected)
on('change', ...) called for context B (unexpected)

Assuming that the identify operations where not started in an overlapping fashion, then you should be getting updates only related to B.

We will take a look. Unrelated to the problem, is there a reason for the event listeners versus the hooks which also re-actively update? Or was this just for debugging purposes?

Thank you,
Ryan

Filed internally as SDK-1215

[tiagoalves]

Thanks for getting back to me so quickly. Ack on the fundamentals, they make sense to me.

Currently it is important to not overlap identify calls, as it may result in some unexpected behavior.

After some tests I became aware that the LD library seems to consider the last-finished evaluation (evalx) request, even if it was started before a subsequent request. If that's the case, it would be great if the library could handle that and discard previous requests when a new one starts.

In any case, it's because of that that I resorted to a promise chain as you can see here in my example repo. I'm pretty sure requests are not overlapping as I also logged when those identify promises were starting and completing.

Assuming that the identify operations where not started in an overlapping fashion, then you should be getting updates only related to B.

Great, that's what I was expecting.

Unrelated to the problem, is there a reason for the event listeners versus the hooks which also re-actively update? Or was this just for debugging purposes?

We're working on a React package that can run by itself (on the web and React Native) and can also be embedded within another React app where we're already using the LaunchDarkly React SDK. I'm under the impression that we can't have multiple LDProvider within the same app so we just created our own "React layer" (provider, hooks, etc) based on the change event, among other things.

Happy to change this in the medium term if there's a better solution but right now we'd need to work with the solution we have.

[kinyoklion]

If that's the case, it would be great if the library could handle that and discard previous requests when a new one starts.

That is a similar approach to that we take in some other SDKs (the first being flutter), but unfortunately has not been implemented in this one yet.

Basically an identify queue with a depth of 1. Where you have 1 active identify and one pending. Calling identify a third time would shed the pending and replace it.

In regards to the event handlers, their usage is fine. I was just curious. Unfortunately when this SDK was implemented the constructor of the implementation type was directly exposed instead of a factory that returns the interface. I noticed the interface itself doesn't expose these events. Which likely needs rectified.

Thank you,
Ryan

[kinyoklion]

@tiagoalves I am not seeing similar when directly interacting with the API of the client.

featureClient.on('change', (context: LDContext, flagKeys: string[]) => {
  console.log(`featureClient.on('change'`, context, flagKeys, ');');
});

featureClient.identify({
  kind: 'user',
  key: 'test-user',
}).then(() => {
  featureClient.identify({
    kind: 'user',
    key: 'bob',
  }).then(() => {
    featureClient.identify({
      kind: 'user',
      key: 'alice',
    });
  });
});

In this case I have used users that will be affected by targeting rules to ensure changes.

Which results in 3 change events in order for the correct contexts.

 LOG  featureClient.on('change' {"kind": "multi", "ld_application": {"envAttributesVersion": "1.0", "id": "ld-rn-test-app", "key": "KDz0jleNHl3uVZkLPXLOrxB36aL0lPbRQ+IL28tIJXc=", "locale": "en_US", "version": "0.0.1"}, "ld_device": {"envAttributesVersion": "1.0", "key": "c786f8ee-0d67-43ab-8b0a-35c8592fe530", "manufacturer": "Google", "model": "sdk_gphone_x86", "os": {"family": "android", "name": "android", "version": "30"}}, "user": {"key": "test-user"}} ["cat", "is-prereq", "my-boolean-flag", "has-prereq-depth-3", "killswitch", "string-flag", "greekTest", "has-prereq-depth-1", "json", "HasBob", "has-prereq-depth-2"] );

 LOG  featureClient.on('change' {"kind": "multi", "ld_application": {"envAttributesVersion": "1.0", "id": "ld-rn-test-app", "key": "KDz0jleNHl3uVZkLPXLOrxB36aL0lPbRQ+IL28tIJXc=", "locale": "en_US", "version": "0.0.1"}, "ld_device": {"envAttributesVersion": "1.0", "key": "c786f8ee-0d67-43ab-8b0a-35c8592fe530", "manufacturer": "Google", "model": "sdk_gphone_x86", "os": {"family": "android", "name": "android", "version": "30"}}, "user": {"key": "bob"}} ["string-flag", "HasBob"] );

 LOG  featureClient.on('change' {"kind": "multi", "ld_application": {"envAttributesVersion": "1.0", "id": "ld-rn-test-app", "key": "KDz0jleNHl3uVZkLPXLOrxB36aL0lPbRQ+IL28tIJXc=", "locale": "en_US", "version": "0.0.1"}, "ld_device": {"envAttributesVersion": "1.0", "key": "c786f8ee-0d67-43ab-8b0a-35c8592fe530", "manufacturer": "Google", "model": "sdk_gphone_x86", "os": {"family": "android", "name": "android", "version": "30"}}, "user": {"key": "alice"}} ["HasBob", "string-flag"] );

I've not tested your example directly yet. But that does make me wonder if there is a react rendering related situation here instead of the SDK specifically. Like strict mode rendering executing things twice resulting in many overlapping requests.

I'll look at your example now and see if I note anything.

Thanks,
Ryan

[kinyoklion]

@tiagoalves Before I run test projects I need to audit them, and there is quite a bit going on inside your test project, so I am not willing to directly run it.

What I would suggest is validating that each hook is running the content you expect one time. For the useEffect hooks I would consider returning cleanup methods that also log to see the lifecycle and wrapping each identify in a before/after log to validate the ordering. Also setting debug:true in the SDK options will produce verbose logging that will make things more apparent. For example when we are starting/stopping connections and the keys in that collection.

Alternatively an example with less code, which is based on the example included in the SDK, would be easier for me to audit and run.

[tiagoalves]

Before I run test projects I need to audit them, and there is quite a bit going on inside your test project, so I am not willing to directly run it.

Totally understandable. I've updated the example repo so that the reproduction code can easily be moved to a clean Expo project. Instructions are here.

However, in the meantime I also ran your example project and indeed I'm not able to reproduce the issue by default. But, after lots of tinkering I found that I can reproduce it if I change the client initialization to use initialConnectionMode: 'polling'. Conversely, if I change my example repro code to use initialConnectionMode: 'streaming' I cannot reproduce it anymore.

Also, the issue does not happen with any type of connection mode in a production build. So, it's most probably something related to Expo dev. But running my example code, I verified that hooks are not re-rendered nor ever unmounted. The LD lib is only initialized once, identify calls are done in sequence and all I'm doing is imperatively call .variationDetail() whenever a change event is triggered. So, I'm always just asking the LD lib to tell me the latest evaluated variant -- without caching anything on my side -- and I get back previous evaluated variants unexpectedly.

Anything you could think of that leads the library to revert to previous evaluations in polling mode? Side note: I think that the issue is reproducible even if we create a new LD client instance for each identify() call. So maybe something in a shared cache?

In summary (edited after more tests):

initialConnectionMode	Dev Build	Prod Build
polling	❌ Unexpected evaluations even in small app	✅ Works fine in small app ❌ Unexpected evaluations in larger app
streaming	✅ Works fine in small app ❌ Unexpected evaluations in larger app	✅ Works fine in small app ❌ Unexpected evaluations in larger app

In the meantime I will change the connection mode to streaming on our side. Please let me know if there are any cons to this in a RN app.

[kinyoklion]

Hello @tiagoalves,

Thank you for the detailed write up.

We have had some problems with the development tooling around React Native before, specifically with the code that intercepts network requests for debugging. The issues had only been with streaming before, because streaming requests would never receive data. Basically the tooling intercepts requests, reads the entire response, then forwards that response on. You cannot read the entire stream, so it would fail. So we have some things here we can look at to see if there is anything unexpected.

Our cache is per environment, so if you initialize multiple SDKs using the same environment, then they will use the same cache and it generally wouldn't be recommended.

We can investigate what is going on in the development configuration.

As another aside, is there a specific reason you were defaulting to polling?

Thank you,
Ryan

[tiagoalves]

We can investigate what is going on in the development configuration.

Thanks! I've tested this more and I can also reproduce it in our main app in a production build -- even though it's rare. I've updated my summary table above.

I've already tried to make flag evaluation as reliable as possible on our layer (with eg promise chaining), and it improved significantly, but sometimes I still get out of order flag re-evaluations. I suspect this is leading to experiment imbalances and people ending up with the incorrect variants, in our case.

I'm also out of ideas of how to mitigate it on our side other than adding delays between identify() calls, which wouldn't be a viable solution.

If you think this can be an issue on our side, let me know and I will investigate more. Otherwise, we'd be thrilled if you could look into this! 🙏

As another aside, is there a specific reason you were defaulting to polling?

Not sure but I guess this is the default behaviour we've always had in our web app so we ported that to our RN app.

[kinyoklion]

@tiagoalves I am doing some testing with a couple PRs that improve the robustness of the identify process. This makes the identify operations queued, and it also disconnects update handlers for closed data sources.

[tiagoalves]

Thanks for the update @kinyoklion. I'm very much looking forward to this as experimentation is currently a bit on hold on our side. I'd happy to help test both #842 and #867 if you think that would help.

[kinyoklion]

@tiagoalves If you want to test #842 that would be excellent. I merged #867, so I will make a release with that shortly.