Remote Config Parameters with Conditions are Inconsistent and Often Incorrect

[ewebfinity]

Describe your environment

• Operating System version: macOS 10.14.6 (18G5033) (Mojave)
• Browser version: Google Chrome (83.0.4103.106 (Official Build) (64-bit))
• Firebase SDK version: 7.15.1
• Firebase Product: remote-config, analytics

Describe the problem

When remote config parameters values are determined by a condition on a user property, the parameter values returned are incorrect or inconsistent. Sometimes they are correct and other times they are wrong. Once they are wrong they appear to get "stuck" in that incorrect state. No amount of refreshing or fetching will get them unstuck.

Steps to reproduce:

Note: jsbin contains fully working sample with playground firebase project

1. Create a user property called "testing_group"
2. Create a remote config parameter called "showDebugInfo" and make it's default to false
3. Create a condition on that parameter with the following condition:
   "testing_group" "contains regex" ".+"
   OR if you like:
   "testing_group" "contains" "dev"
4. In javascript, send up user properties of "testing_group = dev" and retrieve the parameter and it will sometimes be incorrect. It's flaky.

Often times the name of the user property won't even matter. It will return the wrong value.

Relevant Code:

Run this sample.........

Notice: If you get true on your first run and then comment out the code the sends up the user properties and firebase will still return true for that value even though it's conditioned on it being set.

https://jsbin.com/xijolazaka/1/edit?html,console,output

[google-oss-bot]

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

[Feiyang1]

I notice that you didn't send any event after setting the user property. On Web, you need to first send user properties to the analytics backend in order to use it in remote config conditions. This is how it's supposed to work with your example:

1. set user property testing_group on client and send any event to analytics.
2. An unique instance id and testing_group are sent to analytics backend and are recorded.
3. Retrieve parameters from remote config
4. The unique instance id is sent to remote config backend which talks to the analytics backend to check if the condition satisfies for this instance id. (We do some preprocessing so there is no real communication between remote config and analytics, so the lookup is fast)

To remove a user property for an instance id, you will set it to null, then send any event.

If you have used FirebaseAnalytics on Android or iOS, you might notice they work differently where user properties are sent along the remote config request. We are not able to do that on Web because we use gtag.js which doesn't expose an API for reading user properties set on client.

Hope it helps! Let me know if it doesn't explain your issue.

[ewebfinity]

@Feiyang1
That does clarify why it's working on our mobile app, thanks. I had wondered if you had to send up
an event to get the user properties to go up. Thanks for that clarification. Maybe add that to the docs?

So now, I've updated the sample here with a logEvent() and I'm still seeing false:

https://jsbin.com/diruxazigi/1/edit?html,console,output

Here's what I'm doing:

1. In a Chrome incognito window, open the url above
2. Open the developer console and observe the network tab and wait until it sends the user properties to Google analytics (seems to take 2-3 seconds to fire)
3. After the user properties are sent up, click the "Get Remote Config" button
4. Right now it's returning false for me which is incorrect.

Another scenario that's confusing:
Once you magically get it returning true, you can rename the user property to a junk value and it's still stuck in the "true" state.

Sometimes I will get a "true" value returned. In that case I'll change the code and rename the user property to a garbage name so it should always be false and repeat the steps above and it will still return a "true" value even though I'm sending up the wrong user property name. The network call itself will return as true which is wrong. It's as if it's cached somehow server-side even though I'm sending up a garbage user property name now.

[Feiyang1]

I got true now. I opened the link and waited for a couple of minutes before clicking the button. Note that though we process user properties in real time, there is still some delay before they become available for targeting. Can you try now? I believe you will get "true" now.

Another scenario that's confusing:
Once you magically get it returning true, you can rename the user property to a junk value and it's still stuck in the "true" state.

I believe it is working as designed (though I admit it can be confusing). When you rename the user property and send it again, you are appending a new property to the list of all user properties you have set previously. If you want to remove a user property, you need to set it to null and send it again.

I do think our documentation can be improved on explaining how things work, especially when it works differently on Web vs Android and iOS. I will keep the issue open til we update the documentation.

[ewebfinity]

@Feiyang1

Okay, the user property behavior makes more sense. So, what I'm gathering is when we send up the user properties it takes an indeterminate amount of time before they become "active" and so remote-config will return false until that time. Is that correct?

Right now, I do get true values part of the time but not ALL the time. It still seems inconsistent to me.

I've been trying to duplicate this behavior in a new jsbin sample:

https://jsbin.com/desurunaji/1/edit?html,console,output

I've changed the sample to have two buttons to set and nullify the user properties and I'm not able to get the parameters to flip between true and false after minutes of waiting between setting them. Maybe I'm just not patient enough but how long does this typically take?

Also it seems to rarely flip values in the same browser session. We have to open up an incognito window.