Double encoding url rejection

[timed0]

Hey, I am running pyrebase on python 2.7 (my bad!) but I thought that this might interest you anyway. It's possible the urlencoding/requests package behave differently on python3.

Today my requests to firebase began to be rejected for double-quoting of the query string. I guess this is a change on their end which is why I bring it up here.

the request from build_request_url:

http://..firebase./path.json?orderBy=%2522progress%2522&equalTo=0

you can see the " is encoded to %22 and then to %2522.
I replaced the urlencode of parameters wth an unencoded parameters string and it's better:

parameters_string = '&'.join('{0}={1}'.format(k,v) for k,v in parameters.iteritems())
request_ref = '{0}{1}.json?{2}'.format(self.database_url, self.path, parameters_string)

Maybe I should have just removed the quote() of the string parameter...

[timed0]

To follow up myself, I found this
https://firebase.google.com/docs/reference/rest/database/

Which states that change to parameter encoding from Feb 2019. Which is now. Perhaps the X-Firebase-Decoding header is the preferred fix.

[redaben]

@timed0 Having the same pb... Do you have a sample of code of pyrebase with X-Firebase-Decoding header inside ? thx ;)

[RichardForshaw]

Same problem popped up for me on production.

I applied the following workaround: On the file where you import pyrebase, replace the 'quote' function with one that does nothing, so that the resulting URL is only http-escaped once:

# Temporarily replace quote function
def noquote(s):
    return s
pyrebase.pyrebase.quote = noquote

Otherwise, the library should be updated to stop quoting of string parameters to reflect the change to the firebase API.

[sobeki]

I did change to this line,
parameters[param] = '"' + self.build_query[param] + '"'
removing the quote func and it has been working wonders. 👍

[RichardForshaw]

I did change to this line,
parameters[param] = '"' + self.build_query[param] + '"'
removing the quote func and it has been working wonders. 👍

Yes you can do that, but for users of the library it is more difficult to patch the library and deploy it than modify their own code, until the library is correctly patched and released.

[ronhe]

Note that @RichardForshaw's monkey patching solution breaks Storage.get_url method (which needs the quoting functionality).
So you may also need to monkey patch the get_url method (just a copy-and-paste of the original function but using a true quote function).
Here is the full proposed solution:

import pyrebase 
from urllib.parse import quote


# Monkey patch pyrebase: replace quote function in pyrebase to workaround a bug.
# See https://github.com/thisbejim/Pyrebase/issues/294.
pyrebase.pyrebase.quote = lambda s, safe=None: s


# Monkey patch pyrebase: the Storage.get_url method does need quoting :|
def get_url(self, token=None):
    path = self.path
    self.path = None
    if path.startswith('/'):
        path = path[1:]
    if token:
        return "{0}/o/{1}?alt=media&token={2}".format(self.storage_bucket, quote(path, safe=''), token)
    return "{0}/o/{1}?alt=media".format(self.storage_bucket, quote(path, safe=''))


pyrebase.pyrebase.Storage.get_url = lambda self, token=None: \
    get_url(self, token)