Preserve order in BigQuery JSON

[dhermes]

See #2342. Can be done using json.loads(..., object_pairs_hook=OrderedDict) h/t to @c0b

• It's unclear if any methods "need" this behavior.
• It's nice that we don't support Python 2.6 any more because collections.OrderedDict was introduced in 2.7
• If this "feature" is actually worth it, we should determine which subset of payloads need this behavior

---

Here are all the call sites where JSON is parsed from a string to a Python dictionary:

$ git grep -n 'json.loads(' -- google/cloud/bigquery/
google/cloud/bigquery/table.py:961:        return client.job_from_resource(json.loads(response_content))
$
$ git grep -n 'api_request' -- google/cloud/bigquery/
google/cloud/bigquery/client.py:105:        resp = self.connection.api_request(method='GET', path=path,
google/cloud/bigquery/client.py:148:        resp = self.connection.api_request(method='GET', path=path,
google/cloud/bigquery/client.py:239:        resp = self.connection.api_request(method='GET', path=path,
google/cloud/bigquery/dataset.py:424:        api_response = client.connection.api_request(
google/cloud/bigquery/dataset.py:445:            client.connection.api_request(method='GET', path=self.path,
google/cloud/bigquery/dataset.py:465:        api_response = client.connection.api_request(
google/cloud/bigquery/dataset.py:504:        api_response = client.connection.api_request(
google/cloud/bigquery/dataset.py:520:        api_response = client.connection.api_request(
google/cloud/bigquery/dataset.py:536:        client.connection.api_request(method='DELETE', path=self.path)
google/cloud/bigquery/dataset.py:569:        resp = connection.api_request(method='GET', path=path,
google/cloud/bigquery/job.py:318:        api_response = client.connection.api_request(
google/cloud/bigquery/job.py:339:            client.connection.api_request(method='GET', path=self.path,
google/cloud/bigquery/job.py:359:        api_response = client.connection.api_request(
google/cloud/bigquery/job.py:376:        api_response = client.connection.api_request(
google/cloud/bigquery/query.py:335:        api_response = client.connection.api_request(
google/cloud/bigquery/query.py:392:        response = client.connection.api_request(method='GET',
google/cloud/bigquery/table.py:28:from google.cloud.streaming.http_wrapper import make_api_request
google/cloud/bigquery/table.py:490:        api_response = client.connection.api_request(
google/cloud/bigquery/table.py:511:            client.connection.api_request(method='GET', path=self.path,
google/cloud/bigquery/table.py:531:        api_response = client.connection.api_request(
google/cloud/bigquery/table.py:605:        api_response = client.connection.api_request(
google/cloud/bigquery/table.py:621:        api_response = client.connection.api_request(
google/cloud/bigquery/table.py:637:        client.connection.api_request(method='DELETE', path=self.path)
google/cloud/bigquery/table.py:681:        response = client.connection.api_request(method='GET',
google/cloud/bigquery/table.py:767:        response = client.connection.api_request(
google/cloud/bigquery/table.py:955:            http_response = make_api_request(connection.http, request,
$
$ # current hash
$ git log -1 --pretty=%H
d3e28ce8ae5bee1a637a11d11227d294cfa8302c

[c0b]

#2342 (comment)

please fix your internal bq tool in the google cloud SDK. or I would say anyone here want that prettyjson format would like the keys in preserved order

[dhermes]

The maintainers of this repo don't have any pull with the maintainers of gcloud SDK (I don't even know who they are). So we can't get them fixed there.

Is it OK if we close this out @c0b? You can file an issue with them at https://code.google.com/p/google-cloud-sdk/issues/list

[c0b]

or I would say anyone here want that prettyjson format would like the keys in preserved order

users of this library would like to see a preserved keys order option; this is still a valid enhancement request

[dhermes]

Which users?

[c0b]

users of this library

[dhermes]

What I mean to say is that we haven't heard the request from anyone but you and from your previous comments it doesn't seem like you use this library?

[c0b]

I am a potential one, if the library is made really good to use; right now #2354 is a blocker to me, because I have been using Standard SQL nested fields a lot; that is a must fix; but this one is a feature nice to have:

can I request API design like this?

>>> query = client.run_query('SELECT ...', use_legacy_sql=False, use_ordered_dict=True)
>>> query.run()
>>> for row in query.rows:
             ...

[lukesneeringer]

Hello,
One of the challenges of maintaining a large open source project is that sometimes, you can bite off more than you can chew. As the lead maintainer of google-cloud-python, I can definitely say that I have let the issues here pile up.

As part of trying to get things under control (as well as to empower us to provide better customer service in the future), I am declaring a "bankruptcy" of sorts on many of the old issues, especially those likely to have been addressed or made obsolete by more recent updates.

My goal is to close stale issues whose relevance or solution is no longer immediately evident, and which appear to be of lower importance. I believe in good faith that this is one of those issues, but I am scanning quickly and may occasionally be wrong. If this is an issue of high importance, please comment here and we will reconsider. If this is an issue whose solution is trivial, please consider providing a pull request.

Thank you!