Fix get BigQuery datasets in over 50 datasets environment

[katsugeneration]

What type of PR is this? (check all applicable)

• Bug Fix

Description

This implementation fixes getting BigQuery datasets for get_schema operation in over about 50 datasets environment.

BigQuery dataset list API supports pagination. But old implementation do not use it. It causes user to get only several datasets returned by an api call and not to get more than datasets.

New implementation use pagination for fixing it.

Related Tickets & Documents

Mobile & Desktop Screenshots/Recordings (if there are UI changes)

[susodapop]

Thanks for this contribution! I'm happy to merge, but I don't have an environment to test against BigQuery. Can you advise whether this is a drop-in replacement for the previous implementation? Will it break existing data sources?

[katsugeneration]

Thanks for review.

this is a drop-in replacement for the previous implementation?

Yes. This implementation only use same api and return dataset information in the same format for the previous implementation.

Will it break existing data sources?

No. This implementation only change the number of datasets for get_schema operation results by getting additional datasets if use have datasets over than 50.
So, if user will change to this implementation and will get schema first time, redash will behave in the same as being added datasets to BigQuery by user.

[susodapop]

LGTM.

[susodapop]

Thanks for an awesome contribution. This is huge for BQ users 👍