BigQuery: ensure that KeyboardInterrupt during to_dataframeno longer hangs.

[tswast]

I noticed in manually testing to_dataframe that it would stop
the current cell when I hit Ctrl-C, but data kept on downloading in the
background. Trying to exit the Python shell, I'd notice that it would
hang until I pressed Ctrl-C a few more times.

Rather than get the DataFrame for each stream in one big chunk, loop
through each block and exit if the function needs to quit early. This
follows the pattern at https://stackoverflow.com/a/29237343/101923

This depends on:

• Add page iterator to ReadRowsStream #7680, new .pages feature in BQ Storage client
• BigQuery: Raise ValueError when BQ Storage is required but missing #7726, aligns optional import for BQ Storage client with pattern used elsewhere in the client.

[tswast]

✅ Note to self: Need to update the minimum google-cloud-bigquery-storage dependency version after the release at #7716 is complete.

[shollyman]

Mostly I'm trying to reason about scope here. This is a worker pool, but is it possible we're generating multiple independent dataframes that would share the same access to _to_dataframe_finished? Do we need to key the workers to specific invocations, or is the nature of the access always blocking so that this isn't an issue?

[tswast]

There are a couple reasons that I don't think it's an issue:

• Yes, to_dataframe() is a blocking call, so you wouldn't really have multiple going at once.
• RowIterator isn't really something you'd want to use across threads or even more than once anyway. Because of how the pagination state works, once you loop through all the rows once, to_dataframe() returns an empty DataFrame, even in the case where BQ Storage API isn't used.