🐛 [firestore-bigquery-export] schema-views script doesn't work against multiple schema files

[KevinKera]

Step 2: Describe your configuration

• Extension name: firestore-bigquery-export (schema-views script)
• Extension version: 0.1.49
• Configuration values (redact info where appropriate):
  • Cloud Functions location : us-central1
  • BigQuery Dataset location : us
  • BigQuery Project ID: xxx
  • Database ID: (default)
  • Collection path: xxx
  • Enable Wildcard Column field with Parent Firestore Document IDs(Optional): false
  • Dataset ID: xxx
  • Table ID: xxx
  • BigQuery SQL table Time Partitioning option type(Optional): NONE
  • BigQuery Time Partitioning column name(Optional): Parameter not set
  • Firestore Document field name for BigQuery SQL Time Partitioning field option(Optional): Parameter not set
  • BigQuery SQL Time Partitioning table schema field(column) type(Optional): omit
  • BigQuery SQL table clustering(Optional): Parameter not set
  • Maximum number of synced documents per second(Optional): 100
  • Backup Collection Name(Optional): failed_bq_updates
  • Transform function URL(Optional): Parameter not set
  • Use new query syntax for snapshots: yes
  • Exclude old data payloads(Optional): yes
  • Use Collection Group query(Optional): no
  • Cloud KMS key name(Optional): Parameter not set
  • Enabled events: firebase.extensions.big-query-export.v1.sync.start
  • fs-bq-schema-views command:

npx @firebaseextensions/fs-bq-schema-views \
  --non-interactive \
  --project=xxx \
  --dataset=xxx \
  --table-name-prefix=xxx \
  --schema-files=./part_schema_all.json,./part_schema_consented.json,./part_schema_opposed.json

Step 3: Describe the problem

Steps to reproduce:

Setup the extension as mentioned in the documentation. Follow the instructions found in the GENERATE_SCHEMA_VIEWS guide until step 3.

There, the note says

Note: You can run the schema-views script from any directory, but you need to specify the path to your schema file using the --schema-files flag. To run the schema-views script against multiple schema files, specify each file in a comma-separated list (for example: --schema-files=./test_schema.json,./test_schema2.json).

However, when trying to run the script against multiple schema files, it doesn't work and it says the schema files aren't found. When running the same script several times keeping only one schema file (eg. --schema-files=./part_schema_all.json then --schema-files=./part_schema_consented.json), it works though. (this is the only change)

Expected result

Terminale output should be something like

BigQuery creating schema view participants_schema_part_schema_all_changelog:
# parameters
BigQuery created schema view participants_schema_part_schema_all_changelog
...
BigQuery creating schema view participants_schema_part_schema_all_latest:
# parameters
BigQuery created view participants_schema_part_schema_all_latest

BigQuery creating schema view participants_schema_part_schema_consented_changelog:
# parameters
BigQuery created schema view participants_schema_part_schema_consented_changelog
...
BigQuery creating schema view participants_schema_part_schema_consented_latest:
# parameters
BigQuery created view participants_schema_part_schema_consented_latest
done.

Actual result

No schema files found!
done.

[Gustolandia]

Problem fixed in #2223

Awaiting merge into next

[cabljac]

keeping open until merged and released

[Gustolandia]

Changing status back to done, as the issue is fixed, just waiting to merge #2223

[Gustolandia]

Merged