Skip to content

Subcollections not importing via the form #1980

Open
@sorrel-certifyos

Description

[READ] Step 1: Are you in the right place?

Issues filed here should be about bugs for a specific extension in this repository.
If you have a general question, need help debugging, or fall into some
other category use one of these other channels:

  • For general technical questions, post a question on StackOverflow
    with the firebase tag.
  • For general Firebase discussion, use the firebase-talk
    google group.
  • To file a bug against the Firebase Extensions platform, or for an issue affecting multiple extensions, please reach out to
    Firebase support directly.

[REQUIRED] Step 2: Describe your configuration

  • Extension name: firestore-bigquery-export

  • Extension version: 0.1.45

  • Configuration values (redact info where appropriate):

    • collection path my-collection/{myCollectionId}/my-subcollection
    • enable Wildcard Column field with Parent Firestore Document Ids: true
    • importing existing firestore documents into BigQuery: yes
    • existing documents collection collection path my-collection/{myCollectionId}/my-subcollection
    • use collection group query: yes
    • (I have tried both with and without a transform function URL with similar, albeit slightly different results)
    • location: us-central1
    • bigQuery dataset location: us
    • bigQuery project Id: my-project-id
    • dataset Id: my-dataset
    • table id: my-table-id
    • All other values left to default

[REQUIRED] Step 3: Describe the problem

When I configure the extension to a subcollection and select yes for importing existing firestore documents into BigQuery, I expect the subcollection documents to import. They do not. Newly created documents are added to the table in BigQuery via extension, so it appears specifically the import on the subcollection is not working.
I have tried both with and without a transform function. With the transform function, after about an hour I get a document count of how many documents were imported, but no documents were added to the big query table, and the transform function was never invoked per the logs on the transform function.
Without a transform function, the "runtime status" reads, "Configuring BigQuery Sync and running import if configured" but it never imports any documents (though newly created ones are added to the table), and the status does not change (it's been about 3 days now).

My expectation is that the subcollection documents would import into the big query table (as directly importing a collection does)

Steps to reproduce:

What happened? How can we make the problem occur?

Use a configuration similar to above, with your equivalent subcollection and db values
create a new extension

Expected result

Subcollection documents (which, the specific extension is set up only for the subcollection) will import into the extension's bigquery table. All of them. Ideally after being transformed via the transform cloud function specified in the configuration.

Actual result

Although the extension is created, and new documents are created in the raw_changelog table, nothing is imported and there is no feedback of error or issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions