BigQuery traceability labels were not available while using TaskGroup

[jacgeo]

Apache Airflow Provider(s)

google

Versions of Apache Airflow Providers

apache-airflow-providers-google==12.0.0

Apache Airflow version

2.9.3

Operating System

Ubuntu

Deployment

Virtualenv installation

Deployment details

No response

What happened

BigQueryInsertJobOperator set traceability job labels(airflow-dag & airflow-task) in BigQuery jobs. But, when using the BigQueryInsertJobOperator with TaskGroup, the traceability labels were not available

3e2bfb8

What you think should happen instead

traceability_labels = {"airflow-dag": dag_label, "airflow-task": task_label}

The traceability labels should be available for BigQueryInsertJobOperator with TaskGroup

How to reproduce

from datetime import datetime, timedelta
from airflow import DAG
from airflow.providers.google.cloud.operators.bigquery import BigQueryInsertJobOperator
from airflow.operators.dummy import DummyOperator
from airflow.utils.task_group import TaskGroup


default_args = {
    "retries": 1,
    "retry_delay": timedelta(minutes=1),
}


with DAG(
    "test_dag",
    default_args=default_args,
    description="BQ test DAG",
    schedule_interval=timedelta(days=1),
    start_date=datetime(2025, 3, 2),
    catchup=False,
) as dag:
    start = DummyOperator(task_id="start")
    with TaskGroup("taskgroup_1", tooltip="task group #1") as section_1:
        t1 = BigQueryInsertJobOperator(
            task_id="insert_job_1",
            configuration={
                "query": {
                    "query": "SELECT * FROM `project.dataset.table_1`;",
                    "useLegacySql": False,
                }            },
            gcp_conn_id="google_cloud_default",
        )
        t2 = BigQueryInsertJobOperator(
            task_id="insert_job_2",
            configuration={
                "query": {
                    "query": "SELECT * FROM `project.dataset.table_1`;",
                    "useLegacySql": False,
                },
            },
            gcp_conn_id="google_cloud_default",
        )

    end = DummyOperator(task_id="end")

    start >> section_1 >> end

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[RNHTTR]

Can you please test this on the latest version of the bigquery provider (14.0.0)?

[jacgeo]

@RNHTTR
Tested with below version. BigQuery traceability labels were not available while using TaskGroup

• apache-airflow==2.9.3
• apache-airflow-providers-google==14.0.0

[eladkal]

cc @VladaZakharova

[JASNOTJAZZ]

How to Reproduce the Issue
Define a DAG with a TaskGroup containing BigQueryInsertJobOperator tasks.
Execute the DAG and check the BigQuery job metadata.
Observe that the labels "airflow-dag" and "airflow-task" are missing.

Possible Cause
The operator retrieves dag_id and task_id from the task instance, but TaskGroups modify task IDs by prefixing them with the group name (e.g., taskgroup_1.insert_job_1).
The traceability logic may not be properly handling this case when constructing labels.

Solution Approach
Modify BigQueryInsertJobOperator to handle TaskGroups properly when assigning traceability labels.
Ensure the full task ID (including TaskGroup name) is included in labels.
Test the fix using a DAG with a TaskGroup and verify that labels appear in BigQuery job metadata.

[punitchauhan771]

Hi,

It looks like the issue stems from having a period ('.') in the naming convention that separates the TaskGroup name from the Task ID (e.g., {tg.tid}). Could we switch that character from '.' to either '-' or '_' to avoid the conflict while writing it into BigQuery labels ?

google docs :

link

also is it fine if i work on this issue ?