Skip to content

Commit

Permalink
Apply specific ID collation to root_dag_id too
Browse files Browse the repository at this point in the history
In certain databases there is a need to set the collation for ID fields
like dag_id or task_id to something different than the database default.
This is because in MySQL with utf8mb4 the index size becomes too big for
the MySQL limits. In past pull requests this was handled
[apache#7570](apache#7570),
[apache#17729](apache#17729), but the
root_dag_id field on the dag model was missed. Since this field is used
to join with the dag_id in various other models ([and
self-referentially](https://github.com/apache/airflow/blob/451c7cbc42a83a180c4362693508ed33dd1d1dab/airflow/models/dag.py#L2766)),
it also needs to have the same collation as other ID fields.

This can be seen by running `airflow db reset` before and after applying
this change while also specifying `sql_engine_collation_for_ids` in the
configuration.

Other related PRs
[apache#19408](apache#19408)
  • Loading branch information
mpeteuil committed May 9, 2022
1 parent 7813f99 commit b3e6725
Showing 1 changed file with 3 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@
import sqlalchemy as sa
from alembic import op

from airflow.migrations.db_types import StringID

# revision identifiers, used by Alembic.
revision = 'b3b105409875'
down_revision = 'd38e04c12aa2'
Expand All @@ -37,7 +39,7 @@

def upgrade():
"""Apply Add ``root_dag_id`` to ``DAG``"""
op.add_column('dag', sa.Column('root_dag_id', sa.String(length=250), nullable=True))
op.add_column('dag', sa.Column('root_dag_id', StringID(), nullable=True))
op.create_index('idx_root_dag_id', 'dag', ['root_dag_id'], unique=False)


Expand Down

0 comments on commit b3e6725

Please sign in to comment.