Skip to content

Fix the bug that affected the DAG end date. #36144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
eladkal merged 1 commit into from
Feb 9, 2024
Merged

Fix the bug that affected the DAG end date. #36144

eladkal merged 1 commit into from
Feb 9, 2024

Conversation

@avkirilishin
Copy link
Contributor

@avkirilishin avkirilishin commented Dec 9, 2023

Closes: #31555

The state management has been relocated from the mark_task module to the DagRun's set state.
Changes to attributes were implemented in accordance with the following table:

Old State \ New State QUEUED RUNNING SUCCESS FAILED
None queued_at = timezone.utcnow() if empty: start_date = timezone.utcnow()
end_date = None		 end_date = timezone.utcnow() end_date = timezone.utcnow()
QUEUED queued_at = timezone.utcnow() if empty: start_date = timezone.utcnow()
end_date = None		 end_date = timezone.utcnow() end_date = timezone.utcnow()
RUNNING queued_at = timezone.utcnow()
start_date = None
end_date = None		 end_date = timezone.utcnow() end_date = timezone.utcnow()
SUCCESS queued_at = timezone.utcnow()
start_date = None
end_date = None		 start_date = timezone.utcnow()
end_date = None
FAILED queued_at = timezone.utcnow()
start_date = None
end_date = None		 start_date = timezone.utcnow()
end_date = None

Now it's look like:
screen-recording-2023-12-09-at-192842

@boring-cyborg boring-cyborg bot added the area:API Airflow's REST/HTTP API label Dec 9, 2023
@avkirilishin avkirilishin reopened this Dec 9, 2023
@avkirilishin avkirilishin force-pushed the 31555_bug_changed_end_date_after_manually_marking branch from 60493e6 to fd9d0b0 Compare December 25, 2023 12:23
@eladkal eladkal added this to the Airflow 2.8.1 milestone Dec 25, 2023
@eladkal eladkal added the type:bug-fix Changelog: Bug Fixes label Dec 25, 2023
@eladkal eladkal requested a review from uranusjr December 28, 2023 13:05
@potiuk
Copy link
Member

potiuk commented Jan 5, 2024

I think it would be great is some of us more familiar with the dates/scheduling/dag state would look at this one : It looks legit and I like the table description clarification, but I am not sure if I am the right one to validate it :)

ephraimbuddy
ephraimbuddy reviewed Jan 14, 2024
View reviewed changes
@eladkal eladkal force-pushed the 31555_bug_changed_end_date_after_manually_marking branch from e391f6e to 56769c2 Compare February 6, 2024 12:28
@eladkal eladkal merged commit 9f4f208 into apache:main Feb 9, 2024
jedcunningham pushed a commit that referenced this pull request Feb 9, 2024
@avkirilishin @jedcunningham
Fix the bug that affected the DAG end date. (#36144)
be858a5 
(cherry picked from commit 9f4f208)
sc250072 pushed a commit to Teradata/airflow that referenced this pull request Feb 9, 2024
@satish-chinthanippu
Squashed commit of the following:
db35783 
commit 58279b8
Author: Satish Ch <satishchinthanippu@gmail.com>
Date:   Sat Feb 10 00:12:42 2024 +0530

    Update teradata.py

commit ed75e3e
Author: Satish Ch <satishchinthanippu@gmail.com>
Date:   Sat Feb 10 00:09:24 2024 +0530

    D401 support in Teradata Provider

commit f56bede
Merge: e859a1d 00ed467
Author: Satish Ch <satishchinthanippu@gmail.com>
Date:   Fri Feb 9 23:58:15 2024 +0530

    Merge remote-tracking branch 'upstream/main' into pr_teradata_release_1.0.0

commit 00ed467
Author: Vincent <97131062+vincbeck@users.noreply.github.com>
Date:   Fri Feb 9 13:13:36 2024 -0500

    D401 support in fab provider (apache#37283)

commit 48bfb1a
Author: Niko Oliveira <onikolas@amazon.com>
Date:   Fri Feb 9 08:43:32 2024 -0800

    Merge all ECS executor configs following recursive python dict update (apache#37137)

    Also document the behaviour and interaction between exec_config and
    run_task_kwargs config

commit 8317ed9
Author: Amogh Desai <amoghrajesh1999@gmail.com>
Date:   Fri Feb 9 20:17:38 2024 +0530

    Updating the README and visuals for breeze build-docs (apache#37276)

commit 17945fc
Author: Kalyan <kalyan.ben10@live.com>
Date:   Fri Feb 9 20:16:33 2024 +0530

    D401 fixes in Pinecone provider (apache#37270)

commit ab9e2e1
Author: Kalyan <kalyan.ben10@live.com>
Date:   Fri Feb 9 20:15:31 2024 +0530

    fix: D401 lint issues in airflow core (apache#37274)

commit 7835fd2
Author: Jarek Potiuk <jarek@potiuk.com>
Date:   Fri Feb 9 14:59:33 2024 +0100

    The fix-ownership command missed --rm flag and left dangling containers (apache#37277)

    Fixes: apache#37269

commit e859a1d
Author: Satish Ch <satishchinthanippu@gmail.com>
Date:   Fri Feb 9 19:21:28 2024 +0530

    Update teradata.py

commit d1c08e1
Author: Satish Ch <satishchinthanippu@gmail.com>
Date:   Fri Feb 9 19:18:37 2024 +0530

    Update teradata.py

commit e5ac0ec
Author: Satish Ch <satishchinthanippu@gmail.com>
Date:   Fri Feb 9 05:41:42 2024 -0800

    static check issue is fixed

commit ce490f7
Author: Satish Ch <satishchinthanippu@gmail.com>
Date:   Fri Feb 9 19:10:36 2024 +0530

    static format issue fixed

commit f9498c5
Author: Satish Ch <satishchinthanippu@gmail.com>
Date:   Fri Feb 9 05:36:36 2024 -0800

    static check issue is fixed

commit 77bddae
Author: Satish Ch <satishchinthanippu@gmail.com>
Date:   Fri Feb 9 18:23:18 2024 +0530

    common sql unit test failure fixed

commit 9f4f208
Author: Aleksey Kirilishin <54231417+avkirilishin@users.noreply.github.com>
Date:   Fri Feb 9 15:53:04 2024 +0300

    Fix the bug that affected the DAG end date. (apache#36144)

commit 0f8dfeb
Author: Sudipto Baral <sudiptobaral.me@gmail.com>
Date:   Fri Feb 9 06:54:23 2024 -0500

    fix: update  hyperlink to the new documentation section for local virtualenv setup. (apache#37272)

    Signed-off-by: sudipto baral <sudiptobaral.me@gmail.com>
potiuk pushed a commit that referenced this pull request Feb 13, 2024
@avkirilishin @potiuk
Fix the bug that affected the DAG end date. (#36144)
9a7d48c 
(cherry picked from commit 9f4f208)
ephraimbuddy pushed a commit that referenced this pull request Feb 22, 2024
@avkirilishin @ephraimbuddy
Fix the bug that affected the DAG end date. (#36144)
18ea9be 
(cherry picked from commit 9f4f208)
@ephraimbuddy ephraimbuddy mentioned this pull request Feb 22, 2024
Closed
68 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@eladkal eladkal eladkal approved these changes

@potiuk potiuk potiuk approved these changes

@pierrejeambrun pierrejeambrun Awaiting requested review from pierrejeambrun

@kaxil kaxil Awaiting requested review from kaxil

@XD-DENG XD-DENG Awaiting requested review from XD-DENG

@ashb ashb Awaiting requested review from ashb

@uranusjr uranusjr Awaiting requested review from uranusjr

@ephraimbuddy ephraimbuddy Awaiting requested review from ephraimbuddy

Assignees

No one assigned

Labels

area:API Airflow's REST/HTTP API type:bug-fix Changelog: Bug Fixes

Projects

None yet

Milestone

Airflow 2.8.2

Development

Successfully merging this pull request may close these issues.

Manually marking dag run state will change end date

4 participants

@avkirilishin @potiuk @ephraimbuddy @eladkal