Skip to content

Conversation

@leakec
Copy link
Contributor

@leakec leakec commented Jul 19, 2025

The call to DateInterval in latest_only_opeartor is currently passing in a datetime where it should be passing in a DateTime. This PR fixes that issue.

closes: #50911

@leakec
Copy link
Contributor Author

leakec commented Jul 19, 2025

I've tested the fix out, and it resolves the issue I've run into. However, I'm having issues reproducing it in a unit test. I've made an attempt in the PR, but need some help from someone more familiar with airflow unit testing. I've kept the fix and unit test on separate commits to hopefully make things simpler.

Tagging @gopidesupavan for feedback on the PR itself and the unit test.

@gopidesupavan
Copy link
Member

@leakec LGTM, can you please fix the static checks and compat tests?

@gopidesupavan
Copy link
Member

@leakec
Copy link
Contributor Author

leakec commented Jul 24, 2025

@leakec LGTM, can you please fix the static checks and compat tests?

@gopidesupavan Sure thing. I've added a few more commits to patch the issues, and now it passes all the tests. I do still need some help with the unit test though. The one I added passes, but the problem is it doesn't fail before the fix I implemented, i.e., it passes with both versions, so it doesn't really replicate the problem I saw when running this DAG in a normal airflow instance:

from airflow import DAG
from datetime import datetime
from airflow.providers.standard.operators.bash import BashOperator
from airflow.providers.standard.operators.latest_only import LatestOnlyOperator

with DAG(
    "test_job",
    description="Job to test latest only operator.",
    schedule="* * * * *",
    start_date=datetime(2024, 12, 11),
    catchup=False,
):

    # latest only operator
    task1 = LatestOnlyOperator(task_id="latest_only")
    task2 = BashOperator(bash_command="echo test", task_id="task2")

    task1 >> task2

I think there is likely a mistake I'm making when setting up or running tasks in the unit test, but I'm not familiar enough with the airflow testing system to know where. I'll continue reading up on it some more in my free time, but if you see anything obviously amiss, please let me know.

@eladkal eladkal requested review from Lee-W and gopidesupavan July 25, 2025 15:50
@gopidesupavan
Copy link
Member

@leakec LGTM, can you please fix the static checks and compat tests?

@gopidesupavan Sure thing. I've added a few more commits to patch the issues, and now it passes all the tests. I do still need some help with the unit test though. The one I added passes, but the problem is it doesn't fail before the fix I implemented, i.e., it passes with both versions, so it doesn't really replicate the problem I saw when running this DAG in a normal airflow instance:

from airflow import DAG
from datetime import datetime
from airflow.providers.standard.operators.bash import BashOperator
from airflow.providers.standard.operators.latest_only import LatestOnlyOperator

with DAG(
    "test_job",
    description="Job to test latest only operator.",
    schedule="* * * * *",
    start_date=datetime(2024, 12, 11),
    catchup=False,
):

    # latest only operator
    task1 = LatestOnlyOperator(task_id="latest_only")
    task2 = BashOperator(bash_command="echo test", task_id="task2")

    task1 >> task2

I think there is likely a mistake I'm making when setting up or running tasks in the unit test, but I'm not familiar enough with the airflow testing system to know where. I'll continue reading up on it some more in my free time, but if you see anything obviously amiss, please let me know.

the test is fine i just triggered same one , it looks fine, it failed in my case.

image

@gopidesupavan
Copy link
Member

cool , @leakec good work thanks for fixing this.

@gopidesupavan
Copy link
Member

Failures are not related merging it now.

@gopidesupavan gopidesupavan merged commit 55cf82f into apache:main Jul 25, 2025
98 of 101 checks passed
@leakec
Copy link
Contributor Author

leakec commented Jul 25, 2025

Of course, thanks for all the help with the review and PR!

ferruzzi pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 7, 2025
…3541)

* Fixing latest_only_operator. See apache#50911

* Making an attempt at a reg test. However, this does not reproduce the original problem, and I need some help. See apache#50911 (comment)

* Updating to fix static checks and compat.

* Fixing start date.

* This was already failing for godpidespuavan, so good to go.

Co-authored-by: GPK <gopidesupavan@gmail.com>

---------

Co-authored-by: leake <carl.leake@karanadyn.com>
Co-authored-by: GPK <gopidesupavan@gmail.com>
fweilun pushed a commit to fweilun/airflow that referenced this pull request Aug 11, 2025
…3541)

* Fixing latest_only_operator. See apache#50911

* Making an attempt at a reg test. However, this does not reproduce the original problem, and I need some help. See apache#50911 (comment)

* Updating to fix static checks and compat.

* Fixing start date.

* This was already failing for godpidespuavan, so good to go.

Co-authored-by: GPK <gopidesupavan@gmail.com>

---------

Co-authored-by: leake <carl.leake@karanadyn.com>
Co-authored-by: GPK <gopidesupavan@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LatestOnlyOperator fails in airflow-providers-standard >=1.1.0

2 participants