[Airflow 2.2.2] execution_date Proxy object - str formatting error

[jj-ookla]

Apache Airflow version

2.2.2 (latest released)

Operating System

Ubuntu 18.04.6

Versions of Apache Airflow Providers

No response

Deployment

Other Docker-based deployment

Deployment details

No response

What happened

The deprecated variable execution_date raises an error when used in an f string template with date string formatting.

In [1]: execution_date
DeprecationWarning: Accessing 'execution_date' from the template is deprecated and will be removed in a future version. Please use 'logical_date' or 'data_interval_start' instead.
Out[1]: <Proxy at 0x7fb6f9af81c0 wrapping DateTime(2021, 11, 18, 0, 0, 0, tzinfo=Timezone('UTC')) at 0x7fb6f9aeff90 with factory <function TaskInstance.get_template_context.<locals>.deprecated_proxy.<locals>.deprecated_func at 0x7fb6f98699d0>>

In [2]: f"{execution_date:%Y-%m-%d}"
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
----> 1 f"{execution_date:%Y-%m-%d}"

TypeError: unsupported format string passed to Proxy.__format__

What you expected to happen

Executing f"{execution_date:%Y-%m-%d}" should return a string and not raise an error.

How to reproduce

from datetime import datetime
from airflow import DAG
from airflow.operators.python import PythonOperator


def test_str_fmt(execution_date: datetime):
    return f"{execution_date:%Y-%m-%d}"


dag = DAG(
    dag_id="Test_Date_String",
    schedule_interval="@daily",
    catchup=False,
    default_args={
        "depends_on_past": False,
        "start_date": datetime(2021, 11, 1),
        "email": None,
        "email_on_failure": False,
        "email_on_retry": False,
        "retries": 0,
    },
)

with dag:
    test_task = PythonOperator(
        task_id="test_task",
        python_callable=test_str_fmt,
    )

Anything else

from datetime import datetime
...
datetime.fromisoformat(next_ds)
TypeError: fromisoformat: argument must be str

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[uranusjr]

This can be worked around by implementing a proxy specific for datetime:

>>> from lazy_object_proxy import Proxy
>>> class DTProxy(Proxy):
...  def __format__(self, spec):
...   return self.__wrapped__.__format__(spec)
...
>>> from airflow.utils.timezone import utcnow
>>> p = DTProxy(utcnow)
>>> f"{p:%Y-%m-%d}"
'2021-11-19'

[jj-ookla]

@uranusjr I have discovered a similar issue with prev_ds and next_ds.

from datetime import datetime
...
datetime.fromisoformat(next_ds)
TypeError: fromisoformat: argument must be str

[uranusjr]

That is unfixable, I’m afraid, since the datetime API is too limiting to make this work.

But you shouldn’t need to do this in the first place? Those exact values are already available directly in the context as prev_execution_date and next_execution_date.

[jj-ookla]

That is unfixable, I’m afraid, since the datetime API is too limiting to make this work.

But you shouldn’t need to do this in the first place? Those exact values are already available directly in the context as prev_execution_date and next_execution_date.

👍🏻 This was an edge case on our end and I have already updated usage to data_interval_end. Posting related issues here in case they are relevant.

[potiuk]

Yeah. Agree with @uranusjr - we could potentially monkey-patch datetime, but this is not really worthy of that. Almost by definition, there are probably other cases and total compatibility with any python code written out there is not achievable, it will be far easier to fix those cases by the users.