Separate configurations for colorised and json logs in Task SDK / Celery Executor

[ianbuss]

Description

Currently, there is a single enable_pretty_log boolean option in the configure_logging method, which defaults to True.

If True it means structlog will:

• Use colorised output
• Add rich traces to exceptions
• Use "%Y-%m-%d %H:%M:%S.%f" timestamps
• Use console logger

If False it means:

• Use json logger
• Use iso format timestamps
• Use structlog's dict exception renderer

For users who would like a console logger but no colorised output and no rich traces this is not possible right now. Moreover, Celery defaults to enable_pretty_log = True, which makes the logs somewhat difficult to read in log ingestion solutions like Splunk, where the color escape sequences are rendered as raw text.

Use case/motivation

Have the option to have plain text, non-colorised console logger output via the existing https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#colored-console-log config parameter and have that be respected by the executors.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[highlyavailable]

@kaxil I would like to take this on, would you assign it to me?

[amoghrajesh]

@highlyavailable feel free to work on this