Skip to content

Conversation

@insomnes
Copy link
Contributor

@insomnes insomnes commented Feb 16, 2025

  • Remove initial_db_init call from top level of tests/www/views/test_views_rendered.py.
  • Misc: Move overlooked tests/utils/test_dataform.py to providers/google/tests/unit/google/cloud/utils/test_dataform.py

This call significantly slows tests collection.
I was able to drop arbitrary collection time with --collect-only for www tests from 20 to 5-6 seconds. And the whole tests collection from 41-42 to 32-33 seconds. Fast test collection allows various opportunities like dependencies smoke-tests and improves developers' experience.

This call is already provided by tests_common.pytest_plugin, so it should not affect the tests.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@boring-cyborg boring-cyborg bot added area:webserver Webserver related Issues provider:google Google (including GCP) related issues labels Feb 16, 2025
@insomnes
Copy link
Contributor Author

I must mention that I am quite confused by test_rendered_task_detail_env_secret.
I am unsure if I've understood it correctly and if my fixes are right.

In its initial configuration this test depends on parametrized parameters call order cause otherwise monkey would not be marked as secret. So I am also unsure that the test in its original config is right.

@insomnes insomnes force-pushed the speed-up-test-collection branch from 1fe6faa to 1e0863e Compare February 16, 2025 20:53
@insomnes insomnes force-pushed the speed-up-test-collection branch from 1e0863e to 5f76166 Compare February 16, 2025 21:54
Copy link
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NICE!

@potiuk
Copy link
Member

potiuk commented Feb 17, 2025

I must mention that I am quite confused by test_rendered_task_detail_env_secret. I am unsure if I've understood it correctly and if my fixes are right.

In its initial configuration this test depends on parametrized parameters call order cause otherwise monkey would not be marked as secret. So I am also unsure that the test in its original config is right.

Not 100% sure either. But since tests/www will be deleted soon together with the old UI, I woudl not worry too much about it.

@potiuk potiuk merged commit e67c892 into apache:main Feb 17, 2025
90 checks passed
@insomnes
Copy link
Contributor Author

But since tests/www will be deleted soon together with the old UI, I woudl not worry too much about it.

That's awesome! The test collection there is one of the longest. I was researching what's going on with it but decided to postpone a deep investigation because of all 3.0 changes and to fix only this "low-hanging fruit." Even the first phase of old provider code cleanup made a huge difference.

Thank you!

dantonbertuol pushed a commit to dantonbertuol/airflow that referenced this pull request Feb 17, 2025
* Do not run initial_db_init on module top level

* Set default in tests

* Move google provider `test_dataform.py` from tests/utils/ to provider/google/tests/unit/...

* Fix test depending on parametrized params execution order
@insomnes insomnes deleted the speed-up-test-collection branch February 17, 2025 22:27
ntr pushed a commit to ntr/airflow that referenced this pull request Feb 20, 2025
* Do not run initial_db_init on module top level

* Set default in tests

* Move google provider `test_dataform.py` from tests/utils/ to provider/google/tests/unit/...

* Fix test depending on parametrized params execution order
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:webserver Webserver related Issues provider:google Google (including GCP) related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants