Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missed deprecations for cncf #20031

Merged

Conversation

dimon222
Copy link
Contributor

@dimon222 dimon222 commented Dec 3, 2021

This just adds deprecation warnings to backcompat classes for:

  • V1EnvVar
  • V1ResourceRequirements
  • V1ContainerPort

This was split from #19726 (comment)

@jedcunningham

@boring-cyborg boring-cyborg bot added provider:cncf-kubernetes Kubernetes provider related issues area:providers labels Dec 3, 2021
@dimon222
Copy link
Contributor Author

dimon222 commented Dec 3, 2021

Fixed the long lines that flake8 didn't like, but I'm not sure if import is something of concern (why is it not considering it top of the file if above is only license placeholder?)

@dimon222
Copy link
Contributor Author

I would appreciate some advice on linter errors....

@jedcunningham
Copy link
Member

Maybe try with a newline before the first import? I'd play with it locally using pre-commit until you can get it happy, e.g. pre-commit run flake8 --files airflow/providers/cncf/kubernetes/backcompat/pod.py.

@dimon222 dimon222 force-pushed the feature/add_missed_deprecation_backcompat_cncf branch from 7211839 to 7025ead Compare December 16, 2021 22:04
@dimon222
Copy link
Contributor Author

Resolved both, it was because two comment blocks on top of file with """ invoke flake8 to assume we're doing something sketchy. I merged them.

@dimon222
Copy link
Contributor Author

Weird exception in deprecation warning validator. The line is already whitelisted in KNOWN_DEPRECATED_DIRECT_IMPORTS but still not enough for some reason.

@potiuk
Copy link
Member

potiuk commented Dec 17, 2021

It actually detected a mistake :).

It's about the stack_level and the fact that it is imported through another module. The warning is generated one level deeper than other warnings and stack_level must be set to 3, otherwise the generated warning will not show the origin of the import.

As you can see here: https://github.com/apache/airflow/runs/4555876546?check_suite_focus=true#step:6:385 all the warnings are showing "importlib" as the import location but the new warning shows kubernetes_pod as the source:

  /opt/airflow/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py:46: 
  This module is deprecated. Please use `kubernetes.client.models.V1EnvVar`.

  /usr/local/lib/python3.7/importlib/__init__.py:127: This module is deprecated. 
  Please use `airflow.providers.amazon.aws.hooks.dynamodb`.

  /usr/local/lib/python3.7/importlib/__init__.py:127: This module is deprecated. 
  Please use `airflow.hooks.redshift_sql` or `airflow.hooks.redshift_cluster` as 
  appropriate.
```

@dimon222
Copy link
Contributor Author

dimon222 commented Dec 17, 2021

Seems like now EKS block crashes with same

@potiuk
Copy link
Member

potiuk commented Dec 17, 2021

Apparently the common "import" is using different stack levels in different import paths. This probably means that you need to make sure the warning is either generated with the same "stack level" of imports or raise the warning in different imports differently.

@potiuk
Copy link
Member

potiuk commented Dec 17, 2021

The problem is that currently if user import the deprecate import now, they will see a wrong "origin" place of the import, so this is a "real" issue.

@github-actions
Copy link

github-actions bot commented Feb 1, 2022

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Feb 1, 2022
@dimon222 dimon222 force-pushed the feature/add_missed_deprecation_backcompat_cncf branch from fed147d to f6f5a20 Compare February 1, 2022 00:25
@eladkal eladkal removed the stale Stale PRs per the .github/workflows/stale.yml policy file label Feb 1, 2022
@dimon222
Copy link
Contributor Author

dimon222 commented Feb 1, 2022

Should be sorted now. I had to sacrifice old references to imports.
https://github.com/apache/airflow/pull/20031/files#diff-947f9f98b4aedccee6dd81558568c350c738f5da050aad1224d8b48ec43677cd

I feel deprecated class wasn't even intended to be imported by KPD anyway. I suspect users should have started using new V1EnvVar as result of this migration to k8s classes.

@potiuk potiuk merged commit 4a73d8f into apache:main Feb 1, 2022
@dimon222 dimon222 deleted the feature/add_missed_deprecation_backcompat_cncf branch February 1, 2022 12:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers provider:cncf-kubernetes Kubernetes provider related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants