-
Notifications
You must be signed in to change notification settings - Fork 16.4k
Validate JWT tokens in the ExecutionAPI for strong task identity #47885
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
961d886 to
9045edb
Compare
kaxil
approved these changes
Mar 18, 2025
ashb
commented
Mar 18, 2025
pierrejeambrun
approved these changes
Mar 18, 2025
Member
pierrejeambrun
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice I think it's a solid base to keep iterating on, multiple things could be improved but I guess this will be done in follow up PR.
just one remark but not blocking
This adds in Bearer auth token validation for the entire Execution API server. Some notes about how this is achieved: * We make heavy use of FastAPI's Dependency Injection to make this work * We use the `svcs`[1] module to make managing the service, and viewing the state of it, easy * In `.execution_api.routes` there are now two routers used -- one which ensures that _everything_ underneath it has a valid ExecutionAPI JWT, and another one which is "public" (and should contain very few routes) * We ran into an issue (linked in code) with `__future__ annotations` imports, and had to disable it on one specific file. * Refreshing of expiring Task Identity tokens is not handled yet (this will need middleware to support, a Dependency cannot add the response headers) [1]: pypi.org/project/svcs
Without this being set the API server now crash-loops on startup, which adds an extra 2 minutes to the test time for no reason.
9380b97 to
7bcf2e6
Compare
Co-authored-by: Pierre Jeambrun <pierrejbrun@gmail.com>
bugraoz93
approved these changes
Mar 18, 2025
agupta01
pushed a commit
to agupta01/airflow
that referenced
this pull request
Mar 21, 2025
…che#47885) * Validate JWT tokens in the ExecutionAPI for strong task identity This adds in Bearer auth token validation for the entire Execution API server. Some notes about how this is achieved: - We make heavy use of FastAPI's Dependency Injection to make this work - We use the [`svcs`][1] module to make managing the service, and viewing the state of it, easy - In `.execution_api.routes` there are now two routers used -- one which ensures that _everything_ underneath it has a valid ExecutionAPI JWT, and another one which is "public" (and should contain very few routes) - We ran into an issue (linked in code) with `__future__ annotations` imports, and had to disable it on one specific file. - Refreshing of expiring Task Identity tokens is not handled yet (this will need middleware to support, a Dependency cannot add the response headers) [1]: pypi.org/project/svcs * Set `api_auth/jwt_secret` at deploy-time in our Kube tests Without this being set the API server now crash-loops on startup, which adds an extra 2 minutes to the test time for no reason.
nailo2c
pushed a commit
to nailo2c/airflow
that referenced
this pull request
Apr 4, 2025
…che#47885) * Validate JWT tokens in the ExecutionAPI for strong task identity This adds in Bearer auth token validation for the entire Execution API server. Some notes about how this is achieved: - We make heavy use of FastAPI's Dependency Injection to make this work - We use the [`svcs`][1] module to make managing the service, and viewing the state of it, easy - In `.execution_api.routes` there are now two routers used -- one which ensures that _everything_ underneath it has a valid ExecutionAPI JWT, and another one which is "public" (and should contain very few routes) - We ran into an issue (linked in code) with `__future__ annotations` imports, and had to disable it on one specific file. - Refreshing of expiring Task Identity tokens is not handled yet (this will need middleware to support, a Dependency cannot add the response headers) [1]: pypi.org/project/svcs * Set `api_auth/jwt_secret` at deploy-time in our Kube tests Without this being set the API server now crash-loops on startup, which adds an extra 2 minutes to the test time for no reason.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
area:API
Airflow's REST/HTTP API
area:Executors-core
LocalExecutor & SequentialExecutor
area:Scheduler
including HA (high availability) scheduler
area:task-sdk
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds in Bearer auth token validation for the entire Execution API server,
and JWT generation to the Scheduler before handing off to the executor.
Some notes about how this is achieved:
We make heavy use of FastAPI's Dependency Injection to make this work
We use the
svcsmodule to make managing the service, and viewing thestate of it, easy
In
.execution_api.routesthere are now two routers used -- one whichensures that everything underneath it has a valid ExecutionAPI JWT, and
another one which is "public" (and should contain very few routes)
We ran into an issue (linked in code) with
__future__ annotationsimports,and had to disable it on one specific file.
Refreshing of expiring Task Identity tokens is not handled yet (this will
need middleware to support, a Dependency cannot add the response headers)
This could almost certainly use more tests than it has right now.
The API docs know that a JWT Bearer is needed

Closes #45107
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.