Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions dev/README_RELEASE_PROVIDER_PACKAGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -330,13 +330,21 @@ breeze release-management prepare-provider-packages --include-removed-providers
--version-suffix-for-pypi rc1 --package-format both
```

if you only build few packages, run:
If you only build few packages, run:

```shell script
breeze release-management prepare-provider-packages \
--version-suffix-for-pypi rc1 --package-format both PACKAGE PACKAGE ....
```

In case you are ALSO releasing RC2, RC3, etc. for selected packages, they will be skipped automatically because
the `rc1` tag will be created for them already. In this case you should specify the ``rc*`` that you want to
build and specify the package id's you want to build.

```shell script
breeze release-management prepare-provider-packages --version-suffix-for-pypi rc2 --package-format both PACKAGE
```

* Verify the artifacts that would be uploaded:

```shell script
Expand Down Expand Up @@ -416,7 +424,7 @@ git pull --rebase

```shell script
cd "${AIRFLOW_REPO_ROOT}"
breeze build-docs --clean-build apache-airflow-providers all-providers
breeze build-docs --clean-build apache-airflow-providers all-providers --include-removed-providers
```

Usually when we release packages we also build documentation for the "documentation-only" packages. This
Expand Down Expand Up @@ -461,12 +469,6 @@ breeze release-management publish-docs apache-airflow-providers all-providers --
breeze release-management add-back-references all-providers
```

If you see `ModuleNotFoundError: No module named 'docs'`, set:

```
export PYTHONPATH=.:${PYTHONPATH}
```

If you have providers as list of provider ids because you just released them you can build them with

```shell script
Expand Down Expand Up @@ -500,6 +502,8 @@ You can also pass the token as `--github-token` option in the script.
You can also pass list of PR to be excluded from the issue with `--excluded-pr-list`.

```shell script
cd "${AIRFLOW_REPO_ROOT}"

breeze release-management generate-issue-content-providers --only-available-in-dist
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1496,6 +1496,23 @@ def is_package_in_dist(dist_files: list[str], package: str) -> bool:
)


VERSION_MATCH = re.compile(r"([0-9]+)\.([0-9]+)\.([0-9]+)(.*)")


def get_suffix_from_package_in_dist(dist_files: list[str], package: str) -> str | None:
"""Get suffix from package prepared in dist folder."""
for file in dist_files:
if file.startswith(f'apache_airflow_providers_{package.replace(".", "_")}') and file.endswith(
".tar.gz"
):
file = file[: -len(".tar.gz")]
version = file.split("-")[-1]
match = VERSION_MATCH.match(version)
if match:
return match.group(4)
return None


def get_prs_for_package(provider_id: str) -> list[int]:
pr_matcher = re.compile(r".*\(#([0-9]*)\)``$")
prs = []
Expand Down Expand Up @@ -1567,6 +1584,7 @@ class ProviderPRInfo(NamedTuple):
pypi_package_name: str
version: str
pr_list: list[PullRequest.PullRequest | Issue.Issue]
suffix: str

if not provider_packages:
provider_packages = list(DEPENDENCIES.keys())
Expand Down Expand Up @@ -1623,16 +1641,18 @@ class ProviderPRInfo(NamedTuple):
).read_text()
)
if pull_request_list:
package_suffix = get_suffix_from_package_in_dist(files_in_dist, provider_id)
providers[provider_id] = ProviderPRInfo(
version=provider_yaml_dict["versions"][0],
provider_package_id=provider_id,
pypi_package_name=provider_yaml_dict["package-name"],
pr_list=pull_request_list,
suffix=package_suffix if package_suffix else suffix,
)
template = jinja2.Template(
(Path(__file__).parents[1] / "provider_issue_TEMPLATE.md.jinja2").read_text()
)
issue_content = template.render(providers=providers, date=datetime.now(), suffix=suffix)
issue_content = template.render(providers=providers, date=datetime.now())
get_console().print()
get_console().print(
"[green]Below you can find the issue content that you can use "
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Let us know in the comment, whether the issue is addressed.
Those are providers that require testing as there were some substantial changes introduced:

{% for provider_id, provider_info in providers.items() %}
## Provider [{{ provider_id }}: {{ provider_info.version }}{{ suffix }}](https://pypi.org/project/{{ provider_info.pypi_package_name }}/{{ provider_info.version }}{{ suffix }})
## Provider [{{ provider_id }}: {{ provider_info.version }}{{ provider_info.suffix }}](https://pypi.org/project/{{ provider_info.pypi_package_name }}/{{ provider_info.version }}{{ provider_info.suffix }})
{%- for pr in provider_info.pr_list %}
- [ ] [{{ pr.title }} (#{{ pr.number }})]({{ pr.html_url }}): @{{ pr.user.login }}
{%- endfor %}
Expand Down