Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 41 additions & 9 deletions dev/README_RELEASE_PROVIDER_PACKAGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -530,28 +530,60 @@ git push --set-upstream origin "${branch}"

## Prepare issue in GitHub to keep status of testing

Create a GitHub issue with the content generated via manual
execution of the script below. You will use link to that issue in the next step. You need a GITHUB_TOKEN
set as your environment variable.
Create a GitHub issue with the content generated via manual execution of the command below. You will use
link to that issue in the next step.

You can also pass the token as `--github-token` option in the script.
You can also pass list of PR to be excluded from the issue with `--excluded-pr-list`.
```shell script
cd "${AIRFLOW_REPO_ROOT}"

breeze release-management generate-issue-content-providers --only-available-in-dist
```

GitHub API uses rate limiting that is based on the public IP address of client if you do not authenticate
with GitHub, so when you retrieve bigger number of PRs or when you are behind NAT and share your public
IP address with many other Anonymous GitHub API users, issue retrieval will be halted and your API calls
might slow down to a crawl, you will need then a GITHUB_TOKEN set as your
environment variable or pass the token as `--github-token` option in the script.

```shell script
cd "${AIRFLOW_REPO_ROOT}"

breeze release-management generate-issue-content-providers --only-available-in-dist --github-token TOKEN
```

or

```shell script
cd "${AIRFLOW_REPO_ROOT}"
export GITHUB_TOKEN=TOKEN
breeze release-management generate-issue-content-providers --only-available-in-dist
```

You can also generate the token by following
[this link](https://github.com/settings/tokens/new?description=Read%20sssues&scopes=repo:status)
You can generate the token by following
[this link](https://github.com/settings/tokens/new?description=Read%20issues&scopes=repo:status). Since it is easy to generate such token, by following the link, it is recommended to
generate a new token for each release and delete it once you've generated the issue.

If you see in the output that some of the PRs are just "noise" (i.e. there is no need to verify them
as they are misc/documentation kind of changes that have no impact on the actual installation of
the provider or the code of the provider, can optionally pass list of PR to be excluded from
the issue with `--excluded-pr-list`. This might limit the scope of verification. Some providers
might disappear from the list and list of authors that will be pinged in the generated issue.

If you are preparing release for RC2/RC3 candidates, you should add `--suffix` parameter:
You can repeat that and regenerate the issue content until you are happy with the generated issue.

```shell script
breeze release-management generate-issue-content-providers --only-available-in-dist --suffix rc2
cd "${AIRFLOW_REPO_ROOT}"

breeze release-management generate-issue-content-providers --only-available-in-dist --github-token TOKEN \
--excluded-pr-list PR_NUMBER1,PR_NUMBER2
```

It's also OK to manually modify the content of such generated issue before actually creating the
issue. There is a comment generated with NOTE TO RELEASE MANAGER about this in the issue content.
Hit Preview button on "create issue" screen before creating it to verify how it will look like
for the contributors.



## Prepare voting email for Providers release candidate

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
from datetime import datetime
from pathlib import Path
from subprocess import DEVNULL
from typing import IO, TYPE_CHECKING, Any, Generator, NamedTuple
from typing import IO, TYPE_CHECKING, Any, Generator, Iterable, NamedTuple

import click
from rich.progress import Progress
Expand Down Expand Up @@ -1572,6 +1572,25 @@ def get_prs_for_package(provider_id: str) -> list[int]:
return prs


def create_github_issue_url(title: str, body: str, labels: Iterable[str]) -> str:
"""
Creates URL to create the issue with title, body and labels.
:param title: issue title
:param body: issue body
:param labels: labels for the issue
:return: URL to use to create the issue
"""
from urllib.parse import quote

quoted_labels = quote(",".join(labels))
quoted_title = quote(title)
quoted_body = quote(body)
return (
f"https://github.com/apache/airflow/issues/new?labels={quoted_labels}&"
f"title={quoted_title}&body={quoted_body}"
)


@release_management.command(
name="generate-issue-content-providers", help="Generates content for issue to test the release."
)
Expand All @@ -1593,14 +1612,12 @@ def get_prs_for_package(provider_id: str) -> list[int]:
is_flag=True,
help="Only consider package ids with packages prepared in the dist folder",
)
@click.option("--suffix", default="rc1", help="Suffix to add to the version prepared")
@argument_provider_packages
def generate_issue_content_providers(
disable_progress: bool,
excluded_pr_list: str,
github_token: str,
only_available_in_dist: bool,
suffix: str,
provider_packages: list[str],
):
import jinja2
Expand Down Expand Up @@ -1675,7 +1692,7 @@ class ProviderPRInfo(NamedTuple):
provider_package_id=provider_id,
pypi_package_name=provider_yaml_dict["package-name"],
pr_list=pull_request_list,
suffix=package_suffix if package_suffix else suffix,
suffix=package_suffix if package_suffix else "",
)
template = jinja2.Template(
(Path(__file__).parents[1] / "provider_issue_TEMPLATE.md.jinja2").read_text()
Expand All @@ -1689,19 +1706,29 @@ class ProviderPRInfo(NamedTuple):
get_console().print()
get_console().print()
get_console().print(
"Issue title: [yellow]Status of testing Providers that were "
"Issue title: [warning]Status of testing Providers that were "
f"prepared on {datetime.now():%B %d, %Y}[/]"
)
get_console().print()
syntax = Syntax(issue_content, "markdown", theme="ansi_dark")
get_console().print(syntax)
get_console().print()
issue_content += "\n"
users: set[str] = set()
for provider_info in providers.values():
for pr in provider_info.pr_list:
users.add("@" + pr.user.login)
get_console().print("All users involved in the PRs:")
get_console().print(" ".join(users))
issue_content += f"All users involved in the PRs:\n{' '.join(users)}"
syntax = Syntax(issue_content, "markdown", theme="ansi_dark")
get_console().print(syntax)
url_to_create_the_issue = create_github_issue_url(
title=f"Status of testing Providers that were prepared on {datetime.now():%B %d, %Y}",
body=issue_content,
labels=["testing status", "kind:meta"],
)
get_console().print()
get_console().print(
"[info]You can prefill the issue by copy&pasting this link to browser "
"(or Cmd+Click if your terminal supports it):\n"
)
print(url_to_create_the_issue)


def get_all_constraint_files(refresh_constraints: bool, python_version: str) -> None:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,6 @@
"--excluded-pr-list",
"--github-token",
"--only-available-in-dist",
"--suffix",
],
}
],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,7 @@ Those are providers that require testing as there were some substantial changes

NOTE TO RELEASE MANAGER:

Please move here the providers that have doc-only changes or for which changes are trivial, and
you could assess that they are OK. In case

The providers are automatically installed on Airflow 2.3 and latest `main` during the CI, so we know they
are installable. Also, all classes within the providers are imported during the CI run so we know all
providers can be imported.
You can move here the providers that have doc-only changes or for which changes are trivial, and
you could assess that they are OK.

-->
Loading