Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[1.28] Release Team Lead Cycle Progress #2223

Closed
gracenng opened this issue Apr 18, 2023 · 11 comments
Closed

[1.28] Release Team Lead Cycle Progress #2223

gracenng opened this issue Apr 18, 2023 · 11 comments
Assignees
@gracenng
Labels
area/release-eng Issues or PRs related to the Release Engineering subproject area/release-team Issues or PRs related to the release-team subproject needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-priority sig/release Categorizes an issue or PR as relevant to SIG Release.

Comments

@gracenng
Copy link
Member

gracenng commented Apr 18, 2023 •
edited
Loading

Additional information can be found in the release team lead handbook. If tasks are not needed to be done or additional tasks are required, make sure to update the issue template!

Release Lead tasks

1. Before the start of the Release Cycle (Week 0)

  • Captured feedback from previous release cycle retro and planned to incorporate it into the release cycle
  • Release directory named release-1.XX added to k/sig-release/releases
  • Started planning the release schedule by opening a thread in #sig-release
  • Release Lead Shadows are confirmed
  • Team leads notified that all release team members read the release team onboarding document - skipped
  • Update slack channel descriptions for the #sig-release channel and all #release-xxx channels

Release Lead Onboarding:

Create Release Team Documents:

  • Meeting Notes document created
  • Retro document created
  • Release Team contact sheet created
  • Lead rotation sheet created
    • Access: : restricted access, edit rights shared with release team lead shadows individually

One week before the start of the release cycle:

  • "Release Sneak Peak" email to introduce yourself, lead shadows, role leads, and branch manager has been send out - here

2. First weeks of the release cycle up to Enhancements Freeze (1-3 Weeks)

First Week of the release cycle:

  • Checked in with team leads and verified that the release team is complete

  • Reminded release team members to subscribe to the kubernetes-release-team and kubernetes-sig-release google groups and to the kubernetes-release calendar.

  • Notified team leads to update the contact sheet with shadow information bit.ly/k8s1XX-contacts

  • Release schedule finalized

  • Begin paying attention to CI signal, as it may begin degrading soon after the prior release is cut and any slips must be caught and rectified promptly.

  • Meet your Shadows and create a communication channel with them. Establish expectations and share out work - delegate!

  • Request review of this document by the Release Team Lead shadow(s). The shadow(s) should also take all actions in this document around joining groups and requesting access permissions.

  • Update the SIG Release groups in the k/k8s.io/groups/sig-release/groups.yaml with the following:

    • Add Lead and Lead shadows to members of k8s-infra-release-viewers
    • Add EA, Lead, Lead shadows, comms lead, comms shadows to members of release-comms
    • Add EA, Lead and Lead shadows to members of release-managers
    • Add EA, Lead and Lead shadows to managers of release-team
    • Add Role Leads and Role shadows to members of release-team
    • Add Role shadows and Lead shadows to members of release-team-shadows
    • Add EA to manager of release-team-shadows, if EA is owner of release-team-shadows already then add Lead to manager of release-team-shadows
    • Add Bug Triage Lead and shadows to k8s-infra-rbac-triageparty-release

  • Ensured that there is a branch manager available for cutting 1.XX.0-alpha.1

  • Assist the Enhancements Lead in collecting planned work from SIGs

  • Discussed and scheduled a weekly Release Team meetings on a day that is most acceptable to the team. Invite the kubernetes-sig-release group.

  • Poll Release Team membership and schedule a weekly alternate meeting to better enable more attendance outside of the Americas.

  • Major release cycle events have been added to the Kubernetes Release Calendar with one week in advance reminders set. (defer to the handbook for more information)

  • Checked in with ci-signal and branch managers if 1.XX.0-alpha.2 is ok to be released and master-blocking tests are all green

A week before Enhancements Freeze:

  • Remind the community about Enhancements Freeze reminder send out. Example email: 1.26; It may also be useful to send a short version to the #sig-release and #chairs-and-techleads Slack channels referencing to the k-dev email.

3. Enhancements freeze up to Release Halfway Point (~Week 5 - 7)

General Tasks:

  • Bring exceptions to the #sig-release Slack channel and to Release Team meetings, and make sure SIG representatives for the exception(s) know to attend and discuss if necessary.
  • Begin casual observation of issues, CI signal, test flakes, and critical PRs
  • Release Team Retro is scheduled shortly after the "Release Halfway Point" and a host is selected

Release Cut Alpha 3:

  • Checked in with ci-signal and branch managers if 1.XX.0-alpha.3 is ok to be released and master-blocking tests are all green
  • Checked in with Release Notes team on release notes progress after the tag for 1.XX.0-alpha.3 is generated

Release Cut Alpha 4:

  • Checked in with ci-signal and branch managers if 1.XX.0-alpha.4 is ok to be released and master-blocking tests are all green
  • Checked in with Release Notes team on release notes progress after the tag for 1.XX.0-alpha.4 is generated

4. Release Halfway Point up to Code Freeze (~Week 8 - 11)

General Tasks:

  • Send out a "Release Update / State of the Release", example: 1.26
  • Notify SIGs and about upcoming Code Freeze Deadline by sending an email to the kubernetes-dev list
  • The first retrospective meeting is scheduled for the first week of Monday, Wednesday, and Friday burndown meetings, typically mid-week. Confirm the Emeritus Adviser can serve as facilitator. If Emeritus Adviser is unavailable then defer the responsibility as appropriate.
  • Make sure everyone knows the Docs deadline (PRs ready for review) is coming the following week.
  • Started release team meetings on Monday, Wednesday, and Friday
  • Pinged role leads reminding them to start considering succession plans. If they are handing the role off to a successor, identifying them early gives more time for the committed volunteer to get targeted mentoring.

Removals, Deprecations, and Major Changes Blog:

  • Check in with release-comms if they are in contact with the release-enhancements team to collect "deprecations and removals" targeting the release
  • Identified with release-comms & sig-docs if a "Removals, Deprecations, and Major Changes" blog is needed and if so, started drafting it up (ref 1.26 blog)
  • A Release Team representative (ideally from release-comms) should attend the sig-docs meeting to raise awareness about the "Deprecations and Removals blog" for reviews.
  • The Deprecations and Removals blog is scheduled for next week shortly after Code Freeze. A draft of the blog should be started as reviews and iterations will be needed before publication next week
  • Insured that "Removals, Deprecations, and Major Changes"-authors are reviewing the blog before it's being publicized

Major Themes & Release Blog:

  • Followed up with SIGs on major themes of the release
  • Checked in with release-notes and release-comms about the status of major themes collection

5. Around Code Freeze (~Week 13)

Defer to the code freeze section in the release-team-lead handbook.

Shortly before Code freeze

Code Freeze begins, and it’s now the home stretch of the release. SIGs will need to ensure all work moving forward is carefully curated with required merge labels.

  • Monitor the enhancements GitHub project to get an idea of how many PRs are still outstanding leading up to Code Freeze
  • Monitor Testgrid and Prow to understand the stability of the release and PRs getting ready to merge. If Prow and Test grid are not in a good state consult folks from SIG Testing on delaying code freeze by a day if needed.
  • If the release branch is not healthy, stable, and passing tests consistently, notify community through standard channels of need to rectify or code freeze will come early to force focus on stabilization.
  • Reminded the Branch Manager that branch CI jobs will be needed next week
  • Send out a reminder email to kubernetes-dev

With Code Freeze

  • As needed, assist the Bug Triage Lead and Enhancements Lead removing PRs and enhancements from the milestone that aren't merged in time /milestone clear

After Code Freeze

  • Verified that the release-1.XX branch has been automatically created at the start of Test Freeze
  • Verified that the Branch Manager created the CI board on Testgrid for the release cycle (1.XX-blocking & 1.XX-informing)
  • Published the "Removals, Deprecations, and Major Changes blog"

6. Test Freeze up to Release Day (~Week 14)

  • Verified together with the release-docs team that all KEPs with required documentation are ready for review
  • Completed release theme (slogan, logo and explanation text) and add it to the release cycle documentation in k/sig-release/releases/release-1.XX
  • Decided with the Release team if burndown meetings are necessary or updates are done via Slack thread on Tuesdays and Thursdays
  • Discussed Release Lead succession with the EA and sig-release leads
  • Remind team lead to find successors for the upcoming cycle and discuss candidates with the EA
  • The task is now to ensure the release branch is ready to go. This means there are zero pending PRs, no failing 1.XX-blocking tests, no open issues in the milestone. This will continue until release day.
  • Final documentation PRs are reviewed and ready to be merged. Likely, this is not true and some are outstanding, so you need to help convince SIG doc writers to get these in with urgency.
  • Planned something for Release Day. Make the day as fun as you can for the team. Plan ahead for this and do something nice
  • Prepared Release Team gifts
  • The Communications Lead contacts CNCF to gauge media interest, schedule the CNCF Kubernetes release workshop, and publish the release blog post. Stay in the loop for that. If there is media interest in the release, an interview between the journalist will be organized by the CNCF.
  • Check in with the docs lead and verify tasks that should happen before the release day are completed (ref release-docs handbook)
  • The Release Team Retro part 2 and part 3 is scheduled shortly after the "Release Halfway Point" and a host / facilitator is available
  • Remind the release team to add items to the retro meeting agenda

7. Release Day (~Week 15)

  • Note that release day can and should be postponed if any of the conditions outlined in week 11 are not satisfied.

  • Every issue in the milestone is considered release blocking

  • If you have to push the release date back, try to avoid Friday since it makes release publicity extremely difficult. Also, people seem to have patience with delay as long as the reasons are clear and openly communicated. This is your duty. You must over-communicate and ensure the team is also talking to their stakeholders (CNCF, community, press, etc.)

  • The following final actions must occur in order, with successful completion of each being the entry criteria to the next.

    • Release day morning:
      • Go / No-Go: should generally be clear a day or three ahead of release, but the day's burndown provides a final opportunity for the team to affirm things are ready.
      • Release Notes Lead PRs final draft release notes to sig-release, with Release Team Lead approving merge.
      • Branch Manager does mock release build.
      • Branch Manager does mock publication. Validates with Release Team lead and broader team the mock announcement email content.
      • Branch Manager does nomock release build.
    • Starting when ready:
      • Communications Lead begins staging blog post.
      • Branch Manager Lead does nomock publication.
      • Branch Manager coordinates the building and publishing of rpm/deb packages with Google Build Admins.
      • Build Admins affirms build is complete.
      • Verify with the docs lead that the staging release website is in correct shape by verifying the blog posts, the release page, download page, etc.
      • Docs Lead publishes release docs to website (ref tasks todo)
      • Branch Manager does release-notify.
    • Approximately 5pm Pacific: all work is complete and the release team
      lead announces release to k-dev, SIG Leads, and discuss.k8s.io.

  • After the release, it’s time to lift code freeze. The bot will need to be updated.

    • Thaw k/k,
      master branch is then opened for new pull requests.

  • The Docs lead thaws k/website

  • Use all of the appropriate communications channels to announce the lifting of Code Freeze and thawing of
    k/website, this can part of the release announcement.

8. After the Release Day (~Week 16, 17)

  • Release Retrospective Part 2 completed
  • Release Retrospective Part 3 completed or cancelled
  • Contact the Release Managers Google Group to complete the Release Team Lead & Lead Shadows offboarding tasks from the previously-opened onboarding issue
  • Help fill any open positions for the next release milestone
  • Work with the incoming Release Team Lead to establish incoming Release Team.

Release Lead Offboarding tasks:

Further comments

NONE

cc: @kubernetes/release-engineering @kubernetes/release-team
@gracenng gracenng added sig/release Categorizes an issue or PR as relevant to SIG Release. area/release-team Issues or PRs related to the release-team subproject area/release-eng Issues or PRs related to the Release Engineering subproject labels Apr 18, 2023
@gracenng gracenng self-assigned this Apr 18, 2023
@k8s-ci-robot k8s-ci-robot added needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-priority labels Apr 18, 2023
@gracenng
Copy link
Member Author

I have completed the CNCF Inclusive Speaker Orientation training. Certificate ID Number: LF-ocllb99cdq

I agree to abide by the guidelines set forth in the Security Release Process, specifically the embargo on CVE communications.

@mickeyboxell
Copy link
Contributor

I have completed the CNCF Inclusive Speaker Orientation training. Certificate ID Number: LF-ivrocskska

I agree to abide by the guidelines set forth in the Security Release Process, specifically the embargo on CVE communications.

@neoaggelos
Copy link
Contributor

I have completed the CNCF Inclusive Speaker Orientation training. Certificate ID Number: LF-aoiq9wdo0f

I agree to abide by the guidelines set forth in the Security Release Process, specifically the embargo on CVE communications.

@Atharva-Shinde
Copy link
Contributor

I have completed the CNCF Inclusive Speaker Orientation training. Certificate ID Number: LF-99sentagyc

I agree to abide by the guidelines set forth in the Security Release Process, specifically the embargo on CVE communications.

@furkatgofurov7
Copy link
Member

furkatgofurov7 commented May 15, 2023 •
edited
Loading

I have completed the CNCF Inclusive Speaker Orientation training. Certificate ID Number: LF-vw99yqmk08

I agree to abide by the guidelines set forth in the Security Release Process, specifically the embargo on CVE communications.

@sanchita-07
Copy link
Member

I have completed the CNCF Inclusive Speaker Orientation training. Certificate ID Number: LF-5j3swg5mzq

I agree to abide by the guidelines set forth in the Security Release Process, specifically the embargo on CVE communications.

@marosset
Copy link
Contributor

I have completed the CNCF Inclusive Speaker Orientation training. Certificate ID Number: LF-m4rfg689o0

I agree to abide by the guidelines set forth in the Security Release Process, specifically the embargo on CVE communications.

@bradmccoydev
Copy link
Member

I have completed the CNCF Inclusive Speaker Orientation training. Certificate ID Number: LF-3whb91z7zu

I agree to abide by the guidelines set forth in the Security Release Process, specifically the embargo on CVE communications.

@helayoty
Copy link
Member

I have completed the CNCF Inclusive Speaker Orientation training. Certificate ID Number: LF-tl9c2qems9

I agree to abide by the guidelines set forth in the Security Release Process, specifically the embargo on CVE communications.

@Rishit-dagli
Copy link
Member

I have completed the CNCF Inclusive Speaker Orientation training. Certificate ID Number: LF-kil84sfp99

I agree to abide by the guidelines set forth in the Security Release Process, specifically the embargo on CVE communications.

@marosset marosset mentioned this issue Jun 14, 2023
Merged
@marosset marosset mentioned this issue Jul 19, 2023
Closed
@gracenng
Copy link
Member Author

@Priyankasaggu11929 will address the offboarding perm tasks as she update them for 1.29

thanks everyone for a good release >:)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gracenng gracenng

Labels
area/release-eng Issues or PRs related to the Release Engineering subproject area/release-team Issues or PRs related to the release-team subproject needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-priority sig/release Categorizes an issue or PR as relevant to SIG Release.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests