Release 0.16

[BigLep]

Meta

• Release owner: @galargh
• Release reviewer: @ajnavarro
• Expected RC date: 2022-09-22 2022-09-26
• Expected final release date: 2022-09-29 2022-10-03 2022-10-04
• Accompanying PR for improving the release process: docs: init release issue template improvement process v0.16.0 #9283

See the Kubo release process for more info.

Kubo 0.16.0 Release

We're happy to announce Kubo 0.16.0, bla bla...

As usual, this release includes important fixes, some of which may be critical for security. Unless the fix addresses a bug being exploited in the wild, the fix will not be called out in the release notes. Please make sure to update ASAP. See our release process for details.

🗺 What's left for release

• fix: require IPNS V2 signatures #9265
• feat(cmds/add): --to-files option as files cp #8927
• chore: bump go-libp2p v0.23.1 #9285
• feat: Delegated routing with custom configuration. #9274
• feat(gateway): _redirects file support #8890
• feat: add webtransport as an optin transport #9293
• fix: ensure hasher is registered when using a hashing function #9303

Ideal, but not blocking:

• ipfs add with exotic hash functions panics (the daemon and the client) #9297
• feat: configurable ProviderSearchDelay in bitswap #9053

🔦 Highlights

< top highlights for this release notes. For ANY version (final or RCs) >

✅ Release Checklist

• Stage 0 - Prerequisites
  • Ensure that the What's left for release section has all the checkboxes checked. If that's not the case, discuss the open items with Kubo maintainers and update the release schedule accordingly.
  • Create docs-release-vX.Y.Z branch, open a draft PR and keep updating docs/RELEASE_ISSUE_TEMPLATE.md on that branch as you go.
  • Ensure you have GPG key generated and added to your GitHub account. This will enable you to created signed tags.
  • Ensure you have admin access to IPFS Discourse. Admin access is required to globally pin posts and create banners. @2color might be able to assist you.
  • Access to #bifrost channel in FIL Slack might come in handy. Ask the release reviewer to invite you over.
  • After the release is deployed to our internal infrastructure, you're going to need read access to IPFS network metrics dashboards. Open an access request in https://github.com/protocol/pldw/issues/new/choose if you don't have it yet (example).
  • You're also going to need NPM installed on your system. See here for instructions.
  • Prepare changelog proposal in docs/changelogs/vX.Y.md.
    • Skip filling out the ### Changelog section (the one where which lists all the commits and contributors) for now. We will populate it after the release branch is cut.
  • Install ZSH (instructions). It is needed by the changelog creation script.
  • Ensure you have kubo checked out under $(go env GOPATH)/src/github.com/ipfs/kubo. This is required by the changelog creation script.
    • If you want your clone to live in a different location, you can symlink it to the expected location by running mkdir -p $(go env GOPATH)/src/github.com/ipfs && ln -s $(pwd) $(go env GOPATH)/src/github.com/ipfs/kubo.
  • Ensure that README.md is up to date.
• Stage 1 - Initial Preparations
  • Upgrade to the latest patch release of Go that CircleCI has published (currently used version: 1.19.1)
    • See the list here: https://hub.docker.com/r/cimg/go/tags
    • ipfs/distributions: bump this version
    • ipfs/kubo: example PR
  • Fork a new branch (release-vX.Y.Z) from master.
  • Bump the version in version.go in the master branch to vX.(Y+1).0-dev.
• Stage 2 - Release Candidate - if any non-trivial changes need to be included in the release, return to this stage
  • Bump the version in version.go in the release-vX.Y.Z branch to vX.Y.Z-rcN.
  • If applicable, add new commits to the release-vX.Y.Z branch from master using git cherry-pick -x ...
    • Note: release-* branches are protected. You can do all needed updates on a separated branch (e.g. wip-release-vX.Y.Z) and when everything is settled push to release-vX.Y.Z
  • Update the docs/changelogs/vX.Y.md with the new commits and contributors.
    • Run ./bin/mkreleaselog twice to generate the changelog and copy the output.
      • The first run of the script might be poluted with git clone output.
    • Paste the output into the ### Changelog section of the changelog file inside the <details><summary></summary></details> block.
    • Commit the changelog changes.
  • Push the release-vX.Y.Z branch to GitHub (git push origin release-vX.Y.Z) and create a draft PR targetting release branch if it doesn't exist yet (example).
  • Wait for CI to run and complete PR checks. All checks should pass.
  • Tag HEAD release-vX.Y.Z commit with vX.Y.Z-rcN (git tag -s vX.Y.Z-rcN -m 'Pre-release X.Y.Z-rcn')
  • Push the vX.Y.Z-rcN tag to GitHub (git push origin vX.Y.Z-rcN; DO NOT USE git push --tags because it pushes all your local tags).
  • Add artifacts to https://dist.ipfs.tech by making a PR against ipfs/distributions
    • Clone the ipfs/distributions repo locally.
    • Create a new branch (kubo-release-vX.Y.Z-rcn) from master.
    • Run ./dist.sh add-version kubo vX.Y.Z-rcN to add the new version to the versions file (instructions).
      • If you're adding a new RC, dist.sh will print WARNING: not marking pre-release kubo vX.Y.Z-rc1n as the current version..
    • Push the kubo-release-vX.Y.Z-rcn branch to GitHub and create a PR from that branch (example).
    • Wait for PR to build artifacts and generate diff (~30min)
    • Inspect results, merge if CI is green and the diff looks ok
    • Wait for master branch to build. It will automatically update DNSLink at https://dist.ipfs.tech (~30min)
  • Publish the RC to the NPM package by running https://github.com/ipfs/npm-go-ipfs/actions/workflows/main.yml (it happens automatically but it is safe to speed up the process and kick of a run manually)
  • Cut a pre-release on GitHub (instructions, example)
    • Use vX.Y.Z-rcN as the tag.
    • Link to the release issue in the description.
    • Link to the relevant changelog in the description.
    • Check This is a pre-release.
  • Synchronize release artifacts by running sync-release-assets workflow.
  • Announce the RC
    • Create a new post on IPFS Discourse. (example)
      • Use Kubo vX.Y.Z-rcn Release Candidate is out! as the title.
      • Use kubo and go-ipfs as topics.
      • Repeat the title as a heading (##) in the description.
      • Link to the GitHub Release, binaries on IPNS, docker pull command and release notes in the description.
    • Pin the topic globally so that it stays at the top of the category.
    • If there is no more important banner currently set on Discourse (e.g. IPFS Camp announcement), make the topic into a banner.
    • Check if Discourse post was automatically copied to:
      • IPFS Discord #ipfs-chatter
      • FIL Slack #ipfs-chatter
      • Matrix
    • Mention early testers in the comment under the release issue (example).
• Stage 3 - Internal Testing
  • Library Testing.
    • interop
      • Clone the ipfs/interop repo locally.
      • Create a new branch (kubo-release-vX.Y.Z-rcn) from master.
      • Update go-ipfs version to vX.Y.Z-rcN in package.json.
      • Run npm install locally
      • Push the kubo-release-vX.Y.Z-rcn branch to GitHub and create a draft PR from that branch (example).
    • go-ipfs-api
      • Create a branch with kubo version pinned in the test setup action (example).
      • Ensure that CI is green.
      • Delete the branch.
    • go-ipfs-http-client
      • Create a branch with kubo version pinned in the test setup action (example).
      • Ensure that CI is green.
      • Delete the branch.
    • WebUI
      • Run CI workflow with vX.Y.Z-rcN for the kubo-version input.
      • Ensure that CI is green.
  • Infrastructure Testing.
    • Open an issue against bifrost-infra (example).
      • Spell out all that we want updated - gateways, the bootstraper and the cluster/preload nodes
      • Mention @protocol/bifrost-team in the issue if you opened it prior to the release
      • [Optional] Reply under a message about the issue in the #bifrost channel on FIL Slack once the RC is out. Send the message to the channel.
      • Check metrics every day.
        • Compare the metrics trends week over week.
        • If there is an unexpected variation in the trend, message the #bifrost channel on FIL Slack and ask for help investigation the cause.
  • IPFS Application Testing.
    • IPFS Desktop
      • Upgrade to the RC in ipfs-desktop
      • Run npm install to update package-lock.json.
      • Push to a branch (example)
      • Open a draft PR to track through the final release (example)
      • Ensure CI tests pass
    • IPFS Companion
      • Start kubo daemon of the version to release.
      • Start a fresh chromium or chrome instance using chromium --user-data-dir=$(mktemp -d) (macos /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --user-data-dir=$(mktemp -d))
      • Start a fresh firefox instance using firefox --profile $(mktemp -d) (macos /Applications/Firefox.app/Contents/MacOS/firefox --profile $(mktemp -d))
      • Install IPFS Companion from vendor-specific store.
      • Check that the comunication between Kubo daemon and IPFS companion is working properly checking if the number of connected peers changes.
• Stage 5 - Release - ONLY FOR FINAL RELEASE
  • Prepare the release branch.
    • Bump the version in version.go in the release-vX.Y.Z branch to vX.Y.Z.
    • Push the release-vX.Y.Z branch to GitHub (git push origin release-vX.Y.Z)
    • Mark the PR created from release-vX.Y.Z as ready for review.
      • Ensure the PR is targetting release branch.
      • Ensure that CI is green.
      • Have release reviewer review the PR.
    • Merge the PR into release branch using the Create a merge commit (do NOT use Squash and merge nor Rebase and merge because we need to be able to sign the merge commit).
    • Checkout the release branch locally.
      • Remember to pull the latest changes.
    • Create a signed tag for the release.
      • Have release reviewer review the subsequent tagging commits you intend to run.
        • This is a dangerous operation, as it is difficult to reverse due to Go modules and automated Docker image publishing
      • Tag HEAD release commit with vX.Y.Z (git tag -s vX.Y.Z -m 'Release X.Y.Z')
      • Push the vX.Y.Z tag to GitHub (git push origin vX.Y.Z; DO NOT USE git push --tags because it pushes all your local tags).
    • Wait for Publish docker image workflow run initiated by the tag push to finish.
  • Add artifacts to https://dist.ipfs.tech by making a PR against ipfs/distributions
    • Clone the ipfs/distributions repo locally.
    • Create a new branch (kubo-release-vX.Y.Z) from master.
    • Run ./dist.sh add-version kubo vX.Y.Z to add the new version to the versions file (instructions).
    • Push the kubo-release-vX.Y.Z branch to GitHub and create a PR from that branch (example).
    • Wait for PR to build artifacts and generate diff (~30min)
    • Inspect results, merge if CI is green and the diff looks ok
    • Wait for master branch to build. It will automatically update DNSLink at https://dist.ipfs.tech (~30min)
  • Publish the release to the NPM package by running https://github.com/ipfs/npm-go-ipfs/actions/workflows/main.yml (it happens automatically but it is safe to speed up the process and kick of a run manually)
  • Publish the release to chocolatey by running https://github.com/ipfs/choco-go-ipfs/actions/workflows/main.yml (it happens automatically but it is safe to speed up the process and kick of a run manually)
    • Wait for Chocolatey to approve the release (usually takes a few hours)
  • Ensure the release is published to snap
  • Ensure the release is published to arch (flag it out of date)
  • Cut the release on GitHub (instructions, example)
    • Use vX.Y.Z as the tag.
    • Link to the release issue in the description.
    • Copy the relevant changelog into the release description.
      • Keep the release notes as trim as possible (e.g. remove top headers where possible, example)
  • Synchronize release artifacts by running sync-release-assets workflow.
  • Announce the RC
    • Create a new post on IPFS Discourse. (example)
      • Use Kubo vX.Y.Z Release is out! as the title.
      • Use kubo and go-ipfs as topics.
      • Repeat the title as a heading (##) in the description.
      • Link to the GitHub Release, binaries on IPNS, docker pull command and release notes in the description.
    • Pin the topic globally so that it stays at the top of the category.
    • If there is no more important banner currently set on Discourse (e.g. IPFS Camp announcement), make the topic into a banner.
    • Check if Discourse post was automatically copied to:
      • IPFS Discord #ipfs-chatter
      • FIL Slack #ipfs-chatter
      • Matrix
  • Add a link from release notes to Discuss post (like we did here: https://github.com/ipfs/kubo/releases/tag/v0.15.0)
  • Update the draft PR created for interop to use the new release and mark it as ready for review.
  • Update the draft PR created for IPFS Desktop to use the new release and mark it as ready for review.
  • Update docs by merging the auto-created PR in https://github.com/ipfs/ipfs-docs/pulls (example) (they are created automatically every 12 hours)
  • Get the blog post created and shared
    • Submit a request for blog post creation using the form.
      • Notify marketing in #shared-pl-marketing-requests about the blog entry request (since the form tends to go to spam).
      • Don't mark this as done until the blog entry is live.
  • Share link to the release on:
    • Twitter (request in Filecoin Slack channel #shared-pl-marketing-requests)
    • Reddit
• Stage 6 - Post-Release
  • Merge the release branch back into master, ignoring the changes to version.go (keep the -dev version from master).
  • Create an issue using this release issue template for the next release.

⁉️ Do you have questions?

The best place to ask your questions about IPFS, how it works and what you can do with it is at discuss.ipfs.io. We are also available at the #ipfs channel on Freenode, which is also accessible through our Matrix bridge.

Release improvements for next time

< Add any release improvements that were observed this cycle here so they can get incorporated into future releases. >

Items for a separate comment

< Do these as a separate comment to avoid the main issue from getting too large and checkbox updates taking too long. >

Changelog

< changelog generated by bin/mkreleaselog > (add it to a separated comment if it is too big)

❤️ Contributors

< list generated by bin/mkreleaselog >

Would you like to contribute to the IPFS project and don't know how? Well, there are a few places you can get started:

• Check the issues with the help wanted label in the ipfs/kubo repo
• Join an IPFS All Hands, introduce yourself and let us know where you would like to contribute - https://github.com/ipfs/team-mgmt/#weekly-ipfs-all-hands
• Hack with IPFS and show us what you made! The All Hands call is also the perfect venue for demos, join in and show us what you built
• Join the discussion at discuss.ipfs.io and help users finding their answers.
• Join the 🚀 IPFS Core Implementations Weekly Sync 🛰 and be part of the action!

[BigLep]

@ajnavarro @galargh : Slack thread on what's remaining before the release: https://filecoinproject.slack.com/archives/C03L0G3B4RX/p1663819594018819

[galargh]

We're moving the RC1 release to Monday (2022-09-26) to give enough headway for merging What's left for release. As of now, the rest of the schedule remains unchanged.

[BigLep]

Thanks for updating the dates @galargh. To provide a bit more color, we aren't cutting the RC today because we just wrapped up the must-have work of #9274 (and it's EOD Europoe time).

We don't do releases on Friday.

Pushing to Monday 2022-09-26 also gives us the ability to get two other items that were almost complete:

• feat(gateway): _redirects file support #8890
• feat: add webtransport as an optin transport #9293

[galargh]

Early testers ping for RC1 testing 😄

• pacman.store (@RubenKelevra)
• Infura (@MichaelMure)
• Textile (@sanderpick)
• Pinata (@obo20)
• RTrade (@postables)
• QRI (@b5)
• Siderus (@koalalorenzo)
• Charity Engine (@Rytiss, @tristanolive)
• Fission (@bmann)
• OrbitDB (@tabcat)

[BigLep]

Something that needs to improved for next time in my opinion is working on the changelog before the RC. Right now someone doesn't see much on what has changes, what they should try, what they should look out for, etc.

[galargh]

Agreed, it would be beneficial to have it ready in time for the RC. BTW, here's a draft changelog that @ajnavarro prepared - https://github.com/ipfs/kubo/blob/v0.16.0-rc1/docs/changelogs/v0.16.md. It's also linked from the release. At the very least we should formalise how it gets created.

[lidel]

Remaining changelog items added in #9315 (ready for review)

[BigLep]

Here was the state of https://github.com/orgs/ipfs/projects/16/views/7?filterQuery=iteration%3A%22kubo+0.16%22 efore moving things to the next iteration/release. I attached as XLSX because copy/paste is causing column misalignment that I don't have time to figure out.

0.16 status on 2022-10-04.xlsx

[galargh]

We have successfully release v0.16.0: https://github.com/ipfs/kubo/releases/tag/v0.16.0