Rebase to a new version of Fedora (34)

[dustymabe]

Rebase to a new version of Fedora (34)

Release engineering changes

• Verify that a few tags have been created. These should have been created by releng scripts on branching:

• f${releasever}-coreos-signing-pending

• f${releasever}-coreos-continuous

• The tag info for the coreos-pool tag has the new release (N) and next release (N+1) signing keys (just to stay ahead of the curve) and removes the old release (N-2) signing key. The following commands view the current settings and then update the list to 32/33/34 keys. You'll most likely have to get someone from releng to run the second command (edit-tag).

• koji taginfo coreos-pool

• koji edit-tag coreos-pool -x tag2distrepo.keys="12c944d0 9570ff31 45719a39"

• koji untag N-2 packages from the pool (at some point we'll have GC in place to do this for us, but for now we must remember to do this manually or otherwise distRepo will fail once the signed packages are GC'ed). For example the following snippet finds all RPMs signed by the Fedora 32 key and untags them.

Find the key short hash. Usually found here. Then:

f32key=12c944d0
key=$f32key
untaglist=''
for build in $(koji list-tagged --quiet coreos-pool | cut -f1 -d' '); do
    if koji buildinfo $build | grep $key 1>/dev/null; then
        untaglist+="${build} "
        echo "Adding $build to untag list"
    fi
done

Now we have a list of builds to untag. But we need one more sanity check. Let's make sure none of those are actually being used. Fire up the latest FCOS testing-devel and run:

f32key=12c944d0
key=$f32key
rpm -qai | grep -B 8 $key

If there are any RPMs signed by the old key they'll need to be investigated. Maybe they shouldn't be used any longer. Or maybe they're still needed.

After verifying the list looks good:

koji untag-build coreos-pool $untaglist

Now that untagging is done, give a heads up to rpm-ostree developers that N-2 packages have been untagged and that they may need to update their CI compose tests to freeze on a newer FCOS commit.

coreos-installer changes

• Update coreos-installer to know about the signing key used for the future new major version of Fedora (N+1). Note that the signing keys for N+1 may not be created until releng branches and rawhide becomes N+1.

Update fedora-coreos-config next-devel

• Bump releasever in manifest.yaml
• Update the repos in manifest.yaml if needed
• Run cosa fetch --update-lockfile
• PR the result

Ship rebased next

• Ship next
• Set a new update barrier for N-2 on next. In the barrier entry set a link to the docs. See discussion.

Update fedora-coreos-config testing-devel

• Bump releasever in manifest.yaml
• Update the repos in manifest.yaml if needed
• Run cosa fetch --update-lockfile
• Bump the base Fedora version in ci/buildroot/Dockerfile
• PR the result

Ship rebased testing

• Ship testing
• Set a new update barrier for N-2 on testing. In the barrier entry set a link to the docs.

Ship rebased stable

• Ship stable
• Set a new update barrier for N-2 on stable. In the barrier entry set a link to the docs.

Miscellaneous container updates

These are various containers in use throughout our ecosystem. We should update or open a ticket to track updating them once a new Fedora release is out. If you open a ticket instead of doing the update add a link to the ticket as comment.

• Update coreos-assembler or open ticket to update:
  • Dockerfile
• Update coreos-installer
  • Dockerfile
• Update fedora-coreos-cincinnati
  • Dockerfile
• Update config-bot
  • Dockerfile
• Update coreos-koji-tagger
  • Dockerfile
  • ImageStream
  • BuildConfig
• Update coreos-ostree-importer
  • Dockerfile
  • ImageStream
  • BuildConfig

[dustymabe]

coreos/coreos-installer#530

[dustymabe]

coreos/fedora-coreos-config#902

[bgilbert]

coreos/coreos-installer#529

[jlebon]

coreos/fedora-coreos-config#991

[travier]

coreos/fedora-coreos-config#997 > Testing moved to F34

[dustymabe]

We skipped the update barriers since updating from older releases was broken for now, but maybe we should go ahead and put them in place just for consistency?

[dustymabe]

We skipped the update barriers since updating from older releases was broken for now, but maybe we should go ahead and put them in place just for consistency?

coreos/fedora-coreos-streams#323

[dustymabe]

@bgilbert @lucab - Are we unblocked now to move coreos-installer and fedora-coreos-cincinnati to f34 ? Those are our two remaining checkboxes.

[bgilbert]

Looks like F34-based coreos-installer container builds still fail in Quay. The tracking issue is PROJQUAY-2006.

[lucab]

coreos/fedora-coreos-cincinnati#46

[lucab]

The cluster on fedora-infra seems to have the same problem (EPERM/ENOSYS mismatch) possibly due to an outdated container runtime stack. It's tracked at https://pagure.io/fedora-infrastructure/issue/10111.

[bgilbert]

It looks like Quay is fixed now. Submitted coreos/coreos-installer#617.

[bgilbert]

fedora-coreos-cincinnati was reverted in coreos/fedora-coreos-cincinnati#49.

[travier]

We are moving to F35 now. How are we here?

[jlebon]

Yup, this is clearly done. Good catch @travier!

[bgilbert]

We were holding it open for the fedora-coreos-cincinnati Dockerfile bump, but that can equally well be covered by the F35 ticket.