Add deploy-via-container key for image.yaml

[cgwalters]

create_disk: Initialize stateroot earlier

We can do this earlier rather than intermixing it with
the pull/deploy. Prep for further work.

---

Add deploy-via-container key for image.yaml

This builds on the image-format: oci key to switch to using
the new ostree-container code to do the deployment (via rpm-ostree).

Unlike the above though, this is not something we can turn on
by default until we intend to hard cut over to fetching updates
via containers.

The intention is to use this to produce image builds that are
initially provisioned via a container image so that we can more
easily experiment with the "container native" flow.

A side note; I suspect this will indirectly fix
openshift/os#594
because now we're opening a single large file and pulling that
over virtio instead of lots of smaller files.

---

[cgwalters]

To try this out, first you will need an rpm-ostree build from coreos/rpm-ostree#3139

Then do e.g.
echo "deploy-via-container: true" >> src/config.image.yaml
then cosa build etc.

[root@cosa-devsh ~]# rpm-ostree status
State: idle
Deployments:
* ostree-unverified-image:oci-archive:/var/srv/walters/builds/fcos/builds/34.20211027.dev.2/x86_64/fedora-coreos-34.20211027.dev.2-ostree.x86_64.ociarchive
                 Timestamp: 2021-10-27T21:48:38Z
[root@cosa-devsh ~]# 

Now, what we actually want is something like container-reference: ostree-unverified-registry:quay.io/cgwalters/fcos:stable which would still do the deployment via the oscontainer, but then set up the remote as if it came from that reference. But that needs a bit more work in ostree-ext.

Or...I've been thinking about this more, and it feels like longer term we should be reworking the entire build flow around this, where we actually split things up like:

• find new rpm/config change
• build oscontainer and qemu image
• basic sanity tests on qemu (possibly more)
• push oscontainer to registry
• Intermix further testing and builds - things like the metal image etc. would be done by cosa buildprep quay.io/coreos/fcos@sha256:...
• Once testing passes, the container image is promoted to a tag (e.g. :stable) and then the pipeline also synchronizes s3 to that container tag

[cgwalters]

Now, what we actually want is something like container-reference: ostree-unverified-registry:quay.io/cgwalters/fcos:stable which would still do the deployment via the oscontainer, but then set up the remote as if it came from that reference. But that needs a bit more work in ostree-ext.

It turns out instead of typing in English here it was almost equally easy to do ostreedev/ostree-rs-ext#135 there

[jlebon]

Apart from the container target ref, what's the difference in end state between ostree admin deploy and rpm-ostree ex-container image deploy on-disk? Does it make sense to just support the setting the new container-imgref key bit so that rpm-ostree upgrade works OOTB for experimentation?

[cgwalters]

Apart from the container target ref, what's the difference in end state between ostree admin deploy and rpm-ostree ex-container image deploy on-disk?

Deploying via a container in this way also e.g. writes a ref for each blob in the container (today, just one, but in the future multiple), and also creates a new commit which is where the manifest is stored.

Does it make sense to just support the setting the new container-imgref key bit so that rpm-ostree upgrade works OOTB for experimentation?

An important aspect of this per above is that because we write a ref for the blob, we know not to re-download it when upgrading even if the container-imgref is not set.

[cgwalters]

OK, I tested this one in concert with coreos/rpm-ostree#3139 and since this is just an optional thing, nothing blocks landing this before that code lands.

[jlebon]

LGTM!