external: kernel version from depsolve (COMPOSER-2354)

[supakeen]

Provide a (wide) API from the depsolve to query package versions; this is split out from #196 and needs separate discussion; previously:

@mvo5:

This adds a very broad API to this otk external, I personally would start more targeted, we need the kernel name and version and in images the kernel is handled specially only. So maybe we could do the same here? Something like just expose a const.kernel.{name,version}? We will need to adjust fragments/grub2.yam to match the kernel and the string there includes 0-0.noarch right now which also indicates that we need to tweak our tests to make this better testable.

@supakeen:

I chose the broad API to not limit ourselves here also taking some inspiration from osbuild-mpp where it is currently possible to get the version of any package.

[mvo5]

The issue I see (beside api surface) is that currently we support changing the kernel name in images easily, so I expect in our otk files we will have something like:

otk.define:
 kernel_pkgname: cloud-kernel

and with that we would need lookup inside usage of defines, yes? I.e. something like

saved_entry: ffffffffffffffffffffffffffffffff-${packages.os.const.versions.${kernel_pkgname}.version}

which iirc we do not support currently.

We could of course but I'm not sure we should for this use-case since we now need to write a new feature and have a much bigger api surface.

If there is a good use-case for exposing all versions and all details for the versions (i.e. version/epoch/release/arch) beside the kernel then of course we should consider this.

[supakeen]

cc @achilleas-k and @ondrejbudai

[achilleas-k]

I responded on the previous PR but let me copy my comment here:

The only call to the equivalent function in images that's not for looking up the kernel is to check if dracut-config-rescue exists in the package set, but it doesn't care about version.
https://github.com/osbuild/images/blob/67a2b2f65a2e0969039a3de17bf2c266390d01ab/pkg/manifest/os.go#L651

I can imagine a case where this might be useful. We've never had to do this but perhaps a stage's options might change depending on the version of a given package (in the build root or in the OS).

I lean towards simplicity for these externals, but I also like the generalisation of being able to "query" for any package's version by name. It feels useful, so my instinct right now is to generalise it so it's easier to grow in the future.

[achilleas-k]

Have we considered a separate external that consumes the output of the depsolver to look up package info by name?
(widening the API further...)

[supakeen]

Have we considered a separate external that consumes the output of the depsolver to look up package info by name? (widening the API further...)

Kinda, I don't think it particularly adds much unless we want to find a package in multiple package sets (for example). However it does allow us to parametrize the package name more easily as @mvo5 brought up. This might be necessary for custom kernels?

[mvo5]

If we do it like this then the internal json representation of a "package" in dnfjson becomes an (external) API that we will need to maintain, i.e. we can never rename/remove/change e.g. remote_location or checksum. I would suggest to change it so that we only expose what is required for the kernel use-case instead as this is easier to maintain/keep compatible.

Fwiw, this is orthogonal to my point that I'm not sure we should expose the details for all packages by default (but limiting what we expose would certainly reduce it)

[ondrejbudai]

Storing packages by their names can cause issues with multi-arch images (x86_64 and i686) but I wouldn't worry about this too much.

I agree with Michael that directly exposing dnf-json's output creates a risky depedency, so I suggest exposing just VRA for now. :) We can always go bigger later.

Otherwise this lgtm. ;)

[supakeen]

Storing packages by their names can cause issues with multi-arch images (x86_64 and i686) but I wouldn't worry about this too much.

I agree with Michael that directly exposing dnf-json's output creates a risky depedency, so I suggest exposing just VRA for now. :) We can always go bigger later.

Otherwise this lgtm. ;)

Summarizing (also tagging @mvo5), expose all packages but only their Version-Release-Architecture right?

[mvo5]

Storing packages by their names can cause issues with multi-arch images (x86_64 and i686) but I wouldn't worry about this too much.
I agree with Michael that directly exposing dnf-json's output creates a risky depedency, so I suggest exposing just VRA for now. :) We can always go bigger later.
Otherwise this lgtm. ;)

Summarizing (also tagging @mvo5), expose all packages but only their Version-Release-Architecture right?

If we do this we still have the issue that any kernel name like (my-kernel) that is not a valid var in otk could not be represeted AFAICT. So ${packages.os.const.versions.my-kernel.version} does not work, we either need nesting which is a bit of a pain or some other way to access this. The alternative would be to provide a new external like "osbuild-dnf-details" or something that takes a "pkgname" and a resolved tree as input (iirc achilleas idea) - this would not require new "language" features (cc @ondrejbudai )

[supakeen]

Storing packages by their names can cause issues with multi-arch images (x86_64 and i686) but I wouldn't worry about this too much.
I agree with Michael that directly exposing dnf-json's output creates a risky depedency, so I suggest exposing just VRA for now. :) We can always go bigger later.
Otherwise this lgtm. ;)

Summarizing (also tagging @mvo5), expose all packages but only their Version-Release-Architecture right?

If we do this we still have the issue that any kernel name like (my-kernel) that is not a valid var in otk could not be represeted AFAICT. So ${packages.os.const.versions.my-kernel.version} does not work, we either need nesting which is a bit of a pain or some other way to access this. The alternative would be to provide a new external like "osbuild-dnf-details" or something that takes a "pkgname" and a resolved tree as input (iirc achilleas idea) - this would not require new "language" features (cc @ondrejbudai )

This is still an option yes. There's also a separate parallel discussion about names in #213 which would alleviate or make that harder :)

[mvo5]

Storing packages by their names can cause issues with multi-arch images (x86_64 and i686) but I wouldn't worry about this too much.
I agree with Michael that directly exposing dnf-json's output creates a risky depedency, so I suggest exposing just VRA for now. :) We can always go bigger later.
Otherwise this lgtm. ;)

Summarizing (also tagging @mvo5), expose all packages but only their Version-Release-Architecture right?

If we do this we still have the issue that any kernel name like (my-kernel) that is not a valid var in otk could not be represeted AFAICT. So ${packages.os.const.versions.my-kernel.version} does not work, we either need nesting which is a bit of a pain or some other way to access this. The alternative would be to provide a new external like "osbuild-dnf-details" or something that takes a "pkgname" and a resolved tree as input (iirc achilleas idea) - this would not require new "language" features (cc @ondrejbudai )

This is still an option yes. There's also a separate parallel discussion about names in #213 which would alleviate or make that harder :)

Right, if we merge this as is we will be force to solve #213 quickly too because otherwise this feature here will not work for certain package names. So I'm not sure we should before we have a plan for that.

[mvo5]

I opened #226 with an alternative idea that uses an external (iirc achilleas suggested this)

[supakeen]

We're going #226.