[DNF5] Text syntax for more precise definition of transaction objects, possibility to combine different types of objects and actions in the transaction

[jrohel]

Design a text syntax usable for the CLI (command line interface) that allows you to:

• specify the type of transaction object (package/group/module/...); solves [DNF5] Allow to install multiple object types (rpm/group/modules) in one transaction libdnf#1375
• specify other properties of the object (from which the repository comes, what it provides, ...); solves [DNF5] Find a way how to replace repositorypkgs command by CLI syntax libdnf#1376
• work with multiple types of objects (package/group/module) in one transaction
• multiple types of actions (install/remove/upgrade/downgrade/reinstall/enable/disable) in one transaction; more powerfull than DNF command swap, can replace dnf shell
• set reason of installed/modified object (user/dependency/...)

I have some ideas. One example of writing a transaction object is extending the spec argument with query and action:

• spec[?query][@action]
• ?query[@action]

Examples of use:

Install package "abc" in version>=2.4 from repository with id "test". Set reason dependency.
microdnf install abc?version>=2.4&repo=test@reason=dependency
or
microdnf install ?name=abc&version>=2.4&repo=test@reason=dependency

More action types in one transaction. Replace package abc with package xyz - swap.
microdnf install xyz abc@action=remove

Mix instalation of package "abc" and module "xyz" in one transaction.
microdnf install abc xyz?type=module

Installs the package providing the required provide.
microdnf install '?provides=abc > 5'

Examples of supported attributes and actions:

query keys: type (package, module, group, ...), name, epoch, version, release, arch, provides, requires, repo, reason (user, dependency, group, not_installed)

action keys:
reason - (user, dependency, ...), action (install, remove, upgrade, downgrade, reinstall, enable, disable)

This is a proposal. No promise that this will be in microdnf. Please let me know what you think. Alternatively, how to improve the syntax.

[jrohel]

• The '?' is used in spec for pattern matching (matches any single character).
• I discussed with @j-mracek. I will prepare changes to the proposal.

[dmach]

Extended spec syntax would be nice, but does anyone need it now?
Wouldn't be easier to follow what @Conan-Kudo suggested couple months back? :
Introduce -t/--type option that would specify type of all following specs until another --type is specified.

For example:

dnf install --type=pkg bash kernel --type=comps-group core

It might make sense to introduce more options, such as --action and do following:

dnf {do/run}-transaction \
  --type=pkg \
  --action=install bash kernel \
  --action=remove iftop \
  --action=upgrade '*' \
  \
  --type=comps-group \
  --action=install core

You could also combine it with various filters such as --from-repo=<repoid> etc.

[lukash]

Not sure about the --type, that seems cumbersome, I'd just go with e.g. package, @group, %module (the % fits quite well, doesn't it? not sure if it's a special character in any context, seems in bash it would be fine).

But besides that I find the suggested syntax overly complicated. I was thinking along the same lines with the --action= that'd set the action for the following artifacts until another action is specified. Except I think we could just go with --install, --upgrade, --remove, etc. for simplicity.

[dmach]

I'd keep the @group syntax for compatibility reasons, but I don't think we should extend it. We're running out of suitable characters already :)

If a new spec syntax is a thing, then we should make it easy to use and powerful & extensible.
What I'm looking for is:

• work with various types of artifacts (== define "package" type)
• install artifacts as dependencies (so they can get autoremoved - useful for various devel/debug packages)
• narrow down resolving spec to certain repositories
• narrow down resolving spec to certain attributes (e.g. match only package name and nothing else)

Several ideas:

• dnf install <name>,pkg - install a RPM package; 'pkg' is default type so it can be omitted
• dnf install <name>,grp - install a comps group
• dnf install <name>,env - install a comps environment
• dnf install <name>,mod - install a module (profiles?)
• dnf install <name>,dep - install package as a dependency
• dnf install <name>,repo=fedora,repo=updates - install package from either 'fedora' or 'updates' repo
• dnf install <name>,match=n,match=na - narrow down spec resolution to name and name.arch only
• dnf install <name>,match=prv - narrow down spec resolution to provides only
• dnf install <name>,match=fil - narrow down spec resolution to files only

I personally prefer , over ?/& because we're not constructing URLs - it has to be easy to use for the users.

Open questions:

• how to handle local installs?
  • dnf install /foo/bar,pkg,match=pkg/baz.rpm,dep
  • should get resolved to installing /foo/bar,pkg,match=pkg/baz.rpm as a depedency
• how to handle builddep?
  • dnf builddep https://example.com/pkg.spec,dep

For completeness, zypper supports the following options (we probably don't want the options, but we want the features - probably as spec flags/modifiers)

$ zypper install --help
-t, --type <TYPE>           Type of package (package, patch, pattern, product).
-n, --name                  Select packages by plain name, not by capability. Default: false
-C, --capability            Select packages solely by capability. Default: false
    --from <ALIAS|#|URI>    Select packages from the specified repository.
-r, --repo <ALIAS|#|URI>    Work only with the specified repository.

Gentoo's emerge has an interesting option:
https://dev.gentoo.org/~zmedico/portage/doc/man/emerge.1.html

--onlydeps, -o
    Only merge (or pretend to merge) the dependencies of the packages specified, not the packages themselves. 

Arch Linux's pacman:
https://archlinux.org/pacman/pacman.8.html

--asdeps
    Install packages non-explicitly; in other words, fake their install reason to be installed as a dependency. This is useful for makepkg and other build-from-source tools that need to install dependencies before building the package.

--asexplicit
    Install packages explicitly; in other words, fake their install reason to be explicitly installed. This is useful if you want to mark a dependency as explicitly installed so it will not be removed by the --recursive remove operation.

If you know about how the other package managers do it, please link it here.

[j-mracek]

Due to difficulty with parsing I would be very careful with extending spec definition. Even a group syntax with @ was a step in incorrect direction therefore why we should go in that direction and extend that idea. I thing a better direction is to use options.

We would like to install group base and package microdnf in one transaction.
dnf install @base microdnf - dnf4 syntax
dnf install --group=base microdnf - use an option to define group argument

We would like to install module perl:5.26 and package microdnf in one transaction.
dnf install @base microdnf - dnf4 syntax
dnf install --module=perl:5.26 microdnf - use an option to define group argument

Or we can provide a capability of

To add a new option to set install reason for newly installed packages sounds like a good idea.

Anyway I will be happy to see reworked idea from @jrohel related to new capabilities how to define action target.

[Conan-Kudo]

What about <type>/<spec>? For example: dnf install group/base package/microdnf module/perl:5.26 pattern/rpmdev?

[dmach]

What about <type>/<spec>? For example: dnf install group/base package/microdnf module/perl:5.26 pattern/rpmdev?

/ is tricky as it's always part of paths and URLs. We need something that creates as few problems as possible (there's probably nothing perfect) and I'm not sure if this is it.

[lukash]

If you know about how the other package managers do it, please link it here.

Wrt. Gentoo, more importantly than --onlydeps (which I've never used), it has --oneshot (also on the linked manpage), which I've used a lot:

--oneshot, -1
    Emerge as normal, but do not add the packages to the world file for later updating.

The docstring here is somewhat unclear but it basically does the same as --asdeps: installs a package as a dependency, meaning it would get autocleaned later.

But:

dnf install <name>,dep - install package as a dependency

Maybe the dep specifier shouldn't be there at all...

Disregarding the particular syntax for specifying the type of the argument (though I do think we should have special syntax for that), I think we should leave out the action and reason (which is only really the asdep / oneshot flag AFAICS) out of it.

Something like this seems simpler and easier on the users:

dnf operate --install foo?repo=fedora --remove bar?type=group

(using the originally suggested syntax for the specifiers just for the example)

And for the reason userinstalled / asdep flag, it seems only relevant for the install command so in the simplest form we could just have:

dnf operate --install-dep foo

[ppisar]

Container people want to install a package by its checksum (issue #833). The checksum is provided in primary.xml. The checksum could be added to here proposed query tags.