Create policies automatically for custom packages: BE(API)

[georgekarrv]

Epic: #23344

Server will attempt to match bundle id and create a policy based on that, failing to extract a bundle id we will attempt to create a policy based on the application name. If we get neither of those I would say we don't create a policy.

If we are unable to create a policy for any reason we will return 424 Failed Dependency and use the copy we have for if FMA succeeds to upload the software but fails to create the policy.

QA notes

Happy path

Tests with PKGs, MSIs, DEBs and RPMs. Uploading custom packages with "Automatic" will automatically create a policy with name [Install software] $TITLE ($EXTENSION) and linked to the installer (Description should also match the Figma).
Such policy will run on macOS only for PKGs, Windows only for MSIs, and Linux only for DEBs/RPMs.
Such policy should trigger an installation of the uploaded package on hosts that do not have the software installed (so you will need to test on Windows (MSI)+macOS (PKG)+Ubuntu (DEB)+Fedora (RPM) VMs).

PS: If possible, use installers that are not on this list so that we increase our coverage.

EXEs

Uploading an EXE will disable the "Automatic" option on the UI. We should also QA it's disallowed on the API (POST /api/latest/fleet/software/package).

Invalid packages

PKGs

Uploading no_bundle_identifier.pkg should fail because it doesn't have a bundle identifier (should match the Figma error).

MSIs

I was not able to find or generate MSIs without product code (GUID) to be able to test this error when uploading MSIs.

Policy with name exists already on the team

If a policy with the name [Install software] $TITLE ($EXTENSION) exists before uploading an installer with such $TITLE/$EXTENSION, then the generated policy will be [Install software] $TITLE ($EXTENSION) 1, if that one exists, then [Install software] $TITLE ($EXTENSION) 2, etc.

Linux

• For DEB packages we generate a policy that will always pass on Fedora systems (we don't want Fleet to send an RPM to be installed on a Debian-based systems).
• For RPM packages we generate a policy that will always pass on Debian based systems (we don't want Fleet to send a DEB to be installed on RPM-based systems).

[georgekarrv]

Hey team! Please add your planning poker estimate with Zenhub @dantecatalfamo @jahzielv @getvictor @mna

[lucasmrod]

@georgekarrv @marko-lisica I see comments around bundle identifier. Is the goal to start with pkg only for automatic policy? Or do we want to support automatically creating policies for all other package types too (exe, msi, deb, rpm)?

[georgekarrv]

the idea is to support all. Based on our extraction if there is a bundle id choose that first otherwise create a policy based on application name (this is just the easiest, we may need to tweak this logic if this is not very reliable)

[lucasmrod]

the idea is to support all. Based on our extraction if there is a bundle id choose that first otherwise create a policy based on application name (this is just the easiest, we may need to tweak this logic if this is not very reliable)

Gotcha. bundle_identifier is a macOS only thing, so for Linux and Windows we will need to map extracted package name with some field in the programs and deb_packages/rpm_packages tables respectively.
Am guessing the feature won't be 100% reliable with all the kinds of installers and their metadata. Are we ok with this?
In any case we can make sure the most used apps are working as expected. I can come up with a list, but having some advice on this would help.

Also, for Linux we have to make the distinction between a RPM and DEB distro, otherwise we might end up e.g. sending an RPM to install to Ubuntu. See https://github.com/fleetdm/fleet/blob/main/articles/automatic-software-install-in-fleet.md#ubuntu-deb.

@marko-lisica @georgekarrv

[marko-lisica]

so for Linux and Windows we will need to map extracted package name with some field in the programs and deb_packages/rpm_packages tables respectively.

@lucasmrod I think we want to use whatever we used to group Windows and Linux software to titles.

Am guessing the feature won't be 100% reliable with all the kinds of installers and their metadata. Are we ok with this?

That's ok. Users have the option to override policies used to trigger automatic install. I believe most popular apps should work.

In any case we can make sure the most used apps are working as expected. I can come up with a list, but having some advice on this would help.
I can collect a list of apps to test.

Also, for Linux we have to make the distinction between a RPM and DEB distro, otherwise we might end up e.g. sending an RPM to install to Ubuntu. See https://github.com/fleetdm/fleet/blob/main/articles/automatic-software-install-in-fleet.md#ubuntu-deb.

What do you mean, do we need to make sure we run policy only on specific Linux distros based on installer type or we just need different query for .deb and .rem installers?

[lucasmrod]

What do you mean, do we need to make sure we run policy only on specific Linux distros based on installer type or we just need different query for .deb and .rem installers?

Different query for .deb and .rpm installers. If we just do the following when uploading foobar.rpm to a team:

SELECT 1 FROM rpm_packages WHERE name = 'foobar';

Such policy will fail on Ubuntu devices and will trigger a RPM install of foobar.rpm on a Ubuntu (which we don't want).
Same scenario with DEB installers on RPM based distros.

So, the automatic query will need take this into account.

PS: To improve this, we may need a "RPM based Linux" and "DEB based Linux" builtin labels in the future, combined with the label defined software installers (feature being worked currently by the MDM team).

[sharon-fdm]

Lifted the estimation a little per discussion with @lucasmrod and understanding the full extent.

[lucasmrod]

If we are unable to create a policy for any reason we will return 424 Failed Dependency and use the copy we have for if FMA succeeds to upload the software but fails to create the policy.

@georgekarrv This comment in the decription is outdated, right? (FMA creates the policy UI side.)

[georgekarrv]

While FMA does do 2 separate calls today this will be a single call for custom packages. I would even say it would be good to dedup the logic and change FMA to use this param as well (but that's not the scope of this ticket)

The comment is not outdated and this is the expected flow to 424 if the software uploads but the policy fails to create.