FreeBSD packages can disappear from one day to the next

[grahamperrin]

-CURRENT

https://matrix.to/#/!EKNFbsWSwXpDOGLRex:matrix.org/$MdfA8Gvk7rhjk5BBC6tVlsbny7FPH01NapdBXtRdi28?via=matrix.org&via=fosdem.org&via=t2bot.io (2021-01-15):

3 packages disappeared in 13-CURRENT packages...

pkg-static: No packages available to install matching 'virtualbox-ose-additions' have been found in the repositories
pkg-static: No packages available to install matching 'utouch-kmod' have been found in the repositories
pkg-static: No packages available to install matching 'drm-legacy-kmod' have been found in the repositories

Retrospective

Disappearance of drm-legacy-kmod was proper; https://www.freshports.org/graphics/drm-legacy-kmod/#history shows removal.

Looking ahead

1. At any time, seek main-amd64 at https://pkg-status.freebsd.org/builds?type=package
2. then for example, https://pkg-status.freebsd.org/builds/default:default:main-amd64:p565091_sa375ec52a7:beefy18#new_failed New Failed
3. and if you like, compare to the previous build.

[grahamperrin]

End of life (EOL) FreeBSD 12.1-RELEASE

release_2

ISO/build.sh

Line 175 in 4173696

sed -i -e 's|quarterly|release_2|g' "${uzip}/etc/pkg/FreeBSD.conf"

#1 (comment)

… not using this quarter's packages but last quarter's ones, because …

FreeBSD 12.2-RELEASE-p3

quarterly and default

… quarterly can have missing packages any day...

1. At any time, seek 122amd64 at pkg-status.freebsd.org/builds?type=package
2. then for example, http://beefy2.nyi.freebsd.org/jail.html?mastername=122amd64-quarterly for quarterly – or http://beefy6.nyi.freebsd.org/jail.html?mastername=122amd64-default

When a port was updated, it can take some time until the binary package gets updated. How much time? I don't know.

Exceptionally (for an initial build of as much as possible): poudriere might take around four days. See for example:

• http://beefy2.nyi.freebsd.org/build.html?mastername=122amd64-quarterly&build=563981 95:37:36
• http://beefy6.nyi.freebsd.org/build.html?mastername=122amd64-default&build=563982 96:41:58

Normally: poudriere can complete its work in far less time. See for example:

• http://beefy2.nyi.freebsd.org/build.html?mastername=122amd64-quarterly&build=564755 23:59:00
• http://beefy2.nyi.freebsd.org/build.html?mastername=122amd64-quarterly&build=564922 02:57:47
• http://beefy6.nyi.freebsd.org/build.html?mastername=122amd64-default&build=564752 52:42:55

[grahamperrin]

#1 (comment)

… this quarter's packages can disappear from one day to the next, making it impossible to build the ISO).

We still need to find a better way for this. …

#135 (comment)

… cannot use quarterly packages because packages can disappear there from one day to the next, preventing us from doing ISO builds. …

The key to avoiding disappointment will be knowledge of fallout of required packages.

Fallout

We see https://lists.freebsd.org/mailman/listinfo/freebsd-pkg-fallout – freebsd-pkg-fallout -- Fallout logs from package building – however the last post was in 2019.

I do receive fallout-related e-mails. Thirteen hours ago, for example:

[Bug 253484] [exp-run] Update KDE Frameworks to 5.79.0

Re: https://bugs.freebsd.org/bugzilla/show_activity.cgi?id=253484 I'm not a cc recipient of such things, so I need to remind myself of the method through which I chose to receive notifications. Something in Bugzilla, I guess.

[probonopd]

The key to avoiding disappointment will be knowledge of fallout of required packages.

The actual key will be to get to some sort of setup that prevents Fallout from happening. After all, we do not want to know that we cannot build, we want to avoid the situation that we cannot build...

I certainly don't want to keep track of which packages are missing today, and need to be able to build at any time. Not being able to build for a couple of days would be a showstopper for me.

I can't remember that I had ever track down missing packages in e.g., Debian Unstable...

[grahamperrin]

Fair enough,

… Not being able to build for a couple of days would be a showstopper for me. …

Maybe use poudriere to maintain, for that build purpose, a minimal repo.

Will you have a separate computer for builds, or are you limited to 12.1?

Maybe some overlap with #51

[probonopd]

Will you have a separate computer for builds, or are you limited to 12.1?

In fact, my "computer" that does all the building is https://cirrus-ci.com/ and they do a fantastic job at it. Much better than I could ever do.

[grahamperrin]

Is disk space a concern?

[probonopd]

Well, it's a continuous build pipeline without persistent storage.

[probonopd]

Looks like we are not the only ones frustrated by this. @pkgdemon pointed out on IRC:

but the reason i dropped furybsd (...) is things like drm-kmod stay broken in quarterlies for each point release for an enitre quarter. on top of that desktop packages, and even browsers were randomly broken. it lead me to conclude that the only way forward was to invest in a build infrstrucutre to be able to build and test my own
packages and push when they were ready. as much as i hoped stock would work. therefore i joined ghostbsd and rolled my work into that since they already had the donations to support a good build mirror/infrastructure. i'm slowly fixing a lot of the openrc bugs there and things are becoming a lot more polished and on the way to acting more like stock behavior.

[grahamperrin]

NB also the comments on latest. Me:

Interesting observations on quarterly. I very rarely notice breakages with latest.

@pkgdemon replied:

Yeah I’m starting to think also latest is better kept

I’m also using latest now

[crees]

The fallout really falls on the ports committer who committed broken code... though it would be nice for the 'working' previous package to not disappear. I'll find out some more about exactly why this happens.

By the way, you should just remove drm-legacy-kmod as it's not relevant or necessary, but my latest pull request deals with that anyway. #165

[grahamperrin]

#181 (comment) emulators/virtualbox-ose-additions is available in latest and quarterly for FreeBSD:13:amd64