Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CachyOS and NoExtract fix #2

Merged
merged 37 commits into from
May 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
551ae20
Fixed missing files due to NoExtract option
GrabbenD Sep 30, 2023
44dd97d
Added CachyOS base
GrabbenD Sep 30, 2023
654823a
Merge branch 'main' into feature/cachyos
GrabbenD Oct 1, 2023
c7ba824
Fix *.pkg.tar.zst is corrupted (invalid or corrupted package (PGP sig…
GrabbenD Oct 1, 2023
509b05a
Fix WARNING: Possibly missing firmware for module: *
GrabbenD Oct 1, 2023
9d9b274
Use EEVDF-BORE, LTO & V3 optimized kernel
GrabbenD Oct 1, 2023
0f35c8f
Merge branch 'main' into feature/cachyos
GrabbenD Oct 1, 2023
20a093d
Merge branch 'main' into feature/cachyos
GrabbenD Oct 1, 2023
55cd3c6
Drop obsolete dependency
GrabbenD Oct 14, 2023
80fae23
Merge branch 'main' into feature/cachyos
GrabbenD Oct 14, 2023
aa130f0
Use dedicated directory for each base
GrabbenD Oct 14, 2023
c76c9f4
Fix broken HW due to x86_64-v4 packages
GrabbenD Oct 14, 2023
059a1a1
Update ostree.sh
GrabbenD Nov 1, 2023
c435558
Merge branch 'main' into feature/cachyos
GrabbenD Nov 9, 2023
5188e5e
Update ostree.sh
GrabbenD Nov 11, 2023
5545205
Merge branch 'main' into feature/cachyos
GrabbenD Nov 15, 2023
7683873
Update usage with CachyOS
GrabbenD Nov 15, 2023
b4d9b4b
Update ostree.sh
GrabbenD Dec 10, 2023
2e16fb9
Use global CDN
GrabbenD Apr 17, 2024
885cdab
Fix "invalid or corrupted package (PGP signature)"
GrabbenD Apr 17, 2024
9eb4f68
Use newer archlinux builder
GrabbenD Apr 17, 2024
ec9b529
Improve IO performance of example
GrabbenD Apr 17, 2024
985b549
Use long form arguments for Pacman
GrabbenD Apr 17, 2024
6bc2dd3
Improve syntax of Containerfile
GrabbenD Apr 17, 2024
3b9c8f1
Update OSTree GRUB2
GrabbenD Apr 17, 2024
510da83
Allow a existing OSTree system to build another
GrabbenD Apr 19, 2024
7afaf8e
Improve readability of help command
GrabbenD Apr 19, 2024
cdce882
Use Bash exit code for unknown commands
GrabbenD Apr 19, 2024
d5ec4bb
Fix check for read-only filesystem
GrabbenD Apr 19, 2024
6b048cb
Added scope to global variables
GrabbenD Apr 19, 2024
7194925
Fix missing export for Podman in Archiso
GrabbenD Apr 19, 2024
7c469d8
Use TMPDIR for cache
GrabbenD May 20, 2024
363baad
Use single quotes fro consistency
GrabbenD May 20, 2024
f791d89
Use Arch Linux in usage example
GrabbenD May 20, 2024
9514883
Use OSTree bootloader scripts automatically
GrabbenD May 20, 2024
4b7df8c
Fix inconsistent breakage from pipefail
GrabbenD May 20, 2024
fe374c1
Drop unused dependency
GrabbenD May 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 54 additions & 10 deletions Containerfile.host.example
Original file line number Diff line number Diff line change
@@ -1,22 +1,66 @@
## |
## | CUSTOMIZE INSTALLATION
## | OSTREE
## |

# Extend any base image with this
FROM localhost/ostree/base AS host
# Customize your system here
FROM localhost/ostree/base:latest AS host

# Prepre OSTree integration (https://wiki.archlinux.org/title/Mkinitcpio#Common_hooks)
RUN mkdir -p /etc/mkinitcpio.conf.d \
&& echo "HOOKS=(base systemd ostree autodetect modconf kms keyboard sd-vconsole block filesystems fsck)" > /etc/mkinitcpio.conf.d/ostree.conf

# Install kernel, firmware, microcode, filesystem tools, bootloader & ostree and run hooks once:
RUN pacman --noconfirm --sync \
linux \
linux-headers \
\
linux-firmware \
amd-ucode \
\
dosfstools \
xfsprogs \
\
grub \
mkinitcpio \
\
podman \
ostree \
which

# OSTree: Prepare microcode and initramfs
RUN moduledir=$(find /usr/lib/modules -mindepth 1 -maxdepth 1 -type d) \
&& cat /boot/*-ucode.img \
/boot/initramfs-linux-fallback.img \
> ${moduledir}/initramfs.img

# OSTree: Bootloader integration
RUN cp /usr/lib/libostree/* /etc/grub.d \
&& chmod +x /etc/grub.d/15_ostree

# Podman: native Overlay Diff for optimal Podman performance
RUN echo "options overlay metacopy=off redirect_dir=off" > /etc/modprobe.d/disable-overlay-redirect-dir.conf

## |
## | CUSTOMIZE
## |

# Mount disk locations
ARG OSTREE_SYS_BOOT_LABEL
ARG OSTREE_SYS_ROOT_LABEL
ARG OSTREE_SYS_HOME_LABEL
RUN echo "LABEL=${OSTREE_SYS_ROOT_LABEL} / xfs rw,relatime 0 1" >> /etc/fstab && \
echo "LABEL=${OSTREE_SYS_HOME_LABEL} /var/home xfs rw,relatime 0 2" >> /etc/fstab && \
echo "LABEL=${OSTREE_SYS_BOOT_LABEL} /boot/efi vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro 0 2" >> /etc/fstab
RUN echo "LABEL=${OSTREE_SYS_ROOT_LABEL} / xfs rw,relatime,noatime 0 1" >> /etc/fstab \
&& echo "LABEL=${OSTREE_SYS_HOME_LABEL} /var/home xfs rw,relatime,noatime 0 2" >> /etc/fstab \
&& echo "LABEL=${OSTREE_SYS_BOOT_LABEL} /boot/efi vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro 0 2" >> /etc/fstab

# Networking
RUN pacman --noconfirm --sync networkmanager \
&& systemctl enable NetworkManager.service \
&& systemctl mask systemd-networkd-wait-online.service

# Root password (todo move to secret)
# Root password
RUN echo "root:ostree" | chpasswd

# SSHD
RUN pacman --noconfirm -S openssh && \
systemctl enable sshd && \
echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
RUN pacman --noconfirm -S openssh \
&& systemctl enable sshd \
&& echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,5 @@ $ ostree admin unlock
Your persistent cache is out of sync with upstream, this can be resolved with:

```console
$ ./ostree.sh upgrade --no-cache
$ ./ostree.sh upgrade --no-podman-cache
```

82 changes: 26 additions & 56 deletions archlinux/Containerfile.base
Original file line number Diff line number Diff line change
@@ -1,66 +1,36 @@
## |
## | BASE INSTALLATION
## |
# |
# | ROOTFS
# |

# Reusable base template
FROM docker.io/library/archlinux AS base
# Build a clean system in /mnt to avoid missing files from NoExtract option in upstream
FROM docker.io/archlinux/archlinux:latest AS rootfs

# Build in chroot to correctly execute hooks, this uses host's Pacman
RUN curl https://raw.githubusercontent.com/archlinux/svntogit-packages/packages/pacman/trunk/pacman.conf -o /etc/pacman.conf \
&& pacman --noconfirm --sync --needed --refresh archlinux-keyring

# Perform a clean system installation with latest Arch Linux packages in chroot to correctly execute hooks, this uses host's Pacman
RUN pacman --noconfirm --sync --needed arch-install-scripts \
&& pacstrap -K -P /mnt base \
&& cp -av /etc/pacman.d/ /mnt/etc/

# Remove container specific storage optimization in Pacman
RUN sed -i -e "s|^NoExtract.*||g" /etc/pacman.conf && \
pacman --noconfirm -Syu
# |
# | BASE
# |

# Reusable base template
FROM scratch AS base
COPY --from=rootfs /mnt /

# Clock
ARG SYSTEM_OPT_TIMEZONE
RUN ln -sf /usr/share/zoneinfo/${SYSTEM_OPT_TIMEZONE} /etc/localtime
RUN ln --symbolic --force /usr/share/zoneinfo/${SYSTEM_OPT_TIMEZONE} /etc/localtime

# Keymap hook
ARG SYSTEM_OPT_KEYMAP
RUN echo "KEYMAP=${SYSTEM_OPT_KEYMAP}" | tee /etc/vconsole.conf
RUN echo "KEYMAP=${SYSTEM_OPT_KEYMAP}" > /etc/vconsole.conf

# Language
RUN echo "LANG=en_US.UTF-8" | tee /etc/locale.conf && \
echo "en_US.UTF-8 UTF-8" | tee /etc/locale.gen && \
locale-gen

# Networking
RUN pacman --noconfirm -S networkmanager && \
systemctl enable NetworkManager.service && \
systemctl mask systemd-networkd-wait-online.service

## |
## | OSTREE INSTALLATION
## |

# Prepre OSTree integration (https://wiki.archlinux.org/title/Mkinitcpio#Common_hooks)
RUN mkdir -p /etc/mkinitcpio.conf.d && \
echo "HOOKS=(base systemd ostree autodetect modconf kms keyboard sd-vconsole block filesystems fsck)" >> /etc/mkinitcpio.conf.d/ostree.conf

# Install kernel, firmware, microcode, filesystem tools, bootloader, depndencies and run hooks once:
RUN pacman --noconfirm -S \
linux \
linux-headers \
linux-firmware \
amd-ucode \
\
dosfstools \
xfsprogs \
\
grub \
mkinitcpio \
\
podman \
ostree \
which

# OSTree: Prepare microcode and initramfs
RUN moduledir=$(find /usr/lib/modules -mindepth 1 -maxdepth 1 -type d) && \
cat /boot/*-ucode.img \
/boot/initramfs-linux-fallback.img \
> ${moduledir}/initramfs.img

# OSTree: Bootloader integration
RUN curl https://raw.githubusercontent.com/ostreedev/ostree/v2023.6/src/boot/grub2/grub2-15_ostree -o /etc/grub.d/15_ostree && \
chmod +x /etc/grub.d/15_ostree

# Podman: native Overlay Diff for optimal Podman performance
RUN echo "options overlay metacopy=off redirect_dir=off" > /etc/modprobe.d/disable-overlay-redirect-dir.conf
RUN echo 'LANG=en_US.UTF-8' > /etc/locale.conf \
&& echo 'en_US.UTF-8 UTF-8' > /etc/locale.gen \
&& locale-gen
43 changes: 43 additions & 0 deletions cachyos/Containerfile.base
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# |
# | ROOTFS
# |

# Build a clean system in /mnt to avoid missing files from NoExtract option in upstream
FROM docker.io/archlinux/archlinux:latest AS rootfs

# Generate a GPG secret key for signing and import CachyOS keys
RUN pacman-key --init \
&& pacman-key --recv-keys F3B607488DB35A47 --keyserver keyserver.ubuntu.com \
&& pacman-key --lsign-key F3B607488DB35A47

# Import CachyOS x86_64-v3 repositories to current package manager
COPY etc/pacman.conf /etc/
RUN curl https://raw.githubusercontent.com/CachyOS/CachyOS-PKGBUILDS/master/cachyos-mirrorlist/cachyos-mirrorlist -o /etc/pacman.d/cachyos-mirrorlist \
&& curl https://raw.githubusercontent.com/CachyOS/CachyOS-PKGBUILDS/master/cachyos-v3-mirrorlist/cachyos-v3-mirrorlist -o /etc/pacman.d/cachyos-v3-mirrorlist \
&& pacman --noconfirm --sync --needed --refresh archlinux-keyring cachyos-keyring cachyos-mirrorlist cachyos-v3-mirrorlist cachyos-hooks

# Perform a clean system installation with latest CachyOS packages in chroot to correctly execute hooks, this uses host's Pacman
RUN pacman --noconfirm --sync --needed arch-install-scripts \
&& pacstrap -K -P /mnt base \
&& cp -av /etc/pacman.d/ /mnt/etc/

# |
# | BASE
# |

# Reusable base template
FROM scratch AS base
COPY --from=rootfs /mnt /

# Clock
ARG SYSTEM_OPT_TIMEZONE
RUN ln --symbolic --force /usr/share/zoneinfo/${SYSTEM_OPT_TIMEZONE} /etc/localtime

# Keymap hook
ARG SYSTEM_OPT_KEYMAP
RUN echo "KEYMAP=${SYSTEM_OPT_KEYMAP}" > /etc/vconsole.conf

# Language
RUN echo 'LANG=en_US.UTF-8' > /etc/locale.conf \
&& echo 'en_US.UTF-8 UTF-8' > /etc/locale.gen \
&& locale-gen
130 changes: 130 additions & 0 deletions cachyos/etc/pacman.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
#
# /etc/pacman.conf
#
# See the pacman.conf(5) manpage for option and repository directives

#
# GENERAL OPTIONS
#
[options]
# The following paths are commented out with their default values listed.
# If you wish to use different paths, uncomment and update the paths.
#RootDir = /
#DBPath = /var/lib/pacman/
#CacheDir = /var/cache/pacman/pkg/
#LogFile = /var/log/pacman.log
#GPGDir = /etc/pacman.d/gnupg/
#HookDir = /etc/pacman.d/hooks/
HoldPkg = pacman glibc
#XferCommand = /usr/bin/curl -L -C - -f -o %o %u
#XferCommand = /usr/bin/wget --passive-ftp -c -O %o %u
#CleanMethod = KeepInstalled
#UseDelta = 0.7
Architecture = x86_64 x86_64_v3

# Pacman won't upgrade packages listed in IgnorePkg and members of IgnoreGroup
#IgnorePkg =
#IgnoreGroup =

#NoUpgrade =
#NoExtract =

# Misc options
#UseSyslog
Color
ILoveCandy
#NoProgressBar
#CheckSpace
VerbosePkgLists
DisableDownloadTimeout
ParallelDownloads = 10

# By default, pacman accepts packages signed by keys that its local keyring
# trusts (see pacman-key and its man page), as well as unsigned packages.
SigLevel = Required DatabaseOptional
LocalFileSigLevel = Optional
#RemoteFileSigLevel = Required

# NOTE: You must run `pacman-key --init` before first using pacman; the local
# keyring can then be populated with the keys of all official Arch Linux
# packagers with `pacman-key --populate archlinux`.

#
# REPOSITORIES
# - can be defined here or included from another file
# - pacman will search repositories in the order defined here
# - local/custom mirrors can be added here or in separate files
# - repositories listed first will take precedence when packages
# have identical names, regardless of version number
# - URLs will have $repo replaced by the name of the current repo
# - URLs will have $arch replaced by the name of the architecture
#
# Repository entries are of the format:
# [repo-name]
# Server = ServerName
# Include = IncludePath
#
# The header [repo-name] is crucial - it must be present and
# uncommented to enable the repo.
#

# The testing repositories are disabled by default. To enable, uncomment the
# repo name header and Include lines. You can add preferred servers immediately
# after the header, and they will be used before the default mirrors.

# Main (CachyOS)

#[cachyos-testing-v3]
#Include = /etc/pacman.d/mirrorlist
[cachyos-v3]
Include = /etc/pacman.d/cachyos-v3-mirrorlist
[cachyos-core-v3]
Include = /etc/pacman.d/cachyos-v3-mirrorlist
[cachyos-extra-v3]
Include = /etc/pacman.d/cachyos-v3-mirrorlist
[cachyos-community-v3]
Include = /etc/pacman.d/cachyos-v3-mirrorlist
[cachyos]
Include = /etc/pacman.d/cachyos-mirrorlist

# Fallback (Arch Linux)

#[testing]
#Include = /etc/pacman.d/mirrorlist
[core]
Include = /etc/pacman.d/mirrorlist
[extra]
Include = /etc/pacman.d/mirrorlist
#[community-testing]
#Include = /etc/pacman.d/mirrorlist
[community]
Include = /etc/pacman.d/mirrorlist

# If you want to run 32 bit applications on your x86_64 system,
# enable the multilib repositories as required here.

# Wine
#[multilib-testing]
#Include = /etc/pacman.d/mirrorlist
[multilib]
Include = /etc/pacman.d/mirrorlist

# An example of a custom package repository. See the pacman manpage for
# tips on creating your own repositories.
#[custom]
#SigLevel = Optional TrustAll
#Server = file:///home/custompkgs

[options]
#NoExtract = usr/share/help/* !usr/share/help/en* !usr/share/help/C/*
#NoExtract = usr/share/gtk-doc/html/* usr/share/doc/*
#NoExtract = usr/share/locale/* usr/share/X11/locale/* usr/share/i18n/*
#NoExtract = !*locale*/en*/* !usr/share/i18n/charmaps/UTF-8.gz !usr/share/*locale*/locale.*
#NoExtract = !usr/share/*locales/en_?? !usr/share/*locales/i18n* !usr/share/*locales/iso*
#NoExtract = !usr/share/*locales/trans*
#NoExtract = !usr/share/X11/locale/C/*
#NoExtract = !usr/share/X11/locale/compose.dir !usr/share/X11/locale/iso8859-1/*
#NoExtract = !usr/share/*locales/C !usr/share/*locales/POSIX !usr/share/i18n/charmaps/ANSI_X3.4-1968.gz
NoExtract = usr/share/man/* usr/share/info/*
#NoExtract = usr/share/vim/vim*/lang/*
#NoExtract = etc/pacman.conf etc/pacman.d/mirrorlist
Loading