Add initial systemd lifecycle notification support

[jcharaoui]

This adds rudimentary sd_notify() support, and allows service managers to receive a signal when the service is ready.

It could possibly replace the "restart-file" mechanism which seems to serve a similar function, but is somewhat awkward to support.

Issues which might warrant additional work before merging:

• Tests
• Implement watchdog support

[bastelfreak]

@jcharaoui awesome, thanks for the patch! is that AGPL-3 licensed (there's a chance we switch from apache-2 to AGPL-3)? Are you able to take a look at the linter warnings?

[jcharaoui]

The code was authored by @rlbdv, so the licence is at his discretion.

[rlbdv]

AGPL-3 would be just fine.

[ekohl]

I have basically no knowledge about clojure, but I would love to see this implemented.

[ekohl]

The linter shows this:

== Linting puppetlabs.trapperkeeper.internal ==
src/puppetlabs/trapperkeeper/internal.clj:43:17: reflection: call to method getBytes can't be resolved.
src/puppetlabs/trapperkeeper/internal.clj:44:18: reflection: call to java.io.File ctor can't be resolved.
src/puppetlabs/trapperkeeper/internal.clj:158:39: reflection: reference to field error can't be resolved.

Any idea how to solve those? is org.newsclub.net.unix somehow not pulled in project.cli?

[rlbdv]

If trapperkeeper is enforcing "no reflection warnings" globally, then I'd assume those cases just need type hints, e.g. perhaps (.getBytes ^String message "UTF-8"), etc.

[bastelfreak]

Good evening! I know it's been a long time... I rebased this code and fixed the tests in #23. The unit currently looks like this:

# systemctl cat puppetserver
# /usr/lib/systemd/system/puppetserver.service
#
# Local settings can be configured without being overwritten by package upgrades, for example
# if you want to increase puppetserver open-files-limit to 10000,
# you need to increase systemd's LimitNOFILE setting, so create a file named
# "/etc/systemd/system/puppetserver.service.d/limits.conf" containing:
#       [Service]
#       LimitNOFILE=10000
# You can confirm it worked by running systemctl daemon-reload
# then running systemctl show puppetserver | grep LimitNOFILE
#
[Unit]
Description=puppetserver Service
After=syslog.target network.target nss-lookup.target

[Service]
Type=notify
LogsDirectory=puppetlabs/puppetserver
RuntimeDirectory=puppetlabs/puppetserver
EnvironmentFile=/etc/default/puppetserver
User=puppet
TimeoutStartSec=300
TimeoutStopSec=60
Restart=on-failure
StartLimitBurst=5
PrivateTmp=true

# https://tickets.puppetlabs.com/browse/EZ-129
# Prior to systemd v228, TasksMax was unset by default, and unlimited. Starting in 228 a default of '512'
# was implemented. This is low enough to cause problems for certain applications. In systemd 231, the
# default was changed to be 15% of the default kernel limit. This explicitly sets TasksMax to 4915,
# which should match the default in systemd 231 and later.
# See https://github.com/systemd/systemd/issues/3211#issuecomment-233676333
TasksMax=4915

#set default privileges to -rw-r-----
UMask=027


ExecStart=/usr/bin/java $JAVA_ARGS -Dlogappender=F1 \
  '-XX:OnOutOfMemoryError=kill -9 %p' -XX:+CrashOnOutOfMemoryError \
  -XX:ErrorFile="${LOGS_DIRECTORY}/puppetserver_err_pid%p.log" \
  -cp "${INSTALL_DIR}/puppet-server-release.jar" \
  clojure.main \
  -m puppetlabs.trapperkeeper.main \
  --config "${CONFIG}" \
  --bootstrap-config "${BOOTSTRAP_CONFIG}" \
  $TK_ARGS

KillMode=process

ExecReload=kill -HUP $MAINPID
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

The server starts, but the SD_NOTIFY isn't received by systemd I guess. systemctl start puppetserver eventuall fails with:

Job for puppetserver.service failed because a timeout was exceeded.
See "systemctl status puppetserver.service" and "journalctl -xeu puppetserver.service" for details.

I will try to publish the packages later and add some debugging to systemd.

[bastelfreak]

If someone wants to build this on Ubuntu 24.04:

prepare the system:

apt update
apt upgrade -y
apt install -y git podman ruby ruby-dev ruby-bundler
git clone https://github.com/OpenVoxProject/openvox-server.git
cd openvox-server
bundle config set --local path .vendor
bundle install

actual build:

DEP_REBUILD=trapperkeeper DEP_REBUILD_ORG=bastelfreak DEP_REBUILD_BRANCH=sd_notify EZBAKE_VERSION=2.8.0-SNAPSHOT EZBAKE_REPO='https://github.com/bastelfreak/ezbake' EZBAKE_BRANCH='notify' DEB_PLATFORMS='ubuntu-24.04,ubuntu-22.04' RPM_PLATFORMS='' bundle exec rake vox:build

install:

wget https://apt.voxpupuli.org/openvox8-release-ubuntu24.04.deb
apt update
apt install ./output/deb/ubuntu24.04/openvox8/openvox-server_*ubuntu24.04_all.deb

[bastelfreak]

good evening. after some hacking I was able to get this working in #23. thank you all for the patch!