Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

device stopped after timeout #2

Closed
julianbrost opened this issue Jan 21, 2017 · 6 comments
Closed

device stopped after timeout #2

julianbrost opened this issue Jan 21, 2017 · 6 comments

Comments

@julianbrost
Copy link

julianbrost commented Jan 21, 2017 •
edited
Loading

I'm using systemd-tool to remotely unlock my disk crypto via SSH and it works fine so far. But after a timeout of 90 seconds with no password entered, dropbear and the whole network is stopped so I can't enter the password remotely if I miss this time frame.

My setup in detail: Two disks are in a MDADM RAID1, on top of which is a LUKS encrypted volume (to be unlocked at /dev/mapper/crypt_system) which contains the only volume for the LVM VG 'system' providing a LV 'root' with the actual rootfs. So in short: LVM in LUKS in MDADM.

Jan 21 00:47:35 localhost systemd[1]: systemd 232 running in system mode. (+PAM -AUDIT -SELINUX -IMA -APPARMOR +SMACK -SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BL
Jan 21 00:47:35 localhost systemd[1]: Detected architecture x86-64.
Jan 21 00:47:35 localhost systemd[1]: Running in initial RAM disk.
[...]
Jan 21 00:47:35 localhost systemd[1]: Started Initrd Dropbear Service.
Jan 21 00:47:35 localhost dropbear[139]: Not backgrounding
Jan 21 00:47:35 localhost systemd[1]: Started udev Kernel Device Manager.
[...]
Jan 21 00:47:35 localhost systemd[1]: Starting Network Service...
Jan 21 00:47:35 localhost systemd-networkd[187]: Enumeration completed
Jan 21 00:47:35 localhost systemd-networkd[187]: eth1: IPv6 enabled for interface: Success
Jan 21 00:47:35 localhost systemd[1]: Started Network Service.
Jan 21 00:47:36 localhost systemd-networkd[187]: eth0: IPv6 enabled for interface: Success
[...]
Jan 21 00:47:37 localhost systemd[1]: Found device /dev/disk/by-uuid/[...].
Jan 21 00:47:37 localhost systemd[1]: Starting Cryptography Setup for crypt_system...
Jan 21 00:47:37 localhost systemd[1]: Started Initrd Cryptsetup Service.
Jan 21 00:47:37 localhost systemd[1]: Started Dispatch Password Requests to Console.
Jan 21 00:47:37 localhost kernel: device-mapper: uevent: version 1.0.3
Jan 21 00:47:37 localhost kernel: device-mapper: ioctl: 4.35.0-ioctl (2016-06-23) initialised: dm-devel@redhat.com
Jan 21 00:47:37 localhost shell[223]: service/loc info : init
Jan 21 00:47:37 localhost shell[226]: service/loc info : cryptsetup service
Jan 21 00:47:37 localhost shell[230]: service/loc info : crypt jobs
Jan 21 00:47:37 localhost shell[233]: service/loc info : custom agent try #1
Jan 21 00:47:37 localhost kernel: tg3 0000:01:00.1 eth1: Link is up at 100 Mbps, full duplex
Jan 21 00:47:37 localhost kernel: tg3 0000:01:00.1 eth1: Flow control is on for TX and on for RX
Jan 21 00:47:37 localhost kernel: tg3 0000:01:00.1 eth1: EEE is disabled
Jan 21 00:47:37 localhost kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
Jan 21 00:47:37 localhost systemd-networkd[187]: eth1: Gained carrier
Jan 21 00:47:37 localhost shell[242]: service/loc info : query start
Jan 21 00:47:38 localhost systemd-networkd[187]: eth1: Gained IPv6LL
Jan 21 00:47:39 localhost kernel: tg3 0000:01:00.0 eth0: Link is up at 1000 Mbps, full duplex
Jan 21 00:47:39 localhost kernel: tg3 0000:01:00.0 eth0: Flow control is off for TX and off for RX
Jan 21 00:47:39 localhost kernel: tg3 0000:01:00.0 eth0: EEE is enabled
Jan 21 00:47:39 localhost kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Jan 21 00:47:39 localhost systemd-networkd[187]: eth0: Gained carrier
Jan 21 00:47:40 localhost systemd-networkd[187]: eth1: DHCPv4 address [...]/26
Jan 21 00:47:41 localhost systemd-networkd[187]: eth0: Gained IPv6LL
Jan 21 00:47:42 localhost systemd-networkd[187]: eth0: DHCPv4 address [...]/24 via [...]
Jan 21 00:47:42 localhost systemd-networkd[187]: eth0: Configured
Jan 21 00:47:51 localhost systemd-networkd[187]: eth1: Configured
Jan 21 00:49:05 localhost systemd[1]: dev-system-root.device: Job dev-system-root.device/start timed out.
Jan 21 00:49:05 localhost systemd[1]: Timed out waiting for device dev-system-root.device.
Jan 21 00:49:05 localhost systemd[1]: Dependency failed for Initrd Root Device.
Jan 21 00:49:05 localhost systemd[1]: initrd-root-device.target: Job initrd-root-device.target/start failed with result 'dependency'.
Jan 21 00:49:05 localhost systemd[1]: initrd-root-device.target: Triggering OnFailure= dependencies.
Jan 21 00:49:05 localhost systemd[1]: Dependency failed for /sysroot.
Jan 21 00:49:05 localhost systemd[1]: Dependency failed for Initrd Root File System.
Jan 21 00:49:05 localhost dropbear[139]: Early exit: Terminated by signal
Jan 21 00:49:05 localhost systemd[1]: Dependency failed for Reload Configuration from the Real Root.
Jan 21 00:49:05 localhost systemd[1]: initrd-parse-etc.service: Job initrd-parse-etc.service/start failed with result 'dependency'.
Jan 21 00:49:05 localhost systemd[1]: initrd-parse-etc.service: Triggering OnFailure= dependencies.
Jan 21 00:49:05 localhost systemd[1]: initrd-root-fs.target: Job initrd-root-fs.target/start failed with result 'dependency'.
Jan 21 00:49:05 localhost systemd[1]: initrd-root-fs.target: Triggering OnFailure= dependencies.
Jan 21 00:49:05 localhost systemd[1]: sysroot.mount: Job sysroot.mount/start failed with result 'dependency'.
Jan 21 00:49:05 localhost systemd[1]: Dependency failed for File System Check on /dev/system/root.
Jan 21 00:49:05 localhost systemd[1]: systemd-fsck-root.service: Job systemd-fsck-root.service/start failed with result 'dependency'.
Jan 21 00:49:05 localhost systemd[1]: dev-system-root.device: Job dev-system-root.device/start failed with result 'timeout'.
Jan 21 00:49:05 localhost systemd[1]: Reached target Initrd File Systems.
Jan 21 00:49:05 localhost systemd[1]: Started Emergency Shell.
Jan 21 00:49:05 localhost systemd[1]: Reached target Emergency Mode.
Jan 21 00:49:05 localhost sh[251]: initrd-network: disable network devices
Jan 21 00:49:05 localhost systemd[1]: Stopping Initrd Dropbear Service...
Jan 21 00:49:05 localhost systemd[1]: Stopped Initrd Dropbear Service.
Jan 21 00:49:05 localhost systemd[1]: Stopping Initrd Network Service...
Jan 21 00:49:05 localhost systemd-networkd[187]: eth0: Lost carrier
Jan 21 00:49:05 localhost systemd-networkd[187]: eth0: DHCP lease lost
Jan 21 00:49:05 localhost systemd[1]: Stopped Initrd Network Service.
Jan 21 00:49:05 localhost systemd-networkd[187]: eth1: Lost carrier
Jan 21 00:49:05 localhost systemd-networkd[187]: eth1: DHCP lease lost
Jan 21 00:49:05 localhost systemd-networkd[187]: lo: Lost carrier
@Andrei-Pozolotin
Copy link
Collaborator

please test with keep alive settings
https://linux.die.net/man/8/dropbear

-K timeout_seconds
Ensure that traffic is transmitted at a certain interval in seconds. This is useful for working around firewalls or routers that drop connections after a certain period of inactivity. The trade-off is that a session may be closed if there is a temporary lapse of network connectivity. A setting if 0 disables keepalives.

@julianbrost
Copy link
Author

This did not change anything. As far as I can tell, -K only affects open connections. The issue I'm describing here also happens without any open connection. I suspect that there is a dependency missing somewhere.

@julianbrost
Copy link
Author

I found a workaround: adding a file /etc/systemd/system/dev-system-root.device.d/depend-on-cryptsetup.conf to the initrd with the following contents:

[Unit]
JobTimeoutSec=0
Requires=cryptsetup.target

But I'm not sure if that's the best solution and it would be nice if it just works by default.

@Andrei-Pozolotin Andrei-Pozolotin changed the title dropbear stopped after timeout device stopped after timeout Sep 19, 2017
@Andrei-Pozolotin
Copy link
Collaborator

an easier solution for root device timeout could be:

  1. remove root=/dev/mapper/root from kernel command line

  2. provide /sysroot/ folder in the /boot/initramfs-linux.img

  3. declare long timeout in /boot/initramfs-linux.img/ucpio://etc/fstab:

# provide here root partition description (instead of kernel command line)
#  <file system>        <dir>      <type>    <option>                         <dump> <pass>
 /dev/mapper/root     /sysroot    auto     x-systemd.device-timeout=9999h     0     1

see details:
http://man7.org/linux/man-pages/man7/bootup.7.html
https://www.freedesktop.org/software/systemd/man/systemd.mount.html
https://www.freedesktop.org/software/systemd/man/systemd-fstab-generator.html

@accensi
Copy link

accensi commented Sep 28, 2017

Two noob questions:

  1. How to provide the /sysroot folder?
  2. Why not x-systemd.device-timeout=0, for waiting indefinetly?

@Andrei-Pozolotin
Copy link
Collaborator

/sysroot: now included in service by default:
https://github.com/random-archer/mkinitcpio-systemd-tool/blob/master/initrd-cryptsetup.service#L37

x-systemd.device-timeout: there is no default, per docs:
https://www.freedesktop.org/software/systemd/man/systemd.mount.html
but please try and report back

@Andrei-Pozolotin Andrei-Pozolotin mentioned this issue Jan 30, 2018
Closed
@vaminakov vaminakov mentioned this issue Aug 17, 2018
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@julianbrost @Andrei-Pozolotin @accensi