Dependency based boot sequence

Reordering the boot sequence for correctness and speed

There are several boot options available in Debian, and the default SysV-derived one can be tuned and corrected. The talk will present the available options, and go into details on some of the pitfalls and advantages of the default boot system. It will present some results based on the Google summer of code project working on the boot system.

Petter Reinholdtsen
pere@hungry.com
Debconf 2007 Edinburgh, 2007-06-17

About the speaker

Using Linux since 1993
Debian Developer since 2002
Educated at University of Tromsø, Western Australia and Oslo
Employed at University of Oslo to develop system administration tools
Leader of the Norwegian Unix User Group
Co-maintaining sysvinit, maintaining insserv
and a few other (Debian Edu, Debian GIS, Debian Java, Debian Boot, etc)

Summary of this talk

Quick overview of the boot system alternatives in Debian
Some problems with the current system
How to improve the boot sequence ordering
how to prepare packages for this

Based on work done by Carlos Villegas during Google Summer of Code 2006, available from http://initscripts-ng.alioth.debian.org/soc2006-bootsystem/.

Debian include several boot system options

SysV init is only the most used boot system in Debian, not the only one.

file-rc
upstart
init-ng
minit
runit
metainit initiative for init.d scripts

The SysV boot system

Start first all scripts in /etc/rcS.d/
For the runlevel 2, first all K##scripts in /etc/rc2.d/, next all S##script.
All .sh scripts are sourced, the rest are exec-ed

Current sid boot sequence (base install)

/etc/rcS.d:
README                  S30checkfs.sh
S01glibc.sh             S30procps.sh
S02hostname.sh          S35mountall.sh
S02mountkernfs.sh       S36mountall-bootclean.sh
S04mountdevsubfs.sh     S39ifupdown
S05bootlogd             S40networking
S10checkroot.sh         S45mountnfs.sh
S11hwclock.sh           S46mountnfs-bootclean.sh
S12mtab.sh              S55bootmisc.sh
S18ifupdown-clean       S55urandom
S20module-init-tools    S99stop-bootlogd-single
S25libdevmapper1.02.1

/etc/rc2.d:
README                  S89cron
S10sysklogd             S99rc.local
S11klogd                S99rmnologin
S20makedev              S99stop-bootlogd
S20openbsd-inetd

Current sid shutdown sequence (base install)

/etc/rc6.d:
K11cron                 S30urandom
K20makedev              S31umountnfs.sh
K20openbsd-inetd        S35networking
K25hwclock.sh           S36ifupdown
K89klogd                S40umountfs
K90sysklogd             S60umountroot
README                  S90reboot
S20sendsigs

But is it correct?

Problems with the current boot system

Slightly incorrect ordering during startup and shutdown
Hard to pick correct sequence number
The halt/reboot runlevels are inconsistent/confusing (or wrong, depending on your view)
Impossible to parallelise

How can we verify the boot sequence correctness?

Document the dependencies of each script
Verify the scripts are started in dependency order

Example header:

#!/bin/sh -e
### BEGIN INIT INFO
# Provides:          networking
# Required-Start:    mountkernfs ifupdown $local_fs
# Required-Stop:     ifupdown $local_fs
# X-Start-Before:    nis
# X-Stop-After:      nis
# Default-Start:     S
# Default-Stop:      0 6
# Short-Description: Raise network interfaces.
### END INIT INFO

Verifying the current sequence

Using these headers and a script from the insserv package, we can verify the boot order:

# /usr/share/insserv/check-initd-order   -o
LSB header missing in /etc/rcS.d/S11hwclock.sh
LSB header missing in /etc/rcS.d/S25libdevmapper1.02.1
Incorrect order modutils@? > checkfs.sh@30
Incorrect order $all@? > stop-bootlogd-single@99
Incorrect order $time@? > sysklogd@10
Incorrect order $time@? > cron@89
Incorrect order $all@? > rmnologin@99
#

For the packages without dependency information, override files can be read from /usr/share/insserv/overrides/ and /etc/insserv/overrides/.

Verifying the current sequence II

Current status (base task install), with override files.

# /usr/share/insserv/check-initd-order
Incorrect order $local_fs@? > hwclock.sh@11
Incorrect order $remote_fs@? > hwclock.sh@11
Incorrect order modutils@? > libdevmapper1.02.1@25
Incorrect order modutils@? > checkfs.sh@30
Incorrect order $all@? > stop-bootlogd-single@99
Incorrect order $all@? > rmnologin@99
#

Verifying the current sequence III

Current status (desktop task install)

# /usr/share/insserv/check-initd-order -o
LSB header missing in /etc/rcS.d/S11hwclock.sh
LSB header missing in /etc/rcS.d/S25libdevmapper1.02.1
Incorrect order modutils@? > checkfs.sh@30
Incorrect order $all@? > stop-bootlogd-single@99
Incorrect order $time@? > sysklogd@10
LSB header missing in /etc/rc2.d/S19hplip
LSB header missing in /etc/rc2.d/S20cupsys
LSB header missing in /etc/rc2.d/S20hotkey-setup
LSB header missing in /etc/rc2.d/S89anacron
Incorrect order $time@? > cron@89
Incorrect order $all@? > rmnologin@99
#

Reordering based on dependency information

Advantages:

No need to replace existing init.d scripts
Activated by adding headers to the scripts or use override files
Experimental code to enable it is in the insserv package

How to enable this by default

Install the insserv package
Make all start-symlinks in rc0 and rc6 to stop symlinks
Replace update-rc.d
Reorder rc*.d/*

Status of dependency based sysv boot sequence

Majority of common packages provides dependency information
Correct startup sequence for base and desktop tasks when enabled.
Fairly correct shutdown sequence.
Some packages fail to install because of incorrect init.d script registration order

To enable: BAD_INSSERV_HACKER=true dpkg-reconfigure insserv

Testrun:

insserv:/var/tmp/chroot-sid-base-reorder# BAD_INSSERV_HACKER=true dpkg-reconfigure i
info: Backing up existing boot scripts in /var/lib/insserv/bootscripts-20070615T1617.tar.gz
info: Reordering boot system, log to /var/lib/insserv/run-20070615T1617.log
info: Recording new boot sequence in /var/lib/insserv/bootscripts-20070615T1617-after.list
info: Use '/usr/sbin/update-bootsystem-insserv restore' to restore the old boot sequence.
success: Boot system successfully converted
Adding `diversion of /usr/sbin/update-rc.d to /usr/sbin/update-rc.d.distrib by insserv'
#

Similar boot graph

Reordered sid boot sequence (base install)

/etc/rcS.d:
README                  S05mtab.sh
S01glibc.sh             S06mountall.sh
S01libdevmapper1.02.1   S07ifupdown
S02hostname.sh          S07mountall-bootclean.sh
S02mountkernfs.sh       S07urandom
S03mountdevsubfs.sh     S08networking
S03procps.sh            S09mountnfs.sh
S04bootlogd             S10mountnfs-bootclean.sh
S04checkroot.sh         S11bootmisc.sh
S05checkfs.sh           S11hwclock.sh
S05ifupdown-clean       S11stop-bootlogd-single
S05module-init-tools
/etc/rc2.d:
README                  S11openbsd-inetd
S07makedev              S11rc.local
S07sysklogd             S12stop-bootlogd
S08cron                 S13rmnologin
S08klogd

Reordered sid shutdown sequence (base install)

/etc/rc6.d:
K03openbsd-inetd        K31umountnfs.sh
K06cron                 K35networking
K06klogd                K36ifupdown
K07sysklogd             K40umountfs
K20makedev              K60umountroot
K20sendsigs             K90reboot
K25hwclock.sh           README

I'm not brave enough, get me out of here

It is possible to automatically undo the reordering, once, by reconfiguring insserv and disabling the dependency based boot.

# BAD_INSSERV_HACKER=true dpkg-reconfigure i
info: Disabling dependency based boot system
Removing `diversion of /usr/sbin/update-rc.d to /usr/sbin/update-rc.d.distrib by insserv'
info: Restoring using backed up copy of init.d/ and rc*.d/.
info: successfully restored backup of init.d scripts
#

How can package maintainers assist

Update their init.d scripts to include correct dependency information using the LSB headers defined for this purpose, http://wiki.debian.org/LSBInitScripts
Make sure their init.d scripts are registered in dependency order
Start using insserv to detect bugs in the packages

Even running boot scripts in parallel is possible

echo CONCURRENCY=shell >> /etc/default/rcS

Thank you very much

Questions?

http://www.hungry.com/~pere/mypapers/200706-bootseq/