As always, you have the option to compile directly on an Enzian or to cross-compile.

Compiling on Enzian

The main challenge of compiling on Enzian is that we are space constrained due to the 5 GB size limit imposed by the image provided over ISCSI. Further, I recommend you do this on a named volume, so you keep your setup and the installed modules of your freshly built kernel.

Prerequisites

First, you need to install the missing dependencies to build a Linux kernel. For a normal gcc build you will need to run

$ sudo apt update
$ sudo apt install flex bison bc dwarves libelf-dev libssl-dev

For a build using LLVM you need to additionally install a more recent version of clang, its dev libraries and lld.

$ sudo apt install clang-12 libclang-12-dev llvm-12-tools lld-12

Then, to save some disk space run

$ sudo apt clean

If you want to compile a kernel with Rust, additionally set up a Rust toolchain and follow the steps in the kernel Rust Quick Start page.

Finally, you will want to clone the kernel source to your scratch directory.

Configuration

First copy the config of the running system to your kernel source directory:

$ cp /boot/config-$(uname -r) /scratch/<nethz>/kernel-src/.config

Now, whenever you run make in the kernel source you will need to pass LLVM=-12 (or the version that you installed) before the target. Don't forget to also pass this variable during configuration.

To obtain the smallest possible kernel you should run make localmodconfig which updates your configuration to disable all modules that are currently not loaded. Then, proceed to configure your kernel as needed. A good idea is to set a descriptive LOCALVERSION to be able to easily identify which files belong to your custom build.

If you want to configure Rust support, you will most likely need to disable module versioning and structure randomization (search for "RUST" and have a look at the dependencies). If the configuration dependencies are not met, the option to add Rust support will not show up.

You will at some point get an error/warning/notice that the keychain for the trusted system keys for signature checking is not found. The easiest solution is to set CONFIG_SYSTEM_TRUSTED_KEYS="" and CONFIG_SYSTEM_REVOCATION_KEYS="". You can also copy the debian keychains to the system and specify a path there (see Stackoverflow).

Building and Installing

Build the kernel using make -j 47 all (make sure to specify variables needed). If the build succeeds, you will find your compressed image in arch/arm64/boot/Image.gz.

Next, install the modules with make modules_install. If you minimized your kernel sufficiently, this should succeed and you will find the files in /lib/modules/<your kernel version>/. Should you run out of space you can mount a ramdisk to the path so you can still generate an initramfs if needed (see the troubleshooting section).

If your kernel version is much newer than the running kernel or you added new modules that require this, you will need to generate a new initramfs. First, ensure that the module iscsi is present in /etc/initrmafs-tools/modules. Then generate the new initramfs

$ mkinitramfs -o /scratch/<your-nethz>/kernel-src/arch/arm64/boot/initrd.img <your kernel version>

where <your kernel version> matches the directory in /lib/modules/<your kernel version>/.

Booting the compiled kernel

First, you need to ask an admin to create a directory for you in /srv/tftp/userkernels on enzian-gateway. You can then scp the compressed image and (if nessecary) the generated initramfs from the Enzian where you built the kernel into your userkernels directory.

To boot your brand-spanking new kernel image you can supply the emg acquire with -k <your nethz>/<your kernel image> and -i <your nethz>/<your initramfs image>. This will provide your kernel image once you power on Enzian.

Troubleshooting

What Makefile targets do even exist?

The Linux kernel Makefile has the very helpful target make help, which explains all available targets.

How full is the ISCSI provided disk on Enzian?

You can find out with

$ lsblk -o NAME,PATH,FSSIZE,FSAVAIL,FSUSE%,MOUNTPOINT

Why is the ISCSI provided disk on Enzian already full?

You can have a look at the space hogging directories/files using

$ du -BM | sort -nr | less

If you do not wish to see your scratch space throw a grep -n "/scratch" in front of the sort.

How do I mount a ramdisk for some additional space?

First, create a directory to mount to (e.g. /mnt/ramdisk0) and then mount with sudo mount -t tmpfs -o size=5G ext4 <mount point>.

When I boot my kernel I get an error "invalid magic number".

Then you did not supply the correct kernel image. Check again that you supply the compressed kernel image Image.gz (also often renamed to vmlinuz).