LWN.net needs you! Without subscribers, LWN would simply not exist. Please consider signing up for a subscription and helping to keep LWN publishing

The kernel project does not host much user-space code in its repository,
but there are exceptions. One of those, currently found in the tools/include/nolibc
directory, has only been present since the 5.1 release. The nolibc project
aims to provide minimal C-library emulation for small, low-level workloads.
Read on for an overview of nolibc, its history, and future direction
written by its principal contributor.

The nolibc component actually made a discreet entry into the 5.0 kernel
as part of the RCU torture-test suite (“rcutorture”), via commit 66b6f755ad45
(“rcutorture:
Import a copy of nolibc”). This happened after Paul McKenney asked:
“Does anyone do kernel-only deployments, for example, setting up an
embedded device having a Linux kernel and absolutely no userspace
whatsoever?”

He went on:

The mkinitramfs approach results in about 40MB of initrd, and
dracut about 10MB. Most of this is completely useless for
rcutorture, which isn’t interested in mounting filesystems, opening
devices, and almost all of the other interesting things that
mkinitramfs and dracut enable.

Those who know me will not be at all surprised to learn that I went
overboard making the resulting initrd as small as possible. I
started by throwing out everything not absolutely needed by the
dash and sleep binaries, which got me down to about 2.5MB, 1.8MB of
which was libc.

This description felt familiar to me, since I have been solving
similar problems for a long time. The end result (so far) is nolibc — a
minimal C library for times when a system is booted only to run a single
tiny program.

A bit of history: when size matters

For 25 years, I have been building single-floppy-based emergency
routers, firewalls, and traffic generators containing both a kernel and a
root filesystem. I later moved on to credit-card-sized live CDs and,
nowadays, embedding tiny
interactive shells in all of my kernels to help better recover from boot
issues.

All of these had in common a small program called preinit
that is in charge of creating /dev entries, mounting
/proc, optionally mounting a RAM-based filesystem, and loading
some extra modules before executing init. The
characteristic that all my kernels have in common is that all of their
modules are
packaged within the kernel’s builtin initial ramfs, reserving the initial
RAMdisk (initrd) for the root filesystem. When the kernel boots, it pivots
the root and boot mount points to make the pre-packaged modules appear at
their final location, making the root filesystem independent of the kernel
version used. This is extremely convenient when working from flash or
network boot, since you can easily swap kernels without ever touching the
filesystem.

Because it had to fit into 800KB (or smaller) kernels, the preinit code
initially had to be minimal; one of the approaches consisted of strictly
avoiding stdio or high-level C-library functions and reusing code and data
as much as possible (such as merging string tails). This resulted in code
that could be statically built and remain small
(less than 1KB for the original floppy version) thanks to its small
dependencies.

As it evolved, though, the resulting binary size was often dominated by
the C-library initialization code. Switching to diet libc helped but, in 2010, it
was not evolving anymore and still had some limitations. I replaced it
with the slightly larger — but more active — uClibc, with a bunch of functions
replaced by local alternatives to keep the size low (e.g. 640 bytes
were saved on memmove() and strpcy()), and used this
until 2016.

uClibc, in turn, started to show some more annoying limitations which
became a concern when trying to port code to architectures that it did not
support by then (such as aarch64), and its maintenance was falling off. I
started to consider using klibc, which
also had the advantage of being developed and maintained by kernel
developers but, while klibc was much more modern, portable, and closer to
my needs, it still had the same inconvenience of requiring to be built
separately before being linked into the final executable.

It then started to appear obvious that, if the preinit code had so
little dependency on the C library, it could make sense to just define the
system calls directly and be done with it. Thus, in January 2017, some
system-call definitions were moved to macros. With this new ability to
build without relying on any C library at all, a natural-sounding name
quickly emerged for this project: “nolibc”.

The first focus was on being able to build existing code as seamlessly
as possible, with either nolibc or a regular C library, because ifdefs in
the code are painful to deal with. This includes the ability to bypass the
“include” block when nolibc was already included on the command line, and
not to depend on any external files that would require build-process
updates. Because of this, it was decided that only macros and static
definitions would be used, which imposes some limitations that we’ll
cover below.

One nice advantage for quick tests is that it became possible to include
nolibc.h directly from the compiler’s command line, and to rely on
the NOLIBC definition coming from this file to avoid including the
normal C-library headers with a construct like:

    #ifndef NOLIBC
    #include 
    #include 
    /* ... */
    #endif

This allows a program to be built with a regular C library on architectures
lacking nolibc support while making it easy to switch to nolibc when it is
available. This is how rcutorture currently uses it.

Another point to note is that, while the resulting binary is always
statically linked with nolibc (hence it is self-contained and doesn’t
require any
shared library to be installed), it is still usually smaller than with
glibc, with either static or dynamic linking:

    $ size init-glibc-static init-glibc init-nolibc
    text    data     bss     dec     hex  filename
    7075