Firmware – what are we going to do about it?

TL;DR: firmware support in Debian sucks, and we need to change
this. See the “My preference, and rationale” Section below.

In my opinion, the way we deal with (non-free) firmware in Debian is a
mess, and this is hurting many of our users daily. For a long time
we’ve been pretending that supporting and including (non-free)
firmware on Debian systems is not necessary. We don’t want to have
to provide (non-free) firmware to our users, and in an ideal world we
wouldn’t need to. However, it’s very clearly no longer a sensible path
when trying to support lots of common current hardware.

Background – why has (non-free) firmware become an issue?

Firmware is the low-level software that’s designed to make
hardware devices work. Firmware is tightly coupled to the hardware,
exposing its features, providing higher-level functionality and
interfaces for other software to use. For a variety of reasons, it’s
typically not Free Software.

For Debian’s purposes, we typically separate firmware from
software by considering where the code executes (does it run on a
separate processor? Is it visible to the host OS?) but it can be
difficult to define a single reliable dividing line here. Consider the
Intel/AMD CPU microcode packages, or the U-Boot firmware packages as
examples.

In times past, all necessary firmware would normally be included
directly in devices / expansion cards by their vendors. Over time,
however, it has become more and more attractive (and therefore more
common) for device manufacturers to not include complete firmware
on all devices. Instead, some devices just embed a very simple set
of firmware that allows for upload of a more complete firmware “blob”
into memory. Device drivers are then expected to provide that blob
during device initialisation.

There are a couple of key drivers for this change:

• Cost: it’s typically cheaper to fit smaller flash memory (or no
  flash at all) onto a device. The cost difference may seem small in
  many cases, but reducing the bill of materials (BOM) even by a few
  cents can make a substantial difference to the economics of a
  product. For most vendors, they will have to implement device
  drivers anyway and it’s not difficult to include firmware in that
  driver.

• Flexibility: it’s much easier to change the behaviour of a device by
  simply changing to a different blob. This can potentially cover lots
  of different use cases:

  • separating deadlines for hardware and software in manufacturing
    (drivers and firmware can be written and shipped later);
  • bug fixes and security updates (e.g. CPU microcode changes);
  • changing configuration of a device for different users or products
    (e.g. potentially different firmware to enable different
    frequencies on a radio product);
  • changing fundamental device operation (e.g. switching between RAID
    and JBOD functionality on a disk controller).

Due to these reasons, more and more devices in a typical computer now
need firmware to be uploaded at runtime for them to function
correctly. This has grown:

• Going back 10 years or so, most computers only needed firmware
  uploads to make WiFi hardware work.

• A growing number of wired network adapters now demand firmware
  uploads. Some will work in a limited way but depend on extra
  firmware to