One of the most asked about topics to folks working on upstream container technologies is running Podman within a container. Most of this has historically been related to Docker in Docker (DIND), but now, people also want to run Podman in Podman (PINP) or Podman in Docker (PIND).

But Podman can be run in multiple ways, rootful and rootless. We end up with people wanting to run various combinations of rootful and rootless Podman:

• Rootful Podman in rootful Podman
• Rootless Podman in rootful Podman
• Rootful Podman in rootless Podman
• Rootless Podman in rootless Podman

You get the picture.

This blog will attempt to cover each combination, starting with a discussion of privileges. We’ll start with the PINP scenario here in part one. In part two of the series, we’ll cover similar ground but do so within the context of Kubernetes. Be sure to read both articles for a complete picture.

Container engines require privileges

In order to run a container engine like Podman within a container, the first thing you need to understand is that you need a fair amount of privilege.

• Containers require multiple UIDs. Most container images need more than one UID to work. For example, you might have an image with most of the files owned by root, but some owned by the apache user (UID=60).
• Container engines mount file systems and use the system call clone to create user namespaces.

Note: You might need a newer version of Podman.  Examples in this blog were run with Podman 3.2.

Our test image

For the examples in this blog, we’ll use the quay.io/podman/stable image, which was built with the idea of finding the best way to run Podman within a container. You can examine how we build this image from the Dockerfile and containers.conf image in the github.com repo.

# stable/Dockerfile
#
# Build a Podman container image from the latest
# stable version of Podman on the Fedoras Updates System.
# https://bodhi.fedoraproject.org/updates/?search=podman
# This image can be used to create a secured container
# that runs safely with privileges within the container.
#
FROM registry.fedoraproject.org/fedora:latest

# Don't include container-selinux and remove
# directories used by yum that are just taking
# up space.
RUN dnf -y update; yum -y reinstall shadow-utils; 
yum -y install podman fuse-overlayfs --exclude container-selinux; 
rm -rf /var/cache /var/log/dnf* /var/log/yum.*

RUN useradd podman; 
echo podman:10000:5000 > /etc/subuid; 
echo podman:10000:5000 > /etc/subgid;

VOLUME /var/lib/containers
VOLUME /home/podman/.local/share/containers

ADD https://raw.githubusercontent.com/containers/libpod/master/contrib/podmanimage/stable/containers.conf /etc/containers/containers.conf
ADD https://raw.githubusercontent.com/containers/libpod/master/contrib/podmanimage/stable/podman-containers.conf /home/podman/.config/containers/containers.conf

RUN chown podman:podman -R /home/podman

# chmod containers.conf and adjust storage.conf to enable Fuse storage.
RUN chmod 644 /etc/containers/containers.conf; sed -i -e 's|^#mount_program|mount_program|g' -e '/additionalimage.*/a "/var/lib/shared",' -e 's|^mountopt[[:space:]]*=.*$|mountopt = "nodev,fsync=0"|g' /etc/containers/storage.conf
RUN mkdir -p /var/lib/shared/overlay-images /var/lib/shared/overlay-layers /var/lib/shared/vfs-images /var/lib/shared/vfs-layers; touch /var/lib/shared/overlay-images/images.lock; touch /var/lib/shared/overlay-layers/layers.lock; touch /var/lib/shared/vfs-images/images.lock; touch /var/lib/shared/vfs-layers/layers.lock

ENV _CONTAINERS_USERNS_CONFIGURED=""

Let’s examine the Dockerfile.

FROM registry.fedoraproject.org/fedora:latest

# Don't include container-selinux and remove
# directories used by yum that are just taking
# up space.
RUN dnf -y update; yum -y reinstall shadow-utils; 
yum -y install podman fuse-overlayfs --exclude container-selinux; 
rm -rf /var/cache /var/log/dnf* /var/log/yum.*

First pull fedora latest, and then update to the latest packages.  Note it reinstalls shadow-utils, since there is a known issue in the shadow-utils install on the Fedora image where the filecaps on newsubuid and newsubgid are not set.  Reinstalling shadow-utils fixes the problem.  Next, install Podman as well as the