RHEL in Practice #3: Container Workloads — Podman, systemd (quadlet)

Infrastructure Linux RHEL Red Hat
9 min read

In #1 Web Server and #2 DB, we installed nginx and PostgreSQL directly on RHEL and registered them with systemd. This time we bring the same workloads back up as containers. RHEL adopts Podman as its standard container engine instead of Docker, and Podman runs without a daemon — you can run containers with ordinary user privileges. Add quadlet on top and you can treat containers like systemd services, carrying over the operational instincts you built in the earlier posts.

This post moves in three steps. First we cover the basics of bringing up a container with Podman, then rootless operation as a regular user, and finally integration into systemd with quadlet, all the way to automatic startup at boot.

Installing Podman and the First Container #

RHEL 9 and later ship Podman in the default repositories. Install it with dnf.

# install
sudo dnf install -y podman

# check version
podman version

Podman’s command interface is nearly identical to Docker’s, so anyone coming from Docker can move over without friction. Let’s bring up the nginx from #1 as a container.

# pull the image (Red Hat registry)
podman pull registry.access.redhat.com/ubi9/nginx-124

# map host 8080 to container 8080 and run
podman run -d --name web -p 8080:8080 \
  registry.access.redhat.com/ubi9/nginx-124 \
  nginx -g "daemon off;"

-d runs in the background, --name sets the container name, and -p maps the port. Check whether the container came up with podman ps, and get a local response with curl -I http://localhost:8080. For image sources, the UBI (Universal Base Image) family that Red Hat provides is recommended. Built on the same foundation as RHEL, it fits production environments best in terms of compatibility and security updates.

Standing Up the DB with Volumes and Environment Variables #

Running the PostgreSQL from #2 as a container makes the benefits immediately obvious. The data stays in a host volume, while the container itself can be swapped out at any time.

# host directory to hold the data
mkdir -p ~/pgdata

# run the PostgreSQL container
podman run -d --name db \
  -p 5432:5432 \
  -e POSTGRESQL_USER=appuser \
  -e POSTGRESQL_PASSWORD=secret \
  -e POSTGRESQL_DATABASE=appdb \
  -v ~/pgdata:/var/lib/pgsql/data:Z \
  registry.redhat.io/rhel9/postgresql-16

-e passes the initial user and database as environment variables, and -v connects the host’s ~/pgdata to the container’s data directory. The :Z at the end of the volume path is especially important on RHEL.

SELinux Volume Mounts and :Z #

On RHEL, SELinux is enforcing, so by default container processes cannot access host files. Append :Z to the volume option and Podman applies a container-private SELinux label (container_file_t) to that directory so the container can read and write it.

  • :Z (uppercase) is a private label used by only that container. It suits single-container data.
  • :z (lowercase) is a shared label used by multiple containers. Use it when several containers share the same directory.

Leave out :Z and a permission error shows up in the container log and the DB fails to initialize. This is the same context as the SELinux issue from #1, and in containers this one character solves it.

To open the host’s 8080 and 5432 to the outside, open the ports in firewalld exactly as in #1. Being a container does not change the firewall.

# open the ports permanently and apply
sudo firewall-cmd --add-port=8080/tcp --permanent
sudo firewall-cmd --add-port=5432/tcp --permanent
sudo firewall-cmd --reload

Opening the DB port (5432) directly to the outside is not recommended in production. As covered in #2, it is safer to narrow the access scope, or leave it so that only the web container on the same host can reach it.

Rootless Containers: Operating as a Regular User #

Podman can run containers with ordinary user privileges, without root. This is called rootless, and since a compromised container does not spread to host root, it is the default recommendation for security. Run the earlier podman run command as a regular user without sudo and it operates rootless as is. Keep two constraints in mind for rootless.

  • Privileged ports below 1024 cannot be opened directly by a regular user. So bring things up on a port at or above 1024, like 8080 instead of 80, and if needed expose it on 80 through a reverse proxy on the host or by adjusting net.ipv4.ip_unprivileged_port_start.
  • A container is tied to its user. Containers created by root and those created by a regular user are not visible to each other. podman ps shows only the current user’s containers.

The images and storage for rootless containers live under the user’s home (~/.local/share/containers) rather than system-wide. This per-user isolated structure is the foundation of rootless security.

Integrating into systemd with quadlet #

A container brought up with podman run disappears on reboot. In production, a container too must be managed as a systemd service like the nginx in #1, so it gets automatic restart and startup at boot. On RHEL 9.4 and later, quadlet is the standard way to do this.

quadlet is a structure where you place a declarative file such as .container and systemd reads it and turns it into a service unit. You do not write the unit file yourself — you only write the container definition. To use it rootless, place the file in a user directory.

# create the user quadlet directory
mkdir -p ~/.config/containers/systemd

Create a web.container file in this directory.

# ~/.config/containers/systemd/web.container
[Unit]
Description=Nginx web container

[Container]
Image=registry.access.redhat.com/ubi9/nginx-124
PublishPort=8080:8080
Exec=nginx -g "daemon off;"

[Service]
Restart=always

[Install]
WantedBy=default.target

The [Container] section is the heart of quadlet. The main keys are as follows.

  • Image: the image to bring up. Corresponds to the image argument of podman run.
  • PublishPort: port mapping. Corresponds to -p.
  • Volume: volume mount. Corresponds to -v, and you append :Z here too.
  • Environment: environment variable. Corresponds to -e.

Create db.container for the DB container the same way. The only difference is that you write additional Volume and Environment entries in [Container].

# ~/.config/containers/systemd/db.container
[Container]
Image=registry.redhat.io/rhel9/postgresql-16
PublishPort=5432:5432
Volume=%h/pgdata:/var/lib/pgsql/data:Z
Environment=POSTGRESQL_USER=appuser
Environment=POSTGRESQL_PASSWORD=secret
Environment=POSTGRESQL_DATABASE=appdb

[Service]
Restart=always

[Install]
WantedBy=default.target

%h is a systemd specifier that points to the user’s home directory. After placing the file, run daemon-reload so systemd picks up the new definition, then start the service.

# refresh so systemd reads the quadlet files
systemctl --user daemon-reload

# start the service (filename web.container → web service)
systemctl --user start web
systemctl --user start db
systemctl --user status web

quadlet automatically generates a systemd service named web.service from the web.container file. That is why you handle it with an extension-less name like systemctl --user start web. Because the container has become a systemd service, even without a separate enable it comes up automatically at the start of the user session, per [Install] WantedBy=default.target.

Automatic Startup at Boot: linger #

There is one pitfall here. A rootless user service runs only while that user is logged in; after a logout or a reboot without logging in, the container also goes down. To make the service come up at boot even when the user is not logged in, you have to turn on linger.

# enable linger for the current user (requires root)
sudo loginctl enable-linger $USER

# verify
loginctl show-user $USER | grep Linger

Turn on enable-linger and that user’s systemd user session stays alive in the background from boot time, so the quadlet containers start automatically regardless of whether the user is logged in. It is a setting you must always pin down together when operating a server rootless.

System-wide quadlet #

To operate containers as system-wide services rather than per-user, place the files in /etc/containers/systemd/. The format is identical, and only --user drops out of the management commands.

# place the system-wide quadlet file
sudo cp web.container /etc/containers/systemd/

# refresh and start the system systemd
sudo systemctl daemon-reload
sudo systemctl start web

System-wide quadlet runs with root privileges, so it does not need linger and starts automatically at boot like an ordinary systemd service. The catch is that the container then runs as root, and the security benefit of rootless disappears. For a single-user server, go with rootless + linger; when you need to bundle and manage things as system-level services, choose system-wide quadlet.

Logs and Diagnostics #

When a container does not come up or throws an error, there are two places to look. View the container’s own log with podman, and the service log after systemd integration with journalctl.

# container stdout log (-f for live)
podman logs web
podman logs -f web

# quadlet service log (rootless / system-wide)
journalctl --user -u web
sudo journalctl -u web

When SELinux blocks something like a DB volume permission issue, it surfaces as a permission error in podman logs, and is also visible as AVC denied in the host-side audit log (/var/log/audit/audit.log). At that point, as in #1, checking first whether :Z is attached to the volume clears it up in most cases.

Operational Points #

  • Podman has no daemon. Since it does not depend on a resident daemon the way Docker does, each container is managed independently as a systemd service. quadlet is the integration point.
  • Make rootless the default. Bring things up as a regular user and a container breach does not spread to host root. You only need to be aware of the privileged-port and user-binding constraints.
  • Attach :Z to volumes. When mounting a host directory in an SELinux enforcing environment, it is all but mandatory. :Z for a single container, :z for sharing.
  • Automatic startup at boot is linger. A rootless quadlet needs loginctl enable-linger turned on to come up at boot regardless of user login.
  • Ports are still firewalld. Container or direct install, external exposure requires opening the port in firewalld.

Wrap-up #

What we pinned down in this post:

  • Podman basics. Bring up containers with podman pull and podman run, and wire ports, volumes, and environment variables with -p, -v, and -e
  • rootless. Operate as a regular user to gain security isolation, while being aware of the privileged-port and user-binding constraints
  • SELinux volumes. Apply the :Z (private) and :z (shared) labels to host mounts
  • quadlet. Declare containers in ~/.config/containers/systemd/*.container and start them after systemctl --user daemon-reload
  • Automatic startup at boot. loginctl enable-linger for rootless, placement in /etc/containers/systemd/ for system-wide

Next: Monitoring #

Now that the web and DB are up as containers, it is time to look in on whether these workloads are running well.

In #4 Monitoring: Cockpit, PCP, we organize in one cycle how to see the system and containers at a glance with Cockpit, RHEL’s web management console, and the flow of collecting and tracking performance metrics with PCP (Performance Co-Pilot).

X