Red Hat Certified System Administrator (RHCSA) #13: SELinux in depth — contexts, booleans, troubleshooting (audit2allow)

In #12 firewalld and SSH key authentication we opened ports and logged in with keys. But if you opened the firewall and brought up the service and connections are still blocked, the next suspect is almost always SELinux. SELinux is the area where RHCSA candidates trip up most often, and at the same time the area where, once you have the patterns down, you most reliably bank points. This post pins down contexts, booleans, port labels, and troubleshooting from a hands-on exam perspective.

The core of SELinux is simple. Every process and file gets a label (context), and only the combinations the policy allows pass through. Even an operation that clears every standard Linux permission (rwx) will fail if the SELinux policy denies it. Nearly every SELinux problem you meet on RHCSA shows up as “the permissions are right, but it doesn’t work.”

SELinux modes: enforcing, permissive, disabled #

SELinux has three modes.

permissive is the “doesn’t block, but logs what would have been blocked” mode, which makes it very useful during troubleshooting.

Checking the current mode and switching temporarily #

You read the current mode with getenforce. sestatus shows more detailed status.

getenforce
# Enforcing

sestatus
# SELinux status:                 enabled
# Current mode:                   enforcing
# Mode from config file:          enforcing
# Policy from config file:        targeted

setenforce switches between enforcing and permissive immediately. But this switch is a temporary setting that disappears on reboot.

setenforce 0   # switch to permissive
setenforce 1   # switch to enforcing
getenforce
# Permissive

setenforce can’t take you to disabled. It only moves between enforcing and permissive.

Permanent mode: /etc/selinux/config #

The mode that survives a reboot is set by the SELINUX= value in /etc/selinux/config. When RHCSA gives you a task like “set SELinux permanently to enforcing,” this is the file you edit.

cat /etc/selinux/config
# SELINUX=enforcing
# SELINUXTYPE=targeted

Changing the SELINUX= value to one of enforcing, permissive, or disabled takes effect from the next boot. The exam usually asks for permanent enforcing, so the safe move is to apply it immediately with setenforce 1 and also write enforcing into the config file.

setenforce 1
sed -i 's/^SELINUX=.*/SELINUX=enforcing/' /etc/selinux/config
grep '^SELINUX=' /etc/selinux/config
# SELINUX=enforcing

When you switch from disabled to enforcing, labels may be missing across the whole file system, so an automatic relabel happens on the next boot. To force a relabel, touch /.autorelabel and reboot.

Context: the structure of a label #

SELinux attaches a context to every object. A context is made of four parts separated by colons.

user:role:type:level
system_u:object_r:httpd_sys_content_t:s0

The part you actually deal with on RHCSA is almost always just the type (the third field). That’s because most policy decisions are made through the allow rules between types (type enforcement). For example, the type of a web server process is httpd_t, and the type of a web content file is httpd_sys_content_t. The policy allows “httpd_t may read httpd_sys_content_t,” so the web server serves that file.

Viewing file context: ls -Z #

The -Z option shows the SELinux context. You check a file’s context with ls -Z.

ls -Z /var/www/html/
# unconfined_u:object_r:httpd_sys_content_t:s0 index.html

Viewing process context: ps -Z #

You view a running process’s context with ps -Z.

ps -eZ | grep httpd
# system_u:system_r:httpd_t:s0 1234 ? 00:00:00 httpd

id -Z shows the current shell’s context, and ss -Z or netstat -Z shows a socket’s context.

What happens when the context is wrong #

This is the most common scenario on RHCSA. When you put web content somewhere other than the standard location, or move a file in from elsewhere, the context gets out of sync. For example, if you mv a file you created in a home directory to /var/www/html, that file carries over the context of its original location (user_home_t, etc.). As a result the web server process (httpd_t) can’t read it and you get a 403.

There’s a key rule to remember here.

• A cp (copy) picks up the default context of the destination directory.
• An mv (move) preserves the original’s context as-is.

So after moving a file you have to correct the context.

Fixing the context: chcon vs restorecon vs semanage #

There are three ways to change a context, and which one you use can decide your RHCSA score.

chcon: temporary change (avoid it on the exam) #

chcon changes a file’s context directly. It applies immediately, but it is not reflected in the policy database. So if restorecon runs later or a file system relabel happens, it reverts to the original.

chcon -t httpd_sys_content_t /var/www/html/index.html
chcon -R -t httpd_sys_content_t /web/content   # recursive change

Use chcon only for quick tests; it’s not a safe exam answer. If the system gets relabeled before grading, your change will be undone.

semanage fcontext + restorecon: permanent change (the correct answer) #

The correct way to fix a context permanently is two steps.

1. Use semanage fcontext -a -t to add a default context rule to the policy.
2. Use restorecon to apply that rule to the actual files.

For example, to use everything under /web as web content, you do this.

# 1) Add the rule to the policy: define everything under /web as httpd_sys_content_t
semanage fcontext -a -t httpd_sys_content_t "/web(/.*)?"

# 2) Apply the rule to the actual files
restorecon -R -v /web

"/web(/.*)?" is a regular expression that means the /web directory itself and every path beneath it. This way the rule remains in the policy itself, so the context holds even when a relabel happens. This is the permanent application RHCSA asks for.

semanage is in the policycoreutils-python-utils package. If it’s missing, install it.

dnf install -y policycoreutils-python-utils

To check or delete a rule, do this.

semanage fcontext -l | grep '/web'   # check the rule you added
semanage fcontext -d "/web(/.*)?"    # delete the rule

restorecon: reverting to the default context #

restorecon re-applies the default context defined in the policy to a file. A file in a standard location whose context got out of sync from a mv is corrected with just restorecon, no semanage needed. That’s because /var/www/html is already defined as httpd_sys_content_t in the policy.

restorecon -R -v /var/www/html
# Relabeled /var/www/html/index.html
#   from unconfined_u:object_r:user_home_t:s0
#   to   unconfined_u:object_r:httpd_sys_content_t:s0

To sum up: for a standard location, restorecon alone; for a non-standard location, add a rule with semanage fcontext and then restorecon.

boolean: turning policy behavior on and off #

There are cases where things are blocked even when the context is correct. SELinux policy has on/off switches called booleans that allow or block specific behaviors at the policy level. For example, behaviors like a web server serving user home directories, or making outbound network connections, are off by default via booleans.

Listing booleans: getsebool #

You see all booleans with getsebool -a. semanage boolean -l also shows the default value and a description.

getsebool -a | grep httpd
# httpd_can_network_connect --> off
# httpd_enable_homedirs --> off
# httpd_use_nfs --> off

semanage boolean -l | grep httpd_can_network_connect
# httpd_can_network_connect (off , off)  Allow httpd to ...

Setting a boolean: setsebool -P #

You flip a boolean with setsebool. The -P option is the key here. Turn it on without -P and it applies only in memory, so it turns off on reboot. You have to add -P to save it permanently to the policy.

setsebool httpd_can_network_connect on      # temporary. turns off on reboot
setsebool -P httpd_can_network_connect on   # permanent. the correct answer

On RHCSA, a task like “make a web app that connects to a database work” is answered nine times out of ten by turning on httpd_can_network_connect with -P. Leave off -P and you lose points when the system is rebooted at grading time.

Port labels: semanage port #

SELinux labels ports too. If you run a service on a non-standard port, that port’s label won’t match the service’s type and the bind is denied. For example, if you change sshd from port 22 to port 2222, port 2222 has no ssh_port_t label, so sshd can’t open that port. Opening the port with firewalld does no good — it’s blocked at the SELinux layer.

Viewing current port labels #

semanage port -l | grep ssh
# ssh_port_t   tcp   22

semanage port -l | grep http_port_t
# http_port_t  tcp   80, 81, 443, 488, 8008, 8009, 8443, 9000

Adding a port label: semanage port -a -t #

To use a non-standard port, assign the appropriate type to that port.

# to use sshd on port 2222, give that port ssh_port_t
semanage port -a -t ssh_port_t -p tcp 2222
semanage port -l | grep ssh
# ssh_port_t   tcp   2222, 22

If the port is already occupied by another type, use -m (modify) instead of -a (add). When you get an already defined error while adding, switching to -m is the standard move.

semanage port -m -t http_port_t -p tcp 8888

An RHCSA task that moves a service to a non-standard port is only complete when you handle opening the firewalld port + adding the SELinux port label together as one bundle. Do only one of the two and the connection won’t go through.

Troubleshooting: tracing denial logs and building policy #

Every SELinux denial is recorded in the audit log as an AVC (Access Vector Cache) denial. Reading what was blocked and why out of this log is the starting point of troubleshooting. The log is at /var/log/audit/audit.log.

ausearch: finding denials in the audit log #

You collect recent AVC denials with ausearch.

ausearch -m AVC -ts recent
# type=AVC msg=audit(...): avc:  denied  { read } for
#   pid=1234 comm="httpd" name="index.html"
#   scontext=system_u:system_r:httpd_t:s0
#   tcontext=unconfined_u:object_r:user_home_t:s0  tclass=file

The parts you need to read here are clear: comm="httpd" (who), denied { read } (what), tcontext=...user_home_t (against a target with which label). It means the httpd_t process was blocked while trying to read a user_home_t file — a textbook case of a wrong context. The answer is to correct the label with restorecon or semanage fcontext.

sealert: reading setroubleshoot’s advice #

Install the setroubleshoot-server package and, every time a denial occurs, a human-readable analysis and a recommended command are recorded to /var/log/messages. You read the detail with sealert.

dnf install -y setroubleshoot-server
sealert -a /var/log/audit/audit.log

sealert often presents a specific command such as setsebool -P ... or restorecon ..., saying “fix this problem with the following command.” But before following the advice verbatim, you need to judge whether it actually reflects the behavior you intended.

audit2allow: building a policy module from denial logs #

For a denial that custom policy is needed for — one that a standard boolean or context fix won’t resolve — you use audit2allow. It takes a denial log as input and generates a policy rule that allows that behavior.

First, you eyeball the rule to confirm what you’d be allowing.

ausearch -m AVC -ts recent | audit2allow
# allow httpd_t user_home_t:file { read };

When you judge the rule to be sound, build it into a module and load it.

# create a policy module named myhttpd
ausearch -m AVC -ts recent | audit2allow -M myhttpd

# load the generated module
semodule -i myhttpd.pp

audit2allow -M creates a .te (policy source) and a compiled .pp (policy package) file, and semodule -i adds that .pp to the system policy. You check loaded custom modules with semodule -l and remove one with semodule -r myhttpd.

Troubleshooting priority #

audit2allow is powerful but it is the last resort. Building a module that allows a denial unconditionally can paper over the real cause (a wrong context or a boolean that’s off). The recommended order on RHCSA is as follows.

1. Check the context. Look at the label with ls -Z, and if it’s out of sync, correct it with restorecon or semanage fcontext.
2. Check the boolean. See whether the behavior itself is off through a boolean, and if so, turn it on with setsebool -P.
3. Check the port label. If it’s a non-standard port, assign a label with semanage port.
4. Build a module with audit2allow only for a genuinely custom behavior that none of the above resolves.

Briefly switching to permissive is also useful for diagnosis. If the service works normally under permissive, the cause is clearly SELinux, and you can collect and analyze all the denial logs that accumulated in one go. When the diagnosis is done, be sure to return to enforcing.

Exam points #

• chcon vs restorecon. chcon is temporary and comes undone on relabel. The permanent answer is restorecon for a standard location, and semanage fcontext -a -t then restorecon for a non-standard location. Don’t use chcon as an exam answer.
• setsebool’s -P. Leave off -P and it disappears on reboot, losing you points at grading. Always use setsebool -P when turning on a boolean.
• semanage fcontext regex. Write everything under a directory as "/path(/.*)?". Wrap it in quotes so the shell doesn’t interpret the asterisk.
• A non-standard port is two places. Open the port with firewalld, and assign the SELinux label too with semanage port -a -t to complete the connection.
• Permanent mode is /etc/selinux/config. setenforce is temporary. For permanent enforcing, verify down to SELINUX=enforcing in the config file.
• Troubleshooting tools. The log is /var/log/audit/audit.log, denial search is ausearch -m AVC, advice is sealert, and policy generation is audit2allow -M then semodule -i.
• If semanage is missing install policycoreutils-python-utils, and if sealert is missing install setroubleshoot-server.

Wrap-up #

What this post locked in:

• SELinux modes. Temporary switch with getenforce/setenforce, permanent setting with /etc/selinux/config. enforcing , permissive , disabled
• Context. Check with ls -Z/ps -Z. The key is the type field. mv carries the label over, cp takes the destination’s default
• Fixing the context. chcon is temporary; permanent is semanage fcontext -a -t + restorecon. For a standard location, restorecon alone is enough
• boolean. Check with getsebool -a, set permanently with setsebool -P. A per-behavior on/off switch
• Port labels. Assign a type to a non-standard port with semanage port -a -t. One bundle with firewalld
• Troubleshooting. audit.log → ausearch/sealert → a policy module with audit2allow. But check the context, boolean, and port first

SELinux is the area where the concepts you worked through in the RHEL practical track get tightened up from an exam perspective. When you see a denial, don’t panic — get into the habit of checking in the order: context, boolean, port, and SELinux becomes a reliable source of points on RHCSA.

Next — Container management #

We’ve finished the security area too. Now we move on to the last big axis of the RHCSA scope, containers.

In #14 Container management: Podman, systemd integration (quadlet), we’ll type our way through the basic Podman commands for running containers without root, pulling and running an image and mounting storage, and the quadlet integration that registers a container as a systemd service so it starts automatically at boot.