AppArmor is the mandatory access control system enabled by default on every Ubuntu host and on Debian since Buster, yet most operators touch it only when something breaks and the accepted “fix” is aa-teardown. That throws away the point. A good profile confines a compromised daemon to exactly the files, capabilities, and sockets it legitimately needs, with a syntax you can read in an afternoon. This guide authors one properly: a baseline in complain mode, refined from real denials, rolled to enforce without an outage.
If you take one thing away: a
DENIEDline in the kernel log is a specification, not an error. It states precisely what the daemon tried that the profile did not permit. Switching to complain mode and walking away just means you stopped reading the spec.
1. Path-based MAC: AppArmor versus SELinux
Both are Linux Security Modules (LSMs) layering mandatory access control over the standard discretionary (owner/group/mode) permissions. What they key on governs everything downstream:
- SELinux is label-based. Every file carries a security context in the
security.selinuxxattr; rules are written against types. The label is the identity, the path irrelevant. - AppArmor is path-based. Rules name filesystem paths directly. No on-disk labelling, so no relabelling, no
restorecon, no labels lost when youmva file. A profile attaches to a program by the path of its executable.
The consequence: AppArmor profiles are far easier to read and write - a rule is a path plus permission flags - and they survive filesystem operations that wreck SELinux contexts. The trade-off is that path-based confinement can be sidestepped via hard links or alternate paths to the same inode if you are careless. For daemons whose file layout you control, an excellent trade.
Profiles live in /etc/apparmor.d/, named after the binary’s path with dots for slashes (/usr/sbin/nginx becomes usr.sbin.nginx), in one of two modes: enforce (violations blocked and logged) or complain / learning (allowed but logged - how you build a profile safely). Check what is loaded, in which mode, before touching anything:
sudo aa-status # every loaded profile, its mode, and which processes are confined
aa-enabled # prints "Yes" if the LSM is active in the kernel
aa-status is sestatus plus ps -eZ in one command. The tooling lives in two packages - apparmor-utils (the aa-* helpers) and apparmor-profiles (community profiles):
sudo apt install apparmor-utils apparmor-profiles
2. Profile structure: capabilities, file rules, and abstractions
A profile is a block of rules attached to an executable path. A minimal, real one:
#include <tunables/global>
/usr/sbin/mydaemon {
#include <abstractions/base>
#include <abstractions/nameservice>
# capabilities the daemon may use
capability setuid,
capability setgid,
capability net_bind_service,
# file rules: path then permission flags
/usr/sbin/mydaemon mr,
/etc/mydaemon/** r,
/var/log/mydaemon/*.log w,
/run/mydaemon.pid rw,
}
Read the pieces in order.
Capabilities. AppArmor mediates the same CAP_* capabilities the kernel uses. Bind port 80 and you must grant capability net_bind_service; drop privileges after start and you need setuid/setgid. A confined process is denied every capability not listed - one of the strongest parts of the model.
File rules. Each rule is a path followed by mode flags. The ones you use constantly:
| Flag | Permission |
|---|---|
r / w |
read / write |
a |
append-only (a subset of write) |
m |
memory-map executable (PROT_EXEC mmap) |
k / l |
file locking / create hard links |
ix px Px cx ux |
execute transitions (step 7) |
Globbing keeps profiles maintainable: * matches within a path component, ** matches recursively across /, {a,b} is alternation. So /etc/mydaemon/** covers the whole config tree; /var/log/mydaemon/*.log matches only log files in one directory.
Abstractions make profiles short and correct - reusable includes under /etc/apparmor.d/abstractions/ bundling the rules for a common need. Rather than hand-roll the dozen rules to resolve hostnames, #include <abstractions/nameservice>. The ones you reach for most:
| Abstraction | What it grants |
|---|---|
base |
the floor every program needs (libc, /dev/null, /proc/self) - always include |
nameservice |
DNS, NSS, /etc/hosts, /etc/resolv.conf |
ssl_certs |
read the system CA bundle under /etc/ssl |
openssl |
OpenSSL config and engines |
consoles |
tty/pts access |
Always
#include <abstractions/base>. Without it the daemon cannot map libc, failing before it does anything interesting and emitting a flood of confusing denials.baseis the floor, not a convenience. The#include <tunables/global>outside the block defines variables (@{HOME},@{PROC}) the abstractions reference - omit it and some fail to parse.
3. Generate a baseline with aa-genprof and complain mode
Never write a profile from a blank file. aa-genprof is the interactive generator: it creates a skeleton, puts it in complain mode, and watches the audit log while you exercise the program in another terminal.
sudo aa-genprof /usr/sbin/mydaemon
It prints “Profiling” and pauses with (S)can system log / (F)inish. Now, in a second terminal, drive the service through every path that matters - start, reload, hit every endpoint, rotate logs:
sudo systemctl start mydaemon
curl -s http://localhost:8080/health
sudo systemctl reload mydaemon
Return to aa-genprof and press S to scan. For each logged access it presents a menu - the heart of the workflow:
| Key | Action |
|---|---|
A |
Allow the access (adds a rule) |
D |
Deny it (adds a deny rule) |
I |
Allow via an abstraction include if one matches - prefer this |
G |
Allow as a literal glob (/etc/mydaemon/*) |
N |
Allow with a new glob you type (e.g. /var/log/mydaemon/**) |
S |
Save and move on |
F |
Finish |
Prefer
I(abstraction) andG/N(glob) over bare literal paths. A profile full of individual file rules is brittle; one that uses<abstractions/nameservice>and/etc/mydaemon/**survives version upgrades and new config files. You are authoring a policy, not transcribing one run.
On finish, aa-genprof writes /etc/apparmor.d/usr.sbin.mydaemon in complain mode. To start a profile by hand instead, aa-autodep creates the skeleton and aa-complain sets learning mode:
sudo aa-autodep /usr/sbin/mydaemon # generate an empty skeleton profile
sudo aa-complain /usr/sbin/mydaemon # set it to complain (learning) mode
In complain mode the daemon runs normally and is never blocked while you collect its real access pattern - safe on a staging box carrying realistic traffic.
4. Refine from logs with aa-logprof and DENIED events
The iterative loop is aa-logprof - the same engine as aa-genprof, but operating on the accumulated audit log across all profiles, so you run it repeatedly as you find gaps.
First, read a raw denial. AppArmor logs through the audit subsystem; events land in /var/log/syslog and /var/log/kern.log, or /var/log/audit/audit.log if auditd is installed:
type=AVC msg=audit(1717081200.123:456): apparmor="DENIED" operation="open"
profile="/usr/sbin/mydaemon" name="/etc/mydaemon/secret.conf" pid=2310
comm="mydaemon" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Decode it field by field - this is the whole skill:
| Field | Meaning |
|---|---|
apparmor="DENIED" |
refused (reads ALLOWED in complain mode, which logs but permits) |
operation |
the syscall class (open, exec, connect, mknod) |
profile |
which profile (or sub-profile) was in force |
name |
the object - a path, capability, or address |
requested_mask / denied_mask |
bits asked for vs. refused |
comm / pid |
the offending process |
Pull these directly with dmesg or journalctl for a quick look before the interactive refiner:
sudo dmesg | grep -i apparmor | grep DENIED
sudo journalctl -k | grep 'apparmor="DENIED"' | tail -n 30
Now run the refiner. It walks every unhandled event and offers the same A/I/G/N/D menu as aa-genprof:
sudo aa-logprof
The discipline mirrors writing SELinux policy: read what it proposes before accepting. Treat these as red flags worth a deny (D) or a tighter glob, not a blind allow:
- A write where the daemon should only read -
name="/etc/..." requested_mask="w"usually means a bug or an attack, not a missing rule. - A broad capability like
sys_adminordac_override- these hand back most of what confinement removed. - An unexpected
execof a shell (/bin/sh,/bin/dash) - frequently command injection surfacing. You want to see it, not silence it.
After it writes, reload and exercise again to flush out the next layer:
sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.mydaemon
Repeat - exercise, aa-logprof, reload - until a full run produces no new events. That convergence signals the profile is complete enough to enforce.
5. Confine a real service end to end: Nginx
Make it concrete with Nginx, where the layout is well known. The goal: a profile that serves content, reads config and TLS material, writes logs, and binds low ports - nothing else. Start from a skeleton in complain mode so production traffic is unaffected:
sudo aa-autodep /usr/sbin/nginx
sudo aa-complain /usr/sbin/nginx
sudo systemctl restart nginx
Drive real traffic - reload after a config change, request a TLS endpoint, trigger an error page - then run aa-logprof. A hand-finished result:
#include <tunables/global>
/usr/sbin/nginx {
#include <abstractions/base>
#include <abstractions/nameservice>
#include <abstractions/openssl>
#include <abstractions/ssl_certs>
# bind 80/443, then drop to the www-data worker
capability net_bind_service,
capability setuid,
capability setgid,
capability dac_override,
/usr/sbin/nginx mr,
/etc/nginx/** r,
/etc/ssl/private/** r,
/var/log/nginx/*.log w,
/var/www/** r,
/run/nginx.pid rw,
/run/nginx/** rw,
/usr/share/nginx/** r,
/usr/lib/nginx/modules/*.so mr,
# worker temp paths
/var/lib/nginx/** rw,
}
The decisions worth calling out:
/var/www/**isronly. A web server that can write its document root is one upload bug from a webshell. If an app needs uploads, scope a singlerwpath to it, never the whole root./etc/ssl/private/**is explicit and read-only. The private key is the crown jewel; a compromised worker still cannot overwrite it.net_bind_serviceallows binding 80/443 as a non-root worker after the master drops privileges;setuid/setgidallow that drop.
Ubuntu ships maintained extra profiles under
/usr/share/apparmor/extra-profiles/. In production, start from a battle-tested profile and adjust paths rather than generating from scratch - treat the from-zero workflow above as how you understand and extend it, not a mandate to reinvent it.
6. Network rules, signals, and Unix sockets
Modern AppArmor mediates far more than files. Three controls matter for almost every daemon.
Network rules restrict which socket families a process may use. The coarse-grained form gates by address family - universally supported, and what aa-logprof suggests:
network inet tcp, # IPv4 TCP
network inet6 tcp, # IPv6 TCP
network inet udp, # IPv4 UDP (e.g. DNS)
network netlink raw, # some daemons need netlink for interface info
A daemon that should never open a socket omits all network rules and is then incapable of network I/O even if exploited - a powerful default for a local file processor.
Signals are mediated too, which matters when one confined process signals another (a master signalling workers, or systemd reloading). The rule names the signal, a direction, and a peer:
signal (receive) set=(term, hup, quit) peer=unconfined, # signals from systemd
signal (send,receive) set=(term, usr1) peer=/usr/sbin/nginx, # master <-> worker
A blocked reload (SIGHUP) showing operation="signal" after enforce is the classic symptom of a missing rule here.
Unix sockets have their own mediation. For abstract or filesystem IPC sockets (a PHP-FPM socket Nginx connects to), grant the unix rule and, for a filesystem socket, the path:
unix (send,receive,connect) type=stream peer=(addr="@/myapp/*"),
/run/php/php-fpm.sock rw,
When a service “works on the network but the local socket is refused,”
networkrules are not the cause - check for aunixdenial and the socket path rule. Missing either produces the same connection-refused symptom in the app log while the truth sits indmesg.
7. Profile transitions, child profiles, and exec rules
The most security-relevant decision in a profile is what happens when the confined program executes another program. A helper that runs unconfined is a clean escape hatch out of the sandbox. AppArmor’s execute-transition flags decide the child’s fate - choosing right is the difference between real containment and theatre:
| Flag | Child runs under… |
|---|---|
ix |
the same profile (inherit) - child stays in the parent’s confinement |
px |
a separate profile for the child, which must exist (profile transition); fails if none exists |
Px |
like px, with environment scrubbing (strips LD_PRELOAD etc.) - the secure default |
cx |
a child profile defined inline within this profile |
Cx |
inline child profile, with environment scrubbing |
ux / Ux |
unconfined - the child escapes AppArmor entirely. Avoid; Ux at least scrubs the environment |
Rule of thumb: Px for substantial helpers, ix for trivial ones that should share the parent’s box, and treat ux/Ux as a code smell needing justification in review.
Child profiles are the elegant pattern when a daemon shells out to a tightly-scoped helper. Define the helper’s confinement inline with Cx -> name, sandboxing it more tightly than the parent:
/usr/sbin/backupd {
#include <abstractions/base>
/usr/sbin/backupd mr,
/var/backups/** rw,
/bin/gzip Cx -> gzip, # transition into the inline child below
profile gzip {
#include <abstractions/base>
/bin/gzip mr,
/var/backups/** rw, # child touches ONLY backups, nothing else backupd can reach
}
}
Even though backupd may read other paths, the gzip child touches only /var/backups - least privilege per process, not per service.
8. Enforce-mode rollout, troubleshooting, and rollback
You have a converged profile that logs nothing on a full exercise. Now flip it to enforce, carefully. Set the single profile and reload it into the kernel:
sudo aa-enforce /usr/sbin/mydaemon
sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.mydaemon
sudo systemctl restart mydaemon
sudo aa-status | grep mydaemon # confirm it now reports "enforce"
Then exercise it again under enforcement and watch the kernel log live. Anything missed in complain mode surfaces as a real block:
sudo journalctl -k -f | grep 'apparmor="DENIED"'
Troubleshooting follows a fixed order. If the daemon misbehaves after enforcing:
- Confirm AppArmor is the cause. A
DENIEDline with your profile name indmesgis proof; its absence means look elsewhere (a real config bug, not policy). - If it is AppArmor, drop that one profile back to complain - never tear down the whole subsystem:
sudo aa-complain /usr/sbin/mydaemon # this profile learns again; everything else stays enforced
- Reproduce, run
sudo aa-logprof, fold in the legitimate rule, reload, and re-enforce.
Safe rollback. If a profile is breaking production and you need it gone now, unload it from the kernel without deleting the file, so the daemon runs unconfined until you fix the policy. The more durable option symlinks it into disable/ so it stays off across boots:
sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.mydaemon # unload now
sudo ln -s /etc/apparmor.d/usr.sbin.mydaemon /etc/apparmor.d/disable/ # keep it off on boot
To re-enable, remove the symlink and reload.
Resist
sudo aa-teardownandsystemctl stop apparmoras a debugging step. Those unload every profile on the host, silently un-confining your entire fleet to fix one. Per-profileaa-complainandapparmor_parser -Rgive the same relief scoped to the one broken service.
Enterprise scenario
A SaaS platform ran a multi-tenant document-processing service on Ubuntu 22.04: a Go daemon that accepted uploads and shelled out to libreoffice --headless and ghostscript to render thumbnails. Security review flagged the obvious risk - a malicious upload exploiting a Ghostscript parser bug (a recurrent class of CVE) would get code execution as the service user, with read access to every tenant’s files on the shared volume.
The constraint: they could not rewrite the pipeline before the audit deadline, and the converters genuinely needed to read input and write output under /var/spool/render. A blanket profile on the Go daemon would not help - the converters, the actually-dangerous code, would inherit its full file access via ix and still walk the whole spool. The fix was child profiles confining each converter far more tightly than the parent, transitioning with environment scrubbing so a poisoned LD_PRELOAD in an upload could not ride along:
/usr/local/bin/renderd {
#include <abstractions/base>
#include <abstractions/nameservice>
/usr/local/bin/renderd mr,
/var/spool/render/** rw,
network inet tcp,
# converters get their OWN tight profiles, scrubbed, not inheritance
/usr/bin/gs Cx -> ghostscript,
/usr/bin/soffice.bin Px -> libreoffice,
profile ghostscript {
#include <abstractions/base>
/usr/bin/gs mr,
/usr/lib/ghostscript/** mr,
# only the single job directory, passed per-invocation, not the whole spool
/var/spool/render/jobs/** rw,
deny network, # a renderer never needs a socket
deny /etc/shadow r, # belt and braces against credential theft
}
}
The decisive moves: Cx/Px instead of ix, so the converters could not reach the parent’s broad spool access; deny network, so a Ghostscript exploit could not exfiltrate even though the parent daemon legitimately uses the network; and deny /etc/shadow r as defence in depth. They ran renderd and both children in complain mode for a week under production load, harvested residual rules with aa-logprof, then enforced. The audit closed with the exploit chain demonstrably broken - a known Ghostscript PoC executed but could not read a second tenant’s directory or open a socket.
Verify
Confirm the end state explicitly rather than assuming the restart “worked”:
sudo aa-status # profile under "enforce", process counted as confined
cat /proc/$(pgrep -n mydaemon)/attr/current # shows "/usr/sbin/mydaemon (enforce)"
sudo journalctl -k --since "10 min ago" | grep 'apparmor="DENIED"' # empty after a full exercise
sudo apparmor_parser -p /etc/apparmor.d/usr.sbin.mydaemon # profile parses cleanly
The decisive signals: aa-status shows the profile in enforce mode and counts the process as confined, and the kernel log is clean after real traffic through every code path. Reading /proc/<pid>/attr/current is the AppArmor ps -eZ - ground truth that this specific process is confined, not merely that a profile exists.
Checklist
Pitfalls and next steps
A few traps that catch even experienced operators:
aa-teardownto debug. It un-confines the entire host. Scope relief to the broken profile withaa-complain <path>andapparmor_parser -R <file>.- Forgetting to reload after editing. Editing a file in
/etc/apparmor.d/changes nothing untilapparmor_parser -rloads it - the running policy is the loaded one, not the file on disk. ixon a dangerous helper. Inheriting the parent gives a Ghostscript or shell helper all its file access. UsePx/Cxfor anything parsing untrusted input.- Write access to the document root is a webshell waiting to happen. Default to
r, grantrwonly to a specific upload path. - Confusing
networkandunixdenials. A blocked local socket is aunixrule (and path) problem, notnetwork; the app sees the same “connection refused” either way.
For deeper coverage, read the maintained profiles under /usr/share/apparmor/extra-profiles/ and the abstractions in /etc/apparmor.d/abstractions/ - the canonical examples of well-structured policy. The natural progression: mediating mount and ptrace, dbus rules for bus services, confining containers (Docker and LXD both layer AppArmor over workloads), and wiring apparmor_parser -p into CI so every profile change is parsed, diffed, and reviewed like any other code.