known·good

Notes / linuxaclpermissionscertbothome-assistantbackupcron

Default ACL shows #effective:--- on files an app rewrites with mode 0600 (certbot keys, Home Assistant auth)

· by shaun, written up with Claude

A default ACL granting a backup user read access worked, then broke after certbot issued a key or Home Assistant rewrote .storage/auth. Files created with mode 0600 get an ACL mask of ---, so the inherited entry shows #effective:---. Re-running setfacl -m recomputes the mask; run it from cron before each backup.

Tested onArch Linux ARM on a Raspberry Pi 5 with Home Assistant; a cloud Linux server with certbot; POSIX ACLs (getfacl/setfacl); exact versions not recorded

Symptoms #

A nightly backup runs as an unprivileged backup user. Root-owned files were made readable to that user with a default ACL on their directories, and the backup worked for a few days. Then it started reporting one error each night.

On the Home Assistant machine (restic):

texterror: open /var/lib/homeassistant/.storage/auth: permission denied
Warning: at least one source file could not be read

On the certbot machine (rsync), right after a new certificate was issued:

textrsync: [sender] send_files failed to open "/etc/letsencrypt/archive/example.com/privkey1.pem": Permission denied (13)
rsync: [sender] send_files failed to open "/etc/letsencrypt/keys/0062_key-certbot.pem": Permission denied (13)
rsync error: some files/attrs were not transferred (see previous errors) (code 23)

getfacl on the failing file shows the named-user entry is still there, but masked out:

textuser:backupuser:r-x	#effective:---
mask::---

Only the newly written files were affected. Older files in the same directories (for example Home Assistant's auth_provider.homeassistant, mode 0640) stayed readable.

Root cause #

When a file has named-user or named-group ACL entries, it also has a mask entry. The effective permission of every named entry (and of the owning group entry) is its own permission AND the mask. getfacl prints #effective: when the mask removes something.

When a file is created in a directory that has a default ACL, acl(5) ("Object creation and default ACLs") says:

  1. the new file inherits the directory's default ACL as its access ACL, and
  2. the entries that correspond to the file permission bits are then reduced to what the mode argument of open()/creat() allows.

When the ACL has a mask, the group permission bits correspond to the mask entry, not to the owning-group entry (acl(5), "Correspondence between ACL entries and file permission bits"). So the group bits of the create mode cap the mask. An application that creates its secrets with mode 0600 has group bits ---, so the new file gets mask::---, and the inherited user:backupuser:r-x becomes #effective:---.

The same correspondence works in the other direction, so a later chmod 600 on a file with an ACL also sets the mask to ---. (General ACL behaviour from the man page; in these two cases the files were created or rewritten by the application at 0600.)

That is why the default ACL looked like a fix and then stopped working:

  • Home Assistant rewrites .storage/auth (mode 0600) whenever auth or refresh tokens change. The default ACL lasted until the next token rotation.
  • certbot writes new private keys under /etc/letsencrypt/archive/<name>/ and /etc/letsencrypt/keys/ at mode 0600. Issuing a certificate for a new name produced two unreadable files.

A default ACL alone can never give a durable grant on files an application (re)writes at mode 0600.

Fix #

setfacl -m recalculates the mask by default. setfacl(1): the mask is set to "the union of all permissions of the owning group, and all named user and group entries", unless you give a mask entry explicitly or use -n. So re-applying the grant restores mask::r-x.

Immediate fix, as root:

bashsetfacl -R -m u:backupuser:rX /var/lib/homeassistant/.storage
setfacl -R -m u:backupuser:rX /etc/letsencrypt

(X gives execute only to directories and to files that already have execute for someone.)

Durable fix: re-apply it from root's cron a few minutes before the backup runs, for example /etc/cron.d/backup-acl-refresh:

text55 2 * * * root setfacl -R -m u:backupuser:rX /var/lib/homeassistant/.storage 2>/dev/null

with the backup at 03:00, and the same pattern with /etc/letsencrypt on the certbot machine.

Verified: on the Home Assistant machine a manual backup re-run then finished with no errors; on the certbot machine all *.pem files were readable by the backup user and an rsync dry-run was clean.

Caveats:

  • If the application rewrites the file between the cron job and the backup, that night still fails. Home Assistant is idle overnight, so this was accepted as rare.
  • The recalculated mask also re-enables whatever the owning-group entry grants (the mask is the union of that entry and the named entries). Check getfacl on a key file after the fix if the owning group should not read your private keys. This follows from setfacl(1); it was not checked separately in these cases.
  • The alternative is to not need the ACL at all: read the files as root (or have a root job copy them) for the backup.

References #

  • acl(5): "Correspondence between ACL entries and file permission bits" and "Object creation and default ACLs".
  • setfacl(1): -n, --no-mask and --mask.