Comment # 3 on bug 1069906 from Thorsten Kukuk

(In reply to Christian Boltz from comment #2)

> There are more interesting things: apparmor.service has
>     After=var.mount var-lib.mount
> so I'd expect it to start late enough.
> (Note that apparmor.service should start as early as possible - the profiles
> have to be loaded before the programs they confine start.)

/var and /var/lib are no mount points and such this statement is
in this case ignored by systemd. This does not mean you should remove them,
with the cleanup of /var this will become important, only that this is not
enough.

The current design works with a default read-write filesystem like openSUSE
Tumbleweed, Leap and SLE have. If you have a read-only root filesystem like
MicroOS/CaaSP/Kubic or any Live-DVD, this does not work.

I see two solutions:
Add a statement, that systemd should only run the service if /var/lib/apparmor
is writeable (but this is racy, as systemd could still remount it later) or add
var-lib/apparmor.mount to the After line. This should allow systemd to run
apparmor as early as possible, but not before we have a read-write filesystem.
If you want to have apparmor loading the policies as early as possible:
- Load them without creating a cache immeaditly at boot.
- Create a service which will create the cache later in the boot process.

> I don't know what these messages mean exactly, but (assuming/guessing that
> they refer to an overlay mount for /var/lib/) they could be a part of the
> puzzle.

This means systemd is trying to unmount overlayfs, while it has already open
files. Stupid idea from systemd.

> I never tried Kubic or Caas, therefore I can only guess what happens, and
> hope that the above hints help you to understand the problem. If you have an
> idea how to handle this (ideally without delaying loading the profiles),
> please tell me ;-)

As written above: it is a generic problem, Kubic and CaaSP only triggers them,
as every Live-DVD or similar did.
I think we need to fix the dependencies of the apparmor service, and maybe
split it even in two parts.