Comment # 6 on bug 1045886 from Andrei Borzenkov

(In reply to Roger Whittaker from comment #4)
> Failing system has systemd-233-1.1.x86_64

The obvious difference is that with 232 no kernel keyrings were setup by
default at all (we do not have pam_keyinit on TW in default configuration),
while with 233 each session sets up its own keyring. Unfortunately, these
keyrings seem to propagate to user sessions, messing up keys lookup. More
details

1. systemd commit:

commit 74dd6b515fa968c5710b396a7664cac335e25ca8
Author: Lennart Poettering <lennart@poettering.net>
Date:   Fri Dec 2 01:54:41 2016 +0100

    core: run each system service with a fresh session keyring

so far, so good.

2. Now lets look at keyrings immediately after logon

bor@10:~> : Before ecryptfs-setup
bor@10:~> cat /proc/keys
023c3b10 I--Q---     1 perm 0b0b0000     0     0 user      invocation_id: 16
05088f05 I--Q---    88 perm 3f030000     0     0 keyring   _ses: 1
2344184f I--Q---    41 perm 3f030000  1000   100 keyring   _ses: 1
bor@10:~> keyctl show -x @s
Keyring
0x05088f05 --alswrv      0     0  keyring: _ses
0x023c3b10 ----s-rv      0     0   \_ user: invocation_id

Note - our session keyring is owned by user 0!!! So it is the one inherited
from systemd service. (Heck, is there any way to list session keyrings for each
process?)

3. Now let's see what happens after ecryptfs-setup runs

bor@10:~> : After ecryptfs-setup
bor@10:~> cat /proc/keys
023c3b10 I--Q---     1 perm 0b0b0000     0     0 user      invocation_id: 16
041560fc I--Q---     1 perm 1f3f0000  1000 65534 keyring   _uid_ses.1000: 1
05088f05 I--Q---    90 perm 3f030000     0     0 keyring   _ses: 1
2344184f I--Q---    39 perm 3f030000  1000   100 keyring   _ses: 1
2540e350 I--Q---     2 perm 1f3f0000  1000 65534 keyring   _uid.1000: 2
28543b18 I--Q---     1 perm 3f010000  1000   100 user      ae0285ccbff6e3cb:
740
2dec2d68 I--Q---     1 perm 3f010000  1000   100 user      be1ad3e0687105d5:
740
bor@10:~> keyctl show -x @s
Keyring
0x05088f05 --alswrv      0     0  keyring: _ses
0x023c3b10 ----s-rv      0     0   \_ user: invocation_id
bor@10:~> keyctl show -x @us
Keyring
0x041560fc --alswrv   1000 65534  keyring: _uid_ses.1000
0x2540e350 --alswrv   1000 65534   \_ keyring: _uid.1000
0x2dec2d68 --alswrv   1000   100       \_ user: be1ad3e0687105d5
0x28543b18 --alswrv   1000   100       \_ user: ae0285ccbff6e3cb

Oops. ecryptfs adds keys to *user* keyring. This keyring is not used during
default key lookup directly - only if it is linked from session (or in general
other default search path) keyring. But we already have "wrong" session keyring
which does *NOT* link to our user keyring. So default user session is not used
and key is not found.

Adding pam_keyinit.so to default session PAM service fixes it (disclaimer - I
do not know whether it may have some other side effects). Note that systemd 233
actually includes it in recommended PAM configuration shipped with it.