My guess is that you might have hit a problem with an outdated cache file in /var/cache/apparmor/ (the cache is only checked based on the timestamps of the involved files in /etc/apparmor.d/). However, I'm still a bit surprised because the profiles should have a new-enough timestamp (there have been some AppArmor changes unrelated to the profiles, but as part of that, the profiles get a newer timestamp when the package gets built.)
Another possible part of the puzzle is the precompiled cache in /usr/share/apparmor/cache/ - maybe your autogenerated samba profile sniplet was "too old" so that the precompiled cache "won"? But again, there were no recent changes to the profiles, so in this case it {w,sh}ould have been broken for you since quite a while.
Hmm, maybe I have an explanation. It sounds like a combination of corner case and bad luck, but it's still more likely than the guesses above: Did the breakage start with the reboot after a kernel update?
Yes, it started right after I rebooted as prior to do the update one of my other TW machines which had been updated to 20221101 was still connected to the server which was still running 20220908 ( but had just been updated but not rebooted yet)
(Since you came from 20220908, it's likely that you had kernel 5.19.x before, and now have 6.0.x. If in doubt, check /var/log/zypp/history for the date of kernel updates.) I see that a recent kernel update (probably 6.0.x, installed around 2022-10-07) got a new cache hash (which means it has support for new AppArmor features), which also means a fresh cache file in /var/cache/apparmor/$hash/ will be created. But - only if a profile or include file is newer than the precompiled cache. With your timestamp from last year, that would not have been the case, and the precompiled cache might have been used.
I checked and you are correct. When I was running 20220908 I was using kernel 5.19.7-1.1 and after updating to 20221101 I was using kernel 6.0.5-1.1.
I already wanted to ask for a backup of /var/cache/apparmor/ in the broken state, but then checked on my own system (I have samba installed, but only start/use it every few months.)
# ls -l /usr/share/apparmor/cache/201d1af9.0/*smbd \ /var/cache/apparmor/*/*smbd [...] 4. Nov 18:23 /usr/share/apparmor/cache/201d1af9.0/usr.sbin.smbd [...] 7. Okt 21:40 /var/cache/apparmor/3478577f.0/usr.sbin.smbd
Note that there is no /var/cache/apparmor/201d1af9.0/usr.sbin.smbd - my autogenerated profile sniplet is too old: [...] 13. Apr 2022 /etc/apparmor.d/local/usr.sbin.smbd-shares and therefore /usr/share/apparmor/cache/201d1af9.0/usr.sbin.smbd is newer and looks "valid enough" :-/
OK, so we have an explanation.
I guess that's one more reason to remind upstream to move "use a checksum of the /etc/apparmor.d/ files when validating the cache" more to the top of the TODO list.
Here are the results from my system # ls -l /usr/share/apparmor/cache/201d1af9.0/*smbd /var/cache/apparmor/*/*smbd -rw-r--r-- 1 root root 45K Oct 10 15:02 /usr/share/apparmor/cache/201d1af9.0/usr.sbin.smbd -rw------- 1 root root 45K Nov 3 22:41 /var/cache/apparmor/201d1af9.0/usr.sbin.smbd -rw------- 1 root root 42K Jul 22 2021 /var/cache/apparmor/2cfa59e0.0/usr.sbin.smbd -rw------- 1 root root 45K Oct 10 15:02 /var/cache/apparmor/3478577f.0/usr.sbin.smbd -rw------- 1 root root 43K Nov 16 2021 /var/cache/apparmor/b4df858c.0/usr.sbin.smbd
Unless I overlooked something, the DENIED lines were for two files: I did not give you all the denied lines but that had access denied.
Looking at the log I can break the denied lines into 3 groups
1) /var/lib/nscd/netgroup 2) files on smb shares which I have setup 3) lines which are for samba-dcerpcd
Here are some examples of the samba-dcerpcd log lines:
type=AVC msg=audit(1662039375.496:2612): apparmor="DENIED" operation="file_inherit" profile="samba-dcerpcd" name=2F766F6C2F672F4465736B746F702F53686F7720746F2044656E6973652F
In case you wonder - that cryptic name= can be decoded with aa-decode. In this case, it's a directory starting with /vol/g/ and including a space (which is the reason why it was encoded) - so it sums up with "file on the shares".
Thanks for the info on aa-decode. I also found ausearch -i will convert the audit timestamps to human readable.
If the only new log events you still get are for /var/lib/nscd/netgroup, a mail reply is enough. If you get more/different log lines, please open a bugreport and attach your audit.log.
Correct no denied errors since I restart the services.
Does that mean you also don't get denials for /var/lib/nscd/netgroup ? If not, maybe it was related to the strange things you've seen, and we don't need to allow it?
No, it still has DENIED for /var/lib/nscd/netgroup. None for today 11/15/2022 but here are the ones for the last 3 days type=AVC msg=audit(11/12/22 14:40:33.855:2363) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=5876 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/12/22 15:45:40.592:2368) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=8010 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/12/22 16:02:46.111:2509) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=9657 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/12/22 16:09:05.773:2510) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=11716 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/12/22 16:10:02.687:2511) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=12305 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/12/22 16:17:35.934:2512) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=13866 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/12/22 16:18:21.973:2513) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=14018 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/12/22 16:37:19.127:2531) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=16458 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/12/22 16:40:38.650:2532) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=16626 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/12/22 16:44:11.904:2535) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=16840 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/12/22 16:47:22.615:2536) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=17500 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/13/22 10:51:01.809:2760) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=12575 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/13/22 11:18:16.630:2778) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=14627 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/13/22 14:56:43.959:2844) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=1606 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/14/22 10:09:35.969:3021) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=23281 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root type=AVC msg=audit(11/14/22 16:39:15.600:3090) : apparmor=DENIED operation=file_receive profile=smbd name=/var/lib/nscd/netgroup pid=14218 comm=smbd requested_mask=r denied_mask=r fsuid=root ouid=root
So do we have an explanation as to what occurred? Seems odd that the files compare and that the only change is that the smb.conf got a new timestamp causing apparmor to reload the both files still compared to before the update.
So I guess if it happened again, I would just touch smb.conf causing apparmor to reload and then it would work again?
Right. And before doing that, please save /etc/apparmor.d/, /usr/share/apparmor/cache/ and/var/cache/apparmor/ as a tar.gz so that we can easily check for differences and timestamps.
Ok, thanks, I've made a note of that.