[Bug 1209006] New: Document how to secureboot-sign manually built kernel modules on TW kernel >= 6.2.1
http://bugzilla.opensuse.org/show_bug.cgi?id=1209006 Bug ID: 1209006 Summary: Document how to secureboot-sign manually built kernel modules on TW kernel >= 6.2.1 Classification: openSUSE Product: openSUSE Tumbleweed Version: Current Hardware: Other OS: Other Status: NEW Severity: Normal Priority: P5 - None Component: Kernel Assignee: kernel-bugs@opensuse.org Reporter: sndirsch@suse.com QA Contact: qa-bugs@suse.de Found By: --- Blocker: --- Now that we added lockdown patches to our kernel with version 6.2.1 breaking nvidia driver packages it would be time to let our users/developers know how to secureboot-sign manually-built kernel modules (vmware, nvidia, etc.). You can see that there is a need in this mail thread on opensuse-factory ML. :-( https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/A... Documentation for vmware, that doesn't work for TW 6.2.1 kernels https://kb.vmware.com/s/article/2146460 (well, there is missing a -addext "extendedKeyUsage=codeSigning needed already for Leap kernels; so this seems to be outdated anyway) Things that are working for Leap >= 15.2 are: flavor=default privkey=$(mktemp /tmp/MOK.priv.XXXXXX) pubkeydir=/var/lib/nvidia-pubkeys pubkey=$pubkeydir/MOK-nvidia-driver-G06-525.89.02-7.1-$flavor.der # make sure creation of pubkey doesn't fail later test -d pubkeydir || mkdir -p $pubkeydir # Create a key pair (private key, public key) openssl req -new -x509 -newkey rsa:2048 \ -keyout $privkey \ -outform DER -out $pubkey -days 1000 \ -subj "/CN=Local build for nvidia-driver-G06 525.89.02 on $(date+"%Y-%m-%d")/" \ -addext "extendedKeyUsage=codeSigning" \ -nodes # Install the public key to MOK mokutil --import $pubkey --root-pw kver=6.2.1-1-default # Sign the Nvidia modules (weak-updates appears to be broken) for i in /lib/modules/$kver/updates/nvidia*.ko; do /lib/modules/$kver/build/scripts/sign-file sha256 $privkey $pubkey $i done # cleanup: private key no longer needed rm -f $privkey With the TW kernel this results in # dmesg -c > /dev/null # modprobe nvidia modprobe: ERROR: could not insert 'nvidia': Operation not permitted # dmesg [ 58.707747] lockdown_is_locked_down: 2 callbacks suppressed [ 58.707751] Lockdown: modprobe: unsigned module loading is restricted; see man kernel_lockdown.7 # modinfo nvidia filename: /lib/modules/6.2.1-1-default/updates/nvidia.ko firmware: nvidia/525.89.02/gsp_tu10x.bin firmware: nvidia/525.89.02/gsp_ad10x.bin alias: char-major-195-* version: 525.89.02 [...] Takashi asked me to look into our kernel-install-tools. I had a quick look, but I need to say I don't understand them. 1) It works with one file /path/to/certificate for both private and public key? How is this possible? We want to remove private key right after signing but keep public key on the system. 2) There is only a tool (sbtool-sign-kernel) to sign the kernel image, not the kernel modules? Or how is this supposed to work? 3) Signing of modules in KMPs is apparently not being done by the tools in this package, so maybe these are not the right tools at all? -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1209006 Stefan Dirsch <sndirsch@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jeffm@suse.com, | |jslaby@suse.com, | |martin.wilck@suse.com, | |tiwai@suse.com Summary|Document how to |Document how to |secureboot-sign manually |secureboot-sign |built kernel modules on TW |manually-built kernel |kernel >= 6.2.1 |modules on TW kernel >= | |6.2.1 -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1209006 Stefan Dirsch <sndirsch@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jlee@suse.com -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1209006 Stefan Dirsch <sndirsch@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|Normal |Critical -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1209006 http://bugzilla.opensuse.org/show_bug.cgi?id=1209006#c9 --- Comment #9 from Joey Lee <jlee@suse.com> --- (In reply to Michal Suchanek from comment #6)
On Leap we don't need the secondary keyring at all, we have a downstream patch that loads the MOK keys into platform keyring, and verifies modules with platform keyring.
The CONFIG_INTEGRITY_MACHINE_KEYRING .machine keyring needs .secondary trusted keyring.
In upstream the MOK keys are loaded into machine keyring which then should get loaded into secondary keyring.
Looks it's not. Base on v6.2 kernel, keys in .machine keyring still must be trusted(signed) by key in built-in/secondary keyring. It applies restrict_link_to_ima and depends on CONFIG_IMA_KEYRINGS_PERMIT_SIGNED_BY_BUILTIN_OR_SECONDARY. If the key can not pass the restriction check, kernel will put to key to .platform keyring. The logic is in add_to_machine_keyring(), but I didn't test it yet. So, currently the .machine keyring is NOT useful for loading a MOK-signed kernel module. I think all MOKs will be loaded to .platform keyring. Unless Eric Snowberg's MOK CA patch be accepted: Eric Snowberg <eric.snowberg@oracle.com> [PATCH v4 0/6] Add CA enforcement keyring restrictions https://lore.kernel.org/lkml/20230207025958.974056-1-eric.snowberg@oracle.co... [PATCH v4 6/6] integrity: machine keyring CA configuration
This option is for loading additional keys (which the name does not reflect) in a specific way different from the default (which the name does not reflect) and is incompatible with machine keyring (which would have to be patched out to enable it).
Does not seem to work for me, anyway:
cat /proc/keys | grep machine 31db47ec I------ 1 perm 1f0b0000 0 0 keyring .machine: empty
Kernel will check the existence of MokListTrustedRT. So we need to run "mokutil --trust-mok", otherwise all MOK will go to .platform keyring. But, the above tracing, unless the MOK be signed by built-in key. Then it still go to .platform keyring. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1209006 http://bugzilla.opensuse.org/show_bug.cgi?id=1209006#c10 --- Comment #10 from Joey Lee <jlee@suse.com> --- (In reply to Michal Suchanek from comment #6)
On Leap we don't need the secondary keyring at all, we have a downstream patch that loads the MOK keys into platform keyring, and verifies modules with platform keyring.
On the other hand, as Michal mentioned, SLE/Leap applied downstream patch: patches.suse/KEYS-Make-use-of-platform-keyring-for-module-signatu.patch This patch allows modsign function mod_verify_sig() uses .platform keyring to verify kernel module. We didn't have this patch in openSUSE Tumbleweed yet. I have checked the following kernels from different distros, all of them are applied the same downstream patch: Ubuntu ubuntu-stable-kinetic no CONFIG_INTEGRITY_MACHINE_KEYRING CentOS Stream 9 kernel linux-5.14.0-214.el9.x86_64 no CONFIG_INTEGRITY_MACHINE_KEYRING Fedora 37 kernel-6.2.2-301.fc38.src.rpm CONFIG_INTEGRITY_MACHINE_KEYRING=y Fedora 37 has set CONFIG_INTEGRITY_MACHINE_KEYRING, but it still applied downstream patch to allow .platform keyring be used to verify kernel modules. We can also apply this patch to openSUSE Tumbleweed, but this patch may never be accepted by kernel upstream. Because in Eric Snowberg's "Add CA enforcement keyring restrictions" solution, only CA in MOK can be put to .secondary keyring. Other non-CA MOKs still can not be used to verify kernel module. User needs to use keyctl to add it to .secondary keyring after the CA MOK be added. If we don't apply downstream patch and choice waiting Eric's solution is accepted. Then the only solution is disabling secure boot currently. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1209006 http://bugzilla.opensuse.org/show_bug.cgi?id=1209006#c12 --- Comment #12 from Joey Lee <jlee@suse.com> --- (In reply to Stefan Dirsch from comment #11)
Tried "mokutil --trust-mok". After enrolling. Didn't help. Deleted the key from Mok. Then re-enrolling it again. Didn't help either.
I am porting patches.suse/KEYS-Make-use-of-platform-keyring-for-module-signatu.patch to openSUSE Tumbleweed kernel. We need it for Nvidia driver. But we will need to change the rpm scripts of Nvidia driver again when Eric Snowberg's patch set be merged on mainline if we want to follow upstream's approach. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1209006 Darren Davis <ddavis@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ddavis@suse.com -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1209006 http://bugzilla.opensuse.org/show_bug.cgi?id=1209006#c18 --- Comment #18 from Joey Lee <jlee@suse.com> --- Hi Martin, (In reply to Martin Wilck from comment #16)
(In reply to Martin Wilck from comment #15)
(In reply to Joey Lee from comment #9)
Base on v6.2 kernel, keys in .machine keyring still must be trusted(signed) by key in built-in/secondary keyring. It applies restrict_link_to_ima and depends on CONFIG_IMA_KEYRINGS_PERMIT_SIGNED_BY_BUILTIN_OR_SECONDARY.
[...]
So keys could be added from the machine keyring to the secondary keyring without being trusted by the secondary keyring beforehand, but such keys could never have been added to the machine keyring in the first place.
That looks like an upstream bug to me.
Sorry, no. I was wrong.
https://elixir.bootlin.com/linux/v6.2/source/security/integrity/digsig.c#L13... shows that there is no restriction at all for keys in the machine and platform key rings.
But now I fail to see why MoK keys don't make it into the secondary keyring...
You are right! I missed the above code when tracing. I have tested .machine keyring and confirmed that .machine keyring be linked to .secondary keyring and can be used to verify kernel module. First we need run "mokutil --trust-mok" command to request shim to create MokListTrustedRT for linking .machine keyring to .secondary keying. Unfortunately we need shim-15.5 to support MokListTrustedRT, but we only have Microsoft signed shim-15.4 now. Our shim-15.6 and shim 15.7 is waiting shim usptream review and Microsoft signing. If anyone wants to try shim-15.7, it is in openSUSE:Factory:secure-boot repo: https://build.opensuse.org/package/show/openSUSE:Factory:secure-boot/shim You will need to enroll openSUSE CA or signkey to UEFI db. Then you can play shim-15.7. After shim created MokListTrustedRT and boot success, keyctl shows: Vigor135:~ # keyctl list %:.platform 6 keys in keyring: 1067995450: ---lswrv 0 0 asymmetric: Microsoft Windows Production PCA 2011: a92902398e16c49778cd90f99e4f9ae17c55af53 22887073: ---lswrv 0 0 asymmetric: Build time autogenerated kernel key: fc6de3c630f48b63df52f138903ce4d57b4912cf 261332100: ---lswrv 0 0 asymmetric: Build time autogenerated kernel key: ec9a8cdfe96d577ab849ff3e58ee0d84b17d9745 174403700: ---lswrv 0 0 asymmetric: Build time autogenerated kernel key: e9454afc180d18db2c0311d43f6fe8dd65cb96e8 690509329: ---lswrv 0 0 asymmetric: Build time autogenerated kernel key: af96d2c6c40d1afc8b175658d0536fe4514e7501 351948099: ---lswrv 0 0 asymmetric: Microsoft Corporation UEFI CA 2011: 13adbf4309bd82709c8cd54f316ed522988a1bd4 Vigor135:~ # keyctl list %:.machine 6 keys in keyring: 531558656: ---lswrv 0 0 asymmetric: openSUSE Secure Boot CA: 6842600de22c4c477e95be23dfea9513e5971762 905231616: ---lswrv 0 0 asymmetric: openSUSE Secure Boot Signkey: fd9f2c12e599d67cc7f9067541adf426b712469e 113219: ---lswrv 0 0 asymmetric: Kernel OBS Project: 1fb41512acbc8eebdf828d877e4367bf6c719af3 772042873: ---lswrv 0 0 asymmetric: Build time autogenerated kernel key: 959b5e1b135628040e10b510443790b327f6615d 529014248: ---lswrv 0 0 asymmetric: Build time autogenerated kernel key: 55f24d5835fbedf68734252328b769eb29ddfe97 775007952: ---lswrv 0 0 asymmetric: Build time autogenerated kernel key: e9454afc180d18db2c0311d43f6fe8dd65cb96e8 So, all keys in mok unconditionally be added to .machine keyring. And keyring be linked to .secondary keyring: Vigor135:~ # keyctl list %:.secondary_trusted_keys 2 keys in keyring: 236292887: ---lswrv 0 0 keyring: .builtin_trusted_keys 900602128: ---lswrv 0 0 keyring: .machine I have boot to self-built kernel and loaded a openSUSE signkey signed module success. So, I want to fix my comment#9, the .machine keyring works to verify kernel module. The problem is that we need Microsoft signs back openSUSE shim-15.6 or shim-15.7. Otherwise we still need downstream patch KEYS-Make-use-of-platform-keyring-for-module-signatu.patch. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1209006 http://bugzilla.opensuse.org/show_bug.cgi?id=1209006#c21 Joey Lee <jlee@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(jlee@suse.com) | --- Comment #21 from Joey Lee <jlee@suse.com> --- For reference, kernel upstream's plan of .platform and .machine keyrings is here: keyrings, key usage, and trust models https://lore.kernel.org/all/20220928055900.GT4909@linux-l9pv.suse/t/#m3ce7e4... And, a PDF slides. Those pictures may be useful: https://static.sched.com/hosted_files/lssna2022/18/LSS%202022%20trust%20and%... Newest patch set: [PATCH v5 0/6] Add CA enforcement keyring restrictions https://lore.kernel.org/lkml/20230302164652.83571-1-eric.snowberg@oracle.com... Per my understood, "keys in UEFI db" only be trusted to verify booting/kexec. And MOKs also can be used to verify booting/kexec. CA MOKs can be used to verify keys for .ima keyring. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1209006 http://bugzilla.opensuse.org/show_bug.cgi?id=1209006#c30 --- Comment #30 from Joey Lee <jlee@suse.com> --- (In reply to Martin Wilck from comment #24)
From https://lore.kernel.org/all/20220928055900.GT4909@linux-l9pv.suse/t/ #m3ce7e451f1855d9c432965bb896cb7ce0f89e009:
The end-user will now need to enroll two keys. First the CA Key into the MOK and then the leaf cert into the secondary trusted keyring.
HOW would the user add this leaf cert? I am not getting it. From the PDF slides I got the impression that the "Root user" would play a role here, but I couldn't figure out how this would work, in particular how the root user's cert would be added to the kernel.
As Michal mentioned, using keyctl. And using keyctl with dracut in initrd to add leaf cert to .ima/.evm or even .secondary_trusted_keys.
Wrt the general mind set, I agree with Michal.
I agree that the upstream solution is complex, and I do not fully understood the Imputed or Transitive concept in their theory. But I think that those certificates in db/mok must be differentiated based on functionality purpose. The trust is not spread from a purpose to other purposes. Using usage extension in certificates to separate different purposes is a strategy. IMA maintainer uses CA in BasicConstraints, digitalSignature and keyCertSign to identify CA MOK. And NIAP PPOS certification uses codeSign extend key usage.
IMO the biggest issue is that IMA is disabled as soon as a .machine keyring is populated. That looks like a political movement of upstream with the intention to deter people from using MoK (and thus, 3rd party modules).
I have applied Eric Snowberg's patch set on v6.2 kernel and tested. I used the following kernel config, and I got a kernel behavior that is similar with our downstream patch for modsign: CONFIG_INTEGRITY_MACHINE_KEYRING=y <-- enable .machine keyring CONFIG_INTEGRITY_CA_MACHINE_KEYRING=n <-- allow all MOK add to .machine COFNIG_INTEGRITY_TRUSTED_KEYRING=y <-- enable .ima/.evm CONFIG_IMA_KEYRINGS_PERMIT_SIGNED_BY_BUILTIN_OR_SECONDARY=y <-- .secondary keyring trusted key can be added to .ima/.evm Enable MokListTrustedRT, looks that the MokListTrusted be set by default after shim 15.6: commit 2670c6a17edb239949152c471445fc533d8525aa Author: Eric Snowberg <eric.snowberg@oracle.com> Date: Thu Feb 17 18:29:58 2022 -0500 Allow MokListTrusted to be enabled by default Using keyctl tool shows .secondary_trusted_keys and .machine keyrings, confirmed that MOKs are added and linked: Vigor135:~ # keyctl list %:.secondary_trusted_keys 2 keys in keyring: 727581285: ---lswrv 0 0 keyring: .builtin_trusted_keys 978122164: ---lswrv 0 0 keyring: .machine <-- .machine linked Vigor135:~ # keyctl list %:.machine 7 keys in keyring: 661930310: ---lswrv 0 0 asymmetric: openSUSE Secure Boot CA: 6842600de22c4c477e95be23dfea9513e5971762 46964368: ---lswrv 0 0 asymmetric: openSUSE Secure Boot Signkey: fd9f2c12e599d67cc7f9067541adf426b712469e 119545939: ---lswrv 0 0 asymmetric: Build time autogenerated kernel key11111: 92705a4f4be67f7a90a52166573dd590711f9e32 366702073: ---lswrv 0 0 asymmetric: Kernel OBS Project: 1fb41512acbc8eebdf828d877e4367bf6c719af3 392485614: ---lswrv 0 0 asymmetric: Build time autogenerated kernel key: 959b5e1b135628040e10b510443790b327f6615d 820273393: ---lswrv 0 0 asymmetric: Build time autogenerated kernel key: 55f24d5835fbedf68734252328b769eb29ddfe97 1055853409: ---lswrv 0 0 asymmetric: Build time autogenerated kernel key: e9454afc180d18db2c0311d43f6fe8dd65cb96e8 For kernel module sign: No difference with our downstream patch, all MOKs be add to .machine keyring and link to .secondary_trusted_keys. So all MOKs can be used to verify kernel modules. For .ima/.evm All MOKs can be used to verify keys that will go to .ima/.evm keyrings. -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1209006 Petr Vorel <petr.vorel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |petr.vorel@gmail.com -- You are receiving this mail because: You are on the CC list for the bug.
http://bugzilla.opensuse.org/show_bug.cgi?id=1209006 http://bugzilla.opensuse.org/show_bug.cgi?id=1209006#c33 --- Comment #33 from Joey Lee <jlee@suse.com> --- (In reply to Michal Suchanek from comment #31)
(In reply to Joey Lee from comment #30)
I agree that the upstream solution is complex, and I do not fully understood the Imputed or Transitive concept in their theory. But I think that those certificates in db/mok must be differentiated based on functionality purpose. The trust is not spread from a purpose to other purposes.
And the x509 has purpose flags but the kernel trashes all of them except the CA flag. If it wants to differentiate purposes it needs to store those purpose flags.
In the digsig-restrictions branch in Eric Snowberg's git, looks that he is working on add restriction of digitalSignature and codeSigning EKU to MOK: https://github.com/esnowberg/linux/commits/digsig-restrictions But, looks that those restriction are only for CONFIG_INTEGRITY_CA_MACHINE_KEYRING=y. Our plan is CONFIG_INTEGRITY_CA_MACHINE_KEYRING=n. I will find time to test it.
Using usage extension in certificates to separate different purposes is a strategy. IMA maintainer uses CA in BasicConstraints, digitalSignature and keyCertSign to identify CA MOK. And NIAP PPOS certification uses codeSign extend key usage.
Which applies to both kexec and module loading, these are both code. Loaded into the same security context even. Yet kernel upstream inexplicably insists on using different keys for these making their scheme unusable for us.
If it wants to differentiate kexec and module loading, or modules by different vendors it needs to invent new extension for that. Until such certificate extension exists both kernel and modules are code.
The kexec can use .secondary and .platform keyring. And modsign can use .secondary keyring. Base on my testing on comment#30, when CONFIG_INTEGRITY_CA_MACHINE_KEYRING=n, all mok can be used to verify kernel module. Upstream's solution is adding more restrictions when using mok with .ima/.evm. And those restrictions can be turn-off. -- You are receiving this mail because: You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠suse.com