OBS Image builds using device mapper modules broken in Tumbleweed
Hi, We maintain a bunch of integration test builds in obs for testing kiwi features. There are integration tests builds also for Tumbleweed which recently stopped working when they use device mapper features as in luks or raid. Please see the following buildlogs: osc rbl Virtualization:Appliances:Images:Testing_x86:tumbleweed test-image-luks images x86_64 --- [ 289s] [ 266.952479][ T4505] device-mapper: table: 254:3: crypt: Error allocating crypto tfm (-ENOENT) --- or the raid test osc rbl Virtualization:Appliances:Images:Testing_x86:tumbleweed test-image-raid images x86_64 --- [ 90s] modprobe: FATAL: Module md_mod not found in directory /lib/modules/5.16.10-1-default --- For reference all this still works on leap e.g osc rbl Virtualization:Appliances:Images:Testing_x86:leap test-image-luks images x86_64 The testing of image features using device mapper modules or capabilities has stopped to work one or two weeks ago. I assume the kernel has changed in TW and as kernel-obs-build is just a flavor of kernel-default it received all the changes. I'm just guessing, but as of now you will not be able to create images for TW including this sort of features. It would be nice if this can be fixed in the kernel used by obs to start the workers. Neither support: kernel-obs-build nor support: kernel-default has an effect. I assume there is not much I can do to influence this ? Thanks a ton Cheers, Marcus -- Public Key available via: https://keybase.io/marcus_schaefer/key.asc keybase search marcus_schaefer ------------------------------------------------------- Marcus Schäfer Am Unterösch 9 Tel: +49 7562 905437 D-88316 Isny / Rohrdorf Germany -------------------------------------------------------
On Mär 02 2022, Marcus Schäfer wrote:
It would be nice if this can be fixed in the kernel used by obs to start the workers. Neither
support: kernel-obs-build
nor
support: kernel-default
has an effect. I assume there is not much I can do to influence this ?
Not sure what you expect here. You can build a modified kernel-obs-build in your project, and it will be picked up by OBS for use by the build container. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Hi,
Not sure what you expect here.
The expectation was that obs can still build images with dm features supported without requiring the user to rebuild an obs kernel. Basically the working state that it offered before.
You can build a modified kernel-obs-build in your project, and it will be picked up by OBS for use by the build container.
Yes this is correct, but to my understanding kernel-obs-build exists to exactly support the use cases of the buildservice and to prevent users from compiling their own kernel I mean you can argue that you don't care for my use case, that's a fair point because I'm just one uninteresting user of obs. But with the current change _no_ user of obs who wants to build let's say a luks image for Tumbleweed will be able to do it out of the box. What would happen if you delete the filesystem support from the kernel that is needed to mount the obs workers rootfs ? Everything would break down but you could solve the issue by just rebuilding the kernel adding the needed filesystem in your personal project. I assume this wouldn't be a preferred solution ? I'm just saying this is not very user friendly. It would be great if kernel-obs-build can stay stable enough to support common use cases of obs targets independently of changes to the actual distribution kernel (kernel-default). I was under the impression that this distinction exists for exactly that reason. It would cause things to just continue to work and kernel-obs-build is only used in the scope of obs. Unless image building on obs is not a valid use case I think that kernel should offer support for luks, mdadm, lvm just for usability reasons and maybe also because not every user knows how to simply rebuild kernel-obs-build and add missing stuff. Thanks much Regards, Marcus -- Public Key available via: https://keybase.io/marcus_schaefer/key.asc keybase search marcus_schaefer ------------------------------------------------------- Marcus Schäfer Am Unterösch 9 Tel: +49 7562 905437 D-88316 Isny / Rohrdorf Germany -------------------------------------------------------
On Mittwoch, 2. März 2022, 17:39:24 CET Marcus Schäfer wrote:
Hi,
We maintain a bunch of integration test builds in obs for testing kiwi features. There are integration tests builds also for Tumbleweed which recently stopped working when they use device mapper features as in luks or raid.
Please see the following buildlogs:
osc rbl Virtualization:Appliances:Images:Testing_x86:tumbleweed test-image-luks images x86_64
--- [ 289s] [ 266.952479][ T4505] device-mapper: table: 254:3: crypt: Error allocating crypto tfm (-ENOENT) ---
or the raid test
osc rbl Virtualization:Appliances:Images:Testing_x86:tumbleweed test-image-raid images x86_64
--- [ 90s] modprobe: FATAL: Module md_mod not found in directory /lib/modules/5.16.10-1-default ---
For reference all this still works on leap e.g
osc rbl Virtualization:Appliances:Images:Testing_x86:leap test-image-luks images x86_64
The testing of image features using device mapper modules or capabilities has stopped to work one or two weeks ago. I assume the kernel has changed in TW and as kernel-obs-build is just a flavor of kernel-default it received all the changes. I'm just guessing, but as of now you will not be able to create images for TW including this sort of features.
It would be nice if this can be fixed in the kernel used by obs to start the workers. Neither
support: kernel-obs-build
nor
support: kernel-default
has an effect. I assume there is not much I can do to influence this ?
you have also tried both together?
The module from kernel-default should be loadable if the kernel-obs-build kernel
matches.
Otherwise one need to debug why this isn't the case ...
--
Adrian Schroeter
Hi,
you have also tried both together?
The module from kernel-default should be loadable if the kernel-obs-build kernel matches.
Otherwise one need to debug why this isn't the case ...
yes I also tried both together. This is the current setting: https://build.opensuse.org/projects/Virtualization:Appliances:Images:Testing... Thanks Cheers, Marcus -- Public Key available via: https://keybase.io/marcus_schaefer/key.asc keybase search marcus_schaefer ------------------------------------------------------- Marcus Schäfer Am Unterösch 9 Tel: +49 7562 905437 D-88316 Isny / Rohrdorf Germany -------------------------------------------------------
On Mär 03 2022, Marcus Schäfer wrote:
yes I also tried both together. This is the current setting:
https://build.opensuse.org/projects/Virtualization:Appliances:Images:Testing...
You haven't trigger a rebuild for test-image-raid yet (support packages don't count as a dependency). -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Hi,
yes I also tried both together. This is the current setting:
https://build.opensuse.org/projects/Virtualization:Appliances:Images:Testing...
You haven't trigger a rebuild for test-image-raid yet (support packages don't count as a dependency).
ok, triggered the rebuild with "osc rebuild --all" but still facing the same issue: [ 192s] modprobe: FATAL: Module md_mod not found in directory /lib/modules/5.16.10-1-default Thanks Cheers, Marcus -- Public Key available via: https://keybase.io/marcus_schaefer/key.asc keybase search marcus_schaefer ------------------------------------------------------- Marcus Schäfer Am Unterösch 9 Tel: +49 7562 905437 D-88316 Isny / Rohrdorf Germany -------------------------------------------------------
On Mär 03 2022, Marcus Schäfer wrote:
ok, triggered the rebuild with "osc rebuild --all" but still facing the same issue:
[ 192s] modprobe: FATAL: Module md_mod not found in directory /lib/modules/5.16.10-1-default
For some reason, kernel-obs-build is out of date. The version in http://download.opensuse.org/tumbleweed/repo/oss/x86_64/ is 5.16.11, but openSUSE:Tumbleweed/standard contains 5.16.10. $ osc buildinfo Virtualization:Appliances:Images:Testing_x86:tumbleweed test-image-raid images x86_64 | grep kernel- <bdep name="kernel-default" noinstall="1" version="5.16.11" release="1.1" arch="x86_64" hdrmd5="bf9c7358fa8ad1336026074aa6767a5e" project="openSUSE:Tumbleweed" repository="dod"/> <bdep name="kernel-obs-build" vminstall="1" notmeta="1" version="5.16.10" release="1.1" arch="x86_64" hdrmd5="86c3a20fd539aa3c3e998890828420f4" project="openSUSE:Tumbleweed" repository="standard"/> <bdep name="kernel-default" notmeta="1" version="5.16.11" release="1.1" arch="x86_64" hdrmd5="bf9c7358fa8ad1336026074aa6767a5e" project="openSUSE:Tumbleweed" repository="dod"/> -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Am 04.03.22 um 00:29 schrieb Andreas Schwab:
On Mär 03 2022, Marcus Schäfer wrote:
ok, triggered the rebuild with "osc rebuild --all" but still facing the same issue:
[ 192s] modprobe: FATAL: Module md_mod not found in directory /lib/modules/5.16.10-1-default For some reason, kernel-obs-build is out of date. The version in http://download.opensuse.org/tumbleweed/repo/oss/x86_64/ is 5.16.11, but openSUSE:Tumbleweed/standard contains 5.16.10.
$ osc buildinfo Virtualization:Appliances:Images:Testing_x86:tumbleweed test-image-raid images x86_64 | grep kernel- <bdep name="kernel-default" noinstall="1" version="5.16.11" release="1.1" arch="x86_64" hdrmd5="bf9c7358fa8ad1336026074aa6767a5e" project="openSUSE:Tumbleweed" repository="dod"/> <bdep name="kernel-obs-build" vminstall="1" notmeta="1" version="5.16.10" release="1.1" arch="x86_64" hdrmd5="86c3a20fd539aa3c3e998890828420f4" project="openSUSE:Tumbleweed" repository="standard"/> <bdep name="kernel-default" notmeta="1" version="5.16.11" release="1.1" arch="x86_64" hdrmd5="bf9c7358fa8ad1336026074aa6767a5e" project="openSUSE:Tumbleweed" repository="dod"/>
What's the purpose of openSUSE:Tumbleweed/kernel-obs-build anyway? Greetings, Stephan
What's the purpose of openSUSE:Tumbleweed/kernel-obs-build anyway? It's an aggregate that copies from dod to standard that looks 'scheduled' for more
Am 04.03.22 um 07:31 schrieb Stephan Kulow: than a week. I don't know what's going on there, but I could wipe the old binary. Now we will find out what the purpose is :) Greetings, Stephan
On Freitag, 4. März 2022, 07:44:01 CET Stephan Kulow wrote:
What's the purpose of openSUSE:Tumbleweed/kernel-obs-build anyway? It's an aggregate that copies from dod to standard that looks 'scheduled' for more
Am 04.03.22 um 07:31 schrieb Stephan Kulow: than a week. I don't know what's going on there, but I could wipe the old binary. Now we will find out what the purpose is :)
we need the aggregate to let the exportfilters working.
eg. x86_64 kernel is also used on i586 these days, same for aarch64 ones.
dunno exactly why it did hang, but I pushed it to succeeding now...
--
Adrian Schroeter
Hi,
What's the purpose of openSUSE:Tumbleweed/kernel-obs-build anyway? It's an aggregate that copies from dod to standard that looks 'scheduled' for more
Am 04.03.22 um 07:31 schrieb Stephan Kulow: than a week. I don't know what's going on there, but I could wipe the old binary. Now we will find out what the purpose is :)
we need the aggregate to let the exportfilters working. eg. x86_64 kernel is also used on i586 these days, same for aarch64 ones.
dunno exactly why it did hang, but I pushed it to succeeding now...
Thanks much to everybody involved in this issue. It is fixed. I triggered a rebuild and the updated kernel-obs-build was able to build all formerly failing tests Thanks much Cheers, Marcus -- Public Key available via: https://keybase.io/marcus_schaefer/key.asc keybase search marcus_schaefer ------------------------------------------------------- Marcus Schäfer Am Unterösch 9 Tel: +49 7562 905437 D-88316 Isny / Rohrdorf Germany -------------------------------------------------------
participants (4)
-
Adrian Schröter -
Andreas Schwab -
Marcus Schäfer -
Stephan Kulow