On 7/26/19 1:20 PM, Michal Suchánek wrote:

On Fri, 26 Jul 2019 11:56:47 -0500 Larry Finger wrote:

...

What is the magic incantation needed for the VirtualBox spec file so that the kernel modules in the virtualbox-host-kmp package will be built when the kernel changes?

I thought this was being done correctly, but situations such as noted in boo#1142995 keep popping up.

There is a magic incantation in kernel-default-base that reportedly works for Factory. For Leap release repos you need to arrange something with Maintenanace because released packages are not rebuilt otherwise. Technically it should not be required for the release repos, either.

Actually, the failure noted above was for Tumbleweed. Larry -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Fri, 26 Jul 2019 13:55:34 -0500 Larry Finger wrote:

On 7/26/19 1:20 PM, Michal Suchánek wrote:

...

On Fri, 26 Jul 2019 11:56:47 -0500 Larry Finger wrote:

...

What is the magic incantation needed for the VirtualBox spec file so that the kernel modules in the virtualbox-host-kmp package will be built when the kernel changes?

I thought this was being done correctly, but situations such as noted in boo#1142995 keep popping up.

There is a magic incantation in kernel-default-base that reportedly works for Factory. For Leap release repos you need to arrange something with Maintenanace because released packages are not rebuilt otherwise. Technically it should not be required for the release repos, either.

Actually, the failure noted above was for Tumbleweed.

In Factory Virtualbox KMPs should already depend on the kernel version they were built against and become uninstallable when the kernel changes. However, it is not OBS itself checking this rebuild condition. There is a bot somewhere which checks this and rebuilds the packages from time to time. Apparently you can get broken packages released or completely broken Factory due to bad timing. Having a snapshot always checked for uninstallable packages before it is released should be doable, though. Thanks Michal -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Sat, 27 Jul 2019 12:22:20 +0200 Dominique Leuenberger / DimStar wrote:

On Sat, 2019-07-27 at 11:36 +0200, Michal Suchánek wrote:

...

In Factory Virtualbox KMPs should already depend on the kernel version they were built against and become uninstallable when the kernel changes. However, it is not OBS itself checking this rebuild condition. There is a bot somewhere which checks this and rebuilds the packages from time to time. Apparently you can get broken packages released or completely broken Factory due to bad timing. Having a snapshot always checked for uninstallable packages before it is released should be doable, though

Everything is 'doable' - the question is if anybody is willing to pay the price.

e.g. I hear statements like: * No package should ever fail to bvuild * No package should ever be uninstallable * Everything should always work

Juyst to make some coutner statements: * Currently, there are 244 build failures in TW (~ 2% of all packages). Some fairly recent, Some have been failing for as long as 8 months. Shall we *really* block all future snapshots until ALL build fails are fixed? [0]

Build failed and rebuild not attempted are two quite different issues.

* The installcheck report has 32 'unique' packages listed as uninstallable (~0.05%, there are ~ 70k binary packages). Many of them are listed for a long time, one I checked for example received a comment by the bot on the package in Oct 2017 - nothing changed. Shall I block all future snapshots until this is resolved? [1]

If it were fixed by simply triggering a rebuild of the packages, sure.

As for the actual problem with Virtualbox: There is NO indication that it would not be installbale on kernel 5.2.1 in the RPM meta:

...

rpm -qR virtualbox-host-kmp-default | sort -u /bin/sh coreutils grep kernel-default rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(PayloadIsXz) <= 5.2-1

And this is the problem: AFAIK the KMPs in Factory should depend on exact kernel version which is not the case here. So the automation cannot know that Virtualbox needs rebuild. If this is specific to Virtualbox then it should be fixed by using the KMP macros properly. If this is an issue with the kernel macros affecting all KMPs report a bug against kernel. Thanks Michal -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Am 27.07.19 um 13:55 schrieb Michal Suchánek:

On Sat, 27 Jul 2019 11:36:36 +0200 Michal Suchánek wrote:

...

On Fri, 26 Jul 2019 13:55:34 -0500 Larry Finger wrote:

...

On 7/26/19 1:20 PM, Michal Suchánek wrote:

...

On Fri, 26 Jul 2019 11:56:47 -0500 Larry Finger wrote:

...

What is the magic incantation needed for the VirtualBox spec file so that the kernel modules in the virtualbox-host-kmp package will be built when the kernel changes?

I thought this was being done correctly, but situations such as noted in boo#1142995 keep popping up.

There is a magic incantation in kernel-default-base that reportedly works for Factory. For Leap release repos you need to arrange something with Maintenanace because released packages are not rebuilt otherwise. Technically it should not be required for the release repos, either.

Actually, the failure noted above was for Tumbleweed.

In Factory Virtualbox KMPs should already depend on the kernel version they were built against and become uninstallable when the kernel changes. However, it is not OBS itself checking this rebuild condition. There is a bot somewhere which checks this and rebuilds the packages from time to time. Apparently you can get broken packages released or completely broken Factory due to bad timing. Having a snapshot always checked for uninstallable packages before it is released should be doable, though.

And it is done but Virtualbox has been the single KMP that does not have the dependency that triggers the rebuild. This looks like a bug in Virtualbox itself. There are KMP macros provided by kernel that add dependency on kernel version in Factory/TW and kernel symbols on Leap.

The problem AFAICS is, that virtualbox provides two KMP packages, for host and guest. The macros cannot handle that from what I can see (but I might be not reading them correctly). OTOH it looks totally unnecessary to split the KMPs for host and guest: host modules are loaded manually by a service before the vbox service is started. They will not be loaded in a vbox guest anyway. Right now, virtualbox-host-kmp conflicts with virtualox-guest-kmp to enforce "no stupid config", but this could be solved the same by conflictiong virtualbox with virtualbox-guest-tools and having a common virtualbox-kmp package which contains host and guest modules. Size is also not really an issue, because each kmp package is ~850kb installed size. Larry, would you be interested in a submitrequest that changes the package to one common kmp package using the %kernel_module_package macro and shifts around the conflicts in the userspace packages? Best regards, Stefan -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Sat, 27 Jul 2019 16:15:13 +0200 Michal Suchánek wrote:

On Sat, 27 Jul 2019 14:57:43 +0200 Stefan Seyfried wrote:

...

Am 27.07.19 um 13:55 schrieb Michal Suchánek:

...

On Sat, 27 Jul 2019 11:36:36 +0200 Michal Suchánek wrote:

...

On Fri, 26 Jul 2019 13:55:34 -0500 Larry Finger wrote:

...

On 7/26/19 1:20 PM, Michal Suchánek wrote:

...

On Fri, 26 Jul 2019 11:56:47 -0500 Larry Finger wrote:

> What is the magic incantation needed for the VirtualBox spec file so that the > kernel modules in the virtualbox-host-kmp package will be built when the kernel > changes? > > I thought this was being done correctly, but situations such as noted in > boo#1142995 keep popping up.

There is a magic incantation in kernel-default-base that reportedly works for Factory. For Leap release repos you need to arrange something with Maintenanace because released packages are not rebuilt otherwise. Technically it should not be required for the release repos, either.

Actually, the failure noted above was for Tumbleweed.

In Factory Virtualbox KMPs should already depend on the kernel version they were built against and become uninstallable when the kernel changes. However, it is not OBS itself checking this rebuild condition. There is a bot somewhere which checks this and rebuilds the packages from time to time. Apparently you can get broken packages released or completely broken Factory due to bad timing. Having a snapshot always checked for uninstallable packages before it is released should be doable, though.

And it is done but Virtualbox has been the single KMP that does not have the dependency that triggers the rebuild. This looks like a bug in Virtualbox itself. There are KMP macros provided by kernel that add dependency on kernel version in Factory/TW and kernel symbols on Leap.

The problem AFAICS is, that virtualbox provides two KMP packages, for host and guest. The macros cannot handle that from what I can see (but I might be not reading them correctly).

Then just split it into multiple specfiles. There are actually two ways to do it: one with multiple spec files and package links, another with multibuild. Either way will simplify your speci file significantly leading to easier maintenance in the long run.

Clearly the KMP macros are not used by Vbox because it is installed in different location:

/lib/modules/5.1.16-1-default /lib/modules/5.1.16-1-default/extra /lib/modules/5.1.16-1-default/extra/vboxdrv.ko /lib/modules/5.1.16-1-default/extra/vboxnetadp.ko /lib/modules/5.1.16-1-default/extra/vboxnetflt.ko /lib/modules/5.1.16-1-default/extra/vboxpci.ko /lib/modules/5.2.1-1-default /lib/modules/5.2.1-1-default/updates /lib/modules/5.2.1-1-default/updates/crash.ko

That aside the dependency on the current kernel version should be added to any packages that ship kernel modules regardless of the macro used so long as they are recognized by the rpm fint-requires.ksyms script. Clearly the dependency was added neither to vbox nor other KMPs:

rpm -q --requires virtualbox-host-kmp-default-6.0.8_k5.1.16_1-3.2.x86_64 ; echo --- ; rpm -q --requires crash-kmp-default-7.2.6_k5.2.1_1-191.14.x86_64 /bin/sh /bin/sh /bin/sh /bin/sh coreutils grep kernel-default rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(PayloadIsXz) <= 5.2-1 --- /bin/sh /bin/sh /bin/sh /bin/sh coreutils grep kernel-default rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(PayloadIsXz) <= 5.2-1

Which begs the question why is crash rebuilt but Virtualbox not.

Patches to kernel and rpm-config-SUSE to address this for crash are queued. It should work even for VIrtualbox. Thanks Michal -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Sat, Jul 27, 2019 at 1:10 PM Larry Finger wrote:

On 7/27/19 9:57 AM, Michal Suchánek wrote:

...

On Sat, 27 Jul 2019 16:15:13 +0200 Michal Suchánek wrote:

...

On Sat, 27 Jul 2019 14:57:43 +0200

--snip--

...

...

Which begs the question why is crash rebuilt but Virtualbox not.

Patches to kernel and rpm-config-SUSE to address this for crash are queued. It should work even for VIrtualbox.

If those patches work for VirtualBox, that would be great.

Thanks to everyone that contributed to the discussion.

It seems think several answers to the problem have been proposed, but it will take a while to digest all the information.

At first glance, Stefan Seyfried's suggestion to merge the host and guest kmp packages into a single unit seems intriguing. I will explore that locally to see if there are any downsides to this change.

The upstream kernel has included the VirtualBox guest kmods for a while now, so those shouldn't even be *built* anymore in a kmp. -- 真実はいつも一つ！/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On Sat, 27 Jul 2019 12:58:32 -0500 Larry Finger wrote:

On 7/27/19 12:17 PM, Neal Gompa wrote:

...

On Sat, Jul 27, 2019 at 1:10 PM Larry Finger wrote:

...

On 7/27/19 9:57 AM, Michal Suchánek wrote:

...

On Sat, 27 Jul 2019 16:15:13 +0200 Michal Suchánek wrote:

...

On Sat, 27 Jul 2019 14:57:43 +0200

--snip--

...

...

Which begs the question why is crash rebuilt but Virtualbox not.

Patches to kernel and rpm-config-SUSE to address this for crash are queued. It should work even for VIrtualbox.

If those patches work for VirtualBox, that would be great.

Thanks to everyone that contributed to the discussion.

It seems think several answers to the problem have been proposed, but it will take a while to digest all the information.

At first glance, Stefan Seyfried's suggestion to merge the host and guest kmp packages into a single unit seems intriguing. I will explore that locally to see if there are any downsides to this change.

The upstream kernel has included the VirtualBox guest kmods for a while now, so those shouldn't even be *built* anymore in a kmp.

They are not for Tumbleweed. The old kernels in Leap still need the external modules. Unfortunately, the module for mounting host file systems, vboxsf, is not yet in any kernel.

It does not make any sense to have both host and guest modules, and either can potentially cause issues. The host modules are closely tied to Virtualbox itself but building guest modules as a completely separate package should not cause any problems. That is what you have to do for other OS and distribution guests anyway. Thanks Michal -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Am Samstag, 27. Juli 2019, 19:10:19 CEST schrieb Larry Finger:

On 7/27/19 9:57 AM, Michal Suchánek wrote:

...

On Sat, 27 Jul 2019 16:15:13 +0200

Michal Suchánek wrote:

...

On Sat, 27 Jul 2019 14:57:43 +0200

--snip--

At first glance, Stefan Seyfried's suggestion to merge the host and guest kmp packages into a single unit seems intriguing.

Well, from a power virtualbox user POV, this approach feels wrong. (Sorry, Stefan) After all, kernel modules always contain very delicate code. Having both, the guest modules as well as the host modules installed everywhere, whether either host or guest functionality is requested, may open a couple of new issues. What happens, if, for some reason, a mix of these modules is loaded? Any chance, they interfere with each other? DoS.. Especially, the vboxfs module is one, that I don't want to see loaded on any host, that I depend on. To the contrary, I already thought of changes to make this module optional. For the record, I usually add a zypper lock for the guest modules on vbox hosts.

I will explore that locally to see if there are any downsides to this change.

Let's see, how Michals changes to the macros behave. Given the sheer size of virtualbox.spec today, having separate host and guest kmod specs with a simple automated update script sounds like the most attractive approach to me. Cheers, Pete -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 7/28/19 4:29 AM, Hans-Peter Jansen wrote:

Am Samstag, 27. Juli 2019, 19:10:19 CEST schrieb Larry Finger:

...

On 7/27/19 9:57 AM, Michal Suchánek wrote:

...

On Sat, 27 Jul 2019 16:15:13 +0200

Michal Suchánek wrote:

...

On Sat, 27 Jul 2019 14:57:43 +0200

--snip--

At first glance, Stefan Seyfried's suggestion to merge the host and guest kmp packages into a single unit seems intriguing.

Well, from a power virtualbox user POV, this approach feels wrong. (Sorry, Stefan) After all, kernel modules always contain very delicate code.

Having both, the guest modules as well as the host modules installed everywhere, whether either host or guest functionality is requested, may open a couple of new issues. What happens, if, for some reason, a mix of these modules is loaded? Any chance, they interfere with each other? DoS.. Especially, the vboxfs module is one, that I don't want to see loaded on any host, that I depend on. To the contrary, I already thought of changes to make this module optional.

For the record, I usually add a zypper lock for the guest modules on vbox hosts.

...

I will explore that locally to see if there are any downsides to this change.

Let's see, how Michals changes to the macros behave.

Given the sheer size of virtualbox.spec today, having separate host and guest kmod specs with a simple automated update script sounds like the most attractive approach to me.

Pete, For Tumbleweed users, vboxvideo has been part of every kernel for several cycles without any complaints from users. As Stephan notes, it refuses to load. Although vboxguest is also a part of the kernel, we still need to build it as the kernel's version has different symbols, thus vboxsf requires the out-of-kernel version. Adding vboxguest and vboxsf to a single kmp package would add only a few MB to the download and total storage required, thus it should not be a system breaker. In fact, Oracle delivers both kinds of modules and everything else in a single RPM. I would argue that the size and complexity of the virtualbox spec argues against having separate kmod specs. The changes required to combine the two kinds of modules is trivial. In addition, checking that section of the spec has resulted in the finding of several if statements that were wrong, thus we are getting some bug fixes "for free". Keeping a single package also preserves the existing update procedures. Michal's changes to the macros may fix the problem, and changing the virtualbox spec may not help, but I am willing to go ahead with changing the spec. We have also been living with some unintended conflicts that should be fixed. Larry -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 7/29/19 4:28 AM, Stefan Seyfried wrote:

Hi Larry

Am 28.07.19 um 19:43 schrieb Larry Finger:

...

For Tumbleweed users, vboxvideo has been part of every kernel for several cycles without any complaints from users. As Stephan notes, it refuses to load. Although vboxguest is also a part of the kernel, we still need to build it as the kernel's version has different symbols, thus vboxsf requires the out-of-kernel version. Adding vboxguest and vboxsf to a single kmp package would add only a few MB to the download and total storage required, thus it should not be a system breaker. In fact, Oracle delivers both kinds of modules and everything else in a single RPM.

It's only ~850kB per kmp package, so 1.7MB both together. Not worth bothering IMHO.

...

I would argue that the size and complexity of the virtualbox spec argues against having separate kmod specs. The changes required to combine the two kinds of modules is trivial. In addition, checking that section of the spec has resulted in the finding of several if statements that were wrong, thus we are getting some bug fixes "for free". Keeping a single package also preserves the existing update procedures.

Michal's changes to the macros may fix the problem, and changing the virtualbox spec may not help, but I am willing to go ahead with changing the spec. We have also been living with some unintended conflicts that should be fixed.

I did unify the kmp in home:seife:testing/virtualbox and the provides and requires (ksym...) look good, so maybe you can take it just from there.

Thanks for your help, but I made the change yesterday from host and guest kmps to a single package containing all the modules. The result installs and tests correctly on Tumbleweed hosts, TW guests, and Leap15.1 guests. I even tested creating a VM with a TW guest as the host. I would not expect anyone to do something like that, but it works! The only thing left to do is to find the place where the installer selects the guest module package when installing on a VM and change that to select the unified package. The changes were pushed upstream as request id 719647. Larry -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

On 7/29/19 11:47 PM, Stefan Seyfried wrote:

Am 29.07.19 um 17:48 schrieb Larry Finger:

...

The changes were pushed upstream as request id 719647.

I had a quick look at it, and I think your patch for the conflicts fix is not necessary, because the build system already adds a prefix to the symbols.

...

From my version:

strolchi:~ # grep RTMemFreeEx /lib/modules/5.3.0-rc1-3.gc584343-default/modules.symbols alias symbol:VBoxGuest_RTMemFreeEx vboxguest alias symbol:VBoxHost_RTMemFreeEx vboxdrv

...but of course it also does not hurt :-)

Because there was code dealing with that conflict already in the build, I decided to keep it without testing. Now that I have everything working, there will be time to remove it. It doesn't hurt other than every openSUSE patch is another potential patch failure when Oracle updates their source. Larry -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org