[opensuse-factory] Missing kernel modules for VirtualBox

older
[opensuse-factory] New Tumbleweed...

Larry Finger

13 Nov 2017 13 Nov '17

21:03

On 11/13/2017 12:20 PM, Dominique Leuenberger wrote: In the E-mail with subject "New Tumbleweed snapshot 20171112 released!", we see the following:

...

Packages changed: ... kernel-source (4.13.11 -> 4.13.12) ...

As noted, the kernel after updating will be 4.13.12; however, the VirtualBox kernel modules in the repos are for kernel 4.13.10. Accordingly, if you update either a Tumbleweed guest or host, the kernel modules will not be available. In a host, you will be unable to start a VirtualBox Virtual Machine. The popup window will tell you to run '/sbin/bvoxconfig'. Unfortunately, that does not work with the current source. Edit (as root) /sbin/vboxconfig, find the two lines that say echo "Reinstall virtualbox-host-source package. Quitting .." exit 1 and remove them. You may also need to 'sudo zypper in kernel-devel'. The script should now create and install the kernel modules. If not, please post the error messages. These changes to the script will soon be submitted. In a TW guest, the vboxvideo module will be available, but the module used for shared files will not. I am still working on the vboxguestconfig script to make it work in this situation. Larry -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Show replies by thread

Stefan Seyfried

13 Nov 13 Nov

22:41

Am 13.11.2017 um 21:03 schrieb Larry Finger:

...

On 11/13/2017 12:20 PM, Dominique Leuenberger wrote:

In the E-mail with subject "New Tumbleweed snapshot 20171112 released!", we see the following:

...
Packages changed: ... kernel-source (4.13.11 -> 4.13.12) ...

As noted, the kernel after updating will be 4.13.12; however, the VirtualBox kernel modules in the repos are for kernel 4.13.10.

But the weak-modules functionality should catch this, or am I mistaken and this does not work on Factory / Tumbleweed? -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Michal Kubecek

14 Nov 14 Nov

08:33

On Monday, 13 November 2017 22:41 Stefan Seyfried wrote:

...

Am 13.11.2017 um 21:03 schrieb Larry Finger:

...
On 11/13/2017 12:20 PM, Dominique Leuenberger wrote:

In the E-mail with subject "New Tumbleweed snapshot 20171112 released!",> we see the following:

...
Packages changed: ...

...
kernel-source (4.13.11 -> 4.13.12)

...

As noted, the kernel after updating will be 4.13.12; however, the VirtualBox kernel modules in the repos are for kernel 4.13.10.

But the weak-modules functionality should catch this, or am I mistaken and this does not work on Factory / Tumbleweed?

I don't think weak-modules can work reliably in Tumbleweed where we make no attempt to preserve kABI. There is always a chance this particular update doesn't break kABI (or at least not the part relevant for virtualbox modules) but in general, you can't rely on that. Also, a KMP has an explicit dependency on kernel version/release it was built against (no idea if it's the case here). Michal Kubeček -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Larry Finger

15 Nov 15 Nov

16:15

On 11/14/2017 01:33 AM, Michal Kubecek wrote:

...

On Monday, 13 November 2017 22:41 Stefan Seyfried wrote:

...
Am 13.11.2017 um 21:03 schrieb Larry Finger:

...
On 11/13/2017 12:20 PM, Dominique Leuenberger wrote:

In the E-mail with subject "New Tumbleweed snapshot 20171112 released!",> we see the following:

...
Packages changed: ...

...
kernel-source (4.13.11 -> 4.13.12)

...

As noted, the kernel after updating will be 4.13.12; however, the VirtualBox kernel modules in the repos are for kernel 4.13.10.

But the weak-modules functionality should catch this, or am I mistaken and this does not work on Factory / Tumbleweed?

I don't think weak-modules can work reliably in Tumbleweed where we make no attempt to preserve kABI. There is always a chance this particular update doesn't break kABI (or at least not the part relevant for virtualbox modules) but in general, you can't rely on that.

Also, a KMP has an explicit dependency on kernel version/release it was built against (no idea if it's the case here).

Michal Kubeček

Michal, Thanks for that information. Yes, the KMP has such a dependency on the kernel version. Would it be a good thing to add a test for the VB KMP to openQA? At least passing the test would provide some assurance that VMs would function after an update. If such a test would be useful, could someone help with the openQA test? It could be as simple as running "modprobe vboxdrv | lsmod | grep vboxdrv" as root and checking for an output that is not empty. Unfortunately, I have no clue on how to write such a test. I have reworked file /sbin/vboxconfig so that it no longer fails if the host source package is for the wrong version. In most cases, that should work. At least a simple script should not kill a local build of the modules. Thanks, Larry -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Dominique Leuenberger / DimStar

16:26

On Wed, 2017-11-15 at 09:15 -0600, Larry Finger wrote:

...

Would it be a good thing to add a test for the VB KMP to openQA? At least passing the test would provide some assurance that VMs would function after an update. If such a test would be useful, could someone help with the openQA test? It could be as simple as running "modprobe vboxdrv | lsmod | grep vboxdrv" as root and checking for an output that is not empty. Unfortunately, I have no clue on how to write such a test.

There is currently one thing I don't understand yet: virtualbox is part of ring2, which means before a new kernel could be merged, it was guaranteed to build. Also, when looking at the build history of VBox, it had one build fail which was recovered by OBS shortly after (bad buildhost) osc jobhist openSUSE:Factory virtualbox standard x86_64 2017-10-23 19:37:56 virtualbox source change succeeded 3m 11s lamb10:3 2017-10-31 16:12:16 virtualbox source change succeeded 3m 56s lamb08:1 2017-11-04 19:08:25 virtualbox source change failed 0m 12s lamb02:7 2017-11-05 01:22:19 virtualbox retrying bad bu succeeded 2m 55s lamb57:5 2017-11-14 15:40:34 virtualbox source change succeeded 4m 13s lamb57:5 so at least from a 'build' PoV, there should never have been an issue (unless we had an issue with a rebuildtrigger on uninstallable - which is certainly possible) An openQA test might be an option - the question then would rather be if VBox is important enough to delay snapshots and for how long. Cheers Dominique

Dominique Leuenberger / DimStar

16:34

On Tue, 2017-11-14 at 08:33 +0100, Michal Kubecek wrote:

...

...
But the weak-modules functionality should catch this, or am I mistaken and this does not work on Factory / Tumbleweed?

I don't think weak-modules can work reliably in Tumbleweed where we make no attempt to preserve kABI. There is always a chance this particular update doesn't break kABI (or at least not the part relevant for virtualbox modules) but in general, you can't rely on that.

Also, a KMP has an explicit dependency on kernel version/release it was built against (no idea if it's the case here).

Actually, looking at the KMP (checked virtualbox-host-kmp-default) there is only a dependency on the kernel flavor, but not on the kernel version:

...

zypper info --requires virtualbox-host-kmp-default Information for package virtualbox-host-kmp-default:

Repository : 1112 Name : virtualbox-host-kmp-default Version : 5.1.30_k4.13.10_1-3.2 Arch : x86_64 Vendor : openSUSE Installed Size : 824.2 KiB Installed : No Status : not installed Source package : virtualbox-5.1.30-3.2.src Summary : Host kernel module for VirtualBox Description : VirtualBox is an extremely feature rich, high performance product for enterprise customers, it is also the only professional solution that is freely available as Open Source Software under the terms of the GNU Public License (GPL). Requires : [7] coreutils grep kernel-default rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(PayloadIsXz) <= 5.2-1 /bin/sh => that also explains my 'confusion' in my earlier mail about the bot not triggering it: there is simply no indication tnat this would not work with the current kernel. This seems to be an error specific to virtualbox though, since other packages I checked are different there:

...

zypper info --requires bbswitch-kmp-default Information for package bbswitch-kmp-default:

Repository : 1112 Name : bbswitch-kmp-default Version : 0.8_k4.13.12_1-6.2 Arch : x86_64 Vendor : openSUSE Installed Size : 25.3 KiB Installed : No Status : not installed Source package : bbswitch-0.8-6.2.src Summary : Bumblebee ACPI kernel module Description : bbswitch is a kernel module which automatically detects the required ACPI calls for two kinds of Optimus laptops. Requires : [7] coreutils grep rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(PayloadIsXz) <= 5.2-1 kernel-uname-r = 4.13.12-1-default ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /bin/sh If this is corrected in VBox, then the bot will see when it's nescessary to retrigger a build of virtualbox to get a new kmp built. Cheers Dominique

Michal Kubecek

17:12

On Wed, Nov 15, 2017 at 04:34:16PM +0100, Dominique Leuenberger / DimStar wrote:

...

On Tue, 2017-11-14 at 08:33 +0100, Michal Kubecek wrote:

...
Also, a KMP has an explicit dependency on kernel version/release it was built against (no idea if it's the case here).

Actually, looking at the KMP (checked virtualbox-host-kmp-default) there is only a dependency on the kernel flavor, but not on the kernel version:

I'm sorry, I wanted to change the beginning of this sentence from "if a KMP has..." to "a KMP can have..." and this was the result. :-( Michal Kubeček -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Marco Calistri

16 Nov 16 Nov

13:04

Il 13/11/2017 18:03, Larry Finger ha scritto:

...

On 11/13/2017 12:20 PM, Dominique Leuenberger wrote:

In the E-mail with subject "New Tumbleweed snapshot 20171112 released!", we see the following:

...
Packages changed: ... kernel-source (4.13.11 -> 4.13.12) ...

As noted, the kernel after updating will be 4.13.12; however, the VirtualBox kernel modules in the repos are for kernel 4.13.10. Accordingly, if you update either a Tumbleweed guest or host, the kernel modules will not be available. In a host, you will be unable to start a VirtualBox Virtual Machine. The popup window will tell you to run '/sbin/bvoxconfig'. Unfortunately, that does not work with the current source. Edit (as root) /sbin/vboxconfig, find the two lines that say

echo "Reinstall virtualbox-host-source package. Quitting .." exit 1

and remove them. You may also need to 'sudo zypper in kernel-devel'. The script should now create and install the kernel modules. If not, please post the error messages. These changes to the script will soon be submitted.

In a TW guest, the vboxvideo module will be available, but the module used for shared files will not. I am still working on the vboxguestconfig script to make it work in this situation.

Larry Hi,

Just in case others could be encountering same problem, I'm using the Oracle version of VirtualBox but I faced an error during driver compilation: marco@linux-turion64:~> tail /var/log/vbox-install.log /tmp/vbox.0/linux/VBoxNetFlt-linux.c: In function ‘vboxNetFltLinuxForwardToIntNetInner’: /tmp/vbox.0/linux/VBoxNetFlt-linux.c:1526:47: error: ‘SKB_GSO_UDP’ undeclared (first use in this function); did you mean ‘SKB_GSO_ESP’? if ( (skb_shinfo(pBuf)->gso_type & (SKB_GSO_UDP | SKB_GSO_TCPV6 | SKB_GSO_TCPV4)) ^~~~~~~~~~~ SKB_GSO_ESP make[4]: *** [/usr/src/linux-4.14.0-1.gab9e909/scripts/Makefile.build:315: /tmp/vbox.0/linux/VBoxNetFlt-linux.o] Error 1 make[3]: *** [/usr/src/linux-4.14.0-1.gab9e909/Makefile:1508: _module_/tmp/vbox.0] Error 2 make[2]: *** [Makefile:146: sub-make] Error 2 make[1]: *** [Makefile:24: __sub-make] Error 2 make: *** [/tmp/vbox.0/Makefile.include.footer:97: vboxnetflt] Error 2 I resolved by hand-apply the following patch: https://gist.github.com/herbmillerjr/039c129e9c25b047b906e19ad1f23a59 Regards, -- Marco Calistri Linux version : openSUSE Tumbleweed 20171114 Kernel: 4.14.0-1.gab9e909-default - Cinnamon 3.6.2 -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Markus Feilner

13:25

Justmy2cents: I had to deal with proprietary 90ies software again yesternight, and it was horrible :-) Vbox, Windows7 and stuff - on TW. Am I wrong, or are the things I describe on g+ packagers', maintainers' mistakes? I attribute it to Oracle, or is this something we could help fix? https://plus.google.com/+MarkusFeilner/posts/akaRESz18Bj "#blamewarestruck" "I had this feeling today, when I installed virtualbox and started a VM. I was told I am missing an RPM vbox-tool package. Nice, who the fuck forgot this check at packaging? Well, petty details. Install the package. Installed fine, call config tool. It says: "Hey, your vbox-tool doesn't match vbox-main package." WTF? What do we have dependencies for??? So uninstall, update, install both again. This shouldn't be possible. But wait: Call config tool, now it says: You'll need kernel sources. Install them, and the tool says: "Sorry, bro, I don't support your kernel". So much time wasted, and all for crappy proprietary stuff. Thanks, +Oracle! " Maybe I'm just too stupid, but still I think this is not how it should be, but exactly my expectations of proprietary software (like vmware, NVIDIA and others) Am Donnerstag, 16. November 2017, 13:04:19 CET schrieb Marco Calistri:

...

Il 13/11/2017 18:03, Larry Finger ha scritto:

...
On 11/13/2017 12:20 PM, Dominique Leuenberger wrote:

In the E-mail with subject "New Tumbleweed snapshot 20171112 released!",

we see the following:

...
Packages changed: ...

...
kernel-source (4.13.11 -> 4.13.12)

...

As noted, the kernel after updating will be 4.13.12; however, the VirtualBox kernel modules in the repos are for kernel 4.13.10. Accordingly, if you update either a Tumbleweed guest or host, the kernel modules will not be available. In a host, you will be unable to start a VirtualBox Virtual Machine. The popup window will tell you to run '/sbin/bvoxconfig'. Unfortunately, that does not work with the current source. Edit (as root) /sbin/vboxconfig, find the two lines that say

echo "Reinstall virtualbox-host-source package. Quitting .." exit 1

and remove them. You may also need to 'sudo zypper in kernel-devel'. The script should now create and install the kernel modules. If not, please post the error messages. These changes to the script will soon be submitted.

In a TW guest, the vboxvideo module will be available, but the module used for shared files will not. I am still working on the vboxguestconfig script to make it work in this situation.

Larry

Hi,

Just in case others could be encountering same problem, I'm using the Oracle version of VirtualBox but I faced an error during driver compilation:

marco@linux-turion64:~> tail /var/log/vbox-install.log

/tmp/vbox.0/linux/VBoxNetFlt-linux.c: In function ‘vboxNetFltLinuxForwardToIntNetInner’: /tmp/vbox.0/linux/VBoxNetFlt-linux.c:1526:47: error: ‘SKB_GSO_UDP’ undeclared (first use in this function); did you mean ‘SKB_GSO_ESP’? if ( (skb_shinfo(pBuf)->gso_type & (SKB_GSO_UDP | SKB_GSO_TCPV6 | SKB_GSO_TCPV4)) ^~~~~~~~~~~ SKB_GSO_ESP make[4]: *** [/usr/src/linux-4.14.0-1.gab9e909/scripts/Makefile.build:315: /tmp/vbox.0/linux/VBoxNetFlt-linux.o] Error 1 make[3]: *** [/usr/src/linux-4.14.0-1.gab9e909/Makefile:1508: _module_/tmp/vbox.0] Error 2 make[2]: *** [Makefile:146: sub-make] Error 2 make[1]: *** [Makefile:24: __sub-make] Error 2 make: *** [/tmp/vbox.0/Makefile.include.footer:97: vboxnetflt] Error 2

I resolved by hand-apply the following patch:

https://gist.github.com/herbmillerjr/039c129e9c25b047b906e19ad1f23a59

Regards,

-- Markus Feilner Team Lead Documentation P.S.: I moved - new home address: Wöhrdstraße 10, 93059 Regensburg - - - _This incident will be documented._ - - - +49 173 5876 838 (also via Signal), privat: +49 170 302 7092 mfeilner@suse.[com|de] http://www.suse.com G+: https://plus.google.com/+MarkusFeilner Xing: http://www.xing.com/profile/Markus_Feilner LinkedIn: https://www.linkedin.com/in/markusfeilner #mfeilner: Jabber, Skype, Twitter openSUSE: http://www.opensuse.org - - - SUSE Linux GmbH GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

nicholas cunliffe

15:39

from someone not up to speed on this, is it not safe to dup (using vbox from TW oss repo)? -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

J. F. Lemaire

16:03

On 16 November 2017 at 15:39, nicholas cunliffe wrote:

...

from someone not up to speed on this, is it not safe to dup (using vbox from TW oss repo)?

No problem for me after a zypper dup earlier this morning. Cheers, JFL -- Jean-François Lemaire -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Larry Finger

16:22

On 11/16/2017 08:39 AM, nicholas cunliffe wrote:

...

from someone not up to speed on this, is it not safe to dup (using vbox from TW oss repo)?

I am not quite sure what you mean by "safe". The only possible unfavorable outcome is that the VB kernel modules may not match a newly installed kernel. That would cause virtual machines using VirtualBox to fail to run. Your options at that point are to (1) boot the previous kernel, (2) use the suggested script /sbin/vboxconfig to build those modules locally, or (3) install the version of VB from Oracle. The latter option may have problems with the kernel build, as seen in a previous mail in this thread. I am actively working on correcting the build configuration error that allows a new kernel to be shipped without the appropriate kernel modules; however, the spec file for VB is very complicated. Larry -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Larry Finger

16:26

On 11/16/2017 09:03 AM, J. F. Lemaire wrote:

...

On 16 November 2017 at 15:39, nicholas cunliffe wrote:

...
from someone not up to speed on this, is it not safe to dup (using vbox from TW oss repo)?

No problem for me after a zypper dup earlier this morning.

As noted in the release document for the 20171114 snapshot, the VB changes were ==== virtualbox ==== Subpackages: virtualbox-host-kmp-default virtualbox-qt - Update "fixes_for_leap15.patch" for wait queue API changes. - Add file "fixes_for_python.patch". - Modify "fixes_for_leap15.patch" to handle SKB_GSO_UDP API change. - Addresses VUL-0: CVE-2017-10392,CVE-2017-10407,CVE-2017-10408,CVE-2017-3733,CVE-2017-10428 as noted in bsc #1064200 The "new" part of this fix was to fix build errors on Leap 15.0, but the modules were rebuilt for TW, which fixed the problem. Larry -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Stefan Seyfried

17:13

On 16.11.2017 13:25, Markus Feilner wrote:

...

Justmy2cents:

...

told I am missing an RPM vbox-tool package. Nice, who the fuck forgot this check at packaging?

You just offered to help Larry with the virtualbox packaging, right`

...

Well, petty details. Install the package. Installed fine, call config tool. It says: "Hey, your vbox-tool doesn't match vbox-main package." WTF? What do we have dependencies for??? So uninstall, update, install both again. This shouldn't be possible. But wait: Call config tool, now it says: You'll need kernel sources. Install them, and the tool says: "Sorry, bro, I don't support your kernel". So much time wasted, and all for crappy proprietary stuff. Thanks, +Oracle! "

Yes, proprietary GPL'ed software where you have no sources to improve things. Oh. Wait. -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

Larry Finger

17 Nov 17 Nov

23:23

On 11/15/2017 09:34 AM, Dominique Leuenberger / DimStar wrote:

...

On Tue, 2017-11-14 at 08:33 +0100, Michal Kubecek wrote:

...
...
But the weak-modules functionality should catch this, or am I mistaken and this does not work on Factory / Tumbleweed?

I don't think weak-modules can work reliably in Tumbleweed where we make no attempt to preserve kABI. There is always a chance this particular update doesn't break kABI (or at least not the part relevant for virtualbox modules) but in general, you can't rely on that.

Also, a KMP has an explicit dependency on kernel version/release it was built against (no idea if it's the case here).

Actually, looking at the KMP (checked virtualbox-host-kmp-default) there is only a dependency on the kernel flavor, but not on the kernel version:

...
zypper info --requires virtualbox-host-kmp-default Information for package virtualbox-host-kmp-default:

Repository : 1112 Name : virtualbox-host-kmp-default Version : 5.1.30_k4.13.10_1-3.2 Arch : x86_64 Vendor : openSUSE Installed Size : 824.2 KiB Installed : No Status : not installed Source package : virtualbox-5.1.30-3.2.src Summary : Host kernel module for VirtualBox Description : VirtualBox is an extremely feature rich, high performance product for enterprise customers, it is also the only professional solution that is freely available as Open Source Software under the terms of the GNU Public License (GPL). Requires : [7] coreutils grep kernel-default rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(PayloadIsXz) <= 5.2-1 /bin/sh

=> that also explains my 'confusion' in my earlier mail about the bot not triggering it: there is simply no indication tnat this would not work with the current kernel.

This seems to be an error specific to virtualbox though, since other packages I checked are different there:

...
zypper info --requires bbswitch-kmp-default Information for package bbswitch-kmp-default:

Repository : 1112 Name : bbswitch-kmp-default Version : 0.8_k4.13.12_1-6.2 Arch : x86_64 Vendor : openSUSE Installed Size : 25.3 KiB Installed : No Status : not installed Source package : bbswitch-0.8-6.2.src Summary : Bumblebee ACPI kernel module Description : bbswitch is a kernel module which automatically detects the required ACPI calls for two kinds of Optimus laptops. Requires : [7] coreutils grep rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(PayloadIsXz) <= 5.2-1 kernel-uname-r = 4.13.12-1-default ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /bin/sh

If this is corrected in VBox, then the bot will see when it's nescessary to retrigger a build of virtualbox to get a new kmp built.

The only thing I see in the bbswitch spec file is a line that says "BuildRequires: %{kernel_module_package_buildreqs}". Putting a line like that in VB does not change anything , and the kmp still requires kernel-default! Despite bbswitch being so simple, I fail to see why VB fails to match it. Thanks, Larry -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org

2359

Age (days ago)

2363

Last active (days ago)

List overview

Download

14 comments

8 participants

participants (8)

Dominique Leuenberger / DimStar
J. F. Lemaire
Larry Finger
Marco Calistri
Markus Feilner
Michal Kubecek
nicholas cunliffe
Stefan Seyfried