On Mittwoch, 5. Dezember 2018, 19:54:54 CET Keil, Karsten wrote:
Hello,
In our private OBS instance we use KVM workers on SLE12S-SP3. The build servers are HP DL380 Gen9 with 40 cores with 256GB RAM for big workers, 128 GB Ram for smaller ones. On such HW we running for example 10 instances with 3 Jobs. The Problem is that after the initial install stage here are very often (I would say ~ 1/3 of the builds) huge delays in the start of the VM:
A good one: .... [ 10s] booting kvm... [ 10s] ### VM INTERACTION START ### [ 11s] /usr/bin/qemu-kvm -nodefaults -no-reboot -nographic -vga none -object rng-random,filename=/dev/random,id=rng0 -device virtio-rng-pci,rng=rng0 -runas qemu -cpu host -net n [ 11s] [ 1.063999] dracut-pre-udev[161]: modprobe: FATAL: Module kqemu not found. [ 12s] [ 2.047790] dracut-pre-udev[161]: modprobe: FATAL: Module ibmvscsi not found. [ 12s] [ 2.052451] dracut-pre-udev[161]: modprobe: FATAL: Module ibmveth not found. [ 12s] ### VM INTERACTION END ### [ 12s] 2nd stage started in virtual machine [ 12s] machine type: x86_64 ... Bad looks like: ... [ 11s] booting kvm... [ 12s] ### VM INTERACTION START ### [ 12s] /usr/bin/qemu-kvm -nodefaults -no-reboot -nographic -vga none -object rng-random,filename=/dev/random,id=rng0 -device virtio-rng-pci,rng=rng0 -runas qemu -cpu host -net n [ 12s] [ 1.437692] dracut-pre-udev[161]: modprobe: FATAL: Module kqemu not found. [ 111s] [ 98.122901] dracut-pre-udev[161]: modprobe: FATAL: Module ibmvscsi not found. [ 112s] [ 99.982451] dracut-pre-udev[161]: modprobe: FATAL: Module ibmveth not found. [ 112s] ### VM INTERACTION END ### [ 112s] 2nd stage started in virtual machine [ 112s] machine type: x86_64 ... This could also up to 300 seconds, I do not really found a relationship to the size of the build jobs or amount of packages to install. Also the build itself seems not to be much slower after the delay. The host itself does not have any high load at these moments, most of CPUs is idle. So my questions are: What exact happens in that stage of the VM start, are there copies of the initial packages ongoing ? Does somebody see similar things ? OBS version is 2.92.
It seems to be the initrd booting... OBS version should not really matter here, it is more a question which kernel/ initrd is used IMHO. Do you use a kernel-obs-build here or is it the one from the worker system? -- Adrian Schroeter SUSE Linux Products GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org