Issues with vagrant / libvirt since snapshot from this week Tuesday or Wednesday?
Hi all, is anyone else experiencing issues with libvirt since mid of this week? I can no longer start VMs using vagrant. Vagrant hangs at "Waiting for domain to get an IP address..." and never gets anywhere. Existing VMs can be started using virsh or libvirt, so it might be an issue with vagrant itself. But it was working until mid of this week, despite being outdated (due to the licensing change). I noticed some errors regarding missing directories on Feb 8th and today:
Feb 08 13:09:28 baten systemd[1]: Starting libvirt legacy monolithic daemon... Feb 08 13:09:28 baten libvirtd[7841]: libvirt version: 10.0.0 Feb 08 13:09:28 baten libvirtd[7841]: hostname: baten Feb 08 13:09:28 baten libvirtd[7841]: Failed to initialize libnetcontrol. Management of interface devices is disabled Feb 08 13:09:28 baten systemd[1]: Started libvirt legacy monolithic daemon. Feb 08 13:09:29 baten libvirtd[7841]: Interner Fehler: failed to query mdevs from mdevctl: Error: Required directory "/usr/lib/mdevctl/scripts.d/callouts" doesn't exist. This may indicate a packaging or installation error Feb 08 13:11:28 baten systemd[1]: libvirtd.service: Deactivated successfully.
or
Feb 10 18:31:36 baten systemd[1]: Started libvirt legacy monolithic daemon. Feb 10 18:31:37 baten libvirtd[4420]: Interner Fehler: failed to query mdevs from mdevctl: Error: Required directory "/usr/lib/mdevctl/scripts.d/notifiers" doesn't exist. This may indicate a packaging or installation error
And sure enough there are no directories in /usr/lib/mdevctl/scripts.d/ on all of my machines... Kind Regards, Johannes
On 2/10/24 11:53, Johannes Kastl wrote:
Hi all,
is anyone else experiencing issues with libvirt since mid of this week?
I can no longer start VMs using vagrant. Vagrant hangs at "Waiting for domain to get an IP address..." and never gets anywhere.
Existing VMs can be started using virsh or libvirt, so it might be an issue with vagrant itself. But it was working until mid of this week, despite being outdated (due to the licensing change).
I noticed some errors regarding missing directories on Feb 8th and today:
Feb 08 13:09:28 baten systemd[1]: Starting libvirt legacy monolithic daemon... Feb 08 13:09:28 baten libvirtd[7841]: libvirt version: 10.0.0 Feb 08 13:09:28 baten libvirtd[7841]: hostname: baten Feb 08 13:09:28 baten libvirtd[7841]: Failed to initialize libnetcontrol. Management of interface devices is disabled Feb 08 13:09:28 baten systemd[1]: Started libvirt legacy monolithic daemon. Feb 08 13:09:29 baten libvirtd[7841]: Interner Fehler: failed to query mdevs from mdevctl: Error: Required directory "/usr/lib/mdevctl/scripts.d/callouts" doesn't exist. This may indicate a packaging or installation error Feb 08 13:11:28 baten systemd[1]: libvirtd.service: Deactivated successfully.
or
Feb 10 18:31:36 baten systemd[1]: Started libvirt legacy monolithic daemon. Feb 10 18:31:37 baten libvirtd[4420]: Interner Fehler: failed to query mdevs from mdevctl: Error: Required directory "/usr/lib/mdevctl/scripts.d/notifiers" doesn't exist. This may indicate a packaging or installation error
And sure enough there are no directories in /usr/lib/mdevctl/scripts.d/ on all of my machines...
You are probably getting hit by the update of glibc from 2.38 to 2.39. It shows in VirtualBox as no USB passthrough. I added the Slowroll repo and downdgraded glibc along with the 15 dependent packages and the USB access is restored. See boo#1219766 for repo details. Larry
Cc Jim On 10. 02. 24, 18:53, Johannes Kastl wrote:
Hi all,
is anyone else experiencing issues with libvirt since mid of this week?
I can no longer start VMs using vagrant. Vagrant hangs at "Waiting for domain to get an IP address..." and never gets anywhere.
Existing VMs can be started using virsh or libvirt, so it might be an issue with vagrant itself. But it was working until mid of this week, despite being outdated (due to the licensing change).
I noticed some errors regarding missing directories on Feb 8th and today:
Feb 08 13:09:28 baten systemd[1]: Starting libvirt legacy monolithic daemon... Feb 08 13:09:28 baten libvirtd[7841]: libvirt version: 10.0.0 Feb 08 13:09:28 baten libvirtd[7841]: hostname: baten Feb 08 13:09:28 baten libvirtd[7841]: Failed to initialize libnetcontrol. Management of interface devices is disabled Feb 08 13:09:28 baten systemd[1]: Started libvirt legacy monolithic daemon. Feb 08 13:09:29 baten libvirtd[7841]: Interner Fehler: failed to query mdevs from mdevctl: Error: Required directory "/usr/lib/mdevctl/scripts.d/callouts" doesn't exist. This may indicate a packaging or installation error Feb 08 13:11:28 baten systemd[1]: libvirtd.service: Deactivated successfully.
or
Feb 10 18:31:36 baten systemd[1]: Started libvirt legacy monolithic daemon. Feb 10 18:31:37 baten libvirtd[4420]: Interner Fehler: failed to query mdevs from mdevctl: Error: Required directory "/usr/lib/mdevctl/scripts.d/notifiers" doesn't exist. This may indicate a packaging or installation error
And sure enough there are no directories in /usr/lib/mdevctl/scripts.d/ on all of my machines...
I would bet on: https://build.opensuse.org/request/show/1141334 Jim? BTW what happens if you create the two dirs manually? -- js suse labs
On 12. 02. 24, 7:24, Jiri Slaby wrote:
Cc Jim
On 10. 02. 24, 18:53, Johannes Kastl wrote:
Hi all,
is anyone else experiencing issues with libvirt since mid of this week?
I can no longer start VMs using vagrant. Vagrant hangs at "Waiting for domain to get an IP address..." and never gets anywhere.
Existing VMs can be started using virsh or libvirt, so it might be an issue with vagrant itself. But it was working until mid of this week, despite being outdated (due to the licensing change).
I noticed some errors regarding missing directories on Feb 8th and today:
Feb 08 13:09:28 baten systemd[1]: Starting libvirt legacy monolithic daemon... Feb 08 13:09:28 baten libvirtd[7841]: libvirt version: 10.0.0 Feb 08 13:09:28 baten libvirtd[7841]: hostname: baten Feb 08 13:09:28 baten libvirtd[7841]: Failed to initialize libnetcontrol. Management of interface devices is disabled Feb 08 13:09:28 baten systemd[1]: Started libvirt legacy monolithic daemon. Feb 08 13:09:29 baten libvirtd[7841]: Interner Fehler: failed to query mdevs from mdevctl: Error: Required directory "/usr/lib/mdevctl/scripts.d/callouts" doesn't exist. This may indicate a packaging or installation error Feb 08 13:11:28 baten systemd[1]: libvirtd.service: Deactivated successfully.
or
Feb 10 18:31:36 baten systemd[1]: Started libvirt legacy monolithic daemon. Feb 10 18:31:37 baten libvirtd[4420]: Interner Fehler: failed to query mdevs from mdevctl: Error: Required directory "/usr/lib/mdevctl/scripts.d/notifiers" doesn't exist. This may indicate a packaging or installation error
And sure enough there are no directories in /usr/lib/mdevctl/scripts.d/ on all of my machines...
I would bet on: https://build.opensuse.org/request/show/1141334
Jim?
BTW what happens if you create the two dirs manually?
And I think you should create a bug. -- js suse labs
Hi Jiri, On 12.02.24 07:24 Jiri Slaby wrote:
And sure enough there are no directories in /usr/lib/mdevctl/scripts.d/ on all of my machines...
I would bet on: https://build.opensuse.org/request/show/1141334
Jim?
BTW what happens if you create the two dirs manually?
I am not sure the errors are the reason for the non-starting VM, as they do no longer appear. And I tried creating those directories manually, but to no avail. Kind Regards, Johannes
On 2/11/24 23:24, Jiri Slaby wrote:
Cc Jim
On 10. 02. 24, 18:53, Johannes Kastl wrote:
Hi all,
is anyone else experiencing issues with libvirt since mid of this week?
I can no longer start VMs using vagrant. Vagrant hangs at "Waiting for domain to get an IP address..." and never gets anywhere.
Existing VMs can be started using virsh or libvirt, so it might be an issue with vagrant itself. But it was working until mid of this week, despite being outdated (due to the licensing change).
Did you start experiencing the issue after updating libvirt? Do things work again after downgrading libvirt to the previous working version?
I noticed some errors regarding missing directories on Feb 8th and today:
Feb 08 13:09:28 baten systemd[1]: Starting libvirt legacy monolithic daemon... Feb 08 13:09:28 baten libvirtd[7841]: libvirt version: 10.0.0 Feb 08 13:09:28 baten libvirtd[7841]: hostname: baten Feb 08 13:09:28 baten libvirtd[7841]: Failed to initialize libnetcontrol. Management of interface devices is disabled Feb 08 13:09:28 baten systemd[1]: Started libvirt legacy monolithic daemon. Feb 08 13:09:29 baten libvirtd[7841]: Interner Fehler: failed to query mdevs from mdevctl: Error: Required directory "/usr/lib/mdevctl/scripts.d/callouts" doesn't exist. This may indicate a packaging or installation error Feb 08 13:11:28 baten systemd[1]: libvirtd.service: Deactivated successfully.
The libnetcontrol and mdevctl errors are not critical and should not prevent starting VMs.
or
Feb 10 18:31:36 baten systemd[1]: Started libvirt legacy monolithic daemon. Feb 10 18:31:37 baten libvirtd[4420]: Interner Fehler: failed to query mdevs from mdevctl: Error: Required directory "/usr/lib/mdevctl/scripts.d/notifiers" doesn't exist. This may indicate a packaging or installation error
And sure enough there are no directories in /usr/lib/mdevctl/scripts.d/ on all of my machines...
I would bet on: https://build.opensuse.org/request/show/1141334
Nod, the mdevctl errors were introduced by this update. Facepalm for not performing simple tests :-(. I've sent a request to fix it https://build.opensuse.org/request/show/1146259
BTW what happens if you create the two dirs manually?
In my testing, the libvirt error disappears, as does the similar one emitted when running 'mdevctl list'. Regards, Jim
Hi Jim, On 12.02.24 19:45 Jim Fehlig wrote:
Did you start experiencing the issue after updating libvirt? Do things work again after downgrading libvirt to the previous working version?
I am not sure when this started, as I was not using vagrant/libvirt daily during the last week. And I had no time to dig into downgrading libvirt. But I can say that there is another Tumbleweed machine with a "similar" setup, also up to date, where it seems to work. And another one where it doesn't...
The libnetcontrol and mdevctl errors are not critical and should not prevent starting VMs.
As they no longer appeared, but it still does not work, that is what I thought...
Nod, the mdevctl errors were introduced by this update. Facepalm for not performing simple tests :-(. I've sent a request to fix it
Thanks!
BTW what happens if you create the two dirs manually?
In my testing, the libvirt error disappears, as does the similar one emitted when running 'mdevctl list'.
Yes, the error with 'mdevctl list' is gone after creating those directories. Kind Regards, Johannes
On Sat, 2024-02-10 at 18:53 +0100, Johannes Kastl wrote:
Hi all,
is anyone else experiencing issues with libvirt since mid of this week?
I can no longer start VMs using vagrant. Vagrant hangs at "Waiting for domain to get an IP address..." and never gets anywhere.
Existing VMs can be started using virsh or libvirt, so it might be an issue with vagrant itself. But it was working until mid of this week, despite being outdated (due to the licensing change).
I noticed some errors regarding missing directories on Feb 8th and today:
Feb 08 13:09:28 baten systemd[1]: Starting libvirt legacy monolithic daemon... Feb 08 13:09:28 baten libvirtd[7841]: libvirt version: 10.0.0 Feb 08 13:09:28 baten libvirtd[7841]: hostname: baten Feb 08 13:09:28 baten libvirtd[7841]: Failed to initialize libnetcontrol. Management of interface devices is disabled
This looks suspicious to me, given that your VM doesn't obtain an IP address. "failed to initialize libnetcontrol" errors have been reported before for openSUSE, but afaics the existing reports are from 2021 and before.
Feb 08 13:09:28 baten systemd[1]: Started libvirt legacy monolithic daemon. Feb 08 13:09:29 baten libvirtd[7841]: Interner Fehler: failed to query mdevs from mdevctl: Error: Required directory "/usr/lib/mdevctl/scripts.d/callouts" doesn't exist. This may indicate a packaging or installation error Feb 08 13:11:28 baten systemd[1]: libvirtd.service: Deactivated successfully.
or
Feb 10 18:31:36 baten systemd[1]: Started libvirt legacy monolithic daemon. Feb 10 18:31:37 baten libvirtd[4420]: Interner Fehler: failed to query mdevs from mdevctl: Error: Required directory "/usr/lib/mdevctl/scripts.d/notifiers" doesn't exist. This may indicate a packaging or installation error
And sure enough there are no directories in /usr/lib/mdevctl/scripts.d/ on all of my machines...
I know this isn't helpful, but I just tried and my environment started just fine. It might be some detail of how your vagrant box is set up. I not using the "modularized" libvirt setup [1]. Package list here: glibc-2.39-1.2.x86_64 libnetcontrol0-0.3.2-3.2.x86_64 libvirt-daemon-driver-qemu-10.0.0-2.2.x86_64 libvirt-daemon-driver-interface-10.0.0-2.2.x86_64 mdevctl-1.3.0-1.2.x86_64 qemu-8.1.3-1.2.x86_64 vagrant-2.3.7-4.2.noarch vagrant-libvirt-0.11.2-2.6.x86_64 Anyway I'd suggest looking into the libnetcontrol error message. Regards Martin [1] https://doc.opensuse.org/documentation/leap/virtualization/html/book-virtual...
On 2/12/24 02:21, Martin Wilck via openSUSE Factory wrote:
On Sat, 2024-02-10 at 18:53 +0100, Johannes Kastl wrote:
Hi all,
is anyone else experiencing issues with libvirt since mid of this week?
I can no longer start VMs using vagrant. Vagrant hangs at "Waiting for domain to get an IP address..." and never gets anywhere.
Existing VMs can be started using virsh or libvirt, so it might be an issue with vagrant itself. But it was working until mid of this week, despite being outdated (due to the licensing change).
I noticed some errors regarding missing directories on Feb 8th and today:
Feb 08 13:09:28 baten systemd[1]: Starting libvirt legacy monolithic daemon... Feb 08 13:09:28 baten libvirtd[7841]: libvirt version: 10.0.0 Feb 08 13:09:28 baten libvirtd[7841]: hostname: baten Feb 08 13:09:28 baten libvirtd[7841]: Failed to initialize libnetcontrol. Management of interface devices is disabled
This looks suspicious to me, given that your VM doesn't obtain an IP address. "failed to initialize libnetcontrol" errors have been reported before for openSUSE, but afaics the existing reports are from 2021 and before.
It's actually harmless, unless you are trying to manage *host* interface devices using e.g. 'virsh iface-*'. libvirt has always required libnetcontrol+wicked for that. Management of host interface devices has never been supported when using NetworkManager. Regards, Jim
Hi Martin, On 12.02.24 10:21 Martin Wilck via openSUSE Factory wrote:
I know this isn't helpful,
Actually it is. Narrowing down what could be the reason is hard, but getting the information that it works on other machines is a clue.
but I just tried and my environment started just fine. It might be some detail of how your vagrant box is set up. I not using the "modularized" libvirt setup [1]. Package list here:
glibc-2.39-1.2.x86_64 libnetcontrol0-0.3.2-3.2.x86_64 libvirt-daemon-driver-qemu-10.0.0-2.2.x86_64 libvirt-daemon-driver-interface-10.0.0-2.2.x86_64 mdevctl-1.3.0-1.2.x86_64 qemu-8.1.3-1.2.x86_64
I see qemu 8.2.0-1.1 as the only difference.
vagrant-2.3.7-4.2.noarch vagrant-libvirt-0.11.2-2.6.x86_64
Kind Regards, Johannes
On 10.02.24 18:53, Johannes Kastl wrote:
is anyone else experiencing issues with libvirt since mid of this week?
Yes, one of my VMs (a Windows 10 VM) does not start anymore since this week. It runs for some seconds and is then destroyed. I tried to boot this VM with a Linux ISO (tried with Desinfec't from Heise), but also the Linux kernel from Desinfec't panics. I re-created the qemu-img snapshot file to make sure, that the problem is not caused by a new Windows issue, but this also does not help. Probably Libvird with Qemu does not work anymore with some VM configurations since latest Tumbleweed updates. Björn
Hi all, On 10.02.24 18:53 Johannes Kastl wrote:
I can no longer start VMs using vagrant. Vagrant hangs at "Waiting for domain to get an IP address..." and never gets anywhere.
I lost track of this due to other things, but tested again this week and suddenly everything works again. No idea what actually solved it. Kind Regards, Johannes
participants (6)
-
Bjoern Voigt -
Jim Fehlig -
Jiri Slaby -
Johannes Kastl -
Larry Finger -
Martin Wilck