Hello,

You might have seen https://progress.opensuse.org/issues/62204 which is about VMs that appear as "physical" instead of "kvm" to salt, and therefore break applying the highstate.

I had some fun (really!) debugging this yesterday - starting with reading the salt code that does the hardware detection to reading virt-what (used by salt if it's installed) and debugging it.

If you are looking for an entertaining story how a small error was able to cause such a serious problem, read the ticket yourself (ideally while being logged in so that you also see the private comments).

If you only want to hear about the actual reason and solution, scroll down - I'll add some whitespace (to prevent a spoiler) and then add what the actual fix was. However, reading the ticket is much more funny ;-)

Regards,

Christian Boltz

PS: Scroll down for the end of the story ;-)

scroll...

scroll...

scroll...

scroll...

Ok, enough scrolled ;-)

In the end, it turned out that virt-what uses which virt-what-cpuid-helper to find one of its helper scripts. Unfortunately the virt-what package doesn't have "Requires: which" (only "Requires: util-linux", where which lived until 2013) - and without "which" installed, it's not a surprise that which $whatever fails ;-)

The obvious workaround is to install "which".

I added "which" to the package list in pillar/common.sls so that it gets installed on all 15.x machines, and opened a bugreport so that we don't need the same workaround in future releases.

In case someone wonders why this only affected some VMs and not all with Leap 15.1: the condition to trigger this bug is that virt-what is installed and that which is not installed. If virt-what is not installed, salt uses systemd-detect-virt instead, which works without problems.