Comment # 18 on bug 1174365 from Klaus Mueller

(In reply to James Fehlig from comment #17)
> Maybe we can get an idea about who is asking libvirtd to suspend the VMs by
> enabling debug logging in the remote driver and API. E.g. a log_filters in
> libvirtd.conf along the lines of
> 
> log_filters="1:remote 1:libvirt"

Very good idea and thanks for this hint! Test gives:
2021-10-12 06:32:50.862+0000: 8176: debug : handleSystemMessageFunc:579 :
dmn=0x563dd634c820
2021-10-12 06:32:50.863+0000: 9102: debug : daemonStopWorker:518 : Begin stop
dmn=0x563dd634c820
...
2021-10-12 06:32:50.863+0000: 9102: debug : virConnectOpenInternal:1128 :
driver 8 QEMU returned SUCCESS
2021-10-12 06:32:50.863+0000: 9102: debug : virConnectListAllDomains:6642 :
conn=0x7fea100073c0, domains=0x7fe9d49eacb0, flags=0x1
2021-10-12 06:32:50.863+0000: 9102: debug : virDomainGetState:2487 :
dom=0x7fe9a0003410, (VM: name=CentOS-7.x,
uuid=12f858d9-e0cd-a352-1a43-349ab1a7cd21), state=0x7fe9d49eacac, reason=(nil),
flags=0x0
2021-10-12 06:32:50.863+0000: 9102: debug : virDomainSuspend:623 :
dom=0x7fe9a0003410, (VM: name=CentOS-7.x,
uuid=12f858d9-e0cd-a352-1a43-349ab1a7cd21)
2021-10-12 06:32:51.008+0000: 9102: debug : virDomainManagedSave:9547 :
dom=0x7fe9a0003410, (VM: name=CentOS-7.x,
uuid=12f858d9-e0cd-a352-1a43-349ab1a7cd21), flags=0x2
2021-10-12 06:32:53.294+0000: 9102: debug : daemonStopWorker:522 : Completed
stop dmn=0x563dd634c820

(Leap 15.3 host stops Centos 7 VM)

Searching for handleSystemMessageFunc in
https://github.com/paboldin/libvirt/blob/master/daemon/libvirtd.c gives:

static DBusHandlerResult
handleSystemMessageFunc(DBusConnection *connection ATTRIBUTE_UNUSED,
                        DBusMessage *message,
                        void *opaque)
...

Now it's sure, that a message sent via DBUS initiated the stop of the VM. This
interferes with systemd stop services (-> racy). The first one wins ... .

> Yeah, that's a classic problem with save/restore. NTP in the guest helps to
> some extent. If the guest has been suspended for long periods of time,
> 'virsh domtime ...' or the qemu guest agent 'guest-set-time' command can be
> used to correct the  guest's clock
> 
> https://qemu.readthedocs.io/en/latest/interop/qemu-ga-ref.html#qapidoc-19

That's a very helpful hint, too! Thanks for it! I tested it and its working. Do
I understand that I have to execute the guest-set-time myself? Maybe there is a
switch to enable the execution on startup automatically? Or maybe there is a
possibility to add "scripts" to be started after a VM has been resumed?