[opensuse-virtual] should ntp be governing the clock in a guest system?
Continued from opensuse-factory:
Probably other virtualization technology have the same issue, do not use ntp. I do not know for certain.
With xen, my guests run ntpd and have xen.independent_wallclock=1.
And both host and guest try to adjust the speed of the mother board hardware clock chip? (not the bios clock, that's a very different one).
No, only the Dom0 does that. The guests are independent when you use xen.independent_wallclock=1.
http://www.novell.com/documentation/vmserver/config_options/data/b9qzhq5.htm...
But if you run the ntp daemon on guests, it tries to adjust the speed of the clock there. That's how it works.
Of course, that's what you want it to do. You want the guest clock synchronized. It would appear to be a good idea to have ntp on the xen Dom0 govern the clock for all the guests, but in my experience systemd does not like that. I looked into in March 2015 when I was virtualising some systems, and found the Novell page above. The Xen FAQ: https://wiki.xen.org/wiki/Xen_FAQ_DomU also recommends running ntp in each DomU. Finally, Werner Flamme wrote:
When you install SAP systems on top of Xenified VMs, there is the recommendation to do the latter: disable the sync and run a separate ntpd in the DomU. This is also a recommendation for VMs inside a VMWare environment - the problem ist the same there.
Back in March last year I did ask what the typical practice was, but noone had any suggestions. -- Per Jessen, Zürich (25.2°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
On 2016-09-02 16:00, Per Jessen wrote:
Continued from opensuse-factory:
(by the way, this was https://bugzilla.novell.com/show_bug.cgi?id=773323)
But if you run the ntp daemon on guests, it tries to adjust the speed of the clock there. That's how it works.
Of course, that's what you want it to do. You want the guest clock synchronized. It would appear to be a good idea to have ntp on the xen Dom0 govern the clock for all the guests, but in my experience systemd does not like that.
Yes, we want the clock synchronized, but not by adjusting the speed on guests. Just copied from the host. It adds load on the virtualized hardware. I know that vmware does recommends not running ntpd in guests, but instead use: tools.syncTime = "TRUE" I'll try again to find a source for this.
I looked into in March 2015 when I was virtualising some systems, and found the Novell page above.
The Xen FAQ: https://wiki.xen.org/wiki/Xen_FAQ_DomU also recommends running ntp in each DomU.
Finally, Werner Flamme wrote:
When you install SAP systems on top of Xenified VMs, there is the recommendation to do the latter: disable the sync and run a separate ntpd in the DomU. This is also a recommendation for VMs inside a VMWare environment - the problem ist the same there.
Back in March last year I did ask what the typical practice was, but noone had any suggestions.
This is against what I read the vmware people say. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
On 2016-09-02 16:48, Carlos E. R. wrote:
On 2016-09-02 16:00, Per Jessen wrote:
Continued from opensuse-factory:
(by the way, this was https://bugzilla.novell.com/show_bug.cgi?id=773323)
But if you run the ntp daemon on guests, it tries to adjust the speed of the clock there. That's how it works.
Of course, that's what you want it to do. You want the guest clock synchronized. It would appear to be a good idea to have ntp on the xen Dom0 govern the clock for all the guests, but in my experience systemd does not like that.
Yes, we want the clock synchronized, but not by adjusting the speed on guests. Just copied from the host. It adds load on the virtualized hardware.
I know that vmware does recommends not running ntpd in guests, but instead use:
tools.syncTime = "TRUE"
I'll try again to find a source for this.
I have not found the paper where I go that info; instead got another that differs. One old document was <http://www.vmware.com/info?id=34> but that link no longer works. I read a note on one of the posts to google for vmware_timekeeping.pdf, which I found here: <www.vmware.com/pdf/vmware_timekeeping.pdf> This one has recommendations to use vmware internal clock adjusting, or ntp, with appropriate configs. And be careful to never use both. The interesting section is "Synchronizing Virtual Machines and Hosts with Real Time" in page 14. Besides that, the document is very interesting read about PC clocks in general. +++--------------------- VMware Tools periodic clock synchronization has the advantage that it is aware of the virtual machine’s built‐in catch‐up and interacts properly with it. If the guest operating system clock is behind real time by more than the known backlog that is in the process of being caught up, VMware Tools resets the clock and informs the virtual machine to stop catching up, which sets the backlog to zero. An additional advantage of VMware Tools clock synchronization is that it does not require networking to be set up in the guest. However, at this writing, VMware Tools clock synchronization has a serious limitation: it cannot correct the guest clock if it gets ahead of real time (except in the case of NetWare guest operating systems). This limitation applies only to periodic clock synchronization. VMware Tools does a one‐shot correction of the virtual machine clock that may set it either backward or forward in two cases: when the VMware Tools daemon starts (normally while the guest operating system is booting), and when a user toggles the periodic clock synchronization feature from off to on. Native synchronization software has the advantage that it is generally prepared to deal with the virtual machine clock being either ahead of or behind real time. It has the disadvantage that it is not aware of the virtual machine’s built‐in catch‐up and thus typically does not synchronize time as well in a virtual machine as it does when run directly on physical hardware. One specific problem occurs if native synchronization software happens to set the guest operating system clock forward to the correct time while the virtual machine has an interrupt backlog that it is in the process of catching up. Setting the guest operating system clock ahead is a purely software event that the virtual machine cannot be aware of, so it does not know that it should stop the catch‐up process. As a result, the guest operating system clock continues to run fast until catch‐up is complete, and it ends up ahead of the correct time. Fortunately, such events are infrequent, and the native synchronization software generally detects and corrects the error the next time it runs. Another specific problem is that native synchronization software may employ control algorithms that are tuned for the typical rate variation of physical hardware timer devices. Virtual timer devices have a more widely variable rate, which can make it difficult for the synchronization software to lock on to the proper correction factor to make the guest operating system clock run at precisely the rate of real time. As a result, the guest operating system clock tends to oscillate around the correct time to some degree. The native software may even determine that the timer device is broken and give up on correcting the clock. Despite these potential problems, however, testing has shown that NTP in particular behaves fairly well in a virtual machine when appropriately configured (see “Using NTP in Linux and Other Guests” on page 17). NTP is prepared for some of its readings to be anomalous because of network delays, scheduling delays on the local host, and other factors and is effective at filtering out such readings. Generally, it is best to use only one clock synchronization service at a time in a given virtual machine to ensure that multiple services do not attempt to make conflicting changes to the clock. So if you are using native synchronization software, we suggest turning VMware Tools periodic clock synchronization off. ---------------------++- +++--------------------- Using VMware Tools Clock Synchronization VMware Tools includes an optional clock synchronization feature that can check the guest operating system clock against the host operating system clock at regular intervals and correct the guest operating system clock. VMware Tools periodic clock synchronization works in concert with the built‐in catch‐up feature in VMware virtual machines and avoids turning the clock ahead too far. VMware Tools also performs one‐time corrections of the guest operating system clock after certain events, even if periodic synchronization is turned off. ... [Linux] By default, the daemon checks the guest operating system clock only once per minute. You can specify a different period by setting the .vmx configuration file option tools.syncTime.period to the desired time period (value specified in seconds). When the daemon checks the guest operating system clock, if it is much farther behind the host time than the virtual machine’s built‐in catch‐up mechanism expects it to be, the daemon resets the guest operating system clock to host time and cancels any pending catch‐up. As discussed above, this periodic check currently cannot correct the guest operating system time if it is ahead of the host time except in NetWare guest operating systems. ---------------------++- Then comes the section "Using NTP in Linux and Other Guests". I'll copy it here for reference: +++--------------------- The Network Time Protocol is usable in a virtual machine with proper configuration of the NTP daemon. The following points are important: * Do not configure the virtual machine to synchronize to its own (virtual) hardware clock, not even as a fallback with a high stratum number. Some sample ntpd.conf files contain a section specifying the local clock as a potential time server, often marked with the comment “undisciplined local clock.” Delete any such server specification from your ntpd.conf file. * Include the option tinker panic 0 at the top of your ntp.conf file. By default, the NTP daemon sometimes panics and exits if the underlying clock appears to be behaving erratically. This option causes the daemon to keep running instead of panicking. * Follow standard best practices for NTP: Choose a set of servers to synchronize to that have accurate time and adequate redundancy. If you have many virtual or physical client machines to synchronize, set up some internal servers for them to use, so that all your clients are not directly accessing an external low‐stratum NTP server and overloading it with requests. The following sample ntp.conf file is suitable if you have few enough clients that it makes sense for them to access an external NTP server directly. If you have many clients, adapt this file by changing the server names to reference your internal NTP servers. NOTE Any tinker commands used must appear first. # ntpd.conf tinker panic 0 restrict 127.0.0.1 restrict default kod nomodify notrap server 0.vmware.pool.ntp.org server 1.vmware.pool.ntp.org server 2.vmware.pool.ntp.org server 3.vmware.pool.ntp.org Here is a sample /etc/ntp/step-tickers corresponding to the sample ntp.conf file above. # step-tickers 0.vmware.pool.ntp.org 1.vmware.pool.ntp.org Make sure that ntpd is configured to start at boot time. On some distributions this can be accomplished with the command chkconfig ntpd on, but consult your distribution’s documentation for details. On most distributions, you can start ntpd manually with the command /etc/init.d/ntpd start. ---------------------++- (extracts from <www.vmware.com/pdf/vmware_timekeeping.pdf> Copyright © 2008 VMware, Inc. All rights reserved.) This link can be also of interest: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=di... Timekeeping best practices for Linux guests (1006427) It has a list of settings to be using on kernels of different Linux distributions. openSUSE is not included, but SLES is, and SUSE up to 10.3. There is an interesting note at the end: +++--------------------- Virtual hardware clock configuration When configuring the Linux guest operating system, if you are given a choice between keeping the "hardware" clock (that is, the virtual CMOS time of day clock) in UTC or local time, choose UTC. This avoids any confusion when your local time changes between standard and daylight saving time (in England, "summer time"). ---------------------++- ... which is false. If done, the clock is off by hours, the time difference between local and utc. Apparently, the host feeds the local time, not the utc time, to the guest in the cmos emulation. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
Carlos E. R. wrote:
On 2016-09-02 16:00, Per Jessen wrote:
Continued from opensuse-factory:
(by the way, this was https://bugzilla.novell.com/show_bug.cgi?id=773323)
I didn't know that one, but it seems to be about local vs UTC time?
But if you run the ntp daemon on guests, it tries to adjust the speed of the clock there. That's how it works.
Of course, that's what you want it to do. You want the guest clock synchronized. It would appear to be a good idea to have ntp on the xen Dom0 govern the clock for all the guests, but in my experience systemd does not like that.
Yes, we want the clock synchronized, but not by adjusting the speed on guests. Just copied from the host. It adds load on the virtualized hardware.
Yes, the obvious way seems to be to just leave the guests to trust the synchronized clock from the Dom0. The load is negligible, especially on a typical virtual host. -- Per Jessen, Zürich (24.8°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
Per Jessen wrote:
Carlos E. R. wrote:
On 2016-09-02 16:00, Per Jessen wrote:
Continued from opensuse-factory:
(by the way, this was https://bugzilla.novell.com/show_bug.cgi?id=773323)
I didn't know that one, but it seems to be about local vs UTC time?
But if you run the ntp daemon on guests, it tries to adjust the speed of the clock there. That's how it works.
Of course, that's what you want it to do. You want the guest clock synchronized. It would appear to be a good idea to have ntp on the xen Dom0 govern the clock for all the guests, but in my experience systemd does not like that.
Yes, we want the clock synchronized, but not by adjusting the speed on guests. Just copied from the host. It adds load on the virtualized hardware.
Yes, the obvious way seems to be to just leave the guests to trust the synchronized clock from the Dom0. The load is negligible, especially on a typical virtual host.
I don't know if anybody cares much, but on a xen guest running Leap422b1 with ntp disabled, I no longer see those "Time has been changed" messages apart from once or twice. (I used to have them twice a minute). I'll leave it running for a while and see what happens. Does anyone here have an opinion on the matter? -- Per Jessen, Zürich (21.1°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-09-04 18:39, Per Jessen wrote:
I don't know if anybody cares much, but on a xen guest running Leap422b1 with ntp disabled, I no longer see those "Time has been changed" messages apart from once or twice. (I used to have them twice a minute). I'll leave it running for a while and see what happens.
Does anyone here have an opinion on the matter?
That's what I see on my vmware player guest, too. I'd guess that systemd has its own clock adjustment service running and it doesn't like ntp touching the clock as well. What that service might be, I don't know. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlfMcSAACgkQja8UbcUWM1xhXQD9GIFGkFzgyPrlX3zxE3u8Z8ky gEiX/xV5qwEwNlVQ85EA/29UAjQiWOFVquMvShCp2DmbhcoH9WRN3L7BZjebmiJM =yEmt -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2016-09-04 18:39, Per Jessen wrote:
I don't know if anybody cares much, but on a xen guest running Leap422b1 with ntp disabled, I no longer see those "Time has been changed" messages apart from once or twice. (I used to have them twice a minute). I'll leave it running for a while and see what happens.
Does anyone here have an opinion on the matter?
That's what I see on my vmware player guest, too.
I'd guess that systemd has its own clock adjustment service running and it doesn't like ntp touching the clock as well. What that service might be, I don't know.
I doubt that very much. Besides, why should openSUSE have ntp running by default then? I'm beginning to wonder - if Dom0 is actually a little of out sync, ntp would slowly be adjusting the clock, and I guess systemd would notice that. - if Dom0 is actually sync'ed, no adjustment is needed, nothing for systemd to notice. -- Per Jessen, Zürich (15.4°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
On 2016-09-05 08:47, Per Jessen wrote:
Carlos E. R. wrote:
I'm beginning to wonder
- if Dom0 is actually a little of out sync, ntp would slowly be adjusting the clock, and I guess systemd would notice that.
I doubt that systemd would notice unless carefully watching for it, because normally ntpd adjusts the speed of the clock, not the value of the clock itself. Programs can not notice that unless they compare the clock with some other reference and see the difference. Programs can only notice if the value of the clock change. If it jumps suddenly skipping intermediate values. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
Carlos E. R. wrote:
On 2016-09-05 08:47, Per Jessen wrote:
Carlos E. R. wrote:
I'm beginning to wonder
- if Dom0 is actually a little of out sync, ntp would slowly be adjusting the clock, and I guess systemd would notice that.
I doubt that systemd would notice unless carefully watching for it,
I agree, so it clearly does. I suspect there is some event that is triggered when time make a jump or time is not monotone. -- Per Jessen, Zürich (17.6°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse-virtual+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-virtual+owner@opensuse.org
participants (2)
-
Carlos E. R. -
Per Jessen