Comment # 4 on bug 1232489 from Steven Mishos

If I'm reading this FAQ correctly this may be expected behaviour in chrony, but
I'm far from an NTP expert so this may be a wrong interpretation.

6.5. Why is an offset measured between two computers synchronised to each
another?

When two computers are synchronised to each other using the client/server or
symmetric NTP mode, there is an expectation that NTP measurements between the
two computers made on both ends show an average offset close to zero.

With chronyd that can be expected only when the interleaved mode is enabled by
the xleave option. Otherwise, chronyd will use different transmit timestamps
(e.g. daemon timestamp vs kernel timestamp) for serving time and
synchronisation of its own clock, which will cause the other computer to
measure a significant offset.

https://chrony-project.org/faq.html