Mailing List Archive

[BUG, PATCH-RFC] libvirt localtime and rtc_timeoffset handling in xen-sexpr/sxpr/sxp
Hello,

I'm currently tracking a problem in libvirt regarding Xens handling of
localtime and rtc_timeoffset. My current understanding (Xen-3.4.3 and
Xen-4.1.2 under Linux) of Xend (the depcrecated Python one still used by
libvirt) is as this:
- for HV domains, the RTC gets setup to either UTC or localtime depending
on "/domain/image/hvm/localtime" ± "/domain/image/hvm/rtc_offset".
- if the OS of a domU changes its RTC, the rtc_offset gets adjusted and is
saved in XenStore as "/vm/$UUID/rtc/timeoffset".
- if the dom0 accesses its RTC, is accesses the real HW-RTC.
- the Xen-Hypervisor initially read the HW-RTC to setup its Wallclock once,
which is than used to simulate the domU RTCs. (The HW-RTC is otherwise only
accessed on (ACPI-)Suspend and Resume, and with NTP-drift-correction from
dom0).
- on shut-down the rtc_offset is stored by Xend in
the "/var/lib/xend/domains/$uuid/config.sxp" file
in "/domain/image/hvm/rtc_timeoffset", from where it is loaded again on next
start.
- since PV domains don't have a RTC, they somehow(?) get either initialized to
the localtime or UTC time depending on "/domain/image/linux/localtime".

@xen:
Did I figure out that correct?

@xen:
Is there some documentation on the Xen-sxp domain configuration? For the
Python based xen-xm format, I found (and updated)
<http://wiki.xen.org/wiki/XenConfigurationFileOptions>, but for Xen-sxp I so
far found no documentation, especially on what changed between xen-1, xen-2,
xen-3.x, xen-4.x.

@libvirt:
Comparing Xend handling to <http://libvirt.org/formatdomain.html#elementsTime>
the current translation done by libvirt looks wrong; I think is mandates back
to the time when Xen supported only PV-domUs:
libvirt translates the Xen configuration to "localtime" and "utc" ignoring
the "rtc_offset", which exists for HV domains. For localtime=0 this
translates to libvirts offset="variable"-case, but for localtime=1 there is
no matching mapping in libvirt.
Since for PV domains no rtc_timeoffset is tracked, there the mapping to "utc"
and "localtime" looks right.


For libvirt there was a patch
<http://www.redhat.com/archives/libvir-list/2009-January/msg00757.html> which
added some special handling for "localtime" to be either placed
in "/domain/localtime" or "/domain/image/{hvm,linux}/localtime". Xend from
3.4.3 und 4.1.2 seems to accept either one, but /domain/image/hvm/localtime
is preferred and overwrites the first one. When reading back the
configuration the setting is always returned
as /domain/image/{hvm,linux}/localtime.

@John:
Is there a case, where /domain/localtime is returned or is that key
always translated to /domain/linux/{hvm,linux}/localtime? As you had a
sun.com email address, was this some special case when using Xen with
Solaris?

@libvirt:
The attached patch (for 0.8.7) would change the implementation to match the
following:
1. For Xen-PV-domUs, use clock/@offset='utc' and clock/@offset='localtime'.
2. For Xen-HV-domUs, use clock/@offset='variable'.
3. For backward compatibility with old libvirt-XML-files convert
clock/@offset='utc' → (localtime 0)(rtc_timeoffset 0) and
clock/@offset='localtime' → (localtime 1)(rtc_timeosset 0). On readback that
will be returned as clock/@offset='variable'!
4. For Xen-HV-domUs with (localtime=1)(rtc_timeoffset≠0) print a warning that
there is no mapping to libvirts XML.
5. Always put the (localtime)(rtc_offset)-SEXPRs in "(image ({linux,hvm})",
since this is where Xend-3.4 and Xend-4.1 return them.
I also checked Xen-3.2, where this is okay, but the I don't have any older
versions of Xen available (and running), the I can't verify that it still
works there.

Which leads me to a another question: Which versions of Xen are still
supported by libvirt (and must be checked for regressions)? I don't want so
actively remove the code for old Xen versions, but it gets harder and harder
to maintain all those versions. So a statement like "Xen-3.x and Xen-4.y are
actively supported by libvirt-0.a.b; older versions might still work (by
accident ;-)"

Before I forward-port that change to 0.9.10 I'd like to get some comments.
Thanks in advance.

Sincerely
Philipp Hahn
--
Philipp Hahn Open Source Software Engineer hahn@univention.de
Univention GmbH Linux for Your Business fon: +49 421 22 232- 0
Mary-Somerville-Str.1 D-28359 Bremen fax: +49 421 22 232-99
http://www.univention.de/
Re: [BUG, PATCH-RFC] libvirt localtime and rtc_timeoffset handling in xen-sexpr/sxpr/sxp [ In reply to ]
On Wed, 2012-01-11 at 10:20 +0000, Philipp Hahn wrote:
> Hello,
>
> I'm currently tracking a problem in libvirt regarding Xens handling of
> localtime and rtc_timeoffset. My current understanding (Xen-3.4.3 and
> Xen-4.1.2 under Linux) of Xend (the depcrecated Python one still used by
> libvirt) is as this:
> - for HV domains, the RTC gets setup to either UTC or localtime depending
> on "/domain/image/hvm/localtime" ± "/domain/image/hvm/rtc_offset".

Are those xenstore paths or a reference to a config file/sxp var?

I don't see the string "rtc_offset" anywhere under tools in
xen-unstable.

> - if the OS of a domU changes its RTC, the rtc_offset gets adjusted and is
> saved in XenStore as "/vm/$UUID/rtc/timeoffset".

I think so, yes, qemu appears to do this.

> - if the dom0 accesses its RTC, is accesses the real HW-RTC.
> - the Xen-Hypervisor initially read the HW-RTC to setup its Wallclock once,
> which is than used to simulate the domU RTCs. (The HW-RTC is otherwise only
> accessed on (ACPI-)Suspend and Resume, and with NTP-drift-correction from
> dom0).
> - on shut-down the rtc_offset is stored by Xend in
> the "/var/lib/xend/domains/$uuid/config.sxp" file
> in "/domain/image/hvm/rtc_timeoffset", from where it is loaded again on next
> start.
> - since PV domains don't have a RTC, they somehow(?) get either initialized to
> the localtime or UTC time depending on "/domain/image/linux/localtime".

They get PV time direct from the hypervisor which exposes the
hypervisor's wallclock to guests via the shared info. It is always UTC.
See
http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#Struct_shared_info
in particular the wc_* fields.

A pvops domU will only sample this at start of day and then will
maintain free-running wallclock time itself from then on (it is
recommended to run ntp within such domUs).

Classic-Xen kernels by default will take wallclock time from the shared
info for each gettimeofday but can be configured to be free-running
(that is the "independent_wallclock" mode).

> @xen:
> Did I figure out that correct?
>
> @xen:
> Is there some documentation on the Xen-sxp domain configuration? For the
> Python based xen-xm format, I found (and updated)
> <http://wiki.xen.org/wiki/XenConfigurationFileOptions>, but for Xen-sxp I so
> far found no documentation, especially on what changed between xen-1, xen-2,
> xen-3.x, xen-4.x.

Xen-1 and -2 had different architectures and toolstacks IIRC so we can
discount them I think.

I'm afraid I don't know of any resource for the -3/4 sxp stuff.

>
> @libvirt:
> Comparing Xend handling to <http://libvirt.org/formatdomain.html#elementsTime>
> the current translation done by libvirt looks wrong; I think is mandates back
> to the time when Xen supported only PV-domUs:
> libvirt translates the Xen configuration to "localtime" and "utc" ignoring
> the "rtc_offset", which exists for HV domains. For localtime=0 this
> translates to libvirts offset="variable"-case, but for localtime=1 there is
> no matching mapping in libvirt.
> Since for PV domains no rtc_timeoffset is tracked, there the mapping to "utc"
> and "localtime" looks right.
>
>
> For libvirt there was a patch
> <http://www.redhat.com/archives/libvir-list/2009-January/msg00757.html> which
> added some special handling for "localtime" to be either placed
> in "/domain/localtime" or "/domain/image/{hvm,linux}/localtime". Xend from
> 3.4.3 und 4.1.2 seems to accept either one, but /domain/image/hvm/localtime
> is preferred and overwrites the first one. When reading back the
> configuration the setting is always returned
> as /domain/image/{hvm,linux}/localtime.
>
> @John:
> Is there a case, where /domain/localtime is returned or is that key
> always translated to /domain/linux/{hvm,linux}/localtime? As you had a
> sun.com email address, was this some special case when using Xen with
> Solaris?
>
> @libvirt:
> The attached patch (for 0.8.7) would change the implementation to match the
> following:
> 1. For Xen-PV-domUs, use clock/@offset='utc' and clock/@offset='localtime'.
> 2. For Xen-HV-domUs, use clock/@offset='variable'.
> 3. For backward compatibility with old libvirt-XML-files convert
> clock/@offset='utc' → (localtime 0)(rtc_timeoffset 0) and
> clock/@offset='localtime' → (localtime 1)(rtc_timeosset 0). On readback that
> will be returned as clock/@offset='variable'!
> 4. For Xen-HV-domUs with (localtime=1)(rtc_timeoffset≠0) print a warning that
> there is no mapping to libvirts XML.
> 5. Always put the (localtime)(rtc_offset)-SEXPRs in "(image ({linux,hvm})",
> since this is where Xend-3.4 and Xend-4.1 return them.
> I also checked Xen-3.2, where this is okay, but the I don't have any older
> versions of Xen available (and running), the I can't verify that it still
> works there.
>
> Which leads me to a another question: Which versions of Xen are still
> supported by libvirt (and must be checked for regressions)? I don't want so
> actively remove the code for old Xen versions, but it gets harder and harder
> to maintain all those versions. So a statement like "Xen-3.x and Xen-4.y are
> actively supported by libvirt-0.a.b; older versions might still work (by
> accident ;-)"
>
> Before I forward-port that change to 0.9.10 I'd like to get some comments.
> Thanks in advance.
>
> Sincerely
> Philipp Hahn



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: [BUG, PATCH-RFC] libvirt localtime and rtc timeoffset handling in xen-sexpr/sxpr/sxp [ In reply to ]
Hello Ian,

Am Mittwoch 11 Januar 2012 12:54:04 schrieb Ian Campbell:
> On Wed, 2012-01-11 at 10:20 +0000, Philipp Hahn wrote:
> > I'm currently tracking a problem in libvirt regarding Xens handling of
> > localtime and rtc_timeoffset. My current understanding (Xen-3.4.3 and
> > Xen-4.1.2 under Linux) of Xend (the depcrecated Python one still used by
> > libvirt) is as this:
> > - for HV domains, the RTC gets setup to either UTC or localtime depending
> > on "/domain/image/hvm/localtime" ± "/domain/image/hvm/rtc_offset".
>
> Are those xenstore paths or a reference to a config file/sxp var?

Those are XenStore paths.

> I don't see the string "rtc_offset" anywhere under tools in
> xen-unstable.

I missed the "time", it's "rtc_timeoffset". Best search for RegExp
rtc.timeoffset, since so you'll find 'rtc/timeoffset' as well.

...
> They get PV time direct from the hypervisor which exposes the
> hypervisor's wallclock to guests via the shared info. It is always UTC.
> See
> http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#St
>ruct_shared_info in particular the wc_* fields.

Yes, I figured that out as well, but thanks for the link.

> Classic-Xen kernels by default will take wallclock time from the shared
> info for each gettimeofday but can be configured to be free-running
> (that is the "independent_wallclock" mode).

A lot of documentation still talks about /proc/xen/independent_wallclock,
which doesn't exist in 2.6.32 any more. I wish they would mention the Xen
version and Linux kerbel version they were using ...

Thank you for your fast feedback.

Sincerely
Philipp Hahn
--
Philipp Hahn Open Source Software Engineer hahn@univention.de
Univention GmbH Linux for Your Business fon: +49 421 22 232- 0
Mary-Somerville-Str.1 D-28359 Bremen fax: +49 421 22 232-99
http://www.univention.de/