Mailing List Archive

rendezvousing all physical CPUs
Recently I created a Linux patch (in 2.6.18.3 and 2.6.19) that fixes problem
with the detection of the Firmware Hub based random number generator in
Intel chipsets. Little later I considered the consequences in Xen environments:
The patch needs to disable normal BIOS space in order to 'see through' to the
Firmware Hub interface. In native Linux, the stop_machine abstraction is
a very good mechanism to carry out this kind of job, but under Xen this isn't
strong enough, as it affects only those CPUs Dom0 is running on.

Will it be acceptable to create hypercall sub-functions (would probably go
into the platform group, but should be architecture independent) to allow
Dom0 to halt all physical CPUs but the current one, and later restart them?
Or should it rather be a single call with an event-channel based call back
to carry out the operation that must be protected?

Without such functionality, we'd have to disable the Intel RNG in the kernel,
or at least make it ignore any potentially available RNG unless told by the
user that it is safe to use *without* the detection sequence outlined by
Intel's documentation (which btw. has been found to not work when the
BIOS makes use of certain functionality that prevents access to the FWH
space).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: rendezvousing all physical CPUs [ In reply to ]
Question for you. I just found out the FWH parts are now end-of-life;
I was about to put a driver for it in Plan 9 when I learned this, so I
stopped.

So, the question is, given FWH is going away, is this additional
function needed? Does it have some use outside the RNG application? I
don't know one way or another.

thanks

ron

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: rendezvousing all physical CPUs [ In reply to ]
>>> "ron minnich" <rminnich@gmail.com> 30.11.06 17:19 >>>
>Question for you. I just found out the FWH parts are now end-of-life;
>I was about to put a driver for it in Plan 9 when I learned this, so I
>stopped.
>
>So, the question is, given FWH is going away, is this additional
>function needed? Does it have some use outside the RNG application? I
>don't know one way or another.

I don't know, but what I do know is that there's a huge amount of
machines out there having these chipsets.

I would suspect there could be other uses - for instance, I don't think
having the microcode update code in Xen is the right thing, it could
easily live in Dom0 (which would also cause less work merging back
updates from Linux into Xen) if that had a way of bringing all physical
CPUs to a known state.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: rendezvousing all physical CPUs [ In reply to ]
On 11/30/06, Jan Beulich <jbeulich@novell.com> wrote:
> I would suspect there could be other uses - for instance, I don't think
> having the microcode update code in Xen is the right thing, it could
> easily live in Dom0 (which would also cause less work merging back
> updates from Linux into Xen) if that had a way of bringing all physical
> CPUs to a known state.

yeah, make sense to me. I am just getting back into Xen after some
time away, but I'm surprised there is not a way to do this.

thanks

ron

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: rendezvousing all physical CPUs [ In reply to ]
On 30/11/06 16:14, "Jan Beulich" <jbeulich@novell.com> wrote:

> Will it be acceptable to create hypercall sub-functions (would probably go
> into the platform group, but should be architecture independent) to allow
> Dom0 to halt all physical CPUs but the current one, and later restart them?
> Or should it rather be a single call with an event-channel based call back
> to carry out the operation that must be protected?

How about providing the linear address of a chunk of dom0 code that Xen
should run in ring 0 with CPUs in a particular configuration? We could
provide flags to represent useful configurations: e.g., run on all CPUs
atomicaly, run on CPU0 only and quiesce others, etc.

As you say this could be used for things arguably more useful than this RNG
example, like microcode updates and maybe even the MTRR updates could be
done in dom0 too, which would be very nice. :-)

-- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: rendezvousing all physical CPUs [ In reply to ]
On 11/30/06, Keir Fraser <keir@xensource.com> wrote:

> How about providing the linear address of a chunk of dom0 code that Xen
> should run in ring 0 with CPUs in a particular configuration? We could
> provide flags to represent useful configurations: e.g., run on all CPUs
> atomicaly, run on CPU0 only and quiesce others, etc.

I thought the goal of Jan's proposal was to have Dom0 take over this
kind of thing, not extend Xen to do it. So his idea of getting CPUs in
a known state, then letting Dom0 handle it, is different, is it not?
Does this really address his problem?

ron

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: rendezvousing all physical CPUs [ In reply to ]
On 30/11/06 17:12, "ron minnich" <rminnich@gmail.com> wrote:

>> How about providing the linear address of a chunk of dom0 code that Xen
>> should run in ring 0 with CPUs in a particular configuration? We could
>> provide flags to represent useful configurations: e.g., run on all CPUs
>> atomicaly, run on CPU0 only and quiesce others, etc.
>
> I thought the goal of Jan's proposal was to have Dom0 take over this
> kind of thing, not extend Xen to do it. So his idea of getting CPUs in
> a known state, then letting Dom0 handle it, is different, is it not?
> Does this really address his problem?

Dom0 would provide the code in both cases. The CPUs would be in a known
state in both cases. It's a question of which interface is cleaner and/or
easier to implement.

-- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: rendezvousing all physical CPUs [ In reply to ]
>>> Keir Fraser <keir@xensource.com> 30.11.06 17:55 >>>
>On 30/11/06 16:14, "Jan Beulich" <jbeulich@novell.com> wrote:
>
>> Will it be acceptable to create hypercall sub-functions (would probably go
>> into the platform group, but should be architecture independent) to allow
>> Dom0 to halt all physical CPUs but the current one, and later restart them?
>> Or should it rather be a single call with an event-channel based call back
>> to carry out the operation that must be protected?
>
>How about providing the linear address of a chunk of dom0 code that Xen
>should run in ring 0 with CPUs in a particular configuration? We could
>provide flags to represent useful configurations: e.g., run on all CPUs
>atomicaly, run on CPU0 only and quiesce others, etc.

Hmm??? I would have to question why dom0 currently gets run in ring 1 then.

I would at best consider allowing the guest to pass a batch of operations that
it wants carried out - I/O memory accesses (normal RAM not allowed), MSR
reads/writes, port I/O. However, for the specific case of the RNG, PCI config
space accesses would also need to be supported - while they can be reduced
to iomem or port accesses, abstracting this out from the requester and from
Xen would require some thought.

>As you say this could be used for things arguably more useful than this RNG
>example, like microcode updates and maybe even the MTRR updates could be
>done in dom0 too, which would be very nice. :-)

While the RNG example may seem odd or unimportant, the point is that
currently this doesn't present a problem only because apparently no-one
but dom0 can actually see (physical) BIOS memory space (and hence depend
on its contents). I wonder if that is a proper assumption for I/O domains
currently and/or long term, since XEN_DOMCTL_iomem_permission allows
doing such.

Certainly, I agree that using this for MTRR handling (along with microcode
updating) if feasible would be very handy maintenance wise.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: rendezvousing all physical CPUs [ In reply to ]
On 1/12/06 08:16, "Jan Beulich" <jbeulich@novell.com> wrote:

>> How about providing the linear address of a chunk of dom0 code that Xen
>> should run in ring 0 with CPUs in a particular configuration? We could
>> provide flags to represent useful configurations: e.g., run on all CPUs
>> atomicaly, run on CPU0 only and quiesce others, etc.
>
> Hmm??? I would have to question why dom0 currently gets run in ring 1 then.

Uniformity with execution of other domains and it's also a useful safety
mechanism for catching dom0 bugs. It is *not* for security reasons!

> I would at best consider allowing the guest to pass a batch of operations that
> it wants carried out - I/O memory accesses (normal RAM not allowed), MSR
> reads/writes, port I/O. However, for the specific case of the RNG, PCI config
> space accesses would also need to be supported - while they can be reduced
> to iomem or port accesses, abstracting this out from the requester and from
> Xen would require some thought.

Dom0 can map any memory, DMA to any memory, and with your original proposed
hypercall would be able to tear down physical CPUs for unbounded time
periods. It 0wns the machine already. ;-)

-- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel