Mailing List Archive

* Ian Pratt (m+Ian.Pratt@cl.cam.ac.uk) wrote:
> Please excuse me from auto-subscribing you to this list, but I figured
> it would be useful to have a list dedicated for discussion about getting
> arch-xen prepared for sending upstream.

Thanks for setting that up Ian.

> It's pretty clear that we need to move fast on this, lest we get stuck
> with the VMware VMI proposal that just doesn't do what we need to get
> good performance. I'd much rather get our patch in first and add their
> hooks to our stuff, rather than being forced to work within the
> framework of their very low-level approach.

I agree, it makes sense to get things in, and let later work go on with
something concrete in place.

> So, how best to go about this? Can we parallelize the work? Where to
> start?

I've started by simply creating an i386 sub-arch and moving bits over.
(nowhere near compiling, just trying to get a handle on how much work
there is and how it will split out). I use this dumb script that
basically diffs arch/xen/i386/* against arch/i386/*, for example,
to generate the actual changes. This could literally go file by file
with an eye for basic cleanups. We've had a few suggestions about the
cleanups, such as ifdefs based on some config which essentially says
physical machine, to effectively comment code out for Xen, or actual
inlines and macros that are populated by the sub-arch. Likely both are
options depending on the specifics. For example, x86_64 has method for
alternative apic, I wonder if such a scheme might not be useful for i386
as well?

Also, Andi mentioned to me that he'd actually prefer to try and merge
the x86_64 bits directly into the x86_64 port rather than add a sub-arch
there if possible (Andi, please correct me if I misunderstood).

> Anyone interested in a trip to Cambridge to work on this stuff? It's a
> nice time of year for a visit -- nice warm summer evenings sitting in
> beer gardens...

Hmm, quite tempting.

thanks,
-chris

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

Chris Wright wrote:
> * Ian Pratt (m+Ian.Pratt@cl.cam.ac.uk) wrote:
>> Please excuse me from auto-subscribing you to this list, but I
>> figured it would be useful to have a list dedicated for discussion
>> about getting arch-xen prepared for sending upstream.
>
> Thanks for setting that up Ian.
>
>> It's pretty clear that we need to move fast on this, lest we get
>> stuck with the VMware VMI proposal that just doesn't do what we need
>> to get good performance. I'd much rather get our patch in first and
>> add their hooks to our stuff, rather than being forced to work
>> within the framework of their very low-level approach.
>
> I agree, it makes sense to get things in, and let later work go on
> with something concrete in place.
>
>> So, how best to go about this? Can we parallelize the work? Where to
>> start?
>
> I've started by simply creating an i386 sub-arch and moving bits over.
> (nowhere near compiling, just trying to get a handle on how much work
> there is and how it will split out). I use this dumb script that
> basically diffs arch/xen/i386/* against arch/i386/*, for example,
> to generate the actual changes. This could literally go file by file
> with an eye for basic cleanups. We've had a few suggestions about the
> cleanups, such as ifdefs based on some config which essentially says
> physical machine, to effectively comment code out for Xen, or actual
> inlines and macros that are populated by the sub-arch. Likely both
> are options depending on the specifics. For example, x86_64 has
> method for alternative apic, I wonder if such a scheme might not be
> useful for i386 as well?

I'm not sure if there is a VMI patch against x86_64 at this point, but
I'm cleaning up the x86_64 code so that we can share the code with the
native Linux cleanly. At this point, I'm using #ifdef CONFIG_XEN as
agreed with Andi.

>
> Also, Andi mentioned to me that he'd actually prefer to try and merge
> the x86_64 bits directly into the x86_64 port rather than add a
> sub-arch there if possible (Andi, please correct me if I
> misunderstood).

In that case how do we switch the kernels headers for xenlinux?

>
>> Anyone interested in a trip to Cambridge to work on this stuff? It's
>> a nice time of year for a visit -- nice warm summer evenings sitting
>> in beer gardens...
>
> Hmm, quite tempting.
>
> thanks,
> -chris
>

Jun
---
Intel Open Source Technology Center

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

> I've started by simply creating an i386 sub-arch and moving bits
over.
> (nowhere near compiling, just trying to get a handle on how
> much work there is and how it will split out). I use this
> dumb script that basically diffs arch/xen/i386/* against
> arch/i386/*, for example, to generate the actual changes.
> This could literally go file by file with an eye for basic
> cleanups. We've had a few suggestions about the cleanups,
> such as ifdefs based on some config which essentially says
> physical machine, to effectively comment code out for Xen, or
> actual inlines and macros that are populated by the sub-arch.
> Likely both are options depending on the specifics. For
> example, x86_64 has method for alternative apic, I wonder if
> such a scheme might not be useful for i386 as well?

Is it worth us setting up one or more Linux 2.6 mercurial tress on
xenbits that we can use to show each other what we're doing? Patches for
this sort of thing aren't easy to read. (Just send me an ssh v2 public
key to set up 'push' access).

BTW, the people currently on this list are:
ian.pratt@cl.cam.ac.uk
christian.limpach@cl.cam.ac.uk
keir.fraser@cl.cam.ac.uk
vincent.hanquez@cl.cam.ac.uk
chrisw@osdl.org
jun.nakajima@intel.com
mbligh@mbligh.org
ak@suse.de
kurt@garloff.de
riel@redhat.com

> > Anyone interested in a trip to Cambridge to work on this
> stuff? It's a
> > nice time of year for a visit -- nice warm summer evenings
> sitting in
> > beer gardens...
>
> Hmm, quite tempting.

Give it some serious thought...

Cheers,
Ian

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

On Fri, 29 Jul 2005, Ian Pratt wrote:

> Is it worth us setting up one or more Linux 2.6 mercurial tress on
> xenbits that we can use to show each other what we're doing? Patches for
> this sort of thing aren't easy to read.

This worries me. Patches that are not easy to read are
going to be horribly hard to merge into xen-unstable...

Now if we had an idea on what shape would be best for
merging things into the xen-unstable tree, we could
work backwards from there to come up with the kind of
changes to generate.

Of course, we may still come up with the conclusion
that we want the mercurial trees, but at least we'll
know why ;)

--
The Theory of Escalating Commitment: "The cost of continuing mistakes is
borne by others, while the cost of admitting mistakes is borne by yourself."
-- Joseph Stiglitz, Nobel Laureate in Economics

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

> > Is it worth us setting up one or more Linux 2.6 mercurial tress on
> > xenbits that we can use to show each other what we're
> doing? Patches
> > for this sort of thing aren't easy to read.
>
> This worries me. Patches that are not easy to read are going
> to be horribly hard to merge into xen-unstable...

I imagine the patches we submit will consist of a sequence that tidy up
i386 and x86_64 and create all the hooks we need, and then a final patch
that actually adds the Xen support.

The way I would propose going about doing this is to create a Linux hg
tree that has all the re-arrangements in it with xen as a sub-arch, and
then generate a diff that we chop up and arrange into the separate
patches.

The first part of the work is going to be rearranging our sparse tree to
split arch/xen out in to drivers/xen/core and arch/{i386/x86_64}/xen.
Patches for this step would be very messy (mostly file renames) and
aren't worth maintaining as patches, hence the Linux hg tree.

> Now if we had an idea on what shape would be best for merging
> things into the xen-unstable tree, we could work backwards
> from there to come up with the kind of changes to generate.
>
> Of course, we may still come up with the conclusion that we
> want the mercurial trees, but at least we'll know why ;)

Ian

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

> I'm not sure if there is a VMI patch against x86_64 at this point, but
> I'm cleaning up the x86_64 code so that we can share the code with the
> native Linux cleanly.

That sounds like an excellent plan, poking around it seems like the
division between x86 and x86_64 isn't particularly clean - seems to be
a lot of copied code (not Xen stuff, but just pre-existant).

> At this point, I'm using #ifdef CONFIG_XEN as agreed with Andi.
>
>> Also, Andi mentioned to me that he'd actually prefer to try and merge
>> the x86_64 bits directly into the x86_64 port rather than add a
>> sub-arch there if possible (Andi, please correct me if I
>> misunderstood).

Seems odd that Andi would suggest such a thing ... flies directly in
the face of what most people are recommending to do with Linux code,
particularly if it's mid-function. Personally I think subarches do
clean up the code a lot, if they're done sensibly, and we're going
to have to cope with things like summit subarch in x86_64 as well.
I fought against doing it for ia32 at first ... then realized I was
wrong. If we just want to throw some files in subdirs instead of a
"subarch", is good ... but you end up with pretty much the same thing.

Andi, you objected to subarches when we talked at OLS ... what exactly
was the issue, and is it fixable? Not sure it's any worse than the
alternatives, at any rate.

Even if one did not one a subarch, it makes the code much cleaner and
easier to read to define macros at suitable abstraction points, even if
it's just

static inline wibble_with_foo (int bar, int blat) {
#ifdef CONFIG_XEN
do A
#else
do B
#endif.
}

which gets stuck in a header file somewhere. Then we just have
"wibble_with_foo(bar, blat);" in the main code, which makes it much
easier to read (assuming you picked a meaningful function name).
Perhaps I'm just reading too much into your "ifdefs" thing, and this
is more what you're intending.

M.

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

> Please excuse me from auto-subscribing you to this list, but I figured
> it would be useful to have a list dedicated for discussion about getting
> arch-xen prepared for sending upstream.
>
> It's pretty clear that we need to move fast on this, lest we get stuck
> with the VMware VMI proposal that just doesn't do what we need to get
> good performance. I'd much rather get our patch in first and add their
> hooks to our stuff, rather than being forced to work within the
> framework of their very low-level approach.
>
> So, how best to go about this? Can we parallelize the work? Where to
> start?

A couple of basic questions:

1) What version of Xen code are we going to try to merge? People tell me
the development stuff here is in quite a bit of flux right now ...

2) Do you want to end up with something that switches statically at
compile time, or dynamically for all standard x86 kernels? Ways to cope
look to me like:
A) CONFIG option switch
B) Function pointers (like ia32 generic subarch stuff)
C) Dynamic rewriting (like CPU optimization stuff).

Not sure these are actually exclusive ... might be easiest to start with
(A), and evolve it into (C) over time? Certainly (A) will be simpler and
easier to get merged, and if we create the abstraction points right, it
shouldn't be too hard to change over later.

I wouldn't panic too much about the vmware thing, we can stall merging
easily enough, I suspect, as long as it doesn't drag out for months.

M.


_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

On Tue, 2 Aug 2005, Martin J. Bligh wrote:

> 1) What version of Xen code are we going to try to merge? People tell me
> the development stuff here is in quite a bit of flux right now ...

It's in flux, but this is the code base that most people will
want by the time a merge can be completed, so ...

> 2) Do you want to end up with something that switches statically at
> compile time, or dynamically for all standard x86 kernels? Ways to cope
> look to me like:
> A) CONFIG option switch
> B) Function pointers (like ia32 generic subarch stuff)
> C) Dynamic rewriting (like CPU optimization stuff).
>
> Not sure these are actually exclusive ... might be easiest to start with
> (A), and evolve it into (C) over time? Certainly (A) will be simpler and
> easier to get merged, and if we create the abstraction points right, it
> shouldn't be too hard to change over later.

I would definitely prefer to start out with (A). The sooner we
can get xenolinux merged, the less time I need to spend on merging
the same patches into kernel RPMs over and over again, and the
more time I will have to actually work on development of Xen ;)

My main question is, how can we share the work we're doing, and
how can we merge the prepare-for-upstream changes into the
linux-sparse tree ?

--
The Theory of Escalating Commitment: "The cost of continuing mistakes is
borne by others, while the cost of admitting mistakes is borne by yourself."
-- Joseph Stiglitz, Nobel Laureate in Economics

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

On Tue, 2 Aug 2005, Ian Pratt wrote:

> The first part of the work is going to be rearranging our sparse tree to
> split arch/xen out in to drivers/xen/core and arch/{i386/x86_64}/xen.
> Patches for this step would be very messy (mostly file renames) and
> aren't worth maintaining as patches, hence the Linux hg tree.

Could we do the move of non-arch specific code from
arch/xen to drivers/xen/core in the current Xen
linux-sparse tree already ?

We can then do the other bits gradually.

My main worry is that we would end up with an alternative
tree that has an obsolete version of Xen that's mergeable
upstream - and no way to merge that tree with the current
Xen bits.

--
The Theory of Escalating Commitment: "The cost of continuing mistakes is
borne by others, while the cost of admitting mistakes is borne by yourself."
-- Joseph Stiglitz, Nobel Laureate in Economics

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

>> 1) What version of Xen code are we going to try to merge? People tell me
>> the development stuff here is in quite a bit of flux right now ...
>
> It's in flux, but this is the code base that most people will
> want by the time a merge can be completed, so ...

OK, so maybe we should just try to be brave then ;-) I get the feeling
we don't *actually* want to be doing this right now, in a couple of
months would be better ... but still.

>> 2) Do you want to end up with something that switches statically at
>> compile time, or dynamically for all standard x86 kernels? Ways to cope
>> look to me like:
>> A) CONFIG option switch
>> B) Function pointers (like ia32 generic subarch stuff)
>> C) Dynamic rewriting (like CPU optimization stuff).
>>
>> Not sure these are actually exclusive ... might be easiest to start with
>> (A), and evolve it into (C) over time? Certainly (A) will be simpler and
>> easier to get merged, and if we create the abstraction points right, it
>> shouldn't be too hard to change over later.
>
> I would definitely prefer to start out with (A). The sooner we

Is definitely simpler, and easier to prove that you're not going to
piss on anyone else's shoes in the process.

> My main question is, how can we share the work we're doing, and
> how can we merge the prepare-for-upstream changes into the
> linux-sparse tree ?

I'd think it'd be easy enough to break it down by subsystem? Not that
some bits won't be left over, but should break the back of it. We'd
need a common style/approach first though. Apologies for missing the
conversation on Wednesday at OLS, they rudely scheduled me to present
my OLS paper in that timeslot ;-)

My experience from doing the Summit stuff was that the hard bit is
refactoring the existing functions to create suitable abstraction points -
clean breaks for where you want to change the code. That way you don't
end up copying chunks of code everywhere. We could do that fairly easily
in parallel I think, and it probably wouldn't get scuppered too much by
Xen code changes (the hook points are less likely to change). However,
it'll probably cause horrible churn for anyone trying to keep the Xen
code in sync with mainline ;-)

OK, so I forgot question (3).... or rather I think we covered it before
but I forgot the answer. Are we going to try to do i386 or x86_64 first,
or both at the same time? Personally I'd prefer to do i386 first, as the
subarch stuff is in place there, and it should be easier, but maybe that's
just me.

M.


_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

* Rik van Riel (riel@redhat.com) wrote:
> On Fri, 29 Jul 2005, Ian Pratt wrote:
>
> > Is it worth us setting up one or more Linux 2.6 mercurial tress on
> > xenbits that we can use to show each other what we're doing? Patches for
> > this sort of thing aren't easy to read.
>
> This worries me. Patches that are not easy to read are
> going to be horribly hard to merge into xen-unstable...

The bits that are rough to read are largely renames.

> Now if we had an idea on what shape would be best for
> merging things into the xen-unstable tree, we could
> work backwards from there to come up with the kind of
> changes to generate.

I've got something that's mostly compiling, that maybe a useful halfway
step. A good point towards splitting the work apart. I'll clean up
what I've got and post it later today.

thanks,
-chris

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

* Ian Pratt (m+Ian.Pratt@cl.cam.ac.uk) wrote:
>
> > > Is it worth us setting up one or more Linux 2.6 mercurial tress on
> > > xenbits that we can use to show each other what we're
> > doing? Patches
> > > for this sort of thing aren't easy to read.
> >
> > This worries me. Patches that are not easy to read are going
> > to be horribly hard to merge into xen-unstable...
>
> I imagine the patches we submit will consist of a sequence that tidy up
> i386 and x86_64 and create all the hooks we need, and then a final patch
> that actually adds the Xen support.
>
> The way I would propose going about doing this is to create a Linux hg
> tree that has all the re-arrangements in it with xen as a sub-arch, and
> then generate a diff that we chop up and arrange into the separate
> patches.

The chop up and diff part isn't looking too horrible. There will be some
headaches if it takes too long and there's lots of remerging to keep up.

> The first part of the work is going to be rearranging our sparse tree to
> split arch/xen out in to drivers/xen/core and arch/{i386/x86_64}/xen.
> Patches for this step would be very messy (mostly file renames) and
> aren't worth maintaining as patches, hence the Linux hg tree.

Yeah, in fact, I think it can be copies during interim so both sides
can continue to build.

thanks,
-chris

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

* Rik van Riel (riel@redhat.com) wrote:
> On Tue, 2 Aug 2005, Ian Pratt wrote:
>
> > The first part of the work is going to be rearranging our sparse tree to
> > split arch/xen out in to drivers/xen/core and arch/{i386/x86_64}/xen.
> > Patches for this step would be very messy (mostly file renames) and
> > aren't worth maintaining as patches, hence the Linux hg tree.
>
> Could we do the move of non-arch specific code from
> arch/xen to drivers/xen/core in the current Xen
> linux-sparse tree already ?

Yup, I think that could be done quickly in xen-unstable.

> We can then do the other bits gradually.
>
> My main worry is that we would end up with an alternative
> tree that has an obsolete version of Xen that's mergeable
> upstream - and no way to merge that tree with the current
> Xen bits.

I share your concern. So far, it's been manageable, but it's also been
relatively quiet ;-)

thanks,
-chris

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

* Martin J. Bligh (mbligh@mbligh.org) wrote:
> A couple of basic questions:
>
> 1) What version of Xen code are we going to try to merge? People tell me
> the development stuff here is in quite a bit of flux right now ...

I'm working against xen-unstable tip.

> 2) Do you want to end up with something that switches statically at
> compile time, or dynamically for all standard x86 kernels? Ways to cope
> look to me like:
> A) CONFIG option switch
> B) Function pointers (like ia32 generic subarch stuff)
> C) Dynamic rewriting (like CPU optimization stuff).

Not C unless there's no other choice (assuming you are talking
effectively self-modifying code). Right now, I've got a mixture of
essentially A and B (or a variation on that theme ;-).

thanks,
-chris

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

* Martin J. Bligh (mbligh@mbligh.org) wrote:
> OK, so I forgot question (3).... or rather I think we covered it before
> but I forgot the answer. Are we going to try to do i386 or x86_64 first,
> or both at the same time? Personally I'd prefer to do i386 first, as the
> subarch stuff is in place there, and it should be easier, but maybe that's
> just me.

Orignally I had planned to do x86_64 first, but since Andi's not keen on
subarch at this point, I've switched to i386.

thanks,
-chris

_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge

> >> 1) What version of Xen code are we going to try to merge?
> People tell
> >> me the development stuff here is in quite a bit of flux
> right now ...
> >
> > It's in flux, but this is the code base that most people
> will want by
> > the time a merge can be completed, so ...

It's not in quite as much flux as it looks :-)

The ongoing changes are all in the xen-specific drivers, changing them
over to use XenBus rather than the nasty control message API. These are
all fully self contained, so should be no problem.

The new hypervisor time API meant that there have been recent changes to
arch/xen code, but we're not expecting any more, modulo bug fixes.

We just need to pick a Linux version, pick a Xen repo version, do the
merge, and then go through picking up any relevent patches that have
occurred in Linux and Xen arch/xen in the meantime.

> OK, so maybe we should just try to be brave then ;-) I get
> the feeling we don't *actually* want to be doing this right
> now, in a couple of months would be better ... but still.

I don't think it'll be that bad to track changes to arch-xen made in the
Xen repo.

> >> 2) Do you want to end up with something that switches
> statically at
> >> compile time, or dynamically for all standard x86 kernels? Ways to
> >> cope look to me like:
> >> A) CONFIG option switch
> >> B) Function pointers (like ia32 generic subarch stuff)
> >> C) Dynamic rewriting (like CPU optimization stuff).

I'd vote for A in the first instance, but perhaps bareing B and C in
mind.

Ian


_______________________________________________
Xen-merge mailing list
Xen-merge@lists.xensource.com
http://lists.xensource.com/xen-merge