Mailing List Archive

Gentoo "Stable" Portage/Releases
First off, let me just say that this was just an idea I'd cooked up a
while back, so I am sure there's lots of holes in it for you guys to rip
apart. Anyway, without further ado...

The proposal is quite simple insofar as it requires no changes to
portage, whatsoever (though there are possibilities to make extensions
to portage). It introduces no new KEYWORDS and adds no load on the
current ebuild developers, other than those that wish to work on the
stable tree.

Allow me to explain.

First, there is the creation of the "release" tree. This tree is tied
to a specific release of Gentoo Linux. The tree is based on the release
snapshot used to build the release. The tree consists of the highest
version stable ebuild per slot and architecture for each package. This
means if there is no stable version of, say, vmware-player, then the
entire package is omitted. For things like GTK+, there would be at
least 2 versions in the tree, since there are 2 slots and both are
stable on at least one architecture. By only limiting the tree in this
manner, it can be built entirely by a script and require no manual
interactions to repair dependencies, etc.

So let's imagine that 2006.0 is rolling around. The 2006.0 snapshot is
frozen, and the release-building begins. This snapshot tarball is run
through our "stable" script, and a new gentoo-2006.0 CVS module is
created. A corresponding rsync module is created for this tree.

To facilitate "enterprise" usage, we break up the releases into a
"desktop" and "server" set. This means the current
"default-linux/$arch/2006.0" profile would be
"default-linux/$arch/2006.0/desktop" with a
"default-linux/$arch/2006.0/server" profile, also. The stages would be
built against the "default-linux/$arch/2006.0" profile, which would have
any USE, etc. that are common between desktop and server. During
installation, the user can choose to use either the desktop or server
profiles, or stay with the more "generic" 2006.0 profile (good for
developers, etc. that might need components of both, or want a more
minimal default set of USE flags). The desktop and server profiles will
have a defined set of default USE flags that will benefit the most
people, similar to how the current profiles are designed to be "desktop"
profiles, to benefit the most people.

The release media has SYNC set to this rsync module. What we would have
is SYNC="rsync://rsync.gentoo.org/gentoo-2006.0" in make.conf for this
release. Security updates would be applied to gentoo-2006.0, so "emerge
--sync" still allows for getting package updates, as we currently do. A
user can upgrade to 2006.1 by simply changing the SYNC setting and
updating their profile. A simple tool could be created for this, or
portage could be extended to allow for it (emerge --distupgrade 2006.1).

The current "gentoo-x86" CVS module would be the equivalent to Slackware
or FreeBSD's -current tree. Users that wish to participate in Gentoo
development, either via actual patches or simply by assisting in testing
new packages and filing bug reports, can use this tree with no
difference from how they operate now. This would also allow us to get
wider testing on the release profiles before the actual release is made.

Of course, a team would need to be created to work on the gentoo-$relver
trees. However, the trees would already exist with minimal effort on
anyone's part, allowing for an easier transition. In the end, this
would be only marginally more work for Release Engineering, so I don't
see much argument from my project, and minimal to no extra work for any
ebuild developers working on gentoo-x86 that do not also volunteer to
work on gentoo-$relver. Patches from gentoo-$relver would be
"upstreamed" into the gentoo-x86 tree, if they did not originate on
gentoo-x86, so the next gentoo-$relver tree would be already fixed.

Remember that the release trees would only be security fixes. No other
package updates should be happening. This would allow for companies to
actually certify their software against "Gentoo 2006.0", for example.

Even with no updates going into this tree, as would be the case when it
is originally implemented, we would still have a stable, unchanging
platform for external parties to test and verify against. They would
never be caught unaware with an apache upgrade or a gcc upgrade. They
would upgrade to new releases when *they* choose, giving them a stable
platform to build their Gentoo systems against. This concept also
allows for us to take "baby steps" in getting to a fully-supported set
of packages with back-ported security fixes. Because of this, I have
specifically omitted any retention or support periods, since I think
some kind of consensus would need to be reached on that, and I would
prefer that those particular details not cloud this proposal.

Speaking hypothetically, it would even be possible to create a separate
corporate entity that pays developers to work on gentoo-$relver trees by
back-porting patches and providing support to customers. Support and
patches could be maintained on any given tree for any length of time
decided. Yes, it could be possible for Gentoo to sell support.

I'm sure I'm missing a lot and this will be discussed to death, but this
is also the kind of thing that we definitely want to do right if we do
it, so feel free to punch holes all in this proposal (and I know you
will... :P).

--
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux
Re: Gentoo "Stable" Portage/Releases [ In reply to ]
Chris Gianelloni wrote:

>First off, let me just say that this was just an idea I'd cooked up a
>while back, so I am sure there's lots of holes in it for you guys to rip
>apart. Anyway, without further ado...
>
>The proposal is quite simple insofar as it requires no changes to
>portage, whatsoever (though there are possibilities to make extensions
>to portage). It introduces no new KEYWORDS and adds no load on the
>current ebuild developers, other than those that wish to work on the
>stable tree.
>
>Allow me to explain.
>
>First, there is the creation of the "release" tree. This tree is tied
>to a specific release of Gentoo Linux. The tree is based on the release
>snapshot used to build the release. The tree consists of the highest
>version stable ebuild per slot and architecture for each package. This
>means if there is no stable version of, say, vmware-player, then the
>entire package is omitted. For things like GTK+, there would be at
>least 2 versions in the tree, since there are 2 slots and both are
>stable on at least one architecture. By only limiting the tree in this
>manner, it can be built entirely by a script and require no manual
>interactions to repair dependencies, etc.
>
>So let's imagine that 2006.0 is rolling around. The 2006.0 snapshot is
>frozen, and the release-building begins. This snapshot tarball is run
>through our "stable" script, and a new gentoo-2006.0 CVS module is
>created. A corresponding rsync module is created for this tree.
>
>
>
I like this Idea alot actually, the only think I can see as a downside
is that the SYNC=".." could be changed accdentally, making it just
another Gentoo tree.

Another thing that I don't like, is the feel of this method does seem
"offical" enough.. mostly because portage is not 'stable'-aware, Its
just using a stripped down tree.

I think your idea is good, its just the details that need to be worked
out, (how long to keep the trees?)
My little piece on GLEP19 seems to have just been obsoleted.

Perhaps more people could respond so we can see how everyone feels (I
want this route.)

Tux

>To facilitate "enterprise" usage, we break up the releases into a
>"desktop" and "server" set. This means the current
>"default-linux/$arch/2006.0" profile would be
>"default-linux/$arch/2006.0/desktop" with a
>"default-linux/$arch/2006.0/server" profile, also. The stages would be
>built against the "default-linux/$arch/2006.0" profile, which would have
>any USE, etc. that are common between desktop and server. During
>installation, the user can choose to use either the desktop or server
>profiles, or stay with the more "generic" 2006.0 profile (good for
>developers, etc. that might need components of both, or want a more
>minimal default set of USE flags). The desktop and server profiles will
>have a defined set of default USE flags that will benefit the most
>people, similar to how the current profiles are designed to be "desktop"
>profiles, to benefit the most people.
>
>
--
gentoo-dev@gentoo.org mailing list
Re: Gentoo "Stable" Portage/Releases [ In reply to ]
Chris Gianelloni wrote:

>> To facilitate "enterprise" usage, we break up the releases into a
>> "desktop" and "server" set. This means the current
>> "default-linux/$arch/2006.0" profile would be
>> "default-linux/$arch/2006.0/desktop" with a
>> "default-linux/$arch/2006.0/server" profile, also. The stages would be
>> built against the "default-linux/$arch/2006.0" profile, which would have
>> any USE, etc. that are common between desktop and server. During
>> installation, the user can choose to use either the desktop or server
>> profiles, or stay with the more "generic" 2006.0 profile (good for
>> developers, etc. that might need components of both, or want a more
>> minimal default set of USE flags). The desktop and server profiles will
>> have a defined set of default USE flags that will benefit the most
>> people, similar to how the current profiles are designed to be "desktop"
>> profiles, to benefit the most people.
>
Oh yea, I know how it would fit with GLEP 19 nicely, but I think you
might want to make it a seperate issue.

Tux
--
gentoo-dev@gentoo.org mailing list
Re: Gentoo "Stable" Portage/Releases [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andrew Muraco wrote:
| Another thing that I don't like, is the feel of this method does seem
| "offical" enough.. mostly because portage is not 'stable'-aware, Its
| just using a stripped down tree.

What do you want then? If an entire standalone tree distributed by
Gentoo doesn't feel official enough, what will?

Thanks,
Donnie
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)

iD8DBQFDwSWAXVaO67S1rtsRAo7rAJ9YAf+Z3UUsshKfURP71lKqL5PjLwCdGcem
czZJv0hCE0XbT9pjjZOtaiY=
=GdT8
-----END PGP SIGNATURE-----
--
gentoo-dev@gentoo.org mailing list
Re: Gentoo "Stable" Portage/Releases [ In reply to ]
Donnie Berkholz wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Andrew Muraco wrote:
> | Another thing that I don't like, is the feel of this method does seem
> | "offical" enough.. mostly because portage is not 'stable'-aware, Its
> | just using a stripped down tree.
>
> What do you want then? If an entire standalone tree distributed by
> Gentoo doesn't feel official enough, what will?
>
What I meant to say is, having this alternative tree method (as
described here) would mean that portage would handle everything the
exact same as it already does, which means that if someother tree was
accidently sync'd or replaced the local one, portage would react and
want to update everything, because its not 'aware' that there is a
difference in the first place. (now that I think about it, how likely is
it that something like that will happen?)

The method described here would also open up the oppurtunity for "fake"
enterprise trees, without having something to double check that the tree
that we have is indeed the one that is wanted, it would be possible for
a hacked rsync server (or a bogus one) to host the enterprise (stable)
trees with extra ebuilds which could compromise security (/me thinks of
emails warning about Microsoft's patchs and links which point to
infectious websites.)

Maybe this is something thats not very likely to happen, but it still is
important to note that an enterprise tree has to be secure.

Wkr,
Andrew Muraco
--
gentoo-dev@gentoo.org mailing list
Re: Gentoo "Stable" Portage/Releases [ In reply to ]
On Tue, 2006-01-10 at 23:57 -0500, Andrew Muraco wrote:
> What I meant to say is, having this alternative tree method (as
> described here) would mean that portage would handle everything the
> exact same as it already does, which means that if someother tree was
> accidently sync'd or replaced the local one, portage would react and
> want to update everything, because its not 'aware' that there is a
> difference in the first place. (now that I think about it, how likely is
> it that something like that will happen?)

So someone would "accidentally" change the SYNC variable, then
"accidentally" run "emerge --sync"? That seems rather unlikely.

> The method described here would also open up the oppurtunity for "fake"
> enterprise trees, without having something to double check that the tree
> that we have is indeed the one that is wanted, it would be possible for
> a hacked rsync server (or a bogus one) to host the enterprise (stable)
> trees with extra ebuilds which could compromise security (/me thinks of
> emails warning about Microsoft's patchs and links which point to
> infectious websites.)

What exactly makes the release trees any more vulnerable to this than
current portage?

> Maybe this is something thats not very likely to happen, but it still is
> important to note that an enterprise tree has to be secure.

You will notice that I *never* call it an "enterprise" tree, so don't
make that mistake yourself.

--
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux
Re: Re: Gentoo "Stable" Portage/Releases [ In reply to ]
On Wed, 2006-01-11 at 00:03 -0700, Duncan wrote:
> Remember, portage already has a decent amount of signed content
> verification builtin, and is getting more. Just because it's not
> currently used, as the debate on strength and keyring handling hasn't been
> settled, doesn't mean the capacity doesn't exist.

One other advantage with this is we will be starting from a known
portage version. This allows us to not have to worry about backwards
compatibility. Want Manifest2 (and no Manifest/digests)? So long as
the version of portage supports it, we can switch to it completely on
these trees.

> At this point it should be possible to develop a working enterprise
> security model along with the enterprise proposal and tree. Spec it out,
> put the keys in a special dir on a read-only mounted partition, and it'll
> be pretty hard to fake it on the fly, at least.

Again, please don't consider my tree proposal as anything "enterprise",
at all. While it can be used as a *basis* for enterprise work, it does
not need to be relegated to any specific usage. It is simply a release
tree, with frozen package versions.

--
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux
Re: Re: Gentoo "Stable" Portage/Releases [ In reply to ]
On Wed, Jan 11, 2006 at 10:38:30AM -0500, Chris Gianelloni wrote:
> On Wed, 2006-01-11 at 00:03 -0700, Duncan wrote:
> > Remember, portage already has a decent amount of signed content
> > verification builtin, and is getting more. Just because it's not
> > currently used, as the debate on strength and keyring handling hasn't been
> > settled, doesn't mean the capacity doesn't exist.
>
> One other advantage with this is we will be starting from a known
> portage version. This allows us to not have to worry about backwards
> compatibility.

Reliant on portage- we're sitting on forward/backward compatibility
handling for ebuilds (EAPI), few months before we cut over and require
people to be running an EAPI capable portage- that said, we don't have
any versionning yet for profiles.

Proposals welcome for that one, since it's required (recall the 2.0.50
bug for cascaded profiles, anyone? ;).
~harring
Re: Gentoo "Stable" Portage/Releases [ In reply to ]
Chris Gianelloni posted <1136575138.18383.77.camel@cgianelloni.nuvox.net>,
excerpted below, on Fri, 06 Jan 2006 14:18:58 -0500:

> Remember that the release trees would only be security fixes. No other
> package updates should be happening. This would allow for companies to
> actually certify their software against "Gentoo 2006.0", for example.

Not an unreasonable proposal. I've a couple of comments, however.
(Naturally. =8^)

1) AFAIK, most such certification would require nailing down a bit
further, including gcc version used to compile, and CFLAGS, among other
things. Basically, what they'd then be certifying against would be the
GRP releases. This could mean expanding them somewhat, altho it should be
fine to build software unrelated to that being certified, and unrelated to
the necessary boot environment, from source, without destroying the
certification, and that limits the required GRP package count somewhat.

2) I was going to say that without keyword support it might be difficult
to nail down the distinction between those running current and those
running stable, but I just realized it could/should be right there in the
repository and/or profile information, as I'm sure that'll need to be
reported in emerge info, once multi-repository gets full support. It
/will/ be a bit of extra bug tracking for either devs or bug-wranglers or
both, as devs not wanting the work of supporting "stale" packages will,
I'm sure, still get bugs assigned that belong to the stable tree only.
However, that should be minimal and manageable.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


--
gentoo-dev@gentoo.org mailing list
Re: Gentoo "Stable" Portage/Releases [ In reply to ]
Andrew Muraco posted <43C49047.5070302@leetworks.com>, excerpted below,
on Tue, 10 Jan 2006 23:57:43 -0500:

> The method described here would also open up the oppurtunity for "fake"
> enterprise trees, without having something to double check that the tree
> that we have is indeed the one that is wanted, it would be possible for a
> hacked rsync server (or a bogus one) to host the enterprise (stable) trees
> with extra ebuilds which could compromise security (/me thinks of emails
> warning about Microsoft's patchs and links which point to infectious
> websites.)

Remember, portage already has a decent amount of signed content
verification builtin, and is getting more. Just because it's not
currently used, as the debate on strength and keyring handling hasn't been
settled, doesn't mean the capacity doesn't exist.

At this point it should be possible to develop a working enterprise
security model along with the enterprise proposal and tree. Spec it out,
put the keys in a special dir on a read-only mounted partition, and it'll
be pretty hard to fake it on the fly, at least.

IOW, while it's certainly an issue that needs to be addressed, I'd
consider it no worse than anything else on the list, and probably
relatively minor compared to some of the other hurdles to be cleared on
the way to a decent enterprise Gentoo. I believe the biggest hurdles
will be finding the folks to do it and coordinating them to actually get
and keep it going.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


--
gentoo-dev@gentoo.org mailing list
Re: Re: Gentoo "Stable" Portage/Releases [ In reply to ]
Chris Gianelloni posted <1136993910.28257.5.camel@cgianelloni.nuvox.net>,
excerpted below, on Wed, 11 Jan 2006 10:38:30 -0500:

> Again, please don't consider my tree proposal as anything "enterprise", at
> all. While it can be used as a *basis* for enterprise work, it does not
> need to be relegated to any specific usage. It is simply a release tree,
> with frozen package versions.

Good point. "Ultra-stable" tree, then, or as I'm more likely to consider
it, "hoary old archaic" tree. =8^) To each his own, I guess...

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


--
gentoo-dev@gentoo.org mailing list