Mailing List Archive

Re: GLEP 55 (was: A few questions to our nominees)
pioto@pioto.org wrote:
>> 1) Increase of [needless] complexity in filenames/extensions (and only one
>> example of the impact is that searching for ebuild files becomes less
>> straightforward), when things like SLOT, EAPI, etc., etc., seem to
>> naturally belong as part of the script contents/syntax.
>
> Okay... so:
> find /usr/portage -name *ebuild
>
> becomes:
> find /usr/portage -name *ebuild*
>
> That doesn't seem that much harder to me... Same for the file detection
> for editors.

I'm not saying it's a lot harder. But it is more complex and less
elegant. Also, it is error-prone. If someone, by habit, looks for all
"*.ebuild", he will miss a portion of the ebuilds and not even realize
it at first (or ever).

>> 2) Having the same info in more than one place is generally a bad idea in
>> any design - this is true in any discipline. [...]
>
> If you read the proposal more closely, you would notice that it
> specifically says to *not* specify EAPI in more than one place. It belongs
> solely in the filename suffix. The only reason the EAPI variable would be
> recognized inside the file itself is to allow for backwards compatibility
> with the current way EAPI=1 is done -- this behavior would be discouraged
> in all future ebuilds.

Still, the GLEP addresses the very point of what logic has to be
followed if the EAPI exists in the filename, in the file, or both. It
talks of "pre-source" EAPIs and "post-source" EAPIs. Complicated.

>> 3) It uses the extension in a way that goes against common use in computers,
>> especially *nix. No longer could we then say that a file of type
>> "Gentoo ebuild" has the extension "ebuild"
>> (simple/straightforward/standard).
>
> In most unixes, the file extension is completely meaningless. What matters
> is the contents of the file. Nautilus, etc, mostly use detection based
> upon the files contents to identify file types (at least for local files),
> not file extensoins.

That wasn't the point I was trying to make. I am not saying that the
extension has special meaning (even the "." has no meaning, really, in
unix) to software. I mean that people associate certain types of files
with certain extensions. I'm speaking more of the human interface here.

>> 4) It seems that the motivation for this GLEP is so that the tools can
>> determine the EAPI without reading/sourcing the script. In order to
>> get around this, the GLEP suggests exposing this technical information
>> in the filename. This is the wrong place to expose it, and the reasons
>> given are not that this detail should be exposed there but rather because
>> it is inefficient to source the file to get the info. So this is a case
>> of trying to solve a technical issue by changing the interface. I
>> believe it would be more correct to attack the technical problem in a way
>> that does not do this, and I am sure this can be solved.
>
> The reason for this is that, while the current two EAPIs don't cause
> trouble if you try to source them when you don't support them, that
> doesn't mean future ones won't. I'm not talking about anything silly like
> rewriting ebuilds in python, but things like changing syntax for DEPEND
> could potentially confuse older package managers. By adding the EAPI
> specification to the filename instead, old package managers just plain
> won't see packages they don't understand, so there's no need to worry
> about this.

Well, in general, if you rely on extensions changing every time a
program cannot deal with a new feature of a file format, it would be
quite crazy. For example, if C programs had to start using ".c-2",
".c-3", etc., it would get ugly fast. Also, it is easy to build EAPI
checking into portage now, and when the requisite time passes, you only
need to deal with situations where *very* old portage versions are still
in use. Since portage is typically the first thing the system upgrades
after a sync, I don't see a big issue. Or, if that is not acceptable,
see my comment at the end about a one-time change to a new extension
like ".eb".

> Also, sourcing a package to extract one metadata key takes much more time
> than just not loading it in the first place.

I understand there are technical/performance issues to solve, but this
does not, IMHO, justify moving this info into a filename/extension.

>> 1) Littering the filename with this kind of stuff is potentially confusing to
>> both devs and users - I know it's been stated that users shouldn't care,
>> but I don't think that's a good reason/assumption.
>
> New eapis aren't necessarily of any immediate use to people. Those who
> don't see the need for them can continue to just use EAPI=0, plain old
> .ebuild files of the sort that've been around for years, and for many
> packages this is totally appropriate. The only devs who should care are
> the ones who have a need for the new features.

But when they do need the new features, they need to use the new EAPIs.
This is not a matter of "degree" - it is a matter of defining the
filename format. Once you allow freeform ".ebuild-????" extensions,
everyone has to deal with the possibility of their existence.

> Users shouldn't ever have to read the ebuild files themselves. The package
> manager should tell them everything they need to know.

It doesn't matter, users still will look at the builds and the
directories they are in. And I am sure many users do this regularly.
We are not hiding the details of the files and filenames in an opaque
database where none of this would matter (we're not MicroSoft, after all
:). What Gentoo proudly calls the main portage tree is out there for
all to see and examine, which is a good thing.

But what users *really* don't care about is EAPIs, and this GLEP would
expose that technical detail to them in a very blatent way.

Everything else in the filename is something portage exposes to them in
its interface (category, package, version, revision, etc.), so this
would be the first thing in the filename that would *not* match up with
something we want to expose to users. EAPI is really something that
should be under the hood.

>> 2) It is not an elegant filename policy. Many systems have elegance as
>> a design goal.
>
> That's a matter of opinion. It seems perfectly elegant to me -- not to
> mention the various, rather clear technical benefits of it.

Yes, it is a matter of opinion, and I feel GLEP 55 would do things in a
very non-standard and confusing way, for users and for devs.

The fact that it makes the technical solution particularly easy (or an
easy short-cut) is not a justification.

>> 3) Assuming going this route is a mistake, it could be hard to reverse.
>
> Not really. The entire point of this scheme is that we can change at any
> time w/o breaking stuff. If we decide to go with .pbuild files for the
> future, we can just do that. Old package managers still won't care.

Along those lines, as I've said before, migrating to a new extension,
*one-time*, as a solution to this, although not optimal, would be far
more satisfactory than introducing a series of ever-changing extensions.

For example,
http://en.wikipedia.org/wiki/Alphabetical_list_of_file_extensions#E does
not list ".eb", so gentoo could adopt that one, slowly phasing out
".ebuild" over time. At least this would signify that it is an ".eb"
(ebuild) file, not that it is an ".ebuild-4", which happens to be a type
of ebuild file that requires version 4 of EAPI.

-Joe
--
gentoo-dev@lists.gentoo.org mailing list
Re: GLEP 55 (was: A few questions to our nominees) [ In reply to ]
On Mon, 09 Jun 2008 19:49:08 -0600
Joe Peterson <lavajoe@gentoo.org> wrote:
> I'm not saying it's a lot harder. But it is more complex and less
> elegant. Also, it is error-prone. If someone, by habit, looks for
> all "*.ebuild", he will miss a portion of the ebuilds and not even
> realize it at first (or ever).

Yes, if something changes, and people carry on doing the old thing by
habit, then things go wrong.

> Still, the GLEP addresses the very point of what logic has to be
> followed if the EAPI exists in the filename, in the file, or both. It
> talks of "pre-source" EAPIs and "post-source" EAPIs. Complicated.

And if the GLEP didn't address it people would complain that it allowed
undefined behaviour.

> Well, in general, if you rely on extensions changing every time a
> program cannot deal with a new feature of a file format, it would be
> quite crazy. For example, if C programs had to start using ".c-2",
> ".c-3", etc., it would get ugly fast.

Which is why programs that use any major C feature introduced since
1980 use the extension '.cc' or '.cpp'.

> Also, it is easy to build EAPI checking into portage now, and when
> the requisite time passes, you only need to deal with situations
> where *very* old portage versions are still in use. Since portage is
> typically the first thing the system upgrades after a sync, I don't
> see a big issue. Or, if that is not acceptable, see my comment at
> the end about a one-time change to a new extension like ".eb".

You completely miss the point of the GLEP. We need new extensions
precisely because current package managers can't handle future EAPIs
cleanly, and we need to carry on using new extensions because otherwise
we restrict what future EAPIs can do.

> But what users *really* don't care about is EAPIs, and this GLEP would
> expose that technical detail to them in a very blatent way.

Anyone who cares about ebuilds at a file level has to care about EAPIs.

> Along those lines, as I've said before, migrating to a new extension,
> *one-time*, as a solution to this, although not optimal, would be far
> more satisfactory than introducing a series of ever-changing
> extensions.

No it won't. It means future EAPIs will be restricted to some
particular source format.

--
Ciaran McCreesh
Re: GLEP 55 [ In reply to ]
Ciaran McCreesh wrote:
> On Mon, 09 Jun 2008 19:49:08 -0600
> Joe Peterson <lavajoe@gentoo.org> wrote:
>> I'm not saying it's a lot harder. But it is more complex and less
>> elegant. Also, it is error-prone. If someone, by habit, looks for
>> all "*.ebuild", he will miss a portion of the ebuilds and not even
>> realize it at first (or ever).
>
> Yes, if something changes, and people carry on doing the old thing by
> habit, then things go wrong.

Yes, if everyone is perfect and remembers to do things perfectly right,
there would never be issues in many things, but when you make something
more complicated, there will be more errors.

>> Well, in general, if you rely on extensions changing every time a
>> program cannot deal with a new feature of a file format, it would be
>> quite crazy. For example, if C programs had to start using ".c-2",
>> ".c-3", etc., it would get ugly fast.
>
> Which is why programs that use any major C feature introduced since
> 1980 use the extension '.cc' or '.cpp'.

So why would not a one-time new extension (e.g. ".eb") do the trick,
just like ".cc" works for C programs? Unless you are talking about
needing to specify the EAPI in the file if the more advanced features
are to be used, but I see nothing wrong with that requirement - it's not
much different than specifying a slot, keywords, whatever.

>> Also, it is easy to build EAPI checking into portage now, and when
>> the requisite time passes, you only need to deal with situations
>> where *very* old portage versions are still in use. Since portage is
>> typically the first thing the system upgrades after a sync, I don't
>> see a big issue. Or, if that is not acceptable, see my comment at
>> the end about a one-time change to a new extension like ".eb".
>
> You completely miss the point of the GLEP. We need new extensions
> precisely because current package managers can't handle future EAPIs
> cleanly, and we need to carry on using new extensions because otherwise
> we restrict what future EAPIs can do.

No, I get that. But once you develop the concept of an EAPI, the very
next package manager version can be aware of it and check the EAPI of an
ebuild. If the ebuild specifies none, then it is old-style. If it
specifies one that is unknown or newer than what that package manager
version knows it can handle, it can handle that case (ignore it or
whatever). I don't see why you need to keep bumping the
filename/extension every time it changes from that point forward.

>> But what users *really* don't care about is EAPIs, and this GLEP would
>> expose that technical detail to them in a very blatent way.
>
> Anyone who cares about ebuilds at a file level has to care about EAPIs.

Not really. A typical user does not need to know about EAPIs at all,
but he might want to peruse the portage tree to look for ebuilds. He
might also want to grep for KEYWORDS or whatever. The user can delve
into it as much as needed or desired, but if there are these mysterious
EAPI numbers tacked onto the extensions, then it's an added complication
that is not important to all users.

>> Along those lines, as I've said before, migrating to a new extension,
>> *one-time*, as a solution to this, although not optimal, would be far
>> more satisfactory than introducing a series of ever-changing
>> extensions.
>
> No it won't. It means future EAPIs will be restricted to some
> particular source format.

I assume you mean that EAPI needs to be in the file - again, is this
bad? Many file formats specify a file format version as part of the file.

-Joe

--
gentoo-dev@lists.gentoo.org mailing list
Re: GLEP 55 [ In reply to ]
On Mon, 09 Jun 2008 20:15:56 -0600
Joe Peterson <lavajoe@gentoo.org> wrote:
> Yes, if everyone is perfect and remembers to do things perfectly
> right, there would never be issues in many things, but when you make
> something more complicated, there will be more errors.

So we shouldn't ever change anything?

> > Which is why programs that use any major C feature introduced since
> > 1980 use the extension '.cc' or '.cpp'.
>
> So why would not a one-time new extension (e.g. ".eb") do the trick,
> just like ".cc" works for C programs? Unless you are talking about
> needing to specify the EAPI in the file if the more advanced features
> are to be used, but I see nothing wrong with that requirement - it's
> not much different than specifying a slot, keywords, whatever.

Because then we won't be able to change source compatibility again in
the future without introducing yet another new extension.

> > You completely miss the point of the GLEP. We need new extensions
> > precisely because current package managers can't handle future EAPIs
> > cleanly, and we need to carry on using new extensions because
> > otherwise we restrict what future EAPIs can do.
>
> No, I get that. But once you develop the concept of an EAPI, the very
> next package manager version can be aware of it and check the EAPI of
> an ebuild. If the ebuild specifies none, then it is old-style. If it
> specifies one that is unknown or newer than what that package manager
> version knows it can handle, it can handle that case (ignore it or
> whatever). I don't see why you need to keep bumping the
> filename/extension every time it changes from that point forward.

Because the package manager doesn't know how to extract the EAPI from
ebuilds whose EAPI it doesn't support. For example, an EAPI 2 ebuild
might look like this:

require mypackage using ANIMAL="monkey"

How do current package managers understand that the EAPI there is 2?

> >> But what users *really* don't care about is EAPIs, and this GLEP
> >> would expose that technical detail to them in a very blatent way.
> >
> > Anyone who cares about ebuilds at a file level has to care about
> > EAPIs.
>
> Not really. A typical user does not need to know about EAPIs at all,
> but he might want to peruse the portage tree to look for ebuilds. He
> might also want to grep for KEYWORDS or whatever. The user can delve
> into it as much as needed or desired, but if there are these
> mysterious EAPI numbers tacked onto the extensions, then it's an
> added complication that is not important to all users.

The typical user should be using a tool to query that sort of thing.

> >> Along those lines, as I've said before, migrating to a new
> >> extension, *one-time*, as a solution to this, although not
> >> optimal, would be far more satisfactory than introducing a series
> >> of ever-changing extensions.
> >
> > No it won't. It means future EAPIs will be restricted to some
> > particular source format.
>
> I assume you mean that EAPI needs to be in the file - again, is this
> bad? Many file formats specify a file format version as part of the
> file.

It's a pain in the ass, because it means no introducing new global
scope functions and no changing behaviour of existing global scope
functions.

Most file formats don't have to deal with the compatibility issues that
we do. For example, try feeding this C++ program to a C++0x compiler:

void foo(int x)
{
auto bool requires(x == 1);
}

Or this C++0x program to a C++ compiler:

template <std::Regular T_>
T__ && foo(T_ x)
{
std::list<std::list<T_>> l;
return x;
}

In both cases, you get user visible messy errors. That's not something
we have the luxury of being able to do.

--
Ciaran McCreesh
Re: GLEP 55 [ In reply to ]
Ciaran McCreesh wrote:
> On Mon, 09 Jun 2008 20:15:56 -0600
> Joe Peterson <lavajoe@gentoo.org> wrote:
>> Yes, if everyone is perfect and remembers to do things perfectly
>> right, there would never be issues in many things, but when you make
>> something more complicated, there will be more errors.
>
> So we shouldn't ever change anything?

Of course I don't mean that. But humans and computers are each good at
a complementary set of things. Computers handle obscure complexity
easily; humans do not, so it's better to let computers make our lives
easier rather than the reverse when designing systems.

>> So why would not a one-time new extension (e.g. ".eb") do the trick,
>> just like ".cc" works for C programs? Unless you are talking about
>> needing to specify the EAPI in the file if the more advanced features
>> are to be used, but I see nothing wrong with that requirement - it's
>> not much different than specifying a slot, keywords, whatever.
>
> Because then we won't be able to change source compatibility again in
> the future without introducing yet another new extension.

But GLEP 55 is suggesting exactly that: yet another extension for each
new EAPI (I know it is defines this as ".ebuild-<EAPI>", but that is
just semantics).

Source compatibility is not an issue once the EAPI syntax in the file is
defined and the package manager starts to recognize it. At that point
it can handle the ebuild at whatever EAPI the ebuild declares. It is
only the older unaware version of the package manager that would get
confused, but that would be solved by a one-time extension change: the
old one would not even look for the new (e.g.) ".eb" extension (only the
old ".ebuild" one), which is exactly what GLEP 55 tries to address. But
the extension change is only needed once. Once the EAPI syntax is
introduced and the package manager has code to read it, the package
manager is able to determine the EAPI for all future EAPIs. If some new
syntax in an as-yet unsupported EAPI exists, the EAPI-version-aware
package manager will not trip on it, since it can just not bother to
deal with "future" EAPI ebuilds.

Now, even if there is no extension change, if we wait long enough, the
chances of an old machine stubbornly staying at an old (pre-EAPI-aware)
portage version gets slimmer and slimmer. So I'm not even sure this
one-time extension change is really mandatory. But if it is determined
to be so, I still don't see why we need endless extension changes as
suggested in GLEP 55.

> Because the package manager doesn't know how to extract the EAPI from
> ebuilds whose EAPI it doesn't support. For example, an EAPI 2 ebuild
> might look like this:
>
> require mypackage using ANIMAL="monkey"
>
> How do current package managers understand that the EAPI there is 2?

The old (non-aware) package manager version would not, and yes, it would
fail. So there are two alternatives: wait long enough or do a one-time
extension change. In the latter case, the package manager would not
even see the new files. But the new package manager versions would
determine the EAPI from a defined syntax and ignore ebuilds with
"future" EAPIs.

And I do understand the issue of sourcing, since ebuilds are bash. If
sourcing is an issue (and I'm not sure it is an overriding one - that's
a good discussion), I would suggest an out-of-band EAPI specifier, but
not in the filename. Put it in a comment line in the header, like:

# Copyright 1999-2008 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: /var/cvsroot/gentoo-x86/sys-fs/btrfs-...$
# EAPI=2

inherit eutils
...

So, the first non-blank and non-'#' line (in this case, "inherit ...")
would signify the end of the search for the EAPI= string, making parsing
trivial. Therefore, the only rule would have to be that "EAPI=" needs
to be in a header comment. Other rules could be that it needs to be the
only thing on such a header line - whatever.

Again, these are technical details, and I think we can all put our heads
together to come up with the best way to do it.

If sourcing is a better way to go (i.e. to allow EAPI= to be anywhere in
the file with no comment char), caching it might be the answer. How to
make this efficient would become an implementation detail.

>>>> But what users *really* don't care about is EAPIs, and this GLEP
>>>> would expose that technical detail to them in a very blatent way.
>>> Anyone who cares about ebuilds at a file level has to care about
>>> EAPIs.
>> Not really. A typical user does not need to know about EAPIs at all,
>> but he might want to peruse the portage tree to look for ebuilds. He
>> might also want to grep for KEYWORDS or whatever. The user can delve
>> into it as much as needed or desired, but if there are these
>> mysterious EAPI numbers tacked onto the extensions, then it's an
>> added complication that is not important to all users.
>
> The typical user should be using a tool to query that sort of thing.

Sure, but my point is: some users *will* want to explore - why not
encourage this? And if so, why not make the conventions used as clean
and understandable (and elegant) as possible without added noise from
details that do not belong at that (e.g. the filename) level?

Gentoo is a technical distro, and we encourage users to get technical in
every other way. Saying that they should remain at the portage
interface level is not consistent with that philosophy.

>> I assume you mean that EAPI needs to be in the file - again, is this
>> bad? Many file formats specify a file format version as part of the
>> file.
>
> It's a pain in the ass, because it means no introducing new global
> scope functions and no changing behaviour of existing global scope
> functions.
>
> Most file formats don't have to deal with the compatibility issues that
> we do. For example, try feeding this C++ program to a C++0x compiler...

Right: there are two things to consider: 1) do we want EAPI= to be in
the global bash scope or out of band?, and 2) what happens when the
syntax has errors?

#1 needs more discussion, and #2, yes, should throw an error if invalid,
but this also will be caught when the dev tests his/her ebuild (and/or
by repoman), just as it is today, so I don't see a problem there. If it
is an EAPI mismatch that would cause the syntax error, the updated
package manager versions would handle it.

-Joe

--
gentoo-dev@lists.gentoo.org mailing list
Re: GLEP 55 [ In reply to ]
Joe Peterson schrieb:
> Ciaran McCreesh wrote:
>> On Mon, 09 Jun 2008 19:49:08 -0600
>> Joe Peterson <lavajoe@gentoo.org> wrote:
>>> Well, in general, if you rely on extensions changing every time a
>>> program cannot deal with a new feature of a file format, it would be
>>> quite crazy. For example, if C programs had to start using ".c-2",
>>> ".c-3", etc., it would get ugly fast.
>> Which is why programs that use any major C feature introduced since
>> 1980 use the extension '.cc' or '.cpp'.
>
> So why would not a one-time new extension (e.g. ".eb") do the trick,
> just like ".cc" works for C programs? Unless you are talking about
> needing to specify the EAPI in the file if the more advanced features
> are to be used, but I see nothing wrong with that requirement - it's not
> much different than specifying a slot, keywords, whatever.
Because that is about the same "damage" (file ext. changes, people might
get confused etc.) with less capabilities.

>>> Also, it is easy to build EAPI checking into portage now, and when
>>> the requisite time passes, you only need to deal with situations
>>> where *very* old portage versions are still in use. Since portage is
>>> typically the first thing the system upgrades after a sync, I don't
>>> see a big issue. Or, if that is not acceptable, see my comment at
>>> the end about a one-time change to a new extension like ".eb".
>> You completely miss the point of the GLEP. We need new extensions
>> precisely because current package managers can't handle future EAPIs
>> cleanly, and we need to carry on using new extensions because otherwise
>> we restrict what future EAPIs can do.
>
> No, I get that. But once you develop the concept of an EAPI, the very
> next package manager version can be aware of it and check the EAPI of an
> ebuild. If the ebuild specifies none, then it is old-style. If it
> specifies one that is unknown or newer than what that package manager
> version knows it can handle, it can handle that case (ignore it or
> whatever). I don't see why you need to keep bumping the
> filename/extension every time it changes from that point forward.
Because you can change the EAPI in a way that that may not work anymore.
Specifying the EAPI outside the actual ebuild is more flexible.
It doesn't have to be the file extension, but that is the obvious solution.

>>> But what users *really* don't care about is EAPIs, and this GLEP would
>>> expose that technical detail to them in a very blatent way.
>> Anyone who cares about ebuilds at a file level has to care about EAPIs.
>
> Not really. A typical user does not need to know about EAPIs at all,
> but he might want to peruse the portage tree to look for ebuilds. He
> might also want to grep for KEYWORDS or whatever. The user can delve
> into it as much as needed or desired, but if there are these mysterious
> EAPI numbers tacked onto the extensions, then it's an added complication
> that is not important to all users.
No, not really. If you have .txt, .txt-2, .text or .footext in a dir,
you would still realize, that those should be text files.
Of course, a future EAPI could be named .whatevercomestoyourmind, but
first, you can expect people to be smart enough to not do that and
second, you can still identify the packages, because they are still
named foo-version.whatevercomestoyourmind.

Bernd


--
gentoo-dev@lists.gentoo.org mailing list
Re: GLEP 55 [ In reply to ]
On Mon, 09 Jun 2008 21:36:24 -0600
Joe Peterson <lavajoe@gentoo.org> wrote:
> Of course I don't mean that. But humans and computers are each good
> at a complementary set of things. Computers handle obscure complexity
> easily; humans do not, so it's better to let computers make our lives
> easier rather than the reverse when designing systems.

And a file extension is far less obscurely complex than enforcing
arbitrary syntax restrictions upon ebuilds.


> >> So why would not a one-time new extension (e.g. ".eb") do the
> >> trick, just like ".cc" works for C programs? Unless you are
> >> talking about needing to specify the EAPI in the file if the more
> >> advanced features are to be used, but I see nothing wrong with
> >> that requirement - it's not much different than specifying a slot,
> >> keywords, whatever.
> >
> > Because then we won't be able to change source compatibility again
> > in the future without introducing yet another new extension.
>
> But GLEP 55 is suggesting exactly that: yet another extension for each
> new EAPI (I know it is defines this as ".ebuild-<EAPI>", but that is
> just semantics).

GLEP 55 suggests a backwards compatible, forwards compatible way of
dealing with the problem that doesn't involve adding new sets of rules
every few EAPIs.

> Source compatibility is not an issue once the EAPI syntax in the file
> is defined and the package manager starts to recognize it. At that
> point it can handle the ebuild at whatever EAPI the ebuild declares.

No it can't. EAPI has to be known before the source can start. Bash
doesn't provide traps for executing code upon changed variables.

> It is only the older unaware version of the package manager that
> would get confused, but that would be solved by a one-time extension
> change: the old one would not even look for the new (e.g.) ".eb"
> extension (only the old ".ebuild" one), which is exactly what GLEP 55
> tries to address. But the extension change is only needed once.

No, it's only needed once per non-trivial change. So we might as well
just change it for every EAPI.

> Once the EAPI syntax is introduced and the package manager has code
> to read it, the package manager is able to determine the EAPI for all
> future EAPIs.

Which means we can't change anything useful in future EAPIs. Which,
funnily enough, is what the GLEP is designed to solve.

> Now, even if there is no extension change, if we wait long enough, the
> chances of an old machine stubbornly staying at an old
> (pre-EAPI-aware) portage version gets slimmer and slimmer. So I'm
> not even sure this one-time extension change is really mandatory.

Except it is, because current EAPI aware package managers still can't
deal with global scope changes.

> > Because the package manager doesn't know how to extract the EAPI
> > from ebuilds whose EAPI it doesn't support. For example, an EAPI 2
> > ebuild might look like this:
> >
> > require mypackage using ANIMAL="monkey"
> >
> > How do current package managers understand that the EAPI there is 2?
>
> The old (non-aware) package manager version would not, and yes, it
> would fail. So there are two alternatives: wait long enough or do a
> one-time extension change. In the latter case, the package manager
> would not even see the new files. But the new package manager
> versions would determine the EAPI from a defined syntax and ignore
> ebuilds with "future" EAPIs.

And then how do we deal with EAPI 3, where the syntax changes again?

> And I do understand the issue of sourcing, since ebuilds are bash. If
> sourcing is an issue (and I'm not sure it is an overriding one -
> that's a good discussion), I would suggest an out-of-band EAPI
> specifier, but not in the filename. Put it in a comment line in the
> header, like:
>
> # Copyright 1999-2008 Gentoo Foundation
> # Distributed under the terms of the GNU General Public License v2
> # $Header: /var/cvsroot/gentoo-x86/sys-fs/btrfs-...$
> # EAPI=2

Which is way more obscure, complex and arbitrary than a file extension
change. And it still imposes massive restrictions upon future EAPIs.

> Again, these are technical details, and I think we can all put our
> heads together to come up with the best way to do it.

Every issue you've raised so far was already discussed and debunked the
first time this discussion happened. Please read the original
discussions before continuing.

> > The typical user should be using a tool to query that sort of thing.
>
> Sure, but my point is: some users *will* want to explore - why not
> encourage this? And if so, why not make the conventions used as clean
> and understandable (and elegant) as possible without added noise from
> details that do not belong at that (e.g. the filename) level?

And when they do explore, they learn straight away what EAPI is.

> Gentoo is a technical distro, and we encourage users to get technical
> in every other way. Saying that they should remain at the portage
> interface level is not consistent with that philosophy.

And users who get technical knowing what EAPI is is a good thing.

> Right: there are two things to consider: 1) do we want EAPI= to be in
> the global bash scope or out of band?, and 2) what happens when the
> syntax has errors?
>
> #1 needs more discussion

We had that discussion when the GLEP was first proposed.

--
Ciaran McCreesh
Re: GLEP 55 [ In reply to ]
Ciaran McCreesh wrote:
> And a file extension is far less obscurely complex than enforcing
> arbitrary syntax restrictions upon ebuilds.

I disagree. One is exposed to devs only as ebuild syntax; the other is
exposed in an inappropriate location to everyone looking at the portage
tree.

> No it can't. EAPI has to be known before the source can start. Bash
> doesn't provide traps for executing code upon changed variables.

Doing it out-of-band solve this.

> No, it's only needed once per non-trivial change. So we might as well
> just change it for every EAPI.

Huh? If the "new" portage knows how to determine the EAPI definitively
(and that would be defined), it can deal with the differences.

> And then how do we deal with EAPI 3, where the syntax changes again?

Portage (or whatever PM) reads the EAPI, determines it is 3, and goes
from there. If you change the way you declare EAPI each time, yeah,
that's a problem, but I'm not sure why that would ne necessary.

> Which is way more obscure, complex and arbitrary than a file extension
> change. And it still imposes massive restrictions upon future EAPIs.

Massive? Simply a one-line EAPI declaration is not massive nor complex.
And is more elegant than putting it in the filename.

> Every issue you've raised so far was already discussed and debunked the
> first time this discussion happened. Please read the original
> discussions before continuing.

Debunked according to whom? I believe that some, including you, believe
you debunked them, but I do not believe there was wholesale agreement
from the dev community.

> We had that discussion when the GLEP was first proposed.

Yes, but nothing was decided, and agreement was not reached. I'd be
very surprized if I were the only one here who is not entirely satisfied
with GLEP 55's solution to this.

-Joe
--
gentoo-dev@lists.gentoo.org mailing list
Re: GLEP 55 [ In reply to ]
On Mon, 09 Jun 2008 22:09:04 -0600
Joe Peterson <lavajoe@gentoo.org> wrote:
> Ciaran McCreesh wrote:
> > And a file extension is far less obscurely complex than enforcing
> > arbitrary syntax restrictions upon ebuilds.
>
> I disagree. One is exposed to devs only as ebuild syntax; the other
> is exposed in an inappropriate location to everyone looking at the
> portage tree.

You might as well say "We should get rid of Manifest because anyone
looking at the tree is exposed to security internals"...

> > No it can't. EAPI has to be known before the source can start. Bash
> > doesn't provide traps for executing code upon changed variables.
>
> Doing it out-of-band solve this.

Doing it out-of-band but in-the-file means the file format is fixed in
annoying ways.

> > No, it's only needed once per non-trivial change. So we might as
> > well just change it for every EAPI.
>
> Huh? If the "new" portage knows how to determine the EAPI
> definitively (and that would be defined), it can deal with the
> differences.

Sure, until there's another format change. Then we need yet another new
extension. What will we use then? '.ebld'?

The EAPI-in-extension format is cheap. People have been using it for
character sets and languages for decades.

> > And then how do we deal with EAPI 3, where the syntax changes again?
>
> Portage (or whatever PM) reads the EAPI, determines it is 3, and goes
> from there. If you change the way you declare EAPI each time, yeah,
> that's a problem, but I'm not sure why that would ne necessary.

But you're not sure that it's not necessary, so why impose entirely
pointless restrictions that everyone in the future has to stick by?

> > Which is way more obscure, complex and arbitrary than a file
> > extension change. And it still imposes massive restrictions upon
> > future EAPIs.
>
> Massive? Simply a one-line EAPI declaration is not massive nor
> complex. And is more elegant than putting it in the filename.

You're suddenly imposing restrictions upon the content of comments, and
requiring a whole new parser to deal with it. The point of comments is
that they're ignored.

> > Every issue you've raised so far was already discussed and debunked
> > the first time this discussion happened. Please read the original
> > discussions before continuing.
>
> Debunked according to whom? I believe that some, including you,
> believe you debunked them, but I do not believe there was wholesale
> agreement from the dev community.

That doesn't really matter. Most of the dev community don't care to
understand the underlying issue, so all they need to do is go along
with the informed decision that the Council was supposed to have made
on their behalf.

> > We had that discussion when the GLEP was first proposed.
>
> Yes, but nothing was decided, and agreement was not reached. I'd be
> very surprized if I were the only one here who is not entirely
> satisfied with GLEP 55's solution to this.

Yet GLEP 55 is the only solution that's been proposed that solves the
requirements. And your entire argument boils down to "file extension
changes don't look pretty", for some arbitrary value of pretty that
also precludes index.html.en and index.html.utf-8.

--
Ciaran McCreesh
Re: GLEP 55 [ In reply to ]
On 05:20 Tue 10 Jun , Ciaran McCreesh wrote:
> Yet GLEP 55 is the only solution that's been proposed that solves the
> requirements. And your entire argument boils down to "file extension
> changes don't look pretty", for some arbitrary value of pretty that
> also precludes index.html.en and index.html.utf-8.

Did anyone already propose specifying this in metadata.xml? Clearly one
downside is that PMs would need to be able to parse it to do anything,
but that would also enable us to do a lot more with metadata.xml that we
currently don't because we can't have PMs rely on it.

Thanks,
Donnie
--
gentoo-dev@lists.gentoo.org mailing list
Re: GLEP 55 [ In reply to ]
On Mon, 9 Jun 2008 22:35:25 -0700
Donnie Berkholz <dberkholz@gentoo.org> wrote:
> Did anyone already propose specifying this in metadata.xml?

Yup. That's a no-go, since metadata.xml is quite rightly treated as
being "not suitable for anything the package manager really needs".

It also moves the EAPI definition even further away from the ebuild,
which makes it even harder to work with.

And, of course, it's not backwards compatible, so it'd still need a
file extension change.

--
Ciaran McCreesh
Re: Re: GLEP 55 [ In reply to ]
On Tue, 10 Jun 2008 10:33:34 +0200
Tiziano Müller <dev-zero@gentoo.org> wrote:
> Another ugly solution: Having the EAPI on a per-package (like
> $portagedir/cat/package-1) or per-tree basis
> ($portagedir/profiles/eapi) and start providing our tree as overlays
> of more than one tree (will end up in a mess of dependencies, but it
> would still be nice to specify the EAPI for a complete overlay
> instead of having the name all the ebuilds like .eapi-X :-).
> In addition: it wouldn't be possible to identify the EAPI of an
> ebuild by just looking at it...

Kills the upgrade path completely. No good.

--
Ciaran McCreesh
Re: Re: GLEP 55 [ In reply to ]
Tiziano Müller wrote:
> Another ugly solution: Having the EAPI on a per-package (like
> $portagedir/cat/package-1) or per-tree basis ($portagedir/profiles/eapi)

I like the per tree basis and I already asked about that (since makes
things clearer and older portage can be tricked by rsync.

lu

--

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

--
gentoo-dev@lists.gentoo.org mailing list
Re: Re: GLEP 55 [ In reply to ]
Ciaran McCreesh wrote:
> Kills the upgrade path completely. No good.

Not really, you change the rsync paths and older portage will pick a
repo that just has the necessary to upgrade to the next portage.

This kind of things would work better using an scm supporting branches
and tags a bit better.

lu


--

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

--
gentoo-dev@lists.gentoo.org mailing list
Re: Re: GLEP 55 [ In reply to ]
Tiziano Müller wrote:
> Joe Peterson wrote:
>
>> Ciaran McCreesh wrote:
>>> And a file extension is far less obscurely complex than enforcing
>>> arbitrary syntax restrictions upon ebuilds.
>> I disagree. One is exposed to devs only as ebuild syntax; the other is
>> exposed in an inappropriate location to everyone looking at the portage
>> tree.
>>
>>> No it can't. EAPI has to be known before the source can start. Bash
>>> doesn't provide traps for executing code upon changed variables.
>> Doing it out-of-band solve this.
>>
>>> No, it's only needed once per non-trivial change. So we might as well
>>> just change it for every EAPI.
>> Huh? If the "new" portage knows how to determine the EAPI definitively
>> (and that would be defined), it can deal with the differences.
>>
>>> And then how do we deal with EAPI 3, where the syntax changes again?
>> Portage (or whatever PM) reads the EAPI, determines it is 3, and goes
>> from there. If you change the way you declare EAPI each time, yeah,
>> that's a problem, but I'm not sure why that would ne necessary.
> No, that is not the problem.
>
> Example:
> In EAPI 42 we define that the package manager must provide a global function
> extract_depend_from_setup_py() such that it is callable at a global level
> in an ebuild like this
>
> *snip start*
>
> # Copyright 1999-2007 Gentoo Foundation
> # Distributed under the terms of the GNU General Public License v2
> # $Header: $
>
> EAPI=42
>
> DESCRIPTION="A library aiming to support agile and test-driven python
> development on various levels."
> SRC_URI="http://codespeak.net/download/${PN}/${P}.tar.gz"
> HOMEPAGE="http://codespeak.net/py/"
> KEYWORDS="~amd64 ~x86"
> SLOT="0"
> LICENSE="MIT"
> IUSE=""
>
> extract_depend_from_setup_py
>
> *snip end*
>
> Now, a package manager (or a tool) not knowing EAPI 42 will fail when it
> tries to source the above ebuild to determine the EAPI version (as it is
> being currently done as far as I understood it) because
> extract_depend_from_setup_py is undefined.
> So it won't even be able to read out the EAPI.
>
> With the EAPI in the filename tools now knowing EAPI-42 will either ignore
> the above foo-1.0.ebuild-42 or mask it because they may identify the
> EAPI-version without sourcing the ebuild.
>

Check if exists a line EAPI=*$, if does and the rest of the string
matches an understood eapi, go on sourcing, otherwise ignore/mask it...

lu

--

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

--
gentoo-dev@lists.gentoo.org mailing list
Re: Re: GLEP 55 [ In reply to ]
On Tue, 10 Jun 2008 11:40:35 +0200
Luca Barbato <lu_zero@gentoo.org> wrote:
> Ciaran McCreesh wrote:
> > Kills the upgrade path completely. No good.
>
> Not really, you change the rsync paths and older portage will pick a
> repo that just has the necessary to upgrade to the next portage.
>
> This kind of things would work better using an scm supporting
> branches and tags a bit better.

picard_facepalm.jpg

--
Ciaran McCreesh
Re: Re: GLEP 55 [ In reply to ]
Ciaran McCreesh a écrit :
> Kills the upgrade path completely. No good.

Lemme sum this up in layman's terms :

1) EAPI _has_ to be known before sourcing an ebuild. There's no way to
avoid that for various reasons, all 100% valid.

2) Putting the EAPI in the filename :

+ it solves 1)
+ it keeps backward compatibility because old PM won't recognize the
filenames
- it's not very "pretty"

3) Putting the EAPI in metadata.xml or in another external file

+ it solves 1)
+ it keeps pretty file names
- it breaks backwards compatibility
- (specific to metadata.xml) PM will have to learn to read XML (yuck)

That's about it, right?


--
Rémi Cardona
LRI, INRIA
remi.cardona@lri.fr
remi@gentoo.org
--
gentoo-dev@lists.gentoo.org mailing list
Re: Re: Re: GLEP 55 [ In reply to ]
Tiziano Müller wrote:
> ... and package managers which don't do that already still fail.

To put everything in perspective all this discussion is done in order to
workaround the issue of an old and outdated package manager that cannot
be upgraded once it syncs from a too new repository.

The simplest way is to change the syncpoint in the new package manager
and leave the previous uri with a compatibility repo for the older ones.

lu

--

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

--
gentoo-dev@lists.gentoo.org mailing list
Re: Re: Re: GLEP 55 [ In reply to ]
On Tue, 10 Jun 2008 12:22:03 +0200
Luca Barbato <lu_zero@gentoo.org> wrote:
> Tiziano Müller wrote:
> > ... and package managers which don't do that already still fail.
>
> To put everything in perspective all this discussion is done in order
> to workaround the issue of an old and outdated package manager that
> cannot be upgraded once it syncs from a too new repository.
>
> The simplest way is to change the syncpoint in the new package
> manager and leave the previous uri with a compatibility repo for the
> older ones.

So you're volunteering to convert the entire tree to the new EAPI all
in one go every two months?

--
Ciaran McCreesh
Re: Re: GLEP 55 [ In reply to ]
Ciaran McCreesh a écrit :
> picard_facepalm.jpg

I don't think any of us are completely thrilled by either proposals, but
the EAPI-in-a-separate-file does have the potential for more
flexibility, ie package-wide EAPI.

And it does keep filenames simple enough.

--
Rémi Cardona
LRI, INRIA
remi.cardona@lri.fr
remi@gentoo.org
--
gentoo-dev@lists.gentoo.org mailing list
Re: Re: Re: GLEP 55 [ In reply to ]
> The simplest way is to change the syncpoint in the new package manager and
> leave the previous uri with a compatibility repo for the older ones.

So we add a new repo each time a new EAPI comes out? Sounds like a big mess.

--
Best Regards,
Piotr Jaroszyński
éí¢‡^¾X¬¶ÈžÚ(¢¸&j)bž b²
Re: Re: GLEP 55 [ In reply to ]
On 10 Jun 2008, at 12:30, Rémi Cardona wrote:

> Ciaran McCreesh a écrit :
>> picard_facepalm.jpg
>
> I don't think any of us are completely thrilled by either proposals,
> but the EAPI-in-a-separate-file does have the potential for more
> flexibility, ie package-wide EAPI.
>
> And it does keep filenames simple enough.

And it is not backwards compatible.

- ferdy--
gentoo-dev@lists.gentoo.org mailing list
Re: GLEP 55 [ In reply to ]
Ciaran McCreesh wrote:
> On Mon, 09 Jun 2008 22:09:04 -0600
> Joe Peterson <lavajoe@gentoo.org> wrote:
>> Debunked according to whom? I believe that some, including you,
>> believe you debunked them, but I do not believe there was wholesale
>> agreement from the dev community.
>
> That doesn't really matter. Most of the dev community don't care to
> understand the underlying issue, so all they need to do is go along
> with the informed decision that the Council was supposed to have made
> on their behalf.
>

By that logic (that mere ordinary devs need to be guided by their more
informed elected leaders), we're doing EXACTLY what you propose. The
informed decision by the Council was to defer the GLEP. As a result the
lesser devs are going along with that decision and not implementing it.

The fact is that most devs probably do care to understand the underlying
issues. Many just happen to disagree with you.

I still haven't seen a good argument as to why the EAPI can't be handled
as something like a magic number. Maybe make the first line in the ebuild:
#EAPI=foo

ELF vs A.OUT don't use extensions to identify its files - the original
file format was designed to allow file identification from the contents.

The only issue I can see with putting the EAPI on the first line is with
legacy package managers. That could be fixed in a number of ways. The
simplest would be just allowing it to break and sending out the fix well
in advance of any breakage on GMN. Considering the fiasco that we've
had with some GCC/GLIBC upgrades and similar stuff in the past this
would be pretty minor. A less pretty solution might be a one-time file
extension change.

There wouldn't be any need to repeat all of this every time the EAPI
changes - at that point we've standardized the location of the EAPI
within the file.
--
gentoo-dev@lists.gentoo.org mailing list
Re: Re: Re: GLEP 55 [ In reply to ]
Piotr Jaroszyński wrote:
>> The simplest way is to change the syncpoint in the new package manager and
>> leave the previous uri with a compatibility repo for the older ones.
>
> So we add a new repo each time a new EAPI comes out? Sounds like a big mess.

It isn't you just keep 2 repos, one with the minimal eapi and the
minimal set of ebuilds needed to upgrade, one with the latest and greatest.

lu

--

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

--
gentoo-dev@lists.gentoo.org mailing list
Re: Re: Re: GLEP 55 [ In reply to ]
Ciaran McCreesh wrote:
> So you're volunteering to convert the entire tree to the new EAPI all
> in one go every two months?

I don't see the need and I won't see the problem given right now what is
interesting is the set of improvements that aren't forward incompatible.

Being that the case you'd just need 2 trees, managed as overlay and a
marker for each tree on which eapi to use, but I dislike empty theories
or hardly searched corner cases that could be avoided with half of the
effort necessary to get there.


lu

--

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

--
gentoo-dev@lists.gentoo.org mailing list

1 2 3 4 5  View All