Mailing List Archive

Ryan Hill wrote:
> Steve Long wrote:
>> Robert Buchholz wrote:
>>> Since the tree itself is the best database of the packages available,
>>> anything else would be a lot more overhead.
>
>> I really don't agree, altho I could well be missing something. Surely there
>> should be a maintenance/QA database which tracks the tree and where you
>> could put information like this (ie a boolean flag for this instance) which
>> simply shouldn't be exposed to users. There's no need for it, it doesn't
>> effect them, and why should it go in the ebuilds where a maintainer might
>> delete it?
>
> I just use a local db to keep track of stuff like this, but haven't
> thought of a way to turn this into a service and i don't think it's
> really doable. I think you'd need an entry for every ebuild in portage,
> times the number of archs, times an unlimited number of arbitrary fields
> (one for each thing you're tracking). Something like, say, checking
> every package in the tree for GPL-2 or GPL-2+ licenses doesn't need the
> separate arch entries of course, but stuff like GCC testing definitely does.
>
> Even if you did manage to set this up, I wouldn't want to maintain it.

I don't want to sound negative and I like the idea a lot, but two things
are on my mind about this:

It should also sync with changes in the tree, like package removals,
additions and package moves.

When you're talking about it on ebuild base: When a version bump is out,
will it inherit the flags of the version before?


Regards,

Robert
--
gentoo-dev@gentoo.org mailing list

Steve Long schrieb:
>>> In terms of maintaining the metadata, am I right in thinking it's all
>>> just kept within the text files in the tree?
>> Since the tree itself is the best database of the packages available,
>> anything else would be a lot more overhead.
>>
> I really don't agree, altho I could well be missing something. Surely there
> should be a maintenance/QA database which tracks the tree and where you
> could put information like this (ie a boolean flag for this instance) which
> simply shouldn't be exposed to users. There's no need for it, it doesn't
> effect them, and why should it go in the ebuilds where a maintainer might
> delete it?

You are totally right. Though there's one little effect: Before someone
files a bug because the package does not depend on a CC, (s)he will have
to know whether it's already checked. But if the DB is public, that's
not an issue.


>> But I had the impression the idea was discarded anyway. So I should
>> focus my thoughts somewhere else :-)
> Please focus your thoughts wherever you wish. I gotta ask tho; what idea? I
> thought we were just talking about excess dependencies in the tree.

I somehow lost track of the messages in this thread, but the idea of
having large scale dependencies any system package needed seemed not
realistically doable.
If it's just about adding virtual/c-toolchain, was there arguments
against it?

Regards,

Robert
--
gentoo-dev@gentoo.org mailing list

Stephen Bennett wrote:
>> It's seems to be needed sometimes b/c it does change the order of
>> generated deplist(emerge -e world). AFAIK some packages dep on zlib
>> b/c of that.
>
> If you don't know about the unwritten yet near universal exception
> clause then you shouldn't be invoking it.
>
Could you spell out that exception clause, please?

--
gentoo-dev@gentoo.org mailing list

Stephen Bennett wrote:
> Steve Long wrote:
>> Could you spell out that exception clause, please?
> It doesn't translate well into words, but we'll go with something like
> "Unless you know exactly why the rule is there, understand fully the
> implications of breaking it, and know why it's a
> good idea in this particular case."
>
> However, if you're in a position to be invoking that clause, you should
> know about it anyway.

OK; the rule, AFAICT, is don't depend on an app that's in base profile. This
includes portage. There seemed to be probs with /not/ depending on zlib
tho?

--
gentoo-dev@gentoo.org mailing list

Stephen Bennett <spb@gentoo.org> wrote:

> > Could you spell out that exception clause, please?
>
> It doesn't translate well into words, but we'll go with something
> like "Unless you know exactly why the rule is there, understand
> fully the implications of breaking it, and know why it's a
> good idea in this particular case."
>
> However, if you're in a position to be invoking that clause, you
> should know about it anyway.

Can we skip the sekrit rulez crap and just spell it out? Really, how
does this help anyone?

Ciaran McCreesh <ciaranm@ciaranm.org> wrote:

> On Sat, 16 Dec 2006 12:46:30 -0600 Ryan Hill <dirtyepic@gentoo.org>
> wrote:

> | Can we skip the sekrit rulez crap and just spell it out? Really,
> | how does this help anyone?
>
> It's quite simple. You don't do it unless you are fully aware of the
> consequences. If you have to ask, you aren't fully aware of the
> consequences so you mustn't do it.

That's a flawed argument. Not knowing doesn't prevent you from asking,
and asking will inform you of the consequences, assuming the asked
isn't a complete tool.

Ryan Hill <dirtyepic@gentoo.org> wrote:

> Ciaran McCreesh <ciaranm@ciaranm.org> wrote:

> > It's quite simple. You don't do it unless you are fully aware of the
> > consequences. If you have to ask, you aren't fully aware of the
> > consequences so you mustn't do it.
>
> That's a flawed argument. Not knowing doesn't prevent you from
> asking, and asking will inform you of the consequences, assuming the
> asked isn't a complete tool.

Let me try this more diplomatically. How are we supposed to know if a
package that's depending on a system package is a bug or an exception if
we have no idea what the exception(s) is/are?

Stephen Bennett wrote:
> If you don't know about the unwritten yet near universal exception
> clause then you shouldn't be invoking it.

If it's universal, then why isn't it written somewhere? After all
this, we *still* haven't gotten an answer to why some packages
outside of the system target are depending on zlib. Is this a bug? If
not, what's the reason it's there? Let's document this reason, so we
don't have to go through this again in the future. It's that simple.

Alec Warner <antarus@gentoo.org> wrote:

> Hrm, I thought I wrote about this a while ago but I don't see it on
> archives.g.o so lets try again.
>
> > If your package is 'not important' meaning it will never be in
> > 'system' for any profile, you should not depend on anything in
> > 'system', as
> stuff in system should already be installed in a given (sane)
> configuration.
> >
> > If your package may be in 'system' in a given profile, you need to
> ensure your package builds in the proper order, with regards to other
> system packages. The classic example is zlib; if you need zlib to
> uncompress something, then you should put zlib in the deps; that way
> when someone is building say, a stage1, your package will build after
> zlib, instead of before it.
> >
> > You have to be careful in deciding what to specify, as doing
> > things
> incorrectly in this case can often cause dependency loops which are
> sometimes fun to debug; perl and openssl were infamous back in the
> day for this.
> >
> > Enterprising users would specify the 'doc' useflag. openssl
> > requires
> perl to generate its docs and perl requires openssl for some
> encryption stuff. Users would then complain about perl or openssl
> not building, or portage getting really pissed at them; the solution
> being to build openssl twice, once with USE="-doc" and then build
> perl, and then rebuild openssl with USE="doc". This certainly wasn't
> the only case where this occurred (see ML thread about shadow and
> it's dep on some other package I can't remember, although that was a
> while back as well).
> >
> > In conclusion, you need domain knowledge of system packages and
> portage behavior to make good choices here ;)
>
> Wow that pasted nastily; hopefully it shows up ok ;)
>
> In any case I'm sure there are some other exceptions but these are
> the main ones.

Cool, that's exactly what I was looking for.

thanks ;d

Ciaran McCreesh wrote:
> That one pulls us back into the lack of distinction between "stuff
> needed when compiling against this library" and "stuff this library
> needs to run".
>
Wouldn't your c-toolchain or a compiler eg for PERL or Java do?

> | or by using meta-packages.
>
> DEPEND="virtual/c-toolchain" would indeed be nice, but it's a rather
> large change...
>
How so? Isn't it simply a new meta?

--
gentoo-dev@gentoo.org mailing list

Ryan Hill wrote:
> Cool, that's exactly what I was looking for.
>
> thanks ;d

Yeah me too, thanks for a straight reply! ;)

--
gentoo-dev@gentoo.org mailing list

Alec Warner wrote:

> Jason Stubbs wrote:
>>
>>
>> There's ways to manage this complexity, such as putting the dependencies
>> into autotools' RDEPEND (if it can be considered correct) or by using
>> meta-packages. However, your point is against requiring that packages
>> _must_ specify all system dependencies. While I personally believe that
>> packages should specify all dependencies, what I'm arguing against is
>> requiring that packages _must not_ specify any system dependencies.
>>
>> --
>> Jason Stubbs
>
> I agree with your personal belief, however I also find it unmaintainable
> in the current system (metapackages in their current form
> non-withstanding as I don't think they are a great solution, merely duct
> tape if you will, but that is another discussion entirely).
>
> There is no benefit for me as a package maintainer to dep on a system
> package unless there is an existing problem. From a maintainer POV it's
> extra work and extra writing to keep the deps up to date. Also there is
> the whole thought of what to list? Do I list only glibc versions that I
> know work? gcc versions that I know compile my code? Where does the
> line get drawn? What is the point of depending on certain elements if
> say, they are already a dependency of $PACKAGE_MANAGER? It is not
> pragmatic for a distribution to do so IMHO, 'technically correct' or not.
>
I agree but I don't think Jason was saying that; just that he should
be /allowed/ to specify a dep; clearly it should be exceptional, and maybe
tracked as an issue with the pkg.

As you mention the worse is that an extra dep goes in. But if we take the
time to hammer out the policy now (so far I'm reading don't put in a system
dep unless you really have to, and even then it may indicate an upstream
problem) it'll at least be clear.


--
gentoo-dev@gentoo.org mailing list

Ciaran McCreesh wrote:
> On Mon, 18 Dec 2006 07:28:25 +0000 Steve Long wrote:
> | Ciaran McCreesh wrote:
> | > That one pulls us back into the lack of distinction between "stuff
> | > needed when compiling against this library" and "stuff this library
> | > needs to run".
> |
> | Wouldn't your c-toolchain or a compiler eg for PERL or Java do?
>
> You're missing the distinction. The easy example, but not the best, is
> pkg-config: many libraries must be used via pkg-config, so they need to
> RDEPEND upon it to avoid breaking binary packages. However, they don't
> actually require it at runtime. The other option, which is just about
> doable in this one particular case, is to make any package that uses a
> library that uses pkg-config DEPEND upon pkg-config.
>
What do you mean by "stuff needed when compiling against this library"? I'm
not understanding what distinction you're trying to make- a reverse-compile
dep? I can understand compile-time dependencies and run-time dependencies.
You seem to be talking about the compile-time dependency of any pkg
compiling against a library. Doh! I see what you mean, that isn't the same;
so effectively there's another type of dep.

How serious an issue is it in terms of deps on sys pkgs?

> | > DEPEND="virtual/c-toolchain" would indeed be nice, but it's a rather
> | > large change...
> | >
> | How so? Isn't it simply a new meta?
>
> And an entire tree to update before it becomes meaningful.
>
Sure, but the changes can propagate thru as people update their ebuilds, no?

--
gentoo-dev@gentoo.org mailing list

Ciaran McCreesh wrote:
> Steve Long wrote:
> | How serious an issue is it in terms of deps on sys pkgs?
>
> Very. It means we can't realistically handle packages that, by using
> autotools, depend upon the fifty odd system packages that are used by
> autotools-generated code.
>
> | > | > DEPEND="virtual/c-toolchain" would indeed be nice, but it's a
> | > | > rather large change...
> | > | >
> | > | How so? Isn't it simply a new meta?
> | >
> | > And an entire tree to update before it becomes meaningful.
> | >
> | Sure, but the changes can propagate thru as people update their
> | ebuilds, no?
>
> The tricky part then is figuring out whether something doesn't dep upon
> c-compiler because it doesn't need one or because the ebuilds haven't
> been updated.
>
I'm out of my depth here- I can't see where that would be a problem?

--
gentoo-dev@gentoo.org mailing list

Alec Warner wrote:
>>> The tricky part then is figuring out whether something doesn't dep upon
>>> c-compiler because it doesn't need one or because the ebuilds haven't
>>> been updated.
>>>
>> I'm out of my depth here- I can't see where that would be a problem?
>>
>
> Er, his point being that if you don't do the upgrade all at once, you
> have two classes of package.
>
> 1. Packages that don't require C-compiler
> 2. Packages that don't yet depend upon C-compiler
>
> When doing the upgrade over a period of time the two classes of package
> become indistinguishable. Does Foo not need a C compiler, or has it
> just not gotten updated yet, it's impossible to tell without looking, so
> it's very difficult for people to report 'problem packages' because they
> have to unpack and examine the package every time (at worst).
>
I understand that there'd be two types of pkg in the tree; what I don't get
is why that is such a problem. Excuse my missing something obvious. What do
you mean by a `problem package' in this context?

--
gentoo-dev@gentoo.org mailing list

Robert Buchholz wrote:
> A problem package would be one that does not need a C compiler. It can't
> be distinguished from the one which was not yet changed to depend on C.
>
> The problem here is that one can not say when the whole tree is updated
> to the new standard, because for the packages which were not touched, it
> could mean that they needed no change or that they were not looked at yet.
>
I can understand that as a maintenance issue. My query is whether having two
different types of pkg would effect users in any way.

> A solution to distinguish the two categories is to mark the packages
> which were "checked", so you know:
> 1. If it's checked and doesn't depend on cc -> category 1
> 2. If it's not checked and doesn't depend on cc -> category 2
>
> Then, when all packages are checked, the tree is upgraded.
>
Sure, that makes sense. It sounds a heckuva lot like a database ;)

Minor point- how can you tell in cat 2 that it definitely does not need a C
compiler if it hasn't been checked? I'm guessing you were tired :) In any
event in terms of maintenance, we'd need a tri-state: unchecked, checked
and needs compiling, checked and doesn't (eg scripts).

In terms of maintaining the metadata, am I right in thinking it's all just
kept within the text files in the tree?

--
gentoo-dev@gentoo.org mailing list

Robert Buchholz wrote:
> Steve Long wrote:
>> Robert Buchholz wrote:
>>> The problem here is that one can not say when the whole tree is updated
>>> to the new standard, because for the packages which were not touched, it
>>> could mean that they needed no change or that they were not looked at
>>> yet.
>> I can understand that as a maintenance issue. My query is whether having
>> two different types of pkg would effect users in any way.
>
> You're right, it would not be a problem for usability.
>
Well that makes a change ;) Although it sounds like a maintenance nightmare.

>
>>> A solution to distinguish the two categories is to mark the packages
>>> which were "checked", so you know:
>>> 1. If it's checked and doesn't depend on cc -> category 1
>>> 2. If it's not checked and doesn't depend on cc -> category 2
>>>
>>> Then, when all packages are checked, the tree is upgraded.
>>>
>> Sure, that makes sense. It sounds a heckuva lot like a database ;)
>>
>> Minor point- how can you tell in cat 2 that it definitely does not need a
>> C compiler if it hasn't been checked? I'm guessing you were tired :) In
>> any event in terms of maintenance, we'd need a tri-state: unchecked,
>> checked and needs compiling, checked and doesn't (eg scripts).
>
> No, no. I did not mean that "category 2" does not need a C compiler. I
> referred to Alec's mail which had "2. Packages that don't yet depend
> upon C-compiler".
> As for the state problem, when saving "checked" and "unchecked" as a
> package's metadata, you will get the other properties (needs cc, needs
> <insert system dep>) out of its DEPEND.
>
Alec Warner wrote:
> 1. Packages that don't require C-compiler
> 2. Packages that don't yet depend upon C-compiler
>
> When doing the upgrade over a period of time the two classes of package
> become indistinguishable. Does Foo not need a C compiler, or has it
> just not gotten updated yet, it's impossible to tell without looking, so
> it's very difficult for people to report 'problem packages' because they
> have to unpack and examine the package every time (at worst).
>
I understand that it's hard to distinguish a pkg that hasn't been checked,
but might need the C-compiler, from a pkg that doesn't need the compiler
but just hasn't been checked. That's where I was going with the database
stuff.

>
>> In terms of maintaining the metadata, am I right in thinking it's all
>> just kept within the text files in the tree?
>
> Since the tree itself is the best database of the packages available,
> anything else would be a lot more overhead.
>
I really don't agree, altho I could well be missing something. Surely there
should be a maintenance/QA database which tracks the tree and where you
could put information like this (ie a boolean flag for this instance) which
simply shouldn't be exposed to users. There's no need for it, it doesn't
effect them, and why should it go in the ebuilds where a maintainer might
delete it?

> But I had the impression the idea was discarded anyway. So I should
> focus my thoughts somewhere else :-)
>
Please focus your thoughts wherever you wish. I gotta ask tho; what idea? I
thought we were just talking about excess dependencies in the tree.

May your thoughts be pleasant :)


--
gentoo-dev@gentoo.org mailing list

Steve Long wrote:
> Robert Buchholz wrote:

> I understand that it's hard to distinguish a pkg that hasn't been checked,
> but might need the C-compiler, from a pkg that doesn't need the compiler
> but just hasn't been checked. That's where I was going with the database
> stuff.

>> Since the tree itself is the best database of the packages available,
>> anything else would be a lot more overhead.

> I really don't agree, altho I could well be missing something. Surely there
> should be a maintenance/QA database which tracks the tree and where you
> could put information like this (ie a boolean flag for this instance) which
> simply shouldn't be exposed to users. There's no need for it, it doesn't
> effect them, and why should it go in the ebuilds where a maintainer might
> delete it?

I just use a local db to keep track of stuff like this, but haven't
thought of a way to turn this into a service and i don't think it's
really doable. I think you'd need an entry for every ebuild in portage,
times the number of archs, times an unlimited number of arbitrary fields
(one for each thing you're tracking). Something like, say, checking
every package in the tree for GPL-2 or GPL-2+ licenses doesn't need the
separate arch entries of course, but stuff like GCC testing definitely does.

Even if you did manage to set this up, I wouldn't want to maintain it.


--
by design, by neglect
dirtyepic gentoo org for a fact or just for effect
9B81 6C9F E791 83BB 3AB3 5B2D E625 A073 8379 37E8 (0x837937E8)

Robert Buchholz wrote:

> I don't want to sound negative and I like the idea a lot, but two things
> are on my mind about this:
>
> It should also sync with changes in the tree, like package removals,
> additions and package moves.

For sure.

> When you're talking about it on ebuild base: When a version bump is out,
> will it inherit the flags of the version before?

I'd say no. The flags could easily change from version to version, even
from revbump to revbump. For example, a bump could introduce some kind
of QA violation not present in the previous ebuild or fail 'make test'
where the last version worked. Depends on what you're tracking I guess.


--
by design, by neglect
dirtyepic gentoo org for a fact or just for effect
9B81 6C9F E791 83BB 3AB3 5B2D E625 A073 8379 37E8 (0x837937E8)

Robert Buchholz wrote:
>>> But I had the impression the idea was discarded anyway. So I should
>>> focus my thoughts somewhere else :-)
>> Please focus your thoughts wherever you wish. I gotta ask tho; what idea?
>> I thought we were just talking about excess dependencies in the tree.
>
> I somehow lost track of the messages in this thread
Yeah, have to admit I'd been feeling like that too- had to go back and check
what we were actually talking about :P

> , but the idea of
> having large scale dependencies any system package needed seemed not
> realistically doable.
I thought it was to get rid of any dependencies on pkgs that were guaranteed
to be present, but I could be wrong.

> If it's just about adding virtual/c-toolchain, was there arguments
> against it?
Just the implementation difficulties, I thought. Which brought us onto a QA
db.


--
gentoo-dev@gentoo.org mailing list

Ryan Hill wrote:
> I just use a local db to keep track of stuff like this, but haven't
> thought of a way to turn this into a service and i don't think it's
> really doable. I think you'd need an entry for every ebuild in portage,
> times the number of archs, times an unlimited number of arbitrary fields
> (one for each thing you're tracking). Something like, say, checking
> every package in the tree for GPL-2 or GPL-2+ licenses doesn't need the
> separate arch entries of course, but stuff like GCC testing definitely
> does.
>
Agreed. Let's say 15000 ebuilds * 10 architectures or 150000 records of
actual instances of ebuilds. These could be linked to a table of pkg names,
so the first 3 tables and example uses that spring to mind are:

Pkg=> tracking of requires CC
Pkg-version=> ebuild maintenance
Pkg-version-arch=> bugs

It still doesn't strike me as an unfeasible amount of data for any of the
standard db servers. The custom bit would be code to keep it synced to the
portage tree, which doesn't strike me as all that hard. We could simply
capture the rsync output to see the changes for a start.

TBH I have no idea how useful such a db would be, I just find it an
interesting project.

> Even if you did manage to set this up, I wouldn't want to maintain it.
>
No fair enough, nor would I! All I'm up for maintaining is the automation
side, not the individual pkgs.


--
gentoo-dev@gentoo.org mailing list

Ryan Hill wrote:
> Robert Buchholz wrote:
>> I don't want to sound negative and I like the idea a lot, but two things
>> are on my mind about this:
>>
>> It should also sync with changes in the tree, like package removals,
>> additions and package moves.
>
> For sure.
>
>> When you're talking about it on ebuild base: When a version bump is out,
>> will it inherit the flags of the version before?
>
> I'd say no. The flags could easily change from version to version, even
> from revbump to revbump. For example, a bump could introduce some kind
> of QA violation not present in the previous ebuild or fail 'make test'
> where the last version worked. Depends on what you're tracking I guess.
>
++ to both, but I'm not an expert on ebuilds or the tree.


--
gentoo-dev@gentoo.org mailing list