Mailing List Archive

Do (old-ish) Portage QA checks comprise policy?
Hi,

TL;DR: If a QA check is enforced by Portage for a reasonably long time,
does it constitute policy? Or can it be changed unilaterally by Portage
team?


William Hubbs has recently attempted to remove one of Portage's QA
checks [1]. Not only we disagree on the change in question, we also
disagree on whether the original behavior constitutes policy. I'd like
to bring the latter to wider discussion, without focusing on this
particular example.

FWIU, William's argument is that the QA team has not formally approved
such a policy (did QA even exist back then?), therefore it is not a
binding policy and can be changed through internal Portage patch review.

I disagree with this assessment. This check that has been present
in Portage since at least 2005, and has reliably enforced specific way
of writing ebuilds (influencing e.g. gen_usr_ldscript() function).
After 14 years, I believe this certainly counts as de-facto policy
and is not something to be changed lightly. Such change needs to be
discussed on gentoo-dev@, and preferably supported by the research
of the original rationale.

This is not the only QA check in Portage that reliably affects how we
are writing ebuilds today, yet were never formally approved or written
down in developer documentation. I think that this is partially simply
because there were never major disagreement about them, and since
Portage has reliably enforced them there were never any real need to
take them elsewhere. I think they should be considered equally to well-
defined policies.

Hence, my question: should the policies implied by historical Portage
checks be considered official, and be changed with due diligence? Or
should they be merely considered implementation details, and should
Portage developers make unilateral decisions on changing or removing
them?


[1] https://archives.gentoo.org/gentoo-portage-dev/message/6e4cfbb0ef9c36dc6511d4f2003cc458

--
Best regards,
Micha? Górny
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
On 11/4/19 10:01 AM, Micha? Górny wrote:
> Hi,
>
> TL;DR: If a QA check is enforced by Portage for a reasonably long time,
> does it constitute policy? Or can it be changed unilaterally by Portage
> team?
>

To avoid these sorts of questions in the future, it might be worth the
time it would take to vote on each of these policies formally, document
them on the wiki, and then move the related checks to ::gentoo/metadata
where other package managers can benefit from them (and where they can't
be unilaterally nuked). Having a comprehensive list of policies will
also help developers who want to Do The Right Thing and who read up on
these things proactively.

In this case, whether or not this is "policy" is beside the point. No
one else wants to remove this check because it's useful and prevents
developers from accidentally dumping garbage onto users' (often limited)
root filesystems. Some people don't like to do their jobs, though, and
for those developers it's a lot easier to delete the check and make
things worse for everybody than it would be to package software
correctly. Just Say No. That's what QA is for. But again, it would be
easier to veto these obviously-stupid things if they've been documented.
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
On Mon, 2019-11-04 at 10:53 -0500, Michael Orlitzky wrote:
> In this case, whether or not this is "policy" is beside the point. No
> one else wants to remove this check because it's useful and prevents
> developers from accidentally dumping garbage onto users' (often limited)
> root filesystems. Some people don't like to do their jobs, though, and
> for those developers it's a lot easier to delete the check and make
> things worse for everybody than it would be to package software
> correctly. Just Say No. That's what QA is for. But again, it would be
> easier to veto these obviously-stupid things if they've been documented.

I did say no. William claims it doesn't matter.

I did request a QA vote to confirm it. William demands that I close it
[1].

[1] https://archives.gentoo.org/gentoo-portage-dev/message/f655c94bacdb01c3cafcc39a7bf1af93

--
Best regards,
Micha? Górny
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
On 11/4/19 11:02 AM, Micha? Górny wrote:
>
> I did request a QA vote to confirm it. William demands that I close it

Take a page out of the WilliamH playbook and completely ignore him.
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
On Mon, Nov 04, 2019 at 11:07:43AM -0500, Michael Orlitzky wrote:
> On 11/4/19 11:02 AM, Micha? Górny wrote:
> >
> > I did request a QA vote to confirm it. William demands that I close it
>
> Take a page out of the WilliamH playbook and completely ignore him.

As I said on the other list, the ignoring was mostly due to mgorny's
rude personal attack which is pending before the proctors.

Asking that the qa issue be closed was because there is another way
around it, which was supported in a conversation I had with the qa
lead.

That way is not building static libraries at all. If we go that way as
a distro the support for forcing static libraries into /usr/lib* is not
needed because we would just not allow static libraries.

William
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
Hi Michael,

On Mon, Nov 04, 2019 at 10:53:44AM -0500, Michael Orlitzky wrote:
> On 11/4/19 10:01 AM, Micha? Górny wrote:
> > Hi,
> >
> > TL;DR: If a QA check is enforced by Portage for a reasonably long time,
> > does it constitute policy? Or can it be changed unilaterally by Portage
> > team?
> >
>
> To avoid these sorts of questions in the future, it might be worth the
> time it would take to vote on each of these policies formally, document
> them on the wiki, and then move the related checks to ::gentoo/metadata
> where other package managers can benefit from them (and where they can't
> be unilaterally nuked). Having a comprehensive list of policies will
> also help developers who want to Do The Right Thing and who read up on
> these things proactively.

I actually agree with you. I am not a fan of un-written things that we
call policies, and if this is going to be a distro policy it definitely
belongs in ::gentoo not in the package manager, but also see my other
reply.

> In this case, whether or not this is "policy" is beside the point. No
> one else wants to remove this check because it's useful and prevents
> developers from accidentally dumping garbage onto users' (often limited)
> root filesystems. Some people don't like to do their jobs, though, and
> for those developers it's a lot easier to delete the check and make
> things worse for everybody than it would be to package software
> correctly. Just Say No. That's what QA is for. But again, it would be
> easier to veto these obviously-stupid things if they've been documented.

This is a whole other thread I've been talking about for years, but if
we want to be concerned about dumping "garbage" on people's limited root
file systems, there are other things we need to re-consider, like our
notion that we have to install small files everywhere even though they
aren't always used.

So, if you want to talk about that, please start a whole new thread.

William
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
On Mon, Nov 4, 2019 at 1:26 PM William Hubbs <williamh@gentoo.org> wrote:

> That way is not building static libraries at all. If we go that way as
> a distro the support for forcing static libraries into /usr/lib* is not
> needed because we would just not allow static libraries.
>

As an end user, I would be unhappy if static libraries were no longer
supported at all. I use them regularly for my development work.
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
On Mon, 4 Nov 2019 10:53:44 -0500
Michael Orlitzky <mjo@gentoo.org> wrote:

> To avoid these sorts of questions in the future, it might be worth the
> time it would take to vote on each of these policies formally, document
> them on the wiki, and then move the related checks to ::gentoo/metadata
> where other package managers can benefit from them (and where they can't
> be unilaterally nuked). Having a comprehensive list of policies will
> also help developers who want to Do The Right Thing and who read up on
> these things proactively.

I believe the place for these is in the dev-manual[1]

If not the dev-manual, then if there is some other source of authority
where they end up, there should be some mechanism to relay them to the
dev-manual.

Its hard to expect people to follow a policy that is mostly codified in
tools and cultural wisdom.

1: https://archives.gentoo.org/gentoo-dev/message/497c28fb2dab0a480c302ba966481f4f
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
On Mon, Nov 04, 2019 at 02:05:19PM -0600, Michael Jones wrote:
> On Mon, Nov 4, 2019 at 1:26 PM William Hubbs <williamh@gentoo.org> wrote:
>
> > That way is not building static libraries at all. If we go that way as
> > a distro the support for forcing static libraries into /usr/lib* is not
> > needed because we would just not allow static libraries.
> >
>
> As an end user, I would be unhappy if static libraries were no longer
> supported at all. I use them regularly for my development work.

OpenRC as an example has never supported being built statically along
with pam which is turned on in Gentoo, so it would be safe for openrc to
not build them.

Let's move the discussion of static libs to a separate thread however
and continue it there.

William
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
Hi Kent,

On Tue, Nov 05, 2019 at 10:50:09AM +1300, Kent Fredric wrote:
> On Mon, 4 Nov 2019 10:53:44 -0500
> Michael Orlitzky <mjo@gentoo.org> wrote:
>
> > To avoid these sorts of questions in the future, it might be worth the
> > time it would take to vote on each of these policies formally, document
> > them on the wiki, and then move the related checks to ::gentoo/metadata
> > where other package managers can benefit from them (and where they can't
> > be unilaterally nuked). Having a comprehensive list of policies will
> > also help developers who want to Do The Right Thing and who read up on
> > these things proactively.
>
> I believe the place for these is in the dev-manual[1]
>
> If not the dev-manual, then if there is some other source of authority
> where they end up, there should be some mechanism to relay them to the
> dev-manual.
>
> Its hard to expect people to follow a policy that is mostly codified in
> tools and cultural wisdom.

You are correct. When I was on the team, the idea was that the devmanual
was the cannonical source for all qa policies.

I'm not on the team now but I would strongly support this.

William

>
> 1: https://archives.gentoo.org/gentoo-dev/message/497c28fb2dab0a480c302ba966481f4f
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
On 11/4/19 2:40 PM, William Hubbs wrote:
>
> This is a whole other thread I've been talking about for years, but if
> we want to be concerned about dumping "garbage" on people's limited root
> file systems, there are other things we need to re-consider, like our
> notion that we have to install small files everywhere even though they
> aren't always used.
>
> So, if you want to talk about that, please start a whole new thread.
>

1. The idea that we should fix all existing problems before we can
prevent people from creating new ones is ridiculous.

2. Those other files don't get installed to the root filesystem on the
systems that we're talking about.

3. Those other files generally aren't completely useless.

4. "small files" aren't big.
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
On 04/11/19 23:17, Michael Orlitzky wrote:
> On 11/4/19 2:40 PM, William Hubbs wrote:
>>
>> This is a whole other thread I've been talking about for years, but if
>> we want to be concerned about dumping "garbage" on people's limited root
>> file systems, there are other things we need to re-consider, like our
>> notion that we have to install small files everywhere even though they
>> aren't always used.
>>
>> So, if you want to talk about that, please start a whole new thread.
>>
>
> 1. The idea that we should fix all existing problems before we can
>    prevent people from creating new ones is ridiculous.
>
> 2. Those other files don't get installed to the root filesystem on the
>    systems that we're talking about.
>
> 3. Those other files generally aren't completely useless.
>
> 4. "small files" aren't big.
>
>
Straw man anyone? I know Guy Fawkes night is coming up here in the UK ... ;)
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
On Mon, Nov 04, 2019 at 06:17:55PM -0500, Michael Orlitzky wrote:
> On 11/4/19 2:40 PM, William Hubbs wrote:
> >
> > This is a whole other thread I've been talking about for years, but if
> > we want to be concerned about dumping "garbage" on people's limited root
> > file systems, there are other things we need to re-consider, like our
> > notion that we have to install small files everywhere even though they
> > aren't always used.
> >
> > So, if you want to talk about that, please start a whole new thread.
> >
>
> 1. The idea that we should fix all existing problems before we can
> prevent people from creating new ones is ridiculous.

I never said that we should fix all existing problems before we can
prevent people from creating new ones. I just said that along these
lines this is something else I thought we should talk about. It feels
like an inconsistency to me.

> 2. Those other files don't get installed to the root filesystem on the
> systems that we're talking about.

I do not understand what you think I'm referring to and which files you
are talking about.

The way I'm thinking of a root fs is, /bin, maybe /boot, /etc, /lib* and /sbin.

> 3. Those other files generally aren't completely useless.

A number of them are in the default installation.
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
>> 2. Those other files don't get installed to the root filesystem on the
>> systems that we're talking about.
>
> I do not understand what you think I'm referring to and which files you
> are talking about.
>
> The way I'm thinking of a root fs is, /bin, maybe /boot, /etc, /lib* and /sbin.
>

Most junk gets installed to /usr, which is mounted on a separate
partition on the systems we're talking about.

What useless files do we install to /bin, /boot, or /sbin?

In /lib and /etc, some SMALL files that aren't used on every system do
get installed unconditionally. This is an explicit trade-off: the files
are SMALL by definition, and installing them unconditionally means that
we don't have to add a bunch of USE flags, slow down portage, and bog
down users with choices they don't care about. It also means that users
can e.g. switch between init systems or install logrotate without having
to rebuild @world from scratch. Since the files are SMALL, this
trade-off is in everyone's favor.

Your static libraries aren't small, and they aren't ever going to be
useful to anyone. There is no trade-off here.


>> 3. Those other files generally aren't completely useless.
>
> A number of them are in the default installation.
>

What files in the default installation are completely useless to
everyone? Small files that are useless to EVERYONE are not covered by
our existing policy, so please feel free to drop them in src_install.
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
> > TL;DR: If a QA check is enforced by Portage for a reasonably long time,
> > does it constitute policy? Or can it be changed unilaterally by Portage
> > team?

> To avoid these sorts of questions in the future,
[snip since offtopic]

> In this case, whether or not this is "policy" is beside the point. No
> one else wants to remove this check ...

If that really were the case, we wouldnt have this discussion.

And as we all know *any* action by QA will immediately lead to complaints
about abuse of QA powers...

FTR, yes I consider that these old QA checks embody QA policy. 14 years of it.

--
Andreas K. H?ttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)
Re: Do (old-ish) Portage QA checks comprise policy? [ In reply to ]
On Mon, 4 Nov 2019 23:24:57 +0000
Michael 'veremitz' Everitt <gentoo@veremit.xyz> wrote:

> Straw man anyone? I know Guy Fawkes night is coming up here in the UK ... ;)

I love you, but please, less trolling.

This is a developer discussion venue, not a place for cheap-shots like
reddit.