Mailing List Archive

On Wed, 2020-06-24 at 16:03 -0400, Rich Freeman wrote:
> On Wed, Jun 24, 2020 at 3:04 PM Andreas Sturmlechner <asturm@gentoo.org> wrote:
> > > On Wed, Jun 24, 2020 at 11:29 AM Rich Freeman <rich0@gentoo.org> wrote:
> > > > Sure, you can use the portage API to find this info. However, that is
> > > > as easy to do for a list of all impacted packages in the tree with
> > > > their maintainers as for any individual maintainer to obtain this info
> > > > for their own packages.
> >
> > I'm appealing to a more proactive maintenance, not in search for excuses why
> > it is not like that. And ftr I don't mean trying to be "first!" on every
> > upstream version bump; it is just that the python topic has come up often
> > enough that it should have sparked individual head scratching at one point or
> > another.
> >
> > > On Wednesday, 24 June 2020 20:40:58 CEST Alec Warner wrote:
> > > You say there is not a straightforward way, but then you say there is an
> > > api? :p
> >
> > grep all the things! But hey, there's even external tools to help you get an
> > overview:
> >
> > https://repology.org/maintainer/rich0%40gentoo.org
> >
> > You're welcome.
>
> I'm well aware of that. That will get you a list of what you
> maintain, but not which of those things use python2. It is also
> completely external. (I do love that tool though - great for finding
> bumps.)
>
> grep is really not a reliable tool for parsing ebuilds. The API is
> really the right way to do it.
>

$ git grep -l mgorny@gentoo.org '**/metadata.xml' | cut -d/ -f1-2 |
xargs gpy-py2 2>/dev/null



--
Best regards,
Micha? Górny

On Wed, Jun 24, 2020 at 4:08 PM Micha? Górny <mgorny@gentoo.org> wrote:
>
> $ git grep -l mgorny@gentoo.org '**/metadata.xml' | cut -d/ -f1-2 |
> xargs gpy-py2 2>/dev/null
>

I have no idea what gpy-py2 is, but I'll take your word for it.

In any case, the solution in this case is to send a nice email to
-dev-announce saying:

We're removing python2 around <date>. You can help us out by updating
any packages you have that use python2. If you want to easily
identify these packages just do <insert quick script here>.

I think the problem here is that we're basically telling maintainers
that the beatings will continue until morale improves. Then we're
wondering why nothing is getting done.

I'm not saying anybody has to do it a particular way - it just seems
obvious that the way we're doing it is more successful at getting
people upset than actually getting results.

Ideally you would just open a tracker bug and then per-package bugs
for every impacted package. That would be the cleanest solution. If
that is too painful then by all means do some email announcements, but
make it easy for devs to realize when they're missing something.

Having a package mask be the first time a maintainer finds out that
they have a problem isn't good. Now, you can blame that on the
maintainer, or you can blame that on the python team, but either way
the users end up getting exposed to breakage unnecessarily.

--
Rich

On 2020-06-24 16:08, Micha? Górny wrote:
>
> $ git grep -l mgorny@gentoo.org '**/metadata.xml' | cut -d/ -f1-2 |
> xargs gpy-py2 2>/dev/null
>

The big problem with this is that it misses any aliases (like graphics@)
that you're a member of. But let's golf; this is POSIX sh, doesn't use
grep to parse XML, and takes the maintainer's email address as an argument:

REPO=/var/db/repos/gentoo
XPATH="/pkgmetadata/maintainer/email[normalize-space(text()) = '${1}']"

find -L "${REPO}" \
-mindepth 3 \
-maxdepth 3 \
-name 'metadata.xml' \
-exec sh -c "
for f in \"\${@}\"; do
xmllint --xpath \"${XPATH}\" \"\${f}\" >/dev/null 2>&1 && \
echo \"\$(dirname -- \"\${f}\")\" | sed \"s:${REPO%/}/::\"
done
" - {} +

On Thu, Jun 25, 2020 at 07:32:04AM -0400, Michael Orlitzky wrote:
> On 2020-06-24 16:08, Micha? Górny wrote:
> >
> > $ git grep -l mgorny@gentoo.org '**/metadata.xml' | cut -d/ -f1-2 |
> > xargs gpy-py2 2>/dev/null
> >
>
> The big problem with this is that it misses any aliases (like graphics@)
> that you're a member of. But let's golf; this is POSIX sh, doesn't use
> grep to parse XML, and takes the maintainer's email address as an argument:
>
> REPO=/var/db/repos/gentoo
> XPATH="/pkgmetadata/maintainer/email[normalize-space(text()) = '${1}']"
>
> find -L "${REPO}" \
> -mindepth 3 \
> -maxdepth 3 \
> -name 'metadata.xml' \
> -exec sh -c "
> for f in \"\${@}\"; do
> xmllint --xpath \"${XPATH}\" \"\${f}\" >/dev/null 2>&1 && \
> echo \"\$(dirname -- \"\${f}\")\" | sed \"s:${REPO%/}/::\"
> done
> " - {} +
>

We can instead avoid parsing XML at all if we're not averse to using
`pquery`, and we can avoid the limitation of scanning the entire tree
for a single name/email by outputting the maintainers for all of the
problematic packages at once (in this case, packages output by
`gpy-py2`) in a greppable format. Not sure why pquery doesn't see
maintainers for things like automake:1.9, so this implementation is
imperfect, but here:

REPO=/var/db/repos/gentoo

for pkg in $(gpy-py2 -r "${REPO}"); do
maint=$(pquery ${pkg} --one-attr maintainers | tail -1)
if [[ ${maint} ]]; then
echo "${pkg}: ${maint}"
else
echo "${pkg}: maintainer-needed"
fi
done

On Thu, Jun 25, 2020 at 2:45 PM John Helmert III <jchelmert3@posteo.net> wrote:
>
> On Thu, Jun 25, 2020 at 07:32:04AM -0400, Michael Orlitzky wrote:
> > On 2020-06-24 16:08, Micha? Górny wrote:
> > >
> > > $ git grep -l mgorny@gentoo.org '**/metadata.xml' | cut -d/ -f1-2 |
> > > xargs gpy-py2 2>/dev/null
> > find -L "${REPO}" \
> maint=$(pquery ${pkg} --one-attr maintainers | tail -1)

Great, so now we have 4 ways (and counting) to get 4 answers to this
question that hopefully will be mostly the same.

My point is more that it makes more sense for one person to just file
the bugs or send out the list so that maintainers can go fix their
packages, as opposed to playing a game where every developer in Gentoo
independently engineers a solution to the same problem. If some
maintainers decide not to play the game, or play it and make a
mistake, then it ends up being the python team or the users who lose.

If a maintainer ignores a blocker bug for too long nobody is going to
shed a tear over some treecleaning. Spending a bit more time on
communication might save a lot more time in cleanup.

--
Rich

On Wed, Jun 24, 2020 at 04:21:14PM -0400, Rich Freeman wrote:
> On Wed, Jun 24, 2020 at 4:08 PM Micha? Górny <mgorny@gentoo.org> wrote:
> >
> > $ git grep -l mgorny@gentoo.org '**/metadata.xml' | cut -d/ -f1-2 |
> > xargs gpy-py2 2>/dev/null
> >
>
> I have no idea what gpy-py2 is, but I'll take your word for it.
>
> In any case, the solution in this case is to send a nice email to
> -dev-announce saying:
>
> We're removing python2 around <date>. You can help us out by updating
> any packages you have that use python2. If you want to easily
> identify these packages just do <insert quick script here>.
>
> I think the problem here is that we're basically telling maintainers
> that the beatings will continue until morale improves. Then we're
> wondering why nothing is getting done.
>
> I'm not saying anybody has to do it a particular way - it just seems
> obvious that the way we're doing it is more successful at getting
> people upset than actually getting results.
>
> Ideally you would just open a tracker bug and then per-package bugs
> for every impacted package. That would be the cleanest solution. If
> that is too painful then by all means do some email announcements, but
> make it easy for devs to realize when they're missing something.
>
> Having a package mask be the first time a maintainer finds out that
> they have a problem isn't good. Now, you can blame that on the
> maintainer, or you can blame that on the python team, but either way
> the users end up getting exposed to breakage unnecessarily.
>
> --
> Rich
>

I am thoroughly confused here. Some how you have completely changed your
opinion from previous posts. Furthermore, this has turned into a debate
of how to find packages that are Py2 only which is just absurd.

Of all the methods listed in the previous posts, the QA reports, etc.
there is no excuse individuals can't find out if their package is py2
only.

Ironically, it would be a very sad state if an individual doesn't know
what Python interpreter their package is compatible with. This is the
essence of "maintainer" status, correct?

Can we stop finding excuses and let folks fix their packages?

Obviously, the myriad of tools, ML threads, and all the other "avenues"
individual developers have taken to alert others simply doesn't work...
until something is p.masked... people don't budge.

--
Cheers,
Aaron

On Wed, Jun 24, 2020 at 10:12:18AM -0700, Matt Turner wrote:
> offlineimap is widely used and blocks no further work. It can easily
> remain in the tree after all other python2_7 support is gone.
>
> This is not a hill worth dying on.
>

I am confused here... are you saying that due to usage of the package
that it should stay until dev-lang/python:2 is gone?

More importantly, I simply responded to Thomas' mention that the package
has alternatives, but apparently, I don't understand how the package is
used :)

--
Cheers,
Aaron

On Wed, Jun 24, 2020 at 11:52:28AM +0200, Thomas Deutschmann wrote:
> On 2020-06-20 21:24, Aaron Bauman wrote:
> > Thomas, unfortunately, I am shocked at your choice of words here. I
> > think it is reasonable that any developer would understand a lack
> > of forward momentum in removing Py2 only packages only drives
> > stagnation.
> >
> > If you have a more effective method to doing so, I am open to
> > suggestions.
>
> Like I am shocked about your recent actions:
>
> Remember what you did in January. I thought it became clear that next
> time you will share your list before just masking stuff to avoid things
> which happened then.
>

Developers have many tools and *hopefully* the organic ability to determine
which packages are impacted. Especially given previous threads on this
very ML with pleas from other Python team members to assist in cleaning
things up in a deliberate manner.

This is why the QA team interceded... because a couple of individuals
screamed loudly for no reason. Fix it and move on.

> In the beginning of this month you just decided to disband graphics
> project. On your own. Please tell me what gave you the authority to just
> do that? You didn't even share your plan before executing it on any
> mailing list. Something that should be common sense, if not even necessary.
> The whole action was so destructive that you couldn't evenb just undo it
> because you also deleted stuff on Wiki.
>

I will not apologize for doing something that others have lacked the
intestinal fortitude to do.

> Like multiple people have already shown you, many packages from that
> list are not even blocking Py3 transition.
>

This isn't about transitioning to Py3... it is about removing Py2.

> Let me tell you what a mask will cause:
> A mask is destructive and requires user interaction. Therefore a mask
> isn't something to play with, "Oh, let's test if someone will
> complain... it's just a mask, we can just unmask in case...".
>

Is that why you assume I masked these things?

> No, imagine there are people out there using Gentoo in production and
> not as playground. These people maybe have automated build systems which
> are creating systems/images (do you know Dockers for example?). Whenever
> you mask something and that package is referenced in configuration, you
> will break that build.
>
> That's not funny if this is happening for no real reason.
>

I know you use Gentoo in production, but does this mean we (Gentoo)
can't move forward because *you* want to use something that is EOL and
dying? What if you used a major distro that removed Py2 support already?
Why complain here? There are other ways to safely run your tooling with
Py2 if you so choose.

>
> > re: net-mail/offlineimap... there are alternatives.
>
> I think you don't really know that tool. It's an industry standard.
> Sure, there are already successors (however, not in Gentoo). But the
> package itself is still working and actively maintained and when you
> will use it in production you usually have extended/adjusted the tool
> for your environment using the plugin system the tool provides. That's
> not something you will be able to replace with something new in 5 minutes.
>

You continuously speak condescendingly to me. I am truly starting to
regret my nomination for you on both the security project and for
council. Do you speak to others this way simply because you don't agree
with them?

Yes, I use net-mail/offlineimap... I know how it works. No, I really
hope that a tool which has not been maintained in many years is an
"industry standard"

Yes, there are successors in Gentoo.

> And I repeat myself: Especially not when there is no need to do that
> because because the package itself is working fine and there is absolute
> no reason to get rid of it.
>

Take Patrick's approach and move it to an overlay if you want it that
badly.

> Last but not least: Gentoo is about choices. It's not your job to decide
> what people should use. Sure, if you maintained a package and will stop
> using it so it will become maintainer-needed and masked for removal at
> some point because it's outdated, vulnerable and/or not working anymore,
> that's OK. But if someone else will pick up this package... and
> offlineimap in Gentoo is working and up-to-date.
>

Are you implying that because "Gentoo is about choice" that we never
remove an ebuild, interpreter, compiler, etc? Let ::gentoo grow in size
forever to appease the few?

--
Cheers,
Aaron

On Thu, Jun 25, 2020 at 8:14 PM Aaron Bauman <bman@gentoo.org> wrote:
>
> On Wed, Jun 24, 2020 at 10:12:18AM -0700, Matt Turner wrote:
> > offlineimap is widely used and blocks no further work. It can easily
> > remain in the tree after all other python2_7 support is gone.
> >
> > This is not a hill worth dying on.
> >
>
> I am confused here... are you saying that due to usage of the package
> that it should stay until dev-lang/python:2 is gone?
>
> More importantly, I simply responded to Thomas' mention that the package
> has alternatives, but apparently, I don't understand how the package is
> used :)

Two reasons: because it's widely used *and* because its python:2.7
dependency is on only the interpreter itself.

The second reason means that its removal doesn't enable any further clean ups.

Ühel kenal päeval, N, 25.06.2020 kell 23:47, kirjutas Aaron Bauman:
> Yes, there are successors in Gentoo.

Traditionally p.mask entries point these out.

On June 26, 2020 2:45:07 AM EDT, Mart Raudsepp <leio@gentoo.org> wrote:
>Ühel kenal päeval, N, 25.06.2020 kell 23:47, kirjutas Aaron Bauman:
>> Yes, there are successors in Gentoo.
>
>Traditionally p.mask entries point these out.

Feel free to find and report all possible alternatives for all the packages that must go or be ported to remove Py2.

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

On Thu, Jun 25, 2020 at 11:07 PM Aaron Bauman <bman@gentoo.org> wrote:
>
> On Wed, Jun 24, 2020 at 04:21:14PM -0400, Rich Freeman wrote:
> >
> > We're removing python2 around <date>. You can help us out by updating
> > any packages you have that use python2. If you want to easily
> > identify these packages just do <insert quick script here>.
> >
> > I think the problem here is that we're basically telling maintainers
> > that the beatings will continue until morale improves. Then we're
> > wondering why nothing is getting done.
> >
>
> I am thoroughly confused here. Some how you have completely changed your
> opinion from previous posts.

Perhaps we failed to communicate then. My opinion has always been this:

I support letting the python team manage the versions of python
available - if people want legacy versions to stick around they need
to do something to make it happen.

HOWEVER, the python team would also find its job much easier if they
partnered with the myriad of package maintainers to accomplish their
goals, instead of just throwing them over the fence and then breaking
things for users to try to get everybody's attention periodically.

>
> Of all the methods listed in the previous posts, the QA reports, etc.
> there is no excuse individuals can't find out if their package is py2
> only.

None of those methods were posted until a day or two ago, and the
python team has done nothing to actually ensure all the impacted
maintainers are aware of them. Perhaps a communication to
-dev-announce with the preferred approach would be better?

You can't expect every Gentoo dev to independently cobble together a
bunch of scripts to go hunting for py2 reverse deps.

> Ironically, it would be a very sad state if an individual doesn't know
> what Python interpreter their package is compatible with. This is the
> essence of "maintainer" status, correct?

Maintainers generally care about what the package does, and how it
does it is a means to an end. Sure, some care more about the build
system and dependencies than others, and when working on a package you
need to pay more attention to such things. However, I suspect most
package maintainers do not know off the top of their head the
dependency list of all their packages.

> Obviously, the myriad of tools, ML threads, and all the other "avenues"
> individual developers have taken to alert others simply doesn't work...
> until something is p.masked... people don't budge.

At least some devs here seemed surprised about the masks. Did you try
filing a bug?

Masking something for all users is basically like torturing a kitten
to get the attention of its owner. It is a necessary step if the
package is actually to be removed. I don't think it is even allowable
under our policies if no bug was filed.

But if filing bugs is painful at least make things easier on
maintainers. Post a list of packages and owners, for example.

It just seems like you're making things harder on yourself. Gentoo
has done countless migrations like this and for whatever reason in the
past creating a tracker and blocker bugs hasn't been a problem.

I don't think the community will be served by having the python team
work itself into a frenzy until they ragequit. Just give in, send out
a -announce post, and maybe cut your workload in half at least. I get
that you seem to want to stand on some kind of principle that
everybody in Gentoo should care about the py2 migration as much as you
do, but it probably isn't going to happen. Help everybody else help
you...

--
Rich

On June 26, 2020 7:13:07 AM EDT, Rich Freeman <rich0@gentoo.org> wrote:
>On Thu, Jun 25, 2020 at 11:07 PM Aaron Bauman <bman@gentoo.org> wrote:
>>
>> On Wed, Jun 24, 2020 at 04:21:14PM -0400, Rich Freeman wrote:
>> >
>> > We're removing python2 around <date>. You can help us out by
>updating
>> > any packages you have that use python2. If you want to easily
>> > identify these packages just do <insert quick script here>.
>> >
>> > I think the problem here is that we're basically telling
>maintainers
>> > that the beatings will continue until morale improves. Then we're
>> > wondering why nothing is getting done.
>> >
>>
>> I am thoroughly confused here. Some how you have completely changed
>your
>> opinion from previous posts.
>
>Perhaps we failed to communicate then. My opinion has always been
>this:
>
>I support letting the python team manage the versions of python
>available - if people want legacy versions to stick around they need
>to do something to make it happen.
>
>HOWEVER, the python team would also find its job much easier if they
>partnered with the myriad of package maintainers to accomplish their
>goals, instead of just throwing them over the fence and then breaking
>things for users to try to get everybody's attention periodically.
>

How have we *not* partnered with the community of devs? Michal's mails to the list? Previous discussions based on masks...

>>
>> Of all the methods listed in the previous posts, the QA reports, etc.
>> there is no excuse individuals can't find out if their package is py2
>> only.
>
>None of those methods were posted until a day or two ago, and the
>python team has done nothing to actually ensure all the impacted
>maintainers are aware of them. Perhaps a communication to
>-dev-announce with the preferred approach would be better?
>

You should also look at qa-reports. Do we really need to *teach* others "how to fish" here? Why can't folks just ask for assistance?

All of it has been there and widely available for quite some time. Stop finding excuses.

>You can't expect every Gentoo dev to independently cobble together a
>bunch of scripts to go hunting for py2 reverse deps.
>

See above. Qa-reports will output a very nice list (even a graphic!) of such things. Anyway, yes, I do expect devs to understand their packages state if they maintain it. Don't be so myopic.

>> Ironically, it would be a very sad state if an individual doesn't
>know
>> what Python interpreter their package is compatible with. This is the
>> essence of "maintainer" status, correct?
>
>Maintainers generally care about what the package does, and how it
>does it is a means to an end. Sure, some care more about the build
>system and dependencies than others, and when working on a package you
>need to pay more attention to such things. However, I suspect most
>package maintainers do not know off the top of their head the
>dependency list of all their packages.
>
>> Obviously, the myriad of tools, ML threads, and all the other
>"avenues"
>> individual developers have taken to alert others simply doesn't
>work...
>> until something is p.masked... people don't budge.
>
>At least some devs here seemed surprised about the masks. Did you try
>filing a bug?

Have you looked for said bugs?

>
>Masking something for all users is basically like torturing a kitten
>to get the attention of its owner. It is a necessary step if the
>package is actually to be removed. I don't think it is even allowable
>under our policies if no bug was filed.
>

Do tell where said policy is?

>But if filing bugs is painful at least make things easier on
>maintainers. Post a list of packages and owners, for example.
>
>It just seems like you're making things harder on yourself. Gentoo
>has done countless migrations like this and for whatever reason in the
>past creating a tracker and blocker bugs hasn't been a problem.
>

Nothing is really hard about masking packages for removal... honestly. The work comes in defending the position here for the few that complain. If I filed a bug... they would complain or not respond... If I sent out a dev-announce they would complain or not respond.

You see the fun here? Which method is effective? Mask a 100 packages for removal... Someone complains... A few packages get saved and 90 get removed... Life goes on.

-Aaron

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

On Fri, Jun 26, 2020 at 10:36 AM Aaron Bauman <bman@gentoo.org> wrote:
>
> On June 26, 2020 7:13:07 AM EDT, Rich Freeman <rich0@gentoo.org> wrote:
> >> Of all the methods listed in the previous posts, the QA reports, etc.
> >> there is no excuse individuals can't find out if their package is py2
> >> only.
> >
> >None of those methods were posted until a day or two ago, and the
> >python team has done nothing to actually ensure all the impacted
> >maintainers are aware of them. Perhaps a communication to
> >-dev-announce with the preferred approach would be better?
> >
>
> You should also look at qa-reports. Do we really need to *teach* others "how to fish" here? Why can't folks just ask for assistance?

Just looked at it:

1. I had no idea that a list of py2-only packages was listed there.
I don't think I've ever actually looked at that page.

2. The report does not list maintainers, which means nobody is likely
to know they have a package on the list.

>
> See above. Qa-reports will output a very nice list (even a graphic!) of such things. Anyway, yes, I do expect devs to understand their packages state if they maintain it. Don't be so myopic.

Well, you can expect whatever you want, and then you can be frustrated
out of your mind when 95% of devs fail to meet your expectations.

Or you could just work with them where they're at and maybe get your
project completed more quickly and with less effort...

If you want people to look at a qa-report, maybe post on -dev-announce
and ask everybody to do it? Most people aren't going to be following
all the tools used by the python team if they aren't python devs.

> >At least some devs here seemed surprised about the masks. Did you try
> >filing a bug?
>
> Have you looked for said bugs?

Of course. Do you think I'd invite such an obvious reply without
actually checking.

I just went to the first complaint on this list about there not being
a bug, and verified that there wasn't a bug.

As far as I can tell there is no bug for app-misc/golly. If I missed
one feel free to cite it.

>
> >
> >Masking something for all users is basically like torturing a kitten
> >to get the attention of its owner. It is a necessary step if the
> >package is actually to be removed. I don't think it is even allowable
> >under our policies if no bug was filed.
> >
>
> Do tell where said policy is?

https://devmanual.gentoo.org/general-concepts/package-maintainers/index.html

Granted, a mask isn't a package commit, but I think the spirit of the
policy covers it.

In any case, there is no reason not to communicate with a maintainer
before touching a package. That should involve something more than a
generic notice that everybody should become a detective to figure out
if they are covered by an upcoming change. If you have a list of
impacted packages, then just file bugs against them.

>
> Nothing is really hard about masking packages for removal... honestly.

The complaint isn't that masks are hard on you. The complaint is that
it bombards users with unnecessary warnings.

> The work comes in defending the position here for the few that complain.

And how are you enjoying doing all that extra work? Would you prefer
if devs started opening up QA/Comrel bugs that you then have to
formally respond to?

Or maybe you could try notifying devs before masking their packages?

> If I filed a bug... they would complain or not respond... If I sent out a dev-announce they would complain or not respond.

Sometimes, sure. But at least you would have gone through due
process, and you're unlikely to get much push back.

And I suspect a number of those packages would actually get fixed.

>
> You see the fun here? Which method is effective? Mask a 100 packages for removal... Someone complains... A few packages get saved and 90 get removed... Life goes on.

Would you want a dev to just mask one of your packages if they saw a
bug in it, without bothering to open a bug?

--
Rich

On June 26, 2020 11:08:35 AM EDT, Rich Freeman <rich0@gentoo.org> wrote:
>On Fri, Jun 26, 2020 at 10:36 AM Aaron Bauman <bman@gentoo.org> wrote:
>>
>> On June 26, 2020 7:13:07 AM EDT, Rich Freeman <rich0@gentoo.org>
>wrote:
>> >> Of all the methods listed in the previous posts, the QA reports,
>etc.
>> >> there is no excuse individuals can't find out if their package is
>py2
>> >> only.
>> >
>> >None of those methods were posted until a day or two ago, and the
>> >python team has done nothing to actually ensure all the impacted
>> >maintainers are aware of them. Perhaps a communication to
>> >-dev-announce with the preferred approach would be better?
>> >
>>
>> You should also look at qa-reports. Do we really need to *teach*
>others "how to fish" here? Why can't folks just ask for assistance?
>
>Just looked at it:
>
>1. I had no idea that a list of py2-only packages was listed there.
>I don't think I've ever actually looked at that page.
>

Perfect, so you have just shown that you either didn't see the ML posts about QA tools, didn't care to ask other devs what tools are available etc.

If history serves me right, qa-reports has existed for many years (of which I have used it) and mgorny often let's folks know about changes to it (e.g. pkgcore).

So, thanks for proving my point that all the tooling changes, notices, ML posts, etc don't matter. Someone *will* find something to complain about.


>2. The report does not list maintainers, which means nobody is likely
>to know they have a package on the list.
>

Do you argue just to argue? Sad. If someone like Robin (who at one point had like 5% of the tree under his maintainer ship) complained about that I may see it worthwhile.

Just another red herring...

>>
>> See above. Qa-reports will output a very nice list (even a graphic!)
>of such things. Anyway, yes, I do expect devs to understand their
>packages state if they maintain it. Don't be so myopic.
>
>Well, you can expect whatever you want, and then you can be frustrated
>out of your mind when 95% of devs fail to meet your expectations.
>

I am not frustrated. I will continue to perform the same in intervals to drive the removal of Py2.

>Or you could just work with them where they're at and maybe get your
>project completed more quickly and with less effort...
>

::yawn:: see above remarks showing how folks will find a way to complain.

>If you want people to look at a qa-report, maybe post on -dev-announce
>and ask everybody to do it? Most people aren't going to be following
>all the tools used by the python team if they aren't python devs.
>
>> >At least some devs here seemed surprised about the masks. Did you
>try
>> >filing a bug?
>>
>> Have you looked for said bugs?
>
>Of course. Do you think I'd invite such an obvious reply without
>actually checking.
>
>I just went to the first complaint on this list about there not being
>a bug, and verified that there wasn't a bug.
>
>As far as I can tell there is no bug for app-misc/golly. If I missed
>one feel free to cite it.
>
>>
>> >
>> >Masking something for all users is basically like torturing a kitten
>> >to get the attention of its owner. It is a necessary step if the
>> >package is actually to be removed. I don't think it is even
>allowable
>> >under our policies if no bug was filed.
>> >
>>
>> Do tell where said policy is?
>
>https://devmanual.gentoo.org/general-concepts/package-maintainers/index.html
>
>Granted, a mask isn't a package commit, but I think the spirit of the
>policy covers it.
>
>In any case, there is no reason not to communicate with a maintainer
>before touching a package. That should involve something more than a
>generic notice that everybody should become a detective to figure out
>if they are covered by an upcoming change. If you have a list of
>impacted packages, then just file bugs against them.
>
>>
>> Nothing is really hard about masking packages for removal...
>honestly.
>
>The complaint isn't that masks are hard on you. The complaint is that
>it bombards users with unnecessary warnings.
>

Sadly, many users have contributed more than some devs to fix packages. I often get emails directly from users wanting to fix things. I will start forwarding them to you.

>> The work comes in defending the position here for the few that
>complain.
>
>And how are you enjoying doing all that extra work? Would you prefer
>if devs started opening up QA/Comrel bugs that you then have to
>formally respond to?
>

There is one open now. Seems QA hasn't spoken up yet...

>Or maybe you could try notifying devs before masking their packages?
>
>> If I filed a bug... they would complain or not respond... If I sent
>out a dev-announce they would complain or not respond.
>
>Sometimes, sure. But at least you would have gone through due
>process, and you're unlikely to get much push back.
>
>And I suspect a number of those packages would actually get fixed.
>
>>
>> You see the fun here? Which method is effective? Mask a 100 packages
>for removal... Someone complains... A few packages get saved and 90 get
>removed... Life goes on.
>
>Would you want a dev to just mask one of your packages if they saw a
>bug in it, without bothering to open a bug?

Depends on the type of bug. Security RCE? Sure. Deprecated code/EOL interpreter? Sure.

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

On Fri, Jun 26, 2020 at 01:48:31PM -0400, Aaron Bauman wrote:
> So, thanks for proving my point that all the tooling changes, notices,
> ML posts, etc don't matter. Someone *will* find something to complain
> about.
They will also complain about the status quo, you won't win there!

> >2. The report does not list maintainers, which means nobody is likely
> >to know they have a package on the list.
> >
>
> Do you argue just to argue? Sad. If someone like Robin (who at one
> point had like 5% of the tree under his maintainer ship) complained
> about that I may see it worthwhile.
Some of the packages shown on the py27 list I had LONG forgotten that I
maintained: I'll try and get to fixing them now.

I did q-text-as-data this week, because I actually needed it for a quick
project and I haven't used it since I changed my default python away
from py27 last month.

> Just another red herring...
I'm mostly speaking to the QA team here, and Python team indirectly:
Indirectly because the Python team is one of the few teams to step up
and provide QA checks outside of the QA team directly. This thread
however has shown that output of those checks however needs to become
easier to consume.

TL;DR: Please make it easier to search on the QA reports site for
issues, and only show things directly relevant to the search.

A long time ago, there was blizzy's site that listed packages that were
stabilization candidates, and you could filter by developer. It really
helped making it easier to detect and progress.

At a bare minimum, having an on-site way that already expands the data
and makes it searchable by developer.

Have files like https://qa-reports.gentoo.org/output/gpyutils/py2.txt
directly loaded into qa-reports and expanded out (which is cachable) and
then let devs search w/ their browser.

cat/package:slot (reason) (all-direct-maintainers),(expanded-projects)

get-git-file.sh tries very hard to get there for the gentoo-ci output,
needs performance and usability improvements, but it's vastly better
than the py2.txt file already.
Some issues:
- Make it more visible! Right now you have to have a link to it from
somewhere else, and it doesn't accept a branch name
(https://qa-reports.gentoo.org/output/gentoo-ci/HEAD/output.html
returns 404)
- Why does it take 15 seconds to load?
- Add filter by developers (by direct maintainer OR membership in alias)
- Add filter by package/cat
- Add filter by check name
- Machine-readable format should be the same data as human-readable (I
can't just take the .xml and grep it, it doesn't have maintainers at
all)

> >> See above. Qa-reports will output a very nice list (even a graphic!)
> >of such things. Anyway, yes, I do expect devs to understand their
> >packages state if they maintain it. Don't be so myopic.
> >
> >Well, you can expect whatever you want, and then you can be frustrated
> >out of your mind when 95% of devs fail to meet your expectations.
> >
> I am not frustrated. I will continue to perform the same in intervals to drive the removal of Py2.
Can we have that graphic in a searchable text format? <3.

--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

On 6/27/20 2:28 AM, Robin H. Johnson wrote:
>
> TL;DR: Please make it easier to search on the QA reports site for
> issues, and only show things directly relevant to the search.
>
> A long time ago, there was blizzy's site that listed packages that were
> stabilization candidates, and you could filter by developer. It really
> helped making it easier to detect and progress.

$ pkgcheck --color true scan $(git grep -l robbat2@gentoo.org
'**/metadata.xml' | cut -d/ -f1-2) -c StableRequestCheck -R
FormatReporter --format 'stabilize {category}/{package}-{version}? # {desc}'

There's also a bug open to integrate some of the pkgchecks to p.g.o,
https://bugs.gentoo.org/725704



>
> At a bare minimum, having an on-site way that already expands the data
> and makes it searchable by developer.

You can filter the output.html per-dev/per-project, but it's not as
verbose as running pkgcheck locally can be.

https://qa-reports.gentoo.org/output/gentoo-ci/output.html;maintainer=robbat2

https://qa-reports.gentoo.org/output/gentoo-ci/output.verbose.html;maintainer=robbat2

It doesn't straight out show python2 like this, but with the
package.deprecated entry for it, it'll flag newly added py2 commits.

-- juippis

On Sat, Jun 27, 2020 at 10:08:26AM +0300, Joonas Niilola wrote:
>
> On 6/27/20 2:28 AM, Robin H. Johnson wrote:
> >
> > TL;DR: Please make it easier to search on the QA reports site for
> > issues, and only show things directly relevant to the search.
> >
> > A long time ago, there was blizzy's site that listed packages that were
> > stabilization candidates, and you could filter by developer. It really
> > helped making it easier to detect and progress.
>
> $ pkgcheck --color true scan $(git grep -l robbat2@gentoo.org
> '**/metadata.xml' | cut -d/ -f1-2) -c StableRequestCheck -R
> FormatReporter --format 'stabilize {category}/{package}-{version}? # {desc}'
>
> There's also a bug open to integrate some of the pkgchecks to p.g.o,
> https://bugs.gentoo.org/725704
Yep, this ties to the large scope of making it easier to show QA issues
from many sources/checks, filtering by many ways.

> > At a bare minimum, having an on-site way that already expands the data
> > and makes it searchable by developer.
>
> You can filter the output.html per-dev/per-project, but it's not as
> verbose as running pkgcheck locally can be.
>
> https://qa-reports.gentoo.org/output/gentoo-ci/output.html;maintainer=robbat2
> https://qa-reports.gentoo.org/output/gentoo-ci/output.verbose.html;maintainer=robbat2
The maintainer= flag also doesn't seem documented anywhere? (I checked
the source and see the 'include-projects' flag as well, so it's just a
matter of making this stuff much better known and a little bit cleaner.

> It doesn't straight out show python2 like this, but with the
> package.deprecated entry for it, it'll flag newly added py2 commits.
Also it's broken in the XML machine-readable version :-(.
https://qa-reports.gentoo.org/output/gentoo-ci/output.xml;maintainer=robbat2
returns a 404

--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136