Mailing List Archive

[2019-07-17 15:25:10+0200] Micha? Górny:
> The QA team would like to introduce the following policy:
>
> """
> Packages must not disable installing manpages via USE flags (e.g.
> USE=man or USE=doc). If upstream does not ship prebuilt manpages
> and building them requires additional dependencies, the maintainer
> should build them and ship along with the package.
> """

Should this be for any dependency? For example wlroots, sway, … are
using scdoc to transform a form of markdown to manpages, and the
resulting program is very small.

$ qsize scdoc
app-text/scdoc-1.9.3-r1: 5 files, 12 non-files, 59.9K

So in my opinion if the dependency is probably smaller than bundling
the files the dependency should be used.

> Rationale:
>
> Manpages are the basic form of user documentation on Gentoo Linux. Not
> installing them is harmful to our users. On the other hand, requiring
> additional dependencies is inconvenient. Therefore, packaging prebuilt
> manpages (whenever upstream doesn't do that already) is a good
> compromise that provides user with documentation without additional
> dependencies.

I would fully support that as in my opinion all ebuilds should provide
at least usage documentation though manpages and/or via the
non-standard -h / --help.

On Wed, 2019-07-17 at 15:42 +0200, Haelwenn (lanodan) Monnier wrote:
> [2019-07-17 15:25:10+0200] Micha? Górny:
> > The QA team would like to introduce the following policy:
> >
> > """
> > Packages must not disable installing manpages via USE flags (e.g.
> > USE=man or USE=doc). If upstream does not ship prebuilt manpages
> > and building them requires additional dependencies, the maintainer
> > should build them and ship along with the package.
> > """
>
> Should this be for any dependency? For example wlroots, sway, … are
> using scdoc to transform a form of markdown to manpages, and the
> resulting program is very small.
>
> $ qsize scdoc
> app-text/scdoc-1.9.3-r1: 5 files, 12 non-files, 59.9K
>
> So in my opinion if the dependency is probably smaller than bundling
> the files the dependency should be used.
>

Yes, unconditionally requiring the dependency also fits the proposed
wording.


--
Best regards,
Micha? Górny

On Wed, Jul 17, 2019 at 9:25 AM Micha? Górny <mgorny@gentoo.org> wrote:
>
> Hello,
>
> The QA team would like to introduce the following policy:
>
> """
> Packages must not disable installing manpages via USE flags (e.g.
> USE=man or USE=doc). If upstream does not ship prebuilt manpages
> and building them requires additional dependencies, the maintainer
> should build them and ship along with the package.
> """
>
>
> Explanatory note:
>
> This applies to having USE flags that specifically control building
> manpages. It obviously does not affect:
>
> a. USE flags that disable building both a program and its manpage (e.g.
> if USE=gui disables building gfrobnicate, not installing gfrobnicate(1)
> is correct),
>
> b. use of LINGUAS to control installed manpages.
>
>
> Rationale:
>
> Manpages are the basic form of user documentation on Gentoo Linux. Not
> installing them is harmful to our users. On the other hand, requiring
> additional dependencies is inconvenient. Therefore, packaging prebuilt
> manpages (whenever upstream doesn't do that already) is a good
> compromise that provides user with documentation without additional
> dependencies.
>
>
> What are your comments?

I'm against this.

I seriously doubt maintainers will take the time/effort to pre-build
and distribute manpages. The end result of this will be additional
hard dependencies on heavyweight packages.

I would prefer to give users the choice NOT to install these heavy
packages. If USE=doc is not sufficent, introduce a new flag for it.

On 2019.07.17 14:25, Micha? Górny wrote:
> Hello,
>
> The QA team would like to introduce the following policy:
>
> """
> Packages must not disable installing manpages via USE flags (e.g.
> USE=man or USE=doc). If upstream does not ship prebuilt manpages
> and building them requires additional dependencies, the maintainer
> should build them and ship along with the package.
> """
>
>
> Explanatory note:
>
> This applies to having USE flags that specifically control building
> manpages. It obviously does not affect:
>
> a. USE flags that disable building both a program and its manpage
> (e.g.
> if USE=gui disables building gfrobnicate, not installing
> gfrobnicate(1)
> is correct),
>
> b. use of LINGUAS to control installed manpages.
>
>
> Rationale:
>
> Manpages are the basic form of user documentation on Gentoo Linux.
> Not
> installing them is harmful to our users. On the other hand, requiring
> additional dependencies is inconvenient. Therefore, packaging
> prebuilt
> manpages (whenever upstream doesn't do that already) is a good
> compromise that provides user with documentation without additional
> dependencies.
>
>
> What are your comments?
>
> --
> Best regards,
> Micha? Górny
>
>

Micha?,

This works on systems with plenty of resources.
I suspect very few arm users have man/doc/info pages installed.
FEATURES="noman nodoc noinfo" is less than ideal as everything
is still built, so you pay the build time, but not installed.

Does anyone read documentation on embedded class hardware?
I know I don't.

Personally, I don't build much on embedded hardware either but I'm
aware of users that do.

I like to have the choice to not build documentation on low power
systems. Its a part of Gentoos flexibility that should not be removed.

--
Regards,

Roy Bamford
(Neddyseagoon) a member of
elections
gentoo-ops
forum-mods
arm64

On Wed, Jul 17, 2019 at 03:25:10PM +0200, Micha? Górny wrote:
> Hello,
>
> The QA team would like to introduce the following policy:
>
> """
> Packages must not disable installing manpages via USE flags (e.g.
> USE=man or USE=doc). If upstream does not ship prebuilt manpages
> and building them requires additional dependencies, the maintainer
> should build them and ship along with the package.
> """
Reading this wording I feel some of the other response have missed the
intent of this policy.

It would be better as two separate policy statements:
"""
Packages must not disable installing manpages via USE flags.
FEATURES=noman should be used to restrict installation of manpages.
"""

The install part I feel is uncontroversial in itself, as it does leave
open that a USE flag could control dependencies & building the manpages
(yes, I'm rules-lawyering).

"""
Package maintainers must build & distribute pre-built manpages IFF
(upstream does not offer them) AND
(building manpages requires build-time additional dependencies)
"""

I agree with the goals of the shipping manpages, but I'm worried about
the requirements it imposes.

- requires the maintainer has some tooling SOMEWHERE to build & package
the manpages. Does this go into the ebuild somehow (e.g. a special
phase)? If not, where else does it get maintained for each package?
- requires a consistent hosting solution for the additional distfiles
created by this change (projects.g.o?).
- significantly increases the version bump requirements (can't simply
copy & local-build & quick-test & commit)
- may require special handling to build & ship manpages in multiple
languages (admittedly very few packages offer non-english manpages).
=> the maintainer now needs to build manpages for ALL possible
languages. This might require the maintainer has lots of locales on
their local system.

--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

Ühel kenal päeval, K, 17.07.2019 kell 18:05, kirjutas Robin H. Johnson:
> - significantly increases the version bump requirements (can't simply
> copy & local-build & quick-test & commit)

Unrelated to the topic at hand, but I seriously hope this isn't the
standard we aim for in our version bumping. At the very least, the
build system should get checked for dependency changes (minimum deps
and otherwise)...


Mart

On Wed, Jul 17, 2019 at 6:25 AM Micha? Górny <mgorny@gentoo.org> wrote:
>
> Hello,
>
> The QA team would like to introduce the following policy:
>
> """
> Packages must not disable installing manpages via USE flags (e.g.
> USE=man or USE=doc).

Xorg libraries use USE=doc to control the build (sometimes) and
installation of thousands of developer-documentation man pages. 99.9%
of the time users don't want the developer man pages installed.

With USE=-doc the packages still install man pages for the
applications, just not the developer documentation man pages.

Is that not reasonable?

On Wed, Jul 17, 2019 at 09:09:47PM +0300, Mart Raudsepp wrote:
> ?hel kenal p?eval, K, 17.07.2019 kell 18:05, kirjutas Robin H. Johnson:
> > - significantly increases the version bump requirements (can't simply
> > copy & local-build & quick-test & commit)
> Unrelated to the topic at hand, but I seriously hope this isn't the
> standard we aim for in our version bumping. At the very least, the
> build system should get checked for dependency changes (minimum deps
> and otherwise)...
Yes, I usually also compare the release vs the previous one, on
upstream's source control, but that's outside the version bump loop in
my terminal. I'd say that's more in the research BEFORE copying
old.ebuild->new.ebuild

--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

I am also against the part of the proposal about maintainer being
responsible to prebuild the docs.

I'd also like to note that Gentoo users are empowered to locally bump
ebuild versions in this insanely easy way, it almost always works, and
it is really useful at times.

With this policy, this instanely easy way will work with lower
probability, because one would need to work around the missing docs
package for the new version.

Also, I don't see how this would work for proxy-maintained packages and
just with external contributions.

Proxies team members have to trust the docs prebuilt by non-devs?
Or they have to build the docs themselves as part of procedure of
acceptance of external contribution?

Perhaps what they can do to stay safe is to sidestep external
contributions which touch such packages.


P.S.

Not a great argument, but I just want to say this: source-based
nature of Gentoo has ingrained the feeling in me that not only
everything is open for studying, but also that, most of time, the result
of package installation on user's machine is "sterile", not tainted by
any interference from middlemen.

I think the currently discussed issue is not critical. Feels odd to
reduce the source-based nature because of that.

On 2019-07-17 16:56, Mike Gilbert wrote:
> I'm against this.
>
> I seriously doubt maintainers will take the time/effort to pre-build
> and distribute manpages. The end result of this will be additional
> hard dependencies on heavyweight packages.

I second that.

Keep in mind that aside new time requirement for pre-building and
distributing per bump, you often cannot use build system anymore for
installation. For example, please see net-misc/iputils package:
Because it's a base system package, we need to avoid insane doc
dependencies (and this is only xsltproc -- not even the craziest doc
dep hell). Therefore we already ship pre-generated man pages.

But because package provides various USE flags it's becoming a pain to
manually install required docs, see [1]. I don't believe that we want
constructs like that for many packages...


See also:
=========
[1] https://gitweb.gentoo.org/repo/gentoo.git/tree/net-misc/iputils/iputils-99999999.ebuild?id=ec886beb76dc342eec146f7e4d3785437c12bfa9#n159


--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5

On Wed, 17 Jul 2019 15:25:10 +0200
Micha? Górny <mgorny@gentoo.org> wrote:

> What are your comments?

I think there's a situation not covered by this prose which is in a bit
of a grey area as per the intentions behind it, (but I would argue is
otherwise fine).

Some systems ship multiple types of documentation, and some, simply
having the package installed means the documentation is available as
the documentation is part of the installed sources.

In the perl ecosystem, if one wants to read documentation for the
installed module "Foo", one only needs to do `perldoc Foo`.

The perl *installer* toolchain however *also* generates manpages with
the extension ".3pm", which we currently *universally* strip, because
they're surplus to requirements, and most users don't expect to use
`man` to read their perl module documentation.

( Perldoc incidentally converts POD to groff on the fly and so its
pretty much identical )

We have exemptions for shipping man-pages for executables, as they
don't get the '.3pm' extension, and they make sense, because then, "I
have /usr/bin/foo, man foo will tell me about it" as an assumption
works fine.

I don't think any of this is "a problem" as such for us, just
suspecting there are other similar cases out there not covered by the
letter of the policy.

In short, the policy suggests that our blanket removal of man pages
is harmful, even though it really isn't the case. ( That is, man pages
for this system are not primary documentation, merely secondary )

That said, we never plan on gating this removal behind a USE flag, it
would create an *impossible* amount of work.

Micha? Górny schrieb:
> Packages must not disable installing manpages via USE flags

> a. USE flags that disable building both a program and its manpage

I think it seems an implicit goal that this policy should apply to programs
and their manpages?

In that case, I would suggest to at least limit this policy to man section 1
and 8. As mattst88 explained in another post, X.org developer documentation
in man pages is not interesting for non-developers, and does usually not
describe the functions of a program.


Best regards,
Chí-Thanh Christopher Nguy?n

On Wed, 17 Jul 2019 15:25:10 +0200 Micha? Górny wrote:
> Hello,
>
> The QA team would like to introduce the following policy:
>
> """
> Packages must not disable installing manpages via USE flags (e.g.
> USE=man or USE=doc). If upstream does not ship prebuilt manpages
> and building them requires additional dependencies, the maintainer
> should build them and ship along with the package.
> """
>
>
> Explanatory note:
>
> This applies to having USE flags that specifically control building
> manpages. It obviously does not affect:
>
> a. USE flags that disable building both a program and its manpage (e.g.
> if USE=gui disables building gfrobnicate, not installing gfrobnicate(1)
> is correct),
>
> b. use of LINGUAS to control installed manpages.
>
>
> Rationale:
>
> Manpages are the basic form of user documentation on Gentoo Linux. Not
> installing them is harmful to our users. On the other hand, requiring
> additional dependencies is inconvenient. Therefore, packaging prebuilt
> manpages (whenever upstream doesn't do that already) is a good
> compromise that provides user with documentation without additional
> dependencies.
>
>
> What are your comments?

The basic foundation of Gentoo is freedom of choise for our users.
If installing man pages means no additional dependencies, than
proposed rule is ok. However if such dependencies are required it is
up to users to decide if they wan them or not.

Having USE=man (or USE=doc) for such purposes is fine. Having
USE=man enabled by default in user profile is also fine. Forcing
users to install unnecessary dependencies on minimal systems in a
no go and turns Gentoo into something else.

Best regards,
Andrew Savchenko

On Sat, 2019-07-20 at 20:50 +0300, Andrew Savchenko wrote:
> On Wed, 17 Jul 2019 15:25:10 +0200 Micha? Górny wrote:
> > Hello,
> >
> > The QA team would like to introduce the following policy:
> >
> > """
> > Packages must not disable installing manpages via USE flags (e.g.
> > USE=man or USE=doc). If upstream does not ship prebuilt manpages
> > and building them requires additional dependencies, the maintainer
> > should build them and ship along with the package.
> > """
> >
> >
> > Explanatory note:
> >
> > This applies to having USE flags that specifically control building
> > manpages. It obviously does not affect:
> >
> > a. USE flags that disable building both a program and its manpage (e.g.
> > if USE=gui disables building gfrobnicate, not installing gfrobnicate(1)
> > is correct),
> >
> > b. use of LINGUAS to control installed manpages.
> >
> >
> > Rationale:
> >
> > Manpages are the basic form of user documentation on Gentoo Linux. Not
> > installing them is harmful to our users. On the other hand, requiring
> > additional dependencies is inconvenient. Therefore, packaging prebuilt
> > manpages (whenever upstream doesn't do that already) is a good
> > compromise that provides user with documentation without additional
> > dependencies.
> >
> >
> > What are your comments?
>
> The basic foundation of Gentoo is freedom of choise for our users.
> If installing man pages means no additional dependencies, than
> proposed rule is ok. However if such dependencies are required it is
> up to users to decide if they wan them or not.
>
> Having USE=man (or USE=doc) for such purposes is fine. Having
> USE=man enabled by default in user profile is also fine. Forcing
> users to install unnecessary dependencies on minimal systems in a
> no go and turns Gentoo into something else.
>

Could you please read the proposed policy? It explicitly says you are
*not* supposed to force extra deps on users but build manpages for them.

--
Best regards,
Micha? Górny

On Sat, Jul 20, 2019 at 2:28 PM Micha? Górny <mgorny@gentoo.org> wrote:
>
> Could you please read the proposed policy? It explicitly says you are
> *not* supposed to force extra deps on users but build manpages for them.
>

This seems like a significant increase in maintainer effort compared
to just leaving things as they are for very little benefit. Simple
revbumps turn into needing to do a separate build just to build the
manpages, then package those up, host them somewhere, then fetch and
install that from the ebuild. It would be easier to just make the
whole package a binary package since then all the logic happens
outside the ebuild, and all the ebuild does is fetch/install a
tarball, which it would have to do anyway just for the manpages.

Most packages with stable build systems take almost no effort to
revbump, and this would add a fair bit of complexity. I suspect that
you'll find far more maintainers stop going to the trouble to strip
out the dependencies needed for building manpages vs maintaining two
build systems in parallel, with one having no place to host it.

Then whenever a maintainer disappears the package goes to
maintainer-needed, and anybody who wants to touch it has to figure out
how to build the manpages likely without the benefit of any scripts
the original maintainer had lying around.

If we REALLY wanted to do something like this it seems like it would
be better to build some tooling around it. Maybe an eclass combined
with a special USE flag like "man-build". A daemon would check for
packages that have this IUSE and would build the package using it,
which will generate the manpages. It would then store those pages
using a standardized naming convention. The ebuild would inherit the
eclass which would check if man-build was set, and if not it would
automatically fetch and install the manpages built by the build server
from the mirrors.

Then ebuilds that currently have IUSE=man would just inherit the
eclass and change to the man-build flag. That flag would only be used
by the build servers and not by end users, unless they wanted to build
their own manpages from scratch, which would work fine, as the flag
would not suppress building the rest of the package.

Really though I don't see THAT much benefit from doing either.

--
Rich

On Sat, 20 Jul 2019 20:28:39 +0200 Micha? Górny wrote:
> On Sat, 2019-07-20 at 20:50 +0300, Andrew Savchenko wrote:
> > On Wed, 17 Jul 2019 15:25:10 +0200 Micha? Górny wrote:
> > > Hello,
> > >
> > > The QA team would like to introduce the following policy:
> > >
> > > """
> > > Packages must not disable installing manpages via USE flags (e.g.
> > > USE=man or USE=doc). If upstream does not ship prebuilt manpages
> > > and building them requires additional dependencies, the maintainer
> > > should build them and ship along with the package.
> > > """
> > >
> > >
> > > Explanatory note:
> > >
> > > This applies to having USE flags that specifically control building
> > > manpages. It obviously does not affect:
> > >
> > > a. USE flags that disable building both a program and its manpage (e.g.
> > > if USE=gui disables building gfrobnicate, not installing gfrobnicate(1)
> > > is correct),
> > >
> > > b. use of LINGUAS to control installed manpages.
> > >
> > >
> > > Rationale:
> > >
> > > Manpages are the basic form of user documentation on Gentoo Linux. Not
> > > installing them is harmful to our users. On the other hand, requiring
> > > additional dependencies is inconvenient. Therefore, packaging prebuilt
> > > manpages (whenever upstream doesn't do that already) is a good
> > > compromise that provides user with documentation without additional
> > > dependencies.
> > >
> > >
> > > What are your comments?
> >
> > The basic foundation of Gentoo is freedom of choise for our users.
> > If installing man pages means no additional dependencies, than
> > proposed rule is ok. However if such dependencies are required it is
> > up to users to decide if they wan them or not.
> >
> > Having USE=man (or USE=doc) for such purposes is fine. Having
> > USE=man enabled by default in user profile is also fine. Forcing
> > users to install unnecessary dependencies on minimal systems in a
> > no go and turns Gentoo into something else.
> >
>
> Could you please read the proposed policy? It explicitly says you are
> *not* supposed to force extra deps on users but build manpages for them.

Could you please what the other developers have already replied to
you on this matter? This will be a significant increase in
maintenance burden for both developers and advanced users without
much to gain.

Best regards,
Andrew Savchenko

On Sat, 2019-07-20 at 23:04 +0300, Andrew Savchenko wrote:
> On Sat, 20 Jul 2019 20:28:39 +0200 Micha? Górny wrote:
> > On Sat, 2019-07-20 at 20:50 +0300, Andrew Savchenko wrote:
> > > On Wed, 17 Jul 2019 15:25:10 +0200 Micha? Górny wrote:
> > > > Hello,
> > > >
> > > > The QA team would like to introduce the following policy:
> > > >
> > > > """
> > > > Packages must not disable installing manpages via USE flags (e.g.
> > > > USE=man or USE=doc). If upstream does not ship prebuilt manpages
> > > > and building them requires additional dependencies, the maintainer
> > > > should build them and ship along with the package.
> > > > """
> > > >
> > > >
> > > > Explanatory note:
> > > >
> > > > This applies to having USE flags that specifically control building
> > > > manpages. It obviously does not affect:
> > > >
> > > > a. USE flags that disable building both a program and its manpage (e.g.
> > > > if USE=gui disables building gfrobnicate, not installing gfrobnicate(1)
> > > > is correct),
> > > >
> > > > b. use of LINGUAS to control installed manpages.
> > > >
> > > >
> > > > Rationale:
> > > >
> > > > Manpages are the basic form of user documentation on Gentoo Linux. Not
> > > > installing them is harmful to our users. On the other hand, requiring
> > > > additional dependencies is inconvenient. Therefore, packaging prebuilt
> > > > manpages (whenever upstream doesn't do that already) is a good
> > > > compromise that provides user with documentation without additional
> > > > dependencies.
> > > >
> > > >
> > > > What are your comments?
> > >
> > > The basic foundation of Gentoo is freedom of choise for our users.
> > > If installing man pages means no additional dependencies, than
> > > proposed rule is ok. However if such dependencies are required it is
> > > up to users to decide if they wan them or not.
> > >
> > > Having USE=man (or USE=doc) for such purposes is fine. Having
> > > USE=man enabled by default in user profile is also fine. Forcing
> > > users to install unnecessary dependencies on minimal systems in a
> > > no go and turns Gentoo into something else.
> > >
> >
> > Could you please read the proposed policy? It explicitly says you are
> > *not* supposed to force extra deps on users but build manpages for them.
>
> Could you please what the other developers have already replied to
> you on this matter? This will be a significant increase in
> maintenance burden for both developers and advanced users without
> much to gain.
>

Yes, I get it. User experience is not important if it would mean
developers would actually do anything but the bare minimum to get
from one paycheck to another. The usual Gentoo attitude.

--
Best regards,
Micha? Górny

On Sat, Jul 20, 2019 at 4:22 PM Micha? Górny <mgorny@gentoo.org> wrote:
>
>
> Yes, I get it. User experience is not important if it would mean
> developers would actually do anything but the bare minimum to get
> from one paycheck to another. The usual Gentoo attitude.
>

Not sure where I go to sign up for those paychecks. However, even
employers have to accept that policies have a resource cost to them.

Requiring people to do more than the bare minimum often just ensures
that they won't even bother to do the bare minimum. I'm all for
finding ways to standardize things so that everybody benefits at a
very low cost. This doesn't seem that, and honestly requiring
packages to bundle pre-built manpages seems a bit non-Gentooish to
begin with.

--
Rich

On Sat, 2019-07-20 at 18:16 -0400, Rich Freeman wrote:
> On Sat, Jul 20, 2019 at 4:22 PM Micha? Górny <mgorny@gentoo.org> wrote:
> >
> > Yes, I get it. User experience is not important if it would mean
> > developers would actually do anything but the bare minimum to get
> > from one paycheck to another. The usual Gentoo attitude.
> >
>
> Not sure where I go to sign up for those paychecks. However, even
> employers have to accept that policies have a resource cost to them.
>
> Requiring people to do more than the bare minimum often just ensures
> that they won't even bother to do the bare minimum. I'm all for
> finding ways to standardize things so that everybody benefits at a
> very low cost. This doesn't seem that, and honestly requiring
> packages to bundle pre-built manpages seems a bit non-Gentooish to
> begin with.
>

Then you should go and strip all those pregenerated autotools files,
and require eautoreconf from every single package.

--
Best regards,
Micha? Górny

On Sun, 21 Jul 2019 00:33:00 +0200 Micha? Górny wrote:
> On Sat, 2019-07-20 at 18:16 -0400, Rich Freeman wrote:
> > On Sat, Jul 20, 2019 at 4:22 PM Micha? Górny <mgorny@gentoo.org> wrote:
> > >
> > > Yes, I get it. User experience is not important if it would mean
> > > developers would actually do anything but the bare minimum to get
> > > from one paycheck to another. The usual Gentoo attitude.
> > >
> >
> > Not sure where I go to sign up for those paychecks. However, even
> > employers have to accept that policies have a resource cost to them.
> >
> > Requiring people to do more than the bare minimum often just ensures
> > that they won't even bother to do the bare minimum. I'm all for
> > finding ways to standardize things so that everybody benefits at a
> > very low cost. This doesn't seem that, and honestly requiring
> > packages to bundle pre-built manpages seems a bit non-Gentooish to
> > begin with.
> >
>
> Then you should go and strip all those pregenerated autotools files,
> and require eautoreconf from every single package.

This might be a good idea, actually. Version mismatch between
shipped pregenerated files and system autotools occasionally causes
some problems.

Best regards,
Andrew Savchenko

On Sat, Jul 20, 2019 at 1:22 PM Micha? Górny <mgorny@gentoo.org> wrote:
>
> On Sat, 2019-07-20 at 23:04 +0300, Andrew Savchenko wrote:
> > On Sat, 20 Jul 2019 20:28:39 +0200 Micha? Górny wrote:
> > > On Sat, 2019-07-20 at 20:50 +0300, Andrew Savchenko wrote:
> > > > On Wed, 17 Jul 2019 15:25:10 +0200 Micha? Górny wrote:
> > > > > Hello,
> > > > >
> > > > > The QA team would like to introduce the following policy:
> > > > >
> > > > > """
> > > > > Packages must not disable installing manpages via USE flags (e.g.
> > > > > USE=man or USE=doc). If upstream does not ship prebuilt manpages
> > > > > and building them requires additional dependencies, the maintainer
> > > > > should build them and ship along with the package.
> > > > > """
> > > > >
> > > > >
> > > > > Explanatory note:
> > > > >
> > > > > This applies to having USE flags that specifically control building
> > > > > manpages. It obviously does not affect:
> > > > >
> > > > > a. USE flags that disable building both a program and its manpage (e.g.
> > > > > if USE=gui disables building gfrobnicate, not installing gfrobnicate(1)
> > > > > is correct),
> > > > >
> > > > > b. use of LINGUAS to control installed manpages.
> > > > >
> > > > >
> > > > > Rationale:
> > > > >
> > > > > Manpages are the basic form of user documentation on Gentoo Linux. Not
> > > > > installing them is harmful to our users. On the other hand, requiring
> > > > > additional dependencies is inconvenient. Therefore, packaging prebuilt
> > > > > manpages (whenever upstream doesn't do that already) is a good
> > > > > compromise that provides user with documentation without additional
> > > > > dependencies.
> > > > >
> > > > >
> > > > > What are your comments?
> > > >
> > > > The basic foundation of Gentoo is freedom of choise for our users.
> > > > If installing man pages means no additional dependencies, than
> > > > proposed rule is ok. However if such dependencies are required it is
> > > > up to users to decide if they wan them or not.
> > > >
> > > > Having USE=man (or USE=doc) for such purposes is fine. Having
> > > > USE=man enabled by default in user profile is also fine. Forcing
> > > > users to install unnecessary dependencies on minimal systems in a
> > > > no go and turns Gentoo into something else.
> > > >
> > >
> > > Could you please read the proposed policy? It explicitly says you are
> > > *not* supposed to force extra deps on users but build manpages for them.
> >
> > Could you please what the other developers have already replied to
> > you on this matter? This will be a significant increase in
> > maintenance burden for both developers and advanced users without
> > much to gain.
> >
>
> Yes, I get it. User experience is not important if it would mean
> developers would actually do anything but the bare minimum to get
> from one paycheck to another. The usual Gentoo attitude.

I don't understand your reaction, but it's very common with
predictable steps to generate it:

1) You make a proposal
2) People offer feedback and ask questions
3) You respond combatively (or not at all), as if you are upset that
people perhaps are not 100% aligned with your view.

... which honestly shouldn't be at all unexpected and is precisely why
requesting comments on a proposal is valuable.

My question earlier in the thread is relevant and still unaddressed.

On Wed, 2019-07-17 at 12:09 -0700, Matt Turner wrote:
> On Wed, Jul 17, 2019 at 6:25 AM Micha? Górny <mgorny@gentoo.org> wrote:
> > Hello,
> >
> > The QA team would like to introduce the following policy:
> >
> > """
> > Packages must not disable installing manpages via USE flags (e.g.
> > USE=man or USE=doc).
>
> Xorg libraries use USE=doc to control the build (sometimes) and
> installation of thousands of developer-documentation man pages. 99.9%
> of the time users don't want the developer man pages installed.
>
> With USE=-doc the packages still install man pages for the
> applications, just not the developer documentation man pages.
>
> Is that not reasonable?

I think it's a reasonable compromise.

--
Best regards,
Micha? Górny

On Sat, 20 Jul 2019 22:22:39 +0200
Micha? Górny <mgorny@gentoo.org> wrote:

> Yes, I get it. User experience is not important if it would mean
> developers would actually do anything but the bare minimum to get
> from one paycheck to another. The usual Gentoo attitude.

You of course realise putting more demands on The Process of existing
developers, who are in short supply, and their time is also in short
supply, will, ultimately result in a detriment to the userbase, by
virtue of them becoming less capable to achieve the same volume of
tasks?

Its one thing to make this a policy, which I think on the _surface_ is
fine, but this one hints towards substantial changes we don't currently
have tooling support to streamline.

Maybe think about how we can empower developers to achieve this, and
then after it becomes remotely practical to do this _at scale_, we make
it policy to do so?

Yes, yes, I'm suggesting something perverted like a build server or
system for aggregating built man-pages on gentoo-servers automatically
as part of CI, that end users can just trivially fetch. But that's just
one approach, surely, there are others.

On 21-07-2019 23:47:02 +1200, Kent Fredric wrote:
> Yes, yes, I'm suggesting something perverted like a build server or
> system for aggregating built man-pages on gentoo-servers automatically
> as part of CI, that end users can just trivially fetch. But that's just
> one approach, surely, there are others.

Right, or we go for some (official) form of binpkgs distribution.

Fabian

--
Fabian Groffen
Gentoo on a different level

On July 20, 2019 1:22:39 PM PDT, "Micha? Górny" <mgorny@gentoo.org> wrote:
>Yes, I get it. User experience is not important if it would mean
>developers would actually do anything but the bare minimum to get
>from one paycheck to another. The usual Gentoo attitude.

Is my experience as a user really improved if a package I use is dropped because its maintainer no longer has time to maintain it because they now have to spend their N available hours per month building man pages for one package rather than maintaining two?

--
Christopher Head

Hi,

I'm with Rich on this one.  I trust that like me most of the developers
here earn pay checks from elsewhere and that our time here is either
completely volunteer work, or towards a purpose that suits that of our
employers.

Unless there is a way to automate the building of the associated man
pages on a build server in some way.  This might be possible come to
think of it.

USE flag to enable/disable bundled packages.  Any packages that gets
committed with this USE flag goes off to a build server that builds the
package and prepares an install (without bundled) and then the man pages
can be scraped from the prepared install I reckon and placed on a
standard URL, say d.g.o/manpages/${CATEGORY}/${PVR}.tar.gz  ... possibly
an eclass may be required I don't know, haven't thought about it that much.

I am in support of the idea that for any given command there should be a
"sensible" man page (it could be as simple as pointing to online
documentation), but don't see this as a show stopper.

I'll support this proposal if, and only if, there is a way to automate
the building and distribution of "bundled" man pages as proposed.

Kind Regards,
Jaco

On 2019/07/21 00:16, Rich Freeman wrote:

> On Sat, Jul 20, 2019 at 4:22 PM Micha? Górny <mgorny@gentoo.org> wrote:
>>
>> Yes, I get it. User experience is not important if it would mean
>> developers would actually do anything but the bare minimum to get
>> from one paycheck to another. The usual Gentoo attitude.
>>
> Not sure where I go to sign up for those paychecks. However, even
> employers have to accept that policies have a resource cost to them.
>
> Requiring people to do more than the bare minimum often just ensures
> that they won't even bother to do the bare minimum. I'm all for
> finding ways to standardize things so that everybody benefits at a
> very low cost. This doesn't seem that, and honestly requiring
> packages to bundle pre-built manpages seems a bit non-Gentooish to
> begin with.
>

On Mon, 22 Jul 2019 11:00:42 +0200
Jaco Kroon <jaco@uls.co.za> wrote:

> USE flag to enable/disable bundled packages.  Any packages that gets
> committed with this USE flag goes off to a build server that builds the
> package and prepares an install (without bundled) and then the man pages
> can be scraped from the prepared install I reckon and placed on a
> standard URL, say d.g.o/manpages/${CATEGORY}/${PVR}.tar.gz  ... possibly
> an eclass may be required I don't know, haven't thought about it that much.

In general, I really do like these suggestions for a dedicated USE flag
for use on gentoo automated build servers.

I think in general, this idea could even be extended to achieve more
than just pure MAN page generation, just implementation could be a bit
spicy, maybe even need future EAPI changes.

If the end result is a collection of asset tarballs of some kind that
contain various aspects of the same package, where the ebuild itself
dictates how the ebuild should be built on the build server, then it
enables us to approximate debian-esque deployment models where
"maintainer decides what aspects you get", but without the need to
strip end-users of essential utility.

Though I suspect *literally* using USE flags for this as-is might be
the wrong approach, as that just causes user-side pollution :/

Perhaps there's an Out-Of-Band strategy that can be employed, maybe
even using files not currently under PMS control.

IDK. I do get the impression we're "on the right track" with this, just
I don't like the proposals as-stated completely.

On Mon, Jul 22, 2019 at 8:35 AM Kent Fredric <kentnl@gentoo.org> wrote:
>
> Though I suspect *literally* using USE flags for this as-is might be
> the wrong approach, as that just causes user-side pollution :/
>

Maybe in some other situations this might be true, but as I mentioned
in my previous email, users who DO want to build their own manpages
wouldn't want to use the pregenerated ones. Also, packages that have
them need to know on the user side so that they can fetch them. So,
there is a relationship between packages that need to have manpages
pregenerated and the package manager.

--
Rich

On Mon, 22 Jul 2019 09:18:38 -0400
Rich Freeman <rich0@gentoo.org> wrote:

> So,
> there is a relationship between packages that need to have manpages
> pregenerated and the package manager.

My objection re: pollution is more to the point that this propagation
of mechanisms that are inherently "package manager control flow" into
USE flags is an anti-feature.

USE="test" is already objectionable enough, but adding Yet Another Use
Flag, and potentially, adding it to every package is just asking for
trouble, asking for more ways for the portage resolver to get confused,
asking for more ways for annoying auto-unmask mechanics to fire, and
asking for more reasons for people to have to come to #gentoo and have
somebody hold their hand trying to make sense of the huge mountain of
spew ( which is potentially unrelated to the real problem, obscured by
portage pushing past the problem before barfing ), and I'd rather we
reduced that problem, not expanded on it.

Like for instance, a common problem is USE="test" introducing cycles,
some of them are unavoidable, and portage is completely unable to
provide a working merge plan, because it can't even *consider* the
option of temporarily disabling USE="test" to resolve the cycle, and
then restoring USE="test" and building it a second time.

End Users who want to employ FEATURES="test" blanket wide across
portage have to hand-curate a collection of tools and hacks to make it
work.

Obviously, this is also exacerbated as portage can't soft-install a
package or a collection of packages, for the purpose of testing
integrity, prior to deploying them to the system, which would be
necessary for X -> Y -> Z -> X to proceed far enough that X can be
rebuilt, and tested, without potentially deploying a broken X[-test] to
the system, and without potentially deploying Y and Z, which could in
turn be broken due to X[-test] not throwing a failing test.

If portage could do that, there's so many other things it could be
doing too ...

Like, ( and this is getting a bit OT ):

Auto-Reaping build-only dependencies as soon as no targets in the merge
plan need them.

On Sat, Jul 20, 2019 at 08:50:29PM +0300, Andrew Savchenko wrote:
> On Wed, 17 Jul 2019 15:25:10 +0200 Micha? Górny wrote:
> > Hello,
> >
> > The QA team would like to introduce the following policy:
> >
> > """
> > Packages must not disable installing manpages via USE flags (e.g.
> > USE=man or USE=doc). If upstream does not ship prebuilt manpages
> > and building them requires additional dependencies, the maintainer
> > should build them and ship along with the package.
> > """
> >
> >
> > Explanatory note:
> >
> > This applies to having USE flags that specifically control building
> > manpages. It obviously does not affect:
> >
> > a. USE flags that disable building both a program and its manpage (e.g.
> > if USE=gui disables building gfrobnicate, not installing gfrobnicate(1)
> > is correct),
> >
> > b. use of LINGUAS to control installed manpages.
> >
> >
> > Rationale:
> >
> > Manpages are the basic form of user documentation on Gentoo Linux. Not
> > installing them is harmful to our users. On the other hand, requiring
> > additional dependencies is inconvenient. Therefore, packaging prebuilt
> > manpages (whenever upstream doesn't do that already) is a good
> > compromise that provides user with documentation without additional
> > dependencies.
> >
> >
> > What are your comments?
>
> The basic foundation of Gentoo is freedom of choise for our users.
> If installing man pages means no additional dependencies, than
> proposed rule is ok. However if such dependencies are required it is
> up to users to decide if they wan them or not.
>
> Having USE=man (or USE=doc) for such purposes is fine. Having
> USE=man enabled by default in user profile is also fine. Forcing
> users to install unnecessary dependencies on minimal systems in a
> no go and turns Gentoo into something else.
>
> Best regards,
> Andrew Savchenko

I am going to divert topics here... "freedom"... like freedom to post on a
mailing list without restriction (e.g. whitelisting) ?

--
Cheers,
Aaron

On Mon, 22 Jul 2019 21:04:07 -0400
Aaron Bauman <bman@gentoo.org> wrote:

> I am going to divert topics here... "freedom"... like freedom to post on a
> mailing list without restriction (e.g. whitelisting) ?

Please don't, this ain't going anywhere.

On Sat, Jul 20, 2019 at 06:16:24PM -0400, Rich Freeman wrote:
> On Sat, Jul 20, 2019 at 4:22 PM Micha? Górny <mgorny@gentoo.org> wrote:
> >
> >
> > Yes, I get it. User experience is not important if it would mean
> > developers would actually do anything but the bare minimum to get
> > from one paycheck to another. The usual Gentoo attitude.
> >
>
> Not sure where I go to sign up for those paychecks. However, even
> employers have to accept that policies have a resource cost to them.
>
> Requiring people to do more than the bare minimum often just ensures
> that they won't even bother to do the bare minimum. I'm all for

I do like that. I will send you the royalties for quoting you.

> finding ways to standardize things so that everybody benefits at a
> very low cost. This doesn't seem that, and honestly requiring
> packages to bundle pre-built manpages seems a bit non-Gentooish to
> begin with.
>
> --
> Rich
>

Regarding pre-built manpages, I think you have missed Michal's (yea, I don't
fancy letters on my keyboard) point here. He is looking at a compromise of:

1. I want some documentation
2. It doesn't ship from upstream (without crazy extra deps)
3. Gentoo guy hooked me up and packaged it pre-built with it
4. Thanks!

--
Cheers,
Aaron

On Sun, Jul 21, 2019 at 08:30:03AM +0200, Micha? Górny wrote:
> On Wed, 2019-07-17 at 12:09 -0700, Matt Turner wrote:
> > On Wed, Jul 17, 2019 at 6:25 AM Micha? Górny <mgorny@gentoo.org> wrote:
> > > Hello,
> > >
> > > The QA team would like to introduce the following policy:
> > >
> > > """
> > > Packages must not disable installing manpages via USE flags (e.g.
> > > USE=man or USE=doc).
> >
> > Xorg libraries use USE=doc to control the build (sometimes) and
> > installation of thousands of developer-documentation man pages. 99.9%
> > of the time users don't want the developer man pages installed.
> >
> > With USE=-doc the packages still install man pages for the
> > applications, just not the developer documentation man pages.
> >
> > Is that not reasonable?
>
> I think it's a reasonable compromise.
>
> --
> Best regards,
> Micha? Górny
>

If no one noticed. I think the original proposal is looking for some "common
sense" here.

--
Cheers,
Aaron

On Mon, 22 Jul 2019 21:08:51 -0400
Aaron Bauman <bman@gentoo.org> wrote:

> 1. I want some documentation
> 2. It doesn't ship from upstream (without crazy extra deps)
> 3. Gentoo guy hooked me up and packaged it pre-built with it
> 4. Thanks!

The proposal as-stated is:

1. Documentation requires even 1 additional dep
2. Thou may not use a USE flag for this
3. Thus, if you want to elide the dependency from *any* merge graph,
you must elide it from *all* merge graphs.
4. Thus, you must locally perform some non-standard hackery that will
be different for every package to produce these, work out where to put
it which is also not standardised, and also prohibit the user from
being able to update these themselves via a revision bump, _AND_ you
will need to put in place non-standard mechanisms to ensure it gets
updated when you update the package, in order for the documentation
not to diverge from the sources.

There's a lot of "Ummmm, thats bad" in point 4.

Hence, counter-proposals are trying to look at a way to achieve points
2 & 3 in your list, without resorting to barbaric torture and inherent
fragility.

We understand the /achieve, but the mechanism proposed doesn't suit, as
stated.

On 07/22/19 21:08, Aaron Bauman wrote:
> On Sat, Jul 20, 2019 at 06:16:24PM -0400, Rich Freeman wrote:
>> On Sat, Jul 20, 2019 at 4:22 PM Micha? Górny <mgorny@gentoo.org> wrote:
>>>
>>>
>>> Yes, I get it. User experience is not important if it would mean
>>> developers would actually do anything but the bare minimum to get
>>> from one paycheck to another. The usual Gentoo attitude.
>>>
>>
>> Not sure where I go to sign up for those paychecks. However, even
>> employers have to accept that policies have a resource cost to them.
>>
>> Requiring people to do more than the bare minimum often just ensures
>> that they won't even bother to do the bare minimum. I'm all for
>
> I do like that. I will send you the royalties for quoting you.
>
>> finding ways to standardize things so that everybody benefits at a
>> very low cost. This doesn't seem that, and honestly requiring
>> packages to bundle pre-built manpages seems a bit non-Gentooish to
>> begin with.
>>
>> --
>> Rich
>>
>
> Regarding pre-built manpages, I think you have missed Michal's (yea, I don't
> fancy letters on my keyboard) point here. He is looking at a compromise of:
>
> 1. I want some documentation
> 2. It doesn't ship from upstream (without crazy extra deps)
> 3. Gentoo guy hooked me up and packaged it pre-built with it
> 4. Thanks!
>
This seems rather more like an argument for reviving GRP with the flags
to generate man pages enabled [1] and encouraging people to install from
that without installing build-only dependencies if they need all of the
documentation while maintaining a smaller installed footprint and
dependency graphs than it does for requiring ebuild maintainers to jump
through hoops and remove such flags because of yet another not quite
thought through proposal.

[1] No points for guessing where a ready supply of prebuilt man pages
could be found if that were to be done. Nor for realizing that making
manpage packages out of it could probably be robustly scripted in less
time than has been invested in this discussion so far.

Am Dienstag, 23. Juli 2019, 04:00:07 CEST schrieb Kent Fredric:
> On Mon, 22 Jul 2019 21:08:51 -0400
> Aaron Bauman <bman@gentoo.org> wrote:
>
> > 1. I want some documentation
> > 2. It doesn't ship from upstream (without crazy extra deps)
> > 3. Gentoo guy hooked me up and packaged it pre-built with it
> > 4. Thanks!
>
> The proposal as-stated is:
>
> 1. Documentation requires even 1 additional dep
> 2. Thou may not use a USE flag for this
> 3. Thus, if you want to elide the dependency from *any* merge graph,
> you must elide it from *all* merge graphs.
> 4. Thus, you must locally perform some non-standard hackery that will
> be different for every package to produce these, work out where to put
> it which is also not standardised, and also prohibit the user from
> being able to update these themselves via a revision bump, _AND_ you
> will need to put in place non-standard mechanisms to ensure it gets
> updated when you update the package, in order for the documentation
> not to diverge from the sources.
What about a compromise?:
Deliver a (prebuild) manpage as package maintainer by default, but keep
a use flag "man-build" (or whatever) that builds the man page for everyone
(also the maintainer herself) with use of the crazy extra deps. So a user can
do (incomplete) version bumps and gets a manpage and the maintainer
gets the prebuild manpage in a defined way.


>
> There's a lot of "Ummmm, thats bad" in point 4.
>
> Hence, counter-proposals are trying to look at a way to achieve points
> 2 & 3 in your list, without resorting to barbaric torture and inherent
> fragility.
>
> We understand the /achieve, but the mechanism proposed doesn't suit, as
> stated.

Regards,
Gerion

On Tue, 23 Jul 2019 13:38:28 +0200
Gerion Entrup <gerion.entrup@flump.de> wrote:

> What about a compromise?:
> Deliver a (prebuild) manpage as package maintainer by default, but keep
> a use flag "man-build" (or whatever) that builds the man page for everyone
> (also the maintainer herself) with use of the crazy extra deps. So a user can
> do (incomplete) version bumps and gets a manpage and the maintainer
> gets the prebuild manpage in a defined way.

You're missing the part where the maintainer is, by the policy,
required to, for every bump:

1. Ensure the generated documentation is extracted from the build
2. Packaged into a tarball somewhere
3. Uploaded to a server that can host that tarball
4. Update the package to use that.

Failure to do this will mean you're shipping out-dated documentation to
the user.

This series of back-flips is just not practical at present, and
introduces more steps where mistakes can break the ebuild.

On Wed, 2019-07-24 at 00:17 +1200, Kent Fredric wrote:
> On Tue, 23 Jul 2019 13:38:28 +0200
> Gerion Entrup <gerion.entrup@flump.de> wrote:
>
> > What about a compromise?:
> > Deliver a (prebuild) manpage as package maintainer by default, but keep
> > a use flag "man-build" (or whatever) that builds the man page for everyone
> > (also the maintainer herself) with use of the crazy extra deps. So a user can
> > do (incomplete) version bumps and gets a manpage and the maintainer
> > gets the prebuild manpage in a defined way.
>
> You're missing the part where the maintainer is, by the policy,
> required to, for every bump:
>
> 1. Ensure the generated documentation is extracted from the build
> 2. Packaged into a tarball somewhere
> 3. Uploaded to a server that can host that tarball
> 4. Update the package to use that.
>
> Failure to do this will mean you're shipping out-dated documentation to
> the user.

I fail to see how this could happen, unless you'd be using terrible
hacks.

> This series of back-flips is just not practical at present, and
> introduces more steps where mistakes can break the ebuild.

From this thread, it seems that most devs find it impractical to even
test their ebuilds.

--
Best regards,
Micha? Górny

On Tue, 23 Jul 2019 14:39:16 +0200
Micha? Górny <mgorny@gentoo.org> wrote:

> > Failure to do this will mean you're shipping out-dated documentation to
> > the user.
>
> I fail to see how this could happen, unless you'd be using terrible
> hacks.

What part in my series of steps did you not understand?

All that has to happen is somebody does the bump, and doesn't notice
the documentation didn't change when they did the bump, when it in
fact, aught to have changed.

And just because _I'm_ capable of scruitinizing painfully everything
about upstreams changes and actually running tests when I maintain
things, doesn't mean I can actually rely on my fellow devs to do the
same.

The number of bugs I spot that are "somebody bumped a package, didn't
even add the new dependencies that were painfully obvious if you even
looked at upstreams changes" is too damn high, to the point my
intuition is often "see developer bump package, expect them to do it
wrong, look at what they changed, then sigh after predicting correctly"

I don't like this, but there's f-all I can actually do about it.

You have to plan for developers to cock things up, because they're
human, and that's what humans do.

Hi Micha?,

On 2019/07/23 14:39, Micha? Górny wrote:
> On Wed, 2019-07-24 at 00:17 +1200, Kent Fredric wrote:
>> On Tue, 23 Jul 2019 13:38:28 +0200
>> Gerion Entrup <gerion.entrup@flump.de> wrote:
>>
>>> What about a compromise?:
>>> Deliver a (prebuild) manpage as package maintainer by default, but keep
>>> a use flag "man-build" (or whatever) that builds the man page for everyone
>>> (also the maintainer herself) with use of the crazy extra deps. So a user can
>>> do (incomplete) version bumps and gets a manpage and the maintainer
>>> gets the prebuild manpage in a defined way.
>> You're missing the part where the maintainer is, by the policy,
>> required to, for every bump:
>>
>> 1. Ensure the generated documentation is extracted from the build
>> 2. Packaged into a tarball somewhere
>> 3. Uploaded to a server that can host that tarball
>> 4. Update the package to use that.
>>
>> Failure to do this will mean you're shipping out-dated documentation to
>> the user.
> I fail to see how this could happen, unless you'd be using terrible
> hacks.
And therein lies the issue.  We would.
>> This series of back-flips is just not practical at present, and
>> introduces more steps where mistakes can break the ebuild.
> From this thread, it seems that most devs find it impractical to even
> test their ebuilds.

No.  They've been saying that the overhead of maintaining the above
mentioned terrible hacks is unacceptable.  Imagine this:

In order to build man pages I need to pull in 20 additional packages. 
So when I roll new ebuild, I need those 20 ... not an issue for me, so
now I need to build the man pages, and I need to create a tarball with
those.  A tarball which won't exist at the time when I initially build,
so it's not available to generate a Manifest.  So first I have to avoid
those from SRC_URI completely.  Then once I've deployed the pre-built
manpages, I need to re-add it.

So every time there is an upstream version bump, this needs to be
rechecked and determined whether the manpages also needs to be bumped,
or I need to bump unconditionally.  More overhead.

This is outright annoying.  Unless it can be automated properly. And I
believe it might be possible, but it'll involve yet more base complexity
by adding build-time dependencies to build man pages to a separate
depend (or at least flag them with a USE=buildman flag), somehow portage
would need to first sort out the building and deployment of the separate
SRC_URI for man pages before adding to the Manifest.  You get where I'm
going I hope.

Everybody agrees with your base premise:  It's ideal to ship (optional)
documentation along with every single package, especially if it doesn't
have to pull in a boatload of dependencies.

Kind Regards,
Jaco

On Wed, 2019-07-24 at 00:59 +1200, Kent Fredric wrote:
> On Tue, 23 Jul 2019 14:39:16 +0200
> Micha? Górny <mgorny@gentoo.org> wrote:
>
> > > Failure to do this will mean you're shipping out-dated documentation to
> > > the user.
> >
> > I fail to see how this could happen, unless you'd be using terrible
> > hacks.
>
> What part in my series of steps did you not understand?
>
> All that has to happen is somebody does the bump, and doesn't notice
> the documentation didn't change when they did the bump, when it in
> fact, aught to have changed.

Manpages always change because the program version changes. You ought
to include PV in the distfile name. Then you regenerate them always,
and you can't miss it.

--
Best regards,
Micha? Górny

On Mon, Jul 22, 2019 at 09:04:07PM -0400, Aaron Bauman wrote:
> On Sat, Jul 20, 2019 at 08:50:29PM +0300, Andrew Savchenko wrote:
> > On Wed, 17 Jul 2019 15:25:10 +0200 Micha? Górny wrote:
> > > Hello,
> > >
> > > The QA team would like to introduce the following policy:
> > >
> > > """
> > > Packages must not disable installing manpages via USE flags (e.g.
> > > USE=man or USE=doc). If upstream does not ship prebuilt manpages
> > > and building them requires additional dependencies, the maintainer
> > > should build them and ship along with the package.
> > > """
> > >
> > >
> > > Explanatory note:
> > >
> > > This applies to having USE flags that specifically control building
> > > manpages. It obviously does not affect:
> > >
> > > a. USE flags that disable building both a program and its manpage (e.g.
> > > if USE=gui disables building gfrobnicate, not installing gfrobnicate(1)
> > > is correct),
> > >
> > > b. use of LINGUAS to control installed manpages.
> > >
> > >
> > > Rationale:
> > >
> > > Manpages are the basic form of user documentation on Gentoo Linux. Not
> > > installing them is harmful to our users. On the other hand, requiring
> > > additional dependencies is inconvenient. Therefore, packaging prebuilt
> > > manpages (whenever upstream doesn't do that already) is a good
> > > compromise that provides user with documentation without additional
> > > dependencies.
> > >
> > >
> > > What are your comments?
> >
> > The basic foundation of Gentoo is freedom of choise for our users.
> > If installing man pages means no additional dependencies, than
> > proposed rule is ok. However if such dependencies are required it is
> > up to users to decide if they wan them or not.
> >
> > Having USE=man (or USE=doc) for such purposes is fine. Having
> > USE=man enabled by default in user profile is also fine. Forcing
> > users to install unnecessary dependencies on minimal systems in a
> > no go and turns Gentoo into something else.
> >
> > Best regards,
> > Andrew Savchenko
>
> I am going to divert topics here... "freedom"... like freedom to post on a
> mailing list without restriction (e.g. whitelisting) ?
>
> --
> Cheers,
> Aaron

All, my response above was reported to COMREL as "Pure troll/provocation
off-topic on gentoo-dev"

My intent here was to challenge bircoph's apparent contradictions of "freedom"
of choice and "freedom" of posting on mailing lists.

I apologize for appearing to troll/provoke anyone.

--
Cheers,
Aaron

[2019-07-21 08:36:41-0700] Christopher Head:
> On July 20, 2019 1:22:39 PM PDT, "Micha? Górny" <mgorny@gentoo.org> wrote:
> >Yes, I get it. User experience is not important if it would mean
> >developers would actually do anything but the bare minimum to get
> >from one paycheck to another. The usual Gentoo attitude.
>
> Is my experience as a user really improved if a package I use is dropped because its maintainer no longer has time to maintain it because they now have to spend their N available hours per month building man pages for one package rather than maintaining two?
>

Well I guess this is quality versus quantity, which depends if gentoo is tight on some of theses packages missing manpages or not… something which I'm not aware of as a proxy-maint contributor but I can imagine other devs with this issue as well.

On 07/23/19 09:02, Jaco Kroon wrote:
> Hi Micha?,
>
> On 2019/07/23 14:39, Micha? Górny wrote:
>> On Wed, 2019-07-24 at 00:17 +1200, Kent Fredric wrote:
>>> On Tue, 23 Jul 2019 13:38:28 +0200
>>> Gerion Entrup <gerion.entrup@flump.de> wrote:
>>>
>>>> What about a compromise?:
>>>> Deliver a (prebuild) manpage as package maintainer by default, but keep
>>>> a use flag "man-build" (or whatever) that builds the man page for
>>>> everyone
>>>> (also the maintainer herself)  with use of the crazy extra deps. So
>>>> a user can
>>>> do (incomplete) version bumps and gets a manpage and the maintainer
>>>> gets the prebuild manpage in a defined way.
>>> You're missing the part where the maintainer is, by the policy,
>>> required to, for every bump:
>>>
>>> 1. Ensure the generated documentation is extracted from the build
>>> 2. Packaged into a tarball somewhere
>>> 3. Uploaded to a server that can host that tarball
>>> 4. Update the package to use that.
>>>
>>> Failure to do this will mean you're shipping out-dated documentation to
>>> the user.
>> I fail to see how this could happen, unless you'd be using terrible
>> hacks.
> And therein lies the issue.  We would.
>>> This series of back-flips is just not practical at present, and
>>> introduces more steps where mistakes can break the ebuild.
>>  From this thread, it seems that most devs find it impractical to even
>> test their ebuilds.
>
> No.  They've been saying that the overhead of maintaining the above
> mentioned terrible hacks is unacceptable.  Imagine this:
>
> In order to build man pages I need to pull in 20 additional packages. 
> So when I roll new ebuild, I need those 20 ... not an issue for me, so
> now I need to build the man pages, and I need to create a tarball with
> those.  A tarball which won't exist at the time when I initially build,
> so it's not available to generate a Manifest.  So first I have to avoid
> those from SRC_URI completely.  Then once I've deployed the pre-built
> manpages, I need to re-add it.
>
> So every time there is an upstream version bump, this needs to be
> rechecked and determined whether the manpages also needs to be bumped,
> or I need to bump unconditionally.  More overhead.
>
> This is outright annoying.  Unless it can be automated properly. And I
> believe it might be possible, but it'll involve yet more base complexity
> by adding build-time dependencies to build man pages to a separate
> depend (or at least flag them with a USE=buildman flag), somehow portage
> would need to first sort out the building and deployment of the separate
> SRC_URI for man pages before adding to the Manifest.  You get where I'm
> going I hope.
>
> Everybody agrees with your base premise:  It's ideal to ship (optional)
> documentation along with every single package, especially if it doesn't
> have to pull in a boatload of dependencies.
>
As an apparently noncorporeal being, I am curious as to why the opinions
of other apparently noncorporeal beings [1] are not valued. Further, I
would like to remind you that shipping documentation by default does not
necessarily imply forcing workarounds to avoid optional documentation,
while the proposal in question explicitly would.

[1]
https://archives.gentoo.org/gentoo-dev/message/f9503369d19a2efd635eb90ac472a962

> Kind Regards,
> Jaco
>
>
>

On Tue, 23 Jul 2019 23:56:52 -0400
desultory <desultory@gentoo.org> wrote:

> avoid optional documentation,
> while the proposal in question explicitly would

I assume you meant 'optional dependencies' here right? :)

On 07/24/19 10:40, Kent Fredric wrote:
> On Tue, 23 Jul 2019 23:56:52 -0400
> desultory <desultory@gentoo.org> wrote:
>
>> avoid optional documentation,
>> while the proposal in question explicitly would
>
> I assume you meant 'optional dependencies' here right? :)
>
The user-side effects pf the proposal in question, were it to become
policy, would be that anyone seeking to not install what is presently
optional documentation would either be:
(1) wasting build time and space (and, depending on implementation,
dependencies) on their build system (potentially masking the files from
being installed);
(2) wasting the space in their installed image(s) (if they did not mask
the files which would not currently be installed anyway); or
(3) wasting their own time working around what the developers would be
required by policy to implement by repackaging the software themselves
to avoid the time and space drawbacks (though this would generally be
expected to be a rather exceptional case, as it would be relatively
extreme to avoid what would be a distfile and some file masking from the
user side).

Developer-side effects of the proposal in question would explicitly
force developers to use bespoke workarounds to avoid adding optional
dependencies to packages, for questionable gains.

So, while I was commenting on user-side effects, the phrasing applies to
developer-side effects given s/documentation/dependencies/.

As I have noted elsewhere, there is a solution for which the majority of
the tooling exists which could achieve the same ends as the proposal in
question without causing developers in general significant additional
overhead beyond the status quo, while as a side effect providing
additional services to users. However, the proposal in question
specifically avoids offloading the newly generated work to automated
systems in favor of, evidently, optimizing for maximum consumption of
resources with minimal standardization.

On Thu, 25 Jul 2019 00:10:49 -0400
desultory <desultory@gentoo.org> wrote:

> The user-side effects pf the proposal in question, were it to become
> policy, would be that anyone seeking to not install what is presently
> optional documentation would either be:
> (1) wasting build time and space (and, depending on implementation,
> dependencies) on their build system (potentially masking the files from
> being installed);
> (2) wasting the space in their installed image(s) (if they did not mask
> the files which would not currently be installed anyway); or
> (3) wasting their own time working around what the developers would be
> required by policy to implement by repackaging the software themselves
> to avoid the time and space drawbacks (though this would generally be
> expected to be a rather exceptional case, as it would be relatively
> extreme to avoid what would be a distfile and some file masking from the
> user side).

Those concerns as per current policy is unrelated to the
dependency-control-of-USE issue presented here.

Its already established not to use a USE flag to toggle building of man
pages if it doesn't require additional dependencies.

These are _man_ pages we're talking about, not general purpose
documentation.

End users who don't like man pages are encouraged to use the relevant
FEATURES to strip them from the installed image, or use INSTALL_MASK or
something. ( FEATURES=noman )

The GOAL here is to provide man pages in *all* circumstances, and, if
additional dependencies are required to build them, to ALWAYS install
them, or NEVER install them, and then, find a secondary way to obtain
man pages (prebuilt)

The Total Cost of man pages as pure files on the target system is tiny,
and has practically no risk with regards to complexity.

But the cost of *dependencies* has risk, and can inflate to complexity,
causing much more real problems for more users than a few kb of .1.bz2
files.

Hence, why we gate them with USE flags: Because it provides an out for
that complexity ( at the cost of giving a different kind of complexity,
and a reduction in utility if somebody has to opt-out to get around
that initial complexity )

And hence why the counter-proposal to USE flags to solve that
complexity a different way is "prebuild them yourself and ship
them" ( as that eliminates all the dependency complexity and USE flag
complexity user side, while also giving them the man pages -> Quality )

Just the current mechanisms for this counter-proposal shift that
inescapable complexity to a place where it becomes harder to manage in
a standardized way.

And I don't think shifting this complexity to maintainers is the right
step, even though I want the same deliverables.

That's why I'd rather a way to shift this complexity to a build
service, ... albeit, it introduces some complexity of its own with the
building and distribution of these man pages.

Complexity is a pain-in-the-ass, you can't get rid of it, only shift it
around till it stops hurting.

However, all that said, If we're going to shoot some kind of
documentation in the face for the pain its dependency-complexity
introduces, let it be "info".

- Far fewer people enjoy or can actually get useful information out of
info pages
- Its build tooling frequently has dizzying problems in dependency
complexity and fragility, frequently made worse by portage getting
build ordering wrong after perl upgrades.[1]

(Hopefully we've fixed the worst of that, but this is plutonium, a gift
that keeps giving cancer)

1: https://bugs.gentoo.org/buglist.cgi?quicksearch=texinfo

On 07/25/19 04:14, Kent Fredric wrote:
> On Thu, 25 Jul 2019 00:10:49 -0400
> desultory <desultory@gentoo.org> wrote:
>
>> The user-side effects pf the proposal in question, were it to become
>> policy, would be that anyone seeking to not install what is presently
>> optional documentation would either be:
>> (1) wasting build time and space (and, depending on implementation,
>> dependencies) on their build system (potentially masking the files from
>> being installed);
>> (2) wasting the space in their installed image(s) (if they did not mask
>> the files which would not currently be installed anyway); or
>> (3) wasting their own time working around what the developers would be
>> required by policy to implement by repackaging the software themselves
>> to avoid the time and space drawbacks (though this would generally be
>> expected to be a rather exceptional case, as it would be relatively
>> extreme to avoid what would be a distfile and some file masking from the
>> user side).
>
> Those concerns as per current policy is unrelated to the
> dependency-control-of-USE issue presented here.
>
> Its already established not to use a USE flag to toggle building of man
> pages if it doesn't require additional dependencies.
>
> These are _man_ pages we're talking about, not general purpose
> documentation.
>
> End users who don't like man pages are encouraged to use the relevant
> FEATURES to strip them from the installed image, or use INSTALL_MASK or
> something. ( FEATURES=noman )
>
> The GOAL here is to provide man pages in *all* circumstances, and, if
> additional dependencies are required to build them, to ALWAYS install
> them, or NEVER install them, and then, find a secondary way to obtain
> man pages (prebuilt)
>
> The Total Cost of man pages as pure files on the target system is tiny,
> and has practically no risk with regards to complexity.
>
> But the cost of *dependencies* has risk, and can inflate to complexity,
> causing much more real problems for more users than a few kb of .1.bz2
> files.
>
> Hence, why we gate them with USE flags: Because it provides an out for
> that complexity ( at the cost of giving a different kind of complexity,
> and a reduction in utility if somebody has to opt-out to get around
> that initial complexity )
>
So, we more or less concur on those points, or are you attempting to
convey some other meaning by restating points?

> And hence why the counter-proposal to USE flags to solve that
> complexity a different way is "prebuild them yourself and ship
> them" ( as that eliminates all the dependency complexity and USE flag
> complexity user side, while also giving them the man pages -> Quality )
>
> Just the current mechanisms for this counter-proposal shift that
> inescapable complexity to a place where it becomes harder to manage in
> a standardized way.
>
Are you referring to a QA mandate to package or build man pages,
regardless as a counter proposal to the status quo? If so, why? It is a
proposal; the status quo is, at the risk of redundancy, the present
state of things.

> And I don't think shifting this complexity to maintainers is the right
> step, even though I want the same deliverables.
>
> That's why I'd rather a way to shift this complexity to a build
> service, ... albeit, it introduces some complexity of its own with the
> building and distribution of these man pages.
>
As I have noted twice before in discussion of this proposal, such a
build service once existed, and indeed could alternately be provided as
a side effect of one or more of the tinderboxes still in operation, and
could with some additional scripting automatically generate such packages.

> Complexity is a pain-in-the-ass, you can't get rid of it, only shift it
> around till it stops hurting.
>
> However, all that said, If we're going to shoot some kind of
> documentation in the face for the pain its dependency-complexity
> introduces, let it be "info".
>
> - Far fewer people enjoy or can actually get useful information out of
> info pages
> - Its build tooling frequently has dizzying problems in dependency
> complexity and fragility, frequently made worse by portage getting
> build ordering wrong after perl upgrades.[1]
>
> (Hopefully we've fixed the worst of that, but this is plutonium, a gift
> that keeps giving cancer)
>
> 1: https://bugs.gentoo.org/buglist.cgi?quicksearch=texinfo
>
Since when is anyone proposing extirpating man pages on the whole? I am
simply making the rather simple suggestion that pulling in more packages
to support presently optional documentation as newly mandated
documentation when such documentation is neither expected nor desired by
the users of systems onto which it would be installed is not a net
benefit to anyone. Even default on USE flags would be a better "fix" for
the purported problem then making maintainers generate, package, and
publish man pages themselves.

On Thu, 25 Jul 2019 23:56:33 -0400
desultory <desultory@gentoo.org> wrote:

> Since when is anyone proposing extirpating man pages on the whole? I am
> simply making the rather simple suggestion that pulling in more packages
> to support presently optional documentation as newly mandated
> documentation when such documentation is neither expected nor desired by
> the users of systems onto which it would be installed is not a net
> benefit to anyone.

Mostly because all things that provide texinfo files have to depend on
texinfo, and use texinfo tools to compile their info files.

And because presently, the required ubiquitous dependency is causing
problems, due to the dependency graph going pear shaped. ( though we
maaay have solved that, its hard to tell, we worked around it with
bundled deps ... )

This leads to a situation where anything that uses texinfo, *may* want
to provide a way to remove that dependency conditionally to avoid
suffering, and it is reasonable to imagine somebody doing this.

And this is already being done with a USE flag in many packages[1]

But, policy as proposed makes the only way to do this to pre-build
texinfo files yourself and hand-ship them.

Which is amusing, because the info situation is unlike man in one
specific way: That the majority of users probably don't want them.

Yet, all the packages without a USE gating is making these users suffer
problems in portage upgrades.

Making developers hand-bundle prebuilt info files instead of depending
on texinfo with a use flag?

I think you'll just see more people actually opt to solve the
dependency problem by nuking the texinfo generation of build cycle
entirely, and hoping nobody notices.

And unlike USE-gated dependencies that can yelled at by QA using simple
static analysis tools, QA yelling at people for nuking man pages might
be a little harder to implement tools for. ( But FTR, I don't
personally care if texinfo gets shot in the process, it is nothing but
pain for me )

> Even default on USE flags would be a better "fix" for
> the purported problem then making maintainers generate, package, and
> publish man pages themselves.

On that I *kinda* agree, I think? But the reason they're not defaulting
on, is because the complexity it creates can cause breakage, and for
every 1 user that wants to read a man page, there are 10 who just need
the program (or library) to just F-ing install already[2] so they can
go back to focusing on the thing that they actually care about.

So "generate man pages and make installs break for lots of people" is a
bad default.

1: https://qa-reports.gentoo.org/output/genrdeps/dindex/sys-apps/texinfo
2: Lest there be confusion, this is not my rhetoric, this is just me
channelling the average user who has to ask for help in #gentoo yet
again to solve a problem that has had to be solved many dozens of times
over, who is not a deity of package management quirks and struggles to
make sense of portage errors or comprehend random build failures due to
bad build-ordering. Sometimes gentoo is barely usable for even lesser
deities, and we aught to be doing more to put the power in the users
hands to make this crap just stop.