Mailing List Archive

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robin H. Johnson wrote:
> It does NOT check /proc/config.gz presently. The original logic against
> not checking /proc was that we were targeting the kernel being built,
> but that's moot given the use of `uname -r` in OUTPUT_DIR.

I might be reading the eclass wrong, but it looks as though OUTPUT_DIR
is only set to use uname -r if the KV_* variables match the output of
uname -r:

[ "${KV_MAJOR}.${KV_MINOR}.${KV_PATCH}${KV_EXTRA}" == "$(uname -r)" ] && \
OUTPUT_DIR="${OUTPUT_DIR:-/lib/modules/${KV_MAJOR}.${KV_MINOR}.${KV_PATCH}${KV_EXTRA}/build}"

Otherwise it's taken from the KBUILD_DIR and then it even checks inside
the KV_DIR's Makefile to find one. So not checking /proc/config.gz
isn't a moot point anymore, which is not to say it's not a good idea,
but the reason against doing it still stands. To deal with running
kernel's, there's a get_running_version function, that will use uname to
fill out the KV_* variables appropriately.

> Proposed solution:
> ------------------
> We need to be able to reduce user error, so we cannot simply make it
> trust the user by default. So I propose that we add an environment
> variable (I'm not set on the name yet), eg:
> EXTERNALLY_BUILT_KERNEL=1
>
> This option will cause linux-info.eclass to consider ALL kernel option
> checks non-fatal. That way we still get the warnings and logs, but it
> does not stop the builds.

I'm not sure what this is solving? It doesn't seem to reduce user
error, it just allows users a way to override portage errors? It seems
unrelated to config.gz and the discussion going on in the previous
thread. I do like the idea of being able to override portage when it's
stopping us from building (incorrectly), but surely with the capability
to have non-fatal checks, we can try to minimize the times when
portage/the ebuild is wrong using that, instead of a whole new override?

> When is the above NOT enough?
> -----------------------------
> The only time that ANY kernel sources are required is when you are
> building an out-of-tree module. For this purpose, they must be
> configured.

There's a lot of ebuilds that use linux-info, and they can't all be
out-of-tree modules. Are they misusing linux-info, or are you saying
there's a specific instance in which linux-info requires configured
kernel sources?

> The check for having configured kernel sources must only be executed
> when the modules are about to be compiled. Putting it in pkg_preinst
> would block use of binpkgs on (related) machines.
>
> - If a package builds modules AND userspace, we should offer a way to
> build the userspace only, as the user can build their modules
> externally (or patch them into the kernel) [1]
> - For packages that ONLY build modules, and no userspace at all, having
> EXTERNALLY_BUILT_KERNEL=1 means that they should error out? [2]
> (this case might be thrown into the above one).

That all seems fine, but again these just seem like standard guidelines.
Is there not already some "how to write kernel module ebuilds" page
somewhere that documents how you're supposed to use linux-info?

Mike 5:)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)

iEYEARECAAYFAkqa30QACgkQu7rWomwgFXo99gCfd8NbK9QkxtJ9BKalq/n1iBxn
l0QAoLiBsvKXZRzR4Dp8HEtoxrsONCtc
=ZK9i
-----END PGP SIGNATURE-----

On Sun, Aug 30, 2009 at 09:21:24PM +0100, Mike Auty wrote:
> [ "${KV_MAJOR}.${KV_MINOR}.${KV_PATCH}${KV_EXTRA}" == "$(uname -r)" ] && \
> OUTPUT_DIR="${OUTPUT_DIR:-/lib/modules/${KV_MAJOR}.${KV_MINOR}.${KV_PATCH}${KV_EXTRA}/build}"
This check is inside the get_version call, and is ultimately used to set
KV_OUT_DIR, which is in turn used for all the config checks.

The conditional part of the check means that it's only used if the
running kernel is also the one currently pointed to by the sources
variables (which were used further up in the function to populate KV_*).

I also realize on closer reading, that the above bit will never even be
reached unless the kernel sources are present.

> Otherwise it's taken from the KBUILD_DIR and then it even checks inside
> the KV_DIR's Makefile to find one. So not checking /proc/config.gz
> isn't a moot point anymore, which is not to say it's not a good idea,
> but the reason against doing it still stands. To deal with running
> kernel's, there's a get_running_version function, that will use uname to
> fill out the KV_* variables appropriately.
FYI:
get_running_version is used in one single ebuild, in the entire tree:
sys-fs/evms/evms-2.5.5-r10.ebuild
And there it's only for a warning.

> > Proposed solution:
> > ------------------
> > We need to be able to reduce user error, so we cannot simply make it
> > trust the user by default. So I propose that we add an environment
> > variable (I'm not set on the name yet), eg:
> > EXTERNALLY_BUILT_KERNEL=1
> >
> > This option will cause linux-info.eclass to consider ALL kernel option
> > checks non-fatal. That way we still get the warnings and logs, but it
> > does not stop the builds.
> I'm not sure what this is solving? It doesn't seem to reduce user
> error, it just allows users a way to override portage errors? It seems
> unrelated to config.gz and the discussion going on in the previous
> thread. I do like the idea of being able to override portage when it's
> stopping us from building (incorrectly), but surely with the capability
> to have non-fatal checks, we can try to minimize the times when
> portage/the ebuild is wrong using that, instead of a whole new override?
The great majority of CONFIG_CHECK usage in the tree is fatal already.
It only actually needs to be fatal only when it's being used to build a
module.

However, past bugs have shown lots of users complaining that a package
doesn't work, simply because they did not have the relevant option
enabled in their kernel.

This leaves us between hand-holding the basic user's kernel configuration
(exiting if the kernel config option is not enabled), and changing all
non-module instances in the tree to be non-fatal.

If we make all the non-module uses non-fatal, there will be an increase of bugs
due to users not reading the warnings.

> > When is the above NOT enough?
> > -----------------------------
> > The only time that ANY kernel sources are required is when you are
> > building an out-of-tree module. For this purpose, they must be
> > configured.
> There's a lot of ebuilds that use linux-info, and they can't all be
> out-of-tree modules. Are they misusing linux-info, or are you saying
> there's a specific instance in which linux-info requires configured
> kernel sources?
Here's a few that I know of, off the top of my head:

app-mobilephone/gnokii/gnokii-0.6.27-r4.ebuild:
CONFIG_CHECK="UNIX98_PTYS"

/usr/portage/net-print/hplip/hplip-3.9.4b-r1.ebuild:
CONFIG_CHECK="PARPORT PPDEV"

/usr/portage/net-firewall/conntrack-tools/conntrack-tools-0.9.13.ebuild:
CONFIG_CHECK="IP_NF_CONNTRACK_NETLINK"
CONFIG_CHECK="NF_CT_NETLINK"
CONFIG_CHECK="${CONFIG_CHECK} NF_CONNTRACK
linux_chkconfig_present "NF_CONNTRACK_IPV4" || \
linux_chkconfig_present "NF_CONNTRACK_IPV6" || \

If you don't have configured kernel sources, all of these ebuilds fail right
now. They are not building modules, simply userspace. The CONFIG_CHECK elements
are not prefixed with tilde to make them non-fatal.

conntrack-tools is actually even worse, as it has a kernel version check before
the config checks, which would also fail without kernel sources.

> > The check for having configured kernel sources must only be executed
> > when the modules are about to be compiled. Putting it in pkg_preinst
> > would block use of binpkgs on (related) machines.
> >
> > - If a package builds modules AND userspace, we should offer a way to
> > build the userspace only, as the user can build their modules
> > externally (or patch them into the kernel) [1]
> > - For packages that ONLY build modules, and no userspace at all, having
> > EXTERNALLY_BUILT_KERNEL=1 means that they should error out? [2]
> > (this case might be thrown into the above one).
>
> That all seems fine, but again these just seem like standard guidelines.
> Is there not already some "how to write kernel module ebuilds" page
> somewhere that documents how you're supposed to use linux-info?
If you're building modules, most of the time you're using linux-mod, not just
linux-info. There's no document or recommended behavior in the tree for the
above actually, and I'd like to introduce one.

Packages w/ USE=modules
-----------------------
app-emulation/kvm
media-libs/libifp
sys-block/open-iscsi
x11-drivers/ati-drivers
x11-drivers/linuxwacom

No choice, modules and userspace
--------------------------------
net-firewall/ipset
media-tv/ivtv
media-tv/wis-go7007
net-dialup/intel-536ep
(partial list, I just looked for a few).

--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2@gentoo.org
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robin H. Johnson wrote:
> FYI:
> get_running_version is used in one single ebuild, in the entire tree:
> sys-fs/evms/evms-2.5.5-r10.ebuild
> And there it's only for a warning.

Ok, I was just suggesting that if there was an intention to implement
config.gz checks, they should only apply when people ask about the
running version rather than the build version. Since that doesn't seem
popular or even necessary, perhaps neither is the need to check config.gz?

> The great majority of CONFIG_CHECK usage in the tree is fatal already.
> It only actually needs to be fatal only when it's being used to build a
> module.

Ok, I see what you're suggesting now. When people want to build
packages, but portage knows their kernel isn't setup to run them
properly, then it should still build them, but warn them strongly about
it (as opposed to currently, where it'll just die).

> This leaves us between hand-holding the basic user's kernel configuration
> (exiting if the kernel config option is not enabled), and changing all
> non-module instances in the tree to be non-fatal.

Ok, so then the question is do we sacrifice correctness for fewer
(invalid) bugs? Seems like a judgement call. For what it's worth, I'm
not sure adding extra plumbing to allow smart users to bypass the checks
is the right middle ground. I'd either leave it as is, or change the
ebuilds to accurately reflect whether the userspace will build or not.

>> That all seems fine, but again these just seem like standard guidelines.
>> Is there not already some "how to write kernel module ebuilds" page
>> somewhere that documents how you're supposed to use linux-info?
> If you're building modules, most of the time you're using linux-mod, not just
> linux-info. There's no document or recommended behavior in the tree for the
> above actually, and I'd like to introduce one.

Sounds like a good idea, it might also be worth adding to the quizes, if
existing devs are asking how it should be done? I guess that's a call
on how common it is to have kernel config requirements on userspace...

Still, I think I'm on the right page, and even in agreement (which makes
me happy). 5:) Thanks!

Mike 5:)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)

iEYEARECAAYFAkqa9gkACgkQu7rWomwgFXq1PwCfTbp8hqsGZjDmsxKE21gKe1Y8
lYYAoI2EBCn5KwQdlm6Xd8u0q7KGl7gI
=Jrsa
-----END PGP SIGNATURE-----

On Sun, Aug 30, 2009 at 10:58:33PM +0100, Mike Auty wrote:
> Robin H. Johnson wrote:
> > FYI:
> > get_running_version is used in one single ebuild, in the entire tree:
> > sys-fs/evms/evms-2.5.5-r10.ebuild
> > And there it's only for a warning.
> Ok, I was just suggesting that if there was an intention to implement
> config.gz checks, they should only apply when people ask about the
> running version rather than the build version. Since that doesn't seem
> popular or even necessary, perhaps neither is the need to check config.gz?
The running version stuff may be needed to be used more than it is
presently.

If no kernel sources are present at all, all functions that check the
kernel version presently bail out. get_version is also at the very top
of linux-info's pkg_setup function, so it bails as well:
* Determining the location of the kernel source code
* Unable to find kernel sources at /usr/src/linux
...

The existing state is:
- Force the user to install sources

Our choices are:
- `uname -r` output.
- Create an override environment variable for all the checks.

/proc/config.gz comes back here again, in that, we can use it as a
middle ground with `uname -r`, rather than having the environment
variable.

So from our discussion, I propose the following:
Finding the kernel version:
---------------------------
0. (optional) give an env var to bypass entirely.
1. Use existing logic of /usr/src/linux, KERNEL_DIR et al.
2. Fall back to using the running version, with a warning.

Checking a configuration option, for non-module use:
----------------------------------------------------
0. (optional) give an env var to make all checks non-fatal.
1. Use existing logic of .config from /usr/src/linux, KERNEL_DIR et al.
2. Fall back to /proc/config.gz if available.


> > The great majority of CONFIG_CHECK usage in the tree is fatal already.
> > It only actually needs to be fatal only when it's being used to build a
> > module.
> Ok, I see what you're suggesting now. When people want to build
> packages, but portage knows their kernel isn't setup to run them
> properly, then it should still build them, but warn them strongly about
> it (as opposed to currently, where it'll just die).
Two different cases here:
1. Portage knows their kernel is not setup correctly.
2. Portage cannot find out enough info.

It's #2 that bites on virtual machines and hardened servers, and is what
I'd like to fix.

> > This leaves us between hand-holding the basic user's kernel configuration
> > (exiting if the kernel config option is not enabled), and changing all
> > non-module instances in the tree to be non-fatal.
> Ok, so then the question is do we sacrifice correctness for fewer
> (invalid) bugs? Seems like a judgement call. For what it's worth, I'm
> not sure adding extra plumbing to allow smart users to bypass the checks
> is the right middle ground. I'd either leave it as is, or change the
> ebuilds to accurately reflect whether the userspace will build or not.
It's a one-liner to give an override making all checks non-fatal, with
the downside that it can't differentiate why the checks are being made.
So yes, in the case we should simply fix all of the ebuilds (after the
kernel source check is fixed as well).

> > If you're building modules, most of the time you're using linux-mod, not just
> > linux-info. There's no document or recommended behavior in the tree for the
> > above actually, and I'd like to introduce one.
> Sounds like a good idea, it might also be worth adding to the quizes, if
> existing devs are asking how it should be done? I guess that's a call
> on how common it is to have kernel config requirements on userspace...
142 packages inherit linux-mod.
130 packages use functions or the MODULE_NAMES code from linux-mod.
105 packages inherit linux-info.
11 packages inherit both linux-info AND linux-mod.

So I propose this as resolutions from the above:
1. USE=modules added to the base profile.
2. Every package that builds kernel modules must offer USE=modules,
which can be used to disable building the kernel modules, leaving a
pure userspace build of that package.

Most of the changes to the above can be done in the linux-mod eclass, so
I don't think we'll have to touch that many packages directly.

--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2@gentoo.org
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robin H. Johnson wrote:
> The existing state is:
> - Force the user to install sources
>
> Our choices are:
> - `uname -r` output.
> - Create an override environment variable for all the checks.
>
> /proc/config.gz comes back here again, in that, we can use it as a
> middle ground with `uname -r`, rather than having the environment
> variable.

Ok, that seems a very reasonable use.

> So from our discussion, I propose the following:
> Finding the kernel version:
> ---------------------------
> 0. (optional) give an env var to bypass entirely.
> 1. Use existing logic of /usr/src/linux, KERNEL_DIR et al.
> 2. Fall back to using the running version, with a warning.
>
> Checking a configuration option, for non-module use:
> ----------------------------------------------------
> 0. (optional) give an env var to make all checks non-fatal.
> 1. Use existing logic of .config from /usr/src/linux, KERNEL_DIR et al.
> 2. Fall back to /proc/config.gz if available.

Where 2 also issues a warning, presumably? I've had a think and can't
see any repercussions in changing the default behaviour, but I'm
assuming people would only generally hit 2 with virtual machines and/or
hardened servers. Can you see either of these suffering from defaulting
to the running kernel? Do you know of many circumstances where 2 might
get hit by normal users other than those situations?

Also (and potentially off-topic), if we're using linux-info to detect
kernel sources, whether installed or not, should we also check the tree
for ebuilds that require a package which PROVIDES="sources"? I
personally use a single git checkout, since I think (possibly
mistakenly, I honestly haven't checked) that it will leave different
checkouts of the kernel all over my /usr/src directory, whenever a new
version's emerged. I've had to create a fake-sources package to trick
ebuilds that need *some* sources installed, and I'm wondering if that
should be necessary?

> Two different cases here:
> 1. Portage knows their kernel is not setup correctly.
> 2. Portage cannot find out enough info.
>
> It's #2 that bites on virtual machines and hardened servers, and is what
> I'd like to fix.

Ok, I'm all for it, and the solution you've suggested sounds fine.

> It's a one-liner to give an override making all checks non-fatal, with
> the downside that it can't differentiate why the checks are being made.
> So yes, in the case we should simply fix all of the ebuilds (after the
> kernel source check is fixed as well).

I'd definitely try and do things the right way, and a global override
(whilst easy) doesn't sound like it. Fixing all the ebuilds (and the
kernel source check) sounds like the way to go.

> So I propose this as resolutions from the above:
> 1. USE=modules added to the base profile.
> 2. Every package that builds kernel modules must offer USE=modules,
> which can be used to disable building the kernel modules, leaving a
> pure userspace build of that package.
>
> Most of the changes to the above can be done in the linux-mod eclass, so
> I don't think we'll have to touch that many packages directly.
>

Yep, although if the changes are in linux-mod, then those packages that
*only* provide kernel modules will need a way to not present that use
flag (no point installing a package is USE="-modules" and nothing gets
installed). Also, I'd assume +modules would be added as a default.

The whole plan sounds extremely sensible in general! 5:)

Mike 5:)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)

iEYEARECAAYFAkqbC3oACgkQu7rWomwgFXpwywCeKzcB0Znzuul3wq9U9ew5+SuX
scQAnjShkQTVQQrFABb8u2t0RsP9K34z
=LFlY
-----END PGP SIGNATURE-----

On Mon, Aug 31, 2009 at 12:30:02AM +0100, Mike Auty wrote:
> > Checking a configuration option, for non-module use:
> > ----------------------------------------------------
> > 0. (optional) give an env var to make all checks non-fatal.
> > 1. Use existing logic of .config from /usr/src/linux, KERNEL_DIR et al.
> > 2. Fall back to /proc/config.gz if available.
> Where 2 also issues a warning, presumably? I've had a think and can't
> see any repercussions in changing the default behaviour, but I'm
> assuming people would only generally hit 2 with virtual machines and/or
> hardened servers. Can you see either of these suffering from defaulting
> to the running kernel? Do you know of many circumstances where 2 might
> get hit by normal users other than those situations?
Yes, a warning with using /proc/config.gz.

I missed a bit for the config option.
If there is NO source of the config data, what do we do?
Error out or more warnings?

> Also (and potentially off-topic), if we're using linux-info to detect
> kernel sources, whether installed or not, should we also check the tree
> for ebuilds that require a package which PROVIDES="sources"? I
> personally use a single git checkout, since I think (possibly
> mistakenly, I honestly haven't checked) that it will leave different
> checkouts of the kernel all over my /usr/src directory, whenever a new
> version's emerged. I've had to create a fake-sources package to trick
> ebuilds that need *some* sources installed, and I'm wondering if that
> should be necessary?
This is what I use for my home workstation:
/etc/portage/profile/virtuals:virtual/linux-sources sys-kernel/git-sources
/etc/portage/profile/virtuals:virtual/alsa sys-kernel/git-sources
/etc/portage/profile/package.provided:sys-kernel/git-sources-2.6.30

Saves having any fake package like that.

> > It's a one-liner to give an override making all checks non-fatal, with
> > the downside that it can't differentiate why the checks are being made.
> > So yes, in the case we should simply fix all of the ebuilds (after the
> > kernel source check is fixed as well).
> I'd definitely try and do things the right way, and a global override
> (whilst easy) doesn't sound like it. Fixing all the ebuilds (and the
> kernel source check) sounds like the way to go.
Want to lend a hand fixing up the ebuilds then? I'll work on the eclass sources
check.

> > So I propose this as resolutions from the above:
> > 1. USE=modules added to the base profile.
> > 2. Every package that builds kernel modules must offer USE=modules,
> > which can be used to disable building the kernel modules, leaving a
> > pure userspace build of that package.
> Yep, although if the changes are in linux-mod, then those packages that
> *only* provide kernel modules will need a way to not present that use
> flag (no point installing a package is USE="-modules" and nothing gets
> installed).
There IS still a point of having the entirely empty kernel package installed:
- Split userspace/kernel packages where the userspace package has a dependency
on the module-providing package.
- other packages that might have a dependency on the module-providing package.
- /etc/modprobe.d/ files.

USE=-modules will ONLY block files installed to /lib/modules/.

> Also, I'd assume +modules would be added as a default.
I'm not sure we can get away with USE-defaults for this change, due to the
amount of EAPI=0 stuff that will be changing, so it'll be in
profiles/base/make.defaults instead.

--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2@gentoo.org
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robin H. Johnson wrote:
> Yes, a warning with using /proc/config.gz.
>
> I missed a bit for the config option.
> If there is NO source of the config data, what do we do?
> Error out or more warnings?

Well, if we can't determine whether a config option's set or not, if
it's not critical (ie, it's ~CHECK), then we warn. If there's any hard
checks, then we error out I suppose.

> This is what I use for my home workstation:
> /etc/portage/profile/virtuals:virtual/linux-sources sys-kernel/git-sources
> /etc/portage/profile/virtuals:virtual/alsa sys-kernel/git-sources
> /etc/portage/profile/package.provided:sys-kernel/git-sources-2.6.30
>
> Saves having any fake package like that.

Ah cunning, I'll have to try that...

> Want to lend a hand fixing up the ebuilds then? I'll work on the eclass sources
> check.

Sure, as long as I sure I know how we're fixing them, I'd be glad to
help. 5:) I'd just be fixing the userspace ones to make sure they're
~CHECK or would I be doing something else as well? Also should I do
that via bugs and a week's grace period, or would you recommend I just
dive right in?

> There IS still a point of having the entirely empty kernel package installed:
> - Split userspace/kernel packages where the userspace package has a dependency
> on the module-providing package.
> - other packages that might have a dependency on the module-providing package.
> - /etc/modprobe.d/ files.
>
> USE=-modules will ONLY block files installed to /lib/modules/.

Ok, presumably USE=-modules will also stop the kernel checks, otherwise
the following could occur. Let's say the foobar package builds external
modules and explicitly requires that foobar not be set in the kernel.
With USE=-modules, there's no internal version and there's no external
version, but the kernel package is installed, and the deps are happy,
even though they'll bug out when it's showtime.

Presumably there's also packages that aren't split, or depended on,
where not installing the modules is never a good idea. Is there still a
way to disable the USE flag, or will it be a requirement of the
linux-mod package?

> I'm not sure we can get away with USE-defaults for this change, due to the
> amount of EAPI=0 stuff that will be changing, so it'll be in
> profiles/base/make.defaults instead.

That's fine, just so long as it's on and not off, I don't mind what
system we use to do it. 5:)

Mike 5:)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)

iEYEARECAAYFAkqbEqkACgkQu7rWomwgFXq53QCdFi8YlDrKwkh9b0g6EX2DBI35
0t0AnRRRV+oxqQaKPqzcWxGJDtW2lszQ
=oia+
-----END PGP SIGNATURE-----

On Mon, Aug 31, 2009 at 01:00:41AM +0100, Mike Auty wrote:
> > I missed a bit for the config option.
> > If there is NO source of the config data, what do we do?
> > Error out or more warnings?
> Well, if we can't determine whether a config option's set or not, if
> it's not critical (ie, it's ~CHECK), then we warn. If there's any hard
> checks, then we error out I suppose.
+1.

> > This is what I use for my home workstation:
> > /etc/portage/profile/virtuals:virtual/linux-sources sys-kernel/git-sources
> > /etc/portage/profile/virtuals:virtual/alsa sys-kernel/git-sources
> > /etc/portage/profile/package.provided:sys-kernel/git-sources-2.6.30
> > Saves having any fake package like that.
> Ah cunning, I'll have to try that...
We should document it in the kernel web pages.

> > Want to lend a hand fixing up the ebuilds then? I'll work on the eclass sources
> > check.
> Sure, as long as I sure I know how we're fixing them, I'd be glad to
> help. 5:) I'd just be fixing the userspace ones to make sure they're
> ~CHECK or would I be doing something else as well? Also should I do
> that via bugs and a week's grace period, or would you recommend I just
> dive right in?
If you feel like reviewing ~140 packages and filing bugs for them, I
won't stop you. But I'm just going to go and fix the ones that seem
simple enough to me, and only file bugs for the complex ones.

> > There IS still a point of having the entirely empty kernel package installed:
> > - Split userspace/kernel packages where the userspace package has a dependency
> > on the module-providing package.
> > - other packages that might have a dependency on the module-providing package.
> > - /etc/modprobe.d/ files.
> >
> > USE=-modules will ONLY block files installed to /lib/modules/.
> Ok, presumably USE=-modules will also stop the kernel checks, otherwise
> the following could occur. Let's say the foobar package builds external
> modules and explicitly requires that foobar not be set in the kernel.
> With USE=-modules, there's no internal version and there's no external
> version, but the kernel package is installed, and the deps are happy,
> even though they'll bug out when it's showtime.

Err, I'm not following what you claim is the problem here?

cat/foo-mod:
inherit linux-mod
CONFIG_CHECK="!..."
MODULE_NAMES="...."

cat/foo-usr:
RDEPEND="cat/foo"

And say foo-mod provides /dev/foo for some crazy hardware.

The user sets USE=-modules because they have built cat/foo-mod on their
own, into the kernel. foo-mod installs nothing, foo-usr installs fine.
foo-usr runs, and /dev/foo already exists, so it gets used fine.

> Presumably there's also packages that aren't split, or depended on,
> where not installing the modules is never a good idea. Is there still a
> way to disable the USE flag, or will it be a requirement of the
> linux-mod package?
It needs to be ALWAYS available. The ipset bug I linked earlier in the
thread was an example of that. The userspace is useless without the
kernel code, but there is nothing to stop the user patching it into
their kernel and not having it as a module at all (as is the case on a
couple of my work boxes, which is why the bug got filed).

--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2@gentoo.org
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robin H. Johnson wrote:
> If you feel like reviewing ~140 packages and filing bugs for them, I
> won't stop you. But I'm just going to go and fix the ones that seem
> simple enough to me, and only file bugs for the complex ones.

Ok, I'll do what I can tomorrow. I'm off to bed now... 5:)

> Err, I'm not following what you claim is the problem here?
>
> cat/foo-mod:
> inherit linux-mod
> CONFIG_CHECK="!option"
>
> ...
>
> The user sets USE=-modules because they have built cat/foo-mod on their
> own, into the kernel. foo-mod installs nothing

Just checking that it will actually install nothing, rather than failing
out with an error because the kernel has "option" set. I'm just
re-iterating that CONFIG_CHECK should only occur if USE="modules". If
USE="-modules", then the CONFIG_CHECK options should probably be ignored
shouldn't they?

> It needs to be ALWAYS available. The ipset bug I linked earlier in the
> thread was an example of that. The userspace is useless without the
> kernel code, but there is nothing to stop the user patching it into
> their kernel and not having it as a module at all (as is the case on a
> couple of my work boxes, which is why the bug got filed).

Ok, fair enough. However, in that instance, ipset is a userspace
dependency, I was talking about kernel modules that have no userspace
dependencies. I suppose there could be a case where someone is trying
to custom build a userspace tool that isn't in portage yet, but I still
think it might cause confusion to allow USE="modules" on kernel modules
(without any dependencies) that then install nothing. I'm wondering
which will cause more bugs to be filed?

Mike 5:)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)

iEYEARECAAYFAkqbHEEACgkQu7rWomwgFXrEpQCeIMWofCtvwHKJoZsb3/qUyLcP
tTUAoIb3Sr645x6NY82LXK6i/3g75NF5
=uaOB
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ok,

Here's the list of ebuilds that feature a CONFIG_CHECK, and which ones
I've been through to check for either NONEED (already using ~option),
MODULE (builds a module so may really want to bomb out), and FIXED
(packages which I've fixed).

About half the list is done, I'm posting it here as a progress/grab 'em
if you want to do 'em type list (as requested by robbat2).

Mike 5:)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)

iEYEARECAAYFAkqcUhoACgkQu7rWomwgFXp3GwCfddJEgoKsDgaGTLzkouavCdVp
srQAoLUMEEwmobCBwWAHasrwJjjSt9k5
=iMAd
-----END PGP SIGNATURE-----

Robin H. Johnson wrote:
> It does NOT check /proc/config.gz presently. The original logic against
> not checking /proc was that we were targeting the kernel being built,
> but that's moot given the use of `uname -r` in OUTPUT_DIR.

That seems like a bug. OUTPUT_DIR should default to unset which would
cause the internal KV_OUT_DIR to be set to KV_DIR. However I can see
from the code that's not the case, and it's not clear why.

Relying on "uname -r" or /proc/config.gz is still something we should
avoid (unless user configured, perhaps, or in very exceptional
circumstances) however it looks like this isn't a change that you are
proposing.

Thanks for working on this. It's a sensitive area so please take care
and help people pick up the pieces. If you are really motivated, it
might be worth rethinking the whole separate output directory
implementation.

Daniel

On Mon, Sep 07, 2009 at 10:28:11AM +0100, Daniel Drake wrote:
> Robin H. Johnson wrote:
> >It does NOT check /proc/config.gz presently. The original logic against
> >not checking /proc was that we were targeting the kernel being built,
> >but that's moot given the use of `uname -r` in OUTPUT_DIR.
> That seems like a bug. OUTPUT_DIR should default to unset which
> would cause the internal KV_OUT_DIR to be set to KV_DIR. However I
> can see from the code that's not the case, and it's not clear why.
It's messy. I'll maybe tackle it after I'm done the work on configs and
USE=modules.

> Relying on "uname -r" or /proc/config.gz is still something we
> should avoid (unless user configured, perhaps, or in very
> exceptional circumstances) however it looks like this isn't a change
> that you are proposing.
The only time I'm saying that uname -r and /proc/config.gz are looked
as, is where there is no kernel sources or they are installed but not
configured. But that is as part of a larger body of work to fix up all
packages that are merely checking the kernel options during build, not
relying on the config to build.

Kernel modules still require configured sources to be installed.

> Thanks for working on this. It's a sensitive area so please take
> care and help people pick up the pieces. If you are really
> motivated, it might be worth rethinking the whole separate output
> directory implementation.
For anybody following this, we're doing the work on two tracker bugs:
283320 - Changes to linux-info
283900 - USE=modules change for linux-mod and packages.

We've got the core eclass stuff done, but not enabled yet.

If you want to test it it:
1. mv your .config or entire kernel sources away
2. I_KNOW_WHAT_I_AM_DOING=1 emerge udev

You should get warnings, but no errors.

--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2@gentoo.org
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85