Mailing List Archive

On Saturday 18 September 2004 10:57, Anthony Gorecki wrote:
> If Gentoo is committed to offering users a large degree of customizability,
> wouldn't segregating the various KDE components lend itself to continuing
> the evolution of that flexibility? I would cheer at the ability to type
> "emerge kmail" rather than "emerge kdepim".

This defnitely should go onto our FAQ if we have one.

Please see bug #11123 for the previous discussion on the subject.

This is the key paragraph quoted from there:
A kde configure script takes as much as a minute or more to run. Today when
you emerge all of kde you run ~17 configure scripts, i.e. as much as 20
minutes goes there (of course everything depends on the speed of the
machine). If all kde-base packages are split into separate subpackage
ebuilds, you'll get hundreds of subpackags (200+). That means an extra 200
minutes -- 3.5 hours -- added to the compilation time of all of KDE. And most
people do want the whole of kde. This is unacceptable

That said, the recent config-cache patch to portage, if accepted, could make
this rather painless, so now this issue is finally worth raising again. (And
perhaps unsermake could also help somehow?) I haven't tried it out yet
though, so let's have some active KDE maintainer comment ;-)

--
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD 0069 508D 9143 8D5F 8951

> If Gentoo is committed to offering users a large degree of
> customizability,
> wouldn't segregating the various KDE components lend itself to continuing
> the
> evolution of that flexibility? I would cheer at the ability to type
> "emerge
> kmail" rather than "emerge kdepim".

It would be wonderful to have this functionality - there are two hurdles
though. One is that someone needs to implement it. The other is that
someone needs to maintain it. The second part is the harder of the two.

Currently, it's pretty difficult to just maintain the ebuilds in their
current state, if you consider how in flux dependencies are between
releases.

I would love to see this packaging scheme migrated to sub packages within
portage, but after having spent the past year and half with Gentoo/KDE I
can tell you it's not something I'm personally going to try and implement.
If you're up for the challenge, then let's talk.

Caleb




--
gentoo-dev@gentoo.org mailing list

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Caleb Tennis wrote:
| I would love to see this packaging scheme migrated to sub packages within
| portage

Would you mind elaborating in regard to this sub-packages proposition?
Possibly provide an example using a common package? Also, how would
this differ from having a virtual with a set of depends, such as found
in kde-base/kde?

- -Nick Dimiduk
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (Darwin)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBTEDKMRGACoLVQ1kRAgCqAJ41wBpm/FmjHC/DUNHTi++UVE+sCQCeO/2Q
/nCXUojFbSaJWUNzoAMH+iw=
=ui8C
-----END PGP SIGNATURE-----

--
gentoo-dev@gentoo.org mailing list

On Saturday 18 September 2004 2:16 am, Dan Armak wrote:
> A kde configure script takes as much as a minute or more to run. Today when
> you emerge all of kde you run ~17 configure scripts, i.e. as much as 20
> minutes goes there (of course everything depends on the speed of the
> machine). If all kde-base packages are split into separate subpackage
> ebuilds, you'll get hundreds of subpackags (200+).

Perhaps instead of using completely independent packages for the software
applications, a set of "pseudo-packages" could be created to alleviate the
extra configuration requirements?

I've previously used the DO_NOT_COMPILE option for the KDE ebuilds and
successfully screened out many of the unwanted packages. If the dependencies
for any given software application were known (herein lies the large amount
of maintenance), it should then be possible to manipulate that environment
variable to only compile what is necessary for the user. Granted it would
take a fair amount of script-work, however it's an option to consider.


> And most people do want the whole of kde.

Unfortunately, I don't fall into the category of "most people" when it comes
to KDE's software; the above comment is a prime example of why I migrated to
Gentoo and Linux as opposed to windows.


--
Anthony Gorecki
Ectro-Linux Foundation

On Saturday 18 September 2004 21:02, Anthony Gorecki wrote:
> Perhaps instead of using completely independent packages for the software
> applications, a set of "pseudo-packages" could be created to alleviate the
> extra configuration requirements?
I don't understand what you mean by pseudo-packages, unless it's the
DO_NOT_COMPILE thing below. Please elaborate.

>
> I've previously used the DO_NOT_COMPILE option for the KDE ebuilds and
> successfully screened out many of the unwanted packages.
Which doesn't scale, because portage can't manage those dependencies. You
can't depend on just one piece of kdebase (eg khtml) this way, and you can't
add/remove just one piece without also recompiling all other pieces you want
to keep.

This and similar solutions have been discussed to death before now, see bug
#11123.

> If the
> dependencies for any given software application were known (herein lies the
> large amount of maintenance),
IIRC debian and other distributions already provide separate packages for the
individual kde apps, and so presumably have the entire dependency tree mapped
out. Besides, missing dependencies as such are relatively easy to catch and,
once diagnosed, trivial to fix compared, to other problems.

> it should then be possible to manipulate that
> environment variable to only compile what is necessary for the user.
> Granted it would take a fair amount of script-work, however it's an option
> to consider.
See above.

>
> > And most people do want the whole of kde.
>
> Unfortunately, I don't fall into the category of "most people" when it
> comes to KDE's software; the above comment is a prime example of why I
> migrated to Gentoo and Linux as opposed to windows.
I mentioned 'most people' simply to point out that we're assigning limited
resources to the greatest demand. As Caleb says, with enough maintainers this
would be quite doable (provided the config caching portage enhancement
alleviated the performance issue).

--
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD 0069 508D 9143 8D5F 8951

On Saturday 18 September 2004 21:29, Dan Armak wrote:
> Which doesn't scale, because portage can't manage those dependencies. You
> can't depend on just one piece of kdebase (eg khtml) this way, and you
> can't add/remove just one piece without also recompiling all other pieces
> you want to keep.
Err, of course, khtml _is_ in kdelibs. Bad example, sorry.

--
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD 0069 508D 9143 8D5F 8951

On Saturday 18 September 2004 11:29 am, Dan Armak wrote:
> I don't understand what you mean by pseudo-packages, unless it's the
> DO_NOT_COMPILE thing below. Please elaborate.

I would have used "virtual packages", however I don't believe they provide the
functionality that the subsequent example would have required.


> Which doesn't scale, because portage can't manage those dependencies. You
> can't depend on just one piece of kdebase (eg khtml) this way, and you
> can't add/remove just one piece without also recompiling all other pieces
> you want to keep.

It would certainly require a fair amount of new scripting, regardless of how
the system was implemented.

Regarding adding and removing packages, what prevents Portage from compiling
only the KMail components (and dependencies) of the kdepim package, and then
merging it onto the filesystem? Likewise, they could be removed in a similar
fashion. It seems as though the only major difference is that the source
files would be stored in a shared archive rather than an independent archive.


> This and similar solutions have been discussed to death before now, see bug
> #11123.

I have, though we're still no closer to an actual solution. Bug #11123 was
last updated quite a while ago, and I don't believe that twelve replies
constitutes a heavily discussed topic. I don't mean to seem abrasive, however
this issue needs to be addressed, not deferred.


> I mentioned 'most people' simply to point out that we're assigning limited
> resources to the greatest demand. As Caleb says, with enough maintainers
> this would be quite doable (provided the config caching portage enhancement
> alleviated the performance issue).

I realize that community support only extends as far as it is volunteered, but
perhaps instead of simply stating that it would be possible with more
contributions and passing off the topic, a more concrete solution can be
found?

At the very least, if the Gentoo community were to agree on the best way to
implement the KDE package segregation, regardless of the required volunteer
time, it would be a step in the right direction. I would certainly be willing
to volunteer in helping to maintain the packages if they could be properly
handled by Portage.

You mentioned "enough" maintainers. Assuming that the current maintainers are
already strained with keeping packages up-to-date, approximately how many new
volunteers would be needed?


--
Anthony Gorecki
Ectro-Linux Foundation

The way I see it is that part of the problem with dependency resolution
is that we're source based. Debian and the other binary based distros
have an easier time because for them it (probably) boils down to compile
once, and then split the binaries, libraries and headers into separate
binary trees and do the dependency resolution on those. For example,
you compile kdelibs, separate out kdelib and khtml and make the latter
depend on the former, et voila. With Gentoo this process takes on a
much much higher complexity.
--
Seemant Kulleen
http://dev.gentoo.org/~seemant

Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x3458780E
Key fingerprint = 23A9 7CB5 9BBB 4F8D 549B 6593 EDA2 65D8 3458 780E

On Saturday 18 September 2004 22:19, Seemant Kulleen wrote:
> The way I see it is that part of the problem with dependency resolution
> is that we're source based. Debian and the other binary based distros
> have an easier time because for them it (probably) boils down to compile
> once, and then split the binaries, libraries and headers into separate
> binary trees and do the dependency resolution on those. For example,
> you compile kdelibs, separate out kdelib and khtml and make the latter
> depend on the former, et voila. With Gentoo this process takes on a
> much much higher complexity.

I'm probably missing your point. How are the dependencies more complex for us?
KDE apps have very few DEPENDS that aren't also RDEPENDS, so if we had
separate ebuilds for all the apps, it would be relatively easy to enter all
the dependencies - it would only require major effort once.

AFAICS the only complex dependency issue arises if you want to separate
kdelibs into pieces as well, and iirc even debian doesn't go that far; I
don't think we need to, either.

Come to think of it, there is one other issue; some kde makefiles are badly
written, and link against libraries built in the source tree under $S rather
than against installed libraries under /usr/kde/.... To separate the library
and app into two ebuilds we'd need to fix such makefiles. Because I'm
speaking from somewhat dated knowledge here, and have nevr actually tried to
fix such makefiles ;-), I don't know how difficult it'd be or how widespread
the problem is today.

Anything else I'm forgetting?

--
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD 0069 508D 9143 8D5F 8951

On Saturday 18 September 2004 22:09, Anthony Gorecki wrote:
> > Which doesn't scale, because portage can't manage those dependencies. You
> > can't depend on just one piece of kdebase (eg khtml) this way, and you
> > can't add/remove just one piece without also recompiling all other pieces
> > you want to keep.
>
> It would certainly require a fair amount of new scripting, regardless of
> how the system was implemented.
>
> Regarding adding and removing packages, what prevents Portage from
> compiling only the KMail components (and dependencies) of the kdepim
> package, and then merging it onto the filesystem? Likewise, they could be
> removed in a similar fashion. It seems as though the only major difference
> is that the source files would be stored in a shared archive rather than an
> independent archive.
Doesn't that simply kmail etc. are in separate ebuilds? How are your proposed
pseudo-packages different (less costly) from regular ebuilds?

>
> > This and similar solutions have been discussed to death before now, see
> > bug #11123.
>
> I have, though we're still no closer to an actual solution. Bug #11123 was
> last updated quite a while ago, and I don't believe that twelve replies
> constitutes a heavily discussed topic. I don't mean to seem abrasive,
> however this issue needs to be addressed, not deferred.
I'm sorry if I seemed dismissive. 11123 isn't the sole previous discussion of
this issue; it is itself a summary of previous and parallel discussions
gentoo-dev etc, which is why it only has 12 replies - it wasn't used as the
main discussion forum. It may even be that some rejected solution(s) isn't
mentioned in.

The key point of previous discussions, which is also described in 11123, is
the paragraph about the speed of configure i quoted here previously. To me
this always was the major stumbling block. Lack of manpower is rather Caleb's
problem, so I guess I'll let him comment about it ;-)

> At the very least, if the Gentoo community were to agree on the best way to
> implement the KDE package segregation, regardless of the required volunteer
> time, it would be a step in the right direction. I would certainly be
> willing to volunteer in helping to maintain the packages if they could be
> properly handled by Portage.
>
> You mentioned "enough" maintainers. Assuming that the current maintainers
> are already strained with keeping packages up-to-date, approximately how
> many new volunteers would be needed?
IMHO the best way, from the maintainers' POV, would be to be able to use
perfectly ordinary separate ebuilds for KDE apps. And, this would require
something like the config cache to be viable.

A word about my position here; I -used- to be one of the gentoo-KDE
maintainers but left about a year ago (my sig nonwithstanding, since I didn't
use it until now) and came back just a few days ago and straight into this
discussion ;-)

Meanwhile Caleb's been the lead (only?) maintainer, so it's all up to him, my
opinions are just that - opinions. That said I do want to re-join the KDE
maintainers team; it'll probably take me a couple of weeks to get up to
speed, right now I'm still struggling with the gcc 3.4 upgrade.

--
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD 0069 508D 9143 8D5F 8951

On Saturday 18 September 2004 12:45 pm, Dan Armak wrote:
> Doesn't that simply kmail etc. are in separate ebuilds? How are your
> proposed pseudo-packages different (less costly) from regular ebuilds?

Please keep in mind that the example below is only intended to serve as such.

Partially separate, but not quite. What I had in mind is more in line with a
piece of functionality similar to that of a symlink: Instead of manually
tearing apart the individual KDE distribution packages after every revision,
they could be left as-is. When a KMail installation was requested, for
example, its dependencies could be calculated against the existing KDE
packages and configured using DO_NOT_INSTALL to build only the files that
were required for KMail and were not already merged onto the filesystem.

This would add a certain degree of complexity, in that the system would be
working with large collections of software in a closed environment, however
it would lessen the amount of maintenance for the package maintainers, who
would otherwise need to be continually breaking up software components into
dedicated packages. I also suspect that Portage would need to be modified to
accommodate that method of operation.


> I'm sorry if I seemed dismissive. 11123 isn't the sole previous discussion
> of this issue

I wish I could claim that the developers and users in #kde were only being
dismissive, rather than absolute, about their packaging scheme. Individual
packages would save me a lot of time and effort in hacking their software to
suit my workstation :)


> Lack of manpower is rather Caleb's problem

Thankfully.


> IMHO the best way, from the maintainers' POV, would be to be able to use
> perfectly ordinary separate ebuilds for KDE apps. And, this would require
> something like the config cache to be viable.

ccache could be quite helpful, regardless of the package that was being built;
I agree that it would improve KDE compilation time, especially. What do you
mean by "being viable"? I haven't had a chance to experiment with the
software, though I thought it was already working reasonably well for most
computer systems.


--
Anthony Gorecki
Ectro-Linux Foundation

* On Sat Sep-18-2004 at 01:13:20 PM -0700, Anthony Gorecki said:
> On Saturday 18 September 2004 12:45 pm, Dan Armak wrote:
>
> > IMHO the best way, from the maintainers' POV, would be to be able to use
> > perfectly ordinary separate ebuilds for KDE apps. And, this would require
> > something like the config cache to be viable.
>
> ccache could be quite helpful, regardless of the package that was being built;
> I agree that it would improve KDE compilation time, especially. What do you
> mean by "being viable"? I haven't had a chance to experiment with the
> software, though I thought it was already working reasonably well for most
> computer systems.

ccache != config cache!

ccache caches binaries built from C code.

Dan was talking about a configure (as in ./configure before a make)
cache.

--
Sami Samhuri

On Saturday 18 September 2004 1:33 pm, Sami Samhuri wrote:
> ccache != config cache!
>
> ccache caches binaries built from C code.
>
> Dan was talking about a configure (as in ./configure before a make)
> cache.

Ah, my mistake. I should have known better, considering his statement wasn't
sensible when used in reference to ccache.


--
Anthony Gorecki
Ectro-Linux Foundation

On Saturday 18 September 2004 23:13, Anthony Gorecki wrote:
> Partially separate, but not quite. What I had in mind is more in line with
> a piece of functionality similar to that of a symlink: Instead of manually
> tearing apart the individual KDE distribution packages after every
> revision, they could be left as-is. When a KMail installation was
> requested, for example, its dependencies could be calculated against the
> existing KDE packages and configured using DO_NOT_INSTALL to build only the
> files that were required for KMail and were not already merged onto the
> filesystem.
We wouldn't need to 'tear apart' the kde packages except just once, when we
first wrote the broken-out ebuilds. After that each ebuild would essentially
do what you describe: take eg the kdepim tarball and build only kmail using
DO_NOT_COMPILE and related techniques.

> > IMHO the best way, from the maintainers' POV, would be to be able to use
> > perfectly ordinary separate ebuilds for KDE apps. And, this would require
> > something like the config cache to be viable.
>
> ccache could be quite helpful, regardless of the package that was being
> built; I agree that it would improve KDE compilation time, especially. What
> do you mean by "being viable"? I haven't had a chance to experiment with
> the software, though I thought it was already working reasonably well for
> most computer systems.

Misunderstanding here.

ccache=='compiler cache', which caches the work done by gcc/g++.

I was talking about the recently proposed, and still experimental afaik,
'configure cache' patch to portage, posted to this list on September 10th by
Stuart Hebert:
http://dev.gentoo.org/~stuart/confcache/

It caches the tests run by configure scripts. I haven't tried it yet and
haven't seen any reports on how well it works for kde specifically. We -need-
a solution to the configure script run times issue, as described in #11123.
If that's not resolved, no amount of maintainer manpower can help, because
the move to separate ebuilds would mean a large increase in emerging time for
people who do want all of kde. If this configure cache works well and is
accepted into portage, it'd be the best possible solution to the problem.

--
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD 0069 508D 9143 8D5F 8951

On Saturday 18 September 2004 23:38, Dan Armak wrote:
> I was talking about the recently proposed, and still experimental afaik,
> 'configure cache' patch to portage, posted to this list on September 10th
> by Stuart Hebert:
> http://dev.gentoo.org/~stuart/confcache/
>
> It caches the tests run by configure scripts. I haven't tried it yet and
> haven't seen any reports on how well it works for kde specifically. We
> -need- a solution to the configure script run times issue, as described in
> #11123. If that's not resolved, no amount of maintainer manpower can help,
> because the move to separate ebuilds would mean a large increase in
> emerging time for people who do want all of kde. If this configure cache
> works well and is accepted into portage, it'd be the best possible solution
> to the problem.

...Which is why we've been dismissive of this issue until just now: no such
config cache, or an laternative solution to the problem, was available.

--
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD 0069 508D 9143 8D5F 8951

Anthony Gorecki posted <200409181103.01872.anthony@ectrolinux.com>,
excerpted below, on Sat, 18 Sep 2004 11:02:53 -0700:

> On Saturday 18 September 2004 2:16 am, Dan Armak wrote:
>> A kde configure script takes as much as a minute or more to run. Today when
>> you emerge all of kde you run ~17 configure scripts, i.e. as much as 20
>> minutes goes there (of course everything depends on the speed of the
>> machine). If all kde-base packages are split into separate subpackage
>> ebuilds, you'll get hundreds of subpackags (200+).
>
> Perhaps instead of using completely independent packages for the software
> applications, a set of "pseudo-packages" could be created to alleviate the
> extra configuration requirements?
>
> I've previously used the DO_NOT_COMPILE option for the KDE ebuilds and
> successfully screened out many of the unwanted packages. If the dependencies
> for any given software application were known (herein lies the large amount
> of maintenance), it should then be possible to manipulate that environment
> variable to only compile what is necessary for the user. Granted it would
> take a fair amount of script-work, however it's an option to consider.

As a Gentoo and KDE user that hopes to someday be a Gentoo developer..

I to have had a bit of experience with DO_NOT_COMPILE. Namely, with the
KDE 3.3.0 beta ebuilds, on AMD64 (and often running ~amd64 keyword added
then removed toolchain elements, mainly binutils and glibc, so bug reports
might be "noise"), I had to use that to tell KPovModeler (IIRC) from
kdegraphics not to compile, as it was running into an error rather more
advanced than my ability to fix.

I have a suggestion based on the now ~* masked due to linux26-headers
requirement glibc-2.3.4-20040916 and its new method for handling locales.
It has a new "userlocales" use flag, which when activated, only merges the
locales listed in /etc/locales.build. Since I only use en-US and can't
read anything other than en anyway, that's a pretty neat way to cut
compile time for all those locales I don't need. I like the idea!

I'd like to see it expanded, with perhaps several *.build files, maybe
moved out of /etc/ directly into /etc/portage/build or some such. In each
case, the idea would be to allow user configuration of only parts of a
generally large package to be merged. Another perfect example might be
xorg. Create a /etc/portage/build/xorg.build file, which would have two
specifiers, graphics adaptor modules to be compiled, and input modules to
be compiled, so I could simply say ATI (or better yet Radeon) and
mouse/keyboard only. (The make.conf VIDEO_CARDS= thing that I saw hinted
at in some post I read never seems to work, here, possibly because I don't
have it set quite right, as I've never come across anything listing
exactly what the options are and I probably have it wrong. On the last
xorg merge I did I used ebuild and limited it by hand, and found it took
ati not radeon even tho they are separate drivers, and I have radeon in
the make.conf entry, so next time I'm going to try ati and see if it takes
that automatically. It'd help if there was a comment in the ebuild
listing the choices...)

Of course, there'd be a kde.build file as well. It would provide a
section for each kdexxx ebuild, and simply subtract anything not listed
from the build process by adding it to DO_NOT_COMPILE.

The idea here would be to control it in two ways. First, as with glibc,
it'd only be enabled if the required use flag (disabled by default) was
on. Otherwise, as glibc does with the locales, it would just build
all of them. Second, the file would be put there by default, with all
sub-packages in it by default, so even if the use flag was enabled, unless
a user specifically commented out or deleted a subpackage from the list,
it would be compiled as it is now. I'd suggest not even giving the option
of breaking kdelibs apart, and possibly keeping kdebase whole, as well,
simply to minimize the possibility of breaking anything else that depends
on them.

Then, at the top of the file put a big hairy warning about how some
package components, particularly in kdebase, are depended on by others,
and to disable the use flag and recompile all packages if there are
dependency issues, and then leave everything else up to the user. Any
bugs on related dependency issues would be marked invalid, see the warning
in the file, etc. so it wouldn't become a big support issue. Further, all
it would take for implementation would be a function added to kde-dist, to
parse kde.build, and add anything not listed in the appropriate ebuild
section to the DO_NOT_COMPILE env-var. This function could then either
be invoked by the appropriate kde-dist function directly (thus requiring
no individual ebuild mods at all, keeping in mind that it defaults to
current build-all behavior so it could be added without changing current
behavior on current ebuilds except that another use flag would be there
now), or called by the specific ebuilds.

Alternatively, kde.build could list only the packages one did NOT want
merged, in each section. That would probably be simpler to implement with
KDE, since all that would be required is to brainlessly put everything
listed directly in the env-var, but it might be preferable for all the
*.build files to operate the same, and since glibc's locale.build already
operates by inclusion of what's listed, that's the way I wrote it, above.

IMO, this idea fills all the customizability requirements, while at the
same time being quite simple to implement and support, because it doesn't
do any subpackaging with the subsequent dependency and maintenance issues.

The one disadvantage from a user perspective is that it would require
remerging the entire ebuild if another subpackage was desired, but given
the trade-offs, I see that as quite acceptable. Users would either
compile and merge everything as they now do by default, or they'd be
excising subpackages as they so chose, with results no different than a
forced recompile if they changed their mind about use flags.

From my perspective, this would be a perfect way to implement
configurability at a level beyond that which it is practical to do with
use flags, due to their incredible proliferation if one tries. Obviously,
I really like the solution as I've seen it in glibc! =:^)

--
Duncan - List replies preferred. No HTML msgs.
"They that can give up essential liberty to obtain a little
temporary safety, deserve neither liberty nor safety." --
Benjamin Franklin



--
gentoo-dev@gentoo.org mailing list

On Saturday 18 September 2004 21:36, Dan Armak wrote:
> Come to think of it, there is one other issue; some kde makefiles are badly
> written, and link against libraries built in the source tree under $S
> rather than against installed libraries under /usr/kde/.... To separate the
> library and app into two ebuilds we'd need to fix such makefiles. Because
> I'm speaking from somewhat dated knowledge here, and have nevr actually
> tried to fix such makefiles ;-), I don't know how difficult it'd be or how
> widespread the problem is today.
>
> Anything else I'm forgetting?
The aren't badly written, this is the right behavior, For example kmail wants
to compile against libkdepim. You can't link against a library that isn't
already installed o an old installed one with less functions.

Bye!
--
Simone Gotti
<simone.gotti@email.it>
http://kde-bluetooth.sf.net

--
gentoo-dev@gentoo.org mailing list

On Sunday 19 September 2004 11:30, Simone Gotti wrote:
> On Saturday 18 September 2004 21:36, Dan Armak wrote:
> > Come to think of it, there is one other issue; some kde makefiles are
> > badly written, and link against libraries built in the source tree under
> > $S rather than against installed libraries under /usr/kde/.... To
> > separate the library and app into two ebuilds we'd need to fix such
> > makefiles. Because I'm speaking from somewhat dated knowledge here, and
> > have nevr actually tried to fix such makefiles ;-), I don't know how
> > difficult it'd be or how widespread the problem is today.
> >
> > Anything else I'm forgetting?
>
> The aren't badly written, this is the right behavior, For example kmail
> wants to compile against libkdepim. You can't link against a library that
> isn't already installed o an old installed one with less functions.
Should be a fallback IMHO. configure can check for things like that. At least,
we're going to have to make it do that if we go through with this :-)

As for old versions, isn't that what library versions are for?

--
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD 0069 508D 9143 8D5F 8951

On Sunday 19 September 2004 07:43, Duncan wrote:
> Then, at the top of the file put a big hairy warning about how some
> package components, particularly in kdebase, are depended on by others,
> and to disable the use flag and recompile all packages if there are
> dependency issues, and then leave everything else up to the user. Any
> bugs on related dependency issues would be marked invalid, see the warning
> in the file, etc. so it wouldn't become a big support issue.
As you say, this solution is hairy (or anything else that doesn't create real
separate ebuilds for the separate apps). I personally don't like it at all
and don't really want to see it happen even as an alternative to nothing at
all, because it's so ugly. Of course, it's very easy to implement, but it
takes away all the advantages of having a proper package manager - all the
advantages of using portage rather than state-less invocations on the order
of 'ebuild.sh file.ebuild'...

And of course this solution entails more or less not supporting it despite
having it in the portage tree - marking related bugs as invalid etc. That's
another reason I don't like it.

Perhaps I don't have the right to say this at this point, having been incative
for over a year, but this solution simply takes away too much functionality
and support from the user in order to decrease the maintainer's workload.

--
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD 0069 508D 9143 8D5F 8951

On Sunday 19 September 2004 16:48, Dan Armak wrote:
> > The aren't badly written, this is the right behavior, For example kmail
> > wants to compile against libkdepim. You can't link against a library that
> > isn't already installed o an old installed one with less functions.
>
> Should be a fallback IMHO. configure can check for things like that. At
> least, we're going to have to make it do that if we go through with this
> :-)
> As for old versions, isn't that what library versions are for?
You're right. The real problem is that library version usually are the same in
ALL kde 3.X. Because they must be Binary Compatible (B.C.) (this means of
course al source compatible). In c++ you can mantain the B.C. also adding
some new methods to a class, with the exception that this one doesn't have to
be a virtual method. I've noticed that between every version of kde some libs
ends with the same *.so.X.X also if they adds some new functions (I hope to
be wrong...).


By the way, I think that it's possible to split EVERY kde program/libs in a
single package, this can be possibile by heavily patching the Makefiles like
you've said so they can link to an installed lib instead of the local one,
this libs will be a dep of the program and it should be of course a different
ebuild.

I'm already giving a little help to caleb and carlo on the kde ebuilds and if
ypu and they think that a lot of people thinks that having single kde
programs instead of the standard modules will be a better idea I'll be VERY
happy to start hacking on this. My great fear is that it will slow down the
development of the ebuilds because this bring to a lot of additional work!
Think that probably the Makefile's patches must be changed on every kde major
release.

All I need is to know if you (gentoo kde developers) are really interested on
this work. Let me know!

Bye!
--
Simone Gotti
<simone.gotti@email.it>
http://kde-bluetooth.sf.net

--
gentoo-dev@gentoo.org mailing list

On Sunday 19 September 2004 21:08, Simone Gotti wrote:
> By the way, I think that it's possible to split EVERY kde program/libs in a
> single package, this can be possibile by heavily patching the Makefiles
> like you've said so they can link to an installed lib instead of the local
> one, this libs will be a dep of the program and it should be of course a
> different ebuild.
It's certainly possible. I even think (well, intuition...) the patching won't
have to be very heavy.

>
> I'm already giving a little help to caleb and carlo on the kde ebuilds and
> if ypu and they think that a lot of people thinks that having single kde
> programs instead of the standard modules will be a better idea I'll be VERY
> happy to start hacking on this. My great fear is that it will slow down the
> development of the ebuilds because this bring to a lot of additional work!
> Think that probably the Makefile's patches must be changed on every kde
> major release.
>
> All I need is to know if you (gentoo kde developers) are really interested
> on this work. Let me know!
As I said, I'm definitely going to try making this work, but won't have real
time until the weekend after next (and most days after that, hopefully), so
you can beat me to it.

I've just tried the confcache patch. It's integrated into portage .51 cvs
snapshots here http://dev.gentoo.org/~ferringb/ebuild-daemon/51-pre20-cvs/ -
they apparently include other highly experimental stuff and the last one has
some bugs already fixed in cvs, so be in touch with #gentoo-portage if using
these. To enable confcache, add 'confcache' to FEATURES and change the kde
ebuilds to use econf rather than call configure directly. This results in the
kde.eclass attached - the change is trivial, but as I'm on rsync not cvs atm,
I can't make a diff immediately, sorry. The change for split packages should
also be on the order of a five-liner in kde.eclass, apart from the makefile
changes.

On my athlon xp 2600, this makes the kdelibs configure run go down from 66
seconds to 28 seconds. At least 10-12 seconds of each of these two numbers is
due to makefile generation, which will be very much reduced in split-up
ebuilds, so we get 54 sec -> 16 sec, or a 5x improvement.

Also, on slower machines a far larger percent of time spent in (non-cached)
configure is due to slow compilations rather than (as here) IO, so the
benefit will be even greater there. This indicates we should at least
reevluate the emerge performance factor of splitting up the kde ebuilds.

Using my old rough formula, we get:
17 configure scripts (before splitting ebuilds) --> 20-60 minutes total
running time
splitting up --> >=220 packages (? but at least that many, I believe)
splitting up without confcache --> 220-660 minutes, i.e. 3.6-11 hours, total
running time (clearly unacceptable)
splitting up with confcache --> about 15-20% of above, i.e. 1-2.2 hours total
running time

According to these numbers, if we both add confcache and split the ebuilds,
the total time for emerging all or nearly all of kde will increase by 0.6-1.2
hours, plus the overhead from running the emerge cycle 200 more times, which
I hope is relatively negligible (a few minutes?).

Compared with the benefits (to those who don't want all of kde, and
considering that >95% of people typing 'emerge kde' to save package selection
time don't really want kdetoys and kdeedu and such, this seems on first sight
to be a reasonable tradeoff for the added functionality. What do you think,
everyone? Also note that my benchmarking is hardly scientific, so I'd be glad
if someone bothered to repeat it and compare his results to mine.

--
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD 0069 508D 9143 8D5F 8951

On Sunday 19 September 2004 22:33, Dan Armak wrote:
> On my athlon xp 2600, this makes the kdelibs configure run go down from 66
> seconds to 28 seconds. At least 10-12 seconds of each of these two numbers
> is due to makefile generation, which will be very much reduced in split-up
> ebuilds, so we get 54 sec -> 16 sec, or a 5x improvement.
Er. Well. Sorry about that, here are corrected figures:
54 seconds -> 16 seconds == 3.3x improvement.
Therefore:
17 configure scripts (before splitting ebuilds) --> 20-60 minutes total
running time
splitting up --> >=220 packages (? but at least that many, I believe)
splitting up without confcache --> 220-660 minutes, i.e. 3.6-11 hours, total
running time (clearly unacceptable)
splitting up with confcache --> about 25-30% of above, i.e. 1-3.3 hours total
running time

According to these numbers, if we both add confcache and split the ebuilds,
the total time for emerging all or nearly all of kde will increase by 0.6-2.3
hours.

--
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD 0069 508D 9143 8D5F 8951

On Sunday 19 September 2004 16:56, Dan Armak wrote:
> On Sunday 19 September 2004 07:43, Duncan wrote:
> > Then, at the top of the file put a big hairy warning about how some
> > package components, particularly in kdebase, are depended on by others,
> > and to disable the use flag and recompile all packages if there are
> > dependency issues, and then leave everything else up to the user. Any
> > bugs on related dependency issues would be marked invalid, see the
> > warning in the file, etc. so it wouldn't become a big support issue.
>
> As you say, this solution is hairy (or anything else that doesn't create
> real separate ebuilds for the separate apps). I personally don't like it at
> all and don't really want to see it happen even as an alternative to
> nothing at all, because it's so ugly. Of course, it's very easy to
> implement, but it takes away all the advantages of having a proper package
> manager - all the advantages of using portage rather than state-less
> invocations on the order of 'ebuild.sh file.ebuild'...
>
> And of course this solution entails more or less not supporting it despite
> having it in the portage tree - marking related bugs as invalid etc. That's
> another reason I don't like it.
>
> Perhaps I don't have the right to say this at this point, having been
> incative for over a year, but this solution simply takes away too much
> functionality and support from the user in order to decrease the
> maintainer's workload.

Well, I'll second you in any case. It is not a solution and offering something
willingly and then saying it is unsupported is in these kinds of cases a
medicine worse than the cure. The only "possible" solution I see would
involve useflags and useflag dependencies (still "in the works"). But I agree
that we must avoid in any case to get back the stateless mess that the lack
of dependency tracking entails.

Paul

--
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

On Sunday 19 September 2004 22:42, Dan Armak wrote:
> According to these numbers, if we both add confcache and split the ebuilds,
> the total time for emerging all or nearly all of kde will increase by
> 0.6-2.3 hours.

Oh, and some packages (eg kdebase) run automake right now, which takes a lot
more time than a non-cached configure run. Couldn't that be eliminated? Just
distribute pregenerated Makefile.in files, a configure scripts etc generated
with our patches included.

And unsermake is supposed to make disrtibuted/multiprocess compiling of kde
much faster - right now distcc with more than a few boxen isn't really used
by kde compiles - and also possibly to speed up normal compiles through a
nonrecursive makefile system and such nice things.

disclaimer; I haven't tried unsermake myself yet and don't know how mature it
is.

So we might be able to optimize the build system sufficiently that the net
performance price for the added features won't be too high even in the worst
case. Of course, that's where the lack-of-manpower issue comes in...

--
Dan Armak
Gentoo Linux developer (KDE)
Matan, Israel
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD 0069 508D 9143 8D5F 8951