Mailing List Archive

Alan McKinnon wrote:
> After my long time away from Gentoo, I thought perhaps some packages
> that always took ages to compile would have improved. I needed to
> change to ~amd64 anyway (dumb n00b mistake leaving it at amd64). So
> that's what I did and let emerge do it's thing.
>
> chromium has been building since 10:14, it's now 21:16 and still going
> so 9 hours at least on this machine to build a browser - almost as bad
> as openoffice at it's worst (regularly took 12 hours). Nodejs also
> took a while, but I didn't record time.
>
>
> What other packages have huge build times?
>
> --
> Alan McKinnon
> alan dot mckinnon at gmail dot com


I have some software you don't likely use that takes a while but one
that is common is qtwebengine or something.  If it's not that one, it's
qtweb something.  It takes about 4 hours, sometimes 5 or so. 

I think the software takes longer to compile so that we will build new
rigs.  ROFL 

Dale

:-)  :-) 

Chromium and qtwebengine have the longest build times that I have
encountered

On Tue, 12 Sep 2023 at 1:01?AM, Dale <rdalek1967@gmail.com> wrote:

> Alan McKinnon wrote:
>
> After my long time away from Gentoo, I thought perhaps some packages that
> always took ages to compile would have improved. I needed to change to
> ~amd64 anyway (dumb n00b mistake leaving it at amd64). So that's what I did
> and let emerge do it's thing.
>
> chromium has been building since 10:14, it's now 21:16 and still going so
> 9 hours at least on this machine to build a browser - almost as bad as
> openoffice at it's worst (regularly took 12 hours). Nodejs also took a
> while, but I didn't record time.
>
>
> What other packages have huge build times?
>
> --
> Alan McKinnon
> alan dot mckinnon at gmail dot com
>
>
>
> I have some software you don't likely use that takes a while but one that
> is common is qtwebengine or something. If it's not that one, it's qtweb
> something. It takes about 4 hours, sometimes 5 or so.
>
> I think the software takes longer to compile so that we will build new
> rigs. ROFL
>
> Dale
>
> :-) :-)
>

Hi Alan,

just quick and dirty, I am too tired for formalities. :) The following
list contains packages, that may be too big for tmpfs and are most
probably very time consuming to compile:

    $ < /etc/portage/package.env/no_tmpfs.conf
    # custom - 20181121 - rfischer: list packages, which are too big
for tmpfs
    #app-editors/neovim no_tmpfs.conf
    #app-emulation/qemu-kv no_tmpfs.conf
    #app-office/libreoffice no_tmpfs.conf
    #dev-db/mysql no_tmpfs.conf
    #dev-java/icedtea no_tmpfs.conf
    #dev-lang/ghc no_tmpfs.conf
    #dev-lang/ghc no_tmpfs.conf
    #dev-lang/mono no_tmpfs.conf
    #dev-lang/rust no_tmpfs.conf
    #dev-lang/spidermonkey no_tmpfs.conf
    #dev-libs/libpcre no_tmpfs.conf
    #dev-qt/qtwebengine no_tmpfs.conf #throttle_make_emerge.conf
    #mail-client/thunderbird no_tmpfs.conf
    #media-libs/opencv no_tmpfs.conf
    #media-libs/opencv no_tmpfs.conf
    #net-libs/nodejs no_tmpfs.conf
    #net-misc/openssh no_tmpfs.conf
    #sci-libs/tensorflow no_tmpfs.conf
    #sys-apps/iproute2 no_tmpfs.conf
    #sys-devel/clang no_tmpfs.conf
    #sys-devel/gcc no_tmpfs.conf
    #www-client/chromium no_tmpfs.conf #throttle_make_emerge.conf
    #www-client/firefox no_tmpfs.conf
    #www-client/ungoogled-chromium no_tmpfs.conf #throttle_make_emerge.conf

See also:

*
https://wiki.gentoo.org/wiki/Portage_TMPDIR_on_tmpfs#Considering_tmpfs.27_size
*
https://wiki.gentoo.org/wiki/Q_applets#Extracting_information_from_emerge_logs_.28qlop.29

Sleeps away.
-Ramon

On 11/09/2023 21:19, Alan McKinnon wrote:
> After my long time away from Gentoo, I thought perhaps some packages
> that always took ages to compile would have improved. I needed to
> change to ~amd64 anyway (dumb n00b mistake leaving it at amd64). So
> that's what I did and let emerge do it's thing.
>
> chromium has been building since 10:14, it's now 21:16 and still going
> so 9 hours at least on this machine to build a browser - almost as bad
> as openoffice at it's worst (regularly took 12 hours). Nodejs also
> took a while, but I didn't record time.
>
>
> What other packages have huge build times?
>
> --
> Alan McKinnon
> alan dot mckinnon at gmail dot com

--
GPG public key: 5983 98DA 5F4D A464 38FD CF87 155B E264 13E6 99BF

In addition to the reference to "qlop":

    $ qlop ungoogled-chromium | tail
    2022-08-04T19:58:22 >>> www-client/ungoogled-chromium: 9:06:54
    2022-08-05T14:27:44 >>> www-client/ungoogled-chromium: 16:19:06
    2022-08-25T11:45:37 >>> www-client/ungoogled-chromium: 8:01:54
    2022-09-01T10:03:19 >>> www-client/ungoogled-chromium: 15:27:22
    2022-09-06T16:29:49 >>> www-client/ungoogled-chromium: 9:46:16
    2022-09-14T17:48:16 >>> www-client/ungoogled-chromium: 9:30:29
    2022-10-08T03:40:44 >>> www-client/ungoogled-chromium: 1:52:16
    2022-10-21T17:58:43 >>> www-client/ungoogled-chromium: 9:24:55
    2022-12-16T17:47:27 >>> www-client/ungoogled-chromium: 20:56:08
    2023-03-20T14:12:02 <<< www-client/ungoogled-chromium: 2s

Since I am using "ccache", the compilation time is sometimes doubled.

ZzZzZzzz
-Ramon

On 11/09/2023 21:42, Ramon Fischer wrote:
> Hi Alan,
>
> just quick and dirty, I am too tired for formalities. :) The following
> list contains packages, that may be too big for tmpfs and are most
> probably very time consuming to compile:
>
>     $ < /etc/portage/package.env/no_tmpfs.conf
>     # custom - 20181121 - rfischer: list packages, which are too big
> for tmpfs
>     #app-editors/neovim no_tmpfs.conf
>     #app-emulation/qemu-kv no_tmpfs.conf
>     #app-office/libreoffice no_tmpfs.conf
>     #dev-db/mysql no_tmpfs.conf
>     #dev-java/icedtea no_tmpfs.conf
>     #dev-lang/ghc no_tmpfs.conf
>     #dev-lang/ghc no_tmpfs.conf
>     #dev-lang/mono no_tmpfs.conf
>     #dev-lang/rust no_tmpfs.conf
>     #dev-lang/spidermonkey no_tmpfs.conf
>     #dev-libs/libpcre no_tmpfs.conf
>     #dev-qt/qtwebengine no_tmpfs.conf #throttle_make_emerge.conf
>     #mail-client/thunderbird no_tmpfs.conf
>     #media-libs/opencv no_tmpfs.conf
>     #media-libs/opencv no_tmpfs.conf
>     #net-libs/nodejs no_tmpfs.conf
>     #net-misc/openssh no_tmpfs.conf
>     #sci-libs/tensorflow no_tmpfs.conf
>     #sys-apps/iproute2 no_tmpfs.conf
>     #sys-devel/clang no_tmpfs.conf
>     #sys-devel/gcc no_tmpfs.conf
>     #www-client/chromium no_tmpfs.conf #throttle_make_emerge.conf
>     #www-client/firefox no_tmpfs.conf
>     #www-client/ungoogled-chromium no_tmpfs.conf
> #throttle_make_emerge.conf
>
> See also:
>
> *
> https://wiki.gentoo.org/wiki/Portage_TMPDIR_on_tmpfs#Considering_tmpfs.27_size
> *
> https://wiki.gentoo.org/wiki/Q_applets#Extracting_information_from_emerge_logs_.28qlop.29
>
> Sleeps away.
> -Ramon
>
> On 11/09/2023 21:19, Alan McKinnon wrote:
>> After my long time away from Gentoo, I thought perhaps some packages
>> that always took ages to compile would have improved. I needed to
>> change to ~amd64 anyway (dumb n00b mistake leaving it at amd64). So
>> that's what I did and let emerge do it's thing.
>>
>> chromium has been building since 10:14, it's now 21:16 and still
>> going so 9 hours at least on this machine to build a browser - almost
>> as bad as openoffice at it's worst (regularly took 12 hours). Nodejs
>> also took a while, but I didn't record time.
>>
>>
>> What other packages have huge build times?
>>
>> --
>> Alan McKinnon
>> alan dot mckinnon at gmail dot com
>

--
GPG public key: 5983 98DA 5F4D A464 38FD CF87 155B E264 13E6 99BF

qtwebengine! yes that one took forever also. It also said my 16G of RAM was
smaller than the 16G it needed. Weird.

Anyways I enabled a swapfile and left it to run overnight

Alan

On Mon, Sep 11, 2023 at 9:31?PM Dale <rdalek1967@gmail.com> wrote:

> Alan McKinnon wrote:
>
> After my long time away from Gentoo, I thought perhaps some packages that
> always took ages to compile would have improved. I needed to change to
> ~amd64 anyway (dumb n00b mistake leaving it at amd64). So that's what I did
> and let emerge do it's thing.
>
> chromium has been building since 10:14, it's now 21:16 and still going so
> 9 hours at least on this machine to build a browser - almost as bad as
> openoffice at it's worst (regularly took 12 hours). Nodejs also took a
> while, but I didn't record time.
>
>
> What other packages have huge build times?
>
> --
> Alan McKinnon
> alan dot mckinnon at gmail dot com
>
>
>
> I have some software you don't likely use that takes a while but one that
> is common is qtwebengine or something. If it's not that one, it's qtweb
> something. It takes about 4 hours, sometimes 5 or so.
>
> I think the software takes longer to compile so that we will build new
> rigs. ROFL
>
> Dale
>
> :-) :-)
>


--
Alan McKinnon
alan dot mckinnon at gmail dot com

On Mon, 11 Sep 2023 21:19:27 +0200, Alan McKinnon wrote:

> chromium has been building since 10:14, it's now 21:16 and still going
> so 9 hours at least on this machine to build a browser - almost as bad
> as openoffice at it's worst (regularly took 12 hours). Nodejs also took
> a while, but I didn't record time.

Chromium is definitely the worst, and strangely variable. The last few
compiles have taken between 6 and 14 hours. Since it takes longer than
everything else to build, it is usually compiling on its own, so parallel
emerges aren't a factor.

Qtwebengine is also bad, not surprising as it is a cut down Chromium.
Emerging world with --exclude then timing build to coincide with sleep
helps, although I haven't quite reached the age where I need 14 hours of
sleep a day.


--
Neil Bothwick

If it isn't broken, I can fix it.

On Mon, Sep 11, 2023 at 10:05?PM Neil Bothwick <neil@digimed.co.uk> wrote:

> On Mon, 11 Sep 2023 21:19:27 +0200, Alan McKinnon wrote:
>
> > chromium has been building since 10:14, it's now 21:16 and still going
> > so 9 hours at least on this machine to build a browser - almost as bad
> > as openoffice at it's worst (regularly took 12 hours). Nodejs also took
> > a while, but I didn't record time.
>
> Chromium is definitely the worst, and strangely variable. The last few
> compiles have taken between 6 and 14 hours. Since it takes longer than
> everything else to build, it is usually compiling on its own, so parallel
> emerges aren't a factor.
>
> Qtwebengine is also bad, not surprising as it is a cut down Chromium.
> Emerging world with --exclude then timing build to coincide with sleep
> helps, although I haven't quite reached the age where I need 14 hours of
> sleep a day.
>
>
> --
> Neil Bothwick
>
> If it isn't broken, I can fix it.
>

Yup, that jibes with what I see. Oh well, just means that the need for
overnight compiles did not go away haha

Thanks to every one else that replied too - everyone said much the same
thing so I figured one replay to rule them all was the best way


Alan
--
Alan McKinnon
alan dot mckinnon at gmail dot com

On Monday, 11 September 2023 21:21:47 BST Alan McKinnon wrote:
> On Mon, Sep 11, 2023 at 10:05?PM Neil Bothwick <neil@digimed.co.uk> wrote:
> > On Mon, 11 Sep 2023 21:19:27 +0200, Alan McKinnon wrote:
> > > chromium has been building since 10:14, it's now 21:16 and still going
> > > so 9 hours at least on this machine to build a browser - almost as bad
> > > as openoffice at it's worst (regularly took 12 hours). Nodejs also took
> > > a while, but I didn't record time.
> >
> > Chromium is definitely the worst, and strangely variable. The last few
> > compiles have taken between 6 and 14 hours. Since it takes longer than
> > everything else to build, it is usually compiling on its own, so parallel
> > emerges aren't a factor.
> >
> > Qtwebengine is also bad, not surprising as it is a cut down Chromium.
> > Emerging world with --exclude then timing build to coincide with sleep
> > helps, although I haven't quite reached the age where I need 14 hours of
> > sleep a day.
> >
> >
> > --
> > Neil Bothwick
> >
> > If it isn't broken, I can fix it.
>
> Yup, that jibes with what I see. Oh well, just means that the need for
> overnight compiles did not go away haha
>
> Thanks to every one else that replied too - everyone said much the same
> thing so I figured one replay to rule them all was the best way
>
>
> Alan

As the old saying goes, "there ain't no substitute to cubic inches". Moar
cores and moar RAM is almost always the solution, but with laptops and older
PCs in general overnight builds soon become inevitable. Selectively reducing
jobs and adding swap, or for packages like rust placing /var/tmp/portage on
the disk becomes necessary.

A solution I use for older/smaller laptops is to build binaries on a more
powerful PC and emerge these in turn on the weaker PCs.

There's also the option of using bin alternatives where available, e.g.
google-chrome, firefox-bin, libreoffice-bin.

Finally, there is a small scale project to provide systemd based binaries as
an alternative to building your own:

https://wiki.gentoo.org/wiki/Experimental_binary_package_host

On Mon, Sep 11, 2023 at 11:23?PM Michael <confabulate@kintzios.com> wrote:

> On Monday, 11 September 2023 21:21:47 BST Alan McKinnon wrote:
> > On Mon, Sep 11, 2023 at 10:05?PM Neil Bothwick <neil@digimed.co.uk>
> wrote:
> > > On Mon, 11 Sep 2023 21:19:27 +0200, Alan McKinnon wrote:
> > > > chromium has been building since 10:14, it's now 21:16 and still
> going
> > > > so 9 hours at least on this machine to build a browser - almost as
> bad
> > > > as openoffice at it's worst (regularly took 12 hours). Nodejs also
> took
> > > > a while, but I didn't record time.
> > >
> > > Chromium is definitely the worst, and strangely variable. The last few
> > > compiles have taken between 6 and 14 hours. Since it takes longer than
> > > everything else to build, it is usually compiling on its own, so
> parallel
> > > emerges aren't a factor.
> > >
> > > Qtwebengine is also bad, not surprising as it is a cut down Chromium.
> > > Emerging world with --exclude then timing build to coincide with sleep
> > > helps, although I haven't quite reached the age where I need 14 hours
> of
> > > sleep a day.
> > >
> > >
> > > --
> > > Neil Bothwick
> > >
> > > If it isn't broken, I can fix it.
> >
> > Yup, that jibes with what I see. Oh well, just means that the need for
> > overnight compiles did not go away haha
> >
> > Thanks to every one else that replied too - everyone said much the same
> > thing so I figured one replay to rule them all was the best way
> >
> >
> > Alan
>
> As the old saying goes, "there ain't no substitute to cubic inches". Moar
> cores and moar RAM is almost always the solution, but with laptops and
> older
> PCs in general overnight builds soon become inevitable. Selectively
> reducing
> jobs and adding swap, or for packages like rust placing /var/tmp/portage
> on
> the disk becomes necessary.
>
> A solution I use for older/smaller laptops is to build binaries on a more
> powerful PC and emerge these in turn on the weaker PCs.
>
> There's also the option of using bin alternatives where available, e.g.
> google-chrome, firefox-bin, libreoffice-bin.
>
> Finally, there is a small scale project to provide systemd based binaries
> as
> an alternative to building your own:
>
> https://wiki.gentoo.org/wiki/Experimental_binary_package_host
>

As it turns out this laptop is the most powerful machine I have available,
my large collection of previous work laptops are getting older and older.

Although, I *could* create a ginormous build host on one of the
virtualization clusters at work hahaha :-)

That link looks interesting, I'll check it out, thanks!


--
Alan McKinnon
alan dot mckinnon at gmail dot com

You may also want to take a look at "distcc", with which you can set up
compiler farms; this can be even combined with "ccache":

    https://wiki.gentoo.org/wiki/Distcc#With_ccache

-Ramon

On 11/09/2023 23:46, Alan McKinnon wrote:
>
>
> On Mon, Sep 11, 2023 at 11:23?PM Michael <confabulate@kintzios.com> wrote:
>
> On Monday, 11 September 2023 21:21:47 BST Alan McKinnon wrote:
> > On Mon, Sep 11, 2023 at 10:05?PM Neil Bothwick
> <neil@digimed.co.uk> wrote:
> > > On Mon, 11 Sep 2023 21:19:27 +0200, Alan McKinnon wrote:
> > > > chromium has been building since 10:14, it's now 21:16 and
> still going
> > > > so 9 hours at least on this machine to build a browser -
> almost as bad
> > > > as openoffice at it's worst (regularly took 12 hours).
> Nodejs also took
> > > > a while, but I didn't record time.
> > >
> > > Chromium is definitely the worst, and strangely variable. The
> last few
> > > compiles have taken between 6 and 14 hours. Since it takes
> longer than
> > > everything else to build, it is usually compiling on its own,
> so parallel
> > > emerges aren't a factor.
> > >
> > > Qtwebengine is also bad, not surprising as it is a cut down
> Chromium.
> > > Emerging world with --exclude then timing build to coincide
> with sleep
> > > helps, although I haven't quite reached the age where I need
> 14 hours of
> > > sleep a day.
> > >
> > >
> > > --
> > > Neil Bothwick
> > >
> > > If it isn't broken, I can fix it.
> >
> > Yup, that jibes with what I see. Oh well, just means that the
> need for
> > overnight compiles did not go away haha
> >
> > Thanks to every one else that replied too - everyone said much
> the same
> > thing so I figured one replay to rule them all was the best way
> >
> >
> > Alan
>
> As the old saying goes, "there ain't no substitute to cubic
> inches".  Moar
> cores and moar RAM is almost always the solution, but with laptops
> and older
> PCs in general overnight builds soon become inevitable.
> Selectively reducing
> jobs and adding swap, or for packages like rust placing
> /var/tmp/portage on
> the disk becomes necessary.
>
> A solution I use for older/smaller laptops is to build binaries on
> a more
> powerful PC and emerge these in turn on the weaker PCs.
>
> There's also the option of using bin alternatives where available,
> e.g.
> google-chrome, firefox-bin, libreoffice-bin.
>
> Finally, there is a small scale project to provide systemd based
> binaries as
> an alternative to building your own:
>
> https://wiki.gentoo.org/wiki/Experimental_binary_package_host
>
>
> As it turns out this laptop is the most powerful machine I have
> available, my large collection of previous work laptops are getting
> older and older.
>
> Although, I *could* create a ginormous build host on one of the
> virtualization clusters at work hahaha :-)
>
> That link looks interesting, I'll check it out, thanks!
>
>
> --
> Alan McKinnon
> alan dot mckinnon at gmail dot com

--
GPG public key: 5983 98DA 5F4D A464 38FD CF87 155B E264 13E6 99BF

On Tue, Sep 12, 2023 at 1:05?AM Ramon Fischer <Ramon_Fischer@hotmail.de>
wrote:

> You may also want to take a look at "distcc", with which you can set up
> compiler farms; this can be even combined with "ccache":
>
> https://wiki.gentoo.org/wiki/Distcc#With_ccache
>
> -Ramon
>


Hi Ramon,

distcc is way more than I need. I'm not complaining about long compile
times and wanting a solution, I was more curious about which packages these
days take long compared to when I was last here 5/6 years ago

Alan




>
> On 11/09/2023 23:46, Alan McKinnon wrote:
> >
> >
> > On Mon, Sep 11, 2023 at 11:23?PM Michael <confabulate@kintzios.com>
> wrote:
> >
> > On Monday, 11 September 2023 21:21:47 BST Alan McKinnon wrote:
> > > On Mon, Sep 11, 2023 at 10:05?PM Neil Bothwick
> > <neil@digimed.co.uk> wrote:
> > > > On Mon, 11 Sep 2023 21:19:27 +0200, Alan McKinnon wrote:
> > > > > chromium has been building since 10:14, it's now 21:16 and
> > still going
> > > > > so 9 hours at least on this machine to build a browser -
> > almost as bad
> > > > > as openoffice at it's worst (regularly took 12 hours).
> > Nodejs also took
> > > > > a while, but I didn't record time.
> > > >
> > > > Chromium is definitely the worst, and strangely variable. The
> > last few
> > > > compiles have taken between 6 and 14 hours. Since it takes
> > longer than
> > > > everything else to build, it is usually compiling on its own,
> > so parallel
> > > > emerges aren't a factor.
> > > >
> > > > Qtwebengine is also bad, not surprising as it is a cut down
> > Chromium.
> > > > Emerging world with --exclude then timing build to coincide
> > with sleep
> > > > helps, although I haven't quite reached the age where I need
> > 14 hours of
> > > > sleep a day.
> > > >
> > > >
> > > > --
> > > > Neil Bothwick
> > > >
> > > > If it isn't broken, I can fix it.
> > >
> > > Yup, that jibes with what I see. Oh well, just means that the
> > need for
> > > overnight compiles did not go away haha
> > >
> > > Thanks to every one else that replied too - everyone said much
> > the same
> > > thing so I figured one replay to rule them all was the best way
> > >
> > >
> > > Alan
> >
> > As the old saying goes, "there ain't no substitute to cubic
> > inches". Moar
> > cores and moar RAM is almost always the solution, but with laptops
> > and older
> > PCs in general overnight builds soon become inevitable.
> > Selectively reducing
> > jobs and adding swap, or for packages like rust placing
> > /var/tmp/portage on
> > the disk becomes necessary.
> >
> > A solution I use for older/smaller laptops is to build binaries on
> > a more
> > powerful PC and emerge these in turn on the weaker PCs.
> >
> > There's also the option of using bin alternatives where available,
> > e.g.
> > google-chrome, firefox-bin, libreoffice-bin.
> >
> > Finally, there is a small scale project to provide systemd based
> > binaries as
> > an alternative to building your own:
> >
> > https://wiki.gentoo.org/wiki/Experimental_binary_package_host
> >
> >
> > As it turns out this laptop is the most powerful machine I have
> > available, my large collection of previous work laptops are getting
> > older and older.
> >
> > Although, I *could* create a ginormous build host on one of the
> > virtualization clusters at work hahaha :-)
> >
> > That link looks interesting, I'll check it out, thanks!
> >
> >
> > --
> > Alan McKinnon
> > alan dot mckinnon at gmail dot com
>
> --
> GPG public key: 5983 98DA 5F4D A464 38FD CF87 155B E264 13E6 99BF
>
>

--
Alan McKinnon
alan dot mckinnon at gmail dot com

On 11/09/2023 22:19, Alan McKinnon wrote:
> chromium has been building since 10:14, it's now 21:16 and still going
> so 9 hours at least on this machine to build a browser - almost as bad
> as openoffice at it's worst (regularly took 12 hours). Nodejs also took
> a while, but I didn't record time.

What's your CPU and how much RAM? Even on my older system I had (an
4-core i5 2500K) libreoffice took like 2 hours or so to build.


> What other packages have huge build times?

IIRC, dev-qt/qtwebengine is one of the heaviest when it comes to build
times.

Anyway, a nice way to cut down on build times is to build on tmpfs. To
do that however with heavy packages like that, I had to upgrade to 32GB
RAM. There was a large price drop in the memory market a couple months
ago, so I snatched a 32GB DDR4 3600 kit (2x16GB) for like 80€. So now
with plenty of RAM, I configured a 14GB tmpfs in /var/tmp/portage. I
never hit swap when emerging.

On 11/09/2023 23:21, Alan McKinnon wrote:
> Yup, that jibes with what I see. Oh well, just means that the need for
> overnight compiles did not go away haha

Ever since I added the following to my make.conf:

PORTAGE_NICENESS=19
PORTAGE_IONICE_COMMAND="sh -c \"schedtool -D \${PID} && ionice -c 3 -p
\${PID}\""

I never needed overnight compiles again. Make sure sys-process/schedtool
is installed. As long as you have plenty of RAM so the system doesn't
swap, you can use the system normally even while building monster
packages. I can even play video games without issue while portage is
emerging now.

On Monday, 11 September 2023 22:22:28 BST Michael wrote:

> There's also the option of using bin alternatives where available, e.g.
> google-chrome, firefox-bin, libreoffice-bin.

...and rust-bin, which is now the default in at least some desktop profiles.

--
Regards,
Peter.

Le mar. 12 sept. 2023, 12:14, Peter Humphrey <peter@prh.myzen.co.uk> a
écrit :

> On Monday, 11 September 2023 22:22:28 BST Michael wrote:
>
> > There's also the option of using bin alternatives where available, e.g.
> > google-chrome, firefox-bin, libreoffice-bin.
>
> ...and rust-bin, which is now the default in at least some desktop
> profiles.
>
> --
> Regards,
> Peter.
>
>
>
>
> Hello,


I could get rid of webkit-gtk with pleasure and i use rust-bin.
I tried libreoffice-bin but encountered some issues with pdf import, so i
went back to libreoffice.

Cheers,

Jacques



>
>

On Tue, Sep 12, 2023 at 11:19?AM Nikos Chantziaras <realnc@gmail.com> wrote:

> On 11/09/2023 22:19, Alan McKinnon wrote:
> > chromium has been building since 10:14, it's now 21:16 and still going
> > so 9 hours at least on this machine to build a browser - almost as bad
> > as openoffice at it's worst (regularly took 12 hours). Nodejs also took
> > a while, but I didn't record time.
>
> What's your CPU and how much RAM? Even on my older system I had (an
> 4-core i5 2500K) libreoffice took like 2 hours or so to build.
>
>
> > What other packages have huge build times?
>
> IIRC, dev-qt/qtwebengine is one of the heaviest when it comes to build
> times.
>
> Anyway, a nice way to cut down on build times is to build on tmpfs. To
> do that however with heavy packages like that, I had to upgrade to 32GB
> RAM. There was a large price drop in the memory market a couple months
> ago, so I snatched a 32GB DDR4 3600 kit (2x16GB) for like 80€. So now
> with plenty of RAM, I configured a 14GB tmpfs in /var/tmp/portage. I
> never hit swap when emerging.
>

That's not an option for me, this is a corporate laptop with 16G RAM and a
case I may not open :-)
I'm not interested in a remote build host or distcc either

But anyways, this is not really about how to deal with long compiles, I was
asking what current packages take a long time after a 5 year absence.

The answer is what it was always - browsers and libreoffice. I do recall
icu being a bit of a beast back then


Alan




--
Alan McKinnon
alan dot mckinnon at gmail dot com

On 2023-09-12, Alan McKinnon wrote:
[...]
> But anyways, this is not really about how to deal with long compiles, I was
> asking what current packages take a long time after a 5 year absence.
>
> The answer is what it was always - browsers and libreoffice. I do recall
> icu being a bit of a beast back then

I remember insn-attrtab.c making the GCC compilation swap a lot :-)

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=29442

--
Nuno Silva

On 11/09/2023 20:46, Alan McKinnon wrote:
> qtwebengine! yes that one took forever also. It also said my 16G of RAM
> was smaller than the 16G it needed. Weird.
>
> Anyways I enabled a swapfile and left it to run overnight

16GB physical ram <> 16GB usable ram for the compile ...

I concur with others that tmpfs is the way to go - I don't think my
system is set up that way just now, but I always have googols of swap
(twice max physical ram per disk) so I just declare a huge ramdisk for
compiling on.

My current system has four ram slots, two maxed out with 16GB chips
each, so that makes 128GB swap partitions per disk (4 of them) equals
512GB swap ...

(Yes the people at openSUSE said I was being stupid with that much swap)

But declare a 128GB ramdisk, and it'll spill over as required, but
anything that fits in ram will compile very quick.

And yes, I also used to have my systems configured so they had one
shared portage area, matching make.conf, and were set up to "use binary
if it exists, else build one". I had the opposite problem though, my
nice fast system had a habit of crashing, so I used the old slow one to
build most things, because it was more reliable. Hey ho.

There's all sorts of tricks, some work for some people, others work for
others.

Cheers,
Wol

On Tue, 12 Sep 2023 21:01:48 +0200, Alan McKinnon wrote:

> But anyways, this is not really about how to deal with long compiles, I
> was asking what current packages take a long time after a 5 year
> absence.
>
> The answer is what it was always - browsers and libreoffice. I do recall
> icu being a bit of a beast back then

LibreOffice doesn't seem too bad these days. icu and boost are a pain
because of the number of other packages they rebuild.


--
Neil Bothwick

Light travels faster than sound. This is why some people appear bright
until you hear them speak.

On Tuesday, 12 September 2023 22:08:49 BST Wol wrote:

> There's all sorts of tricks, some work for some people, others work for
> others.

Quite so. Here I have two swap partitions: 8GB priority 20 on NVME and 50GB
priority 10 on SSD. I've never noticed either of them being used, so I suppose
I could dispense with the smaller one. When I bought the machine I wanted it
to be as powerful as I could reasonably justify (to run BOINC projects, making
my own small contribution to the state of knowlege in astrophysics), so it has
64 GB RAM in its four slots.

Tmpfs earns its keep here. I don't set limits on it, preferring to let the
kernel manage it itself. One tmpfs is on /tmp, the other on /var/tmp/portage.
With all that RAM to play in, swap is rarely used, if ever.

A thought on compiling, which I hope some devs will read: I was tempted to
push the system hard at first, with load average and jobs as high as I thought
I could set them. I've come to believe, though, that job control by portage
and /usr/bin/make is weak at very high loads, because I would usually find that
a few packages had failed to compile; also that some complex programs were
sometimes unstable. Therefore I've had to throttle the system to be sure(r) of
correctness. Seems a waste. Thus:

$ grep '\-j' /etc/portage/make.conf
EMERGE_DEFAULT_OPTS="--jobs=4 --load-average=32 [...]"
MAKEOPTS="-j14"

That 14 will revert to its previous 12 if I find things going bump in the night
again, or perhaps go up a bit more if not.

--
Regards,
Peter.

On 13/09/2023 12:28, Peter Humphrey wrote:
> A thought on compiling, which I hope some devs will read: I was tempted to
> push the system hard at first, with load average and jobs as high as I thought
> I could set them. I've come to believe, though, that job control by portage
> and /usr/bin/make is weak at very high loads, because I would usually find that
> a few packages had failed to compile; also that some complex programs were
> sometimes unstable. Therefore I've had to throttle the system to be sure(r) of
> correctness. Seems a waste. Thus:

Bear in mind a lot of systems are thermally limited and can't run at
full pelt anyway ...

You might find it's actually better (and more efficient) to run at lower
loading. Certainly following the kernel lists you get the impression
that the CPU regularly goes into thermal throttling under heavy load,
and also that using a couple of cores lightly is more efficient than
using one core heavily.

It's so difficult to know what's best ... (because too many people make
decisions based on their interests, and then when you come along their
decisions may conflict with each other and certainly conflict with you ...)

Cheers,
Wol

Am Wed, Sep 13, 2023 at 12:50:20PM +0100 schrieb Wols Lists:

> Bear in mind a lot of systems are thermally limited and can't run at full
> pelt anyway ...

Usually those are space-constrained systems like mini PCs or laptops.
Typical Desktops shouldn’t be limited; even the stock CPU coolers should be
capable of dissipating all heat, as long as the case has enough air flow.

> You might find it's actually better (and more efficient) to run at lower
> loading. Certainly following the kernel lists you get the impression that
> the CPU regularly goes into thermal throttling under heavy load, and also
> that using a couple of cores lightly is more efficient than using one core
> heavily.

At least very current CPUs tend to go into so high clock speeds that they
become quite inefficient. If you set a 105 W Ryzen 7700X to 65 W eco mode in
the BIOS (which means that the actual maximum intake goes down from 144 W to
84 W), you reduce consuption by a third, but only lose ~15 % in performance.

At very low figures (15 W), Ryzen 5000 and 7000 CPUs are almost as efficient
as Apple M1.

--
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Einstein is dead. Newton is dead. I’m feeling sick, too.

On Wednesday, 13 September 2023 12:50:20 BST Wols Lists wrote:
> On 13/09/2023 12:28, Peter Humphrey wrote:
> > A thought on compiling, which I hope some devs will read: I was tempted to
> > push the system hard at first, with load average and jobs as high as I
> > thought I could set them. I've come to believe, though, that job control
> > by portage and /usr/bin/make is weak at very high loads, because I would
> > usually find that a few packages had failed to compile; also that some
> > complex programs were sometimes unstable. Therefore I've had to throttle
> > the system to be sure(r) of correctness. Seems a waste.

> Bear in mind a lot of systems are thermally limited and can't run at
> full pelt anyway ...

No doubt, but apparently not this box: I run it 24x7 with all 24 CPU threads
fully loaded with floating-point calculations, which make a good deal more heat
than 'mere' compiling with (I assume) integer arithmetic. :)

> You might find it's actually better (and more efficient) to run at lower
> loading. Certainly following the kernel lists you get the impression
> that the CPU regularly goes into thermal throttling under heavy load,
> and also that using a couple of cores lightly is more efficient than
> using one core heavily.

See above; besides, I have to limit the load anyway when compiling, for the
reasons I gave last time.

> It's so difficult to know what's best ... (because too many people make
> decisions based on their interests, and then when you come along their
> decisions may conflict with each other and certainly conflict with you ...)

I agree with you there Wol, even without the parenthesis. :)

--
Regards,
Peter.

On Wednesday, 13 September 2023 13:41:00 BST Peter Humphrey wrote:
> On Wednesday, 13 September 2023 12:50:20 BST Wols Lists wrote:
> > On 13/09/2023 12:28, Peter Humphrey wrote:
> > > A thought on compiling, which I hope some devs will read: I was tempted
> > > to
> > > push the system hard at first, with load average and jobs as high as I
> > > thought I could set them. I've come to believe, though, that job control
> > > by portage and /usr/bin/make is weak at very high loads, because I would
> > > usually find that a few packages had failed to compile; also that some
> > > complex programs were sometimes unstable. Therefore I've had to throttle
> > > the system to be sure(r) of correctness. Seems a waste.
> >
> > Bear in mind a lot of systems are thermally limited and can't run at
> > full pelt anyway ...
>
> No doubt, but apparently not this box: I run it 24x7 with all 24 CPU threads
> fully loaded with floating-point calculations, which make a good deal more
> heat than 'mere' compiling with (I assume) integer arithmetic. :)

I recall this being discussed in a previous thread, but if your CPU has 24
threads and you've set:

EMERGE_DEFAULT_OPTS="--jobs=4 --load-average=32
MAKEOPTS="-j14"

You will be asking emerge to run up to 4 x 14 = 56 threads, which could
potentially eat up to 2G of RAM each, i.e. 112G of RAM. This will exhaust
your 64G of RAM, not taking into account whatever else the OS will be trying
to run at the time. The --load-average=32 is normally expected to be a
floating point number indicating a percentage of load x the number of cores;
e.g. for 12 cores you could set it at 12 x 0.9 = 10.8 to limit the load to 90%
so as maintain some system responsiveness.

Of course, not all emerges use make and you may never or rarely emerge 4 x
monster packages in parallel to need 2G of RAM for each thread at the same
time.

If only we had at our disposal some AI algorithm to calculate dynamically each
time we run emerge the optimal combo of parallel emerge jobs and number of
make tasks, so as to achieve the highest total time saving Vs energy spent!
Or just the highest total time saving. ;-)

I haven't performed any meaningful comparisons to determine where the greatest
gains are to be achieved. Parallel emerges of many small packages, or a large
number of make jobs for big packages. The combination would change each time
according to the individual packages waiting for an update. In my use case,
instinctively feels more beneficial reducing the time I have to wait for huge
packages like qtwebengine to finish, than accelerating the updates of half a
dozen smaller packages. Therefore, as a rule I leave EMERGE_DEFAULT_OPTS
unset. I set MAKEOPTS jobs to the number of CPU threads +1 and the load
average at 95%, so I can continue using the PC without any noticeable latency.
On PCs where RAM is constrained I reduce the MAKEOPTS in /etc/portage/
package.env for any large packages which are bound to start swapping and
thrashing the disk.