Mailing List Archive

On Tue, 2021-04-27 at 20:53 +0000, Alan Mackenzie wrote:
> I know I could plump for the -bin package.  Maybe I should.

I did, recently. 

Anecdotes/opinions/rant follows. Feel free to ignore, I just wanted to
vent because the issue is fresh in my mind.

...

Some things just aren't worth building from source. Rust is so heavy,
convoluted, ill-supported and slow that I just can't be bothered to
spend hours of compile time on getting a slightly-less-awful version of
it for my system(s). And given how a desktop environment now requires
a Javascript engine to parse configurations (that's a separate
complaint), and spidermonkey requires rust to build, there's no
avoiding it for my needs, so I just cut bait.

The same can be said for GHC (the Glasgow Haskell Compiler). I can't
do my day-to-day without shellcheck, but spending 8 hours to build a
compiler for shellcheck (and sometimes pandoc) frankly sucks. This
isn't gentoo's fault, or anyone's fault really. Languages are complex
and the code to bootstrap compile them isn't easy. Its above my skill
level, that's for sure.

8+ years of gentoo has taught me many lessons, chief among them is to
pick my battles. If there is some must-have or must-avoid USE flag, or
a killer feature that's missing from the bin build, then I'll swallow
the pill and take the time to compile it. If not, I've got better
things to do with my electricity.

On Tuesday, 27 April 2021 22:17:20 BST Matt Connell (Gmail) wrote:
> On Tue, 2021-04-27 at 20:53 +0000, Alan Mackenzie wrote:
> > I know I could plump for the -bin package. Maybe I should.
>
> I did, recently.
>
> Anecdotes/opinions/rant follows. Feel free to ignore, I just wanted to
> vent because the issue is fresh in my mind.
>
> ...
>
> Some things just aren't worth building from source. Rust is so heavy,
> convoluted, ill-supported and slow that I just can't be bothered to
> spend hours of compile time on getting a slightly-less-awful version of
> it for my system(s). And given how a desktop environment now requires
> a Javascript engine to parse configurations (that's a separate
> complaint), and spidermonkey requires rust to build, there's no
> avoiding it for my needs, so I just cut bait.
>
> The same can be said for GHC (the Glasgow Haskell Compiler). I can't
> do my day-to-day without shellcheck, but spending 8 hours to build a
> compiler for shellcheck (and sometimes pandoc) frankly sucks. This
> isn't gentoo's fault, or anyone's fault really. Languages are complex
> and the code to bootstrap compile them isn't easy. Its above my skill
> level, that's for sure.
>
> 8+ years of gentoo has taught me many lessons, chief among them is to
> pick my battles. If there is some must-have or must-avoid USE flag, or
> a killer feature that's missing from the bin build, then I'll swallow
> the pill and take the time to compile it. If not, I've got better
> things to do with my electricity.

Heh! I had a go more than once increasing RAM allocated to /var/tmp/portage
also running out of memory. I think this is the second time it happened to me
in the last 6 months or so. There's a bug on this:

https://bugs.gentoo.org/show_bug.cgi?id=757276

There's three options, I can think of:

1. Use dev-lang/rust-bin, as Matt suggested above.

2. Buy more RAM, or use a surrogate PC with more RAM to cross-compile it.

3. Use a partition with enough space on it to bind mount /var/tmp/portage, for
this package only.

I use the 3rd option, but I'm wondering if option 1 may be the smartest move
for my needs.

On 2021-04-27 20:53+0000 Alan Mackenzie <acm@muc.de> wrote:

> Hello, Gentoo.
>
> I'm having problems building rust.
>
> I build everything in a ram disk, and last night my 13 Gb ram disk
> proved too small to build rust in. So I increased its size to 14 Gb,
> and tried again this evening. Same result. The pre-check on the disk
> size gave an OK both times, and both runs lasted about 45 minutes
> before running out of space.
>
> Help! What am I supposed to do? I've got 16 Gb RAM (I'm _not_ going
> to use the word "only" here), and wondering just how big a chunk a
> ram disk can take out of that before the machine siezes up
> altogether. But if I increase the ram disk to 15 Gb, even assuming
> it'd work, it's only going to be a small number of releases before
> the clever people at rust increase their bloat even more.

You could use zram[1], it's like a ramdisk but compressed. Source code
compresses very good, binary files compress to ~50% in my tests.

> I know I could plump for the -bin package. Maybe I should. But
> before I do that, is it possible to redirect this one ebuild away from
> /var/tmp/portage (my ram disk) without disturbing the other builds?
> If so, how would I do this (or where should I look for documentation)?

You could create /var/tmp/portage/dev-lang/rust after creating the
ramdisk and then bind-mount another directory to it. But then you'd
have to tell portage to not delete it with FEATURES="keepwork"[2]
(see`man 5 make.conf`) … I wouldn't recommend it.

[1] <https://wiki.gentoo.org/wiki/Zram>
[2] <https://wiki.gentoo.org/wiki//etc/portage/package.env>

kind regards, tastytea

--
Get my PGP key with `gpg --locate-keys tastytea@tastytea.de` or at
<https://tastytea.de/tastytea.asc>.

On Tue, Apr 27, 2021 at 08:53:11PM +0000, Alan Mackenzie wrote:
>But if I increase the ram disk to 15 Gb, even assuming it'd work

Probably wouldn't, had same issue recently, 16GB was not enough,
increased to 24GB just in case and it went through.

>is it possible to redirect this one ebuild away from
>/var/tmp/portage (my ram disk) without disturbing the other builds? If
>so, how would I do this (or where should I look for documentation)?

What you want is to look up package.env page on wiki, it allows you to
customize enviromental variables on per-package basis, PORTAGE_TMPDIR
being one of them. That's what I used to do for bigger packages when I
had less RAM.

Sincerely,
Teru

On Tue, 27 Apr 2021 20:53:11 +0000, Alan Mackenzie wrote:

> Help! What am I supposed to do? I've got 16 Gb RAM (I'm _not_ going to
> use the word "only" here), and wondering just how big a chunk a ram disk
> can take out of that before the machine siezes up altogether. But if I
> increase the ram disk to 15 Gb, even assuming it'd work, it's only going
> to be a small number of releases before the clever people at rust
> increase their bloat even more.

Setting the RAM disk so high will mean you don't have enough memory for
the compilation, which will start swapping and everything will grind to a
near halt.

> I know I could plump for the -bin package. Maybe I should. But before
> I do that, is it possible to redirect this one ebuild away from
> /var/tmp/portage (my ram disk) without disturbing the other builds? If
> so, how would I do this (or where should I look for documentation)?

Lookup package.env on the wiki. I use rust-bin now, so this isn't an
issue for me, but my laptop has only 8GB and this is how I have it set
for chromium:

% cat /etc/portage/env/disk-tmpdir.conf
PORTAGE_TMPDIR="/mnt/scratch"

% cat /etc/portage/package.env/chromium
www-client/chromium disk-tmpdir.conf

I do similar for libreoffice and a couple of other heavyweights.

Another option, to be used instead of or as well as this would be distcc.
Once again, you can use package.env to apply this to selected packages:

% cat /etc/portage/env/distcc.conf
FEATURES="distcc buildpkg distlocks"
MAKEOPTS="-j36 -l4"
CFLAGS="-march=broadwell -O2 -pipe"
CXXFLAGS="${CFLAGS}"


--
Neil Bothwick

I'm firm. You're obstinate. He's a pigheaded fool.

On Tuesday, April 27, 2021 10:53:11 PM CEST Alan Mackenzie wrote:
> Hello, Gentoo.
>
> I'm having problems building rust.
>
> I build everything in a ram disk, and last night my 13 Gb ram disk
> proved too small to build rust in. So I increased its size to 14 Gb,
> and tried again this evening. Same result. The pre-check on the disk
> size gave an OK both times, and both runs lasted about 45 minutes before
> running out of space.
>
> Help! What am I supposed to do? I've got 16 Gb RAM (I'm _not_ going to
> use the word "only" here), and wondering just how big a chunk a ram disk
> can take out of that before the machine siezes up altogether. But if I
> increase the ram disk to 15 Gb, even assuming it'd work, it's only going
> to be a small number of releases before the clever people at rust
> increase their bloat even more.
>
> I know I could plump for the -bin package. Maybe I should. But before
> I do that, is it possible to redirect this one ebuild away from
> /var/tmp/portage (my ram disk) without disturbing the other builds? If
> so, how would I do this (or where should I look for documentation)?
>
> Thanks for the help!

I managed to get it to compile with a 16GB ramdisk using "system-llvm":


$ grep -r rust /etc/portage
/etc/portage/profile/package.use.mask:dev-lang/rust -system-llvm
/etc/portage/package.use/rust:dev-lang/rust system-llvm parallel-compiler

It claims it wants:
Checking for at least 13312 MiB

From the build-log:

======
* Final size of build directory: 12344956 KiB ( 11.7 GiB)
* Final size of installed tree: 366492 KiB (357.9 MiB)

* The ebuild is installing to one or more unexpected paths:
*
* /usr/share/doc/rust-1.51.0
*
* Please fix the ebuild to use correct FHS/Gentoo policy paths.
=======

Hope this helps,

Joost

On 28/4/21 3:04 pm, J. Roeleveld wrote:
> On Tuesday, April 27, 2021 10:53:11 PM CEST Alan Mackenzie wrote:
>> Hello, Gentoo.
>>
>> I'm having problems building rust.
[snip]
...
...
[snip]
>
> Hope this helps,
>
> Joost
>
>
>

I fall into the same camp as Matt. My machine uses Rust only for the
Mozilla family of stuff so the "-bin" is fine for me. A 30 sec. download
Vs many hours compiling feels about right to me. Obviously, if you are
actually doing development using Rust you may want to tweak things so a
compile might be the way to go there.

Andrew

On 27/04/21 23:00, Michael wrote:
> There's three options, I can think of:
>
> 1. Use dev-lang/rust-bin, as Matt suggested above.
>
> 2. Buy more RAM, or use a surrogate PC with more RAM to cross-compile it.
>
> 3. Use a partition with enough space on it to bind mount /var/tmp/portage, for
> this package only.
>
> I use the 3rd option, but I'm wondering if option 1 may be the smartest move
> for my needs.

4. Add gazillibytes of swap. With a maximum of 16GB on my mobo, my two
disks each have a 32GB swap partition so that's 80GB of "ram" available
to my system. My /var/tmp/portage tmpfs is 30GB.

So obviously I use option 4 :-)

Cheers,
Wol

On Wednesday, 28 April 2021 09:58:57 BST Wols Lists wrote:
> On 27/04/21 23:00, Michael wrote:
> > There's three options, I can think of:
> >
> > 1. Use dev-lang/rust-bin, as Matt suggested above.
> >
> > 2. Buy more RAM, or use a surrogate PC with more RAM to cross-compile it.
> >
> > 3. Use a partition with enough space on it to bind mount /var/tmp/portage,
> > for this package only.
> >
> > I use the 3rd option, but I'm wondering if option 1 may be the smartest
> > move for my needs.
>
> 4. Add gazillibytes of swap. With a maximum of 16GB on my mobo, my two
> disks each have a 32GB swap partition so that's 80GB of "ram" available
> to my system. My /var/tmp/portage tmpfs is 30GB.
>
> So obviously I use option 4 :-)
>
> Cheers,
> Wol

The "add more swap" solution remains feasible, but only for PCs which already
have adequate RAM. By 'adequate RAM' I mean enough RAM to run the OS, plus at
least a single compilation thread. With, say, <4G RAM, even a single
compilation thread of a monster package will end up thrashing your disk
continuously. The solutions are to set MAKEOPTS="-j1", plus deploy zram for /
var/tmp/portage, plus add adequate swap and if your swap is on a spinning disk
also use a better scheduler:

echo bfq > /sys/block/sda/queue/scheduler

Nevertheless, if a bin package exists and does what you need it to do, the
above gymnastics are probably akin to ricing with substandard hardware. :-)

Hello, Neil.

On Wed, Apr 28, 2021 at 00:09:12 +0100, Neil Bothwick wrote:
> On Tue, 27 Apr 2021 20:53:11 +0000, Alan Mackenzie wrote:

> > Help! What am I supposed to do? I've got 16 Gb RAM (I'm _not_ going to
> > use the word "only" here), and wondering just how big a chunk a ram disk
> > can take out of that before the machine siezes up altogether. But if I
> > increase the ram disk to 15 Gb, even assuming it'd work, it's only going
> > to be a small number of releases before the clever people at rust
> > increase their bloat even more.

> Setting the RAM disk so high will mean you don't have enough memory for
> the compilation, which will start swapping and everything will grind to a
> near halt.

I think that was happening when I tried to use a 14 GB ram disk.

> > I know I could plump for the -bin package. Maybe I should. But before
> > I do that, is it possible to redirect this one ebuild away from
> > /var/tmp/portage (my ram disk) without disturbing the other builds? If
> > so, how would I do this (or where should I look for documentation)?

> Lookup package.env on the wiki.

Many thanks for the tip. I did this, and set up my PORTAGE_TMPDIR to
point into /home/portage. Not the ideal place, but the only place with
enough room, without messing around with mdadm to create more.

> I use rust-bin now, so this isn't an issue for me, but my laptop has
> only 8GB and this is how I have it set for chromium:

> % cat /etc/portage/env/disk-tmpdir.conf
> PORTAGE_TMPDIR="/mnt/scratch"

> % cat /etc/portage/package.env/chromium
> www-client/chromium disk-tmpdir.conf

> I do similar for libreoffice and a couple of other heavyweights.

With a like setup, building rust took ~45 minutes. This is about the
same as it took on the ram disk before giving up. So, I'm not losing a
lot, if anything, by using /home/portage (On mirrored nvme SSDs).

> Another option, to be used instead of or as well as this would be distcc.
> Once again, you can use package.env to apply this to selected packages:

That's for building on a different (more powerful) machine, isn't it? I
don't actually have a more powerful machine. ;-)

> % cat /etc/portage/env/distcc.conf
> FEATURES="distcc buildpkg distlocks"
> MAKEOPTS="-j36 -l4"
> CFLAGS="-march=broadwell -O2 -pipe"
> CXXFLAGS="${CFLAGS}"


> --
> Neil Bothwick

> I'm firm. You're obstinate. He's a pigheaded fool.

At times, I'm all three!

--
Alan Mackenzie (Nuremberg, Germany).

On Wed, 28 Apr 2021 11:12:33 +0000, Alan Mackenzie wrote:

> > I use rust-bin now, so this isn't an issue for me, but my laptop has
> > only 8GB and this is how I have it set for chromium:
>
> > % cat /etc/portage/env/disk-tmpdir.conf
> > PORTAGE_TMPDIR="/mnt/scratch"
>
> > % cat /etc/portage/package.env/chromium
> > www-client/chromium disk-tmpdir.conf
>
> > I do similar for libreoffice and a couple of other heavyweights.
>
> With a like setup, building rust took ~45 minutes. This is about the
> same as it took on the ram disk before giving up. So, I'm not losing a
> lot, if anything, by using /home/portage (On mirrored nvme SSDs).

There have been several tests posted over the years that show that using
a tmpfs for TMPDIR doesn't give as much of a gain as you would expect.
With NVMe drives, the difference is likely to be even smaller.

>
> > Another option, to be used instead of or as well as this would be
> > distcc. Once again, you can use package.env to apply this to selected
> > packages:
>
> That's for building on a different (more powerful) machine, isn't it? I
> don't actually have a more powerful machine. ;-)

A different machine, not necessarily more powerful. The point is that you
are splitting the workload between the different computers. My desktop is
currently emerging chromium, and is sending some of the work to my MythTV
backend, which is actually the old desktop machine it replaced.


--
Neil Bothwick

Time for a diet! -- [NO FLABBIER].

On 28/04/2021 13:03, Neil Bothwick wrote:
> There have been several tests posted over the years that show that using
> a tmpfs for TMPDIR doesn't give as much of a gain as you would expect.
> With NVMe drives, the difference is likely to be even smaller.

But that's not the only reason for a tmpfs. I guess the reason tmpfs
doesn't necessarily give much gain is because linux caches everything so
it's quite possible stuff never gets written to disk.

And I know gentoo does clean up behind itself, but I use tmpfs so the
system cleans up behind it too, and it's got as much room as it needs.
When you see things like firefox allegedly needing 10s of gigs, I
thought "just give it oodles and let it take what it wants".

Cheers,
Wol

On Wed, 28 Apr 2021 23:05:35 +0100, antlists wrote:

> > There have been several tests posted over the years that show that
> > using a tmpfs for TMPDIR doesn't give as much of a gain as you would
> > expect. With NVMe drives, the difference is likely to be even
> > smaller.
>
> But that's not the only reason for a tmpfs. I guess the reason tmpfs
> doesn't necessarily give much gain is because linux caches everything
> so it's quite possible stuff never gets written to disk.

Agreed, but I was only responding to the comment that building on an SSD
was no slower than tmpfs.

> And I know gentoo does clean up behind itself, but I use tmpfs so the
> system cleans up behind it too, and it's got as much room as it needs.

I can't agree with the second part, a tmpfs has far less space than a
disk. I had a gcc build fail this morning when /tmp filled up. Not
because gcc needs more than I had but because another emerge had failed
and left a G or two in /tmp. You'll always have more disk space than
RAM - embedded aside.


--
Neil Bothwick

Forget the Joneses...I can't keep up with The Simpsons.

On Wednesday, April 28, 2021, Alan Mackenzie <acm@muc.de> wrote:

> Hello, Gentoo.
>
> I'm having problems building rust.


On systems that I setup assuming I wouldn’t need swap, I set swappiness low
and add a swap file (on ssd if possible) so the bigger builds can at least
finish, albeit more slowly. Generally swap never gets touched. I also use
zswap with zstd or lz4, which perform better than the old lzo deflate/zlib
options, sometimes by a wide margin.

I don’t bother with /var/tmp/portage on tmpfs any more as it seemed to be
more trouble than it was worth.