Mailing List Archive

On 2021.04.03 07:15, karl@aspodata.se wrote:
> Is there a way to run emerge step by step to find out why it fails ?
>
> ///
>
> I get this:
> make[1]: Entering directory
> '/Net/gentoo/tmpdir/portage/dev-lang/R-4.0.4/work/R-4.0.4/doc'
> make[1]: Leaving directory
> '/Net/gentoo/tmpdir/portage/dev-lang/R-4.0.4/work/R-4.0.4'
> help2man: can't get `--version' info from ../bin/R
> make[1]: *** [Makefile:46: R.1] Error 255
> make[1]: *** Waiting for unfinished jobs....
Try running with -j1


> Error: package or namespace load failed for ?<80><98>methods?<80><99>:
> .onLoad failed in loadNamespace() for 'methods', details:
> call: (function (n)
> error: lazy-load database
> '/Net/gentoo/tmpdir/portage/dev-lang/R-4.0.4/work/R-4.0.4/library/methods/R/methods.rdb'
> is corrupt
> During startup - Warning messages:
> 1: In (function (n) : internal error -4 in R_decompress1
> 2: package "methods" in options("defaultPackages") was not found
> make[1]: Leaving directory
> '/Net/gentoo/tmpdir/portage/dev-lang/R-4.0.4/work/R-4.0.4/doc'
>
> but ../bin/R --version works:
> # cd /Net/gentoo/tmpdir/portage/dev-lang/R-4.0.4/work/R-4.0.4/doc
> # ../bin/R --version
> WARNING: ignoring environment value of R_HOME
> R version 4.0.4 (2021-02-15) -- "Lost Library Book"
> Copyright (C) 2021 The R Foundation for Statistical Computing
> Platform: x86_64-pc-linux-gnu (64-bit)
>
> R is free software and comes with ABSOLUTELY NO WARRANTY.
> You are welcome to redistribute it under the terms of the
> GNU General Public License versions 2 or 3.
> For more information about these matters see
> https://www.gnu.org/licenses/.
>
> #
>
> ///
>
> I also get segfault:
> begin installing recommended package MASS
> begin installing recommended package lattice
> Segmentation fault
> make[2]: *** [Makefile:53: MASS.ts] Error 1
> make[2]: *** Waiting for unfinished jobs....
> * installing *source* package ?<80><98>codetools?<80><99> ...
> ** package ?<80><98>codetools?<80><99> successfully unpacked and MD5
> sums checked
> ** using non-staged installation
>
> ///
>
> Full build log is here:
> http://aspodata.se/tmp/build.log
>
> Regards,
> /Karl Hammar
>
>
>

On Sat, 2021-04-03 at 13:15 +0200, karl@aspodata.se wrote:
> Is there a way to run emerge step by step to find out why it fails ?
>

No easy way. You can `cp -a` the source/build directories out of
/var/tmp/portage and then re-run `make -j<whatever>`. That should re-
start the build more or less where it failed.

Feel free to file a bug with what you discover and CC me on it.

Jack:
> On 2021.04.03 07:15, karl@aspodata.se wrote:
...
> > make[1]: *** Waiting for unfinished jobs....
> Try running with -j1
...

I already have:
EMERGE_DEFAULT_OPTS="--jobs 1 ...
in make.conf. Don't know why it still wants to run in parallell.

Regards,
/Karl Hammar

Michael:
> On Sat, 2021-04-03 at 13:15 +0200, karl@aspodata.se wrote:
> > Is there a way to run emerge step by step to find out why it fails ?
> >
>
> No easy way. You can `cp -a` the source/build directories out of
> /var/tmp/portage and then re-run `make -j<whatever>`. That should re-
> start the build more or less where it failed.

$ cp -a ...
$ cd ...
$ make -j1
...
$ echo $?
0

It builds without failure in that case.

Regards,
/Karl Hammar

On Mon, 2021-04-05 at 00:44 +0200, karl@aspodata.se wrote:
> Michael:
>
> $ cp -a ...
> $ cd ...
> $ make -j1
> ...
> $ echo $?
> 0
>
> It builds without failure in that case.
>

That's (potentially) good news. Can you now try it with the
CFLAGS/CXXFLAGS and LDFLAGS from your `emerge --info`? If the build
fails in that case, then we know it's one of those flags causing a
problem.

On 4/4/21 6:44 PM, karl@aspodata.se wrote:
> Jack:
>> On 2021.04.03 07:15, karl@aspodata.se wrote:
> ...
>>> make[1]: *** Waiting for unfinished jobs....
>> Try running with -j1
> ...
>
> I already have:
> EMERGE_DEFAULT_OPTS="--jobs 1 ...
> in make.conf. Don't know why it still wants to run in parallel.
--jobs=1 tells emerge to only emerge one package at a time.  The -j 1
(actually needs to be part of 'MAKEOPTS="-j1 ...."' is passed in to make
to tell it to only compile one piece at a time.  Using a higher number
for -j sometimes fails if it needs some intermediate result that has not
yet been produced.  Sometimes that failure is consistently repeatable,
and sometimes not.

> On Mon, 2021-04-05 at 00:44 +0200, karl@aspodata.se wrote:
...
> > $ cp -a ...
> > $ cd ...
> > $ make -j1
> > ...
> > $ echo $?
> > 0
> >
> > It builds without failure in that case.
> >
>
> That's (potentially) good news. Can you now try it with the
> CFLAGS/CXXFLAGS and LDFLAGS from your `emerge --info`? If the build
> fails in that case, then we know it's one of those flags causing a
> problem.

It seems -j1 succeeds where -j4 fails, and it might be memory related.

///

# make clean
# make

does fail

# make clean
# ./configure
...
R is now configured for x86_64-pc-linux-gnu

Source directory: .
Installation directory: /usr/local

C compiler: gcc -g -O2
Fortran fixed-form compiler: gfortran -fno-optimize-sibling-calls -g -O2

Default C++ compiler: g++ -std=gnu++11 -g -O2
C++14 compiler: g++ -std=gnu++14 -g -O2
C++17 compiler: g++ -std=gnu++17 -g -O2
C++20 compiler:
Fortran free-form compiler: gfortran -fno-optimize-sibling-calls -g -O2
Obj-C compiler: gcc -g -O2 -fobjc-exceptions

Interfaces supported: X11, tcltk
External libraries: pcre2, readline, curl
Additional capabilities: PNG, JPEG, TIFF, NLS, cairo, ICU
Options enabled: shared BLAS, R profiling

Capabilities skipped:
Options not enabled: memory profiling

Recommended packages: yes

# make

ok (=succeeds)

adding (as environment variables)
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe"
make clean; make
ok

CHOST="x86_64-pc-linux-gnu"
CXXFLAGS="-O2 -pipe"
ok

FCFLAGS="-O2 -pipe"
FFLAGS="-O2 -pipe"
ok

LDFLAGS="-Wl,-O1 -Wl,--as-needed"
ok

From ..../portage/dev-lang/R-4.0.4/work/R-4.0.4/config.log
# make distclean
# ./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --docdir=/usr/share/doc/R-4.0.4 --htmldir=/usr/share/doc/R-4.0.4/html --with-sysroot=/ --libdir=/usr/lib64 --enable-byte-compiled-packages --enable-R-shlib --disable-R-framework --with-blas=-lblas rdocdir=/usr/share/doc/R-4.0.4 --disable-java --enable-nls --enable-openmp --disable-R-profiling --disable-memory-profiling --enable-static --enable-R-static-lib --without-cairo --without-ICU --with-jpeglib --without-lapack --with-recommended-packages --with-libpng --with-readline --with-libtiff --without-tcltk --without-tk-config --without-tcl-config --with-x
...
R is now configured for x86_64-pc-linux-gnu

Source directory: .
Installation directory: /usr

C compiler: x86_64-pc-linux-gnu-gcc -g -O2
Fortran fixed-form compiler: x86_64-pc-linux-gnu-gfortran -fno-optimize-sibling-calls -g -O2

Default C++ compiler: x86_64-pc-linux-gnu-g++ -std=gnu++11 -g -O2
C++14 compiler: x86_64-pc-linux-gnu-g++ -std=gnu++14 -g -O2
C++17 compiler: x86_64-pc-linux-gnu-g++ -std=gnu++17 -g -O2
C++20 compiler:
Fortran free-form compiler: x86_64-pc-linux-gnu-gfortran -fno-optimize-sibling-calls -g -O2
Obj-C compiler: x86_64-pc-linux-gnu-gcc -g -O2 -fobjc-exceptions

Interfaces supported: X11
External libraries: pcre2, readline, BLAS(generic), curl
Additional capabilities: PNG, JPEG, TIFF, NLS
Options enabled: shared R library

Capabilities skipped: cairo, ICU
Options not enabled: shared BLAS, R profiling, memory profiling

Recommended packages: yes

# make
ok

add CFLAGS="-O2 -pipe" CXXFLAGS="-O2 -pipe" FCFLAGS="-O2 -pipe" FFLAGS="-O2 -pipe" LDFLAGS="-Wl,-O1 -Wl,--as-needed"
# make distclean
# ./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --docdir=/usr/share/doc/R-4.0.4 --htmldir=/usr/share/doc/R-4.0.4/html --with-sysroot=/ --libdir=/usr/lib64 --enable-byte-compiled-packages --enable-R-shlib --disable-R-framework --with-blas=-lblas rdocdir=/usr/share/doc/R-4.0.4 --disable-java --enable-nls --enable-openmp --disable-R-profiling --disable-memory-profiling --enable-static --enable-R-static-lib --without-cairo --without-ICU --with-jpeglib --without-lapack --with-recommended-packages --with-libpng --with-readline --with-libtiff --without-tcltk --without-tk-config --without-tcl-config --with-x CFLAGS="-O2 -pipe" CXXFLAGS="-O2 -pipe" FCFLAGS="-O2 -pipe" FFLAGS="-O2 -pipe" LDFLAGS="-Wl,-O1 -Wl,--as-needed"
...
R is now configured for x86_64-pc-linux-gnu

Source directory: .
Installation directory: /usr

C compiler: x86_64-pc-linux-gnu-gcc -O2 -pipe
Fortran fixed-form compiler: x86_64-pc-linux-gnu-gfortran -fno-optimize-sibling-calls -O2 -pipe

Default C++ compiler: x86_64-pc-linux-gnu-g++ -std=gnu++11 -O2 -pipe
C++14 compiler: x86_64-pc-linux-gnu-g++ -std=gnu++14 -O2 -pipe
C++17 compiler: x86_64-pc-linux-gnu-g++ -std=gnu++17 -O2 -pipe
C++20 compiler:
Fortran free-form compiler: x86_64-pc-linux-gnu-gfortran -fno-optimize-sibling-calls -O2 -pipe
Obj-C compiler: x86_64-pc-linux-gnu-gcc -g -O2 -fobjc-exceptions

Interfaces supported: X11
External libraries: pcre2, readline, BLAS(generic), curl
Additional capabilities: PNG, JPEG, TIFF, NLS
Options enabled: shared R library

Capabilities skipped: cairo, ICU
Options not enabled: shared BLAS, R profiling, memory profiling

Recommended packages: yes

# make -j1
ok

# make clean
# make -j4
...
make[1]: Leaving directory '/home/sys/R-4.0.4/work/R-4.0.4/tests'
you should 'make docs' now ...
make[1]: Entering directory '/home/sys/R-4.0.4/work/R-4.0.4/doc'
make[1]: Entering directory '/home/sys/R-4.0.4/work/R-4.0.4'
make[1]: 'stamp-java' is up to date.
make[1]: Leaving directory '/home/sys/R-4.0.4/work/R-4.0.4'
WARNING: ignoring environment value of R_HOME
WARNING: ignoring environment value of R_HOME
WARNING: ignoring environment value of R_HOME
Segmentation fault
make[1]: *** [Makefile:77: NEWS.3.rds] Error 139
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/home/sys/R-4.0.4/work/R-4.0.4/doc'
make: [Makefile:72: docs] Error 2 (ignored)
make[1]: Entering directory '/home/sys/R-4.0.4/work/R-4.0.4/src/library'
make[2]: Entering directory '/home/sys/R-4.0.4/work/R-4.0.4/src/library'
installing parsed Rd
make[3]: Entering directory '/home/sys/R-4.0.4/work/R-4.0.4/src/library'
WARNING: ignoring environment value of R_HOME
WARNING: ignoring environment value of R_HOME
WARNING: ignoring environment value of R_HOME
WARNING: ignoring environment value of R_HOME
Segmentation fault
make[3]: *** [Makefile:73: stats.Rdts] Error 1
make[3]: *** Waiting for unfinished jobs....
Error in readRDS(mapfile) : error reading from connection
Execution halted
Warning message:
system call failed: Cannot allocate memory
Segmentation fault
make[3]: *** [Makefile:73: methods.Rdts] Error 1
make[3]: *** [Makefile:73: base.Rdts] Error 1
utils
make[3]: Leaving directory '/home/sys/R-4.0.4/work/R-4.0.4/src/library'
make[2]: *** [Makefile:81: Rdobjects] Error 2
make[2]: Leaving directory '/home/sys/R-4.0.4/work/R-4.0.4/src/library'
make[1]: *** [Makefile:62: docs] Error 2
make[1]: Leaving directory '/home/sys/R-4.0.4/work/R-4.0.4/src/library'
make: [Makefile:73: docs] Error 2 (ignored)
make[1]: Entering directory '/home/sys/R-4.0.4/work/R-4.0.4/src/library/Recommended'
make[2]: Entering directory '/home/sys/R-4.0.4/work/R-4.0.4/src/library/Recommended'
begin installing recommended package MASS
begin installing recommended package lattice
begin installing recommended package codetools
begin installing recommended package foreign
WARNING: ignoring environment value of R_HOME
Fatal error: couldn't allocate memory for pointer stack
Segmentation fault
make[2]: *** [Makefile:53: lattice.ts] Error 1
make[2]: *** Waiting for unfinished jobs....
WARNING: ignoring environment value of R_HOME
Segmentation fault
make[2]: *** [Makefile:53: codetools.ts] Error 1
WARNING: ignoring environment value of R_HOME
* installing *source* package ?foreign? ...
** package ?foreign? successfully unpacked and MD5 sums checked
** using non-staged installation
** libs
make[3]: Entering directory '/tmp/Rtmpyf37hQ/R.INSTALL6e75db12a27/foreign/src'
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c R_systat.c -o R_systat.o
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c Rdbfread.c -o Rdbfread.o
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c Rdbfwrite.c -o Rdbfwrite.o
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c SASxport.c -o SASxport.o
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c avl.c -o avl.o
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c dbfopen.c -o dbfopen.o
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c file-handle.c -o file-handle.o
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c format.c -o format.o
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c init.c -o init.o
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c minitab.c -o minitab.o
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c pfm-read.c -o pfm-read.o
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c sfm-read.c -o sfm-read.o
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c spss.c -o spss.o
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c stataread.c -o stataread.o
x86_64-pc-linux-gnu-gcc -shared -L/home/sys/R-4.0.4/work/R-4.0.4/lib -Wl,-O1 -Wl,--as-needed -o foreign.so R_systat.o Rdbfread.o Rdbfwrite.o SASxport.o avl.o dbfopen.o file-handle.o format.o init.o minitab.o pfm-read.o sfm-read.o spss.o stataread.o -L/home/sys/R-4.0.4/work/R-4.0.4/lib -lR
make[3]: Leaving directory '/tmp/Rtmpyf37hQ/R.INSTALL6e75db12a27/foreign/src'
installing to /home/sys/R-4.0.4/work/R-4.0.4/library/foreign/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (foreign)
WARNING: ignoring environment value of R_HOME
* installing *source* package ?MASS? ...
** package ?MASS? successfully unpacked and MD5 sums checked
** using non-staged installation
** libs
make[3]: Entering directory '/tmp/RtmpY6iU2f/R.INSTALL6e078f85fff/MASS/src'
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c MASS.c -o MASS.o
x86_64-pc-linux-gnu-gcc -I"/home/sys/R-4.0.4/work/R-4.0.4/include" -DNDEBUG -fpic -O2 -pipe -c lqs.c -o lqs.o
x86_64-pc-linux-gnu-gcc -shared -L/home/sys/R-4.0.4/work/R-4.0.4/lib -Wl,-O1 -Wl,--as-needed -o MASS.so MASS.o lqs.o -L/home/sys/R-4.0.4/work/R-4.0.4/lib -lR
make[3]: Leaving directory '/tmp/RtmpY6iU2f/R.INSTALL6e078f85fff/MASS/src'
installing to /home/sys/R-4.0.4/work/R-4.0.4/library/MASS/libs
** R
** data
*** moving datasets to lazyload DB
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (MASS)
make[2]: Leaving directory '/home/sys/R-4.0.4/work/R-4.0.4/src/library/Recommended'
make[1]: *** [Makefile:39: recommended-packages] Error 2
make[1]: Leaving directory '/home/sys/R-4.0.4/work/R-4.0.4/src/library/Recommended'
make: *** [Makefile:77: stamp-recommended] Error 2

remove -pipe
# ./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --docdir=/usr/share/doc/R-4.0.4 --htmldir=/usr/share/doc/R-4.0.4/html --with-sysroot=/ --libdir=/usr/lib64 --enable-byte-compiled-packages --enable-R-shlib --disable-R-framework --with-blas=-lblas rdocdir=/usr/share/doc/R-4.0.4 --disable-java --enable-nls --enable-openmp --disable-R-profiling --disable-memory-profiling --enable-static --enable-R-static-lib --without-cairo --without-ICU --with-jpeglib --without-lapack --with-recommended-packages --with-libpng --with-readline --with-libtiff --without-tcltk --without-tk-config --without-tcl-config --with-x CFLAGS="-O2" CXXFLAGS="-O2" FCFLAGS="-O2" FFLAGS="-O2" LDFLAGS="-Wl,-O1 -Wl,--as-needed"
...
R is now configured for x86_64-pc-linux-gnu

Source directory: .
Installation directory: /usr

C compiler: x86_64-pc-linux-gnu-gcc -O2
Fortran fixed-form compiler: x86_64-pc-linux-gnu-gfortran -fno-optimize-sibling-calls -O2

Default C++ compiler: x86_64-pc-linux-gnu-g++ -std=gnu++11 -O2
C++14 compiler: x86_64-pc-linux-gnu-g++ -std=gnu++14 -O2
C++17 compiler: x86_64-pc-linux-gnu-g++ -std=gnu++17 -O2
C++20 compiler:
Fortran free-form compiler: x86_64-pc-linux-gnu-gfortran -fno-optimize-sibling-calls -O2
Obj-C compiler: x86_64-pc-linux-gnu-gcc -g -O2 -fobjc-exceptions

Interfaces supported: X11
External libraries: pcre2, readline, BLAS(generic), curl
Additional capabilities: PNG, JPEG, TIFF, NLS
Options enabled: shared R library

Capabilities skipped: cairo, ICU
Options not enabled: shared BLAS, R profiling, memory profiling

Recommended packages: yes

# make -j4
...
make[1]: Leaving directory '/home/sys/R-4.0.4/work/R-4.0.4/tests'
you should 'make docs' now ...
make[1]: Entering directory '/home/sys/R-4.0.4/work/R-4.0.4/doc'
make[1]: Entering directory '/home/sys/R-4.0.4/work/R-4.0.4'
WARNING: ignoring environment value of R_HOME
make[1]: Leaving directory '/home/sys/R-4.0.4/work/R-4.0.4'
WARNING: ignoring environment value of R_HOME
WARNING: ignoring environment value of R_HOME
Segmentation fault
make[1]: *** [Makefile:77: NEWS.3.rds] Error 139
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/home/sys/R-4.0.4/work/R-4.0.4/doc'
...

Regards,
/Karl Hammar

On Mon, 2021-04-05 at 16:10 +0200, karl@aspodata.se wrote:
> system call failed: Cannot allocate memory
> Segmentation fault

Is this a low-memory machine? If so, there's no much you can do here
except set a lower number of jobs in MAKEOPTS for the dev-lang/R build:

https://wiki.gentoo.org/wiki//etc/portage/package.env

But if you don't think you're running out of RAM, we can keep digging.
For example you might have some memory limits set in /etc/limits that
you're hitting.

Michael:
> On Mon, 2021-04-05 at 16:10 +0200, karl@aspodata.se wrote:
> > system call failed: Cannot allocate memory
> > Segmentation fault
>
> Is this a low-memory machine?

So, how much memory does R need to build ?
I have:

$ free
total used free shared buff/cache available
Mem: 6103628 1807560 2631444 146376 1664624 3107868
Swap: 8788924 670108 8118816

///

Somewhere in the log it says that something failed to get
a sigaltstack(). Did the R devs anticipate out of memory and sigseg and
tried to handle it ?

> If so, there's no much you can do here
> except set a lower number of jobs in MAKEOPTS for the dev-lang/R build:
>
> https://wiki.gentoo.org/wiki//etc/portage/package.env

I already have (in make.conf):
EMERGE_DEFAULT_OPTS="--jobs 1 ..."

This seems to solved the issue:
# cat /etc/portage/env/j1
MAKEOPTS=-j1
# cat /etc/portage/package.env/R
dev-lang/R j1

> But if you don't think you're running out of RAM, we can keep digging.
> For example you might have some memory limits set in /etc/limits that
> you're hitting.

I don't have an /etc/limits, and /etc/security/limits.conf is just
comments.

# prlimit -d -l -s
RESOURCE DESCRIPTION SOFT HARD UNITS
DATA max data size unlimited unlimited bytes
MEMLOCK max locked-in-memory address space unlimited unlimited bytes
STACK max stack size 8388608 unlimited bytes

Regards,
/Karl Hammar

On Mon, 2021-04-05 at 18:29 +0200, karl@aspodata.se wrote:
>
> So, how much memory does R need to build ?
> I have:
>
> $ free
>                total used free shared buff/cache available
> Mem: 6103628 1807560 2631444 146376 1664624 3107868
> Swap: 8788924 670108 8118816
>

We have the same amount of RAM, but building dev-lang/R doesn't come
close to using it all on my machine, even with MAKEOPTS="-j4"...

>
>  I already have (in make.conf):
> EMERGE_DEFAULT_OPTS="--jobs 1 ..."
>
>   This seems to solved the issue:
> # cat /etc/portage/env/j1
> MAKEOPTS=-j1
> # cat /etc/portage/package.env/R
> dev-lang/R j1

That will work around the issue, and you may have to use that trick for
some other packages like firefox/LLVM with "only" 6GB of RAM. But it
doesn't explain why I can build dev-lang/R and you can't.

What USE flags does "emerge -pv1 dev-lang/R" show? Are you willing to
watch "htop" in another window while the dev-lang/R build fails to see
how much memory is free?

Michael:
...
> What USE flags does "emerge -pv1 dev-lang/R" show?

# emerge -pv1 dev-lang/R
...
[ebuild R ] dev-lang/R-4.0.4::gentoo USE="X jpeg nls openmp perl png readline static-libs tiff -cairo -doc -icu -java -lapack -minimal (-prefix) -profile -test -tk" 0 KiB

> Are you willing to watch "htop" in another window while the dev-lang/R
> build fails to see how much memory is free?

htop -u portage shows high cpu usage and low memory usage (something
like less than 2%).

if I use this instead of htop:
# grep -m1 ^MemFree /proc/meminfo
MemFree: 253332 kB

I get for -j1:
mean = 397662.00592
min = 196772
max = 762304
std = 161394.91895

I have put up data from some -j4 runs at:
http://aspodata.se/tmp/R/
where mem4* are made with ./memwatch.pl > mem4_<run#>.log. It's columns are
just seconds, MemFree, and MemAvailable, taken from:

$ head -3 /proc/meminfo
MemTotal: 6103628 kB
MemFree: 1028012 kB
MemAvailable: 2326588 kB

///

from mem4_<run#>.log, I see that the minimum free memory is:
run 1 203492
run 2 202392
i.e. about 200MB.

build_1.log points the finger to sandbox. So it might be that there
i no problem at all with R, it might be that sandbox doesn't work here.
I don't know what sandbox requires from the kernel so there might be
some syscall missing.

I did run 2 as
FEATURES="-sandbox -usersandbox" emerge -aqv1 dev-lang/R

In build_2.log it just says Segment fault in the vincinity of
NEWS.2.rds, NEWS.rds, base.Rdts, and methods.Rdts. The first
two is mentioned in doc/Makefile, and the latter two in
src/library/Makefile. Going into doc and src/library and makeing
thoose four works without any errors.

Regards,
/Karl Hammar

On Wed, 2021-04-07 at 12:54 +0200, karl@aspodata.se wrote:

> # emerge -pv1 dev-lang/R
> ...
> [ebuild R ] dev-lang/R-4.0.4::gentoo USE="... static-libs ..."

Thanks, this is really good debugging information. Is that USE=static-
libs a global flag on your system? That may explain why your build is
using more memory than mine.

Michael:
> On Wed, 2021-04-07 at 12:54 +0200, karl@aspodata.se wrote:
> > # emerge -pv1 dev-lang/R
> > ...
> > [ebuild R ] dev-lang/R-4.0.4::gentoo USE="... static-libs ..."
>
> Thanks, this is really good debugging information. Is that USE=static-
> libs a global flag on your system? That may explain why your build is
> using more memory than mine.

yea, it was a try to make c++ programs to behave under changing
compiler and library situations. Seems that some such programs
don't want to be built statically so they break whenever some "random"
lib changes.

Regards,
/Karl Hammar

On Wed, 2021-04-07 at 22:15 +0200, karl@aspodata.se wrote:
>
> yea, it was a try to make c++ programs to behave under changing
> compiler and library situations. Seems that some such programs
> don't want to be built statically so they break whenever some "random"
> lib changes.
>

That's my best guess as to why your build is using so much more RAM.
Every library with a static-libs USE flag will have a bunch of other
libraries "copy & pasted" into it, and all of those copies take up
their own separate chunk of RAM.

Personally, I would disable static-libs. Within Gentoo the flag is
mostly there because GNU autotools makes it easy to do -- not because
it's a well-thought-out and reliable feature. But if it solves a more-
annoying problem for you, then maybe it's worth it =) In that case
lowering the -j flag for specific packages when you get an allocation
failure is a reasonable workaround.