Mailing List Archive

On Sat, Sep 17, 2022 at 10:40:53AM +0800, johnstrass wrote

> Compiling gcc uses alot of memory, sometimes only less then 10MB
> left free ( I am doing this on a small yeeloong netbook with only
> 1GB memory).

Ouch, I'm surprised it compiles at all. The Gentoo install handbook
https://wiki.gentoo.org/wiki/Handbook:AMD64/Full/Installation#MAKEOPTS
has a dire warning about memory usage...

> Warning
> Using a large number of jobs can significantly impact memory
> consumption. A good recommendation is to have at least 2 GiB of RAM
> for every job specified (so, e.g. -j6 requires at least 12 GiB). To
> avoid running out of memory, lower the number of jobs to fit the
> available memory.

You simply don't have enough ram. You might reduce the pain and the
swapping by trying something like the following in make.conf (assuming
you haven't already done it)

MAKEOPTS="-j1"

> If this happens, what is the best way to reboot or shutdown?

Try ye olde {CTRL}{ALT}{DEL}

--
I've seen things, you people wouldn't believe; Gopher, Netscape with
frames, the first Browser Wars. Searching for pages with AltaVista,
pop-up windows self-replicating, trying to uninstall RealPlayer. All
those moments, will be lost in time like tears in rain... time to die.

At 2022-09-17 14:15:51, "Walter Dnes" <waltdnes@waltdnes.org> wrote:

>On Sat, Sep 17, 2022 at 10:40:53AM +0800, johnstrass wrote
>
>> Compiling gcc uses alot of memory, sometimes only less then 10MB
>> left free ( I am doing this on a small yeeloong netbook with only
>> 1GB memory).
>
> Ouch, I'm surprised it compiles at all. The Gentoo install handbook
>https://wiki.gentoo.org/wiki/Handbook:AMD64/Full/Installation#MAKEOPTS
>has a dire warning about memory usage...

>



Yep, I've got the gcc-12.2.0 natively compiled on this netbook using gcc-11.3.0, after 5+ days + a crash + 5more days.

I set up a large swap and I've also set up the ccache which may speedup a little after the crash.


>> Warning
>> Using a large number of jobs can significantly impact memory
>> consumption. A good recommendation is to have at least 2 GiB of RAM
>> for every job specified (so, e.g. -j6 requires at least 12 GiB). To
>> avoid running out of memory, lower the number of jobs to fit the
>> available memory.
>
> You simply don't have enough ram. You might reduce the pain and the
>swapping by trying something like the following in make.conf (assuming
>you haven't already done it)
>

>MAKEOPTS="-j1"


I will try next time. Thanks.


>
>> If this happens, what is the best way to reboot or shutdown?
>

> Try ye olde {CTRL}{ALT}{DEL}




I remember that I did try "CTRL+ALT+DEL" in such situations before, and most times it did not work but for one or two times it worked occasionally. I will try it again next time.




>
>--
>I've seen things, you people wouldn't believe; Gopher, Netscape with
>frames, the first Browser Wars. Searching for pages with AltaVista,
>pop-up windows self-replicating, trying to uninstall RealPlayer. All
>those moments, will be lost in time like tears in rain... time to die.

On 2022-09-17 17:37+0800 johnstrass <johnstrass@163.com> wrote:

> At 2022-09-17 14:15:51, "Walter Dnes" <waltdnes@waltdnes.org> wrote:
>
> >On Sat, Sep 17, 2022 at 10:40:53AM +0800, johnstrass wrote
> >
> >> Compiling gcc uses alot of memory, sometimes only less then 10MB
> >> left free ( I am doing this on a small yeeloong netbook with only
> >> 1GB memory).
> >
> > Ouch, I'm surprised it compiles at all. The Gentoo install
> > handbook
> >https://wiki.gentoo.org/wiki/Handbook:AMD64/Full/Installation#MAKEOPTS
> >has a dire warning about memory usage...
>
> >
>
>
>
> Yep, I've got the gcc-12.2.0 natively compiled on this netbook using
> gcc-11.3.0, after 5+ days + a crash + 5more days.
>
> I set up a large swap and I've also set up the ccache which may
> speedup a little after the crash.
>
>
> >> Warning
> >> Using a large number of jobs can significantly impact memory
> >> consumption. A good recommendation is to have at least 2 GiB of RAM
> >> for every job specified (so, e.g. -j6 requires at least 12 GiB). To
> >> avoid running out of memory, lower the number of jobs to fit the
> >> available memory.
> >
> > You simply don't have enough ram. You might reduce the pain and
> > the
> >swapping by trying something like the following in make.conf
> >(assuming you haven't already done it)
> >
>
> >MAKEOPTS="-j1"
>
>
> I will try next time. Thanks.
>
>
> >
> >> If this happens, what is the best way to reboot or shutdown?
> >
>
> > Try ye olde {CTRL}{ALT}{DEL}
>
>
>
>
> I remember that I did try "CTRL+ALT+DEL" in such situations before,
> and most times it did not work but for one or two times it worked
> occasionally. I will try it again next time.

You could also use “Magic SysRQ”s[1] to reboot the system forcefully
but somewhat controlled. The traditional sequence is:
r e i s u b (raising elephants is so utterly boring)[2]

[1] <https://wiki.gentoo.org/wiki/Magic_SysRQ>
[2] <https://www.kernel.org/doc/html/latest/admin-guide/sysrq.html#what-are-the-command-keys>

On Saturday, 17 September 2022 12:11:56 BST tastytea wrote:
> On 2022-09-17 17:37+0800 johnstrass <johnstrass@163.com> wrote:
> > At 2022-09-17 14:15:51, "Walter Dnes" <waltdnes@waltdnes.org> wrote:
> > >On Sat, Sep 17, 2022 at 10:40:53AM +0800, johnstrass wrote
> > >
> > >> Compiling gcc uses alot of memory, sometimes only less then 10MB
> > >> left free ( I am doing this on a small yeeloong netbook with only
> > >> 1GB memory).
> > >>
> > > Ouch, I'm surprised it compiles at all. The Gentoo install
> > >
> > > handbook
> > >
> > >https://wiki.gentoo.org/wiki/Handbook:AMD64/Full/Installation#MAKEOPTS
> > >has a dire warning about memory usage...
> >
> > Yep, I've got the gcc-12.2.0 natively compiled on this netbook using
> > gcc-11.3.0, after 5+ days + a crash + 5more days.
> >
> > I set up a large swap and I've also set up the ccache which may
> > speedup a little after the crash.
> >
> > >> Warning
> > >>
> > >> Using a large number of jobs can significantly impact memory
> > >> consumption. A good recommendation is to have at least 2 GiB of RAM
> > >> for every job specified (so, e.g. -j6 requires at least 12 GiB). To
> > >> avoid running out of memory, lower the number of jobs to fit the
> > >> available memory.
> > >>
> > > You simply don't have enough ram. You might reduce the pain and
> > >
> > > the
> > >
> > >swapping by trying something like the following in make.conf
> > >(assuming you haven't already done it)
> > >
> > >
> > >MAKEOPTS="-j1"
> >
> > I will try next time. Thanks.
> >
> > >> If this happens, what is the best way to reboot or shutdown?
> > >>
> > > Try ye olde {CTRL}{ALT}{DEL}
> >
> > I remember that I did try "CTRL+ALT+DEL" in such situations before,
> > and most times it did not work but for one or two times it worked
> > occasionally. I will try it again next time.
>
> You could also use “Magic SysRQ”s[1] to reboot the system forcefully
> but somewhat controlled. The traditional sequence is:
> r e i s u b (raising elephants is so utterly boring)[2]
>
> [1] <https://wiki.gentoo.org/wiki/Magic_SysRQ>
> [2]
> <https://www.kernel.org/doc/html/latest/admin-guide/sysrq.html#what-are-the
> -command-keys>

Using MAKEOPTS="-j1" will help and a bigger swap will stop it crashing with
OOM, but a more sensible solution is to compile and build big packages like
gcc on a faster PC as binaries, then transfer and emerge on the slow PC with
--usepkg, or --usepkgonly options. More info:

https://wiki.gentoo.org/wiki/Binary_package_guide

OK.


However, my real confusion or worry in fact is not the compilation of gcc, but about the logind. This obvious happens not only in compilation of gcc. Whenever there are programs consuming almost all the memory, the logind will be disconnected and not come back again. Why is the logind so fragile? Why cannot it be brought up again after the memeory become available again? Or why should the systemd depend on such a faulty logind in order to do important jobs like shutdown? Don't you think it is a serious weakness of the system?














At 2022-09-17 19:43:54, "Michael" <confabulate@kintzios.com> wrote:
>On Saturday, 17 September 2022 12:11:56 BST tastytea wrote:
>> On 2022-09-17 17:37+0800 johnstrass <johnstrass@163.com> wrote:
>> > At 2022-09-17 14:15:51, "Walter Dnes" <waltdnes@waltdnes.org> wrote:
>> > >On Sat, Sep 17, 2022 at 10:40:53AM +0800, johnstrass wrote
>> > >
>> > >> Compiling gcc uses alot of memory, sometimes only less then 10MB
>> > >> left free ( I am doing this on a small yeeloong netbook with only
>> > >> 1GB memory).
>> > >>
>> > > Ouch, I'm surprised it compiles at all. The Gentoo install
>> > >
>> > > handbook
>> > >
>> > >https://wiki.gentoo.org/wiki/Handbook:AMD64/Full/Installation#MAKEOPTS
>> > >has a dire warning about memory usage...
>> >
>> > Yep, I've got the gcc-12.2.0 natively compiled on this netbook using
>> > gcc-11.3.0, after 5+ days + a crash + 5more days.
>> >
>> > I set up a large swap and I've also set up the ccache which may
>> > speedup a little after the crash.
>> >
>> > >> Warning
>> > >>
>> > >> Using a large number of jobs can significantly impact memory
>> > >> consumption. A good recommendation is to have at least 2 GiB of RAM
>> > >> for every job specified (so, e.g. -j6 requires at least 12 GiB). To
>> > >> avoid running out of memory, lower the number of jobs to fit the
>> > >> available memory.
>> > >>
>> > > You simply don't have enough ram. You might reduce the pain and
>> > >
>> > > the
>> > >
>> > >swapping by trying something like the following in make.conf
>> > >(assuming you haven't already done it)
>> > >
>> > >
>> > >MAKEOPTS="-j1"
>> >
>> > I will try next time. Thanks.
>> >
>> > >> If this happens, what is the best way to reboot or shutdown?
>> > >>
>> > > Try ye olde {CTRL}{ALT}{DEL}
>> >
>> > I remember that I did try "CTRL+ALT+DEL" in such situations before,
>> > and most times it did not work but for one or two times it worked
>> > occasionally. I will try it again next time.
>>
>> You could also use ??Magic SysRQ??s[1] to reboot the system forcefully
>> but somewhat controlled. The traditional sequence is:
>> r e i s u b (raising elephants is so utterly boring)[2]
>>
>> [1] <https://wiki.gentoo.org/wiki/Magic_SysRQ>
>> [2]
>> <https://www.kernel.org/doc/html/latest/admin-guide/sysrq.html#what-are-the
>> -command-keys>
>
>Using MAKEOPTS="-j1" will help and a bigger swap will stop it crashing with
>OOM, but a more sensible solution is to compile and build big packages like
>gcc on a faster PC as binaries, then transfer and emerge on the slow PC with
>--usepkg, or --usepkgonly options. More info:
>
>https://wiki.gentoo.org/wiki/Binary_package_guide

On Sat, Sep 17, 2022 at 8:21 AM johnstrass <johnstrass@163.com> wrote:
>
>
> Why is the logind so fragile?

Have you checked your logs. I'm guessing that the kernel OOM killer
is killing it, and it is kind of hard for a process to not die when
the kernel kills it.

> Why cannot it be brought up again after the memeory become available again?

I suspect it probably could be - probably not a failure mode upstream
has paid much attention to. If the OOM killer is loose on your
system, general breakage is to be expected. It is actually surprising
though that it didn't go after gcc itself. I'd check the logs.

You probably could tweak the unit setting so that logind is less
likely to get prioritized. Then it might go after something less
essential like sshd or postfix. :)

Also possible that it isn't logind itself but something else it uses
for IPC. Haven't looked into the gory details of how it works.

I'm guessing systemd could be coaxed to shut down without it. It
might actually do that on its own if you give it time, but I haven't
checked the settings.

--
Rich

Your answer seems to be on the right track I expected.





At 2022-09-17 21:08:21, "Rich Freeman" <rich0@gentoo.org> wrote:
>On Sat, Sep 17, 2022 at 8:21 AM johnstrass <johnstrass@163.com> wrote:
>>
>>
>> Why is the logind so fragile?
>
>Have you checked your logs. I'm guessing that the kernel OOM killer
>is killing it, and it is kind of hard for a process to not die when
>the kernel kills it.
>
>> Why cannot it be brought up again after the memeory become available again?
>
>I suspect it probably could be - probably not a failure mode upstream
>has paid much attention to. If the OOM killer is loose on your
>system, general breakage is to be expected. It is actually surprising
>though that it didn't go after gcc itself. I'd check the logs.
>
>You probably could tweak the unit setting so that logind is less
>likely to get prioritized. Then it might go after something less
>essential like sshd or postfix. :)
>
>Also possible that it isn't logind itself but something else it uses
>for IPC. Haven't looked into the gory details of how it works.
>
>I'm guessing systemd could be coaxed to shut down without it. It
>might actually do that on its own if you give it time, but I haven't
>checked the settings.

>


If you have some clue, I would like to know how to coax it.
If you would like to have a look at some logs on my machine, I would love to send them to you next time I meet such problems.


>--
>Rich

Am Sat, Sep 17, 2022 at 02:15:51AM -0400 schrieb Walter Dnes:
> On Sat, Sep 17, 2022 at 10:40:53AM +0800, johnstrass wrote
>
> > Compiling gcc uses alot of memory, sometimes only less then 10MB
> > left free ( I am doing this on a small yeeloong netbook with only
> > 1GB memory).
>
> Ouch, I'm surprised it compiles at all. The Gentoo install handbook
> https://wiki.gentoo.org/wiki/Handbook:AMD64/Full/Installation#MAKEOPTS
> has a dire warning about memory usage...
>
> […]
>
> You simply don't have enough ram. You might reduce the pain and the
> swapping by trying something like the following in make.conf (assuming
> you haven't already done it)
>
> MAKEOPTS="-j1"

You could also have a look at zram/zswap to compress your RAM content.
https://wiki.gentoo.org/wiki/Zram
https://wiki.archlinux.org/title/Zswap

--
Grüße | Greetings | Salut | Qapla’
There are two kinds of computer people: 1 and 0.