Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Gentoo>
  3. Hardened
clone(2) blocks for 30 seconds
powerman at powerman
Dec 23, 2012, 6:16 PM
Post #1 of 3 (1277 views)
Permalink
Hi!

Please take a look at
http://serverfault.com/questions/460429/clone2-30-sec-delay-in-apache

I didn't think it may be related to hardened but I've just found this in
kernel logs:

2012-12-23_20:45:19.15938 kern.alert: grsec: From 75.101.174.3: Segmentation fault occurred at 000014e2 in /usr/sbin/apache2[apache2:5346] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:1391] uid/euid:0/0 gid/egid:0/0
2012-12-23_20:45:19.17936 kern.alert: grsec: From 75.101.174.3: Segmentation fault occurred at 000014b6 in /usr/sbin/apache2[apache2:5302] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:1391] uid/euid:0/0 gid/egid:0/0
2012-12-23_22:28:17.53334 kern.alert: grsec: From 91.207.5.222: Segmentation fault occurred at 00003e5a in /usr/sbin/apache2[apache2:15962] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:5461] uid/euid:0/0 gid/egid:0/0
2012-12-23_22:28:17.69334 kern.alert: grsec: From 91.207.5.222: Segmentation fault occurred at 00003c0e in /usr/sbin/apache2[apache2:15374] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:5461] uid/euid:0/0 gid/egid:0/0
2012-12-23_22:28:17.92335 kern.alert: grsec: From 91.207.5.222: Segmentation fault occurred at 00004214 in /usr/sbin/apache2[apache2:16916] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:5461] uid/euid:0/0 gid/egid:0/0
2012-12-23_22:28:18.75335 kern.alert: grsec: From 91.207.5.222: Segmentation fault occurred at 00003fa4 in /usr/sbin/apache2[apache2:16292] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:5461] uid/euid:0/0 gid/egid:0/0

I can't check this right now - I have to wait until this bug happens again -
but looks like this bug happens after about 10-20 such records in logs.
So, maybe grsec break something in kernel while handling these segfaults
which in turn result in delay in clone(2)? Or maybe I have to run memtest. :(

--
WBR, Alex.
Re: clone(2) blocks for 30 seconds [ In reply to ]
klondike at gentoo
Dec 23, 2012, 6:37 PM
Post #2 of 3 (1221 views)
Permalink
El 24/12/12 03:16, Alex Efros escribió:
> Hi!
>
> Please take a look at
> http://serverfault.com/questions/460429/clone2-30-sec-delay-in-apache
>
> I didn't think it may be related to hardened but I've just found this in
> kernel logs:
>
> 2012-12-23_20:45:19.15938 kern.alert: grsec: From 75.101.174.3: Segmentation fault occurred at 000014e2 in /usr/sbin/apache2[apache2:5346] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:1391] uid/euid:0/0 gid/egid:0/0
> 2012-12-23_20:45:19.17936 kern.alert: grsec: From 75.101.174.3: Segmentation fault occurred at 000014b6 in /usr/sbin/apache2[apache2:5302] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:1391] uid/euid:0/0 gid/egid:0/0
> 2012-12-23_22:28:17.53334 kern.alert: grsec: From 91.207.5.222: Segmentation fault occurred at 00003e5a in /usr/sbin/apache2[apache2:15962] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:5461] uid/euid:0/0 gid/egid:0/0
> 2012-12-23_22:28:17.69334 kern.alert: grsec: From 91.207.5.222: Segmentation fault occurred at 00003c0e in /usr/sbin/apache2[apache2:15374] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:5461] uid/euid:0/0 gid/egid:0/0
> 2012-12-23_22:28:17.92335 kern.alert: grsec: From 91.207.5.222: Segmentation fault occurred at 00004214 in /usr/sbin/apache2[apache2:16916] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:5461] uid/euid:0/0 gid/egid:0/0
> 2012-12-23_22:28:18.75335 kern.alert: grsec: From 91.207.5.222: Segmentation fault occurred at 00003fa4 in /usr/sbin/apache2[apache2:16292] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:5461] uid/euid:0/0 gid/egid:0/0
>
> I can't check this right now - I have to wait until this bug happens again -
> but looks like this bug happens after about 10-20 such records in logs.
> So, maybe grsec break something in kernel while handling these segfaults
> which in turn result in delay in clone(2)? Or maybe I have to run memtest. :(
>
┌──────────────────────────────────────────────────────────────────────────────
Deter exploit bruteforcing
───────────────────────────────────────────────────────────────────────────────┐
│ CONFIG_GRKERNSEC_BRUTE: │
│ │
│ If you say Y here, attempts to bruteforce exploits against forking │
│ daemons such as apache or sshd, as well as against suid/sgid binaries │
│ will be deterred. When a child of a forking daemon is killed by PaX │
│ or crashes due to an illegal instruction or other suspicious signal, │
│ the parent process will be delayed 30 seconds upon every subsequent │
│ fork until the administrator is able to assess the situation and │
│ restart the daemon. │
│ In the suid/sgid case, the attempt is logged, the user has all their │
│ processes terminated, and they are prevented from executing any further │
│ processes for 15 minutes. │
│ It is recommended that you also enable signal logging in the auditing │
│ section so that logs are generated when a process triggers a suspicious │
│ signal. │
│ If the sysctl option is enabled, a sysctl option with name │
│ "deter_bruteforce" is created. │

Likely your culprit.

Attached Files:

Re: clone(2) blocks for 30 seconds [ In reply to ]
powerman at powerman
Dec 23, 2012, 6:58 PM
Post #3 of 3 (1227 views)
Permalink
Hi!

On Mon, Dec 24, 2012 at 03:37:14AM +0100, Francisco Blas Izquierdo Riera (klondike) wrote:
> El 24/12/12 03:16, Alex Efros escribió:
> > 2012-12-23_20:45:19.15938 kern.alert: grsec: From 75.101.174.3: Segmentation fault occurred at 000014e2 in /usr/sbin/apache2[apache2:5346] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:1391] uid/euid:0/0 gid/egid:0/0
> > 2012-12-23_20:45:19.17936 kern.alert: grsec: From 75.101.174.3: Segmentation fault occurred at 000014b6 in /usr/sbin/apache2[apache2:5302] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:1391] uid/euid:0/0 gid/egid:0/0
> > 2012-12-23_22:28:17.53334 kern.alert: grsec: From 91.207.5.222: Segmentation fault occurred at 00003e5a in /usr/sbin/apache2[apache2:15962] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:5461] uid/euid:0/0 gid/egid:0/0
> > 2012-12-23_22:28:17.69334 kern.alert: grsec: From 91.207.5.222: Segmentation fault occurred at 00003c0e in /usr/sbin/apache2[apache2:15374] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:5461] uid/euid:0/0 gid/egid:0/0
> > 2012-12-23_22:28:17.92335 kern.alert: grsec: From 91.207.5.222: Segmentation fault occurred at 00004214 in /usr/sbin/apache2[apache2:16916] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:5461] uid/euid:0/0 gid/egid:0/0
> > 2012-12-23_22:28:18.75335 kern.alert: grsec: From 91.207.5.222: Segmentation fault occurred at 00003fa4 in /usr/sbin/apache2[apache2:16292] uid/euid:81/81 gid/egid:81/81, parent /usr/sbin/apache2[apache2:5461] uid/euid:0/0 gid/egid:0/0
> │ CONFIG_GRKERNSEC_BRUTE: │
> │ │
> │ the parent process will be delayed 30 seconds upon every subsequent │
> │ fork until the administrator is able to assess the situation and │
> │ restart the daemon. │
>
> Likely your culprit.

Yes, thanks a lot!

I've checked these requests in logs - they are not exploits, just usual
requests. Most from bots. Few crashes happens even on requests sent by
wget running on same server by my monitoring script.

Restarting apache doesn't helps because it will crash again few hours later.
And this is production server and I've no idea how to debug this bug in apache.

So - only option I've is to disable this "feature"? :-/

--
WBR, Alex.

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal