Mailing List Archive

v2.2.[01] nmap.nasl + OS detect problem + solution
Tested Systems: Linux Fedora Core 2 & 3, Nessus v2.2.0 & v2.2.1, OpenSSL
0.9.7d, nmap v3.70 & v3.75.

Hi,

Ever since I installed v2.2.0, I've noticed scans that have no portscan
results. This, of course, leads to incorrect scan results, since Nessus
thinks there are no 'live' ports, and the proper scripts aren't run, etc.
To get to the bottom of the issue, I wrapped the nmap binary with a shell
script, and saw that it was segfaulting either before or while outputting
its results. After examining a series of core dumps, I found that nmap was
crashing in several places, none of which made any sense - there was plenty
of free RAM, the load was low, there was just one target, etc. All of these
conditions seemed to be ideal for a scan, yet nmap was still crashing. When
I got v2.2.1 today, I hoped that it might fix the issue, but it did not.

After some work, a friend & I tracked this issue down to the 'Identify
remote OS' NMAP wrapper option. When this option was off, nmap scans worked
every time, with it on, they failed pretty much every time.

After a bit of thinking, we realized that the only difference between the
two options from Nessus' point of view (at least until the NMap process
exited) was the amount of memory taken up. After some searching through the
code, we found the issue...

//--------------------- CUT HERE ---------------------//

<file nessus-core/nessusd/nasl_plugins.c)

#ifdef RLIMIT_RSS
{
struct rlimit rlim;
getrlimit(RLIMIT_RSS, &rlim);
rlim.rlim_cur = 1024*1024*20;
rlim.rlim_max = 1024*1024*20;
setrlimit(RLIMIT_RSS, &rlim);
}
#endif

#ifdef RLIMIT_AS
... same as above but for the max address space size...
#endif

#ifdef RLIMIT_DATA
... same as above but for the max data size...
#endif

//--------------------- CUT HERE ---------------------//

Basically, nessusd sets the max resident set, max address space size and max
data segment size to 20MB. Before the advent of trusted scripts, this was
probably PLENTY of RAM for the biggest hog of a NASL script. However, now
that nasl can run external applications, this can (and has) cause problems.
I don't know how to track the maxiumum amount of memory used by a process,
but I was able to see by using vmstat and ps that nmap came very close to
going over its limit with its resident segment size, and went over 20MB with
its data size (24MB when 40 ports were open).

After switching the 20's to 40's (so that it matches the rlimits set by
nes_plugins.c) and rebuilding / restarting nessusd, the problem went away,
and now nessusd runs as designed.

Can anyone think of any potential downside to this mod, other than the
potential for runaway scripts taking up over 20MB of RAM before being denied
additional memory? As signed scripts become more and more popular, and Nmap
adds more and more functionality, it seems like the potential for this 20MB
limit will become more and more likely to be too little.

One last question - it seems strange to me that Nessusd didn't catch the
return value from the crashing Nmap. Is that a missing feature in the NASL
language, an oversight in design, or was it done on purpose? It just seemed
like it would've been a lot easier to track down this issue if Nessusd had
failed the scan. Oh well.

Hopefully this information re: the crashes can be of assistance to some
people. When trying to track down any fixe, I found a bunch of people who
had similar sounding issues, so hopefully this ends up being the answer
they've been looking for.

Let me know if anyone has a question, feel free to ask here.


---
btx
btx@calyx.net
Re: v2.2.[01] nmap.nasl + OS detect problem + solution [ In reply to ]
On Tue, Dec 07, 2004 at 07:09:05PM -0500, Brian Costello wrote:

Hi,

> After some work, a friend & I tracked this issue down to the 'Identify
> remote OS' NMAP wrapper option. When this option was off, nmap scans worked
> every time, with it on, they failed pretty much every time.

Good analysis, thanks. I thought we had removed OS recognition from
nmap.nasl.

I think the cleanest fix would be to have nasl perform some laxist
getrlimits() when the script is privileged. I'll add that on my todo
list, thanks.


-- Renaud