Mailing List Archive

Better false positive detection for CGI scanning (Modified by charl van der walt)
Hi there,

This is a re-post of a submission we sent just before Christmas last
year. We received ... 0 ... response at the time, which either means
that this is an idiotic submission, or that you were all simply too
busy with other holiday-related things. ;) In the hope that it's the
latter, we're posting this again. Please forgive the re-posting. We
really wouldn't do it if we didn't think it was worthwhile. We've also
posted to two lists. You'll have to forgive us for that also. The
point...

Some time ago we released a tool called Wikto. Wikto used the Nikto
scan database. It's a .NET tool. The main difference between Nikto and
Wikto was that you could set Wikto to ignore error codes and just rely
on message content to decide if the CGI is there or not. This was
achieved by building a database of responses for non-existent files and
doing a content compare between that and the response for the actual
request (this is done in real time BTW).

The method greatly decreases the occurance of false positives. It also
adds a little bandwidth (and time) overhead.

Currently Nessus uses no404.nasl for the detection of "friendly" 404
messages. This works fairly well..except when the server has different
behaviour for certain extentions, or responds differntly for different
locations. Wikto tries to solve this by storing responses according to
location and extention.

Extracting locations and extentions from attacks are not always easy.
Clearly /scripts/login.asp is an ASP file and located in /scripts/, but
how about /%c0%af/.../test?go=/cgi-bin/\\admin.pl ? Because of this
nastiness these parameters can be specified manually. If they are not
specified (as all NASLs will be currently) best effort is done to
extract it correctly. If the extention cannot be determined (or if the
request has
no extention) the value of "default" is given.

The response is saved in the KB (which dramatically increases the size
of the KB [with about 120k]) with a key of www/<port>/<path>/<ext>. A
typical key would be "www/80//webcart/config/txt". This means that the
checker does not need to replay the dummy request if it is already in
the KB.

So - how well does it work? Tested it against 2 servers:

1) Server comes back with a friendly 200 Message if the resource cannot
be found.
------------------------------------------------
Without no404.nasl and without wikto style is_cgi: 55 holes, 27
warnings [all false positives]

With no404.nasl and no wikto style is_cgi: 1 hole, 5 warnings [no false
positives]

Without no404.nasl and with wikto style is_cgi: 1 hole, 5 warnings [no
false positives]

2) Server responds with 200s on EXEs but normal 404 on resources it
cannot find
-------------------------------------------------
Without no404.nasl and without wikto style is_cgi:
18 holes, 3 infos [all false positives]

With no404.nasl and no wikto style is_cgi:
18 holes, 2 infos [all false positives]

Without no404.nasl and wikto style is_cgi:
0 holes, 1 info [no false positives]

During the same tests we also includes some vulnerabilities on the
servers to make sure that these will be detected (and it did so
nicely).

When the two messages are compared (VERY primative at the moment, but
works) a index is given - this index shows how close the messages
match. Because of changing server time the messages never quite match
perfectly. The value of the index range from 0-100 (and sometimes >100,
I said it was primative). At the moment the nasl triggers for anything
below 50. If the community likes the concept I guess we could have this
as a tweakable value in the config. Keep in mind that the lower you
have the value, the less false positives you would get...but the
greater the chance of missing stuff.

The script is a "drop-in" replacement for is_cgi_installed_ka. It takes
4
parameters:

function is_cgi_installed_ka(item, port, location, ext)

item is backwards compatible item...
port is .. well..the port
location is the location of the scripts - e.g. /scripts/
ext is the extension (without a dot) - e.g. dll

The last two parameters is optional - as said before - if you dont
trust the ext and location extraction tools please populate this.

If you want to play with this do the following:

1. Add the new functions to the end of http_keepalive.inc

2. Change the current is_cgi_installed_ka to is_cgi_installed_ka_old
within http_keepalive.inc.

3. If you want to see it work without no404.nasl then just do an
exit(0) early on in the script. There's no need for this tho..

If you have a mission critical server..make sure you test it properly
before you use it. It might not be everyone's cup of tea.

PS: I suspect the badblue_null.nasl is broken (you will see it show up
every time with this tech). Line 57 reads:

r = string("/ext.ini.% 00.txt");

which is exactly how it appears on Bugtraq. Wikto sees the path as /
and the ext as txt. It now request /SP_never_SP.txt, and gets the page
(let's for now assume it's a 404 Not Found). If you request /ext.ini.%
00.txt you almost certainly get back a 400 Bad Request. Because the
requests dont match it is treated as a positive. So..shouldn't it be
/ext.ini%00.txt? Makes more sense to me, unless I am missing the point?

Anyhow..enjoy and please send flames and comments to
research@sensepost.com.

Regards,
SensePost.