I finally bit the bullet and finished this thing up today, looking for some
feedback before I go any further.
The point of this script is to locate valid directories in the web root of a
server. The list of discovered directories will be saved in the KB and be
used by any plugins which need this information. One good example would be a
plugin which looks for copies of WS_FTP.LOG in the web root, it could dimply
depend on this plugin, then iterate through all directories in the KB looking
for WS_FTP.LOG in each directory. Another example would be a plugin which
looks for directory indexes.
With some minor modifications to webmirror.nasl, it could use the list of
directories found by this plugin to perform a much more complete crawl. The
results of the crawl could be used to even further populate the directory
list in the KB. The way I would like to see this implemented:
[DDI_Directory_Scanner.nasl] -> [webmirror.nasl] -> [ws_ftp_log.nasl]
Where ws_ftp_log has a script_dependencie() for webmirror and webmirror
depends on DDI_Directory_Scanner. You could even flip that relationship and
allow the directory scanner to search the subdirectories of those found in
webmirror...
The current implementation uses 200, 403, and 401 responses to determine if a
directory exists. Only the 200 and 403 responses are considered when
populating the KB tree (not implemented yet). There is a built-in
"no404.nasl"-like check which looks for common error messages and works
around them, for servers which respond with a 200 OK for non-existent
directories. The 403 check will try to verify whether the directory itself is
protected or whether just directory indexes are disabled and there is no
default page.
The first chunk of the directory array was taken from nikto and whisker, I
browsed a few dozen vhosts on some colo machines I have access to and
populated it further based on what I found. PLEASE submit more directories if
you can and custom 404 messages that arent already handled.
I would like to take this one further and recursively search the
subdirectories of those already found, but I wanted to get something out
today for you all to play with.
Please test using the 'nasl' command line interpreter only please, there is no
script_id assigned so just dropping it into your plugin directory wont do
much do good. Here is an example run (apologies to Renaud):
hdm@masada:~> nasl DDI_Directory_Scanner.nasl -t www.nessus.org
** WARNING : packet forgery will not work
** as NASL is not running as root
DDI_Directory_Scanner.nasl : Warning : evaluating unknown variable -
description
Discovered: doc
Discovered: icons
The following directories were discovered: /doc, /icons
-HD
feedback before I go any further.
The point of this script is to locate valid directories in the web root of a
server. The list of discovered directories will be saved in the KB and be
used by any plugins which need this information. One good example would be a
plugin which looks for copies of WS_FTP.LOG in the web root, it could dimply
depend on this plugin, then iterate through all directories in the KB looking
for WS_FTP.LOG in each directory. Another example would be a plugin which
looks for directory indexes.
With some minor modifications to webmirror.nasl, it could use the list of
directories found by this plugin to perform a much more complete crawl. The
results of the crawl could be used to even further populate the directory
list in the KB. The way I would like to see this implemented:
[DDI_Directory_Scanner.nasl] -> [webmirror.nasl] -> [ws_ftp_log.nasl]
Where ws_ftp_log has a script_dependencie() for webmirror and webmirror
depends on DDI_Directory_Scanner. You could even flip that relationship and
allow the directory scanner to search the subdirectories of those found in
webmirror...
The current implementation uses 200, 403, and 401 responses to determine if a
directory exists. Only the 200 and 403 responses are considered when
populating the KB tree (not implemented yet). There is a built-in
"no404.nasl"-like check which looks for common error messages and works
around them, for servers which respond with a 200 OK for non-existent
directories. The 403 check will try to verify whether the directory itself is
protected or whether just directory indexes are disabled and there is no
default page.
The first chunk of the directory array was taken from nikto and whisker, I
browsed a few dozen vhosts on some colo machines I have access to and
populated it further based on what I found. PLEASE submit more directories if
you can and custom 404 messages that arent already handled.
I would like to take this one further and recursively search the
subdirectories of those already found, but I wanted to get something out
today for you all to play with.
Please test using the 'nasl' command line interpreter only please, there is no
script_id assigned so just dropping it into your plugin directory wont do
much do good. Here is an example run (apologies to Renaud):
hdm@masada:~> nasl DDI_Directory_Scanner.nasl -t www.nessus.org
** WARNING : packet forgery will not work
** as NASL is not running as root
DDI_Directory_Scanner.nasl : Warning : evaluating unknown variable -
description
Discovered: doc
Discovered: icons
The following directories were discovered: /doc, /icons
-HD
Attached Files:
-
DDI_Directory_Scanner.nasl (8.41 KB)