[.I have been thinking about this for a time and I recently talked
about all this privately with a couple of people]
Last year, we had less than 2000 Nessus scripts if I remember well.
Today we have 5570. Although some of them are related to a brand new
feature (local tests) and are automatically generated, it seems that
the number of "classical" plugins increase quicker than Moore's Law
(double every 18 months).
In a very far future, we will not be able to find a computer quick
enough to run the full test set before the target machine is
obsolete. So far it is only a nice topic for a geek scifi story :-)
The situation is not dramatic: most "local tests" are skipped when you
run "optimized" tests. As I did not like some implications of
script_exclude_keys() I added an "optimization_level" setting in
nessusd.conf - default (old behaviour) is 3, but I set it to 2
instead, which only enforce script_require_ports() and
script_require_keys(). Lowering it to 1 is not recommended as the
useless local tests will be run (script_require_keys is not enforce
any more, only script_require_ports)
However, as we pile new tests up, scans are expected to run (a
little?) slower.
Removing old tests is definitely a bad idea, as I have seen machines
that had never been patched for more than 5 years connected to
Internet! Why they were not rooted remains a mystery...
I have come with several possibilities to reduce the overall scanning
time:
1. Cut down the number of banner matching plugins and replace them
with generic tests. Thus, we might even find unpublished flaws.
But:
- everything cannot be tested easily
- most security advisories are fuzzy and reproducing the bug is not
easy.
- generic tests are often dangerous and would be declared as
ACT_DESTRUCTIVE_ATTACK, i.e. disabled by people who run "safe checks".
IMHO, "safe checks" should not be used most of the time. Users ask for
99.99% uptime and ultra high reliability but they often do not need
it. Exceptions are few: medical life support systems (don't even scan
them!), big banking / money transfer systems (but you may have time
for play between two cut off), round the clock worldwide airplane
reservation systems...
2. Rewrite the NASL interpretor using a VM. According to gforth /
vmgen developers, such an interpretor might be 10 to 100 times quicker
than a simple "syntax tree walker" (just like the current NASL2).
But:
- some people say the speed gain is not that great, and definitely not
worth the trouble
- the current code is clean and flexible, a stack machine based
interpretor might not be.
- CPU is not that an issue but on huge network. On my point of view,
memory is (because I usually use an old laptop to scan a couple of
machines, maybe? :)
3. Try other "simple" NASL optimizations. I have implemented a cache
for include files (they are only parsed once for all plugins) in the
CVS head branch. Even if it saves only 1% of CPU, it is worth it
because the code is simple & safe (less than one page of C, two
exported functions)
Another idea would be to convert some very common NASL functions into
C so that they run quicker.
4. Make sure that all plugins do not perform redundant operations.
e.g. if a plugin need a banner, it should be grabbed from the KB, not
read on the network.
I guess that there are few ill designed plugins (maybe none).
Any comment? New ideas?
--
arboi@alussinan.org http://arboi.da.ru
NASL2 reference manual http://michel.arboi.free.fr/nasl2ref/
about all this privately with a couple of people]
Last year, we had less than 2000 Nessus scripts if I remember well.
Today we have 5570. Although some of them are related to a brand new
feature (local tests) and are automatically generated, it seems that
the number of "classical" plugins increase quicker than Moore's Law
(double every 18 months).
In a very far future, we will not be able to find a computer quick
enough to run the full test set before the target machine is
obsolete. So far it is only a nice topic for a geek scifi story :-)
The situation is not dramatic: most "local tests" are skipped when you
run "optimized" tests. As I did not like some implications of
script_exclude_keys() I added an "optimization_level" setting in
nessusd.conf - default (old behaviour) is 3, but I set it to 2
instead, which only enforce script_require_ports() and
script_require_keys(). Lowering it to 1 is not recommended as the
useless local tests will be run (script_require_keys is not enforce
any more, only script_require_ports)
However, as we pile new tests up, scans are expected to run (a
little?) slower.
Removing old tests is definitely a bad idea, as I have seen machines
that had never been patched for more than 5 years connected to
Internet! Why they were not rooted remains a mystery...
I have come with several possibilities to reduce the overall scanning
time:
1. Cut down the number of banner matching plugins and replace them
with generic tests. Thus, we might even find unpublished flaws.
But:
- everything cannot be tested easily
- most security advisories are fuzzy and reproducing the bug is not
easy.
- generic tests are often dangerous and would be declared as
ACT_DESTRUCTIVE_ATTACK, i.e. disabled by people who run "safe checks".
IMHO, "safe checks" should not be used most of the time. Users ask for
99.99% uptime and ultra high reliability but they often do not need
it. Exceptions are few: medical life support systems (don't even scan
them!), big banking / money transfer systems (but you may have time
for play between two cut off), round the clock worldwide airplane
reservation systems...
2. Rewrite the NASL interpretor using a VM. According to gforth /
vmgen developers, such an interpretor might be 10 to 100 times quicker
than a simple "syntax tree walker" (just like the current NASL2).
But:
- some people say the speed gain is not that great, and definitely not
worth the trouble
- the current code is clean and flexible, a stack machine based
interpretor might not be.
- CPU is not that an issue but on huge network. On my point of view,
memory is (because I usually use an old laptop to scan a couple of
machines, maybe? :)
3. Try other "simple" NASL optimizations. I have implemented a cache
for include files (they are only parsed once for all plugins) in the
CVS head branch. Even if it saves only 1% of CPU, it is worth it
because the code is simple & safe (less than one page of C, two
exported functions)
Another idea would be to convert some very common NASL functions into
C so that they run quicker.
4. Make sure that all plugins do not perform redundant operations.
e.g. if a plugin need a banner, it should be grabbed from the KB, not
read on the network.
I guess that there are few ill designed plugins (maybe none).
Any comment? New ideas?
--
arboi@alussinan.org http://arboi.da.ru
NASL2 reference manual http://michel.arboi.free.fr/nasl2ref/