Mailing List Archive

I thik there might have been plans to add mysql backend to this
you might want to wait a couple of days to hear back before starting.

--
Michael Scheidell
Secnap Network Security, LLC
scheidell@secnap.net 1+(561) 368-9561
See updated IT Security News at http://www.fdma.com/

On Sat, Jan 05, 2002 at 12:35:57AM +0530, Rajkumar S wrote:
> Hi,
>
> I don't know if what I am proposing is of any practical use. I plan to
> add a database back end to nessus so that the current KB which is
> stored in a file is stored in a DB. Some more modifications that I
> plan are to get all the config options from a config file also, if
> available so that nessusd can be made an independent daemon with out
> the client, checking any given network from the config continously.
> The database can be queried by a web front end to produce reports
> which can give the summary of the problems found in any given time
> frame. This will transform nessusd to some thing similar to snort with
> DB back end (in the way the daemon runs and not in function)
>
> One use of this functionality is when security scanning is done
> remotely by an out source agency and the suites wants to see things like
> how many times the network was scanned and how many vuln was found
> out in a particular period of time etc...
>
> Is any work going on to change the NTP to some other protocol? I have
> one of my friend who is interested in taking up that work. He is
> looking at using the BEEP framework (www.beepcore.org) to build the
> replacement for NTP.
>
> Any feed back in this is most welcome, most importantly please tell me
> if this a really stupid idea :)
>
> Wish you all a Happy and Safe New Year!!!
>
> raj

I know that I am also in the minority, but I would love to see that come to
pass. I am currently using the commercial product ISS (http://www.iss.net) to
scan a very large number of hosts and have built a DB backend for that. It
isn't pretty and it doesn't always work. When I was designing it, I had an
idea that might be useful to those on this list. Perhaps some kind of backend
format could be defined that would make it possible to plug in different scanners
according to their strengths and weaknesses. The thoughts that I had on the
idea stemmed from there being an already established unique identifier for
each vulnerability (CVE). Since I didn't have time to flesh out any code or
give further substance to the idea, I cannot say what other advantages or
disadvantages would be.

Long and short of this is that I'd like to see it happen and might even be
contributing a patch or two if others think it is a good idea.

-alex

--
________________
Alex Lovell-Troy WWW: http://guacamole.cso.uiuc.edu
alovell@uiuc.edu PGP: http://guacamole.cso.uiuc.edu
WSG System Admin PHONE: (217) 265-0989
Asst. Linux Guru FAX: (217) 244-7089
________________

On Sat, 5 Jan 2002, Rajkumar S wrote:

> Hi,
>
> I don't know if what I am proposing is of any practical use. I plan to
> add a database back end to nessus so that the current KB which is
> stored in a file is stored in a DB. Some more modifications that I
> plan are to get all the config options from a config file also, if
> available so that nessusd can be made an independent daemon with out
> the client, checking any given network from the config continously.
> The database can be queried by a web front end to produce reports
> which can give the summary of the problems found in any given time
> frame. This will transform nessusd to some thing similar to snort with
> DB back end (in the way the daemon runs and not in function)

FWIW,

I've been working on something similar. A db that would be used to
store scan results from multiple servers. I could then wrap a web
front end around it an provided it as a method for local management to
view the reports. I have been planing on building in statistical
reports as well as filtering on type of vulnerability, and severity.

I have been using the xml reports as the base. Maybe that's not the
best way to proceed ??

-Alan

> One use of this functionality is when security scanning is done
> remotely by an out source agency and the suites wants to see things like
> how many times the network was scanned and how many vuln was found
> out in a particular period of time etc...
>
> Is any work going on to change the NTP to some other protocol? I have
> one of my friend who is interested in taking up that work. He is
> looking at using the BEEP framework (www.beepcore.org) to build the
> replacement for NTP.
>
> Any feed back in this is most welcome, most importantly please tell me
> if this a really stupid idea :)
>
> Wish you all a Happy and Safe New Year!!!
>
> raj
>


--
-----------------------------------------------------
Alan Pitts | E-Mail: amp@uu.net
UUNet Technologies | Ph: 614.723.4954
-----------------------------------------------------

Thanks for all your feedback. So after all it is not as bad an idea as
i imagined ;)

On Fri, 4 Jan 2002, Geoff Galitz wrote:

> I did some work in getting nessus to interface to MySQL, but not
> the way you outline below.

Can you share your idea with us. I do not have a real concrete idea as
of now, only a vague idea that we must come up with some DB format so
that security scanners can log their output into it.

> I know the nessus development team has talked about ideas to
> integrate nessus with MySQL, but I haven't heard if anything
> concrete.

I am ready to take up this work, with help from all of you.

> If you (or anyone else) is interested, let me know. I'll also see
> if I can forward any notes I may have made on getting this all to
> work together.

I am certainly interested in it and it will be great to share notes
with you.

raj

Hi,

Having decided to go on with the DB back end, I have started thinking
about the how part of it. Here are some of my ideas. please send in
your comments.

To use the DB mode nessus will have to be started in detached mode.
The place I see is the place where the scan results are written to the
email file. I plan to add the db write routine in place of the email
file write routine and when selecting the detached scan mode the user
can select whether emails should be send or the data should be
appended to the DB.

Though eventually the nessusd should be able to use a conf file to get
the params like the network to scan, time between the scan etc and
launch a detached scan to send periodic emails and/or add the entries
to the DB.

I also have to take care of the results of the previous scan so that
if the current scan do not differ from the previous scan then no db
write needs to be done. I figure this is some what different from the
KB that we have for nessus. If I understand correctly the KB is used
to avoid some test if it is already done some time before. What I am
thinking about is that all tests will be done during every scan and
it's result will be compared with the previous scan and if their are
any changes the result will be written to the DB. This will enable us
to produce the output like the down time of the hosts, How many hours
was the machine remain with a vulnerable version of the daemon etc..

Another issue is the design of the data base. I want it to be modular
in the sense that it should be able to accept data from any of the
security scanners. Can some one tell me if other packages generate
more data than nessus and if so what are they?

As I have said before these are just some of the
ideas that are floating around, and your comments are most welcome.

I am going through the code now and kind of figured out how the
detached stuff happens. Before the attack happens a file is written
with the email headers and during the attack the results are written
to the file and at the end it is send. Please correct me if I am
wrong. But so fat I have not able to figure out where/how the attack
results are written to the email file. Any help in explaining this
will be much appreciated.


raj

On Tue, Jan 08, 2002 at 01:23:29AM +0530, Rajkumar S wrote:
> Hi,
>
> Having decided to go on with the DB back end, I have started thinking
> about the how part of it. Here are some of my ideas. please send in
> your comments.
>
> To use the DB mode nessus will have to be started in detached mode.
> The place I see is the place where the scan results are written to the
> email file. I plan to add the db write routine in place of the email
> file write routine and when selecting the detached scan mode the user
> can select whether emails should be send or the data should be
> appended to the DB.

This seems like an un-necessary and inconvient restriction. I run my
nessus scans from cron, which allows the things you mention in the next
paragraph. Is there any reason not to just make a new output module
which writes to the database rather than producing a report (like html
or nsr) so that detached mode is not required. Another option is to parse
the output NSR/XML file....

>
> Though eventually the nessusd should be able to use a conf file to get
> the params like the network to scan, time between the scan etc and
> launch a detached scan to send periodic emails and/or add the entries
> to the DB.
>
> I also have to take care of the results of the previous scan so that
> if the current scan do not differ from the previous scan then no db
> write needs to be done. I figure this is some what different from the
> KB that we have for nessus. If I understand correctly the KB is used
> to avoid some test if it is already done some time before. What I am
> thinking about is that all tests will be done during every scan and
> it's result will be compared with the previous scan and if their are
> any changes the result will be written to the DB. This will enable us
> to produce the output like the down time of the hosts, How many hours
> was the machine remain with a vulnerable version of the daemon etc..

Humm, The problem with only registering differences is that you loose
assurances that the scan was run. Forgetting problems with fake data
being inserted, it would be nice to be able to check that a particular
plugin was run for scan1, scan2, scan3, etc... If you only log
differences then this capability will be lost. And no, you can't just
trust Nessus correctly run a scan every time.

> I am going through the code now and kind of figured out how the
> detached stuff happens. Before the attack happens a file is written
> with the email headers and during the attack the results are written
> to the file and at the end it is send. Please correct me if I am
> wrong. But so fat I have not able to figure out where/how the attack
> results are written to the email file. Any help in explaining this
> will be much appreciated.

This brings up a suggestion that I have for Nessus. Have the developers
considered spooling results to disk for all scans? Or using something
like Berkeley DB for the KB? The two advantages of this would be first,
lower memory usage, and second, partial results even if the scan blows
up. You may even be able to checkpoint scans, so that they can be
stoped and started later.

I am pushing Nessus a bit, and have experienced Out Of Memory conditions
before adding RAM to our Nessus box.

--
Devin Kowatch
devink@sdsc.edu

I've built a system in the past (not for Nessus) that builds tables dynamically & populates them with data... so for example you scan
10.10.10.10, and it creates a table called "RESULTS_10_10_10_10_02082002". It makes for a lot of tables, but by doing a 'show tables'
in the code you can easily find all the hosts/dates you have results for, and then the contents of those tables gives you the ability to do
diffs, keep for archives, etc.

Once the data is successfully stored, it's all just the user interface after that...

> > I also have to take care of the results of the previous scan so that
> > if the current scan do not differ from the previous scan then no db
> > write needs to be done. I figure this is some what different from the
> > KB that we have for nessus. If I understand correctly the KB is used
> > to avoid some test if it is already done some time before. What I am
> > thinking about is that all tests will be done during every scan and
> > it's result will be compared with the previous scan and if their are
> > any changes the result will be written to the DB. This will enable us
> > to produce the output like the down time of the hosts, How many hours
> > was the machine remain with a vulnerable version of the daemon etc..
>
> Humm, The problem with only registering differences is that you loose
> assurances that the scan was run. Forgetting problems with fake data
> being inserted, it would be nice to be able to check that a particular
> plugin was run for scan1, scan2, scan3, etc... If you only log
> differences then this capability will be lost. And no, you can't just
> trust Nessus correctly run a scan every time.


Chris Sullo
____________________________________________________
http://www.cirt.net/
Default Passwords, Ports, SSIDs & more

On Tue, 8 Jan 2002, Devin Kowatch wrote:

> This seems like an un-necessary and inconvient restriction. I run
> my nessus scans from cron, which allows the things you mention in
> the next paragraph.

I suggested this because this was mearly an extension of the email
scan. But if you are running from cron it may be difficult to write
only the difference to the DB.

> Is there any reason not to just make a new output module which
> writes to the database rather than producing a report (like html
> or nsr) so that detached mode is not required.

None. except that if detached scan is used then setting this under
cron is not required.

> Another option is to parse the output NSR/XML file....

Hm... Some thing I am doing already.


> Humm, The problem with only registering differences is that you
> loose assurances that the scan was run.

We can write the time a scan was finished in one table, whether it was
different or not along with the number of plugins the scan ran. But
the results go to another table.

> And no, you can't just trust Nessus correctly run a scan every
> time.

What happens now when the nessus fails on a scan and email scan is
used?

raj

On Tue, Jan 08, 2002 at 11:43:50AM -0800, Devin Kowatch wrote:
> > I am going through the code now and kind of figured out how the
> > detached stuff happens. Before the attack happens a file is written
> > with the email headers and during the attack the results are written
> > to the file and at the end it is send. Please correct me if I am
> > wrong. But so fat I have not able to figure out where/how the attack
> > results are written to the email file. Any help in explaining this
> > will be much appreciated.
>
> This brings up a suggestion that I have for Nessus. Have the developers
> considered spooling results to disk for all scans?

Yes, in 1.1.x. That's an issue I also met.

> Or using something
> like Berkeley DB for the KB? The two advantages of this would be first,
> lower memory usage, and second, partial results even if the scan blows
> up. You may even be able to checkpoint scans, so that they can be
> stoped and started later.

That needs to be improved, but this is basically done in 1.1.x. There
are some nasty issue to solve, but we can do that, technically speaking.


-- Renaud

On Wed, Jan 09, 2002 at 02:29:06AM +0530, Rajkumar S wrote:
> On Tue, 8 Jan 2002, Devin Kowatch wrote:
>
> > This seems like an un-necessary and inconvient restriction. I run
> > my nessus scans from cron, which allows the things you mention in
> > the next paragraph.
>
> I suggested this because this was mearly an extension of the email
> scan. But if you are running from cron it may be difficult to write
> only the difference to the DB.
As noted below this is not (in my mind) a problem.

>
> > Is there any reason not to just make a new output module which
> > writes to the database rather than producing a report (like html
> > or nsr) so that detached mode is not required.
>
> None. except that if detached scan is used then setting this under
> cron is not required.
But then it only works if you are doing a detached scan. Perhaps I am
mis-understanding your intent with this. If you are simply trying to
use the detached scan with a database backend ... then your ideas are
fine.

But if you want to build a general database backend onto Nessus I don't
think that detached scans are the place to do them.
>
> > Another option is to parse the output NSR/XML file....
>
> Hm... Some thing I am doing already.

Truthfully, this is, I think, The Right Answer. But then I also believe
that Nessus should only have one output format (XML) and pretty report
generation should be done external to Nessus. I believe that Renaud
considered this a while back, though I don't know what the final
decision was.

>
> > Humm, The problem with only registering differences is that you
> > loose assurances that the scan was run.
>
> We can write the time a scan was finished in one table, whether it was
> different or not along with the number of plugins the scan ran. But
> the results go to another table.

You would still have to record _which_ plugins ran. I doubt that I'm
alone in updating my plugins regularly (between scans) so the absolute
number may or may not be representitive.
>
> > And no, you can't just trust Nessus correctly run a scan every
> > time.
>
> What happens now when the nessus fails on a scan and email scan is
> used?

I considered using detached scans originally, but have since tossed the
idea. There are several reasons for this, including having too many
target lists (about 5 - 10 class C networks, many of which needed to be
split to /25 or /26 networks so that Nessus could complete a run). And
I only want scans to run twice a month, at specific times.

--
Devin Kowatch
devink@sdsc.edu

On Wed, 9 Jan 2002, Devin Kowatch wrote:

> But if you want to build a general database backend onto Nessus I
> don't think that detached scans are the place to do them.

Ok, then another choice will be to make the DB writing an optional
output module so that it can be selected with any mode of operation.

Their will be a check box in nessus to enable this. We can also make
provision for a DB only scan where all the other output modules are
disabled.

> But then I also believe that Nessus should only have one output
> format (XML)

Yes, I also believe that the NTP should also be XML based. Is their
any work being done in that direction?

> and pretty report generation should be done external to Nessus.

I guess this is the case already. nessusd sends the scan results back
and nessus is the one which does the pretty reporting.

> You would still have to record _which_ plugins ran. I doubt that
> I'm alone in updating my plugins regularly (between scans) so the
> absolute number may or may not be representitive.

Ok, understood.


raj