Mailing List Archive

On Jan 13, 2004, at 11:25 AM, Jeff Dell wrote:

I'll let Javier handle your questions.

> 4. Is there any talk about NESSUS_SQL merging in with NESSUS?

It's planned to be merged as an experimental feature in Nessus 2.1.x,
so really
soon.

-- Renaud

Re: [Database-devel] questions... [ In reply to ]

Jan 13, 2004, 11:24 AM

Post #3 of 18 (3014 views)

Jeff Dell wrote:

> I have been playing with NESSUS_SQL and I have a few questions:
>
> 1. In he README.txt file it says to use import-nbe.pl to import old nessus
> reports into the database. I can't seem to find it. It is not located where
> it says.

It's available in the nessus-tools CVS directory, branch NESSUS_SQL
http://cvsweb.nessus.org/cgi-bin/cvsweb.cgi/nessus-tools/, more
precisely in
http://cvsweb.nessus.org/cgi-bin/cvsweb.cgi/nessus-tools/nessus-extract/?only_with_tag=NESSUS_SQL

>
> 2. How do I start nessus so it utilizes the database?

If you have configured it properly (i.e. 'configure --with-mysql') you
have to configure nessusd.conf properly with:

db_user = USER_TO_CONNECT_WITH
db_pass = PASSWORD_TO_USE_FOR_CONNECTION
db_host = DATABASE_HOST
db_database = DATABASE_NAME (default name is 'Nessus_Scans')
db_port = REMOTE_DATABASE_PORT (default depends on the DB server)
db_make_tables = yes|no
db_compress = yes|no

This step is not in the readme I will add it there, sorry.
>
> 3. When extracting the plugins with nessus -qSp it uses a table called
> plugins, when using the script nessus-extract.pl, it uses the table
> nessus-plugindata. Which one is really used by nessus?

Ummm... are you using the nessus_extract.pl from the NESSUS_SQL
branch? The one from the NESSUS_SQL branch _does_ use the 'PLUGINS'
table. Since you do not see import_nbe either that might be the reason
you are seeing those differences.

>
> 4. Is there any talk about NESSUS_SQL merging in with NESSUS?

As Renaud said, it will probably be added soon (but disabled per
default since it needs wider testing).

Regards

Javi

RE: [Database-devel] questions... [ In reply to ]

Jan 13, 2004, 3:46 PM

Post #4 of 18 (3022 views)

1. I would not recommend using the import-nbe.pl script. The NESSUS_SQL code captures a lot of state information (such as knowledgebase, execution time, status, etc) This information is not captured in the nbe output. Going this route will lead to incomplete data. I also believe that the import-nbe.pl scripts do not work with the NESSUS_SQL code as is.

With regard to #2, the db_compress feature is not currently supported.

With regard to #3, I have an updated version of nessus-extract.pl that works with the code in NESSUS_SQL. I have also updated some of the regular expressions to be more robust and case insensitive.

On a more general note, the tables have been renamed to all lowercase (it is quite a pain to type uppercase column names in the SQL command line)

I also have a Perl script that generates a lot of different reports from the database. These include standard reports (similar to nessus html save), executive summary reports, flat text files and Excel spread sheets (thanks to Spreadsheet::WriteExcel). The Perl reporting code also auto remediates. This means that if an issue was detected on scan #1 and then checked for but not found on scan #6, this issue will be included in the report as fixed. This is a time consuming feature but quite worthwhile.

I also have updated SQL code with speed improvements and support for auto magically creating inno db tables for MySQL. Postgre support should be soon now.

I will try and get the new code to Javier some time in the next week.

-Cory

-----Original Message-----
From: Javier Fernandez-Sanguino [mailto:jfernandez@germinus.com]
Sent: Tuesday, January 13, 2004 11:25 AM
To: nessus-devel@list.nessus.org
Subject: Re: [Nessus-devel] [Database-devel] questions...

Jeff Dell wrote:

> I have been playing with NESSUS_SQL and I have a few questions:
>
> 1. In he README.txt file it says to use import-nbe.pl to import old nessus
> reports into the database. I can't seem to find it. It is not located where
> it says.

It's available in the nessus-tools CVS directory, branch NESSUS_SQL
http://cvsweb.nessus.org/cgi-bin/cvsweb.cgi/nessus-tools/, more
precisely in
http://cvsweb.nessus.org/cgi-bin/cvsweb.cgi/nessus-tools/nessus-extract/?only_with_tag=NESSUS_SQL

>
> 2. How do I start nessus so it utilizes the database?

If you have configured it properly (i.e. 'configure --with-mysql') you
have to configure nessusd.conf properly with:

db_user = USER_TO_CONNECT_WITH
db_pass = PASSWORD_TO_USE_FOR_CONNECTION
db_host = DATABASE_HOST
db_database = DATABASE_NAME (default name is 'Nessus_Scans')
db_port = REMOTE_DATABASE_PORT (default depends on the DB server)
db_make_tables = yes|no
db_compress = yes|no

This step is not in the readme I will add it there, sorry.
>
> 3. When extracting the plugins with nessus -qSp it uses a table called
> plugins, when using the script nessus-extract.pl, it uses the table
> nessus-plugindata. Which one is really used by nessus?

Ummm... are you using the nessus_extract.pl from the NESSUS_SQL
branch? The one from the NESSUS_SQL branch _does_ use the 'PLUGINS'
table. Since you do not see import_nbe either that might be the reason
you are seeing those differences.

>
> 4. Is there any talk about NESSUS_SQL merging in with NESSUS?

As Renaud said, it will probably be added soon (but disabled per
default since it needs wider testing).

Regards

Javi
_______________________________________________
Nessus-devel mailing list
Nessus-devel@list.nessus.org
http://mail.nessus.org/mailman/listinfo/nessus-devel

[INFO] -- Access Manager:
This transmission may contain information that is privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. A2

RE: [Database-devel] questions... [ In reply to ]

Jan 14, 2004, 10:34 AM

Post #5 of 18 (3030 views)

Thanks for all the info... With some minor tweaking, I was able to get
almost everything to work. From looking at the source, It looks like the
table DETECTEDVULNERABILITY that was created with the nessus_db_schema.mysql
should be really called vulnerability. I added this table and changed all of
the other table names to lowercase, but DETECTEDVULNERABILITY or
vulnerability are still not getting populated. It looks like most of the
other tables are getting populated though. I checked the nessusd.dump and
nessusd.messages and I am not getting any errors or messages regarding this
problem (I did get messages when I had uppercase tables names). Does anyone
have any hits?

Thanks,
Jeff

-----Original Message-----
From: nessus-devel-bounces@list.nessus.org
[mailto:nessus-devel-bounces@list.nessus.org] On Behalf Of Marsh, Cory
Sent: Tuesday, January 13, 2004 5:47 PM
To: Javier Fernandez-Sanguino
Cc: nessus-devel@list.nessus.org
Subject: RE: [Nessus-devel] [Database-devel] questions...

1. I would not recommend using the import-nbe.pl script. The NESSUS_SQL
code captures a lot of state information (such as knowledgebase, execution
time, status, etc) This information is not captured in the nbe output.
Going this route will lead to incomplete data. I also believe that the
import-nbe.pl scripts do not work with the NESSUS_SQL code as is.

With regard to #2, the db_compress feature is not currently supported.

With regard to #3, I have an updated version of nessus-extract.pl that works
with the code in NESSUS_SQL. I have also updated some of the regular
expressions to be more robust and case insensitive.

On a more general note, the tables have been renamed to all lowercase (it is
quite a pain to type uppercase column names in the SQL command line)

I also have a Perl script that generates a lot of different reports from the
database. These include standard reports (similar to nessus html save),
executive summary reports, flat text files and Excel spread sheets (thanks
to Spreadsheet::WriteExcel). The Perl reporting code also auto remediates.
This means that if an issue was detected on scan #1 and then checked for but
not found on scan #6, this issue will be included in the report as fixed.
This is a time consuming feature but quite worthwhile.

I also have updated SQL code with speed improvements and support for auto
magically creating inno db tables for MySQL. Postgre support should be soon
now.

I will try and get the new code to Javier some time in the next week.

-Cory

RE: [Database-devel] questions... [ In reply to ]

Jan 14, 2004, 11:42 AM

Post #6 of 18 (3020 views)

That's odd about the vulnerability table. You can try this:

create a new database from scratch. in the nessusd.conf file set the option: 'db_make_tables = yes'. This will tell nessus to create the tables you need with the proper case (lowercase). This may help you. The code is written such that failed SQL queries should generate log messages in nessusd.messages (I think that's the log file). So if the statements are failing they should be listed there. If you still are scanning and getting vulnerabilities that are not showing up in the database, the only thing I can think of is that the code is not being called.

nessusd/attack.c lines 512,513 should call db_update_host_scan() and db_dump_kb() respectively. You should be able to see the status in the hostsession table set to "Finished" when this runs. You should also see the entire knowledgebase in the knowledgebase table. The db_dump_kb() function reads the knowledgebase for keys 'SentData/TYPE/' where TYPE can be any of the set { 'NOTE', 'INFO', 'HOLE' } and inserts these knowledgebase values into the vulnerability table. db_dump_kb() also updates the service table with open ports, and takes keys of the form 'Success/PLUGINID' and sets the appropriate executedplugin status to 'Success'; If you see either of these updated status messages in the db, you know the db_dump_kb() function is running and you should see results in the vulnerability table.

A few other notes about db_dump_kb(), it currently replaces values for the key 'SMB/password' to 8 stars '********' before inserting into the database. Keys that begin with '/tmp' are not inserted. db_dump_kb() also checks for the keys 'HOST/name' and 'HOST/mac' and hard sets the host table to these entries. I use this feature to fill out the MAC address of windows systems that are not on my broadcast domain by setting the knowledgebase key 'HOST/mac' to the MAC address returned in the plugin 'netbios_name_get.nasl'. Everywhere you see 'set_kb_item(name:'SMB/name', value:name);' add the line: 'set_kb_item(name:'HOST/name', value:name);'. Also, right BEFORE you see the line: 'if(adapter_nbame == "0x00 0x00 0x00 0x00 0x00 ") {' (line 248 in my script), add the line: 'set_kb_item(name:'HOST/name', value:mac_address);'. Adding these two elements to the netbios_name_get.nasl script will ensure that the host table reflects the computer name for the system, and the MAC address. the db_dump_kb() function also uses these values to determine if it has seen a host before. If the nessus server is unsure if the host it is scanning exists in the host table or not, it will create a new host. If any of the plugins set the 'HOST/name' or 'HOST/mac' values, the db_dump_kb() function that runs after the scan will search the host table to matching entries and update the new host data to map to the existing host in the database. Obviously the MAC address is treated as the ultimate determining factor if two hosts are in fact the same. This happens entirely automatically, but it is a good idea to change to netbios_name_get.nasl script to help NESSUS_SQL in determining exactly what host it is scanning.

Regards, Cory

-----Original Message-----
From: Jeff Dell [mailto:jdell@activeworx.com]
Sent: Wednesday, January 14, 2004 10:35 AM
To: Marsh, Cory; 'Javier Fernandez-Sanguino'
Cc: nessus-devel@list.nessus.org
Subject: RE: [Nessus-devel] [Database-devel] questions...

Thanks for all the info... With some minor tweaking, I was able to get
almost everything to work. From looking at the source, It looks like the
table DETECTEDVULNERABILITY that was created with the nessus_db_schema.mysql
should be really called vulnerability. I added this table and changed all of
the other table names to lowercase, but DETECTEDVULNERABILITY or
vulnerability are still not getting populated. It looks like most of the
other tables are getting populated though. I checked the nessusd.dump and
nessusd.messages and I am not getting any errors or messages regarding this
problem (I did get messages when I had uppercase tables names). Does anyone
have any hits?

Thanks,
Jeff

-----Original Message-----
From: nessus-devel-bounces@list.nessus.org
[mailto:nessus-devel-bounces@list.nessus.org] On Behalf Of Marsh, Cory
Sent: Tuesday, January 13, 2004 5:47 PM
To: Javier Fernandez-Sanguino
Cc: nessus-devel@list.nessus.org
Subject: RE: [Nessus-devel] [Database-devel] questions...

1. I would not recommend using the import-nbe.pl script. The NESSUS_SQL
code captures a lot of state information (such as knowledgebase, execution
time, status, etc) This information is not captured in the nbe output.
Going this route will lead to incomplete data. I also believe that the
import-nbe.pl scripts do not work with the NESSUS_SQL code as is.

With regard to #2, the db_compress feature is not currently supported.

With regard to #3, I have an updated version of nessus-extract.pl that works
with the code in NESSUS_SQL. I have also updated some of the regular
expressions to be more robust and case insensitive.

On a more general note, the tables have been renamed to all lowercase (it is
quite a pain to type uppercase column names in the SQL command line)

I also have a Perl script that generates a lot of different reports from the
database. These include standard reports (similar to nessus html save),
executive summary reports, flat text files and Excel spread sheets (thanks
to Spreadsheet::WriteExcel). The Perl reporting code also auto remediates.
This means that if an issue was detected on scan #1 and then checked for but
not found on scan #6, this issue will be included in the report as fixed.
This is a time consuming feature but quite worthwhile.

I also have updated SQL code with speed improvements and support for auto
magically creating inno db tables for MySQL. Postgre support should be soon
now.

I will try and get the new code to Javier some time in the next week.

-Cory

_______________________________________________
Nessus-devel mailing list
Nessus-devel@list.nessus.org
http://mail.nessus.org/mailman/listinfo/nessus-devel

This transmission may contain information that is privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. A1.

RE: [Database-devel] questions... [ In reply to ]

Jan 14, 2004, 2:04 PM

Post #7 of 18 (3024 views)

I found the problem I am having... It looks like I am getting a SIGSEGV
error on line 802 of save_db.c when using nessuswx and not storing anything
in the knowledgebase or vulnerability tables. It worked fine with the
standard nessus client. The tables userlist and usersession are still not
being populated though.. I didn't find any code for this either.. Did I miss
something or has it not been implemented yet?

One last question... There are a few tables in the perl script that are not
created with nessusd. Have they just not been implemented yet? Or are they
part of an old schema?

Thanks for all the help,

Jeff

-----Original Message-----
From: Marsh, Cory [mailto:CMarsh@idahopower.com]
Sent: Wednesday, January 14, 2004 1:43 PM
To: Jeff Dell; Javier Fernandez-Sanguino
Cc: nessus-devel@list.nessus.org
Subject: RE: [Nessus-devel] [Database-devel] questions...

That's odd about the vulnerability table. You can try this:

create a new database from scratch. in the nessusd.conf file set the
option: 'db_make_tables = yes'. This will tell nessus to create the tables
you need with the proper case (lowercase). This may help you. The code is
written such that failed SQL queries should generate log messages in
nessusd.messages (I think that's the log file). So if the statements are
failing they should be listed there. If you still are scanning and getting
vulnerabilities that are not showing up in the database, the only thing I
can think of is that the code is not being called.

nessusd/attack.c lines 512,513 should call db_update_host_scan() and
db_dump_kb() respectively. You should be able to see the status in the
hostsession table set to "Finished" when this runs. You should also see the
entire knowledgebase in the knowledgebase table. The db_dump_kb() function
reads the knowledgebase for keys 'SentData/TYPE/' where TYPE can be any of
the set { 'NOTE', 'INFO', 'HOLE' } and inserts these knowledgebase values
into the vulnerability table. db_dump_kb() also updates the service table
with open ports, and takes keys of the form 'Success/PLUGINID' and sets the
appropriate executedplugin status to 'Success'; If you see either of these
updated status messages in the db, you know the db_dump_kb() function is
running and you should see results in the vulnerability table.

A few other notes about db_dump_kb(), it currently replaces values for the
key 'SMB/password' to 8 stars '********' before inserting into the database.
Keys that begin with '/tmp' are not inserted. db_dump_kb() also checks for
the keys 'HOST/name' and 'HOST/mac' and hard sets the host table to these
entries. I use this feature to fill out the MAC address of windows systems
that are not on my broadcast domain by setting the knowledgebase key
'HOST/mac' to the MAC address returned in the plugin
'netbios_name_get.nasl'. Everywhere you see 'set_kb_item(name:'SMB/name',
value:name);' add the line: 'set_kb_item(name:'HOST/name', value:name);'.
Also, right BEFORE you see the line: 'if(adapter_nbame == "0x00 0x00 0x00
0x00 0x00 ") {' (line 248 in my script), add the line:
'set_kb_item(name:'HOST/name', value:mac_address);'. Adding these two
elements to the netbios_name_get.nasl script will ensure that the host table
reflects the computer name for the system, and the MAC address. the
db_dump_kb() function also uses these values to determine if it has seen a
host before. If the nessus server is unsure if the host it is scanning
exists in the host table or not, it will create a new host. If any of the
plugins set the 'HOST/name' or 'HOST/mac' values, the db_dump_kb() function
that runs after the scan will search the host table to matching entries and
update the new host data to map to the existing host in the database.
Obviously the MAC address is treated as the ultimate determining factor if
two hosts are in fact the same. This happens entirely automatically, but it
is a good idea to change to netbios_name_get.nasl script to help NESSUS_SQL
in determining exactly what host it is scanning.

Regards, Cory

Re: [Database-devel] questions... [ In reply to ]

Jan 15, 2004, 1:44 AM

Post #8 of 18 (3029 views)

Jeff Dell wrote:
> Thanks for all the info... With some minor tweaking, I was able to get
> almost everything to work. From looking at the source, It looks like the
> table DETECTEDVULNERABILITY that was created with the nessus_db_schema.mysql
> should be really called vulnerability. I added this table and changed all of
> the other table names to lowercase, but DETECTEDVULNERABILITY or
(...)

Correct, there are some minor discrepancies between the code and the
schema. I've documented them in the latest README, I will make updates
to both the .dia file and the SQL schemas under the documentation
directory soon.

Regards

Javi

Re: [Database-devel] questions... [ In reply to ]

Jan 15, 2004, 2:32 AM

Post #9 of 18 (3025 views)

Jeff Dell wrote:

> I found the problem I am having... It looks like I am getting a SIGSEGV
> error on line 802 of save_db.c when using nessuswx and not storing anything
> in the knowledgebase or vulnerability tables. It worked fine with the

That's strange.... it says:

798 /* iterate through the KB */
799 while ( key->next != NULL ) {
800
801 /* this is vulnerability data sent to the client */
802 if ( strncmp( key->name, "SentData", 8 ) == 0 ) {
803 int plugid = 0;
804 int risk = 0;
805 int execution_id = 0;

Maybe that SIGSEGV happens because key == NULL ? Can you change line
799 to
while ( key != NULL && key->next != NULL ) {

And see if still segfaults?

> standard nessus client. The tables userlist and usersession are still not
> being populated though.. I didn't find any code for this either.. Did I miss
> something or has it not been implemented yet?

It has not been implemented yet.

>
> One last question... There are a few tables in the perl script that are not
> created with nessusd. Have they just not been implemented yet? Or are they
> part of an old schema?
>

Those are there in order to be able to generate more detailed reports.
Currently all the vulnerability information is taken from what
Nessus provides in the KB, but there's certainly more vulnerability
information in the plugins themselves.

Also the schema has been thought out in order to provide some way to
maintain a list of older plugins/revisions. This yas _not_ yet been
implemented.

If you take a look at the current code the pluginid is not extracted
from the 'Plugin' table but, instead, is based on the Nessus
script_id. Making it able to extract the current pluginid based on the
'Plugin' table would be more complex but it would introduce a way to
generate the _exact_ report that was generated, say, a year ago when a
scan was run.

This covers the case of running a scan, storing the information in the
database, updating the plugins and having the need to restore the
report as it was before the plugins were updated. It also provides a
way to be able to determine if new vulnerabilities are being detected
because they are _new_ in the scanned host or because the plugin has
been updated or a _new_ plugin is introduced (when updating). Consider
the following (alternative) case:

0.- Update the Nessus plugins
1.- Scan host A, returns report with N vulnerabilities.
(time passes by....)
2.- Update the Nessus plugins
3.- Scan host A, returns a new report with M vulnerabilities.

Now, if N > M this can be because:

a) The host has changed and now is more vulnerable than it was before
b) The plugins have changed (bug fixes) and are now able to properly
detect a vulnerability that _were_ present before and were known when
it was first scanned.
c) There are new plugins that are now able to detect a vulnerability
that _was_ present before but was unknown when it was first scanned.

A security team would act differently, presumably, in either case. a)
might mean you have to determine what has changed in the host. b)
means you have to schedule a patch fix for the host, c) means you have
to schedule a (probably) more urgen patch fix for the host (because
the vulnerability has been open for more time that it might be
reasonable, for example)

There's currently no way to do this, but it's a very important feature
for enterprise scans and that's why its designed in the schema.

Hopefully this rant (sorry :-) has explained why there might be
significant differences between the documented/designed schema and the
current implementation. It will take some time for the implementation
to fill all the gaps, and more code will need to be developed to
implement the design or the design will need to be change if it's
determined that it's good in theory but not so in practice (cannot be
implemented, leads to scans that are too heavy in the database, the
SQL queries are way too complex, etc.)

Regards

Javi

RE: [Database-devel] questions... [ In reply to ]

Jan 15, 2004, 6:30 AM

Post #10 of 18 (3019 views)

>Maybe that SIGSEGV happens because key == NULL ? Can you change line
>799 to
> while ( key != NULL && key->next != NULL ) {
>
>And see if still segfaults?

I made the change and It is still segfaulting.

>Hopefully this rant (sorry :-) has explained why there might be
>significant differences between the documented/designed schema and the

It is actually really nice to hear more about the Database development that
is going on. There is so little information about it out there and I think
it is a really great feature.

Regards,

Jeff

Re: [Database-devel] questions... [ In reply to ]

rrich at gstisecurity

Jan 15, 2004, 7:47 AM

Post #11 of 18 (3018 views)

I've been following this thread with interest and have been
building/testing the NESSUS_SQL code.

I do have a question that may relate to Jeff's problem. If i build
nessusd out of NESSUS_SQL, do i have to use the nessus client from that
same codebase? The system i'm building nessusd on has some gtk issues,
so i've been using both the gtk client and windows (NessusWX) client
from different systems running code from HEAD. While i haven't seen any
segfaults as indicated by Jeff below, i also have not seen any data in
the knowledgebase table.

Thoughts?

Thanks

Bob

Jeff Dell wrote:

>
>
>
>>Maybe that SIGSEGV happens because key == NULL ? Can you change line
>>799 to
>> while ( key != NULL && key->next != NULL ) {
>>
>>And see if still segfaults?
>>
>>
>
>I made the change and It is still segfaulting.
>
>
>
>>Hopefully this rant (sorry :-) has explained why there might be
>>significant differences between the documented/designed schema and the
>>
>>
>
>It is actually really nice to hear more about the Database development that
>is going on. There is so little information about it out there and I think
>it is a really great feature.
>
>Regards,
>
>Jeff
>
>_______________________________________________
>Nessus-devel mailing list
>Nessus-devel@list.nessus.org
>http://mail.nessus.org/mailman/listinfo/nessus-devel
>
>

Re: [Database-devel] questions... [ In reply to ]

Jan 15, 2004, 9:47 AM

Post #12 of 18 (3026 views)

Robert Rich wrote:

> I've been following this thread with interest and have been
> building/testing the NESSUS_SQL code.
> I do have a question that may relate to Jeff's problem. If i build
> nessusd out of NESSUS_SQL, do i have to use the nessus client from that
> same codebase? The system i'm building nessusd on has some gtk issues,
> so i've been using both the gtk client and windows (NessusWX) client
> from different systems running code from HEAD. While i haven't seen any
> segfaults as indicated by Jeff below, i also have not seen any data in
> the knowledgebase table.
> Thoughts?
>

There's really no reason why you shouldn't be able to use a different
client. Client/server communication is standarised and it should not
be affected by the SQL code. We will try to add some debug information
in the code in order to see what's going on.

Do you see if there is any database communication at all? (snooping
the loopback interface for example) Do you see if the host / session
/hostsession tables are being updated when you run a scan?

As a matter of fact you should be albe to see information from the
scan in real time when you are using the database code, since the
tables are updated when the session starts/ends, while the host is
being scanned, etc.

Regards

Javi

Re: [Database-devel] questions... [ In reply to ]

rrich at gstisecurity

Jan 15, 2004, 10:10 AM

Post #13 of 18 (3030 views)

There is definitely database communication. Based on some previous
discussion, i created the db but left it empty and simply allowed
nessusd to create the schema.

After three scans (the last one still running), this is the number of
rows in each table:

executedplugin: 9476
host: 15
hostsession: 54
knowledgebase: 0
service: 0
session: 3
userlist: 0
usersession: 0
vulnerability: 0

Your last point related to 'realtime' status is part of what makes this
so interesting to me. I've been watching the tables grow as time
progresses. I do see the following in nessus.messages:

$ grep kb nessusd.messages | tail -20 | head -4
[Thu Jan 15 09:30:56 2004][1480] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.239
[Thu Jan 15 09:30:58 2004][1484] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.240
[Thu Jan 15 09:31:00 2004][1486] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.241
[Thu Jan 15 09:31:02 2004][1487] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.242

Here were my configure args from the build of nessus-core

$ grep CONFIGURE_ARGS nessus.tmpl
CONFIGURE_ARGS = --prefix=/export/home/rrich/local --enable-release
--disable-gtk --enable-save-sessions --enable-save-kb
--with-mysql=/usr/local/mysql --with-x

Is it possible the --enable-save-kb is conflicting?

Javier Fernandez-Sanguino wrote:

> Robert Rich wrote:
>
>> I've been following this thread with interest and have been
>> building/testing the NESSUS_SQL code.
>> I do have a question that may relate to Jeff's problem. If i build
>> nessusd out of NESSUS_SQL, do i have to use the nessus client from
>> that same codebase? The system i'm building nessusd on has some gtk
>> issues, so i've been using both the gtk client and windows (NessusWX)
>> client from different systems running code from HEAD. While i
>> haven't seen any segfaults as indicated by Jeff below, i also have
>> not seen any data in the knowledgebase table.
>> Thoughts?
>>
>
> There's really no reason why you shouldn't be able to use a different
> client. Client/server communication is standarised and it should not
> be affected by the SQL code. We will try to add some debug information
> in the code in order to see what's going on.
>
> Do you see if there is any database communication at all? (snooping
> the loopback interface for example) Do you see if the host / session
> /hostsession tables are being updated when you run a scan?
>
> As a matter of fact you should be albe to see information from the
> scan in real time when you are using the database code, since the
> tables are updated when the session starts/ends, while the host is
> being scanned, etc.
>
> Regards
>
> Javi

RE: [Database-devel] questions... [ In reply to ]

Jan 15, 2004, 12:17 PM

Post #14 of 18 (3015 views)

NESSUS_SQL has *NOT* been tested with the knowledgebase code. I am going to guess that this is your problem. If the knowledgebase saving feature deletes the knowledgebase AFTER saving it and BEFORE the DB code is called, you will see nothing in the knowledgebase OR the vulnerability table. I recommend recompiling your nessusd WITHOUT --enable-save-kb. If both the SQL save AND a static file knowledgebase save is a required (or useful) feature, we may want to look at doing SQL reads to get the data for SQL users, or modifying the save_knowledgebase code to support the DB. I see both as valid options and renaud may have more input.

-Cory

-----Original Message-----
From: Robert Rich [mailto:rrich@gstisecurity.com]
Sent: Thursday, January 15, 2004 10:11 AM
To: Javier Fernandez-Sanguino
Cc: Jeff Dell; Marsh, Cory; nessus-devel@list.nessus.org
Subject: Re: [Nessus-devel] [Database-devel] questions...

There is definitely database communication. Based on some previous
discussion, i created the db but left it empty and simply allowed
nessusd to create the schema.

After three scans (the last one still running), this is the number of
rows in each table:

executedplugin: 9476
host: 15
hostsession: 54
knowledgebase: 0
service: 0
session: 3
userlist: 0
usersession: 0
vulnerability: 0

Your last point related to 'realtime' status is part of what makes this
so interesting to me. I've been watching the tables grow as time
progresses. I do see the following in nessus.messages:

$ grep kb nessusd.messages | tail -20 | head -4
[Thu Jan 15 09:30:56 2004][1480] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.239
[Thu Jan 15 09:30:58 2004][1484] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.240
[Thu Jan 15 09:31:00 2004][1486] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.241
[Thu Jan 15 09:31:02 2004][1487] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.242

Here were my configure args from the build of nessus-core

$ grep CONFIGURE_ARGS nessus.tmpl
CONFIGURE_ARGS = --prefix=/export/home/rrich/local --enable-release
--disable-gtk --enable-save-sessions --enable-save-kb
--with-mysql=/usr/local/mysql --with-x

Is it possible the --enable-save-kb is conflicting?

Javier Fernandez-Sanguino wrote:

> Robert Rich wrote:
>
>> I've been following this thread with interest and have been
>> building/testing the NESSUS_SQL code.
>> I do have a question that may relate to Jeff's problem. If i build
>> nessusd out of NESSUS_SQL, do i have to use the nessus client from
>> that same codebase? The system i'm building nessusd on has some gtk
>> issues, so i've been using both the gtk client and windows (NessusWX)
>> client from different systems running code from HEAD. While i
>> haven't seen any segfaults as indicated by Jeff below, i also have
>> not seen any data in the knowledgebase table.
>> Thoughts?
>>
>
> There's really no reason why you shouldn't be able to use a different
> client. Client/server communication is standarised and it should not
> be affected by the SQL code. We will try to add some debug information
> in the code in order to see what's going on.
>
> Do you see if there is any database communication at all? (snooping
> the loopback interface for example) Do you see if the host / session
> /hostsession tables are being updated when you run a scan?
>
> As a matter of fact you should be albe to see information from the
> scan in real time when you are using the database code, since the
> tables are updated when the session starts/ends, while the host is
> being scanned, etc.
>
> Regards
>
> Javi

This transmission may contain information that is privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. A1.

RE: [Database-devel] questions... [ In reply to ]

Jan 15, 2004, 12:33 PM

Post #15 of 18 (3023 views)

I personally feel that being able to report to the database on knowledgebase
information would be a really nice feature.

As far as the segfault that I was getting... I was seeing it in the
nessusd.messages file after the scan completed and it was a few lines up
from the bottom. Perform a scan with NessusWX and tail the file. When it
tries to populate the knowledgebase and vulnerability tables after the scan,
I get a message that says "[time] [pid] SIGSEGV occurred !".

Jeff

-----Original Message-----
From: nessus-devel-bounces@list.nessus.org
[mailto:nessus-devel-bounces@list.nessus.org] On Behalf Of Marsh, Cory
Sent: Thursday, January 15, 2004 2:17 PM
To: nessus-devel@list.nessus.org
Subject: RE: [Nessus-devel] [Database-devel] questions...

NESSUS_SQL has *NOT* been tested with the knowledgebase code. I am going to
guess that this is your problem. If the knowledgebase saving feature
deletes the knowledgebase AFTER saving it and BEFORE the DB code is called,
you will see nothing in the knowledgebase OR the vulnerability table. I
recommend recompiling your nessusd WITHOUT --enable-save-kb. If both the
SQL save AND a static file knowledgebase save is a required (or useful)
feature, we may want to look at doing SQL reads to get the data for SQL
users, or modifying the save_knowledgebase code to support the DB. I see
both as valid options and renaud may have more input.

-Cory

-----Original Message-----
From: Robert Rich [mailto:rrich@gstisecurity.com]
Sent: Thursday, January 15, 2004 10:11 AM
To: Javier Fernandez-Sanguino
Cc: Jeff Dell; Marsh, Cory; nessus-devel@list.nessus.org
Subject: Re: [Nessus-devel] [Database-devel] questions...

There is definitely database communication. Based on some previous
discussion, i created the db but left it empty and simply allowed
nessusd to create the schema.

After three scans (the last one still running), this is the number of
rows in each table:

executedplugin: 9476
host: 15
hostsession: 54
knowledgebase: 0
service: 0
session: 3
userlist: 0
usersession: 0
vulnerability: 0

Your last point related to 'realtime' status is part of what makes this
so interesting to me. I've been watching the tables grow as time
progresses. I do see the following in nessus.messages:

$ grep kb nessusd.messages | tail -20 | head -4
[Thu Jan 15 09:30:56 2004][1480] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.239
[Thu Jan 15 09:30:58 2004][1484] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.240
[Thu Jan 15 09:31:00 2004][1486] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.241
[Thu Jan 15 09:31:02 2004][1487] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.242

Here were my configure args from the build of nessus-core

$ grep CONFIGURE_ARGS nessus.tmpl
CONFIGURE_ARGS = --prefix=/export/home/rrich/local --enable-release
--disable-gtk --enable-save-sessions --enable-save-kb
--with-mysql=/usr/local/mysql --with-x

Is it possible the --enable-save-kb is conflicting?

Javier Fernandez-Sanguino wrote:

> Robert Rich wrote:
>
>> I've been following this thread with interest and have been
>> building/testing the NESSUS_SQL code.
>> I do have a question that may relate to Jeff's problem. If i build
>> nessusd out of NESSUS_SQL, do i have to use the nessus client from
>> that same codebase? The system i'm building nessusd on has some gtk
>> issues, so i've been using both the gtk client and windows (NessusWX)
>> client from different systems running code from HEAD. While i
>> haven't seen any segfaults as indicated by Jeff below, i also have
>> not seen any data in the knowledgebase table.
>> Thoughts?
>>
>
> There's really no reason why you shouldn't be able to use a different
> client. Client/server communication is standarised and it should not
> be affected by the SQL code. We will try to add some debug information
> in the code in order to see what's going on.
>
> Do you see if there is any database communication at all? (snooping
> the loopback interface for example) Do you see if the host / session
> /hostsession tables are being updated when you run a scan?
>
> As a matter of fact you should be albe to see information from the
> scan in real time when you are using the database code, since the
> tables are updated when the session starts/ends, while the host is
> being scanned, etc.
>
> Regards
>
> Javi

This transmission may contain information that is privileged, confidential
and/or exempt from disclosure under applicable law. If you are not the
intended recipient, you are hereby notified that any disclosure, copying,
distribution, or use of the information contained herein (including any
reliance thereon) is STRICTLY PROHIBITED. If you received this transmission
in error, please immediately contact the sender and destroy the material in
its entirety, whether in electronic or hard copy format. Thank you. A1.

_______________________________________________
Nessus-devel mailing list
Nessus-devel@list.nessus.org
http://mail.nessus.org/mailman/listinfo/nessus-devel

RE: [Database-devel] questions... [ In reply to ]

Jan 15, 2004, 12:46 PM

Post #16 of 18 (3023 views)

Perhaps it wasn't clear that I was referring to the knowledgebase saving feature not being tested with the sql code. I was not saying that the knowledgebase is not saved to the database. This is the default case.

-----Original Message-----
From: Jeff Dell [mailto:jdell@activeworx.com]
Sent: Thursday, January 15, 2004 12:33 PM
To: nessus-devel@list.nessus.org
Subject: RE: [Nessus-devel] [Database-devel] questions...

I personally feel that being able to report to the database on knowledgebase
information would be a really nice feature.

As far as the segfault that I was getting... I was seeing it in the
nessusd.messages file after the scan completed and it was a few lines up
from the bottom. Perform a scan with NessusWX and tail the file. When it
tries to populate the knowledgebase and vulnerability tables after the scan,
I get a message that says "[time] [pid] SIGSEGV occurred !".

Jeff

-----Original Message-----
From: nessus-devel-bounces@list.nessus.org
[mailto:nessus-devel-bounces@list.nessus.org] On Behalf Of Marsh, Cory
Sent: Thursday, January 15, 2004 2:17 PM
To: nessus-devel@list.nessus.org
Subject: RE: [Nessus-devel] [Database-devel] questions...

NESSUS_SQL has *NOT* been tested with the knowledgebase code. I am going to
guess that this is your problem. If the knowledgebase saving feature
deletes the knowledgebase AFTER saving it and BEFORE the DB code is called,
you will see nothing in the knowledgebase OR the vulnerability table. I
recommend recompiling your nessusd WITHOUT --enable-save-kb. If both the
SQL save AND a static file knowledgebase save is a required (or useful)
feature, we may want to look at doing SQL reads to get the data for SQL
users, or modifying the save_knowledgebase code to support the DB. I see
both as valid options and renaud may have more input.

-Cory

-----Original Message-----
From: Robert Rich [mailto:rrich@gstisecurity.com]
Sent: Thursday, January 15, 2004 10:11 AM
To: Javier Fernandez-Sanguino
Cc: Jeff Dell; Marsh, Cory; nessus-devel@list.nessus.org
Subject: Re: [Nessus-devel] [Database-devel] questions...

There is definitely database communication. Based on some previous
discussion, i created the db but left it empty and simply allowed
nessusd to create the schema.

After three scans (the last one still running), this is the number of
rows in each table:

executedplugin: 9476
host: 15
hostsession: 54
knowledgebase: 0
service: 0
session: 3
userlist: 0
usersession: 0
vulnerability: 0

Your last point related to 'realtime' status is part of what makes this
so interesting to me. I've been watching the tables grow as time
progresses. I do see the following in nessus.messages:

$ grep kb nessusd.messages | tail -20 | head -4
[Thu Jan 15 09:30:56 2004][1480] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.239
[Thu Jan 15 09:30:58 2004][1484] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.240
[Thu Jan 15 09:31:00 2004][1486] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.241
[Thu Jan 15 09:31:02 2004][1487] user rrich : new KB will be saved as
/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.242

Here were my configure args from the build of nessus-core

$ grep CONFIGURE_ARGS nessus.tmpl
CONFIGURE_ARGS = --prefix=/export/home/rrich/local --enable-release
--disable-gtk --enable-save-sessions --enable-save-kb
--with-mysql=/usr/local/mysql --with-x

Is it possible the --enable-save-kb is conflicting?

Javier Fernandez-Sanguino wrote:

> Robert Rich wrote:
>
>> I've been following this thread with interest and have been
>> building/testing the NESSUS_SQL code.
>> I do have a question that may relate to Jeff's problem. If i build
>> nessusd out of NESSUS_SQL, do i have to use the nessus client from
>> that same codebase? The system i'm building nessusd on has some gtk
>> issues, so i've been using both the gtk client and windows (NessusWX)
>> client from different systems running code from HEAD. While i
>> haven't seen any segfaults as indicated by Jeff below, i also have
>> not seen any data in the knowledgebase table.
>> Thoughts?
>>
>
> There's really no reason why you shouldn't be able to use a different
> client. Client/server communication is standarised and it should not
> be affected by the SQL code. We will try to add some debug information
> in the code in order to see what's going on.
>
> Do you see if there is any database communication at all? (snooping
> the loopback interface for example) Do you see if the host / session
> /hostsession tables are being updated when you run a scan?
>
> As a matter of fact you should be albe to see information from the
> scan in real time when you are using the database code, since the
> tables are updated when the session starts/ends, while the host is
> being scanned, etc.
>
> Regards
>
> Javi

This transmission may contain information that is privileged, confidential
and/or exempt from disclosure under applicable law. If you are not the
intended recipient, you are hereby notified that any disclosure, copying,
distribution, or use of the information contained herein (including any
reliance thereon) is STRICTLY PROHIBITED. If you received this transmission
in error, please immediately contact the sender and destroy the material in
its entirety, whether in electronic or hard copy format. Thank you. A1.

_______________________________________________
Nessus-devel mailing list
Nessus-devel@list.nessus.org
http://mail.nessus.org/mailman/listinfo/nessus-devel

_______________________________________________
Nessus-devel mailing list
Nessus-devel@list.nessus.org
http://mail.nessus.org/mailman/listinfo/nessus-devel

[INFO] -- Access Manager:
This transmission may contain information that is privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. A2

Re: [Database-devel] questions... [ In reply to ]

rrich at gstisecurity

Jan 15, 2004, 1:43 PM

Post #17 of 18 (3028 views)

Cory,

I'll recompile without the save_kb and see what happens.

Thanks!!!

Bob

Marsh, Cory wrote:

>NESSUS_SQL has *NOT* been tested with the knowledgebase code. I am going to guess that this is your problem. If the knowledgebase saving feature deletes the knowledgebase AFTER saving it and BEFORE the DB code is called, you will see nothing in the knowledgebase OR the vulnerability table. I recommend recompiling your nessusd WITHOUT --enable-save-kb. If both the SQL save AND a static file knowledgebase save is a required (or useful) feature, we may want to look at doing SQL reads to get the data for SQL users, or modifying the save_knowledgebase code to support the DB. I see both as valid options and renaud may have more input.
>
>-Cory
>
>-----Original Message-----
>From: Robert Rich [mailto:rrich@gstisecurity.com]
>Sent: Thursday, January 15, 2004 10:11 AM
>To: Javier Fernandez-Sanguino
>Cc: Jeff Dell; Marsh, Cory; nessus-devel@list.nessus.org
>Subject: Re: [Nessus-devel] [Database-devel] questions...
>
>
>There is definitely database communication. Based on some previous
>discussion, i created the db but left it empty and simply allowed
>nessusd to create the schema.
>
>After three scans (the last one still running), this is the number of
>rows in each table:
>
>executedplugin: 9476
>host: 15
>hostsession: 54
>knowledgebase: 0
>service: 0
>session: 3
>userlist: 0
>usersession: 0
>vulnerability: 0
>
>
>Your last point related to 'realtime' status is part of what makes this
>so interesting to me. I've been watching the tables grow as time
>progresses. I do see the following in nessus.messages:
>
>$ grep kb nessusd.messages | tail -20 | head -4
>[Thu Jan 15 09:30:56 2004][1480] user rrich : new KB will be saved as
>/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.239
>[Thu Jan 15 09:30:58 2004][1484] user rrich : new KB will be saved as
>/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.240
>[Thu Jan 15 09:31:00 2004][1486] user rrich : new KB will be saved as
>/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.241
>[Thu Jan 15 09:31:02 2004][1487] user rrich : new KB will be saved as
>/export/home/rrich/local/var/nessus/users/rrich/kbs/10.10.10.242
>
>Here were my configure args from the build of nessus-core
>
>$ grep CONFIGURE_ARGS nessus.tmpl
>CONFIGURE_ARGS = --prefix=/export/home/rrich/local --enable-release
>--disable-gtk --enable-save-sessions --enable-save-kb
>--with-mysql=/usr/local/mysql --with-x
>
>Is it possible the --enable-save-kb is conflicting?
>
>
>Javier Fernandez-Sanguino wrote:
>
>
>
>>Robert Rich wrote:
>>
>>
>>
>>>I've been following this thread with interest and have been
>>>building/testing the NESSUS_SQL code.
>>>I do have a question that may relate to Jeff's problem. If i build
>>>nessusd out of NESSUS_SQL, do i have to use the nessus client from
>>>that same codebase? The system i'm building nessusd on has some gtk
>>>issues, so i've been using both the gtk client and windows (NessusWX)
>>>client from different systems running code from HEAD. While i
>>>haven't seen any segfaults as indicated by Jeff below, i also have
>>>not seen any data in the knowledgebase table.
>>>Thoughts?
>>>
>>>
>>>
>>There's really no reason why you shouldn't be able to use a different
>>client. Client/server communication is standarised and it should not
>>be affected by the SQL code. We will try to add some debug information
>>in the code in order to see what's going on.
>>
>>Do you see if there is any database communication at all? (snooping
>>the loopback interface for example) Do you see if the host / session
>>/hostsession tables are being updated when you run a scan?
>>
>>As a matter of fact you should be albe to see information from the
>>scan in real time when you are using the database code, since the
>>tables are updated when the session starts/ends, while the host is
>>being scanned, etc.
>>
>>Regards
>>
>>Javi
>>
>>
>
>
>
>
>This transmission may contain information that is privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. A1.
>
>
>_______________________________________________
>Nessus-devel mailing list
>Nessus-devel@list.nessus.org
>http://mail.nessus.org/mailman/listinfo/nessus-devel
>
>

RE: [Database-devel] questions... [ In reply to ]

Jan 16, 2004, 10:02 AM

Post #18 of 18 (3022 views)