On Tuesday 11 February 2003 05:04, Javier Fernandez-Sanguino wrote:
> Cool. I'll take this into account. I've worked with Dia before, could
> you please send this script to the Dia upstream developers so they
> include it in the contrib/ section of it?
Thanks for the suggestion. I'm looking into that.
> > Here's the list of changes I made, and my reasoning:
> > =========================================================================
> >======= Deleted ExecutionID column from Vulnerability table.
> > Deleted fk relationship Vulnerability_ExecutedPlugins_ExecutionID.
> > Created PluginID column in Vulnerability table.
> > Created fk relationship Vulnerability_NessusPlugins_PluginID.
> > -------------------------------------------------------------------------
> >------- A plugin will address a given vulnerability. ExecutedPlugins
> > relates plugins to sessions. Vulnerabilities are related to plugins
> > regardless of scans performed.
>
> The fact is, you've gotten this wrong IMHO.
Okay, your opinion is noted...care for a little debate? :-)
> Vulnerabilities are associated with hosts. That is, the vulnerability
> table are vulnerabilities that have been _found_ while testing a host.
> Not "theoric" vulnerabilities.
I can understand what you're shooting for, but I think it's the wrong approach
to get there. True, a host may have a given vulnerability and we want Nessus
to tell us that for each case. However, a vulnerability is still a discrete
entity (a condition of a piece of software). That's not saying it's
"theoretic", theoretic is when the vulnerability is determined through
analysis but no *known* exploit exists. By placing making an fk relationship
from the Plugin table to the Vulnerability table, we're maintaining the
proper relationship of two differnt entities.
> Why are ExecutedPlugins and Vulnerabilities separated then? Because you
> can execute a plugin against a host but don't detect any vulnerability
> associated with it.
That's why a boolean column value called "FoundVulnerable" or some such thing
should be in the ExecutedPlugins table...because it's an attribute (the
result) of the action of running the plugin (which is what the
ExecutedPlugins table represents).
> The "AdditionalInformation" stuff is a placeholder for information that
> a Plugin might have retrieved from a host. Sample: banners from a telnet
> server, open shares from a Netbios host.
If you want to store data gathered during a plugin, just make it a column of
the ExecutedPlugins table. It's a much more logical place for it, since that
table is the record of a moment in time - each row representing some given
state of the system we're scanning.
> That's the main reasoning of having an "executedplugin -> vulnerability"
Unfortunately, this is putting the cart before the horse. What we have here
is a hierarchical relationship between two entities and one state. A
vulnerability is discovered. Renaurd et al write a plugin against it. You
and I run our Nessus scanners against our respective networks, executing the
plugin, where we get a postive or negative. That's the hierarchical order of
these three things...to express it any other way is incorrect.
> Maybe I should have named it "Detected_Vulnerability" and not just
> "Vulnerability". However, the point is: I want an easy way to determine
> the exact # of vulnerabilties in my infraestructure.
To have a table that gets rows for every vulnerability on every host is
wasteful. Let's make up an example (which you can find at
http://mywebpages.comcast.net/earnoth/example.tar! ;) where there are three
hosts who all have the same two plugins executed against them (let's just
arbitrarily pick "WS FTP overflows" (CAN-2001-1021) and "Canna Overflow"
(CVE-2000-0584). Let's now pretend that all three are vulnerable to both.
Now the Vulnerability table is going to be populated with no less than four
duplicate rows. Now lets imagine this for a scan against 200 hosts where
there are common vulnerabilities between hosts (let's pick an average of 40
per host). That scales out to about 7,960 (200*40 - 40) duplicate rows that
store redunant data. This a terrible waste of space.
So what do we buy with the space we're throwing away?
To get the vulnerabilities for a host in the original v->e->p schema, we might
use something like this:
--------------------------------------------------------------------------------
jefferson8:40pm[121]> cat query.v1.sql
select h.name, v.type
from host h, vulnerability v, executedplugins e
where
v.executionid = e.executionid and e.sessionid = 1 and
h.hostid = e.hostid
--------------------------------------------------------------------------------
A simple query indeed. To get the vulnerabilities for a host in the e->p->v
schema, we might use something like this:
--------------------------------------------------------------------------------
jefferson8:44pm[122]> cat query.v3.sql
select h.name, v.type
from host h, vulnerability v, executedplugins e, nessusplugin p
where
p.pluginid = e.pluginid and e.sessionid = 1 and e.foundvulnerable=TRUE
and
h.hostid = e.hostid and p.vulnerabilityid = v.vulnerabilityid
--------------------------------------------------------------------------------
(nessusdb_v3 is the same as v2, but I changed the FoundVulnerable column type
from bit to boolean)
That's not too much more complex, though. So what exactly are we gaining for
all that wasted space? Okay, so we might take a small hit on performance,
but even for large result sets, I can't a see how a small query performance
save is worth the waste shown above.
> I can then link
> this to a ticket-based management system that provides me with a view
> with which vulnerabilities are still "open" (i.e. have not been fixed by
> the admins) and issue a ticket whenever I find a vulnerability.
I can understand this viewpoint entirely. I work in a massive corporation,
and this kind of thing is the bane of our existance. However, I think the
examples I've posted here show that the revisions I posted can achieve the
same thing. It's simply a matter of how one builds one's additional table
structure around the foundation, however that foundation is built. Moreover,
if the foundation is built around how the data being modeled relates to one
another, additional structure will acually work better, in the long-run.
>
> > =========================================================================
> >======= Deleted PluginCVE table, added CVE column to Vulnerability table.
> > -------------------------------------------------------------------------
> >------- A CVE names a vulnerability, not a Plugin. Plugins may reference
> > vulnerabilities through the CVE number, but CVE's are not published
> > against Nessus plugins.
> > A CVE entry only accounts for a single vulnerability. As such, each
> > vulnerability listed in the Vulnerability table should only need one
> > entry.
>
> Not correct.
Really? The CVE naming process documentation at
http://cve.mitre.org/docs/docs2000/naming_process.html references the terms
vulnerability and exposure in the singular term.
Also, this quote seems to suggest vulnerabilities and exposures are expressed
in the singular form (from
http://cve.mitre.org/about/faq.html#C1):
------------------------------------------------------------------------------------------
C1. What information is included in a CVE entry?
CVE names include the name (also referred to as the CVE "number"), a brief
description of the security vulnerability or exposure, and any pertinent
references. The CVE number, for example, CVE-2000-0067, is an encoding of the
year and a unique number N for the Nth candidate proposed that year.
------------------------------------------------------------------------------------------
I have yet to see a CVE that refers to more than one vuln...could you please
give an example where a single CVE references more than one discrete
vulnerability? Even if that is the case, then simply breaking out the CVE
into its own table as it was originally done is a simple solution.
> > =========================================================================
> >======= Deleted BID, added BugtraqID column to Vulnerability table.
> > -------------------------------------------------------------------------
> >------- Same reasoning as CVE
>
> Ditto. There are three wrong assumptions here:
>
> - a vulnerability is a "theoric" one. Which is not, it's one that has
> been detected
Again, a theoretical vulnerability is one which has no *known* exploit against
it. I understand the want to link a host with it's vulnerabilities (that's
the whole point of Nessus, of course!) but that doesn't change the fact that
a vulnerability is a discrete concept, or entity, and should be properly
represented in the database design.
> - there is always one CVE entry for each Bugtraq entry
> - there is always one CVE/Bugtraq entry for each plugin.
>
> As a matter of fact there are plugins which test more than a given
> vulnerability. NASL won't allow this to be placed in the reports just
> yet, but should do in the future.
Here again, could you please provide an example? I admit I could be
completely wrong here (as with the CVE matter) but I have yet to find an
example. I did check the plugin list on the web, but was unable to find
anything.
> > =========================================================================
> >======= Deleted IsFalsePositive column from Vulnerability table.
> > Added IsFalsePositive column to ExecutedPlugins table.
> > -------------------------------------------------------------------------
> >------- In keeping with the model that differentiates a vulnerability from
> > the execution iof a plugin n a session, a false positive is an aspect of
> > a scan, not the vulnerability.
>
> Since this assumption of vulnerability not associated with plugin
> execution is false (or at least not properly understood), I'm afraid
> this shouldn't be done either.
See my comments above.
>
> > =========================================================================
> >======= Deleted DetectedService_NessusPlugin_PluginID fk relationship.
> > Deleted Services_ExecutedPlugins_ServiceID fk relationship.
> > Reversed Services_DetectedService_ServiceID fk relationship.
> > Deleted ServiceID column from ExecutedPlugins table.
> > Deleted PluginID column from DetecteService table.
> > Created ServiceID column to NessusPlugin table.
> > Created NessusPlugin_Services_ServiceID fk relationship.
> > -------------------------------------------------------------------------
> >------- A plugin will only probe a vulnerability in a single service. As
> > such, linking each NessusPlugin row will have an associated row in the
> > Services table, regardless of any scan run with the plugin
> > (ExecutedPlugins).
>
> That's not correct. There might be plugins which test more than one
> service. Simple example: a port scanner or the hydra brute force
> scanner. You cannot have a 1:1 relationship between plugins and
> services, that's just not correct.
Quite right, I see that there are plugins which do scan multiple ports. A
simple many-to-many table structure can solve that.
>
> > =========================================================================
> >======= Created NessusVersion column in Session table.
> > -------------------------------------------------------------------------
> >------- It might be useful to know what version of Nessus was used in a
> > given scan
>
> Yes. This one might be useful.
Glad we agree somewhere...hopefully through this debate we can find more to
agree on. ;-)
> > =========================================================================
> >======= Added FoundVulnerable column to ExecutedPlugins table.
> > Deleted PluginID column from Vulnerability table.
> > Added VulnerabilityID column to NessusPlugin table.
> > Added Description column to Vulnerability table.
> > Deleted Vulnerability_NessusPlugin_PluginID fk reference.
> > Created NessusPlugin_Vulnerability_VulnerabilityID fk reference.
> > -------------------------------------------------------------------------
> >------- Furthering the seperation of the Vulnerability, Plugin, and scan
> > sessions concepts.
>
> Since vulnerabilities are not "generic" ones but specific ones. There
> is no way you can do these changes. Note that you could have this schema
> and add it whatever additional information you want to (for example
> CERT's/CVE's/Bugtraq's database) which would provide the detailed
> description information you are talking about.
It's not a matter of "generic" versus "specific". I've already pretty well
beaten this topic to death, but I'll just respond to this last one by again
stressing that a vulnerability is a discrete entity that should be
appropriately represented in both table and surrounding structure (correct
foriegn key relationships, etc). The association of a host with a
vulnerability is properly achieved through the
ExecutedPlugins->Plugins->Vulnerability structure.
It is good to note that the Executed->Plugin->Vuln structure parallels the
DetectedService->Services->Protocol layout in the original design. The two
sets of tables show very similar hierarchical relationships. (A host has a
service listening on a given port over a given protocol, and a host has a
plugin result from the an execution of a plugin which was written to address
a vulnerability).
>
> > -------------------------------------------------------------------------
> >------- Other thoughts:
> >
> > There may be an issue with the NessusPlugin table. Due to the nature of
> > all the columns specified there, each one demands a large varchar()
> > datatype. However, having too large of a max row size can be problematic.
> > However, the plugin is a discrete conceptual unit, so the columns all
> > make sense to be in that table.
>
> I'm not sure this is an issue. I have the whole plugin information in a
> table with this schema and I'm not seeing problems with this.
This will be DBMS specific, and may depend on each database supported in
turn...
> > Should ExecutedPlugins have timestamps as it does now? That is, does
> > nessus record when each plugin is stopped & started? If not, then the
> > StartTime & EndTime should probably be removed from ExecutedPlugins.
>
> Nessusd should (check Michael's code) introduce this information in the
> database:
>
> - right after calling a plugin -> make an SQL insert
> - right after finishing a plugin call -> make an SQL update
D'oh! Where was my brain on that one....?!?
> If the nessus server fails you can still see which plugins were not
> finished by checking which ones do not have an end time. This
> information is also useful to make estimates of how much a scan will
> take place, given the same condition. I.e. determine roughly how much on
> average a plugin against a given host will run.
>
> However, it might make sense to add a column for the nessus server to
> add to the database when a given plugin was killed or did not end
> correctly (maybe a boolean?)
A good idea, indeed. Boolean would work, I see in the nessus code that there
is a return value specifying "killed plugin" (or some such thing, it was a
few nights ago I looked...)
> How about adding the .dia and .mysql, .pgsql files in the Nessus CVS
> source so we can keep track of what changes have been done to the schema?
Has someone set up an independent CVS server for this database project, or is
there a module in the main Nessus CVS? If the first, I know know where it
is, if the second, I don't have write access... How does one get access?
Just as a note, I plan to write some PostgreSQL code, similar in nature to
what William has written, though after looking at his code I have some
additional ideas for additional functions that should be implemented. Of
course, C design shouldn't be taken too far until a database design is
finalized.
--
Eric I. Arnoth CISSP (
http://www.isc2.org)
earnoth@comcast.net
http://mywebpages.comcast.net/earnoth ¤ø,¸¸,ø¤º°*°º¤ø,¸¸,ø¤ø,¸¸,ø¤º°*°º¤ø,¸¸,ø¤ø,¸¸,ø¤º°*°º¤ø,¸¸,ø¤ø,¸¸,ø¤º°*°º¤ø,¸¸,ø
Nessus-database-schema.png (10.5 KB)