Mailing List Archive: Data Persistence between Ntop sessions?

Data Persistence between Ntop sessions?

Jun 13, 2001, 11:54 AM

Post #1 of 6 (2109 views)

Hello -

I've read the documentation and man page, and I can't seem to find an
answer to my question.

When I run ntop, it will core dump periodically. I want to have ntop
running for months at a time, so I can keep track of the bandwidth
used by colocated servers on my network. ntop crashing isn't the
problem... I can restart it with cron if need be. The problem is
that when it restarts it loses all its data. Is there a way to keep
the data persistently between sessions, so when it's restarted by
cron the only lost data will be the data since it crashed?

Thanks in advance for any help...
Preston Wilson
--

-- --
-- Preston Wilson -- Data Center Manager -- Interactive Ink, Inc. --
-- --

Re: Data Persistence between Ntop sessions? [ In reply to ]

patsmailinglists at dresden-online

Jun 13, 2001, 12:55 PM

Post #2 of 6 (2079 views)

Permalink

hello Preston!

on Wednesday, June 13, 2001, 8:54:11 PM, you wrote:

PW> Hello -

PW> I've read the documentation and man page, and I can't seem to find an
PW> answer to my question.

PW> When I run ntop, it will core dump periodically. I want to have ntop
PW> running for months at a time, so I can keep track of the bandwidth
PW> used by colocated servers on my network. ntop crashing isn't the
PW> problem... I can restart it with cron if need be. The problem is
PW> that when it restarts it loses all its data. Is there a way to keep
PW> the data persistently between sessions, so when it's restarted by
PW> cron the only lost data will be the data since it crashed?
i asked the same question days ago and still looking for the
answer.. i remember reading a solution somewhere in google or
deja.. but i just can't find it anymore...

%}

PAT
--

vcard/LDAP/PGP: http://dresden-online.com/perler/identity.html
PGP fingerprint: DAC6 2FDA 1ED7 AD55 BD1F 5142 3D5F 72BF

RE: Data Persistence between Ntop sessions? [ In reply to ]

dcolquho at opentext

Jun 13, 2001, 1:06 PM

Post #3 of 6 (2079 views)

Permalink

Depending on your needs RRD can be used and fed data by a perl script.
Have a look at the example in the www directory, and
/dumpTrafficData.html to get an idea of what you can store.
/dumpData.html provides much more information, but at the expense of
being bloody huge. RRD should be quick and easy to setup, and graphs
are a cinch. As I just want pretty pictures of protocol information, I
use RRD over some other solutions like SQL.

Also, the very recent CVS snapshots seems to store persistent data
correctly, but again it's been my experience that it requires huge
amounts of disk space.

> -----Original Message-----
> From: ntop-admin@unipi.it [mailto:ntop-admin@unipi.it] On
> Behalf Of Patrick Erler
> Sent: June 13, 2001 3:56 PM
> To: Preston Wilson
> Subject: Re: [Ntop] Data Persistence between Ntop sessions?
>
>
> hello Preston!
>
> on Wednesday, June 13, 2001, 8:54:11 PM, you wrote:
>
> PW> Hello -
>
> PW> I've read the documentation and man page, and I can't
> seem to find an
> PW> answer to my question.
>
> PW> When I run ntop, it will core dump periodically. I want
> to have ntop
> PW> running for months at a time, so I can keep track of the
> bandwidth
> PW> used by colocated servers on my network. ntop crashing isn't the
> PW> problem... I can restart it with cron if need be. The problem is
> PW> that when it restarts it loses all its data. Is there a
> way to keep
> PW> the data persistently between sessions, so when it's restarted by
> PW> cron the only lost data will be the data since it crashed?
> i asked the same question days ago and still looking for the
> answer.. i remember reading a solution somewhere in google or
> deja.. but i just can't find it anymore...
>
> %}
>
> PAT
> --
>
>
>
> vcard/LDAP/PGP: http://dresden-online.com/perler/identity.html
> PGP fingerprint: DAC6 2FDA 1ED7 AD55 BD1F 5142 3D5F 72BF
>
> _______________________________________________
> Ntop mailing list
> Ntop@unipi.it
> http://listmanager.unipi.it/mailman/listinfo/ntop
>

Re: Data Persistence between Ntop sessions? [ In reply to ]

davem at mitre

Jun 13, 2001, 1:30 PM

Post #4 of 6 (2077 views)

Permalink

We took that approach initially. The problem is that even if you save the information in a separate
database, when NTOP restarts all of the statistics start over with a new baseline. Now you have two
sets that can't be merged in a meaningful fashion.

Either NTOP needs to be able to restart from a saved baseline, or it needs to output events instead
of statistics for external calculations. Without taking one of these approaches, I don't see any way
to merge the data either from multiple instances of NTOP or multiple restarts.

Also, /dumpTrafficData.html does not export all of the data. For example "Session" data and "Alert"
data.

Dan Colquhoun wrote:
>
> Depending on your needs RRD can be used and fed data by a perl script.
> Have a look at the example in the www directory, and
> /dumpTrafficData.html to get an idea of what you can store.
> /dumpData.html provides much more information, but at the expense of
> being bloody huge. RRD should be quick and easy to setup, and graphs
> are a cinch. As I just want pretty pictures of protocol information, I
> use RRD over some other solutions like SQL.
>
> Also, the very recent CVS snapshots seems to store persistent data
> correctly, but again it's been my experience that it requires huge
> amounts of disk space.
>
> > -----Original Message-----
> > From: ntop-admin@unipi.it [mailto:ntop-admin@unipi.it] On
> > Behalf Of Patrick Erler
> > Sent: June 13, 2001 3:56 PM
> > To: Preston Wilson
> > Subject: Re: [Ntop] Data Persistence between Ntop sessions?
> >
> >
> > hello Preston!
> >
> > on Wednesday, June 13, 2001, 8:54:11 PM, you wrote:
> >
> > PW> Hello -
> >
> > PW> I've read the documentation and man page, and I can't
> > seem to find an
> > PW> answer to my question.
> >
> > PW> When I run ntop, it will core dump periodically. I want
> > to have ntop
> > PW> running for months at a time, so I can keep track of the
> > bandwidth
> > PW> used by colocated servers on my network. ntop crashing isn't the
> > PW> problem... I can restart it with cron if need be. The problem is
> > PW> that when it restarts it loses all its data. Is there a
> > way to keep
> > PW> the data persistently between sessions, so when it's restarted by
> > PW> cron the only lost data will be the data since it crashed?
> > i asked the same question days ago and still looking for the
> > answer.. i remember reading a solution somewhere in google or
> > deja.. but i just can't find it anymore...
> >
> > %}
> >
> > PAT
> > --
> >
> >
> >
> > vcard/LDAP/PGP: http://dresden-online.com/perler/identity.html
> > PGP fingerprint: DAC6 2FDA 1ED7 AD55 BD1F 5142 3D5F 72BF
> >
> > _______________________________________________
> > Ntop mailing list
> > Ntop@unipi.it
> > http://listmanager.unipi.it/mailman/listinfo/ntop
> >
>
> _______________________________________________
> Ntop mailing list
> Ntop@unipi.it
> http://listmanager.unipi.it/mailman/listinfo/ntop

RE: Data Persistence between Ntop sessions? [ In reply to ]

preston at interactiveink

Jun 13, 2001, 1:51 PM

Post #5 of 6 (2078 views)

Permalink

Dan -

Thanks for your reply.

You mention the very recent CVS snapshots. Did you ever figure out
what was wrong with that "can't chdir(/home/tmp)" error with CVS?
I'm getting that same error.

I'll try to get a snapshot from snapshot.ntop.org. If the newest
snapshot fixes this persistence problem that would be very cool.

Thanks...
Preston

>Depending on your needs RRD can be used and fed data by a perl script.
>Have a look at the example in the www directory, and
>/dumpTrafficData.html to get an idea of what you can store.
>/dumpData.html provides much more information, but at the expense of
>being bloody huge. RRD should be quick and easy to setup, and graphs
>are a cinch. As I just want pretty pictures of protocol information, I
>use RRD over some other solutions like SQL.
>
>
>Also, the very recent CVS snapshots seems to store persistent data
>correctly, but again it's been my experience that it requires huge
>amounts of disk space.
>
>
>> -----Original Message-----
>> From: ntop-admin@unipi.it [mailto:ntop-admin@unipi.it] On
>> Behalf Of Patrick Erler
>> Sent: June 13, 2001 3:56 PM
>> To: Preston Wilson
>> Subject: Re: [Ntop] Data Persistence between Ntop sessions?
>>
>>
>> hello Preston!
>>
>> on Wednesday, June 13, 2001, 8:54:11 PM, you wrote:
>>
>> PW> Hello -
>>
>> PW> I've read the documentation and man page, and I can't
>> seem to find an
>> PW> answer to my question.
>>
>> PW> When I run ntop, it will core dump periodically. I want
>> to have ntop
>> PW> running for months at a time, so I can keep track of the
>> bandwidth
>> PW> used by colocated servers on my network. ntop crashing isn't the
>> PW> problem... I can restart it with cron if need be. The problem is
>> PW> that when it restarts it loses all its data. Is there a
>> way to keep
>> PW> the data persistently between sessions, so when it's restarted by
>> PW> cron the only lost data will be the data since it crashed?
>> i asked the same question days ago and still looking for the
>> answer.. i remember reading a solution somewhere in google or
>> deja.. but i just can't find it anymore...
>>
>> %}
>>
>> PAT
>> --
>>
>>
>>
>> vcard/LDAP/PGP: http://dresden-online.com/perler/identity.html
>> PGP fingerprint: DAC6 2FDA 1ED7 AD55 BD1F 5142 3D5F 72BF
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@unipi.it
>> http://listmanager.unipi.it/mailman/listinfo/ntop
>>
>
>_______________________________________________
>Ntop mailing list
>Ntop@unipi.it
>http://listmanager.unipi.it/mailman/listinfo/ntop

--

-- --
-- Preston Wilson -- Data Center Manager -- Interactive Ink, Inc. --
-- --

RE: Data Persistence between Ntop sessions? [ In reply to ]

dcolquho at opentext

Jun 13, 2001, 2:00 PM

Post #6 of 6 (2090 views)

Permalink

No luck with the CVS today, I'm only mildy acquainted with CVS anyway.
I suspect it may have something to do with the recent move of the
repository to /export/home/ntop, but I don't see how exactly. Luca told
me he will be away until Monday, so I doubt anything will change in CVS
until then anyway.

> -----Original Message-----
> From: ntop-admin@unipi.it [mailto:ntop-admin@unipi.it] On
> Behalf Of Preston Wilson
> Sent: June 13, 2001 4:52 PM
> To: ntop@unipi.it
> Subject: RE: [Ntop] Data Persistence between Ntop sessions?
>
>
> Dan -
>
> Thanks for your reply.
>
> You mention the very recent CVS snapshots. Did you ever figure out
> what was wrong with that "can't chdir(/home/tmp)" error with CVS?
> I'm getting that same error.
>
> I'll try to get a snapshot from snapshot.ntop.org. If the newest
> snapshot fixes this persistence problem that would be very cool.
>
> Thanks...
> Preston
>
> >Depending on your needs RRD can be used and fed data by a
> perl script.
> >Have a look at the example in the www directory, and
> >/dumpTrafficData.html to get an idea of what you can store.
> >/dumpData.html provides much more information, but at the expense of
> >being bloody huge. RRD should be quick and easy to setup, and graphs
> >are a cinch. As I just want pretty pictures of protocol
> information, I
> >use RRD over some other solutions like SQL.
> >
> >
> >Also, the very recent CVS snapshots seems to store persistent data
> >correctly, but again it's been my experience that it requires huge
> >amounts of disk space.
> >
> >
> >> -----Original Message-----
> >> From: ntop-admin@unipi.it [mailto:ntop-admin@unipi.it] On
> >> Behalf Of Patrick Erler
> >> Sent: June 13, 2001 3:56 PM
> >> To: Preston Wilson
> >> Subject: Re: [Ntop] Data Persistence between Ntop sessions?
> >>
> >>
> >> hello Preston!
> >>
> >> on Wednesday, June 13, 2001, 8:54:11 PM, you wrote:
> >>
> >> PW> Hello -
> >>
> >> PW> I've read the documentation and man page, and I can't
> >> seem to find an
> >> PW> answer to my question.
> >>
> >> PW> When I run ntop, it will core dump periodically. I want
> >> to have ntop
> >> PW> running for months at a time, so I can keep track of the
> >> bandwidth
> >> PW> used by colocated servers on my network. ntop
> crashing isn't the
> >> PW> problem... I can restart it with cron if need be.
> The problem is
> >> PW> that when it restarts it loses all its data. Is there a
> >> way to keep
> >> PW> the data persistently between sessions, so when it's
> restarted by
> >> PW> cron the only lost data will be the data since it crashed?
> >> i asked the same question days ago and still looking for the
> >> answer.. i remember reading a solution somewhere in google or
> >> deja.. but i just can't find it anymore...
> >>
> >> %}
> >>
> >> PAT
> >> --
> >>
> >>
> >>
> >> vcard/LDAP/PGP: http://dresden-online.com/perler/identity.html
> >> PGP fingerprint: DAC6 2FDA 1ED7 AD55 BD1F 5142 3D5F 72BF
> >>
> >> _______________________________________________
> >> Ntop mailing list
> >> Ntop@unipi.it
> >> http://listmanager.unipi.it/mailman/listinfo/ntop
> >>
> >
> >_______________________________________________
> >Ntop mailing list
> >Ntop@unipi.it
> >http://listmanager.unipi.it/mailman/listinfo/ntop
>
>
> --
>
> --
> --
> -- Preston Wilson -- Data Center Manager -- Interactive
> Ink, Inc. --
> --
> --
> _______________________________________________
> Ntop mailing list
> Ntop@unipi.it
> http://listmanager.unipi.it/mailman/listinfo/ntop
>