Mailing List Archive

On Tue, Jan 20, 2009 at 06:00, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
> are there some folks on this list who are working in the computer
> forensics space? I wonder how syslog, and rsyslog in specific, works in
> forensics.

Could you clarify what you're asking here? There are two clearly
delineated portions of the computer forensics space: that which is
analyzed and that which performs the analysis. Are you looking more
to improve analysis of rsyslog instances or to integrate into back-end
tools?

> Most importantly, I am interested in what stops acceptance in
> the forensics field (or what nurtures it). I am interested in feedback
> to help shape the medium to long term schedule for rsyslog (including
> those initiatives that I should learn more about).

Law Enforcement. LE is by far the biggest driver in industry
acceptance, nearly regardless of technology. The "primary" forensics
tool, EnCase, is a perfect example: there are many arguably better
products on the market, but because huge numbers of extremely
non-technical police officers are comfortable with it (since Guidance
gives steep LE discounts), it is by far the biggest player.

There isn't a huge amount of logging to be done in the analysis space.
Although centralized solutions are becoming more prevalent, most of
the critical logs are being (or will be) stored with the
encrypted/signed forensic data for non-repudiation. Even so, there is
more effort going into improving analysis (carvers, documenting
formats, etc.) than building up proper logging and storage.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

On Tue, 20 Jan 2009, RB wrote:

> On Tue, Jan 20, 2009 at 06:00, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
>> are there some folks on this list who are working in the computer
>> forensics space? I wonder how syslog, and rsyslog in specific, works in
>> forensics.
>
> Could you clarify what you're asking here? There are two clearly
> delineated portions of the computer forensics space: that which is
> analyzed and that which performs the analysis. Are you looking more
> to improve analysis of rsyslog instances or to integrate into back-end
> tools?
>
>> Most importantly, I am interested in what stops acceptance in
>> the forensics field (or what nurtures it). I am interested in feedback
>> to help shape the medium to long term schedule for rsyslog (including
>> those initiatives that I should learn more about).

I think that what he is asking about is what makes logs acceptable or not
acceptable when doing forensics, and what configurations of rsyslog would
be acceptable.

for example, rsyslog can be configured to use disk-based queues on
redundant drives and RELP for network communication, and the result will
be that rsyslog is _very_ reliable in terms of preserving messages that
get to it (at the cost of performance, but you can throw hardware at it to
deal with that)

this is probably acceptable as a log for forensics type work.

but what about the more normal settings? (tcp or udp network
communications with memory-based queues). those settings can loose data,
but won't under normal conditions (assuming the network isn't so busy that
it drops UDP packets)

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

On Tue, Jan 20, 2009 at 12:54, <david@lang.hm> wrote:
> I think that what he is asking about is what makes logs acceptable or not
> acceptable when doing forensics, and what configurations of rsyslog would
> be acceptable.

That's still unclear as to whether the logging instances are being
analyzed or they are part of the analysis process (i.e. logging
investigator actions, "interesting" items, etc.).

> for example, rsyslog can be configured to use disk-based queues on
> redundant drives and RELP for network communication, and the result will
> be that rsyslog is _very_ reliable in terms of preserving messages that
> get to it (at the cost of performance, but you can throw hardware at it to
> deal with that)
>
> this is probably acceptable as a log for forensics type work.
>
> but what about the more normal settings? (tcp or udp network
> communications with memory-based queues). those settings can loose data,
> but won't under normal conditions (assuming the network isn't so busy that
> it drops UDP packets)

Generally speaking, forensics prefers the "save everything, impossible
to lose" approach. A single lost message probably won't break a given
case, but the possibility is definitely there. RELP with disk queues
on hardware-redundant drives would probably be a good start if you're
looking to ease future analysis, but it is my opinion that networked
logging of the forensic process is both unlikely and overkill, as most
analysis processes want their logs integrated instead of held as a
separate source.

One item I have had on my wish-list for quite some time is the ability
to log directly to a UDF VAT filesystem (incremental writes on
write-once optical media). Poor man's WORM, if you will. It would
enable physical assurance that log data is unmodified up to the point
of compromise. Add in the idea of incremental checksums or signing,
and you have an extremely controlled, verifiable log source. Of
course, it doesn't have to be solved in rsyslog-space, but it'd
definitely be useful.


RB
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

On Wed, 21 Jan 2009, RB wrote:

> On Tue, Jan 20, 2009 at 12:54, <david@lang.hm> wrote:
>> I think that what he is asking about is what makes logs acceptable or not
>> acceptable when doing forensics, and what configurations of rsyslog would
>> be acceptable.
>
> That's still unclear as to whether the logging instances are being
> analyzed or they are part of the analysis process (i.e. logging
> investigator actions, "interesting" items, etc.).

I think it's the logs being analysed, not logging investigator actions
(other than the extent that things the investigators do would be logged if
anyone did them)

>> for example, rsyslog can be configured to use disk-based queues on
>> redundant drives and RELP for network communication, and the result will
>> be that rsyslog is _very_ reliable in terms of preserving messages that
>> get to it (at the cost of performance, but you can throw hardware at it to
>> deal with that)
>>
>> this is probably acceptable as a log for forensics type work.
>>
>> but what about the more normal settings? (tcp or udp network
>> communications with memory-based queues). those settings can loose data,
>> but won't under normal conditions (assuming the network isn't so busy that
>> it drops UDP packets)
>
> Generally speaking, forensics prefers the "save everything, impossible
> to lose" approach. A single lost message probably won't break a given
> case, but the possibility is definitely there.

this is the most paranoid/conservative view, and by this definition there
are basicly no logs in existance that meet the forensics requirements

> RELP with disk queues
> on hardware-redundant drives would probably be a good start if you're
> looking to ease future analysis, but it is my opinion that networked
> logging of the forensic process is both unlikely and overkill, as most
> analysis processes want their logs integrated instead of held as a
> separate source.
>
> One item I have had on my wish-list for quite some time is the ability
> to log directly to a UDF VAT filesystem (incremental writes on
> write-once optical media). Poor man's WORM, if you will. It would
> enable physical assurance that log data is unmodified up to the point
> of compromise. Add in the idea of incremental checksums or signing,
> and you have an extremely controlled, verifiable log source. Of
> course, it doesn't have to be solved in rsyslog-space, but it'd
> definitely be useful.

frankly, if you really need write-only media, the best thing to do (volume
permitting) is to dump to a printer.

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

On Wed, Jan 21, 2009 at 12:55, <david@lang.hm> wrote:
> this is the most paranoid/conservative view, and by this definition there
> are basicly no logs in existance that meet the forensics requirements

Rather than set an unattainable standard, my intent was to communicate
the conservative approach forensics would rather take. Edge cases and
mitigating controls are acceptable as long as they are well-documented
- that's basic security practice. I would rather see a solution that
has 100 well-documented lossy edge cases than one that claims to be
lossless with no proofs to back it.

> frankly, if you really need write-only media, the best thing to do (volume
> permitting) is to dump to a printer.

You may want to recalculate; even 6-point font on large (14.875x11.5")
tractor-feed paper only fits ~80MB per 3500-sheet box. Or, put
another way, 2 512-byte events per second will burn through a $70 case
per day. Or 6.5 reams of US Letter per day. Extremely limited
volume.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

Hi all,

Sorry for posting the question and then being offline. I had a meeting
and was after that a bit more swamped than I expected ;)

Thanks for the good answers so far. My question was vague, but that
reflected that I actually do not exactly know what to ask for. While I
took a look at forensics every now and then, this is not an area where I
have really any deep expertise.

However, I should have stated that I am primarily interested on the
event detection/gathering, transmission and storage part of the picture.
That's where rsyslog can play a role (that limits the "event detection"
process to listening to whoever wants to talk to it). The analysis part
is beyond my scope right now (and probably will be for quite some time).
As I said, I do not have an immediate need, but would like to understand
the needs a bit better (and you have already provided good advise so far
:)).

The root cause of my question is that I would like to refine my medium,
may be long term vision. While I think I can not implement any of the
outcome, it helps my tune the implementation of things I do in a way
that facilitates forensic needs (at least in cases where I have a
choice). Without that information, I would probably do things in ways
that will require much more effort once I get to "forensics-readiness".

I hope this clarifies and sorry for not replying sooner. I will probably
be a bit swamped 'til the end of the week, but will try to be more
responsive now :)

Thanks again for all that fine information, please keep it flowing. It
is very useful.

Rainer


> -----Original Message-----
> From: rsyslog-bounces@lists.adiscon.com
> [mailto:rsyslog-bounces@lists.adiscon.com] On Behalf Of RB
> Sent: Wednesday, January 21, 2009 9:59 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Anyone in Computer Forensics?
>
> On Wed, Jan 21, 2009 at 12:55, <david@lang.hm> wrote:
> > this is the most paranoid/conservative view, and by this
> definition there
> > are basicly no logs in existance that meet the forensics
> requirements
>
> Rather than set an unattainable standard, my intent was to communicate
> the conservative approach forensics would rather take. Edge cases and
> mitigating controls are acceptable as long as they are well-documented
> - that's basic security practice. I would rather see a solution that
> has 100 well-documented lossy edge cases than one that claims to be
> lossless with no proofs to back it.
>
> > frankly, if you really need write-only media, the best
> thing to do (volume
> > permitting) is to dump to a printer.
>
> You may want to recalculate; even 6-point font on large (14.875x11.5")
> tractor-feed paper only fits ~80MB per 3500-sheet box. Or, put
> another way, 2 512-byte events per second will burn through a $70 case
> per day. Or 6.5 reams of US Letter per day. Extremely limited
> volume.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

On Wed, 21 Jan 2009, RB wrote:

> On Wed, Jan 21, 2009 at 12:55, <david@lang.hm> wrote:
>> this is the most paranoid/conservative view, and by this definition there
>> are basicly no logs in existance that meet the forensics requirements
>
> Rather than set an unattainable standard, my intent was to communicate
> the conservative approach forensics would rather take. Edge cases and
> mitigating controls are acceptable as long as they are well-documented
> - that's basic security practice. I would rather see a solution that
> has 100 well-documented lossy edge cases than one that claims to be
> lossless with no proofs to back it.

the problem is that so many forensics people list the perfect situation
and tell people that anything less won't stand up in court.

like everything else, it's a reliability/performance/cost trade-off

but we really aren't answering the initial question here (or rather we are
demonstrating that there isn't a clear answer to the question)

>> franklk, if you really need write-only media, the best thing to do
(volume
>> permitting) is to dump to a printer.
>
> You may want to recalculate; even 6-point font on large (14.875x11.5")
> tractor-feed paper only fits ~80MB per 3500-sheet box. Or, put
> another way, 2 512-byte events per second will burn through a $70 case
> per day. Or 6.5 reams of US Letter per day. Extremely limited
> volume.

that's why I said volume permitting (and for your most critical logs the
volume is probably fairly low)

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com