In the interest of brevity, I'm leaving out details...if you need
more, just ask.
We have a security event monitoring system that processes probably in
the neighborhood of 100 million syslog messages per day (I know
precisely how many events it has processed, but it doesn't break them
down by protocol). In some of our WAN sites, I would like to
implement a local system that will receive all the local syslog
messages and ship some/all back to the main collector on our LAN. The
main collector on the LAN would receive the bulk of its message from
other LAN systems. Then the main collect would ship some/all to the
SEM environment. I'm leaning towards using rsyslog for this task and
have a few questions:
1) What kind of system [rough estimate] would I need for the main
collector if assume 200 million syslog messages per day and peak that
is triple that average rate (~7000 eps)?
2) Can I enable rate limiting in a way that will:
A1) start dropping messages beyond a given threshold
A2) start intelligently dropping messages beyond a given threshold
(i.e. start dropping events matching this regex)
B) allow me to alert someone that this is occurring (is written to
log file, etc)
more, just ask.
We have a security event monitoring system that processes probably in
the neighborhood of 100 million syslog messages per day (I know
precisely how many events it has processed, but it doesn't break them
down by protocol). In some of our WAN sites, I would like to
implement a local system that will receive all the local syslog
messages and ship some/all back to the main collector on our LAN. The
main collector on the LAN would receive the bulk of its message from
other LAN systems. Then the main collect would ship some/all to the
SEM environment. I'm leaning towards using rsyslog for this task and
have a few questions:
1) What kind of system [rough estimate] would I need for the main
collector if assume 200 million syslog messages per day and peak that
is triple that average rate (~7000 eps)?
2) Can I enable rate limiting in a way that will:
A1) start dropping messages beyond a given threshold
A2) start intelligently dropping messages beyond a given threshold
(i.e. start dropping events matching this regex)
B) allow me to alert someone that this is occurring (is written to
log file, etc)