From: Stephen Malenshek [mailto:smalenshek@skyline-ats.com]
Sent: Monday, April 28, 2008 8:55 AM
To: rsyslog at lists.adiscon.com; Rainer Gerhards
Subject: [rsyslog] RE: RSYSLOG "Best Practices" & General Questions
On Fri, 2008-04-25 at 10:05 -0600, Stephen Malenshek wrote:
> I am currently setting up creating a managed service platform from
> various open source products out on the market and I would like to use
> your product as the standard SYSLOG replacement on all our sites. I
> have a couple of questions related to this and would like you provide
> some input on the best ways to achieve specific objectives.
>
>
>
> 1) At the present time, I have started the configuration on the
> "central" server, which will act as the central repository for all
> data from the remote sites. I am configuring it to store all SYSLOG
> data with in the database, but I have followed the recommendations
> made to "buffer" it to a spool first. My question is this, I do not
> want to just write the information to the database, for governmental
> compliance, I need to keep a duplicate copy in "standard" log format
> on the drive, which I will rotate and gzip daily, for long term log
> retention. I have looked around and did not find anything that
> specifically addresses this. It looked like it should able to be
> done, but I am just not sure the best way to accomplish this.
>
Storing to the database is in no way special. So you can just add as
many other selector lines as you like. For example:
*.* :ommysql*...
*.* /var/log/soxstore
works perfectly. What you need to be aware is that the buffer parameters
($Action...) work on the NEXT action, so you want to have them directly
in front of the action in question.
In my present configuration, I am doing an if/then like the following:
if $source == 'somemachine' \
and $syslogfacility-text == 'authpriv' \
then ?LocalSecure
$template LocalSecure,"/var/log/secure"
With this, how can I define multiple directions within this template?
> 2) At each customer site, there will be a server called a
> "collector" that will accept all SYSLOG related information for that
> site. This server will store a copy of the log files for the local
> network as a repository, but it also needs to send it to the central
> server for processing. My question is whether it will be more
> efficient to write the information directly to the database, or to
> just send it using normal SYSLOG directives, 'I.E. *.* @{IP Address},
> and let the server process and insert like it would local logs?
>
that's a very good question. I'd say it depends on a lot of factors,
probablky most importantly from the clients ability to talk to the
database directly (this is often impossible due to firewalls). Also, you
may consider where the plain text files need to be written. If they
should be on a central system, I'd opt for moving everything to the
central server and writing it to database and/or text file there.
Please note that UDP (@IP) is a really bad protocol choice. You will
lose a lot of messages and it is definitely not
"compliance-compliant" ;). TCP based syslog is much better but not
ideal, see:
http://rgerhards.blogspot.com/2008/04/on-unreliability-of-plain-tcp-syslog.h
tml
rsyslog's solution is the relp protocol, which prevents message loss.
> 3) Within the scenario listed in question 2, how can I, 1)
> preserve all the original IP addresses of the machines that are
> transmitting information,
If you use rsyslog on the clients, too, this is automatically done for
you.
At each site, there will be a single server tat is aggregating all SYSLOG
data before transmitting it to the central repository. Are you talking
about the machine sending the aggregate information or the hosts, I.E.
router, server, switch, etc., that we will be monitoring?
> and 2) tag that information with a specific account code identifying
> the site that the information was sent from. Within the database, I
> have created a column called "customerid" that I would like to do this
> with. In this, I would like to designate a name or integer like "1"
> for site A, "2" for site B, etc. The reason for this is that I will
> run into situations where multiple customers will have the same IP
> addressing scheme. I figure this could be passed from the site's
> collector as a site identifier, but I am not sure how to accomplish
> this. I think I can accomplish this on the central server, if I have
> to, with a subquery within the insert query to another table to lookup
> this value, but I am looking for a much more "elegant" method.
You should look into template definitions. On the client, create a
template that contains the customer ID somewhere AFTER the tag. Maybe
immediately behind it with the rest of the message being separated by a
comma.
Then, on the server, look into the property replace. Use field-based
extraction to pull out that value and insert it. The property replace is
part of the template system. It's quite powerful, but you need to play a
bit with it.
I need a little more direction on this. I believe I understand, but I want
to make absolutely sure. Could you provide me an example of what would be
within the remote server's config and what the central server config would
look like?
>
> 4) During the processing of this information, whether it is the
> logs or the database inserts, we need to be able to parse this
> information, attempt to match using defined regular expressions and
> generate an email with the information matched. I saw an example of
> this somewhere, but after looking some, not a lot, I just have not
> found it again. Would you provide me with a few examples of efficient
> ways to accomplish this.?
I have none at hand, but the http://wiki.rsyslog.com probably has some.
If not, post one or two actual cases and I'll create a few samples of
property replacer statenets. You friend is this doc page:
http://www.rsyslog.com/doc-property_replacer.html
I want to look for things like the following:
Apr 28 07:31:28 centralca01 kernel: audit(1209393088.758:15174): user
pid=9509 uid=0 auid=0 msg='PAM: authentication acct="root" :
exe="/usr/sbin/sshd" (hostname=XXXXXXXXXXX, addr=XXX.XXX.XXX.XXX,
terminal=ssh res=success)'
With this, I want to be notified with the host information, address, etc.
via e-mail if someone logs into a particular host as "root".
>
> 5) Lastly, I am going to strongly recommend to all our clients use
> the products related to SYSLOG from Adiscon for us to be able to
> process information within this environment for Windows based
> machines. I would like to use the same table that is used for storage
> of the rest of SYSLOG data, and I have the associated columns already
> built. I just want to make sure that what I setup now will be
> completely compatible and able to process NT Event Log information.
>
The schema that comes with rsyslog is the "MonitorWare schema", which is
the same for the commercial softwares too.
I also suggest to have a look at the recently announced
http://www.phplogcon.org. As part of phpLogCon, we will define a special
Windows Event Format (based on our existing techology) that will be
understood by phpLogCon. I will make sure it is documented, so that you
can also subparse it. phpLogCon (also GPLv3) will contain the parser in
php, so you should be quite easy be able to adopt it.
I have taken a good look at it and will integrate it within our
administration interface. It looks like an excellent product and will save
me a lot of code to write.
>
>
> Once again, thanks for your time and look forward to hearing your
> thoughts related to this implementation. I have used SYSLOG-NG for
> years and found that it was great in some respects, but disappointing
> being able to do storage to MySQL. You have a great product here.
>
thanks, appreciated. Our long-term vision specifically includes
compliance, so I would be most interested in any requirement you have.
For example, I am currently implementing TLS, first for plain tcp
syslog, then for RELP. Digital message signatures are also on my list.
So any ideas/actual requirements are most welcome and I will happily
work together with you to get things done. My vision is *much* broader
than syslog-ng (at least as far as I know that project).
Rainer
Sent: Monday, April 28, 2008 8:55 AM
To: rsyslog at lists.adiscon.com; Rainer Gerhards
Subject: [rsyslog] RE: RSYSLOG "Best Practices" & General Questions
On Fri, 2008-04-25 at 10:05 -0600, Stephen Malenshek wrote:
> I am currently setting up creating a managed service platform from
> various open source products out on the market and I would like to use
> your product as the standard SYSLOG replacement on all our sites. I
> have a couple of questions related to this and would like you provide
> some input on the best ways to achieve specific objectives.
>
>
>
> 1) At the present time, I have started the configuration on the
> "central" server, which will act as the central repository for all
> data from the remote sites. I am configuring it to store all SYSLOG
> data with in the database, but I have followed the recommendations
> made to "buffer" it to a spool first. My question is this, I do not
> want to just write the information to the database, for governmental
> compliance, I need to keep a duplicate copy in "standard" log format
> on the drive, which I will rotate and gzip daily, for long term log
> retention. I have looked around and did not find anything that
> specifically addresses this. It looked like it should able to be
> done, but I am just not sure the best way to accomplish this.
>
Storing to the database is in no way special. So you can just add as
many other selector lines as you like. For example:
*.* :ommysql*...
*.* /var/log/soxstore
works perfectly. What you need to be aware is that the buffer parameters
($Action...) work on the NEXT action, so you want to have them directly
in front of the action in question.
In my present configuration, I am doing an if/then like the following:
if $source == 'somemachine' \
and $syslogfacility-text == 'authpriv' \
then ?LocalSecure
$template LocalSecure,"/var/log/secure"
With this, how can I define multiple directions within this template?
> 2) At each customer site, there will be a server called a
> "collector" that will accept all SYSLOG related information for that
> site. This server will store a copy of the log files for the local
> network as a repository, but it also needs to send it to the central
> server for processing. My question is whether it will be more
> efficient to write the information directly to the database, or to
> just send it using normal SYSLOG directives, 'I.E. *.* @{IP Address},
> and let the server process and insert like it would local logs?
>
that's a very good question. I'd say it depends on a lot of factors,
probablky most importantly from the clients ability to talk to the
database directly (this is often impossible due to firewalls). Also, you
may consider where the plain text files need to be written. If they
should be on a central system, I'd opt for moving everything to the
central server and writing it to database and/or text file there.
Please note that UDP (@IP) is a really bad protocol choice. You will
lose a lot of messages and it is definitely not
"compliance-compliant" ;). TCP based syslog is much better but not
ideal, see:
http://rgerhards.blogspot.com/2008/04/on-unreliability-of-plain-tcp-syslog.h
tml
rsyslog's solution is the relp protocol, which prevents message loss.
> 3) Within the scenario listed in question 2, how can I, 1)
> preserve all the original IP addresses of the machines that are
> transmitting information,
If you use rsyslog on the clients, too, this is automatically done for
you.
At each site, there will be a single server tat is aggregating all SYSLOG
data before transmitting it to the central repository. Are you talking
about the machine sending the aggregate information or the hosts, I.E.
router, server, switch, etc., that we will be monitoring?
> and 2) tag that information with a specific account code identifying
> the site that the information was sent from. Within the database, I
> have created a column called "customerid" that I would like to do this
> with. In this, I would like to designate a name or integer like "1"
> for site A, "2" for site B, etc. The reason for this is that I will
> run into situations where multiple customers will have the same IP
> addressing scheme. I figure this could be passed from the site's
> collector as a site identifier, but I am not sure how to accomplish
> this. I think I can accomplish this on the central server, if I have
> to, with a subquery within the insert query to another table to lookup
> this value, but I am looking for a much more "elegant" method.
You should look into template definitions. On the client, create a
template that contains the customer ID somewhere AFTER the tag. Maybe
immediately behind it with the rest of the message being separated by a
comma.
Then, on the server, look into the property replace. Use field-based
extraction to pull out that value and insert it. The property replace is
part of the template system. It's quite powerful, but you need to play a
bit with it.
I need a little more direction on this. I believe I understand, but I want
to make absolutely sure. Could you provide me an example of what would be
within the remote server's config and what the central server config would
look like?
>
> 4) During the processing of this information, whether it is the
> logs or the database inserts, we need to be able to parse this
> information, attempt to match using defined regular expressions and
> generate an email with the information matched. I saw an example of
> this somewhere, but after looking some, not a lot, I just have not
> found it again. Would you provide me with a few examples of efficient
> ways to accomplish this.?
I have none at hand, but the http://wiki.rsyslog.com probably has some.
If not, post one or two actual cases and I'll create a few samples of
property replacer statenets. You friend is this doc page:
http://www.rsyslog.com/doc-property_replacer.html
I want to look for things like the following:
Apr 28 07:31:28 centralca01 kernel: audit(1209393088.758:15174): user
pid=9509 uid=0 auid=0 msg='PAM: authentication acct="root" :
exe="/usr/sbin/sshd" (hostname=XXXXXXXXXXX, addr=XXX.XXX.XXX.XXX,
terminal=ssh res=success)'
With this, I want to be notified with the host information, address, etc.
via e-mail if someone logs into a particular host as "root".
>
> 5) Lastly, I am going to strongly recommend to all our clients use
> the products related to SYSLOG from Adiscon for us to be able to
> process information within this environment for Windows based
> machines. I would like to use the same table that is used for storage
> of the rest of SYSLOG data, and I have the associated columns already
> built. I just want to make sure that what I setup now will be
> completely compatible and able to process NT Event Log information.
>
The schema that comes with rsyslog is the "MonitorWare schema", which is
the same for the commercial softwares too.
I also suggest to have a look at the recently announced
http://www.phplogcon.org. As part of phpLogCon, we will define a special
Windows Event Format (based on our existing techology) that will be
understood by phpLogCon. I will make sure it is documented, so that you
can also subparse it. phpLogCon (also GPLv3) will contain the parser in
php, so you should be quite easy be able to adopt it.
I have taken a good look at it and will integrate it within our
administration interface. It looks like an excellent product and will save
me a lot of code to write.
>
>
> Once again, thanks for your time and look forward to hearing your
> thoughts related to this implementation. I have used SYSLOG-NG for
> years and found that it was great in some respects, but disappointing
> being able to do storage to MySQL. You have a great product here.
>
thanks, appreciated. Our long-term vision specifically includes
compliance, so I would be most interested in any requirement you have.
For example, I am currently implementing TLS, first for plain tcp
syslog, then for RELP. Digital message signatures are also on my list.
So any ideas/actual requirements are most welcome and I will happily
work together with you to get things done. My vision is *much* broader
than syslog-ng (at least as far as I know that project).
Rainer