Mailing List Archive

On Mon, 20 Oct 2008, Mikel Jimenez wrote:

> Date: Mon, 20 Oct 2008 21:54:42 +0200
> From: Mikel Jimenez <mikel@irontec.com>
> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
> To: rsyslog-users <rsyslog@lists.adiscon.com>
> Subject: [rsyslog] db template how to
>
> Hello
> How can I make a template of sql in rsyslog.conf that makes that? :
>
> hostname1 >database.hostname1
> hostname2 >database.hostname2
> hostanem3 >database.hostname3
> .....

first a disclaimer, I haven't used the database output yet, so I could be
wrong or missing something easier that what I'm suggesting.



putting them into different databases would be difficult and expensive,
but possible.

if you are putting them into different tables inside one database you
could change from something like

"insert into syslog(message) values ('%msg%')"

to something like

"insert into syslog||'.'||%hostname% (message) values ('%msg%')"

since you are issuing SQL commands, you could do something along the lines
of the following to do different databases

"connect database||'.'||%hostname% ; insert into syslog(message) values ('%msg%'):

to switch the the appropriate databases. (note that this is usually _very_
inefficiant to do)

I don't know if you can do multiple database output modules going to
different systems, but if you can you may be able to do something nicer by
defining a different output module for each hostname and use filters to
seperate the traffic (this will obviously only work for a limited number
of hostnames, and they all have to be known ahead of time)

>
> and
>
> hostnameaa
> hostnameab
> hostnamedd
> hostnamede
>
> hostnamea*>database.a
> hostnamed*>database.d

this could be similar, but you would have to do string manipulation of the
hostname value to extract out just the character you need.

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

On Mon, Oct 20, 2008 at 14:44, <david@lang.hm> wrote:
> putting them into different databases would be difficult and expensive,
> but possible.

Hopefully we're not talking different databases, per se, maybe different tables?

My suggestion wouldn't be to put any of that logic at all in your
rsyslog configuration - the application & configurations are complex
enough. Rather (if you have a proper database engine) use table
partitioning, even constraint-based partitioning to do what you need.
Much less horrifying on the client side and any competent DBA will
know what you're doing and be able to handle it. Off the top of my
head I know Oracle, PostgreSQL, and MySQL support partitioning and I'd
be willing to bet MSSQL does as well.

Perhaps a more appropriate question would be why? Other than your
mental organization and maybe some access-control concerns, why would
you want to divide your database by host? Not only does it make
global correlation more complex, it's practically guaranteed to have
an uneven distribution.

When doing log tables, I always arrange the MERGE/partition constraint
by date or the like so it's easier and often faster for limiting
searches as well as to age data out (DROP TABLE versus DELETE FROM).
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com