Mailing List Archive: Right regex format for property based filters

Right regex format for property based filters

Mar 4, 2009, 7:20 AM

Post #1 of 6 (3188 views)

Hello list.

I have a question regarding to rsyslog configuration. What is the
correct syntax of property based
filter with regex.

I'm using this configuration right now, and would like to create date
based logfiles for each host - hostA, hostB, hostC.
But it doesn't work this way.

$template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%-%$DAY%.log"
:HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile

And when running rsyslog with -d, I got only false matches on this
regex. I seems that it tries to match all the text
inside quotes instead of regexp. As I have red in man page, and html
docs, then regexp should be in POSIX RE format
(tryed also everything enclosed in braces). For example, if I change
regex like this:

:HOSTNAME, regex, "host" -?TplFile

Then it works and matches all the hosts (A,B,C), and creates the files
for each (well it's the same as using contains).
But that doesn't solve the problem, when there isn't equal start
prefixes for all hosts.
For example if I want to match hosts - dog,cat,cow.

Best regards
--janis
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

Re: Right regex format for property based filters [ In reply to ]

rgerhards at hq

Mar 4, 2009, 7:35 AM

Post #2 of 6 (3123 views)

Permalink

Hi Janis,

the regex is Posix BRE, nor ERE. I think the syntax you use is not
supported in BRE (as a side-note, this reminds me that I wanted to check
what it takes to upgrade them to use ERE, too).

Rainer

> -----Original Message-----
> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
> bounces@lists.adiscon.com] On Behalf Of Janis
> Sent: Wednesday, March 04, 2009 4:20 PM
> To: rsyslog@lists.adiscon.com
> Subject: [rsyslog] Right regex format for property based filters
>
> Hello list.
>
> I have a question regarding to rsyslog configuration. What is the
> correct syntax of property based
> filter with regex.
>
> I'm using this configuration right now, and would like to create date
> based logfiles for each host - hostA, hostB, hostC.
> But it doesn't work this way.
>
> $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%-
> %$DAY%.log"
> :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile
>
> And when running rsyslog with -d, I got only false matches on this
> regex. I seems that it tries to match all the text
> inside quotes instead of regexp. As I have red in man page, and html
> docs, then regexp should be in POSIX RE format
> (tryed also everything enclosed in braces). For example, if I change
> regex like this:
>
> :HOSTNAME, regex, "host" -?TplFile
>
> Then it works and matches all the hosts (A,B,C), and creates the files
> for each (well it's the same as using contains).
> But that doesn't solve the problem, when there isn't equal start
> prefixes for all hosts.
> For example if I want to match hosts - dog,cat,cow.
>
> Best regards
> --janis
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

Re: Right regex format for property based filters [ In reply to ]

rgerhards at hq

Mar 4, 2009, 9:03 AM

Post #3 of 6 (3122 views)

Permalink

ERE looks trivial - just seeing if I get it in...

> -----Original Message-----
> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
> bounces@lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Wednesday, March 04, 2009 4:36 PM
> To: janisg@latnetdc.lv; rsyslog-users
> Subject: Re: [rsyslog] Right regex format for property based filters
>
> Hi Janis,
>
> the regex is Posix BRE, nor ERE. I think the syntax you use is not
> supported in BRE (as a side-note, this reminds me that I wanted to
> check
> what it takes to upgrade them to use ERE, too).
>
> Rainer
>
> > -----Original Message-----
> > From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
> > bounces@lists.adiscon.com] On Behalf Of Janis
> > Sent: Wednesday, March 04, 2009 4:20 PM
> > To: rsyslog@lists.adiscon.com
> > Subject: [rsyslog] Right regex format for property based filters
> >
> > Hello list.
> >
> > I have a question regarding to rsyslog configuration. What is the
> > correct syntax of property based
> > filter with regex.
> >
> > I'm using this configuration right now, and would like to create
date
> > based logfiles for each host - hostA, hostB, hostC.
> > But it doesn't work this way.
> >
> > $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%-
> > %$DAY%.log"
> > :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile
> >
> > And when running rsyslog with -d, I got only false matches on this
> > regex. I seems that it tries to match all the text
> > inside quotes instead of regexp. As I have red in man page, and html
> > docs, then regexp should be in POSIX RE format
> > (tryed also everything enclosed in braces). For example, if I change
> > regex like this:
> >
> > :HOSTNAME, regex, "host" -?TplFile
> >
> > Then it works and matches all the hosts (A,B,C), and creates the
> files
> > for each (well it's the same as using contains).
> > But that doesn't solve the problem, when there isn't equal start
> > prefixes for all hosts.
> > For example if I want to match hosts - dog,cat,cow.
> >
> > Best regards
> > --janis
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

Re: Right regex format for property based filters [ In reply to ]

rgerhards at hq

Mar 4, 2009, 9:38 AM

Post #4 of 6 (3118 views)

Permalink

Janis,

I have added ERE filter support to the devel branch and your use case
described below now works - you just need to use "ereregexp" instead of
"regexp". No release tarball yet, the patch is here:

http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=5005bce38763051b5b12e48ac60c3ff17097a952

I did some quick checks, but would appreciate if some others try it out.

Rainer

On Wed, 2009-03-04 at 17:20 +0200, Janis wrote:
> Hello list.
>
> I have a question regarding to rsyslog configuration. What is the
> correct syntax of property based
> filter with regex.
>
> I'm using this configuration right now, and would like to create date
> based logfiles for each host - hostA, hostB, hostC.
> But it doesn't work this way.
>
> $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%-%$DAY%.log"
> :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile
>
> And when running rsyslog with -d, I got only false matches on this
> regex. I seems that it tries to match all the text
> inside quotes instead of regexp. As I have red in man page, and html
> docs, then regexp should be in POSIX RE format
> (tryed also everything enclosed in braces). For example, if I change
> regex like this:
>
> :HOSTNAME, regex, "host" -?TplFile
>
> Then it works and matches all the hosts (A,B,C), and creates the files
> for each (well it's the same as using contains).
> But that doesn't solve the problem, when there isn't equal start
> prefixes for all hosts.
> For example if I want to match hosts - dog,cat,cow.
>
> Best regards
> --janis
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

Re: Right regex format for property based filters [ In reply to ]

rgerhards at hq

Mar 4, 2009, 9:56 AM

Post #5 of 6 (3118 views)

Permalink

All,

I introduced a memory leak with the ERE enhancement. It is fixed now. So
be sure to apply all patches after the one I mentioned.

For your convenience, I created a temporary tarball based on the fixed
version. It is available at

http://download.rsyslog.com/rsyslog/tmp.tar.gz

The tarball claims to contain 4.1.4, but you should not count on that it
is equal to the released version. I will *not* care any more about this
tarball. But I think it is useful to have a version right at hand. Also,
this doesn't require any autotools tricks ;)

Rainer

On Wed, 2009-03-04 at 18:38 +0100, Rainer Gerhards wrote:
> Janis,
>
> I have added ERE filter support to the devel branch and your use case
> described below now works - you just need to use "ereregexp" instead of
> "regexp". No release tarball yet, the patch is here:
>
> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=5005bce38763051b5b12e48ac60c3ff17097a952
>
> I did some quick checks, but would appreciate if some others try it out.
>
> Rainer
>
> On Wed, 2009-03-04 at 17:20 +0200, Janis wrote:
> > Hello list.
> >
> > I have a question regarding to rsyslog configuration. What is the
> > correct syntax of property based
> > filter with regex.
> >
> > I'm using this configuration right now, and would like to create date
> > based logfiles for each host - hostA, hostB, hostC.
> > But it doesn't work this way.
> >
> > $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%-%$DAY%.log"
> > :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile
> >
> > And when running rsyslog with -d, I got only false matches on this
> > regex. I seems that it tries to match all the text
> > inside quotes instead of regexp. As I have red in man page, and html
> > docs, then regexp should be in POSIX RE format
> > (tryed also everything enclosed in braces). For example, if I change
> > regex like this:
> >
> > :HOSTNAME, regex, "host" -?TplFile
> >
> > Then it works and matches all the hosts (A,B,C), and creates the files
> > for each (well it's the same as using contains).
> > But that doesn't solve the problem, when there isn't equal start
> > prefixes for all hosts.
> > For example if I want to match hosts - dog,cat,cow.
> >
> > Best regards
> > --janis
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

Re: Right regex format for property based filters [ In reply to ]

mbiebl at gmail

Mar 4, 2009, 11:39 AM

Post #6 of 6 (3122 views)

Permalink

2009/3/4 Rainer Gerhards <rgerhards@hq.adiscon.com>:
> All,
>
> I introduced a memory leak with the ERE enhancement. It is fixed now. So
> be sure to apply all patches after the one I mentioned.
>
> For your convenience, I created a temporary tarball based on the fixed
> version. It is available at
>
> http://download.rsyslog.com/rsyslog/tmp.tar.gz
>
> The tarball claims to contain 4.1.4, but you should not count on that it
> is equal to the released version. I will *not* care any more about this
> tarball. But I think it is useful to have a version right at hand. Also,
> this doesn't require any autotools tricks ;)

Rainer, gitweb has the nice snapshot feature, which allows to download
a tarball for a given SHA1 [1]
It doesn't contain the build system, so requires a "autoreconf -vfi"
run, but otherwise it should work just fine.

Cheers,
Michael

[1] http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=42db7de5968d2db0fa855a9f029f6bccc0a30650;sf=tgz
--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com