Mailing List Archive: rsyslog with apache and per vhost log

rsyslog with apache and per vhost log

Mar 11, 2008, 5:01 AM

Post #1 of 18 (8849 views)

Hi all,
i set up an apache webserver wich logs with directives:
ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com"
CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " "combined"

then syslog forwards them with
local5.err @10.10.10.89
local6.info @10.10.10.89
and it works good!

now on 10.10.10.89 i set up an rsyslog server with directives
$template ApacheRemoteErr,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Err.log"
local5.err -?ApacheRemoteErr
$template ApacheRemoteCustom,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Custom.log"
local6.info -?ApacheRemoteCustom

the result is that i get these files in /var/log/rsyslog/Apache/
./www.mysite.com:/Custom.log
with customLog

./error_www.mysite.com:/Err.log
with ErroLog

./message/Err.log
with lines like
Mar 11 12:32:52 last message repeated 9 times
Mar 11 12:32:52 last message repeated 5 times
(Note that this is very annoying!!!)

Now this a simple way i have found to divide logs per VHost.
I don't know if there is a better way, but the real question is different:
In the Custom or Error logs i have lines like:
Mar 11 12:32:53 php03 www.mysite.com: 10.10.10.2 - -
[11/Mar/2008:12:35:07 +0100] "GET /pagina.php?id=61 HTTP/1.1" 200 93
"http://www.mysite.com/home.php" "Mozilla/5.0 (X11; U; Linux i686;
en-US; rv:1.7.13) Gecko/20060607"

i don't want the "Mar 11 12:32:53 php03 www.mysite.com:" part
do i have to use a double template setup? one for the file and one for
the content?

the template in the file seems to be this:
$template TraditionalFormat,%timegenerated% %HOSTNAME% %syslogtag%%msg%\n"
but i need just
$template TheOneINeed,%msg%\n"
i want only the %msg%\n in my file

Thanks in advance

--
mr

rsyslog with apache and per vhost log [ In reply to ]

rgerhards at hq

Mar 11, 2008, 1:45 PM

Post #2 of 18 (8642 views)

Permalink

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
> Maurizio Rottin
> Sent: Tuesday, March 11, 2008 1:02 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] rsyslog with apache and per vhost log
>
> Hi all,
> i set up an apache webserver wich logs with directives:
> ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com"
> CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com
> " "combined"
>
> then syslog forwards them with
> local5.err @10.10.10.89
> local6.info @10.10.10.89
> and it works good!
>
> now on 10.10.10.89 i set up an rsyslog server with directives
> $template
> ApacheRemoteErr,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Err.log"
> local5.err -?ApacheRemoteErr
> $template
> ApacheRemoteCustom,%msg%"/var/log/rsyslog/Apache/%syslogtag%/C
> ustom.log"
> local6.info -?ApacheRemoteCustom
>
> the result is that i get these files in /var/log/rsyslog/Apache/
> ./www.mysite.com:/Custom.log
> with customLog
>
> ./error_www.mysite.com:/Err.log
> with ErroLog
>
> ./message/Err.log
> with lines like
> Mar 11 12:32:52 last message repeated 9 times
> Mar 11 12:32:52 last message repeated 5 times
> (Note that this is very annoying!!!)

Add -e to rsyslogd startup and they go away ;) (this is part of sysklogd
compatibility, -e stand for "log *e*very message")

>
>
> Now this a simple way i have found to divide logs per VHost.
> I don't know if there is a better way, but the real question
> is different:
> In the Custom or Error logs i have lines like:
> Mar 11 12:32:53 php03 www.mysite.com: 10.10.10.2 - -
> [11/Mar/2008:12:35:07 +0100] "GET /pagina.php?id=61 HTTP/1.1" 200 93
> "http://www.mysite.com/home.php" "Mozilla/5.0 (X11; U; Linux i686;
> en-US; rv:1.7.13) Gecko/20060607"
>
> i don't want the "Mar 11 12:32:53 php03 www.mysite.com:" part
> do i have to use a double template setup? one for the file and one for
> the content?
>
This is caused by the default template, which is what most peiople
expect in their syslog files. But, as you write, the soluton is quite
easy, just use another template ;)

> the template in the file seems to be this:
> $template TraditionalFormat,%timegenerated% %HOSTNAME%
> %syslogtag%%msg%\n"
> but i need just
> $template TheOneINeed,%msg%\n"
> i want only the %msg%\n in my file
>

So, use

local6.info -?ApacheRemoteCustom;TheOneINeed

HTH, feel free to ask any follow-up questions. :)

Rainer
>
> Thanks in advance
>
> --
> mr
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>

rsyslog with apache and per vhost log [ In reply to ]

maurizio.rottin at gmail

Mar 12, 2008, 2:47 AM

Post #3 of 18 (8629 views)

Permalink

2008/3/11, Rainer Gerhards <rgerhards at hq.adiscon.com>:
>
>
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com
> > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
> > Maurizio Rottin
> > Sent: Tuesday, March 11, 2008 1:02 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: [rsyslog] rsyslog with apache and per vhost log
> >
> > Hi all,
> > i set up an apache webserver wich logs with directives:
> > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com"
> > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com
> > " "combined"
> >
> > then syslog forwards them with
> > local5.err @10.10.10.89
> > local6.info @10.10.10.89
> > and it works good!
> >
> > now on 10.10.10.89 i set up an rsyslog server with directives
> > $template
> > ApacheRemoteErr,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Err.log"
> > local5.err -?ApacheRemoteErr
> > $template
> > ApacheRemoteCustom,%msg%"/var/log/rsyslog/Apache/%syslogtag%/C
> > ustom.log"
> > local6.info -?ApacheRemoteCustom
> >
> > the result is that i get these files in /var/log/rsyslog/Apache/
> > ./www.mysite.com:/Custom.log
> > with customLog
> >
> > ./error_www.mysite.com:/Err.log
> > with ErroLog
> >
> > ./message/Err.log
> > with lines like
> > Mar 11 12:32:52 last message repeated 9 times
> > Mar 11 12:32:52 last message repeated 5 times
> > (Note that this is very annoying!!!)
>
>
> Add -e to rsyslogd startup and they go away ;) (this is part of sysklogd
> compatibility, -e stand for "log *e*very message")

And i red about the -e options a lot of times...thank!

>
> >
> >
> > Now this a simple way i have found to divide logs per VHost.
> > I don't know if there is a better way, but the real question
> > is different:
> > In the Custom or Error logs i have lines like:
> > Mar 11 12:32:53 php03 www.mysite.com: 10.10.10.2 - -
> > [11/Mar/2008:12:35:07 +0100] "GET /pagina.php?id=61 HTTP/1.1" 200 93
> > "http://www.mysite.com/home.php" "Mozilla/5.0 (X11; U; Linux i686;
> > en-US; rv:1.7.13) Gecko/20060607"
> >
> > i don't want the "Mar 11 12:32:53 php03 www.mysite.com:" part
> > do i have to use a double template setup? one for the file and one for
> > the content?
> >
>
> This is caused by the default template, which is what most peiople
> expect in their syslog files. But, as you write, the soluton is quite
> easy, just use another template ;)
>
>
> > the template in the file seems to be this:
> > $template TraditionalFormat,%timegenerated% %HOSTNAME%
> > %syslogtag%%msg%\n"
> > but i need just
> > $template TheOneINeed,%msg%\n"
> > i want only the %msg%\n in my file
> >
>
>
> So, use
>
> local6.info -?ApacheRemoteCustom;TheOneINeed

works great man! Thanks!

> HTH, feel free to ask any follow-up questions. :)
>

just one question, do you know any way to pass the website name in
"/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an
other program...i know only logger...

--
mr

rsyslog with apache and per vhost log [ In reply to ]

rgerhards at hq

Mar 12, 2008, 2:50 AM

Post #4 of 18 (8618 views)

Permalink

[happy snip]
> just one question, do you know any way to pass the website name in
> "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an
> other program...i know only logger...

Well, you could use the imfile plugin to read the log files directly:

http://www.rsyslog.com/doc-imfile.html

Then, I'd set the site name as tag and so you can process it from the
tag value.

HTH
Rainer

rsyslog with apache and per vhost log [ In reply to ]

maurizio.rottin at gmail

Mar 12, 2008, 3:24 AM

Post #5 of 18 (8624 views)

Permalink

2008/3/12, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> [happy snip]
>
> > just one question, do you know any way to pass the website name in
> > "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an
> > other program...i know only logger...
>
>
> Well, you could use the imfile plugin to read the log files directly:
>
> http://www.rsyslog.com/doc-imfile.html
>
> Then, I'd set the site name as tag and so you can process it from the
> tag value.
>
> HTH
>
> Rainer
>
mmmmm...i'll check that better but at a glance it seems a heavy
process with big logs.

--
mr

rsyslog with apache and per vhost log [ In reply to ]

rgerhards at hq

Mar 12, 2008, 3:25 AM

Post #6 of 18 (8610 views)

Permalink

> mmmmm...i'll check that better but at a glance it seems a heavy
> process with big logs.

Sorry, I don't really understand what you mean ;) But the problem is
that I probably do not really know how what you do with logger really
works. It is part of the apache config, is this right? Maybe you could
post a full config sample?

Rainer

rsyslog with apache and per vhost log [ In reply to ]

maurizio.rottin at gmail

Mar 12, 2008, 6:37 AM

Post #7 of 18 (8612 views)

Permalink

2008/3/12, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > mmmmm...i'll check that better but at a glance it seems a heavy
> > process with big logs.
>
>
> Sorry, I don't really understand what you mean ;) But the problem is
> that I probably do not really know how what you do with logger really
> works. It is part of the apache config, is this right? Maybe you could
> post a full config sample?
>
but i did...
this is the log part in the <virtualhost> directive
ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com"
CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " "combined"

in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with
facility "-p local*.xxx" with tag "-t www.mysite.com"

then i can use the tag to split logs by website in the remote rsyslog machine ;)

--
mr

rsyslog with apache and per vhost log [ In reply to ]

rgerhards at hq

Mar 12, 2008, 6:44 AM

Post #8 of 18 (8614 views)

Permalink

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin
> Sent: Wednesday, March 12, 2008 2:37 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog with apache and per vhost log
>
> 2008/3/12, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > mmmmm...i'll check that better but at a glance it seems a heavy
> > > process with big logs.
> >
> >
> > Sorry, I don't really understand what you mean ;) But the problem is
> > that I probably do not really know how what you do with logger
> really
> > works. It is part of the apache config, is this right? Maybe you
> could
> > post a full config sample?
> >
> but i did...
> this is the log part in the <virtualhost> directive
> ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com"
> CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com "
> "combined"
>
> in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with
> facility "-p local*.xxx" with tag "-t www.mysite.com"
>
> then i can use the tag to split logs by website in the remote rsyslog
> machine ;)

Ah, OK, I didn't realize this was actual Apache config. So your original
question

> just one question, do you know any way to pass the website name in
> "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an
> other program...i know only logger...

was how you could use the same apache config command for each log, so
that you don't need to have separate apache config statements for each
virtual server?

Please bear with me if I am getting it wrong, I probably have too much
rsyslog on my mind. I am trying to help but I don't get a grip at what
the real problem is...

Rainer

rsyslog with apache and per vhost log [ In reply to ]

maurizio.rottin at gmail

Mar 13, 2008, 1:03 AM

Post #9 of 18 (8627 views)

Permalink

2008/3/12, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin
>
> > Sent: Wednesday, March 12, 2008 2:37 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] rsyslog with apache and per vhost log
> >
> > 2008/3/12, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > > mmmmm...i'll check that better but at a glance it seems a heavy
> > > > process with big logs.
> > >
> > >
> > > Sorry, I don't really understand what you mean ;) But the problem is
> > > that I probably do not really know how what you do with logger
> > really
> > > works. It is part of the apache config, is this right? Maybe you
> > could
> > > post a full config sample?
> > >
> > but i did...
> > this is the log part in the <virtualhost> directive
> > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com"
> > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com "
> > "combined"
> >
> > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with
> > facility "-p local*.xxx" with tag "-t www.mysite.com"
> >
> > then i can use the tag to split logs by website in the remote rsyslog
> > machine ;)
>
>
> Ah, OK, I didn't realize this was actual Apache config. So your original
> question
>
>
> > just one question, do you know any way to pass the website name in
> > "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an
> > other program...i know only logger...
>
>
> was how you could use the same apache config command for each log, so
> that you don't need to have separate apache config statements for each
> virtual server?
>
> Please bear with me if I am getting it wrong, I probably have too much
> rsyslog on my mind. I am trying to help but I don't get a grip at what
> the real problem is...
>
>

Not really...i don't want to use logger, it becomes a heavy process
under high traffic load....but this should be a question for the
apache ML...

--
mr

rsyslog with apache and per vhost log [ In reply to ]

rgerhards at hq

Mar 13, 2008, 1:06 AM

Post #10 of 18 (8622 views)

Permalink

[sniiiiip]
> Not really...i don't want to use logger, it becomes a heavy process
> under high traffic load....but this should be a question for the
> apache ML...

OK, I don't know what makes it a heavy process... anyhow... If you come
up with an explanation or what would be less resource-intense (from an
apache point of view), please let me know. Maybe I can add a simple
little tool to do the job. But obviously I need to know what's wrong
with logger before even thinking about that ;)

Rainer

rsyslog with apache and per vhost log [ In reply to ]

maurizio.rottin at gmail

Mar 20, 2008, 7:20 AM

Post #11 of 18 (8641 views)

Permalink

2008/3/13, Rainer Gerhards <rgerhards at hq.adiscon.com>:

wow,
> [sniiiiip]
>
> > Not really...i don't want to use logger, it becomes a heavy process
> > under high traffic load....but this should be a question for the
> > apache ML...
>
>
> OK, I don't know what makes it a heavy process... anyhow... If you come
> up with an explanation or what would be less resource-intense (from an
> apache point of view), please let me know. Maybe I can add a simple
> little tool to do the job. But obviously I need to know what's wrong
> with logger before even thinking about that ;)
>
>

ok my ideas are a little more clear now, as you said:
> was how you could use the same apache config command for each log, so
> that you don't need to have separate apache config statements for each
> virtual server?
>
> Please bear with me if I am getting it wrong, I probably have too much
> rsyslog on my mind. I am trying to help but I don't get a grip at what
> the real problem is...
>
>

you were right...this is what i need!

just one command for ErrorLog and one for CustomLog

so as i posted now i'm using:
> > this is the log part in the <virtualhost> directive
> > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com"
> > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com "
> > "combined"
> >
> > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with
> > facility "-p local*.xxx" with tag "-t www.mysite.com"

for each virtualhost, but this mean that i have 2 loggers running for
every virtualhost.
This is why logger becomes a heavy process.

But if i use one default directive for every vhost, i don't know how
to split the log per vhost in the remote machine which collects the
logs...

Hope now is a little bit clearer...

now, new config in httpd.conf
"LogLevel warn
ErrorLog "|/usr/bin/logger -p local5.err"
LogFormat "%v %h %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" vcombined
CustomLog "|/usr/bin/logger -p local6.info " vcombined"

and the %v gives the virtualhost name, so now i've got only two loggers.

At this point how can i say to rsyslog to split the files according to
the first word which is in this case the vhost name?

Thanks for being patient!

--
mr

rsyslog with apache and per vhost log [ In reply to ]

rgerhards at hq

Mar 20, 2008, 7:25 AM

Post #12 of 18 (8620 views)

Permalink

Can you send me a handful of the logline to play with? Probably not this
week, but next...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin
> Sent: Thursday, March 20, 2008 3:20 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog with apache and per vhost log
>
> 2008/3/13, Rainer Gerhards <rgerhards at hq.adiscon.com>:
>
> wow,
> > [sniiiiip]
> >
> > > Not really...i don't want to use logger, it becomes a heavy
process
> > > under high traffic load....but this should be a question for the
> > > apache ML...
> >
> >
> > OK, I don't know what makes it a heavy process... anyhow... If you
> come
> > up with an explanation or what would be less resource-intense (from
> an
> > apache point of view), please let me know. Maybe I can add a simple
> > little tool to do the job. But obviously I need to know what's
wrong
> > with logger before even thinking about that ;)
> >
> >
>
> ok my ideas are a little more clear now, as you said:
> > was how you could use the same apache config command for each log,
so
> > that you don't need to have separate apache config statements for
> each
> > virtual server?
> >
> > Please bear with me if I am getting it wrong, I probably have too
> much
> > rsyslog on my mind. I am trying to help but I don't get a grip at
> what
> > the real problem is...
> >
> >
>
> you were right...this is what i need!
>
> just one command for ErrorLog and one for CustomLog
>
> so as i posted now i'm using:
> > > this is the log part in the <virtualhost> directive
> > > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com"
> > > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com "
> > > "combined"
> > >
> > > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger
> with
> > > facility "-p local*.xxx" with tag "-t www.mysite.com"
>
> for each virtualhost, but this mean that i have 2 loggers running for
> every virtualhost.
> This is why logger becomes a heavy process.
>
> But if i use one default directive for every vhost, i don't know how
> to split the log per vhost in the remote machine which collects the
> logs...
>
> Hope now is a little bit clearer...
>
> now, new config in httpd.conf
> "LogLevel warn
> ErrorLog "|/usr/bin/logger -p local5.err"
> LogFormat "%v %h %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
> vcombined
> CustomLog "|/usr/bin/logger -p local6.info " vcombined"
>
> and the %v gives the virtualhost name, so now i've got only two
> loggers.
>
> At this point how can i say to rsyslog to split the files according to
> the first word which is in this case the vhost name?
>
> Thanks for being patient!
>
> --
> mr
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

rsyslog with apache and per vhost log [ In reply to ]

maurizio.rottin at gmail

Mar 20, 2008, 7:46 AM

Post #13 of 18 (8614 views)

Permalink

2008/3/20, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Can you send me a handful of the logline to play with? Probably not this
> week, but next...
>

www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET
/images/wm001.jpg HTTP/1.1" 304 - "http://www.mysite.com/webmail.htm"
"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607"

i'm trying to use the regexp but with no success in this way:
$template MsgFormat,"%msg%\n"
$template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\
[a-z,\.]*--end%_az.log"
local6.info -?ApacheRemoteCustom;MsgFormat

from the documentation: "the property replacer will return the part of
the property text that matches the regular expression" which should be
" www.mysite.com"
but i get a file named _az.log

--
mr

rsyslog with apache and per vhost log [ In reply to ]

rgerhards at hq

Mar 20, 2008, 7:49 AM

Post #14 of 18 (8615 views)

Permalink

Let me try to avoid the regexp (its expensive and I can not debug it now
;)): so you search for the string that is at the start of the msg and
delimited by the first space?

Rainer
> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin
> Sent: Thursday, March 20, 2008 3:47 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog with apache and per vhost log
>
> 2008/3/20, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Can you send me a handful of the logline to play with? Probably not
> this
> > week, but next...
> >
>
> www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET
> /images/wm001.jpg HTTP/1.1" 304 - "http://www.mysite.com/webmail.htm"
> "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607"
>
> i'm trying to use the regexp but with no success in this way:
> $template MsgFormat,"%msg%\n"
> $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\
> [a-z,\.]*--end%_az.log"
> local6.info -?ApacheRemoteCustom;MsgFormat
>
> from the documentation: "the property replacer will return the part of
> the property text that matches the regular expression" which should be
> " www.mysite.com"
> but i get a file named _az.log
>
> --
> mr
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

rsyslog with apache and per vhost log [ In reply to ]

maurizio.rottin at gmail

Mar 20, 2008, 7:59 AM

Post #15 of 18 (8622 views)

Permalink

yes! but actually there is a space at the beginning and hostname can
contain the dash -, numbers, and letters.

2008/3/20, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Let me try to avoid the regexp (its expensive and I can not debug it now
> ;)): so you search for the string that is at the start of the msg and
> delimited by the first space?
>
>
>
> Rainer
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin
>
> > Sent: Thursday, March 20, 2008 3:47 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] rsyslog with apache and per vhost log
> >
>
> > 2008/3/20, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > this
> > > week, but next...
> > >
> >
> > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET
> > /images/wm001.jpg HTTP/1.1" 304 - "http://www.mysite.com/webmail.htm"
> > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607"
> >
> > i'm trying to use the regexp but with no success in this way:
> > $template MsgFormat,"%msg%\n"
> > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\
> > [a-z,\.]*--end%_az.log"
> > local6.info -?ApacheRemoteCustom;MsgFormat
> >
> > from the documentation: "the property replacer will return the part of
> > the property text that matches the regular expression" which should be
> > " www.mysite.com"
> > but i get a file named _az.log
> >
> > --
> > mr
>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>

--
mr

rsyslog with apache and per vhost log [ In reply to ]

rgerhards at hq

Mar 20, 2008, 8:09 AM

Post #16 of 18 (8626 views)

Permalink

Should work with fields (much faster). I can't try it out due to relp
work, but try:

%msg:F,32:2% [32 is USASCII SP, the delimiter here]

But maybe %msg:F,32:1% - you need to experiment a bit. In any case, that
should work...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin
> Sent: Thursday, March 20, 2008 3:59 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog with apache and per vhost log
>
> yes! but actually there is a space at the beginning and hostname can
> contain the dash -, numbers, and letters.
>
> 2008/3/20, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Let me try to avoid the regexp (its expensive and I can not debug it
> now
> > ;)): so you search for the string that is at the start of the msg
> and
> > delimited by the first space?
> >
> >
> >
> > Rainer
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin
> >
> > > Sent: Thursday, March 20, 2008 3:47 PM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log
> > >
> >
> > > 2008/3/20, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > > Can you send me a handful of the logline to play with? Probably
> not
> > > this
> > > > week, but next...
> > > >
> > >
> > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET
> > > /images/wm001.jpg HTTP/1.1" 304 -
> "http://www.mysite.com/webmail.htm"
> > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13)
> Gecko/20060607"
> > >
> > > i'm trying to use the regexp but with no success in this way:
> > > $template MsgFormat,"%msg%\n"
> > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\
> > > [a-z,\.]*--end%_az.log"
> > > local6.info -?ApacheRemoteCustom;MsgFormat
> > >
> > > from the documentation: "the property replacer will return the
> part of
> > > the property text that matches the regular expression" which
> should be
> > > " www.mysite.com"
> > > but i get a file named _az.log
> > >
> > > --
> > > mr
> >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
>
>
> --
> mr
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

rsyslog with apache and per vhost log [ In reply to ]

maurizio.rottin at gmail

Mar 20, 2008, 8:22 AM

Post #17 of 18 (8613 views)

Permalink

wow!
%msg:F,32:2% works!
i completely misunderstood how Fielf works...i thought the first
number was the FromChar and second the ToChar...
Thanks a lot!!!

2008/3/20, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Should work with fields (much faster). I can't try it out due to relp
> work, but try:
>
> %msg:F,32:2% [32 is USASCII SP, the delimiter here]
>
> But maybe %msg:F,32:1% - you need to experiment a bit. In any case, that
> should work...
>
>
> Rainer
>
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin
>
> > Sent: Thursday, March 20, 2008 3:59 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] rsyslog with apache and per vhost log
> >
> > yes! but actually there is a space at the beginning and hostname can
> > contain the dash -, numbers, and letters.
> >
> > 2008/3/20, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > Let me try to avoid the regexp (its expensive and I can not debug it
> > now
> > > ;)): so you search for the string that is at the start of the msg
> > and
> > > delimited by the first space?
> > >
> > >
> > >
> > > Rainer
> > > > -----Original Message-----
> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin
> > >
> > > > Sent: Thursday, March 20, 2008 3:47 PM
> > > > To: rsyslog-users
> > > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log
> > > >
> > >
> > > > 2008/3/20, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > > > Can you send me a handful of the logline to play with? Probably
> > not
> > > > this
> > > > > week, but next...
> > > > >
> > > >
> > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET
> > > > /images/wm001.jpg HTTP/1.1" 304 -
> > "http://www.mysite.com/webmail.htm"
> > > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13)
> > Gecko/20060607"
> > > >
> > > > i'm trying to use the regexp but with no success in this way:
> > > > $template MsgFormat,"%msg%\n"
> > > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\
> > > > [a-z,\.]*--end%_az.log"
> > > > local6.info -?ApacheRemoteCustom;MsgFormat
> > > >
> > > > from the documentation: "the property replacer will return the
> > part of
> > > > the property text that matches the regular expression" which
> > should be
> > > > " www.mysite.com"
> > > > but i get a file named _az.log
> > > >
> > > > --
> > > > mr
> > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >
> >
> >
> > --
> > mr
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>

--
mr

rsyslog with apache and per vhost log [ In reply to ]

rgerhards at hq

Mar 20, 2008, 8:39 AM

Post #18 of 18 (8641 views)

Permalink

Excellent, good to hear :)

Just a side-note: when I am done with relp, I'll hopefully have time to
come back to the config format. Then we'll also see much more powerful
ways to extract values...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin
> Sent: Thursday, March 20, 2008 4:22 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog with apache and per vhost log
>
> wow!
> %msg:F,32:2% works!
> i completely misunderstood how Fielf works...i thought the first
> number was the FromChar and second the ToChar...
> Thanks a lot!!!
>
> 2008/3/20, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Should work with fields (much faster). I can't try it out due to
relp
> > work, but try:
> >
> > %msg:F,32:2% [32 is USASCII SP, the delimiter here]
> >
> > But maybe %msg:F,32:1% - you need to experiment a bit. In any case,
> that
> > should work...
> >
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin
> >
> > > Sent: Thursday, March 20, 2008 3:59 PM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log
> > >
> > > yes! but actually there is a space at the beginning and hostname
> can
> > > contain the dash -, numbers, and letters.
> > >
> > > 2008/3/20, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > > Let me try to avoid the regexp (its expensive and I can not
> debug it
> > > now
> > > > ;)): so you search for the string that is at the start of the
> msg
> > > and
> > > > delimited by the first space?
> > > >
> > > >
> > > >
> > > > Rainer
> > > > > -----Original Message-----
> > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin
> > > >
> > > > > Sent: Thursday, March 20, 2008 3:47 PM
> > > > > To: rsyslog-users
> > > > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log
> > > > >
> > > >
> > > > > 2008/3/20, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > > > > Can you send me a handful of the logline to play with?
> Probably
> > > not
> > > > > this
> > > > > > week, but next...
> > > > > >
> > > > >
> > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100]
> "GET
> > > > > /images/wm001.jpg HTTP/1.1" 304 -
> > > "http://www.mysite.com/webmail.htm"
> > > > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13)
> > > Gecko/20060607"
> > > > >
> > > > > i'm trying to use the regexp but with no success in this
way:
> > > > > $template MsgFormat,"%msg%\n"
> > > > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\
> > > > > [a-z,\.]*--end%_az.log"
> > > > > local6.info -?ApacheRemoteCustom;MsgFormat
> > > > >
> > > > > from the documentation: "the property replacer will return
> the
> > > part of
> > > > > the property text that matches the regular expression" which
> > > should be
> > > > > " www.mysite.com"
> > > > > but i get a file named _az.log
> > > > >
> > > > > --
> > > > > mr
> > > >
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >
> > >
> > >
> > > --
> > > mr
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
>
>
> --
> mr
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog