Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. RSyslog>
  3. users
properties not getting filled in correctly
david at lang
Mar 6, 2009, 10:53 AM
Post #1 of 11 (3366 views)
Permalink
I'm running into problems trying to do filtering. it looks as if the log
parsing is not properly filling in the properties.

what I've run into so far

when I use the property 'programname' the content that I see is what I
would expect in 'hostname'

when I use the property 'hostname' the content that I see is what I would
expect in 'fromhost'

I haven't checked all the other properties, but my guess is that somehow
rsyslog is off-by-one in filling them in.

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
Re: properties not getting filled in correctly [ In reply to ]
david at lang
Mar 6, 2009, 10:54 AM
Post #2 of 11 (3337 views)
Permalink
On Fri, 6 Mar 2009, david@lang.hm wrote:

> I'm running into problems trying to do filtering. it looks as if the log
> parsing is not properly filling in the properties.
>
> what I've run into so far
>
> when I use the property 'programname' the content that I see is what I would
> expect in 'hostname'
>
> when I use the property 'hostname' the content that I see is what I would
> expect in 'fromhost'
>
> I haven't checked all the other properties, but my guess is that somehow
> rsyslog is off-by-one in filling them in.

having said this, date, fromhost, and from-ip appear to be filled in
correctly.

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
Re: properties not getting filled in correctly [ In reply to ]
rgerhards at hq
Mar 6, 2009, 10:54 AM
Post #3 of 11 (3345 views)
Permalink
That's why I am after the log samples :) I just termed a new acronym
this afternoon:
YAMSF - yet another malformed syslog format ;)

http://blog.gerhards.net/2009/02/calling-for-log-samples.html

I try hard to get the fields right, but often this is impossible,
resulting in the issues you see.

Rainer

> -----Original Message-----
> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
> bounces@lists.adiscon.com] On Behalf Of david@lang.hm
> Sent: Friday, March 06, 2009 7:54 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] properties not getting filled in correctly
>
> On Fri, 6 Mar 2009, david@lang.hm wrote:
>
> > I'm running into problems trying to do filtering. it looks as if the
> log
> > parsing is not properly filling in the properties.
> >
> > what I've run into so far
> >
> > when I use the property 'programname' the content that I see is what
> I would
> > expect in 'hostname'
> >
> > when I use the property 'hostname' the content that I see is what I
> would
> > expect in 'fromhost'
> >
> > I haven't checked all the other properties, but my guess is that
> somehow
> > rsyslog is off-by-one in filling them in.
>
> having said this, date, fromhost, and from-ip appear to be filled in
> correctly.
>
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
Re: properties not getting filled in correctly [ In reply to ]
david at lang
Mar 6, 2009, 5:25 PM
Post #4 of 11 (3336 views)
Permalink
On Fri, 6 Mar 2009, Rainer Gerhards wrote:

> That's why I am after the log samples :) I just termed a new acronym
> this afternoon:
> YAMSF - yet another malformed syslog format ;)
>
> http://blog.gerhards.net/2009/02/calling-for-log-samples.html
>
> I try hard to get the fields right, but often this is impossible,
> resulting in the issues you see.

these logs come from several different servers, including different OSs,
but all are misparsed by rsyslog.

I am not seeing anything obviously wrong with them

<167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601
<29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029
<29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1
<29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667
<22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=<blah@HOTMAIL.COM>, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106@w31.diginsite.com> Queued mail for delivery)
<29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw

David Lang

> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
>> bounces@lists.adiscon.com] On Behalf Of david@lang.hm
>> Sent: Friday, March 06, 2009 7:54 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] properties not getting filled in correctly
>>
>> On Fri, 6 Mar 2009, david@lang.hm wrote:
>>
>>> I'm running into problems trying to do filtering. it looks as if the
>> log
>>> parsing is not properly filling in the properties.
>>>
>>> what I've run into so far
>>>
>>> when I use the property 'programname' the content that I see is what
>> I would
>>> expect in 'hostname'
>>>
>>> when I use the property 'hostname' the content that I see is what I
>> would
>>> expect in 'fromhost'
>>>
>>> I haven't checked all the other properties, but my guess is that
>> somehow
>>> rsyslog is off-by-one in filling them in.
>>
>> having said this, date, fromhost, and from-ip appear to be filled in
>> correctly.
>>
>> David Lang
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
Re: properties not getting filled in correctly [ In reply to ]
david at lang
Mar 6, 2009, 6:55 PM
Post #5 of 11 (3342 views)
Permalink
On Fri, 6 Mar 2009, david@lang.hm wrote:

> On Fri, 6 Mar 2009, Rainer Gerhards wrote:
>
>> That's why I am after the log samples :) I just termed a new acronym
>> this afternoon:
>> YAMSF - yet another malformed syslog format ;)
>>
>> http://blog.gerhards.net/2009/02/calling-for-log-samples.html
>>
>> I try hard to get the fields right, but often this is impossible,
>> resulting in the issues you see.
>
> these logs come from several different servers, including different OSs,
> but all are misparsed by rsyslog.
>
> I am not seeing anything obviously wrong with them
>
> <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601
> <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029
> <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1
> <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667
> <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=<blah@HOTMAIL.COM>, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106@w31.diginsite.com> Queued mail for delivery)
> <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw

doing some more digging I see some very definant problems

I created the following template

$template DumpAll,"msg =%msg%\nrawmsg =%rawmsg%\nuxtradmsg =%uxtradmsg%\nhostname =%hostname%\nsource =%source%\nfromhost =%fromhost%\nfromhost-ip =%fromhost-ip%\nsyslogtag =%syslogtag%\nprogramname =%programname%\npri =%pri%\npri-text =%pri-text%\niut =%iut%\nsyslogfacility =%syslogfacility%\nsyslogfacility-text =%syslogfacility-text%\nsyslogseverity =%syslogseverity%\nsyslogseverity-text =%syslogseverity-text%\nsyslogpriority =%syslogpriority%\nsyslogpriority-text =%syslogpriority-text%\ntimegenerated =%timegenerated%\ntimereported =%timereported%\ntimestamp =%timestamp%\nprotocol-version =%protocol-version%\nstructured-data =%structured-data%\napp-name =%app-name%\nprocid =%procid%\nmsgid =%msgid%\ninputname =%inputname%\n\n"

which creates a nice table for each log message showing what's in each
property.

things that I am seeing

hostname and source are fromhost rather than the name/IP that's in the
record.

msg includes the programname

programname and appname are what hostname should be

David Lang

msg = %PIX-7-710005: UDP request discarded from BOK37UAT/3683 to test_app:255.255.255.255/61601
rawmsg =<167>Mar 6 18:33:47 172.20.245.8 %PIX-7-710005: UDP request discarded from BOK37UAT/3683 to test_app:255.255.255.255/61601
uxtradmsg =Mar 6 18:33:47 172.20.245.8 %PIX-7-710005: UDP request discarded from BOK37UAT/3683 to test_app:255.255.255.255/61601
hostname =itascan1a-p
source =itascan1a-p
fromhost =itascan1a-p
fromhost-ip =192.168.210.6
syslogtag =172.20.245.8
programname =172.20.245.8
pri =167
pri-text =local4.debug<167>
iut =1
syslogfacility =20
syslogfacility-text =local4
syslogseverity =7
syslogseverity-text =debug
syslogpriority =7
syslogpriority-text =debug
timegenerated =Mar 7 02:33:47
timereported =Mar 6 18:33:47
timestamp =Mar 6 18:33:47
protocol-version =0
structured-data =-
app-name =172.20.245.8
procid =-
msgid =-
inputname =imudp

msg = plug-gw[28055]: disconnect host= /192.168.242.212 destination=179.50.100.130/12773 in=0 out=0 duration=0
rawmsg =<29>Mar 6 18:33:47 methane1d-b plug-gw[28055]: disconnect host= /192.168.242.212 destination=179.50.100.130/12773 in=0 out=0 duration=0
uxtradmsg =Mar 6 18:33:47 methane1d-b plug-gw[28055]: disconnect host= /192.168.242.212 destination=179.50.100.130/12773 in=0 out=0 duration=0
hostname =itascan1a-p
source =itascan1a-p
fromhost =itascan1a-p
fromhost-ip =192.168.210.6
syslogtag =methane1d-b
programname =methane1d-b
pri =29
pri-text =daemon.notice<29>
iut =1
syslogfacility =3
syslogfacility-text =daemon
syslogseverity =5
syslogseverity-text =notice
syslogpriority =5
syslogpriority-text =notice
timegenerated =Mar 7 02:33:47
timereported =Mar 6 18:33:47
timestamp =Mar 6 18:33:47
protocol-version =0
structured-data =-
app-name =methane1d-b
procid =-
msgid =-
inputname =imudp

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
Re: properties not getting filled in correctly [ In reply to ]
rgerhards at hq
Mar 7, 2009, 1:47 AM
Post #6 of 11 (3335 views)
Permalink
The messages indeed look ok. I'll feed them into my parser and will see what happens.

rainer

----- Ursprüngliche Nachricht -----
Von: "david@lang.hm" <david@lang.hm>
An: "rsyslog-users" <rsyslog@lists.adiscon.com>
Gesendet: 07.03.09 02:20
Betreff: Re: [rsyslog] properties not getting filled in correctly

On Fri, 6 Mar 2009, Rainer Gerhards wrote:

> That's why I am after the log samples :) I just termed a new acronym
> this afternoon:
> YAMSF - yet another malformed syslog format ;)
>
> http://blog.gerhards.net/2009/02/calling-for-log-samples.html
>
> I try hard to get the fields right, but often this is impossible,
> resulting in the issues you see.

these logs come from several different servers, including different OSs,
but all are misparsed by rsyslog.

I am not seeing anything obviously wrong with them

<167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601
<29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029
<29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1
<29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667
<22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=<blah@HOTMAIL.COM>, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106@w31.diginsite.com> Queued mail for delivery)
<29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw

David Lang

> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
>> bounces@lists.adiscon.com] On Behalf Of david@lang.hm
>> Sent: Friday, March 06, 2009 7:54 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] properties not getting filled in correctly
>>
>> On Fri, 6 Mar 2009, david@lang.hm wrote:
>>
>>> I'm running into problems trying to do filtering. it looks as if the
>> log
>>> parsing is not properly filling in the properties.
>>>
>>> what I've run into so far
>>>
>>> when I use the property 'programname' the content that I see is what
>> I would
>>> expect in 'hostname'
>>>
>>> when I use the property 'hostname' the content that I see is what I
>> would
>>> expect in 'fromhost'
>>>
>>> I haven't checked all the other properties, but my guess is that
>> somehow
>>> rsyslog is off-by-one in filling them in.
>>
>> having said this, date, fromhost, and from-ip appear to be filled in
>> correctly.
>>
>> David Lang
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
Re: properties not getting filled in correctly [ In reply to ]
david at lang
Mar 10, 2009, 8:21 AM
Post #7 of 11 (3335 views)
Permalink
On Sat, 7 Mar 2009, Rainer Gerhards wrote:

> The messages indeed look ok. I'll feed them into my parser and will see what happens.

any idea what's happening here yet?

David Lang

> rainer
>
> ----- Urspr?ngliche Nachricht -----
> Von: "david@lang.hm" <david@lang.hm>
> An: "rsyslog-users" <rsyslog@lists.adiscon.com>
> Gesendet: 07.03.09 02:20
> Betreff: Re: [rsyslog] properties not getting filled in correctly
>
> On Fri, 6 Mar 2009, Rainer Gerhards wrote:
>
>> That's why I am after the log samples :) I just termed a new acronym
>> this afternoon:
>> YAMSF - yet another malformed syslog format ;)
>>
>> http://blog.gerhards.net/2009/02/calling-for-log-samples.html
>>
>> I try hard to get the fields right, but often this is impossible,
>> resulting in the issues you see.
>
> these logs come from several different servers, including different OSs,
> but all are misparsed by rsyslog.
>
> I am not seeing anything obviously wrong with them
>
> <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601
> <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029
> <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1
> <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667
> <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=<blah@HOTMAIL.COM>, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106@w31.diginsite.com> Queued mail for delivery)
> <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw
>
> David Lang
>
>> Rainer
>>
>>> -----Original Message-----
>>> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
>>> bounces@lists.adiscon.com] On Behalf Of david@lang.hm
>>> Sent: Friday, March 06, 2009 7:54 PM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] properties not getting filled in correctly
>>>
>>> On Fri, 6 Mar 2009, david@lang.hm wrote:
>>>
>>>> I'm running into problems trying to do filtering. it looks as if the
>>> log
>>>> parsing is not properly filling in the properties.
>>>>
>>>> what I've run into so far
>>>>
>>>> when I use the property 'programname' the content that I see is what
>>> I would
>>>> expect in 'hostname'
>>>>
>>>> when I use the property 'hostname' the content that I see is what I
>>> would
>>>> expect in 'fromhost'
>>>>
>>>> I haven't checked all the other properties, but my guess is that
>>> somehow
>>>> rsyslog is off-by-one in filling them in.
>>>
>>> having said this, date, fromhost, and from-ip appear to be filled in
>>> correctly.
>>>
>>> David Lang
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
Re: properties not getting filled in correctly [ In reply to ]
rgerhards at hq
Mar 10, 2009, 8:24 AM
Post #8 of 11 (3339 views)
Permalink
Not at the moment, I am currently looking into the scripting engine (for
stringlength-based evaluations)

I highly suggest

http://twitter.com/rgerhards

to keep track of what I am looking at. You do NOT need to be subscribed to
twitter to use this service.

Rainer

> -----Original Message-----
> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
> bounces@lists.adiscon.com] On Behalf Of david@lang.hm
> Sent: Tuesday, March 10, 2009 4:22 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] properties not getting filled in correctly
>
> On Sat, 7 Mar 2009, Rainer Gerhards wrote:
>
> > The messages indeed look ok. I'll feed them into my parser and will
> see what happens.
>
> any idea what's happening here yet?
>
> David Lang
>
> > rainer
> >
> > ----- Urspr?ngliche Nachricht -----
> > Von: "david@lang.hm" <david@lang.hm>
> > An: "rsyslog-users" <rsyslog@lists.adiscon.com>
> > Gesendet: 07.03.09 02:20
> > Betreff: Re: [rsyslog] properties not getting filled in correctly
> >
> > On Fri, 6 Mar 2009, Rainer Gerhards wrote:
> >
> >> That's why I am after the log samples :) I just termed a new acronym
> >> this afternoon:
> >> YAMSF - yet another malformed syslog format ;)
> >>
> >> http://blog.gerhards.net/2009/02/calling-for-log-samples.html
> >>
> >> I try hard to get the fields right, but often this is impossible,
> >> resulting in the issues you see.
> >
> > these logs come from several different servers, including different
> OSs,
> > but all are misparsed by rsyslog.
> >
> > I am not seeing anything obviously wrong with them
> >
> > <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request
> discarded from SERVER1/2741 to test_app:255.255.255.255/61601
> > <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host=
> /192.168.243.37 destination=179.50.100.130/60029
> > <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host=
> /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71
> duration=1
> > <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host=
> /192.168.22.8 destination=192.168.104.31/5667
> > <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326:
> to=<blah@HOTMAIL.COM>, delay=00:00:01, xdelay=00:00:01, mailer=esmtp,
> pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent (
> <200903070057.n270vrL174106@w31.diginsite.com> Queued mail for
> delivery)
> > <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host=
> /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw
> >
> > David Lang
> >
> >> Rainer
> >>
> >>> -----Original Message-----
> >>> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
> >>> bounces@lists.adiscon.com] On Behalf Of david@lang.hm
> >>> Sent: Friday, March 06, 2009 7:54 PM
> >>> To: rsyslog-users
> >>> Subject: Re: [rsyslog] properties not getting filled in correctly
> >>>
> >>> On Fri, 6 Mar 2009, david@lang.hm wrote:
> >>>
> >>>> I'm running into problems trying to do filtering. it looks as if
> the
> >>> log
> >>>> parsing is not properly filling in the properties.
> >>>>
> >>>> what I've run into so far
> >>>>
> >>>> when I use the property 'programname' the content that I see is
> what
> >>> I would
> >>>> expect in 'hostname'
> >>>>
> >>>> when I use the property 'hostname' the content that I see is what
> I
> >>> would
> >>>> expect in 'fromhost'
> >>>>
> >>>> I haven't checked all the other properties, but my guess is that
> >>> somehow
> >>>> rsyslog is off-by-one in filling them in.
> >>>
> >>> having said this, date, fromhost, and from-ip appear to be filled
> in
> >>> correctly.
> >>>
> >>> David Lang
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
Re: properties not getting filled in correctly [ In reply to ]
rgerhards at hq
Mar 11, 2009, 5:49 AM
Post #9 of 11 (3338 views)
Permalink
David,

the issue is in v4 only (and so far UDP only, too). It was introduced by the
optimizations, which pass some wrong parameters to the now-decoupled parser.
Need to find root cause, though.

Will keep you posted.

Rainer

> -----Original Message-----
> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
> bounces@lists.adiscon.com] On Behalf Of david@lang.hm
> Sent: Tuesday, March 10, 2009 4:22 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] properties not getting filled in correctly
>
> On Sat, 7 Mar 2009, Rainer Gerhards wrote:
>
> > The messages indeed look ok. I'll feed them into my parser and will
> see what happens.
>
> any idea what's happening here yet?
>
> David Lang
>
> > rainer
> >
> > ----- Urspr?ngliche Nachricht -----
> > Von: "david@lang.hm" <david@lang.hm>
> > An: "rsyslog-users" <rsyslog@lists.adiscon.com>
> > Gesendet: 07.03.09 02:20
> > Betreff: Re: [rsyslog] properties not getting filled in correctly
> >
> > On Fri, 6 Mar 2009, Rainer Gerhards wrote:
> >
> >> That's why I am after the log samples :) I just termed a new acronym
> >> this afternoon:
> >> YAMSF - yet another malformed syslog format ;)
> >>
> >> http://blog.gerhards.net/2009/02/calling-for-log-samples.html
> >>
> >> I try hard to get the fields right, but often this is impossible,
> >> resulting in the issues you see.
> >
> > these logs come from several different servers, including different
> OSs,
> > but all are misparsed by rsyslog.
> >
> > I am not seeing anything obviously wrong with them
> >
> > <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request
> discarded from SERVER1/2741 to test_app:255.255.255.255/61601
> > <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host=
> /192.168.243.37 destination=179.50.100.130/60029
> > <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host=
> /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71
> duration=1
> > <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host=
> /192.168.22.8 destination=192.168.104.31/5667
> > <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326:
> to=<blah@HOTMAIL.COM>, delay=00:00:01, xdelay=00:00:01, mailer=esmtp,
> pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent (
> <200903070057.n270vrL174106@w31.diginsite.com> Queued mail for
> delivery)
> > <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host=
> /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw
> >
> > David Lang
> >
> >> Rainer
> >>
> >>> -----Original Message-----
> >>> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
> >>> bounces@lists.adiscon.com] On Behalf Of david@lang.hm
> >>> Sent: Friday, March 06, 2009 7:54 PM
> >>> To: rsyslog-users
> >>> Subject: Re: [rsyslog] properties not getting filled in correctly
> >>>
> >>> On Fri, 6 Mar 2009, david@lang.hm wrote:
> >>>
> >>>> I'm running into problems trying to do filtering. it looks as if
> the
> >>> log
> >>>> parsing is not properly filling in the properties.
> >>>>
> >>>> what I've run into so far
> >>>>
> >>>> when I use the property 'programname' the content that I see is
> what
> >>> I would
> >>>> expect in 'hostname'
> >>>>
> >>>> when I use the property 'hostname' the content that I see is what
> I
> >>> would
> >>>> expect in 'fromhost'
> >>>>
> >>>> I haven't checked all the other properties, but my guess is that
> >>> somehow
> >>>> rsyslog is off-by-one in filling them in.
> >>>
> >>> having said this, date, fromhost, and from-ip appear to be filled
> in
> >>> correctly.
> >>>
> >>> David Lang
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
Re: properties not getting filled in correctly [ In reply to ]
david at lang
Mar 11, 2009, 5:51 AM
Post #10 of 11 (3337 views)
Permalink
On Wed, 11 Mar 2009, Rainer Gerhards wrote:

> David,
>
> the issue is in v4 only (and so far UDP only, too). It was introduced by the
> optimizations, which pass some wrong parameters to the now-decoupled parser.
> Need to find root cause, though.
>
> Will keep you posted.

thanks.

David Lang

> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
>> bounces@lists.adiscon.com] On Behalf Of david@lang.hm
>> Sent: Tuesday, March 10, 2009 4:22 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] properties not getting filled in correctly
>>
>> On Sat, 7 Mar 2009, Rainer Gerhards wrote:
>>
>>> The messages indeed look ok. I'll feed them into my parser and will
>> see what happens.
>>
>> any idea what's happening here yet?
>>
>> David Lang
>>
>>> rainer
>>>
>>> ----- Urspr?ngliche Nachricht -----
>>> Von: "david@lang.hm" <david@lang.hm>
>>> An: "rsyslog-users" <rsyslog@lists.adiscon.com>
>>> Gesendet: 07.03.09 02:20
>>> Betreff: Re: [rsyslog] properties not getting filled in correctly
>>>
>>> On Fri, 6 Mar 2009, Rainer Gerhards wrote:
>>>
>>>> That's why I am after the log samples :) I just termed a new acronym
>>>> this afternoon:
>>>> YAMSF - yet another malformed syslog format ;)
>>>>
>>>> http://blog.gerhards.net/2009/02/calling-for-log-samples.html
>>>>
>>>> I try hard to get the fields right, but often this is impossible,
>>>> resulting in the issues you see.
>>>
>>> these logs come from several different servers, including different
>> OSs,
>>> but all are misparsed by rsyslog.
>>>
>>> I am not seeing anything obviously wrong with them
>>>
>>> <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request
>> discarded from SERVER1/2741 to test_app:255.255.255.255/61601
>>> <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host=
>> /192.168.243.37 destination=179.50.100.130/60029
>>> <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host=
>> /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71
>> duration=1
>>> <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host=
>> /192.168.22.8 destination=192.168.104.31/5667
>>> <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326:
>> to=<blah@HOTMAIL.COM>, delay=00:00:01, xdelay=00:00:01, mailer=esmtp,
>> pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent (
>> <200903070057.n270vrL174106@w31.diginsite.com> Queued mail for
>> delivery)
>>> <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host=
>> /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw
>>>
>>> David Lang
>>>
>>>> Rainer
>>>>
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
>>>>> bounces@lists.adiscon.com] On Behalf Of david@lang.hm
>>>>> Sent: Friday, March 06, 2009 7:54 PM
>>>>> To: rsyslog-users
>>>>> Subject: Re: [rsyslog] properties not getting filled in correctly
>>>>>
>>>>> On Fri, 6 Mar 2009, david@lang.hm wrote:
>>>>>
>>>>>> I'm running into problems trying to do filtering. it looks as if
>> the
>>>>> log
>>>>>> parsing is not properly filling in the properties.
>>>>>>
>>>>>> what I've run into so far
>>>>>>
>>>>>> when I use the property 'programname' the content that I see is
>> what
>>>>> I would
>>>>>> expect in 'hostname'
>>>>>>
>>>>>> when I use the property 'hostname' the content that I see is what
>> I
>>>>> would
>>>>>> expect in 'fromhost'
>>>>>>
>>>>>> I haven't checked all the other properties, but my guess is that
>>>>> somehow
>>>>>> rsyslog is off-by-one in filling them in.
>>>>>
>>>>> having said this, date, fromhost, and from-ip appear to be filled
>> in
>>>>> correctly.
>>>>>
>>>>> David Lang
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
Re: properties not getting filled in correctly [ In reply to ]
rgerhards at hq
Mar 11, 2009, 6:32 AM
Post #11 of 11 (3340 views)
Permalink
David,

there is now a patch available:

http://git.adiscon.com/?p=rsyslog.git;a=commit;h=59192611db992e7357337beb8e68
ec6cee5b3fec

I will release a new devel today, and it will include the patch. I expect to
release another one next week, which will then have the Solaris work plus the
script engine with functions (feedback on that is still appreciated).

Rainer

> -----Original Message-----
> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
> bounces@lists.adiscon.com] On Behalf Of david@lang.hm
> Sent: Wednesday, March 11, 2009 1:51 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] properties not getting filled in correctly
>
> On Wed, 11 Mar 2009, Rainer Gerhards wrote:
>
> > David,
> >
> > the issue is in v4 only (and so far UDP only, too). It was introduced
> by the
> > optimizations, which pass some wrong parameters to the now-decoupled
> parser.
> > Need to find root cause, though.
> >
> > Will keep you posted.
>
> thanks.
>
> David Lang
>
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
> >> bounces@lists.adiscon.com] On Behalf Of david@lang.hm
> >> Sent: Tuesday, March 10, 2009 4:22 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] properties not getting filled in correctly
> >>
> >> On Sat, 7 Mar 2009, Rainer Gerhards wrote:
> >>
> >>> The messages indeed look ok. I'll feed them into my parser and will
> >> see what happens.
> >>
> >> any idea what's happening here yet?
> >>
> >> David Lang
> >>
> >>> rainer
> >>>
> >>> ----- Urspr?ngliche Nachricht -----
> >>> Von: "david@lang.hm" <david@lang.hm>
> >>> An: "rsyslog-users" <rsyslog@lists.adiscon.com>
> >>> Gesendet: 07.03.09 02:20
> >>> Betreff: Re: [rsyslog] properties not getting filled in correctly
> >>>
> >>> On Fri, 6 Mar 2009, Rainer Gerhards wrote:
> >>>
> >>>> That's why I am after the log samples :) I just termed a new
> acronym
> >>>> this afternoon:
> >>>> YAMSF - yet another malformed syslog format ;)
> >>>>
> >>>> http://blog.gerhards.net/2009/02/calling-for-log-samples.html
> >>>>
> >>>> I try hard to get the fields right, but often this is impossible,
> >>>> resulting in the issues you see.
> >>>
> >>> these logs come from several different servers, including different
> >> OSs,
> >>> but all are misparsed by rsyslog.
> >>>
> >>> I am not seeing anything obviously wrong with them
> >>>
> >>> <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request
> >> discarded from SERVER1/2741 to test_app:255.255.255.255/61601
> >>> <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host=
> >> /192.168.243.37 destination=179.50.100.130/60029
> >>> <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host=
> >> /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71
> >> duration=1
> >>> <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host=
> >> /192.168.22.8 destination=192.168.104.31/5667
> >>> <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326:
> >> to=<blah@HOTMAIL.COM>, delay=00:00:01, xdelay=00:00:01,
> mailer=esmtp,
> >> pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0,
> stat=Sent (
> >> <200903070057.n270vrL174106@w31.diginsite.com> Queued mail for
> >> delivery)
> >>> <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host=
> >> /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw
> >>>
> >>> David Lang
> >>>
> >>>> Rainer
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
> >>>>> bounces@lists.adiscon.com] On Behalf Of david@lang.hm
> >>>>> Sent: Friday, March 06, 2009 7:54 PM
> >>>>> To: rsyslog-users
> >>>>> Subject: Re: [rsyslog] properties not getting filled in correctly
> >>>>>
> >>>>> On Fri, 6 Mar 2009, david@lang.hm wrote:
> >>>>>
> >>>>>> I'm running into problems trying to do filtering. it looks as if
> >> the
> >>>>> log
> >>>>>> parsing is not properly filling in the properties.
> >>>>>>
> >>>>>> what I've run into so far
> >>>>>>
> >>>>>> when I use the property 'programname' the content that I see is
> >> what
> >>>>> I would
> >>>>>> expect in 'hostname'
> >>>>>>
> >>>>>> when I use the property 'hostname' the content that I see is
> what
> >> I
> >>>>> would
> >>>>>> expect in 'fromhost'
> >>>>>>
> >>>>>> I haven't checked all the other properties, but my guess is that
> >>>>> somehow
> >>>>>> rsyslog is off-by-one in filling them in.
> >>>>>
> >>>>> having said this, date, fromhost, and from-ip appear to be filled
> >> in
> >>>>> correctly.
> >>>>>
> >>>>> David Lang
> >>>>> _______________________________________________
> >>>>> rsyslog mailing list
> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>> http://www.rsyslog.com
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>> http://www.rsyslog.com
> >>>>
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal