Michael,
I have reviewed some of the code in question. It looks like a bug. It's
actually not a klog driver issue, the PRI parsing is done at an upper
klog layer and thus should work for linux as well. I would like to
generate a version with some additional instrumentation. Could you run
it and report the results back (maybe via private mail to keep the list
free of noise).
Rainer
> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Tuesday, July 15, 2008 3:04 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] slight kernel log format inconsistency between
> 3.16.2and 3.18.0
>
> On Tue, 2008-07-15 at 14:54 +0200, Michael Biebl wrote:
> > 2008/7/14 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > Hi all,
> > >
> > > Michael Biebl noticed a small inconsistency in kernel log format:
> > >
> > > 3.16.x (and before):
> > > Jul 11 17:33:28 pluto kernel: [14177.432349]
> ADDRCONF(NETDEV_CHANGE):
> > > wlan0: link becomes ready
> > > Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped.
> > > <upgrade auf 3.18.0>
> > >
> > > 3.18.0:
> > > Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source =
/proc/kmsg
> > > started.
> > > Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped.
> >
> > another example:
> > Jul 15 13:43:42 pluto kernel:<6>[47539.265140] eth0: link down
> > See the <6> after the kernel: tag. It's also new.
> > Is this some kind of log level, set by the kernel?
>
> This does not look correct. Under FreeBSD, it is typical to have a PRI
> inside kernel messages. But for linux kernels I tested with, I did not
> see it and (I think) so I did not support it. But if you see it on
> Debian, it seems to happen. A cure is to use the same logic that is
> used
> for BSD.
>
> > I have never seen, that other syslogger (like syslog-ng, sysklogd
set
> this).
> >
> > >
> > > As you can see, there is a space <SP> after kernel: in the pre-
> 3.18.0
> > > messages. This also brings us down to a general inconsistency:
> > >
> > > Unfortunately, RFC 3164 defines the <SP> after TAG not as a
> delimiter
> > > but as part of the CONTENT of the message. This leads to some
> > > inconsistency in practice. Depending on who generated the message,
> there
> > > may be a space after the TAG or not. If it is, that space must be
> > > submitted as part of the message itself.
> > >
> > > In versions up to 3.16.x, kernel log messages were generated by
old
> > > code, which did not know about tag and simply generated a non-
> structured
> > > message. With 3.17.x and above, the klog code is rewritten and
> correctly
> > > fills in all properties. This leads to the "missing" space,
because
> the
> > > space is neither part of the tag nor part of the message.
> > >
> > > I have now three options:
> > >
> > > 1. leave as is (without a space)
> > > In that case, some log parsers may run into troubles
> >
> > This was a rather vague concern of mine. I don't actually know, if
> > other software is affected by this change.
>
> I share this concern, but also have no additional information. I
wanted
> to point out the options. I have to admit I am still undecided. But
> that
> nobody violently objected so far makes this look like it is not so
> important...
>
> >
> > > 2. add a <SP> at the end of the TAG
> > > That will probably lead to even more confusion, as matches against
> TAG
> > > would need to include that space.
> > >
> > > 3. add a <SP> in front of each kernel message
> > > This comes close to the previous mode, but is "a bit" clumpsy and
> > > hackish. It will also make all database tables etc have messages
> > > starting with <SP>. Similar as 2, MSG matching rules are affected
> (but I
> > > consider this less severe, as matches inside the MSG are usually
> driven
> > > by searches and not absolute positions).
> >
> > Just curious: For other log messages, that are not generated by the
> > kernel, you have a space after the programname: tag, e.g.
> > Jul 15 13:43:50 pluto dhclient: DHCPACK from 192.168.2.1
> >
> > Is the space after dhclient: part of the message or part of the
> > :programname tag?
>
> In our private communication, I thought it is part of TAG. But that is
> wrong. It actually is part of MSG - at least for legacy/RFC3164
syslog.
>
> For the new IETF syslog RFC series (syslog-protocol, -tls et al) it is
> well-defined and part of neither. There, it is a delimiter (as one
> would
> expect). I you use rsyslog only with syslog-protocol messages (and the
> syslog-protocol templates), this is more or less no issue. But who
does
> today...? ;)
>
> The real problem is that legacy syslog has many different
> interpretations and we break one or the other in either way...
>
> >
> > Imho, if there is no clear definition in the RFC, how the message
> > should be written, it's best to stay backwards compatible.
>
> Is that a vote for option 3, stick a space in front of each kernel
> message? ;)
>
> >
> >
> > Cheers,
> > Michael
> >
> >
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
I have reviewed some of the code in question. It looks like a bug. It's
actually not a klog driver issue, the PRI parsing is done at an upper
klog layer and thus should work for linux as well. I would like to
generate a version with some additional instrumentation. Could you run
it and report the results back (maybe via private mail to keep the list
free of noise).
Rainer
> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Tuesday, July 15, 2008 3:04 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] slight kernel log format inconsistency between
> 3.16.2and 3.18.0
>
> On Tue, 2008-07-15 at 14:54 +0200, Michael Biebl wrote:
> > 2008/7/14 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > Hi all,
> > >
> > > Michael Biebl noticed a small inconsistency in kernel log format:
> > >
> > > 3.16.x (and before):
> > > Jul 11 17:33:28 pluto kernel: [14177.432349]
> ADDRCONF(NETDEV_CHANGE):
> > > wlan0: link becomes ready
> > > Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped.
> > > <upgrade auf 3.18.0>
> > >
> > > 3.18.0:
> > > Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source =
/proc/kmsg
> > > started.
> > > Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped.
> >
> > another example:
> > Jul 15 13:43:42 pluto kernel:<6>[47539.265140] eth0: link down
> > See the <6> after the kernel: tag. It's also new.
> > Is this some kind of log level, set by the kernel?
>
> This does not look correct. Under FreeBSD, it is typical to have a PRI
> inside kernel messages. But for linux kernels I tested with, I did not
> see it and (I think) so I did not support it. But if you see it on
> Debian, it seems to happen. A cure is to use the same logic that is
> used
> for BSD.
>
> > I have never seen, that other syslogger (like syslog-ng, sysklogd
set
> this).
> >
> > >
> > > As you can see, there is a space <SP> after kernel: in the pre-
> 3.18.0
> > > messages. This also brings us down to a general inconsistency:
> > >
> > > Unfortunately, RFC 3164 defines the <SP> after TAG not as a
> delimiter
> > > but as part of the CONTENT of the message. This leads to some
> > > inconsistency in practice. Depending on who generated the message,
> there
> > > may be a space after the TAG or not. If it is, that space must be
> > > submitted as part of the message itself.
> > >
> > > In versions up to 3.16.x, kernel log messages were generated by
old
> > > code, which did not know about tag and simply generated a non-
> structured
> > > message. With 3.17.x and above, the klog code is rewritten and
> correctly
> > > fills in all properties. This leads to the "missing" space,
because
> the
> > > space is neither part of the tag nor part of the message.
> > >
> > > I have now three options:
> > >
> > > 1. leave as is (without a space)
> > > In that case, some log parsers may run into troubles
> >
> > This was a rather vague concern of mine. I don't actually know, if
> > other software is affected by this change.
>
> I share this concern, but also have no additional information. I
wanted
> to point out the options. I have to admit I am still undecided. But
> that
> nobody violently objected so far makes this look like it is not so
> important...
>
> >
> > > 2. add a <SP> at the end of the TAG
> > > That will probably lead to even more confusion, as matches against
> TAG
> > > would need to include that space.
> > >
> > > 3. add a <SP> in front of each kernel message
> > > This comes close to the previous mode, but is "a bit" clumpsy and
> > > hackish. It will also make all database tables etc have messages
> > > starting with <SP>. Similar as 2, MSG matching rules are affected
> (but I
> > > consider this less severe, as matches inside the MSG are usually
> driven
> > > by searches and not absolute positions).
> >
> > Just curious: For other log messages, that are not generated by the
> > kernel, you have a space after the programname: tag, e.g.
> > Jul 15 13:43:50 pluto dhclient: DHCPACK from 192.168.2.1
> >
> > Is the space after dhclient: part of the message or part of the
> > :programname tag?
>
> In our private communication, I thought it is part of TAG. But that is
> wrong. It actually is part of MSG - at least for legacy/RFC3164
syslog.
>
> For the new IETF syslog RFC series (syslog-protocol, -tls et al) it is
> well-defined and part of neither. There, it is a delimiter (as one
> would
> expect). I you use rsyslog only with syslog-protocol messages (and the
> syslog-protocol templates), this is more or less no issue. But who
does
> today...? ;)
>
> The real problem is that legacy syslog has many different
> interpretations and we break one or the other in either way...
>
> >
> > Imho, if there is no clear definition in the RFC, how the message
> > should be written, it's best to stay backwards compatible.
>
> Is that a vote for option 3, stick a space in front of each kernel
> message? ;)
>
> >
> >
> > Cheers,
> > Michael
> >
> >
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog