Mailing List Archive

Multiple mmnormalize calls on same message
Hi Rsyslog Team,

We are planning to call mmnormalize multiple times for a message,

action(type="mmnormalize"
ruleBase="/etc/rsyslog.d/container_log_parse.rulebase" path="$!parsedmsg")
set $!msg = $!parsedmsg!log;
.......
.......
action(type="mmnormalize" rule=["rule=:%severity:word% %message:rest%"]
path="$!normalized_msg" variable="$!msg")

We found that, its working perfectly fine. However in docs
<https://www.rsyslog.com/doc/v8-stable/configuration/modules/mmnormalize.html>,
it says
"Note that mmnormalize should only be called once on each message.
Behaviour is undefined if multiple calls to mmnormalize happen for the same
message."

Couldn't understand completely what the statement means and the reasoning
behind it. Can we use mmnormalize like above or we should replace one of
mmnormalize functionality with other functions ? (we are using first
mmnormalize to split the message into multiple fields using space
delimiter. So, we can use re_extract function or property replacers or some
other function)

--
Regards,
Rajesh KSV
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Multiple mmnormalize calls on same message [ In reply to ]
I have used multiple mmnormalize calls on the same message without a problem.

That being said, if you can combine your rulesets, it's going to be more
effiecient than having to make multiple calls.

I believe the note in the docs is out of date.

David Lang

On Thu, 3 Jun 2021, rajeshksv via rsyslog wrote:

> Date: Thu, 3 Jun 2021 10:48:15 +0530
> From: rajeshksv via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog-users <rsyslog@lists.adiscon.com>
> Cc: rajeshksv <rajeshksv37@gmail.com>
> Subject: [rsyslog] Multiple mmnormalize calls on same message
>
> Hi Rsyslog Team,
>
> We are planning to call mmnormalize multiple times for a message,
>
> action(type="mmnormalize"
> ruleBase="/etc/rsyslog.d/container_log_parse.rulebase" path="$!parsedmsg")
> set $!msg = $!parsedmsg!log;
> .......
> .......
> action(type="mmnormalize" rule=["rule=:%severity:word% %message:rest%"]
> path="$!normalized_msg" variable="$!msg")
>
> We found that, its working perfectly fine. However in docs
> <https://www.rsyslog.com/doc/v8-stable/configuration/modules/mmnormalize.html>,
> it says
> "Note that mmnormalize should only be called once on each message.
> Behaviour is undefined if multiple calls to mmnormalize happen for the same
> message."
>
> Couldn't understand completely what the statement means and the reasoning
> behind it. Can we use mmnormalize like above or we should replace one of
> mmnormalize functionality with other functions ? (we are using first
> mmnormalize to split the message into multiple fields using space
> delimiter. So, we can use re_extract function or property replacers or some
> other function)
>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Multiple mmnormalize calls on same message [ In reply to ]
In this scenario, we can't combine them as the first call comes default
from the custom package (over rsyslog) users install. Other calls might be
added by users who install the custom package. Users might have multiple
mmnormalize calls as well

If docs are outdated and we can use mmnormalize multiple times, then
awesome it is!

On Thu, Jun 3, 2021 at 11:19 AM David Lang <david@lang.hm> wrote:

> I have used multiple mmnormalize calls on the same message without a
> problem.
>
> That being said, if you can combine your rulesets, it's going to be more
> effiecient than having to make multiple calls.
>
> I believe the note in the docs is out of date.
>
> David Lang
>
> On Thu, 3 Jun 2021, rajeshksv via rsyslog wrote:
>
> > Date: Thu, 3 Jun 2021 10:48:15 +0530
> > From: rajeshksv via rsyslog <rsyslog@lists.adiscon.com>
> > To: rsyslog-users <rsyslog@lists.adiscon.com>
> > Cc: rajeshksv <rajeshksv37@gmail.com>
> > Subject: [rsyslog] Multiple mmnormalize calls on same message
> >
> > Hi Rsyslog Team,
> >
> > We are planning to call mmnormalize multiple times for a message,
> >
> > action(type="mmnormalize"
> > ruleBase="/etc/rsyslog.d/container_log_parse.rulebase"
> path="$!parsedmsg")
> > set $!msg = $!parsedmsg!log;
> > .......
> > .......
> > action(type="mmnormalize" rule=["rule=:%severity:word% %message:rest%"]
> > path="$!normalized_msg" variable="$!msg")
> >
> > We found that, its working perfectly fine. However in docs
> > <
> https://www.rsyslog.com/doc/v8-stable/configuration/modules/mmnormalize.html
> >,
> > it says
> > "Note that mmnormalize should only be called once on each message.
> > Behaviour is undefined if multiple calls to mmnormalize happen for the
> same
> > message."
> >
> > Couldn't understand completely what the statement means and the reasoning
> > behind it. Can we use mmnormalize like above or we should replace one of
> > mmnormalize functionality with other functions ? (we are using first
> > mmnormalize to split the message into multiple fields using space
> > delimiter. So, we can use re_extract function or property replacers or
> some
> > other function)
> >
> >
>


--
Regards,
Rajesh KSV
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Multiple mmnormalize calls on same message [ In reply to ]
I think this was probably a doc bug from day one on (or heavily
outdated). It is supported, and we do it in several cases.

I'll update the doc later today.

Rainer

El vie, 4 jun 2021 a las 7:46, rajeshksv via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> In this scenario, we can't combine them as the first call comes default
> from the custom package (over rsyslog) users install. Other calls might be
> added by users who install the custom package. Users might have multiple
> mmnormalize calls as well
>
> If docs are outdated and we can use mmnormalize multiple times, then
> awesome it is!
>
> On Thu, Jun 3, 2021 at 11:19 AM David Lang <david@lang.hm> wrote:
>
> > I have used multiple mmnormalize calls on the same message without a
> > problem.
> >
> > That being said, if you can combine your rulesets, it's going to be more
> > effiecient than having to make multiple calls.
> >
> > I believe the note in the docs is out of date.
> >
> > David Lang
> >
> > On Thu, 3 Jun 2021, rajeshksv via rsyslog wrote:
> >
> > > Date: Thu, 3 Jun 2021 10:48:15 +0530
> > > From: rajeshksv via rsyslog <rsyslog@lists.adiscon.com>
> > > To: rsyslog-users <rsyslog@lists.adiscon.com>
> > > Cc: rajeshksv <rajeshksv37@gmail.com>
> > > Subject: [rsyslog] Multiple mmnormalize calls on same message
> > >
> > > Hi Rsyslog Team,
> > >
> > > We are planning to call mmnormalize multiple times for a message,
> > >
> > > action(type="mmnormalize"
> > > ruleBase="/etc/rsyslog.d/container_log_parse.rulebase"
> > path="$!parsedmsg")
> > > set $!msg = $!parsedmsg!log;
> > > .......
> > > .......
> > > action(type="mmnormalize" rule=["rule=:%severity:word% %message:rest%"]
> > > path="$!normalized_msg" variable="$!msg")
> > >
> > > We found that, its working perfectly fine. However in docs
> > > <
> > https://www.rsyslog.com/doc/v8-stable/configuration/modules/mmnormalize.html
> > >,
> > > it says
> > > "Note that mmnormalize should only be called once on each message.
> > > Behaviour is undefined if multiple calls to mmnormalize happen for the
> > same
> > > message."
> > >
> > > Couldn't understand completely what the statement means and the reasoning
> > > behind it. Can we use mmnormalize like above or we should replace one of
> > > mmnormalize functionality with other functions ? (we are using first
> > > mmnormalize to split the message into multiple fields using space
> > > delimiter. So, we can use re_extract function or property replacers or
> > some
> > > other function)
> > >
> > >
> >
>
>
> --
> Regards,
> Rajesh KSV
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Multiple mmnormalize calls on same message [ In reply to ]
Awesome. Thanks!

On Fri, Jun 4, 2021, 12:02 PM Rainer Gerhards <rgerhards@hq.adiscon.com>
wrote:

> I think this was probably a doc bug from day one on (or heavily
> outdated). It is supported, and we do it in several cases.
>
> I'll update the doc later today.
>
> Rainer
>
> El vie, 4 jun 2021 a las 7:46, rajeshksv via rsyslog
> (<rsyslog@lists.adiscon.com>) escribió:
> >
> > In this scenario, we can't combine them as the first call comes default
> > from the custom package (over rsyslog) users install. Other calls might
> be
> > added by users who install the custom package. Users might have multiple
> > mmnormalize calls as well
> >
> > If docs are outdated and we can use mmnormalize multiple times, then
> > awesome it is!
> >
> > On Thu, Jun 3, 2021 at 11:19 AM David Lang <david@lang.hm> wrote:
> >
> > > I have used multiple mmnormalize calls on the same message without a
> > > problem.
> > >
> > > That being said, if you can combine your rulesets, it's going to be
> more
> > > effiecient than having to make multiple calls.
> > >
> > > I believe the note in the docs is out of date.
> > >
> > > David Lang
> > >
> > > On Thu, 3 Jun 2021, rajeshksv via rsyslog wrote:
> > >
> > > > Date: Thu, 3 Jun 2021 10:48:15 +0530
> > > > From: rajeshksv via rsyslog <rsyslog@lists.adiscon.com>
> > > > To: rsyslog-users <rsyslog@lists.adiscon.com>
> > > > Cc: rajeshksv <rajeshksv37@gmail.com>
> > > > Subject: [rsyslog] Multiple mmnormalize calls on same message
> > > >
> > > > Hi Rsyslog Team,
> > > >
> > > > We are planning to call mmnormalize multiple times for a message,
> > > >
> > > > action(type="mmnormalize"
> > > > ruleBase="/etc/rsyslog.d/container_log_parse.rulebase"
> > > path="$!parsedmsg")
> > > > set $!msg = $!parsedmsg!log;
> > > > .......
> > > > .......
> > > > action(type="mmnormalize" rule=["rule=:%severity:word%
> %message:rest%"]
> > > > path="$!normalized_msg" variable="$!msg")
> > > >
> > > > We found that, its working perfectly fine. However in docs
> > > > <
> > >
> https://www.rsyslog.com/doc/v8-stable/configuration/modules/mmnormalize.html
> > > >,
> > > > it says
> > > > "Note that mmnormalize should only be called once on each message.
> > > > Behaviour is undefined if multiple calls to mmnormalize happen for
> the
> > > same
> > > > message."
> > > >
> > > > Couldn't understand completely what the statement means and the
> reasoning
> > > > behind it. Can we use mmnormalize like above or we should replace
> one of
> > > > mmnormalize functionality with other functions ? (we are using first
> > > > mmnormalize to split the message into multiple fields using space
> > > > delimiter. So, we can use re_extract function or property replacers
> or
> > > some
> > > > other function)
> > > >
> > > >
> > >
> >
> >
> > --
> > Regards,
> > Rajesh KSV
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Multiple mmnormalize calls on same message [ In reply to ]
just FYI: https://github.com/rsyslog/rsyslog-doc/pull/931

Rainer

El vie, 4 jun 2021 a las 10:00, rajeshksv (<rajeshksv37@gmail.com>) escribió:
>
> Awesome. Thanks!
>
> On Fri, Jun 4, 2021, 12:02 PM Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
>>
>> I think this was probably a doc bug from day one on (or heavily
>> outdated). It is supported, and we do it in several cases.
>>
>> I'll update the doc later today.
>>
>> Rainer
>>
>> El vie, 4 jun 2021 a las 7:46, rajeshksv via rsyslog
>> (<rsyslog@lists.adiscon.com>) escribió:
>> >
>> > In this scenario, we can't combine them as the first call comes default
>> > from the custom package (over rsyslog) users install. Other calls might be
>> > added by users who install the custom package. Users might have multiple
>> > mmnormalize calls as well
>> >
>> > If docs are outdated and we can use mmnormalize multiple times, then
>> > awesome it is!
>> >
>> > On Thu, Jun 3, 2021 at 11:19 AM David Lang <david@lang.hm> wrote:
>> >
>> > > I have used multiple mmnormalize calls on the same message without a
>> > > problem.
>> > >
>> > > That being said, if you can combine your rulesets, it's going to be more
>> > > effiecient than having to make multiple calls.
>> > >
>> > > I believe the note in the docs is out of date.
>> > >
>> > > David Lang
>> > >
>> > > On Thu, 3 Jun 2021, rajeshksv via rsyslog wrote:
>> > >
>> > > > Date: Thu, 3 Jun 2021 10:48:15 +0530
>> > > > From: rajeshksv via rsyslog <rsyslog@lists.adiscon.com>
>> > > > To: rsyslog-users <rsyslog@lists.adiscon.com>
>> > > > Cc: rajeshksv <rajeshksv37@gmail.com>
>> > > > Subject: [rsyslog] Multiple mmnormalize calls on same message
>> > > >
>> > > > Hi Rsyslog Team,
>> > > >
>> > > > We are planning to call mmnormalize multiple times for a message,
>> > > >
>> > > > action(type="mmnormalize"
>> > > > ruleBase="/etc/rsyslog.d/container_log_parse.rulebase"
>> > > path="$!parsedmsg")
>> > > > set $!msg = $!parsedmsg!log;
>> > > > .......
>> > > > .......
>> > > > action(type="mmnormalize" rule=["rule=:%severity:word% %message:rest%"]
>> > > > path="$!normalized_msg" variable="$!msg")
>> > > >
>> > > > We found that, its working perfectly fine. However in docs
>> > > > <
>> > > https://www.rsyslog.com/doc/v8-stable/configuration/modules/mmnormalize.html
>> > > >,
>> > > > it says
>> > > > "Note that mmnormalize should only be called once on each message.
>> > > > Behaviour is undefined if multiple calls to mmnormalize happen for the
>> > > same
>> > > > message."
>> > > >
>> > > > Couldn't understand completely what the statement means and the reasoning
>> > > > behind it. Can we use mmnormalize like above or we should replace one of
>> > > > mmnormalize functionality with other functions ? (we are using first
>> > > > mmnormalize to split the message into multiple fields using space
>> > > > delimiter. So, we can use re_extract function or property replacers or
>> > > some
>> > > > other function)
>> > > >
>> > > >
>> > >
>> >
>> >
>> > --
>> > Regards,
>> > Rajesh KSV
>> > _______________________________________________
>> > rsyslog mailing list
>> > https://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com/professional-services/
>> > What's up with rsyslog? Follow https://twitter.com/rgerhards
>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.