Mailing List Archive

Action params with exec.OnlyEveryNthTime
Good morning/afternoon!
I?m looking to implement some rate limiting due to an especially chatty set of vSphere hosts. First there is nothing I can do on the vmware side, so I?ve been exploring the rate limiting options in Rsyslog and so far what looks the best in my case is setting the action.execOnlyEveryNthTime. The problem is, going by the rsyslog.com docs I?m unsure of how to actually employ it.

If I have a ruleset like:

ruleset(name="r_vmhost"){
action(type="omfile" Dynafile="d_vmhost")
}

Should the parameter with within the action ()?s like:

ruleset(name="r_vmhost"){
action(type="omfile" Dynafile="d_vmhost" execOnlyEveryNthTime=?5?)
}

I did look at setting a global rate limiting within rsyslog.conf however I?d rather apply this to a specific ruleset or maybe a couple but not everything.

Thanks!



[Jamf]


Ben Hart
IT Systems Administrator II
100 Washington Ave S, Minneapolis, MN 55401
[Phone]
+00 1 989 424 0187
[Email]
ben.hart@jamf.com
[Web]
www.jamf.com<https://www.jamf.com>
[Facebook] [Twitter] [LinkedIn] [YouTube]
Re: Action params with exec.OnlyEveryNthTime [ In reply to ]
Update: Ok I found an example from a couple years ago using the ommail module. So now my updated syntax is:

if ($msg contains " Syslog-I ") then { action (type="omfile" file="/var/log/firewalld" action.execOnlyEveryNthTime="5" action.execOnlyEveryNthTimeout="15") stop }

Except the error running rsyslogd- N1 is on or before line 3: parameter 'action.execOnlyEveryNthTimeout' not known.
So I searched thru the changelog document for ?action.exec.OnlyEveryNth? and found notes under the DEVEL version 3.21.3 from 2008.

So Im left wondering what the status is of this functionality? Docs show it?s valie parameters under omfile (and possibly other modules), but with no mention in the changelog for a recent/current version I?m confused.

Thanks!


From: Ben Hart <ben.hart@jamf.com>
Date: Monday, September 26, 2022 at 12:10 PM
To: Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
Subject: Action params with exec.OnlyEveryNthTime
Good morning/afternoon!
I?m looking to implement some rate limiting due to an especially chatty set of vSphere hosts. First there is nothing I can do on the vmware side, so I?ve been exploring the rate limiting options in Rsyslog and so far what looks the best in my case is setting the action.execOnlyEveryNthTime. The problem is, going by the rsyslog.com docs I?m unsure of how to actually employ it.

If I have a ruleset like:

ruleset(name="r_vmhost"){
action(type="omfile" Dynafile="d_vmhost")
}

Should the parameter with within the action ()?s like:

ruleset(name="r_vmhost"){
action(type="omfile" Dynafile="d_vmhost" execOnlyEveryNthTime=?5?)
}

I did look at setting a global rate limiting within rsyslog.conf however I?d rather apply this to a specific ruleset or maybe a couple but not everything.

Thanks!



[Jamf]


Ben Hart
IT Systems Administrator II
100 Washington Ave S, Minneapolis, MN 55401
[Phone]
+00 1 989 424 0187
[Email]
ben.hart@jamf.com
[Web]
www.jamf.com<https://www.jamf.com>
[Facebook] [Twitter] [LinkedIn] [YouTube]
Re: Action params with exec.OnlyEveryNthTime [ In reply to ]
I think it's a valid parameter to ommail, but I would be surprised to see it
under omfile (and if it is, check what version you are running vs the version
you are seeing it documented)

rsyslog is not an event correlation engine, it has some minimal things, but they
really aren't very good.

What I do in a situation like yours is to filter the messages and send them to
an external event correlation engine (I tend to use Simple Event Correlator) and
then have it generate alert or summary messages, feeding them back in to rsyslog
(watch out that you don't generate a loop in this process)

David Lang

_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Action params with exec.OnlyEveryNthTime [ In reply to ]
David, So yeah you are right.. I didn?t dig into omfile and assumed from the Actions page those were available to most. So I have a .conf with 13 rulesets with matching inputs and I?ve found that you can ratelimit on the input side however this source is local (firewalld).
If I am loading modules imudp, imtcp and imuxsock and since I do have a .conf to send all of the logs from that source to a specific file:



From: David Lang <david@lang.hm>
Date: Monday, September 26, 2022 at 2:52 PM
To: Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
Cc: Ben Hart <ben.hart@jamf.com>
Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
I think it's a valid parameter to ommail, but I would be surprised to see it
under omfile (and if it is, check what version you are running vs the version
you are seeing it documented)

rsyslog is not an event correlation engine, it has some minimal things, but they
really aren't very good.

What I do in a situation like yours is to filter the messages and send them to
an external event correlation engine (I tend to use Simple Event Correlator) and
then have it generate alert or summary messages, feeding them back in to rsyslog
(watch out that you don't generate a loop in this process)

David Lang
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Action params with exec.OnlyEveryNthTime [ In reply to ]
(Apologies, somehow I sent this before it was complete.)

David, So yeah you are right.. I didn?t dig into omfile and assumed from the Actions page those were available to most. So I have a .conf with 13 rulesets with matching inputs and I?ve found that you can ratelimit on the input side however this source is local (firewalld).
If I am loading modules imudp, imtcp and imuxsock and since I do have a .conf to send all of the logs from that source to a specific file:

:msg,contains," Syslog-In " /var/log/firewalld
& stop

Am I correct in thinking that an input type for this would be imuxsock? And if so, could I setup a ruleset with matching inputs to then attempt to rate limit it?

Ben


From: David Lang <david@lang.hm>
Date: Monday, September 26, 2022 at 2:52 PM
To: Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
Cc: Ben Hart <ben.hart@jamf.com>
Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
I think it's a valid parameter to ommail, but I would be surprised to see it
under omfile (and if it is, check what version you are running vs the version
you are seeing it documented)

rsyslog is not an event correlation engine, it has some minimal things, but they
really aren't very good.

What I do in a situation like yours is to filter the messages and send them to
an external event correlation engine (I tend to use Simple Event Correlator) and
then have it generate alert or summary messages, feeding them back in to rsyslog
(watch out that you don't generate a loop in this process)

David Lang
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Action params with exec.OnlyEveryNthTime [ In reply to ]
also note that ratelimiting on the input throws away all messages, not just
messages of a particular type.

David Lang
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Action params with exec.OnlyEveryNthTime [ In reply to ]
log the message with the template RSYSLOG_DebugFormat and it will show you most
of the parsed properties from the message.

(I think you are correct, but it's best to be sure)

David Lang

On Mon, 26 Sep 2022, Ben Hart wrote:

> (Apologies, somehow I sent this before it was complete.)
>
> David, So yeah you are right.. I didn?t dig into omfile and assumed from the Actions page those were available to most. So I have a .conf with 13 rulesets with matching inputs and I?ve found that you can ratelimit on the input side however this source is local (firewalld).
> If I am loading modules imudp, imtcp and imuxsock and since I do have a .conf to send all of the logs from that source to a specific file:
>
> :msg,contains," Syslog-In " /var/log/firewalld
> & stop
>
> Am I correct in thinking that an input type for this would be imuxsock? And if so, could I setup a ruleset with matching inputs to then attempt to rate limit it?
>
> Ben
>
>
> From: David Lang <david@lang.hm>
> Date: Monday, September 26, 2022 at 2:52 PM
> To: Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Cc: Ben Hart <ben.hart@jamf.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
> I think it's a valid parameter to ommail, but I would be surprised to see it
> under omfile (and if it is, check what version you are running vs the version
> you are seeing it documented)
>
> rsyslog is not an event correlation engine, it has some minimal things, but they
> really aren't very good.
>
> What I do in a situation like yours is to filter the messages and send them to
> an external event correlation engine (I tend to use Simple Event Correlator) and
> then have it generate alert or summary messages, feeding them back in to rsyslog
> (watch out that you don't generate a loop in this process)
>
> David Lang
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Action params with exec.OnlyEveryNthTime [ In reply to ]
Understood, So I’ve never actually changed the default template, is there a way to engage that from within a .conf or does it need to be in rsyslog.conf? This server is currently handling logs from about 15 sources and SplunkUF is handing them off to Splunk Cloud so, unless I have to I’ don’t wanna mess with those.

From: David Lang <david@lang.hm>
Date: Monday, September 26, 2022 at 3:47 PM
To: Ben Hart <ben.hart@jamf.com>
Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
log the message with the template RSYSLOG_DebugFormat and it will show you most
of the parsed properties from the message.

(I think you are correct, but it's best to be sure)

David Lang

On Mon, 26 Sep 2022, Ben Hart wrote:

> (Apologies, somehow I sent this before it was complete.)
>
> David, So yeah you are right.. I didn?t dig into omfile and assumed from the Actions page those were available to most. So I have a .conf with 13 rulesets with matching inputs and I?ve found that you can ratelimit on the input side however this source is local (firewalld).
> If I am loading modules imudp, imtcp and imuxsock and since I do have a .conf to send all of the logs from that source to a specific file:
>
> :msg,contains," Syslog-In " /var/log/firewalld
> & stop
>
> Am I correct in thinking that an input type for this would be imuxsock? And if so, could I setup a ruleset with matching inputs to then attempt to rate limit it?
>
> Ben
>
>
> From: David Lang <david@lang.hm>
> Date: Monday, September 26, 2022 at 2:52 PM
> To: Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Cc: Ben Hart <ben.hart@jamf.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
> I think it's a valid parameter to ommail, but I would be surprised to see it
> under omfile (and if it is, check what version you are running vs the version
> you are seeing it documented)
>
> rsyslog is not an event correlation engine, it has some minimal things, but they
> really aren't very good.
>
> What I do in a situation like yours is to filter the messages and send them to
> an external event correlation engine (I tend to use Simple Event Correlator) and
> then have it generate alert or summary messages, feeding them back in to rsyslog
> (watch out that you don't generate a loop in this process)
>
> David Lang
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Action params with exec.OnlyEveryNthTime [ In reply to ]
Ok wait I think I got it.. I added the $DebugFile parameters to rsyslog.conf and bounced the daemon. Tons of info to parse through now ????

From: David Lang <david@lang.hm>
Date: Monday, September 26, 2022 at 3:47 PM
To: Ben Hart <ben.hart@jamf.com>
Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
log the message with the template RSYSLOG_DebugFormat and it will show you most
of the parsed properties from the message.

(I think you are correct, but it's best to be sure)

David Lang

On Mon, 26 Sep 2022, Ben Hart wrote:

> (Apologies, somehow I sent this before it was complete.)
>
> David, So yeah you are right.. I didn?t dig into omfile and assumed from the Actions page those were available to most. So I have a .conf with 13 rulesets with matching inputs and I?ve found that you can ratelimit on the input side however this source is local (firewalld).
> If I am loading modules imudp, imtcp and imuxsock and since I do have a .conf to send all of the logs from that source to a specific file:
>
> :msg,contains," Syslog-In " /var/log/firewalld
> & stop
>
> Am I correct in thinking that an input type for this would be imuxsock? And if so, could I setup a ruleset with matching inputs to then attempt to rate limit it?
>
> Ben
>
>
> From: David Lang <david@lang.hm>
> Date: Monday, September 26, 2022 at 2:52 PM
> To: Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Cc: Ben Hart <ben.hart@jamf.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
> I think it's a valid parameter to ommail, but I would be surprised to see it
> under omfile (and if it is, check what version you are running vs the version
> you are seeing it documented)
>
> rsyslog is not an event correlation engine, it has some minimal things, but they
> really aren't very good.
>
> What I do in a situation like yours is to filter the messages and send them to
> an external event correlation engine (I tend to use Simple Event Correlator) and
> then have it generate alert or summary messages, feeding them back in to rsyslog
> (watch out that you don't generate a loop in this process)
>
> David Lang
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Action params with exec.OnlyEveryNthTime [ In reply to ]
Ok here’s a sanitized blurb from the debug file grepped to the events I’m trying to limit.

2165.510789426:main Q:Reg/w0 : ruleset.c: Filter: check for property 'msg' (value '[2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.30 LEN=361 TOS=0x00 PREC=0x00 TTL=64 ID=29068 DF PROTO=UDP SPT=52225 DPT=9999 LEN=341 ') contains 'filter_': FALSE
2165.510961295:main Q:Reg/w0 : rainerscript.c: rainerscript: (string) var 1: '[2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.30 LEN=361 TOS=0x00 PREC=0x00 TTL=64 ID=29068 DF PROTO=UDP SPT=52225 DPT=9999 LEN=341 '
2165.512976011:main Q:Reg/w0 : omfile.c: omfile: write to stream, pData->pStrm 0x7f9e88001890, lenBuf 254, strt data Sep 26 14:56:05 rsyslog-server kernel: [2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08
2165.513114940:main Q:Reg/w0 : omfile.c: omfile: write to stream, pData->pStrm 0x7f9e88004040, lenBuf 254, strt data Sep 26 14:56:05 rsyslog-server kernel: [2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08
2165.545162782:vcsa9999:Reg/w0: rainerscript.c: 2165.545175184:main Q:Reg/w0 : ruleset.c: processBATCH: next msg 0: [2070020.676314] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.3


So it’s def hitting the main queue, and it’s matching correctly. Could not find any mention of imuxsock, or the word socket along with firewalld. Oh wait a minute, FWD natively logs to syslog so maybe I can still use an input for imuxsock but then possibly filter my programname or msg to impose the limits?


From: Ben Hart <ben.hart@jamf.com>
Date: Monday, September 26, 2022 at 3:51 PM
To: David Lang <david@lang.hm>
Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
Understood, So I’ve never actually changed the default template, is there a way to engage that from within a .conf or does it need to be in rsyslog.conf? This server is currently handling logs from about 15 sources and SplunkUF is handing them off to Splunk Cloud so, unless I have to I’ don’t wanna mess with those.

From: David Lang <david@lang.hm>
Date: Monday, September 26, 2022 at 3:47 PM
To: Ben Hart <ben.hart@jamf.com>
Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
log the message with the template RSYSLOG_DebugFormat and it will show you most
of the parsed properties from the message.

(I think you are correct, but it's best to be sure)

David Lang

On Mon, 26 Sep 2022, Ben Hart wrote:

> (Apologies, somehow I sent this before it was complete.)
>
> David, So yeah you are right.. I didn?t dig into omfile and assumed from the Actions page those were available to most. So I have a .conf with 13 rulesets with matching inputs and I?ve found that you can ratelimit on the input side however this source is local (firewalld).
> If I am loading modules imudp, imtcp and imuxsock and since I do have a .conf to send all of the logs from that source to a specific file:
>
> :msg,contains," Syslog-In " /var/log/firewalld
> & stop
>
> Am I correct in thinking that an input type for this would be imuxsock? And if so, could I setup a ruleset with matching inputs to then attempt to rate limit it?
>
> Ben
>
>
> From: David Lang <david@lang.hm>
> Date: Monday, September 26, 2022 at 2:52 PM
> To: Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Cc: Ben Hart <ben.hart@jamf.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
> I think it's a valid parameter to ommail, but I would be surprised to see it
> under omfile (and if it is, check what version you are running vs the version
> you are seeing it documented)
>
> rsyslog is not an event correlation engine, it has some minimal things, but they
> really aren't very good.
>
> What I do in a situation like yours is to filter the messages and send them to
> an external event correlation engine (I tend to use Simple Event Correlator) and
> then have it generate alert or summary messages, feeding them back in to rsyslog
> (watch out that you don't generate a loop in this process)
>
> David Lang
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Action params with exec.OnlyEveryNthTime [ In reply to ]
currently you have

>> :msg,contains," Syslog-In " /var/log/firewalld
>> & stop

change this to
:msg,contains," Syslog-In " {
/var/log/firewalld
/var/log/firewalld-debug;RSYSLOG_DebugFormat
stop
}

(a little easier to read than using &)

then look at the firewalld-debug file

David Lang

On Mon, 26 Sep 2022, Ben Hart wrote:

> Date: Mon, 26 Sep 2022 20:09:40 +0000
> From: Ben Hart <ben.hart@jamf.com>
> To: David Lang <david@lang.hm>
> Cc: David Lang <david@lang.hm>,
> Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>
> Ok here’s a sanitized blurb from the debug file grepped to the events I’m trying to limit.
>
> 2165.510789426:main Q:Reg/w0 : ruleset.c: Filter: check for property 'msg' (value '[2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.30 LEN=361 TOS=0x00 PREC=0x00 TTL=64 ID=29068 DF PROTO=UDP SPT=52225 DPT=9999 LEN=341 ') contains 'filter_': FALSE
> 2165.510961295:main Q:Reg/w0 : rainerscript.c: rainerscript: (string) var 1: '[2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.30 LEN=361 TOS=0x00 PREC=0x00 TTL=64 ID=29068 DF PROTO=UDP SPT=52225 DPT=9999 LEN=341 '
> 2165.512976011:main Q:Reg/w0 : omfile.c: omfile: write to stream, pData->pStrm 0x7f9e88001890, lenBuf 254, strt data Sep 26 14:56:05 rsyslog-server kernel: [2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08
> 2165.513114940:main Q:Reg/w0 : omfile.c: omfile: write to stream, pData->pStrm 0x7f9e88004040, lenBuf 254, strt data Sep 26 14:56:05 rsyslog-server kernel: [2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08
> 2165.545162782:vcsa9999:Reg/w0: rainerscript.c: 2165.545175184:main Q:Reg/w0 : ruleset.c: processBATCH: next msg 0: [2070020.676314] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.3
>
>
> So it’s def hitting the main queue, and it’s matching correctly. Could not find any mention of imuxsock, or the word socket along with firewalld. Oh wait a minute, FWD natively logs to syslog so maybe I can still use an input for imuxsock but then possibly filter my programname or msg to impose the limits?
>
>
> From: Ben Hart <ben.hart@jamf.com>
> Date: Monday, September 26, 2022 at 3:51 PM
> To: David Lang <david@lang.hm>
> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
> Understood, So I’ve never actually changed the default template, is there a way to engage that from within a .conf or does it need to be in rsyslog.conf? This server is currently handling logs from about 15 sources and SplunkUF is handing them off to Splunk Cloud so, unless I have to I’ don’t wanna mess with those.
>
> From: David Lang <david@lang.hm>
> Date: Monday, September 26, 2022 at 3:47 PM
> To: Ben Hart <ben.hart@jamf.com>
> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
> log the message with the template RSYSLOG_DebugFormat and it will show you most
> of the parsed properties from the message.
>
> (I think you are correct, but it's best to be sure)
>
> David Lang
>
> On Mon, 26 Sep 2022, Ben Hart wrote:
>
>> (Apologies, somehow I sent this before it was complete.)
>>
>> David, So yeah you are right.. I didn?t dig into omfile and assumed from the Actions page those were available to most. So I have a .conf with 13 rulesets with matching inputs and I?ve found that you can ratelimit on the input side however this source is local (firewalld).
>> If I am loading modules imudp, imtcp and imuxsock and since I do have a .conf to send all of the logs from that source to a specific file:
>>
>> :msg,contains," Syslog-In " /var/log/firewalld
>> & stop
>>
>> Am I correct in thinking that an input type for this would be imuxsock? And if so, could I setup a ruleset with matching inputs to then attempt to rate limit it?
>>
>> Ben
>>
>>
>> From: David Lang <david@lang.hm>
>> Date: Monday, September 26, 2022 at 2:52 PM
>> To: Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>> Cc: Ben Hart <ben.hart@jamf.com>
>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>> I think it's a valid parameter to ommail, but I would be surprised to see it
>> under omfile (and if it is, check what version you are running vs the version
>> you are seeing it documented)
>>
>> rsyslog is not an event correlation engine, it has some minimal things, but they
>> really aren't very good.
>>
>> What I do in a situation like yours is to filter the messages and send them to
>> an external event correlation engine (I tend to use Simple Event Correlator) and
>> then have it generate alert or summary messages, feeding them back in to rsyslog
>> (watch out that you don't generate a loop in this process)
>>
>> David Lang
>>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Action params with exec.OnlyEveryNthTime [ In reply to ]
OK I imagine I did something wrong.. changed the firewalld.conf to exactly what you suggested, bounced rsyslog and gave it a minute. The new fierwalld-debug file remained empty. Do I need to paste that DebugFormat template in somewhere? Also tried pasting the template definition into the bottom of this same file. It did not correct the problem.

I do see numerous omfile suspension warnings in systemctl status rsyslog, I presume it’s referring to the -debug file. Rsyslogd -N1 gives no errors. Also the rules no longer matching on the “” value. Goes back to defaults which is only logging denied packets to the file and all else into syslog.

After struggling a bit, then trying to simplify it I just appended the template onto the tail of the original line. I let it run for a few minutes then reverted back to original. But I now have a sample of log with the debugformat matching my “” below. And it’s using the imklog module, but reading about that module I get the impression that it’s nor normal nor probably a good idea to be handling it. So with that in mind, maybe it’s a better idea to try to keep this data out of that module entirely.

But, TBH I have no idea where to go from here. I still need to limit this traffic but since it’s hitting the kernel, and it’s local, and I cannot restrict it from the source… all suggestions welcome.
Thanks

From: David Lang <david@lang.hm>
Date: Monday, September 26, 2022 at 4:13 PM
To: Ben Hart <ben.hart@jamf.com>
Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
currently you have

>> :msg,contains," Syslog-In " /var/log/firewalld
>> & stop

change this to
:msg,contains," Syslog-In " {
/var/log/firewalld
/var/log/firewalld-debug;RSYSLOG_DebugFormat
stop
}

(a little easier to read than using &)

then look at the firewalld-debug file

David Lang

On Mon, 26 Sep 2022, Ben Hart wrote:

> Date: Mon, 26 Sep 2022 20:09:40 +0000
> From: Ben Hart <ben.hart@jamf.com>
> To: David Lang <david@lang.hm>
> Cc: David Lang <david@lang.hm>,
> Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>
> Ok here’s a sanitized blurb from the debug file grepped to the events I’m trying to limit.
>
> 2165.510789426:main Q:Reg/w0 : ruleset.c: Filter: check for property 'msg' (value '[2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.30 LEN=361 TOS=0x00 PREC=0x00 TTL=64 ID=29068 DF PROTO=UDP SPT=52225 DPT=9999 LEN=341 ') contains 'filter_': FALSE
> 2165.510961295:main Q:Reg/w0 : rainerscript.c: rainerscript: (string) var 1: '[2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.30 LEN=361 TOS=0x00 PREC=0x00 TTL=64 ID=29068 DF PROTO=UDP SPT=52225 DPT=9999 LEN=341 '
> 2165.512976011:main Q:Reg/w0 : omfile.c: omfile: write to stream, pData->pStrm 0x7f9e88001890, lenBuf 254, strt data Sep 26 14:56:05 rsyslog-server kernel: [2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08
> 2165.513114940:main Q:Reg/w0 : omfile.c: omfile: write to stream, pData->pStrm 0x7f9e88004040, lenBuf 254, strt data Sep 26 14:56:05 rsyslog-server kernel: [2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08
> 2165.545162782:vcsa9999:Reg/w0: rainerscript.c: 2165.545175184:main Q:Reg/w0 : ruleset.c: processBATCH: next msg 0: [2070020.676314] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.3
>
>
> So it’s def hitting the main queue, and it’s matching correctly. Could not find any mention of imuxsock, or the word socket along with firewalld. Oh wait a minute, FWD natively logs to syslog so maybe I can still use an input for imuxsock but then possibly filter my programname or msg to impose the limits?
>
>
> From: Ben Hart <ben.hart@jamf.com>
> Date: Monday, September 26, 2022 at 3:51 PM
> To: David Lang <david@lang.hm>
> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
> Understood, So I’ve never actually changed the default template, is there a way to engage that from within a .conf or does it need to be in rsyslog.conf? This server is currently handling logs from about 15 sources and SplunkUF is handing them off to Splunk Cloud so, unless I have to I’ don’t wanna mess with those.
>
> From: David Lang <david@lang.hm>
> Date: Monday, September 26, 2022 at 3:47 PM
> To: Ben Hart <ben.hart@jamf.com>
> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
> log the message with the template RSYSLOG_DebugFormat and it will show you most
> of the parsed properties from the message.
>
> (I think you are correct, but it's best to be sure)
>
> David Lang
>
> On Mon, 26 Sep 2022, Ben Hart wrote:
>
>> (Apologies, somehow I sent this before it was complete.)
>>
>> David, So yeah you are right.. I didn?t dig into omfile and assumed from the Actions page those were available to most. So I have a .conf with 13 rulesets with matching inputs and I?ve found that you can ratelimit on the input side however this source is local (firewalld).
>> If I am loading modules imudp, imtcp and imuxsock and since I do have a .conf to send all of the logs from that source to a specific file:
>>
>> :msg,contains," Syslog-In " /var/log/firewalld
>> & stop
>>
>> Am I correct in thinking that an input type for this would be imuxsock? And if so, could I setup a ruleset with matching inputs to then attempt to rate limit it?
>>
>> Ben
>>
>>
>> From: David Lang <david@lang.hm>
>> Date: Monday, September 26, 2022 at 2:52 PM
>> To: Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>> Cc: Ben Hart <ben.hart@jamf.com>
>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>> I think it's a valid parameter to ommail, but I would be surprised to see it
>> under omfile (and if it is, check what version you are running vs the version
>> you are seeing it documented)
>>
>> rsyslog is not an event correlation engine, it has some minimal things, but they
>> really aren't very good.
>>
>> What I do in a situation like yours is to filter the messages and send them to
>> an external event correlation engine (I tend to use Simple Event Correlator) and
>> then have it generate alert or summary messages, feeding them back in to rsyslog
>> (watch out that you don't generate a loop in this process)
>>
>> David Lang
>>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Action params with exec.OnlyEveryNthTime [ In reply to ]
ok, I don't see the sample of the debug format message, please send one or two
of these to the body of the message rather than trying to use an attachment (I
assume that's what you tried)

your original approach to filter these out to a separate file and then throw
them away is the place to start, you just need to do that before any config
lines that do other things with the logs.

remember that you can filter on any variable, and you can use the simple
if...then format (where you can combine multiple criteria) rather than the old
format you used below.

David Lang


On Mon, 26 Sep 2022, Ben Hart wrote:

> Date: Mon, 26 Sep 2022 21:19:43 +0000
> From: Ben Hart <ben.hart@jamf.com>
> To: David Lang <david@lang.hm>
> Cc: David Lang <david@lang.hm>,
> Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>
> OK I imagine I did something wrong.. changed the firewalld.conf to exactly what you suggested, bounced rsyslog and gave it a minute. The new fierwalld-debug file remained empty. Do I need to paste that DebugFormat template in somewhere? Also tried pasting the template definition into the bottom of this same file. It did not correct the problem.
>
> I do see numerous omfile suspension warnings in systemctl status rsyslog, I presume it’s referring to the -debug file. Rsyslogd -N1 gives no errors. Also the rules no longer matching on the “” value. Goes back to defaults which is only logging denied packets to the file and all else into syslog.
>
> After struggling a bit, then trying to simplify it I just appended the template onto the tail of the original line. I let it run for a few minutes then reverted back to original. But I now have a sample of log with the debugformat matching my “” below. And it’s using the imklog module, but reading about that module I get the impression that it’s nor normal nor probably a good idea to be handling it. So with that in mind, maybe it’s a better idea to try to keep this data out of that module entirely.
>
> But, TBH I have no idea where to go from here. I still need to limit this traffic but since it’s hitting the kernel, and it’s local, and I cannot restrict it from the source… all suggestions welcome.
> Thanks
>
> From: David Lang <david@lang.hm>
> Date: Monday, September 26, 2022 at 4:13 PM
> To: Ben Hart <ben.hart@jamf.com>
> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
> currently you have
>
>>> :msg,contains," Syslog-In " /var/log/firewalld
>>> & stop
>
> change this to
> :msg,contains," Syslog-In " {
> /var/log/firewalld
> /var/log/firewalld-debug;RSYSLOG_DebugFormat
> stop
> }
>
> (a little easier to read than using &)
>
> then look at the firewalld-debug file
>
> David Lang
>
> On Mon, 26 Sep 2022, Ben Hart wrote:
>
>> Date: Mon, 26 Sep 2022 20:09:40 +0000
>> From: Ben Hart <ben.hart@jamf.com>
>> To: David Lang <david@lang.hm>
>> Cc: David Lang <david@lang.hm>,
>> Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>>
>> Ok here’s a sanitized blurb from the debug file grepped to the events I’m trying to limit.
>>
>> 2165.510789426:main Q:Reg/w0 : ruleset.c: Filter: check for property 'msg' (value '[2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.30 LEN=361 TOS=0x00 PREC=0x00 TTL=64 ID=29068 DF PROTO=UDP SPT=52225 DPT=9999 LEN=341 ') contains 'filter_': FALSE
>> 2165.510961295:main Q:Reg/w0 : rainerscript.c: rainerscript: (string) var 1: '[2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.30 LEN=361 TOS=0x00 PREC=0x00 TTL=64 ID=29068 DF PROTO=UDP SPT=52225 DPT=9999 LEN=341 '
>> 2165.512976011:main Q:Reg/w0 : omfile.c: omfile: write to stream, pData->pStrm 0x7f9e88001890, lenBuf 254, strt data Sep 26 14:56:05 rsyslog-server kernel: [2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08
>> 2165.513114940:main Q:Reg/w0 : omfile.c: omfile: write to stream, pData->pStrm 0x7f9e88004040, lenBuf 254, strt data Sep 26 14:56:05 rsyslog-server kernel: [2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08
>> 2165.545162782:vcsa9999:Reg/w0: rainerscript.c: 2165.545175184:main Q:Reg/w0 : ruleset.c: processBATCH: next msg 0: [2070020.676314] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.3
>>
>>
>> So it’s def hitting the main queue, and it’s matching correctly. Could not find any mention of imuxsock, or the word socket along with firewalld. Oh wait a minute, FWD natively logs to syslog so maybe I can still use an input for imuxsock but then possibly filter my programname or msg to impose the limits?
>>
>>
>> From: Ben Hart <ben.hart@jamf.com>
>> Date: Monday, September 26, 2022 at 3:51 PM
>> To: David Lang <david@lang.hm>
>> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>> Understood, So I’ve never actually changed the default template, is there a way to engage that from within a .conf or does it need to be in rsyslog.conf? This server is currently handling logs from about 15 sources and SplunkUF is handing them off to Splunk Cloud so, unless I have to I’ don’t wanna mess with those.
>>
>> From: David Lang <david@lang.hm>
>> Date: Monday, September 26, 2022 at 3:47 PM
>> To: Ben Hart <ben.hart@jamf.com>
>> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>> log the message with the template RSYSLOG_DebugFormat and it will show you most
>> of the parsed properties from the message.
>>
>> (I think you are correct, but it's best to be sure)
>>
>> David Lang
>>
>> On Mon, 26 Sep 2022, Ben Hart wrote:
>>
>>> (Apologies, somehow I sent this before it was complete.)
>>>
>>> David, So yeah you are right.. I didn?t dig into omfile and assumed from the Actions page those were available to most. So I have a .conf with 13 rulesets with matching inputs and I?ve found that you can ratelimit on the input side however this source is local (firewalld).
>>> If I am loading modules imudp, imtcp and imuxsock and since I do have a .conf to send all of the logs from that source to a specific file:
>>>
>>> :msg,contains," Syslog-In " /var/log/firewalld
>>> & stop
>>>
>>> Am I correct in thinking that an input type for this would be imuxsock? And if so, could I setup a ruleset with matching inputs to then attempt to rate limit it?
>>>
>>> Ben
>>>
>>>
>>> From: David Lang <david@lang.hm>
>>> Date: Monday, September 26, 2022 at 2:52 PM
>>> To: Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>>> Cc: Ben Hart <ben.hart@jamf.com>
>>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>>> I think it's a valid parameter to ommail, but I would be surprised to see it
>>> under omfile (and if it is, check what version you are running vs the version
>>> you are seeing it documented)
>>>
>>> rsyslog is not an event correlation engine, it has some minimal things, but they
>>> really aren't very good.
>>>
>>> What I do in a situation like yours is to filter the messages and send them to
>>> an external event correlation engine (I tend to use Simple Event Correlator) and
>>> then have it generate alert or summary messages, feeding them back in to rsyslog
>>> (watch out that you don't generate a loop in this process)
>>>
>>> David Lang
>>>
>>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Action params with exec.OnlyEveryNthTime [ In reply to ]
Yeah sorry, here’s a block of matched return:

FROMHOST: 'rsyslog-server', fromhost-ip: '127.0.0.1', HOSTNAME: 'rsyslog-server', PRI: 6,
syslogtag 'kernel:', programname: 'kernel', APP-NAME: 'kernel', PROCID: '-', MSGID: '-',
TIMESTAMP: 'Sep 26 16:10:01', STRUCTURED-DATA: '-',
msg: '[2074456.628051] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.160.115 DST=1.2.160.30 LEN=205 TOS=0x00 PREC=0x00 TTL=64 ID=65500 DF PROTO=UDP SPT=36775 DPT=20525 LEN=185 '
escaped msg: '[2074456.628051] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.160.115 DST=1.2.160.30 LEN=205 TOS=0x00 PREC=0x00 TTL=64 ID=65500 DF PROTO=UDP SPT=36775 DPT=20525 LEN=185 '
inputname: imklog rawmsg: '[2074456.628051] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.160.115 DST=1.2.160.30 LEN=205 TOS=0x00 PREC=0x00 TTL=64 ID=65500 DF PROTO=UDP SPT=36775 DPT=20525 LEN=185 '
$!:
$.:
$/:

I do have another conf file that I found some excellent examples of that utilizes a catch_all ruleset/queue with if/thens below it. I’d rather use the modern syntaxes when I can so Im thinking of trying to adapt something like that to this. Except, in this case the format I’d mimic is directing matches to 3 different log files. I don’t need to do that, they all need to stay in the firewalld file except with the VCSA match being rate limited.

Here's the one I’m talking about:

ruleset(name="f_all" queue.type="LinkedList" queue.workerthreads="4" queue.size="100000") {
# WLC
if ($hostname contains 'wlc') then{
call r_wlc
stop
}
# ESXI
if ($hostname contains 'vmhost') then{
call r_vmhost
stop
}
# Catch All
call r_catch_all
stop
}
# add vmhosts #
template(name="d_catch_all" type="string" string="/var/log/remote-syslog/undefined.log")
template(name="d_wlc" type="string" string="/var/log/remote-syslog/wlc.log")
template(name="d_vmhost" type="string" string="/var/log/remote-syslog/esxi.log")

ruleset(name="r_wlc"){
action(type="omfile" Dynafile="d_wlc")
}
ruleset(name="r_vmhost"){
action(type="omfile" Dynafile="d_vmhost")
}
ruleset(name="r_catch_all"){
action(type="omfile" Dynafile="d_catch_all")
}

input(type="imudp" port="514" ruleset="f_all")


From: David Lang <david@lang.hm>
Date: Monday, September 26, 2022 at 5:31 PM
To: Ben Hart <ben.hart@jamf.com>
Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
ok, I don't see the sample of the debug format message, please send one or two
of these to the body of the message rather than trying to use an attachment (I
assume that's what you tried)

your original approach to filter these out to a separate file and then throw
them away is the place to start, you just need to do that before any config
lines that do other things with the logs.

remember that you can filter on any variable, and you can use the simple
if...then format (where you can combine multiple criteria) rather than the old
format you used below.

David Lang


On Mon, 26 Sep 2022, Ben Hart wrote:

> Date: Mon, 26 Sep 2022 21:19:43 +0000
> From: Ben Hart <ben.hart@jamf.com>
> To: David Lang <david@lang.hm>
> Cc: David Lang <david@lang.hm>,
> Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>
> OK I imagine I did something wrong.. changed the firewalld.conf to exactly what you suggested, bounced rsyslog and gave it a minute. The new fierwalld-debug file remained empty. Do I need to paste that DebugFormat template in somewhere? Also tried pasting the template definition into the bottom of this same file. It did not correct the problem.
>
> I do see numerous omfile suspension warnings in systemctl status rsyslog, I presume it’s referring to the -debug file. Rsyslogd -N1 gives no errors. Also the rules no longer matching on the “” value. Goes back to defaults which is only logging denied packets to the file and all else into syslog.
>
> After struggling a bit, then trying to simplify it I just appended the template onto the tail of the original line. I let it run for a few minutes then reverted back to original. But I now have a sample of log with the debugformat matching my “” below. And it’s using the imklog module, but reading about that module I get the impression that it’s nor normal nor probably a good idea to be handling it. So with that in mind, maybe it’s a better idea to try to keep this data out of that module entirely.
>
> But, TBH I have no idea where to go from here. I still need to limit this traffic but since it’s hitting the kernel, and it’s local, and I cannot restrict it from the source… all suggestions welcome.
> Thanks
>
> From: David Lang <david@lang.hm>
> Date: Monday, September 26, 2022 at 4:13 PM
> To: Ben Hart <ben.hart@jamf.com>
> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
> currently you have
>
>>> :msg,contains," Syslog-In " /var/log/firewalld
>>> & stop
>
> change this to
> :msg,contains," Syslog-In " {
> /var/log/firewalld
> /var/log/firewalld-debug;RSYSLOG_DebugFormat
> stop
> }
>
> (a little easier to read than using &)
>
> then look at the firewalld-debug file
>
> David Lang
>
> On Mon, 26 Sep 2022, Ben Hart wrote:
>
>> Date: Mon, 26 Sep 2022 20:09:40 +0000
>> From: Ben Hart <ben.hart@jamf.com>
>> To: David Lang <david@lang.hm>
>> Cc: David Lang <david@lang.hm>,
>> Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>>
>> Ok here’s a sanitized blurb from the debug file grepped to the events I’m trying to limit.
>>
>> 2165.510789426:main Q:Reg/w0 : ruleset.c: Filter: check for property 'msg' (value '[2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.30 LEN=361 TOS=0x00 PREC=0x00 TTL=64 ID=29068 DF PROTO=UDP SPT=52225 DPT=9999 LEN=341 ') contains 'filter_': FALSE
>> 2165.510961295:main Q:Reg/w0 : rainerscript.c: rainerscript: (string) var 1: '[2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.30 LEN=361 TOS=0x00 PREC=0x00 TTL=64 ID=29068 DF PROTO=UDP SPT=52225 DPT=9999 LEN=341 '
>> 2165.512976011:main Q:Reg/w0 : omfile.c: omfile: write to stream, pData->pStrm 0x7f9e88001890, lenBuf 254, strt data Sep 26 14:56:05 rsyslog-server kernel: [2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08
>> 2165.513114940:main Q:Reg/w0 : omfile.c: omfile: write to stream, pData->pStrm 0x7f9e88004040, lenBuf 254, strt data Sep 26 14:56:05 rsyslog-server kernel: [2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08
>> 2165.545162782:vcsa9999:Reg/w0: rainerscript.c: 2165.545175184:main Q:Reg/w0 : ruleset.c: processBATCH: next msg 0: [2070020.676314] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.3
>>
>>
>> So it’s def hitting the main queue, and it’s matching correctly. Could not find any mention of imuxsock, or the word socket along with firewalld. Oh wait a minute, FWD natively logs to syslog so maybe I can still use an input for imuxsock but then possibly filter my programname or msg to impose the limits?
>>
>>
>> From: Ben Hart <ben.hart@jamf.com>
>> Date: Monday, September 26, 2022 at 3:51 PM
>> To: David Lang <david@lang.hm>
>> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>> Understood, So I’ve never actually changed the default template, is there a way to engage that from within a .conf or does it need to be in rsyslog.conf? This server is currently handling logs from about 15 sources and SplunkUF is handing them off to Splunk Cloud so, unless I have to I’ don’t wanna mess with those.
>>
>> From: David Lang <david@lang.hm>
>> Date: Monday, September 26, 2022 at 3:47 PM
>> To: Ben Hart <ben.hart@jamf.com>
>> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>> log the message with the template RSYSLOG_DebugFormat and it will show you most
>> of the parsed properties from the message.
>>
>> (I think you are correct, but it's best to be sure)
>>
>> David Lang
>>
>> On Mon, 26 Sep 2022, Ben Hart wrote:
>>
>>> (Apologies, somehow I sent this before it was complete.)
>>>
>>> David, So yeah you are right.. I didn?t dig into omfile and assumed from the Actions page those were available to most. So I have a .conf with 13 rulesets with matching inputs and I?ve found that you can ratelimit on the input side however this source is local (firewalld).
>>> If I am loading modules imudp, imtcp and imuxsock and since I do have a .conf to send all of the logs from that source to a specific file:
>>>
>>> :msg,contains," Syslog-In " /var/log/firewalld
>>> & stop
>>>
>>> Am I correct in thinking that an input type for this would be imuxsock? And if so, could I setup a ruleset with matching inputs to then attempt to rate limit it?
>>>
>>> Ben
>>>
>>>
>>> From: David Lang <david@lang.hm>
>>> Date: Monday, September 26, 2022 at 2:52 PM
>>> To: Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>>> Cc: Ben Hart <ben.hart@jamf.com>
>>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>>> I think it's a valid parameter to ommail, but I would be surprised to see it
>>> under omfile (and if it is, check what version you are running vs the version
>>> you are seeing it documented)
>>>
>>> rsyslog is not an event correlation engine, it has some minimal things, but they
>>> really aren't very good.
>>>
>>> What I do in a situation like yours is to filter the messages and send them to
>>> an external event correlation engine (I tend to use Simple Event Correlator) and
>>> then have it generate alert or summary messages, feeding them back in to rsyslog
>>> (watch out that you don't generate a loop in this process)
>>>
>>> David Lang
>>>
>>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Action params with exec.OnlyEveryNthTime [ In reply to ]
On Mon, 26 Sep 2022, Ben Hart wrote:

> Yeah sorry, here’s a block of matched return:
>
> FROMHOST: 'rsyslog-server', fromhost-ip: '127.0.0.1', HOSTNAME: 'rsyslog-server', PRI: 6,
> syslogtag 'kernel:', programname: 'kernel', APP-NAME: 'kernel', PROCID: '-', MSGID: '-',
> TIMESTAMP: 'Sep 26 16:10:01', STRUCTURED-DATA: '-',
> msg: '[2074456.628051] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.160.115 DST=1.2.160.30 LEN=205 TOS=0x00 PREC=0x00 TTL=64 ID=65500 DF PROTO=UDP SPT=36775 DPT=20525 LEN=185 '
> escaped msg: '[2074456.628051] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.160.115 DST=1.2.160.30 LEN=205 TOS=0x00 PREC=0x00 TTL=64 ID=65500 DF PROTO=UDP SPT=36775 DPT=20525 LEN=185 '
> inputname: imklog rawmsg: '[2074456.628051] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.160.115 DST=1.2.160.30 LEN=205 TOS=0x00 PREC=0x00 TTL=64 ID=65500 DF PROTO=UDP SPT=36775 DPT=20525 LEN=185 '
> $!:
> $.:
> $/:

thanks, that gives good info. Yes, the iptables logging is by/from the kernel,
so you would need to match VCSA or Syslog-IN as you note.

> I do have another conf file that I found some excellent examples of that
> utilizes a catch_all ruleset/queue with if/thens below it. I’d rather use the
> modern syntaxes when I can so Im thinking of trying to adapt something like
> that to this. Except, in this case the format I’d mimic is directing matches
> to 3 different log files. I don’t need to do that, they all need to stay in
> the firewalld file except with the VCSA match being rate limited.

note that the different files all get merged together before rsyslog looks at
them, start rsyslog with -o /path/to/file and the file will have the combined
file as rsyslog sees it.

Because of how the files are combined, trying to put things into an included
file may not have the effect you wish it did because of how the files are
combined.

you can put a ruleset on the inkern module, or you can just put the filter in
the default ruleset


This is a nice configuration, it keeps the remote messages out of the main
queue, putting them in their own queue, and then classifies the messages,
sending them to separate rulesets (which could have their own queues if needed)

> Here's the one I’m talking about:
>
> ruleset(name="f_all" queue.type="LinkedList" queue.workerthreads="4" queue.size="100000") {

note that lots of worker threads is likely to hurt performance rather than help
it. Only increase worker threads if you see that one of them is maxing out a
core (running top, hit 'H" to see the threads and look for a thread that is
using 100% cpu), too many threads reduces batch size and causing thrashing on
the locks, eating CPU without increasing throughput.

> # WLC
> if ($hostname contains 'wlc') then{
> call r_wlc
> stop
> }
> # ESXI
> if ($hostname contains 'vmhost') then{
> call r_vmhost
> stop
> }
> # Catch All
> call r_catch_all
> stop
> }
> # add vmhosts #
> template(name="d_catch_all" type="string" string="/var/log/remote-syslog/undefined.log")
> template(name="d_wlc" type="string" string="/var/log/remote-syslog/wlc.log")
> template(name="d_vmhost" type="string" string="/var/log/remote-syslog/esxi.log")

note that none of these templates include a variable in them, so you don't need
them to be dynamic (i.e. you could use file instead of dynafile), dynafile is if
you wanted something like /var/log/remote-syslog/type/hostname.log, and if you
wanted that you could do

if ($hostname contains 'wlc') then{
set $.type='wlc';
} else if ($hostname contains 'vmhost') then{
set $.type='esxi';
} else set $.type='undefined';

$template type_output,"/var/log/remotelog/%$.type%/%hostname%.log"
?type_output

(I much prefer the new syntax for conditionals, but for simple output and simple
templates, the old syntax is more compact and still very readable)

David Lang


> ruleset(name="r_wlc"){
> action(type="omfile" Dynafile="d_wlc")
> }
> ruleset(name="r_vmhost"){
> action(type="omfile" Dynafile="d_vmhost")
> }
> ruleset(name="r_catch_all"){
> action(type="omfile" Dynafile="d_catch_all")
> }
>
> input(type="imudp" port="514" ruleset="f_all")
>
>
> From: David Lang <david@lang.hm>
> Date: Monday, September 26, 2022 at 5:31 PM
> To: Ben Hart <ben.hart@jamf.com>
> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
> ok, I don't see the sample of the debug format message, please send one or two
> of these to the body of the message rather than trying to use an attachment (I
> assume that's what you tried)
>
> your original approach to filter these out to a separate file and then throw
> them away is the place to start, you just need to do that before any config
> lines that do other things with the logs.
>
> remember that you can filter on any variable, and you can use the simple
> if...then format (where you can combine multiple criteria) rather than the old
> format you used below.
>
> David Lang
>
>
> On Mon, 26 Sep 2022, Ben Hart wrote:
>
>> Date: Mon, 26 Sep 2022 21:19:43 +0000
>> From: Ben Hart <ben.hart@jamf.com>
>> To: David Lang <david@lang.hm>
>> Cc: David Lang <david@lang.hm>,
>> Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>>
>> OK I imagine I did something wrong.. changed the firewalld.conf to exactly what you suggested, bounced rsyslog and gave it a minute. The new fierwalld-debug file remained empty. Do I need to paste that DebugFormat template in somewhere? Also tried pasting the template definition into the bottom of this same file. It did not correct the problem.
>>
>> I do see numerous omfile suspension warnings in systemctl status rsyslog, I presume it’s referring to the -debug file. Rsyslogd -N1 gives no errors. Also the rules no longer matching on the “” value. Goes back to defaults which is only logging denied packets to the file and all else into syslog.
>>
>> After struggling a bit, then trying to simplify it I just appended the template onto the tail of the original line. I let it run for a few minutes then reverted back to original. But I now have a sample of log with the debugformat matching my “” below. And it’s using the imklog module, but reading about that module I get the impression that it’s nor normal nor probably a good idea to be handling it. So with that in mind, maybe it’s a better idea to try to keep this data out of that module entirely.
>>
>> But, TBH I have no idea where to go from here. I still need to limit this traffic but since it’s hitting the kernel, and it’s local, and I cannot restrict it from the source… all suggestions welcome.
>> Thanks
>>
>> From: David Lang <david@lang.hm>
>> Date: Monday, September 26, 2022 at 4:13 PM
>> To: Ben Hart <ben.hart@jamf.com>
>> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>> currently you have
>>
>>>> :msg,contains," Syslog-In " /var/log/firewalld
>>>> & stop
>>
>> change this to
>> :msg,contains," Syslog-In " {
>> /var/log/firewalld
>> /var/log/firewalld-debug;RSYSLOG_DebugFormat
>> stop
>> }
>>
>> (a little easier to read than using &)
>>
>> then look at the firewalld-debug file
>>
>> David Lang
>>
>> On Mon, 26 Sep 2022, Ben Hart wrote:
>>
>>> Date: Mon, 26 Sep 2022 20:09:40 +0000
>>> From: Ben Hart <ben.hart@jamf.com>
>>> To: David Lang <david@lang.hm>
>>> Cc: David Lang <david@lang.hm>,
>>> Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>>>
>>> Ok here’s a sanitized blurb from the debug file grepped to the events I’m trying to limit.
>>>
>>> 2165.510789426:main Q:Reg/w0 : ruleset.c: Filter: check for property 'msg' (value '[2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.30 LEN=361 TOS=0x00 PREC=0x00 TTL=64 ID=29068 DF PROTO=UDP SPT=52225 DPT=9999 LEN=341 ') contains 'filter_': FALSE
>>> 2165.510961295:main Q:Reg/w0 : rainerscript.c: rainerscript: (string) var 1: '[2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.30 LEN=361 TOS=0x00 PREC=0x00 TTL=64 ID=29068 DF PROTO=UDP SPT=52225 DPT=9999 LEN=341 '
>>> 2165.512976011:main Q:Reg/w0 : omfile.c: omfile: write to stream, pData->pStrm 0x7f9e88001890, lenBuf 254, strt data Sep 26 14:56:05 rsyslog-server kernel: [2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08
>>> 2165.513114940:main Q:Reg/w0 : omfile.c: omfile: write to stream, pData->pStrm 0x7f9e88004040, lenBuf 254, strt data Sep 26 14:56:05 rsyslog-server kernel: [2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08
>>> 2165.545162782:vcsa9999:Reg/w0: rainerscript.c: 2165.545175184:main Q:Reg/w0 : ruleset.c: processBATCH: next msg 0: [2070020.676314] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.3
>>>
>>>
>>> So it’s def hitting the main queue, and it’s matching correctly. Could not find any mention of imuxsock, or the word socket along with firewalld. Oh wait a minute, FWD natively logs to syslog so maybe I can still use an input for imuxsock but then possibly filter my programname or msg to impose the limits?
>>>
>>>
>>> From: Ben Hart <ben.hart@jamf.com>
>>> Date: Monday, September 26, 2022 at 3:51 PM
>>> To: David Lang <david@lang.hm>
>>> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>>> Understood, So I’ve never actually changed the default template, is there a way to engage that from within a .conf or does it need to be in rsyslog.conf? This server is currently handling logs from about 15 sources and SplunkUF is handing them off to Splunk Cloud so, unless I have to I’ don’t wanna mess with those.
>>>
>>> From: David Lang <david@lang.hm>
>>> Date: Monday, September 26, 2022 at 3:47 PM
>>> To: Ben Hart <ben.hart@jamf.com>
>>> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>>> log the message with the template RSYSLOG_DebugFormat and it will show you most
>>> of the parsed properties from the message.
>>>
>>> (I think you are correct, but it's best to be sure)
>>>
>>> David Lang
>>>
>>> On Mon, 26 Sep 2022, Ben Hart wrote:
>>>
>>>> (Apologies, somehow I sent this before it was complete.)
>>>>
>>>> David, So yeah you are right.. I didn?t dig into omfile and assumed from the Actions page those were available to most. So I have a .conf with 13 rulesets with matching inputs and I?ve found that you can ratelimit on the input side however this source is local (firewalld).
>>>> If I am loading modules imudp, imtcp and imuxsock and since I do have a .conf to send all of the logs from that source to a specific file:
>>>>
>>>> :msg,contains," Syslog-In " /var/log/firewalld
>>>> & stop
>>>>
>>>> Am I correct in thinking that an input type for this would be imuxsock? And if so, could I setup a ruleset with matching inputs to then attempt to rate limit it?
>>>>
>>>> Ben
>>>>
>>>>
>>>> From: David Lang <david@lang.hm>
>>>> Date: Monday, September 26, 2022 at 2:52 PM
>>>> To: Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>>>> Cc: Ben Hart <ben.hart@jamf.com>
>>>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>>>> I think it's a valid parameter to ommail, but I would be surprised to see it
>>>> under omfile (and if it is, check what version you are running vs the version
>>>> you are seeing it documented)
>>>>
>>>> rsyslog is not an event correlation engine, it has some minimal things, but they
>>>> really aren't very good.
>>>>
>>>> What I do in a situation like yours is to filter the messages and send them to
>>>> an external event correlation engine (I tend to use Simple Event Correlator) and
>>>> then have it generate alert or summary messages, feeding them back in to rsyslog
>>>> (watch out that you don't generate a loop in this process)
>>>>
>>>> David Lang
>>>>
>>>
>>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.