Yeah sorry, here’s a block of matched return:
FROMHOST: 'rsyslog-server', fromhost-ip: '127.0.0.1', HOSTNAME: 'rsyslog-server', PRI: 6,
syslogtag 'kernel:', programname: 'kernel', APP-NAME: 'kernel', PROCID: '-', MSGID: '-',
TIMESTAMP: 'Sep 26 16:10:01', STRUCTURED-DATA: '-',
msg: '[2074456.628051] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.160.115 DST=1.2.160.30 LEN=205 TOS=0x00 PREC=0x00 TTL=64 ID=65500 DF PROTO=UDP SPT=36775 DPT=20525 LEN=185 '
escaped msg: '[2074456.628051] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.160.115 DST=1.2.160.30 LEN=205 TOS=0x00 PREC=0x00 TTL=64 ID=65500 DF PROTO=UDP SPT=36775 DPT=20525 LEN=185 '
inputname: imklog rawmsg: '[2074456.628051] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.160.115 DST=1.2.160.30 LEN=205 TOS=0x00 PREC=0x00 TTL=64 ID=65500 DF PROTO=UDP SPT=36775 DPT=20525 LEN=185 '
$!:
$.:
$/:
I do have another conf file that I found some excellent examples of that utilizes a catch_all ruleset/queue with if/thens below it. I’d rather use the modern syntaxes when I can so Im thinking of trying to adapt something like that to this. Except, in this case the format I’d mimic is directing matches to 3 different log files. I don’t need to do that, they all need to stay in the firewalld file except with the VCSA match being rate limited.
Here's the one I’m talking about:
ruleset(name="f_all" queue.type="LinkedList" queue.workerthreads="4" queue.size="100000") {
# WLC
if ($hostname contains 'wlc') then{
call r_wlc
stop
}
# ESXI
if ($hostname contains 'vmhost') then{
call r_vmhost
stop
}
# Catch All
call r_catch_all
stop
}
# add vmhosts #
template(name="d_catch_all" type="string" string="/var/log/remote-syslog/undefined.log")
template(name="d_wlc" type="string" string="/var/log/remote-syslog/wlc.log")
template(name="d_vmhost" type="string" string="/var/log/remote-syslog/esxi.log")
ruleset(name="r_wlc"){
action(type="omfile" Dynafile="d_wlc")
}
ruleset(name="r_vmhost"){
action(type="omfile" Dynafile="d_vmhost")
}
ruleset(name="r_catch_all"){
action(type="omfile" Dynafile="d_catch_all")
}
input(type="imudp" port="514" ruleset="f_all")
From: David Lang <david@lang.hm>
Date: Monday, September 26, 2022 at 5:31 PM
To: Ben Hart <ben.hart@jamf.com>
Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
ok, I don't see the sample of the debug format message, please send one or two
of these to the body of the message rather than trying to use an attachment (I
assume that's what you tried)
your original approach to filter these out to a separate file and then throw
them away is the place to start, you just need to do that before any config
lines that do other things with the logs.
remember that you can filter on any variable, and you can use the simple
if...then format (where you can combine multiple criteria) rather than the old
format you used below.
David Lang
On Mon, 26 Sep 2022, Ben Hart wrote:
> Date: Mon, 26 Sep 2022 21:19:43 +0000
> From: Ben Hart <ben.hart@jamf.com>
> To: David Lang <david@lang.hm>
> Cc: David Lang <david@lang.hm>,
> Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>
> OK I imagine I did something wrong.. changed the firewalld.conf to exactly what you suggested, bounced rsyslog and gave it a minute. The new fierwalld-debug file remained empty. Do I need to paste that DebugFormat template in somewhere? Also tried pasting the template definition into the bottom of this same file. It did not correct the problem.
>
> I do see numerous omfile suspension warnings in systemctl status rsyslog, I presume it’s referring to the -debug file. Rsyslogd -N1 gives no errors. Also the rules no longer matching on the “” value. Goes back to defaults which is only logging denied packets to the file and all else into syslog.
>
> After struggling a bit, then trying to simplify it I just appended the template onto the tail of the original line. I let it run for a few minutes then reverted back to original. But I now have a sample of log with the debugformat matching my “” below. And it’s using the imklog module, but reading about that module I get the impression that it’s nor normal nor probably a good idea to be handling it. So with that in mind, maybe it’s a better idea to try to keep this data out of that module entirely.
>
> But, TBH I have no idea where to go from here. I still need to limit this traffic but since it’s hitting the kernel, and it’s local, and I cannot restrict it from the source… all suggestions welcome.
> Thanks
>
> From: David Lang <david@lang.hm>
> Date: Monday, September 26, 2022 at 4:13 PM
> To: Ben Hart <ben.hart@jamf.com>
> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
> currently you have
>
>>> :msg,contains," Syslog-In " /var/log/firewalld
>>> & stop
>
> change this to
> :msg,contains," Syslog-In " {
> /var/log/firewalld
> /var/log/firewalld-debug;RSYSLOG_DebugFormat
> stop
> }
>
> (a little easier to read than using &)
>
> then look at the firewalld-debug file
>
> David Lang
>
> On Mon, 26 Sep 2022, Ben Hart wrote:
>
>> Date: Mon, 26 Sep 2022 20:09:40 +0000
>> From: Ben Hart <ben.hart@jamf.com>
>> To: David Lang <david@lang.hm>
>> Cc: David Lang <david@lang.hm>,
>> Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>>
>> Ok here’s a sanitized blurb from the debug file grepped to the events I’m trying to limit.
>>
>> 2165.510789426:main Q:Reg/w0 : ruleset.c: Filter: check for property 'msg' (value '[2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.30 LEN=361 TOS=0x00 PREC=0x00 TTL=64 ID=29068 DF PROTO=UDP SPT=52225 DPT=9999 LEN=341 ') contains 'filter_': FALSE
>> 2165.510961295:main Q:Reg/w0 : rainerscript.c: rainerscript: (string) var 1: '[2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.30 LEN=361 TOS=0x00 PREC=0x00 TTL=64 ID=29068 DF PROTO=UDP SPT=52225 DPT=9999 LEN=341 '
>> 2165.512976011:main Q:Reg/w0 : omfile.c: omfile: write to stream, pData->pStrm 0x7f9e88001890, lenBuf 254, strt data Sep 26 14:56:05 rsyslog-server kernel: [2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08
>> 2165.513114940:main Q:Reg/w0 : omfile.c: omfile: write to stream, pData->pStrm 0x7f9e88004040, lenBuf 254, strt data Sep 26 14:56:05 rsyslog-server kernel: [2070020.637509] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08
>> 2165.545162782:vcsa9999:Reg/w0: rainerscript.c: 2165.545175184:main Q:Reg/w0 : ruleset.c: processBATCH: next msg 0: [2070020.676314] VCSA Syslog-IN IN=ens192 OUT= MAC=00:50:56:a0:03:90:00:0c:29:cb:fc:14:08:00 SRC=1.2.3.4160.115 DST=1.2.3.4160.3
>>
>>
>> So it’s def hitting the main queue, and it’s matching correctly. Could not find any mention of imuxsock, or the word socket along with firewalld. Oh wait a minute, FWD natively logs to syslog so maybe I can still use an input for imuxsock but then possibly filter my programname or msg to impose the limits?
>>
>>
>> From: Ben Hart <ben.hart@jamf.com>
>> Date: Monday, September 26, 2022 at 3:51 PM
>> To: David Lang <david@lang.hm>
>> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>> Understood, So I’ve never actually changed the default template, is there a way to engage that from within a .conf or does it need to be in rsyslog.conf? This server is currently handling logs from about 15 sources and SplunkUF is handing them off to Splunk Cloud so, unless I have to I’ don’t wanna mess with those.
>>
>> From: David Lang <david@lang.hm>
>> Date: Monday, September 26, 2022 at 3:47 PM
>> To: Ben Hart <ben.hart@jamf.com>
>> Cc: David Lang <david@lang.hm>, Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>> log the message with the template RSYSLOG_DebugFormat and it will show you most
>> of the parsed properties from the message.
>>
>> (I think you are correct, but it's best to be sure)
>>
>> David Lang
>>
>> On Mon, 26 Sep 2022, Ben Hart wrote:
>>
>>> (Apologies, somehow I sent this before it was complete.)
>>>
>>> David, So yeah you are right.. I didn?t dig into omfile and assumed from the Actions page those were available to most. So I have a .conf with 13 rulesets with matching inputs and I?ve found that you can ratelimit on the input side however this source is local (firewalld).
>>> If I am loading modules imudp, imtcp and imuxsock and since I do have a .conf to send all of the logs from that source to a specific file:
>>>
>>> :msg,contains," Syslog-In " /var/log/firewalld
>>> & stop
>>>
>>> Am I correct in thinking that an input type for this would be imuxsock? And if so, could I setup a ruleset with matching inputs to then attempt to rate limit it?
>>>
>>> Ben
>>>
>>>
>>> From: David Lang <david@lang.hm>
>>> Date: Monday, September 26, 2022 at 2:52 PM
>>> To: Ben Hart via rsyslog <rsyslog@lists.adiscon.com>
>>> Cc: Ben Hart <ben.hart@jamf.com>
>>> Subject: Re: [rsyslog] Action params with exec.OnlyEveryNthTime
>>> I think it's a valid parameter to ommail, but I would be surprised to see it
>>> under omfile (and if it is, check what version you are running vs the version
>>> you are seeing it documented)
>>>
>>> rsyslog is not an event correlation engine, it has some minimal things, but they
>>> really aren't very good.
>>>
>>> What I do in a situation like yours is to filter the messages and send them to
>>> an external event correlation engine (I tend to use Simple Event Correlator) and
>>> then have it generate alert or summary messages, feeding them back in to rsyslog
>>> (watch out that you don't generate a loop in this process)
>>>
>>> David Lang
>>>
>>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow
https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.