Mailing List Archive

rate limiting output will just cause things to back up, it doesn't throw away
the messages. It would be better for you to detect these messages and feed them
into an external event correlation engine (Simple Event Correltator for
example), and have that engine then send you logs that you keep (which could be
'system X sent Y OOM messages in the last 5 min' or something like that)

David Lang



On Tue, 2 Apr 2024, Prasad Koya via rsyslog wrote:

> Date: Tue, 2 Apr 2024 18:25:01 -0700
> From: Prasad Koya via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog-users <rsyslog@lists.adiscon.com>
> Cc: Prasad Koya <prasad@arista.com>
> Subject: [rsyslog] rate limiting logs written to a file
>
> Hi
>
>
> module( load="imuxsock" )
> module( load="imklog" )
>
> if ($syslogfacility-text == 'kern' and $msg contains "Out of memory") then {
> action(type="omfile" queue.size="1000" queue.type="LinkedList"
> queue.dequeueSlowDown="3600000000" file="/var/log/oom")
> }
>
>
> If we have a flurry of, for example, "Out of memory" messages from the
> facility 'kern', we'd like to keep only, say, one per hour in a
> separate file.
>
> https://rsyslog.readthedocs.io/en/latest/rainerscript/queue_parameters.html
> says do not set low value for queue.size.
>
> Above page also says queue.dequeueslowdown can be used as
> rate-limiting. Maybe that doesn't apply for "omfile" as I don't see
> above rsyslog.conf working. high and low watermarks are not applicable
> here as it's not a disk based queue. Appreciate any pointers.
>
> Thank you.
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

This sounds a bit like you are looking for this:

https://www.rsyslog.com/doc/tutorials/log_sampling.html

HTH
Rainer

El mié, 3 abr 2024 a las 3:25, Prasad Koya via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> Hi
>
>
> module( load="imuxsock" )
> module( load="imklog" )
>
> if ($syslogfacility-text == 'kern' and $msg contains "Out of memory") then {
> action(type="omfile" queue.size="1000" queue.type="LinkedList"
> queue.dequeueSlowDown="3600000000" file="/var/log/oom")
> }
>
>
> If we have a flurry of, for example, "Out of memory" messages from the
> facility 'kern', we'd like to keep only, say, one per hour in a
> separate file.
>
> https://rsyslog.readthedocs.io/en/latest/rainerscript/queue_parameters.html
> says do not set low value for queue.size.
>
> Above page also says queue.dequeueslowdown can be used as
> rate-limiting. Maybe that doesn't apply for "omfile" as I don't see
> above rsyslog.conf working. high and low watermarks are not applicable
> here as it's not a disk based queue. Appreciate any pointers.
>
> Thank you.
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

A piece of feedback against
https://www.rsyslog.com/doc/tutorials/random_sampling.html

I know it's obvious for some people but for some it might not be - it
will collect _about_ 20% of logs provided the overall number of events
is big enough. Since it's a random-based mechanism, it's... random.

Additionally, I had a similar thing implemented for load-balancing to
different "backends" and for some reason it didn't work uniformly (I
think I already wrote about this) - one of four backends was never
chosen. The thing is I was choosing not based directly on random(4) but
on (random(some_bigger_integer) % 4) which I thought would yield more
uniform randomness. It appears I was wrong.

MK


On 3.04.2024 08:51, Rainer Gerhards via rsyslog wrote:
> This sounds a bit like you are looking for this:
>
> https://www.rsyslog.com/doc/tutorials/log_sampling.html
>
> HTH
> Rainer
>
> El mié, 3 abr 2024 a las 3:25, Prasad Koya via rsyslog
> (<rsyslog@lists.adiscon.com>) escribió:
>> Hi
>>
>>
>> module( load="imuxsock" )
>> module( load="imklog" )
>>
>> if ($syslogfacility-text == 'kern' and $msg contains "Out of memory") then {
>> action(type="omfile" queue.size="1000" queue.type="LinkedList"
>> queue.dequeueSlowDown="3600000000" file="/var/log/oom")
>> }
>>
>>
>> If we have a flurry of, for example, "Out of memory" messages from the
>> facility 'kern', we'd like to keep only, say, one per hour in a
>> separate file.
>>
>> https://rsyslog.readthedocs.io/en/latest/rainerscript/queue_parameters.html
>> says do not set low value for queue.size.
>>
>> Above page also says queue.dequeueslowdown can be used as
>> rate-limiting. Maybe that doesn't apply for "omfile" as I don't see
>> above rsyslog.conf working. high and low watermarks are not applicable
>> here as it's not a disk based queue. Appreciate any pointers.
>>
>> Thank you.
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Thanks for the replies.

My requirement is that I have a daemon that may generate a burst of
syslogs, say, every minute (when a certain subsystem is overloaded).
We do not want to write all these syslogs to /var/log/messages. We
also do not want to forward so many of those syslogs to a remote
syslog server. We wanted to rate limit them, maybe 1 per hour or so.
For now, I have tried 'omprog' module and sending such syslogs to an
external program.


module( load="omprog" )

if ($syslogfacility-text == 'daemon' and $msg contains "Out of memory") then {
action(type="omprog"
binary="/usr/bin/rate_limit_daemon_syslogs.sh"
template="RSYSLOG_FileFormat")
stop
}

/usr/bin/rate_limit_daemon_syslogs.sh maintains state on when was the
last syslog forwarded or written locally. If this script decides its
ok to syslog, then it will call 'logger' with differently worded
message and different facility.

In our case we'd like to limit these logs at the source and not at an
intermediate syslog collector. I took a quick look at the sampling
docs and that may not apply to our scenario.

Thanks.

On Wed, Apr 3, 2024 at 12:07?AM Mariusz Kruk via rsyslog
<rsyslog@lists.adiscon.com> wrote:
>
> A piece of feedback against
> https://www.rsyslog.com/doc/tutorials/random_sampling.html
>
> I know it's obvious for some people but for some it might not be - it
> will collect _about_ 20% of logs provided the overall number of events
> is big enough. Since it's a random-based mechanism, it's... random.
>
> Additionally, I had a similar thing implemented for load-balancing to
> different "backends" and for some reason it didn't work uniformly (I
> think I already wrote about this) - one of four backends was never
> chosen. The thing is I was choosing not based directly on random(4) but
> on (random(some_bigger_integer) % 4) which I thought would yield more
> uniform randomness. It appears I was wrong.
>
> MK
>
>
> On 3.04.2024 08:51, Rainer Gerhards via rsyslog wrote:
> > This sounds a bit like you are looking for this:
> >
> > https://www.rsyslog.com/doc/tutorials/log_sampling.html
> >
> > HTH
> > Rainer
> >
> > El mié, 3 abr 2024 a las 3:25, Prasad Koya via rsyslog
> > (<rsyslog@lists.adiscon.com>) escribió:
> >> Hi
> >>
> >>
> >> module( load="imuxsock" )
> >> module( load="imklog" )
> >>
> >> if ($syslogfacility-text == 'kern' and $msg contains "Out of memory") then {
> >> action(type="omfile" queue.size="1000" queue.type="LinkedList"
> >> queue.dequeueSlowDown="3600000000" file="/var/log/oom")
> >> }
> >>
> >>
> >> If we have a flurry of, for example, "Out of memory" messages from the
> >> facility 'kern', we'd like to keep only, say, one per hour in a
> >> separate file.
> >>
> >> https://rsyslog.readthedocs.io/en/latest/rainerscript/queue_parameters.html
> >> says do not set low value for queue.size.
> >>
> >> Above page also says queue.dequeueslowdown can be used as
> >> rate-limiting. Maybe that doesn't apply for "omfile" as I don't see
> >> above rsyslog.conf working. high and low watermarks are not applicable
> >> here as it's not a disk based queue. Appreciate any pointers.
> >>
> >> Thank you.
> >> _______________________________________________
> >> rsyslog mailing list
> >> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com/professional-services/
> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

This sounds like you are looking for

action.execOnlyOnceEveryInterval="3600"

for the action in question.

Doc:
https://www.rsyslog.com/doc/configuration/actions.html

Sample use in ommal doc:
https://www.rsyslog.com/doc/configuration/modules/ommail.html

HTH
Rainer

El lun, 8 abr 2024 a las 8:36, Prasad Koya via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> Thanks for the replies.
>
> My requirement is that I have a daemon that may generate a burst of
> syslogs, say, every minute (when a certain subsystem is overloaded).
> We do not want to write all these syslogs to /var/log/messages. We
> also do not want to forward so many of those syslogs to a remote
> syslog server. We wanted to rate limit them, maybe 1 per hour or so.
> For now, I have tried 'omprog' module and sending such syslogs to an
> external program.
>
>
> module( load="omprog" )
>
> if ($syslogfacility-text == 'daemon' and $msg contains "Out of memory") then {
> action(type="omprog"
> binary="/usr/bin/rate_limit_daemon_syslogs.sh"
> template="RSYSLOG_FileFormat")
> stop
> }
>
> /usr/bin/rate_limit_daemon_syslogs.sh maintains state on when was the
> last syslog forwarded or written locally. If this script decides its
> ok to syslog, then it will call 'logger' with differently worded
> message and different facility.
>
> In our case we'd like to limit these logs at the source and not at an
> intermediate syslog collector. I took a quick look at the sampling
> docs and that may not apply to our scenario.
>
> Thanks.
>
> On Wed, Apr 3, 2024 at 12:07?AM Mariusz Kruk via rsyslog
> <rsyslog@lists.adiscon.com> wrote:
> >
> > A piece of feedback against
> > https://www.rsyslog.com/doc/tutorials/random_sampling.html
> >
> > I know it's obvious for some people but for some it might not be - it
> > will collect _about_ 20% of logs provided the overall number of events
> > is big enough. Since it's a random-based mechanism, it's... random.
> >
> > Additionally, I had a similar thing implemented for load-balancing to
> > different "backends" and for some reason it didn't work uniformly (I
> > think I already wrote about this) - one of four backends was never
> > chosen. The thing is I was choosing not based directly on random(4) but
> > on (random(some_bigger_integer) % 4) which I thought would yield more
> > uniform randomness. It appears I was wrong.
> >
> > MK
> >
> >
> > On 3.04.2024 08:51, Rainer Gerhards via rsyslog wrote:
> > > This sounds a bit like you are looking for this:
> > >
> > > https://www.rsyslog.com/doc/tutorials/log_sampling.html
> > >
> > > HTH
> > > Rainer
> > >
> > > El mié, 3 abr 2024 a las 3:25, Prasad Koya via rsyslog
> > > (<rsyslog@lists.adiscon.com>) escribió:
> > >> Hi
> > >>
> > >>
> > >> module( load="imuxsock" )
> > >> module( load="imklog" )
> > >>
> > >> if ($syslogfacility-text == 'kern' and $msg contains "Out of memory") then {
> > >> action(type="omfile" queue.size="1000" queue.type="LinkedList"
> > >> queue.dequeueSlowDown="3600000000" file="/var/log/oom")
> > >> }
> > >>
> > >>
> > >> If we have a flurry of, for example, "Out of memory" messages from the
> > >> facility 'kern', we'd like to keep only, say, one per hour in a
> > >> separate file.
> > >>
> > >> https://rsyslog.readthedocs.io/en/latest/rainerscript/queue_parameters.html
> > >> says do not set low value for queue.size.
> > >>
> > >> Above page also says queue.dequeueslowdown can be used as
> > >> rate-limiting. Maybe that doesn't apply for "omfile" as I don't see
> > >> above rsyslog.conf working. high and low watermarks are not applicable
> > >> here as it's not a disk based queue. Appreciate any pointers.
> > >>
> > >> Thank you.
> > >> _______________________________________________
> > >> rsyslog mailing list
> > >> https://lists.adiscon.net/mailman/listinfo/rsyslog
> > >> http://www.rsyslog.com/professional-services/
> > >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> > > _______________________________________________
> > > rsyslog mailing list
> > > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

you would need to do the filtering with omprog on the sending machines.

I would suggest that rather than throwing them away, you generate a log every
rate-limiting period along the lines of "X number of messages happened in the
last period"

This is really a job for an event correlation engine, since rsyslog is
multi-threaded for performance, it can't count on anything internal seeing every
message, which is why filtering for the messages and sending them to an external
app (single threaded or a very complex multi-threaded thing that correlates
across it's threads) is the right thing to do.

David Lang

On Sun, 7 Apr 2024, Prasad Koya via rsyslog wrote:

> Thanks for the replies.
>
> My requirement is that I have a daemon that may generate a burst of
> syslogs, say, every minute (when a certain subsystem is overloaded).
> We do not want to write all these syslogs to /var/log/messages. We
> also do not want to forward so many of those syslogs to a remote
> syslog server. We wanted to rate limit them, maybe 1 per hour or so.
> For now, I have tried 'omprog' module and sending such syslogs to an
> external program.
>
>
> module( load="omprog" )
>
> if ($syslogfacility-text == 'daemon' and $msg contains "Out of memory") then {
> action(type="omprog"
> binary="/usr/bin/rate_limit_daemon_syslogs.sh"
> template="RSYSLOG_FileFormat")
> stop
> }
>
> /usr/bin/rate_limit_daemon_syslogs.sh maintains state on when was the
> last syslog forwarded or written locally. If this script decides its
> ok to syslog, then it will call 'logger' with differently worded
> message and different facility.
>
> In our case we'd like to limit these logs at the source and not at an
> intermediate syslog collector. I took a quick look at the sampling
> docs and that may not apply to our scenario.
>
> Thanks.
>
> On Wed, Apr 3, 2024 at 12:07?AM Mariusz Kruk via rsyslog
> <rsyslog@lists.adiscon.com> wrote:
>>
>> A piece of feedback against
>> https://www.rsyslog.com/doc/tutorials/random_sampling.html
>>
>> I know it's obvious for some people but for some it might not be - it
>> will collect _about_ 20% of logs provided the overall number of events
>> is big enough. Since it's a random-based mechanism, it's... random.
>>
>> Additionally, I had a similar thing implemented for load-balancing to
>> different "backends" and for some reason it didn't work uniformly (I
>> think I already wrote about this) - one of four backends was never
>> chosen. The thing is I was choosing not based directly on random(4) but
>> on (random(some_bigger_integer) % 4) which I thought would yield more
>> uniform randomness. It appears I was wrong.
>>
>> MK
>>
>>
>> On 3.04.2024 08:51, Rainer Gerhards via rsyslog wrote:
>> > This sounds a bit like you are looking for this:
>> >
>> > https://www.rsyslog.com/doc/tutorials/log_sampling.html
>> >
>> > HTH
>> > Rainer
>> >
>> > El mié, 3 abr 2024 a las 3:25, Prasad Koya via rsyslog
>> > (<rsyslog@lists.adiscon.com>) escribió:
>> >> Hi
>> >>
>> >>
>> >> module( load="imuxsock" )
>> >> module( load="imklog" )
>> >>
>> >> if ($syslogfacility-text == 'kern' and $msg contains "Out of memory") then {
>> >> action(type="omfile" queue.size="1000" queue.type="LinkedList"
>> >> queue.dequeueSlowDown="3600000000" file="/var/log/oom")
>> >> }
>> >>
>> >>
>> >> If we have a flurry of, for example, "Out of memory" messages from the
>> >> facility 'kern', we'd like to keep only, say, one per hour in a
>> >> separate file.
>> >>
>> >> https://rsyslog.readthedocs.io/en/latest/rainerscript/queue_parameters.html
>> >> says do not set low value for queue.size.
>> >>
>> >> Above page also says queue.dequeueslowdown can be used as
>> >> rate-limiting. Maybe that doesn't apply for "omfile" as I don't see
>> >> above rsyslog.conf working. high and low watermarks are not applicable
>> >> here as it's not a disk based queue. Appreciate any pointers.
>> >>
>> >> Thank you.
>> >> _______________________________________________
>> >> rsyslog mailing list
>> >> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> >> http://www.rsyslog.com/professional-services/
>> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>> > _______________________________________________
>> > rsyslog mailing list
>> > https://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com/professional-services/
>> > What's up with rsyslog? Follow https://twitter.com/rgerhards
>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Thank you. "action.execOnlyOnceEveryInterval" is very useful.

We intend to keep all unfiltered syslogs in a local file, say file1,
but log a rate limited message to file2 and forward that message to a
syslog collector.

Is it possible to change the facility, severity or message for the next actions?

Thanks again.

On Sun, Apr 7, 2024 at 11:42?PM David Lang <david@lang.hm> wrote:
>
> you would need to do the filtering with omprog on the sending machines.
>
> I would suggest that rather than throwing them away, you generate a log every
> rate-limiting period along the lines of "X number of messages happened in the
> last period"
>
> This is really a job for an event correlation engine, since rsyslog is
> multi-threaded for performance, it can't count on anything internal seeing every
> message, which is why filtering for the messages and sending them to an external
> app (single threaded or a very complex multi-threaded thing that correlates
> across it's threads) is the right thing to do.
>
> David Lang
>
> On Sun, 7 Apr 2024, Prasad Koya via rsyslog wrote:
>
> > Thanks for the replies.
> >
> > My requirement is that I have a daemon that may generate a burst of
> > syslogs, say, every minute (when a certain subsystem is overloaded).
> > We do not want to write all these syslogs to /var/log/messages. We
> > also do not want to forward so many of those syslogs to a remote
> > syslog server. We wanted to rate limit them, maybe 1 per hour or so.
> > For now, I have tried 'omprog' module and sending such syslogs to an
> > external program.
> >
> >
> > module( load="omprog" )
> >
> > if ($syslogfacility-text == 'daemon' and $msg contains "Out of memory") then {
> > action(type="omprog"
> > binary="/usr/bin/rate_limit_daemon_syslogs.sh"
> > template="RSYSLOG_FileFormat")
> > stop
> > }
> >
> > /usr/bin/rate_limit_daemon_syslogs.sh maintains state on when was the
> > last syslog forwarded or written locally. If this script decides its
> > ok to syslog, then it will call 'logger' with differently worded
> > message and different facility.
> >
> > In our case we'd like to limit these logs at the source and not at an
> > intermediate syslog collector. I took a quick look at the sampling
> > docs and that may not apply to our scenario.
> >
> > Thanks.
> >
> > On Wed, Apr 3, 2024 at 12:07?AM Mariusz Kruk via rsyslog
> > <rsyslog@lists.adiscon.com> wrote:
> >>
> >> A piece of feedback against
> >> https://www.rsyslog.com/doc/tutorials/random_sampling.html
> >>
> >> I know it's obvious for some people but for some it might not be - it
> >> will collect _about_ 20% of logs provided the overall number of events
> >> is big enough. Since it's a random-based mechanism, it's... random.
> >>
> >> Additionally, I had a similar thing implemented for load-balancing to
> >> different "backends" and for some reason it didn't work uniformly (I
> >> think I already wrote about this) - one of four backends was never
> >> chosen. The thing is I was choosing not based directly on random(4) but
> >> on (random(some_bigger_integer) % 4) which I thought would yield more
> >> uniform randomness. It appears I was wrong.
> >>
> >> MK
> >>
> >>
> >> On 3.04.2024 08:51, Rainer Gerhards via rsyslog wrote:
> >> > This sounds a bit like you are looking for this:
> >> >
> >> > https://www.rsyslog.com/doc/tutorials/log_sampling.html
> >> >
> >> > HTH
> >> > Rainer
> >> >
> >> > El mié, 3 abr 2024 a las 3:25, Prasad Koya via rsyslog
> >> > (<rsyslog@lists.adiscon.com>) escribió:
> >> >> Hi
> >> >>
> >> >>
> >> >> module( load="imuxsock" )
> >> >> module( load="imklog" )
> >> >>
> >> >> if ($syslogfacility-text == 'kern' and $msg contains "Out of memory") then {
> >> >> action(type="omfile" queue.size="1000" queue.type="LinkedList"
> >> >> queue.dequeueSlowDown="3600000000" file="/var/log/oom")
> >> >> }
> >> >>
> >> >>
> >> >> If we have a flurry of, for example, "Out of memory" messages from the
> >> >> facility 'kern', we'd like to keep only, say, one per hour in a
> >> >> separate file.
> >> >>
> >> >> https://rsyslog.readthedocs.io/en/latest/rainerscript/queue_parameters.html
> >> >> says do not set low value for queue.size.
> >> >>
> >> >> Above page also says queue.dequeueslowdown can be used as
> >> >> rate-limiting. Maybe that doesn't apply for "omfile" as I don't see
> >> >> above rsyslog.conf working. high and low watermarks are not applicable
> >> >> here as it's not a disk based queue. Appreciate any pointers.
> >> >>
> >> >> Thank you.
> >> >> _______________________________________________
> >> >> rsyslog mailing list
> >> >> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >> http://www.rsyslog.com/professional-services/
> >> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> >> > _______________________________________________
> >> > rsyslog mailing list
> >> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > http://www.rsyslog.com/professional-services/
> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> >> _______________________________________________
> >> rsyslog mailing list
> >> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com/professional-services/
> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

not easily within rsyslog, with an event correlation engine, you have a lot more
capability (which is why I keep pushing you that way :-) )

you can create a template to use when outputting a log, and that log can say
anything (it can be arbitrary text unrelated to the log you received), but that
gets really ugly really quickly

you would combine it essentially with a if/else type arrangement to only be done
if the prior action didn't take place. I don't know if the onceeveryinterval
will work with that.

David Lang


On Mon, 8 Apr 2024, Prasad Koya wrote:

> Thank you. "action.execOnlyOnceEveryInterval" is very useful.
>
> We intend to keep all unfiltered syslogs in a local file, say file1,
> but log a rate limited message to file2 and forward that message to a
> syslog collector.
>
> Is it possible to change the facility, severity or message for the next actions?
>
> Thanks again.
>
> On Sun, Apr 7, 2024 at 11:42?PM David Lang <david@lang.hm> wrote:
>>
>> you would need to do the filtering with omprog on the sending machines.
>>
>> I would suggest that rather than throwing them away, you generate a log every
>> rate-limiting period along the lines of "X number of messages happened in the
>> last period"
>>
>> This is really a job for an event correlation engine, since rsyslog is
>> multi-threaded for performance, it can't count on anything internal seeing every
>> message, which is why filtering for the messages and sending them to an external
>> app (single threaded or a very complex multi-threaded thing that correlates
>> across it's threads) is the right thing to do.
>>
>> David Lang
>>
>> On Sun, 7 Apr 2024, Prasad Koya via rsyslog wrote:
>>
>>> Thanks for the replies.
>>>
>>> My requirement is that I have a daemon that may generate a burst of
>>> syslogs, say, every minute (when a certain subsystem is overloaded).
>>> We do not want to write all these syslogs to /var/log/messages. We
>>> also do not want to forward so many of those syslogs to a remote
>>> syslog server. We wanted to rate limit them, maybe 1 per hour or so.
>>> For now, I have tried 'omprog' module and sending such syslogs to an
>>> external program.
>>>
>>>
>>> module( load="omprog" )
>>>
>>> if ($syslogfacility-text == 'daemon' and $msg contains "Out of memory") then {
>>> action(type="omprog"
>>> binary="/usr/bin/rate_limit_daemon_syslogs.sh"
>>> template="RSYSLOG_FileFormat")
>>> stop
>>> }
>>>
>>> /usr/bin/rate_limit_daemon_syslogs.sh maintains state on when was the
>>> last syslog forwarded or written locally. If this script decides its
>>> ok to syslog, then it will call 'logger' with differently worded
>>> message and different facility.
>>>
>>> In our case we'd like to limit these logs at the source and not at an
>>> intermediate syslog collector. I took a quick look at the sampling
>>> docs and that may not apply to our scenario.
>>>
>>> Thanks.
>>>
>>> On Wed, Apr 3, 2024 at 12:07?AM Mariusz Kruk via rsyslog
>>> <rsyslog@lists.adiscon.com> wrote:
>>>>
>>>> A piece of feedback against
>>>> https://www.rsyslog.com/doc/tutorials/random_sampling.html
>>>>
>>>> I know it's obvious for some people but for some it might not be - it
>>>> will collect _about_ 20% of logs provided the overall number of events
>>>> is big enough. Since it's a random-based mechanism, it's... random.
>>>>
>>>> Additionally, I had a similar thing implemented for load-balancing to
>>>> different "backends" and for some reason it didn't work uniformly (I
>>>> think I already wrote about this) - one of four backends was never
>>>> chosen. The thing is I was choosing not based directly on random(4) but
>>>> on (random(some_bigger_integer) % 4) which I thought would yield more
>>>> uniform randomness. It appears I was wrong.
>>>>
>>>> MK
>>>>
>>>>
>>>> On 3.04.2024 08:51, Rainer Gerhards via rsyslog wrote:
>>>>> This sounds a bit like you are looking for this:
>>>>>
>>>>> https://www.rsyslog.com/doc/tutorials/log_sampling.html
>>>>>
>>>>> HTH
>>>>> Rainer
>>>>>
>>>>> El mié, 3 abr 2024 a las 3:25, Prasad Koya via rsyslog
>>>>> (<rsyslog@lists.adiscon.com>) escribió:
>>>>>> Hi
>>>>>>
>>>>>>
>>>>>> module( load="imuxsock" )
>>>>>> module( load="imklog" )
>>>>>>
>>>>>> if ($syslogfacility-text == 'kern' and $msg contains "Out of memory") then {
>>>>>> action(type="omfile" queue.size="1000" queue.type="LinkedList"
>>>>>> queue.dequeueSlowDown="3600000000" file="/var/log/oom")
>>>>>> }
>>>>>>
>>>>>>
>>>>>> If we have a flurry of, for example, "Out of memory" messages from the
>>>>>> facility 'kern', we'd like to keep only, say, one per hour in a
>>>>>> separate file.
>>>>>>
>>>>>> https://rsyslog.readthedocs.io/en/latest/rainerscript/queue_parameters.html
>>>>>> says do not set low value for queue.size.
>>>>>>
>>>>>> Above page also says queue.dequeueslowdown can be used as
>>>>>> rate-limiting. Maybe that doesn't apply for "omfile" as I don't see
>>>>>> above rsyslog.conf working. high and low watermarks are not applicable
>>>>>> here as it's not a disk based queue. Appreciate any pointers.
>>>>>>
>>>>>> Thank you.
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com/professional-services/
>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com/professional-services/
>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com/professional-services/
>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>>> _______________________________________________
>>> rsyslog mailing list
>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Yes, we get a lot of flexibility with omprog.

However, changing a syslog's facility/priority isn't possible right?
Please see below.

if ($syslogfacility-text == 'daemon' and $msg contains "Out of memory") then {
action(type="omfile"
action.execOnlyOnceEveryInterval="3600"
template="RSYSLOG_FileFormat")

# Change the facility and message severity forever here for the
next action(s). for example to user.alert
}

# Forward all 'user'.crit and severe to syslog collector.
user.crit action(type="omfwd" target="10.11.12.13" protocol="udp"
port="514" template="MsgForwardFormat")

On Mon, Apr 8, 2024 at 6:37?PM David Lang <david@lang.hm> wrote:
>
> not easily within rsyslog, with an event correlation engine, you have a lot more
> capability (which is why I keep pushing you that way :-) )
>
> you can create a template to use when outputting a log, and that log can say
> anything (it can be arbitrary text unrelated to the log you received), but that
> gets really ugly really quickly
>
> you would combine it essentially with a if/else type arrangement to only be done
> if the prior action didn't take place. I don't know if the onceeveryinterval
> will work with that.
>
> David Lang
>
>
> On Mon, 8 Apr 2024, Prasad Koya wrote:
>
> > Thank you. "action.execOnlyOnceEveryInterval" is very useful.
> >
> > We intend to keep all unfiltered syslogs in a local file, say file1,
> > but log a rate limited message to file2 and forward that message to a
> > syslog collector.
> >
> > Is it possible to change the facility, severity or message for the next actions?
> >
> > Thanks again.
> >
> > On Sun, Apr 7, 2024 at 11:42?PM David Lang <david@lang.hm> wrote:
> >>
> >> you would need to do the filtering with omprog on the sending machines.
> >>
> >> I would suggest that rather than throwing them away, you generate a log every
> >> rate-limiting period along the lines of "X number of messages happened in the
> >> last period"
> >>
> >> This is really a job for an event correlation engine, since rsyslog is
> >> multi-threaded for performance, it can't count on anything internal seeing every
> >> message, which is why filtering for the messages and sending them to an external
> >> app (single threaded or a very complex multi-threaded thing that correlates
> >> across it's threads) is the right thing to do.
> >>
> >> David Lang
> >>
> >> On Sun, 7 Apr 2024, Prasad Koya via rsyslog wrote:
> >>
> >>> Thanks for the replies.
> >>>
> >>> My requirement is that I have a daemon that may generate a burst of
> >>> syslogs, say, every minute (when a certain subsystem is overloaded).
> >>> We do not want to write all these syslogs to /var/log/messages. We
> >>> also do not want to forward so many of those syslogs to a remote
> >>> syslog server. We wanted to rate limit them, maybe 1 per hour or so.
> >>> For now, I have tried 'omprog' module and sending such syslogs to an
> >>> external program.
> >>>
> >>>
> >>> module( load="omprog" )
> >>>
> >>> if ($syslogfacility-text == 'daemon' and $msg contains "Out of memory") then {
> >>> action(type="omprog"
> >>> binary="/usr/bin/rate_limit_daemon_syslogs.sh"
> >>> template="RSYSLOG_FileFormat")
> >>> stop
> >>> }
> >>>
> >>> /usr/bin/rate_limit_daemon_syslogs.sh maintains state on when was the
> >>> last syslog forwarded or written locally. If this script decides its
> >>> ok to syslog, then it will call 'logger' with differently worded
> >>> message and different facility.
> >>>
> >>> In our case we'd like to limit these logs at the source and not at an
> >>> intermediate syslog collector. I took a quick look at the sampling
> >>> docs and that may not apply to our scenario.
> >>>
> >>> Thanks.
> >>>
> >>> On Wed, Apr 3, 2024 at 12:07?AM Mariusz Kruk via rsyslog
> >>> <rsyslog@lists.adiscon.com> wrote:
> >>>>
> >>>> A piece of feedback against
> >>>> https://www.rsyslog.com/doc/tutorials/random_sampling.html
> >>>>
> >>>> I know it's obvious for some people but for some it might not be - it
> >>>> will collect _about_ 20% of logs provided the overall number of events
> >>>> is big enough. Since it's a random-based mechanism, it's... random.
> >>>>
> >>>> Additionally, I had a similar thing implemented for load-balancing to
> >>>> different "backends" and for some reason it didn't work uniformly (I
> >>>> think I already wrote about this) - one of four backends was never
> >>>> chosen. The thing is I was choosing not based directly on random(4) but
> >>>> on (random(some_bigger_integer) % 4) which I thought would yield more
> >>>> uniform randomness. It appears I was wrong.
> >>>>
> >>>> MK
> >>>>
> >>>>
> >>>> On 3.04.2024 08:51, Rainer Gerhards via rsyslog wrote:
> >>>>> This sounds a bit like you are looking for this:
> >>>>>
> >>>>> https://www.rsyslog.com/doc/tutorials/log_sampling.html
> >>>>>
> >>>>> HTH
> >>>>> Rainer
> >>>>>
> >>>>> El mié, 3 abr 2024 a las 3:25, Prasad Koya via rsyslog
> >>>>> (<rsyslog@lists.adiscon.com>) escribió:
> >>>>>> Hi
> >>>>>>
> >>>>>>
> >>>>>> module( load="imuxsock" )
> >>>>>> module( load="imklog" )
> >>>>>>
> >>>>>> if ($syslogfacility-text == 'kern' and $msg contains "Out of memory") then {
> >>>>>> action(type="omfile" queue.size="1000" queue.type="LinkedList"
> >>>>>> queue.dequeueSlowDown="3600000000" file="/var/log/oom")
> >>>>>> }
> >>>>>>
> >>>>>>
> >>>>>> If we have a flurry of, for example, "Out of memory" messages from the
> >>>>>> facility 'kern', we'd like to keep only, say, one per hour in a
> >>>>>> separate file.
> >>>>>>
> >>>>>> https://rsyslog.readthedocs.io/en/latest/rainerscript/queue_parameters.html
> >>>>>> says do not set low value for queue.size.
> >>>>>>
> >>>>>> Above page also says queue.dequeueslowdown can be used as
> >>>>>> rate-limiting. Maybe that doesn't apply for "omfile" as I don't see
> >>>>>> above rsyslog.conf working. high and low watermarks are not applicable
> >>>>>> here as it's not a disk based queue. Appreciate any pointers.
> >>>>>>
> >>>>>> Thank you.
> >>>>>> _______________________________________________
> >>>>>> rsyslog mailing list
> >>>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>>> http://www.rsyslog.com/professional-services/
> >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> >>>>> _______________________________________________
> >>>>> rsyslog mailing list
> >>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>> http://www.rsyslog.com/professional-services/
> >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>> http://www.rsyslog.com/professional-services/
> >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com/professional-services/
> >>> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> >
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

On Mon, 8 Apr 2024, Prasad Koya wrote:

> Yes, we get a lot of flexibility with omprog.
>
> However, changing a syslog's facility/priority isn't possible right?
> Please see below.
>
> if ($syslogfacility-text == 'daemon' and $msg contains "Out of memory") then {
> action(type="omfile"
> action.execOnlyOnceEveryInterval="3600"
> template="RSYSLOG_FileFormat")
>
> # Change the facility and message severity forever here for the
> next action(s). for example to user.alert
> }
>
> # Forward all 'user'.crit and severe to syslog collector.
> user.crit action(type="omfwd" target="10.11.12.13" protocol="udp"
> port="514" template="MsgForwardFormat")

you cannot change the $facility, $severity, and $pri variables, but you can make
your own $.pri variable and create a new template that is identical to
MsgForwardFormat that users $.pri instead of $pri (the outbound message is just
text, so you can make it say anything)

David Lang

> On Mon, Apr 8, 2024 at 6:37?PM David Lang <david@lang.hm> wrote:
>>
>> not easily within rsyslog, with an event correlation engine, you have a lot more
>> capability (which is why I keep pushing you that way :-) )
>>
>> you can create a template to use when outputting a log, and that log can say
>> anything (it can be arbitrary text unrelated to the log you received), but that
>> gets really ugly really quickly
>>
>> you would combine it essentially with a if/else type arrangement to only be done
>> if the prior action didn't take place. I don't know if the onceeveryinterval
>> will work with that.
>>
>> David Lang
>>
>>
>> On Mon, 8 Apr 2024, Prasad Koya wrote:
>>
>>> Thank you. "action.execOnlyOnceEveryInterval" is very useful.
>>>
>>> We intend to keep all unfiltered syslogs in a local file, say file1,
>>> but log a rate limited message to file2 and forward that message to a
>>> syslog collector.
>>>
>>> Is it possible to change the facility, severity or message for the next actions?
>>>
>>> Thanks again.
>>>
>>> On Sun, Apr 7, 2024 at 11:42?PM David Lang <david@lang.hm> wrote:
>>>>
>>>> you would need to do the filtering with omprog on the sending machines.
>>>>
>>>> I would suggest that rather than throwing them away, you generate a log every
>>>> rate-limiting period along the lines of "X number of messages happened in the
>>>> last period"
>>>>
>>>> This is really a job for an event correlation engine, since rsyslog is
>>>> multi-threaded for performance, it can't count on anything internal seeing every
>>>> message, which is why filtering for the messages and sending them to an external
>>>> app (single threaded or a very complex multi-threaded thing that correlates
>>>> across it's threads) is the right thing to do.
>>>>
>>>> David Lang
>>>>
>>>> On Sun, 7 Apr 2024, Prasad Koya via rsyslog wrote:
>>>>
>>>>> Thanks for the replies.
>>>>>
>>>>> My requirement is that I have a daemon that may generate a burst of
>>>>> syslogs, say, every minute (when a certain subsystem is overloaded).
>>>>> We do not want to write all these syslogs to /var/log/messages. We
>>>>> also do not want to forward so many of those syslogs to a remote
>>>>> syslog server. We wanted to rate limit them, maybe 1 per hour or so.
>>>>> For now, I have tried 'omprog' module and sending such syslogs to an
>>>>> external program.
>>>>>
>>>>>
>>>>> module( load="omprog" )
>>>>>
>>>>> if ($syslogfacility-text == 'daemon' and $msg contains "Out of memory") then {
>>>>> action(type="omprog"
>>>>> binary="/usr/bin/rate_limit_daemon_syslogs.sh"
>>>>> template="RSYSLOG_FileFormat")
>>>>> stop
>>>>> }
>>>>>
>>>>> /usr/bin/rate_limit_daemon_syslogs.sh maintains state on when was the
>>>>> last syslog forwarded or written locally. If this script decides its
>>>>> ok to syslog, then it will call 'logger' with differently worded
>>>>> message and different facility.
>>>>>
>>>>> In our case we'd like to limit these logs at the source and not at an
>>>>> intermediate syslog collector. I took a quick look at the sampling
>>>>> docs and that may not apply to our scenario.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> On Wed, Apr 3, 2024 at 12:07?AM Mariusz Kruk via rsyslog
>>>>> <rsyslog@lists.adiscon.com> wrote:
>>>>>>
>>>>>> A piece of feedback against
>>>>>> https://www.rsyslog.com/doc/tutorials/random_sampling.html
>>>>>>
>>>>>> I know it's obvious for some people but for some it might not be - it
>>>>>> will collect _about_ 20% of logs provided the overall number of events
>>>>>> is big enough. Since it's a random-based mechanism, it's... random.
>>>>>>
>>>>>> Additionally, I had a similar thing implemented for load-balancing to
>>>>>> different "backends" and for some reason it didn't work uniformly (I
>>>>>> think I already wrote about this) - one of four backends was never
>>>>>> chosen. The thing is I was choosing not based directly on random(4) but
>>>>>> on (random(some_bigger_integer) % 4) which I thought would yield more
>>>>>> uniform randomness. It appears I was wrong.
>>>>>>
>>>>>> MK
>>>>>>
>>>>>>
>>>>>> On 3.04.2024 08:51, Rainer Gerhards via rsyslog wrote:
>>>>>>> This sounds a bit like you are looking for this:
>>>>>>>
>>>>>>> https://www.rsyslog.com/doc/tutorials/log_sampling.html
>>>>>>>
>>>>>>> HTH
>>>>>>> Rainer
>>>>>>>
>>>>>>> El mié, 3 abr 2024 a las 3:25, Prasad Koya via rsyslog
>>>>>>> (<rsyslog@lists.adiscon.com>) escribió:
>>>>>>>> Hi
>>>>>>>>
>>>>>>>>
>>>>>>>> module( load="imuxsock" )
>>>>>>>> module( load="imklog" )
>>>>>>>>
>>>>>>>> if ($syslogfacility-text == 'kern' and $msg contains "Out of memory") then {
>>>>>>>> action(type="omfile" queue.size="1000" queue.type="LinkedList"
>>>>>>>> queue.dequeueSlowDown="3600000000" file="/var/log/oom")
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> If we have a flurry of, for example, "Out of memory" messages from the
>>>>>>>> facility 'kern', we'd like to keep only, say, one per hour in a
>>>>>>>> separate file.
>>>>>>>>
>>>>>>>> https://rsyslog.readthedocs.io/en/latest/rainerscript/queue_parameters.html
>>>>>>>> says do not set low value for queue.size.
>>>>>>>>
>>>>>>>> Above page also says queue.dequeueslowdown can be used as
>>>>>>>> rate-limiting. Maybe that doesn't apply for "omfile" as I don't see
>>>>>>>> above rsyslog.conf working. high and low watermarks are not applicable
>>>>>>>> here as it's not a disk based queue. Appreciate any pointers.
>>>>>>>>
>>>>>>>> Thank you.
>>>>>>>> _______________________________________________
>>>>>>>> rsyslog mailing list
>>>>>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>>> http://www.rsyslog.com/professional-services/
>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>>>>>>> _______________________________________________
>>>>>>> rsyslog mailing list
>>>>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>> http://www.rsyslog.com/professional-services/
>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com/professional-services/
>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com/professional-services/
>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>>>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.