Mailing List Archive

Suppress duplicate messages
A stateless application starts, runs, and stops every 1 second. If the
application encounters an error, that error is logged. The system is
running rsyslogd 8.1901.0.

When the error occurs, the log is spammed every 1 second until the error is
resolved. We only need to see the error once. The application, being
stateless, does not "remember" that it had already logged the error.

The $RepeatedMsgReduction on option is not a recommended solution, for
various reasons.

Here's a first attempt:

if $msg contains "app error" then set $/app!error!count = $/app!error!count
+ 1;

if $/app!error!count == 10 then set $/app!error!count = 1;

if not $/app!error!count == 1 then ~;

Then:

logger "app error"
logger "app error"
sudo tail -f /var/log/syslog

Shows:

2022-07-06T13:24:42.697247-07:00 host username: app error
2022-07-06T13:24:42.880985-07:00 host username: app error

How would you codify something like the following logic in rsyslog's
scripting language:

IF NOT DEFINED error_count THEN error_count = 0
IF message CONTAINS "{{error message}}" THEN error_count = error_count
+ 1
IF error_count > 10 THEN error_count = 1
IF error_count != 1 THEN stop

Essentially, the first time the error is encountered, log the message;
otherwise, when the error message has occurred 10 times, log the error
again. In effect, this will log the error once every 10 seconds. A more
rigorous solution would also take into consideration the timestamp to reset
the count. Possibly:

IF NOT DEFINED error_count THEN error_count = 0
IF NOT DEFINED error_timestamp THEN error_timestamp = current_timestamp
IF message CONTAINS "{{error message}}" THEN {error_count = error_count
+ 1; error_timestamp = current_timestamp}
IF current_timestamp - error_timestamp > 10 seconds THEN {error_count =
1; error_timestamp = current_timestamp}
IF error_count > 10 THEN error_count = 1
IF error_count != 1 THEN stop

Thank you!
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Suppress duplicate messages [ In reply to ]
rsyslog processes messages one at a time, so there isn't state from prior
messages to reference (in the general case), and messages may be processed in
parallel.

that being said, rsyslog does have 'global variables' that persist from message
to message that could be used for something like this, but be aware that there
is a performance penalty for using them.

I would suggest that you use a tool better designed for the task, Simple Event
Correlator is my go-to tool for anything where I need to do analysis across
multiple messages. To keep SEC from being a bottleneck, I use rsyslog to filter
and direct the logs so that each instance of SEC only gets messages that it is
going to care about, rather than sending it everything and letting it filter it.

David Lang

On Wed, 6 Jul 2022, Thangalin via rsyslog wrote:

> Date: Wed, 6 Jul 2022 14:01:13 -0700
> From: Thangalin via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog@lists.adiscon.com
> Cc: Thangalin <thangalin@gmail.com>
> Subject: [rsyslog] Suppress duplicate messages
>
> A stateless application starts, runs, and stops every 1 second. If the
> application encounters an error, that error is logged. The system is
> running rsyslogd 8.1901.0.
>
> When the error occurs, the log is spammed every 1 second until the error is
> resolved. We only need to see the error once. The application, being
> stateless, does not "remember" that it had already logged the error.
>
> The $RepeatedMsgReduction on option is not a recommended solution, for
> various reasons.
>
> Here's a first attempt:
>
> if $msg contains "app error" then set $/app!error!count = $/app!error!count
> + 1;
>
> if $/app!error!count == 10 then set $/app!error!count = 1;
>
> if not $/app!error!count == 1 then ~;
>
> Then:
>
> logger "app error"
> logger "app error"
> sudo tail -f /var/log/syslog
>
> Shows:
>
> 2022-07-06T13:24:42.697247-07:00 host username: app error
> 2022-07-06T13:24:42.880985-07:00 host username: app error
>
> How would you codify something like the following logic in rsyslog's
> scripting language:
>
> IF NOT DEFINED error_count THEN error_count = 0
> IF message CONTAINS "{{error message}}" THEN error_count = error_count
> + 1
> IF error_count > 10 THEN error_count = 1
> IF error_count != 1 THEN stop
>
> Essentially, the first time the error is encountered, log the message;
> otherwise, when the error message has occurred 10 times, log the error
> again. In effect, this will log the error once every 10 seconds. A more
> rigorous solution would also take into consideration the timestamp to reset
> the count. Possibly:
>
> IF NOT DEFINED error_count THEN error_count = 0
> IF NOT DEFINED error_timestamp THEN error_timestamp = current_timestamp
> IF message CONTAINS "{{error message}}" THEN {error_count = error_count
> + 1; error_timestamp = current_timestamp}
> IF current_timestamp - error_timestamp > 10 seconds THEN {error_count =
> 1; error_timestamp = current_timestamp}
> IF error_count > 10 THEN error_count = 1
> IF error_count != 1 THEN stop
>
> Thank you!
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.