Mailing List Archive

just an idea: could you filter out certain strings?

Rainer

El lun., 12 oct. 2020 a las 10:00, Unam via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> Hi there,
>
> I am using rsyslog to send all our logs on a remote graylog server. The config on every host (Debian 9 & 10) is quite simple, we send by default everything.
> All our servers are rebooted every night and we receive tons of logs that we don't care about.
>
> My question is maybe very simple but, is there an option in rsyslog to filters this logs and don't send them to the remote ?
>
> I searched in the doc but nothing seems to talk about boot sequence. I already tried to edit grub.cfg to make the boot process silent without success.
>
> Thanks for your hints.
>
> Regards,
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Yep, I can filter directly into graylog but the reboot of 150 servers generate thousand of lines into graylog.

I would like to avoid consuming bandwidth and storage on my graylog server instead of filter on it.

October 12, 2020 10:32 AM, "Rainer Gerhards" <rgerhards@hq.adiscon.com> wrote:

> just an idea: could you filter out certain strings?
>
> Rainer
>
> El lun., 12 oct. 2020 a las 10:00, Unam via rsyslog
> (<rsyslog@lists.adiscon.com>) escribió:
>
>> Hi there,
>>
>> I am using rsyslog to send all our logs on a remote graylog server. The config on every host
>> (Debian 9 & 10) is quite simple, we send by default everything.
>> All our servers are rebooted every night and we receive tons of logs that we don't care about.
>>
>> My question is maybe very simple but, is there an option in rsyslog to filters this logs and don't
>> send them to the remote ?
>>
>> I searched in the doc but nothing seems to talk about boot sequence. I already tried to edit
>> grub.cfg to make the boot process silent without success.
>>
>> Thanks for your hints.
>>
>> Regards,
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
>> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

if you know how to filter at graylog, you can also filter out and
discard those messages at rsyslog ;-)

Rainer

El lun., 12 oct. 2020 a las 10:40, Unam via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> Yep, I can filter directly into graylog but the reboot of 150 servers generate thousand of lines into graylog.
>
> I would like to avoid consuming bandwidth and storage on my graylog server instead of filter on it.
>
> October 12, 2020 10:32 AM, "Rainer Gerhards" <rgerhards@hq.adiscon.com> wrote:
>
> > just an idea: could you filter out certain strings?
> >
> > Rainer
> >
> > El lun., 12 oct. 2020 a las 10:00, Unam via rsyslog
> > (<rsyslog@lists.adiscon.com>) escribió:
> >
> >> Hi there,
> >>
> >> I am using rsyslog to send all our logs on a remote graylog server. The config on every host
> >> (Debian 9 & 10) is quite simple, we send by default everything.
> >> All our servers are rebooted every night and we receive tons of logs that we don't care about.
> >>
> >> My question is maybe very simple but, is there an option in rsyslog to filters this logs and don't
> >> send them to the remote ?
> >>
> >> I searched in the doc but nothing seems to talk about boot sequence. I already tried to edit
> >> grub.cfg to make the boot process silent without success.
> >>
> >> Thanks for your hints.
> >>
> >> Regards,
> >> _______________________________________________
> >> rsyslog mailing list
> >> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com/professional-services
> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
> >> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

> >> All our servers are rebooted every night
> >>

I am bit curious, why/when does one decide to do such a thing?

_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

On 12.10.2020 11:36, Marc Roos via rsyslog wrote:
>>>> All our servers are rebooted every night
>>>>
> I am bit curious, why/when does one decide to do such a thing?
>
I used to have (some 10+ years ago) a Novel eDirectory server which
would run out of contiguous allocatable memory every few days and crash
(only 32-bit version suffered from this; 64-bit version supposedly ran
fine but migration from 32 to 64-bit was quite some task AFAIR). So we
restarted the main server's process every night in controlled manner. I
suppose we could as well just reboot the server. If you have automatic
updates configured, you also have new kernel version installation as a
side effect (but of course if anything goes wrong and the servers go
down and don't come up, you're done for big time since there's high
probability that it happens to all your infrastructure).


Mariusz Kruk
Ekspert ds. Bezpiecze?stwa IT
COMP S.A.
Pion Cyberbezpiecze?stwa i Zarz?dzania Ryzykiem
e-mail: mariusz.kruk@comp.com.pl
e-mail: mariusz.kruk@safecomp.com
tel: +48 608 623 299

_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

I tried a solution, disabling kern.debug with :

```
kern.debug stop
kern.* -/var/log/kern.log
```

This turned the log from 1000 lines to 150 after a reboot.

Just to confirm, is this the correct way ? Does the kern.* still produces logs except for kern.debug ?

Regards,

October 12, 2020 11:05 AM, "Rainer Gerhards" <rgerhards@hq.adiscon.com> wrote:

> if you know how to filter at graylog, you can also filter out and
> discard those messages at rsyslog ;-)
>
> Rainer
>
> El lun., 12 oct. 2020 a las 10:40, Unam via rsyslog
> (<rsyslog@lists.adiscon.com>) escribió:
>
>> Yep, I can filter directly into graylog but the reboot of 150 servers generate thousand of lines
>> into graylog.
>>
>> I would like to avoid consuming bandwidth and storage on my graylog server instead of filter on it.
>>
>> October 12, 2020 10:32 AM, "Rainer Gerhards" <rgerhards@hq.adiscon.com> wrote:
>>
>> just an idea: could you filter out certain strings?
>>
>> Rainer
>>
>> El lun., 12 oct. 2020 a las 10:00, Unam via rsyslog
>> (<rsyslog@lists.adiscon.com>) escribió:
>>
>> Hi there,
>>
>> I am using rsyslog to send all our logs on a remote graylog server. The config on every host
>> (Debian 9 & 10) is quite simple, we send by default everything.
>> All our servers are rebooted every night and we receive tons of logs that we don't care about.
>>
>> My question is maybe very simple but, is there an option in rsyslog to filters this logs and don't
>> send them to the remote ?
>>
>> I searched in the doc but nothing seems to talk about boot sequence. I already tried to edit
>> grub.cfg to make the boot process silent without success.
>>
>> Thanks for your hints.
>>
>> Regards,
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
>> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
>> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

> I tried a solution, disabling kern.debug with :
>
> ```
> kern.debug stop
> kern.* -/var/log/kern.log
> ```
>
> This turned the log from 1000 lines to 150 after a reboot.
>
> Just to confirm, is this the correct way ? Does the kern.* still produces logs except for kern.debug ?

yes

As I said, you can also filter out based on some e.g. string inside
the message. Just so that you know.

Rainer

>
> Regards,
>
> October 12, 2020 11:05 AM, "Rainer Gerhards" <rgerhards@hq.adiscon.com> wrote:
>
> > if you know how to filter at graylog, you can also filter out and
> > discard those messages at rsyslog ;-)
> >
> > Rainer
> >
> > El lun., 12 oct. 2020 a las 10:40, Unam via rsyslog
> > (<rsyslog@lists.adiscon.com>) escribió:
> >
> >> Yep, I can filter directly into graylog but the reboot of 150 servers generate thousand of lines
> >> into graylog.
> >>
> >> I would like to avoid consuming bandwidth and storage on my graylog server instead of filter on it.
> >>
> >> October 12, 2020 10:32 AM, "Rainer Gerhards" <rgerhards@hq.adiscon.com> wrote:
> >>
> >> just an idea: could you filter out certain strings?
> >>
> >> Rainer
> >>
> >> El lun., 12 oct. 2020 a las 10:00, Unam via rsyslog
> >> (<rsyslog@lists.adiscon.com>) escribió:
> >>
> >> Hi there,
> >>
> >> I am using rsyslog to send all our logs on a remote graylog server. The config on every host
> >> (Debian 9 & 10) is quite simple, we send by default everything.
> >> All our servers are rebooted every night and we receive tons of logs that we don't care about.
> >>
> >> My question is maybe very simple but, is there an option in rsyslog to filters this logs and don't
> >> send them to the remote ?
> >>
> >> I searched in the doc but nothing seems to talk about boot sequence. I already tried to edit
> >> grub.cfg to make the boot process silent without success.
> >>
> >> Thanks for your hints.
> >>
> >> Regards,
> >> _______________________________________________
> >> rsyslog mailing list
> >> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com/professional-services
> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
> >> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> >> _______________________________________________
> >> rsyslog mailing list
> >> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com/professional-services
> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
> >> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Yes, I tried by filtering string but there many patterns to look for.

I will stop kern.debug on few servers and will apply it globally if everything ok.

Thanks for all your quick reply.

Regards,

October 13, 2020 9:43 AM, "Rainer Gerhards" <rgerhards@hq.adiscon.com> wrote:

>> I tried a solution, disabling kern.debug with :
>>
>> ```
>> kern.debug stop
>> kern.* -/var/log/kern.log
>> ```
>>
>> This turned the log from 1000 lines to 150 after a reboot.
>>
>> Just to confirm, is this the correct way ? Does the kern.* still produces logs except for
>> kern.debug ?
>
> yes
>
> As I said, you can also filter out based on some e.g. string inside
> the message. Just so that you know.
>
> Rainer
>
>> Regards,
>>
>> October 12, 2020 11:05 AM, "Rainer Gerhards" <rgerhards@hq.adiscon.com> wrote:
>>
>> if you know how to filter at graylog, you can also filter out and
>> discard those messages at rsyslog ;-)
>>
>> Rainer
>>
>> El lun., 12 oct. 2020 a las 10:40, Unam via rsyslog
>> (<rsyslog@lists.adiscon.com>) escribió:
>>
>> Yep, I can filter directly into graylog but the reboot of 150 servers generate thousand of lines
>> into graylog.
>>
>> I would like to avoid consuming bandwidth and storage on my graylog server instead of filter on it.
>>
>> October 12, 2020 10:32 AM, "Rainer Gerhards" <rgerhards@hq.adiscon.com> wrote:
>>
>> just an idea: could you filter out certain strings?
>>
>> Rainer
>>
>> El lun., 12 oct. 2020 a las 10:00, Unam via rsyslog
>> (<rsyslog@lists.adiscon.com>) escribió:
>>
>> Hi there,
>>
>> I am using rsyslog to send all our logs on a remote graylog server. The config on every host
>> (Debian 9 & 10) is quite simple, we send by default everything.
>> All our servers are rebooted every night and we receive tons of logs that we don't care about.
>>
>> My question is maybe very simple but, is there an option in rsyslog to filters this logs and don't
>> send them to the remote ?
>>
>> I searched in the doc but nothing seems to talk about boot sequence. I already tried to edit
>> grub.cfg to make the boot process silent without success.
>>
>> Thanks for your hints.
>>
>> Regards,
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
>> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
>> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.