Mailing List Archive

If every input maps to its own unique ruleset then you must do some type of duplication. I would define a separate ruleset with the stop logic (and anything else you’d want to do in all cases) and then call that ruleset from each of the others. I.E. the first action in each ruleset is the call to the new one.

Regards,

> On Dec 8, 2022, at 09:15, Steven D via rsyslog <rsyslog@lists.adiscon.com> wrote:
>
> Rsyslog gurus
>
> I have a config that accepts connections from remote hosts and steers logs to files based on port. Pretty straightforward... what i'm looking to do is "globally" prevent certain ip addresses from ending up in the logs. (Internal vulnerability scanners I have no control over).
>
> I've tried a few different ways but not coming across anything that works globally. Adding something like "if $fromhost-ip '1.2.3.4' then stop" works just fine on an individual ruleset.
>
> Is there a way I can do this without having to enter duplicate lines in every ruleset (I have like 30 rulesets) ?
>
> Thanks,
> Steven
>
> Config snippet below: "#logname01/02#" is replaced by the relevant product in the configuration.
>
> module(load="imudp")
> module(load="imtcp" MaxListeners="100" AddtlFrameDelimiter="000" KeepAlive="on" KeepAlive.Probes="1" KeepAlive.Time="10")
>
> input(type="imudp" port="24514" ruleset="#logname01#_rule")
> input(type="imtcp" port="24514" ruleset="#logname01#_rule")
> template(name="#logname01#_logs" type="string" string="/data/logs/#logname01#/24514/%fromhost-ip%/syslog.log")
> ruleset(name="#logname01#_rule") {
> action(name="#logname01#_rule"
> type="omfile"
> FileCreateMode="0744"
> DirCreateMode="0755"
> FileOwner="SIEM"
> FileGroup="SIEM"
> DirOwner="SIEM"
> DirGroup="SIEM"
> DynaFile="#logname01#_logs"
> DynaFileCacheSize = "50")
> }
>
> input(type="imudp" port="25514" ruleset="#logname02#_rule")
> input(type="imtcp" port="25514" ruleset="#logname02#_rule")
> template(name="#logname02#_logs" type="string" string="/data/logs/#logname02#/25514/%fromhost-ip%/syslog.log")
> ruleset(name="#logname02#_rule") {
> action(name="#logname02#_rule"
> type="omfile"
> FileCreateMode="0744"
> DirCreateMode="0755"
> FileOwner="SIEM"
> FileGroup="SIEM"
> DirOwner="SIEM"
> DirGroup="SIEM"
> DynaFile="#logname02#_logs"
> DynaFileCacheSize = "50")
> }
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

You can have a global ruleset that does the fitlering and then dispatch
the logs to different rulesets.
There's a very power feature to help you here : the "call_indirect"
(https://www.rsyslog.com/doc/master/rainerscript/rainerscript_call_indirect.html)

Here's an example that calls a specific ruleset based on the syslogtag
that I'm sure you can adapt to be based on the port instead :

# ruleset 1
ruleset(name="my_tag1") {
action(
type="omfile"
file="/var/log/my_tag1.log"
)
}

# ruleset 2
ruleset(name="my_tag2") {
action(
type="omfile"
file="/var/log/my_tag2.log"
)
}

# global ruleset
ruleset(name="dispatcher"){
# IP filter
if $fromhost-ip '1.2.3.4' then stop

# if the IP passed the filter, then use a ruleset based on the
syslogtag
if (
$syslogtag == 'my_tag1'
or $syslogtag == 'my_tag2'
) then {
call_indirect $syslogtag;
stop
}
}

# tell imtcp to use the global ruleset
input(type="imtcp" port="514" ruleset="dispatcher")




Le 2022-12-08 16:15, Steven D via rsyslog a écrit :
> Rsyslog gurus
>
> I have a config that accepts connections from remote hosts and steers
> logs
> to files based on port. Pretty straightforward... what i'm looking to
> do
> is "globally" prevent certain ip addresses from ending up in the logs.
> (Internal vulnerability scanners I have no control over).
>
> I've tried a few different ways but not coming across anything that
> works
> globally. Adding something like "if $fromhost-ip '1.2.3.4' then stop"
> works just fine on an individual ruleset.
>
> Is there a way I can do this without having to enter duplicate lines in
> every ruleset (I have like 30 rulesets) ?
>
> Thanks,
> Steven
>
> Config snippet below: "#logname01/02#" is replaced by the relevant
> product
> in the configuration.
>
> module(load="imudp")
> module(load="imtcp" MaxListeners="100" AddtlFrameDelimiter="000"
> KeepAlive="on" KeepAlive.Probes="1" KeepAlive.Time="10")
>
> input(type="imudp" port="24514" ruleset="#logname01#_rule")
> input(type="imtcp" port="24514" ruleset="#logname01#_rule")
> template(name="#logname01#_logs" type="string"
> string="/data/logs/#logname01#/24514/%fromhost-ip%/syslog.log")
> ruleset(name="#logname01#_rule") {
> action(name="#logname01#_rule"
> type="omfile"
> FileCreateMode="0744"
> DirCreateMode="0755"
> FileOwner="SIEM"
> FileGroup="SIEM"
> DirOwner="SIEM"
> DirGroup="SIEM"
> DynaFile="#logname01#_logs"
> DynaFileCacheSize = "50")
> }
>
> input(type="imudp" port="25514" ruleset="#logname02#_rule")
> input(type="imtcp" port="25514" ruleset="#logname02#_rule")
> template(name="#logname02#_logs" type="string"
> string="/data/logs/#logname02#/25514/%fromhost-ip%/syslog.log")
> ruleset(name="#logname02#_rule") {
> action(name="#logname02#_rule"
> type="omfile"
> FileCreateMode="0744"
> DirCreateMode="0755"
> FileOwner="SIEM"
> FileGroup="SIEM"
> DirOwner="SIEM"
> DirGroup="SIEM"
> DynaFile="#logname02#_logs"
> DynaFileCacheSize = "50")
> }
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

There is also the table lookup that can do efficient translation between ranges
of IP addresses and tags that you can then use in something like this.

along with mmnormalize to parse logs into known variables you can do a lot of
classification very efficiently.

David Lang
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

I know that it has already been answered but let me add my three cents ;-)

Firstly, adding condition to $fromhost-ip suggests that you want to
limit based on the source IP, not on the event's content. Which raises
the question - why not simply _not_ send from that host? Or at least
filter it out on the local firewall (most probably iptables).

Secondly, expanding on nope's response - you can have multiple rulesets
chained together so that you have some common "subroutine" and then
branch to specific ruleset depending on how you want to process given
source or data type. You can use lookups or conditions to dynamically
decide to which ruleset you want to route your event to. The
possibilities are endless :-) (and you can end up creating a ruleset
loop and crashing your rsyslogd XD)

Something like.

ruleset(name="ruleset1") {
    set $.destination_ruleset="dest1";
    call intermediate_ruleset
}

ruleset(name="ruleset2") {
    set $.destination_ruleset="dest2";
    call intermediate_ruleset
}

ruleset(name="intermediate_ruleset") {
    set $.this=$that;
// and other stuff
    call_indirect $.destination_ruleset;
}

ruleset(name="dest1") {
  ...
}

ruleset(name="dest2") {
  ...
}

MK

On 8.12.2022 16:15, Steven D via rsyslog wrote:
> Rsyslog gurus
>
> I have a config that accepts connections from remote hosts and steers logs to files based on port. Pretty straightforward... what i'm looking to do is "globally" prevent certain ip addresses from ending up in the logs. (Internal vulnerability scanners I have no control over).
>
> I've tried a few different ways but not coming across anything that works globally. Adding something like "if $fromhost-ip '1.2.3.4' then stop" works just fine on an individual ruleset.
>
> Is there a way I can do this without having to enter duplicate lines in every ruleset (I have like 30 rulesets) ?
>
> Thanks,
> Steven
>
> Config snippet below: "#logname01/02#" is replaced by the relevant product in the configuration.
>
> module(load="imudp")
> module(load="imtcp" MaxListeners="100" AddtlFrameDelimiter="000" KeepAlive="on" KeepAlive.Probes="1" KeepAlive.Time="10")
>
> input(type="imudp" port="24514" ruleset="#logname01#_rule")
> input(type="imtcp" port="24514" ruleset="#logname01#_rule")
> template(name="#logname01#_logs" type="string" string="/data/logs/#logname01#/24514/%fromhost-ip%/syslog.log")
> ruleset(name="#logname01#_rule") {
> action(name="#logname01#_rule"
> type="omfile"
> FileCreateMode="0744"
> DirCreateMode="0755"
> FileOwner="SIEM"
> FileGroup="SIEM"
> DirOwner="SIEM"
> DirGroup="SIEM"
> DynaFile="#logname01#_logs"
> DynaFileCacheSize = "50")
> }
>
> input(type="imudp" port="25514" ruleset="#logname02#_rule")
> input(type="imtcp" port="25514" ruleset="#logname02#_rule")
> template(name="#logname02#_logs" type="string" string="/data/logs/#logname02#/25514/%fromhost-ip%/syslog.log")
> ruleset(name="#logname02#_rule") {
> action(name="#logname02#_rule"
> type="omfile"
> FileCreateMode="0744"
> DirCreateMode="0755"
> FileOwner="SIEM"
> FileGroup="SIEM"
> DirOwner="SIEM"
> DirGroup="SIEM"
> DynaFile="#logname02#_logs"
> DynaFileCacheSize = "50")
> }
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Thanks for all the input, ok so some answers inline

Nope - So this looks straight forward to me, but correct some ignorance on my end. The way you've defined the "syslogtag" rulesets, it makes me assume that the syslog config processed in a top down manner, otherwise logically it doesn't work.

Is that true?

The way we have our syslog config set up is to group all the related template/ruleset/input stanzas together... Just from an ease of modification PoV it makes it easier.

Kurk Mariusz - I don't have control over the sending hosts, they are vulnerability scanners that indiscriminately scan the environment. I can't just iptable drop those same devices from touching the syslog servers as that will cause other operational issues.
________________________________
From: rsyslog <rsyslog-bounces@lists.adiscon.com> on behalf of Mariusz Kruk via rsyslog <rsyslog@lists.adiscon.com>
Sent: Friday, December 9, 2022 5:30 AM
To: rsyslog@lists.adiscon.com <rsyslog@lists.adiscon.com>
Cc: Mariusz Kruk <kruk@epsilon.eu.org>
Subject: Re: [rsyslog] "Global" fromhost-ip blocklist?

I know that it has already been answered but let me add my three cents ;-)

Firstly, adding condition to $fromhost-ip suggests that you want to
limit based on the source IP, not on the event's content. Which raises
the question - why not simply _not_ send from that host? Or at least
filter it out on the local firewall (most probably iptables).

Secondly, expanding on nope's response - you can have multiple rulesets
chained together so that you have some common "subroutine" and then
branch to specific ruleset depending on how you want to process given
source or data type. You can use lookups or conditions to dynamically
decide to which ruleset you want to route your event to. The
possibilities are endless :-) (and you can end up creating a ruleset
loop and crashing your rsyslogd XD)

Something like.

ruleset(name="ruleset1") {
set $.destination_ruleset="dest1";
call intermediate_ruleset
}

ruleset(name="ruleset2") {
set $.destination_ruleset="dest2";
call intermediate_ruleset
}

ruleset(name="intermediate_ruleset") {
set $.this=$that;
// and other stuff
call_indirect $.destination_ruleset;
}

ruleset(name="dest1") {
...
}

ruleset(name="dest2") {
...
}

MK

On 8.12.2022 16:15, Steven D via rsyslog wrote:
> Rsyslog gurus
>
> I have a config that accepts connections from remote hosts and steers logs to files based on port. Pretty straightforward... what i'm looking to do is "globally" prevent certain ip addresses from ending up in the logs. (Internal vulnerability scanners I have no control over).
>
> I've tried a few different ways but not coming across anything that works globally. Adding something like "if $fromhost-ip '1.2.3.4' then stop" works just fine on an individual ruleset.
>
> Is there a way I can do this without having to enter duplicate lines in every ruleset (I have like 30 rulesets) ?
>
> Thanks,
> Steven
>
> Config snippet below: "#logname01/02#" is replaced by the relevant product in the configuration.
>
> module(load="imudp")
> module(load="imtcp" MaxListeners="100" AddtlFrameDelimiter="000" KeepAlive="on" KeepAlive.Probes="1" KeepAlive.Time="10")
>
> input(type="imudp" port="24514" ruleset="#logname01#_rule")
> input(type="imtcp" port="24514" ruleset="#logname01#_rule")
> template(name="#logname01#_logs" type="string" string="/data/logs/#logname01#/24514/%fromhost-ip%/syslog.log")
> ruleset(name="#logname01#_rule") {
> action(name="#logname01#_rule"
> type="omfile"
> FileCreateMode="0744"
> DirCreateMode="0755"
> FileOwner="SIEM"
> FileGroup="SIEM"
> DirOwner="SIEM"
> DirGroup="SIEM"
> DynaFile="#logname01#_logs"
> DynaFileCacheSize = "50")
> }
>
> input(type="imudp" port="25514" ruleset="#logname02#_rule")
> input(type="imtcp" port="25514" ruleset="#logname02#_rule")
> template(name="#logname02#_logs" type="string" string="/data/logs/#logname02#/25514/%fromhost-ip%/syslog.log")
> ruleset(name="#logname02#_rule") {
> action(name="#logname02#_rule"
> type="omfile"
> FileCreateMode="0744"
> DirCreateMode="0755"
> FileOwner="SIEM"
> FileGroup="SIEM"
> DirOwner="SIEM"
> DirGroup="SIEM"
> DynaFile="#logname02#_logs"
> DynaFileCacheSize = "50")
> }
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Depends on what you means by "processed".

It is read from the main config (default or explicitly specified in
command line) and if there is any include directive, the corresponding
file(s) is getting inserted in this spot. So you end up with an
"effective config" which you can see with "-o fullconf" option.

This config is intepreted (we're still talking about reading config, not
processing events!) from the top to bottom so that for most things if
you have some dependency, it must be defined earlier (like you must have
a lookup defined earlier in the config to be able to call that lookup
later).

Then comes the part of event processing - if some operation is not
defined within a specific ruleset it's assumed to be in an implicit
"main" ruleset. And similarily - if an input is not associated with a
specific ruleset, events received by that input are processed by that
implicit main ruleset.


BTW, if you're planning on just dropping the events from the
vulnerability scanners you could just drop events on this one port but I
don't know your precise circumstances. It's just that it seems that
you're trying to overengineer something that could be much simpler ;-)


MK


On 9.12.2022 13:15, Steven D wrote:
> Thanks for all the input, ok so some answers inline
>
> Nope - So this /looks /straight forward to me, but correct some
> ignorance on my end. The way you've defined the "syslogtag" rulesets,
> it makes me assume that the syslog config processed in a top down
> manner, otherwise logically it doesn't work.
>
> Is that true?
>
> The way we have our syslog config set up is to group all the related
> template/ruleset/input stanzas together... Just from an ease of
> modification PoV it makes it easier.
>
> Kurk Mariusz - I don't have control over the sending hosts, they are
> vulnerability scanners that indiscriminately scan the environment. I
> can't just iptable drop those same devices from touching the syslog
> servers as that will cause other operational issues.
> ------------------------------------------------------------------------
> *From:* rsyslog <rsyslog-bounces@lists.adiscon.com> on behalf of
> Mariusz Kruk via rsyslog <rsyslog@lists.adiscon.com>
> *Sent:* Friday, December 9, 2022 5:30 AM
> *To:* rsyslog@lists.adiscon.com <rsyslog@lists.adiscon.com>
> *Cc:* Mariusz Kruk <kruk@epsilon.eu.org>
> *Subject:* Re: [rsyslog] "Global" fromhost-ip blocklist?
> I know that it has already been answered but let me add my three cents ;-)
>
> Firstly, adding condition to $fromhost-ip suggests that you want to
> limit based on the source IP, not on the event's content. Which raises
> the question - why not simply _not_ send from that host? Or at least
> filter it out on the local firewall (most probably iptables).
>
> Secondly, expanding on nope's response - you can have multiple rulesets
> chained together so that you have some common "subroutine" and then
> branch to specific ruleset depending on how you want to process given
> source or data type. You can use lookups or conditions to dynamically
> decide to which ruleset you want to route your event to. The
> possibilities are endless :-) (and you can end up creating a ruleset
> loop and crashing your rsyslogd XD)
>
> Something like.
>
> ruleset(name="ruleset1") {
>      set $.destination_ruleset="dest1";
>      call intermediate_ruleset
> }
>
> ruleset(name="ruleset2") {
>      set $.destination_ruleset="dest2";
>      call intermediate_ruleset
> }
>
> ruleset(name="intermediate_ruleset") {
>      set $.this=$that;
> // and other stuff
>      call_indirect $.destination_ruleset;
> }
>
> ruleset(name="dest1") {
>    ...
> }
>
> ruleset(name="dest2") {
>    ...
> }
>
> MK
>
> On 8.12.2022 16:15, Steven D via rsyslog wrote:
> > Rsyslog gurus
> >
> > I have a config that accepts connections from remote hosts and
> steers logs to files based on port. Pretty straightforward... what i'm
> looking to do is "globally" prevent certain ip addresses from ending
> up in the logs. (Internal vulnerability scanners I have no control over).
> >
> > I've tried a few different ways but not coming across anything that
> works globally. Adding something like "if $fromhost-ip '1.2.3.4' then
> stop" works just fine on an individual ruleset.
> >
> > Is there a way I can do this without having to enter duplicate lines
> in every ruleset (I have like 30 rulesets) ?
> >
> > Thanks,
> > Steven
> >
> > Config snippet below: "#logname01/02#" is replaced by the relevant
> product in the configuration.
> >
> > module(load="imudp")
> > module(load="imtcp" MaxListeners="100" AddtlFrameDelimiter="000"
> KeepAlive="on" KeepAlive.Probes="1" KeepAlive.Time="10")
> >
> > input(type="imudp" port="24514" ruleset="#logname01#_rule")
> > input(type="imtcp" port="24514" ruleset="#logname01#_rule")
> > template(name="#logname01#_logs" type="string"
> string="/data/logs/#logname01#/24514/%fromhost-ip%/syslog.log")
> > ruleset(name="#logname01#_rule") {
> > action(name="#logname01#_rule"
> >        type="omfile"
> >        FileCreateMode="0744"
> >        DirCreateMode="0755"
> >        FileOwner="SIEM"
> >        FileGroup="SIEM"
> >        DirOwner="SIEM"
> >        DirGroup="SIEM"
> >        DynaFile="#logname01#_logs"
> >        DynaFileCacheSize = "50")
> > }
> >
> > input(type="imudp" port="25514" ruleset="#logname02#_rule")
> > input(type="imtcp" port="25514" ruleset="#logname02#_rule")
> > template(name="#logname02#_logs" type="string"
> string="/data/logs/#logname02#/25514/%fromhost-ip%/syslog.log")
> > ruleset(name="#logname02#_rule") {
> > action(name="#logname02#_rule"
> >        type="omfile"
> >        FileCreateMode="0744"
> >        DirCreateMode="0755"
> >        FileOwner="SIEM"
> >        FileGroup="SIEM"
> >        DirOwner="SIEM"
> >        DirGroup="SIEM"
> >        DynaFile="#logname02#_logs"
> >        DynaFileCacheSize = "50")
> > }
> >
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
> if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
> if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

That explanation was very helpful and insightful. Thank you sir.

We may be at a misunderstanding of what I mean about vulnerability scanner events. What is happening is the vulnerability scanners in our environment are scanning all ports across my syslog servers. The scanners connection attempts or vulnerability tests are being written to disc as events. Which in turn end up in my logging system and are just noise I don't need.

So yeah you're right it is a bit of an overengineering issue, but I don't really have a choice. Either I find a way to drop traffic from specific sources to all of my open ports for our syslog only. Or I just use IP tables and block the scanners altogether, as you suggested earlier.... Unfortunately that means I'd have to do manual vulnerability checks on my servers, and I really don't want to have to deal with that, hahaha



Regards,

Steven.



-------- Original message --------
From: Mariusz Kruk <kruk@epsilon.eu.org>
Date: 12/9/22 8:19 AM (GMT-05:00)
To: Steven D <pheerless@hotmail.com>, rsyslog@lists.adiscon.com
Subject: Re: [rsyslog] "Global" fromhost-ip blocklist?


Depends on what you means by "processed".

It is read from the main config (default or explicitly specified in command line) and if there is any include directive, the corresponding file(s) is getting inserted in this spot. So you end up with an "effective config" which you can see with "-o fullconf" option.

This config is intepreted (we're still talking about reading config, not processing events!) from the top to bottom so that for most things if you have some dependency, it must be defined earlier (like you must have a lookup defined earlier in the config to be able to call that lookup later).

Then comes the part of event processing - if some operation is not defined within a specific ruleset it's assumed to be in an implicit "main" ruleset. And similarily - if an input is not associated with a specific ruleset, events received by that input are processed by that implicit main ruleset.


BTW, if you're planning on just dropping the events from the vulnerability scanners you could just drop events on this one port but I don't know your precise circumstances. It's just that it seems that you're trying to overengineer something that could be much simpler ;-)


MK


On 9.12.2022 13:15, Steven D wrote:
Thanks for all the input, ok so some answers inline

Nope - So this looks straight forward to me, but correct some ignorance on my end. The way you've defined the "syslogtag" rulesets, it makes me assume that the syslog config processed in a top down manner, otherwise logically it doesn't work.

Is that true?

The way we have our syslog config set up is to group all the related template/ruleset/input stanzas together... Just from an ease of modification PoV it makes it easier.

Kurk Mariusz - I don't have control over the sending hosts, they are vulnerability scanners that indiscriminately scan the environment. I can't just iptable drop those same devices from touching the syslog servers as that will cause other operational issues.
________________________________
From: rsyslog <rsyslog-bounces@lists.adiscon.com><mailto:rsyslog-bounces@lists.adiscon.com> on behalf of Mariusz Kruk via rsyslog <rsyslog@lists.adiscon.com><mailto:rsyslog@lists.adiscon.com>
Sent: Friday, December 9, 2022 5:30 AM
To: rsyslog@lists.adiscon.com<mailto:rsyslog@lists.adiscon.com> <rsyslog@lists.adiscon.com><mailto:rsyslog@lists.adiscon.com>
Cc: Mariusz Kruk <kruk@epsilon.eu.org><mailto:kruk@epsilon.eu.org>
Subject: Re: [rsyslog] "Global" fromhost-ip blocklist?

I know that it has already been answered but let me add my three cents ;-)

Firstly, adding condition to $fromhost-ip suggests that you want to
limit based on the source IP, not on the event's content. Which raises
the question - why not simply _not_ send from that host? Or at least
filter it out on the local firewall (most probably iptables).

Secondly, expanding on nope's response - you can have multiple rulesets
chained together so that you have some common "subroutine" and then
branch to specific ruleset depending on how you want to process given
source or data type. You can use lookups or conditions to dynamically
decide to which ruleset you want to route your event to. The
possibilities are endless :-) (and you can end up creating a ruleset
loop and crashing your rsyslogd XD)

Something like.

ruleset(name="ruleset1") {
set $.destination_ruleset="dest1";
call intermediate_ruleset
}

ruleset(name="ruleset2") {
set $.destination_ruleset="dest2";
call intermediate_ruleset
}

ruleset(name="intermediate_ruleset") {
set $.this=$that;
// and other stuff
call_indirect $.destination_ruleset;
}

ruleset(name="dest1") {
...
}

ruleset(name="dest2") {
...
}

MK

On 8.12.2022 16:15, Steven D via rsyslog wrote:
> Rsyslog gurus
>
> I have a config that accepts connections from remote hosts and steers logs to files based on port. Pretty straightforward... what i'm looking to do is "globally" prevent certain ip addresses from ending up in the logs. (Internal vulnerability scanners I have no control over).
>
> I've tried a few different ways but not coming across anything that works globally. Adding something like "if $fromhost-ip '1.2.3.4' then stop" works just fine on an individual ruleset.
>
> Is there a way I can do this without having to enter duplicate lines in every ruleset (I have like 30 rulesets) ?
>
> Thanks,
> Steven
>
> Config snippet below: "#logname01/02#" is replaced by the relevant product in the configuration.
>
> module(load="imudp")
> module(load="imtcp" MaxListeners="100" AddtlFrameDelimiter="000" KeepAlive="on" KeepAlive.Probes="1" KeepAlive.Time="10")
>
> input(type="imudp" port="24514" ruleset="#logname01#_rule")
> input(type="imtcp" port="24514" ruleset="#logname01#_rule")
> template(name="#logname01#_logs" type="string" string="/data/logs/#logname01#/24514/%fromhost-ip%/syslog.log")
> ruleset(name="#logname01#_rule") {
> action(name="#logname01#_rule"
> type="omfile"
> FileCreateMode="0744"
> DirCreateMode="0755"
> FileOwner="SIEM"
> FileGroup="SIEM"
> DirOwner="SIEM"
> DirGroup="SIEM"
> DynaFile="#logname01#_logs"
> DynaFileCacheSize = "50")
> }
>
> input(type="imudp" port="25514" ruleset="#logname02#_rule")
> input(type="imtcp" port="25514" ruleset="#logname02#_rule")
> template(name="#logname02#_logs" type="string" string="/data/logs/#logname02#/25514/%fromhost-ip%/syslog.log")
> ruleset(name="#logname02#_rule") {
> action(name="#logname02#_rule"
> type="omfile"
> FileCreateMode="0744"
> DirCreateMode="0755"
> FileOwner="SIEM"
> FileGroup="SIEM"
> DirOwner="SIEM"
> DirGroup="SIEM"
> DynaFile="#logname02#_logs"
> DynaFileCacheSize = "50")
> }
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

> Nope - So this looks straight forward to me, but correct some ignorance
> on
> my end. The way you've defined the "syslogtag" rulesets, it makes me
> assume that the syslog config processed in a top down manner, otherwise
> logically it doesn't work.
>
> Is that true?
>
> The way we have our syslog config set up is to group all the related
> template/ruleset/input stanzas together... Just from an ease of
> modification PoV it makes it easier.

The rulesets and templates can be defined in any order.
Each ruleset and template can be in a separate file and loaded in any
order. Eg.:

# /etc/rsyslog.d/00_input.conf
input(type="imtcp" port="514" ruleset="dispatcher")

# /etc/rsyslog.d/00_dispatcher.conf
ruleset(name="dispatcher"){
# use a ruleset based on the syslogtag
if ( $syslogtag == 'my_tag1' or $syslogtag == 'my_tag2' ) then {
call_indirect $syslogtag;
stop
}
}

# /etc/rsyslog.d/10_my_tag1.conf
ruleset(name="my_tag1") {
action( type="omfile" file="/var/log/my_tag1.log" )
}

# /etc/rsyslog.d/20_my_tag2.conf
ruleset(name="my_tag2") {
action( type="omfile" file="/var/log/my_tag2.log")
}






Le 2022-12-09 13:15, Steven D via rsyslog a écrit :
> Thanks for all the input, ok so some answers inline
>
> Nope - So this looks straight forward to me, but correct some ignorance
> on
> my end. The way you've defined the "syslogtag" rulesets, it makes me
> assume that the syslog config processed in a top down manner, otherwise
> logically it doesn't work.
>
> Is that true?
>
> The way we have our syslog config set up is to group all the related
> template/ruleset/input stanzas together... Just from an ease of
> modification PoV it makes it easier.
>
> Kurk Mariusz - I don't have control over the sending hosts, they are
> vulnerability scanners that indiscriminately scan the environment. I
> can't
> just iptable drop those same devices from touching the syslog servers
> as
> that will cause other operational issues.
> ________________________________
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> on behalf of Mariusz
> Kruk via rsyslog <rsyslog@lists.adiscon.com>
> Sent: Friday, December 9, 2022 5:30 AM
> To: rsyslog@lists.adiscon.com <rsyslog@lists.adiscon.com>
> Cc: Mariusz Kruk <kruk@epsilon.eu.org>
> Subject: Re: [rsyslog] "Global" fromhost-ip blocklist?
>
> I know that it has already been answered but let me add my three cents
> ;-)
>
> Firstly, adding condition to $fromhost-ip suggests that you want to
> limit based on the source IP, not on the event's content. Which raises
> the question - why not simply _not_ send from that host? Or at least
> filter it out on the local firewall (most probably iptables).
>
> Secondly, expanding on nope's response - you can have multiple rulesets
> chained together so that you have some common "subroutine" and then
> branch to specific ruleset depending on how you want to process given
> source or data type. You can use lookups or conditions to dynamically
> decide to which ruleset you want to route your event to. The
> possibilities are endless :-) (and you can end up creating a ruleset
> loop and crashing your rsyslogd XD)
>
> Something like.
>
> ruleset(name="ruleset1") {
> set $.destination_ruleset="dest1";
> call intermediate_ruleset
> }
>
> ruleset(name="ruleset2") {
> set $.destination_ruleset="dest2";
> call intermediate_ruleset
> }
>
> ruleset(name="intermediate_ruleset") {
> set $.this=$that;
> // and other stuff
> call_indirect $.destination_ruleset;
> }
>
> ruleset(name="dest1") {
> ...
> }
>
> ruleset(name="dest2") {
> ...
> }
>
> MK
>

_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

I've run into the same thing, and I either accept the noise (and filter it out
during log processing/repoting), or do something similar to what you are doing
and drop it in a ruleset.

I don't want to block those sources with iptables because I want the
vulnerability scanners to be able to see what ports are open and poke at them
rather than getting a false sense of what ports are open, which is what would
happen if they were blocked by iptables

While these events are dropped from the log on the receiving side (either in
processing or in logging), they are not dropped as far as the vulnerability
scanners go, so the scanners see what ports are open, what versions are there
(and in the case of ssh, it tests common username/password combinations, and
while you would think that test would never find anything, accidents happen and
I've seen them find things)

David Lang


On Fri, 9 Dec 2022, Steven D via rsyslog wrote:

> Date: Fri, 9 Dec 2022 13:30:55 +0000
> From: Steven D via rsyslog <rsyslog@lists.adiscon.com>
> To: Mariusz Kruk <kruk@epsilon.eu.org>,
> "rsyslog@lists.adiscon.com" <rsyslog@lists.adiscon.com>
> Cc: Steven D <pheerless@hotmail.com>
> Subject: Re: [rsyslog] "Global" fromhost-ip blocklist?
>
> That explanation was very helpful and insightful. Thank you sir.
>
> We may be at a misunderstanding of what I mean about vulnerability scanner events. What is happening is the vulnerability scanners in our environment are scanning all ports across my syslog servers. The scanners connection attempts or vulnerability tests are being written to disc as events. Which in turn end up in my logging system and are just noise I don't need.
>
> So yeah you're right it is a bit of an overengineering issue, but I don't really have a choice. Either I find a way to drop traffic from specific sources to all of my open ports for our syslog only. Or I just use IP tables and block the scanners altogether, as you suggested earlier.... Unfortunately that means I'd have to do manual vulnerability checks on my servers, and I really don't want to have to deal with that, hahaha
>
>
>
> Regards,
>
> Steven.
>
>
>
> -------- Original message --------
> From: Mariusz Kruk <kruk@epsilon.eu.org>
> Date: 12/9/22 8:19 AM (GMT-05:00)
> To: Steven D <pheerless@hotmail.com>, rsyslog@lists.adiscon.com
> Subject: Re: [rsyslog] "Global" fromhost-ip blocklist?
>
>
> Depends on what you means by "processed".
>
> It is read from the main config (default or explicitly specified in command line) and if there is any include directive, the corresponding file(s) is getting inserted in this spot. So you end up with an "effective config" which you can see with "-o fullconf" option.
>
> This config is intepreted (we're still talking about reading config, not processing events!) from the top to bottom so that for most things if you have some dependency, it must be defined earlier (like you must have a lookup defined earlier in the config to be able to call that lookup later).
>
> Then comes the part of event processing - if some operation is not defined within a specific ruleset it's assumed to be in an implicit "main" ruleset. And similarily - if an input is not associated with a specific ruleset, events received by that input are processed by that implicit main ruleset.
>
>
> BTW, if you're planning on just dropping the events from the vulnerability scanners you could just drop events on this one port but I don't know your precise circumstances. It's just that it seems that you're trying to overengineer something that could be much simpler ;-)
>
>
> MK
>
>
> On 9.12.2022 13:15, Steven D wrote:
> Thanks for all the input, ok so some answers inline
>
> Nope - So this looks straight forward to me, but correct some ignorance on my end. The way you've defined the "syslogtag" rulesets, it makes me assume that the syslog config processed in a top down manner, otherwise logically it doesn't work.
>
> Is that true?
>
> The way we have our syslog config set up is to group all the related template/ruleset/input stanzas together... Just from an ease of modification PoV it makes it easier.
>
> Kurk Mariusz - I don't have control over the sending hosts, they are vulnerability scanners that indiscriminately scan the environment. I can't just iptable drop those same devices from touching the syslog servers as that will cause other operational issues.
> ________________________________
> From: rsyslog <rsyslog-bounces@lists.adiscon.com><mailto:rsyslog-bounces@lists.adiscon.com> on behalf of Mariusz Kruk via rsyslog <rsyslog@lists.adiscon.com><mailto:rsyslog@lists.adiscon.com>
> Sent: Friday, December 9, 2022 5:30 AM
> To: rsyslog@lists.adiscon.com<mailto:rsyslog@lists.adiscon.com> <rsyslog@lists.adiscon.com><mailto:rsyslog@lists.adiscon.com>
> Cc: Mariusz Kruk <kruk@epsilon.eu.org><mailto:kruk@epsilon.eu.org>
> Subject: Re: [rsyslog] "Global" fromhost-ip blocklist?
>
> I know that it has already been answered but let me add my three cents ;-)
>
> Firstly, adding condition to $fromhost-ip suggests that you want to
> limit based on the source IP, not on the event's content. Which raises
> the question - why not simply _not_ send from that host? Or at least
> filter it out on the local firewall (most probably iptables).
>
> Secondly, expanding on nope's response - you can have multiple rulesets
> chained together so that you have some common "subroutine" and then
> branch to specific ruleset depending on how you want to process given
> source or data type. You can use lookups or conditions to dynamically
> decide to which ruleset you want to route your event to. The
> possibilities are endless :-) (and you can end up creating a ruleset
> loop and crashing your rsyslogd XD)
>
> Something like.
>
> ruleset(name="ruleset1") {
> set $.destination_ruleset="dest1";
> call intermediate_ruleset
> }
>
> ruleset(name="ruleset2") {
> set $.destination_ruleset="dest2";
> call intermediate_ruleset
> }
>
> ruleset(name="intermediate_ruleset") {
> set $.this=$that;
> // and other stuff
> call_indirect $.destination_ruleset;
> }
>
> ruleset(name="dest1") {
> ...
> }
>
> ruleset(name="dest2") {
> ...
> }
>
> MK
>
> On 8.12.2022 16:15, Steven D via rsyslog wrote:
>> Rsyslog gurus
>>
>> I have a config that accepts connections from remote hosts and steers logs to files based on port. Pretty straightforward... what i'm looking to do is "globally" prevent certain ip addresses from ending up in the logs. (Internal vulnerability scanners I have no control over).
>>
>> I've tried a few different ways but not coming across anything that works globally. Adding something like "if $fromhost-ip '1.2.3.4' then stop" works just fine on an individual ruleset.
>>
>> Is there a way I can do this without having to enter duplicate lines in every ruleset (I have like 30 rulesets) ?
>>
>> Thanks,
>> Steven
>>
>> Config snippet below: "#logname01/02#" is replaced by the relevant product in the configuration.
>>
>> module(load="imudp")
>> module(load="imtcp" MaxListeners="100" AddtlFrameDelimiter="000" KeepAlive="on" KeepAlive.Probes="1" KeepAlive.Time="10")
>>
>> input(type="imudp" port="24514" ruleset="#logname01#_rule")
>> input(type="imtcp" port="24514" ruleset="#logname01#_rule")
>> template(name="#logname01#_logs" type="string" string="/data/logs/#logname01#/24514/%fromhost-ip%/syslog.log")
>> ruleset(name="#logname01#_rule") {
>> action(name="#logname01#_rule"
>> type="omfile"
>> FileCreateMode="0744"
>> DirCreateMode="0755"
>> FileOwner="SIEM"
>> FileGroup="SIEM"
>> DirOwner="SIEM"
>> DirGroup="SIEM"
>> DynaFile="#logname01#_logs"
>> DynaFileCacheSize = "50")
>> }
>>
>> input(type="imudp" port="25514" ruleset="#logname02#_rule")
>> input(type="imtcp" port="25514" ruleset="#logname02#_rule")
>> template(name="#logname02#_logs" type="string" string="/data/logs/#logname02#/25514/%fromhost-ip%/syslog.log")
>> ruleset(name="#logname02#_rule") {
>> action(name="#logname02#_rule"
>> type="omfile"
>> FileCreateMode="0744"
>> DirCreateMode="0755"
>> FileOwner="SIEM"
>> FileGroup="SIEM"
>> DirOwner="SIEM"
>> DirGroup="SIEM"
>> DynaFile="#logname02#_logs"
>> DynaFileCacheSize = "50")
>> }
>>
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Well, depends on how you're sending the events. If your L4 protocol is
UDP, then there's no difference between accepting and processing the
datagram and dropping it silently. (If you close the port and send the
ICMP unreachable, that's different).

But yes, I also tend to drop events about vulnerability scanners but in
a different case - I filter out their network sessions from the firewall
logs (but this involves some parsing and matching events from hosts
A,B,C and so on to check whether they are _about_ hosts D, E and F).

MK

On 9.12.2022 15:27, David Lang via rsyslog wrote:
> I've run into the same thing, and I either accept the noise (and
> filter it out during log processing/repoting), or do something similar
> to what you are doing and drop it in a ruleset.
>
> I don't want to block those sources with iptables because I want the
> vulnerability scanners to be able to see what ports are open and poke
> at them rather than getting a false sense of what ports are open,
> which is what would happen if they were blocked by iptables
>
> While these events are dropped from the log on the receiving side
> (either in processing or in logging), they are not dropped as far as
> the vulnerability scanners go, so the scanners see what ports are
> open, what versions are there (and in the case of ssh, it tests common
> username/password combinations, and while you would think that test
> would never find anything, accidents happen and I've seen them find
> things)
>
> David Lang
>
>
>  On Fri, 9 Dec 2022, Steven D via rsyslog wrote:
>
>> Date: Fri, 9 Dec 2022 13:30:55 +0000
>> From: Steven D via rsyslog <rsyslog@lists.adiscon.com>
>> To: Mariusz Kruk <kruk@epsilon.eu.org>,
>>     "rsyslog@lists.adiscon.com" <rsyslog@lists.adiscon.com>
>> Cc: Steven D <pheerless@hotmail.com>
>> Subject: Re: [rsyslog] "Global" fromhost-ip blocklist?
>>
>> That explanation was very helpful and insightful. Thank you sir.
>>
>> We may be at a misunderstanding of what I mean about vulnerability
>> scanner events. What is happening is the vulnerability scanners in
>> our environment are scanning all ports across my syslog servers. The
>> scanners connection attempts or vulnerability tests are being written
>> to disc as events. Which in turn end up in my logging system and are
>> just noise I don't need.
>>
>> So yeah you're right it is a bit of an overengineering issue, but I
>> don't really have a choice. Either I find a way to drop traffic from
>> specific sources to all of my open ports for our syslog only. Or I
>> just use IP tables and block the scanners altogether, as you
>> suggested earlier.... Unfortunately that means I'd have to do manual
>> vulnerability checks on my servers, and I really don't want to have
>> to deal with that, hahaha
>>
>>
>>
>> Regards,
>>
>> Steven.
>>
>>
>>
>> -------- Original message --------
>> From: Mariusz Kruk <kruk@epsilon.eu.org>
>> Date: 12/9/22 8:19 AM (GMT-05:00)
>> To: Steven D <pheerless@hotmail.com>, rsyslog@lists.adiscon.com
>> Subject: Re: [rsyslog] "Global" fromhost-ip blocklist?
>>
>>
>> Depends on what you means by "processed".
>>
>> It is read from the main config (default or explicitly specified in
>> command line) and if there is any include directive, the
>> corresponding file(s) is getting inserted in this spot. So you end up
>> with an "effective config" which you can see with "-o fullconf" option.
>>
>> This config is intepreted (we're still talking about reading config,
>> not processing events!) from the top to bottom so that for most
>> things if you have some dependency, it must be defined earlier (like
>> you must have a lookup defined earlier in the config to be able to
>> call that lookup later).
>>
>> Then comes the part of event processing - if some operation is not
>> defined within a specific ruleset it's assumed to be in an implicit
>> "main" ruleset. And similarily - if an input is not associated with a
>> specific ruleset, events received by that input are processed by that
>> implicit main ruleset.
>>
>>
>> BTW, if you're planning on just dropping the events from the
>> vulnerability scanners you could just drop events on this one port
>> but I don't know your precise circumstances. It's just that it seems
>> that you're trying to overengineer something that could be much
>> simpler ;-)
>>
>>
>> MK
>>
>>
>> On 9.12.2022 13:15, Steven D wrote:
>> Thanks for all the input, ok so some answers inline
>>
>> Nope - So this looks straight forward to me, but correct some
>> ignorance on my end. The way you've defined the "syslogtag" rulesets,
>> it makes me assume that the syslog config processed in a top down
>> manner, otherwise logically it doesn't work.
>>
>> Is that true?
>>
>> The way we have our syslog config set up is to group all the related
>> template/ruleset/input stanzas together... Just from an ease of
>> modification PoV it makes it easier.
>>
>> Kurk Mariusz - I don't have control over the sending hosts, they are
>> vulnerability scanners that indiscriminately scan the environment. I
>> can't just iptable drop those same devices from touching the syslog
>> servers as that will cause other operational issues.
>> ________________________________
>> From: rsyslog
>> <rsyslog-bounces@lists.adiscon.com><mailto:rsyslog-bounces@lists.adiscon.com>
>> on behalf of Mariusz Kruk via rsyslog
>> <rsyslog@lists.adiscon.com><mailto:rsyslog@lists.adiscon.com>
>> Sent: Friday, December 9, 2022 5:30 AM
>> To: rsyslog@lists.adiscon.com<mailto:rsyslog@lists.adiscon.com>
>> <rsyslog@lists.adiscon.com><mailto:rsyslog@lists.adiscon.com>
>> Cc: Mariusz Kruk <kruk@epsilon.eu.org><mailto:kruk@epsilon.eu.org>
>> Subject: Re: [rsyslog] "Global" fromhost-ip blocklist?
>>
>> I know that it has already been answered but let me add my three
>> cents ;-)
>>
>> Firstly, adding condition to $fromhost-ip suggests that you want to
>> limit based on the source IP, not on the event's content. Which raises
>> the question - why not simply _not_ send from that host? Or at least
>> filter it out on the local firewall (most probably iptables).
>>
>> Secondly, expanding on nope's response - you can have multiple rulesets
>> chained together so that you have some common "subroutine" and then
>> branch to specific ruleset depending on how you want to process given
>> source or data type. You can use lookups or conditions to dynamically
>> decide to which ruleset you want to route your event to. The
>> possibilities are endless :-) (and you can end up creating a ruleset
>> loop and crashing your rsyslogd XD)
>>
>> Something like.
>>
>> ruleset(name="ruleset1") {
>>     set $.destination_ruleset="dest1";
>>     call intermediate_ruleset
>> }
>>
>> ruleset(name="ruleset2") {
>>     set $.destination_ruleset="dest2";
>>     call intermediate_ruleset
>> }
>>
>> ruleset(name="intermediate_ruleset") {
>>     set $.this=$that;
>> // and other stuff
>>     call_indirect $.destination_ruleset;
>> }
>>
>> ruleset(name="dest1") {
>>   ...
>> }
>>
>> ruleset(name="dest2") {
>>   ...
>> }
>>
>> MK
>>
>> On 8.12.2022 16:15, Steven D via rsyslog wrote:
>>> Rsyslog gurus
>>>
>>> I have a config that accepts connections from remote hosts and
>>> steers logs to files based on port. Pretty straightforward... what
>>> i'm looking to do is "globally" prevent certain ip addresses from
>>> ending up in the logs. (Internal vulnerability scanners I have no
>>> control over).
>>>
>>> I've tried a few different ways but not coming across anything that
>>> works globally. Adding something like "if $fromhost-ip '1.2.3.4'
>>> then stop" works just fine on an individual ruleset.
>>>
>>> Is there a way I can do this without having to enter duplicate lines
>>> in every ruleset (I have like 30 rulesets) ?
>>>
>>> Thanks,
>>> Steven
>>>
>>> Config snippet below: "#logname01/02#" is replaced by the relevant
>>> product in the configuration.
>>>
>>> module(load="imudp")
>>> module(load="imtcp" MaxListeners="100" AddtlFrameDelimiter="000"
>>> KeepAlive="on" KeepAlive.Probes="1" KeepAlive.Time="10")
>>>
>>> input(type="imudp" port="24514" ruleset="#logname01#_rule")
>>> input(type="imtcp" port="24514" ruleset="#logname01#_rule")
>>> template(name="#logname01#_logs" type="string"
>>> string="/data/logs/#logname01#/24514/%fromhost-ip%/syslog.log")
>>> ruleset(name="#logname01#_rule") {
>>> action(name="#logname01#_rule"
>>>        type="omfile"
>>>        FileCreateMode="0744"
>>>        DirCreateMode="0755"
>>>        FileOwner="SIEM"
>>>        FileGroup="SIEM"
>>>        DirOwner="SIEM"
>>>        DirGroup="SIEM"
>>>        DynaFile="#logname01#_logs"
>>>        DynaFileCacheSize = "50")
>>> }
>>>
>>> input(type="imudp" port="25514" ruleset="#logname02#_rule")
>>> input(type="imtcp" port="25514" ruleset="#logname02#_rule")
>>> template(name="#logname02#_logs" type="string"
>>> string="/data/logs/#logname02#/25514/%fromhost-ip%/syslog.log")
>>> ruleset(name="#logname02#_rule") {
>>> action(name="#logname02#_rule"
>>>        type="omfile"
>>>        FileCreateMode="0744"
>>>        DirCreateMode="0755"
>>>        FileOwner="SIEM"
>>>        FileGroup="SIEM"
>>>        DirOwner="SIEM"
>>>        DirGroup="SIEM"
>>>        DynaFile="#logname02#_logs"
>>>        DynaFileCacheSize = "50")
>>> }
>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
>>> POST if you DON'T LIKE THAT.
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
>> POST if you DON'T LIKE THAT.
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
>> POST if you DON'T LIKE THAT.
>>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
> if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Just wanted to close loop on this and thank everyone for the input/suggestions.


I ended up implementing it similar to below, incase anyone else needs a starting point:

---------------------------------------------
module(load="imudp")
module(load="imtcp")

## Blocklist to remove vuln scanning/other noise
ruleset(name="common_rule") {
#drop known scanners
if $fromhost-ip == '1.2.3.4' then stop;
else if $fromhost-ip == '1.2.3.5' then stop;
else call_indirect $.dest_ruleset;

}
## Blocklist

input(type="imudp" port="10514" ruleset="firewall_rule")
input(type="imtcp" port="10514" ruleset="firewall_rule")
template(name="firewall_logs" type="string" string="/data/logs/firewall/10514/%fromhost-ip%/syslog.log")
ruleset(name="firewall_rule") {
set $.dest_ruleset="firewall_rule_do";
call common_rule
}
ruleset(name="firewall_rule_do") {
action(name="firewall_rule_do"
type="omfile"
FileCreateMode="0744"
DirCreateMode="0755"
FileOwner="user"
FileGroup="user"
DirOwner="user"
DirGroup="user"
DynaFile="firewall_logs"
DynaFileCacheSize = "50")
}
--------------------------------------------------------------
________________________________
From: rsyslog <rsyslog-bounces@lists.adiscon.com> on behalf of Mariusz Kruk via rsyslog <rsyslog@lists.adiscon.com>
Sent: Friday, December 9, 2022 9:40 AM
To: rsyslog@lists.adiscon.com <rsyslog@lists.adiscon.com>
Cc: Mariusz Kruk <kruk@epsilon.eu.org>
Subject: Re: [rsyslog] "Global" fromhost-ip blocklist?

Well, depends on how you're sending the events. If your L4 protocol is
UDP, then there's no difference between accepting and processing the
datagram and dropping it silently. (If you close the port and send the
ICMP unreachable, that's different).

But yes, I also tend to drop events about vulnerability scanners but in
a different case - I filter out their network sessions from the firewall
logs (but this involves some parsing and matching events from hosts
A,B,C and so on to check whether they are _about_ hosts D, E and F).

MK

On 9.12.2022 15:27, David Lang via rsyslog wrote:
> I've run into the same thing, and I either accept the noise (and
> filter it out during log processing/repoting), or do something similar
> to what you are doing and drop it in a ruleset.
>
> I don't want to block those sources with iptables because I want the
> vulnerability scanners to be able to see what ports are open and poke
> at them rather than getting a false sense of what ports are open,
> which is what would happen if they were blocked by iptables
>
> While these events are dropped from the log on the receiving side
> (either in processing or in logging), they are not dropped as far as
> the vulnerability scanners go, so the scanners see what ports are
> open, what versions are there (and in the case of ssh, it tests common
> username/password combinations, and while you would think that test
> would never find anything, accidents happen and I've seen them find
> things)
>
> David Lang
>
>
> On Fri, 9 Dec 2022, Steven D via rsyslog wrote:
>
>> Date: Fri, 9 Dec 2022 13:30:55 +0000
>> From: Steven D via rsyslog <rsyslog@lists.adiscon.com>
>> To: Mariusz Kruk <kruk@epsilon.eu.org>,
>> "rsyslog@lists.adiscon.com" <rsyslog@lists.adiscon.com>
>> Cc: Steven D <pheerless@hotmail.com>
>> Subject: Re: [rsyslog] "Global" fromhost-ip blocklist?
>>
>> That explanation was very helpful and insightful. Thank you sir.
>>
>> We may be at a misunderstanding of what I mean about vulnerability
>> scanner events. What is happening is the vulnerability scanners in
>> our environment are scanning all ports across my syslog servers. The
>> scanners connection attempts or vulnerability tests are being written
>> to disc as events. Which in turn end up in my logging system and are
>> just noise I don't need.
>>
>> So yeah you're right it is a bit of an overengineering issue, but I
>> don't really have a choice. Either I find a way to drop traffic from
>> specific sources to all of my open ports for our syslog only. Or I
>> just use IP tables and block the scanners altogether, as you
>> suggested earlier.... Unfortunately that means I'd have to do manual
>> vulnerability checks on my servers, and I really don't want to have
>> to deal with that, hahaha
>>
>>
>>
>> Regards,
>>
>> Steven.
>>
>>
>>
>> -------- Original message --------
>> From: Mariusz Kruk <kruk@epsilon.eu.org>
>> Date: 12/9/22 8:19 AM (GMT-05:00)
>> To: Steven D <pheerless@hotmail.com>, rsyslog@lists.adiscon.com
>> Subject: Re: [rsyslog] "Global" fromhost-ip blocklist?
>>
>>
>> Depends on what you means by "processed".
>>
>> It is read from the main config (default or explicitly specified in
>> command line) and if there is any include directive, the
>> corresponding file(s) is getting inserted in this spot. So you end up
>> with an "effective config" which you can see with "-o fullconf" option.
>>
>> This config is intepreted (we're still talking about reading config,
>> not processing events!) from the top to bottom so that for most
>> things if you have some dependency, it must be defined earlier (like
>> you must have a lookup defined earlier in the config to be able to
>> call that lookup later).
>>
>> Then comes the part of event processing - if some operation is not
>> defined within a specific ruleset it's assumed to be in an implicit
>> "main" ruleset. And similarily - if an input is not associated with a
>> specific ruleset, events received by that input are processed by that
>> implicit main ruleset.
>>
>>
>> BTW, if you're planning on just dropping the events from the
>> vulnerability scanners you could just drop events on this one port
>> but I don't know your precise circumstances. It's just that it seems
>> that you're trying to overengineer something that could be much
>> simpler ;-)
>>
>>
>> MK
>>
>>
>> On 9.12.2022 13:15, Steven D wrote:
>> Thanks for all the input, ok so some answers inline
>>
>> Nope - So this looks straight forward to me, but correct some
>> ignorance on my end. The way you've defined the "syslogtag" rulesets,
>> it makes me assume that the syslog config processed in a top down
>> manner, otherwise logically it doesn't work.
>>
>> Is that true?
>>
>> The way we have our syslog config set up is to group all the related
>> template/ruleset/input stanzas together... Just from an ease of
>> modification PoV it makes it easier.
>>
>> Kurk Mariusz - I don't have control over the sending hosts, they are
>> vulnerability scanners that indiscriminately scan the environment. I
>> can't just iptable drop those same devices from touching the syslog
>> servers as that will cause other operational issues.
>> ________________________________
>> From: rsyslog
>> <rsyslog-bounces@lists.adiscon.com><mailto:rsyslog-bounces@lists.adiscon.com>
>> on behalf of Mariusz Kruk via rsyslog
>> <rsyslog@lists.adiscon.com><mailto:rsyslog@lists.adiscon.com>
>> Sent: Friday, December 9, 2022 5:30 AM
>> To: rsyslog@lists.adiscon.com<mailto:rsyslog@lists.adiscon.com>
>> <rsyslog@lists.adiscon.com><mailto:rsyslog@lists.adiscon.com>
>> Cc: Mariusz Kruk <kruk@epsilon.eu.org><mailto:kruk@epsilon.eu.org>
>> Subject: Re: [rsyslog] "Global" fromhost-ip blocklist?
>>
>> I know that it has already been answered but let me add my three
>> cents ;-)
>>
>> Firstly, adding condition to $fromhost-ip suggests that you want to
>> limit based on the source IP, not on the event's content. Which raises
>> the question - why not simply _not_ send from that host? Or at least
>> filter it out on the local firewall (most probably iptables).
>>
>> Secondly, expanding on nope's response - you can have multiple rulesets
>> chained together so that you have some common "subroutine" and then
>> branch to specific ruleset depending on how you want to process given
>> source or data type. You can use lookups or conditions to dynamically
>> decide to which ruleset you want to route your event to. The
>> possibilities are endless :-) (and you can end up creating a ruleset
>> loop and crashing your rsyslogd XD)
>>
>> Something like.
>>
>> ruleset(name="ruleset1") {
>> set $.destination_ruleset="dest1";
>> call intermediate_ruleset
>> }
>>
>> ruleset(name="ruleset2") {
>> set $.destination_ruleset="dest2";
>> call intermediate_ruleset
>> }
>>
>> ruleset(name="intermediate_ruleset") {
>> set $.this=$that;
>> // and other stuff
>> call_indirect $.destination_ruleset;
>> }
>>
>> ruleset(name="dest1") {
>> ...
>> }
>>
>> ruleset(name="dest2") {
>> ...
>> }
>>
>> MK
>>
>> On 8.12.2022 16:15, Steven D via rsyslog wrote:
>>> Rsyslog gurus
>>>
>>> I have a config that accepts connections from remote hosts and
>>> steers logs to files based on port. Pretty straightforward... what
>>> i'm looking to do is "globally" prevent certain ip addresses from
>>> ending up in the logs. (Internal vulnerability scanners I have no
>>> control over).
>>>
>>> I've tried a few different ways but not coming across anything that
>>> works globally. Adding something like "if $fromhost-ip '1.2.3.4'
>>> then stop" works just fine on an individual ruleset.
>>>
>>> Is there a way I can do this without having to enter duplicate lines
>>> in every ruleset (I have like 30 rulesets) ?
>>>
>>> Thanks,
>>> Steven
>>>
>>> Config snippet below: "#logname01/02#" is replaced by the relevant
>>> product in the configuration.
>>>
>>> module(load="imudp")
>>> module(load="imtcp" MaxListeners="100" AddtlFrameDelimiter="000"
>>> KeepAlive="on" KeepAlive.Probes="1" KeepAlive.Time="10")
>>>
>>> input(type="imudp" port="24514" ruleset="#logname01#_rule")
>>> input(type="imtcp" port="24514" ruleset="#logname01#_rule")
>>> template(name="#logname01#_logs" type="string"
>>> string="/data/logs/#logname01#/24514/%fromhost-ip%/syslog.log")
>>> ruleset(name="#logname01#_rule") {
>>> action(name="#logname01#_rule"
>>> type="omfile"
>>> FileCreateMode="0744"
>>> DirCreateMode="0755"
>>> FileOwner="SIEM"
>>> FileGroup="SIEM"
>>> DirOwner="SIEM"
>>> DirGroup="SIEM"
>>> DynaFile="#logname01#_logs"
>>> DynaFileCacheSize = "50")
>>> }
>>>
>>> input(type="imudp" port="25514" ruleset="#logname02#_rule")
>>> input(type="imtcp" port="25514" ruleset="#logname02#_rule")
>>> template(name="#logname02#_logs" type="string"
>>> string="/data/logs/#logname02#/25514/%fromhost-ip%/syslog.log")
>>> ruleset(name="#logname02#_rule") {
>>> action(name="#logname02#_rule"
>>> type="omfile"
>>> FileCreateMode="0744"
>>> DirCreateMode="0755"
>>> FileOwner="SIEM"
>>> FileGroup="SIEM"
>>> DirOwner="SIEM"
>>> DirGroup="SIEM"
>>> DynaFile="#logname02#_logs"
>>> DynaFileCacheSize = "50")
>>> }
>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
>>> POST if you DON'T LIKE THAT.
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
>> POST if you DON'T LIKE THAT.
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
>> POST if you DON'T LIKE THAT.
>>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
> if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.