Mailing List Archive

Problem making multiple rulesets work
Hi there,

I'm pretty new to rsyslog, so it's likely I'm doing something wrong. I'm
trying to build a dockerised syslog router based on rsyslog.

The docker container is listening on UDP514 and receiving syslogs from
multiple sources and I want to route the raw syslogs to some destinations,
as well as parse them and send them on to different Elasticsearch
destinations, and perhaps Kafka in the future.

To start with, I'm just trying to forward on the syslogs received on UDP
514 to another syslog server also listening on UDP 514.

I would also like to monitor the local Alpine server in the container and
have those logs writing only to stdout which can then get picked up by
filebeat which is monitoring all my containers, or seen by the "docker
logs" command.

What I don't want is for the container syslogs being written to the UDP 514
outbound stream or the syslogs being received on UDP 514 being written to
the local container syslogs via stdout.

I tried making two different multiple rulesets and actions and binding them
to the UDP 514 and the linux socket inputs. When I tried this, I could
only see the forwarded UDP 514 syslogs, but not the stdout local container
syslogs. However for some reason, when I only used one ruleset for the UDP
traffic, and then had a default rule for the local syslogs, it seemed to
work.

I'd love to know why my multiple ruleset configuration didn't work. I
tried following the documentation here:
https://www.rsyslog.com/doc/v8-stable/concepts/multi_ruleset.html and
Rainer's helpful tutorial here:
https://rainer.gerhards.net/2019/10/rsyslog-relay-messages-only-no-local-storage.html

Here is my working configuration with the one ruleset and the default:

# configure inputs

# local system logging
module(load="imuxsock")
input(type="imuxsock" Socket="/var/run/rsyslog/dev/log" CreatePath="on")

# run UDP syslog on syslog port 514
module(load="imudp")
input(type="imudp" port="514" ruleset="syslogin")

# configure output actions with rulesets for UDP 514 outbound

ruleset(name="syslogin"){
action(type="omfwd" target="my-remote-server.com" port="514"
protocol="udp")
}

# default is to send all output to stdout, in this case only
localfiles, as UDP 514 input is forwarded to 514 UDP.
$ModLoad omstdout.so
*.* :omstdout:

But the one with two rulesets bound to each of the inputs only
forwards the UDP 514 stream:

$ModLoad omstdout.so

# configure inputs

# local system logging
module(load="imuxsock")
input(type="imuxsock" Socket="/var/run/rsyslog/dev/log"
CreatePath="on" ruleset="local")

# run UDP syslog on syslog port 514
module(load="imudp")
input(type="imudp" port="514" ruleset="syslogin")

# configure output actions via rulesets

ruleset(name="syslogin"){
action(type="omfwd" target="my-remote-server.com" port="514"
protocol="udp")
}

ruleset(name="local"){
action(type="omstdout")
}

Thanks for any help with this!
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Problem making multiple rulesets work [ In reply to ]
as I answered elsewhere, I think the issue is imuxsock not supporting rulesets,
what version of rsyslog are you running?

David Lang

On Fri, 18 Oct 2019, Stephen Greszczyszyn via rsyslog wrote:

> Date: Fri, 18 Oct 2019 20:29:35 +0100
> From: Stephen Greszczyszyn via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog@lists.adiscon.com
> Cc: Stephen Greszczyszyn <sgreszcz@gmail.com>
> Subject: [rsyslog] Problem making multiple rulesets work
>
> Hi there,
>
> I'm pretty new to rsyslog, so it's likely I'm doing something wrong. I'm
> trying to build a dockerised syslog router based on rsyslog.
>
> The docker container is listening on UDP514 and receiving syslogs from
> multiple sources and I want to route the raw syslogs to some destinations,
> as well as parse them and send them on to different Elasticsearch
> destinations, and perhaps Kafka in the future.
>
> To start with, I'm just trying to forward on the syslogs received on UDP
> 514 to another syslog server also listening on UDP 514.
>
> I would also like to monitor the local Alpine server in the container and
> have those logs writing only to stdout which can then get picked up by
> filebeat which is monitoring all my containers, or seen by the "docker
> logs" command.
>
> What I don't want is for the container syslogs being written to the UDP 514
> outbound stream or the syslogs being received on UDP 514 being written to
> the local container syslogs via stdout.
>
> I tried making two different multiple rulesets and actions and binding them
> to the UDP 514 and the linux socket inputs. When I tried this, I could
> only see the forwarded UDP 514 syslogs, but not the stdout local container
> syslogs. However for some reason, when I only used one ruleset for the UDP
> traffic, and then had a default rule for the local syslogs, it seemed to
> work.
>
> I'd love to know why my multiple ruleset configuration didn't work. I
> tried following the documentation here:
> https://www.rsyslog.com/doc/v8-stable/concepts/multi_ruleset.html and
> Rainer's helpful tutorial here:
> https://rainer.gerhards.net/2019/10/rsyslog-relay-messages-only-no-local-storage.html
>
> Here is my working configuration with the one ruleset and the default:
>
> # configure inputs
>
> # local system logging
> module(load="imuxsock")
> input(type="imuxsock" Socket="/var/run/rsyslog/dev/log" CreatePath="on")
>
> # run UDP syslog on syslog port 514
> module(load="imudp")
> input(type="imudp" port="514" ruleset="syslogin")
>
> # configure output actions with rulesets for UDP 514 outbound
>
> ruleset(name="syslogin"){
> action(type="omfwd" target="my-remote-server.com" port="514"
> protocol="udp")
> }
>
> # default is to send all output to stdout, in this case only
> localfiles, as UDP 514 input is forwarded to 514 UDP.
> $ModLoad omstdout.so
> *.* :omstdout:
>
> But the one with two rulesets bound to each of the inputs only
> forwards the UDP 514 stream:
>
> $ModLoad omstdout.so
>
> # configure inputs
>
> # local system logging
> module(load="imuxsock")
> input(type="imuxsock" Socket="/var/run/rsyslog/dev/log"
> CreatePath="on" ruleset="local")
>
> # run UDP syslog on syslog port 514
> module(load="imudp")
> input(type="imudp" port="514" ruleset="syslogin")
>
> # configure output actions via rulesets
>
> ruleset(name="syslogin"){
> action(type="omfwd" target="my-remote-server.com" port="514"
> protocol="udp")
> }
>
> ruleset(name="local"){
> action(type="omstdout")
> }
>
> Thanks for any help with this!
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.