Mailing List Archive

log with the template RSYSLOG_DebugFormat so you can see what's in each
variable. I think that will answer your question (I don't think it's what you
think it is)

David Lang

On Mon, 18 Jan 2021, Alex via rsyslog wrote:

> Date: Mon, 18 Jan 2021 21:52:28 -0500
> From: Alex via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog@lists.adiscon.com
> Cc: Alex <mysqlstudent@gmail.com>
> Subject: [rsyslog] logging remote clients based on facility
>
> Hi,
>
> I'm using rsyslog-8.2010.0 on fedora33 as a remote logging server. I'd
> like to consider logging mail.* messages to individual files based on
> hostname. Primarily I want mail messages from the log server (xavier)
> to go in the regular /var/log/maillog file.
>
> This is what I've come up with so far, but it doesn't appear to do
> anything. It also doesn't report any errors when I reload rsyslog.
> What am I doing wrong?
>
> if $hostname == 'xavier' then {
> if $facility == 2 then action(type="omfile" file="/var/log/maillog")
> if $facility != 2 then action(type="omfile" file="/var/log/maillog-other")
> }
>
> I've seen references to $syslogfacility instead of just $facility in
> the documentation, but that doesn't appear to make a difference
> either.
>
> Thanks,
> Alex
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Firstly, the second nested conditional (if != 2) is a bit pointless.
It's what "else" is for ;-)

Secondly, $hostname contains the hostname contained within the message,
not resolved from the source host IP. So if the sender is misconfigured
and sends - for example - localhost in logs, you'll have localhost as
$hostname.

Thirdly, unless you're very very sure about the contents of the logged
events I'd rather go with sorting incoming logs by sender's IP
($fromhost-ip). Caveat: if you're relaying messages for some hosts
further down the chain you'd have to somehow pass the original IP; it
can get tricky.

And finally, if you're planning on extending this mechanism to splitting
to different files for many different sources consider either dynamicaly
templated filenames or using lookups to find appropriate action instead
of if/else if chains.

Of course the advice of logging with debug template is good for every
problem with event processing.

On 19/01/2021 03:52, Alex via rsyslog wrote:
> Hi,
>
> I'm using rsyslog-8.2010.0 on fedora33 as a remote logging server. I'd
> like to consider logging mail.* messages to individual files based on
> hostname. Primarily I want mail messages from the log server (xavier)
> to go in the regular /var/log/maillog file.
>
> This is what I've come up with so far, but it doesn't appear to do
> anything. It also doesn't report any errors when I reload rsyslog.
> What am I doing wrong?
>
> if $hostname == 'xavier' then {
> if $facility == 2 then action(type="omfile" file="/var/log/maillog")
> if $facility != 2 then action(type="omfile" file="/var/log/maillog-other")
> }
>
> I've seen references to $syslogfacility instead of just $facility in
> the documentation, but that doesn't appear to make a difference
> either.
>
> Thanks,
> Alex
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Hi,

Thanks very much for your help.

> Firstly, the second nested conditional (if != 2) is a bit pointless.
> It's what "else" is for ;-)
>
> Secondly, $hostname contains the hostname contained within the message,
> not resolved from the source host IP. So if the sender is misconfigured
> and sends - for example - localhost in logs, you'll have localhost as
> $hostname.
>
> Thirdly, unless you're very very sure about the contents of the logged
> events I'd rather go with sorting incoming logs by sender's IP
> ($fromhost-ip). Caveat: if you're relaying messages for some hosts
> further down the chain you'd have to somehow pass the original IP; it
> can get tricky.
>
> And finally, if you're planning on extending this mechanism to splitting
> to different files for many different sources consider either dynamicaly
> templated filenames or using lookups to find appropriate action instead
> of if/else if chains.
>
> Of course the advice of logging with debug template is good for every
> problem with event processing.

This is what I've come up with.

if $fromhost-ip == '209.216.111.114' then {
if $facility == 2 then { action(type="omfile" file="/var/log/maillog")
} else {
action(type="omfile" file="/var/log/maillog-other")
}

but it then logged nothing after restarting successfully and produced no
indication of what was wrong in /var/log/messages. It also never
produces the /var/log/maillog-other file.

I also tried to enable debugging in my rsyslog.conf:

$DebugLevel 2
$DebugFile /var/log/rsyslog.log

and while it produced too much output to be helpful, I did see that it
at least recorded that IP address.

I'm really unsure what to do next.

Thanks,
Alex
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

On 19/01/2021 17:27, Alex Regan via rsyslog wrote:
> Hi,
>
> Thanks very much for your help.
>
>> Firstly, the second nested conditional (if != 2) is a bit pointless.
>> It's what "else" is for ;-)
>>
>> Secondly, $hostname contains the hostname contained within the
>> message, not resolved from the source host IP. So if the sender is
>> misconfigured and sends - for example - localhost in logs, you'll
>> have localhost as $hostname.
>>
>> Thirdly, unless you're very very sure about the contents of the
>> logged events I'd rather go with sorting incoming logs by sender's IP
>> ($fromhost-ip). Caveat: if you're relaying messages for some hosts
>> further down the chain you'd have to somehow pass the original IP; it
>> can get tricky.
>>
>> And finally, if you're planning on extending this mechanism to
>> splitting to different files for many different sources consider
>> either dynamicaly templated filenames or using lookups to find
>> appropriate action instead of if/else if chains.
>>
>> Of course the advice of logging with debug template is good for every
>> problem with event processing.
>
> This is what I've come up with.
>
> if $fromhost-ip == '209.216.111.114' then {
>    if $facility == 2 then { action(type="omfile" file="/var/log/maillog")
>    } else {
>      action(type="omfile" file="/var/log/maillog-other")
> }
>
> but it then logged nothing after restarting successfully and produced
> no indication of what was wrong in /var/log/messages. It also never
> produces the /var/log/maillog-other file.
>
> I also tried to enable debugging in my rsyslog.conf:
>
> $DebugLevel 2
> $DebugFile /var/log/rsyslog.log
>
> and while it produced too much output to be helpful, I did see that it
> at least recorded that IP address.
>

Hard to tell without really seeing what is in those events that you're
receiving and without seeing whole config.

But.

This form of debugging is _not_ what you need. It's for debugging the
rsyslogd itself, not your rules. Get rid of it.

As David already wrote, you want to use RSYSLOG_DebugFormat to write
full event debug data to a file and see what properties and variables
you have associated with the event. Watch out though because it logs
huge amounts of data so your file will quickly grow beyond your
expectations.

So I'd go with:

action(type="omfile" file="/tmp/debug.log" template="RSYSLOG_DebugFormat")

_before_ this whole config snippet.

Therefore you'd see what you're working on.

If you get no output in the debug.log file, check your permissions (and
selinux if it can be involved) and prior rules. Maybe you have some
rules before this snippet that block further processing and you don't
even get here?


_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

>> This is what I've come up with.
>>
>> if $fromhost-ip == '209.216.111.114' then {
>>    if $facility == 2 then { action(type="omfile" file="/var/log/maillog")
>>    } else {
>>      action(type="omfile" file="/var/log/maillog-other")
>> }
>>
>> but it then logged nothing after restarting successfully and produced
>> no indication of what was wrong in /var/log/messages. It also never
>> produces the /var/log/maillog-other file.
>>
>> I also tried to enable debugging in my rsyslog.conf:
>>
>> $DebugLevel 2
>> $DebugFile /var/log/rsyslog.log
>>
>> and while it produced too much output to be helpful, I did see that it
>> at least recorded that IP address.
>>
>
> Hard to tell without really seeing what is in those events that you're
> receiving and without seeing whole config.
>
> But.
>
> This form of debugging is _not_ what you need. It's for debugging the
> rsyslogd itself, not your rules. Get rid of it.
>
> As David already wrote, you want to use RSYSLOG_DebugFormat to write
> full event debug data to a file and see what properties and variables
> you have associated with the event. Watch out though because it logs
> huge amounts of data so your file will quickly grow beyond your
> expectations.
>
> So I'd go with:
>
> action(type="omfile" file="/tmp/debug.log" template="RSYSLOG_DebugFormat")

Okay, now I understand. It's produced output like:

FROMHOST: 'xavier', fromhost-ip: '127.0.0.1', HOSTNAME: 'xavier', PRI:
22,syslogtag 'postfix-117/qmgr[496743]:', programname: 'postfix-117',
APP-NAME: 'postfix-117', PROCID: '496743', MSGID: '-',TIMESTAMP: 'Jan 20
08:39:54', STRUCTURED-DATA: '-',msg: '6B1B930668306: removed' escaped
msg: '6B1B930668306: removed' inputname: imjournal rawmsg:
'6B1B930668306: removed' $!:{ "PRIORITY": "6", "_BOOT_ID":
"6ff20e0e797d45789b7c38229e26f928", "_MACHINE_ID":
"c4b32aa0d25c4a5d85432835f7c2e2ac", "_HOSTNAME": "xavier.example.com",
"_TRANSPORT": "syslog", "SYSLOG_FACILITY": "2", "_UID": "89", "_GID":
"89", "_CAP_EFFECTIVE": "0", "_SYSTEMD_CGROUP":
"\/system.slice\/postfix.service", "_SYSTEMD_UNIT": "postfix.service",
"_SYSTEMD_SLICE": "system.slice", "_SYSTEMD_INVOCATION_ID":
"dde7fdbb530148f89ad2ee01b46615ac", "_COMM": "qmgr", "_EXE":
"\/usr\/libexec\/postfix\/qmgr", "_CMDLINE": "qmgr -l -t unix -u",
"SYSLOG_IDENTIFIER": "postfix-117\/qmgr", "SYSLOG_PID": "496743",
"_PID": "496743", "SYSLOG_TIMESTAMP": "Jan 20 08:39:54 ", "MESSAGE":
"6B1B930668306: removed", "_SOURCE_REALTIME_TIMESTAMP": "1611149994883159" }

I've modified my config to the following:

if $fromhost-ip == "127.0.0.1" then {
if $syslogfacility == 2 then { action(type="omfile"
file="/var/log/maillog") }
} else {
action(type="omfile" file="/var/log/maillog-other")
}
}

It now logs only messages from the local host to /var/log/maillog, but
the maillog-other file is not logging messages from the other hosts
sending their maillogs to this server. What am I missing?

Here is my full config.

$MaxMessageSize 65536
$ModLoad imjournal # provides access to the systemd journal
$ModLoad imudp
$UDPServerRun 514
$ModLoad imtcp
$InputTCPServerRun 10514
$InputTCPServerBindRuleset remote
$ModLoad imuxsock
$ModLoad imklog
$ActionQueueFileName fwdRule1
$ActionQueueMaxDiskSpace 1g
$ActionQueueSaveOnShutdown on
$ActionQueueType LinkedList
$ActionResumeRetryCount -1
$SystemLogRateLimitInterval 0
:msg,contains,"LOGDROP " /var/log/iptables.log
& stop
if $programname == 'audit' then {
action(type="omfile" file="/var/log/kernel.audit.log")
# if $syslogseverity >= 4 then stop # warning
if $syslogseverity >= 5 then stop # notice
# if $syslogseverity >= 6 then stop # info
}
$WorkDirectory /var/lib/rsyslog
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$IncludeConfig /etc/rsyslog.d/*.conf
$IMJournalStateFile imjournal.state
$IMJournalIgnorePreviousMessages on
kern.none /dev/console
kern.* /var/log/kern.log
*.info;kern.none;mail.none;authpriv.none;cron.none /var/log/messages
authpriv.* /var/log/secure
action(type="omfile" file="/var/log/rsyslog.log"
template="RSYSLOG_DebugFormat")
if $fromhost-ip == "127.0.0.1" then {
if $syslogfacility == 2 then { action(type="omfile"
file="/var/log/maillog") }
} else {
action(type="omfile" file="/var/log/maillog-other")
}
}
cron.* /var/log/cron
*.emerg :omusrmsg:*
uucp,news.crit /var/log/spooler
local7.* /var/log/boot.log




_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

On 20/01/2021 15:04, Alex Regan via rsyslog wrote:
>
> Okay, now I understand. It's produced output like:
>
> FROMHOST: 'xavier', fromhost-ip: '127.0.0.1', HOSTNAME: 'xavier', PRI:
> 22,syslogtag 'postfix-117/qmgr[496743]:', programname: 'postfix-117',
> APP-NAME: 'postfix-117', PROCID: '496743', MSGID: '-',TIMESTAMP: 'Jan
> 20 08:39:54', STRUCTURED-DATA: '-',msg: '6B1B930668306: removed'
> escaped msg: '6B1B930668306: removed' inputname: imjournal rawmsg:
> '6B1B930668306: removed' $!:{ "PRIORITY": "6", "_BOOT_ID":
> "6ff20e0e797d45789b7c38229e26f928", "_MACHINE_ID":
> "c4b32aa0d25c4a5d85432835f7c2e2ac", "_HOSTNAME": "xavier.example.com",
> "_TRANSPORT": "syslog", "SYSLOG_FACILITY": "2", "_UID": "89", "_GID":
> "89", "_CAP_EFFECTIVE": "0", "_SYSTEMD_CGROUP":
> "\/system.slice\/postfix.service", "_SYSTEMD_UNIT": "postfix.service",
> "_SYSTEMD_SLICE": "system.slice", "_SYSTEMD_INVOCATION_ID":
> "dde7fdbb530148f89ad2ee01b46615ac", "_COMM": "qmgr", "_EXE":
> "\/usr\/libexec\/postfix\/qmgr", "_CMDLINE": "qmgr -l -t unix -u",
> "SYSLOG_IDENTIFIER": "postfix-117\/qmgr", "SYSLOG_PID": "496743",
> "_PID": "496743", "SYSLOG_TIMESTAMP": "Jan 20 08:39:54 ", "MESSAGE":
> "6B1B930668306: removed", "_SOURCE_REALTIME_TIMESTAMP":
> "1611149994883159" }
>
> I've modified my config to the following:
>
> if $fromhost-ip == "127.0.0.1" then {
>    if $syslogfacility == 2 then { action(type="omfile"
> file="/var/log/maillog") }
>    } else {
>      action(type="omfile" file="/var/log/maillog-other")
>    }
> }
>
> It now logs only messages from the local host to /var/log/maillog, but
> the maillog-other file is not logging messages from the other hosts
> sending their maillogs to this server. What am I missing?
>
Does this config properly validate? It seems that you have some
unbalanced brackets.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

>> I've modified my config to the following:
>>
>> if $fromhost-ip == "127.0.0.1" then {
>>    if $syslogfacility == 2 then { action(type="omfile"
>> file="/var/log/maillog") }
>>    } else {
>>      action(type="omfile" file="/var/log/maillog-other")
>>    }
>> }
>>
>> It now logs only messages from the local host to /var/log/maillog, but
>> the maillog-other file is not logging messages from the other hosts
>> sending their maillogs to this server. What am I missing?
>>
> Does this config properly validate? It seems that you have some
> unbalanced brackets.

Ah, thanks. Looking at it more closely helped me also realize the logic
was wrong. This is what I wanted, and now works properly.

if $fromhost-ip == "127.0.0.1" then {
#action(type="omfile" file="/var/log/maillog")
if $syslogfacility == 2 then { action(type="omfile"
file="/var/log/maillog") }
} else {
if $syslogfacility == 2 then { action(type="omfile"
file="/var/log/maillog-other") }
}





_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.