Mailing List Archive

Hostname field changed in rsyslog proxy
Dear rsyslog mailing list,

First time coming out of the shadow here for a particular behavior that
I've not well understood in our stack here.
So, we are monitoring our network using syslog using rsyslog as a proxy
to be certain of the version format used, as the equipements are very
diverse and some respects some rfc, some others, and some
implementations are buggous. Last time we had an issue were we had an
alert on a log volume on a equipement hostname that is not anymore in
our network. The IP of the equipement was reused somewhere else.

It seemed that instead of keeping the hostname "as is" the rsyslog proxy
was using it's own name resolution and replaced the hostname first
present in the packet by the one in it cache, resulting in the wrong
hostname beeing transmeted to the promtail/loki infrastructure behind.

Here is what we have as a remote.conf: We have made that for a buggous
implementation where APPNAME is seen as NULL (not empty string)
resulting in failure of rsyslog. I don't think this behavious is
especialy linked to the %HOSTNAME% but I want to be sure with you.


```
$ModLoad omfwd
:app-name, !isequal, "" {
action(type="omfwd" protocol="tcp" target= "localhost" port="1515"
Template="RSYSLOG_SyslogProtocol23Format" TCP_Framing="octet-counted"
KeepAlive="on")
}

template(name="missingAppName" type="string" string="<%PRI%>1
%TIMESTAMP:::date-rfc3339% %HOSTNAME% NOAPPNAME %PROCID% %MSGID%
%STRUCTURED-DATA% %msg% \n")

:app-name, isequal, "" {
action(type="omfwd" protocol="tcp" target= "localhost" port="1515"
Template="missingAppName" TCP_Framing="octet-counted" KeepAlive="on")
}
```



Could you please clarify how rsyslog interact with the system name
resolution cache as we thought it wouldn't be touching the value of
hostname before retransmiting it. It has been fixed by removing the
cache entry. The server is a Debian bullseye with a 8.2102.0-2+deb11u1
(version might be incorrect as I've had the issue couple month ago and
haven't noted here. I will try to find it back if needed).

Best regards.


_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Hostname field changed in rsyslog proxy [ In reply to ]
hostname should not change, fromhost and fromhost-ip will change.

David Lang


On Thu, 2 Nov 2023, Martin Passard via rsyslog wrote:

> Date: Thu, 2 Nov 2023 14:46:34 +0000
> From: Martin Passard via rsyslog <rsyslog@lists.adiscon.com>
> To: "rsyslog@lists.adiscon.com" <rsyslog@lists.adiscon.com>
> Cc: Martin Passard <martin.passard@ielo.net>
> Subject: [rsyslog] Hostname field changed in rsyslog proxy
>
> Dear rsyslog mailing list,
>
> First time coming out of the shadow here for a particular behavior that
> I've not well understood in our stack here.
> So, we are monitoring our network using syslog using rsyslog as a proxy
> to be certain of the version format used, as the equipements are very
> diverse and some respects some rfc, some others, and some
> implementations are buggous. Last time we had an issue were we had an
> alert on a log volume on a equipement hostname that is not anymore in
> our network. The IP of the equipement was reused somewhere else.
>
> It seemed that instead of keeping the hostname "as is" the rsyslog proxy
> was using it's own name resolution and replaced the hostname first
> present in the packet by the one in it cache, resulting in the wrong
> hostname beeing transmeted to the promtail/loki infrastructure behind.
>
> Here is what we have as a remote.conf: We have made that for a buggous
> implementation where APPNAME is seen as NULL (not empty string)
> resulting in failure of rsyslog. I don't think this behavious is
> especialy linked to the %HOSTNAME% but I want to be sure with you.
>
>
> ```
> $ModLoad omfwd
> :app-name, !isequal, "" {
> action(type="omfwd" protocol="tcp" target= "localhost" port="1515"
> Template="RSYSLOG_SyslogProtocol23Format" TCP_Framing="octet-counted"
> KeepAlive="on")
> }
>
> template(name="missingAppName" type="string" string="<%PRI%>1
> %TIMESTAMP:::date-rfc3339% %HOSTNAME% NOAPPNAME %PROCID% %MSGID%
> %STRUCTURED-DATA% %msg% \n")
>
> :app-name, isequal, "" {
> action(type="omfwd" protocol="tcp" target= "localhost" port="1515"
> Template="missingAppName" TCP_Framing="octet-counted" KeepAlive="on")
> }
> ```
>
>
>
> Could you please clarify how rsyslog interact with the system name
> resolution cache as we thought it wouldn't be touching the value of
> hostname before retransmiting it. It has been fixed by removing the
> cache entry. The server is a Debian bullseye with a 8.2102.0-2+deb11u1
> (version might be incorrect as I've had the issue couple month ago and
> haven't noted here. I will try to find it back if needed).
>
> Best regards.
>
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Hostname field changed in rsyslog proxy [ In reply to ]
Well, you say that the equipment does not necessarily follow the RFC.
In that case, rsyslog might not be able to read the hostname from the
message. We try hard to handle "well known malformed formats", but we
cannot let everything slip through. Especially if it really is unclear
where the hostname is present.

In such cases, rsyslog falls back to the sender IP address. This *may*
be what you are seeing.

We would need to see the original message from the device where the
"wrong hostname" will be shown later on.

HTH
Rainer

El jue, 2 nov 2023 a las 15:46, Martin Passard via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> Dear rsyslog mailing list,
>
> First time coming out of the shadow here for a particular behavior that
> I've not well understood in our stack here.
> So, we are monitoring our network using syslog using rsyslog as a proxy
> to be certain of the version format used, as the equipements are very
> diverse and some respects some rfc, some others, and some
> implementations are buggous. Last time we had an issue were we had an
> alert on a log volume on a equipement hostname that is not anymore in
> our network. The IP of the equipement was reused somewhere else.
>
> It seemed that instead of keeping the hostname "as is" the rsyslog proxy
> was using it's own name resolution and replaced the hostname first
> present in the packet by the one in it cache, resulting in the wrong
> hostname beeing transmeted to the promtail/loki infrastructure behind.
>
> Here is what we have as a remote.conf: We have made that for a buggous
> implementation where APPNAME is seen as NULL (not empty string)
> resulting in failure of rsyslog. I don't think this behavious is
> especialy linked to the %HOSTNAME% but I want to be sure with you.
>
>
> ```
> $ModLoad omfwd
> :app-name, !isequal, "" {
> action(type="omfwd" protocol="tcp" target= "localhost" port="1515"
> Template="RSYSLOG_SyslogProtocol23Format" TCP_Framing="octet-counted"
> KeepAlive="on")
> }
>
> template(name="missingAppName" type="string" string="<%PRI%>1
> %TIMESTAMP:::date-rfc3339% %HOSTNAME% NOAPPNAME %PROCID% %MSGID%
> %STRUCTURED-DATA% %msg% \n")
>
> :app-name, isequal, "" {
> action(type="omfwd" protocol="tcp" target= "localhost" port="1515"
> Template="missingAppName" TCP_Framing="octet-counted" KeepAlive="on")
> }
> ```
>
>
>
> Could you please clarify how rsyslog interact with the system name
> resolution cache as we thought it wouldn't be touching the value of
> hostname before retransmiting it. It has been fixed by removing the
> cache entry. The server is a Debian bullseye with a 8.2102.0-2+deb11u1
> (version might be incorrect as I've had the issue couple month ago and
> haven't noted here. I will try to find it back if needed).
>
> Best regards.
>
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.