Mailing List Archive: Short source hostname

Short source hostname

Feb 23, 2024, 7:13 AM

Post #1 of 6 (136 views)

Hello rsyslog experts,

newbie writing his first post to the list here...

I am setting up a central rsyslog server, collecting all syslog messages from any type of computer and other devices. Clients run different variants of syslog, up to networked IoT devices sending dumb messages. So I can't format messages on the client. All I can do is making the rsyslog server to cope with it somehow.

Collected messages are stored in a separate subdirectory named after the client. I would prefer to use the short client name for this directory. But I keep getting the FQDN.

I switched the global option PreserveFQDN to off, but it had no effect. Is this option only meant for the sending clients or is it supposed to work on the receiving server as well?

What other ways would be possible to shorten the path for the storage? Would I need to build this into the template?
My current template is

$template RemoteLogs,"/var/log/loghost/%FROMHOST%/syslog

Is there another variable as replacement for FROMHOST containing only the short name of the sending client? I tried HOSTNAME , but this also contained the FQDN.

Any tipps or (learning) hints are appreciated.

Best regards,
Frank
--
Frank Morawietz

IT Systems Administrator Global Computational Chemistry & Biology
Healthcare | Discovery Development Technologies
Merck Healthcare KGaA | Frankfurter Str. 250 | Postcode: D050/902 | 64293 Darmstadt | Germany

This message and any attachment are confidential and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you must not copy this message or attachment or disclose the contents to any other person. If you have received this transmission in error, please notify the sender immediately and delete the message and any attachment from your system. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not accept liability for any omissions or errors in this message which may arise as a result of E-Mail-transmission or for damages resulting from any unauthorized changes of the content of this message and any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not guarantee that this message is free of viruses and does not accept liability for any damages caused by any virus transmitted therewith.

Click merckgroup.com/disclaimer<https://www.merckgroup.com/en/legal-disclaimer/mail-disclaimer.html> to access the German, French, Spanish, Portuguese, Turkish, Polish and Slovak versions of this disclaimer.

Please find our Privacy Statement information by clicking here: merckgroup.com/privacy-statements-by-location<https://www.merckgroup.com/en/privacy-statement/privacy-statements-by-location.html>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Re: Short source hostname [ In reply to ]

rsyslog at lists

Feb 23, 2024, 7:31 AM

Post #2 of 6 (136 views)

Permalink

I'd simply go for IP instead of hostname. (or keep your local /etc/hosts
database up to date but that's ugly).

Also remember that fromhost and fromhost-ip are from the _connection_.
Hostname is the value from the message (might be completely bonkers if
the source doesn't follow standards).

You could also just do your own field based on the $fromhost value, just
split at first dot.

set $.loghost=field($hostname,46,1);

Then you can use the $.loghost variable in your template.

MK

On 23.02.2024 16:13, Frank Morawietz via rsyslog wrote:
> Hello rsyslog experts,
>
> newbie writing his first post to the list here...
>
> I am setting up a central rsyslog server, collecting all syslog messages from any type of computer and other devices. Clients run different variants of syslog, up to networked IoT devices sending dumb messages. So I can't format messages on the client. All I can do is making the rsyslog server to cope with it somehow.
>
> Collected messages are stored in a separate subdirectory named after the client. I would prefer to use the short client name for this directory. But I keep getting the FQDN.
>
> I switched the global option PreserveFQDN to off, but it had no effect. Is this option only meant for the sending clients or is it supposed to work on the receiving server as well?
>
> What other ways would be possible to shorten the path for the storage? Would I need to build this into the template?
> My current template is
>
> $template RemoteLogs,"/var/log/loghost/%FROMHOST%/syslog
>
> Is there another variable as replacement for FROMHOST containing only the short name of the sending client? I tried HOSTNAME , but this also contained the FQDN.
>
> Any tipps or (learning) hints are appreciated.
>
> Best regards,
> Frank
> --
> Frank Morawietz
>
> IT Systems Administrator Global Computational Chemistry & Biology
> Healthcare | Discovery Development Technologies
> Merck Healthcare KGaA | Frankfurter Str. 250 | Postcode: D050/902 | 64293 Darmstadt | Germany
>
>
>
>
> This message and any attachment are confidential and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you must not copy this message or attachment or disclose the contents to any other person. If you have received this transmission in error, please notify the sender immediately and delete the message and any attachment from your system. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not accept liability for any omissions or errors in this message which may arise as a result of E-Mail-transmission or for damages resulting from any unauthorized changes of the content of this message and any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not guarantee that this message is free of viruses and does not accept liability for any damages caused by any virus transmitted therewith.
>
>
>
> Click merckgroup.com/disclaimer<https://www.merckgroup.com/en/legal-disclaimer/mail-disclaimer.html> to access the German, French, Spanish, Portuguese, Turkish, Polish and Slovak versions of this disclaimer.
>
>
>
> Please find our Privacy Statement information by clicking here: merckgroup.com/privacy-statements-by-location<https://www.merckgroup.com/en/privacy-statement/privacy-statements-by-location.html>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Re: Short source hostname [ In reply to ]

rsyslog at lists

Feb 23, 2024, 11:29 AM

Post #3 of 6 (134 views)

Permalink

Thanks for your reply, Mariusz.

> Also remember that fromhost and fromhost-ip are from the _connection_.
> Hostname is the value from the message (might be completely bonkers if
> the source doesn't follow standards).

This is exactly why I switched to FROMHOST. To much messages with junk instead of the hostname.

> You could also just do your own field based on the $fromhost value, just
> split at first dot.
>
>set $.loghost=field($hostname,46,1);
>
>Then you can use the $.loghost variable in your template.

Got it. I already tried this adaption of my template and it's looking like I wanted now!

Best regards,
Frank

P.S.: So the global option PreserveFQDN is not used on the receiving server?
--
Frank Morawietz

IT Systems Administrator Global Computational Chemistry & Biology
Healthcare | Discovery Development Technologies
Merck Healthcare KGaA | Frankfurter Str. 250 | Postcode: D050/902 | 64293 Darmstadt | Germany

This message and any attachment are confidential and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you must not copy this message or attachment or disclose the contents to any other person. If you have received this transmission in error, please notify the sender immediately and delete the message and any attachment from your system. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not accept liability for any omissions or errors in this message which may arise as a result of E-Mail-transmission or for damages resulting from any unauthorized changes of the content of this message and any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not guarantee that this message is free of viruses and does not accept liability for any damages caused by any virus transmitted therewith.

Click merckgroup.com/disclaimer<https://www.merckgroup.com/en/legal-disclaimer/mail-disclaimer.html> to access the German, French, Spanish, Portuguese, Turkish, Polish and Slovak versions of this disclaimer.

Please find our Privacy Statement information by clicking here: merckgroup.com/privacy-statements-by-location<https://www.merckgroup.com/en/privacy-statement/privacy-statements-by-location.html>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Re: Short source hostname [ In reply to ]

rsyslog at lists

Feb 23, 2024, 1:29 PM

Post #4 of 6 (134 views)

Permalink

$fromhost is the result of a name lookup of the IP of the connection, it is
always the full result of that lookup (which will normally be a FQDN from DNS

David Lang

On Fri, 23 Feb 2024, Frank Morawietz via rsyslog wrote:

> Thanks for your reply, Mariusz.
>
>> Also remember that fromhost and fromhost-ip are from the _connection_.
>> Hostname is the value from the message (might be completely bonkers if
>> the source doesn't follow standards).
>
> This is exactly why I switched to FROMHOST. To much messages with junk instead of the hostname.
>
>> You could also just do your own field based on the $fromhost value, just
>> split at first dot.
>>
>> set $.loghost=field($hostname,46,1);
>>
>> Then you can use the $.loghost variable in your template.
>
> Got it. I already tried this adaption of my template and it's looking like I wanted now!
>
> Best regards,
> Frank
>
> P.S.: So the global option PreserveFQDN is not used on the receiving server?
> --
> Frank Morawietz
>
> IT Systems Administrator Global Computational Chemistry & Biology
> Healthcare | Discovery Development Technologies
> Merck Healthcare KGaA | Frankfurter Str. 250 | Postcode: D050/902 | 64293 Darmstadt | Germany
>
>
> This message and any attachment are confidential and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you must not copy this message or attachment or disclose the contents to any other person. If you have received this transmission in error, please notify the sender immediately and delete the message and any attachment from your system. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not accept liability for any omissions or errors in this message which may arise as a result of E-Mail-transmission or for damages resulting from any unauthorized changes of the content of this message and any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not guarantee that this message is free of viruses and does not accept liability for any damages caused by any virus transmitted therewith.
>
>
>
> Click merckgroup.com/disclaimer<https://www.merckgroup.com/en/legal-disclaimer/mail-disclaimer.html> to access the German, French, Spanish, Portuguese, Turkish, Polish and Slovak versions of this disclaimer.
>
>
>
> Please find our Privacy Statement information by clicking here: merckgroup.com/privacy-statements-by-location<https://www.merckgroup.com/en/privacy-statement/privacy-statements-by-location.html>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Re: Short source hostname [ In reply to ]

rsyslog at lists

Feb 24, 2024, 2:25 AM

Post #5 of 6 (134 views)

Permalink

On 23.02.2024 20:29, Frank Morawietz via rsyslog wrote:
>> You could also just do your own field based on the $fromhost value, just
>> split at first dot.
>>
>> set $.loghost=field($hostname,46,1);
>>
>> Then you can use the $.loghost variable in your template.
> Got it. I already tried this adaption of my template and it's looking like I wanted now!

One thing though. As far as I remember, if your receiver cannot resolve
the hostname, your $fromhost will be identical to $fromhost-ip.

So trimming it after first dot will yield unwanted results.

> P.S.: So the global option PreserveFQDN is not used on the receiving server?
Honestly - have no idea. I don't use it :-)
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Re: Short source hostname [ In reply to ]

rsyslog at lists

Feb 24, 2024, 2:35 AM

Post #6 of 6 (134 views)

Permalink

On Sat, 24 Feb 2024, Mariusz Kruk via rsyslog wrote:

> On 23.02.2024 20:29, Frank Morawietz via rsyslog wrote:
>>> You could also just do your own field based on the $fromhost value, just
>>> split at first dot.
>>>
>>> set $.loghost=field($hostname,46,1);
>>>
>>> Then you can use the $.loghost variable in your template.
>> Got it. I already tried this adaption of my template and it's looking like
>> I wanted now!
>
> One thing though. As far as I remember, if your receiver cannot resolve the
> hostname, your $fromhost will be identical to $fromhost-ip.
>
> So trimming it after first dot will yield unwanted results.

correct

>> P.S.: So the global option PreserveFQDN is not used on the receiving
>> server?
> Honestly - have no idea. I don't use it :-)

I think that has to do with the local hostname for the logs it sends, but I
don't remember.

lots of log analysis stuff breaks if the short hostname isn't unique, so while
you can frequently get away with using a FQDN and not having the short name be
unique, it is likely bite you at some point. Best to avoid it.

David Lang
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.