Mailing List Archive

FROMHOST missing on central log collector
Hello,

I have a central log server, many of them, using rsyslog to aggregate logs
from remote servers. Everything works great but I have a new challenge and
am hoping for some recommendations.

I have a number of AWS auto-scaling groups where compute resources are
dynamically scaled up and down. Each of these will have a custom rsyslog
configuration pulled from the AWS AMI.

These dynamic resources are not added to DNS due to their dynamic nature so
they will not have DNS assigned FQDNs.

Because of the lack of a hostname, my central log server is getting only
IP. I aggregate based on FROMHOST-FROMHOST-IP.

So what I'm seeing today looks like '10.38.134.77-10.38.134.77' where I
want to see ause1oagbtst03.mydomain.com-10.41.102.168

What I'd want to do is have easy resource send using the same hostname and
current IP. This later will allow me to aggregate all resources by name.

I did not see any way of affecting the FROMHOST information unless, on the
collector, I have rules based on IP address which isn't optimal given the
dynamic nature of the IPs changing.

Any suggestion is appreciated.

*Scott Slattery*

*Sr. Enterprise/Cloud Architect*

*Cloud, Compute, Information & Architecture Team*

motorolasolutions.com

*O: 602.529.8226*

*E*: Scott.Slattery@MotorolaSolutions.com

--


*For more information on how and why we collect your personal
information, please visit our Privacy Policy
<https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement>.*
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
fromhost is the result of a name lookup of fromhost-ip. On the receiver, you can
control this with your name resolution (DNS, /etc/hosts, other mechanisms)

but a better option would probably be to set the hostname on the sender. The
hostname field in the message is under the full control of the sender.

David Lang

On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:

> Date: Tue, 16 Nov 2021 14:56:09 -0700
> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog-users <rsyslog@lists.adiscon.com>
> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
> Subject: [rsyslog] FROMHOST missing on central log collector
>
> Hello,
>
> I have a central log server, many of them, using rsyslog to aggregate logs
> from remote servers. Everything works great but I have a new challenge and
> am hoping for some recommendations.
>
> I have a number of AWS auto-scaling groups where compute resources are
> dynamically scaled up and down. Each of these will have a custom rsyslog
> configuration pulled from the AWS AMI.
>
> These dynamic resources are not added to DNS due to their dynamic nature so
> they will not have DNS assigned FQDNs.
>
> Because of the lack of a hostname, my central log server is getting only
> IP. I aggregate based on FROMHOST-FROMHOST-IP.
>
> So what I'm seeing today looks like '10.38.134.77-10.38.134.77' where I
> want to see ause1oagbtst03.mydomain.com-10.41.102.168
>
> What I'd want to do is have easy resource send using the same hostname and
> current IP. This later will allow me to aggregate all resources by name.
>
> I did not see any way of affecting the FROMHOST information unless, on the
> collector, I have rules based on IP address which isn't optimal given the
> dynamic nature of the IPs changing.
>
> Any suggestion is appreciated.
>
> *Scott Slattery*
>
> *Sr. Enterprise/Cloud Architect*
>
> *Cloud, Compute, Information & Architecture Team*
>
> motorolasolutions.com
>
> *O: 602.529.8226*
>
> *E*: Scott.Slattery@MotorolaSolutions.com
>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
Thanks, David, I was hoping this was possible. Since the compute resources
are dynamic, using any sort of local /etc/hosts would be impossible since
the IP are unpredictable. Can you point me to how I would do this on the
client-server?

Thanks

*Scott Slattery*

*Sr. Enterprise/Cloud Architect*

*Cloud, Compute, Information & Architecture Team*

motorolasolutions.com

*O: 602.529.8226*

*E*: Scott.Slattery@MotorolaSolutions.com




On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm> wrote:

> fromhost is the result of a name lookup of fromhost-ip. On the receiver,
> you can
> control this with your name resolution (DNS, /etc/hosts, other mechanisms)
>
> but a better option would probably be to set the hostname on the sender.
> The
> hostname field in the message is under the full control of the sender.
>
> David Lang
>
> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
>
> > Date: Tue, 16 Nov 2021 14:56:09 -0700
> > From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> > To: rsyslog-users <rsyslog@lists.adiscon.com>
> > Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
> > Subject: [rsyslog] FROMHOST missing on central log collector
> >
> > Hello,
> >
> > I have a central log server, many of them, using rsyslog to aggregate
> logs
> > from remote servers. Everything works great but I have a new challenge
> and
> > am hoping for some recommendations.
> >
> > I have a number of AWS auto-scaling groups where compute resources are
> > dynamically scaled up and down. Each of these will have a custom rsyslog
> > configuration pulled from the AWS AMI.
> >
> > These dynamic resources are not added to DNS due to their dynamic nature
> so
> > they will not have DNS assigned FQDNs.
> >
> > Because of the lack of a hostname, my central log server is getting only
> > IP. I aggregate based on FROMHOST-FROMHOST-IP.
> >
> > So what I'm seeing today looks like '10.38.134.77-10.38.134.77' where I
> > want to see ause1oagbtst03.mydomain.com-10.41.102.168
> >
> > What I'd want to do is have easy resource send using the same hostname
> and
> > current IP. This later will allow me to aggregate all resources by name.
> >
> > I did not see any way of affecting the FROMHOST information unless, on
> the
> > collector, I have rules based on IP address which isn't optimal given the
> > dynamic nature of the IPs changing.
> >
> > Any suggestion is appreciated.
> >
> > *Scott Slattery*
> >
> > *Sr. Enterprise/Cloud Architect*
> >
> > *Cloud, Compute, Information & Architecture Team*
> >
> > motorolasolutions.com
> >
> > *O: 602.529.8226*
> >
> > *E*: Scott.Slattery@MotorolaSolutions.com
> >
> >
>

--


*For more information on how and why we collect your personal
information, please visit our Privacy Policy
<https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement>.*
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
Linux has a rather sophisticated mechanism for plugging in arbitrary ways of
doing name resolution. DNS has 'won' but hitorically there have been many other
options. Research nsswitch (/etc/nsswitch.conf) and see if there is something
that you can leverage.

or, if you can set the hostname of the resources as they are created to be some
predicatable pattern rather than the AWS default of IP based, you can then make
your logic use that. (This is the approach I would look into). What mechanism
this will be will depend on how you are configuring/provisioning the systems.

David Lang



On Tue, 16 Nov 2021, Scott Slattery wrote:

> Date: Tue, 16 Nov 2021 15:14:51 -0700
> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> To: David Lang <david@lang.hm>
> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>
> Thanks, David, I was hoping this was possible. Since the compute resources
> are dynamic, using any sort of local /etc/hosts would be impossible since
> the IP are unpredictable. Can you point me to how I would do this on the
> client-server?
>
> Thanks
>
> *Scott Slattery*
>
> *Sr. Enterprise/Cloud Architect*
>
> *Cloud, Compute, Information & Architecture Team*
>
> motorolasolutions.com
>
> *O: 602.529.8226*
>
> *E*: Scott.Slattery@MotorolaSolutions.com
>
>
>
>
> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm> wrote:
>
>> fromhost is the result of a name lookup of fromhost-ip. On the receiver,
>> you can
>> control this with your name resolution (DNS, /etc/hosts, other mechanisms)
>>
>> but a better option would probably be to set the hostname on the sender.
>> The
>> hostname field in the message is under the full control of the sender.
>>
>> David Lang
>>
>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
>>
>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
>>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
>>> Subject: [rsyslog] FROMHOST missing on central log collector
>>>
>>> Hello,
>>>
>>> I have a central log server, many of them, using rsyslog to aggregate
>> logs
>>> from remote servers. Everything works great but I have a new challenge
>> and
>>> am hoping for some recommendations.
>>>
>>> I have a number of AWS auto-scaling groups where compute resources are
>>> dynamically scaled up and down. Each of these will have a custom rsyslog
>>> configuration pulled from the AWS AMI.
>>>
>>> These dynamic resources are not added to DNS due to their dynamic nature
>> so
>>> they will not have DNS assigned FQDNs.
>>>
>>> Because of the lack of a hostname, my central log server is getting only
>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
>>>
>>> So what I'm seeing today looks like '10.38.134.77-10.38.134.77' where I
>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
>>>
>>> What I'd want to do is have easy resource send using the same hostname
>> and
>>> current IP. This later will allow me to aggregate all resources by name.
>>>
>>> I did not see any way of affecting the FROMHOST information unless, on
>> the
>>> collector, I have rules based on IP address which isn't optimal given the
>>> dynamic nature of the IPs changing.
>>>
>>> Any suggestion is appreciated.
>>>
>>> *Scott Slattery*
>>>
>>> *Sr. Enterprise/Cloud Architect*
>>>
>>> *Cloud, Compute, Information & Architecture Team*
>>>
>>> motorolasolutions.com
>>>
>>> *O: 602.529.8226*
>>>
>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>
>>>
>>
>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
Thanks David, I could be wrong but the resolution seems to be happening at
the log collection server, not the client end. Given this, I'm not sure
anything outside of rsyslog on the client would affect what the receiving
collection server is seeing.

My hope was that this could be affected by RSYSLOG on the client device but
perhaps not. I'll also look into AWS to see if a dynamically created
compute resource can automatically be registered with DNS.

If anything else comes to mind, let me know. As always, I appreciate your
feedback.

*Scott Slattery*

*Sr. Enterprise/Cloud Architect*

*Cloud, Compute, Information & Architecture Team*

motorolasolutions.com

*O: 602.529.8226*

*E*: Scott.Slattery@MotorolaSolutions.com




On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm> wrote:

> Linux has a rather sophisticated mechanism for plugging in arbitrary ways
> of
> doing name resolution. DNS has 'won' but hitorically there have been many
> other
> options. Research nsswitch (/etc/nsswitch.conf) and see if there is
> something
> that you can leverage.
>
> or, if you can set the hostname of the resources as they are created to be
> some
> predicatable pattern rather than the AWS default of IP based, you can then
> make
> your logic use that. (This is the approach I would look into). What
> mechanism
> this will be will depend on how you are configuring/provisioning the
> systems.
>
> David Lang
>
>
>
> On Tue, 16 Nov 2021, Scott Slattery wrote:
>
> > Date: Tue, 16 Nov 2021 15:14:51 -0700
> > From: Scott Slattery <scott.slattery@motorolasolutions.com>
> > To: David Lang <david@lang.hm>
> > Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> > Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >
> > Thanks, David, I was hoping this was possible. Since the compute
> resources
> > are dynamic, using any sort of local /etc/hosts would be impossible since
> > the IP are unpredictable. Can you point me to how I would do this on the
> > client-server?
> >
> > Thanks
> >
> > *Scott Slattery*
> >
> > *Sr. Enterprise/Cloud Architect*
> >
> > *Cloud, Compute, Information & Architecture Team*
> >
> > motorolasolutions.com
> >
> > *O: 602.529.8226*
> >
> > *E*: Scott.Slattery@MotorolaSolutions.com
> >
> >
> >
> >
> > On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm> wrote:
> >
> >> fromhost is the result of a name lookup of fromhost-ip. On the receiver,
> >> you can
> >> control this with your name resolution (DNS, /etc/hosts, other
> mechanisms)
> >>
> >> but a better option would probably be to set the hostname on the sender.
> >> The
> >> hostname field in the message is under the full control of the sender.
> >>
> >> David Lang
> >>
> >> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
> >>
> >>> Date: Tue, 16 Nov 2021 14:56:09 -0700
> >>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>> To: rsyslog-users <rsyslog@lists.adiscon.com>
> >>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>> Subject: [rsyslog] FROMHOST missing on central log collector
> >>>
> >>> Hello,
> >>>
> >>> I have a central log server, many of them, using rsyslog to aggregate
> >> logs
> >>> from remote servers. Everything works great but I have a new challenge
> >> and
> >>> am hoping for some recommendations.
> >>>
> >>> I have a number of AWS auto-scaling groups where compute resources are
> >>> dynamically scaled up and down. Each of these will have a custom
> rsyslog
> >>> configuration pulled from the AWS AMI.
> >>>
> >>> These dynamic resources are not added to DNS due to their dynamic
> nature
> >> so
> >>> they will not have DNS assigned FQDNs.
> >>>
> >>> Because of the lack of a hostname, my central log server is getting
> only
> >>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
> >>>
> >>> So what I'm seeing today looks like '10.38.134.77-10.38.134.77' where I
> >>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
> >>>
> >>> What I'd want to do is have easy resource send using the same hostname
> >> and
> >>> current IP. This later will allow me to aggregate all resources by
> name.
> >>>
> >>> I did not see any way of affecting the FROMHOST information unless, on
> >> the
> >>> collector, I have rules based on IP address which isn't optimal given
> the
> >>> dynamic nature of the IPs changing.
> >>>
> >>> Any suggestion is appreciated.
> >>>
> >>> *Scott Slattery*
> >>>
> >>> *Sr. Enterprise/Cloud Architect*
> >>>
> >>> *Cloud, Compute, Information & Architecture Team*
> >>>
> >>> motorolasolutions.com
> >>>
> >>> *O: 602.529.8226*
> >>>
> >>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>
> >>>
> >>
> >
> >
>

--


*For more information on how and why we collect your personal
information, please visit our Privacy Policy
<https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement>.*
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
the translation from fromhost-ip to fromhost is done at the collector, but the
sender sets the hostname field. If you can trust that hostname was set
correctly, there is no reason to use fromhost

David Lang

On Tue, 16 Nov 2021, Scott Slattery wrote:

> Date: Tue, 16 Nov 2021 16:53:19 -0700
> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> To: David Lang <david@lang.hm>
> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>
> Thanks David, I could be wrong but the resolution seems to be happening at
> the log collection server, not the client end. Given this, I'm not sure
> anything outside of rsyslog on the client would affect what the receiving
> collection server is seeing.
>
> My hope was that this could be affected by RSYSLOG on the client device but
> perhaps not. I'll also look into AWS to see if a dynamically created
> compute resource can automatically be registered with DNS.
>
> If anything else comes to mind, let me know. As always, I appreciate your
> feedback.
>
> *Scott Slattery*
>
> *Sr. Enterprise/Cloud Architect*
>
> *Cloud, Compute, Information & Architecture Team*
>
> motorolasolutions.com
>
> *O: 602.529.8226*
>
> *E*: Scott.Slattery@MotorolaSolutions.com
>
>
>
>
> On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm> wrote:
>
>> Linux has a rather sophisticated mechanism for plugging in arbitrary ways
>> of
>> doing name resolution. DNS has 'won' but hitorically there have been many
>> other
>> options. Research nsswitch (/etc/nsswitch.conf) and see if there is
>> something
>> that you can leverage.
>>
>> or, if you can set the hostname of the resources as they are created to be
>> some
>> predicatable pattern rather than the AWS default of IP based, you can then
>> make
>> your logic use that. (This is the approach I would look into). What
>> mechanism
>> this will be will depend on how you are configuring/provisioning the
>> systems.
>>
>> David Lang
>>
>>
>>
>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>
>>> Date: Tue, 16 Nov 2021 15:14:51 -0700
>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>> To: David Lang <david@lang.hm>
>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>
>>> Thanks, David, I was hoping this was possible. Since the compute
>> resources
>>> are dynamic, using any sort of local /etc/hosts would be impossible since
>>> the IP are unpredictable. Can you point me to how I would do this on the
>>> client-server?
>>>
>>> Thanks
>>>
>>> *Scott Slattery*
>>>
>>> *Sr. Enterprise/Cloud Architect*
>>>
>>> *Cloud, Compute, Information & Architecture Team*
>>>
>>> motorolasolutions.com
>>>
>>> *O: 602.529.8226*
>>>
>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>
>>>
>>>
>>>
>>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm> wrote:
>>>
>>>> fromhost is the result of a name lookup of fromhost-ip. On the receiver,
>>>> you can
>>>> control this with your name resolution (DNS, /etc/hosts, other
>> mechanisms)
>>>>
>>>> but a better option would probably be to set the hostname on the sender.
>>>> The
>>>> hostname field in the message is under the full control of the sender.
>>>>
>>>> David Lang
>>>>
>>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
>>>>
>>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
>>>>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>> Subject: [rsyslog] FROMHOST missing on central log collector
>>>>>
>>>>> Hello,
>>>>>
>>>>> I have a central log server, many of them, using rsyslog to aggregate
>>>> logs
>>>>> from remote servers. Everything works great but I have a new challenge
>>>> and
>>>>> am hoping for some recommendations.
>>>>>
>>>>> I have a number of AWS auto-scaling groups where compute resources are
>>>>> dynamically scaled up and down. Each of these will have a custom
>> rsyslog
>>>>> configuration pulled from the AWS AMI.
>>>>>
>>>>> These dynamic resources are not added to DNS due to their dynamic
>> nature
>>>> so
>>>>> they will not have DNS assigned FQDNs.
>>>>>
>>>>> Because of the lack of a hostname, my central log server is getting
>> only
>>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
>>>>>
>>>>> So what I'm seeing today looks like '10.38.134.77-10.38.134.77' where I
>>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
>>>>>
>>>>> What I'd want to do is have easy resource send using the same hostname
>>>> and
>>>>> current IP. This later will allow me to aggregate all resources by
>> name.
>>>>>
>>>>> I did not see any way of affecting the FROMHOST information unless, on
>>>> the
>>>>> collector, I have rules based on IP address which isn't optimal given
>> the
>>>>> dynamic nature of the IPs changing.
>>>>>
>>>>> Any suggestion is appreciated.
>>>>>
>>>>> *Scott Slattery*
>>>>>
>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>
>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>
>>>>> motorolasolutions.com
>>>>>
>>>>> *O: 602.529.8226*
>>>>>
>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
My follow-on question woudl be how do I set the hostname at the client end?
Other than what's in /etc/hosts, /etc/hostname, etc. I don't know how else
I would affect the log being sent to ensure it's going over.

*Scott Slattery*

*Sr. Enterprise/Cloud Architect*

*Cloud, Compute, Information & Architecture Team*

motorolasolutions.com

*O: 602.529.8226*

*E*: Scott.Slattery@MotorolaSolutions.com




On Tue, Nov 16, 2021 at 4:55 PM David Lang <david@lang.hm> wrote:

> the translation from fromhost-ip to fromhost is done at the collector, but
> the
> sender sets the hostname field. If you can trust that hostname was set
> correctly, there is no reason to use fromhost
>
> David Lang
>
> On Tue, 16 Nov 2021, Scott Slattery wrote:
>
> > Date: Tue, 16 Nov 2021 16:53:19 -0700
> > From: Scott Slattery <scott.slattery@motorolasolutions.com>
> > To: David Lang <david@lang.hm>
> > Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> > Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >
> > Thanks David, I could be wrong but the resolution seems to be happening
> at
> > the log collection server, not the client end. Given this, I'm not sure
> > anything outside of rsyslog on the client would affect what the receiving
> > collection server is seeing.
> >
> > My hope was that this could be affected by RSYSLOG on the client device
> but
> > perhaps not. I'll also look into AWS to see if a dynamically created
> > compute resource can automatically be registered with DNS.
> >
> > If anything else comes to mind, let me know. As always, I appreciate your
> > feedback.
> >
> > *Scott Slattery*
> >
> > *Sr. Enterprise/Cloud Architect*
> >
> > *Cloud, Compute, Information & Architecture Team*
> >
> > motorolasolutions.com
> >
> > *O: 602.529.8226*
> >
> > *E*: Scott.Slattery@MotorolaSolutions.com
> >
> >
> >
> >
> > On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm> wrote:
> >
> >> Linux has a rather sophisticated mechanism for plugging in arbitrary
> ways
> >> of
> >> doing name resolution. DNS has 'won' but hitorically there have been
> many
> >> other
> >> options. Research nsswitch (/etc/nsswitch.conf) and see if there is
> >> something
> >> that you can leverage.
> >>
> >> or, if you can set the hostname of the resources as they are created to
> be
> >> some
> >> predicatable pattern rather than the AWS default of IP based, you can
> then
> >> make
> >> your logic use that. (This is the approach I would look into). What
> >> mechanism
> >> this will be will depend on how you are configuring/provisioning the
> >> systems.
> >>
> >> David Lang
> >>
> >>
> >>
> >> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>
> >>> Date: Tue, 16 Nov 2021 15:14:51 -0700
> >>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>> To: David Lang <david@lang.hm>
> >>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>
> >>> Thanks, David, I was hoping this was possible. Since the compute
> >> resources
> >>> are dynamic, using any sort of local /etc/hosts would be impossible
> since
> >>> the IP are unpredictable. Can you point me to how I would do this on
> the
> >>> client-server?
> >>>
> >>> Thanks
> >>>
> >>> *Scott Slattery*
> >>>
> >>> *Sr. Enterprise/Cloud Architect*
> >>>
> >>> *Cloud, Compute, Information & Architecture Team*
> >>>
> >>> motorolasolutions.com
> >>>
> >>> *O: 602.529.8226*
> >>>
> >>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>
> >>>
> >>>
> >>>
> >>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm> wrote:
> >>>
> >>>> fromhost is the result of a name lookup of fromhost-ip. On the
> receiver,
> >>>> you can
> >>>> control this with your name resolution (DNS, /etc/hosts, other
> >> mechanisms)
> >>>>
> >>>> but a better option would probably be to set the hostname on the
> sender.
> >>>> The
> >>>> hostname field in the message is under the full control of the sender.
> >>>>
> >>>> David Lang
> >>>>
> >>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
> >>>>
> >>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
> >>>>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
> >>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>> Subject: [rsyslog] FROMHOST missing on central log collector
> >>>>>
> >>>>> Hello,
> >>>>>
> >>>>> I have a central log server, many of them, using rsyslog to aggregate
> >>>> logs
> >>>>> from remote servers. Everything works great but I have a new
> challenge
> >>>> and
> >>>>> am hoping for some recommendations.
> >>>>>
> >>>>> I have a number of AWS auto-scaling groups where compute resources
> are
> >>>>> dynamically scaled up and down. Each of these will have a custom
> >> rsyslog
> >>>>> configuration pulled from the AWS AMI.
> >>>>>
> >>>>> These dynamic resources are not added to DNS due to their dynamic
> >> nature
> >>>> so
> >>>>> they will not have DNS assigned FQDNs.
> >>>>>
> >>>>> Because of the lack of a hostname, my central log server is getting
> >> only
> >>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
> >>>>>
> >>>>> So what I'm seeing today looks like '10.38.134.77-10.38.134.77'
> where I
> >>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
> >>>>>
> >>>>> What I'd want to do is have easy resource send using the same
> hostname
> >>>> and
> >>>>> current IP. This later will allow me to aggregate all resources by
> >> name.
> >>>>>
> >>>>> I did not see any way of affecting the FROMHOST information unless,
> on
> >>>> the
> >>>>> collector, I have rules based on IP address which isn't optimal given
> >> the
> >>>>> dynamic nature of the IPs changing.
> >>>>>
> >>>>> Any suggestion is appreciated.
> >>>>>
> >>>>> *Scott Slattery*
> >>>>>
> >>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>
> >>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>
> >>>>> motorolasolutions.com
> >>>>>
> >>>>> *O: 602.529.8226*
> >>>>>
> >>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>
> >
> >
>

--


*For more information on how and why we collect your personal
information, please visit our Privacy Policy
<https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement>.*
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
the hostname command will let you set the hostname (you want to do that before
you start rsyslog). I would expect that the orcastration tool you use to create
the systems will have some 'correct for that tool' way to set the hostname as it
starts the instance (sorry I can't provide more specifics, if you can mention
what you are using, possibly someone else can chime in on the best way to set
the hostname with that tool)

David Lang

On Tue, 16 Nov 2021, Scott Slattery wrote:

> Date: Tue, 16 Nov 2021 16:59:17 -0700
> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> To: David Lang <david@lang.hm>
> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>
> My follow-on question woudl be how do I set the hostname at the client end?
> Other than what's in /etc/hosts, /etc/hostname, etc. I don't know how else
> I would affect the log being sent to ensure it's going over.
>
> *Scott Slattery*
>
> *Sr. Enterprise/Cloud Architect*
>
> *Cloud, Compute, Information & Architecture Team*
>
> motorolasolutions.com
>
> *O: 602.529.8226*
>
> *E*: Scott.Slattery@MotorolaSolutions.com
>
>
>
>
> On Tue, Nov 16, 2021 at 4:55 PM David Lang <david@lang.hm> wrote:
>
>> the translation from fromhost-ip to fromhost is done at the collector, but
>> the
>> sender sets the hostname field. If you can trust that hostname was set
>> correctly, there is no reason to use fromhost
>>
>> David Lang
>>
>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>
>>> Date: Tue, 16 Nov 2021 16:53:19 -0700
>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>> To: David Lang <david@lang.hm>
>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>
>>> Thanks David, I could be wrong but the resolution seems to be happening
>> at
>>> the log collection server, not the client end. Given this, I'm not sure
>>> anything outside of rsyslog on the client would affect what the receiving
>>> collection server is seeing.
>>>
>>> My hope was that this could be affected by RSYSLOG on the client device
>> but
>>> perhaps not. I'll also look into AWS to see if a dynamically created
>>> compute resource can automatically be registered with DNS.
>>>
>>> If anything else comes to mind, let me know. As always, I appreciate your
>>> feedback.
>>>
>>> *Scott Slattery*
>>>
>>> *Sr. Enterprise/Cloud Architect*
>>>
>>> *Cloud, Compute, Information & Architecture Team*
>>>
>>> motorolasolutions.com
>>>
>>> *O: 602.529.8226*
>>>
>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>
>>>
>>>
>>>
>>> On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm> wrote:
>>>
>>>> Linux has a rather sophisticated mechanism for plugging in arbitrary
>> ways
>>>> of
>>>> doing name resolution. DNS has 'won' but hitorically there have been
>> many
>>>> other
>>>> options. Research nsswitch (/etc/nsswitch.conf) and see if there is
>>>> something
>>>> that you can leverage.
>>>>
>>>> or, if you can set the hostname of the resources as they are created to
>> be
>>>> some
>>>> predicatable pattern rather than the AWS default of IP based, you can
>> then
>>>> make
>>>> your logic use that. (This is the approach I would look into). What
>>>> mechanism
>>>> this will be will depend on how you are configuring/provisioning the
>>>> systems.
>>>>
>>>> David Lang
>>>>
>>>>
>>>>
>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>>>
>>>>> Date: Tue, 16 Nov 2021 15:14:51 -0700
>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>> To: David Lang <david@lang.hm>
>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>>>
>>>>> Thanks, David, I was hoping this was possible. Since the compute
>>>> resources
>>>>> are dynamic, using any sort of local /etc/hosts would be impossible
>> since
>>>>> the IP are unpredictable. Can you point me to how I would do this on
>> the
>>>>> client-server?
>>>>>
>>>>> Thanks
>>>>>
>>>>> *Scott Slattery*
>>>>>
>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>
>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>
>>>>> motorolasolutions.com
>>>>>
>>>>> *O: 602.529.8226*
>>>>>
>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm> wrote:
>>>>>
>>>>>> fromhost is the result of a name lookup of fromhost-ip. On the
>> receiver,
>>>>>> you can
>>>>>> control this with your name resolution (DNS, /etc/hosts, other
>>>> mechanisms)
>>>>>>
>>>>>> but a better option would probably be to set the hostname on the
>> sender.
>>>>>> The
>>>>>> hostname field in the message is under the full control of the sender.
>>>>>>
>>>>>> David Lang
>>>>>>
>>>>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
>>>>>>
>>>>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
>>>>>>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>>>> Subject: [rsyslog] FROMHOST missing on central log collector
>>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> I have a central log server, many of them, using rsyslog to aggregate
>>>>>> logs
>>>>>>> from remote servers. Everything works great but I have a new
>> challenge
>>>>>> and
>>>>>>> am hoping for some recommendations.
>>>>>>>
>>>>>>> I have a number of AWS auto-scaling groups where compute resources
>> are
>>>>>>> dynamically scaled up and down. Each of these will have a custom
>>>> rsyslog
>>>>>>> configuration pulled from the AWS AMI.
>>>>>>>
>>>>>>> These dynamic resources are not added to DNS due to their dynamic
>>>> nature
>>>>>> so
>>>>>>> they will not have DNS assigned FQDNs.
>>>>>>>
>>>>>>> Because of the lack of a hostname, my central log server is getting
>>>> only
>>>>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
>>>>>>>
>>>>>>> So what I'm seeing today looks like '10.38.134.77-10.38.134.77'
>> where I
>>>>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
>>>>>>>
>>>>>>> What I'd want to do is have easy resource send using the same
>> hostname
>>>>>> and
>>>>>>> current IP. This later will allow me to aggregate all resources by
>>>> name.
>>>>>>>
>>>>>>> I did not see any way of affecting the FROMHOST information unless,
>> on
>>>>>> the
>>>>>>> collector, I have rules based on IP address which isn't optimal given
>>>> the
>>>>>>> dynamic nature of the IPs changing.
>>>>>>>
>>>>>>> Any suggestion is appreciated.
>>>>>>>
>>>>>>> *Scott Slattery*
>>>>>>>
>>>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>>>
>>>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>>>
>>>>>>> motorolasolutions.com
>>>>>>>
>>>>>>> *O: 602.529.8226*
>>>>>>>
>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
Thanks David, the hostname is currently set in the AMI (Amazon Master
Image) which is the source image for all instances that are dynamically
created and I can verify that, if you login to one of these dynamic
instances, the hostname is in fact set correctly.

The issue doesn't seem particularly related to what is set in
/etc/hostname, /etc/hosts, or what was set using 'hostname' command. I
think you can see this is the source of my frustration. It appears the
central log collector relies only on DNS resolution unless there's some
hidden magic inside RSYSLOG to force the sent logs to include a host header
(vs DNS).

I don't want to continue wasting your time but again, it is much
appreciated. I'll look into some way of dynamically adding these hosts to
DNS in AWS Route53. It appears rsyslog simply can't do what I'm after.


*Scott Slattery*

*Sr. Enterprise/Cloud Architect*

*Cloud, Compute, Information & Architecture Team*

motorolasolutions.com

*O: 602.529.8226*

*E*: Scott.Slattery@MotorolaSolutions.com




On Tue, Nov 16, 2021 at 5:02 PM David Lang <david@lang.hm> wrote:

> the hostname command will let you set the hostname (you want to do that
> before
> you start rsyslog). I would expect that the orcastration tool you use to
> create
> the systems will have some 'correct for that tool' way to set the hostname
> as it
> starts the instance (sorry I can't provide more specifics, if you can
> mention
> what you are using, possibly someone else can chime in on the best way to
> set
> the hostname with that tool)
>
> David Lang
>
> On Tue, 16 Nov 2021, Scott Slattery wrote:
>
> > Date: Tue, 16 Nov 2021 16:59:17 -0700
> > From: Scott Slattery <scott.slattery@motorolasolutions.com>
> > To: David Lang <david@lang.hm>
> > Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> > Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >
> > My follow-on question woudl be how do I set the hostname at the client
> end?
> > Other than what's in /etc/hosts, /etc/hostname, etc. I don't know how
> else
> > I would affect the log being sent to ensure it's going over.
> >
> > *Scott Slattery*
> >
> > *Sr. Enterprise/Cloud Architect*
> >
> > *Cloud, Compute, Information & Architecture Team*
> >
> > motorolasolutions.com
> >
> > *O: 602.529.8226*
> >
> > *E*: Scott.Slattery@MotorolaSolutions.com
> >
> >
> >
> >
> > On Tue, Nov 16, 2021 at 4:55 PM David Lang <david@lang.hm> wrote:
> >
> >> the translation from fromhost-ip to fromhost is done at the collector,
> but
> >> the
> >> sender sets the hostname field. If you can trust that hostname was set
> >> correctly, there is no reason to use fromhost
> >>
> >> David Lang
> >>
> >> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>
> >>> Date: Tue, 16 Nov 2021 16:53:19 -0700
> >>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>> To: David Lang <david@lang.hm>
> >>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>
> >>> Thanks David, I could be wrong but the resolution seems to be happening
> >> at
> >>> the log collection server, not the client end. Given this, I'm not sure
> >>> anything outside of rsyslog on the client would affect what the
> receiving
> >>> collection server is seeing.
> >>>
> >>> My hope was that this could be affected by RSYSLOG on the client device
> >> but
> >>> perhaps not. I'll also look into AWS to see if a dynamically created
> >>> compute resource can automatically be registered with DNS.
> >>>
> >>> If anything else comes to mind, let me know. As always, I appreciate
> your
> >>> feedback.
> >>>
> >>> *Scott Slattery*
> >>>
> >>> *Sr. Enterprise/Cloud Architect*
> >>>
> >>> *Cloud, Compute, Information & Architecture Team*
> >>>
> >>> motorolasolutions.com
> >>>
> >>> *O: 602.529.8226*
> >>>
> >>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>
> >>>
> >>>
> >>>
> >>> On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm> wrote:
> >>>
> >>>> Linux has a rather sophisticated mechanism for plugging in arbitrary
> >> ways
> >>>> of
> >>>> doing name resolution. DNS has 'won' but hitorically there have been
> >> many
> >>>> other
> >>>> options. Research nsswitch (/etc/nsswitch.conf) and see if there is
> >>>> something
> >>>> that you can leverage.
> >>>>
> >>>> or, if you can set the hostname of the resources as they are created
> to
> >> be
> >>>> some
> >>>> predicatable pattern rather than the AWS default of IP based, you can
> >> then
> >>>> make
> >>>> your logic use that. (This is the approach I would look into). What
> >>>> mechanism
> >>>> this will be will depend on how you are configuring/provisioning the
> >>>> systems.
> >>>>
> >>>> David Lang
> >>>>
> >>>>
> >>>>
> >>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>>>
> >>>>> Date: Tue, 16 Nov 2021 15:14:51 -0700
> >>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>> To: David Lang <david@lang.hm>
> >>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>>>
> >>>>> Thanks, David, I was hoping this was possible. Since the compute
> >>>> resources
> >>>>> are dynamic, using any sort of local /etc/hosts would be impossible
> >> since
> >>>>> the IP are unpredictable. Can you point me to how I would do this on
> >> the
> >>>>> client-server?
> >>>>>
> >>>>> Thanks
> >>>>>
> >>>>> *Scott Slattery*
> >>>>>
> >>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>
> >>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>
> >>>>> motorolasolutions.com
> >>>>>
> >>>>> *O: 602.529.8226*
> >>>>>
> >>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm> wrote:
> >>>>>
> >>>>>> fromhost is the result of a name lookup of fromhost-ip. On the
> >> receiver,
> >>>>>> you can
> >>>>>> control this with your name resolution (DNS, /etc/hosts, other
> >>>> mechanisms)
> >>>>>>
> >>>>>> but a better option would probably be to set the hostname on the
> >> sender.
> >>>>>> The
> >>>>>> hostname field in the message is under the full control of the
> sender.
> >>>>>>
> >>>>>> David Lang
> >>>>>>
> >>>>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
> >>>>>>
> >>>>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
> >>>>>>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
> >>>>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>>>> Subject: [rsyslog] FROMHOST missing on central log collector
> >>>>>>>
> >>>>>>> Hello,
> >>>>>>>
> >>>>>>> I have a central log server, many of them, using rsyslog to
> aggregate
> >>>>>> logs
> >>>>>>> from remote servers. Everything works great but I have a new
> >> challenge
> >>>>>> and
> >>>>>>> am hoping for some recommendations.
> >>>>>>>
> >>>>>>> I have a number of AWS auto-scaling groups where compute resources
> >> are
> >>>>>>> dynamically scaled up and down. Each of these will have a custom
> >>>> rsyslog
> >>>>>>> configuration pulled from the AWS AMI.
> >>>>>>>
> >>>>>>> These dynamic resources are not added to DNS due to their dynamic
> >>>> nature
> >>>>>> so
> >>>>>>> they will not have DNS assigned FQDNs.
> >>>>>>>
> >>>>>>> Because of the lack of a hostname, my central log server is getting
> >>>> only
> >>>>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
> >>>>>>>
> >>>>>>> So what I'm seeing today looks like '10.38.134.77-10.38.134.77'
> >> where I
> >>>>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
> >>>>>>>
> >>>>>>> What I'd want to do is have easy resource send using the same
> >> hostname
> >>>>>> and
> >>>>>>> current IP. This later will allow me to aggregate all resources by
> >>>> name.
> >>>>>>>
> >>>>>>> I did not see any way of affecting the FROMHOST information unless,
> >> on
> >>>>>> the
> >>>>>>> collector, I have rules based on IP address which isn't optimal
> given
> >>>> the
> >>>>>>> dynamic nature of the IPs changing.
> >>>>>>>
> >>>>>>> Any suggestion is appreciated.
> >>>>>>>
> >>>>>>> *Scott Slattery*
> >>>>>>>
> >>>>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>>>
> >>>>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>>>
> >>>>>>> motorolasolutions.com
> >>>>>>>
> >>>>>>> *O: 602.529.8226*
> >>>>>>>
> >>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>
> >
> >
>

--


*For more information on how and why we collect your personal
information, please visit our Privacy Policy
<https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement>.*
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
if you login to one of the systems, you should find that the name returned by
the hostname command should match what you get in the syslog message that is
delivered to your central collector. (if it doesn't, try restarting rsyslog and
see if it changes to match)

then the question becomes what mechansims does AMI provide for customizing the
hostname

a quick google search shows a new hostnamectl command
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-hostname.html
https://www.cyberciti.biz/faq/set-change-hostname-in-amazon-linux-ec2-instance-server/

I know there is a way for you to specify a script to run when an instance is
started, that script can then set things like this. I don't know enough to point
you at specifically how to do that.

David Lang


On Tue, 16 Nov 2021, Scott Slattery wrote:

> Date: Tue, 16 Nov 2021 17:07:47 -0700
> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> To: David Lang <david@lang.hm>
> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>
> Thanks David, the hostname is currently set in the AMI (Amazon Master
> Image) which is the source image for all instances that are dynamically
> created and I can verify that, if you login to one of these dynamic
> instances, the hostname is in fact set correctly.
>
> The issue doesn't seem particularly related to what is set in
> /etc/hostname, /etc/hosts, or what was set using 'hostname' command. I
> think you can see this is the source of my frustration. It appears the
> central log collector relies only on DNS resolution unless there's some
> hidden magic inside RSYSLOG to force the sent logs to include a host header
> (vs DNS).
>
> I don't want to continue wasting your time but again, it is much
> appreciated. I'll look into some way of dynamically adding these hosts to
> DNS in AWS Route53. It appears rsyslog simply can't do what I'm after.
>
>
> *Scott Slattery*
>
> *Sr. Enterprise/Cloud Architect*
>
> *Cloud, Compute, Information & Architecture Team*
>
> motorolasolutions.com
>
> *O: 602.529.8226*
>
> *E*: Scott.Slattery@MotorolaSolutions.com
>
>
>
>
> On Tue, Nov 16, 2021 at 5:02 PM David Lang <david@lang.hm> wrote:
>
>> the hostname command will let you set the hostname (you want to do that
>> before
>> you start rsyslog). I would expect that the orcastration tool you use to
>> create
>> the systems will have some 'correct for that tool' way to set the hostname
>> as it
>> starts the instance (sorry I can't provide more specifics, if you can
>> mention
>> what you are using, possibly someone else can chime in on the best way to
>> set
>> the hostname with that tool)
>>
>> David Lang
>>
>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>
>>> Date: Tue, 16 Nov 2021 16:59:17 -0700
>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>> To: David Lang <david@lang.hm>
>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>
>>> My follow-on question woudl be how do I set the hostname at the client
>> end?
>>> Other than what's in /etc/hosts, /etc/hostname, etc. I don't know how
>> else
>>> I would affect the log being sent to ensure it's going over.
>>>
>>> *Scott Slattery*
>>>
>>> *Sr. Enterprise/Cloud Architect*
>>>
>>> *Cloud, Compute, Information & Architecture Team*
>>>
>>> motorolasolutions.com
>>>
>>> *O: 602.529.8226*
>>>
>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>
>>>
>>>
>>>
>>> On Tue, Nov 16, 2021 at 4:55 PM David Lang <david@lang.hm> wrote:
>>>
>>>> the translation from fromhost-ip to fromhost is done at the collector,
>> but
>>>> the
>>>> sender sets the hostname field. If you can trust that hostname was set
>>>> correctly, there is no reason to use fromhost
>>>>
>>>> David Lang
>>>>
>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>>>
>>>>> Date: Tue, 16 Nov 2021 16:53:19 -0700
>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>> To: David Lang <david@lang.hm>
>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>>>
>>>>> Thanks David, I could be wrong but the resolution seems to be happening
>>>> at
>>>>> the log collection server, not the client end. Given this, I'm not sure
>>>>> anything outside of rsyslog on the client would affect what the
>> receiving
>>>>> collection server is seeing.
>>>>>
>>>>> My hope was that this could be affected by RSYSLOG on the client device
>>>> but
>>>>> perhaps not. I'll also look into AWS to see if a dynamically created
>>>>> compute resource can automatically be registered with DNS.
>>>>>
>>>>> If anything else comes to mind, let me know. As always, I appreciate
>> your
>>>>> feedback.
>>>>>
>>>>> *Scott Slattery*
>>>>>
>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>
>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>
>>>>> motorolasolutions.com
>>>>>
>>>>> *O: 602.529.8226*
>>>>>
>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm> wrote:
>>>>>
>>>>>> Linux has a rather sophisticated mechanism for plugging in arbitrary
>>>> ways
>>>>>> of
>>>>>> doing name resolution. DNS has 'won' but hitorically there have been
>>>> many
>>>>>> other
>>>>>> options. Research nsswitch (/etc/nsswitch.conf) and see if there is
>>>>>> something
>>>>>> that you can leverage.
>>>>>>
>>>>>> or, if you can set the hostname of the resources as they are created
>> to
>>>> be
>>>>>> some
>>>>>> predicatable pattern rather than the AWS default of IP based, you can
>>>> then
>>>>>> make
>>>>>> your logic use that. (This is the approach I would look into). What
>>>>>> mechanism
>>>>>> this will be will depend on how you are configuring/provisioning the
>>>>>> systems.
>>>>>>
>>>>>> David Lang
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>>>>>
>>>>>>> Date: Tue, 16 Nov 2021 15:14:51 -0700
>>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>>>> To: David Lang <david@lang.hm>
>>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>>>>>
>>>>>>> Thanks, David, I was hoping this was possible. Since the compute
>>>>>> resources
>>>>>>> are dynamic, using any sort of local /etc/hosts would be impossible
>>>> since
>>>>>>> the IP are unpredictable. Can you point me to how I would do this on
>>>> the
>>>>>>> client-server?
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> *Scott Slattery*
>>>>>>>
>>>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>>>
>>>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>>>
>>>>>>> motorolasolutions.com
>>>>>>>
>>>>>>> *O: 602.529.8226*
>>>>>>>
>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm> wrote:
>>>>>>>
>>>>>>>> fromhost is the result of a name lookup of fromhost-ip. On the
>>>> receiver,
>>>>>>>> you can
>>>>>>>> control this with your name resolution (DNS, /etc/hosts, other
>>>>>> mechanisms)
>>>>>>>>
>>>>>>>> but a better option would probably be to set the hostname on the
>>>> sender.
>>>>>>>> The
>>>>>>>> hostname field in the message is under the full control of the
>> sender.
>>>>>>>>
>>>>>>>> David Lang
>>>>>>>>
>>>>>>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
>>>>>>>>
>>>>>>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
>>>>>>>>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>>>>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>>>>>> Subject: [rsyslog] FROMHOST missing on central log collector
>>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> I have a central log server, many of them, using rsyslog to
>> aggregate
>>>>>>>> logs
>>>>>>>>> from remote servers. Everything works great but I have a new
>>>> challenge
>>>>>>>> and
>>>>>>>>> am hoping for some recommendations.
>>>>>>>>>
>>>>>>>>> I have a number of AWS auto-scaling groups where compute resources
>>>> are
>>>>>>>>> dynamically scaled up and down. Each of these will have a custom
>>>>>> rsyslog
>>>>>>>>> configuration pulled from the AWS AMI.
>>>>>>>>>
>>>>>>>>> These dynamic resources are not added to DNS due to their dynamic
>>>>>> nature
>>>>>>>> so
>>>>>>>>> they will not have DNS assigned FQDNs.
>>>>>>>>>
>>>>>>>>> Because of the lack of a hostname, my central log server is getting
>>>>>> only
>>>>>>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
>>>>>>>>>
>>>>>>>>> So what I'm seeing today looks like '10.38.134.77-10.38.134.77'
>>>> where I
>>>>>>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
>>>>>>>>>
>>>>>>>>> What I'd want to do is have easy resource send using the same
>>>> hostname
>>>>>>>> and
>>>>>>>>> current IP. This later will allow me to aggregate all resources by
>>>>>> name.
>>>>>>>>>
>>>>>>>>> I did not see any way of affecting the FROMHOST information unless,
>>>> on
>>>>>>>> the
>>>>>>>>> collector, I have rules based on IP address which isn't optimal
>> given
>>>>>> the
>>>>>>>>> dynamic nature of the IPs changing.
>>>>>>>>>
>>>>>>>>> Any suggestion is appreciated.
>>>>>>>>>
>>>>>>>>> *Scott Slattery*
>>>>>>>>>
>>>>>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>>>>>
>>>>>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>>>>>
>>>>>>>>> motorolasolutions.com
>>>>>>>>>
>>>>>>>>> *O: 602.529.8226*
>>>>>>>>>
>>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
Thanks, David, I think you've done more than enough to try and help me on
this. I need to do some reading on Amazon (and the link you shared) to see
what my options are. I agree with you, it's likely workable.

I've confirmed that the results from the 'hostname' command do match so
it's a bit of a mystery why rsyslog doesn't detect this but, i think you're
on the right track, we need to run a post-deployment script to get these
instances registered in Route53.


*Scott Slattery*

*Sr. Enterprise/Cloud Architect*

*Cloud, Compute, Information & Architecture Team*

motorolasolutions.com

*O: 602.529.8226*

*E*: Scott.Slattery@MotorolaSolutions.com




On Tue, Nov 16, 2021 at 5:20 PM David Lang <david@lang.hm> wrote:

> if you login to one of the systems, you should find that the name returned
> by
> the hostname command should match what you get in the syslog message that
> is
> delivered to your central collector. (if it doesn't, try restarting
> rsyslog and
> see if it changes to match)
>
> then the question becomes what mechansims does AMI provide for customizing
> the
> hostname
>
> a quick google search shows a new hostnamectl command
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.aws.amazon.com_AWSEC2_latest_UserGuide_set-2Dhostname.html&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=WR-Pz8svN0d8vqg4ZKSNj2dbxtcngaMJ4iiRXCPpD6c&e=
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.cyberciti.biz_faq_set-2Dchange-2Dhostname-2Din-2Damazon-2Dlinux-2Dec2-2Dinstance-2Dserver_&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=2RI1Khq-fBEBJxckXk9nWDESN8pTJxGiGv6xpsbYhzE&e=
>
> I know there is a way for you to specify a script to run when an instance
> is
> started, that script can then set things like this. I don't know enough to
> point
> you at specifically how to do that.
>
> David Lang
>
>
> On Tue, 16 Nov 2021, Scott Slattery wrote:
>
> > Date: Tue, 16 Nov 2021 17:07:47 -0700
> > From: Scott Slattery <scott.slattery@motorolasolutions.com>
> > To: David Lang <david@lang.hm>
> > Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> > Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >
> > Thanks David, the hostname is currently set in the AMI (Amazon Master
> > Image) which is the source image for all instances that are dynamically
> > created and I can verify that, if you login to one of these dynamic
> > instances, the hostname is in fact set correctly.
> >
> > The issue doesn't seem particularly related to what is set in
> > /etc/hostname, /etc/hosts, or what was set using 'hostname' command. I
> > think you can see this is the source of my frustration. It appears the
> > central log collector relies only on DNS resolution unless there's some
> > hidden magic inside RSYSLOG to force the sent logs to include a host
> header
> > (vs DNS).
> >
> > I don't want to continue wasting your time but again, it is much
> > appreciated. I'll look into some way of dynamically adding these hosts to
> > DNS in AWS Route53. It appears rsyslog simply can't do what I'm after.
> >
> >
> > *Scott Slattery*
> >
> > *Sr. Enterprise/Cloud Architect*
> >
> > *Cloud, Compute, Information & Architecture Team*
> >
> > motorolasolutions.com
> >
> > *O: 602.529.8226*
> >
> > *E*: Scott.Slattery@MotorolaSolutions.com
> >
> >
> >
> >
> > On Tue, Nov 16, 2021 at 5:02 PM David Lang <david@lang.hm> wrote:
> >
> >> the hostname command will let you set the hostname (you want to do that
> >> before
> >> you start rsyslog). I would expect that the orcastration tool you use to
> >> create
> >> the systems will have some 'correct for that tool' way to set the
> hostname
> >> as it
> >> starts the instance (sorry I can't provide more specifics, if you can
> >> mention
> >> what you are using, possibly someone else can chime in on the best way
> to
> >> set
> >> the hostname with that tool)
> >>
> >> David Lang
> >>
> >> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>
> >>> Date: Tue, 16 Nov 2021 16:59:17 -0700
> >>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>> To: David Lang <david@lang.hm>
> >>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>
> >>> My follow-on question woudl be how do I set the hostname at the client
> >> end?
> >>> Other than what's in /etc/hosts, /etc/hostname, etc. I don't know how
> >> else
> >>> I would affect the log being sent to ensure it's going over.
> >>>
> >>> *Scott Slattery*
> >>>
> >>> *Sr. Enterprise/Cloud Architect*
> >>>
> >>> *Cloud, Compute, Information & Architecture Team*
> >>>
> >>> motorolasolutions.com
> >>>
> >>> *O: 602.529.8226*
> >>>
> >>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>
> >>>
> >>>
> >>>
> >>> On Tue, Nov 16, 2021 at 4:55 PM David Lang <david@lang.hm> wrote:
> >>>
> >>>> the translation from fromhost-ip to fromhost is done at the collector,
> >> but
> >>>> the
> >>>> sender sets the hostname field. If you can trust that hostname was set
> >>>> correctly, there is no reason to use fromhost
> >>>>
> >>>> David Lang
> >>>>
> >>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>>>
> >>>>> Date: Tue, 16 Nov 2021 16:53:19 -0700
> >>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>> To: David Lang <david@lang.hm>
> >>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>>>
> >>>>> Thanks David, I could be wrong but the resolution seems to be
> happening
> >>>> at
> >>>>> the log collection server, not the client end. Given this, I'm not
> sure
> >>>>> anything outside of rsyslog on the client would affect what the
> >> receiving
> >>>>> collection server is seeing.
> >>>>>
> >>>>> My hope was that this could be affected by RSYSLOG on the client
> device
> >>>> but
> >>>>> perhaps not. I'll also look into AWS to see if a dynamically created
> >>>>> compute resource can automatically be registered with DNS.
> >>>>>
> >>>>> If anything else comes to mind, let me know. As always, I appreciate
> >> your
> >>>>> feedback.
> >>>>>
> >>>>> *Scott Slattery*
> >>>>>
> >>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>
> >>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>
> >>>>> motorolasolutions.com
> >>>>>
> >>>>> *O: 602.529.8226*
> >>>>>
> >>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm> wrote:
> >>>>>
> >>>>>> Linux has a rather sophisticated mechanism for plugging in arbitrary
> >>>> ways
> >>>>>> of
> >>>>>> doing name resolution. DNS has 'won' but hitorically there have been
> >>>> many
> >>>>>> other
> >>>>>> options. Research nsswitch (/etc/nsswitch.conf) and see if there is
> >>>>>> something
> >>>>>> that you can leverage.
> >>>>>>
> >>>>>> or, if you can set the hostname of the resources as they are created
> >> to
> >>>> be
> >>>>>> some
> >>>>>> predicatable pattern rather than the AWS default of IP based, you
> can
> >>>> then
> >>>>>> make
> >>>>>> your logic use that. (This is the approach I would look into). What
> >>>>>> mechanism
> >>>>>> this will be will depend on how you are configuring/provisioning the
> >>>>>> systems.
> >>>>>>
> >>>>>> David Lang
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>>>>>
> >>>>>>> Date: Tue, 16 Nov 2021 15:14:51 -0700
> >>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>>>> To: David Lang <david@lang.hm>
> >>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>>>>>
> >>>>>>> Thanks, David, I was hoping this was possible. Since the compute
> >>>>>> resources
> >>>>>>> are dynamic, using any sort of local /etc/hosts would be impossible
> >>>> since
> >>>>>>> the IP are unpredictable. Can you point me to how I would do this
> on
> >>>> the
> >>>>>>> client-server?
> >>>>>>>
> >>>>>>> Thanks
> >>>>>>>
> >>>>>>> *Scott Slattery*
> >>>>>>>
> >>>>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>>>
> >>>>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>>>
> >>>>>>> motorolasolutions.com
> >>>>>>>
> >>>>>>> *O: 602.529.8226*
> >>>>>>>
> >>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm> wrote:
> >>>>>>>
> >>>>>>>> fromhost is the result of a name lookup of fromhost-ip. On the
> >>>> receiver,
> >>>>>>>> you can
> >>>>>>>> control this with your name resolution (DNS, /etc/hosts, other
> >>>>>> mechanisms)
> >>>>>>>>
> >>>>>>>> but a better option would probably be to set the hostname on the
> >>>> sender.
> >>>>>>>> The
> >>>>>>>> hostname field in the message is under the full control of the
> >> sender.
> >>>>>>>>
> >>>>>>>> David Lang
> >>>>>>>>
> >>>>>>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
> >>>>>>>>
> >>>>>>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
> >>>>>>>>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
> >>>>>>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>>>>>> Subject: [rsyslog] FROMHOST missing on central log collector
> >>>>>>>>>
> >>>>>>>>> Hello,
> >>>>>>>>>
> >>>>>>>>> I have a central log server, many of them, using rsyslog to
> >> aggregate
> >>>>>>>> logs
> >>>>>>>>> from remote servers. Everything works great but I have a new
> >>>> challenge
> >>>>>>>> and
> >>>>>>>>> am hoping for some recommendations.
> >>>>>>>>>
> >>>>>>>>> I have a number of AWS auto-scaling groups where compute
> resources
> >>>> are
> >>>>>>>>> dynamically scaled up and down. Each of these will have a custom
> >>>>>> rsyslog
> >>>>>>>>> configuration pulled from the AWS AMI.
> >>>>>>>>>
> >>>>>>>>> These dynamic resources are not added to DNS due to their dynamic
> >>>>>> nature
> >>>>>>>> so
> >>>>>>>>> they will not have DNS assigned FQDNs.
> >>>>>>>>>
> >>>>>>>>> Because of the lack of a hostname, my central log server is
> getting
> >>>>>> only
> >>>>>>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
> >>>>>>>>>
> >>>>>>>>> So what I'm seeing today looks like '10.38.134.77-10.38.134.77'
> >>>> where I
> >>>>>>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
> >>>>>>>>>
> >>>>>>>>> What I'd want to do is have easy resource send using the same
> >>>> hostname
> >>>>>>>> and
> >>>>>>>>> current IP. This later will allow me to aggregate all resources
> by
> >>>>>> name.
> >>>>>>>>>
> >>>>>>>>> I did not see any way of affecting the FROMHOST information
> unless,
> >>>> on
> >>>>>>>> the
> >>>>>>>>> collector, I have rules based on IP address which isn't optimal
> >> given
> >>>>>> the
> >>>>>>>>> dynamic nature of the IPs changing.
> >>>>>>>>>
> >>>>>>>>> Any suggestion is appreciated.
> >>>>>>>>>
> >>>>>>>>> *Scott Slattery*
> >>>>>>>>>
> >>>>>>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>>>>>
> >>>>>>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>>>>>
> >>>>>>>>> motorolasolutions.com
> >>>>>>>>>
> >>>>>>>>> *O: 602.529.8226*
> >>>>>>>>>
> >>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>
> >
> >
>

--


*For more information on how and why we collect your personal
information, please visit our Privacy Policy
<https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement>.*
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
Rsyslog looks up the hostname as it starts up, so if something after rsyslog
starts changes the hostname, rsyslog isn't going to notice until you restart
rsyslog.

again, fromhost is a receiver side lookup of the name to match fromhost-ip, so
if hostname is getting set correctly, filter on that instead of on fromhost.

David Lang

On Tue, 16 Nov 2021, Scott Slattery wrote:

> Date: Tue, 16 Nov 2021 17:28:15 -0700
> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> To: David Lang <david@lang.hm>
> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>
> Thanks, David, I think you've done more than enough to try and help me on
> this. I need to do some reading on Amazon (and the link you shared) to see
> what my options are. I agree with you, it's likely workable.
>
> I've confirmed that the results from the 'hostname' command do match so
> it's a bit of a mystery why rsyslog doesn't detect this but, i think you're
> on the right track, we need to run a post-deployment script to get these
> instances registered in Route53.
>
>
> *Scott Slattery*
>
> *Sr. Enterprise/Cloud Architect*
>
> *Cloud, Compute, Information & Architecture Team*
>
> motorolasolutions.com
>
> *O: 602.529.8226*
>
> *E*: Scott.Slattery@MotorolaSolutions.com
>
>
>
>
> On Tue, Nov 16, 2021 at 5:20 PM David Lang <david@lang.hm> wrote:
>
>> if you login to one of the systems, you should find that the name returned
>> by
>> the hostname command should match what you get in the syslog message that
>> is
>> delivered to your central collector. (if it doesn't, try restarting
>> rsyslog and
>> see if it changes to match)
>>
>> then the question becomes what mechansims does AMI provide for customizing
>> the
>> hostname
>>
>> a quick google search shows a new hostnamectl command
>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.aws.amazon.com_AWSEC2_latest_UserGuide_set-2Dhostname.html&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=WR-Pz8svN0d8vqg4ZKSNj2dbxtcngaMJ4iiRXCPpD6c&e=
>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.cyberciti.biz_faq_set-2Dchange-2Dhostname-2Din-2Damazon-2Dlinux-2Dec2-2Dinstance-2Dserver_&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=2RI1Khq-fBEBJxckXk9nWDESN8pTJxGiGv6xpsbYhzE&e=
>>
>> I know there is a way for you to specify a script to run when an instance
>> is
>> started, that script can then set things like this. I don't know enough to
>> point
>> you at specifically how to do that.
>>
>> David Lang
>>
>>
>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>
>>> Date: Tue, 16 Nov 2021 17:07:47 -0700
>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>> To: David Lang <david@lang.hm>
>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>
>>> Thanks David, the hostname is currently set in the AMI (Amazon Master
>>> Image) which is the source image for all instances that are dynamically
>>> created and I can verify that, if you login to one of these dynamic
>>> instances, the hostname is in fact set correctly.
>>>
>>> The issue doesn't seem particularly related to what is set in
>>> /etc/hostname, /etc/hosts, or what was set using 'hostname' command. I
>>> think you can see this is the source of my frustration. It appears the
>>> central log collector relies only on DNS resolution unless there's some
>>> hidden magic inside RSYSLOG to force the sent logs to include a host
>> header
>>> (vs DNS).
>>>
>>> I don't want to continue wasting your time but again, it is much
>>> appreciated. I'll look into some way of dynamically adding these hosts to
>>> DNS in AWS Route53. It appears rsyslog simply can't do what I'm after.
>>>
>>>
>>> *Scott Slattery*
>>>
>>> *Sr. Enterprise/Cloud Architect*
>>>
>>> *Cloud, Compute, Information & Architecture Team*
>>>
>>> motorolasolutions.com
>>>
>>> *O: 602.529.8226*
>>>
>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>
>>>
>>>
>>>
>>> On Tue, Nov 16, 2021 at 5:02 PM David Lang <david@lang.hm> wrote:
>>>
>>>> the hostname command will let you set the hostname (you want to do that
>>>> before
>>>> you start rsyslog). I would expect that the orcastration tool you use to
>>>> create
>>>> the systems will have some 'correct for that tool' way to set the
>> hostname
>>>> as it
>>>> starts the instance (sorry I can't provide more specifics, if you can
>>>> mention
>>>> what you are using, possibly someone else can chime in on the best way
>> to
>>>> set
>>>> the hostname with that tool)
>>>>
>>>> David Lang
>>>>
>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>>>
>>>>> Date: Tue, 16 Nov 2021 16:59:17 -0700
>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>> To: David Lang <david@lang.hm>
>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>>>
>>>>> My follow-on question woudl be how do I set the hostname at the client
>>>> end?
>>>>> Other than what's in /etc/hosts, /etc/hostname, etc. I don't know how
>>>> else
>>>>> I would affect the log being sent to ensure it's going over.
>>>>>
>>>>> *Scott Slattery*
>>>>>
>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>
>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>
>>>>> motorolasolutions.com
>>>>>
>>>>> *O: 602.529.8226*
>>>>>
>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Nov 16, 2021 at 4:55 PM David Lang <david@lang.hm> wrote:
>>>>>
>>>>>> the translation from fromhost-ip to fromhost is done at the collector,
>>>> but
>>>>>> the
>>>>>> sender sets the hostname field. If you can trust that hostname was set
>>>>>> correctly, there is no reason to use fromhost
>>>>>>
>>>>>> David Lang
>>>>>>
>>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>>>>>
>>>>>>> Date: Tue, 16 Nov 2021 16:53:19 -0700
>>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>>>> To: David Lang <david@lang.hm>
>>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>>>>>
>>>>>>> Thanks David, I could be wrong but the resolution seems to be
>> happening
>>>>>> at
>>>>>>> the log collection server, not the client end. Given this, I'm not
>> sure
>>>>>>> anything outside of rsyslog on the client would affect what the
>>>> receiving
>>>>>>> collection server is seeing.
>>>>>>>
>>>>>>> My hope was that this could be affected by RSYSLOG on the client
>> device
>>>>>> but
>>>>>>> perhaps not. I'll also look into AWS to see if a dynamically created
>>>>>>> compute resource can automatically be registered with DNS.
>>>>>>>
>>>>>>> If anything else comes to mind, let me know. As always, I appreciate
>>>> your
>>>>>>> feedback.
>>>>>>>
>>>>>>> *Scott Slattery*
>>>>>>>
>>>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>>>
>>>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>>>
>>>>>>> motorolasolutions.com
>>>>>>>
>>>>>>> *O: 602.529.8226*
>>>>>>>
>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm> wrote:
>>>>>>>
>>>>>>>> Linux has a rather sophisticated mechanism for plugging in arbitrary
>>>>>> ways
>>>>>>>> of
>>>>>>>> doing name resolution. DNS has 'won' but hitorically there have been
>>>>>> many
>>>>>>>> other
>>>>>>>> options. Research nsswitch (/etc/nsswitch.conf) and see if there is
>>>>>>>> something
>>>>>>>> that you can leverage.
>>>>>>>>
>>>>>>>> or, if you can set the hostname of the resources as they are created
>>>> to
>>>>>> be
>>>>>>>> some
>>>>>>>> predicatable pattern rather than the AWS default of IP based, you
>> can
>>>>>> then
>>>>>>>> make
>>>>>>>> your logic use that. (This is the approach I would look into). What
>>>>>>>> mechanism
>>>>>>>> this will be will depend on how you are configuring/provisioning the
>>>>>>>> systems.
>>>>>>>>
>>>>>>>> David Lang
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>>>>>>>
>>>>>>>>> Date: Tue, 16 Nov 2021 15:14:51 -0700
>>>>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>>>>>> To: David Lang <david@lang.hm>
>>>>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>>>>>>>
>>>>>>>>> Thanks, David, I was hoping this was possible. Since the compute
>>>>>>>> resources
>>>>>>>>> are dynamic, using any sort of local /etc/hosts would be impossible
>>>>>> since
>>>>>>>>> the IP are unpredictable. Can you point me to how I would do this
>> on
>>>>>> the
>>>>>>>>> client-server?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> *Scott Slattery*
>>>>>>>>>
>>>>>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>>>>>
>>>>>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>>>>>
>>>>>>>>> motorolasolutions.com
>>>>>>>>>
>>>>>>>>> *O: 602.529.8226*
>>>>>>>>>
>>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm> wrote:
>>>>>>>>>
>>>>>>>>>> fromhost is the result of a name lookup of fromhost-ip. On the
>>>>>> receiver,
>>>>>>>>>> you can
>>>>>>>>>> control this with your name resolution (DNS, /etc/hosts, other
>>>>>>>> mechanisms)
>>>>>>>>>>
>>>>>>>>>> but a better option would probably be to set the hostname on the
>>>>>> sender.
>>>>>>>>>> The
>>>>>>>>>> hostname field in the message is under the full control of the
>>>> sender.
>>>>>>>>>>
>>>>>>>>>> David Lang
>>>>>>>>>>
>>>>>>>>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
>>>>>>>>>>
>>>>>>>>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
>>>>>>>>>>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>>>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>>>>>>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>>>>>>>> Subject: [rsyslog] FROMHOST missing on central log collector
>>>>>>>>>>>
>>>>>>>>>>> Hello,
>>>>>>>>>>>
>>>>>>>>>>> I have a central log server, many of them, using rsyslog to
>>>> aggregate
>>>>>>>>>> logs
>>>>>>>>>>> from remote servers. Everything works great but I have a new
>>>>>> challenge
>>>>>>>>>> and
>>>>>>>>>>> am hoping for some recommendations.
>>>>>>>>>>>
>>>>>>>>>>> I have a number of AWS auto-scaling groups where compute
>> resources
>>>>>> are
>>>>>>>>>>> dynamically scaled up and down. Each of these will have a custom
>>>>>>>> rsyslog
>>>>>>>>>>> configuration pulled from the AWS AMI.
>>>>>>>>>>>
>>>>>>>>>>> These dynamic resources are not added to DNS due to their dynamic
>>>>>>>> nature
>>>>>>>>>> so
>>>>>>>>>>> they will not have DNS assigned FQDNs.
>>>>>>>>>>>
>>>>>>>>>>> Because of the lack of a hostname, my central log server is
>> getting
>>>>>>>> only
>>>>>>>>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
>>>>>>>>>>>
>>>>>>>>>>> So what I'm seeing today looks like '10.38.134.77-10.38.134.77'
>>>>>> where I
>>>>>>>>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
>>>>>>>>>>>
>>>>>>>>>>> What I'd want to do is have easy resource send using the same
>>>>>> hostname
>>>>>>>>>> and
>>>>>>>>>>> current IP. This later will allow me to aggregate all resources
>> by
>>>>>>>> name.
>>>>>>>>>>>
>>>>>>>>>>> I did not see any way of affecting the FROMHOST information
>> unless,
>>>>>> on
>>>>>>>>>> the
>>>>>>>>>>> collector, I have rules based on IP address which isn't optimal
>>>> given
>>>>>>>> the
>>>>>>>>>>> dynamic nature of the IPs changing.
>>>>>>>>>>>
>>>>>>>>>>> Any suggestion is appreciated.
>>>>>>>>>>>
>>>>>>>>>>> *Scott Slattery*
>>>>>>>>>>>
>>>>>>>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>>>>>>>
>>>>>>>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>>>>>>>
>>>>>>>>>>> motorolasolutions.com
>>>>>>>>>>>
>>>>>>>>>>> *O: 602.529.8226*
>>>>>>>>>>>
>>>>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
Hello!

Just a reminder that a hostname field in a syslog message is just a string
sent from sender to collector. So you can craft a custom template with the
hostname field defined as you'd like. Though I'd call this a "fallback" way
of fixing the issue. The right way is to set the proper hostname on a
sender system before rsyslog starts I'd say.

On Wed, 17 Nov 2021 at 08:33, David Lang via rsyslog <
rsyslog@lists.adiscon.com> wrote:

> Rsyslog looks up the hostname as it starts up, so if something after
> rsyslog
> starts changes the hostname, rsyslog isn't going to notice until you
> restart
> rsyslog.
>
> again, fromhost is a receiver side lookup of the name to match
> fromhost-ip, so
> if hostname is getting set correctly, filter on that instead of on
> fromhost.
>
> David Lang
>
> On Tue, 16 Nov 2021, Scott Slattery wrote:
>
> > Date: Tue, 16 Nov 2021 17:28:15 -0700
> > From: Scott Slattery <scott.slattery@motorolasolutions.com>
> > To: David Lang <david@lang.hm>
> > Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> > Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >
> > Thanks, David, I think you've done more than enough to try and help me on
> > this. I need to do some reading on Amazon (and the link you shared) to
> see
> > what my options are. I agree with you, it's likely workable.
> >
> > I've confirmed that the results from the 'hostname' command do match so
> > it's a bit of a mystery why rsyslog doesn't detect this but, i think
> you're
> > on the right track, we need to run a post-deployment script to get these
> > instances registered in Route53.
> >
> >
> > *Scott Slattery*
> >
> > *Sr. Enterprise/Cloud Architect*
> >
> > *Cloud, Compute, Information & Architecture Team*
> >
> > motorolasolutions.com
> >
> > *O: 602.529.8226*
> >
> > *E*: Scott.Slattery@MotorolaSolutions.com
> >
> >
> >
> >
> > On Tue, Nov 16, 2021 at 5:20 PM David Lang <david@lang.hm> wrote:
> >
> >> if you login to one of the systems, you should find that the name
> returned
> >> by
> >> the hostname command should match what you get in the syslog message
> that
> >> is
> >> delivered to your central collector. (if it doesn't, try restarting
> >> rsyslog and
> >> see if it changes to match)
> >>
> >> then the question becomes what mechansims does AMI provide for
> customizing
> >> the
> >> hostname
> >>
> >> a quick google search shows a new hostnamectl command
> >>
> >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.aws.amazon.com_AWSEC2_latest_UserGuide_set-2Dhostname.html&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=WR-Pz8svN0d8vqg4ZKSNj2dbxtcngaMJ4iiRXCPpD6c&e=
> >>
> >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.cyberciti.biz_faq_set-2Dchange-2Dhostname-2Din-2Damazon-2Dlinux-2Dec2-2Dinstance-2Dserver_&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=2RI1Khq-fBEBJxckXk9nWDESN8pTJxGiGv6xpsbYhzE&e=
> >>
> >> I know there is a way for you to specify a script to run when an
> instance
> >> is
> >> started, that script can then set things like this. I don't know enough
> to
> >> point
> >> you at specifically how to do that.
> >>
> >> David Lang
> >>
> >>
> >> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>
> >>> Date: Tue, 16 Nov 2021 17:07:47 -0700
> >>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>> To: David Lang <david@lang.hm>
> >>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>
> >>> Thanks David, the hostname is currently set in the AMI (Amazon Master
> >>> Image) which is the source image for all instances that are dynamically
> >>> created and I can verify that, if you login to one of these dynamic
> >>> instances, the hostname is in fact set correctly.
> >>>
> >>> The issue doesn't seem particularly related to what is set in
> >>> /etc/hostname, /etc/hosts, or what was set using 'hostname' command. I
> >>> think you can see this is the source of my frustration. It appears the
> >>> central log collector relies only on DNS resolution unless there's some
> >>> hidden magic inside RSYSLOG to force the sent logs to include a host
> >> header
> >>> (vs DNS).
> >>>
> >>> I don't want to continue wasting your time but again, it is much
> >>> appreciated. I'll look into some way of dynamically adding these hosts
> to
> >>> DNS in AWS Route53. It appears rsyslog simply can't do what I'm after.
> >>>
> >>>
> >>> *Scott Slattery*
> >>>
> >>> *Sr. Enterprise/Cloud Architect*
> >>>
> >>> *Cloud, Compute, Information & Architecture Team*
> >>>
> >>> motorolasolutions.com
> >>>
> >>> *O: 602.529.8226*
> >>>
> >>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>
> >>>
> >>>
> >>>
> >>> On Tue, Nov 16, 2021 at 5:02 PM David Lang <david@lang.hm> wrote:
> >>>
> >>>> the hostname command will let you set the hostname (you want to do
> that
> >>>> before
> >>>> you start rsyslog). I would expect that the orcastration tool you use
> to
> >>>> create
> >>>> the systems will have some 'correct for that tool' way to set the
> >> hostname
> >>>> as it
> >>>> starts the instance (sorry I can't provide more specifics, if you can
> >>>> mention
> >>>> what you are using, possibly someone else can chime in on the best way
> >> to
> >>>> set
> >>>> the hostname with that tool)
> >>>>
> >>>> David Lang
> >>>>
> >>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>>>
> >>>>> Date: Tue, 16 Nov 2021 16:59:17 -0700
> >>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>> To: David Lang <david@lang.hm>
> >>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>>>
> >>>>> My follow-on question woudl be how do I set the hostname at the
> client
> >>>> end?
> >>>>> Other than what's in /etc/hosts, /etc/hostname, etc. I don't know how
> >>>> else
> >>>>> I would affect the log being sent to ensure it's going over.
> >>>>>
> >>>>> *Scott Slattery*
> >>>>>
> >>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>
> >>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>
> >>>>> motorolasolutions.com
> >>>>>
> >>>>> *O: 602.529.8226*
> >>>>>
> >>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Tue, Nov 16, 2021 at 4:55 PM David Lang <david@lang.hm> wrote:
> >>>>>
> >>>>>> the translation from fromhost-ip to fromhost is done at the
> collector,
> >>>> but
> >>>>>> the
> >>>>>> sender sets the hostname field. If you can trust that hostname was
> set
> >>>>>> correctly, there is no reason to use fromhost
> >>>>>>
> >>>>>> David Lang
> >>>>>>
> >>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>>>>>
> >>>>>>> Date: Tue, 16 Nov 2021 16:53:19 -0700
> >>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>>>> To: David Lang <david@lang.hm>
> >>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>>>>>
> >>>>>>> Thanks David, I could be wrong but the resolution seems to be
> >> happening
> >>>>>> at
> >>>>>>> the log collection server, not the client end. Given this, I'm not
> >> sure
> >>>>>>> anything outside of rsyslog on the client would affect what the
> >>>> receiving
> >>>>>>> collection server is seeing.
> >>>>>>>
> >>>>>>> My hope was that this could be affected by RSYSLOG on the client
> >> device
> >>>>>> but
> >>>>>>> perhaps not. I'll also look into AWS to see if a dynamically
> created
> >>>>>>> compute resource can automatically be registered with DNS.
> >>>>>>>
> >>>>>>> If anything else comes to mind, let me know. As always, I
> appreciate
> >>>> your
> >>>>>>> feedback.
> >>>>>>>
> >>>>>>> *Scott Slattery*
> >>>>>>>
> >>>>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>>>
> >>>>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>>>
> >>>>>>> motorolasolutions.com
> >>>>>>>
> >>>>>>> *O: 602.529.8226*
> >>>>>>>
> >>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm> wrote:
> >>>>>>>
> >>>>>>>> Linux has a rather sophisticated mechanism for plugging in
> arbitrary
> >>>>>> ways
> >>>>>>>> of
> >>>>>>>> doing name resolution. DNS has 'won' but hitorically there have
> been
> >>>>>> many
> >>>>>>>> other
> >>>>>>>> options. Research nsswitch (/etc/nsswitch.conf) and see if there
> is
> >>>>>>>> something
> >>>>>>>> that you can leverage.
> >>>>>>>>
> >>>>>>>> or, if you can set the hostname of the resources as they are
> created
> >>>> to
> >>>>>> be
> >>>>>>>> some
> >>>>>>>> predicatable pattern rather than the AWS default of IP based, you
> >> can
> >>>>>> then
> >>>>>>>> make
> >>>>>>>> your logic use that. (This is the approach I would look into).
> What
> >>>>>>>> mechanism
> >>>>>>>> this will be will depend on how you are configuring/provisioning
> the
> >>>>>>>> systems.
> >>>>>>>>
> >>>>>>>> David Lang
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>>>>>>>
> >>>>>>>>> Date: Tue, 16 Nov 2021 15:14:51 -0700
> >>>>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>>>>>> To: David Lang <david@lang.hm>
> >>>>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>>>>>>>
> >>>>>>>>> Thanks, David, I was hoping this was possible. Since the compute
> >>>>>>>> resources
> >>>>>>>>> are dynamic, using any sort of local /etc/hosts would be
> impossible
> >>>>>> since
> >>>>>>>>> the IP are unpredictable. Can you point me to how I would do this
> >> on
> >>>>>> the
> >>>>>>>>> client-server?
> >>>>>>>>>
> >>>>>>>>> Thanks
> >>>>>>>>>
> >>>>>>>>> *Scott Slattery*
> >>>>>>>>>
> >>>>>>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>>>>>
> >>>>>>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>>>>>
> >>>>>>>>> motorolasolutions.com
> >>>>>>>>>
> >>>>>>>>> *O: 602.529.8226*
> >>>>>>>>>
> >>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm>
> wrote:
> >>>>>>>>>
> >>>>>>>>>> fromhost is the result of a name lookup of fromhost-ip. On the
> >>>>>> receiver,
> >>>>>>>>>> you can
> >>>>>>>>>> control this with your name resolution (DNS, /etc/hosts, other
> >>>>>>>> mechanisms)
> >>>>>>>>>>
> >>>>>>>>>> but a better option would probably be to set the hostname on the
> >>>>>> sender.
> >>>>>>>>>> The
> >>>>>>>>>> hostname field in the message is under the full control of the
> >>>> sender.
> >>>>>>>>>>
> >>>>>>>>>> David Lang
> >>>>>>>>>>
> >>>>>>>>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
> >>>>>>>>>>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>>>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
> >>>>>>>>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>>>>>>>> Subject: [rsyslog] FROMHOST missing on central log collector
> >>>>>>>>>>>
> >>>>>>>>>>> Hello,
> >>>>>>>>>>>
> >>>>>>>>>>> I have a central log server, many of them, using rsyslog to
> >>>> aggregate
> >>>>>>>>>> logs
> >>>>>>>>>>> from remote servers. Everything works great but I have a new
> >>>>>> challenge
> >>>>>>>>>> and
> >>>>>>>>>>> am hoping for some recommendations.
> >>>>>>>>>>>
> >>>>>>>>>>> I have a number of AWS auto-scaling groups where compute
> >> resources
> >>>>>> are
> >>>>>>>>>>> dynamically scaled up and down. Each of these will have a
> custom
> >>>>>>>> rsyslog
> >>>>>>>>>>> configuration pulled from the AWS AMI.
> >>>>>>>>>>>
> >>>>>>>>>>> These dynamic resources are not added to DNS due to their
> dynamic
> >>>>>>>> nature
> >>>>>>>>>> so
> >>>>>>>>>>> they will not have DNS assigned FQDNs.
> >>>>>>>>>>>
> >>>>>>>>>>> Because of the lack of a hostname, my central log server is
> >> getting
> >>>>>>>> only
> >>>>>>>>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
> >>>>>>>>>>>
> >>>>>>>>>>> So what I'm seeing today looks like '10.38.134.77-10.38.134.77'
> >>>>>> where I
> >>>>>>>>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
> >>>>>>>>>>>
> >>>>>>>>>>> What I'd want to do is have easy resource send using the same
> >>>>>> hostname
> >>>>>>>>>> and
> >>>>>>>>>>> current IP. This later will allow me to aggregate all resources
> >> by
> >>>>>>>> name.
> >>>>>>>>>>>
> >>>>>>>>>>> I did not see any way of affecting the FROMHOST information
> >> unless,
> >>>>>> on
> >>>>>>>>>> the
> >>>>>>>>>>> collector, I have rules based on IP address which isn't optimal
> >>>> given
> >>>>>>>> the
> >>>>>>>>>>> dynamic nature of the IPs changing.
> >>>>>>>>>>>
> >>>>>>>>>>> Any suggestion is appreciated.
> >>>>>>>>>>>
> >>>>>>>>>>> *Scott Slattery*
> >>>>>>>>>>>
> >>>>>>>>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>>>>>>>
> >>>>>>>>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>>>>>>>
> >>>>>>>>>>> motorolasolutions.com
> >>>>>>>>>>>
> >>>>>>>>>>> *O: 602.529.8226*
> >>>>>>>>>>>
> >>>>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>
> >
> >
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>


--
Yury Bushmelev
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
Thanks for your feedback. There seems to be some understanding that the
hostname is not set properly on the client-side. This is not the case, the
hostname displays properly on the host itself and is also properly
configured from a linux perspective. This is precisely why I'm inquiring
about alternatives. The only differentiating factor with respect to these
dynamically created hosts is that they do not get registered in DNS since
their life is, or can be, quite short based on computing demand.

I was under the impression that the hostname used by the server-side
(collector) was the result of a server-side DNS lookup, which will not
resolve for these hosts. This is why I was looking for a rsyslog solution
that didn't involved DNS.

Yuri, if I understand you correctly you're saying a custom template using
HOSTNAME vs FROMHOST-IP may be an option. I'll look into this to understand
it better. Thanks for this suggestion, it sound like it completely removes
the DNS constraint. I'll give it a try.

*Scott Slattery*

*Sr. Enterprise/Cloud Architect*

*Cloud, Compute, Information & Architecture Team*

motorolasolutions.com

*O: 602.529.8226*

*E*: Scott.Slattery@MotorolaSolutions.com




On Tue, Nov 16, 2021 at 7:15 PM Yuri Bushmelev <jay4mail@gmail.com> wrote:

> Hello!
>
> Just a reminder that a hostname field in a syslog message is just a string
> sent from sender to collector. So you can craft a custom template with the
> hostname field defined as you'd like. Though I'd call this a "fallback" way
> of fixing the issue. The right way is to set the proper hostname on a
> sender system before rsyslog starts I'd say.
>
> On Wed, 17 Nov 2021 at 08:33, David Lang via rsyslog <
> rsyslog@lists.adiscon.com> wrote:
>
>> Rsyslog looks up the hostname as it starts up, so if something after
>> rsyslog
>> starts changes the hostname, rsyslog isn't going to notice until you
>> restart
>> rsyslog.
>>
>> again, fromhost is a receiver side lookup of the name to match
>> fromhost-ip, so
>> if hostname is getting set correctly, filter on that instead of on
>> fromhost.
>>
>> David Lang
>>
>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>
>> > Date: Tue, 16 Nov 2021 17:28:15 -0700
>> > From: Scott Slattery <scott.slattery@motorolasolutions.com>
>> > To: David Lang <david@lang.hm>
>> > Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>> > Subject: Re: [rsyslog] FROMHOST missing on central log collector
>> >
>> > Thanks, David, I think you've done more than enough to try and help me
>> on
>> > this. I need to do some reading on Amazon (and the link you shared) to
>> see
>> > what my options are. I agree with you, it's likely workable.
>> >
>> > I've confirmed that the results from the 'hostname' command do match so
>> > it's a bit of a mystery why rsyslog doesn't detect this but, i think
>> you're
>> > on the right track, we need to run a post-deployment script to get these
>> > instances registered in Route53.
>> >
>> >
>> > *Scott Slattery*
>> >
>> > *Sr. Enterprise/Cloud Architect*
>> >
>> > *Cloud, Compute, Information & Architecture Team*
>> >
>> > motorolasolutions.com
>> >
>> > *O: 602.529.8226*
>> >
>> > *E*: Scott.Slattery@MotorolaSolutions.com
>> >
>> >
>> >
>> >
>> > On Tue, Nov 16, 2021 at 5:20 PM David Lang <david@lang.hm> wrote:
>> >
>> >> if you login to one of the systems, you should find that the name
>> returned
>> >> by
>> >> the hostname command should match what you get in the syslog message
>> that
>> >> is
>> >> delivered to your central collector. (if it doesn't, try restarting
>> >> rsyslog and
>> >> see if it changes to match)
>> >>
>> >> then the question becomes what mechansims does AMI provide for
>> customizing
>> >> the
>> >> hostname
>> >>
>> >> a quick google search shows a new hostnamectl command
>> >>
>> >>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.aws.amazon.com_AWSEC2_latest_UserGuide_set-2Dhostname.html&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=WR-Pz8svN0d8vqg4ZKSNj2dbxtcngaMJ4iiRXCPpD6c&e=
>> >>
>> >>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.cyberciti.biz_faq_set-2Dchange-2Dhostname-2Din-2Damazon-2Dlinux-2Dec2-2Dinstance-2Dserver_&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=2RI1Khq-fBEBJxckXk9nWDESN8pTJxGiGv6xpsbYhzE&e=
>> >>
>> >> I know there is a way for you to specify a script to run when an
>> instance
>> >> is
>> >> started, that script can then set things like this. I don't know
>> enough to
>> >> point
>> >> you at specifically how to do that.
>> >>
>> >> David Lang
>> >>
>> >>
>> >> On Tue, 16 Nov 2021, Scott Slattery wrote:
>> >>
>> >>> Date: Tue, 16 Nov 2021 17:07:47 -0700
>> >>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>> >>> To: David Lang <david@lang.hm>
>> >>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>> >>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>> >>>
>> >>> Thanks David, the hostname is currently set in the AMI (Amazon Master
>> >>> Image) which is the source image for all instances that are
>> dynamically
>> >>> created and I can verify that, if you login to one of these dynamic
>> >>> instances, the hostname is in fact set correctly.
>> >>>
>> >>> The issue doesn't seem particularly related to what is set in
>> >>> /etc/hostname, /etc/hosts, or what was set using 'hostname' command. I
>> >>> think you can see this is the source of my frustration. It appears the
>> >>> central log collector relies only on DNS resolution unless there's
>> some
>> >>> hidden magic inside RSYSLOG to force the sent logs to include a host
>> >> header
>> >>> (vs DNS).
>> >>>
>> >>> I don't want to continue wasting your time but again, it is much
>> >>> appreciated. I'll look into some way of dynamically adding these
>> hosts to
>> >>> DNS in AWS Route53. It appears rsyslog simply can't do what I'm after.
>> >>>
>> >>>
>> >>> *Scott Slattery*
>> >>>
>> >>> *Sr. Enterprise/Cloud Architect*
>> >>>
>> >>> *Cloud, Compute, Information & Architecture Team*
>> >>>
>> >>> motorolasolutions.com
>> >>>
>> >>> *O: 602.529.8226*
>> >>>
>> >>> *E*: Scott.Slattery@MotorolaSolutions.com
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> On Tue, Nov 16, 2021 at 5:02 PM David Lang <david@lang.hm> wrote:
>> >>>
>> >>>> the hostname command will let you set the hostname (you want to do
>> that
>> >>>> before
>> >>>> you start rsyslog). I would expect that the orcastration tool you
>> use to
>> >>>> create
>> >>>> the systems will have some 'correct for that tool' way to set the
>> >> hostname
>> >>>> as it
>> >>>> starts the instance (sorry I can't provide more specifics, if you can
>> >>>> mention
>> >>>> what you are using, possibly someone else can chime in on the best
>> way
>> >> to
>> >>>> set
>> >>>> the hostname with that tool)
>> >>>>
>> >>>> David Lang
>> >>>>
>> >>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>> >>>>
>> >>>>> Date: Tue, 16 Nov 2021 16:59:17 -0700
>> >>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>> >>>>> To: David Lang <david@lang.hm>
>> >>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>> >>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>> >>>>>
>> >>>>> My follow-on question woudl be how do I set the hostname at the
>> client
>> >>>> end?
>> >>>>> Other than what's in /etc/hosts, /etc/hostname, etc. I don't know
>> how
>> >>>> else
>> >>>>> I would affect the log being sent to ensure it's going over.
>> >>>>>
>> >>>>> *Scott Slattery*
>> >>>>>
>> >>>>> *Sr. Enterprise/Cloud Architect*
>> >>>>>
>> >>>>> *Cloud, Compute, Information & Architecture Team*
>> >>>>>
>> >>>>> motorolasolutions.com
>> >>>>>
>> >>>>> *O: 602.529.8226*
>> >>>>>
>> >>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Tue, Nov 16, 2021 at 4:55 PM David Lang <david@lang.hm> wrote:
>> >>>>>
>> >>>>>> the translation from fromhost-ip to fromhost is done at the
>> collector,
>> >>>> but
>> >>>>>> the
>> >>>>>> sender sets the hostname field. If you can trust that hostname was
>> set
>> >>>>>> correctly, there is no reason to use fromhost
>> >>>>>>
>> >>>>>> David Lang
>> >>>>>>
>> >>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>> >>>>>>
>> >>>>>>> Date: Tue, 16 Nov 2021 16:53:19 -0700
>> >>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>> >>>>>>> To: David Lang <david@lang.hm>
>> >>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>> >>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>> >>>>>>>
>> >>>>>>> Thanks David, I could be wrong but the resolution seems to be
>> >> happening
>> >>>>>> at
>> >>>>>>> the log collection server, not the client end. Given this, I'm not
>> >> sure
>> >>>>>>> anything outside of rsyslog on the client would affect what the
>> >>>> receiving
>> >>>>>>> collection server is seeing.
>> >>>>>>>
>> >>>>>>> My hope was that this could be affected by RSYSLOG on the client
>> >> device
>> >>>>>> but
>> >>>>>>> perhaps not. I'll also look into AWS to see if a dynamically
>> created
>> >>>>>>> compute resource can automatically be registered with DNS.
>> >>>>>>>
>> >>>>>>> If anything else comes to mind, let me know. As always, I
>> appreciate
>> >>>> your
>> >>>>>>> feedback.
>> >>>>>>>
>> >>>>>>> *Scott Slattery*
>> >>>>>>>
>> >>>>>>> *Sr. Enterprise/Cloud Architect*
>> >>>>>>>
>> >>>>>>> *Cloud, Compute, Information & Architecture Team*
>> >>>>>>>
>> >>>>>>> motorolasolutions.com
>> >>>>>>>
>> >>>>>>> *O: 602.529.8226*
>> >>>>>>>
>> >>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm> wrote:
>> >>>>>>>
>> >>>>>>>> Linux has a rather sophisticated mechanism for plugging in
>> arbitrary
>> >>>>>> ways
>> >>>>>>>> of
>> >>>>>>>> doing name resolution. DNS has 'won' but hitorically there have
>> been
>> >>>>>> many
>> >>>>>>>> other
>> >>>>>>>> options. Research nsswitch (/etc/nsswitch.conf) and see if there
>> is
>> >>>>>>>> something
>> >>>>>>>> that you can leverage.
>> >>>>>>>>
>> >>>>>>>> or, if you can set the hostname of the resources as they are
>> created
>> >>>> to
>> >>>>>> be
>> >>>>>>>> some
>> >>>>>>>> predicatable pattern rather than the AWS default of IP based, you
>> >> can
>> >>>>>> then
>> >>>>>>>> make
>> >>>>>>>> your logic use that. (This is the approach I would look into).
>> What
>> >>>>>>>> mechanism
>> >>>>>>>> this will be will depend on how you are configuring/provisioning
>> the
>> >>>>>>>> systems.
>> >>>>>>>>
>> >>>>>>>> David Lang
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>> >>>>>>>>
>> >>>>>>>>> Date: Tue, 16 Nov 2021 15:14:51 -0700
>> >>>>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>> >>>>>>>>> To: David Lang <david@lang.hm>
>> >>>>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>> >>>>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>> >>>>>>>>>
>> >>>>>>>>> Thanks, David, I was hoping this was possible. Since the compute
>> >>>>>>>> resources
>> >>>>>>>>> are dynamic, using any sort of local /etc/hosts would be
>> impossible
>> >>>>>> since
>> >>>>>>>>> the IP are unpredictable. Can you point me to how I would do
>> this
>> >> on
>> >>>>>> the
>> >>>>>>>>> client-server?
>> >>>>>>>>>
>> >>>>>>>>> Thanks
>> >>>>>>>>>
>> >>>>>>>>> *Scott Slattery*
>> >>>>>>>>>
>> >>>>>>>>> *Sr. Enterprise/Cloud Architect*
>> >>>>>>>>>
>> >>>>>>>>> *Cloud, Compute, Information & Architecture Team*
>> >>>>>>>>>
>> >>>>>>>>> motorolasolutions.com
>> >>>>>>>>>
>> >>>>>>>>> *O: 602.529.8226*
>> >>>>>>>>>
>> >>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm>
>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> fromhost is the result of a name lookup of fromhost-ip. On the
>> >>>>>> receiver,
>> >>>>>>>>>> you can
>> >>>>>>>>>> control this with your name resolution (DNS, /etc/hosts, other
>> >>>>>>>> mechanisms)
>> >>>>>>>>>>
>> >>>>>>>>>> but a better option would probably be to set the hostname on
>> the
>> >>>>>> sender.
>> >>>>>>>>>> The
>> >>>>>>>>>> hostname field in the message is under the full control of the
>> >>>> sender.
>> >>>>>>>>>>
>> >>>>>>>>>> David Lang
>> >>>>>>>>>>
>> >>>>>>>>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
>> >>>>>>>>>>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>> >>>>>>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>> >>>>>>>>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
>> >>>>>>>>>>> Subject: [rsyslog] FROMHOST missing on central log collector
>> >>>>>>>>>>>
>> >>>>>>>>>>> Hello,
>> >>>>>>>>>>>
>> >>>>>>>>>>> I have a central log server, many of them, using rsyslog to
>> >>>> aggregate
>> >>>>>>>>>> logs
>> >>>>>>>>>>> from remote servers. Everything works great but I have a new
>> >>>>>> challenge
>> >>>>>>>>>> and
>> >>>>>>>>>>> am hoping for some recommendations.
>> >>>>>>>>>>>
>> >>>>>>>>>>> I have a number of AWS auto-scaling groups where compute
>> >> resources
>> >>>>>> are
>> >>>>>>>>>>> dynamically scaled up and down. Each of these will have a
>> custom
>> >>>>>>>> rsyslog
>> >>>>>>>>>>> configuration pulled from the AWS AMI.
>> >>>>>>>>>>>
>> >>>>>>>>>>> These dynamic resources are not added to DNS due to their
>> dynamic
>> >>>>>>>> nature
>> >>>>>>>>>> so
>> >>>>>>>>>>> they will not have DNS assigned FQDNs.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Because of the lack of a hostname, my central log server is
>> >> getting
>> >>>>>>>> only
>> >>>>>>>>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
>> >>>>>>>>>>>
>> >>>>>>>>>>> So what I'm seeing today looks like
>> '10.38.134.77-10.38.134.77'
>> >>>>>> where I
>> >>>>>>>>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
>> >>>>>>>>>>>
>> >>>>>>>>>>> What I'd want to do is have easy resource send using the same
>> >>>>>> hostname
>> >>>>>>>>>> and
>> >>>>>>>>>>> current IP. This later will allow me to aggregate all
>> resources
>> >> by
>> >>>>>>>> name.
>> >>>>>>>>>>>
>> >>>>>>>>>>> I did not see any way of affecting the FROMHOST information
>> >> unless,
>> >>>>>> on
>> >>>>>>>>>> the
>> >>>>>>>>>>> collector, I have rules based on IP address which isn't
>> optimal
>> >>>> given
>> >>>>>>>> the
>> >>>>>>>>>>> dynamic nature of the IPs changing.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Any suggestion is appreciated.
>> >>>>>>>>>>>
>> >>>>>>>>>>> *Scott Slattery*
>> >>>>>>>>>>>
>> >>>>>>>>>>> *Sr. Enterprise/Cloud Architect*
>> >>>>>>>>>>>
>> >>>>>>>>>>> *Cloud, Compute, Information & Architecture Team*
>> >>>>>>>>>>>
>> >>>>>>>>>>> motorolasolutions.com
>> >>>>>>>>>>>
>> >>>>>>>>>>> *O: 602.529.8226*
>> >>>>>>>>>>>
>> >>>>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>>
>> >>
>> >
>> >
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=db-lyqaTcrex58uwzOcY54hh137E9JMAF6vN-1IWnsA&e=>
>> http://www.rsyslog.com/professional-services/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=eGWs1Xi6yCyCD3OYNlbvl3fIYBADttEDYjwGyicAZbk&e=>
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=KIBqHKSAQtwhZA0rXY7Uh_or50wek4ABsH6-S4pxX0c&e=>
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
>
>
> --
> Yury Bushmelev
>

--


*For more information on how and why we collect your personal
information, please visit our Privacy Policy
<https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement>.*
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
Think relay: fromhost is the last hop (socket layer sender), hostname
is the original sender (syslog layer) - iff the sender works according
to RFCs, unfortunately.

Rainer

El mié, 17 nov 2021 a las 17:44, Scott Slattery via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> Thanks for your feedback. There seems to be some understanding that the
> hostname is not set properly on the client-side. This is not the case, the
> hostname displays properly on the host itself and is also properly
> configured from a linux perspective. This is precisely why I'm inquiring
> about alternatives. The only differentiating factor with respect to these
> dynamically created hosts is that they do not get registered in DNS since
> their life is, or can be, quite short based on computing demand.
>
> I was under the impression that the hostname used by the server-side
> (collector) was the result of a server-side DNS lookup, which will not
> resolve for these hosts. This is why I was looking for a rsyslog solution
> that didn't involved DNS.
>
> Yuri, if I understand you correctly you're saying a custom template using
> HOSTNAME vs FROMHOST-IP may be an option. I'll look into this to understand
> it better. Thanks for this suggestion, it sound like it completely removes
> the DNS constraint. I'll give it a try.
>
> *Scott Slattery*
>
> *Sr. Enterprise/Cloud Architect*
>
> *Cloud, Compute, Information & Architecture Team*
>
> motorolasolutions.com
>
> *O: 602.529.8226*
>
> *E*: Scott.Slattery@MotorolaSolutions.com
>
>
>
>
> On Tue, Nov 16, 2021 at 7:15 PM Yuri Bushmelev <jay4mail@gmail.com> wrote:
>
> > Hello!
> >
> > Just a reminder that a hostname field in a syslog message is just a string
> > sent from sender to collector. So you can craft a custom template with the
> > hostname field defined as you'd like. Though I'd call this a "fallback" way
> > of fixing the issue. The right way is to set the proper hostname on a
> > sender system before rsyslog starts I'd say.
> >
> > On Wed, 17 Nov 2021 at 08:33, David Lang via rsyslog <
> > rsyslog@lists.adiscon.com> wrote:
> >
> >> Rsyslog looks up the hostname as it starts up, so if something after
> >> rsyslog
> >> starts changes the hostname, rsyslog isn't going to notice until you
> >> restart
> >> rsyslog.
> >>
> >> again, fromhost is a receiver side lookup of the name to match
> >> fromhost-ip, so
> >> if hostname is getting set correctly, filter on that instead of on
> >> fromhost.
> >>
> >> David Lang
> >>
> >> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>
> >> > Date: Tue, 16 Nov 2021 17:28:15 -0700
> >> > From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >> > To: David Lang <david@lang.hm>
> >> > Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >> > Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >> >
> >> > Thanks, David, I think you've done more than enough to try and help me
> >> on
> >> > this. I need to do some reading on Amazon (and the link you shared) to
> >> see
> >> > what my options are. I agree with you, it's likely workable.
> >> >
> >> > I've confirmed that the results from the 'hostname' command do match so
> >> > it's a bit of a mystery why rsyslog doesn't detect this but, i think
> >> you're
> >> > on the right track, we need to run a post-deployment script to get these
> >> > instances registered in Route53.
> >> >
> >> >
> >> > *Scott Slattery*
> >> >
> >> > *Sr. Enterprise/Cloud Architect*
> >> >
> >> > *Cloud, Compute, Information & Architecture Team*
> >> >
> >> > motorolasolutions.com
> >> >
> >> > *O: 602.529.8226*
> >> >
> >> > *E*: Scott.Slattery@MotorolaSolutions.com
> >> >
> >> >
> >> >
> >> >
> >> > On Tue, Nov 16, 2021 at 5:20 PM David Lang <david@lang.hm> wrote:
> >> >
> >> >> if you login to one of the systems, you should find that the name
> >> returned
> >> >> by
> >> >> the hostname command should match what you get in the syslog message
> >> that
> >> >> is
> >> >> delivered to your central collector. (if it doesn't, try restarting
> >> >> rsyslog and
> >> >> see if it changes to match)
> >> >>
> >> >> then the question becomes what mechansims does AMI provide for
> >> customizing
> >> >> the
> >> >> hostname
> >> >>
> >> >> a quick google search shows a new hostnamectl command
> >> >>
> >> >>
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.aws.amazon.com_AWSEC2_latest_UserGuide_set-2Dhostname.html&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=WR-Pz8svN0d8vqg4ZKSNj2dbxtcngaMJ4iiRXCPpD6c&e=
> >> >>
> >> >>
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.cyberciti.biz_faq_set-2Dchange-2Dhostname-2Din-2Damazon-2Dlinux-2Dec2-2Dinstance-2Dserver_&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=2RI1Khq-fBEBJxckXk9nWDESN8pTJxGiGv6xpsbYhzE&e=
> >> >>
> >> >> I know there is a way for you to specify a script to run when an
> >> instance
> >> >> is
> >> >> started, that script can then set things like this. I don't know
> >> enough to
> >> >> point
> >> >> you at specifically how to do that.
> >> >>
> >> >> David Lang
> >> >>
> >> >>
> >> >> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >> >>
> >> >>> Date: Tue, 16 Nov 2021 17:07:47 -0700
> >> >>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >> >>> To: David Lang <david@lang.hm>
> >> >>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >> >>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >> >>>
> >> >>> Thanks David, the hostname is currently set in the AMI (Amazon Master
> >> >>> Image) which is the source image for all instances that are
> >> dynamically
> >> >>> created and I can verify that, if you login to one of these dynamic
> >> >>> instances, the hostname is in fact set correctly.
> >> >>>
> >> >>> The issue doesn't seem particularly related to what is set in
> >> >>> /etc/hostname, /etc/hosts, or what was set using 'hostname' command. I
> >> >>> think you can see this is the source of my frustration. It appears the
> >> >>> central log collector relies only on DNS resolution unless there's
> >> some
> >> >>> hidden magic inside RSYSLOG to force the sent logs to include a host
> >> >> header
> >> >>> (vs DNS).
> >> >>>
> >> >>> I don't want to continue wasting your time but again, it is much
> >> >>> appreciated. I'll look into some way of dynamically adding these
> >> hosts to
> >> >>> DNS in AWS Route53. It appears rsyslog simply can't do what I'm after.
> >> >>>
> >> >>>
> >> >>> *Scott Slattery*
> >> >>>
> >> >>> *Sr. Enterprise/Cloud Architect*
> >> >>>
> >> >>> *Cloud, Compute, Information & Architecture Team*
> >> >>>
> >> >>> motorolasolutions.com
> >> >>>
> >> >>> *O: 602.529.8226*
> >> >>>
> >> >>> *E*: Scott.Slattery@MotorolaSolutions.com
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> On Tue, Nov 16, 2021 at 5:02 PM David Lang <david@lang.hm> wrote:
> >> >>>
> >> >>>> the hostname command will let you set the hostname (you want to do
> >> that
> >> >>>> before
> >> >>>> you start rsyslog). I would expect that the orcastration tool you
> >> use to
> >> >>>> create
> >> >>>> the systems will have some 'correct for that tool' way to set the
> >> >> hostname
> >> >>>> as it
> >> >>>> starts the instance (sorry I can't provide more specifics, if you can
> >> >>>> mention
> >> >>>> what you are using, possibly someone else can chime in on the best
> >> way
> >> >> to
> >> >>>> set
> >> >>>> the hostname with that tool)
> >> >>>>
> >> >>>> David Lang
> >> >>>>
> >> >>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >> >>>>
> >> >>>>> Date: Tue, 16 Nov 2021 16:59:17 -0700
> >> >>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >> >>>>> To: David Lang <david@lang.hm>
> >> >>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >> >>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >> >>>>>
> >> >>>>> My follow-on question woudl be how do I set the hostname at the
> >> client
> >> >>>> end?
> >> >>>>> Other than what's in /etc/hosts, /etc/hostname, etc. I don't know
> >> how
> >> >>>> else
> >> >>>>> I would affect the log being sent to ensure it's going over.
> >> >>>>>
> >> >>>>> *Scott Slattery*
> >> >>>>>
> >> >>>>> *Sr. Enterprise/Cloud Architect*
> >> >>>>>
> >> >>>>> *Cloud, Compute, Information & Architecture Team*
> >> >>>>>
> >> >>>>> motorolasolutions.com
> >> >>>>>
> >> >>>>> *O: 602.529.8226*
> >> >>>>>
> >> >>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>> On Tue, Nov 16, 2021 at 4:55 PM David Lang <david@lang.hm> wrote:
> >> >>>>>
> >> >>>>>> the translation from fromhost-ip to fromhost is done at the
> >> collector,
> >> >>>> but
> >> >>>>>> the
> >> >>>>>> sender sets the hostname field. If you can trust that hostname was
> >> set
> >> >>>>>> correctly, there is no reason to use fromhost
> >> >>>>>>
> >> >>>>>> David Lang
> >> >>>>>>
> >> >>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >> >>>>>>
> >> >>>>>>> Date: Tue, 16 Nov 2021 16:53:19 -0700
> >> >>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >> >>>>>>> To: David Lang <david@lang.hm>
> >> >>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >> >>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >> >>>>>>>
> >> >>>>>>> Thanks David, I could be wrong but the resolution seems to be
> >> >> happening
> >> >>>>>> at
> >> >>>>>>> the log collection server, not the client end. Given this, I'm not
> >> >> sure
> >> >>>>>>> anything outside of rsyslog on the client would affect what the
> >> >>>> receiving
> >> >>>>>>> collection server is seeing.
> >> >>>>>>>
> >> >>>>>>> My hope was that this could be affected by RSYSLOG on the client
> >> >> device
> >> >>>>>> but
> >> >>>>>>> perhaps not. I'll also look into AWS to see if a dynamically
> >> created
> >> >>>>>>> compute resource can automatically be registered with DNS.
> >> >>>>>>>
> >> >>>>>>> If anything else comes to mind, let me know. As always, I
> >> appreciate
> >> >>>> your
> >> >>>>>>> feedback.
> >> >>>>>>>
> >> >>>>>>> *Scott Slattery*
> >> >>>>>>>
> >> >>>>>>> *Sr. Enterprise/Cloud Architect*
> >> >>>>>>>
> >> >>>>>>> *Cloud, Compute, Information & Architecture Team*
> >> >>>>>>>
> >> >>>>>>> motorolasolutions.com
> >> >>>>>>>
> >> >>>>>>> *O: 602.529.8226*
> >> >>>>>>>
> >> >>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm> wrote:
> >> >>>>>>>
> >> >>>>>>>> Linux has a rather sophisticated mechanism for plugging in
> >> arbitrary
> >> >>>>>> ways
> >> >>>>>>>> of
> >> >>>>>>>> doing name resolution. DNS has 'won' but hitorically there have
> >> been
> >> >>>>>> many
> >> >>>>>>>> other
> >> >>>>>>>> options. Research nsswitch (/etc/nsswitch.conf) and see if there
> >> is
> >> >>>>>>>> something
> >> >>>>>>>> that you can leverage.
> >> >>>>>>>>
> >> >>>>>>>> or, if you can set the hostname of the resources as they are
> >> created
> >> >>>> to
> >> >>>>>> be
> >> >>>>>>>> some
> >> >>>>>>>> predicatable pattern rather than the AWS default of IP based, you
> >> >> can
> >> >>>>>> then
> >> >>>>>>>> make
> >> >>>>>>>> your logic use that. (This is the approach I would look into).
> >> What
> >> >>>>>>>> mechanism
> >> >>>>>>>> this will be will depend on how you are configuring/provisioning
> >> the
> >> >>>>>>>> systems.
> >> >>>>>>>>
> >> >>>>>>>> David Lang
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >> >>>>>>>>
> >> >>>>>>>>> Date: Tue, 16 Nov 2021 15:14:51 -0700
> >> >>>>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >> >>>>>>>>> To: David Lang <david@lang.hm>
> >> >>>>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >> >>>>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >> >>>>>>>>>
> >> >>>>>>>>> Thanks, David, I was hoping this was possible. Since the compute
> >> >>>>>>>> resources
> >> >>>>>>>>> are dynamic, using any sort of local /etc/hosts would be
> >> impossible
> >> >>>>>> since
> >> >>>>>>>>> the IP are unpredictable. Can you point me to how I would do
> >> this
> >> >> on
> >> >>>>>> the
> >> >>>>>>>>> client-server?
> >> >>>>>>>>>
> >> >>>>>>>>> Thanks
> >> >>>>>>>>>
> >> >>>>>>>>> *Scott Slattery*
> >> >>>>>>>>>
> >> >>>>>>>>> *Sr. Enterprise/Cloud Architect*
> >> >>>>>>>>>
> >> >>>>>>>>> *Cloud, Compute, Information & Architecture Team*
> >> >>>>>>>>>
> >> >>>>>>>>> motorolasolutions.com
> >> >>>>>>>>>
> >> >>>>>>>>> *O: 602.529.8226*
> >> >>>>>>>>>
> >> >>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm>
> >> wrote:
> >> >>>>>>>>>
> >> >>>>>>>>>> fromhost is the result of a name lookup of fromhost-ip. On the
> >> >>>>>> receiver,
> >> >>>>>>>>>> you can
> >> >>>>>>>>>> control this with your name resolution (DNS, /etc/hosts, other
> >> >>>>>>>> mechanisms)
> >> >>>>>>>>>>
> >> >>>>>>>>>> but a better option would probably be to set the hostname on
> >> the
> >> >>>>>> sender.
> >> >>>>>>>>>> The
> >> >>>>>>>>>> hostname field in the message is under the full control of the
> >> >>>> sender.
> >> >>>>>>>>>>
> >> >>>>>>>>>> David Lang
> >> >>>>>>>>>>
> >> >>>>>>>>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
> >> >>>>>>>>>>
> >> >>>>>>>>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
> >> >>>>>>>>>>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >> >>>>>>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
> >> >>>>>>>>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
> >> >>>>>>>>>>> Subject: [rsyslog] FROMHOST missing on central log collector
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Hello,
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> I have a central log server, many of them, using rsyslog to
> >> >>>> aggregate
> >> >>>>>>>>>> logs
> >> >>>>>>>>>>> from remote servers. Everything works great but I have a new
> >> >>>>>> challenge
> >> >>>>>>>>>> and
> >> >>>>>>>>>>> am hoping for some recommendations.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> I have a number of AWS auto-scaling groups where compute
> >> >> resources
> >> >>>>>> are
> >> >>>>>>>>>>> dynamically scaled up and down. Each of these will have a
> >> custom
> >> >>>>>>>> rsyslog
> >> >>>>>>>>>>> configuration pulled from the AWS AMI.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> These dynamic resources are not added to DNS due to their
> >> dynamic
> >> >>>>>>>> nature
> >> >>>>>>>>>> so
> >> >>>>>>>>>>> they will not have DNS assigned FQDNs.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Because of the lack of a hostname, my central log server is
> >> >> getting
> >> >>>>>>>> only
> >> >>>>>>>>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> So what I'm seeing today looks like
> >> '10.38.134.77-10.38.134.77'
> >> >>>>>> where I
> >> >>>>>>>>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> What I'd want to do is have easy resource send using the same
> >> >>>>>> hostname
> >> >>>>>>>>>> and
> >> >>>>>>>>>>> current IP. This later will allow me to aggregate all
> >> resources
> >> >> by
> >> >>>>>>>> name.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> I did not see any way of affecting the FROMHOST information
> >> >> unless,
> >> >>>>>> on
> >> >>>>>>>>>> the
> >> >>>>>>>>>>> collector, I have rules based on IP address which isn't
> >> optimal
> >> >>>> given
> >> >>>>>>>> the
> >> >>>>>>>>>>> dynamic nature of the IPs changing.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Any suggestion is appreciated.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> *Scott Slattery*
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> *Sr. Enterprise/Cloud Architect*
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> *Cloud, Compute, Information & Architecture Team*
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> motorolasolutions.com
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> *O: 602.529.8226*
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>
> >> >>>
> >> >>>
> >> >>
> >> >
> >> >
> >> _______________________________________________
> >> rsyslog mailing list
> >> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=db-lyqaTcrex58uwzOcY54hh137E9JMAF6vN-1IWnsA&e=>
> >> http://www.rsyslog.com/professional-services/
> >> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=eGWs1Xi6yCyCD3OYNlbvl3fIYBADttEDYjwGyicAZbk&e=>
> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> <https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=KIBqHKSAQtwhZA0rXY7Uh_or50wek4ABsH6-S4pxX0c&e=>
> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> >> DON'T LIKE THAT.
> >>
> >
> >
> > --
> > Yury Bushmelev
> >
>
> --
>
>
> *For more information on how and why we collect your personal
> information, please visit our Privacy Policy
> <https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement>.*
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
Thanks Rainer, in my case the sender is the last hop. So they should be the
same.

*Scott Slattery*

*Sr. Enterprise/Cloud Architect*

*Cloud, Compute, Information & Architecture Team*

motorolasolutions.com

*O: 602.529.8226*

*E*: Scott.Slattery@MotorolaSolutions.com




On Wed, Nov 17, 2021 at 9:53 AM Rainer Gerhards <rgerhards@hq.adiscon.com>
wrote:

> Think relay: fromhost is the last hop (socket layer sender), hostname
> is the original sender (syslog layer) - iff the sender works according
> to RFCs, unfortunately.
>
> Rainer
>
> El mié, 17 nov 2021 a las 17:44, Scott Slattery via rsyslog
> (<rsyslog@lists.adiscon.com>) escribió:
> >
> > Thanks for your feedback. There seems to be some understanding that the
> > hostname is not set properly on the client-side. This is not the case,
> the
> > hostname displays properly on the host itself and is also properly
> > configured from a linux perspective. This is precisely why I'm inquiring
> > about alternatives. The only differentiating factor with respect to these
> > dynamically created hosts is that they do not get registered in DNS since
> > their life is, or can be, quite short based on computing demand.
> >
> > I was under the impression that the hostname used by the server-side
> > (collector) was the result of a server-side DNS lookup, which will not
> > resolve for these hosts. This is why I was looking for a rsyslog solution
> > that didn't involved DNS.
> >
> > Yuri, if I understand you correctly you're saying a custom template using
> > HOSTNAME vs FROMHOST-IP may be an option. I'll look into this to
> understand
> > it better. Thanks for this suggestion, it sound like it completely
> removes
> > the DNS constraint. I'll give it a try.
> >
> > *Scott Slattery*
> >
> > *Sr. Enterprise/Cloud Architect*
> >
> > *Cloud, Compute, Information & Architecture Team*
> >
> > motorolasolutions.com
> >
> > *O: 602.529.8226*
> >
> > *E*: Scott.Slattery@MotorolaSolutions.com
> >
> >
> >
> >
> > On Tue, Nov 16, 2021 at 7:15 PM Yuri Bushmelev <jay4mail@gmail.com>
> wrote:
> >
> > > Hello!
> > >
> > > Just a reminder that a hostname field in a syslog message is just a
> string
> > > sent from sender to collector. So you can craft a custom template with
> the
> > > hostname field defined as you'd like. Though I'd call this a
> "fallback" way
> > > of fixing the issue. The right way is to set the proper hostname on a
> > > sender system before rsyslog starts I'd say.
> > >
> > > On Wed, 17 Nov 2021 at 08:33, David Lang via rsyslog <
> > > rsyslog@lists.adiscon.com> wrote:
> > >
> > >> Rsyslog looks up the hostname as it starts up, so if something after
> > >> rsyslog
> > >> starts changes the hostname, rsyslog isn't going to notice until you
> > >> restart
> > >> rsyslog.
> > >>
> > >> again, fromhost is a receiver side lookup of the name to match
> > >> fromhost-ip, so
> > >> if hostname is getting set correctly, filter on that instead of on
> > >> fromhost.
> > >>
> > >> David Lang
> > >>
> > >> On Tue, 16 Nov 2021, Scott Slattery wrote:
> > >>
> > >> > Date: Tue, 16 Nov 2021 17:28:15 -0700
> > >> > From: Scott Slattery <scott.slattery@motorolasolutions.com>
> > >> > To: David Lang <david@lang.hm>
> > >> > Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> > >> > Subject: Re: [rsyslog] FROMHOST missing on central log collector
> > >> >
> > >> > Thanks, David, I think you've done more than enough to try and help
> me
> > >> on
> > >> > this. I need to do some reading on Amazon (and the link you shared)
> to
> > >> see
> > >> > what my options are. I agree with you, it's likely workable.
> > >> >
> > >> > I've confirmed that the results from the 'hostname' command do
> match so
> > >> > it's a bit of a mystery why rsyslog doesn't detect this but, i think
> > >> you're
> > >> > on the right track, we need to run a post-deployment script to get
> these
> > >> > instances registered in Route53.
> > >> >
> > >> >
> > >> > *Scott Slattery*
> > >> >
> > >> > *Sr. Enterprise/Cloud Architect*
> > >> >
> > >> > *Cloud, Compute, Information & Architecture Team*
> > >> >
> > >> > motorolasolutions.com
> > >> >
> > >> > *O: 602.529.8226*
> > >> >
> > >> > *E*: Scott.Slattery@MotorolaSolutions.com
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > On Tue, Nov 16, 2021 at 5:20 PM David Lang <david@lang.hm> wrote:
> > >> >
> > >> >> if you login to one of the systems, you should find that the name
> > >> returned
> > >> >> by
> > >> >> the hostname command should match what you get in the syslog
> message
> > >> that
> > >> >> is
> > >> >> delivered to your central collector. (if it doesn't, try restarting
> > >> >> rsyslog and
> > >> >> see if it changes to match)
> > >> >>
> > >> >> then the question becomes what mechansims does AMI provide for
> > >> customizing
> > >> >> the
> > >> >> hostname
> > >> >>
> > >> >> a quick google search shows a new hostnamectl command
> > >> >>
> > >> >>
> > >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.aws.amazon.com_AWSEC2_latest_UserGuide_set-2Dhostname.html&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=WR-Pz8svN0d8vqg4ZKSNj2dbxtcngaMJ4iiRXCPpD6c&e=
> > >> >>
> > >> >>
> > >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.cyberciti.biz_faq_set-2Dchange-2Dhostname-2Din-2Damazon-2Dlinux-2Dec2-2Dinstance-2Dserver_&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=2RI1Khq-fBEBJxckXk9nWDESN8pTJxGiGv6xpsbYhzE&e=
> > >> >>
> > >> >> I know there is a way for you to specify a script to run when an
> > >> instance
> > >> >> is
> > >> >> started, that script can then set things like this. I don't know
> > >> enough to
> > >> >> point
> > >> >> you at specifically how to do that.
> > >> >>
> > >> >> David Lang
> > >> >>
> > >> >>
> > >> >> On Tue, 16 Nov 2021, Scott Slattery wrote:
> > >> >>
> > >> >>> Date: Tue, 16 Nov 2021 17:07:47 -0700
> > >> >>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> > >> >>> To: David Lang <david@lang.hm>
> > >> >>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> > >> >>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> > >> >>>
> > >> >>> Thanks David, the hostname is currently set in the AMI (Amazon
> Master
> > >> >>> Image) which is the source image for all instances that are
> > >> dynamically
> > >> >>> created and I can verify that, if you login to one of these
> dynamic
> > >> >>> instances, the hostname is in fact set correctly.
> > >> >>>
> > >> >>> The issue doesn't seem particularly related to what is set in
> > >> >>> /etc/hostname, /etc/hosts, or what was set using 'hostname'
> command. I
> > >> >>> think you can see this is the source of my frustration. It
> appears the
> > >> >>> central log collector relies only on DNS resolution unless there's
> > >> some
> > >> >>> hidden magic inside RSYSLOG to force the sent logs to include a
> host
> > >> >> header
> > >> >>> (vs DNS).
> > >> >>>
> > >> >>> I don't want to continue wasting your time but again, it is much
> > >> >>> appreciated. I'll look into some way of dynamically adding these
> > >> hosts to
> > >> >>> DNS in AWS Route53. It appears rsyslog simply can't do what I'm
> after.
> > >> >>>
> > >> >>>
> > >> >>> *Scott Slattery*
> > >> >>>
> > >> >>> *Sr. Enterprise/Cloud Architect*
> > >> >>>
> > >> >>> *Cloud, Compute, Information & Architecture Team*
> > >> >>>
> > >> >>> motorolasolutions.com
> > >> >>>
> > >> >>> *O: 602.529.8226*
> > >> >>>
> > >> >>> *E*: Scott.Slattery@MotorolaSolutions.com
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >>> On Tue, Nov 16, 2021 at 5:02 PM David Lang <david@lang.hm> wrote:
> > >> >>>
> > >> >>>> the hostname command will let you set the hostname (you want to
> do
> > >> that
> > >> >>>> before
> > >> >>>> you start rsyslog). I would expect that the orcastration tool you
> > >> use to
> > >> >>>> create
> > >> >>>> the systems will have some 'correct for that tool' way to set the
> > >> >> hostname
> > >> >>>> as it
> > >> >>>> starts the instance (sorry I can't provide more specifics, if
> you can
> > >> >>>> mention
> > >> >>>> what you are using, possibly someone else can chime in on the
> best
> > >> way
> > >> >> to
> > >> >>>> set
> > >> >>>> the hostname with that tool)
> > >> >>>>
> > >> >>>> David Lang
> > >> >>>>
> > >> >>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> > >> >>>>
> > >> >>>>> Date: Tue, 16 Nov 2021 16:59:17 -0700
> > >> >>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> > >> >>>>> To: David Lang <david@lang.hm>
> > >> >>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> > >> >>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> > >> >>>>>
> > >> >>>>> My follow-on question woudl be how do I set the hostname at the
> > >> client
> > >> >>>> end?
> > >> >>>>> Other than what's in /etc/hosts, /etc/hostname, etc. I don't
> know
> > >> how
> > >> >>>> else
> > >> >>>>> I would affect the log being sent to ensure it's going over.
> > >> >>>>>
> > >> >>>>> *Scott Slattery*
> > >> >>>>>
> > >> >>>>> *Sr. Enterprise/Cloud Architect*
> > >> >>>>>
> > >> >>>>> *Cloud, Compute, Information & Architecture Team*
> > >> >>>>>
> > >> >>>>> motorolasolutions.com
> > >> >>>>>
> > >> >>>>> *O: 602.529.8226*
> > >> >>>>>
> > >> >>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> > >> >>>>>
> > >> >>>>>
> > >> >>>>>
> > >> >>>>>
> > >> >>>>> On Tue, Nov 16, 2021 at 4:55 PM David Lang <david@lang.hm>
> wrote:
> > >> >>>>>
> > >> >>>>>> the translation from fromhost-ip to fromhost is done at the
> > >> collector,
> > >> >>>> but
> > >> >>>>>> the
> > >> >>>>>> sender sets the hostname field. If you can trust that hostname
> was
> > >> set
> > >> >>>>>> correctly, there is no reason to use fromhost
> > >> >>>>>>
> > >> >>>>>> David Lang
> > >> >>>>>>
> > >> >>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> > >> >>>>>>
> > >> >>>>>>> Date: Tue, 16 Nov 2021 16:53:19 -0700
> > >> >>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> > >> >>>>>>> To: David Lang <david@lang.hm>
> > >> >>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> > >> >>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log
> collector
> > >> >>>>>>>
> > >> >>>>>>> Thanks David, I could be wrong but the resolution seems to be
> > >> >> happening
> > >> >>>>>> at
> > >> >>>>>>> the log collection server, not the client end. Given this,
> I'm not
> > >> >> sure
> > >> >>>>>>> anything outside of rsyslog on the client would affect what
> the
> > >> >>>> receiving
> > >> >>>>>>> collection server is seeing.
> > >> >>>>>>>
> > >> >>>>>>> My hope was that this could be affected by RSYSLOG on the
> client
> > >> >> device
> > >> >>>>>> but
> > >> >>>>>>> perhaps not. I'll also look into AWS to see if a dynamically
> > >> created
> > >> >>>>>>> compute resource can automatically be registered with DNS.
> > >> >>>>>>>
> > >> >>>>>>> If anything else comes to mind, let me know. As always, I
> > >> appreciate
> > >> >>>> your
> > >> >>>>>>> feedback.
> > >> >>>>>>>
> > >> >>>>>>> *Scott Slattery*
> > >> >>>>>>>
> > >> >>>>>>> *Sr. Enterprise/Cloud Architect*
> > >> >>>>>>>
> > >> >>>>>>> *Cloud, Compute, Information & Architecture Team*
> > >> >>>>>>>
> > >> >>>>>>> motorolasolutions.com
> > >> >>>>>>>
> > >> >>>>>>> *O: 602.529.8226*
> > >> >>>>>>>
> > >> >>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm>
> wrote:
> > >> >>>>>>>
> > >> >>>>>>>> Linux has a rather sophisticated mechanism for plugging in
> > >> arbitrary
> > >> >>>>>> ways
> > >> >>>>>>>> of
> > >> >>>>>>>> doing name resolution. DNS has 'won' but hitorically there
> have
> > >> been
> > >> >>>>>> many
> > >> >>>>>>>> other
> > >> >>>>>>>> options. Research nsswitch (/etc/nsswitch.conf) and see if
> there
> > >> is
> > >> >>>>>>>> something
> > >> >>>>>>>> that you can leverage.
> > >> >>>>>>>>
> > >> >>>>>>>> or, if you can set the hostname of the resources as they are
> > >> created
> > >> >>>> to
> > >> >>>>>> be
> > >> >>>>>>>> some
> > >> >>>>>>>> predicatable pattern rather than the AWS default of IP
> based, you
> > >> >> can
> > >> >>>>>> then
> > >> >>>>>>>> make
> > >> >>>>>>>> your logic use that. (This is the approach I would look
> into).
> > >> What
> > >> >>>>>>>> mechanism
> > >> >>>>>>>> this will be will depend on how you are
> configuring/provisioning
> > >> the
> > >> >>>>>>>> systems.
> > >> >>>>>>>>
> > >> >>>>>>>> David Lang
> > >> >>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> > >> >>>>>>>>
> > >> >>>>>>>>> Date: Tue, 16 Nov 2021 15:14:51 -0700
> > >> >>>>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> > >> >>>>>>>>> To: David Lang <david@lang.hm>
> > >> >>>>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> > >> >>>>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log
> collector
> > >> >>>>>>>>>
> > >> >>>>>>>>> Thanks, David, I was hoping this was possible. Since the
> compute
> > >> >>>>>>>> resources
> > >> >>>>>>>>> are dynamic, using any sort of local /etc/hosts would be
> > >> impossible
> > >> >>>>>> since
> > >> >>>>>>>>> the IP are unpredictable. Can you point me to how I would do
> > >> this
> > >> >> on
> > >> >>>>>> the
> > >> >>>>>>>>> client-server?
> > >> >>>>>>>>>
> > >> >>>>>>>>> Thanks
> > >> >>>>>>>>>
> > >> >>>>>>>>> *Scott Slattery*
> > >> >>>>>>>>>
> > >> >>>>>>>>> *Sr. Enterprise/Cloud Architect*
> > >> >>>>>>>>>
> > >> >>>>>>>>> *Cloud, Compute, Information & Architecture Team*
> > >> >>>>>>>>>
> > >> >>>>>>>>> motorolasolutions.com
> > >> >>>>>>>>>
> > >> >>>>>>>>> *O: 602.529.8226*
> > >> >>>>>>>>>
> > >> >>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm>
> > >> wrote:
> > >> >>>>>>>>>
> > >> >>>>>>>>>> fromhost is the result of a name lookup of fromhost-ip. On
> the
> > >> >>>>>> receiver,
> > >> >>>>>>>>>> you can
> > >> >>>>>>>>>> control this with your name resolution (DNS, /etc/hosts,
> other
> > >> >>>>>>>> mechanisms)
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> but a better option would probably be to set the hostname
> on
> > >> the
> > >> >>>>>> sender.
> > >> >>>>>>>>>> The
> > >> >>>>>>>>>> hostname field in the message is under the full control of
> the
> > >> >>>> sender.
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> David Lang
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
> > >> >>>>>>>>>>
> > >> >>>>>>>>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
> > >> >>>>>>>>>>> From: Scott Slattery via rsyslog <
> rsyslog@lists.adiscon.com>
> > >> >>>>>>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
> > >> >>>>>>>>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
> > >> >>>>>>>>>>> Subject: [rsyslog] FROMHOST missing on central log
> collector
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> Hello,
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> I have a central log server, many of them, using rsyslog
> to
> > >> >>>> aggregate
> > >> >>>>>>>>>> logs
> > >> >>>>>>>>>>> from remote servers. Everything works great but I have a
> new
> > >> >>>>>> challenge
> > >> >>>>>>>>>> and
> > >> >>>>>>>>>>> am hoping for some recommendations.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> I have a number of AWS auto-scaling groups where compute
> > >> >> resources
> > >> >>>>>> are
> > >> >>>>>>>>>>> dynamically scaled up and down. Each of these will have a
> > >> custom
> > >> >>>>>>>> rsyslog
> > >> >>>>>>>>>>> configuration pulled from the AWS AMI.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> These dynamic resources are not added to DNS due to their
> > >> dynamic
> > >> >>>>>>>> nature
> > >> >>>>>>>>>> so
> > >> >>>>>>>>>>> they will not have DNS assigned FQDNs.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> Because of the lack of a hostname, my central log server
> is
> > >> >> getting
> > >> >>>>>>>> only
> > >> >>>>>>>>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> So what I'm seeing today looks like
> > >> '10.38.134.77-10.38.134.77'
> > >> >>>>>> where I
> > >> >>>>>>>>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> What I'd want to do is have easy resource send using the
> same
> > >> >>>>>> hostname
> > >> >>>>>>>>>> and
> > >> >>>>>>>>>>> current IP. This later will allow me to aggregate all
> > >> resources
> > >> >> by
> > >> >>>>>>>> name.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> I did not see any way of affecting the FROMHOST
> information
> > >> >> unless,
> > >> >>>>>> on
> > >> >>>>>>>>>> the
> > >> >>>>>>>>>>> collector, I have rules based on IP address which isn't
> > >> optimal
> > >> >>>> given
> > >> >>>>>>>> the
> > >> >>>>>>>>>>> dynamic nature of the IPs changing.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> Any suggestion is appreciated.
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> *Scott Slattery*
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> *Sr. Enterprise/Cloud Architect*
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> *Cloud, Compute, Information & Architecture Team*
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> motorolasolutions.com
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> *O: 602.529.8226*
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>>
> > >> >>>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>
> > >> >>>>>
> > >> >>>>>
> > >> >>>>
> > >> >>>
> > >> >>>
> > >> >>
> > >> >
> > >> >
> > >> _______________________________________________
> > >> rsyslog mailing list
> > >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=byZQ4B17TpYXwp7w16sjgq1YwmV4o6O3wLuGRq16WDJzp5U81Xv4xRKNkRfPWUL_&s=LRn-pDlin6SMuY_Ncd7AA0guWZdN3zaGu3Hu0ozAlxA&e=
> > >> <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=db-lyqaTcrex58uwzOcY54hh137E9JMAF6vN-1IWnsA&e=
> >
> > >>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=byZQ4B17TpYXwp7w16sjgq1YwmV4o6O3wLuGRq16WDJzp5U81Xv4xRKNkRfPWUL_&s=BDcnrfghWuu5lVf7v9HG439toMBot9PrkNGcZcJblv8&e=
> > >> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=eGWs1Xi6yCyCD3OYNlbvl3fIYBADttEDYjwGyicAZbk&e=
> >
> > >> What's up with rsyslog? Follow
> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=byZQ4B17TpYXwp7w16sjgq1YwmV4o6O3wLuGRq16WDJzp5U81Xv4xRKNkRfPWUL_&s=T8KQImFltarg-g3d4mjbAC_qvd8mFV1z8kxk_CBWj8k&e=
> > >> <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=KIBqHKSAQtwhZA0rXY7Uh_or50wek4ABsH6-S4pxX0c&e=
> >
> > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad
> > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> > >> DON'T LIKE THAT.
> > >>
> > >
> > >
> > > --
> > > Yury Bushmelev
> > >
> >
> > --
> >
> >
> > *For more information on how and why we collect your personal
> > information, please visit our Privacy Policy
> > <
> https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement
> >.*
> > _______________________________________________
> > rsyslog mailing list
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=byZQ4B17TpYXwp7w16sjgq1YwmV4o6O3wLuGRq16WDJzp5U81Xv4xRKNkRfPWUL_&s=LRn-pDlin6SMuY_Ncd7AA0guWZdN3zaGu3Hu0ozAlxA&e=
> >
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=byZQ4B17TpYXwp7w16sjgq1YwmV4o6O3wLuGRq16WDJzp5U81Xv4xRKNkRfPWUL_&s=BDcnrfghWuu5lVf7v9HG439toMBot9PrkNGcZcJblv8&e=
> > What's up with rsyslog? Follow
> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=byZQ4B17TpYXwp7w16sjgq1YwmV4o6O3wLuGRq16WDJzp5U81Xv4xRKNkRfPWUL_&s=T8KQImFltarg-g3d4mjbAC_qvd8mFV1z8kxk_CBWj8k&e=
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>

--


*For more information on how and why we collect your personal
information, please visit our Privacy Policy
<https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement>.*
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
But not if you override the default template and use fromhost -- which is
what it looks like...

Sent from phone, thus brief.

Scott Slattery <scott.slattery@motorolasolutions.com> schrieb am Mi., 17.
Nov. 2021, 18:13:

> Thanks Rainer, in my case the sender is the last hop. So they should be
> the same.
>
> *Scott Slattery*
>
> *Sr. Enterprise/Cloud Architect*
>
> *Cloud, Compute, Information & Architecture Team*
>
> motorolasolutions.com
>
> *O: 602.529.8226*
>
> *E*: Scott.Slattery@MotorolaSolutions.com
>
>
>
>
> On Wed, Nov 17, 2021 at 9:53 AM Rainer Gerhards <rgerhards@hq.adiscon.com>
> wrote:
>
>> Think relay: fromhost is the last hop (socket layer sender), hostname
>> is the original sender (syslog layer) - iff the sender works according
>> to RFCs, unfortunately.
>>
>> Rainer
>>
>> El mié, 17 nov 2021 a las 17:44, Scott Slattery via rsyslog
>> (<rsyslog@lists.adiscon.com>) escribió:
>> >
>> > Thanks for your feedback. There seems to be some understanding that the
>> > hostname is not set properly on the client-side. This is not the case,
>> the
>> > hostname displays properly on the host itself and is also properly
>> > configured from a linux perspective. This is precisely why I'm inquiring
>> > about alternatives. The only differentiating factor with respect to
>> these
>> > dynamically created hosts is that they do not get registered in DNS
>> since
>> > their life is, or can be, quite short based on computing demand.
>> >
>> > I was under the impression that the hostname used by the server-side
>> > (collector) was the result of a server-side DNS lookup, which will not
>> > resolve for these hosts. This is why I was looking for a rsyslog
>> solution
>> > that didn't involved DNS.
>> >
>> > Yuri, if I understand you correctly you're saying a custom template
>> using
>> > HOSTNAME vs FROMHOST-IP may be an option. I'll look into this to
>> understand
>> > it better. Thanks for this suggestion, it sound like it completely
>> removes
>> > the DNS constraint. I'll give it a try.
>> >
>> > *Scott Slattery*
>> >
>> > *Sr. Enterprise/Cloud Architect*
>> >
>> > *Cloud, Compute, Information & Architecture Team*
>> >
>> > motorolasolutions.com
>> >
>> > *O: 602.529.8226*
>> >
>> > *E*: Scott.Slattery@MotorolaSolutions.com
>> >
>> >
>> >
>> >
>> > On Tue, Nov 16, 2021 at 7:15 PM Yuri Bushmelev <jay4mail@gmail.com>
>> wrote:
>> >
>> > > Hello!
>> > >
>> > > Just a reminder that a hostname field in a syslog message is just a
>> string
>> > > sent from sender to collector. So you can craft a custom template
>> with the
>> > > hostname field defined as you'd like. Though I'd call this a
>> "fallback" way
>> > > of fixing the issue. The right way is to set the proper hostname on a
>> > > sender system before rsyslog starts I'd say.
>> > >
>> > > On Wed, 17 Nov 2021 at 08:33, David Lang via rsyslog <
>> > > rsyslog@lists.adiscon.com> wrote:
>> > >
>> > >> Rsyslog looks up the hostname as it starts up, so if something after
>> > >> rsyslog
>> > >> starts changes the hostname, rsyslog isn't going to notice until you
>> > >> restart
>> > >> rsyslog.
>> > >>
>> > >> again, fromhost is a receiver side lookup of the name to match
>> > >> fromhost-ip, so
>> > >> if hostname is getting set correctly, filter on that instead of on
>> > >> fromhost.
>> > >>
>> > >> David Lang
>> > >>
>> > >> On Tue, 16 Nov 2021, Scott Slattery wrote:
>> > >>
>> > >> > Date: Tue, 16 Nov 2021 17:28:15 -0700
>> > >> > From: Scott Slattery <scott.slattery@motorolasolutions.com>
>> > >> > To: David Lang <david@lang.hm>
>> > >> > Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>> > >> > Subject: Re: [rsyslog] FROMHOST missing on central log collector
>> > >> >
>> > >> > Thanks, David, I think you've done more than enough to try and
>> help me
>> > >> on
>> > >> > this. I need to do some reading on Amazon (and the link you
>> shared) to
>> > >> see
>> > >> > what my options are. I agree with you, it's likely workable.
>> > >> >
>> > >> > I've confirmed that the results from the 'hostname' command do
>> match so
>> > >> > it's a bit of a mystery why rsyslog doesn't detect this but, i
>> think
>> > >> you're
>> > >> > on the right track, we need to run a post-deployment script to get
>> these
>> > >> > instances registered in Route53.
>> > >> >
>> > >> >
>> > >> > *Scott Slattery*
>> > >> >
>> > >> > *Sr. Enterprise/Cloud Architect*
>> > >> >
>> > >> > *Cloud, Compute, Information & Architecture Team*
>> > >> >
>> > >> > motorolasolutions.com
>> > >> >
>> > >> > *O: 602.529.8226*
>> > >> >
>> > >> > *E*: Scott.Slattery@MotorolaSolutions.com
>> > >> >
>> > >> >
>> > >> >
>> > >> >
>> > >> > On Tue, Nov 16, 2021 at 5:20 PM David Lang <david@lang.hm> wrote:
>> > >> >
>> > >> >> if you login to one of the systems, you should find that the name
>> > >> returned
>> > >> >> by
>> > >> >> the hostname command should match what you get in the syslog
>> message
>> > >> that
>> > >> >> is
>> > >> >> delivered to your central collector. (if it doesn't, try
>> restarting
>> > >> >> rsyslog and
>> > >> >> see if it changes to match)
>> > >> >>
>> > >> >> then the question becomes what mechansims does AMI provide for
>> > >> customizing
>> > >> >> the
>> > >> >> hostname
>> > >> >>
>> > >> >> a quick google search shows a new hostnamectl command
>> > >> >>
>> > >> >>
>> > >>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.aws.amazon.com_AWSEC2_latest_UserGuide_set-2Dhostname.html&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=WR-Pz8svN0d8vqg4ZKSNj2dbxtcngaMJ4iiRXCPpD6c&e=
>> > >> >>
>> > >> >>
>> > >>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.cyberciti.biz_faq_set-2Dchange-2Dhostname-2Din-2Damazon-2Dlinux-2Dec2-2Dinstance-2Dserver_&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=2RI1Khq-fBEBJxckXk9nWDESN8pTJxGiGv6xpsbYhzE&e=
>> > >> >>
>> > >> >> I know there is a way for you to specify a script to run when an
>> > >> instance
>> > >> >> is
>> > >> >> started, that script can then set things like this. I don't know
>> > >> enough to
>> > >> >> point
>> > >> >> you at specifically how to do that.
>> > >> >>
>> > >> >> David Lang
>> > >> >>
>> > >> >>
>> > >> >> On Tue, 16 Nov 2021, Scott Slattery wrote:
>> > >> >>
>> > >> >>> Date: Tue, 16 Nov 2021 17:07:47 -0700
>> > >> >>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>> > >> >>> To: David Lang <david@lang.hm>
>> > >> >>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>> > >> >>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>> > >> >>>
>> > >> >>> Thanks David, the hostname is currently set in the AMI (Amazon
>> Master
>> > >> >>> Image) which is the source image for all instances that are
>> > >> dynamically
>> > >> >>> created and I can verify that, if you login to one of these
>> dynamic
>> > >> >>> instances, the hostname is in fact set correctly.
>> > >> >>>
>> > >> >>> The issue doesn't seem particularly related to what is set in
>> > >> >>> /etc/hostname, /etc/hosts, or what was set using 'hostname'
>> command. I
>> > >> >>> think you can see this is the source of my frustration. It
>> appears the
>> > >> >>> central log collector relies only on DNS resolution unless
>> there's
>> > >> some
>> > >> >>> hidden magic inside RSYSLOG to force the sent logs to include a
>> host
>> > >> >> header
>> > >> >>> (vs DNS).
>> > >> >>>
>> > >> >>> I don't want to continue wasting your time but again, it is much
>> > >> >>> appreciated. I'll look into some way of dynamically adding these
>> > >> hosts to
>> > >> >>> DNS in AWS Route53. It appears rsyslog simply can't do what I'm
>> after.
>> > >> >>>
>> > >> >>>
>> > >> >>> *Scott Slattery*
>> > >> >>>
>> > >> >>> *Sr. Enterprise/Cloud Architect*
>> > >> >>>
>> > >> >>> *Cloud, Compute, Information & Architecture Team*
>> > >> >>>
>> > >> >>> motorolasolutions.com
>> > >> >>>
>> > >> >>> *O: 602.529.8226*
>> > >> >>>
>> > >> >>> *E*: Scott.Slattery@MotorolaSolutions.com
>> > >> >>>
>> > >> >>>
>> > >> >>>
>> > >> >>>
>> > >> >>> On Tue, Nov 16, 2021 at 5:02 PM David Lang <david@lang.hm>
>> wrote:
>> > >> >>>
>> > >> >>>> the hostname command will let you set the hostname (you want to
>> do
>> > >> that
>> > >> >>>> before
>> > >> >>>> you start rsyslog). I would expect that the orcastration tool
>> you
>> > >> use to
>> > >> >>>> create
>> > >> >>>> the systems will have some 'correct for that tool' way to set
>> the
>> > >> >> hostname
>> > >> >>>> as it
>> > >> >>>> starts the instance (sorry I can't provide more specifics, if
>> you can
>> > >> >>>> mention
>> > >> >>>> what you are using, possibly someone else can chime in on the
>> best
>> > >> way
>> > >> >> to
>> > >> >>>> set
>> > >> >>>> the hostname with that tool)
>> > >> >>>>
>> > >> >>>> David Lang
>> > >> >>>>
>> > >> >>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>> > >> >>>>
>> > >> >>>>> Date: Tue, 16 Nov 2021 16:59:17 -0700
>> > >> >>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>> > >> >>>>> To: David Lang <david@lang.hm>
>> > >> >>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>> > >> >>>>> Subject: Re: [rsyslog] FROMHOST missing on central log
>> collector
>> > >> >>>>>
>> > >> >>>>> My follow-on question woudl be how do I set the hostname at the
>> > >> client
>> > >> >>>> end?
>> > >> >>>>> Other than what's in /etc/hosts, /etc/hostname, etc. I don't
>> know
>> > >> how
>> > >> >>>> else
>> > >> >>>>> I would affect the log being sent to ensure it's going over.
>> > >> >>>>>
>> > >> >>>>> *Scott Slattery*
>> > >> >>>>>
>> > >> >>>>> *Sr. Enterprise/Cloud Architect*
>> > >> >>>>>
>> > >> >>>>> *Cloud, Compute, Information & Architecture Team*
>> > >> >>>>>
>> > >> >>>>> motorolasolutions.com
>> > >> >>>>>
>> > >> >>>>> *O: 602.529.8226*
>> > >> >>>>>
>> > >> >>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>> > >> >>>>>
>> > >> >>>>>
>> > >> >>>>>
>> > >> >>>>>
>> > >> >>>>> On Tue, Nov 16, 2021 at 4:55 PM David Lang <david@lang.hm>
>> wrote:
>> > >> >>>>>
>> > >> >>>>>> the translation from fromhost-ip to fromhost is done at the
>> > >> collector,
>> > >> >>>> but
>> > >> >>>>>> the
>> > >> >>>>>> sender sets the hostname field. If you can trust that
>> hostname was
>> > >> set
>> > >> >>>>>> correctly, there is no reason to use fromhost
>> > >> >>>>>>
>> > >> >>>>>> David Lang
>> > >> >>>>>>
>> > >> >>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>> > >> >>>>>>
>> > >> >>>>>>> Date: Tue, 16 Nov 2021 16:53:19 -0700
>> > >> >>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>> > >> >>>>>>> To: David Lang <david@lang.hm>
>> > >> >>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>> > >> >>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log
>> collector
>> > >> >>>>>>>
>> > >> >>>>>>> Thanks David, I could be wrong but the resolution seems to be
>> > >> >> happening
>> > >> >>>>>> at
>> > >> >>>>>>> the log collection server, not the client end. Given this,
>> I'm not
>> > >> >> sure
>> > >> >>>>>>> anything outside of rsyslog on the client would affect what
>> the
>> > >> >>>> receiving
>> > >> >>>>>>> collection server is seeing.
>> > >> >>>>>>>
>> > >> >>>>>>> My hope was that this could be affected by RSYSLOG on the
>> client
>> > >> >> device
>> > >> >>>>>> but
>> > >> >>>>>>> perhaps not. I'll also look into AWS to see if a dynamically
>> > >> created
>> > >> >>>>>>> compute resource can automatically be registered with DNS.
>> > >> >>>>>>>
>> > >> >>>>>>> If anything else comes to mind, let me know. As always, I
>> > >> appreciate
>> > >> >>>> your
>> > >> >>>>>>> feedback.
>> > >> >>>>>>>
>> > >> >>>>>>> *Scott Slattery*
>> > >> >>>>>>>
>> > >> >>>>>>> *Sr. Enterprise/Cloud Architect*
>> > >> >>>>>>>
>> > >> >>>>>>> *Cloud, Compute, Information & Architecture Team*
>> > >> >>>>>>>
>> > >> >>>>>>> motorolasolutions.com
>> > >> >>>>>>>
>> > >> >>>>>>> *O: 602.529.8226*
>> > >> >>>>>>>
>> > >> >>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>> On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm>
>> wrote:
>> > >> >>>>>>>
>> > >> >>>>>>>> Linux has a rather sophisticated mechanism for plugging in
>> > >> arbitrary
>> > >> >>>>>> ways
>> > >> >>>>>>>> of
>> > >> >>>>>>>> doing name resolution. DNS has 'won' but hitorically there
>> have
>> > >> been
>> > >> >>>>>> many
>> > >> >>>>>>>> other
>> > >> >>>>>>>> options. Research nsswitch (/etc/nsswitch.conf) and see if
>> there
>> > >> is
>> > >> >>>>>>>> something
>> > >> >>>>>>>> that you can leverage.
>> > >> >>>>>>>>
>> > >> >>>>>>>> or, if you can set the hostname of the resources as they are
>> > >> created
>> > >> >>>> to
>> > >> >>>>>> be
>> > >> >>>>>>>> some
>> > >> >>>>>>>> predicatable pattern rather than the AWS default of IP
>> based, you
>> > >> >> can
>> > >> >>>>>> then
>> > >> >>>>>>>> make
>> > >> >>>>>>>> your logic use that. (This is the approach I would look
>> into).
>> > >> What
>> > >> >>>>>>>> mechanism
>> > >> >>>>>>>> this will be will depend on how you are
>> configuring/provisioning
>> > >> the
>> > >> >>>>>>>> systems.
>> > >> >>>>>>>>
>> > >> >>>>>>>> David Lang
>> > >> >>>>>>>>
>> > >> >>>>>>>>
>> > >> >>>>>>>>
>> > >> >>>>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>> > >> >>>>>>>>
>> > >> >>>>>>>>> Date: Tue, 16 Nov 2021 15:14:51 -0700
>> > >> >>>>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com
>> >
>> > >> >>>>>>>>> To: David Lang <david@lang.hm>
>> > >> >>>>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>> > >> >>>>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log
>> collector
>> > >> >>>>>>>>>
>> > >> >>>>>>>>> Thanks, David, I was hoping this was possible. Since the
>> compute
>> > >> >>>>>>>> resources
>> > >> >>>>>>>>> are dynamic, using any sort of local /etc/hosts would be
>> > >> impossible
>> > >> >>>>>> since
>> > >> >>>>>>>>> the IP are unpredictable. Can you point me to how I would
>> do
>> > >> this
>> > >> >> on
>> > >> >>>>>> the
>> > >> >>>>>>>>> client-server?
>> > >> >>>>>>>>>
>> > >> >>>>>>>>> Thanks
>> > >> >>>>>>>>>
>> > >> >>>>>>>>> *Scott Slattery*
>> > >> >>>>>>>>>
>> > >> >>>>>>>>> *Sr. Enterprise/Cloud Architect*
>> > >> >>>>>>>>>
>> > >> >>>>>>>>> *Cloud, Compute, Information & Architecture Team*
>> > >> >>>>>>>>>
>> > >> >>>>>>>>> motorolasolutions.com
>> > >> >>>>>>>>>
>> > >> >>>>>>>>> *O: 602.529.8226*
>> > >> >>>>>>>>>
>> > >> >>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>> > >> >>>>>>>>>
>> > >> >>>>>>>>>
>> > >> >>>>>>>>>
>> > >> >>>>>>>>>
>> > >> >>>>>>>>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm>
>> > >> wrote:
>> > >> >>>>>>>>>
>> > >> >>>>>>>>>> fromhost is the result of a name lookup of fromhost-ip.
>> On the
>> > >> >>>>>> receiver,
>> > >> >>>>>>>>>> you can
>> > >> >>>>>>>>>> control this with your name resolution (DNS, /etc/hosts,
>> other
>> > >> >>>>>>>> mechanisms)
>> > >> >>>>>>>>>>
>> > >> >>>>>>>>>> but a better option would probably be to set the hostname
>> on
>> > >> the
>> > >> >>>>>> sender.
>> > >> >>>>>>>>>> The
>> > >> >>>>>>>>>> hostname field in the message is under the full control
>> of the
>> > >> >>>> sender.
>> > >> >>>>>>>>>>
>> > >> >>>>>>>>>> David Lang
>> > >> >>>>>>>>>>
>> > >> >>>>>>>>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
>> > >> >>>>>>>>>>
>> > >> >>>>>>>>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
>> > >> >>>>>>>>>>> From: Scott Slattery via rsyslog <
>> rsyslog@lists.adiscon.com>
>> > >> >>>>>>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>> > >> >>>>>>>>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com
>> >
>> > >> >>>>>>>>>>> Subject: [rsyslog] FROMHOST missing on central log
>> collector
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> Hello,
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> I have a central log server, many of them, using rsyslog
>> to
>> > >> >>>> aggregate
>> > >> >>>>>>>>>> logs
>> > >> >>>>>>>>>>> from remote servers. Everything works great but I have a
>> new
>> > >> >>>>>> challenge
>> > >> >>>>>>>>>> and
>> > >> >>>>>>>>>>> am hoping for some recommendations.
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> I have a number of AWS auto-scaling groups where compute
>> > >> >> resources
>> > >> >>>>>> are
>> > >> >>>>>>>>>>> dynamically scaled up and down. Each of these will have a
>> > >> custom
>> > >> >>>>>>>> rsyslog
>> > >> >>>>>>>>>>> configuration pulled from the AWS AMI.
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> These dynamic resources are not added to DNS due to their
>> > >> dynamic
>> > >> >>>>>>>> nature
>> > >> >>>>>>>>>> so
>> > >> >>>>>>>>>>> they will not have DNS assigned FQDNs.
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> Because of the lack of a hostname, my central log server
>> is
>> > >> >> getting
>> > >> >>>>>>>> only
>> > >> >>>>>>>>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> So what I'm seeing today looks like
>> > >> '10.38.134.77-10.38.134.77'
>> > >> >>>>>> where I
>> > >> >>>>>>>>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> What I'd want to do is have easy resource send using the
>> same
>> > >> >>>>>> hostname
>> > >> >>>>>>>>>> and
>> > >> >>>>>>>>>>> current IP. This later will allow me to aggregate all
>> > >> resources
>> > >> >> by
>> > >> >>>>>>>> name.
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> I did not see any way of affecting the FROMHOST
>> information
>> > >> >> unless,
>> > >> >>>>>> on
>> > >> >>>>>>>>>> the
>> > >> >>>>>>>>>>> collector, I have rules based on IP address which isn't
>> > >> optimal
>> > >> >>>> given
>> > >> >>>>>>>> the
>> > >> >>>>>>>>>>> dynamic nature of the IPs changing.
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> Any suggestion is appreciated.
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> *Scott Slattery*
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> *Sr. Enterprise/Cloud Architect*
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> *Cloud, Compute, Information & Architecture Team*
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> motorolasolutions.com
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> *O: 602.529.8226*
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>>
>> > >> >>>>>>>>>>
>> > >> >>>>>>>>>
>> > >> >>>>>>>>>
>> > >> >>>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>>
>> > >> >>>>>>
>> > >> >>>>>
>> > >> >>>>>
>> > >> >>>>
>> > >> >>>
>> > >> >>>
>> > >> >>
>> > >> >
>> > >> >
>> > >> _______________________________________________
>> > >> rsyslog mailing list
>> > >>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=byZQ4B17TpYXwp7w16sjgq1YwmV4o6O3wLuGRq16WDJzp5U81Xv4xRKNkRfPWUL_&s=LRn-pDlin6SMuY_Ncd7AA0guWZdN3zaGu3Hu0ozAlxA&e=
>> > >> <
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=db-lyqaTcrex58uwzOcY54hh137E9JMAF6vN-1IWnsA&e=
>> >
>> > >>
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=byZQ4B17TpYXwp7w16sjgq1YwmV4o6O3wLuGRq16WDJzp5U81Xv4xRKNkRfPWUL_&s=BDcnrfghWuu5lVf7v9HG439toMBot9PrkNGcZcJblv8&e=
>> > >> <
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=eGWs1Xi6yCyCD3OYNlbvl3fIYBADttEDYjwGyicAZbk&e=
>> >
>> > >> What's up with rsyslog? Follow
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=byZQ4B17TpYXwp7w16sjgq1YwmV4o6O3wLuGRq16WDJzp5U81Xv4xRKNkRfPWUL_&s=T8KQImFltarg-g3d4mjbAC_qvd8mFV1z8kxk_CBWj8k&e=
>> > >> <
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=KIBqHKSAQtwhZA0rXY7Uh_or50wek4ABsH6-S4pxX0c&e=
>> >
>> > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad
>> > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
>> you
>> > >> DON'T LIKE THAT.
>> > >>
>> > >
>> > >
>> > > --
>> > > Yury Bushmelev
>> > >
>> >
>> > --
>> >
>> >
>> > *For more information on how and why we collect your personal
>> > information, please visit our Privacy Policy
>> > <
>> https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement
>> >.*
>> > _______________________________________________
>> > rsyslog mailing list
>> >
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=byZQ4B17TpYXwp7w16sjgq1YwmV4o6O3wLuGRq16WDJzp5U81Xv4xRKNkRfPWUL_&s=LRn-pDlin6SMuY_Ncd7AA0guWZdN3zaGu3Hu0ozAlxA&e=
>> >
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=byZQ4B17TpYXwp7w16sjgq1YwmV4o6O3wLuGRq16WDJzp5U81Xv4xRKNkRfPWUL_&s=BDcnrfghWuu5lVf7v9HG439toMBot9PrkNGcZcJblv8&e=
>> > What's up with rsyslog? Follow
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=byZQ4B17TpYXwp7w16sjgq1YwmV4o6O3wLuGRq16WDJzp5U81Xv4xRKNkRfPWUL_&s=T8KQImFltarg-g3d4mjbAC_qvd8mFV1z8kxk_CBWj8k&e=
>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
>> you DON'T LIKE THAT.
>>
>
> *For more information on how and why we collect your personal information,
> please visit our Privacy Policy
> <https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement>.*
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
on the receiver:

HOSTNAME is what the sender put in the message
FROMHOST-IP is the IP of the last hop before the receiver (in your case the
sender)
FROMHOST is a name lookup of FROMHOST-IP

if everything has the correct name, and that name is in DNS/hosts, then FROMHOST
and HOSTNAME may be the same, or FROMHOST may be FQDN while HOSTNAME is just the
short name (depends on the sending system)

if you can set the HOSTNAME before rsyslog starts so that it's in the message
itself correctly, then you don't need to depend on updating name resolution (and
name lookups cost time and there is always a lag between a system starting up
and when the update will show up to a lookup on the receiver)

David Lang


On Wed, 17 Nov 2021, Scott Slattery wrote:

> Date: Wed, 17 Nov 2021 09:43:47 -0700
> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> To: Yuri Bushmelev <jay4mail@gmail.com>
> Cc: rsyslog-users <rsyslog@lists.adiscon.com>, David Lang <david@lang.hm>
> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>
> Thanks for your feedback. There seems to be some understanding that the
> hostname is not set properly on the client-side. This is not the case, the
> hostname displays properly on the host itself and is also properly
> configured from a linux perspective. This is precisely why I'm inquiring
> about alternatives. The only differentiating factor with respect to these
> dynamically created hosts is that they do not get registered in DNS since
> their life is, or can be, quite short based on computing demand.
>
> I was under the impression that the hostname used by the server-side
> (collector) was the result of a server-side DNS lookup, which will not
> resolve for these hosts. This is why I was looking for a rsyslog solution
> that didn't involved DNS.
>
> Yuri, if I understand you correctly you're saying a custom template using
> HOSTNAME vs FROMHOST-IP may be an option. I'll look into this to understand
> it better. Thanks for this suggestion, it sound like it completely removes
> the DNS constraint. I'll give it a try.
>
> *Scott Slattery*
>
> *Sr. Enterprise/Cloud Architect*
>
> *Cloud, Compute, Information & Architecture Team*
>
> motorolasolutions.com
>
> *O: 602.529.8226*
>
> *E*: Scott.Slattery@MotorolaSolutions.com
>
>
>
>
> On Tue, Nov 16, 2021 at 7:15 PM Yuri Bushmelev <jay4mail@gmail.com> wrote:
>
>> Hello!
>>
>> Just a reminder that a hostname field in a syslog message is just a string
>> sent from sender to collector. So you can craft a custom template with the
>> hostname field defined as you'd like. Though I'd call this a "fallback" way
>> of fixing the issue. The right way is to set the proper hostname on a
>> sender system before rsyslog starts I'd say.
>>
>> On Wed, 17 Nov 2021 at 08:33, David Lang via rsyslog <
>> rsyslog@lists.adiscon.com> wrote:
>>
>>> Rsyslog looks up the hostname as it starts up, so if something after
>>> rsyslog
>>> starts changes the hostname, rsyslog isn't going to notice until you
>>> restart
>>> rsyslog.
>>>
>>> again, fromhost is a receiver side lookup of the name to match
>>> fromhost-ip, so
>>> if hostname is getting set correctly, filter on that instead of on
>>> fromhost.
>>>
>>> David Lang
>>>
>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>>
>>>> Date: Tue, 16 Nov 2021 17:28:15 -0700
>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>> To: David Lang <david@lang.hm>
>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>>
>>>> Thanks, David, I think you've done more than enough to try and help me
>>> on
>>>> this. I need to do some reading on Amazon (and the link you shared) to
>>> see
>>>> what my options are. I agree with you, it's likely workable.
>>>>
>>>> I've confirmed that the results from the 'hostname' command do match so
>>>> it's a bit of a mystery why rsyslog doesn't detect this but, i think
>>> you're
>>>> on the right track, we need to run a post-deployment script to get these
>>>> instances registered in Route53.
>>>>
>>>>
>>>> *Scott Slattery*
>>>>
>>>> *Sr. Enterprise/Cloud Architect*
>>>>
>>>> *Cloud, Compute, Information & Architecture Team*
>>>>
>>>> motorolasolutions.com
>>>>
>>>> *O: 602.529.8226*
>>>>
>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Nov 16, 2021 at 5:20 PM David Lang <david@lang.hm> wrote:
>>>>
>>>>> if you login to one of the systems, you should find that the name
>>> returned
>>>>> by
>>>>> the hostname command should match what you get in the syslog message
>>> that
>>>>> is
>>>>> delivered to your central collector. (if it doesn't, try restarting
>>>>> rsyslog and
>>>>> see if it changes to match)
>>>>>
>>>>> then the question becomes what mechansims does AMI provide for
>>> customizing
>>>>> the
>>>>> hostname
>>>>>
>>>>> a quick google search shows a new hostnamectl command
>>>>>
>>>>>
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.aws.amazon.com_AWSEC2_latest_UserGuide_set-2Dhostname.html&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=WR-Pz8svN0d8vqg4ZKSNj2dbxtcngaMJ4iiRXCPpD6c&e=
>>>>>
>>>>>
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.cyberciti.biz_faq_set-2Dchange-2Dhostname-2Din-2Damazon-2Dlinux-2Dec2-2Dinstance-2Dserver_&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=2RI1Khq-fBEBJxckXk9nWDESN8pTJxGiGv6xpsbYhzE&e=
>>>>>
>>>>> I know there is a way for you to specify a script to run when an
>>> instance
>>>>> is
>>>>> started, that script can then set things like this. I don't know
>>> enough to
>>>>> point
>>>>> you at specifically how to do that.
>>>>>
>>>>> David Lang
>>>>>
>>>>>
>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>>>>
>>>>>> Date: Tue, 16 Nov 2021 17:07:47 -0700
>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>>> To: David Lang <david@lang.hm>
>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>>>>
>>>>>> Thanks David, the hostname is currently set in the AMI (Amazon Master
>>>>>> Image) which is the source image for all instances that are
>>> dynamically
>>>>>> created and I can verify that, if you login to one of these dynamic
>>>>>> instances, the hostname is in fact set correctly.
>>>>>>
>>>>>> The issue doesn't seem particularly related to what is set in
>>>>>> /etc/hostname, /etc/hosts, or what was set using 'hostname' command. I
>>>>>> think you can see this is the source of my frustration. It appears the
>>>>>> central log collector relies only on DNS resolution unless there's
>>> some
>>>>>> hidden magic inside RSYSLOG to force the sent logs to include a host
>>>>> header
>>>>>> (vs DNS).
>>>>>>
>>>>>> I don't want to continue wasting your time but again, it is much
>>>>>> appreciated. I'll look into some way of dynamically adding these
>>> hosts to
>>>>>> DNS in AWS Route53. It appears rsyslog simply can't do what I'm after.
>>>>>>
>>>>>>
>>>>>> *Scott Slattery*
>>>>>>
>>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>>
>>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>>
>>>>>> motorolasolutions.com
>>>>>>
>>>>>> *O: 602.529.8226*
>>>>>>
>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Nov 16, 2021 at 5:02 PM David Lang <david@lang.hm> wrote:
>>>>>>
>>>>>>> the hostname command will let you set the hostname (you want to do
>>> that
>>>>>>> before
>>>>>>> you start rsyslog). I would expect that the orcastration tool you
>>> use to
>>>>>>> create
>>>>>>> the systems will have some 'correct for that tool' way to set the
>>>>> hostname
>>>>>>> as it
>>>>>>> starts the instance (sorry I can't provide more specifics, if you can
>>>>>>> mention
>>>>>>> what you are using, possibly someone else can chime in on the best
>>> way
>>>>> to
>>>>>>> set
>>>>>>> the hostname with that tool)
>>>>>>>
>>>>>>> David Lang
>>>>>>>
>>>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>>>>>>
>>>>>>>> Date: Tue, 16 Nov 2021 16:59:17 -0700
>>>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>>>>> To: David Lang <david@lang.hm>
>>>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>>>>>>
>>>>>>>> My follow-on question woudl be how do I set the hostname at the
>>> client
>>>>>>> end?
>>>>>>>> Other than what's in /etc/hosts, /etc/hostname, etc. I don't know
>>> how
>>>>>>> else
>>>>>>>> I would affect the log being sent to ensure it's going over.
>>>>>>>>
>>>>>>>> *Scott Slattery*
>>>>>>>>
>>>>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>>>>
>>>>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>>>>
>>>>>>>> motorolasolutions.com
>>>>>>>>
>>>>>>>> *O: 602.529.8226*
>>>>>>>>
>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Nov 16, 2021 at 4:55 PM David Lang <david@lang.hm> wrote:
>>>>>>>>
>>>>>>>>> the translation from fromhost-ip to fromhost is done at the
>>> collector,
>>>>>>> but
>>>>>>>>> the
>>>>>>>>> sender sets the hostname field. If you can trust that hostname was
>>> set
>>>>>>>>> correctly, there is no reason to use fromhost
>>>>>>>>>
>>>>>>>>> David Lang
>>>>>>>>>
>>>>>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>>>>>>>>
>>>>>>>>>> Date: Tue, 16 Nov 2021 16:53:19 -0700
>>>>>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>>>>>>> To: David Lang <david@lang.hm>
>>>>>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>>>>>>>>
>>>>>>>>>> Thanks David, I could be wrong but the resolution seems to be
>>>>> happening
>>>>>>>>> at
>>>>>>>>>> the log collection server, not the client end. Given this, I'm not
>>>>> sure
>>>>>>>>>> anything outside of rsyslog on the client would affect what the
>>>>>>> receiving
>>>>>>>>>> collection server is seeing.
>>>>>>>>>>
>>>>>>>>>> My hope was that this could be affected by RSYSLOG on the client
>>>>> device
>>>>>>>>> but
>>>>>>>>>> perhaps not. I'll also look into AWS to see if a dynamically
>>> created
>>>>>>>>>> compute resource can automatically be registered with DNS.
>>>>>>>>>>
>>>>>>>>>> If anything else comes to mind, let me know. As always, I
>>> appreciate
>>>>>>> your
>>>>>>>>>> feedback.
>>>>>>>>>>
>>>>>>>>>> *Scott Slattery*
>>>>>>>>>>
>>>>>>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>>>>>>
>>>>>>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>>>>>>
>>>>>>>>>> motorolasolutions.com
>>>>>>>>>>
>>>>>>>>>> *O: 602.529.8226*
>>>>>>>>>>
>>>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm> wrote:
>>>>>>>>>>
>>>>>>>>>>> Linux has a rather sophisticated mechanism for plugging in
>>> arbitrary
>>>>>>>>> ways
>>>>>>>>>>> of
>>>>>>>>>>> doing name resolution. DNS has 'won' but hitorically there have
>>> been
>>>>>>>>> many
>>>>>>>>>>> other
>>>>>>>>>>> options. Research nsswitch (/etc/nsswitch.conf) and see if there
>>> is
>>>>>>>>>>> something
>>>>>>>>>>> that you can leverage.
>>>>>>>>>>>
>>>>>>>>>>> or, if you can set the hostname of the resources as they are
>>> created
>>>>>>> to
>>>>>>>>> be
>>>>>>>>>>> some
>>>>>>>>>>> predicatable pattern rather than the AWS default of IP based, you
>>>>> can
>>>>>>>>> then
>>>>>>>>>>> make
>>>>>>>>>>> your logic use that. (This is the approach I would look into).
>>> What
>>>>>>>>>>> mechanism
>>>>>>>>>>> this will be will depend on how you are configuring/provisioning
>>> the
>>>>>>>>>>> systems.
>>>>>>>>>>>
>>>>>>>>>>> David Lang
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Date: Tue, 16 Nov 2021 15:14:51 -0700
>>>>>>>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>>>>>>>>> To: David Lang <david@lang.hm>
>>>>>>>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>>>>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks, David, I was hoping this was possible. Since the compute
>>>>>>>>>>> resources
>>>>>>>>>>>> are dynamic, using any sort of local /etc/hosts would be
>>> impossible
>>>>>>>>> since
>>>>>>>>>>>> the IP are unpredictable. Can you point me to how I would do
>>> this
>>>>> on
>>>>>>>>> the
>>>>>>>>>>>> client-server?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks
>>>>>>>>>>>>
>>>>>>>>>>>> *Scott Slattery*
>>>>>>>>>>>>
>>>>>>>>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>>>>>>>>
>>>>>>>>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>>>>>>>>
>>>>>>>>>>>> motorolasolutions.com
>>>>>>>>>>>>
>>>>>>>>>>>> *O: 602.529.8226*
>>>>>>>>>>>>
>>>>>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm>
>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> fromhost is the result of a name lookup of fromhost-ip. On the
>>>>>>>>> receiver,
>>>>>>>>>>>>> you can
>>>>>>>>>>>>> control this with your name resolution (DNS, /etc/hosts, other
>>>>>>>>>>> mechanisms)
>>>>>>>>>>>>>
>>>>>>>>>>>>> but a better option would probably be to set the hostname on
>>> the
>>>>>>>>> sender.
>>>>>>>>>>>>> The
>>>>>>>>>>>>> hostname field in the message is under the full control of the
>>>>>>> sender.
>>>>>>>>>>>>>
>>>>>>>>>>>>> David Lang
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
>>>>>>>>>>>>>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
>>>>>>>>>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>>>>>>>>>>>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
>>>>>>>>>>>>>> Subject: [rsyslog] FROMHOST missing on central log collector
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have a central log server, many of them, using rsyslog to
>>>>>>> aggregate
>>>>>>>>>>>>> logs
>>>>>>>>>>>>>> from remote servers. Everything works great but I have a new
>>>>>>>>> challenge
>>>>>>>>>>>>> and
>>>>>>>>>>>>>> am hoping for some recommendations.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have a number of AWS auto-scaling groups where compute
>>>>> resources
>>>>>>>>> are
>>>>>>>>>>>>>> dynamically scaled up and down. Each of these will have a
>>> custom
>>>>>>>>>>> rsyslog
>>>>>>>>>>>>>> configuration pulled from the AWS AMI.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> These dynamic resources are not added to DNS due to their
>>> dynamic
>>>>>>>>>>> nature
>>>>>>>>>>>>> so
>>>>>>>>>>>>>> they will not have DNS assigned FQDNs.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Because of the lack of a hostname, my central log server is
>>>>> getting
>>>>>>>>>>> only
>>>>>>>>>>>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So what I'm seeing today looks like
>>> '10.38.134.77-10.38.134.77'
>>>>>>>>> where I
>>>>>>>>>>>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What I'd want to do is have easy resource send using the same
>>>>>>>>> hostname
>>>>>>>>>>>>> and
>>>>>>>>>>>>>> current IP. This later will allow me to aggregate all
>>> resources
>>>>> by
>>>>>>>>>>> name.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I did not see any way of affecting the FROMHOST information
>>>>> unless,
>>>>>>>>> on
>>>>>>>>>>>>> the
>>>>>>>>>>>>>> collector, I have rules based on IP address which isn't
>>> optimal
>>>>>>> given
>>>>>>>>>>> the
>>>>>>>>>>>>>> dynamic nature of the IPs changing.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Any suggestion is appreciated.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Scott Slattery*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Sr. Enterprise/Cloud Architect*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Cloud, Compute, Information & Architecture Team*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> motorolasolutions.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *O: 602.529.8226*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=db-lyqaTcrex58uwzOcY54hh137E9JMAF6vN-1IWnsA&e=>
>>> http://www.rsyslog.com/professional-services/
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=eGWs1Xi6yCyCD3OYNlbvl3fIYBADttEDYjwGyicAZbk&e=>
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=KIBqHKSAQtwhZA0rXY7Uh_or50wek4ABsH6-S4pxX0c&e=>
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>> DON'T LIKE THAT.
>>>
>>
>>
>> --
>> Yury Bushmelev
>>
>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: FROMHOST missing on central log collector [ In reply to ]
Thank you David, I agree with everything you just said. I'm hoping to get
back to this little issue but today has been a bit of a disaster. My thanks
to everyone who responded with so many thoughts. I will report back
hopefully tomorrow once I've made some changes and tested them.

*Scott Slattery*

*Sr. Enterprise/Cloud Architect*

*Cloud, Compute, Information & Architecture Team*

motorolasolutions.com

*O: 602.529.8226*

*E*: Scott.Slattery@MotorolaSolutions.com




On Wed, Nov 17, 2021 at 10:51 AM David Lang <david@lang.hm> wrote:

> on the receiver:
>
> HOSTNAME is what the sender put in the message
> FROMHOST-IP is the IP of the last hop before the receiver (in your case
> the
> sender)
> FROMHOST is a name lookup of FROMHOST-IP
>
> if everything has the correct name, and that name is in DNS/hosts, then
> FROMHOST
> and HOSTNAME may be the same, or FROMHOST may be FQDN while HOSTNAME is
> just the
> short name (depends on the sending system)
>
> if you can set the HOSTNAME before rsyslog starts so that it's in the
> message
> itself correctly, then you don't need to depend on updating name
> resolution (and
> name lookups cost time and there is always a lag between a system starting
> up
> and when the update will show up to a lookup on the receiver)
>
> David Lang
>
>
> On Wed, 17 Nov 2021, Scott Slattery wrote:
>
> > Date: Wed, 17 Nov 2021 09:43:47 -0700
> > From: Scott Slattery <scott.slattery@motorolasolutions.com>
> > To: Yuri Bushmelev <jay4mail@gmail.com>
> > Cc: rsyslog-users <rsyslog@lists.adiscon.com>, David Lang <david@lang.hm
> >
> > Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >
> > Thanks for your feedback. There seems to be some understanding that the
> > hostname is not set properly on the client-side. This is not the case,
> the
> > hostname displays properly on the host itself and is also properly
> > configured from a linux perspective. This is precisely why I'm inquiring
> > about alternatives. The only differentiating factor with respect to these
> > dynamically created hosts is that they do not get registered in DNS since
> > their life is, or can be, quite short based on computing demand.
> >
> > I was under the impression that the hostname used by the server-side
> > (collector) was the result of a server-side DNS lookup, which will not
> > resolve for these hosts. This is why I was looking for a rsyslog solution
> > that didn't involved DNS.
> >
> > Yuri, if I understand you correctly you're saying a custom template using
> > HOSTNAME vs FROMHOST-IP may be an option. I'll look into this to
> understand
> > it better. Thanks for this suggestion, it sound like it completely
> removes
> > the DNS constraint. I'll give it a try.
> >
> > *Scott Slattery*
> >
> > *Sr. Enterprise/Cloud Architect*
> >
> > *Cloud, Compute, Information & Architecture Team*
> >
> > motorolasolutions.com
> >
> > *O: 602.529.8226*
> >
> > *E*: Scott.Slattery@MotorolaSolutions.com
> >
> >
> >
> >
> > On Tue, Nov 16, 2021 at 7:15 PM Yuri Bushmelev <jay4mail@gmail.com>
> wrote:
> >
> >> Hello!
> >>
> >> Just a reminder that a hostname field in a syslog message is just a
> string
> >> sent from sender to collector. So you can craft a custom template with
> the
> >> hostname field defined as you'd like. Though I'd call this a "fallback"
> way
> >> of fixing the issue. The right way is to set the proper hostname on a
> >> sender system before rsyslog starts I'd say.
> >>
> >> On Wed, 17 Nov 2021 at 08:33, David Lang via rsyslog <
> >> rsyslog@lists.adiscon.com> wrote:
> >>
> >>> Rsyslog looks up the hostname as it starts up, so if something after
> >>> rsyslog
> >>> starts changes the hostname, rsyslog isn't going to notice until you
> >>> restart
> >>> rsyslog.
> >>>
> >>> again, fromhost is a receiver side lookup of the name to match
> >>> fromhost-ip, so
> >>> if hostname is getting set correctly, filter on that instead of on
> >>> fromhost.
> >>>
> >>> David Lang
> >>>
> >>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>>
> >>>> Date: Tue, 16 Nov 2021 17:28:15 -0700
> >>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>> To: David Lang <david@lang.hm>
> >>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>>
> >>>> Thanks, David, I think you've done more than enough to try and help me
> >>> on
> >>>> this. I need to do some reading on Amazon (and the link you shared) to
> >>> see
> >>>> what my options are. I agree with you, it's likely workable.
> >>>>
> >>>> I've confirmed that the results from the 'hostname' command do match
> so
> >>>> it's a bit of a mystery why rsyslog doesn't detect this but, i think
> >>> you're
> >>>> on the right track, we need to run a post-deployment script to get
> these
> >>>> instances registered in Route53.
> >>>>
> >>>>
> >>>> *Scott Slattery*
> >>>>
> >>>> *Sr. Enterprise/Cloud Architect*
> >>>>
> >>>> *Cloud, Compute, Information & Architecture Team*
> >>>>
> >>>> motorolasolutions.com
> >>>>
> >>>> *O: 602.529.8226*
> >>>>
> >>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Tue, Nov 16, 2021 at 5:20 PM David Lang <david@lang.hm> wrote:
> >>>>
> >>>>> if you login to one of the systems, you should find that the name
> >>> returned
> >>>>> by
> >>>>> the hostname command should match what you get in the syslog message
> >>> that
> >>>>> is
> >>>>> delivered to your central collector. (if it doesn't, try restarting
> >>>>> rsyslog and
> >>>>> see if it changes to match)
> >>>>>
> >>>>> then the question becomes what mechansims does AMI provide for
> >>> customizing
> >>>>> the
> >>>>> hostname
> >>>>>
> >>>>> a quick google search shows a new hostnamectl command
> >>>>>
> >>>>>
> >>>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.aws.amazon.com_AWSEC2_latest_UserGuide_set-2Dhostname.html&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=WR-Pz8svN0d8vqg4ZKSNj2dbxtcngaMJ4iiRXCPpD6c&e=
> >>>>>
> >>>>>
> >>>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.cyberciti.biz_faq_set-2Dchange-2Dhostname-2Din-2Damazon-2Dlinux-2Dec2-2Dinstance-2Dserver_&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=1JiTvUcvjB8RxLP9vfPbVSsbAQyitkPK6AzRBhEBUjRBWl-3tAtfR73TCtIFhdHZ&s=2RI1Khq-fBEBJxckXk9nWDESN8pTJxGiGv6xpsbYhzE&e=
> >>>>>
> >>>>> I know there is a way for you to specify a script to run when an
> >>> instance
> >>>>> is
> >>>>> started, that script can then set things like this. I don't know
> >>> enough to
> >>>>> point
> >>>>> you at specifically how to do that.
> >>>>>
> >>>>> David Lang
> >>>>>
> >>>>>
> >>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>>>>
> >>>>>> Date: Tue, 16 Nov 2021 17:07:47 -0700
> >>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>>> To: David Lang <david@lang.hm>
> >>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>>>>
> >>>>>> Thanks David, the hostname is currently set in the AMI (Amazon
> Master
> >>>>>> Image) which is the source image for all instances that are
> >>> dynamically
> >>>>>> created and I can verify that, if you login to one of these dynamic
> >>>>>> instances, the hostname is in fact set correctly.
> >>>>>>
> >>>>>> The issue doesn't seem particularly related to what is set in
> >>>>>> /etc/hostname, /etc/hosts, or what was set using 'hostname'
> command. I
> >>>>>> think you can see this is the source of my frustration. It appears
> the
> >>>>>> central log collector relies only on DNS resolution unless there's
> >>> some
> >>>>>> hidden magic inside RSYSLOG to force the sent logs to include a host
> >>>>> header
> >>>>>> (vs DNS).
> >>>>>>
> >>>>>> I don't want to continue wasting your time but again, it is much
> >>>>>> appreciated. I'll look into some way of dynamically adding these
> >>> hosts to
> >>>>>> DNS in AWS Route53. It appears rsyslog simply can't do what I'm
> after.
> >>>>>>
> >>>>>>
> >>>>>> *Scott Slattery*
> >>>>>>
> >>>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>>
> >>>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>>
> >>>>>> motorolasolutions.com
> >>>>>>
> >>>>>> *O: 602.529.8226*
> >>>>>>
> >>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Nov 16, 2021 at 5:02 PM David Lang <david@lang.hm> wrote:
> >>>>>>
> >>>>>>> the hostname command will let you set the hostname (you want to do
> >>> that
> >>>>>>> before
> >>>>>>> you start rsyslog). I would expect that the orcastration tool you
> >>> use to
> >>>>>>> create
> >>>>>>> the systems will have some 'correct for that tool' way to set the
> >>>>> hostname
> >>>>>>> as it
> >>>>>>> starts the instance (sorry I can't provide more specifics, if you
> can
> >>>>>>> mention
> >>>>>>> what you are using, possibly someone else can chime in on the best
> >>> way
> >>>>> to
> >>>>>>> set
> >>>>>>> the hostname with that tool)
> >>>>>>>
> >>>>>>> David Lang
> >>>>>>>
> >>>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>>>>>>
> >>>>>>>> Date: Tue, 16 Nov 2021 16:59:17 -0700
> >>>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>>>>> To: David Lang <david@lang.hm>
> >>>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>>>>>>
> >>>>>>>> My follow-on question woudl be how do I set the hostname at the
> >>> client
> >>>>>>> end?
> >>>>>>>> Other than what's in /etc/hosts, /etc/hostname, etc. I don't know
> >>> how
> >>>>>>> else
> >>>>>>>> I would affect the log being sent to ensure it's going over.
> >>>>>>>>
> >>>>>>>> *Scott Slattery*
> >>>>>>>>
> >>>>>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>>>>
> >>>>>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>>>>
> >>>>>>>> motorolasolutions.com
> >>>>>>>>
> >>>>>>>> *O: 602.529.8226*
> >>>>>>>>
> >>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Nov 16, 2021 at 4:55 PM David Lang <david@lang.hm> wrote:
> >>>>>>>>
> >>>>>>>>> the translation from fromhost-ip to fromhost is done at the
> >>> collector,
> >>>>>>> but
> >>>>>>>>> the
> >>>>>>>>> sender sets the hostname field. If you can trust that hostname
> was
> >>> set
> >>>>>>>>> correctly, there is no reason to use fromhost
> >>>>>>>>>
> >>>>>>>>> David Lang
> >>>>>>>>>
> >>>>>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>>>>>>>>
> >>>>>>>>>> Date: Tue, 16 Nov 2021 16:53:19 -0700
> >>>>>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>>>>>>> To: David Lang <david@lang.hm>
> >>>>>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>>>>>>>>
> >>>>>>>>>> Thanks David, I could be wrong but the resolution seems to be
> >>>>> happening
> >>>>>>>>> at
> >>>>>>>>>> the log collection server, not the client end. Given this, I'm
> not
> >>>>> sure
> >>>>>>>>>> anything outside of rsyslog on the client would affect what the
> >>>>>>> receiving
> >>>>>>>>>> collection server is seeing.
> >>>>>>>>>>
> >>>>>>>>>> My hope was that this could be affected by RSYSLOG on the client
> >>>>> device
> >>>>>>>>> but
> >>>>>>>>>> perhaps not. I'll also look into AWS to see if a dynamically
> >>> created
> >>>>>>>>>> compute resource can automatically be registered with DNS.
> >>>>>>>>>>
> >>>>>>>>>> If anything else comes to mind, let me know. As always, I
> >>> appreciate
> >>>>>>> your
> >>>>>>>>>> feedback.
> >>>>>>>>>>
> >>>>>>>>>> *Scott Slattery*
> >>>>>>>>>>
> >>>>>>>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>>>>>>
> >>>>>>>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>>>>>>
> >>>>>>>>>> motorolasolutions.com
> >>>>>>>>>>
> >>>>>>>>>> *O: 602.529.8226*
> >>>>>>>>>>
> >>>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm>
> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Linux has a rather sophisticated mechanism for plugging in
> >>> arbitrary
> >>>>>>>>> ways
> >>>>>>>>>>> of
> >>>>>>>>>>> doing name resolution. DNS has 'won' but hitorically there have
> >>> been
> >>>>>>>>> many
> >>>>>>>>>>> other
> >>>>>>>>>>> options. Research nsswitch (/etc/nsswitch.conf) and see if
> there
> >>> is
> >>>>>>>>>>> something
> >>>>>>>>>>> that you can leverage.
> >>>>>>>>>>>
> >>>>>>>>>>> or, if you can set the hostname of the resources as they are
> >>> created
> >>>>>>> to
> >>>>>>>>> be
> >>>>>>>>>>> some
> >>>>>>>>>>> predicatable pattern rather than the AWS default of IP based,
> you
> >>>>> can
> >>>>>>>>> then
> >>>>>>>>>>> make
> >>>>>>>>>>> your logic use that. (This is the approach I would look into).
> >>> What
> >>>>>>>>>>> mechanism
> >>>>>>>>>>> this will be will depend on how you are
> configuring/provisioning
> >>> the
> >>>>>>>>>>> systems.
> >>>>>>>>>>>
> >>>>>>>>>>> David Lang
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Date: Tue, 16 Nov 2021 15:14:51 -0700
> >>>>>>>>>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>>>>>>>>> To: David Lang <david@lang.hm>
> >>>>>>>>>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>>>>>>>>> Subject: Re: [rsyslog] FROMHOST missing on central log
> collector
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks, David, I was hoping this was possible. Since the
> compute
> >>>>>>>>>>> resources
> >>>>>>>>>>>> are dynamic, using any sort of local /etc/hosts would be
> >>> impossible
> >>>>>>>>> since
> >>>>>>>>>>>> the IP are unpredictable. Can you point me to how I would do
> >>> this
> >>>>> on
> >>>>>>>>> the
> >>>>>>>>>>>> client-server?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks
> >>>>>>>>>>>>
> >>>>>>>>>>>> *Scott Slattery*
> >>>>>>>>>>>>
> >>>>>>>>>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>>>>>>>>
> >>>>>>>>>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>>>>>>>>
> >>>>>>>>>>>> motorolasolutions.com
> >>>>>>>>>>>>
> >>>>>>>>>>>> *O: 602.529.8226*
> >>>>>>>>>>>>
> >>>>>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm>
> >>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> fromhost is the result of a name lookup of fromhost-ip. On
> the
> >>>>>>>>> receiver,
> >>>>>>>>>>>>> you can
> >>>>>>>>>>>>> control this with your name resolution (DNS, /etc/hosts,
> other
> >>>>>>>>>>> mechanisms)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> but a better option would probably be to set the hostname on
> >>> the
> >>>>>>>>> sender.
> >>>>>>>>>>>>> The
> >>>>>>>>>>>>> hostname field in the message is under the full control of
> the
> >>>>>>> sender.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> David Lang
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
> >>>>>>>>>>>>>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com
> >
> >>>>>>>>>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
> >>>>>>>>>>>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>>>>>>>>>>> Subject: [rsyslog] FROMHOST missing on central log collector
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hello,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I have a central log server, many of them, using rsyslog to
> >>>>>>> aggregate
> >>>>>>>>>>>>> logs
> >>>>>>>>>>>>>> from remote servers. Everything works great but I have a new
> >>>>>>>>> challenge
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>> am hoping for some recommendations.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I have a number of AWS auto-scaling groups where compute
> >>>>> resources
> >>>>>>>>> are
> >>>>>>>>>>>>>> dynamically scaled up and down. Each of these will have a
> >>> custom
> >>>>>>>>>>> rsyslog
> >>>>>>>>>>>>>> configuration pulled from the AWS AMI.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> These dynamic resources are not added to DNS due to their
> >>> dynamic
> >>>>>>>>>>> nature
> >>>>>>>>>>>>> so
> >>>>>>>>>>>>>> they will not have DNS assigned FQDNs.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Because of the lack of a hostname, my central log server is
> >>>>> getting
> >>>>>>>>>>> only
> >>>>>>>>>>>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> So what I'm seeing today looks like
> >>> '10.38.134.77-10.38.134.77'
> >>>>>>>>> where I
> >>>>>>>>>>>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> What I'd want to do is have easy resource send using the
> same
> >>>>>>>>> hostname
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>> current IP. This later will allow me to aggregate all
> >>> resources
> >>>>> by
> >>>>>>>>>>> name.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I did not see any way of affecting the FROMHOST information
> >>>>> unless,
> >>>>>>>>> on
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> collector, I have rules based on IP address which isn't
> >>> optimal
> >>>>>>> given
> >>>>>>>>>>> the
> >>>>>>>>>>>>>> dynamic nature of the IPs changing.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Any suggestion is appreciated.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> *Scott Slattery*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> motorolasolutions.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> *O: 602.529.8226*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=OTkFDMK4UI8CSuwUwUCQdf5V_5uP3xqV7DY2_qNb66RaJ3J7rqv3Dvv1r--1xkB2&s=VAJbMOODQwtGY_sg9fOx6BNjaFZo6oN-ZsOrmoml28k&e=
> >>> <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=db-lyqaTcrex58uwzOcY54hh137E9JMAF6vN-1IWnsA&e=
> >
> >>>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=OTkFDMK4UI8CSuwUwUCQdf5V_5uP3xqV7DY2_qNb66RaJ3J7rqv3Dvv1r--1xkB2&s=WYLn-D7Km8_dqGDaOm9uhF6VkWlH7JeSaMQGYnBeWtQ&e=
> >>> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=eGWs1Xi6yCyCD3OYNlbvl3fIYBADttEDYjwGyicAZbk&e=
> >
> >>> What's up with rsyslog? Follow
> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIBAg&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=OTkFDMK4UI8CSuwUwUCQdf5V_5uP3xqV7DY2_qNb66RaJ3J7rqv3Dvv1r--1xkB2&s=bRac5V8wSaaS-HxKWzbJVzw_fWGgGIxKNE34w-ZeXsU&e=
> >>> <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwMFaQ&c=q3cDpHe1hF8lXU5EFjNM_C93KOmcBXCBnhee2v6PYlc&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=lFiAzSG4O_IwoKCCbEi8i_yQYNFz5X0OWXMx9xGKGjYlvLeLrnYvixFB3egNPybF&s=KIBqHKSAQtwhZA0rXY7Uh_or50wek4ABsH6-S4pxX0c&e=
> >
> >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad
> >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> >>> DON'T LIKE THAT.
> >>>
> >>
> >>
> >> --
> >> Yury Bushmelev
> >>
> >
> >
>

--


*For more information on how and why we collect your personal
information, please visit our Privacy Policy
<https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement>.*
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.