Thanks David, the hostname is currently set in the AMI (Amazon Master
Image) which is the source image for all instances that are dynamically
created and I can verify that, if you login to one of these dynamic
instances, the hostname is in fact set correctly.
The issue doesn't seem particularly related to what is set in
/etc/hostname, /etc/hosts, or what was set using 'hostname' command. I
think you can see this is the source of my frustration. It appears the
central log collector relies only on DNS resolution unless there's some
hidden magic inside RSYSLOG to force the sent logs to include a host header
(vs DNS).
I don't want to continue wasting your time but again, it is much
appreciated. I'll look into some way of dynamically adding these hosts to
DNS in AWS Route53. It appears rsyslog simply can't do what I'm after.
*Scott Slattery*
*Sr. Enterprise/Cloud Architect*
*Cloud, Compute, Information & Architecture Team*
motorolasolutions.com
*O: 602.529.8226*
*E*: Scott.Slattery@MotorolaSolutions.com
On Tue, Nov 16, 2021 at 5:02 PM David Lang <david@lang.hm> wrote:
> the hostname command will let you set the hostname (you want to do that
> before
> you start rsyslog). I would expect that the orcastration tool you use to
> create
> the systems will have some 'correct for that tool' way to set the hostname
> as it
> starts the instance (sorry I can't provide more specifics, if you can
> mention
> what you are using, possibly someone else can chime in on the best way to
> set
> the hostname with that tool)
>
> David Lang
>
> On Tue, 16 Nov 2021, Scott Slattery wrote:
>
> > Date: Tue, 16 Nov 2021 16:59:17 -0700
> > From: Scott Slattery <scott.slattery@motorolasolutions.com>
> > To: David Lang <david@lang.hm>
> > Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> > Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >
> > My follow-on question woudl be how do I set the hostname at the client
> end?
> > Other than what's in /etc/hosts, /etc/hostname, etc. I don't know how
> else
> > I would affect the log being sent to ensure it's going over.
> >
> > *Scott Slattery*
> >
> > *Sr. Enterprise/Cloud Architect*
> >
> > *Cloud, Compute, Information & Architecture Team*
> >
> > motorolasolutions.com
> >
> > *O: 602.529.8226*
> >
> > *E*: Scott.Slattery@MotorolaSolutions.com
> >
> >
> >
> >
> > On Tue, Nov 16, 2021 at 4:55 PM David Lang <david@lang.hm> wrote:
> >
> >> the translation from fromhost-ip to fromhost is done at the collector,
> but
> >> the
> >> sender sets the hostname field. If you can trust that hostname was set
> >> correctly, there is no reason to use fromhost
> >>
> >> David Lang
> >>
> >> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>
> >>> Date: Tue, 16 Nov 2021 16:53:19 -0700
> >>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>> To: David Lang <david@lang.hm>
> >>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>
> >>> Thanks David, I could be wrong but the resolution seems to be happening
> >> at
> >>> the log collection server, not the client end. Given this, I'm not sure
> >>> anything outside of rsyslog on the client would affect what the
> receiving
> >>> collection server is seeing.
> >>>
> >>> My hope was that this could be affected by RSYSLOG on the client device
> >> but
> >>> perhaps not. I'll also look into AWS to see if a dynamically created
> >>> compute resource can automatically be registered with DNS.
> >>>
> >>> If anything else comes to mind, let me know. As always, I appreciate
> your
> >>> feedback.
> >>>
> >>> *Scott Slattery*
> >>>
> >>> *Sr. Enterprise/Cloud Architect*
> >>>
> >>> *Cloud, Compute, Information & Architecture Team*
> >>>
> >>> motorolasolutions.com
> >>>
> >>> *O: 602.529.8226*
> >>>
> >>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>
> >>>
> >>>
> >>>
> >>> On Tue, Nov 16, 2021 at 4:37 PM David Lang <david@lang.hm> wrote:
> >>>
> >>>> Linux has a rather sophisticated mechanism for plugging in arbitrary
> >> ways
> >>>> of
> >>>> doing name resolution. DNS has 'won' but hitorically there have been
> >> many
> >>>> other
> >>>> options. Research nsswitch (/etc/nsswitch.conf) and see if there is
> >>>> something
> >>>> that you can leverage.
> >>>>
> >>>> or, if you can set the hostname of the resources as they are created
> to
> >> be
> >>>> some
> >>>> predicatable pattern rather than the AWS default of IP based, you can
> >> then
> >>>> make
> >>>> your logic use that. (This is the approach I would look into). What
> >>>> mechanism
> >>>> this will be will depend on how you are configuring/provisioning the
> >>>> systems.
> >>>>
> >>>> David Lang
> >>>>
> >>>>
> >>>>
> >>>> On Tue, 16 Nov 2021, Scott Slattery wrote:
> >>>>
> >>>>> Date: Tue, 16 Nov 2021 15:14:51 -0700
> >>>>> From: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>> To: David Lang <david@lang.hm>
> >>>>> Cc: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>> Subject: Re: [rsyslog] FROMHOST missing on central log collector
> >>>>>
> >>>>> Thanks, David, I was hoping this was possible. Since the compute
> >>>> resources
> >>>>> are dynamic, using any sort of local /etc/hosts would be impossible
> >> since
> >>>>> the IP are unpredictable. Can you point me to how I would do this on
> >> the
> >>>>> client-server?
> >>>>>
> >>>>> Thanks
> >>>>>
> >>>>> *Scott Slattery*
> >>>>>
> >>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>
> >>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>
> >>>>> motorolasolutions.com
> >>>>>
> >>>>> *O: 602.529.8226*
> >>>>>
> >>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Tue, Nov 16, 2021 at 2:59 PM David Lang <david@lang.hm> wrote:
> >>>>>
> >>>>>> fromhost is the result of a name lookup of fromhost-ip. On the
> >> receiver,
> >>>>>> you can
> >>>>>> control this with your name resolution (DNS, /etc/hosts, other
> >>>> mechanisms)
> >>>>>>
> >>>>>> but a better option would probably be to set the hostname on the
> >> sender.
> >>>>>> The
> >>>>>> hostname field in the message is under the full control of the
> sender.
> >>>>>>
> >>>>>> David Lang
> >>>>>>
> >>>>>> On Tue, 16 Nov 2021, Scott Slattery via rsyslog wrote:
> >>>>>>
> >>>>>>> Date: Tue, 16 Nov 2021 14:56:09 -0700
> >>>>>>> From: Scott Slattery via rsyslog <rsyslog@lists.adiscon.com>
> >>>>>>> To: rsyslog-users <rsyslog@lists.adiscon.com>
> >>>>>>> Cc: Scott Slattery <scott.slattery@motorolasolutions.com>
> >>>>>>> Subject: [rsyslog] FROMHOST missing on central log collector
> >>>>>>>
> >>>>>>> Hello,
> >>>>>>>
> >>>>>>> I have a central log server, many of them, using rsyslog to
> aggregate
> >>>>>> logs
> >>>>>>> from remote servers. Everything works great but I have a new
> >> challenge
> >>>>>> and
> >>>>>>> am hoping for some recommendations.
> >>>>>>>
> >>>>>>> I have a number of AWS auto-scaling groups where compute resources
> >> are
> >>>>>>> dynamically scaled up and down. Each of these will have a custom
> >>>> rsyslog
> >>>>>>> configuration pulled from the AWS AMI.
> >>>>>>>
> >>>>>>> These dynamic resources are not added to DNS due to their dynamic
> >>>> nature
> >>>>>> so
> >>>>>>> they will not have DNS assigned FQDNs.
> >>>>>>>
> >>>>>>> Because of the lack of a hostname, my central log server is getting
> >>>> only
> >>>>>>> IP. I aggregate based on FROMHOST-FROMHOST-IP.
> >>>>>>>
> >>>>>>> So what I'm seeing today looks like '10.38.134.77-10.38.134.77'
> >> where I
> >>>>>>> want to see ause1oagbtst03.mydomain.com-10.41.102.168
> >>>>>>>
> >>>>>>> What I'd want to do is have easy resource send using the same
> >> hostname
> >>>>>> and
> >>>>>>> current IP. This later will allow me to aggregate all resources by
> >>>> name.
> >>>>>>>
> >>>>>>> I did not see any way of affecting the FROMHOST information unless,
> >> on
> >>>>>> the
> >>>>>>> collector, I have rules based on IP address which isn't optimal
> given
> >>>> the
> >>>>>>> dynamic nature of the IPs changing.
> >>>>>>>
> >>>>>>> Any suggestion is appreciated.
> >>>>>>>
> >>>>>>> *Scott Slattery*
> >>>>>>>
> >>>>>>> *Sr. Enterprise/Cloud Architect*
> >>>>>>>
> >>>>>>> *Cloud, Compute, Information & Architecture Team*
> >>>>>>>
> >>>>>>> motorolasolutions.com
> >>>>>>>
> >>>>>>> *O: 602.529.8226*
> >>>>>>>
> >>>>>>> *E*: Scott.Slattery@MotorolaSolutions.com
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>
> >
> >
>
--
*For more information on how and why we collect your personal
information, please visit our Privacy Policy
<
https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement>.*
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow
https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.