Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. RSyslog>
  3. users
impstats _sender_stat messages reported as string, not integer
rsyslog at lists
Jun 7, 2021, 12:41 PM
Post #1 of 11 (1054 views)
Permalink
In version 8.2102 the impstats _sender_stat messages field is being reported as a string value, not an integer.


{"name":"_sender_stat", "sender":"192.168.10.18", "messages":"0"}


This is in contrast to essentially everything else that is reported as integers.


{"name": "udp-15144-out queue", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 4 }


Config is generated from blow directive…


module(load="impstats"
interval="60"
resetCounters="on"
format="json"
severity="7"
log.file="/var/log/impstats/impstats.json”
)


I rather doubt this is intended, and it’s trivial to work around, but the maintainers would need to say for sure. I can enter an issue if desired.

Thanks,

_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: impstats _sender_stat messages reported as string, not integer [ In reply to ]
rsyslog at lists
Jun 7, 2021, 9:21 PM
Post #2 of 11 (1054 views)
Permalink
Hello!

Hmm.. from what I see here you're right:
https://github.com/rsyslog/rsyslog/blob/master/runtime/statsobj.c#L500

Number is formatted as a string.. And there is no "origin" field as well.
I'd prefer to export this data in the same form as `dynstats` counters are.
E.g.:

{ "name": "_sender_stats", "origin": "runtime", "values": {
"192.168.10.18": 1, "192.168.10.19": 12 } }

This way it's much easier to reuse existing parsers.

Or... Ideally we don't need this kind of stats at all because a user can
register a dynstats counter for this (I guess).. but I may miss some
internal details here..

P.S. I have personal interest here as I made the rsyslog_exporter in Python
:)


On Tue, 8 Jun 2021 at 03:41, John Chivian via rsyslog <
rsyslog@lists.adiscon.com> wrote:

> In version 8.2102 the impstats _sender_stat messages field is being
> reported as a string value, not an integer.
>
>
> {"name":"_sender_stat", "sender":"192.168.10.18", "messages":"0"}
>
>
> This is in contrast to essentially everything else that is reported as
> integers.
>
>
> {"name": "udp-15144-out queue", "origin": "core.queue", "size": 0,
> "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0,
> "maxqsize": 4 }
>
>
> Config is generated from blow directive…
>
>
> module(load="impstats"
> interval="60"
> resetCounters="on"
> format="json"
> severity="7"
> log.file="/var/log/impstats/impstats.json”
> )
>
>
> I rather doubt this is intended, and it’s trivial to work around, but the
> maintainers would need to say for sure. I can enter an issue if desired.
>
> Thanks,
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.



--
Yury Bushmelev
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: impstats _sender_stat messages reported as string, not integer [ In reply to ]
rsyslog at lists
Jun 8, 2021, 4:11 AM
Post #3 of 11 (1054 views)
Permalink
mhhh... this is a bit unfortunate: I obviously did not spot it with
the initial commit.

Question now: will we break existing scripts when we fix this? Do we
need to add an option to impstats to export numbers as, well, numbers?
;-)

Opinions?

Rainer

El mar, 8 jun 2021 a las 6:21, Yuri Bushmelev via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> Hello!
>
> Hmm.. from what I see here you're right:
> https://github.com/rsyslog/rsyslog/blob/master/runtime/statsobj.c#L500
>
> Number is formatted as a string.. And there is no "origin" field as well.
> I'd prefer to export this data in the same form as `dynstats` counters are.
> E.g.:
>
> { "name": "_sender_stats", "origin": "runtime", "values": {
> "192.168.10.18": 1, "192.168.10.19": 12 } }
>
> This way it's much easier to reuse existing parsers.
>
> Or... Ideally we don't need this kind of stats at all because a user can
> register a dynstats counter for this (I guess).. but I may miss some
> internal details here..
>
> P.S. I have personal interest here as I made the rsyslog_exporter in Python
> :)
>
>
> On Tue, 8 Jun 2021 at 03:41, John Chivian via rsyslog <
> rsyslog@lists.adiscon.com> wrote:
>
> > In version 8.2102 the impstats _sender_stat messages field is being
> > reported as a string value, not an integer.
> >
> >
> > {"name":"_sender_stat", "sender":"192.168.10.18", "messages":"0"}
> >
> >
> > This is in contrast to essentially everything else that is reported as
> > integers.
> >
> >
> > {"name": "udp-15144-out queue", "origin": "core.queue", "size": 0,
> > "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0,
> > "maxqsize": 4 }
> >
> >
> > Config is generated from blow directive…
> >
> >
> > module(load="impstats"
> > interval="60"
> > resetCounters="on"
> > format="json"
> > severity="7"
> > log.file="/var/log/impstats/impstats.json”
> > )
> >
> >
> > I rather doubt this is intended, and it’s trivial to work around, but the
> > maintainers would need to say for sure. I can enter an issue if desired.
> >
> > Thanks,
> >
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> > DON'T LIKE THAT.
>
>
>
> --
> Yury Bushmelev
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: impstats _sender_stat messages reported as string, not integer [ In reply to ]
rsyslog at lists
Jun 8, 2021, 4:12 AM
Post #4 of 11 (1054 views)
Permalink
I forgot: I would not like to refactor this into dynstats, as this is
IMHO harder to use for non-pro users.

Rainer

El mar, 8 jun 2021 a las 13:11, Rainer Gerhards
(<rgerhards@hq.adiscon.com>) escribió:
>
> mhhh... this is a bit unfortunate: I obviously did not spot it with
> the initial commit.
>
> Question now: will we break existing scripts when we fix this? Do we
> need to add an option to impstats to export numbers as, well, numbers?
> ;-)
>
> Opinions?
>
> Rainer
>
> El mar, 8 jun 2021 a las 6:21, Yuri Bushmelev via rsyslog
> (<rsyslog@lists.adiscon.com>) escribió:
> >
> > Hello!
> >
> > Hmm.. from what I see here you're right:
> > https://github.com/rsyslog/rsyslog/blob/master/runtime/statsobj.c#L500
> >
> > Number is formatted as a string.. And there is no "origin" field as well.
> > I'd prefer to export this data in the same form as `dynstats` counters are.
> > E.g.:
> >
> > { "name": "_sender_stats", "origin": "runtime", "values": {
> > "192.168.10.18": 1, "192.168.10.19": 12 } }
> >
> > This way it's much easier to reuse existing parsers.
> >
> > Or... Ideally we don't need this kind of stats at all because a user can
> > register a dynstats counter for this (I guess).. but I may miss some
> > internal details here..
> >
> > P.S. I have personal interest here as I made the rsyslog_exporter in Python
> > :)
> >
> >
> > On Tue, 8 Jun 2021 at 03:41, John Chivian via rsyslog <
> > rsyslog@lists.adiscon.com> wrote:
> >
> > > In version 8.2102 the impstats _sender_stat messages field is being
> > > reported as a string value, not an integer.
> > >
> > >
> > > {"name":"_sender_stat", "sender":"192.168.10.18", "messages":"0"}
> > >
> > >
> > > This is in contrast to essentially everything else that is reported as
> > > integers.
> > >
> > >
> > > {"name": "udp-15144-out queue", "origin": "core.queue", "size": 0,
> > > "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0,
> > > "maxqsize": 4 }
> > >
> > >
> > > Config is generated from blow directive…
> > >
> > >
> > > module(load="impstats"
> > > interval="60"
> > > resetCounters="on"
> > > format="json"
> > > severity="7"
> > > log.file="/var/log/impstats/impstats.json”
> > > )
> > >
> > >
> > > I rather doubt this is intended, and it’s trivial to work around, but the
> > > maintainers would need to say for sure. I can enter an issue if desired.
> > >
> > > Thanks,
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> > > DON'T LIKE THAT.
> >
> >
> >
> > --
> > Yury Bushmelev
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: impstats _sender_stat messages reported as string, not integer [ In reply to ]
rsyslog at lists
Jun 8, 2021, 4:28 AM
Post #5 of 11 (1054 views)
Permalink
Hello!

I just found I was never using `senders.keepTrack` before.. though it's a 5
years old feature.. I was always using dynstats for this :-D

I'd vote for a global option to have sender stats in old format or in
dynstats-like format.
Something like "senders.reportFormat = [old/dynstats]" with "old" as a
default value.
This way we can keep compatibility and do not create yet another format to
deal with.

Thank you!

On Tue, 8 Jun 2021 at 19:13, Rainer Gerhards <rgerhards@hq.adiscon.com>
wrote:

> I forgot: I would not like to refactor this into dynstats, as this is
> IMHO harder to use for non-pro users.
>
> Rainer
>
> El mar, 8 jun 2021 a las 13:11, Rainer Gerhards
> (<rgerhards@hq.adiscon.com>) escribió:
> >
> > mhhh... this is a bit unfortunate: I obviously did not spot it with
> > the initial commit.
> >
> > Question now: will we break existing scripts when we fix this? Do we
> > need to add an option to impstats to export numbers as, well, numbers?
> > ;-)
> >
> > Opinions?
> >
> > Rainer
> >
> > El mar, 8 jun 2021 a las 6:21, Yuri Bushmelev via rsyslog
> > (<rsyslog@lists.adiscon.com>) escribió:
> > >
> > > Hello!
> > >
> > > Hmm.. from what I see here you're right:
> > > https://github.com/rsyslog/rsyslog/blob/master/runtime/statsobj.c#L500
> > >
> > > Number is formatted as a string.. And there is no "origin" field as
> well.
> > > I'd prefer to export this data in the same form as `dynstats` counters
> are.
> > > E.g.:
> > >
> > > { "name": "_sender_stats", "origin": "runtime", "values": {
> > > "192.168.10.18": 1, "192.168.10.19": 12 } }
> > >
> > > This way it's much easier to reuse existing parsers.
> > >
> > > Or... Ideally we don't need this kind of stats at all because a user
> can
> > > register a dynstats counter for this (I guess).. but I may miss some
> > > internal details here..
> > >
> > > P.S. I have personal interest here as I made the rsyslog_exporter in
> Python
> > > :)
> > >
> > >
> > > On Tue, 8 Jun 2021 at 03:41, John Chivian via rsyslog <
> > > rsyslog@lists.adiscon.com> wrote:
> > >
> > > > In version 8.2102 the impstats _sender_stat messages field is being
> > > > reported as a string value, not an integer.
> > > >
> > > >
> > > > {"name":"_sender_stat", "sender":"192.168.10.18", "messages":"0"}
> > > >
> > > >
> > > > This is in contrast to essentially everything else that is reported
> as
> > > > integers.
> > > >
> > > >
> > > > {"name": "udp-15144-out queue", "origin": "core.queue", "size": 0,
> > > > "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0,
> > > > "maxqsize": 4 }
> > > >
> > > >
> > > > Config is generated from blow directive…
> > > >
> > > >
> > > > module(load="impstats"
> > > > interval="60"
> > > > resetCounters="on"
> > > > format="json"
> > > > severity="7"
> > > > log.file="/var/log/impstats/impstats.json”
> > > > )
> > > >
> > > >
> > > > I rather doubt this is intended, and it’s trivial to work around,
> but the
> > > > maintainers would need to say for sure. I can enter an issue if
> desired.
> > > >
> > > > Thanks,
> > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com/professional-services/
> > > > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad
> > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> you
> > > > DON'T LIKE THAT.
> > >
> > >
> > >
> > > --
> > > Yury Bushmelev
> > > _______________________________________________
> > > rsyslog mailing list
> > > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> you DON'T LIKE THAT.
>


--
Yury Bushmelev
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: impstats _sender_stat messages reported as string, not integer [ In reply to ]
rsyslog at lists
Jun 8, 2021, 4:42 AM
Post #6 of 11 (1054 views)
Permalink
Hi Rainer:

I would prefer the existing format, not the dynstats format or methodology as you are correct, that is not so friendly to not advanced users.

From my perspective, changing the messages field to be emitted as an integer will not break anything. That information is fed into a modern SIEM which is intelligent enough handle the string correctly. It requires conversion to an integer for use (which is why I brought it up), but that’s just an extra step not a heavy lift.

I would vote for simply correcting the existing output format as this doesn’t fundamentally change the behavior or configuration of the application.

Regards,



> On Jun 8, 2021, at 06:28, Yuri Bushmelev via rsyslog <rsyslog@lists.adiscon.com> wrote:
>
> Hello!
>
> I just found I was never using `senders.keepTrack` before.. though it's a 5
> years old feature.. I was always using dynstats for this :-D
>
> I'd vote for a global option to have sender stats in old format or in
> dynstats-like format.
> Something like "senders.reportFormat = [old/dynstats]" with "old" as a
> default value.
> This way we can keep compatibility and do not create yet another format to
> deal with.
>
> Thank you!
>
> On Tue, 8 Jun 2021 at 19:13, Rainer Gerhards <rgerhards@hq.adiscon.com>
> wrote:
>
>> I forgot: I would not like to refactor this into dynstats, as this is
>> IMHO harder to use for non-pro users.
>>
>> Rainer
>>
>> El mar, 8 jun 2021 a las 13:11, Rainer Gerhards
>> (<rgerhards@hq.adiscon.com>) escribió:
>>>
>>> mhhh... this is a bit unfortunate: I obviously did not spot it with
>>> the initial commit.
>>>
>>> Question now: will we break existing scripts when we fix this? Do we
>>> need to add an option to impstats to export numbers as, well, numbers?
>>> ;-)
>>>
>>> Opinions?
>>>
>>> Rainer
>>>
>>> El mar, 8 jun 2021 a las 6:21, Yuri Bushmelev via rsyslog
>>> (<rsyslog@lists.adiscon.com>) escribió:
>>>>
>>>> Hello!
>>>>
>>>> Hmm.. from what I see here you're right:
>>>> https://github.com/rsyslog/rsyslog/blob/master/runtime/statsobj.c#L500
>>>>
>>>> Number is formatted as a string.. And there is no "origin" field as
>> well.
>>>> I'd prefer to export this data in the same form as `dynstats` counters
>> are.
>>>> E.g.:
>>>>
>>>> { "name": "_sender_stats", "origin": "runtime", "values": {
>>>> "192.168.10.18": 1, "192.168.10.19": 12 } }
>>>>
>>>> This way it's much easier to reuse existing parsers.
>>>>
>>>> Or... Ideally we don't need this kind of stats at all because a user
>> can
>>>> register a dynstats counter for this (I guess).. but I may miss some
>>>> internal details here..
>>>>
>>>> P.S. I have personal interest here as I made the rsyslog_exporter in
>> Python
>>>> :)
>>>>
>>>>
>>>> On Tue, 8 Jun 2021 at 03:41, John Chivian via rsyslog <
>>>> rsyslog@lists.adiscon.com> wrote:
>>>>
>>>>> In version 8.2102 the impstats _sender_stat messages field is being
>>>>> reported as a string value, not an integer.
>>>>>
>>>>>
>>>>> {"name":"_sender_stat", "sender":"192.168.10.18", "messages":"0"}
>>>>>
>>>>>
>>>>> This is in contrast to essentially everything else that is reported
>> as
>>>>> integers.
>>>>>
>>>>>
>>>>> {"name": "udp-15144-out queue", "origin": "core.queue", "size": 0,
>>>>> "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0,
>>>>> "maxqsize": 4 }
>>>>>
>>>>>
>>>>> Config is generated from blow directive…
>>>>>
>>>>>
>>>>> module(load="impstats"
>>>>> interval="60"
>>>>> resetCounters="on"
>>>>> format="json"
>>>>> severity="7"
>>>>> log.file="/var/log/impstats/impstats.json”
>>>>> )
>>>>>
>>>>>
>>>>> I rather doubt this is intended, and it’s trivial to work around,
>> but the
>>>>> maintainers would need to say for sure. I can enter an issue if
>> desired.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com/professional-services/
>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad
>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
>> you
>>>>> DON'T LIKE THAT.
>>>>
>>>>
>>>>
>>>> --
>>>> Yury Bushmelev
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com/professional-services/
>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
>> you DON'T LIKE THAT.
>>
>
>
> --
> Yury Bushmelev
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: impstats _sender_stat messages reported as string, not integer [ In reply to ]
rsyslog at lists
Jun 9, 2021, 5:35 AM
Post #7 of 11 (1054 views)
Permalink
Yuri,

would that proposal "just fix it" work for you?

While I am really going strongly for keeping backwards compatibility,
John's argument sounds convincing. One reason for this thought is that
experience tells me many people will not turn on such a "bug-fixing
option".

Or maybe "just fix it and add the option if there are real complaints"?

Rainer

El mar, 8 jun 2021 a las 13:42, John Chivian (<jchivian@chivian.com>) escribió:
>
> Hi Rainer:
>
> I would prefer the existing format, not the dynstats format or methodology as you are correct, that is not so friendly to not advanced users.
>
> From my perspective, changing the messages field to be emitted as an integer will not break anything. That information is fed into a modern SIEM which is intelligent enough handle the string correctly. It requires conversion to an integer for use (which is why I brought it up), but that’s just an extra step not a heavy lift.
>
> I would vote for simply correcting the existing output format as this doesn’t fundamentally change the behavior or configuration of the application.
>
> Regards,
>
>
>
> > On Jun 8, 2021, at 06:28, Yuri Bushmelev via rsyslog <rsyslog@lists.adiscon.com> wrote:
> >
> > Hello!
> >
> > I just found I was never using `senders.keepTrack` before.. though it's a 5
> > years old feature.. I was always using dynstats for this :-D
> >
> > I'd vote for a global option to have sender stats in old format or in
> > dynstats-like format.
> > Something like "senders.reportFormat = [old/dynstats]" with "old" as a
> > default value.
> > This way we can keep compatibility and do not create yet another format to
> > deal with.
> >
> > Thank you!
> >
> > On Tue, 8 Jun 2021 at 19:13, Rainer Gerhards <rgerhards@hq.adiscon.com>
> > wrote:
> >
> >> I forgot: I would not like to refactor this into dynstats, as this is
> >> IMHO harder to use for non-pro users.
> >>
> >> Rainer
> >>
> >> El mar, 8 jun 2021 a las 13:11, Rainer Gerhards
> >> (<rgerhards@hq.adiscon.com>) escribió:
> >>>
> >>> mhhh... this is a bit unfortunate: I obviously did not spot it with
> >>> the initial commit.
> >>>
> >>> Question now: will we break existing scripts when we fix this? Do we
> >>> need to add an option to impstats to export numbers as, well, numbers?
> >>> ;-)
> >>>
> >>> Opinions?
> >>>
> >>> Rainer
> >>>
> >>> El mar, 8 jun 2021 a las 6:21, Yuri Bushmelev via rsyslog
> >>> (<rsyslog@lists.adiscon.com>) escribió:
> >>>>
> >>>> Hello!
> >>>>
> >>>> Hmm.. from what I see here you're right:
> >>>> https://github.com/rsyslog/rsyslog/blob/master/runtime/statsobj.c#L500
> >>>>
> >>>> Number is formatted as a string.. And there is no "origin" field as
> >> well.
> >>>> I'd prefer to export this data in the same form as `dynstats` counters
> >> are.
> >>>> E.g.:
> >>>>
> >>>> { "name": "_sender_stats", "origin": "runtime", "values": {
> >>>> "192.168.10.18": 1, "192.168.10.19": 12 } }
> >>>>
> >>>> This way it's much easier to reuse existing parsers.
> >>>>
> >>>> Or... Ideally we don't need this kind of stats at all because a user
> >> can
> >>>> register a dynstats counter for this (I guess).. but I may miss some
> >>>> internal details here..
> >>>>
> >>>> P.S. I have personal interest here as I made the rsyslog_exporter in
> >> Python
> >>>> :)
> >>>>
> >>>>
> >>>> On Tue, 8 Jun 2021 at 03:41, John Chivian via rsyslog <
> >>>> rsyslog@lists.adiscon.com> wrote:
> >>>>
> >>>>> In version 8.2102 the impstats _sender_stat messages field is being
> >>>>> reported as a string value, not an integer.
> >>>>>
> >>>>>
> >>>>> {"name":"_sender_stat", "sender":"192.168.10.18", "messages":"0"}
> >>>>>
> >>>>>
> >>>>> This is in contrast to essentially everything else that is reported
> >> as
> >>>>> integers.
> >>>>>
> >>>>>
> >>>>> {"name": "udp-15144-out queue", "origin": "core.queue", "size": 0,
> >>>>> "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0,
> >>>>> "maxqsize": 4 }
> >>>>>
> >>>>>
> >>>>> Config is generated from blow directive…
> >>>>>
> >>>>>
> >>>>> module(load="impstats"
> >>>>> interval="60"
> >>>>> resetCounters="on"
> >>>>> format="json"
> >>>>> severity="7"
> >>>>> log.file="/var/log/impstats/impstats.json”
> >>>>> )
> >>>>>
> >>>>>
> >>>>> I rather doubt this is intended, and it’s trivial to work around,
> >> but the
> >>>>> maintainers would need to say for sure. I can enter an issue if
> >> desired.
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> _______________________________________________
> >>>>> rsyslog mailing list
> >>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>> http://www.rsyslog.com/professional-services/
> >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> >> myriad
> >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> >> you
> >>>>> DON'T LIKE THAT.
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Yury Bushmelev
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>> http://www.rsyslog.com/professional-services/
> >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> >> you DON'T LIKE THAT.
> >>
> >
> >
> > --
> > Yury Bushmelev
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: impstats _sender_stat messages reported as string, not integer [ In reply to ]
rsyslog at lists
Jun 9, 2021, 7:41 AM
Post #8 of 11 (1054 views)
Permalink
Hello!

Well.. I'm ok to just change string to integer as John requested..
But my internal perfectionist would like to have "proper & beautiful fix"
:-D
My main point here is the sender stats are non-compliant compared to the
rest of metrics. I'd say it should be reimplemented as dynstats internally.
Or maybe just deleted putting an dynstats configuration example in docs
instead. But this is against overall rsyslog policy as I see it.

Let's just do a quickfix now. One day someone will refactor it maybe..

Thank you!

On Wed, 9 Jun 2021 at 20:35, Rainer Gerhards <rgerhards@hq.adiscon.com>
wrote:

> Yuri,
>
> would that proposal "just fix it" work for you?
>
> While I am really going strongly for keeping backwards compatibility,
> John's argument sounds convincing. One reason for this thought is that
> experience tells me many people will not turn on such a "bug-fixing
> option".
>
> Or maybe "just fix it and add the option if there are real complaints"?
>
> Rainer
>
> El mar, 8 jun 2021 a las 13:42, John Chivian (<jchivian@chivian.com>)
> escribió:
> >
> > Hi Rainer:
> >
> > I would prefer the existing format, not the dynstats format or
> methodology as you are correct, that is not so friendly to not advanced
> users.
> >
> > From my perspective, changing the messages field to be emitted as an
> integer will not break anything. That information is fed into a modern
> SIEM which is intelligent enough handle the string correctly. It requires
> conversion to an integer for use (which is why I brought it up), but that’s
> just an extra step not a heavy lift.
> >
> > I would vote for simply correcting the existing output format as this
> doesn’t fundamentally change the behavior or configuration of the
> application.
> >
> > Regards,
> >
> >
> >
> > > On Jun 8, 2021, at 06:28, Yuri Bushmelev via rsyslog <
> rsyslog@lists.adiscon.com> wrote:
> > >
> > > Hello!
> > >
> > > I just found I was never using `senders.keepTrack` before.. though
> it's a 5
> > > years old feature.. I was always using dynstats for this :-D
> > >
> > > I'd vote for a global option to have sender stats in old format or in
> > > dynstats-like format.
> > > Something like "senders.reportFormat = [old/dynstats]" with "old" as a
> > > default value.
> > > This way we can keep compatibility and do not create yet another
> format to
> > > deal with.
> > >
> > > Thank you!
> > >
> > > On Tue, 8 Jun 2021 at 19:13, Rainer Gerhards <rgerhards@hq.adiscon.com
> >
> > > wrote:
> > >
> > >> I forgot: I would not like to refactor this into dynstats, as this is
> > >> IMHO harder to use for non-pro users.
> > >>
> > >> Rainer
> > >>
> > >> El mar, 8 jun 2021 a las 13:11, Rainer Gerhards
> > >> (<rgerhards@hq.adiscon.com>) escribió:
> > >>>
> > >>> mhhh... this is a bit unfortunate: I obviously did not spot it with
> > >>> the initial commit.
> > >>>
> > >>> Question now: will we break existing scripts when we fix this? Do we
> > >>> need to add an option to impstats to export numbers as, well,
> numbers?
> > >>> ;-)
> > >>>
> > >>> Opinions?
> > >>>
> > >>> Rainer
> > >>>
> > >>> El mar, 8 jun 2021 a las 6:21, Yuri Bushmelev via rsyslog
> > >>> (<rsyslog@lists.adiscon.com>) escribió:
> > >>>>
> > >>>> Hello!
> > >>>>
> > >>>> Hmm.. from what I see here you're right:
> > >>>>
> https://github.com/rsyslog/rsyslog/blob/master/runtime/statsobj.c#L500
> > >>>>
> > >>>> Number is formatted as a string.. And there is no "origin" field as
> > >> well.
> > >>>> I'd prefer to export this data in the same form as `dynstats`
> counters
> > >> are.
> > >>>> E.g.:
> > >>>>
> > >>>> { "name": "_sender_stats", "origin": "runtime", "values": {
> > >>>> "192.168.10.18": 1, "192.168.10.19": 12 } }
> > >>>>
> > >>>> This way it's much easier to reuse existing parsers.
> > >>>>
> > >>>> Or... Ideally we don't need this kind of stats at all because a user
> > >> can
> > >>>> register a dynstats counter for this (I guess).. but I may miss some
> > >>>> internal details here..
> > >>>>
> > >>>> P.S. I have personal interest here as I made the rsyslog_exporter in
> > >> Python
> > >>>> :)
> > >>>>
> > >>>>
> > >>>> On Tue, 8 Jun 2021 at 03:41, John Chivian via rsyslog <
> > >>>> rsyslog@lists.adiscon.com> wrote:
> > >>>>
> > >>>>> In version 8.2102 the impstats _sender_stat messages field is being
> > >>>>> reported as a string value, not an integer.
> > >>>>>
> > >>>>>
> > >>>>> {"name":"_sender_stat", "sender":"192.168.10.18", "messages":"0"}
> > >>>>>
> > >>>>>
> > >>>>> This is in contrast to essentially everything else that is reported
> > >> as
> > >>>>> integers.
> > >>>>>
> > >>>>>
> > >>>>> {"name": "udp-15144-out queue", "origin": "core.queue", "size": 0,
> > >>>>> "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0,
> > >>>>> "maxqsize": 4 }
> > >>>>>
> > >>>>>
> > >>>>> Config is generated from blow directive…
> > >>>>>
> > >>>>>
> > >>>>> module(load="impstats"
> > >>>>> interval="60"
> > >>>>> resetCounters="on"
> > >>>>> format="json"
> > >>>>> severity="7"
> > >>>>> log.file="/var/log/impstats/impstats.json”
> > >>>>> )
> > >>>>>
> > >>>>>
> > >>>>> I rather doubt this is intended, and it’s trivial to work around,
> > >> but the
> > >>>>> maintainers would need to say for sure. I can enter an issue if
> > >> desired.
> > >>>>>
> > >>>>> Thanks,
> > >>>>>
> > >>>>> _______________________________________________
> > >>>>> rsyslog mailing list
> > >>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
> > >>>>> http://www.rsyslog.com/professional-services/
> > >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
> > >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> > >> myriad
> > >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> > >> you
> > >>>>> DON'T LIKE THAT.
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Yury Bushmelev
> > >>>> _______________________________________________
> > >>>> rsyslog mailing list
> > >>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
> > >>>> http://www.rsyslog.com/professional-services/
> > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
> > >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> > >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
> POST if
> > >> you DON'T LIKE THAT.
> > >>
> > >
> > >
> > > --
> > > Yury Bushmelev
> > > _______________________________________________
> > > rsyslog mailing list
> > > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> you DON'T LIKE THAT.
> >
>


--
Yury Bushmelev
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: impstats _sender_stat messages reported as string, not integer [ In reply to ]
rsyslog at lists
Jun 9, 2021, 9:44 AM
Post #9 of 11 (1054 views)
Permalink
To Yuri’s point, the _sender_stat message is the only one without an origin, but I see that as ancillary and not entirely relevant. In my view the format change is simply a bug fix.

Thanks to all!

> On Jun 9, 2021, at 09:41, Yuri Bushmelev <jay4mail@gmail.com> wrote:
>
> Hello!
>
> Well.. I'm ok to just change string to integer as John requested..
> But my internal perfectionist would like to have "proper & beautiful fix" :-D
> My main point here is the sender stats are non-compliant compared to the rest of metrics. I'd say it should be reimplemented as dynstats internally. Or maybe just deleted putting an dynstats configuration example in docs instead. But this is against overall rsyslog policy as I see it.
>
> Let's just do a quickfix now. One day someone will refactor it maybe..
>
> Thank you!
>
> On Wed, 9 Jun 2021 at 20:35, Rainer Gerhards <rgerhards@hq.adiscon.com <mailto:rgerhards@hq.adiscon.com>> wrote:
> Yuri,
>
> would that proposal "just fix it" work for you?
>
> While I am really going strongly for keeping backwards compatibility,
> John's argument sounds convincing. One reason for this thought is that
> experience tells me many people will not turn on such a "bug-fixing
> option".
>
> Or maybe "just fix it and add the option if there are real complaints"?
>
> Rainer
>
> El mar, 8 jun 2021 a las 13:42, John Chivian (<jchivian@chivian.com <mailto:jchivian@chivian.com>>) escribió:
> >
> > Hi Rainer:
> >
> > I would prefer the existing format, not the dynstats format or methodology as you are correct, that is not so friendly to not advanced users.
> >
> > From my perspective, changing the messages field to be emitted as an integer will not break anything. That information is fed into a modern SIEM which is intelligent enough handle the string correctly. It requires conversion to an integer for use (which is why I brought it up), but that’s just an extra step not a heavy lift.
> >
> > I would vote for simply correcting the existing output format as this doesn’t fundamentally change the behavior or configuration of the application.
> >
> > Regards,
> >
> >
> >
> > > On Jun 8, 2021, at 06:28, Yuri Bushmelev via rsyslog <rsyslog@lists.adiscon.com <mailto:rsyslog@lists.adiscon.com>> wrote:
> > >
> > > Hello!
> > >
> > > I just found I was never using `senders.keepTrack` before.. though it's a 5
> > > years old feature.. I was always using dynstats for this :-D
> > >
> > > I'd vote for a global option to have sender stats in old format or in
> > > dynstats-like format.
> > > Something like "senders.reportFormat = [old/dynstats]" with "old" as a
> > > default value.
> > > This way we can keep compatibility and do not create yet another format to
> > > deal with.
> > >
> > > Thank you!
> > >
> > > On Tue, 8 Jun 2021 at 19:13, Rainer Gerhards <rgerhards@hq.adiscon.com <mailto:rgerhards@hq.adiscon.com>>
> > > wrote:
> > >
> > >> I forgot: I would not like to refactor this into dynstats, as this is
> > >> IMHO harder to use for non-pro users.
> > >>
> > >> Rainer
> > >>
> > >> El mar, 8 jun 2021 a las 13:11, Rainer Gerhards
> > >> (<rgerhards@hq.adiscon.com <mailto:rgerhards@hq.adiscon.com>>) escribió:
> > >>>
> > >>> mhhh... this is a bit unfortunate: I obviously did not spot it with
> > >>> the initial commit.
> > >>>
> > >>> Question now: will we break existing scripts when we fix this? Do we
> > >>> need to add an option to impstats to export numbers as, well, numbers?
> > >>> ;-)
> > >>>
> > >>> Opinions?
> > >>>
> > >>> Rainer
> > >>>
> > >>> El mar, 8 jun 2021 a las 6:21, Yuri Bushmelev via rsyslog
> > >>> (<rsyslog@lists.adiscon.com <mailto:rsyslog@lists.adiscon.com>>) escribió:
> > >>>>
> > >>>> Hello!
> > >>>>
> > >>>> Hmm.. from what I see here you're right:
> > >>>> https://github.com/rsyslog/rsyslog/blob/master/runtime/statsobj.c#L500 <https://github.com/rsyslog/rsyslog/blob/master/runtime/statsobj.c#L500>
> > >>>>
> > >>>> Number is formatted as a string.. And there is no "origin" field as
> > >> well.
> > >>>> I'd prefer to export this data in the same form as `dynstats` counters
> > >> are.
> > >>>> E.g.:
> > >>>>
> > >>>> { "name": "_sender_stats", "origin": "runtime", "values": {
> > >>>> "192.168.10.18": 1, "192.168.10.19": 12 } }
> > >>>>
> > >>>> This way it's much easier to reuse existing parsers.
> > >>>>
> > >>>> Or... Ideally we don't need this kind of stats at all because a user
> > >> can
> > >>>> register a dynstats counter for this (I guess).. but I may miss some
> > >>>> internal details here..
> > >>>>
> > >>>> P.S. I have personal interest here as I made the rsyslog_exporter in
> > >> Python
> > >>>> :)
> > >>>>
> > >>>>
> > >>>> On Tue, 8 Jun 2021 at 03:41, John Chivian via rsyslog <
> > >>>> rsyslog@lists.adiscon.com <mailto:rsyslog@lists.adiscon.com>> wrote:
> > >>>>
> > >>>>> In version 8.2102 the impstats _sender_stat messages field is being
> > >>>>> reported as a string value, not an integer.
> > >>>>>
> > >>>>>
> > >>>>> {"name":"_sender_stat", "sender":"192.168.10.18", "messages":"0"}
> > >>>>>
> > >>>>>
> > >>>>> This is in contrast to essentially everything else that is reported
> > >> as
> > >>>>> integers.
> > >>>>>
> > >>>>>
> > >>>>> {"name": "udp-15144-out queue", "origin": "core.queue", "size": 0,
> > >>>>> "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf <http://discarded.nf/>": 0,
> > >>>>> "maxqsize": 4 }
> > >>>>>
> > >>>>>
> > >>>>> Config is generated from blow directive…
> > >>>>>
> > >>>>>
> > >>>>> module(load="impstats"
> > >>>>> interval="60"
> > >>>>> resetCounters="on"
> > >>>>> format="json"
> > >>>>> severity="7"
> > >>>>> log.file="/var/log/impstats/impstats.json”
> > >>>>> )
> > >>>>>
> > >>>>>
> > >>>>> I rather doubt this is intended, and it’s trivial to work around,
> > >> but the
> > >>>>> maintainers would need to say for sure. I can enter an issue if
> > >> desired.
> > >>>>>
> > >>>>> Thanks,
> > >>>>>
> > >>>>> _______________________________________________
> > >>>>> rsyslog mailing list
> > >>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog <https://lists.adiscon.net/mailman/listinfo/rsyslog>
> > >>>>> http://www.rsyslog.com/professional-services/ <http://www.rsyslog.com/professional-services/>
> > >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards <https://twitter.com/rgerhards>
> > >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> > >> myriad
> > >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> > >> you
> > >>>>> DON'T LIKE THAT.
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Yury Bushmelev
> > >>>> _______________________________________________
> > >>>> rsyslog mailing list
> > >>>> https://lists.adiscon.net/mailman/listinfo/rsyslog <https://lists.adiscon.net/mailman/listinfo/rsyslog>
> > >>>> http://www.rsyslog.com/professional-services/ <http://www.rsyslog.com/professional-services/>
> > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards <https://twitter.com/rgerhards>
> > >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> > >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> > >> you DON'T LIKE THAT.
> > >>
> > >
> > >
> > > --
> > > Yury Bushmelev
> > > _______________________________________________
> > > rsyslog mailing list
> > > https://lists.adiscon.net/mailman/listinfo/rsyslog <https://lists.adiscon.net/mailman/listinfo/rsyslog>
> > > http://www.rsyslog.com/professional-services/ <http://www.rsyslog.com/professional-services/>
> > > What's up with rsyslog? Follow https://twitter.com/rgerhards <https://twitter.com/rgerhards>
> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> >
>
>
> --
> Yury Bushmelev

_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: impstats _sender_stat messages reported as string, not integer [ In reply to ]
rsyslog at lists
Jun 9, 2021, 11:13 AM
Post #10 of 11 (1051 views)
Permalink
normally I am very strict on not breaking backwards compatibility, but I think
this is a case where consumers are rare, and a large number of them aren't going
to care if it's a sting or integer to start with, so I think it's better to fix
it and include a warning in the release notes for the next 6 months to a year
that there was a backwards incompaticle change in version X (with a link) rather
than adding the complexity of a config option.

David Lang


On
Wed, 9 Jun 2021, John Chivian via rsyslog wrote:

> Date: Wed, 9 Jun 2021 11:44:07 -0500
> From: John Chivian via rsyslog <rsyslog@lists.adiscon.com>
> To: Yuri Bushmelev <jay4mail@gmail.com>
> Cc: John Chivian <jchivian@chivian.com>,
> rsyslog-users <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] impstats _sender_stat messages reported as string,
> not integer
>
> To Yuri’s point, the _sender_stat message is the only one without an origin, but I see that as ancillary and not entirely relevant. In my view the format change is simply a bug fix.
>
> Thanks to all!
>
>> On Jun 9, 2021, at 09:41, Yuri Bushmelev <jay4mail@gmail.com> wrote:
>>
>> Hello!
>>
>> Well.. I'm ok to just change string to integer as John requested..
>> But my internal perfectionist would like to have "proper & beautiful fix" :-D
>> My main point here is the sender stats are non-compliant compared to the rest of metrics. I'd say it should be reimplemented as dynstats internally. Or maybe just deleted putting an dynstats configuration example in docs instead. But this is against overall rsyslog policy as I see it.
>>
>> Let's just do a quickfix now. One day someone will refactor it maybe..
>>
>> Thank you!
>>
>> On Wed, 9 Jun 2021 at 20:35, Rainer Gerhards <rgerhards@hq.adiscon.com <mailto:rgerhards@hq.adiscon.com>> wrote:
>> Yuri,
>>
>> would that proposal "just fix it" work for you?
>>
>> While I am really going strongly for keeping backwards compatibility,
>> John's argument sounds convincing. One reason for this thought is that
>> experience tells me many people will not turn on such a "bug-fixing
>> option".
>>
>> Or maybe "just fix it and add the option if there are real complaints"?
>>
>> Rainer
>>
>> El mar, 8 jun 2021 a las 13:42, John Chivian (<jchivian@chivian.com <mailto:jchivian@chivian.com>>) escribió:
>> >
>> > Hi Rainer:
>> >
>> > I would prefer the existing format, not the dynstats format or methodology as you are correct, that is not so friendly to not advanced users.
>> >
>> > From my perspective, changing the messages field to be emitted as an integer will not break anything. That information is fed into a modern SIEM which is intelligent enough handle the string correctly. It requires conversion to an integer for use (which is why I brought it up), but that’s just an extra step not a heavy lift.
>> >
>> > I would vote for simply correcting the existing output format as this doesn’t fundamentally change the behavior or configuration of the application.
>> >
>> > Regards,
>> >
>> >
>> >
>> > > On Jun 8, 2021, at 06:28, Yuri Bushmelev via rsyslog <rsyslog@lists.adiscon.com <mailto:rsyslog@lists.adiscon.com>> wrote:
>> > >
>> > > Hello!
>> > >
>> > > I just found I was never using `senders.keepTrack` before.. though it's a 5
>> > > years old feature.. I was always using dynstats for this :-D
>> > >
>> > > I'd vote for a global option to have sender stats in old format or in
>> > > dynstats-like format.
>> > > Something like "senders.reportFormat = [old/dynstats]" with "old" as a
>> > > default value.
>> > > This way we can keep compatibility and do not create yet another format to
>> > > deal with.
>> > >
>> > > Thank you!
>> > >
>> > > On Tue, 8 Jun 2021 at 19:13, Rainer Gerhards <rgerhards@hq.adiscon.com <mailto:rgerhards@hq.adiscon.com>>
>> > > wrote:
>> > >
>> > >> I forgot: I would not like to refactor this into dynstats, as this is
>> > >> IMHO harder to use for non-pro users.
>> > >>
>> > >> Rainer
>> > >>
>> > >> El mar, 8 jun 2021 a las 13:11, Rainer Gerhards
>> > >> (<rgerhards@hq.adiscon.com <mailto:rgerhards@hq.adiscon.com>>) escribió:
>> > >>>
>> > >>> mhhh... this is a bit unfortunate: I obviously did not spot it with
>> > >>> the initial commit.
>> > >>>
>> > >>> Question now: will we break existing scripts when we fix this? Do we
>> > >>> need to add an option to impstats to export numbers as, well, numbers?
>> > >>> ;-)
>> > >>>
>> > >>> Opinions?
>> > >>>
>> > >>> Rainer
>> > >>>
>> > >>> El mar, 8 jun 2021 a las 6:21, Yuri Bushmelev via rsyslog
>> > >>> (<rsyslog@lists.adiscon.com <mailto:rsyslog@lists.adiscon.com>>) escribió:
>> > >>>>
>> > >>>> Hello!
>> > >>>>
>> > >>>> Hmm.. from what I see here you're right:
>> > >>>> https://github.com/rsyslog/rsyslog/blob/master/runtime/statsobj.c#L500 <https://github.com/rsyslog/rsyslog/blob/master/runtime/statsobj.c#L500>
>> > >>>>
>> > >>>> Number is formatted as a string.. And there is no "origin" field as
>> > >> well.
>> > >>>> I'd prefer to export this data in the same form as `dynstats` counters
>> > >> are.
>> > >>>> E.g.:
>> > >>>>
>> > >>>> { "name": "_sender_stats", "origin": "runtime", "values": {
>> > >>>> "192.168.10.18": 1, "192.168.10.19": 12 } }
>> > >>>>
>> > >>>> This way it's much easier to reuse existing parsers.
>> > >>>>
>> > >>>> Or... Ideally we don't need this kind of stats at all because a user
>> > >> can
>> > >>>> register a dynstats counter for this (I guess).. but I may miss some
>> > >>>> internal details here..
>> > >>>>
>> > >>>> P.S. I have personal interest here as I made the rsyslog_exporter in
>> > >> Python
>> > >>>> :)
>> > >>>>
>> > >>>>
>> > >>>> On Tue, 8 Jun 2021 at 03:41, John Chivian via rsyslog <
>> > >>>> rsyslog@lists.adiscon.com <mailto:rsyslog@lists.adiscon.com>> wrote:
>> > >>>>
>> > >>>>> In version 8.2102 the impstats _sender_stat messages field is being
>> > >>>>> reported as a string value, not an integer.
>> > >>>>>
>> > >>>>>
>> > >>>>> {"name":"_sender_stat", "sender":"192.168.10.18", "messages":"0"}
>> > >>>>>
>> > >>>>>
>> > >>>>> This is in contrast to essentially everything else that is reported
>> > >> as
>> > >>>>> integers.
>> > >>>>>
>> > >>>>>
>> > >>>>> {"name": "udp-15144-out queue", "origin": "core.queue", "size": 0,
>> > >>>>> "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf <http://discarded.nf/>": 0,
>> > >>>>> "maxqsize": 4 }
>> > >>>>>
>> > >>>>>
>> > >>>>> Config is generated from blow directive…
>> > >>>>>
>> > >>>>>
>> > >>>>> module(load="impstats"
>> > >>>>> interval="60"
>> > >>>>> resetCounters="on"
>> > >>>>> format="json"
>> > >>>>> severity="7"
>> > >>>>> log.file="/var/log/impstats/impstats.json”
>> > >>>>> )
>> > >>>>>
>> > >>>>>
>> > >>>>> I rather doubt this is intended, and it’s trivial to work around,
>> > >> but the
>> > >>>>> maintainers would need to say for sure. I can enter an issue if
>> > >> desired.
>> > >>>>>
>> > >>>>> Thanks,
>> > >>>>>
>> > >>>>> _______________________________________________
>> > >>>>> rsyslog mailing list
>> > >>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog <https://lists.adiscon.net/mailman/listinfo/rsyslog>
>> > >>>>> http://www.rsyslog.com/professional-services/ <http://www.rsyslog.com/professional-services/>
>> > >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards <https://twitter.com/rgerhards>
>> > >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> > >> myriad
>> > >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
>> > >> you
>> > >>>>> DON'T LIKE THAT.
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> --
>> > >>>> Yury Bushmelev
>> > >>>> _______________________________________________
>> > >>>> rsyslog mailing list
>> > >>>> https://lists.adiscon.net/mailman/listinfo/rsyslog <https://lists.adiscon.net/mailman/listinfo/rsyslog>
>> > >>>> http://www.rsyslog.com/professional-services/ <http://www.rsyslog.com/professional-services/>
>> > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards <https://twitter.com/rgerhards>
>> > >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> > >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
>> > >> you DON'T LIKE THAT.
>> > >>
>> > >
>> > >
>> > > --
>> > > Yury Bushmelev
>> > > _______________________________________________
>> > > rsyslog mailing list
>> > > https://lists.adiscon.net/mailman/listinfo/rsyslog <https://lists.adiscon.net/mailman/listinfo/rsyslog>
>> > > http://www.rsyslog.com/professional-services/ <http://www.rsyslog.com/professional-services/>
>> > > What's up with rsyslog? Follow https://twitter.com/rgerhards <https://twitter.com/rgerhards>
>> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>> >
>>
>>
>> --
>> Yury Bushmelev
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: impstats _sender_stat messages reported as string, not integer [ In reply to ]
rsyslog at lists
Jun 10, 2021, 12:45 AM
Post #11 of 11 (1050 views)
Permalink
Thx all for the feedback. I have created a proposed PR, so far with
just the bugfix.

https://github.com/rsyslog/rsyslog/pull/4601

Comments, if any, are appreciated in the PR so that we have a record of them.

I have also just reached out to our support customers to get their
opinion on the potential option.

My plan is to add it either way to this month's scheduled stable release.

Rainer

El mié, 9 jun 2021 a las 20:13, David Lang via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> normally I am very strict on not breaking backwards compatibility, but I think
> this is a case where consumers are rare, and a large number of them aren't going
> to care if it's a sting or integer to start with, so I think it's better to fix
> it and include a warning in the release notes for the next 6 months to a year
> that there was a backwards incompaticle change in version X (with a link) rather
> than adding the complexity of a config option.
>
> David Lang
>
>
> On
> Wed, 9 Jun 2021, John Chivian via rsyslog wrote:
>
> > Date: Wed, 9 Jun 2021 11:44:07 -0500
> > From: John Chivian via rsyslog <rsyslog@lists.adiscon.com>
> > To: Yuri Bushmelev <jay4mail@gmail.com>
> > Cc: John Chivian <jchivian@chivian.com>,
> > rsyslog-users <rsyslog@lists.adiscon.com>
> > Subject: Re: [rsyslog] impstats _sender_stat messages reported as string,
> > not integer
> >
> > To Yuri’s point, the _sender_stat message is the only one without an origin, but I see that as ancillary and not entirely relevant. In my view the format change is simply a bug fix.
> >
> > Thanks to all!
> >
> >> On Jun 9, 2021, at 09:41, Yuri Bushmelev <jay4mail@gmail.com> wrote:
> >>
> >> Hello!
> >>
> >> Well.. I'm ok to just change string to integer as John requested..
> >> But my internal perfectionist would like to have "proper & beautiful fix" :-D
> >> My main point here is the sender stats are non-compliant compared to the rest of metrics. I'd say it should be reimplemented as dynstats internally. Or maybe just deleted putting an dynstats configuration example in docs instead. But this is against overall rsyslog policy as I see it.
> >>
> >> Let's just do a quickfix now. One day someone will refactor it maybe..
> >>
> >> Thank you!
> >>
> >> On Wed, 9 Jun 2021 at 20:35, Rainer Gerhards <rgerhards@hq.adiscon.com <mailto:rgerhards@hq.adiscon.com>> wrote:
> >> Yuri,
> >>
> >> would that proposal "just fix it" work for you?
> >>
> >> While I am really going strongly for keeping backwards compatibility,
> >> John's argument sounds convincing. One reason for this thought is that
> >> experience tells me many people will not turn on such a "bug-fixing
> >> option".
> >>
> >> Or maybe "just fix it and add the option if there are real complaints"?
> >>
> >> Rainer
> >>
> >> El mar, 8 jun 2021 a las 13:42, John Chivian (<jchivian@chivian.com <mailto:jchivian@chivian.com>>) escribió:
> >> >
> >> > Hi Rainer:
> >> >
> >> > I would prefer the existing format, not the dynstats format or methodology as you are correct, that is not so friendly to not advanced users.
> >> >
> >> > From my perspective, changing the messages field to be emitted as an integer will not break anything. That information is fed into a modern SIEM which is intelligent enough handle the string correctly. It requires conversion to an integer for use (which is why I brought it up), but that’s just an extra step not a heavy lift.
> >> >
> >> > I would vote for simply correcting the existing output format as this doesn’t fundamentally change the behavior or configuration of the application.
> >> >
> >> > Regards,
> >> >
> >> >
> >> >
> >> > > On Jun 8, 2021, at 06:28, Yuri Bushmelev via rsyslog <rsyslog@lists.adiscon.com <mailto:rsyslog@lists.adiscon.com>> wrote:
> >> > >
> >> > > Hello!
> >> > >
> >> > > I just found I was never using `senders.keepTrack` before.. though it's a 5
> >> > > years old feature.. I was always using dynstats for this :-D
> >> > >
> >> > > I'd vote for a global option to have sender stats in old format or in
> >> > > dynstats-like format.
> >> > > Something like "senders.reportFormat = [old/dynstats]" with "old" as a
> >> > > default value.
> >> > > This way we can keep compatibility and do not create yet another format to
> >> > > deal with.
> >> > >
> >> > > Thank you!
> >> > >
> >> > > On Tue, 8 Jun 2021 at 19:13, Rainer Gerhards <rgerhards@hq.adiscon.com <mailto:rgerhards@hq.adiscon.com>>
> >> > > wrote:
> >> > >
> >> > >> I forgot: I would not like to refactor this into dynstats, as this is
> >> > >> IMHO harder to use for non-pro users.
> >> > >>
> >> > >> Rainer
> >> > >>
> >> > >> El mar, 8 jun 2021 a las 13:11, Rainer Gerhards
> >> > >> (<rgerhards@hq.adiscon.com <mailto:rgerhards@hq.adiscon.com>>) escribió:
> >> > >>>
> >> > >>> mhhh... this is a bit unfortunate: I obviously did not spot it with
> >> > >>> the initial commit.
> >> > >>>
> >> > >>> Question now: will we break existing scripts when we fix this? Do we
> >> > >>> need to add an option to impstats to export numbers as, well, numbers?
> >> > >>> ;-)
> >> > >>>
> >> > >>> Opinions?
> >> > >>>
> >> > >>> Rainer
> >> > >>>
> >> > >>> El mar, 8 jun 2021 a las 6:21, Yuri Bushmelev via rsyslog
> >> > >>> (<rsyslog@lists.adiscon.com <mailto:rsyslog@lists.adiscon.com>>) escribió:
> >> > >>>>
> >> > >>>> Hello!
> >> > >>>>
> >> > >>>> Hmm.. from what I see here you're right:
> >> > >>>> https://github.com/rsyslog/rsyslog/blob/master/runtime/statsobj.c#L500 <https://github.com/rsyslog/rsyslog/blob/master/runtime/statsobj.c#L500>
> >> > >>>>
> >> > >>>> Number is formatted as a string.. And there is no "origin" field as
> >> > >> well.
> >> > >>>> I'd prefer to export this data in the same form as `dynstats` counters
> >> > >> are.
> >> > >>>> E.g.:
> >> > >>>>
> >> > >>>> { "name": "_sender_stats", "origin": "runtime", "values": {
> >> > >>>> "192.168.10.18": 1, "192.168.10.19": 12 } }
> >> > >>>>
> >> > >>>> This way it's much easier to reuse existing parsers.
> >> > >>>>
> >> > >>>> Or... Ideally we don't need this kind of stats at all because a user
> >> > >> can
> >> > >>>> register a dynstats counter for this (I guess).. but I may miss some
> >> > >>>> internal details here..
> >> > >>>>
> >> > >>>> P.S. I have personal interest here as I made the rsyslog_exporter in
> >> > >> Python
> >> > >>>> :)
> >> > >>>>
> >> > >>>>
> >> > >>>> On Tue, 8 Jun 2021 at 03:41, John Chivian via rsyslog <
> >> > >>>> rsyslog@lists.adiscon.com <mailto:rsyslog@lists.adiscon.com>> wrote:
> >> > >>>>
> >> > >>>>> In version 8.2102 the impstats _sender_stat messages field is being
> >> > >>>>> reported as a string value, not an integer.
> >> > >>>>>
> >> > >>>>>
> >> > >>>>> {"name":"_sender_stat", "sender":"192.168.10.18", "messages":"0"}
> >> > >>>>>
> >> > >>>>>
> >> > >>>>> This is in contrast to essentially everything else that is reported
> >> > >> as
> >> > >>>>> integers.
> >> > >>>>>
> >> > >>>>>
> >> > >>>>> {"name": "udp-15144-out queue", "origin": "core.queue", "size": 0,
> >> > >>>>> "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf <http://discarded.nf/>": 0,
> >> > >>>>> "maxqsize": 4 }
> >> > >>>>>
> >> > >>>>>
> >> > >>>>> Config is generated from blow directive…
> >> > >>>>>
> >> > >>>>>
> >> > >>>>> module(load="impstats"
> >> > >>>>> interval="60"
> >> > >>>>> resetCounters="on"
> >> > >>>>> format="json"
> >> > >>>>> severity="7"
> >> > >>>>> log.file="/var/log/impstats/impstats.json”
> >> > >>>>> )
> >> > >>>>>
> >> > >>>>>
> >> > >>>>> I rather doubt this is intended, and it’s trivial to work around,
> >> > >> but the
> >> > >>>>> maintainers would need to say for sure. I can enter an issue if
> >> > >> desired.
> >> > >>>>>
> >> > >>>>> Thanks,
> >> > >>>>>
> >> > >>>>> _______________________________________________
> >> > >>>>> rsyslog mailing list
> >> > >>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog <https://lists.adiscon.net/mailman/listinfo/rsyslog>
> >> > >>>>> http://www.rsyslog.com/professional-services/ <http://www.rsyslog.com/professional-services/>
> >> > >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards <https://twitter.com/rgerhards>
> >> > >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> >> > >> myriad
> >> > >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> >> > >> you
> >> > >>>>> DON'T LIKE THAT.
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> --
> >> > >>>> Yury Bushmelev
> >> > >>>> _______________________________________________
> >> > >>>> rsyslog mailing list
> >> > >>>> https://lists.adiscon.net/mailman/listinfo/rsyslog <https://lists.adiscon.net/mailman/listinfo/rsyslog>
> >> > >>>> http://www.rsyslog.com/professional-services/ <http://www.rsyslog.com/professional-services/>
> >> > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards <https://twitter.com/rgerhards>
> >> > >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> >> > >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> >> > >> you DON'T LIKE THAT.
> >> > >>
> >> > >
> >> > >
> >> > > --
> >> > > Yury Bushmelev
> >> > > _______________________________________________
> >> > > rsyslog mailing list
> >> > > https://lists.adiscon.net/mailman/listinfo/rsyslog <https://lists.adiscon.net/mailman/listinfo/rsyslog>
> >> > > http://www.rsyslog.com/professional-services/ <http://www.rsyslog.com/professional-services/>
> >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards <https://twitter.com/rgerhards>
> >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> >> >
> >>
> >>
> >> --
> >> Yury Bushmelev
> >
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal