Mailing List Archive

Time. What is time anyway?  The easiest way to solve your problem is to
punt and log everything in high resolution UTC.  Or you could log
everything with high resolution local time (including TZ offset). Or you
could set your entire logging infrastructure use UTC only.  At least
then you wouldn't have the problem of the wobbling hour.

All that aside, there is already an enhancement request for a high
resolution $now but it's a low priority item with not much momentum. 
Such a thing would return server local time regardless of the timezone
setting on the server (which would of course be UTC).

The current $now (as you've discovered) is a low resolution string, not
an actual timestamp that can be cast with a format.

Regards,


On 11/2/20 12:14 PM, Nicholas Brown via rsyslog wrote:
> Hi,
>
> I'm looking for a useful monatomic timestamp property that can be used for
> rate-limiting calculations in rules.
>
> 'parse_time($timegenerated)' mostly works, but when there are local time
> adjustments happen like daylight saving and time goes backwards.
>
> https://www.rsyslog.com/doc/master/configuration/properties.html#system-properties
> https://www.rsyslog.com/doc/master/rainerscript/functions/rs-parse_time.html
>
> A $timegenerated-utc property would not have daylight saving timezone
> issues, but does not appear to be available. Similar $now-utc is available,
> but it does not have second resolution to be useful to convert to a unix
> timestamp with parse_time().
>
> Is there a high precision utc timestamp property available?
>
> Thanks,
> Nick
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.


_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Hi,

Thanks for the reply.

I had found these issues:
https://github.com/rsyslog/rsyslog/issues/3329 (Enhancement to $now
property)
https://github.com/rsyslog/rsyslog/issues/3930 (Properties timegenerated
and timereported are not high precision in the RainerScript context)

For our use we are looking for something with second granularity, and in
UTC.
$timegenerated has second granularity, but there is no $timegenerated-utc
available.
$now and $now-utc both only have granularity to the day.

A simple example of the use case for the property in a rules might be:

# Get Unix timestamp of current message
set $.tnow = parse_time($timegenerated);

# Rate limit info to 5 every 60 seconds
if ($!severity == 6 and $!facility == 17) then {
if (($.tnow - $/trate) > 60) then {
# 5 seconds window expired, allow more messages
set $/trate = $.tnow;
set $/ratecount = 0;
}
if ($/ratecount > 5) then {
# discard message
stop
} else {
set $/ratecount = $/ratecount + 1;
}
}

Would it be possible to add a $timegenerated-utc property?
Any pointers into the rsyslog code on where this might be implemented would
be welcome, and we could take a look.

Thanks,
Nick

On Mon, 2 Nov 2020 at 20:59, John Chivian via rsyslog <
rsyslog@lists.adiscon.com> wrote:

> Time. What is time anyway? The easiest way to solve your problem is to
> punt and log everything in high resolution UTC. Or you could log
> everything with high resolution local time (including TZ offset). Or you
> could set your entire logging infrastructure use UTC only. At least
> then you wouldn't have the problem of the wobbling hour.
>
> All that aside, there is already an enhancement request for a high
> resolution $now but it's a low priority item with not much momentum.
> Such a thing would return server local time regardless of the timezone
> setting on the server (which would of course be UTC).
>
> The current $now (as you've discovered) is a low resolution string, not
> an actual timestamp that can be cast with a format.
>
> Regards,
>
>
> On 11/2/20 12:14 PM, Nicholas Brown via rsyslog wrote:
> > Hi,
> >
> > I'm looking for a useful monatomic timestamp property that can be used
> for
> > rate-limiting calculations in rules.
> >
> > 'parse_time($timegenerated)' mostly works, but when there are local time
> > adjustments happen like daylight saving and time goes backwards.
> >
> >
> https://www.rsyslog.com/doc/master/configuration/properties.html#system-properties
> >
> https://www.rsyslog.com/doc/master/rainerscript/functions/rs-parse_time.html
> >
> > A $timegenerated-utc property would not have daylight saving timezone
> > issues, but does not appear to be available. Similar $now-utc is
> available,
> > but it does not have second resolution to be useful to convert to a unix
> > timestamp with parse_time().
> >
> > Is there a high precision utc timestamp property available?
> >
> > Thanks,
> > Nick
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

An alternative that could be useful would be just make a unix timestamp
(with full second resolution) directly available as a variable.
Perhaps something like $now-unix?
The implementation of this could even avoid having to parse time formats,
thus be quick.
Looking at
https://github.com/rsyslog/rsyslog/blob/11b58367dda661c6b991122174faeb6463e5a919/runtime/datetime.c#L175:13
it appears
getCurrTime can already provide the unix seconds, and getTime() is
available that does only that.

On Tue, 3 Nov 2020 at 10:24, Nicholas Brown <nickbroon+rsyslog@gmail.com>
wrote:

> Hi,
>
> Thanks for the reply.
>
> I had found these issues:
> https://github.com/rsyslog/rsyslog/issues/3329 (Enhancement to $now
> property)
> https://github.com/rsyslog/rsyslog/issues/3930 (Properties timegenerated
> and timereported are not high precision in the RainerScript context)
>
> For our use we are looking for something with second granularity, and in
> UTC.
> $timegenerated has second granularity, but there is no $timegenerated-utc
> available.
> $now and $now-utc both only have granularity to the day.
>
> A simple example of the use case for the property in a rules might be:
>
> # Get Unix timestamp of current message
> set $.tnow = parse_time($timegenerated);
>
> # Rate limit info to 5 every 60 seconds
> if ($!severity == 6 and $!facility == 17) then {
> if (($.tnow - $/trate) > 60) then {
> # 5 seconds window expired, allow more messages
> set $/trate = $.tnow;
> set $/ratecount = 0;
> }
> if ($/ratecount > 5) then {
> # discard message
> stop
> } else {
> set $/ratecount = $/ratecount + 1;
> }
> }
>
> Would it be possible to add a $timegenerated-utc property?
> Any pointers into the rsyslog code on where this might be implemented
> would be welcome, and we could take a look.
>
> Thanks,
> Nick
>
> On Mon, 2 Nov 2020 at 20:59, John Chivian via rsyslog <
> rsyslog@lists.adiscon.com> wrote:
>
>> Time. What is time anyway? The easiest way to solve your problem is to
>> punt and log everything in high resolution UTC. Or you could log
>> everything with high resolution local time (including TZ offset). Or you
>> could set your entire logging infrastructure use UTC only. At least
>> then you wouldn't have the problem of the wobbling hour.
>>
>> All that aside, there is already an enhancement request for a high
>> resolution $now but it's a low priority item with not much momentum.
>> Such a thing would return server local time regardless of the timezone
>> setting on the server (which would of course be UTC).
>>
>> The current $now (as you've discovered) is a low resolution string, not
>> an actual timestamp that can be cast with a format.
>>
>> Regards,
>>
>>
>> On 11/2/20 12:14 PM, Nicholas Brown via rsyslog wrote:
>> > Hi,
>> >
>> > I'm looking for a useful monatomic timestamp property that can be used
>> for
>> > rate-limiting calculations in rules.
>> >
>> > 'parse_time($timegenerated)' mostly works, but when there are local time
>> > adjustments happen like daylight saving and time goes backwards.
>> >
>> >
>> https://www.rsyslog.com/doc/master/configuration/properties.html#system-properties
>> >
>> https://www.rsyslog.com/doc/master/rainerscript/functions/rs-parse_time.html
>> >
>> > A $timegenerated-utc property would not have daylight saving timezone
>> > issues, but does not appear to be available. Similar $now-utc is
>> available,
>> > but it does not have second resolution to be useful to convert to a unix
>> > timestamp with parse_time().
>> >
>> > Is there a high precision utc timestamp property available?
>> >
>> > Thanks,
>> > Nick
>> > _______________________________________________
>> > rsyslog mailing list
>> > https://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com/professional-services/
>> > What's up with rsyslog? Follow https://twitter.com/rgerhards
>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
>> you DON'T LIKE THAT.
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

sounds good! --> https://github.com/rsyslog/rsyslog/pull/4456

Rainer

El mar., 3 nov. 2020 a las 12:12, Nicholas Brown via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> An alternative that could be useful would be just make a unix timestamp
> (with full second resolution) directly available as a variable.
> Perhaps something like $now-unix?
> The implementation of this could even avoid having to parse time formats,
> thus be quick.
> Looking at
> https://github.com/rsyslog/rsyslog/blob/11b58367dda661c6b991122174faeb6463e5a919/runtime/datetime.c#L175:13
> it appears
> getCurrTime can already provide the unix seconds, and getTime() is
> available that does only that.
>
> On Tue, 3 Nov 2020 at 10:24, Nicholas Brown <nickbroon+rsyslog@gmail.com>
> wrote:
>
> > Hi,
> >
> > Thanks for the reply.
> >
> > I had found these issues:
> > https://github.com/rsyslog/rsyslog/issues/3329 (Enhancement to $now
> > property)
> > https://github.com/rsyslog/rsyslog/issues/3930 (Properties timegenerated
> > and timereported are not high precision in the RainerScript context)
> >
> > For our use we are looking for something with second granularity, and in
> > UTC.
> > $timegenerated has second granularity, but there is no $timegenerated-utc
> > available.
> > $now and $now-utc both only have granularity to the day.
> >
> > A simple example of the use case for the property in a rules might be:
> >
> > # Get Unix timestamp of current message
> > set $.tnow = parse_time($timegenerated);
> >
> > # Rate limit info to 5 every 60 seconds
> > if ($!severity == 6 and $!facility == 17) then {
> > if (($.tnow - $/trate) > 60) then {
> > # 5 seconds window expired, allow more messages
> > set $/trate = $.tnow;
> > set $/ratecount = 0;
> > }
> > if ($/ratecount > 5) then {
> > # discard message
> > stop
> > } else {
> > set $/ratecount = $/ratecount + 1;
> > }
> > }
> >
> > Would it be possible to add a $timegenerated-utc property?
> > Any pointers into the rsyslog code on where this might be implemented
> > would be welcome, and we could take a look.
> >
> > Thanks,
> > Nick
> >
> > On Mon, 2 Nov 2020 at 20:59, John Chivian via rsyslog <
> > rsyslog@lists.adiscon.com> wrote:
> >
> >> Time. What is time anyway? The easiest way to solve your problem is to
> >> punt and log everything in high resolution UTC. Or you could log
> >> everything with high resolution local time (including TZ offset). Or you
> >> could set your entire logging infrastructure use UTC only. At least
> >> then you wouldn't have the problem of the wobbling hour.
> >>
> >> All that aside, there is already an enhancement request for a high
> >> resolution $now but it's a low priority item with not much momentum.
> >> Such a thing would return server local time regardless of the timezone
> >> setting on the server (which would of course be UTC).
> >>
> >> The current $now (as you've discovered) is a low resolution string, not
> >> an actual timestamp that can be cast with a format.
> >>
> >> Regards,
> >>
> >>
> >> On 11/2/20 12:14 PM, Nicholas Brown via rsyslog wrote:
> >> > Hi,
> >> >
> >> > I'm looking for a useful monatomic timestamp property that can be used
> >> for
> >> > rate-limiting calculations in rules.
> >> >
> >> > 'parse_time($timegenerated)' mostly works, but when there are local time
> >> > adjustments happen like daylight saving and time goes backwards.
> >> >
> >> >
> >> https://www.rsyslog.com/doc/master/configuration/properties.html#system-properties
> >> >
> >> https://www.rsyslog.com/doc/master/rainerscript/functions/rs-parse_time.html
> >> >
> >> > A $timegenerated-utc property would not have daylight saving timezone
> >> > issues, but does not appear to be available. Similar $now-utc is
> >> available,
> >> > but it does not have second resolution to be useful to convert to a unix
> >> > timestamp with parse_time().
> >> >
> >> > Is there a high precision utc timestamp property available?
> >> >
> >> > Thanks,
> >> > Nick
> >> > _______________________________________________
> >> > rsyslog mailing list
> >> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > http://www.rsyslog.com/professional-services/
> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> >> you DON'T LIKE THAT.
> >>
> >>
> >> _______________________________________________
> >> rsyslog mailing list
> >> https://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com/professional-services/
> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> >> DON'T LIKE THAT.
> >
> >
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

I started to integrate the snippet from github into the doc... In a
sense, this looks like what you really after is a native ratelimiter
object that can be called during rule processing. Isn't it?

Rainer

El mié., 4 nov. 2020 a las 12:26, Rainer Gerhards
(<rgerhards@hq.adiscon.com>) escribió:
>
> sounds good! --> https://github.com/rsyslog/rsyslog/pull/4456
>
> Rainer
>
> El mar., 3 nov. 2020 a las 12:12, Nicholas Brown via rsyslog
> (<rsyslog@lists.adiscon.com>) escribió:
> >
> > An alternative that could be useful would be just make a unix timestamp
> > (with full second resolution) directly available as a variable.
> > Perhaps something like $now-unix?
> > The implementation of this could even avoid having to parse time formats,
> > thus be quick.
> > Looking at
> > https://github.com/rsyslog/rsyslog/blob/11b58367dda661c6b991122174faeb6463e5a919/runtime/datetime.c#L175:13
> > it appears
> > getCurrTime can already provide the unix seconds, and getTime() is
> > available that does only that.
> >
> > On Tue, 3 Nov 2020 at 10:24, Nicholas Brown <nickbroon+rsyslog@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > Thanks for the reply.
> > >
> > > I had found these issues:
> > > https://github.com/rsyslog/rsyslog/issues/3329 (Enhancement to $now
> > > property)
> > > https://github.com/rsyslog/rsyslog/issues/3930 (Properties timegenerated
> > > and timereported are not high precision in the RainerScript context)
> > >
> > > For our use we are looking for something with second granularity, and in
> > > UTC.
> > > $timegenerated has second granularity, but there is no $timegenerated-utc
> > > available.
> > > $now and $now-utc both only have granularity to the day.
> > >
> > > A simple example of the use case for the property in a rules might be:
> > >
> > > # Get Unix timestamp of current message
> > > set $.tnow = parse_time($timegenerated);
> > >
> > > # Rate limit info to 5 every 60 seconds
> > > if ($!severity == 6 and $!facility == 17) then {
> > > if (($.tnow - $/trate) > 60) then {
> > > # 5 seconds window expired, allow more messages
> > > set $/trate = $.tnow;
> > > set $/ratecount = 0;
> > > }
> > > if ($/ratecount > 5) then {
> > > # discard message
> > > stop
> > > } else {
> > > set $/ratecount = $/ratecount + 1;
> > > }
> > > }
> > >
> > > Would it be possible to add a $timegenerated-utc property?
> > > Any pointers into the rsyslog code on where this might be implemented
> > > would be welcome, and we could take a look.
> > >
> > > Thanks,
> > > Nick
> > >
> > > On Mon, 2 Nov 2020 at 20:59, John Chivian via rsyslog <
> > > rsyslog@lists.adiscon.com> wrote:
> > >
> > >> Time. What is time anyway? The easiest way to solve your problem is to
> > >> punt and log everything in high resolution UTC. Or you could log
> > >> everything with high resolution local time (including TZ offset). Or you
> > >> could set your entire logging infrastructure use UTC only. At least
> > >> then you wouldn't have the problem of the wobbling hour.
> > >>
> > >> All that aside, there is already an enhancement request for a high
> > >> resolution $now but it's a low priority item with not much momentum.
> > >> Such a thing would return server local time regardless of the timezone
> > >> setting on the server (which would of course be UTC).
> > >>
> > >> The current $now (as you've discovered) is a low resolution string, not
> > >> an actual timestamp that can be cast with a format.
> > >>
> > >> Regards,
> > >>
> > >>
> > >> On 11/2/20 12:14 PM, Nicholas Brown via rsyslog wrote:
> > >> > Hi,
> > >> >
> > >> > I'm looking for a useful monatomic timestamp property that can be used
> > >> for
> > >> > rate-limiting calculations in rules.
> > >> >
> > >> > 'parse_time($timegenerated)' mostly works, but when there are local time
> > >> > adjustments happen like daylight saving and time goes backwards.
> > >> >
> > >> >
> > >> https://www.rsyslog.com/doc/master/configuration/properties.html#system-properties
> > >> >
> > >> https://www.rsyslog.com/doc/master/rainerscript/functions/rs-parse_time.html
> > >> >
> > >> > A $timegenerated-utc property would not have daylight saving timezone
> > >> > issues, but does not appear to be available. Similar $now-utc is
> > >> available,
> > >> > but it does not have second resolution to be useful to convert to a unix
> > >> > timestamp with parse_time().
> > >> >
> > >> > Is there a high precision utc timestamp property available?
> > >> >
> > >> > Thanks,
> > >> > Nick
> > >> > _______________________________________________
> > >> > rsyslog mailing list
> > >> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > >> > http://www.rsyslog.com/professional-services/
> > >> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> > >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> > >> you DON'T LIKE THAT.
> > >>
> > >>
> > >> _______________________________________________
> > >> rsyslog mailing list
> > >> https://lists.adiscon.net/mailman/listinfo/rsyslog
> > >> http://www.rsyslog.com/professional-services/
> > >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> > >> DON'T LIKE THAT.
> > >
> > >
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Yes a native ratelimiter object would be useful.
The only things currently available are rather limited in
their expressiveness for rate-limiting. Ingress rate-limiting in the input
modules is too coarse and all or nothing. dequeueSlowDown only slows
processing and has complex interaction with other queue parameters needed
to control the drop needed for rate-limiting, and action
execOnlyEveryNthTime/Timeout cannot be used to define a rate-limit and
could result in no messages.
So a native rate-limiting object would be a very useful thing for rsyslog
to have.

Thanks
Nick

On Wed, 4 Nov 2020 at 15:01, Rainer Gerhards <rgerhards@hq.adiscon.com>
wrote:

> I started to integrate the snippet from github into the doc... In a
> sense, this looks like what you really after is a native ratelimiter
> object that can be called during rule processing. Isn't it?
>
> Rainer
>
> El mié., 4 nov. 2020 a las 12:26, Rainer Gerhards
> (<rgerhards@hq.adiscon.com>) escribió:
> >
> > sounds good! --> https://github.com/rsyslog/rsyslog/pull/4456
> >
> > Rainer
> >
> > El mar., 3 nov. 2020 a las 12:12, Nicholas Brown via rsyslog
> > (<rsyslog@lists.adiscon.com>) escribió:
> > >
> > > An alternative that could be useful would be just make a unix timestamp
> > > (with full second resolution) directly available as a variable.
> > > Perhaps something like $now-unix?
> > > The implementation of this could even avoid having to parse time
> formats,
> > > thus be quick.
> > > Looking at
> > >
> https://github.com/rsyslog/rsyslog/blob/11b58367dda661c6b991122174faeb6463e5a919/runtime/datetime.c#L175:13
> > > it appears
> > > getCurrTime can already provide the unix seconds, and getTime() is
> > > available that does only that.
> > >
> > > On Tue, 3 Nov 2020 at 10:24, Nicholas Brown <
> nickbroon+rsyslog@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > Thanks for the reply.
> > > >
> > > > I had found these issues:
> > > > https://github.com/rsyslog/rsyslog/issues/3329 (Enhancement to $now
> > > > property)
> > > > https://github.com/rsyslog/rsyslog/issues/3930 (Properties
> timegenerated
> > > > and timereported are not high precision in the RainerScript context)
> > > >
> > > > For our use we are looking for something with second granularity,
> and in
> > > > UTC.
> > > > $timegenerated has second granularity, but there is no
> $timegenerated-utc
> > > > available.
> > > > $now and $now-utc both only have granularity to the day.
> > > >
> > > > A simple example of the use case for the property in a rules might
> be:
> > > >
> > > > # Get Unix timestamp of current message
> > > > set $.tnow = parse_time($timegenerated);
> > > >
> > > > # Rate limit info to 5 every 60 seconds
> > > > if ($!severity == 6 and $!facility == 17) then {
> > > > if (($.tnow - $/trate) > 60) then {
> > > > # 5 seconds window expired, allow more messages
> > > > set $/trate = $.tnow;
> > > > set $/ratecount = 0;
> > > > }
> > > > if ($/ratecount > 5) then {
> > > > # discard message
> > > > stop
> > > > } else {
> > > > set $/ratecount = $/ratecount + 1;
> > > > }
> > > > }
> > > >
> > > > Would it be possible to add a $timegenerated-utc property?
> > > > Any pointers into the rsyslog code on where this might be implemented
> > > > would be welcome, and we could take a look.
> > > >
> > > > Thanks,
> > > > Nick
> > > >
> > > > On Mon, 2 Nov 2020 at 20:59, John Chivian via rsyslog <
> > > > rsyslog@lists.adiscon.com> wrote:
> > > >
> > > >> Time. What is time anyway? The easiest way to solve your problem
> is to
> > > >> punt and log everything in high resolution UTC. Or you could log
> > > >> everything with high resolution local time (including TZ offset).
> Or you
> > > >> could set your entire logging infrastructure use UTC only. At least
> > > >> then you wouldn't have the problem of the wobbling hour.
> > > >>
> > > >> All that aside, there is already an enhancement request for a high
> > > >> resolution $now but it's a low priority item with not much momentum.
> > > >> Such a thing would return server local time regardless of the
> timezone
> > > >> setting on the server (which would of course be UTC).
> > > >>
> > > >> The current $now (as you've discovered) is a low resolution string,
> not
> > > >> an actual timestamp that can be cast with a format.
> > > >>
> > > >> Regards,
> > > >>
> > > >>
> > > >> On 11/2/20 12:14 PM, Nicholas Brown via rsyslog wrote:
> > > >> > Hi,
> > > >> >
> > > >> > I'm looking for a useful monatomic timestamp property that can be
> used
> > > >> for
> > > >> > rate-limiting calculations in rules.
> > > >> >
> > > >> > 'parse_time($timegenerated)' mostly works, but when there are
> local time
> > > >> > adjustments happen like daylight saving and time goes backwards.
> > > >> >
> > > >> >
> > > >>
> https://www.rsyslog.com/doc/master/configuration/properties.html#system-properties
> > > >> >
> > > >>
> https://www.rsyslog.com/doc/master/rainerscript/functions/rs-parse_time.html
> > > >> >
> > > >> > A $timegenerated-utc property would not have daylight saving
> timezone
> > > >> > issues, but does not appear to be available. Similar $now-utc is
> > > >> available,
> > > >> > but it does not have second resolution to be useful to convert to
> a unix
> > > >> > timestamp with parse_time().
> > > >> >
> > > >> > Is there a high precision utc timestamp property available?
> > > >> >
> > > >> > Thanks,
> > > >> > Nick
> > > >> > _______________________________________________
> > > >> > rsyslog mailing list
> > > >> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >> > http://www.rsyslog.com/professional-services/
> > > >> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> > > >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
> POST if
> > > >> you DON'T LIKE THAT.
> > > >>
> > > >>
> > > >> _______________________________________________
> > > >> rsyslog mailing list
> > > >> https://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >> http://www.rsyslog.com/professional-services/
> > > >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad
> > > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> you
> > > >> DON'T LIKE THAT.
> > > >
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.