Mailing List Archive

There is not a graceful way to do what you're asking, nor would you want
to.  UTC never shifts, other time zones do and if you don't account for
this events get displaced on the timeline.  It's best to deliver the
events to a system (like a SIEM) that will put events on the timeline
correctly regardless of timezone.

Regards,

On 6/9/20 1:25 AM, Eero Volotinen via rsyslog wrote:
> Hi,
>
> My cisco asa support only utc timestamp or no timestamp in syslogs.
>
> Is it possible to modify timestamp in rsyslog and then resend to remote
> syslogger?
>
> How?
>
> Eero
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.


_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

it is possible, but not easy. you would create a custom template to use to send
to the remote system and then have to do the math to change the timestamp. We
have functions to start doing this, but I think you are likely to run into
issues where you need just a little bit more functionality than we currently
provide (but please try and help us find where we have gaps)

David Lang

On Tue, 9 Jun 2020,
Eero Volotinen via rsyslog wrote:

> Date: Tue, 9 Jun 2020 09:25:32 +0300
> From: Eero Volotinen via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog@lists.adiscon.com
> Cc: Eero Volotinen <eero.volotinen@iki.fi>
> Subject: [rsyslog] stupid question about timestamp modification
>
> Hi,
>
> My cisco asa support only utc timestamp or no timestamp in syslogs.
>
> Is it possible to modify timestamp in rsyslog and then resend to remote
> syslogger?
>
> How?
>
> Eero
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

John, SIEMs and other systems can only work with what they are given, if they
get invalid timestamps, they have to be able to figure out what the correct
timestamp is, and that is sometimes far harder than it should be if the logs are
being forwarded

David Lang

On Tue, 9 Jun 2020, John Chivian via rsyslog wrote:

> Date: Tue, 9 Jun 2020 05:59:14 -0500
> From: John Chivian via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog@lists.adiscon.com
> Cc: John Chivian <jchivian@chivian.com>
> Subject: Re: [rsyslog] stupid question about timestamp modification
>
> There is not a graceful way to do what you're asking, nor would you want
> to.  UTC never shifts, other time zones do and if you don't account for
> this events get displaced on the timeline.  It's best to deliver the
> events to a system (like a SIEM) that will put events on the timeline
> correctly regardless of timezone.
>
> Regards,
>
> On 6/9/20 1:25 AM, Eero Volotinen via rsyslog wrote:
>> Hi,
>>
>> My cisco asa support only utc timestamp or no timestamp in syslogs.
>>
>> Is it possible to modify timestamp in rsyslog and then resend to remote
>> syslogger?
>>
>> How?
>>
>> Eero
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
>
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

This is true, and why ALL timestamps should contain an offset.

Also, your note about custom templates and somewhat incomplete
functionality is exactly why I replied that there is not a "graceful"
way to do it.  I've tried it, and it's not easy or straightforward even
with the new syntax.

Unless I missed something, you have to use format_time and parse_time. 
Doing so loses any fractional seconds.  I was also unable to figure out
how to make rsyslog tell you the TZ offset of the server it's running
on.  This would be useful for appending to events that you know are from
the same timezone, but don't have the offset specified within.

Thanks for the reply,


On 6/9/20 1:47 PM, David Lang wrote:
> John, SIEMs and other systems can only work with what they are given,
> if they get invalid timestamps, they have to be able to figure out
> what the correct timestamp is, and that is sometimes far harder than
> it should be if the logs are being forwarded
>
> David Lang
>
>  On Tue, 9 Jun 2020, John Chivian via rsyslog wrote:
>
>> Date: Tue, 9 Jun 2020 05:59:14 -0500
>> From: John Chivian via rsyslog <rsyslog@lists.adiscon.com>
>> To: rsyslog@lists.adiscon.com
>> Cc: John Chivian <jchivian@chivian.com>
>> Subject: Re: [rsyslog] stupid question about timestamp modification
>>
>> There is not a graceful way to do what you're asking, nor would you
>> want to.  UTC never shifts, other time zones do and if you don't
>> account for this events get displaced on the timeline. It's best to
>> deliver the events to a system (like a SIEM) that will put events on
>> the timeline correctly regardless of timezone.
>>
>> Regards,
>>
>> On 6/9/20 1:25 AM, Eero Volotinen via rsyslog wrote:
>>> Hi,
>>>
>>> My cisco asa support only utc timestamp or no timestamp in syslogs.
>>>
>>> Is it possible to modify timestamp in rsyslog and then resend to remote
>>> syslogger?
>>>
>>> How?
>>>
>>> Eero
>>> _______________________________________________
>>> rsyslog mailing list
>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>>> myriad of
>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
>> POST if you DON'T LIKE THAT.


_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

As a long-time SIEM engineer (QRadar), disagree.

First, a lot of legacy devices can only generate RFC 3164 messages, which don't have an offset.

Second, while a lot of shops have a well-thought-out time synchronization infrastructure, that's worthless in the face of a brain-dead admin who configures a system to synch with N.pool.ntp.org (which has been blocked at the firewall because of said sync infra).

And that's only for systems that are capable of time synchronization, quite a bit of legacy cannot.

Third is latency. There are any number of potential buffer-bloat issues that can delay a message from source to logging destination.

The net result is that you simply can't rely on the timestamp in the message. You can never be certain by just looking at the message if the timestamp was accurate or not.

Because of that, the source of time truth for a SIEM is its local clock. It's why the SIEM stamps a message with the time it was received and bases all correlations, searches, reports and whatever else off of that timestamp.

Sure, the timestamp in the message is parsed as well, but it is just a data point not a point of reference.

Regards,

Jack Radigan

?On 6/9/20, 3:19 PM, "rsyslog on behalf of John Chivian via rsyslog" <rsyslog-bounces@lists.adiscon.com on behalf of rsyslog@lists.adiscon.com> wrote:

This is true, and why ALL timestamps should contain an offset.

Also, your note about custom templates and somewhat incomplete
functionality is exactly why I replied that there is not a "graceful"
way to do it. I've tried it, and it's not easy or straightforward even
with the new syntax.

Unless I missed something, you have to use format_time and parse_time.
Doing so loses any fractional seconds. I was also unable to figure out
how to make rsyslog tell you the TZ offset of the server it's running
on. This would be useful for appending to events that you know are from
the same timezone, but don't have the offset specified within.

Thanks for the reply,


On 6/9/20 1:47 PM, David Lang wrote:
> John, SIEMs and other systems can only work with what they are given,
> if they get invalid timestamps, they have to be able to figure out
> what the correct timestamp is, and that is sometimes far harder than
> it should be if the logs are being forwarded
>
> David Lang
>
> On Tue, 9 Jun 2020, John Chivian via rsyslog wrote:
>
>> Date: Tue, 9 Jun 2020 05:59:14 -0500
>> From: John Chivian via rsyslog <rsyslog@lists.adiscon.com>
>> To: rsyslog@lists.adiscon.com
>> Cc: John Chivian <jchivian@chivian.com>
>> Subject: Re: [rsyslog] stupid question about timestamp modification
>>
>> There is not a graceful way to do what you're asking, nor would you
>> want to. UTC never shifts, other time zones do and if you don't
>> account for this events get displaced on the timeline. It's best to
>> deliver the events to a system (like a SIEM) that will put events on
>> the timeline correctly regardless of timezone.
>>
>> Regards,
>>
>> On 6/9/20 1:25 AM, Eero Volotinen via rsyslog wrote:
>>> Hi,
>>>
>>> My cisco asa support only utc timestamp or no timestamp in syslogs.
>>>
>>> Is it possible to modify timestamp in rsyslog and then resend to remote
>>> syslogger?
>>>
>>> How?
>>>
>>> Eero
>>> _______________________________________________
>>> rsyslog mailing list
>>> https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.adiscon.net%2Fmailman%2Flistinfo%2Frsyslog&amp;data=02%7C01%7Cjack.radigan%40refinitiv.com%7Cbf791aecaea74456043408d80caa10ff%7C71ad2f6261e244fc9e8586c2827f6de9%7C0%7C0%7C637273271856238719&amp;sdata=gN4yi8WjEe5of8%2FFNyk1Sr4JcmW92g2niVJ3Aq7tpsk%3D&amp;reserved=0
>>> https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.rsyslog.com%2Fprofessional-services%2F&amp;data=02%7C01%7Cjack.radigan%40refinitiv.com%7Cbf791aecaea74456043408d80caa10ff%7C71ad2f6261e244fc9e8586c2827f6de9%7C0%7C0%7C637273271856238719&amp;sdata=92OtoM80IGg1uvOUiVkzJB%2BUv8CMEBgxDjUA4uAt1dw%3D&amp;reserved=0
>>> What's up with rsyslog? Follow https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Frgerhards&amp;data=02%7C01%7Cjack.radigan%40refinitiv.com%7Cbf791aecaea74456043408d80caa10ff%7C71ad2f6261e244fc9e8586c2827f6de9%7C0%7C0%7C637273271856238719&amp;sdata=BZ9YPZMiEYSzE7eZkyqOw3Qe5%2BxJJIvgguFzHpJyH7c%3D&amp;reserved=0
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>>> myriad of
>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.adiscon.net%2Fmailman%2Flistinfo%2Frsyslog&amp;data=02%7C01%7Cjack.radigan%40refinitiv.com%7Cbf791aecaea74456043408d80caa10ff%7C71ad2f6261e244fc9e8586c2827f6de9%7C0%7C0%7C637273271856238719&amp;sdata=gN4yi8WjEe5of8%2FFNyk1Sr4JcmW92g2niVJ3Aq7tpsk%3D&amp;reserved=0
>> https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.rsyslog.com%2Fprofessional-services%2F&amp;data=02%7C01%7Cjack.radigan%40refinitiv.com%7Cbf791aecaea74456043408d80caa10ff%7C71ad2f6261e244fc9e8586c2827f6de9%7C0%7C0%7C637273271856238719&amp;sdata=92OtoM80IGg1uvOUiVkzJB%2BUv8CMEBgxDjUA4uAt1dw%3D&amp;reserved=0
>> What's up with rsyslog? Follow https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Frgerhards&amp;data=02%7C01%7Cjack.radigan%40refinitiv.com%7Cbf791aecaea74456043408d80caa10ff%7C71ad2f6261e244fc9e8586c2827f6de9%7C0%7C0%7C637273271856238719&amp;sdata=BZ9YPZMiEYSzE7eZkyqOw3Qe5%2BxJJIvgguFzHpJyH7c%3D&amp;reserved=0
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
>> POST if you DON'T LIKE THAT.


_______________________________________________
rsyslog mailing list
https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.adiscon.net%2Fmailman%2Flistinfo%2Frsyslog&amp;data=02%7C01%7Cjack.radigan%40refinitiv.com%7Cbf791aecaea74456043408d80caa10ff%7C71ad2f6261e244fc9e8586c2827f6de9%7C0%7C0%7C637273271856248713&amp;sdata=XCpP%2FoL%2FAzk5HD%2B4hvVi3GmQNsZGcxWGm4H05cj%2Bofw%3D&amp;reserved=0
https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.rsyslog.com%2Fprofessional-services%2F&amp;data=02%7C01%7Cjack.radigan%40refinitiv.com%7Cbf791aecaea74456043408d80caa10ff%7C71ad2f6261e244fc9e8586c2827f6de9%7C0%7C0%7C637273271856248713&amp;sdata=Kk6zQyt8IEIxnQvCOciJH8%2FkFFh76%2Bj85zrQji%2B21Nw%3D&amp;reserved=0
What's up with rsyslog? Follow https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Frgerhards&amp;data=02%7C01%7Cjack.radigan%40refinitiv.com%7Cbf791aecaea74456043408d80caa10ff%7C71ad2f6261e244fc9e8586c2827f6de9%7C0%7C0%7C637273271856248713&amp;sdata=Z1SDkYNTKdKQ%2F%2Bah%2F09QhjE02OEk0Lc%2BjTl29807fok%3D&amp;reserved=0
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.


_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.