Mailing List Archive

There does appear to be some problem with 8.1905, so far we don't have enough
information to figure out what. If it's possible to get a debug log and/or
pstats output from a box that's not logging it would help (ideally a debug log
that includes when it stops, but that's harder to get)

one bug we do know about is if the main queue and an action/ruleset queue are
configured for disk assist mode, rsyslog will segfault, but that doesn't match
what you are seeing.

We have at least one other report of 8.1905 stopping after a half hour or so.

David Lang

On Mon, 3 Jun 2019,
Heaton, Tobias via rsyslog wrote:

> Date: Mon, 3 Jun 2019 18:06:53 +0000
> From: "Heaton, Tobias via rsyslog" <rsyslog@lists.adiscon.com>
> To: "rsyslog@lists.adiscon.com" <rsyslog@lists.adiscon.com>
> Cc: "Heaton, Tobias" <tobias.heaton@unh.edu>
> Subject: [rsyslog] 8.1905.0 Logging Stops
>
> I have two EL6 hosts running rsyslog, one of which writes log files to disk while the other writes to a MySQL database.
>
> I upgraded each host to 8.1905.0 Friday, 2019-05-30 and since that time, logging on both systems seems to stop for no apparent reason. I don't see any error messages.
>
> Restarting rsyslog on each host restores functionality for 45-60 minutes, then it stops logging again. I have rolled back to 8.1904.0 and haven't seen this behavior repeat.
>
> Is there a potential problem with 8.1905.0? The rsyslog configs on each EL host haven't changed since the upgrade Friday, nor have any other packages been installed/uninstalled.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

I'm trying to bump this thread - just tried upgrading again to 8.11905.0-2 and after about 10 minutes, all logging stops. No indication of any errors. Restarting rsyslog brings things back, but then 10 minutes later again, everything stops.

This is happening on two separate EL6 boxes. Rolling back to 8.1904.0-1 restores all functionality.

-----Original Message-----
From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
Sent: Monday, June 3, 2019 2:07 PM
To: rsyslog@lists.adiscon.com
Cc: Heaton, Tobias <tobias.heaton@unh.edu>
Subject: [rsyslog] 8.1905.0 Logging Stops

I have two EL6 hosts running rsyslog, one of which writes log files to disk while the other writes to a MySQL database.

I upgraded each host to 8.1905.0 Friday, 2019-05-30 and since that time, logging on both systems seems to stop for no apparent reason. I don't see any error messages.

Restarting rsyslog on each host restores functionality for 45-60 minutes, then it stops logging again. I have rolled back to 8.1904.0 and haven't seen this behavior repeat.

Is there a potential problem with 8.1905.0? The rsyslog configs on each EL host haven't changed since the upgrade Friday, nor have any other packages been installed/uninstalled.
_______________________________________________
rsyslog mailing list
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwICAg&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=69LNvD0eBSW_bd9n5hRj31IPXxnM93WteRS1aO85B5Y&s=2q-GoaOF6VeslY-8UgQExfBrLbo0ZaXisN9i5cDINbM&e=
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwICAg&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=69LNvD0eBSW_bd9n5hRj31IPXxnM93WteRS1aO85B5Y&s=U7eRJEj9cb1xwPtzTjtpJjVSEk0l54In0HgVbTdb_6M&e=
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

right now we have limited info on this problem

do you have impstats setup? can you show us what it outputs when logging stops?

if you can manage to get debug output over the time that it stops, that would be
wonderful.

David Lang

On Mon, 10 Jun 2019, Heaton, Tobias via rsyslog wrote:

> Date: Mon, 10 Jun 2019 14:28:50 +0000
> From: "Heaton, Tobias via rsyslog" <rsyslog@lists.adiscon.com>
> To: rsyslog-users <rsyslog@lists.adiscon.com>
> Cc: "Heaton, Tobias" <tobias.heaton@unh.edu>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> I'm trying to bump this thread - just tried upgrading again to 8.11905.0-2 and after about 10 minutes, all logging stops. No indication of any errors. Restarting rsyslog brings things back, but then 10 minutes later again, everything stops.
>
> This is happening on two separate EL6 boxes. Rolling back to 8.1904.0-1 restores all functionality.
>
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> Sent: Monday, June 3, 2019 2:07 PM
> To: rsyslog@lists.adiscon.com
> Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> Subject: [rsyslog] 8.1905.0 Logging Stops
>
> I have two EL6 hosts running rsyslog, one of which writes log files to disk while the other writes to a MySQL database.
>
> I upgraded each host to 8.1905.0 Friday, 2019-05-30 and since that time, logging on both systems seems to stop for no apparent reason. I don't see any error messages.
>
> Restarting rsyslog on each host restores functionality for 45-60 minutes, then it stops logging again. I have rolled back to 8.1904.0 and haven't seen this behavior repeat.
>
> Is there a potential problem with 8.1905.0? The rsyslog configs on each EL host haven't changed since the upgrade Friday, nor have any other packages been installed/uninstalled.
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwICAg&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=69LNvD0eBSW_bd9n5hRj31IPXxnM93WteRS1aO85B5Y&s=2q-GoaOF6VeslY-8UgQExfBrLbo0ZaXisN9i5cDINbM&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwICAg&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=69LNvD0eBSW_bd9n5hRj31IPXxnM93WteRS1aO85B5Y&s=U7eRJEj9cb1xwPtzTjtpJjVSEk0l54In0HgVbTdb_6M&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

I configured impstats, updated rsyslog, restarted and awaited the failure. Tailing the busiest logs, behavior starts normal. Over the course of a few minutes, the logs seems to arrive in chunks...or at least appear to write that way to disk.

That behavior continues with logs populating incrementally slower and slower until it just stops. The impstats don't show anything of consequence - the repeated stats are all normal and the same both prior to and after the logs stop writing.

Rsyslog appears still running and impstats continue to populate again with no appreciable changes - very strange!


-----Original Message-----
From: David Lang <david@lang.hm>
Sent: Monday, June 10, 2019 2:08 PM
To: Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
Cc: Heaton, Tobias <tobias.heaton@unh.edu>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

Caution - External Email;;;

********************************************************************************************************************************************
right now we have limited info on this problem



do you have impstats setup? can you show us what it outputs when logging stops?



if you can manage to get debug output over the time that it stops, that would be

wonderful.



David Lang



On Mon, 10 Jun 2019, Heaton, Tobias via rsyslog wrote:



> Date: Mon, 10 Jun 2019 14:28:50 +0000

> From: "Heaton, Tobias via rsyslog" <rsyslog@lists.adiscon.com>

> To: rsyslog-users <rsyslog@lists.adiscon.com>

> Cc: "Heaton, Tobias" <tobias.heaton@unh.edu>

> Subject: Re: [rsyslog] 8.1905.0 Logging Stops

>

> I'm trying to bump this thread - just tried upgrading again to 8.11905.0-2 and after about 10 minutes, all logging stops. No indication of any errors. Restarting rsyslog brings things back, but then 10 minutes later again, everything stops.

>

> This is happening on two separate EL6 boxes. Rolling back to 8.1904.0-1 restores all functionality.

>

> -----Original Message-----

> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog

> Sent: Monday, June 3, 2019 2:07 PM

> To: rsyslog@lists.adiscon.com

> Cc: Heaton, Tobias <tobias.heaton@unh.edu>

> Subject: [rsyslog] 8.1905.0 Logging Stops

>

> I have two EL6 hosts running rsyslog, one of which writes log files to disk while the other writes to a MySQL database.

>

> I upgraded each host to 8.1905.0 Friday, 2019-05-30 and since that time, logging on both systems seems to stop for no apparent reason. I don't see any error messages.

>

> Restarting rsyslog on each host restores functionality for 45-60 minutes, then it stops logging again. I have rolled back to 8.1904.0 and haven't seen this behavior repeat.

>

> Is there a potential problem with 8.1905.0? The rsyslog configs on each EL host haven't changed since the upgrade Friday, nor have any other packages been installed/uninstalled.

> _______________________________________________

> rsyslog mailing list

> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwICAg&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=69LNvD0eBSW_bd9n5hRj31IPXxnM93WteRS1aO85B5Y&s=2q-GoaOF6VeslY-8UgQExfBrLbo0ZaXisN9i5cDINbM&e=

> http://www.rsyslog.com/professional-services/

> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwICAg&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=69LNvD0eBSW_bd9n5hRj31IPXxnM93WteRS1aO85B5Y&s=U7eRJEj9cb1xwPtzTjtpJjVSEk0l54In0HgVbTdb_6M&e=

> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

> _______________________________________________

> rsyslog mailing list

> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIBAg&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=0w2XdtyZl5a52NEqzpJvoEWkZGPONmoDHL776Egzeuw&s=d-SZrocyZ9NIBE99melwdUvrw0yiTQKyli-e6xOcCy8&e=

> http://www.rsyslog.com/professional-services/

> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIBAg&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=0w2XdtyZl5a52NEqzpJvoEWkZGPONmoDHL776Egzeuw&s=FsdDeXNoDiy4qXbSpxVO7RTEdP8mRG-blTsbcPjCVTs&e=

> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

>

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

On Mon, 10 Jun 2019, Heaton, Tobias wrote:

> I configured impstats, updated rsyslog, restarted and awaited the failure. Tailing the busiest logs, behavior starts normal. Over the course of a few minutes, the logs seems to arrive in chunks...or at least appear to write that way to disk.
>
> That behavior continues with logs populating incrementally slower and slower until it just stops. The impstats don't show anything of consequence - the repeated stats are all normal and the same both prior to and after the logs stop writing.
>
> Rsyslog appears still running and impstats continue to populate again with no appreciable changes - very strange!

after it stops, does impstats show new messages arriving (and queues building)?
or does it show outputs failing and messages being discarded?
or something else?

a couple cycles of the pstats output after it's failed would be useful

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Attached is a snippet of the impstats. Logs stopped writing at 15:00:01.

-----Original Message-----
From: David Lang <david@lang.hm>
Sent: Monday, June 10, 2019 3:29 PM
To: Heaton, Tobias <tobias.heaton@unh.edu>
Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] 8.1905.0 Logging Stops

Caution - External Email;;;

********************************************************************************************************************************************
On Mon, 10 Jun 2019, Heaton, Tobias wrote:

> I configured impstats, updated rsyslog, restarted and awaited the failure. Tailing the busiest logs, behavior starts normal. Over the course of a few minutes, the logs seems to arrive in chunks...or at least appear to write that way to disk.
>
> That behavior continues with logs populating incrementally slower and slower until it just stops. The impstats don't show anything of consequence - the repeated stats are all normal and the same both prior to and after the logs stop writing.
>
> Rsyslog appears still running and impstats continue to populate again with no appreciable changes - very strange!

after it stops, does impstats show new messages arriving (and queues building)?
or does it show outputs failing and messages being discarded?
or something else?

a couple cycles of the pstats output after it's failed would be useful

David Lang

do you have impstats configured to reset it's counters each time it outputs?

David Lang

On Mon, 10 Jun 2019, Heaton, Tobias wrote:

> Date: Mon, 10 Jun 2019 19:37:31 +0000
> From: "Heaton, Tobias" <tobias.heaton@unh.edu>
> To: David Lang <david@lang.hm>
> Cc: "Heaton, Tobias via rsyslog" <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>
> Attached is a snippet of the impstats. Logs stopped writing at 15:00:01.
>
> -----Original Message-----
> From: David Lang <david@lang.hm>
> Sent: Monday, June 10, 2019 3:29 PM
> To: Heaton, Tobias <tobias.heaton@unh.edu>
> Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> On Mon, 10 Jun 2019, Heaton, Tobias wrote:
>
>> I configured impstats, updated rsyslog, restarted and awaited the failure. Tailing the busiest logs, behavior starts normal. Over the course of a few minutes, the logs seems to arrive in chunks...or at least appear to write that way to disk.
>>
>> That behavior continues with logs populating incrementally slower and slower until it just stops. The impstats don't show anything of consequence - the repeated stats are all normal and the same both prior to and after the logs stop writing.
>>
>> Rsyslog appears still running and impstats continue to populate again with no appreciable changes - very strange!
>
> after it stops, does impstats show new messages arriving (and queues building)?
> or does it show outputs failing and messages being discarded?
> or something else?
>
> a couple cycles of the pstats output after it's failed would be useful
>
> David Lang
>

I do - should I not?

-----Original Message-----
From: David Lang <david@lang.hm>
Sent: Monday, June 10, 2019 3:43 PM
To: Heaton, Tobias <tobias.heaton@unh.edu>
Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] 8.1905.0 Logging Stops

Caution - External Email;;;

********************************************************************************************************************************************
do you have impstats configured to reset it's counters each time it outputs?

David Lang

On Mon, 10 Jun 2019, Heaton, Tobias wrote:

> Date: Mon, 10 Jun 2019 19:37:31 +0000
> From: "Heaton, Tobias" <tobias.heaton@unh.edu>
> To: David Lang <david@lang.hm>
> Cc: "Heaton, Tobias via rsyslog" <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>
> Attached is a snippet of the impstats. Logs stopped writing at 15:00:01.
>
> -----Original Message-----
> From: David Lang <david@lang.hm>
> Sent: Monday, June 10, 2019 3:29 PM
> To: Heaton, Tobias <tobias.heaton@unh.edu>
> Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> On Mon, 10 Jun 2019, Heaton, Tobias wrote:
>
>> I configured impstats, updated rsyslog, restarted and awaited the failure. Tailing the busiest logs, behavior starts normal. Over the course of a few minutes, the logs seems to arrive in chunks...or at least appear to write that way to disk.
>>
>> That behavior continues with logs populating incrementally slower and slower until it just stops. The impstats don't show anything of consequence - the repeated stats are all normal and the same both prior to and after the logs stop writing.
>>
>> Rsyslog appears still running and impstats continue to populate again with no appreciable changes - very strange!
>
> after it stops, does impstats show new messages arriving (and queues building)?
> or does it show outputs failing and messages being discarded?
> or something else?
>
> a couple cycles of the pstats output after it's failed would be useful
>
> David Lang
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

by the way, looking at your evicted stats, you probably want to tweak your
dynafilecachesize on some of your outputs (performance issue, probably not
related to this issue)

data was still being processed by outputs through the 15:00:17 window. It didn't
stop until the 15:00:22 window.

If you can get a debug log (send to Rainer) through the time it crashes, that
would be helpful

Also, can you post your config?

David Lang

On Mon, 10 Jun 2019, Heaton, Tobias wrote:

> Date: Mon, 10 Jun 2019 19:37:31 +0000
> From: "Heaton, Tobias" <tobias.heaton@unh.edu>
> To: David Lang <david@lang.hm>
> Cc: "Heaton, Tobias via rsyslog" <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>
> Attached is a snippet of the impstats. Logs stopped writing at 15:00:01.
>
> -----Original Message-----
> From: David Lang <david@lang.hm>
> Sent: Monday, June 10, 2019 3:29 PM
> To: Heaton, Tobias <tobias.heaton@unh.edu>
> Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> On Mon, 10 Jun 2019, Heaton, Tobias wrote:
>
>> I configured impstats, updated rsyslog, restarted and awaited the failure. Tailing the busiest logs, behavior starts normal. Over the course of a few minutes, the logs seems to arrive in chunks...or at least appear to write that way to disk.
>>
>> That behavior continues with logs populating incrementally slower and slower until it just stops. The impstats don't show anything of consequence - the repeated stats are all normal and the same both prior to and after the logs stop writing.
>>
>> Rsyslog appears still running and impstats continue to populate again with no appreciable changes - very strange!
>
> after it stops, does impstats show new messages arriving (and queues building)?
> or does it show outputs failing and messages being discarded?
> or something else?
>
> a couple cycles of the pstats output after it's failed would be useful
>
> David Lang
>

On Mon, 10 Jun 2019, Heaton, Tobias wrote:

> I do - should I not?

the stats are a little more inaccurate (there is a race condition between
the time that the stats are reported and when they are cleared that new events
can be lost) beyond that it's mostly preference, but the stats analysis tool on
the rsyslog.com website assumes that the counters are not reset.

I don't think it's a big deal, just confirming.

David Lang

> -----Original Message-----
> From: David Lang <david@lang.hm>
> Sent: Monday, June 10, 2019 3:43 PM
> To: Heaton, Tobias <tobias.heaton@unh.edu>
> Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> do you have impstats configured to reset it's counters each time it outputs?
>
> David Lang
>
> On Mon, 10 Jun 2019, Heaton, Tobias wrote:
>
>> Date: Mon, 10 Jun 2019 19:37:31 +0000
>> From: "Heaton, Tobias" <tobias.heaton@unh.edu>
>> To: David Lang <david@lang.hm>
>> Cc: "Heaton, Tobias via rsyslog" <rsyslog@lists.adiscon.com>
>> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>>
>> Attached is a snippet of the impstats. Logs stopped writing at 15:00:01.
>>
>> -----Original Message-----
>> From: David Lang <david@lang.hm>
>> Sent: Monday, June 10, 2019 3:29 PM
>> To: Heaton, Tobias <tobias.heaton@unh.edu>
>> Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
>> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>>
>> Caution - External Email;;;
>>
>> ********************************************************************************************************************************************
>> On Mon, 10 Jun 2019, Heaton, Tobias wrote:
>>
>>> I configured impstats, updated rsyslog, restarted and awaited the failure. Tailing the busiest logs, behavior starts normal. Over the course of a few minutes, the logs seems to arrive in chunks...or at least appear to write that way to disk.
>>>
>>> That behavior continues with logs populating incrementally slower and slower until it just stops. The impstats don't show anything of consequence - the repeated stats are all normal and the same both prior to and after the logs stop writing.
>>>
>>> Rsyslog appears still running and impstats continue to populate again with no appreciable changes - very strange!
>>
>> after it stops, does impstats show new messages arriving (and queues building)?
>> or does it show outputs failing and messages being discarded?
>> or something else?
>>
>> a couple cycles of the pstats output after it's failed would be useful
>>
>> David Lang
>>
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

My apologies - I'm a little na?ve when it comes to the debugging.

What's your recommendation on the best way to capture debug?

-----Original Message-----
From: David Lang <david@lang.hm>
Sent: Monday, June 10, 2019 3:50 PM
To: Heaton, Tobias <tobias.heaton@unh.edu>
Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] 8.1905.0 Logging Stops

Caution - External Email;;;

********************************************************************************************************************************************
by the way, looking at your evicted stats, you probably want to tweak your
dynafilecachesize on some of your outputs (performance issue, probably not
related to this issue)

data was still being processed by outputs through the 15:00:17 window. It didn't
stop until the 15:00:22 window.

If you can get a debug log (send to Rainer) through the time it crashes, that
would be helpful

Also, can you post your config?

David Lang

On Mon, 10 Jun 2019, Heaton, Tobias wrote:

> Date: Mon, 10 Jun 2019 19:37:31 +0000
> From: "Heaton, Tobias" <tobias.heaton@unh.edu>
> To: David Lang <david@lang.hm>
> Cc: "Heaton, Tobias via rsyslog" <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>
> Attached is a snippet of the impstats. Logs stopped writing at 15:00:01.
>
> -----Original Message-----
> From: David Lang <david@lang.hm>
> Sent: Monday, June 10, 2019 3:29 PM
> To: Heaton, Tobias <tobias.heaton@unh.edu>
> Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> On Mon, 10 Jun 2019, Heaton, Tobias wrote:
>
>> I configured impstats, updated rsyslog, restarted and awaited the failure. Tailing the busiest logs, behavior starts normal. Over the course of a few minutes, the logs seems to arrive in chunks...or at least appear to write that way to disk.
>>
>> That behavior continues with logs populating incrementally slower and slower until it just stops. The impstats don't show anything of consequence - the repeated stats are all normal and the same both prior to and after the logs stop writing.
>>
>> Rsyslog appears still running and impstats continue to populate again with no appreciable changes - very strange!
>
> after it stops, does impstats show new messages arriving (and queues building)?
> or does it show outputs failing and messages being discarded?
> or something else?
>
> a couple cycles of the pstats output after it's failed would be useful
>
> David Lang
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Hi Tobias,

this is the "quick guide" to generate a debug log:
https://www.rsyslog.com/doc/v8-stable/troubleshooting/howtodebug.html
- just do the "regular debug" part and let it run until the problem
manifests.

We also have more elaborate ones, but I guess this one is sufficient
(let's keep it smooth and simple ;-)).

Rainer

El lun., 10 jun. 2019 a las 21:58, Heaton, Tobias via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> My apologies - I'm a little naïve when it comes to the debugging.
>
> What's your recommendation on the best way to capture debug?
>
> -----Original Message-----
> From: David Lang <david@lang.hm>
> Sent: Monday, June 10, 2019 3:50 PM
> To: Heaton, Tobias <tobias.heaton@unh.edu>
> Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> by the way, looking at your evicted stats, you probably want to tweak your
> dynafilecachesize on some of your outputs (performance issue, probably not
> related to this issue)
>
> data was still being processed by outputs through the 15:00:17 window. It didn't
> stop until the 15:00:22 window.
>
> If you can get a debug log (send to Rainer) through the time it crashes, that
> would be helpful
>
> Also, can you post your config?
>
> David Lang
>
> On Mon, 10 Jun 2019, Heaton, Tobias wrote:
>
> > Date: Mon, 10 Jun 2019 19:37:31 +0000
> > From: "Heaton, Tobias" <tobias.heaton@unh.edu>
> > To: David Lang <david@lang.hm>
> > Cc: "Heaton, Tobias via rsyslog" <rsyslog@lists.adiscon.com>
> > Subject: RE: [rsyslog] 8.1905.0 Logging Stops
> >
> > Attached is a snippet of the impstats. Logs stopped writing at 15:00:01.
> >
> > -----Original Message-----
> > From: David Lang <david@lang.hm>
> > Sent: Monday, June 10, 2019 3:29 PM
> > To: Heaton, Tobias <tobias.heaton@unh.edu>
> > Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> > Subject: RE: [rsyslog] 8.1905.0 Logging Stops
> >
> > Caution - External Email;;;
> >
> > ********************************************************************************************************************************************
> > On Mon, 10 Jun 2019, Heaton, Tobias wrote:
> >
> >> I configured impstats, updated rsyslog, restarted and awaited the failure. Tailing the busiest logs, behavior starts normal. Over the course of a few minutes, the logs seems to arrive in chunks...or at least appear to write that way to disk.
> >>
> >> That behavior continues with logs populating incrementally slower and slower until it just stops. The impstats don't show anything of consequence - the repeated stats are all normal and the same both prior to and after the logs stop writing.
> >>
> >> Rsyslog appears still running and impstats continue to populate again with no appreciable changes - very strange!
> >
> > after it stops, does impstats show new messages arriving (and queues building)?
> > or does it show outputs failing and messages being discarded?
> > or something else?
> >
> > a couple cycles of the pstats output after it's failed would be useful
> >
> > David Lang
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Hello Rainer,

Thank you for the information. I set the debug configuration, updated rsyslog and restarted. After about 5 minutes, the logs are starting to slow and the debug log is well over 12GB in size.

It seems having debug enabled slows general performance considerably and given the rate at which the debug log is building, finding relevant, helpful data might be a challenge...not to mention getting you the file itself!

Tobias

-----Original Message-----
From: Rainer Gerhards <rgerhards@hq.adiscon.com>
Sent: Tuesday, June 11, 2019 7:33 AM
To: rsyslog-users <rsyslog@lists.adiscon.com>
Cc: David Lang <david@lang.hm>; Heaton, Tobias <tobias.heaton@unh.edu>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

Caution - External Email;;;

********************************************************************************************************************************************
Hi Tobias,

this is the "quick guide" to generate a debug log:
https://www.rsyslog.com/doc/v8-stable/troubleshooting/howtodebug.html
- just do the "regular debug" part and let it run until the problem
manifests.

We also have more elaborate ones, but I guess this one is sufficient
(let's keep it smooth and simple ;-)).

Rainer

El lun., 10 jun. 2019 a las 21:58, Heaton, Tobias via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> My apologies - I'm a little naïve when it comes to the debugging.
>
> What's your recommendation on the best way to capture debug?
>
> -----Original Message-----
> From: David Lang <david@lang.hm>
> Sent: Monday, June 10, 2019 3:50 PM
> To: Heaton, Tobias <tobias.heaton@unh.edu>
> Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> by the way, looking at your evicted stats, you probably want to tweak your
> dynafilecachesize on some of your outputs (performance issue, probably not
> related to this issue)
>
> data was still being processed by outputs through the 15:00:17 window. It didn't
> stop until the 15:00:22 window.
>
> If you can get a debug log (send to Rainer) through the time it crashes, that
> would be helpful
>
> Also, can you post your config?
>
> David Lang
>
> On Mon, 10 Jun 2019, Heaton, Tobias wrote:
>
> > Date: Mon, 10 Jun 2019 19:37:31 +0000
> > From: "Heaton, Tobias" <tobias.heaton@unh.edu>
> > To: David Lang <david@lang.hm>
> > Cc: "Heaton, Tobias via rsyslog" <rsyslog@lists.adiscon.com>
> > Subject: RE: [rsyslog] 8.1905.0 Logging Stops
> >
> > Attached is a snippet of the impstats. Logs stopped writing at 15:00:01.
> >
> > -----Original Message-----
> > From: David Lang <david@lang.hm>
> > Sent: Monday, June 10, 2019 3:29 PM
> > To: Heaton, Tobias <tobias.heaton@unh.edu>
> > Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> > Subject: RE: [rsyslog] 8.1905.0 Logging Stops
> >
> > Caution - External Email;;;
> >
> > ********************************************************************************************************************************************
> > On Mon, 10 Jun 2019, Heaton, Tobias wrote:
> >
> >> I configured impstats, updated rsyslog, restarted and awaited the failure. Tailing the busiest logs, behavior starts normal. Over the course of a few minutes, the logs seems to arrive in chunks...or at least appear to write that way to disk.
> >>
> >> That behavior continues with logs populating incrementally slower and slower until it just stops. The impstats don't show anything of consequence - the repeated stats are all normal and the same both prior to and after the logs stop writing.
> >>
> >> Rsyslog appears still running and impstats continue to populate again with no appreciable changes - very strange!
> >
> > after it stops, does impstats show new messages arriving (and queues building)?
> > or does it show outputs failing and messages being discarded?
> > or something else?
> >
> > a couple cycles of the pstats output after it's failed would be useful
> >
> > David Lang
> >
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=lhTjg_QDGIeHOeCAcLX5bXSSgEoxAn4zMhoKMn39_Q0&s=AIKzntUgGqWkTqo_j2lfE7vckKRK0v9bS913ZhCug6k&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=lhTjg_QDGIeHOeCAcLX5bXSSgEoxAn4zMhoKMn39_Q0&s=Wv4TJP-48wkRkAh4kLGr1DRn9AJcn69V5upqPFFcXkY&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

El mar., 11 jun. 2019 a las 14:33, Heaton, Tobias
(<tobias.heaton@unh.edu>) escribió:
>
> Hello Rainer,
>
> Thank you for the information. I set the debug configuration, updated rsyslog and restarted. After about 5 minutes, the logs are starting to slow and the debug log is well over 12GB in size.
>
> It seems having debug enabled slows general performance considerably and given the rate at which the debug log is building, finding relevant, helpful data might be a challenge...not to mention getting you the file itself!

If the 12GiB contain the actual error case, than it is not too bad - I
am used to wading though large debug files. These files usually
compress very well (xz recommended), so the transfer is also not too
bad. It would be best, though, if we can go to a point where the log
really stalls.

For starters, I would guess that the last minute or so of data could
be interesting.

Rainer
>
> Tobias
>
> -----Original Message-----
> From: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Sent: Tuesday, June 11, 2019 7:33 AM
> To: rsyslog-users <rsyslog@lists.adiscon.com>
> Cc: David Lang <david@lang.hm>; Heaton, Tobias <tobias.heaton@unh.edu>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> Hi Tobias,
>
> this is the "quick guide" to generate a debug log:
> https://www.rsyslog.com/doc/v8-stable/troubleshooting/howtodebug.html
> - just do the "regular debug" part and let it run until the problem
> manifests.
>
> We also have more elaborate ones, but I guess this one is sufficient
> (let's keep it smooth and simple ;-)).
>
> Rainer
>
> El lun., 10 jun. 2019 a las 21:58, Heaton, Tobias via rsyslog
> (<rsyslog@lists.adiscon.com>) escribió:
> >
> > My apologies - I'm a little naïve when it comes to the debugging.
> >
> > What's your recommendation on the best way to capture debug?
> >
> > -----Original Message-----
> > From: David Lang <david@lang.hm>
> > Sent: Monday, June 10, 2019 3:50 PM
> > To: Heaton, Tobias <tobias.heaton@unh.edu>
> > Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> > Subject: RE: [rsyslog] 8.1905.0 Logging Stops
> >
> > Caution - External Email;;;
> >
> > ********************************************************************************************************************************************
> > by the way, looking at your evicted stats, you probably want to tweak your
> > dynafilecachesize on some of your outputs (performance issue, probably not
> > related to this issue)
> >
> > data was still being processed by outputs through the 15:00:17 window. It didn't
> > stop until the 15:00:22 window.
> >
> > If you can get a debug log (send to Rainer) through the time it crashes, that
> > would be helpful
> >
> > Also, can you post your config?
> >
> > David Lang
> >
> > On Mon, 10 Jun 2019, Heaton, Tobias wrote:
> >
> > > Date: Mon, 10 Jun 2019 19:37:31 +0000
> > > From: "Heaton, Tobias" <tobias.heaton@unh.edu>
> > > To: David Lang <david@lang.hm>
> > > Cc: "Heaton, Tobias via rsyslog" <rsyslog@lists.adiscon.com>
> > > Subject: RE: [rsyslog] 8.1905.0 Logging Stops
> > >
> > > Attached is a snippet of the impstats. Logs stopped writing at 15:00:01.
> > >
> > > -----Original Message-----
> > > From: David Lang <david@lang.hm>
> > > Sent: Monday, June 10, 2019 3:29 PM
> > > To: Heaton, Tobias <tobias.heaton@unh.edu>
> > > Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> > > Subject: RE: [rsyslog] 8.1905.0 Logging Stops
> > >
> > > Caution - External Email;;;
> > >
> > > ********************************************************************************************************************************************
> > > On Mon, 10 Jun 2019, Heaton, Tobias wrote:
> > >
> > >> I configured impstats, updated rsyslog, restarted and awaited the failure. Tailing the busiest logs, behavior starts normal. Over the course of a few minutes, the logs seems to arrive in chunks...or at least appear to write that way to disk.
> > >>
> > >> That behavior continues with logs populating incrementally slower and slower until it just stops. The impstats don't show anything of consequence - the repeated stats are all normal and the same both prior to and after the logs stop writing.
> > >>
> > >> Rsyslog appears still running and impstats continue to populate again with no appreciable changes - very strange!
> > >
> > > after it stops, does impstats show new messages arriving (and queues building)?
> > > or does it show outputs failing and messages being discarded?
> > > or something else?
> > >
> > > a couple cycles of the pstats output after it's failed would be useful
> > >
> > > David Lang
> > >
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=lhTjg_QDGIeHOeCAcLX5bXSSgEoxAn4zMhoKMn39_Q0&s=AIKzntUgGqWkTqo_j2lfE7vckKRK0v9bS913ZhCug6k&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=lhTjg_QDGIeHOeCAcLX5bXSSgEoxAn4zMhoKMn39_Q0&s=Wv4TJP-48wkRkAh4kLGr1DRn9AJcn69V5upqPFFcXkY&e=
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Ok - I will set things up and do my best to capture the failure and let you know when I have logs.

Thanks for your help - appreciate it!

> On Jun 11, 2019, at 8:53 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> El mar., 11 jun. 2019 a las 14:33, Heaton, Tobias
> (<tobias.heaton@unh.edu>) escribió:
>>
>> Hello Rainer,
>>
>> Thank you for the information. I set the debug configuration, updated rsyslog and restarted. After about 5 minutes, the logs are starting to slow and the debug log is well over 12GB in size.
>>
>> It seems having debug enabled slows general performance considerably and given the rate at which the debug log is building, finding relevant, helpful data might be a challenge...not to mention getting you the file itself!
>
> If the 12GiB contain the actual error case, than it is not too bad - I
> am used to wading though large debug files. These files usually
> compress very well (xz recommended), so the transfer is also not too
> bad. It would be best, though, if we can go to a point where the log
> really stalls.
>
> For starters, I would guess that the last minute or so of data could
> be interesting.
>
> Rainer
>>
>> Tobias
>>
>> -----Original Message-----
>> From: Rainer Gerhards <rgerhards@hq.adiscon.com>
>> Sent: Tuesday, June 11, 2019 7:33 AM
>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>> Cc: David Lang <david@lang.hm>; Heaton, Tobias <tobias.heaton@unh.edu>
>> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>>
>> Caution - External Email;;;
>>
>> ********************************************************************************************************************************************
>> Hi Tobias,
>>
>> this is the "quick guide" to generate a debug log:
>> https://www.rsyslog.com/doc/v8-stable/troubleshooting/howtodebug.html
>> - just do the "regular debug" part and let it run until the problem
>> manifests.
>>
>> We also have more elaborate ones, but I guess this one is sufficient
>> (let's keep it smooth and simple ;-)).
>>
>> Rainer
>>
>> El lun., 10 jun. 2019 a las 21:58, Heaton, Tobias via rsyslog
>> (<rsyslog@lists.adiscon.com>) escribió:
>>>
>>> My apologies - I'm a little naïve when it comes to the debugging.
>>>
>>> What's your recommendation on the best way to capture debug?
>>>
>>> -----Original Message-----
>>> From: David Lang <david@lang.hm>
>>> Sent: Monday, June 10, 2019 3:50 PM
>>> To: Heaton, Tobias <tobias.heaton@unh.edu>
>>> Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
>>> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>>>
>>> Caution - External Email;;;
>>>
>>> ********************************************************************************************************************************************
>>> by the way, looking at your evicted stats, you probably want to tweak your
>>> dynafilecachesize on some of your outputs (performance issue, probably not
>>> related to this issue)
>>>
>>> data was still being processed by outputs through the 15:00:17 window. It didn't
>>> stop until the 15:00:22 window.
>>>
>>> If you can get a debug log (send to Rainer) through the time it crashes, that
>>> would be helpful
>>>
>>> Also, can you post your config?
>>>
>>> David Lang
>>>
>>>> On Mon, 10 Jun 2019, Heaton, Tobias wrote:
>>>>
>>>> Date: Mon, 10 Jun 2019 19:37:31 +0000
>>>> From: "Heaton, Tobias" <tobias.heaton@unh.edu>
>>>> To: David Lang <david@lang.hm>
>>>> Cc: "Heaton, Tobias via rsyslog" <rsyslog@lists.adiscon.com>
>>>> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>>>>
>>>> Attached is a snippet of the impstats. Logs stopped writing at 15:00:01.
>>>>
>>>> -----Original Message-----
>>>> From: David Lang <david@lang.hm>
>>>> Sent: Monday, June 10, 2019 3:29 PM
>>>> To: Heaton, Tobias <tobias.heaton@unh.edu>
>>>> Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
>>>> Subject: RE: [rsyslog] 8.1905.0 Logging Stops
>>>>
>>>> Caution - External Email;;;
>>>>
>>>> ********************************************************************************************************************************************
>>>>> On Mon, 10 Jun 2019, Heaton, Tobias wrote:
>>>>>
>>>>> I configured impstats, updated rsyslog, restarted and awaited the failure. Tailing the busiest logs, behavior starts normal. Over the course of a few minutes, the logs seems to arrive in chunks...or at least appear to write that way to disk.
>>>>>
>>>>> That behavior continues with logs populating incrementally slower and slower until it just stops. The impstats don't show anything of consequence - the repeated stats are all normal and the same both prior to and after the logs stop writing.
>>>>>
>>>>> Rsyslog appears still running and impstats continue to populate again with no appreciable changes - very strange!
>>>>
>>>> after it stops, does impstats show new messages arriving (and queues building)?
>>>> or does it show outputs failing and messages being discarded?
>>>> or something else?
>>>>
>>>> a couple cycles of the pstats output after it's failed would be useful
>>>>
>>>> David Lang
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=lhTjg_QDGIeHOeCAcLX5bXSSgEoxAn4zMhoKMn39_Q0&s=AIKzntUgGqWkTqo_j2lfE7vckKRK0v9bS913ZhCug6k&e=
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=lhTjg_QDGIeHOeCAcLX5bXSSgEoxAn4zMhoKMn39_Q0&s=Wv4TJP-48wkRkAh4kLGr1DRn9AJcn69V5upqPFFcXkY&e=
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Rainer,

I traced my steps and allowed rsyslog to run for 45 minutes without seeing a failure.

The debug puts considerable strain on the system, so much so that the log timestamps started drifting. I was seeing logs stamped at 13:18 EST at 13:31EST actual time.

Furthermore the debug log is nearly 70GB, closing in on the available disk space.

Is there anything else I can try to help troubleshoot this problem?

Tobias

-----Original Message-----
From: Rainer Gerhards <rgerhards@hq.adiscon.com>
Sent: Tuesday, June 11, 2019 8:53 AM
To: Heaton, Tobias <tobias.heaton@unh.edu>
Cc: rsyslog-users <rsyslog@lists.adiscon.com>; David Lang <david@lang.hm>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

Caution - External Email;;;

********************************************************************************************************************************************
El mar., 11 jun. 2019 a las 14:33, Heaton, Tobias
(<tobias.heaton@unh.edu>) escribió:
>
> Hello Rainer,
>
> Thank you for the information. I set the debug configuration, updated rsyslog and restarted. After about 5 minutes, the logs are starting to slow and the debug log is well over 12GB in size.
>
> It seems having debug enabled slows general performance considerably and given the rate at which the debug log is building, finding relevant, helpful data might be a challenge...not to mention getting you the file itself!

If the 12GiB contain the actual error case, than it is not too bad - I
am used to wading though large debug files. These files usually
compress very well (xz recommended), so the transfer is also not too
bad. It would be best, though, if we can go to a point where the log
really stalls.

For starters, I would guess that the last minute or so of data could
be interesting.

Rainer
>
> Tobias
>
> -----Original Message-----
> From: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Sent: Tuesday, June 11, 2019 7:33 AM
> To: rsyslog-users <rsyslog@lists.adiscon.com>
> Cc: David Lang <david@lang.hm>; Heaton, Tobias <tobias.heaton@unh.edu>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> Hi Tobias,
>
> this is the "quick guide" to generate a debug log:
> https://www.rsyslog.com/doc/v8-stable/troubleshooting/howtodebug.html
> - just do the "regular debug" part and let it run until the problem
> manifests.
>
> We also have more elaborate ones, but I guess this one is sufficient
> (let's keep it smooth and simple ;-)).
>
> Rainer
>
> El lun., 10 jun. 2019 a las 21:58, Heaton, Tobias via rsyslog
> (<rsyslog@lists.adiscon.com>) escribió:
> >
> > My apologies - I'm a little naïve when it comes to the debugging.
> >
> > What's your recommendation on the best way to capture debug?
> >
> > -----Original Message-----
> > From: David Lang <david@lang.hm>
> > Sent: Monday, June 10, 2019 3:50 PM
> > To: Heaton, Tobias <tobias.heaton@unh.edu>
> > Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> > Subject: RE: [rsyslog] 8.1905.0 Logging Stops
> >
> > Caution - External Email;;;
> >
> > ********************************************************************************************************************************************
> > by the way, looking at your evicted stats, you probably want to tweak your
> > dynafilecachesize on some of your outputs (performance issue, probably not
> > related to this issue)
> >
> > data was still being processed by outputs through the 15:00:17 window. It didn't
> > stop until the 15:00:22 window.
> >
> > If you can get a debug log (send to Rainer) through the time it crashes, that
> > would be helpful
> >
> > Also, can you post your config?
> >
> > David Lang
> >
> > On Mon, 10 Jun 2019, Heaton, Tobias wrote:
> >
> > > Date: Mon, 10 Jun 2019 19:37:31 +0000
> > > From: "Heaton, Tobias" <tobias.heaton@unh.edu>
> > > To: David Lang <david@lang.hm>
> > > Cc: "Heaton, Tobias via rsyslog" <rsyslog@lists.adiscon.com>
> > > Subject: RE: [rsyslog] 8.1905.0 Logging Stops
> > >
> > > Attached is a snippet of the impstats. Logs stopped writing at 15:00:01.
> > >
> > > -----Original Message-----
> > > From: David Lang <david@lang.hm>
> > > Sent: Monday, June 10, 2019 3:29 PM
> > > To: Heaton, Tobias <tobias.heaton@unh.edu>
> > > Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> > > Subject: RE: [rsyslog] 8.1905.0 Logging Stops
> > >
> > > Caution - External Email;;;
> > >
> > > ********************************************************************************************************************************************
> > > On Mon, 10 Jun 2019, Heaton, Tobias wrote:
> > >
> > >> I configured impstats, updated rsyslog, restarted and awaited the failure. Tailing the busiest logs, behavior starts normal. Over the course of a few minutes, the logs seems to arrive in chunks...or at least appear to write that way to disk.
> > >>
> > >> That behavior continues with logs populating incrementally slower and slower until it just stops. The impstats don't show anything of consequence - the repeated stats are all normal and the same both prior to and after the logs stop writing.
> > >>
> > >> Rsyslog appears still running and impstats continue to populate again with no appreciable changes - very strange!
> > >
> > > after it stops, does impstats show new messages arriving (and queues building)?
> > > or does it show outputs failing and messages being discarded?
> > > or something else?
> > >
> > > a couple cycles of the pstats output after it's failed would be useful
> > >
> > > David Lang
> > >
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=lhTjg_QDGIeHOeCAcLX5bXSSgEoxAn4zMhoKMn39_Q0&s=AIKzntUgGqWkTqo_j2lfE7vckKRK0v9bS913ZhCug6k&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=lhTjg_QDGIeHOeCAcLX5bXSSgEoxAn4zMhoKMn39_Q0&s=Wv4TJP-48wkRkAh4kLGr1DRn9AJcn69V5upqPFFcXkY&e=
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

El mar., 11 jun. 2019 a las 19:23, Heaton, Tobias
(<tobias.heaton@unh.edu>) escribió:
>
> Rainer,
>
> I traced my steps and allowed rsyslog to run for 45 minutes without seeing a failure.
>
> The debug puts considerable strain on the system, so much so that the log timestamps started drifting. I was seeing logs stamped at 13:18 EST at 13:31EST actual time.
>
> Furthermore the debug log is nearly 70GB, closing in on the available disk space.
>
> Is there anything else I can try to help troubleshoot this problem?

Not really. Maybe the first 12 GiB actually contain info that get's us
a bit closer. So if you could share that, it would be useful. We can
fine-tune the debug output, but as we have not the slightest idea of
what may be the root cause that's no option.

Rainer
>
> Tobias
>
> -----Original Message-----
> From: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Sent: Tuesday, June 11, 2019 8:53 AM
> To: Heaton, Tobias <tobias.heaton@unh.edu>
> Cc: rsyslog-users <rsyslog@lists.adiscon.com>; David Lang <david@lang.hm>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> El mar., 11 jun. 2019 a las 14:33, Heaton, Tobias
> (<tobias.heaton@unh.edu>) escribió:
> >
> > Hello Rainer,
> >
> > Thank you for the information. I set the debug configuration, updated rsyslog and restarted. After about 5 minutes, the logs are starting to slow and the debug log is well over 12GB in size.
> >
> > It seems having debug enabled slows general performance considerably and given the rate at which the debug log is building, finding relevant, helpful data might be a challenge...not to mention getting you the file itself!
>
> If the 12GiB contain the actual error case, than it is not too bad - I
> am used to wading though large debug files. These files usually
> compress very well (xz recommended), so the transfer is also not too
> bad. It would be best, though, if we can go to a point where the log
> really stalls.
>
> For starters, I would guess that the last minute or so of data could
> be interesting.
>
> Rainer
> >
> > Tobias
> >
> > -----Original Message-----
> > From: Rainer Gerhards <rgerhards@hq.adiscon.com>
> > Sent: Tuesday, June 11, 2019 7:33 AM
> > To: rsyslog-users <rsyslog@lists.adiscon.com>
> > Cc: David Lang <david@lang.hm>; Heaton, Tobias <tobias.heaton@unh.edu>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > Caution - External Email;;;
> >
> > ********************************************************************************************************************************************
> > Hi Tobias,
> >
> > this is the "quick guide" to generate a debug log:
> > https://www.rsyslog.com/doc/v8-stable/troubleshooting/howtodebug.html
> > - just do the "regular debug" part and let it run until the problem
> > manifests.
> >
> > We also have more elaborate ones, but I guess this one is sufficient
> > (let's keep it smooth and simple ;-)).
> >
> > Rainer
> >
> > El lun., 10 jun. 2019 a las 21:58, Heaton, Tobias via rsyslog
> > (<rsyslog@lists.adiscon.com>) escribió:
> > >
> > > My apologies - I'm a little naïve when it comes to the debugging.
> > >
> > > What's your recommendation on the best way to capture debug?
> > >
> > > -----Original Message-----
> > > From: David Lang <david@lang.hm>
> > > Sent: Monday, June 10, 2019 3:50 PM
> > > To: Heaton, Tobias <tobias.heaton@unh.edu>
> > > Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> > > Subject: RE: [rsyslog] 8.1905.0 Logging Stops
> > >
> > > Caution - External Email;;;
> > >
> > > ********************************************************************************************************************************************
> > > by the way, looking at your evicted stats, you probably want to tweak your
> > > dynafilecachesize on some of your outputs (performance issue, probably not
> > > related to this issue)
> > >
> > > data was still being processed by outputs through the 15:00:17 window. It didn't
> > > stop until the 15:00:22 window.
> > >
> > > If you can get a debug log (send to Rainer) through the time it crashes, that
> > > would be helpful
> > >
> > > Also, can you post your config?
> > >
> > > David Lang
> > >
> > > On Mon, 10 Jun 2019, Heaton, Tobias wrote:
> > >
> > > > Date: Mon, 10 Jun 2019 19:37:31 +0000
> > > > From: "Heaton, Tobias" <tobias.heaton@unh.edu>
> > > > To: David Lang <david@lang.hm>
> > > > Cc: "Heaton, Tobias via rsyslog" <rsyslog@lists.adiscon.com>
> > > > Subject: RE: [rsyslog] 8.1905.0 Logging Stops
> > > >
> > > > Attached is a snippet of the impstats. Logs stopped writing at 15:00:01.
> > > >
> > > > -----Original Message-----
> > > > From: David Lang <david@lang.hm>
> > > > Sent: Monday, June 10, 2019 3:29 PM
> > > > To: Heaton, Tobias <tobias.heaton@unh.edu>
> > > > Cc: David Lang <david@lang.hm>; Heaton, Tobias via rsyslog <rsyslog@lists.adiscon.com>
> > > > Subject: RE: [rsyslog] 8.1905.0 Logging Stops
> > > >
> > > > Caution - External Email;;;
> > > >
> > > > ********************************************************************************************************************************************
> > > > On Mon, 10 Jun 2019, Heaton, Tobias wrote:
> > > >
> > > >> I configured impstats, updated rsyslog, restarted and awaited the failure. Tailing the busiest logs, behavior starts normal. Over the course of a few minutes, the logs seems to arrive in chunks...or at least appear to write that way to disk.
> > > >>
> > > >> That behavior continues with logs populating incrementally slower and slower until it just stops. The impstats don't show anything of consequence - the repeated stats are all normal and the same both prior to and after the logs stop writing.
> > > >>
> > > >> Rsyslog appears still running and impstats continue to populate again with no appreciable changes - very strange!
> > > >
> > > > after it stops, does impstats show new messages arriving (and queues building)?
> > > > or does it show outputs failing and messages being discarded?
> > > > or something else?
> > > >
> > > > a couple cycles of the pstats output after it's failed would be useful
> > > >
> > > > David Lang
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=lhTjg_QDGIeHOeCAcLX5bXSSgEoxAn4zMhoKMn39_Q0&s=AIKzntUgGqWkTqo_j2lfE7vckKRK0v9bS913ZhCug6k&e=
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=lhTjg_QDGIeHOeCAcLX5bXSSgEoxAn4zMhoKMn39_Q0&s=Wv4TJP-48wkRkAh4kLGr1DRn9AJcn69V5upqPFFcXkY&e=
> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Am Di., 11. Juni 2019 um 19:26 Uhr schrieb Rainer Gerhards via rsyslog
<rsyslog@lists.adiscon.com>:
>
> El mar., 11 jun. 2019 a las 19:23, Heaton, Tobias
> (<tobias.heaton@unh.edu>) escribió:
> >
> > Rainer,
> >
> > I traced my steps and allowed rsyslog to run for 45 minutes without seeing a failure.
> >
> > The debug puts considerable strain on the system, so much so that the log timestamps started drifting. I was seeing logs stamped at 13:18 EST at 13:31EST actual time.
> >
> > Furthermore the debug log is nearly 70GB, closing in on the available disk space.
> >
> > Is there anything else I can try to help troubleshoot this problem?
>
> Not really. Maybe the first 12 GiB actually contain info that get's us
> a bit closer. So if you could share that, it would be useful. We can
> fine-tune the debug output, but as we have not the slightest idea of
> what may be the root cause that's no option.
>

If the problem is reliably reproducible (which seems to be on affected
systems), maybe a git bisect would actually be the quickest way to
find the commit causing this.


--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
> If the problem is reliably reproducible (which seems to be on affected
> systems), maybe a git bisect would actually be the quickest way to
> find the commit causing this.

Good point!

@tobias: can you build from source?

If not, we could work with git bisect and custom packages, but that
looks very time-consuming.

Rainer
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

FYI: I guess this thread here is a duplicate of
https://github.com/rsyslog/rsyslog/issues/3686

Knowing the full config would be useful. Can be created via

$ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf

where -f should point to the main config file and the file pointed to
by -o contains the full config. Post the latter.

Rainer

El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
(<rgerhards@hq.adiscon.com>) escribió:
>
> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
> > If the problem is reliably reproducible (which seems to be on affected
> > systems), maybe a git bisect would actually be the quickest way to
> > find the commit causing this.
>
> Good point!
>
> @tobias: can you build from source?
>
> If not, we could work with git bisect and custom packages, but that
> looks very time-consuming.
>
> Rainer
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Hello Rainer,

Am 12.06.2019 um 12:48 schrieb Rainer Gerhards via rsyslog:
> FYI: I guess this thread here is a duplicate of
> https://github.com/rsyslog/rsyslog/issues/3686
>
> Knowing the full config would be useful. Can be created via
>
> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
>
> where -f should point to the main config file and the file pointed to
> by -o contains the full config. Post the latter.

[...]

I tried this on
- openSUSE 4.12.14-lp150.12.61-default
rsyslogd 8.33.1
- Ubuntu 4.18.0-20-generic
rsyslogd 8.32.0
- Eisfair-64 3.16.68-eisfair-64-VIRT
rsyslogd 8.1904.0

The result is always

/usr/sbin/rsyslogd: invalid option -- 'o'
usage: rsyslogd [options]
use "man rsyslogd" for details. To run rsyslog interactively, use
"rsyslogd -n"
to run it in debug mode use "rsyslogd -dn"
For further information see https://www.rsyslog.com/doc/

Looks like a bug?

Thanks.
Ansgar
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Looking at the aforementioned link, my config is considerably simpler. I did read about the suggested bump to the devel release, but given our syslog server is production I hesitate to continue testing and risk further loss of data.

> On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> FYI: I guess this thread here is a duplicate of
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAdnCB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
>
> Knowing the full config would be useful. Can be created via
>
> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
>
> where -f should point to the main config file and the file pointed to
> by -o contains the full config. Post the latter.
>
> Rainer
>
> El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
> (<rgerhards@hq.adiscon.com>) escribió:
>>
>>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
>>> If the problem is reliably reproducible (which seems to be on affected
>>> systems), maybe a git bisect would actually be the quickest way to
>>> find the commit causing this.
>>
>> Good point!
>>
>> @tobias: can you build from source?
>>
>> If not, we could work with git bisect and custom packages, but that
>> looks very time-consuming.
>>
>> Rainer
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Hi Ansgar,

glad to hear from you. What you have is simply a too-old version, the
-o switch is a recent addition of 8.1905.0 (2 weeks ago or so). The
version you have, I think, is around 9 month old.

Do you experience the same issue? That would be very interesting.

Rainer

El jue., 13 jun. 2019 a las 17:49, Ansgar Puester via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
> Hello Rainer,
>
> Am 12.06.2019 um 12:48 schrieb Rainer Gerhards via rsyslog:
> > FYI: I guess this thread here is a duplicate of
> > https://github.com/rsyslog/rsyslog/issues/3686
> >
> > Knowing the full config would be useful. Can be created via
> >
> > $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
> >
> > where -f should point to the main config file and the file pointed to
> > by -o contains the full config. Post the latter.
>
> [...]
>
> I tried this on
> - openSUSE 4.12.14-lp150.12.61-default
> rsyslogd 8.33.1
> - Ubuntu 4.18.0-20-generic
> rsyslogd 8.32.0
> - Eisfair-64 3.16.68-eisfair-64-VIRT
> rsyslogd 8.1904.0
>
> The result is always
>
> /usr/sbin/rsyslogd: invalid option -- 'o'
> usage: rsyslogd [options]
> use "man rsyslogd" for details. To run rsyslog interactively, use
> "rsyslogd -n"
> to run it in debug mode use "rsyslogd -dn"
> For further information see https://www.rsyslog.com/doc/
>
> Looks like a bug?
>
> Thanks.
> Ansgar
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

El jue., 13 jun. 2019 a las 19:04, Heaton, Tobias
(<tobias.heaton@unh.edu>) escribió:
>
> Looking at the aforementioned link, my config is considerably simpler. I did read about the suggested bump to the devel release, but given our syslog server is production I hesitate to continue testing and risk further loss of data.

The daily stable build is actually a stable build, just updated more
frequently. It's called "devel" for historical reasons. Have a look
here:

https://www.rsyslog.com/downloads/download-daily-build/

Rainer
>
> > On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
> >
> > Caution - External Email;;;
> >
> > ********************************************************************************************************************************************
> > FYI: I guess this thread here is a duplicate of
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAdnCB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
> >
> > Knowing the full config would be useful. Can be created via
> >
> > $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
> >
> > where -f should point to the main config file and the file pointed to
> > by -o contains the full config. Post the latter.
> >
> > Rainer
> >
> > El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
> > (<rgerhards@hq.adiscon.com>) escribió:
> >>
> >>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
> >>> If the problem is reliably reproducible (which seems to be on affected
> >>> systems), maybe a git bisect would actually be the quickest way to
> >>> find the commit causing this.
> >>
> >> Good point!
> >>
> >> @tobias: can you build from source?
> >>
> >> If not, we could work with git bisect and custom packages, but that
> >> looks very time-consuming.
> >>
> >> Rainer
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Hello Rainer,

you are absolute right. openSUSE and Ubuntu use quite
old versions and my version was also a little old.

New try:

- Eisfair-64 3.16.68-eisfair-64-VIRT
rsyslogd 8.1905.0

eis64test # rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
eis64test # ls -la /tmp/full_rsyslog_conf
-rw-r--r-- 1 root root 3485 Jun 13 19:39 /tmp/full_rsyslog_conf

It works! Sorry, my fault.

By the way:

Where can I find a command line option reference
on https://www.rsyslog.com/doc?

Examples on
https://www.rsyslog.com/doc/v8-stable/troubleshooting/troubleshoot.html#configuration-problems
require some spaces after option -f

Have a nice day.
Ansgar

Am 13.06.2019 um 19:08 schrieb Rainer Gerhards:
> Hi Ansgar,
>
> glad to hear from you. What you have is simply a too-old version, the
> -o switch is a recent addition of 8.1905.0 (2 weeks ago or so). The
> version you have, I think, is around 9 month old.
>
> Do you experience the same issue? That would be very interesting.
>
> Rainer

[...]
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

I've been following the thread at https://github.com/rsyslog/rsyslog/issues/3686.

I upgraded rsyslog on my RHEL6 host and after about 20 minutes, logging again stops.

Has the aforementioned patch not yet been released for RedHat/yum?

-----Original Message-----
From: Heaton, Tobias
Sent: Thursday, June 13, 2019 1:05 PM
To: Rainer Gerhards <rgerhards@hq.adiscon.com>
Cc: Michael Biebl <mbiebl@gmail.com>; rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

Looking at the aforementioned link, my config is considerably simpler. I did read about the suggested bump to the devel release, but given our syslog server is production I hesitate to continue testing and risk further loss of data.

> On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> FYI: I guess this thread here is a duplicate of
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAdnCB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
>
> Knowing the full config would be useful. Can be created via
>
> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
>
> where -f should point to the main config file and the file pointed to
> by -o contains the full config. Post the latter.
>
> Rainer
>
> El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
> (<rgerhards@hq.adiscon.com>) escribió:
>>
>>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
>>> If the problem is reliably reproducible (which seems to be on affected
>>> systems), maybe a git bisect would actually be the quickest way to
>>> find the commit causing this.
>>
>> Good point!
>>
>> @tobias: can you build from source?
>>
>> If not, we could work with git bisect and custom packages, but that
>> looks very time-consuming.
>>
>> Rainer
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Bumping this thread - still seeing 8.1905.0-2.el6 for RHEL6, which still causes rsyslog to fail.

Is there any update on when RedHat will have the updated code?

-----Original Message-----
From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
Sent: Thursday, June 27, 2019 9:03 AM
To: Rainer Gerhards <rgerhards@hq.adiscon.com>
Cc: Heaton, Tobias <tobias.heaton@unh.edu>; rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

I've been following the thread at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=Rhk0gSd_i0S7JJbXgQv8IxckTRylqq94K5ypZhg1tgc&e= .

I upgraded rsyslog on my RHEL6 host and after about 20 minutes, logging again stops.

Has the aforementioned patch not yet been released for RedHat/yum?

-----Original Message-----
From: Heaton, Tobias
Sent: Thursday, June 13, 2019 1:05 PM
To: Rainer Gerhards <rgerhards@hq.adiscon.com>
Cc: Michael Biebl <mbiebl@gmail.com>; rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

Looking at the aforementioned link, my config is considerably simpler. I did read about the suggested bump to the devel release, but given our syslog server is production I hesitate to continue testing and risk further loss of data.

> On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> FYI: I guess this thread here is a duplicate of
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAdnCB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
>
> Knowing the full config would be useful. Can be created via
>
> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
>
> where -f should point to the main config file and the file pointed to
> by -o contains the full config. Post the latter.
>
> Rainer
>
> El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
> (<rgerhards@hq.adiscon.com>) escribió:
>>
>>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
>>> If the problem is reliably reproducible (which seems to be on affected
>>> systems), maybe a git bisect would actually be the quickest way to
>>> find the commit causing this.
>>
>> Good point!
>>
>> @tobias: can you build from source?
>>
>> If not, we could work with git bisect and custom packages, but that
>> looks very time-consuming.
>>
>> Rainer
_______________________________________________
rsyslog mailing list
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=eCE_uw1iz27utOr31oN-0_3M4_4L8Kfe4WebMYOuy1o&e=
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=X2Jy3MZhZvUU2y8SREXpuYme4VAi3hEn2FDlEfdNt6E&e=
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Hi Rainer,
I'm also experiencing similar logging stopping issue on AWS Linux 2018.03 when would fix in https://github.com/rsyslog/rsyslog/issues/3686 be made available for RHEL06 ?
Thanks.
Mike
-----Original Message-----
From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
Sent: Tuesday, July 9, 2019 8:41 AM
To: rsyslog-users <rsyslog@lists.adiscon.com>; Rainer Gerhards <rgerhards@hq.adiscon.com>
Cc: Heaton, Tobias <tobias.heaton@unh.edu>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

EXTERNAL: Verify sender before opening attachments or links.

Bumping this thread - still seeing 8.1905.0-2.el6 for RHEL6, which still causes rsyslog to fail.

Is there any update on when RedHat will have the updated code?

-----Original Message-----
From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
Sent: Thursday, June 27, 2019 9:03 AM
To: Rainer Gerhards <rgerhards@hq.adiscon.com>
Cc: Heaton, Tobias <tobias.heaton@unh.edu>; rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

I've been following the thread at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=Rhk0gSd_i0S7JJbXgQv8IxckTRylqq94K5ypZhg1tgc&e= .

I upgraded rsyslog on my RHEL6 host and after about 20 minutes, logging again stops.

Has the aforementioned patch not yet been released for RedHat/yum?

-----Original Message-----
From: Heaton, Tobias
Sent: Thursday, June 13, 2019 1:05 PM
To: Rainer Gerhards <rgerhards@hq.adiscon.com>
Cc: Michael Biebl <mbiebl@gmail.com>; rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

Looking at the aforementioned link, my config is considerably simpler. I did read about the suggested bump to the devel release, but given our syslog server is production I hesitate to continue testing and risk further loss of data.

> On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
>
> Caution - External Email;;;
>
> **********************************************************************
> **********************************************************************
> FYI: I guess this thread here is a duplicate of
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslo
> g_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaL
> sgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAdn
> CB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
>
> Knowing the full config would be useful. Can be created via
>
> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
>
> where -f should point to the main config file and the file pointed to
> by -o contains the full config. Post the latter.
>
> Rainer
>
> El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
> (<rgerhards@hq.adiscon.com>) escribió:
>>
>>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
>>> If the problem is reliably reproducible (which seems to be on
>>> affected systems), maybe a git bisect would actually be the quickest
>>> way to find the commit causing this.
>>
>> Good point!
>>
>> @tobias: can you build from source?
>>
>> If not, we could work with git bisect and custom packages, but that
>> looks very time-consuming.
>>
>> Rainer
_______________________________________________
rsyslog mailing list
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=eCE_uw1iz27utOr31oN-0_3M4_4L8Kfe4WebMYOuy1o&e=
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=X2Jy3MZhZvUU2y8SREXpuYme4VAi3hEn2FDlEfdNt6E&e=
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Confidentiality Notice:: This email, including attachments, may include non-public, proprietary, confidential or legally privileged information. If you are not an intended recipient or an authorized agent of an intended recipient, you are hereby notified that any dissemination, distribution or copying of the information contained in or transmitted with this e-mail is unauthorized and strictly prohibited. If you have received this email in error, please notify the sender by replying to this message and permanently delete this e-mail, its attachments, and any copies of it immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

there are 8.1907 packages for centos in the adiscon repo, but it looks like the
OBS repo built 8.1905 in July instead of 8.1907

David Lang

On Tue, 9 Jul 2019, Li, Mike via rsyslog wrote:

> Date: Tue, 9 Jul 2019 13:35:29 +0000
> From: "Li, Mike via rsyslog" <rsyslog@lists.adiscon.com>
> To: rsyslog-users <rsyslog@lists.adiscon.com>,
> Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: "Li, Mike" <Mike.Li@finra.org>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Hi Rainer,
> I'm also experiencing similar logging stopping issue on AWS Linux 2018.03 when would fix in https://github.com/rsyslog/rsyslog/issues/3686 be made available for RHEL06 ?
> Thanks.
> Mike
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> Sent: Tuesday, July 9, 2019 8:41 AM
> To: rsyslog-users <rsyslog@lists.adiscon.com>; Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> EXTERNAL: Verify sender before opening attachments or links.
>
> Bumping this thread - still seeing 8.1905.0-2.el6 for RHEL6, which still causes rsyslog to fail.
>
> Is there any update on when RedHat will have the updated code?
>
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> Sent: Thursday, June 27, 2019 9:03 AM
> To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: Heaton, Tobias <tobias.heaton@unh.edu>; rsyslog-users <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> I've been following the thread at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=Rhk0gSd_i0S7JJbXgQv8IxckTRylqq94K5ypZhg1tgc&e= .
>
> I upgraded rsyslog on my RHEL6 host and after about 20 minutes, logging again stops.
>
> Has the aforementioned patch not yet been released for RedHat/yum?
>
> -----Original Message-----
> From: Heaton, Tobias
> Sent: Thursday, June 13, 2019 1:05 PM
> To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: Michael Biebl <mbiebl@gmail.com>; rsyslog-users <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Looking at the aforementioned link, my config is considerably simpler. I did read about the suggested bump to the devel release, but given our syslog server is production I hesitate to continue testing and risk further loss of data.
>
>> On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
>>
>> Caution - External Email;;;
>>
>> **********************************************************************
>> **********************************************************************
>> FYI: I guess this thread here is a duplicate of
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslo
>> g_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaL
>> sgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAdn
>> CB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
>>
>> Knowing the full config would be useful. Can be created via
>>
>> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
>>
>> where -f should point to the main config file and the file pointed to
>> by -o contains the full config. Post the latter.
>>
>> Rainer
>>
>> El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
>> (<rgerhards@hq.adiscon.com>) escribió:
>>>
>>>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
>>>> If the problem is reliably reproducible (which seems to be on
>>>> affected systems), maybe a git bisect would actually be the quickest
>>>> way to find the commit causing this.
>>>
>>> Good point!
>>>
>>> @tobias: can you build from source?
>>>
>>> If not, we could work with git bisect and custom packages, but that
>>> looks very time-consuming.
>>>
>>> Rainer
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=eCE_uw1iz27utOr31oN-0_3M4_4L8Kfe4WebMYOuy1o&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=X2Jy3MZhZvUU2y8SREXpuYme4VAi3hEn2FDlEfdNt6E&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
> Confidentiality Notice:: This email, including attachments, may include non-public, proprietary, confidential or legally privileged information. If you are not an intended recipient or an authorized agent of an intended recipient, you are hereby notified that any dissemination, distribution or copying of the information contained in or transmitted with this e-mail is unauthorized and strictly prohibited. If you have received this email in error, please notify the sender by replying to this message and permanently delete this e-mail, its attachments, and any copies of it immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

I updated my RHEL6 boxes to rsyslog 8.1907.0 at 7:27am this morning and at 8:07am, logging stopped just as before. Rolling back to 8.1904.0 and so far, so good.

-----Original Message-----
From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of David Lang via rsyslog
Sent: Tuesday, July 9, 2019 1:52 PM
To: Li, Mike via rsyslog <rsyslog@lists.adiscon.com>
Cc: David Lang <david@lang.hm>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

Caution - External Email;;;

********************************************************************************************************************************************
there are 8.1907 packages for centos in the adiscon repo, but it looks like the
OBS repo built 8.1905 in July instead of 8.1907

David Lang

On Tue, 9 Jul 2019, Li, Mike via rsyslog wrote:

> Date: Tue, 9 Jul 2019 13:35:29 +0000
> From: "Li, Mike via rsyslog" <rsyslog@lists.adiscon.com>
> To: rsyslog-users <rsyslog@lists.adiscon.com>,
> Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: "Li, Mike" <Mike.Li@finra.org>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Hi Rainer,
> I'm also experiencing similar logging stopping issue on AWS Linux 2018.03 when would fix in https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=lDNfB6yzWn9mIeLe13n59qDGZyKDBsH254e8pj476mM&e= be made available for RHEL06 ?
> Thanks.
> Mike
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> Sent: Tuesday, July 9, 2019 8:41 AM
> To: rsyslog-users <rsyslog@lists.adiscon.com>; Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> EXTERNAL: Verify sender before opening attachments or links.
>
> Bumping this thread - still seeing 8.1905.0-2.el6 for RHEL6, which still causes rsyslog to fail.
>
> Is there any update on when RedHat will have the updated code?
>
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> Sent: Thursday, June 27, 2019 9:03 AM
> To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: Heaton, Tobias <tobias.heaton@unh.edu>; rsyslog-users <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> I've been following the thread at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=Rhk0gSd_i0S7JJbXgQv8IxckTRylqq94K5ypZhg1tgc&e= .
>
> I upgraded rsyslog on my RHEL6 host and after about 20 minutes, logging again stops.
>
> Has the aforementioned patch not yet been released for RedHat/yum?
>
> -----Original Message-----
> From: Heaton, Tobias
> Sent: Thursday, June 13, 2019 1:05 PM
> To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: Michael Biebl <mbiebl@gmail.com>; rsyslog-users <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Looking at the aforementioned link, my config is considerably simpler. I did read about the suggested bump to the devel release, but given our syslog server is production I hesitate to continue testing and risk further loss of data.
>
>> On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
>>
>> Caution - External Email;;;
>>
>> **********************************************************************
>> **********************************************************************
>> FYI: I guess this thread here is a duplicate of
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslo
>> g_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaL
>> sgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAdn
>> CB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
>>
>> Knowing the full config would be useful. Can be created via
>>
>> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
>>
>> where -f should point to the main config file and the file pointed to
>> by -o contains the full config. Post the latter.
>>
>> Rainer
>>
>> El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
>> (<rgerhards@hq.adiscon.com>) escribió:
>>>
>>>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
>>>> If the problem is reliably reproducible (which seems to be on
>>>> affected systems), maybe a git bisect would actually be the quickest
>>>> way to find the commit causing this.
>>>
>>> Good point!
>>>
>>> @tobias: can you build from source?
>>>
>>> If not, we could work with git bisect and custom packages, but that
>>> looks very time-consuming.
>>>
>>> Rainer
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=eCE_uw1iz27utOr31oN-0_3M4_4L8Kfe4WebMYOuy1o&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=X2Jy3MZhZvUU2y8SREXpuYme4VAi3hEn2FDlEfdNt6E&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e= NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
> Confidentiality Notice:: This email, including attachments, may include non-public, proprietary, confidential or legally privileged information. If you are not an intended recipient or an authorized agent of an intended recipient, you are hereby notified that any dissemination, distribution or copying of the information contained in or transmitted with this e-mail is unauthorized and strictly prohibited. If you have received this email in error, please notify the sender by replying to this message and permanently delete this e-mail, its attachments, and any copies of it immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

FWIW, we have an Ubuntu test receiver configured to use the latest nightly development builds and more days than not it has the same issue. We have Nagios setup to regularly probe the ports on the box and it is usually the imrelp port probes that result in the condition.

Since the update to 8.1908.0~20190709150537-0adiscon1xenial we've not (yet) seen the problem reoccur. That said, we've not upgraded our main receivers to 8.1907 yet and haven't had a lot of time to closely monitor the specifics of the logging failures on the test node in recent weeks, so I understand that this feedback is of limited value.

-----Original Message-----
From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
Sent: Wednesday, July 10, 2019 7:35 AM
To: rsyslog-users <rsyslog@lists.adiscon.com>
Cc: Heaton, Tobias <tobias.heaton@unh.edu>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops
Importance: High

I updated my RHEL6 boxes to rsyslog 8.1907.0 at 7:27am this morning and at 8:07am, logging stopped just as before. Rolling back to 8.1904.0 and so far, so good.

-----Original Message-----
From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of David Lang via rsyslog
Sent: Tuesday, July 9, 2019 1:52 PM
To: Li, Mike via rsyslog <rsyslog@lists.adiscon.com>
Cc: David Lang <david@lang.hm>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

Caution - External Email;;;

********************************************************************************************************************************************
there are 8.1907 packages for centos in the adiscon repo, but it looks like the
OBS repo built 8.1905 in July instead of 8.1907

David Lang

On Tue, 9 Jul 2019, Li, Mike via rsyslog wrote:

> Date: Tue, 9 Jul 2019 13:35:29 +0000
> From: "Li, Mike via rsyslog" <rsyslog@lists.adiscon.com>
> To: rsyslog-users <rsyslog@lists.adiscon.com>,
> Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: "Li, Mike" <Mike.Li@finra.org>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Hi Rainer,
> I'm also experiencing similar logging stopping issue on AWS Linux 2018.03 when would fix in https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=lDNfB6yzWn9mIeLe13n59qDGZyKDBsH254e8pj476mM&e= be made available for RHEL06 ?
> Thanks.
> Mike
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> Sent: Tuesday, July 9, 2019 8:41 AM
> To: rsyslog-users <rsyslog@lists.adiscon.com>; Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> EXTERNAL: Verify sender before opening attachments or links.
>
> Bumping this thread - still seeing 8.1905.0-2.el6 for RHEL6, which still causes rsyslog to fail.
>
> Is there any update on when RedHat will have the updated code?
>
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> Sent: Thursday, June 27, 2019 9:03 AM
> To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: Heaton, Tobias <tobias.heaton@unh.edu>; rsyslog-users <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> I've been following the thread at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=Rhk0gSd_i0S7JJbXgQv8IxckTRylqq94K5ypZhg1tgc&e= .
>
> I upgraded rsyslog on my RHEL6 host and after about 20 minutes, logging again stops.
>
> Has the aforementioned patch not yet been released for RedHat/yum?
>
> -----Original Message-----
> From: Heaton, Tobias
> Sent: Thursday, June 13, 2019 1:05 PM
> To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: Michael Biebl <mbiebl@gmail.com>; rsyslog-users <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Looking at the aforementioned link, my config is considerably simpler. I did read about the suggested bump to the devel release, but given our syslog server is production I hesitate to continue testing and risk further loss of data.
>
>> On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
>>
>> Caution - External Email;;;
>>
>> **********************************************************************
>> **********************************************************************
>> FYI: I guess this thread here is a duplicate of
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslo
>> g_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaL
>> sgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAdn
>> CB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
>>
>> Knowing the full config would be useful. Can be created via
>>
>> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
>>
>> where -f should point to the main config file and the file pointed to
>> by -o contains the full config. Post the latter.
>>
>> Rainer
>>
>> El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
>> (<rgerhards@hq.adiscon.com>) escribió:
>>>
>>>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
>>>> If the problem is reliably reproducible (which seems to be on
>>>> affected systems), maybe a git bisect would actually be the quickest
>>>> way to find the commit causing this.
>>>
>>> Good point!
>>>
>>> @tobias: can you build from source?
>>>
>>> If not, we could work with git bisect and custom packages, but that
>>> looks very time-consuming.
>>>
>>> Rainer
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=eCE_uw1iz27utOr31oN-0_3M4_4L8Kfe4WebMYOuy1o&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=X2Jy3MZhZvUU2y8SREXpuYme4VAi3hEn2FDlEfdNt6E&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e= NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
> Confidentiality Notice:: This email, including attachments, may include non-public, proprietary, confidential or legally privileged information. If you are not an intended recipient or an authorized agent of an intended recipient, you are hereby notified that any dissemination, distribution or copying of the information contained in or transmitted with this e-mail is unauthorized and strictly prohibited. If you have received this email in error, please notify the sender by replying to this message and permanently delete this e-mail, its attachments, and any copies of it immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

I'm also seeing more of the segfault messages in 1907 and restarts of rsyslogd
[455973.614327] rs:main Q:Reg[31261]: segfault at 0 ip (null) sp 00007fc7d7ffea48 error 14 in rsyslogd[55c41427f000+a0000]
[459113.551029] rs:main Q:Reg[2193]: segfault at 0 ip (null) sp 00007fd4aade7a48 error 14 in rsyslogd[556d32a70000+a0000]
[501789.568177] rs:main Q:Reg[18096]: segfault at 0 ip (null) sp 00007f0e3bffea48 error 14 in rsyslogd[55c3fdcee000+a0000]
[502288.762233] rs:main Q:Reg[27491]: segfault at 0 ip (null) sp 00007fda17ffea48 error 14 in rsyslogd[560a70a46000+a0000]
[515018.787635] traps: rs:main Q:Reg[29669] general protection ip:7f9049967310 sp:7f902fbde8d8 error:0 in libc-2.17.so[7f904980e000+1c2000]
[516723.736981] traps: rs:main Q:Reg[660] general protection ip:7f0435de3310 sp:7f041bffe8d8 error:0 in libc-2.17.so[7f0435c8a000+1c2000]
[517408.745655] rs:main Q:Reg[2854]: segfault at a203065 ip 0000555ff5641781 sp 00007f324fbfda30 error 4 in rsyslogd[555ff55ee000+a0000]
[613874.114663] rs:main Q:Reg[25493]: segfault at 7f0d38021cb8 ip 00007f0d61d1c10e sp 00007f0d435e9a20 error 4 in libpthread-2.17.so[7f0d61d0c000+17000]

Reverted back to 1904 also.
Thanks.
Mike

-----Original Message-----
From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
Sent: Wednesday, July 10, 2019 8:35 AM
To: rsyslog-users <rsyslog@lists.adiscon.com>
Cc: Heaton, Tobias <tobias.heaton@unh.edu>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops
Importance: High

EXTERNAL: Verify sender before opening attachments or links.

I updated my RHEL6 boxes to rsyslog 8.1907.0 at 7:27am this morning and at 8:07am, logging stopped just as before. Rolling back to 8.1904.0 and so far, so good.

-----Original Message-----
From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of David Lang via rsyslog
Sent: Tuesday, July 9, 2019 1:52 PM
To: Li, Mike via rsyslog <rsyslog@lists.adiscon.com>
Cc: David Lang <david@lang.hm>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

Caution - External Email;;;

********************************************************************************************************************************************
there are 8.1907 packages for centos in the adiscon repo, but it looks like the OBS repo built 8.1905 in July instead of 8.1907

David Lang

On Tue, 9 Jul 2019, Li, Mike via rsyslog wrote:

> Date: Tue, 9 Jul 2019 13:35:29 +0000
> From: "Li, Mike via rsyslog" <rsyslog@lists.adiscon.com>
> To: rsyslog-users <rsyslog@lists.adiscon.com>,
> Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: "Li, Mike" <Mike.Li@finra.org>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Hi Rainer,
> I'm also experiencing similar logging stopping issue on AWS Linux 2018.03 when would fix in https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=lDNfB6yzWn9mIeLe13n59qDGZyKDBsH254e8pj476mM&e= be made available for RHEL06 ?
> Thanks.
> Mike
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton,
> Tobias via rsyslog
> Sent: Tuesday, July 9, 2019 8:41 AM
> To: rsyslog-users <rsyslog@lists.adiscon.com>; Rainer Gerhards
> <rgerhards@hq.adiscon.com>
> Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> EXTERNAL: Verify sender before opening attachments or links.
>
> Bumping this thread - still seeing 8.1905.0-2.el6 for RHEL6, which still causes rsyslog to fail.
>
> Is there any update on when RedHat will have the updated code?
>
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton,
> Tobias via rsyslog
> Sent: Thursday, June 27, 2019 9:03 AM
> To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: Heaton, Tobias <tobias.heaton@unh.edu>; rsyslog-users
> <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> I've been following the thread at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=Rhk0gSd_i0S7JJbXgQv8IxckTRylqq94K5ypZhg1tgc&e= .
>
> I upgraded rsyslog on my RHEL6 host and after about 20 minutes, logging again stops.
>
> Has the aforementioned patch not yet been released for RedHat/yum?
>
> -----Original Message-----
> From: Heaton, Tobias
> Sent: Thursday, June 13, 2019 1:05 PM
> To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: Michael Biebl <mbiebl@gmail.com>; rsyslog-users
> <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Looking at the aforementioned link, my config is considerably simpler. I did read about the suggested bump to the devel release, but given our syslog server is production I hesitate to continue testing and risk further loss of data.
>
>> On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
>>
>> Caution - External Email;;;
>>
>> *********************************************************************
>> *
>> *********************************************************************
>> *
>> FYI: I guess this thread here is a duplicate of
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsysl
>> o
>> g_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBa
>> L
>> sgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAd
>> n CB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
>>
>> Knowing the full config would be useful. Can be created via
>>
>> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
>>
>> where -f should point to the main config file and the file pointed to
>> by -o contains the full config. Post the latter.
>>
>> Rainer
>>
>> El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
>> (<rgerhards@hq.adiscon.com>) escribió:
>>>
>>>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
>>>> If the problem is reliably reproducible (which seems to be on
>>>> affected systems), maybe a git bisect would actually be the
>>>> quickest way to find the commit causing this.
>>>
>>> Good point!
>>>
>>> @tobias: can you build from source?
>>>
>>> If not, we could work with git bisect and custom packages, but that
>>> looks very time-consuming.
>>>
>>> Rainer
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_
> mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cw
> BaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUA
> no-jOYzTu8&s=eCE_uw1iz27utOr31oN-0_3M4_4L8Kfe4WebMYOuy1o&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow
> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerh
> ards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EU
> vddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=X2Jy3MZ
> hZvUU2y8SREXpuYme4VAi3hEn2FDlEfdNt6E&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_
> mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cw
> BaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982j
> k-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e= NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
> Confidentiality Notice:: This email, including attachments, may include non-public, proprietary, confidential or legally privileged information. If you are not an intended recipient or an authorized agent of an intended recipient, you are hereby notified that any dissemination, distribution or copying of the information contained in or transmitted with this e-mail is unauthorized and strictly prohibited. If you have received this email in error, please notify the sender by replying to this message and permanently delete this e-mail, its attachments, and any copies of it immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_
> mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cw
> BaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982j
> k-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow
> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerh
> ards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EU
> vddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmP
> xD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Does anyone have any updates on a stable update for RHEL6? Thank you.

-----Original Message-----
From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
Sent: Wednesday, July 10, 2019 8:35 AM
To: rsyslog-users <rsyslog@lists.adiscon.com>
Cc: Heaton, Tobias <tobias.heaton@unh.edu>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops
Importance: High

I updated my RHEL6 boxes to rsyslog 8.1907.0 at 7:27am this morning and at 8:07am, logging stopped just as before. Rolling back to 8.1904.0 and so far, so good.

-----Original Message-----
From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of David Lang via rsyslog
Sent: Tuesday, July 9, 2019 1:52 PM
To: Li, Mike via rsyslog <rsyslog@lists.adiscon.com>
Cc: David Lang <david@lang.hm>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

Caution - External Email;;;

********************************************************************************************************************************************
there are 8.1907 packages for centos in the adiscon repo, but it looks like the
OBS repo built 8.1905 in July instead of 8.1907

David Lang

On Tue, 9 Jul 2019, Li, Mike via rsyslog wrote:

> Date: Tue, 9 Jul 2019 13:35:29 +0000
> From: "Li, Mike via rsyslog" <rsyslog@lists.adiscon.com>
> To: rsyslog-users <rsyslog@lists.adiscon.com>,
> Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: "Li, Mike" <Mike.Li@finra.org>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Hi Rainer,
> I'm also experiencing similar logging stopping issue on AWS Linux 2018.03 when would fix in https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=lDNfB6yzWn9mIeLe13n59qDGZyKDBsH254e8pj476mM&e= be made available for RHEL06 ?
> Thanks.
> Mike
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> Sent: Tuesday, July 9, 2019 8:41 AM
> To: rsyslog-users <rsyslog@lists.adiscon.com>; Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> EXTERNAL: Verify sender before opening attachments or links.
>
> Bumping this thread - still seeing 8.1905.0-2.el6 for RHEL6, which still causes rsyslog to fail.
>
> Is there any update on when RedHat will have the updated code?
>
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> Sent: Thursday, June 27, 2019 9:03 AM
> To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: Heaton, Tobias <tobias.heaton@unh.edu>; rsyslog-users <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> I've been following the thread at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=Rhk0gSd_i0S7JJbXgQv8IxckTRylqq94K5ypZhg1tgc&e= .
>
> I upgraded rsyslog on my RHEL6 host and after about 20 minutes, logging again stops.
>
> Has the aforementioned patch not yet been released for RedHat/yum?
>
> -----Original Message-----
> From: Heaton, Tobias
> Sent: Thursday, June 13, 2019 1:05 PM
> To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Cc: Michael Biebl <mbiebl@gmail.com>; rsyslog-users <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Looking at the aforementioned link, my config is considerably simpler. I did read about the suggested bump to the devel release, but given our syslog server is production I hesitate to continue testing and risk further loss of data.
>
>> On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
>>
>> Caution - External Email;;;
>>
>> **********************************************************************
>> **********************************************************************
>> FYI: I guess this thread here is a duplicate of
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslo
>> g_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaL
>> sgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAdn
>> CB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
>>
>> Knowing the full config would be useful. Can be created via
>>
>> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
>>
>> where -f should point to the main config file and the file pointed to
>> by -o contains the full config. Post the latter.
>>
>> Rainer
>>
>> El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
>> (<rgerhards@hq.adiscon.com>) escribió:
>>>
>>>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
>>>> If the problem is reliably reproducible (which seems to be on
>>>> affected systems), maybe a git bisect would actually be the quickest
>>>> way to find the commit causing this.
>>>
>>> Good point!
>>>
>>> @tobias: can you build from source?
>>>
>>> If not, we could work with git bisect and custom packages, but that
>>> looks very time-consuming.
>>>
>>> Rainer
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=eCE_uw1iz27utOr31oN-0_3M4_4L8Kfe4WebMYOuy1o&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=X2Jy3MZhZvUU2y8SREXpuYme4VAi3hEn2FDlEfdNt6E&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e= NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
> Confidentiality Notice:: This email, including attachments, may include non-public, proprietary, confidential or legally privileged information. If you are not an intended recipient or an authorized agent of an intended recipient, you are hereby notified that any dissemination, distribution or copying of the information contained in or transmitted with this e-mail is unauthorized and strictly prohibited. If you have received this email in error, please notify the sender by replying to this message and permanently delete this e-mail, its attachments, and any copies of it immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=GU19wL68jlENObb7o1-k6kI7_7hYyYr9vrHYca1TxOI&s=L8jzjEaRlubwQt3WPTDcONsCrfsTtsbLcVcmHMjwWXs&e=
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=GU19wL68jlENObb7o1-k6kI7_7hYyYr9vrHYca1TxOI&s=GyLXb3i3V-7QPuCALwX5mogCa1es_Y5Z-2mgpdCuBDQ&e=
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Can you get me a backtrace of one of these aborts. Best with debug symbols.

Rainer

Sent from phone, thus brief.

Li, Mike via rsyslog <rsyslog@lists.adiscon.com> schrieb am Mi., 10. Juli
2019, 15:01:

> I'm also seeing more of the segfault messages in 1907 and restarts of
> rsyslogd
> [455973.614327] rs:main Q:Reg[31261]: segfault at 0 ip (null) sp
> 00007fc7d7ffea48 error 14 in rsyslogd[55c41427f000+a0000]
> [459113.551029] rs:main Q:Reg[2193]: segfault at 0 ip (null) sp
> 00007fd4aade7a48 error 14 in rsyslogd[556d32a70000+a0000]
> [501789.568177] rs:main Q:Reg[18096]: segfault at 0 ip (null) sp
> 00007f0e3bffea48 error 14 in rsyslogd[55c3fdcee000+a0000]
> [502288.762233] rs:main Q:Reg[27491]: segfault at 0 ip (null) sp
> 00007fda17ffea48 error 14 in rsyslogd[560a70a46000+a0000]
> [515018.787635] traps: rs:main Q:Reg[29669] general protection
> ip:7f9049967310 sp:7f902fbde8d8 error:0 in libc-2.17.so
> [7f904980e000+1c2000]
> [516723.736981] traps: rs:main Q:Reg[660] general protection
> ip:7f0435de3310 sp:7f041bffe8d8 error:0 in libc-2.17.so
> [7f0435c8a000+1c2000]
> [517408.745655] rs:main Q:Reg[2854]: segfault at a203065 ip
> 0000555ff5641781 sp 00007f324fbfda30 error 4 in rsyslogd[555ff55ee000+a0000]
> [613874.114663] rs:main Q:Reg[25493]: segfault at 7f0d38021cb8 ip
> 00007f0d61d1c10e sp 00007f0d435e9a20 error 4 in libpthread-2.17.so
> [7f0d61d0c000+17000]
>
> Reverted back to 1904 also.
> Thanks.
> Mike
>
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton,
> Tobias via rsyslog
> Sent: Wednesday, July 10, 2019 8:35 AM
> To: rsyslog-users <rsyslog@lists.adiscon.com>
> Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> Importance: High
>
> EXTERNAL: Verify sender before opening attachments or links.
>
> I updated my RHEL6 boxes to rsyslog 8.1907.0 at 7:27am this morning and at
> 8:07am, logging stopped just as before. Rolling back to 8.1904.0 and so
> far, so good.
>
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of David Lang
> via rsyslog
> Sent: Tuesday, July 9, 2019 1:52 PM
> To: Li, Mike via rsyslog <rsyslog@lists.adiscon.com>
> Cc: David Lang <david@lang.hm>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
>
> ********************************************************************************************************************************************
> there are 8.1907 packages for centos in the adiscon repo, but it looks
> like the OBS repo built 8.1905 in July instead of 8.1907
>
> David Lang
>
> On Tue, 9 Jul 2019, Li, Mike via rsyslog wrote:
>
> > Date: Tue, 9 Jul 2019 13:35:29 +0000
> > From: "Li, Mike via rsyslog" <rsyslog@lists.adiscon.com>
> > To: rsyslog-users <rsyslog@lists.adiscon.com>,
> > Rainer Gerhards <rgerhards@hq.adiscon.com>
> > Cc: "Li, Mike" <Mike.Li@finra.org>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > Hi Rainer,
> > I'm also experiencing similar logging stopping issue on AWS Linux
> 2018.03 when would fix in
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=lDNfB6yzWn9mIeLe13n59qDGZyKDBsH254e8pj476mM&e=
> be made available for RHEL06 ?
> > Thanks.
> > Mike
> > -----Original Message-----
> > From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton,
> > Tobias via rsyslog
> > Sent: Tuesday, July 9, 2019 8:41 AM
> > To: rsyslog-users <rsyslog@lists.adiscon.com>; Rainer Gerhards
> > <rgerhards@hq.adiscon.com>
> > Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > EXTERNAL: Verify sender before opening attachments or links.
> >
> > Bumping this thread - still seeing 8.1905.0-2.el6 for RHEL6, which still
> causes rsyslog to fail.
> >
> > Is there any update on when RedHat will have the updated code?
> >
> > -----Original Message-----
> > From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton,
> > Tobias via rsyslog
> > Sent: Thursday, June 27, 2019 9:03 AM
> > To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> > Cc: Heaton, Tobias <tobias.heaton@unh.edu>; rsyslog-users
> > <rsyslog@lists.adiscon.com>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > I've been following the thread at
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=Rhk0gSd_i0S7JJbXgQv8IxckTRylqq94K5ypZhg1tgc&e=
> .
> >
> > I upgraded rsyslog on my RHEL6 host and after about 20 minutes, logging
> again stops.
> >
> > Has the aforementioned patch not yet been released for RedHat/yum?
> >
> > -----Original Message-----
> > From: Heaton, Tobias
> > Sent: Thursday, June 13, 2019 1:05 PM
> > To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> > Cc: Michael Biebl <mbiebl@gmail.com>; rsyslog-users
> > <rsyslog@lists.adiscon.com>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > Looking at the aforementioned link, my config is considerably simpler. I
> did read about the suggested bump to the devel release, but given our
> syslog server is production I hesitate to continue testing and risk further
> loss of data.
> >
> >> On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com>
> wrote:
> >>
> >> Caution - External Email;;;
> >>
> >> *********************************************************************
> >> *
> >> *********************************************************************
> >> *
> >> FYI: I guess this thread here is a duplicate of
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsysl
> >> o
> >> g_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBa
> >> L
> >> sgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAd
> >> n CB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
> >>
> >> Knowing the full config would be useful. Can be created via
> >>
> >> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
> >>
> >> where -f should point to the main config file and the file pointed to
> >> by -o contains the full config. Post the latter.
> >>
> >> Rainer
> >>
> >> El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
> >> (<rgerhards@hq.adiscon.com>) escribió:
> >>>
> >>>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>)
> escribió:
> >>>> If the problem is reliably reproducible (which seems to be on
> >>>> affected systems), maybe a git bisect would actually be the
> >>>> quickest way to find the commit causing this.
> >>>
> >>> Good point!
> >>>
> >>> @tobias: can you build from source?
> >>>
> >>> If not, we could work with git bisect and custom packages, but that
> >>> looks very time-consuming.
> >>>
> >>> Rainer
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_
> > mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cw
> > BaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUA
> > no-jOYzTu8&s=eCE_uw1iz27utOr31oN-0_3M4_4L8Kfe4WebMYOuy1o&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerh
> > ards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EU
> > vddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=X2Jy3MZ
> > hZvUU2y8SREXpuYme4VAi3hEn2FDlEfdNt6E&e=
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_
> > mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cw
> > BaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982j
> > k-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow
> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
> >
> > Confidentiality Notice:: This email, including attachments, may include
> non-public, proprietary, confidential or legally privileged information.
> If you are not an intended recipient or an authorized agent of an intended
> recipient, you are hereby notified that any dissemination, distribution or
> copying of the information contained in or transmitted with this e-mail is
> unauthorized and strictly prohibited. If you have received this email in
> error, please notify the sender by replying to this message and permanently
> delete this e-mail, its attachments, and any copies of it immediately. You
> should not retain, copy or use this e-mail or any attachment for any
> purpose, nor disclose all or any part of the contents to any other person.
> Thank you.
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_
> > mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cw
> > BaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982j
> > k-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerh
> > ards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EU
> > vddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmP
> > xD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow
> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL:
> This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
> THAT.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

It looks like the OBS packages were not updated. I did this myself
yesterday. So 8.1907.0 is now available via OBS.

Rainer

El mar., 16 jul. 2019 a las 18:58, Heaton, Tobias
(<tobias.heaton@unh.edu>) escribió:
>
> Does anyone have any updates on a stable update for RHEL6? Thank you.
>
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> Sent: Wednesday, July 10, 2019 8:35 AM
> To: rsyslog-users <rsyslog@lists.adiscon.com>
> Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> Importance: High
>
> I updated my RHEL6 boxes to rsyslog 8.1907.0 at 7:27am this morning and at 8:07am, logging stopped just as before. Rolling back to 8.1904.0 and so far, so good.
>
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of David Lang via rsyslog
> Sent: Tuesday, July 9, 2019 1:52 PM
> To: Li, Mike via rsyslog <rsyslog@lists.adiscon.com>
> Cc: David Lang <david@lang.hm>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> there are 8.1907 packages for centos in the adiscon repo, but it looks like the
> OBS repo built 8.1905 in July instead of 8.1907
>
> David Lang
>
> On Tue, 9 Jul 2019, Li, Mike via rsyslog wrote:
>
> > Date: Tue, 9 Jul 2019 13:35:29 +0000
> > From: "Li, Mike via rsyslog" <rsyslog@lists.adiscon.com>
> > To: rsyslog-users <rsyslog@lists.adiscon.com>,
> > Rainer Gerhards <rgerhards@hq.adiscon.com>
> > Cc: "Li, Mike" <Mike.Li@finra.org>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > Hi Rainer,
> > I'm also experiencing similar logging stopping issue on AWS Linux 2018.03 when would fix in https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=lDNfB6yzWn9mIeLe13n59qDGZyKDBsH254e8pj476mM&e= be made available for RHEL06 ?
> > Thanks.
> > Mike
> > -----Original Message-----
> > From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> > Sent: Tuesday, July 9, 2019 8:41 AM
> > To: rsyslog-users <rsyslog@lists.adiscon.com>; Rainer Gerhards <rgerhards@hq.adiscon.com>
> > Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > EXTERNAL: Verify sender before opening attachments or links.
> >
> > Bumping this thread - still seeing 8.1905.0-2.el6 for RHEL6, which still causes rsyslog to fail.
> >
> > Is there any update on when RedHat will have the updated code?
> >
> > -----Original Message-----
> > From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> > Sent: Thursday, June 27, 2019 9:03 AM
> > To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> > Cc: Heaton, Tobias <tobias.heaton@unh.edu>; rsyslog-users <rsyslog@lists.adiscon.com>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > I've been following the thread at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=Rhk0gSd_i0S7JJbXgQv8IxckTRylqq94K5ypZhg1tgc&e= .
> >
> > I upgraded rsyslog on my RHEL6 host and after about 20 minutes, logging again stops.
> >
> > Has the aforementioned patch not yet been released for RedHat/yum?
> >
> > -----Original Message-----
> > From: Heaton, Tobias
> > Sent: Thursday, June 13, 2019 1:05 PM
> > To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> > Cc: Michael Biebl <mbiebl@gmail.com>; rsyslog-users <rsyslog@lists.adiscon.com>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > Looking at the aforementioned link, my config is considerably simpler. I did read about the suggested bump to the devel release, but given our syslog server is production I hesitate to continue testing and risk further loss of data.
> >
> >> On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
> >>
> >> Caution - External Email;;;
> >>
> >> **********************************************************************
> >> **********************************************************************
> >> FYI: I guess this thread here is a duplicate of
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslo
> >> g_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaL
> >> sgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAdn
> >> CB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
> >>
> >> Knowing the full config would be useful. Can be created via
> >>
> >> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
> >>
> >> where -f should point to the main config file and the file pointed to
> >> by -o contains the full config. Post the latter.
> >>
> >> Rainer
> >>
> >> El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
> >> (<rgerhards@hq.adiscon.com>) escribió:
> >>>
> >>>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
> >>>> If the problem is reliably reproducible (which seems to be on
> >>>> affected systems), maybe a git bisect would actually be the quickest
> >>>> way to find the commit causing this.
> >>>
> >>> Good point!
> >>>
> >>> @tobias: can you build from source?
> >>>
> >>> If not, we could work with git bisect and custom packages, but that
> >>> looks very time-consuming.
> >>>
> >>> Rainer
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=eCE_uw1iz27utOr31oN-0_3M4_4L8Kfe4WebMYOuy1o&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=X2Jy3MZhZvUU2y8SREXpuYme4VAi3hEn2FDlEfdNt6E&e=
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e= NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> >
> > Confidentiality Notice:: This email, including attachments, may include non-public, proprietary, confidential or legally privileged information. If you are not an intended recipient or an authorized agent of an intended recipient, you are hereby notified that any dissemination, distribution or copying of the information contained in or transmitted with this e-mail is unauthorized and strictly prohibited. If you have received this email in error, please notify the sender by replying to this message and permanently delete this e-mail, its attachments, and any copies of it immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=GU19wL68jlENObb7o1-k6kI7_7hYyYr9vrHYca1TxOI&s=L8jzjEaRlubwQt3WPTDcONsCrfsTtsbLcVcmHMjwWXs&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=GU19wL68jlENObb7o1-k6kI7_7hYyYr9vrHYca1TxOI&s=GyLXb3i3V-7QPuCALwX5mogCa1es_Y5Z-2mgpdCuBDQ&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Thank you Rainer.

I updated to 8.1907.0-1 at 7:26am EST today and at 8:50am, logging once again stopped. I reverted back to 8.1904.

-----Original Message-----
From: Rainer Gerhards <rgerhards@hq.adiscon.com>
Sent: Friday, July 19, 2019 2:44 AM
To: Heaton, Tobias <tobias.heaton@unh.edu>
Cc: rsyslog-users <rsyslog@lists.adiscon.com>; David Lang <david@lang.hm>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

Caution - External Email;;;

********************************************************************************************************************************************
It looks like the OBS packages were not updated. I did this myself
yesterday. So 8.1907.0 is now available via OBS.

Rainer

El mar., 16 jul. 2019 a las 18:58, Heaton, Tobias
(<tobias.heaton@unh.edu>) escribió:
>
> Does anyone have any updates on a stable update for RHEL6? Thank you.
>
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> Sent: Wednesday, July 10, 2019 8:35 AM
> To: rsyslog-users <rsyslog@lists.adiscon.com>
> Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> Importance: High
>
> I updated my RHEL6 boxes to rsyslog 8.1907.0 at 7:27am this morning and at 8:07am, logging stopped just as before. Rolling back to 8.1904.0 and so far, so good.
>
> -----Original Message-----
> From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of David Lang via rsyslog
> Sent: Tuesday, July 9, 2019 1:52 PM
> To: Li, Mike via rsyslog <rsyslog@lists.adiscon.com>
> Cc: David Lang <david@lang.hm>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> there are 8.1907 packages for centos in the adiscon repo, but it looks like the
> OBS repo built 8.1905 in July instead of 8.1907
>
> David Lang
>
> On Tue, 9 Jul 2019, Li, Mike via rsyslog wrote:
>
> > Date: Tue, 9 Jul 2019 13:35:29 +0000
> > From: "Li, Mike via rsyslog" <rsyslog@lists.adiscon.com>
> > To: rsyslog-users <rsyslog@lists.adiscon.com>,
> > Rainer Gerhards <rgerhards@hq.adiscon.com>
> > Cc: "Li, Mike" <Mike.Li@finra.org>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > Hi Rainer,
> > I'm also experiencing similar logging stopping issue on AWS Linux 2018.03 when would fix in https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=lDNfB6yzWn9mIeLe13n59qDGZyKDBsH254e8pj476mM&e= be made available for RHEL06 ?
> > Thanks.
> > Mike
> > -----Original Message-----
> > From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> > Sent: Tuesday, July 9, 2019 8:41 AM
> > To: rsyslog-users <rsyslog@lists.adiscon.com>; Rainer Gerhards <rgerhards@hq.adiscon.com>
> > Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > EXTERNAL: Verify sender before opening attachments or links.
> >
> > Bumping this thread - still seeing 8.1905.0-2.el6 for RHEL6, which still causes rsyslog to fail.
> >
> > Is there any update on when RedHat will have the updated code?
> >
> > -----Original Message-----
> > From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> > Sent: Thursday, June 27, 2019 9:03 AM
> > To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> > Cc: Heaton, Tobias <tobias.heaton@unh.edu>; rsyslog-users <rsyslog@lists.adiscon.com>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > I've been following the thread at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=Rhk0gSd_i0S7JJbXgQv8IxckTRylqq94K5ypZhg1tgc&e= .
> >
> > I upgraded rsyslog on my RHEL6 host and after about 20 minutes, logging again stops.
> >
> > Has the aforementioned patch not yet been released for RedHat/yum?
> >
> > -----Original Message-----
> > From: Heaton, Tobias
> > Sent: Thursday, June 13, 2019 1:05 PM
> > To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> > Cc: Michael Biebl <mbiebl@gmail.com>; rsyslog-users <rsyslog@lists.adiscon.com>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > Looking at the aforementioned link, my config is considerably simpler. I did read about the suggested bump to the devel release, but given our syslog server is production I hesitate to continue testing and risk further loss of data.
> >
> >> On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
> >>
> >> Caution - External Email;;;
> >>
> >> **********************************************************************
> >> **********************************************************************
> >> FYI: I guess this thread here is a duplicate of
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslo
> >> g_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaL
> >> sgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAdn
> >> CB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
> >>
> >> Knowing the full config would be useful. Can be created via
> >>
> >> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
> >>
> >> where -f should point to the main config file and the file pointed to
> >> by -o contains the full config. Post the latter.
> >>
> >> Rainer
> >>
> >> El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
> >> (<rgerhards@hq.adiscon.com>) escribió:
> >>>
> >>>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
> >>>> If the problem is reliably reproducible (which seems to be on
> >>>> affected systems), maybe a git bisect would actually be the quickest
> >>>> way to find the commit causing this.
> >>>
> >>> Good point!
> >>>
> >>> @tobias: can you build from source?
> >>>
> >>> If not, we could work with git bisect and custom packages, but that
> >>> looks very time-consuming.
> >>>
> >>> Rainer
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=eCE_uw1iz27utOr31oN-0_3M4_4L8Kfe4WebMYOuy1o&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=X2Jy3MZhZvUU2y8SREXpuYme4VAi3hEn2FDlEfdNt6E&e=
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e= NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> >
> > Confidentiality Notice:: This email, including attachments, may include non-public, proprietary, confidential or legally privileged information. If you are not an intended recipient or an authorized agent of an intended recipient, you are hereby notified that any dissemination, distribution or copying of the information contained in or transmitted with this e-mail is unauthorized and strictly prohibited. If you have received this email in error, please notify the sender by replying to this message and permanently delete this e-mail, its attachments, and any copies of it immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=GU19wL68jlENObb7o1-k6kI7_7hYyYr9vrHYca1TxOI&s=L8jzjEaRlubwQt3WPTDcONsCrfsTtsbLcVcmHMjwWXs&e=
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=GU19wL68jlENObb7o1-k6kI7_7hYyYr9vrHYca1TxOI&s=GyLXb3i3V-7QPuCALwX5mogCa1es_Y5Z-2mgpdCuBDQ&e=
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

El vie., 19 jul. 2019 a las 14:51, Heaton, Tobias
(<tobias.heaton@unh.edu>) escribió:
>
> Thank you Rainer.
>
> I updated to 8.1907.0-1 at 7:26am EST today and at 8:50am, logging once again stopped. I reverted back to 8.1904.

Can you help debug this? With me having no idea what is going on and
not the slightest chance to reproduce it, a fix is extremely unlikely.

For starters, can you describe the situation where the hang occurs? Is
it immediately? Or during HUP? Random? Does it really stop (a.k.a
properly terminate)? Or does it hang and you need to abort it?

If you can reproduce, a debug log would possibly be very useful.
Having more information on the issue enables us to set a plan to
tackle it. For that, a github issue tracker would also be good.

Thx,
Rainer
>
> -----Original Message-----
> From: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Sent: Friday, July 19, 2019 2:44 AM
> To: Heaton, Tobias <tobias.heaton@unh.edu>
> Cc: rsyslog-users <rsyslog@lists.adiscon.com>; David Lang <david@lang.hm>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> It looks like the OBS packages were not updated. I did this myself
> yesterday. So 8.1907.0 is now available via OBS.
>
> Rainer
>
> El mar., 16 jul. 2019 a las 18:58, Heaton, Tobias
> (<tobias.heaton@unh.edu>) escribió:
> >
> > Does anyone have any updates on a stable update for RHEL6? Thank you.
> >
> > -----Original Message-----
> > From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> > Sent: Wednesday, July 10, 2019 8:35 AM
> > To: rsyslog-users <rsyslog@lists.adiscon.com>
> > Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> > Importance: High
> >
> > I updated my RHEL6 boxes to rsyslog 8.1907.0 at 7:27am this morning and at 8:07am, logging stopped just as before. Rolling back to 8.1904.0 and so far, so good.
> >
> > -----Original Message-----
> > From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of David Lang via rsyslog
> > Sent: Tuesday, July 9, 2019 1:52 PM
> > To: Li, Mike via rsyslog <rsyslog@lists.adiscon.com>
> > Cc: David Lang <david@lang.hm>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > Caution - External Email;;;
> >
> > ********************************************************************************************************************************************
> > there are 8.1907 packages for centos in the adiscon repo, but it looks like the
> > OBS repo built 8.1905 in July instead of 8.1907
> >
> > David Lang
> >
> > On Tue, 9 Jul 2019, Li, Mike via rsyslog wrote:
> >
> > > Date: Tue, 9 Jul 2019 13:35:29 +0000
> > > From: "Li, Mike via rsyslog" <rsyslog@lists.adiscon.com>
> > > To: rsyslog-users <rsyslog@lists.adiscon.com>,
> > > Rainer Gerhards <rgerhards@hq.adiscon.com>
> > > Cc: "Li, Mike" <Mike.Li@finra.org>
> > > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> > >
> > > Hi Rainer,
> > > I'm also experiencing similar logging stopping issue on AWS Linux 2018.03 when would fix in https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=lDNfB6yzWn9mIeLe13n59qDGZyKDBsH254e8pj476mM&e= be made available for RHEL06 ?
> > > Thanks.
> > > Mike
> > > -----Original Message-----
> > > From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> > > Sent: Tuesday, July 9, 2019 8:41 AM
> > > To: rsyslog-users <rsyslog@lists.adiscon.com>; Rainer Gerhards <rgerhards@hq.adiscon.com>
> > > Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> > > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> > >
> > > EXTERNAL: Verify sender before opening attachments or links.
> > >
> > > Bumping this thread - still seeing 8.1905.0-2.el6 for RHEL6, which still causes rsyslog to fail.
> > >
> > > Is there any update on when RedHat will have the updated code?
> > >
> > > -----Original Message-----
> > > From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> > > Sent: Thursday, June 27, 2019 9:03 AM
> > > To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> > > Cc: Heaton, Tobias <tobias.heaton@unh.edu>; rsyslog-users <rsyslog@lists.adiscon.com>
> > > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> > >
> > > I've been following the thread at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=Rhk0gSd_i0S7JJbXgQv8IxckTRylqq94K5ypZhg1tgc&e= .
> > >
> > > I upgraded rsyslog on my RHEL6 host and after about 20 minutes, logging again stops.
> > >
> > > Has the aforementioned patch not yet been released for RedHat/yum?
> > >
> > > -----Original Message-----
> > > From: Heaton, Tobias
> > > Sent: Thursday, June 13, 2019 1:05 PM
> > > To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> > > Cc: Michael Biebl <mbiebl@gmail.com>; rsyslog-users <rsyslog@lists.adiscon.com>
> > > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> > >
> > > Looking at the aforementioned link, my config is considerably simpler. I did read about the suggested bump to the devel release, but given our syslog server is production I hesitate to continue testing and risk further loss of data.
> > >
> > >> On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
> > >>
> > >> Caution - External Email;;;
> > >>
> > >> **********************************************************************
> > >> **********************************************************************
> > >> FYI: I guess this thread here is a duplicate of
> > >> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslo
> > >> g_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaL
> > >> sgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAdn
> > >> CB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
> > >>
> > >> Knowing the full config would be useful. Can be created via
> > >>
> > >> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
> > >>
> > >> where -f should point to the main config file and the file pointed to
> > >> by -o contains the full config. Post the latter.
> > >>
> > >> Rainer
> > >>
> > >> El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
> > >> (<rgerhards@hq.adiscon.com>) escribió:
> > >>>
> > >>>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
> > >>>> If the problem is reliably reproducible (which seems to be on
> > >>>> affected systems), maybe a git bisect would actually be the quickest
> > >>>> way to find the commit causing this.
> > >>>
> > >>> Good point!
> > >>>
> > >>> @tobias: can you build from source?
> > >>>
> > >>> If not, we could work with git bisect and custom packages, but that
> > >>> looks very time-consuming.
> > >>>
> > >>> Rainer
> > > _______________________________________________
> > > rsyslog mailing list
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=eCE_uw1iz27utOr31oN-0_3M4_4L8Kfe4WebMYOuy1o&e=
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=X2Jy3MZhZvUU2y8SREXpuYme4VAi3hEn2FDlEfdNt6E&e=
> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> > > _______________________________________________
> > > rsyslog mailing list
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e= NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> > >
> > > Confidentiality Notice:: This email, including attachments, may include non-public, proprietary, confidential or legally privileged information. If you are not an intended recipient or an authorized agent of an intended recipient, you are hereby notified that any dissemination, distribution or copying of the information contained in or transmitted with this e-mail is unauthorized and strictly prohibited. If you have received this email in error, please notify the sender by replying to this message and permanently delete this e-mail, its attachments, and any copies of it immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.
> > > _______________________________________________
> > > rsyslog mailing list
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=GU19wL68jlENObb7o1-k6kI7_7hYyYr9vrHYca1TxOI&s=L8jzjEaRlubwQt3WPTDcONsCrfsTtsbLcVcmHMjwWXs&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=GU19wL68jlENObb7o1-k6kI7_7hYyYr9vrHYca1TxOI&s=GyLXb3i3V-7QPuCALwX5mogCa1es_Y5Z-2mgpdCuBDQ&e=
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Rsyslog does not stop per say - rather logs simply stop populating. From the time rsyslog starts, it usually takes anywhere from 30-90 minutes to fail. A restart of services restore functionality temporarily.

A debug log is difficult because I cannot predict when rsyslog will fail. This is also one of two production rsyslog receivers I'm running, neither of which I can afford to have offline for more than a few seconds. I can say this behavior is reproducible on both systems.

-----Original Message-----
From: Rainer Gerhards <rgerhards@hq.adiscon.com>
Sent: Friday, July 19, 2019 9:25 AM
To: Heaton, Tobias <tobias.heaton@unh.edu>
Cc: rsyslog-users <rsyslog@lists.adiscon.com>; David Lang <david@lang.hm>
Subject: Re: [rsyslog] 8.1905.0 Logging Stops

Caution - External Email;;;

********************************************************************************************************************************************
El vie., 19 jul. 2019 a las 14:51, Heaton, Tobias
(<tobias.heaton@unh.edu>) escribió:
>
> Thank you Rainer.
>
> I updated to 8.1907.0-1 at 7:26am EST today and at 8:50am, logging once again stopped. I reverted back to 8.1904.

Can you help debug this? With me having no idea what is going on and
not the slightest chance to reproduce it, a fix is extremely unlikely.

For starters, can you describe the situation where the hang occurs? Is
it immediately? Or during HUP? Random? Does it really stop (a.k.a
properly terminate)? Or does it hang and you need to abort it?

If you can reproduce, a debug log would possibly be very useful.
Having more information on the issue enables us to set a plan to
tackle it. For that, a github issue tracker would also be good.

Thx,
Rainer
>
> -----Original Message-----
> From: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Sent: Friday, July 19, 2019 2:44 AM
> To: Heaton, Tobias <tobias.heaton@unh.edu>
> Cc: rsyslog-users <rsyslog@lists.adiscon.com>; David Lang <david@lang.hm>
> Subject: Re: [rsyslog] 8.1905.0 Logging Stops
>
> Caution - External Email;;;
>
> ********************************************************************************************************************************************
> It looks like the OBS packages were not updated. I did this myself
> yesterday. So 8.1907.0 is now available via OBS.
>
> Rainer
>
> El mar., 16 jul. 2019 a las 18:58, Heaton, Tobias
> (<tobias.heaton@unh.edu>) escribió:
> >
> > Does anyone have any updates on a stable update for RHEL6? Thank you.
> >
> > -----Original Message-----
> > From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> > Sent: Wednesday, July 10, 2019 8:35 AM
> > To: rsyslog-users <rsyslog@lists.adiscon.com>
> > Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> > Importance: High
> >
> > I updated my RHEL6 boxes to rsyslog 8.1907.0 at 7:27am this morning and at 8:07am, logging stopped just as before. Rolling back to 8.1904.0 and so far, so good.
> >
> > -----Original Message-----
> > From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of David Lang via rsyslog
> > Sent: Tuesday, July 9, 2019 1:52 PM
> > To: Li, Mike via rsyslog <rsyslog@lists.adiscon.com>
> > Cc: David Lang <david@lang.hm>
> > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> >
> > Caution - External Email;;;
> >
> > ********************************************************************************************************************************************
> > there are 8.1907 packages for centos in the adiscon repo, but it looks like the
> > OBS repo built 8.1905 in July instead of 8.1907
> >
> > David Lang
> >
> > On Tue, 9 Jul 2019, Li, Mike via rsyslog wrote:
> >
> > > Date: Tue, 9 Jul 2019 13:35:29 +0000
> > > From: "Li, Mike via rsyslog" <rsyslog@lists.adiscon.com>
> > > To: rsyslog-users <rsyslog@lists.adiscon.com>,
> > > Rainer Gerhards <rgerhards@hq.adiscon.com>
> > > Cc: "Li, Mike" <Mike.Li@finra.org>
> > > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> > >
> > > Hi Rainer,
> > > I'm also experiencing similar logging stopping issue on AWS Linux 2018.03 when would fix in https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=lDNfB6yzWn9mIeLe13n59qDGZyKDBsH254e8pj476mM&e= be made available for RHEL06 ?
> > > Thanks.
> > > Mike
> > > -----Original Message-----
> > > From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> > > Sent: Tuesday, July 9, 2019 8:41 AM
> > > To: rsyslog-users <rsyslog@lists.adiscon.com>; Rainer Gerhards <rgerhards@hq.adiscon.com>
> > > Cc: Heaton, Tobias <tobias.heaton@unh.edu>
> > > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> > >
> > > EXTERNAL: Verify sender before opening attachments or links.
> > >
> > > Bumping this thread - still seeing 8.1905.0-2.el6 for RHEL6, which still causes rsyslog to fail.
> > >
> > > Is there any update on when RedHat will have the updated code?
> > >
> > > -----Original Message-----
> > > From: rsyslog <rsyslog-bounces@lists.adiscon.com> On Behalf Of Heaton, Tobias via rsyslog
> > > Sent: Thursday, June 27, 2019 9:03 AM
> > > To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> > > Cc: Heaton, Tobias <tobias.heaton@unh.edu>; rsyslog-users <rsyslog@lists.adiscon.com>
> > > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> > >
> > > I've been following the thread at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslog_rsyslog_issues_3686&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=Rhk0gSd_i0S7JJbXgQv8IxckTRylqq94K5ypZhg1tgc&e= .
> > >
> > > I upgraded rsyslog on my RHEL6 host and after about 20 minutes, logging again stops.
> > >
> > > Has the aforementioned patch not yet been released for RedHat/yum?
> > >
> > > -----Original Message-----
> > > From: Heaton, Tobias
> > > Sent: Thursday, June 13, 2019 1:05 PM
> > > To: Rainer Gerhards <rgerhards@hq.adiscon.com>
> > > Cc: Michael Biebl <mbiebl@gmail.com>; rsyslog-users <rsyslog@lists.adiscon.com>
> > > Subject: Re: [rsyslog] 8.1905.0 Logging Stops
> > >
> > > Looking at the aforementioned link, my config is considerably simpler. I did read about the suggested bump to the devel release, but given our syslog server is production I hesitate to continue testing and risk further loss of data.
> > >
> > >> On Jun 12, 2019, at 6:49 AM, Rainer Gerhards <rgerhards@hq.adiscon.com> wrote:
> > >>
> > >> Caution - External Email;;;
> > >>
> > >> **********************************************************************
> > >> **********************************************************************
> > >> FYI: I guess this thread here is a duplicate of
> > >> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rsyslo
> > >> g_rsyslog_issues_3686&d=DwIFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaL
> > >> sgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=S3Rb53yyV69wLOmt0FRCSHHGXxB8GD7vZAdn
> > >> CB7qtYo&s=kstKSKXQNf__Rvz2xQVW5tWJfOyx-yNejVXfraa7SxY&e=
> > >>
> > >> Knowing the full config would be useful. Can be created via
> > >>
> > >> $ rsyslogd -f /etc/rsyslog.conf -o /tmp/full_rsyslog_conf
> > >>
> > >> where -f should point to the main config file and the file pointed to
> > >> by -o contains the full config. Post the latter.
> > >>
> > >> Rainer
> > >>
> > >> El mié., 12 jun. 2019 a las 10:48, Rainer Gerhards
> > >> (<rgerhards@hq.adiscon.com>) escribió:
> > >>>
> > >>>> El mar., 11 jun. 2019 a las 20:16, Michael Biebl (<mbiebl@gmail.com>) escribió:
> > >>>> If the problem is reliably reproducible (which seems to be on
> > >>>> affected systems), maybe a git bisect would actually be the quickest
> > >>>> way to find the commit causing this.
> > >>>
> > >>> Good point!
> > >>>
> > >>> @tobias: can you build from source?
> > >>>
> > >>> If not, we could work with git bisect and custom packages, but that
> > >>> looks very time-consuming.
> > >>>
> > >>> Rainer
> > > _______________________________________________
> > > rsyslog mailing list
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=eCE_uw1iz27utOr31oN-0_3M4_4L8Kfe4WebMYOuy1o&e=
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=xweK2GTzEkqcrBBZIJq3bnF0uiFtOepUAno-jOYzTu8&s=X2Jy3MZhZvUU2y8SREXpuYme4VAi3hEn2FDlEfdNt6E&e=
> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> > > _______________________________________________
> > > rsyslog mailing list
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e= NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> > >
> > > Confidentiality Notice:: This email, including attachments, may include non-public, proprietary, confidential or legally privileged information. If you are not an intended recipient or an authorized agent of an intended recipient, you are hereby notified that any dissemination, distribution or copying of the information contained in or transmitted with this e-mail is unauthorized and strictly prohibited. If you have received this email in error, please notify the sender by replying to this message and permanently delete this e-mail, its attachments, and any copies of it immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.
> > > _______________________________________________
> > > rsyslog mailing list
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=NcjAJ2Kf_t5sCfkgLq55TKYlm8NXDcnqo82J3ivgRJ8&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=-vr4j0BXX9DOYxocIscwHayDvQ2Xx982jk-DMJvZ2Ro&s=hkiSFmPxD44uanS98WxuE34ktlghA-EI5XW3jB23l6k&e=
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> > _______________________________________________
> > rsyslog mailing list
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=GU19wL68jlENObb7o1-k6kI7_7hYyYr9vrHYca1TxOI&s=L8jzjEaRlubwQt3WPTDcONsCrfsTtsbLcVcmHMjwWXs&e=
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=Gev2oD_cwBaLsgGrQ_lnBXqGXR9EUvddO5r7UxzqP5s&m=GU19wL68jlENObb7o1-k6kI7_7hYyYr9vrHYca1TxOI&s=GyLXb3i3V-7QPuCALwX5mogCa1es_Y5Z-2mgpdCuBDQ&e=
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

can you post your config?

also, there is a way to enable debugging with a signal (debug on demand), if you
can let a machine get into a bad state and then enable the debug, it's far more
useful than nothing (even though not as good as a debug that catches when it
starts having problems)

David Lang

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.