Mailing List Archive

rsyslogd HUP and logrotate
Hello,

I've got a general question about logrotate postrotate script and rsyslog HUP signal handling interaction.

Lets say I have this logrotate section:

/var/log/messages
{
rotate 4
daily
dateext
missingok
notifempty
compress
postrotate
/usr/bin/systemctl kill -s HUP rsyslog.service >/dev/null 2>&1 || true
endscript
}

And logrotate fires up. Something like might happen:

1. /var/log/messages is rotated to /var/log/messages-20210827
2. rsyslog is still writing to /var/log/messages-20210827
3. postrotate script runs, the kill signal command returns immediately
4. logrotate start compressing /var/log/messages-20210827

The "real" handling of SIGUP happens in rsyslogd mainloop and could happen anywhere after 3, which means also after 4.

When the /var/log/messages activity is high (about 100 messages/sec), I often have logrotate compression step
complaining about "file size changed while zipping".

I can workaround this using the delaycompress option, but I'd like to understand if my understanding is correct and if
there is a better/cleaner solution ?

If I'm correct until this point, I can only see two options:

- using delaycompress in logrotate block
- having a command line tool that tells rsyslogd to close and reopen the log files AND that does not exit before
its done and use this command in postrotate

What are your thoughts ? Any suggestions ?

Thank you.

Jean-Baptiste





_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: rsyslogd HUP and logrotate [ In reply to ]
signals (like HUP) are sent and processed asynchronously, so it can take a
little bit of time to be handled. Your understanding is correct.

what I normally do is not use logrotate, but do the rotation myself in a script
run from cron that does the mv of the files, sends the hub, then sleeps for a
second before starting the compression (and then normally I mv the compressed
file to a date based directory hierarchy, when you have very high log volumes,
you may want to rotate files frequently, I've done it as frequently as every
minute, and that results is LOTS of files, more than you want in one directory)

I don't know if this can be done in the postrotate stanza of logrotate.

David Lang

On Fri, 27 Aug 2021, Jean-Baptiste Denis via rsyslog wrote:

> Date: Fri, 27 Aug 2021 16:34:51 +0200
> From: Jean-Baptiste Denis via rsyslog <rsyslog@lists.adiscon.com>
> Reply-To: jbdenis@pasteur.fr, rsyslog-users <rsyslog@lists.adiscon.com>
> To: rsyslog@lists.adiscon.com
> Cc: Jean-Baptiste Denis <jbdenis@pasteur.fr>
> Subject: [rsyslog] rsyslogd HUP and logrotate
>
> Hello,
>
> I've got a general question about logrotate postrotate script and rsyslog HUP
> signal handling interaction.
>
> Lets say I have this logrotate section:
>
> /var/log/messages
> {
> rotate 4
> daily
> dateext
> missingok
> notifempty
> compress
> postrotate
> /usr/bin/systemctl kill -s HUP rsyslog.service >/dev/null
> 2>&1 || true
> endscript
> }
>
> And logrotate fires up. Something like might happen:
>
> 1. /var/log/messages is rotated to /var/log/messages-20210827
> 2. rsyslog is still writing to /var/log/messages-20210827
> 3. postrotate script runs, the kill signal command returns immediately
> 4. logrotate start compressing /var/log/messages-20210827
>
> The "real" handling of SIGUP happens in rsyslogd mainloop and could happen
> anywhere after 3, which means also after 4.
>
> When the /var/log/messages activity is high (about 100 messages/sec), I often
> have logrotate compression step complaining about "file size changed while
> zipping".
>
> I can workaround this using the delaycompress option, but I'd like to
> understand if my understanding is correct and if there is a better/cleaner
> solution ?
>
> If I'm correct until this point, I can only see two options:
>
> - using delaycompress in logrotate block
> - having a command line tool that tells rsyslogd to close and reopen the log
> files AND that does not exit before
> its done and use this command in postrotate
>
> What are your thoughts ? Any suggestions ?
>
> Thank you.
>
> Jean-Baptiste
>
>
>
>
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: rsyslogd HUP and logrotate [ In reply to ]
Hello David,

thank you for the explanation and confirmation !

So, this is a kind of gray area unless rsyslog offers some kind of blocking mechanism that do the close/reopen and exits
when its done. rsyslog could also handle the rotation, but I have only seen a size criteria using outchannel
(https://www.rsyslog.com/doc/master/tutorials/log_rotation_fix_size.html).

I could also add a sleep X after sending the HUP in the postrotate script to achieve something similar to what you
suggest. But relying on sleep to ensure that the HUP has been fully processed does not sound reliable (at least it is
not satisfactory to me).

Jean-Baptiste

On 8/27/21 10:18 PM, David Lang wrote:
> signals (like HUP) are sent and processed asynchronously, so it can take a little bit of time to be handled. Your
> understanding is correct.
>
> what I normally do is not use logrotate, but do the rotation myself in a script run from cron that does the mv of the
> files, sends the hub, then sleeps fora second before starting the compression (and then normally I mv the compressed
> file to a date based directory hierarchy, when you have very high log volumes, you may want to rotate files frequently,
> I've done it as frequently as every minute, and that results is LOTS of files, more than you want in one directory)
>
> I don't know if this can be done in the postrotate stanza of logrotate.
>
> David Lang
>
> On Fri, 27 Aug 2021, Jean-Baptiste Denis via rsyslog wrote:
>
>> Date: Fri, 27 Aug 2021 16:34:51 +0200
>> From: Jean-Baptiste Denis via rsyslog <rsyslog@lists.adiscon.com>
>> Reply-To: jbdenis@pasteur.fr, rsyslog-users <rsyslog@lists.adiscon.com>
>> To: rsyslog@lists.adiscon.com
>> Cc: Jean-Baptiste Denis <jbdenis@pasteur.fr>
>> Subject: [rsyslog] rsyslogd HUP and logrotate
>>
>> Hello,
>>
>> I've got a general question about logrotate postrotate script and rsyslogHUP signal handling interaction.
>>
>> Lets say I have this logrotate section:
>>
>> /var/log/messages
>> {
>>     rotate 4
>>     daily
>>        dateext
>>     missingok
>>     notifempty
>>     compress
>>     postrotate
>>         /usr/bin/systemctl kill -s HUP rsyslog.service >/dev/null 2>&1 || true
>>     endscript
>> }
>>
>> And logrotate fires up. Something like might happen:
>>
>> 1. /var/log/messages is rotated to /var/log/messages-20210827
>> 2. rsyslog is still writing to /var/log/messages-20210827
>> 3. postrotate script runs, the kill signal command returns immediately
>> 4. logrotate start compressing /var/log/messages-20210827
>>
>> The "real" handling of SIGUP happens in rsyslogd mainloop and could happen anywhere after 3, which means also after 4.
>>
>> When the /var/log/messages activity is high (about 100 messages/sec), I often have logrotate compression step
>> complaining about "file size changed while zipping".
>>
>> I can workaround this using the delaycompress option, but I'd like to understand if my understanding is correct and if
>> there is a better/cleaner solution ?
>>
>> If I'm correct until this point, I can only see two options:
>>
>> - using delaycompress in logrotate block
>> - having a command line tool that tells rsyslogd to close and reopen the log files AND that does not exit before
>>  its done and use this command in postrotate
>>
>> What are your thoughts ? Any suggestions ?
>>
>> Thank you.
>>
>> Jean-Baptiste
>>
>>
>>
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> https://urldefense.com/v3/__https://lists.adiscon.net/mailman/listinfo/rsyslog__;!!JFdNOqOXpB6UZW0!4KbwKsaSUOggdRvK-LPY-1Q1vlVOQ26d_yxjlWF8IeLQNVAxcYLHIzJzGA9hqLmX$
>> https://urldefense.com/v3/__http://www.rsyslog.com/professional-services/__;!!JFdNOqOXpB6UZW0!4KbwKsaSUOggdRvK-LPY-1Q1vlVOQ26d_yxjlWF8IeLQNVAxcYLHIzJzGLbhsE0Z$
>> What's up with rsyslog? Follow
>> https://urldefense.com/v3/__https://twitter.com/rgerhards__;!!JFdNOqOXpB6UZW0!4KbwKsaSUOggdRvK-LPY-1Q1vlVOQ26d_yxjlWF8IeLQNVAxcYLHIzJzGMjZw-NT$
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE
>> UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>>

_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: rsyslogd HUP and logrotate [ In reply to ]
You could poll using lsof to detect when lsof no longer has the rotated file open. Not an elegant solution, but it would reliably tell you when it's safe to compress.

________________________________
From: rsyslog <rsyslog-bounces@lists.adiscon.com> on behalf of Jean-Baptiste Denis via rsyslog <rsyslog@lists.adiscon.com>
Sent: Friday, August 27, 2021 16:39
To: David Lang <david@lang.hm>; rsyslog-users <rsyslog@lists.adiscon.com>
Cc: Jean-Baptiste Denis <jbdenis@pasteur.fr>
Subject: Re: [rsyslog] rsyslogd HUP and logrotate

USE CAUTION. This message is from outside of N.C. A&T.
________________________________

Hello David,

thank you for the explanation and confirmation !

So, this is a kind of gray area unless rsyslog offers some kind of blocking mechanism that do the close/reopen and exits
when its done. rsyslog could also handle the rotation, but I have only seen a size criteria using outchannel
(https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.rsyslog.com%2Fdoc%2Fmaster%2Ftutorials%2Flog_rotation_fix_size.html&amp;data=04%7C01%7Casinger%40ncat.edu%7C10c250b5a98f4eefe02708d969a32328%7Cd844dd75a4d74b1fbd33bc0b1c796c38%7C0%7C0%7C637656972418202636%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=BsyPcnpd8wrbPBc6cnX9J3hTT%2FuVVoT2albLXGGtO%2BI%3D&amp;reserved=0).

I could also add a sleep X after sending the HUP in the postrotate script to achieve something similar to what you
suggest. But relying on sleep to ensure that the HUP has been fully processed does not sound reliable (at least it is
not satisfactory to me).

Jean-Baptiste

On 8/27/21 10:18 PM, David Lang wrote:
> signals (like HUP) are sent and processed asynchronously, so it can take a little bit of time to be handled. Your
> understanding is correct.
>
> what I normally do is not use logrotate, but do the rotation myself in a script run from cron that does the mv of the
> files, sends the hub, then sleeps fora second before starting the compression (and then normally I mv the compressed
> file to a date based directory hierarchy, when you have very high log volumes, you may want to rotate files frequently,
> I've done it as frequently as every minute, and that results is LOTS of files, more than you want in one directory)
>
> I don't know if this can be done in the postrotate stanza of logrotate.
>
> David Lang
>
> On Fri, 27 Aug 2021, Jean-Baptiste Denis via rsyslog wrote:
>
>> Date: Fri, 27 Aug 2021 16:34:51 +0200
>> From: Jean-Baptiste Denis via rsyslog <rsyslog@lists.adiscon.com>
>> Reply-To: jbdenis@pasteur.fr, rsyslog-users <rsyslog@lists.adiscon.com>
>> To: rsyslog@lists.adiscon.com
>> Cc: Jean-Baptiste Denis <jbdenis@pasteur.fr>
>> Subject: [rsyslog] rsyslogd HUP and logrotate
>>
>> Hello,
>>
>> I've got a general question about logrotate postrotate script and rsyslogHUP signal handling interaction.
>>
>> Lets say I have this logrotate section:
>>
>> /var/log/messages
>> {
>> rotate 4
>> daily
>> dateext
>> missingok
>> notifempty
>> compress
>> postrotate
>> /usr/bin/systemctl kill -s HUP rsyslog.service >/dev/null 2>&1 || true
>> endscript
>> }
>>
>> And logrotate fires up. Something like might happen:
>>
>> 1. /var/log/messages is rotated to /var/log/messages-20210827
>> 2. rsyslog is still writing to /var/log/messages-20210827
>> 3. postrotate script runs, the kill signal command returns immediately
>> 4. logrotate start compressing /var/log/messages-20210827
>>
>> The "real" handling of SIGUP happens in rsyslogd mainloop and could happen anywhere after 3, which means also after 4.
>>
>> When the /var/log/messages activity is high (about 100 messages/sec), I often have logrotate compression step
>> complaining about "file size changed while zipping".
>>
>> I can workaround this using the delaycompress option, but I'd like to understand if my understanding is correct and if
>> there is a better/cleaner solution ?
>>
>> If I'm correct until this point, I can only see two options:
>>
>> - using delaycompress in logrotate block
>> - having a command line tool that tells rsyslogd to close and reopen the log files AND that does not exit before
>> its done and use this command in postrotate
>>
>> What are your thoughts ? Any suggestions ?
>>
>> Thank you.
>>
>> Jean-Baptiste
>>
>>
>>
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Flists.adiscon.net%2Fmailman%2Flistinfo%2Frsyslog__%3B!!JFdNOqOXpB6UZW0!4KbwKsaSUOggdRvK-LPY-1Q1vlVOQ26d_yxjlWF8IeLQNVAxcYLHIzJzGA9hqLmX%24&amp;data=04%7C01%7Casinger%40ncat.edu%7C10c250b5a98f4eefe02708d969a32328%7Cd844dd75a4d74b1fbd33bc0b1c796c38%7C0%7C0%7C637656972418202636%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=K0YZjGTLX3tRKMStPHe359qu5MgxHxgPEBHuas3ewLQ%3D&amp;reserved=0
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2F%2Fwww.rsyslog.com%2Fprofessional-services%2F__%3B!!JFdNOqOXpB6UZW0!4KbwKsaSUOggdRvK-LPY-1Q1vlVOQ26d_yxjlWF8IeLQNVAxcYLHIzJzGLbhsE0Z%24&amp;data=04%7C01%7Casinger%40ncat.edu%7C10c250b5a98f4eefe02708d969a32328%7Cd844dd75a4d74b1fbd33bc0b1c796c38%7C0%7C0%7C637656972418202636%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=VhkhFdE7ngK0O2eON%2B8cvftRrmsoQ%2FPxisC%2B%2FsaAiQA%3D&amp;reserved=0
>> What's up with rsyslog? Follow
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Ftwitter.com%2Frgerhards__%3B!!JFdNOqOXpB6UZW0!4KbwKsaSUOggdRvK-LPY-1Q1vlVOQ26d_yxjlWF8IeLQNVAxcYLHIzJzGMjZw-NT%24&amp;data=04%7C01%7Casinger%40ncat.edu%7C10c250b5a98f4eefe02708d969a32328%7Cd844dd75a4d74b1fbd33bc0b1c796c38%7C0%7C0%7C637656972418202636%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=yUB4phnkP%2B8laU93L7F7q3wBcVblaXWDlh2HrcLBVrI%3D&amp;reserved=0
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE
>> UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>>

_______________________________________________
rsyslog mailing list
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.adiscon.net%2Fmailman%2Flistinfo%2Frsyslog&amp;data=04%7C01%7Casinger%40ncat.edu%7C10c250b5a98f4eefe02708d969a32328%7Cd844dd75a4d74b1fbd33bc0b1c796c38%7C0%7C0%7C637656972418202636%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=srtaZpnPBZIX0gn%2FEk%2BA3iAXk0bpXw1RLiYthFnUCQ4%3D&amp;reserved=0
https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.rsyslog.com%2Fprofessional-services%2F&amp;data=04%7C01%7Casinger%40ncat.edu%7C10c250b5a98f4eefe02708d969a32328%7Cd844dd75a4d74b1fbd33bc0b1c796c38%7C0%7C0%7C637656972418202636%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=6W1PLMyoCIaC5OEB0gWXrWhtd2ISToUgjCFKH44Ygzg%3D&amp;reserved=0
What's up with rsyslog? Follow https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Frgerhards&amp;data=04%7C01%7Casinger%40ncat.edu%7C10c250b5a98f4eefe02708d969a32328%7Cd844dd75a4d74b1fbd33bc0b1c796c38%7C0%7C0%7C637656972418202636%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=tzMQ4rITYSzaWG9s2RvVYAWR%2FN2o86KsT0LL5nmH%2FH4%3D&amp;reserved=0
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
------- NOTICE: This e-mail correspondence is subject to Public Records Law and may be disclosed to third parties. --------
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: rsyslogd HUP and logrotate [ In reply to ]
On Fri, 27 Aug 2021, Jean-Baptiste Denis wrote:

> So, this is a kind of gray area unless rsyslog offers some kind of blocking mechanism that do the close/reopen and exits
> when its done. rsyslog could also handle the rotation, but I have only seen a size criteria using outchannel
> (https://www.rsyslog.com/doc/master/tutorials/log_rotation_fix_size.html).

Rsyslog does not offer any blocking mechanism to close/reopen logs (I'm not
aware of any programs that do, especially not ones that work with signals as
most do, signals do not provide any option for feedback)

I guess you could query the OS to ensure that no other program has the file
open, but that seems fragile as well (imagine to programs monitoring the same
file, rsyslog may close it but they could detect each other), possibly an
inotify mechanism to detect the close, but again, how do you make sure the close
from rsyslog is what you detect.

> I could also add a sleep X after sending the HUP in the postrotate script to achieve something similar to what you
> suggest. But relying on sleep to ensure that the HUP has been fully processed does not sound reliable (at least it is
> not satisfactory to me).

the delay between the signal delivery and the close will be sub-second (unless
the system is under a huge overload), so even a 1 sec delay would be very
reliable.

David Lang

> Jean-Baptiste
>
> On 8/27/21 10:18 PM, David Lang wrote:
>> signals (like HUP) are sent and processed asynchronously, so it can take a little bit of time to be handled. Your
>> understanding is correct.
>>
>> what I normally do is not use logrotate, but do the rotation myself in a script run from cron that does the mv of the
>> files, sends the hub, then sleeps fora second before starting the compression (and then normally I mv the compressed
>> file to a date based directory hierarchy, when you have very high log volumes, you may want to rotate files frequently,
>> I've done it as frequently as every minute, and that results is LOTS of files, more than you want in one directory)
>>
>> I don't know if this can be done in the postrotate stanza of logrotate.
>>
>> David Lang
>>
>> On Fri, 27 Aug 2021, Jean-Baptiste Denis via rsyslog wrote:
>>
>>> Date: Fri, 27 Aug 2021 16:34:51 +0200
>>> From: Jean-Baptiste Denis via rsyslog <rsyslog@lists.adiscon.com>
>>> Reply-To: jbdenis@pasteur.fr, rsyslog-users <rsyslog@lists.adiscon.com>
>>> To: rsyslog@lists.adiscon.com
>>> Cc: Jean-Baptiste Denis <jbdenis@pasteur.fr>
>>> Subject: [rsyslog] rsyslogd HUP and logrotate
>>>
>>> Hello,
>>>
>>> I've got a general question about logrotate postrotate script and rsyslogHUP signal handling interaction.
>>>
>>> Lets say I have this logrotate section:
>>>
>>> /var/log/messages
>>> {
>>>     rotate 4
>>>     daily
>>>        dateext
>>>     missingok
>>>     notifempty
>>>     compress
>>>     postrotate
>>>         /usr/bin/systemctl kill -s HUP rsyslog.service >/dev/null 2>&1 || true
>>>     endscript
>>> }
>>>
>>> And logrotate fires up. Something like might happen:
>>>
>>> 1. /var/log/messages is rotated to /var/log/messages-20210827
>>> 2. rsyslog is still writing to /var/log/messages-20210827
>>> 3. postrotate script runs, the kill signal command returns immediately
>>> 4. logrotate start compressing /var/log/messages-20210827
>>>
>>> The "real" handling of SIGUP happens in rsyslogd mainloop and could happen anywhere after 3, which means also after 4.
>>>
>>> When the /var/log/messages activity is high (about 100 messages/sec), I often have logrotate compression step
>>> complaining about "file size changed while zipping".
>>>
>>> I can workaround this using the delaycompress option, but I'd like to understand if my understanding is correct and if
>>> there is a better/cleaner solution ?
>>>
>>> If I'm correct until this point, I can only see two options:
>>>
>>> - using delaycompress in logrotate block
>>> - having a command line tool that tells rsyslogd to close and reopen the log files AND that does not exit before
>>>  its done and use this command in postrotate
>>>
>>> What are your thoughts ? Any suggestions ?
>>>
>>> Thank you.
>>>
>>> Jean-Baptiste
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> https://urldefense.com/v3/__https://lists.adiscon.net/mailman/listinfo/rsyslog__;!!JFdNOqOXpB6UZW0!4KbwKsaSUOggdRvK-LPY-1Q1vlVOQ26d_yxjlWF8IeLQNVAxcYLHIzJzGA9hqLmX$
>>> https://urldefense.com/v3/__http://www.rsyslog.com/professional-services/__;!!JFdNOqOXpB6UZW0!4KbwKsaSUOggdRvK-LPY-1Q1vlVOQ26d_yxjlWF8IeLQNVAxcYLHIzJzGLbhsE0Z$
>>> What's up with rsyslog? Follow
>>> https://urldefense.com/v3/__https://twitter.com/rgerhards__;!!JFdNOqOXpB6UZW0!4KbwKsaSUOggdRvK-LPY-1Q1vlVOQ26d_yxjlWF8IeLQNVAxcYLHIzJzGMjZw-NT$
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE
>>> UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>>>
>
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: rsyslogd HUP and logrotate [ In reply to ]
Hey all,

Another variant is to write to date named files and then you can
compress them directly after midnight.

Can't rotate on anything else based on time and date though so YMMV.

Example configuration:

template(name="omfileTemplate"
type="string"
string="/srv/logs/%$year%/%fromhost%/%fromhost%-%$now%.log"
)

action(type="omfile"
dynaFile="omfileTemplate"
template="SU_FileFormat"
)


BR,
- Simon

On Fri, 2021-08-27 at 16:34:51 +0200, Jean-Baptiste Denis via rsyslog
wrote:
> Hello,
>
> I've got a general question about logrotate postrotate script and rsyslog HUP signal handling interaction.
>
> Lets say I have this logrotate section:
>
> /var/log/messages
> {
> rotate 4
> daily
> dateext
> missingok
> notifempty
> compress
> postrotate
> /usr/bin/systemctl kill -s HUP rsyslog.service >/dev/null 2>&1 || true
> endscript
> }
>
> And logrotate fires up. Something like might happen:
>
> 1. /var/log/messages is rotated to /var/log/messages-20210827
> 2. rsyslog is still writing to /var/log/messages-20210827
> 3. postrotate script runs, the kill signal command returns immediately
> 4. logrotate start compressing /var/log/messages-20210827
>
> The "real" handling of SIGUP happens in rsyslogd mainloop and could happen anywhere after 3, which means also after 4.
>
> When the /var/log/messages activity is high (about 100 messages/sec), I often have logrotate compression step
> complaining about "file size changed while zipping".
>
> I can workaround this using the delaycompress option, but I'd like to understand if my understanding is correct and if
> there is a better/cleaner solution ?
>
> If I'm correct until this point, I can only see two options:
>
> - using delaycompress in logrotate block
> - having a command line tool that tells rsyslogd to close and reopen the log files AND that does not exit before
> its done and use this command in postrotate
>
> What are your thoughts ? Any suggestions ?
>
> Thank you.
>
> Jean-Baptiste
>
>
>
>
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.