Mailing List Archive

Equivalent omfile action to a file prefixed by '-'?
In the old style of rsyslog configuration, you can say that you want to buffer a file, rather than flush it after every write, with syntax like this (prefixing the filename with '-'):

local4.info -/var/log/mylog

Reading the documentation of the omfile Output Module, I don't see an exact way to specify the exact same end result documented, such that you could write something like:

local4.info {
action( type="omfile" file="/var/log/mylog" ...)
}

I see all the following, but not sure of the exact combined config to equate to the old syntax:

* flushInterval
* asyncWriting
* flushOnTxEnd
* ioBufferSize
* sync

Thanks for any insight into achieving this goal.

GM
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Equivalent omfile action to a file prefixed by '-'? [ In reply to ]
If I am not mistaken, buffering is the default in the new syntax such that it is not necessary to state it explicitly. The maintainers will know for sure.



> On Sep 29, 2022, at 15:54, Gordon Marler (BLOOMBERG/ 120 PARK) via rsyslog <rsyslog@lists.adiscon.com> wrote:
>
> In the old style of rsyslog configuration, you can say that you want to buffer a file, rather than flush it after every write, with syntax like this (prefixing the filename with '-'):
>
> local4.info -/var/log/mylog
>
> Reading the documentation of the omfile Output Module, I don't see an exact way to specify the exact same end result documented, such that you could write something like:
>
> local4.info {
> action( type="omfile" file="/var/log/mylog" ...)
> }
>
> I see all the following, but not sure of the exact combined config to equate to the old syntax:
>
> * flushInterval
> * asyncWriting
> * flushOnTxEnd
> * ioBufferSize
> * sync
>
> Thanks for any insight into achieving this goal.
>
> GM
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Equivalent omfile action to a file prefixed by '-'? [ In reply to ]
The - does not throw a syntax error, but is ignored for many years. iIRC it
caused a sync after each write, which would be crazy for actual systems.

Rainer

Sent from phone, thus brief.

John Chivian via rsyslog <rsyslog@lists.adiscon.com> schrieb am Do., 29.
Sep. 2022, 23:25:

> If I am not mistaken, buffering is the default in the new syntax such that
> it is not necessary to state it explicitly. The maintainers will know for
> sure.
>
>
>
> > On Sep 29, 2022, at 15:54, Gordon Marler (BLOOMBERG/ 120 PARK) via
> rsyslog <rsyslog@lists.adiscon.com> wrote:
> >
> > In the old style of rsyslog configuration, you can say that you want to
> buffer a file, rather than flush it after every write, with syntax like
> this (prefixing the filename with '-'):
> >
> > local4.info -/var/log/mylog
> >
> > Reading the documentation of the omfile Output Module, I don't see an
> exact way to specify the exact same end result documented, such that you
> could write something like:
> >
> > local4.info {
> > action( type="omfile" file="/var/log/mylog" ...)
> > }
> >
> > I see all the following, but not sure of the exact combined config to
> equate to the old syntax:
> >
> > * flushInterval
> > * asyncWriting
> > * flushOnTxEnd
> > * ioBufferSize
> > * sync
> >
> > Thanks for any insight into achieving this goal.
> >
> > GM
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Equivalent omfile action to a file prefixed by '-'? [ In reply to ]
it is possible to get the 'unbuffered' mode, but you have to work hard to do it
and the performance will utterly suck (even if you have a
battery-backed-ramdisk, the system calls and serialization will cripple your
performance)

I think this has been the default since at least rsyslog 3, around 20 years ago.

David Lang


On Fri, 30 Sep 2022, Rainer Gerhards via rsyslog wrote:

> Date: Fri, 30 Sep 2022 00:02:28 +0200
> From: Rainer Gerhards via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog-users <rsyslog@lists.adiscon.com>
> Cc: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Subject: Re: [rsyslog] Equivalent omfile action to a file prefixed by '-'?
>
> The - does not throw a syntax error, but is ignored for many years. iIRC it
> caused a sync after each write, which would be crazy for actual systems.
>
> Rainer
>
> Sent from phone, thus brief.
>
> John Chivian via rsyslog <rsyslog@lists.adiscon.com> schrieb am Do., 29.
> Sep. 2022, 23:25:
>
>> If I am not mistaken, buffering is the default in the new syntax such that
>> it is not necessary to state it explicitly. The maintainers will know for
>> sure.
>>
>>
>>
>>> On Sep 29, 2022, at 15:54, Gordon Marler (BLOOMBERG/ 120 PARK) via
>> rsyslog <rsyslog@lists.adiscon.com> wrote:
>>>
>>> In the old style of rsyslog configuration, you can say that you want to
>> buffer a file, rather than flush it after every write, with syntax like
>> this (prefixing the filename with '-'):
>>>
>>> local4.info -/var/log/mylog
>>>
>>> Reading the documentation of the omfile Output Module, I don't see an
>> exact way to specify the exact same end result documented, such that you
>> could write something like:
>>>
>>> local4.info {
>>> action( type="omfile" file="/var/log/mylog" ...)
>>> }
>>>
>>> I see all the following, but not sure of the exact combined config to
>> equate to the old syntax:
>>>
>>> * flushInterval
>>> * asyncWriting
>>> * flushOnTxEnd
>>> * ioBufferSize
>>> * sync
>>>
>>> Thanks for any insight into achieving this goal.
>>>
>>> GM
>>> _______________________________________________
>>> rsyslog mailing list
>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Equivalent omfile action to a file prefixed by '-'? [ In reply to ]
No, not looking for unbuffered mode, but since we were converting from the old
to new config style, and many of our heavily used logs prefixed with '-', just
wanted to be sure that we weren't accidentally going backward to unbuffered mode.

If anything, we'd more likely want to increase the size of the buffer for specific
logs, and knowing that '-' didn't change any of the defaults anymore is quite helpful.

Thanks to all for clarifying.

From: rsyslog@lists.adiscon.com At: 09/29/22 19:01:44 UTC-4:00To: rsyslog@lists.adiscon.com
Cc: david@lang.hm
Subject: Re: [rsyslog] Equivalent omfile action to a file prefixed by '-'?

it is possible to get the 'unbuffered' mode, but you have to work hard to do it
and the performance will utterly suck (even if you have a
battery-backed-ramdisk, the system calls and serialization will cripple your
performance)

I think this has been the default since at least rsyslog 3, around 20 years ago.

David Lang


On Fri, 30 Sep 2022, Rainer Gerhards via rsyslog wrote:

> Date: Fri, 30 Sep 2022 00:02:28 +0200
> From: Rainer Gerhards via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog-users <rsyslog@lists.adiscon.com>
> Cc: Rainer Gerhards <rgerhards@hq.adiscon.com>
> Subject: Re: [rsyslog] Equivalent omfile action to a file prefixed by '-'?
>
> The - does not throw a syntax error, but is ignored for many years. iIRC it
> caused a sync after each write, which would be crazy for actual systems.
>
> Rainer
>
> Sent from phone, thus brief.
>
> John Chivian via rsyslog <rsyslog@lists.adiscon.com> schrieb am Do., 29.
> Sep. 2022, 23:25:
>
>> If I am not mistaken, buffering is the default in the new syntax such that
>> it is not necessary to state it explicitly. The maintainers will know for
>> sure.
>>
>>
>>
>>> On Sep 29, 2022, at 15:54, Gordon Marler (BLOOMBERG/ 120 PARK) via
>> rsyslog <rsyslog@lists.adiscon.com> wrote:
>>>
>>> In the old style of rsyslog configuration, you can say that you want to
>> buffer a file, rather than flush it after every write, with syntax like
>> this (prefixing the filename with '-'):
>>>
>>> local4.info -/var/log/mylog
>>>
>>> Reading the documentation of the omfile Output Module, I don't see an
>> exact way to specify the exact same end result documented, such that you
>> could write something like:
>>>
>>> local4.info {
>>> action( type="omfile" file="/var/log/mylog" ...)
>>> }
>>>
>>> I see all the following, but not sure of the exact combined config to
>> equate to the old syntax:
>>>
>>> * flushInterval
>>> * asyncWriting
>>> * flushOnTxEnd
>>> * ioBufferSize
>>> * sync
>>>
>>> Thanks for any insight into achieving this goal.
>>>
>>> GM
>>> _______________________________________________
>>> rsyslog mailing list
>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.


_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Equivalent omfile action to a file prefixed by '-'? [ In reply to ]
adding a queue to an omfile output will probably just slow it down (the one
excetion being if you are having rsyslog compress/encrypt the file)

the overhead of lockinf, adding to a queue, then unlockinh (and have another
process lock, read, unlock the queue) is larger than the time needed to
just write the log out

David Lang

On Thu, 29 Sep 2022, Gordon Marler (BLOOMBERG/ 120 PARK) wrote:

> Date: Thu, 29 Sep 2022 23:17:30 -0000
> From: "Gordon Marler (BLOOMBERG/ 120 PARK)" <gmarler@bloomberg.net>
> To: rsyslog@lists.adiscon.com
> Cc: david@lang.hm
> Subject: Re: [rsyslog] Equivalent omfile action to a file prefixed by '-'?
>
> No, not looking for unbuffered mode, but since we were converting from the old
> to new config style, and many of our heavily used logs prefixed with '-', just
> wanted to be sure that we weren't accidentally going backward to unbuffered mode.
>
> If anything, we'd more likely want to increase the size of the buffer for specific
> logs, and knowing that '-' didn't change any of the defaults anymore is quite helpful.
>
> Thanks to all for clarifying.
>
> From: rsyslog@lists.adiscon.com At: 09/29/22 19:01:44 UTC-4:00To: rsyslog@lists.adiscon.com
> Cc: david@lang.hm
> Subject: Re: [rsyslog] Equivalent omfile action to a file prefixed by '-'?
>
> it is possible to get the 'unbuffered' mode, but you have to work hard to do it
> and the performance will utterly suck (even if you have a
> battery-backed-ramdisk, the system calls and serialization will cripple your
> performance)
>
> I think this has been the default since at least rsyslog 3, around 20 years ago.
>
> David Lang
>
>
> On Fri, 30 Sep 2022, Rainer Gerhards via rsyslog wrote:
>
>> Date: Fri, 30 Sep 2022 00:02:28 +0200
>> From: Rainer Gerhards via rsyslog <rsyslog@lists.adiscon.com>
>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>> Cc: Rainer Gerhards <rgerhards@hq.adiscon.com>
>> Subject: Re: [rsyslog] Equivalent omfile action to a file prefixed by '-'?
>>
>> The - does not throw a syntax error, but is ignored for many years. iIRC it
>> caused a sync after each write, which would be crazy for actual systems.
>>
>> Rainer
>>
>> Sent from phone, thus brief.
>>
>> John Chivian via rsyslog <rsyslog@lists.adiscon.com> schrieb am Do., 29.
>> Sep. 2022, 23:25:
>>
>>> If I am not mistaken, buffering is the default in the new syntax such that
>>> it is not necessary to state it explicitly. The maintainers will know for
>>> sure.
>>>
>>>
>>>
>>>> On Sep 29, 2022, at 15:54, Gordon Marler (BLOOMBERG/ 120 PARK) via
>>> rsyslog <rsyslog@lists.adiscon.com> wrote:
>>>>
>>>> In the old style of rsyslog configuration, you can say that you want to
>>> buffer a file, rather than flush it after every write, with syntax like
>>> this (prefixing the filename with '-'):
>>>>
>>>> local4.info -/var/log/mylog
>>>>
>>>> Reading the documentation of the omfile Output Module, I don't see an
>>> exact way to specify the exact same end result documented, such that you
>>> could write something like:
>>>>
>>>> local4.info {
>>>> action( type="omfile" file="/var/log/mylog" ...)
>>>> }
>>>>
>>>> I see all the following, but not sure of the exact combined config to
>>> equate to the old syntax:
>>>>
>>>> * flushInterval
>>>> * asyncWriting
>>>> * flushOnTxEnd
>>>> * ioBufferSize
>>>> * sync
>>>>
>>>> Thanks for any insight into achieving this goal.
>>>>
>>>> GM
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com/professional-services/
>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>> DON'T LIKE THAT.
>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>> DON'T LIKE THAT.
>>>
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
> THAT.
>>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
> THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.