Mailing List Archive: Query regarding Failover Syslog Server

Query regarding Failover Syslog Server

Jun 6, 2020, 6:51 AM

Post #1 of 5 (702 views)

Hi All,

As per documentation
https://www.rsyslog.com/doc/v8-stable/tutorials/failover_syslog_server.html,
it says

"Failover will *not* work when you define queues on the actions".

Proposed solution is

"If you would like to use a queue on the forwarding process as whole, the
solution is to put all actions into a ruleset and assign a queue to the
ruleset"

While I understand the rationale behind assigning a queue to the ruleset, I
don't understand how the last example given in doc works and first example
doesn't work.

In the last example, Ruleset consisted of 3 actions. Each action didn't
specify any queue parameters - which means they have a default direct
queue. Since actions have a queue, even the last example should fail (Just
like the first example in the doc where actions didn't have a queue
explicitly defined in config). Please correct me where I got wrong.

ruleset(name="forwarding" queue.type="linkedList" queue.filename="fwdq") {
action(type="omfwd" target="primary-syslog.example.com" port="10514"
protocol="tcp")
action(type="omfwd" target="secondary-1-syslog.example.com" port="10514"
action.execOnlyWhenPreviousIsSuspended="on")
action(type="omfwd" target="secondary-2-syslog.example.com" port="10514"
action.execOnlyWhenPreviousIsSuspended="on")
action(type="omfile" tag="failover" file="/var/log/localbuffer"
action.execOnlyWhenPreviousIsSuspended="on")
}

if($msg contains "error") then {
call forwarding
}

--
Regards,
Rajesh KSV
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Re: Query regarding Failover Syslog Server [ In reply to ]

rsyslog at lists

Jun 6, 2020, 8:16 AM

Post #2 of 5 (702 views)

Permalink

"Each action didn't specify any queue parameters - which means they have
a default direct queue."

That statement is not correct. Actions do not have queues by default.

If you define a queue on an action, then the action never suspends,
hence the waterfall of "only when previous suspended" will never
accomplish the failover. If an action does not have a queue, and cannot
complete, THEN it will suspend and the next action be attempted. The
queue on the ruleset only comes into play if ALL of the actions are
suspended.

Regards,

On 6/6/20 8:51 AM, rajeshksv via rsyslog wrote:
> Hi All,
>
> As per documentation
> https://www.rsyslog.com/doc/v8-stable/tutorials/failover_syslog_server.html,
> it says
>
> "Failover will *not* work when you define queues on the actions".
>
> Proposed solution is
>
> "If you would like to use a queue on the forwarding process as whole, the
> solution is to put all actions into a ruleset and assign a queue to the
> ruleset"
>
> While I understand the rationale behind assigning a queue to the ruleset, I
> don't understand how the last example given in doc works and first example
> doesn't work.
>
> In the last example, Ruleset consisted of 3 actions. Each action didn't
> specify any queue parameters - which means they have a default direct
> queue. Since actions have a queue, even the last example should fail (Just
> like the first example in the doc where actions didn't have a queue
> explicitly defined in config). Please correct me where I got wrong.
>
> ruleset(name="forwarding" queue.type="linkedList" queue.filename="fwdq") {
> action(type="omfwd" target="primary-syslog.example.com" port="10514"
> protocol="tcp")
> action(type="omfwd" target="secondary-1-syslog.example.com" port="10514"
> action.execOnlyWhenPreviousIsSuspended="on")
> action(type="omfwd" target="secondary-2-syslog.example.com" port="10514"
> action.execOnlyWhenPreviousIsSuspended="on")
> action(type="omfile" tag="failover" file="/var/log/localbuffer"
> action.execOnlyWhenPreviousIsSuspended="on")
> }
>
> if($msg contains "error") then {
> call forwarding
> }
>
>
>

_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Re: Query regarding Failover Syslog Server [ In reply to ]

rsyslog at lists

Jun 7, 2020, 11:37 AM

Post #3 of 5 (701 views)

Permalink

In queues doc - https://www.rsyslog.com/doc/v8-stable/concepts/queues.html,
it says

"There are multiple action queues, one for each configured action. By
default, these queues operate in direct (non-queueing) mode"

May be, when doc says

"Failover will *not* work when you define queues on the actions".

It actually means - "when you define non-direct queues on the actions"

(As Direct queues on actions are fine for failover - Since return status is
passed back to the producer in case of direct queue. )

Is my understanding correct?

On Sat, Jun 6, 2020 at 8:46 PM John Chivian via rsyslog <
rsyslog@lists.adiscon.com> wrote:

> "Each action didn't specify any queue parameters - which means they have
> a default direct queue."
>
> That statement is not correct. Actions do not have queues by default.
>
> If you define a queue on an action, then the action never suspends,
> hence the waterfall of "only when previous suspended" will never
> accomplish the failover. If an action does not have a queue, and cannot
> complete, THEN it will suspend and the next action be attempted. The
> queue on the ruleset only comes into play if ALL of the actions are
> suspended.
>
> Regards,
>
>
> On 6/6/20 8:51 AM, rajeshksv via rsyslog wrote:
> > Hi All,
> >
> > As per documentation
> >
> https://www.rsyslog.com/doc/v8-stable/tutorials/failover_syslog_server.html
> ,
> > it says
> >
> > "Failover will *not* work when you define queues on the actions".
> >
> > Proposed solution is
> >
> > "If you would like to use a queue on the forwarding process as whole, the
> > solution is to put all actions into a ruleset and assign a queue to the
> > ruleset"
> >
> > While I understand the rationale behind assigning a queue to the
> ruleset, I
> > don't understand how the last example given in doc works and first
> example
> > doesn't work.
> >
> > In the last example, Ruleset consisted of 3 actions. Each action didn't
> > specify any queue parameters - which means they have a default direct
> > queue. Since actions have a queue, even the last example should fail
> (Just
> > like the first example in the doc where actions didn't have a queue
> > explicitly defined in config). Please correct me where I got wrong.
> >
> > ruleset(name="forwarding" queue.type="linkedList" queue.filename="fwdq")
> {
> > action(type="omfwd" target="primary-syslog.example.com"
> port="10514"
> > protocol="tcp")
> > action(type="omfwd" target="secondary-1-syslog.example.com"
> port="10514"
> > action.execOnlyWhenPreviousIsSuspended="on")
> > action(type="omfwd" target="secondary-2-syslog.example.com"
> port="10514"
> > action.execOnlyWhenPreviousIsSuspended="on")
> > action(type="omfile" tag="failover" file="/var/log/localbuffer"
> > action.execOnlyWhenPreviousIsSuspended="on")
> > }
> >
> > if($msg contains "error") then {
> > call forwarding
> > }
> >
> >
> >
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.

--
Regards,
Rajesh KSV
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Re: Query regarding Failover Syslog Server [ In reply to ]

rsyslog at lists

Jun 7, 2020, 11:57 AM

Post #4 of 5 (701 views)

Permalink

It is true that rsyslog has "queues" for all actions. However by
default they don't do anything, so if no queuing is done is it really a
queue? Semantics, nothing more.

It is better to say that all actions support possible queuing, hence
there is an object called "a queue" associated with the action
regardless of whether or not it actually does anything.

To my way of thinking a "direct mode queue" is a bit of an oxymoron, but
as a developer I understand the need and the context. And technically
the documentation as written is correct, the keyword being "you" as in
when you (the implementer) define queuing on the actions.

Hope that helps,

On 6/7/20 1:37 PM, rajeshksv wrote:
> In queues doc -
> https://www.rsyslog.com/doc/v8-stable/concepts/queues.html, it says
>
> "There are multiple action queues, one for each configured action. By
> default, these queues operate in direct (non-queueing) mode"
>
> May be, when doc says
>
> "Failover will *not* work when you define queues on the actions".
>
> It actually means - "when you define non-direct queues on the actions"
>
> (As Direct queues on actions are fine for failover - Since return
> status is passed back to the producer in case of direct queue. )
>
> Is my understanding correct?
>
>
> On Sat, Jun 6, 2020 at 8:46 PM John Chivian via rsyslog
> <rsyslog@lists.adiscon.com <mailto:rsyslog@lists.adiscon.com>> wrote:
>
> "Each action didn't specify any queue parameters - which means
> they have
> a default direct queue."
>
> That statement is not correct. Actions do not have queues by default.
>
> If you define a queue on an action, then the action never suspends,
> hence the waterfall of "only when previous suspended" will never
> accomplish the failover. If an action does not have a queue, and
> cannot
> complete, THEN it will suspend and the next action be attempted. The
> queue on the ruleset only comes into play if ALL of the actions are
> suspended.
>
> Regards,
>

_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Re: Query regarding Failover Syslog Server [ In reply to ]

rsyslog at lists

Jun 7, 2020, 12:09 PM

Post #5 of 5 (701 views)

Permalink

Gotcha. Thanks

With the "Direct Queues" concept in rsyslog, there is scope for confusion
at few places. When documentation says "queue", it is sometimes confusing
if it's referring to the "queue" concept in general or semantic meaning of
"queues" wrt rsyslog (which includes direct queue as well). I should start
taking it as general "queue" concept

Thanks again.

On Mon, Jun 8, 2020 at 12:27 AM John Chivian <jchivian@chivian.com> wrote:

> It is true that rsyslog has "queues" for all actions. However by default
> they don't do anything, so if no queuing is done is it really a queue?
> Semantics, nothing more.
>
> It is better to say that all actions support possible queuing, hence there
> is an object called "a queue" associated with the action regardless of
> whether or not it actually does anything.
>
> To my way of thinking a "direct mode queue" is a bit of an oxymoron, but
> as a developer I understand the need and the context. And technically the
> documentation as written is correct, the keyword being "you" as in when you
> (the implementer) define queuing on the actions.
>
> Hope that helps,
>
>
> On 6/7/20 1:37 PM, rajeshksv wrote:
>
> In queues doc - https://www.rsyslog.com/doc/v8-stable/concepts/queues.html,
> it says
>
> "There are multiple action queues, one for each configured action. By
> default, these queues operate in direct (non-queueing) mode"
>
> May be, when doc says
>
> "Failover will *not* work when you define queues on the actions".
>
> It actually means - "when you define non-direct queues on the actions"
>
> (As Direct queues on actions are fine for failover - Since return status
> is passed back to the producer in case of direct queue. )
>
> Is my understanding correct?
>
>
> On Sat, Jun 6, 2020 at 8:46 PM John Chivian via rsyslog <
> rsyslog@lists.adiscon.com> wrote:
>
>> "Each action didn't specify any queue parameters - which means they have
>> a default direct queue."
>>
>> That statement is not correct. Actions do not have queues by default.
>>
>> If you define a queue on an action, then the action never suspends,
>> hence the waterfall of "only when previous suspended" will never
>> accomplish the failover. If an action does not have a queue, and cannot
>> complete, THEN it will suspend and the next action be attempted. The
>> queue on the ruleset only comes into play if ALL of the actions are
>> suspended.
>>
>> Regards,
>
>
>

--
Regards,
Rajesh KSV
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.