We have received complaints about rsyslog documentation repeatedly, We have
a lot of detail, but it's all written for someone already fairly familiar
with things.
Here is a 3am first pass from me at writing an overview of how rsyslog works,
with the idea that this could be made pretty with diagrams, click through links
to more specific pages with detail, etc.
I'm replying to the github issue to see if the user who complained about the
documentation and RainerScript would find this more useful, but also to
rsyslog-users to get feedback from others on this.
some of the sections here should possibly be broken into sub-pages (some
sub-pages already exist that cover some of these and can/should be simplified),
or it make make sense to have a simple version on an overview page with the
ability to click down for the gory details.
David Lang
Rsyslog architecture is very straightforward, but in it's simplicity it hides a lot of flexibility.
Rsyslog has one or more inputs that each receive one or more messages and pass the batch of messages to a ruleset
Each input runs the incoming log through a stack of possible parser modules
until it hits one that reports success in parsing the log (pointer to parser
module documentation and the default stack)
Multiple inputs can feed to the same ruleset (by default, all inputs feed to
the Default ruleset which uses the 'main' queue) [1]
Worker threads pull batches of logs from a queue, then process the logs in the
batch using the statements in a ruleset
Conceptually, it really is that trivial. As always, looking at details makes it
seem more complicated.
Rsyslog config file(s)
Rsyslog reads in the config file and all included files and combines them before
evaluating anything (see -o option for how files are combined), which file a
statement is in has no impact (other than as part of the ordering of
statements).
(insert link to Rainer's recent post on mis-use of config includes??)
At startup time, Rsyslog evaluates the combined config and implements all module
loading, input definition, template definitions and other global settings.
All other statements get put into the default ruleset unless a ruleset is
specified. None of these statements are evaluated (beyond syntax checking) at
startup.
The Rsyslog team believes very strongly in maintaining backwards compatibility
(a config that works should never break or change behavior when rsyslog is
updated to a new version) as such there are multiple ways of doing the same
thing, and some ways are no longer recommended. When you see that something
is depriciated, that means it is recommended not to use it in a new
configuration for confusion/feature reasons, not that it is scheduled to go
away/break in a new version.
The config statments that existed prior to v6 of rsyslog were an evolution of
the syslog format from the 90's, doing complex things by setting a bunch of
values that then got used by later statements. By v5 of rsyslog, this was
resulting in such complex interactions that even core developers were having
trouble understanding what complex configs did. V6 introduced RainerScript,
which deliberately requires you to specify all options rather than 'inheriting'
them from prior statements. This can be significantly more verbose as it
requires you to specify all options each time, but makes it much clearer exactly
what is happening. There are times when the old syntax is shorter and more
obvious to use than the new syntax, and in those cases, it's recommended to use
the old syntax. But if the old syntax requires multiple lines to do something,
you are probably better off using the new syntax.
Rulesets are the heart of log processing, defining what happens with each log
message. The statements in a ruleset are evaluated for every log message as it
is processed.
Rulesets and Actions can have a queue defined for them (insert link to queue
turn lane post, possibnly with updates). The 'default' ruleset uses the 'main'
queue.
The contents of a ruleset are a series of statements, which can be:
1. call an action to use an output module
1a. legacy formats:
/var/log/messages (write to a file)
@1.2.3.4 (send to a remote system via UDP
@@1.2.3.4 (send to a remote system via TCP)
1b. action() format
2. set/clear variables (link to functions)
3. call a message modification module (can modify the log message being processed and set variables) commonly used to parse messages
4. call another ruleset and then retur
5. statement block
{ statement statement }
usually used after a filter to have the filter apply to multiple statements
& <statememt>
apply the last filter to this statement [2]
6. stop processing this log message
6a. ~ [2]
6b. stop
ignore all following statements in the ruleset. Rsyslog will warn you if you have statements after an unconditional stop
7. apply a filter
7a. legacy syslog facility.serverity filters
i.e. mail.info /var/log/mail
7b. rsyslog property based filters [2]
i.e. :msg, contains, "foo" <statemnent>
7c. expression based filters (if..then..else with continue) (link to
functions and conditionals)
i.d. if $msg contains("foo") then <statement>
8. atomic stats update (see impstats module)
9. foreach execute a block of statements on each value in a variable array
Variable types:
built-in/legacy properties start with $ or $$ (link to property page)
user modifyable variables exist as a tree internally represented as a json structure. There are three trees that can be used:
"normal variables" start with $!
"local variables" start with $. (exist so that you can include all $! variables in a template without including everything)
"global variables" start with $\ (persist past the log message where they were set, performance pigs)
Templates are used by output modules. They are used to create larger strings that use variable values for use with the module. These allow you to change the format of the output, what file or database table the log gets written to and other similar things. The details of what the result of the template means varies from one module to another. There is a common misconception that a template can be used to match and parse a log message. Templates are output only.
[1] it may make sense to have a 'are you sure' message at startup about inputs
that feed to rulesets that don't have a queue defined for the ruleset. I don't
think a new warning would be a breaking change
[2] supported for backwards compatibility, use is discouraged
(note, there should possibly be two versions of this, one showing the
straighforward, single-message-at-a-time process, and a second one that shows
the advanced, batch supporting, version that includes showing where locking
happens, the atomic stats and foreach would be in the advanced version)
On Wed, 1 Nov 2023, computerquip-work wrote:
> The documentation is so poor that it's almost unusable. Using RainerScript is hands down the most painful thing I've ever used in software.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
a lot of detail, but it's all written for someone already fairly familiar
with things.
Here is a 3am first pass from me at writing an overview of how rsyslog works,
with the idea that this could be made pretty with diagrams, click through links
to more specific pages with detail, etc.
I'm replying to the github issue to see if the user who complained about the
documentation and RainerScript would find this more useful, but also to
rsyslog-users to get feedback from others on this.
some of the sections here should possibly be broken into sub-pages (some
sub-pages already exist that cover some of these and can/should be simplified),
or it make make sense to have a simple version on an overview page with the
ability to click down for the gory details.
David Lang
Rsyslog architecture is very straightforward, but in it's simplicity it hides a lot of flexibility.
Rsyslog has one or more inputs that each receive one or more messages and pass the batch of messages to a ruleset
Each input runs the incoming log through a stack of possible parser modules
until it hits one that reports success in parsing the log (pointer to parser
module documentation and the default stack)
Multiple inputs can feed to the same ruleset (by default, all inputs feed to
the Default ruleset which uses the 'main' queue) [1]
Worker threads pull batches of logs from a queue, then process the logs in the
batch using the statements in a ruleset
Conceptually, it really is that trivial. As always, looking at details makes it
seem more complicated.
Rsyslog config file(s)
Rsyslog reads in the config file and all included files and combines them before
evaluating anything (see -o option for how files are combined), which file a
statement is in has no impact (other than as part of the ordering of
statements).
(insert link to Rainer's recent post on mis-use of config includes??)
At startup time, Rsyslog evaluates the combined config and implements all module
loading, input definition, template definitions and other global settings.
All other statements get put into the default ruleset unless a ruleset is
specified. None of these statements are evaluated (beyond syntax checking) at
startup.
The Rsyslog team believes very strongly in maintaining backwards compatibility
(a config that works should never break or change behavior when rsyslog is
updated to a new version) as such there are multiple ways of doing the same
thing, and some ways are no longer recommended. When you see that something
is depriciated, that means it is recommended not to use it in a new
configuration for confusion/feature reasons, not that it is scheduled to go
away/break in a new version.
The config statments that existed prior to v6 of rsyslog were an evolution of
the syslog format from the 90's, doing complex things by setting a bunch of
values that then got used by later statements. By v5 of rsyslog, this was
resulting in such complex interactions that even core developers were having
trouble understanding what complex configs did. V6 introduced RainerScript,
which deliberately requires you to specify all options rather than 'inheriting'
them from prior statements. This can be significantly more verbose as it
requires you to specify all options each time, but makes it much clearer exactly
what is happening. There are times when the old syntax is shorter and more
obvious to use than the new syntax, and in those cases, it's recommended to use
the old syntax. But if the old syntax requires multiple lines to do something,
you are probably better off using the new syntax.
Rulesets are the heart of log processing, defining what happens with each log
message. The statements in a ruleset are evaluated for every log message as it
is processed.
Rulesets and Actions can have a queue defined for them (insert link to queue
turn lane post, possibnly with updates). The 'default' ruleset uses the 'main'
queue.
The contents of a ruleset are a series of statements, which can be:
1. call an action to use an output module
1a. legacy formats:
/var/log/messages (write to a file)
@1.2.3.4 (send to a remote system via UDP
@@1.2.3.4 (send to a remote system via TCP)
1b. action() format
2. set/clear variables (link to functions)
3. call a message modification module (can modify the log message being processed and set variables) commonly used to parse messages
4. call another ruleset and then retur
5. statement block
{ statement statement }
usually used after a filter to have the filter apply to multiple statements
& <statememt>
apply the last filter to this statement [2]
6. stop processing this log message
6a. ~ [2]
6b. stop
ignore all following statements in the ruleset. Rsyslog will warn you if you have statements after an unconditional stop
7. apply a filter
7a. legacy syslog facility.serverity filters
i.e. mail.info /var/log/mail
7b. rsyslog property based filters [2]
i.e. :msg, contains, "foo" <statemnent>
7c. expression based filters (if..then..else with continue) (link to
functions and conditionals)
i.d. if $msg contains("foo") then <statement>
8. atomic stats update (see impstats module)
9. foreach execute a block of statements on each value in a variable array
Variable types:
built-in/legacy properties start with $ or $$ (link to property page)
user modifyable variables exist as a tree internally represented as a json structure. There are three trees that can be used:
"normal variables" start with $!
"local variables" start with $. (exist so that you can include all $! variables in a template without including everything)
"global variables" start with $\ (persist past the log message where they were set, performance pigs)
Templates are used by output modules. They are used to create larger strings that use variable values for use with the module. These allow you to change the format of the output, what file or database table the log gets written to and other similar things. The details of what the result of the template means varies from one module to another. There is a common misconception that a template can be used to match and parse a log message. Templates are output only.
[1] it may make sense to have a 'are you sure' message at startup about inputs
that feed to rulesets that don't have a queue defined for the ruleset. I don't
think a new warning would be a breaking change
[2] supported for backwards compatibility, use is discouraged
(note, there should possibly be two versions of this, one showing the
straighforward, single-message-at-a-time process, and a second one that shows
the advanced, batch supporting, version that includes showing where locking
happens, the atomic stats and foreach would be in the advanced version)
On Wed, 1 Nov 2023, computerquip-work wrote:
> The documentation is so poor that it's almost unusable. Using RainerScript is hands down the most painful thing I've ever used in software.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.