list members,
i am looking to separate the PID from the SysLogTag in all messages, and
have separate fields in the LogAnalyser database. i have several hosts
that send messages via RELP to a central cluster of receivers, and those
cluster members then submit the messages into MariaDB.
as i understand it, i need to:
1) modify each source hosts messages to split out the PID from the
SysLogTag field
2) modify the central clusters inserts to MariaDB, to specify the PID
independent of the SysLogTag
3) modify the view used in LogAnalyzer to show the PID column
for the first bullet, my typical syslog source has the following config:
global(workDirectory="/var/lib/rsyslog")
module(load="builtin:omfile" Template="RSYSLOG_TraditionalFileFormat")
module(load="imuxsock"
SysSock.Use="off")
module(load="imjournal"
StateFile="imjournal.state")
include(file="/etc/rsyslog.d/*.conf" mode="optional")
module(load="omrelp") # needs to be done just once
action(type="omrelp" target="relp.bpk2.com" port="20514")
&~
i believe i need to create my own Template, and specify that instead of
"RSYSLOG_TraditionalFileFormat". Being that the source hosts do not
directly insert the messages into the database, i dont need to specify
an insert at this point. i believe RSYSLOG_TraditionalFileFormat has
the following fields:
$template TraditionalFileFormat,"%TIMESTAMP% %HOSTNAME%
%syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
what fields do i need to change, to use the raw SysLogTag, and PID as
separate fields to send to the central receivers?
for the second bullet, i insert messages to the database using the
following config:
global(workDirectory="/var/lib/rsyslog")
module(load="builtin:omfile" Template="RSYSLOG_TraditionalFileFormat")
include(file="/etc/rsyslog.d/*.conf" mode="optional")
module(load="imuxsock"
SysSock.Use="off")
module(load="imjournal"
StateFile="imjournal.state")
module(load="imudp")
input(type="imudp" port="514")
module(load="imtcp")
input(type="imtcp" port="514")
module(load="imrelp")
input(type="imrelp" port="20514")
module(load="ommysql")
action(type="ommysql" server="database.domain.tld" serverport="3306"
db="Syslog" uid="" pwd="")
&~
i figure i need to specify another template here, and use that for the
inserts. i found the following, and want to sanity check it, as i dont
know alot about RegEx, which seems to be used:
$template dbFormat,"insert into SystemEvents (Message, Facility,
FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID,
SysLogTag, processid) values ('%msg%', %syslogfacility%,
'%HOSTNAME%',%syslogpriority%, '%timereported:::date-mysql%',
'%timegenerated:::date-mysql%', %iut%,
'%syslogtag:R,ERE,1,FIELD:(.+)(\[[0-9]{1,10}\]).*--end%',
'%syslogtag:R,ERE,1,BLANK:\[([0-9]{1,10})\]--end%')",sql
with the above Template, called dbFormat, i would modify the "action" of
my ommysql module, to be:
...
action(type="ommysql" server="database.domain.tld" serverport="3306"
db="Syslog" uid="" pwd="" template="dbFormat")
am i tracking properly on this? the database does have the processid
field defined already, so it seems that i just need to populate that field.
for the third bullet, i have installed the helper pieces for the
phpLogCon/LogAnalyzer, so i just need to update the configs in the
database to show the ProcessID field in the view i defined and set as
the default. again, am i thinking about this in the right terms? are
there any steps i am overlooking?
thanks in advance,
brendan kearney
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
i am looking to separate the PID from the SysLogTag in all messages, and
have separate fields in the LogAnalyser database. i have several hosts
that send messages via RELP to a central cluster of receivers, and those
cluster members then submit the messages into MariaDB.
as i understand it, i need to:
1) modify each source hosts messages to split out the PID from the
SysLogTag field
2) modify the central clusters inserts to MariaDB, to specify the PID
independent of the SysLogTag
3) modify the view used in LogAnalyzer to show the PID column
for the first bullet, my typical syslog source has the following config:
global(workDirectory="/var/lib/rsyslog")
module(load="builtin:omfile" Template="RSYSLOG_TraditionalFileFormat")
module(load="imuxsock"
SysSock.Use="off")
module(load="imjournal"
StateFile="imjournal.state")
include(file="/etc/rsyslog.d/*.conf" mode="optional")
module(load="omrelp") # needs to be done just once
action(type="omrelp" target="relp.bpk2.com" port="20514")
&~
i believe i need to create my own Template, and specify that instead of
"RSYSLOG_TraditionalFileFormat". Being that the source hosts do not
directly insert the messages into the database, i dont need to specify
an insert at this point. i believe RSYSLOG_TraditionalFileFormat has
the following fields:
$template TraditionalFileFormat,"%TIMESTAMP% %HOSTNAME%
%syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
what fields do i need to change, to use the raw SysLogTag, and PID as
separate fields to send to the central receivers?
for the second bullet, i insert messages to the database using the
following config:
global(workDirectory="/var/lib/rsyslog")
module(load="builtin:omfile" Template="RSYSLOG_TraditionalFileFormat")
include(file="/etc/rsyslog.d/*.conf" mode="optional")
module(load="imuxsock"
SysSock.Use="off")
module(load="imjournal"
StateFile="imjournal.state")
module(load="imudp")
input(type="imudp" port="514")
module(load="imtcp")
input(type="imtcp" port="514")
module(load="imrelp")
input(type="imrelp" port="20514")
module(load="ommysql")
action(type="ommysql" server="database.domain.tld" serverport="3306"
db="Syslog" uid="" pwd="")
&~
i figure i need to specify another template here, and use that for the
inserts. i found the following, and want to sanity check it, as i dont
know alot about RegEx, which seems to be used:
$template dbFormat,"insert into SystemEvents (Message, Facility,
FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID,
SysLogTag, processid) values ('%msg%', %syslogfacility%,
'%HOSTNAME%',%syslogpriority%, '%timereported:::date-mysql%',
'%timegenerated:::date-mysql%', %iut%,
'%syslogtag:R,ERE,1,FIELD:(.+)(\[[0-9]{1,10}\]).*--end%',
'%syslogtag:R,ERE,1,BLANK:\[([0-9]{1,10})\]--end%')",sql
with the above Template, called dbFormat, i would modify the "action" of
my ommysql module, to be:
...
action(type="ommysql" server="database.domain.tld" serverport="3306"
db="Syslog" uid="" pwd="" template="dbFormat")
am i tracking properly on this? the database does have the processid
field defined already, so it seems that i just need to populate that field.
for the third bullet, i have installed the helper pieces for the
phpLogCon/LogAnalyzer, so i just need to update the configs in the
database to show the ProcessID field in the view i defined and set as
the default. again, am i thinking about this in the right terms? are
there any steps i am overlooking?
thanks in advance,
brendan kearney
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.