Hi all,
We are trying to use mmnormalize to process incoming CEF messages from a
fortigate firewall. Unfortunately the fortigate doesn't seem to include a
tag and so the 'CEF:' component of the message actually gets parsed as the
tag. This means that mmnormalize is then unable to parse the CEF message.
Here is an example incoming message:
Oct 21 03:07:16 CassiniTest
CEF:0|Fortinet|Fortigate|v6.4.2|40704|event:system
perf-stats|3|deviceExternalId=FGTAWS0-FPMC6A76 FTNTFGTlogid=0100040704
cat=event:system FTNTFGTsubtype=system FTNTFGTlevel=notice FTNTFGTvd=root
FTNTFGTeventtime=1603274837221415347 FTNTFGTtz=-0700 FTNTFGTlogdesc=System
performance statistics act=perf-stats FTNTFGTcpu=0 FTNTFGTmem=43
FTNTFGTtotalsession=12 FTNTFGTdisk=1 FTNTFGTbandwidth=1/1
FTNTFGTsetuprate=0 FTNTFGTdisklograte=0 FTNTFGTfazlograte=0
FTNTFGTfreediskstorage=28514 FTNTFGTsysuptime=10525 FTNTFGTwaninfo=N/A
msg=Performance statistics: average CPU: 0, memory: 43, concurrent
sessions: 12, setup-rate: 0
Here is one that I have modified by hand that works - I just added the "-"
character to indicate a nil tag value:
Oct 21 03:07:16 CassiniTest -
CEF:0|Fortinet|Fortigate|v6.4.2|40704|event:system
perf-stats|3|deviceExternalId=FGTAWS0-FPMC6A76 FTNTFGTlogid=0100040704
cat=event:system FTNTFGTsubtype=system FTNTFGTlevel=notice FTNTFGTvd=root
FTNTFGTeventtime=1603274837221415347 FTNTFGTtz=-0700 FTNTFGTlogdesc=System
performance statistics act=perf-stats FTNTFGTcpu=0 FTNTFGTmem=43
FTNTFGTtotalsession=12 FTNTFGTdisk=1 FTNTFGTbandwidth=1/1
FTNTFGTsetuprate=0 FTNTFGTdisklograte=0 FTNTFGTfazlograte=0
FTNTFGTfreediskstorage=28514 FTNTFGTsysuptime=10525 FTNTFGTwaninfo=N/A
msg=Performance statistics: average CPU: 0, memory: 43, concurrent
sessions: 12, setup-rate: 0
This is what my test config looks like:
module(load="mmnormalize")
template(name="csv" type="list") {
# syslog properties
property(name="timegenerated")
constant(value=",")
property(name="hostname")
# CSV
constant(value=",")
property(name="$!cef!DeviceVendor" format="csv")
constant(value=",")
property(name="$!cef!DeviceProduct" format="csv")
constant(value=",")
property(name="$!cef!DeviceVersion" format="csv")
constant(value=",")
property(name="$!cef!SignatureID" format="csv")
constant(value=",")
property(name="$!cef!Name" format="csv")
constant(value=",")
property(name="$!cef!Severity" format="csv")
constant(value=",")
property(name="$!cef!Extensions!act" format="csv")
constant(value="\n")
}
ruleset(name="cef_ruleset") {
action(type="mmnormalize" ruleBase="/etc/rsyslog.d/cef.rb" useRawMsg="off")
action(type="omfile" dirCreateMode="0700" FileCreateMode="0644"
File="/var/log/fortigate/parsed.log" template="csv")
}
input(type="imtcp" port="514" ruleset="cef_ruleset")
Is there an action I can use to add the missing tag field? I did look at
the mmtaghostname module but it's not in the ubuntu or centos builds (and
looks like it won't be added).
Many thanks,
James
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
We are trying to use mmnormalize to process incoming CEF messages from a
fortigate firewall. Unfortunately the fortigate doesn't seem to include a
tag and so the 'CEF:' component of the message actually gets parsed as the
tag. This means that mmnormalize is then unable to parse the CEF message.
Here is an example incoming message:
Oct 21 03:07:16 CassiniTest
CEF:0|Fortinet|Fortigate|v6.4.2|40704|event:system
perf-stats|3|deviceExternalId=FGTAWS0-FPMC6A76 FTNTFGTlogid=0100040704
cat=event:system FTNTFGTsubtype=system FTNTFGTlevel=notice FTNTFGTvd=root
FTNTFGTeventtime=1603274837221415347 FTNTFGTtz=-0700 FTNTFGTlogdesc=System
performance statistics act=perf-stats FTNTFGTcpu=0 FTNTFGTmem=43
FTNTFGTtotalsession=12 FTNTFGTdisk=1 FTNTFGTbandwidth=1/1
FTNTFGTsetuprate=0 FTNTFGTdisklograte=0 FTNTFGTfazlograte=0
FTNTFGTfreediskstorage=28514 FTNTFGTsysuptime=10525 FTNTFGTwaninfo=N/A
msg=Performance statistics: average CPU: 0, memory: 43, concurrent
sessions: 12, setup-rate: 0
Here is one that I have modified by hand that works - I just added the "-"
character to indicate a nil tag value:
Oct 21 03:07:16 CassiniTest -
CEF:0|Fortinet|Fortigate|v6.4.2|40704|event:system
perf-stats|3|deviceExternalId=FGTAWS0-FPMC6A76 FTNTFGTlogid=0100040704
cat=event:system FTNTFGTsubtype=system FTNTFGTlevel=notice FTNTFGTvd=root
FTNTFGTeventtime=1603274837221415347 FTNTFGTtz=-0700 FTNTFGTlogdesc=System
performance statistics act=perf-stats FTNTFGTcpu=0 FTNTFGTmem=43
FTNTFGTtotalsession=12 FTNTFGTdisk=1 FTNTFGTbandwidth=1/1
FTNTFGTsetuprate=0 FTNTFGTdisklograte=0 FTNTFGTfazlograte=0
FTNTFGTfreediskstorage=28514 FTNTFGTsysuptime=10525 FTNTFGTwaninfo=N/A
msg=Performance statistics: average CPU: 0, memory: 43, concurrent
sessions: 12, setup-rate: 0
This is what my test config looks like:
module(load="mmnormalize")
template(name="csv" type="list") {
# syslog properties
property(name="timegenerated")
constant(value=",")
property(name="hostname")
# CSV
constant(value=",")
property(name="$!cef!DeviceVendor" format="csv")
constant(value=",")
property(name="$!cef!DeviceProduct" format="csv")
constant(value=",")
property(name="$!cef!DeviceVersion" format="csv")
constant(value=",")
property(name="$!cef!SignatureID" format="csv")
constant(value=",")
property(name="$!cef!Name" format="csv")
constant(value=",")
property(name="$!cef!Severity" format="csv")
constant(value=",")
property(name="$!cef!Extensions!act" format="csv")
constant(value="\n")
}
ruleset(name="cef_ruleset") {
action(type="mmnormalize" ruleBase="/etc/rsyslog.d/cef.rb" useRawMsg="off")
action(type="omfile" dirCreateMode="0700" FileCreateMode="0644"
File="/var/log/fortigate/parsed.log" template="csv")
}
input(type="imtcp" port="514" ruleset="cef_ruleset")
Is there an action I can use to add the missing tag field? I did look at
the mmtaghostname module but it's not in the ubuntu or centos builds (and
looks like it won't be added).
Many thanks,
James
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.