Hi David
thank you for your suggest.
This is the debug:
Debug line with all properties:
FROMHOST: 'public.', fromhost-ip: 'xxx.xxx.xxx.xxx', HOSTNAME:
'xxx.xxx.xxx.xxx', PRI: 14,
syslogtag 'TOOL:', programname: 'TOOL', APP-NAME: 'TOOL', PROCID: '-',
MSGID: '-',
TIMESTAMP: 'Mar 2 10:10:45', STRUCTURED-DATA: '-',
msg: ' LogRecord {id='null', date=1677748245215,
applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
domainName='public', serverIp='yyy.yyy.yyy.yyy', serverPort=9999,
clientIp='94.138.173.87', clientPort=64012,
sessionId='859b383e-7c51-48cf-9365-6e67a84cd29d', username='',
clientRequest='null', clientMessage='null', serverStatus='session
started', serverMessage='SFTP/SCP', inboundBytes=null,
outboundBytes=null}'
escaped msg: ' LogRecord {id='null', date=1677748245215,
applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
domainName='public', serverIp='yyy.yyy.yyy.yyy', serverPort=9999,
clientIp='94.138.173.87', clientPort=64012,
sessionId='859b383e-7c51-48cf-9365-6e67a84cd29d', username='',
clientRequest='null', clientMessage='null', serverStatus='session
started', serverMessage='SFTP/SCP', inboundBytes=null,
outboundBytes=null}'
inputname: prt_65309udp rawmsg: '<14>Mar 2 10:10:45 xxx.xxx.xxx.xxx
TOOL: LogRecord {id='null', date=1677748245215,
applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
domainName='public', serverIp='yyy.yyy.yyy.yyy', serverPort=9999,
clientIp='94.138.173.87', clientPort=64012,
sessionId='859b383e-7c51-48cf-9365-6e67a84cd29d', username='',
clientRequest='null', clientMessage='null', serverStatus='session
started', serverMessage='SFTP/SCP', inboundBytes=null,
outboundBytes=null}'
$!:
$.:{ "mprefix": "public2", "mstoredir": "\/data\/logs" }
$/:
I have create a file rulebase.rb:
rule=:%date:date-rfc3164% %clientIp:ipv4% %clientPort:number%
With a template to print into file like this:
#template to add source ip
template( name="addFrmHstUDP" type="list")
{
property( name="fromhost-ip" )
constant( value=" " )
property( name="$!msg!date" )
constant( value=" " )
property( name="$!msg!clientIp" )
constant( value=" " )
property( name="$!msg!clientPort" )
#property( name="msg" )
constant( value="\n" )
}
but it's print only variable fromhost-ip.
Please how can I get the values?
Thank you
Cheers,
Maurizio
------ Messaggio Originale ------
Da: david@lang.hm
A: rsyslog@lists.adiscon.com
Cc: amaury@tin.it
Inviato: mercoledì 1 marzo 2023 19:00
Oggetto: Re: [rsyslog] R: Re: received json log format
Ok, in that case you will need to use mmnormalize to parse the
log lines, and then a template to use the variables that you parse.
the json type will be your friend for parsing the json, then log the
line with the RSYSLOG_DebugFormat to see all the variables it creates
and use them ($!foo!bar for multi-level json structures) and then use
those variables in a template.
David Lang
On Wed, 1 Mar 2023, amaury--- via rsyslog wrote:
> Date: Wed, 1 Mar 2023 16:44:20 +0100 (CET)
> From: amaury--- via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog@lists.adiscon.com
> Cc: "amaury@tin.it"
> Subject: [rsyslog] R: Re: received json log format
>
> Hi David
> til the vendor have't changed the log ouput format I received the
logs
> with this:
>
> # Add this to reset the umask#
> $umask 0000
>
> #BEGIN TEMPLATE
>
> #template to add source ip
> template( name="addFrmHstUDP" type="list")
> {
> property( name="fromhost-ip" )
> constant( value=" " )
> property( name="msg" )
> constant( value="\n" )
> }
>
> #template set log filename
> template( name="rmt_sys_netlogs_udp" type="list")
> {
> property( name="$.mstoredir" )
> constant( value="/" )
> property( name="$.mprefix" )
> constant( value="." )
> property( name="timegenerated" dateformat="year" )
> property( name="timegenerated" dateformat="month" )
> property( name="timegenerated" dateformat="day" )
> property( name="timegenerated" dateformat="hour" )
> constant( value="00" )
> }
> #END TEMPLATE
>
> ruleset( name="udp-netlogs-tool"){
> reset $.mstoredir = "/data/logs;
> action( name="logs-udp"
> type="omfile"
> DynaFile="rmt_sys_netlogs_udp"
> DirOwner="root"
> DirGroup="netlogs"
> FileOwner="root"
> FileGroup="netlogs"
> DirCreateMode="0750"
> FileCreateMode="0640"
> template="addFrmHstUDP"
> closeTimeout="2"
> dynaFileCacheSize="2"
> )
> }
>
> ruleset( name="dp65309udp" )
> {
> reset $.mprefix = "public2;
> call udp-netlogs-tool
> }
>
> Now I receive the log that I have show you.
>
> Cheers,
> Maurizio
> ------ Messaggio Originale ------
> Da: david@lang.hm
> A: rsyslog@lists.adiscon.com
> Cc: amaury@tin.it
> Inviato: mercoledì 1 marzo 2023 14:10
> Oggetto: Re: [rsyslog] received json log format
>
> please post your config so we can understand if what youa
re
> showing us is the result of your config or what is being sent to
you.
>
> If it's what is being sent to you, you would use mmnormalize to
parse
> it into variables, then create a custom template to assemble the
message
> format that you want to write out and then output the message with
your
> template
>
> David Lang
>
> On Wed, 1 Mar 2023, amaury--- via rsyslog wrote:
>
>> Date: Wed, 1 Mar 2023 14:01:49 +0100 (CET)
>> From: amaury--- via rsyslog <rsyslog@lists.adiscon.com>
>> To: rsyslog@lists.adiscon.com
>> Cc: "amaury@tin.it"
>> Subject: [rsyslog] received json log format
>>
>> Hello
>> I receive on rsyslog-8.2102 log json format like this:
>> LogRecord {id='null', date=1677669932610,
>> applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
>> domainName='public', serverIp='null', serverPort=null,
> clientIp='null',
>> clientPort=null, sessionId='null', username='null',
> clientRequest='null',
>> clientMessage='null', serverStatus='trigger completed',
>> serverMessage='trigger=move document;
>> eventId=8a8d22d9-fd5c-451b-817b-699c706db5d6', inboundBytes=null,
>> outboundBytes=null}
>> LogRecord {id='null', date=1677669932644,
>> applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
>> domainName='public', serverIp='ddd.ddd.ddd.ddd', serverPort=hhhh,
>> clientIp='kkk.kkk.kkk.kkk', clientPort=9999,
>> sessionId='134e7eed-af8b-48a5-bd7c-0cb48013dfda',
username='user01',
>> clientRequest='null', clientMessage='null', serverStatus='logged
> out',
>> serverMessage='null', inboundBytes=null, outboundBytes=null}
>> LogRecord {id='null', date=1677669932645,
>> applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
>> domainName='public', serverIp='ddd.ddd.ddd.ddd', serverPort=hhhh,
>> clientIp='kkk.kkk.kkk.kkk', clientPort=9999,
>> sessionId='134e7eed-af8b-48a5-bd7c-0cb48013dfda',
username='user01',
>> clientRequest='null', clientMessage='null', serverStatus='session
> closed',
>> serverMessage='SFTP/SCP', inboundBytes=null, outboundBytes=null}
>>
>> Please how I can to convert rewrite in log file something like
>> null 1677669932610 5fc42f05-36ab-45ff-908d-e7b978a88269 public null
> null null
>> null null null null trigger completed trigger=move document
>> 8a8d22d9-fd5c-451b-817b-699c706db5d6 null null
>> ?
>> Thank you
>> Maurizio
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad of
>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T
>> LIKE THAT.
>>
>
>
>
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T
> LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
thank you for your suggest.
This is the debug:
Debug line with all properties:
FROMHOST: 'public.', fromhost-ip: 'xxx.xxx.xxx.xxx', HOSTNAME:
'xxx.xxx.xxx.xxx', PRI: 14,
syslogtag 'TOOL:', programname: 'TOOL', APP-NAME: 'TOOL', PROCID: '-',
MSGID: '-',
TIMESTAMP: 'Mar 2 10:10:45', STRUCTURED-DATA: '-',
msg: ' LogRecord {id='null', date=1677748245215,
applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
domainName='public', serverIp='yyy.yyy.yyy.yyy', serverPort=9999,
clientIp='94.138.173.87', clientPort=64012,
sessionId='859b383e-7c51-48cf-9365-6e67a84cd29d', username='',
clientRequest='null', clientMessage='null', serverStatus='session
started', serverMessage='SFTP/SCP', inboundBytes=null,
outboundBytes=null}'
escaped msg: ' LogRecord {id='null', date=1677748245215,
applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
domainName='public', serverIp='yyy.yyy.yyy.yyy', serverPort=9999,
clientIp='94.138.173.87', clientPort=64012,
sessionId='859b383e-7c51-48cf-9365-6e67a84cd29d', username='',
clientRequest='null', clientMessage='null', serverStatus='session
started', serverMessage='SFTP/SCP', inboundBytes=null,
outboundBytes=null}'
inputname: prt_65309udp rawmsg: '<14>Mar 2 10:10:45 xxx.xxx.xxx.xxx
TOOL: LogRecord {id='null', date=1677748245215,
applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
domainName='public', serverIp='yyy.yyy.yyy.yyy', serverPort=9999,
clientIp='94.138.173.87', clientPort=64012,
sessionId='859b383e-7c51-48cf-9365-6e67a84cd29d', username='',
clientRequest='null', clientMessage='null', serverStatus='session
started', serverMessage='SFTP/SCP', inboundBytes=null,
outboundBytes=null}'
$!:
$.:{ "mprefix": "public2", "mstoredir": "\/data\/logs" }
$/:
I have create a file rulebase.rb:
rule=:%date:date-rfc3164% %clientIp:ipv4% %clientPort:number%
With a template to print into file like this:
#template to add source ip
template( name="addFrmHstUDP" type="list")
{
property( name="fromhost-ip" )
constant( value=" " )
property( name="$!msg!date" )
constant( value=" " )
property( name="$!msg!clientIp" )
constant( value=" " )
property( name="$!msg!clientPort" )
#property( name="msg" )
constant( value="\n" )
}
but it's print only variable fromhost-ip.
Please how can I get the values?
Thank you
Cheers,
Maurizio
------ Messaggio Originale ------
Da: david@lang.hm
A: rsyslog@lists.adiscon.com
Cc: amaury@tin.it
Inviato: mercoledì 1 marzo 2023 19:00
Oggetto: Re: [rsyslog] R: Re: received json log format
Ok, in that case you will need to use mmnormalize to parse the
log lines, and then a template to use the variables that you parse.
the json type will be your friend for parsing the json, then log the
line with the RSYSLOG_DebugFormat to see all the variables it creates
and use them ($!foo!bar for multi-level json structures) and then use
those variables in a template.
David Lang
On Wed, 1 Mar 2023, amaury--- via rsyslog wrote:
> Date: Wed, 1 Mar 2023 16:44:20 +0100 (CET)
> From: amaury--- via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog@lists.adiscon.com
> Cc: "amaury@tin.it"
> Subject: [rsyslog] R: Re: received json log format
>
> Hi David
> til the vendor have't changed the log ouput format I received the
logs
> with this:
>
> # Add this to reset the umask#
> $umask 0000
>
> #BEGIN TEMPLATE
>
> #template to add source ip
> template( name="addFrmHstUDP" type="list")
> {
> property( name="fromhost-ip" )
> constant( value=" " )
> property( name="msg" )
> constant( value="\n" )
> }
>
> #template set log filename
> template( name="rmt_sys_netlogs_udp" type="list")
> {
> property( name="$.mstoredir" )
> constant( value="/" )
> property( name="$.mprefix" )
> constant( value="." )
> property( name="timegenerated" dateformat="year" )
> property( name="timegenerated" dateformat="month" )
> property( name="timegenerated" dateformat="day" )
> property( name="timegenerated" dateformat="hour" )
> constant( value="00" )
> }
> #END TEMPLATE
>
> ruleset( name="udp-netlogs-tool"){
> reset $.mstoredir = "/data/logs;
> action( name="logs-udp"
> type="omfile"
> DynaFile="rmt_sys_netlogs_udp"
> DirOwner="root"
> DirGroup="netlogs"
> FileOwner="root"
> FileGroup="netlogs"
> DirCreateMode="0750"
> FileCreateMode="0640"
> template="addFrmHstUDP"
> closeTimeout="2"
> dynaFileCacheSize="2"
> )
> }
>
> ruleset( name="dp65309udp" )
> {
> reset $.mprefix = "public2;
> call udp-netlogs-tool
> }
>
> Now I receive the log that I have show you.
>
> Cheers,
> Maurizio
> ------ Messaggio Originale ------
> Da: david@lang.hm
> A: rsyslog@lists.adiscon.com
> Cc: amaury@tin.it
> Inviato: mercoledì 1 marzo 2023 14:10
> Oggetto: Re: [rsyslog] received json log format
>
> please post your config so we can understand if what youa
re
> showing us is the result of your config or what is being sent to
you.
>
> If it's what is being sent to you, you would use mmnormalize to
parse
> it into variables, then create a custom template to assemble the
message
> format that you want to write out and then output the message with
your
> template
>
> David Lang
>
> On Wed, 1 Mar 2023, amaury--- via rsyslog wrote:
>
>> Date: Wed, 1 Mar 2023 14:01:49 +0100 (CET)
>> From: amaury--- via rsyslog <rsyslog@lists.adiscon.com>
>> To: rsyslog@lists.adiscon.com
>> Cc: "amaury@tin.it"
>> Subject: [rsyslog] received json log format
>>
>> Hello
>> I receive on rsyslog-8.2102 log json format like this:
>> LogRecord {id='null', date=1677669932610,
>> applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
>> domainName='public', serverIp='null', serverPort=null,
> clientIp='null',
>> clientPort=null, sessionId='null', username='null',
> clientRequest='null',
>> clientMessage='null', serverStatus='trigger completed',
>> serverMessage='trigger=move document;
>> eventId=8a8d22d9-fd5c-451b-817b-699c706db5d6', inboundBytes=null,
>> outboundBytes=null}
>> LogRecord {id='null', date=1677669932644,
>> applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
>> domainName='public', serverIp='ddd.ddd.ddd.ddd', serverPort=hhhh,
>> clientIp='kkk.kkk.kkk.kkk', clientPort=9999,
>> sessionId='134e7eed-af8b-48a5-bd7c-0cb48013dfda',
username='user01',
>> clientRequest='null', clientMessage='null', serverStatus='logged
> out',
>> serverMessage='null', inboundBytes=null, outboundBytes=null}
>> LogRecord {id='null', date=1677669932645,
>> applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
>> domainName='public', serverIp='ddd.ddd.ddd.ddd', serverPort=hhhh,
>> clientIp='kkk.kkk.kkk.kkk', clientPort=9999,
>> sessionId='134e7eed-af8b-48a5-bd7c-0cb48013dfda',
username='user01',
>> clientRequest='null', clientMessage='null', serverStatus='session
> closed',
>> serverMessage='SFTP/SCP', inboundBytes=null, outboundBytes=null}
>>
>> Please how I can to convert rewrite in log file something like
>> null 1677669932610 5fc42f05-36ab-45ff-908d-e7b978a88269 public null
> null null
>> null null null null trigger completed trigger=move document
>> 8a8d22d9-fd5c-451b-817b-699c706db5d6 null null
>> ?
>> Thank you
>> Maurizio
>> _______________________________________________
>> rsyslog mailing list
>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad of
>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T
>> LIKE THAT.
>>
>
>
>
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T
> LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.