Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. RSyslog>
  3. users
R: Re: received json log format
rsyslog at lists
Mar 1, 2023, 7:44 AM
Post #1 of 2 (124 views)
Permalink
Hi David
til the vendor have't changed the log ouput format I received the logs
with this:

# Add this to reset the umask#
$umask 0000

#BEGIN TEMPLATE

#template to add source ip
template( name="addFrmHstUDP" type="list")
{
property( name="fromhost-ip" )
constant( value=" " )
property( name="msg" )
constant( value="\n" )
}

#template set log filename
template( name="rmt_sys_netlogs_udp" type="list")
{
property( name="$.mstoredir" )
constant( value="/" )
property( name="$.mprefix" )
constant( value="." )
property( name="timegenerated" dateformat="year" )
property( name="timegenerated" dateformat="month" )
property( name="timegenerated" dateformat="day" )
property( name="timegenerated" dateformat="hour" )
constant( value="00" )
}
#END TEMPLATE

ruleset( name="udp-netlogs-tool"){
reset $.mstoredir = "/data/logs;
action( name="logs-udp"
type="omfile"
DynaFile="rmt_sys_netlogs_udp"
DirOwner="root"
DirGroup="netlogs"
FileOwner="root"
FileGroup="netlogs"
DirCreateMode="0750"
FileCreateMode="0640"
template="addFrmHstUDP"
closeTimeout="2"
dynaFileCacheSize="2"
)
}

ruleset( name="dp65309udp" )
{
reset $.mprefix = "public2;
call udp-netlogs-tool
}

Now I receive the log that I have show you.

Cheers,
Maurizio
------ Messaggio Originale ------
Da: david@lang.hm
A: rsyslog@lists.adiscon.com
Cc: amaury@tin.it
Inviato: mercoledì 1 marzo 2023 14:10
Oggetto: Re: [rsyslog] received json log format

please post your config so we can understand if what youa re
showing us is the result of your config or what is being sent to you.

If it's what is being sent to you, you would use mmnormalize to parse
it into variables, then create a custom template to assemble the message
format that you want to write out and then output the message with your
template

David Lang

On Wed, 1 Mar 2023, amaury--- via rsyslog wrote:

> Date: Wed, 1 Mar 2023 14:01:49 +0100 (CET)
> From: amaury--- via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog@lists.adiscon.com
> Cc: "amaury@tin.it"
> Subject: [rsyslog] received json log format
>
> Hello
> I receive on rsyslog-8.2102 log json format like this:
> LogRecord {id='null', date=1677669932610,
> applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
> domainName='public', serverIp='null', serverPort=null,
clientIp='null',
> clientPort=null, sessionId='null', username='null',
clientRequest='null',
> clientMessage='null', serverStatus='trigger completed',
> serverMessage='trigger=move document;
> eventId=8a8d22d9-fd5c-451b-817b-699c706db5d6', inboundBytes=null,
> outboundBytes=null}
> LogRecord {id='null', date=1677669932644,
> applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
> domainName='public', serverIp='ddd.ddd.ddd.ddd', serverPort=hhhh,
> clientIp='kkk.kkk.kkk.kkk', clientPort=9999,
> sessionId='134e7eed-af8b-48a5-bd7c-0cb48013dfda', username='user01',
> clientRequest='null', clientMessage='null', serverStatus='logged
out',
> serverMessage='null', inboundBytes=null, outboundBytes=null}
> LogRecord {id='null', date=1677669932645,
> applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
> domainName='public', serverIp='ddd.ddd.ddd.ddd', serverPort=hhhh,
> clientIp='kkk.kkk.kkk.kkk', clientPort=9999,
> sessionId='134e7eed-af8b-48a5-bd7c-0cb48013dfda', username='user01',
> clientRequest='null', clientMessage='null', serverStatus='session
closed',
> serverMessage='SFTP/SCP', inboundBytes=null, outboundBytes=null}
>
> Please how I can to convert rewrite in log file something like
> null 1677669932610 5fc42f05-36ab-45ff-908d-e7b978a88269 public null
null null
> null null null null trigger completed trigger=move document
> 8a8d22d9-fd5c-451b-817b-699c706db5d6 null null
> ?
> Thank you
> Maurizio
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T
> LIKE THAT.
>




_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: R: Re: received json log format [ In reply to ]
rsyslog at lists
Mar 1, 2023, 10:00 AM
Post #2 of 2 (124 views)
Permalink
Ok, in that case you will need to use mmnormalize to parse the log lines, and
then a template to use the variables that you parse.

the json type will be your friend for parsing the json, then log the line with
the RSYSLOG_DebugFormat to see all the variables it creates and use them
($!foo!bar for multi-level json structures) and then use those variables in a
template.

David Lang

On Wed, 1 Mar 2023, amaury--- via rsyslog wrote:

> Date: Wed, 1 Mar 2023 16:44:20 +0100 (CET)
> From: amaury--- via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog@lists.adiscon.com
> Cc: "amaury@tin.it" <amaury@tin.it>
> Subject: [rsyslog] R: Re: received json log format
>
> Hi David
> til the vendor have't changed the log ouput format I received the logs
> with this:
>
> # Add this to reset the umask#
> $umask 0000
>
> #BEGIN TEMPLATE
>
> #template to add source ip
> template( name="addFrmHstUDP" type="list")
> {
> property( name="fromhost-ip" )
> constant( value=" " )
> property( name="msg" )
> constant( value="\n" )
> }
>
> #template set log filename
> template( name="rmt_sys_netlogs_udp" type="list")
> {
> property( name="$.mstoredir" )
> constant( value="/" )
> property( name="$.mprefix" )
> constant( value="." )
> property( name="timegenerated" dateformat="year" )
> property( name="timegenerated" dateformat="month" )
> property( name="timegenerated" dateformat="day" )
> property( name="timegenerated" dateformat="hour" )
> constant( value="00" )
> }
> #END TEMPLATE
>
> ruleset( name="udp-netlogs-tool"){
> reset $.mstoredir = "/data/logs;
> action( name="logs-udp"
> type="omfile"
> DynaFile="rmt_sys_netlogs_udp"
> DirOwner="root"
> DirGroup="netlogs"
> FileOwner="root"
> FileGroup="netlogs"
> DirCreateMode="0750"
> FileCreateMode="0640"
> template="addFrmHstUDP"
> closeTimeout="2"
> dynaFileCacheSize="2"
> )
> }
>
> ruleset( name="dp65309udp" )
> {
> reset $.mprefix = "public2;
> call udp-netlogs-tool
> }
>
> Now I receive the log that I have show you.
>
> Cheers,
> Maurizio
> ------ Messaggio Originale ------
> Da: david@lang.hm
> A: rsyslog@lists.adiscon.com
> Cc: amaury@tin.it
> Inviato: mercoledì 1 marzo 2023 14:10
> Oggetto: Re: [rsyslog] received json log format
>
> please post your config so we can understand if what youa re
> showing us is the result of your config or what is being sent to you.
>
> If it's what is being sent to you, you would use mmnormalize to parse
> it into variables, then create a custom template to assemble the message
> format that you want to write out and then output the message with your
> template
>
> David Lang
>
> On Wed, 1 Mar 2023, amaury--- via rsyslog wrote:
>
> > Date: Wed, 1 Mar 2023 14:01:49 +0100 (CET)
> > From: amaury--- via rsyslog <rsyslog@lists.adiscon.com>
> > To: rsyslog@lists.adiscon.com
> > Cc: "amaury@tin.it"
> > Subject: [rsyslog] received json log format
> >
> > Hello
> > I receive on rsyslog-8.2102 log json format like this:
> > LogRecord {id='null', date=1677669932610,
> > applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
> > domainName='public', serverIp='null', serverPort=null,
> clientIp='null',
> > clientPort=null, sessionId='null', username='null',
> clientRequest='null',
> > clientMessage='null', serverStatus='trigger completed',
> > serverMessage='trigger=move document;
> > eventId=8a8d22d9-fd5c-451b-817b-699c706db5d6', inboundBytes=null,
> > outboundBytes=null}
> > LogRecord {id='null', date=1677669932644,
> > applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
> > domainName='public', serverIp='ddd.ddd.ddd.ddd', serverPort=hhhh,
> > clientIp='kkk.kkk.kkk.kkk', clientPort=9999,
> > sessionId='134e7eed-af8b-48a5-bd7c-0cb48013dfda', username='user01',
> > clientRequest='null', clientMessage='null', serverStatus='logged
> out',
> > serverMessage='null', inboundBytes=null, outboundBytes=null}
> > LogRecord {id='null', date=1677669932645,
> > applicationInstanceId='5fc42f05-36ab-45ff-908d-e7b978a88269',
> > domainName='public', serverIp='ddd.ddd.ddd.ddd', serverPort=hhhh,
> > clientIp='kkk.kkk.kkk.kkk', clientPort=9999,
> > sessionId='134e7eed-af8b-48a5-bd7c-0cb48013dfda', username='user01',
> > clientRequest='null', clientMessage='null', serverStatus='session
> closed',
> > serverMessage='SFTP/SCP', inboundBytes=null, outboundBytes=null}
> >
> > Please how I can to convert rewrite in log file something like
> > null 1677669932610 5fc42f05-36ab-45ff-908d-e7b978a88269 public null
> null null
> > null null null null trigger completed trigger=move document
> > 8a8d22d9-fd5c-451b-817b-699c706db5d6 null null
> > ?
> > Thank you
> > Maurizio
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad of
> > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T
> > LIKE THAT.
> >
>
>
>
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal