Mailing List Archive: JSON parse array question

Hello Guys,

I'm facing a strange case. I send log messages to syslog, and process them
via mmjsonparse and ommongodb. When I send texts and objects, everything
works fine. But when I try to send arrays, I get objects in Mongo. Very
strange, because the stored raw message shows array yet (check the user_ids
at the end):

@cee:{"tag":"auth","message":"Sign in
success","details":{"ip":"192.168.54.6"},"user_ids":[7]}

However, I found this part in Mongo:

... "user_ids":{"0":7}

My System:
Debian 11 Bullseye, Rsyslog 8.2102.0, MongoDB v5.0.5.

The template I use:
template(name="outfmt" type="list" option.jsonf="on") {
property(outname="time" name="timereported"
dateFormat="rfc3339" format="json")
property(outname="host" name="hostname" format="json")
property(outname="severity" name="syslogseverity-text"
caseConversion="upper" format="json")
property(outname="facility" name="syslogfacility-text"
format="json")
property(outname="tag" name="$!tag" format="jsonf")
property(outname="message" name="$!message" format="jsonf")
property(outname="rawmsg" name="msg" controlcharacters="drop")
property(outname="details" name="$!details" format="jsonf")
property(outname="level" name="$!level" format="jsonf")
property(outname="level_name" name="$!level_name" format="jsonf")
property(outname="user_ids" name="$!user_ids")
property(outname="user_ids" name="$!mesg")
property(outname="extra" name="$!extra" format="jsonf")
property(outname="channel" name="$!channel" format="jsonf")
property(outname="datetime" name="$!datetime" dateFormat="rfc3339"
format="jsonf")
}
(I tried format="json", "jsonf", "csv" for the user_ids, but no effect.)

And the action is:
if ($syslogfacility-text == 'Local6') then {
action(type="mmutf8fix")
action(type="mmjsonparse")
if $parsesuccess == "OK" then {
action(type="ommongodb" server="192.168.1.11:27017" db="syslog"
collection="events" template="outfmt")
}
stop
}

Could anybody help me in this context? What should I do to get arrays
instead of objects?

Kind regards,
Gabor
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Hi Gabor,

I see that you have two mentions of user_ids in the template:

property(outname="user_ids" name="$!user_ids")
property(outname="user_ids" name="$!mesg")

Maybe that's why? I would normally expect just one instance of the same
variable.

My usual way of troubleshooting templates is to write the output to a file
(using the same template). At least this way I would know if the problem is
specific to my destination or to my template.

Best regards,
Radu
--
Elasticsearch/OpenSearch & Solr Consulting, Production Support & Training
Sematext Cloud - Full Stack Observability
https://sematext.com/ <http://sematext.com/>

On Tue, Jul 12, 2022 at 8:22 AM Gábor Takács via rsyslog <
rsyslog@lists.adiscon.com> wrote:

> Hello Guys,
>
> I'm facing a strange case. I send log messages to syslog, and process them
> via mmjsonparse and ommongodb. When I send texts and objects, everything
> works fine. But when I try to send arrays, I get objects in Mongo. Very
> strange, because the stored raw message shows array yet (check the user_ids
> at the end):
>
> @cee:{"tag":"auth","message":"Sign in
> success","details":{"ip":"192.168.54.6"},"user_ids":[7]}
>
> However, I found this part in Mongo:
>
> ... "user_ids":{"0":7}
>
> My System:
> Debian 11 Bullseye, Rsyslog 8.2102.0, MongoDB v5.0.5.
>
> The template I use:
> template(name="outfmt" type="list" option.jsonf="on") {
> property(outname="time" name="timereported"
> dateFormat="rfc3339" format="json")
> property(outname="host" name="hostname" format="json")
> property(outname="severity" name="syslogseverity-text"
> caseConversion="upper" format="json")
> property(outname="facility" name="syslogfacility-text"
> format="json")
> property(outname="tag" name="$!tag" format="jsonf")
> property(outname="message" name="$!message" format="jsonf")
> property(outname="rawmsg" name="msg" controlcharacters="drop")
> property(outname="details" name="$!details" format="jsonf")
> property(outname="level" name="$!level" format="jsonf")
> property(outname="level_name" name="$!level_name" format="jsonf")
> property(outname="user_ids" name="$!user_ids")
> property(outname="user_ids" name="$!mesg")
> property(outname="extra" name="$!extra" format="jsonf")
> property(outname="channel" name="$!channel" format="jsonf")
> property(outname="datetime" name="$!datetime" dateFormat="rfc3339"
> format="jsonf")
> }
> (I tried format="json", "jsonf", "csv" for the user_ids, but no effect.)
>
> And the action is:
> if ($syslogfacility-text == 'Local6') then {
> action(type="mmutf8fix")
> action(type="mmjsonparse")
> if $parsesuccess == "OK" then {
> action(type="ommongodb" server="192.168.1.11:27017" db="syslog"
> collection="events" template="outfmt")
> }
> stop
> }
>
> Could anybody help me in this context? What should I do to get arrays
> instead of objects?
>
> Kind regards,
> Gabor
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.