> -----Original Message-----
> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
> bounces@lists.adiscon.com] On Behalf Of Luis Fernando Muñoz MejÃas
> Sent: Thursday, April 02, 2009 5:34 PM
> To: rsyslog@lists.adiscon.com
> Subject: Re: [rsyslog] RFC: On rsyslog output modules and
> supportforbatchoperations
>
> El Jueves, 2 de Abril de 2009 12:03, Rainer Gerhards escribió:
> > > Making properties in CSV format is indeed a good idea.
> >
> > I'll add a "csv" option to the property replacer. It will format a
> > field according to RFC 4180:
> >
> Thanks. I'll start working on it. For handling CSVs I'm planning to use
> libmba, which is distributed with Red Hat distros. Is it OK to have
> such
> a dependency on a module?
That's the module author's decison. It just should be documented. I don't
think it is any problem at all. I think it is available on all major
platforms, isn't it?
I am more strict with the core, core dependencies need to be loaded on every
(actually!) system that uses rsyslog as default syslogd (think Fedore, Debian
and variants). That would be much more of a problem.
>
> > I will always enclose values in double quotes to keep the code simple
> > (I think also on the parser side). Let me know if you think this
> > approach is useful.
>
> Looks useful. I'm starting to play with it.
>
> > I have done some review of the code. Not in-depth, but I think good
> > enough. I do have the message object available (thankfully, the
> > optimization to store just the strings did not yet take place
> > ;)). Creating a deep copy is not problematic and I'd assume typically
> > bears roughly equivalent cost to creating the template string (which
> > is kind of expensive). There obviously must be a new interface
> > declared for this and your plugin should support both the new and the
> > old interface. It may be OK if you return RS_RET_DISABLED on the
> first
> > call to the old interface, what means a downlevel engine can not use
> > your plugin (acceptable from my POV).
> >
> Sounds acceptable.
>
> > Feature-Wise, however, I think you lose a couple of things. Most
> > importantly, the template processor & property replacer allows you to
> > rewrite parts of the message. If we pass in the plain message object,
> > you lose this ability. So any modifications must be made directly
> from
> > within the plugin. I'd say that's a big disadvantage.
> >
> A huge one. If, for instance, I want to extract user name and IP
> address
> from SSH log in messages, I don't want my plugin to be aware of SSH
> messages. I prefer to have it done by rsyslog, as it is done now.
>
> If we have to make the modules extract fields on their own, we are
> duplicating existing code and introducing many new bugs. That's not
> acceptable. I wonder if, however, we can get any access to the
> properties rsyslog processes, before they are concatenated into a
> single
> string.
>
> For instance, if I specify:
>
> %fromhost%,%timestamp:::date-rfc3339%,%msg%
>
> Can I have an array of pointers to each already processed property?
I will investigate that. It just occured to me that I have the template
compiled as a linked list. It should not be too much of an effort to turn
values into a linked list rather than a string. Will check, but no promise.
>
> If this can't be done, CSV parsing is already excellent for our needs.
>
> > > I think it's better to pass a deep copy, free it once the module
> > > call returns, and do it only for modules that actually need that
> new
> > > entry point. If such deep copies are expensive, then we are just
> > > fine the way we are now.
> >
> > Agreed, but I think a deep copy does not address anything. With the
> > template system (in theory but not yet in practice), a plugin can not
> > access any information other than what the users has configured in
> the
> > template. If the full object is passed, this can not prevented.
>
> Indeed. I was thinking of the set of things I suppose you have right
> before generating the string that is passed to doAction. Anything else
> is a security problem.
>
> Again, if it's difficult or overkill, discard this idea.
It is good to have this discussion, it was dangeling for quite some while
(and I miss some participants ;)). It may turn out to help us toward a more
powerful interface for future development.
Rainer
>
> Cheers.
> --
> Luis Fernando Muñoz MejÃas
> Luis.Fernando.Munoz.Mejias@cern.ch
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com