> > There are a couple of subtleties, but I think it can work. In
> essence,
> > you need a template that feeds into the values via ($template!) and
> > also a config string for the prepared statement. It's actually not
> > even that hard to do. It may be useful (and of course doable) to
> > enable the property replace to escape special characters, so that,
> for
> > example, we could use CSV and replace commas by two of them.
>
> Making properties in CSV format is indeed a good idea.
I'll add a "csv" option to the property replacer. It will format a field
according to RFC 4180:
http://tools.ietf.org/html/rfc4180
I will always enclose values in double quotes to keep the code simple (I
think also on the parser side). Let me know if you think this approach is
useful.
>
> > > I bet it works. But it's probably too ugly for users. Cleaner ways
> > > may need deeper changes into rsyslog's API so that the module gets
> > > direct access to each field. That's probably a lot of work and I
> > > can't wait for that.
>
> > I need to check if there are actually larger changes required. The
> > main reason for this interface initially was security (do not pass to
> > the module the full object).
>
> It's a good reason. If it's easy to generate and pass a deep copy of
> the
> object (and it's not a performance killer, it shouldn't), we can
> discuss
> it. Otherwise, I don't think this is worth the effort.
I have done some review of the code. Not in-depth, but I think good enough. I
do have the message object available (thankfully, the optimization to store
just the strings did not yet take place ;)). Creating a deep copy is not
problematic and I'd assume typically bears roughly equivalent cost to
creating the template string (which is kind of expensive). There obviously
must be a new interface declared for this and your plugin should support both
the new and the old interface. It may be OK if you return RS_RET_DISABLED on
the first call to the old interface, what means a downlevel engine can not
use your plugin (acceptable from my POV).
Feature-Wise, however, I think you lose a couple of things. Most importantly,
the template processor & property replacer allows you to rewrite parts of the
message. If we pass in the plain message object, you lose this ability. So
any modifications must be made directly from within the plugin. I'd say
that's a big disadvantage.
As the scripting engine evolves, we will probably be able to overcome this
limitation by permitting modifications to message objects, but that's a long
way until we are there...
> > Assuming that I have the object available
> > at the time of the plugin call, I could use a different entry point
> to
> > pass that data in. If so, that would not be too much effort. Security
> > concerns could be (somewhat) addressed by a config statement which
> > enables such object access for the next action, so one could
> > specifically grant that privilege.
>
> I'm not quite sure about this: if two entries request direct access to
> the same object, one is buggy and modifies it, then the second one can
> suffer unpredictable consequences. I think it's better to pass a deep
> copy, free it once the module call returns, and do it only for modules
> that actually need that new entry point. If such deep copies are
> expensive, then we are just fine the way we are now.
Agreed, but I think a deep copy does not address anything. With the template
system (in theory but not yet in practice), a plugin can not access any
information other than what the users has configured in the template. If the
full object is passed, this can not prevented. In practice, today, plugins
are loaded in-process and as such can access the whole process space. But
there are ideas to create an out-of-process plugin interface for very
security sensitive environments. They would be hurt (or require additional
configuration) but the "full object access" approach.
Feedback is appreciated.
Rainer
>
> Cheers.
>
>
> --
> Luis Fernando Muñoz MejÃas
> Luis.Fernando.Munoz.Mejias@cern.ch
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
> essence,
> > you need a template that feeds into the values via ($template!) and
> > also a config string for the prepared statement. It's actually not
> > even that hard to do. It may be useful (and of course doable) to
> > enable the property replace to escape special characters, so that,
> for
> > example, we could use CSV and replace commas by two of them.
>
> Making properties in CSV format is indeed a good idea.
I'll add a "csv" option to the property replacer. It will format a field
according to RFC 4180:
http://tools.ietf.org/html/rfc4180
I will always enclose values in double quotes to keep the code simple (I
think also on the parser side). Let me know if you think this approach is
useful.
>
> > > I bet it works. But it's probably too ugly for users. Cleaner ways
> > > may need deeper changes into rsyslog's API so that the module gets
> > > direct access to each field. That's probably a lot of work and I
> > > can't wait for that.
>
> > I need to check if there are actually larger changes required. The
> > main reason for this interface initially was security (do not pass to
> > the module the full object).
>
> It's a good reason. If it's easy to generate and pass a deep copy of
> the
> object (and it's not a performance killer, it shouldn't), we can
> discuss
> it. Otherwise, I don't think this is worth the effort.
I have done some review of the code. Not in-depth, but I think good enough. I
do have the message object available (thankfully, the optimization to store
just the strings did not yet take place ;)). Creating a deep copy is not
problematic and I'd assume typically bears roughly equivalent cost to
creating the template string (which is kind of expensive). There obviously
must be a new interface declared for this and your plugin should support both
the new and the old interface. It may be OK if you return RS_RET_DISABLED on
the first call to the old interface, what means a downlevel engine can not
use your plugin (acceptable from my POV).
Feature-Wise, however, I think you lose a couple of things. Most importantly,
the template processor & property replacer allows you to rewrite parts of the
message. If we pass in the plain message object, you lose this ability. So
any modifications must be made directly from within the plugin. I'd say
that's a big disadvantage.
As the scripting engine evolves, we will probably be able to overcome this
limitation by permitting modifications to message objects, but that's a long
way until we are there...
> > Assuming that I have the object available
> > at the time of the plugin call, I could use a different entry point
> to
> > pass that data in. If so, that would not be too much effort. Security
> > concerns could be (somewhat) addressed by a config statement which
> > enables such object access for the next action, so one could
> > specifically grant that privilege.
>
> I'm not quite sure about this: if two entries request direct access to
> the same object, one is buggy and modifies it, then the second one can
> suffer unpredictable consequences. I think it's better to pass a deep
> copy, free it once the module call returns, and do it only for modules
> that actually need that new entry point. If such deep copies are
> expensive, then we are just fine the way we are now.
Agreed, but I think a deep copy does not address anything. With the template
system (in theory but not yet in practice), a plugin can not access any
information other than what the users has configured in the template. If the
full object is passed, this can not prevented. In practice, today, plugins
are loaded in-process and as such can access the whole process space. But
there are ideas to create an out-of-process plugin interface for very
security sensitive environments. They would be hurt (or require additional
configuration) but the "full object access" approach.
Feedback is appreciated.
Rainer
>
> Cheers.
>
>
> --
> Luis Fernando Muñoz MejÃas
> Luis.Fernando.Munoz.Mejias@cern.ch
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com