Mailing List Archive

Michael Biebl wrote:
> the current IncludeConfig implementation has some issues, especially
> for distributers:
> All files in the directory as specified by $IncludeConfig are included.
> This means, temp files like *~ *.bak etc that are left behind by
> editors, or stuff like *.rpmsave or *.dpkg-old/*.dpkg-new.
> This is obviously unfortunate.
>
> How shall we solve this?
> 1.) I patched the Debian package to only include files ending with
> *.conf. If we document that properly this is imho an acceptable
> solution.
> 2.) Implement an internal blacklist of endings that are ignored (see above)
> 3.) Instead of hardcoding the file ending to *.conf (as in 1.), use
> the same syntax as e.g. apache, which allows to specify shell globs:
> example:
> Include /etc/apache2/conf.d/*.conf

i would opt for the third option - thou i'm not sure how much work
this creates ;)

cheers,
raoul
--
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at
Barawitzkagasse 10/2/2/11 email. office at ipax.at
1190 Wien tel. +43 1 3670030
FN 277995t HG Wien fax. +43 1 3670030 15
____________________________________________________________________

Lol... the initial message seems not to have passed my spam filter... ;)

I, too, tend to option three. The problem, though is
backwards-compatibility... Any suggestions and concerns are appreciated.

I'll look at the code to see how much work that creates. I think the
complexity is quite acceptable. I am right now in a threading redesign,
so I'd like to push this away for a little while... Anyhow, please keep
comments floating - I'll listen and drop in any thoughts I have.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX]
> Sent: Thursday, December 13, 2007 9:56 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with IncludeConfig
>
> Michael Biebl wrote:
> > the current IncludeConfig implementation has some issues, especially
> > for distributers:
> > All files in the directory as specified by $IncludeConfig are
> included.
> > This means, temp files like *~ *.bak etc that are left behind by
> > editors, or stuff like *.rpmsave or *.dpkg-old/*.dpkg-new.
> > This is obviously unfortunate.
> >
> > How shall we solve this?
> > 1.) I patched the Debian package to only include files ending with
> > *.conf. If we document that properly this is imho an acceptable
> > solution.
> > 2.) Implement an internal blacklist of endings that are ignored (see
> above)
> > 3.) Instead of hardcoding the file ending to *.conf (as in 1.), use
> > the same syntax as e.g. apache, which allows to specify shell globs:
> > example:
> > Include /etc/apache2/conf.d/*.conf
>
> i would opt for the third option - thou i'm not sure how much work
> this creates ;)
>
> cheers,
> raoul
> --
> ____________________________________________________________________
> DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at
> Technischer Leiter
>
> IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at
> Barawitzkagasse 10/2/2/11 email. office at ipax.at
> 1190 Wien tel. +43 1 3670030
> FN 277995t HG Wien fax. +43 1 3670030 15
> ____________________________________________________________________
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

2007/12/13, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Lol... the initial message seems not to have passed my spam filter... ;)
>
> I, too, tend to option three. The problem, though is

Yeah, it's the most flexible one.

> backwards-compatibility... Any suggestions and concerns are appreciated.

This feature has been pretty new, and as such probably isn't widely used.
I would simply emphasize this change in the release notes and the Changelog.
The worst that can happen is, that a file is not included anymore.

>
> I'll look at the code to see how much work that creates. I think the
> complexity is quite acceptable. I am right now in a threading redesign,
> so I'd like to push this away for a little while... Anyhow, please keep
> comments floating - I'll listen and drop in any thoughts I have.

Should be fairly straightforward to implement using
http://www.gnu.org/software/libc/manual/html_node/Globbing.html#Globbing

Cheers,
Michael


--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

2007/12/14, Michael Biebl <mbiebl at gmail.com>:
> 2007/12/13, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Lol... the initial message seems not to have passed my spam filter... ;)
> >
> > I, too, tend to option three. The problem, though is
>
> Yeah, it's the most flexible one.
>
> > backwards-compatibility... Any suggestions and concerns are appreciated.
>
> This feature has been pretty new, and as such probably isn't widely used.
> I would simply emphasize this change in the release notes and the Changelog.
> The worst that can happen is, that a file is not included anymore.
>
> >
> > I'll look at the code to see how much work that creates. I think the
> > complexity is quite acceptable. I am right now in a threading redesign,
> > so I'd like to push this away for a little while... Anyhow, please keep
> > comments floating - I'll listen and drop in any thoughts I have.
>
> Should be fairly straightforward to implement using
> http://www.gnu.org/software/libc/manual/html_node/Globbing.html#Globbing
>

Small example using glob:

#include <glob.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(int argc, char **argv)
{
glob_t files;
int ret = 0;
int i = 0;
char *path;

if (argc > 1) {
path = argv[1];
} else {
path = strdup("/etc/rsyslog.d/*.conf");
}

ret = glob(path, GLOB_ERR, NULL, &files);
printf("All files matching %s:\n", path);
for (i = 0; i < files.gl_pathc; i++) {
printf("\t%s\n", files.gl_pathv[i]);
}

globfree(&files);
printf("Finished\n");

return EXIT_SUCCESS;
}

man 3 glob has more information.

If wanted, I can prepare a patch for rsyslog.

Cheers,
Michael
--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

Michael,

sorry for the sluggish reply. I am right in the middle of the initial
threading code (have a look at CVS ;)) and that blocks most of
everything else ;)

It would be great if you could supply a patch, I'd apply it as soon as I
got something together with the new code. I try to do the threading in
an as atomic code change as possible, but it will proabbyl busy me for a
while ;)

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, December 14, 2007 5:24 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with IncludeConfig
>
> 2007/12/14, Michael Biebl <mbiebl at gmail.com>:
> > 2007/12/13, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > Lol... the initial message seems not to have passed my spam
> filter... ;)
> > >
> > > I, too, tend to option three. The problem, though is
> >
> > Yeah, it's the most flexible one.
> >
> > > backwards-compatibility... Any suggestions and concerns are
> appreciated.
> >
> > This feature has been pretty new, and as such probably isn't widely
> used.
> > I would simply emphasize this change in the release notes and the
> Changelog.
> > The worst that can happen is, that a file is not included anymore.
> >
> > >
> > > I'll look at the code to see how much work that creates. I think
> the
> > > complexity is quite acceptable. I am right now in a threading
> redesign,
> > > so I'd like to push this away for a little while... Anyhow, please
> keep
> > > comments floating - I'll listen and drop in any thoughts I have.
> >
> > Should be fairly straightforward to implement using
> >
>
http://www.gnu.org/software/libc/manual/html_node/Globbing.html#Globbin
> g
> >
>
> Small example using glob:
>
> #include <glob.h>
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
>
> int main(int argc, char **argv)
> {
> glob_t files;
> int ret = 0;
> int i = 0;
> char *path;
>
> if (argc > 1) {
> path = argv[1];
> } else {
> path = strdup("/etc/rsyslog.d/*.conf");
> }
>
> ret = glob(path, GLOB_ERR, NULL, &files);
> printf("All files matching %s:\n", path);
> for (i = 0; i < files.gl_pathc; i++) {
> printf("\t%s\n", files.gl_pathv[i]);
> }
>
> globfree(&files);
> printf("Finished\n");
>
> return EXIT_SUCCESS;
> }
>
> man 3 glob has more information.
>
> If wanted, I can prepare a patch for rsyslog.
>
> Cheers,
> Michael
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

2007/12/14, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Michael,
>
> sorry for the sluggish reply. I am right in the middle of the initial
> threading code (have a look at CVS ;)) and that blocks most of
> everything else ;)
>
> It would be great if you could supply a patch, I'd apply it as soon as I
> got something together with the new code. I try to do the threading in
> an as atomic code change as possible, but it will proabbyl busy me for a
> while ;)

Unfortunately I can't set the mimetype via gmail, so the patch is at

http://debs.michaelbiebl.de/rsyslog/rsyslog-glob.patch

I didn't check the return values for glob. I wanted I can implement
that and printout debugging messsages in that case.
The patch also doesn't contain any changes to the documentation (yet).
I wanted to get some feedback first.

Michael

--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

2007/12/16, Michael Biebl <mbiebl at gmail.com>:
> 2007/12/14, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Michael,
> >
> > sorry for the sluggish reply. I am right in the middle of the initial
> > threading code (have a look at CVS ;)) and that blocks most of
> > everything else ;)
> >
> > It would be great if you could supply a patch, I'd apply it as soon as I
> > got something together with the new code. I try to do the threading in
> > an as atomic code change as possible, but it will proabbyl busy me for a
> > while ;)
>
> Unfortunately I can't set the mimetype via gmail, so the patch is at
>
> http://debs.michaelbiebl.de/rsyslog/rsyslog-glob.patch
>

I have an updated patch
http://debs.michaelbiebl.de/rsyslog/rsyslog-glob-2.patch
which is backward compatible.
It allows the following

include config files:
$IncludeConfig /etc/rsyslog.d/foo.conf
include config directories:
$IncludeConfig /etc/rsyslog.d/ (with or without trailing slash)
include via shell glob:
$IncludeConfig /etc/rsyslog.d/*.conf

One remaining issue is, how to handle symlinks. I made the patch
consistent to the current behaviour of doIncludeDirectories to ignore
symlinks.

I know that the apache Include directive allows symlinks, and
supporting it could be handy.
So my question is, what do others think about it. Please let me know.

Cheers,
Michael

P.S: A personal wish: as the list is subscriber only, would it be
possible to allow attachements (other than text/plain, gmail is
obviously too stupid)?


--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

2007/12/17, Michael Biebl <mbiebl at gmail.com>:
> 2007/12/16, Michael Biebl <mbiebl at gmail.com>:
> > 2007/12/14, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> I have an updated patch
> http://debs.michaelbiebl.de/rsyslog/rsyslog-glob-2.patch

Spotted a minor typo, so here goes v3
http://debs.michaelbiebl.de/rsyslog/rsyslog-glob-3.patch

Michael

Btw. gmail uses
Content-Type: multipart/mixed
and then
Content-Type: text/plain for the mail
and
Content-Type: text/x-diff for the patch
--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

Hi Michael,

I like this patch, especially it being backwards-compatible. And I also
tend to agree that symlinks would be nice to have.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Monday, December 17, 2007 5:27 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with IncludeConfig
>
> 2007/12/17, Michael Biebl <mbiebl at gmail.com>:
> > 2007/12/16, Michael Biebl <mbiebl at gmail.com>:
> > > 2007/12/14, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > I have an updated patch
> > http://debs.michaelbiebl.de/rsyslog/rsyslog-glob-2.patch
>
> Spotted a minor typo, so here goes v3
> http://debs.michaelbiebl.de/rsyslog/rsyslog-glob-3.patch
>
> Michael
>
> Btw. gmail uses
> Content-Type: multipart/mixed
> and then
> Content-Type: text/plain for the mail
> and
> Content-Type: text/x-diff for the patch
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

This patch now also made it to the repository. Applied to both v2
(previously 1.20.2) and v3.

Rainer

On Mon, 2007-12-17 at 05:09 +0100, Michael Biebl wrote:
> 2007/12/16, Michael Biebl <mbiebl at gmail.com>:
> > 2007/12/14, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > Michael,
> > >
> > > sorry for the sluggish reply. I am right in the middle of the initial
> > > threading code (have a look at CVS ;)) and that blocks most of
> > > everything else ;)
> > >
> > > It would be great if you could supply a patch, I'd apply it as soon as I
> > > got something together with the new code. I try to do the threading in
> > > an as atomic code change as possible, but it will proabbyl busy me for a
> > > while ;)
> >
> > Unfortunately I can't set the mimetype via gmail, so the patch is at
> >
> > http://debs.michaelbiebl.de/rsyslog/rsyslog-glob.patch
> >
>
> I have an updated patch
> http://debs.michaelbiebl.de/rsyslog/rsyslog-glob-2.patch
> which is backward compatible.
> It allows the following
>
> include config files:
> $IncludeConfig /etc/rsyslog.d/foo.conf
> include config directories:
> $IncludeConfig /etc/rsyslog.d/ (with or without trailing slash)
> include via shell glob:
> $IncludeConfig /etc/rsyslog.d/*.conf
>
> One remaining issue is, how to handle symlinks. I made the patch
> consistent to the current behaviour of doIncludeDirectories to ignore
> symlinks.
>
> I know that the apache Include directive allows symlinks, and
> supporting it could be handy.
> So my question is, what do others think about it. Please let me know.
>
> Cheers,
> Michael
>
> P.S: A personal wish: as the list is subscriber only, would it be
> possible to allow attachements (other than text/plain, gmail is
> obviously too stupid)?
>
>

2007/12/18, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> This patch now also made it to the repository. Applied to both v2
> (previously 1.20.2) and v3.

Cool, thanks.
Remains to update the documentation doc/rsconf1_includeconfig.html.
I'm usually not good a writing docs, but if you want I can provide a patch ;-)

Cheers,
Michael
--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

Oops... this time I forgot the doc (I always try to have at least a
minimal description there). I just added two lines, maybe you have a
look. I am a bit under pressure because I just released 1.21.1, but now
re-created the tarball with the somewhat enhanced doc. If you are up for
a better doc, just sent me a patch ;) It'll appear in the next release.

Release notification is due in a few minutes...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Wednesday, December 19, 2007 9:27 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with IncludeConfig
>
> 2007/12/18, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > This patch now also made it to the repository. Applied to both v2
> > (previously 1.20.2) and v3.
>
> Cool, thanks.
> Remains to update the documentation doc/rsconf1_includeconfig.html.
> I'm usually not good a writing docs, but if you want I can provide a
> patch ;-)
>
> Cheers,
> Michael
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog