Mailing List Archive

syslog-ng misbehaving
Before I seek out a mailing list for syslog-ng, I was hoping I could get
some tips from people here. I recently started trying to separate logs
into various functions rather than letting everything go to
/var/log/messages. So I created three filters in syslog-ng. One is
intended to separate sshd messages, one to separate samba messages, and
the other to say "everything ELSE". The problem  I seem to be having is
that the everything ELSE log still shows things that should have been
removed! For example, when I login via ssh I get identical notification
in the /var/log/messages and in /var/sshd/sshd.log. So I'm lost. I'm
including my syslog-ng.conf. Perhaps someone here can tell me what I'm
doing wrong.

Thanks!

---------- CUT HERE - syslog_ng.conf ----------
@version: 3.30

@include "scl.conf"

options {
    threaded(yes);
    chain_hostnames(no);
    stats_freq(43200);
    mark_freq(3600);
};

source src { system(); internal(); };

filter samba { program("samba"); };
filter ssh_messages { facility("AUTH") and level("INFO"); };
filter syslog { not filter("ssh_messages") and not filter("samba"); };

destination console { file("/dev/tty12"); };
destination messages { file("/var/log/messages"); };
destination sshd_log { file("/var/log/sshd/sshd.log"); };
destination smb_logs { file("/var/log/samba/samba.log"); };

log { source(src); destination(smb_logs); filter(samba); flags(final); );
log { source(src); destination(sshd_log); filter(ssh_messages);
flags(final); };
log { source(src); destination(console); filter(syslog); };
log { source(src); destination(messages); filter(syslog); };
---------- CUT HERE --------------
Re: syslog-ng misbehaving [ In reply to ]
On Tue, 6 Apr 2021 23:11:15 -0600, Dan Egli wrote:

> Before I seek out a mailing list for syslog-ng, I was hoping I could
> get some tips from people here. I recently started trying to separate
> logs into various functions rather than letting everything go to
> /var/log/messages. So I created three filters in syslog-ng. One is
> intended to separate sshd messages, one to separate samba messages, and
> the other to say "everything ELSE". The problem  I seem to be having is
> that the everything ELSE log still shows things that should have been
> removed! For example, when I login via ssh I get identical notification
> in the /var/log/messages and in /var/sshd/sshd.log. So I'm lost. I'm
> including my syslog-ng.conf. Perhaps someone here can tell me what I'm
> doing wrong.

> filter samba { program("samba"); };
> filter ssh_messages { facility("AUTH") and level("INFO"); };
> filter syslog { not filter("ssh_messages") and not filter("samba"); };

Shouldn't this be an or? You are filtering out anything that is in both
groups, which will be a very small set of messages.


--
Neil Bothwick

Ralph's Observation - It is a mistake to allow any mechanical object
to realize that you are in a hurry.
Re: syslog-ng misbehaving [ In reply to ]
It's worth a shot. I never completely got boolean logic, so you may be
right.

And the result is no joy. I changed it to or, restarted syslog-ng and
tailed /var/log/messages, just in time to catch a botnet trying a
brute-force attack. Since all the sshd messages are comming in through
/var/log/messages, then changing to "or" vs. "and" made no difference.


On 4/7/2021 1:40 AM, Neil Bothwick wrote:
> On Tue, 6 Apr 2021 23:11:15 -0600, Dan Egli wrote:
>
>> Before I seek out a mailing list for syslog-ng, I was hoping I could
>> get some tips from people here. I recently started trying to separate
>> logs into various functions rather than letting everything go to
>> /var/log/messages. So I created three filters in syslog-ng. One is
>> intended to separate sshd messages, one to separate samba messages, and
>> the other to say "everything ELSE". The problem  I seem to be having is
>> that the everything ELSE log still shows things that should have been
>> removed! For example, when I login via ssh I get identical notification
>> in the /var/log/messages and in /var/sshd/sshd.log. So I'm lost. I'm
>> including my syslog-ng.conf. Perhaps someone here can tell me what I'm
>> doing wrong.
>> filter samba { program("samba"); };
>> filter ssh_messages { facility("AUTH") and level("INFO"); };
>> filter syslog { not filter("ssh_messages") and not filter("samba"); };
> Shouldn't this be an or? You are filtering out anything that is in both
> groups, which will be a very small set of messages.
>
>
Re: syslog-ng misbehaving [ In reply to ]
On Wednesday, 7 April 2021 08:40:44 BST Neil Bothwick wrote:
> On Tue, 6 Apr 2021 23:11:15 -0600, Dan Egli wrote:
> > Before I seek out a mailing list for syslog-ng, I was hoping I could
> > get some tips from people here. I recently started trying to separate
> > logs into various functions rather than letting everything go to
> > /var/log/messages. So I created three filters in syslog-ng. One is
> > intended to separate sshd messages, one to separate samba messages, and
> > the other to say "everything ELSE". The problem I seem to be having is
> > that the everything ELSE log still shows things that should have been
> > removed! For example, when I login via ssh I get identical notification
> > in the /var/log/messages and in /var/sshd/sshd.log. So I'm lost. I'm
> > including my syslog-ng.conf. Perhaps someone here can tell me what I'm
> > doing wrong.
> >
> > filter samba { program("samba"); };
> > filter ssh_messages { facility("AUTH") and level("INFO"); };
> > filter syslog { not filter("ssh_messages") and not filter("samba"); };
>
> Shouldn't this be an or? You are filtering out anything that is in both
> groups, which will be a very small set of messages.

Nope. NOT a AND NOT b matches anything that is in neither a nor b. The rule
you're suggesting would be NOT (a AND b).

--
Regards,
Peter.
Re: syslog-ng misbehaving [ In reply to ]
On Wed, 2021-04-07 at 02:02 -0600, Dan Egli wrote:
> It's worth a shot. I never completely got boolean logic, so you may be
> right.
>

It depends on an implicit order of operations. Usually "not" has higher
precedence than "and" and "or", but personally I wouldn't count on it
unless the documentation says so (or you've tried it).

It's like when you see a+b/c*d. Some people use PEMDAS, some people use
BEDMAS. Neither of them work, and they give different answers. It's no
wonder nobody understands this stuff.

In any case, post your whole syslog-ng.conf file. Some later directive
could be shoveling things into /var/log/messages.
Re: syslog-ng misbehaving [ In reply to ]
Dan,

On Tuesday, 2021-04-06 23:11:15 -0600, you wrote:

> ...
> log { source(src); destination(smb_logs); filter(samba); flags(final); );

According to the documentation at

https://support.oneidentity.com/technical-documents/syslog-ng-open-source-edition/3.30/administration-guide/56#TOPIC-1595065

the filter clause should be specified between source and destination.

Sincerely,
Rainer
Re: syslog-ng misbehaving [ In reply to ]
I had posted the whole file. But I can do it again easy enough.

--------------------------------
@version: 3.30

@include "scl.conf"

options {
    threaded(yes);
    chain_hostnames(no);
    stats_freq(43200);
    mark_freq(3600);
};

source src { system(); internal(); };

filter samba { program("samba"); };
filter ssh_messages { facility("AUTH") and level("INFO"); };
filter syslog { not filter("ssh_messages") and not filter("samba"); };

destination console { file("/dev/tty12"); };
destination messages { file("/var/log/messages"); };
destination sshd_log { file("/var/log/sshd/sshd.log"); };
destination smb_logs { file("/var/log/samba/samba.log"); };

log { source(src); filter(samba); destination(smb_logs); flags(final); );
log { source(src); filter(ssh_messages); destination(sshd_log);
flags(final); };
log { source(src); filter(syslog); destination(console); };
log { source(src); filter(syslog); destination(messages); };
----------------------------

There, that's the ENTIRE file, sans comments.

On 4/7/2021 5:47 AM, Michael Orlitzky wrote:
> On Wed, 2021-04-07 at 02:02 -0600, Dan Egli wrote:
>> It's worth a shot. I never completely got boolean logic, so you may be
>> right.
>>
> It depends on an implicit order of operations. Usually "not" has higher
> precedence than "and" and "or", but personally I wouldn't count on it
> unless the documentation says so (or you've tried it).
>
> It's like when you see a+b/c*d. Some people use PEMDAS, some people use
> BEDMAS. Neither of them work, and they give different answers. It's no
> wonder nobody understands this stuff.
>
> In any case, post your whole syslog-ng.conf file. Some later directive
> could be shoveling things into /var/log/messages.
>
>
>
Re: syslog-ng misbehaving [ In reply to ]
That was fixed a bit ago. See my most recent post for a copy of the
current config file.

On 4/7/2021 8:46 AM, Dr Rainer Woitok wrote:
> Dan,
>
> On Tuesday, 2021-04-06 23:11:15 -0600, you wrote:
>
>> ...
>> log { source(src); destination(smb_logs); filter(samba); flags(final); );
> According to the documentation at
>
> https://support.oneidentity.com/technical-documents/syslog-ng-open-source-edition/3.30/administration-guide/56#TOPIC-1595065
>
> the filter clause should be specified between source and destination.
>
> Sincerely,
> Rainer
>
Re: syslog-ng misbehaving [ In reply to ]
Dan,

On Wednesday, 2021-04-07 12:05:10 -0600, you wrote:

> I had posted the whole file. But I can do it again easy enough.
> ...
> filter samba { program("samba"); };
> filter ssh_messages { facility("AUTH") and level("INFO"); };
> filter syslog { not filter("ssh_messages") and not filter("samba"); };

Omit the double quotes in this last line. You're needing the NAMES of
the filters here.

Sincerely,
Rainer
Re: syslog-ng misbehaving [ In reply to ]
On 4/8/2021 9:59 AM, Dr Rainer Woitok wrote:
> Dan,
>
> On Wednesday, 2021-04-07 12:05:10 -0600, you wrote:
>
>> I had posted the whole file. But I can do it again easy enough.
>> ...
>> filter samba { program("samba"); };
>> filter ssh_messages { facility("AUTH") and level("INFO"); };
>> filter syslog { not filter("ssh_messages") and not filter("samba"); };
> Omit the double quotes in this last line. You're needing the NAMES of
> the filters here.
>

I'm afraid that didn't work either.  I did as you said, and changed the
syslog filter line to read: filter syslog { not filter(sshd) and not
filter (samba); }; which would match the previous lines (see URL below).
I still see sshd messages in /var/log/messages when I ssh into the
machine. I'm totally lost. I've posted relevant files for everyone to
see. All are updated in real time becuase they are either symlinks to
the actual files, or are the target of a redirection directly:

https://www.newideatest.site/syslog-conf = /etc/syslog-ng/syslog-ng.conf
https://www.newideatest.site/syslog-out = output of syslog-ng -Fdav
https://www.newideatest.site/system_log = /var/log/messages


Any further ideas are most welcome.
Re: syslog-ng misbehaving [ In reply to ]
I swear Thunderbird is sending to the MTA when it should be saving on
the IMAP server. Ignore this one for the next one. IT is complete. This
one is not.

On 4/8/2021 12:13 PM, Dan Egli wrote:
> On 4/8/2021 9:59 AM, Dr Rainer Woitok wrote:
>> Dan,
>>
>> On Wednesday, 2021-04-07 12:05:10 -0600, you wrote:
>>
>>> I had posted the whole file. But I can do it again easy enough.
>>> ...
>>> filter samba { program("samba"); };
>>> filter ssh_messages { facility("AUTH") and level("INFO"); };
>>> filter syslog { not filter("ssh_messages") and not filter("samba"); };
>> Omit the double quotes in this last line. You're needing the NAMES of
>> the filters here.
>>
>
> I'm afraid that didn't work either.  I did as you said, and changed
> the syslog filter line to read: filter syslog { not filter(sshd) and
> not filter (samba); }; which would match the previous lines (see URL
> below). I still see sshd messages in /var/log/messages when I ssh into
> the machine.
>
Re: syslog-ng misbehaving [ In reply to ]
"Dan Egli" <dan@newideatest.site>, 08.04.2021, 20:15:

> I'm afraid that didn't work either. I did as you said, and changed the syslog filter line to read: filter syslog { not filter(sshd) and not filter (samba); }; which would match the previous lines (see URL below). I still see sshd messages in /var/log/messages when I ssh into the machine. I'm totally lost. I've posted relevant files for everyone to see. All are updated in real time becuase they are either symlinks to the actual files, or are the target of a redirection directly:

> https://www.newideatest.site/syslog-conf = /etc/syslog-ng/syslog-ng.conf

Is the filter definition correct?

filter sshd { program("ssdhd"); };
^ ???

s.

> https://www.newideatest.site/syslog-out = output of syslog-ng -Fdav
> https://www.newideatest.site/system_log = /var/log/messages


> Any further ideas are most welcome.
Re: syslog-ng misbehaving [ In reply to ]
Yea, that was it. I can't believe I missed that. Boy do I feel dumb now.
Thanks!

On 4/8/2021 11:11 PM, Stefan Schmiedl wrote:
> "Dan Egli" <dan@newideatest.site>, 08.04.2021, 20:15:
>
>> I'm afraid that didn't work either. I did as you said, and changed the syslog filter line to read: filter syslog { not filter(sshd) and not filter (samba); }; which would match the previous lines (see URL below). I still see sshd messages in /var/log/messages when I ssh into the machine. I'm totally lost. I've posted relevant files for everyone to see. All are updated in real time becuase they are either symlinks to the actual files, or are the target of a redirection directly:
>> https://www.newideatest.site/syslog-conf = /etc/syslog-ng/syslog-ng.conf
> Is the filter definition correct?
>
> filter sshd { program("ssdhd"); };
> ^ ???
>
> s.
>
>> https://www.newideatest.site/syslog-out = output of syslog-ng -Fdav
>> https://www.newideatest.site/system_log = /var/log/messages
>
>> Any further ideas are most welcome.
>
>