Mailing List Archive

On Nov 21, 2008, at 4:11 AM, Jan-Frode Myklebust wrote:

> For my usage, I need two modes of operation for syslog daemons.
>
> 1 - local syslog. Requires privileges to on local devices (/dev/
> log,
> /dev/klogd or similar), write to local log-files, and send to
> remote log server.
>
> 2 - central log server. Only listen on the needed network ports
> (514/udp/tcp), and write to local log files (possibly also send
> to other remote log servers).
>
> For #1 I think it's OK not being able to chroot, keep more privileges,
> etc., as the attacks against it will mostly be from local processes.
>
> #2 needs to be *very* openly available on the network ports, since all
> my servers needs to send logs to it. #2 will also be holding a lot
> more
> sensitive data than #1, so I think this server needs to be protected
> as
> much as possible. If chroot'ing, or dropping privileges prevents it
> from
> reading from local /proc og /dev, I think that wouldn't matter much.
> One
> could always run two instances on these few central servers, i.e. #1
> sending to #2.

Yes, your two modes as defined above are exactly how I run syslog, and
I think the distinction between the two use cases
local-syslog and central-log-server is very useful and important in
this discussion.

And I agree with your prioritization of security between the two
modes, the local-syslog does not need to open a network port
to listen for log messages, and so it is only at risk from other
processes on the same box.

The central-log-server does need to open and read a network port, and
needs to be open to lots of devices on the network, and thus it is at
the most risk.
That is the version that most needs privilege separation and chroot.
And I agree that the server that runs the central-log-server could
also run an instance of the local-syslog, the local-syslog would
handle /dev/log on that machine, and forward msgs to be centrally
logged onto the central-log-server running on the same machine.

From another sub-thread:

> On Nov 19, 2008, at 4:32 AM, Rainer Gerhards wrote:

> Yes, the traditional HUP behavior is simply incompatible with dropping
> privileges. But I assume that someone who configures rsyslogd in
> such a
> way knows what he does and thus changes config-reload scripts away
> from
> HUP to a real reload.

I am not sure about the details of the implementation, but named on
OpenBSD supports privilege separation, and it provides
some of your HUP functionality, although possibly in different ways:

Here is how named looks from ps:

root 23404 0.0 0.0 2148 880 ?? Is 2:19PM 0:00.00
named: [priv] (named)
named 21080 0.0 0.2 8744 9452 ?? I 2:19PM 0:00.24
named -4

Here is what the named man page says about HUP:

SIGNALS
In routine operation, signals should not be used to con-
trol the nameserver; rndc should be used instead.

SIGHUP Force a reload of the server.

SIGINT, SIGTERM
Shut down the server.

The result of sending any other signals to the server is
undefined.

And, as stated above, rndc is the preferred way of controlling named
(asking it to reload zone files, etc.)

The config file for rsyslogd could/should reside within the chroot, so
it should always be accessible.


On yet another sub-thread:

On Nov 19, 2008, at 4:46 AM, Rainer Gerhards wrote:
> I also agree to David Lang's argument that some
> degree of design change would be necessary to get the requested
> features
> done 100% correct.

It is generally difficult to add security ex post facto, and the more
code that is written prior to the implementation of those design
changes will tend to increase the difficultly/cost of that
implementation.

On Nov 18, 2008, at 2:29 PM, (private) HKS wrote:
> FWIW, while I believe this discussion is relevant and the security
> changes proposed are important, I don't see this as a high priority
> feature. The scripting language/syntax, greater performance, continued
> stability enhancements, and true RELP support are all far more
> important. As far as security goes, I would much rather see two-way
> host authentication than a chroot/unprivilieged framework implemented.

I don't know what the author's goals for rsyslog are, so it is
difficult to prioritize.

What *I* would like rsyslog to be is the clear choice to replace
syslog on all the machines in my network, on all the OSes I care about
(OpenBSD, Solaris, FreeBSD, etc),
basically "total world domination of the syslog space" :-)

I would ask Ranier and others here to reflect on the need/importance
of security vis-a-vis their ultimate goals for rsyslog, and if
security is going to be important (I think it is crucial, but you need
to come to that conclusion yourself), and the cost of implementing
security design changes is going to increase over time, to consider
making those design changes sooner rather than later.

In closing, I am learning a lot and enjoying the thoughtful and
patient responses to my original email on this topic, kudos to the
rsyslog community, it is a welcome change from some other mailing
lists :-)

Best regards,

Don













_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

On Sat, Nov 22, 2008 at 04:17, Jan-Frode Myklebust <janfrode@tanso.net> wrote:
> Did you see the comment from Rainer about the secpath-replace property?
> I think that proves my basis is not flawed. The chroot here protects
> against code flaws, or configuration flaws that could otherwise give
> attacker the possibility of overwriting system files.

I most certainly did, but all the secpath-replace property protects
against is an administrator foolishly trusting unvalidated data from a
remote host to generate file paths. Chroot does not protect against
code flaws, full stop. That is the continuing flaw in your argument.
Chroot only prevents code flaws from affecting anything _outside_ of
the chroot environment.

> Also you pointing at "most exploits bring their own executable .. operate
> purely in-memory" argument is flawed when I'm arguing for chroot being
> *more* secure. Not 100% secure against all flaws, but *more* secure than
> a non-chrooted process.

You mistake what I am arguing - there is no question that chroot()
helps protect the host system's files. What it does _not_ protect
against is abuse of anything within the chroot(). Your statement was:

> so I would argue that my sensitive syslogged data is more secure in a
> chroot environment

If your log data is being stored inside that chroot, you are utterly
wrong. Please actually understand what chroot() does before you
continue making wild, unfounded statements.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

Hi all,

please let me jump into the middle of that discussion.

Maybe it helps if someone else comments. From my (external) point of
view, it looks like your arguments hit more or less the same targets,
but it also looks like you seem to apply different co-notations to
things. So let me try to formalize things a bit.

Let me think of security as a probability of security breach. S_curr is
the security of the reference system without a root jail. S_total is the
security of a hypothetical system that is "totally secure" (knowing well
that no such system exists). In other words, the probability S_total
equals 0.

I think the common ground is that a root jail does not worsen security.
Note that I do not say it improves security, only that it does not
reduce a system's security. S_jail is the security of a system that is
otherwise identical to the reference system, but with a root jail. Than
S_jail <= s_curr, because we assume that the security of the system is
not reduced.

I think it is also common ground that the probability of a security
breach is reduced if the number of attack vectors is reduced, without
any new attack vectors being added. [.There is one generic "attack
vector", the "thought of being secure and thus becoming careless" which
always increases as risk is reduced - I will not include that vector in
my thoughts]

We seem to be in agreement that a root jail is able to prevent some
attacks from being successful. I can't enumerate them and it is probably
useless to try to do so (because attackers invent new attacks each day),
but there exist some attacks which can be prevented by a root jail. I do
not try to weigh them by their importance.

For obvious reasons, there exist other attacks which are not affected by
the root jail. Some of them have been mentions, like the class of
in-memory based attacks, code injection and many more.

I tend to think that the set of attack vectors that can be prevented by
a root jail is much smaller than the set of those which can not. I also
tend to think that the later class contains the more serious attack
vectors.

But even then, a root jail seems to remove a subset of the attack
vectors that otherwise exists and so it reduces the probably of security
breach. So it benefits security. We can only argue that it does not
benefit security if we can show that in all cases we can think of (and
those we can not), security is not improved. However, some cases have
been show, where it improves, so it can not be that security is not
improved in all cases. As such, a root jail improves security, or more
precisely the probability of a security breach is

0 < S_jail < S_curr

We can identify the benefit we gain is the difference between the
reference system's probability of security breach and the system with
the jail. Be S_impr this improvement, than

S_impr = S_curr - S_jail

Now the root jail is just one potential security measure. We could now
try to calculate S_impr for all kinds of security measures, for example
a privilege drop. I find it hard to do the actual probability
calculations, but I would guess that S_impr_privdop > S_impr_jail.

Based on the improvements, one may finally decide what to implement
first (either at the code or admin level), all of this of course
weighted with the importance of the numbers.

In any case, I think I have show that both is correct:

- the root jail is a security improvement
- there exist numerous other improvements, many of them probably more
efficient than the jail

And if I got you right, I think your arguments meant exactly this ;)

For rsyslog as a project, this also means we need to weigh security
measures against functionality. Based on this discussion, I wonder if it
would be a useful undertaking to try document at least the security
options at hand and try to provide some helpful notes for those that are
not so deeply knowledgeable about these issues (including me, someone
who is always surprised by the numerous ways people find to circumvent
what one has considered to be secure ;)).

Feedback is appreciated.

Rainer


> -----Original Message-----
> From: rsyslog-bounces@lists.adiscon.com [mailto:rsyslog-
> bounces@lists.adiscon.com] On Behalf Of RB
> Sent: Saturday, November 22, 2008 11:49 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslogd security questions/comments
>
> On Sat, Nov 22, 2008 at 04:17, Jan-Frode Myklebust
<janfrode@tanso.net>
> wrote:
> > Did you see the comment from Rainer about the secpath-replace
> property?
> > I think that proves my basis is not flawed. The chroot here protects
> > against code flaws, or configuration flaws that could otherwise give
> > attacker the possibility of overwriting system files.
>
> I most certainly did, but all the secpath-replace property protects
> against is an administrator foolishly trusting unvalidated data from a
> remote host to generate file paths. Chroot does not protect against
> code flaws, full stop. That is the continuing flaw in your argument.
> Chroot only prevents code flaws from affecting anything _outside_ of
> the chroot environment.
>
> > Also you pointing at "most exploits bring their own executable ..
> operate
> > purely in-memory" argument is flawed when I'm arguing for chroot
> being
> > *more* secure. Not 100% secure against all flaws, but *more* secure
> than
> > a non-chrooted process.
>
> You mistake what I am arguing - there is no question that chroot()
> helps protect the host system's files. What it does _not_ protect
> against is abuse of anything within the chroot(). Your statement was:
>
> > so I would argue that my sensitive syslogged data is more secure in
a
> > chroot environment
>
> If your log data is being stored inside that chroot, you are utterly
> wrong. Please actually understand what chroot() does before you
> continue making wild, unfounded statements.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

Hi Don,

thanks for your careful thoughts. I would also appreciate if you could
try out the experimental branch, but on the other hand I think I will be
able to create a regular devel branch based on some of the discussion
this week.

On Sat, 2008-11-22 at 12:34 -0800, Don Jackson wrote:
> On Nov 21, 2008, at 4:11 AM, Jan-Frode Myklebust wrote:
>
> > For my usage, I need two modes of operation for syslog daemons.
> >
> > 1 - local syslog. Requires privileges to on local devices (/dev/
> > log,
> > /dev/klogd or similar), write to local log-files, and send to
> > remote log server.
> >
> > 2 - central log server. Only listen on the needed network ports
> > (514/udp/tcp), and write to local log files (possibly also send
> > to other remote log servers).
> >
> > For #1 I think it's OK not being able to chroot, keep more privileges,
> > etc., as the attacks against it will mostly be from local processes.
> >
> > #2 needs to be *very* openly available on the network ports, since all
> > my servers needs to send logs to it. #2 will also be holding a lot
> > more
> > sensitive data than #1, so I think this server needs to be protected
> > as
> > much as possible. If chroot'ing, or dropping privileges prevents it
> > from
> > reading from local /proc og /dev, I think that wouldn't matter much.
> > One
> > could always run two instances on these few central servers, i.e. #1
> > sending to #2.
>
> Yes, your two modes as defined above are exactly how I run syslog, and
> I think the distinction between the two use cases
> local-syslog and central-log-server is very useful and important in
> this discussion.

Even further, I think this could be different (but compatible) use cases
in a to-be-written rsyslog security doc.

>
> And I agree with your prioritization of security between the two
> modes, the local-syslog does not need to open a network port
> to listen for log messages, and so it is only at risk from other
> processes on the same box.
>
> The central-log-server does need to open and read a network port, and
> needs to be open to lots of devices on the network, and thus it is at
> the most risk.
> That is the version that most needs privilege separation and chroot.
> And I agree that the server that runs the central-log-server could
> also run an instance of the local-syslog, the local-syslog would
> handle /dev/log on that machine, and forward msgs to be centrally
> logged onto the central-log-server running on the same machine.
>
> From another sub-thread:
>
> > On Nov 19, 2008, at 4:32 AM, Rainer Gerhards wrote:
>
> > Yes, the traditional HUP behavior is simply incompatible with dropping
> > privileges. But I assume that someone who configures rsyslogd in
> > such a
> > way knows what he does and thus changes config-reload scripts away
> > from
> > HUP to a real reload.
>
> I am not sure about the details of the implementation, but named on
> OpenBSD supports privilege separation, and it provides
> some of your HUP functionality, although possibly in different ways:
[snip]

It would be too much info to go in full detail, but HUP is ugly and it
is a full restart. I'd like to phase out that mode, but will probably
need to keep it for "admin compatibility". Anyhow, I do not see any
issue if, in a secure system, HUP does not work as a full restart. (One
way to make it work would be to have a master running all the time and
the actual rsyslogd being a child where the parent handles HUP and
restarts the "real" rsyslogd - but I don't like to make it so compliated
"just" to satisfy old habits... [who likes this can always contribute
the code ;)]).

> On yet another sub-thread:
>
> On Nov 19, 2008, at 4:46 AM, Rainer Gerhards wrote:
> > I also agree to David Lang's argument that some
> > degree of design change would be necessary to get the requested
> > features
> > done 100% correct.
>
> It is generally difficult to add security ex post facto, and the more
> code that is written prior to the implementation of those design
> changes will tend to increase the difficultly/cost of that
> implementation.

That's totally right, and rsyslog is a good example. First of all it is
important to remember that it was not designed from scratch. We forked
it off from sysklogd. And while almost no sysklogd code remains today,
we had a long chain of design decisions which were based on the need to
keep compatible with sysklogd. If you traverse that chain, you'll see
why some things are as they are.

The security approach was not to be worse than sysklogd and better where
possible, but there always was a strong focus on deliverving something.
You may argue that it is foolish not doing the "right thing" right from
the beginning. My view is that we live in an imperfect (and risky) world
and it is important to get some things done. So I don't want to limit
myself in using a better solution because that better solution is not
better in some aspect than the previous solution, which I would be using
otherwise.

In somewhat more formal words. If sysklogd's number if features is
F_sysklogd and it's probably of security breach is S_sysklog and the
exact same concepts for rsyslog are F_rsyslog and S_rsyslog, then I
don't see any reason not to use rsyslog over sysklogd if F_rsyslog >
F_sysklogd and R_rsyslog <= R_sysklogd. In even other words, if I would
not use rsyslog because it is not as secure as sysklogd, I would ditch
it without getting any benefit and I would also not get the benefits
from the new features it offers.

One may of course argue that rsyslog's addition feature inherently
increase the risk of using it, in which case S_rsyslog < S_sysklogd. The
only argument I have against this is that you can disable most of these
new features so that you can limit the potential risk exposure.

In any case, you are right that the ultimate goal should be to keep as
secure *as possible* and this is why I immediately followed up on your
questions.

But I think it is still very important to not let security be a show
stopper in a case if it is not worsened. I think if I had started the
rsyslog project with a totally secure design, rsyslog would be obsolete
by now because anybody would already have moved over to syslog-ng and no
other solution would have a chance to rise.

I am not saying that syslog-ng is a bad project, nor am I saying it is
less secure than rsyslog (right now, it probably is more secure). But a
world with just syslog-ng (or one with just rsyslog!) is a monoculture a
monoculture has inherent and very large security risk.

I sincerely believe (bash me) that Windows is *not* bad software (or at
least not worse than Linux). It "just" is (well, "was" ;)) a monoculture
and that lead to much more interest in attacking it. The financial
benefit of attacking it was much greater than the benefit of attacking
Linux.

If you look from that angle (and I do), you'll find that a world with a
not-so-secure rsyslog AND syslog-ng is more secure than a world with a
totally-secure rsyslog (that nobody uses) and and always-present
syslog-ng (being a monoculture on the vast majority of systems). So less
security in a "subsystem" brought more security to the overall picture.

Still, I strongly favor an as-secure-as possible rsyslog, but I need to
abide to the constraints given (and with the financial crises and loss
of sponsorship these constraints for the rsyslog project unfortunately
have not become more permitting...).

>
> On Nov 18, 2008, at 2:29 PM, (private) HKS wrote:
> > FWIW, while I believe this discussion is relevant and the security
> > changes proposed are important, I don't see this as a high priority
> > feature. The scripting language/syntax, greater performance, continued
> > stability enhancements, and true RELP support are all far more
> > important. As far as security goes, I would much rather see two-way
> > host authentication than a chroot/unprivilieged framework implemented.
>
> I don't know what the author's goals for rsyslog are, so it is
> difficult to prioritize.
>
> What *I* would like rsyslog to be is the clear choice to replace
> syslog on all the machines in my network, on all the OSes I care about
> (OpenBSD, Solaris, FreeBSD, etc),
> basically "total world domination of the syslog space" :-)

as said above, "total world dominition" is sweet fruit, but it has a big
risk associated with it...

>
> I would ask Ranier and others here to reflect on the need/importance
> of security vis-a-vis their ultimate goals for rsyslog, and if
> security is going to be important (I think it is crucial, but you need
> to come to that conclusion yourself), and the cost of implementing
> security design changes is going to increase over time, to consider
> making those design changes sooner rather than later.

I hope I outlined these thoughts above. Please keep asking/suggesting.
My mind is always open to change and I like good arguments ;)

Rainer
>
> In closing, I am learning a lot and enjoying the thoughtful and
> patient responses to my original email on this topic, kudos to the
> rsyslog community, it is a welcome change from some other mailing
> lists :-)
>
> Best regards,
>
> Don
>
>
>
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com