Mailing List Archive

1 2 3  View All
Aw: Re: Howto log multiple sftpd instances with their chroot shared via NFS [ In reply to ]
> 4192 /usr/sbin/sftpd is likely unrelated to OpenSSH.
> I guess that this is a TLS FTP server.
> Try ps fax to see process relationships.
>
> The "sftpd:" prefix for the sshd processes may be related to SELinux;
> nothing in OpenSSH is called sftpd, except the "sftpd_t" SELinux context.

Sorry Peter I should have explained this at once.

To have a clean seperation of the standard sshd service on port 22 and our sftp server that listens on another port,
and both have and shall have really nothing to do which each other,
I created a new sshd instance named "sftpd":

/usr/sbin/sftpd is a symlink to /usr/sbin/sshd

Copied /etc/ssh/ and /etc/default/ssh to /etc/sftpd/ and /etc/default/sftpd

Copied /lib/systemd/system/ssh.service to /etc/systemd/system/sftpd.service

end replaced in the copy all occurences of regex "sshd?" with sftpd

E.g.

[Service]
EnvironmentFile=-/etc/default/sftpd
ExecStartPre=/usr/sbin/sftpd -t
ExecStart=/usr/sbin/sftpd -D $SFTPD_OPTS -f /etc/sftpd/sftpd_config

Important for correct autostart of sftpd on boot is:
[Unit]
After=network.target auditd.service sshd.service

Because sftpd needs to be started after sshd because only sshd creates /run/sshd/
and that is needed by sftpd (hard compiled in sshd).

Pitfall:
When stopping sshd /run/sshd/ gets deleted and you get the following sftpd error:
fatal: Missing privilege separation directory: /run/sshd
and sftpd does not accept new connections anymore etc.


Here ps fax output :)

12981 ? Ss 0:02 /usr/sbin/sftpd -D -f /etc/sftpd/sftpd_config
18846 ? Ss 0:00 \_ sftpd: sftp_nagios [priv]
18848 ? S 0:00 \_ sftpd: sftp_nagios@notty
18849 ? Ss 0:00 \_ sftpd: sftp_nagios@internal-sftp

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Howto log multiple sftpd instances with their chroot shared via NFS [ In reply to ]
Hi Hildgard,

Hildegard Meier wrote:
> > 4192 /usr/sbin/sftpd is likely unrelated to OpenSSH.
>
> To have a clean seperation of the standard sshd service on port 22
> and our sftp server that listens on another port, and both have and
> shall have really nothing to do which each other, I created a new
> sshd instance named "sftpd":

I understand! This could actually make it much easier to use a local
build for the standalone sftp server. Does the patch idea seem viable?


> [Service]
> EnvironmentFile=-/etc/default/sftpd
> ExecStartPre=/usr/sbin/sftpd -t
> ExecStart=/usr/sbin/sftpd -D $SFTPD_OPTS -f /etc/sftpd/sftpd_config
>
> Important for correct autostart of sftpd on boot is:
> [Unit]
> After=network.target auditd.service sshd.service
>
> Because sftpd needs to be started after sshd because only sshd
> creates /run/sshd/ and that is needed by sftpd (hard compiled in sshd).

systemd creates /run/sshd for the sshd.service, it's not great that
the sftpd service needs to use the same path.

A local sshd build allows cleanly solving that as well.


These commands build an OpenSSH-8.8p1 sshd with my sftp-server patch
and the alternate privsep directory:

git clone https://anongit.mindrot.org/openssh.git &&
cd openssh &&
git checkout -b v88_sftplog V_8_8_P1 &&
wget -O 0001-sftplog.patch \
https://lists.mindrot.org/pipermail/openssh-unix-dev/attachments/20210930/9b40e2d4/attachment.bin &&
git am 0001-sftplog.patch &&
autoreconf -fi -I m4 &&
./configure --prefix=/usr/local/sftpd --with-privsep-path=/run/sftpd &&
make sshd

Thanks to --prefix this sshd will not interfere with anything installed
via package management. Place configuration and keys (or symlinks) in
/usr/local/sftpd/etc/ and remember PidFile /run/sftpd.pid in sshd_config.


In any case you can express the current dependency to systemd to avoid
errors:

> When stopping sshd /run/sshd/ gets deleted and you get the
> following sftpd error:
> fatal: Missing privilege separation directory: /run/sshd
> and sftpd does not accept new connections anymore etc.

Set BindsTo=sshd.service in the sftpd.service [Unit] section to
tell systemd that sftpd requires sshd, and should be stopped first
if sshd is being stopped.

Also create a Wants:

mkdir /etc/systemd/system/sshd.service.wants
ln -s ../sftpd.service /etc/systemd/system/sshd.service.wants/

..to tell systemd that it should try to start sftpd when sshd starts.

Keep After=sshd.service in sftpd.service.

With those changes you can stop and start sftpd while sshd continues
running like before, if you stop sshd then sftpd is now stopped first,
and if you start either sshd or sftpd when neither is running then
both are started in the right order.


> Here ps fax output :)

That makes sense now. :)


Kind regards

//Peter
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Howto log multiple sftpd instances with their chroot shared via NFS [ In reply to ]
I'm staring at this thread, and all of this complexity, and trying to
figure out "why are you hurting yourself doing this"? In the amount of
time burned trying to integrate hundreds of NFS mounts and chroot cage
configurations, you could have "vsftpd" to support FTPS rather than
SFTP, gotten clean isolated upload and download cages without having
to embed /dev/log in the chroot cages, enabled much simpler and stable
automounting, and spent the rest of the time on the lobbying effort
necessary to convince your user community to switch to FTPS enabled
clients.

Unless you have some compelling need for SSH key based access, why are
you spending these cycles on this work?
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Howto log multiple sftpd instances with their chroot shared via NFS [ In reply to ]
Following up on my own thread.

I was able to get logging from sshd to stderr using 'sshd -e', but it appears as if internal-sftp does not send messages to stderr (or, at least, to the same file handle). With 'LogLevel VERBOSE', sshd emits expected messages about connection and accepting keys, then:

User child is on pid 1234
Starting session: subsystem'sftp' for user1 from 1.2.3.4 port 5678 id 0

It's not until closing the session that we get more:

Close session: user user1 from 1.2.3.4 port 5678 id 0
Received disconnect from 1.2.3.4 port 5678:12: disconnected by user
[...]

As expected, command-line flags in sshd_config for the internal-sftp command have no effect ('-e -l DEBUG', for example).

I haven't looked at the code to see whether there's an easy path here or not. Too bad, it seemed like a reasonable approach -- and could still be, if someone were to implement a reasonable "log-to-stderr" for internal sftp.

The only other approach that's seems at all reasonable to me is to add code and a config twiddle to sshd to send log messages directly to, say, an RFC5425 syslog-over-TLS server at a configurable address and port. That seems like a lot of code for not so much benefit.

--
jmk

> On Sep 29, 2021, at 20:44, Jim Knoble <jmknoble@pobox.com> wrote:
>
> ?
> If 'internal-sftp' means that the SFTP server is just a forked child of sshd, would the 'sshd -e' flag work to send log messages from internal-sftp to stderr?
>
> If so, it may be worth just sending all logging from sshd to stderr (sowie [daemontools]).
>
> Someone with time and ingenuity on their hands may even be able to send the stderr output on to syslog-ng and dispense with any need to filter it or send it to a file....
>
> If internal-sftp doesn't work that way for ${reasons}, it may be worth adding that functionality for this sort of use case.
>
> ____________________
> [daemontools]: https://cr.yp.to/daemontools.html
>
>
> --
> jmk
>
>>> On Sep 29, 2021, at 19:44, Peter Stuge <peter@stuge.se> wrote:
>>>
>> ?Small clarification:
>>
>> Hildegard Meier wrote:
>>> ps auxww |grep sftp
>>> root 4192 0.0 0.1 72304 6512 ? Ss 11:01 0:00 /usr/sbin/sftpd -D -f /etc/sftpd/sftpd_config
>>> root 4590 0.2 0.1 74736 6632 ? Ss 11:05 0:00 sftpd: sftp_nagios [priv]
>>> sftp_na+ 4592 0.0 0.0 74736 3432 ? S 11:05 0:00 sftpd: sftp_nagios@notty
>>> sftp_na+ 4593 0.0 0.0 74736 3108 ? Ss 11:05 0:00 sftpd: sftp_nagios@internal-sftp
>>
>> 4192 /usr/sbin/sftpd is likely unrelated to OpenSSH.
>> I guess that this is a TLS FTP server.
>> Try ps fax to see process relationships.
>>
>> The "sftpd:" prefix for the sshd processes may be related to SELinux;
>> nothing in OpenSSH is called sftpd, except the "sftpd_t" SELinux context.
>>
>>
>> //Peter
>> _______________________________________________
>> openssh-unix-dev mailing list
>> openssh-unix-dev@mindrot.org
>> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Howto log multiple sftpd instances with their chroot shared via NFS [ In reply to ]
Hello all, first let me thank you all for your input and suggestions.

I think I have found now a workaround for the problem.

> Since we have 800 users, it would be impractical unrobust to use user-specifc e.g. bind) mounts (e.g. 800 bind-over-mounts). To keep it simple, clear and coherent, all user's homes must be on the same one singular NFS-Share.

It seems, that I cannot uphold this requirement of not having user-specific bind-over-mounts. (while the requirement that there is only exactly one nfs mount is still true).

My workaround is the following:

sudo mkdir /var/data/dev # Create a directroy under which user subdirectories are created

For very username <username> and the user's primary group <groupname> do the following:

sudo mkdir /var/data/dev/<username>
sudo chmod 550 /var/data/dev/<username> # This restrictive permission is a requirement I think
sudo chgrp <groupname> /var/data/dev/<username> # so the user can read the directory

So the new directory is exactly the same as the existing /var/data/chroot/<username>/dev directory (which is on the nfs mount /var/data/chroot/).

Then do
mount --bind /var/data/dev/<username> /var/data/chroot/<username>/dev

so /var/data/chroot/<username>/dev is now effectively local on the sftp server, not anymore on nfs mount.

Then change the syslog-ng config

from
source s_chroot_<username> { unix-stream("/var/data/chroot/<username>/dev/log" optional(yes) ); };

to
source s_chroot_<username> { unix-stream("/var/data/dev/<username>/log" optional(yes) ); };

(this is not strictly needed, but I think it's nice having syslog-ng definitely now only reading from local file, guaranteed not from nfs mount anymore)

I have tested this successfully with one user so far - whether the user logs in on the one sftp server or the other, syslog-ng can now log the sftp session on the affected sftp server.

While it is still a mess to have 800 (or in the future maybe 2000 or 3000) bind mounts which I very much would like to avoid, I think this workaround is acceptable because the impact is limited to the logging functionality and not the sftp service itself. Speaking, if there are problems with the bind mounts, the impact is only that there is no sftp session logging.
Also, this is very simple, uniform and clear.
And one has 100% reliable sftp session logging.
No need to change anything else of the well established production system.

I think I will create systemd unit files for every bind mount (I experienced systemd dependency cycles with bind mounts in /etc/fstab on Ubuntu18 boot before anyway, so systemd units are needed anyway).
Maybe one can define a systemd dependency so that when you unmount the nfs mount /var/data/chroot/, all the bind mounts under it are automatically unmounted before (sftpd needs to be stopped first of course, otherwise mount would be busy), and also the bind-mounts automatically be mounted after /var/dat/chroot is mounted again.

So there should not be much hassle with all those simple uniform bind mounts.

I think I will implement a nagios monitoring check that checks if for every username /var/data/chroot/<username>/dev is a mountpoint, so logging is assured.

According to https://serverfault.com/questions/102588/maximum-numbers-for-file-system-mounts-in-linux/927860#927860 the Linux Kernel mount maximum is 100 000 so hopefully 2000 or 3000 will not be a problem.

--
Possible alternative
I guess (not tested) an alternative could be to hardlink (so needs to be on the same (nfs) file system) every /var/data/chroot/<username>/dev/log to e.g. /var/data/chroot/general_dev_log (which would need access permission for all users of course)
and do
mount --bind /var/data/general_dev_log /var/data/chroot/general_dev_log

so every sftp session would log to the one and only log device (unix socket), again using the local bind mount.
So you could write all the sftp session logs of all users into one log file with syslog-ng.
Advantage:
- only one bind mount needed
- only one syslog-ng unix-stream configuration needed

Disadvantage:
You need to parse and filter the big one session log to create user-specific log files which only conatin the sessions for that user.
I have no experience with such tools and I think this filtering can not be 100% reliable, there will always be some cases where the filter will fail and log entries will be missing.

Best regards
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Aw: Re: Howto log multiple sftpd instances with their chroot shared via NFS [ In reply to ]
> Does the patch idea seem viable?
> A local sshd build allows cleanly solving that as well.

Thanks Peter, but one reason for the new sftp-server cluster HA architecture (and therefore the central NFS mount for the users, leading to this problem)
was to be able to have maintenance of the single sftp servers without service outage, to be able to apply operating system security patches delivered by the distribution (Ubuntu in this case).
I have no capacity to follow the OpenSSH security issues myself an then if needed re-compile newer patched versions (and not even then apply your patch additionally every time to it :)
We do here industry production service and need to stick with vanilla distribution OpenSSH and leave the delivery security patches to Ubuntu.


> Set BindsTo=sshd.service in the sftpd.service [Unit] section to
> tell systemd that sftpd requires sshd, and should be stopped first
> if sshd is being stopped.
>
> Also create a Wants:
>
> mkdir /etc/systemd/system/sshd.service.wants
> ln -s ../sftpd.service /etc/systemd/system/sshd.service.wants/
>
> ..to tell systemd that it should try to start sftpd when sshd starts.
>
> Keep After=sshd.service in sftpd.service.
>

Thanks for the hint, I will look into it, I am not so experienced with systemd yet.
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Howto log multiple sftpd instances with their chroot shared via NFS [ In reply to ]
Sorry for answering this off topic post.

> you could have "vsftpd" to support FTPS rather than SFTP

In the past we offered FTPS to our industry clients, but my collegues which admister the old FTPS server told that FTPS would often be problematic with customers because of firewall/port/certificate issues or something like that (I do not know FTPS), so we migrate customers from FTPS to SFTP, which has simply one TCP port (and luckily, PKI is not established in practice with OpenSSH!).
(if customers insist to use FTPS they can use it on the old server, though)

> convince your user community to switch to FTPS enabled clients

Well we do not have a "user community" but industry clients, some of them big companies, so to change this is practically impossible.
The decision for SFTP was made ten years ago, and we need to support SFTP for decades now on :)

It took years to achieve the sftp service to become reasonably robust stable, there popped up many problems through the years which was then fixed, and with so many customers you experience ever new unexpected fancy client behaviour.
And then we have the forwarding from the sftp service to the backend processing and back, which is also not trivial.

Just to mention, 18 million files transferred via SFTP last month:
inbound 8 252 591
outbound 10 074 881

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Howto log multiple sftpd instances with their chroot shared via NFS [ In reply to ]
Hi Hildegard,

On 1/10/21 2:44 pm, Hildegard Meier wrote:
> mount --bind/var/data/dev/<username>/var/data/chroot/<username>/dev
>
> so/var/data/chroot/<username>/dev is now effectively local on the sftp server, not anymore on nfs mount.

That's not right.  This doesn't magically turn an NFS mount into a local
mount.  If /var/data/chroot/<username>/dev is NFS mounted,
/var/data/dev/<username> is also NFS mounted.

I believe I explained what you need to do but for some reason you didn't
like it, and when I asked why, you never replied.

To repeat myself:

1. Create /var/data/chroot/dev, /var/data/chroot/home and
/var/data/chroot/lib on both machines.

2. Configure your syslog daemon to read from /var/data/chroot/dev/log.

3. Configure SSH to chroot for the SFTP users to /var/data/chroot.

4. Mount your current NFS share which contains all 800 user directories
over /var/data/chroot/home.

For extra points, instead of step 4, although I think is the bit you
didn't like, while at the same time I think is the bit that you said you
really want, so there's cognitive dissonance for you:

4. Mount a NFS share which contains empty directories for all 800 users
over /var/data/chroot/home.

5. When a user logs in use automount to NSF mount their home directory
over /var/data/chroot/home/<username>.

This is not hard.

Regards,

David

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Howto log multiple sftpd instances with their chroot shared via NFS [ In reply to ]
Hi Hildegard,

Hildegard Meier wrote:
> > Does the patch idea seem viable?
..
> I have no capacity to follow the OpenSSH security issues myself an
> then if needed re-compile newer patched versions (and not even then
> apply your patch additionally every time to it :)

Fair enough, although it's not too much effort..

OpenSSH has a few releases per year (including security updates), there's
a release announcement mailing list with no other traffic, and the build
commands I sent need only minimal adjustment for new version numbers in
the git checkout command - or remove the git checkout command to always
build the very latest code, then the commands stay identical, including
applying the patch. The patch is small so should apply cleanly for some
time.


> We do here industry production service and need to stick with
> vanilla distribution OpenSSH and leave the delivery security
> patches to Ubuntu.

Ack. I consider upstream releases to have higher quality, because
distributions usually ship OpenSSH with their own patches, which
every now and then introduce some problems.

And Ubuntu like all other distributions usually take maybe a day or two
even for critical updates, full new releases can take months or years,
so a local build on release day is much faster, but you are right:

A local build does require tracking releases and running commands other
than the standard update process, and while few packages are as easy to
build as OpenSSH I understand that it can still be out of scope.



> > BindsTo=sshd.service
> > mkdir /etc/systemd/system/sshd.service.wants
> > ln -s ../sftpd.service /etc/systemd/system/sshd.service.wants/
> > Keep After=sshd.service in sftpd.service.
>
> Thanks for the hint, I will look into it, I am not so experienced
> with systemd yet.

The systemd dependency data model is simple but very capable and allows
for neat yet precise configuration. I like it a lot. Can recommend.


Hildegard Meier wrote:
> My workaround is the following:
..
> mount --bind /var/data/dev/<username> /var/data/chroot/<username>/dev
>
> so /var/data/chroot/<username>/dev is now effectively local on the
> sftp server, not anymore on nfs mount.

This solution is perfect if per-user mounts are acceptable.


> Then change the syslog-ng config
>
> from
> source s_chroot_<username> { unix-stream("/var/data/chroot/<username>/dev/log" optional(yes) ); };
>
> to
> source s_chroot_<username> { unix-stream("/var/data/dev/<username>/log" optional(yes) ); };
>
> (this is not strictly needed, but I think it's nice having syslog-ng
> definitely now only reading from local file, guaranteed not from nfs
> mount anymore)

I agree, I think this is an important step for reliability.


> I think I will create systemd unit files for every bind mount
> (I experienced systemd dependency cycles with bind mounts in /etc/fstab
> on Ubuntu18 boot before anyway, so systemd units are needed anyway).

If users have local accounts then a generator[1] can create units
dynamically (on boot and on systemctl daemon-reload), otherwise units
must be created and deleted explicitly by user account management.


I've attached a suggested generator script that reads /etc/passwd and
creates units for all members of the group set at the top of the script.
The sshd_config you sent matches on group sftp-cust so I used that.

The generator can be tested in a temporary directory:

dir=$(mktemp -d); ./sftplog-generator.sh "$dir" && ls -lR "$dir"


When all is good copy the script to /etc/systemd/system-generators/
and after running systemctl daemon-reload the generated units will
be in /run/systemd/system-generators/ (never stored on disk) and have
all been loaded.

systemctl restart sftpd.service then mounts everything before
starting the new sftpd process.


> Maybe one can define a systemd dependency so that when you unmount
> the nfs mount /var/data/chroot/, all the bind mounts under it are
> automatically unmounted before (sftpd needs to be stopped first of
> course, otherwise mount would be busy),

Yes, but it requires using systemctl stop var-data-chroot.mount
to unmount. systemd doesn't know about unsuccessful attempts to
directly umount a busy /var/data/chroot, but when stopping the
mount unit it knows to first stop the dependency tree. sftpd can
be included there.


> and also the bind-mounts automatically be mounted after /var/dat/chroot
> is mounted again.

Yes, that's a great idea. So looking at the dependencies:

systemd implicitly[2] adds a Requires= dependency with After= ordering
for directories above the target of mount units, so the generated units
get this for /var/data/chroot automatically, ensuring that /var/data/chroot
is mounted first.

The generator creates units with Before=sftpd.service ordering, it creates
Wants= dependencies for var-data-chroot.mount on each bind mount using a
symlink in var-data-chroot.mount.wants/, and finally it creates
BindsTo= dependencies for sftpd.service on each bind mount using so-called
drop-in[3] files in sftpd.service.d/.

The Wants= symlinks automatically mount the bind mounts when /var/data/chroot
is mounted, and the BindsTo= drop-in files ensure that when sftpd.service
is started all bind mounts are mounted first, and also that if any bind
mount is unmounted then sftpd.service is stopped.

A good example of the dependency model capabilities!


> I think I will implement a nagios monitoring check that checks if
> for every username /var/data/chroot/<username>/dev is a mountpoint,
> so logging is assured.

Can do, but with the above dependencies systemd will never start sftpd
without all bind mounts and will stop sftpd if any bind mount is unmounted.

If stopping sftpd.service is too violent then change BindsTo=${unit}
to Requires=${unit} near the bottom of the generator, but add
BindsTo=var-data-chroot.mount to sftpd.service, otherwise sftpd will not
be stopped when stopping var-data-chroot.mount, causing that to fail.

I like how BindsTo= ensures that sftpd either works fully or not at all,
but that may not suit everyone.



> Possible alternative
> --
> I guess (not tested) an alternative could be to hardlink (so needs
> to be on the same (nfs) file system) every
> /var/data/chroot/<username>/dev/log

Unfortunately it doesn't work; the hardlinks must be (re)created after
syslog-ng creates the general socket, but syslog-ng A and syslog-ng B
will create different general socket inodes, the later will overwrite
the earlier, so it's back to square one.


> Disadvantage:
> You need to parse and filter the big one session log to create
> user-specific log files which only conatin the sessions for that user.
> I have no experience with such tools and I think this filtering can
> not be 100% reliable

Exactly right; sftp-server doesn't include the username in most
messages so it's difficult to reliably associate messages with a user.


Kind regards

//Peter

[1] https://www.freedesktop.org/software/systemd/man/systemd.generator.html
[2] https://www.freedesktop.org/software/systemd/man/systemd.mount.html
[3] https://www.freedesktop.org/software/systemd/man/systemd.unit.html
Re: Howto log multiple sftpd instances with their chroot shared via NFS [ In reply to ]
David,

David Newall wrote:
> I believe I explained what you need to do but for some reason you didn't
> like it, and when I asked why, you never replied.
..
> This is not hard.

I find your tone condescending, even if unintended, and I probably
wouldn't reply to it either. If you'd like to discuss that further
please email me off-list.


Have a nice weekend everyone

//Peter
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Howto log multiple sftpd instances with their chroot shared via NFS [ In reply to ]
All,

On 2/10/21 1:52 am, Peter Stuge wrote:
> David,
>
> David Newall wrote:
>> I believe I explained what you need to do but for some reason you didn't
>> like it, and when I asked why, you never replied.
> ..
>> This is not hard.
> I find your tone condescending, even if unintended

Apologies.  I didn't intend to be condescending.  I was trying to help
with what seems to me to be an already solved problem.

David

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Howto log multiple sftpd instances with their chroot shared via NFS [ In reply to ]
Can confirm this is working in production successfully now.

> Gesendet: Freitag, 01. Oktober 2021 um 06:14 Uhr
> Von: "Hildegard Meier" <daku8938@gmx.de>
> Cc: openssh-unix-dev@mindrot.org
> Betreff: Re: Howto log multiple sftpd instances with their chroot shared via NFS
>
> My workaround is the following:
>
> sudo mkdir /var/data/dev # Create a directroy under which user subdirectories are created
>
> For very username <username> and the user's primary group <groupname> do the following:
>
> sudo mkdir /var/data/dev/<username>
> sudo chmod 550 /var/data/dev/<username> # This restrictive permission is a requirement I think
> sudo chgrp <groupname> /var/data/dev/<username> # so the user can read the directory
>
> So the new directory is exactly the same as the existing /var/data/chroot/<username>/dev directory (which is on the nfs mount /var/data/chroot/).
>
> Then do
> mount --bind /var/data/dev/<username> /var/data/chroot/<username>/dev
>
> so /var/data/chroot/<username>/dev is now effectively local on the sftp server, not anymore on nfs mount.
>
> Then change the syslog-ng config
>
> from
> source s_chroot_<username> { unix-stream("/var/data/chroot/<username>/dev/log" optional(yes) ); };
>
> to
> source s_chroot_<username> { unix-stream("/var/data/dev/<username>/log" optional(yes) ); };
>
> (this is not strictly needed, but I think it's nice having syslog-ng definitely now only reading from local file, guaranteed not from nfs mount anymore)
>
> I have tested this successfully with one user so far - whether the user logs in on the one sftp server or the other, syslog-ng can now log the sftp session on the affected sftp server.
>
> While it is still a mess to have 800 (or in the future maybe 2000 or 3000) bind mounts which I very much would like to avoid, I think this workaround is acceptable because the impact is limited to the logging functionality and not the sftp service itself. Speaking, if there are problems with the bind mounts, the impact is only that there is no sftp session logging.
> Also, this is very simple, uniform and clear.
> And one has 100% reliable sftp session logging.
> No need to change anything else of the well established production system.
>
> I think I will create systemd unit files for every bind mount (I experienced systemd dependency cycles with bind mounts in /etc/fstab on Ubuntu18 boot before anyway, so systemd units are needed anyway).
> Maybe one can define a systemd dependency so that when you unmount the nfs mount /var/data/chroot/, all the bind mounts under it are automatically unmounted before (sftpd needs to be stopped first of course, otherwise mount would be busy), and also the bind-mounts automatically be mounted after /var/dat/chroot is mounted again.
>
> So there should not be much hassle with all those simple uniform bind mounts.
>
> I think I will implement a nagios monitoring check that checks if for every username /var/data/chroot/<username>/dev is a mountpoint, so logging is assured.
>
> According to https://serverfault.com/questions/102588/maximum-numbers-for-file-system-mounts-in-linux/927860#927860 the Linux Kernel mount maximum is 100 000 so hopefully 2000 or 3000 will not be a problem.

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev

1 2 3  View All