Mailing List Archive

On Tue, Sep 1, 2020 at 5:08 AM Richard Damon <Richard@damon-family.org> wrote:
> The file descriptor could remember the path used to get to it. chroot
> shows that .. needs to be somewhat special, as it needs to go away for
> anyone that . is their current root.

But my point is that there might not actually *be* a valid path that
gets you to a file descriptor. It can't remember something that
doesn't exist. (And it's pretty impractical to do that even if it
does.)

> I see no problem with it being a hardlink, and in fact, executables know
> the name they were executed by, so directories knowing the path isn't
> that different.

Actually no, they don't. They are all taught, and being taught,
believe, that their first argument is their name.

> The key differnce between a hardlink and a symlink is
> that hardlinks maintain existance, and always point to something that
> exists (things know how many hardlinks refer to them). symlinks don't
> reference the actual file object, but the symbolic path to it, which may
> or may not actually exist, and who doesn't know such a link exists.

Symlinks refer to a path, which may be relative. Hardlinks refer to an
inode (or whatever other way you choose to identify an actual file's
contents). It's entirely possible to have an open file or directory
that no longer has any actual path referring to it; in fact, things
don't know how many hardlinks refer to them, just how many references
there are.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On 8/31/20 6:05 PM, Chris Angelico wrote:
> On Tue, Sep 1, 2020 at 5:08 AM Richard Damon <Richard@damon-family.org> wrote:
>> The file descriptor could remember the path used to get to it. chroot
>> shows that .. needs to be somewhat special, as it needs to go away for
>> anyone that . is their current root.
> But my point is that there might not actually *be* a valid path that
> gets you to a file descriptor. It can't remember something that
> doesn't exist. (And it's pretty impractical to do that even if it
> does.)

Remember, we are talking about a hypothetical OS that handles hardlinks
to directories, and defines that .. will point to the parent used to
come to it, NOT just having current *NIX allowing hardlinks to
directories with no remediation of the issues cause.

One result of the definition, is that when you open a file/directory, if
it might be a directory, the system WILL need to remember the path to it
(so as to provide a value for ..) and that memory will provide a
'reference' for the directories so they can't go away (just like an
unlinked file stays around will someone has it open). The normal way to
get a file descriptor starts from a path, so the path had to exist in
the first place, and since we are assuming that to get .., it keep a
reference to that path, it can't truly go away.

>
>> I see no problem with it being a hardlink, and in fact, executables know
>> the name they were executed by, so directories knowing the path isn't
>> that different.
> Actually no, they don't. They are all taught, and being taught,
> believe, that their first argument is their name.
>
>> The key differnce between a hardlink and a symlink is
>> that hardlinks maintain existance, and always point to something that
>> exists (things know how many hardlinks refer to them). symlinks don't
>> reference the actual file object, but the symbolic path to it, which may
>> or may not actually exist, and who doesn't know such a link exists.
> Symlinks refer to a path, which may be relative. Hardlinks refer to an
> inode (or whatever other way you choose to identify an actual file's
> contents). It's entirely possible to have an open file or directory
> that no longer has any actual path referring to it; in fact, things
> don't know how many hardlinks refer to them, just how many references
> there are.
>
> ChrisA


--
Richard Damon

--
https://mail.python.org/mailman/listinfo/python-list

On Tue, Sep 1, 2020 at 10:57 PM Richard Damon <Richard@damon-family.org> wrote:
>
> On 8/31/20 6:05 PM, Chris Angelico wrote:
> > On Tue, Sep 1, 2020 at 5:08 AM Richard Damon <Richard@damon-family.org> wrote:
> >> The file descriptor could remember the path used to get to it. chroot
> >> shows that .. needs to be somewhat special, as it needs to go away for
> >> anyone that . is their current root.
> > But my point is that there might not actually *be* a valid path that
> > gets you to a file descriptor. It can't remember something that
> > doesn't exist. (And it's pretty impractical to do that even if it
> > does.)
>
> Remember, we are talking about a hypothetical OS that handles hardlinks
> to directories, and defines that .. will point to the parent used to
> come to it, NOT just having current *NIX allowing hardlinks to
> directories with no remediation of the issues cause.

Yes. I'm assuming that you can define things any way you like.

> One result of the definition, is that when you open a file/directory, if
> it might be a directory, the system WILL need to remember the path to it
> (so as to provide a value for ..) and that memory will provide a
> 'reference' for the directories so they can't go away (just like an
> unlinked file stays around will someone has it open). The normal way to
> get a file descriptor starts from a path, so the path had to exist in
> the first place, and since we are assuming that to get .., it keep a
> reference to that path, it can't truly go away.
>

But as I've pointed out, you don't always *have* a valid path. File
descriptors can be passed from process to process (most commonly by
being retained when you fork and not closed when you exec, but there
are other ways too, eg sockets), so even though starting with a path
is the most common, it isn't the only way, and you could easily have
an FD with no associated path, and then read it to find out what ".."
is.

Also, even if all that could be solved, I don't like the idea that
reading the same directory from two different sources leads to
different results. Is it really the same directory if reading it in
different ways gives different results?

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On 2020-09-01, Richard Damon <Richard@Damon-Family.org> wrote:

> Remember, we are talking about a hypothetical OS that handles hardlinks
> to directories, and defines that .. will point to the parent used to
> come to it, NOT just having current *NIX allowing hardlinks to
> directories with no remediation of the issues cause.

I can testify from personal experience that SunOS 3/4 was in the
latter category. After creating a hard-link to a directory, things
like fsck and the filesystem dump backup utility got very upset and
confused. IIRC the only way to recover was to nuke the involved
inodes then let fsck try to pick up the pieces and put them in the
lost+found. IIRC, I managed to recover without losing any files, but
it wasn't a fun day.

--
Grant


--
https://mail.python.org/mailman/listinfo/python-list

On 9/1/20, Chris Angelico <rosuav@gmail.com> wrote:
>
> Also, even if all that could be solved, I don't like the idea that
> reading the same directory from two different sources leads to
> different results. Is it really the same directory if reading it in
> different ways gives different results?

What's your take on the following example in Linux?

"test2/spam" is a bind mount for "test1/spam", and note that `mount
--bind` in Linux is a namespace operation, i.e. it's not a new device:

>>> os.lstat('test1/spam').st_dev == os.lstat('test2/spam').st_dev
True
>>> os.lstat('test1/spam').st_ino == os.lstat('test2/spam').st_ino
True

According to POSIX (st_dev, st_ino), it's the same directory, yet the
".." entry evaluates depending on the path parsing context:

>>> os.lstat('test1/spam/..').st_ino == os.lstat('test1').st_ino
True
>>> os.lstat('test2/spam/..').st_ino == os.lstat('test2').st_ino
True
--
https://mail.python.org/mailman/listinfo/python-list

On Wed, Sep 2, 2020 at 4:55 AM Eryk Sun <eryksun@gmail.com> wrote:
>
> On 9/1/20, Chris Angelico <rosuav@gmail.com> wrote:
> >
> > Also, even if all that could be solved, I don't like the idea that
> > reading the same directory from two different sources leads to
> > different results. Is it really the same directory if reading it in
> > different ways gives different results?
>
> What's your take on the following example in Linux?
>
> "test2/spam" is a bind mount for "test1/spam", and note that `mount
> --bind` in Linux is a namespace operation, i.e. it's not a new device:
>
> >>> os.lstat('test1/spam').st_dev == os.lstat('test2/spam').st_dev
> True
> >>> os.lstat('test1/spam').st_ino == os.lstat('test2/spam').st_ino
> True
>
> According to POSIX (st_dev, st_ino), it's the same directory, yet the
> ".." entry evaluates depending on the path parsing context:
>
> >>> os.lstat('test1/spam/..').st_ino == os.lstat('test1').st_ino
> True
> >>> os.lstat('test2/spam/..').st_ino == os.lstat('test2').st_ino
> True

When you add ".." to the end of a path, it is very frequently handled
textually. Can you do this exercise by actually opening the
directories and then using openat or statat?

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On 9/1/20, Chris Angelico <rosuav@gmail.com> wrote:
> On Wed, Sep 2, 2020 at 4:55 AM Eryk Sun <eryksun@gmail.com> wrote:
>
>> "test2/spam" is a bind mount for "test1/spam", and note that `mount
>> --bind` in Linux is a namespace operation, i.e. it's not a new device:
>>
>> >>> os.lstat('test1/spam').st_dev == os.lstat('test2/spam').st_dev
>> True
>> >>> os.lstat('test1/spam').st_ino == os.lstat('test2/spam').st_ino
>> True
>>
>> According to POSIX (st_dev, st_ino), it's the same directory, yet the
>> ".." entry evaluates depending on the path parsing context:
>>
>> >>> os.lstat('test1/spam/..').st_ino == os.lstat('test1').st_ino
>> True
>> >>> os.lstat('test2/spam/..').st_ino == os.lstat('test2').st_ino
>> True
>
> When you add ".." to the end of a path, it is very frequently handled
> textually. Can you do this exercise by actually opening the
> directories and then using openat or statat?

The Python example doesn't process the input path to normalize or
resolve it, so I assume you mean text-based processing in the kernel
for the lstat system call. I can switch to using dir_fd, which is
implemented via fstatat in Linux.

Showing again that the bind mountpoint is the same directory:

>>> fd1 = os.open('test1/spam', 0)
>>> fd2 = os.open('test2/spam', 0)
>>> fs1, fs2 = os.fstat(fd1), os.fstat(fd2)
>>> (fs1.st_dev, fs1.st_ino) == (fs2.st_dev, fs2.st_ino)
True

In agreement with the previous example, the ".." entry evaluates
depending on the original path parsing context of the opened fd:

>>> os.lstat('..', dir_fd=fd1).st_ino == os.lstat('test1').st_ino
True
>>> os.lstat('..', dir_fd=fd2).st_ino == os.lstat('test2').st_ino
True
--
https://mail.python.org/mailman/listinfo/python-list

On 9/1/20 9:03 AM, Chris Angelico wrote:
> On Tue, Sep 1, 2020 at 10:57 PM Richard Damon <Richard@damon-family.org> wrote:
>> On 8/31/20 6:05 PM, Chris Angelico wrote:
>>> On Tue, Sep 1, 2020 at 5:08 AM Richard Damon <Richard@damon-family.org> wrote:
>>>> The file descriptor could remember the path used to get to it. chroot
>>>> shows that .. needs to be somewhat special, as it needs to go away for
>>>> anyone that . is their current root.
>>> But my point is that there might not actually *be* a valid path that
>>> gets you to a file descriptor. It can't remember something that
>>> doesn't exist. (And it's pretty impractical to do that even if it
>>> does.)
>> Remember, we are talking about a hypothetical OS that handles hardlinks
>> to directories, and defines that .. will point to the parent used to
>> come to it, NOT just having current *NIX allowing hardlinks to
>> directories with no remediation of the issues cause.
> Yes. I'm assuming that you can define things any way you like.
>
>> One result of the definition, is that when you open a file/directory, if
>> it might be a directory, the system WILL need to remember the path to it
>> (so as to provide a value for ..) and that memory will provide a
>> 'reference' for the directories so they can't go away (just like an
>> unlinked file stays around will someone has it open). The normal way to
>> get a file descriptor starts from a path, so the path had to exist in
>> the first place, and since we are assuming that to get .., it keep a
>> reference to that path, it can't truly go away.
>>
> But as I've pointed out, you don't always *have* a valid path. File
> descriptors can be passed from process to process (most commonly by
> being retained when you fork and not closed when you exec, but there
> are other ways too, eg sockets), so even though starting with a path
> is the most common, it isn't the only way, and you could easily have
> an FD with no associated path, and then read it to find out what ".."
> is.
>
> Also, even if all that could be solved, I don't like the idea that
> reading the same directory from two different sources leads to
> different results. Is it really the same directory if reading it in
> different ways gives different results?
>
> ChrisA

But when you pass the file descriptors to another process, the OS knows
this, so the data that was pointed by the descriptor (which is where you
would keep it anyway) still has it.  There is no normal way that I know
of from user land to get to a directory except from a path or at least
an object that could have remembered a path. For a FD, that FD started
with a path, so it could still remember it.

Part of your issue is likely that you are used to thinking of the file
system as a pure tree, and *nix likes to do that too. Once you accept
hard links to directories as 'normal', suddenly the meaning of .. gets
funny, as there is not a unique parent, but getting the parent as you
got there could be useful for recursive file system searches (which will
need protection from getting stuck in a loop).

My research says that Unix System V allowed them, but restricted them to
super users, to avoid the bigger problems with them. I don't know how it
handle the issue of ..

--
Richard Damon

--
https://mail.python.org/mailman/listinfo/python-list

On 01Sep2020 18:42, Grant Edwards <grant.b.edwards@gmail.com> wrote:
>On 2020-09-01, Richard Damon <Richard@Damon-Family.org> wrote:
>> Remember, we are talking about a hypothetical OS that handles
>> hardlinks
>> to directories, and defines that .. will point to the parent used to
>> come to it, NOT just having current *NIX allowing hardlinks to
>> directories with no remediation of the issues cause.
>
>I can testify from personal experience that SunOS 3/4 was in the
>latter category. After creating a hard-link to a directory, things
>like fsck and the filesystem dump backup utility got very upset and
>confused. IIRC the only way to recover was to nuke the involved
>inodes then let fsck try to pick up the pieces and put them in the
>lost+found. IIRC, I managed to recover without losing any files, but
>it wasn't a fun day.

IIRC this is very old behaviour indeed, in that when I was using V7 UNIX
the manual mentioned that only root could make a hard link to a
directory, and I think there was some allusion to this being how rename
was done for directories, just like files (no rename OS call, just link
to new name, unlink old). That would mean "mv" was setuid root at that
time, which does _not_ ring any bells. So I may be misremembering the
details.

Cheers,
Cameron Simpson <cs@cskk.id.au>
--
https://mail.python.org/mailman/listinfo/python-list

On 2020-09-01, Richard Damon <Richard@Damon-Family.org> wrote:

> My research says that Unix System V allowed them, but restricted them to
> super users, to avoid the bigger problems with them. I don't know how it
> handle the issue of ..

SunOS-3/4 (a BSD 4.x derivative) allowed them, but restricted them to
root. It did not handle the issue of '..' at all, and if you did much
of anything at all with the directory while the "extra" link was
present (which meant the filesystem was no longer a tree) it didn't go
well. It wasn't something you did a second time.

--
Grant




--
https://mail.python.org/mailman/listinfo/python-list

On 2/09/20 6:55 am, Eryk Sun wrote:
> According to POSIX (st_dev, st_ino), it's the same directory, yet the
> ".." entry evaluates depending on the path parsing context:
>
> >>> os.lstat('test1/spam/..').st_ino == os.lstat('test1').st_ino
> True
> >>> os.lstat('test2/spam/..').st_ino == os.lstat('test2').st_ino
> True

What happens if you go one level deeper? I.e. is
os.lstat('test1/spam/eggs/../..').st_ino == os.lstat('test1').st_ino
and
os.lstat('test2/spam/eggs/../..').st_ino == os.lstat('test2').st_ino
?

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

On 9/3/20, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
> On 2/09/20 6:55 am, Eryk Sun wrote:
>> According to POSIX (st_dev, st_ino), it's the same directory, yet the
>> ".." entry evaluates depending on the path parsing context:
>>
>> >>> os.lstat('test1/spam/..').st_ino == os.lstat('test1').st_ino
>> True
>> >>> os.lstat('test2/spam/..').st_ino == os.lstat('test2').st_ino
>> True
>
> What happens if you go one level deeper? I.e. is
> os.lstat('test1/spam/eggs/../..').st_ino == os.lstat('test1').st_ino
> and
> os.lstat('test2/spam/eggs/../..').st_ino == os.lstat('test2').st_ino
> ?

Those two examples return true. Going a level deeper to 'spam/eggs'
doesn't change the behavior of the graft at "test2/spam".

An interesting case is bind mounting a directory on one of its child
directories. For example, if "spam" is mounted on "spam/mount_spam":

>>> s = os.lstat('spam/mount_spam/mount_spam/..')
>>> s.st_ino == os.lstat('spam').st_ino
True

The namespace graft only occurs at "spam/mount_spam". So
"spam/mount_spam/mount_spam" is the original filesystem directory, for
which the ".." entry references "spam". It's also consistent with
"spam/mount_spam" being bound to "spam".
--
https://mail.python.org/mailman/listinfo/python-list

In article <mailman.856.1598858817.9580.python-list@python.org>,
Cameron Simpson <cs@cskk.id.au> wrote:
>
>Yeah, makes me ill. That's because these days "pwd" is usually a shell
>builtin with funny semantics and a cache/sanity=check against $PWD
>(which gets computed as you cd around, typically). And if has a -P
>option and friends explicitly because of this hideous stuff.

Would you be a fan of nash , the Never Again so complicated SHell ?
The base idea that it is a Forth interpreter that has a few
shell like features tucked onto it, like executing programs.
And if you type pwd,
sure as hell it would look through $PATH and execute the exact
program you want.
(You could defined `pwd' in the interpreter yourself to do something
different. If you do that, probably you want it.)

>
>Cheers,
>Cameron Simpson <cs@cskk.id.au>

Groetjes Albert
--
This is the first day of the end of your life.
It may not kill you, but it does make your weaker.
If you can't beat them, too bad.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst
--
https://mail.python.org/mailman/listinfo/python-list

In article <mailman.879.1598986729.9580.python-list@python.org>,
Grant Edwards <grant.b.edwards@gmail.com> wrote:
>On 2020-09-01, Richard Damon <Richard@Damon-Family.org> wrote:
>
>> Remember, we are talking about a hypothetical OS that handles hardlinks
>> to directories, and defines that .. will point to the parent used to
>> come to it, NOT just having current *NIX allowing hardlinks to
>> directories with no remediation of the issues cause.
>
>I can testify from personal experience that SunOS 3/4 was in the
>latter category. After creating a hard-link to a directory, things
>like fsck and the filesystem dump backup utility got very upset and
>confused. IIRC the only way to recover was to nuke the involved
>inodes then let fsck try to pick up the pieces and put them in the
>lost+found. IIRC, I managed to recover without losing any files, but
>it wasn't a fun day.

It was a defect ("bug") in the SUNOS that this was possible.
So reread this thread.
"Let us imagine the situation that a severe, known defect was
reintroduced in linux, just for the fun of it."

>
>--
>Grant

Groetjes Albert
--
This is the first day of the end of your life.
It may not kill you, but it does make your weaker.
If you can't beat them, too bad.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst
--
https://mail.python.org/mailman/listinfo/python-list