Mailing List Archive

On 3/8/2024 1:03 PM, Albert-Jan Roskam via Python-list wrote:
> Hi,
> I was replacing some os.path stuff with Pathlib and I discovered this:
> Path(256 * "x").is_file() # OSError
> os.path.isfile(256 * "x") # bool
> Is this intended? Does pathlib try to resemble os.path as closely as
> possible?

You must have an very old version of Python. I'm running 3.12.2 and it
returns False. Either that or that path name exists and throws some
kind of unexpected exception.

The Python docs say

"Return True if the path points to a regular file (or a symbolic link
pointing to a regular file), False if it points to another kind of file.

False is also returned if the path doesn’t exist or is a broken symlink;
other errors (such as permission errors) are propagated"
--
https://mail.python.org/mailman/listinfo/python-list

On 2024-03-08, Thomas Passin via Python-list <python-list@python.org> wrote:
> On 3/8/2024 1:03 PM, Albert-Jan Roskam via Python-list wrote:
>> Hi,
>> I was replacing some os.path stuff with Pathlib and I discovered this:
>> Path(256 * "x").is_file() # OSError
>> os.path.isfile(256 * "x") # bool
>> Is this intended? Does pathlib try to resemble os.path as closely as
>> possible?
>
> You must have an very old version of Python. I'm running 3.12.2 and it
> returns False.

It throws OSError with Python 3.11.8 on Linux.

$ python
Python 3.11.8 (main, Feb 23 2024, 16:11:29) [GCC 13.2.1 20240113] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> pathlib.Path(256 * "x").is_file()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.11/pathlib.py", line 1267, in is_file
return S_ISREG(self.stat().st_mode)
^^^^^^^^^^^
File "/usr/lib/python3.11/pathlib.py", line 1013, in stat
return os.stat(self, follow_symlinks=follow_symlinks)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 36] File name too long: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
>>>
>>> import os
>>> os.path.isfile(256 * "x")
False

--
https://mail.python.org/mailman/listinfo/python-list

On 2024-03-08, Grant Edwards via Python-list <python-list@python.org> wrote:
> On 2024-03-08, Thomas Passin via Python-list <python-list@python.org> wrote:
>> On 3/8/2024 1:03 PM, Albert-Jan Roskam via Python-list wrote:
>>> Hi,
>>> I was replacing some os.path stuff with Pathlib and I discovered this:
>>> Path(256 * "x").is_file() # OSError
>>> os.path.isfile(256 * "x") # bool
>>> Is this intended? Does pathlib try to resemble os.path as closely as
>>> possible?
>>
>> You must have an very old version of Python. I'm running 3.12.2 and it
>> returns False.
>
> It throws OSError with Python 3.11.8 on Linux.

> OSError: [Errno 36] File name too long: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

On all of the Linux filesystems I know about, the max length for a
filename is 255 bytes, so the OSError is too surprising, and it does
seem to follow the documentation.

>>>> import os
>>>> os.path.isfile(256 * "x")
> False

However, os.path.isfile() apprently masks that error somehow and
returns False instead.

I notice that the os.path.isfile() documentation does not specify what
happens if the path is not a file or is illegal. It only specifies
that True is returned if the path is a regular file. Presumably
something other than "return True" is supposed to happen, but exactly
what is not specified.


--
https://mail.python.org/mailman/listinfo/python-list

On 2024-03-08, Grant Edwards via Python-list <python-list@python.org> wrote:

>> OSError: [Errno 36] File name too long: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
>
> On all of the Linux filesystems I know about, the max length for a
> filename is 255 bytes, so the OSError is too surprising, and it does
> seem to follow the documentation.

Doh. I meant "is not too surprising".
--
https://mail.python.org/mailman/listinfo/python-list

On 3/8/2024 2:21 PM, Grant Edwards via Python-list wrote:
> On 2024-03-08, Thomas Passin via Python-list <python-list@python.org> wrote:
>> On 3/8/2024 1:03 PM, Albert-Jan Roskam via Python-list wrote:
>>> Hi,
>>> I was replacing some os.path stuff with Pathlib and I discovered this:
>>> Path(256 * "x").is_file() # OSError
>>> os.path.isfile(256 * "x") # bool
>>> Is this intended? Does pathlib try to resemble os.path as closely as
>>> possible?
>>
>> You must have an very old version of Python. I'm running 3.12.2 and it
>> returns False.
>
> It throws OSError with Python 3.11.8 on Linux.

Sorry, I should have said on Windows.

>
> $ python
> Python 3.11.8 (main, Feb 23 2024, 16:11:29) [GCC 13.2.1 20240113] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import pathlib
>>>> pathlib.Path(256 * "x").is_file()
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/usr/lib/python3.11/pathlib.py", line 1267, in is_file
> return S_ISREG(self.stat().st_mode)
> ^^^^^^^^^^^
> File "/usr/lib/python3.11/pathlib.py", line 1013, in stat
> return os.stat(self, follow_symlinks=follow_symlinks)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> OSError: [Errno 36] File name too long: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
>>>>
>>>> import os
>>>> os.path.isfile(256 * "x")
> False
>

--
https://mail.python.org/mailman/listinfo/python-list

On Mar 8, 2024 19:35, Thomas Passin via Python-list
<python-list@python.org> wrote:

On 3/8/2024 1:03 PM, Albert-Jan Roskam via Python-list wrote:
> Hi,
> I was replacing some os.path stuff with Pathlib and I discovered
this:
> Path(256 * "x").is_file() # OSError
> os.path.isfile(256 * "x") # bool
> Is this intended? Does pathlib try to resemble os.path as closely
as
> possible?

You must have an very old version of Python. I'm running 3.12.2 and it
returns False. Either that or that path name exists and throws some
kind of unexpected exception.

====
Hi, I tested this with Python 3.8. Good to know that this was fixed!
--
https://mail.python.org/mailman/listinfo/python-list

On 3/8/2024 5:14 PM, Albert-Jan Roskam wrote:
>
>
> On Mar 8, 2024 19:35, Thomas Passin via Python-list
> <python-list@python.org> wrote:
>
> On 3/8/2024 1:03 PM, Albert-Jan Roskam via Python-list wrote:
> >     Hi,
> >     I was replacing some os.path stuff with Pathlib and I
> discovered this:
> >     Path(256 * "x").is_file()  # OSError
> >     os.path.isfile(256 * "x")  # bool
> >     Is this intended? Does pathlib try to resemble os.path as
> closely as
> >     possible?
>
> You must have an very old version of Python.  I'm running 3.12.2 and it
> returns False.  Either that or that path name exists and throws some
> kind of unexpected exception.
>
>
>
> ====
>
> Hi, I tested this with Python 3.8. Good to know that this was fixed!

We just learned a few posts back that it might be specific to Linux; I
ran it on Windows.

--
https://mail.python.org/mailman/listinfo/python-list

On 2024-03-08, Thomas Passin via Python-list <python-list@python.org> wrote:
>
>> Hi, I tested this with Python 3.8. Good to know that this was fixed!
>
> We just learned a few posts back that it might be specific to Linux; I
> ran it on Windows.

On Linux, the limit is imposed by the filesystem. Most of the "real"
filesystems on Linux have a 255 character limit, a few support 256,
and some of the legacy filesystems have lower limits. Reiser4 is the
only one that's even remotely common which supports more than 256 --
according to https://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits
it supports filenames up to 3976 bytes long.

NB: The behavior when the limit is exceeded might also vary from one
filesystem to another.

In any case, the pathlib docs for is_file() are explicit: any errors
from the underlying OS and libraries will be propogated. There is
nothing to fix.

https://docs.python.org/3/library/pathlib.html#pathlib.Path.is_file

Path.is_file()

Return True if the path points to a regular file (or a symbolic
link pointing to a regular file), False if it points to another
kind of file.

False is also returned if the path doesn’t exist or is a broken
symlink; other errors (such as permission errors) are propagated.
--
https://mail.python.org/mailman/listinfo/python-list

> On 8 Mar 2024, at 23:19, Thomas Passin via Python-list <python-list@python.org> wrote:
>
> We just learned a few posts back that it might be specific to Linux; I ran it on Windows.

Depending on the exact win32 api used there is a 257 limit on windows.
The 257 includes 2 for the device, C:, and 255 for the path part that will use 1 for the leading \. Getting an error for a name that is 255 is not surprising.

Other api allow for 65535 limit, not sure on its additional limits.

Barry


--
https://mail.python.org/mailman/listinfo/python-list

On 3/10/2024 6:17 AM, Barry wrote:
>
>
>> On 8 Mar 2024, at 23:19, Thomas Passin via Python-list <python-list@python.org> wrote:
>>
>> We just learned a few posts back that it might be specific to Linux; I ran it on Windows.
>
> Depending on the exact win32 api used there is a 257 limit on windows.
> The 257 includes 2 for the device, C:, and 255 for the path part that will use 1 for the leading \. Getting an error for a name that is 255 is not surprising.
>
> Other api allow for 65535 limit, not sure on its additional limits.

I seem to remember there is a setting to allow longer paths, but I
forget any details.

--
https://mail.python.org/mailman/listinfo/python-list

On Mar 10, 2024 12:59, Thomas Passin via Python-list
<python-list@python.org> wrote:

On 3/10/2024 6:17 AM, Barry wrote:
>
>
>> On 8 Mar 2024, at 23:19, Thomas Passin via Python-list
<python-list@python.org> wrote:
>>
>> We just learned a few posts back that it might be specific to Linux;
I ran it on Windows.
>
> Depending on the exact win32 api used there is a 257 limit on windows.
> The 257 includes 2 for the device, C:, and 255 for the path part that
will use 1 for the leading \. Getting an error for a name that is 255 is
not surprising.
>
> Other api allow for 65535 limit, not sure on its additional limits.

I seem to remember there is a setting to allow longer paths, but I
forget any details.

=====
You mean the "\\?\" prefix?
https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry
--
https://mail.python.org/mailman/listinfo/python-list

On 3/10/2024 9:33 AM, Albert-Jan Roskam wrote:
>
>
> On Mar 10, 2024 12:59, Thomas Passin via Python-list
> <python-list@python.org> wrote:
>
> On 3/10/2024 6:17 AM, Barry wrote:
> >
> >
> >> On 8 Mar 2024, at 23:19, Thomas Passin via Python-list
> <python-list@python.org> wrote:
> >>
> >> We just learned a few posts back that it might be specific to
> Linux; I ran it on Windows.
> >
> > Depending on the exact win32 api used there is a 257 limit on
> windows.
> > The 257 includes 2 for the device, C:, and 255 for the path part
> that will use 1 for the leading \. Getting an error for a name that
> is 255 is not surprising.
> >
> > Other api allow for 65535 limit, not sure on its additional limits.
>
> I seem to remember there is a setting to allow longer paths, but I
> forget any details.
>
>
>
> =====
>
> You mean the "\\?\" prefix?
>
> https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry

That and there's a registry setting:

https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation



--
https://mail.python.org/mailman/listinfo/python-list

> On 10 Mar 2024, at 14:49, Thomas Passin via Python-list <python-list@python.org> wrote:
>
> That and there's a registry setting:
>
> https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation
>
Yep that and rules about size of parts of the path.

Barry


--
https://mail.python.org/mailman/listinfo/python-list