Mailing List Archive

Re: [Python-ideas] Re: open functions in dbm submodule need to support path-like object
I attempted to do this today, as my first actual contribution to CPython
itself. I think the prior attempt went down a wrong path, which is why
neither PR could actually pass tests.

I've been looking at `posixmodule.c` for comparison, specifically. The key
thing, I believe, is not to use `PyObject *filename` in the
`dpmopen_impl()` calls at all, but rather `path_t *path`. I tried some
absolutely crude instrumentation to see why the `PyOS_FSPath` macro and the
rest wasn't doing what it seems like it should (i.e. sticking `printf()`s
in the .c files). Eventually I realized that argument clinic is stopping
me from even getting to the body of the function when `filename` was a
PosixPath, so anything that PyOS_FSPath does is moot.

I think in the end the new code should be even simpler than the main (3.11)
branch. All I really need to do with a `path_t` is be able to call:

PyObject *self = newgdbmobject(state, name, iflags, mode);

That is, `name` is simply the filename as char* (the other parameters are
independent of the path-like issue).

Does anyone have pointers to an example or documentation about doing this?
The patterns in posixmodule.c are suggestive, but I'm not sure of the
missing pieces.

Also, I copied the typedef for path_t from posixmodule.c to both
`_dbmmodule.c` and `_gdbmmodule.c`. I know that's also going to be wrong,
and it should be pulled into an .h file. Any idea about which one?
`posixmodule.h` seems like a possibility, but then I'd need to repeat it in
`winreparse.h`. Maybe a new name?



On Tue, Sep 7, 2021 at 1:11 PM Guido van Rossum <guido@python.org> wrote:

> If someone posted a Pull Request that added this, it would be looked upon
> favorably I'm sure.
>
> On Tue, Sep 7, 2021 at 8:31 AM Christopher Barker <pythonchb@gmail.com>
> wrote:
>
>> it looks like this as had a BPO for over a year:
>>
>> https://bugs.python.org/issue40563
>>
>> I suggest pinging that (and maybe python-dev) to see what's up.
>>
>> NOTE: first check 3.10 :-)
>>
>> -CHB
>>
>>
>> On Tue, Sep 7, 2021 at 5:05 AM Evan Greenup via Python-ideas <
>> python-ideas@python.org> wrote:
>>
>>> Currently, in Python 3.9, `dbm.open()`, `dbm.gnu.open()` and
>>> `dbm.ndbm.open()` doesn't support path-like object, class defined in
>>> `pathlib`.
>>>
>>> It would be nice to add support with it.
>>>
>>>
Re: [Python-ideas] Re: open functions in dbm submodule need to support path-like object [ In reply to ]
08.09.21 08:19, David Mertz, Ph.D. ????:
> I attempted to do this today, as my first actual contribution to CPython
> itself.  I think the prior attempt went down a wrong path, which is why
> neither PR could actually pass tests.
>
> I've been looking at `posixmodule.c` for comparison, specifically.

The code in posixmodule.c is a bad example, because it is too general
and supports many options. It gives the patch as char* and wchat_t* (on
Windows), supports file descriptors and None, and format error messages
for functions supporting multiple types. But if you only need a path as
char*, you can just use PyUnicode_FSConverter().

There is an existing PR for this issue. It looks correct in general, but
I left some comments.

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LQGU6DM5ZSDPCAXKLEII6YIF4HQI52NG/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Re: open functions in dbm submodule need to support path-like object [ In reply to ]
Thanks Serhiy,

I've made the few additional changes you noted in the PR. I took out my
attempt with path_t. However, here is why I think argument clinic (or
something else?!) is actually intercepting the attempted call:

With my temporary debugging, I have this function in Modules/_gdbmmodule.c:

[clinic start generated code]*/

static PyObject *
dbmopen_impl(PyObject *module, PyObject *filename, const char *flags,
int mode)
/*[clinic end generated code: output=9527750f5df90764
input=812b7d74399ceb0e]*/
{
PyObject_Print(filename, stdout, 0);
printf(" from _gdbmmodule.c (XXX)\n");
/* ... rest of function ...*/

And I have a very simplified test script:

import _gdbm
import sys
from pathlib import Path

print(sys.version)
path = '/tmp/tmp.db'

db = _gdbm.open(path, 'c')
print("Opened with string path")
db.close()

db = _gdbm.open(Path(path), 'c')
print("Opened with path-like")
db.close()

The output of running this is:

3.11.0a0 (heads/bpo-45133-dirty:0376feb030, Sep 8 2021, 00:39:39) [GCC
10.3.0]
'/tmp/tmp.db' from _gdbmmodule.c (XXX)
Opened with string path
Traceback (most recent call last):
File "/home/dmertz/tmp/pathlike-dbm.py", line 12, in <module>
db = _gdbm.open(Path(path), 'c')
^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: open() argument 1 must be str, not PosixPath

So before I get to the first line of the _gdbm.open() function, the
TypeError is already occurring when passed a PosixPath.

On Wed, Sep 8, 2021 at 3:49 AM Serhiy Storchaka <storchaka@gmail.com> wrote:

> 08.09.21 08:19, David Mertz, Ph.D. ????:
> > I attempted to do this today, as my first actual contribution to CPython
> > itself. I think the prior attempt went down a wrong path, which is why
> > neither PR could actually pass tests.
> >
> > I've been looking at `posixmodule.c` for comparison, specifically.
>
> The code in posixmodule.c is a bad example, because it is too general
> and supports many options. It gives the patch as char* and wchat_t* (on
> Windows), supports file descriptors and None, and format error messages
> for functions supporting multiple types. But if you only need a path as
> char*, you can just use PyUnicode_FSConverter().
>
> There is an existing PR for this issue. It looks correct in general, but
> I left some comments.
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/LQGU6DM5ZSDPCAXKLEII6YIF4HQI52NG/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


--
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons. Intellectual property is
to the 21st century what the slave trade was to the 16th.