Mailing List Archive

Is "shelve" thread-safe on Windows?
Hi All,

I hope to use the shelve module to store data which may be accessed from
several Python CGI scripts at once. I have studied the source-code and it
seems that the anydbm code (that shelve relies on) falls-back to using the
dumbdbm module on Windows which is not safe for concurrent access (according
to comments in its source code).

Can anyone suggest a simple way to gain the functionality of "shelve" on a
Windows platform, but which is safe for handling concurrent accesses (like
from CGI scripts)?

Cheers,

Ian.

--
Any views expressed in this email are my own and not those of Logica UK.
Is "shelve" thread-safe on Windows? [ In reply to ]
Andrew> Can anyone suggest a simple way to gain the functionality of
Andrew> "shelve" on a Windows platform, but which is safe for handling
Andrew> concurrent accesses (like from CGI scripts)?

You might take a look at file locking. On Unix there's the fcntl module.
Not sure if there is similar functionality on Windows.

Skip Montanaro | http://www.mojam.com/
skip@mojam.com | http://www.musi-cal.com/~skip/
847-475-3758
Is "shelve" thread-safe on Windows? [ In reply to ]
Skip Montanaro <skip@mojam.com> wrote:
> Andrew> Can anyone suggest a simple way to gain the functionality of
> Andrew> "shelve" on a Windows platform, but which is safe for handling
> Andrew> concurrent accesses (like from CGI scripts)?
>
> You might take a look at file locking. On Unix there's the fcntl module.
> Not sure if there is similar functionality on Windows.

http://www.python.org/doc/current/lib/msvcrt-files.html

msvcrt.locking(file.fileno(), mode, bytes)

if anyone knows how to use the "mode" argument,
please let us know... (and send doc patches to
Fred!).

</F>
Is "shelve" thread-safe on Windows? [ In reply to ]
I wrote:
>
> http://www.python.org/doc/current/lib/msvcrt-files.html
>
> msvcrt.locking(file.fileno(), mode, bytes)
>
> if anyone knows how to use the "mode" argument,
> please let us know...

okay, okay. Snipped from the Microsoft
docs:

The locking function locks or unlocks nbytes
bytes of the file specified by handle. Locking
bytes in a file prevents access to those bytes
by other processes. All locking or unlocking
begins at the current position of the file
pointer and proceeds for the next nbytes
bytes. It is possible to lock bytes past end
of file.

mode must be one of the following manifest
constants:

_LK_LOCK = 1 # lock the file region

Locks the specified bytes. If the bytes cannot
be locked, the program immediately tries again
after 1 second. If, after 10 attempts, the bytes
cannot be locked, the constant returns an error.

_LK_NBLCK = 2 # non-blocking lock

Locks the specified bytes. If the bytes cannot
be locked, the constant returns an error.

_LK_NBRLCK = 4 # non-blocking lock for writing)

Same as _LK_NBLCK.

_LK_RLCK = 3 # lock for writing

Same as _LK_LOCK.

_LK_UNLCK = 0 # unlock the file region

Unlocks the specified bytes, which must have
been previously locked.

Multiple regions of a file that do not overlap
can be locked. A region being unlocked must
have been previously locked. locking does
not merge adjacent regions; if two locked
regions are adjacent, each region must be
unlocked separately. Regions should be
locked only briefly and should be unlocked
before closing a file or exiting the program.

I'm sure Fred Drake would love if someone could take
this, rephrase it (so it doesn't look as it's cut right out
of Microsoft's copyrighted docs), and submit patches
to the library reference. don't forget to emphasize
that you must use seek (or possibly os.seek(file.fileno())?)
to go to the beginning of the region before you issue
the locking command.

Extra bonus points if you tweak msvcrt so it exports
the various mode flags.

And even more bonus points if you add docstrings
to msvcrt.

Supermega many bonus points for anyone who
implements and documents a locking abstraction
which works on all major unixes *and* windows.

</F>
Is "shelve" thread-safe on Windows? [ In reply to ]
I see a bsddb.pyd included with Python 1.5.1, and I assume with 1.5.2. I
think this means Python shouldn't fall back to dumbdbm on a normal
installation. If I am wrong about that, e.g. if bsddbm.pyd is based on the
1.x series dbm that didn't support concurrent use, I think you can get a
safe version of dbm from www.SleepyCat.com, or maybe the gdbm from
www.roth.net/libs. It should be quite simple to plug in one of those to the
bsddbm or gdbm Python module source.