Mailing List Archive: ANN: mxProxy Package

ANN: mxProxy Package - Version 0.2.0

Aug 19, 1999, 2:02 PM

Post #1 of 3 (192 views)

ANNOUNCING:

mxProxy Version 0.2.0

A Python extension package providing a configurable
transparent proxy object implementation with
fine grained access control, data hiding and
weak references for Python objects

WHAT IT IS:

mxProxy uses an extension type to wrap objects that you pass
to its contructor in essentially two different ways:

1. via a strong reference which works just like any other
reference in Python or

2. using a weak reference which allows you to build circular
reference models in your data without producing the other-
wise difficult to handle memory leakage induced by this.

Furthermore, the implementation offers some very convenient
access and cleanup protocols which help you control access to
the object (even on very low levels such as type slots) and
provide automatic mechanisms for object cleanup prior to
normal garbage collection.

Wrapping is done in a transparent way, so that you normally
don't even have to recode programs to have them use the
proxy instead of the real object -- only the type possibly
changes.

mxProxy enables you to build secure environment without
having to use techniques like the standard library module
Bastion or restricted environments on a per-object basis.

Several different higher level interfaces for the generic type
are included in the package, such as InstanceProxy, caching
mix-ins, etc.

WHAT'S NEW ?

The 0.2.0 release added a powerful weak reference mechanism
to the package and also fixed some minor bugs.

This release also includes a precompiled Windows DLL so that you can
use the package right away. Installing on that platform boils down
to a simple unzip in the \Python\Lib directory.

WHERE CAN I GET IT ?

The full documentation and instructions for downloading and installing
can be found at:

http://starship.skyport.net/~lemburg/mxProxy.html

WHAT DOES IT COST ?

It comes with a Python-type license, but is free otherwise.

WHERE CAN I GET SUPPORT ?

I am offering commercial support for this package through
Python Professional Services Inc. (http://www.pythonpros.com).
Look on their support pages for details or contact me directly.

REFERENCE:

<P><A HREF="http://starship.skyport.net/~lemburg/mxProxy.html>
mxProxy 0.2.0</A> - generic proxy object providing low-level
access control and weak references. (19-Aug-1999)

--
Marc-Andre Lemburg
______________________________________________________________________
Y2000: 134 days left
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/

ANN: mxProxy Package - Version 0.2.0 [ In reply to ]

nascheme at ucalgary

Aug 19, 1999, 4:14 PM

Post #2 of 3 (189 views)

Permalink

M.-A. Lemburg <mal@lemburg.com> wrote:
>2. using a weak reference which allows you to build circular
> reference models in your data without producing the other-
> wise difficult to handle memory leakage induced by this.

This is an interesting implementation of weak references.
Thanks. Attached is a weak reference implementation I cooked up
yesterday. It is more of a hack then your approach but has the
advantage that referenced objects die sooner. Unfortunately, my
proxy object is incomplete.

Neil

"""
Weak References in Python

This is a hack. It works okay in the current version of CPython. Any object
that has settable attributes can have weak references to it. To get a weak
reference to an object use WeakRef(object). When the object is destroyed any
attempts to access it though the weak reference will raise an exception.

This module requires the refhack extension module, available from:

http://www.ucalgary.ca/~nascheme/python/

Neil Schemenauer <nascheme@ucalgary.ca>
August, 1999
"""

import refhack

# something that cannot conflict with normal Python code
MAGIC_NAME = ' __weakref_cleaner'

WeakRefError = 'WeakRefError'

class WeakRef:
"""Create a weak reference and return a proxy object"""
def __init__(self, obj):
self._alive = 1
self._obj = obj
if not hasattr(self._obj, MAGIC_NAME):
cleaner = WeakRefCleaner(self._obj)
else:
cleaner = getattr(self._obj, MAGIC_NAME)
cleaner.add_ref(self)
refhack.decref(self._obj)

def _destroy(self):
if self._alive:
# _obj is now gone, don't touch it. The refcount of _obj will go
# negitive but this is okay in the current Python implementation
self._alive = 0

def __getattr__(self, name):
if not self._alive:
raise WeakRefError, 'object has been destroyed'
return getattr(self.__dict__['_obj'], name)

#def __setattr__(self, name, value):
# if not self._alive:
# raise WeakRefError, 'object has been destroyed'
# self.__dict__['_obj'].setattr(name, value)

class WeakRefCleaner:
"""A little object to die with the weak referenced object. It
will inform all weak references of the objects death"""
def __init__(self, obj):
self.refs = []
# the object must have settable attributes
setattr(obj, MAGIC_NAME, self)

def add_ref(self, ref):
self.refs.append(ref)

def remove_ref(self, ref):
self.refs.remove(ref)

def __del__(self):
for ref in self.refs:
ref._destroy()

def test():
class A:
pass
a = A()
a.b = 1
r = WeakRef(a)
print r.b
del a
print r.b

if __name__ == '__main__':
test()

ANN: mxProxy Package - Version 0.2.0 [ In reply to ]

mal at lemburg

Aug 20, 1999, 12:45 AM

Post #3 of 3 (191 views)

Permalink

Neil Schemenauer wrote:
>
> M.-A. Lemburg <mal@lemburg.com> wrote:
> >2. using a weak reference which allows you to build circular
> > reference models in your data without producing the other-
> > wise difficult to handle memory leakage induced by this.
>
> This is an interesting implementation of weak references.
> Thanks. Attached is a weak reference implementation I cooked up
> yesterday. It is more of a hack then your approach but has the
> advantage that referenced objects die sooner. Unfortunately, my
> proxy object is incomplete.

You have an interesting implementation there too :-) Both ways
have their merrit I guess. The problem with your idea is that
you need setattr() to work on the object -- this doesn't work
on builtin types, but does do the trick for all Python instances.
My implementation builds on a true proxy that also covers
builtin types (because it also handles type slots, well, at least
most of them: the numeric slots don't work yet because of coercion
issues) and has the advantage of not modifying the referenced
object at all.

Jack Janssen just recently suggested adding a weak bit flag
to all Python objects. This would in the end cause all objects
to grow by about 2-4 bytes depending on struct alignment. With this
flag one could do much the same as suggest in your approach
(using an additional global dict like I do). However the amount
of extra storage needed is significant and probably not even
used in most cases, so the weak bit idea is controversial.

BTW, calling the checkweakrefs() function in mxProxy is pretty
much equivalent to explicitly invoking the GC in JPython.

--
Marc-Andre Lemburg
______________________________________________________________________
Y2000: 133 days left
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/