Mailing List Archive

PEP Idea: Real private attribute
Python currently uses name mangling for double-underscore attributes. Name
mangling is not an ideal method to avoid name conflicting. There are
various normal programming patterns that can simply cause name conflicting
in double-underscore members. A typical example is when a class is
re-decorated using the same decorator. The decorator can not take
double-underscore members without name conflicts. For example:

```
@custom_decorator("a")
@custom_decorator("b")
class C:
pass
```

The `@custom_decorator` wrapper may need to hold private members, but
Python's current name conflict resolution does not provide any solution and
the decorator cannot hold private members without applying tricky
programming methods.

Another example is when a class inherits from a base class of the same name.

```
class View:
"""A class representing a view of an object; similar to
numpy.ndarray.view"""
pass

class Object:
class View(View):
"""A view class costumized for objects of type Object"""
pass
```

Again, in this example, class `Object.View` can't take double-underscore
names without conflicting with `View`'s.

My idea is to introduce real private members (by which I do not mean to be
inaccessible from outside the class, but to be guaranteed not to conflict
with other private members of the same object). These private members are
started with triple underscores and are stored in a separate dictionary
named `__privs__`. Unlike `__dict__` that takes 'str' keys, `__privs__`
will be a double layer dictionary that takes 'type' keys in the first
level, and 'str' keys in the second level.

For example, assume that the user runs the following code:
```
class C:
def __init__(self, value):
self.___member = value

c = C("my value")
```

On the last line, Python's attribute setter creates a new entry in the
dictionary with key `C`, adds the value "my value" to a new entry with the
key 'member'.

The user can then retrieve `c.___member` by invoking the `__privs__`
dictionary:

```
print(c.__privs__[C]['member']) # prints 'my value'
```

Note that, unlike class names, class objects are unique and there will not
be any conflicts. Python classes are hashable and can be dictionary keys.
Personally, I do not see any disadvantage of using __privs__ over name
mangling/double-underscores. While name mangling does not truly guarantee
conflict resolution, __privs__ does.

Please discuss the idea, let me know what you think about it, whether there
are possible disadvantages, and if you think it will be approved as a PEP.

Thanks,
Mehrzad Saremi

AI M.Sc. grad. from AUT
--
https://mail.python.org/mailman/listinfo/python-list
Re: PEP Idea: Real private attribute [ In reply to ]
On Sun, Aug 29, 2021 at 7:40 AM Mehrzad Saremi <mehrzad.1024@gmail.com> wrote:
>
> Python currently uses name mangling for double-underscore attributes. Name
> mangling is not an ideal method to avoid name conflicting. There are
> various normal programming patterns that can simply cause name conflicting
> in double-underscore members. A typical example is when a class is
> re-decorated using the same decorator. The decorator can not take
> double-underscore members without name conflicts. For example:
>
> ```
> @custom_decorator("a")
> @custom_decorator("b")
> class C:
> pass
> ```
>
> The `@custom_decorator` wrapper may need to hold private members, but
> Python's current name conflict resolution does not provide any solution and
> the decorator cannot hold private members without applying tricky
> programming methods.
>
> Another example is when a class inherits from a base class of the same name.
>
> ```
> class View:
> """A class representing a view of an object; similar to
> numpy.ndarray.view"""
> pass
>
> class Object:
> class View(View):
> """A view class costumized for objects of type Object"""
> pass
> ```
>
> Again, in this example, class `Object.View` can't take double-underscore
> names without conflicting with `View`'s.
>
> My idea is to introduce real private members (by which I do not mean to be
> inaccessible from outside the class, but to be guaranteed not to conflict
> with other private members of the same object). These private members are
> started with triple underscores and are stored in a separate dictionary
> named `__privs__`. Unlike `__dict__` that takes 'str' keys, `__privs__`
> will be a double layer dictionary that takes 'type' keys in the first
> level, and 'str' keys in the second level.
>
> For example, assume that the user runs the following code:
> ```
> class C:
> def __init__(self, value):
> self.___member = value
>
> c = C("my value")
> ```
>
> On the last line, Python's attribute setter creates a new entry in the
> dictionary with key `C`, adds the value "my value" to a new entry with the
> key 'member'.
>
> The user can then retrieve `c.___member` by invoking the `__privs__`
> dictionary:
>
> ```
> print(c.__privs__[C]['member']) # prints 'my value'
> ```
>
> Note that, unlike class names, class objects are unique and there will not
> be any conflicts. Python classes are hashable and can be dictionary keys.
> Personally, I do not see any disadvantage of using __privs__ over name
> mangling/double-underscores. While name mangling does not truly guarantee
> conflict resolution, __privs__ does.

Not entirely sure how it would know the right type to use (subclassing
makes that tricky), but whatever your definition is, there's nothing
stopping you from doing it yourself. Don't forget that you have
__class__ available if you need to refer to "the class that I'm
lexically inside" (that's how the zero-arg super() function works), so
you might do something like self.__privs__[__class__, "foo"] to refer
to a thing.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: PEP Idea: Real private attribute [ In reply to ]
No, a class ("the class that I'm lexically inside") cannot be accessed from
outside of the class. This is why I'm planning to offer it as a core
feature because only the parser would know. There's apparently no elegant
solution if you want to implement it yourself. You'll need to write
self.__privs__[__class__, "foo"], whenever you want to use the feature and
even wrapping it in superclasses won't remedy it, because the parent class
isn't aware which class you're inside. It seems to me name mangling must
have been an ad-hoc solution in a language that it doesn't really fit when
it could have been implemented in a much more cogent way.

Best,
[image: image.gif][image: image.gif]

Mehrzad


On Sun, 29 Aug 2021 at 02:18, Chris Angelico <rosuav@gmail.com> wrote:

> On Sun, Aug 29, 2021 at 7:40 AM Mehrzad Saremi <mehrzad.1024@gmail.com>
> wrote:
> >
> > Python currently uses name mangling for double-underscore attributes.
> Name
> > mangling is not an ideal method to avoid name conflicting. There are
> > various normal programming patterns that can simply cause name
> conflicting
> > in double-underscore members. A typical example is when a class is
> > re-decorated using the same decorator. The decorator can not take
> > double-underscore members without name conflicts. For example:
> >
> > ```
> > @custom_decorator("a")
> > @custom_decorator("b")
> > class C:
> > pass
> > ```
> >
> > The `@custom_decorator` wrapper may need to hold private members, but
> > Python's current name conflict resolution does not provide any solution
> and
> > the decorator cannot hold private members without applying tricky
> > programming methods.
> >
> > Another example is when a class inherits from a base class of the same
> name.
> >
> > ```
> > class View:
> > """A class representing a view of an object; similar to
> > numpy.ndarray.view"""
> > pass
> >
> > class Object:
> > class View(View):
> > """A view class costumized for objects of type Object"""
> > pass
> > ```
> >
> > Again, in this example, class `Object.View` can't take double-underscore
> > names without conflicting with `View`'s.
> >
> > My idea is to introduce real private members (by which I do not mean to
> be
> > inaccessible from outside the class, but to be guaranteed not to conflict
> > with other private members of the same object). These private members are
> > started with triple underscores and are stored in a separate dictionary
> > named `__privs__`. Unlike `__dict__` that takes 'str' keys, `__privs__`
> > will be a double layer dictionary that takes 'type' keys in the first
> > level, and 'str' keys in the second level.
> >
> > For example, assume that the user runs the following code:
> > ```
> > class C:
> > def __init__(self, value):
> > self.___member = value
> >
> > c = C("my value")
> > ```
> >
> > On the last line, Python's attribute setter creates a new entry in the
> > dictionary with key `C`, adds the value "my value" to a new entry with
> the
> > key 'member'.
> >
> > The user can then retrieve `c.___member` by invoking the `__privs__`
> > dictionary:
> >
> > ```
> > print(c.__privs__[C]['member']) # prints 'my value'
> > ```
> >
> > Note that, unlike class names, class objects are unique and there will
> not
> > be any conflicts. Python classes are hashable and can be dictionary keys.
> > Personally, I do not see any disadvantage of using __privs__ over name
> > mangling/double-underscores. While name mangling does not truly guarantee
> > conflict resolution, __privs__ does.
>
> Not entirely sure how it would know the right type to use (subclassing
> makes that tricky), but whatever your definition is, there's nothing
> stopping you from doing it yourself. Don't forget that you have
> __class__ available if you need to refer to "the class that I'm
> lexically inside" (that's how the zero-arg super() function works), so
> you might do something like self.__privs__[__class__, "foo"] to refer
> to a thing.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: PEP Idea: Real private attribute [ In reply to ]
On Mon, Aug 30, 2021 at 5:49 AM Mehrzad Saremi <mehrzad.1024@gmail.com> wrote:
>
> No, a class ("the class that I'm lexically inside") cannot be accessed from
> outside of the class. This is why I'm planning to offer it as a core
> feature because only the parser would know. There's apparently no elegant
> solution if you want to implement it yourself. You'll need to write
> self.__privs__[__class__, "foo"], whenever you want to use the feature and
> even wrapping it in superclasses won't remedy it, because the parent class
> isn't aware which class you're inside. It seems to me name mangling must
> have been an ad-hoc solution in a language that it doesn't really fit when
> it could have been implemented in a much more cogent way.
>

If the parent class isn't aware which class you're in, how is the
language going to define it?

Can you give a full run-down of the semantics of your proposed privs,
and how it's different from something like you just used above -
self.__privs__[__class__, "foo"] - ? If the problem is the ugliness
alone, then say so; but also, how this would work with decorators,
since you specifically mention them as a use-case.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: PEP Idea: Real private attribute [ In reply to ]
The proposed semantics would be the same as self.__privs__[__class__,
"foo"]; yes I can say the problem is ugliness. The following is an example
where name mangling can be problematic (of course there are workarounds,
yet if double-underscores are meant to represent class-specific members,
the following behavior is an infringement of the purpose).

```
class Weighable:
def weight(self):
raise NotImplementedError()


class AddWeight:
def __init__(self, weight):
self.weight = weight

def __call__(self, cls):
class Wrapper(cls, Weighable):
__weight = self.weight

def weight(self):
return self.__weight + (cls.weight(self) if issubclass(cls, Weighable) else
0) # Unexpected behavior

return Wrapper


@AddWeight(2.0)
@AddWeight(1.0)
class C:
pass


print(C().weight())
```

> If the parent class isn't aware which class you're in, how is the
language going to define it?

I mean if you want to implement self.__privs__[__class__, "foo"] in a
parent class using __setattr__/__getattribute__ the __class__ value is
unknown.


On Mon, 30 Aug 2021 at 00:26, Chris Angelico <rosuav@gmail.com> wrote:

> On Mon, Aug 30, 2021 at 5:49 AM Mehrzad Saremi <mehrzad.1024@gmail.com>
> wrote:
> >
> > No, a class ("the class that I'm lexically inside") cannot be accessed
> from
> > outside of the class. This is why I'm planning to offer it as a core
> > feature because only the parser would know. There's apparently no elegant
> > solution if you want to implement it yourself. You'll need to write
> > self.__privs__[__class__, "foo"], whenever you want to use the feature
> and
> > even wrapping it in superclasses won't remedy it, because the parent
> class
> > isn't aware which class you're inside. It seems to me name mangling must
> > have been an ad-hoc solution in a language that it doesn't really fit
> when
> > it could have been implemented in a much more cogent way.
> >
>
> If the parent class isn't aware which class you're in, how is the
> language going to define it?
>
> Can you give a full run-down of the semantics of your proposed privs,
> and how it's different from something like you just used above -
> self.__privs__[__class__, "foo"] - ? If the problem is the ugliness
> alone, then say so; but also, how this would work with decorators,
> since you specifically mention them as a use-case.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: PEP Idea: Real private attribute [ In reply to ]
The right way for those decorators to hold some private information, imho,
isn't to put anything on the decorated object at all, but to use a weak-ref
dictionary using the target object as a key.

On Sat, Aug 28, 2021 at 5:42 PM Mehrzad Saremi <mehrzad.1024@gmail.com>
wrote:

> Python currently uses name mangling for double-underscore attributes. Name
> mangling is not an ideal method to avoid name conflicting. There are
> various normal programming patterns that can simply cause name conflicting
> in double-underscore members. A typical example is when a class is
> re-decorated using the same decorator. The decorator can not take
> double-underscore members without name conflicts. For example:
>
> ```
> @custom_decorator("a")
> @custom_decorator("b")
> class C:
> pass
> ```
>
> The `@custom_decorator` wrapper may need to hold private members, but
> Python's current name conflict resolution does not provide any solution and
> the decorator cannot hold private members without applying tricky
> programming methods.
>
> Another example is when a class inherits from a base class of the same
> name.
>
> ```
> class View:
> """A class representing a view of an object; similar to
> numpy.ndarray.view"""
> pass
>
> class Object:
> class View(View):
> """A view class costumized for objects of type Object"""
> pass
> ```
>
> Again, in this example, class `Object.View` can't take double-underscore
> names without conflicting with `View`'s.
>
> My idea is to introduce real private members (by which I do not mean to be
> inaccessible from outside the class, but to be guaranteed not to conflict
> with other private members of the same object). These private members are
> started with triple underscores and are stored in a separate dictionary
> named `__privs__`. Unlike `__dict__` that takes 'str' keys, `__privs__`
> will be a double layer dictionary that takes 'type' keys in the first
> level, and 'str' keys in the second level.
>
> For example, assume that the user runs the following code:
> ```
> class C:
> def __init__(self, value):
> self.___member = value
>
> c = C("my value")
> ```
>
> On the last line, Python's attribute setter creates a new entry in the
> dictionary with key `C`, adds the value "my value" to a new entry with the
> key 'member'.
>
> The user can then retrieve `c.___member` by invoking the `__privs__`
> dictionary:
>
> ```
> print(c.__privs__[C]['member']) # prints 'my value'
> ```
>
> Note that, unlike class names, class objects are unique and there will not
> be any conflicts. Python classes are hashable and can be dictionary keys.
> Personally, I do not see any disadvantage of using __privs__ over name
> mangling/double-underscores. While name mangling does not truly guarantee
> conflict resolution, __privs__ does.
>
> Please discuss the idea, let me know what you think about it, whether there
> are possible disadvantages, and if you think it will be approved as a PEP.
>
> Thanks,
> Mehrzad Saremi
>
> AI M.Sc. grad. from AUT
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>

--

CALVIN SPEALMAN

SENIOR QUALITY ENGINEER

calvin.spealman@redhat.com M: +1.336.210.5107
[image: https://red.ht/sig] <https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
--
https://mail.python.org/mailman/listinfo/python-list
Re: PEP Idea: Real private attribute [ In reply to ]
Calvin, even if the language offered truly private members?

On Tue, 31 Aug 2021 at 17:31, Calvin Spealman <cspealma@redhat.com> wrote:

> The right way for those decorators to hold some private information, imho,
> isn't to put anything on the decorated object at all, but to use a weak-ref
> dictionary using the target object as a key.
>
> On Sat, Aug 28, 2021 at 5:42 PM Mehrzad Saremi <mehrzad.1024@gmail.com>
> wrote:
>
>> Python currently uses name mangling for double-underscore attributes. Name
>> mangling is not an ideal method to avoid name conflicting. There are
>> various normal programming patterns that can simply cause name conflicting
>> in double-underscore members. A typical example is when a class is
>> re-decorated using the same decorator. The decorator can not take
>> double-underscore members without name conflicts. For example:
>>
>> ```
>> @custom_decorator("a")
>> @custom_decorator("b")
>> class C:
>> pass
>> ```
>>
>> The `@custom_decorator` wrapper may need to hold private members, but
>> Python's current name conflict resolution does not provide any solution
>> and
>> the decorator cannot hold private members without applying tricky
>> programming methods.
>>
>> Another example is when a class inherits from a base class of the same
>> name.
>>
>> ```
>> class View:
>> """A class representing a view of an object; similar to
>> numpy.ndarray.view"""
>> pass
>>
>> class Object:
>> class View(View):
>> """A view class costumized for objects of type Object"""
>> pass
>> ```
>>
>> Again, in this example, class `Object.View` can't take double-underscore
>> names without conflicting with `View`'s.
>>
>> My idea is to introduce real private members (by which I do not mean to be
>> inaccessible from outside the class, but to be guaranteed not to conflict
>> with other private members of the same object). These private members are
>> started with triple underscores and are stored in a separate dictionary
>> named `__privs__`. Unlike `__dict__` that takes 'str' keys, `__privs__`
>> will be a double layer dictionary that takes 'type' keys in the first
>> level, and 'str' keys in the second level.
>>
>> For example, assume that the user runs the following code:
>> ```
>> class C:
>> def __init__(self, value):
>> self.___member = value
>>
>> c = C("my value")
>> ```
>>
>> On the last line, Python's attribute setter creates a new entry in the
>> dictionary with key `C`, adds the value "my value" to a new entry with the
>> key 'member'.
>>
>> The user can then retrieve `c.___member` by invoking the `__privs__`
>> dictionary:
>>
>> ```
>> print(c.__privs__[C]['member']) # prints 'my value'
>> ```
>>
>> Note that, unlike class names, class objects are unique and there will not
>> be any conflicts. Python classes are hashable and can be dictionary keys.
>> Personally, I do not see any disadvantage of using __privs__ over name
>> mangling/double-underscores. While name mangling does not truly guarantee
>> conflict resolution, __privs__ does.
>>
>> Please discuss the idea, let me know what you think about it, whether
>> there
>> are possible disadvantages, and if you think it will be approved as a PEP.
>>
>> Thanks,
>> Mehrzad Saremi
>>
>> AI M.Sc. grad. from AUT
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>>
>
> --
>
> CALVIN SPEALMAN
>
> SENIOR QUALITY ENGINEER
>
> calvin.spealman@redhat.com M: +1.336.210.5107
> [image: https://red.ht/sig] <https://red.ht/sig>
> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: PEP Idea: Real private attribute [ In reply to ]
On Tue, Aug 31, 2021 at 1:19 PM Mehrzad Saremi <mehrzad.1024@gmail.com>
wrote:

> Calvin, even if the language offered truly private members?
>

I'm saying I don't think they're necessary, especially not for the use case
posited here. Private members in other languages are about things internal
to the class of the object itself. If we *did* have them in Python, you
couldn't use private members of an object from a decorator on it because
the decorator is external to the target class.


> On Tue, 31 Aug 2021 at 17:31, Calvin Spealman <cspealma@redhat.com> wrote:
>
>> The right way for those decorators to hold some private information,
>> imho, isn't to put anything on the decorated object at all, but to use a
>> weak-ref dictionary using the target object as a key.
>>
>> On Sat, Aug 28, 2021 at 5:42 PM Mehrzad Saremi <mehrzad.1024@gmail.com>
>> wrote:
>>
>>> Python currently uses name mangling for double-underscore attributes.
>>> Name
>>> mangling is not an ideal method to avoid name conflicting. There are
>>> various normal programming patterns that can simply cause name
>>> conflicting
>>> in double-underscore members. A typical example is when a class is
>>> re-decorated using the same decorator. The decorator can not take
>>> double-underscore members without name conflicts. For example:
>>>
>>> ```
>>> @custom_decorator("a")
>>> @custom_decorator("b")
>>> class C:
>>> pass
>>> ```
>>>
>>> The `@custom_decorator` wrapper may need to hold private members, but
>>> Python's current name conflict resolution does not provide any solution
>>> and
>>> the decorator cannot hold private members without applying tricky
>>> programming methods.
>>>
>>> Another example is when a class inherits from a base class of the same
>>> name.
>>>
>>> ```
>>> class View:
>>> """A class representing a view of an object; similar to
>>> numpy.ndarray.view"""
>>> pass
>>>
>>> class Object:
>>> class View(View):
>>> """A view class costumized for objects of type Object"""
>>> pass
>>> ```
>>>
>>> Again, in this example, class `Object.View` can't take double-underscore
>>> names without conflicting with `View`'s.
>>>
>>> My idea is to introduce real private members (by which I do not mean to
>>> be
>>> inaccessible from outside the class, but to be guaranteed not to conflict
>>> with other private members of the same object). These private members are
>>> started with triple underscores and are stored in a separate dictionary
>>> named `__privs__`. Unlike `__dict__` that takes 'str' keys, `__privs__`
>>> will be a double layer dictionary that takes 'type' keys in the first
>>> level, and 'str' keys in the second level.
>>>
>>> For example, assume that the user runs the following code:
>>> ```
>>> class C:
>>> def __init__(self, value):
>>> self.___member = value
>>>
>>> c = C("my value")
>>> ```
>>>
>>> On the last line, Python's attribute setter creates a new entry in the
>>> dictionary with key `C`, adds the value "my value" to a new entry with
>>> the
>>> key 'member'.
>>>
>>> The user can then retrieve `c.___member` by invoking the `__privs__`
>>> dictionary:
>>>
>>> ```
>>> print(c.__privs__[C]['member']) # prints 'my value'
>>> ```
>>>
>>> Note that, unlike class names, class objects are unique and there will
>>> not
>>> be any conflicts. Python classes are hashable and can be dictionary keys.
>>> Personally, I do not see any disadvantage of using __privs__ over name
>>> mangling/double-underscores. While name mangling does not truly guarantee
>>> conflict resolution, __privs__ does.
>>>
>>> Please discuss the idea, let me know what you think about it, whether
>>> there
>>> are possible disadvantages, and if you think it will be approved as a
>>> PEP.
>>>
>>> Thanks,
>>> Mehrzad Saremi
>>>
>>> AI M.Sc. grad. from AUT
>>> --
>>> https://mail.python.org/mailman/listinfo/python-list
>>>
>>>
>>
>> --
>>
>> CALVIN SPEALMAN
>>
>> SENIOR QUALITY ENGINEER
>>
>> calvin.spealman@redhat.com M: +1.336.210.5107
>> [image: https://red.ht/sig] <https://red.ht/sig>
>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>
>

--

CALVIN SPEALMAN

SENIOR QUALITY ENGINEER

calvin.spealman@redhat.com M: +1.336.210.5107
[image: https://red.ht/sig] <https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
--
https://mail.python.org/mailman/listinfo/python-list