Mailing List Archive

Descriptors in dataclasses fields
I've been exploring dataclasses for a few months now and they've proven to be very useful.

The only downside is that there's not a simple way to use descriptors.

Descriptors only work on class attributes (as per the docs: https://docs.python.org/3/howto/descriptor.html#closing-thoughts). This means that to use a descriptor in a data class we have to use typing.ClassVar like this

@dataclass
class Item:
name: str
price: typing.ClassVar[Decimal] = PriceValidator()

Which is totally fine because of how descriptors work the previous syntax is a feature in dataclasses, IMHO.

But, there's not a straight forward way to pass a value to the descriptor on init. Because ClassVars are not used by @dataclass to do its thing (as per the docs: https://docs.python.org/3/library/dataclasses.html#class-variables)

This means that in the example above `price` is not going to be a parameter of the class Item's __init__ method. So the only way to do this is to either create an InitVar field or a regular field and then pass the value to the descriptor in __post_init__

@dataclass
class Item:
name: str
price_val: InitVar[Decimal]
price: typing.ClassVar[Decimal] = PriceValidator()

def __post_init__(self, price_val: Decimal) -> None:
self.price = price_val

When using a regular field we can double the field's purpose by making it the field the descriptor is going to use:

@dataclass
class Item:
name: str
_price: Decimal
price: typing.ClassVar[Decimal] = PriceValidator()

def __post_init__(self) -> None:
self.price = self._price

And then in the descriptor implement __set_name__ like so:

def __set_name(self, owner, name):
self.name = f"_{name}"


Personally, I don't like either option because it adds noice to the data class definition. Using an InitVar is the better option because that variable is clearly defined as init only and it's not present in an instance. Using a regular field adds unnecessary noice to the data class.

Also, I think it clashes with the intent of descriptors since they're supposed to manage their data in any way they want.

My questions are:

- Are my assumptions correct?
- Is this the intended behavior? Or what's the preferred way of using a descriptor in a dataclass field?
- If this is not intended, could it be possible to add a `descriptor` parameter to the `field` method and treat the field accordingly?

I couldn't find any information on the docs or the PEP. I could"ve missed something, sorry if this is the case :)

Thanks!

--
Josue
https://www.rmcomplexity.com
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UOMBDIVNRG3DS6UHWSOF4JTLIPXEENCT/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: Descriptors in dataclasses fields [ In reply to ]
I think you are complicating things just because there is no easy way to
tell mypy
that although you are assigning a descriptor to a class variable
in the class body, it will be used as a normal instance attribute
afterwards - and
it is the type used in the instance attribute that mypy should care for,

And then
maybe, the specs of static type checking could get a feature to allow
assigning one thing at class declaration and another thing at instance
working
(otherwise, not only dataclasses, but anything using custom descriptors
would not work with static type checking).

As a workaround, you could just "cast" our descriptor instance to the
type the attribute will actually hold - mypy should not complain:
```
@dataclass
class Item:
price: Decimal = typing.cast(Decimal, MyDescriptor())

```

On Sun, 3 Jan 2021 at 22:48, Josue Balandrano Coronel <jbc@rmcomplexity.com>
wrote:

> I've been exploring dataclasses for a few months now and they've proven to
> be very useful.
>
> The only downside is that there's not a simple way to use descriptors.
>
> Descriptors only work on class attributes (as per the docs:
> https://docs.python.org/3/howto/descriptor.html#closing-thoughts). This
> means that to use a descriptor in a data class we have to use
> typing.ClassVar like this
>
> @dataclass
> class Item:
> name: str
> price: typing.ClassVar[Decimal] = PriceValidator()
>
> Which is totally fine because of how descriptors work the previous syntax
> is a feature in dataclasses, IMHO.
>
> But, there's not a straight forward way to pass a value to the descriptor
> on init. Because ClassVars are not used by @dataclass to do its thing (as
> per the docs:
> https://docs.python.org/3/library/dataclasses.html#class-variables)
>
> This means that in the example above `price` is not going to be a
> parameter of the class Item's __init__ method. So the only way to do this
> is to either create an InitVar field or a regular field and then pass the
> value to the descriptor in __post_init__
>
> @dataclass
> class Item:
> name: str
> price_val: InitVar[Decimal]
> price: typing.ClassVar[Decimal] = PriceValidator()
>
> def __post_init__(self, price_val: Decimal) -> None:
> self.price = price_val
>
> When using a regular field we can double the field's purpose by making it
> the field the descriptor is going to use:
>
> @dataclass
> class Item:
> name: str
> _price: Decimal
> price: typing.ClassVar[Decimal] = PriceValidator()
>
> def __post_init__(self) -> None:
> self.price = self._price
>
> And then in the descriptor implement __set_name__ like so:
>
> def __set_name(self, owner, name):
> self.name = f"_{name}"
>
>
> Personally, I don't like either option because it adds noice to the data
> class definition. Using an InitVar is the better option because that
> variable is clearly defined as init only and it's not present in an
> instance. Using a regular field adds unnecessary noice to the data class.
>
> Also, I think it clashes with the intent of descriptors since they're
> supposed to manage their data in any way they want.
>
> My questions are:
>
> - Are my assumptions correct?
> - Is this the intended behavior? Or what's the preferred way of using a
> descriptor in a dataclass field?
> - If this is not intended, could it be possible to add a `descriptor`
> parameter to the `field` method and treat the field accordingly?
>
> I couldn't find any information on the docs or the PEP. I could"ve missed
> something, sorry if this is the case :)
>
> Thanks!
>
> --
> Josue
> https://www.rmcomplexity.com
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/UOMBDIVNRG3DS6UHWSOF4JTLIPXEENCT/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
Re: Descriptors in dataclasses fields [ In reply to ]
Ah, very interesting.

Just tried this and it worked. That's awesome, thanks!

On Mon, Jan 4, 2021, at 15:45, Joao S. O. Bueno wrote:
> I think you are complicating things just because there is no easy way
> to tell mypy
> that although you are assigning a descriptor to a class variable
> in the class body, it will be used as a normal instance attribute
> afterwards - and
> it is the type used in the instance attribute that mypy should care for,
>
> And then
> maybe, the specs of static type checking could get a feature to allow
> assigning one thing at class declaration and another thing at instance working
> (otherwise, not only dataclasses, but anything using custom descriptors
> would not work with static type checking).
>
> As a workaround, you could just "cast" our descriptor instance to the
> type the attribute will actually hold - mypy should not complain:
> ```
> @dataclass
> class Item:
> price: Decimal = typing.cast(Decimal, MyDescriptor())
>
> ```
>
> On Sun, 3 Jan 2021 at 22:48, Josue Balandrano Coronel
> <jbc@rmcomplexity.com> wrote:
> > I've been exploring dataclasses for a few months now and they've proven to be very useful.
> >
> > The only downside is that there's not a simple way to use descriptors.
> >
> > Descriptors only work on class attributes (as per the docs: https://docs.python.org/3/howto/descriptor.html#closing-thoughts). This means that to use a descriptor in a data class we have to use typing.ClassVar like this
> >
> > @dataclass
> > class Item:
> > name: str
> > price: typing.ClassVar[Decimal] = PriceValidator()
> >
> > Which is totally fine because of how descriptors work the previous syntax is a feature in dataclasses, IMHO.
> >
> > But, there's not a straight forward way to pass a value to the descriptor on init. Because ClassVars are not used by @dataclass to do its thing (as per the docs: https://docs.python.org/3/library/dataclasses.html#class-variables)
> >
> > This means that in the example above `price` is not going to be a parameter of the class Item's __init__ method. So the only way to do this is to either create an InitVar field or a regular field and then pass the value to the descriptor in __post_init__
> >
> > @dataclass
> > class Item:
> > name: str
> > price_val: InitVar[Decimal]
> > price: typing.ClassVar[Decimal] = PriceValidator()
> >
> > def __post_init__(self, price_val: Decimal) -> None:
> > self.price = price_val
> >
> > When using a regular field we can double the field's purpose by making it the field the descriptor is going to use:
> >
> > @dataclass
> > class Item:
> > name: str
> > _price: Decimal
> > price: typing.ClassVar[Decimal] = PriceValidator()
> >
> > def __post_init__(self) -> None:
> > self.price = self._price
> >
> > And then in the descriptor implement __set_name__ like so:
> >
> > def __set_name(self, owner, name):
> > self.name = f"_{name}"
> >
> >
> > Personally, I don't like either option because it adds noice to the data class definition. Using an InitVar is the better option because that variable is clearly defined as init only and it's not present in an instance. Using a regular field adds unnecessary noice to the data class.
> >
> > Also, I think it clashes with the intent of descriptors since they're supposed to manage their data in any way they want.
> >
> > My questions are:
> >
> > - Are my assumptions correct?
> > - Is this the intended behavior? Or what's the preferred way of using a descriptor in a dataclass field?
> > - If this is not intended, could it be possible to add a `descriptor` parameter to the `field` method and treat the field accordingly?
> >
> > I couldn't find any information on the docs or the PEP. I could"ve missed something, sorry if this is the case :)
> >
> > Thanks!
> >
> > --
> > Josue
> > https://www.rmcomplexity.com
> > _______________________________________________
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-leave@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UOMBDIVNRG3DS6UHWSOF4JTLIPXEENCT/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/HRIDJO4VX4F4MNOHODMZJNAOV7NP2VL7/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

--
https://www.rmcomplexity.com
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7CFKKYMVEUQV545R4DWXQOPC2F6UJMCY/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: Descriptors in dataclasses fields [ In reply to ]
Then this is more of an issue with type hints rather than dataclasses. Interesting.

On Mon, Jan 4, 2021, at 16:01, Josue Balandrano Coronel wrote:
> Ah, very interesting.
>
> Just tried this and it worked. That's awesome, thanks!
>
> On Mon, Jan 4, 2021, at 15:45, Joao S. O. Bueno wrote:
> > I think you are complicating things just because there is no easy way
> > to tell mypy
> > that although you are assigning a descriptor to a class variable
> > in the class body, it will be used as a normal instance attribute
> > afterwards - and
> > it is the type used in the instance attribute that mypy should care for,
> >
> > And then
> > maybe, the specs of static type checking could get a feature to allow
> > assigning one thing at class declaration and another thing at instance working
> > (otherwise, not only dataclasses, but anything using custom descriptors
> > would not work with static type checking).
> >
> > As a workaround, you could just "cast" our descriptor instance to the
> > type the attribute will actually hold - mypy should not complain:
> > ```
> > @dataclass
> > class Item:
> > price: Decimal = typing.cast(Decimal, MyDescriptor())
> >
> > ```
> >
> > On Sun, 3 Jan 2021 at 22:48, Josue Balandrano Coronel
> > <jbc@rmcomplexity.com> wrote:
> > > I've been exploring dataclasses for a few months now and they've proven to be very useful.
> > >
> > > The only downside is that there's not a simple way to use descriptors.
> > >
> > > Descriptors only work on class attributes (as per the docs: https://docs.python.org/3/howto/descriptor.html#closing-thoughts). This means that to use a descriptor in a data class we have to use typing.ClassVar like this
> > >
> > > @dataclass
> > > class Item:
> > > name: str
> > > price: typing.ClassVar[Decimal] = PriceValidator()
> > >
> > > Which is totally fine because of how descriptors work the previous syntax is a feature in dataclasses, IMHO.
> > >
> > > But, there's not a straight forward way to pass a value to the descriptor on init. Because ClassVars are not used by @dataclass to do its thing (as per the docs: https://docs.python.org/3/library/dataclasses.html#class-variables)
> > >
> > > This means that in the example above `price` is not going to be a parameter of the class Item's __init__ method. So the only way to do this is to either create an InitVar field or a regular field and then pass the value to the descriptor in __post_init__
> > >
> > > @dataclass
> > > class Item:
> > > name: str
> > > price_val: InitVar[Decimal]
> > > price: typing.ClassVar[Decimal] = PriceValidator()
> > >
> > > def __post_init__(self, price_val: Decimal) -> None:
> > > self.price = price_val
> > >
> > > When using a regular field we can double the field's purpose by making it the field the descriptor is going to use:
> > >
> > > @dataclass
> > > class Item:
> > > name: str
> > > _price: Decimal
> > > price: typing.ClassVar[Decimal] = PriceValidator()
> > >
> > > def __post_init__(self) -> None:
> > > self.price = self._price
> > >
> > > And then in the descriptor implement __set_name__ like so:
> > >
> > > def __set_name(self, owner, name):
> > > self.name = f"_{name}"
> > >
> > >
> > > Personally, I don't like either option because it adds noice to the data class definition. Using an InitVar is the better option because that variable is clearly defined as init only and it's not present in an instance. Using a regular field adds unnecessary noice to the data class.
> > >
> > > Also, I think it clashes with the intent of descriptors since they're supposed to manage their data in any way they want.
> > >
> > > My questions are:
> > >
> > > - Are my assumptions correct?
> > > - Is this the intended behavior? Or what's the preferred way of using a descriptor in a dataclass field?
> > > - If this is not intended, could it be possible to add a `descriptor` parameter to the `field` method and treat the field accordingly?
> > >
> > > I couldn't find any information on the docs or the PEP. I could"ve missed something, sorry if this is the case :)
> > >
> > > Thanks!
> > >
> > > --
> > > Josue
> > > https://www.rmcomplexity.com
> > > _______________________________________________
> > > Python-Dev mailing list -- python-dev@python.org
> > > To unsubscribe send an email to python-dev-leave@python.org
> > > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > > Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UOMBDIVNRG3DS6UHWSOF4JTLIPXEENCT/
> > > Code of Conduct: http://python.org/psf/codeofconduct/
> > _______________________________________________
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-leave@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> > https://mail.python.org/archives/list/python-dev@python.org/message/HRIDJO4VX4F4MNOHODMZJNAOV7NP2VL7/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >
>
> --
> https://www.rmcomplexity.com
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/7CFKKYMVEUQV545R4DWXQOPC2F6UJMCY/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

--
https://www.rmcomplexity.com
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/A6XL5GVEO74VT2OTWZMDS2FUNAC5FN3E/
Code of Conduct: http://python.org/psf/codeofconduct/