Mailing List Archive

On Thu, Apr 15, 2021 at 11:09 AM Larry Hastings <larry@hastings.org> wrote:
>
> Thanks for doing this! I don't think PEP 649 is going to be accepted or rejected based on either performance or memory usage, but it's nice to see you confirmed that its performance and memory impact is acceptable.
>
>
> If I run "ann_test.py 1", the annotations are already turned into strings. Why do you do it that way? It makes stock semantics look better, because manually stringized annotations are much faster than evaluating real expressions.
>

Because `if TYPE_CHECKING` and manually stringified annotation is used
in real world applications.
I want to mix both use cases.

--
Inada Naoki <songofacandy@gmail.com>
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MKAGFBTVZ4LXYJC5X6P3LXXXRD7P2WH5/
Code of Conduct: http://python.org/psf/codeofconduct/

I updated the benchmark little:

* Added no annotation mode for baseline performance.
* Better stats output.

https://gist.github.com/methane/abb509e5f781cc4a103cc450e1e7925d

```
# No annotation (master + GH-25419)
$ ./python ~/ann_test.py 0
code size: 102967 bytes
memory: 181288 bytes
unmarshal: avg: 299.301ms +/-1.257ms
exec: avg: 104.019ms +/-0.038ms

# PEP 563 (master + GH-25419)
$ ./python ~/ann_test.py 2
code size: 110488 bytes
memory: 193572 bytes
unmarshal: avg: 313.032ms +/-0.068ms
exec: avg: 108.456ms +/-0.048ms

# PEP 649 (co_annotations + GH-25419 + GH-23056)
$ ./python ~/ann_test.py 3
code size: 204963 bytes
memory: 209257 bytes
unmarshal: avg: 587.336ms +/-2.073ms
exec: avg: 97.056ms +/-0.046ms

# Python 3.9
$ python3 ann_test.py 0
code size: 108955 bytes
memory: 173296 bytes
unmarshal: avg: 333.527ms +/-1.750ms
exec: avg: 90.810ms +/-0.347ms

$ python3 ann_test.py 1
code size: 121011 bytes
memory: 385200 bytes
unmarshal: avg: 334.929ms +/-0.055ms
exec: avg: 400.260ms +/-0.249ms
```

## Rough estimation of annotation overhead

Python 3.9 w/o PEP 563
code (pyc) size: +11%
memory usage: +122% (211bytes / function)
import time: +73% (*)

PEP 563
code (pyc) size: +7.3%
memory usage: +0.68% (13.3bytes / function)
import time: +4.5%

PEP 649
code (pyc) size: +99%
memory usage: +15% (28 bytes / function)
import time: +70%

(*) import time can be much more slower for complex annotations.

## Conclusion

* PEP 563 is close to "zero overhead" in memory consumption. And
import time overhead is ~5%. Users can write type annotations without
caring overhead.

* PEP 649 is much better than old semantics for memory usage and
import time. But import time is still longer than no annotation code.

* The import time overhead is coming from unmarshal, not from
eval(). If we implement a "lazy load" mechanizm for docstrings and
annotations, overhead will become cheaper.
* pyc file become bigger (but who cares?)

I will read PEP 649 implementation to find missing optimizations other
than GH-25419 and GH-23056.

--
Inada Naoki <songofacandy@gmail.com>
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/K5PFGS6DQUHUG63UWRXYNLLMXAVELP32/
Code of Conduct: http://python.org/psf/codeofconduct/

Thanks Brett Cannon for suggesting to get Samuel Colvin (Pydantic) and me, Sebastián Ramírez (FastAPI and Typer) involved in this.

TL;DR: it seems to me PEP 649 would be incredibly important/useful for Pydantic, FastAPI, Typer, and similar tools, and their communities.

## About FastAPI, Pydantic, Typer

Some of you probably don't know what these tools are, so, in short, FastAPI is a web API framework based on Pydantic. It has had massive growth and adoption, and very positive feedback.

FastAPI was included for the first time in the last Python developers survey and it's already the third most used web framework, and apparently the fastest growing one: https://www.jetbrains.com/lp/python-developers-survey-2020/.

It was also recently recommended by ThoughtWorks for the enterprises: https://www.thoughtworks.com/radar/languages-and-frameworks?blipid=202104087

And it's currently used in lots of organizations, some widely known, chances are your orgs already use it in some way.

Pydantic, in very short, is a library that looks a lot like dataclasses (and also supports them), but it uses the same type annotations not only for type hints, but also for data validation, serialization (e.g. to JSON) and documentation (JSON Schema).

The key feature of FastAPI (thanks to Pydantic) is using the same type annotations for _more_ than just type hinting: data validation, serialization, and documentation. All those features are provided by default when building an API with FastAPI and Pydantic.

Typer is a library for building CLIs, based on Click, but using the same ideas from type annotations, from FastAPI and Pydantic.

## Why PEP 649

You can read Samuel's message to this mailing list here: https://mail.python.org/archives/list/python-dev@python.org/thread/7VMJWFGHVXDSRQFHMXVJKDDOVT47B54T/

And a longer discussion of how PEP 563 affects Pydantic here: https://github.com/samuelcolvin/pydantic/issues/2678

He has done most of the work to support these additional features from type annotations. So he would have the deepest insight into the tradeoffs/issues.

>From my point of view, just being able to use local variables in Pydantic models would be enough to justify PEP 649. With PEP 563, if a developer decides to create a Pydantic model inside a function (say a factory function) they would probably get an error. And it would probably not be obvious that they have to declare the model in the top level of the module.

The main feature of FastAPI and Pydantic is that they are very easy/intuitive to use. People from many backgrounds are now quickly and efficiently building APIs with best practices.

I've read some isolated comments of people that were against type annotations in general, saying that these tools justify adopting them. And I've also seen comments from people coming from other languages and fields, and adopting Python just to be able to use these tools.

Many of these developers are not Python experts, and supporting them and their intuition as much as possible when using these tools would help towards the PSF goal to:

> [...] support and facilitate the growth of a diverse and international community of Python programmers.

## Community support

To avoid asking people to spam here, Samuel and I are collecting "likes" in:

* This tweet: https://twitter.com/tiangolo/status/1382800928982642692
* This issue: https://github.com/samuelcolvin/pydantic/issues/2678

I just sent that tweet, I expect/hope it will collect some likes in support by the time you see it.

## Questions

I'm not very familiar with the internals of Python, and I'm not sure how the new syntax for `Union`s using the vertical bar character ("pipe", "|") work.

But would PEP 649 still support things like this?:

def run(arg: int | str = 0): pass

And would it be inspectable at runtime?

## Additional comments

The original author of PEP 563 was ?ukasz Langa.

I was recently chatting with him about Typer and annotations. And he showed interest, support, and help.

I think he probably knows the benefits of the way these libraries use type annotations and I would like/hope to read his opinion on all this.

Or alternatively, any possible ideas for how to handle these things in tools like Pydantic.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/J7YKCCCV2GL7HUDKG7DX4INAT65SXYRF/
Code of Conduct: http://python.org/psf/codeofconduct/

On 4/15/21 2:02 PM, Sebastián Ramírez wrote:
> ## Questions
>
> I'm not very familiar with the internals of Python, and I'm not sure how the new syntax for `Union`s using the vertical bar character ("pipe", "|") work.
>
> But would PEP 649 still support things like this?:
>
> def run(arg: int | str = 0): pass
>
> And would it be inspectable at runtime?


As far as I can tell, absolutely PEP 649 would support this feature. 
Under the covers, all PEP 649 is really doing is changing the
destination that annotation expressions get compiled to.  So anything
that works in an annotation with "stock" semantics would work fine with
PEP 649 semantics too, barring the exceptions specifically listed in the
PEP (e.g. annotations defined in conditionals, walrus operator, etc).


Cheers,


//arry/

>
> I will read PEP 649 implementation to find missing optimizations other
> than GH-25419 and GH-23056.
>

I found each "__co_annotation__" has own name like "func0.__co_annotation__".
It increased pyc size a little.
I created a draft pull request for cherry-picking GH-25419 and
GH-23056 and using just "__co_annotation__" as a name.
https://github.com/larryhastings/co_annotations/pull/9/commits/48a99e0aafa2dd006d72194bc1d7d47443900502

```
# previous result
$ ./python ~/ann_test.py 3
code size: 204963 bytes
memory: 209257 bytes
unmarshal: avg: 587.336ms +/-2.073ms
exec: avg: 97.056ms +/-0.046ms

# Use single name
$ ./python ~/ann_test.py 3
code size: 182088 bytes
memory: 209257 bytes
unmarshal: avg: 539.841ms +/-0.227ms
exec: avg: 98.351ms +/-0.064ms
```

It reduced code size and unmarshal time by 10%.
I confirmed GH-25419 and GH-23056 works very well. All same constants
are shared.

Unmarshal time is still slow. It is caused by unmarshaling code object.
But I don't know where is the bottleneck: parsing marshal file, or
creating code object.

---

Then, I tried to measure method annotation overhead.
Code: https://gist.github.com/methane/abb509e5f781cc4a103cc450e1e7925d#file-ann_test_method-py
Result:

```
# No annotation
$ ./python ~/ann_test_method.py 0
code size: 113019 bytes
memory: 256008 bytes
unmarshal: avg: 336.665ms +/-6.185ms
exec: avg: 176.791ms +/-3.067ms

# PEP 563
$ ./python ~/ann_test_method.py 2
code size: 120532 bytes
memory: 269004 bytes
unmarshal: avg: 348.285ms +/-0.102ms
exec: avg: 176.933ms +/-4.343ms

# PEP 649 (all optimization included)
$ ./python ~/ann_test_method.py 3
code size: 196135 bytes
memory: 436565 bytes
unmarshal: avg: 579.680ms +/-0.147ms
exec: avg: 259.781ms +/-7.087ms
```

PEP 563 vs 649
* code size: +63%
* memory: +62%
* import time: +60%

PEP 649 annotation overhead (compared with no annotation):
* code size: +83 byte/method
* memory: +180 byte/method
* import time: +326 us/method

It is disappointing because having thousands methods is very common
for web applications.

Unlike simple function case, PEP 649 creates function object instead
of code object for __co_annotation__ of methods.
It cause this overhead. Can we avoid creating functions for each annotation?

--
Inada Naoki <songofacandy@gmail.com>
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4KUXC373ZOP7YHZI6N4NKOOOB3DCL7NW/
Code of Conduct: http://python.org/psf/codeofconduct/

On 4/15/21 9:24 PM, Inada Naoki wrote:
> Unlike simple function case, PEP 649 creates function object instead
> of code object for __co_annotation__ of methods.
> It cause this overhead. Can we avoid creating functions for each annotation?


As the implementation of PEP 649 currently stands, there are two reasons
why the compiler might pre-bind the __co_annotations__ code object to a
function, instead of simply storing the code object:

* If the annotations refer to a closure ("freevars" is nonzero), or
* If the annotations /possibly/ refer to a class variable (the
annotations code object contains either LOAD_NAME or LOAD_CLASSDEREF).

If the annotations refer to a closure, then the code object also needs
to be bound with the "closure" tuple.  If the annotations possibly refer
to a class variable, then the code object also needs to be bound with
the current "f_locals" dict.  (Both could be true.)

Unfortunately, when generating annotations on a method, references to
builtins (e.g. "int", "str") seem to generate LOAD_NAME instructions
instead of LOAD_GLOBAL.  Which means pre-binding the function happens
pretty often for methods.  I believe in your benchmark it will happen
every time.  There's a lot of code, and a lot of runtime data
structures, inside compile.c and symtable.c behind the compiler's
decision about whether something is NAME vs GLOBAL vs DEREF etc, and I
wasn't comfortable with seeing if I could fix it.

Anyway I assume it wasn't "fixable".  The compiler would presumably
already prefer to generate LOAD_GLOBAL vs LOAD_NAME, because LOAD_GLOBAL
would be cheaper every time for a global or builtin.  The fact that it
already doesn't do so implies that it can't.


At the moment I have only one idea for a possible optimization, as
follows.  Instead of binding the function object immediately, it /might/
be cheaper to write the needed values into a tuple, then only actually
bind the function object on demand (like normal).

I haven't tried this because I assumed the difference at runtime would
be negligible.  On one hand, you're creating a function object; on the
other you're creating a tuple.  Either way you're creating an object at
runtime, and I assumed that bound functions weren't /that/ much more
expensive than tuples.  Of course I could be very wrong about that.

The other thing is, it would be a lot of work to even try the
experiment.  Also, it's an optimization, and I was more concerned with
correctness... and getting it done and getting this discussion underway.


What follows are my specific thoughts about how to implement this
optimization.

In this scenario, the logic in the compiler that generates the code
object would change to something like this:

has_closure = co.co_freevars != 0
has_load_name = co.co_code does not contain LOAD_NAME or
LOAD_CLASSDEREF bytecodes
if not (has_closure or has_load_name):
    co_ann = co
elif has_closure and (not has_load_name):
    co_ann = (co, freevars)
elif (not has_closure) and has_load_name:
    co_ann = (co, f_locals)
else:
    co_ann = (co, freevars, f_locals)
setattr(o, "__co_annotations__", co_ann)

(The compiler would have to generate instructions creating the tuple and
setting its members, then storing the resulting object on the object
with the annotations.)

Sadly, we can't pre-create this "co_ann" tuple as a constant and store
it in the .pyc file, because the whole point of the tuple is to contain
one or more objects only created at runtime.


The code implementing __co_annotations__ in the three objects (function,
class, module) would examine the object it got.  If it was a code
object, it would bind it; if it was a tuple, it would unpack the tuple
and use the values based on their type:

// co_ann = internal storage for __co_annotations__
if isinstance(co_ann, FunctionType) or (co_ann == None):
    return co_ann
co, freevars, locals = None
if isinstance(co_ann, CodeType):
    co = co_ann
else:
    assert isinstance(co_ann, tuple)
    assert 1 <= len(co_ann) <= 3
    for o in co_ann:
        if isinstance(o, CodeObject):
            assert not co
            co = o
        elif isinstance(o, tuple):
            assert not freevars
            freevars = o
        elif isinstance(o, dict):
            assert not locals
            locals = o
        else:
            raise ValueError(f"illegal value in co_annotations
tuple: {o!r}")
co_ann = make_function(co, freevars=freevars, locals=locals)
return co_ann


If you experiment with this approach, I'd be glad to answer questions
about it, either here or on Github, etc.


Cheers,


//arry/

On Fri, 16 Apr 2021, 3:14 pm Larry Hastings, <larry@hastings.org> wrote:

>
> Anyway I assume it wasn't "fixable". The compiler would presumably
> already prefer to generate LOAD_GLOBAL vs LOAD_NAME, because LOAD_GLOBAL
> would be cheaper every time for a global or builtin. The fact that it
> already doesn't do so implies that it can't.
>

Metaclass __prepare__ methods can inject names into the class namespace
that the compiler doesn't know about, so yeah, it unfortunately has to be
conservative and use LOAD_NAME in class level code.

Cheers,
Nick.

>
>

El sáb, 17 abr 2021 a las 8:30, Nick Coghlan (<ncoghlan@gmail.com>)
escribió:

>
>
> On Fri, 16 Apr 2021, 3:14 pm Larry Hastings, <larry@hastings.org> wrote:
>
>>
>> Anyway I assume it wasn't "fixable". The compiler would presumably
>> already prefer to generate LOAD_GLOBAL vs LOAD_NAME, because LOAD_GLOBAL
>> would be cheaper every time for a global or builtin. The fact that it
>> already doesn't do so implies that it can't.
>>
>
> Metaclass __prepare__ methods can inject names into the class namespace
> that the compiler doesn't know about, so yeah, it unfortunately has to be
> conservative and use LOAD_NAME in class level code.
>
> But of course, most metaclasses don't. I wonder if there are cases where
the compiler can statically figure out that there are no metaclass
shenanigans going on, and emit LOAD_GLOBAL anyway. It seems safe at least
when the class has no base classes and no metaclass=.


> Cheers,
> Nick.
>
>>
>> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/IZJYDHWJNMMMICUE32M3O7DGMSMVIOQ3/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Sun, 18 Apr 2021, 1:59 am Jelle Zijlstra, <jelle.zijlstra@gmail.com>
wrote:

> El sáb, 17 abr 2021 a las 8:30, Nick Coghlan (<ncoghlan@gmail.com>)
> escribió:.
>
>>
>> Metaclass __prepare__ methods can inject names into the class namespace
>> that the compiler doesn't know about, so yeah, it unfortunately has to be
>> conservative and use LOAD_NAME in class level code.
>>
>> But of course, most metaclasses don't. I wonder if there are cases where
> the compiler can statically figure out that there are no metaclass
> shenanigans going on, and emit LOAD_GLOBAL anyway. It seems safe at least
> when the class has no base classes and no metaclass=.
>

Aye, that particular case is one the symtable pass could at least
theoretically identify.

As soon as there is a name to resolve in the class header, though, it's no
longer safe for the compiler to make assumptions :(

Cheers,
Nick.

>


>>