Breaking out the discussion about lazy imports.
It seems somewhat OT in the declarative imports thread.
On Mon, Apr 11, 2022 at 10:50 AM Barry Warsaw <barry@python.org> wrote:
> Thanks Carl,
>
> > On Apr 9, 2022, at 08:25, Carl Meyer <carl@oddbird.net> wrote:
> >
> > Our experience in practice, though, has been that universally lazy
> > imports is somewhat easier to adopt than Strict Modules, and has had a
> > much bigger overall impact on reducing startup time for big CLIs (and
> > a big web server too; as you note it's not as serious an issue for a
> > web server in production, but restart time still does make a
> > difference to dev speed / experience.)
>
> Excellent point about the impact of restarts and development time. That’s
> been an issue for us a bit, but not an overwhelming motivation to rewrite
> in other languages[1].
>
Lazy imports had been very significant to both CLI startup time, as
well as service reload time during local development - we have
more details in the docs [1], and a related blog post coming up soon.
>
> > Removing slow stuff happening
> > at import time helps, but it'll never match the speed of not doing the
> > import at all! We've seen startup time improvements up to 70% in
> > real-world CLIs just by making imports lazy. We've also opened an
> > issue to discuss the possibility of upstreaming this. [2]
> >
> > [1] https://github.com/facebookincubator/cinder/#strict-modules
> > [2] https://bugs.python.org/issue46963
>
> Post-GH-issues-migration link for the issue:
> https://github.com/python/cpython/issues/91119
>
> I’ve put some questions and comments there, but I’m also really curious
> about the technical details for your lazy imports. Have you gotten as far
> as thinking about a PR or PEP?
>
Yes and Yes (at least for the "thinking about" part).
Germán Méndez Bravo (Kronuz) will be drafting a PEP,
and is leading work internally to port this from 3.8 (first
to 3.10, then to the cpython main branch).
We are planning to get started on this during PyCon sprints,
it would be nice to connect in-person with anyone who's
interested in this and happens to be in the sprints too!
>
> -Barry
>
> [1] Not that there aren’t other reasons folks give for rewriting, such as
> multicore performance, ecosystem alignment (e.g. SREs being more
> comfortable in Go), etc.
>
Itamar.
[1]
https://github.com/facebookincubator/cinder/blob/cinder/3.8/CinderDoc/lazy_imports.rst#lazy-imports
It seems somewhat OT in the declarative imports thread.
On Mon, Apr 11, 2022 at 10:50 AM Barry Warsaw <barry@python.org> wrote:
> Thanks Carl,
>
> > On Apr 9, 2022, at 08:25, Carl Meyer <carl@oddbird.net> wrote:
> >
> > Our experience in practice, though, has been that universally lazy
> > imports is somewhat easier to adopt than Strict Modules, and has had a
> > much bigger overall impact on reducing startup time for big CLIs (and
> > a big web server too; as you note it's not as serious an issue for a
> > web server in production, but restart time still does make a
> > difference to dev speed / experience.)
>
> Excellent point about the impact of restarts and development time. That’s
> been an issue for us a bit, but not an overwhelming motivation to rewrite
> in other languages[1].
>
Lazy imports had been very significant to both CLI startup time, as
well as service reload time during local development - we have
more details in the docs [1], and a related blog post coming up soon.
>
> > Removing slow stuff happening
> > at import time helps, but it'll never match the speed of not doing the
> > import at all! We've seen startup time improvements up to 70% in
> > real-world CLIs just by making imports lazy. We've also opened an
> > issue to discuss the possibility of upstreaming this. [2]
> >
> > [1] https://github.com/facebookincubator/cinder/#strict-modules
> > [2] https://bugs.python.org/issue46963
>
> Post-GH-issues-migration link for the issue:
> https://github.com/python/cpython/issues/91119
>
> I’ve put some questions and comments there, but I’m also really curious
> about the technical details for your lazy imports. Have you gotten as far
> as thinking about a PR or PEP?
>
Yes and Yes (at least for the "thinking about" part).
Germán Méndez Bravo (Kronuz) will be drafting a PEP,
and is leading work internally to port this from 3.8 (first
to 3.10, then to the cpython main branch).
We are planning to get started on this during PyCon sprints,
it would be nice to connect in-person with anyone who's
interested in this and happens to be in the sprints too!
>
> -Barry
>
> [1] Not that there aren’t other reasons folks give for rewriting, such as
> multicore performance, ecosystem alignment (e.g. SREs being more
> comfortable in Go), etc.
>
Itamar.
[1]
https://github.com/facebookincubator/cinder/blob/cinder/3.8/CinderDoc/lazy_imports.rst#lazy-imports
