Mailing List Archive

Eric Snow <ericsnowcurrently@gmail.com> added the comment:


New changeset a65c86889e208dddb26a7ebe7840c24edbcca775 by Eric Snow in branch 'main':
bpo-45020: Add -X frozen_modules=[on|off] to explicitly control use of frozen modules. (gh-28320)
https://github.com/python/cpython/commit/a65c86889e208dddb26a7ebe7840c24edbcca775


----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Change by Terry J. Reedy <tjreedy@udel.edu>:


----------
nosy: +terry.reedy
nosy_count: 14.0 -> 15.0
pull_requests: +26756
pull_request: https://github.com/python/cpython/pull/28344

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Terry J. Reedy <tjreedy@udel.edu> added the comment:


New changeset 369bf949ccbb689cd4638b29b4c0c12db79b927c by Terry Jan Reedy in branch 'main':
bpo-45020: Don't test IDLE with frozen module. (GH-28344)
https://github.com/python/cpython/commit/369bf949ccbb689cd4638b29b4c0c12db79b927c


----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>:


----------
nosy: +miss-islington
nosy_count: 15.0 -> 16.0
pull_requests: +26758
pull_request: https://github.com/python/cpython/pull/28345

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>:


----------
pull_requests: +26759
pull_request: https://github.com/python/cpython/pull/28346

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

miss-islington <mariatta.wijaya+miss-islington@gmail.com> added the comment:


New changeset 8a9396cf1d9e1ce558841095e1ce0d3c23b7a8aa by Miss Islington (bot) in branch '3.10':
bpo-45020: Don't test IDLE with frozen module. (GH-28344)
https://github.com/python/cpython/commit/8a9396cf1d9e1ce558841095e1ce0d3c23b7a8aa


----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

miss-islington <mariatta.wijaya+miss-islington@gmail.com> added the comment:


New changeset f71b86e0ae194613d235086755c6a44266978be1 by Miss Islington (bot) in branch '3.9':
bpo-45020: Don't test IDLE with frozen module. (GH-28344)
https://github.com/python/cpython/commit/f71b86e0ae194613d235086755c6a44266978be1


----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Eric Snow <ericsnowcurrently@gmail.com> added the comment:


New changeset cbeb81971057d6c382f45ecce92df2b204d4106a by Eric Snow in branch 'main':
bpo-45020: Freeze some of the modules imported during startup. (gh-28335)
https://github.com/python/cpython/commit/cbeb81971057d6c382f45ecce92df2b204d4106a


----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Eric Snow <ericsnowcurrently@gmail.com> added the comment:

At this point the fundamental work is done. Here are some follow-up tasks to wrap up this issue:

* freeze the remaining stdlib modules imported during startup (os, site, codecs, encodings.*)
+ blocked by bpo-45186 and bpo-45188
* default to "on" (except if actually running out of the source tree)
+ blocked by bpo-45211 (if we want to minimize disk access)
* always default to "on" if it's a PGO build, even if running out of the source tree
* stop tracking the frozen module .h files in the repo
+ blocked by bpo-45188
* (maybe) freeze modules imported for "python -m ..." (e.g. runpy)
* (maybe) freeze a small subset of the encodings (e.g. UTF-8)
+ blocked by a valid __path__ (see below) and consequently bpo-45211

Other related follow-up tasks:

* (maybe) make frozen stdlib modules more like source modules, with __file__ and __path__ (see bpo-21736)
+ blocked by bpo-45211
+ __path__ allows us to have frozen stdlib packages with non-frozen submodules (e.g. encodings)
* (maybe) freeze other modules (e.g. commonly used modules, sysconfig)
* (maybe) use something other than linear search for looking up frozen modules (see bpo-45213)

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Guido van Rossum <guido@python.org> added the comment:

I would move "default to "on" (except if actually running out of the source tree)" to the "maybe" category. I left a few comments in other deps. I think we should start by turning this on by default in PGO builds.

Separately, I encourage you to collect reliable performance numbers. It would be nice to see a dip on speed.python.org for this benchmark:

https://speed.python.org/timeline/#/?exe=12&ben=python_startup&env=1&revs=50&equid=off&quarts=on&extr=on

(but that won't show up until we turn this on by default for PGO builds).

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Change by Eric Snow <ericsnowcurrently@gmail.com>:


----------
pull_requests: +26790
pull_request: https://github.com/python/cpython/pull/28375

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Eric Snow <ericsnowcurrently@gmail.com> added the comment:

On Wed, Sep 15, 2021 at 12:03 PM Guido van Rossum
<report@bugs.python.org> wrote:
> I would move "default to "on" (except if actually running out of the source tree)" to the "maybe" category. I left a few comments in other deps. I think we should start by turning this on by default in PGO builds.

Sounds good.

> Separately, I encourage you to collect reliable performance numbers. It would be nice to see a dip on speed.python.org for this benchmark:

I'll do that.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Eric Snow <ericsnowcurrently@gmail.com> added the comment:


New changeset a9757bf34d8b4cb3c24bbb70d50a06c815e2e8f3 by Eric Snow in branch 'main':
bpo-45020: Drop the frozen .h files from the repo. (gh-28375)
https://github.com/python/cpython/commit/a9757bf34d8b4cb3c24bbb70d50a06c815e2e8f3


----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Eric Snow <ericsnowcurrently@gmail.com> added the comment:

Looks like that last commit broke one of the buildbots:

https://buildbot.python.org/all/#/builders/483/builds/812

I'll fix that right away.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Guido van Rossum <guido@python.org> added the comment:

FWIW, Python/frozen_modules/__hello__.h is still in the repo somehow.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Eric Snow <ericsnowcurrently@gmail.com> added the comment:

Never mind. It's pretty late here so I'm going to revert it and sort it out in the morning.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Change by Eric Snow <ericsnowcurrently@gmail.com>:


----------
pull_requests: +26795
pull_request: https://github.com/python/cpython/pull/28380

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Eric Snow <ericsnowcurrently@gmail.com> added the comment:


New changeset 9fd87a5fe5c468cf94265365091267838b004b7f by Eric Snow in branch 'main':
bpo-45020: Revert "Drop the frozen .h files from the repo." (gh-28380)
https://github.com/python/cpython/commit/9fd87a5fe5c468cf94265365091267838b004b7f


----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

STINNER Victor <vstinner@python.org> added the comment:

Is bpo-45220 "Windows builds sometimes fail on Azure and GitHub Action: fatal error RC1116: RC terminating after preprocessor errors" related to this issue?

----------
nosy: +vstinner

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Eric Snow <ericsnowcurrently@gmail.com> added the comment:

I left a comment on bpo-45220.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Change by Eric Snow <ericsnowcurrently@gmail.com>:


----------
pull_requests: +26804
pull_request: https://github.com/python/cpython/pull/28392

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Eric Snow <ericsnowcurrently@gmail.com> added the comment:


New changeset fdc6b3d9316501d2f0068a1bf4334debc1949e62 by Eric Snow in branch 'main':
bpo-45020: Drop the frozen .h files from the repo. (gh-28392)
https://github.com/python/cpython/commit/fdc6b3d9316501d2f0068a1bf4334debc1949e62


----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Change by Eric Snow <ericsnowcurrently@gmail.com>:


----------
pull_requests: +26812
pull_request: https://github.com/python/cpython/pull/28398

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

STINNER Victor <vstinner@python.org> added the comment:

It's no longer possible to build Python from a different directory. Example:

git clean -fdx
mkdir build
cd build/
../configure --with-pydebug
make

It fails with:

../Programs/_freeze_module importlib._bootstrap ../Lib/importlib/_bootstrap.py ../Python/frozen_modules/importlib__bootstrap.h
make: ../Programs/_freeze_module: No such file or directory

I'm working on a fix.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

STINNER Victor <vstinner@python.org> added the comment:

There is also a minor issue, "make regen-frozen" changes the end of line of two PCbuild/ files: see bpo-45231.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45020>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com