Mailing List Archive

gh-92913: Clarify changes to PyInitConfig.module_search_paths[_set] fields (GH-92980)
https://github.com/python/cpython/commit/403d16fa28764718dcd0536ccb3ab8d05768465d
commit: 403d16fa28764718dcd0536ccb3ab8d05768465d
branch: main
author: Steve Dower <steve.dower@python.org>
committer: zooba <steve.dower@microsoft.com>
date: 2022-05-19T20:23:53+01:00
summary:

gh-92913: Clarify changes to PyInitConfig.module_search_paths[_set] fields (GH-92980)

files:
A Misc/NEWS.d/next/C API/2022-05-19-18-05-51.gh-issue-92913.Ass1Hv.rst
M Doc/c-api/init_config.rst
M Doc/whatsnew/3.11.rst
M Lib/test/_test_embed_set_config.py
M Modules/getpath.py

diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst
index cc223a7fa4b94..d6a12375cd56f 100644
--- a/Doc/c-api/init_config.rst
+++ b/Doc/c-api/init_config.rst
@@ -1289,7 +1289,11 @@ Example setting the program name::
}

More complete example modifying the default configuration, read the
-configuration, and then override some parameters::
+configuration, and then override some parameters. Note that since
+3.11, many parameters are not calculated until initialization, and
+so values cannot be read from the configuration structure. Any values
+set before initialize is called will be left unchanged by
+initialization::

PyStatus init_python(const char *program_name)
{
@@ -1314,7 +1318,15 @@ configuration, and then override some parameters::
goto done;
}

- /* Append our custom search path to sys.path */
+ /* Specify sys.path explicitly */
+ /* To calculate the default and then modify, finish initialization and
+ then use PySys_GetObject("path") to get the list. */
+ condig.module_search_paths_set = 1
+ status = PyWideStringList_Append(&config.module_search_paths,
+ L"/path/to/stdlib");
+ if (PyStatus_Exception(status)) {
+ goto done;
+ }
status = PyWideStringList_Append(&config.module_search_paths,
L"/path/to/more/modules");
if (PyStatus_Exception(status)) {
@@ -1417,8 +1429,8 @@ It is possible to completely ignore the function calculating the default
path configuration by setting explicitly all path configuration output
fields listed above. A string is considered as set even if it is non-empty.
``module_search_paths`` is considered as set if
-``module_search_paths_set`` is set to ``1``. In this case, path
-configuration input fields are ignored as well.
+``module_search_paths_set`` is set to ``1``. In this case,
+``module_search_paths`` will be used without modification.

Set :c:member:`~PyConfig.pathconfig_warnings` to ``0`` to suppress warnings when
calculating the path configuration (Unix only, Windows does not log any warning).
diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst
index 1fd5b35695ee3..1f88d2557aa3d 100644
--- a/Doc/whatsnew/3.11.rst
+++ b/Doc/whatsnew/3.11.rst
@@ -403,6 +403,11 @@ Other CPython Implementation Changes
instead of prepending them.
(Contributed by Bastian Neuburger in :issue:`44934`.)

+* The :c:member:`PyConfig.module_search_paths_set` field must now be set to 1 for
+ initialization to use :c:member:`PyConfig.module_search_paths` to initialize
+ :data:`sys.path`. Otherwise, initialization will recalculate the path and replace
+ any values added to ``module_search_paths``.
+

New Modules
===========
@@ -1861,6 +1866,16 @@ Porting to Python 3.11
* Distributors are encouraged to build Python with the optimized Blake2
library `libb2`_.

+* The :c:member:`PyConfig.module_search_paths_set` field must now be set to 1 for
+ initialization to use :c:member:`PyConfig.module_search_paths` to initialize
+ :data:`sys.path`. Otherwise, initialization will recalculate the path and replace
+ any values added to ``module_search_paths``.
+
+* :c:func:`PyConfig_Read` no longer calculates the initial search path, and will not
+ fill any values into :c:member:`PyConfig.module_search_paths`. To calculate default
+ paths and then modify them, finish initialization and use :c:func:`PySys_GetObject`
+ to retrieve :data:`sys.path` as a Python list object and modify it directly.
+

Deprecated
----------
diff --git a/Lib/test/_test_embed_set_config.py b/Lib/test/_test_embed_set_config.py
index e7b7106e17851..7ff641b37bf18 100644
--- a/Lib/test/_test_embed_set_config.py
+++ b/Lib/test/_test_embed_set_config.py
@@ -236,10 +236,11 @@ def test_path(self):
module_search_paths=['a', 'b', 'c'])
self.assertEqual(sys.path, ['a', 'b', 'c'])

- # Leave sys.path unchanged if module_search_paths_set=0
+ # sys.path is reset if module_search_paths_set=0
self.set_config(module_search_paths_set=0,
module_search_paths=['new_path'])
- self.assertEqual(sys.path, ['a', 'b', 'c'])
+ self.assertNotEqual(sys.path, ['a', 'b', 'c'])
+ self.assertNotEqual(sys.path, ['new_path'])

def test_argv(self):
self.set_config(parse_argv=0,
diff --git a/Misc/NEWS.d/next/C API/2022-05-19-18-05-51.gh-issue-92913.Ass1Hv.rst b/Misc/NEWS.d/next/C API/2022-05-19-18-05-51.gh-issue-92913.Ass1Hv.rst
new file mode 100644
index 0000000000000..c448c64029d82
--- /dev/null
+++ b/Misc/NEWS.d/next/C API/2022-05-19-18-05-51.gh-issue-92913.Ass1Hv.rst
@@ -0,0 +1,2 @@
+Ensures changes to :c:member:`PyConfig.module_search_paths` are ignored
+unless :c:member:`PyConfig.module_search_paths_set` is set
diff --git a/Modules/getpath.py b/Modules/getpath.py
index 9aff19c0af7ed..47f075caf5551 100644
--- a/Modules/getpath.py
+++ b/Modules/getpath.py
@@ -228,6 +228,7 @@ def search_up(prefix, *landmarks, test=isfile):
use_environment = config.get('use_environment', 1)

pythonpath = config.get('module_search_paths')
+pythonpath_was_set = config.get('module_search_paths_set')

real_executable_dir = None
stdlib_dir = None
@@ -626,8 +627,8 @@ def search_up(prefix, *landmarks, test=isfile):
config['module_search_paths'] = py_setpath.split(DELIM)
config['module_search_paths_set'] = 1

-elif not pythonpath:
- # If pythonpath was already set, we leave it alone.
+elif not pythonpath_was_set:
+ # If pythonpath was already explicitly set or calculated, we leave it alone.
# This won't matter in normal use, but if an embedded host is trying to
# recalculate paths while running then we do not want to change it.
pythonpath = []

_______________________________________________
Python-checkins mailing list
Python-checkins@python.org
https://mail.python.org/mailman/listinfo/python-checkins