Mailing List Archive

[3.10] bpo-37788: Fix reference leak when Thread is never joined (GH-26103) (GH-26138)
https://github.com/python/cpython/commit/71dca6ea73aaf215fafa094512e8c748248c16b0
commit: 71dca6ea73aaf215fafa094512e8c748248c16b0
branch: 3.10
author: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com>
committer: miss-islington <31488909+miss-islington@users.noreply.github.com>
date: 2021-05-15T02:24:44-07:00
summary:

[3.10] bpo-37788: Fix reference leak when Thread is never joined (GH-26103) (GH-26138)



When a Thread is not joined after it has stopped, its lock may remain in the _shutdown_locks set until interpreter shutdown. If many threads are created this way, the _shutdown_locks set could therefore grow endlessly. To avoid such a situation, purge expired locks each time a new one is added or removed.
(cherry picked from commit c10c2ec7a0e06975e8010c56c9c3270f8ea322ec)


Co-authored-by: Antoine Pitrou <antoine@python.org>

files:
A Misc/NEWS.d/next/Library/2021-05-13-19-07-28.bpo-37788.adeFcf.rst
M Lib/test/test_threading.py
M Lib/threading.py

diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py
index 546773e3329afa..08c0ccd9a79b2a 100644
--- a/Lib/test/test_threading.py
+++ b/Lib/test/test_threading.py
@@ -907,6 +907,13 @@ def __call__(self):
thread.join()
self.assertTrue(target.ran)

+ def test_leak_without_join(self):
+ # bpo-37788: Test that a thread which is not joined explicitly
+ # does not leak. Test written for reference leak checks.
+ def noop(): pass
+ with threading_helper.wait_threads_exit():
+ threading.Thread(target=noop).start()
+ # Thread.join() is not called


class ThreadJoinOnShutdown(BaseTestCase):
diff --git a/Lib/threading.py b/Lib/threading.py
index fb70abd17aff8b..6c3d49c2d52679 100644
--- a/Lib/threading.py
+++ b/Lib/threading.py
@@ -780,12 +780,27 @@ def _newname(name_template):
_active = {} # maps thread id to Thread object
_limbo = {}
_dangling = WeakSet()
+
# Set of Thread._tstate_lock locks of non-daemon threads used by _shutdown()
# to wait until all Python thread states get deleted:
# see Thread._set_tstate_lock().
_shutdown_locks_lock = _allocate_lock()
_shutdown_locks = set()

+def _maintain_shutdown_locks():
+ """
+ Drop any shutdown locks that don't correspond to running threads anymore.
+
+ Calling this from time to time avoids an ever-growing _shutdown_locks
+ set when Thread objects are not joined explicitly. See bpo-37788.
+
+ This must be called with _shutdown_locks_lock acquired.
+ """
+ # If a lock was released, the corresponding thread has exited
+ to_remove = [lock for lock in _shutdown_locks if not lock.locked()]
+ _shutdown_locks.difference_update(to_remove)
+
+
# Main class for threads

class Thread:
@@ -968,6 +983,7 @@ def _set_tstate_lock(self):

if not self.daemon:
with _shutdown_locks_lock:
+ _maintain_shutdown_locks()
_shutdown_locks.add(self._tstate_lock)

def _bootstrap_inner(self):
@@ -1023,7 +1039,8 @@ def _stop(self):
self._tstate_lock = None
if not self.daemon:
with _shutdown_locks_lock:
- _shutdown_locks.discard(lock)
+ # Remove our lock and other released locks from _shutdown_locks
+ _maintain_shutdown_locks()

def _delete(self):
"Remove current thread from the dict of currently running threads."
diff --git a/Misc/NEWS.d/next/Library/2021-05-13-19-07-28.bpo-37788.adeFcf.rst b/Misc/NEWS.d/next/Library/2021-05-13-19-07-28.bpo-37788.adeFcf.rst
new file mode 100644
index 00000000000000..0c33923e992452
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2021-05-13-19-07-28.bpo-37788.adeFcf.rst
@@ -0,0 +1 @@
+Fix a reference leak when a Thread object is never joined.

_______________________________________________
Python-checkins mailing list
Python-checkins@python.org
https://mail.python.org/mailman/listinfo/python-checkins