Mailing List Archive

Need help to debug a ssl crash on Windows which prevents merging PRs
Hi,

In the 3.10 branch, it became really hard to merge PRs because the
following ssl crashs on Windows:
https://bugs.python.org/issue44252

It has a failure rate 1/2 (on average) on the "Windows x86" and
"Windows x64" jobs of GitHub Action and on the Win32 and Win64 jobs of
the Azure Pipelines. I failed to reproduce it on an up to date Windows
10 with an up to date Visual Studio 2019.

I cannot say if it's a race condition, if it's a bug in Python, if
it's a bug in the tests (it sounds unlikely, it worked well previously
and it's a hard crash, not a Python exception), if it's a bug in the C
compiler or in Windows itself...

Since there are other random test failures like test_asyncio, it now
requires multiple "re-run jobs" on GitHub Actions. Example of
test_asyncio test which fails frequently on Windows:
https://bugs.python.org/issue41682

If someone can reproduce https://bugs.python.org/issue44252 crash on
Windows, can you please provide me a SSH access to your machine so I
can debug it? Here is my public SSH key:
https://github.com/vstinner.keys

Is there a way to get a SSH access to a machine of the GitHub Action
CI job or of an Azure Pipelines CI job?

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/B7DIDGAZWV3WET227Z3LPXI5LUZX2DFP/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: Need help to debug a ssl crash on Windows which prevents merging PRs [ In reply to ]
On Fri, May 28, 2021 at 6:40 PM Victor Stinner <vstinner@python.org> wrote:
> In the 3.10 branch, it became really hard to merge PRs because the
> following ssl crashs on Windows:
> https://bugs.python.org/issue44252

Update on this bug which blocked the Python 3.10 beta 2 release. It's
now fully fixed!

It was a simple bug in the _ssl.SSLError exception. The problem was
that the crash only occurred on Windows and only if tests were run in
a very specific way. On CIs, the crash was deterministic. When I
debugged the issue manually, I failed to reproduce it. I tried many
different ways to run the tests, none worked.

I recall an old hack: run "import gc; gc.set_threshold(5)" at startup.
It makes crashes related to GC way more likely (the default threshold
of GC generation 0 is 700). I used this hack 3 years ago to debug
another GC bug really hard to reproduce:
https://mail.python.org/pipermail/python-dev/2018-June/153857.html
https://docs.python.org/dev/library/gc.html#gc.set_threshold

Not only the _ssl.SSLError bug is fixed, but Pablo also fixed the
documentation to explain clearly that a traverse function must be
implemented if Py_TPFLAGS_HAVE_GC is set:
https://github.com/python/cpython/commit/8b55bc3f93a655bc803bff79725d5fe3f124e2f0

Moreover, for people who don't read the documentation ;-), I also made
sure that it's no longer possible to create a type with
Py_TPFLAGS_HAVE_GC but with no traverse function:
https://github.com/python/cpython/commit/ee7637596d8de25f54261bbeabc602d31e74f482

By the way, I had to fix two stdlib types (_testcapi and _decimal
modules) which didn't respect that!

Victor
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7FHMUDJIZSO4EPREJ5XX6WTYLR3SQTAI/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: Need help to debug a ssl crash on Windows which prevents merging PRs [ In reply to ]
Well done Victor!
This stuff is way over my head, but rest assured that humble Python
programmers like me appreciate all the effort put in from guys like you
into improving Python.
Rob Cliffe

On 01/06/2021 23:14, Victor Stinner wrote:
> On Fri, May 28, 2021 at 6:40 PM Victor Stinner <vstinner@python.org> wrote:
>> In the 3.10 branch, it became really hard to merge PRs because the
>> following ssl crashs on Windows:
>> https://bugs.python.org/issue44252
> Update on this bug which blocked the Python 3.10 beta 2 release. It's
> now fully fixed!
>
> It was a simple bug in the _ssl.SSLError exception.
[snip]
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TDBYDARGKANV2ZIS27UTZW4NOKQKE6WE/
Code of Conduct: http://python.org/psf/codeofconduct/