Mailing List Archive: [issue41401] Using non-ascii that require UTF-8 breaks AIX testing

[issue41401] Using non-ascii that require UTF-8 breaks AIX testing

Jul 26, 2020, 7:07 AM

Post #1 of 11 (66 views)

New submission from Michael Felt <aixtools@felt.demon.nl>:

issue41069 introduces tests for paths/files containing non-ascii characters.

On AIX - since the merge of PR21035 and PR21156 - the bots have been broken, i.e., returning test failed.

commit 700cfa8c90a90016638bac13c4efd03786b2b2a0
Author: Serhiy Storchaka <storchaka@gmail.com>
Date: Thu Jun 25 17:56:31 2020 +0300

bpo-41069: Make TESTFN and the CWD for tests containing non-ascii characters. (GH-21035)

commit f925407a19eeb9bf5f7640143979638adce2c677
Author: Serhiy Storchaka <storchaka@gmail.com>
Date: Thu Jun 25 20:39:12 2020 +0300

[3.9] bpo-41069: Make TESTFN and the CWD for tests containing non-ascii characters. (GH-21035). (GH-21156)

(cherry picked from commit 700cfa8c90a90016638bac13c4efd03786b2b2a0)

++++++++

Sadly, I cannot determine - exactly - where it is going wrong as the verbose results ends (says SUCCESS, but there is an ENV change, so bot says FAILED) as:

----------------------------------------------------------------------

Ran 614 tests in 59.122s

OK (skipped=8)
Warning -- files was modified by test_io
Before: []
After: ['@test_23134518_tmpæ']
test_io failed (env changed) in 59.7 sec

== Tests result: SUCCESS ==

1 test altered the execution environment:
test_io

Total duration: 59.8 sec

----------
components: IO, Tests
messages: 374312
nosy: Michael.Felt
priority: normal
severity: normal
status: open
title: Using non-ascii that require UTF-8 breaks AIX testing
type: behavior
versions: Python 3.10, Python 3.9

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41401>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41401] Using non-ascii that require UTF-8 breaks AIX testing [ In reply to ]

Jul 26, 2020, 7:15 AM

Post #2 of 11 (66 views)

Change by Karthikeyan Singaravelan <tir.karthi@gmail.com>:

----------
nosy: +serhiy.storchaka

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41401>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41401] Using non-ascii that require UTF-8 breaks AIX testing [ In reply to ]

Jul 26, 2020, 7:32 AM

Post #3 of 11 (66 views)

Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment:

Thank you for your report (and for all other heroic work for maintaining AIX). Breaking only one test on AIX looks like a good news.

Do you able to run tests manually? Could you please run the following command?

./python -m test --list-cases -uall test_io | xargs -n 1 ./python -m test -vuall test_io -m

It may help to determine what exactly the test was broken.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41401>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41401] Using non-ascii that require UTF-8 breaks AIX testing [ In reply to ]

Jul 26, 2020, 1:56 PM

Post #4 of 11 (66 views)

Michael Felt <aixtools@felt.demon.nl> added the comment:

Neat! extra arguments!!

The warnings - extracted:

== CPython 3.10.0a0 (heads/master-dirty:b1a8730, Jul 26 2020, 14:00:34) [GCC 7.2.0]
== AIX-2-00F9C1964C00-powerpc-32bit big-endian
== cwd: /home/aixtools/cpython/cpython-master/build/test_python_27984450?
== CPU count: 32
== encodings: locale=ISO8859-1, FS=iso8859-1
0:00:00 Run tests sequentially
0:00:00 [1/1] test_io
test_fspath_support (test.test_io.CIOTest) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.026s

OK
Warning -- files was modified by test_io
Before: []
After: ['@test_27984450_tmpæ']
test_io failed (env changed)

+++++
== CPython 3.10.0a0 (heads/master-dirty:b1a8730, Jul 26 2020, 14:00:34) [GCC 7.2.0]
== AIX-2-00F9C1964C00-powerpc-32bit big-endian
== cwd: /home/aixtools/cpython/cpython-master/build/test_python_28246552?
== CPU count: 32
== encodings: locale=ISO8859-1, FS=iso8859-1
0:00:00 Run tests sequentially
0:00:00 [1/1] test_io
test_fspath_support (test.test_io.PyIOTest) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.026s

OK
Warning -- files was modified by test_io
Before: []
After: ['@test_28246552_tmpæ']
test_io failed (env changed)

== Tests result: SUCCESS ==

1 test altered the execution environment:
test_io

++++

== CPython 3.10.0a0 (heads/master-dirty:b1a8730, Jul 26 2020, 14:00:34) [GCC 7.2.0]
== AIX-2-00F9C1964C00-powerpc-32bit big-endian
== cwd: /home/aixtools/cpython/cpython-master/build/test_python_33620404?
== CPU count: 32
== encodings: locale=ISO8859-1, FS=iso8859-1
0:00:00 Run tests sequentially
0:00:00 [1/1] test_io
test_reader_writer_close_error_on_close (test.test_io.CBufferedRWPairTest) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.001s

OK
Warning -- Unraisable exception
Exception ignored in: <_io.BufferedRWPair object at 0x30689750>
Traceback (most recent call last):
File "/home/aixtools/cpython/cpython-master/Lib/test/support/__init__.py", line 613, in gc_collect
gc.collect()
ValueError: flush of closed file
Warning -- Unraisable exception
Exception ignored in: <_io.BufferedWriter>
Traceback (most recent call last):
File "/home/aixtools/cpython/cpython-master/Lib/test/support/__init__.py", line 613, in gc_collect
gc.collect()
ValueError: flush of closed file
test_io failed (env changed)

== Tests result: SUCCESS ==

1 test altered the execution environment:
test_io

Hope this helps!

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41401>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41401] Using non-ascii that require UTF-8 breaks AIX testing [ In reply to ]

Jul 27, 2020, 7:14 AM

Post #5 of 11 (62 views)

Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment:

There are two errors in test_io, both are not related to AIX.

The first one is in test_fspath_support. It is exposed on non-UTF-8 locales. AIX by accident use ISO8859-1 locale by default.

The second one is in test_reader_writer_close_error_on_close and related to garbage collector. Seems it is only exposed when run this test separately.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41401>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41401] Using non-ascii that require UTF-8 breaks AIX testing [ In reply to ]

Jul 27, 2020, 7:16 AM

Post #6 of 11 (62 views)

Change by Serhiy Storchaka <storchaka+cpython@gmail.com>:

----------
keywords: +patch
pull_requests: +20781
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/21640

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41401>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41401] Using non-ascii that require UTF-8 breaks AIX testing [ In reply to ]

Jul 27, 2020, 7:26 AM

Post #7 of 11 (62 views)

Michael Felt <aixtools@felt.demon.nl> added the comment:

Excellent!!

aixtools@gcc119:[/home/aixtools/cpython/cpython-master]git pr 21640
remote: Enumerating objects: 50, done.
remote: Counting objects: 100% (50/50), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 58 (delta 46), reused 48 (delta 46), pack-reused 8
Unpacking objects: 100% (58/58), done.
>From https://github.com/python/cpython
* [new ref] refs/pull/21640/head -> pr_21640

aixtools@gcc119:[/home/aixtools/cpython/cpython-master]./python -m test
test_io
0:00:00 Run tests sequentially
0:00:00 [1/1] test_io
test_io passed in 1 min

== Tests result: SUCCESS ==

1 test OK.

Total duration: 1 min
Tests result: SUCCESS

On 27/07/2020 16:16, Serhiy Storchaka wrote:
> Change by Serhiy Storchaka <storchaka+cpython@gmail.com>:
>
>
> ----------
> keywords: +patch
> pull_requests: +20781
> stage: -> patch review
> pull_request: https://github.com/python/cpython/pull/21640
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue41401>
> _______________________________________
>

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41401>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41401] Using non-ascii that require UTF-8 breaks AIX testing [ In reply to ]

Jul 27, 2020, 10:58 AM

Post #8 of 11 (62 views)

Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment:

New changeset 67987acd5dc9776f55f4e139e2b3d9e7a6434d9f by Serhiy Storchaka in branch 'master':
bpo-41401: Fix test_fspath_support in test_io. (GH-21640)
https://github.com/python/cpython/commit/67987acd5dc9776f55f4e139e2b3d9e7a6434d9f

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41401>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41401] Using non-ascii that require UTF-8 breaks AIX testing [ In reply to ]

Jul 27, 2020, 11:07 AM

Post #9 of 11 (62 views)

Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment:

Automatic backport does not work due to changes in the test.support module. Victor, do you mind to backport PR 21640 to 3.9 and 3.8?

----------
nosy: +vstinner

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41401>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41401] Using non-ascii that require UTF-8 breaks AIX testing [ In reply to ]

Jul 29, 2020, 10:36 PM

Post #10 of 11 (61 views)

Michael Felt <aixtools@felt.demon.nl> added the comment:

The 'master' branch bot is working again, the 3.9 branch is still broken, and the 3.8 branch seems, as yet, unaffected.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41401>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41401] Using non-ascii that require UTF-8 breaks AIX testing [ In reply to ]

Aug 3, 2020, 4:06 PM

Post #11 of 11 (59 views)

Change by STINNER Victor <vstinner@python.org>:

----------
nosy: -vstinner

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41401>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com