Mailing List Archive

On 2021-02-24 at 15:29:58 +0100,
lucas <lucas@bourneuf.net> wrote:

> I'm currently trying to understand an error when using the dokuwikixmlrpc
> python module, allowing to easily work with DokuWiki RPC interface.
>
> Another description of the problem :
> https://github.com/kynan/dokuwikixmlrpc/issues/8
>
> Here is the code, tailored to work with the DokuWiki RPC interface:
>
> from urllib.parse import urlencode
> import xmlrpc.client as xmlrpclib
>
> URL = 'wiki.example.net'
> USER = 'user'
> PASSWD = 'password'
> USER_AGENT = 'DokuWikiXMLRPC 1.0 for testing'
>
> script = '/lib/exe/xmlrpc.php'
> url = URL + script + '?' + urlencode({'u': USER, 'p': PASSWD})
> xmlrpclib.Transport.user_agent = USER_AGENT
> xmlrpclib.SafeTransport.user_agent = USER_AGENT
> proxy = xmlrpclib.ServerProxy(url)
>
> v = proxy.dokuwiki.getVersion()
> print(v)
>
> When ran with Python 3.7 (a personnal debian server, or a personal windows
> computer), i obtain the expected 'Release 2018-04-22a "Greebo"' as ouput.
> When ran with Python 3.9 (my personnal, manjaro machine), i obtain the
> following stacktrace:
>
> Traceback (most recent call last):
> File "/home/project/read.py", line 32, in <module>
> v = proxy.dokuwiki.getVersion()
> File "/usr/lib/python3.9/xmlrpc/client.py", line 1116, in __call__
> return self.__send(self.__name, args)
> File "/usr/lib/python3.9/xmlrpc/client.py", line 1456, in __request
> response = self.__transport.request(
> File "/usr/lib/python3.9/xmlrpc/client.py", line 1160, in request
> return self.single_request(host, handler, request_body, verbose)
> File "/usr/lib/python3.9/xmlrpc/client.py", line 1190, in
> single_request
> raise ProtocolError(
> xmlrpc.client.ProtocolError: <ProtocolError for
> wiki.example.net/lib/exe/xmlrpc.php: 401 Unauthorized>

At the risk of stating the obvious, it might actually be an
authentication problem. In addition to double checking the password:

(1) make sure all of your certificates (under Arch Linux, on which
Manjaro is based), that's the ca-certificates package) are up to date;
and

(2) check with whoever owns the wiki about any other certificates they
require.

After that, all I know about XML RPC is to avoid it. :-)

> I don't have the possibility to test this on python 3.8 specifically, but
> since the XML and XMLRPC modules have been updated in 3.8, and since 3.9
> doesn't seems to introduce any change for them, i would expect 3.8 to
> introduce some change that dokuwikixmlrpc has somehow to take into
> consideration.
>
> Can anyone help me with that one ? I don't know anything about RPC and XML,
> i don't know what i need to do know to fix dokuwikixmlrpc.
--
https://mail.python.org/mailman/listinfo/python-list

Hi, thanks for your answer !

I updated everything, including certificates, while upgrading to python
3.9, and retried today (no new certificates to install). I am the
administrator of the wiki i try to access, and didn't do black magic in
the configuration..

The error really seems to came from 3.8 or 3.9, since i can still reach
the wiki nominally with my (some being non-updated) machines using
python 3.7. This didn't evolve since i discovered the problem few days ago.

--lucas


On 24/02/2021 16:20, 2QdxY4RzWzUUiLuE@potatochowder.com wrote:
> On 2021-02-24 at 15:29:58 +0100,
> lucas <lucas@bourneuf.net> wrote:
>
>> I'm currently trying to understand an error when using the dokuwikixmlrpc
>> python module, allowing to easily work with DokuWiki RPC interface.
>>
>> Another description of the problem :
>> https://github.com/kynan/dokuwikixmlrpc/issues/8
>>
>> Here is the code, tailored to work with the DokuWiki RPC interface:
>>
>> from urllib.parse import urlencode
>> import xmlrpc.client as xmlrpclib
>>
>> URL = 'wiki.example.net'
>> USER = 'user'
>> PASSWD = 'password'
>> USER_AGENT = 'DokuWikiXMLRPC 1.0 for testing'
>>
>> script = '/lib/exe/xmlrpc.php'
>> url = URL + script + '?' + urlencode({'u': USER, 'p': PASSWD})
>> xmlrpclib.Transport.user_agent = USER_AGENT
>> xmlrpclib.SafeTransport.user_agent = USER_AGENT
>> proxy = xmlrpclib.ServerProxy(url)
>>
>> v = proxy.dokuwiki.getVersion()
>> print(v)
>>
>> When ran with Python 3.7 (a personnal debian server, or a personal windows
>> computer), i obtain the expected 'Release 2018-04-22a "Greebo"' as ouput.
>> When ran with Python 3.9 (my personnal, manjaro machine), i obtain the
>> following stacktrace:
>>
>> Traceback (most recent call last):
>> File "/home/project/read.py", line 32, in <module>
>> v = proxy.dokuwiki.getVersion()
>> File "/usr/lib/python3.9/xmlrpc/client.py", line 1116, in __call__
>> return self.__send(self.__name, args)
>> File "/usr/lib/python3.9/xmlrpc/client.py", line 1456, in __request
>> response = self.__transport.request(
>> File "/usr/lib/python3.9/xmlrpc/client.py", line 1160, in request
>> return self.single_request(host, handler, request_body, verbose)
>> File "/usr/lib/python3.9/xmlrpc/client.py", line 1190, in
>> single_request
>> raise ProtocolError(
>> xmlrpc.client.ProtocolError: <ProtocolError for
>> wiki.example.net/lib/exe/xmlrpc.php: 401 Unauthorized>
>
> At the risk of stating the obvious, it might actually be an
> authentication problem. In addition to double checking the password:
>
> (1) make sure all of your certificates (under Arch Linux, on which
> Manjaro is based), that's the ca-certificates package) are up to date;
> and
>
> (2) check with whoever owns the wiki about any other certificates they
> require.
>
> After that, all I know about XML RPC is to avoid it. :-)
>
>> I don't have the possibility to test this on python 3.8 specifically, but
>> since the XML and XMLRPC modules have been updated in 3.8, and since 3.9
>> doesn't seems to introduce any change for them, i would expect 3.8 to
>> introduce some change that dokuwikixmlrpc has somehow to take into
>> consideration.
>>
>> Can anyone help me with that one ? I don't know anything about RPC and XML,
>> i don't know what i need to do know to fix dokuwikixmlrpc.
--
https://mail.python.org/mailman/listinfo/python-list

On Thu, Feb 25, 2021 at 2:02 AM lucas <lucas@bourneuf.net> wrote:
>
> Hi everyone,
>
> (Sorry for the double-send if any, i'm not sure the first send was
> performed, maybe because of bounce errors according to mailman.)
>
>
> I'm currently trying to understand an error when using the
> dokuwikixmlrpc python module, allowing to easily work with DokuWiki RPC
> interface.
>
> Another description of the problem :
> https://github.com/kynan/dokuwikixmlrpc/issues/8
>
> Here is the code, tailored to work with the DokuWiki RPC interface:
>
> from urllib.parse import urlencode
> import xmlrpc.client as xmlrpclib
>
> URL = 'wiki.example.net'
> USER = 'user'
> PASSWD = 'password'
> USER_AGENT = 'DokuWikiXMLRPC 1.0 for testing'
>
> script = '/lib/exe/xmlrpc.php'
> url = URL + script + '?' + urlencode({'u': USER, 'p': PASSWD})
> xmlrpclib.Transport.user_agent = USER_AGENT
> xmlrpclib.SafeTransport.user_agent = USER_AGENT
> proxy = xmlrpclib.ServerProxy(url)
>
> v = proxy.dokuwiki.getVersion()
> print(v)
>
> When ran with Python 3.7 (a personnal debian server, or a personal
> windows computer), i obtain the expected 'Release 2018-04-22a "Greebo"'
> as ouput.
> When ran with Python 3.9 (my personnal, manjaro machine), i obtain the
> following stacktrace:
>
> Traceback (most recent call last):
> File "/home/project/read.py", line 32, in <module>
> v = proxy.dokuwiki.getVersion()
> File "/usr/lib/python3.9/xmlrpc/client.py", line 1116, in __call__
> return self.__send(self.__name, args)
> File "/usr/lib/python3.9/xmlrpc/client.py", line 1456, in __request
> response = self.__transport.request(
> File "/usr/lib/python3.9/xmlrpc/client.py", line 1160, in request
> return self.single_request(host, handler, request_body, verbose)
> File "/usr/lib/python3.9/xmlrpc/client.py", line 1190, in
> single_request
> raise ProtocolError(
> xmlrpc.client.ProtocolError: <ProtocolError for
> wiki.example.net/lib/exe/xmlrpc.php: 401 Unauthorized>
>

A properly-formed URL will start with a protocol. I don't know
specifically what changed, but it's looking like something started
rejecting malformed URLs. Try adding "http://" or "https://" to your
URL (whichever is appropriate) and see if both versions will accept
it.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On 24/02/2021 18:00, Chris Angelico wrote:
> On Thu, Feb 25, 2021 at 2:02 AM lucas <lucas@bourneuf.net> wrote:
>>
>> Hi everyone,
>>
>> (Sorry for the double-send if any, i'm not sure the first send was
>> performed, maybe because of bounce errors according to mailman.)
>>
>>
>> I'm currently trying to understand an error when using the
>> dokuwikixmlrpc python module, allowing to easily work with DokuWiki RPC
>> interface.
>>
>> Another description of the problem :
>> https://github.com/kynan/dokuwikixmlrpc/issues/8
>>
>> Here is the code, tailored to work with the DokuWiki RPC interface:
>>
>> from urllib.parse import urlencode
>> import xmlrpc.client as xmlrpclib
>>
>> URL = 'wiki.example.net'
>> USER = 'user'
>> PASSWD = 'password'
>> USER_AGENT = 'DokuWikiXMLRPC 1.0 for testing'
>>
>> script = '/lib/exe/xmlrpc.php'
>> url = URL + script + '?' + urlencode({'u': USER, 'p': PASSWD})
>> xmlrpclib.Transport.user_agent = USER_AGENT
>> xmlrpclib.SafeTransport.user_agent = USER_AGENT
>> proxy = xmlrpclib.ServerProxy(url)
>>
>> v = proxy.dokuwiki.getVersion()
>> print(v)
>>
>> When ran with Python 3.7 (a personnal debian server, or a personal
>> windows computer), i obtain the expected 'Release 2018-04-22a "Greebo"'
>> as ouput.
>> When ran with Python 3.9 (my personnal, manjaro machine), i obtain the
>> following stacktrace:
>>
>> Traceback (most recent call last):
>> File "/home/project/read.py", line 32, in <module>
>> v = proxy.dokuwiki.getVersion()
>> File "/usr/lib/python3.9/xmlrpc/client.py", line 1116, in __call__
>> return self.__send(self.__name, args)
>> File "/usr/lib/python3.9/xmlrpc/client.py", line 1456, in __request
>> response = self.__transport.request(
>> File "/usr/lib/python3.9/xmlrpc/client.py", line 1160, in request
>> return self.single_request(host, handler, request_body, verbose)
>> File "/usr/lib/python3.9/xmlrpc/client.py", line 1190, in
>> single_request
>> raise ProtocolError(
>> xmlrpc.client.ProtocolError: <ProtocolError for
>> wiki.example.net/lib/exe/xmlrpc.php: 401 Unauthorized>
>>
>
> A properly-formed URL will start with a protocol. I don't know
> specifically what changed, but it's looking like something started
> rejecting malformed URLs. Try adding "http://" or "https://" to your
> URL (whichever is appropriate) and see if both versions will accept
> it.
>
> ChrisA
>

I did that, obtaining the following URL

https://wiki.[…]/lib/exe/xmlrpc.php?u=[…]&p=[…]

on my two currently available computer (laptop on 3.9, remote on 3.7),
and the results is the same.

For information, i'm uploading the same python program to my remote
server (debian, 3.7), and running it with the same parameters as my
laptop (manjaro, 3.9). My laptop is getting the 401, my server is
getting the expected dokuwiki version.

--lucas
--
https://mail.python.org/mailman/listinfo/python-list

On Thu, Feb 25, 2021 at 4:36 AM lucas <lucas@bourneuf.net> wrote:
> > A properly-formed URL will start with a protocol. I don't know
> > specifically what changed, but it's looking like something started
> > rejecting malformed URLs. Try adding "http://" or "https://" to your
> > URL (whichever is appropriate) and see if both versions will accept
> > it.
> >
> > ChrisA
> >
>
> I did that, obtaining the following URL
>
> https://wiki.[…]/lib/exe/xmlrpc.php?u=[…]&p=[…]
>
> on my two currently available computer (laptop on 3.9, remote on 3.7),
> and the results is the same.
>
> For information, i'm uploading the same python program to my remote
> server (debian, 3.7), and running it with the same parameters as my
> laptop (manjaro, 3.9). My laptop is getting the 401, my server is
> getting the expected dokuwiki version.

(I'm aware that you have some other actual domain, but I'll continue
using "wiki.example.net" as per your original post.)

Is it possible that there's some kind of server-based restriction?
What happens if you call socket.gethostbyname("wiki.example.net") on
both the laptop and the server - do you get back the same IP address?
(Or use gethostbyname_ex, or possibly getaddrinfo if ipv6 support
matters.)

Also, just to make sure there's nothing stupid happening, try printing
out the URL on both machines. Obviously censor the password before
sharing it here, but mainly, make sure that the two are generating the
exact same URL, just in case. Both are running 3.7+, so dict iteration
order shouldn't be getting in your way, but I've seen crazier things
before :)

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On 24/02/2021 18:48, Chris Angelico wrote:
> On Thu, Feb 25, 2021 at 4:36 AM lucas <lucas@bourneuf.net> wrote:
>>> A properly-formed URL will start with a protocol. I don't know
>>> specifically what changed, but it's looking like something started
>>> rejecting malformed URLs. Try adding "http://" or "https://" to your
>>> URL (whichever is appropriate) and see if both versions will accept
>>> it.
>>>
>>> ChrisA
>>>
>>
>> I did that, obtaining the following URL
>>
>> https://wiki.[…]/lib/exe/xmlrpc.php?u=[…]&p=[…]
>>
>> on my two currently available computer (laptop on 3.9, remote on 3.7),
>> and the results is the same.
>>
>> For information, i'm uploading the same python program to my remote
>> server (debian, 3.7), and running it with the same parameters as my
>> laptop (manjaro, 3.9). My laptop is getting the 401, my server is
>> getting the expected dokuwiki version.
>
> (I'm aware that you have some other actual domain, but I'll continue
> using "wiki.example.net" as per your original post.)
>
> Is it possible that there's some kind of server-based restriction?
> What happens if you call socket.gethostbyname("wiki.example.net") on
> both the laptop and the server - do you get back the same IP address?
> (Or use gethostbyname_ex, or possibly getaddrinfo if ipv6 support
> matters.)
>
> Also, just to make sure there's nothing stupid happening, try printing
> out the URL on both machines. Obviously censor the password before
> sharing it here, but mainly, make sure that the two are generating the
> exact same URL, just in case. Both are running 3.7+, so dict iteration
> order shouldn't be getting in your way, but I've seen crazier things
> before :)
>
> ChrisA
>

Thanks for taking time to help me !

I added socket.gethostbyname("wiki.example.net") (i removed the https://
since it, obviously now i think about it, led to a socket error)
in the program, so i could verify both the URL and IP are equivalent.

I got the exact same URL and IP. To be sure, i let python run the
comparison instead of only relying on my eyes.

>>> 'https://wiki.example.net/lib/exe/xmlrpc.php?u=user&p=password' ==
'https://wiki.example.net/lib/exe/xmlrpc.php?u=user&p=password'
True
>>> 'xx.xxx.xxx.197' == 'xx.xxx.xxx.197'
True

You gave me an idea: i checked the nginx log of the server hosting the
dokuwiki instance, and got something of interest :

[SERVER IP] - - [24/Feb/2021:19:08:31 +0100] "POST
/lib/exe/xmlrpc.php?u=[USER]&p=[PASSWORD] HTTP/1.1" 200 209 "-"
"Python-xmlrpc/3.7"
[LAPTOP IP] - - [24/Feb/2021:19:08:35 +0100] "POST /lib/exe/xmlrpc.php
HTTP/1.1" 401 433 "-" "Python-xmlrpc/3.9"

It seems that the laptop is not sending the arguments, despite them
being fed in the python code ?


Now i think about it, my remote server host both the python program into
3.7, and the dokuwiki. One may think it could be the source of the problem.
To prove it's not, i will test with another laptop (the one under
windows, python 3.7, which successfully accessed the wiki while i was
discovering the error on my manjaro laptop) as soon as possible.

--lucas
--
https://mail.python.org/mailman/listinfo/python-list

On Thu, Feb 25, 2021 at 5:12 AM lucas <lucas@bourneuf.net> wrote:
>
> On 24/02/2021 18:48, Chris Angelico wrote:
> I added socket.gethostbyname("wiki.example.net") (i removed the https://
> since it, obviously now i think about it, led to a socket error)
> in the program, so i could verify both the URL and IP are equivalent.
>
> I got the exact same URL and IP. To be sure, i let python run the
> comparison instead of only relying on my eyes.
>
> >>> 'https://wiki.example.net/lib/exe/xmlrpc.php?u=user&p=password' ==
> 'https://wiki.example.net/lib/exe/xmlrpc.php?u=user&p=password'
> True
> >>> 'xx.xxx.xxx.197' == 'xx.xxx.xxx.197'
> True

Those URLs were printed out from each script, just before attempting the call?

> You gave me an idea: i checked the nginx log of the server hosting the
> dokuwiki instance, and got something of interest :
>
> [SERVER IP] - - [24/Feb/2021:19:08:31 +0100] "POST
> /lib/exe/xmlrpc.php?u=[USER]&p=[PASSWORD] HTTP/1.1" 200 209 "-"
> "Python-xmlrpc/3.7"
> [LAPTOP IP] - - [24/Feb/2021:19:08:35 +0100] "POST /lib/exe/xmlrpc.php
> HTTP/1.1" 401 433 "-" "Python-xmlrpc/3.9"
>
> It seems that the laptop is not sending the arguments, despite them
> being fed in the python code ?

Fascinating! Well, that does at least simplify things somewhat -
instead of "why is this coming back 401", it's "why is this not
sending credentials".

> Now i think about it, my remote server host both the python program into
> 3.7, and the dokuwiki. One may think it could be the source of the problem.
> To prove it's not, i will test with another laptop (the one under
> windows, python 3.7, which successfully accessed the wiki while i was
> discovering the error on my manjaro laptop) as soon as possible.
>

More data is always good.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On 24/02/2021 19:22, Chris Angelico wrote:
> On Thu, Feb 25, 2021 at 5:12 AM lucas <lucas@bourneuf.net> wrote:
>>
>> On 24/02/2021 18:48, Chris Angelico wrote:
>> I added socket.gethostbyname("wiki.example.net") (i removed the https://
>> since it, obviously now i think about it, led to a socket error)
>> in the program, so i could verify both the URL and IP are equivalent.
>>
>> I got the exact same URL and IP. To be sure, i let python run the
>> comparison instead of only relying on my eyes.
>>
>> >>> 'https://wiki.example.net/lib/exe/xmlrpc.php?u=user&p=password' ==
>> 'https://wiki.example.net/lib/exe/xmlrpc.php?u=user&p=password'
>> True
>> >>> 'xx.xxx.xxx.197' == 'xx.xxx.xxx.197'
>> True
>
> Those URLs were printed out from each script, just before attempting the call?

Yes. The program is that one :

import socket
import xmlrpc.client as xmlrpclib
from xml.parsers.expat import ExpatError
from urllib.parse import urlencode

URL = 'https://wiki.example.net'
USER_AGENT = 'DokuWikiXMLRPC 1.0 for testing'

script = '/lib/exe/xmlrpc.php'
url = URL + script + '?' + urlencode({'u': USER, 'p': PASSWD})
print(url)
print(socket.gethostbyname('wiki.example.net'))
# xmlrpclib.Transport.user_agent = USER_AGENT
# xmlrpclib.SafeTransport.user_agent = USER_AGENT

proxy = xmlrpclib.ServerProxy(url)
v = proxy.dokuwiki.getVersion()
print(v)


>
>> You gave me an idea: i checked the nginx log of the server hosting the
>> dokuwiki instance, and got something of interest :
>>
>> [SERVER IP] - - [24/Feb/2021:19:08:31 +0100] "POST
>> /lib/exe/xmlrpc.php?u=[USER]&p=[PASSWORD] HTTP/1.1" 200 209 "-"
>> "Python-xmlrpc/3.7"
>> [LAPTOP IP] - - [24/Feb/2021:19:08:35 +0100] "POST /lib/exe/xmlrpc.php
>> HTTP/1.1" 401 433 "-" "Python-xmlrpc/3.9"
>>
>> It seems that the laptop is not sending the arguments, despite them
>> being fed in the python code ?
>
> Fascinating! Well, that does at least simplify things somewhat -
> instead of "why is this coming back 401", it's "why is this not
> sending credentials".

Yes, we are going forward :)


>>
>
> More data is always good.
>
> ChrisA

I tested from the windows computer (Python 3.8, it appears, not 3.7 as i
thought), and got the following nginx log:

[LAPTOP IP] - - [24/Feb/2021:20:06:42 +0100] "POST
/lib/exe/xmlrpc.php?u=[user]&p=[password] HTTP/1.1" 200 209 "-"
"DokuWikiXMLRPC 1.0 for testing windows"

That other laptop also had an ubuntu installed, with python 3.6.9, so i
ran the program and got the expected output, and the following nginx log:
[LAPTOP IP] - - [24/Feb/2021:20:04:04 +0100] "POST
/lib/exe/xmlrpc.php?u=[user]&p=[password] HTTP/1.1" 200 209 "-"
"DokuWikiXMLRPC 1.0 for testing ubuntu"

Both accessed correctly the dokuwiki version. Unless this is a platform
specific problem, it seems to narrow it to 3.9.

--lucas
--
https://mail.python.org/mailman/listinfo/python-list

On Thu, Feb 25, 2021 at 6:14 AM lucas <lucas@bourneuf.net> wrote:
> I tested from the windows computer (Python 3.8, it appears, not 3.7 as i
> thought), and got the following nginx log:
>
> [LAPTOP IP] - - [24/Feb/2021:20:06:42 +0100] "POST
> /lib/exe/xmlrpc.php?u=[user]&p=[password] HTTP/1.1" 200 209 "-"
> "DokuWikiXMLRPC 1.0 for testing windows"
>
> That other laptop also had an ubuntu installed, with python 3.6.9, so i
> ran the program and got the expected output, and the following nginx log:
> [LAPTOP IP] - - [24/Feb/2021:20:04:04 +0100] "POST
> /lib/exe/xmlrpc.php?u=[user]&p=[password] HTTP/1.1" 200 209 "-"
> "DokuWikiXMLRPC 1.0 for testing ubuntu"
>
> Both accessed correctly the dokuwiki version. Unless this is a platform
> specific problem, it seems to narrow it to 3.9.
>

Curious.

I think, at this point, we need a repeatable test case. Wonder how
hard it would be to make a single Python script that has a tiny server
and a tiny client, and tries to connect them.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On 24/02/2021 20:21, Chris Angelico wrote:
> On Thu, Feb 25, 2021 at 6:14 AM lucas <lucas@bourneuf.net> wrote:
>> I tested from the windows computer (Python 3.8, it appears, not 3.7 as i
>> thought), and got the following nginx log:
>>
>> [LAPTOP IP] - - [24/Feb/2021:20:06:42 +0100] "POST
>> /lib/exe/xmlrpc.php?u=[user]&p=[password] HTTP/1.1" 200 209 "-"
>> "DokuWikiXMLRPC 1.0 for testing windows"
>>
>> That other laptop also had an ubuntu installed, with python 3.6.9, so i
>> ran the program and got the expected output, and the following nginx log:
>> [LAPTOP IP] - - [24/Feb/2021:20:04:04 +0100] "POST
>> /lib/exe/xmlrpc.php?u=[user]&p=[password] HTTP/1.1" 200 209 "-"
>> "DokuWikiXMLRPC 1.0 for testing ubuntu"
>>
>> Both accessed correctly the dokuwiki version. Unless this is a platform
>> specific problem, it seems to narrow it to 3.9.
>>
>
> Curious.
>
> I think, at this point, we need a repeatable test case. Wonder how
> hard it would be to make a single Python script that has a tiny server
> and a tiny client, and tries to connect them.
>
> ChrisA
>
I will work on that tomorrow.

Thanks for your guidance, i will come back with a repeatable test case.

Best regards,
--lucas
--
https://mail.python.org/mailman/listinfo/python-list

Following our previous discussion:
https://www.talkend.net/post/287193.html

I finally took time (thanks to Florian R.) to get a reproducible example
of my problem, as asked previously by ChrisA.

The following code is implementing a webserver with Flask, and a client
with the XMLRPC client:

import sys
from xmlrpc import server, client
from urllib.parse import urlencode
from flask import Flask, request


PORT = 23456
USER, PASSWD = 'user', 'password'
URL = '127.0.0.1:' + str(PORT)

if len(sys.argv) > 1 and sys.argv[1] == 'server':
app = Flask(__name__)
@app.route('/', methods=['POST'])
@app.route('/RPC2', methods=['POST'])
def login():
print('REQUEST:', request.args)
return 'ok'
app.run(debug=True, host='localhost', port=PORT)

else:
url = 'http://' + URL + '?' + urlencode({'u': USER, 'p': PASSWD})
print(url)
proxy = client.ServerProxy(url)
print(proxy, dir(proxy))
print(proxy.add(2, 3))


You can run the client with `python3 p.py`, and the server with `python3
p.py server`.

On debian, Python 3.7, i got:

lucas@debianserver:~$ python3.7 p.py server
* Serving Flask app "p" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a
production deployment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://localhost:23456/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 249-992-288
127.0.0.1 - - [27/Mar/2021 18:31:13] "POST /?u=user&p=password
HTTP/1.1" 404 -


On Arch, python 3.9, i got:

aluriak@arch? python3.9 p.py server
* Serving Flask app "p" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a
production deployment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://localhost:23456/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 821-276-100
127.0.0.1 - - [27/Mar/2021 18:35:45] "POST /RPC2 HTTP/1.1" 200 -


Both systems return the same output on command `pip3 freeze | grep
Flask`, which is `Flask==1.1.2`.


Note that the two outputs differs in two ways:

127.0.0.1 - - [27/Mar/2021 18:31:13] "POST /?u=user&p=password
HTTP/1.1" 404 -
127.0.0.1 - - [27/Mar/2021 18:35:45] "POST /RPC2 HTTP/1.1" 200 -

The first contains the arguments, and the second contains the path.
Sounds like there is something wrong with both modules.

This should be a reproducible example ; plus i'm not the only one to
encounter that problem:
https://github.com/kynan/dokuwikixmlrpc/issues/8#issuecomment-808755244

Best regards,
--lucas
--
https://mail.python.org/mailman/listinfo/python-list

And, in my outputs, a key part is missing: the received arguments as
parsed by Flask:

Python 3.7:
REQUEST: ImmutableMultiDict([('u', 'user'), ('p', 'password')])

Python 3.9:
REQUEST: ImmutableMultiDict([])


Have a good day everyone,
--lucas


On 27/03/2021 18:53, lucas wrote:
> Following our previous discussion:
>     https://www.talkend.net/post/287193.html
>
> I finally took time (thanks to Florian R.) to get a reproducible example
> of my problem, as asked previously by ChrisA.
>
> The following code is implementing a webserver with Flask, and a client
> with the XMLRPC client:
>
>     import sys
>     from xmlrpc import server, client
>     from urllib.parse import urlencode
>     from flask import Flask, request
>
>
>     PORT = 23456
>     USER, PASSWD = 'user', 'password'
>     URL = '127.0.0.1:' + str(PORT)
>
>     if len(sys.argv) > 1 and sys.argv[1] == 'server':
>         app = Flask(__name__)
>         @app.route('/', methods=['POST'])
>         @app.route('/RPC2', methods=['POST'])
>         def login():
>             print('REQUEST:', request.args)
>             return 'ok'
>         app.run(debug=True, host='localhost', port=PORT)
>
>     else:
>         url = 'http://' + URL + '?' + urlencode({'u': USER, 'p': PASSWD})
>         print(url)
>         proxy = client.ServerProxy(url)
>         print(proxy, dir(proxy))
>         print(proxy.add(2, 3))
>
>
> You can run the client with `python3 p.py`, and the server with `python3
> p.py server`.
>
> On debian, Python 3.7, i got:
>
>     lucas@debianserver:~$ python3.7 p.py server
>      * Serving Flask app "p" (lazy loading)
>      * Environment: production
>        WARNING: This is a development server. Do not use it in a
> production deployment.
>        Use a production WSGI server instead.
>      * Debug mode: on
>      * Running on http://localhost:23456/ (Press CTRL+C to quit)
>      * Restarting with stat
>      * Debugger is active!
>      * Debugger PIN: 249-992-288
>     127.0.0.1 - - [27/Mar/2021 18:31:13] "POST /?u=user&p=password
> HTTP/1.1" 404 -
>
>
> On Arch, python 3.9, i got:
>
>      aluriak@arch? python3.9 p.py server
>      * Serving Flask app "p" (lazy loading)
>      * Environment: production
>        WARNING: This is a development server. Do not use it in a
> production deployment.
>        Use a production WSGI server instead.
>      * Debug mode: on
>      * Running on http://localhost:23456/ (Press CTRL+C to quit)
>      * Restarting with stat
>      * Debugger is active!
>      * Debugger PIN: 821-276-100
>     127.0.0.1 - - [27/Mar/2021 18:35:45] "POST /RPC2 HTTP/1.1" 200 -
>
>
> Both systems return the same output on command `pip3 freeze | grep
> Flask`, which is `Flask==1.1.2`.
>
>
> Note that the two outputs differs in two ways:
>
>     127.0.0.1 - - [27/Mar/2021 18:31:13] "POST /?u=user&p=password
> HTTP/1.1" 404 -
>     127.0.0.1 - - [27/Mar/2021 18:35:45] "POST /RPC2 HTTP/1.1" 200 -
>
> The first contains the arguments, and the second contains the path.
> Sounds like there is something wrong with both modules.
>
> This should be a reproducible example ; plus i'm not the only one to
> encounter that problem:
>
> https://github.com/kynan/dokuwikixmlrpc/issues/8#issuecomment-808755244
>
> Best regards,
> --lucas
--
https://mail.python.org/mailman/listinfo/python-list

On Sun, Mar 28, 2021 at 5:00 AM lucas <lucas@bourneuf.net> wrote:
> I finally took time (thanks to Florian R.) to get a reproducible example
> of my problem, as asked previously by ChrisA.

Thanks! With this in hand, I can play around with it.

> On debian, Python 3.7, i got:
>
> 127.0.0.1 - - [27/Mar/2021 18:31:13] "POST /?u=user&p=password
> HTTP/1.1" 404 -
>
> On Arch, python 3.9, i got:
>
> aluriak@arch? python3.9 p.py server
> 127.0.0.1 - - [27/Mar/2021 18:35:45] "POST /RPC2 HTTP/1.1" 200 -

On Debian, with both versions having been built from source, the same
occurs. Furthermore, Python 3.8 behaves as 3.7 does. This definitely
changed in 3.9.

> Note that the two outputs differs in two ways:
>
> 127.0.0.1 - - [27/Mar/2021 18:31:13] "POST /?u=user&p=password
> HTTP/1.1" 404 -
> 127.0.0.1 - - [27/Mar/2021 18:35:45] "POST /RPC2 HTTP/1.1" 200 -
>
> The first contains the arguments, and the second contains the path.
> Sounds like there is something wrong with both modules.

One point of note is that the request as given actually doesn't have a
slash. I think that's technically wrong, but a lot of systems will
just implicitly add the slash. That, coupled with commit 9c4c45, is
why you're seeing "/RPC2" in there. That distinction vanishes if you
change your client thusly:

url = 'http://' + URL + '/?' + urlencode({'u': USER, 'p': PASSWD})

Actually, it looks like all the changes came in with that commit. The
old way used some internal functions from urllib.parse, and the new
way uses the public function urllib.parse.urlparse(), and there are
some subtle differences. For one thing, the old way would implicitly
readd the missing slash, thus hiding the above issue; the new way
leaves the path empty (thus triggering the "/RPC2" replacement). But
perhaps more significantly, the old way left query parameters in the
"path" portion, where the new way has a separate "query" portion that
is being lost. Here's the relevant BPO:

https://bugs.python.org/issue38038

It seems to have been intended as a pure refactor, so I'd call this a
regression. Fortunately, it's not difficult to fix; but I'm not sure
if there are any other subtle changes.

The regression's already been reported so I'm adding to this issue:

https://bugs.python.org/issue43433

Hopefully that solves the problem!

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Thank you ChrisA !

I hope it will solve it too. Do i need to do anything ?

Thank you for your time and help.

Best wishes,
--lucas



On 27/03/2021 22:49, Chris Angelico wrote:
> On Sun, Mar 28, 2021 at 5:00 AM lucas <lucas@bourneuf.net> wrote:
>> I finally took time (thanks to Florian R.) to get a reproducible example
>> of my problem, as asked previously by ChrisA.
>
> Thanks! With this in hand, I can play around with it.
>
>> On debian, Python 3.7, i got:
>>
>> 127.0.0.1 - - [27/Mar/2021 18:31:13] "POST /?u=user&p=password
>> HTTP/1.1" 404 -
>>
>> On Arch, python 3.9, i got:
>>
>> aluriak@arch? python3.9 p.py server
>> 127.0.0.1 - - [27/Mar/2021 18:35:45] "POST /RPC2 HTTP/1.1" 200 -
>
> On Debian, with both versions having been built from source, the same
> occurs. Furthermore, Python 3.8 behaves as 3.7 does. This definitely
> changed in 3.9.
>
>> Note that the two outputs differs in two ways:
>>
>> 127.0.0.1 - - [27/Mar/2021 18:31:13] "POST /?u=user&p=password
>> HTTP/1.1" 404 -
>> 127.0.0.1 - - [27/Mar/2021 18:35:45] "POST /RPC2 HTTP/1.1" 200 -
>>
>> The first contains the arguments, and the second contains the path.
>> Sounds like there is something wrong with both modules.
>
> One point of note is that the request as given actually doesn't have a
> slash. I think that's technically wrong, but a lot of systems will
> just implicitly add the slash. That, coupled with commit 9c4c45, is
> why you're seeing "/RPC2" in there. That distinction vanishes if you
> change your client thusly:
>
> url = 'http://' + URL + '/?' + urlencode({'u': USER, 'p': PASSWD})
>
> Actually, it looks like all the changes came in with that commit. The
> old way used some internal functions from urllib.parse, and the new
> way uses the public function urllib.parse.urlparse(), and there are
> some subtle differences. For one thing, the old way would implicitly
> readd the missing slash, thus hiding the above issue; the new way
> leaves the path empty (thus triggering the "/RPC2" replacement). But
> perhaps more significantly, the old way left query parameters in the
> "path" portion, where the new way has a separate "query" portion that
> is being lost. Here's the relevant BPO:
>
> https://bugs.python.org/issue38038
>
> It seems to have been intended as a pure refactor, so I'd call this a
> regression. Fortunately, it's not difficult to fix; but I'm not sure
> if there are any other subtle changes.
>
> The regression's already been reported so I'm adding to this issue:
>
> https://bugs.python.org/issue43433
>
> Hopefully that solves the problem!
>
> ChrisA
>
--
https://mail.python.org/mailman/listinfo/python-list

On 3/27/2021 5:49 PM, Chris Angelico wrote:

> https://bugs.python.org/issue38038
>
> It seems to have been intended as a pure refactor, so I'd call this a
> regression. Fortunately, it's not difficult to fix; but I'm not sure
> if there are any other subtle changes.
>
> The regression's already been reported so I'm adding to this issue:
>
> https://bugs.python.org/issue43433

You added a PR, which is great, but useless unless a currently active
coredev reviews and merges. I requested such from the author and merger
of the apparently offending commit.

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

On 3/27/2021 6:10 PM, lucas wrote:

> I hope it will solve it too. Do i need to do anything ?

Review the patch by trying it out on your system. If necessary because
you do not have a local cpython clone, backup installed 3.9
Lib/xmlrpc.py and hand-edit. Then report OS, python used, and result.
(If it works, you can keep the patched version ;-).

For many issues, reviews are the bottleneck for getting fixes released.

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

On Sun, Mar 28, 2021 at 9:12 AM lucas <lucas@bourneuf.net> wrote:
>
> Thank you ChrisA !
>
> I hope it will solve it too. Do i need to do anything ?
>
> Thank you for your time and help.
>

There are a couple of things you can do actually! First off, here's
the pull request, which will be where further comments happen:

https://github.com/python/cpython/pull/25045

Since you actually use xmlrpc, you'd be the best person to devise some
useful tests. Also, the question has been raised regarding the
fragment. I suspect that a fragment is inappropriate here, since it's
effectively an external URL, but that's for you to answer; if
fragments are to be stripped, that definitely needs to be mentioned.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

lucas wrote at 2021-3-27 18:53 +0100:
>Following our previous discussion:
> https://www.talkend.net/post/287193.html
>
>I finally took time (thanks to Florian R.) to get a reproducible example
>of my problem, as asked previously by ChrisA.

I compared `xmlrpc.client.ServerProxy.__init__`
for Python 3.6 and Python 3.9(a5):
Python 3.6 uses `splittype` and `splithost` to parse *uri*
while Python 3.9 uses `urlparse` to complete decompose *uri*
and then uses `scheme`, `netloc` and `path` from the result.
As a consequence Python 3.9 misses any information in the "query string".

In my view, Python 3.6 behaves as documented (in the `ServerProxy` "docstring")
while Python 3.9 does not. At your place, I would file a bug
report at "https://bugs.python.org/".
--
https://mail.python.org/mailman/listinfo/python-list