Mailing List Archive

[ python-Bugs-1424152 ] urllib/urllib2: HTTPS over (Squid) Proxy fails
Bugs item #1424152, was opened at 2006-02-04 18:50
Message generated for change (Comment added) made by doko
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1424152&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: kxroberto (kxroberto)
Assigned to: Nobody/Anonymous (nobody)
>Summary: urllib/urllib2: HTTPS over (Squid) Proxy fails

Initial Comment:
py2.4.2/win32

The proxy mechanism of python fails on https and does
work completely wrong (not using the CONNECT scheme).

(after urlopen some minute(s) freeze then EOF error)

Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for
more information.
>>> import urllib
>>> urllib.getproxies()
{'ftp': 'ftp://vserver:3128', 'http':
'http://vserver:3128', 'gopher': 'gopher:/
/vserver:3128', 'https': 'https://vserver:3128'}
>>> urllib.urlopen('https://www.elster.de')
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "C:\Python24\lib\urllib.py", line 77, in urlopen
return opener.open(url)
File "C:\Python24\lib\urllib.py", line 185, in open
return getattr(self, name)(url)
File "C:\Python24\lib\urllib.py", line 386, in open_https
h.endheaders()
File "C:\Python24\lib\httplib.py", line 795, in
endheaders
self._send_output()
File "C:\Python24\lib\httplib.py", line 676, in
_send_output
self.send(msg)
File "C:\Python24\lib\httplib.py", line 643, in send
self.connect()
File "C:\Python24\lib\httplib.py", line 1071, in connect
ssl = socket.ssl(sock, self.key_file, self.cert_file)
File "C:\Python24\lib\socket.py", line 74, in ssl
return _realssl(sock, keyfile, certfile)
IOError: [Errno socket error] (8, 'EOF occurred in
violation of protocol')
>>>



no CONNECT even appears in the squid proxy access log.

Robert


----------------------------------------------------------------------

>Comment By: Matthias Klose (doko)
Date: 2007-08-14 14:29

Message:
Logged In: YES
user_id=60903
Originator: NO

seen with urllib2 as well:

https://bugs.launchpad.net/ubuntu/+source/python2.5/+bug/122551


----------------------------------------------------------------------

Comment By: kxroberto (kxroberto)
Date: 2006-02-11 19:28

Message:
Logged In: YES
user_id=972995

Meanwhile I wrote my own CONNECT quick hack. As indeed
this hack works correct for all proxied environments tested
up to now (>30) I wonder how open_https (in urllib and
urllib2) ever in the past managed to come through a proxy,
because there is some differentiation in open_https for the
case, that there is a proxy!? Who has written that
if..else's? Are there proxies which really do
SSL-handshaking directly and make an extra connection to the
target server? I guess that would even make certificate
handling very strange... I cannot immagine and never saw
one. But maybe such proxies exist. I am not a real expert
for such networking questions, but I guess CONNECT is widely
used and in my own proxies I can see in the log file, that
all common browsers use a HTTP CONNECT request for https
proxying. CONNECT should at least be implemented as an
option in urllibX

Robert


----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1424152&group_id=5470
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[ python-Bugs-1424152 ] urllib/urllib2: HTTPS over (Squid) Proxy fails [ In reply to ]
Bugs item #1424152, was opened at 2006-02-04 23:20
Message generated for change (Comment added) made by orsenthil
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1424152&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: kxroberto (kxroberto)
Assigned to: Nobody/Anonymous (nobody)
Summary: urllib/urllib2: HTTPS over (Squid) Proxy fails

Initial Comment:
py2.4.2/win32

The proxy mechanism of python fails on https and does
work completely wrong (not using the CONNECT scheme).

(after urlopen some minute(s) freeze then EOF error)

Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for
more information.
>>> import urllib
>>> urllib.getproxies()
{'ftp': 'ftp://vserver:3128', 'http':
'http://vserver:3128', 'gopher': 'gopher:/
/vserver:3128', 'https': 'https://vserver:3128'}
>>> urllib.urlopen('https://www.elster.de')
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "C:\Python24\lib\urllib.py", line 77, in urlopen
return opener.open(url)
File "C:\Python24\lib\urllib.py", line 185, in open
return getattr(self, name)(url)
File "C:\Python24\lib\urllib.py", line 386, in open_https
h.endheaders()
File "C:\Python24\lib\httplib.py", line 795, in
endheaders
self._send_output()
File "C:\Python24\lib\httplib.py", line 676, in
_send_output
self.send(msg)
File "C:\Python24\lib\httplib.py", line 643, in send
self.connect()
File "C:\Python24\lib\httplib.py", line 1071, in connect
ssl = socket.ssl(sock, self.key_file, self.cert_file)
File "C:\Python24\lib\socket.py", line 74, in ssl
return _realssl(sock, keyfile, certfile)
IOError: [Errno socket error] (8, 'EOF occurred in
violation of protocol')
>>>



no CONNECT even appears in the squid proxy access log.

Robert


----------------------------------------------------------------------

Comment By: O.R.Senthil Kumaran (orsenthil)
Date: 2007-08-15 01:40

Message:
Logged In: YES
user_id=942711
Originator: NO

Please verify if this recipe is of any help:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/456195

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2007-08-14 17:59

Message:
Logged In: YES
user_id=60903
Originator: NO

seen with urllib2 as well:

https://bugs.launchpad.net/ubuntu/+source/python2.5/+bug/122551


----------------------------------------------------------------------

Comment By: kxroberto (kxroberto)
Date: 2006-02-11 23:58

Message:
Logged In: YES
user_id=972995

Meanwhile I wrote my own CONNECT quick hack. As indeed
this hack works correct for all proxied environments tested
up to now (>30) I wonder how open_https (in urllib and
urllib2) ever in the past managed to come through a proxy,
because there is some differentiation in open_https for the
case, that there is a proxy!? Who has written that
if..else's? Are there proxies which really do
SSL-handshaking directly and make an extra connection to the
target server? I guess that would even make certificate
handling very strange... I cannot immagine and never saw
one. But maybe such proxies exist. I am not a real expert
for such networking questions, but I guess CONNECT is widely
used and in my own proxies I can see in the log file, that
all common browsers use a HTTP CONNECT request for https
proxying. CONNECT should at least be implemented as an
option in urllibX

Robert


----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1424152&group_id=5470
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com