Mailing List Archive

Hi there,

On Fri, 2 Jul 2021, James Brown via clamav-users wrote:

> Trying to get the Google Safebrowsing python script to work. ...
>
> It worked yesterday [...] But since then I get: [...]
>
> 2021-04-08 14:33:46,539 DEBUG:googleapiclient.discovery:URL being requested: POST https://safebrowsing.googleapis.com/v4/threatListUpdates:fetch?key=AIwzaSyTheRestOfMyKeyc&alt=json <https://safebrowsing.googleapis.com/v4/threatListUpdates:fetch?key=AIwzaSyTheRestOfMyKeyc&alt=json>
> Can anyone help me with this?
>
> I posted on the Issues page in GitHub but got no reply.

It seems you're running on a Mac but you don't mention that in your
mail. In your mail, which AFAICT was sent on 2 July 2021, you say

"It worked yesterday ..."

yet the log gives the date at 8th April 2021 and your Github issue is
from the same date.

The posted log line which I haven't trimmed contains

...Updates:fetch?key=AIwzaSyTheRestOfMyKeyc&alt=json

which looks like either you've redacted some of the text without
telling us or you're sending an invalid request. Either way, it
appears that the request is being dropped by the server as the
connection attempts (and retries) are failing.

Perhaps you can clarify all the above.

I should say that I haven't used safebrowsing since it was dropped
from ClamAV's database servers in 2019; I've never tried to use the
Python scripts.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

> On 2 Jul 2021, at 6:06 pm, G.W. Haywood via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> Hi there,
>
> On Fri, 2 Jul 2021, James Brown via clamav-users wrote:
>
>> Trying to get the Google Safebrowsing python script to work. ...
>>
>> It worked yesterday [...] But since then I get: [...]
>>
>> 2021-04-08 14:33:46,539 DEBUG:googleapiclient.discovery:URL being requested: POST https://safebrowsing.googleapis.com/v4/threatListUpdates:fetch?key=AIwzaSyTheRestOfMyKeyc&alt=json <https://safebrowsing.googleapis.com/v4/threatListUpdates:fetch?key=AIwzaSyTheRestOfMyKeyc&alt=json>
>> Can anyone help me with this?
>>
>> I posted on the Issues page in GitHub but got no reply.
>
> It seems you're running on a Mac but you don't mention that in your
> mail. In your mail, which AFAICT was sent on 2 July 2021, you say
>
> "It worked yesterday ..."
>
> yet the log gives the date at 8th April 2021 and your Github issue is
> from the same date.
>
> The posted log line which I haven't trimmed contains
>
> ...Updates:fetch?key=AIwzaSyTheRestOfMyKeyc&alt=json
>
> which looks like either you've redacted some of the text without
> telling us or you're sending an invalid request. Either way, it
> appears that the request is being dropped by the server as the
> connection attempts (and retries) are failing.
>
> Perhaps you can clarify all the above.
>
> I should say that I haven't used safebrowsing since it was dropped
> from ClamAV's database servers in 2019; I've never tried to use the
> Python scripts.
> --
>
> 73,
> Ged.

Thanks for replying Ged.

Yes, running on macOS.

Yes I redacted some of the key in the URL.

Everything looks fine on console.cloud.google.com.

Just tried it again and it worked:

after the URL POST request the log continues with:


2021-07-02 19:03:42,442 DEBUG:UpdateClient:retrieving threats updates success
2021-07-02 19:03:42,445 INFO:UpdateClient:processing changes for MALWARE.URL.ALL_PLATFORMS from Cg0IARAGGAEiAzAwMTABEMiJCRoCGAwrTbwx to Cg0IARAGGAEiAzAwMBABEKeKCRoCGMwUrkuJ
2021-07-02 19:03:42,445 DEBUG:UpdateClient:processing 251 indice removals for MALWARE.URL.ALL_PLATFORMS
2021-07-02 19:03:43,836 DEBUG:UpdateClient:rm 000464f4 :: 000464f4

etc.

Not sure what was causing the problem.

Glad it’s working now (for the moment anyway).

Thanks again for your help.

James.



_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Hi there,

On Fri, 2 Jul 2021, James Brown via clamav-users wrote:

> ...
> Just tried it again and it worked:
> ...

Did it really fail for nearly three months or did you just not try often?
If the former perhaps better mention that in the Github issue, otherwise
it's probably best to close it.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

> On 2 Jul 2021, at 7:19 pm, G.W. Haywood via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> Hi there,
>
> On Fri, 2 Jul 2021, James Brown via clamav-users wrote:
>
>> ...
>> Just tried it again and it worked:
>> …

I spoke too soon. Just timed out again. Last log lines were:

2021-07-02 19:30:33,539 DEBUG:UpdateClient:add b'0d5f1ddd'
2021-07-02 19:30:33,550 DEBUG:UpdateClient:add b'0d646934'
2021-07-02 19:30:33,553 DEBUG:googleapiclient.discovery:URL being requested: POST https://safebrowsing.googleapis.com/v4/fullHashes:find?key=AIzaSyTheRestOfMyKeyc&alt=json

and Terminals said:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/httplib2/__init__.py", line 1376, in _conn_request
response = conn.getresponse()
File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1345, in getresponse
response.begin()
File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 307, in begin
version, status, reason = self._read_status()
File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 276, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/me/Downloads/clamav-safebrowsing-master/clamsbsync.py", line 599, in <module>
client.Sync()
File "/Users/me/Downloads/clamav-safebrowsing-master/clamsbsync.py", line 518, in Sync
self._handle_additions(session, listobj, list_update['additions'])
File "/Users/me/Downloads/clamav-safebrowsing-master/clamsbsync.py", line 419, in _handle_additions
hashes = self._retrieve_fullhashes([listobj], prefixset)
File "/Users/me/Downloads/clamav-safebrowsing-master/clamsbsync.py", line 104, in _retrieve_fullhashes
gapi_resp = self.gapi.get_full_hashes(listobjs, prefix_set)
File "/Users/me/Downloads/clamav-safebrowsing-master/clamsb/googleapi.py", line 69, in get_full_hashes
response = self._service.fullHashes().find(body=request_body).execute()
File "/usr/local/lib/python3.9/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/googleapiclient/http.py", line 920, in execute
resp, content = _retry_request(
File "/usr/local/lib/python3.9/site-packages/googleapiclient/http.py", line 222, in _retry_request
raise exception
File "/usr/local/lib/python3.9/site-packages/googleapiclient/http.py", line 191, in _retry_request
resp, content = http.request(uri, method, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/httplib2/__init__.py", line 1708, in request
(response, content) = self._request(
File "/usr/local/lib/python3.9/site-packages/httplib2/__init__.py", line 1424, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "/usr/local/lib/python3.9/site-packages/httplib2/__init__.py", line 1385, in _conn_request
conn.connect()
File "/usr/local/lib/python3.9/site-packages/httplib2/__init__.py", line 1136, in connect
sock.connect((self.host, self.port))
socket.timeout: timed out

So Google gets bored with me and hangs up?

Very hard to work out what is going on when you only see one side!

James.

I have to wonder why bother when Safari most other macOS browsers already use Google SafeBrowsing to screen for fraudulent websites, as long as you leave it enabled.

-Al-


Powered by Mailbutler <https://www.mailbutler.io/?utm_source=watermark&utm_medium=email&utm_campaign=watermark-essential-email>, the email extension that does it all

> On Jul 2, 2021, at 02:08, James Brown via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> Yes, running on macOS.

Hi there,

On Fri, 2 Jul 2021, James Brown via clamav-users wrote:
>> On Fri, 2 Jul 2021, James Brown via clamav-users wrote:
>>> ...
>>> Just tried it again and it worked:
>>> …
>
> I spoke too soon. Just timed out again. Last log lines were:
> ...
>
> So Google gets bored with me and hangs up?

Wouldn't surprise me (nor concern me) in the least, but that's just
between me and Google - largely because I see Google as the seccond
biggest problem on the Internet (at the moment, after Microsoft).

> Very hard to work out what is going on when you only see one side!

If I wanted to delve into it more I'd probably record the conversation
e.g. with tcpdump and then look at it with Wireshark (although I'm not
sure what that will tell you that you don't already know).

FWIW when I was using safebrowsing I don't remember seeing it catch
anything in the mail stream here, so I didn't mourn its loss.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

> On 2 Jul 2021, at 7:19 pm, G.W. Haywood via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> Hi there,
>
> On Fri, 2 Jul 2021, James Brown via clamav-users wrote:
>
>> ...
>> Just tried it again and it worked:
>> ...
>
> Did it really fail for nearly three months or did you just not try often?
> If the former perhaps better mention that in the Github issue, otherwise
> it's probably best to close it.

No, it didn’t work back in April and I just gave up on it.

Then this week I decided to retry.

James.


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml