Mailing List Archive

Hello,

I'm maintainer of FreeBSD ClamAV port. Today I recieved problem report
from our user that download of source archive fails with 403
forbitten. I checked it and confirmed it really happens as following.

----------------------------------------------------------------------
yasu@rolling-vm-freebsd2[1058]% make fetch
=> clamav-0.104.1.tar.gz doesn't seem to exist in /net/freebsd/ports/distfiles/.
=> Attempting to fetch https://www.clamav.net/downloads/production/clamav-0.104.1.tar.gz
fetch: https://www.clamav.net/downloads/production/clamav-0.104.1.tar.gz: Forbidden
=> Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/clamav-0.104.1.tar.gz
fetch: http://distcache.FreeBSD.org/ports-distfiles/clamav-0.104.1.tar.gz: Not Found
=> Couldn't fetch it - please try to retrieve this
=> port manually into /net/freebsd/ports/distfiles/ and try again.
*** Error code 1

Stop.
make: stopped in /usr/ports/security/clamav
yasu@rolling-vm-freebsd2[1059]%
----------------------------------------------------------------------

In the base system of FreeBSD there is a program named 'fetch'. Fetch
is a utility to retieve a file by URL, similar one to famous GNU Wget
and curl. FreeBSD ports framework uses fetch to download files through
network. And 403 forbitten happens when I tried to download source
archive with fetch.

----------------------------------------------------------------------
yasu@rolling-vm-freebsd2[1066]% fetch https://www.clamav.net/downloads/production/clamav-0.103.4.tar.gz
fetch: https://www.clamav.net/downloads/production/clamav-0.103.4.tar.gz: Forbidden
yasu@rolling-vm-freebsd2[1067]%
----------------------------------------------------------------------

I also tried download with GNU Wget and curl and it also fails with
same reason.

----------------------------------------------------------------------
yasu@rolling-vm-freebsd2[1069]% wget https://www.clamav.net/downloads/production/clamav-0.103.4.tar.gz
--2021-11-15 08:53:12-- https://www.clamav.net/downloads/production/clamav-0.103.4.tar.gz
Resolving www.clamav.net (www.clamav.net)... 104.16.218.84, 104.16.219.84, 2606:4700::6810:da54, ...
Connecting to www.clamav.net (www.clamav.net)|104.16.218.84|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2021-11-15 08:53:12 ERROR 403: Forbidden.

yasu@rolling-vm-freebsd2[1070]% curl -i https://www.clamav.net/downloads/production/clamav-0.103.4.tar.gz
HTTP/2 403
date: Sun, 14 Nov 2021 23:53:17 GMT
content-type: text/plain; charset=UTF-8
content-length: 16
x-frame-options: SAMEORIGIN
referrer-policy: same-origin
cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
expires: Thu, 01 Jan 1970 00:00:01 GMT
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
strict-transport-security: max-age=15552000
x-content-type-options: nosniff
server: cloudflare
cf-ray: 6ae4288c495f206b-NRT

error code: 1020%
yasu@rolling-vm-freebsd2[1071]%
----------------------------------------------------------------------

But download succeeds if I use web browser such as Chrome, Edge and
Firefox. it seems server checks User-Agent header of HTTP request. So
I tried to confirm it.

Our fetch utility has '--user-agent' options that changes the value of
User-Agent header. And if I specify 'Mozilla/5.0' as it, then source
archive can be downloaded successfully.

----------------------------------------------------------------------
yasu@rolling-vm-freebsd2[1072]% fetch --user-agent=Mozilla/5.0 https://www.clamav.net/downloads/production/clamav-0.103.4.tar.gz
clamav-0.103.4.tar.gz 15 MB 2092 kBps 07s
yasu@rolling-vm-freebsd2[1073]%
----------------------------------------------------------------------

These results means server checks User-Agent header of HTTP request
and returns 403 forbitten if the value doesn't look like that of web
browser.

Then is it intened change?

Best Regards.

---
Yasuhiro Kimura

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml