Mailing List Archive

[clamav-users] Linode Clam AV Updates
hi,

    Clam AV has put it's database behind Cloudflare...  as a result the
updates no longer work because Cloudflare is blocking Linode.com
machines...  the updates are getting a 429 error saying that we are
"rate limited"...  if this continues it will make Clam AV useless
because eventually the database will be so out of date that new viruses
will get through...

    There should be some way for Clam AV to tell Cloudflare  to unhide
the AV database and fix the 429 errors....

thanks,

~bill speidel~

--

William H. Speidel, President
ENER G Systems, Inc.
117 Green Street Suite 1
Warrenton, VA 20186
540-547-6005


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
Hi there,

On Fri, 19 Mar 2021, Bill Speidel wrote:

> Clam AV has put it's database behind Cloudflare...? as a result the
> updates no longer work because Cloudflare is blocking Linode.com machines...?

Nope. That's not right. ClamAV has been using Cloudflare for ages.
But some protection against chronic abuse has recently been implemented.
Also, access by some very old versions of the ClamAV products has been
blocked - this was announced beforehand but I guess you didn't see that.

> the updates are getting a 429 error saying that we are "rate limited"...?

That's probably because you've been abusing ClamAV's service. I feel
sure that it was unwittingly.

> if this continues it will make Clam AV useless because eventually
> the database will be so out of date that new viruses will get
> through...

Your logic is flawed. If you think this way about *any* anti-virus
product you are going to have compromised systems. New viruses will
get through anyway.

> There should be some way for Clam AV to tell Cloudflare to unhide
> the AV database and fix the 429 errors....

Have you just joined this mailing list? Please read all the posts to
it for the past couple of weeks. After that by all means please feel
free to get back to us with information about your usage of ClamAV if
you actually need help.

Incidentally I block all Linode servers as a matter of routine, and I
have done for several years. The amount of abuse which emanates from
them is truly staggering.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
On 3/19/21 4:57 PM, Bill Speidel wrote:
> hi,

Hi,

>     Clam AV has put it's database behind Cloudflare...  as a result the
> updates no longer work because Cloudflare is blocking Linode.com
> machines...  the updates are getting a 429 error saying that we are
> "rate limited"...  if this continues it will make Clam AV useless
> because eventually the database will be so out of date that new viruses
> will get through...

Um ...

My Linode seems to still be getting updates.

I'm not seeing any errors in the freshclam log file.

Note: I don't know if it makes any difference or not, but I am using my
own DNS server and not Linode's.

>     There should be some way for Clam AV to tell Cloudflare  to unhide
> the AV database and fix the 429 errors....
>
> thanks,

I'd like to learn more about the format of the current.cvd.clamav.net
TXT record. The numbers in the end of my freshclam log file match some
of the numbers in the TXT record. But I can't quite grock the pattern
to know for sure.

# host -t txt current.cvd.clamav.net; perl -e 'printf "%d\n", time;'
current.cvd.clamav.net descriptive text
"0.103.1:59:26113:1616196540:0:63:49191:333"
1616197107
Fri Mar 19 17:00:25 2021 -> --------------------------------------
Fri Mar 19 18:00:25 2021 -> Received signal: wake up
Fri Mar 19 18:00:25 2021 -> ClamAV update process started at Fri Mar 19
18:00:25 2021
Fri Mar 19 18:00:26 2021 -> main.cvd is up to date (version: 59, sigs:
4564902, f-level: 60, builder: sigmgr)
Fri Mar 19 18:00:26 2021 -> daily.cld is up to date (version: 26113,
sigs: 3964163, f-level: 63, builder: raynman)
Fri Mar 19 18:00:26 2021 -> bytecode.cld is up to date (version: 333,
sigs: 92, f-level: 63, builder: awillia2)
Fri Mar 19 18:00:26 2021 -> --------------------------------------





--
Grant. . . .
unix || die
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
Hi Grant,

On Fri, 19 Mar 2021, Grant Taylor via clamav-users wrote:

> I'd like to learn more about the format of the current.cvd.clamav.net TXT
> record. The numbers in the end of my freshclam log file match some of the
> numbers in the TXT record. But I can't quite grock the pattern to know for
> sure.

Did you see this?

https://blog.clamav.net/2021/03/clamav-cvds-cdiffs-and-magic-behind.html

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
Linode is our second biggest abuser.

Slow your updater down.

Sent from my ? iPhone

> On Mar 19, 2021, at 19:40, Grant Taylor via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> ?On 3/19/21 4:57 PM, Bill Speidel wrote:
>> hi,
>
> Hi,
>
>> Clam AV has put it's database behind Cloudflare... as a result the updates no longer work because Cloudflare is blocking Linode.com machines... the updates are getting a 429 error saying that we are "rate limited"... if this continues it will make Clam AV useless because eventually the database will be so out of date that new viruses will get through...
>
> Um ...
>
> My Linode seems to still be getting updates.
>
> I'm not seeing any errors in the freshclam log file.
>
> Note: I don't know if it makes any difference or not, but I am using my own DNS server and not Linode's.
>
>> There should be some way for Clam AV to tell Cloudflare to unhide the AV database and fix the 429 errors....
>> thanks,
>
> I'd like to learn more about the format of the current.cvd.clamav.net TXT record. The numbers in the end of my freshclam log file match some of the numbers in the TXT record. But I can't quite grock the pattern to know for sure.
>
> # host -t txt current.cvd.clamav.net; perl -e 'printf "%d\n", time;'
> current.cvd.clamav.net descriptive text "0.103.1:59:26113:1616196540:0:63:49191:333"
> 1616197107
> Fri Mar 19 17:00:25 2021 -> --------------------------------------
> Fri Mar 19 18:00:25 2021 -> Received signal: wake up
> Fri Mar 19 18:00:25 2021 -> ClamAV update process started at Fri Mar 19 18:00:25 2021
> Fri Mar 19 18:00:26 2021 -> main.cvd is up to date (version: 59, sigs: 4564902, f-level: 60, builder: sigmgr)
> Fri Mar 19 18:00:26 2021 -> daily.cld is up to date (version: 26113, sigs: 3964163, f-level: 63, builder: raynman)
> Fri Mar 19 18:00:26 2021 -> bytecode.cld is up to date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
> Fri Mar 19 18:00:26 2021 -> --------------------------------------
>
>
>
>
>
> --
> Grant. . . .
> unix || die
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
On 3/19/21 5:52 PM, Joel Esler (jesler) via clamav-users wrote:
> Linode is our second biggest abuser.

I'm sorry to hear that.

> Slow your updater down.

Are you saying that as a knee jerk reaction to Linode? Or do you have
evidence that my system is the problem? Or do you know that some
defaults in Linode are a bad configuration (from Linode)?

I'm fairly certain that my system is still configured with the defaults
freshclam configuration. It looks like it's checking once an hour.

If that's too fast, what is a new recommendation?



--
Grant. . . .
unix || die
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
Sweet, who’s first OVH or Hetzner, they are probably the biggest spammers I see on MailOps, but I’m also a subscriber.
I’m thinking Amazon is just considered too big, or too much of a PIMA to outright RBL.
Seriously though, I wouldn’t complain either way, because I know the repercussions from subscribing to these providers irl.

Sincerely,

Eric Tykwinski
TrueNet, Inc.
P: 610-429-8300

> On Mar 19, 2021, at 7:52 PM, Joel Esler (jesler) via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> Linode is our second biggest abuser.
>
> Slow your updater down.
>
> Sent from my ? iPhone
>
>> On Mar 19, 2021, at 19:40, Grant Taylor via clamav-users <clamav-users@lists.clamav.net> wrote:
>>
>> ?On 3/19/21 4:57 PM, Bill Speidel wrote:
>>> hi,
>>
>> Hi,
>>
>>> Clam AV has put it's database behind Cloudflare... as a result the updates no longer work because Cloudflare is blocking Linode.com machines... the updates are getting a 429 error saying that we are "rate limited"... if this continues it will make Clam AV useless because eventually the database will be so out of date that new viruses will get through...
>>
>> Um ...
>>
>> My Linode seems to still be getting updates.
>>
>> I'm not seeing any errors in the freshclam log file.
>>
>> Note: I don't know if it makes any difference or not, but I am using my own DNS server and not Linode's.
>>
>>> There should be some way for Clam AV to tell Cloudflare to unhide the AV database and fix the 429 errors....
>>> thanks,
>>
>> I'd like to learn more about the format of the current.cvd.clamav.net TXT record. The numbers in the end of my freshclam log file match some of the numbers in the TXT record. But I can't quite grock the pattern to know for sure.
>>
>> # host -t txt current.cvd.clamav.net; perl -e 'printf "%d\n", time;'
>> current.cvd.clamav.net descriptive text "0.103.1:59:26113:1616196540:0:63:49191:333"
>> 1616197107
>> Fri Mar 19 17:00:25 2021 -> --------------------------------------
>> Fri Mar 19 18:00:25 2021 -> Received signal: wake up
>> Fri Mar 19 18:00:25 2021 -> ClamAV update process started at Fri Mar 19 18:00:25 2021
>> Fri Mar 19 18:00:26 2021 -> main.cvd is up to date (version: 59, sigs: 4564902, f-level: 60, builder: sigmgr)
>> Fri Mar 19 18:00:26 2021 -> daily.cld is up to date (version: 26113, sigs: 3964163, f-level: 63, builder: raynman)
>> Fri Mar 19 18:00:26 2021 -> bytecode.cld is up to date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
>> Fri Mar 19 18:00:26 2021 -> --------------------------------------
>>
>>
>>
>>
>>
>> --
>> Grant. . . .
>> unix || die
>>
>>
>> _______________________________________________
>>
>> clamav-users mailing list
>> clamav-users@lists.clamav.net
>> https://lists.clamav.net/mailman/listinfo/clamav-users
>>
>>
>> Help us build a comprehensive ClamAV guide:
>> https://github.com/vrtadmin/clamav-faq
>>
>> http://www.clamav.net/contact.html#ml
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
On 3/19/21 5:44 PM, G.W. Haywood via clamav-users wrote:
> Hi Grant,

Hi Ged,

> Did you see this?
>
> https://blog.clamav.net/2021/03/clamav-cvds-cdiffs-and-magic-behind.html

Thank you for that link.

It got me started down a twisty and windy path.

I ended up with this.

% dig +short current.cvd.clamav.net txt | sed 's/"//g;s/:/ /g' | read
newVersion mainVersion dailyVersion recordTime versionWarning
remoteFLevel safebrowsingVersion bytecodeVersion && printf " Scanning
Engine: %s\n Current Time: %s\n DNS Record Time: %s\n
main.cvd: %s\n daily.cvd: %s\n bytecode.cvd:
%s\nsafebrowsing.cvd: %s\n Remote F-Level: %s\n Version Warning: %s\n"
$newVersion "$(date "+%a %b %d %H:%M:%S %Y")" "$(date "+%a %b %d
%H:%M:%S %Y" -d "@$recordTime")" $mainVersion $dailyVersion
$bytecodeVersion $safebrowsingVersion $remoteFLevel $versionWarning
Scanning Engine: 0.103.1
Current Time: Fri Mar 19 21:06:56 2021
DNS Record Time: Fri Mar 19 19:29:00 2021
main.cvd: 59
daily.cvd: 26113
bytecode.cvd: 333
safebrowsing.cvd: 49191
Remote F-Level: 63
Version Warning: 0

Note: The read works in Zsh, but may not work as is in Bash.

Also, Linux date, macOS date is cranky.



--
Grant. . . .
unix || die
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
hi,

    thanks for the response...  i'm new to the clam users list...  i
did see that the freshclam routine was pinging every 5 seconds after
getting a 429 error so i stopped freshclam...  then i waited several
hours and tried again...  same 420 response...

    then i tried using cvd update but again got the 429 errors...  i
only let it run for about seven or eight tries... and am waiting for an
entire day to see if that helps...

    the problem i see is that i don't know if it's my IP in particular,
all of linode's IP addresses or a subnet...

    i block a lot of sites myself (Digital Ocean anyone?) so i
understand the desire to stop abusers...  i would be happy to only try
updating once every day or two if it's just my IP address...

    on the other hand if all of Linode is blocked then there's not much
i can do...

~bill~

--
William H. Speidel, President
ENER G Systems, Inc.
117 Green Street Suite 1
Warrenton, VA 20186
540-547-6005


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
--
William H. Speidel, President
ENER G Systems, Inc.
117 Green Street Suite 1
Warrenton, VA 20186
540-547-6005


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
hi,

    thanks for the response...  i'm new to the clam users list... i did
see that the freshclam routine was pinging every 5 seconds after getting
a 429 error so i stopped freshclam...  then i waited several hours and
tried again...  same 420 response...

    then i tried using cvd update but again got the 429 errors... i
only let it run for about seven or eight tries... and am waiting for an
entire day to see if that helps...

    the problem i see is that i don't know if it's my IP in particular,
all of linode's IP addresses or a subnet...

    i block a lot of sites myself (Digital Ocean anyone?) so i
understand the desire to stop abusers...  i would be happy to only try
updating once every day or two if it's just my IP address...

    on the other hand if all of Linode is blocked then there's not much
i can do...

~bill~

--
William H. Speidel, President
ENER G Systems, Inc.
117 Green Street Suite 1
Warrenton, VA 20186
540-547-6005


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
On 3/19/21 9:11 PM, Bill Speidel wrote:
> hi,
>
>     thanks for the response...  i'm new to the clam users list...  i
> did see that the freshclam routine was pinging every 5 seconds after
> getting a 429 error so i stopped freshclam...  then i waited several
> hours and tried again...  same 420 response...

I noticed that freshclam had problems when it tried to get
safebrowsing.cvd and that it tried every five seconds for three or five
times. But it gave up relatively quickly and is falling back to it's
regularly scheduled once an hour cycle.

>     the problem i see is that i don't know if it's my IP in particular,
> all of linode's IP addresses or a subnet...

My experience has been that Cloudflare has usually been good about per
IP filtering vs per IP /block/ filtering.

>     on the other hand if all of Linode is blocked then there's not much
> i can do...

Per the freshclam.conf man page, it looks like the code's default is
once every two hours.

I would hope -> expect that to be satisfactory.

Though comments in the man page say to check the safebrowsing file every
30 minutes.



--
Grant. . . .
unix || die
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
Please check out cvdupdate or Freshclam for your updates. Once or twice a day to check is fine.

Sent from my ? iPhone

> On Mar 19, 2021, at 23:16, Bill Speidel <bill@energsys.com> wrote:
>
> ?hi,
>
> thanks for the response... i'm new to the clam users list... i did see that the freshclam routine was pinging every 5 seconds after getting a 429 error so i stopped freshclam... then i waited several hours and tried again... same 420 response...
>
> then i tried using cvd update but again got the 429 errors... i only let it run for about seven or eight tries... and am waiting for an entire day to see if that helps...
>
> the problem i see is that i don't know if it's my IP in particular, all of linode's IP addresses or a subnet...
>
> i block a lot of sites myself (Digital Ocean anyone?) so i understand the desire to stop abusers... i would be happy to only try updating once every day or two if it's just my IP address...
>
> on the other hand if all of Linode is blocked then there's not much i can do...
>
> ~bill~
>
> --
> William H. Speidel, President
> ENER G Systems, Inc.
> 117 Green Street Suite 1
> Warrenton, VA 20186
> 540-547-6005
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
On 20/03/2021 14:12, Bill Speidel wrote:
[SNIP]
>     on the other hand if all of Linode is blocked then there's not much
> i can do...
>
Well, complaining to them and indicating a willingness to move to a
different provider if they don't clean up their act /might/ help.

But probably not...

Cheers,
Gary B-)


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
They aren’t blocked. They fall into the same rate limit that the rest of the planet does

Sent from my ? iPhone

> On Mar 20, 2021, at 00:37, Gary R. Schmidt <grschmidt@acm.org> wrote:
>
> ?On 20/03/2021 14:12, Bill Speidel wrote:
> [SNIP]
>> on the other hand if all of Linode is blocked then there's not much i can do...
> Well, complaining to them and indicating a willingness to move to a different provider if they don't clean up their act /might/ help.
>
> But probably not...
>
> Cheers,
> Gary B-)
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
We haven’t published an updated safebrowsing file in about 3 or 4 years.

https://blog.clamav.net/2020/06/the-future-of-clamav-safebrowsing.html<https://blog.clamav.net/2020/06/the-future-of-clamav-safebrowsing.html?m=1>


Sent from my ? iPhone

On Mar 20, 2021, at 00:21, Grant Taylor via clamav-users <clamav-users@lists.clamav.net> wrote:

?On 3/19/21 9:11 PM, Bill Speidel wrote:
hi,
thanks for the response... i'm new to the clam users list... i did see that the freshclam routine was pinging every 5 seconds after getting a 429 error so i stopped freshclam... then i waited several hours and tried again... same 420 response...

I noticed that freshclam had problems when it tried to get safebrowsing.cvd and that it tried every five seconds for three or five times. But it gave up relatively quickly and is falling back to it's regularly scheduled once an hour cycle.

the problem i see is that i don't know if it's my IP in particular, all of linode's IP addresses or a subnet...

My experience has been that Cloudflare has usually been good about per IP filtering vs per IP /block/ filtering.

on the other hand if all of Linode is blocked then there's not much i can do...

Per the freshclam.conf man page, it looks like the code's default is once every two hours.

I would hope -> expect that to be satisfactory.

Though comments in the man page say to check the safebrowsing file every 30 minutes.



--
Grant. . . .
unix || die


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
Slight exaggeration, 2 years.

If you aren’t using a private safebrowing engine as shown in the blog post below. Disable safebrowsing in your freshclam.conf. A future version of Freshclam will help you with this. Don’t wait! Get rid of those 403’s today!

Sent from my ? iPhone

On Mar 20, 2021, at 09:22, Joel Esler (jesler) <jesler@cisco.com> wrote:

? We haven’t published an updated safebrowsing file in about 3 or 4 years.

https://blog.clamav.net/2020/06/the-future-of-clamav-safebrowsing.html<https://blog.clamav.net/2020/06/the-future-of-clamav-safebrowsing.html?m=1>


Sent from my ? iPhone

On Mar 20, 2021, at 00:21, Grant Taylor via clamav-users <clamav-users@lists.clamav.net> wrote:

?On 3/19/21 9:11 PM, Bill Speidel wrote:
hi,
thanks for the response... i'm new to the clam users list... i did see that the freshclam routine was pinging every 5 seconds after getting a 429 error so i stopped freshclam... then i waited several hours and tried again... same 420 response...

I noticed that freshclam had problems when it tried to get safebrowsing.cvd and that it tried every five seconds for three or five times. But it gave up relatively quickly and is falling back to it's regularly scheduled once an hour cycle.

the problem i see is that i don't know if it's my IP in particular, all of linode's IP addresses or a subnet...

My experience has been that Cloudflare has usually been good about per IP filtering vs per IP /block/ filtering.

on the other hand if all of Linode is blocked then there's not much i can do...

Per the freshclam.conf man page, it looks like the code's default is once every two hours.

I would hope -> expect that to be satisfactory.

Though comments in the man page say to check the safebrowsing file every 30 minutes.



--
Grant. . . .
unix || die


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
On 21/03/2021 00:21, Joel Esler (jesler) via clamav-users wrote:
>
> Sent from my ? iPhone
>
>> On Mar 20, 2021, at 00:37, Gary R. Schmidt <grschmidt@acm.org> wrote:
>>
>> ?On 20/03/2021 14:12, Bill Speidel wrote:
>> [SNIP]
>>> on the other hand if all of Linode is blocked then there's not much i can do...
>> Well, complaining to them and indicating a willingness to move to a different provider if they don't clean up their act /might/ help.
>>
>> But probably not...
>>
> They aren’t blocked. They fall into the same rate limit that the
rest of the planet does

Ah, I just realised that what I wrote may be misinterpreted, it was the
service provided by Linode I was referring to moving away from, not ClamAV.

Cheers,
Gary B-)

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
On 20/03/2021 04:31, Joel Esler (jesler) via clamav-users wrote:
> Please check out cvdupdate or Freshclam for your updates. Once or twice a day to check is fine.
>
FWIW, running cvdupdate only once or twice a day is a BAD idea.

If you are running a private mirror, then if Freshclam tries to get the
latest CDIFF (according to DNS) from the private mirror, and it's not
there, it immediately downloads the full CVD from the private mirror.

So, if CDIFF 26116 is advertised in DNS but has not been downloaded by
cvdupdate yet, then the private mirror gets hammered by all the
Freshclam clients getting the full CVD - and the next time all the
Freshclams check, they will get the full CVD *again*, and *again*, until
cvdupdate finally updates the private mirror with the latest CDIFF.

So, you need to run cvdupdate at least every hour or so, so that
hopefully each Freshclam instance doesn't download the full CVD more
than once per released CDIFF...

Hopefully there'll soon either be a documented way to run our own
'DNSDatabaseInfo' server in conjunction with cvdupdate, or a Freshclam
update will make it be less impatient before it downloads the full CVD
after a new CDIFF is published.


--
Paul


--


Paul Smith Computer Services
Tel: 01484 855800
Vat No: GB 685 6987 53

Sign up for news & updates at http://www.pscs.co.uk/go/subscribe

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
Hi there,

On Sat, 20 Mar 2021, Paul Smith via clamav-users wrote:

> On 20/03/2021 04:31, Joel Esler (jesler) via clamav-users wrote:
>> Please check out cvdupdate or Freshclam for your updates. Once or twice a
>> day to check is fine.
>>
> FWIW, running cvdupdate only once or twice a day is a BAD idea.
>
> If you are running a private mirror, then if Freshclam tries to get the
> latest CDIFF (according to DNS) from the private mirror ...

My understanding is that if you're using a private mirror you're supposed
to set the 'PrivateMirror' option, which does not use DNS to check for the
existence of updated files, but checks the files themselves directly.

Quoting 'man freshclam.conf':

PrivateMirror STR
This option allows you to easily point freshclam to private
mirrors. If PrivateMirror is set, freshclam does not attempt
to use DNS to determine whether its databases are out-of-date,
instead it will use the If-Modified-Since request or directly
check the headers of the remote database files. For each
database, freshclam first attempts to download the CLD file.
If that fails, it tries to download the CVD file. This option
overrides DatabaseMirror, DNSDatabaseInfo and ScriptedUpdates.
It can be used multiple times to provide fall-back mirrors.
Default: disabled

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
Ged is correct.

Sent from my ? iPhone

> On Mar 20, 2021, at 13:14, G.W. Haywood via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> ?Hi there,
>
>> On Sat, 20 Mar 2021, Paul Smith via clamav-users wrote:
>>
>>> On 20/03/2021 04:31, Joel Esler (jesler) via clamav-users wrote:
>>> Please check out cvdupdate or Freshclam for your updates. Once or twice a day to check is fine.
>> FWIW, running cvdupdate only once or twice a day is a BAD idea.
>>
>> If you are running a private mirror, then if Freshclam tries to get the latest CDIFF (according to DNS) from the private mirror ...
>
> My understanding is that if you're using a private mirror you're supposed
> to set the 'PrivateMirror' option, which does not use DNS to check for the
> existence of updated files, but checks the files themselves directly.
>
> Quoting 'man freshclam.conf':
>
> PrivateMirror STR
> This option allows you to easily point freshclam to private
> mirrors. If PrivateMirror is set, freshclam does not attempt
> to use DNS to determine whether its databases are out-of-date,
> instead it will use the If-Modified-Since request or directly
> check the headers of the remote database files. For each
> database, freshclam first attempts to download the CLD file.
> If that fails, it tries to download the CVD file. This option
> overrides DatabaseMirror, DNSDatabaseInfo and ScriptedUpdates.
> It can be used multiple times to provide fall-back mirrors.
> Default: disabled
>
> --
>
> 73,
> Ged.
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
Sent from my iPad

On Mar 20, 2021, at 09:51, Paul Smith via clamav-users <clamav-users@lists.clamav.net> wrote:
> On 20/03/2021 04:31, Joel Esler (jesler) via clamav-users wrote:
>> Please check out cvdupdate or Freshclam for your updates. Once or twice a day to check is fine.
>>
> FWIW, running cvdupdate only once or twice a day is a BAD idea.

And just to be clear, Joel's advise was for use on the Private Server, not the clients.

-Al-

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
On 20/03/2021 17:12, G.W. Haywood via clamav-users wrote:
> On 20/03/2021 04:31, Joel Esler (jesler) via clamav-users wrote:
>>> Please check out cvdupdate or Freshclam for your updates.  Once or
>>> twice a day to check is fine.
>>>
>> FWIW, running cvdupdate only once or twice a day is a BAD idea.
>>
>> If you are running a private mirror, then if Freshclam tries to get
>> the latest CDIFF (according to DNS) from the private mirror ...
>
> My understanding is that if you're using a private mirror you're supposed
> to set the 'PrivateMirror' option, which does not use DNS to check for the
> existence of updated files, but checks the files themselves directly.
On 20/03/2021 19:08, Joel Esler (jesler) via clamav-users wrote:
> Ged is correct.

I'm sorry, but this is definitively NOT what the website says!

https://www.clamav.net/documents/private-local-mirrors

Option (2) (which is still documented but won't work any more) says "For
this to work you have to change freshclam.conf on each client so that it
reads

PrivateMirror machine1.mylan
ScriptedUpdates no"

This is NOT what we are doing!

Option (3) (using cvdupdate) says: "Set up your Freshclam clients’
freshclam.conf config file to point to:

DatabaseMirror http://machine1.mylan"

So, the cvdupdate method is meant to use 'DatabaseMirror' NOT
'PrivateMirror'

The 'PrivateMirror' option means that Freshclam does not download CDIFF
files at all, but that is how the 'cvdupdate' method expects the clients
to work. Cvdupdate makes CDIFF files available to the mirror 'clients',
just like the normal ClamAV method does. It is designed to be bandwidth
efficient by allowing clients to get the CDIFFs, as opposed to the
'PrivateMirror' method which requires them to get the full CVD file

It works absolutely fine, and wonderfully, as long as the private mirror
is up to date, so cvdupdate needs to be run frequently. It will not
download anything unless the DNS TXT record has updated.


Also, in case of doubt: https://github.com/micahsnyder/cvdupdate says

"You can test it by running freshclam or freshclam.exe locally, where
you've configured freshclam.conf with:

DatabaseMirror http://localhost:8000"

(There is no mention of the 'PrivateMirror' configuration option in the
cvdupdate docs)


--
Paul



--


Paul Smith Computer Services
Tel: 01484 855800
Vat No: GB 685 6987 53

Sign up for news & updates at http://www.pscs.co.uk/go/subscribe
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
Hi there,

On Sun, 21 Mar 2021, Paul Smith via clamav-users wrote:
> On 20/03/2021 17:12, G.W. Haywood via clamav-users wrote:
>>
>> My understanding is that if you're using a private mirror you're supposed
>> to set the 'PrivateMirror' option, which does not use DNS to check for the
>> existence of updated files, but checks the files themselves directly.
> ...
>
> I'm sorry, but this is definitively NOT what the website says!
>
> https://www.clamav.net/documents/private-local-mirrors
>
> Option (2) (which is still documented but won't work any more) says ...

Maybe I've missed something. Can you explain why it won't work?

As I understand it, as far as the Cloudflare service is concerned,
option 2 effectively makes a bunch of clients into a single client.

The single client is your Webserver - which behaves as any ordinary
client, in that it uses freshclam in the 'conventional' way. It uses
DNS to find the latest versions of the databases, and downloads cdiff
files if and when it needs to update the databases. But the database
files are now in the Webserver's document store; they are distributed
to the Webserver's clients (which are the remainder of your computers)
by running freshclam on _those_ computers in the 'unconventional' way,
i.e. with the 'PrivateMirror' option set. Your Webserver won't have
implemented DOS protection such as the ClamAV team has been obliged to
do by the ongoing abuse, and won't care that on every update freshclam
fetches the full database files instead of a few difference files; and
your LAN will probably have at least Gigabit/s capacity, so grabbing a
few hundred megabytes of files per day is a few seconds of traffic per
day per machine and isn't likely to be an issue. If your network is
larger than can be supported by a single mirror you could daisy-chain
more secondary mirrors from it (or perhaps something more creative)
but I'd expect you'd to be able to deal with that if you're managing
such a large network.

Anyway, the Cloudflare servers just see a single, well-behaved client.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Linode Clam AV Updates [ In reply to ]
On 21/03/2021 18:29, G.W. Haywood via clamav-users wrote:
> Hi there,
>
> On Sun, 21 Mar 2021, Paul Smith via clamav-users wrote:
>> On 20/03/2021 17:12, G.W. Haywood via clamav-users wrote:
>>>
>>> My understanding is that if you're using a private mirror you're
>>> supposed
>>> to set the 'PrivateMirror' option, which does not use DNS to check
>>> for the
>>> existence of updated files, but checks the files themselves directly.
>> ...
>>
>> I'm sorry, but this is definitively NOT what the website says!
>>
>> https://www.clamav.net/documents/private-local-mirrors
>>
>> Option (2) (which is still documented but won't work any more) says ...
>
> Maybe I've missed something.  Can you explain why it won't work?
>
> As I understand it, as far as the Cloudflare service is concerned,
> option 2 effectively makes a bunch of clients into a single client.


I tested it and couldn't get this to work to download the CVD files. I
believed it was because it was trying to repeatedly download full CVD
files rather than the CDIFFs, but maybe it was something else at my end
or the Cloudflare throttling not liking my tests at the time I tried it.
Because cvdupdate worked well and is better, I didn't persue this
option. If it should work, then I apologise.

> and your LAN will probably have at least Gigabit/s capacity

Yes, this option is fine if all your clients are on a fast LAN, but not
when private mirror serves clients over WANs, VPNs, remote Internet
users, etc

This is exactly what the cvdupdate method is supposed to help with. That
does NOT use the 'PrivateMirror' option with the private mirror as you
originally said it did.


--
Paul


--


Paul Smith Computer Services
Tel: 01484 855800
Vat No: GB 685 6987 53

Sign up for news & updates at http://www.pscs.co.uk/go/subscribe

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

1 2  View All