Mailing List Archive

i'd say: look at the APIs and pick based on what's the most convenient for
you, and also, check the underlying libraries and supported databases.

The vmods don't do much but rather leverage libmaxminddb and libgeoip, so
they are the real point of focus here in terms or speed and resources.

--
Guillaume Quintard

On Sat, Nov 26, 2016 at 3:22 PM, Andrei <lagged@gmail.com> wrote:

> Hello all,
>
> I was wondering if there were any preferences among the community on which
> vmod (from https://varnish-cache.org/vmods) to use for geolocation, and
> why. My main concerns are of course speed, resources and accuracy. The ones
> I'm looking over from the vmods page are:
>
> ip2location - https://github.com/thlc/libvmod-ip2location
> geoip2 - https://github.com/fgsch/libvmod-geoip2
> maxminddb - https://github.com/simonvik/libvmod_maxminddb
>
> -- Andrei
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>

On Sat, Nov 26, 2016 at 3:22 PM, Andrei <lagged@gmail.com> wrote:
> Hello all,
>
> I was wondering if there were any preferences among the community on which
> vmod (from https://varnish-cache.org/vmods) to use for geolocation, and why.
> My main concerns are of course speed, resources and accuracy. The ones I'm
> looking over from the vmods page are:
>
> ip2location - https://github.com/thlc/libvmod-ip2location

Hello Andrei,

I will talk only about ip2location since I'm the developer behind it.
As it only implements some bindings to the C library provided by
ip2location, the overhead is pretty minimal.

The accuracy depends on the freshness of you IP2location database. Its
UNIX mtime is checked at each call and the database is reloaded on the
fly if the file has been updated. Our varnish servers don't handle a
lot of requests per second so I can't say wether it is able to cope
with more than 1000 req/s without increasing the CPU load.

I only implemented the bindings we needed, that's why I don't provide
full bindings to the whole library yet. I can add some if you need
them.

Thanks,

--
Thomas Lecomte | Sysadmin @ Virtual Expo

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Thank you both for the awesome input. Running stat on a file each time a
request comes in is something I would like to avoid. Would it be possible
to go by the varnishd uptime instead, and trigger database checks every N
seconds instead? We're looking to push around 5k req/s on average which is
why I'm trying to avoid the added syscalls.

On Mon, Nov 28, 2016 at 9:16 AM, Thomas Lecomte <
thomas.lecomte@virtual-expo.com> wrote:

> On Sat, Nov 26, 2016 at 3:22 PM, Andrei <lagged@gmail.com> wrote:
> > Hello all,
> >
> > I was wondering if there were any preferences among the community on
> which
> > vmod (from https://varnish-cache.org/vmods) to use for geolocation, and
> why.
> > My main concerns are of course speed, resources and accuracy. The ones
> I'm
> > looking over from the vmods page are:
> >
> > ip2location - https://github.com/thlc/libvmod-ip2location
>
> Hello Andrei,
>
> I will talk only about ip2location since I'm the developer behind it.
> As it only implements some bindings to the C library provided by
> ip2location, the overhead is pretty minimal.
>
> The accuracy depends on the freshness of you IP2location database. Its
> UNIX mtime is checked at each call and the database is reloaded on the
> fly if the file has been updated. Our varnish servers don't handle a
> lot of requests per second so I can't say wether it is able to cope
> with more than 1000 req/s without increasing the CPU load.
>
> I only implemented the bindings we needed, that's why I don't provide
> full bindings to the whole library yet. I can add some if you need
> them.
>
> Thanks,
>
> --
> Thomas Lecomte | Sysadmin @ Virtual Expo
>

On Mon, Nov 28, 2016 at 8:34 AM, Andrei <lagged@gmail.com> wrote:
> Thank you both for the awesome input. Running stat on a file each time a
> request comes in is something I would like to avoid. Would it be possible to
> go by the varnishd uptime instead, and trigger database checks every N
> seconds instead? We're looking to push around 5k req/s on average which is
> why I'm trying to avoid the added syscalls.

Indeed it would make more sense. If you are interested in ip2location
and plan to use this vmod, I can change the its behavior as you
suggested, it should be pretty straightforward.

--
Thomas Lecomte | Sysadmin @ Virtual Expo

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

This change would definitely help! From what I'm seeing both ip2loc and
maxmind are just as accurate when it comes to country tags, which is what
we're aiming for.

On Mon, Nov 28, 2016 at 9:41 AM, Thomas Lecomte <
thomas.lecomte@virtual-expo.com> wrote:

> On Mon, Nov 28, 2016 at 8:34 AM, Andrei <lagged@gmail.com> wrote:
> > Thank you both for the awesome input. Running stat on a file each time a
> > request comes in is something I would like to avoid. Would it be
> possible to
> > go by the varnishd uptime instead, and trigger database checks every N
> > seconds instead? We're looking to push around 5k req/s on average which
> is
> > why I'm trying to avoid the added syscalls.
>
> Indeed it would make more sense. If you are interested in ip2location
> and plan to use this vmod, I can change the its behavior as you
> suggested, it should be pretty straightforward.
>
> --
> Thomas Lecomte | Sysadmin @ Virtual Expo
>

Hi Andrel and Thomas,

I also wrote an implementation of ip2location as a vmod:
https://github.com/controversy187/libvmod-ip2location

I'm not well versed in C, and this is my first vmod, but maybe it could be
helpful. And I'm also open to suggestions for improvement :)

Brett

On Mon, Nov 28, 2016 at 3:38 AM Andrei <lagged@gmail.com> wrote:

> This change would definitely help! From what I'm seeing both ip2loc and
> maxmind are just as accurate when it comes to country tags, which is what
> we're aiming for.
>
> On Mon, Nov 28, 2016 at 9:41 AM, Thomas Lecomte <
> thomas.lecomte@virtual-expo.com> wrote:
>
> On Mon, Nov 28, 2016 at 8:34 AM, Andrei <lagged@gmail.com> wrote:
> > Thank you both for the awesome input. Running stat on a file each time a
> > request comes in is something I would like to avoid. Would it be
> possible to
> > go by the varnishd uptime instead, and trigger database checks every N
> > seconds instead? We're looking to push around 5k req/s on average which
> is
> > why I'm trying to avoid the added syscalls.
>
> Indeed it would make more sense. If you are interested in ip2location
> and plan to use this vmod, I can change the its behavior as you
> suggested, it should be pretty straightforward.
>
> --
> Thomas Lecomte | Sysadmin @ Virtual Expo
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Hi Brett,

Thanks for the heads up! Looks pretty lightweight and the db is only loaded
on init, which is also preferable in my case to avoid the numerous syscalls.


On Mon, Nov 28, 2016 at 6:41 AM, Brett Fitzgerald <
brettgfitzgerald@gmail.com> wrote:

> Hi Andrel and Thomas,
>
> I also wrote an implementation of ip2location as a vmod:
> https://github.com/controversy187/libvmod-ip2location
>
> I'm not well versed in C, and this is my first vmod, but maybe it could be
> helpful. And I'm also open to suggestions for improvement :)
>
> Brett
>
> On Mon, Nov 28, 2016 at 3:38 AM Andrei <lagged@gmail.com> wrote:
>
>> This change would definitely help! From what I'm seeing both ip2loc and
>> maxmind are just as accurate when it comes to country tags, which is what
>> we're aiming for.
>>
>> On Mon, Nov 28, 2016 at 9:41 AM, Thomas Lecomte <
>> thomas.lecomte@virtual-expo.com> wrote:
>>
>> On Mon, Nov 28, 2016 at 8:34 AM, Andrei <lagged@gmail.com> wrote:
>> > Thank you both for the awesome input. Running stat on a file each time a
>> > request comes in is something I would like to avoid. Would it be
>> possible to
>> > go by the varnishd uptime instead, and trigger database checks every N
>> > seconds instead? We're looking to push around 5k req/s on average which
>> is
>> > why I'm trying to avoid the added syscalls.
>>
>> Indeed it would make more sense. If you are interested in ip2location
>> and plan to use this vmod, I can change the its behavior as you
>> suggested, it should be pretty straightforward.
>>
>> --
>> Thomas Lecomte | Sysadmin @ Virtual Expo
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc@varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>