Mailing List Archive

Google IPv6 measurements in Europe appear heading down...
For a couple of weeks, it seems that Google IPv6 measurements are heading down mainly for Europe. For example, here is a link to a presentation of the Google measurements for several European countries and USA. There is a clear drop in the last days/weeks for European countries but not for USA.

This includes a big drop for my country (BE) :-O and I have checked with all Belgian ISP and they have no explanation as for them 'business as usual'. Apnic also does not show such a big drop.

So, I am guessing either a 'bug' in Google measurements infrastructure in Europe or could it be that the IPv6 latency to Google has increased a lot so that Happy Eyeball prefers IPv4? Recent measurement of dual-stack latency to www.google.com from several Belgian ISP gave 10% slower over IPv6.

Any clue will be welcome

-éric
Re: Google IPv6 measurements in Europe appear heading down... [ In reply to ]
Dear Eric,

on the http://www.ipv6matrix.org project we are also seeing rises &
falls depending on areas.
We have attributed this to several potential causes:

1. temporary allocation of IPv6 addresses & roll-back when faced with a
tech problem
2. a less inter-connected IPv6 network thus affected by changes in
peering agreements & BGP routing. (this has been quite a significant
challenge)
3. changes in hosting for some large content providers - this sometimes
happens when content providers strike a deal with another hosting
company or merge

Kind regards,

Olivier



On 23/10/2014 17:38, Eric Vyncke (evyncke) wrote:
> For a couple of weeks, it seems that Google IPv6 measurements are
> heading down mainly for Europe. For example, here is a link to a
> presentation of the Google measurements for several European countries
> and USA. There is a clear drop in the last days/weeks for European
> countries but not for USA.
>
> This includes a big drop for my country (BE) :-O and I have checked
> with all Belgian ISP and they have no explanation as for them
> 'business as usual'. Apnic also does not show such a big drop.
>
> So, I am guessing either a 'bug' in Google measurements infrastructure
> in Europe or could it be that the IPv6 latency to Google has increased
> a lot so that Happy Eyeball prefers IPv4? Recent measurement of
> dual-stack latency to www.google.com from several Belgian ISP gave 10%
> slower over IPv6.
>
> Any clue will be welcome
>
> -éric
Re: Google IPv6 measurements in Europe appear heading down... [ In reply to ]
With the link, it is probably better… still need some caffein

https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,de,us,lu

From: Eric Vyncke <evyncke@cisco.com<mailto:evyncke@cisco.com>>
Date: jeudi 23 octobre 2014 09:38
To: "ipv6-ops@lists.cluenet.de<mailto:ipv6-ops@lists.cluenet.de>" <ipv6-ops@lists.cluenet.de<mailto:ipv6-ops@lists.cluenet.de>>
Subject: Google IPv6 measurements in Europe appear heading down...

For a couple of weeks, it seems that Google IPv6 measurements are heading down mainly for Europe. For example, here is a link to a presentation of the Google measurements for several European countries and USA. There is a clear drop in the last days/weeks for European countries but not for USA.

This includes a big drop for my country (BE) :-O and I have checked with all Belgian ISP and they have no explanation as for them 'business as usual'. Apnic also does not show such a big drop.

So, I am guessing either a 'bug' in Google measurements infrastructure in Europe or could it be that the IPv6 latency to Google has increased a lot so that Happy Eyeball prefers IPv4? Recent measurement of dual-stack latency to www.google.com from several Belgian ISP gave 10% slower over IPv6.

Any clue will be welcome

-éric
Re: Google IPv6 measurements in Europe appear heading down... [ In reply to ]
Not seeing this in the Akamai data. See for Germany and Belgium.

Longer-term data at a lower frequency for Belgium
(from slightly different filtering so some data may not line up exactly):

2013-12-11 5.4
2014-01-08 5.3
2014-02-12 11.2
2014-02-26 17.4
2014-03-12 19.6
2014-04-16 20.7
2014-05-14 23.6
2014-05-28 24.4
2014-06-04 25.1
2014-06-11 26.1
2014-07-09 27.3
2014-07-16 26.0
2014-08-13 28.1
2014-08-27 28.1
2014-08-28 28.1
2014-09-10 26.7
2014-09-24 27.4
2014-10-08 27.8
2014-10-15 28.1





On Thu, Oct 23, 2014 at 2:03 PM, Eric Vyncke (evyncke) <evyncke@cisco.com>
wrote:

> With the link, it is probably better… still need some caffein
>
>
> https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,de,us,lu
>
> From: Eric Vyncke <evyncke@cisco.com>
> Date: jeudi 23 octobre 2014 09:38
> To: "ipv6-ops@lists.cluenet.de" <ipv6-ops@lists.cluenet.de>
> Subject: Google IPv6 measurements in Europe appear heading down...
>
> For a couple of weeks, it seems that Google IPv6 measurements are
> heading down mainly for Europe. For example, here is a link to a
> presentation of the Google measurements for several European countries and
> USA. There is a clear drop in the last days/weeks for European countries
> but not for USA.
>
> This includes a big drop for my country (BE) :-O and I have checked with
> all Belgian ISP and they have no explanation as for them 'business as
> usual'. Apnic also does not show such a big drop.
>
> So, I am guessing either a 'bug' in Google measurements infrastructure
> in Europe or could it be that the IPv6 latency to Google has increased a
> lot so that Happy Eyeball prefers IPv4? Recent measurement of dual-stack
> latency to www.google.com from several Belgian ISP gave 10% slower over
> IPv6.
>
> Any clue will be welcome
>
> -éric
>
>
Re: Google IPv6 measurements in Europe appear heading down... [ In reply to ]
Hi Erik,

> Not seeing this in the Akamai data. See for Germany and Belgium.

Your graphs show the best results (even going over 30% occasionally for Belgium) so let's go with your data. :)

Cheers,
Sander

/me likes picking the data that best represents what I *want* to see ;)
SV: Google IPv6 measurements in Europe appear heading down... [ In reply to ]
Telenor Norway has had an pretty steep growth in IPv6 enabled subscribers since the summer. We are the larges ISP in Norway, so rollouts we do usually are somewhat reflected in the graphs. On the fixed access (DSL and fiber) we had approx. 60.000 lines 1. oct. Today (24.oct) we have more than 100.000 lines activated. Yet the graph for Norway shows an flattening in the same time period. https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=no.


-Erik


________________________________________
Fra: ipv6-ops-bounces+erik.taraldsen=telenor.com@lists.cluenet.de [ipv6-ops-bounces+erik.taraldsen=telenor.com@lists.cluenet.de] p&#229; vegne av Sander Steffann [sander@steffann.nl]
Sendt: 24. oktober 2014 02:25
Til: Erik Nygren
Kopi: Eric Vyncke; ipv6-ops@lists.cluenet.de
Emne: Re: Google IPv6 measurements in Europe appear heading down...

Hi Erik,

> Not seeing this in the Akamai data. See for Germany and Belgium.

Your graphs show the best results (even going over 30% occasionally for Belgium) so let's go with your data. :)

Cheers,
Sander

/me likes picking the data that best represents what I *want* to see ;)
Re: Google IPv6 measurements in Europe appear heading down... [ In reply to ]
* erik.taraldsen@telenor.com

> Telenor Norway has had an pretty steep growth in IPv6 enabled
> subscribers since the summer. We are the larges ISP in Norway, so
> rollouts we do usually are somewhat reflected in the graphs. On the
> fixed access (DSL and fiber) we had approx. 60.000 lines 1. oct.
> Today (24.oct) we have more than 100.000 lines activated. Yet the
> graph for Norway shows an flattening in the same time period.
> https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=no.

I've been somewhat puzzled about Google's Norway graph as well. If you
zoom in, there is a dramatic increase between the 25th of September and
the 7th of October, from 4.22% to 5.85%. In the same period, my own
data from VG has been rather stable, peaking at about 4.5% (see
http://goo.gl/XvoNLE).

I do see a more gradual increase during October due to
Telenor's roll-out (see http://goo.gl/7OWJqL), especially visible in the
last few weeks. However, as Erik points out, in the same period (after
the 7th of October), the Google graph has hardly moved. Very odd.

Tore
Re: Google IPv6 measurements in Europe appear heading down... [ In reply to ]
…and, does anyone know why today there is no IPv6 shown in the APNIC statistics of
http://labs.apnic.net/dists/v6dcc.htm <http://labs.apnic.net/dists/v6dcc.htm> ?

Marco

> On 23/ott/2014, at 18:38, Eric Vyncke (evyncke) <evyncke@cisco.com> wrote:
>
> For a couple of weeks, it seems that Google IPv6 measurements are heading down mainly for Europe. For example, here is a link to a presentation of the Google measurements for several European countries and USA. There is a clear drop in the last days/weeks for European countries but not for USA.
>
> This includes a big drop for my country (BE) :-O and I have checked with all Belgian ISP and they have no explanation as for them 'business as usual'. Apnic also does not show such a big drop.
>
> So, I am guessing either a 'bug' in Google measurements infrastructure in Europe or could it be that the IPv6 latency to Google has increased a lot so that Happy Eyeball prefers IPv4? Recent measurement of dual-stack latency to www.google.com from several Belgian ISP gave 10% slower over IPv6.
>
> Any clue will be welcome
>
> -éric

--
Marco Sommani
Consiglio Nazionale delle Ricerche
Istituto di Informatica e Telematica
Via Giuseppe Moruzzi 1
56124 Pisa - Italia
work: +390506212127
mobile: +393487981019
fax: +390503158327
mailto:marco.sommani@cnr.it
Re: Google IPv6 measurements in Europe appear heading down... [ In reply to ]
…and, does anyone know why today there is no IPv6 shown in the APNIC statistics of
> http://labs.apnic.net/dists/v6dcc.htm ?

this:
http://labs.apnic.net/dists/v6dcc.html ?



--
Re: Google IPv6 measurements in Europe appear heading down... [ In reply to ]
Hi Éric,

I'm puzzled, not about the recent decline (and now near-recovery) but at the
odd 'kick' in the stats on Aug 17, mainly for Europe.

Compare Belgium and most of the high-IPv6 EU countries (for clarity I left a
couple out like FR and CH, but they have small kicks as well):
https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,de,lu,ro,cz,no

- with high-IPv6 non-EU (BE left in for comparison):
https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,us,jp,pe,my
The US and Peru show small increments too, but certainly not like the EU countries.

Looking closely, it appears something increased the measurements in Europe a
little on 13 Aug, then a lot on 17 Aug. Erik Taraldsen from Telenor Norway
said they'd been rolling out v6 since summer - but that sounds more gradual
than a sudden increment in mid-August. Any ideas ...?

Regards,
Kate


On Thu, Oct 23, 2014 at 06:03:42PM +0000, Eric Vyncke (evyncke) wrote:
> With the link, it is probably better… still need some caffein
>
> [1]https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,
> de,us,lu
>
> From: Eric Vyncke <[2]evyncke@cisco.com>
> Date: jeudi 23 octobre 2014 09:38
> To: "[3]ipv6-ops@lists.cluenet.de" <[4]ipv6-ops@lists.cluenet.de>
> Subject: Google IPv6 measurements in Europe appear heading down...
>
> For a couple of weeks, it seems that Google IPv6 measurements are
> heading down mainly for Europe. For example, here is a link to a
> presentation of the Google measurements for several European countries
> and USA. There is a clear drop in the last days/weeks for European
> countries but not for USA.
> This includes a big drop for my country (BE) :-O and I have checked
> with all Belgian ISP and they have no explanation as for them 'business
> as usual'. Apnic also does not show such a big drop.
> So, I am guessing either a 'bug' in Google measurements infrastructure
> in Europe or could it be that the IPv6 latency to Google has increased
> a lot so that Happy Eyeball prefers IPv4? Recent measurement of
> dual-stack latency to www.google.com from several Belgian ISP gave 10%
> slower over IPv6.
> Any clue will be welcome
> -éric
>
> References
>
> 1. https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,de,us,lu
> 2. mailto:evyncke@cisco.com
> 3. mailto:ipv6-ops@lists.cluenet.de
> 4. mailto:ipv6-ops@lists.cluenet.de
Re: Google IPv6 measurements in Europe appear heading down... [ In reply to ]
So far all the conversation I've seen about this has been on the eyeball
side. Has anyone looked into whether or not the content networks have
made changes (removing/adding AAAAs for example) that would be
responsible for the temporary skew?

Doug


On 11/5/14 7:23 PM, Kate Lance wrote:
> Hi Éric,
>
> I'm puzzled, not about the recent decline (and now near-recovery) but at the
> odd 'kick' in the stats on Aug 17, mainly for Europe.
>
> Compare Belgium and most of the high-IPv6 EU countries (for clarity I left a
> couple out like FR and CH, but they have small kicks as well):
> https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,de,lu,ro,cz,no
>
> - with high-IPv6 non-EU (BE left in for comparison):
> https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,us,jp,pe,my
> The US and Peru show small increments too, but certainly not like the EU countries.
>
> Looking closely, it appears something increased the measurements in Europe a
> little on 13 Aug, then a lot on 17 Aug. Erik Taraldsen from Telenor Norway
> said they'd been rolling out v6 since summer - but that sounds more gradual
> than a sudden increment in mid-August. Any ideas ...?
>
> Regards,
> Kate
>
>
> On Thu, Oct 23, 2014 at 06:03:42PM +0000, Eric Vyncke (evyncke) wrote:
>> With the link, it is probably better… still need some caffein
>>
>> [1]https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,
>> de,us,lu
>>
>> From: Eric Vyncke <[2]evyncke@cisco.com>
>> Date: jeudi 23 octobre 2014 09:38
>> To: "[3]ipv6-ops@lists.cluenet.de" <[4]ipv6-ops@lists.cluenet.de>
>> Subject: Google IPv6 measurements in Europe appear heading down...
>>
>> For a couple of weeks, it seems that Google IPv6 measurements are
>> heading down mainly for Europe. For example, here is a link to a
>> presentation of the Google measurements for several European countries
>> and USA. There is a clear drop in the last days/weeks for European
>> countries but not for USA.
>> This includes a big drop for my country (BE) :-O and I have checked
>> with all Belgian ISP and they have no explanation as for them 'business
>> as usual'. Apnic also does not show such a big drop.
>> So, I am guessing either a 'bug' in Google measurements infrastructure
>> in Europe or could it be that the IPv6 latency to Google has increased
>> a lot so that Happy Eyeball prefers IPv4? Recent measurement of
>> dual-stack latency to www.google.com from several Belgian ISP gave 10%
>> slower over IPv6.
>> Any clue will be welcome
>> -éric
>>
>> References
>>
>> 1. https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,de,us,lu
>> 2. mailto:evyncke@cisco.com
>> 3. mailto:ipv6-ops@lists.cluenet.de
>> 4. mailto:ipv6-ops@lists.cluenet.de
Re: Google IPv6 measurements in Europe appear heading down... [ In reply to ]
Telenor is big in Norway and is now picking up momentum on a deployment:
http://stats.labs.apnic.net/ipv6/AS2119?c=NO&g=&w=10&x=1


Geoff


> On 6 Nov 2014, at 2:23 pm, Kate Lance <kate@ipv6now.com.au> wrote:
>
> Hi Éric,
>
> I'm puzzled, not about the recent decline (and now near-recovery) but at the
> odd 'kick' in the stats on Aug 17, mainly for Europe.
>
> Compare Belgium and most of the high-IPv6 EU countries (for clarity I left a
> couple out like FR and CH, but they have small kicks as well):
> https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,de,lu,ro,cz,no
>
> - with high-IPv6 non-EU (BE left in for comparison):
> https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,us,jp,pe,my
> The US and Peru show small increments too, but certainly not like the EU countries.
>
> Looking closely, it appears something increased the measurements in Europe a
> little on 13 Aug, then a lot on 17 Aug. Erik Taraldsen from Telenor Norway
> said they'd been rolling out v6 since summer - but that sounds more gradual
> than a sudden increment in mid-August. Any ideas ...?
>
> Regards,
> Kate
>
>
> On Thu, Oct 23, 2014 at 06:03:42PM +0000, Eric Vyncke (evyncke) wrote:
>> With the link, it is probably better… still need some caffein
>>
>> [1]https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,
>> de,us,lu
>>
>> From: Eric Vyncke <[2]evyncke@cisco.com>
>> Date: jeudi 23 octobre 2014 09:38
>> To: "[3]ipv6-ops@lists.cluenet.de" <[4]ipv6-ops@lists.cluenet.de>
>> Subject: Google IPv6 measurements in Europe appear heading down...
>>
>> For a couple of weeks, it seems that Google IPv6 measurements are
>> heading down mainly for Europe. For example, here is a link to a
>> presentation of the Google measurements for several European countries
>> and USA. There is a clear drop in the last days/weeks for European
>> countries but not for USA.
>> This includes a big drop for my country (BE) :-O and I have checked
>> with all Belgian ISP and they have no explanation as for them 'business
>> as usual'. Apnic also does not show such a big drop.
>> So, I am guessing either a 'bug' in Google measurements infrastructure
>> in Europe or could it be that the IPv6 latency to Google has increased
>> a lot so that Happy Eyeball prefers IPv4? Recent measurement of
>> dual-stack latency to www.google.com from several Belgian ISP gave 10%
>> slower over IPv6.
>> Any clue will be welcome
>> -éric
>>
>> References
>>
>> 1. https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,de,us,lu
>> 2. mailto:evyncke@cisco.com
>> 3. mailto:ipv6-ops@lists.cluenet.de
>> 4. mailto:ipv6-ops@lists.cluenet.de
Re: Google IPv6 measurements in Europe appear heading down... [ In reply to ]
On Thu, Nov 06, 2014 at 04:32:34PM +1100, Geoff Huston wrote:
> Telenor is big in Norway and is now picking up momentum on a deployment:
> http://stats.labs.apnic.net/ipv6/AS2119?c=NO&g=&w=10&x=1

Hi Geoff,

https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,no shows
that over August 2014, Norway rose from roughly 2% to 4%, while Belgium rose
from 22% to 29%. But that's percentages, not per capita.

Norway's population is roughly 5 million, Belgium's 11 million. So a 2% rise
among 5 million people isn't likely to stimulate a 7% rise in a country with
more than twice the population.

But looking at the APNIC stats from August, they don't show the coordinated
EU kick - so now I'm wondering if it's an artifact of Google's measurement
technique instead.

Two months earlier, in mid-June, the Google graphs for most European countries
show an odd semi-outage, and from then, compared to the APNIC numbers, they
seem to *undercount* till mid-August, when they all rise together to again be
similar to APNIC's.

Certainly it's apples and oranges comparing APNIC and Google stats, but this
might suggest there wasn't really a rise in IPv6 in mid-August (as measured by
Google), but more of a correction to a previous undercounting. Interesting.

Regards, Kate


> > On 6 Nov 2014, at 2:23 pm, Kate Lance <kate@ipv6now.com.au> wrote:
> >
> > Hi Éric,
> >
> > I'm puzzled, not about the recent decline (and now near-recovery) but at the
> > odd 'kick' in the stats on Aug 17, mainly for Europe.
> >
> > Compare Belgium and most of the high-IPv6 EU countries (for clarity I left a
> > couple out like FR and CH, but they have small kicks as well):
> > https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,de,lu,ro,cz,no
> >
> > - with high-IPv6 non-EU (BE left in for comparison):
> > https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,us,jp,pe,my
> > The US and Peru show small increments too, but certainly not like the EU countries.
> >
> > Looking closely, it appears something increased the measurements in Europe a
> > little on 13 Aug, then a lot on 17 Aug. Erik Taraldsen from Telenor Norway
> > said they'd been rolling out v6 since summer - but that sounds more gradual
> > than a sudden increment in mid-August. Any ideas ...?
> >
> > Regards,
> > Kate
> >
> >
> > On Thu, Oct 23, 2014 at 06:03:42PM +0000, Eric Vyncke (evyncke) wrote:
> >> With the link, it is probably better… still need some caffein
> >>
> >> [1]https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,
> >> de,us,lu
> >>
> >> From: Eric Vyncke <[2]evyncke@cisco.com>
> >> Date: jeudi 23 octobre 2014 09:38
> >> To: "[3]ipv6-ops@lists.cluenet.de" <[4]ipv6-ops@lists.cluenet.de>
> >> Subject: Google IPv6 measurements in Europe appear heading down...
> >>
> >> For a couple of weeks, it seems that Google IPv6 measurements are
> >> heading down mainly for Europe. For example, here is a link to a
> >> presentation of the Google measurements for several European countries
> >> and USA. There is a clear drop in the last days/weeks for European
> >> countries but not for USA.
> >> This includes a big drop for my country (BE) :-O and I have checked
> >> with all Belgian ISP and they have no explanation as for them 'business
> >> as usual'. Apnic also does not show such a big drop.
> >> So, I am guessing either a 'bug' in Google measurements infrastructure
> >> in Europe or could it be that the IPv6 latency to Google has increased
> >> a lot so that Happy Eyeball prefers IPv4? Recent measurement of
> >> dual-stack latency to www.google.com from several Belgian ISP gave 10%
> >> slower over IPv6.
> >> Any clue will be welcome
> >> -éric
> >>
> >> References
> >>
> >> 1. https://www.vyncke.org/ipv6status/compare.php?metric=p&countries=be,de,us,lu
> >> 2. mailto:evyncke@cisco.com
> >> 3. mailto:ipv6-ops@lists.cluenet.de
> >> 4. mailto:ipv6-ops@lists.cluenet.de
Re: Google IPv6 measurements in Europe appear heading down... [ In reply to ]
> Certainly it's apples and oranges comparing APNIC and Google stats, but this
> might suggest there wasn't really a rise in IPv6 in mid-August (as measured by
> Google), but more of a correction to a previous undercounting. Interesting.

There have been, and will likely continue to be, adjustments to the
way in which the experiment runs and the data is analysed. In general
I might not obsess to strongly over individual day-to-day or even
week-to-week fluctuations, but rather keep an eye out for longer term
trends.

-ek