Mailing List Archive

On Sat, Jun 20, 2020 at 09:24:11AM -0700, William Herrin wrote:
> Howdy,
>
> Why is latency between the east and west coasts so bad? Speed of light
> accounts for about 15ms each direction for a 30ms round trip. Where
> does the other 30ms come from and why haven't we gotten rid of it?
>
> c = 186,282 miles/second
> 2742 miles from Seattle to Washington DC mainly driving I-90
>
> 2742/186282 ~= 0.015 seconds

Speed of light in a fiber is more like 124K miles per second. It
depends on the refractive index. And of course amplifiers and stuff.

... JG
--
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"The strain of anti-intellectualism has been a constant thread winding its way
through our political and cultural life, nurtured by the false notion that
democracy means that 'my ignorance is just as good as your knowledge.'"-Asimov

Speed of light in glass ~200 km/s

100 km rtt = 1ms

Coast-to-coast ~6000 km ~60ms

Tim:>

On Sat, Jun 20, 2020 at 12:27 PM William Herrin <bill@herrin.us> wrote:

> Howdy,
>
> Why is latency between the east and west coasts so bad? Speed of light
> accounts for about 15ms each direction for a 30ms round trip. Where
> does the other 30ms come from and why haven't we gotten rid of it?
>
> c = 186,282 miles/second
> 2742 miles from Seattle to Washington DC mainly driving I-90
>
> 2742/186282 ~= 0.015 seconds
>
> Thanks,
> Bill Herrin
>
> --
> William Herrin
> bill@herrin.us
> https://bill.herrin.us/
>


--
Tim:>

And of course in your more realistic example:

2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path

On Sat, Jun 20, 2020 at 12:36 PM Tim Durack <tdurack@gmail.com> wrote:

> Speed of light in glass ~200 km/s
>
> 100 km rtt = 1ms
>
> Coast-to-coast ~6000 km ~60ms
>
> Tim:>
>
> On Sat, Jun 20, 2020 at 12:27 PM William Herrin <bill@herrin.us> wrote:
>
>> Howdy,
>>
>> Why is latency between the east and west coasts so bad? Speed of light
>> accounts for about 15ms each direction for a 30ms round trip. Where
>> does the other 30ms come from and why haven't we gotten rid of it?
>>
>> c = 186,282 miles/second
>> 2742 miles from Seattle to Washington DC mainly driving I-90
>>
>> 2742/186282 ~= 0.015 seconds
>>
>> Thanks,
>> Bill Herrin
>>
>> --
>> William Herrin
>> bill@herrin.us
>> https://bill.herrin.us/
>>
>
>
> --
> Tim:>
>


--
Tim:>

The speed of light in fiber is only about 2/3 the speed of light in a vacuum, so that 15 ms is really about 22.5 ms. That brings the total to about 45 ms.


Some would come from how many miles of extra glass in that 2,742 miles in the form of slack loops.


Some would come from fiber routes not being straight lines. Allied Fiber's formerly planned route from the Westin Building to Equinix Ashburn was about 4,464 miles. That's about 38% longer than your 2,742 miles. Add that 38% to the previous 45 ms and you're at 62.1 ms.




-----
Mike Hammett
Intelligent Computing Solutions
http://www.ics-il.com

Midwest-IX
http://www.midwest-ix.com

----- Original Message -----

From: "William Herrin" <bill@herrin.us>
To: nanog@nanog.org
Sent: Saturday, June 20, 2020 11:24:11 AM
Subject: 60 ms cross-continent

Howdy,

Why is latency between the east and west coasts so bad? Speed of light
accounts for about 15ms each direction for a 30ms round trip. Where
does the other 30ms come from and why haven't we gotten rid of it?

c = 186,282 miles/second
2742 miles from Seattle to Washington DC mainly driving I-90

2742/186282 ~= 0.015 seconds

Thanks,
Bill Herrin

--
William Herrin
bill@herrin.us
https://bill.herrin.us/

Besides the refractive index of glass that makes like go about 2/3rds it
can in a vacuum, "Stuff" also includes many other things like
modulation/demodulation, buffers, etc. I did a quora answer on this you
can find at:

https://www.quora.com/How-can-one-describe-the-delay-characteristics-of-ping-and-traceroute-commands/answer/Tim-Pozar

On 6/20/20 9:29 AM, Joe Greco wrote:
> On Sat, Jun 20, 2020 at 09:24:11AM -0700, William Herrin wrote:
>> Howdy,
>>
>> Why is latency between the east and west coasts so bad? Speed of light
>> accounts for about 15ms each direction for a 30ms round trip. Where
>> does the other 30ms come from and why haven't we gotten rid of it?
>>
>> c = 186,282 miles/second
>> 2742 miles from Seattle to Washington DC mainly driving I-90
>>
>> 2742/186282 ~= 0.015 seconds
>
> Speed of light in a fiber is more like 124K miles per second. It
> depends on the refractive index. And of course amplifiers and stuff.
>
> ... JG
>

Sent from my iPhone

> On Jun 20, 2020, at 9:27 AM, William Herrin <bill@herrin.us> wrote:
>
> ?Howdy,
>
> Why is latency between the east and west coasts so bad? Speed of light
> accounts for about 15ms each direction for a 30ms round trip. Where
> does the other 30ms come from and why haven't we gotten rid of it?
>
> c = 186,282 miles/second

This is c in a vacuum. Light transmission through a medium is slower. In the case of an optical fiber about 31% slower.

My lowest latency transit paths Palo Alto to the ashburn area are around 58ms. the great circle route for the two dcs involved is a distance 2408 miles which gives you a 39.6ms Lower bound.

The path isn’t quite a straight as that, but if you eliminate the 6 routers in the path and count up the oeo regens I’m sure you can account most of the extra in the form of distance.

> 2742 miles from Seattle to Washington DC mainly driving I-90
>
> 2742/186282 ~= 0.015 seconds
>
> Thanks,
> Bill Herrin
>
> --
> William Herrin
> bill@herrin.us
> https://bill.herrin.us/
>

Doing some rough back of the napkin math, an ultra low-latency path from, say, the Westin to 1275 K in Seattle will be in the 59 ms range. This is considerably longer than the I-90 driving distance would suggest because:
- Best case optical distance is more like 5500 km, in part because the path actually will go Chicago-NJ-WDC and in part because a distance of 5000 km by right-of-way will be more like 5500 km when you account for things like maintenance coils, in-building wiring, etc.
- You’ll need (at least) three OEO regens on that distance, since there’s no value in spending 5x to deploy an optical system that wouldn’t need to (like the ones that would manage that distance subsea). This is in addition to ~60 in-line amplification nodes, although that adds significantly less latency even in aggregate

Some of that is simply due to cost savings. In theory, you could probably spend a boatload of money to build a route that cuts off some of the distance inefficiency and gets you closer to 4500 km optical distance with minimal slack coil, and maybe no regens, so you get a real-world performance of 46 ms. But there are no algo trading sites of importance in DC, and for everybody else there’s not enough money in the difference between 46 and 59 ms for someone to go invest in that type of deployment.

Dave Cohen
craetdave@gmail.com

> On Jun 20, 2020, at 12:44 PM, Tim Durack <tdurack@gmail.com> wrote:
>
> ?
> And of course in your more realistic example:
>
> 2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path
>
>> On Sat, Jun 20, 2020 at 12:36 PM Tim Durack <tdurack@gmail.com> wrote:
>> Speed of light in glass ~200 km/s
>>
>> 100 km rtt = 1ms
>>
>> Coast-to-coast ~6000 km ~60ms
>>
>> Tim:>
>>
>>> On Sat, Jun 20, 2020 at 12:27 PM William Herrin <bill@herrin.us> wrote:
>>> Howdy,
>>>
>>> Why is latency between the east and west coasts so bad? Speed of light
>>> accounts for about 15ms each direction for a 30ms round trip. Where
>>> does the other 30ms come from and why haven't we gotten rid of it?
>>>
>>> c = 186,282 miles/second
>>> 2742 miles from Seattle to Washington DC mainly driving I-90
>>>
>>> 2742/186282 ~= 0.015 seconds
>>>
>>> Thanks,
>>> Bill Herrin
>>>
>>> --
>>> William Herrin
>>> bill@herrin.us
>>> https://bill.herrin.us/
>>
>>
>> --
>> Tim:>
>
>
> --
> Tim:>

An intriguing development in fiber optic media is hollow core optical fiber, which achieves 99.7% of the speed of light in a vacuum.

https://www.extremetech.com/computing/151498-researchers-create-fiber-network-that-operates-at-99-7-speed-of-light-smashes-speed-and-latency-records

-mel

On Jun 20, 2020, at 10:14 AM, Dave Cohen <craetdave@gmail.com> wrote:

? Doing some rough back of the napkin math, an ultra low-latency path from, say, the Westin to 1275 K in Seattle will be in the 59 ms range. This is considerably longer than the I-90 driving distance would suggest because:
- Best case optical distance is more like 5500 km, in part because the path actually will go Chicago-NJ-WDC and in part because a distance of 5000 km by right-of-way will be more like 5500 km when you account for things like maintenance coils, in-building wiring, etc.
- You’ll need (at least) three OEO regens on that distance, since there’s no value in spending 5x to deploy an optical system that wouldn’t need to (like the ones that would manage that distance subsea). This is in addition to ~60 in-line amplification nodes, although that adds significantly less latency even in aggregate

Some of that is simply due to cost savings. In theory, you could probably spend a boatload of money to build a route that cuts off some of the distance inefficiency and gets you closer to 4500 km optical distance with minimal slack coil, and maybe no regens, so you get a real-world performance of 46 ms. But there are no algo trading sites of importance in DC, and for everybody else there’s not enough money in the difference between 46 and 59 ms for someone to go invest in that type of deployment.

Dave Cohen
craetdave@gmail.com

On Jun 20, 2020, at 12:44 PM, Tim Durack <tdurack@gmail.com> wrote:

?
And of course in your more realistic example:

2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path

On Sat, Jun 20, 2020 at 12:36 PM Tim Durack <tdurack@gmail.com<mailto:tdurack@gmail.com>> wrote:
Speed of light in glass ~200 km/s

100 km rtt = 1ms

Coast-to-coast ~6000 km ~60ms

Tim:>

On Sat, Jun 20, 2020 at 12:27 PM William Herrin <bill@herrin.us<mailto:bill@herrin.us>> wrote:
Howdy,

Why is latency between the east and west coasts so bad? Speed of light
accounts for about 15ms each direction for a 30ms round trip. Where
does the other 30ms come from and why haven't we gotten rid of it?

c = 186,282 miles/second
2742 miles from Seattle to Washington DC mainly driving I-90

2742/186282 ~= 0.015 seconds

Thanks,
Bill Herrin

--
William Herrin
bill@herrin.us<mailto:bill@herrin.us>
https://bill.herrin.us/


--
Tim:>


--
Tim:>

And thus far, no one has mentioned switching speed and other
electronic overhead such as the transceivers (that's the big one,
IIRC.)

I also don't recall if anyone mentioned that the 30ms is as the
photon flies, not fiber distance.

-Wayne

On Sat, Jun 20, 2020 at 05:32:30PM +0000, Mel Beckman wrote:
> An intriguing development in fiber optic media is hollow core optical fiber, which achieves 99.7% of the speed of light in a vacuum.
>
> https://www.extremetech.com/computing/151498-researchers-create-fiber-network-that-operates-at-99-7-speed-of-light-smashes-speed-and-latency-records
>
> -mel
>
> On Jun 20, 2020, at 10:14 AM, Dave Cohen <craetdave@gmail.com> wrote:
>
> ??? Doing some rough back of the napkin math, an ultra low-latency path from, say, the Westin to 1275 K in Seattle will be in the 59 ms range. This is considerably longer than the I-90 driving distance would suggest because:
> - Best case optical distance is more like 5500 km, in part because the path actually will go Chicago-NJ-WDC and in part because a distance of 5000 km by right-of-way will be more like 5500 km when you account for things like maintenance coils, in-building wiring, etc.
> - You???ll need (at least) three OEO regens on that distance, since there???s no value in spending 5x to deploy an optical system that wouldn???t need to (like the ones that would manage that distance subsea). This is in addition to ~60 in-line amplification nodes, although that adds significantly less latency even in aggregate
>
> Some of that is simply due to cost savings. In theory, you could probably spend a boatload of money to build a route that cuts off some of the distance inefficiency and gets you closer to 4500 km optical distance with minimal slack coil, and maybe no regens, so you get a real-world performance of 46 ms. But there are no algo trading sites of importance in DC, and for everybody else there???s not enough money in the difference between 46 and 59 ms for someone to go invest in that type of deployment.
>
> Dave Cohen
> craetdave@gmail.com
>
> On Jun 20, 2020, at 12:44 PM, Tim Durack <tdurack@gmail.com> wrote:
>
> ???
> And of course in your more realistic example:
>
> 2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path
>
> On Sat, Jun 20, 2020 at 12:36 PM Tim Durack <tdurack@gmail.com<mailto:tdurack@gmail.com>> wrote:
> Speed of light in glass ~200 km/s
>
> 100 km rtt = 1ms
>
> Coast-to-coast ~6000 km ~60ms
>
> Tim:>
>
> On Sat, Jun 20, 2020 at 12:27 PM William Herrin <bill@herrin.us<mailto:bill@herrin.us>> wrote:
> Howdy,
>
> Why is latency between the east and west coasts so bad? Speed of light
> accounts for about 15ms each direction for a 30ms round trip. Where
> does the other 30ms come from and why haven't we gotten rid of it?
>
> c = 186,282 miles/second
> 2742 miles from Seattle to Washington DC mainly driving I-90
>
> 2742/186282 ~= 0.015 seconds
>
> Thanks,
> Bill Herrin
>
> --
> William Herrin
> bill@herrin.us<mailto:bill@herrin.us>
> https://bill.herrin.us/
>
>
> --
> Tim:>
>
>
> --
> Tim:>

---
Wayne Bouchard
web@typo.org
Network Dude
http://www.typo.org/~web/

On Sat, 20 Jun 2020 at 20:52, Wayne Bouchard <web@typo.org> wrote:

> And thus far, no one has mentioned switching speed and other
> electronic overhead such as the transceivers (that's the big one,
> IIRC.)

This will be something from tens of meters (low lat swich), to few
hundred meters (typical pipeline), to 2km delay (NPU+FAB+NPU) per
active IP device. If that is a big one, I guess it depends, cross
atlantic, no, inside rack, maybe.

--
++ytti

On 2020-06-20, at 19:07, Joel Jaeggli <joelja@bogus.com> wrote:
>
> This is c in a vacuum. Light transmission through a medium is slower.

Ob-movie: https://en.wikipedia.org/wiki/The_Hummingbird_Project

Grüße, Carsten

Hello,

  Taking advantage of this thread may I ask something?. I have heard of
"wireless fiber optic", something like an antenna with a laser pointing
from one building to the other, having said this I can assume this link
with have lower RTT than a laser thru a fiber optic made of glass?


Thanks,


Alejandro,


On 6/20/20 1:11 PM, Dave Cohen wrote:
> Doing some rough back of the napkin math, an ultra low-latency path
> from, say, the Westin to 1275 K in Seattle will be in the 59 ms range.
> This is considerably longer than the I-90 driving distance would
> suggest because:
> - Best case optical distance is more like 5500 km, in part because the
> path actually will go Chicago-NJ-WDC and in part because a distance of
> 5000 km by right-of-way will be more like 5500 km when you account for
> things like maintenance coils, in-building wiring, etc.
> - You’ll need (at least) three OEO regens on that distance, since
> there’s no value in spending 5x to deploy an optical system that
> wouldn’t need to (like the ones that would manage that distance
> subsea). This is in addition to ~60 in-line amplification nodes,
> although that adds significantly less latency even in aggregate
>
> Some of that is simply due to cost savings. In theory, you could
> probably spend a boatload of money to build a route that cuts off some
> of the distance inefficiency and gets you closer to 4500 km optical
> distance with minimal slack coil, and maybe no regens, so you get a
> real-world performance of 46 ms. But there are no algo trading sites
> of importance in DC, and for everybody else there’s not enough money
> in the difference between 46 and 59 ms for someone to go invest in
> that type of deployment.
>
> Dave Cohen
> craetdave@gmail.com
>
>> On Jun 20, 2020, at 12:44 PM, Tim Durack <tdurack@gmail.com> wrote:
>>
>> ?
>> And of course in your more realistic example:
>>
>> 2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path
>>
>> On Sat, Jun 20, 2020 at 12:36 PM Tim Durack <tdurack@gmail.com
>> <mailto:tdurack@gmail.com>> wrote:
>>
>> Speed of light in glass ~200 km/s
>>
>> 100 km rtt = 1ms
>>
>> Coast-to-coast ~6000 km ~60ms
>>
>> Tim:>
>>
>> On Sat, Jun 20, 2020 at 12:27 PM William Herrin <bill@herrin.us
>> <mailto:bill@herrin.us>> wrote:
>>
>> Howdy,
>>
>> Why is latency between the east and west coasts so bad? Speed
>> of light
>> accounts for about 15ms each direction for a 30ms round trip.
>> Where
>> does the other 30ms come from and why haven't we gotten rid
>> of it?
>>
>> c = 186,282 miles/second
>> 2742 miles from Seattle to Washington DC mainly driving I-90
>>
>> 2742/186282 ~= 0.015 seconds
>>
>> Thanks,
>> Bill Herrin
>>
>> --
>> William Herrin
>> bill@herrin.us <mailto:bill@herrin.us>
>> https://bill.herrin.us/
>>
>>
>>
>> --
>> Tim:>
>>
>>
>>
>> --
>> Tim:>

This was also pitched as one of the killer-apps for the SpaceX
Starlink satellite array, particularly for cross-Atlantic and
cross-Pacific trading.

https://blogs.cfainstitute.org/marketintegrity/2019/06/25/fspacex-is-opening-up-the-next-frontier-for-hft/

"Several commentators quickly caught onto the fact that an extremely
expensive network whose main selling point is long-distance,
low-latency coverage has a unique chance to fund its growth by
addressing the needs of a wealthy market that has a high willingness
to pay — high-frequency traders."

Regards
Marshall

On Sat, Jun 20, 2020 at 2:01 PM Carsten Bormann <cabo@tzi.org> wrote:
>
> On 2020-06-20, at 19:07, Joel Jaeggli <joelja@bogus.com> wrote:
> >
> > This is c in a vacuum. Light transmission through a medium is slower.
>
> Ob-movie: https://en.wikipedia.org/wiki/The_Hummingbird_Project
>
> Grüße, Carsten
>

On 6/20/20 1:56 PM, Saku Ytti wrote:
> On Sat, 20 Jun 2020 at 20:52, Wayne Bouchard <web@typo.org> wrote:
>
>> And thus far, no one has mentioned switching speed and other
>> electronic overhead such as the transceivers (that's the big one,
>> IIRC.)
> This will be something from tens of meters (low lat swich), to few
> hundred meters (typical pipeline), to 2km delay (NPU+FAB+NPU) per
> active IP device. If that is a big one, I guess it depends, cross
> atlantic, no, inside rack, maybe.

I think he might be referring to the newer modulation types (QAM) on long haul
transport. There's quite a bit of time in uS that the encoding takes into QAM
and adding FEC. You typically won't see this at the plug-able level between
switches and stuff.

60ms is nothing really, and I'm happy I don't need to play in the HFT space
anymore. I do wish my home connection wasn't 60 ms across town as spectrum
wants takes TPA-ATL-DCA-DEN-NY to get to my rack. :-)
--
Bryan Fields

727-409-1194 - Voice
http://bryanfields.net

On Sat, Jun 20, 2020 at 12:56 PM Alejandro Acosta <
alejandroacostaalamo@gmail.com> wrote:

> Hello,
>
> Taking advantage of this thread may I ask something?. I have heard of
> "wireless fiber optic", something like an antenna with a laser pointing
> from one building to the other, having said this I can assume this link
> with have lower RTT than a laser thru a fiber optic made of glass?
>
>
See: Terrabeam from about the year 2000.

--
Joe Hamelin, W7COM, Tulalip, WA, +1 (360) 474-7474

On Sat, Jun 20, 2020 at 16:14 Bryan Fields <Bryan@bryanfields.net> wrote:

> On 6/20/20 1:56 PM, Saku Ytti wrote:
> > On Sat, 20 Jun 2020 at 20:52, Wayne Bouchard <web@typo.org> wrote:
> >
> >> And thus far, no one has mentioned switching speed and other
> >> electronic overhead such as the transceivers (that's the big one,
> >> IIRC.)
> > This will be something from tens of meters (low lat swich), to few
> > hundred meters (typical pipeline), to 2km delay (NPU+FAB+NPU) per
> > active IP device. If that is a big one, I guess it depends, cross
> > atlantic, no, inside rack, maybe.
>
> I think he might be referring to the newer modulation types (QAM) on long
> haul
> transport. There's quite a bit of time in uS that the encoding takes into
> QAM
> and adding FEC. You typically won't see this at the plug-able level
> between
> switches and stuff.
>
> 60ms is nothing really, and I'm happy I don't need to play in the HFT space
> anymore. I do wish my home connection wasn't 60 ms across town as spectrum
> wants takes TPA-ATL-DCA-DEN-NY to get to my rack. :-)



working on that ...... :-)






> --
> Bryan Fields
>
> 727-409-1194 - Voice
> http://bryanfields.net
>

Did you not read my posting on Quora?

Tim

On 6/20/20 10:49 AM, Wayne Bouchard wrote:
> And thus far, no one has mentioned switching speed and other
> electronic overhead such as the transceivers (that's the big one,
> IIRC.)
>
> I also don't recall if anyone mentioned that the 30ms is as the
> photon flies, not fiber distance.
>
> -Wayne
>
> On Sat, Jun 20, 2020 at 05:32:30PM +0000, Mel Beckman wrote:
>> An intriguing development in fiber optic media is hollow core optical fiber, which achieves 99.7% of the speed of light in a vacuum.
>>
>> https://www.extremetech.com/computing/151498-researchers-create-fiber-network-that-operates-at-99-7-speed-of-light-smashes-speed-and-latency-records
>>
>> -mel
>>
>> On Jun 20, 2020, at 10:14 AM, Dave Cohen <craetdave@gmail.com> wrote:
>>
>> ??? Doing some rough back of the napkin math, an ultra low-latency path from, say, the Westin to 1275 K in Seattle will be in the 59 ms range. This is considerably longer than the I-90 driving distance would suggest because:
>> - Best case optical distance is more like 5500 km, in part because the path actually will go Chicago-NJ-WDC and in part because a distance of 5000 km by right-of-way will be more like 5500 km when you account for things like maintenance coils, in-building wiring, etc.
>> - You???ll need (at least) three OEO regens on that distance, since there???s no value in spending 5x to deploy an optical system that wouldn???t need to (like the ones that would manage that distance subsea). This is in addition to ~60 in-line amplification nodes, although that adds significantly less latency even in aggregate
>>
>> Some of that is simply due to cost savings. In theory, you could probably spend a boatload of money to build a route that cuts off some of the distance inefficiency and gets you closer to 4500 km optical distance with minimal slack coil, and maybe no regens, so you get a real-world performance of 46 ms. But there are no algo trading sites of importance in DC, and for everybody else there???s not enough money in the difference between 46 and 59 ms for someone to go invest in that type of deployment.
>>
>> Dave Cohen
>> craetdave@gmail.com
>>
>> On Jun 20, 2020, at 12:44 PM, Tim Durack <tdurack@gmail.com> wrote:
>>
>> ???
>> And of course in your more realistic example:
>>
>> 2742 miles = 4412 km ~ 44 ms optical rtt with no OEO in the path
>>
>> On Sat, Jun 20, 2020 at 12:36 PM Tim Durack <tdurack@gmail.com<mailto:tdurack@gmail.com>> wrote:
>> Speed of light in glass ~200 km/s
>>
>> 100 km rtt = 1ms
>>
>> Coast-to-coast ~6000 km ~60ms
>>
>> Tim:>
>>
>> On Sat, Jun 20, 2020 at 12:27 PM William Herrin <bill@herrin.us<mailto:bill@herrin.us>> wrote:
>> Howdy,
>>
>> Why is latency between the east and west coasts so bad? Speed of light
>> accounts for about 15ms each direction for a 30ms round trip. Where
>> does the other 30ms come from and why haven't we gotten rid of it?
>>
>> c = 186,282 miles/second
>> 2742 miles from Seattle to Washington DC mainly driving I-90
>>
>> 2742/186282 ~= 0.015 seconds
>>
>> Thanks,
>> Bill Herrin
>>
>> --
>> William Herrin
>> bill@herrin.us<mailto:bill@herrin.us>
>> https://bill.herrin.us/
>>
>>
>> --
>> Tim:>
>>
>>
>> --
>> Tim:>
>
> ---
> Wayne Bouchard
> web@typo.org
> Network Dude
> http://www.typo.org/~web/
>

On Sat, 20 Jun 2020 at 23:14, Bryan Fields <Bryan@bryanfields.net> wrote:

> I think he might be referring to the newer modulation types (QAM) on long haul
> transport. There's quite a bit of time in uS that the encoding takes into QAM
> and adding FEC. You typically won't see this at the plug-able level between
> switches and stuff.

FEC is low tens of meters (i.e. low tens of nanoseconds), QAM is less.
Won't impact the pipeline or NPU scenarios meaningfully, will impact
the low latency scenario.

--
++ytti

Mel Beckman <mel@beckman.org> wrote:

> An intriguing development in fiber optic media is hollow core optical
> fiber, which achieves 99.7% of the speed of light in a vacuum.
>
> https://www.extremetech.com/computing/151498-researchers-create-fiber-network-that-operates-at-99-7-speed-of-light-smashes-speed-and-latency-records

Here's an update from 7 years after that article which hints at the
downside of hollow core fibre:

https://phys.org/news/2020-03-hollow-core-fiber-technology-mainstream-optical.html

It sounds like attenuation was a big problem: "in the space of 18 months
the attenuation in data-transmitting hollow-core fibers has been reduced
by over a factor of 10, from 3.5dB/km to only 0.28 dB/km within a factor
of two of the attenuation of conventional all-glass fiber technology."

Tony.
--
f.anthony.n.finch <dot@dotat.at> http://dotat.at/
Shetland Isles: Southeasterly 5 or 6, veering southerly or southwesterly 3 or
4, then backing southeasterly 5 later in southwest. Slight or moderate,
occasionally rough later in far west. Occasional rain then mainly fair, but
showers far in east. Good, occasionally moderate.

On Sat, Jun 20, 2020 at 5:05 PM Marshall Eubanks <marshall.eubanks@gmail.com>
wrote:

> This was also pitched as one of the killer-apps for the SpaceX
> Starlink satellite array, particularly for cross-Atlantic and
> cross-Pacific trading.
>
>
> https://blogs.cfainstitute.org/marketintegrity/2019/06/25/fspacex-is-opening-up-the-next-frontier-for-hft/
>
> "Several commentators quickly caught onto the fact that an extremely
> expensive network whose main selling point is long-distance,
> low-latency coverage has a unique chance to fund its growth by
> addressing the needs of a wealthy market that has a high willingness
> to pay — high-frequency traders."
>
>
This is a nice plot for a movie, but not how HFT is really done. It's so
much easier to colocate on the same datacenter of the exchange and run
algorithms from there; while those algorithms need humans to guide their
strategy, the human thought process takes a couple of seconds anyways. So
the real HFTs keep using the defined strategy while the human controller
doesn't tell it otherwise.

And in order to preserve equality among traders, each exchange already adds
physically (loops of fiber or copper cable) some ns to closer racks so
everyone gets at the system at the same time.

And then comes a really high added latency of the trade risk controller,
which limits what a trader is allowed to expose itself to what is deposited
or agreed with the exchange. And this comes with both latency and jitter
due to its implementation, making even the faster HFT only faster on
average, not faster at every transaction.


Rubens

On Sun, Jun 21, 2020 at 02:17:08PM -0300, Rubens Kuhl wrote:
> On Sat, Jun 20, 2020 at 5:05 PM Marshall Eubanks <marshall.eubanks@gmail.com>
> wrote:
>
> > This was also pitched as one of the killer-apps for the SpaceX
> > Starlink satellite array, particularly for cross-Atlantic and
> > cross-Pacific trading.
> >
> >
> > https://blogs.cfainstitute.org/marketintegrity/2019/06/25/fspacex-is-opening-up-the-next-frontier-for-hft/
> >
> > "Several commentators quickly caught onto the fact that an extremely
> > expensive network whose main selling point is long-distance,
> > low-latency coverage has a unique chance to fund its growth by
> > addressing the needs of a wealthy market that has a high willingness
> > to pay — high-frequency traders."
> >
> >
> This is a nice plot for a movie, but not how HFT is really done. It's so
> much easier to colocate on the same datacenter of the exchange and run
> algorithms from there; while those algorithms need humans to guide their
> strategy, the human thought process takes a couple of seconds anyways. So
> the real HFTs keep using the defined strategy while the human controller
> doesn't tell it otherwise.

For faster access to one exchange, yes, absolutely, colocate at the
exchange. But there's more then one exchange.

As one example, many index futures trade in Chicago. The stocks that
make up those indices mostly trade in New York. There's money to be
made on the arbitrage, if your Chicago algorithms get faster
information from New York (and vice versa) than everyone else's
algorithms.

More expensive but shorter fiber routes have been build between NYC and
Chicago for this reason, as have a microwave paths (to get
speed-of-light in air rather than in glass). There's competition to
have the microwave towers as close as possible to the data centers,
because the last mile is fiber so the longer your last mile, the less
valuable your network.

https://www.bloomberg.com/news/features/2019-03-08/the-gazillion-dollar-standoff-over-two-high-frequency-trading-towers

-- Brett

On 6/21/20 1:53 PM, Brett Frankenberger wrote:
> On Sun, Jun 21, 2020 at 02:17:08PM -0300, Rubens Kuhl wrote:
>> On Sat, Jun 20, 2020 at 5:05 PM Marshall Eubanks <marshall.eubanks@gmail.com>
>> wrote:
>>
>>> This was also pitched as one of the killer-apps for the SpaceX
>>> Starlink satellite array, particularly for cross-Atlantic and
>>> cross-Pacific trading.
>>>
>>>
>>> https://blogs.cfainstitute.org/marketintegrity/2019/06/25/fspacex-is-opening-up-the-next-frontier-for-hft/
>>>
>>> "Several commentators quickly caught onto the fact that an extremely
>>> expensive network whose main selling point is long-distance,
>>> low-latency coverage has a unique chance to fund its growth by
>>> addressing the needs of a wealthy market that has a high willingness
>>> to pay — high-frequency traders."
>>>
>>>
>> This is a nice plot for a movie, but not how HFT is really done. It's so
>> much easier to colocate on the same datacenter of the exchange and run
>> algorithms from there; while those algorithms need humans to guide their
>> strategy, the human thought process takes a couple of seconds anyways. So
>> the real HFTs keep using the defined strategy while the human controller
>> doesn't tell it otherwise.
> For faster access to one exchange, yes, absolutely, colocate at the
> exchange. But there's more then one exchange.
>
> As one example, many index futures trade in Chicago. The stocks that
> make up those indices mostly trade in New York. There's money to be
> made on the arbitrage, if your Chicago algorithms get faster
> information from New York (and vice versa) than everyone else's
> algorithms.
>
> More expensive but shorter fiber routes have been build between NYC and
> Chicago for this reason, as have a microwave paths (to get
> speed-of-light in air rather than in glass). There's competition to
> have the microwave towers as close as possible to the data centers,
> because the last mile is fiber so the longer your last mile, the less
> valuable your network.
>
> https://www.bloomberg.com/news/features/2019-03-08/the-gazillion-dollar-standoff-over-two-high-frequency-trading-towers


... and similar to this:
https://www.extremetech.com/extreme/122989-1-5-billion-the-cost-of-cutting-london-toyko-latency-by-60ms


>
> -- Brett

> > This is a nice plot for a movie, but not how HFT is really done. It's so
> > much easier to colocate on the same datacenter of the exchange and run
> > algorithms from there; while those algorithms need humans to guide their
> > strategy, the human thought process takes a couple of seconds anyways. So
> > the real HFTs keep using the defined strategy while the human controller
> > doesn't tell it otherwise.
>
> For faster access to one exchange, yes, absolutely, colocate at the
> exchange. But there's more then one exchange.
>

Yes, but to do real HFT you will need to colocate at each exchange.
Otherwise your competitors have a head start on you.


>
> As one example, many index futures trade in Chicago. The stocks that
> make up those indices mostly trade in New York. There's money to be
> made on the arbitrage, if your Chicago algorithms get faster
> information from New York (and vice versa) than everyone else's
> algorithms.
>

Most traded index futures are longer than just that day closing, usually
months to a year in advance.
They are influenced mostly by traders perception on economic futures, and
the current stocks valuation is a poor proxy for it.
There is more chance in reading the news feeds and speculating its impact
on perception than stocks.

Rubens

Serious HFT moved to shortwave years ago. The chicago-NYC routes by
microwave still exist, but are only for things that need higher data rates
(as measured in kbps). It's hard to hide a giant log-periodic or yagi-uda
antenna. The sites near Chicago that are aimed at London are well known to
those in the industry.



On Sun, Jun 21, 2020 at 10:53 AM Brett Frankenberger <rbf+nanog@panix.com>
wrote:

> On Sun, Jun 21, 2020 at 02:17:08PM -0300, Rubens Kuhl wrote:
> > On Sat, Jun 20, 2020 at 5:05 PM Marshall Eubanks <
> marshall.eubanks@gmail.com>
> > wrote:
> >
> > > This was also pitched as one of the killer-apps for the SpaceX
> > > Starlink satellite array, particularly for cross-Atlantic and
> > > cross-Pacific trading.
> > >
> > >
> > >
> https://blogs.cfainstitute.org/marketintegrity/2019/06/25/fspacex-is-opening-up-the-next-frontier-for-hft/
> > >
> > > "Several commentators quickly caught onto the fact that an extremely
> > > expensive network whose main selling point is long-distance,
> > > low-latency coverage has a unique chance to fund its growth by
> > > addressing the needs of a wealthy market that has a high willingness
> > > to pay — high-frequency traders."
> > >
> > >
> > This is a nice plot for a movie, but not how HFT is really done. It's so
> > much easier to colocate on the same datacenter of the exchange and run
> > algorithms from there; while those algorithms need humans to guide their
> > strategy, the human thought process takes a couple of seconds anyways. So
> > the real HFTs keep using the defined strategy while the human controller
> > doesn't tell it otherwise.
>
> For faster access to one exchange, yes, absolutely, colocate at the
> exchange. But there's more then one exchange.
>
> As one example, many index futures trade in Chicago. The stocks that
> make up those indices mostly trade in New York. There's money to be
> made on the arbitrage, if your Chicago algorithms get faster
> information from New York (and vice versa) than everyone else's
> algorithms.
>
> More expensive but shorter fiber routes have been build between NYC and
> Chicago for this reason, as have a microwave paths (to get
> speed-of-light in air rather than in glass). There's competition to
> have the microwave towers as close as possible to the data centers,
> because the last mile is fiber so the longer your last mile, the less
> valuable your network.
>
>
> https://www.bloomberg.com/news/features/2019-03-08/the-gazillion-dollar-standoff-over-two-high-frequency-trading-towers
>
> -- Brett
>