Mailing List Archive

hey,

> I need to push 100G over 100 km of dark fiber. Since there are no 100G pluggable optics with this reach (~25 dB), I have been offered coherent transport systems to solve my problem. This is all good and well, except total system costs start from high five figures.

100G-ZR4 QSFP28 is on the market and works. Just watch out for power
limitations, your typical DC switch might not support it but proper
stuff has no problems providing 6.5W of power.

--
tarko

Hi Jared,

If it’s just a single 100G channel needed you could try 100GBASE-ZR4. Specified for 80km, 30db power budget they could actually reach more the 80km.
Dispersion should also be „no" problem in the 1310nm length. I have to say that I never tried this on 100km distance without coherent solutions.

Best regards,
Vincentz
PS: https://www.flexoptix.net/en/transceiver/q-161hg-80.html?co10728=100306

> Am 30.10.2020 um 15:19 schrieb Jared Brown <nanog-isp@mail.com>:
>
> Hello NANOG!
>
> I need to push 100G over 100 km of dark fiber. Since there are no 100G pluggable optics with this reach (~25 dB), I have been offered coherent transport systems to solve my problem. This is all good and well, except total system costs start from high five figures.
>
> So, my question is, do I have any other options?
>
> I can't help noticing that you can break out a 100G QSFP into four 25G QSFPs. 25G DWDM systems are relatively inexpensive (low five figures), but can you make 25G DWDM go 100 km?
>
> I only need the one 100G, so I don't really need a highly scalable DWDM system. I can't put anything midspan, or if I could it would cost more than just going with a coherent system.
>
>
> Jared

You may also find that 100G PAM4 could work. There are some vendors that
sell the optic, and an outboard EDFA + DCF pizza box.

Dale

Thus spake Tarko Tikan (tarko@lanparty.ee) on Fri, Oct 30, 2020 at 04:25:58PM +0200:
> hey,
>
> > I need to push 100G over 100 km of dark fiber. Since there are no 100G pluggable optics with this reach (~25 dB), I have been offered coherent transport systems to solve my problem. This is all good and well, except total system costs start from high five figures.
>
> 100G-ZR4 QSFP28 is on the market and works. Just watch out for power
> limitations, your typical DC switch might not support it but proper stuff
> has no problems providing 6.5W of power.
>
> --
> tarko

On 10/30/20 10:27 AM, Vincentz Petzholtz wrote:
> If it’s just a single 100G channel needed you could try 100GBASE-ZR4. Specified for 80km, 30db power budget they could actually reach more the 80km.
> Dispersion should also be „no" problem in the 1310nm length. I have to say that I never tried this on 100km distance without coherent solutions.

If you end up marginal with the ZR4 at the receive end, you may be able
to use a silicon optical preamp to make up the margin.

If you are at the noise floor already (ZR4 receivers are pretty good),
you may just need to dump more power into the line. I recently came
across the PDFA (like an EDFA but the fiber is doped with Praseodymium
instead of Erbium) which operate on the 1310 band and can push upwards
of 20-23dBm of power out. They are not cheap (low 5 figures), but they
are cheaper than coherent and will still let you run on 1310 so you
don't have chromatic dispersion issues. With good (ER4 or ZR4)
receivers good down to -20dBm or so, you've got 30-33dB of link budget
which should just about make your 100km if it's decent fiber. They're
also usable as a pre-amp (different configuration optimized for gain and
noise figure instead of power, similar to EDFA preamps) and have
performance potentially better than silicon amps in that situation.

The 1550 PAM4 pizza box EDFA+Mux+DCM that others have mentioned may also
work and end up even cheaper, but I normally see those for 40-60km or
"up to" 80km with them really pushing the link budget at that point.

Honestly, I'd be tempted to just suck it up and do a coherent solution,
though I admit it would probably be at least 2x the cost. You can
probably get a 200G carrier, though.
--
Brandon Martin

Hi jared

as others have pointed out there are lots of options


inphi offers these
https://www.inphi.com/products/colorz/


or use a box like packetlight, here is a Arista solution brief
https://www.arista.com/assets/data/pdf/Whitepapers/Arista_Packetlight_100G_Extension_Solution.pdf
and if you serch for openline systems there are a few that do smaller systems 2/4/8 ports that are available
FS even offers this
https://www.fs.com/de-en/specials/100g-fmx-transport-platform-103.html
more expensive than optics but an alternative and you can stay in low 5 figures.

cisco has the ncs55a2 with these
https://www.cisco.com/c/en/us/products/collateral/routers/network-convergence-system-5500-series/datasheet-c78-743732.html
that costs like a gazzilion dollars but your company may have great discounts...


HTH

Brian


> -----Original Message-----
> From: NANOG <nanog-bounces+b.turnbow=twt.it@nanog.org> On Behalf Of
> Jared Brown
> Sent: Friday, October 30, 2020 3:19 PM
> To: nanog@nanog.org
> Subject: 100G over 100 km of dark fiber
>
> Hello NANOG!
>
> I need to push 100G over 100 km of dark fiber. Since there are no 100G
> pluggable optics with this reach (~25 dB), I have been offered coherent
> transport systems to solve my problem. This is all good and well, except total
> system costs start from high five figures.
>
> So, my question is, do I have any other options?
>
> I can't help noticing that you can break out a 100G QSFP into four 25G QSFPs.
> 25G DWDM systems are relatively inexpensive (low five figures), but can you
> make 25G DWDM go 100 km?
>
> I only need the one 100G, so I don't really need a highly scalable DWDM
> system. I can't put anything midspan, or if I could it would cost more than just
> going with a coherent system.
>
>
> Jared

Dale W. Carder wrote on 30/10/2020 14:33:
> You may also find that 100G PAM4 could work.

not at 100km. This would be outside the dispersion tolerance limits for
pam4.

Nick

You could break this into 10x 10g coherent lanes, but you’re going to end up back close to coherent 100g prices.

You’re at the threshold distance where you’re past all the short range tech and are seriously pushing it - whereas the 100g coherent tech is just taking off.

How important is this link?

Ms. Benjamin PD Cannon, ASCE
6x7 Networks & 6x7 Telecom, LLC
CEO
ben@6by7.net
"The only fully end-to-end encrypted global telecommunications company in the world.”

FCC License KJ6FJJ



> On Oct 30, 2020, at 7:19 AM, Jared Brown <nanog-isp@mail.com> wrote:
>
> ?Hello NANOG!
>
> I need to push 100G over 100 km of dark fiber. Since there are no 100G pluggable optics with this reach (~25 dB), I have been offered coherent transport systems to solve my problem. This is all good and well, except total system costs start from high five figures.
>
> So, my question is, do I have any other options?
>
> I can't help noticing that you can break out a 100G QSFP into four 25G QSFPs. 25G DWDM systems are relatively inexpensive (low five figures), but can you make 25G DWDM go 100 km?
>
> I only need the one 100G, so I don't really need a highly scalable DWDM system. I can't put anything midspan, or if I could it would cost more than just going with a coherent system.
>
>
> Jared

hey,

> If it’s just a single 100G channel needed you could try 100GBASE-ZR4. Specified for 80km, 30db power budget they could actually reach more the 80km.
> Dispersion should also be „no" problem in the 1310nm length. I have to say that I never tried this on 100km distance without coherent solutions.

Just to add to my original suggestion, we just did 100G-ZR4 over 30dB
link with pre-FEC BER 3.174E-11.

As OP is asking for a solution for 25dB I don't see any reason why ZR4
would not work and why you would need coherent, amplifiers or any other
additional solution except when you are limited by QSFP28 SFF power.

--
tarko

Very impressive! Can you share your fiber type and link-loss?
—L.B.

Lady Benjamin PD Cannon, ASCE
6x7 Networks & 6x7 Telecom, LLC
CEO
ben@6by7.net <mailto:ben@6by7.net>
"The only fully end-to-end encrypted global telecommunications company in the world.”
FCC License KJ6FJJ



> On Oct 30, 2020, at 9:03 AM, Tarko Tikan <tarko@lanparty.ee> wrote:
>
> hey,
>
>> If it’s just a single 100G channel needed you could try 100GBASE-ZR4. Specified for 80km, 30db power budget they could actually reach more the 80km.
>> Dispersion should also be „no" problem in the 1310nm length. I have to say that I never tried this on 100km distance without coherent solutions.
>
> Just to add to my original suggestion, we just did 100G-ZR4 over 30dB link with pre-FEC BER 3.174E-11.
>
> As OP is asking for a solution for 25dB I don't see any reason why ZR4 would not work and why you would need coherent, amplifiers or any other additional solution except when you are limited by QSFP28 SFF power.
>
> --
> tarko

The 100 km leg completes a ring.
 

Jared
 

Sent: Friday, October 30, 2020
From: "Ben Cannon" <ben@6by7.net>
To: "Jared Brown" <nanog-isp@mail.com>
Cc: nanog@nanog.org
Subject: Re: 100G over 100 km of dark fiber

You could break this into 10x 10g coherent lanes, but you’re going to end up back close to coherent 100g prices.
 
You’re at the threshold distance where you’re past all the short range tech and are seriously pushing it - whereas the 100g coherent tech is just taking off.  
 
How important is this link?
 

Ms. Benjamin PD Cannon, ASCE
6x7 Networks & 6x7 Telecom, LLC 
CEO 
ben@6by7.net[mailto:ben@6by7.net]
"The only fully end-to-end encrypted global telecommunications company in the world.”
FCC License KJ6FJJ

 On Oct 30, 2020, at 7:19 AM, Jared Brown <nanog-isp@mail.com> wrote:
 
?Hello NANOG!

I need to push 100G over 100 km of dark fiber. Since there are no 100G pluggable optics with this reach (~25 dB), I have been offered coherent transport systems to solve my problem. This is all good and well, except total system costs start from high five figures.

So, my question is, do I have any other options?

I can't help noticing that you can break out a 100G QSFP into four 25G QSFPs. 25G DWDM systems are relatively inexpensive (low five figures), but can you make 25G DWDM go 100 km?

I only need the one 100G, so I don't really need a highly scalable DWDM system. I can't put anything midspan, or if I could it would cost more than just going with a coherent system.


Jared

Hi,

On 30-10-2020 15:33, Dale W. Carder wrote:
>
> You may also find that 100G PAM4 could work. There are some vendors that
> sell the optic, and an outboard EDFA + DCF pizza box.

We are about to deploy these on a couple of dark fibers:
https://www.solid-optics.com/product/edfamux-multiplexer-amplifier-dispersion-compensation-dwdm-mux-edfa/

They have amplified and dispersion compensated 8x100G to be used with
PAM4 optics, and a pass-through port to connect existing 1G/10G MUXes to
(which can have their own amplification if necessary).

They can provide models with different sets of channels of you need that
(nice when cascading them with existing 1/10G MUXes). IIRC next year
they can also build in a power meter so you can do remote monitoring.

If you're interested I can let you know how much we like them in a few
months ;)

Cheers,
Sander

Hi Jared,

4x25Gbit/s, 'directly detected' MAY work but it won't be easy at all. As many already have suggested, coherent detection will give you much much less headaches.

If you want to go with directly detected for financial reasons than first make sure you know the type(s) of fibers and the dispersion you can expect. Fiber-operators tend to splice different types of fiber together. Don't just assume that it will be all G.652 compatible fiber.

If there is G.655 or even G.653 compatible fiber in your path then it is unlikely that you can operate in the O-band (1310nm).

In that case you need to go with the C-band (1550nm), which is easier to amplify than the O-band, because you can use commonly available EDFAs.

You probably need to compensate the chromatic dispersion in the C-band. For that it is again very important to know the fiber type(s) in your path.Because a '100km DCM', assumes 100km G.652 fiber. If your path consists of 50km G.652 and 50km G.655 then you need to compensate for 50km.

Last but not least, your hardware or optics probably need to be able to do FEC to get a bit of a decent BER.

-- Ariën



> On 30 Oct 2020, at 15:19, Jared Brown <nanog-isp@mail.com> wrote:
>
> Hello NANOG!
>
> I need to push 100G over 100 km of dark fiber. Since there are no 100G pluggable optics with this reach (~25 dB), I have been offered coherent transport systems to solve my problem. This is all good and well, except total system costs start from high five figures.
>
> So, my question is, do I have any other options?
>
> I can't help noticing that you can break out a 100G QSFP into four 25G QSFPs. 25G DWDM systems are relatively inexpensive (low five figures), but can you make 25G DWDM go 100 km?
>
> I only need the one 100G, so I don't really need a highly scalable DWDM system. I can't put anything midspan, or if I could it would cost more than just going with a coherent system.
>
>
> Jared

Hi Jared,

Easiest thing would be breaking into sub-100G channels (e.g. 4x 25G) and
using DWDM optics + Amp + Compensation + Pre-Amp to transport it across the
100km.

Same parts for all configs:
*) 1x Amplifier => 1 slot
*) 1x 100km dispersion compensation => separate 1U
*) 1x Pre-Amplifier => 1 slot

Sample config per site:

*) 1U chassis (4 slots)
*) 1x Mgmt card => 1 slot
*) [same parts for all configs]
*) 1x 25G converter (100G client port split into 4x 25G SFP28 line ports) =>
1 slot
*) 1x 8 channel DWDM Mux/Demux => separate 1U
*) 4x 25G SFP28 DWDM 10km
*) 1x 100G QSFP28 SR/LR (this is only for the client port on the converter -
you need a second one for your device)

=> footprint per site ~6200 USD for 1x 100G

The above setup requires 3U of rack space per site and provides 4 more DWDM
channels as spares for further upgrades and the DWDM unit will have an
separate upgrade port, to expand the number of filters (if you do not want
to replace the whole filter with a bigger one).

If you you want to go for a solution which starts with 4x 100G you're on the
13k USD range (per site):

*) 2U chassis (8 slots)
*) 1x Mgmt card => 1 slot
*) [same parts for all configs]
*) 4x 25G converter (100G client port split into 4x 25G SFP28 line ports) =>
4 slots
*) 1x 16 channel DWDM Mux/Demux => separate 1U
*) 16x 25G SFP28 DWDM 10km
*) 4x 100G QSFP28 SR/LR (this is only for the client port on the converter -
you need a second one for your device)

=> footprint per site ~13000 USD for 4x 100G


Even expansion to 12x 100G is more or less "cheap":

*) 5U chassis (16 slots)
*) 1x Mgmt card => 1 slot
*) [same parts for all configs]
*) 12x 25G converter (100G client port split into 4x 25G SFP28 line ports)
=> 12 slots
*) 1x 48 channel DWDM Mux/Demux => separate 1U
*) 48x 25G SFP28 DWDM 10km
*) 12x 100G QSFP28 SR/LR (this is only for the client port on the converter
- you need a second one for your device)

=> footprint per site ~31000 USD for 12x 100G

I typically go for Sintai, FS.com and Edgeoptics components. Regarding the
compensation and AMPs stuff: it's just math, nothing else :).


Best regards
J?rgen

Turns out I was wrong, again.

There is at least one vendor that offers pluggable 100G QSFP28 optics that can reach 120 km with amplification.

Thanks to everybody that reached out! I appreciate all the input and lessons learned.


Jared


-------------------------------------------------------------------------------------------------------

Hello NANOG!

I need to push 100G over 100 km of dark fiber. Since there are no 100G pluggable optics with this reach (~25 dB), I have been offered coherent transport systems to solve my problem. This is all good and well, except total system costs start from high five figures.

So, my question is, do I have any other options?

I can't help noticing that you can break out a 100G QSFP into four 25G QSFPs. 25G DWDM systems are relatively inexpensive (low five figures), but can you make 25G DWDM go 100 km?

I only need the one 100G, so I don't really need a highly scalable DWDM system. I can't put anything midspan, or if I could it would cost more than just going with a coherent system.


Jared