Mailing List Archive

> james list
> Sent: Saturday, September 26, 2020 11:57 AM
>
> Dear experts
> I have a project to connect at layer2 level 16 servers (two interfaces
> each) with a total of 32 x 10Gbs server interfaces in order to setup a big
data
> solution.
>
> These interfaces must have in normal conditions full L2 bandwidth
available
> to transmit among themselves and redundancy of switch (if there is a fault
> condition 160 Gbs is enough).
>
> Since this kind of huge bandwidth requirement could cause bottlenecks in
> Datacenter Lan environment I was thinking to setup a separate lan
> architecture.
>
> I was thinking to setup a virtual chassis environment with 2 x qfx5100
with
> multiple 40 Gbs interfaces to set vc ports.
>
Couple of thoughts,

Routing vs switching
First of all try to push for a L3 solution if the app itself doesn't support
it look into routing on host (cRDP, etc...) -so that all you need to worry
about outside of hosts is routing (not switching).
With L3 you don't need to worry about split brain scenarios with virtual
chassis or multi-chassis lag,

Scalability and throughput,
Note that creating a dedicated 2 x qfx5100 pod will limit the solution to
the bandwidth of a single qfx5100, which may or may not be sufficient to
grow the solution in future.
Migration to a standard folded Clos fabric will then be complicated if
needed.
Also note that qfx5100 might have throughput limitations at smaller than
1500B packet sizes, so check with the app team on expected packet size and
compare that with the switch specs or best do your performance testing.

Buffers,
If you start to mix and match 40s and 10s (in failure condition) you need to
worry about buffers in the stepdown from 40 to 10.


adam


_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Hi
Can you elaborate the concept of L3 vs L2?

I was thinking to have L2 versus the server in A/A (vc or mclag) or A/P
(plain vlan among two switches) and a vrrp over the two switches.

Are you talking to have L3 (routed port) on each switch interface versus
server ?

Cheers
James

Il Lun 28 Set 2020, 09:42 <adamv0025@netconsultings.com> ha scritto:

> > james list
> > Sent: Saturday, September 26, 2020 11:57 AM
> >
> > Dear experts
> > I have a project to connect at layer2 level 16 servers (two interfaces
> > each) with a total of 32 x 10Gbs server interfaces in order to setup a
> big
> data
> > solution.
> >
> > These interfaces must have in normal conditions full L2 bandwidth
> available
> > to transmit among themselves and redundancy of switch (if there is a
> fault
> > condition 160 Gbs is enough).
> >
> > Since this kind of huge bandwidth requirement could cause bottlenecks in
> > Datacenter Lan environment I was thinking to setup a separate lan
> > architecture.
> >
> > I was thinking to setup a virtual chassis environment with 2 x qfx5100
> with
> > multiple 40 Gbs interfaces to set vc ports.
> >
> Couple of thoughts,
>
> Routing vs switching
> First of all try to push for a L3 solution if the app itself doesn't
> support
> it look into routing on host (cRDP, etc...) -so that all you need to worry
> about outside of hosts is routing (not switching).
> With L3 you don't need to worry about split brain scenarios with virtual
> chassis or multi-chassis lag,
>
> Scalability and throughput,
> Note that creating a dedicated 2 x qfx5100 pod will limit the solution to
> the bandwidth of a single qfx5100, which may or may not be sufficient to
> grow the solution in future.
> Migration to a standard folded Clos fabric will then be complicated if
> needed.
> Also note that qfx5100 might have throughput limitations at smaller than
> 1500B packet sizes, so check with the app team on expected packet size and
> compare that with the switch specs or best do your performance testing.
>
> Buffers,
> If you start to mix and match 40s and 10s (in failure condition) you need
> to
> worry about buffers in the stepdown from 40 to 10.
>
>
> adam
>
>
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Yes I was recommending to have the QFX switch ports as L3 ports with p2p L3 links to servers -where each server would be engaging in routing with the QFX-es (understanding it can load-share across its two links to get to the other 15 servers). (this could be accomplished by static routing on the server (+BFD for next hop reachability) or by running some routing protocol on the host.

Within routed environment it’s easy to achieve active-active and route around a failure, in L2 environment you need to rely on hacks like virtual chassis or multi-chassis-lag etc… which could fail in a in a split-brain scenario.



adam



From: james list <jameslist72@gmail.com>
Sent: Monday, September 28, 2020 7:57 PM
To: adamv0025@netconsultings.com
Cc: Juniper List <juniper-nsp@puck.nether.net>
Subject: Re: [j-nsp] Big flows up to 320 Gbs



Hi

Can you elaborate the concept of L3 vs L2?



I was thinking to have L2 versus the server in A/A (vc or mclag) or A/P (plain vlan among two switches) and a vrrp over the two switches.



Are you talking to have L3 (routed port) on each switch interface versus server ?



Cheers

James

Il Lun 28 Set 2020, 09:42 <adamv0025@netconsultings.com <mailto:adamv0025@netconsultings.com> > ha scritto:

> james list
> Sent: Saturday, September 26, 2020 11:57 AM
>
> Dear experts
> I have a project to connect at layer2 level 16 servers (two interfaces
> each) with a total of 32 x 10Gbs server interfaces in order to setup a big
data
> solution.
>
> These interfaces must have in normal conditions full L2 bandwidth
available
> to transmit among themselves and redundancy of switch (if there is a fault
> condition 160 Gbs is enough).
>
> Since this kind of huge bandwidth requirement could cause bottlenecks in
> Datacenter Lan environment I was thinking to setup a separate lan
> architecture.
>
> I was thinking to setup a virtual chassis environment with 2 x qfx5100
with
> multiple 40 Gbs interfaces to set vc ports.
>
Couple of thoughts,

Routing vs switching
First of all try to push for a L3 solution if the app itself doesn't support
it look into routing on host (cRDP, etc...) -so that all you need to worry
about outside of hosts is routing (not switching).
With L3 you don't need to worry about split brain scenarios with virtual
chassis or multi-chassis lag,

Scalability and throughput,
Note that creating a dedicated 2 x qfx5100 pod will limit the solution to
the bandwidth of a single qfx5100, which may or may not be sufficient to
grow the solution in future.
Migration to a standard folded Clos fabric will then be complicated if
needed.
Also note that qfx5100 might have throughput limitations at smaller than
1500B packet sizes, so check with the app team on expected packet size and
compare that with the switch specs or best do your performance testing.

Buffers,
If you start to mix and match 40s and 10s (in failure condition) you need to
worry about buffers in the stepdown from 40 to 10.


adam



_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Hardware wise I would go with at least QFX5110-48S, then you have
4x1000Gbps uplinks, but even better go with QFX5120-48Y for 8x100G uplinks.
Also, don't use VC or MC-LAG unless you really must. Instead use EVPN-VXLAN
to provide multihoming, L2 stretch, Anycasted default GW etc etc.

Regards
Roger

On Sat, Sep 26, 2020 at 1:00 PM james list <jameslist72@gmail.com> wrote:

> Dear experts
> I have a project to connect at layer2 level 16 servers (two interfaces
> each) with a total of 32 x 10Gbs server interfaces in order to setup a big
> data solution.
>
> These interfaces must have in normal conditions full L2 bandwidth available
> to transmit among themselves and redundancy of switch (if there is a fault
> condition 160 Gbs is enough).
>
> Since this kind of huge bandwidth requirement could cause bottlenecks in
> Datacenter Lan environment I was thinking to setup a separate lan
> architecture.
>
> I was thinking to setup a virtual chassis environment with 2 x qfx5100 with
> multiple 40 Gbs interfaces to set vc ports.
>
> Do you think it's a good architecture or what would you setup?
>
> Cheers
> James
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp