Mailing List Archive

Mellanox / Cumulus
Curious to hear if the community has had any real-world experience using
Mellanox/Cumulus (nVidia) for L2/L3 things outside of the datacenter.

Like other vendors, notably Arista, they seem to be trying to move out
of the datacenter and target SPs and the layer 3 market. Personally, I
think Arista has worked out most of the kinks over the last few years,
and we've been happy with their L3 solutions (e.g., the 7280s)

While Mellanox's chipset is intriguing, I get a sense of feature-itis
from their marketing. (BGP, OSPF, NAT, we do it all etc.) No IS-IS
support, I'm told ...

Anybody using these in production in an SP environment? And if so, any
opinions, good or bad?

Feel free to reach out off-list if you prefer. Thank you,

- bryan
Re: Mellanox / Cumulus [ In reply to ]
On 02/11/2020 17:52, Bryan Holloway wrote:
> Anybody using these in production in an SP environment? And if so, any
> opinions, good or bad?

I haven't used them in an SP environment precisely because the Mellanox
hardware - while miles better than equivalent Broadcom designs - does
not cater to anyone with more than the most basic of QoS requirements.

Realistically, the ASIC designs are brilliant for data
centre/storage/HPC use, but they do not have (last I was briefed) any
hardware that would replace even an access switch, let alone a capable
border router.

I would *love* for that to change, so please correct me if I'm outdated.

On the software side, Cumulus Linux is very capable, and a joy to work
with. However, the business case to support even the Broadcom DNX range
(e.g. Arista 7280R) just wasn't there /before/ their acquisition. Again,
if that's changed it would be a fine software suite to investigate.

Regards,

--
Tom
RE: Mellanox / Cumulus [ In reply to ]
One of my CDN caching providers sent a Mellanox SN2700 with their servers. Seems to be running well. They manage them, I just give them rack, power, and a couple 10 gig links into my core

-Aaron

-----Original Message-----
From: NANOG <nanog-bounces+aaron1=gvtc.com@nanog.org> On Behalf Of Tom Hill
Sent: Wednesday, November 4, 2020 9:37 AM
To: nanog@nanog.org
Subject: Re: Mellanox / Cumulus

On 02/11/2020 17:52, Bryan Holloway wrote:
> Anybody using these in production in an SP environment? And if so, any
> opinions, good or bad?

I haven't used them in an SP environment precisely because the Mellanox hardware - while miles better than equivalent Broadcom designs - does not cater to anyone with more than the most basic of QoS requirements.

Realistically, the ASIC designs are brilliant for data centre/storage/HPC use, but they do not have (last I was briefed) any hardware that would replace even an access switch, let alone a capable border router.

I would *love* for that to change, so please correct me if I'm outdated.

On the software side, Cumulus Linux is very capable, and a joy to work with. However, the business case to support even the Broadcom DNX range (e.g. Arista 7280R) just wasn't there /before/ their acquisition. Again, if that's changed it would be a fine software suite to investigate.

Regards,

--
Tom
Re: Mellanox / Cumulus [ In reply to ]
On 04/11/2020 16:02, aaron1@gvtc.com wrote:
> One of my CDN caching providers sent a Mellanox SN2700 with their
> servers. Seems to be running well. They manage them, I just give
> them rack, power, and a couple 10 gig links into my core


At this point, we may descend into a "what does SP mean" debate, but I
am not at all surprised that CDN providers would like the Spectrum
ASICs: they're excellent at pushing lots of bits, quickly.

Because they lack buffer-laden features (by design) a lot of SPs - say,
access ISPs - will stop and say, "What? This is a DC switch!"

--
Tom