Mailing List Archive

Any sign of supply chain returning to normal?
I'm not sure if this is the right place for this discussion but I can't think of anywhere better to ask.

Has anyone seen any progress whatsoever on supply chain issues with networking hardware?

I've noticed that primary market lead times have been increasing and at the same time secondary market pricing has also been going higher at the same time, still.

What have you seen?
Re: Any sign of supply chain returning to normal? [ In reply to ]
Basically, anything that uses Broadcom or other commodity silicon is
currently 55+ weeks out according to most of the vendors I work with.
Custom Silicon is a bit better or so I'm told, but I've not had to order
much gear with custom silicon lately, so I've not got a clear read on lead
times there.

I wouldn't be surprised to see some recent gear go End of Sales early just
because of component shortages and fabs moving to produce the more
in-demand parts over older less profitable parts.



On Fri, Apr 22, 2022 at 8:25 AM Drew Weaver <drew.weaver@thenap.com> wrote:

> I’m not sure if this is the right place for this discussion but I can’t
> think of anywhere better to ask.
>
>
>
> Has anyone seen any progress whatsoever on supply chain issues with
> networking hardware?
>
>
>
> I’ve noticed that primary market lead times have been increasing and at
> the same time secondary market pricing has also been going higher at the
> same time, still.
>
>
>
> What have you seen?
>
>
>
>
>
>
>
Re: Any sign of supply chain returning to normal? [ In reply to ]
A company I work for designs a lot of our own hardware and we’ve had a number of critical components go EOL suddenly and without warning, such as FPGAs, ADCs, clock generators, and SOMs just to name a few. Just a few weeks ago we were informed that a large order of FPGAs was not going to be filled at all and the order was cancelled. Of the parts that aren’t EOL (yet), many have 52-week lead times which is just a place holder for “we have no idea when we’ll get these” and not an actual delivery estimate. Older product lines and lower volume product lines are being cancelled. We had an ADC go EOL because the only factory in Japan making this part burned down so not necessarily related to what we think of as supply chain issues, but it is of a different sort.

> On Apr 22, 2022, at 8:50 AM, Joe Freeman <joe@netbyjoe.com> wrote:
>
> ?
> Basically, anything that uses Broadcom or other commodity silicon is currently 55+ weeks out according to most of the vendors I work with. Custom Silicon is a bit better or so I'm told, but I've not had to order much gear with custom silicon lately, so I've not got a clear read on lead times there.
>
> I wouldn't be surprised to see some recent gear go End of Sales early just because of component shortages and fabs moving to produce the more in-demand parts over older less profitable parts.
RE: Any sign of supply chain returning to normal? [ In reply to ]
I bought (3) MX204's 10/2021 and received them 2/2022 so about 5 months to
receive those. Also received a couple SRX300's in that same purchase.



I'll add that I can't say the same for the other stuff I also ordered
10/2021.



- MX480



- MX240



- MPC10E-10C





.which is due in around 5/2022. So about 8 months for that stuff, but,
actually remains to be seen because we still haven't got it yet.





-Aaron
Re: Any sign of supply chain returning to normal? [ In reply to ]
------- Original Message -------
On Friday, April 22nd, 2022 at 13:24, Drew Weaver <drew.weaver@thenap.com> wrote:

> Has anyone seen any progress whatsoever on supply chain issues with networking hardware?

Nope.

Personally speaking I'm struggling on everything from simple Intel network cards to half-decent switches.

On the grapevine, I am hearing the same. I was, for example, talking to a rep from $major_IX ... he said they've got more than one customer with pending port connections due to lack of kit, and one customer told him some router card was not due til Q1 2023.

I think the reality is it's going to get bad before it gets better. There's probably an almighty backlog that needs to be processed before new orders reach the front of the queue.

On another grapevine I've heard the situation is, shall we say, not helped by the hyperscalers. The hyperscalers are (allegedly) queue jumping using hard $$$ ... i.e. saying to vendors "I'll give you this chunky order in return for a place higher up the queue". Vendors of course being vain beasts who don't care for much apart from next quarter's financial results are (allegedly) lapping up the "free" money.
RE: Any sign of supply chain returning to normal? [ In reply to ]
Anecdotally, I had a pair of Nexus 93180s that I ordered in May 2021 show up in February 2022, so 9 months. The estimated ship date got punted several times (probably due to being preempted by folks employing the approach Laura outlined ;-) ).

I haven't ordered anything since then, but I understand that 4-8 months isn't unexpected, still.

- Jima

From: NANOG <nanog-bounces@nanog.org> On Behalf Of Drew Weaver
Sent: Friday, April 22, 2022 07:24
To: 'nanog@nanog.org' <nanog@nanog.org>
Subject: Any sign of supply chain returning to normal?

I'm not sure if this is the right place for this discussion but I can't think of anywhere better to ask.

Has anyone seen any progress whatsoever on supply chain issues with networking hardware?

I've noticed that primary market lead times have been increasing and at the same time secondary market pricing has also been going higher at the same time, still.

What have you seen?
Re: Any sign of supply chain returning to normal? [ In reply to ]
There's some queue-jumping happening for other reasons -
medical/hospital a significant portion of that - but even there I'm
hearing 6+ months for some switch hardware and Cisco APs are pretty
uniformly "if you didn't order before March, you won't see them for
over a year".

On Fri, Apr 22, 2022 at 6:29 PM nanog@jima.us <nanog@jima.us> wrote:
>
> Anecdotally, I had a pair of Nexus 93180s that I ordered in May 2021 show up in February 2022, so 9 months. The estimated ship date got punted several times (probably due to being preempted by folks employing the approach Laura outlined ;-) ).
>
> I haven't ordered anything since then, but I understand that 4-8 months isn't unexpected, still.
>
> - Jima
>
> From: NANOG <nanog-bounces@nanog.org> On Behalf Of Drew Weaver
> Sent: Friday, April 22, 2022 07:24
> To: 'nanog@nanog.org' <nanog@nanog.org>
> Subject: Any sign of supply chain returning to normal?
>
> I'm not sure if this is the right place for this discussion but I can't think of anywhere better to ask.
>
> Has anyone seen any progress whatsoever on supply chain issues with networking hardware?
>
> I've noticed that primary market lead times have been increasing and at the same time secondary market pricing has also been going higher at the same time, still.
>
> What have you seen?
>
>
>
Re: Any sign of supply chain returning to normal? [ In reply to ]
Go virtual. x86 servers are still 5-8 weeks from our usual suppliers,
although some NICs are 12 weeks and DC Power Supplies are also
52-weeks/'no-idea'.

-- Tom


On Fri, Apr 22, 2022 at 6:21 AM Ryan Wilkins <ryan@deadfrog.net> wrote:

> A company I work for designs a lot of our own hardware and we’ve had a
> number of critical components go EOL suddenly and without warning, such as
> FPGAs, ADCs, clock generators, and SOMs just to name a few. Just a few
> weeks ago we were informed that a large order of FPGAs was not going to be
> filled at all and the order was cancelled. Of the parts that aren’t EOL
> (yet), many have 52-week lead times which is just a place holder for “we
> have no idea when we’ll get these” and not an actual delivery estimate.
> Older product lines and lower volume product lines are being cancelled. We
> had an ADC go EOL because the only factory in Japan making this part burned
> down so not necessarily related to what we think of as supply chain issues,
> but it is of a different sort.
>
> > On Apr 22, 2022, at 8:50 AM, Joe Freeman <joe@netbyjoe.com> wrote:
> >
> > ?
> > Basically, anything that uses Broadcom or other commodity silicon is
> currently 55+ weeks out according to most of the vendors I work with.
> Custom Silicon is a bit better or so I'm told, but I've not had to order
> much gear with custom silicon lately, so I've not got a clear read on lead
> times there.
> >
> > I wouldn't be surprised to see some recent gear go End of Sales early
> just because of component shortages and fabs moving to produce the more
> in-demand parts over older less profitable parts.
>
Re: Any sign of supply chain returning to normal? [ In reply to ]
so i am chatting with the volvo mechanic this morning. he said that 25
years ago volvo had essentially two engines, carb and fuel injection.
from the late '90s on, the variations grew; and the parts and tools one
needed exploded.

he started feeling supply chain issues early. and now the number of
variations being designed is narrowing again.

i suspect that, in years of overabundant late stage capitalism, folk
went nuts. and we are now paying for it. one of my fave quotes

I thought of it in a slightly different way--like a space that we
were exploring and, in the early days, we figured out this
consistent path through the space: IP, TCP, and so on. What's been
happening over the last few years is that the IETF is filling the
rest of the space with every alternative approach, not necessarily
any better. Every possible alternative is now being written down.
And it's not useful. -- Jon Postel

randy
Re: Any sign of supply chain returning to normal? [ In reply to ]
Who are you buying servers from, because I'm going on a year waiting on
servers from HPE, and about 6 months on servers from Dell, although
that may have to do with the types of NICs I need.
I'm told HPE is holding back capacity for some of their large "Government"
contracts which have stiff performance penalties.

For the last year and a half, I have been working on fitting out a $20
Million dollar telco network lab (x86 and network gear), and while I work
for a VERY LARGE company, we can't even get escalations with the vendors.
In fact about 6 months ago, we were told that Juniper, Cisco and HPE all
stopped accepting VP level escalations, which can normally get you ahead of
the line.

Juniper has definitely been the worst in terms of delivery, I have
equipment from them which was ordered last March and has delivery dates as
far out as September.
I tried to order a pair of AC power supplies for a Nexus 9K last week and
was told delivery was quoted as 55 weeks, so maybe Cisco is no better.
About a month ago, I had a quote from Extreme Networks for a pair of 1G
switches, and before I could submit the order, they came back and raised
the price almost 20%, so it seems some vendors may be trying to reduce
demand by increasing prices.
And don't even try to order an ACC100 Accelerator card, many vendors have
simply stopped accepting orders.

It is definitely going to get worse before it gets better.

Shane



On Fri, Apr 22, 2022 at 6:58 PM Tom Mitchell <tmitchell@netelastic.com>
wrote:

> Go virtual. x86 servers are still 5-8 weeks from our usual suppliers,
> although some NICs are 12 weeks and DC Power Supplies are also
> 52-weeks/'no-idea'.
>
> -- Tom
>
>
> On Fri, Apr 22, 2022 at 6:21 AM Ryan Wilkins <ryan@deadfrog.net> wrote:
>
>> A company I work for designs a lot of our own hardware and we’ve had a
>> number of critical components go EOL suddenly and without warning, such as
>> FPGAs, ADCs, clock generators, and SOMs just to name a few. Just a few
>> weeks ago we were informed that a large order of FPGAs was not going to be
>> filled at all and the order was cancelled. Of the parts that aren’t EOL
>> (yet), many have 52-week lead times which is just a place holder for “we
>> have no idea when we’ll get these” and not an actual delivery estimate.
>> Older product lines and lower volume product lines are being cancelled. We
>> had an ADC go EOL because the only factory in Japan making this part burned
>> down so not necessarily related to what we think of as supply chain issues,
>> but it is of a different sort.
>>
>> > On Apr 22, 2022, at 8:50 AM, Joe Freeman <joe@netbyjoe.com> wrote:
>> >
>> > ?
>> > Basically, anything that uses Broadcom or other commodity silicon is
>> currently 55+ weeks out according to most of the vendors I work with.
>> Custom Silicon is a bit better or so I'm told, but I've not had to order
>> much gear with custom silicon lately, so I've not got a clear read on lead
>> times there.
>> >
>> > I wouldn't be surprised to see some recent gear go End of Sales early
>> just because of component shortages and fabs moving to produce the more
>> in-demand parts over older less profitable parts.
>>
>
Re: Any sign of supply chain returning to normal? [ In reply to ]
On 23/04/2022 01:28, nanog@jima.us wrote:

Ordered a pair of ASR9906s in Jan 2022 with delivery Aug 2022.

-Hank

> Anecdotally, I had a pair of Nexus 93180s that I ordered in May 2021 show up in February 2022, so 9 months. The estimated ship date got punted several times (probably due to being preempted by folks employing the approach Laura outlined ;-) ).
>
> I haven't ordered anything since then, but I understand that 4-8 months isn't unexpected, still.
>
> - Jima
>
> From: NANOG <nanog-bounces@nanog.org> On Behalf Of Drew Weaver
> Sent: Friday, April 22, 2022 07:24
> To: 'nanog@nanog.org' <nanog@nanog.org>
> Subject: Any sign of supply chain returning to normal?
>
> I'm not sure if this is the right place for this discussion but I can't think of anywhere better to ask.
>
> Has anyone seen any progress whatsoever on supply chain issues with networking hardware?
>
> I've noticed that primary market lead times have been increasing and at the same time secondary market pricing has also been going higher at the same time, still.
>
> What have you seen?
>
>
>
Re: Any sign of supply chain returning to normal? [ In reply to ]
Randy Bush wrote:

> i suspect that, in years of overabundant late stage capitalism, folk
> went nuts. and we are now paying for it. one of my fave quotes
>
> I thought of it in a slightly different way--like a space that we
> were exploring and, in the early days, we figured out this
> consistent path through the space: IP, TCP, and so on. What's been
> happening over the last few years is that the IETF is filling the
> rest of the space with every alternative approach, not necessarily
> any better. Every possible alternative is now being written down.
> And it's not useful. -- Jon Postel

And Steve Deering agreed with Jon saying "Exactly".

That's so funny because the statement was published in Oct. 1998
and the first rfc on IPv6 was published in Dec. 1995.

Masataka Ohta
Re: Any sign of supply chain returning to normal? [ In reply to ]
I happen to have a few MX204’s (some MX204 “old version” and some MX204-HW-BASE left. In case anyone is interested please contact me directly.

Cheers,
Greg

From: NANOG <nanog-bounces+gregory=emxcore.com@nanog.org> on behalf of "aaron1@gvtc.com" <aaron1@gvtc.com>
Date: Friday, 22 April 2022 at 20:51
To: 'Drew Weaver' <drew.weaver@thenap.com>, "nanog@nanog.org" <nanog@nanog.org>
Subject: RE: Any sign of supply chain returning to normal?


I bought (3) MX204’s 10/2021 and received them 2/2022 so about 5 months to receive those. Also received a couple SRX300’s in that same purchase.



I’ll add that I can’t say the same for the other stuff I also ordered 10/2021…



- MX480



- MX240



- MPC10E-10C





…which is due in around 5/2022. So about 8 months for that stuff, but, actually remains to be seen because we still haven’t got it yet.





-Aaron
Re: Any sign of supply chain returning to normal? [ In reply to ]
As I've been saying for a while, instead of buying new kit, perhaps we
could spend some time on getting better software onto our older kit?
Getting stuff to multiplex better, be more reliable, last longer?

It isn't just me wanting to upgrade a billion+ routers with existing
crappy software to openwrt, is it?

https://docs.google.com/document/d/1T21on7g1MqQZoK91epUdxLYFGdtyLRgBat0VXoC9e3I/edit
Re: Any sign of supply chain returning to normal? [ In reply to ]
I'd bet it's cheaper and easier to quantify new hardware than software.
Labor was super expensive and now it is ready to implode.

On Thu, May 19, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:

> As I've been saying for a while, instead of buying new kit, perhaps we
> could spend some time on getting better software onto our older kit?
> Getting stuff to multiplex better, be more reliable, last longer?
>
> It isn't just me wanting to upgrade a billion+ routers with existing
> crappy software to openwrt, is it?
>
>
> https://docs.google.com/document/d/1T21on7g1MqQZoK91epUdxLYFGdtyLRgBat0VXoC9e3I/edit
>
Re: Any sign of supply chain returning to normal? [ In reply to ]
As someone who works within the "secondary market" for networking hardware,
there is a ton of demand spilling over into the "pre-owned/vendor
refurbished" market.

Market prices on pre-owned equipment are rapidly increasing in step with
increased demand and dwindling supply.

Market prices on 1G - 10G switching products, wireless infrastructure
devices, etc have been rising precipitously. Even semi "legacy" stuff going
back 2-3 generations (EOL/EOS) from current gen have doubled, tripled, even
quadrupled in price.

I've been involved in the hardware business for 20 years and the current
market landscape is unprecedented.



Cory J. Andrews
++++++++++++++
NetEquity.com (Formerly CiscoBuy.com)
4519 Northgate Court
Sarasota, FL 34234
++++++++++++++
TF/FAX 877.582.4726
E - sales@netequity.com

On Thu, May 19, 2022, 9:48 AM Josh Luthman <josh@imaginenetworksllc.com>
wrote:

> I'd bet it's cheaper and easier to quantify new hardware than software.
> Labor was super expensive and now it is ready to implode.
>
> On Thu, May 19, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
>
>> As I've been saying for a while, instead of buying new kit, perhaps we
>> could spend some time on getting better software onto our older kit?
>> Getting stuff to multiplex better, be more reliable, last longer?
>>
>> It isn't just me wanting to upgrade a billion+ routers with existing
>> crappy software to openwrt, is it?
>>
>>
>> https://docs.google.com/document/d/1T21on7g1MqQZoK91epUdxLYFGdtyLRgBat0VXoC9e3I/edit
>>
>
Re: Any sign of supply chain returning to normal? [ In reply to ]
On Thu, May 19, 2022 at 9:07 AM NetEquity Sales <sales@netequity.com> wrote:
>
> As someone who works within the "secondary market" for networking hardware, there is a ton of demand spilling over into the "pre-owned/vendor refurbished" market.

I just wish there were people putting in a value-add, like reflashing
with better software, first.

>
> Market prices on pre-owned equipment are rapidly increasing in step with increased demand and dwindling supply.
>
> Market prices on 1G - 10G switching products, wireless infrastructure devices, etc have been rising precipitously. Even semi "legacy" stuff going back 2-3 generations (EOL/EOS) from current gen have doubled, tripled, even quadrupled in price.
>
> I've been involved in the hardware business for 20 years and the current market landscape is unprecedented.
>
>
>
> Cory J. Andrews
> ++++++++++++++
> NetEquity.com (Formerly CiscoBuy.com)
> 4519 Northgate Court
> Sarasota, FL 34234
> ++++++++++++++
> TF/FAX 877.582.4726
> E - sales@netequity.com
>
> On Thu, May 19, 2022, 9:48 AM Josh Luthman <josh@imaginenetworksllc.com> wrote:
>>
>> I'd bet it's cheaper and easier to quantify new hardware than software. Labor was super expensive and now it is ready to implode.
>>
>> On Thu, May 19, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
>>>
>>> As I've been saying for a while, instead of buying new kit, perhaps we
>>> could spend some time on getting better software onto our older kit?
>>> Getting stuff to multiplex better, be more reliable, last longer?
>>>
>>> It isn't just me wanting to upgrade a billion+ routers with existing
>>> crappy software to openwrt, is it?
>>>
>>> https://docs.google.com/document/d/1T21on7g1MqQZoK91epUdxLYFGdtyLRgBat0VXoC9e3I/edit



--
FQ World Domination pending: https://blog.cerowrt.org/post/state_of_fq_codel/
Dave Täht CEO, TekLibre, LLC
Re: Any sign of supply chain returning to normal? [ In reply to ]
Who's going to support that reflashed device? Certainly not the OEM vendor.

On Thu, May 19, 2022 at 10:30 AM Dave Taht <dave.taht@gmail.com> wrote:

> On Thu, May 19, 2022 at 9:07 AM NetEquity Sales <sales@netequity.com>
> wrote:
> >
> > As someone who works within the "secondary market" for networking
> hardware, there is a ton of demand spilling over into the "pre-owned/vendor
> refurbished" market.
>
> I just wish there were people putting in a value-add, like reflashing
> with better software, first.
>
> >
> > Market prices on pre-owned equipment are rapidly increasing in step with
> increased demand and dwindling supply.
> >
> > Market prices on 1G - 10G switching products, wireless infrastructure
> devices, etc have been rising precipitously. Even semi "legacy" stuff going
> back 2-3 generations (EOL/EOS) from current gen have doubled, tripled, even
> quadrupled in price.
> >
> > I've been involved in the hardware business for 20 years and the current
> market landscape is unprecedented.
> >
> >
> >
> > Cory J. Andrews
> > ++++++++++++++
> > NetEquity.com (Formerly CiscoBuy.com)
> > 4519 Northgate Court
> > Sarasota, FL 34234
> > ++++++++++++++
> > TF/FAX 877.582.4726
> > E - sales@netequity.com
> >
> > On Thu, May 19, 2022, 9:48 AM Josh Luthman <josh@imaginenetworksllc.com>
> wrote:
> >>
> >> I'd bet it's cheaper and easier to quantify new hardware than
> software. Labor was super expensive and now it is ready to implode.
> >>
> >> On Thu, May 19, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
> >>>
> >>> As I've been saying for a while, instead of buying new kit, perhaps we
> >>> could spend some time on getting better software onto our older kit?
> >>> Getting stuff to multiplex better, be more reliable, last longer?
> >>>
> >>> It isn't just me wanting to upgrade a billion+ routers with existing
> >>> crappy software to openwrt, is it?
> >>>
> >>>
> https://docs.google.com/document/d/1T21on7g1MqQZoK91epUdxLYFGdtyLRgBat0VXoC9e3I/edit
>
>
>
> --
> FQ World Domination pending:
> https://blog.cerowrt.org/post/state_of_fq_codel/
> Dave Täht CEO, TekLibre, LLC
>


--
Jason
Re: Any sign of supply chain returning to normal? [ In reply to ]
On Thu, May 19, 2022 at 10:33 AM Jason Biel <jason@biel-tech.com> wrote:
>
> Who's going to support that reflashed device? Certainly not the OEM vendor.

The oem ain't gonna support the resold device either.

Yes, arguably, someone or someones doing a value add would have to be
making money at it somehow.

However, at least in my world, volunteers make the world round, still.
It would kind of suck,
I suppose, if someone unleashed a few hundred thousand reflashed
routers like the TIP openwifi
effort ( https://telecominfraproject.com/ ) seem to intent on doing...

... but if the OS is good enough to not need support, the impact is minimal.
>
> On Thu, May 19, 2022 at 10:30 AM Dave Taht <dave.taht@gmail.com> wrote:
>>
>> On Thu, May 19, 2022 at 9:07 AM NetEquity Sales <sales@netequity.com> wrote:
>> >
>> > As someone who works within the "secondary market" for networking hardware, there is a ton of demand spilling over into the "pre-owned/vendor refurbished" market.
>>
>> I just wish there were people putting in a value-add, like reflashing
>> with better software, first.
>>
>> >
>> > Market prices on pre-owned equipment are rapidly increasing in step with increased demand and dwindling supply.
>> >
>> > Market prices on 1G - 10G switching products, wireless infrastructure devices, etc have been rising precipitously. Even semi "legacy" stuff going back 2-3 generations (EOL/EOS) from current gen have doubled, tripled, even quadrupled in price.
>> >
>> > I've been involved in the hardware business for 20 years and the current market landscape is unprecedented.
>> >
>> >
>> >
>> > Cory J. Andrews
>> > ++++++++++++++
>> > NetEquity.com (Formerly CiscoBuy.com)
>> > 4519 Northgate Court
>> > Sarasota, FL 34234
>> > ++++++++++++++
>> > TF/FAX 877.582.4726
>> > E - sales@netequity.com
>> >
>> > On Thu, May 19, 2022, 9:48 AM Josh Luthman <josh@imaginenetworksllc.com> wrote:
>> >>
>> >> I'd bet it's cheaper and easier to quantify new hardware than software. Labor was super expensive and now it is ready to implode.
>> >>
>> >> On Thu, May 19, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
>> >>>
>> >>> As I've been saying for a while, instead of buying new kit, perhaps we
>> >>> could spend some time on getting better software onto our older kit?
>> >>> Getting stuff to multiplex better, be more reliable, last longer?
>> >>>
>> >>> It isn't just me wanting to upgrade a billion+ routers with existing
>> >>> crappy software to openwrt, is it?
>> >>>
>> >>> https://docs.google.com/document/d/1T21on7g1MqQZoK91epUdxLYFGdtyLRgBat0VXoC9e3I/edit
>>
>>
>>
>> --
>> FQ World Domination pending: https://blog.cerowrt.org/post/state_of_fq_codel/
>> Dave Täht CEO, TekLibre, LLC
>
>
>
> --
> Jason



--
FQ World Domination pending: https://blog.cerowrt.org/post/state_of_fq_codel/
Dave Täht CEO, TekLibre, LLC
Re: Any sign of supply chain returning to normal? [ In reply to ]
> The oem ain't gonna support the resold device either.

Many vendors support resold gear through a recertification cost in order to
bring it back under a support contract.

On Thu, May 19, 2022 at 10:58 AM Dave Taht <dave.taht@gmail.com> wrote:

> On Thu, May 19, 2022 at 10:33 AM Jason Biel <jason@biel-tech.com> wrote:
> >
> > Who's going to support that reflashed device? Certainly not the OEM
> vendor.
>
> The oem ain't gonna support the resold device either.
>
> Yes, arguably, someone or someones doing a value add would have to be
> making money at it somehow.
>
> However, at least in my world, volunteers make the world round, still.
> It would kind of suck,
> I suppose, if someone unleashed a few hundred thousand reflashed
> routers like the TIP openwifi
> effort ( https://telecominfraproject.com/ ) seem to intent on doing...
>
> ... but if the OS is good enough to not need support, the impact is
> minimal.
> >
> > On Thu, May 19, 2022 at 10:30 AM Dave Taht <dave.taht@gmail.com> wrote:
> >>
> >> On Thu, May 19, 2022 at 9:07 AM NetEquity Sales <sales@netequity.com>
> wrote:
> >> >
> >> > As someone who works within the "secondary market" for networking
> hardware, there is a ton of demand spilling over into the "pre-owned/vendor
> refurbished" market.
> >>
> >> I just wish there were people putting in a value-add, like reflashing
> >> with better software, first.
> >>
> >> >
> >> > Market prices on pre-owned equipment are rapidly increasing in step
> with increased demand and dwindling supply.
> >> >
> >> > Market prices on 1G - 10G switching products, wireless infrastructure
> devices, etc have been rising precipitously. Even semi "legacy" stuff going
> back 2-3 generations (EOL/EOS) from current gen have doubled, tripled, even
> quadrupled in price.
> >> >
> >> > I've been involved in the hardware business for 20 years and the
> current market landscape is unprecedented.
> >> >
> >> >
> >> >
> >> > Cory J. Andrews
> >> > ++++++++++++++
> >> > NetEquity.com (Formerly CiscoBuy.com)
> >> > 4519 Northgate Court
> >> > Sarasota, FL 34234
> >> > ++++++++++++++
> >> > TF/FAX 877.582.4726
> >> > E - sales@netequity.com
> >> >
> >> > On Thu, May 19, 2022, 9:48 AM Josh Luthman <
> josh@imaginenetworksllc.com> wrote:
> >> >>
> >> >> I'd bet it's cheaper and easier to quantify new hardware than
> software. Labor was super expensive and now it is ready to implode.
> >> >>
> >> >> On Thu, May 19, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com>
> wrote:
> >> >>>
> >> >>> As I've been saying for a while, instead of buying new kit, perhaps
> we
> >> >>> could spend some time on getting better software onto our older kit?
> >> >>> Getting stuff to multiplex better, be more reliable, last longer?
> >> >>>
> >> >>> It isn't just me wanting to upgrade a billion+ routers with existing
> >> >>> crappy software to openwrt, is it?
> >> >>>
> >> >>>
> https://docs.google.com/document/d/1T21on7g1MqQZoK91epUdxLYFGdtyLRgBat0VXoC9e3I/edit
> >>
> >>
> >>
> >> --
> >> FQ World Domination pending:
> https://blog.cerowrt.org/post/state_of_fq_codel/
> >> Dave Täht CEO, TekLibre, LLC
> >
> >
> >
> > --
> > Jason
>
>
>
> --
> FQ World Domination pending:
> https://blog.cerowrt.org/post/state_of_fq_codel/
> Dave Täht CEO, TekLibre, LLC
>


--
Jason
Re: Any sign of supply chain returning to normal? [ In reply to ]
On Thu, May 19, 2022 at 11:01 AM Jason Biel <jason@biel-tech.com> wrote:
>
> > The oem ain't gonna support the resold device either.
>
> Many vendors support resold gear through a recertification cost in order to bring it back under a support contract.

In my world, support ends after 6 months. Period.

It's even worse than that. Mediatek, for example, provides a devkit to
new customers, still, based on the obsolete
LEDE-17 release of openwrt, e.g. 6+ year code. I recently pointed out
to a marketing manager pimping how wifi-7 was going to fix latency on
wifi in 3-4 years, how crappy the factory driver was, compared to
what's now in linux

https://blog.cerowrt.org/post/fq-codel-unifi6/

and asked when they were going to ship that instead, to a blank
stare... And yes, I know of several "new" customers for that chipset
that are using that obsolete code, too scared and incompentent to make
the jump to a more current OS.

If you think that's bad, qualcomm is worse, and I just established a
new record, I think, with truly ancient broadcom's openwrt based
devkit that just shipped with the "NEW" triband tp-link deco series...
https://www.tp-link.com/us/deco-mesh-wifi/product-family/deco-xe75/ -
I can hardly bring myself to talk to the sea of CVEs and incompetence
in there... you can start with them STILL shipping dnsmasq 2.62... and
linux 3.3.8.

I used to be really proud that openwrt was used by all these major
manufacturers, but I'd also thought that they'd
have been responsible enough to at least keep up with CVEs, and stay
within a few years of the mainline.

If you are wondering why WiFi-6 works so badly out of the box, or why
ipv6 is not rolling out, you don't have to look much further. The
really, really sad thing, is that the ODM in these cases, just slaps
the devkit and a fancy gui
on top of it, and ships the product, with no further support.

So I kind of view recycling routers, with newer software, as a great
way to clean up the present ecosystem. And if you looked at the first
url I pasted above, with 4x more throughput, and 10x less latency, on
"obsolete", hw.

> On Thu, May 19, 2022 at 10:58 AM Dave Taht <dave.taht@gmail.com> wrote:
>>
>> On Thu, May 19, 2022 at 10:33 AM Jason Biel <jason@biel-tech.com> wrote:
>> >
>> > Who's going to support that reflashed device? Certainly not the OEM vendor.
>>
>> The oem ain't gonna support the resold device either.
>>
>> Yes, arguably, someone or someones doing a value add would have to be
>> making money at it somehow.
>>
>> However, at least in my world, volunteers make the world round, still.
>> It would kind of suck,
>> I suppose, if someone unleashed a few hundred thousand reflashed
>> routers like the TIP openwifi
>> effort ( https://telecominfraproject.com/ ) seem to intent on doing...
>>
>> ... but if the OS is good enough to not need support, the impact is minimal.
>> >
>> > On Thu, May 19, 2022 at 10:30 AM Dave Taht <dave.taht@gmail.com> wrote:
>> >>
>> >> On Thu, May 19, 2022 at 9:07 AM NetEquity Sales <sales@netequity.com> wrote:
>> >> >
>> >> > As someone who works within the "secondary market" for networking hardware, there is a ton of demand spilling over into the "pre-owned/vendor refurbished" market.
>> >>
>> >> I just wish there were people putting in a value-add, like reflashing
>> >> with better software, first.
>> >>
>> >> >
>> >> > Market prices on pre-owned equipment are rapidly increasing in step with increased demand and dwindling supply.
>> >> >
>> >> > Market prices on 1G - 10G switching products, wireless infrastructure devices, etc have been rising precipitously. Even semi "legacy" stuff going back 2-3 generations (EOL/EOS) from current gen have doubled, tripled, even quadrupled in price.
>> >> >
>> >> > I've been involved in the hardware business for 20 years and the current market landscape is unprecedented.
>> >> >
>> >> >
>> >> >
>> >> > Cory J. Andrews
>> >> > ++++++++++++++
>> >> > NetEquity.com (Formerly CiscoBuy.com)
>> >> > 4519 Northgate Court
>> >> > Sarasota, FL 34234
>> >> > ++++++++++++++
>> >> > TF/FAX 877.582.4726
>> >> > E - sales@netequity.com
>> >> >
>> >> > On Thu, May 19, 2022, 9:48 AM Josh Luthman <josh@imaginenetworksllc.com> wrote:
>> >> >>
>> >> >> I'd bet it's cheaper and easier to quantify new hardware than software. Labor was super expensive and now it is ready to implode.
>> >> >>
>> >> >> On Thu, May 19, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
>> >> >>>
>> >> >>> As I've been saying for a while, instead of buying new kit, perhaps we
>> >> >>> could spend some time on getting better software onto our older kit?
>> >> >>> Getting stuff to multiplex better, be more reliable, last longer?
>> >> >>>
>> >> >>> It isn't just me wanting to upgrade a billion+ routers with existing
>> >> >>> crappy software to openwrt, is it?
>> >> >>>
>> >> >>> https://docs.google.com/document/d/1T21on7g1MqQZoK91epUdxLYFGdtyLRgBat0VXoC9e3I/edit
>> >>
>> >>
>> >>
>> >> --
>> >> FQ World Domination pending: https://blog.cerowrt.org/post/state_of_fq_codel/
>> >> Dave Täht CEO, TekLibre, LLC
>> >
>> >
>> >
>> > --
>> > Jason
>>
>>
>>
>> --
>> FQ World Domination pending: https://blog.cerowrt.org/post/state_of_fq_codel/
>> Dave Täht CEO, TekLibre, LLC
>
>
>
> --
> Jason



--
FQ World Domination pending: https://blog.cerowrt.org/post/state_of_fq_codel/
Dave Täht CEO, TekLibre, LLC
RE: Any sign of supply chain returning to normal? [ In reply to ]
We have customers being forced to use EOL products that they previously replaces as they continue to wait on the vendor for new EQ.
Paul

-----Original Message-----
From: NANOG <nanog-bounces+razor=meganet.net@nanog.org> On Behalf Of Dave Taht
Sent: Thursday, May 19, 2022 12:16 PM
To: Jason Biel <jason@biel-tech.com>
Cc: NANOG <nanog@nanog.org>
Subject: Re: Any sign of supply chain returning to normal?

On Thu, May 19, 2022 at 11:01 AM Jason Biel <jason@biel-tech.com> wrote:
>
> > The oem ain't gonna support the resold device either.
>
> Many vendors support resold gear through a recertification cost in order to bring it back under a support contract.

In my world, support ends after 6 months. Period.

It's even worse than that. Mediatek, for example, provides a devkit to new customers, still, based on the obsolete
LEDE-17 release of openwrt, e.g. 6+ year code. I recently pointed out to a marketing manager pimping how wifi-7 was going to fix latency on wifi in 3-4 years, how crappy the factory driver was, compared to what's now in linux

https://blog.cerowrt.org/post/fq-codel-unifi6/

and asked when they were going to ship that instead, to a blank stare... And yes, I know of several "new" customers for that chipset that are using that obsolete code, too scared and incompentent to make the jump to a more current OS.

If you think that's bad, qualcomm is worse, and I just established a new record, I think, with truly ancient broadcom's openwrt based devkit that just shipped with the "NEW" triband tp-link deco series...
https://www.tp-link.com/us/deco-mesh-wifi/product-family/deco-xe75/ - I can hardly bring myself to talk to the sea of CVEs and incompetence in there... you can start with them STILL shipping dnsmasq 2.62... and linux 3.3.8.

I used to be really proud that openwrt was used by all these major manufacturers, but I'd also thought that they'd have been responsible enough to at least keep up with CVEs, and stay within a few years of the mainline.

If you are wondering why WiFi-6 works so badly out of the box, or why
ipv6 is not rolling out, you don't have to look much further. The really, really sad thing, is that the ODM in these cases, just slaps the devkit and a fancy gui on top of it, and ships the product, with no further support.

So I kind of view recycling routers, with newer software, as a great way to clean up the present ecosystem. And if you looked at the first url I pasted above, with 4x more throughput, and 10x less latency, on "obsolete", hw.

> On Thu, May 19, 2022 at 10:58 AM Dave Taht <dave.taht@gmail.com> wrote:
>>
>> On Thu, May 19, 2022 at 10:33 AM Jason Biel <jason@biel-tech.com> wrote:
>> >
>> > Who's going to support that reflashed device? Certainly not the OEM vendor.
>>
>> The oem ain't gonna support the resold device either.
>>
>> Yes, arguably, someone or someones doing a value add would have to be
>> making money at it somehow.
>>
>> However, at least in my world, volunteers make the world round, still.
>> It would kind of suck,
>> I suppose, if someone unleashed a few hundred thousand reflashed
>> routers like the TIP openwifi effort (
>> https://telecominfraproject.com/ ) seem to intent on doing...
>>
>> ... but if the OS is good enough to not need support, the impact is minimal.
>> >
>> > On Thu, May 19, 2022 at 10:30 AM Dave Taht <dave.taht@gmail.com> wrote:
>> >>
>> >> On Thu, May 19, 2022 at 9:07 AM NetEquity Sales <sales@netequity.com> wrote:
>> >> >
>> >> > As someone who works within the "secondary market" for networking hardware, there is a ton of demand spilling over into the "pre-owned/vendor refurbished" market.
>> >>
>> >> I just wish there were people putting in a value-add, like
>> >> reflashing with better software, first.
>> >>
>> >> >
>> >> > Market prices on pre-owned equipment are rapidly increasing in step with increased demand and dwindling supply.
>> >> >
>> >> > Market prices on 1G - 10G switching products, wireless infrastructure devices, etc have been rising precipitously. Even semi "legacy" stuff going back 2-3 generations (EOL/EOS) from current gen have doubled, tripled, even quadrupled in price.
>> >> >
>> >> > I've been involved in the hardware business for 20 years and the current market landscape is unprecedented.
>> >> >
>> >> >
>> >> >
>> >> > Cory J. Andrews
>> >> > ++++++++++++++
>> >> > NetEquity.com (Formerly CiscoBuy.com)
>> >> > 4519 Northgate Court
>> >> > Sarasota, FL 34234
>> >> > ++++++++++++++
>> >> > TF/FAX 877.582.4726
>> >> > E - sales@netequity.com
>> >> >
>> >> > On Thu, May 19, 2022, 9:48 AM Josh Luthman <josh@imaginenetworksllc.com> wrote:
>> >> >>
>> >> >> I'd bet it's cheaper and easier to quantify new hardware than software. Labor was super expensive and now it is ready to implode.
>> >> >>
>> >> >> On Thu, May 19, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
>> >> >>>
>> >> >>> As I've been saying for a while, instead of buying new kit,
>> >> >>> perhaps we could spend some time on getting better software onto our older kit?
>> >> >>> Getting stuff to multiplex better, be more reliable, last longer?
>> >> >>>
>> >> >>> It isn't just me wanting to upgrade a billion+ routers with
>> >> >>> existing crappy software to openwrt, is it?
>> >> >>>
>> >> >>> https://docs.google.com/document/d/1T21on7g1MqQZoK91epUdxLYFGd
>> >> >>> tyLRgBat0VXoC9e3I/edit
>> >>
>> >>
>> >>
>> >> --
>> >> FQ World Domination pending:
>> >> https://blog.cerowrt.org/post/state_of_fq_codel/
>> >> Dave Täht CEO, TekLibre, LLC
>> >
>> >
>> >
>> > --
>> > Jason
>>
>>
>>
>> --
>> FQ World Domination pending:
>> https://blog.cerowrt.org/post/state_of_fq_codel/
>> Dave Täht CEO, TekLibre, LLC
>
>
>
> --
> Jason



--
FQ World Domination pending: https://blog.cerowrt.org/post/state_of_fq_codel/
Dave Täht CEO, TekLibre, LLC
Re: Any sign of supply chain returning to normal? [ In reply to ]
On 5/19/22 16:07, NetEquity Sales wrote:

> As someone who works within the "secondary market" for networking
> hardware, there is a ton of demand spilling over into the
> "pre-owned/vendor refurbished" market.
>
> Market prices on pre-owned equipment are rapidly increasing in step
> with increased demand and dwindling supply.
>
> Market prices on 1G - 10G switching products, wireless infrastructure
> devices, etc have been rising precipitously. Even semi "legacy" stuff
> going back 2-3 generations (EOL/EOS) from current gen have doubled,
> tripled, even quadrupled in price.
>
> I've been involved in the hardware business for 20 years and the
> current market landscape is unprecedented.

And this is the case for Transport gear as well, not just IP/MPLS/Ethernet.

Mark.
Re: Any sign of supply chain returning to normal? [ In reply to ]
On 5/19/22 15:27, Dave Taht wrote:

> As I've been saying for a while, instead of buying new kit, perhaps we
> could spend some time on getting better software onto our older kit?
> Getting stuff to multiplex better, be more reliable, last longer?

So we've been running Arista's 7508E devices as core switches in data
centres to support 1Gbps, 10Gbps and 100Gbps internal cabling (purely
Layer 2, no IP).

We suddenly got told that they are now EoS/EoL some time back (I
probably should have done a better job tracking this, but I tend to rely
on vendor notifications for that based on my Cisco/Juniper experience).
So Arista advised we move to the 7508R3, which doesn't make sense for us
because we are currently only using 2x slots from the chassis, and
nowhere close to max'ing out the line card or switch fabric capacity.

It just didn't make sense to us to spend hundreds-of-thousands of $$ for
no extra benefit in performance or technology (Layer 2 is very simple).

So we decided to keep the the 7508E, even if they are EoL. The box is
brain-dead, runs fine, shifts bits nice and good, and hums along quietly.

I can't fathom why a box like that has already been EoL'ed. It isn't
long in the tooth, and still has plenty of bite in it. But, we also need
to use common sense, and for us, swapping it out just to maintain
"support" isn't worth the cash. There are options for cold sparing...

Mark.
Re: Any sign of supply chain returning to normal? [ In reply to ]
On 5/19/22 18:15, Dave Taht wrote:

> So I kind of view recycling routers, with newer software, as a great
> way to clean up the present ecosystem. And if you looked at the first
> url I pasted above, with 4x more throughput, and 10x less latency, on
> "obsolete", hw.

When I had my battle with Cisco over LDPv6 vs. SR(v6) in 2020, I told
them that Covid has really changed the landscape, and people (read:
their customers) no longer have money to spend like they did, for a
multitude of reasons, not the top of which is a lack. of. money.

I said to them that they need to focus on helping customers answer their
"why", and not continue with the old model of having sales meetings and
assigning $$ values against customer names for the year, as if ants
visit PoP's and chew routers down to smithereens as a matter of course.

People no longer have money to spend on things that don't add value. And
while routers do add value, how vendors choose to make money from them
beyond selling the hardware and providing decent support is what erodes
that value, and customer trust.

Users will delete an app in 5 seconds if they launch it and it doesn't
do what it claims in a way they perceive as value. Service providers
will do the exact same thing to vendors that act like disappointing apps
competing for space on your phone and space in your mind.

Mark.
Re: Any sign of supply chain returning to normal? [ In reply to ]
On Fri, 20 May 2022 at 11:21, Mark Tinka <mark@tinka.africa> wrote:

> I can't fathom why a box like that has already been EoL'ed. It isn't long in the tooth, and still has plenty of bite in it. But, we also need to use common sense, and for us, swapping it out just to maintain "support" isn't worth the cash. There are options for cold sparing...

That's engineering, understanding what risks and compromises are worth
carrying. If you do it by the book, you're not even needed, just
0-rate AS/PS services to your RFP and the vendor is happy to do it by
the book for you.
And fully agreed, in many cases it makes sense to run boxes to the
ground until they physically stop working. You just need to figure out
how to handle the MTTR well.

--
++ytti
Re: Any sign of supply chain returning to normal? [ In reply to ]
On 5/20/22 10:24, Saku Ytti wrote:

> That's engineering, understanding what risks and compromises are worth
> carrying. If you do it by the book, you're not even needed, just
> 0-rate AS/PS services to your RFP and the vendor is happy to do it by
> the book for you.
> And fully agreed, in many cases it makes sense to run boxes to the
> ground until they physically stop working. You just need to figure out
> how to handle the MTTR well.

We are going to need more 100Gbps data centre switching in other PoP's
in the coming months, and we like the 7508E for this, even if Arista
have EoL'ed it. So for those builds, we'll grab them off the open
market, and cold spare with a full chassis in copy. Way cheaper than
when we bought them from Arista, even with the ongoing mark-up on the
pre-owned space.

Arista are also not developing anymore new or maintenance code for the
supervisor that runs on the 7508E, but we are fine with that because
this is not an Internet-facing box, and is only doing Layer 2 with
features that will never change (802.1Q and LACP is about as advanced as
it gets, for us).

Mark.
Re: Any sign of supply chain returning to normal? [ In reply to ]
On Fri May 20, 2022 at 10:15:14am +0200, Mark Tinka wrote:
> We suddenly got told that they are now EoS/EoL some time back (I probably
> should have done a better job tracking this, but I tend to rely on vendor
> notifications for that based on my Cisco/Juniper experience).

I've heard that some vendors are prematurely EoS/EoL'ing kit as a result of
the silicon shortages - and redesigning kit to use silicon that's easier to
get hold of.

Simon
Re: Any sign of supply chain returning to normal? [ In reply to ]
Simon,

That is correct in most cases. Fabs are changing and the time needed to manufacture is limited and most vendors are opt’ing to EoL/EoS gear early to move to newer tech and cannot risk the time to manufacture older silicon and spares that could be used for newer silicon.

On Fri, May 20, 2022, at 07:44, Simon Lockhart wrote:
> On Fri May 20, 2022 at 10:15:14am +0200, Mark Tinka wrote:
> > We suddenly got told that they are now EoS/EoL some time back (I probably
> > should have done a better job tracking this, but I tend to rely on vendor
> > notifications for that based on my Cisco/Juniper experience).
>
> I've heard that some vendors are prematurely EoS/EoL'ing kit as a result of
> the silicon shortages - and redesigning kit to use silicon that's easier to
> get hold of.
>
> Simon
>

----
Jason
Re: Any sign of supply chain returning to normal? [ In reply to ]
On 5/20/22 14:44, Simon Lockhart wrote:

> I've heard that some vendors are prematurely EoS/EoL'ing kit as a result of
> the silicon shortages - and redesigning kit to use silicon that's easier to
> get hold of.

This was my suspicion, because we started using this box in 2017. I
can't remember when it launched, but I'd imagine a year or two earlier.

We have boxes from 2014 with other vendors still in service and fully
supported. So we were not going to be bullied into new hardware we
didn't need.

Mark.