Mailing List Archive

hey,

> 7280R, Jericho.

What is the motivation to run jericho in your L2-only setup (instead
trident)? Only buffer space?

--
tarko
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

On 23/Sep/19 13:14, Tarko Tikan wrote:

>  
>
> What is the motivation to run jericho in your L2-only setup (instead
> trident)? Only buffer space?

We were chasing the actual switch, as it met all of what we needed,
including the larger buffer space.

Mark.

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

> On Sep 23, 2019, at 5:11 AM, Mark Tinka <mark.tinka@seacom.mu> wrote:
>
> This is the major driving reason behind us avoiding the NCS540
> for the Metro.

What are your other requirements? Who/what else are you looking at?
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

On 23/Sep/19 14:07, Jason Lixfeld wrote:
> What are your other requirements? Who/what else are you looking at?

We were the first ISP in the world to run IP/MPLS all the way into the
Access back in 2009 - TIME dotCom, Malaysia - on the Cisco ME3600X.

I haven't operated that network since 2012, but it keeps growing from
what I know, now having migrated to the Cisco ASR920.

We looked at other vendors, and the only one that came close to Cisco's
competition was Brocade with their CES/CER2000 NetIron. But Cisco's
overall price, design and feature set on the ME3600X was just too good.

Juniper have dropped the ball here for years. Until the MX204. However,
the MX204 is good if you run 10Gbps customers in the Metro. Otherwise,
for now, nothing beats the ASR920, IMHO.

So in short, we want a full-blown router in the Access, but designed
cheaply enough for Metro applications.

Mark.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

> On Sep 23, 2019, at 2:17 PM, Mark Tinka <mark.tinka@seacom.mu> wrote:
>
> On 23/Sep/19 14:07, Jason Lixfeld wrote:
>> What are your other requirements? Who/what else are you looking at?
>
> We were the first ISP in the world to run IP/MPLS all the way into the
> Access back in 2009 - TIME dotCom, Malaysia - on the Cisco ME3600X.
>
> I haven't operated that network since 2012, but it keeps growing from
> what I know, now having migrated to the Cisco ASR920.
>
> We looked at other vendors, and the only one that came close to Cisco's
> competition was Brocade with their CES/CER2000 NetIron. But Cisco's
> overall price, design and feature set on the ME3600X was just too good.
>
> Juniper have dropped the ball here for years. Until the MX204. However,
> the MX204 is good if you run 10Gbps customers in the Metro. Otherwise,
> for now, nothing beats the ASR920, IMHO.

Is the MX204 not a prohibitively expensive 10G port?

> So in short, we want a full-blown router in the Access, but designed
> cheaply enough for Metro applications.

If you were considering the NCS540, then you’re OK with BCM there? If so, something like the ACX5448, except far more in-line with the NCS540 price per port. NCS540 pricing/10G port is much more realistic to what a 10G port should cost these days vs. MX204 or ACX5448.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Hi,

On Mon, Sep 23, 2019 at 02:37:08PM -0400, Jason Lixfeld wrote:
> > Juniper have dropped the ball here for years. Until the MX204. However,
> > the MX204 is good if you run 10Gbps customers in the Metro. Otherwise,
> > for now, nothing beats the ASR920, IMHO.
>
> Is the MX204 not a prohibitively expensive 10G port?

Among routers (full flexibility, large tables, buffers, ...) it's a
fairly good bargain - compare this to ASR9001 or ASR9901 from the Cisco
camp.

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

On 23/Sep/19 20:37, Jason Lixfeld wrote:

> Is the MX204 not a prohibitively expensive 10G port?

When you consider that you can:

    - Drive a Metro-E ring at 100Gbps with no special Transport gear.
    - Hang an ASR920 or two off of those 10Gbps ports to manage
low-speed customers.
    - Place your "serious" premium 10Gbps customers into one of them at
low volume with decent margin.

It makes plenty sense if you want to maintain an intelligent Access network.

If your design is to have a dumb Layer 2 Access/Metro and haul
everything into a central "IPGW", ah well...


> If you were considering the NCS540, then you’re OK with BCM there?

If I'm honest, IOS XR was the first thing that put us off. We didn't
make it far enough to even worry about the chipset :-).


> If so, something like the ACX5448, except far more in-line with the NCS540 price per port.

The ACX5000 was a box we looked at back in 2015 and just moved on.
Nothing has changed there.


> NCS540 pricing/10G port is much more realistic to what a 10G port should cost these days vs. MX204 or ACX5448..

Agree, but with us, it's not all just feeds and speeds. A box is more
than the sum of its ports. I'm happy to take a reasonably less dense box
if I am making money on it, than a dense, cheaper box where I'm still
making money but also staying up at 3AM every night.

Mark.

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

On 23/Sep/19 20:39, Gert Doering wrote:

> Among routers (full flexibility, large tables, buffers, ...) it's a
> fairly good bargain - compare this to ASR9001 or ASR9901 from the Cisco
> camp.

This.

Mark.

Hi,


Up grade issues !

This is very valid point, we are totally fed up with,

Upgrade is a horrible

We got round 2 and half years to upgrade around 200 Ex4200 devices.
Initially every one in three devices failed to boot after the upgrade.
Juniper keep on preaching us go to the site and format install.
They don’t seems understand how impractical to upgrade large number of
devices on console.

Finally long chase up they agreed they have issue and provided working long
process which took lots of time to script. Still more than 5% failed
causing incidents,

Worse thing is after 6 months all these hard works. We found some nasty
bugs, now they recommend new version.


Two months back i started up grading around 120 numbers of ACx2100 devices.
Process is very complicated. Firstly we found Re filters are broken in
interim Version loosing remote access. Some has to go to site and remove RE
filters.
That is fixed by removing RE filters before the upgrade then we found ssh
folders are deleted during the request system software add process this
cause access issues when upgrade fails due to validation errors. There are
were many incompatible configs between versions.
It took two solid months and many sleepless nights to get smooth running
process, JTAC response to these issues are not very helpfull some times.
there response is always workarounds not to the root course.

On MX upgrades we had very horrible stories too. Large number of DPC cards
didnt boot up after the upgrades, They are like price of a luxury car when
we bought them. I cant imagine why they are so delicate. Our upgrades put
on hold few times due large number of failures. We still have one MX960
with old version, no one likes to touch because it had many DPC cards on it.

I agree that Junos is very mature OS but these issues causing the network
hard to maintain.

Thank you

Kamal

On Monday, September 23, 2019, Karl Gerhard <karl_gerh@gmx.at> wrote:

> Hi,
>
> I'd like to point out one more thing because I feel that this point hasn't
> been stressed enough:
> Upgrading Junos might be more time consuming than many people expect it to
> be.
>
> The reason for this is that quite often, things that previously worked in
> Junos will break in a new release. This affects very, very basic things
> too. Even if you don't do magic fairy-stuff like EVPN-MPLS to EVPN-VXLAN
> handoff or any MPLS at all, you might get bitten by this at some point.
>
> One example:
> After an upgrade we noticed that local-preference just stopped working.
> After investigating this, we realized that traffic for some subnets was
> being diverted, while for others it was still following the old path.
> Reason for this was that we had BGP PIC enabled. We had to downgrade to
> another version because BGP PIC is important to us.
>
> Another, more nasty example of very basic things failing after an upgrade
> is this:
> We had packet loss on an IBGP link due to a dirty fiber. No problem, we
> run LAGs everywhere, so we just disabled the link and send a technician to
> the datacenter to clean the fiber. However, at the moment when we disabled
> the dirty fiber some customers went completely offline. At this point a
> minor incident had turned into a major one because customers were feeling
> the impact and we started receiving calls from them. However, the routers
> did not show any errors on either end. LACP was still working fine on the
> three other links, BGP was still fine, lots of traffic was being forwarded.
> But we were dropping packets somewhere, some customers were offline and
> network engineers couldn't tell their boss what was wrong. Not a very
> pleasant situation to be in.
> In the end, we found out that the last upgrade introduced a new bug: If
> you disable an interface that is part of a LAG, Junos would still hash
> traffic on to that interface, so with a LAG consisting of 4 interfaces
> where you disabled one, a quater of the traffic would just be blackholed.
>
> Be ready to put up with shit like this if you buy Juniper. The officially
> recommended Juniper releases won't save you either: They too contain newly
> introduced bugs that break things that previously worked flawlessly. This
> is the main problem I have with Juniper: If you upgrade, you might spend
> days debugging stuff that used to work flawlessly for years. Even the most
> basic things like LAGs aren't safe. You might think that you found a
> version that works for you, but then, weeks later, you find something that
> got broken with the upgrade and then you need to schedule a new
> upgrade/downgrade.
>
> The big companies have fancy and expensive labs and employees that spend
> weeks testing new releases. However, we're a small hosting provider running
> a bunch of MX480ies and other Juniper stuff. I need routers that I can
> upgrade without fearing that my network will explode. Can't have that with
> Juniper.
>
> If I were to rebuild our network again, I'd take a very good look at
> Arista/ANET. As Saku already mentioned, they're the ones that have the best
> practices in developing software. They might not have all the bells and
> whistles that Juniper have, but at least I might get more sleep and peace
> of mind when upgrading those than I got with my Juniper gear.
>
> Regards
> Karl
>
>
>
>
> On 19.09.19 09:09, Gert Doering wrote:
> > Hi,
> >
> > On Thu, Sep 19, 2019 at 05:04:54PM +1200, Phil Reilly wrote:
> >> MX104's are the dual brain unit of the 204. Though a 204 has 40/100G
> >> capabilities. If I read your original request correctly about ip
> >> routing. Not sure the 104/204 is grunty enough to deal with multiple
> >> internet tables. Thats a demanding task these days best left to the
> >> larger chassis.
> > You can't really compare MX104 to MX204.
> >
> > The MX104 is ppc based and *slow*, and should have never ever shipped.
> >
> > The MX204 is a really really nice box, with a fast intel RE and 40/100G
> > ports (though some - documented - restrictions on how they can be
> combined),
> > and from a RE/BGP point of view, en par with the larger MXes.
> >
> > And given the price point of the MX204, if the amount of interfaces is
> > sufficient, just get two of them :-)
> >
> > gert
> >
> > _______________________________________________
> > juniper-nsp mailing list juniper-nsp@puck.nether.net
> > https://puck.nether.net/mailman/listinfo/juniper-nsp
>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Hello Kamal

thanks for sharing your valuable insight. Very interesting to see what happens when one tries to upgrade hundreds of Juniper devices.
Upon reading your mail one could get the impression that the upgrade woes and all the bugs+regressions are a structural problem related to how Junos is being developed.

Two more things:
*The two bugs I mentioned in my last mail affected Junos 17.4 and Junos 18.1. I wish QA was bad just for Junos 14/15, but no, the problems seem to persist.
*We've solved the problem you described Kamal by never upgrading any of our access switches (EX4300 and EX4200) at all. We install a tried and tested firmware before deploying them to prod and then we're done, hopefully forever. After lots of experimentation, we use our EX4200 and EX4300 only for switching, me0 is the only interface with an IP address. However, I do understand that this only feasible if you run a very uniform environment as we do.

Regards
Karl



On 01.10.19 00:40, Kamal Dissanayaka wrote:
> Hi,
>
>
> Up grade issues !
>
> This is very valid point, we are totally fed up with,
>
> Upgrade is a horrible
>
> We got round 2 and half years to upgrade around 200 Ex4200 devices.
> Initially every one in three devices failed to boot after the upgrade.
> Juniper keep on preaching us go to the site and format install.
> They don’t seems understand how impractical to upgrade large number of
> devices on console.
>
> Finally long chase up they agreed they have issue and provided working long
> process which took lots of time to script. Still more than 5% failed
> causing incidents,
>
> Worse thing is after 6 months all these hard works. We found some nasty
> bugs, now they recommend new version.
>
>
> Two months back i started up grading around 120 numbers of ACx2100 devices.
> Process is very complicated. Firstly we found Re filters are broken in
> interim Version loosing remote access. Some has to go to site and remove RE
> filters.
> That is fixed by removing RE filters before the upgrade then we found ssh
> folders are deleted during the request system software add process this
> cause access issues when upgrade fails due to validation errors. There are
> were many incompatible configs between versions.
> It took two solid months and many sleepless nights to get smooth running
> process, JTAC response to these issues are not very helpfull some times.
> there response is always workarounds not to the root course.
>
> On MX upgrades we had very horrible stories too. Large number of DPC cards
> didnt boot up after the upgrades, They are like price of a luxury car when
> we bought them. I cant imagine why they are so delicate. Our upgrades put
> on hold few times due large number of failures. We still have one MX960
> with old version, no one likes to touch because it had many DPC cards on it.
>
> I agree that Junos is very mature OS but these issues causing the network
> hard to maintain.
>
> Thank you
>
> Kamal
>
> On Monday, September 23, 2019, Karl Gerhard <karl_gerh@gmx.at> wrote:
>
>> Hi,
>>
>> I'd like to point out one more thing because I feel that this point hasn't
>> been stressed enough:
>> Upgrading Junos might be more time consuming than many people expect it to
>> be.
>>
>> The reason for this is that quite often, things that previously worked in
>> Junos will break in a new release. This affects very, very basic things
>> too. Even if you don't do magic fairy-stuff like EVPN-MPLS to EVPN-VXLAN
>> handoff or any MPLS at all, you might get bitten by this at some point.
>>
>> One example:
>> After an upgrade we noticed that local-preference just stopped working.
>> After investigating this, we realized that traffic for some subnets was
>> being diverted, while for others it was still following the old path.
>> Reason for this was that we had BGP PIC enabled. We had to downgrade to
>> another version because BGP PIC is important to us.
>>
>> Another, more nasty example of very basic things failing after an upgrade
>> is this:
>> We had packet loss on an IBGP link due to a dirty fiber. No problem, we
>> run LAGs everywhere, so we just disabled the link and send a technician to
>> the datacenter to clean the fiber. However, at the moment when we disabled
>> the dirty fiber some customers went completely offline. At this point a
>> minor incident had turned into a major one because customers were feeling
>> the impact and we started receiving calls from them. However, the routers
>> did not show any errors on either end. LACP was still working fine on the
>> three other links, BGP was still fine, lots of traffic was being forwarded.
>> But we were dropping packets somewhere, some customers were offline and
>> network engineers couldn't tell their boss what was wrong. Not a very
>> pleasant situation to be in.
>> In the end, we found out that the last upgrade introduced a new bug: If
>> you disable an interface that is part of a LAG, Junos would still hash
>> traffic on to that interface, so with a LAG consisting of 4 interfaces
>> where you disabled one, a quater of the traffic would just be blackholed.
>>
>> Be ready to put up with shit like this if you buy Juniper. The officially
>> recommended Juniper releases won't save you either: They too contain newly
>> introduced bugs that break things that previously worked flawlessly. This
>> is the main problem I have with Juniper: If you upgrade, you might spend
>> days debugging stuff that used to work flawlessly for years. Even the most
>> basic things like LAGs aren't safe. You might think that you found a
>> version that works for you, but then, weeks later, you find something that
>> got broken with the upgrade and then you need to schedule a new
>> upgrade/downgrade.
>>
>> The big companies have fancy and expensive labs and employees that spend
>> weeks testing new releases. However, we're a small hosting provider running
>> a bunch of MX480ies and other Juniper stuff. I need routers that I can
>> upgrade without fearing that my network will explode. Can't have that with
>> Juniper.
>>
>> If I were to rebuild our network again, I'd take a very good look at
>> Arista/ANET. As Saku already mentioned, they're the ones that have the best
>> practices in developing software. They might not have all the bells and
>> whistles that Juniper have, but at least I might get more sleep and peace
>> of mind when upgrading those than I got with my Juniper gear.
>>
>> Regards
>> Karl
>>
>>
>>
>>
>> On 19.09.19 09:09, Gert Doering wrote:
>>> Hi,
>>>
>>> On Thu, Sep 19, 2019 at 05:04:54PM +1200, Phil Reilly wrote:
>>>> MX104's are the dual brain unit of the 204. Though a 204 has 40/100G
>>>> capabilities. If I read your original request correctly about ip
>>>> routing. Not sure the 104/204 is grunty enough to deal with multiple
>>>> internet tables. Thats a demanding task these days best left to the
>>>> larger chassis.
>>> You can't really compare MX104 to MX204.
>>>
>>> The MX104 is ppc based and *slow*, and should have never ever shipped.
>>>
>>> The MX204 is a really really nice box, with a fast intel RE and 40/100G
>>> ports (though some - documented - restrictions on how they can be
>> combined),
>>> and from a RE/BGP point of view, en par with the larger MXes.
>>>
>>> And given the price point of the MX204, if the amount of interfaces is
>>> sufficient, just get two of them :-)
>>>
>>> gert
>>>
>>> _______________________________________________
>>> juniper-nsp mailing list juniper-nsp@puck.nether.net
>>> https://puck.nether.net/mailman/listinfo/juniper-nsp
>> _______________________________________________
>> juniper-nsp mailing list juniper-nsp@puck.nether.net
>> https://puck.nether.net/mailman/listinfo/juniper-nsp
>>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp