Mailing List Archive

Hi,

> On Sep 18, 2019, at 5:15 PM, Howard Leadmon <howard@leadmon.net> wrote:
>
>
> I am looking to replace an older Cisco I have sitting down in Equinix, and have l have had a few tell me that I should look at the Juniper routers as well.

Diving into Juniper/JunOS isn’t for the faint of heart. It’s a completely different animal.

JunOS is a slick, powerful OS, no doubt, but there’s so much nuanced configuration to hardware compatibility, and more feature-x-stops-working-after-software-upgrade headaches than I could have ever imagined from a vendor. Their policy and filter language is very comprehensive, but very complicated and takes a long time to get familiar with and understand.

For the past year I’ve been testing Juniper/JunOS for the first time, and you know, I’ve got to tell you, you might want to stick to Cisco if that’s what you're comfortable with. Testing has been long and frustrating, and of the 4 platform that I was looking at, all for various tasks, I decided that the only role I’m comfortable having played by Juniper is an MX204 running as a BFD/ISIS/LDP/MPLS enabled peering/transit border router. So far, it seems to do that *really* well. However, based on feedback received from people who have been running Juniper for a long time, and reading what folks have been saying here and elsewhere, we feel that it’s just too risky to put this stuff anywhere else.

Don’t get me wrong, Cisco definitely owns their share of BS, but they seem to be predictable in how they are going to screw you. Juniper will screw you, but how they are going to screw you seems to be predictably unpredictable.

FWIW, you may want to check out Arista’s 7280R. We’ve just deployed a pair of these for EVPN-MPLS and they’re slick, and from what I understand, they have the FIB scale to be able to act as a border router. It’s a very IOS-like CLI (but so many things about the CLI are so much more refined than IOS) so it may be more familiar, unless you’re Cisco experience is limited to IOS-XR. It’s about USD$50K list for 48 x SFP+ / 6 x 40/100G, including licensing.

It’s a BCM Jericho based pizza box, so that’s redundant powered, but not “redundant” in so far as there are no redundant supervisor/management cards. But, for the number of times I’ve had that kind of failure on any of my boxes that have had redundant cards, I don’t think it’s worth the cost or the rack space, especially if it’s just a border router where you’ve probably got a bunch of other border router that can accommodate a crash or a reboot or whatever.

Hope that helps.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

> However, based on feedback received from people who have been running Juniper for a long time, and reading what folks have been saying here and elsewhere, we feel that it’s just too risky to put this stuff anywhere else.

We have been using Juniper since 2003 and that is definitely _not_ my
experience. We have used them extensively as P, PE, and Internet Edges
in a tier 1 regional carrier. (We also have used ASRs in the same
capabilities). IMO both do Internet Edge well. But I and most of whom I
know would take an MX960 over an ASR9k any day, the BGP functionality
and extensions are well developed in JUNOS. Also note that Junipers main
philosophy and focus is based on MPLS so thats why its dominant in
carriers for that function.

Cisco do EVPN/vxlan well. They do CE well, (though I'd argue the SRX is
a good candidate for this too). But they just cant compete against
Juniper for SP stuff IMO. Same for Arista. That are good in the DC but
they don't have much carrier market share I believe. Anyway you will
note my bias leans towards Juniper in the IP routing world.

I find the juniper licensing a bit more upfront and honest too. Though
with the higher speed capabilities that's eroding somewhat.

MX104's are the dual brain unit of the 204. Though a 204 has 40/100G
capabilities. If I read your original request correctly about ip
routing. Not sure the 104/204 is grunty enough to deal with multiple
internet tables. Thats a demanding task these days best left to the
larger chassis.

On 19/09/2019 10:52 AM, Jason Lixfeld wrote:
> Hi,
>
>> On Sep 18, 2019, at 5:15 PM, Howard Leadmon <howard@leadmon.net> wrote:
>>
>>
>> I am looking to replace an older Cisco I have sitting down in Equinix, and have l have had a few tell me that I should look at the Juniper routers as well.
> Diving into Juniper/JunOS isn’t for the faint of heart. It’s a completely different animal.
>
> JunOS is a slick, powerful OS, no doubt, but there’s so much nuanced configuration to hardware compatibility, and more feature-x-stops-working-after-software-upgrade headaches than I could have ever imagined from a vendor. Their policy and filter language is very comprehensive, but very complicated and takes a long time to get familiar with and understand.
>
> For the past year I’ve been testing Juniper/JunOS for the first time, and you know, I’ve got to tell you, you might want to stick to Cisco if that’s what you're comfortable with. Testing has been long and frustrating, and of the 4 platform that I was looking at, all for various tasks, I decided that the only role I’m comfortable having played by Juniper is an MX204 running as a BFD/ISIS/LDP/MPLS enabled peering/transit border router. So far, it seems to do that *really* well. However, based on feedback received from people who have been running Juniper for a long time, and reading what folks have been saying here and elsewhere, we feel that it’s just too risky to put this stuff anywhere else.
>
> Don’t get me wrong, Cisco definitely owns their share of BS, but they seem to be predictable in how they are going to screw you. Juniper will screw you, but how they are going to screw you seems to be predictably unpredictable.
>
> FWIW, you may want to check out Arista’s 7280R. We’ve just deployed a pair of these for EVPN-MPLS and they’re slick, and from what I understand, they have the FIB scale to be able to act as a border router. It’s a very IOS-like CLI (but so many things about the CLI are so much more refined than IOS) so it may be more familiar, unless you’re Cisco experience is limited to IOS-XR. It’s about USD$50K list for 48 x SFP+ / 6 x 40/100G, including licensing.
>
> It’s a BCM Jericho based pizza box, so that’s redundant powered, but not “redundant” in so far as there are no redundant supervisor/management cards. But, for the number of times I’ve had that kind of failure on any of my boxes that have had redundant cards, I don’t think it’s worth the cost or the rack space, especially if it’s just a border router where you’ve probably got a bunch of other border router that can accommodate a crash or a reboot or whatever.
>
> Hope that helps.
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Hi,

On Thu, Sep 19, 2019 at 05:04:54PM +1200, Phil Reilly wrote:
> MX104's are the dual brain unit of the 204. Though a 204 has 40/100G
> capabilities. If I read your original request correctly about ip
> routing. Not sure the 104/204 is grunty enough to deal with multiple
> internet tables. Thats a demanding task these days best left to the
> larger chassis.

You can't really compare MX104 to MX204.

The MX104 is ppc based and *slow*, and should have never ever shipped.

The MX204 is a really really nice box, with a fast intel RE and 40/100G
ports (though some - documented - restrictions on how they can be combined),
and from a RE/BGP point of view, en par with the larger MXes.

And given the price point of the MX204, if the amount of interfaces is
sufficient, just get two of them :-)

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

On Thu, 19 Sep 2019 at 01:52, Jason Lixfeld <jason-jnsp@lixfeld.ca> wrote:

> Don’t get me wrong, Cisco definitely owns their share of BS, but they seem to be predictable in how they are going to screw you. Juniper will screw you, but how they are going to screw you seems to be predictably unpredictable.

I have to say this is entirely subjective and my subjective experience
is rather different. But it is to be expected, if we would have very
homogenous view on which vendor is superior, competing vendors would
disappear.

My subjective views where JNPR shines compared to market

a) model driven configuration since day1, this is a big thing, much
bigger than YANG or Netconf or whatever other automation hype is going
on. Junos doesn't actively resist automation.
b) Trio is the only platform I know how to protect control-plane so
that I don't know how to break it
c) very debuggable (CSCO is good too here, this is more complaint to
Other Options).
d) I will echo your sentiment about complexity and agree to it, junos
gives a lot of power to the operator, where CSCO might have some
specific feature they support, have a name for, and have single line
config, JNPR doesn't have anything, because the feature has always
been possible due to the inherent flexibility and expressiveness of
the system. Some particular examples, BGP GTSM, VRF source selection,
egress-acl-at-ingress, there are plenty of examples in this, but can
be very involved to get right.

I don't have any dislike for CSCO, I'm sure you can support your
business case with both vendors, or Nokia or Huawei. Subjectively for
SP role, I think Nokia and Huawei have best port/chassis options. I
think Juniper has best NPU and best NOS on the market. For SP role
particularly, from the big names, today I think CSCO has least
attractive portfolio, but this is normal timing related thing, at some
point any vendor is behind, once they release some new generation
stuff they'll move ahead and start regressing. I believe ANET has best
practices in developing software and considering how young they are on
SP market, it's quite impressive where they are.

--
++ytti
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

gert@greenie.muc.de (Gert Doering) wrote:

> And given the price point of the MX204, if the amount of interfaces is
> sufficient, just get two of them :-)

Alternatively, just add QFXn as satellites.

El El
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

> Phil Reilly
> Sent: Thursday, September 19, 2019 6:05 AM
>
> the BGP functionality and extensions are well
> developed in JUNOS.
Ha... :)
Just a few examples when you change export policy it resets the peer or the cockup with RR clearing all sessions or the fact BGP is part of very complex RDP monolith -to me that's not really "carrier grade" implementation

adam

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

On Thu, 19 Sep 2019 at 14:22, <adamv0025@netconsultings.com> wrote:

> Just a few examples when you change export policy it resets the peer or the cockup with RR clearing all sessions or the fact BGP is part of very complex RDP monolith -to me that's not really "carrier grade" implementation

This happens when export policy breaks update-group. It may sometimes
be difficult for operator to understand if it will do that or not, so
it's fair concern. Perhaps system should not clear, but tell manual
clear is needed for policy change to take effect.

If monolith is good or bad, I'm not sure. If you thread you have high
performance with some risk. If you have process separation you have
IPC problem, and you have low performance and many will solve this by
duplicating state. Junos is moving towards multi process model with
Junos Evolved, if this will be positive or negative direction remains
to be seen.

Operationally speaking, BGP in JunOS for us works great, on IOS-XR
right now we have sessions where policy isn't what is configured and
there is no way to verify which one, and we've propagated leaks
because acting configuration isn't the one we've configured. We've not
had similar problems in JunOS. This is anecdote, not data.



--
++ytti
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

> From: Saku Ytti <saku@ytti.fi>
> Sent: Thursday, September 19, 2019 12:33 PM
>
> On Thu, 19 Sep 2019 at 14:22, <adamv0025@netconsultings.com> wrote:
>
> > Just a few examples when you change export policy it resets the peer
> > or the cockup with RR clearing all sessions or the fact BGP is part of
> > very complex RDP monolith -to me that's not really "carrier grade"
> > implementation
>
> This happens when export policy breaks update-group. It may sometimes be
> difficult for operator to understand if it will do that or not, so it's fair concern.
> Perhaps system should not clear, but tell manual clear is needed for policy
> change to take effect.
>
Ideally I'd like to see equivalent of Cisco's dynamic update peer-groups in Junos.

> If monolith is good or bad, I'm not sure. If you thread you have high
> performance with some risk. If you have process separation you have IPC
> problem, and you have low performance and many will solve this by
> duplicating state. Junos is moving towards multi process model with Junos
> Evolved, if this will be positive or negative direction remains to be seen.
>
I like where XR and Junos Evolved is heading,
In future I'd like to have the option to install only stuff I need on a particular type of node/deployment and not worry about the rest all the way to being able to mix and match protocols of different vendors.
Although cRPD is also interesting development pathway, but again cBGP would be even better :)

> Operationally speaking, BGP in JunOS for us works great, on IOS-XR right now
> we have sessions where policy isn't what is configured and there is no way to
> verify which one, and we've propagated leaks because acting configuration
> isn't the one we've configured. We've not had similar problems in JunOS. This
> is anecdote, not data.
>
Right this inconsistency between configured and operational state in my opinion is THE biggest problem of XR, I'm afraid it has to be something fundamental since they haven't been able to consistently address these inconsistencies across the board for years now (or ASR9k HW? Not sure if these types of issues can be experienced on other platforms).
Though usually it's CP state does not trickle down to DP correctly/completely where what you described seems to be CP only.

adam

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

On Thu, 19 Sep 2019 at 15:11, <adamv0025@netconsultings.com> wrote:

> Ideally I'd like to see equivalent of Cisco's dynamic update peer-groups in Junos.

They are dynamic, but once you make export change which affects subset
of members in peer-group, that member gets reset while placed to new
update-group.

> Right this inconsistency between configured and operational state in my opinion is THE biggest problem of XR, I'm afraid it has to be something fundamental since they haven't been able to consistently address these inconsistencies across the board for years now (or ASR9k HW? Not sure if these types of issues can be experienced on other platforms).

It is fundamental with non-monolithic design, because processA has no
access to the memory of processB you need IPC, which for many things
is too slow, so you replicate state and sync the state, which
increases complexity and adds bugs.
The argument against monolithic design is that single component
failure will bring the whole house of cards down. I think this is
debatable, if it's BGP or ISIS or LDP or oror which fails, my whole
system is dead anyhow, router isn't multiservice multitenant in most
cases, all pieces are needed and any piece failure is total failure.

Hopefully Juniper avoids the pitfalls with Evolved, I'm not sure how.
It seems like a complex problem to me. Having own DSL which compiles
with many safety guarantees, to tune of rust, perhaps monolithic
design is what you want, perhaps using the kernel facilities for
multiprocess isn't the right solution, I don't think it's
axiomatically true multiprocess is right solution.



--
++ytti
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

> From: Saku Ytti <saku@ytti.fi>
> Sent: Thursday, September 19, 2019 1:32 PM
>
> On Thu, 19 Sep 2019 at 15:11, <adamv0025@netconsultings.com> wrote:
>
> > Ideally I'd like to see equivalent of Cisco's dynamic update peer-groups in
> Junos.
>
> They are dynamic, but once you make export change which affects subset of
> members in peer-group, that member gets reset while placed to new
> update-group.
>
The dynamic in Cisco implementation means that peers are automatically placed to update groups based on commonalities in export policies, regardless of the configuration.
In juniper case you can actually have two peers with exact same export policies be part of different update groups just because they have been configured that way -which is inefficient, but who cares throw more CPU at it and you're done -real operational problem is the meaningless peer reset.

> > Right this inconsistency between configured and operational state in my
> opinion is THE biggest problem of XR, I'm afraid it has to be something
> fundamental since they haven't been able to consistently address these
> inconsistencies across the board for years now (or ASR9k HW? Not sure if
> these types of issues can be experienced on other platforms).
>
> It is fundamental with non-monolithic design, because processA has no
> access to the memory of processB you need IPC, which for many things is too
> slow, so you replicate state and sync the state, which increases complexity
> and adds bugs.
True but the argument can go both ways -in case of common memory space there's no protection so one needs to worry about leaking into different parts of memory.

> The argument against monolithic design is that single component failure will
> bring the whole house of cards down. I think this is debatable, if it's BGP or
> ISIS or LDP or oror which fails, my whole system is dead anyhow, router isn't
> multiservice multitenant in most cases, all pieces are needed and any piece
> failure is total failure.
>
Touche, you're right in routers for control plane functions is mostly all or nothing.
Management plane is different, but that one is separate in both Junos and XR.

Then there's the flexibility aspect, for example being able to run multiple completely separate instances of BGP would be useful in some cases.

> Hopefully Juniper avoids the pitfalls with Evolved, I'm not sure how.
> It seems like a complex problem to me. Having own DSL which compiles with
> many safety guarantees, to tune of rust, perhaps monolithic design is what
> you want, perhaps using the kernel facilities for multiprocess isn't the right
> solution, I don't think it's axiomatically true multiprocess is right solution.
>
Hmm, but we wouldn’t second guess this multi-process approach for desktop/mobile operating systems right?

adam

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

On Thu, 19 Sep 2019 at 16:29, <adamv0025@netconsultings.com> wrote:

> The dynamic in Cisco implementation means that peers are automatically placed to update groups based on commonalities in export policies, regardless of the configuration.
> In juniper case you can actually have two peers with exact same export policies be part of different update groups just because they have been configured that way -which is inefficient, but who cares throw more CPU at it and you're done -real operational problem is the meaningless peer reset.

I'm not sure I've seen this, but I'm ready to accept it to be true,
but seems like something that could get PR and get fixed. Do you have
a particular example where neighbours which will export the same set
of routes and have same behaviour get separate update-groups?

> True but the argument can go both ways -in case of common memory space there's no protection so one needs to worry about leaking into different parts of memory.

Yes, the whole house of cards effect, which is very relevant in
multitenant time-shared systems like Linux server, just because sshd
crashes doesn't mean httpd should suffer. But I'm not sure if it's so
important in router.

> Hmm, but we wouldn’t second guess this multi-process approach for desktop/mobile operating systems right?

They are much less controlled system, you install nillywilly random
applications to your phone, and it's not good if phone crashes just
because some background app I already forgot crashed. And servers have
multiple tenants. I think in most cases routers have very different
profile, vendor knows exactly what is running there and it's exactly
one use-case.
Note I'm not saying multiprocess is worse than monolithic, I'm just
saying, it's not obvious to me multiprocess is the superior in this
use-case. Certainly C has exuberated the problem as it makes it too
easy to write memory unsafe code, but considering how long lived NOS
are, I think every NOS should be written in its own DSL which compiles
to C++ or Rust. Then you have more choices where given problem is best
solved.

Like forwarding code, should we write it in C or P4 which compiles to
whatever language the NPU can consume? Less capable developers will be
able to write good code in P4 and can communicate to the smaller set
of developers who write the P4 compiler about missing features. And if
the P4 compiler developer notices some nasty hacks the P4 coders need
to write to solve some problem they can move the solution to the
compiler and have the P4 coders write something less hacky.
I think DSL as an abstraction is justified in any complex long lived project.


--
++ytti
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

>
> > Ideally I'd like to see equivalent of Cisco's dynamic update peer-groups
> in Junos.
>
> They are dynamic, but once you make export change which affects subset
> of members in peer-group, that member gets reset while placed to new
> update-group.


And that is how dynamic update groups works by design. They automatically
group peers based on their export policy configuration.

To reset or not reset the peers really depends on the nature of the policy
change. In some cases clear soft out or in will do just fine without
bringing down the session. Of course if you enable soft reconfig in and
policy is just a new import filter no reset of the peer is required if OS
still does it they can fix it easily.

R.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

> From: Saku Ytti <saku@ytti.fi>
> Sent: Thursday, September 19, 2019 2:42 PM
>
> On Thu, 19 Sep 2019 at 16:29, <adamv0025@netconsultings.com> wrote:
>
> > The dynamic in Cisco implementation means that peers are automatically
> placed to update groups based on commonalities in export policies,
> regardless of the configuration.
> > In juniper case you can actually have two peers with exact same export
> policies be part of different update groups just because they have been
> configured that way -which is inefficient, but who cares throw more CPU at it
> and you're done -real operational problem is the meaningless peer reset.
>
> I'm not sure I've seen this, but I'm ready to accept it to be true, but seems
> like something that could get PR and get fixed. Do you have a particular
> example where neighbours which will export the same set of routes and
> have same behaviour get separate update-groups?
>
+ group test-group1 {
+ type external;
+ family inet {
+ unicast;
+ }
+ peer-as 65000;
+ neighbor 192.0.2.10;
+ }
+ group test-group2 {
+ type external;
+ family inet {
+ unicast;
+ }
+ peer-as 65000;
+ neighbor 192.0.2.20;
+ }
+ group test-group3 {
+ type internal;
+ family inet {
+ unicast;
+ }
+ neighbor 192.0.2.30;
+ }
+ group test-group4 {
+ type internal;
+ family inet {
+ unicast;
+ }
+ neighbor 192.0.2.40
+ neighbor 192.0.2.41;
+ }

Group Type: External Local AS: 31655
Name: test-group1 Index: 8 Flags: <>
Holdtime: 0
Total peers: 1 Established: 0
192.0.2.10

Group Type: External Local AS: 31655
Name: test-group2 Index: 9 Flags: <>
Holdtime: 0
Total peers: 1 Established: 0
192.0.2.20

Group Type: Internal AS: 31655 Local AS: 31655
Name: test-group3 Index: 10 Flags: <>
Holdtime: 0
Total peers: 1 Established: 0
192.0.2.30

Group Type: Internal AS: 31655 Local AS: 31655
Name: test-group4 Index: 11 Flags: <>
Holdtime: 0
Total peers: 2 Established: 0
192.0.2.40
192.0.2.41

In cisco these should be under common update group

What is dynamic is that the peer 192.0.2.41 will be kicked out of group 11 if configured with different export policy
And reset in that process unfortunately

set protocols bgp group test-group4 neighbor 192.0.2.41 export INTERNET_EX

Group Type: Internal AS: 31655 Local AS: 31655
Name: test-group4 Index: 12 Flags: <>
Export: [ INTERNET_EX ]
Holdtime: 0
Total peers: 1 Established: 0
192.0.2.41



> > True but the argument can go both ways -in case of common memory
> space there's no protection so one needs to worry about leaking into
> different parts of memory.
>
> Yes, the whole house of cards effect, which is very relevant in multitenant
> time-shared systems like Linux server, just because sshd crashes doesn't
> mean httpd should suffer. But I'm not sure if it's so important in router.
>
> > Hmm, but we wouldn’t second guess this multi-process approach for
> desktop/mobile operating systems right?
>
> They are much less controlled system, you install nillywilly random
> applications to your phone, and it's not good if phone crashes just because
> some background app I already forgot crashed. And servers have multiple
> tenants. I think in most cases routers have very different profile, vendor
> knows exactly what is running there and it's exactly one use-case.
> Note I'm not saying multiprocess is worse than monolithic, I'm just saying, it's
> not obvious to me multiprocess is the superior in this use-case. Certainly C
> has exuberated the problem as it makes it too easy to write memory unsafe
> code, but considering how long lived NOS are, I think every NOS should be
> written in its own DSL which compiles to C++ or Rust. Then you have more
> choices where given problem is best solved.
>
> Like forwarding code, should we write it in C or P4 which compiles to
> whatever language the NPU can consume? Less capable developers will be
> able to write good code in P4 and can communicate to the smaller set of
> developers who write the P4 compiler about missing features. And if the P4
> compiler developer notices some nasty hacks the P4 coders need to write to
> solve some problem they can move the solution to the compiler and have
> the P4 coders write something less hacky.
> I think DSL as an abstraction is justified in any complex long lived project.
>
Your point on all or nothing and single use-case nature of NOSes is certainly interesting. I'll ponder it some more.

adam

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

On Thu, 19 Sep 2019 at 18:36, <adamv0025@netconsultings.com> wrote:

> In cisco these should be under common update group

This is a fair point. I've not tried to create unique peer-groups with
non-unique policies so this has escaped me.

> What is dynamic is that the peer 192.0.2.41 will be kicked out of group 11 if configured with different export policy
> And reset in that process unfortunately

Quite, so I guess to be eligible for update group junos has two
constraints and ios has one, junos constraints are same egress-policy
and same peer-group, ios constraints same egress-policy. I can
understand thne rationale to both, junos rationale makes sense in a
context where you want to force update-group separation when you know
the two sets are not guaranteed to have the same policy every time.

Of course the ideal solution would be to never reset and use as few
update groups as theoretically possible. But for me as an operator,
both behaviour are good enough and don't really bite operationally, as
I can design configuration around it.

--
++ytti
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Thu, Sep 19, 2019 at 06:42:58PM +0300, Saku Ytti:
> Quite, so I guess to be eligible for update group junos has two
> constraints and ios has one, junos constraints are same egress-policy
> and same peer-group, ios constraints same egress-policy. I can
> understand thne rationale to both, junos rationale makes sense in a
> context where you want to force update-group separation when you know
> the two sets are not guaranteed to have the same policy every time.

And, why wouldn't you want that control?
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

On 9/19/19, 1:05 AM, "juniper-nsp on behalf of Phil Reilly" <juniper-nsp-bounces@puck.nether.net on behalf of philr@jaspers.co.nz> wrote:

MX104's are the dual brain unit of the 204. Though a 204 has 40/100G
capabilities. If I read your original request correctly about ip
routing. Not sure the 104/204 is grunty enough to deal with multiple
internet tables. Thats a demanding task these days best left to the
larger chassis.

The MX204 should work fine with multiple full table peers. The MX104 is the dual-RE version of the MX80 PPC-powered router. The MX204 is essentially an MPC7E, has a 64-bit Junos that runs the OS in a VM and has 16GB of RAM. MX204 is pretty powerful for what it is, which is a 1U router with 100G/40G/10G/1G capabilities, but you cannot use all ports simultaneously if you enable 100G or 40G. It's a bit convoluted, but it's explained here:

https://www.juniper.net/documentation/en_US/junos/topics/concept/port-speed-capability-mx204router.html

They offer a port-checker tool to verify configurations here:

https://apps.juniper.net/home/port-checker/index.html

-evt

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

I have to play devils advocate around “Right this inconsistency between configured and operational state in my opinion is THE biggest problem of XR”

Have had this problem occur multiple times in Junos, as recently as Junos 17, where what was being advertised did not reflect configured policy. Worse still, the only recovery was complete deletion of the policy (and any peers using it) then re-adding it.

So it’s definitely not a XR only problem.

Sent from my iPhone

On Sep 19, 2019, at 8:11 AM, "adamv0025@netconsultings.com" <adamv0025@netconsultings.com> wrote:

>> From: Saku Ytti <saku@ytti.fi>
>> Sent: Thursday, September 19, 2019 12:33 PM
>>
>>> On Thu, 19 Sep 2019 at 14:22, <adamv0025@netconsultings.com> wrote:
>>>
>>> Just a few examples when you change export policy it resets the peer
>>> or the cockup with RR clearing all sessions or the fact BGP is part of
>>> very complex RDP monolith -to me that's not really "carrier grade"
>>> implementation
>>
>> This happens when export policy breaks update-group. It may sometimes be
>> difficult for operator to understand if it will do that or not, so it's fair concern.
>> Perhaps system should not clear, but tell manual clear is needed for policy
>> change to take effect.
>>
> Ideally I'd like to see equivalent of Cisco's dynamic update peer-groups in Junos.
>
>> If monolith is good or bad, I'm not sure. If you thread you have high
>> performance with some risk. If you have process separation you have IPC
>> problem, and you have low performance and many will solve this by
>> duplicating state. Junos is moving towards multi process model with Junos
>> Evolved, if this will be positive or negative direction remains to be seen.
>>
> I like where XR and Junos Evolved is heading,
> In future I'd like to have the option to install only stuff I need on a particular type of node/deployment and not worry about the rest all the way to being able to mix and match protocols of different vendors.
> Although cRPD is also interesting development pathway, but again cBGP would be even better :)
>
>> Operationally speaking, BGP in JunOS for us works great, on IOS-XR right now
>> we have sessions where policy isn't what is configured and there is no way to
>> verify which one, and we've propagated leaks because acting configuration
>> isn't the one we've configured. We've not had similar problems in JunOS. This
>> is anecdote, not data.
>>
> Right this inconsistency between configured and operational state in my opinion is THE biggest problem of XR, I'm afraid it has to be something fundamental since they haven't been able to consistently address these inconsistencies across the board for years now (or ASR9k HW? Not sure if these types of issues can be experienced on other platforms).
> Though usually it's CP state does not trickle down to DP correctly/completely where what you described seems to be CP only.
>
> adam
>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

On 19/Sep/19 00:52, Jason Lixfeld wrote:

> FWIW, you may want to check out Arista’s 7280R. We’ve just deployed a pair of these for EVPN-MPLS and they’re slick, and from what I understand, they have the FIB scale to be able to act as a border router. It’s a very IOS-like CLI (but so many things about the CLI are so much more refined than IOS) so it may be more familiar, unless you’re Cisco experience is limited to IOS-XR. It’s about USD$50K list for 48 x SFP+ / 6 x 40/100G, including licensing.
>
> It’s a BCM Jericho based pizza box, so that’s redundant powered, but not “redundant” in so far as there are no redundant supervisor/management cards. But, for the number of times I’ve had that kind of failure on any of my boxes that have had redundant cards, I don’t think it’s worth the cost or the rack space, especially if it’s just a border router where you’ve probably got a bunch of other border router that can accommodate a crash or a reboot or whatever.

We recently migrated our Juniper Ethernet switches over to Arista (Layer
2-only aggregation).

We hit an issue where policing did not work, despite being activated. We
then realized we had to explicitly enable "l2 qos" for our TCAM profile.
This is traffic-affecting. You then verify by bumping the hardware ACL
counters.

Now, where this gets hairy, is when you update the TCAM profile to
enable policing, all TCAM resources are then exhausted because the
default slicing of the TCAM in EOS shares it across various protocols
and features, including ACL's, MPLS, IPv6, non-IP, PBR, pcap, e.t.c. So
in order to support policing, you have to give up some of these features

Luckily for us, this is a pure Layer 2-only device, so we don't need a
bunch of these things; just ACL's to protect terminal sessions. We are
now going to test this again and hope nothing else breaks.

Definitely not something we were expecting, and a bit of a surprise for
the Arista TAC too.

Takes me back several steps with merchant silicon... ah well.

Mark.

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Hi,

On Mon, Sep 23, 2019 at 09:15:48AM +0200, Mark Tinka wrote:
> We hit an issue where policing did not work, despite being activated. We
> then realized we had to explicitly enable "l2 qos" for our TCAM profile.
> This is traffic-affecting. You then verify by bumping the hardware ACL
> counters.
>
> Now, where this gets hairy, is when you update the TCAM profile to
> enable policing, all TCAM resources are then exhausted because the
> default slicing of the TCAM in EOS shares it across various protocols
> and features, including ACL's, MPLS, IPv6, non-IP, PBR, pcap, e.t.c. So
> in order to support policing, you have to give up some of these features

Which box was this? Trident or Jericho?

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

On 23/Sep/19 09:23, Gert Doering wrote:

>
> Which box was this? Trident or Jericho?

7280R, Jericho.

Mark.

Hi,

On Mon, Sep 23, 2019 at 09:28:38AM +0200, Mark Tinka wrote:
> > Which box was this? Trident or Jericho?
> 7280R, Jericho.

Ewww... thanks for the heads up. I have one of those "incoming" as
new "full table, many features, external links" L3 device, and that
one will definitely need QoS + netflow/ipfix + L3...

Will test very thoroughly :-)

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

On 23/Sep/19 09:44, Gert Doering wrote:

> Ewww... thanks for the heads up. I have one of those "incoming" as
> new "full table, many features, external links" L3 device, and that
> one will definitely need QoS + netflow/ipfix + L3...
>
> Will test very thoroughly :-)

I'll also let you know once we are done lab'ing Arista's solution to all
of this.

Of course, if you are doing IP/MPLS on this box, you will have way more
considerations about this than we will.

Mark.

Hi,

I'd like to point out one more thing because I feel that this point hasn't been stressed enough:
Upgrading Junos might be more time consuming than many people expect it to be.

The reason for this is that quite often, things that previously worked in Junos will break in a new release. This affects very, very basic things too. Even if you don't do magic fairy-stuff like EVPN-MPLS to EVPN-VXLAN handoff or any MPLS at all, you might get bitten by this at some point.

One example:
After an upgrade we noticed that local-preference just stopped working. After investigating this, we realized that traffic for some subnets was being diverted, while for others it was still following the old path. Reason for this was that we had BGP PIC enabled. We had to downgrade to another version because BGP PIC is important to us.

Another, more nasty example of very basic things failing after an upgrade is this:
We had packet loss on an IBGP link due to a dirty fiber. No problem, we run LAGs everywhere, so we just disabled the link and send a technician to the datacenter to clean the fiber. However, at the moment when we disabled the dirty fiber some customers went completely offline. At this point a minor incident had turned into a major one because customers were feeling the impact and we started receiving calls from them. However, the routers did not show any errors on either end. LACP was still working fine on the three other links, BGP was still fine, lots of traffic was being forwarded. But we were dropping packets somewhere, some customers were offline and network engineers couldn't tell their boss what was wrong. Not a very pleasant situation to be in.
In the end, we found out that the last upgrade introduced a new bug: If you disable an interface that is part of a LAG, Junos would still hash traffic on to that interface, so with a LAG consisting of 4 interfaces where you disabled one, a quater of the traffic would just be blackholed.

Be ready to put up with shit like this if you buy Juniper. The officially recommended Juniper releases won't save you either: They too contain newly introduced bugs that break things that previously worked flawlessly. This is the main problem I have with Juniper: If you upgrade, you might spend days debugging stuff that used to work flawlessly for years. Even the most basic things like LAGs aren't safe. You might think that you found a version that works for you, but then, weeks later, you find something that got broken with the upgrade and then you need to schedule a new upgrade/downgrade.

The big companies have fancy and expensive labs and employees that spend weeks testing new releases. However, we're a small hosting provider running a bunch of MX480ies and other Juniper stuff. I need routers that I can upgrade without fearing that my network will explode. Can't have that with Juniper.

If I were to rebuild our network again, I'd take a very good look at Arista/ANET. As Saku already mentioned, they're the ones that have the best practices in developing software. They might not have all the bells and whistles that Juniper have, but at least I might get more sleep and peace of mind when upgrading those than I got with my Juniper gear.

Regards
Karl




On 19.09.19 09:09, Gert Doering wrote:
> Hi,
>
> On Thu, Sep 19, 2019 at 05:04:54PM +1200, Phil Reilly wrote:
>> MX104's are the dual brain unit of the 204. Though a 204 has 40/100G
>> capabilities. If I read your original request correctly about ip
>> routing. Not sure the 104/204 is grunty enough to deal with multiple
>> internet tables. Thats a demanding task these days best left to the
>> larger chassis.
> You can't really compare MX104 to MX204.
>
> The MX104 is ppc based and *slow*, and should have never ever shipped.
>
> The MX204 is a really really nice box, with a fast intel RE and 40/100G
> ports (though some - documented - restrictions on how they can be combined),
> and from a RE/BGP point of view, en par with the larger MXes.
>
> And given the price point of the MX204, if the amount of interfaces is
> sufficient, just get two of them :-)
>
> gert
>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

On 23/Sep/19 10:58, Karl Gerhard wrote:

>
> The big companies have fancy and expensive labs and employees that spend weeks testing new releases. However, we're a small hosting provider running a bunch of MX480ies and other Juniper stuff. I need routers that I can upgrade without fearing that my network will explode. Can't have that with Juniper.

If you've been doing this for as long as many of us on here have, you'll
note that what you're experiencing with Juniper is not unique to them.

I've been running IOS since 10.

I've been running Junos since 7.

I've been running IOS XR since 3.

I've been running FreeBSD since 4.

I've been running SuSE Linux since 5.

I've been running Windows since 3.

There is always a risk with doing upgrades, both with basic and exotic
features. That's why testing as much and for as long as you can before
the activity is not to be underestimated. And even then, you'll still
get caught out. This is not about to change, but will get slightly better.

Saku and others will tell you that QA for Juniper got really bad between
10 and 15, and only started to get better again around 16 onward. But
also, they moved away from a single-OS marketing line to forking for
various platforms. This is to be expected, and don't expect Arista to
avoid this as they grow.


>
> If I were to rebuild our network again, I'd take a very good look at Arista/ANET. As Saku already mentioned, they're the ones that have the best practices in developing software. They might not have all the bells and whistles that Juniper have, but at least I might get more sleep and peace of mind when upgrading those than I got with my Juniper gear.

See my previous post about our ongoing Arista woes (and this is only in
a Layer 2 application). Point is, you can't escape it, whichever vendor
you go for, especially when all the new ones on the block are running
merchant silicon and all the pleasure & joy that brings.

For me, the biggest thing I enjoy about Junos vs. IOS XR is the upgrade
process. This is the major driving reason behind us avoiding the NCS540
for the Metro. Too many devices to upgrade the XR way.

Mark.

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp