Mailing List Archive

1 2  View All
Re: How to pick JUNOS Version [ In reply to ]
1 - https://kb.juniper.net/InfoCenter/index?page=content&id=KB33988
(pretty sure that avoids the usb craziness, if memory serves me right, I
think Juniper created that KB from my lab 5048 test back in March 2019. Now
I'm wondering if I tried to go to like 15.x.D61 or something prior to trying
to go to 17.3 if that would've helped)

2 - Virtual Chassis on an ACX5048? Can you do that?

-Aaron

-----Original Message-----
From: Kody Vicknair <kvicknair@reservetele.com>
Sent: Tuesday, September 1, 2020 7:52 PM
To: aaron1@gvtc.com; 'Roger Wiklund' <roger.wiklund@gmail.com>; 'Colton
Conor' <colton.conor@gmail.com>
Cc: 'Juniper List' <juniper-nsp@puck.nether.net>
Subject: RE: [j-nsp] How to pick JUNOS Version

1. I did a direct USB recovery format boot from 17.4r2-S11, and afterwards
recovered configs from FTP. The reason for this was because 15.1x54D51
screws up your /tmp directory and you wont have enough free space no matter
what you try. - at least, this was the case for my installs

2. One caveat of mgmt_junos routing-instance is you wont be able to utilize
vme interfaces (so if you have a virtual chassis, it still won't work), to
get around this I did the following:

set system management-instance
rename interfaces vme.0 to em0.0
set routing-instances mgmt_junos description "OOB Mgmt Instance"
set routing-instances mgmt_junos routing-options static route 0.0.0.0/0
next-hop x.x.x.x set snmp interface em0.0 set snmp routing-instance-access
set system ntp server x.x.x.x routing-instance mgmt_junos set system ntp
server x.x.x.x routing-instance mgmt_junos

and deleted the static route for management out of the default inet.0 table

Goodluck!

-KV


-----Original Message-----
From: aaron1@gvtc.com <aaron1@gvtc.com>
Sent: Tuesday, September 1, 2020 5:34 PM
To: Kody Vicknair <kvicknair@reservetele.com>; 'Roger Wiklund'
<roger.wiklund@gmail.com>; 'Colton Conor' <colton.conor@gmail.com>
Cc: 'Juniper List' <juniper-nsp@puck.nether.net>
Subject: RE: [j-nsp] How to pick JUNOS Version

*External Email: Use Caution*

Thanks Kody, 2 questions sir... I recently began moving towards that same
version (17.4R2-S11) as I was hitting PR1419761 high cpu.

1 - did you upgrade straight from 15.1x54D51 to 17.4R2-S11 , or did you take
an intermediate step? Asking since JTAC recently told me that this was too
far of a jump and I should go through 17.1 on my way to 17.4
15.1X54--------17.1----------17.4

2 - do you use that mgmt_junos route instance? I would've expected to see
em0 or fxp0 in there or something like that, but I don't.

name@acx5048> show system information | grep Junos
Family: junos
Junos: 17.4R2-S11

name@acx5048> show route instance mgmt_junos detail
mgmt_junos:
Router ID: 0.0.0.0
Type: forwarding State: Active

-Aaron



_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: How to pick JUNOS Version [ In reply to ]
Aaron,

1. yes, I've seen this kb, but when I had someone I trust look at the diff, + the nature of the changes they were suggesting, I just wasn't comfortable making those changes. I personally avoided this at all cost. In my case, installs took about ~7 minutes per acx in a maintenance window and a majority of the routers are local, so it wasn't a big deal.

2. Not that I'm aware. I was including the caveat because for some reason during initial config we used vme.0 interfaces instead of em0.0 and I ran into issues with it not working properly in a lab setting and vme.0 was the gotcha for me.

By the way, I was seeing the same PR1419761 issues you are seeing, those have been resolved since upgrading.

-kv


-----Original Message-----
From: aaron1@gvtc.com <aaron1@gvtc.com>
Sent: Wednesday, September 2, 2020 1:22 AM
To: Kody Vicknair <kvicknair@reservetele.com>; 'Roger Wiklund' <roger.wiklund@gmail.com>; 'Colton Conor' <colton.conor@gmail.com>
Cc: 'Juniper List' <juniper-nsp@puck.nether.net>
Subject: RE: [j-nsp] How to pick JUNOS Version

*External Email: Use Caution*

1 - https://link.edgepilot.com/s/6890e15f/hA34W4NyWUumQulQkNIdtw?u=https://kb.juniper.net/InfoCenter/index?page=content%26id=KB33988
(pretty sure that avoids the usb craziness, if memory serves me right, I think Juniper created that KB from my lab 5048 test back in March 2019. Now I'm wondering if I tried to go to like 15.x.D61 or something prior to trying to go to 17.3 if that would've helped)

2 - Virtual Chassis on an ACX5048? Can you do that?

-Aaron

-----Original Message-----
From: Kody Vicknair <kvicknair@reservetele.com>
Sent: Tuesday, September 1, 2020 7:52 PM
To: aaron1@gvtc.com; 'Roger Wiklund' <roger.wiklund@gmail.com>; 'Colton Conor' <colton.conor@gmail.com>
Cc: 'Juniper List' <juniper-nsp@puck.nether.net>
Subject: RE: [j-nsp] How to pick JUNOS Version

1. I did a direct USB recovery format boot from 17.4r2-S11, and afterwards recovered configs from FTP. The reason for this was because 15.1x54D51 screws up your /tmp directory and you wont have enough free space no matter what you try. - at least, this was the case for my installs

2. One caveat of mgmt_junos routing-instance is you wont be able to utilize vme interfaces (so if you have a virtual chassis, it still won't work), to get around this I did the following:

set system management-instance
rename interfaces vme.0 to em0.0
set routing-instances mgmt_junos description "OOB Mgmt Instance"
set routing-instances mgmt_junos routing-options static route 0.0.0.0/0 next-hop x.x.x.x set snmp interface em0.0 set snmp routing-instance-access set system ntp server x.x.x.x routing-instance mgmt_junos set system ntp server x.x.x.x routing-instance mgmt_junos

and deleted the static route for management out of the default inet.0 table

Goodluck!

-KV


-----Original Message-----
From: aaron1@gvtc.com <aaron1@gvtc.com>
Sent: Tuesday, September 1, 2020 5:34 PM
To: Kody Vicknair <kvicknair@reservetele.com>; 'Roger Wiklund'
<roger.wiklund@gmail.com>; 'Colton Conor' <colton.conor@gmail.com>
Cc: 'Juniper List' <juniper-nsp@puck.nether.net>
Subject: RE: [j-nsp] How to pick JUNOS Version

*External Email: Use Caution*

Thanks Kody, 2 questions sir... I recently began moving towards that same version (17.4R2-S11) as I was hitting PR1419761 high cpu.

1 - did you upgrade straight from 15.1x54D51 to 17.4R2-S11 , or did you take an intermediate step? Asking since JTAC recently told me that this was too far of a jump and I should go through 17.1 on my way to 17.4
15.1X54--------17.1----------17.4

2 - do you use that mgmt_junos route instance? I would've expected to see
em0 or fxp0 in there or something like that, but I don't.

name@acx5048> show system information | grep Junos
Family: junos
Junos: 17.4R2-S11

name@acx5048> show route instance mgmt_junos detail
mgmt_junos:
Router ID: 0.0.0.0
Type: forwarding State: Active

-Aaron





Links contained in this email have been replaced. If you click on a link in the email above, the link will be analyzed for known threats. If a known threat is found, you will not be able to proceed to the destination. If suspicious content is detected, you will see a warning.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: How to pick JUNOS Version [ In reply to ]
So - been giving kinda a fair bit of thought to the original question about how one picks their junos version.

I think - for me it comes down to a whole range of variables - and a lot of them relate to resources among other things.


1. Don't upgrade if you don't need to - if there is absolutely nothing you need in the new code - and no serious fixes - don't fix what aint broke

That's rule number #1

Then - it gets a little more difficult - well in our case anyway - we do it this like:


1. Join the beta program - and every beta that comes out - the first thing we do is run the production configs through it in the lab - make damn sure everything we have CURRENTLY is working - if it aint - file the bug reports and try and get it fixed before release
2. Start looking at the new features - decide what may be useful - if anything - and start testing to that to death - again preferably before release so that the fixes can be in when it is released
3. If there are no new features that we see that are of interest - skip the release - though we tend to be pretty early adopters of a lot of tech - so we upgrade pretty frequently.

The con of the approach we take - is that it requires resources - both time and human resource.

The advantage of it is - we know exactly where the trip points are going into each release - we know what we can live with and what we can't. I've also found that it develops your skills immensely - because when you're working with stuff pre-release its not like you can go running to jtac - ideally you wanna be able to get to the point where you can identify the bug - and go as low level as possible so when you submit your reports you can point to the exact case and say - this is where there is a problem. The net result of this is that you learn one hell of a lot about the devices and the protocols in this process.

I realize that what I've said above is almost certainly not applicable to most companies - but it is how we approach things - get it early - test it to death in the lab - and then decide if its worth deploying. I'd also say - the more people who go out there and test early - the better advantage for everyone else - because the fact is an operator is going to always find things and see things that a vendor probably won't - purely because of diversity of the networks and how much stuff the operators are likely to throw at the box.

Anyway - that's the kinda approach we like to take.

Thanks

Andrew


From: juniper-nsp <juniper-nsp-bounces@puck.nether.net> On Behalf Of aaron1@gvtc.com
Sent: Wednesday, 2 September 2020 01:34
To: 'Kody Vicknair' <kvicknair@reservetele.com>; 'Roger Wiklund' <roger.wiklund@gmail.com>; 'Colton Conor' <colton.conor@gmail.com>
Cc: 'Juniper List' <juniper-nsp@puck.nether.net>
Subject: Re: [j-nsp] How to pick JUNOS Version

Thanks Kody, 2 questions sir... I recently began moving towards that same
version (17.4R2-S11) as I was hitting PR1419761 high cpu.

1 - did you upgrade straight from 15.1x54D51 to 17.4R2-S11 , or did you take
an intermediate step? Asking since JTAC recently told me that this was too
far of a jump and I should go through 17.1 on my way to 17.4
15.1X54--------17.1----------17.4

2 - do you use that mgmt_junos route instance? I would've expected to see
em0 or fxp0 in there or something like that, but I don't.

name@acx5048> show system information | grep Junos
Family: junos
Junos: 17.4R2-S11

name@acx5048> show route instance mgmt_junos detail
mgmt_junos:
Router ID: 0.0.0.0
Type: forwarding State: Active

-Aaron


_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp<https://puck.nether.net/mailman/listinfo/juniper-nsp>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: How to pick JUNOS Version [ In reply to ]
On Wed, 2 Sep 2020 at 10:23, Andrew Alston
<Andrew.Alston@liquidtelecom.com> wrote:

> 2. Start looking at the new features - decide what may be useful - if anything - and start testing to that to death - again preferably before release so that the fixes can be in when it is released

How do people measure this? Vendors spend tens or hundreds millions
annually on testing, and still deliver absolute trash NOS, to every
vendor, and there is no change that I can observe +20 years in
quality. Basic things are broken, and everyone finds new fundamental
bugs all the time.

I think NOS are shit, because shit NOS is a good business case and
good NOS is a bad business case, I know it sounds outrageous, but let
me explain. Vendor revenue is support contract, not HW sales. And a
lot of us don't need help on configuring or troubleshooting, a lot of
us have access to community which outperforms TAC on how to get that
box working. But none of us has access to the code, we can't commit
and push a fix. If the NOS would work, like Windows, Macos or Linux
that you rarely find bugs, a lot of us would opt out from support
contracts, and would just buy spare HW, destroying the vendor's
business case.

I don't think vendors sit in scary skull towers and plans for shit
NOS, I think it's emergent behaviour from how the market is modelled.
And there are ways I think the market could change, but I'm already
venturing too far from the question to explore that topic.



Now when it comes to testing, many claim it is important and it
matters. I'm not convinced. And I don't think people are looking at
this in any formality, it's more like religion, and its utility is to
improve comfort-to-deploy in the organisation, it doesn't do much
towards probability-of- success in my mind. I've worked for companies
who test not at all, companies who boot it in the lab and companies
who had a team doing just testing, and I can't say I've seen different
amounts of TAC cases on software issues.

People who invest lot on testing, and are not comfortable with idea
that value is just 'comfort-to-deploy' (that may be sufficiently
important value), I recommend looking at TAC cases you had which
actually did cause customer outage, then try to evaluate 'was this
reasonable to catch in the lab', try to be honest.
The problem I see, whole NOS quality is shit, it's not so shit that
it's always broken, the problems that manifest require usually more
than one condition, then if you start to do back-of-the-envelope math
on testing everything with every permutations, you will notice no
amount of money can fix the fact that you're limited by
heat-death-of-universe on the wall clock. So now you're venturing into
an area where you gotta choose, what to test and what not to test, and
you don't have nearly enough outages and faults to apply statistical
analysis on it, so you're actually just guessing.
It's religion, which has some utility, but not the utility we think it has.


Note I'm not saying testing wholesale is useless, I'm more saying it
has an exponentially or worse diminishing return. I would say push 1
packet through all your products in the lab, and you're done, you're
as far as you're reasonably gonna get.
And start thinking in terms 'the NOS is shit and I exercise no power
over it', what actions work in that world? Staging pop with real but
outage insensitive subscriptions?



--
++ytti
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: How to pick JUNOS Version [ In reply to ]
Saku,

I think for us - the testing we do is to validate against our own configurations and our own designs. This is not done hand in hand with the vendor - it is done against what we use every day as a starting point - to give us some indication of where the trip points potentially are.

And you are 100% correct - the vendors spend huge amounts testing - but it is impossible for any vendor to test every scenario of every operator out there - which is why I advocate for operators to test against their specific needs - those tests can - and do - show up things occasionally that can be rectified.

That testing as I said - we do in our own labs - against our own templates - and our own methodologies - and its worked pretty well for us so far.

Andrew


-----Original Message-----
From: Saku Ytti <saku@ytti.fi>
Sent: Wednesday, 2 September 2020 10:37
To: Andrew Alston <Andrew.Alston@liquidtelecom.com>
Cc: aaron1@gvtc.com; Kody Vicknair <kvicknair@reservetele.com>; Roger Wiklund <roger.wiklund@gmail.com>; Colton Conor <colton.conor@gmail.com>; Juniper List <juniper-nsp@puck.nether.net>
Subject: Re: [j-nsp] How to pick JUNOS Version

On Wed, 2 Sep 2020 at 10:23, Andrew Alston <Andrew.Alston@liquidtelecom.com> wrote:

> 2. Start looking at the new features - decide what may be useful -
> if anything - and start testing to that to death - again preferably
> before release so that the fixes can be in when it is released

How do people measure this? Vendors spend tens or hundreds millions annually on testing, and still deliver absolute trash NOS, to every vendor, and there is no change that I can observe +20 years in quality. Basic things are broken, and everyone finds new fundamental bugs all the time.

I think NOS are shit, because shit NOS is a good business case and good NOS is a bad business case, I know it sounds outrageous, but let me explain. Vendor revenue is support contract, not HW sales. And a lot of us don't need help on configuring or troubleshooting, a lot of us have access to community which outperforms TAC on how to get that box working. But none of us has access to the code, we can't commit and push a fix. If the NOS would work, like Windows, Macos or Linux that you rarely find bugs, a lot of us would opt out from support contracts, and would just buy spare HW, destroying the vendor's business case.

I don't think vendors sit in scary skull towers and plans for shit NOS, I think it's emergent behaviour from how the market is modelled.
And there are ways I think the market could change, but I'm already venturing too far from the question to explore that topic.



Now when it comes to testing, many claim it is important and it matters. I'm not convinced. And I don't think people are looking at this in any formality, it's more like religion, and its utility is to improve comfort-to-deploy in the organisation, it doesn't do much towards probability-of- success in my mind. I've worked for companies who test not at all, companies who boot it in the lab and companies who had a team doing just testing, and I can't say I've seen different amounts of TAC cases on software issues.

People who invest lot on testing, and are not comfortable with idea that value is just 'comfort-to-deploy' (that may be sufficiently important value), I recommend looking at TAC cases you had which actually did cause customer outage, then try to evaluate 'was this reasonable to catch in the lab', try to be honest.
The problem I see, whole NOS quality is shit, it's not so shit that it's always broken, the problems that manifest require usually more than one condition, then if you start to do back-of-the-envelope math on testing everything with every permutations, you will notice no amount of money can fix the fact that you're limited by heat-death-of-universe on the wall clock. So now you're venturing into an area where you gotta choose, what to test and what not to test, and you don't have nearly enough outages and faults to apply statistical analysis on it, so you're actually just guessing.
It's religion, which has some utility, but not the utility we think it has.


Note I'm not saying testing wholesale is useless, I'm more saying it has an exponentially or worse diminishing return. I would say push 1 packet through all your products in the lab, and you're done, you're as far as you're reasonably gonna get.
And start thinking in terms 'the NOS is shit and I exercise no power over it', what actions work in that world? Staging pop with real but outage insensitive subscriptions?



--
++ytti

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: How to pick JUNOS Version [ In reply to ]
Thanks KV, also, if you just wanted to clear that high cpu thing, you can do this...

show system processes extensive | except 0.0

* get the process ID of the fxpc hogging process...in this case it's 1820

me@acx5048> show system processes extensive | except 0.0
last pid: 7231; load averages: 0.89, 0.93, 0.91 up 516+22:09:03 23:07:23
144 processes: 5 running, 118 sleeping, 21 waiting

Mem: 625M Active, 296M Inact, 471M Wired, 425M Cache, 69M Buf, 32M Free
Swap:

PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND
1820 root 2 -51 -51 1185M 501M select ??? 96.97% fxpc

* NOTE, when you hit enter on the KILL command, cursor may continue to blink, and you might think it didn't take your input, but it did... pings will start to die

start shell user root
(enter root pwd)

kill -9 1820

-Aaron


-----Original Message-----
From: Kody Vicknair <kvicknair@reservetele.com>
Sent: Wednesday, September 2, 2020 2:08 AM
To: aaron1@gvtc.com; 'Roger Wiklund' <roger.wiklund@gmail.com>; 'Colton Conor' <colton.conor@gmail.com>
Cc: 'Juniper List' <juniper-nsp@puck.nether.net>
Subject: RE: [j-nsp] How to pick JUNOS Version

Aaron,

1. yes, I've seen this kb, but when I had someone I trust look at the diff, + the nature of the changes they were suggesting, I just wasn't comfortable making those changes. I personally avoided this at all cost. In my case, installs took about ~7 minutes per acx in a maintenance window and a majority of the routers are local, so it wasn't a big deal.

2. Not that I'm aware. I was including the caveat because for some reason during initial config we used vme.0 interfaces instead of em0.0 and I ran into issues with it not working properly in a lab setting and vme.0 was the gotcha for me.

By the way, I was seeing the same PR1419761 issues you are seeing, those have been resolved since upgrading.

-kv


-----Original Message-----
From: aaron1@gvtc.com <aaron1@gvtc.com>
Sent: Wednesday, September 2, 2020 1:22 AM
To: Kody Vicknair <kvicknair@reservetele.com>; 'Roger Wiklund' <roger.wiklund@gmail.com>; 'Colton Conor' <colton.conor@gmail.com>
Cc: 'Juniper List' <juniper-nsp@puck.nether.net>
Subject: RE: [j-nsp] How to pick JUNOS Version

*External Email: Use Caution*

1 - https://link.edgepilot.com/s/6890e15f/hA34W4NyWUumQulQkNIdtw?u=https://kb.juniper.net/InfoCenter/index?page=content%26id=KB33988
(pretty sure that avoids the usb craziness, if memory serves me right, I think Juniper created that KB from my lab 5048 test back in March 2019. Now I'm wondering if I tried to go to like 15.x.D61 or something prior to trying to go to 17.3 if that would've helped)

2 - Virtual Chassis on an ACX5048? Can you do that?

-Aaron

-----Original Message-----
From: Kody Vicknair <kvicknair@reservetele.com>
Sent: Tuesday, September 1, 2020 7:52 PM
To: aaron1@gvtc.com; 'Roger Wiklund' <roger.wiklund@gmail.com>; 'Colton Conor' <colton.conor@gmail.com>
Cc: 'Juniper List' <juniper-nsp@puck.nether.net>
Subject: RE: [j-nsp] How to pick JUNOS Version

1. I did a direct USB recovery format boot from 17.4r2-S11, and afterwards recovered configs from FTP. The reason for this was because 15.1x54D51 screws up your /tmp directory and you wont have enough free space no matter what you try. - at least, this was the case for my installs

2. One caveat of mgmt_junos routing-instance is you wont be able to utilize vme interfaces (so if you have a virtual chassis, it still won't work), to get around this I did the following:

set system management-instance
rename interfaces vme.0 to em0.0
set routing-instances mgmt_junos description "OOB Mgmt Instance"
set routing-instances mgmt_junos routing-options static route 0.0.0.0/0 next-hop x.x.x.x set snmp interface em0.0 set snmp routing-instance-access set system ntp server x.x.x.x routing-instance mgmt_junos set system ntp server x.x.x.x routing-instance mgmt_junos

and deleted the static route for management out of the default inet.0 table

Goodluck!

-KV


-----Original Message-----
From: aaron1@gvtc.com <aaron1@gvtc.com>
Sent: Tuesday, September 1, 2020 5:34 PM
To: Kody Vicknair <kvicknair@reservetele.com>; 'Roger Wiklund'
<roger.wiklund@gmail.com>; 'Colton Conor' <colton.conor@gmail.com>
Cc: 'Juniper List' <juniper-nsp@puck.nether.net>
Subject: RE: [j-nsp] How to pick JUNOS Version

*External Email: Use Caution*

Thanks Kody, 2 questions sir... I recently began moving towards that same version (17.4R2-S11) as I was hitting PR1419761 high cpu.

1 - did you upgrade straight from 15.1x54D51 to 17.4R2-S11 , or did you take an intermediate step? Asking since JTAC recently told me that this was too far of a jump and I should go through 17.1 on my way to 17.4
15.1X54--------17.1----------17.4

2 - do you use that mgmt_junos route instance? I would've expected to see
em0 or fxp0 in there or something like that, but I don't.

name@acx5048> show system information | grep Junos
Family: junos
Junos: 17.4R2-S11

name@acx5048> show route instance mgmt_junos detail
mgmt_junos:
Router ID: 0.0.0.0
Type: forwarding State: Active

-Aaron





Links contained in this email have been replaced. If you click on a link in the email above, the link will be analyzed for known threats. If a known threat is found, you will not be able to proceed to the destination. If suspicious content is detected, you will see a warning.

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: How to pick JUNOS Version [ In reply to ]
Agreed. I like your philosophy about network software upgrades. if you
don't absolutely need to, then don't. I don't like to change my network if
it's running along just fine.



Another reason for upgrades is that a vendor is no longer going to work with
you because of EoS code. I've had that happen to me, where the TAC won't
talk to you about something since your code is End of Support.



-Aaron



_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: How to pick JUNOS Version [ In reply to ]
Thanks ytti

We still test drive a used car before purchasing, even though, the real test will be how it perform all day long up and down the highway...day after day.

Yeah I don't test at scale for pps, and load of any and every protocol. Geez, that's a lot of testing. I think IXIA and Spirent and others make test platforms to load up your box. I don't do that currently, but have been at places that did. The real test is out in the wild in the environment you put it in... and, as we know, with time, things change, and load changes, and results change.

I hear you though about your frustration with vendors putting out code that have issues.

-Aaron


_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: How to pick JUNOS Version [ In reply to ]
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: How to pick JUNOS Version [ In reply to ]
On 2/Sep/20 09:37, Saku Ytti wrote:

> How do people measure this? Vendors spend tens or hundreds millions
> annually on testing, and still deliver absolute trash NOS, to every
> vendor, and there is no change that I can observe +20 years in
> quality. Basic things are broken, and everyone finds new fundamental
> bugs all the time.
>
> I think NOS are shit, because shit NOS is a good business case and
> good NOS is a bad business case, I know it sounds outrageous, but let
> me explain. Vendor revenue is support contract, not HW sales. And a
> lot of us don't need help on configuring or troubleshooting, a lot of
> us have access to community which outperforms TAC on how to get that
> box working. But none of us has access to the code, we can't commit
> and push a fix. If the NOS would work, like Windows, Macos or Linux
> that you rarely find bugs, a lot of us would opt out from support
> contracts, and would just buy spare HW, destroying the vendor's
> business case.
>
> I don't think vendors sit in scary skull towers and plans for shit
> NOS, I think it's emergent behaviour from how the market is modelled.
> And there are ways I think the market could change, but I'm already
> venturing too far from the question to explore that topic.
>
>
>
> Now when it comes to testing, many claim it is important and it
> matters. I'm not convinced. And I don't think people are looking at
> this in any formality, it's more like religion, and its utility is to
> improve comfort-to-deploy in the organisation, it doesn't do much
> towards probability-of- success in my mind. I've worked for companies
> who test not at all, companies who boot it in the lab and companies
> who had a team doing just testing, and I can't say I've seen different
> amounts of TAC cases on software issues.
>
> People who invest lot on testing, and are not comfortable with idea
> that value is just 'comfort-to-deploy' (that may be sufficiently
> important value), I recommend looking at TAC cases you had which
> actually did cause customer outage, then try to evaluate 'was this
> reasonable to catch in the lab', try to be honest.
> The problem I see, whole NOS quality is shit, it's not so shit that
> it's always broken, the problems that manifest require usually more
> than one condition, then if you start to do back-of-the-envelope math
> on testing everything with every permutations, you will notice no
> amount of money can fix the fact that you're limited by
> heat-death-of-universe on the wall clock. So now you're venturing into
> an area where you gotta choose, what to test and what not to test, and
> you don't have nearly enough outages and faults to apply statistical
> analysis on it, so you're actually just guessing.
> It's religion, which has some utility, but not the utility we think it has.
>
>
> Note I'm not saying testing wholesale is useless, I'm more saying it
> has an exponentially or worse diminishing return. I would say push 1
> packet through all your products in the lab, and you're done, you're
> as far as you're reasonably gonna get.
> And start thinking in terms 'the NOS is shit and I exercise no power
> over it', what actions work in that world? Staging pop with real but
> outage insensitive subscriptions?

100% agreed with everything Saku said here.

As we age, we need to pick our battles, since what's real and important,
begins to reveal itself.

Mark.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: How to pick JUNOS Version [ In reply to ]
> Saku Ytti
> Sent: Wednesday, September 2, 2020 8:37 AM
>
> On Wed, 2 Sep 2020 at 10:23, Andrew Alston
> <Andrew.Alston@liquidtelecom.com> wrote:
>
> > 2. Start looking at the new features - decide what may be useful -
> > if anything - and start testing to that to death - again preferably
> > before release so that the fixes can be in when it is released
>
> How do people measure this? Vendors spend tens or hundreds millions
> annually on testing, and still deliver absolute trash NOS, to every
vendor, and
> there is no change that I can observe +20 years in quality. Basic things
are
> broken, and everyone finds new fundamental bugs all the time.
>
> I think NOS are shit, because shit NOS is a good business case and good
NOS
> is a bad business case, I know it sounds outrageous, but let me explain.
> Vendor revenue is support contract, not HW sales. And a lot of us don't
need
> help on configuring or troubleshooting, a lot of us have access to
community
> which outperforms TAC on how to get that box working. But none of us has
> access to the code, we can't commit and push a fix. If the NOS would work,
> like Windows, Macos or Linux that you rarely find bugs, a lot of us would
opt
> out from support contracts, and would just buy spare HW, destroying the
> vendor's business case.
>
> I don't think vendors sit in scary skull towers and plans for shit NOS, I
think it's
> emergent behaviour from how the market is modelled.
> And there are ways I think the market could change, but I'm already
> venturing too far from the question to explore that topic.
>
>
>
> Now when it comes to testing, many claim it is important and it matters.
I'm
> not convinced. And I don't think people are looking at this in any
formality,
> it's more like religion, and its utility is to improve comfort-to-deploy
in the
> organisation, it doesn't do much towards probability-of- success in my
mind.
> I've worked for companies who test not at all, companies who boot it in
the
> lab and companies who had a team doing just testing, and I can't say I've
> seen different amounts of TAC cases on software issues.
>
> People who invest lot on testing, and are not comfortable with idea that
> value is just 'comfort-to-deploy' (that may be sufficiently important
value), I
> recommend looking at TAC cases you had which actually did cause customer
> outage, then try to evaluate 'was this reasonable to catch in the lab',
try to be
> honest.
> The problem I see, whole NOS quality is shit, it's not so shit that it's
always
> broken, the problems that manifest require usually more than one
condition,
> then if you start to do back-of-the-envelope math on testing everything
with
> every permutations, you will notice no amount of money can fix the fact
that
> you're limited by heat-death-of-universe on the wall clock. So now you're
> venturing into an area where you gotta choose, what to test and what not
to
> test, and you don't have nearly enough outages and faults to apply
statistical
> analysis on it, so you're actually just guessing.
> It's religion, which has some utility, but not the utility we think it
has.
>
>
> Note I'm not saying testing wholesale is useless, I'm more saying it has
an
> exponentially or worse diminishing return. I would say push 1 packet
through
> all your products in the lab, and you're done, you're as far as you're
> reasonably gonna get.
> And start thinking in terms 'the NOS is shit and I exercise no power over
it',
> what actions work in that world? Staging pop with real but outage
insensitive
> subscriptions?
>
>
Food for thoughts indeed,
With regards to the infinite amount of failure cases and inability to test
for all of these,
Actually the name of the game is "what is the minimum number of features you
can get away with while realizing all your business cases".
so while the search space is infinite I'd say the graph starts with a big
and narrow peak followed by an infinitely long tail (x axes = number of all
possible failure cases, y axes= probability)

Also testing can also be divided into functional, performance and scaling
testing.
So you take your minimum number of features you can get away with and
perform
(I advise to perform these as multidimensional testing -all features
combined)

Functional testing
-to see if the features actually work as intended and work when combined
-this is where you'll find some bugs and can adjust your feature-set
accordingly (yes the search space still equals to infinity albeit smaller
infinity than complete search space containing all features)

Performance testing
-only related to HW/HW-related features
-to figure out your pps/bps rate head room -weird stuff happens if you cross
certain thresholds (this shields you from potential known and unknow failure
cases and bugs)

Scaling testing
-related to HW and SW resources of the box
-to figure out your features scale and derive your head room -weird stuff
happens if you cross certain thresholds (this shields you from potential
known and unknow failure cases and bugs)


adam


_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

1 2  View All