Mailing List Archive

Did you check the CAM utilization? It's probably full, perhaps you can
partition it differently.

On Wed, Sep 6, 2017 at 2:30 PM, Brian Rak <brak@gameservers.com> wrote:

> Has anyone else seem problems with the CER's being unable to add routes
> before? We have multiple devices (all 2024C-4X) reporting one or both of
> these messages:
>
> IPv6 Route ADD: CAM entry creation FAILED
> IPv4 Network Route ADD: CAM entry creation FAILED
>
> They're learning a full v4+v6 table, but we are well below the limits
> defined in `sh default values`. From past experience, a reload seems to be
> the only thing that will actually correct this, but that is a pretty
> annoying thing to have to do.
>
> Any ideas for what may be causing this? We're on 5.6.0m right now.
>
> _______________________________________________
> foundry-nsp mailing list
> foundry-nsp@puck.nether.net
> http://puck.nether.net/mailman/listinfo/foundry-nsp




--
Be Well,

Derek Labian

CER's don't have CAM profiles.  From the information I can find, they
don't actually use TCAM anyway.


On 9/6/2017 3:32 PM, Derek wrote:
> Did you check the CAM utilization?  It's probably full, perhaps you
> can partition it differently.
>
> On Wed, Sep 6, 2017 at 2:30 PM, Brian Rak <brak@gameservers.com
> <mailto:brak@gameservers.com>> wrote:
>
> Has anyone else seem problems with the CER's being unable to add
> routes before?  We have multiple devices (all 2024C-4X) reporting
> one or both of these messages:
>
> IPv6 Route ADD: CAM entry creation FAILED
> IPv4 Network Route ADD: CAM entry creation FAILED
>
> They're learning a full v4+v6 table, but we are well below the
> limits defined in `sh default values`.  From past experience, a
> reload seems to be the only thing that will actually correct this,
> but that is a pretty annoying thing to have to do.
>
> Any ideas for what may be causing this?  We're on 5.6.0m right now.
>
> _______________________________________________
> foundry-nsp mailing list
> foundry-nsp@puck.nether.net <mailto:foundry-nsp@puck.nether.net>
> http://puck.nether.net/mailman/listinfo/foundry-nsp
> <http://puck.nether.net/mailman/listinfo/foundry-nsp>
>
>
>
>
> --
> Be Well,
>
> Derek Labian

Yeah, you are right, profiles aren't supported on the CER series. It still
seems your error is related to over-utilization, hence the fix with a
reboot. Check the utilization.



On Wed, Sep 6, 2017 at 2:37 PM, Brian Rak <brak@gameservers.com> wrote:

> CER's don't have CAM profiles. From the information I can find, they
> don't actually use TCAM anyway.
>
> On 9/6/2017 3:32 PM, Derek wrote:
>
> Did you check the CAM utilization? It's probably full, perhaps you can
> partition it differently.
>
> On Wed, Sep 6, 2017 at 2:30 PM, Brian Rak <brak@gameservers.com> wrote:
>
>> Has anyone else seem problems with the CER's being unable to add routes
>> before? We have multiple devices (all 2024C-4X) reporting one or both of
>> these messages:
>>
>> IPv6 Route ADD: CAM entry creation FAILED
>> IPv4 Network Route ADD: CAM entry creation FAILED
>>
>> They're learning a full v4+v6 table, but we are well below the limits
>> defined in `sh default values`. From past experience, a reload seems to be
>> the only thing that will actually correct this, but that is a pretty
>> annoying thing to have to do.
>>
>> Any ideas for what may be causing this? We're on 5.6.0m right now.
>>
>> _______________________________________________
>> foundry-nsp mailing list
>> foundry-nsp@puck.nether.net
>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>
>
>
>
> --
> Be Well,
>
> Derek Labian
>
>
>


--
Be Well,

Derek Labian

Been there, suffer from it everyday.

They might not have CAM profiles ala MLXe, but they still have CAMs... so they have limited and counted ressources.

Best regards.



> Le 6 sept. 2017 à 21:37, Brian Rak <brak@gameservers.com> a écrit :
>
> CER's don't have CAM profiles. From the information I can find, they don't actually use TCAM anyway.
>
>> On 9/6/2017 3:32 PM, Derek wrote:
>> Did you check the CAM utilization? It's probably full, perhaps you can partition it differently.
>>
>>> On Wed, Sep 6, 2017 at 2:30 PM, Brian Rak <brak@gameservers.com> wrote:
>>> Has anyone else seem problems with the CER's being unable to add routes before? We have multiple devices (all 2024C-4X) reporting one or both of these messages:
>>>
>>> IPv6 Route ADD: CAM entry creation FAILED
>>> IPv4 Network Route ADD: CAM entry creation FAILED
>>>
>>> They're learning a full v4+v6 table, but we are well below the limits defined in `sh default values`. From past experience, a reload seems to be the only thing that will actually correct this, but that is a pretty annoying thing to have to do.
>>>
>>> Any ideas for what may be causing this? We're on 5.6.0m right now.
>>>
>>> _______________________________________________
>>> foundry-nsp mailing list
>>> foundry-nsp@puck.nether.net
>>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>>
>>
>>
>> --
>> Be Well,
>>
>> Derek Labian
>
> _______________________________________________
> foundry-nsp mailing list
> foundry-nsp@puck.nether.net
> http://puck.nether.net/mailman/listinfo/foundry-nsp

None of the cam-partition commands are even supported on the CER, so I'm
not sure how to even begin to look at utilization.  The only real
commands related to this are 'show cam l4 SLOT/PORT', which doesn't
really tell me a whole lot.

I've checked that our route counts are below what we have defined in
system-max... if there's another limit here it's not well documented.


On 9/6/2017 3:43 PM, Derek wrote:
> Yeah, you are right, profiles aren't supported on the CER series.  It
> still seems your error is related to over-utilization, hence the fix
> with a reboot.  Check the utilization.
>
>
>
> On Wed, Sep 6, 2017 at 2:37 PM, Brian Rak <brak@gameservers.com
> <mailto:brak@gameservers.com>> wrote:
>
> CER's don't have CAM profiles.  From the information I can find,
> they don't actually use TCAM anyway.
>
>
> On 9/6/2017 3:32 PM, Derek wrote:
>> Did you check the CAM utilization? It's probably full, perhaps
>> you can partition it differently.
>>
>> On Wed, Sep 6, 2017 at 2:30 PM, Brian Rak <brak@gameservers.com
>> <mailto:brak@gameservers.com>> wrote:
>>
>> Has anyone else seem problems with the CER's being unable to
>> add routes before?  We have multiple devices (all 2024C-4X)
>> reporting one or both of these messages:
>>
>> IPv6 Route ADD: CAM entry creation FAILED
>> IPv4 Network Route ADD: CAM entry creation FAILED
>>
>> They're learning a full v4+v6 table, but we are well below
>> the limits defined in `sh default values`.  From past
>> experience, a reload seems to be the only thing that will
>> actually correct this, but that is a pretty annoying thing to
>> have to do.
>>
>> Any ideas for what may be causing this?  We're on 5.6.0m
>> right now.
>>
>> _______________________________________________
>> foundry-nsp mailing list
>> foundry-nsp@puck.nether.net <mailto:foundry-nsp@puck.nether.net>
>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>> <http://puck.nether.net/mailman/listinfo/foundry-nsp>
>>
>>
>>
>>
>> --
>> Be Well,
>>
>> Derek Labian
>
>
>
>
> --
> Be Well,
>
> Derek Labian

What sort of resource limits are you running into here?  Did you find
any sort of workaround, or are you just reloading all the time?


On 9/6/2017 3:45 PM, Youssef Bengelloun-Zahr wrote:
> Been there, suffer from it everyday.
>
> They might not have CAM profiles ala MLXe, but they still have CAMs...
> so they have limited and counted ressources.
>
> Best regards.
>
>
>
> Le 6 sept. 2017 à 21:37, Brian Rak <brak@gameservers.com
> <mailto:brak@gameservers.com>> a écrit :
>
>> CER's don't have CAM profiles.  From the information I can find, they
>> don't actually use TCAM anyway.
>>
>>
>> On 9/6/2017 3:32 PM, Derek wrote:
>>> Did you check the CAM utilization?  It's probably full, perhaps you
>>> can partition it differently.
>>>
>>> On Wed, Sep 6, 2017 at 2:30 PM, Brian Rak <brak@gameservers.com
>>> <mailto:brak@gameservers.com>> wrote:
>>>
>>> Has anyone else seem problems with the CER's being unable to add
>>> routes before?  We have multiple devices (all 2024C-4X)
>>> reporting one or both of these messages:
>>>
>>> IPv6 Route ADD: CAM entry creation FAILED
>>> IPv4 Network Route ADD: CAM entry creation FAILED
>>>
>>> They're learning a full v4+v6 table, but we are well below the
>>> limits defined in `sh default values`.  From past experience, a
>>> reload seems to be the only thing that will actually correct
>>> this, but that is a pretty annoying thing to have to do.
>>>
>>> Any ideas for what may be causing this?  We're on 5.6.0m right now.
>>>
>>> _______________________________________________
>>> foundry-nsp mailing list
>>> foundry-nsp@puck.nether.net <mailto:foundry-nsp@puck.nether.net>
>>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>>> <http://puck.nether.net/mailman/listinfo/foundry-nsp>
>>>
>>>
>>>
>>>
>>> --
>>> Be Well,
>>>
>>> Derek Labian
>>
>> _______________________________________________
>> foundry-nsp mailing list
>> foundry-nsp@puck.nether.net <mailto:foundry-nsp@puck.nether.net>
>> http://puck.nether.net/mailman/listinfo/foundry-nsp

On the CER, the equivalent of "show cam-partition usage" is "show
resource". I believe the IP entries include both IPv4 & IPv6.

"show default values" is also useful because you can see if your
"system-max ipv6-route" is high enough - the default is only 8192 and
your IPv6 DFZ table obviously won't fit if your system-max is set to
such a low value. We tend to run it at 262144 because system-max is a
fairly pointless feature anyway, it doesn't carve up your CAM, just
restricts certain features from over-consuming resources that are shared
with different features.

And, of course, after messing around with system-max you're going to
have to reload the device to apply the new settings.

Best regards,
Martijn

On 06-09-17 21:53, Brian Rak wrote:
>
> What sort of resource limits are you running into here? Did you find
> any sort of workaround, or are you just reloading all the time?
>
>
> On 9/6/2017 3:45 PM, Youssef Bengelloun-Zahr wrote:
>> Been there, suffer from it everyday.
>>
>> They might not have CAM profiles ala MLXe, but they still have
>> CAMs... so they have limited and counted ressources.
>>
>> Best regards.
>>
>>
>>
>> Le 6 sept. 2017 à 21:37, Brian Rak <brak@gameservers.com
>> <mailto:brak@gameservers.com>> a écrit :
>>
>>> CER's don't have CAM profiles. From the information I can find,
>>> they don't actually use TCAM anyway.
>>>
>>>
>>> On 9/6/2017 3:32 PM, Derek wrote:
>>>> Did you check the CAM utilization? It's probably full, perhaps you
>>>> can partition it differently.
>>>>
>>>> On Wed, Sep 6, 2017 at 2:30 PM, Brian Rak <brak@gameservers.com
>>>> <mailto:brak@gameservers.com>> wrote:
>>>>
>>>> Has anyone else seem problems with the CER's being unable to
>>>> add routes before? We have multiple devices (all 2024C-4X)
>>>> reporting one or both of these messages:
>>>>
>>>> IPv6 Route ADD: CAM entry creation FAILED
>>>> IPv4 Network Route ADD: CAM entry creation FAILED
>>>>
>>>> They're learning a full v4+v6 table, but we are well below the
>>>> limits defined in `sh default values`. From past experience, a
>>>> reload seems to be the only thing that will actually correct
>>>> this, but that is a pretty annoying thing to have to do.
>>>>
>>>> Any ideas for what may be causing this? We're on 5.6.0m right now.
>>>>
>>>> _______________________________________________
>>>> foundry-nsp mailing list
>>>> foundry-nsp@puck.nether.net <mailto:foundry-nsp@puck.nether.net>
>>>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>>>> <http://puck.nether.net/mailman/listinfo/foundry-nsp>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Be Well,
>>>>
>>>> Derek Labian
>>>
>>> _______________________________________________
>>> foundry-nsp mailing list
>>> foundry-nsp@puck.nether.net <mailto:foundry-nsp@puck.nether.net>
>>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>
>
>
> _______________________________________________
> foundry-nsp mailing list
> foundry-nsp@puck.nether.net
> http://puck.nether.net/mailman/listinfo/foundry-nsp

--
Met vriendelijke groet / Kindest regards,
Martijn Schmidt


i3D.net performance hosting
*Martijn Schmidt | Network Architect*
Email: martijnschmidt@i3d.net <mailto://martijnschmidt@i3d.net> | Tel:
+31 10 8900070

*i3D.net B.V. | Global Backbone AS49544*
Rivium 1e Straat 1, 2909 LE Capelle aan den IJssel, The Netherlands
VAT: NL 8202.63.886.B01

Website
<http://www.i3d.net/?utm_source=emailsignature&utm_medium=email&utm_campaign=home>
| Case Studies
<http://www.i3d.net/partners/?utm_source=emailsignature&utm_medium=email&utm_campaign=case-studies>
| LinkedIn <https://www.linkedin.com/company/i3d-net>

Hmm, thanks!

'show resource' at least gives us a counter of failed allocations.  It
doesn't make a whole lot of sense to me, because we're seeing failed
allocations despite only being ~70% used:

            [IP]1102400(size), 339702(free), 069.18%(used), 4198683(failed)

We're running with a system-max ip cache/route of 1000000 and ivp6
cache/route of 102400.  'sh default values' confirms those are actually
set, we ran into those limits ages ago (and have rebooted many times
since raising them)


On 9/6/2017 4:09 PM, i3D.net - Martijn Schmidt wrote:
> On the CER, the equivalent of "show cam-partition usage" is "show
> resource". I believe the IP entries include both IPv4 & IPv6.
>
> "show default values" is also useful because you can see if your
> "system-max ipv6-route" is high enough - the default is only 8192 and
> your IPv6 DFZ table obviously won't fit if your system-max is set to
> such a low value. We tend to run it at 262144 because system-max is a
> fairly pointless feature anyway, it doesn't carve up your CAM, just
> restricts certain features from over-consuming resources that are
> shared with different features.
>
> And, of course, after messing around with system-max you're going to
> have to reload the device to apply the new settings.
>
> Best regards,
> Martijn
>
> On 06-09-17 21:53, Brian Rak wrote:
>>
>> What sort of resource limits are you running into here?  Did you find
>> any sort of workaround, or are you just reloading all the time?
>>
>>
>> On 9/6/2017 3:45 PM, Youssef Bengelloun-Zahr wrote:
>>> Been there, suffer from it everyday.
>>>
>>> They might not have CAM profiles ala MLXe, but they still have
>>> CAMs... so they have limited and counted ressources.
>>>
>>> Best regards.
>>>
>>>
>>>
>>> Le 6 sept. 2017 à 21:37, Brian Rak <brak@gameservers.com
>>> <mailto:brak@gameservers.com>> a écrit :
>>>
>>>> CER's don't have CAM profiles.  From the information I can find,
>>>> they don't actually use TCAM anyway.
>>>>
>>>>
>>>> On 9/6/2017 3:32 PM, Derek wrote:
>>>>> Did you check the CAM utilization?  It's probably full, perhaps
>>>>> you can partition it differently.
>>>>>
>>>>> On Wed, Sep 6, 2017 at 2:30 PM, Brian Rak <brak@gameservers.com
>>>>> <mailto:brak@gameservers.com>> wrote:
>>>>>
>>>>> Has anyone else seem problems with the CER's being unable to
>>>>> add routes before?  We have multiple devices (all 2024C-4X)
>>>>> reporting one or both of these messages:
>>>>>
>>>>> IPv6 Route ADD: CAM entry creation FAILED
>>>>> IPv4 Network Route ADD: CAM entry creation FAILED
>>>>>
>>>>> They're learning a full v4+v6 table, but we are well below the
>>>>> limits defined in `sh default values`.  From past experience,
>>>>> a reload seems to be the only thing that will actually correct
>>>>> this, but that is a pretty annoying thing to have to do.
>>>>>
>>>>> Any ideas for what may be causing this?  We're on 5.6.0m right
>>>>> now.
>>>>>
>>>>> _______________________________________________
>>>>> foundry-nsp mailing list
>>>>> foundry-nsp@puck.nether.net <mailto:foundry-nsp@puck.nether.net>
>>>>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>>>>> <http://puck.nether.net/mailman/listinfo/foundry-nsp>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Be Well,
>>>>>
>>>>> Derek Labian
>>>>
>>>> _______________________________________________
>>>> foundry-nsp mailing list
>>>> foundry-nsp@puck.nether.net <mailto:foundry-nsp@puck.nether.net>
>>>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>>
>>
>>
>> _______________________________________________
>> foundry-nsp mailing list
>> foundry-nsp@puck.nether.net
>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>
> --
> Met vriendelijke groet / Kindest regards,
> Martijn Schmidt
>
>
> i3D.net performance hosting
> *Martijn Schmidt | Network Architect*
> Email: martijnschmidt@i3d.net <mailto://martijnschmidt@i3d.net> | Tel:
> +31 10 8900070
>
> *i3D.net B.V. | Global Backbone AS49544*
> Rivium 1e Straat 1, 2909 LE Capelle aan den IJssel, The Netherlands
> VAT: NL 8202.63.886.B01
>
> Website
> <http://www.i3d.net/?utm_source=emailsignature&utm_medium=email&utm_campaign=home>
> | Case Studies
> <http://www.i3d.net/partners/?utm_source=emailsignature&utm_medium=email&utm_campaign=case-studies>
> | LinkedIn <https://www.linkedin.com/company/i3d-net>
>
>

Sounds like it's time to open a TAC case, then! :-)

Best regards,
Martijn

On 06-09-17 22:33, Brian Rak wrote:
>
> Hmm, thanks!
>
> 'show resource' at least gives us a counter of failed allocations. It
> doesn't make a whole lot of sense to me, because we're seeing failed
> allocations despite only being ~70% used:
>
> [IP]1102400(size), 339702(free), 069.18%(used),
> 4198683(failed)
>
> We're running with a system-max ip cache/route of 1000000 and ivp6
> cache/route of 102400. 'sh default values' confirms those are
> actually set, we ran into those limits ages ago (and have rebooted
> many times since raising them)
>
>
> On 9/6/2017 4:09 PM, i3D.net - Martijn Schmidt wrote:
>> On the CER, the equivalent of "show cam-partition usage" is "show
>> resource". I believe the IP entries include both IPv4 & IPv6.
>>
>> "show default values" is also useful because you can see if your
>> "system-max ipv6-route" is high enough - the default is only 8192 and
>> your IPv6 DFZ table obviously won't fit if your system-max is set to
>> such a low value. We tend to run it at 262144 because system-max is a
>> fairly pointless feature anyway, it doesn't carve up your CAM, just
>> restricts certain features from over-consuming resources that are
>> shared with different features.
>>
>> And, of course, after messing around with system-max you're going to
>> have to reload the device to apply the new settings.
>>
>> Best regards,
>> Martijn
>>
>> On 06-09-17 21:53, Brian Rak wrote:
>>>
>>> What sort of resource limits are you running into here? Did you
>>> find any sort of workaround, or are you just reloading all the time?
>>>
>>>
>>> On 9/6/2017 3:45 PM, Youssef Bengelloun-Zahr wrote:
>>>> Been there, suffer from it everyday.
>>>>
>>>> They might not have CAM profiles ala MLXe, but they still have
>>>> CAMs... so they have limited and counted ressources.
>>>>
>>>> Best regards.
>>>>
>>>>
>>>>
>>>> Le 6 sept. 2017 à 21:37, Brian Rak <brak@gameservers.com
>>>> <mailto:brak@gameservers.com>> a écrit :
>>>>
>>>>> CER's don't have CAM profiles. From the information I can find,
>>>>> they don't actually use TCAM anyway.
>>>>>
>>>>>
>>>>> On 9/6/2017 3:32 PM, Derek wrote:
>>>>>> Did you check the CAM utilization? It's probably full, perhaps
>>>>>> you can partition it differently.
>>>>>>
>>>>>> On Wed, Sep 6, 2017 at 2:30 PM, Brian Rak <brak@gameservers.com
>>>>>> <mailto:brak@gameservers.com>> wrote:
>>>>>>
>>>>>> Has anyone else seem problems with the CER's being unable to
>>>>>> add routes before? We have multiple devices (all 2024C-4X)
>>>>>> reporting one or both of these messages:
>>>>>>
>>>>>> IPv6 Route ADD: CAM entry creation FAILED
>>>>>> IPv4 Network Route ADD: CAM entry creation FAILED
>>>>>>
>>>>>> They're learning a full v4+v6 table, but we are well below
>>>>>> the limits defined in `sh default values`. From past
>>>>>> experience, a reload seems to be the only thing that will
>>>>>> actually correct this, but that is a pretty annoying thing to
>>>>>> have to do.
>>>>>>
>>>>>> Any ideas for what may be causing this? We're on 5.6.0m
>>>>>> right now.
>>>>>>
>>>>>> _______________________________________________
>>>>>> foundry-nsp mailing list
>>>>>> foundry-nsp@puck.nether.net <mailto:foundry-nsp@puck.nether.net>
>>>>>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>>>>>> <http://puck.nether.net/mailman/listinfo/foundry-nsp>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Be Well,
>>>>>>
>>>>>> Derek Labian
>>>>>
>>>>> _______________________________________________
>>>>> foundry-nsp mailing list
>>>>> foundry-nsp@puck.nether.net <mailto:foundry-nsp@puck.nether.net>
>>>>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>>>
>>>
>>>
>>> _______________________________________________
>>> foundry-nsp mailing list
>>> foundry-nsp@puck.nether.net
>>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>>
>> --
>> Met vriendelijke groet / Kindest regards,
>> Martijn Schmidt
>>
>>
>> i3D.net performance hosting
>> *Martijn Schmidt | Network Architect*
>> Email: martijnschmidt@i3d.net <mailto://martijnschmidt@i3d.net> |
>> Tel: +31 10 8900070
>>
>> *i3D.net B.V. | Global Backbone AS49544*
>> Rivium 1e Straat 1, 2909 LE Capelle aan den IJssel, The Netherlands
>> VAT: NL 8202.63.886.B01
>>
>> Website
>> <http://www.i3d.net/?utm_source=emailsignature&utm_medium=email&utm_campaign=home>
>> | Case Studies
>> <http://www.i3d.net/partners/?utm_source=emailsignature&utm_medium=email&utm_campaign=case-studies>
>> | LinkedIn <https://www.linkedin.com/company/i3d-net>
>>
>>
>

--
Met vriendelijke groet / Kindest regards,
Martijn Schmidt


i3D.net performance hosting
*Martijn Schmidt | Network Architect*
Email: martijnschmidt@i3d.net <mailto://martijnschmidt@i3d.net> | Tel:
+31 10 8900070

*i3D.net B.V. | Global Backbone AS49544*
Rivium 1e Straat 1, 2909 LE Capelle aan den IJssel, The Netherlands
VAT: NL 8202.63.886.B01

Website
<http://www.i3d.net/?utm_source=emailsignature&utm_medium=email&utm_campaign=home>
| Case Studies
<http://www.i3d.net/partners/?utm_source=emailsignature&utm_medium=email&utm_campaign=case-studies>
| LinkedIn <https://www.linkedin.com/company/i3d-net>

Brian,

My guess is still the cam partitioning. Just because it's not configurable
on the CER doesn't mean the CAM isn't partitioned. I.e. you may have free
cam space, and still, have resource allocation failing depending on the
type of resource. Since 'sh resources' isn't giving you the details, I
might try a 'sh tech' and dig around in the results to see if you can get
more details on a breakdown.

Derek

On Wed, Sep 6, 2017 at 3:33 PM, Brian Rak <brak@gameservers.com> wrote:

> Hmm, thanks!
>
> 'show resource' at least gives us a counter of failed allocations. It
> doesn't make a whole lot of sense to me, because we're seeing failed
> allocations despite only being ~70% used:
>
> [IP]1102400(size), 339702(free), 069.18%(used),
> 4198683(failed)
>
> We're running with a system-max ip cache/route of 1000000 and ivp6
> cache/route of 102400. 'sh default values' confirms those are actually
> set, we ran into those limits ages ago (and have rebooted many times since
> raising them)
>
> On 9/6/2017 4:09 PM, i3D.net - Martijn Schmidt wrote:
>
> On the CER, the equivalent of "show cam-partition usage" is "show
> resource". I believe the IP entries include both IPv4 & IPv6.
>
> "show default values" is also useful because you can see if your
> "system-max ipv6-route" is high enough - the default is only 8192 and your
> IPv6 DFZ table obviously won't fit if your system-max is set to such a low
> value. We tend to run it at 262144 because system-max is a fairly pointless
> feature anyway, it doesn't carve up your CAM, just restricts certain
> features from over-consuming resources that are shared with different
> features.
>
> And, of course, after messing around with system-max you're going to have
> to reload the device to apply the new settings.
>
> Best regards,
> Martijn
>
> On 06-09-17 21:53, Brian Rak wrote:
>
> What sort of resource limits are you running into here? Did you find any
> sort of workaround, or are you just reloading all the time?
>
> On 9/6/2017 3:45 PM, Youssef Bengelloun-Zahr wrote:
>
> Been there, suffer from it everyday.
>
> They might not have CAM profiles ala MLXe, but they still have CAMs... so
> they have limited and counted ressources.
>
> Best regards.
>
>
>
> Le 6 sept. 2017 à 21:37, Brian Rak <brak@gameservers.com> a écrit :
>
> CER's don't have CAM profiles. From the information I can find, they
> don't actually use TCAM anyway.
>
> On 9/6/2017 3:32 PM, Derek wrote:
>
> Did you check the CAM utilization? It's probably full, perhaps you can
> partition it differently.
>
> On Wed, Sep 6, 2017 at 2:30 PM, Brian Rak <brak@gameservers.com> wrote:
>
>> Has anyone else seem problems with the CER's being unable to add routes
>> before? We have multiple devices (all 2024C-4X) reporting one or both of
>> these messages:
>>
>> IPv6 Route ADD: CAM entry creation FAILED
>> IPv4 Network Route ADD: CAM entry creation FAILED
>>
>> They're learning a full v4+v6 table, but we are well below the limits
>> defined in `sh default values`. From past experience, a reload seems to be
>> the only thing that will actually correct this, but that is a pretty
>> annoying thing to have to do.
>>
>> Any ideas for what may be causing this? We're on 5.6.0m right now.
>>
>> _______________________________________________
>> foundry-nsp mailing list
>> foundry-nsp@puck.nether.net
>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>
>
>
>
> --
> Be Well,
>
> Derek Labian
>
>
> _______________________________________________
> foundry-nsp mailing list
> foundry-nsp@puck.nether.net
> http://puck.nether.net/mailman/listinfo/foundry-nsp
>
>
>
>
> _______________________________________________
> foundry-nsp mailing listfoundry-nsp@puck.nether.nethttp://puck.nether.net/mailman/listinfo/foundry-nsp
>
>
> --
> Met vriendelijke groet / Kindest regards,
> Martijn Schmidt
>
>
> [image: i3D.net performance hosting]
> *Martijn Schmidt | Network Architect*
> Email: martijnschmidt@i3d.net <//martijnschmidt@i3d.net> | Tel: +31 10
> 8900070 <+31%2010%20890%200070>
>
> *i3D.net B.V. | Global Backbone AS49544*
> Rivium 1e Straat 1, 2909 LE Capelle aan den IJssel, The Netherlands
> VAT: NL 8202.63.886.B01
>
> Website
> <http://www.i3d.net/?utm_source=emailsignature&utm_medium=email&utm_campaign=home>
> | Case Studies
> <http://www.i3d.net/partners/?utm_source=emailsignature&utm_medium=email&utm_campaign=case-studies>
> | LinkedIn <https://www.linkedin.com/company/i3d-net>
>
>
>
> _______________________________________________
> foundry-nsp mailing list
> foundry-nsp@puck.nether.net
> http://puck.nether.net/mailman/listinfo/foundry-nsp
>



--
Be Well,

Derek Labian

What he said :-)

Best regards.



> Le 6 sept. 2017 à 22:09, i3D.net - Martijn Schmidt <martijnschmidt@i3d.net> a écrit :
>
> On the CER, the equivalent of "show cam-partition usage" is "show resource". I believe the IP entries include both IPv4 & IPv6.
>
> "show default values" is also useful because you can see if your "system-max ipv6-route" is high enough - the default is only 8192 and your IPv6 DFZ table obviously won't fit if your system-max is set to such a low value. We tend to run it at 262144 because system-max is a fairly pointless feature anyway, it doesn't carve up your CAM, just restricts certain features from over-consuming resources that are shared with different features.
>
> And, of course, after messing around with system-max you're going to have to reload the device to apply the new settings.
>
> Best regards,
> Martijn
>
>> On 06-09-17 21:53, Brian Rak wrote:
>> What sort of resource limits are you running into here? Did you find any sort of workaround, or are you just reloading all the time?
>>
>>> On 9/6/2017 3:45 PM, Youssef Bengelloun-Zahr wrote:
>>> Been there, suffer from it everyday.
>>>
>>> They might not have CAM profiles ala MLXe, but they still have CAMs... so they have limited and counted ressources.
>>>
>>> Best regards.
>>>
>>>
>>>
>>> Le 6 sept. 2017 à 21:37, Brian Rak <brak@gameservers.com> a écrit :
>>>
>>>> CER's don't have CAM profiles. From the information I can find, they don't actually use TCAM anyway.
>>>>
>>>>> On 9/6/2017 3:32 PM, Derek wrote:
>>>>> Did you check the CAM utilization? It's probably full, perhaps you can partition it differently.
>>>>>
>>>>>> On Wed, Sep 6, 2017 at 2:30 PM, Brian Rak <brak@gameservers.com> wrote:
>>>>>> Has anyone else seem problems with the CER's being unable to add routes before? We have multiple devices (all 2024C-4X) reporting one or both of these messages:
>>>>>>
>>>>>> IPv6 Route ADD: CAM entry creation FAILED
>>>>>> IPv4 Network Route ADD: CAM entry creation FAILED
>>>>>>
>>>>>> They're learning a full v4+v6 table, but we are well below the limits defined in `sh default values`. From past experience, a reload seems to be the only thing that will actually correct this, but that is a pretty annoying thing to have to do.
>>>>>>
>>>>>> Any ideas for what may be causing this? We're on 5.6.0m right now.
>>>>>>
>>>>>> _______________________________________________
>>>>>> foundry-nsp mailing list
>>>>>> foundry-nsp@puck.nether.net
>>>>>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Be Well,
>>>>>
>>>>> Derek Labian
>>>>
>>>> _______________________________________________
>>>> foundry-nsp mailing list
>>>> foundry-nsp@puck.nether.net
>>>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>>
>>
>>
>> _______________________________________________
>> foundry-nsp mailing list
>> foundry-nsp@puck.nether.net
>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>
> --
> Met vriendelijke groet / Kindest regards,
> Martijn Schmidt
>
>
>
> Martijn Schmidt | Network Architect
> Email: martijnschmidt@i3d.net | Tel: +31 10 8900070
>
> i3D.net B.V. | Global Backbone AS49544
> Rivium 1e Straat 1, 2909 LE Capelle aan den IJssel, The Netherlands
> VAT: NL 8202.63.886.B01
>
> Website | Case Studies | LinkedIn
>

I am also wondering if there is some local condition than can trigger
this. Running into a IPv6 route condition: Ok - running also into a IPv4
condition and also on all CER-devices: Well!

What about the mac-address size table? Is the rest of the network as
stable as it should be like? Do you run 05.6.00m on all devices? Can you
upgrade one CER e.g. to a newer 6.1? I have found DEFECT000549512 <=
5.9, but this should be fixed in the m-version.

Can you post
show default values
show ip cache
show ipv6 cache

Jörg

On 6 Sep 2017, at 22:35, i3D.net - Martijn Schmidt wrote:

> Sounds like it's time to open a TAC case, then! :-)
>
> Best regards,
> Martijn
>
> On 06-09-17 22:33, Brian Rak wrote:
>>
>> Hmm, thanks!
>>
>> 'show resource' at least gives us a counter of failed allocations.
>> It
>> doesn't make a whole lot of sense to me, because we're seeing failed
>> allocations despite only being ~70% used:
>>
>> [IP]1102400(size), 339702(free), 069.18%(used),
>> 4198683(failed)
>>
>> We're running with a system-max ip cache/route of 1000000 and ivp6
>> cache/route of 102400. 'sh default values' confirms those are
>> actually set, we ran into those limits ages ago (and have rebooted
>> many times since raising them)
>>
>>
>> On 9/6/2017 4:09 PM, i3D.net - Martijn Schmidt wrote:
>>> On the CER, the equivalent of "show cam-partition usage" is "show
>>> resource". I believe the IP entries include both IPv4 & IPv6.
>>>
>>> "show default values" is also useful because you can see if your
>>> "system-max ipv6-route" is high enough - the default is only 8192
>>> and
>>> your IPv6 DFZ table obviously won't fit if your system-max is set to
>>> such a low value. We tend to run it at 262144 because system-max is
>>> a
>>> fairly pointless feature anyway, it doesn't carve up your CAM, just
>>> restricts certain features from over-consuming resources that are
>>> shared with different features.
>>>
>>> And, of course, after messing around with system-max you're going to
>>> have to reload the device to apply the new settings.
>>>
>>> Best regards,
>>> Martijn
>>>
>>> On 06-09-17 21:53, Brian Rak wrote:
>>>>
>>>> What sort of resource limits are you running into here? Did you
>>>> find any sort of workaround, or are you just reloading all the
>>>> time?
>>>>
>>>>
>>>> On 9/6/2017 3:45 PM, Youssef Bengelloun-Zahr wrote:
>>>>> Been there, suffer from it everyday.
>>>>>
>>>>> They might not have CAM profiles ala MLXe, but they still have
>>>>> CAMs... so they have limited and counted ressources.
>>>>>
>>>>> Best regards.
>>>>>
>>>>>
>>>>>
>>>>> Le 6 sept. 2017 à 21:37, Brian Rak <brak@gameservers.com
>>>>> <mailto:brak@gameservers.com>> a écrit :
>>>>>
>>>>>> CER's don't have CAM profiles. From the information I can find,
>>>>>> they don't actually use TCAM anyway.
>>>>>>
>>>>>>
>>>>>> On 9/6/2017 3:32 PM, Derek wrote:
>>>>>>> Did you check the CAM utilization? It's probably full, perhaps
>>>>>>> you can partition it differently.
>>>>>>>
>>>>>>> On Wed, Sep 6, 2017 at 2:30 PM, Brian Rak <brak@gameservers.com
>>>>>>> <mailto:brak@gameservers.com>> wrote:
>>>>>>>
>>>>>>> Has anyone else seem problems with the CER's being unable to
>>>>>>> add routes before? We have multiple devices (all 2024C-4X)
>>>>>>> reporting one or both of these messages:
>>>>>>>
>>>>>>> IPv6 Route ADD: CAM entry creation FAILED
>>>>>>> IPv4 Network Route ADD: CAM entry creation FAILED
>>>>>>>
>>>>>>> They're learning a full v4+v6 table, but we are well below
>>>>>>> the limits defined in `sh default values`. From past
>>>>>>> experience, a reload seems to be the only thing that will
>>>>>>> actually correct this, but that is a pretty annoying thing
>>>>>>> to
>>>>>>> have to do.
>>>>>>>
>>>>>>> Any ideas for what may be causing this? We're on 5.6.0m
>>>>>>> right now.
_______________________________________________
foundry-nsp mailing list
foundry-nsp@puck.nether.net
http://puck.nether.net/mailman/listinfo/foundry-nsp