Mailing List Archive: GigE: M10 Cisco Catalyst 6509: oversized and corrupted frames

GigE: M10 <-> Cisco Catalyst 6509: oversized and corrupted frames

Mar 24, 2003, 9:28 AM

Post #1 of 8 (5117 views)

I've connected Juniper M10 (JUNOS 5.6R2.4) and Cisco Catalyst 6509
(IOS 12.1(13)E4) to each other via gigabit ethernet using multimode
fibre. Both ends complain about the packets they are seeing. On M10,
the number in oversized frames counter increases. The 6509 complains
about corrupted IP packets, too.

I did check the MTUs but they are the same on the both ends. There are
no other errors which could be the cause of this beaviour.

The relevant counters on M10 look like this:

> show interfaces ge-0/1/0 extensive
Physical interface: ge-0/1/0, Enabled, Physical link is Up
Interface index: 14, SNMP ifIndex: 17, Generation: 13
Link-level type: Ethernet, MTU: 1518, Speed: 1000mbps, Loopback: Disabled,
[cut]
MAC statistics: Receive Transmit
Total octets 49678735078 23364511178
Total packets 264424668 266312599
Unicast packets 259715065 265837157
Broadcast packets 242 17960
Multicast packets 4362526 457482
CRC/Align errors 0 0
FIFO errors 0 0
MAC control frames 0 0
MAC pause frames 0 0
Oversized frames 346835
Jabber frames 0
Fragment frames 0
VLAN tagged frames 263506071
Code violations 0

Cisco just logs this:

%MLS_STAT-SP-4-IP_CSUM_ERR: IP checksum errors

The local Cisco support informed me that sometimes Juniper sends GigE
packets too rapidly. I understood this so that sometimes the interval
between two frames Juniper sends is shorter than the GigE specs allow.
Is anyone able to confirm that information? I've no reason not to
believe that, but I'm curious whether anyone else has encountered the
same problem. Also, I'm very interested in finding a workaround.

Cheers,

--
- Matti -

GigE: M10 <-> Cisco Catalyst 6509: oversized and corrupted frames [ In reply to ]

josefb at juniper

Mar 24, 2003, 10:08 AM

Post #2 of 8 (5086 views)

Permalink

At 03:29 PM 3/24/2003, Matti Saarinen wrote:

>I've connected Juniper M10 (JUNOS 5.6R2.4) and Cisco Catalyst 6509
>(IOS 12.1(13)E4) to each other via gigabit ethernet using multimode
>fibre. Both ends complain about the packets they are seeing. On M10,
>the number in oversized frames counter increases.

this might be normal if you use tagged interface connections
and the frame is longer then 1518 including FCS. Although all
works perfect the counter is increasing as oversized frames.
So you would get a frame of 1522 including FCS in this case which
will increase the oversize counter.

> The 6509 complains
>about corrupted IP packets, too.

so you get checksum error when you ping the interface address
of the cisco ? is this consistent ?
you can run one test on the juniper where you perform an external
loopback between TX and RX and you ping your local interface
address with the bypass routing knob. This way you check if the
local PIC and/or forwarding path is working correct.

Josef

>I did check the MTUs but they are the same on the both ends. There are
>no other errors which could be the cause of this beaviour.
>
>The relevant counters on M10 look like this:
>
> > show interfaces ge-0/1/0 extensive
>Physical interface: ge-0/1/0, Enabled, Physical link is Up
> Interface index: 14, SNMP ifIndex: 17, Generation: 13
> Link-level type: Ethernet, MTU: 1518, Speed: 1000mbps, Loopback: Disabled,
>[cut]
> MAC statistics: Receive Transmit
> Total octets 49678735078 23364511178
> Total packets 264424668 266312599
> Unicast packets 259715065 265837157
> Broadcast packets 242 17960
> Multicast packets 4362526 457482
> CRC/Align errors 0 0
> FIFO errors 0 0
> MAC control frames 0 0
> MAC pause frames 0 0
> Oversized frames 346835
> Jabber frames 0
> Fragment frames 0
> VLAN tagged frames 263506071
> Code violations 0
>
>
>
>Cisco just logs this:
>
>%MLS_STAT-SP-4-IP_CSUM_ERR: IP checksum errors
>
>
>The local Cisco support informed me that sometimes Juniper sends GigE
>packets too rapidly. I understood this so that sometimes the interval
>between two frames Juniper sends is shorter than the GigE specs allow.
>Is anyone able to confirm that information? I've no reason not to
>believe that, but I'm curious whether anyone else has encountered the
>same problem. Also, I'm very interested in finding a workaround.
>
>Cheers,
>
>--
>- Matti -
>_______________________________________________
>juniper-nsp mailing list juniper-nsp@puck.nether.net
>http://puck.nether.net/mailman/listinfo/juniper-nsp

GigE: M10 <-> Cisco Catalyst 6509: oversized and corrupted frames [ In reply to ]

mjs at cc

Mar 28, 2003, 1:13 AM

Post #3 of 8 (5078 views)

Permalink

Josef Buchsteiner <josefb@juniper.net> writes:

>> Both ends complain about the packets they are seeing. On M10, the
>> number in oversized frames counter increases.
>
> this might be normal if you use tagged interface connections
> and the frame is longer then 1518 including FCS. Although all
> works perfect the counter is increasing as oversizd frames.

Thanks, now I'm able to iterpret the counters a little bit
better. I do indeed use tagged frames.

>> The 6509 complains about corrupted IP packets, too.
>
> so you get checksum error when you ping the interface address
> of the cisco ? is this consistent ?

No, I don't get any errors and the errors do show up a few times
in a day. The interval between the errors isn't contant.

I attached an analyser to a another GigE port on Cisco and
configured the switch to copy all the traffic it receives from
M10 to that port. The analyser has now been scanning the packets
for a three days and it has seen no errors. I could be that the
Cisco doesn't mirror packet containing errors.

Still, I think that there's nothing wrong in a way Juniper sends
the packets. The errors must be caused by some other equipment.

Thanks,

--
- Matti -

GigE: M10 <-> Cisco Catalyst 6509: oversized and corrupted frames [ In reply to ]

josefb at juniper

Mar 28, 2003, 3:00 AM

Post #4 of 8 (5087 views)

Permalink

At 07:14 AM 3/28/2003, Matti Saarinen wrote:
>Josef Buchsteiner <josefb@juniper.net> writes:
>
> >> Both ends complain about the packets they are seeing. On M10, the
> >> number in oversized frames counter increases.
> >
> > this might be normal if you use tagged interface connections
> > and the frame is longer then 1518 including FCS. Although all
> > works perfect the counter is increasing as oversizd frames.
>
> Thanks, now I'm able to iterpret the counters a little bit
> better. I do indeed use tagged frames.
>
> >> The 6509 complains about corrupted IP packets, too.
> >
> > so you get checksum error when you ping the interface address
> > of the cisco ? is this consistent ?
>
> No, I don't get any errors and the errors do show up a few times
> in a day. The interval between the errors isn't contant.
>
> I attached an analyser to a another GigE port on Cisco and
> configured the switch to copy all the traffic it receives from
> M10 to that port. The analyser has now been scanning the packets
> for a three days and it has seen no errors. I could be that the
> Cisco doesn't mirror packet containing errors.

We need to differentiate between L2/L1 errors and L3 errors. In your
case it is L3. Which means also that your Analyser needs to perform
a data integrity check to catch a corrupted IP packet. It would be
good to understand if the ip header or the ip payload is corrupted
and if your Analyzer is really checking the payload for corruption.
When you performed the mirror function have you still seen that the
6509 complains about corrupted IP packets ?

> Still, I think that there's nothing wrong in a way Juniper sends
> the packets. The errors must be caused by some other equipment.
>
> Thanks,
>
>--
>- Matti -

GigE: M10 <-> Cisco Catalyst 6509: oversized and corrupted frames [ In reply to ]

mjs at cc

Mar 28, 2003, 4:00 AM

Post #5 of 8 (5078 views)

Permalink

Josef Buchsteiner <josefb@juniper.net> writes:

> We need to differentiate between L2/L1 errors and L3 errors. In your
> case it is L3. Which means also that your Analyser needs to perform
> a data integrity check to catch a corrupted IP packet.

I think (but don't know for sure. I haven't read the manual
carefully enough. The analyser is Fluke OptiView.) that the
analyser also checks the IP header. At least, it shows the
network utilisation per TCP and UDP port etc.

> When you performed the mirror function have you still seen that the
> 6509 complains about corrupted IP packets ?

Yes, I have. I've changed the routing topology as well so that
more traffic is going via the GigE link. The error messages
haven't been showing up more frequently.

--
- Matti -

GigE: M10 <-> Cisco Catalyst 6509: oversized and corrupted frames [ In reply to ]

josefb at juniper

Mar 28, 2003, 4:21 AM

Post #6 of 8 (5079 views)

Permalink

At 10:01 AM 3/28/2003, Matti Saarinen wrote:
>Josef Buchsteiner <josefb@juniper.net> writes:
>
> > We need to differentiate between L2/L1 errors and L3 errors. In your
> > case it is L3. Which means also that your Analyser needs to perform
> > a data integrity check to catch a corrupted IP packet.
>
> I think (but don't know for sure. I haven't read the manual
> carefully enough. The analyser is Fluke OptiView.) that the
> analyser also checks the IP header. At least, it shows the
> network utilisation per TCP and UDP port etc.
>
> > When you performed the mirror function have you still seen that the
> > 6509 complains about corrupted IP packets ?
>
> Yes, I have. I've changed the routing topology as well so that
> more traffic is going via the GigE link. The error messages
> haven't been showing up more frequently.

I just got informed from the cisco tac that corrupted ip packets will be
dropped
by the switch engine PFC/PFC2 which is reason you can not see any error
on the analyzer at all when mirror the data ...

>--
>- Matti -
>_______________________________________________
>juniper-nsp mailing list juniper-nsp@puck.nether.net
>http://puck.nether.net/mailman/listinfo/juniper-nsp

GigE: M10 <-> Cisco Catalyst 6509: oversized and corrupted frames [ In reply to ]

dek at hades

Mar 30, 2003, 4:15 AM

Post #7 of 8 (5072 views)

Permalink

Hi Matti,

I may sound stupid, but have you tried to replace the fibre cable between
your Juniper and Cat to the ONE THAT IS KNOW TO BE GOOD? We had a problem at
one point when the cable appeared to be good but was starting to give errors
if bumped or disturbed by an aircon airflow.

May I also suggest obtaining an optical splitter for your GigE link so that
you do actually capture what is going on on the glass, rather than trust the
Cat to copy stuff for you (which as I understand was already proved useless,
as it wouldn't forward broken frames, and I do understand the reasons behind
it).

There's also one more quetion: is your Cat running in native mode or hybrid?
If it is hybrid, then where do you see the errors about corrupt IP headers
(I would imagine it being MSFC, but if it's on Sup side, then it could be
something to do with the way MLS switching is done) (?).

How periodical are your errors and do they concide (J and C side)?

More questions than answers but... ;)

SY,
--
D.K.

GigE: M10 <-> Cisco Catalyst 6509: oversized and corrupted frames [ In reply to ]

mjs at cc

Apr 9, 2003, 1:46 AM

Post #8 of 8 (5079 views)

Permalink

Dmitri Kalintsev <dek@hades.uz> writes:

> I may sound stupid, but have you tried to replace the fibre cable between
> your Juniper and Cat to the ONE THAT IS KNOW TO BE GOOD?

No, I haven't. At the moment, I'm fairly sure that there's no
problem in the connection or in the communication between the M10
and the 6509.

> May I also suggest obtaining an optical splitter for your GigE link
> so that you do actually capture what is going on on the glass,

Such a thing would be really nice to have. I have to start
looking around whether anyone here sells the splitters.

> There's also one more quetion: is your Cat running in native mode or
> hybrid?

Native.

> How periodical are your errors and do they concide (J and C side)?

The only one reporting the errors is the Cat and the errors
appear once in a few hours. M10 has nothing to complain about
(well, there's a tiny memory leak, but the box doesn't complain
about it yet...)

This really is a Cisco issue more than a Juniper one

Thanks to all who helped.

Cheers,

--
- Matti -