Mailing List Archive

tilting at windmills discussion 2
In an attempt to reduce the signal to noise ratio of NANOG, I felt like
ATM should get equal time with registry policy for unending discussion of
week award. In Bob Melcalfe's concept that all we care about is working
code, I have included a PERL program for your benefit.

The programs reads a histogram file like the ones kc generates from
mae-west (http://www.nlanr.net/NA/Learn/packetsizes.html) and computes
the overhead for various framing methods. Hopefully taking
empirical data from the internet and giving people code will reduce
the discussions of what the "cell tax" for real traffic is. Don't
like my traffic, collect your own. Fighting over conclusions will
be left to others.

Jerry

Here's the output from the 15 minute collection run on Feb 10th, '96:

% packettax.pl < packetsizes.data
total packets seen = 11708789, total payload bytes seen = 3010380871
HDLC framing bytes = 3080633605 HDLC efficiency = 97.72
ATM framing bytes = 3644304857 ATM efficiency = 82.61
ATM w/snap framing bytes = 3862101043 ATM w/snap efficiency = 77.95

and here's the crude but working code (copyright courtesy of the ISC)

#! /usr/bin/perl

# Copyright (c) 1996 by Internet Software Consortium.
#
# Permission to use, copy, modify, and distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.
#
# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SOFTWARE CONSORTIUM DISCLAIMS
# ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES
# OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL INTERNET SOFTWARE
# CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL
# DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR
# PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS
# ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS
# SOFTWARE.
#
# Jerry Scharf, 5/96

# perl script to take a packet histogram and compute framing efficiency
# the expected format is:
# packetsize count rest
# for each packetsize, the program computes the frame size for HDLC
# and, ATM/AAL5 with and without SNAP, and multiplies them by count and
# adds them to running totals. At the end, it prints the 3 totals, and the
# efficiency of each framing method (payload/framedsize).
# (I had done PPP, but there are too many possible header sizes.)
#
# this was specifically designed to take data from the NLANR project to
# generate traffic studies, courtesy of k claffy. Kent England checked the
# ATM cell calcualtions.
#

$payload = 0.0;
$hdlcload = 0.0;
# $pppload = 0.0;
$atmload = 0.0;
$atmsnapload = 0.0;
$numpackets = 0.0;

while (<>) {
chop;
($packetsize,$packetcount,@rest) = split;
$hdlcframe = $packetsize + 6;
# $pppframe = $packetsize + 8;
$atmframe = int(($packetsize+8+47)/48) * 53;
$atmsnapframe = int(($packetsize+16+47)/48) * 53;
$payload += $packetsize*$packetcount;
$hdlcload += $hdlcframe*$packetcount;
# $pppload += $pppframe*$packetcount;
$atmload += $atmframe*$packetcount;
$atmsnapload += $atmsnapframe*$packetcount;
$numpackets += $packetcount;
}

printf("total packets seen = %12.0f, total payload bytes seen = %12.0f\n",
$numpackets, $payload);
printf("HDLC framing bytes = %12.0f HDLC efficiency = %2.2f\%\n",
$hdlcload, 100*$payload/$hdlcload);
# printf("PPP framing bytes = %12.0f PPP efficiency = %2.2f\%\n",
# $pppload, 100*$payload/$pppload);
printf("ATM framing bytes = %12.0f ATM efficiency = %2.2f\%\n",
$atmload, 100*$payload/$atmload);
printf("ATM w/snap framing bytes = %12.0f ATM w/snap efficiency = %2.2f\%\n",
$atmsnapload, 100*$payload/$atmsnapload);
- - - - - - - - - - - - - - - - -
Re: tilting at windmills discussion 2 [ In reply to ]
Thanks for the hard figures, Jerry!

--vadim
- - - - - - - - - - - - - - - - -
Re: tilting at windmills discussion 2 [ In reply to ]
>IP packet size distribution (970780293 total packets):
> 1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480
> .000 .497 .094 .013 .009 .006 .007 .010 .010 .014 .004 .005 .002 .002 .002
> ^^^^
> 512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
> .004 .002 .189 .000 .069 .053 .000 .000 .000 .000 .000
>
>Collected at a private interconnect between CICNet, OARnet and OSU. over a
>period of a week or so.

This is not surprising if you think of it. Short packets are
pure ACKs, telnet keystrokes and SYNs. What is worrying is the
peak at 576. It means TCP unable to grasp that path MTU is
really 1.5K.

--vadim
- - - - - - - - - - - - - - - - -
Re: tilting at windmills discussion 2 [ In reply to ]
On Tue, 18 Jun 1996, Jerry Scharf wrote:

> the overhead for various framing methods. Hopefully taking
> empirical data from the internet and giving people code will reduce
> the discussions of what the "cell tax" for real traffic is. Don't
> like my traffic, collect your own. Fighting over conclusions will
> be left to others.

In a similar vein, something to think about:

IP packet size distribution (970780293 total packets):
1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480
.000 .497 .094 .013 .009 .006 .007 .010 .010 .014 .004 .005 .002 .002 .002
^^^^
512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
.004 .002 .189 .000 .069 .053 .000 .000 .000 .000 .000

Collected at a private interconnect between CICNet, OARnet and OSU. over a
period of a week or so.

-dorian

- - - - - - - - - - - - - - - - -
Re: tilting at windmills discussion 2 [ In reply to ]
> In a similar vein, something to think about:
>
> IP packet size distribution (970780293 total packets):
> 1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480
> .000 .497 .094 .013 .009 .006 .007 .010 .010 .014 .004 .005 .002 .002 .002
> ^^^^
> 512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
> .004 .002 .189 .000 .069 .053 .000 .000 .000 .000 .000
>
> Collected at a private interconnect between CICNet, OARnet and OSU. over a
> period of a week or so.

It would be really interesting if you could expand the range of 32 to 64
byte packets to display in one byte increments. Then we could
try to guess:

o How many packets are nothing but TCP and IP headers?

o How many packets appear to be TCP/IP headers with one byte of
data?

o How many of these packets, with some sort of header compression
scheme, could fit in one ATM cell?

o Etc., etc.

-tjs

- - - - - - - - - - - - - - - - -
Re: tilting at windmills discussion 2 [ In reply to ]
Do you also happen to have a breakdown of what ports this raffic was going
to/from? The reason I ask iss that this seems to lead to a conclusion that
a lot of the data was for "interactive" services such as telnet and the like
as opposed to batch transfer type of connections such as ftp, and sendmail.


At 03:29 AM 6/19/96 -0400, Dorian Kim wrote:
>On Tue, 18 Jun 1996, Jerry Scharf wrote:
>
>> the overhead for various framing methods. Hopefully taking
>> empirical data from the internet and giving people code will reduce
>> the discussions of what the "cell tax" for real traffic is. Don't
>> like my traffic, collect your own. Fighting over conclusions will
>> be left to others.
>
>In a similar vein, something to think about:
>
>IP packet size distribution (970780293 total packets):
> 1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480
> .000 .497 .094 .013 .009 .006 .007 .010 .010 .014 .004 .005 .002 .002 .002
> ^^^^
> 512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
> .004 .002 .189 .000 .069 .053 .000 .000 .000 .000 .000
>
>Collected at a private interconnect between CICNet, OARnet and OSU. over a
>period of a week or so.
>
>-dorian
>
>
>

Justin Newton
Internet Architect
Erol's Internet Services

- - - - - - - - - - - - - - - - -
Re: tilting at windmills discussion 2 [ In reply to ]
> > In a similar vein, something to think about:
> >
> > IP packet size distribution (970780293 total packets):
> > 1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480
> > .000 .497 .094 .013 .009 .006 .007 .010 .010 .014 .004 .005 .002 .002 .002
> > ^^^^
> > 512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
> > .004 .002 .189 .000 .069 .053 .000 .000 .000 .000 .000


Ah, finally some data pertaining to the "it's really important that
interconnects support large mtus" discussion.

- - - - - - - - - - - - - - - - -
Re: tilting at windmills discussion 2 [ In reply to ]
Justin W. Newton <justin@erols.com> writes:

> Do you also happen to have a breakdown of what ports this raffic was going
> to/from? The reason I ask iss that this seems to lead to a conclusion that
> a lot of the data was for "interactive" services such as telnet and the like
> as opposed to batch transfer type of connections such as ftp, and sendmail.

go look at http://www.nlanr.net/NA and the parent directory of the URL in the
original mail. k claffey and folks have done a great job of collecting loads
of stats and analyzing them 17 different ways. Anyone interested in doing
traffic statistics work should start there.

Jerry
- - - - - - - - - - - - - - - - -
Re: tilting at windmills discussion 2 [ In reply to ]
On Wed, 19 Jun 1996, Vadim Antonov wrote:

> > 512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
> > .004 .002 .189 .000 .069 .053 .000 .000 .000 .000 .000
> >
> >Collected at a private interconnect between CICNet, OARnet and OSU. over a
> >period of a week or so.
>
> This is not surprising if you think of it. Short packets are
> pure ACKs, telnet keystrokes and SYNs. What is worrying is the
> peak at 576. It means TCP unable to grasp that path MTU is
> really 1.5K.

Agreed. There's alot of bad/old implementations out there, and that's
something that should cause some worry..

-dorian

- - - - - - - - - - - - - - - - -