Mailing List Archive: Netcraft Web Surver Survey

Netcraft Web Surver Survey

andrew at epprod

Sep 6, 1995, 7:44 PM

Post #1 of 6 (575 views)

Hi,

after much midnight hacking and after almost getting blown out of the
water by a similar offering published last week I'm releved^Hproud
to announce the unveiling of the Netcraft Web Surver Survey (relax, I'm not
trying to sell you anything).

You can read the hype when you visit the site but briefly we'll be visiting
24,000+ websites a month and presenting our findings in the form of tables,
and soon charts, depicting the demographic spread of different Web Server
software families (and distinct versions) in all domains on the net.

I'll let you draw your own conlusions from the information in the pages but
I would like to draw your attention to the magic 10% of the *.COM domain
that is attributed to sites running Apache, and to some of the top sites in
that 10%.

Thanks to Rob H, Paul R, Beth F and rst for showing enough interest to make
the weekend worth losing to a rewrite, and to you all for putting up with
my repeated claims about having 'some juicy info, but...'.

When you get the time, take a look at this place, and don't forget to come
back next month (whoops, more promises, sorry ;)

http://www.netcraft.co.uk/Survey/

Cheers,
Ay.

Andrew Wilson URL: http://www.cm.cf.ac.uk/User/Andrew.Wilson/
Elsevier Science, Oxford Office: +44 01865 843155 Mobile: +44 0589 616144

Re: Netcraft Web Surver Survey [ In reply to ]

brian at organic

Sep 6, 1995, 12:02 PM

Post #2 of 6 (575 views)

On Wed, 6 Sep 1995, Andrew Wilson wrote:
> You can read the hype when you visit the site but briefly we'll be visiting
> 24,000+ websites a month and presenting our findings in the form of tables,
> and soon charts, depicting the demographic spread of different Web Server
> software families (and distinct versions) in all domains on the net.

In the interests of fairness.. should VirtualHosts count as separate servers?
I noticed that BESTWWWD counted for 131 hosts.. and that Organic's hosts were
well represented even though they are all on the same machine. Hmm, I
suppose there's no way to catch that since the IP numbers are different - at
least a catch to make sure no two hosts have the same IP number would be
appropriate methinks.

Otherwise, it looks very thorough.

Brian

--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
brian@organic.com brian@hyperreal.com http://www.[hyperreal,organic].com/

Re: Netcraft Web Surver Survey [ In reply to ]

paul at netcraft

Sep 6, 1995, 2:02 PM

Post #3 of 6 (573 views)

In reply to Brian Behlendorf who said
>
> In the interests of fairness.. should VirtualHosts count as separate servers?
> I noticed that BESTWWWD counted for 131 hosts.. and that Organic's hosts were
> well represented even though they are all on the same machine. Hmm, I
> suppose there's no way to catch that since the IP numbers are different - at
> least a catch to make sure no two hosts have the same IP number would be
> appropriate methinks.

Actually, I like the fact that all virtual hosts are found since it's
usefull to know that www.foo.bar is hosted on a particular server. It
skews the survey unfairly if what you want to know is how many installations
of a server are out there though. Maybe next month we can provide both
sets of results.

--
Paul Richards, Netcraft Ltd.
Internet: paul@netcraft.co.uk, http://www.netcraft.co.uk
Phone: 0370 462071 (Mobile), +44 1225 447500 (work)

Re: Netcraft Web Surver Survey [ In reply to ]

Sep 6, 1995, 4:23 PM

Post #4 of 6 (582 views)

Re: Netcraft Web Surver Survey [ In reply to ]

Sep 6, 1995, 8:24 PM

Post #5 of 6 (571 views)

> In the interests of fairness.. should VirtualHosts count as separate servers?
> I noticed that BESTWWWD counted for 131 hosts.. and that Organic's hosts were
> well represented even though they are all on the same machine. Hmm, I
> suppose there's no way to catch that since the IP numbers are different - at
> least a catch to make sure no two hosts have the same IP number would be
> appropriate methinks.

Valid points. But suppose the question was not:

"How many apache servers are there out there?"

but rather:

"How many people choose to have their information hosted
on an apache server?"

Checking for 'same IP' is relatively easy and worth considering. We may
be able to give an answer to:

"How many Apache server's are used in Virtual Hosting mode?"

shortly.

> Otherwise, it looks very thorough.

There are bound to be faults, the moreso in our own interpretation, which is
why we've published as much as we can. People with better analytical minds, or
with different points of view should be able to make what they will of this
stuff.

Incidentally, I'd have published this myself on COMMA if I'd been unable to
find support with the current host site. If nothing else, I feel that this
information in its raw state can be of great use to software developers.

> Brian
>
> --=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
> brian@organic.com brian@hyperreal.com http://www.[hyperreal,organic].com/

Cheers,
Ay.

Andrew Wilson URL: http://www.cm.cf.ac.uk/User/Andrew.Wilson/
Elsevier Science, Oxford Office: +44 01865 843155 Mobile: +44 0589 616144

ps. I realise this stuff isn't of direct relevence to the development
of software, so I'm happy to chat off list of people would prefer.
'sup 2 U. ;)

Re: Netcraft Web Surver Survey [ In reply to ]

Sep 7, 1995, 9:55 AM

Post #6 of 6 (572 views)

> Nice stuff. I'd be happy to assist with the statistics if you need it.
> In the mean time, it'd be nice to:
>
> * track which & when sites change servers - this gives a reasonably strong
> indicator of what's up.

So we can track how close people follow the new releases. We've got the data
so it might be interesting to work out something like

"percentage of servers who changed from FooBar/1.2 to
BazBumble/2.2a-beta in the preceeding month"

> * add a registration facility for sites to be added to the next round - this
> only makes your job easier & your results more compelling.

Yep, we're considering this.

> * growth rates for the number of new sites, new servers, etc.

We could have done this already I suppose. We actually have 3 surveys already
compiled (1 aug, 16 aug, 1 sep), so perhaps we can build a rate of change
analysis from these. I'm wary of setting a precedent and getting into a war
with other suveys where we might eventually be running one survey a day. Don't
want to be accused of running a really aggressive webcrawler. At any rate
presently a 1monthly sample seems to give good results in terms of %point changes
for server usage (sounds like the stock market).

> Regards,
> Jim.

Cheers,
Ay.

Andrew Wilson URL: http://www.cm.cf.ac.uk/User/Andrew.Wilson/
Elsevier Science, Oxford Office: +44 01865 843155 Mobile: +44 0589 616144