Hi,
I noticed errors in my Apache logs :
[Sat Jul 03 08:52:59 2010] [error] [client 66.249.71.237] File does not
exist: /data/davical/htdocs/robots.txt
[Sat Jul 03 14:53:02 2010] [error] [client 66.249.68.211] File does not
exist: /data/davical/htdocs/robots.txt
[Sun Jul 04 02:53:07 2010] [error] [client 66.249.68.211] File does not
exist: /data/davical/htdocs/robots.txt
[Sun Jul 04 14:53:11 2010] [error] [client 66.249.71.237] File does not
exist: /data/davical/htdocs/robots.txt
[Mon Jul 05 02:53:16 2010] [error] [client 66.249.71.237] File does not
exist: /data/davical/htdocs/robots.txt
[Mon Jul 05 14:53:21 2010] [error] [client 66.249.71.237] File does not
exist: /data/davical/htdocs/robots.txt
It would maybe make sense to add a htdocs/robots.txt in the davical
tarball, so we are avoided by these things ?
Mine looks like this :
------------------------------
User-agent: *
Disallow: /
------------------------------
Regs,
Charles
I noticed errors in my Apache logs :
[Sat Jul 03 08:52:59 2010] [error] [client 66.249.71.237] File does not
exist: /data/davical/htdocs/robots.txt
[Sat Jul 03 14:53:02 2010] [error] [client 66.249.68.211] File does not
exist: /data/davical/htdocs/robots.txt
[Sun Jul 04 02:53:07 2010] [error] [client 66.249.68.211] File does not
exist: /data/davical/htdocs/robots.txt
[Sun Jul 04 14:53:11 2010] [error] [client 66.249.71.237] File does not
exist: /data/davical/htdocs/robots.txt
[Mon Jul 05 02:53:16 2010] [error] [client 66.249.71.237] File does not
exist: /data/davical/htdocs/robots.txt
[Mon Jul 05 14:53:21 2010] [error] [client 66.249.71.237] File does not
exist: /data/davical/htdocs/robots.txt
It would maybe make sense to add a htdocs/robots.txt in the davical
tarball, so we are avoided by these things ?
Mine looks like this :
------------------------------
User-agent: *
Disallow: /
------------------------------
Regs,
Charles