Mailing List Archive: Duplicate files in an NFS directory.

We have a NFS file system exported from our filer (it is also a CIFS shared folder).
One of the sub-directories in this file system has duplicate files and directories.
The duplicated files and directories have the same inode number. See the example below:

#ls -i
758588 NPR1/ 758583 docs/
758588 NPR1/ 758583 docs/
758587 README.version 758584 images/
758587 README.version 758584 images/
758586 bin/ 758585 licensing/
758586 bin/ 758585 licensing/
758582 cgi/ 758589 nprinfo
758582 cgi/ 758589 nprinfo

None of the sub-directories in this particular directory has duplicate files and directories.
The directories and files above this directory in the file system tree do not have duplicate files
and directories. Has anyone seen this issue before?

One of my colleagues has resolved this problem before by using the UNIX touch command to change the
timestamp on a file in a duplicated directory. I have tried this and it did not resolve the issue
this time. Thank you in advance for your help.

The problem seems to be solved (this time) by creating a new file in the directory:

metrica:laurel> ls -lrt
total 96
drwxrwxr-x 2 metrica metrica 4096 Jan 23 2002 images/
drwxrwxr-x 2 metrica metrica 4096 Jan 23 2002 images/
drwxrwxr-x 2 metrica metrica 4096 Jan 23 2002 docs/
drwxrwxr-x 2 metrica metrica 4096 Jan 23 2002 docs/
-r--r--r-- 1 metrica metrica 53 Jan 23 2002 README.version
-r--r--r-- 1 metrica metrica 53 Jan 23 2002 README.version
drwxrwxr-x 5 metrica metrica 4096 Jan 23 2002 bin/
drwxrwxr-x 5 metrica metrica 4096 Jan 23 2002 bin/
drwxrwxr-x 4 metrica metrica 4096 Jan 23 2002 cgi/
drwxrwxr-x 4 metrica metrica 4096 Jan 23 2002 cgi/
drwxrwxr-x 2 metrica metrica 4096 Jan 23 2002 licensing/
drwxrwxr-x 2 metrica metrica 4096 Jan 23 2002 licensing/
drwxrwxr-x 13 metrica metrica 4096 Aug 29 16:30 NPR1/
drwxrwxr-x 13 metrica metrica 4096 Aug 29 16:30 NPR1/
-rw-r--r-- 1 metrica metrica 63 Oct 8 14:51 nprinfo
-rw-r--r-- 1 metrica metrica 63 Oct 8 14:51 nprinfo

metrica:laurel> touch fred

metrica:laurel> ls
NPR1/ cgi/ images/
README.version docs/ licensing/
bin/ fred nprinfo
metrica:laurel> ls -i
758588 NPR1/ 1281403 fred
758587 README.version 758584 images/
758586 bin/ 758585 licensing/
758582 cgi/ 758589 nprinfo
758583 docs/

Why is this problem happening? How do I stop this from happening?

Netapp suggests using snoop to try and capture this problem, but they have
not told us what they are looking for.

We are also seeing this issue on more than one Solaris 8 box. Has anyone
seen this issue before?

regards,

Paul O'Neill

Paul_O'Neill@adc.com writes:
>
> We have a NFS file system exported from our filer (it is also a CIFS shared folder).
> One of the sub-directories in this file system has duplicate files and directories.
> The duplicated files and directories have the same inode number. See the example below:
>
> #ls -i
> 758588 NPR1/ 758583 docs/
> 758588 NPR1/ 758583 docs/
> 758587 README.version 758584 images/
> 758587 README.version 758584 images/
> 758586 bin/ 758585 licensing/
> 758586 bin/ 758585 licensing/
> 758582 cgi/ 758589 nprinfo
> 758582 cgi/ 758589 nprinfo

One thing I would check on any future occurrence of the problem is what
"ls -f" shows on the directory, i.e. displaying the entries in directory
order. There are two fairly likely possibilities: an A A B B C C D D order
or an A B C D A B C D one. It would be a useful clue to where the bug is.

[.... various ways of making the problem go away snipped ...]

> Why is this problem happening? How do I stop this from happening?
>
> Netapp suggests using snoop to try and capture this problem, but they have
> not told us what they are looking for.

I would imagine they want to see what the READDIR(PLUS) responses to the
client actually look like, to distinguish between an ONTAP bug (filer
duplicating the entries in its response) and a client one (possibly
an incorrect replay from the directory cache).

> We are also seeing this issue on more than one Solaris 8 box. Has anyone
> seen this issue before?

Certainly I haven't, and we use Solaris 8 clients. Some obvious questions:
which ONTAP release are you using? What patches are on your Solaris 8 clients,
and especially which versions of the kernel patch (108528-xx) and NFS patch
(108727-xx)?

Chris Thompson
Email: cet1@cam.ac.uk