Mailing List Archive

chris@psychofx.com (Chris Miles) writes
>
> Hi all, are there any limits to the maximum number of files (or
> subdirectories) a Netapp NFS mounted directory can handle?
>
> In our case (if it matters) we are looking at Linux 2.4.18 using NFS
> v3/tcp to a Netapp running 6.1.2R3.
>
> We are thinking of a directory with 500,000+ files in it.

Look at "options wafl.maxdirsize" (or, in more recent ONTAP versions,
"vol options maxdirsize"). The default is 10 MB, and a directory that
size will hold 327680 entries if the names are small enough: less
than that otherwise.

You can increase the value, but directories this size are not really
a very good idea, even with the ONTAP fast lookup enhancements.

Whether the Linux client matters depends on what it thinks of caching
NFS readdir results of that size. Not with great enthusiasm, I would
guess. :-)

Does your application really need a flat directory structure this wide?

Chris Thompson
Email: cet1@cam.ac.uk

On Wed, Aug 28, 2002 at 06:03:38PM +0100, Chris Thompson wrote:
> Look at "options wafl.maxdirsize" (or, in more recent ONTAP versions,
> "vol options maxdirsize"). The default is 10 MB, and a directory that
> size will hold 327680 entries if the names are small enough: less
> than that otherwise.

Great, thanks.

> You can increase the value, but directories this size are not really
> a very good idea, even with the ONTAP fast lookup enhancements.
>
> Whether the Linux client matters depends on what it thinks of caching
> NFS readdir results of that size. Not with great enthusiasm, I would
> guess. :-)
>
> Does your application really need a flat directory structure this wide?

Well that is being debated, but at least I know it is technically
possible. If it gets to that we'll do some performance testing.

Cheers,
CM

--
Chris Miles
chris@psychofx.com
http://www.psychofx.com/chris/

I remember we had a problem with Linux (probably RedHat 7.1) where we could
only access 32767 files before things started failing. I believe we found
it was a limitation on the Linux side, not a NetApp issues.

-dale
----- Original Message -----
From: "Chris Miles" <chris@psychofx.com>
To: <toasters@mathworks.com>
Sent: Wednesday, August 28, 2002 1:00 PM
Subject: limit to number of files in a directory


> Hi all, are there any limits to the maximum number of files (or
> subdirectories) a Netapp NFS mounted directory can handle?
>
> In our case (if it matters) we are looking at Linux 2.4.18 using NFS
> v3/tcp to a Netapp running 6.1.2R3.
>
> We are thinking of a directory with 500,000+ files in it.
>
> Cheers,
> CM
>
> --
> Chris Miles
> chris@psychofx.com
> http://www.psychofx.com/chris/
>

Hmmm.... another thought. You asked:

> Hi all, are there any limits to the maximum number of files (or
> subdirectories) a Netapp NFS mounted directory can handle?

There's a bound specifically on the number of subdirectories that
I didn't mention. The link count on the parent directory is limited
to 65535, which means no more than 65533 subdirectories. (Some Unix
filing systems use signed shorts for the link count, but ONTAP uses
unsigned ones.)

Chris Thompson
Email: cet1@cam.ac.uk

We have noticed similar problems with large directories on a NetApp
filer. The resolution involved emptying the directory, removing it and
recreating it. That brings performance back to normal.

No changes were on the client side so we have concluded that the issue
was with the Filer.

--Walter

Dale Gass wrote:

>I remember we had a problem with Linux (probably RedHat 7.1) where we could
>only access 32767 files before things started failing. I believe we found
>it was a limitation on the Linux side, not a NetApp issues.
>
>-dale
>----- Original Message -----
>From: "Chris Miles" <chris@psychofx.com>
>To: <toasters@mathworks.com>
>Sent: Wednesday, August 28, 2002 1:00 PM
>Subject: limit to number of files in a directory
>
>
>
>
>>Hi all, are there any limits to the maximum number of files (or
>>subdirectories) a Netapp NFS mounted directory can handle?
>>
>>In our case (if it matters) we are looking at Linux 2.4.18 using NFS
>>v3/tcp to a Netapp running 6.1.2R3.
>>
>>We are thinking of a directory with 500,000+ files in it.
>>
>>Cheers,
>>CM
>>
>>--
>>Chris Miles
>>chris@psychofx.com
>>http://www.psychofx.com/chris/
>>
>>