Mailing List Archive

Garrett,

We use NFS over TCP over 5000KM (from RTP to SJ) but only for bare
necessities. We place as much replicated content as possible at each
site and only go over the WAN for things that cannot be replicated.

I would not recommend primary NFS storage over a WAN unless latency is
VERY low (i.e. < 5ms). Depending on the application, bandwidth may or
may not be as important. For homedirs, bandwidth is not a big deal
usually. For large data archives, of course that would be a different
story.

/Brian/

On Mon, 2002-12-09 at 09:45, Garrett Burke wrote:
> Hi All,
>
> I'm wondering if anyone out there has experience with running NFS with
> filers over long distances - of the order of 10s of KM.
>
> I'm particularly interested to find out experiences when bandwidth was and
> wasn't an issue. For example, running at 34Mbs/155Mbs against running over
> GigE. Was tweaking the NFS request size required? Was TCP used and if so
> was it because pkt loss was impacting NFS performance? Did latency become
> an issue?
>
> Thanks,
> GB
>
>
> ----
> Garrett Burke,
> Service Implementation & Support Manager,
> Eircomnet.
> http://www.eircom.net
>

On Mon, 9 Dec 2002, Garrett Burke wrote:
>
> I'm wondering if anyone out there has experience with running NFS with
> filers over long distances - of the order of 10s of KM.

Not recently, but we currently have an electronic fax application
that runs on a W2K server in Calgary, saves decoded TIFFs over CIFS to
a filer in Toronto, where a UNIX box runs ImageMagick over it and does
funky stuff to present it to the user via Apache. CIFS seems to be
chattier and much more finicky than NFS in terms of less-than-ideal
network conditions too. I don't know what the wire distance is, but
it is probably in the 3000 km range.

> I'm particularly interested to find out experiences when bandwidth
> was and wasn't an issue. For example, running at 34Mbs/155Mbs
> against running over GigE. Was tweaking the NFS request size
> required? Was TCP used and if so was it because pkt loss was
> impacting NFS performance? Did latency become an issue?

I find that bandwidth tends to be less of an issue than latency
and packet loss. With 6000 km for a round trip, speed of light
becomes noticeable (20 ms right there). At only 10 km, you should be
able to run Gigabit Ethernet over fiber without too much complexity
slowing things down... sub-millisecond latency should be possible,
although I don't have anything in that configuration right now to back
up that assertion.
--
Brian Tao (BT300, taob@risc.org)
"Though this be madness, yet there is method in't"