Mailing List Archive: NFS performance not good enough?

NFS performance not good enough?

Jun 5, 2014, 12:15 AM

Post #1 of 4 (1413 views)

I have a problem with my recordings not finishing. On examining the
logs I see that it is due to disk performance. I am writing to a disk
over NFS. It should be able to take it, I thought, and these problems
have only started recently.

(See below for my logs and why I thing this is the problem)

So I have 2 questions for this list:

(1) Should I expect NFS to manage? Searching the internet does not turn
up many other people having problems. I have got file caching turned on
(using the 'fsc' option and cachefilesd. (I'm turning that off now to
see if it helps)

(2) I have a local 32G ssd drive that is hardly used. Can I get Myth to
record to it and move the recordings later as a scheduled job to the NFS
system?

For example I recorded Back Benches and at 16:17 the recording just stops.

The recording started at 22:29

The log said (mythbackend.log):

Jun 4 22:29:00 gently mythbackend[1425]: I Scheduler scheduler.cpp:2514
(HandleRecordingStatusChange) Tuning recording: "Back Benches": channel
1010
on cardid 5, sourceid 1
Jun 4 22:29:01 gently mythbackend[1425]: I CoreContext
scheduler.cpp:637 (UpdateRecStatus) Updating status for "Back Benches"
on cardid 5 (Tuning =>
Recording)
Jun 4 22:29:01 gently mythbackend[1425]: I TVRecEvent tv_rec.cpp:3989
(TuningNewRecorder) TVRec(5): rec->GetPathname():
'/media/Recordings/1010_20140604222900.mpg'

I assume it is writing to 1010_20140604222900

I get a few messages like:

Jun 4 22:42:50 gently mythbackend[1425]: W TFWWrite
ThreadedFileWriter.cpp:499 (DiskLoop)
TFW(/media/Recordings/1010_20140604222900.mpg:62): write(58280) cnt 90
total 5226776 -- took a long time, 14230 ms

Then the last mention of the file is:

Jun 4 22:51:33 gently mythbackend[1425]: E DVBRead
ThreadedFileWriter.cpp:216 (Write)
TFW(/media/Recordings/1010_20140604222900.mpg:62): Maximum buffer size
exceeded.#012#011#011#011file will be truncated, no further writing will
be done.#012#011#011#011This generally indicates your disk performance
#012#011#011#011is insufficient to deal with the number of on-going
#012#011#011#011recordings, or you have a disk failure.
Jun 4 22:51:40 gently mythbackend[1425]: W TFWWrite
ThreadedFileWriter.cpp:499 (DiskLoop)
TFW(/media/Recordings/1010_20140604222900.mpg:62): write(58092) cnt 2308
total 134209816 -- took a long time, 16819 ms

Worik

--
The only true evil is turning people into things....
Granny Weatherwax
worik.stanton@gmail.com 021-1680650, (03) 4821804
Aotearoa (New Zealand)

Re: NFS performance not good enough? [ In reply to ]

pieter at insync

Jun 5, 2014, 12:28 AM

Post #2 of 4 (1372 views)

Permalink

Hi,

On 5/06/2014 19:15, Worik Stanton wrote:
> I have a problem with my recordings not finishing. On examining the
> logs I see that it is due to disk performance. I am writing to a disk
> over NFS. It should be able to take it, I thought, and these problems
> have only started recently.
>
> (See below for my logs and why I thing this is the problem)
>
> So I have 2 questions for this list:
>
> (1) Should I expect NFS to manage? Searching the internet does not turn
> up many other people having problems. I have got file caching turned on
> (using the 'fsc' option and cachefilesd. (I'm turning that off now to
> see if it helps)
It should manage, but you need to tell us *way* more about your setup.
> (2) I have a local 32G ssd drive that is hardly used. Can I get Myth to
> record to it and move the recordings later as a scheduled job to the NFS
> system?
Personally, I would avoid this as the ssd will wear out
> For example I recorded Back Benches and at 16:17 the recording just stops.
>
> The recording started at 22:29
>
> The log said (mythbackend.log):
>
> Jun 4 22:29:00 gently mythbackend[1425]: I Scheduler scheduler.cpp:2514
> (HandleRecordingStatusChange) Tuning recording: "Back Benches": channel
> 1010
> on cardid 5, sourceid 1
> Jun 4 22:29:01 gently mythbackend[1425]: I CoreContext
> scheduler.cpp:637 (UpdateRecStatus) Updating status for "Back Benches"
> on cardid 5 (Tuning =>
> Recording)
> Jun 4 22:29:01 gently mythbackend[1425]: I TVRecEvent tv_rec.cpp:3989
> (TuningNewRecorder) TVRec(5): rec->GetPathname():
> '/media/Recordings/1010_20140604222900.mpg'
>
> I assume it is writing to 1010_20140604222900
>
> I get a few messages like:
>
> Jun 4 22:42:50 gently mythbackend[1425]: W TFWWrite
> ThreadedFileWriter.cpp:499 (DiskLoop)
> TFW(/media/Recordings/1010_20140604222900.mpg:62): write(58280) cnt 90
> total 5226776 -- took a long time, 14230 ms
>
> Then the last mention of the file is:
>
> Jun 4 22:51:33 gently mythbackend[1425]: E DVBRead
> ThreadedFileWriter.cpp:216 (Write)
> TFW(/media/Recordings/1010_20140604222900.mpg:62): Maximum buffer size
> exceeded.#012#011#011#011file will be truncated, no further writing will
> be done.#012#011#011#011This generally indicates your disk performance
> #012#011#011#011is insufficient to deal with the number of on-going
> #012#011#011#011recordings, or you have a disk failure.
> Jun 4 22:51:40 gently mythbackend[1425]: W TFWWrite
> ThreadedFileWriter.cpp:499 (DiskLoop)
> TFW(/media/Recordings/1010_20140604222900.mpg:62): write(58092) cnt 2308
> total 134209816 -- took a long time, 16819 ms
>
>
> Worik
>
>
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.linuxnut.co.nz
> http://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

Re: NFS performance not good enough? [ In reply to ]

jyavenard at gmail

Jun 5, 2014, 3:47 PM

Post #3 of 4 (1367 views)

Permalink

On 5 June 2014 17:15, Worik Stanton <worik.stanton@gmail.com> wrote:
> I have a problem with my recordings not finishing. On examining the
> logs I see that it is due to disk performance. I am writing to a disk
> over NFS. It should be able to take it, I thought, and these problems
> have only started recently.
>
> (See below for my logs and why I thing this is the problem)
>
> So I have 2 questions for this list:
>
> (1) Should I expect NFS to manage? Searching the internet does not turn
> up many other people having problems. I have got file caching turned on
> (using the 'fsc' option and cachefilesd. (I'm turning that off now to
> see if it helps)

I write over NFS to a FreeNAS ZFS array without issues.

What I did find however, is that all the fancy NFS tutorial on how to
increase performance are all wrong, and in fact compromise speed
enormously.
Leave the read/write buffer as they are calculated by the system,
don't play with it.

I mount the NFS share using the following option:
192.168.200.13:/mnt/pool/data/videos/recordings /external/recordings
nfs proto=tcp,timeo=14,noatime,actimeo=1 0 0

that is it.. e.g. leave it to the system to calculate buffers, and
don't update the creation and last modification time

I use gigabit jumbo frame on the dedicated link between my myth
backend and the NFS server, but it actually makes little difference in
my experience

I do over 100MB/s writing to the server, which it more than enough to
record a dozen HD stream simultaneously

>
> Jun 4 22:51:33 gently mythbackend[1425]: E DVBRead
> ThreadedFileWriter.cpp:216 (Write)
> TFW(/media/Recordings/1010_20140604222900.mpg:62): Maximum buffer size
> exceeded.#012#011#011#011file will be truncated, no further writing will
> be done.#012#011#011#011This generally indicates your disk performance
> #012#011#011#011is insufficient to deal with the number of on-going
> #012#011#011#011recordings, or you have a disk failure.
> Jun 4 22:51:40 gently mythbackend[1425]: W TFWWrite
> ThreadedFileWriter.cpp:499 (DiskLoop)
> TFW(/media/Recordings/1010_20140604222900.mpg:62): write(58092) cnt 2308
> total 134209816 -- took a long time, 16819 ms
>
make sure you are running the latest fixes/0.27 build, several
improvements were made in this area.

but yes, it does appear your disk is too slow

_______________________________________________
mythtvnz mailing list
mythtvnz@lists.linuxnut.co.nz
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

Re: NFS performance not good enough? [ In reply to ]

worik.stanton at gmail

Jun 5, 2014, 5:20 PM

Post #4 of 4 (1374 views)

Permalink

On 05/06/14 19:28, Pieter De Wit wrote:
> Hi,
[snip]

>

>>
>> (1) Should I expect NFS to manage? Searching the internet does not turn
>> up many other people having problems. I have got file caching turned on
>> (using the 'fsc' option and cachefilesd. (I'm turning that off now to
>> see if it helps)
> It should manage, but you need to tell us *way* more about your setup.

Cool. That is a good initial reaction. "NFS is unsuitable for HD
files" would get me woried.

On closer examination my NAS is poked. Which explains a lot. When I
came to benchmark the NFS share it has zero performance. That explains
a lot.

sigh
W

>> (2) I have a local 32G ssd drive that is hardly used. Can I get Myth to
>> record to it and move the recordings later as a scheduled job to the NFS
>> system?
> Personally, I would avoid this as the ssd will wear out
>> For example I recorded Back Benches and at 16:17 the recording just
>> stops.
>>
>> The recording started at 22:29
>>
>> The log said (mythbackend.log):
>>
>> Jun 4 22:29:00 gently mythbackend[1425]: I Scheduler scheduler.cpp:2514
>> (HandleRecordingStatusChange) Tuning recording: "Back Benches": channel
>> 1010
>> on cardid 5, sourceid 1
>> Jun 4 22:29:01 gently mythbackend[1425]: I CoreContext
>> scheduler.cpp:637 (UpdateRecStatus) Updating status for "Back Benches"
>> on cardid 5 (Tuning =>
>> Recording)
>> Jun 4 22:29:01 gently mythbackend[1425]: I TVRecEvent tv_rec.cpp:3989
>> (TuningNewRecorder) TVRec(5): rec->GetPathname():
>> '/media/Recordings/1010_20140604222900.mpg'
>>
>> I assume it is writing to 1010_20140604222900
>>
>> I get a few messages like:
>>
>> Jun 4 22:42:50 gently mythbackend[1425]: W TFWWrite
>> ThreadedFileWriter.cpp:499 (DiskLoop)
>> TFW(/media/Recordings/1010_20140604222900.mpg:62): write(58280) cnt 90
>> total 5226776 -- took a long time, 14230 ms
>>
>> Then the last mention of the file is:
>>
>> Jun 4 22:51:33 gently mythbackend[1425]: E DVBRead
>> ThreadedFileWriter.cpp:216 (Write)
>> TFW(/media/Recordings/1010_20140604222900.mpg:62): Maximum buffer size
>> exceeded.#012#011#011#011file will be truncated, no further writing will
>> be done.#012#011#011#011This generally indicates your disk performance
>> #012#011#011#011is insufficient to deal with the number of on-going
>> #012#011#011#011recordings, or you have a disk failure.
>> Jun 4 22:51:40 gently mythbackend[1425]: W TFWWrite
>> ThreadedFileWriter.cpp:499 (DiskLoop)
>> TFW(/media/Recordings/1010_20140604222900.mpg:62): write(58092) cnt 2308
>> total 134209816 -- took a long time, 16819 ms
>>
>>
>> Worik
>>
>>
>>
>> _______________________________________________
>> mythtvnz mailing list
>> mythtvnz@lists.linuxnut.co.nz
>> http://lists.ourshack.com/mailman/listinfo/mythtvnz
>> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>
>
>
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz@lists.linuxnut.co.nz
> http://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/
>

--
The only true evil is turning people into things....
Granny Weatherwax
worik.stanton@gmail.com 021-1680650, (03) 4821804
Aotearoa (New Zealand)

Mailing List Archive

Attached Files:

Attached Files: