Mailing List Archive: No space left on device Error using 0.20.03

Anyone seen this error:

Error in function kino_FSFileDes_fdwrite at
c_src/KinoSearch/Store/FSFileDes.c:100: File operation failed: No space
left on device
at
/usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi/KinoSearch/Index/CompoundFileWriter.pm
line 52

KinoSearch::Index::CompoundFileWriter::finish('KinoSearch::Index::CompoundFileWriter=HASH(0x23c24a24)')
called at
/usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi/KinoSearch/Index/SegWriter.pm
line 194

KinoSearch::Index::SegWriter::finish('KinoSearch::Index::SegWriter=HASH(0xb63ff978)')
called at
/usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi/KinoSearch/InvIndexer.pm
line 209

KinoSearch::InvIndexer::finish('KinoSearch::InvIndexer=HASH(0xb5c1a564)',
'optimize', 1) called at email_indexer.pl line 55

There are around 9.6 GB of files in the index directory and the
partition is only 65% used.

On May 18, 2007, at 10:31 AM, Roger Dooley wrote:

> Error in function kino_FSFileDes_fdwrite at c_src/KinoSearch/Store/
> FSFileDes.c:100: File operation failed: No space left on device

I suspect this happened for real. Here's line 100 of FSFileDes.c:

CHECK_IO_OP( fflush(self->fhandle) ); /* TODO -- kill this? */

The TODO comment refers to whether or not it makes sense to let the
system determine when to flush its own streams. It wouldn't have
affected this.

The CHECK_IO_OP macro is just the error-catching mechanism that threw
the error.

The largest amount of data that could have suddenly been flushed to
disk via fflush() is the amount held by the standard FILE* object
self->fhandle. That's no more than a few k.

> There are around 9.6 GB of files in the index directory and the
> partition is only 65% used.

That's big enough. I notice from elsewhere in the stack trace that
you had optimize set to 1.

KinoSearch::InvIndexer::finish('KinoSearch::InvIndexer=HASH
(0xb5c1a564)', 'optimize', 1) called at email_indexer.pl line 55

That's guarantees that all existing segments will be rewritten to the
latest segment. They need to be copied over -- so you need scratch
space of at least double the index size (bigger, actually, due to how
CompoundFileWriter works). It can also happen when optimize is not
set to 1 and the algo decides it's time to merge all segments, but
seeing optimize => 1 tells us for sure.

You're gonna need a bigger disk.

The only question is why the disk wasn't left full at the end. I'd
suspect the phantom space was occupied by a file that got unlinked
and was released when the process quit. But I'm not sure where that
would come from, because Invindexer->_purge_unused doesn't happen
until after writing completes.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/