Mailing List Archive

Sorry to bother y'all again.
Found an answer in the archives under the Thread "Indexing problem".

About to try using RAMDirectory first.

pax et bonum. p.

----- Original Message -----
From: "Paul Friedman" <paulsanfordfriedman@yahoo.com>
To: "Lucene Users List" <lucene-user@jakarta.apache.org>
Sent: Thursday, February 28, 2002 1:23 PM
Subject: optimizing index - too many open files


> Hello all,
>
> I am running into an error:
> java.io.FileNotFoundException: /lucene/index/_2vx.tii ( too many open
> files )
> after my class calls IndexWriter.optimize().
>
> Does anybody know what causes this error?
> Any help is appreciated.
>
> ( By the way, the site that I am indexing is huge.
> I have a crawler run through the site calling many .jsps, .pdfs, and
.html
> docs.
> It ran fine two days ago after indexing 3700+ pages. )
>
> Could the index be too large for Lucene to handle?
>
> The error:
> java.io.FileNotFoundException: /lucene/index/-2vx.tii ( too many open
> files )
> at java.io.RandomAccessFile.open( Native Method )
> at java.io.RandomAccessFile.<init>
> at java.io.RandomAccessFile.<init>
> at org.apache.lucene.store.FSInputStream$Descriptor.<init>
> at org.apache.lucene.store.FSInputStream.<init>
> at org.apache.lucene.store.FSDirectory.openFile
> at org.apache.lucene.index.TermInfosReader.readIndex
> at org.apache.lucene.index.TermInfosReader.<init>
> at org.apache.lucene.index.SegmentReader.<init>
> at org.apache.lucene.index.IndexWriter.mergeSegments
> at org.apache.lucene.index.IndexWriter.optimize
>
>
> _________________________________________________________
> Do You Yahoo!?
> Get your free @yahoo.com address at http://mail.yahoo.com
>
>
> --
> To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>


_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com


--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>

I have to index 1650mb of documents, and eventually i will get "out of
memory" with a RAMDir and get "too many open files" with a FSdir, so to get
around this i am indexing 100 documents at a time in a RAMDir, then merge
this RAMDir into a FSDir before i index the next set of 100 files. This made
me work around both "out of memory" and "too many files" exceptions...

mvh karl øie



-----Original Message-----
From: Paul Friedman [mailto:paulsanfordfriedman@yahoo.com]
Sent: 28. februar 2002 21:38
To: Lucene Users List
Subject: Re: optimizing index - too many open files


Sorry to bother y'all again.
Found an answer in the archives under the Thread "Indexing problem".

About to try using RAMDirectory first.

pax et bonum. p.

----- Original Message -----
From: "Paul Friedman" <paulsanfordfriedman@yahoo.com>
To: "Lucene Users List" <lucene-user@jakarta.apache.org>
Sent: Thursday, February 28, 2002 1:23 PM
Subject: optimizing index - too many open files


> Hello all,
>
> I am running into an error:
> java.io.FileNotFoundException: /lucene/index/_2vx.tii ( too many open
> files )
> after my class calls IndexWriter.optimize().
>
> Does anybody know what causes this error?
> Any help is appreciated.
>
> ( By the way, the site that I am indexing is huge.
> I have a crawler run through the site calling many .jsps, .pdfs, and
.html
> docs.
> It ran fine two days ago after indexing 3700+ pages. )
>
> Could the index be too large for Lucene to handle?
>
> The error:
> java.io.FileNotFoundException: /lucene/index/-2vx.tii ( too many open
> files )
> at java.io.RandomAccessFile.open( Native Method )
> at java.io.RandomAccessFile.<init>
> at java.io.RandomAccessFile.<init>
> at org.apache.lucene.store.FSInputStream$Descriptor.<init>
> at org.apache.lucene.store.FSInputStream.<init>
> at org.apache.lucene.store.FSDirectory.openFile
> at org.apache.lucene.index.TermInfosReader.readIndex
> at org.apache.lucene.index.TermInfosReader.<init>
> at org.apache.lucene.index.SegmentReader.<init>
> at org.apache.lucene.index.IndexWriter.mergeSegments
> at org.apache.lucene.index.IndexWriter.optimize
>
>
> _________________________________________________________
> Do You Yahoo!?
> Get your free @yahoo.com address at http://mail.yahoo.com
>
>
> --
> To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>


_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com


--
To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>


--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>