Mailing List Archive

IndexReader and IndexWriter on the same index
In the FAQ in Searching question 41, it says about index
modification: "The problems are only when you add documents or
optimize an index, and then search with an IndexReader that was
constructed before those changes to the index were made."

I'm in a situation where before I add any document I need to remove
any old versions of it from the index. I'd like to keep both an
IndexReader and IndexWriter around for efficiency's sake, but I'm
concerned about them interacting properly. Does the delete(Term)
member function count as a "search" in the above statement? Can I
delete with an IndexReader, add with an IndexWriter, delete with that
IndexReader, add with that IndexWriter, etc. without concern, or must
I recreate one (or both) of them each time?

Thanks.

Avi
--
Avi Drissman
avi_drissman@baseview.com
Bit bashing since 1977

--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>
RE: IndexReader and IndexWriter on the same index [ In reply to ]
If you are performing additions and deletions then you should serially
create an IndexReader to do deletions, close it, then create an IndexWriter
to do additions, close it, and so on. Note that typically one will use a
different IndexReader for deletions than is used for searching, so that
searches may continue while adding documents.

Doug

> -----Original Message-----
> From: Avi Drissman [mailto:avi_drissman@baseview.com]
> Sent: Tuesday, November 27, 2001 1:47 PM
> To: Lucene Users List
> Subject: IndexReader and IndexWriter on the same index
>
>
> In the FAQ in Searching question 41, it says about index
> modification: "The problems are only when you add documents or
> optimize an index, and then search with an IndexReader that was
> constructed before those changes to the index were made."
>
> I'm in a situation where before I add any document I need to remove
> any old versions of it from the index. I'd like to keep both an
> IndexReader and IndexWriter around for efficiency's sake, but I'm
> concerned about them interacting properly. Does the delete(Term)
> member function count as a "search" in the above statement? Can I
> delete with an IndexReader, add with an IndexWriter, delete with that
> IndexReader, add with that IndexWriter, etc. without concern, or must
> I recreate one (or both) of them each time?
>
> Thanks.
>
> Avi
> --
> Avi Drissman
> avi_drissman@baseview.com
> Bit bashing since 1977
>
> --
> To unsubscribe, e-mail:
> <mailto:lucene-user-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail:
> <mailto:lucene-user-help@jakarta.apache.org>
>

--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>
RE: IndexReader and IndexWriter on the same index [ In reply to ]
At 2:14 PM -0800 11/27/01, you wrote:

>If you are performing additions and deletions then you should serially
>create an IndexReader to do deletions, close it, then create an IndexWriter
>to do additions, close it, and so on.

But if I need to do a deletion before every addition, then there's
the overhead of all those reader and writer creations. There's no way
around it?

Avi

--
Avi Drissman
avi_drissman@baseview.com
Bit bashing since 1977

--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>
RE: IndexReader and IndexWriter on the same index [ In reply to ]
> From: Avi Drissman [mailto:avi_drissman@baseview.com]
>
> But if I need to do a deletion before every addition, then there's
> the overhead of all those reader and writer creations. There's no way
> around it?

Batch your deletions, then batch your additions. If you need these changes
to appear atomically to searchers, do not re-open the IndexReader that
you're using for searching until you have closed the IndexWriter.

Doug

--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>
Re: IndexReader and IndexWriter on the same index [ In reply to ]
Doug Cutting wrote:
>
> > From: Avi Drissman [mailto:avi_drissman@baseview.com]
> >
> > But if I need to do a deletion before every addition, then there's
> > the overhead of all those reader and writer creations. There's no way
> > around it?
>
> Batch your deletions, then batch your additions. If you need these changes
> to appear atomically to searchers, do not re-open the IndexReader that
> you're using for searching until you have closed the IndexWriter.
>
> Doug
>
I am faced with a similar problem to Avi and I have a question :

I have multiple threads peforming inserts, and searches on an index.
The IndexReader/IndexSearcher instances are refreshed when a search
thread detects that the modification time of the index changes (before
IndexSearcher.search() is called).
Refresh includes :
o synchcronise on all readers and writers.
o peform any queued deletes using IndexReader.delete().
o call IndexReader.close() to flush deletes to the index.
o open new IndexReader/IndexSearcher.

Do I also need to re-create the IndexWriter instance?

Many Thanks for any answers!

Nick Smith

--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>