Mailing List Archive: Can an indexreader/indexsearcher survive index edits?

Hi

Lucene 8.6.3

In a prototype application I build a Lucene index with a single process and
query it with another. Every operation is a new process. When the data
changes I simply recreate the index and future searches pick up the new
index. Of course performance is sub-optimal.

So I am changing this so that after the initial build subsequent data
changes will update the index rather than rebuilding the entire index.

I am also changing the search method so that I have a single service which
creates an IndexReader and IndexSearcher at startup, and reads and responds
to search requests through a socket.

I know that an existing index can be maintained with selective deletions and
additions, but I am not sure if the process holding the reader and searcher
objects can continue running without having to close and recreate them when
the index is modified.

Is it safe to do that?

cheers

T

Hi,

If you continue to use your code without any changes your searcher should
still work but it won't return newly indexed documents or reflect deletes.
You can consider using a SearcherManager in your searching process and
periodically (use a thread maybe?) ask it to `maybeRefresh()`. Then the
next time you call `acquire()` on this searcherManager, you will get an
updated Searcher that can reflect the new incremental changes the other
thread has made on the index.

Useful references: Search using a SearcherManager (has a code example
similar to your situation)
<https://blog.mikemccandless.com/2011/09/lucenes-searchermanager-simplifies.html>
Near real time search with a SearcherManager (faster than the above
approach)
<https://blog.mikemccandless.com/2011/11/near-real-time-readers-with-lucenes.html>
Similar stackoverflow question
<https://stackoverflow.com/questions/45275557/lucene-near-real-time-search>

-
Gautam Worah.

On Wed, Sep 22, 2021 at 1:10 PM Trevor Nicholls <trevor@castingthevoid.com>
wrote:

> Hi
>
>
>
> Lucene 8.6.3
>
>
>
> In a prototype application I build a Lucene index with a single process and
> query it with another. Every operation is a new process. When the data
> changes I simply recreate the index and future searches pick up the new
> index. Of course performance is sub-optimal.
>
>
>
> So I am changing this so that after the initial build subsequent data
> changes will update the index rather than rebuilding the entire index.
>
> I am also changing the search method so that I have a single service which
> creates an IndexReader and IndexSearcher at startup, and reads and responds
> to search requests through a socket.
>
>
>
> I know that an existing index can be maintained with selective deletions
> and
> additions, but I am not sure if the process holding the reader and searcher
> objects can continue running without having to close and recreate them when
> the index is modified.
>
> Is it safe to do that?
>
>
>
> cheers
>
> T
>
>
>
>