Mailing List Archive

Lucene Index Writer in a distributed system
Hello Team,

I am new to Lucene and want to use Lucene in a distributed system to write
in a Amazon EFS index.

As per my understanding, the index writer for a particular index needs to
be opened by 1 server only. Is there a way we can achieved this in
distributed system to write parallelly in Lucence

Any document/article where i can read about this problem will be of great
help, Many thanks in advance!!

Gopal
Re: Lucene Index Writer in a distributed system [ In reply to ]
Hi Gopal,

Indeed, for a single Lucene index, only one writer may be open at a time.
Lucene tries to catch you if you mess this up, using file-based locking.

If you really need concurrent indexing, you could have N IndexWriters each
writing into a private Directory, and then periodically use addIndexes to
"reduce" these indices down into a single index which you then use for
searching.

Mike McCandless

http://blog.mikemccandless.com


On Thu, Oct 19, 2023 at 5:49?AM Gopal Sharma <gopal.sharma@algonomy.com>
wrote:

> Hello Team,
>
> I am new to Lucene and want to use Lucene in a distributed system to write
> in a Amazon EFS index.
>
> As per my understanding, the index writer for a particular index needs to
> be opened by 1 server only. Is there a way we can achieved this in
> distributed system to write parallelly in Lucence
>
> Any document/article where i can read about this problem will be of great
> help, Many thanks in advance!!
>
> Gopal
>
Re: Lucene Index Writer in a distributed system [ In reply to ]
Zookeeper, right? Look how Zookeeper is used in Solr, but Zookeeper does exactly what you want, I believe.

Sent from my iPhone

> On Oct 19, 2023, at 3:49 AM, Gopal Sharma <gopal.sharma@algonomy.com> wrote:
>
> ?Hello Team,
>
> I am new to Lucene and want to use Lucene in a distributed system to write
> in a Amazon EFS index.
>
> As per my understanding, the index writer for a particular index needs to
> be opened by 1 server only. Is there a way we can achieved this in
> distributed system to write parallelly in Lucence
>
> Any document/article where i can read about this problem will be of great
> help, Many thanks in advance!!
>
> Gopal

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org