Mailing List Archive

IndexWriter WRITE_LOCK_TIMEOUT
Hi,

We have written a multi-threaded indexing framework that uses Lucene. To avoid contention on a single IndexWriter, we are using multiple writers. We have enountered a situation where we need to check if a writer object exists for a particular index directory and create it if it doesn't. We do this by catching the exception that IndexWriter throws. We found from the code that the WRITE_LOCK_TIMEOUT is one second. This value is a bit high for our requirements.

My query is
* Why is it hard-coded to minimum 1 sec.
* Will anything break if I change to value to be very low, say 5 millisecond and use it.

Thanks

~ Mukul Joshi
Senior Member of Technical Staff
Great Software Laboratory, Pune
Re: IndexWriter WRITE_LOCK_TIMEOUT [ In reply to ]
> We have written a multi-threaded indexing framework that uses
> Lucene. To avoid contention on a single IndexWriter, we are using
> multiple writers. We have enountered a situation where we need to
> check if a writer object exists for a particular index directory and
> create it if it doesn't. We do this by catching the exception that
> IndexWriter throws. We found from the code that the WRITE_LOCK_TIMEOUT
> is one second. This value is a bit high for our requirements.

I'm assuming you're actually creating multiple indices and then one
IndexWriter per index?

> * Why is it hard-coded to minimum 1 sec.

This is just the default value. You can change it by calling
setWriteLockTimeout on your IndexWriter instance.

Oh, I see -- this is too late for you, because that timeout is used when
you first construct the IndexWriter before you have a chance to change it.

Before Lucene 1.9 you could set a system property to achieve this, but
that tie-in was removed in 1.9 (as part of the "don't use system
properties for defaults" cleanup effort).

Furthermore, the fields were marked final, so you can't set them
externally on the class. It feels like we need to add setter/getter for
the static values, eg "static setDefaultWriteLockTimeout()". I'll open
a Jira issue for this.

But as a workaround for your case, you could instead go directly to the
current Lock API to test if anything is holding the write lock against a
given index. For example, call this:

directory.makeLock(IndexWriter.WRITE_LOCK_NAME).isLocked()

And if that returns true then something is holding the write lock. This
call returns immediately (ie, no wait-polling takes place), so there are
no timeouts that apply. Note that if you delete documents from an
IndexReader, that would also hold the write lock.

Mike

Mukul Joshi wrote:
> Hi,
>
> We have written a multi-threaded indexing framework that uses Lucene. To avoid contention on a single IndexWriter, we are using multiple writers. We have enountered a situation where we need to check if a writer object exists for a particular index directory and create it if it doesn't. We do this by catching the exception that IndexWriter throws. We found from the code that the WRITE_LOCK_TIMEOUT is one second. This value is a bit high for our requirements.
>
> My query is
> * Why is it hard-coded to minimum 1 sec.
> * Will anything break if I change to value to be very low, say 5 millisecond and use it.
>
> Thanks
>
> ~ Mukul Joshi
> Senior Member of Technical Staff
> Great Software Laboratory, Pune
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: IndexWriter WRITE_LOCK_TIMEOUT [ In reply to ]
> We found from the code that the WRITE_LOCK_TIMEOUT
> > is one second. This value is a bit high for our requirements.
>
> I'm assuming you're actually creating multiple indices and then one
> IndexWriter per index?

Yes.

Thanks for the help, Michael.

~ Mukul Joshi
Senior Member of Technical Staff
Great Software Laboratory
www.gs-lab.com




---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org