I am not a Lucene expert but I would like to understand the threading issues
also, and I'm wondering if the following is true when using Lucene in a
multithreaded application.
I understand there are three modes for using IndexReader and IndexWriter:
A- IndexReader for reading only, not deleting
B- IndexReader for deleting (and reading)
C- IndexWriter (for adding and optimizing)
Any number of readers may be used concurrently in mode A. But for B and C
the reader or writer may not be kept open for long periods. Write
operations create a lock, and closing the reader or writer is the only way
to release the lock. In theory a single writer could be kept open, but its
lock will prevent deletions (which are performed with a separate reader).
Therefore for B and C each set of changes should be made inside a
synchronized block where the reader or writer is opened and closed. This
prevents multiple writers (or readers used for deleting) from being open at
once. The synchronization should be done on an object that identifies a
particular index, e.g., on a global object if there is only one index. For
example:
class myindex {
static final Object INDEX_LOCK = new Object();
void delete(int[] docs) {
synchronized (INDEX_LOCK) {
IndexReader reader = IndexReader.open(...);
try {
for (int i = 0; i < docs.length; i++) {
reader.delete(docs[i]);
}
} finally {
reader.close();
}
}
}
void add(Document[] docs) {
synchronized (INDEX_LOCK) {
IndexWriter writer = new IndexWriter(...);
try {
for (int i = 0; i < docs.length; i++) {
writer.add(docs[i]);
}
writer.optimize();
} finally {
writer.close();
}
}
}
}
Of course there are other techniques for global locking such as 'static
synchronized' methods. Locking on a separate object per index is the
general case (where multiple indexes are present).
Is this correct? Or should Lucene be waiting on the write lock instead of
throwing an exception?
mark
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
> Sent: Sunday, February 24, 2002 9:22 PM
> To: Lucene Users List
> Subject: RE: Index Locked For Write
>
>
>
> --- "Howk, Michael" <MHowk@FSC.Follett.com> wrote:
> > Out of curiosity, why didn't we need to close the writer in rc2 or
> > rc3?
> >
> > When you suggest a "synchronized keyword", are you suggesting that
> > the
> > writer is not inherently thread-safe? Do we need to write our own
> > thread
> > management on top of Lucene?
>
> Sorry, that might have been a wrong suggestion, IndexWriter (at least
> the add method) is supposed to be thread safe.
>
> Otis
>
>
> > -----Original Message-----
> > From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
> > Sent: Thursday, February 21, 2002 4:07 PM
> > To: Lucene Users List
> > Subject: RE: Index Locked For Write
> >
> >
> > You could use synchronized keyword and use IndexReader.isLocked() or
> > something like that, no?
> >
> > Otis
> >
> > --- "Howk, Michael" <MHowk@FSC.Follett.com> wrote:
> > > Thank you for your quick responses. But in our application, we're
> > > working in
> > > a transactional environment where multiple threads are accessing a
> > > single
> > > writer using the recommended singleton pattern. Since no
> thread has
> > > exclusive access to the writer, how can we have one thread
> > > arbitrarily
> > > decide to close the writer?
> > >
> > > Michael
> > >
> > > -----Original Message-----
> > > From: Mark Tucker [mailto:MTucker@infoimage.com]
> > > Sent: Thursday, February 21, 2002 3:51 PM
> > > To: Lucene Users List
> > > Subject: RE: Index Locked For Write
> > >
> > >
> > > You forgot to close your writer after the call to optimize.
> > >
> > > -----Original Message-----
> > > From: Howk, Michael [mailto:MHowk@FSC.Follett.com]
> > > Sent: Thursday, February 21, 2002 2:49 PM
> > > To: Lucene Mailing List (E-mail)
> > > Subject: Index Locked For Write
> > >
> > >
> > > We just got the newest daily build (to try to fix some NullPointer
> > > errors
> > > with "?" and "_" characters), and we're getting the same problem
> > that
> > > Daniel
> > > Calvo mentioned: Index Locked for Write. Here's basically what our
> > > code is
> > > doing:
> > > IndexWriter writer = new IndexWriter(path,
> analyzer, create);
> > > try {
> > > Document doc = new Document();
> > > doc.add(Field.Keyword("DOC_ID", "14"));
> > > doc.add(Field.UnStored("ANY", "mushu"));
> > > writer.addDocument(doc);
> > > writer.optimize();
> > >
> > > // Search the document for our keyword
> > > {
> > > IndexReader reader = IndexReader.open(path);
> > > IndexSearcher searcher = new IndexSearcher(reader);
> > > Vector returnStuff = searcher.search("mushu");
> > > }
> > >
> > > // Verify that we got one record back
> > > assertNotNull(returnStuff);
> > > assertEquals(1, returnStuff.size());
> > > }
> > > finally {
> > > // Clean up after ourselves
> > > IndexReader reader = IndexReader.open(path);
> > > reader.delete(new Term("DOC_ID", "14"));
> > > reader.close();
> > > }
> > >
> > > And the exception we're getting on the "reader.delete" line in the
> > > finally
> > > clause:
> > >
> > > java.io.IOException: Index locked for write:
> > > Lock@C:\devtools\JBossTomcat\jboss\indexes\marc\write.lock at
> > >
> >
> sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer
> (StreamRemoteC
> > > all.java:245) at
> > >
> >
> sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCal
> l.java:220)
> > > at
> > > sun.rmi.server.UnicastRef.invoke(UnicastRef.java:122) at
> > >
> >
> org.jboss.ejb.plugins.jrmp.server.JRMPContainerInvoker_Stub.in
> voke(Unknown
> > > Source) at
> > >
> >
> org.jboss.ejb.plugins.jrmp.interfaces.GenericProxy.invokeConta
> iner(GenericPr
> > > oxy.java:357) at
> > >
> >
> org.jboss.ejb.plugins.jrmp.interfaces.StatelessSessionProxy.in
> voke(Stateless
> > > SessionProxy.java:123) at
> > > $Proxy5.deleteDocument(Unknown Source)
> > >
> > > Are we using the right approach? Any suggestions? Thank you.
> > >
> > > Michael Howk
> > >
> > > --
> > > To unsubscribe, e-mail:
> > > <mailto:lucene-user-unsubscribe@jakarta.apache.org>
> > > For additional commands, e-mail:
> > > <mailto:lucene-user-help@jakarta.apache.org>
> > >
> > >
> > > --
> > > To unsubscribe, e-mail:
> > > <mailto:lucene-user-unsubscribe@jakarta.apache.org>
> > > For additional commands, e-mail:
> > > <mailto:lucene-user-help@jakarta.apache.org>
> > >
> > > --
> > > To unsubscribe, e-mail:
> > > <mailto:lucene-user-unsubscribe@jakarta.apache.org>
> > > For additional commands, e-mail:
> > > <mailto:lucene-user-help@jakarta.apache.org>
> > >
> >
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Yahoo! Sports - Coverage of the 2002 Olympic Games
> > http://sports.yahoo.com
> >
> > --
> > To unsubscribe, e-mail:
> > <mailto:lucene-user-unsubscribe@jakarta.apache.org>
> > For additional commands, e-mail:
> > <mailto:lucene-user-help@jakarta.apache.org>
> >
> > --
> > To unsubscribe, e-mail:
> > <mailto:lucene-user-unsubscribe@jakarta.apache.org>
> > For additional commands, e-mail:
> > <mailto:lucene-user-help@jakarta.apache.org>
> >
>
>
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Sports - Coverage of the 2002 Olympic Games
> http://sports.yahoo.com
>
> --
> To unsubscribe, e-mail:
> <mailto:lucene-user-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail:
> <mailto:lucene-user-help@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>