Mailing List Archive

synchronization problem / bug?
hi

while testing the SqlDirectory, I found some really strange thing: scenario
is concurrent writer and searcher:
1. a IndexWriter is started and creates a write.lock until the close method
is called. this cleanly prevents other writers to access the index at the
same time and is ok.
2. go on indexing ...

but now, concurrently, the following process goes on:
1. a Searcher is created with searcher = new IndexSearcher().
2. this process creates a commit.lock as expected and reads some files.
3. the commit.lock is released. (immediately)
4. now, the querying is done and the hits.doc(i) is read. during this, no
commit.lock is set, but again, some files are accessed (the
InputStream.readInternal method is called).
5. the searcher.close() method is called which closes all open InputStreams.
(no commit.lock released or created)

like that, from time to time, a exception occures because the file has been
changed by the IndexWriter process running the same time.

Any ideas about this? this should also occur with FSDirectory or
RAMDirectory, but more rarely, because these are faster in reading
results...

cheers
marc



--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>
RE: synchronization problem / bug? [ In reply to ]
Lucene assumes that if an open file is deleted then it will continue to be
readable at least until it is closed. So if your SQL Directory is
permitting open files to be deleted then you will encounter problems. All
open files should be somehow locked against deletion. If a file deletion
fails Lucene will try again later.

RAMDirectory and FSDirectory implement this correctly.

Doug

> -----Original Message-----
> From: Marc Kramis [mailto:mkramis@student.ethz.ch]
> Sent: Tuesday, November 27, 2001 2:10 PM
> To: lucene-user@jakarta.apache.org
> Subject: synchronization problem / bug?
>
>
> hi
>
> while testing the SqlDirectory, I found some really strange
> thing: scenario
> is concurrent writer and searcher:
> 1. a IndexWriter is started and creates a write.lock until
> the close method
> is called. this cleanly prevents other writers to access the
> index at the
> same time and is ok.
> 2. go on indexing ...
>
> but now, concurrently, the following process goes on:
> 1. a Searcher is created with searcher = new IndexSearcher().
> 2. this process creates a commit.lock as expected and reads
> some files.
> 3. the commit.lock is released. (immediately)
> 4. now, the querying is done and the hits.doc(i) is read.
> during this, no
> commit.lock is set, but again, some files are accessed (the
> InputStream.readInternal method is called).
> 5. the searcher.close() method is called which closes all
> open InputStreams.
> (no commit.lock released or created)
>
> like that, from time to time, a exception occures because the
> file has been
> changed by the IndexWriter process running the same time.
>
> Any ideas about this? this should also occur with FSDirectory or
> RAMDirectory, but more rarely, because these are faster in reading
> results...
>
> cheers
> marc
>
>
>
> --
> To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>

--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>