Aug 7, 2002, 10:47 AM
Post #6 of 22
(813 views)
Permalink
Ok. Here's a new, simplified version that doesn't lock the index:
/** Returns the time the index in this directory was last modified. */
public static long lastModified(final Directory directory) throws
IOException {
long last = 0;
SegmentInfos infos = new SegmentInfos();
infos.read(directory);
for (int i = 0; i < infos.size(); i++) {
String delFile = infos.info(i).name + ".del";
if (directory.fileExists(delFile)) {
long lastDel = directory.fileModified(delFile); // last delete
time (per segment)
if (lastDel > last) last = lastDel;
}
}
long lastAdd = directory.fileModified("segments"); // last add time
return (lastAdd > last) ? lastAdd : last;
}
Comments?
Scott
> -----Original Message-----
> From: Scott Ganyo [mailto:scott.ganyo@eTapestry.com]
> Sent: Wednesday, August 07, 2002 12:41 PM
> To: 'Lucene Developers List'
> Subject: RE: IndexReader.lastModified() not correct
>
>
> I lock the index because I copied the code from the open()
> method... and
> because it seemed to be the right thing to do. I was
> concerned that the
> index files not change during my iterating over them. For
> example, if the
> index segments were merged into a new one between when I read
> the "segments"
> file and when I iterated over the segment's deleted files, I wouldn't
> necessarily get the right answer. On the other hand, I could
> check the last
> modified date on the "segments" file *after* checking the individual
> segments. Maybe I'll do that instead.
>
> At any rate, though, you're right. It isn't truly safe
> unless the lock is
> held during the entire process...
>
> Scott
>
> P.S. I don't know if this could have caused your Exception.
> I actually
> can't think of a scenario off-hand where you would get that
> Exception except
> in a corrupt index... is your index intact?
>
> > -----Original Message-----
> > From: Halácsy Péter [mailto:halacsy.peter@axelero.com]
> > Sent: Wednesday, August 07, 2002 12:23 PM
> > To: Lucene Developers List
> > Subject: RE: IndexReader.lastModified() not correct
> >
> >
> > Scott,
> > why do you lock the index while iterating del files?
> >
> > if you don't lock it, you can calculate wrong time but has it
> > effect on caching? one can modify the index between checking
> > the lastModified value and creating new searcher.
> >
> > peter
> >
> > > -----Original Message-----
> > > From: Scott Ganyo [mailto:scott.ganyo@eTapestry.com]
> > > Sent: Wednesday, August 07, 2002 6:32 PM
> > > To: 'Lucene Developers List'
> > > Subject: RE: IndexReader.lastModified() not correct
> > >
> > >
> > > Ok. I've dug through all the code and although this isn't
> > > too pretty, it
> > > looks to my untrained eye that this is probably the best way
> > > to implement a
> > > corrected version that takes into consideration the deleted list.
> > > Basically, it checks the "segments" file as before, then
> checks the
> > > "segment.del" file in each segment, and takes the last time
> > > any of those
> > > files were modified:
> > >
> > > public static long lastModified(final Directory
> directory) throws
> > > IOException {
> > > synchronized (directory) { //
> > > in- & inter-process
> > > sync
> > > Long lastMod = (Long)new
> > > Lock.With(directory.makeLock("commit.lock"))
> > > {
> > > public Object doBody() throws IOException {
> > > long last = directory.fileModified("segments"); //
> > > last add time
> > > SegmentInfos infos = new SegmentInfos();
> > > infos.read(directory);
> > > for (int i = 0; i < infos.size(); i++) {
> > > String delFile = infos.info(i).name + ".del";
> > > if (directory.fileExists(delFile)) {
> > > long lastDel =
> > > directory.fileModified(delFile); // last
> > > delete time (per segment)
> > > if (lastDel > last) last = lastDel;
> > > }
> > > }
> > > return new Long(last);
> > > }
> > > }.run();
> > > return lastMod.longValue();
> > > }
> > > }
> > >
> > > Please let me know if I've missed any shortcuts or if you
> > > know of a better
> > > way...
> > >
> > > Scott
> > >
> > > > -----Original Message-----
> > > > From: Scott Ganyo [mailto:scott.ganyo@eTapestry.com]
> > > > Sent: Wednesday, August 07, 2002 9:27 AM
> > > > To: Lucene-Dev (E-mail)
> > > > Subject: IndexReader.lastModified() not correct
> > > >
> > > >
> > > > The current implementation of IndexReader.lastModified() does
> > > > not return the
> > > > results I am expecting. Here is the implementation:
> > > >
> > > > /** Returns the time the index in the named directory was
> > > > last modified.
> > > > */
> > > > public static long lastModified(File directory) throws
> > > IOException {
> > > > return FSDirectory.fileModified(directory, "segments");
> > > > }
> > > >
> > > > The problem is that the "segments" file is apparently not
> > > > updated when just
> > > > doing a delete from an index. This is causing me problems
> > > > because I am
> > > > attempting to rely on on the lastModified() command for
> > > IndexSearcher
> > > > caching. The only solution that I have thought of so far
> > > > (without changing
> > > > other parts of Lucene) is to make the lastModified() command
> > > > look at all the
> > > > files in the last segment for the last modified date.
> > > >
> > > > Thoughts?
> > > >
> > > > Thanks,
> > > > Scott
> > > >
> > >
> >
> > --
> > To unsubscribe, e-mail:
> > <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
> > For additional commands, e-mail:
> > <mailto:lucene-dev-help@jakarta.apache.org>
> >
>