I've been able to reproduce a crash we are seeing in our product with
newer Lucene versions.
I'm attaching a small Java code that reproduces this. It might look
weird, it's the result of removing every custom thing we are applying to
the query while still seeing the bug.
This is the crash I see with this code (with assertions disabled it
crashes in a different place):
Exception in thread "main" java.lang.AssertionError
??? at
org.apache.lucene.codecs.lucene84.Lucene84PostingsReader$EverythingEnum.nextPosition(Lucene84PostingsReader.java:940)
??? at
org.apache.lucene.search.PhrasePositions.nextPosition(PhrasePositions.java:57)
??? at
org.apache.lucene.search.PhrasePositions.firstPosition(PhrasePositions.java:46)
??? at
org.apache.lucene.search.SloppyPhraseMatcher.initSimple(SloppyPhraseMatcher.java:368)
??? at
org.apache.lucene.search.SloppyPhraseMatcher.initPhrasePositions(SloppyPhraseMatcher.java:356)
??? at
org.apache.lucene.search.SloppyPhraseMatcher.reset(SloppyPhraseMatcher.java:153)
??? at
org.apache.lucene.search.PhraseScorer$1.matches(PhraseScorer.java:49)
??? at
org.apache.lucene.search.DoubleValuesSource$WeightDoubleValuesSource$1.advanceExact(DoubleValuesSource.java:631)
??? at
org.apache.lucene.queries.function.FunctionScoreQuery$QueryBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:343)
??? at
org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
??? at
org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
??? at
org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:270)
??? at
org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:228)
??? at
org.apache.lucene.search.DoubleValuesSource$2.doubleValue(DoubleValuesSource.java:344)
??? at
org.apache.lucene.search.DoubleValues$1.doubleValue(DoubleValues.java:48)
??? at
org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.doubleValue(FunctionScoreQuery.java:265)
??? at
org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:229)
??? at
org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:76)
??? at
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:276)
??? at
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:232)
??? at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
??? at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:661)
??? at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:445)
??? at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:574)
??? at
org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:421)
??? at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:432)
??? at com.wolfram.textsearch.LuceneCrash.main(LuceneCrash.java:48)
Interestingly, the bug does not happen if the index is created on a
ByteBuffersDirectory.
I hope this is useful!
Thanks!
newer Lucene versions.
I'm attaching a small Java code that reproduces this. It might look
weird, it's the result of removing every custom thing we are applying to
the query while still seeing the bug.
This is the crash I see with this code (with assertions disabled it
crashes in a different place):
Exception in thread "main" java.lang.AssertionError
??? at
org.apache.lucene.codecs.lucene84.Lucene84PostingsReader$EverythingEnum.nextPosition(Lucene84PostingsReader.java:940)
??? at
org.apache.lucene.search.PhrasePositions.nextPosition(PhrasePositions.java:57)
??? at
org.apache.lucene.search.PhrasePositions.firstPosition(PhrasePositions.java:46)
??? at
org.apache.lucene.search.SloppyPhraseMatcher.initSimple(SloppyPhraseMatcher.java:368)
??? at
org.apache.lucene.search.SloppyPhraseMatcher.initPhrasePositions(SloppyPhraseMatcher.java:356)
??? at
org.apache.lucene.search.SloppyPhraseMatcher.reset(SloppyPhraseMatcher.java:153)
??? at
org.apache.lucene.search.PhraseScorer$1.matches(PhraseScorer.java:49)
??? at
org.apache.lucene.search.DoubleValuesSource$WeightDoubleValuesSource$1.advanceExact(DoubleValuesSource.java:631)
??? at
org.apache.lucene.queries.function.FunctionScoreQuery$QueryBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:343)
??? at
org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
??? at
org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
??? at
org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:270)
??? at
org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:228)
??? at
org.apache.lucene.search.DoubleValuesSource$2.doubleValue(DoubleValuesSource.java:344)
??? at
org.apache.lucene.search.DoubleValues$1.doubleValue(DoubleValues.java:48)
??? at
org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.doubleValue(FunctionScoreQuery.java:265)
??? at
org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:229)
??? at
org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:76)
??? at
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:276)
??? at
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:232)
??? at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
??? at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:661)
??? at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:445)
??? at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:574)
??? at
org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:421)
??? at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:432)
??? at com.wolfram.textsearch.LuceneCrash.main(LuceneCrash.java:48)
Interestingly, the bug does not happen if the index is created on a
ByteBuffersDirectory.
I hope this is useful!
Thanks!