Hi all,
After upgrading projects from Lucene 3.0.0 to 8.5.2, I'm getting an error appearing sometimes during a load test.
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 128
at org.apache.lucene.util.BytesRefHash$1.get(BytesRefHash.java:181) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.StringMSBRadixSorter.byteAt(StringMSBRadixSorter.java:30) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.MSBRadixSorter.getBucket(MSBRadixSorter.java:199) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.MSBRadixSorter.buildHistogram(MSBRadixSorter.java:249) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.MSBRadixSorter.computeCommonPrefixLengthAndBuildHistogram(MSBRadixSorter.java:236) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.MSBRadixSorter.radixSort(MSBRadixSorter.java:148) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.MSBRadixSorter.sort(MSBRadixSorter.java:128) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.MSBRadixSorter.sort(MSBRadixSorter.java:121) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.BytesRefHash.sort(BytesRefHash.java:185) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.index.memory.MemoryIndex$Info.sortTerms(MemoryIndex.java:858) ~[lucene-memory-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:43]
at org.apache.lucene.index.memory.MemoryIndex$MemoryIndexReader$MemoryTermsEnum.<init>(MemoryIndex.java:1342) ~[lucene-memory-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:43]
at org.apache.lucene.index.memory.MemoryIndex$MemoryIndexReader$MemoryFields$1.iterator(MemoryIndex.java:1279) ~[lucene-memory-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:43]
at org.apache.lucene.index.TermStates.loadTermsEnum(TermStates.java:123) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.index.TermStates.build(TermStates.java:109) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.search.TermQuery.createWeight(TermQuery.java:199) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:726) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.search.BooleanWeight.<init>(BooleanWeight.java:63) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:231) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:726) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:445) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.index.memory.MemoryIndex.search(MemoryIndex.java:690) ~[lucene-memory-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:43]
It's not easily reproducible as it doesn't happen for the same queries or documents each time the load test is repeated.
The queries are generally quite simple field:value ones e.g. "doc_lead:mayenne"
The queries are evaluated in concurrent threads and I noticed the freeze() method had been added since 3.0.0 but even calling this between populating the index and evaluating the queries did not help.
Thanks in advance for any help,
Albert
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
After upgrading projects from Lucene 3.0.0 to 8.5.2, I'm getting an error appearing sometimes during a load test.
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 128
at org.apache.lucene.util.BytesRefHash$1.get(BytesRefHash.java:181) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.StringMSBRadixSorter.byteAt(StringMSBRadixSorter.java:30) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.MSBRadixSorter.getBucket(MSBRadixSorter.java:199) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.MSBRadixSorter.buildHistogram(MSBRadixSorter.java:249) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.MSBRadixSorter.computeCommonPrefixLengthAndBuildHistogram(MSBRadixSorter.java:236) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.MSBRadixSorter.radixSort(MSBRadixSorter.java:148) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.MSBRadixSorter.sort(MSBRadixSorter.java:128) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.MSBRadixSorter.sort(MSBRadixSorter.java:121) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.util.BytesRefHash.sort(BytesRefHash.java:185) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.index.memory.MemoryIndex$Info.sortTerms(MemoryIndex.java:858) ~[lucene-memory-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:43]
at org.apache.lucene.index.memory.MemoryIndex$MemoryIndexReader$MemoryTermsEnum.<init>(MemoryIndex.java:1342) ~[lucene-memory-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:43]
at org.apache.lucene.index.memory.MemoryIndex$MemoryIndexReader$MemoryFields$1.iterator(MemoryIndex.java:1279) ~[lucene-memory-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:43]
at org.apache.lucene.index.TermStates.loadTermsEnum(TermStates.java:123) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.index.TermStates.build(TermStates.java:109) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.search.TermQuery.createWeight(TermQuery.java:199) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:726) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.search.BooleanWeight.<init>(BooleanWeight.java:63) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:231) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:726) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:445) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
at org.apache.lucene.index.memory.MemoryIndex.search(MemoryIndex.java:690) ~[lucene-memory-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:43]
It's not easily reproducible as it doesn't happen for the same queries or documents each time the load test is repeated.
The queries are generally quite simple field:value ones e.g. "doc_lead:mayenne"
The queries are evaluated in concurrent threads and I noticed the freeze() method had been added since 3.0.0 but even calling this between populating the index and evaluating the queries did not help.
Thanks in advance for any help,
Albert
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org