Mailing List Archive

I've seen this failure before here:
https://github.com/apache/lucene/issues/11754

From what I remember, seems something blows up with the multiplier
that causes the usage.

On Fri, Sep 30, 2022 at 3:17 AM Uwe Schindler <uwe@thetaphi.de> wrote:
>
> Hi,
>
> I have never seen this before. It looks like something in this test ran
> out of memory (too many clauses, to large top-n?).
>
> Uwe
>
> Am 30.09.2022 um 07:48 schrieb Policeman Jenkins Server:
> > Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/5866/
> > Java: 64bit/jdk-18 -XX:-UseCompressedOops -XX:+UseParallelGC
> >
> > 1 tests failed.
> > FAILED: org.apache.lucene.search.TestBoolean2.testRandomQueries
> >
> > Error Message:
> > java.lang.OutOfMemoryError: Java heap space
> >
> > Stack Trace:
> > java.lang.OutOfMemoryError: Java heap space
> > at __randomizedtesting.SeedInfo.seed([1ACF9F67D361176D:44E42F8BEBBB3AF3]:0)
> > at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:96)
> > at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:43)
> > at org.apache.lucene.search.FieldValueHitQueue.<init>(FieldValueHitQueue.java:123)
> > at org.apache.lucene.search.FieldValueHitQueue$OneComparatorFieldValueHitQueue.<init>(FieldValueHitQueue.java:59)
> > at org.apache.lucene.search.FieldValueHitQueue.create(FieldValueHitQueue.java:159)
> > at org.apache.lucene.search.TopFieldCollector.create(TopFieldCollector.java:454)
> > at org.apache.lucene.search.TopFieldCollector$1.newCollector(TopFieldCollector.java:501)
> > at org.apache.lucene.search.TopFieldCollector$1.newCollector(TopFieldCollector.java:493)
> > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:669)
> > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:656)
> > at org.apache.lucene.search.TestBoolean2.testRandomQueries(TestBoolean2.java:406)
> > at java.base/java.lang.invoke.LambdaForm$DMH/0x0000000800d60000.invokeVirtual(LambdaForm$DMH)
> > at java.base/java.lang.invoke.LambdaForm$MH/0x0000000800ca4800.invoke(LambdaForm$MH)
> > at java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
> > at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(DirectMethodHandleAccessor.java:154)
> > at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
> > at java.base/java.lang.reflect.Method.invoke(Method.java:577)
> > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
> > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
> > at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
> > at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
> > at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
> > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
> > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
> > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: builds-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: builds-help@lucene.apache.org
>
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Thanks Robert for the link to issue. I have to get used to Github issues
:-). I was looking in JIRA :-)

We should have a big warning on JIRA that it is not only read-only but
also not uptodate. Spring framework redirects all JIRA issues and the
JIRA homepage to the corresponding Github issues. This can be done by a
RewriteMap in the Proxy Server before JIRA. The map could easily
generated from our collected metadata (JIRA issue number -> Github number).

Uwe

Am 30.09.2022 um 09:51 schrieb Robert Muir:
> I've seen this failure before here:
> https://github.com/apache/lucene/issues/11754
>
> From what I remember, seems something blows up with the multiplier
> that causes the usage.
>
> On Fri, Sep 30, 2022 at 3:17 AM Uwe Schindler <uwe@thetaphi.de> wrote:
>> Hi,
>>
>> I have never seen this before. It looks like something in this test ran
>> out of memory (too many clauses, to large top-n?).
>>
>> Uwe
>>
>> Am 30.09.2022 um 07:48 schrieb Policeman Jenkins Server:
>>> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/5866/
>>> Java: 64bit/jdk-18 -XX:-UseCompressedOops -XX:+UseParallelGC
>>>
>>> 1 tests failed.
>>> FAILED: org.apache.lucene.search.TestBoolean2.testRandomQueries
>>>
>>> Error Message:
>>> java.lang.OutOfMemoryError: Java heap space
>>>
>>> Stack Trace:
>>> java.lang.OutOfMemoryError: Java heap space
>>> at __randomizedtesting.SeedInfo.seed([1ACF9F67D361176D:44E42F8BEBBB3AF3]:0)
>>> at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:96)
>>> at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:43)
>>> at org.apache.lucene.search.FieldValueHitQueue.<init>(FieldValueHitQueue.java:123)
>>> at org.apache.lucene.search.FieldValueHitQueue$OneComparatorFieldValueHitQueue.<init>(FieldValueHitQueue.java:59)
>>> at org.apache.lucene.search.FieldValueHitQueue.create(FieldValueHitQueue.java:159)
>>> at org.apache.lucene.search.TopFieldCollector.create(TopFieldCollector.java:454)
>>> at org.apache.lucene.search.TopFieldCollector$1.newCollector(TopFieldCollector.java:501)
>>> at org.apache.lucene.search.TopFieldCollector$1.newCollector(TopFieldCollector.java:493)
>>> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:669)
>>> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:656)
>>> at org.apache.lucene.search.TestBoolean2.testRandomQueries(TestBoolean2.java:406)
>>> at java.base/java.lang.invoke.LambdaForm$DMH/0x0000000800d60000.invokeVirtual(LambdaForm$DMH)
>>> at java.base/java.lang.invoke.LambdaForm$MH/0x0000000800ca4800.invoke(LambdaForm$MH)
>>> at java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
>>> at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(DirectMethodHandleAccessor.java:154)
>>> at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
>>> at java.base/java.lang.reflect.Method.invoke(Method.java:577)
>>> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
>>> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
>>> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
>>> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
>>> at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
>>> at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>> at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
>>> at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>> at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
>>> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
>>> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: builds-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: builds-help@lucene.apache.org
>> --
>> Uwe Schindler
>> Achterdiek 19, D-28357 Bremen
>> https://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
--
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

As far as the issue, I've been meaning to look into it... just have
not found the time yet. I think it will work to run the reproducing
seed with -Ptests.profile=true -Ptests.profile.mode=heap, to debug the
heap allocations with JFR and see where its blowing up wrt to the
multiplier setting. I don't have the energy in me to inspect a heap
dump :)

On Fri, Sep 30, 2022 at 4:01 AM Uwe Schindler <uwe@thetaphi.de> wrote:
>
> Thanks Robert for the link to issue. I have to get used to Github issues
> :-). I was looking in JIRA :-)
>
> We should have a big warning on JIRA that it is not only read-only but
> also not uptodate. Spring framework redirects all JIRA issues and the
> JIRA homepage to the corresponding Github issues. This can be done by a
> RewriteMap in the Proxy Server before JIRA. The map could easily
> generated from our collected metadata (JIRA issue number -> Github number).
>
> Uwe
>
> Am 30.09.2022 um 09:51 schrieb Robert Muir:
> > I've seen this failure before here:
> > https://github.com/apache/lucene/issues/11754
> >
> > From what I remember, seems something blows up with the multiplier
> > that causes the usage.
> >
> > On Fri, Sep 30, 2022 at 3:17 AM Uwe Schindler <uwe@thetaphi.de> wrote:
> >> Hi,
> >>
> >> I have never seen this before. It looks like something in this test ran
> >> out of memory (too many clauses, to large top-n?).
> >>
> >> Uwe
> >>
> >> Am 30.09.2022 um 07:48 schrieb Policeman Jenkins Server:
> >>> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/5866/
> >>> Java: 64bit/jdk-18 -XX:-UseCompressedOops -XX:+UseParallelGC
> >>>
> >>> 1 tests failed.
> >>> FAILED: org.apache.lucene.search.TestBoolean2.testRandomQueries
> >>>
> >>> Error Message:
> >>> java.lang.OutOfMemoryError: Java heap space
> >>>
> >>> Stack Trace:
> >>> java.lang.OutOfMemoryError: Java heap space
> >>> at __randomizedtesting.SeedInfo.seed([1ACF9F67D361176D:44E42F8BEBBB3AF3]:0)
> >>> at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:96)
> >>> at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:43)
> >>> at org.apache.lucene.search.FieldValueHitQueue.<init>(FieldValueHitQueue.java:123)
> >>> at org.apache.lucene.search.FieldValueHitQueue$OneComparatorFieldValueHitQueue.<init>(FieldValueHitQueue.java:59)
> >>> at org.apache.lucene.search.FieldValueHitQueue.create(FieldValueHitQueue.java:159)
> >>> at org.apache.lucene.search.TopFieldCollector.create(TopFieldCollector.java:454)
> >>> at org.apache.lucene.search.TopFieldCollector$1.newCollector(TopFieldCollector.java:501)
> >>> at org.apache.lucene.search.TopFieldCollector$1.newCollector(TopFieldCollector.java:493)
> >>> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:669)
> >>> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:656)
> >>> at org.apache.lucene.search.TestBoolean2.testRandomQueries(TestBoolean2.java:406)
> >>> at java.base/java.lang.invoke.LambdaForm$DMH/0x0000000800d60000.invokeVirtual(LambdaForm$DMH)
> >>> at java.base/java.lang.invoke.LambdaForm$MH/0x0000000800ca4800.invoke(LambdaForm$MH)
> >>> at java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)
> >>> at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(DirectMethodHandleAccessor.java:154)
> >>> at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
> >>> at java.base/java.lang.reflect.Method.invoke(Method.java:577)
> >>> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
> >>> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
> >>> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
> >>> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
> >>> at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
> >>> at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> >>> at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> >>> at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> >>> at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> >>> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> >>> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> >>> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
> >>> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
> >>> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: builds-unsubscribe@lucene.apache.org
> >>> For additional commands, e-mail: builds-help@lucene.apache.org
> >> --
> >> Uwe Schindler
> >> Achterdiek 19, D-28357 Bremen
> >> https://www.thetaphi.de
> >> eMail: uwe@thetaphi.de
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: dev-help@lucene.apache.org
> >>
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org