Mailing List Archive

Hi Uwe,

Will dig tomorrow. Maybe Dawid has an idea? It looks like the alternate
> runtime is correctly detected, but why is Gradle passing compiler runzine
> options without -J in just this case. In Main the same works where Gradle
> runs with Java17-Temurin and J9 is used as runtime.
>

I think I know what this is - please let me verify and provide a PR, if
it's indeed that.

Dawid

Thanks.

I have the feeling it is because of same major version. If Gradle JDK is Eclipse Temurin and runtime JDK is same version but OpenJ9 it fails.

Interestingly only in 11 (branch 9x). On main it worked for long time. Jdk17 as Gradle runtime by Temurin and openj9 as runtime.

If runtime and Gradle VM is identical home dir it does not fork. Here is a flight difference and that matters. The alternate VM info is printed, but it looks like it f*cks up when providing options and something thinks it does not fork, but it does.

> Task :altJvmWarning
NOTE: Alternative java toolchain will be used for compilation and tests:
Project will use 11 (IBM JDK 11.0.20.1+1, home at: /home/jenkins/tools/java/64bit/openj9/jdk-11.0.20)
Gradle runs with 11 (Eclipse Temurin JDK 11.0.21+9, home at: /home/jenkins/tools/java/64bit/hotspot/jdk-11.0.21)

On main branch it works, no idea why:

> Task :altJvmWarning
NOTE: Alternative java toolchain will be used for compilation and tests:
Project will use 17 (IBM JDK 17.0.8.1+1, home at: /home/jenkins/tools/java/64bit/openj9/jdk-17.0.8)
Gradle runs with 17 (Eclipse Temurin JDK 17.0.9+9, home at: /home/jenkins/tools/java/64bit/hotspot/jdk-17.0.9)

In case you ask, die to randomized jvms the Jenkins job Always runs with default temurin minimum jdk for Gradle and just exchanges runtime for testing (because Gradle itself can't run on all JDKs out there).

One other thing: if you set OpenJ9 as default jdk and run Gradle with it, it fails while building buildSrc: it can't compile the profiling stuff as flight recorder module does not ship with OpenJ9. We should possibly fix this, maybe by rewriting the code with pure Groovy class and not compile it at all.

Uwe

Am 12. November 2023 21:08:41 MEZ schrieb Dawid Weiss <dawid.weiss@gmail.com>:
>Hi Uwe,
>
>Will dig tomorrow. Maybe Dawid has an idea? It looks like the alternate
>> runtime is correctly detected, but why is Gradle passing compiler runzine
>> options without -J in just this case. In Main the same works where Gradle
>> runs with Java17-Temurin and J9 is used as runtime.
>>
>
>I think I know what this is - please let me verify and provide a PR, if
>it's indeed that.
>
>Dawid

--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de

One addition: Solr is affected on same way.

Am 12. November 2023 21:39:07 MEZ schrieb Uwe Schindler <uwe@thetaphi.de>:
>Thanks.
>
>I have the feeling it is because of same major version. If Gradle JDK is Eclipse Temurin and runtime JDK is same version but OpenJ9 it fails.
>
>Interestingly only in 11 (branch 9x). On main it worked for long time. Jdk17 as Gradle runtime by Temurin and openj9 as runtime.
>
>If runtime and Gradle VM is identical home dir it does not fork. Here is a flight difference and that matters. The alternate VM info is printed, but it looks like it f*cks up when providing options and something thinks it does not fork, but it does.
>
>> Task :altJvmWarning
>NOTE: Alternative java toolchain will be used for compilation and tests:
> Project will use 11 (IBM JDK 11.0.20.1+1, home at: /home/jenkins/tools/java/64bit/openj9/jdk-11.0.20)
> Gradle runs with 11 (Eclipse Temurin JDK 11.0.21+9, home at: /home/jenkins/tools/java/64bit/hotspot/jdk-11.0.21)
>
>On main branch it works, no idea why:
>
>> Task :altJvmWarning
>NOTE: Alternative java toolchain will be used for compilation and tests:
> Project will use 17 (IBM JDK 17.0.8.1+1, home at: /home/jenkins/tools/java/64bit/openj9/jdk-17.0.8)
> Gradle runs with 17 (Eclipse Temurin JDK 17.0.9+9, home at: /home/jenkins/tools/java/64bit/hotspot/jdk-17.0.9)
>
>In case you ask, die to randomized jvms the Jenkins job Always runs with default temurin minimum jdk for Gradle and just exchanges runtime for testing (because Gradle itself can't run on all JDKs out there).
>
>One other thing: if you set OpenJ9 as default jdk and run Gradle with it, it fails while building buildSrc: it can't compile the profiling stuff as flight recorder module does not ship with OpenJ9. We should possibly fix this, maybe by rewriting the code with pure Groovy class and not compile it at all.
>
>Uwe
>
>Am 12. November 2023 21:08:41 MEZ schrieb Dawid Weiss <dawid.weiss@gmail.com>:
>>Hi Uwe,
>>
>>Will dig tomorrow. Maybe Dawid has an idea? It looks like the alternate
>>> runtime is correctly detected, but why is Gradle passing compiler runzine
>>> options without -J in just this case. In Main the same works where Gradle
>>> runs with Java17-Temurin and J9 is used as runtime.
>>>
>>
>>I think I know what this is - please let me verify and provide a PR, if
>>it's indeed that.
>>
>>Dawid
>
>--
>Uwe Schindler
>Achterdiek 19, 28357 Bremen
>https://www.thetaphi.de
--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de

I can't reproduce this though - used exactly the same JVMs (on Windows):

> gradlew :lucene:distribution.tests:compileTestJava --rerun-tasks
--console=plain
Generating gradle.properties
...
> Task :altJvmWarning
NOTE: Alternative java toolchain will be used for compilation and tests:
Project will use 11 (IBM JDK 11.0.20.1+1, home at:
c:\_tmp\jdk-11.0.20.1+1)
Gradle runs with 11 (Eclipse Temurin JDK 11.0.21+9, home at:
C:\_tmp\jdk-11.0.21+9)
...
> Task :lucene:distribution.tests:compileJava NO-SOURCE
> Task :lucene:distribution.tests:classes UP-TO-DATE
> Task :lucene:distribution.tests:compileTestJava

BUILD SUCCESSFUL in 23s
5 actionable tasks: 5 executed

On main branch it works, no idea why:
>

O thought it's because of this:

https://github.com/apache/lucene/commit/2e12a35c876a

but I don't think so... seems to work for me on Windows on branch_9x just
fine?

D.

>

This one was my first idea, too.

It fails only with IBM Semeru in combination with Gradle using Temurin.

I will dig tomorrow on Jenkins server and print all debug info.

Uwe

Am 12. November 2023 21:48:54 MEZ schrieb Dawid Weiss <dawid.weiss@gmail.com>:
>I can't reproduce this though - used exactly the same JVMs (on Windows):
>
>> gradlew :lucene:distribution.tests:compileTestJava --rerun-tasks
>--console=plain
>Generating gradle.properties
>...
>> Task :altJvmWarning
>NOTE: Alternative java toolchain will be used for compilation and tests:
> Project will use 11 (IBM JDK 11.0.20.1+1, home at:
>c:\_tmp\jdk-11.0.20.1+1)
> Gradle runs with 11 (Eclipse Temurin JDK 11.0.21+9, home at:
>C:\_tmp\jdk-11.0.21+9)
>...
>> Task :lucene:distribution.tests:compileJava NO-SOURCE
>> Task :lucene:distribution.tests:classes UP-TO-DATE
>> Task :lucene:distribution.tests:compileTestJava
>
>BUILD SUCCESSFUL in 23s
>5 actionable tasks: 5 executed
>
>On main branch it works, no idea why:
>>
>
>O thought it's because of this:
>
>https://github.com/apache/lucene/commit/2e12a35c876a
>
>but I don't think so... seems to work for me on Windows on branch_9x just
>fine?
>
>D.
>
>>

--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de

Hi Uwe,

Can you reproduce this on Windows with the same JVM versions though? Seems
like I have exactly the same setup and yet this works for me just fine.
Strange.

Dawid

On Sun, Nov 12, 2023 at 9:52?PM Uwe Schindler <uwe@thetaphi.de> wrote:

> This one was my first idea, too.
>
> It fails only with IBM Semeru in combination with Gradle using Temurin.
>
> I will dig tomorrow on Jenkins server and print all debug info.
>
> Uwe
>
>
> Am 12. November 2023 21:48:54 MEZ schrieb Dawid Weiss <
> dawid.weiss@gmail.com>:
>
>>
>> I can't reproduce this though - used exactly the same JVMs (on Windows):
>>
>> > gradlew :lucene:distribution.tests:compileTestJava --rerun-tasks
>> --console=plain
>> Generating gradle.properties
>> ...
>> > Task :altJvmWarning
>> NOTE: Alternative java toolchain will be used for compilation and tests:
>> Project will use 11 (IBM JDK 11.0.20.1+1, home at:
>> c:\_tmp\jdk-11.0.20.1+1)
>> Gradle runs with 11 (Eclipse Temurin JDK 11.0.21+9, home at:
>> C:\_tmp\jdk-11.0.21+9)
>> ...
>> > Task :lucene:distribution.tests:compileJava NO-SOURCE
>> > Task :lucene:distribution.tests:classes UP-TO-DATE
>> > Task :lucene:distribution.tests:compileTestJava
>>
>> BUILD SUCCESSFUL in 23s
>> 5 actionable tasks: 5 executed
>>
>> On main branch it works, no idea why:
>>>
>>
>> O thought it's because of this:
>>
>> https://github.com/apache/lucene/commit/2e12a35c876a
>>
>> but I don't think so... seems to work for me on Windows on branch_9x just
>> fine?
>>
>> D.
>>
>>> --
> Uwe Schindler
> Achterdiek 19, 28357 Bremen
> https://www.thetaphi.de
>

Will check tomorrow, it's too late now.

On Jenkins there were no windows builds with IBM and Java 11 yet:
https://jenkins.thetaphi.de/job/Lucene-9.x-Windows/

Am 12.11.2023 um 22:00 schrieb Dawid Weiss:
>
> Hi Uwe,
>
> Can you reproduce this on Windows with the same JVM versions though?
> Seems like I have exactly the same setup and yet this works for me
> just fine. Strange.
>
> Dawid
>
> On Sun, Nov 12, 2023 at 9:52?PM Uwe Schindler <uwe@thetaphi.de> wrote:
>
> This one was my first idea, too.
>
> It fails only with IBM Semeru in combination with Gradle using
> Temurin.
>
> I will dig tomorrow on Jenkins server and print all debug info.
>
> Uwe
>
>
> Am 12. November 2023 21:48:54 MEZ schrieb Dawid Weiss
> <dawid.weiss@gmail.com>:
>
>
> I can't reproduce this though - used exactly the same JVMs (on
> Windows):
>
> > gradlew :lucene:distribution.tests:compileTestJava
> --rerun-tasks --console=plain
> Generating gradle.properties
> ...
> > Task :altJvmWarning
> NOTE: Alternative java toolchain will be used for compilation
> and tests:
>   Project will use 11 (IBM JDK 11.0.20.1+1, home at:
> c:\_tmp\jdk-11.0.20.1+1)
>   Gradle runs with 11 (Eclipse Temurin JDK 11.0.21+9, home at:
> C:\_tmp\jdk-11.0.21+9)
> ...
> > Task :lucene:distribution.tests:compileJava NO-SOURCE
> > Task :lucene:distribution.tests:classes UP-TO-DATE
> > Task :lucene:distribution.tests:compileTestJava
>
> BUILD SUCCESSFUL in 23s
> 5 actionable tasks: 5 executed
>
> On main branch it works, no idea why:
>
>
> O thought it's because of this:
>
> https://github.com/apache/lucene/commit/2e12a35c876a
>
> but I don't think so... seems to work for me on Windows on
> branch_9x just fine?
>
> D.
>
> --
> Uwe Schindler
> Achterdiek 19, 28357 Bremen
> https://www.thetaphi.de
>
--
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail:uwe@thetaphi.de

Sure, thanks. What's strange is that we don't use add-opens anywhere, I
think (there is a mention of it I left in one of the
comments, but nothing else across the codebase uses this directive).

> Task :lucene:distribution.tests:compileTestJava
warning: [options] --add-opens has no effect at compile time



On Sun, Nov 12, 2023 at 10:56?PM Uwe Schindler <uwe@thetaphi.de> wrote:

> Will check tomorrow, it's too late now.
>
> On Jenkins there were no windows builds with IBM and Java 11 yet:
> https://jenkins.thetaphi.de/job/Lucene-9.x-Windows/
> Am 12.11.2023 um 22:00 schrieb Dawid Weiss:
>
>
> Hi Uwe,
>
> Can you reproduce this on Windows with the same JVM versions though? Seems
> like I have exactly the same setup and yet this works for me just fine.
> Strange.
>
> Dawid
>
> On Sun, Nov 12, 2023 at 9:52?PM Uwe Schindler <uwe@thetaphi.de> wrote:
>
>> This one was my first idea, too.
>>
>> It fails only with IBM Semeru in combination with Gradle using Temurin.
>>
>> I will dig tomorrow on Jenkins server and print all debug info.
>>
>> Uwe
>>
>>
>> Am 12. November 2023 21:48:54 MEZ schrieb Dawid Weiss <
>> dawid.weiss@gmail.com>:
>>
>>>
>>> I can't reproduce this though - used exactly the same JVMs (on Windows):
>>>
>>> > gradlew :lucene:distribution.tests:compileTestJava --rerun-tasks
>>> --console=plain
>>> Generating gradle.properties
>>> ...
>>> > Task :altJvmWarning
>>> NOTE: Alternative java toolchain will be used for compilation and tests:
>>> Project will use 11 (IBM JDK 11.0.20.1+1, home at:
>>> c:\_tmp\jdk-11.0.20.1+1)
>>> Gradle runs with 11 (Eclipse Temurin JDK 11.0.21+9, home at:
>>> C:\_tmp\jdk-11.0.21+9)
>>> ...
>>> > Task :lucene:distribution.tests:compileJava NO-SOURCE
>>> > Task :lucene:distribution.tests:classes UP-TO-DATE
>>> > Task :lucene:distribution.tests:compileTestJava
>>>
>>> BUILD SUCCESSFUL in 23s
>>> 5 actionable tasks: 5 executed
>>>
>>> On main branch it works, no idea why:
>>>>
>>>
>>> O thought it's because of this:
>>>
>>> https://github.com/apache/lucene/commit/2e12a35c876a
>>>
>>> but I don't think so... seems to work for me on Windows on branch_9x
>>> just fine?
>>>
>>> D.
>>>
>> --
>> Uwe Schindler
>> Achterdiek 19, 28357 Bremen
>> https://www.thetaphi.de
>>
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremenhttps://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>

I linked to this thread on the upstream (OpenJ9) issue about recent Lucene
CI build failures: https://github.com/eclipse-openj9/openj9/issues/18400

Mike McCandless

http://blog.mikemccandless.com


On Mon, Nov 13, 2023 at 3:09?AM Dawid Weiss <dawid.weiss@gmail.com> wrote:

>
> Sure, thanks. What's strange is that we don't use add-opens anywhere, I
> think (there is a mention of it I left in one of the
> comments, but nothing else across the codebase uses this directive).
>
> > Task :lucene:distribution.tests:compileTestJava
> warning: [options] --add-opens has no effect at compile time
>
>
>
> On Sun, Nov 12, 2023 at 10:56?PM Uwe Schindler <uwe@thetaphi.de> wrote:
>
>> Will check tomorrow, it's too late now.
>>
>> On Jenkins there were no windows builds with IBM and Java 11 yet:
>> https://jenkins.thetaphi.de/job/Lucene-9.x-Windows/
>> Am 12.11.2023 um 22:00 schrieb Dawid Weiss:
>>
>>
>> Hi Uwe,
>>
>> Can you reproduce this on Windows with the same JVM versions though?
>> Seems like I have exactly the same setup and yet this works for me just
>> fine. Strange.
>>
>> Dawid
>>
>> On Sun, Nov 12, 2023 at 9:52?PM Uwe Schindler <uwe@thetaphi.de> wrote:
>>
>>> This one was my first idea, too.
>>>
>>> It fails only with IBM Semeru in combination with Gradle using Temurin.
>>>
>>> I will dig tomorrow on Jenkins server and print all debug info.
>>>
>>> Uwe
>>>
>>>
>>> Am 12. November 2023 21:48:54 MEZ schrieb Dawid Weiss <
>>> dawid.weiss@gmail.com>:
>>>
>>>>
>>>> I can't reproduce this though - used exactly the same JVMs (on Windows):
>>>>
>>>> > gradlew :lucene:distribution.tests:compileTestJava --rerun-tasks
>>>> --console=plain
>>>> Generating gradle.properties
>>>> ...
>>>> > Task :altJvmWarning
>>>> NOTE: Alternative java toolchain will be used for compilation and tests:
>>>> Project will use 11 (IBM JDK 11.0.20.1+1, home at:
>>>> c:\_tmp\jdk-11.0.20.1+1)
>>>> Gradle runs with 11 (Eclipse Temurin JDK 11.0.21+9, home at:
>>>> C:\_tmp\jdk-11.0.21+9)
>>>> ...
>>>> > Task :lucene:distribution.tests:compileJava NO-SOURCE
>>>> > Task :lucene:distribution.tests:classes UP-TO-DATE
>>>> > Task :lucene:distribution.tests:compileTestJava
>>>>
>>>> BUILD SUCCESSFUL in 23s
>>>> 5 actionable tasks: 5 executed
>>>>
>>>> On main branch it works, no idea why:
>>>>>
>>>>
>>>> O thought it's because of this:
>>>>
>>>> https://github.com/apache/lucene/commit/2e12a35c876a
>>>>
>>>> but I don't think so... seems to work for me on Windows on branch_9x
>>>> just fine?
>>>>
>>>> D.
>>>>
>>> --
>>> Uwe Schindler
>>> Achterdiek 19, 28357 Bremen
>>> https://www.thetaphi.de
>>>
>> --
>> Uwe Schindler
>> Achterdiek 19, D-28357 Bremenhttps://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>

Hi Dawid,

Hah, the issue happens only if you pass CI=true (this is set by CI
systems), so errorprone is enabled. so do "export CI=true" and then
build with that config.

So it looks like a combination of errorprone enabled with Java 11 OpenJ9.

Uwe

Am 13.11.2023 um 09:09 schrieb Dawid Weiss:
>
> Sure, thanks. What's strange is that we don't use add-opens anywhere,
> I think (there is a mention of it I left in one of the
> comments, but nothing else across the codebase uses this directive).
>
> > Task :lucene:distribution.tests:compileTestJava
> warning: [options] --add-opens has no effect at compile time
>
>
> On Sun, Nov 12, 2023 at 10:56?PM Uwe Schindler <uwe@thetaphi.de> wrote:
>
> Will check tomorrow, it's too late now.
>
> On Jenkins there were no windows builds with IBM and Java 11 yet:
> https://jenkins.thetaphi.de/job/Lucene-9.x-Windows/
>
> Am 12.11.2023 um 22:00 schrieb Dawid Weiss:
>>
>> Hi Uwe,
>>
>> Can you reproduce this on Windows with the same JVM versions
>> though? Seems like I have exactly the same setup and yet this
>> works for me just fine. Strange.
>>
>> Dawid
>>
>> On Sun, Nov 12, 2023 at 9:52?PM Uwe Schindler <uwe@thetaphi.de>
>> wrote:
>>
>> This one was my first idea, too.
>>
>> It fails only with IBM Semeru in combination with Gradle
>> using Temurin.
>>
>> I will dig tomorrow on Jenkins server and print all debug info.
>>
>> Uwe
>>
>>
>> Am 12. November 2023 21:48:54 MEZ schrieb Dawid Weiss
>> <dawid.weiss@gmail.com>:
>>
>>
>> I can't reproduce this though - used exactly the same
>> JVMs (on Windows):
>>
>> > gradlew :lucene:distribution.tests:compileTestJava
>> --rerun-tasks --console=plain
>> Generating gradle.properties
>> ...
>> > Task :altJvmWarning
>> NOTE: Alternative java toolchain will be used for
>> compilation and tests:
>>   Project will use 11 (IBM JDK 11.0.20.1+1, home at:
>> c:\_tmp\jdk-11.0.20.1+1)
>>   Gradle runs with 11 (Eclipse Temurin JDK 11.0.21+9,
>> home at: C:\_tmp\jdk-11.0.21+9)
>> ...
>> > Task :lucene:distribution.tests:compileJava NO-SOURCE
>> > Task :lucene:distribution.tests:classes UP-TO-DATE
>> > Task :lucene:distribution.tests:compileTestJava
>>
>> BUILD SUCCESSFUL in 23s
>> 5 actionable tasks: 5 executed
>>
>> On main branch it works, no idea why:
>>
>>
>> O thought it's because of this:
>>
>> https://github.com/apache/lucene/commit/2e12a35c876a
>>
>> but I don't think so... seems to work for me on Windows
>> on branch_9x just fine?
>>
>> D.
>>
>> --
>> Uwe Schindler
>> Achterdiek 19, 28357 Bremen
>> https://www.thetaphi.de
>>
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail:uwe@thetaphi.de
>
--
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail:uwe@thetaphi.de

Hi Uwe,

Hah, the issue happens only if you pass CI=true (this is set by CI
> systems), so errorprone is enabled. so do "export CI=true" and then build
> with that config.
>

Darn... I hate that flag. I'll take a look unless you beat me to it.

D.

>

Hi Dawid,

The problem does not happen on Java 17, because errorprone is not
enabled when the forked JDK is > Java 15. We did this because earlier
versions worked correctly. But new versions of errorprone always fail
when the JDK is forked while compiling.

if (rootProject.usesAltJvm && rootProject.runtimeJavaVersion >
JavaVersion.VERSION_15) {
  skipReason = "won't work with JDK
${rootProject.runtimeJavaVersion} if used as alternative java toolchain"
}

if (!propertyOrDefault("validation.errorprone",
isCIBuild).asBoolean()) {
  skipReason = "skipped on builds not running inside CI
environments, pass -Pvalidation.errorprone=true to enable"
}

So it looks like the errorprone plugin got broken by a recent upgrade.
It now always fails when forked JDK is used. So we shold disable it in
this case. We just did not notice, as previously it was only disabled
when the runtime java version was > 17.

Nowadays we no longer run alternate JVMs with Java 12, 13, 14, 15. We
run with Java 11, 17, 19, 20, 21. So it is always disabled except for
Java 11. With RUNTIME_JAVA_HOME==JAVA_HOME we never fork, but as we use
OpenJ9, we fork an BOOOOOM.

I will post a PR soon.

Uwe

Am 14.11.2023 um 19:06 schrieb Uwe Schindler:
>
> Hi Dawid,
>
> Hah, the issue happens only if you pass CI=true (this is set by CI
> systems), so errorprone is enabled. so do "export CI=true" and then
> build with that config.
>
> So it looks like a combination of errorprone enabled with Java 11 OpenJ9.
>
> Uwe
>
> Am 13.11.2023 um 09:09 schrieb Dawid Weiss:
>>
>> Sure, thanks. What's strange is that we don't use add-opens anywhere,
>> I think (there is a mention of it I left in one of the
>> comments, but nothing else across the codebase uses this directive).
>>
>> > Task :lucene:distribution.tests:compileTestJava
>> warning: [options] --add-opens has no effect at compile time
>>
>>
>> On Sun, Nov 12, 2023 at 10:56?PM Uwe Schindler <uwe@thetaphi.de> wrote:
>>
>> Will check tomorrow, it's too late now.
>>
>> On Jenkins there were no windows builds with IBM and Java 11 yet:
>> https://jenkins.thetaphi.de/job/Lucene-9.x-Windows/
>>
>> Am 12.11.2023 um 22:00 schrieb Dawid Weiss:
>>>
>>> Hi Uwe,
>>>
>>> Can you reproduce this on Windows with the same JVM versions
>>> though? Seems like I have exactly the same setup and yet this
>>> works for me just fine. Strange.
>>>
>>> Dawid
>>>
>>> On Sun, Nov 12, 2023 at 9:52?PM Uwe Schindler <uwe@thetaphi.de>
>>> wrote:
>>>
>>> This one was my first idea, too.
>>>
>>> It fails only with IBM Semeru in combination with Gradle
>>> using Temurin.
>>>
>>> I will dig tomorrow on Jenkins server and print all debug info.
>>>
>>> Uwe
>>>
>>>
>>> Am 12. November 2023 21:48:54 MEZ schrieb Dawid Weiss
>>> <dawid.weiss@gmail.com>:
>>>
>>>
>>> I can't reproduce this though - used exactly the same
>>> JVMs (on Windows):
>>>
>>> > gradlew :lucene:distribution.tests:compileTestJava
>>> --rerun-tasks --console=plain
>>> Generating gradle.properties
>>> ...
>>> > Task :altJvmWarning
>>> NOTE: Alternative java toolchain will be used for
>>> compilation and tests:
>>>   Project will use 11 (IBM JDK 11.0.20.1+1, home at:
>>> c:\_tmp\jdk-11.0.20.1+1)
>>>   Gradle runs with 11 (Eclipse Temurin JDK 11.0.21+9,
>>> home at: C:\_tmp\jdk-11.0.21+9)
>>> ...
>>> > Task :lucene:distribution.tests:compileJava NO-SOURCE
>>> > Task :lucene:distribution.tests:classes UP-TO-DATE
>>> > Task :lucene:distribution.tests:compileTestJava
>>>
>>> BUILD SUCCESSFUL in 23s
>>> 5 actionable tasks: 5 executed
>>>
>>> On main branch it works, no idea why:
>>>
>>>
>>> O thought it's because of this:
>>>
>>> https://github.com/apache/lucene/commit/2e12a35c876a
>>>
>>> but I don't think so... seems to work for me on Windows
>>> on branch_9x just fine?
>>>
>>> D.
>>>
>>> --
>>> Uwe Schindler
>>> Achterdiek 19, 28357 Bremen
>>> https://www.thetaphi.de
>>>
>> --
>> Uwe Schindler
>> Achterdiek 19, D-28357 Bremen
>> https://www.thetaphi.de
>> eMail:uwe@thetaphi.de
>>
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail:uwe@thetaphi.de

--
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail:uwe@thetaphi.de

Hi,

For now the simplest is to disable always is an alternate JVM is used,
just remove the second part of the first IF statement. In Main it is no
longer relevant, as the runtime JDK is always >= 17, so it would always
trigger the first if.

I would not spend too much time, until errorprone gets its issues fixed.
Actually they have some hacks but they only work with toolkits. Looks
like the combination fork=true and not using toolkits breaks their
construction of parameters to pass.

Same for Solr.

Am 14.11.2023 um 19:17 schrieb Uwe Schindler:
>
> Hi Dawid,
>
> The problem does not happen on Java 17, because errorprone is not
> enabled when the forked JDK is > Java 15. We did this because earlier
> versions worked correctly. But new versions of errorprone always fail
> when the JDK is forked while compiling.
>
> if (rootProject.usesAltJvm && rootProject.runtimeJavaVersion >
> JavaVersion.VERSION_15) {
>   skipReason = "won't work with JDK
> ${rootProject.runtimeJavaVersion} if used as alternative java
> toolchain"
> }
>
> if (!propertyOrDefault("validation.errorprone",
> isCIBuild).asBoolean()) {
>   skipReason = "skipped on builds not running inside CI
> environments, pass -Pvalidation.errorprone=true to enable"
> }
>
> So it looks like the errorprone plugin got broken by a recent upgrade.
> It now always fails when forked JDK is used. So we shold disable it in
> this case. We just did not notice, as previously it was only disabled
> when the runtime java version was > 17.
>
> Nowadays we no longer run alternate JVMs with Java 12, 13, 14, 15. We
> run with Java 11, 17, 19, 20, 21. So it is always disabled except for
> Java 11. With RUNTIME_JAVA_HOME==JAVA_HOME we never fork, but as we
> use OpenJ9, we fork an BOOOOOM.
>
> I will post a PR soon.
>
> Uwe
>
> Am 14.11.2023 um 19:06 schrieb Uwe Schindler:
>>
>> Hi Dawid,
>>
>> Hah, the issue happens only if you pass CI=true (this is set by CI
>> systems), so errorprone is enabled. so do "export CI=true" and then
>> build with that config.
>>
>> So it looks like a combination of errorprone enabled with Java 11 OpenJ9.
>>
>> Uwe
>>
>> Am 13.11.2023 um 09:09 schrieb Dawid Weiss:
>>>
>>> Sure, thanks. What's strange is that we don't use add-opens
>>> anywhere, I think (there is a mention of it I left in one of the
>>> comments, but nothing else across the codebase uses this directive).
>>>
>>> > Task :lucene:distribution.tests:compileTestJava
>>> warning: [options] --add-opens has no effect at compile time
>>>
>>>
>>> On Sun, Nov 12, 2023 at 10:56?PM Uwe Schindler <uwe@thetaphi.de> wrote:
>>>
>>> Will check tomorrow, it's too late now.
>>>
>>> On Jenkins there were no windows builds with IBM and Java 11
>>> yet: https://jenkins.thetaphi.de/job/Lucene-9.x-Windows/
>>>
>>> Am 12.11.2023 um 22:00 schrieb Dawid Weiss:
>>>>
>>>> Hi Uwe,
>>>>
>>>> Can you reproduce this on Windows with the same JVM versions
>>>> though? Seems like I have exactly the same setup and yet this
>>>> works for me just fine. Strange.
>>>>
>>>> Dawid
>>>>
>>>> On Sun, Nov 12, 2023 at 9:52?PM Uwe Schindler <uwe@thetaphi.de>
>>>> wrote:
>>>>
>>>> This one was my first idea, too.
>>>>
>>>> It fails only with IBM Semeru in combination with Gradle
>>>> using Temurin.
>>>>
>>>> I will dig tomorrow on Jenkins server and print all debug info.
>>>>
>>>> Uwe
>>>>
>>>>
>>>> Am 12. November 2023 21:48:54 MEZ schrieb Dawid Weiss
>>>> <dawid.weiss@gmail.com>:
>>>>
>>>>
>>>> I can't reproduce this though - used exactly the same
>>>> JVMs (on Windows):
>>>>
>>>> > gradlew :lucene:distribution.tests:compileTestJava
>>>> --rerun-tasks --console=plain
>>>> Generating gradle.properties
>>>> ...
>>>> > Task :altJvmWarning
>>>> NOTE: Alternative java toolchain will be used for
>>>> compilation and tests:
>>>>   Project will use 11 (IBM JDK 11.0.20.1+1, home at:
>>>> c:\_tmp\jdk-11.0.20.1+1)
>>>>   Gradle runs with 11 (Eclipse Temurin JDK 11.0.21+9,
>>>> home at: C:\_tmp\jdk-11.0.21+9)
>>>> ...
>>>> > Task :lucene:distribution.tests:compileJava NO-SOURCE
>>>> > Task :lucene:distribution.tests:classes UP-TO-DATE
>>>> > Task :lucene:distribution.tests:compileTestJava
>>>>
>>>> BUILD SUCCESSFUL in 23s
>>>> 5 actionable tasks: 5 executed
>>>>
>>>> On main branch it works, no idea why:
>>>>
>>>>
>>>> O thought it's because of this:
>>>>
>>>> https://github.com/apache/lucene/commit/2e12a35c876a
>>>>
>>>> but I don't think so... seems to work for me on Windows
>>>> on branch_9x just fine?
>>>>
>>>> D.
>>>>
>>>> --
>>>> Uwe Schindler
>>>> Achterdiek 19, 28357 Bremen
>>>> https://www.thetaphi.de
>>>>
>>> --
>>> Uwe Schindler
>>> Achterdiek 19, D-28357 Bremen
>>> https://www.thetaphi.de
>>> eMail:uwe@thetaphi.de
>>>
>> --
>> Uwe Schindler
>> Achterdiek 19, D-28357 Bremen
>> https://www.thetaphi.de
>> eMail:uwe@thetaphi.de
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail:uwe@thetaphi.de

--
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail:uwe@thetaphi.de

Thanks Uwe!

On Tue, Nov 14, 2023 at 7:27?PM Uwe Schindler <uwe@thetaphi.de> wrote:

> Hi,
>
> For now the simplest is to disable always is an alternate JVM is used,
> just remove the second part of the first IF statement. In Main it is no
> longer relevant, as the runtime JDK is always >= 17, so it would always
> trigger the first if.
>
> I would not spend too much time, until errorprone gets its issues fixed.
> Actually they have some hacks but they only work with toolkits. Looks like
> the combination fork=true and not using toolkits breaks their construction
> of parameters to pass.
>
> Same for Solr.
> Am 14.11.2023 um 19:17 schrieb Uwe Schindler:
>
> Hi Dawid,
>
> The problem does not happen on Java 17, because errorprone is not enabled
> when the forked JDK is > Java 15. We did this because earlier versions
> worked correctly. But new versions of errorprone always fail when the JDK
> is forked while compiling.
>
> if (rootProject.usesAltJvm && rootProject.runtimeJavaVersion >
> JavaVersion.VERSION_15) {
> skipReason = "won't work with JDK ${rootProject.runtimeJavaVersion} if
> used as alternative java toolchain"
> }
>
> if (!propertyOrDefault("validation.errorprone", isCIBuild).asBoolean()) {
> skipReason = "skipped on builds not running inside CI environments, pass
> -Pvalidation.errorprone=true to enable"
> }
>
> So it looks like the errorprone plugin got broken by a recent upgrade. It
> now always fails when forked JDK is used. So we shold disable it in this
> case. We just did not notice, as previously it was only disabled when the
> runtime java version was > 17.
>
> Nowadays we no longer run alternate JVMs with Java 12, 13, 14, 15. We run
> with Java 11, 17, 19, 20, 21. So it is always disabled except for Java 11.
> With RUNTIME_JAVA_HOME==JAVA_HOME we never fork, but as we use OpenJ9, we
> fork an BOOOOOM.
>
> I will post a PR soon.
>
> Uwe
> Am 14.11.2023 um 19:06 schrieb Uwe Schindler:
>
> Hi Dawid,
>
> Hah, the issue happens only if you pass CI=true (this is set by CI
> systems), so errorprone is enabled. so do "export CI=true" and then build
> with that config.
>
> So it looks like a combination of errorprone enabled with Java 11 OpenJ9.
>
> Uwe
> Am 13.11.2023 um 09:09 schrieb Dawid Weiss:
>
>
> Sure, thanks. What's strange is that we don't use add-opens anywhere, I
> think (there is a mention of it I left in one of the
> comments, but nothing else across the codebase uses this directive).
>
> > Task :lucene:distribution.tests:compileTestJava
> warning: [options] --add-opens has no effect at compile time
>
>
>
> On Sun, Nov 12, 2023 at 10:56?PM Uwe Schindler <uwe@thetaphi.de> wrote:
>
>> Will check tomorrow, it's too late now.
>>
>> On Jenkins there were no windows builds with IBM and Java 11 yet:
>> https://jenkins.thetaphi.de/job/Lucene-9.x-Windows/
>> Am 12.11.2023 um 22:00 schrieb Dawid Weiss:
>>
>>
>> Hi Uwe,
>>
>> Can you reproduce this on Windows with the same JVM versions though?
>> Seems like I have exactly the same setup and yet this works for me just
>> fine. Strange.
>>
>> Dawid
>>
>> On Sun, Nov 12, 2023 at 9:52?PM Uwe Schindler <uwe@thetaphi.de> wrote:
>>
>>> This one was my first idea, too.
>>>
>>> It fails only with IBM Semeru in combination with Gradle using Temurin.
>>>
>>> I will dig tomorrow on Jenkins server and print all debug info.
>>>
>>> Uwe
>>>
>>>
>>> Am 12. November 2023 21:48:54 MEZ schrieb Dawid Weiss <
>>> dawid.weiss@gmail.com>:
>>>
>>>>
>>>> I can't reproduce this though - used exactly the same JVMs (on Windows):
>>>>
>>>> > gradlew :lucene:distribution.tests:compileTestJava --rerun-tasks
>>>> --console=plain
>>>> Generating gradle.properties
>>>> ...
>>>> > Task :altJvmWarning
>>>> NOTE: Alternative java toolchain will be used for compilation and tests:
>>>> Project will use 11 (IBM JDK 11.0.20.1+1, home at:
>>>> c:\_tmp\jdk-11.0.20.1+1)
>>>> Gradle runs with 11 (Eclipse Temurin JDK 11.0.21+9, home at:
>>>> C:\_tmp\jdk-11.0.21+9)
>>>> ...
>>>> > Task :lucene:distribution.tests:compileJava NO-SOURCE
>>>> > Task :lucene:distribution.tests:classes UP-TO-DATE
>>>> > Task :lucene:distribution.tests:compileTestJava
>>>>
>>>> BUILD SUCCESSFUL in 23s
>>>> 5 actionable tasks: 5 executed
>>>>
>>>> On main branch it works, no idea why:
>>>>>
>>>>
>>>> O thought it's because of this:
>>>>
>>>> https://github.com/apache/lucene/commit/2e12a35c876a
>>>>
>>>> but I don't think so... seems to work for me on Windows on branch_9x
>>>> just fine?
>>>>
>>>> D.
>>>>
>>> --
>>> Uwe Schindler
>>> Achterdiek 19, 28357 Bremen
>>> https://www.thetaphi.de
>>>
>> --
>> Uwe Schindler
>> Achterdiek 19, D-28357 Bremenhttps://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremenhttps://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremenhttps://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremenhttps://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>

Hi,

Here is the PR to disable errorprone with forked compiler:
https://github.com/apache/lucene/pull/12808; it makes no difference in
main, but on 9.x it will fix the issue.

I will merge the commit also to Solr as Jenkins fails there the same way.

Uwe

Am 14.11.2023 um 19:35 schrieb Dawid Weiss:
>
> Thanks Uwe!
>
> On Tue, Nov 14, 2023 at 7:27?PM Uwe Schindler <uwe@thetaphi.de> wrote:
>
> Hi,
>
> For now the simplest is to disable always is an alternate JVM is
> used, just remove the second part of the first IF statement. In
> Main it is no longer relevant, as the runtime JDK is always >= 17,
> so it would always trigger the first if.
>
> I would not spend too much time, until errorprone gets its issues
> fixed. Actually they have some hacks but they only work with
> toolkits. Looks like the combination fork=true and not using
> toolkits breaks their construction of parameters to pass.
>
> Same for Solr.
>
> Am 14.11.2023 um 19:17 schrieb Uwe Schindler:
>>
>> Hi Dawid,
>>
>> The problem does not happen on Java 17, because errorprone is not
>> enabled when the forked JDK is > Java 15. We did this because
>> earlier versions worked correctly. But new versions of errorprone
>> always fail when the JDK is forked while compiling.
>>
>> if (rootProject.usesAltJvm && rootProject.runtimeJavaVersion
>> > JavaVersion.VERSION_15) {
>>   skipReason = "won't work with JDK
>> ${rootProject.runtimeJavaVersion} if used as alternative java
>> toolchain"
>> }
>>
>> if (!propertyOrDefault("validation.errorprone",
>> isCIBuild).asBoolean()) {
>>   skipReason = "skipped on builds not running inside CI
>> environments, pass -Pvalidation.errorprone=true to enable"
>> }
>>
>> So it looks like the errorprone plugin got broken by a recent
>> upgrade. It now always fails when forked JDK is used. So we shold
>> disable it in this case. We just did not notice, as previously it
>> was only disabled when the runtime java version was > 17.
>>
>> Nowadays we no longer run alternate JVMs with Java 12, 13, 14,
>> 15. We run with Java 11, 17, 19, 20, 21. So it is always disabled
>> except for Java 11. With RUNTIME_JAVA_HOME==JAVA_HOME we never
>> fork, but as we use OpenJ9, we fork an BOOOOOM.
>>
>> I will post a PR soon.
>>
>> Uwe
>>
>> Am 14.11.2023 um 19:06 schrieb Uwe Schindler:
>>>
>>> Hi Dawid,
>>>
>>> Hah, the issue happens only if you pass CI=true (this is set by
>>> CI systems), so errorprone is enabled. so do "export CI=true"
>>> and then build with that config.
>>>
>>> So it looks like a combination of errorprone enabled with Java
>>> 11 OpenJ9.
>>>
>>> Uwe
>>>
>>> Am 13.11.2023 um 09:09 schrieb Dawid Weiss:
>>>>
>>>> Sure, thanks. What's strange is that we don't use add-opens
>>>> anywhere, I think (there is a mention of it I left in one of the
>>>> comments, but nothing else across the codebase uses this
>>>> directive).
>>>>
>>>> > Task :lucene:distribution.tests:compileTestJava
>>>> warning: [options] --add-opens has no effect at compile time
>>>>
>>>>
>>>> On Sun, Nov 12, 2023 at 10:56?PM Uwe Schindler
>>>> <uwe@thetaphi.de> wrote:
>>>>
>>>> Will check tomorrow, it's too late now.
>>>>
>>>> On Jenkins there were no windows builds with IBM and Java
>>>> 11 yet: https://jenkins.thetaphi.de/job/Lucene-9.x-Windows/
>>>>
>>>> Am 12.11.2023 um 22:00 schrieb Dawid Weiss:
>>>>>
>>>>> Hi Uwe,
>>>>>
>>>>> Can you reproduce this on Windows with the same JVM
>>>>> versions though? Seems like I have exactly the same setup
>>>>> and yet this works for me just fine. Strange.
>>>>>
>>>>> Dawid
>>>>>
>>>>> On Sun, Nov 12, 2023 at 9:52?PM Uwe Schindler
>>>>> <uwe@thetaphi.de> wrote:
>>>>>
>>>>> This one was my first idea, too.
>>>>>
>>>>> It fails only with IBM Semeru in combination with
>>>>> Gradle using Temurin.
>>>>>
>>>>> I will dig tomorrow on Jenkins server and print all
>>>>> debug info.
>>>>>
>>>>> Uwe
>>>>>
>>>>>
>>>>> Am 12. November 2023 21:48:54 MEZ schrieb Dawid Weiss
>>>>> <dawid.weiss@gmail.com>:
>>>>>
>>>>>
>>>>> I can't reproduce this though - used exactly the
>>>>> same JVMs (on Windows):
>>>>>
>>>>> > gradlew
>>>>> :lucene:distribution.tests:compileTestJava
>>>>> --rerun-tasks --console=plain
>>>>> Generating gradle.properties
>>>>> ...
>>>>> > Task :altJvmWarning
>>>>> NOTE: Alternative java toolchain will be used for
>>>>> compilation and tests:
>>>>>   Project will use 11 (IBM JDK 11.0.20.1+1, home
>>>>> at: c:\_tmp\jdk-11.0.20.1+1)
>>>>>   Gradle runs with 11 (Eclipse Temurin JDK
>>>>> 11.0.21+9, home at: C:\_tmp\jdk-11.0.21+9)
>>>>> ...
>>>>> > Task :lucene:distribution.tests:compileJava
>>>>> NO-SOURCE
>>>>> > Task :lucene:distribution.tests:classes UP-TO-DATE
>>>>> > Task :lucene:distribution.tests:compileTestJava
>>>>>
>>>>> BUILD SUCCESSFUL in 23s
>>>>> 5 actionable tasks: 5 executed
>>>>>
>>>>> On main branch it works, no idea why:
>>>>>
>>>>>
>>>>> O thought it's because of this:
>>>>>
>>>>> https://github.com/apache/lucene/commit/2e12a35c876a
>>>>>
>>>>> but I don't think so... seems to work for me on
>>>>> Windows on branch_9x just fine?
>>>>>
>>>>> D.
>>>>>
>>>>> --
>>>>> Uwe Schindler
>>>>> Achterdiek 19, 28357 Bremen
>>>>> https://www.thetaphi.de
>>>>>
>>>> --
>>>> Uwe Schindler
>>>> Achterdiek 19, D-28357 Bremen
>>>> https://www.thetaphi.de
>>>> eMail:uwe@thetaphi.de
>>>>
>>> --
>>> Uwe Schindler
>>> Achterdiek 19, D-28357 Bremen
>>> https://www.thetaphi.de
>>> eMail:uwe@thetaphi.de
>> --
>> Uwe Schindler
>> Achterdiek 19, D-28357 Bremen
>> https://www.thetaphi.de
>> eMail:uwe@thetaphi.de
>
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail:uwe@thetaphi.de
>
--
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail:uwe@thetaphi.de