After generating a pre-compiled image lucene-core (8.3.0) with jaotc (JDK
13.0.1), RamUsageEstimator class is never loaded - it fails to complete the
static initializer.
Steps to reproduce:
1)Generate the image with
>jaotc --info --ignore-errors --jar ~/.m2/repository/org/apache/lucene/lucene-core/8.3.0/lucene-core-8.3.0.jar --output lucene-core.so
2)Create a simple test class to trigger class loading
import java.io.IOException;
import java.nio.file.Path;
import java.nio.file.Paths;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.store.MMapDirectory;
public class TestAOT {
public static void main(String...args){
run();
System.out.println("Done");
}
static void run(){
try {
var iw = new IndexWriter(MMapDirectory.open(Paths.get("dir")), new
IndexWriterConfig());
iw.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
3)Run the class with the pre-compiled image
>java -XX:AOTLibrary=./lucene-core.so -XX:+PrintAOT -cp ~/.m2/repository/org/apache/lucene/lucene-core/8.3.0/lucene-core-8.3.0.jar TestAOT.java
4)The program never completes. After inspecting it with jstack <pid> it
shows that it's stuck in line 195 of RamUsageEstimator
"main" #1 prio=5 os_prio=0 cpu=174827,04ms elapsed=173,91s
tid=0x00007fe984018800 nid=0x2daf runnable [0x00007fe98bc3c000]
java.lang.Thread.State: RUNNABLE
at
org.apache.lucene.util.RamUsageEstimator.<clinit>(RamUsageEstimator.java:195)
at org.apache.lucene.util.ArrayUtil.<clinit>(ArrayUtil.java:31)
at org.apache.lucene.index.IndexWriter.<clinit>(IndexWriter.java:276)
at TestAOT.run(TestAOT.java:20)
at TestAOT.main(TestAOT.java:14)
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@13.0.1/Native
Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@13.0.1/NativeMethodAccessorImpl.java:62)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@13.0.1/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@13.0.1/Method.java:567)
at com.sun.tools.javac.launcher.Main.execute(jdk.compiler@13.0.1/Main.java:415)
at com.sun.tools.javac.launcher.Main.run(jdk.compiler@13.0.1/Main.java:192)
at com.sun.tools.javac.launcher.Main.main(jdk.compiler@13.0.1
/Main.java:132)
My guess is that the AOT compiler aggressive optimizations like
inlining/scalar replacing calls to Long.valueOf, are working against the
code's desired semantics.
Perhaps the logic to determine LONG_CACHE_[MIN/MAX]_VALUE could be
conditioned to VM version/vendor in Constants class. From [Open|Oracle]JDK
7 onwards (don't know about older versions) Long cache has a fixed range of
[-128,127], so there's no need to loop until a non-cached boxed value shows
up.
I know this compiler bug isn't Lucene's fault and as a workaround one could
use --compile-for-tiered or exclude RamUsageEstimator's methods from jaotc,
however, it would be nice to have a faster, allocation/loop-free
initializer for RamUsageEstimator nevertheless.
Cheers
13.0.1), RamUsageEstimator class is never loaded - it fails to complete the
static initializer.
Steps to reproduce:
1)Generate the image with
>jaotc --info --ignore-errors --jar ~/.m2/repository/org/apache/lucene/lucene-core/8.3.0/lucene-core-8.3.0.jar --output lucene-core.so
2)Create a simple test class to trigger class loading
import java.io.IOException;
import java.nio.file.Path;
import java.nio.file.Paths;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.store.MMapDirectory;
public class TestAOT {
public static void main(String...args){
run();
System.out.println("Done");
}
static void run(){
try {
var iw = new IndexWriter(MMapDirectory.open(Paths.get("dir")), new
IndexWriterConfig());
iw.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
3)Run the class with the pre-compiled image
>java -XX:AOTLibrary=./lucene-core.so -XX:+PrintAOT -cp ~/.m2/repository/org/apache/lucene/lucene-core/8.3.0/lucene-core-8.3.0.jar TestAOT.java
4)The program never completes. After inspecting it with jstack <pid> it
shows that it's stuck in line 195 of RamUsageEstimator
"main" #1 prio=5 os_prio=0 cpu=174827,04ms elapsed=173,91s
tid=0x00007fe984018800 nid=0x2daf runnable [0x00007fe98bc3c000]
java.lang.Thread.State: RUNNABLE
at
org.apache.lucene.util.RamUsageEstimator.<clinit>(RamUsageEstimator.java:195)
at org.apache.lucene.util.ArrayUtil.<clinit>(ArrayUtil.java:31)
at org.apache.lucene.index.IndexWriter.<clinit>(IndexWriter.java:276)
at TestAOT.run(TestAOT.java:20)
at TestAOT.main(TestAOT.java:14)
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@13.0.1/Native
Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@13.0.1/NativeMethodAccessorImpl.java:62)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@13.0.1/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@13.0.1/Method.java:567)
at com.sun.tools.javac.launcher.Main.execute(jdk.compiler@13.0.1/Main.java:415)
at com.sun.tools.javac.launcher.Main.run(jdk.compiler@13.0.1/Main.java:192)
at com.sun.tools.javac.launcher.Main.main(jdk.compiler@13.0.1
/Main.java:132)
My guess is that the AOT compiler aggressive optimizations like
inlining/scalar replacing calls to Long.valueOf, are working against the
code's desired semantics.
Perhaps the logic to determine LONG_CACHE_[MIN/MAX]_VALUE could be
conditioned to VM version/vendor in Constants class. From [Open|Oracle]JDK
7 onwards (don't know about older versions) Long cache has a fixed range of
[-128,127], so there's no need to loop until a non-cached boxed value shows
up.
I know this compiler bug isn't Lucene's fault and as a workaround one could
use --compile-for-tiered or exclude RamUsageEstimator's methods from jaotc,
however, it would be nice to have a faster, allocation/loop-free
initializer for RamUsageEstimator nevertheless.
Cheers