Mailing List Archive

Re: Running 10.0 build with a custom lucene 9.5
You are probably missing the contents of WEB-INF in your custom jar?
Roughly speaking the files in there define run-time-bound "services"
that are looked up by name by the JDK's service-loader API.

On Sat, May 13, 2023 at 9:33?AM Gus Heck <gus.heck@gmail.com> wrote:
>
> Cross posting to lucene on the possibility that folks here are more likely to add customized lucene to Solr and recognize what I'm stumbling on? (zero responses on solr list)
>
> Note that the specific test that I happened to copy is not the issue, all tests are doing this (or at least so many tests are failing I can't see the ones that are passing easily).
>
> ---------- Forwarded message ---------
> From: Gus Heck <gus.heck@gmail.com>
> Date: Wed, May 10, 2023 at 6:50?PM
> Subject: Running 10.0 build with a custom lucene 9.5
> To: <dev@solr.apache.org>
>
>
> Lucene:
>
> I made a tweak to lucene for something I'm investigating, gave it a new version, deployed to mavenLocal()
> I have verified that the jars are built with correct META-INF/services files
>
> Solr:
>
> I added mavenLocal() in gradle/globals.gradle
> I removed the license file sha1 sigs for the default lucene & creates signatures for my test version
> I updated versions.props
> I updated versions.lock
>
> Now when I run individual solr tests via my ide they seem to pass, but virtually every test run via gradle fails with something like:
>
> org.apache.solr.embedded.TestJettySolrRunner > classMethod FAILED
> java.lang.ExceptionInInitializerError
> at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
> at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> Caused by:
> java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: []
> at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
> at org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> ... 19 more
>
> org.apache.solr.embedded.TestJettySolrRunner > classMethod FAILED
> java.lang.NullPointerException
> at java.base/java.util.Objects.requireNonNull(Objects.java:221)
> at org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
> at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
> at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> org.apache.solr.embedded.TestJettySolrRunner > test suite's output saved to /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt, copied below:
> > java.lang.ExceptionInInitializerError
> > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
> > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > at java.base/java.lang.Thread.run(Thread.java:829)
> >
> > Caused by:
> > java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: []
> > at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
> > at org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> > ... 19 more
> > java.lang.NullPointerException
> > at java.base/java.util.Objects.requireNonNull(Objects.java:221)
> > at org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
> > at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
> > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > at java.base/java.lang.Thread.run(Thread.java:829)
> 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc. 11.0.19 (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
> 2> NOTE: All tests run in this JVM: [TestJettySolrRunner]
> 2> NOTE: reproduce with: gradlew test --tests TestJettySolrRunner -Dtests.seed=2FB768B5A303764 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>
> The only difference I've been able to determine is that the IDE does not run the test with a security manager, but that feels like a red herring since nothing is throwing security exception (or if it is, it's getting swallowed!).
>
> I have debugged and I can see that the classloaders have the new lucene jars I built on the classpath (in /Users/gus/.m2/repository as they should), but I haven't yet figuree out why the services are not loading.
>
> -Gus
>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Running 10.0 build with a custom lucene 9.5 [ In reply to ]
sorry - META-INF not WEB-INF

On Sat, May 13, 2023 at 10:12?AM Michael Sokolov <msokolov@gmail.com> wrote:
>
> You are probably missing the contents of WEB-INF in your custom jar?
> Roughly speaking the files in there define run-time-bound "services"
> that are looked up by name by the JDK's service-loader API.
>
> On Sat, May 13, 2023 at 9:33?AM Gus Heck <gus.heck@gmail.com> wrote:
> >
> > Cross posting to lucene on the possibility that folks here are more likely to add customized lucene to Solr and recognize what I'm stumbling on? (zero responses on solr list)
> >
> > Note that the specific test that I happened to copy is not the issue, all tests are doing this (or at least so many tests are failing I can't see the ones that are passing easily).
> >
> > ---------- Forwarded message ---------
> > From: Gus Heck <gus.heck@gmail.com>
> > Date: Wed, May 10, 2023 at 6:50?PM
> > Subject: Running 10.0 build with a custom lucene 9.5
> > To: <dev@solr.apache.org>
> >
> >
> > Lucene:
> >
> > I made a tweak to lucene for something I'm investigating, gave it a new version, deployed to mavenLocal()
> > I have verified that the jars are built with correct META-INF/services files
> >
> > Solr:
> >
> > I added mavenLocal() in gradle/globals.gradle
> > I removed the license file sha1 sigs for the default lucene & creates signatures for my test version
> > I updated versions.props
> > I updated versions.lock
> >
> > Now when I run individual solr tests via my ide they seem to pass, but virtually every test run via gradle fails with something like:
> >
> > org.apache.solr.embedded.TestJettySolrRunner > classMethod FAILED
> > java.lang.ExceptionInInitializerError
> > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
> > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > at java.base/java.lang.Thread.run(Thread.java:829)
> >
> > Caused by:
> > java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: []
> > at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
> > at org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> > ... 19 more
> >
> > org.apache.solr.embedded.TestJettySolrRunner > classMethod FAILED
> > java.lang.NullPointerException
> > at java.base/java.util.Objects.requireNonNull(Objects.java:221)
> > at org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
> > at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
> > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > at java.base/java.lang.Thread.run(Thread.java:829)
> >
> > org.apache.solr.embedded.TestJettySolrRunner > test suite's output saved to /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt, copied below:
> > > java.lang.ExceptionInInitializerError
> > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > > at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
> > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >
> > > Caused by:
> > > java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: []
> > > at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
> > > at org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> > > ... 19 more
> > > java.lang.NullPointerException
> > > at java.base/java.util.Objects.requireNonNull(Objects.java:221)
> > > at org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
> > > at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
> > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > > at java.base/java.lang.Thread.run(Thread.java:829)
> > 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc. 11.0.19 (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
> > 2> NOTE: All tests run in this JVM: [TestJettySolrRunner]
> > 2> NOTE: reproduce with: gradlew test --tests TestJettySolrRunner -Dtests.seed=2FB768B5A303764 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
> >
> > The only difference I've been able to determine is that the IDE does not run the test with a security manager, but that feels like a red herring since nothing is throwing security exception (or if it is, it's getting swallowed!).
> >
> > I have debugged and I can see that the classloaders have the new lucene jars I built on the classpath (in /Users/gus/.m2/repository as they should), but I haven't yet figuree out why the services are not loading.
> >
> > -Gus
> >
> >
> > --
> > http://www.needhamsoftware.com (work)
> > http://www.the111shift.com (play)
> >
> >
> > --
> > http://www.needhamsoftware.com (work)
> > http://www.the111shift.com (play)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Running 10.0 build with a custom lucene 9.5 [ In reply to ]
doh I actually read your email and you said you already checked that -
I'm going to send out one of those "sokolov would like to retract the
previous email" emails. Does GMail even pretend to do that? I don't
know what's going on there! sorry

On Sat, May 13, 2023 at 10:13?AM Michael Sokolov <msokolov@gmail.com> wrote:
>
> sorry - META-INF not WEB-INF
>
> On Sat, May 13, 2023 at 10:12?AM Michael Sokolov <msokolov@gmail.com> wrote:
> >
> > You are probably missing the contents of WEB-INF in your custom jar?
> > Roughly speaking the files in there define run-time-bound "services"
> > that are looked up by name by the JDK's service-loader API.
> >
> > On Sat, May 13, 2023 at 9:33?AM Gus Heck <gus.heck@gmail.com> wrote:
> > >
> > > Cross posting to lucene on the possibility that folks here are more likely to add customized lucene to Solr and recognize what I'm stumbling on? (zero responses on solr list)
> > >
> > > Note that the specific test that I happened to copy is not the issue, all tests are doing this (or at least so many tests are failing I can't see the ones that are passing easily).
> > >
> > > ---------- Forwarded message ---------
> > > From: Gus Heck <gus.heck@gmail.com>
> > > Date: Wed, May 10, 2023 at 6:50?PM
> > > Subject: Running 10.0 build with a custom lucene 9.5
> > > To: <dev@solr.apache.org>
> > >
> > >
> > > Lucene:
> > >
> > > I made a tweak to lucene for something I'm investigating, gave it a new version, deployed to mavenLocal()
> > > I have verified that the jars are built with correct META-INF/services files
> > >
> > > Solr:
> > >
> > > I added mavenLocal() in gradle/globals.gradle
> > > I removed the license file sha1 sigs for the default lucene & creates signatures for my test version
> > > I updated versions.props
> > > I updated versions.lock
> > >
> > > Now when I run individual solr tests via my ide they seem to pass, but virtually every test run via gradle fails with something like:
> > >
> > > org.apache.solr.embedded.TestJettySolrRunner > classMethod FAILED
> > > java.lang.ExceptionInInitializerError
> > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > > at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
> > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >
> > > Caused by:
> > > java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: []
> > > at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
> > > at org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> > > ... 19 more
> > >
> > > org.apache.solr.embedded.TestJettySolrRunner > classMethod FAILED
> > > java.lang.NullPointerException
> > > at java.base/java.util.Objects.requireNonNull(Objects.java:221)
> > > at org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
> > > at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
> > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >
> > > org.apache.solr.embedded.TestJettySolrRunner > test suite's output saved to /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt, copied below:
> > > > java.lang.ExceptionInInitializerError
> > > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > > > at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
> > > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > > > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > > > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > > > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > > > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > > > at java.base/java.lang.Thread.run(Thread.java:829)
> > > >
> > > > Caused by:
> > > > java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: []
> > > > at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
> > > > at org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> > > > ... 19 more
> > > > java.lang.NullPointerException
> > > > at java.base/java.util.Objects.requireNonNull(Objects.java:221)
> > > > at org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
> > > > at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
> > > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > > > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > > > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > > > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > > > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > > > at java.base/java.lang.Thread.run(Thread.java:829)
> > > 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc. 11.0.19 (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
> > > 2> NOTE: All tests run in this JVM: [TestJettySolrRunner]
> > > 2> NOTE: reproduce with: gradlew test --tests TestJettySolrRunner -Dtests.seed=2FB768B5A303764 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
> > >
> > > The only difference I've been able to determine is that the IDE does not run the test with a security manager, but that feels like a red herring since nothing is throwing security exception (or if it is, it's getting swallowed!).
> > >
> > > I have debugged and I can see that the classloaders have the new lucene jars I built on the classpath (in /Users/gus/.m2/repository as they should), but I haven't yet figuree out why the services are not loading.
> > >
> > > -Gus
> > >
> > >
> > > --
> > > http://www.needhamsoftware.com (work)
> > > http://www.the111shift.com (play)
> > >
> > >
> > > --
> > > http://www.needhamsoftware.com (work)
> > > http://www.the111shift.com (play)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Running 10.0 build with a custom lucene 9.5 [ In reply to ]
Gus, I haven't done this myself, but are you using the instructions
provided in Solr's "gradle/lucene-dev/lucene-dev-repo-composite.gradle"?

It looks like you need to specify the development lucene version
differently than other dependencies...

- Houston

On Sat, May 13, 2023 at 10:14?AM Michael Sokolov <msokolov@gmail.com> wrote:

> doh I actually read your email and you said you already checked that -
> I'm going to send out one of those "sokolov would like to retract the
> previous email" emails. Does GMail even pretend to do that? I don't
> know what's going on there! sorry
>
> On Sat, May 13, 2023 at 10:13?AM Michael Sokolov <msokolov@gmail.com>
> wrote:
> >
> > sorry - META-INF not WEB-INF
> >
> > On Sat, May 13, 2023 at 10:12?AM Michael Sokolov <msokolov@gmail.com>
> wrote:
> > >
> > > You are probably missing the contents of WEB-INF in your custom jar?
> > > Roughly speaking the files in there define run-time-bound "services"
> > > that are looked up by name by the JDK's service-loader API.
> > >
> > > On Sat, May 13, 2023 at 9:33?AM Gus Heck <gus.heck@gmail.com> wrote:
> > > >
> > > > Cross posting to lucene on the possibility that folks here are more
> likely to add customized lucene to Solr and recognize what I'm stumbling
> on? (zero responses on solr list)
> > > >
> > > > Note that the specific test that I happened to copy is not the
> issue, all tests are doing this (or at least so many tests are failing I
> can't see the ones that are passing easily).
> > > >
> > > > ---------- Forwarded message ---------
> > > > From: Gus Heck <gus.heck@gmail.com>
> > > > Date: Wed, May 10, 2023 at 6:50?PM
> > > > Subject: Running 10.0 build with a custom lucene 9.5
> > > > To: <dev@solr.apache.org>
> > > >
> > > >
> > > > Lucene:
> > > >
> > > > I made a tweak to lucene for something I'm investigating, gave it a
> new version, deployed to mavenLocal()
> > > > I have verified that the jars are built with correct
> META-INF/services files
> > > >
> > > > Solr:
> > > >
> > > > I added mavenLocal() in gradle/globals.gradle
> > > > I removed the license file sha1 sigs for the default lucene &
> creates signatures for my test version
> > > > I updated versions.props
> > > > I updated versions.lock
> > > >
> > > > Now when I run individual solr tests via my ide they seem to pass,
> but virtually every test run via gradle fails with something like:
> > > >
> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod FAILED
> > > > java.lang.ExceptionInInitializerError
> > > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > > > at
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
> > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > > > at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > > > at java.base/java.lang.Thread.run(Thread.java:829)
> > > >
> > > > Caused by:
> > > > java.lang.IllegalArgumentException: An SPI class of type
> org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist. You
> need to add the corresponding JAR file supporting this SPI to your
> classpath. The current classpath supports the following names: []
> > > > at
> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
> > > > at
> org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> > > > ... 19 more
> > > >
> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod FAILED
> > > > java.lang.NullPointerException
> > > > at
> java.base/java.util.Objects.requireNonNull(Objects.java:221)
> > > > at org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
> > > > at
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
> > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > > > at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > > > at java.base/java.lang.Thread.run(Thread.java:829)
> > > >
> > > > org.apache.solr.embedded.TestJettySolrRunner > test suite's output
> saved to
> /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt,
> copied below:
> > > > > java.lang.ExceptionInInitializerError
> > > > > at
> org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > > > > at
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
> > > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > > at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > > at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > > > > at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > > > > at java.base/java.lang.Thread.run(Thread.java:829)
> > > > >
> > > > > Caused by:
> > > > > java.lang.IllegalArgumentException: An SPI class of
> type org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist.
> You need to add the corresponding JAR file supporting this SPI to your
> classpath. The current classpath supports the following names: []
> > > > > at
> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
> > > > > at
> org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> > > > > ... 19 more
> > > > > java.lang.NullPointerException
> > > > > at
> java.base/java.util.Objects.requireNonNull(Objects.java:221)
> > > > > at
> org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
> > > > > at
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
> > > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > > at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > > at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > > > > at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > > > > at java.base/java.lang.Thread.run(Thread.java:829)
> > > > 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc. 11.0.19
> (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
> > > > 2> NOTE: All tests run in this JVM: [TestJettySolrRunner]
> > > > 2> NOTE: reproduce with: gradlew test --tests TestJettySolrRunner
> -Dtests.seed=2FB768B5A303764 -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> > > >
> > > > The only difference I've been able to determine is that the IDE does
> not run the test with a security manager, but that feels like a red herring
> since nothing is throwing security exception (or if it is, it's getting
> swallowed!).
> > > >
> > > > I have debugged and I can see that the classloaders have the new
> lucene jars I built on the classpath (in /Users/gus/.m2/repository as they
> should), but I haven't yet figuree out why the services are not loading.
> > > >
> > > > -Gus
> > > >
> > > >
> > > > --
> > > > http://www.needhamsoftware.com (work)
> > > > http://www.the111shift.com (play)
> > > >
> > > >
> > > > --
> > > > http://www.needhamsoftware.com (work)
> > > > http://www.the111shift.com (play)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: Running 10.0 build with a custom lucene 9.5 [ In reply to ]
I hadn't seen that one. Thanks, I'll look at it. It already looks a bit
confusing though since it seems to have options for pointing to a repo, but
I appear to be pulling the jars successfully from .m2/repository already...
(except then they don't work, so successful means I see them in the
classpath of the relevant classloader). And if we can't deploy a valid jar
to mavenLocal for some reason (tweaked the solr build so it sees
mavenLocal()), (or solr can't consume such a jar) that seems like an issue
for whichever one is breaking that.

Debugging: The JDK appears to be attempting to load the services file from
modules, but not seeing the lucene module. (just the jdk ones) Also it
passes through a block that says:

// not in a package of a module defined to this loader
for (URL url : findMiscResource(name)) {

(but then
iterates jdk.internal.loader.BuiltinClassLoader#nameToModule.values() to
load things anyway)

-Gus

On Mon, May 15, 2023 at 10:54?AM Houston Putman <houstonputman@gmail.com>
wrote:

> Gus, I haven't done this myself, but are you using the instructions
> provided in Solr's "gradle/lucene-dev/lucene-dev-repo-composite.gradle"?
>
> It looks like you need to specify the development lucene version
> differently than other dependencies...
>
> - Houston
>
> On Sat, May 13, 2023 at 10:14?AM Michael Sokolov <msokolov@gmail.com>
> wrote:
>
>> doh I actually read your email and you said you already checked that -
>> I'm going to send out one of those "sokolov would like to retract the
>> previous email" emails. Does GMail even pretend to do that? I don't
>> know what's going on there! sorry
>>
>> On Sat, May 13, 2023 at 10:13?AM Michael Sokolov <msokolov@gmail.com>
>> wrote:
>> >
>> > sorry - META-INF not WEB-INF
>> >
>> > On Sat, May 13, 2023 at 10:12?AM Michael Sokolov <msokolov@gmail.com>
>> wrote:
>> > >
>> > > You are probably missing the contents of WEB-INF in your custom jar?
>> > > Roughly speaking the files in there define run-time-bound "services"
>> > > that are looked up by name by the JDK's service-loader API.
>> > >
>> > > On Sat, May 13, 2023 at 9:33?AM Gus Heck <gus.heck@gmail.com> wrote:
>> > > >
>> > > > Cross posting to lucene on the possibility that folks here are more
>> likely to add customized lucene to Solr and recognize what I'm stumbling
>> on? (zero responses on solr list)
>> > > >
>> > > > Note that the specific test that I happened to copy is not the
>> issue, all tests are doing this (or at least so many tests are failing I
>> can't see the ones that are passing easily).
>> > > >
>> > > > ---------- Forwarded message ---------
>> > > > From: Gus Heck <gus.heck@gmail.com>
>> > > > Date: Wed, May 10, 2023 at 6:50?PM
>> > > > Subject: Running 10.0 build with a custom lucene 9.5
>> > > > To: <dev@solr.apache.org>
>> > > >
>> > > >
>> > > > Lucene:
>> > > >
>> > > > I made a tweak to lucene for something I'm investigating, gave it a
>> new version, deployed to mavenLocal()
>> > > > I have verified that the jars are built with correct
>> META-INF/services files
>> > > >
>> > > > Solr:
>> > > >
>> > > > I added mavenLocal() in gradle/globals.gradle
>> > > > I removed the license file sha1 sigs for the default lucene &
>> creates signatures for my test version
>> > > > I updated versions.props
>> > > > I updated versions.lock
>> > > >
>> > > > Now when I run individual solr tests via my ide they seem to pass,
>> but virtually every test run via gradle fails with something like:
>> > > >
>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod FAILED
>> > > > java.lang.ExceptionInInitializerError
>> > > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>> > > > at
>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>> > > > at
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>> > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > at
>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>> > > > at
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > > > at
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > at
>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>> > > > at
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>> > > > at
>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>> > > > at
>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>> > > > at
>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>> > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > at
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>> > > > at
>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
>> > > >
>> > > > Caused by:
>> > > > java.lang.IllegalArgumentException: An SPI class of type
>> org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist. You
>> need to add the corresponding JAR file supporting this SPI to your
>> classpath. The current classpath supports the following names: []
>> > > > at
>> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>> > > > at
>> org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>> > > > ... 19 more
>> > > >
>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod FAILED
>> > > > java.lang.NullPointerException
>> > > > at
>> java.base/java.util.Objects.requireNonNull(Objects.java:221)
>> > > > at org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>> > > > at
>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>> > > > at
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>> > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > at
>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>> > > > at
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > > > at
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > at
>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>> > > > at
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>> > > > at
>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>> > > > at
>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>> > > > at
>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>> > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > at
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>> > > > at
>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
>> > > >
>> > > > org.apache.solr.embedded.TestJettySolrRunner > test suite's output
>> saved to
>> /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt,
>> copied below:
>> > > > > java.lang.ExceptionInInitializerError
>> > > > > at
>> org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>> > > > > at
>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>> > > > > at
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>> > > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > > at
>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>> > > > > at
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > > > > at
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > > at
>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>> > > > > at
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>> > > > > at
>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>> > > > > at
>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>> > > > > at
>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>> > > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>> > > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > > at
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>> > > > > at
>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>> > > > > at java.base/java.lang.Thread.run(Thread.java:829)
>> > > > >
>> > > > > Caused by:
>> > > > > java.lang.IllegalArgumentException: An SPI class of
>> type org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist.
>> You need to add the corresponding JAR file supporting this SPI to your
>> classpath. The current classpath supports the following names: []
>> > > > > at
>> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>> > > > > at
>> org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>> > > > > ... 19 more
>> > > > > java.lang.NullPointerException
>> > > > > at
>> java.base/java.util.Objects.requireNonNull(Objects.java:221)
>> > > > > at
>> org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>> > > > > at
>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>> > > > > at
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>> > > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > > at
>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>> > > > > at
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > > > > at
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > > at
>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>> > > > > at
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>> > > > > at
>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>> > > > > at
>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>> > > > > at
>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>> > > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>> > > > > at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > > > > at
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>> > > > > at
>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>> > > > > at java.base/java.lang.Thread.run(Thread.java:829)
>> > > > 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc. 11.0.19
>> (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
>> > > > 2> NOTE: All tests run in this JVM: [TestJettySolrRunner]
>> > > > 2> NOTE: reproduce with: gradlew test --tests TestJettySolrRunner
>> -Dtests.seed=2FB768B5A303764 -Dtests.asserts=true
>> -Dtests.file.encoding=UTF-8
>> > > >
>> > > > The only difference I've been able to determine is that the IDE
>> does not run the test with a security manager, but that feels like a red
>> herring since nothing is throwing security exception (or if it is, it's
>> getting swallowed!).
>> > > >
>> > > > I have debugged and I can see that the classloaders have the new
>> lucene jars I built on the classpath (in /Users/gus/.m2/repository as they
>> should), but I haven't yet figuree out why the services are not loading.
>> > > >
>> > > > -Gus
>> > > >
>> > > >
>> > > > --
>> > > > http://www.needhamsoftware.com (work)
>> > > > http://www.the111shift.com (play)
>> > > >
>> > > >
>> > > > --
>> > > > http://www.needhamsoftware.com (work)
>> > > > http://www.the111shift.com (play)
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>

--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: Running 10.0 build with a custom lucene 9.5 [ In reply to ]
random guess - does it have something to do with modules?

On Mon, May 15, 2023 at 11:14?AM Gus Heck <gus.heck@gmail.com> wrote:
>
> I hadn't seen that one. Thanks, I'll look at it. It already looks a bit confusing though since it seems to have options for pointing to a repo, but I appear to be pulling the jars successfully from .m2/repository already... (except then they don't work, so successful means I see them in the classpath of the relevant classloader). And if we can't deploy a valid jar to mavenLocal for some reason (tweaked the solr build so it sees mavenLocal()), (or solr can't consume such a jar) that seems like an issue for whichever one is breaking that.
>
> Debugging: The JDK appears to be attempting to load the services file from modules, but not seeing the lucene module. (just the jdk ones) Also it passes through a block that says:
>
> // not in a package of a module defined to this loader
> for (URL url : findMiscResource(name)) {
>
> (but then iterates jdk.internal.loader.BuiltinClassLoader#nameToModule.values() to load things anyway)
>
> -Gus
>
> On Mon, May 15, 2023 at 10:54?AM Houston Putman <houstonputman@gmail.com> wrote:
>>
>> Gus, I haven't done this myself, but are you using the instructions provided in Solr's "gradle/lucene-dev/lucene-dev-repo-composite.gradle"?
>>
>> It looks like you need to specify the development lucene version differently than other dependencies...
>>
>> - Houston
>>
>> On Sat, May 13, 2023 at 10:14?AM Michael Sokolov <msokolov@gmail.com> wrote:
>>>
>>> doh I actually read your email and you said you already checked that -
>>> I'm going to send out one of those "sokolov would like to retract the
>>> previous email" emails. Does GMail even pretend to do that? I don't
>>> know what's going on there! sorry
>>>
>>> On Sat, May 13, 2023 at 10:13?AM Michael Sokolov <msokolov@gmail.com> wrote:
>>> >
>>> > sorry - META-INF not WEB-INF
>>> >
>>> > On Sat, May 13, 2023 at 10:12?AM Michael Sokolov <msokolov@gmail.com> wrote:
>>> > >
>>> > > You are probably missing the contents of WEB-INF in your custom jar?
>>> > > Roughly speaking the files in there define run-time-bound "services"
>>> > > that are looked up by name by the JDK's service-loader API.
>>> > >
>>> > > On Sat, May 13, 2023 at 9:33?AM Gus Heck <gus.heck@gmail.com> wrote:
>>> > > >
>>> > > > Cross posting to lucene on the possibility that folks here are more likely to add customized lucene to Solr and recognize what I'm stumbling on? (zero responses on solr list)
>>> > > >
>>> > > > Note that the specific test that I happened to copy is not the issue, all tests are doing this (or at least so many tests are failing I can't see the ones that are passing easily).
>>> > > >
>>> > > > ---------- Forwarded message ---------
>>> > > > From: Gus Heck <gus.heck@gmail.com>
>>> > > > Date: Wed, May 10, 2023 at 6:50?PM
>>> > > > Subject: Running 10.0 build with a custom lucene 9.5
>>> > > > To: <dev@solr.apache.org>
>>> > > >
>>> > > >
>>> > > > Lucene:
>>> > > >
>>> > > > I made a tweak to lucene for something I'm investigating, gave it a new version, deployed to mavenLocal()
>>> > > > I have verified that the jars are built with correct META-INF/services files
>>> > > >
>>> > > > Solr:
>>> > > >
>>> > > > I added mavenLocal() in gradle/globals.gradle
>>> > > > I removed the license file sha1 sigs for the default lucene & creates signatures for my test version
>>> > > > I updated versions.props
>>> > > > I updated versions.lock
>>> > > >
>>> > > > Now when I run individual solr tests via my ide they seem to pass, but virtually every test run via gradle fails with something like:
>>> > > >
>>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod FAILED
>>> > > > java.lang.ExceptionInInitializerError
>>> > > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>>> > > > at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>>> > > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>>> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>> > > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>> > > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>> > > > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>> > > > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>> > > > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>> > > > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
>>> > > >
>>> > > > Caused by:
>>> > > > java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: []
>>> > > > at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>>> > > > at org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>> > > > ... 19 more
>>> > > >
>>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod FAILED
>>> > > > java.lang.NullPointerException
>>> > > > at java.base/java.util.Objects.requireNonNull(Objects.java:221)
>>> > > > at org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>>> > > > at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>>> > > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>>> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>> > > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>> > > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>> > > > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>> > > > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>> > > > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>> > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>> > > > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
>>> > > >
>>> > > > org.apache.solr.embedded.TestJettySolrRunner > test suite's output saved to /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt, copied below:
>>> > > > > java.lang.ExceptionInInitializerError
>>> > > > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>>> > > > > at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>>> > > > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>>> > > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>> > > > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > > > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>> > > > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>> > > > > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>> > > > > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>> > > > > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>> > > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>> > > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>> > > > > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>> > > > > at java.base/java.lang.Thread.run(Thread.java:829)
>>> > > > >
>>> > > > > Caused by:
>>> > > > > java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: []
>>> > > > > at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>>> > > > > at org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>> > > > > ... 19 more
>>> > > > > java.lang.NullPointerException
>>> > > > > at java.base/java.util.Objects.requireNonNull(Objects.java:221)
>>> > > > > at org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>>> > > > > at org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>>> > > > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>>> > > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>> > > > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > > > > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>> > > > > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>> > > > > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>> > > > > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>> > > > > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>> > > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>> > > > > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > > > > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>> > > > > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>> > > > > at java.base/java.lang.Thread.run(Thread.java:829)
>>> > > > 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc. 11.0.19 (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
>>> > > > 2> NOTE: All tests run in this JVM: [TestJettySolrRunner]
>>> > > > 2> NOTE: reproduce with: gradlew test --tests TestJettySolrRunner -Dtests.seed=2FB768B5A303764 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>>> > > >
>>> > > > The only difference I've been able to determine is that the IDE does not run the test with a security manager, but that feels like a red herring since nothing is throwing security exception (or if it is, it's getting swallowed!).
>>> > > >
>>> > > > I have debugged and I can see that the classloaders have the new lucene jars I built on the classpath (in /Users/gus/.m2/repository as they should), but I haven't yet figuree out why the services are not loading.
>>> > > >
>>> > > > -Gus
>>> > > >
>>> > > >
>>> > > > --
>>> > > > http://www.needhamsoftware.com (work)
>>> > > > http://www.the111shift.com (play)
>>> > > >
>>> > > >
>>> > > > --
>>> > > > http://www.needhamsoftware.com (work)
>>> > > > http://www.the111shift.com (play)
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Running 10.0 build with a custom lucene 9.5 [ In reply to ]
Found it.

It's a solr thing made worse by the interaction of lucene testutils and
jdk.internal.loader.URLClassPath's decision to hide anything gone wrong
when checking a URL
/*
* Checks whether the resource URL should be returned.
* Returns null on security check failure.
* Called by java.net.URLClassLoader.
*/
public static URL checkURL(URL url) {
if (url != null) {
try {
check(url);
} catch (Exception e) {
return null;
}
}
return url;
}

Yay. Fun. JDK classes swallowing exceptions silently.

At the start of this it only took me a little while to discover that there
was a security manager in play via debugging. Remembering that I saw emails
about that, I went to jira, found the ticket enabling it by default in 9.x
and eventually tracked down the name of the security policy file by reading
solr.in.sh and /bin/solr... The key issue that tripped me up is that the
tests have a *separate* security policy file, and there was pretty much no
way to know this without extensive reading of the build. Thus I got thrown
off track when

permission java.io.FilePermission
"${user.home}${/}.m2${/}repository${/}-", "read";

To solr/server/etc/security.policy had no effect. That and the fact that
no security exception was reported, led me to start chasing increasingly
improbable hypotheses. Many hours later when I went back to debugging
deeply into class loading, I found that the code was actually reading the
jar files in question, and then I finally caught it throwing a security
exception during my debugging.

It turns out that adding the above permission to
gradle/testing/randomization/policies/solr-tests.policy allows the test to
pass. [1]

I think we need to document this somewhere (or someone needs to point me to
the doc I missed, FWIW I hit this basically following the process in
dev-docs/dependency-upgrades.adoc treating lucene like a dependency, and
unaware that there is a "shortcut" mode for lucene specifically in
gradle/lucene-dev/lucene-dev-repo-composite.gradle and I find reading that
file none-to clear anyway)

That's the solr part, the lucene part is that the security exception is hit
when in org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
when org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv#before
does

savedCodec = Codec.getDefault();

The error message "An SPI class of type org.apache.lucene.codecs.Codec with
name 'Lucene95' does not exist." was moderately misleading because the file
and the services files in the jar definitely did exist. This message should
vary if there is an installed security manager, maybe saying something like:

"An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95'
does not exist. We have detected that a security manager is installed so it
is also possible that the jar containing the codec is inaccessible under
the current security policy. (Java does not throw SecurityException if this
is the case, it just ignores the jar!)" [2]

Having that explicitly called out would have been SUPER helpful.

-Gus

[1]: https://issues.apache.org/jira/browse/SOLR-16804
[2]: https://github.com/apache/lucene/issues/12300


On Mon, May 15, 2023 at 3:17?PM Michael Sokolov <msokolov@gmail.com> wrote:

> random guess - does it have something to do with modules?
>
> On Mon, May 15, 2023 at 11:14?AM Gus Heck <gus.heck@gmail.com> wrote:
> >
> > I hadn't seen that one. Thanks, I'll look at it. It already looks a bit
> confusing though since it seems to have options for pointing to a repo, but
> I appear to be pulling the jars successfully from .m2/repository already...
> (except then they don't work, so successful means I see them in the
> classpath of the relevant classloader). And if we can't deploy a valid jar
> to mavenLocal for some reason (tweaked the solr build so it sees
> mavenLocal()), (or solr can't consume such a jar) that seems like an issue
> for whichever one is breaking that.
> >
> > Debugging: The JDK appears to be attempting to load the services file
> from modules, but not seeing the lucene module. (just the jdk ones) Also it
> passes through a block that says:
> >
> > // not in a package of a module defined to this loader
> > for (URL url : findMiscResource(name)) {
> >
> > (but then iterates
> jdk.internal.loader.BuiltinClassLoader#nameToModule.values() to load things
> anyway)
> >
> > -Gus
> >
> > On Mon, May 15, 2023 at 10:54?AM Houston Putman <houstonputman@gmail.com>
> wrote:
> >>
> >> Gus, I haven't done this myself, but are you using the instructions
> provided in Solr's "gradle/lucene-dev/lucene-dev-repo-composite.gradle"?
> >>
> >> It looks like you need to specify the development lucene version
> differently than other dependencies...
> >>
> >> - Houston
> >>
> >> On Sat, May 13, 2023 at 10:14?AM Michael Sokolov <msokolov@gmail.com>
> wrote:
> >>>
> >>> doh I actually read your email and you said you already checked that -
> >>> I'm going to send out one of those "sokolov would like to retract the
> >>> previous email" emails. Does GMail even pretend to do that? I don't
> >>> know what's going on there! sorry
> >>>
> >>> On Sat, May 13, 2023 at 10:13?AM Michael Sokolov <msokolov@gmail.com>
> wrote:
> >>> >
> >>> > sorry - META-INF not WEB-INF
> >>> >
> >>> > On Sat, May 13, 2023 at 10:12?AM Michael Sokolov <msokolov@gmail.com>
> wrote:
> >>> > >
> >>> > > You are probably missing the contents of WEB-INF in your custom
> jar?
> >>> > > Roughly speaking the files in there define run-time-bound
> "services"
> >>> > > that are looked up by name by the JDK's service-loader API.
> >>> > >
> >>> > > On Sat, May 13, 2023 at 9:33?AM Gus Heck <gus.heck@gmail.com>
> wrote:
> >>> > > >
> >>> > > > Cross posting to lucene on the possibility that folks here are
> more likely to add customized lucene to Solr and recognize what I'm
> stumbling on? (zero responses on solr list)
> >>> > > >
> >>> > > > Note that the specific test that I happened to copy is not the
> issue, all tests are doing this (or at least so many tests are failing I
> can't see the ones that are passing easily).
> >>> > > >
> >>> > > > ---------- Forwarded message ---------
> >>> > > > From: Gus Heck <gus.heck@gmail.com>
> >>> > > > Date: Wed, May 10, 2023 at 6:50?PM
> >>> > > > Subject: Running 10.0 build with a custom lucene 9.5
> >>> > > > To: <dev@solr.apache.org>
> >>> > > >
> >>> > > >
> >>> > > > Lucene:
> >>> > > >
> >>> > > > I made a tweak to lucene for something I'm investigating, gave
> it a new version, deployed to mavenLocal()
> >>> > > > I have verified that the jars are built with correct
> META-INF/services files
> >>> > > >
> >>> > > > Solr:
> >>> > > >
> >>> > > > I added mavenLocal() in gradle/globals.gradle
> >>> > > > I removed the license file sha1 sigs for the default lucene &
> creates signatures for my test version
> >>> > > > I updated versions.props
> >>> > > > I updated versions.lock
> >>> > > >
> >>> > > > Now when I run individual solr tests via my ide they seem to
> pass, but virtually every test run via gradle fails with something like:
> >>> > > >
> >>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod FAILED
> >>> > > > java.lang.ExceptionInInitializerError
> >>> > > > at
> org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> >>> > > > at
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
> >>> > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
> >>> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> >>> > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> >>> > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> >>> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> >>> > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> >>> > > > at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> >>> > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> >>> > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> >>> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> >>> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> >>> > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
> >>> > > >
> >>> > > > Caused by:
> >>> > > > java.lang.IllegalArgumentException: An SPI class of type
> org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist. You
> need to add the corresponding JAR file supporting this SPI to your
> classpath. The current classpath supports the following names: []
> >>> > > > at
> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
> >>> > > > at
> org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> >>> > > > ... 19 more
> >>> > > >
> >>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod FAILED
> >>> > > > java.lang.NullPointerException
> >>> > > > at
> java.base/java.util.Objects.requireNonNull(Objects.java:221)
> >>> > > > at
> org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
> >>> > > > at
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
> >>> > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
> >>> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> >>> > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> >>> > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> >>> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> >>> > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> >>> > > > at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> >>> > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> >>> > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> >>> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> >>> > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> >>> > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
> >>> > > >
> >>> > > > org.apache.solr.embedded.TestJettySolrRunner > test suite's
> output saved to
> /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt,
> copied below:
> >>> > > > > java.lang.ExceptionInInitializerError
> >>> > > > > at
> org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> >>> > > > > at
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
> >>> > > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > > at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > > at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> >>> > > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> >>> > > > > at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> >>> > > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> >>> > > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> >>> > > > > at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> >>> > > > > at java.base/java.lang.Thread.run(Thread.java:829)
> >>> > > > >
> >>> > > > > Caused by:
> >>> > > > > java.lang.IllegalArgumentException: An SPI class of
> type org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist.
> You need to add the corresponding JAR file supporting this SPI to your
> classpath. The current classpath supports the following names: []
> >>> > > > > at
> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
> >>> > > > > at
> org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> >>> > > > > ... 19 more
> >>> > > > > java.lang.NullPointerException
> >>> > > > > at
> java.base/java.util.Objects.requireNonNull(Objects.java:221)
> >>> > > > > at
> org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
> >>> > > > > at
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
> >>> > > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > > at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > > at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> >>> > > > > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> >>> > > > > at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> >>> > > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> >>> > > > > at
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> >>> > > > > at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> >>> > > > > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> >>> > > > > at java.base/java.lang.Thread.run(Thread.java:829)
> >>> > > > 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc. 11.0.19
> (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
> >>> > > > 2> NOTE: All tests run in this JVM: [TestJettySolrRunner]
> >>> > > > 2> NOTE: reproduce with: gradlew test --tests
> TestJettySolrRunner -Dtests.seed=2FB768B5A303764 -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> >>> > > >
> >>> > > > The only difference I've been able to determine is that the IDE
> does not run the test with a security manager, but that feels like a red
> herring since nothing is throwing security exception (or if it is, it's
> getting swallowed!).
> >>> > > >
> >>> > > > I have debugged and I can see that the classloaders have the new
> lucene jars I built on the classpath (in /Users/gus/.m2/repository as they
> should), but I haven't yet figuree out why the services are not loading.
> >>> > > >
> >>> > > > -Gus
> >>> > > >
> >>> > > >
> >>> > > > --
> >>> > > > http://www.needhamsoftware.com (work)
> >>> > > > http://www.the111shift.com (play)
> >>> > > >
> >>> > > >
> >>> > > > --
> >>> > > > http://www.needhamsoftware.com (work)
> >>> > > > http://www.the111shift.com (play)
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>> For additional commands, e-mail: dev-help@lucene.apache.org
> >>>
> >
> >
> > --
> > http://www.needhamsoftware.com (work)
> > http://www.the111shift.com (play)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: Running 10.0 build with a custom lucene 9.5 [ In reply to ]
> Having that explicitly called out would have been SUPER helpful.

Blaming Java in an exception thrown by Lucene is a ridiculous idea.

On Wed, 17 May, 2023, 3:33 am Gus Heck, <gus.heck@gmail.com> wrote:

> Found it.
>
> It's a solr thing made worse by the interaction of lucene testutils and
> jdk.internal.loader.URLClassPath's decision to hide anything gone wrong
> when checking a URL
> /*
> * Checks whether the resource URL should be returned.
> * Returns null on security check failure.
> * Called by java.net.URLClassLoader.
> */
> public static URL checkURL(URL url) {
> if (url != null) {
> try {
> check(url);
> } catch (Exception e) {
> return null;
> }
> }
> return url;
> }
>
> Yay. Fun. JDK classes swallowing exceptions silently.
>
> At the start of this it only took me a little while to discover that there
> was a security manager in play via debugging. Remembering that I saw emails
> about that, I went to jira, found the ticket enabling it by default in 9.x
> and eventually tracked down the name of the security policy file by reading
> solr.in.sh and /bin/solr... The key issue that tripped me up is that the
> tests have a *separate* security policy file, and there was pretty much no
> way to know this without extensive reading of the build. Thus I got thrown
> off track when
>
> permission java.io.FilePermission
> "${user.home}${/}.m2${/}repository${/}-", "read";
>
> To solr/server/etc/security.policy had no effect. That and the fact that
> no security exception was reported, led me to start chasing increasingly
> improbable hypotheses. Many hours later when I went back to debugging
> deeply into class loading, I found that the code was actually reading the
> jar files in question, and then I finally caught it throwing a security
> exception during my debugging.
>
> It turns out that adding the above permission to
> gradle/testing/randomization/policies/solr-tests.policy allows the test to
> pass. [1]
>
> I think we need to document this somewhere (or someone needs to point me to
> the doc I missed, FWIW I hit this basically following the process in
> dev-docs/dependency-upgrades.adoc treating lucene like a dependency, and
> unaware that there is a "shortcut" mode for lucene specifically in
> gradle/lucene-dev/lucene-dev-repo-composite.gradle and I find reading that
> file none-to clear anyway)
>
> That's the solr part, the lucene part is that the security exception is hit
> when in org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> when org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv#before
> does
>
> savedCodec = Codec.getDefault();
>
> The error message "An SPI class of type org.apache.lucene.codecs.Codec with
> name 'Lucene95' does not exist." was moderately misleading because the file
> and the services files in the jar definitely did exist. This message should
> vary if there is an installed security manager, maybe saying something
> like:
>
> "An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95'
> does not exist. We have detected that a security manager is installed so it
> is also possible that the jar containing the codec is inaccessible under
> the current security policy. (Java does not throw SecurityException if this
> is the case, it just ignores the jar!)" [2]
>
> Having that explicitly called out would have been SUPER helpful.
>
> -Gus
>
> [1]: https://issues.apache.org/jira/browse/SOLR-16804
> [2]: https://github.com/apache/lucene/issues/12300
>
>
> On Mon, May 15, 2023 at 3:17?PM Michael Sokolov <msokolov@gmail.com>
> wrote:
>
> > random guess - does it have something to do with modules?
> >
> > On Mon, May 15, 2023 at 11:14?AM Gus Heck <gus.heck@gmail.com> wrote:
> > >
> > > I hadn't seen that one. Thanks, I'll look at it. It already looks a bit
> > confusing though since it seems to have options for pointing to a repo,
> but
> > I appear to be pulling the jars successfully from .m2/repository
> already...
> > (except then they don't work, so successful means I see them in the
> > classpath of the relevant classloader). And if we can't deploy a valid
> jar
> > to mavenLocal for some reason (tweaked the solr build so it sees
> > mavenLocal()), (or solr can't consume such a jar) that seems like an
> issue
> > for whichever one is breaking that.
> > >
> > > Debugging: The JDK appears to be attempting to load the services file
> > from modules, but not seeing the lucene module. (just the jdk ones) Also
> it
> > passes through a block that says:
> > >
> > > // not in a package of a module defined to this loader
> > > for (URL url : findMiscResource(name)) {
> > >
> > > (but then iterates
> > jdk.internal.loader.BuiltinClassLoader#nameToModule.values() to load
> things
> > anyway)
> > >
> > > -Gus
> > >
> > > On Mon, May 15, 2023 at 10:54?AM Houston Putman <
> houstonputman@gmail.com>
> > wrote:
> > >>
> > >> Gus, I haven't done this myself, but are you using the instructions
> > provided in Solr's "gradle/lucene-dev/lucene-dev-repo-composite.gradle"?
> > >>
> > >> It looks like you need to specify the development lucene version
> > differently than other dependencies...
> > >>
> > >> - Houston
> > >>
> > >> On Sat, May 13, 2023 at 10:14?AM Michael Sokolov <msokolov@gmail.com>
> > wrote:
> > >>>
> > >>> doh I actually read your email and you said you already checked that
> -
> > >>> I'm going to send out one of those "sokolov would like to retract the
> > >>> previous email" emails. Does GMail even pretend to do that? I don't
> > >>> know what's going on there! sorry
> > >>>
> > >>> On Sat, May 13, 2023 at 10:13?AM Michael Sokolov <msokolov@gmail.com
> >
> > wrote:
> > >>> >
> > >>> > sorry - META-INF not WEB-INF
> > >>> >
> > >>> > On Sat, May 13, 2023 at 10:12?AM Michael Sokolov <
> msokolov@gmail.com>
> > wrote:
> > >>> > >
> > >>> > > You are probably missing the contents of WEB-INF in your custom
> > jar?
> > >>> > > Roughly speaking the files in there define run-time-bound
> > "services"
> > >>> > > that are looked up by name by the JDK's service-loader API.
> > >>> > >
> > >>> > > On Sat, May 13, 2023 at 9:33?AM Gus Heck <gus.heck@gmail.com>
> > wrote:
> > >>> > > >
> > >>> > > > Cross posting to lucene on the possibility that folks here are
> > more likely to add customized lucene to Solr and recognize what I'm
> > stumbling on? (zero responses on solr list)
> > >>> > > >
> > >>> > > > Note that the specific test that I happened to copy is not the
> > issue, all tests are doing this (or at least so many tests are failing I
> > can't see the ones that are passing easily).
> > >>> > > >
> > >>> > > > ---------- Forwarded message ---------
> > >>> > > > From: Gus Heck <gus.heck@gmail.com>
> > >>> > > > Date: Wed, May 10, 2023 at 6:50?PM
> > >>> > > > Subject: Running 10.0 build with a custom lucene 9.5
> > >>> > > > To: <dev@solr.apache.org>
> > >>> > > >
> > >>> > > >
> > >>> > > > Lucene:
> > >>> > > >
> > >>> > > > I made a tweak to lucene for something I'm investigating, gave
> > it a new version, deployed to mavenLocal()
> > >>> > > > I have verified that the jars are built with correct
> > META-INF/services files
> > >>> > > >
> > >>> > > > Solr:
> > >>> > > >
> > >>> > > > I added mavenLocal() in gradle/globals.gradle
> > >>> > > > I removed the license file sha1 sigs for the default lucene &
> > creates signatures for my test version
> > >>> > > > I updated versions.props
> > >>> > > > I updated versions.lock
> > >>> > > >
> > >>> > > > Now when I run individual solr tests via my ide they seem to
> > pass, but virtually every test run via gradle fails with something like:
> > >>> > > >
> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod
> FAILED
> > >>> > > > java.lang.ExceptionInInitializerError
> > >>> > > > at
> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > >>> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >>> > > >
> > >>> > > > Caused by:
> > >>> > > > java.lang.IllegalArgumentException: An SPI class of
> type
> > org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist. You
> > need to add the corresponding JAR file supporting this SPI to your
> > classpath. The current classpath supports the following names: []
> > >>> > > > at
> > org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
> > >>> > > > at
> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> > >>> > > > ... 19 more
> > >>> > > >
> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod
> FAILED
> > >>> > > > java.lang.NullPointerException
> > >>> > > > at
> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
> > >>> > > > at
> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > >>> > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > >>> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > >>> > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >>> > > >
> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > test suite's
> > output saved to
> >
> /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt,
> > copied below:
> > >>> > > > > java.lang.ExceptionInInitializerError
> > >>> > > > > at
> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > >>> > > > > at
> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > >>> > > > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >>> > > > >
> > >>> > > > > Caused by:
> > >>> > > > > java.lang.IllegalArgumentException: An SPI class
> of
> > type org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist.
> > You need to add the corresponding JAR file supporting this SPI to your
> > classpath. The current classpath supports the following names: []
> > >>> > > > > at
> > org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
> > >>> > > > > at
> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
> > >>> > > > > ... 19 more
> > >>> > > > > java.lang.NullPointerException
> > >>> > > > > at
> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
> > >>> > > > > at
> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > >>> > > > > at
> >
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > >>> > > > > at
> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > >>> > > > > at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > >>> > > > > at java.base/java.lang.Thread.run(Thread.java:829)
> > >>> > > > 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc. 11.0.19
> > (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
> > >>> > > > 2> NOTE: All tests run in this JVM: [TestJettySolrRunner]
> > >>> > > > 2> NOTE: reproduce with: gradlew test --tests
> > TestJettySolrRunner -Dtests.seed=2FB768B5A303764 -Dtests.asserts=true
> > -Dtests.file.encoding=UTF-8
> > >>> > > >
> > >>> > > > The only difference I've been able to determine is that the IDE
> > does not run the test with a security manager, but that feels like a red
> > herring since nothing is throwing security exception (or if it is, it's
> > getting swallowed!).
> > >>> > > >
> > >>> > > > I have debugged and I can see that the classloaders have the
> new
> > lucene jars I built on the classpath (in /Users/gus/.m2/repository as
> they
> > should), but I haven't yet figuree out why the services are not loading.
> > >>> > > >
> > >>> > > > -Gus
> > >>> > > >
> > >>> > > >
> > >>> > > > --
> > >>> > > > http://www.needhamsoftware.com (work)
> > >>> > > > http://www.the111shift.com (play)
> > >>> > > >
> > >>> > > >
> > >>> > > > --
> > >>> > > > http://www.needhamsoftware.com (work)
> > >>> > > > http://www.the111shift.com (play)
> > >>>
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > >>> For additional commands, e-mail: dev-help@lucene.apache.org
> > >>>
> > >
> > >
> > > --
> > > http://www.needhamsoftware.com (work)
> > > http://www.the111shift.com (play)
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
> >
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>
Re: Running 10.0 build with a custom lucene 9.5 [ In reply to ]
Blaming?

On Tue, May 16, 2023 at 10:05?PM Ishan Chattopadhyaya <
ichattopadhyaya@gmail.com> wrote:

> > Having that explicitly called out would have been SUPER helpful.
>
> Blaming Java in an exception thrown by Lucene is a ridiculous idea.
>
> On Wed, 17 May, 2023, 3:33 am Gus Heck, <gus.heck@gmail.com> wrote:
>
>> Found it.
>>
>> It's a solr thing made worse by the interaction of lucene testutils and
>> jdk.internal.loader.URLClassPath's decision to hide anything gone wrong
>> when checking a URL
>> /*
>> * Checks whether the resource URL should be returned.
>> * Returns null on security check failure.
>> * Called by java.net.URLClassLoader.
>> */
>> public static URL checkURL(URL url) {
>> if (url != null) {
>> try {
>> check(url);
>> } catch (Exception e) {
>> return null;
>> }
>> }
>> return url;
>> }
>>
>> Yay. Fun. JDK classes swallowing exceptions silently.
>>
>> At the start of this it only took me a little while to discover that there
>> was a security manager in play via debugging. Remembering that I saw
>> emails
>> about that, I went to jira, found the ticket enabling it by default in 9.x
>> and eventually tracked down the name of the security policy file by
>> reading
>> solr.in.sh and /bin/solr... The key issue that tripped me up is that the
>> tests have a *separate* security policy file, and there was pretty much no
>> way to know this without extensive reading of the build. Thus I got thrown
>> off track when
>>
>> permission java.io.FilePermission
>> "${user.home}${/}.m2${/}repository${/}-", "read";
>>
>> To solr/server/etc/security.policy had no effect. That and the fact that
>> no security exception was reported, led me to start chasing increasingly
>> improbable hypotheses. Many hours later when I went back to debugging
>> deeply into class loading, I found that the code was actually reading the
>> jar files in question, and then I finally caught it throwing a security
>> exception during my debugging.
>>
>> It turns out that adding the above permission to
>> gradle/testing/randomization/policies/solr-tests.policy allows the test to
>> pass. [1]
>>
>> I think we need to document this somewhere (or someone needs to point me
>> to
>> the doc I missed, FWIW I hit this basically following the process in
>> dev-docs/dependency-upgrades.adoc treating lucene like a dependency, and
>> unaware that there is a "shortcut" mode for lucene specifically in
>> gradle/lucene-dev/lucene-dev-repo-composite.gradle and I find reading that
>> file none-to clear anyway)
>>
>> That's the solr part, the lucene part is that the security exception is
>> hit
>> when in org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>> when org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv#before
>> does
>>
>> savedCodec = Codec.getDefault();
>>
>> The error message "An SPI class of type org.apache.lucene.codecs.Codec
>> with
>> name 'Lucene95' does not exist." was moderately misleading because the
>> file
>> and the services files in the jar definitely did exist. This message
>> should
>> vary if there is an installed security manager, maybe saying something
>> like:
>>
>> "An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95'
>> does not exist. We have detected that a security manager is installed so
>> it
>> is also possible that the jar containing the codec is inaccessible under
>> the current security policy. (Java does not throw SecurityException if
>> this
>> is the case, it just ignores the jar!)" [2]
>>
>> Having that explicitly called out would have been SUPER helpful.
>>
>> -Gus
>>
>> [1]: https://issues.apache.org/jira/browse/SOLR-16804
>> [2]: https://github.com/apache/lucene/issues/12300
>>
>>
>> On Mon, May 15, 2023 at 3:17?PM Michael Sokolov <msokolov@gmail.com>
>> wrote:
>>
>> > random guess - does it have something to do with modules?
>> >
>> > On Mon, May 15, 2023 at 11:14?AM Gus Heck <gus.heck@gmail.com> wrote:
>> > >
>> > > I hadn't seen that one. Thanks, I'll look at it. It already looks a
>> bit
>> > confusing though since it seems to have options for pointing to a repo,
>> but
>> > I appear to be pulling the jars successfully from .m2/repository
>> already...
>> > (except then they don't work, so successful means I see them in the
>> > classpath of the relevant classloader). And if we can't deploy a valid
>> jar
>> > to mavenLocal for some reason (tweaked the solr build so it sees
>> > mavenLocal()), (or solr can't consume such a jar) that seems like an
>> issue
>> > for whichever one is breaking that.
>> > >
>> > > Debugging: The JDK appears to be attempting to load the services file
>> > from modules, but not seeing the lucene module. (just the jdk ones)
>> Also it
>> > passes through a block that says:
>> > >
>> > > // not in a package of a module defined to this loader
>> > > for (URL url : findMiscResource(name)) {
>> > >
>> > > (but then iterates
>> > jdk.internal.loader.BuiltinClassLoader#nameToModule.values() to load
>> things
>> > anyway)
>> > >
>> > > -Gus
>> > >
>> > > On Mon, May 15, 2023 at 10:54?AM Houston Putman <
>> houstonputman@gmail.com>
>> > wrote:
>> > >>
>> > >> Gus, I haven't done this myself, but are you using the instructions
>> > provided in Solr's "gradle/lucene-dev/lucene-dev-repo-composite.gradle"?
>> > >>
>> > >> It looks like you need to specify the development lucene version
>> > differently than other dependencies...
>> > >>
>> > >> - Houston
>> > >>
>> > >> On Sat, May 13, 2023 at 10:14?AM Michael Sokolov <msokolov@gmail.com
>> >
>> > wrote:
>> > >>>
>> > >>> doh I actually read your email and you said you already checked
>> that -
>> > >>> I'm going to send out one of those "sokolov would like to retract
>> the
>> > >>> previous email" emails. Does GMail even pretend to do that? I don't
>> > >>> know what's going on there! sorry
>> > >>>
>> > >>> On Sat, May 13, 2023 at 10:13?AM Michael Sokolov <
>> msokolov@gmail.com>
>> > wrote:
>> > >>> >
>> > >>> > sorry - META-INF not WEB-INF
>> > >>> >
>> > >>> > On Sat, May 13, 2023 at 10:12?AM Michael Sokolov <
>> msokolov@gmail.com>
>> > wrote:
>> > >>> > >
>> > >>> > > You are probably missing the contents of WEB-INF in your custom
>> > jar?
>> > >>> > > Roughly speaking the files in there define run-time-bound
>> > "services"
>> > >>> > > that are looked up by name by the JDK's service-loader API.
>> > >>> > >
>> > >>> > > On Sat, May 13, 2023 at 9:33?AM Gus Heck <gus.heck@gmail.com>
>> > wrote:
>> > >>> > > >
>> > >>> > > > Cross posting to lucene on the possibility that folks here are
>> > more likely to add customized lucene to Solr and recognize what I'm
>> > stumbling on? (zero responses on solr list)
>> > >>> > > >
>> > >>> > > > Note that the specific test that I happened to copy is not the
>> > issue, all tests are doing this (or at least so many tests are failing I
>> > can't see the ones that are passing easily).
>> > >>> > > >
>> > >>> > > > ---------- Forwarded message ---------
>> > >>> > > > From: Gus Heck <gus.heck@gmail.com>
>> > >>> > > > Date: Wed, May 10, 2023 at 6:50?PM
>> > >>> > > > Subject: Running 10.0 build with a custom lucene 9.5
>> > >>> > > > To: <dev@solr.apache.org>
>> > >>> > > >
>> > >>> > > >
>> > >>> > > > Lucene:
>> > >>> > > >
>> > >>> > > > I made a tweak to lucene for something I'm investigating, gave
>> > it a new version, deployed to mavenLocal()
>> > >>> > > > I have verified that the jars are built with correct
>> > META-INF/services files
>> > >>> > > >
>> > >>> > > > Solr:
>> > >>> > > >
>> > >>> > > > I added mavenLocal() in gradle/globals.gradle
>> > >>> > > > I removed the license file sha1 sigs for the default lucene &
>> > creates signatures for my test version
>> > >>> > > > I updated versions.props
>> > >>> > > > I updated versions.lock
>> > >>> > > >
>> > >>> > > > Now when I run individual solr tests via my ide they seem to
>> > pass, but virtually every test run via gradle fails with something like:
>> > >>> > > >
>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod
>> FAILED
>> > >>> > > > java.lang.ExceptionInInitializerError
>> > >>> > > > at
>> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>> > >>> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>> > >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
>> > >>> > > >
>> > >>> > > > Caused by:
>> > >>> > > > java.lang.IllegalArgumentException: An SPI class of
>> type
>> > org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist. You
>> > need to add the corresponding JAR file supporting this SPI to your
>> > classpath. The current classpath supports the following names: []
>> > >>> > > > at
>> > org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>> > >>> > > > at
>> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>> > >>> > > > ... 19 more
>> > >>> > > >
>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod
>> FAILED
>> > >>> > > > java.lang.NullPointerException
>> > >>> > > > at
>> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
>> > >>> > > > at
>> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>> > >>> > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>> > >>> > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>> > >>> > > > at
>> >
>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>> > >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
>> > >>> > > >
>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > test suite's
>> > output saved to
>> >
>> /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt,
>> > copied below:
>> > >>> > > > > java.lang.ExceptionInInitializerError
>> > >>> > > > > at
>> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>> > >>> > > > > at
>> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>> > >>> > > > > at
>> java.base/java.lang.Thread.run(Thread.java:829)
>> > >>> > > > >
>> > >>> > > > > Caused by:
>> > >>> > > > > java.lang.IllegalArgumentException: An SPI class
>> of
>> > type org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist.
>> > You need to add the corresponding JAR file supporting this SPI to your
>> > classpath. The current classpath supports the following names: []
>> > >>> > > > > at
>> > org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>> > >>> > > > > at
>> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>> > >>> > > > > ... 19 more
>> > >>> > > > > java.lang.NullPointerException
>> > >>> > > > > at
>> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
>> > >>> > > > > at
>> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>> > >>> > > > > at
>> >
>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>> > >>> > > > > at
>> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>> > >>> > > > > at
>> >
>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>> > >>> > > > > at
>> java.base/java.lang.Thread.run(Thread.java:829)
>> > >>> > > > 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc. 11.0.19
>> > (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
>> > >>> > > > 2> NOTE: All tests run in this JVM: [TestJettySolrRunner]
>> > >>> > > > 2> NOTE: reproduce with: gradlew test --tests
>> > TestJettySolrRunner -Dtests.seed=2FB768B5A303764 -Dtests.asserts=true
>> > -Dtests.file.encoding=UTF-8
>> > >>> > > >
>> > >>> > > > The only difference I've been able to determine is that the
>> IDE
>> > does not run the test with a security manager, but that feels like a red
>> > herring since nothing is throwing security exception (or if it is, it's
>> > getting swallowed!).
>> > >>> > > >
>> > >>> > > > I have debugged and I can see that the classloaders have the
>> new
>> > lucene jars I built on the classpath (in /Users/gus/.m2/repository as
>> they
>> > should), but I haven't yet figuree out why the services are not loading.
>> > >>> > > >
>> > >>> > > > -Gus
>> > >>> > > >
>> > >>> > > >
>> > >>> > > > --
>> > >>> > > > http://www.needhamsoftware.com (work)
>> > >>> > > > http://www.the111shift.com (play)
>> > >>> > > >
>> > >>> > > >
>> > >>> > > > --
>> > >>> > > > http://www.needhamsoftware.com (work)
>> > >>> > > > http://www.the111shift.com (play)
>> > >>>
>> > >>>
>> ---------------------------------------------------------------------
>> > >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > >>> For additional commands, e-mail: dev-help@lucene.apache.org
>> > >>>
>> > >
>> > >
>> > > --
>> > > http://www.needhamsoftware.com (work)
>> > > http://www.the111shift.com (play)
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: dev-help@lucene.apache.org
>> >
>> >
>>
>> --
>> http://www.needhamsoftware.com (work)
>> http://www.the111shift.com (play)
>>
>

--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: Running 10.0 build with a custom lucene 9.5 [ In reply to ]
You propose to throw an exception containing this, right?

> Java does not throw SecurityException if this
is the case, it just ignores the jar!

Are you serious?

On Wed, 17 May, 2023, 8:02 am Gus Heck, <gus.heck@gmail.com> wrote:

> Blaming?
>
> On Tue, May 16, 2023 at 10:05?PM Ishan Chattopadhyaya <
> ichattopadhyaya@gmail.com> wrote:
>
>> > Having that explicitly called out would have been SUPER helpful.
>>
>> Blaming Java in an exception thrown by Lucene is a ridiculous idea.
>>
>> On Wed, 17 May, 2023, 3:33 am Gus Heck, <gus.heck@gmail.com> wrote:
>>
>>> Found it.
>>>
>>> It's a solr thing made worse by the interaction of lucene testutils and
>>> jdk.internal.loader.URLClassPath's decision to hide anything gone wrong
>>> when checking a URL
>>> /*
>>> * Checks whether the resource URL should be returned.
>>> * Returns null on security check failure.
>>> * Called by java.net.URLClassLoader.
>>> */
>>> public static URL checkURL(URL url) {
>>> if (url != null) {
>>> try {
>>> check(url);
>>> } catch (Exception e) {
>>> return null;
>>> }
>>> }
>>> return url;
>>> }
>>>
>>> Yay. Fun. JDK classes swallowing exceptions silently.
>>>
>>> At the start of this it only took me a little while to discover that
>>> there
>>> was a security manager in play via debugging. Remembering that I saw
>>> emails
>>> about that, I went to jira, found the ticket enabling it by default in
>>> 9.x
>>> and eventually tracked down the name of the security policy file by
>>> reading
>>> solr.in.sh and /bin/solr... The key issue that tripped me up is that
>>> the
>>> tests have a *separate* security policy file, and there was pretty much
>>> no
>>> way to know this without extensive reading of the build. Thus I got
>>> thrown
>>> off track when
>>>
>>> permission java.io.FilePermission
>>> "${user.home}${/}.m2${/}repository${/}-", "read";
>>>
>>> To solr/server/etc/security.policy had no effect. That and the fact that
>>> no security exception was reported, led me to start chasing increasingly
>>> improbable hypotheses. Many hours later when I went back to debugging
>>> deeply into class loading, I found that the code was actually reading the
>>> jar files in question, and then I finally caught it throwing a security
>>> exception during my debugging.
>>>
>>> It turns out that adding the above permission to
>>> gradle/testing/randomization/policies/solr-tests.policy allows the test
>>> to
>>> pass. [1]
>>>
>>> I think we need to document this somewhere (or someone needs to point me
>>> to
>>> the doc I missed, FWIW I hit this basically following the process in
>>> dev-docs/dependency-upgrades.adoc treating lucene like a dependency, and
>>> unaware that there is a "shortcut" mode for lucene specifically in
>>> gradle/lucene-dev/lucene-dev-repo-composite.gradle and I find reading
>>> that
>>> file none-to clear anyway)
>>>
>>> That's the solr part, the lucene part is that the security exception is
>>> hit
>>> when in org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>> when org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv#before
>>> does
>>>
>>> savedCodec = Codec.getDefault();
>>>
>>> The error message "An SPI class of type org.apache.lucene.codecs.Codec
>>> with
>>> name 'Lucene95' does not exist." was moderately misleading because the
>>> file
>>> and the services files in the jar definitely did exist. This message
>>> should
>>> vary if there is an installed security manager, maybe saying something
>>> like:
>>>
>>> "An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95'
>>> does not exist. We have detected that a security manager is installed so
>>> it
>>> is also possible that the jar containing the codec is inaccessible under
>>> the current security policy. (Java does not throw SecurityException if
>>> this
>>> is the case, it just ignores the jar!)" [2]
>>>
>>> Having that explicitly called out would have been SUPER helpful.
>>>
>>> -Gus
>>>
>>> [1]: https://issues.apache.org/jira/browse/SOLR-16804
>>> [2]: https://github.com/apache/lucene/issues/12300
>>>
>>>
>>> On Mon, May 15, 2023 at 3:17?PM Michael Sokolov <msokolov@gmail.com>
>>> wrote:
>>>
>>> > random guess - does it have something to do with modules?
>>> >
>>> > On Mon, May 15, 2023 at 11:14?AM Gus Heck <gus.heck@gmail.com> wrote:
>>> > >
>>> > > I hadn't seen that one. Thanks, I'll look at it. It already looks a
>>> bit
>>> > confusing though since it seems to have options for pointing to a
>>> repo, but
>>> > I appear to be pulling the jars successfully from .m2/repository
>>> already...
>>> > (except then they don't work, so successful means I see them in the
>>> > classpath of the relevant classloader). And if we can't deploy a valid
>>> jar
>>> > to mavenLocal for some reason (tweaked the solr build so it sees
>>> > mavenLocal()), (or solr can't consume such a jar) that seems like an
>>> issue
>>> > for whichever one is breaking that.
>>> > >
>>> > > Debugging: The JDK appears to be attempting to load the services file
>>> > from modules, but not seeing the lucene module. (just the jdk ones)
>>> Also it
>>> > passes through a block that says:
>>> > >
>>> > > // not in a package of a module defined to this loader
>>> > > for (URL url : findMiscResource(name)) {
>>> > >
>>> > > (but then iterates
>>> > jdk.internal.loader.BuiltinClassLoader#nameToModule.values() to load
>>> things
>>> > anyway)
>>> > >
>>> > > -Gus
>>> > >
>>> > > On Mon, May 15, 2023 at 10:54?AM Houston Putman <
>>> houstonputman@gmail.com>
>>> > wrote:
>>> > >>
>>> > >> Gus, I haven't done this myself, but are you using the instructions
>>> > provided in Solr's
>>> "gradle/lucene-dev/lucene-dev-repo-composite.gradle"?
>>> > >>
>>> > >> It looks like you need to specify the development lucene version
>>> > differently than other dependencies...
>>> > >>
>>> > >> - Houston
>>> > >>
>>> > >> On Sat, May 13, 2023 at 10:14?AM Michael Sokolov <
>>> msokolov@gmail.com>
>>> > wrote:
>>> > >>>
>>> > >>> doh I actually read your email and you said you already checked
>>> that -
>>> > >>> I'm going to send out one of those "sokolov would like to retract
>>> the
>>> > >>> previous email" emails. Does GMail even pretend to do that? I don't
>>> > >>> know what's going on there! sorry
>>> > >>>
>>> > >>> On Sat, May 13, 2023 at 10:13?AM Michael Sokolov <
>>> msokolov@gmail.com>
>>> > wrote:
>>> > >>> >
>>> > >>> > sorry - META-INF not WEB-INF
>>> > >>> >
>>> > >>> > On Sat, May 13, 2023 at 10:12?AM Michael Sokolov <
>>> msokolov@gmail.com>
>>> > wrote:
>>> > >>> > >
>>> > >>> > > You are probably missing the contents of WEB-INF in your custom
>>> > jar?
>>> > >>> > > Roughly speaking the files in there define run-time-bound
>>> > "services"
>>> > >>> > > that are looked up by name by the JDK's service-loader API.
>>> > >>> > >
>>> > >>> > > On Sat, May 13, 2023 at 9:33?AM Gus Heck <gus.heck@gmail.com>
>>> > wrote:
>>> > >>> > > >
>>> > >>> > > > Cross posting to lucene on the possibility that folks here
>>> are
>>> > more likely to add customized lucene to Solr and recognize what I'm
>>> > stumbling on? (zero responses on solr list)
>>> > >>> > > >
>>> > >>> > > > Note that the specific test that I happened to copy is not
>>> the
>>> > issue, all tests are doing this (or at least so many tests are failing
>>> I
>>> > can't see the ones that are passing easily).
>>> > >>> > > >
>>> > >>> > > > ---------- Forwarded message ---------
>>> > >>> > > > From: Gus Heck <gus.heck@gmail.com>
>>> > >>> > > > Date: Wed, May 10, 2023 at 6:50?PM
>>> > >>> > > > Subject: Running 10.0 build with a custom lucene 9.5
>>> > >>> > > > To: <dev@solr.apache.org>
>>> > >>> > > >
>>> > >>> > > >
>>> > >>> > > > Lucene:
>>> > >>> > > >
>>> > >>> > > > I made a tweak to lucene for something I'm investigating,
>>> gave
>>> > it a new version, deployed to mavenLocal()
>>> > >>> > > > I have verified that the jars are built with correct
>>> > META-INF/services files
>>> > >>> > > >
>>> > >>> > > > Solr:
>>> > >>> > > >
>>> > >>> > > > I added mavenLocal() in gradle/globals.gradle
>>> > >>> > > > I removed the license file sha1 sigs for the default lucene &
>>> > creates signatures for my test version
>>> > >>> > > > I updated versions.props
>>> > >>> > > > I updated versions.lock
>>> > >>> > > >
>>> > >>> > > > Now when I run individual solr tests via my ide they seem to
>>> > pass, but virtually every test run via gradle fails with something
>>> like:
>>> > >>> > > >
>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod
>>> FAILED
>>> > >>> > > > java.lang.ExceptionInInitializerError
>>> > >>> > > > at
>>> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>> > >>> > > > at
>>> org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>> > >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
>>> > >>> > > >
>>> > >>> > > > Caused by:
>>> > >>> > > > java.lang.IllegalArgumentException: An SPI class of
>>> type
>>> > org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist.
>>> You
>>> > need to add the corresponding JAR file supporting this SPI to your
>>> > classpath. The current classpath supports the following names: []
>>> > >>> > > > at
>>> > org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>>> > >>> > > > at
>>> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>> > >>> > > > ... 19 more
>>> > >>> > > >
>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod
>>> FAILED
>>> > >>> > > > java.lang.NullPointerException
>>> > >>> > > > at
>>> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
>>> > >>> > > > at
>>> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>> > >>> > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>> > >>> > > > at
>>> org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>> > >>> > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>> > >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
>>> > >>> > > >
>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > test suite's
>>> > output saved to
>>> >
>>> /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt,
>>> > copied below:
>>> > >>> > > > > java.lang.ExceptionInInitializerError
>>> > >>> > > > > at
>>> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>> > >>> > > > > at
>>> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>> > >>> > > > > at
>>> java.base/java.lang.Thread.run(Thread.java:829)
>>> > >>> > > > >
>>> > >>> > > > > Caused by:
>>> > >>> > > > > java.lang.IllegalArgumentException: An SPI
>>> class of
>>> > type org.apache.lucene.codecs.Codec with name 'Lucene95' does not
>>> exist.
>>> > You need to add the corresponding JAR file supporting this SPI to your
>>> > classpath. The current classpath supports the following names: []
>>> > >>> > > > > at
>>> > org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>>> > >>> > > > > at
>>> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>> > >>> > > > > ... 19 more
>>> > >>> > > > > java.lang.NullPointerException
>>> > >>> > > > > at
>>> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
>>> > >>> > > > > at
>>> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>> > >>> > > > > at
>>> >
>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>> > >>> > > > > at
>>> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>> > >>> > > > > at
>>> >
>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>> > >>> > > > > at
>>> java.base/java.lang.Thread.run(Thread.java:829)
>>> > >>> > > > 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc. 11.0.19
>>> > (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
>>> > >>> > > > 2> NOTE: All tests run in this JVM: [TestJettySolrRunner]
>>> > >>> > > > 2> NOTE: reproduce with: gradlew test --tests
>>> > TestJettySolrRunner -Dtests.seed=2FB768B5A303764 -Dtests.asserts=true
>>> > -Dtests.file.encoding=UTF-8
>>> > >>> > > >
>>> > >>> > > > The only difference I've been able to determine is that the
>>> IDE
>>> > does not run the test with a security manager, but that feels like a
>>> red
>>> > herring since nothing is throwing security exception (or if it is, it's
>>> > getting swallowed!).
>>> > >>> > > >
>>> > >>> > > > I have debugged and I can see that the classloaders have the
>>> new
>>> > lucene jars I built on the classpath (in /Users/gus/.m2/repository as
>>> they
>>> > should), but I haven't yet figuree out why the services are not
>>> loading.
>>> > >>> > > >
>>> > >>> > > > -Gus
>>> > >>> > > >
>>> > >>> > > >
>>> > >>> > > > --
>>> > >>> > > > http://www.needhamsoftware.com (work)
>>> > >>> > > > http://www.the111shift.com (play)
>>> > >>> > > >
>>> > >>> > > >
>>> > >>> > > > --
>>> > >>> > > > http://www.needhamsoftware.com (work)
>>> > >>> > > > http://www.the111shift.com (play)
>>> > >>>
>>> > >>>
>>> ---------------------------------------------------------------------
>>> > >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> > >>> For additional commands, e-mail: dev-help@lucene.apache.org
>>> > >>>
>>> > >
>>> > >
>>> > > --
>>> > > http://www.needhamsoftware.com (work)
>>> > > http://www.the111shift.com (play)
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>> >
>>> >
>>>
>>> --
>>> http://www.needhamsoftware.com (work)
>>> http://www.the111shift.com (play)
>>>
>>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>
Re: Running 10.0 build with a custom lucene 9.5 [ In reply to ]
I propose to improve the message on an exception already thrown.

On Tue, May 16, 2023 at 11:04?PM Ishan Chattopadhyaya <
ichattopadhyaya@gmail.com> wrote:

> You propose to throw an exception containing this, right?
>
> > Java does not throw SecurityException if this
> is the case, it just ignores the jar!
>
> Are you serious?
>
> On Wed, 17 May, 2023, 8:02 am Gus Heck, <gus.heck@gmail.com> wrote:
>
>> Blaming?
>>
>> On Tue, May 16, 2023 at 10:05?PM Ishan Chattopadhyaya <
>> ichattopadhyaya@gmail.com> wrote:
>>
>>> > Having that explicitly called out would have been SUPER helpful.
>>>
>>> Blaming Java in an exception thrown by Lucene is a ridiculous idea.
>>>
>>> On Wed, 17 May, 2023, 3:33 am Gus Heck, <gus.heck@gmail.com> wrote:
>>>
>>>> Found it.
>>>>
>>>> It's a solr thing made worse by the interaction of lucene testutils and
>>>> jdk.internal.loader.URLClassPath's decision to hide anything gone wrong
>>>> when checking a URL
>>>> /*
>>>> * Checks whether the resource URL should be returned.
>>>> * Returns null on security check failure.
>>>> * Called by java.net.URLClassLoader.
>>>> */
>>>> public static URL checkURL(URL url) {
>>>> if (url != null) {
>>>> try {
>>>> check(url);
>>>> } catch (Exception e) {
>>>> return null;
>>>> }
>>>> }
>>>> return url;
>>>> }
>>>>
>>>> Yay. Fun. JDK classes swallowing exceptions silently.
>>>>
>>>> At the start of this it only took me a little while to discover that
>>>> there
>>>> was a security manager in play via debugging. Remembering that I saw
>>>> emails
>>>> about that, I went to jira, found the ticket enabling it by default in
>>>> 9.x
>>>> and eventually tracked down the name of the security policy file by
>>>> reading
>>>> solr.in.sh and /bin/solr... The key issue that tripped me up is that
>>>> the
>>>> tests have a *separate* security policy file, and there was pretty much
>>>> no
>>>> way to know this without extensive reading of the build. Thus I got
>>>> thrown
>>>> off track when
>>>>
>>>> permission java.io.FilePermission
>>>> "${user.home}${/}.m2${/}repository${/}-", "read";
>>>>
>>>> To solr/server/etc/security.policy had no effect. That and the fact
>>>> that
>>>> no security exception was reported, led me to start chasing increasingly
>>>> improbable hypotheses. Many hours later when I went back to debugging
>>>> deeply into class loading, I found that the code was actually reading
>>>> the
>>>> jar files in question, and then I finally caught it throwing a security
>>>> exception during my debugging.
>>>>
>>>> It turns out that adding the above permission to
>>>> gradle/testing/randomization/policies/solr-tests.policy allows the test
>>>> to
>>>> pass. [1]
>>>>
>>>> I think we need to document this somewhere (or someone needs to point
>>>> me to
>>>> the doc I missed, FWIW I hit this basically following the process in
>>>> dev-docs/dependency-upgrades.adoc treating lucene like a dependency, and
>>>> unaware that there is a "shortcut" mode for lucene specifically in
>>>> gradle/lucene-dev/lucene-dev-repo-composite.gradle and I find reading
>>>> that
>>>> file none-to clear anyway)
>>>>
>>>> That's the solr part, the lucene part is that the security exception is
>>>> hit
>>>> when in org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>>> when org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv#before
>>>> does
>>>>
>>>> savedCodec = Codec.getDefault();
>>>>
>>>> The error message "An SPI class of type org.apache.lucene.codecs.Codec
>>>> with
>>>> name 'Lucene95' does not exist." was moderately misleading because the
>>>> file
>>>> and the services files in the jar definitely did exist. This message
>>>> should
>>>> vary if there is an installed security manager, maybe saying something
>>>> like:
>>>>
>>>> "An SPI class of type org.apache.lucene.codecs.Codec with name
>>>> 'Lucene95'
>>>> does not exist. We have detected that a security manager is installed
>>>> so it
>>>> is also possible that the jar containing the codec is inaccessible under
>>>> the current security policy. (Java does not throw SecurityException if
>>>> this
>>>> is the case, it just ignores the jar!)" [2]
>>>>
>>>> Having that explicitly called out would have been SUPER helpful.
>>>>
>>>> -Gus
>>>>
>>>> [1]: https://issues.apache.org/jira/browse/SOLR-16804
>>>> [2]: https://github.com/apache/lucene/issues/12300
>>>>
>>>>
>>>> On Mon, May 15, 2023 at 3:17?PM Michael Sokolov <msokolov@gmail.com>
>>>> wrote:
>>>>
>>>> > random guess - does it have something to do with modules?
>>>> >
>>>> > On Mon, May 15, 2023 at 11:14?AM Gus Heck <gus.heck@gmail.com> wrote:
>>>> > >
>>>> > > I hadn't seen that one. Thanks, I'll look at it. It already looks a
>>>> bit
>>>> > confusing though since it seems to have options for pointing to a
>>>> repo, but
>>>> > I appear to be pulling the jars successfully from .m2/repository
>>>> already...
>>>> > (except then they don't work, so successful means I see them in the
>>>> > classpath of the relevant classloader). And if we can't deploy a
>>>> valid jar
>>>> > to mavenLocal for some reason (tweaked the solr build so it sees
>>>> > mavenLocal()), (or solr can't consume such a jar) that seems like an
>>>> issue
>>>> > for whichever one is breaking that.
>>>> > >
>>>> > > Debugging: The JDK appears to be attempting to load the services
>>>> file
>>>> > from modules, but not seeing the lucene module. (just the jdk ones)
>>>> Also it
>>>> > passes through a block that says:
>>>> > >
>>>> > > // not in a package of a module defined to this loader
>>>> > > for (URL url : findMiscResource(name)) {
>>>> > >
>>>> > > (but then iterates
>>>> > jdk.internal.loader.BuiltinClassLoader#nameToModule.values() to load
>>>> things
>>>> > anyway)
>>>> > >
>>>> > > -Gus
>>>> > >
>>>> > > On Mon, May 15, 2023 at 10:54?AM Houston Putman <
>>>> houstonputman@gmail.com>
>>>> > wrote:
>>>> > >>
>>>> > >> Gus, I haven't done this myself, but are you using the instructions
>>>> > provided in Solr's
>>>> "gradle/lucene-dev/lucene-dev-repo-composite.gradle"?
>>>> > >>
>>>> > >> It looks like you need to specify the development lucene version
>>>> > differently than other dependencies...
>>>> > >>
>>>> > >> - Houston
>>>> > >>
>>>> > >> On Sat, May 13, 2023 at 10:14?AM Michael Sokolov <
>>>> msokolov@gmail.com>
>>>> > wrote:
>>>> > >>>
>>>> > >>> doh I actually read your email and you said you already checked
>>>> that -
>>>> > >>> I'm going to send out one of those "sokolov would like to retract
>>>> the
>>>> > >>> previous email" emails. Does GMail even pretend to do that? I
>>>> don't
>>>> > >>> know what's going on there! sorry
>>>> > >>>
>>>> > >>> On Sat, May 13, 2023 at 10:13?AM Michael Sokolov <
>>>> msokolov@gmail.com>
>>>> > wrote:
>>>> > >>> >
>>>> > >>> > sorry - META-INF not WEB-INF
>>>> > >>> >
>>>> > >>> > On Sat, May 13, 2023 at 10:12?AM Michael Sokolov <
>>>> msokolov@gmail.com>
>>>> > wrote:
>>>> > >>> > >
>>>> > >>> > > You are probably missing the contents of WEB-INF in your
>>>> custom
>>>> > jar?
>>>> > >>> > > Roughly speaking the files in there define run-time-bound
>>>> > "services"
>>>> > >>> > > that are looked up by name by the JDK's service-loader API.
>>>> > >>> > >
>>>> > >>> > > On Sat, May 13, 2023 at 9:33?AM Gus Heck <gus.heck@gmail.com>
>>>> > wrote:
>>>> > >>> > > >
>>>> > >>> > > > Cross posting to lucene on the possibility that folks here
>>>> are
>>>> > more likely to add customized lucene to Solr and recognize what I'm
>>>> > stumbling on? (zero responses on solr list)
>>>> > >>> > > >
>>>> > >>> > > > Note that the specific test that I happened to copy is not
>>>> the
>>>> > issue, all tests are doing this (or at least so many tests are
>>>> failing I
>>>> > can't see the ones that are passing easily).
>>>> > >>> > > >
>>>> > >>> > > > ---------- Forwarded message ---------
>>>> > >>> > > > From: Gus Heck <gus.heck@gmail.com>
>>>> > >>> > > > Date: Wed, May 10, 2023 at 6:50?PM
>>>> > >>> > > > Subject: Running 10.0 build with a custom lucene 9.5
>>>> > >>> > > > To: <dev@solr.apache.org>
>>>> > >>> > > >
>>>> > >>> > > >
>>>> > >>> > > > Lucene:
>>>> > >>> > > >
>>>> > >>> > > > I made a tweak to lucene for something I'm investigating,
>>>> gave
>>>> > it a new version, deployed to mavenLocal()
>>>> > >>> > > > I have verified that the jars are built with correct
>>>> > META-INF/services files
>>>> > >>> > > >
>>>> > >>> > > > Solr:
>>>> > >>> > > >
>>>> > >>> > > > I added mavenLocal() in gradle/globals.gradle
>>>> > >>> > > > I removed the license file sha1 sigs for the default lucene
>>>> &
>>>> > creates signatures for my test version
>>>> > >>> > > > I updated versions.props
>>>> > >>> > > > I updated versions.lock
>>>> > >>> > > >
>>>> > >>> > > > Now when I run individual solr tests via my ide they seem to
>>>> > pass, but virtually every test run via gradle fails with something
>>>> like:
>>>> > >>> > > >
>>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod
>>>> FAILED
>>>> > >>> > > > java.lang.ExceptionInInitializerError
>>>> > >>> > > > at
>>>> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>> > >>> > > > at
>>>> org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>> > >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
>>>> > >>> > > >
>>>> > >>> > > > Caused by:
>>>> > >>> > > > java.lang.IllegalArgumentException: An SPI class of
>>>> type
>>>> > org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist.
>>>> You
>>>> > need to add the corresponding JAR file supporting this SPI to your
>>>> > classpath. The current classpath supports the following names: []
>>>> > >>> > > > at
>>>> > org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>>>> > >>> > > > at
>>>> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>>> > >>> > > > ... 19 more
>>>> > >>> > > >
>>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod
>>>> FAILED
>>>> > >>> > > > java.lang.NullPointerException
>>>> > >>> > > > at
>>>> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
>>>> > >>> > > > at
>>>> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>> > >>> > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>> > >>> > > > at
>>>> org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>> > >>> > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>> > >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
>>>> > >>> > > >
>>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > test suite's
>>>> > output saved to
>>>> >
>>>> /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt,
>>>> > copied below:
>>>> > >>> > > > > java.lang.ExceptionInInitializerError
>>>> > >>> > > > > at
>>>> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>> > >>> > > > > at
>>>> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>> > >>> > > > > at
>>>> java.base/java.lang.Thread.run(Thread.java:829)
>>>> > >>> > > > >
>>>> > >>> > > > > Caused by:
>>>> > >>> > > > > java.lang.IllegalArgumentException: An SPI
>>>> class of
>>>> > type org.apache.lucene.codecs.Codec with name 'Lucene95' does not
>>>> exist.
>>>> > You need to add the corresponding JAR file supporting this SPI to your
>>>> > classpath. The current classpath supports the following names: []
>>>> > >>> > > > > at
>>>> > org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>>>> > >>> > > > > at
>>>> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>>> > >>> > > > > ... 19 more
>>>> > >>> > > > > java.lang.NullPointerException
>>>> > >>> > > > > at
>>>> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
>>>> > >>> > > > > at
>>>> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>> > >>> > > > > at
>>>> >
>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>> > >>> > > > > at
>>>> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>> > >>> > > > > at
>>>> >
>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>> > >>> > > > > at
>>>> java.base/java.lang.Thread.run(Thread.java:829)
>>>> > >>> > > > 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc. 11.0.19
>>>> > (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
>>>> > >>> > > > 2> NOTE: All tests run in this JVM: [TestJettySolrRunner]
>>>> > >>> > > > 2> NOTE: reproduce with: gradlew test --tests
>>>> > TestJettySolrRunner -Dtests.seed=2FB768B5A303764 -Dtests.asserts=true
>>>> > -Dtests.file.encoding=UTF-8
>>>> > >>> > > >
>>>> > >>> > > > The only difference I've been able to determine is that the
>>>> IDE
>>>> > does not run the test with a security manager, but that feels like a
>>>> red
>>>> > herring since nothing is throwing security exception (or if it is,
>>>> it's
>>>> > getting swallowed!).
>>>> > >>> > > >
>>>> > >>> > > > I have debugged and I can see that the classloaders have
>>>> the new
>>>> > lucene jars I built on the classpath (in /Users/gus/.m2/repository as
>>>> they
>>>> > should), but I haven't yet figuree out why the services are not
>>>> loading.
>>>> > >>> > > >
>>>> > >>> > > > -Gus
>>>> > >>> > > >
>>>> > >>> > > >
>>>> > >>> > > > --
>>>> > >>> > > > http://www.needhamsoftware.com (work)
>>>> > >>> > > > http://www.the111shift.com (play)
>>>> > >>> > > >
>>>> > >>> > > >
>>>> > >>> > > > --
>>>> > >>> > > > http://www.needhamsoftware.com (work)
>>>> > >>> > > > http://www.the111shift.com (play)
>>>> > >>>
>>>> > >>>
>>>> ---------------------------------------------------------------------
>>>> > >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> > >>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>> > >>>
>>>> > >
>>>> > >
>>>> > > --
>>>> > > http://www.needhamsoftware.com (work)
>>>> > > http://www.the111shift.com (play)
>>>> >
>>>> > ---------------------------------------------------------------------
>>>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>>> >
>>>> >
>>>>
>>>> --
>>>> http://www.needhamsoftware.com (work)
>>>> http://www.the111shift.com (play)
>>>>
>>>
>>
>> --
>> http://www.needhamsoftware.com (work)
>> http://www.the111shift.com (play)
>>
>

--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: Running 10.0 build with a custom lucene 9.5 [ In reply to ]
Oh hmm the google UI hid the quoted bit. If you don't like message let's
improve it. (actually, it should probably say the "file in the jar"... or
something a little more specific... not the jar entirely. The class loads,
but the service loader cant access the file in the same jar without the
FilePermission to (re?) access the jar it seems)

On Tue, May 16, 2023 at 11:05?PM Gus Heck <gus.heck@gmail.com> wrote:

> I propose to improve the message on an exception already thrown.
>
> On Tue, May 16, 2023 at 11:04?PM Ishan Chattopadhyaya <
> ichattopadhyaya@gmail.com> wrote:
>
>> You propose to throw an exception containing this, right?
>>
>> > Java does not throw SecurityException if this
>> is the case, it just ignores the jar!
>>
>> Are you serious?
>>
>> On Wed, 17 May, 2023, 8:02 am Gus Heck, <gus.heck@gmail.com> wrote:
>>
>>> Blaming?
>>>
>>> On Tue, May 16, 2023 at 10:05?PM Ishan Chattopadhyaya <
>>> ichattopadhyaya@gmail.com> wrote:
>>>
>>>> > Having that explicitly called out would have been SUPER helpful.
>>>>
>>>> Blaming Java in an exception thrown by Lucene is a ridiculous idea.
>>>>
>>>> On Wed, 17 May, 2023, 3:33 am Gus Heck, <gus.heck@gmail.com> wrote:
>>>>
>>>>> Found it.
>>>>>
>>>>> It's a solr thing made worse by the interaction of lucene testutils and
>>>>> jdk.internal.loader.URLClassPath's decision to hide anything gone wrong
>>>>> when checking a URL
>>>>> /*
>>>>> * Checks whether the resource URL should be returned.
>>>>> * Returns null on security check failure.
>>>>> * Called by java.net.URLClassLoader.
>>>>> */
>>>>> public static URL checkURL(URL url) {
>>>>> if (url != null) {
>>>>> try {
>>>>> check(url);
>>>>> } catch (Exception e) {
>>>>> return null;
>>>>> }
>>>>> }
>>>>> return url;
>>>>> }
>>>>>
>>>>> Yay. Fun. JDK classes swallowing exceptions silently.
>>>>>
>>>>> At the start of this it only took me a little while to discover that
>>>>> there
>>>>> was a security manager in play via debugging. Remembering that I saw
>>>>> emails
>>>>> about that, I went to jira, found the ticket enabling it by default in
>>>>> 9.x
>>>>> and eventually tracked down the name of the security policy file by
>>>>> reading
>>>>> solr.in.sh and /bin/solr... The key issue that tripped me up is that
>>>>> the
>>>>> tests have a *separate* security policy file, and there was pretty
>>>>> much no
>>>>> way to know this without extensive reading of the build. Thus I got
>>>>> thrown
>>>>> off track when
>>>>>
>>>>> permission java.io.FilePermission
>>>>> "${user.home}${/}.m2${/}repository${/}-", "read";
>>>>>
>>>>> To solr/server/etc/security.policy had no effect. That and the fact
>>>>> that
>>>>> no security exception was reported, led me to start chasing
>>>>> increasingly
>>>>> improbable hypotheses. Many hours later when I went back to debugging
>>>>> deeply into class loading, I found that the code was actually reading
>>>>> the
>>>>> jar files in question, and then I finally caught it throwing a security
>>>>> exception during my debugging.
>>>>>
>>>>> It turns out that adding the above permission to
>>>>> gradle/testing/randomization/policies/solr-tests.policy allows the
>>>>> test to
>>>>> pass. [1]
>>>>>
>>>>> I think we need to document this somewhere (or someone needs to point
>>>>> me to
>>>>> the doc I missed, FWIW I hit this basically following the process in
>>>>> dev-docs/dependency-upgrades.adoc treating lucene like a dependency,
>>>>> and
>>>>> unaware that there is a "shortcut" mode for lucene specifically in
>>>>> gradle/lucene-dev/lucene-dev-repo-composite.gradle and I find reading
>>>>> that
>>>>> file none-to clear anyway)
>>>>>
>>>>> That's the solr part, the lucene part is that the security exception
>>>>> is hit
>>>>> when in org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>>>> when
>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv#before
>>>>> does
>>>>>
>>>>> savedCodec = Codec.getDefault();
>>>>>
>>>>> The error message "An SPI class of type org.apache.lucene.codecs.Codec
>>>>> with
>>>>> name 'Lucene95' does not exist." was moderately misleading because the
>>>>> file
>>>>> and the services files in the jar definitely did exist. This message
>>>>> should
>>>>> vary if there is an installed security manager, maybe saying something
>>>>> like:
>>>>>
>>>>> "An SPI class of type org.apache.lucene.codecs.Codec with name
>>>>> 'Lucene95'
>>>>> does not exist. We have detected that a security manager is installed
>>>>> so it
>>>>> is also possible that the jar containing the codec is inaccessible
>>>>> under
>>>>> the current security policy. (Java does not throw SecurityException if
>>>>> this
>>>>> is the case, it just ignores the jar!)" [2]
>>>>>
>>>>> Having that explicitly called out would have been SUPER helpful.
>>>>>
>>>>> -Gus
>>>>>
>>>>> [1]: https://issues.apache.org/jira/browse/SOLR-16804
>>>>> [2]: https://github.com/apache/lucene/issues/12300
>>>>>
>>>>>
>>>>> On Mon, May 15, 2023 at 3:17?PM Michael Sokolov <msokolov@gmail.com>
>>>>> wrote:
>>>>>
>>>>> > random guess - does it have something to do with modules?
>>>>> >
>>>>> > On Mon, May 15, 2023 at 11:14?AM Gus Heck <gus.heck@gmail.com>
>>>>> wrote:
>>>>> > >
>>>>> > > I hadn't seen that one. Thanks, I'll look at it. It already looks
>>>>> a bit
>>>>> > confusing though since it seems to have options for pointing to a
>>>>> repo, but
>>>>> > I appear to be pulling the jars successfully from .m2/repository
>>>>> already...
>>>>> > (except then they don't work, so successful means I see them in the
>>>>> > classpath of the relevant classloader). And if we can't deploy a
>>>>> valid jar
>>>>> > to mavenLocal for some reason (tweaked the solr build so it sees
>>>>> > mavenLocal()), (or solr can't consume such a jar) that seems like an
>>>>> issue
>>>>> > for whichever one is breaking that.
>>>>> > >
>>>>> > > Debugging: The JDK appears to be attempting to load the services
>>>>> file
>>>>> > from modules, but not seeing the lucene module. (just the jdk ones)
>>>>> Also it
>>>>> > passes through a block that says:
>>>>> > >
>>>>> > > // not in a package of a module defined to this loader
>>>>> > > for (URL url : findMiscResource(name)) {
>>>>> > >
>>>>> > > (but then iterates
>>>>> > jdk.internal.loader.BuiltinClassLoader#nameToModule.values() to load
>>>>> things
>>>>> > anyway)
>>>>> > >
>>>>> > > -Gus
>>>>> > >
>>>>> > > On Mon, May 15, 2023 at 10:54?AM Houston Putman <
>>>>> houstonputman@gmail.com>
>>>>> > wrote:
>>>>> > >>
>>>>> > >> Gus, I haven't done this myself, but are you using the
>>>>> instructions
>>>>> > provided in Solr's
>>>>> "gradle/lucene-dev/lucene-dev-repo-composite.gradle"?
>>>>> > >>
>>>>> > >> It looks like you need to specify the development lucene version
>>>>> > differently than other dependencies...
>>>>> > >>
>>>>> > >> - Houston
>>>>> > >>
>>>>> > >> On Sat, May 13, 2023 at 10:14?AM Michael Sokolov <
>>>>> msokolov@gmail.com>
>>>>> > wrote:
>>>>> > >>>
>>>>> > >>> doh I actually read your email and you said you already checked
>>>>> that -
>>>>> > >>> I'm going to send out one of those "sokolov would like to
>>>>> retract the
>>>>> > >>> previous email" emails. Does GMail even pretend to do that? I
>>>>> don't
>>>>> > >>> know what's going on there! sorry
>>>>> > >>>
>>>>> > >>> On Sat, May 13, 2023 at 10:13?AM Michael Sokolov <
>>>>> msokolov@gmail.com>
>>>>> > wrote:
>>>>> > >>> >
>>>>> > >>> > sorry - META-INF not WEB-INF
>>>>> > >>> >
>>>>> > >>> > On Sat, May 13, 2023 at 10:12?AM Michael Sokolov <
>>>>> msokolov@gmail.com>
>>>>> > wrote:
>>>>> > >>> > >
>>>>> > >>> > > You are probably missing the contents of WEB-INF in your
>>>>> custom
>>>>> > jar?
>>>>> > >>> > > Roughly speaking the files in there define run-time-bound
>>>>> > "services"
>>>>> > >>> > > that are looked up by name by the JDK's service-loader API.
>>>>> > >>> > >
>>>>> > >>> > > On Sat, May 13, 2023 at 9:33?AM Gus Heck <gus.heck@gmail.com
>>>>> >
>>>>> > wrote:
>>>>> > >>> > > >
>>>>> > >>> > > > Cross posting to lucene on the possibility that folks here
>>>>> are
>>>>> > more likely to add customized lucene to Solr and recognize what I'm
>>>>> > stumbling on? (zero responses on solr list)
>>>>> > >>> > > >
>>>>> > >>> > > > Note that the specific test that I happened to copy is not
>>>>> the
>>>>> > issue, all tests are doing this (or at least so many tests are
>>>>> failing I
>>>>> > can't see the ones that are passing easily).
>>>>> > >>> > > >
>>>>> > >>> > > > ---------- Forwarded message ---------
>>>>> > >>> > > > From: Gus Heck <gus.heck@gmail.com>
>>>>> > >>> > > > Date: Wed, May 10, 2023 at 6:50?PM
>>>>> > >>> > > > Subject: Running 10.0 build with a custom lucene 9.5
>>>>> > >>> > > > To: <dev@solr.apache.org>
>>>>> > >>> > > >
>>>>> > >>> > > >
>>>>> > >>> > > > Lucene:
>>>>> > >>> > > >
>>>>> > >>> > > > I made a tweak to lucene for something I'm investigating,
>>>>> gave
>>>>> > it a new version, deployed to mavenLocal()
>>>>> > >>> > > > I have verified that the jars are built with correct
>>>>> > META-INF/services files
>>>>> > >>> > > >
>>>>> > >>> > > > Solr:
>>>>> > >>> > > >
>>>>> > >>> > > > I added mavenLocal() in gradle/globals.gradle
>>>>> > >>> > > > I removed the license file sha1 sigs for the default
>>>>> lucene &
>>>>> > creates signatures for my test version
>>>>> > >>> > > > I updated versions.props
>>>>> > >>> > > > I updated versions.lock
>>>>> > >>> > > >
>>>>> > >>> > > > Now when I run individual solr tests via my ide they seem
>>>>> to
>>>>> > pass, but virtually every test run via gradle fails with something
>>>>> like:
>>>>> > >>> > > >
>>>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod
>>>>> FAILED
>>>>> > >>> > > > java.lang.ExceptionInInitializerError
>>>>> > >>> > > > at
>>>>> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>>> > >>> > > > at
>>>>> org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>>> > >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
>>>>> > >>> > > >
>>>>> > >>> > > > Caused by:
>>>>> > >>> > > > java.lang.IllegalArgumentException: An SPI class
>>>>> of type
>>>>> > org.apache.lucene.codecs.Codec with name 'Lucene95' does not exist.
>>>>> You
>>>>> > need to add the corresponding JAR file supporting this SPI to your
>>>>> > classpath. The current classpath supports the following names: []
>>>>> > >>> > > > at
>>>>> > org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>>>>> > >>> > > > at
>>>>> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>>>> > >>> > > > ... 19 more
>>>>> > >>> > > >
>>>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > classMethod
>>>>> FAILED
>>>>> > >>> > > > java.lang.NullPointerException
>>>>> > >>> > > > at
>>>>> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
>>>>> > >>> > > > at
>>>>> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>>> > >>> > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>>> > >>> > > > at
>>>>> org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>>> > >>> > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>>> > >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
>>>>> > >>> > > >
>>>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > test suite's
>>>>> > output saved to
>>>>> >
>>>>> /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt,
>>>>> > copied below:
>>>>> > >>> > > > > java.lang.ExceptionInInitializerError
>>>>> > >>> > > > > at
>>>>> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>>> > >>> > > > > at
>>>>> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>>> > >>> > > > > at
>>>>> java.base/java.lang.Thread.run(Thread.java:829)
>>>>> > >>> > > > >
>>>>> > >>> > > > > Caused by:
>>>>> > >>> > > > > java.lang.IllegalArgumentException: An SPI
>>>>> class of
>>>>> > type org.apache.lucene.codecs.Codec with name 'Lucene95' does not
>>>>> exist.
>>>>> > You need to add the corresponding JAR file supporting this SPI to
>>>>> your
>>>>> > classpath. The current classpath supports the following names: []
>>>>> > >>> > > > > at
>>>>> > org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>>>>> > >>> > > > > at
>>>>> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>>>> > >>> > > > > ... 19 more
>>>>> > >>> > > > > java.lang.NullPointerException
>>>>> > >>> > > > > at
>>>>> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
>>>>> > >>> > > > > at
>>>>> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>>> > >>> > > > > at
>>>>> >
>>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>>> > >>> > > > > at
>>>>> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>>> > >>> > > > > at
>>>>> >
>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>>> > >>> > > > > at
>>>>> java.base/java.lang.Thread.run(Thread.java:829)
>>>>> > >>> > > > 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc. 11.0.19
>>>>> > (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
>>>>> > >>> > > > 2> NOTE: All tests run in this JVM: [TestJettySolrRunner]
>>>>> > >>> > > > 2> NOTE: reproduce with: gradlew test --tests
>>>>> > TestJettySolrRunner -Dtests.seed=2FB768B5A303764 -Dtests.asserts=true
>>>>> > -Dtests.file.encoding=UTF-8
>>>>> > >>> > > >
>>>>> > >>> > > > The only difference I've been able to determine is that
>>>>> the IDE
>>>>> > does not run the test with a security manager, but that feels like a
>>>>> red
>>>>> > herring since nothing is throwing security exception (or if it is,
>>>>> it's
>>>>> > getting swallowed!).
>>>>> > >>> > > >
>>>>> > >>> > > > I have debugged and I can see that the classloaders have
>>>>> the new
>>>>> > lucene jars I built on the classpath (in /Users/gus/.m2/repository
>>>>> as they
>>>>> > should), but I haven't yet figuree out why the services are not
>>>>> loading.
>>>>> > >>> > > >
>>>>> > >>> > > > -Gus
>>>>> > >>> > > >
>>>>> > >>> > > >
>>>>> > >>> > > > --
>>>>> > >>> > > > http://www.needhamsoftware.com (work)
>>>>> > >>> > > > http://www.the111shift.com (play)
>>>>> > >>> > > >
>>>>> > >>> > > >
>>>>> > >>> > > > --
>>>>> > >>> > > > http://www.needhamsoftware.com (work)
>>>>> > >>> > > > http://www.the111shift.com (play)
>>>>> > >>>
>>>>> > >>>
>>>>> ---------------------------------------------------------------------
>>>>> > >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> > >>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>> > >>>
>>>>> > >
>>>>> > >
>>>>> > > --
>>>>> > > http://www.needhamsoftware.com (work)
>>>>> > > http://www.the111shift.com (play)
>>>>> >
>>>>> > ---------------------------------------------------------------------
>>>>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>>>> >
>>>>> >
>>>>>
>>>>> --
>>>>> http://www.needhamsoftware.com (work)
>>>>> http://www.the111shift.com (play)
>>>>>
>>>>
>>>
>>> --
>>> http://www.needhamsoftware.com (work)
>>> http://www.the111shift.com (play)
>>>
>>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>


--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: Running 10.0 build with a custom lucene 9.5 [ In reply to ]
Ok reading my last message I realize it still might not be clear. Here's
what I observed:

The class Codec clearly loaded, (from the lucene-core jar) when
Codec$Holder tried to load the class initializer code went looking for the
service definitions. It failed to find any of the the
META-INF/services/org.apache.lucene.codec.Codec files in any of the jars
residing in ~/.m2/repository (where the class evidently loads from),
including the one in **the same jar that Codec had loaded from** and I
observed it throwing Security exception by debugging at a level underneath
the quoted method in my longer message above.

So what it seems to imply is that the ability of java to load a class from
a jar on the class path is not related to the FilePermission needed to load
the services file.

So what I want to communicate to the user is 2 things:

1. They are running a security manager, so the policy file is relevant.
2. The fact they are not seeing a SecurityException does not eliminate the
possibility that they are missing permissions.

We should craft a message that is clearer and communicates those two points.

Adding a permission to the policy for the tests fixed everything (once I
found the right policy file).

-Gus

On Tue, May 16, 2023 at 11:10?PM Gus Heck <gus.heck@gmail.com> wrote:

> Oh hmm the google UI hid the quoted bit. If you don't like message let's
> improve it. (actually, it should probably say the "file in the jar"... or
> something a little more specific... not the jar entirely. The class loads,
> but the service loader cant access the file in the same jar without the
> FilePermission to (re?) access the jar it seems)
>
> On Tue, May 16, 2023 at 11:05?PM Gus Heck <gus.heck@gmail.com> wrote:
>
>> I propose to improve the message on an exception already thrown.
>>
>> On Tue, May 16, 2023 at 11:04?PM Ishan Chattopadhyaya <
>> ichattopadhyaya@gmail.com> wrote:
>>
>>> You propose to throw an exception containing this, right?
>>>
>>> > Java does not throw SecurityException if this
>>> is the case, it just ignores the jar!
>>>
>>> Are you serious?
>>>
>>> On Wed, 17 May, 2023, 8:02 am Gus Heck, <gus.heck@gmail.com> wrote:
>>>
>>>> Blaming?
>>>>
>>>> On Tue, May 16, 2023 at 10:05?PM Ishan Chattopadhyaya <
>>>> ichattopadhyaya@gmail.com> wrote:
>>>>
>>>>> > Having that explicitly called out would have been SUPER helpful.
>>>>>
>>>>> Blaming Java in an exception thrown by Lucene is a ridiculous idea.
>>>>>
>>>>> On Wed, 17 May, 2023, 3:33 am Gus Heck, <gus.heck@gmail.com> wrote:
>>>>>
>>>>>> Found it.
>>>>>>
>>>>>> It's a solr thing made worse by the interaction of lucene testutils
>>>>>> and
>>>>>> jdk.internal.loader.URLClassPath's decision to hide anything gone
>>>>>> wrong
>>>>>> when checking a URL
>>>>>> /*
>>>>>> * Checks whether the resource URL should be returned.
>>>>>> * Returns null on security check failure.
>>>>>> * Called by java.net.URLClassLoader.
>>>>>> */
>>>>>> public static URL checkURL(URL url) {
>>>>>> if (url != null) {
>>>>>> try {
>>>>>> check(url);
>>>>>> } catch (Exception e) {
>>>>>> return null;
>>>>>> }
>>>>>> }
>>>>>> return url;
>>>>>> }
>>>>>>
>>>>>> Yay. Fun. JDK classes swallowing exceptions silently.
>>>>>>
>>>>>> At the start of this it only took me a little while to discover that
>>>>>> there
>>>>>> was a security manager in play via debugging. Remembering that I saw
>>>>>> emails
>>>>>> about that, I went to jira, found the ticket enabling it by default
>>>>>> in 9.x
>>>>>> and eventually tracked down the name of the security policy file by
>>>>>> reading
>>>>>> solr.in.sh and /bin/solr... The key issue that tripped me up is
>>>>>> that the
>>>>>> tests have a *separate* security policy file, and there was pretty
>>>>>> much no
>>>>>> way to know this without extensive reading of the build. Thus I got
>>>>>> thrown
>>>>>> off track when
>>>>>>
>>>>>> permission java.io.FilePermission
>>>>>> "${user.home}${/}.m2${/}repository${/}-", "read";
>>>>>>
>>>>>> To solr/server/etc/security.policy had no effect. That and the fact
>>>>>> that
>>>>>> no security exception was reported, led me to start chasing
>>>>>> increasingly
>>>>>> improbable hypotheses. Many hours later when I went back to debugging
>>>>>> deeply into class loading, I found that the code was actually reading
>>>>>> the
>>>>>> jar files in question, and then I finally caught it throwing a
>>>>>> security
>>>>>> exception during my debugging.
>>>>>>
>>>>>> It turns out that adding the above permission to
>>>>>> gradle/testing/randomization/policies/solr-tests.policy allows the
>>>>>> test to
>>>>>> pass. [1]
>>>>>>
>>>>>> I think we need to document this somewhere (or someone needs to point
>>>>>> me to
>>>>>> the doc I missed, FWIW I hit this basically following the process in
>>>>>> dev-docs/dependency-upgrades.adoc treating lucene like a dependency,
>>>>>> and
>>>>>> unaware that there is a "shortcut" mode for lucene specifically in
>>>>>> gradle/lucene-dev/lucene-dev-repo-composite.gradle and I find reading
>>>>>> that
>>>>>> file none-to clear anyway)
>>>>>>
>>>>>> That's the solr part, the lucene part is that the security exception
>>>>>> is hit
>>>>>> when in org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>>>>> when
>>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv#before
>>>>>> does
>>>>>>
>>>>>> savedCodec = Codec.getDefault();
>>>>>>
>>>>>> The error message "An SPI class of type
>>>>>> org.apache.lucene.codecs.Codec with
>>>>>> name 'Lucene95' does not exist." was moderately misleading because
>>>>>> the file
>>>>>> and the services files in the jar definitely did exist. This message
>>>>>> should
>>>>>> vary if there is an installed security manager, maybe saying
>>>>>> something like:
>>>>>>
>>>>>> "An SPI class of type org.apache.lucene.codecs.Codec with name
>>>>>> 'Lucene95'
>>>>>> does not exist. We have detected that a security manager is installed
>>>>>> so it
>>>>>> is also possible that the jar containing the codec is inaccessible
>>>>>> under
>>>>>> the current security policy. (Java does not throw SecurityException
>>>>>> if this
>>>>>> is the case, it just ignores the jar!)" [2]
>>>>>>
>>>>>> Having that explicitly called out would have been SUPER helpful.
>>>>>>
>>>>>> -Gus
>>>>>>
>>>>>> [1]: https://issues.apache.org/jira/browse/SOLR-16804
>>>>>> [2]: https://github.com/apache/lucene/issues/12300
>>>>>>
>>>>>>
>>>>>> On Mon, May 15, 2023 at 3:17?PM Michael Sokolov <msokolov@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> > random guess - does it have something to do with modules?
>>>>>> >
>>>>>> > On Mon, May 15, 2023 at 11:14?AM Gus Heck <gus.heck@gmail.com>
>>>>>> wrote:
>>>>>> > >
>>>>>> > > I hadn't seen that one. Thanks, I'll look at it. It already looks
>>>>>> a bit
>>>>>> > confusing though since it seems to have options for pointing to a
>>>>>> repo, but
>>>>>> > I appear to be pulling the jars successfully from .m2/repository
>>>>>> already...
>>>>>> > (except then they don't work, so successful means I see them in the
>>>>>> > classpath of the relevant classloader). And if we can't deploy a
>>>>>> valid jar
>>>>>> > to mavenLocal for some reason (tweaked the solr build so it sees
>>>>>> > mavenLocal()), (or solr can't consume such a jar) that seems like
>>>>>> an issue
>>>>>> > for whichever one is breaking that.
>>>>>> > >
>>>>>> > > Debugging: The JDK appears to be attempting to load the services
>>>>>> file
>>>>>> > from modules, but not seeing the lucene module. (just the jdk ones)
>>>>>> Also it
>>>>>> > passes through a block that says:
>>>>>> > >
>>>>>> > > // not in a package of a module defined to this loader
>>>>>> > > for (URL url : findMiscResource(name)) {
>>>>>> > >
>>>>>> > > (but then iterates
>>>>>> > jdk.internal.loader.BuiltinClassLoader#nameToModule.values() to
>>>>>> load things
>>>>>> > anyway)
>>>>>> > >
>>>>>> > > -Gus
>>>>>> > >
>>>>>> > > On Mon, May 15, 2023 at 10:54?AM Houston Putman <
>>>>>> houstonputman@gmail.com>
>>>>>> > wrote:
>>>>>> > >>
>>>>>> > >> Gus, I haven't done this myself, but are you using the
>>>>>> instructions
>>>>>> > provided in Solr's
>>>>>> "gradle/lucene-dev/lucene-dev-repo-composite.gradle"?
>>>>>> > >>
>>>>>> > >> It looks like you need to specify the development lucene version
>>>>>> > differently than other dependencies...
>>>>>> > >>
>>>>>> > >> - Houston
>>>>>> > >>
>>>>>> > >> On Sat, May 13, 2023 at 10:14?AM Michael Sokolov <
>>>>>> msokolov@gmail.com>
>>>>>> > wrote:
>>>>>> > >>>
>>>>>> > >>> doh I actually read your email and you said you already checked
>>>>>> that -
>>>>>> > >>> I'm going to send out one of those "sokolov would like to
>>>>>> retract the
>>>>>> > >>> previous email" emails. Does GMail even pretend to do that? I
>>>>>> don't
>>>>>> > >>> know what's going on there! sorry
>>>>>> > >>>
>>>>>> > >>> On Sat, May 13, 2023 at 10:13?AM Michael Sokolov <
>>>>>> msokolov@gmail.com>
>>>>>> > wrote:
>>>>>> > >>> >
>>>>>> > >>> > sorry - META-INF not WEB-INF
>>>>>> > >>> >
>>>>>> > >>> > On Sat, May 13, 2023 at 10:12?AM Michael Sokolov <
>>>>>> msokolov@gmail.com>
>>>>>> > wrote:
>>>>>> > >>> > >
>>>>>> > >>> > > You are probably missing the contents of WEB-INF in your
>>>>>> custom
>>>>>> > jar?
>>>>>> > >>> > > Roughly speaking the files in there define run-time-bound
>>>>>> > "services"
>>>>>> > >>> > > that are looked up by name by the JDK's service-loader API.
>>>>>> > >>> > >
>>>>>> > >>> > > On Sat, May 13, 2023 at 9:33?AM Gus Heck <
>>>>>> gus.heck@gmail.com>
>>>>>> > wrote:
>>>>>> > >>> > > >
>>>>>> > >>> > > > Cross posting to lucene on the possibility that folks
>>>>>> here are
>>>>>> > more likely to add customized lucene to Solr and recognize what I'm
>>>>>> > stumbling on? (zero responses on solr list)
>>>>>> > >>> > > >
>>>>>> > >>> > > > Note that the specific test that I happened to copy is
>>>>>> not the
>>>>>> > issue, all tests are doing this (or at least so many tests are
>>>>>> failing I
>>>>>> > can't see the ones that are passing easily).
>>>>>> > >>> > > >
>>>>>> > >>> > > > ---------- Forwarded message ---------
>>>>>> > >>> > > > From: Gus Heck <gus.heck@gmail.com>
>>>>>> > >>> > > > Date: Wed, May 10, 2023 at 6:50?PM
>>>>>> > >>> > > > Subject: Running 10.0 build with a custom lucene 9.5
>>>>>> > >>> > > > To: <dev@solr.apache.org>
>>>>>> > >>> > > >
>>>>>> > >>> > > >
>>>>>> > >>> > > > Lucene:
>>>>>> > >>> > > >
>>>>>> > >>> > > > I made a tweak to lucene for something I'm investigating,
>>>>>> gave
>>>>>> > it a new version, deployed to mavenLocal()
>>>>>> > >>> > > > I have verified that the jars are built with correct
>>>>>> > META-INF/services files
>>>>>> > >>> > > >
>>>>>> > >>> > > > Solr:
>>>>>> > >>> > > >
>>>>>> > >>> > > > I added mavenLocal() in gradle/globals.gradle
>>>>>> > >>> > > > I removed the license file sha1 sigs for the default
>>>>>> lucene &
>>>>>> > creates signatures for my test version
>>>>>> > >>> > > > I updated versions.props
>>>>>> > >>> > > > I updated versions.lock
>>>>>> > >>> > > >
>>>>>> > >>> > > > Now when I run individual solr tests via my ide they seem
>>>>>> to
>>>>>> > pass, but virtually every test run via gradle fails with something
>>>>>> like:
>>>>>> > >>> > > >
>>>>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner >
>>>>>> classMethod FAILED
>>>>>> > >>> > > > java.lang.ExceptionInInitializerError
>>>>>> > >>> > > > at
>>>>>> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>>>> > >>> > > > at
>>>>>> org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>>>> > >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
>>>>>> > >>> > > >
>>>>>> > >>> > > > Caused by:
>>>>>> > >>> > > > java.lang.IllegalArgumentException: An SPI class
>>>>>> of type
>>>>>> > org.apache.lucene.codecs.Codec with name 'Lucene95' does not
>>>>>> exist. You
>>>>>> > need to add the corresponding JAR file supporting this SPI to your
>>>>>> > classpath. The current classpath supports the following names: []
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>>>>>> > >>> > > > at
>>>>>> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>>>>> > >>> > > > ... 19 more
>>>>>> > >>> > > >
>>>>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner >
>>>>>> classMethod FAILED
>>>>>> > >>> > > > java.lang.NullPointerException
>>>>>> > >>> > > > at
>>>>>> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
>>>>>> > >>> > > > at
>>>>>> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>>>> > >>> > > > at
>>>>>> org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>>>> > >>> > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>>>> > >>> > > > at java.base/java.lang.Thread.run(Thread.java:829)
>>>>>> > >>> > > >
>>>>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > test
>>>>>> suite's
>>>>>> > output saved to
>>>>>> >
>>>>>> /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt,
>>>>>> > copied below:
>>>>>> > >>> > > > > java.lang.ExceptionInInitializerError
>>>>>> > >>> > > > > at
>>>>>> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>>>> > >>> > > > > at
>>>>>> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>>>> > >>> > > > > at
>>>>>> java.base/java.lang.Thread.run(Thread.java:829)
>>>>>> > >>> > > > >
>>>>>> > >>> > > > > Caused by:
>>>>>> > >>> > > > > java.lang.IllegalArgumentException: An SPI
>>>>>> class of
>>>>>> > type org.apache.lucene.codecs.Codec with name 'Lucene95' does not
>>>>>> exist.
>>>>>> > You need to add the corresponding JAR file supporting this SPI to
>>>>>> your
>>>>>> > classpath. The current classpath supports the following names: []
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>>>>>> > >>> > > > > at
>>>>>> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>>>>> > >>> > > > > ... 19 more
>>>>>> > >>> > > > > java.lang.NullPointerException
>>>>>> > >>> > > > > at
>>>>>> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
>>>>>> > >>> > > > > at
>>>>>> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>>>> > >>> > > > > at
>>>>>> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>>>> > >>> > > > > at
>>>>>> >
>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>>>> > >>> > > > > at
>>>>>> java.base/java.lang.Thread.run(Thread.java:829)
>>>>>> > >>> > > > 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc.
>>>>>> 11.0.19
>>>>>> > (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
>>>>>> > >>> > > > 2> NOTE: All tests run in this JVM:
>>>>>> [TestJettySolrRunner]
>>>>>> > >>> > > > 2> NOTE: reproduce with: gradlew test --tests
>>>>>> > TestJettySolrRunner -Dtests.seed=2FB768B5A303764
>>>>>> -Dtests.asserts=true
>>>>>> > -Dtests.file.encoding=UTF-8
>>>>>> > >>> > > >
>>>>>> > >>> > > > The only difference I've been able to determine is that
>>>>>> the IDE
>>>>>> > does not run the test with a security manager, but that feels like
>>>>>> a red
>>>>>> > herring since nothing is throwing security exception (or if it is,
>>>>>> it's
>>>>>> > getting swallowed!).
>>>>>> > >>> > > >
>>>>>> > >>> > > > I have debugged and I can see that the classloaders have
>>>>>> the new
>>>>>> > lucene jars I built on the classpath (in /Users/gus/.m2/repository
>>>>>> as they
>>>>>> > should), but I haven't yet figuree out why the services are not
>>>>>> loading.
>>>>>> > >>> > > >
>>>>>> > >>> > > > -Gus
>>>>>> > >>> > > >
>>>>>> > >>> > > >
>>>>>> > >>> > > > --
>>>>>> > >>> > > > http://www.needhamsoftware.com (work)
>>>>>> > >>> > > > http://www.the111shift.com (play)
>>>>>> > >>> > > >
>>>>>> > >>> > > >
>>>>>> > >>> > > > --
>>>>>> > >>> > > > http://www.needhamsoftware.com (work)
>>>>>> > >>> > > > http://www.the111shift.com (play)
>>>>>> > >>>
>>>>>> > >>>
>>>>>> ---------------------------------------------------------------------
>>>>>> > >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>> > >>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>> > >>>
>>>>>> > >
>>>>>> > >
>>>>>> > > --
>>>>>> > > http://www.needhamsoftware.com (work)
>>>>>> > > http://www.the111shift.com (play)
>>>>>> >
>>>>>> >
>>>>>> ---------------------------------------------------------------------
>>>>>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>> >
>>>>>> >
>>>>>>
>>>>>> --
>>>>>> http://www.needhamsoftware.com (work)
>>>>>> http://www.the111shift.com (play)
>>>>>>
>>>>>
>>>>
>>>> --
>>>> http://www.needhamsoftware.com (work)
>>>> http://www.the111shift.com (play)
>>>>
>>>
>>
>> --
>> http://www.needhamsoftware.com (work)
>> http://www.the111shift.com (play)
>>
>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>


--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: Running 10.0 build with a custom lucene 9.5 [ In reply to ]
Ok pushed an attempt at a clearer message. LMK what you think.

On Tue, May 16, 2023 at 11:30?PM Gus Heck <gus.heck@gmail.com> wrote:

> Ok reading my last message I realize it still might not be clear. Here's
> what I observed:
>
> The class Codec clearly loaded, (from the lucene-core jar) when
> Codec$Holder tried to load the class initializer code went looking for the
> service definitions. It failed to find any of the the
> META-INF/services/org.apache.lucene.codec.Codec files in any of the jars
> residing in ~/.m2/repository (where the class evidently loads from),
> including the one in **the same jar that Codec had loaded from** and I
> observed it throwing Security exception by debugging at a level underneath
> the quoted method in my longer message above.
>
> So what it seems to imply is that the ability of java to load a class from
> a jar on the class path is not related to the FilePermission needed to load
> the services file.
>
> So what I want to communicate to the user is 2 things:
>
> 1. They are running a security manager, so the policy file is relevant.
> 2. The fact they are not seeing a SecurityException does not eliminate the
> possibility that they are missing permissions.
>
> We should craft a message that is clearer and communicates those two
> points.
>
> Adding a permission to the policy for the tests fixed everything (once I
> found the right policy file).
>
> -Gus
>
> On Tue, May 16, 2023 at 11:10?PM Gus Heck <gus.heck@gmail.com> wrote:
>
>> Oh hmm the google UI hid the quoted bit. If you don't like message let's
>> improve it. (actually, it should probably say the "file in the jar"... or
>> something a little more specific... not the jar entirely. The class loads,
>> but the service loader cant access the file in the same jar without the
>> FilePermission to (re?) access the jar it seems)
>>
>> On Tue, May 16, 2023 at 11:05?PM Gus Heck <gus.heck@gmail.com> wrote:
>>
>>> I propose to improve the message on an exception already thrown.
>>>
>>> On Tue, May 16, 2023 at 11:04?PM Ishan Chattopadhyaya <
>>> ichattopadhyaya@gmail.com> wrote:
>>>
>>>> You propose to throw an exception containing this, right?
>>>>
>>>> > Java does not throw SecurityException if this
>>>> is the case, it just ignores the jar!
>>>>
>>>> Are you serious?
>>>>
>>>> On Wed, 17 May, 2023, 8:02 am Gus Heck, <gus.heck@gmail.com> wrote:
>>>>
>>>>> Blaming?
>>>>>
>>>>> On Tue, May 16, 2023 at 10:05?PM Ishan Chattopadhyaya <
>>>>> ichattopadhyaya@gmail.com> wrote:
>>>>>
>>>>>> > Having that explicitly called out would have been SUPER helpful.
>>>>>>
>>>>>> Blaming Java in an exception thrown by Lucene is a ridiculous idea.
>>>>>>
>>>>>> On Wed, 17 May, 2023, 3:33 am Gus Heck, <gus.heck@gmail.com> wrote:
>>>>>>
>>>>>>> Found it.
>>>>>>>
>>>>>>> It's a solr thing made worse by the interaction of lucene testutils
>>>>>>> and
>>>>>>> jdk.internal.loader.URLClassPath's decision to hide anything gone
>>>>>>> wrong
>>>>>>> when checking a URL
>>>>>>> /*
>>>>>>> * Checks whether the resource URL should be returned.
>>>>>>> * Returns null on security check failure.
>>>>>>> * Called by java.net.URLClassLoader.
>>>>>>> */
>>>>>>> public static URL checkURL(URL url) {
>>>>>>> if (url != null) {
>>>>>>> try {
>>>>>>> check(url);
>>>>>>> } catch (Exception e) {
>>>>>>> return null;
>>>>>>> }
>>>>>>> }
>>>>>>> return url;
>>>>>>> }
>>>>>>>
>>>>>>> Yay. Fun. JDK classes swallowing exceptions silently.
>>>>>>>
>>>>>>> At the start of this it only took me a little while to discover that
>>>>>>> there
>>>>>>> was a security manager in play via debugging. Remembering that I saw
>>>>>>> emails
>>>>>>> about that, I went to jira, found the ticket enabling it by default
>>>>>>> in 9.x
>>>>>>> and eventually tracked down the name of the security policy file by
>>>>>>> reading
>>>>>>> solr.in.sh and /bin/solr... The key issue that tripped me up is
>>>>>>> that the
>>>>>>> tests have a *separate* security policy file, and there was pretty
>>>>>>> much no
>>>>>>> way to know this without extensive reading of the build. Thus I got
>>>>>>> thrown
>>>>>>> off track when
>>>>>>>
>>>>>>> permission java.io.FilePermission
>>>>>>> "${user.home}${/}.m2${/}repository${/}-", "read";
>>>>>>>
>>>>>>> To solr/server/etc/security.policy had no effect. That and the fact
>>>>>>> that
>>>>>>> no security exception was reported, led me to start chasing
>>>>>>> increasingly
>>>>>>> improbable hypotheses. Many hours later when I went back to debugging
>>>>>>> deeply into class loading, I found that the code was actually
>>>>>>> reading the
>>>>>>> jar files in question, and then I finally caught it throwing a
>>>>>>> security
>>>>>>> exception during my debugging.
>>>>>>>
>>>>>>> It turns out that adding the above permission to
>>>>>>> gradle/testing/randomization/policies/solr-tests.policy allows the
>>>>>>> test to
>>>>>>> pass. [1]
>>>>>>>
>>>>>>> I think we need to document this somewhere (or someone needs to
>>>>>>> point me to
>>>>>>> the doc I missed, FWIW I hit this basically following the process in
>>>>>>> dev-docs/dependency-upgrades.adoc treating lucene like a dependency,
>>>>>>> and
>>>>>>> unaware that there is a "shortcut" mode for lucene specifically in
>>>>>>> gradle/lucene-dev/lucene-dev-repo-composite.gradle and I find
>>>>>>> reading that
>>>>>>> file none-to clear anyway)
>>>>>>>
>>>>>>> That's the solr part, the lucene part is that the security exception
>>>>>>> is hit
>>>>>>> when in org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>>>>>> when
>>>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv#before
>>>>>>> does
>>>>>>>
>>>>>>> savedCodec = Codec.getDefault();
>>>>>>>
>>>>>>> The error message "An SPI class of type
>>>>>>> org.apache.lucene.codecs.Codec with
>>>>>>> name 'Lucene95' does not exist." was moderately misleading because
>>>>>>> the file
>>>>>>> and the services files in the jar definitely did exist. This message
>>>>>>> should
>>>>>>> vary if there is an installed security manager, maybe saying
>>>>>>> something like:
>>>>>>>
>>>>>>> "An SPI class of type org.apache.lucene.codecs.Codec with name
>>>>>>> 'Lucene95'
>>>>>>> does not exist. We have detected that a security manager is
>>>>>>> installed so it
>>>>>>> is also possible that the jar containing the codec is inaccessible
>>>>>>> under
>>>>>>> the current security policy. (Java does not throw SecurityException
>>>>>>> if this
>>>>>>> is the case, it just ignores the jar!)" [2]
>>>>>>>
>>>>>>> Having that explicitly called out would have been SUPER helpful.
>>>>>>>
>>>>>>> -Gus
>>>>>>>
>>>>>>> [1]: https://issues.apache.org/jira/browse/SOLR-16804
>>>>>>> [2]: https://github.com/apache/lucene/issues/12300
>>>>>>>
>>>>>>>
>>>>>>> On Mon, May 15, 2023 at 3:17?PM Michael Sokolov <msokolov@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> > random guess - does it have something to do with modules?
>>>>>>> >
>>>>>>> > On Mon, May 15, 2023 at 11:14?AM Gus Heck <gus.heck@gmail.com>
>>>>>>> wrote:
>>>>>>> > >
>>>>>>> > > I hadn't seen that one. Thanks, I'll look at it. It already
>>>>>>> looks a bit
>>>>>>> > confusing though since it seems to have options for pointing to a
>>>>>>> repo, but
>>>>>>> > I appear to be pulling the jars successfully from .m2/repository
>>>>>>> already...
>>>>>>> > (except then they don't work, so successful means I see them in the
>>>>>>> > classpath of the relevant classloader). And if we can't deploy a
>>>>>>> valid jar
>>>>>>> > to mavenLocal for some reason (tweaked the solr build so it sees
>>>>>>> > mavenLocal()), (or solr can't consume such a jar) that seems like
>>>>>>> an issue
>>>>>>> > for whichever one is breaking that.
>>>>>>> > >
>>>>>>> > > Debugging: The JDK appears to be attempting to load the services
>>>>>>> file
>>>>>>> > from modules, but not seeing the lucene module. (just the jdk
>>>>>>> ones) Also it
>>>>>>> > passes through a block that says:
>>>>>>> > >
>>>>>>> > > // not in a package of a module defined to this
>>>>>>> loader
>>>>>>> > > for (URL url : findMiscResource(name)) {
>>>>>>> > >
>>>>>>> > > (but then iterates
>>>>>>> > jdk.internal.loader.BuiltinClassLoader#nameToModule.values() to
>>>>>>> load things
>>>>>>> > anyway)
>>>>>>> > >
>>>>>>> > > -Gus
>>>>>>> > >
>>>>>>> > > On Mon, May 15, 2023 at 10:54?AM Houston Putman <
>>>>>>> houstonputman@gmail.com>
>>>>>>> > wrote:
>>>>>>> > >>
>>>>>>> > >> Gus, I haven't done this myself, but are you using the
>>>>>>> instructions
>>>>>>> > provided in Solr's
>>>>>>> "gradle/lucene-dev/lucene-dev-repo-composite.gradle"?
>>>>>>> > >>
>>>>>>> > >> It looks like you need to specify the development lucene version
>>>>>>> > differently than other dependencies...
>>>>>>> > >>
>>>>>>> > >> - Houston
>>>>>>> > >>
>>>>>>> > >> On Sat, May 13, 2023 at 10:14?AM Michael Sokolov <
>>>>>>> msokolov@gmail.com>
>>>>>>> > wrote:
>>>>>>> > >>>
>>>>>>> > >>> doh I actually read your email and you said you already
>>>>>>> checked that -
>>>>>>> > >>> I'm going to send out one of those "sokolov would like to
>>>>>>> retract the
>>>>>>> > >>> previous email" emails. Does GMail even pretend to do that? I
>>>>>>> don't
>>>>>>> > >>> know what's going on there! sorry
>>>>>>> > >>>
>>>>>>> > >>> On Sat, May 13, 2023 at 10:13?AM Michael Sokolov <
>>>>>>> msokolov@gmail.com>
>>>>>>> > wrote:
>>>>>>> > >>> >
>>>>>>> > >>> > sorry - META-INF not WEB-INF
>>>>>>> > >>> >
>>>>>>> > >>> > On Sat, May 13, 2023 at 10:12?AM Michael Sokolov <
>>>>>>> msokolov@gmail.com>
>>>>>>> > wrote:
>>>>>>> > >>> > >
>>>>>>> > >>> > > You are probably missing the contents of WEB-INF in your
>>>>>>> custom
>>>>>>> > jar?
>>>>>>> > >>> > > Roughly speaking the files in there define run-time-bound
>>>>>>> > "services"
>>>>>>> > >>> > > that are looked up by name by the JDK's service-loader API.
>>>>>>> > >>> > >
>>>>>>> > >>> > > On Sat, May 13, 2023 at 9:33?AM Gus Heck <
>>>>>>> gus.heck@gmail.com>
>>>>>>> > wrote:
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > Cross posting to lucene on the possibility that folks
>>>>>>> here are
>>>>>>> > more likely to add customized lucene to Solr and recognize what I'm
>>>>>>> > stumbling on? (zero responses on solr list)
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > Note that the specific test that I happened to copy is
>>>>>>> not the
>>>>>>> > issue, all tests are doing this (or at least so many tests are
>>>>>>> failing I
>>>>>>> > can't see the ones that are passing easily).
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > ---------- Forwarded message ---------
>>>>>>> > >>> > > > From: Gus Heck <gus.heck@gmail.com>
>>>>>>> > >>> > > > Date: Wed, May 10, 2023 at 6:50?PM
>>>>>>> > >>> > > > Subject: Running 10.0 build with a custom lucene 9.5
>>>>>>> > >>> > > > To: <dev@solr.apache.org>
>>>>>>> > >>> > > >
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > Lucene:
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > I made a tweak to lucene for something I'm
>>>>>>> investigating, gave
>>>>>>> > it a new version, deployed to mavenLocal()
>>>>>>> > >>> > > > I have verified that the jars are built with correct
>>>>>>> > META-INF/services files
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > Solr:
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > I added mavenLocal() in gradle/globals.gradle
>>>>>>> > >>> > > > I removed the license file sha1 sigs for the default
>>>>>>> lucene &
>>>>>>> > creates signatures for my test version
>>>>>>> > >>> > > > I updated versions.props
>>>>>>> > >>> > > > I updated versions.lock
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > Now when I run individual solr tests via my ide they
>>>>>>> seem to
>>>>>>> > pass, but virtually every test run via gradle fails with something
>>>>>>> like:
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner >
>>>>>>> classMethod FAILED
>>>>>>> > >>> > > > java.lang.ExceptionInInitializerError
>>>>>>> > >>> > > > at
>>>>>>> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>>>>> > >>> > > > at
>>>>>>> org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>>>>> > >>> > > > at
>>>>>>> java.base/java.lang.Thread.run(Thread.java:829)
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > Caused by:
>>>>>>> > >>> > > > java.lang.IllegalArgumentException: An SPI class
>>>>>>> of type
>>>>>>> > org.apache.lucene.codecs.Codec with name 'Lucene95' does not
>>>>>>> exist. You
>>>>>>> > need to add the corresponding JAR file supporting this SPI to your
>>>>>>> > classpath. The current classpath supports the following names: []
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>>>>>>> > >>> > > > at
>>>>>>> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>>>>>> > >>> > > > ... 19 more
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner >
>>>>>>> classMethod FAILED
>>>>>>> > >>> > > > java.lang.NullPointerException
>>>>>>> > >>> > > > at
>>>>>>> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
>>>>>>> > >>> > > > at
>>>>>>> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>>>>> > >>> > > > at
>>>>>>> org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>>>>> > >>> > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>>>>> > >>> > > > at
>>>>>>> java.base/java.lang.Thread.run(Thread.java:829)
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > org.apache.solr.embedded.TestJettySolrRunner > test
>>>>>>> suite's
>>>>>>> > output saved to
>>>>>>> >
>>>>>>> /Users/gus/projects/apache/solr/code/solr/solr/test-framework/build/test-results/test/outputs/OUTPUT-org.apache.solr.embedded.TestJettySolrRunner.txt,
>>>>>>> > copied below:
>>>>>>> > >>> > > > > java.lang.ExceptionInInitializerError
>>>>>>> > >>> > > > > at
>>>>>>> > org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:137)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:42)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>>>>> > >>> > > > > at
>>>>>>> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>>>>> > >>> > > > > at
>>>>>>> java.base/java.lang.Thread.run(Thread.java:829)
>>>>>>> > >>> > > > >
>>>>>>> > >>> > > > > Caused by:
>>>>>>> > >>> > > > > java.lang.IllegalArgumentException: An SPI
>>>>>>> class of
>>>>>>> > type org.apache.lucene.codecs.Codec with name 'Lucene95' does not
>>>>>>> exist.
>>>>>>> > You need to add the corresponding JAR file supporting this SPI to
>>>>>>> your
>>>>>>> > classpath. The current classpath supports the following names: []
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
>>>>>>> > >>> > > > > at
>>>>>>> > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58)
>>>>>>> > >>> > > > > ... 19 more
>>>>>>> > >>> > > > > java.lang.NullPointerException
>>>>>>> > >>> > > > > at
>>>>>>> > java.base/java.util.Objects.requireNonNull(Objects.java:221)
>>>>>>> > >>> > > > > at
>>>>>>> > org.apache.lucene.codecs.Codec.setDefault(Codec.java:151)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv.after(TestRuleSetupAndRestoreClassEnv.java:292)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:49)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
>>>>>>> > >>> > > > > at
>>>>>>> > org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
>>>>>>> > >>> > > > > at
>>>>>>> >
>>>>>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
>>>>>>> > >>> > > > > at
>>>>>>> java.base/java.lang.Thread.run(Thread.java:829)
>>>>>>> > >>> > > > 2> NOTE: Mac OS X 12.6 aarch64/Azul Systems, Inc.
>>>>>>> 11.0.19
>>>>>>> > (64-bit)/cpus=10,threads=1,free=235002360,total=268435456
>>>>>>> > >>> > > > 2> NOTE: All tests run in this JVM:
>>>>>>> [TestJettySolrRunner]
>>>>>>> > >>> > > > 2> NOTE: reproduce with: gradlew test --tests
>>>>>>> > TestJettySolrRunner -Dtests.seed=2FB768B5A303764
>>>>>>> -Dtests.asserts=true
>>>>>>> > -Dtests.file.encoding=UTF-8
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > The only difference I've been able to determine is that
>>>>>>> the IDE
>>>>>>> > does not run the test with a security manager, but that feels like
>>>>>>> a red
>>>>>>> > herring since nothing is throwing security exception (or if it is,
>>>>>>> it's
>>>>>>> > getting swallowed!).
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > I have debugged and I can see that the classloaders have
>>>>>>> the new
>>>>>>> > lucene jars I built on the classpath (in /Users/gus/.m2/repository
>>>>>>> as they
>>>>>>> > should), but I haven't yet figuree out why the services are not
>>>>>>> loading.
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > -Gus
>>>>>>> > >>> > > >
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > --
>>>>>>> > >>> > > > http://www.needhamsoftware.com (work)
>>>>>>> > >>> > > > http://www.the111shift.com (play)
>>>>>>> > >>> > > >
>>>>>>> > >>> > > >
>>>>>>> > >>> > > > --
>>>>>>> > >>> > > > http://www.needhamsoftware.com (work)
>>>>>>> > >>> > > > http://www.the111shift.com (play)
>>>>>>> > >>>
>>>>>>> > >>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> > >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>>> > >>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>>> > >>>
>>>>>>> > >
>>>>>>> > >
>>>>>>> > > --
>>>>>>> > > http://www.needhamsoftware.com (work)
>>>>>>> > > http://www.the111shift.com (play)
>>>>>>> >
>>>>>>> >
>>>>>>> ---------------------------------------------------------------------
>>>>>>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>>> >
>>>>>>> >
>>>>>>>
>>>>>>> --
>>>>>>> http://www.needhamsoftware.com (work)
>>>>>>> http://www.the111shift.com (play)
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> http://www.needhamsoftware.com (work)
>>>>> http://www.the111shift.com (play)
>>>>>
>>>>
>>>
>>> --
>>> http://www.needhamsoftware.com (work)
>>> http://www.the111shift.com (play)
>>>
>>
>>
>> --
>> http://www.needhamsoftware.com (work)
>> http://www.the111shift.com (play)
>>
>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>


--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)