Mailing List Archive

Gradle HELP
I am trying to get Lucene to build in a very "special" build system
based on Gradle, and I am stumbling across all kinds of things I don't
really understand (which is most of everything about gradle at this
point). Cookie cutter cargo cultism is getting me pretty far, but one
thing I don't understand: we have this in the precommit task:

// Root-level validation tasks.
dependsOn ":verifyLocks"
dependsOn ":versionsPropsAreSorted"
dependsOn ":checkWorkingCopyClean"

What is the "verifyLocks" task? I can't seem to find it defined anywhere

-Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Gradle HELP [ In reply to ]
I believe it is one of the built in gradle tasks available to us.
https://docs.gradle.org/current/userguide/dependency_locking.html

On Wed, Aug 18, 2021 at 3:22 PM Michael Sokolov <msokolov@gmail.com> wrote:
>
> I am trying to get Lucene to build in a very "special" build system
> based on Gradle, and I am stumbling across all kinds of things I don't
> really understand (which is most of everything about gradle at this
> point). Cookie cutter cargo cultism is getting me pretty far, but one
> thing I don't understand: we have this in the precommit task:
>
> // Root-level validation tasks.
> dependsOn ":verifyLocks"
> dependsOn ":versionsPropsAreSorted"
> dependsOn ":checkWorkingCopyClean"
>
> What is the "verifyLocks" task? I can't seem to find it defined anywhere
>
> -Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Gradle HELP [ In reply to ]
ah interesting. I may have broken it in the midst of debugging
something else. I did manage to get most of the module jars built, and
I'm moving on to trying to get tests to run. I'm running into what
looks like more dependency/classpath resolution problems, but with our
module-module dependencies. If anyone has a clue what might be going
on or can suggest some futher debugging tips, I'd be very
appreciative. Here are the gory details...


For example, tests in o.a.l.codecs are failing because:

org.apache.lucene.codecs.uniformsplit.TestUniformSplitPostingFormat >
classMethod FAILED
java.lang.ExceptionInInitializerError
at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
at org.apache.lucene.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:133)
...
java.lang.IllegalArgumentException: An SPI class of type
org.apache.lucene.codecs.Codec with name 'Lucene90' does not exist.
You need to add the corresponding JAR file supporting this SPI to your
classpath. The current classpath supports the following names:
[SimpleText]

and I print out the classpath used to run the test by:

tasks.withType(Test) {
doFirst {
println "Test $allJvmArgs"
println "Test classpath " + classpath.getFiles().toString()
}
}

and the classpath includes

...build/lib/lucene-core-9.0.jar

and that jar includes

-rw-r--r-- 843 18-Aug-2021 21:40:50
META-INF/services/org.apache.lucene.codecs.Codec

which has the expected content -- so I am kind of confused. Are these
Codec files supposed to be merged together? Or is this a conflict
resolved by classpath-order?

and there are other dependency issues, but I think it would just be
more confusing to list them all here.

PS: full listing below:

Test classpath [./local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/classes/java/test,
/local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/resources/test,
/local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/classes/java/main,
/local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/resources/main,
/local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-test-framework-9.0_az.jar,
/local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-codecs-9.0_az.jar,
/local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-core-9.0_az.jar,
/local/home/sokolovm/workspace/lucene/env/gradle-cache-2/brazil/junit/4.12/junit-4.12.jar,
/local/home/sokolovm/workspace/lucene/env/gradle-cache-2/brazil/Hamcrest/1.3/Hamcrest-1.3.jar,
/local/home/sokolovm/.gradle/caches/modules-2/files-2.1/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.7.2/84ed6b5f70906f9ef173db67ccde657b43ea60df/randomizedtesting-runner-2.7.2.jar,
/local/home/sokolovm/.gradle/caches/modules-2/files-2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-4.12.jar,
/local/home/sokolovm/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest-core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-1.3.jar,
/local/home/sokolovm/.gradle/caches/modules-2/files-2.1/org.hamcrest/java-hamcrest/2.0.0.0/f1c8853ade0ecf707f5a261c830e98893983813/java-hamcrest-2.0.0.0.jar]

On Wed, Aug 18, 2021 at 4:30 PM Mike Drob <mdrob@mdrob.com> wrote:
>
> I believe it is one of the built in gradle tasks available to us.
> https://docs.gradle.org/current/userguide/dependency_locking.html
>
> On Wed, Aug 18, 2021 at 3:22 PM Michael Sokolov <msokolov@gmail.com> wrote:
> >
> > I am trying to get Lucene to build in a very "special" build system
> > based on Gradle, and I am stumbling across all kinds of things I don't
> > really understand (which is most of everything about gradle at this
> > point). Cookie cutter cargo cultism is getting me pretty far, but one
> > thing I don't understand: we have this in the precommit task:
> >
> > // Root-level validation tasks.
> > dependsOn ":verifyLocks"
> > dependsOn ":versionsPropsAreSorted"
> > dependsOn ":checkWorkingCopyClean"
> >
> > What is the "verifyLocks" task? I can't seem to find it defined anywhere
> >
> > -Mike
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Gradle HELP [ In reply to ]
Update: running tests with -Dtests.useSecurityManager=false got over
that hump. Not sure why the security manager would be behaving
differently in this setup...

On Wed, Aug 18, 2021 at 6:41 PM Michael Sokolov <msokolov@gmail.com> wrote:
>
> ah interesting. I may have broken it in the midst of debugging
> something else. I did manage to get most of the module jars built, and
> I'm moving on to trying to get tests to run. I'm running into what
> looks like more dependency/classpath resolution problems, but with our
> module-module dependencies. If anyone has a clue what might be going
> on or can suggest some futher debugging tips, I'd be very
> appreciative. Here are the gory details...
>
>
> For example, tests in o.a.l.codecs are failing because:
>
> org.apache.lucene.codecs.uniformsplit.TestUniformSplitPostingFormat >
> classMethod FAILED
> java.lang.ExceptionInInitializerError
> at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> at org.apache.lucene.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:133)
> ...
> java.lang.IllegalArgumentException: An SPI class of type
> org.apache.lucene.codecs.Codec with name 'Lucene90' does not exist.
> You need to add the corresponding JAR file supporting this SPI to your
> classpath. The current classpath supports the following names:
> [SimpleText]
>
> and I print out the classpath used to run the test by:
>
> tasks.withType(Test) {
> doFirst {
> println "Test $allJvmArgs"
> println "Test classpath " + classpath.getFiles().toString()
> }
> }
>
> and the classpath includes
>
> ...build/lib/lucene-core-9.0.jar
>
> and that jar includes
>
> -rw-r--r-- 843 18-Aug-2021 21:40:50
> META-INF/services/org.apache.lucene.codecs.Codec
>
> which has the expected content -- so I am kind of confused. Are these
> Codec files supposed to be merged together? Or is this a conflict
> resolved by classpath-order?
>
> and there are other dependency issues, but I think it would just be
> more confusing to list them all here.
>
> PS: full listing below:
>
> Test classpath [./local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/classes/java/test,
> /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/resources/test,
> /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/classes/java/main,
> /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/resources/main,
> /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-test-framework-9.0_az.jar,
> /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-codecs-9.0_az.jar,
> /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-core-9.0_az.jar,
> /local/home/sokolovm/workspace/lucene/env/gradle-cache-2/brazil/junit/4.12/junit-4.12.jar,
> /local/home/sokolovm/workspace/lucene/env/gradle-cache-2/brazil/Hamcrest/1.3/Hamcrest-1.3.jar,
> /local/home/sokolovm/.gradle/caches/modules-2/files-2.1/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.7.2/84ed6b5f70906f9ef173db67ccde657b43ea60df/randomizedtesting-runner-2.7.2.jar,
> /local/home/sokolovm/.gradle/caches/modules-2/files-2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-4.12.jar,
> /local/home/sokolovm/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest-core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-1.3.jar,
> /local/home/sokolovm/.gradle/caches/modules-2/files-2.1/org.hamcrest/java-hamcrest/2.0.0.0/f1c8853ade0ecf707f5a261c830e98893983813/java-hamcrest-2.0.0.0.jar]
>
> On Wed, Aug 18, 2021 at 4:30 PM Mike Drob <mdrob@mdrob.com> wrote:
> >
> > I believe it is one of the built in gradle tasks available to us.
> > https://docs.gradle.org/current/userguide/dependency_locking.html
> >
> > On Wed, Aug 18, 2021 at 3:22 PM Michael Sokolov <msokolov@gmail.com> wrote:
> > >
> > > I am trying to get Lucene to build in a very "special" build system
> > > based on Gradle, and I am stumbling across all kinds of things I don't
> > > really understand (which is most of everything about gradle at this
> > > point). Cookie cutter cargo cultism is getting me pretty far, but one
> > > thing I don't understand: we have this in the precommit task:
> > >
> > > // Root-level validation tasks.
> > > dependsOn ":verifyLocks"
> > > dependsOn ":versionsPropsAreSorted"
> > > dependsOn ":checkWorkingCopyClean"
> > >
> > > What is the "verifyLocks" task? I can't seem to find it defined anywhere
> > >
> > > -Mike
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: dev-help@lucene.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Gradle HELP [ In reply to ]
OK I see it's because this build system puts the output in a different
folder that is not covered by the security policy. phew.

On Wed, Aug 18, 2021 at 10:30 PM Michael Sokolov <msokolov@gmail.com> wrote:
>
> Update: running tests with -Dtests.useSecurityManager=false got over
> that hump. Not sure why the security manager would be behaving
> differently in this setup...
>
> On Wed, Aug 18, 2021 at 6:41 PM Michael Sokolov <msokolov@gmail.com> wrote:
> >
> > ah interesting. I may have broken it in the midst of debugging
> > something else. I did manage to get most of the module jars built, and
> > I'm moving on to trying to get tests to run. I'm running into what
> > looks like more dependency/classpath resolution problems, but with our
> > module-module dependencies. If anyone has a clue what might be going
> > on or can suggest some futher debugging tips, I'd be very
> > appreciative. Here are the gory details...
> >
> >
> > For example, tests in o.a.l.codecs are failing because:
> >
> > org.apache.lucene.codecs.uniformsplit.TestUniformSplitPostingFormat >
> > classMethod FAILED
> > java.lang.ExceptionInInitializerError
> > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > at org.apache.lucene.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetupAndRestoreClassEnv.java:133)
> > ...
> > java.lang.IllegalArgumentException: An SPI class of type
> > org.apache.lucene.codecs.Codec with name 'Lucene90' does not exist.
> > You need to add the corresponding JAR file supporting this SPI to your
> > classpath. The current classpath supports the following names:
> > [SimpleText]
> >
> > and I print out the classpath used to run the test by:
> >
> > tasks.withType(Test) {
> > doFirst {
> > println "Test $allJvmArgs"
> > println "Test classpath " + classpath.getFiles().toString()
> > }
> > }
> >
> > and the classpath includes
> >
> > ...build/lib/lucene-core-9.0.jar
> >
> > and that jar includes
> >
> > -rw-r--r-- 843 18-Aug-2021 21:40:50
> > META-INF/services/org.apache.lucene.codecs.Codec
> >
> > which has the expected content -- so I am kind of confused. Are these
> > Codec files supposed to be merged together? Or is this a conflict
> > resolved by classpath-order?
> >
> > and there are other dependency issues, but I think it would just be
> > more confusing to list them all here.
> >
> > PS: full listing below:
> >
> > Test classpath [./local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/classes/java/test,
> > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/resources/test,
> > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/classes/java/main,
> > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/resources/main,
> > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-test-framework-9.0_az.jar,
> > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-codecs-9.0_az.jar,
> > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-core-9.0_az.jar,
> > /local/home/sokolovm/workspace/lucene/env/gradle-cache-2/brazil/junit/4.12/junit-4.12.jar,
> > /local/home/sokolovm/workspace/lucene/env/gradle-cache-2/brazil/Hamcrest/1.3/Hamcrest-1.3.jar,
> > /local/home/sokolovm/.gradle/caches/modules-2/files-2.1/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.7.2/84ed6b5f70906f9ef173db67ccde657b43ea60df/randomizedtesting-runner-2.7.2.jar,
> > /local/home/sokolovm/.gradle/caches/modules-2/files-2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-4.12.jar,
> > /local/home/sokolovm/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest-core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-1.3.jar,
> > /local/home/sokolovm/.gradle/caches/modules-2/files-2.1/org.hamcrest/java-hamcrest/2.0.0.0/f1c8853ade0ecf707f5a261c830e98893983813/java-hamcrest-2.0.0.0.jar]
> >
> > On Wed, Aug 18, 2021 at 4:30 PM Mike Drob <mdrob@mdrob.com> wrote:
> > >
> > > I believe it is one of the built in gradle tasks available to us.
> > > https://docs.gradle.org/current/userguide/dependency_locking.html
> > >
> > > On Wed, Aug 18, 2021 at 3:22 PM Michael Sokolov <msokolov@gmail.com> wrote:
> > > >
> > > > I am trying to get Lucene to build in a very "special" build system
> > > > based on Gradle, and I am stumbling across all kinds of things I don't
> > > > really understand (which is most of everything about gradle at this
> > > > point). Cookie cutter cargo cultism is getting me pretty far, but one
> > > > thing I don't understand: we have this in the precommit task:
> > > >
> > > > // Root-level validation tasks.
> > > > dependsOn ":verifyLocks"
> > > > dependsOn ":versionsPropsAreSorted"
> > > > dependsOn ":checkWorkingCopyClean"
> > > >
> > > > What is the "verifyLocks" task? I can't seem to find it defined anywhere
> > > >
> > > > -Mike
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: dev-help@lucene.apache.org
> > >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Gradle HELP [ In reply to ]
The "verifyLocks" is in gradle... but in Lucene/ Solr it's actually
done by the palantir's plugin:
https://github.com/palantir/gradle-consistent-versions

To quote from their site:
"There is a verifyLocks task (automatically run as part of check) that
will ensure versions.lock is still consistent with the current
dependencies."

If you change the global dependency in versions.props the
versions.lock file needs to be updated manually via --write-locks, as
explained here:

https://github.com/apache/lucene/blob/main/help/dependencies.txt#L50-L59

Can I ask why you need to rewrite gradle to gradle so that the other
build system works? This seems a tad insane?

Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: Gradle HELP [ In reply to ]
Hi,

would you give us some hints what you are trying to do? Is this related to the Lucene build system and are you preparing a PR or is this some Amazon-only rewrite? In the latter case, can't you not wrap the execution of gradlew in your "internal" build system? You can also configure gradle to fetch dependencies from other repositories than maven central.

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Michael Sokolov <msokolov@gmail.com>
> Sent: Thursday, August 19, 2021 5:18 AM
> To: Lucene Dev <dev@lucene.apache.org>
> Subject: Re: Gradle HELP
>
> OK I see it's because this build system puts the output in a different
> folder that is not covered by the security policy. phew.
>
> On Wed, Aug 18, 2021 at 10:30 PM Michael Sokolov <msokolov@gmail.com>
> wrote:
> >
> > Update: running tests with -Dtests.useSecurityManager=false got over
> > that hump. Not sure why the security manager would be behaving
> > differently in this setup...
> >
> > On Wed, Aug 18, 2021 at 6:41 PM Michael Sokolov <msokolov@gmail.com>
> wrote:
> > >
> > > ah interesting. I may have broken it in the midst of debugging
> > > something else. I did manage to get most of the module jars built, and
> > > I'm moving on to trying to get tests to run. I'm running into what
> > > looks like more dependency/classpath resolution problems, but with our
> > > module-module dependencies. If anyone has a clue what might be going
> > > on or can suggest some futher debugging tips, I'd be very
> > > appreciative. Here are the gory details...
> > >
> > >
> > > For example, tests in o.a.l.codecs are failing because:
> > >
> > > org.apache.lucene.codecs.uniformsplit.TestUniformSplitPostingFormat >
> > > classMethod FAILED
> > > java.lang.ExceptionInInitializerError
> > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > > at
> org.apache.lucene.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetup
> AndRestoreClassEnv.java:133)
> > > ...
> > > java.lang.IllegalArgumentException: An SPI class of type
> > > org.apache.lucene.codecs.Codec with name 'Lucene90' does not exist.
> > > You need to add the corresponding JAR file supporting this SPI to your
> > > classpath. The current classpath supports the following names:
> > > [SimpleText]
> > >
> > > and I print out the classpath used to run the test by:
> > >
> > > tasks.withType(Test) {
> > > doFirst {
> > > println "Test $allJvmArgs"
> > > println "Test classpath " + classpath.getFiles().toString()
> > > }
> > > }
> > >
> > > and the classpath includes
> > >
> > > ...build/lib/lucene-core-9.0.jar
> > >
> > > and that jar includes
> > >
> > > -rw-r--r-- 843 18-Aug-2021 21:40:50
> > > META-INF/services/org.apache.lucene.codecs.Codec
> > >
> > > which has the expected content -- so I am kind of confused. Are these
> > > Codec files supposed to be merged together? Or is this a conflict
> > > resolved by classpath-order?
> > >
> > > and there are other dependency issues, but I think it would just be
> > > more confusing to list them all here.
> > >
> > > PS: full listing below:
> > >
> > > Test classpath
> [/local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> ses/java/test,
> > >
> /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> urces/test,
> > >
> /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> ses/java/main,
> > >
> /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> urces/main,
> > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-test-
> framework-9.0_az.jar,
> > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-
> codecs-9.0_az.jar,
> > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-core-
> 9.0_az.jar,
> > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> 2/brazil/junit/4.12/junit-4.12.jar,
> > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> 2/brazil/Hamcrest/1.3/Hamcrest-1.3.jar,
> > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> 2.1/com.carrotsearch.randomizedtesting/randomizedtesting-
> runner/2.7.2/84ed6b5f70906f9ef173db67ccde657b43ea60df/randomizedtesti
> ng-runner-2.7.2.jar,
> > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> 2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-
> 4.12.jar,
> > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> 2.1/org.hamcrest/hamcrest-
> core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-
> 1.3.jar,
> > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> 2.1/org.hamcrest/java-
> hamcrest/2.0.0.0/f1c8853ade0ecf707f5a261c830e98893983813/java-
> hamcrest-2.0.0.0.jar]
> > >
> > > On Wed, Aug 18, 2021 at 4:30 PM Mike Drob <mdrob@mdrob.com> wrote:
> > > >
> > > > I believe it is one of the built in gradle tasks available to us.
> > > > https://docs.gradle.org/current/userguide/dependency_locking.html
> > > >
> > > > On Wed, Aug 18, 2021 at 3:22 PM Michael Sokolov
> <msokolov@gmail.com> wrote:
> > > > >
> > > > > I am trying to get Lucene to build in a very "special" build system
> > > > > based on Gradle, and I am stumbling across all kinds of things I don't
> > > > > really understand (which is most of everything about gradle at this
> > > > > point). Cookie cutter cargo cultism is getting me pretty far, but one
> > > > > thing I don't understand: we have this in the precommit task:
> > > > >
> > > > > // Root-level validation tasks.
> > > > > dependsOn ":verifyLocks"
> > > > > dependsOn ":versionsPropsAreSorted"
> > > > > dependsOn ":checkWorkingCopyClean"
> > > > >
> > > > > What is the "verifyLocks" task? I can't seem to find it defined anywhere
> > > > >
> > > > > -Mike
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Gradle HELP [ In reply to ]
> This seems a tad insane?

Agree, and yes it's an Amazon thing. Basically our build servers are
not connected to the internet, and we use an internal repository and
corresponding gradle plugin for resolving dependencies that is
somewhat idiosyncratic (not Maven or anything else familiar, but acts
kind of like Maven repo). Uwe, I am basically doing what you say,
re-using as much as I can from our excellent Lucene build system, but
wrapping alone is insufficient: I have to make invasive changes here
and there. Believe me this will be a big improvement from the hacked
ant build we have been using.

On Thu, Aug 19, 2021 at 5:00 AM Uwe Schindler <uwe@thetaphi.de> wrote:
>
> Hi,
>
> would you give us some hints what you are trying to do? Is this related to the Lucene build system and are you preparing a PR or is this some Amazon-only rewrite? In the latter case, can't you not wrap the execution of gradlew in your "internal" build system? You can also configure gradle to fetch dependencies from other repositories than maven central.
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
> > -----Original Message-----
> > From: Michael Sokolov <msokolov@gmail.com>
> > Sent: Thursday, August 19, 2021 5:18 AM
> > To: Lucene Dev <dev@lucene.apache.org>
> > Subject: Re: Gradle HELP
> >
> > OK I see it's because this build system puts the output in a different
> > folder that is not covered by the security policy. phew.
> >
> > On Wed, Aug 18, 2021 at 10:30 PM Michael Sokolov <msokolov@gmail.com>
> > wrote:
> > >
> > > Update: running tests with -Dtests.useSecurityManager=false got over
> > > that hump. Not sure why the security manager would be behaving
> > > differently in this setup...
> > >
> > > On Wed, Aug 18, 2021 at 6:41 PM Michael Sokolov <msokolov@gmail.com>
> > wrote:
> > > >
> > > > ah interesting. I may have broken it in the midst of debugging
> > > > something else. I did manage to get most of the module jars built, and
> > > > I'm moving on to trying to get tests to run. I'm running into what
> > > > looks like more dependency/classpath resolution problems, but with our
> > > > module-module dependencies. If anyone has a clue what might be going
> > > > on or can suggest some futher debugging tips, I'd be very
> > > > appreciative. Here are the gory details...
> > > >
> > > >
> > > > For example, tests in o.a.l.codecs are failing because:
> > > >
> > > > org.apache.lucene.codecs.uniformsplit.TestUniformSplitPostingFormat >
> > > > classMethod FAILED
> > > > java.lang.ExceptionInInitializerError
> > > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > > > at
> > org.apache.lucene.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetup
> > AndRestoreClassEnv.java:133)
> > > > ...
> > > > java.lang.IllegalArgumentException: An SPI class of type
> > > > org.apache.lucene.codecs.Codec with name 'Lucene90' does not exist.
> > > > You need to add the corresponding JAR file supporting this SPI to your
> > > > classpath. The current classpath supports the following names:
> > > > [SimpleText]
> > > >
> > > > and I print out the classpath used to run the test by:
> > > >
> > > > tasks.withType(Test) {
> > > > doFirst {
> > > > println "Test $allJvmArgs"
> > > > println "Test classpath " + classpath.getFiles().toString()
> > > > }
> > > > }
> > > >
> > > > and the classpath includes
> > > >
> > > > ...build/lib/lucene-core-9.0.jar
> > > >
> > > > and that jar includes
> > > >
> > > > -rw-r--r-- 843 18-Aug-2021 21:40:50
> > > > META-INF/services/org.apache.lucene.codecs.Codec
> > > >
> > > > which has the expected content -- so I am kind of confused. Are these
> > > > Codec files supposed to be merged together? Or is this a conflict
> > > > resolved by classpath-order?
> > > >
> > > > and there are other dependency issues, but I think it would just be
> > > > more confusing to list them all here.
> > > >
> > > > PS: full listing below:
> > > >
> > > > Test classpath
> > [/local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> > ses/java/test,
> > > >
> > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> > urces/test,
> > > >
> > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> > ses/java/main,
> > > >
> > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> > urces/main,
> > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-test-
> > framework-9.0_az.jar,
> > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-
> > codecs-9.0_az.jar,
> > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-core-
> > 9.0_az.jar,
> > > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> > 2/brazil/junit/4.12/junit-4.12.jar,
> > > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> > 2/brazil/Hamcrest/1.3/Hamcrest-1.3.jar,
> > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > 2.1/com.carrotsearch.randomizedtesting/randomizedtesting-
> > runner/2.7.2/84ed6b5f70906f9ef173db67ccde657b43ea60df/randomizedtesti
> > ng-runner-2.7.2.jar,
> > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > 2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-
> > 4.12.jar,
> > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > 2.1/org.hamcrest/hamcrest-
> > core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-
> > 1.3.jar,
> > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > 2.1/org.hamcrest/java-
> > hamcrest/2.0.0.0/f1c8853ade0ecf707f5a261c830e98893983813/java-
> > hamcrest-2.0.0.0.jar]
> > > >
> > > > On Wed, Aug 18, 2021 at 4:30 PM Mike Drob <mdrob@mdrob.com> wrote:
> > > > >
> > > > > I believe it is one of the built in gradle tasks available to us.
> > > > > https://docs.gradle.org/current/userguide/dependency_locking.html
> > > > >
> > > > > On Wed, Aug 18, 2021 at 3:22 PM Michael Sokolov
> > <msokolov@gmail.com> wrote:
> > > > > >
> > > > > > I am trying to get Lucene to build in a very "special" build system
> > > > > > based on Gradle, and I am stumbling across all kinds of things I don't
> > > > > > really understand (which is most of everything about gradle at this
> > > > > > point). Cookie cutter cargo cultism is getting me pretty far, but one
> > > > > > thing I don't understand: we have this in the precommit task:
> > > > > >
> > > > > > // Root-level validation tasks.
> > > > > > dependsOn ":verifyLocks"
> > > > > > dependsOn ":versionsPropsAreSorted"
> > > > > > dependsOn ":checkWorkingCopyClean"
> > > > > >
> > > > > > What is the "verifyLocks" task? I can't seem to find it defined anywhere
> > > > > >
> > > > > > -Mike
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Gradle HELP [ In reply to ]
Ok, clear. I understand the reasons these things are shielded from the
internet. I'm sure gradle folks have thought about such scenarios
though - gradle is used across enterprises with similar setups? I
really know nothing about it but I feel pretty confident artifact
resolution can be redirected to only known locations. Sorry you have
to hack around!

Dawid

On Thu, Aug 19, 2021 at 2:00 PM Michael Sokolov <msokolov@gmail.com> wrote:
>
> > This seems a tad insane?
>
> Agree, and yes it's an Amazon thing. Basically our build servers are
> not connected to the internet, and we use an internal repository and
> corresponding gradle plugin for resolving dependencies that is
> somewhat idiosyncratic (not Maven or anything else familiar, but acts
> kind of like Maven repo). Uwe, I am basically doing what you say,
> re-using as much as I can from our excellent Lucene build system, but
> wrapping alone is insufficient: I have to make invasive changes here
> and there. Believe me this will be a big improvement from the hacked
> ant build we have been using.
>
> On Thu, Aug 19, 2021 at 5:00 AM Uwe Schindler <uwe@thetaphi.de> wrote:
> >
> > Hi,
> >
> > would you give us some hints what you are trying to do? Is this related to the Lucene build system and are you preparing a PR or is this some Amazon-only rewrite? In the latter case, can't you not wrap the execution of gradlew in your "internal" build system? You can also configure gradle to fetch dependencies from other repositories than maven central.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > https://www.thetaphi.de
> > eMail: uwe@thetaphi.de
> >
> > > -----Original Message-----
> > > From: Michael Sokolov <msokolov@gmail.com>
> > > Sent: Thursday, August 19, 2021 5:18 AM
> > > To: Lucene Dev <dev@lucene.apache.org>
> > > Subject: Re: Gradle HELP
> > >
> > > OK I see it's because this build system puts the output in a different
> > > folder that is not covered by the security policy. phew.
> > >
> > > On Wed, Aug 18, 2021 at 10:30 PM Michael Sokolov <msokolov@gmail.com>
> > > wrote:
> > > >
> > > > Update: running tests with -Dtests.useSecurityManager=false got over
> > > > that hump. Not sure why the security manager would be behaving
> > > > differently in this setup...
> > > >
> > > > On Wed, Aug 18, 2021 at 6:41 PM Michael Sokolov <msokolov@gmail.com>
> > > wrote:
> > > > >
> > > > > ah interesting. I may have broken it in the midst of debugging
> > > > > something else. I did manage to get most of the module jars built, and
> > > > > I'm moving on to trying to get tests to run. I'm running into what
> > > > > looks like more dependency/classpath resolution problems, but with our
> > > > > module-module dependencies. If anyone has a clue what might be going
> > > > > on or can suggest some futher debugging tips, I'd be very
> > > > > appreciative. Here are the gory details...
> > > > >
> > > > >
> > > > > For example, tests in o.a.l.codecs are failing because:
> > > > >
> > > > > org.apache.lucene.codecs.uniformsplit.TestUniformSplitPostingFormat >
> > > > > classMethod FAILED
> > > > > java.lang.ExceptionInInitializerError
> > > > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > > > > at
> > > org.apache.lucene.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetup
> > > AndRestoreClassEnv.java:133)
> > > > > ...
> > > > > java.lang.IllegalArgumentException: An SPI class of type
> > > > > org.apache.lucene.codecs.Codec with name 'Lucene90' does not exist.
> > > > > You need to add the corresponding JAR file supporting this SPI to your
> > > > > classpath. The current classpath supports the following names:
> > > > > [SimpleText]
> > > > >
> > > > > and I print out the classpath used to run the test by:
> > > > >
> > > > > tasks.withType(Test) {
> > > > > doFirst {
> > > > > println "Test $allJvmArgs"
> > > > > println "Test classpath " + classpath.getFiles().toString()
> > > > > }
> > > > > }
> > > > >
> > > > > and the classpath includes
> > > > >
> > > > > ...build/lib/lucene-core-9.0.jar
> > > > >
> > > > > and that jar includes
> > > > >
> > > > > -rw-r--r-- 843 18-Aug-2021 21:40:50
> > > > > META-INF/services/org.apache.lucene.codecs.Codec
> > > > >
> > > > > which has the expected content -- so I am kind of confused. Are these
> > > > > Codec files supposed to be merged together? Or is this a conflict
> > > > > resolved by classpath-order?
> > > > >
> > > > > and there are other dependency issues, but I think it would just be
> > > > > more confusing to list them all here.
> > > > >
> > > > > PS: full listing below:
> > > > >
> > > > > Test classpath
> > > [/local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> > > ses/java/test,
> > > > >
> > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> > > urces/test,
> > > > >
> > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> > > ses/java/main,
> > > > >
> > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> > > urces/main,
> > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-test-
> > > framework-9.0_az.jar,
> > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-
> > > codecs-9.0_az.jar,
> > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-core-
> > > 9.0_az.jar,
> > > > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> > > 2/brazil/junit/4.12/junit-4.12.jar,
> > > > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> > > 2/brazil/Hamcrest/1.3/Hamcrest-1.3.jar,
> > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > 2.1/com.carrotsearch.randomizedtesting/randomizedtesting-
> > > runner/2.7.2/84ed6b5f70906f9ef173db67ccde657b43ea60df/randomizedtesti
> > > ng-runner-2.7.2.jar,
> > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > 2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-
> > > 4.12.jar,
> > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > 2.1/org.hamcrest/hamcrest-
> > > core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-
> > > 1.3.jar,
> > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > 2.1/org.hamcrest/java-
> > > hamcrest/2.0.0.0/f1c8853ade0ecf707f5a261c830e98893983813/java-
> > > hamcrest-2.0.0.0.jar]
> > > > >
> > > > > On Wed, Aug 18, 2021 at 4:30 PM Mike Drob <mdrob@mdrob.com> wrote:
> > > > > >
> > > > > > I believe it is one of the built in gradle tasks available to us.
> > > > > > https://docs.gradle.org/current/userguide/dependency_locking.html
> > > > > >
> > > > > > On Wed, Aug 18, 2021 at 3:22 PM Michael Sokolov
> > > <msokolov@gmail.com> wrote:
> > > > > > >
> > > > > > > I am trying to get Lucene to build in a very "special" build system
> > > > > > > based on Gradle, and I am stumbling across all kinds of things I don't
> > > > > > > really understand (which is most of everything about gradle at this
> > > > > > > point). Cookie cutter cargo cultism is getting me pretty far, but one
> > > > > > > thing I don't understand: we have this in the precommit task:
> > > > > > >
> > > > > > > // Root-level validation tasks.
> > > > > > > dependsOn ":verifyLocks"
> > > > > > > dependsOn ":versionsPropsAreSorted"
> > > > > > > dependsOn ":checkWorkingCopyClean"
> > > > > > >
> > > > > > > What is the "verifyLocks" task? I can't seem to find it defined anywhere
> > > > > > >
> > > > > > > -Mike
> > > > > > >
> > > > > > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > > >
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: dev-help@lucene.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Gradle HELP [ In reply to ]
Thanks for your sympathy :) I am told the situation here should
improve soon... After working around the security policy I am able to
run some tests, but I still have some dependency issues around junit
methods -- I think maybe it's some issue w/hamcrest? What I see is
most methods get resolved correctly, but eg assertThrows does not.

I see we have this tricksy logic in test-framework's build.gradle:

- api ("com.carrotsearch.randomizedtesting:randomizedtesting-runner",
{
- exclude group: "junit"
- })
- api ("junit:junit", {
- exclude group: "org.hamcrest"
- })
- api ('org.hamcrest:hamcrest')

and I wonder why we need these exclusions?


On Thu, Aug 19, 2021 at 8:44 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>
> Ok, clear. I understand the reasons these things are shielded from the
> internet. I'm sure gradle folks have thought about such scenarios
> though - gradle is used across enterprises with similar setups? I
> really know nothing about it but I feel pretty confident artifact
> resolution can be redirected to only known locations. Sorry you have
> to hack around!
>
> Dawid
>
> On Thu, Aug 19, 2021 at 2:00 PM Michael Sokolov <msokolov@gmail.com> wrote:
> >
> > > This seems a tad insane?
> >
> > Agree, and yes it's an Amazon thing. Basically our build servers are
> > not connected to the internet, and we use an internal repository and
> > corresponding gradle plugin for resolving dependencies that is
> > somewhat idiosyncratic (not Maven or anything else familiar, but acts
> > kind of like Maven repo). Uwe, I am basically doing what you say,
> > re-using as much as I can from our excellent Lucene build system, but
> > wrapping alone is insufficient: I have to make invasive changes here
> > and there. Believe me this will be a big improvement from the hacked
> > ant build we have been using.
> >
> > On Thu, Aug 19, 2021 at 5:00 AM Uwe Schindler <uwe@thetaphi.de> wrote:
> > >
> > > Hi,
> > >
> > > would you give us some hints what you are trying to do? Is this related to the Lucene build system and are you preparing a PR or is this some Amazon-only rewrite? In the latter case, can't you not wrap the execution of gradlew in your "internal" build system? You can also configure gradle to fetch dependencies from other repositories than maven central.
> > >
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > Achterdiek 19, D-28357 Bremen
> > > https://www.thetaphi.de
> > > eMail: uwe@thetaphi.de
> > >
> > > > -----Original Message-----
> > > > From: Michael Sokolov <msokolov@gmail.com>
> > > > Sent: Thursday, August 19, 2021 5:18 AM
> > > > To: Lucene Dev <dev@lucene.apache.org>
> > > > Subject: Re: Gradle HELP
> > > >
> > > > OK I see it's because this build system puts the output in a different
> > > > folder that is not covered by the security policy. phew.
> > > >
> > > > On Wed, Aug 18, 2021 at 10:30 PM Michael Sokolov <msokolov@gmail.com>
> > > > wrote:
> > > > >
> > > > > Update: running tests with -Dtests.useSecurityManager=false got over
> > > > > that hump. Not sure why the security manager would be behaving
> > > > > differently in this setup...
> > > > >
> > > > > On Wed, Aug 18, 2021 at 6:41 PM Michael Sokolov <msokolov@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > ah interesting. I may have broken it in the midst of debugging
> > > > > > something else. I did manage to get most of the module jars built, and
> > > > > > I'm moving on to trying to get tests to run. I'm running into what
> > > > > > looks like more dependency/classpath resolution problems, but with our
> > > > > > module-module dependencies. If anyone has a clue what might be going
> > > > > > on or can suggest some futher debugging tips, I'd be very
> > > > > > appreciative. Here are the gory details...
> > > > > >
> > > > > >
> > > > > > For example, tests in o.a.l.codecs are failing because:
> > > > > >
> > > > > > org.apache.lucene.codecs.uniformsplit.TestUniformSplitPostingFormat >
> > > > > > classMethod FAILED
> > > > > > java.lang.ExceptionInInitializerError
> > > > > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > > > > > at
> > > > org.apache.lucene.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetup
> > > > AndRestoreClassEnv.java:133)
> > > > > > ...
> > > > > > java.lang.IllegalArgumentException: An SPI class of type
> > > > > > org.apache.lucene.codecs.Codec with name 'Lucene90' does not exist.
> > > > > > You need to add the corresponding JAR file supporting this SPI to your
> > > > > > classpath. The current classpath supports the following names:
> > > > > > [SimpleText]
> > > > > >
> > > > > > and I print out the classpath used to run the test by:
> > > > > >
> > > > > > tasks.withType(Test) {
> > > > > > doFirst {
> > > > > > println "Test $allJvmArgs"
> > > > > > println "Test classpath " + classpath.getFiles().toString()
> > > > > > }
> > > > > > }
> > > > > >
> > > > > > and the classpath includes
> > > > > >
> > > > > > ...build/lib/lucene-core-9.0.jar
> > > > > >
> > > > > > and that jar includes
> > > > > >
> > > > > > -rw-r--r-- 843 18-Aug-2021 21:40:50
> > > > > > META-INF/services/org.apache.lucene.codecs.Codec
> > > > > >
> > > > > > which has the expected content -- so I am kind of confused. Are these
> > > > > > Codec files supposed to be merged together? Or is this a conflict
> > > > > > resolved by classpath-order?
> > > > > >
> > > > > > and there are other dependency issues, but I think it would just be
> > > > > > more confusing to list them all here.
> > > > > >
> > > > > > PS: full listing below:
> > > > > >
> > > > > > Test classpath
> > > > [/local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> > > > ses/java/test,
> > > > > >
> > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> > > > urces/test,
> > > > > >
> > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> > > > ses/java/main,
> > > > > >
> > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> > > > urces/main,
> > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-test-
> > > > framework-9.0_az.jar,
> > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-
> > > > codecs-9.0_az.jar,
> > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-core-
> > > > 9.0_az.jar,
> > > > > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> > > > 2/brazil/junit/4.12/junit-4.12.jar,
> > > > > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> > > > 2/brazil/Hamcrest/1.3/Hamcrest-1.3.jar,
> > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > 2.1/com.carrotsearch.randomizedtesting/randomizedtesting-
> > > > runner/2.7.2/84ed6b5f70906f9ef173db67ccde657b43ea60df/randomizedtesti
> > > > ng-runner-2.7.2.jar,
> > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > 2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-
> > > > 4.12.jar,
> > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > 2.1/org.hamcrest/hamcrest-
> > > > core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-
> > > > 1.3.jar,
> > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > 2.1/org.hamcrest/java-
> > > > hamcrest/2.0.0.0/f1c8853ade0ecf707f5a261c830e98893983813/java-
> > > > hamcrest-2.0.0.0.jar]
> > > > > >
> > > > > > On Wed, Aug 18, 2021 at 4:30 PM Mike Drob <mdrob@mdrob.com> wrote:
> > > > > > >
> > > > > > > I believe it is one of the built in gradle tasks available to us.
> > > > > > > https://docs.gradle.org/current/userguide/dependency_locking.html
> > > > > > >
> > > > > > > On Wed, Aug 18, 2021 at 3:22 PM Michael Sokolov
> > > > <msokolov@gmail.com> wrote:
> > > > > > > >
> > > > > > > > I am trying to get Lucene to build in a very "special" build system
> > > > > > > > based on Gradle, and I am stumbling across all kinds of things I don't
> > > > > > > > really understand (which is most of everything about gradle at this
> > > > > > > > point). Cookie cutter cargo cultism is getting me pretty far, but one
> > > > > > > > thing I don't understand: we have this in the precommit task:
> > > > > > > >
> > > > > > > > // Root-level validation tasks.
> > > > > > > > dependsOn ":verifyLocks"
> > > > > > > > dependsOn ":versionsPropsAreSorted"
> > > > > > > > dependsOn ":checkWorkingCopyClean"
> > > > > > > >
> > > > > > > > What is the "verifyLocks" task? I can't seem to find it defined anywhere
> > > > > > > >
> > > > > > > > -Mike
> > > > > > > >
> > > > > > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > > > >
> > > > > > >
> > > > > > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: dev-help@lucene.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Gradle HELP [ In reply to ]
ah nvm I see it is the wrong version of junit - I had 4.12 and need to
get to 4.13

On Thu, Aug 19, 2021 at 4:21 PM Michael Sokolov <msokolov@gmail.com> wrote:
>
> Thanks for your sympathy :) I am told the situation here should
> improve soon... After working around the security policy I am able to
> run some tests, but I still have some dependency issues around junit
> methods -- I think maybe it's some issue w/hamcrest? What I see is
> most methods get resolved correctly, but eg assertThrows does not.
>
> I see we have this tricksy logic in test-framework's build.gradle:
>
> - api ("com.carrotsearch.randomizedtesting:randomizedtesting-runner",
> {
> - exclude group: "junit"
> - })
> - api ("junit:junit", {
> - exclude group: "org.hamcrest"
> - })
> - api ('org.hamcrest:hamcrest')
>
> and I wonder why we need these exclusions?
>
>
> On Thu, Aug 19, 2021 at 8:44 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> >
> > Ok, clear. I understand the reasons these things are shielded from the
> > internet. I'm sure gradle folks have thought about such scenarios
> > though - gradle is used across enterprises with similar setups? I
> > really know nothing about it but I feel pretty confident artifact
> > resolution can be redirected to only known locations. Sorry you have
> > to hack around!
> >
> > Dawid
> >
> > On Thu, Aug 19, 2021 at 2:00 PM Michael Sokolov <msokolov@gmail.com> wrote:
> > >
> > > > This seems a tad insane?
> > >
> > > Agree, and yes it's an Amazon thing. Basically our build servers are
> > > not connected to the internet, and we use an internal repository and
> > > corresponding gradle plugin for resolving dependencies that is
> > > somewhat idiosyncratic (not Maven or anything else familiar, but acts
> > > kind of like Maven repo). Uwe, I am basically doing what you say,
> > > re-using as much as I can from our excellent Lucene build system, but
> > > wrapping alone is insufficient: I have to make invasive changes here
> > > and there. Believe me this will be a big improvement from the hacked
> > > ant build we have been using.
> > >
> > > On Thu, Aug 19, 2021 at 5:00 AM Uwe Schindler <uwe@thetaphi.de> wrote:
> > > >
> > > > Hi,
> > > >
> > > > would you give us some hints what you are trying to do? Is this related to the Lucene build system and are you preparing a PR or is this some Amazon-only rewrite? In the latter case, can't you not wrap the execution of gradlew in your "internal" build system? You can also configure gradle to fetch dependencies from other repositories than maven central.
> > > >
> > > > Uwe
> > > >
> > > > -----
> > > > Uwe Schindler
> > > > Achterdiek 19, D-28357 Bremen
> > > > https://www.thetaphi.de
> > > > eMail: uwe@thetaphi.de
> > > >
> > > > > -----Original Message-----
> > > > > From: Michael Sokolov <msokolov@gmail.com>
> > > > > Sent: Thursday, August 19, 2021 5:18 AM
> > > > > To: Lucene Dev <dev@lucene.apache.org>
> > > > > Subject: Re: Gradle HELP
> > > > >
> > > > > OK I see it's because this build system puts the output in a different
> > > > > folder that is not covered by the security policy. phew.
> > > > >
> > > > > On Wed, Aug 18, 2021 at 10:30 PM Michael Sokolov <msokolov@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > Update: running tests with -Dtests.useSecurityManager=false got over
> > > > > > that hump. Not sure why the security manager would be behaving
> > > > > > differently in this setup...
> > > > > >
> > > > > > On Wed, Aug 18, 2021 at 6:41 PM Michael Sokolov <msokolov@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > ah interesting. I may have broken it in the midst of debugging
> > > > > > > something else. I did manage to get most of the module jars built, and
> > > > > > > I'm moving on to trying to get tests to run. I'm running into what
> > > > > > > looks like more dependency/classpath resolution problems, but with our
> > > > > > > module-module dependencies. If anyone has a clue what might be going
> > > > > > > on or can suggest some futher debugging tips, I'd be very
> > > > > > > appreciative. Here are the gory details...
> > > > > > >
> > > > > > >
> > > > > > > For example, tests in o.a.l.codecs are failing because:
> > > > > > >
> > > > > > > org.apache.lucene.codecs.uniformsplit.TestUniformSplitPostingFormat >
> > > > > > > classMethod FAILED
> > > > > > > java.lang.ExceptionInInitializerError
> > > > > > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > > > > > > at
> > > > > org.apache.lucene.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetup
> > > > > AndRestoreClassEnv.java:133)
> > > > > > > ...
> > > > > > > java.lang.IllegalArgumentException: An SPI class of type
> > > > > > > org.apache.lucene.codecs.Codec with name 'Lucene90' does not exist.
> > > > > > > You need to add the corresponding JAR file supporting this SPI to your
> > > > > > > classpath. The current classpath supports the following names:
> > > > > > > [SimpleText]
> > > > > > >
> > > > > > > and I print out the classpath used to run the test by:
> > > > > > >
> > > > > > > tasks.withType(Test) {
> > > > > > > doFirst {
> > > > > > > println "Test $allJvmArgs"
> > > > > > > println "Test classpath " + classpath.getFiles().toString()
> > > > > > > }
> > > > > > > }
> > > > > > >
> > > > > > > and the classpath includes
> > > > > > >
> > > > > > > ...build/lib/lucene-core-9.0.jar
> > > > > > >
> > > > > > > and that jar includes
> > > > > > >
> > > > > > > -rw-r--r-- 843 18-Aug-2021 21:40:50
> > > > > > > META-INF/services/org.apache.lucene.codecs.Codec
> > > > > > >
> > > > > > > which has the expected content -- so I am kind of confused. Are these
> > > > > > > Codec files supposed to be merged together? Or is this a conflict
> > > > > > > resolved by classpath-order?
> > > > > > >
> > > > > > > and there are other dependency issues, but I think it would just be
> > > > > > > more confusing to list them all here.
> > > > > > >
> > > > > > > PS: full listing below:
> > > > > > >
> > > > > > > Test classpath
> > > > > [/local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> > > > > ses/java/test,
> > > > > > >
> > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> > > > > urces/test,
> > > > > > >
> > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> > > > > ses/java/main,
> > > > > > >
> > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> > > > > urces/main,
> > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-test-
> > > > > framework-9.0_az.jar,
> > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-
> > > > > codecs-9.0_az.jar,
> > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-core-
> > > > > 9.0_az.jar,
> > > > > > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> > > > > 2/brazil/junit/4.12/junit-4.12.jar,
> > > > > > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> > > > > 2/brazil/Hamcrest/1.3/Hamcrest-1.3.jar,
> > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > 2.1/com.carrotsearch.randomizedtesting/randomizedtesting-
> > > > > runner/2.7.2/84ed6b5f70906f9ef173db67ccde657b43ea60df/randomizedtesti
> > > > > ng-runner-2.7.2.jar,
> > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > 2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-
> > > > > 4.12.jar,
> > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > 2.1/org.hamcrest/hamcrest-
> > > > > core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-
> > > > > 1.3.jar,
> > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > 2.1/org.hamcrest/java-
> > > > > hamcrest/2.0.0.0/f1c8853ade0ecf707f5a261c830e98893983813/java-
> > > > > hamcrest-2.0.0.0.jar]
> > > > > > >
> > > > > > > On Wed, Aug 18, 2021 at 4:30 PM Mike Drob <mdrob@mdrob.com> wrote:
> > > > > > > >
> > > > > > > > I believe it is one of the built in gradle tasks available to us.
> > > > > > > > https://docs.gradle.org/current/userguide/dependency_locking.html
> > > > > > > >
> > > > > > > > On Wed, Aug 18, 2021 at 3:22 PM Michael Sokolov
> > > > > <msokolov@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > I am trying to get Lucene to build in a very "special" build system
> > > > > > > > > based on Gradle, and I am stumbling across all kinds of things I don't
> > > > > > > > > really understand (which is most of everything about gradle at this
> > > > > > > > > point). Cookie cutter cargo cultism is getting me pretty far, but one
> > > > > > > > > thing I don't understand: we have this in the precommit task:
> > > > > > > > >
> > > > > > > > > // Root-level validation tasks.
> > > > > > > > > dependsOn ":verifyLocks"
> > > > > > > > > dependsOn ":versionsPropsAreSorted"
> > > > > > > > > dependsOn ":checkWorkingCopyClean"
> > > > > > > > >
> > > > > > > > > What is the "verifyLocks" task? I can't seem to find it defined anywhere
> > > > > > > > >
> > > > > > > > > -Mike
> > > > > > > > >
> > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > > > > >
> > > > > > > >
> > > > > > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: dev-help@lucene.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Gradle HELP [ In reply to ]
In the midst of all this hackery I have been using Gradle 6.9 and it
seems to have support for Java 15 - should we switch? Might be easier
than going to 7

On Thu, Aug 19, 2021 at 4:26 PM Michael Sokolov <msokolov@gmail.com> wrote:
>
> ah nvm I see it is the wrong version of junit - I had 4.12 and need to
> get to 4.13
>
> On Thu, Aug 19, 2021 at 4:21 PM Michael Sokolov <msokolov@gmail.com> wrote:
> >
> > Thanks for your sympathy :) I am told the situation here should
> > improve soon... After working around the security policy I am able to
> > run some tests, but I still have some dependency issues around junit
> > methods -- I think maybe it's some issue w/hamcrest? What I see is
> > most methods get resolved correctly, but eg assertThrows does not.
> >
> > I see we have this tricksy logic in test-framework's build.gradle:
> >
> > - api ("com.carrotsearch.randomizedtesting:randomizedtesting-runner",
> > {
> > - exclude group: "junit"
> > - })
> > - api ("junit:junit", {
> > - exclude group: "org.hamcrest"
> > - })
> > - api ('org.hamcrest:hamcrest')
> >
> > and I wonder why we need these exclusions?
> >
> >
> > On Thu, Aug 19, 2021 at 8:44 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> > >
> > > Ok, clear. I understand the reasons these things are shielded from the
> > > internet. I'm sure gradle folks have thought about such scenarios
> > > though - gradle is used across enterprises with similar setups? I
> > > really know nothing about it but I feel pretty confident artifact
> > > resolution can be redirected to only known locations. Sorry you have
> > > to hack around!
> > >
> > > Dawid
> > >
> > > On Thu, Aug 19, 2021 at 2:00 PM Michael Sokolov <msokolov@gmail.com> wrote:
> > > >
> > > > > This seems a tad insane?
> > > >
> > > > Agree, and yes it's an Amazon thing. Basically our build servers are
> > > > not connected to the internet, and we use an internal repository and
> > > > corresponding gradle plugin for resolving dependencies that is
> > > > somewhat idiosyncratic (not Maven or anything else familiar, but acts
> > > > kind of like Maven repo). Uwe, I am basically doing what you say,
> > > > re-using as much as I can from our excellent Lucene build system, but
> > > > wrapping alone is insufficient: I have to make invasive changes here
> > > > and there. Believe me this will be a big improvement from the hacked
> > > > ant build we have been using.
> > > >
> > > > On Thu, Aug 19, 2021 at 5:00 AM Uwe Schindler <uwe@thetaphi.de> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > would you give us some hints what you are trying to do? Is this related to the Lucene build system and are you preparing a PR or is this some Amazon-only rewrite? In the latter case, can't you not wrap the execution of gradlew in your "internal" build system? You can also configure gradle to fetch dependencies from other repositories than maven central.
> > > > >
> > > > > Uwe
> > > > >
> > > > > -----
> > > > > Uwe Schindler
> > > > > Achterdiek 19, D-28357 Bremen
> > > > > https://www.thetaphi.de
> > > > > eMail: uwe@thetaphi.de
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Michael Sokolov <msokolov@gmail.com>
> > > > > > Sent: Thursday, August 19, 2021 5:18 AM
> > > > > > To: Lucene Dev <dev@lucene.apache.org>
> > > > > > Subject: Re: Gradle HELP
> > > > > >
> > > > > > OK I see it's because this build system puts the output in a different
> > > > > > folder that is not covered by the security policy. phew.
> > > > > >
> > > > > > On Wed, Aug 18, 2021 at 10:30 PM Michael Sokolov <msokolov@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > Update: running tests with -Dtests.useSecurityManager=false got over
> > > > > > > that hump. Not sure why the security manager would be behaving
> > > > > > > differently in this setup...
> > > > > > >
> > > > > > > On Wed, Aug 18, 2021 at 6:41 PM Michael Sokolov <msokolov@gmail.com>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > ah interesting. I may have broken it in the midst of debugging
> > > > > > > > something else. I did manage to get most of the module jars built, and
> > > > > > > > I'm moving on to trying to get tests to run. I'm running into what
> > > > > > > > looks like more dependency/classpath resolution problems, but with our
> > > > > > > > module-module dependencies. If anyone has a clue what might be going
> > > > > > > > on or can suggest some futher debugging tips, I'd be very
> > > > > > > > appreciative. Here are the gory details...
> > > > > > > >
> > > > > > > >
> > > > > > > > For example, tests in o.a.l.codecs are failing because:
> > > > > > > >
> > > > > > > > org.apache.lucene.codecs.uniformsplit.TestUniformSplitPostingFormat >
> > > > > > > > classMethod FAILED
> > > > > > > > java.lang.ExceptionInInitializerError
> > > > > > > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > > > > > > > at
> > > > > > org.apache.lucene.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetup
> > > > > > AndRestoreClassEnv.java:133)
> > > > > > > > ...
> > > > > > > > java.lang.IllegalArgumentException: An SPI class of type
> > > > > > > > org.apache.lucene.codecs.Codec with name 'Lucene90' does not exist.
> > > > > > > > You need to add the corresponding JAR file supporting this SPI to your
> > > > > > > > classpath. The current classpath supports the following names:
> > > > > > > > [SimpleText]
> > > > > > > >
> > > > > > > > and I print out the classpath used to run the test by:
> > > > > > > >
> > > > > > > > tasks.withType(Test) {
> > > > > > > > doFirst {
> > > > > > > > println "Test $allJvmArgs"
> > > > > > > > println "Test classpath " + classpath.getFiles().toString()
> > > > > > > > }
> > > > > > > > }
> > > > > > > >
> > > > > > > > and the classpath includes
> > > > > > > >
> > > > > > > > ...build/lib/lucene-core-9.0.jar
> > > > > > > >
> > > > > > > > and that jar includes
> > > > > > > >
> > > > > > > > -rw-r--r-- 843 18-Aug-2021 21:40:50
> > > > > > > > META-INF/services/org.apache.lucene.codecs.Codec
> > > > > > > >
> > > > > > > > which has the expected content -- so I am kind of confused. Are these
> > > > > > > > Codec files supposed to be merged together? Or is this a conflict
> > > > > > > > resolved by classpath-order?
> > > > > > > >
> > > > > > > > and there are other dependency issues, but I think it would just be
> > > > > > > > more confusing to list them all here.
> > > > > > > >
> > > > > > > > PS: full listing below:
> > > > > > > >
> > > > > > > > Test classpath
> > > > > > [/local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> > > > > > ses/java/test,
> > > > > > > >
> > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> > > > > > urces/test,
> > > > > > > >
> > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> > > > > > ses/java/main,
> > > > > > > >
> > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> > > > > > urces/main,
> > > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-test-
> > > > > > framework-9.0_az.jar,
> > > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-
> > > > > > codecs-9.0_az.jar,
> > > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-core-
> > > > > > 9.0_az.jar,
> > > > > > > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> > > > > > 2/brazil/junit/4.12/junit-4.12.jar,
> > > > > > > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> > > > > > 2/brazil/Hamcrest/1.3/Hamcrest-1.3.jar,
> > > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > > 2.1/com.carrotsearch.randomizedtesting/randomizedtesting-
> > > > > > runner/2.7.2/84ed6b5f70906f9ef173db67ccde657b43ea60df/randomizedtesti
> > > > > > ng-runner-2.7.2.jar,
> > > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > > 2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-
> > > > > > 4.12.jar,
> > > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > > 2.1/org.hamcrest/hamcrest-
> > > > > > core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-
> > > > > > 1.3.jar,
> > > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > > 2.1/org.hamcrest/java-
> > > > > > hamcrest/2.0.0.0/f1c8853ade0ecf707f5a261c830e98893983813/java-
> > > > > > hamcrest-2.0.0.0.jar]
> > > > > > > >
> > > > > > > > On Wed, Aug 18, 2021 at 4:30 PM Mike Drob <mdrob@mdrob.com> wrote:
> > > > > > > > >
> > > > > > > > > I believe it is one of the built in gradle tasks available to us.
> > > > > > > > > https://docs.gradle.org/current/userguide/dependency_locking.html
> > > > > > > > >
> > > > > > > > > On Wed, Aug 18, 2021 at 3:22 PM Michael Sokolov
> > > > > > <msokolov@gmail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > I am trying to get Lucene to build in a very "special" build system
> > > > > > > > > > based on Gradle, and I am stumbling across all kinds of things I don't
> > > > > > > > > > really understand (which is most of everything about gradle at this
> > > > > > > > > > point). Cookie cutter cargo cultism is getting me pretty far, but one
> > > > > > > > > > thing I don't understand: we have this in the precommit task:
> > > > > > > > > >
> > > > > > > > > > // Root-level validation tasks.
> > > > > > > > > > dependsOn ":verifyLocks"
> > > > > > > > > > dependsOn ":versionsPropsAreSorted"
> > > > > > > > > > dependsOn ":checkWorkingCopyClean"
> > > > > > > > > >
> > > > > > > > > > What is the "verifyLocks" task? I can't seem to find it defined anywhere
> > > > > > > > > >
> > > > > > > > > > -Mike
> > > > > > > > > >
> > > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > > > > >
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: dev-help@lucene.apache.org
> > >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Gradle HELP [ In reply to ]
I'm actually looking into switching to gradle 7... The only
problematic part I see so far is jvm registries. Don't know how to
solve it yet. They've changed a lot and I need to figure out whether
it's possible to somehow keep the existing behavior or if we need to
adopt to the new conventions (I'd rather stick to what we're used to).

Dawid

On Fri, Aug 20, 2021 at 6:13 PM Michael Sokolov <msokolov@gmail.com> wrote:
>
> In the midst of all this hackery I have been using Gradle 6.9 and it
> seems to have support for Java 15 - should we switch? Might be easier
> than going to 7
>
> On Thu, Aug 19, 2021 at 4:26 PM Michael Sokolov <msokolov@gmail.com> wrote:
> >
> > ah nvm I see it is the wrong version of junit - I had 4.12 and need to
> > get to 4.13
> >
> > On Thu, Aug 19, 2021 at 4:21 PM Michael Sokolov <msokolov@gmail.com> wrote:
> > >
> > > Thanks for your sympathy :) I am told the situation here should
> > > improve soon... After working around the security policy I am able to
> > > run some tests, but I still have some dependency issues around junit
> > > methods -- I think maybe it's some issue w/hamcrest? What I see is
> > > most methods get resolved correctly, but eg assertThrows does not.
> > >
> > > I see we have this tricksy logic in test-framework's build.gradle:
> > >
> > > - api ("com.carrotsearch.randomizedtesting:randomizedtesting-runner",
> > > {
> > > - exclude group: "junit"
> > > - })
> > > - api ("junit:junit", {
> > > - exclude group: "org.hamcrest"
> > > - })
> > > - api ('org.hamcrest:hamcrest')
> > >
> > > and I wonder why we need these exclusions?
> > >
> > >
> > > On Thu, Aug 19, 2021 at 8:44 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> > > >
> > > > Ok, clear. I understand the reasons these things are shielded from the
> > > > internet. I'm sure gradle folks have thought about such scenarios
> > > > though - gradle is used across enterprises with similar setups? I
> > > > really know nothing about it but I feel pretty confident artifact
> > > > resolution can be redirected to only known locations. Sorry you have
> > > > to hack around!
> > > >
> > > > Dawid
> > > >
> > > > On Thu, Aug 19, 2021 at 2:00 PM Michael Sokolov <msokolov@gmail.com> wrote:
> > > > >
> > > > > > This seems a tad insane?
> > > > >
> > > > > Agree, and yes it's an Amazon thing. Basically our build servers are
> > > > > not connected to the internet, and we use an internal repository and
> > > > > corresponding gradle plugin for resolving dependencies that is
> > > > > somewhat idiosyncratic (not Maven or anything else familiar, but acts
> > > > > kind of like Maven repo). Uwe, I am basically doing what you say,
> > > > > re-using as much as I can from our excellent Lucene build system, but
> > > > > wrapping alone is insufficient: I have to make invasive changes here
> > > > > and there. Believe me this will be a big improvement from the hacked
> > > > > ant build we have been using.
> > > > >
> > > > > On Thu, Aug 19, 2021 at 5:00 AM Uwe Schindler <uwe@thetaphi.de> wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > would you give us some hints what you are trying to do? Is this related to the Lucene build system and are you preparing a PR or is this some Amazon-only rewrite? In the latter case, can't you not wrap the execution of gradlew in your "internal" build system? You can also configure gradle to fetch dependencies from other repositories than maven central.
> > > > > >
> > > > > > Uwe
> > > > > >
> > > > > > -----
> > > > > > Uwe Schindler
> > > > > > Achterdiek 19, D-28357 Bremen
> > > > > > https://www.thetaphi.de
> > > > > > eMail: uwe@thetaphi.de
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Michael Sokolov <msokolov@gmail.com>
> > > > > > > Sent: Thursday, August 19, 2021 5:18 AM
> > > > > > > To: Lucene Dev <dev@lucene.apache.org>
> > > > > > > Subject: Re: Gradle HELP
> > > > > > >
> > > > > > > OK I see it's because this build system puts the output in a different
> > > > > > > folder that is not covered by the security policy. phew.
> > > > > > >
> > > > > > > On Wed, Aug 18, 2021 at 10:30 PM Michael Sokolov <msokolov@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > Update: running tests with -Dtests.useSecurityManager=false got over
> > > > > > > > that hump. Not sure why the security manager would be behaving
> > > > > > > > differently in this setup...
> > > > > > > >
> > > > > > > > On Wed, Aug 18, 2021 at 6:41 PM Michael Sokolov <msokolov@gmail.com>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > ah interesting. I may have broken it in the midst of debugging
> > > > > > > > > something else. I did manage to get most of the module jars built, and
> > > > > > > > > I'm moving on to trying to get tests to run. I'm running into what
> > > > > > > > > looks like more dependency/classpath resolution problems, but with our
> > > > > > > > > module-module dependencies. If anyone has a clue what might be going
> > > > > > > > > on or can suggest some futher debugging tips, I'd be very
> > > > > > > > > appreciative. Here are the gory details...
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > For example, tests in o.a.l.codecs are failing because:
> > > > > > > > >
> > > > > > > > > org.apache.lucene.codecs.uniformsplit.TestUniformSplitPostingFormat >
> > > > > > > > > classMethod FAILED
> > > > > > > > > java.lang.ExceptionInInitializerError
> > > > > > > > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141)
> > > > > > > > > at
> > > > > > > org.apache.lucene.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetup
> > > > > > > AndRestoreClassEnv.java:133)
> > > > > > > > > ...
> > > > > > > > > java.lang.IllegalArgumentException: An SPI class of type
> > > > > > > > > org.apache.lucene.codecs.Codec with name 'Lucene90' does not exist.
> > > > > > > > > You need to add the corresponding JAR file supporting this SPI to your
> > > > > > > > > classpath. The current classpath supports the following names:
> > > > > > > > > [SimpleText]
> > > > > > > > >
> > > > > > > > > and I print out the classpath used to run the test by:
> > > > > > > > >
> > > > > > > > > tasks.withType(Test) {
> > > > > > > > > doFirst {
> > > > > > > > > println "Test $allJvmArgs"
> > > > > > > > > println "Test classpath " + classpath.getFiles().toString()
> > > > > > > > > }
> > > > > > > > > }
> > > > > > > > >
> > > > > > > > > and the classpath includes
> > > > > > > > >
> > > > > > > > > ...build/lib/lucene-core-9.0.jar
> > > > > > > > >
> > > > > > > > > and that jar includes
> > > > > > > > >
> > > > > > > > > -rw-r--r-- 843 18-Aug-2021 21:40:50
> > > > > > > > > META-INF/services/org.apache.lucene.codecs.Codec
> > > > > > > > >
> > > > > > > > > which has the expected content -- so I am kind of confused. Are these
> > > > > > > > > Codec files supposed to be merged together? Or is this a conflict
> > > > > > > > > resolved by classpath-order?
> > > > > > > > >
> > > > > > > > > and there are other dependency issues, but I think it would just be
> > > > > > > > > more confusing to list them all here.
> > > > > > > > >
> > > > > > > > > PS: full listing below:
> > > > > > > > >
> > > > > > > > > Test classpath
> > > > > > > [/local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> > > > > > > ses/java/test,
> > > > > > > > >
> > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> > > > > > > urces/test,
> > > > > > > > >
> > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/clas
> > > > > > > ses/java/main,
> > > > > > > > >
> > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/lucene/codecs/build/reso
> > > > > > > urces/main,
> > > > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-test-
> > > > > > > framework-9.0_az.jar,
> > > > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-
> > > > > > > codecs-9.0_az.jar,
> > > > > > > > > /local/home/sokolovm/workspace/lucene/src/Lucene/build/lib/lucene-core-
> > > > > > > 9.0_az.jar,
> > > > > > > > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> > > > > > > 2/brazil/junit/4.12/junit-4.12.jar,
> > > > > > > > > /local/home/sokolovm/workspace/lucene/env/gradle-cache-
> > > > > > > 2/brazil/Hamcrest/1.3/Hamcrest-1.3.jar,
> > > > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > > > 2.1/com.carrotsearch.randomizedtesting/randomizedtesting-
> > > > > > > runner/2.7.2/84ed6b5f70906f9ef173db67ccde657b43ea60df/randomizedtesti
> > > > > > > ng-runner-2.7.2.jar,
> > > > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > > > 2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-
> > > > > > > 4.12.jar,
> > > > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > > > 2.1/org.hamcrest/hamcrest-
> > > > > > > core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-
> > > > > > > 1.3.jar,
> > > > > > > > > /local/home/sokolovm/.gradle/caches/modules-2/files-
> > > > > > > 2.1/org.hamcrest/java-
> > > > > > > hamcrest/2.0.0.0/f1c8853ade0ecf707f5a261c830e98893983813/java-
> > > > > > > hamcrest-2.0.0.0.jar]
> > > > > > > > >
> > > > > > > > > On Wed, Aug 18, 2021 at 4:30 PM Mike Drob <mdrob@mdrob.com> wrote:
> > > > > > > > > >
> > > > > > > > > > I believe it is one of the built in gradle tasks available to us.
> > > > > > > > > > https://docs.gradle.org/current/userguide/dependency_locking.html
> > > > > > > > > >
> > > > > > > > > > On Wed, Aug 18, 2021 at 3:22 PM Michael Sokolov
> > > > > > > <msokolov@gmail.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > I am trying to get Lucene to build in a very "special" build system
> > > > > > > > > > > based on Gradle, and I am stumbling across all kinds of things I don't
> > > > > > > > > > > really understand (which is most of everything about gradle at this
> > > > > > > > > > > point). Cookie cutter cargo cultism is getting me pretty far, but one
> > > > > > > > > > > thing I don't understand: we have this in the precommit task:
> > > > > > > > > > >
> > > > > > > > > > > // Root-level validation tasks.
> > > > > > > > > > > dependsOn ":verifyLocks"
> > > > > > > > > > > dependsOn ":versionsPropsAreSorted"
> > > > > > > > > > > dependsOn ":checkWorkingCopyClean"
> > > > > > > > > > >
> > > > > > > > > > > What is the "verifyLocks" task? I can't seem to find it defined anywhere
> > > > > > > > > > >
> > > > > > > > > > > -Mike
> > > > > > > > > > >
> > > > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > > > > > >
> > > > > > >
> > > > > > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > >
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org