Mailing List Archive

Tests and module paths
Hello everyone,

Hope (some of) you had a good break.

I've placed a draft PR pushing module system compatibility changes a
bit further, the PR is here:

https://github.com/apache/lucene/pull/571

I essentially got to the point of making tests in most subprojects run
with dependencies (cross-project and external) on module-path. This
already revealed a few issues where the module system turns out to
behave differently (resource lookups) and I expect more of these to
appear in the future. It's a bit like two different worlds.

I had to make a few changes to the codebase and the reasons for these
changes will also apply in the future.

* I had to convert spatial test fixtures to a subproject. Gradle
plugins that add or modify source sets and their paths will be a
problem. They are basically an orthogonal dimension that complicates
(or even makes it impossible) to assemble the module path correctly. I
gave up on trying to make the test fixtures plugin working.

* There are very few patterns and best practices on how to work with
projects that have to run in both the classpath and modular world. A
good writeup on how one can organize tests, for example, was written
by Christian Stein (mentioned by Uwe previously) -

https://sormuras.github.io/blog/2018-09-11-testing-in-the-modular-world.html

However, I found the black-box automatisation (in both Maven and
gradle) only gets you to a certain point and then stops working - you
have to rewrite most of the stuff yourself anyway. All the different
tasks that use classpath (javac, javadoc, ecj, tests) will require the
same or, in many cases, different treatments for providing arguments
and none of the built-in transparent tweaks work across tools, IDEs,
etc. It quickly gets into a complicated mess of customized hacks.

So, what works in PR 571 and what doesn't?

The tests mostly work from the classpath, with project dependencies on
module-path if they're modular dependencies. This is best explained on
an example:

> gradlew -p lucene\analysis\morfologik -Pbuild.debug.paths=true test
...
Modular extension, runtime paths, mode=DEPENDENCIES_ON_MODULE_PATH:
Module path:
...\randomizedtesting-runner-2.7.6.jar
...\junit-4.13.1.jar
...\morfologik-fsa-2.1.8.jar
...\morfologik-polish-2.1.8.jar
...\morfologik-stemming-2.1.8.jar
...\hamcrest-2.2.jar
...\morfologik-ukrainian-search-4.9.1.jar
...\lucene\analysis\common\build\libs\lucene-analysis-common-10.0.0-SNAPSHOT.jar
...\lucene\codecs\build\libs\lucene-codecs-10.0.0-SNAPSHOT.jar
...\lucene\core\build\libs\lucene-core-10.0.0-SNAPSHOT.jar
...\lucene\test-framework\build\libs\lucene-test-framework-10.0.0-SNAPSHOT.jar
Class path:
...\lucene\analysis\morfologik\build\classes\java\main
...\lucene\analysis\morfologik\build\classes\java\test
...\lucene\analysis\morfologik\build\resources\main
...\analysis\morfologik\build\resources\test

I've removed some extra leading paths above but you can see that the
module path includes external dependencies (both named modules and
automatic modules) and cross-project dependencies (including the test
framework).

The split of classes and resources (a convention introduced by Maven)
and split-packages between tests and main classes of the project are
the reason why we have the main source set's and test source set's
artifacts on classpath:

...\lucene\analysis\morfologik\build\classes\java\main
...\lucene\analysis\morfologik\build\classes\java\test
...\lucene\analysis\morfologik\build\resources\main
...\lucene\analysis\morfologik\build\resources\test

While there are technically ways of combining these folders into a
module I found it too magical, at least for now. So PR 571 does not
attempt to solve this. Fully modular tests are possible but you have
to create a different subproject (with an extension '.tests' and then
reference the tested module from there. Obviously the test module name
and packages used have to be unique then. This is demonstrated on two
subprojects - core.tests and morfologik.tests. For example, compare
the printed paths for this:

> gradlew -p lucene\core.tests -Pbuild.debug.paths=true test

Modular extension, runtime paths, source set=test (module),
mode=DEPENDENCIES_ON_MODULE_PATH:
Module path:
...\junit-4.13.1.jar
...\hamcrest-2.2.jar
...\lucene\core.tests\build\libs\lucene-core.tests-10.0.0-SNAPSHOT-test.jar
...\lucene\core.tests\build\libs\lucene-core.tests-10.0.0-SNAPSHOT.jar
...\lucene\core\build\libs\lucene-core-10.0.0-SNAPSHOT.jar
Class path: [empty]

As can be seen, everything ended up on module-path. The background of
how this works is pretty ugly (to me) but in short terms, the source
set outputs are converted to JARs - the main source set would
correspond to lucene-core.tests-10.0.0-SNAPSHOT.jar and the test
source set is in lucene-core.tests-10.0.0-SNAPSHOT-test.jar.

It does work and it works in full module-mode.

What's remaining? Well, exploration. I've had some prototype work that
had module descriptors both in the main source set and in the test
source set - couldn't get it to work with IntelliJ (confused the heck
out of it). I also have an idea how module patching could work to
allow non-modular tests to work with a modular main source set... but
I didn't have time to try it yet and I have a strong suspicion it will
break IDEs again, so leaving it for now.

Finally, there are problems of the chicken-end-egg nature - the
following subprojects are explicitly using full classpath mode for
compilation/tests (modules.java):

":lucene:core",
":lucene:codecs",
":lucene:spatial3d",
":lucene:spatial-extras",
":lucene:test-framework",


these are there because of cyclic dependencies. For example, the tests
of the test framework are on classpath (including the test framework
itself) but the core (and test secrets accessors) are on module path -
this causes runtime module layer exceptions that test secrets cannot
be accessed from an unnamed module. Spatial modules use a
split-package dependency.

If anybody is willing to jump in, the PR contains the code. It's not
easy - it's touching quite a lot of gradle APIs - but it's fairly
self-contained in modules.java. Feel free to hack around and suggest
improvements, I'm slowly running out of ideas if it can be made any
easier/ better.

Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org