Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Lucene>
  3. Java-Dev
HEADS UP: Java 9 Module System (JMS) module names for Lucene artifacts
uwe at thetaphi
Nov 29, 2021, 3:04 PM
Post #1 of 5 (293 views)
Permalink
Hi,

I stopped the Lucene 9.0 release because of some inconsistencies with Java
Module System (JMS) module names. We will respin, but in preparation to full
module system support (in later Lucene 9.x versions), I changed the so
called "automatic module name" of all JAR artifacts so they are consistent
with naming conventions by the ASF and suggested by module developers at
OpenJDK and Maven people.

Long story:

There were already lengthy discussions on Maven and OpenJDK mailing list on
"how to name a module". If you define a module name though
"automatic-module-name" in the JAR manifest or by an explicit
module-info.java (see https://issues.apache.org/jira/browse/LUCENE-10255,
which is draft) the module name must be well thought. Christian Stein
(Member of the OpenJDK group and also Junit committer, also well involved in
development of Apache Maven) wrote some blog post about how a module name
should look like, so any code downstream can import it into their own
modules. The names must be valid Java identifiers and should be formatted
like package names:
https://sormuras.github.io/blog/2019-08-04-maven-coordinates-and-java-module
-names.html

It concludes this very well:
- The Java module name should have the Maven Group ID as prefix, followed by
"." and then a local module descriptor. E.g., "org.apache.lucene.core"
- The prefix of exported package names inside each module *should* be
prefixed by the module name (we can't do this for Lucene, but we should at
least share the same prefix: "org.apache.lucene").
- The version name inside the module should follow module system syntax (so
at least "9.0.0", no prefix/suffix => parseable by ModuleDescriptor.Version)

Here is a statistic of module names used on Maven by different artifacts,
have a look at examples like Log4J, Apache TIKA and others:
https://github.com/sormuras/modules/blob/main/doc/Top1000-2020.txt.md

For my detailed arguments see the discussion here (comments following this
one):
<https://issues.apache.org/jira/browse/LUCENE-10255?focusedCommentId=1745032
7&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comm
ent-17450327>

My proposal is to do the following before release, now implemented in
https://github.com/apache/lucene/pull/487:

> Task :showModuleNames
lucene-benchmark-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.benchmark
lucene-backward-codecs-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.backward_codecs
lucene-classification-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.classification
lucene-codecs-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.codecs
lucene-core-10.0.0-SNAPSHOT.jar -> org.apache.lucene.core
lucene-demo-10.0.0-SNAPSHOT.jar -> org.apache.lucene.demo
lucene-expressions-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.expressions
lucene-facet-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.facet
lucene-grouping-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.grouping
lucene-highlighter-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.highlighter
lucene-join-10.0.0-SNAPSHOT.jar -> org.apache.lucene.join
lucene-luke-10.0.0-SNAPSHOT.jar -> org.apache.lucene.luke
lucene-memory-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.memory
lucene-misc-10.0.0-SNAPSHOT.jar -> org.apache.lucene.misc
lucene-monitor-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.monitor
lucene-queries-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.queries
lucene-queryparser-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.queryparser
lucene-replicator-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.replicator
lucene-sandbox-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.sandbox
lucene-spatial-extras-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.spatial_extras
lucene-spatial3d-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.spatial3d
lucene-suggest-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.suggest
lucene-test-framework-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.test_framework
lucene-analysis-common-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.analysis.common
lucene-analysis-icu-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.analysis.icu
lucene-analysis-kuromoji-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.analysis.kuromoji
lucene-analysis-morfologik-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.analysis.morfologik
lucene-analysis-nori-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.analysis.nori
lucene-analysis-opennlp-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.analysis.opennlp
lucene-analysis-phonetic-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.analysis.phonetic
lucene-analysis-smartcn-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.analysis.smartcn
lucene-analysis-stempel-10.0.0-SNAPSHOT.jar ->
org.apache.lucene.analysis.stempel

These names can be used in your own "module-info.java" and Java will
automatically link Lucene's modules, which will enable you to write a module
based application with Java 11.

Later for 9.x we replace the automatic module name by a real
module-info.java (see PR: https://github.com/apache/lucene/pull/470) where
we may hide internal packages and other hidden stuff (e.g. the
analysis-common module may only export the SPI names and make all classes
hidden implementation details, so one can only reach the analysis components
using SPI names -- that's just an example, won't do this! Or the Codecs
module could hide all implementation details of codecs, analysis common may
not export the internal Snowball generated package,...)

Dawid will merge the PR tomorrow and we will respin. If you have any
complaints, speak up! It would also be interesting to hear from
Elasticsearch people who are on progress to move to modules.

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: HEADS UP: Java 9 Module System (JMS) module names for Lucene artifacts [ In reply to ]
christopher.hegarty at elastic
Nov 30, 2021, 2:50 AM
Post #2 of 5 (293 views)
Permalink
Hi Uwe,

Thank you very much for noticing and fixing this. I’m relatively new
here and certainly don’t get a vote or anything, but it is a big +1
from me for this change.

I'm actively investigaing the modularization of the Elasticsearch
platform and I noticed, when prototyping our module declarations, that
the lucene module names were less than ideal. Getting this fixed in
9.0.0 will allow us to leverage these stable module names in requires
directives (without needing to change them anything soon).

Thanks again,
-Chris.

> On 29 Nov 2021, at 23:04, Uwe Schindler <uwe@thetaphi.de> wrote:
>
> Hi,
>
> I stopped the Lucene 9.0 release because of some inconsistencies with Java
> Module System (JMS) module names. We will respin, but in preparation to full
> module system support (in later Lucene 9.x versions), I changed the so
> called "automatic module name" of all JAR artifacts so they are consistent
> with naming conventions by the ASF and suggested by module developers at
> OpenJDK and Maven people.
>
> Long story:
>
> There were already lengthy discussions on Maven and OpenJDK mailing list on
> "how to name a module". If you define a module name though
> "automatic-module-name" in the JAR manifest or by an explicit
> module-info.java (see https://issues.apache.org/jira/browse/LUCENE-10255,
> which is draft) the module name must be well thought. Christian Stein
> (Member of the OpenJDK group and also Junit committer, also well involved in
> development of Apache Maven) wrote some blog post about how a module name
> should look like, so any code downstream can import it into their own
> modules. The names must be valid Java identifiers and should be formatted
> like package names:
> https://sormuras.github.io/blog/2019-08-04-maven-coordinates-and-java-module
> -names.html
>
> It concludes this very well:
> - The Java module name should have the Maven Group ID as prefix, followed by
> "." and then a local module descriptor. E.g., "org.apache.lucene.core"
> - The prefix of exported package names inside each module *should* be
> prefixed by the module name (we can't do this for Lucene, but we should at
> least share the same prefix: "org.apache.lucene").
> - The version name inside the module should follow module system syntax (so
> at least "9.0.0", no prefix/suffix => parseable by ModuleDescriptor.Version)
>
> Here is a statistic of module names used on Maven by different artifacts,
> have a look at examples like Log4J, Apache TIKA and others:
> https://github.com/sormuras/modules/blob/main/doc/Top1000-2020.txt.md
>
> For my detailed arguments see the discussion here (comments following this
> one):
> <https://issues.apache.org/jira/browse/LUCENE-10255?focusedCommentId=1745032
> 7&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comm
> ent-17450327>
>
> My proposal is to do the following before release, now implemented in
> https://github.com/apache/lucene/pull/487:
>
>> Task :showModuleNames
> lucene-benchmark-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.benchmark
> lucene-backward-codecs-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.backward_codecs
> lucene-classification-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.classification
> lucene-codecs-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.codecs
> lucene-core-10.0.0-SNAPSHOT.jar -> org.apache.lucene.core
> lucene-demo-10.0.0-SNAPSHOT.jar -> org.apache.lucene.demo
> lucene-expressions-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.expressions
> lucene-facet-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.facet
> lucene-grouping-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.grouping
> lucene-highlighter-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.highlighter
> lucene-join-10.0.0-SNAPSHOT.jar -> org.apache.lucene.join
> lucene-luke-10.0.0-SNAPSHOT.jar -> org.apache.lucene.luke
> lucene-memory-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.memory
> lucene-misc-10.0.0-SNAPSHOT.jar -> org.apache.lucene.misc
> lucene-monitor-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.monitor
> lucene-queries-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.queries
> lucene-queryparser-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.queryparser
> lucene-replicator-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.replicator
> lucene-sandbox-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.sandbox
> lucene-spatial-extras-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.spatial_extras
> lucene-spatial3d-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.spatial3d
> lucene-suggest-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.suggest
> lucene-test-framework-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.test_framework
> lucene-analysis-common-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.analysis.common
> lucene-analysis-icu-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.analysis.icu
> lucene-analysis-kuromoji-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.analysis.kuromoji
> lucene-analysis-morfologik-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.analysis.morfologik
> lucene-analysis-nori-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.analysis.nori
> lucene-analysis-opennlp-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.analysis.opennlp
> lucene-analysis-phonetic-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.analysis.phonetic
> lucene-analysis-smartcn-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.analysis.smartcn
> lucene-analysis-stempel-10.0.0-SNAPSHOT.jar ->
> org.apache.lucene.analysis.stempel
>
> These names can be used in your own "module-info.java" and Java will
> automatically link Lucene's modules, which will enable you to write a module
> based application with Java 11.
>
> Later for 9.x we replace the automatic module name by a real
> module-info.java (see PR: https://github.com/apache/lucene/pull/470) where
> we may hide internal packages and other hidden stuff (e.g. the
> analysis-common module may only export the SPI names and make all classes
> hidden implementation details, so one can only reach the analysis components
> using SPI names -- that's just an example, won't do this! Or the Codecs
> module could hide all implementation details of codecs, analysis common may
> not export the internal Snowball generated package,...)
>
> Dawid will merge the PR tomorrow and we will respin. If you have any
> complaints, speak up! It would also be interesting to hear from
> Elasticsearch people who are on progress to move to modules.
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: HEADS UP: Java 9 Module System (JMS) module names for Lucene artifacts [ In reply to ]
dawid.weiss at gmail
Nov 30, 2021, 4:18 AM
Post #3 of 5 (293 views)
Permalink
>
> Getting this fixed in 9.0.0 will allow us to leverage these stable module
> names in requires
> directives (without needing to change them anything soon).
>

For the record - they are not considered stable in 9.0.0 -- see Uwe's note
below (although this was my original
intention for introducing automatic module names, even if the actual
compact naming has been so
fiercely criticized...).

LUCENE-10234: Added Automatic-Module-Name to all JARs. This is a first step
to enable full Java
module system (JMS) support in later Lucene versions. At the moment, the
automatic names should
not be considered stable. (Dawid Weiss, Uwe Schindler)

D.
RE: HEADS UP: Java 9 Module System (JMS) module names for Lucene artifacts [ In reply to ]
uwe at thetaphi
Nov 30, 2021, 5:01 AM
Post #4 of 5 (293 views)
Permalink
Hi Chris,



yes they are not declared “stable” for 9.0 as those are just “automatic module names” assigned through manifest based on the gradle module name only. But I would say they are still more “stable” than anything we had in 8.x where the module system guesses just something from JAR file name.



With the statement I wanted to say: We try to keep the names as noted in my previous mail when we will switch to full-featured module system with module-info.java, all SPIs declared, exports of packages,… (https://issues.apache.org/jira/browse/LUCENE-10255). But there may be the requirement to refactor and rename a module.



As you were also involved with the JMS @ Oracle before you started at Elasticsearch, you may also give some comments about the naming issue. To my understanding the new and almost stable module names are safe to be released as auto-modules in Lucene 9.0.



Uwe



P.S.: Not sure with which version you are testing Elasticsearch. The automodules were added 2 weeks ago and yesterday we renamed them to have fully-qualified domain names. We will merge the PR later and backport to 9.0 and 9.x branch.



-----

Uwe Schindler

Achterdiek 19, D-28357 Bremen

<https://www.thetaphi.de> https://www.thetaphi.de

eMail: uwe@thetaphi.de



From: Dawid Weiss <dawid.weiss@gmail.com>
Sent: Tuesday, November 30, 2021 1:19 PM
To: Lucene Dev <dev@lucene.apache.org>
Subject: Re: HEADS UP: Java 9 Module System (JMS) module names for Lucene artifacts



Getting this fixed in 9.0.0 will allow us to leverage these stable module names in requires
directives (without needing to change them anything soon).



For the record - they are not considered stable in 9.0.0 -- see Uwe's note below (although this was my original

intention for introducing automatic module names, even if the actual compact naming has been so

fiercely criticized...).



LUCENE-10234: Added Automatic-Module-Name to all JARs. This is a first step to enable full Java
module system (JMS) support in later Lucene versions. At the moment, the automatic names should
not be considered stable. (Dawid Weiss, Uwe Schindler)



D.
Re: HEADS UP: Java 9 Module System (JMS) module names for Lucene artifacts [ In reply to ]
christopher.hegarty at elastic
Nov 30, 2021, 9:31 AM
Post #5 of 5 (293 views)
Permalink
Hi Uwe,

> On 30 Nov 2021, at 13:01, Uwe Schindler <uwe@thetaphi.de> wrote:
>
> Hi Chris,
>
> yes they are not declared “stable” for 9.0 as those are just “automatic module names” assigned through manifest based on the gradle module name only. But I would say they are still more “stable” than anything we had in 8.x where the module system guesses just something from JAR file name.

Right - “more stable”, which is fine.

> With the statement I wanted to say: We try to keep the names as noted in my previous mail when we will switch to full-featured module system with module-info.java, all SPIs declared, exports of packages,… (https://issues.apache.org/jira/browse/LUCENE-10255 <https://issues.apache.org/jira/browse/LUCENE-10255>). But there may be the requirement to refactor and rename a module.

That sounds reasonable.

> As you were also involved with the JMS @ Oracle before you started at Elasticsearch, you may also give some comments about the naming issue. To my understanding the new and almost stable module names are safe to be released as auto-modules in Lucene 9.0.

Yes, that is my understanding also.

> Uwe
>
> P.S.: Not sure with which version you are testing Elasticsearch. The automodules were added 2 weeks ago and yesterday we renamed them to have fully-qualified domain names. We will merge the PR later and backport to 9.0 and 9.x branch.

I’m experimenting with a recent 9.0 snapshot. I’ll update once the latest name changes find their way into 9.0.

> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de <https://www.thetaphi.de/>
> eMail: uwe@thetaphi.de <mailto:uwe@thetaphi.de>
>
> From: Dawid Weiss <dawid.weiss@gmail.com>
> Sent: Tuesday, November 30, 2021 1:19 PM
> To: Lucene Dev <dev@lucene.apache.org>
> Subject: Re: HEADS UP: Java 9 Module System (JMS) module names for Lucene artifacts
>
>> Getting this fixed in 9.0.0 will allow us to leverage these stable module names in requires
>> directives (without needing to change them anything soon).
>
> For the record - they are not considered stable in 9.0.0 -- see Uwe's note below (although this was my original
> intention for introducing automatic module names, even if the actual compact naming has been so
> fiercely criticized...).
>
> LUCENE-10234: Added Automatic-Module-Name to all JARs. This is a first step to enable full Java
> module system (JMS) support in later Lucene versions. At the moment, the automatic names should
> not be considered stable. (Dawid Weiss, Uwe Schindler)
>
> D.

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal