Mailing List Archive

Do we gain much from such a requirement? Are there APIs that make things
run faster or perform better?

The downside for me is that if Lucene requires Java 17 then all downstream
projects will be forced to require Java 17. And Java 11 LTS is still
perfectly fine and supported. So unless there is a gain, I wouldn't force
folks to upgrade.

Dawid

On Wed, Nov 3, 2021 at 3:50 PM Adrien Grand <jpountz@gmail.com> wrote:

> Hello,
>
> Now that the main branch is the future 10.0 version, would there be any
> concern if we bumped the minimum Java version to 17 instead of 11?
>
> --
> Adrien
>

Who would we be forcing to upgrade? It is the 'main' branch: unreleased.

+1 to bump to 17

On Wed, Nov 3, 2021 at 11:19 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>
>
> Do we gain much from such a requirement? Are there APIs that make things run faster or perform better?
>
> The downside for me is that if Lucene requires Java 17 then all downstream projects will be forced to require Java 17. And Java 11 LTS is still perfectly fine and supported. So unless there is a gain, I wouldn't force folks to upgrade.
>
> Dawid
>
> On Wed, Nov 3, 2021 at 3:50 PM Adrien Grand <jpountz@gmail.com> wrote:
>>
>> Hello,
>>
>> Now that the main branch is the future 10.0 version, would there be any concern if we bumped the minimum Java version to 17 instead of 11?
>>
>> --
>> Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

> Who would we be forcing to upgrade? It is the 'main' branch: unreleased.
>

It will be released, eventually, right? In six months, a year maybe? Then
it's people like me who would be affected: we use Lucene internally and
this one dependency would push the entire stack to Java 17. I wouldn't mind
this at all if there was something super-attractive there to gain but all
the benefits of Java 17 to me at the moment lie at source-code level... I
just don't see much benefit at runtime. So I personally don't feel the need
to upgrade - this applies to Lucene, my own software and even my shell
default...

D.


>
> +1 to bump to 17
>
> On Wed, Nov 3, 2021 at 11:19 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> >
> >
> > Do we gain much from such a requirement? Are there APIs that make things
> run faster or perform better?
> >
> > The downside for me is that if Lucene requires Java 17 then all
> downstream projects will be forced to require Java 17. And Java 11 LTS is
> still perfectly fine and supported. So unless there is a gain, I wouldn't
> force folks to upgrade.
> >
> > Dawid
> >
> > On Wed, Nov 3, 2021 at 3:50 PM Adrien Grand <jpountz@gmail.com> wrote:
> >>
> >> Hello,
> >>
> >> Now that the main branch is the future 10.0 version, would there be any
> concern if we bumped the minimum Java version to 17 instead of 11?
> >>
> >> --
> >> Adrien
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

On Wed, Nov 3, 2021 at 12:32 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>
>
>> Who would we be forcing to upgrade? It is the 'main' branch: unreleased.
>
>
> It will be released, eventually, right? In six months, a year maybe? Then it's people like me who would be affected: we use Lucene internally and this one dependency would push the entire stack to Java 17. I wouldn't mind this at all if there was something super-attractive there to gain but all the benefits of Java 17 to me at the moment lie at source-code level... I just don't see much benefit at runtime. So I personally don't feel the need to upgrade - this applies to Lucene, my own software and even my shell default...

six months to a year? I think it has been 2.5 years since the last
major release!

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

On Wed, Nov 3, 2021 at 12:51 PM Robert Muir <rcmuir@gmail.com> wrote:
> >
> > It will be released, eventually, right? In six months, a year maybe? Then it's people like me who would be affected: we use Lucene internally and this one dependency would push the entire stack to Java 17. I wouldn't mind this at all if there was something super-attractive there to gain but all the benefits of Java 17 to me at the moment lie at source-code level... I just don't see much benefit at runtime. So I personally don't feel the need to upgrade - this applies to Lucene, my own software and even my shell default...
>

Separately, but if you want to stay on old stuff, then stay on old
stuff (e.g. stay on lucene 9 with java 11). This is how people do it
with all other software. If you are happy with java 11, then you
"personally don't feel the need to upgrade" and you can just stay on
lucene 9, or lucene 8, or lucene 7, or whatever!

But there should be a place where we can move forwards: I think the
'main' branch is appropriate.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

+1 for JDK 17 or maybe 18. In main (to eventually be Lucene 10.0, a couple
years from now?) we should try to be free to use the latest JDK and Java
language improvements.

Mike McCandless

http://blog.mikemccandless.com


On Wed, Nov 3, 2021 at 12:57 PM Robert Muir <rcmuir@gmail.com> wrote:

> On Wed, Nov 3, 2021 at 12:32 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> >
> >
> >> Who would we be forcing to upgrade? It is the 'main' branch: unreleased.
> >
> >
> > It will be released, eventually, right? In six months, a year maybe?
> Then it's people like me who would be affected: we use Lucene internally
> and this one dependency would push the entire stack to Java 17. I wouldn't
> mind this at all if there was something super-attractive there to gain but
> all the benefits of Java 17 to me at the moment lie at source-code level...
> I just don't see much benefit at runtime. So I personally don't feel the
> need to upgrade - this applies to Lucene, my own software and even my shell
> default...
>
> six months to a year? I think it has been 2.5 years since the last
> major release!
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

We'll be going to Java 18 or 19 as a minimum for MMapDirectory using the
new Panama APIs once those stabilize, right?

We could probably benefit today some from record classes, but I'm not sure
how much of a hint those are to the runtime VM for optimizations or if it
is entirely a source code syntactic sugar thing.

On Wed, Nov 3, 2021 at 11:53 AM Robert Muir <rcmuir@gmail.com> wrote:

> On Wed, Nov 3, 2021 at 12:51 PM Robert Muir <rcmuir@gmail.com> wrote:
> > >
> > > It will be released, eventually, right? In six months, a year maybe?
> Then it's people like me who would be affected: we use Lucene internally
> and this one dependency would push the entire stack to Java 17. I wouldn't
> mind this at all if there was something super-attractive there to gain but
> all the benefits of Java 17 to me at the moment lie at source-code level...
> I just don't see much benefit at runtime. So I personally don't feel the
> need to upgrade - this applies to Lucene, my own software and even my shell
> default...
> >
>
> Separately, but if you want to stay on old stuff, then stay on old
> stuff (e.g. stay on lucene 9 with java 11). This is how people do it
> with all other software. If you are happy with java 11, then you
> "personally don't feel the need to upgrade" and you can just stay on
> lucene 9, or lucene 8, or lucene 7, or whatever!
>
> But there should be a place where we can move forwards: I think the
> 'main' branch is appropriate.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

>
> six months to a year? I think it has been 2.5 years since the last
> major release!
>

I was hoping for an improvement here! :)

> we should try to be free to use the latest JDK and Java language
improvements.

I principally agree with you - we should leverage new Java features and I'm
all for it. I just don't see much difference between
Java 11 and 17 in the context of Lucene... Upgrading for the sake of
upgrading doesn't justify the move to
me. But if you can point at a feature of Java 17 and say - here, this is
great and was not there before, it's worth using, then I'm all in.

D.

On Wed, Nov 3, 2021 at 1:36 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>
> I principally agree with you - we should leverage new Java features and I'm all for it. I just don't see much difference between
> Java 11 and 17 in the context of Lucene... Upgrading for the sake of upgrading doesn't justify the move to
> me. But if you can point at a feature of Java 17 and say - here, this is great and was not there before, it's worth using, then I'm all in.
>
> D.

absolute-bulk-get methods on Byte/Short/Int/Long/Float/DoubleBuffers?

I think we should investigate it for MMapDirectory and
ByteBuffersDirectory at least? Maybe it can create new opportunities,
e.g. reduce overhead vs position()+get(). Or maybe expand our
random-access API to include it, and perhaps bit-unpacking can be
simplified or sped up (e.g. DirectReader). Especially now that we have
varhandles it seems to make more things possible. Or maybe there's no
performance win for us and it only simplifies existing code in the
short-term.

I like the new PRNGs, maybe we should replace our handrolled
xorshift128 stuff that is used for segment IDs (see StringHelper). The
new API has nice set of algorithms:
https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/random/package-summary.html.
Good to look at for the HNSW vector stuff, too. Maybe, we should
switch over unit tests eventually too.

The JFR runtime streaming api looks interesting, maybe we could
improve tests.profile to use it, or mike's benchmark.
I like that they fixed multicast api to work correctly (IIRC
previously you had to implicitly bind to all interfaces, couldn't even
bind to just localhost). Theoretically it could be more efficient for
stuff like replication, but there's still practical issues (e.g. you
have to deal with UDP, some cloud environments have spotty support,
etc).
Unix Sockets! Now they work with windows, so java exposed them. I use
these heavily at work, curl --unix-socket is my GOTO. it's a nice
bonus you can protect them with ordinary file permissions on Linux at
least. I love apps like haproxy that expose stats/control interfaces
first-class over unix sockets. Infostream logging is nice, but maybe
we should provide other options to make it easy to get
metrics/statistics counters and such "live".
I like that the Unicode version is bumped, that helps the lucene
analyzers based on the JDK.
I'm also a fan of the HexFormat to replace any hand-rolled
hex-printers. Could probably clean up some test code at least.

There's a lot of little improvements to the API like this:
https://docs.oracle.com/en/java/javase/17/docs/api/new-list.html

As far as the language/major features, sure they are just sugar
sometime, but often it makes sense to refactor the code to use them.
E.G. having text block support, it is just one of those little things
that can make the code much easier. And I get spoiled by other
languages that all seem to have this.
There's a new packaging tool that might be appropriate for Luke, not sure.
Maybe lucene/expressions should use Hidden Classes? I can't remember
the details, but I think we make a private child classloader, register
the class there, to try to prevent GC hell. But why register it at
all?
The switch expressions looks interesting, because we could remove some
of the horrors of forgotten-break statements and stuff? Haven't looked
in detail, I think we are doing stuff with ecj to try to detect these
mistakes already. Seems potentially less error-prone to use the new
syntax.

You can easily see the full list of these language/major features since java 11:
https://openjdk.java.net/projects/jdk/12/
https://openjdk.java.net/projects/jdk/13/
https://openjdk.java.net/projects/jdk/14/
https://openjdk.java.net/projects/jdk/15/
https://openjdk.java.net/projects/jdk/16/
https://openjdk.java.net/projects/jdk/17/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Now you're talking.
+1.


On Thu, Nov 4, 2021 at 1:49 AM Robert Muir <rcmuir@gmail.com> wrote:

> On Wed, Nov 3, 2021 at 1:36 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> >
> > I principally agree with you - we should leverage new Java features and
> I'm all for it. I just don't see much difference between
> > Java 11 and 17 in the context of Lucene... Upgrading for the sake of
> upgrading doesn't justify the move to
> > me. But if you can point at a feature of Java 17 and say - here, this is
> great and was not there before, it's worth using, then I'm all in.
> >
> > D.
>
> absolute-bulk-get methods on Byte/Short/Int/Long/Float/DoubleBuffers?
>
> I think we should investigate it for MMapDirectory and
> ByteBuffersDirectory at least? Maybe it can create new opportunities,
> e.g. reduce overhead vs position()+get(). Or maybe expand our
> random-access API to include it, and perhaps bit-unpacking can be
> simplified or sped up (e.g. DirectReader). Especially now that we have
> varhandles it seems to make more things possible. Or maybe there's no
> performance win for us and it only simplifies existing code in the
> short-term.
>
> I like the new PRNGs, maybe we should replace our handrolled
> xorshift128 stuff that is used for segment IDs (see StringHelper). The
> new API has nice set of algorithms:
>
> https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/random/package-summary.html
> .
> Good to look at for the HNSW vector stuff, too. Maybe, we should
> switch over unit tests eventually too.
>
> The JFR runtime streaming api looks interesting, maybe we could
> improve tests.profile to use it, or mike's benchmark.
> I like that they fixed multicast api to work correctly (IIRC
> previously you had to implicitly bind to all interfaces, couldn't even
> bind to just localhost). Theoretically it could be more efficient for
> stuff like replication, but there's still practical issues (e.g. you
> have to deal with UDP, some cloud environments have spotty support,
> etc).
> Unix Sockets! Now they work with windows, so java exposed them. I use
> these heavily at work, curl --unix-socket is my GOTO. it's a nice
> bonus you can protect them with ordinary file permissions on Linux at
> least. I love apps like haproxy that expose stats/control interfaces
> first-class over unix sockets. Infostream logging is nice, but maybe
> we should provide other options to make it easy to get
> metrics/statistics counters and such "live".
> I like that the Unicode version is bumped, that helps the lucene
> analyzers based on the JDK.
> I'm also a fan of the HexFormat to replace any hand-rolled
> hex-printers. Could probably clean up some test code at least.
>
> There's a lot of little improvements to the API like this:
> https://docs.oracle.com/en/java/javase/17/docs/api/new-list.html
>
> As far as the language/major features, sure they are just sugar
> sometime, but often it makes sense to refactor the code to use them.
> E.G. having text block support, it is just one of those little things
> that can make the code much easier. And I get spoiled by other
> languages that all seem to have this.
> There's a new packaging tool that might be appropriate for Luke, not sure.
> Maybe lucene/expressions should use Hidden Classes? I can't remember
> the details, but I think we make a private child classloader, register
> the class there, to try to prevent GC hell. But why register it at
> all?
> The switch expressions looks interesting, because we could remove some
> of the horrors of forgotten-break statements and stuff? Haven't looked
> in detail, I think we are doing stuff with ecj to try to detect these
> mistakes already. Seems potentially less error-prone to use the new
> syntax.
>
> You can easily see the full list of these language/major features since
> java 11:
> https://openjdk.java.net/projects/jdk/12/
> https://openjdk.java.net/projects/jdk/13/
> https://openjdk.java.net/projects/jdk/14/
> https://openjdk.java.net/projects/jdk/15/
> https://openjdk.java.net/projects/jdk/16/
> https://openjdk.java.net/projects/jdk/17/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

Hi,



I agree with this plan, lets go to JDK17 in Lucene 10 (main), but whenever a new Java version comes out, update Gradle and the –release=XX switch. Plain simple! The stable branch has a defined java version (currently “11”), “main” should be always latest. I don’t think this is a problem, because the Java release cycles have changed and people who are old-syled are still on 8 (so would be stuck with Lucene 8). For some larger companies they stick with officially “Oracle supported LTS” versions, but those people won’t upgrade soo. Nowadays with Docker and Kubernetes, it is so easy to start Solr or Elasticsearch with any Java version (and you don’t care, you just take what’s shipped with your image), so beeing bleeding edge on main is perfectly fine. When we release a new major version, we take what’s latest at that time (based on main branch, hopefully with Panama).



Based on my previous statement, JDK 17 is not the final goal for Lucene 10, and not even 18 it is: JDK 18 won’t contain Panama (they have a second icubator of Total-Panama), so it is likely to be part of “java.base” Module in JDK 19 (still requiring some extra enabler-command line param).



About 17: What I like most is the multiline-Strings and the new switch statement. In addition to Robert’s comment: I like it not only because of the break-hell, more because it is not a simple statement, but an expression (having return value). So the anti-pattern like a variable and then a switch stament assigning a value to this variable in each case is then finally obsolete. You have then “variable = switch(….)”. And finally we will get a switch for instanceofs a bit later (hopefully at same time when Panama comes out) ????



Records are bullshit, sorry. It’s only useful for the Hibernate/Spring/Foobar-like Entities-For-Everything business logic. It may be useful at some point when they are no instances on heap anymore and just data wrappers, but based on classes I see no reason to use them for Lucene.



Uwe



-----

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: uwe@thetaphi.de



From: Dawid Weiss <dawid.weiss@gmail.com>
Sent: Thursday, November 4, 2021 8:27 AM
To: Lucene Dev <dev@lucene.apache.org>
Subject: Re: Bump minimum Java version to 17 on main (10.0)





Now you're talking.

+1.





On Thu, Nov 4, 2021 at 1:49 AM Robert Muir <rcmuir@gmail.com <mailto:rcmuir@gmail.com> > wrote:

On Wed, Nov 3, 2021 at 1:36 PM Dawid Weiss <dawid.weiss@gmail.com <mailto:dawid.weiss@gmail.com> > wrote:
>
> I principally agree with you - we should leverage new Java features and I'm all for it. I just don't see much difference between
> Java 11 and 17 in the context of Lucene... Upgrading for the sake of upgrading doesn't justify the move to
> me. But if you can point at a feature of Java 17 and say - here, this is great and was not there before, it's worth using, then I'm all in.
>
> D.

absolute-bulk-get methods on Byte/Short/Int/Long/Float/DoubleBuffers?

I think we should investigate it for MMapDirectory and
ByteBuffersDirectory at least? Maybe it can create new opportunities,
e.g. reduce overhead vs position()+get(). Or maybe expand our
random-access API to include it, and perhaps bit-unpacking can be
simplified or sped up (e.g. DirectReader). Especially now that we have
varhandles it seems to make more things possible. Or maybe there's no
performance win for us and it only simplifies existing code in the
short-term.

I like the new PRNGs, maybe we should replace our handrolled
xorshift128 stuff that is used for segment IDs (see StringHelper). The
new API has nice set of algorithms:
https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/random/package-summary.html.
Good to look at for the HNSW vector stuff, too. Maybe, we should
switch over unit tests eventually too.

The JFR runtime streaming api looks interesting, maybe we could
improve tests.profile to use it, or mike's benchmark.
I like that they fixed multicast api to work correctly (IIRC
previously you had to implicitly bind to all interfaces, couldn't even
bind to just localhost). Theoretically it could be more efficient for
stuff like replication, but there's still practical issues (e.g. you
have to deal with UDP, some cloud environments have spotty support,
etc).
Unix Sockets! Now they work with windows, so java exposed them. I use
these heavily at work, curl --unix-socket is my GOTO. it's a nice
bonus you can protect them with ordinary file permissions on Linux at
least. I love apps like haproxy that expose stats/control interfaces
first-class over unix sockets. Infostream logging is nice, but maybe
we should provide other options to make it easy to get
metrics/statistics counters and such "live".
I like that the Unicode version is bumped, that helps the lucene
analyzers based on the JDK.
I'm also a fan of the HexFormat to replace any hand-rolled
hex-printers. Could probably clean up some test code at least.

There's a lot of little improvements to the API like this:
https://docs.oracle.com/en/java/javase/17/docs/api/new-list.html

As far as the language/major features, sure they are just sugar
sometime, but often it makes sense to refactor the code to use them.
E.G. having text block support, it is just one of those little things
that can make the code much easier. And I get spoiled by other
languages that all seem to have this.
There's a new packaging tool that might be appropriate for Luke, not sure.
Maybe lucene/expressions should use Hidden Classes? I can't remember
the details, but I think we make a private child classloader, register
the class there, to try to prevent GC hell. But why register it at
all?
The switch expressions looks interesting, because we could remove some
of the horrors of forgotten-break statements and stuff? Haven't looked
in detail, I think we are doing stuff with ecj to try to detect these
mistakes already. Seems potentially less error-prone to use the new
syntax.

You can easily see the full list of these language/major features since java 11:
https://openjdk.java.net/projects/jdk/12/
https://openjdk.java.net/projects/jdk/13/
https://openjdk.java.net/projects/jdk/14/
https://openjdk.java.net/projects/jdk/15/
https://openjdk.java.net/projects/jdk/16/
https://openjdk.java.net/projects/jdk/17/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>

I prefer that we require JDK 17 for build/test but allow our artifacts
(except lucene-test-framework maybe) to be run on JDK 11 (or 14?) via
setting the "target". This allows us some time to appreciate some of the
benefits of Java/JDK 17 without insisting that our users switch. This
approach doesn't prevent us from fully-committing to JDK 17 for Lucene 10
if we want. When we consider that Lucene is a library and not a full app,
we should be somewhat conservative here.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Thu, Nov 4, 2021 at 6:10 AM Uwe Schindler <uwe@thetaphi.de> wrote:

> Hi,
>
>
>
> I agree with this plan, lets go to JDK17 in Lucene 10 (main), but whenever
> a new Java version comes out, update Gradle and the –release=XX switch.
> Plain simple! The stable branch has a defined java version (currently
> “11”), “main” should be always latest. I don’t think this is a problem,
> because the Java release cycles have changed and people who are old-syled
> are still on 8 (so would be stuck with Lucene 8). For some larger companies
> they stick with officially “Oracle supported LTS” versions, but those
> people won’t upgrade soo. Nowadays with Docker and Kubernetes, it is so
> easy to start Solr or Elasticsearch with any Java version (and you don’t
> care, you just take what’s shipped with your image), so beeing bleeding
> edge on main is perfectly fine. When we release a new major version, we
> take what’s latest at that time (based on main branch, hopefully with
> Panama).
>
>
>
> Based on my previous statement, JDK 17 is not the final goal for Lucene
> 10, and not even 18 it is: JDK 18 won’t contain Panama (they have a second
> icubator of Total-Panama), so it is likely to be part of “java.base” Module
> in JDK 19 (still requiring some extra enabler-command line param).
>
>
>
> About 17: What I like most is the multiline-Strings and the new switch
> statement. In addition to Robert’s comment: I like it not only because of
> the break-hell, more because it is not a simple statement, but an
> expression (having return value). So the anti-pattern like a variable and
> then a switch stament assigning a value to this variable in each case is
> then finally obsolete. You have then “variable = switch(….)”. And finally
> we will get a switch for instanceofs a bit later (hopefully at same time
> when Panama comes out) ????
>
>
>
> Records are bullshit, sorry. It’s only useful for the
> Hibernate/Spring/Foobar-like Entities-For-Everything business logic. It may
> be useful at some point when they are no instances on heap anymore and just
> data wrappers, but based on classes I see no reason to use them for Lucene.
>
>
>
> Uwe
>
>
>
> -----
>
> Uwe Schindler
>
> Achterdiek 19, D-28357 Bremen
>
> https://www.thetaphi.de
>
> eMail: uwe@thetaphi.de
>
>
>
> *From:* Dawid Weiss <dawid.weiss@gmail.com>
> *Sent:* Thursday, November 4, 2021 8:27 AM
> *To:* Lucene Dev <dev@lucene.apache.org>
> *Subject:* Re: Bump minimum Java version to 17 on main (10.0)
>
>
>
>
>
> Now you're talking.
>
> +1.
>
>
>
>
>
> On Thu, Nov 4, 2021 at 1:49 AM Robert Muir <rcmuir@gmail.com> wrote:
>
> On Wed, Nov 3, 2021 at 1:36 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> >
> > I principally agree with you - we should leverage new Java features and
> I'm all for it. I just don't see much difference between
> > Java 11 and 17 in the context of Lucene... Upgrading for the sake of
> upgrading doesn't justify the move to
> > me. But if you can point at a feature of Java 17 and say - here, this is
> great and was not there before, it's worth using, then I'm all in.
> >
> > D.
>
> absolute-bulk-get methods on Byte/Short/Int/Long/Float/DoubleBuffers?
>
> I think we should investigate it for MMapDirectory and
> ByteBuffersDirectory at least? Maybe it can create new opportunities,
> e.g. reduce overhead vs position()+get(). Or maybe expand our
> random-access API to include it, and perhaps bit-unpacking can be
> simplified or sped up (e.g. DirectReader). Especially now that we have
> varhandles it seems to make more things possible. Or maybe there's no
> performance win for us and it only simplifies existing code in the
> short-term.
>
> I like the new PRNGs, maybe we should replace our handrolled
> xorshift128 stuff that is used for segment IDs (see StringHelper). The
> new API has nice set of algorithms:
>
> https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/random/package-summary.html
> .
> Good to look at for the HNSW vector stuff, too. Maybe, we should
> switch over unit tests eventually too.
>
> The JFR runtime streaming api looks interesting, maybe we could
> improve tests.profile to use it, or mike's benchmark.
> I like that they fixed multicast api to work correctly (IIRC
> previously you had to implicitly bind to all interfaces, couldn't even
> bind to just localhost). Theoretically it could be more efficient for
> stuff like replication, but there's still practical issues (e.g. you
> have to deal with UDP, some cloud environments have spotty support,
> etc).
> Unix Sockets! Now they work with windows, so java exposed them. I use
> these heavily at work, curl --unix-socket is my GOTO. it's a nice
> bonus you can protect them with ordinary file permissions on Linux at
> least. I love apps like haproxy that expose stats/control interfaces
> first-class over unix sockets. Infostream logging is nice, but maybe
> we should provide other options to make it easy to get
> metrics/statistics counters and such "live".
> I like that the Unicode version is bumped, that helps the lucene
> analyzers based on the JDK.
> I'm also a fan of the HexFormat to replace any hand-rolled
> hex-printers. Could probably clean up some test code at least.
>
> There's a lot of little improvements to the API like this:
> https://docs.oracle.com/en/java/javase/17/docs/api/new-list.html
>
> As far as the language/major features, sure they are just sugar
> sometime, but often it makes sense to refactor the code to use them.
> E.G. having text block support, it is just one of those little things
> that can make the code much easier. And I get spoiled by other
> languages that all seem to have this.
> There's a new packaging tool that might be appropriate for Luke, not sure.
> Maybe lucene/expressions should use Hidden Classes? I can't remember
> the details, but I think we make a private child classloader, register
> the class there, to try to prevent GC hell. But why register it at
> all?
> The switch expressions looks interesting, because we could remove some
> of the horrors of forgotten-break statements and stuff? Haven't looked
> in detail, I think we are doing stuff with ecj to try to detect these
> mistakes already. Seems potentially less error-prone to use the new
> syntax.
>
> You can easily see the full list of these language/major features since
> java 11:
> https://openjdk.java.net/projects/jdk/12/
> https://openjdk.java.net/projects/jdk/13/
> https://openjdk.java.net/projects/jdk/14/
> https://openjdk.java.net/projects/jdk/15/
> https://openjdk.java.net/projects/jdk/16/
> https://openjdk.java.net/projects/jdk/17/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

With the current release frequency, JDK 11 will be EOL by the time
Lucene 10 is released.

Users can use lucene 9 if they want to stay on old outdated JDKs.

On Thu, Nov 4, 2021 at 7:44 AM David Smiley <dsmiley@apache.org> wrote:
>
> I prefer that we require JDK 17 for build/test but allow our artifacts (except lucene-test-framework maybe) to be run on JDK 11 (or 14?) via setting the "target". This allows us some time to appreciate some of the benefits of Java/JDK 17 without insisting that our users switch. This approach doesn't prevent us from fully-committing to JDK 17 for Lucene 10 if we want. When we consider that Lucene is a library and not a full app, we should be somewhat conservative here.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Thu, Nov 4, 2021 at 6:10 AM Uwe Schindler <uwe@thetaphi.de> wrote:
>>
>> Hi,
>>
>>
>>
>> I agree with this plan, lets go to JDK17 in Lucene 10 (main), but whenever a new Java version comes out, update Gradle and the –release=XX switch. Plain simple! The stable branch has a defined java version (currently “11”), “main” should be always latest. I don’t think this is a problem, because the Java release cycles have changed and people who are old-syled are still on 8 (so would be stuck with Lucene 8). For some larger companies they stick with officially “Oracle supported LTS” versions, but those people won’t upgrade soo. Nowadays with Docker and Kubernetes, it is so easy to start Solr or Elasticsearch with any Java version (and you don’t care, you just take what’s shipped with your image), so beeing bleeding edge on main is perfectly fine. When we release a new major version, we take what’s latest at that time (based on main branch, hopefully with Panama).
>>
>>
>>
>> Based on my previous statement, JDK 17 is not the final goal for Lucene 10, and not even 18 it is: JDK 18 won’t contain Panama (they have a second icubator of Total-Panama), so it is likely to be part of “java.base” Module in JDK 19 (still requiring some extra enabler-command line param).
>>
>>
>>
>> About 17: What I like most is the multiline-Strings and the new switch statement. In addition to Robert’s comment: I like it not only because of the break-hell, more because it is not a simple statement, but an expression (having return value). So the anti-pattern like a variable and then a switch stament assigning a value to this variable in each case is then finally obsolete. You have then “variable = switch(….)”. And finally we will get a switch for instanceofs a bit later (hopefully at same time when Panama comes out) ????
>>
>>
>>
>> Records are bullshit, sorry. It’s only useful for the Hibernate/Spring/Foobar-like Entities-For-Everything business logic. It may be useful at some point when they are no instances on heap anymore and just data wrappers, but based on classes I see no reason to use them for Lucene.
>>
>>
>>
>> Uwe
>>
>>
>>
>> -----
>>
>> Uwe Schindler
>>
>> Achterdiek 19, D-28357 Bremen
>>
>> https://www.thetaphi.de
>>
>> eMail: uwe@thetaphi.de
>>
>>
>>
>> From: Dawid Weiss <dawid.weiss@gmail.com>
>> Sent: Thursday, November 4, 2021 8:27 AM
>> To: Lucene Dev <dev@lucene.apache.org>
>> Subject: Re: Bump minimum Java version to 17 on main (10.0)
>>
>>
>>
>>
>>
>> Now you're talking.
>>
>> +1.
>>
>>
>>
>>
>>
>> On Thu, Nov 4, 2021 at 1:49 AM Robert Muir <rcmuir@gmail.com> wrote:
>>
>> On Wed, Nov 3, 2021 at 1:36 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>> >
>> > I principally agree with you - we should leverage new Java features and I'm all for it. I just don't see much difference between
>> > Java 11 and 17 in the context of Lucene... Upgrading for the sake of upgrading doesn't justify the move to
>> > me. But if you can point at a feature of Java 17 and say - here, this is great and was not there before, it's worth using, then I'm all in.
>> >
>> > D.
>>
>> absolute-bulk-get methods on Byte/Short/Int/Long/Float/DoubleBuffers?
>>
>> I think we should investigate it for MMapDirectory and
>> ByteBuffersDirectory at least? Maybe it can create new opportunities,
>> e.g. reduce overhead vs position()+get(). Or maybe expand our
>> random-access API to include it, and perhaps bit-unpacking can be
>> simplified or sped up (e.g. DirectReader). Especially now that we have
>> varhandles it seems to make more things possible. Or maybe there's no
>> performance win for us and it only simplifies existing code in the
>> short-term.
>>
>> I like the new PRNGs, maybe we should replace our handrolled
>> xorshift128 stuff that is used for segment IDs (see StringHelper). The
>> new API has nice set of algorithms:
>> https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/random/package-summary.html.
>> Good to look at for the HNSW vector stuff, too. Maybe, we should
>> switch over unit tests eventually too.
>>
>> The JFR runtime streaming api looks interesting, maybe we could
>> improve tests.profile to use it, or mike's benchmark.
>> I like that they fixed multicast api to work correctly (IIRC
>> previously you had to implicitly bind to all interfaces, couldn't even
>> bind to just localhost). Theoretically it could be more efficient for
>> stuff like replication, but there's still practical issues (e.g. you
>> have to deal with UDP, some cloud environments have spotty support,
>> etc).
>> Unix Sockets! Now they work with windows, so java exposed them. I use
>> these heavily at work, curl --unix-socket is my GOTO. it's a nice
>> bonus you can protect them with ordinary file permissions on Linux at
>> least. I love apps like haproxy that expose stats/control interfaces
>> first-class over unix sockets. Infostream logging is nice, but maybe
>> we should provide other options to make it easy to get
>> metrics/statistics counters and such "live".
>> I like that the Unicode version is bumped, that helps the lucene
>> analyzers based on the JDK.
>> I'm also a fan of the HexFormat to replace any hand-rolled
>> hex-printers. Could probably clean up some test code at least.
>>
>> There's a lot of little improvements to the API like this:
>> https://docs.oracle.com/en/java/javase/17/docs/api/new-list.html
>>
>> As far as the language/major features, sure they are just sugar
>> sometime, but often it makes sense to refactor the code to use them.
>> E.G. having text block support, it is just one of those little things
>> that can make the code much easier. And I get spoiled by other
>> languages that all seem to have this.
>> There's a new packaging tool that might be appropriate for Luke, not sure.
>> Maybe lucene/expressions should use Hidden Classes? I can't remember
>> the details, but I think we make a private child classloader, register
>> the class there, to try to prevent GC hell. But why register it at
>> all?
>> The switch expressions looks interesting, because we could remove some
>> of the horrors of forgotten-break statements and stuff? Haven't looked
>> in detail, I think we are doing stuff with ecj to try to detect these
>> mistakes already. Seems potentially less error-prone to use the new
>> syntax.
>>
>> You can easily see the full list of these language/major features since java 11:
>> https://openjdk.java.net/projects/jdk/12/
>> https://openjdk.java.net/projects/jdk/13/
>> https://openjdk.java.net/projects/jdk/14/
>> https://openjdk.java.net/projects/jdk/15/
>> https://openjdk.java.net/projects/jdk/16/
>> https://openjdk.java.net/projects/jdk/17/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

I can imagine some users might like to keep abreast of main, at least
in some kind of testing setup, but aren't ready to cut over their JDK
for some reason (as Dawid was describing), but they can always do a
small patch and build Lucene themselves, applying the -target for
compatibility with their environments. And if we ever get to the point
of really *requiring* compilation for the latest JDK, then it will
probably be for a good reason, and we should prepare users for that.

On Thu, Nov 4, 2021 at 7:54 AM Robert Muir <rcmuir@gmail.com> wrote:
>
> With the current release frequency, JDK 11 will be EOL by the time
> Lucene 10 is released.
>
> Users can use lucene 9 if they want to stay on old outdated JDKs.
>
> On Thu, Nov 4, 2021 at 7:44 AM David Smiley <dsmiley@apache.org> wrote:
> >
> > I prefer that we require JDK 17 for build/test but allow our artifacts (except lucene-test-framework maybe) to be run on JDK 11 (or 14?) via setting the "target". This allows us some time to appreciate some of the benefits of Java/JDK 17 without insisting that our users switch. This approach doesn't prevent us from fully-committing to JDK 17 for Lucene 10 if we want. When we consider that Lucene is a library and not a full app, we should be somewhat conservative here.
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > On Thu, Nov 4, 2021 at 6:10 AM Uwe Schindler <uwe@thetaphi.de> wrote:
> >>
> >> Hi,
> >>
> >>
> >>
> >> I agree with this plan, lets go to JDK17 in Lucene 10 (main), but whenever a new Java version comes out, update Gradle and the –release=XX switch. Plain simple! The stable branch has a defined java version (currently “11”), “main” should be always latest. I don’t think this is a problem, because the Java release cycles have changed and people who are old-syled are still on 8 (so would be stuck with Lucene 8). For some larger companies they stick with officially “Oracle supported LTS” versions, but those people won’t upgrade soo. Nowadays with Docker and Kubernetes, it is so easy to start Solr or Elasticsearch with any Java version (and you don’t care, you just take what’s shipped with your image), so beeing bleeding edge on main is perfectly fine. When we release a new major version, we take what’s latest at that time (based on main branch, hopefully with Panama).
> >>
> >>
> >>
> >> Based on my previous statement, JDK 17 is not the final goal for Lucene 10, and not even 18 it is: JDK 18 won’t contain Panama (they have a second icubator of Total-Panama), so it is likely to be part of “java.base” Module in JDK 19 (still requiring some extra enabler-command line param).
> >>
> >>
> >>
> >> About 17: What I like most is the multiline-Strings and the new switch statement. In addition to Robert’s comment: I like it not only because of the break-hell, more because it is not a simple statement, but an expression (having return value). So the anti-pattern like a variable and then a switch stament assigning a value to this variable in each case is then finally obsolete. You have then “variable = switch(….)”. And finally we will get a switch for instanceofs a bit later (hopefully at same time when Panama comes out) ????
> >>
> >>
> >>
> >> Records are bullshit, sorry. It’s only useful for the Hibernate/Spring/Foobar-like Entities-For-Everything business logic. It may be useful at some point when they are no instances on heap anymore and just data wrappers, but based on classes I see no reason to use them for Lucene.
> >>
> >>
> >>
> >> Uwe
> >>
> >>
> >>
> >> -----
> >>
> >> Uwe Schindler
> >>
> >> Achterdiek 19, D-28357 Bremen
> >>
> >> https://www.thetaphi.de
> >>
> >> eMail: uwe@thetaphi.de
> >>
> >>
> >>
> >> From: Dawid Weiss <dawid.weiss@gmail.com>
> >> Sent: Thursday, November 4, 2021 8:27 AM
> >> To: Lucene Dev <dev@lucene.apache.org>
> >> Subject: Re: Bump minimum Java version to 17 on main (10.0)
> >>
> >>
> >>
> >>
> >>
> >> Now you're talking.
> >>
> >> +1.
> >>
> >>
> >>
> >>
> >>
> >> On Thu, Nov 4, 2021 at 1:49 AM Robert Muir <rcmuir@gmail.com> wrote:
> >>
> >> On Wed, Nov 3, 2021 at 1:36 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> >> >
> >> > I principally agree with you - we should leverage new Java features and I'm all for it. I just don't see much difference between
> >> > Java 11 and 17 in the context of Lucene... Upgrading for the sake of upgrading doesn't justify the move to
> >> > me. But if you can point at a feature of Java 17 and say - here, this is great and was not there before, it's worth using, then I'm all in.
> >> >
> >> > D.
> >>
> >> absolute-bulk-get methods on Byte/Short/Int/Long/Float/DoubleBuffers?
> >>
> >> I think we should investigate it for MMapDirectory and
> >> ByteBuffersDirectory at least? Maybe it can create new opportunities,
> >> e.g. reduce overhead vs position()+get(). Or maybe expand our
> >> random-access API to include it, and perhaps bit-unpacking can be
> >> simplified or sped up (e.g. DirectReader). Especially now that we have
> >> varhandles it seems to make more things possible. Or maybe there's no
> >> performance win for us and it only simplifies existing code in the
> >> short-term.
> >>
> >> I like the new PRNGs, maybe we should replace our handrolled
> >> xorshift128 stuff that is used for segment IDs (see StringHelper). The
> >> new API has nice set of algorithms:
> >> https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/random/package-summary.html.
> >> Good to look at for the HNSW vector stuff, too. Maybe, we should
> >> switch over unit tests eventually too.
> >>
> >> The JFR runtime streaming api looks interesting, maybe we could
> >> improve tests.profile to use it, or mike's benchmark.
> >> I like that they fixed multicast api to work correctly (IIRC
> >> previously you had to implicitly bind to all interfaces, couldn't even
> >> bind to just localhost). Theoretically it could be more efficient for
> >> stuff like replication, but there's still practical issues (e.g. you
> >> have to deal with UDP, some cloud environments have spotty support,
> >> etc).
> >> Unix Sockets! Now they work with windows, so java exposed them. I use
> >> these heavily at work, curl --unix-socket is my GOTO. it's a nice
> >> bonus you can protect them with ordinary file permissions on Linux at
> >> least. I love apps like haproxy that expose stats/control interfaces
> >> first-class over unix sockets. Infostream logging is nice, but maybe
> >> we should provide other options to make it easy to get
> >> metrics/statistics counters and such "live".
> >> I like that the Unicode version is bumped, that helps the lucene
> >> analyzers based on the JDK.
> >> I'm also a fan of the HexFormat to replace any hand-rolled
> >> hex-printers. Could probably clean up some test code at least.
> >>
> >> There's a lot of little improvements to the API like this:
> >> https://docs.oracle.com/en/java/javase/17/docs/api/new-list.html
> >>
> >> As far as the language/major features, sure they are just sugar
> >> sometime, but often it makes sense to refactor the code to use them.
> >> E.G. having text block support, it is just one of those little things
> >> that can make the code much easier. And I get spoiled by other
> >> languages that all seem to have this.
> >> There's a new packaging tool that might be appropriate for Luke, not sure.
> >> Maybe lucene/expressions should use Hidden Classes? I can't remember
> >> the details, but I think we make a private child classloader, register
> >> the class there, to try to prevent GC hell. But why register it at
> >> all?
> >> The switch expressions looks interesting, because we could remove some
> >> of the horrors of forgotten-break statements and stuff? Haven't looked
> >> in detail, I think we are doing stuff with ecj to try to detect these
> >> mistakes already. Seems potentially less error-prone to use the new
> >> syntax.
> >>
> >> You can easily see the full list of these language/major features since java 11:
> >> https://openjdk.java.net/projects/jdk/12/
> >> https://openjdk.java.net/projects/jdk/13/
> >> https://openjdk.java.net/projects/jdk/14/
> >> https://openjdk.java.net/projects/jdk/15/
> >> https://openjdk.java.net/projects/jdk/16/
> >> https://openjdk.java.net/projects/jdk/17/
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: dev-help@lucene.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

we don't have to "support" users on our main trunk branch. That's why
we make releases.

On Thu, Nov 4, 2021 at 9:59 AM Michael Sokolov <msokolov@gmail.com> wrote:
>
> I can imagine some users might like to keep abreast of main, at least
> in some kind of testing setup, but aren't ready to cut over their JDK
> for some reason (as Dawid was describing), but they can always do a
> small patch and build Lucene themselves, applying the -target for
> compatibility with their environments. And if we ever get to the point
> of really *requiring* compilation for the latest JDK, then it will
> probably be for a good reason, and we should prepare users for that.
>
> On Thu, Nov 4, 2021 at 7:54 AM Robert Muir <rcmuir@gmail.com> wrote:
> >
> > With the current release frequency, JDK 11 will be EOL by the time
> > Lucene 10 is released.
> >
> > Users can use lucene 9 if they want to stay on old outdated JDKs.
> >
> > On Thu, Nov 4, 2021 at 7:44 AM David Smiley <dsmiley@apache.org> wrote:
> > >
> > > I prefer that we require JDK 17 for build/test but allow our artifacts (except lucene-test-framework maybe) to be run on JDK 11 (or 14?) via setting the "target". This allows us some time to appreciate some of the benefits of Java/JDK 17 without insisting that our users switch. This approach doesn't prevent us from fully-committing to JDK 17 for Lucene 10 if we want. When we consider that Lucene is a library and not a full app, we should be somewhat conservative here.
> > >
> > > ~ David Smiley
> > > Apache Lucene/Solr Search Developer
> > > http://www.linkedin.com/in/davidwsmiley
> > >
> > >
> > > On Thu, Nov 4, 2021 at 6:10 AM Uwe Schindler <uwe@thetaphi.de> wrote:
> > >>
> > >> Hi,
> > >>
> > >>
> > >>
> > >> I agree with this plan, lets go to JDK17 in Lucene 10 (main), but whenever a new Java version comes out, update Gradle and the –release=XX switch. Plain simple! The stable branch has a defined java version (currently “11”), “main” should be always latest. I don’t think this is a problem, because the Java release cycles have changed and people who are old-syled are still on 8 (so would be stuck with Lucene 8). For some larger companies they stick with officially “Oracle supported LTS” versions, but those people won’t upgrade soo. Nowadays with Docker and Kubernetes, it is so easy to start Solr or Elasticsearch with any Java version (and you don’t care, you just take what’s shipped with your image), so beeing bleeding edge on main is perfectly fine. When we release a new major version, we take what’s latest at that time (based on main branch, hopefully with Panama).
> > >>
> > >>
> > >>
> > >> Based on my previous statement, JDK 17 is not the final goal for Lucene 10, and not even 18 it is: JDK 18 won’t contain Panama (they have a second icubator of Total-Panama), so it is likely to be part of “java.base” Module in JDK 19 (still requiring some extra enabler-command line param).
> > >>
> > >>
> > >>
> > >> About 17: What I like most is the multiline-Strings and the new switch statement. In addition to Robert’s comment: I like it not only because of the break-hell, more because it is not a simple statement, but an expression (having return value). So the anti-pattern like a variable and then a switch stament assigning a value to this variable in each case is then finally obsolete. You have then “variable = switch(….)”. And finally we will get a switch for instanceofs a bit later (hopefully at same time when Panama comes out) ????
> > >>
> > >>
> > >>
> > >> Records are bullshit, sorry. It’s only useful for the Hibernate/Spring/Foobar-like Entities-For-Everything business logic. It may be useful at some point when they are no instances on heap anymore and just data wrappers, but based on classes I see no reason to use them for Lucene.
> > >>
> > >>
> > >>
> > >> Uwe
> > >>
> > >>
> > >>
> > >> -----
> > >>
> > >> Uwe Schindler
> > >>
> > >> Achterdiek 19, D-28357 Bremen
> > >>
> > >> https://www.thetaphi.de
> > >>
> > >> eMail: uwe@thetaphi.de
> > >>
> > >>
> > >>
> > >> From: Dawid Weiss <dawid.weiss@gmail.com>
> > >> Sent: Thursday, November 4, 2021 8:27 AM
> > >> To: Lucene Dev <dev@lucene.apache.org>
> > >> Subject: Re: Bump minimum Java version to 17 on main (10.0)
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> Now you're talking.
> > >>
> > >> +1.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> On Thu, Nov 4, 2021 at 1:49 AM Robert Muir <rcmuir@gmail.com> wrote:
> > >>
> > >> On Wed, Nov 3, 2021 at 1:36 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> > >> >
> > >> > I principally agree with you - we should leverage new Java features and I'm all for it. I just don't see much difference between
> > >> > Java 11 and 17 in the context of Lucene... Upgrading for the sake of upgrading doesn't justify the move to
> > >> > me. But if you can point at a feature of Java 17 and say - here, this is great and was not there before, it's worth using, then I'm all in.
> > >> >
> > >> > D.
> > >>
> > >> absolute-bulk-get methods on Byte/Short/Int/Long/Float/DoubleBuffers?
> > >>
> > >> I think we should investigate it for MMapDirectory and
> > >> ByteBuffersDirectory at least? Maybe it can create new opportunities,
> > >> e.g. reduce overhead vs position()+get(). Or maybe expand our
> > >> random-access API to include it, and perhaps bit-unpacking can be
> > >> simplified or sped up (e.g. DirectReader). Especially now that we have
> > >> varhandles it seems to make more things possible. Or maybe there's no
> > >> performance win for us and it only simplifies existing code in the
> > >> short-term.
> > >>
> > >> I like the new PRNGs, maybe we should replace our handrolled
> > >> xorshift128 stuff that is used for segment IDs (see StringHelper). The
> > >> new API has nice set of algorithms:
> > >> https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/random/package-summary.html.
> > >> Good to look at for the HNSW vector stuff, too. Maybe, we should
> > >> switch over unit tests eventually too.
> > >>
> > >> The JFR runtime streaming api looks interesting, maybe we could
> > >> improve tests.profile to use it, or mike's benchmark.
> > >> I like that they fixed multicast api to work correctly (IIRC
> > >> previously you had to implicitly bind to all interfaces, couldn't even
> > >> bind to just localhost). Theoretically it could be more efficient for
> > >> stuff like replication, but there's still practical issues (e.g. you
> > >> have to deal with UDP, some cloud environments have spotty support,
> > >> etc).
> > >> Unix Sockets! Now they work with windows, so java exposed them. I use
> > >> these heavily at work, curl --unix-socket is my GOTO. it's a nice
> > >> bonus you can protect them with ordinary file permissions on Linux at
> > >> least. I love apps like haproxy that expose stats/control interfaces
> > >> first-class over unix sockets. Infostream logging is nice, but maybe
> > >> we should provide other options to make it easy to get
> > >> metrics/statistics counters and such "live".
> > >> I like that the Unicode version is bumped, that helps the lucene
> > >> analyzers based on the JDK.
> > >> I'm also a fan of the HexFormat to replace any hand-rolled
> > >> hex-printers. Could probably clean up some test code at least.
> > >>
> > >> There's a lot of little improvements to the API like this:
> > >> https://docs.oracle.com/en/java/javase/17/docs/api/new-list.html
> > >>
> > >> As far as the language/major features, sure they are just sugar
> > >> sometime, but often it makes sense to refactor the code to use them.
> > >> E.G. having text block support, it is just one of those little things
> > >> that can make the code much easier. And I get spoiled by other
> > >> languages that all seem to have this.
> > >> There's a new packaging tool that might be appropriate for Luke, not sure.
> > >> Maybe lucene/expressions should use Hidden Classes? I can't remember
> > >> the details, but I think we make a private child classloader, register
> > >> the class there, to try to prevent GC hell. But why register it at
> > >> all?
> > >> The switch expressions looks interesting, because we could remove some
> > >> of the horrors of forgotten-break statements and stuff? Haven't looked
> > >> in detail, I think we are doing stuff with ecj to try to detect these
> > >> mistakes already. Seems potentially less error-prone to use the new
> > >> syntax.
> > >>
> > >> You can easily see the full list of these language/major features since java 11:
> > >> https://openjdk.java.net/projects/jdk/12/
> > >> https://openjdk.java.net/projects/jdk/13/
> > >> https://openjdk.java.net/projects/jdk/14/
> > >> https://openjdk.java.net/projects/jdk/15/
> > >> https://openjdk.java.net/projects/jdk/16/
> > >> https://openjdk.java.net/projects/jdk/17/
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > >> For additional commands, e-mail: dev-help@lucene.apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

> we don't have to "support" users on our main trunk branch. That's why
> we make releases.

Nah. I don't agree with this one - you have to be aware about who is
using your project (especially a library) and in what context. Maybe
we differ in our opinion here. Your previous argument is much more
convincing (thank you for laying it out so clearly).

D.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

That does not work. You can set -release or -target, but not both with incompatible variants.



-----

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: uwe@thetaphi.de



From: David Smiley <dsmiley@apache.org>
Sent: Thursday, November 4, 2021 12:44 PM
To: lucene-dev <dev@lucene.apache.org>
Subject: Re: Bump minimum Java version to 17 on main (10.0)



I prefer that we require JDK 17 for build/test but allow our artifacts (except lucene-test-framework maybe) to be run on JDK 11 (or 14?) via setting the "target". This allows us some time to appreciate some of the benefits of Java/JDK 17 without insisting that our users switch. This approach doesn't prevent us from fully-committing to JDK 17 for Lucene 10 if we want. When we consider that Lucene is a library and not a full app, we should be somewhat conservative here.




~ David Smiley

Apache Lucene/Solr Search Developer

http://www.linkedin.com/in/davidwsmiley





On Thu, Nov 4, 2021 at 6:10 AM Uwe Schindler <uwe@thetaphi.de <mailto:uwe@thetaphi.de> > wrote:

Hi,



I agree with this plan, lets go to JDK17 in Lucene 10 (main), but whenever a new Java version comes out, update Gradle and the –release=XX switch. Plain simple! The stable branch has a defined java version (currently “11”), “main” should be always latest. I don’t think this is a problem, because the Java release cycles have changed and people who are old-syled are still on 8 (so would be stuck with Lucene 8). For some larger companies they stick with officially “Oracle supported LTS” versions, but those people won’t upgrade soo. Nowadays with Docker and Kubernetes, it is so easy to start Solr or Elasticsearch with any Java version (and you don’t care, you just take what’s shipped with your image), so beeing bleeding edge on main is perfectly fine. When we release a new major version, we take what’s latest at that time (based on main branch, hopefully with Panama).



Based on my previous statement, JDK 17 is not the final goal for Lucene 10, and not even 18 it is: JDK 18 won’t contain Panama (they have a second icubator of Total-Panama), so it is likely to be part of “java.base” Module in JDK 19 (still requiring some extra enabler-command line param).



About 17: What I like most is the multiline-Strings and the new switch statement. In addition to Robert’s comment: I like it not only because of the break-hell, more because it is not a simple statement, but an expression (having return value). So the anti-pattern like a variable and then a switch stament assigning a value to this variable in each case is then finally obsolete. You have then “variable = switch(….)”. And finally we will get a switch for instanceofs a bit later (hopefully at same time when Panama comes out) ????



Records are bullshit, sorry. It’s only useful for the Hibernate/Spring/Foobar-like Entities-For-Everything business logic. It may be useful at some point when they are no instances on heap anymore and just data wrappers, but based on classes I see no reason to use them for Lucene.



Uwe



-----

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: uwe@thetaphi.de <mailto:uwe@thetaphi.de>



From: Dawid Weiss <dawid.weiss@gmail.com <mailto:dawid.weiss@gmail.com> >
Sent: Thursday, November 4, 2021 8:27 AM
To: Lucene Dev <dev@lucene.apache.org <mailto:dev@lucene.apache.org> >
Subject: Re: Bump minimum Java version to 17 on main (10.0)





Now you're talking.

+1.





On Thu, Nov 4, 2021 at 1:49 AM Robert Muir <rcmuir@gmail.com <mailto:rcmuir@gmail.com> > wrote:

On Wed, Nov 3, 2021 at 1:36 PM Dawid Weiss <dawid.weiss@gmail.com <mailto:dawid.weiss@gmail.com> > wrote:
>
> I principally agree with you - we should leverage new Java features and I'm all for it. I just don't see much difference between
> Java 11 and 17 in the context of Lucene... Upgrading for the sake of upgrading doesn't justify the move to
> me. But if you can point at a feature of Java 17 and say - here, this is great and was not there before, it's worth using, then I'm all in.
>
> D.

absolute-bulk-get methods on Byte/Short/Int/Long/Float/DoubleBuffers?

I think we should investigate it for MMapDirectory and
ByteBuffersDirectory at least? Maybe it can create new opportunities,
e.g. reduce overhead vs position()+get(). Or maybe expand our
random-access API to include it, and perhaps bit-unpacking can be
simplified or sped up (e.g. DirectReader). Especially now that we have
varhandles it seems to make more things possible. Or maybe there's no
performance win for us and it only simplifies existing code in the
short-term.

I like the new PRNGs, maybe we should replace our handrolled
xorshift128 stuff that is used for segment IDs (see StringHelper). The
new API has nice set of algorithms:
https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/random/package-summary.html.
Good to look at for the HNSW vector stuff, too. Maybe, we should
switch over unit tests eventually too.

The JFR runtime streaming api looks interesting, maybe we could
improve tests.profile to use it, or mike's benchmark.
I like that they fixed multicast api to work correctly (IIRC
previously you had to implicitly bind to all interfaces, couldn't even
bind to just localhost). Theoretically it could be more efficient for
stuff like replication, but there's still practical issues (e.g. you
have to deal with UDP, some cloud environments have spotty support,
etc).
Unix Sockets! Now they work with windows, so java exposed them. I use
these heavily at work, curl --unix-socket is my GOTO. it's a nice
bonus you can protect them with ordinary file permissions on Linux at
least. I love apps like haproxy that expose stats/control interfaces
first-class over unix sockets. Infostream logging is nice, but maybe
we should provide other options to make it easy to get
metrics/statistics counters and such "live".
I like that the Unicode version is bumped, that helps the lucene
analyzers based on the JDK.
I'm also a fan of the HexFormat to replace any hand-rolled
hex-printers. Could probably clean up some test code at least.

There's a lot of little improvements to the API like this:
https://docs.oracle.com/en/java/javase/17/docs/api/new-list.html

As far as the language/major features, sure they are just sugar
sometime, but often it makes sense to refactor the code to use them.
E.G. having text block support, it is just one of those little things
that can make the code much easier. And I get spoiled by other
languages that all seem to have this.
There's a new packaging tool that might be appropriate for Luke, not sure.
Maybe lucene/expressions should use Hidden Classes? I can't remember
the details, but I think we make a private child classloader, register
the class there, to try to prevent GC hell. But why register it at
all?
The switch expressions looks interesting, because we could remove some
of the horrors of forgotten-break statements and stuff? Haven't looked
in detail, I think we are doing stuff with ecj to try to detect these
mistakes already. Seems potentially less error-prone to use the new
syntax.

You can easily see the full list of these language/major features since java 11:
https://openjdk.java.net/projects/jdk/12/
https://openjdk.java.net/projects/jdk/13/
https://openjdk.java.net/projects/jdk/14/
https://openjdk.java.net/projects/jdk/15/
https://openjdk.java.net/projects/jdk/16/
https://openjdk.java.net/projects/jdk/17/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>

Hi,

> I can imagine some users might like to keep abreast of main, at least
> in some kind of testing setup, but aren't ready to cut over their JDK
> for some reason (as Dawid was describing), but they can always do a
> small patch and build Lucene themselves, applying the -target for
> compatibility with their environments. And if we ever get to the point
> of really *requiring* compilation for the latest JDK, then it will
> probably be for a good reason, and we should prepare users for that.

I think you misunderstood what I said: We still have a stable branch where we backport our stuff to. So the main branch is really for "trying out and working on new stuff". That's what a main branch is thought for.

And if you read my original mail, that's what I am behind. We should focus on latest JDK with our main branch and try to optimize our APIs to use that, otherwise we will stay in a loop of ever supporting old bullshit forever.

If a change in the main branch is so exhaustive that we think: "oh this will annoy users", then it is the point to create a new stable branch before and release it as new major version, before the crazy stuff gets in. At this point, we provide users with some brand new version that works with the brand new JDK. What's wrong with that?

And before that we have the stable branch like we always did. If we can backport stuff we should do it. It may involve some nuking of JDK features we can't use in the stable branch, but this will bring pressure to the users: if you want newest features, upgrade your JDK. If you can't your stuck with old Lucene. But this is what our users are like: The conservative ones are also conservative with JDK versions. The hipster ones have hipster kubernetes clusters and don't care about the JDK version. They just install their app that fits their needs and the JDK is bundled.

We're far ahead Java 5 where people were complaining about generics (says the generics policeman) and everybody had their JBoss Tomcat running Slowlar inside with JDK 1.4.

Uwe

> On Thu, Nov 4, 2021 at 7:54 AM Robert Muir <rcmuir@gmail.com> wrote:
> >
> > With the current release frequency, JDK 11 will be EOL by the time
> > Lucene 10 is released.
> >
> > Users can use lucene 9 if they want to stay on old outdated JDKs.
> >
> > On Thu, Nov 4, 2021 at 7:44 AM David Smiley <dsmiley@apache.org> wrote:
> > >
> > > I prefer that we require JDK 17 for build/test but allow our artifacts (except
> lucene-test-framework maybe) to be run on JDK 11 (or 14?) via setting the
> "target". This allows us some time to appreciate some of the benefits of
> Java/JDK 17 without insisting that our users switch. This approach doesn't
> prevent us from fully-committing to JDK 17 for Lucene 10 if we want. When
> we consider that Lucene is a library and not a full app, we should be somewhat
> conservative here.
> > >
> > > ~ David Smiley
> > > Apache Lucene/Solr Search Developer
> > > http://www.linkedin.com/in/davidwsmiley
> > >
> > >
> > > On Thu, Nov 4, 2021 at 6:10 AM Uwe Schindler <uwe@thetaphi.de> wrote:
> > >>
> > >> Hi,
> > >>
> > >>
> > >>
> > >> I agree with this plan, lets go to JDK17 in Lucene 10 (main), but whenever
> a new Java version comes out, update Gradle and the –release=XX switch. Plain
> simple! The stable branch has a defined java version (currently “11”), “main”
> should be always latest. I don’t think this is a problem, because the Java release
> cycles have changed and people who are old-syled are still on 8 (so would be
> stuck with Lucene 8). For some larger companies they stick with officially
> “Oracle supported LTS” versions, but those people won’t upgrade soo.
> Nowadays with Docker and Kubernetes, it is so easy to start Solr or
> Elasticsearch with any Java version (and you don’t care, you just take what’s
> shipped with your image), so beeing bleeding edge on main is perfectly fine.
> When we release a new major version, we take what’s latest at that time
> (based on main branch, hopefully with Panama).
> > >>
> > >>
> > >>
> > >> Based on my previous statement, JDK 17 is not the final goal for Lucene
> 10, and not even 18 it is: JDK 18 won’t contain Panama (they have a second
> icubator of Total-Panama), so it is likely to be part of “java.base” Module in JDK
> 19 (still requiring some extra enabler-command line param).
> > >>
> > >>
> > >>
> > >> About 17: What I like most is the multiline-Strings and the new switch
> statement. In addition to Robert’s comment: I like it not only because of the
> break-hell, more because it is not a simple statement, but an expression
> (having return value). So the anti-pattern like a variable and then a switch
> stament assigning a value to this variable in each case is then finally obsolete.
> You have then “variable = switch(….)”. And finally we will get a switch for
> instanceofs a bit later (hopefully at same time when Panama comes out) ????
> > >>
> > >>
> > >>
> > >> Records are bullshit, sorry. It’s only useful for the
> Hibernate/Spring/Foobar-like Entities-For-Everything business logic. It may be
> useful at some point when they are no instances on heap anymore and just
> data wrappers, but based on classes I see no reason to use them for Lucene.
> > >>
> > >>
> > >>
> > >> Uwe
> > >>
> > >>
> > >>
> > >> -----
> > >>
> > >> Uwe Schindler
> > >>
> > >> Achterdiek 19, D-28357 Bremen
> > >>
> > >> https://www.thetaphi.de
> > >>
> > >> eMail: uwe@thetaphi.de
> > >>
> > >>
> > >>
> > >> From: Dawid Weiss <dawid.weiss@gmail.com>
> > >> Sent: Thursday, November 4, 2021 8:27 AM
> > >> To: Lucene Dev <dev@lucene.apache.org>
> > >> Subject: Re: Bump minimum Java version to 17 on main (10.0)
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> Now you're talking.
> > >>
> > >> +1.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> On Thu, Nov 4, 2021 at 1:49 AM Robert Muir <rcmuir@gmail.com> wrote:
> > >>
> > >> On Wed, Nov 3, 2021 at 1:36 PM Dawid Weiss <dawid.weiss@gmail.com>
> wrote:
> > >> >
> > >> > I principally agree with you - we should leverage new Java features and
> I'm all for it. I just don't see much difference between
> > >> > Java 11 and 17 in the context of Lucene... Upgrading for the sake of
> upgrading doesn't justify the move to
> > >> > me. But if you can point at a feature of Java 17 and say - here, this is
> great and was not there before, it's worth using, then I'm all in.
> > >> >
> > >> > D.
> > >>
> > >> absolute-bulk-get methods on Byte/Short/Int/Long/Float/DoubleBuffers?
> > >>
> > >> I think we should investigate it for MMapDirectory and
> > >> ByteBuffersDirectory at least? Maybe it can create new opportunities,
> > >> e.g. reduce overhead vs position()+get(). Or maybe expand our
> > >> random-access API to include it, and perhaps bit-unpacking can be
> > >> simplified or sped up (e.g. DirectReader). Especially now that we have
> > >> varhandles it seems to make more things possible. Or maybe there's no
> > >> performance win for us and it only simplifies existing code in the
> > >> short-term.
> > >>
> > >> I like the new PRNGs, maybe we should replace our handrolled
> > >> xorshift128 stuff that is used for segment IDs (see StringHelper). The
> > >> new API has nice set of algorithms:
> > >>
> https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/rando
> m/package-summary.html.
> > >> Good to look at for the HNSW vector stuff, too. Maybe, we should
> > >> switch over unit tests eventually too.
> > >>
> > >> The JFR runtime streaming api looks interesting, maybe we could
> > >> improve tests.profile to use it, or mike's benchmark.
> > >> I like that they fixed multicast api to work correctly (IIRC
> > >> previously you had to implicitly bind to all interfaces, couldn't even
> > >> bind to just localhost). Theoretically it could be more efficient for
> > >> stuff like replication, but there's still practical issues (e.g. you
> > >> have to deal with UDP, some cloud environments have spotty support,
> > >> etc).
> > >> Unix Sockets! Now they work with windows, so java exposed them. I use
> > >> these heavily at work, curl --unix-socket is my GOTO. it's a nice
> > >> bonus you can protect them with ordinary file permissions on Linux at
> > >> least. I love apps like haproxy that expose stats/control interfaces
> > >> first-class over unix sockets. Infostream logging is nice, but maybe
> > >> we should provide other options to make it easy to get
> > >> metrics/statistics counters and such "live".
> > >> I like that the Unicode version is bumped, that helps the lucene
> > >> analyzers based on the JDK.
> > >> I'm also a fan of the HexFormat to replace any hand-rolled
> > >> hex-printers. Could probably clean up some test code at least.
> > >>
> > >> There's a lot of little improvements to the API like this:
> > >> https://docs.oracle.com/en/java/javase/17/docs/api/new-list.html
> > >>
> > >> As far as the language/major features, sure they are just sugar
> > >> sometime, but often it makes sense to refactor the code to use them.
> > >> E.G. having text block support, it is just one of those little things
> > >> that can make the code much easier. And I get spoiled by other
> > >> languages that all seem to have this.
> > >> There's a new packaging tool that might be appropriate for Luke, not sure.
> > >> Maybe lucene/expressions should use Hidden Classes? I can't remember
> > >> the details, but I think we make a private child classloader, register
> > >> the class there, to try to prevent GC hell. But why register it at
> > >> all?
> > >> The switch expressions looks interesting, because we could remove some
> > >> of the horrors of forgotten-break statements and stuff? Haven't looked
> > >> in detail, I think we are doing stuff with ecj to try to detect these
> > >> mistakes already. Seems potentially less error-prone to use the new
> > >> syntax.
> > >>
> > >> You can easily see the full list of these language/major features since java
> 11:
> > >> https://openjdk.java.net/projects/jdk/12/
> > >> https://openjdk.java.net/projects/jdk/13/
> > >> https://openjdk.java.net/projects/jdk/14/
> > >> https://openjdk.java.net/projects/jdk/15/
> > >> https://openjdk.java.net/projects/jdk/16/
> > >> https://openjdk.java.net/projects/jdk/17/
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > >> For additional commands, e-mail: dev-help@lucene.apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

> I think you misunderstood what I said: We still have a stable branch where we backport our stuff to. So the main branch is really for "trying out and working on new stuff". That's what a main branch is thought for. ...

Sorry if I did. I am definitely mixing up -target and --release - old
ways die hard. Just to be clear - I am in support of tracking the
latest thing on main. I am also kind of curious to know what would
break if one changed the --release flag to 11 on some local build of
main, though.

On Thu, Nov 4, 2021 at 1:37 PM Uwe Schindler <uwe@thetaphi.de> wrote:
>
> Hi,
>
> > I can imagine some users might like to keep abreast of main, at least
> > in some kind of testing setup, but aren't ready to cut over their JDK
> > for some reason (as Dawid was describing), but they can always do a
> > small patch and build Lucene themselves, applying the -target for
> > compatibility with their environments. And if we ever get to the point
> > of really *requiring* compilation for the latest JDK, then it will
> > probably be for a good reason, and we should prepare users for that.
>
> I think you misunderstood what I said: We still have a stable branch where we backport our stuff to. So the main branch is really for "trying out and working on new stuff". That's what a main branch is thought for.
>
> And if you read my original mail, that's what I am behind. We should focus on latest JDK with our main branch and try to optimize our APIs to use that, otherwise we will stay in a loop of ever supporting old bullshit forever.
>
> If a change in the main branch is so exhaustive that we think: "oh this will annoy users", then it is the point to create a new stable branch before and release it as new major version, before the crazy stuff gets in. At this point, we provide users with some brand new version that works with the brand new JDK. What's wrong with that?
>
> And before that we have the stable branch like we always did. If we can backport stuff we should do it. It may involve some nuking of JDK features we can't use in the stable branch, but this will bring pressure to the users: if you want newest features, upgrade your JDK. If you can't your stuck with old Lucene. But this is what our users are like: The conservative ones are also conservative with JDK versions. The hipster ones have hipster kubernetes clusters and don't care about the JDK version. They just install their app that fits their needs and the JDK is bundled.
>
> We're far ahead Java 5 where people were complaining about generics (says the generics policeman) and everybody had their JBoss Tomcat running Slowlar inside with JDK 1.4.
>
> Uwe
>
> > On Thu, Nov 4, 2021 at 7:54 AM Robert Muir <rcmuir@gmail.com> wrote:
> > >
> > > With the current release frequency, JDK 11 will be EOL by the time
> > > Lucene 10 is released.
> > >
> > > Users can use lucene 9 if they want to stay on old outdated JDKs.
> > >
> > > On Thu, Nov 4, 2021 at 7:44 AM David Smiley <dsmiley@apache.org> wrote:
> > > >
> > > > I prefer that we require JDK 17 for build/test but allow our artifacts (except
> > lucene-test-framework maybe) to be run on JDK 11 (or 14?) via setting the
> > "target". This allows us some time to appreciate some of the benefits of
> > Java/JDK 17 without insisting that our users switch. This approach doesn't
> > prevent us from fully-committing to JDK 17 for Lucene 10 if we want. When
> > we consider that Lucene is a library and not a full app, we should be somewhat
> > conservative here.
> > > >
> > > > ~ David Smiley
> > > > Apache Lucene/Solr Search Developer
> > > > http://www.linkedin.com/in/davidwsmiley
> > > >
> > > >
> > > > On Thu, Nov 4, 2021 at 6:10 AM Uwe Schindler <uwe@thetaphi.de> wrote:
> > > >>
> > > >> Hi,
> > > >>
> > > >>
> > > >>
> > > >> I agree with this plan, lets go to JDK17 in Lucene 10 (main), but whenever
> > a new Java version comes out, update Gradle and the –release=XX switch. Plain
> > simple! The stable branch has a defined java version (currently “11”), “main”
> > should be always latest. I don’t think this is a problem, because the Java release
> > cycles have changed and people who are old-syled are still on 8 (so would be
> > stuck with Lucene 8). For some larger companies they stick with officially
> > “Oracle supported LTS” versions, but those people won’t upgrade soo.
> > Nowadays with Docker and Kubernetes, it is so easy to start Solr or
> > Elasticsearch with any Java version (and you don’t care, you just take what’s
> > shipped with your image), so beeing bleeding edge on main is perfectly fine.
> > When we release a new major version, we take what’s latest at that time
> > (based on main branch, hopefully with Panama).
> > > >>
> > > >>
> > > >>
> > > >> Based on my previous statement, JDK 17 is not the final goal for Lucene
> > 10, and not even 18 it is: JDK 18 won’t contain Panama (they have a second
> > icubator of Total-Panama), so it is likely to be part of “java.base” Module in JDK
> > 19 (still requiring some extra enabler-command line param).
> > > >>
> > > >>
> > > >>
> > > >> About 17: What I like most is the multiline-Strings and the new switch
> > statement. In addition to Robert’s comment: I like it not only because of the
> > break-hell, more because it is not a simple statement, but an expression
> > (having return value). So the anti-pattern like a variable and then a switch
> > stament assigning a value to this variable in each case is then finally obsolete.
> > You have then “variable = switch(….)”. And finally we will get a switch for
> > instanceofs a bit later (hopefully at same time when Panama comes out) ????
> > > >>
> > > >>
> > > >>
> > > >> Records are bullshit, sorry. It’s only useful for the
> > Hibernate/Spring/Foobar-like Entities-For-Everything business logic. It may be
> > useful at some point when they are no instances on heap anymore and just
> > data wrappers, but based on classes I see no reason to use them for Lucene.
> > > >>
> > > >>
> > > >>
> > > >> Uwe
> > > >>
> > > >>
> > > >>
> > > >> -----
> > > >>
> > > >> Uwe Schindler
> > > >>
> > > >> Achterdiek 19, D-28357 Bremen
> > > >>
> > > >> https://www.thetaphi.de
> > > >>
> > > >> eMail: uwe@thetaphi.de
> > > >>
> > > >>
> > > >>
> > > >> From: Dawid Weiss <dawid.weiss@gmail.com>
> > > >> Sent: Thursday, November 4, 2021 8:27 AM
> > > >> To: Lucene Dev <dev@lucene.apache.org>
> > > >> Subject: Re: Bump minimum Java version to 17 on main (10.0)
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> Now you're talking.
> > > >>
> > > >> +1.
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Thu, Nov 4, 2021 at 1:49 AM Robert Muir <rcmuir@gmail.com> wrote:
> > > >>
> > > >> On Wed, Nov 3, 2021 at 1:36 PM Dawid Weiss <dawid.weiss@gmail.com>
> > wrote:
> > > >> >
> > > >> > I principally agree with you - we should leverage new Java features and
> > I'm all for it. I just don't see much difference between
> > > >> > Java 11 and 17 in the context of Lucene... Upgrading for the sake of
> > upgrading doesn't justify the move to
> > > >> > me. But if you can point at a feature of Java 17 and say - here, this is
> > great and was not there before, it's worth using, then I'm all in.
> > > >> >
> > > >> > D.
> > > >>
> > > >> absolute-bulk-get methods on Byte/Short/Int/Long/Float/DoubleBuffers?
> > > >>
> > > >> I think we should investigate it for MMapDirectory and
> > > >> ByteBuffersDirectory at least? Maybe it can create new opportunities,
> > > >> e.g. reduce overhead vs position()+get(). Or maybe expand our
> > > >> random-access API to include it, and perhaps bit-unpacking can be
> > > >> simplified or sped up (e.g. DirectReader). Especially now that we have
> > > >> varhandles it seems to make more things possible. Or maybe there's no
> > > >> performance win for us and it only simplifies existing code in the
> > > >> short-term.
> > > >>
> > > >> I like the new PRNGs, maybe we should replace our handrolled
> > > >> xorshift128 stuff that is used for segment IDs (see StringHelper). The
> > > >> new API has nice set of algorithms:
> > > >>
> > https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/rando
> > m/package-summary.html.
> > > >> Good to look at for the HNSW vector stuff, too. Maybe, we should
> > > >> switch over unit tests eventually too.
> > > >>
> > > >> The JFR runtime streaming api looks interesting, maybe we could
> > > >> improve tests.profile to use it, or mike's benchmark.
> > > >> I like that they fixed multicast api to work correctly (IIRC
> > > >> previously you had to implicitly bind to all interfaces, couldn't even
> > > >> bind to just localhost). Theoretically it could be more efficient for
> > > >> stuff like replication, but there's still practical issues (e.g. you
> > > >> have to deal with UDP, some cloud environments have spotty support,
> > > >> etc).
> > > >> Unix Sockets! Now they work with windows, so java exposed them. I use
> > > >> these heavily at work, curl --unix-socket is my GOTO. it's a nice
> > > >> bonus you can protect them with ordinary file permissions on Linux at
> > > >> least. I love apps like haproxy that expose stats/control interfaces
> > > >> first-class over unix sockets. Infostream logging is nice, but maybe
> > > >> we should provide other options to make it easy to get
> > > >> metrics/statistics counters and such "live".
> > > >> I like that the Unicode version is bumped, that helps the lucene
> > > >> analyzers based on the JDK.
> > > >> I'm also a fan of the HexFormat to replace any hand-rolled
> > > >> hex-printers. Could probably clean up some test code at least.
> > > >>
> > > >> There's a lot of little improvements to the API like this:
> > > >> https://docs.oracle.com/en/java/javase/17/docs/api/new-list.html
> > > >>
> > > >> As far as the language/major features, sure they are just sugar
> > > >> sometime, but often it makes sense to refactor the code to use them.
> > > >> E.G. having text block support, it is just one of those little things
> > > >> that can make the code much easier. And I get spoiled by other
> > > >> languages that all seem to have this.
> > > >> There's a new packaging tool that might be appropriate for Luke, not sure.
> > > >> Maybe lucene/expressions should use Hidden Classes? I can't remember
> > > >> the details, but I think we make a private child classloader, register
> > > >> the class there, to try to prevent GC hell. But why register it at
> > > >> all?
> > > >> The switch expressions looks interesting, because we could remove some
> > > >> of the horrors of forgotten-break statements and stuff? Haven't looked
> > > >> in detail, I think we are doing stuff with ecj to try to detect these
> > > >> mistakes already. Seems potentially less error-prone to use the new
> > > >> syntax.
> > > >>
> > > >> You can easily see the full list of these language/major features since java
> > 11:
> > > >> https://openjdk.java.net/projects/jdk/12/
> > > >> https://openjdk.java.net/projects/jdk/13/
> > > >> https://openjdk.java.net/projects/jdk/14/
> > > >> https://openjdk.java.net/projects/jdk/15/
> > > >> https://openjdk.java.net/projects/jdk/16/
> > > >> https://openjdk.java.net/projects/jdk/17/
> > > >>
> > > >> ---------------------------------------------------------------------
> > > >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > >> For additional commands, e-mail: dev-help@lucene.apache.org
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: dev-help@lucene.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

On Thu, Nov 4, 2021 at 10:42 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>
> > we don't have to "support" users on our main trunk branch. That's why
> > we make releases.
>
> Nah. I don't agree with this one - you have to be aware about who is
> using your project (especially a library) and in what context. Maybe
> we differ in our opinion here. Your previous argument is much more
> convincing (thank you for laying it out so clearly).
>

Let's try again. First of all, I feel Uwe's reply says it perfectly,
so it is difficult to elaborate more.

But there should be a place to add "latest-and-greatest" changes, that
take advantage of whatever new slim pickings java will give us. This
allows the project to move forward: both in terms of new
opportunities, but also basic janitorial work: so you don't have a ton
of slow suboptimal code that doesn't "keep with the times".

This is what a 'main' branch is all about: it is the latest and
greatest. We have stable branches such as 'branch_9x' which stay
stable on their old java 11, and maybe we spin 10 releases for them
for 2.5 years just like we did for 8.x Plenty of bugfixes/improvements
can be backported to 'branch_9x', maybe they require some small
modifications because 'branch_9x' is on java 11. When this gets to be
painful it is a sign that lucene 10 is overdue, and we branch again :)

So it really shouldn't be controversial for the main branch to have
the latest dependencies and so on. But yet we suffer through this
battle *EVERY SINGLE MAJOR LUCENE RELEASE*. Like pulling teeth to bump
the jdk version so that we can improve its features. Doesn't make any
sense as the change won't have user impact for 2.5 years. In this
case, in 2.5 years (2023), JDK11 will actually be EOL already. No
sense in holding back 10.x like that.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

I opened https://issues.apache.org/jira/browse/LUCENE-10283.

On Fri, Nov 5, 2021 at 3:46 AM Robert Muir <rcmuir@gmail.com> wrote:
>
> On Thu, Nov 4, 2021 at 10:42 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> >
> > > we don't have to "support" users on our main trunk branch. That's why
> > > we make releases.
> >
> > Nah. I don't agree with this one - you have to be aware about who is
> > using your project (especially a library) and in what context. Maybe
> > we differ in our opinion here. Your previous argument is much more
> > convincing (thank you for laying it out so clearly).
> >
>
> Let's try again. First of all, I feel Uwe's reply says it perfectly,
> so it is difficult to elaborate more.
>
> But there should be a place to add "latest-and-greatest" changes, that
> take advantage of whatever new slim pickings java will give us. This
> allows the project to move forward: both in terms of new
> opportunities, but also basic janitorial work: so you don't have a ton
> of slow suboptimal code that doesn't "keep with the times".
>
> This is what a 'main' branch is all about: it is the latest and
> greatest. We have stable branches such as 'branch_9x' which stay
> stable on their old java 11, and maybe we spin 10 releases for them
> for 2.5 years just like we did for 8.x Plenty of bugfixes/improvements
> can be backported to 'branch_9x', maybe they require some small
> modifications because 'branch_9x' is on java 11. When this gets to be
> painful it is a sign that lucene 10 is overdue, and we branch again :)
>
> So it really shouldn't be controversial for the main branch to have
> the latest dependencies and so on. But yet we suffer through this
> battle *EVERY SINGLE MAJOR LUCENE RELEASE*. Like pulling teeth to bump
> the jdk version so that we can improve its features. Doesn't make any
> sense as the change won't have user impact for 2.5 years. In this
> case, in 2.5 years (2023), JDK11 will actually be EOL already. No
> sense in holding back 10.x like that.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>


--
Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org