Mailing List Archive

Lucene 9.4.0 release
Hi, I'd like to start the ball rolling for a 9.4.0 release. We don't
have a large number of things, but there are a few nice high-impact
changes; see excerpt from CHANGE.txt below. I'll volunteer to be
release manager. I'd just like to know if anybody has any outstanding
work that is nearing completion and that they'd like to include,
and/or if we have any known blocker issues*.

* -- I am having a little trouble searching github for blocker issues.
I think there may not be any, but I tried
https://github.com/apache/lucene/issues?q=is%3Aissue+is%3Aopen+legacy-jira-priority%5C%3AMajor+
to find Major issues, for example, and it seems to only find one Minor
one? Does anyone have better github-search-fu?

API Changes
---------------------
* LUCENE-10577: Add VectorEncoding to enable byte-encoded HNSW vectors
(Michael Sokolov, Julie Tibshirani)

New Features
---------------------
* LUCENE-10654: Add new ShapeDocValuesField for LatLonShape and
XYShape. (Nick Knize)

* LUCENE-10629: Support match set filtering with a query in
MatchingFacetSetCounts. (Stefan Vodita, Shai Erera)

* LUCENE-10633: SortField#setOptimizeSortWithIndexedData and
SortField#getOptimizeSortWithIndexedData were introduce to provide
an option to disable sort optimization for various sort fields.
(Mayya Sharipova)

Improvements
---------------------
* LUCENE-10592: Build HNSW Graph on indexing. (Mayya Sharipova, Adrien
Grand, Julie Tibshirani)

* GITHUB#11715: Add Integer awareness to RamUsageEstimator.sizeOf (Mike Drob)

Optimizations
---------------------

* LUCENE-10661: Reduce memory copy in BytesStore. (luyuncheng)

* GITHUB#1020: Support #scoreSupplier and small optimizations to
DocValuesRewriteMethod. (Greg Miller)

* LUCENE-10633: Added support for dynamic pruning to queries sorted by a string
field that is indexed with terms and SORTED or SORTED_SET doc values.
(Adrien Grand)

* LUCENE-10627: Using ByteBuffersDataInput reduce memory copy on
compressing data. (luyuncheng)

* GITHUB#1062: Optimize TermInSetQuery when a term is present that
matches all docs in a segment.
(Greg Miller)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Lucene 9.4.0 release [ In reply to ]
I believe we decided to use GitHub Milestone for release planning and I
wrote documentation about it.
https://github.com/apache/lucene/blob/main/dev-docs/github-issues-howto.md

Is this unclear to you?


2022?8?31?(?) 23:13 Michael Sokolov <msokolov@gmail.com>:

> Hi, I'd like to start the ball rolling for a 9.4.0 release. We don't
> have a large number of things, but there are a few nice high-impact
> changes; see excerpt from CHANGE.txt below. I'll volunteer to be
> release manager. I'd just like to know if anybody has any outstanding
> work that is nearing completion and that they'd like to include,
> and/or if we have any known blocker issues*.
>
> * -- I am having a little trouble searching github for blocker issues.
> I think there may not be any, but I tried
>
> https://github.com/apache/lucene/issues?q=is%3Aissue+is%3Aopen+legacy-jira-priority%5C%3AMajor+
> to find Major issues, for example, and it seems to only find one Minor
> one? Does anyone have better github-search-fu?
>
> API Changes
> ---------------------
> * LUCENE-10577: Add VectorEncoding to enable byte-encoded HNSW vectors
> (Michael Sokolov, Julie Tibshirani)
>
> New Features
> ---------------------
> * LUCENE-10654: Add new ShapeDocValuesField for LatLonShape and
> XYShape. (Nick Knize)
>
> * LUCENE-10629: Support match set filtering with a query in
> MatchingFacetSetCounts. (Stefan Vodita, Shai Erera)
>
> * LUCENE-10633: SortField#setOptimizeSortWithIndexedData and
> SortField#getOptimizeSortWithIndexedData were introduce to provide
> an option to disable sort optimization for various sort fields.
> (Mayya Sharipova)
>
> Improvements
> ---------------------
> * LUCENE-10592: Build HNSW Graph on indexing. (Mayya Sharipova, Adrien
> Grand, Julie Tibshirani)
>
> * GITHUB#11715: Add Integer awareness to RamUsageEstimator.sizeOf (Mike
> Drob)
>
> Optimizations
> ---------------------
>
> * LUCENE-10661: Reduce memory copy in BytesStore. (luyuncheng)
>
> * GITHUB#1020: Support #scoreSupplier and small optimizations to
> DocValuesRewriteMethod. (Greg Miller)
>
> * LUCENE-10633: Added support for dynamic pruning to queries sorted by a
> string
> field that is indexed with terms and SORTED or SORTED_SET doc values.
> (Adrien Grand)
>
> * LUCENE-10627: Using ByteBuffersDataInput reduce memory copy on
> compressing data. (luyuncheng)
>
> * GITHUB#1062: Optimize TermInSetQuery when a term is present that
> matches all docs in a segment.
> (Greg Miller)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: Lucene 9.4.0 release [ In reply to ]
Ok in short, you can find blockers for 9.4.0 here.
https://github.com/apache/lucene/milestone/3

If there are other issues/PRs that are still not resolved but planned to be
delivered in 9.4.0, just set their Milestone as 9.4.0. Then they will
appear in the list.



2022?9?1?(?) 0:00 Tomoko Uchida <tomoko.uchida.1111@gmail.com>:

> I believe we decided to use GitHub Milestone for release planning and I
> wrote documentation about it.
> https://github.com/apache/lucene/blob/main/dev-docs/github-issues-howto.md
>
> Is this unclear to you?
>
>
> 2022?8?31?(?) 23:13 Michael Sokolov <msokolov@gmail.com>:
>
>> Hi, I'd like to start the ball rolling for a 9.4.0 release. We don't
>> have a large number of things, but there are a few nice high-impact
>> changes; see excerpt from CHANGE.txt below. I'll volunteer to be
>> release manager. I'd just like to know if anybody has any outstanding
>> work that is nearing completion and that they'd like to include,
>> and/or if we have any known blocker issues*.
>>
>> * -- I am having a little trouble searching github for blocker issues.
>> I think there may not be any, but I tried
>>
>> https://github.com/apache/lucene/issues?q=is%3Aissue+is%3Aopen+legacy-jira-priority%5C%3AMajor+
>> to find Major issues, for example, and it seems to only find one Minor
>> one? Does anyone have better github-search-fu?
>>
>> API Changes
>> ---------------------
>> * LUCENE-10577: Add VectorEncoding to enable byte-encoded HNSW vectors
>> (Michael Sokolov, Julie Tibshirani)
>>
>> New Features
>> ---------------------
>> * LUCENE-10654: Add new ShapeDocValuesField for LatLonShape and
>> XYShape. (Nick Knize)
>>
>> * LUCENE-10629: Support match set filtering with a query in
>> MatchingFacetSetCounts. (Stefan Vodita, Shai Erera)
>>
>> * LUCENE-10633: SortField#setOptimizeSortWithIndexedData and
>> SortField#getOptimizeSortWithIndexedData were introduce to provide
>> an option to disable sort optimization for various sort fields.
>> (Mayya Sharipova)
>>
>> Improvements
>> ---------------------
>> * LUCENE-10592: Build HNSW Graph on indexing. (Mayya Sharipova, Adrien
>> Grand, Julie Tibshirani)
>>
>> * GITHUB#11715: Add Integer awareness to RamUsageEstimator.sizeOf (Mike
>> Drob)
>>
>> Optimizations
>> ---------------------
>>
>> * LUCENE-10661: Reduce memory copy in BytesStore. (luyuncheng)
>>
>> * GITHUB#1020: Support #scoreSupplier and small optimizations to
>> DocValuesRewriteMethod. (Greg Miller)
>>
>> * LUCENE-10633: Added support for dynamic pruning to queries sorted by a
>> string
>> field that is indexed with terms and SORTED or SORTED_SET doc values.
>> (Adrien Grand)
>>
>> * LUCENE-10627: Using ByteBuffersDataInput reduce memory copy on
>> compressing data. (luyuncheng)
>>
>> * GITHUB#1062: Optimize TermInSetQuery when a term is present that
>> matches all docs in a segment.
>> (Greg Miller)
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
Re: Lucene 9.4.0 release [ In reply to ]
Thanks for the links, Tomoko. I thought it would be helpful to ask on
the list in case this new system has not been fully internalized by
all the developers. And I confess I didn't know about that nice
milestone view! It makes this very easy, assuming of course that
everyone follows the guidelines and labels the issues with the
milestone.

On Wed, Aug 31, 2022 at 11:16 AM Tomoko Uchida
<tomoko.uchida.1111@gmail.com> wrote:
>
> Ok in short, you can find blockers for 9.4.0 here.
> https://github.com/apache/lucene/milestone/3
>
> If there are other issues/PRs that are still not resolved but planned to be delivered in 9.4.0, just set their Milestone as 9.4.0. Then they will appear in the list.
>
>
>
> 2022?9?1?(?) 0:00 Tomoko Uchida <tomoko.uchida.1111@gmail.com>:
>>
>> I believe we decided to use GitHub Milestone for release planning and I wrote documentation about it.
>> https://github.com/apache/lucene/blob/main/dev-docs/github-issues-howto.md
>>
>> Is this unclear to you?
>>
>>
>> 2022?8?31?(?) 23:13 Michael Sokolov <msokolov@gmail.com>:
>>>
>>> Hi, I'd like to start the ball rolling for a 9.4.0 release. We don't
>>> have a large number of things, but there are a few nice high-impact
>>> changes; see excerpt from CHANGE.txt below. I'll volunteer to be
>>> release manager. I'd just like to know if anybody has any outstanding
>>> work that is nearing completion and that they'd like to include,
>>> and/or if we have any known blocker issues*.
>>>
>>> * -- I am having a little trouble searching github for blocker issues.
>>> I think there may not be any, but I tried
>>> https://github.com/apache/lucene/issues?q=is%3Aissue+is%3Aopen+legacy-jira-priority%5C%3AMajor+
>>> to find Major issues, for example, and it seems to only find one Minor
>>> one? Does anyone have better github-search-fu?
>>>
>>> API Changes
>>> ---------------------
>>> * LUCENE-10577: Add VectorEncoding to enable byte-encoded HNSW vectors
>>> (Michael Sokolov, Julie Tibshirani)
>>>
>>> New Features
>>> ---------------------
>>> * LUCENE-10654: Add new ShapeDocValuesField for LatLonShape and
>>> XYShape. (Nick Knize)
>>>
>>> * LUCENE-10629: Support match set filtering with a query in
>>> MatchingFacetSetCounts. (Stefan Vodita, Shai Erera)
>>>
>>> * LUCENE-10633: SortField#setOptimizeSortWithIndexedData and
>>> SortField#getOptimizeSortWithIndexedData were introduce to provide
>>> an option to disable sort optimization for various sort fields.
>>> (Mayya Sharipova)
>>>
>>> Improvements
>>> ---------------------
>>> * LUCENE-10592: Build HNSW Graph on indexing. (Mayya Sharipova, Adrien
>>> Grand, Julie Tibshirani)
>>>
>>> * GITHUB#11715: Add Integer awareness to RamUsageEstimator.sizeOf (Mike Drob)
>>>
>>> Optimizations
>>> ---------------------
>>>
>>> * LUCENE-10661: Reduce memory copy in BytesStore. (luyuncheng)
>>>
>>> * GITHUB#1020: Support #scoreSupplier and small optimizations to
>>> DocValuesRewriteMethod. (Greg Miller)
>>>
>>> * LUCENE-10633: Added support for dynamic pruning to queries sorted by a string
>>> field that is indexed with terms and SORTED or SORTED_SET doc values.
>>> (Adrien Grand)
>>>
>>> * LUCENE-10627: Using ByteBuffersDataInput reduce memory copy on
>>> compressing data. (luyuncheng)
>>>
>>> * GITHUB#1062: Optimize TermInSetQuery when a term is present that
>>> matches all docs in a segment.
>>> (Greg Miller)
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Lucene 9.4.0 release [ In reply to ]
Hi Mike,
thanks for your response.

I'm actually not familiar with the whole release procedure that may be
built on Jira.
I hope other people help to interpret Jira-related things on the way into
the language of GitHub issues.

Tomoko

2022?9?1?(?) 3:40 Michael Sokolov <msokolov@gmail.com>:

> Thanks for the links, Tomoko. I thought it would be helpful to ask on
> the list in case this new system has not been fully internalized by
> all the developers. And I confess I didn't know about that nice
> milestone view! It makes this very easy, assuming of course that
> everyone follows the guidelines and labels the issues with the
> milestone.
>
> On Wed, Aug 31, 2022 at 11:16 AM Tomoko Uchida
> <tomoko.uchida.1111@gmail.com> wrote:
> >
> > Ok in short, you can find blockers for 9.4.0 here.
> > https://github.com/apache/lucene/milestone/3
> >
> > If there are other issues/PRs that are still not resolved but planned to
> be delivered in 9.4.0, just set their Milestone as 9.4.0. Then they will
> appear in the list.
> >
> >
> >
> > 2022?9?1?(?) 0:00 Tomoko Uchida <tomoko.uchida.1111@gmail.com>:
> >>
> >> I believe we decided to use GitHub Milestone for release planning and I
> wrote documentation about it.
> >>
> https://github.com/apache/lucene/blob/main/dev-docs/github-issues-howto.md
> >>
> >> Is this unclear to you?
> >>
> >>
> >> 2022?8?31?(?) 23:13 Michael Sokolov <msokolov@gmail.com>:
> >>>
> >>> Hi, I'd like to start the ball rolling for a 9.4.0 release. We don't
> >>> have a large number of things, but there are a few nice high-impact
> >>> changes; see excerpt from CHANGE.txt below. I'll volunteer to be
> >>> release manager. I'd just like to know if anybody has any outstanding
> >>> work that is nearing completion and that they'd like to include,
> >>> and/or if we have any known blocker issues*.
> >>>
> >>> * -- I am having a little trouble searching github for blocker issues.
> >>> I think there may not be any, but I tried
> >>>
> https://github.com/apache/lucene/issues?q=is%3Aissue+is%3Aopen+legacy-jira-priority%5C%3AMajor+
> >>> to find Major issues, for example, and it seems to only find one Minor
> >>> one? Does anyone have better github-search-fu?
> >>>
> >>> API Changes
> >>> ---------------------
> >>> * LUCENE-10577: Add VectorEncoding to enable byte-encoded HNSW vectors
> >>> (Michael Sokolov, Julie Tibshirani)
> >>>
> >>> New Features
> >>> ---------------------
> >>> * LUCENE-10654: Add new ShapeDocValuesField for LatLonShape and
> >>> XYShape. (Nick Knize)
> >>>
> >>> * LUCENE-10629: Support match set filtering with a query in
> >>> MatchingFacetSetCounts. (Stefan Vodita, Shai Erera)
> >>>
> >>> * LUCENE-10633: SortField#setOptimizeSortWithIndexedData and
> >>> SortField#getOptimizeSortWithIndexedData were introduce to provide
> >>> an option to disable sort optimization for various sort fields.
> >>> (Mayya Sharipova)
> >>>
> >>> Improvements
> >>> ---------------------
> >>> * LUCENE-10592: Build HNSW Graph on indexing. (Mayya Sharipova, Adrien
> >>> Grand, Julie Tibshirani)
> >>>
> >>> * GITHUB#11715: Add Integer awareness to RamUsageEstimator.sizeOf
> (Mike Drob)
> >>>
> >>> Optimizations
> >>> ---------------------
> >>>
> >>> * LUCENE-10661: Reduce memory copy in BytesStore. (luyuncheng)
> >>>
> >>> * GITHUB#1020: Support #scoreSupplier and small optimizations to
> >>> DocValuesRewriteMethod. (Greg Miller)
> >>>
> >>> * LUCENE-10633: Added support for dynamic pruning to queries sorted by
> a string
> >>> field that is indexed with terms and SORTED or SORTED_SET doc values.
> >>> (Adrien Grand)
> >>>
> >>> * LUCENE-10627: Using ByteBuffersDataInput reduce memory copy on
> >>> compressing data. (luyuncheng)
> >>>
> >>> * GITHUB#1062: Optimize TermInSetQuery when a term is present that
> >>> matches all docs in a segment.
> >>> (Greg Miller)
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>> For additional commands, e-mail: dev-help@lucene.apache.org
> >>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: Lucene 9.4.0 release [ In reply to ]
I started to go through the release process, which has been scripted very
clearly by Jan and others, and found Jira-specific steps there. I think the
translations to GitHub will be pretty straightforward. One thing is that
for JIra, we needed to acquire special permissions in order to be able to
add new release versions, but I don't think that is true for GitHub ... Any
committer should be able to do that, right?

On Wed, Aug 31, 2022, 8:42 PM Tomoko Uchida <tomoko.uchida.1111@gmail.com>
wrote:

> Hi Mike,
> thanks for your response.
>
> I'm actually not familiar with the whole release procedure that may be
> built on Jira.
> I hope other people help to interpret Jira-related things on the way into
> the language of GitHub issues.
>
> Tomoko
>
> 2022?9?1?(?) 3:40 Michael Sokolov <msokolov@gmail.com>:
>
>> Thanks for the links, Tomoko. I thought it would be helpful to ask on
>> the list in case this new system has not been fully internalized by
>> all the developers. And I confess I didn't know about that nice
>> milestone view! It makes this very easy, assuming of course that
>> everyone follows the guidelines and labels the issues with the
>> milestone.
>>
>> On Wed, Aug 31, 2022 at 11:16 AM Tomoko Uchida
>> <tomoko.uchida.1111@gmail.com> wrote:
>> >
>> > Ok in short, you can find blockers for 9.4.0 here.
>> > https://github.com/apache/lucene/milestone/3
>> >
>> > If there are other issues/PRs that are still not resolved but planned
>> to be delivered in 9.4.0, just set their Milestone as 9.4.0. Then they will
>> appear in the list.
>> >
>> >
>> >
>> > 2022?9?1?(?) 0:00 Tomoko Uchida <tomoko.uchida.1111@gmail.com>:
>> >>
>> >> I believe we decided to use GitHub Milestone for release planning and
>> I wrote documentation about it.
>> >>
>> https://github.com/apache/lucene/blob/main/dev-docs/github-issues-howto.md
>> >>
>> >> Is this unclear to you?
>> >>
>> >>
>> >> 2022?8?31?(?) 23:13 Michael Sokolov <msokolov@gmail.com>:
>> >>>
>> >>> Hi, I'd like to start the ball rolling for a 9.4.0 release. We don't
>> >>> have a large number of things, but there are a few nice high-impact
>> >>> changes; see excerpt from CHANGE.txt below. I'll volunteer to be
>> >>> release manager. I'd just like to know if anybody has any outstanding
>> >>> work that is nearing completion and that they'd like to include,
>> >>> and/or if we have any known blocker issues*.
>> >>>
>> >>> * -- I am having a little trouble searching github for blocker issues.
>> >>> I think there may not be any, but I tried
>> >>>
>> https://github.com/apache/lucene/issues?q=is%3Aissue+is%3Aopen+legacy-jira-priority%5C%3AMajor+
>> >>> to find Major issues, for example, and it seems to only find one Minor
>> >>> one? Does anyone have better github-search-fu?
>> >>>
>> >>> API Changes
>> >>> ---------------------
>> >>> * LUCENE-10577: Add VectorEncoding to enable byte-encoded HNSW vectors
>> >>> (Michael Sokolov, Julie Tibshirani)
>> >>>
>> >>> New Features
>> >>> ---------------------
>> >>> * LUCENE-10654: Add new ShapeDocValuesField for LatLonShape and
>> >>> XYShape. (Nick Knize)
>> >>>
>> >>> * LUCENE-10629: Support match set filtering with a query in
>> >>> MatchingFacetSetCounts. (Stefan Vodita, Shai Erera)
>> >>>
>> >>> * LUCENE-10633: SortField#setOptimizeSortWithIndexedData and
>> >>> SortField#getOptimizeSortWithIndexedData were introduce to provide
>> >>> an option to disable sort optimization for various sort fields.
>> >>> (Mayya Sharipova)
>> >>>
>> >>> Improvements
>> >>> ---------------------
>> >>> * LUCENE-10592: Build HNSW Graph on indexing. (Mayya Sharipova, Adrien
>> >>> Grand, Julie Tibshirani)
>> >>>
>> >>> * GITHUB#11715: Add Integer awareness to RamUsageEstimator.sizeOf
>> (Mike Drob)
>> >>>
>> >>> Optimizations
>> >>> ---------------------
>> >>>
>> >>> * LUCENE-10661: Reduce memory copy in BytesStore. (luyuncheng)
>> >>>
>> >>> * GITHUB#1020: Support #scoreSupplier and small optimizations to
>> >>> DocValuesRewriteMethod. (Greg Miller)
>> >>>
>> >>> * LUCENE-10633: Added support for dynamic pruning to queries sorted
>> by a string
>> >>> field that is indexed with terms and SORTED or SORTED_SET doc
>> values.
>> >>> (Adrien Grand)
>> >>>
>> >>> * LUCENE-10627: Using ByteBuffersDataInput reduce memory copy on
>> >>> compressing data. (luyuncheng)
>> >>>
>> >>> * GITHUB#1062: Optimize TermInSetQuery when a term is present that
>> >>> matches all docs in a segment.
>> >>> (Greg Miller)
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >>> For additional commands, e-mail: dev-help@lucene.apache.org
>> >>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
Re: Lucene 9.4.0 release [ In reply to ]
> One thing is that for JIra, we needed to acquire special permissions in
order to be able to add new release versions, but I don't think that is
true for GitHub ...

You do not need any special permissions to make new Milestones on GitHub.
Every committer already has permission to create/close/delete Milestones,
you can test it here.
https://github.com/apache/lucene/milestones

Tomoko


2022?9?1?(?) 9:57 Michael Sokolov <msokolov@gmail.com>:

> I started to go through the release process, which has been scripted very
> clearly by Jan and others, and found Jira-specific steps there. I think the
> translations to GitHub will be pretty straightforward. One thing is that
> for JIra, we needed to acquire special permissions in order to be able to
> add new release versions, but I don't think that is true for GitHub ... Any
> committer should be able to do that, right?
>
> On Wed, Aug 31, 2022, 8:42 PM Tomoko Uchida <tomoko.uchida.1111@gmail.com>
> wrote:
>
>> Hi Mike,
>> thanks for your response.
>>
>> I'm actually not familiar with the whole release procedure that may be
>> built on Jira.
>> I hope other people help to interpret Jira-related things on the way into
>> the language of GitHub issues.
>>
>> Tomoko
>>
>> 2022?9?1?(?) 3:40 Michael Sokolov <msokolov@gmail.com>:
>>
>>> Thanks for the links, Tomoko. I thought it would be helpful to ask on
>>> the list in case this new system has not been fully internalized by
>>> all the developers. And I confess I didn't know about that nice
>>> milestone view! It makes this very easy, assuming of course that
>>> everyone follows the guidelines and labels the issues with the
>>> milestone.
>>>
>>> On Wed, Aug 31, 2022 at 11:16 AM Tomoko Uchida
>>> <tomoko.uchida.1111@gmail.com> wrote:
>>> >
>>> > Ok in short, you can find blockers for 9.4.0 here.
>>> > https://github.com/apache/lucene/milestone/3
>>> >
>>> > If there are other issues/PRs that are still not resolved but planned
>>> to be delivered in 9.4.0, just set their Milestone as 9.4.0. Then they will
>>> appear in the list.
>>> >
>>> >
>>> >
>>> > 2022?9?1?(?) 0:00 Tomoko Uchida <tomoko.uchida.1111@gmail.com>:
>>> >>
>>> >> I believe we decided to use GitHub Milestone for release planning and
>>> I wrote documentation about it.
>>> >>
>>> https://github.com/apache/lucene/blob/main/dev-docs/github-issues-howto.md
>>> >>
>>> >> Is this unclear to you?
>>> >>
>>> >>
>>> >> 2022?8?31?(?) 23:13 Michael Sokolov <msokolov@gmail.com>:
>>> >>>
>>> >>> Hi, I'd like to start the ball rolling for a 9.4.0 release. We don't
>>> >>> have a large number of things, but there are a few nice high-impact
>>> >>> changes; see excerpt from CHANGE.txt below. I'll volunteer to be
>>> >>> release manager. I'd just like to know if anybody has any outstanding
>>> >>> work that is nearing completion and that they'd like to include,
>>> >>> and/or if we have any known blocker issues*.
>>> >>>
>>> >>> * -- I am having a little trouble searching github for blocker
>>> issues.
>>> >>> I think there may not be any, but I tried
>>> >>>
>>> https://github.com/apache/lucene/issues?q=is%3Aissue+is%3Aopen+legacy-jira-priority%5C%3AMajor+
>>> >>> to find Major issues, for example, and it seems to only find one
>>> Minor
>>> >>> one? Does anyone have better github-search-fu?
>>> >>>
>>> >>> API Changes
>>> >>> ---------------------
>>> >>> * LUCENE-10577: Add VectorEncoding to enable byte-encoded HNSW
>>> vectors
>>> >>> (Michael Sokolov, Julie Tibshirani)
>>> >>>
>>> >>> New Features
>>> >>> ---------------------
>>> >>> * LUCENE-10654: Add new ShapeDocValuesField for LatLonShape and
>>> >>> XYShape. (Nick Knize)
>>> >>>
>>> >>> * LUCENE-10629: Support match set filtering with a query in
>>> >>> MatchingFacetSetCounts. (Stefan Vodita, Shai Erera)
>>> >>>
>>> >>> * LUCENE-10633: SortField#setOptimizeSortWithIndexedData and
>>> >>> SortField#getOptimizeSortWithIndexedData were introduce to provide
>>> >>> an option to disable sort optimization for various sort fields.
>>> >>> (Mayya Sharipova)
>>> >>>
>>> >>> Improvements
>>> >>> ---------------------
>>> >>> * LUCENE-10592: Build HNSW Graph on indexing. (Mayya Sharipova,
>>> Adrien
>>> >>> Grand, Julie Tibshirani)
>>> >>>
>>> >>> * GITHUB#11715: Add Integer awareness to RamUsageEstimator.sizeOf
>>> (Mike Drob)
>>> >>>
>>> >>> Optimizations
>>> >>> ---------------------
>>> >>>
>>> >>> * LUCENE-10661: Reduce memory copy in BytesStore. (luyuncheng)
>>> >>>
>>> >>> * GITHUB#1020: Support #scoreSupplier and small optimizations to
>>> >>> DocValuesRewriteMethod. (Greg Miller)
>>> >>>
>>> >>> * LUCENE-10633: Added support for dynamic pruning to queries sorted
>>> by a string
>>> >>> field that is indexed with terms and SORTED or SORTED_SET doc
>>> values.
>>> >>> (Adrien Grand)
>>> >>>
>>> >>> * LUCENE-10627: Using ByteBuffersDataInput reduce memory copy on
>>> >>> compressing data. (luyuncheng)
>>> >>>
>>> >>> * GITHUB#1062: Optimize TermInSetQuery when a term is present that
>>> >>> matches all docs in a segment.
>>> >>> (Greg Miller)
>>> >>>
>>> >>> ---------------------------------------------------------------------
>>> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> >>> For additional commands, e-mail: dev-help@lucene.apache.org
>>> >>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
Re: Lucene 9.4.0 release [ In reply to ]
Hi Mike,
to be precise, on GitHub I think release managers would go through these
steps:

1. Figure out blocker issues/PRs and associate them with the target
Milestone, e.g., "9.4.0"
2. Create a new Milestone for the next version here
<https://github.com/apache/lucene/milestones>, e.g., "9.5.0"
3. Address blockers - some may be resolved by developers, and others may be
re-associated with the next Milestone.
4. Make release artifacts and publish them as usual.
5. Close (not Delete!) the target Milestone.

You need no special permission on GitHub to handle Milestones, and
everything should be fairly straightforward.
If you need any help, I'd be happy to review the necessary updates on the
release wizard tool. I have little time to make a patch for that myself
right now, sorry.

Tomoko


2022?9?1?(?) 10:02 Tomoko Uchida <tomoko.uchida.1111@gmail.com>:

> > One thing is that for JIra, we needed to acquire special permissions in
> order to be able to add new release versions, but I don't think that is
> true for GitHub ...
>
> You do not need any special permissions to make new Milestones on GitHub.
> Every committer already has permission to create/close/delete Milestones,
> you can test it here.
> https://github.com/apache/lucene/milestones
>
> Tomoko
>
>
> 2022?9?1?(?) 9:57 Michael Sokolov <msokolov@gmail.com>:
>
>> I started to go through the release process, which has been scripted very
>> clearly by Jan and others, and found Jira-specific steps there. I think the
>> translations to GitHub will be pretty straightforward. One thing is that
>> for JIra, we needed to acquire special permissions in order to be able to
>> add new release versions, but I don't think that is true for GitHub ... Any
>> committer should be able to do that, right?
>>
>> On Wed, Aug 31, 2022, 8:42 PM Tomoko Uchida <tomoko.uchida.1111@gmail.com>
>> wrote:
>>
>>> Hi Mike,
>>> thanks for your response.
>>>
>>> I'm actually not familiar with the whole release procedure that may be
>>> built on Jira.
>>> I hope other people help to interpret Jira-related things on the way
>>> into the language of GitHub issues.
>>>
>>> Tomoko
>>>
>>> 2022?9?1?(?) 3:40 Michael Sokolov <msokolov@gmail.com>:
>>>
>>>> Thanks for the links, Tomoko. I thought it would be helpful to ask on
>>>> the list in case this new system has not been fully internalized by
>>>> all the developers. And I confess I didn't know about that nice
>>>> milestone view! It makes this very easy, assuming of course that
>>>> everyone follows the guidelines and labels the issues with the
>>>> milestone.
>>>>
>>>> On Wed, Aug 31, 2022 at 11:16 AM Tomoko Uchida
>>>> <tomoko.uchida.1111@gmail.com> wrote:
>>>> >
>>>> > Ok in short, you can find blockers for 9.4.0 here.
>>>> > https://github.com/apache/lucene/milestone/3
>>>> >
>>>> > If there are other issues/PRs that are still not resolved but planned
>>>> to be delivered in 9.4.0, just set their Milestone as 9.4.0. Then they will
>>>> appear in the list.
>>>> >
>>>> >
>>>> >
>>>> > 2022?9?1?(?) 0:00 Tomoko Uchida <tomoko.uchida.1111@gmail.com>:
>>>> >>
>>>> >> I believe we decided to use GitHub Milestone for release planning
>>>> and I wrote documentation about it.
>>>> >>
>>>> https://github.com/apache/lucene/blob/main/dev-docs/github-issues-howto.md
>>>> >>
>>>> >> Is this unclear to you?
>>>> >>
>>>> >>
>>>> >> 2022?8?31?(?) 23:13 Michael Sokolov <msokolov@gmail.com>:
>>>> >>>
>>>> >>> Hi, I'd like to start the ball rolling for a 9.4.0 release. We don't
>>>> >>> have a large number of things, but there are a few nice high-impact
>>>> >>> changes; see excerpt from CHANGE.txt below. I'll volunteer to be
>>>> >>> release manager. I'd just like to know if anybody has any
>>>> outstanding
>>>> >>> work that is nearing completion and that they'd like to include,
>>>> >>> and/or if we have any known blocker issues*.
>>>> >>>
>>>> >>> * -- I am having a little trouble searching github for blocker
>>>> issues.
>>>> >>> I think there may not be any, but I tried
>>>> >>>
>>>> https://github.com/apache/lucene/issues?q=is%3Aissue+is%3Aopen+legacy-jira-priority%5C%3AMajor+
>>>> >>> to find Major issues, for example, and it seems to only find one
>>>> Minor
>>>> >>> one? Does anyone have better github-search-fu?
>>>> >>>
>>>> >>> API Changes
>>>> >>> ---------------------
>>>> >>> * LUCENE-10577: Add VectorEncoding to enable byte-encoded HNSW
>>>> vectors
>>>> >>> (Michael Sokolov, Julie Tibshirani)
>>>> >>>
>>>> >>> New Features
>>>> >>> ---------------------
>>>> >>> * LUCENE-10654: Add new ShapeDocValuesField for LatLonShape and
>>>> >>> XYShape. (Nick Knize)
>>>> >>>
>>>> >>> * LUCENE-10629: Support match set filtering with a query in
>>>> >>> MatchingFacetSetCounts. (Stefan Vodita, Shai Erera)
>>>> >>>
>>>> >>> * LUCENE-10633: SortField#setOptimizeSortWithIndexedData and
>>>> >>> SortField#getOptimizeSortWithIndexedData were introduce to provide
>>>> >>> an option to disable sort optimization for various sort fields.
>>>> >>> (Mayya Sharipova)
>>>> >>>
>>>> >>> Improvements
>>>> >>> ---------------------
>>>> >>> * LUCENE-10592: Build HNSW Graph on indexing. (Mayya Sharipova,
>>>> Adrien
>>>> >>> Grand, Julie Tibshirani)
>>>> >>>
>>>> >>> * GITHUB#11715: Add Integer awareness to RamUsageEstimator.sizeOf
>>>> (Mike Drob)
>>>> >>>
>>>> >>> Optimizations
>>>> >>> ---------------------
>>>> >>>
>>>> >>> * LUCENE-10661: Reduce memory copy in BytesStore. (luyuncheng)
>>>> >>>
>>>> >>> * GITHUB#1020: Support #scoreSupplier and small optimizations to
>>>> >>> DocValuesRewriteMethod. (Greg Miller)
>>>> >>>
>>>> >>> * LUCENE-10633: Added support for dynamic pruning to queries sorted
>>>> by a string
>>>> >>> field that is indexed with terms and SORTED or SORTED_SET doc
>>>> values.
>>>> >>> (Adrien Grand)
>>>> >>>
>>>> >>> * LUCENE-10627: Using ByteBuffersDataInput reduce memory copy on
>>>> >>> compressing data. (luyuncheng)
>>>> >>>
>>>> >>> * GITHUB#1062: Optimize TermInSetQuery when a term is present that
>>>> >>> matches all docs in a segment.
>>>> >>> (Greg Miller)
>>>> >>>
>>>> >>>
>>>> ---------------------------------------------------------------------
>>>> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> >>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>> >>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>
>>>>
Re: Lucene 9.4.0 release [ In reply to ]
Thanks Tomoko - I appreciate the offer to review the changes needed. I
will take care of updating the release script/template.

I think I managed to get a GPG key registered and signed (thanks Mike
McCandless!) although we'll see if this all stands up to scrutiny!
It's quite tricky to navigate this setup, seems very secure to me, but
what do I know. Anyway the releaseWizard seems to be satisfied.

So -- I plan to cut a branch_9_4 release branch tomorrow, Sept 2 and
late it bake for a week, until Sept 7, after which point it will be
frozen for release. Does that seem like enough time? If you have any
concerns, please reply here.

On Thu, Sep 1, 2022 at 7:46 AM Tomoko Uchida
<tomoko.uchida.1111@gmail.com> wrote:
>
> Hi Mike,
> to be precise, on GitHub I think release managers would go through these steps:
>
> 1. Figure out blocker issues/PRs and associate them with the target Milestone, e.g., "9.4.0"
> 2. Create a new Milestone for the next version here, e.g., "9.5.0"
> 3. Address blockers - some may be resolved by developers, and others may be re-associated with the next Milestone.
> 4. Make release artifacts and publish them as usual.
> 5. Close (not Delete!) the target Milestone.
>
> You need no special permission on GitHub to handle Milestones, and everything should be fairly straightforward.
> If you need any help, I'd be happy to review the necessary updates on the release wizard tool. I have little time to make a patch for that myself right now, sorry.
>
> Tomoko
>
>
> 2022?9?1?(?) 10:02 Tomoko Uchida <tomoko.uchida.1111@gmail.com>:
>>
>> > One thing is that for JIra, we needed to acquire special permissions in order to be able to add new release versions, but I don't think that is true for GitHub ...
>>
>> You do not need any special permissions to make new Milestones on GitHub. Every committer already has permission to create/close/delete Milestones, you can test it here.
>> https://github.com/apache/lucene/milestones
>>
>> Tomoko
>>
>>
>> 2022?9?1?(?) 9:57 Michael Sokolov <msokolov@gmail.com>:
>>>
>>> I started to go through the release process, which has been scripted very clearly by Jan and others, and found Jira-specific steps there. I think the translations to GitHub will be pretty straightforward. One thing is that for JIra, we needed to acquire special permissions in order to be able to add new release versions, but I don't think that is true for GitHub ... Any committer should be able to do that, right?
>>>
>>> On Wed, Aug 31, 2022, 8:42 PM Tomoko Uchida <tomoko.uchida.1111@gmail.com> wrote:
>>>>
>>>> Hi Mike,
>>>> thanks for your response.
>>>>
>>>> I'm actually not familiar with the whole release procedure that may be built on Jira.
>>>> I hope other people help to interpret Jira-related things on the way into the language of GitHub issues.
>>>>
>>>> Tomoko
>>>>
>>>> 2022?9?1?(?) 3:40 Michael Sokolov <msokolov@gmail.com>:
>>>>>
>>>>> Thanks for the links, Tomoko. I thought it would be helpful to ask on
>>>>> the list in case this new system has not been fully internalized by
>>>>> all the developers. And I confess I didn't know about that nice
>>>>> milestone view! It makes this very easy, assuming of course that
>>>>> everyone follows the guidelines and labels the issues with the
>>>>> milestone.
>>>>>
>>>>> On Wed, Aug 31, 2022 at 11:16 AM Tomoko Uchida
>>>>> <tomoko.uchida.1111@gmail.com> wrote:
>>>>> >
>>>>> > Ok in short, you can find blockers for 9.4.0 here.
>>>>> > https://github.com/apache/lucene/milestone/3
>>>>> >
>>>>> > If there are other issues/PRs that are still not resolved but planned to be delivered in 9.4.0, just set their Milestone as 9.4.0. Then they will appear in the list.
>>>>> >
>>>>> >
>>>>> >
>>>>> > 2022?9?1?(?) 0:00 Tomoko Uchida <tomoko.uchida.1111@gmail.com>:
>>>>> >>
>>>>> >> I believe we decided to use GitHub Milestone for release planning and I wrote documentation about it.
>>>>> >> https://github.com/apache/lucene/blob/main/dev-docs/github-issues-howto.md
>>>>> >>
>>>>> >> Is this unclear to you?
>>>>> >>
>>>>> >>
>>>>> >> 2022?8?31?(?) 23:13 Michael Sokolov <msokolov@gmail.com>:
>>>>> >>>
>>>>> >>> Hi, I'd like to start the ball rolling for a 9.4.0 release. We don't
>>>>> >>> have a large number of things, but there are a few nice high-impact
>>>>> >>> changes; see excerpt from CHANGE.txt below. I'll volunteer to be
>>>>> >>> release manager. I'd just like to know if anybody has any outstanding
>>>>> >>> work that is nearing completion and that they'd like to include,
>>>>> >>> and/or if we have any known blocker issues*.
>>>>> >>>
>>>>> >>> * -- I am having a little trouble searching github for blocker issues.
>>>>> >>> I think there may not be any, but I tried
>>>>> >>> https://github.com/apache/lucene/issues?q=is%3Aissue+is%3Aopen+legacy-jira-priority%5C%3AMajor+
>>>>> >>> to find Major issues, for example, and it seems to only find one Minor
>>>>> >>> one? Does anyone have better github-search-fu?
>>>>> >>>
>>>>> >>> API Changes
>>>>> >>> ---------------------
>>>>> >>> * LUCENE-10577: Add VectorEncoding to enable byte-encoded HNSW vectors
>>>>> >>> (Michael Sokolov, Julie Tibshirani)
>>>>> >>>
>>>>> >>> New Features
>>>>> >>> ---------------------
>>>>> >>> * LUCENE-10654: Add new ShapeDocValuesField for LatLonShape and
>>>>> >>> XYShape. (Nick Knize)
>>>>> >>>
>>>>> >>> * LUCENE-10629: Support match set filtering with a query in
>>>>> >>> MatchingFacetSetCounts. (Stefan Vodita, Shai Erera)
>>>>> >>>
>>>>> >>> * LUCENE-10633: SortField#setOptimizeSortWithIndexedData and
>>>>> >>> SortField#getOptimizeSortWithIndexedData were introduce to provide
>>>>> >>> an option to disable sort optimization for various sort fields.
>>>>> >>> (Mayya Sharipova)
>>>>> >>>
>>>>> >>> Improvements
>>>>> >>> ---------------------
>>>>> >>> * LUCENE-10592: Build HNSW Graph on indexing. (Mayya Sharipova, Adrien
>>>>> >>> Grand, Julie Tibshirani)
>>>>> >>>
>>>>> >>> * GITHUB#11715: Add Integer awareness to RamUsageEstimator.sizeOf (Mike Drob)
>>>>> >>>
>>>>> >>> Optimizations
>>>>> >>> ---------------------
>>>>> >>>
>>>>> >>> * LUCENE-10661: Reduce memory copy in BytesStore. (luyuncheng)
>>>>> >>>
>>>>> >>> * GITHUB#1020: Support #scoreSupplier and small optimizations to
>>>>> >>> DocValuesRewriteMethod. (Greg Miller)
>>>>> >>>
>>>>> >>> * LUCENE-10633: Added support for dynamic pruning to queries sorted by a string
>>>>> >>> field that is indexed with terms and SORTED or SORTED_SET doc values.
>>>>> >>> (Adrien Grand)
>>>>> >>>
>>>>> >>> * LUCENE-10627: Using ByteBuffersDataInput reduce memory copy on
>>>>> >>> compressing data. (luyuncheng)
>>>>> >>>
>>>>> >>> * GITHUB#1062: Optimize TermInSetQuery when a term is present that
>>>>> >>> matches all docs in a segment.
>>>>> >>> (Greg Miller)
>>>>> >>>
>>>>> >>> ---------------------------------------------------------------------
>>>>> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> >>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>> >>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Lucene 9.4.0 release [ In reply to ]
Hello Michael,
thanks for starting the discussion for the 9.4.0 release.

We are investigating the impact of LUCENE-10592 (Build HNSW Graph on
indexing) on memory used during indexing.
In our Elasticsearch tests, we have observed much more segments being
created.
It would be nice if we have an extra week for feature freeze (considering
extended holidays in North America).
Thank you.


On Thu, Sep 1, 2022 at 11:04 AM Michael Sokolov <msokolov@gmail.com> wrote:

> Thanks Tomoko - I appreciate the offer to review the changes needed. I
> will take care of updating the release script/template.
>
> I think I managed to get a GPG key registered and signed (thanks Mike
> McCandless!) although we'll see if this all stands up to scrutiny!
> It's quite tricky to navigate this setup, seems very secure to me, but
> what do I know. Anyway the releaseWizard seems to be satisfied.
>
> So -- I plan to cut a branch_9_4 release branch tomorrow, Sept 2 and
> late it bake for a week, until Sept 7, after which point it will be
> frozen for release. Does that seem like enough time? If you have any
> concerns, please reply here.
>
> On Thu, Sep 1, 2022 at 7:46 AM Tomoko Uchida
> <tomoko.uchida.1111@gmail.com> wrote:
> >
> > Hi Mike,
> > to be precise, on GitHub I think release managers would go through these
> steps:
> >
> > 1. Figure out blocker issues/PRs and associate them with the target
> Milestone, e.g., "9.4.0"
> > 2. Create a new Milestone for the next version here, e.g., "9.5.0"
> > 3. Address blockers - some may be resolved by developers, and others may
> be re-associated with the next Milestone.
> > 4. Make release artifacts and publish them as usual.
> > 5. Close (not Delete!) the target Milestone.
> >
> > You need no special permission on GitHub to handle Milestones, and
> everything should be fairly straightforward.
> > If you need any help, I'd be happy to review the necessary updates on
> the release wizard tool. I have little time to make a patch for that myself
> right now, sorry.
> >
> > Tomoko
> >
> >
> > 2022?9?1?(?) 10:02 Tomoko Uchida <tomoko.uchida.1111@gmail.com>:
> >>
> >> > One thing is that for JIra, we needed to acquire special permissions
> in order to be able to add new release versions, but I don't think that is
> true for GitHub ...
> >>
> >> You do not need any special permissions to make new Milestones on
> GitHub. Every committer already has permission to create/close/delete
> Milestones, you can test it here.
> >> https://github.com/apache/lucene/milestones
> >>
> >> Tomoko
> >>
> >>
> >> 2022?9?1?(?) 9:57 Michael Sokolov <msokolov@gmail.com>:
> >>>
> >>> I started to go through the release process, which has been scripted
> very clearly by Jan and others, and found Jira-specific steps there. I
> think the translations to GitHub will be pretty straightforward. One thing
> is that for JIra, we needed to acquire special permissions in order to be
> able to add new release versions, but I don't think that is true for GitHub
> ... Any committer should be able to do that, right?
> >>>
> >>> On Wed, Aug 31, 2022, 8:42 PM Tomoko Uchida <
> tomoko.uchida.1111@gmail.com> wrote:
> >>>>
> >>>> Hi Mike,
> >>>> thanks for your response.
> >>>>
> >>>> I'm actually not familiar with the whole release procedure that may
> be built on Jira.
> >>>> I hope other people help to interpret Jira-related things on the way
> into the language of GitHub issues.
> >>>>
> >>>> Tomoko
> >>>>
> >>>> 2022?9?1?(?) 3:40 Michael Sokolov <msokolov@gmail.com>:
> >>>>>
> >>>>> Thanks for the links, Tomoko. I thought it would be helpful to ask on
> >>>>> the list in case this new system has not been fully internalized by
> >>>>> all the developers. And I confess I didn't know about that nice
> >>>>> milestone view! It makes this very easy, assuming of course that
> >>>>> everyone follows the guidelines and labels the issues with the
> >>>>> milestone.
> >>>>>
> >>>>> On Wed, Aug 31, 2022 at 11:16 AM Tomoko Uchida
> >>>>> <tomoko.uchida.1111@gmail.com> wrote:
> >>>>> >
> >>>>> > Ok in short, you can find blockers for 9.4.0 here.
> >>>>> > https://github.com/apache/lucene/milestone/3
> >>>>> >
> >>>>> > If there are other issues/PRs that are still not resolved but
> planned to be delivered in 9.4.0, just set their Milestone as 9.4.0. Then
> they will appear in the list.
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> > 2022?9?1?(?) 0:00 Tomoko Uchida <tomoko.uchida.1111@gmail.com>:
> >>>>> >>
> >>>>> >> I believe we decided to use GitHub Milestone for release planning
> and I wrote documentation about it.
> >>>>> >>
> https://github.com/apache/lucene/blob/main/dev-docs/github-issues-howto.md
> >>>>> >>
> >>>>> >> Is this unclear to you?
> >>>>> >>
> >>>>> >>
> >>>>> >> 2022?8?31?(?) 23:13 Michael Sokolov <msokolov@gmail.com>:
> >>>>> >>>
> >>>>> >>> Hi, I'd like to start the ball rolling for a 9.4.0 release. We
> don't
> >>>>> >>> have a large number of things, but there are a few nice
> high-impact
> >>>>> >>> changes; see excerpt from CHANGE.txt below. I'll volunteer to be
> >>>>> >>> release manager. I'd just like to know if anybody has any
> outstanding
> >>>>> >>> work that is nearing completion and that they'd like to include,
> >>>>> >>> and/or if we have any known blocker issues*.
> >>>>> >>>
> >>>>> >>> * -- I am having a little trouble searching github for blocker
> issues.
> >>>>> >>> I think there may not be any, but I tried
> >>>>> >>>
> https://github.com/apache/lucene/issues?q=is%3Aissue+is%3Aopen+legacy-jira-priority%5C%3AMajor+
> >>>>> >>> to find Major issues, for example, and it seems to only find one
> Minor
> >>>>> >>> one? Does anyone have better github-search-fu?
> >>>>> >>>
> >>>>> >>> API Changes
> >>>>> >>> ---------------------
> >>>>> >>> * LUCENE-10577: Add VectorEncoding to enable byte-encoded HNSW
> vectors
> >>>>> >>> (Michael Sokolov, Julie Tibshirani)
> >>>>> >>>
> >>>>> >>> New Features
> >>>>> >>> ---------------------
> >>>>> >>> * LUCENE-10654: Add new ShapeDocValuesField for LatLonShape and
> >>>>> >>> XYShape. (Nick Knize)
> >>>>> >>>
> >>>>> >>> * LUCENE-10629: Support match set filtering with a query in
> >>>>> >>> MatchingFacetSetCounts. (Stefan Vodita, Shai Erera)
> >>>>> >>>
> >>>>> >>> * LUCENE-10633: SortField#setOptimizeSortWithIndexedData and
> >>>>> >>> SortField#getOptimizeSortWithIndexedData were introduce to
> provide
> >>>>> >>> an option to disable sort optimization for various sort fields.
> >>>>> >>> (Mayya Sharipova)
> >>>>> >>>
> >>>>> >>> Improvements
> >>>>> >>> ---------------------
> >>>>> >>> * LUCENE-10592: Build HNSW Graph on indexing. (Mayya Sharipova,
> Adrien
> >>>>> >>> Grand, Julie Tibshirani)
> >>>>> >>>
> >>>>> >>> * GITHUB#11715: Add Integer awareness to
> RamUsageEstimator.sizeOf (Mike Drob)
> >>>>> >>>
> >>>>> >>> Optimizations
> >>>>> >>> ---------------------
> >>>>> >>>
> >>>>> >>> * LUCENE-10661: Reduce memory copy in BytesStore. (luyuncheng)
> >>>>> >>>
> >>>>> >>> * GITHUB#1020: Support #scoreSupplier and small optimizations to
> >>>>> >>> DocValuesRewriteMethod. (Greg Miller)
> >>>>> >>>
> >>>>> >>> * LUCENE-10633: Added support for dynamic pruning to queries
> sorted by a string
> >>>>> >>> field that is indexed with terms and SORTED or SORTED_SET doc
> values.
> >>>>> >>> (Adrien Grand)
> >>>>> >>>
> >>>>> >>> * LUCENE-10627: Using ByteBuffersDataInput reduce memory copy on
> >>>>> >>> compressing data. (luyuncheng)
> >>>>> >>>
> >>>>> >>> * GITHUB#1062: Optimize TermInSetQuery when a term is present
> that
> >>>>> >>> matches all docs in a segment.
> >>>>> >>> (Greg Miller)
> >>>>> >>>
> >>>>> >>>
> ---------------------------------------------------------------------
> >>>>> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>>>> >>> For additional commands, e-mail: dev-help@lucene.apache.org
> >>>>> >>>
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>>>> For additional commands, e-mail: dev-help@lucene.apache.org
> >>>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: Lucene 9.4.0 release [ In reply to ]
OK, thanks for letting us know about that, Mayya. Let's say 9/2-9/16
for a freeze period then.

On Thu, Sep 1, 2022 at 1:27 PM Mayya Sharipova
<mayya.sharipova@elastic.co.invalid> wrote:
>
> Hello Michael,
> thanks for starting the discussion for the 9.4.0 release.
>
> We are investigating the impact of LUCENE-10592 (Build HNSW Graph on indexing) on memory used during indexing.
> In our Elasticsearch tests, we have observed much more segments being created.
> It would be nice if we have an extra week for feature freeze (considering extended holidays in North America).
> Thank you.
>
>
> On Thu, Sep 1, 2022 at 11:04 AM Michael Sokolov <msokolov@gmail.com> wrote:
>>
>> Thanks Tomoko - I appreciate the offer to review the changes needed. I
>> will take care of updating the release script/template.
>>
>> I think I managed to get a GPG key registered and signed (thanks Mike
>> McCandless!) although we'll see if this all stands up to scrutiny!
>> It's quite tricky to navigate this setup, seems very secure to me, but
>> what do I know. Anyway the releaseWizard seems to be satisfied.
>>
>> So -- I plan to cut a branch_9_4 release branch tomorrow, Sept 2 and
>> late it bake for a week, until Sept 7, after which point it will be
>> frozen for release. Does that seem like enough time? If you have any
>> concerns, please reply here.
>>
>> On Thu, Sep 1, 2022 at 7:46 AM Tomoko Uchida
>> <tomoko.uchida.1111@gmail.com> wrote:
>> >
>> > Hi Mike,
>> > to be precise, on GitHub I think release managers would go through these steps:
>> >
>> > 1. Figure out blocker issues/PRs and associate them with the target Milestone, e.g., "9.4.0"
>> > 2. Create a new Milestone for the next version here, e.g., "9.5.0"
>> > 3. Address blockers - some may be resolved by developers, and others may be re-associated with the next Milestone.
>> > 4. Make release artifacts and publish them as usual.
>> > 5. Close (not Delete!) the target Milestone.
>> >
>> > You need no special permission on GitHub to handle Milestones, and everything should be fairly straightforward.
>> > If you need any help, I'd be happy to review the necessary updates on the release wizard tool. I have little time to make a patch for that myself right now, sorry.
>> >
>> > Tomoko
>> >
>> >
>> > 2022?9?1?(?) 10:02 Tomoko Uchida <tomoko.uchida.1111@gmail.com>:
>> >>
>> >> > One thing is that for JIra, we needed to acquire special permissions in order to be able to add new release versions, but I don't think that is true for GitHub ...
>> >>
>> >> You do not need any special permissions to make new Milestones on GitHub. Every committer already has permission to create/close/delete Milestones, you can test it here.
>> >> https://github.com/apache/lucene/milestones
>> >>
>> >> Tomoko
>> >>
>> >>
>> >> 2022?9?1?(?) 9:57 Michael Sokolov <msokolov@gmail.com>:
>> >>>
>> >>> I started to go through the release process, which has been scripted very clearly by Jan and others, and found Jira-specific steps there. I think the translations to GitHub will be pretty straightforward. One thing is that for JIra, we needed to acquire special permissions in order to be able to add new release versions, but I don't think that is true for GitHub ... Any committer should be able to do that, right?
>> >>>
>> >>> On Wed, Aug 31, 2022, 8:42 PM Tomoko Uchida <tomoko.uchida.1111@gmail.com> wrote:
>> >>>>
>> >>>> Hi Mike,
>> >>>> thanks for your response.
>> >>>>
>> >>>> I'm actually not familiar with the whole release procedure that may be built on Jira.
>> >>>> I hope other people help to interpret Jira-related things on the way into the language of GitHub issues.
>> >>>>
>> >>>> Tomoko
>> >>>>
>> >>>> 2022?9?1?(?) 3:40 Michael Sokolov <msokolov@gmail.com>:
>> >>>>>
>> >>>>> Thanks for the links, Tomoko. I thought it would be helpful to ask on
>> >>>>> the list in case this new system has not been fully internalized by
>> >>>>> all the developers. And I confess I didn't know about that nice
>> >>>>> milestone view! It makes this very easy, assuming of course that
>> >>>>> everyone follows the guidelines and labels the issues with the
>> >>>>> milestone.
>> >>>>>
>> >>>>> On Wed, Aug 31, 2022 at 11:16 AM Tomoko Uchida
>> >>>>> <tomoko.uchida.1111@gmail.com> wrote:
>> >>>>> >
>> >>>>> > Ok in short, you can find blockers for 9.4.0 here.
>> >>>>> > https://github.com/apache/lucene/milestone/3
>> >>>>> >
>> >>>>> > If there are other issues/PRs that are still not resolved but planned to be delivered in 9.4.0, just set their Milestone as 9.4.0. Then they will appear in the list.
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> > 2022?9?1?(?) 0:00 Tomoko Uchida <tomoko.uchida.1111@gmail.com>:
>> >>>>> >>
>> >>>>> >> I believe we decided to use GitHub Milestone for release planning and I wrote documentation about it.
>> >>>>> >> https://github.com/apache/lucene/blob/main/dev-docs/github-issues-howto.md
>> >>>>> >>
>> >>>>> >> Is this unclear to you?
>> >>>>> >>
>> >>>>> >>
>> >>>>> >> 2022?8?31?(?) 23:13 Michael Sokolov <msokolov@gmail.com>:
>> >>>>> >>>
>> >>>>> >>> Hi, I'd like to start the ball rolling for a 9.4.0 release. We don't
>> >>>>> >>> have a large number of things, but there are a few nice high-impact
>> >>>>> >>> changes; see excerpt from CHANGE.txt below. I'll volunteer to be
>> >>>>> >>> release manager. I'd just like to know if anybody has any outstanding
>> >>>>> >>> work that is nearing completion and that they'd like to include,
>> >>>>> >>> and/or if we have any known blocker issues*.
>> >>>>> >>>
>> >>>>> >>> * -- I am having a little trouble searching github for blocker issues.
>> >>>>> >>> I think there may not be any, but I tried
>> >>>>> >>> https://github.com/apache/lucene/issues?q=is%3Aissue+is%3Aopen+legacy-jira-priority%5C%3AMajor+
>> >>>>> >>> to find Major issues, for example, and it seems to only find one Minor
>> >>>>> >>> one? Does anyone have better github-search-fu?
>> >>>>> >>>
>> >>>>> >>> API Changes
>> >>>>> >>> ---------------------
>> >>>>> >>> * LUCENE-10577: Add VectorEncoding to enable byte-encoded HNSW vectors
>> >>>>> >>> (Michael Sokolov, Julie Tibshirani)
>> >>>>> >>>
>> >>>>> >>> New Features
>> >>>>> >>> ---------------------
>> >>>>> >>> * LUCENE-10654: Add new ShapeDocValuesField for LatLonShape and
>> >>>>> >>> XYShape. (Nick Knize)
>> >>>>> >>>
>> >>>>> >>> * LUCENE-10629: Support match set filtering with a query in
>> >>>>> >>> MatchingFacetSetCounts. (Stefan Vodita, Shai Erera)
>> >>>>> >>>
>> >>>>> >>> * LUCENE-10633: SortField#setOptimizeSortWithIndexedData and
>> >>>>> >>> SortField#getOptimizeSortWithIndexedData were introduce to provide
>> >>>>> >>> an option to disable sort optimization for various sort fields.
>> >>>>> >>> (Mayya Sharipova)
>> >>>>> >>>
>> >>>>> >>> Improvements
>> >>>>> >>> ---------------------
>> >>>>> >>> * LUCENE-10592: Build HNSW Graph on indexing. (Mayya Sharipova, Adrien
>> >>>>> >>> Grand, Julie Tibshirani)
>> >>>>> >>>
>> >>>>> >>> * GITHUB#11715: Add Integer awareness to RamUsageEstimator.sizeOf (Mike Drob)
>> >>>>> >>>
>> >>>>> >>> Optimizations
>> >>>>> >>> ---------------------
>> >>>>> >>>
>> >>>>> >>> * LUCENE-10661: Reduce memory copy in BytesStore. (luyuncheng)
>> >>>>> >>>
>> >>>>> >>> * GITHUB#1020: Support #scoreSupplier and small optimizations to
>> >>>>> >>> DocValuesRewriteMethod. (Greg Miller)
>> >>>>> >>>
>> >>>>> >>> * LUCENE-10633: Added support for dynamic pruning to queries sorted by a string
>> >>>>> >>> field that is indexed with terms and SORTED or SORTED_SET doc values.
>> >>>>> >>> (Adrien Grand)
>> >>>>> >>>
>> >>>>> >>> * LUCENE-10627: Using ByteBuffersDataInput reduce memory copy on
>> >>>>> >>> compressing data. (luyuncheng)
>> >>>>> >>>
>> >>>>> >>> * GITHUB#1062: Optimize TermInSetQuery when a term is present that
>> >>>>> >>> matches all docs in a segment.
>> >>>>> >>> (Greg Miller)
>> >>>>> >>>
>> >>>>> >>> ---------------------------------------------------------------------
>> >>>>> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >>>>> >>> For additional commands, e-mail: dev-help@lucene.apache.org
>> >>>>> >>>
>> >>>>>
>> >>>>> ---------------------------------------------------------------------
>> >>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>> >>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org