Mailing List Archive

8.10.1 Patch release?
Hello everyone!
Thank you, Timothy, for the recent 8.10 release.

I wonder if we are ok to do a 8.10.1 patch release and do it fairly soon?
this week?
Nhat fixed a bad bug <https://issues.apache.org/jira/browse/LUCENE-10126>where
"sort with after" on a numeric field can incorrectly miss documents. This
bug only manifests when sort optimization on numeric fields is explicitly
enabled.

Given that we are not sure when the next minor release will be, it would be
useful to have a patch release. If nobody is opposed to it, I can
volunteer to be the Release Manager.

Thanks.
Re: 8.10.1 Patch release? [ In reply to ]
FYI Looks like there are already six items currently listed under
"Bugfixes" for 8.11.0, so those could be candidates for the patch
release.

Bug Fixes
---------------------
* LUCENE-10110: MultiCollector now handles single leaf collector that
wants to skip low-scoring hits
but the combined score mode doesn't allow it. (Jim Ferenczi)
* LUCENE-10111: Missing calculating the bytes used of DocsWithFieldSet
in NormValuesWriter.
(Lu Xugang)
* LUCENE-10116: Missing calculating the bytes used of DocsWithFieldSet
and currentValues in SortedSetDocValuesWriter.
(Lu Xugang)
* LUCENE-10070 Skip deleted docs when accumulating facet counts for
all docs. (Ankur Goel, Greg Miller)
* LUCENE-10126: Sort optimization with a chunked bulk scorer
can wrongly skip documents (Nhat Nguyen, Mayya Sharipova)
* LUCENE-10134: ConcurrentSortedSetDocValuesFacetCounts shouldn't
share liveDocs Bits across threads.
(Ankur Goel)

On Mon, Oct 4, 2021 at 7:46 PM Mayya Sharipova <mayya@apache.org> wrote:
>
> Hello everyone!
> Thank you, Timothy, for the recent 8.10 release.
>
> I wonder if we are ok to do a 8.10.1 patch release and do it fairly soon? this week?
> Nhat fixed a bad bug where "sort with after" on a numeric field can incorrectly miss documents. This bug only manifests when sort optimization on numeric fields is explicitly enabled.
>
> Given that we are not sure when the next minor release will be, it would be useful to have a patch release. If nobody is opposed to it, I can volunteer to be the Release Manager.
>
> Thanks.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: 8.10.1 Patch release? [ In reply to ]
Thanks for the update, Robert. Would be nice to have these bug fixes as
well.

On Mon, Oct 4, 2021 at 7:56 PM Robert Muir <rcmuir@gmail.com> wrote:

> FYI Looks like there are already six items currently listed under
> "Bugfixes" for 8.11.0, so those could be candidates for the patch
> release.
>
> Bug Fixes
> ---------------------
> * LUCENE-10110: MultiCollector now handles single leaf collector that
> wants to skip low-scoring hits
> but the combined score mode doesn't allow it. (Jim Ferenczi)
> * LUCENE-10111: Missing calculating the bytes used of DocsWithFieldSet
> in NormValuesWriter.
> (Lu Xugang)
> * LUCENE-10116: Missing calculating the bytes used of DocsWithFieldSet
> and currentValues in SortedSetDocValuesWriter.
> (Lu Xugang)
> * LUCENE-10070 Skip deleted docs when accumulating facet counts for
> all docs. (Ankur Goel, Greg Miller)
> * LUCENE-10126: Sort optimization with a chunked bulk scorer
> can wrongly skip documents (Nhat Nguyen, Mayya Sharipova)
> * LUCENE-10134: ConcurrentSortedSetDocValuesFacetCounts shouldn't
> share liveDocs Bits across threads.
> (Ankur Goel)
>
> On Mon, Oct 4, 2021 at 7:46 PM Mayya Sharipova <mayya@apache.org> wrote:
> >
> > Hello everyone!
> > Thank you, Timothy, for the recent 8.10 release.
> >
> > I wonder if we are ok to do a 8.10.1 patch release and do it fairly
> soon? this week?
> > Nhat fixed a bad bug where "sort with after" on a numeric field can
> incorrectly miss documents. This bug only manifests when sort optimization
> on numeric fields is explicitly enabled.
> >
> > Given that we are not sure when the next minor release will be, it would
> be useful to have a patch release. If nobody is opposed to it, I can
> volunteer to be the Release Manager.
> >
> > Thanks.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: 8.10.1 Patch release? [ In reply to ]
+1 to a 8.10.1 patch release

On Tue, Oct 5, 2021 at 2:03 AM Mayya Sharipova
<mayya.sharipova@elastic.co.invalid> wrote:

> Thanks for the update, Robert. Would be nice to have these bug fixes as
> well.
>
> On Mon, Oct 4, 2021 at 7:56 PM Robert Muir <rcmuir@gmail.com> wrote:
>
>> FYI Looks like there are already six items currently listed under
>> "Bugfixes" for 8.11.0, so those could be candidates for the patch
>> release.
>>
>> Bug Fixes
>> ---------------------
>> * LUCENE-10110: MultiCollector now handles single leaf collector that
>> wants to skip low-scoring hits
>> but the combined score mode doesn't allow it. (Jim Ferenczi)
>> * LUCENE-10111: Missing calculating the bytes used of DocsWithFieldSet
>> in NormValuesWriter.
>> (Lu Xugang)
>> * LUCENE-10116: Missing calculating the bytes used of DocsWithFieldSet
>> and currentValues in SortedSetDocValuesWriter.
>> (Lu Xugang)
>> * LUCENE-10070 Skip deleted docs when accumulating facet counts for
>> all docs. (Ankur Goel, Greg Miller)
>> * LUCENE-10126: Sort optimization with a chunked bulk scorer
>> can wrongly skip documents (Nhat Nguyen, Mayya Sharipova)
>> * LUCENE-10134: ConcurrentSortedSetDocValuesFacetCounts shouldn't
>> share liveDocs Bits across threads.
>> (Ankur Goel)
>>
>> On Mon, Oct 4, 2021 at 7:46 PM Mayya Sharipova <mayya@apache.org> wrote:
>> >
>> > Hello everyone!
>> > Thank you, Timothy, for the recent 8.10 release.
>> >
>> > I wonder if we are ok to do a 8.10.1 patch release and do it fairly
>> soon? this week?
>> > Nhat fixed a bad bug where "sort with after" on a numeric field can
>> incorrectly miss documents. This bug only manifests when sort optimization
>> on numeric fields is explicitly enabled.
>> >
>> > Given that we are not sure when the next minor release will be, it
>> would be useful to have a patch release. If nobody is opposed to it, I
>> can volunteer to be the Release Manager.
>> >
>> > Thanks.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>

--
Adrien
Re: 8.10.1 Patch release? [ In reply to ]
Is the bug new in 8.10? If it affects older versions as well then I feel
like 8.10.1 might be less urgent.

Mike

On Tue, Oct 5, 2021 at 2:14 PM Adrien Grand <jpountz@gmail.com> wrote:

> +1 to a 8.10.1 patch release
>
> On Tue, Oct 5, 2021 at 2:03 AM Mayya Sharipova
> <mayya.sharipova@elastic.co.invalid> wrote:
>
>> Thanks for the update, Robert. Would be nice to have these bug fixes as
>> well.
>>
>> On Mon, Oct 4, 2021 at 7:56 PM Robert Muir <rcmuir@gmail.com> wrote:
>>
>>> FYI Looks like there are already six items currently listed under
>>> "Bugfixes" for 8.11.0, so those could be candidates for the patch
>>> release.
>>>
>>> Bug Fixes
>>> ---------------------
>>> * LUCENE-10110: MultiCollector now handles single leaf collector that
>>> wants to skip low-scoring hits
>>> but the combined score mode doesn't allow it. (Jim Ferenczi)
>>> * LUCENE-10111: Missing calculating the bytes used of DocsWithFieldSet
>>> in NormValuesWriter.
>>> (Lu Xugang)
>>> * LUCENE-10116: Missing calculating the bytes used of DocsWithFieldSet
>>> and currentValues in SortedSetDocValuesWriter.
>>> (Lu Xugang)
>>> * LUCENE-10070 Skip deleted docs when accumulating facet counts for
>>> all docs. (Ankur Goel, Greg Miller)
>>> * LUCENE-10126: Sort optimization with a chunked bulk scorer
>>> can wrongly skip documents (Nhat Nguyen, Mayya Sharipova)
>>> * LUCENE-10134: ConcurrentSortedSetDocValuesFacetCounts shouldn't
>>> share liveDocs Bits across threads.
>>> (Ankur Goel)
>>>
>>> On Mon, Oct 4, 2021 at 7:46 PM Mayya Sharipova <mayya@apache.org> wrote:
>>> >
>>> > Hello everyone!
>>> > Thank you, Timothy, for the recent 8.10 release.
>>> >
>>> > I wonder if we are ok to do a 8.10.1 patch release and do it fairly
>>> soon? this week?
>>> > Nhat fixed a bad bug where "sort with after" on a numeric field can
>>> incorrectly miss documents. This bug only manifests when sort optimization
>>> on numeric fields is explicitly enabled.
>>> >
>>> > Given that we are not sure when the next minor release will be, it
>>> would be useful to have a patch release. If nobody is opposed to it, I
>>> can volunteer to be the Release Manager.
>>> >
>>> > Thanks.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
>
> --
> Adrien
>
Re: 8.10.1 Patch release? [ In reply to ]
No, the bug is not new and was present in the previous versions as well,
but was discovered quite recently.

On Tue, Oct 5, 2021 at 3:54 PM Mike Drob <mdrob@apache.org> wrote:

> Is the bug new in 8.10? If it affects older versions as well then I feel
> like 8.10.1 might be less urgent.
>
> Mike
>
> On Tue, Oct 5, 2021 at 2:14 PM Adrien Grand <jpountz@gmail.com> wrote:
>
>> +1 to a 8.10.1 patch release
>>
>> On Tue, Oct 5, 2021 at 2:03 AM Mayya Sharipova
>> <mayya.sharipova@elastic.co.invalid> wrote:
>>
>>> Thanks for the update, Robert. Would be nice to have these bug fixes
>>> as well.
>>>
>>> On Mon, Oct 4, 2021 at 7:56 PM Robert Muir <rcmuir@gmail.com> wrote:
>>>
>>>> FYI Looks like there are already six items currently listed under
>>>> "Bugfixes" for 8.11.0, so those could be candidates for the patch
>>>> release.
>>>>
>>>> Bug Fixes
>>>> ---------------------
>>>> * LUCENE-10110: MultiCollector now handles single leaf collector that
>>>> wants to skip low-scoring hits
>>>> but the combined score mode doesn't allow it. (Jim Ferenczi)
>>>> * LUCENE-10111: Missing calculating the bytes used of DocsWithFieldSet
>>>> in NormValuesWriter.
>>>> (Lu Xugang)
>>>> * LUCENE-10116: Missing calculating the bytes used of DocsWithFieldSet
>>>> and currentValues in SortedSetDocValuesWriter.
>>>> (Lu Xugang)
>>>> * LUCENE-10070 Skip deleted docs when accumulating facet counts for
>>>> all docs. (Ankur Goel, Greg Miller)
>>>> * LUCENE-10126: Sort optimization with a chunked bulk scorer
>>>> can wrongly skip documents (Nhat Nguyen, Mayya Sharipova)
>>>> * LUCENE-10134: ConcurrentSortedSetDocValuesFacetCounts shouldn't
>>>> share liveDocs Bits across threads.
>>>> (Ankur Goel)
>>>>
>>>> On Mon, Oct 4, 2021 at 7:46 PM Mayya Sharipova <mayya@apache.org>
>>>> wrote:
>>>> >
>>>> > Hello everyone!
>>>> > Thank you, Timothy, for the recent 8.10 release.
>>>> >
>>>> > I wonder if we are ok to do a 8.10.1 patch release and do it fairly
>>>> soon? this week?
>>>> > Nhat fixed a bad bug where "sort with after" on a numeric field can
>>>> incorrectly miss documents. This bug only manifests when sort optimization
>>>> on numeric fields is explicitly enabled.
>>>> >
>>>> > Given that we are not sure when the next minor release will be, it
>>>> would be useful to have a patch release. If nobody is opposed to it, I
>>>> can volunteer to be the Release Manager.
>>>> >
>>>> > Thanks.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>
>>>>
>>
>> --
>> Adrien
>>
>
Re: 8.10.1 Patch release? [ In reply to ]
Mike, it's unclear to me if you are suggesting waiting before doing a
8.10.1 release? On my end I'm good with doing a 8.10.1 release now, we
could still do a 8.10.2 release later in case we find new bugs?

On Tue, Oct 5, 2021 at 10:41 PM Mayya Sharipova
<mayya.sharipova@elastic.co.invalid> wrote:

> No, the bug is not new and was present in the previous versions as well,
> but was discovered quite recently.
>
> On Tue, Oct 5, 2021 at 3:54 PM Mike Drob <mdrob@apache.org> wrote:
>
>> Is the bug new in 8.10? If it affects older versions as well then I feel
>> like 8.10.1 might be less urgent.
>>
>> Mike
>>
>> On Tue, Oct 5, 2021 at 2:14 PM Adrien Grand <jpountz@gmail.com> wrote:
>>
>>> +1 to a 8.10.1 patch release
>>>
>>> On Tue, Oct 5, 2021 at 2:03 AM Mayya Sharipova
>>> <mayya.sharipova@elastic.co.invalid> wrote:
>>>
>>>> Thanks for the update, Robert. Would be nice to have these bug fixes
>>>> as well.
>>>>
>>>> On Mon, Oct 4, 2021 at 7:56 PM Robert Muir <rcmuir@gmail.com> wrote:
>>>>
>>>>> FYI Looks like there are already six items currently listed under
>>>>> "Bugfixes" for 8.11.0, so those could be candidates for the patch
>>>>> release.
>>>>>
>>>>> Bug Fixes
>>>>> ---------------------
>>>>> * LUCENE-10110: MultiCollector now handles single leaf collector that
>>>>> wants to skip low-scoring hits
>>>>> but the combined score mode doesn't allow it. (Jim Ferenczi)
>>>>> * LUCENE-10111: Missing calculating the bytes used of DocsWithFieldSet
>>>>> in NormValuesWriter.
>>>>> (Lu Xugang)
>>>>> * LUCENE-10116: Missing calculating the bytes used of DocsWithFieldSet
>>>>> and currentValues in SortedSetDocValuesWriter.
>>>>> (Lu Xugang)
>>>>> * LUCENE-10070 Skip deleted docs when accumulating facet counts for
>>>>> all docs. (Ankur Goel, Greg Miller)
>>>>> * LUCENE-10126: Sort optimization with a chunked bulk scorer
>>>>> can wrongly skip documents (Nhat Nguyen, Mayya Sharipova)
>>>>> * LUCENE-10134: ConcurrentSortedSetDocValuesFacetCounts shouldn't
>>>>> share liveDocs Bits across threads.
>>>>> (Ankur Goel)
>>>>>
>>>>> On Mon, Oct 4, 2021 at 7:46 PM Mayya Sharipova <mayya@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > Hello everyone!
>>>>> > Thank you, Timothy, for the recent 8.10 release.
>>>>> >
>>>>> > I wonder if we are ok to do a 8.10.1 patch release and do it fairly
>>>>> soon? this week?
>>>>> > Nhat fixed a bad bug where "sort with after" on a numeric field can
>>>>> incorrectly miss documents. This bug only manifests when sort optimization
>>>>> on numeric fields is explicitly enabled.
>>>>> >
>>>>> > Given that we are not sure when the next minor release will be, it
>>>>> would be useful to have a patch release. If nobody is opposed to it, I
>>>>> can volunteer to be the Release Manager.
>>>>> >
>>>>> > Thanks.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>
>>>>>
>>>
>>> --
>>> Adrien
>>>
>>

--
Adrien
Re: 8.10.1 Patch release? [ In reply to ]
It feels weird to say that I’m against releases, but generally I feel like
bug fix releases should be scoped either for a regression discovered in
that release or for rapid security fixes. Otherwise, what’s the harm in
waiting for the next release train?

Obviously any committee is free to create a release candidate on any
commit, and if there are three PMC members in support then a release can
happen, but I don’t want to be putting pressure on ourselves where we are
constantly in the middle of a release cycle.

Or waiting a month and doing 8.11 seems fine too?

On Wed, Oct 6, 2021 at 8:17 AM Adrien Grand <jpountz@gmail.com> wrote:

> Mike, it's unclear to me if you are suggesting waiting before doing a
> 8.10.1 release? On my end I'm good with doing a 8.10.1 release now, we
> could still do a 8.10.2 release later in case we find new bugs?
>
> On Tue, Oct 5, 2021 at 10:41 PM Mayya Sharipova
> <mayya.sharipova@elastic.co.invalid> wrote:
>
>> No, the bug is not new and was present in the previous versions as well,
>> but was discovered quite recently.
>>
>> On Tue, Oct 5, 2021 at 3:54 PM Mike Drob <mdrob@apache.org> wrote:
>>
>>> Is the bug new in 8.10? If it affects older versions as well then I feel
>>> like 8.10.1 might be less urgent.
>>>
>>> Mike
>>>
>>> On Tue, Oct 5, 2021 at 2:14 PM Adrien Grand <jpountz@gmail.com> wrote:
>>>
>>>> +1 to a 8.10.1 patch release
>>>>
>>>> On Tue, Oct 5, 2021 at 2:03 AM Mayya Sharipova
>>>> <mayya.sharipova@elastic.co.invalid> wrote:
>>>>
>>>>> Thanks for the update, Robert. Would be nice to have these bug fixes
>>>>> as well.
>>>>>
>>>>> On Mon, Oct 4, 2021 at 7:56 PM Robert Muir <rcmuir@gmail.com> wrote:
>>>>>
>>>>>> FYI Looks like there are already six items currently listed under
>>>>>> "Bugfixes" for 8.11.0, so those could be candidates for the patch
>>>>>> release.
>>>>>>
>>>>>> Bug Fixes
>>>>>> ---------------------
>>>>>> * LUCENE-10110: MultiCollector now handles single leaf collector that
>>>>>> wants to skip low-scoring hits
>>>>>> but the combined score mode doesn't allow it. (Jim Ferenczi)
>>>>>> * LUCENE-10111: Missing calculating the bytes used of DocsWithFieldSet
>>>>>> in NormValuesWriter.
>>>>>> (Lu Xugang)
>>>>>> * LUCENE-10116: Missing calculating the bytes used of DocsWithFieldSet
>>>>>> and currentValues in SortedSetDocValuesWriter.
>>>>>> (Lu Xugang)
>>>>>> * LUCENE-10070 Skip deleted docs when accumulating facet counts for
>>>>>> all docs. (Ankur Goel, Greg Miller)
>>>>>> * LUCENE-10126: Sort optimization with a chunked bulk scorer
>>>>>> can wrongly skip documents (Nhat Nguyen, Mayya Sharipova)
>>>>>> * LUCENE-10134: ConcurrentSortedSetDocValuesFacetCounts shouldn't
>>>>>> share liveDocs Bits across threads.
>>>>>> (Ankur Goel)
>>>>>>
>>>>>> On Mon, Oct 4, 2021 at 7:46 PM Mayya Sharipova <mayya@apache.org>
>>>>>> wrote:
>>>>>> >
>>>>>> > Hello everyone!
>>>>>> > Thank you, Timothy, for the recent 8.10 release.
>>>>>> >
>>>>>> > I wonder if we are ok to do a 8.10.1 patch release and do it fairly
>>>>>> soon? this week?
>>>>>> > Nhat fixed a bad bug where "sort with after" on a numeric field can
>>>>>> incorrectly miss documents. This bug only manifests when sort optimization
>>>>>> on numeric fields is explicitly enabled.
>>>>>> >
>>>>>> > Given that we are not sure when the next minor release will be, it
>>>>>> would be useful to have a patch release. If nobody is opposed to it, I
>>>>>> can volunteer to be the Release Manager.
>>>>>> >
>>>>>> > Thanks.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>>
>>>>>>
>>>>
>>>> --
>>>> Adrien
>>>>
>>>
>
> --
> Adrien
>
Re: 8.10.1 Patch release? [ In reply to ]
I agree with Mike on this one as well. In addition, I'm surprised
nobody asked to halt the RC1 and make RC2 with Nhat's fix while I was
doing 8.10. Nhat made it sound like it was not a big deal at the time,
but now there's some urgency in releasing it?

Tim

On Wed, Oct 6, 2021 at 11:15 AM Mike Drob <mdrob@mdrob.com> wrote:
>
> It feels weird to say that I’m against releases, but generally I feel like bug fix releases should be scoped either for a regression discovered in that release or for rapid security fixes. Otherwise, what’s the harm in waiting for the next release train?
>
> Obviously any committee is free to create a release candidate on any commit, and if there are three PMC members in support then a release can happen, but I don’t want to be putting pressure on ourselves where we are constantly in the middle of a release cycle.
>
> Or waiting a month and doing 8.11 seems fine too?
>
> On Wed, Oct 6, 2021 at 8:17 AM Adrien Grand <jpountz@gmail.com> wrote:
>>
>> Mike, it's unclear to me if you are suggesting waiting before doing a 8.10.1 release? On my end I'm good with doing a 8.10.1 release now, we could still do a 8.10.2 release later in case we find new bugs?
>>
>> On Tue, Oct 5, 2021 at 10:41 PM Mayya Sharipova <mayya.sharipova@elastic.co.invalid> wrote:
>>>
>>> No, the bug is not new and was present in the previous versions as well, but was discovered quite recently.
>>>
>>> On Tue, Oct 5, 2021 at 3:54 PM Mike Drob <mdrob@apache.org> wrote:
>>>>
>>>> Is the bug new in 8.10? If it affects older versions as well then I feel like 8.10.1 might be less urgent.
>>>>
>>>> Mike
>>>>
>>>> On Tue, Oct 5, 2021 at 2:14 PM Adrien Grand <jpountz@gmail.com> wrote:
>>>>>
>>>>> +1 to a 8.10.1 patch release
>>>>>
>>>>> On Tue, Oct 5, 2021 at 2:03 AM Mayya Sharipova <mayya.sharipova@elastic.co.invalid> wrote:
>>>>>>
>>>>>> Thanks for the update, Robert. Would be nice to have these bug fixes as well.
>>>>>>
>>>>>> On Mon, Oct 4, 2021 at 7:56 PM Robert Muir <rcmuir@gmail.com> wrote:
>>>>>>>
>>>>>>> FYI Looks like there are already six items currently listed under
>>>>>>> "Bugfixes" for 8.11.0, so those could be candidates for the patch
>>>>>>> release.
>>>>>>>
>>>>>>> Bug Fixes
>>>>>>> ---------------------
>>>>>>> * LUCENE-10110: MultiCollector now handles single leaf collector that
>>>>>>> wants to skip low-scoring hits
>>>>>>> but the combined score mode doesn't allow it. (Jim Ferenczi)
>>>>>>> * LUCENE-10111: Missing calculating the bytes used of DocsWithFieldSet
>>>>>>> in NormValuesWriter.
>>>>>>> (Lu Xugang)
>>>>>>> * LUCENE-10116: Missing calculating the bytes used of DocsWithFieldSet
>>>>>>> and currentValues in SortedSetDocValuesWriter.
>>>>>>> (Lu Xugang)
>>>>>>> * LUCENE-10070 Skip deleted docs when accumulating facet counts for
>>>>>>> all docs. (Ankur Goel, Greg Miller)
>>>>>>> * LUCENE-10126: Sort optimization with a chunked bulk scorer
>>>>>>> can wrongly skip documents (Nhat Nguyen, Mayya Sharipova)
>>>>>>> * LUCENE-10134: ConcurrentSortedSetDocValuesFacetCounts shouldn't
>>>>>>> share liveDocs Bits across threads.
>>>>>>> (Ankur Goel)
>>>>>>>
>>>>>>> On Mon, Oct 4, 2021 at 7:46 PM Mayya Sharipova <mayya@apache.org> wrote:
>>>>>>> >
>>>>>>> > Hello everyone!
>>>>>>> > Thank you, Timothy, for the recent 8.10 release.
>>>>>>> >
>>>>>>> > I wonder if we are ok to do a 8.10.1 patch release and do it fairly soon? this week?
>>>>>>> > Nhat fixed a bad bug where "sort with after" on a numeric field can incorrectly miss documents. This bug only manifests when sort optimization on numeric fields is explicitly enabled.
>>>>>>> >
>>>>>>> > Given that we are not sure when the next minor release will be, it would be useful to have a patch release. If nobody is opposed to it, I can volunteer to be the Release Manager.
>>>>>>> >
>>>>>>> > Thanks.
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Adrien
>>
>>
>>
>> --
>> Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: 8.10.1 Patch release? [ In reply to ]
Hello everyone!

Thank you for discussing this.

There were two bugs during the release of 8.10.0. I considered the first
bug wasn't a blocker to respin 8.10.0 RC1 and it made into 8.10.0 RC2.
However, Mayya discovered that the second bug had a severe impact on search
after requests with sort, and we didn't fully understand its severity until
8.10.0 was out.
Although the bug has been in the previous versions, I am +1 to release
8.10.1 to reduce the impact.

Best Regards,
Nhat

On Wed, Oct 6, 2021 at 2:36 PM Timothy Potter <thelabdude@gmail.com> wrote:

> I agree with Mike on this one as well. In addition, I'm surprised
> nobody asked to halt the RC1 and make RC2 with Nhat's fix while I was
> doing 8.10. Nhat made it sound like it was not a big deal at the time,
> but now there's some urgency in releasing it?
>
> Tim
>
> On Wed, Oct 6, 2021 at 11:15 AM Mike Drob <mdrob@mdrob.com> wrote:
> >
> > It feels weird to say that I’m against releases, but generally I feel
> like bug fix releases should be scoped either for a regression discovered
> in that release or for rapid security fixes. Otherwise, what’s the harm in
> waiting for the next release train?
> >
> > Obviously any committee is free to create a release candidate on any
> commit, and if there are three PMC members in support then a release can
> happen, but I don’t want to be putting pressure on ourselves where we are
> constantly in the middle of a release cycle.
> >
> > Or waiting a month and doing 8.11 seems fine too?
> >
> > On Wed, Oct 6, 2021 at 8:17 AM Adrien Grand <jpountz@gmail.com> wrote:
> >>
> >> Mike, it's unclear to me if you are suggesting waiting before doing a
> 8.10.1 release? On my end I'm good with doing a 8.10.1 release now, we
> could still do a 8.10.2 release later in case we find new bugs?
> >>
> >> On Tue, Oct 5, 2021 at 10:41 PM Mayya Sharipova <
> mayya.sharipova@elastic.co.invalid> wrote:
> >>>
> >>> No, the bug is not new and was present in the previous versions as
> well, but was discovered quite recently.
> >>>
> >>> On Tue, Oct 5, 2021 at 3:54 PM Mike Drob <mdrob@apache.org> wrote:
> >>>>
> >>>> Is the bug new in 8.10? If it affects older versions as well then I
> feel like 8.10.1 might be less urgent.
> >>>>
> >>>> Mike
> >>>>
> >>>> On Tue, Oct 5, 2021 at 2:14 PM Adrien Grand <jpountz@gmail.com>
> wrote:
> >>>>>
> >>>>> +1 to a 8.10.1 patch release
> >>>>>
> >>>>> On Tue, Oct 5, 2021 at 2:03 AM Mayya Sharipova <
> mayya.sharipova@elastic.co.invalid> wrote:
> >>>>>>
> >>>>>> Thanks for the update, Robert. Would be nice to have these bug
> fixes as well.
> >>>>>>
> >>>>>> On Mon, Oct 4, 2021 at 7:56 PM Robert Muir <rcmuir@gmail.com>
> wrote:
> >>>>>>>
> >>>>>>> FYI Looks like there are already six items currently listed under
> >>>>>>> "Bugfixes" for 8.11.0, so those could be candidates for the patch
> >>>>>>> release.
> >>>>>>>
> >>>>>>> Bug Fixes
> >>>>>>> ---------------------
> >>>>>>> * LUCENE-10110: MultiCollector now handles single leaf collector
> that
> >>>>>>> wants to skip low-scoring hits
> >>>>>>> but the combined score mode doesn't allow it. (Jim Ferenczi)
> >>>>>>> * LUCENE-10111: Missing calculating the bytes used of
> DocsWithFieldSet
> >>>>>>> in NormValuesWriter.
> >>>>>>> (Lu Xugang)
> >>>>>>> * LUCENE-10116: Missing calculating the bytes used of
> DocsWithFieldSet
> >>>>>>> and currentValues in SortedSetDocValuesWriter.
> >>>>>>> (Lu Xugang)
> >>>>>>> * LUCENE-10070 Skip deleted docs when accumulating facet counts for
> >>>>>>> all docs. (Ankur Goel, Greg Miller)
> >>>>>>> * LUCENE-10126: Sort optimization with a chunked bulk scorer
> >>>>>>> can wrongly skip documents (Nhat Nguyen, Mayya Sharipova)
> >>>>>>> * LUCENE-10134: ConcurrentSortedSetDocValuesFacetCounts shouldn't
> >>>>>>> share liveDocs Bits across threads.
> >>>>>>> (Ankur Goel)
> >>>>>>>
> >>>>>>> On Mon, Oct 4, 2021 at 7:46 PM Mayya Sharipova <mayya@apache.org>
> wrote:
> >>>>>>> >
> >>>>>>> > Hello everyone!
> >>>>>>> > Thank you, Timothy, for the recent 8.10 release.
> >>>>>>> >
> >>>>>>> > I wonder if we are ok to do a 8.10.1 patch release and do it
> fairly soon? this week?
> >>>>>>> > Nhat fixed a bad bug where "sort with after" on a numeric field
> can incorrectly miss documents. This bug only manifests when sort
> optimization on numeric fields is explicitly enabled.
> >>>>>>> >
> >>>>>>> > Given that we are not sure when the next minor release will be,
> it would be useful to have a patch release. If nobody is opposed to it, I
> can volunteer to be the Release Manager.
> >>>>>>> >
> >>>>>>> > Thanks.
> >>>>>>>
> >>>>>>>
> ---------------------------------------------------------------------
> >>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
> >>>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Adrien
> >>
> >>
> >>
> >> --
> >> Adrien
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: 8.10.1 Patch release? [ In reply to ]
Ah ok thanks for clarifying Nhat. In light of this, doing an 8.10.1
makes sense to me.

Cheers,
Tim

On Wed, Oct 6, 2021 at 12:57 PM Nhat Nguyen
<nhat.nguyen@elastic.co.invalid> wrote:
>
> Hello everyone!
>
> Thank you for discussing this.
>
> There were two bugs during the release of 8.10.0. I considered the first bug wasn't a blocker to respin 8.10.0 RC1 and it made into 8.10.0 RC2. However, Mayya discovered that the second bug had a severe impact on search after requests with sort, and we didn't fully understand its severity until 8.10.0 was out.
> Although the bug has been in the previous versions, I am +1 to release 8.10.1 to reduce the impact.
>
> Best Regards,
> Nhat
>
> On Wed, Oct 6, 2021 at 2:36 PM Timothy Potter <thelabdude@gmail.com> wrote:
>>
>> I agree with Mike on this one as well. In addition, I'm surprised
>> nobody asked to halt the RC1 and make RC2 with Nhat's fix while I was
>> doing 8.10. Nhat made it sound like it was not a big deal at the time,
>> but now there's some urgency in releasing it?
>>
>> Tim
>>
>> On Wed, Oct 6, 2021 at 11:15 AM Mike Drob <mdrob@mdrob.com> wrote:
>> >
>> > It feels weird to say that I’m against releases, but generally I feel like bug fix releases should be scoped either for a regression discovered in that release or for rapid security fixes. Otherwise, what’s the harm in waiting for the next release train?
>> >
>> > Obviously any committee is free to create a release candidate on any commit, and if there are three PMC members in support then a release can happen, but I don’t want to be putting pressure on ourselves where we are constantly in the middle of a release cycle.
>> >
>> > Or waiting a month and doing 8.11 seems fine too?
>> >
>> > On Wed, Oct 6, 2021 at 8:17 AM Adrien Grand <jpountz@gmail.com> wrote:
>> >>
>> >> Mike, it's unclear to me if you are suggesting waiting before doing a 8.10.1 release? On my end I'm good with doing a 8.10.1 release now, we could still do a 8.10.2 release later in case we find new bugs?
>> >>
>> >> On Tue, Oct 5, 2021 at 10:41 PM Mayya Sharipova <mayya.sharipova@elastic.co.invalid> wrote:
>> >>>
>> >>> No, the bug is not new and was present in the previous versions as well, but was discovered quite recently.
>> >>>
>> >>> On Tue, Oct 5, 2021 at 3:54 PM Mike Drob <mdrob@apache.org> wrote:
>> >>>>
>> >>>> Is the bug new in 8.10? If it affects older versions as well then I feel like 8.10.1 might be less urgent.
>> >>>>
>> >>>> Mike
>> >>>>
>> >>>> On Tue, Oct 5, 2021 at 2:14 PM Adrien Grand <jpountz@gmail.com> wrote:
>> >>>>>
>> >>>>> +1 to a 8.10.1 patch release
>> >>>>>
>> >>>>> On Tue, Oct 5, 2021 at 2:03 AM Mayya Sharipova <mayya.sharipova@elastic.co.invalid> wrote:
>> >>>>>>
>> >>>>>> Thanks for the update, Robert. Would be nice to have these bug fixes as well.
>> >>>>>>
>> >>>>>> On Mon, Oct 4, 2021 at 7:56 PM Robert Muir <rcmuir@gmail.com> wrote:
>> >>>>>>>
>> >>>>>>> FYI Looks like there are already six items currently listed under
>> >>>>>>> "Bugfixes" for 8.11.0, so those could be candidates for the patch
>> >>>>>>> release.
>> >>>>>>>
>> >>>>>>> Bug Fixes
>> >>>>>>> ---------------------
>> >>>>>>> * LUCENE-10110: MultiCollector now handles single leaf collector that
>> >>>>>>> wants to skip low-scoring hits
>> >>>>>>> but the combined score mode doesn't allow it. (Jim Ferenczi)
>> >>>>>>> * LUCENE-10111: Missing calculating the bytes used of DocsWithFieldSet
>> >>>>>>> in NormValuesWriter.
>> >>>>>>> (Lu Xugang)
>> >>>>>>> * LUCENE-10116: Missing calculating the bytes used of DocsWithFieldSet
>> >>>>>>> and currentValues in SortedSetDocValuesWriter.
>> >>>>>>> (Lu Xugang)
>> >>>>>>> * LUCENE-10070 Skip deleted docs when accumulating facet counts for
>> >>>>>>> all docs. (Ankur Goel, Greg Miller)
>> >>>>>>> * LUCENE-10126: Sort optimization with a chunked bulk scorer
>> >>>>>>> can wrongly skip documents (Nhat Nguyen, Mayya Sharipova)
>> >>>>>>> * LUCENE-10134: ConcurrentSortedSetDocValuesFacetCounts shouldn't
>> >>>>>>> share liveDocs Bits across threads.
>> >>>>>>> (Ankur Goel)
>> >>>>>>>
>> >>>>>>> On Mon, Oct 4, 2021 at 7:46 PM Mayya Sharipova <mayya@apache.org> wrote:
>> >>>>>>> >
>> >>>>>>> > Hello everyone!
>> >>>>>>> > Thank you, Timothy, for the recent 8.10 release.
>> >>>>>>> >
>> >>>>>>> > I wonder if we are ok to do a 8.10.1 patch release and do it fairly soon? this week?
>> >>>>>>> > Nhat fixed a bad bug where "sort with after" on a numeric field can incorrectly miss documents. This bug only manifests when sort optimization on numeric fields is explicitly enabled.
>> >>>>>>> >
>> >>>>>>> > Given that we are not sure when the next minor release will be, it would be useful to have a patch release. If nobody is opposed to it, I can volunteer to be the Release Manager.
>> >>>>>>> >
>> >>>>>>> > Thanks.
>> >>>>>>>
>> >>>>>>> ---------------------------------------------------------------------
>> >>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >>>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>> >>>>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Adrien
>> >>
>> >>
>> >>
>> >> --
>> >> Adrien
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: 8.10.1 Patch release? [ In reply to ]
Thanks everyone,
Sorry for the trouble, I understand it is not a good experience for us to
have frequent releases.
And I aspire to write better tests that can catch bugs earlier.
If nobody is strongly against it, I am adding my +1, and we can proceed
with doing the patch release.

I will be sending another email notifying everyone about the upcoming
release.

On Wed, Oct 6, 2021 at 3:29 PM Timothy Potter <thelabdude@gmail.com> wrote:

> Ah ok thanks for clarifying Nhat. In light of this, doing an 8.10.1
> makes sense to me.
>
> Cheers,
> Tim
>
> On Wed, Oct 6, 2021 at 12:57 PM Nhat Nguyen
> <nhat.nguyen@elastic.co.invalid> wrote:
> >
> > Hello everyone!
> >
> > Thank you for discussing this.
> >
> > There were two bugs during the release of 8.10.0. I considered the first
> bug wasn't a blocker to respin 8.10.0 RC1 and it made into 8.10.0 RC2.
> However, Mayya discovered that the second bug had a severe impact on search
> after requests with sort, and we didn't fully understand its severity until
> 8.10.0 was out.
> > Although the bug has been in the previous versions, I am +1 to release
> 8.10.1 to reduce the impact.
> >
> > Best Regards,
> > Nhat
> >
> > On Wed, Oct 6, 2021 at 2:36 PM Timothy Potter <thelabdude@gmail.com>
> wrote:
> >>
> >> I agree with Mike on this one as well. In addition, I'm surprised
> >> nobody asked to halt the RC1 and make RC2 with Nhat's fix while I was
> >> doing 8.10. Nhat made it sound like it was not a big deal at the time,
> >> but now there's some urgency in releasing it?
> >>
> >> Tim
> >>
> >> On Wed, Oct 6, 2021 at 11:15 AM Mike Drob <mdrob@mdrob.com> wrote:
> >> >
> >> > It feels weird to say that I’m against releases, but generally I feel
> like bug fix releases should be scoped either for a regression discovered
> in that release or for rapid security fixes. Otherwise, what’s the harm in
> waiting for the next release train?
> >> >
> >> > Obviously any committee is free to create a release candidate on any
> commit, and if there are three PMC members in support then a release can
> happen, but I don’t want to be putting pressure on ourselves where we are
> constantly in the middle of a release cycle.
> >> >
> >> > Or waiting a month and doing 8.11 seems fine too?
> >> >
> >> > On Wed, Oct 6, 2021 at 8:17 AM Adrien Grand <jpountz@gmail.com>
> wrote:
> >> >>
> >> >> Mike, it's unclear to me if you are suggesting waiting before doing
> a 8.10.1 release? On my end I'm good with doing a 8.10.1 release now, we
> could still do a 8.10.2 release later in case we find new bugs?
> >> >>
> >> >> On Tue, Oct 5, 2021 at 10:41 PM Mayya Sharipova <
> mayya.sharipova@elastic.co.invalid> wrote:
> >> >>>
> >> >>> No, the bug is not new and was present in the previous versions as
> well, but was discovered quite recently.
> >> >>>
> >> >>> On Tue, Oct 5, 2021 at 3:54 PM Mike Drob <mdrob@apache.org> wrote:
> >> >>>>
> >> >>>> Is the bug new in 8.10? If it affects older versions as well then
> I feel like 8.10.1 might be less urgent.
> >> >>>>
> >> >>>> Mike
> >> >>>>
> >> >>>> On Tue, Oct 5, 2021 at 2:14 PM Adrien Grand <jpountz@gmail.com>
> wrote:
> >> >>>>>
> >> >>>>> +1 to a 8.10.1 patch release
> >> >>>>>
> >> >>>>> On Tue, Oct 5, 2021 at 2:03 AM Mayya Sharipova <
> mayya.sharipova@elastic.co.invalid> wrote:
> >> >>>>>>
> >> >>>>>> Thanks for the update, Robert. Would be nice to have these bug
> fixes as well.
> >> >>>>>>
> >> >>>>>> On Mon, Oct 4, 2021 at 7:56 PM Robert Muir <rcmuir@gmail.com>
> wrote:
> >> >>>>>>>
> >> >>>>>>> FYI Looks like there are already six items currently listed
> under
> >> >>>>>>> "Bugfixes" for 8.11.0, so those could be candidates for the
> patch
> >> >>>>>>> release.
> >> >>>>>>>
> >> >>>>>>> Bug Fixes
> >> >>>>>>> ---------------------
> >> >>>>>>> * LUCENE-10110: MultiCollector now handles single leaf
> collector that
> >> >>>>>>> wants to skip low-scoring hits
> >> >>>>>>> but the combined score mode doesn't allow it. (Jim Ferenczi)
> >> >>>>>>> * LUCENE-10111: Missing calculating the bytes used of
> DocsWithFieldSet
> >> >>>>>>> in NormValuesWriter.
> >> >>>>>>> (Lu Xugang)
> >> >>>>>>> * LUCENE-10116: Missing calculating the bytes used of
> DocsWithFieldSet
> >> >>>>>>> and currentValues in SortedSetDocValuesWriter.
> >> >>>>>>> (Lu Xugang)
> >> >>>>>>> * LUCENE-10070 Skip deleted docs when accumulating facet counts
> for
> >> >>>>>>> all docs. (Ankur Goel, Greg Miller)
> >> >>>>>>> * LUCENE-10126: Sort optimization with a chunked bulk scorer
> >> >>>>>>> can wrongly skip documents (Nhat Nguyen, Mayya Sharipova)
> >> >>>>>>> * LUCENE-10134: ConcurrentSortedSetDocValuesFacetCounts
> shouldn't
> >> >>>>>>> share liveDocs Bits across threads.
> >> >>>>>>> (Ankur Goel)
> >> >>>>>>>
> >> >>>>>>> On Mon, Oct 4, 2021 at 7:46 PM Mayya Sharipova <
> mayya@apache.org> wrote:
> >> >>>>>>> >
> >> >>>>>>> > Hello everyone!
> >> >>>>>>> > Thank you, Timothy, for the recent 8.10 release.
> >> >>>>>>> >
> >> >>>>>>> > I wonder if we are ok to do a 8.10.1 patch release and do it
> fairly soon? this week?
> >> >>>>>>> > Nhat fixed a bad bug where "sort with after" on a numeric
> field can incorrectly miss documents. This bug only manifests when sort
> optimization on numeric fields is explicitly enabled.
> >> >>>>>>> >
> >> >>>>>>> > Given that we are not sure when the next minor release will
> be, it would be useful to have a patch release. If nobody is opposed to
> it, I can volunteer to be the Release Manager.
> >> >>>>>>> >
> >> >>>>>>> > Thanks.
> >> >>>>>>>
> >> >>>>>>>
> ---------------------------------------------------------------------
> >> >>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> >>>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
> >> >>>>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>> --
> >> >>>>> Adrien
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Adrien
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: dev-help@lucene.apache.org
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>