Mailing List Archive

New branch and feature freeze for Lucene/Solr 8.10.0
NOTICE:

Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.

Please observe the normal rules:

* No new features may be committed to the branch.

* Documentation patches, build patches and serious bug fixes may be
committed to the branch. However, you should submit all patches you
want to commit to Jira first to give others the chance to review
and possibly vote against the patch. Keep in mind that it is our
main intention to keep the branch as stable as possible.

* All patches that are intended for the branch should first be committed
to the unstable branch, merged into the stable branch, and then into
the current release branch.

* Normal unstable and stable branch development may continue as usual.
However, if you plan to commit a big change to the unstable branch
while the branch feature freeze is in effect, think twice: can't the
addition wait a couple more days? Merges of bug fixes into the branch
may become more difficult.

* Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
a release candidate build.
----

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
the schema designer. I haven't built the RC yet, so going to see if I
can get this in today.

On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
>
> NOTICE:
>
> Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
>
> Please observe the normal rules:
>
> * No new features may be committed to the branch.
>
> * Documentation patches, build patches and serious bug fixes may be
> committed to the branch. However, you should submit all patches you
> want to commit to Jira first to give others the chance to review
> and possibly vote against the patch. Keep in mind that it is our
> main intention to keep the branch as stable as possible.
>
> * All patches that are intended for the branch should first be committed
> to the unstable branch, merged into the stable branch, and then into
> the current release branch.
>
> * Normal unstable and stable branch development may continue as usual.
> However, if you plan to commit a big change to the unstable branch
> while the branch feature freeze is in effect, think twice: can't the
> addition wait a couple more days? Merges of bug fixes into the branch
> may become more difficult.
>
> * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
> a release candidate build.
> ----

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
both look pretty good, but I've got a few last unit tests that I need
to chase down. Hopefully taken care of by today or tomorrow, I'll be
sure to keep you updated though.


On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com> wrote:
>
> I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
> the schema designer. I haven't built the RC yet, so going to see if I
> can get this in today.
>
> On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
> >
> > NOTICE:
> >
> > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
> >
> > Please observe the normal rules:
> >
> > * No new features may be committed to the branch.
> >
> > * Documentation patches, build patches and serious bug fixes may be
> > committed to the branch. However, you should submit all patches you
> > want to commit to Jira first to give others the chance to review
> > and possibly vote against the patch. Keep in mind that it is our
> > main intention to keep the branch as stable as possible.
> >
> > * All patches that are intended for the branch should first be committed
> > to the unstable branch, merged into the stable branch, and then into
> > the current release branch.
> >
> > * Normal unstable and stable branch development may continue as usual.
> > However, if you plan to commit a big change to the unstable branch
> > while the branch feature freeze is in effect, think twice: can't the
> > addition wait a couple more days? Merges of bug fixes into the branch
> > may become more difficult.
> >
> > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
> > a release candidate build.
> > ----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
Thanks for the update Mike!

I'm backporting SOLR-15620 right now and am cooking up a quick PR for
SOLR-15621, which looks like an easy win for the issue Cassandra
reported on Slack earlier today.

Cheers,
Tim

On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
>
> Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
> both look pretty good, but I've got a few last unit tests that I need
> to chase down. Hopefully taken care of by today or tomorrow, I'll be
> sure to keep you updated though.
>
>
> On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com> wrote:
> >
> > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
> > the schema designer. I haven't built the RC yet, so going to see if I
> > can get this in today.
> >
> > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
> > >
> > > NOTICE:
> > >
> > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
> > >
> > > Please observe the normal rules:
> > >
> > > * No new features may be committed to the branch.
> > >
> > > * Documentation patches, build patches and serious bug fixes may be
> > > committed to the branch. However, you should submit all patches you
> > > want to commit to Jira first to give others the chance to review
> > > and possibly vote against the patch. Keep in mind that it is our
> > > main intention to keep the branch as stable as possible.
> > >
> > > * All patches that are intended for the branch should first be committed
> > > to the unstable branch, merged into the stable branch, and then into
> > > the current release branch.
> > >
> > > * Normal unstable and stable branch development may continue as usual.
> > > However, if you plan to commit a big change to the unstable branch
> > > while the branch feature freeze is in effect, think twice: can't the
> > > addition wait a couple more days? Merges of bug fixes into the branch
> > > may become more difficult.
> > >
> > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
> > > a release candidate build.
> > > ----
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
Building RC1 now ... stay tuned.

On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com> wrote:
>
> Thanks for the update Mike!
>
> I'm backporting SOLR-15620 right now and am cooking up a quick PR for
> SOLR-15621, which looks like an easy win for the issue Cassandra
> reported on Slack earlier today.
>
> Cheers,
> Tim
>
> On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
> >
> > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
> > both look pretty good, but I've got a few last unit tests that I need
> > to chase down. Hopefully taken care of by today or tomorrow, I'll be
> > sure to keep you updated though.
> >
> >
> > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com> wrote:
> > >
> > > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
> > > the schema designer. I haven't built the RC yet, so going to see if I
> > > can get this in today.
> > >
> > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
> > > >
> > > > NOTICE:
> > > >
> > > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
> > > >
> > > > Please observe the normal rules:
> > > >
> > > > * No new features may be committed to the branch.
> > > >
> > > > * Documentation patches, build patches and serious bug fixes may be
> > > > committed to the branch. However, you should submit all patches you
> > > > want to commit to Jira first to give others the chance to review
> > > > and possibly vote against the patch. Keep in mind that it is our
> > > > main intention to keep the branch as stable as possible.
> > > >
> > > > * All patches that are intended for the branch should first be committed
> > > > to the unstable branch, merged into the stable branch, and then into
> > > > the current release branch.
> > > >
> > > > * Normal unstable and stable branch development may continue as usual.
> > > > However, if you plan to commit a big change to the unstable branch
> > > > while the branch feature freeze is in effect, think twice: can't the
> > > > addition wait a couple more days? Merges of bug fixes into the branch
> > > > may become more difficult.
> > > >
> > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
> > > > a release candidate build.
> > > > ----
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: dev-help@lucene.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
All the best, this is the worst step.

On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com> wrote:

> Building RC1 now ... stay tuned.
>
> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com>
> wrote:
> >
> > Thanks for the update Mike!
> >
> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
> > SOLR-15621, which looks like an easy win for the issue Cassandra
> > reported on Slack earlier today.
> >
> > Cheers,
> > Tim
> >
> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
> > >
> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
> > > both look pretty good, but I've got a few last unit tests that I need
> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
> > > sure to keep you updated though.
> > >
> > >
> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com>
> wrote:
> > > >
> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while
> testing
> > > > the schema designer. I haven't built the RC yet, so going to see if I
> > > > can get this in today.
> > > >
> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <
> thelabdude@apache.org> wrote:
> > > > >
> > > > > NOTICE:
> > > > >
> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on
> stable branch.
> > > > >
> > > > > Please observe the normal rules:
> > > > >
> > > > > * No new features may be committed to the branch.
> > > > >
> > > > > * Documentation patches, build patches and serious bug fixes may be
> > > > > committed to the branch. However, you should submit all patches
> you
> > > > > want to commit to Jira first to give others the chance to review
> > > > > and possibly vote against the patch. Keep in mind that it is our
> > > > > main intention to keep the branch as stable as possible.
> > > > >
> > > > > * All patches that are intended for the branch should first be
> committed
> > > > > to the unstable branch, merged into the stable branch, and then
> into
> > > > > the current release branch.
> > > > >
> > > > > * Normal unstable and stable branch development may continue as
> usual.
> > > > > However, if you plan to commit a big change to the unstable
> branch
> > > > > while the branch feature freeze is in effect, think twice: can't
> the
> > > > > addition wait a couple more days? Merges of bug fixes into the
> branch
> > > > > may become more difficult.
> > > > >
> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker"
> will delay
> > > > > a release candidate build.
> > > > > ----
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: dev-help@lucene.apache.org
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
headed out on vacation tomorrow, back at keys on Monday, Sept 20
unless someone else wants to pick up the RM duties before then?

After failing the test suite at various places and other weirdness
like .asc files not getting created, I finally got to the smoke test
part, which is now failing with:

File "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
line 176, in checkJARMetaData
raise RuntimeError('%s is missing "%s" inside its
META-INF/MANIFEST.MF (wrong git revision?)' % \
RuntimeError: JAR file
"/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
is missing "Implementation-Version: 8.10.0
ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
META-INF/MANIFEST.MF (wrong git revision?)

FWIW, I verified that the other Lucene JAR files have this line in
them, such as core:

Manifest-Version: 1.0
Ant-Version: Apache Ant 1.9.15
Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
Extension-Name: org.apache.lucene
Specification-Title: Lucene Search Engine: core
Specification-Version: 8.10.0
Specification-Vendor: The Apache Software Foundation
Implementation-Title: org.apache.lucene
Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
7 - tjp - 2021-09-14 19:08:42
Implementation-Vendor: The Apache Software Foundation
X-Compile-Source-JDK: 8
X-Compile-Target-JDK: 8
Multi-Release: true

On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
<ichattopadhyaya@gmail.com> wrote:
>
> All the best, this is the worst step.
>
> On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com> wrote:
>>
>> Building RC1 now ... stay tuned.
>>
>> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com> wrote:
>> >
>> > Thanks for the update Mike!
>> >
>> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
>> > SOLR-15621, which looks like an easy win for the issue Cassandra
>> > reported on Slack earlier today.
>> >
>> > Cheers,
>> > Tim
>> >
>> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
>> > >
>> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
>> > > both look pretty good, but I've got a few last unit tests that I need
>> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
>> > > sure to keep you updated though.
>> > >
>> > >
>> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com> wrote:
>> > > >
>> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
>> > > > the schema designer. I haven't built the RC yet, so going to see if I
>> > > > can get this in today.
>> > > >
>> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
>> > > > >
>> > > > > NOTICE:
>> > > > >
>> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
>> > > > >
>> > > > > Please observe the normal rules:
>> > > > >
>> > > > > * No new features may be committed to the branch.
>> > > > >
>> > > > > * Documentation patches, build patches and serious bug fixes may be
>> > > > > committed to the branch. However, you should submit all patches you
>> > > > > want to commit to Jira first to give others the chance to review
>> > > > > and possibly vote against the patch. Keep in mind that it is our
>> > > > > main intention to keep the branch as stable as possible.
>> > > > >
>> > > > > * All patches that are intended for the branch should first be committed
>> > > > > to the unstable branch, merged into the stable branch, and then into
>> > > > > the current release branch.
>> > > > >
>> > > > > * Normal unstable and stable branch development may continue as usual.
>> > > > > However, if you plan to commit a big change to the unstable branch
>> > > > > while the branch feature freeze is in effect, think twice: can't the
>> > > > > addition wait a couple more days? Merges of bug fixes into the branch
>> > > > > may become more difficult.
>> > > > >
>> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
>> > > > > a release candidate build.
>> > > > > ----
>> > > >
>> > > > ---------------------------------------------------------------------
>> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > > > For additional commands, e-mail: dev-help@lucene.apache.org
>> > > >
>> > >
>> > > ---------------------------------------------------------------------
>> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > > For additional commands, e-mail: dev-help@lucene.apache.org
>> > >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> For additional commands, e-mail: dev-help@solr.apache.org
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
Hello everyone,
We have discovered a bug and fixed a bug in Lucene sort optimization
<https://github.com/apache/lucene/pull/300> (LUCENE-10106) and would like
to merge it to Lucene 8.10 if it is not too late.
I apologize for the inconvenience, the bug was discovered just yesterday.

On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org>
wrote:

> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
> headed out on vacation tomorrow, back at keys on Monday, Sept 20
> unless someone else wants to pick up the RM duties before then?
>
> After failing the test suite at various places and other weirdness
> like .asc files not getting created, I finally got to the smoke test
> part, which is now failing with:
>
> File
> "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
> line 176, in checkJARMetaData
> raise RuntimeError('%s is missing "%s" inside its
> META-INF/MANIFEST.MF (wrong git revision?)' % \
> RuntimeError: JAR file
>
> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
> is missing "Implementation-Version: 8.10.0
> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
> META-INF/MANIFEST.MF (wrong git revision?)
>
> FWIW, I verified that the other Lucene JAR files have this line in
> them, such as core:
>
> Manifest-Version: 1.0
> Ant-Version: Apache Ant 1.9.15
> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
> Extension-Name: org.apache.lucene
> Specification-Title: Lucene Search Engine: core
> Specification-Version: 8.10.0
> Specification-Vendor: The Apache Software Foundation
> Implementation-Title: org.apache.lucene
> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
> 7 - tjp - 2021-09-14 19:08:42
> Implementation-Vendor: The Apache Software Foundation
> X-Compile-Source-JDK: 8
> X-Compile-Target-JDK: 8
> Multi-Release: true
>
> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
> <ichattopadhyaya@gmail.com> wrote:
> >
> > All the best, this is the worst step.
> >
> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com>
> wrote:
> >>
> >> Building RC1 now ... stay tuned.
> >>
> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com>
> wrote:
> >> >
> >> > Thanks for the update Mike!
> >> >
> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
> >> > reported on Slack earlier today.
> >> >
> >> > Cheers,
> >> > Tim
> >> >
> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
> >> > >
> >> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
> >> > > both look pretty good, but I've got a few last unit tests that I
> need
> >> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
> >> > > sure to keep you updated though.
> >> > >
> >> > >
> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <
> thelabdude@gmail.com> wrote:
> >> > > >
> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while
> testing
> >> > > > the schema designer. I haven't built the RC yet, so going to see
> if I
> >> > > > can get this in today.
> >> > > >
> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <
> thelabdude@apache.org> wrote:
> >> > > > >
> >> > > > > NOTICE:
> >> > > > >
> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on
> stable branch.
> >> > > > >
> >> > > > > Please observe the normal rules:
> >> > > > >
> >> > > > > * No new features may be committed to the branch.
> >> > > > >
> >> > > > > * Documentation patches, build patches and serious bug fixes
> may be
> >> > > > > committed to the branch. However, you should submit all
> patches you
> >> > > > > want to commit to Jira first to give others the chance to
> review
> >> > > > > and possibly vote against the patch. Keep in mind that it is
> our
> >> > > > > main intention to keep the branch as stable as possible.
> >> > > > >
> >> > > > > * All patches that are intended for the branch should first be
> committed
> >> > > > > to the unstable branch, merged into the stable branch, and
> then into
> >> > > > > the current release branch.
> >> > > > >
> >> > > > > * Normal unstable and stable branch development may continue as
> usual.
> >> > > > > However, if you plan to commit a big change to the unstable
> branch
> >> > > > > while the branch feature freeze is in effect, think twice:
> can't the
> >> > > > > addition wait a couple more days? Merges of bug fixes into
> the branch
> >> > > > > may become more difficult.
> >> > > > >
> >> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker"
> will delay
> >> > > > > a release candidate build.
> >> > > > > ----
> >> > > >
> >> > > >
> ---------------------------------------------------------------------
> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
> >> > > >
> >> > >
> >> > >
> ---------------------------------------------------------------------
> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
> >> > >
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> >> For additional commands, e-mail: dev-help@solr.apache.org
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
I think since Tim is out on vacation, it's probably not too late. That
looks like a good fix to have, do we know how long the bug has been present?

On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova
<mayya.sharipova@elastic.co.invalid> wrote:

> Hello everyone,
> We have discovered a bug and fixed a bug in Lucene sort optimization
> <https://github.com/apache/lucene/pull/300> (LUCENE-10106) and would like
> to merge it to Lucene 8.10 if it is not too late.
> I apologize for the inconvenience, the bug was discovered just yesterday.
>
> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org>
> wrote:
>
>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
>> unless someone else wants to pick up the RM duties before then?
>>
>> After failing the test suite at various places and other weirdness
>> like .asc files not getting created, I finally got to the smoke test
>> part, which is now failing with:
>>
>> File
>> "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
>> line 176, in checkJARMetaData
>> raise RuntimeError('%s is missing "%s" inside its
>> META-INF/MANIFEST.MF (wrong git revision?)' % \
>> RuntimeError: JAR file
>>
>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
>> is missing "Implementation-Version: 8.10.0
>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
>> META-INF/MANIFEST.MF (wrong git revision?)
>>
>> FWIW, I verified that the other Lucene JAR files have this line in
>> them, such as core:
>>
>> Manifest-Version: 1.0
>> Ant-Version: Apache Ant 1.9.15
>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
>> Extension-Name: org.apache.lucene
>> Specification-Title: Lucene Search Engine: core
>> Specification-Version: 8.10.0
>> Specification-Vendor: The Apache Software Foundation
>> Implementation-Title: org.apache.lucene
>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
>> 7 - tjp - 2021-09-14 19:08:42
>> Implementation-Vendor: The Apache Software Foundation
>> X-Compile-Source-JDK: 8
>> X-Compile-Target-JDK: 8
>> Multi-Release: true
>>
>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
>> <ichattopadhyaya@gmail.com> wrote:
>> >
>> > All the best, this is the worst step.
>> >
>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com>
>> wrote:
>> >>
>> >> Building RC1 now ... stay tuned.
>> >>
>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com>
>> wrote:
>> >> >
>> >> > Thanks for the update Mike!
>> >> >
>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
>> >> > reported on Slack earlier today.
>> >> >
>> >> > Cheers,
>> >> > Tim
>> >> >
>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
>> >> > >
>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
>> >> > > both look pretty good, but I've got a few last unit tests that I
>> need
>> >> > > to chase down. Hopefully taken care of by today or tomorrow, I'll
>> be
>> >> > > sure to keep you updated though.
>> >> > >
>> >> > >
>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <
>> thelabdude@gmail.com> wrote:
>> >> > > >
>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while
>> testing
>> >> > > > the schema designer. I haven't built the RC yet, so going to see
>> if I
>> >> > > > can get this in today.
>> >> > > >
>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <
>> thelabdude@apache.org> wrote:
>> >> > > > >
>> >> > > > > NOTICE:
>> >> > > > >
>> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11
>> on stable branch.
>> >> > > > >
>> >> > > > > Please observe the normal rules:
>> >> > > > >
>> >> > > > > * No new features may be committed to the branch.
>> >> > > > >
>> >> > > > > * Documentation patches, build patches and serious bug fixes
>> may be
>> >> > > > > committed to the branch. However, you should submit all
>> patches you
>> >> > > > > want to commit to Jira first to give others the chance to
>> review
>> >> > > > > and possibly vote against the patch. Keep in mind that it is
>> our
>> >> > > > > main intention to keep the branch as stable as possible.
>> >> > > > >
>> >> > > > > * All patches that are intended for the branch should first be
>> committed
>> >> > > > > to the unstable branch, merged into the stable branch, and
>> then into
>> >> > > > > the current release branch.
>> >> > > > >
>> >> > > > > * Normal unstable and stable branch development may continue
>> as usual.
>> >> > > > > However, if you plan to commit a big change to the unstable
>> branch
>> >> > > > > while the branch feature freeze is in effect, think twice:
>> can't the
>> >> > > > > addition wait a couple more days? Merges of bug fixes into
>> the branch
>> >> > > > > may become more difficult.
>> >> > > > >
>> >> > > > > * Only Jira issues with Fix version 8.10 and priority
>> "Blocker" will delay
>> >> > > > > a release candidate build.
>> >> > > > > ----
>> >> > > >
>> >> > > >
>> ---------------------------------------------------------------------
>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
>> >> > > >
>> >> > >
>> >> > >
>> ---------------------------------------------------------------------
>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
>> >> > >
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> >> For additional commands, e-mail: dev-help@solr.apache.org
>> >>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
Thanks Mayya and Mike! I will backport it to the 8.10 branch.

On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com> wrote:

> I think since Tim is out on vacation, it's probably not too late. That
> looks like a good fix to have, do we know how long the bug has been present?
>
> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova
> <mayya.sharipova@elastic.co.invalid> wrote:
>
>> Hello everyone,
>> We have discovered a bug and fixed a bug in Lucene sort optimization
>> <https://github.com/apache/lucene/pull/300> (LUCENE-10106) and would
>> like to merge it to Lucene 8.10 if it is not too late.
>> I apologize for the inconvenience, the bug was discovered just yesterday.
>>
>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org>
>> wrote:
>>
>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
>>> unless someone else wants to pick up the RM duties before then?
>>>
>>> After failing the test suite at various places and other weirdness
>>> like .asc files not getting created, I finally got to the smoke test
>>> part, which is now failing with:
>>>
>>> File
>>> "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
>>> line 176, in checkJARMetaData
>>> raise RuntimeError('%s is missing "%s" inside its
>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
>>> RuntimeError: JAR file
>>>
>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
>>> is missing "Implementation-Version: 8.10.0
>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
>>> META-INF/MANIFEST.MF (wrong git revision?)
>>>
>>> FWIW, I verified that the other Lucene JAR files have this line in
>>> them, such as core:
>>>
>>> Manifest-Version: 1.0
>>> Ant-Version: Apache Ant 1.9.15
>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
>>> Extension-Name: org.apache.lucene
>>> Specification-Title: Lucene Search Engine: core
>>> Specification-Version: 8.10.0
>>> Specification-Vendor: The Apache Software Foundation
>>> Implementation-Title: org.apache.lucene
>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
>>> 7 - tjp - 2021-09-14 19:08:42
>>> Implementation-Vendor: The Apache Software Foundation
>>> X-Compile-Source-JDK: 8
>>> X-Compile-Target-JDK: 8
>>> Multi-Release: true
>>>
>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
>>> <ichattopadhyaya@gmail.com> wrote:
>>> >
>>> > All the best, this is the worst step.
>>> >
>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com>
>>> wrote:
>>> >>
>>> >> Building RC1 now ... stay tuned.
>>> >>
>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com>
>>> wrote:
>>> >> >
>>> >> > Thanks for the update Mike!
>>> >> >
>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR
>>> for
>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
>>> >> > reported on Slack earlier today.
>>> >> >
>>> >> > Cheers,
>>> >> > Tim
>>> >> >
>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
>>> >> > >
>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
>>> >> > > both look pretty good, but I've got a few last unit tests that I
>>> need
>>> >> > > to chase down. Hopefully taken care of by today or tomorrow, I'll
>>> be
>>> >> > > sure to keep you updated though.
>>> >> > >
>>> >> > >
>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <
>>> thelabdude@gmail.com> wrote:
>>> >> > > >
>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while
>>> testing
>>> >> > > > the schema designer. I haven't built the RC yet, so going to
>>> see if I
>>> >> > > > can get this in today.
>>> >> > > >
>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <
>>> thelabdude@apache.org> wrote:
>>> >> > > > >
>>> >> > > > > NOTICE:
>>> >> > > > >
>>> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11
>>> on stable branch.
>>> >> > > > >
>>> >> > > > > Please observe the normal rules:
>>> >> > > > >
>>> >> > > > > * No new features may be committed to the branch.
>>> >> > > > >
>>> >> > > > > * Documentation patches, build patches and serious bug fixes
>>> may be
>>> >> > > > > committed to the branch. However, you should submit all
>>> patches you
>>> >> > > > > want to commit to Jira first to give others the chance to
>>> review
>>> >> > > > > and possibly vote against the patch. Keep in mind that it
>>> is our
>>> >> > > > > main intention to keep the branch as stable as possible.
>>> >> > > > >
>>> >> > > > > * All patches that are intended for the branch should first
>>> be committed
>>> >> > > > > to the unstable branch, merged into the stable branch, and
>>> then into
>>> >> > > > > the current release branch.
>>> >> > > > >
>>> >> > > > > * Normal unstable and stable branch development may continue
>>> as usual.
>>> >> > > > > However, if you plan to commit a big change to the unstable
>>> branch
>>> >> > > > > while the branch feature freeze is in effect, think twice:
>>> can't the
>>> >> > > > > addition wait a couple more days? Merges of bug fixes into
>>> the branch
>>> >> > > > > may become more difficult.
>>> >> > > > >
>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority
>>> "Blocker" will delay
>>> >> > > > > a release candidate build.
>>> >> > > > > ----
>>> >> > > >
>>> >> > > >
>>> ---------------------------------------------------------------------
>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
>>> >> > > >
>>> >> > >
>>> >> > >
>>> ---------------------------------------------------------------------
>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
>>> >> > >
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>> >> For additional commands, e-mail: dev-help@solr.apache.org
>>> >>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
can someone also please look into that benchmark jar issue?

Sent from my iPhone

> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co.invalid> wrote:
>
> ?
> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
>
>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com> wrote:
>> I think since Tim is out on vacation, it's probably not too late. That looks like a good fix to have, do we know how long the bug has been present?
>>
>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova <mayya.sharipova@elastic.co.invalid> wrote:
>>> Hello everyone,
>>> We have discovered a bug and fixed a bug in Lucene sort optimization (LUCENE-10106) and would like to merge it to Lucene 8.10 if it is not too late.
>>> I apologize for the inconvenience, the bug was discovered just yesterday.
>>>
>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org> wrote:
>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
>>>> unless someone else wants to pick up the RM duties before then?
>>>>
>>>> After failing the test suite at various places and other weirdness
>>>> like .asc files not getting created, I finally got to the smoke test
>>>> part, which is now failing with:
>>>>
>>>> File "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
>>>> line 176, in checkJARMetaData
>>>> raise RuntimeError('%s is missing "%s" inside its
>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
>>>> RuntimeError: JAR file
>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
>>>> is missing "Implementation-Version: 8.10.0
>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
>>>> META-INF/MANIFEST.MF (wrong git revision?)
>>>>
>>>> FWIW, I verified that the other Lucene JAR files have this line in
>>>> them, such as core:
>>>>
>>>> Manifest-Version: 1.0
>>>> Ant-Version: Apache Ant 1.9.15
>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
>>>> Extension-Name: org.apache.lucene
>>>> Specification-Title: Lucene Search Engine: core
>>>> Specification-Version: 8.10.0
>>>> Specification-Vendor: The Apache Software Foundation
>>>> Implementation-Title: org.apache.lucene
>>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
>>>> 7 - tjp - 2021-09-14 19:08:42
>>>> Implementation-Vendor: The Apache Software Foundation
>>>> X-Compile-Source-JDK: 8
>>>> X-Compile-Target-JDK: 8
>>>> Multi-Release: true
>>>>
>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
>>>> <ichattopadhyaya@gmail.com> wrote:
>>>> >
>>>> > All the best, this is the worst step.
>>>> >
>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com> wrote:
>>>> >>
>>>> >> Building RC1 now ... stay tuned.
>>>> >>
>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com> wrote:
>>>> >> >
>>>> >> > Thanks for the update Mike!
>>>> >> >
>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
>>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
>>>> >> > reported on Slack earlier today.
>>>> >> >
>>>> >> > Cheers,
>>>> >> > Tim
>>>> >> >
>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
>>>> >> > >
>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
>>>> >> > > both look pretty good, but I've got a few last unit tests that I need
>>>> >> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
>>>> >> > > sure to keep you updated though.
>>>> >> > >
>>>> >> > >
>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com> wrote:
>>>> >> > > >
>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
>>>> >> > > > the schema designer. I haven't built the RC yet, so going to see if I
>>>> >> > > > can get this in today.
>>>> >> > > >
>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
>>>> >> > > > >
>>>> >> > > > > NOTICE:
>>>> >> > > > >
>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
>>>> >> > > > >
>>>> >> > > > > Please observe the normal rules:
>>>> >> > > > >
>>>> >> > > > > * No new features may be committed to the branch.
>>>> >> > > > >
>>>> >> > > > > * Documentation patches, build patches and serious bug fixes may be
>>>> >> > > > > committed to the branch. However, you should submit all patches you
>>>> >> > > > > want to commit to Jira first to give others the chance to review
>>>> >> > > > > and possibly vote against the patch. Keep in mind that it is our
>>>> >> > > > > main intention to keep the branch as stable as possible.
>>>> >> > > > >
>>>> >> > > > > * All patches that are intended for the branch should first be committed
>>>> >> > > > > to the unstable branch, merged into the stable branch, and then into
>>>> >> > > > > the current release branch.
>>>> >> > > > >
>>>> >> > > > > * Normal unstable and stable branch development may continue as usual.
>>>> >> > > > > However, if you plan to commit a big change to the unstable branch
>>>> >> > > > > while the branch feature freeze is in effect, think twice: can't the
>>>> >> > > > > addition wait a couple more days? Merges of bug fixes into the branch
>>>> >> > > > > may become more difficult.
>>>> >> > > > >
>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
>>>> >> > > > > a release candidate build.
>>>> >> > > > > ----
>>>> >> > > >
>>>> >> > > > ---------------------------------------------------------------------
>>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
>>>> >> > > >
>>>> >> > >
>>>> >> > > ---------------------------------------------------------------------
>>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
>>>> >> > >
>>>> >>
>>>> >> ---------------------------------------------------------------------
>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>> >> For additional commands, e-mail: dev-help@solr.apache.org
>>>> >>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
The benchmark jar has the info we need… sort of. When I built it, it has:

Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
1 - mdrob - 2021-09-15 11:40:36


and it’s looking for Implementation-Version: 8.10.0
75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.

Because 8.10 is a character longer than 8.9, we happen to wrap the last
character of the git commit sha. From the manifest spec:

No line may be longer than 72 bytes (not characters), in its UTF8-encoded form.
If a value would make the initial line longer than this, it should be continued
on extra lines (each starting with a single SPACE).

And we were already teetering on the edge of that limit. We'll run into
this problem again in a few years when we try to release version 10.0.0, so
solving it now has practical benefits down the line.

There's a few options that I can come up with -
1. Use the short-hash when we generate the jar
2. Use the short-hash when we check the contents in the smoke test
3. Do some line join magic in the smoke test.

I'm leaning towards number 1 as I feel that would still be unique enough
for our needs, but would like to hear from others as well.

On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <thelabdude@gmail.com> wrote:

> can someone also please look into that benchmark jar issue?
>
> Sent from my iPhone
>
> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co.invalid>
> wrote:
>
> ?
> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
>
> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com> wrote:
>
>> I think since Tim is out on vacation, it's probably not too late. That
>> looks like a good fix to have, do we know how long the bug has been present?
>>
>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova
>> <mayya.sharipova@elastic.co.invalid> wrote:
>>
>>> Hello everyone,
>>> We have discovered a bug and fixed a bug in Lucene sort optimization
>>> <https://github.com/apache/lucene/pull/300> (LUCENE-10106) and would
>>> like to merge it to Lucene 8.10 if it is not too late.
>>> I apologize for the inconvenience, the bug was discovered just
>>> yesterday.
>>>
>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org>
>>> wrote:
>>>
>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
>>>> unless someone else wants to pick up the RM duties before then?
>>>>
>>>> After failing the test suite at various places and other weirdness
>>>> like .asc files not getting created, I finally got to the smoke test
>>>> part, which is now failing with:
>>>>
>>>> File
>>>> "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
>>>> line 176, in checkJARMetaData
>>>> raise RuntimeError('%s is missing "%s" inside its
>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
>>>> RuntimeError: JAR file
>>>>
>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
>>>> is missing "Implementation-Version: 8.10.0
>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
>>>> META-INF/MANIFEST.MF (wrong git revision?)
>>>>
>>>> FWIW, I verified that the other Lucene JAR files have this line in
>>>> them, such as core:
>>>>
>>>> Manifest-Version: 1.0
>>>> Ant-Version: Apache Ant 1.9.15
>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
>>>> Extension-Name: org.apache.lucene
>>>> Specification-Title: Lucene Search Engine: core
>>>> Specification-Version: 8.10.0
>>>> Specification-Vendor: The Apache Software Foundation
>>>> Implementation-Title: org.apache.lucene
>>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
>>>> 7 - tjp - 2021-09-14 19:08:42
>>>> Implementation-Vendor: The Apache Software Foundation
>>>> X-Compile-Source-JDK: 8
>>>> X-Compile-Target-JDK: 8
>>>> Multi-Release: true
>>>>
>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
>>>> <ichattopadhyaya@gmail.com> wrote:
>>>> >
>>>> > All the best, this is the worst step.
>>>> >
>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> Building RC1 now ... stay tuned.
>>>> >>
>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com>
>>>> wrote:
>>>> >> >
>>>> >> > Thanks for the update Mike!
>>>> >> >
>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR
>>>> for
>>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
>>>> >> > reported on Slack earlier today.
>>>> >> >
>>>> >> > Cheers,
>>>> >> > Tim
>>>> >> >
>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org>
>>>> wrote:
>>>> >> > >
>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and
>>>> benchmarking
>>>> >> > > both look pretty good, but I've got a few last unit tests that I
>>>> need
>>>> >> > > to chase down. Hopefully taken care of by today or tomorrow,
>>>> I'll be
>>>> >> > > sure to keep you updated though.
>>>> >> > >
>>>> >> > >
>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <
>>>> thelabdude@gmail.com> wrote:
>>>> >> > > >
>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620
>>>> while testing
>>>> >> > > > the schema designer. I haven't built the RC yet, so going to
>>>> see if I
>>>> >> > > > can get this in today.
>>>> >> > > >
>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <
>>>> thelabdude@apache.org> wrote:
>>>> >> > > > >
>>>> >> > > > > NOTICE:
>>>> >> > > > >
>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11
>>>> on stable branch.
>>>> >> > > > >
>>>> >> > > > > Please observe the normal rules:
>>>> >> > > > >
>>>> >> > > > > * No new features may be committed to the branch.
>>>> >> > > > >
>>>> >> > > > > * Documentation patches, build patches and serious bug fixes
>>>> may be
>>>> >> > > > > committed to the branch. However, you should submit all
>>>> patches you
>>>> >> > > > > want to commit to Jira first to give others the chance to
>>>> review
>>>> >> > > > > and possibly vote against the patch. Keep in mind that it
>>>> is our
>>>> >> > > > > main intention to keep the branch as stable as possible.
>>>> >> > > > >
>>>> >> > > > > * All patches that are intended for the branch should first
>>>> be committed
>>>> >> > > > > to the unstable branch, merged into the stable branch, and
>>>> then into
>>>> >> > > > > the current release branch.
>>>> >> > > > >
>>>> >> > > > > * Normal unstable and stable branch development may continue
>>>> as usual.
>>>> >> > > > > However, if you plan to commit a big change to the
>>>> unstable branch
>>>> >> > > > > while the branch feature freeze is in effect, think twice:
>>>> can't the
>>>> >> > > > > addition wait a couple more days? Merges of bug fixes into
>>>> the branch
>>>> >> > > > > may become more difficult.
>>>> >> > > > >
>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority
>>>> "Blocker" will delay
>>>> >> > > > > a release candidate build.
>>>> >> > > > > ----
>>>> >> > > >
>>>> >> > > >
>>>> ---------------------------------------------------------------------
>>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
>>>> >> > > >
>>>> >> > >
>>>> >> > >
>>>> ---------------------------------------------------------------------
>>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
>>>> >> > >
>>>> >>
>>>> >> ---------------------------------------------------------------------
>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>> >> For additional commands, e-mail: dev-help@solr.apache.org
>>>> >>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>
>>>>
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
+1 for option #1 Let's control the hash to a reasonable size, so that
the version number will "fit"

We would hit this same problem with 10.0.0 and then we'd have to
revisit it again with 10.10.0, so let's not try to solve it
onesy-twosey at this time

On Wed, Sep 15, 2021 at 2:46 PM Mike Drob <mdrob@mdrob.com> wrote:
>
> The benchmark jar has the info we need… sort of. When I built it, it has:
>
> Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
> 1 - mdrob - 2021-09-15 11:40:36
>
>
> and it’s looking for Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
>
> Because 8.10 is a character longer than 8.9, we happen to wrap the last character of the git commit sha. From the manifest spec:
>
> No line may be longer than 72 bytes (not characters), in its UTF8-encoded form.
> If a value would make the initial line longer than this, it should be continued
> on extra lines (each starting with a single SPACE).
>
> And we were already teetering on the edge of that limit. We'll run into this problem again in a few years when we try to release version 10.0.0, so solving it now has practical benefits down the line.
>
> There's a few options that I can come up with -
> 1. Use the short-hash when we generate the jar
> 2. Use the short-hash when we check the contents in the smoke test
> 3. Do some line join magic in the smoke test.
>
> I'm leaning towards number 1 as I feel that would still be unique enough for our needs, but would like to hear from others as well.
>
> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <thelabdude@gmail.com> wrote:
>>
>> can someone also please look into that benchmark jar issue?
>>
>> Sent from my iPhone
>>
>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co.invalid> wrote:
>>
>> ?
>> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
>>
>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com> wrote:
>>>
>>> I think since Tim is out on vacation, it's probably not too late. That looks like a good fix to have, do we know how long the bug has been present?
>>>
>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova <mayya.sharipova@elastic.co.invalid> wrote:
>>>>
>>>> Hello everyone,
>>>> We have discovered a bug and fixed a bug in Lucene sort optimization (LUCENE-10106) and would like to merge it to Lucene 8.10 if it is not too late.
>>>> I apologize for the inconvenience, the bug was discovered just yesterday.
>>>>
>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org> wrote:
>>>>>
>>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
>>>>> unless someone else wants to pick up the RM duties before then?
>>>>>
>>>>> After failing the test suite at various places and other weirdness
>>>>> like .asc files not getting created, I finally got to the smoke test
>>>>> part, which is now failing with:
>>>>>
>>>>> File "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
>>>>> line 176, in checkJARMetaData
>>>>> raise RuntimeError('%s is missing "%s" inside its
>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
>>>>> RuntimeError: JAR file
>>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
>>>>> is missing "Implementation-Version: 8.10.0
>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
>>>>> META-INF/MANIFEST.MF (wrong git revision?)
>>>>>
>>>>> FWIW, I verified that the other Lucene JAR files have this line in
>>>>> them, such as core:
>>>>>
>>>>> Manifest-Version: 1.0
>>>>> Ant-Version: Apache Ant 1.9.15
>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
>>>>> Extension-Name: org.apache.lucene
>>>>> Specification-Title: Lucene Search Engine: core
>>>>> Specification-Version: 8.10.0
>>>>> Specification-Vendor: The Apache Software Foundation
>>>>> Implementation-Title: org.apache.lucene
>>>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
>>>>> 7 - tjp - 2021-09-14 19:08:42
>>>>> Implementation-Vendor: The Apache Software Foundation
>>>>> X-Compile-Source-JDK: 8
>>>>> X-Compile-Target-JDK: 8
>>>>> Multi-Release: true
>>>>>
>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
>>>>> <ichattopadhyaya@gmail.com> wrote:
>>>>> >
>>>>> > All the best, this is the worst step.
>>>>> >
>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com> wrote:
>>>>> >>
>>>>> >> Building RC1 now ... stay tuned.
>>>>> >>
>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com> wrote:
>>>>> >> >
>>>>> >> > Thanks for the update Mike!
>>>>> >> >
>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
>>>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
>>>>> >> > reported on Slack earlier today.
>>>>> >> >
>>>>> >> > Cheers,
>>>>> >> > Tim
>>>>> >> >
>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
>>>>> >> > >
>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
>>>>> >> > > both look pretty good, but I've got a few last unit tests that I need
>>>>> >> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
>>>>> >> > > sure to keep you updated though.
>>>>> >> > >
>>>>> >> > >
>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com> wrote:
>>>>> >> > > >
>>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
>>>>> >> > > > the schema designer. I haven't built the RC yet, so going to see if I
>>>>> >> > > > can get this in today.
>>>>> >> > > >
>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
>>>>> >> > > > >
>>>>> >> > > > > NOTICE:
>>>>> >> > > > >
>>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
>>>>> >> > > > >
>>>>> >> > > > > Please observe the normal rules:
>>>>> >> > > > >
>>>>> >> > > > > * No new features may be committed to the branch.
>>>>> >> > > > >
>>>>> >> > > > > * Documentation patches, build patches and serious bug fixes may be
>>>>> >> > > > > committed to the branch. However, you should submit all patches you
>>>>> >> > > > > want to commit to Jira first to give others the chance to review
>>>>> >> > > > > and possibly vote against the patch. Keep in mind that it is our
>>>>> >> > > > > main intention to keep the branch as stable as possible.
>>>>> >> > > > >
>>>>> >> > > > > * All patches that are intended for the branch should first be committed
>>>>> >> > > > > to the unstable branch, merged into the stable branch, and then into
>>>>> >> > > > > the current release branch.
>>>>> >> > > > >
>>>>> >> > > > > * Normal unstable and stable branch development may continue as usual.
>>>>> >> > > > > However, if you plan to commit a big change to the unstable branch
>>>>> >> > > > > while the branch feature freeze is in effect, think twice: can't the
>>>>> >> > > > > addition wait a couple more days? Merges of bug fixes into the branch
>>>>> >> > > > > may become more difficult.
>>>>> >> > > > >
>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
>>>>> >> > > > > a release candidate build.
>>>>> >> > > > > ----
>>>>> >> > > >
>>>>> >> > > > ---------------------------------------------------------------------
>>>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
>>>>> >> > > >
>>>>> >> > >
>>>>> >> > > ---------------------------------------------------------------------
>>>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
>>>>> >> > >
>>>>> >>
>>>>> >> ---------------------------------------------------------------------
>>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>>> >> For additional commands, e-mail: dev-help@solr.apache.org
>>>>> >>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
This is valid manifest line-breaking though... Can we read the manifest
properly on the smoke tester side somehow (for example, run a Java process
that reads and extracts the required attribute)? This way we wouldn't care
about the implementation details of how manifest wraps the lines (or
escapes characters).

D.

On Wed, Sep 15, 2021 at 8:46 PM Mike Drob <mdrob@mdrob.com> wrote:

> The benchmark jar has the info we need… sort of. When I built it, it has:
>
> Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
> 1 - mdrob - 2021-09-15 11:40:36
>
>
> and it’s looking for Implementation-Version: 8.10.0
> 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
>
> Because 8.10 is a character longer than 8.9, we happen to wrap the last
> character of the git commit sha. From the manifest spec:
>
> No line may be longer than 72 bytes (not characters), in its UTF8-encoded form.
> If a value would make the initial line longer than this, it should be continued
> on extra lines (each starting with a single SPACE).
>
> And we were already teetering on the edge of that limit. We'll run into
> this problem again in a few years when we try to release version 10.0.0, so
> solving it now has practical benefits down the line.
>
> There's a few options that I can come up with -
> 1. Use the short-hash when we generate the jar
> 2. Use the short-hash when we check the contents in the smoke test
> 3. Do some line join magic in the smoke test.
>
> I'm leaning towards number 1 as I feel that would still be unique enough
> for our needs, but would like to hear from others as well.
>
> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <thelabdude@gmail.com>
> wrote:
>
>> can someone also please look into that benchmark jar issue?
>>
>> Sent from my iPhone
>>
>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co.invalid>
>> wrote:
>>
>> ?
>> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
>>
>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com> wrote:
>>
>>> I think since Tim is out on vacation, it's probably not too late. That
>>> looks like a good fix to have, do we know how long the bug has been present?
>>>
>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova
>>> <mayya.sharipova@elastic.co.invalid> wrote:
>>>
>>>> Hello everyone,
>>>> We have discovered a bug and fixed a bug in Lucene sort optimization
>>>> <https://github.com/apache/lucene/pull/300> (LUCENE-10106) and would
>>>> like to merge it to Lucene 8.10 if it is not too late.
>>>> I apologize for the inconvenience, the bug was discovered just
>>>> yesterday.
>>>>
>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org>
>>>> wrote:
>>>>
>>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
>>>>> unless someone else wants to pick up the RM duties before then?
>>>>>
>>>>> After failing the test suite at various places and other weirdness
>>>>> like .asc files not getting created, I finally got to the smoke test
>>>>> part, which is now failing with:
>>>>>
>>>>> File
>>>>> "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
>>>>> line 176, in checkJARMetaData
>>>>> raise RuntimeError('%s is missing "%s" inside its
>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
>>>>> RuntimeError: JAR file
>>>>>
>>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
>>>>> is missing "Implementation-Version: 8.10.0
>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
>>>>> META-INF/MANIFEST.MF (wrong git revision?)
>>>>>
>>>>> FWIW, I verified that the other Lucene JAR files have this line in
>>>>> them, such as core:
>>>>>
>>>>> Manifest-Version: 1.0
>>>>> Ant-Version: Apache Ant 1.9.15
>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
>>>>> Extension-Name: org.apache.lucene
>>>>> Specification-Title: Lucene Search Engine: core
>>>>> Specification-Version: 8.10.0
>>>>> Specification-Vendor: The Apache Software Foundation
>>>>> Implementation-Title: org.apache.lucene
>>>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
>>>>> 7 - tjp - 2021-09-14 19:08:42
>>>>> Implementation-Vendor: The Apache Software Foundation
>>>>> X-Compile-Source-JDK: 8
>>>>> X-Compile-Target-JDK: 8
>>>>> Multi-Release: true
>>>>>
>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
>>>>> <ichattopadhyaya@gmail.com> wrote:
>>>>> >
>>>>> > All the best, this is the worst step.
>>>>> >
>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com>
>>>>> wrote:
>>>>> >>
>>>>> >> Building RC1 now ... stay tuned.
>>>>> >>
>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com>
>>>>> wrote:
>>>>> >> >
>>>>> >> > Thanks for the update Mike!
>>>>> >> >
>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR
>>>>> for
>>>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
>>>>> >> > reported on Slack earlier today.
>>>>> >> >
>>>>> >> > Cheers,
>>>>> >> > Tim
>>>>> >> >
>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org>
>>>>> wrote:
>>>>> >> > >
>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and
>>>>> benchmarking
>>>>> >> > > both look pretty good, but I've got a few last unit tests that
>>>>> I need
>>>>> >> > > to chase down. Hopefully taken care of by today or tomorrow,
>>>>> I'll be
>>>>> >> > > sure to keep you updated though.
>>>>> >> > >
>>>>> >> > >
>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <
>>>>> thelabdude@gmail.com> wrote:
>>>>> >> > > >
>>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620
>>>>> while testing
>>>>> >> > > > the schema designer. I haven't built the RC yet, so going to
>>>>> see if I
>>>>> >> > > > can get this in today.
>>>>> >> > > >
>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <
>>>>> thelabdude@apache.org> wrote:
>>>>> >> > > > >
>>>>> >> > > > > NOTICE:
>>>>> >> > > > >
>>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to
>>>>> 8.11 on stable branch.
>>>>> >> > > > >
>>>>> >> > > > > Please observe the normal rules:
>>>>> >> > > > >
>>>>> >> > > > > * No new features may be committed to the branch.
>>>>> >> > > > >
>>>>> >> > > > > * Documentation patches, build patches and serious bug
>>>>> fixes may be
>>>>> >> > > > > committed to the branch. However, you should submit all
>>>>> patches you
>>>>> >> > > > > want to commit to Jira first to give others the chance to
>>>>> review
>>>>> >> > > > > and possibly vote against the patch. Keep in mind that it
>>>>> is our
>>>>> >> > > > > main intention to keep the branch as stable as possible.
>>>>> >> > > > >
>>>>> >> > > > > * All patches that are intended for the branch should first
>>>>> be committed
>>>>> >> > > > > to the unstable branch, merged into the stable branch,
>>>>> and then into
>>>>> >> > > > > the current release branch.
>>>>> >> > > > >
>>>>> >> > > > > * Normal unstable and stable branch development may
>>>>> continue as usual.
>>>>> >> > > > > However, if you plan to commit a big change to the
>>>>> unstable branch
>>>>> >> > > > > while the branch feature freeze is in effect, think
>>>>> twice: can't the
>>>>> >> > > > > addition wait a couple more days? Merges of bug fixes
>>>>> into the branch
>>>>> >> > > > > may become more difficult.
>>>>> >> > > > >
>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority
>>>>> "Blocker" will delay
>>>>> >> > > > > a release candidate build.
>>>>> >> > > > > ----
>>>>> >> > > >
>>>>> >> > > >
>>>>> ---------------------------------------------------------------------
>>>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
>>>>> >> > > >
>>>>> >> > >
>>>>> >> > >
>>>>> ---------------------------------------------------------------------
>>>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
>>>>> >> > >
>>>>> >>
>>>>> >>
>>>>> ---------------------------------------------------------------------
>>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>>> >> For additional commands, e-mail: dev-help@solr.apache.org
>>>>> >>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>
>>>>>
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
>Can we read the manifest properly on the smoke tester side somehow

This would be my option #3?


These are apparently going to need different solutions for ant and gradle,
so we'll handle them separately.

Created https://issues.apache.org/jira/browse/LUCENE-10107 for 8.x and
https://issues.apache.org/jira/browse/LUCENE-10108 for 9.x+

On Wed, Sep 15, 2021 at 2:25 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:

>
> This is valid manifest line-breaking though... Can we read the manifest
> properly on the smoke tester side somehow (for example, run a Java process
> that reads and extracts the required attribute)? This way we wouldn't care
> about the implementation details of how manifest wraps the lines (or
> escapes characters).
>
> D.
>
> On Wed, Sep 15, 2021 at 8:46 PM Mike Drob <mdrob@mdrob.com> wrote:
>
>> The benchmark jar has the info we need… sort of. When I built it, it has:
>>
>> Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
>> 1 - mdrob - 2021-09-15 11:40:36
>>
>>
>> and it’s looking for Implementation-Version: 8.10.0
>> 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
>>
>> Because 8.10 is a character longer than 8.9, we happen to wrap the last
>> character of the git commit sha. From the manifest spec:
>>
>> No line may be longer than 72 bytes (not characters), in its UTF8-encoded form.
>> If a value would make the initial line longer than this, it should be continued
>> on extra lines (each starting with a single SPACE).
>>
>> And we were already teetering on the edge of that limit. We'll run into
>> this problem again in a few years when we try to release version 10.0.0, so
>> solving it now has practical benefits down the line.
>>
>> There's a few options that I can come up with -
>> 1. Use the short-hash when we generate the jar
>> 2. Use the short-hash when we check the contents in the smoke test
>> 3. Do some line join magic in the smoke test.
>>
>> I'm leaning towards number 1 as I feel that would still be unique enough
>> for our needs, but would like to hear from others as well.
>>
>> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <thelabdude@gmail.com>
>> wrote:
>>
>>> can someone also please look into that benchmark jar issue?
>>>
>>> Sent from my iPhone
>>>
>>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co.invalid>
>>> wrote:
>>>
>>> ?
>>> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
>>>
>>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com> wrote:
>>>
>>>> I think since Tim is out on vacation, it's probably not too late. That
>>>> looks like a good fix to have, do we know how long the bug has been present?
>>>>
>>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova
>>>> <mayya.sharipova@elastic.co.invalid> wrote:
>>>>
>>>>> Hello everyone,
>>>>> We have discovered a bug and fixed a bug in Lucene sort optimization
>>>>> <https://github.com/apache/lucene/pull/300> (LUCENE-10106) and would
>>>>> like to merge it to Lucene 8.10 if it is not too late.
>>>>> I apologize for the inconvenience, the bug was discovered just
>>>>> yesterday.
>>>>>
>>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
>>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
>>>>>> unless someone else wants to pick up the RM duties before then?
>>>>>>
>>>>>> After failing the test suite at various places and other weirdness
>>>>>> like .asc files not getting created, I finally got to the smoke test
>>>>>> part, which is now failing with:
>>>>>>
>>>>>> File
>>>>>> "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
>>>>>> line 176, in checkJARMetaData
>>>>>> raise RuntimeError('%s is missing "%s" inside its
>>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
>>>>>> RuntimeError: JAR file
>>>>>>
>>>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
>>>>>> is missing "Implementation-Version: 8.10.0
>>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
>>>>>> META-INF/MANIFEST.MF (wrong git revision?)
>>>>>>
>>>>>> FWIW, I verified that the other Lucene JAR files have this line in
>>>>>> them, such as core:
>>>>>>
>>>>>> Manifest-Version: 1.0
>>>>>> Ant-Version: Apache Ant 1.9.15
>>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
>>>>>> Extension-Name: org.apache.lucene
>>>>>> Specification-Title: Lucene Search Engine: core
>>>>>> Specification-Version: 8.10.0
>>>>>> Specification-Vendor: The Apache Software Foundation
>>>>>> Implementation-Title: org.apache.lucene
>>>>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
>>>>>> 7 - tjp - 2021-09-14 19:08:42
>>>>>> Implementation-Vendor: The Apache Software Foundation
>>>>>> X-Compile-Source-JDK: 8
>>>>>> X-Compile-Target-JDK: 8
>>>>>> Multi-Release: true
>>>>>>
>>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
>>>>>> <ichattopadhyaya@gmail.com> wrote:
>>>>>> >
>>>>>> > All the best, this is the worst step.
>>>>>> >
>>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <
>>>>>> thelabdude@gmail.com> wrote:
>>>>>> >>
>>>>>> >> Building RC1 now ... stay tuned.
>>>>>> >>
>>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <
>>>>>> thelabdude@gmail.com> wrote:
>>>>>> >> >
>>>>>> >> > Thanks for the update Mike!
>>>>>> >> >
>>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick
>>>>>> PR for
>>>>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
>>>>>> >> > reported on Slack earlier today.
>>>>>> >> >
>>>>>> >> > Cheers,
>>>>>> >> > Tim
>>>>>> >> >
>>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org>
>>>>>> wrote:
>>>>>> >> > >
>>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and
>>>>>> benchmarking
>>>>>> >> > > both look pretty good, but I've got a few last unit tests that
>>>>>> I need
>>>>>> >> > > to chase down. Hopefully taken care of by today or tomorrow,
>>>>>> I'll be
>>>>>> >> > > sure to keep you updated though.
>>>>>> >> > >
>>>>>> >> > >
>>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <
>>>>>> thelabdude@gmail.com> wrote:
>>>>>> >> > > >
>>>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620
>>>>>> while testing
>>>>>> >> > > > the schema designer. I haven't built the RC yet, so going to
>>>>>> see if I
>>>>>> >> > > > can get this in today.
>>>>>> >> > > >
>>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <
>>>>>> thelabdude@apache.org> wrote:
>>>>>> >> > > > >
>>>>>> >> > > > > NOTICE:
>>>>>> >> > > > >
>>>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to
>>>>>> 8.11 on stable branch.
>>>>>> >> > > > >
>>>>>> >> > > > > Please observe the normal rules:
>>>>>> >> > > > >
>>>>>> >> > > > > * No new features may be committed to the branch.
>>>>>> >> > > > >
>>>>>> >> > > > > * Documentation patches, build patches and serious bug
>>>>>> fixes may be
>>>>>> >> > > > > committed to the branch. However, you should submit all
>>>>>> patches you
>>>>>> >> > > > > want to commit to Jira first to give others the chance
>>>>>> to review
>>>>>> >> > > > > and possibly vote against the patch. Keep in mind that
>>>>>> it is our
>>>>>> >> > > > > main intention to keep the branch as stable as possible.
>>>>>> >> > > > >
>>>>>> >> > > > > * All patches that are intended for the branch should
>>>>>> first be committed
>>>>>> >> > > > > to the unstable branch, merged into the stable branch,
>>>>>> and then into
>>>>>> >> > > > > the current release branch.
>>>>>> >> > > > >
>>>>>> >> > > > > * Normal unstable and stable branch development may
>>>>>> continue as usual.
>>>>>> >> > > > > However, if you plan to commit a big change to the
>>>>>> unstable branch
>>>>>> >> > > > > while the branch feature freeze is in effect, think
>>>>>> twice: can't the
>>>>>> >> > > > > addition wait a couple more days? Merges of bug fixes
>>>>>> into the branch
>>>>>> >> > > > > may become more difficult.
>>>>>> >> > > > >
>>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority
>>>>>> "Blocker" will delay
>>>>>> >> > > > > a release candidate build.
>>>>>> >> > > > > ----
>>>>>> >> > > >
>>>>>> >> > > >
>>>>>> ---------------------------------------------------------------------
>>>>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>> >> > > >
>>>>>> >> > >
>>>>>> >> > >
>>>>>> ---------------------------------------------------------------------
>>>>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>> >> > >
>>>>>> >>
>>>>>> >>
>>>>>> ---------------------------------------------------------------------
>>>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>>>> >> For additional commands, e-mail: dev-help@solr.apache.org
>>>>>> >>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>>
>>>>>>
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
But its irrelevant that is "valid" when virtually no tools match it.

In other words, I'd agree with you if the "jar" command had some
ability to read these manifests and print stuff to stdout, e.g. if
there was ANY interop at all here.

But there isn't. So IMO it makes no sense to cause confusion and chaos
by adding an unnecessarily long git commit hash.

On Wed, Sep 15, 2021 at 3:26 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>
>
> This is valid manifest line-breaking though... Can we read the manifest properly on the smoke tester side somehow (for example, run a Java process that reads and extracts the required attribute)? This way we wouldn't care about the implementation details of how manifest wraps the lines (or escapes characters).
>
> D.
>
> On Wed, Sep 15, 2021 at 8:46 PM Mike Drob <mdrob@mdrob.com> wrote:
>>
>> The benchmark jar has the info we need… sort of. When I built it, it has:
>>
>> Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
>> 1 - mdrob - 2021-09-15 11:40:36
>>
>>
>> and it’s looking for Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
>>
>> Because 8.10 is a character longer than 8.9, we happen to wrap the last character of the git commit sha. From the manifest spec:
>>
>> No line may be longer than 72 bytes (not characters), in its UTF8-encoded form.
>> If a value would make the initial line longer than this, it should be continued
>> on extra lines (each starting with a single SPACE).
>>
>> And we were already teetering on the edge of that limit. We'll run into this problem again in a few years when we try to release version 10.0.0, so solving it now has practical benefits down the line.
>>
>> There's a few options that I can come up with -
>> 1. Use the short-hash when we generate the jar
>> 2. Use the short-hash when we check the contents in the smoke test
>> 3. Do some line join magic in the smoke test.
>>
>> I'm leaning towards number 1 as I feel that would still be unique enough for our needs, but would like to hear from others as well.
>>
>> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <thelabdude@gmail.com> wrote:
>>>
>>> can someone also please look into that benchmark jar issue?
>>>
>>> Sent from my iPhone
>>>
>>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co.invalid> wrote:
>>>
>>> ?
>>> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
>>>
>>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com> wrote:
>>>>
>>>> I think since Tim is out on vacation, it's probably not too late. That looks like a good fix to have, do we know how long the bug has been present?
>>>>
>>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova <mayya.sharipova@elastic.co.invalid> wrote:
>>>>>
>>>>> Hello everyone,
>>>>> We have discovered a bug and fixed a bug in Lucene sort optimization (LUCENE-10106) and would like to merge it to Lucene 8.10 if it is not too late.
>>>>> I apologize for the inconvenience, the bug was discovered just yesterday.
>>>>>
>>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org> wrote:
>>>>>>
>>>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
>>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
>>>>>> unless someone else wants to pick up the RM duties before then?
>>>>>>
>>>>>> After failing the test suite at various places and other weirdness
>>>>>> like .asc files not getting created, I finally got to the smoke test
>>>>>> part, which is now failing with:
>>>>>>
>>>>>> File "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
>>>>>> line 176, in checkJARMetaData
>>>>>> raise RuntimeError('%s is missing "%s" inside its
>>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
>>>>>> RuntimeError: JAR file
>>>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
>>>>>> is missing "Implementation-Version: 8.10.0
>>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
>>>>>> META-INF/MANIFEST.MF (wrong git revision?)
>>>>>>
>>>>>> FWIW, I verified that the other Lucene JAR files have this line in
>>>>>> them, such as core:
>>>>>>
>>>>>> Manifest-Version: 1.0
>>>>>> Ant-Version: Apache Ant 1.9.15
>>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
>>>>>> Extension-Name: org.apache.lucene
>>>>>> Specification-Title: Lucene Search Engine: core
>>>>>> Specification-Version: 8.10.0
>>>>>> Specification-Vendor: The Apache Software Foundation
>>>>>> Implementation-Title: org.apache.lucene
>>>>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
>>>>>> 7 - tjp - 2021-09-14 19:08:42
>>>>>> Implementation-Vendor: The Apache Software Foundation
>>>>>> X-Compile-Source-JDK: 8
>>>>>> X-Compile-Target-JDK: 8
>>>>>> Multi-Release: true
>>>>>>
>>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
>>>>>> <ichattopadhyaya@gmail.com> wrote:
>>>>>> >
>>>>>> > All the best, this is the worst step.
>>>>>> >
>>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com> wrote:
>>>>>> >>
>>>>>> >> Building RC1 now ... stay tuned.
>>>>>> >>
>>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com> wrote:
>>>>>> >> >
>>>>>> >> > Thanks for the update Mike!
>>>>>> >> >
>>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
>>>>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
>>>>>> >> > reported on Slack earlier today.
>>>>>> >> >
>>>>>> >> > Cheers,
>>>>>> >> > Tim
>>>>>> >> >
>>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
>>>>>> >> > >
>>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
>>>>>> >> > > both look pretty good, but I've got a few last unit tests that I need
>>>>>> >> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
>>>>>> >> > > sure to keep you updated though.
>>>>>> >> > >
>>>>>> >> > >
>>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com> wrote:
>>>>>> >> > > >
>>>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
>>>>>> >> > > > the schema designer. I haven't built the RC yet, so going to see if I
>>>>>> >> > > > can get this in today.
>>>>>> >> > > >
>>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
>>>>>> >> > > > >
>>>>>> >> > > > > NOTICE:
>>>>>> >> > > > >
>>>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
>>>>>> >> > > > >
>>>>>> >> > > > > Please observe the normal rules:
>>>>>> >> > > > >
>>>>>> >> > > > > * No new features may be committed to the branch.
>>>>>> >> > > > >
>>>>>> >> > > > > * Documentation patches, build patches and serious bug fixes may be
>>>>>> >> > > > > committed to the branch. However, you should submit all patches you
>>>>>> >> > > > > want to commit to Jira first to give others the chance to review
>>>>>> >> > > > > and possibly vote against the patch. Keep in mind that it is our
>>>>>> >> > > > > main intention to keep the branch as stable as possible.
>>>>>> >> > > > >
>>>>>> >> > > > > * All patches that are intended for the branch should first be committed
>>>>>> >> > > > > to the unstable branch, merged into the stable branch, and then into
>>>>>> >> > > > > the current release branch.
>>>>>> >> > > > >
>>>>>> >> > > > > * Normal unstable and stable branch development may continue as usual.
>>>>>> >> > > > > However, if you plan to commit a big change to the unstable branch
>>>>>> >> > > > > while the branch feature freeze is in effect, think twice: can't the
>>>>>> >> > > > > addition wait a couple more days? Merges of bug fixes into the branch
>>>>>> >> > > > > may become more difficult.
>>>>>> >> > > > >
>>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
>>>>>> >> > > > > a release candidate build.
>>>>>> >> > > > > ----
>>>>>> >> > > >
>>>>>> >> > > > ---------------------------------------------------------------------
>>>>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>> >> > > >
>>>>>> >> > >
>>>>>> >> > > ---------------------------------------------------------------------
>>>>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>> >> > >
>>>>>> >>
>>>>>> >> ---------------------------------------------------------------------
>>>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>>>> >> For additional commands, e-mail: dev-help@solr.apache.org
>>>>>> >>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
Jar command doesn't have it, true. But it's fairly trivial to do, even
with an inline snippet like this one?

public class PrintManifest {
public static void main(String[] jars) throws IOException {
for (var jar : jars) {
var manifest = new JarFile(Paths.get(jar).toFile()).getManifest();
var attrs = manifest.getMainAttributes();
System.out.println(jar + ": " + attrs.getValue("Implementation-Version"));
}
}
}

I have this in my lucene-core-9.0.0-SNAPSHOT.jar:

Implementation-Version: 9.0.0-SNAPSHOT de45b68c909815ce5ea7b6b9e1a2ce337
5b6334d [snapshot build, details omitted]

and running:

java PrintManifest.java lucene-core-9.0.0-SNAPSHOT.jar

shows:

lucene-core-9.0.0-SNAPSHOT.jar: 9.0.0-SNAPSHOT
de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details
omitted]

This seems easier to me than trying to remember and keep the length of
that line shorter than an arbitrary limit.

Dawid


On Wed, Sep 15, 2021 at 9:46 PM Robert Muir <rcmuir@gmail.com> wrote:
>
> But its irrelevant that is "valid" when virtually no tools match it.
>
> In other words, I'd agree with you if the "jar" command had some
> ability to read these manifests and print stuff to stdout, e.g. if
> there was ANY interop at all here.
>
> But there isn't. So IMO it makes no sense to cause confusion and chaos
> by adding an unnecessarily long git commit hash.
>
> On Wed, Sep 15, 2021 at 3:26 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> >
> >
> > This is valid manifest line-breaking though... Can we read the manifest properly on the smoke tester side somehow (for example, run a Java process that reads and extracts the required attribute)? This way we wouldn't care about the implementation details of how manifest wraps the lines (or escapes characters).
> >
> > D.
> >
> > On Wed, Sep 15, 2021 at 8:46 PM Mike Drob <mdrob@mdrob.com> wrote:
> >>
> >> The benchmark jar has the info we need… sort of. When I built it, it has:
> >>
> >> Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
> >> 1 - mdrob - 2021-09-15 11:40:36
> >>
> >>
> >> and it’s looking for Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
> >>
> >> Because 8.10 is a character longer than 8.9, we happen to wrap the last character of the git commit sha. From the manifest spec:
> >>
> >> No line may be longer than 72 bytes (not characters), in its UTF8-encoded form.
> >> If a value would make the initial line longer than this, it should be continued
> >> on extra lines (each starting with a single SPACE).
> >>
> >> And we were already teetering on the edge of that limit. We'll run into this problem again in a few years when we try to release version 10.0.0, so solving it now has practical benefits down the line.
> >>
> >> There's a few options that I can come up with -
> >> 1. Use the short-hash when we generate the jar
> >> 2. Use the short-hash when we check the contents in the smoke test
> >> 3. Do some line join magic in the smoke test.
> >>
> >> I'm leaning towards number 1 as I feel that would still be unique enough for our needs, but would like to hear from others as well.
> >>
> >> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <thelabdude@gmail.com> wrote:
> >>>
> >>> can someone also please look into that benchmark jar issue?
> >>>
> >>> Sent from my iPhone
> >>>
> >>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co.invalid> wrote:
> >>>
> >>> ?
> >>> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
> >>>
> >>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com> wrote:
> >>>>
> >>>> I think since Tim is out on vacation, it's probably not too late. That looks like a good fix to have, do we know how long the bug has been present?
> >>>>
> >>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova <mayya.sharipova@elastic.co.invalid> wrote:
> >>>>>
> >>>>> Hello everyone,
> >>>>> We have discovered a bug and fixed a bug in Lucene sort optimization (LUCENE-10106) and would like to merge it to Lucene 8.10 if it is not too late.
> >>>>> I apologize for the inconvenience, the bug was discovered just yesterday.
> >>>>>
> >>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org> wrote:
> >>>>>>
> >>>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
> >>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
> >>>>>> unless someone else wants to pick up the RM duties before then?
> >>>>>>
> >>>>>> After failing the test suite at various places and other weirdness
> >>>>>> like .asc files not getting created, I finally got to the smoke test
> >>>>>> part, which is now failing with:
> >>>>>>
> >>>>>> File "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
> >>>>>> line 176, in checkJARMetaData
> >>>>>> raise RuntimeError('%s is missing "%s" inside its
> >>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
> >>>>>> RuntimeError: JAR file
> >>>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
> >>>>>> is missing "Implementation-Version: 8.10.0
> >>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
> >>>>>> META-INF/MANIFEST.MF (wrong git revision?)
> >>>>>>
> >>>>>> FWIW, I verified that the other Lucene JAR files have this line in
> >>>>>> them, such as core:
> >>>>>>
> >>>>>> Manifest-Version: 1.0
> >>>>>> Ant-Version: Apache Ant 1.9.15
> >>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
> >>>>>> Extension-Name: org.apache.lucene
> >>>>>> Specification-Title: Lucene Search Engine: core
> >>>>>> Specification-Version: 8.10.0
> >>>>>> Specification-Vendor: The Apache Software Foundation
> >>>>>> Implementation-Title: org.apache.lucene
> >>>>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
> >>>>>> 7 - tjp - 2021-09-14 19:08:42
> >>>>>> Implementation-Vendor: The Apache Software Foundation
> >>>>>> X-Compile-Source-JDK: 8
> >>>>>> X-Compile-Target-JDK: 8
> >>>>>> Multi-Release: true
> >>>>>>
> >>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
> >>>>>> <ichattopadhyaya@gmail.com> wrote:
> >>>>>> >
> >>>>>> > All the best, this is the worst step.
> >>>>>> >
> >>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com> wrote:
> >>>>>> >>
> >>>>>> >> Building RC1 now ... stay tuned.
> >>>>>> >>
> >>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com> wrote:
> >>>>>> >> >
> >>>>>> >> > Thanks for the update Mike!
> >>>>>> >> >
> >>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
> >>>>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
> >>>>>> >> > reported on Slack earlier today.
> >>>>>> >> >
> >>>>>> >> > Cheers,
> >>>>>> >> > Tim
> >>>>>> >> >
> >>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
> >>>>>> >> > >
> >>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
> >>>>>> >> > > both look pretty good, but I've got a few last unit tests that I need
> >>>>>> >> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
> >>>>>> >> > > sure to keep you updated though.
> >>>>>> >> > >
> >>>>>> >> > >
> >>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com> wrote:
> >>>>>> >> > > >
> >>>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
> >>>>>> >> > > > the schema designer. I haven't built the RC yet, so going to see if I
> >>>>>> >> > > > can get this in today.
> >>>>>> >> > > >
> >>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
> >>>>>> >> > > > >
> >>>>>> >> > > > > NOTICE:
> >>>>>> >> > > > >
> >>>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
> >>>>>> >> > > > >
> >>>>>> >> > > > > Please observe the normal rules:
> >>>>>> >> > > > >
> >>>>>> >> > > > > * No new features may be committed to the branch.
> >>>>>> >> > > > >
> >>>>>> >> > > > > * Documentation patches, build patches and serious bug fixes may be
> >>>>>> >> > > > > committed to the branch. However, you should submit all patches you
> >>>>>> >> > > > > want to commit to Jira first to give others the chance to review
> >>>>>> >> > > > > and possibly vote against the patch. Keep in mind that it is our
> >>>>>> >> > > > > main intention to keep the branch as stable as possible.
> >>>>>> >> > > > >
> >>>>>> >> > > > > * All patches that are intended for the branch should first be committed
> >>>>>> >> > > > > to the unstable branch, merged into the stable branch, and then into
> >>>>>> >> > > > > the current release branch.
> >>>>>> >> > > > >
> >>>>>> >> > > > > * Normal unstable and stable branch development may continue as usual.
> >>>>>> >> > > > > However, if you plan to commit a big change to the unstable branch
> >>>>>> >> > > > > while the branch feature freeze is in effect, think twice: can't the
> >>>>>> >> > > > > addition wait a couple more days? Merges of bug fixes into the branch
> >>>>>> >> > > > > may become more difficult.
> >>>>>> >> > > > >
> >>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
> >>>>>> >> > > > > a release candidate build.
> >>>>>> >> > > > > ----
> >>>>>> >> > > >
> >>>>>> >> > > > ---------------------------------------------------------------------
> >>>>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>>>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
> >>>>>> >> > > >
> >>>>>> >> > >
> >>>>>> >> > > ---------------------------------------------------------------------
> >>>>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>>>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
> >>>>>> >> > >
> >>>>>> >>
> >>>>>> >> ---------------------------------------------------------------------
> >>>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> >>>>>> >> For additional commands, e-mail: dev-help@solr.apache.org
> >>>>>> >>
> >>>>>>
> >>>>>> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
> >>>>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
I was referring to doing this with languages other than java.

I'm also assuming that exceeding this limit is going to cause indirect
hassles for users of lucene, e.g. breaking various security / supply
chain tools. We know a lot of these are total crap but people in the
corporate world have to suffer under them.

Its super-easy to use a short hash here and avoid problems.

On Thu, Sep 16, 2021 at 3:03 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>
> Jar command doesn't have it, true. But it's fairly trivial to do, even
> with an inline snippet like this one?
>
> public class PrintManifest {
> public static void main(String[] jars) throws IOException {
> for (var jar : jars) {
> var manifest = new JarFile(Paths.get(jar).toFile()).getManifest();
> var attrs = manifest.getMainAttributes();
> System.out.println(jar + ": " + attrs.getValue("Implementation-Version"));
> }
> }
> }
>
> I have this in my lucene-core-9.0.0-SNAPSHOT.jar:
>
> Implementation-Version: 9.0.0-SNAPSHOT de45b68c909815ce5ea7b6b9e1a2ce337
> 5b6334d [snapshot build, details omitted]
>
> and running:
>
> java PrintManifest.java lucene-core-9.0.0-SNAPSHOT.jar
>
> shows:
>
> lucene-core-9.0.0-SNAPSHOT.jar: 9.0.0-SNAPSHOT
> de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details
> omitted]
>
> This seems easier to me than trying to remember and keep the length of
> that line shorter than an arbitrary limit.
>
> Dawid
>
>
> On Wed, Sep 15, 2021 at 9:46 PM Robert Muir <rcmuir@gmail.com> wrote:
> >
> > But its irrelevant that is "valid" when virtually no tools match it.
> >
> > In other words, I'd agree with you if the "jar" command had some
> > ability to read these manifests and print stuff to stdout, e.g. if
> > there was ANY interop at all here.
> >
> > But there isn't. So IMO it makes no sense to cause confusion and chaos
> > by adding an unnecessarily long git commit hash.
> >
> > On Wed, Sep 15, 2021 at 3:26 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> > >
> > >
> > > This is valid manifest line-breaking though... Can we read the manifest properly on the smoke tester side somehow (for example, run a Java process that reads and extracts the required attribute)? This way we wouldn't care about the implementation details of how manifest wraps the lines (or escapes characters).
> > >
> > > D.
> > >
> > > On Wed, Sep 15, 2021 at 8:46 PM Mike Drob <mdrob@mdrob.com> wrote:
> > >>
> > >> The benchmark jar has the info we need… sort of. When I built it, it has:
> > >>
> > >> Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
> > >> 1 - mdrob - 2021-09-15 11:40:36
> > >>
> > >>
> > >> and it’s looking for Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
> > >>
> > >> Because 8.10 is a character longer than 8.9, we happen to wrap the last character of the git commit sha. From the manifest spec:
> > >>
> > >> No line may be longer than 72 bytes (not characters), in its UTF8-encoded form.
> > >> If a value would make the initial line longer than this, it should be continued
> > >> on extra lines (each starting with a single SPACE).
> > >>
> > >> And we were already teetering on the edge of that limit. We'll run into this problem again in a few years when we try to release version 10.0.0, so solving it now has practical benefits down the line.
> > >>
> > >> There's a few options that I can come up with -
> > >> 1. Use the short-hash when we generate the jar
> > >> 2. Use the short-hash when we check the contents in the smoke test
> > >> 3. Do some line join magic in the smoke test.
> > >>
> > >> I'm leaning towards number 1 as I feel that would still be unique enough for our needs, but would like to hear from others as well.
> > >>
> > >> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <thelabdude@gmail.com> wrote:
> > >>>
> > >>> can someone also please look into that benchmark jar issue?
> > >>>
> > >>> Sent from my iPhone
> > >>>
> > >>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co.invalid> wrote:
> > >>>
> > >>> ?
> > >>> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
> > >>>
> > >>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com> wrote:
> > >>>>
> > >>>> I think since Tim is out on vacation, it's probably not too late. That looks like a good fix to have, do we know how long the bug has been present?
> > >>>>
> > >>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova <mayya.sharipova@elastic.co.invalid> wrote:
> > >>>>>
> > >>>>> Hello everyone,
> > >>>>> We have discovered a bug and fixed a bug in Lucene sort optimization (LUCENE-10106) and would like to merge it to Lucene 8.10 if it is not too late.
> > >>>>> I apologize for the inconvenience, the bug was discovered just yesterday.
> > >>>>>
> > >>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org> wrote:
> > >>>>>>
> > >>>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
> > >>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
> > >>>>>> unless someone else wants to pick up the RM duties before then?
> > >>>>>>
> > >>>>>> After failing the test suite at various places and other weirdness
> > >>>>>> like .asc files not getting created, I finally got to the smoke test
> > >>>>>> part, which is now failing with:
> > >>>>>>
> > >>>>>> File "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
> > >>>>>> line 176, in checkJARMetaData
> > >>>>>> raise RuntimeError('%s is missing "%s" inside its
> > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
> > >>>>>> RuntimeError: JAR file
> > >>>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
> > >>>>>> is missing "Implementation-Version: 8.10.0
> > >>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
> > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)
> > >>>>>>
> > >>>>>> FWIW, I verified that the other Lucene JAR files have this line in
> > >>>>>> them, such as core:
> > >>>>>>
> > >>>>>> Manifest-Version: 1.0
> > >>>>>> Ant-Version: Apache Ant 1.9.15
> > >>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
> > >>>>>> Extension-Name: org.apache.lucene
> > >>>>>> Specification-Title: Lucene Search Engine: core
> > >>>>>> Specification-Version: 8.10.0
> > >>>>>> Specification-Vendor: The Apache Software Foundation
> > >>>>>> Implementation-Title: org.apache.lucene
> > >>>>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
> > >>>>>> 7 - tjp - 2021-09-14 19:08:42
> > >>>>>> Implementation-Vendor: The Apache Software Foundation
> > >>>>>> X-Compile-Source-JDK: 8
> > >>>>>> X-Compile-Target-JDK: 8
> > >>>>>> Multi-Release: true
> > >>>>>>
> > >>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
> > >>>>>> <ichattopadhyaya@gmail.com> wrote:
> > >>>>>> >
> > >>>>>> > All the best, this is the worst step.
> > >>>>>> >
> > >>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com> wrote:
> > >>>>>> >>
> > >>>>>> >> Building RC1 now ... stay tuned.
> > >>>>>> >>
> > >>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com> wrote:
> > >>>>>> >> >
> > >>>>>> >> > Thanks for the update Mike!
> > >>>>>> >> >
> > >>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
> > >>>>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
> > >>>>>> >> > reported on Slack earlier today.
> > >>>>>> >> >
> > >>>>>> >> > Cheers,
> > >>>>>> >> > Tim
> > >>>>>> >> >
> > >>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
> > >>>>>> >> > >
> > >>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
> > >>>>>> >> > > both look pretty good, but I've got a few last unit tests that I need
> > >>>>>> >> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
> > >>>>>> >> > > sure to keep you updated though.
> > >>>>>> >> > >
> > >>>>>> >> > >
> > >>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com> wrote:
> > >>>>>> >> > > >
> > >>>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
> > >>>>>> >> > > > the schema designer. I haven't built the RC yet, so going to see if I
> > >>>>>> >> > > > can get this in today.
> > >>>>>> >> > > >
> > >>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
> > >>>>>> >> > > > >
> > >>>>>> >> > > > > NOTICE:
> > >>>>>> >> > > > >
> > >>>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
> > >>>>>> >> > > > >
> > >>>>>> >> > > > > Please observe the normal rules:
> > >>>>>> >> > > > >
> > >>>>>> >> > > > > * No new features may be committed to the branch.
> > >>>>>> >> > > > >
> > >>>>>> >> > > > > * Documentation patches, build patches and serious bug fixes may be
> > >>>>>> >> > > > > committed to the branch. However, you should submit all patches you
> > >>>>>> >> > > > > want to commit to Jira first to give others the chance to review
> > >>>>>> >> > > > > and possibly vote against the patch. Keep in mind that it is our
> > >>>>>> >> > > > > main intention to keep the branch as stable as possible.
> > >>>>>> >> > > > >
> > >>>>>> >> > > > > * All patches that are intended for the branch should first be committed
> > >>>>>> >> > > > > to the unstable branch, merged into the stable branch, and then into
> > >>>>>> >> > > > > the current release branch.
> > >>>>>> >> > > > >
> > >>>>>> >> > > > > * Normal unstable and stable branch development may continue as usual.
> > >>>>>> >> > > > > However, if you plan to commit a big change to the unstable branch
> > >>>>>> >> > > > > while the branch feature freeze is in effect, think twice: can't the
> > >>>>>> >> > > > > addition wait a couple more days? Merges of bug fixes into the branch
> > >>>>>> >> > > > > may become more difficult.
> > >>>>>> >> > > > >
> > >>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
> > >>>>>> >> > > > > a release candidate build.
> > >>>>>> >> > > > > ----
> > >>>>>> >> > > >
> > >>>>>> >> > > > ---------------------------------------------------------------------
> > >>>>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > >>>>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > >>>>>> >> > > >
> > >>>>>> >> > >
> > >>>>>> >> > > ---------------------------------------------------------------------
> > >>>>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > >>>>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
> > >>>>>> >> > >
> > >>>>>> >>
> > >>>>>> >> ---------------------------------------------------------------------
> > >>>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > >>>>>> >> For additional commands, e-mail: dev-help@solr.apache.org
> > >>>>>> >>
> > >>>>>>
> > >>>>>> ---------------------------------------------------------------------
> > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > >>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
> > >>>>>>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > For additional commands, e-mail: dev-help@solr.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
The shit that mike's python tool is doing, i'd expect bugs like that.
If you honestly disagree with me, go fix his python code to obey "the
rules". Be sure you have "the rules" correct as well (count bytes, not
characters, count EOL, etc).

Try to imagine how much software is going to get this wrong. Various
tools scanning for security problems that aren't written in java and
hence aren't using Jar class. Your security team isn't writing fucking
java code. Your ops team isn't writing fucking java code, so it really
does not matter if it is "easy in java".

This is *exactly* what i mean when i say this crazy shit has NO
INTEROP. It is 100% pure garbage, java showing its true colors.

On Thu, Sep 16, 2021 at 5:53 AM Robert Muir <rcmuir@gmail.com> wrote:
>
> I was referring to doing this with languages other than java.
>
> I'm also assuming that exceeding this limit is going to cause indirect
> hassles for users of lucene, e.g. breaking various security / supply
> chain tools. We know a lot of these are total crap but people in the
> corporate world have to suffer under them.
>
> Its super-easy to use a short hash here and avoid problems.
>
> On Thu, Sep 16, 2021 at 3:03 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> >
> > Jar command doesn't have it, true. But it's fairly trivial to do, even
> > with an inline snippet like this one?
> >
> > public class PrintManifest {
> > public static void main(String[] jars) throws IOException {
> > for (var jar : jars) {
> > var manifest = new JarFile(Paths.get(jar).toFile()).getManifest();
> > var attrs = manifest.getMainAttributes();
> > System.out.println(jar + ": " + attrs.getValue("Implementation-Version"));
> > }
> > }
> > }
> >
> > I have this in my lucene-core-9.0.0-SNAPSHOT.jar:
> >
> > Implementation-Version: 9.0.0-SNAPSHOT de45b68c909815ce5ea7b6b9e1a2ce337
> > 5b6334d [snapshot build, details omitted]
> >
> > and running:
> >
> > java PrintManifest.java lucene-core-9.0.0-SNAPSHOT.jar
> >
> > shows:
> >
> > lucene-core-9.0.0-SNAPSHOT.jar: 9.0.0-SNAPSHOT
> > de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details
> > omitted]
> >
> > This seems easier to me than trying to remember and keep the length of
> > that line shorter than an arbitrary limit.
> >
> > Dawid
> >
> >
> > On Wed, Sep 15, 2021 at 9:46 PM Robert Muir <rcmuir@gmail.com> wrote:
> > >
> > > But its irrelevant that is "valid" when virtually no tools match it.
> > >
> > > In other words, I'd agree with you if the "jar" command had some
> > > ability to read these manifests and print stuff to stdout, e.g. if
> > > there was ANY interop at all here.
> > >
> > > But there isn't. So IMO it makes no sense to cause confusion and chaos
> > > by adding an unnecessarily long git commit hash.
> > >
> > > On Wed, Sep 15, 2021 at 3:26 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> > > >
> > > >
> > > > This is valid manifest line-breaking though... Can we read the manifest properly on the smoke tester side somehow (for example, run a Java process that reads and extracts the required attribute)? This way we wouldn't care about the implementation details of how manifest wraps the lines (or escapes characters).
> > > >
> > > > D.
> > > >
> > > > On Wed, Sep 15, 2021 at 8:46 PM Mike Drob <mdrob@mdrob.com> wrote:
> > > >>
> > > >> The benchmark jar has the info we need… sort of. When I built it, it has:
> > > >>
> > > >> Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
> > > >> 1 - mdrob - 2021-09-15 11:40:36
> > > >>
> > > >>
> > > >> and it’s looking for Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
> > > >>
> > > >> Because 8.10 is a character longer than 8.9, we happen to wrap the last character of the git commit sha. From the manifest spec:
> > > >>
> > > >> No line may be longer than 72 bytes (not characters), in its UTF8-encoded form.
> > > >> If a value would make the initial line longer than this, it should be continued
> > > >> on extra lines (each starting with a single SPACE).
> > > >>
> > > >> And we were already teetering on the edge of that limit. We'll run into this problem again in a few years when we try to release version 10.0.0, so solving it now has practical benefits down the line.
> > > >>
> > > >> There's a few options that I can come up with -
> > > >> 1. Use the short-hash when we generate the jar
> > > >> 2. Use the short-hash when we check the contents in the smoke test
> > > >> 3. Do some line join magic in the smoke test.
> > > >>
> > > >> I'm leaning towards number 1 as I feel that would still be unique enough for our needs, but would like to hear from others as well.
> > > >>
> > > >> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <thelabdude@gmail.com> wrote:
> > > >>>
> > > >>> can someone also please look into that benchmark jar issue?
> > > >>>
> > > >>> Sent from my iPhone
> > > >>>
> > > >>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co.invalid> wrote:
> > > >>>
> > > >>> ?
> > > >>> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
> > > >>>
> > > >>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com> wrote:
> > > >>>>
> > > >>>> I think since Tim is out on vacation, it's probably not too late. That looks like a good fix to have, do we know how long the bug has been present?
> > > >>>>
> > > >>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova <mayya.sharipova@elastic.co.invalid> wrote:
> > > >>>>>
> > > >>>>> Hello everyone,
> > > >>>>> We have discovered a bug and fixed a bug in Lucene sort optimization (LUCENE-10106) and would like to merge it to Lucene 8.10 if it is not too late.
> > > >>>>> I apologize for the inconvenience, the bug was discovered just yesterday.
> > > >>>>>
> > > >>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org> wrote:
> > > >>>>>>
> > > >>>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
> > > >>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
> > > >>>>>> unless someone else wants to pick up the RM duties before then?
> > > >>>>>>
> > > >>>>>> After failing the test suite at various places and other weirdness
> > > >>>>>> like .asc files not getting created, I finally got to the smoke test
> > > >>>>>> part, which is now failing with:
> > > >>>>>>
> > > >>>>>> File "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
> > > >>>>>> line 176, in checkJARMetaData
> > > >>>>>> raise RuntimeError('%s is missing "%s" inside its
> > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
> > > >>>>>> RuntimeError: JAR file
> > > >>>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
> > > >>>>>> is missing "Implementation-Version: 8.10.0
> > > >>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
> > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)
> > > >>>>>>
> > > >>>>>> FWIW, I verified that the other Lucene JAR files have this line in
> > > >>>>>> them, such as core:
> > > >>>>>>
> > > >>>>>> Manifest-Version: 1.0
> > > >>>>>> Ant-Version: Apache Ant 1.9.15
> > > >>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
> > > >>>>>> Extension-Name: org.apache.lucene
> > > >>>>>> Specification-Title: Lucene Search Engine: core
> > > >>>>>> Specification-Version: 8.10.0
> > > >>>>>> Specification-Vendor: The Apache Software Foundation
> > > >>>>>> Implementation-Title: org.apache.lucene
> > > >>>>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
> > > >>>>>> 7 - tjp - 2021-09-14 19:08:42
> > > >>>>>> Implementation-Vendor: The Apache Software Foundation
> > > >>>>>> X-Compile-Source-JDK: 8
> > > >>>>>> X-Compile-Target-JDK: 8
> > > >>>>>> Multi-Release: true
> > > >>>>>>
> > > >>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
> > > >>>>>> <ichattopadhyaya@gmail.com> wrote:
> > > >>>>>> >
> > > >>>>>> > All the best, this is the worst step.
> > > >>>>>> >
> > > >>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com> wrote:
> > > >>>>>> >>
> > > >>>>>> >> Building RC1 now ... stay tuned.
> > > >>>>>> >>
> > > >>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com> wrote:
> > > >>>>>> >> >
> > > >>>>>> >> > Thanks for the update Mike!
> > > >>>>>> >> >
> > > >>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
> > > >>>>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
> > > >>>>>> >> > reported on Slack earlier today.
> > > >>>>>> >> >
> > > >>>>>> >> > Cheers,
> > > >>>>>> >> > Tim
> > > >>>>>> >> >
> > > >>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
> > > >>>>>> >> > >
> > > >>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
> > > >>>>>> >> > > both look pretty good, but I've got a few last unit tests that I need
> > > >>>>>> >> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
> > > >>>>>> >> > > sure to keep you updated though.
> > > >>>>>> >> > >
> > > >>>>>> >> > >
> > > >>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com> wrote:
> > > >>>>>> >> > > >
> > > >>>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
> > > >>>>>> >> > > > the schema designer. I haven't built the RC yet, so going to see if I
> > > >>>>>> >> > > > can get this in today.
> > > >>>>>> >> > > >
> > > >>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
> > > >>>>>> >> > > > >
> > > >>>>>> >> > > > > NOTICE:
> > > >>>>>> >> > > > >
> > > >>>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
> > > >>>>>> >> > > > >
> > > >>>>>> >> > > > > Please observe the normal rules:
> > > >>>>>> >> > > > >
> > > >>>>>> >> > > > > * No new features may be committed to the branch.
> > > >>>>>> >> > > > >
> > > >>>>>> >> > > > > * Documentation patches, build patches and serious bug fixes may be
> > > >>>>>> >> > > > > committed to the branch. However, you should submit all patches you
> > > >>>>>> >> > > > > want to commit to Jira first to give others the chance to review
> > > >>>>>> >> > > > > and possibly vote against the patch. Keep in mind that it is our
> > > >>>>>> >> > > > > main intention to keep the branch as stable as possible.
> > > >>>>>> >> > > > >
> > > >>>>>> >> > > > > * All patches that are intended for the branch should first be committed
> > > >>>>>> >> > > > > to the unstable branch, merged into the stable branch, and then into
> > > >>>>>> >> > > > > the current release branch.
> > > >>>>>> >> > > > >
> > > >>>>>> >> > > > > * Normal unstable and stable branch development may continue as usual.
> > > >>>>>> >> > > > > However, if you plan to commit a big change to the unstable branch
> > > >>>>>> >> > > > > while the branch feature freeze is in effect, think twice: can't the
> > > >>>>>> >> > > > > addition wait a couple more days? Merges of bug fixes into the branch
> > > >>>>>> >> > > > > may become more difficult.
> > > >>>>>> >> > > > >
> > > >>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
> > > >>>>>> >> > > > > a release candidate build.
> > > >>>>>> >> > > > > ----
> > > >>>>>> >> > > >
> > > >>>>>> >> > > > ---------------------------------------------------------------------
> > > >>>>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > >>>>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > >>>>>> >> > > >
> > > >>>>>> >> > >
> > > >>>>>> >> > > ---------------------------------------------------------------------
> > > >>>>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > >>>>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > >>>>>> >> > >
> > > >>>>>> >>
> > > >>>>>> >> ---------------------------------------------------------------------
> > > >>>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > >>>>>> >> For additional commands, e-mail: dev-help@solr.apache.org
> > > >>>>>> >>
> > > >>>>>>
> > > >>>>>> ---------------------------------------------------------------------
> > > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > >>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
> > > >>>>>>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > For additional commands, e-mail: dev-help@solr.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
Ok, fine.

D.

On Thu, Sep 16, 2021 at 12:00 PM Robert Muir <rcmuir@gmail.com> wrote:
>
> The shit that mike's python tool is doing, i'd expect bugs like that.
> If you honestly disagree with me, go fix his python code to obey "the
> rules". Be sure you have "the rules" correct as well (count bytes, not
> characters, count EOL, etc).
>
> Try to imagine how much software is going to get this wrong. Various
> tools scanning for security problems that aren't written in java and
> hence aren't using Jar class. Your security team isn't writing fucking
> java code. Your ops team isn't writing fucking java code, so it really
> does not matter if it is "easy in java".
>
> This is *exactly* what i mean when i say this crazy shit has NO
> INTEROP. It is 100% pure garbage, java showing its true colors.
>
> On Thu, Sep 16, 2021 at 5:53 AM Robert Muir <rcmuir@gmail.com> wrote:
> >
> > I was referring to doing this with languages other than java.
> >
> > I'm also assuming that exceeding this limit is going to cause indirect
> > hassles for users of lucene, e.g. breaking various security / supply
> > chain tools. We know a lot of these are total crap but people in the
> > corporate world have to suffer under them.
> >
> > Its super-easy to use a short hash here and avoid problems.
> >
> > On Thu, Sep 16, 2021 at 3:03 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> > >
> > > Jar command doesn't have it, true. But it's fairly trivial to do, even
> > > with an inline snippet like this one?
> > >
> > > public class PrintManifest {
> > > public static void main(String[] jars) throws IOException {
> > > for (var jar : jars) {
> > > var manifest = new JarFile(Paths.get(jar).toFile()).getManifest();
> > > var attrs = manifest.getMainAttributes();
> > > System.out.println(jar + ": " + attrs.getValue("Implementation-Version"));
> > > }
> > > }
> > > }
> > >
> > > I have this in my lucene-core-9.0.0-SNAPSHOT.jar:
> > >
> > > Implementation-Version: 9.0.0-SNAPSHOT de45b68c909815ce5ea7b6b9e1a2ce337
> > > 5b6334d [snapshot build, details omitted]
> > >
> > > and running:
> > >
> > > java PrintManifest.java lucene-core-9.0.0-SNAPSHOT.jar
> > >
> > > shows:
> > >
> > > lucene-core-9.0.0-SNAPSHOT.jar: 9.0.0-SNAPSHOT
> > > de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details
> > > omitted]
> > >
> > > This seems easier to me than trying to remember and keep the length of
> > > that line shorter than an arbitrary limit.
> > >
> > > Dawid
> > >
> > >
> > > On Wed, Sep 15, 2021 at 9:46 PM Robert Muir <rcmuir@gmail.com> wrote:
> > > >
> > > > But its irrelevant that is "valid" when virtually no tools match it.
> > > >
> > > > In other words, I'd agree with you if the "jar" command had some
> > > > ability to read these manifests and print stuff to stdout, e.g. if
> > > > there was ANY interop at all here.
> > > >
> > > > But there isn't. So IMO it makes no sense to cause confusion and chaos
> > > > by adding an unnecessarily long git commit hash.
> > > >
> > > > On Wed, Sep 15, 2021 at 3:26 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> > > > >
> > > > >
> > > > > This is valid manifest line-breaking though... Can we read the manifest properly on the smoke tester side somehow (for example, run a Java process that reads and extracts the required attribute)? This way we wouldn't care about the implementation details of how manifest wraps the lines (or escapes characters).
> > > > >
> > > > > D.
> > > > >
> > > > > On Wed, Sep 15, 2021 at 8:46 PM Mike Drob <mdrob@mdrob.com> wrote:
> > > > >>
> > > > >> The benchmark jar has the info we need… sort of. When I built it, it has:
> > > > >>
> > > > >> Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
> > > > >> 1 - mdrob - 2021-09-15 11:40:36
> > > > >>
> > > > >>
> > > > >> and it’s looking for Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
> > > > >>
> > > > >> Because 8.10 is a character longer than 8.9, we happen to wrap the last character of the git commit sha. From the manifest spec:
> > > > >>
> > > > >> No line may be longer than 72 bytes (not characters), in its UTF8-encoded form.
> > > > >> If a value would make the initial line longer than this, it should be continued
> > > > >> on extra lines (each starting with a single SPACE).
> > > > >>
> > > > >> And we were already teetering on the edge of that limit. We'll run into this problem again in a few years when we try to release version 10.0.0, so solving it now has practical benefits down the line.
> > > > >>
> > > > >> There's a few options that I can come up with -
> > > > >> 1. Use the short-hash when we generate the jar
> > > > >> 2. Use the short-hash when we check the contents in the smoke test
> > > > >> 3. Do some line join magic in the smoke test.
> > > > >>
> > > > >> I'm leaning towards number 1 as I feel that would still be unique enough for our needs, but would like to hear from others as well.
> > > > >>
> > > > >> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <thelabdude@gmail.com> wrote:
> > > > >>>
> > > > >>> can someone also please look into that benchmark jar issue?
> > > > >>>
> > > > >>> Sent from my iPhone
> > > > >>>
> > > > >>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co.invalid> wrote:
> > > > >>>
> > > > >>> ?
> > > > >>> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
> > > > >>>
> > > > >>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com> wrote:
> > > > >>>>
> > > > >>>> I think since Tim is out on vacation, it's probably not too late. That looks like a good fix to have, do we know how long the bug has been present?
> > > > >>>>
> > > > >>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova <mayya.sharipova@elastic.co.invalid> wrote:
> > > > >>>>>
> > > > >>>>> Hello everyone,
> > > > >>>>> We have discovered a bug and fixed a bug in Lucene sort optimization (LUCENE-10106) and would like to merge it to Lucene 8.10 if it is not too late.
> > > > >>>>> I apologize for the inconvenience, the bug was discovered just yesterday.
> > > > >>>>>
> > > > >>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org> wrote:
> > > > >>>>>>
> > > > >>>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
> > > > >>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
> > > > >>>>>> unless someone else wants to pick up the RM duties before then?
> > > > >>>>>>
> > > > >>>>>> After failing the test suite at various places and other weirdness
> > > > >>>>>> like .asc files not getting created, I finally got to the smoke test
> > > > >>>>>> part, which is now failing with:
> > > > >>>>>>
> > > > >>>>>> File "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
> > > > >>>>>> line 176, in checkJARMetaData
> > > > >>>>>> raise RuntimeError('%s is missing "%s" inside its
> > > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
> > > > >>>>>> RuntimeError: JAR file
> > > > >>>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
> > > > >>>>>> is missing "Implementation-Version: 8.10.0
> > > > >>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
> > > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)
> > > > >>>>>>
> > > > >>>>>> FWIW, I verified that the other Lucene JAR files have this line in
> > > > >>>>>> them, such as core:
> > > > >>>>>>
> > > > >>>>>> Manifest-Version: 1.0
> > > > >>>>>> Ant-Version: Apache Ant 1.9.15
> > > > >>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
> > > > >>>>>> Extension-Name: org.apache.lucene
> > > > >>>>>> Specification-Title: Lucene Search Engine: core
> > > > >>>>>> Specification-Version: 8.10.0
> > > > >>>>>> Specification-Vendor: The Apache Software Foundation
> > > > >>>>>> Implementation-Title: org.apache.lucene
> > > > >>>>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
> > > > >>>>>> 7 - tjp - 2021-09-14 19:08:42
> > > > >>>>>> Implementation-Vendor: The Apache Software Foundation
> > > > >>>>>> X-Compile-Source-JDK: 8
> > > > >>>>>> X-Compile-Target-JDK: 8
> > > > >>>>>> Multi-Release: true
> > > > >>>>>>
> > > > >>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
> > > > >>>>>> <ichattopadhyaya@gmail.com> wrote:
> > > > >>>>>> >
> > > > >>>>>> > All the best, this is the worst step.
> > > > >>>>>> >
> > > > >>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com> wrote:
> > > > >>>>>> >>
> > > > >>>>>> >> Building RC1 now ... stay tuned.
> > > > >>>>>> >>
> > > > >>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com> wrote:
> > > > >>>>>> >> >
> > > > >>>>>> >> > Thanks for the update Mike!
> > > > >>>>>> >> >
> > > > >>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
> > > > >>>>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
> > > > >>>>>> >> > reported on Slack earlier today.
> > > > >>>>>> >> >
> > > > >>>>>> >> > Cheers,
> > > > >>>>>> >> > Tim
> > > > >>>>>> >> >
> > > > >>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
> > > > >>>>>> >> > >
> > > > >>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
> > > > >>>>>> >> > > both look pretty good, but I've got a few last unit tests that I need
> > > > >>>>>> >> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
> > > > >>>>>> >> > > sure to keep you updated though.
> > > > >>>>>> >> > >
> > > > >>>>>> >> > >
> > > > >>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com> wrote:
> > > > >>>>>> >> > > >
> > > > >>>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
> > > > >>>>>> >> > > > the schema designer. I haven't built the RC yet, so going to see if I
> > > > >>>>>> >> > > > can get this in today.
> > > > >>>>>> >> > > >
> > > > >>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
> > > > >>>>>> >> > > > >
> > > > >>>>>> >> > > > > NOTICE:
> > > > >>>>>> >> > > > >
> > > > >>>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
> > > > >>>>>> >> > > > >
> > > > >>>>>> >> > > > > Please observe the normal rules:
> > > > >>>>>> >> > > > >
> > > > >>>>>> >> > > > > * No new features may be committed to the branch.
> > > > >>>>>> >> > > > >
> > > > >>>>>> >> > > > > * Documentation patches, build patches and serious bug fixes may be
> > > > >>>>>> >> > > > > committed to the branch. However, you should submit all patches you
> > > > >>>>>> >> > > > > want to commit to Jira first to give others the chance to review
> > > > >>>>>> >> > > > > and possibly vote against the patch. Keep in mind that it is our
> > > > >>>>>> >> > > > > main intention to keep the branch as stable as possible.
> > > > >>>>>> >> > > > >
> > > > >>>>>> >> > > > > * All patches that are intended for the branch should first be committed
> > > > >>>>>> >> > > > > to the unstable branch, merged into the stable branch, and then into
> > > > >>>>>> >> > > > > the current release branch.
> > > > >>>>>> >> > > > >
> > > > >>>>>> >> > > > > * Normal unstable and stable branch development may continue as usual.
> > > > >>>>>> >> > > > > However, if you plan to commit a big change to the unstable branch
> > > > >>>>>> >> > > > > while the branch feature freeze is in effect, think twice: can't the
> > > > >>>>>> >> > > > > addition wait a couple more days? Merges of bug fixes into the branch
> > > > >>>>>> >> > > > > may become more difficult.
> > > > >>>>>> >> > > > >
> > > > >>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
> > > > >>>>>> >> > > > > a release candidate build.
> > > > >>>>>> >> > > > > ----
> > > > >>>>>> >> > > >
> > > > >>>>>> >> > > > ---------------------------------------------------------------------
> > > > >>>>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > >>>>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > >>>>>> >> > > >
> > > > >>>>>> >> > >
> > > > >>>>>> >> > > ---------------------------------------------------------------------
> > > > >>>>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > >>>>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
> > > > >>>>>> >> > >
> > > > >>>>>> >>
> > > > >>>>>> >> ---------------------------------------------------------------------
> > > > >>>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > > >>>>>> >> For additional commands, e-mail: dev-help@solr.apache.org
> > > > >>>>>> >>
> > > > >>>>>>
> > > > >>>>>> ---------------------------------------------------------------------
> > > > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > > >>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
> > > > >>>>>>
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > > For additional commands, e-mail: dev-help@solr.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: dev-help@lucene.apache.org
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
: I was referring to doing this with languages other than java.
:
: I'm also assuming that exceeding this limit is going to cause indirect
: hassles for users of lucene, e.g. breaking various security / supply
: chain tools. We know a lot of these are total crap but people in the
: corporate world have to suffer under them.

Just to be clear -- our 'Implementation-Version:' has been exceeding the
72 byte "single line" limit for a LOOOOONG time -- worrying about how
"security / supply chain" tools will handle parsing that line now seems
kind of silly...

If tools can't handle a line wrap in the 8.10 jars, then they haven't
been able to handle the line wrap since we switched from svn to git (when
the Implementation Version values switched from being based svn version#
to git sha)

The *ONLY* thing that's new here is where in the value the line wrap
happens (with 8.10.0 it happens in the middle of the SHA) and that our
smoketest tool isn't smart enough to parse the values properly.

This is not even the first time we've even had a conversation about the
smoke tester and Implementation Version line wraps: LUCENE-7023.


: Its super-easy to use a short hash here and avoid problems.


There is however an actual and practical downside to switching our
implementation version to using a "short" SHA, and that's that we would
lose the ability to garuntee that the information in the
Implementation-Version uniquely identifies what commit a given jar was
built from. Multiple commits with the same short(end) hash are possible
-- Multiple commits with identical (full) commits is not.

Folks may think that using git tags is useful enough for figuring this
out from official releases, but being able to look at the jar metadata
from arbitrary builds off of arbitrary branches and sanity check where
exactly they come from has been very useful to me for 10+ years.


: On Thu, Sep 16, 2021 at 3:03 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
: >
: > Jar command doesn't have it, true. But it's fairly trivial to do, even
: > with an inline snippet like this one?
: >
: > public class PrintManifest {
: > public static void main(String[] jars) throws IOException {
: > for (var jar : jars) {
: > var manifest = new JarFile(Paths.get(jar).toFile()).getManifest();
: > var attrs = manifest.getMainAttributes();
: > System.out.println(jar + ": " + attrs.getValue("Implementation-Version"));
: > }
: > }
: > }
: >
: > I have this in my lucene-core-9.0.0-SNAPSHOT.jar:
: >
: > Implementation-Version: 9.0.0-SNAPSHOT de45b68c909815ce5ea7b6b9e1a2ce337
: > 5b6334d [snapshot build, details omitted]
: >
: > and running:
: >
: > java PrintManifest.java lucene-core-9.0.0-SNAPSHOT.jar
: >
: > shows:
: >
: > lucene-core-9.0.0-SNAPSHOT.jar: 9.0.0-SNAPSHOT
: > de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details
: > omitted]
: >
: > This seems easier to me than trying to remember and keep the length of
: > that line shorter than an arbitrary limit.
: >
: > Dawid
: >
: >
: > On Wed, Sep 15, 2021 at 9:46 PM Robert Muir <rcmuir@gmail.com> wrote:
: > >
: > > But its irrelevant that is "valid" when virtually no tools match it.
: > >
: > > In other words, I'd agree with you if the "jar" command had some
: > > ability to read these manifests and print stuff to stdout, e.g. if
: > > there was ANY interop at all here.
: > >
: > > But there isn't. So IMO it makes no sense to cause confusion and chaos
: > > by adding an unnecessarily long git commit hash.
: > >
: > > On Wed, Sep 15, 2021 at 3:26 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
: > > >
: > > >
: > > > This is valid manifest line-breaking though... Can we read the manifest properly on the smoke tester side somehow (for example, run a Java process that reads and extracts the required attribute)? This way we wouldn't care about the implementation details of how manifest wraps the lines (or escapes characters).
: > > >
: > > > D.
: > > >
: > > > On Wed, Sep 15, 2021 at 8:46 PM Mike Drob <mdrob@mdrob.com> wrote:
: > > >>
: > > >> The benchmark jar has the info we need… sort of. When I built it, it has:
: > > >>
: > > >> Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
: > > >> 1 - mdrob - 2021-09-15 11:40:36
: > > >>
: > > >>
: > > >> and it’s looking for Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
: > > >>
: > > >> Because 8.10 is a character longer than 8.9, we happen to wrap the last character of the git commit sha. From the manifest spec:
: > > >>
: > > >> No line may be longer than 72 bytes (not characters), in its UTF8-encoded form.
: > > >> If a value would make the initial line longer than this, it should be continued
: > > >> on extra lines (each starting with a single SPACE).
: > > >>
: > > >> And we were already teetering on the edge of that limit. We'll run into this problem again in a few years when we try to release version 10.0.0, so solving it now has practical benefits down the line.
: > > >>
: > > >> There's a few options that I can come up with -
: > > >> 1. Use the short-hash when we generate the jar
: > > >> 2. Use the short-hash when we check the contents in the smoke test
: > > >> 3. Do some line join magic in the smoke test.
: > > >>
: > > >> I'm leaning towards number 1 as I feel that would still be unique enough for our needs, but would like to hear from others as well.
: > > >>
: > > >> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <thelabdude@gmail.com> wrote:
: > > >>>
: > > >>> can someone also please look into that benchmark jar issue?
: > > >>>
: > > >>> Sent from my iPhone
: > > >>>
: > > >>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co.invalid> wrote:
: > > >>>
: > > >>> ?
: > > >>> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
: > > >>>
: > > >>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com> wrote:
: > > >>>>
: > > >>>> I think since Tim is out on vacation, it's probably not too late. That looks like a good fix to have, do we know how long the bug has been present?
: > > >>>>
: > > >>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova <mayya.sharipova@elastic.co.invalid> wrote:
: > > >>>>>
: > > >>>>> Hello everyone,
: > > >>>>> We have discovered a bug and fixed a bug in Lucene sort optimization (LUCENE-10106) and would like to merge it to Lucene 8.10 if it is not too late.
: > > >>>>> I apologize for the inconvenience, the bug was discovered just yesterday.
: > > >>>>>
: > > >>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org> wrote:
: > > >>>>>>
: > > >>>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
: > > >>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
: > > >>>>>> unless someone else wants to pick up the RM duties before then?
: > > >>>>>>
: > > >>>>>> After failing the test suite at various places and other weirdness
: > > >>>>>> like .asc files not getting created, I finally got to the smoke test
: > > >>>>>> part, which is now failing with:
: > > >>>>>>
: > > >>>>>> File "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
: > > >>>>>> line 176, in checkJARMetaData
: > > >>>>>> raise RuntimeError('%s is missing "%s" inside its
: > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
: > > >>>>>> RuntimeError: JAR file
: > > >>>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
: > > >>>>>> is missing "Implementation-Version: 8.10.0
: > > >>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
: > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)
: > > >>>>>>
: > > >>>>>> FWIW, I verified that the other Lucene JAR files have this line in
: > > >>>>>> them, such as core:
: > > >>>>>>
: > > >>>>>> Manifest-Version: 1.0
: > > >>>>>> Ant-Version: Apache Ant 1.9.15
: > > >>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
: > > >>>>>> Extension-Name: org.apache.lucene
: > > >>>>>> Specification-Title: Lucene Search Engine: core
: > > >>>>>> Specification-Version: 8.10.0
: > > >>>>>> Specification-Vendor: The Apache Software Foundation
: > > >>>>>> Implementation-Title: org.apache.lucene
: > > >>>>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
: > > >>>>>> 7 - tjp - 2021-09-14 19:08:42
: > > >>>>>> Implementation-Vendor: The Apache Software Foundation
: > > >>>>>> X-Compile-Source-JDK: 8
: > > >>>>>> X-Compile-Target-JDK: 8
: > > >>>>>> Multi-Release: true
: > > >>>>>>
: > > >>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
: > > >>>>>> <ichattopadhyaya@gmail.com> wrote:
: > > >>>>>> >
: > > >>>>>> > All the best, this is the worst step.
: > > >>>>>> >
: > > >>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com> wrote:
: > > >>>>>> >>
: > > >>>>>> >> Building RC1 now ... stay tuned.
: > > >>>>>> >>
: > > >>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com> wrote:
: > > >>>>>> >> >
: > > >>>>>> >> > Thanks for the update Mike!
: > > >>>>>> >> >
: > > >>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
: > > >>>>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
: > > >>>>>> >> > reported on Slack earlier today.
: > > >>>>>> >> >
: > > >>>>>> >> > Cheers,
: > > >>>>>> >> > Tim
: > > >>>>>> >> >
: > > >>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
: > > >>>>>> >> > >
: > > >>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
: > > >>>>>> >> > > both look pretty good, but I've got a few last unit tests that I need
: > > >>>>>> >> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
: > > >>>>>> >> > > sure to keep you updated though.
: > > >>>>>> >> > >
: > > >>>>>> >> > >
: > > >>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com> wrote:
: > > >>>>>> >> > > >
: > > >>>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
: > > >>>>>> >> > > > the schema designer. I haven't built the RC yet, so going to see if I
: > > >>>>>> >> > > > can get this in today.
: > > >>>>>> >> > > >
: > > >>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > NOTICE:
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > Please observe the normal rules:
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > * No new features may be committed to the branch.
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > * Documentation patches, build patches and serious bug fixes may be
: > > >>>>>> >> > > > > committed to the branch. However, you should submit all patches you
: > > >>>>>> >> > > > > want to commit to Jira first to give others the chance to review
: > > >>>>>> >> > > > > and possibly vote against the patch. Keep in mind that it is our
: > > >>>>>> >> > > > > main intention to keep the branch as stable as possible.
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > * All patches that are intended for the branch should first be committed
: > > >>>>>> >> > > > > to the unstable branch, merged into the stable branch, and then into
: > > >>>>>> >> > > > > the current release branch.
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > * Normal unstable and stable branch development may continue as usual.
: > > >>>>>> >> > > > > However, if you plan to commit a big change to the unstable branch
: > > >>>>>> >> > > > > while the branch feature freeze is in effect, think twice: can't the
: > > >>>>>> >> > > > > addition wait a couple more days? Merges of bug fixes into the branch
: > > >>>>>> >> > > > > may become more difficult.
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
: > > >>>>>> >> > > > > a release candidate build.
: > > >>>>>> >> > > > > ----
: > > >>>>>> >> > > >
: > > >>>>>> >> > > > ---------------------------------------------------------------------
: > > >>>>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: > > >>>>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
: > > >>>>>> >> > > >
: > > >>>>>> >> > >
: > > >>>>>> >> > > ---------------------------------------------------------------------
: > > >>>>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: > > >>>>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
: > > >>>>>> >> > >
: > > >>>>>> >>
: > > >>>>>> >> ---------------------------------------------------------------------
: > > >>>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
: > > >>>>>> >> For additional commands, e-mail: dev-help@solr.apache.org
: > > >>>>>> >>
: > > >>>>>>
: > > >>>>>> ---------------------------------------------------------------------
: > > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: > > >>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
: > > >>>>>>
: > >
: > > ---------------------------------------------------------------------
: > > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
: > > For additional commands, e-mail: dev-help@solr.apache.org
: > >
: >
: > ---------------------------------------------------------------------
: > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: > For additional commands, e-mail: dev-help@lucene.apache.org
: >
:
: ---------------------------------------------------------------------
: To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: For additional commands, e-mail: dev-help@lucene.apache.org
:
:

-Hoss
http://www.lucidworks.com/
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
We could do a few things to keep everyone happy -

1) keep abbreviated hash in the Implementat-Version and use a separate
manifest entry to store a full hash.
2) use a longer version for git show (abbrev=num) so that the chance of
collisions in the future is minimized. It's still not a full hash but a
long(er) forced prefix.

D.

On Fri, Sep 17, 2021 at 12:21 AM Chris Hostetter <hossman_lucene@fucit.org>
wrote:

>
> : I was referring to doing this with languages other than java.
> :
> : I'm also assuming that exceeding this limit is going to cause indirect
> : hassles for users of lucene, e.g. breaking various security / supply
> : chain tools. We know a lot of these are total crap but people in the
> : corporate world have to suffer under them.
>
> Just to be clear -- our 'Implementation-Version:' has been exceeding the
> 72 byte "single line" limit for a LOOOOONG time -- worrying about how
> "security / supply chain" tools will handle parsing that line now seems
> kind of silly...
>
> If tools can't handle a line wrap in the 8.10 jars, then they haven't
> been able to handle the line wrap since we switched from svn to git (when
> the Implementation Version values switched from being based svn version#
> to git sha)
>
> The *ONLY* thing that's new here is where in the value the line wrap
> happens (with 8.10.0 it happens in the middle of the SHA) and that our
> smoketest tool isn't smart enough to parse the values properly.
>
> This is not even the first time we've even had a conversation about the
> smoke tester and Implementation Version line wraps: LUCENE-7023.
>
>
> : Its super-easy to use a short hash here and avoid problems.
>
>
> There is however an actual and practical downside to switching our
> implementation version to using a "short" SHA, and that's that we would
> lose the ability to garuntee that the information in the
> Implementation-Version uniquely identifies what commit a given jar was
> built from. Multiple commits with the same short(end) hash are possible
> -- Multiple commits with identical (full) commits is not.
>
> Folks may think that using git tags is useful enough for figuring this
> out from official releases, but being able to look at the jar metadata
> from arbitrary builds off of arbitrary branches and sanity check where
> exactly they come from has been very useful to me for 10+ years.
>
>
> : On Thu, Sep 16, 2021 at 3:03 AM Dawid Weiss <dawid.weiss@gmail.com>
> wrote:
> : >
> : > Jar command doesn't have it, true. But it's fairly trivial to do, even
> : > with an inline snippet like this one?
> : >
> : > public class PrintManifest {
> : > public static void main(String[] jars) throws IOException {
> : > for (var jar : jars) {
> : > var manifest = new
> JarFile(Paths.get(jar).toFile()).getManifest();
> : > var attrs = manifest.getMainAttributes();
> : > System.out.println(jar + ": " +
> attrs.getValue("Implementation-Version"));
> : > }
> : > }
> : > }
> : >
> : > I have this in my lucene-core-9.0.0-SNAPSHOT.jar:
> : >
> : > Implementation-Version: 9.0.0-SNAPSHOT
> de45b68c909815ce5ea7b6b9e1a2ce337
> : > 5b6334d [snapshot build, details omitted]
> : >
> : > and running:
> : >
> : > java PrintManifest.java lucene-core-9.0.0-SNAPSHOT.jar
> : >
> : > shows:
> : >
> : > lucene-core-9.0.0-SNAPSHOT.jar: 9.0.0-SNAPSHOT
> : > de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details
> : > omitted]
> : >
> : > This seems easier to me than trying to remember and keep the length of
> : > that line shorter than an arbitrary limit.
> : >
> : > Dawid
> : >
> : >
> : > On Wed, Sep 15, 2021 at 9:46 PM Robert Muir <rcmuir@gmail.com> wrote:
> : > >
> : > > But its irrelevant that is "valid" when virtually no tools match it.
> : > >
> : > > In other words, I'd agree with you if the "jar" command had some
> : > > ability to read these manifests and print stuff to stdout, e.g. if
> : > > there was ANY interop at all here.
> : > >
> : > > But there isn't. So IMO it makes no sense to cause confusion and
> chaos
> : > > by adding an unnecessarily long git commit hash.
> : > >
> : > > On Wed, Sep 15, 2021 at 3:26 PM Dawid Weiss <dawid.weiss@gmail.com>
> wrote:
> : > > >
> : > > >
> : > > > This is valid manifest line-breaking though... Can we read the
> manifest properly on the smoke tester side somehow (for example, run a Java
> process that reads and extracts the required attribute)? This way we
> wouldn't care about the implementation details of how manifest wraps the
> lines (or escapes characters).
> : > > >
> : > > > D.
> : > > >
> : > > > On Wed, Sep 15, 2021 at 8:46 PM Mike Drob <mdrob@mdrob.com> wrote:
> : > > >>
> : > > >> The benchmark jar has the info we need… sort of. When I built it,
> it has:
> : > > >>
> : > > >> Implementation-Version: 8.10.0
> 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
> : > > >> 1 - mdrob - 2021-09-15 11:40:36
> : > > >>
> : > > >>
> : > > >> and it’s looking for Implementation-Version: 8.10.0
> 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
> : > > >>
> : > > >> Because 8.10 is a character longer than 8.9, we happen to wrap
> the last character of the git commit sha. From the manifest spec:
> : > > >>
> : > > >> No line may be longer than 72 bytes (not characters), in its
> UTF8-encoded form.
> : > > >> If a value would make the initial line longer than this, it
> should be continued
> : > > >> on extra lines (each starting with a single SPACE).
> : > > >>
> : > > >> And we were already teetering on the edge of that limit. We'll
> run into this problem again in a few years when we try to release version
> 10.0.0, so solving it now has practical benefits down the line.
> : > > >>
> : > > >> There's a few options that I can come up with -
> : > > >> 1. Use the short-hash when we generate the jar
> : > > >> 2. Use the short-hash when we check the contents in the smoke test
> : > > >> 3. Do some line join magic in the smoke test.
> : > > >>
> : > > >> I'm leaning towards number 1 as I feel that would still be unique
> enough for our needs, but would like to hear from others as well.
> : > > >>
> : > > >> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <
> thelabdude@gmail.com> wrote:
> : > > >>>
> : > > >>> can someone also please look into that benchmark jar issue?
> : > > >>>
> : > > >>> Sent from my iPhone
> : > > >>>
> : > > >>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co.invalid>
> wrote:
> : > > >>>
> : > > >>> ?
> : > > >>> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
> : > > >>>
> : > > >>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com>
> wrote:
> : > > >>>>
> : > > >>>> I think since Tim is out on vacation, it's probably not too
> late. That looks like a good fix to have, do we know how long the bug has
> been present?
> : > > >>>>
> : > > >>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova <
> mayya.sharipova@elastic.co.invalid> wrote:
> : > > >>>>>
> : > > >>>>> Hello everyone,
> : > > >>>>> We have discovered a bug and fixed a bug in Lucene sort
> optimization (LUCENE-10106) and would like to merge it to Lucene 8.10 if it
> is not too late.
> : > > >>>>> I apologize for the inconvenience, the bug was discovered just
> yesterday.
> : > > >>>>>
> : > > >>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <
> thelabdude@apache.org> wrote:
> : > > >>>>>>
> : > > >>>>>> Ahem ... unfortunately there will not be an 8.10 RC this
> week. I'm
> : > > >>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept
> 20
> : > > >>>>>> unless someone else wants to pick up the RM duties before
> then?
> : > > >>>>>>
> : > > >>>>>> After failing the test suite at various places and other
> weirdness
> : > > >>>>>> like .asc files not getting created, I finally got to the
> smoke test
> : > > >>>>>> part, which is now failing with:
> : > > >>>>>>
> : > > >>>>>> File
> "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
> : > > >>>>>> line 176, in checkJARMetaData
> : > > >>>>>> raise RuntimeError('%s is missing "%s" inside its
> : > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
> : > > >>>>>> RuntimeError: JAR file
> : > > >>>>>>
> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
> : > > >>>>>> is missing "Implementation-Version: 8.10.0
> : > > >>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
> : > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)
> : > > >>>>>>
> : > > >>>>>> FWIW, I verified that the other Lucene JAR files have this
> line in
> : > > >>>>>> them, such as core:
> : > > >>>>>>
> : > > >>>>>> Manifest-Version: 1.0
> : > > >>>>>> Ant-Version: Apache Ant 1.9.15
> : > > >>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
> : > > >>>>>> Extension-Name: org.apache.lucene
> : > > >>>>>> Specification-Title: Lucene Search Engine: core
> : > > >>>>>> Specification-Version: 8.10.0
> : > > >>>>>> Specification-Vendor: The Apache Software Foundation
> : > > >>>>>> Implementation-Title: org.apache.lucene
> : > > >>>>>> Implementation-Version: 8.10.0
> ecf5c747e6df418dd05a18af327c20051f0584d
> : > > >>>>>> 7 - tjp - 2021-09-14 19:08:42
> : > > >>>>>> Implementation-Vendor: The Apache Software Foundation
> : > > >>>>>> X-Compile-Source-JDK: 8
> : > > >>>>>> X-Compile-Target-JDK: 8
> : > > >>>>>> Multi-Release: true
> : > > >>>>>>
> : > > >>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
> : > > >>>>>> <ichattopadhyaya@gmail.com> wrote:
> : > > >>>>>> >
> : > > >>>>>> > All the best, this is the worst step.
> : > > >>>>>> >
> : > > >>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <
> thelabdude@gmail.com> wrote:
> : > > >>>>>> >>
> : > > >>>>>> >> Building RC1 now ... stay tuned.
> : > > >>>>>> >>
> : > > >>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <
> thelabdude@gmail.com> wrote:
> : > > >>>>>> >> >
> : > > >>>>>> >> > Thanks for the update Mike!
> : > > >>>>>> >> >
> : > > >>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a
> quick PR for
> : > > >>>>>> >> > SOLR-15621, which looks like an easy win for the issue
> Cassandra
> : > > >>>>>> >> > reported on Slack earlier today.
> : > > >>>>>> >> >
> : > > >>>>>> >> > Cheers,
> : > > >>>>>> >> > Tim
> : > > >>>>>> >> >
> : > > >>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <
> mdrob@apache.org> wrote:
> : > > >>>>>> >> > >
> : > > >>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and
> benchmarking
> : > > >>>>>> >> > > both look pretty good, but I've got a few last unit
> tests that I need
> : > > >>>>>> >> > > to chase down. Hopefully taken care of by today or
> tomorrow, I'll be
> : > > >>>>>> >> > > sure to keep you updated though.
> : > > >>>>>> >> > >
> : > > >>>>>> >> > >
> : > > >>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <
> thelabdude@gmail.com> wrote:
> : > > >>>>>> >> > > >
> : > > >>>>>> >> > > > I found
> https://issues.apache.org/jira/browse/SOLR-15620 while testing
> : > > >>>>>> >> > > > the schema designer. I haven't built the RC yet, so
> going to see if I
> : > > >>>>>> >> > > > can get this in today.
> : > > >>>>>> >> > > >
> : > > >>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <
> thelabdude@apache.org> wrote:
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > NOTICE:
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > Branch branch_8_10 has been cut and versions
> updated to 8.11 on stable branch.
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > Please observe the normal rules:
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > * No new features may be committed to the branch.
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > * Documentation patches, build patches and serious
> bug fixes may be
> : > > >>>>>> >> > > > > committed to the branch. However, you should
> submit all patches you
> : > > >>>>>> >> > > > > want to commit to Jira first to give others the
> chance to review
> : > > >>>>>> >> > > > > and possibly vote against the patch. Keep in
> mind that it is our
> : > > >>>>>> >> > > > > main intention to keep the branch as stable as
> possible.
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > * All patches that are intended for the branch
> should first be committed
> : > > >>>>>> >> > > > > to the unstable branch, merged into the stable
> branch, and then into
> : > > >>>>>> >> > > > > the current release branch.
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > * Normal unstable and stable branch development
> may continue as usual.
> : > > >>>>>> >> > > > > However, if you plan to commit a big change to
> the unstable branch
> : > > >>>>>> >> > > > > while the branch feature freeze is in effect,
> think twice: can't the
> : > > >>>>>> >> > > > > addition wait a couple more days? Merges of bug
> fixes into the branch
> : > > >>>>>> >> > > > > may become more difficult.
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and
> priority "Blocker" will delay
> : > > >>>>>> >> > > > > a release candidate build.
> : > > >>>>>> >> > > > > ----
> : > > >>>>>> >> > > >
> : > > >>>>>> >> > > >
> ---------------------------------------------------------------------
> : > > >>>>>> >> > > > To unsubscribe, e-mail:
> dev-unsubscribe@lucene.apache.org
> : > > >>>>>> >> > > > For additional commands, e-mail:
> dev-help@lucene.apache.org
> : > > >>>>>> >> > > >
> : > > >>>>>> >> > >
> : > > >>>>>> >> > >
> ---------------------------------------------------------------------
> : > > >>>>>> >> > > To unsubscribe, e-mail:
> dev-unsubscribe@lucene.apache.org
> : > > >>>>>> >> > > For additional commands, e-mail:
> dev-help@lucene.apache.org
> : > > >>>>>> >> > >
> : > > >>>>>> >>
> : > > >>>>>> >>
> ---------------------------------------------------------------------
> : > > >>>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> : > > >>>>>> >> For additional commands, e-mail: dev-help@solr.apache.org
> : > > >>>>>> >>
> : > > >>>>>>
> : > > >>>>>>
> ---------------------------------------------------------------------
> : > > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> : > > >>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
> : > > >>>>>>
> : > >
> : > > ---------------------------------------------------------------------
> : > > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> : > > For additional commands, e-mail: dev-help@solr.apache.org
> : > >
> : >
> : > ---------------------------------------------------------------------
> : > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> : > For additional commands, e-mail: dev-help@lucene.apache.org
> : >
> :
> : ---------------------------------------------------------------------
> : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> : For additional commands, e-mail: dev-help@lucene.apache.org
> :
> :
>
> -Hoss
> http://www.lucidworks.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
Let's just follow the spec and move on.

Just tested this python package, which has no problem parsing the problematic manifest https://pypi.org/project/jarmanifest/

>>> manifest.getAttributes("/tmp/lucene-manifest.mf")
[{'implementationversion': '9.0.0-SNAPSHOT de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details omitted]'}]

Jan

> 17. sep. 2021 kl. 09:32 skrev Dawid Weiss <dawid.weiss@gmail.com>:
>
>
> We could do a few things to keep everyone happy -
>
> 1) keep abbreviated hash in the Implementat-Version and use a separate manifest entry to store a full hash.
> 2) use a longer version for git show (abbrev=num) so that the chance of collisions in the future is minimized. It's still not a full hash but a
> long(er) forced prefix.
>
> D.
>
> On Fri, Sep 17, 2021 at 12:21 AM Chris Hostetter <hossman_lucene@fucit.org <mailto:hossman_lucene@fucit.org>> wrote:
>
> : I was referring to doing this with languages other than java.
> :
> : I'm also assuming that exceeding this limit is going to cause indirect
> : hassles for users of lucene, e.g. breaking various security / supply
> : chain tools. We know a lot of these are total crap but people in the
> : corporate world have to suffer under them.
>
> Just to be clear -- our 'Implementation-Version:' has been exceeding the
> 72 byte "single line" limit for a LOOOOONG time -- worrying about how
> "security / supply chain" tools will handle parsing that line now seems
> kind of silly...
>
> If tools can't handle a line wrap in the 8.10 jars, then they haven't
> been able to handle the line wrap since we switched from svn to git (when
> the Implementation Version values switched from being based svn version#
> to git sha)
>
> The *ONLY* thing that's new here is where in the value the line wrap
> happens (with 8.10.0 it happens in the middle of the SHA) and that our
> smoketest tool isn't smart enough to parse the values properly.
>
> This is not even the first time we've even had a conversation about the
> smoke tester and Implementation Version line wraps: LUCENE-7023.
>
>
> : Its super-easy to use a short hash here and avoid problems.
>
>
> There is however an actual and practical downside to switching our
> implementation version to using a "short" SHA, and that's that we would
> lose the ability to garuntee that the information in the
> Implementation-Version uniquely identifies what commit a given jar was
> built from. Multiple commits with the same short(end) hash are possible
> -- Multiple commits with identical (full) commits is not.
>
> Folks may think that using git tags is useful enough for figuring this
> out from official releases, but being able to look at the jar metadata
> from arbitrary builds off of arbitrary branches and sanity check where
> exactly they come from has been very useful to me for 10+ years.
>
>
> : On Thu, Sep 16, 2021 at 3:03 AM Dawid Weiss <dawid.weiss@gmail.com <mailto:dawid.weiss@gmail.com>> wrote:
> : >
> : > Jar command doesn't have it, true. But it's fairly trivial to do, even
> : > with an inline snippet like this one?
> : >
> : > public class PrintManifest {
> : > public static void main(String[] jars) throws IOException {
> : > for (var jar : jars) {
> : > var manifest = new JarFile(Paths.get(jar).toFile()).getManifest();
> : > var attrs = manifest.getMainAttributes();
> : > System.out.println(jar + ": " + attrs.getValue("Implementation-Version"));
> : > }
> : > }
> : > }
> : >
> : > I have this in my lucene-core-9.0.0-SNAPSHOT.jar:
> : >
> : > Implementation-Version: 9.0.0-SNAPSHOT de45b68c909815ce5ea7b6b9e1a2ce337
> : > 5b6334d [snapshot build, details omitted]
> : >
> : > and running:
> : >
> : > java PrintManifest.java lucene-core-9.0.0-SNAPSHOT.jar
> : >
> : > shows:
> : >
> : > lucene-core-9.0.0-SNAPSHOT.jar: 9.0.0-SNAPSHOT
> : > de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details
> : > omitted]
> : >
> : > This seems easier to me than trying to remember and keep the length of
> : > that line shorter than an arbitrary limit.
> : >
> : > Dawid
> : >
> : >
> : > On Wed, Sep 15, 2021 at 9:46 PM Robert Muir <rcmuir@gmail.com <mailto:rcmuir@gmail.com>> wrote:
> : > >
> : > > But its irrelevant that is "valid" when virtually no tools match it.
> : > >
> : > > In other words, I'd agree with you if the "jar" command had some
> : > > ability to read these manifests and print stuff to stdout, e.g. if
> : > > there was ANY interop at all here.
> : > >
> : > > But there isn't. So IMO it makes no sense to cause confusion and chaos
> : > > by adding an unnecessarily long git commit hash.
> : > >
> : > > On Wed, Sep 15, 2021 at 3:26 PM Dawid Weiss <dawid.weiss@gmail.com <mailto:dawid.weiss@gmail.com>> wrote:
> : > > >
> : > > >
> : > > > This is valid manifest line-breaking though... Can we read the manifest properly on the smoke tester side somehow (for example, run a Java process that reads and extracts the required attribute)? This way we wouldn't care about the implementation details of how manifest wraps the lines (or escapes characters).
> : > > >
> : > > > D.
> : > > >
> : > > > On Wed, Sep 15, 2021 at 8:46 PM Mike Drob <mdrob@mdrob.com <mailto:mdrob@mdrob.com>> wrote:
> : > > >>
> : > > >> The benchmark jar has the info we need… sort of. When I built it, it has:
> : > > >>
> : > > >> Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
> : > > >> 1 - mdrob - 2021-09-15 11:40:36
> : > > >>
> : > > >>
> : > > >> and it’s looking for Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
> : > > >>
> : > > >> Because 8.10 is a character longer than 8.9, we happen to wrap the last character of the git commit sha. From the manifest spec:
> : > > >>
> : > > >> No line may be longer than 72 bytes (not characters), in its UTF8-encoded form.
> : > > >> If a value would make the initial line longer than this, it should be continued
> : > > >> on extra lines (each starting with a single SPACE).
> : > > >>
> : > > >> And we were already teetering on the edge of that limit. We'll run into this problem again in a few years when we try to release version 10.0.0, so solving it now has practical benefits down the line.
> : > > >>
> : > > >> There's a few options that I can come up with -
> : > > >> 1. Use the short-hash when we generate the jar
> : > > >> 2. Use the short-hash when we check the contents in the smoke test
> : > > >> 3. Do some line join magic in the smoke test.
> : > > >>
> : > > >> I'm leaning towards number 1 as I feel that would still be unique enough for our needs, but would like to hear from others as well.
> : > > >>
> : > > >> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <thelabdude@gmail.com <mailto:thelabdude@gmail.com>> wrote:
> : > > >>>
> : > > >>> can someone also please look into that benchmark jar issue?
> : > > >>>
> : > > >>> Sent from my iPhone
> : > > >>>
> : > > >>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co <mailto:nhat.nguyen@elastic.co>.invalid> wrote:
> : > > >>>
> : > > >>> ?
> : > > >>> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
> : > > >>>
> : > > >>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com <mailto:mdrob@mdrob.com>> wrote:
> : > > >>>>
> : > > >>>> I think since Tim is out on vacation, it's probably not too late. That looks like a good fix to have, do we know how long the bug has been present?
> : > > >>>>
> : > > >>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova <mayya.sharipova@elastic.co <mailto:mayya.sharipova@elastic.co>.invalid> wrote:
> : > > >>>>>
> : > > >>>>> Hello everyone,
> : > > >>>>> We have discovered a bug and fixed a bug in Lucene sort optimization (LUCENE-10106) and would like to merge it to Lucene 8.10 if it is not too late.
> : > > >>>>> I apologize for the inconvenience, the bug was discovered just yesterday.
> : > > >>>>>
> : > > >>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org <mailto:thelabdude@apache.org>> wrote:
> : > > >>>>>>
> : > > >>>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
> : > > >>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
> : > > >>>>>> unless someone else wants to pick up the RM duties before then?
> : > > >>>>>>
> : > > >>>>>> After failing the test suite at various places and other weirdness
> : > > >>>>>> like .asc files not getting created, I finally got to the smoke test
> : > > >>>>>> part, which is now failing with:
> : > > >>>>>>
> : > > >>>>>> File "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
> : > > >>>>>> line 176, in checkJARMetaData
> : > > >>>>>> raise RuntimeError('%s is missing "%s" inside its
> : > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
> : > > >>>>>> RuntimeError: JAR file
> : > > >>>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
> : > > >>>>>> is missing "Implementation-Version: 8.10.0
> : > > >>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
> : > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)
> : > > >>>>>>
> : > > >>>>>> FWIW, I verified that the other Lucene JAR files have this line in
> : > > >>>>>> them, such as core:
> : > > >>>>>>
> : > > >>>>>> Manifest-Version: 1.0
> : > > >>>>>> Ant-Version: Apache Ant 1.9.15
> : > > >>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
> : > > >>>>>> Extension-Name: org.apache.lucene
> : > > >>>>>> Specification-Title: Lucene Search Engine: core
> : > > >>>>>> Specification-Version: 8.10.0
> : > > >>>>>> Specification-Vendor: The Apache Software Foundation
> : > > >>>>>> Implementation-Title: org.apache.lucene
> : > > >>>>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
> : > > >>>>>> 7 - tjp - 2021-09-14 19:08:42
> : > > >>>>>> Implementation-Vendor: The Apache Software Foundation
> : > > >>>>>> X-Compile-Source-JDK: 8
> : > > >>>>>> X-Compile-Target-JDK: 8
> : > > >>>>>> Multi-Release: true
> : > > >>>>>>
> : > > >>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
> : > > >>>>>> <ichattopadhyaya@gmail.com <mailto:ichattopadhyaya@gmail.com>> wrote:
> : > > >>>>>> >
> : > > >>>>>> > All the best, this is the worst step.
> : > > >>>>>> >
> : > > >>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com <mailto:thelabdude@gmail.com>> wrote:
> : > > >>>>>> >>
> : > > >>>>>> >> Building RC1 now ... stay tuned.
> : > > >>>>>> >>
> : > > >>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com <mailto:thelabdude@gmail.com>> wrote:
> : > > >>>>>> >> >
> : > > >>>>>> >> > Thanks for the update Mike!
> : > > >>>>>> >> >
> : > > >>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
> : > > >>>>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
> : > > >>>>>> >> > reported on Slack earlier today.
> : > > >>>>>> >> >
> : > > >>>>>> >> > Cheers,
> : > > >>>>>> >> > Tim
> : > > >>>>>> >> >
> : > > >>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org <mailto:mdrob@apache.org>> wrote:
> : > > >>>>>> >> > >
> : > > >>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
> : > > >>>>>> >> > > both look pretty good, but I've got a few last unit tests that I need
> : > > >>>>>> >> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
> : > > >>>>>> >> > > sure to keep you updated though.
> : > > >>>>>> >> > >
> : > > >>>>>> >> > >
> : > > >>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com <mailto:thelabdude@gmail.com>> wrote:
> : > > >>>>>> >> > > >
> : > > >>>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 <https://issues.apache.org/jira/browse/SOLR-15620> while testing
> : > > >>>>>> >> > > > the schema designer. I haven't built the RC yet, so going to see if I
> : > > >>>>>> >> > > > can get this in today.
> : > > >>>>>> >> > > >
> : > > >>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org <mailto:thelabdude@apache.org>> wrote:
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > NOTICE:
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > Please observe the normal rules:
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > * No new features may be committed to the branch.
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > * Documentation patches, build patches and serious bug fixes may be
> : > > >>>>>> >> > > > > committed to the branch. However, you should submit all patches you
> : > > >>>>>> >> > > > > want to commit to Jira first to give others the chance to review
> : > > >>>>>> >> > > > > and possibly vote against the patch. Keep in mind that it is our
> : > > >>>>>> >> > > > > main intention to keep the branch as stable as possible.
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > * All patches that are intended for the branch should first be committed
> : > > >>>>>> >> > > > > to the unstable branch, merged into the stable branch, and then into
> : > > >>>>>> >> > > > > the current release branch.
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > * Normal unstable and stable branch development may continue as usual.
> : > > >>>>>> >> > > > > However, if you plan to commit a big change to the unstable branch
> : > > >>>>>> >> > > > > while the branch feature freeze is in effect, think twice: can't the
> : > > >>>>>> >> > > > > addition wait a couple more days? Merges of bug fixes into the branch
> : > > >>>>>> >> > > > > may become more difficult.
> : > > >>>>>> >> > > > >
> : > > >>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
> : > > >>>>>> >> > > > > a release candidate build.
> : > > >>>>>> >> > > > > ----
> : > > >>>>>> >> > > >
> : > > >>>>>> >> > > > ---------------------------------------------------------------------
> : > > >>>>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
> : > > >>>>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>
> : > > >>>>>> >> > > >
> : > > >>>>>> >> > >
> : > > >>>>>> >> > > ---------------------------------------------------------------------
> : > > >>>>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
> : > > >>>>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>
> : > > >>>>>> >> > >
> : > > >>>>>> >>
> : > > >>>>>> >> ---------------------------------------------------------------------
> : > > >>>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org <mailto:dev-unsubscribe@solr.apache.org>
> : > > >>>>>> >> For additional commands, e-mail: dev-help@solr.apache.org <mailto:dev-help@solr.apache.org>
> : > > >>>>>> >>
> : > > >>>>>>
> : > > >>>>>> ---------------------------------------------------------------------
> : > > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
> : > > >>>>>> For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>
> : > > >>>>>>
> : > >
> : > > ---------------------------------------------------------------------
> : > > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org <mailto:dev-unsubscribe@solr.apache.org>
> : > > For additional commands, e-mail: dev-help@solr.apache.org <mailto:dev-help@solr.apache.org>
> : > >
> : >
> : > ---------------------------------------------------------------------
> : > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
> : > For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>
> : >
> :
> : ---------------------------------------------------------------------
> : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
> : For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>
> :
> :
>
> -Hoss
> http://www.lucidworks.com/ <http://www.lucidworks.com/>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org <mailto:dev-unsubscribe@solr.apache.org>
> For additional commands, e-mail: dev-help@solr.apache.org <mailto:dev-help@solr.apache.org>
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
Sure, but that package is archived/read-only, GPLv3. with 3 watchers and 1 star.

On Fri, Sep 17, 2021 at 4:27 AM Jan Høydahl <jan.asf@cominvent.com> wrote:
>
> Let's just follow the spec and move on.
>
> Just tested this python package, which has no problem parsing the problematic manifest https://pypi.org/project/jarmanifest/
>
> [{'implementationversion': '9.0.0-SNAPSHOT de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details omitted]'}]
>
> Jan
>
> 17. sep. 2021 kl. 09:32 skrev Dawid Weiss <dawid.weiss@gmail.com>:
>
>
> We could do a few things to keep everyone happy -
>
> 1) keep abbreviated hash in the Implementat-Version and use a separate manifest entry to store a full hash.
> 2) use a longer version for git show (abbrev=num) so that the chance of collisions in the future is minimized. It's still not a full hash but a
> long(er) forced prefix.
>
> D.
>
> On Fri, Sep 17, 2021 at 12:21 AM Chris Hostetter <hossman_lucene@fucit.org> wrote:
>>
>>
>> : I was referring to doing this with languages other than java.
>> :
>> : I'm also assuming that exceeding this limit is going to cause indirect
>> : hassles for users of lucene, e.g. breaking various security / supply
>> : chain tools. We know a lot of these are total crap but people in the
>> : corporate world have to suffer under them.
>>
>> Just to be clear -- our 'Implementation-Version:' has been exceeding the
>> 72 byte "single line" limit for a LOOOOONG time -- worrying about how
>> "security / supply chain" tools will handle parsing that line now seems
>> kind of silly...
>>
>> If tools can't handle a line wrap in the 8.10 jars, then they haven't
>> been able to handle the line wrap since we switched from svn to git (when
>> the Implementation Version values switched from being based svn version#
>> to git sha)
>>
>> The *ONLY* thing that's new here is where in the value the line wrap
>> happens (with 8.10.0 it happens in the middle of the SHA) and that our
>> smoketest tool isn't smart enough to parse the values properly.
>>
>> This is not even the first time we've even had a conversation about the
>> smoke tester and Implementation Version line wraps: LUCENE-7023.
>>
>>
>> : Its super-easy to use a short hash here and avoid problems.
>>
>>
>> There is however an actual and practical downside to switching our
>> implementation version to using a "short" SHA, and that's that we would
>> lose the ability to garuntee that the information in the
>> Implementation-Version uniquely identifies what commit a given jar was
>> built from. Multiple commits with the same short(end) hash are possible
>> -- Multiple commits with identical (full) commits is not.
>>
>> Folks may think that using git tags is useful enough for figuring this
>> out from official releases, but being able to look at the jar metadata
>> from arbitrary builds off of arbitrary branches and sanity check where
>> exactly they come from has been very useful to me for 10+ years.
>>
>>
>> : On Thu, Sep 16, 2021 at 3:03 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>> : >
>> : > Jar command doesn't have it, true. But it's fairly trivial to do, even
>> : > with an inline snippet like this one?
>> : >
>> : > public class PrintManifest {
>> : > public static void main(String[] jars) throws IOException {
>> : > for (var jar : jars) {
>> : > var manifest = new JarFile(Paths.get(jar).toFile()).getManifest();
>> : > var attrs = manifest.getMainAttributes();
>> : > System.out.println(jar + ": " + attrs.getValue("Implementation-Version"));
>> : > }
>> : > }
>> : > }
>> : >
>> : > I have this in my lucene-core-9.0.0-SNAPSHOT.jar:
>> : >
>> : > Implementation-Version: 9.0.0-SNAPSHOT de45b68c909815ce5ea7b6b9e1a2ce337
>> : > 5b6334d [snapshot build, details omitted]
>> : >
>> : > and running:
>> : >
>> : > java PrintManifest.java lucene-core-9.0.0-SNAPSHOT.jar
>> : >
>> : > shows:
>> : >
>> : > lucene-core-9.0.0-SNAPSHOT.jar: 9.0.0-SNAPSHOT
>> : > de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details
>> : > omitted]
>> : >
>> : > This seems easier to me than trying to remember and keep the length of
>> : > that line shorter than an arbitrary limit.
>> : >
>> : > Dawid
>> : >
>> : >
>> : > On Wed, Sep 15, 2021 at 9:46 PM Robert Muir <rcmuir@gmail.com> wrote:
>> : > >
>> : > > But its irrelevant that is "valid" when virtually no tools match it.
>> : > >
>> : > > In other words, I'd agree with you if the "jar" command had some
>> : > > ability to read these manifests and print stuff to stdout, e.g. if
>> : > > there was ANY interop at all here.
>> : > >
>> : > > But there isn't. So IMO it makes no sense to cause confusion and chaos
>> : > > by adding an unnecessarily long git commit hash.
>> : > >
>> : > > On Wed, Sep 15, 2021 at 3:26 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>> : > > >
>> : > > >
>> : > > > This is valid manifest line-breaking though... Can we read the manifest properly on the smoke tester side somehow (for example, run a Java process that reads and extracts the required attribute)? This way we wouldn't care about the implementation details of how manifest wraps the lines (or escapes characters).
>> : > > >
>> : > > > D.
>> : > > >
>> : > > > On Wed, Sep 15, 2021 at 8:46 PM Mike Drob <mdrob@mdrob.com> wrote:
>> : > > >>
>> : > > >> The benchmark jar has the info we need… sort of. When I built it, it has:
>> : > > >>
>> : > > >> Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
>> : > > >> 1 - mdrob - 2021-09-15 11:40:36
>> : > > >>
>> : > > >>
>> : > > >> and it’s looking for Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
>> : > > >>
>> : > > >> Because 8.10 is a character longer than 8.9, we happen to wrap the last character of the git commit sha. From the manifest spec:
>> : > > >>
>> : > > >> No line may be longer than 72 bytes (not characters), in its UTF8-encoded form.
>> : > > >> If a value would make the initial line longer than this, it should be continued
>> : > > >> on extra lines (each starting with a single SPACE).
>> : > > >>
>> : > > >> And we were already teetering on the edge of that limit. We'll run into this problem again in a few years when we try to release version 10.0.0, so solving it now has practical benefits down the line.
>> : > > >>
>> : > > >> There's a few options that I can come up with -
>> : > > >> 1. Use the short-hash when we generate the jar
>> : > > >> 2. Use the short-hash when we check the contents in the smoke test
>> : > > >> 3. Do some line join magic in the smoke test.
>> : > > >>
>> : > > >> I'm leaning towards number 1 as I feel that would still be unique enough for our needs, but would like to hear from others as well.
>> : > > >>
>> : > > >> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <thelabdude@gmail.com> wrote:
>> : > > >>>
>> : > > >>> can someone also please look into that benchmark jar issue?
>> : > > >>>
>> : > > >>> Sent from my iPhone
>> : > > >>>
>> : > > >>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co.invalid> wrote:
>> : > > >>>
>> : > > >>> ?
>> : > > >>> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
>> : > > >>>
>> : > > >>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com> wrote:
>> : > > >>>>
>> : > > >>>> I think since Tim is out on vacation, it's probably not too late. That looks like a good fix to have, do we know how long the bug has been present?
>> : > > >>>>
>> : > > >>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova <mayya.sharipova@elastic.co.invalid> wrote:
>> : > > >>>>>
>> : > > >>>>> Hello everyone,
>> : > > >>>>> We have discovered a bug and fixed a bug in Lucene sort optimization (LUCENE-10106) and would like to merge it to Lucene 8.10 if it is not too late.
>> : > > >>>>> I apologize for the inconvenience, the bug was discovered just yesterday.
>> : > > >>>>>
>> : > > >>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org> wrote:
>> : > > >>>>>>
>> : > > >>>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
>> : > > >>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
>> : > > >>>>>> unless someone else wants to pick up the RM duties before then?
>> : > > >>>>>>
>> : > > >>>>>> After failing the test suite at various places and other weirdness
>> : > > >>>>>> like .asc files not getting created, I finally got to the smoke test
>> : > > >>>>>> part, which is now failing with:
>> : > > >>>>>>
>> : > > >>>>>> File "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
>> : > > >>>>>> line 176, in checkJARMetaData
>> : > > >>>>>> raise RuntimeError('%s is missing "%s" inside its
>> : > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
>> : > > >>>>>> RuntimeError: JAR file
>> : > > >>>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
>> : > > >>>>>> is missing "Implementation-Version: 8.10.0
>> : > > >>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
>> : > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)
>> : > > >>>>>>
>> : > > >>>>>> FWIW, I verified that the other Lucene JAR files have this line in
>> : > > >>>>>> them, such as core:
>> : > > >>>>>>
>> : > > >>>>>> Manifest-Version: 1.0
>> : > > >>>>>> Ant-Version: Apache Ant 1.9.15
>> : > > >>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
>> : > > >>>>>> Extension-Name: org.apache.lucene
>> : > > >>>>>> Specification-Title: Lucene Search Engine: core
>> : > > >>>>>> Specification-Version: 8.10.0
>> : > > >>>>>> Specification-Vendor: The Apache Software Foundation
>> : > > >>>>>> Implementation-Title: org.apache.lucene
>> : > > >>>>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
>> : > > >>>>>> 7 - tjp - 2021-09-14 19:08:42
>> : > > >>>>>> Implementation-Vendor: The Apache Software Foundation
>> : > > >>>>>> X-Compile-Source-JDK: 8
>> : > > >>>>>> X-Compile-Target-JDK: 8
>> : > > >>>>>> Multi-Release: true
>> : > > >>>>>>
>> : > > >>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
>> : > > >>>>>> <ichattopadhyaya@gmail.com> wrote:
>> : > > >>>>>> >
>> : > > >>>>>> > All the best, this is the worst step.
>> : > > >>>>>> >
>> : > > >>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com> wrote:
>> : > > >>>>>> >>
>> : > > >>>>>> >> Building RC1 now ... stay tuned.
>> : > > >>>>>> >>
>> : > > >>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com> wrote:
>> : > > >>>>>> >> >
>> : > > >>>>>> >> > Thanks for the update Mike!
>> : > > >>>>>> >> >
>> : > > >>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
>> : > > >>>>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
>> : > > >>>>>> >> > reported on Slack earlier today.
>> : > > >>>>>> >> >
>> : > > >>>>>> >> > Cheers,
>> : > > >>>>>> >> > Tim
>> : > > >>>>>> >> >
>> : > > >>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
>> : > > >>>>>> >> > >
>> : > > >>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
>> : > > >>>>>> >> > > both look pretty good, but I've got a few last unit tests that I need
>> : > > >>>>>> >> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
>> : > > >>>>>> >> > > sure to keep you updated though.
>> : > > >>>>>> >> > >
>> : > > >>>>>> >> > >
>> : > > >>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com> wrote:
>> : > > >>>>>> >> > > >
>> : > > >>>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
>> : > > >>>>>> >> > > > the schema designer. I haven't built the RC yet, so going to see if I
>> : > > >>>>>> >> > > > can get this in today.
>> : > > >>>>>> >> > > >
>> : > > >>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
>> : > > >>>>>> >> > > > >
>> : > > >>>>>> >> > > > > NOTICE:
>> : > > >>>>>> >> > > > >
>> : > > >>>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
>> : > > >>>>>> >> > > > >
>> : > > >>>>>> >> > > > > Please observe the normal rules:
>> : > > >>>>>> >> > > > >
>> : > > >>>>>> >> > > > > * No new features may be committed to the branch.
>> : > > >>>>>> >> > > > >
>> : > > >>>>>> >> > > > > * Documentation patches, build patches and serious bug fixes may be
>> : > > >>>>>> >> > > > > committed to the branch. However, you should submit all patches you
>> : > > >>>>>> >> > > > > want to commit to Jira first to give others the chance to review
>> : > > >>>>>> >> > > > > and possibly vote against the patch. Keep in mind that it is our
>> : > > >>>>>> >> > > > > main intention to keep the branch as stable as possible.
>> : > > >>>>>> >> > > > >
>> : > > >>>>>> >> > > > > * All patches that are intended for the branch should first be committed
>> : > > >>>>>> >> > > > > to the unstable branch, merged into the stable branch, and then into
>> : > > >>>>>> >> > > > > the current release branch.
>> : > > >>>>>> >> > > > >
>> : > > >>>>>> >> > > > > * Normal unstable and stable branch development may continue as usual.
>> : > > >>>>>> >> > > > > However, if you plan to commit a big change to the unstable branch
>> : > > >>>>>> >> > > > > while the branch feature freeze is in effect, think twice: can't the
>> : > > >>>>>> >> > > > > addition wait a couple more days? Merges of bug fixes into the branch
>> : > > >>>>>> >> > > > > may become more difficult.
>> : > > >>>>>> >> > > > >
>> : > > >>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
>> : > > >>>>>> >> > > > > a release candidate build.
>> : > > >>>>>> >> > > > > ----
>> : > > >>>>>> >> > > >
>> : > > >>>>>> >> > > > ---------------------------------------------------------------------
>> : > > >>>>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> : > > >>>>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
>> : > > >>>>>> >> > > >
>> : > > >>>>>> >> > >
>> : > > >>>>>> >> > > ---------------------------------------------------------------------
>> : > > >>>>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> : > > >>>>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
>> : > > >>>>>> >> > >
>> : > > >>>>>> >>
>> : > > >>>>>> >> ---------------------------------------------------------------------
>> : > > >>>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> : > > >>>>>> >> For additional commands, e-mail: dev-help@solr.apache.org
>> : > > >>>>>> >>
>> : > > >>>>>>
>> : > > >>>>>> ---------------------------------------------------------------------
>> : > > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> : > > >>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>> : > > >>>>>>
>> : > >
>> : > > ---------------------------------------------------------------------
>> : > > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> : > > For additional commands, e-mail: dev-help@solr.apache.org
>> : > >
>> : >
>> : > ---------------------------------------------------------------------
>> : > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> : > For additional commands, e-mail: dev-help@lucene.apache.org
>> : >
>> :
>> : ---------------------------------------------------------------------
>> : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> : For additional commands, e-mail: dev-help@lucene.apache.org
>> :
>> :
>>
>> -Hoss
>> http://www.lucidworks.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> For additional commands, e-mail: dev-help@solr.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: New branch and feature freeze for Lucene/Solr 8.10.0 [ In reply to ]
That was not my point, there are several choices - this from 2020 which also work ootb: https://pypi.org/project/java-manifest/
It has TWO stars :), conforms to the specs and solves our needs in pure python... If other tools cannot follow a spec then IMO that's their problem, not ours.

Jan

> 17. sep. 2021 kl. 13:46 skrev Robert Muir <rcmuir@gmail.com>:
>
> Sure, but that package is archived/read-only, GPLv3. with 3 watchers and 1 star.
>
> On Fri, Sep 17, 2021 at 4:27 AM Jan Høydahl <jan.asf@cominvent.com> wrote:
>>
>> Let's just follow the spec and move on.
>>
>> Just tested this python package, which has no problem parsing the problematic manifest https://pypi.org/project/jarmanifest/
>>
>>>>> manifest.getAttributes("/tmp/lucene-manifest.mf")
>> [{'implementationversion': '9.0.0-SNAPSHOT de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details omitted]'}]
>>
>> Jan
>>
>> 17. sep. 2021 kl. 09:32 skrev Dawid Weiss <dawid.weiss@gmail.com>:
>>
>>
>> We could do a few things to keep everyone happy -
>>
>> 1) keep abbreviated hash in the Implementat-Version and use a separate manifest entry to store a full hash.
>> 2) use a longer version for git show (abbrev=num) so that the chance of collisions in the future is minimized. It's still not a full hash but a
>> long(er) forced prefix.
>>
>> D.
>>
>> On Fri, Sep 17, 2021 at 12:21 AM Chris Hostetter <hossman_lucene@fucit.org> wrote:
>>>
>>>
>>> : I was referring to doing this with languages other than java.
>>> :
>>> : I'm also assuming that exceeding this limit is going to cause indirect
>>> : hassles for users of lucene, e.g. breaking various security / supply
>>> : chain tools. We know a lot of these are total crap but people in the
>>> : corporate world have to suffer under them.
>>>
>>> Just to be clear -- our 'Implementation-Version:' has been exceeding the
>>> 72 byte "single line" limit for a LOOOOONG time -- worrying about how
>>> "security / supply chain" tools will handle parsing that line now seems
>>> kind of silly...
>>>
>>> If tools can't handle a line wrap in the 8.10 jars, then they haven't
>>> been able to handle the line wrap since we switched from svn to git (when
>>> the Implementation Version values switched from being based svn version#
>>> to git sha)
>>>
>>> The *ONLY* thing that's new here is where in the value the line wrap
>>> happens (with 8.10.0 it happens in the middle of the SHA) and that our
>>> smoketest tool isn't smart enough to parse the values properly.
>>>
>>> This is not even the first time we've even had a conversation about the
>>> smoke tester and Implementation Version line wraps: LUCENE-7023.
>>>
>>>
>>> : Its super-easy to use a short hash here and avoid problems.
>>>
>>>
>>> There is however an actual and practical downside to switching our
>>> implementation version to using a "short" SHA, and that's that we would
>>> lose the ability to garuntee that the information in the
>>> Implementation-Version uniquely identifies what commit a given jar was
>>> built from. Multiple commits with the same short(end) hash are possible
>>> -- Multiple commits with identical (full) commits is not.
>>>
>>> Folks may think that using git tags is useful enough for figuring this
>>> out from official releases, but being able to look at the jar metadata
>>> from arbitrary builds off of arbitrary branches and sanity check where
>>> exactly they come from has been very useful to me for 10+ years.
>>>
>>>
>>> : On Thu, Sep 16, 2021 at 3:03 AM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>>> : >
>>> : > Jar command doesn't have it, true. But it's fairly trivial to do, even
>>> : > with an inline snippet like this one?
>>> : >
>>> : > public class PrintManifest {
>>> : > public static void main(String[] jars) throws IOException {
>>> : > for (var jar : jars) {
>>> : > var manifest = new JarFile(Paths.get(jar).toFile()).getManifest();
>>> : > var attrs = manifest.getMainAttributes();
>>> : > System.out.println(jar + ": " + attrs.getValue("Implementation-Version"));
>>> : > }
>>> : > }
>>> : > }
>>> : >
>>> : > I have this in my lucene-core-9.0.0-SNAPSHOT.jar:
>>> : >
>>> : > Implementation-Version: 9.0.0-SNAPSHOT de45b68c909815ce5ea7b6b9e1a2ce337
>>> : > 5b6334d [snapshot build, details omitted]
>>> : >
>>> : > and running:
>>> : >
>>> : > java PrintManifest.java lucene-core-9.0.0-SNAPSHOT.jar
>>> : >
>>> : > shows:
>>> : >
>>> : > lucene-core-9.0.0-SNAPSHOT.jar: 9.0.0-SNAPSHOT
>>> : > de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details
>>> : > omitted]
>>> : >
>>> : > This seems easier to me than trying to remember and keep the length of
>>> : > that line shorter than an arbitrary limit.
>>> : >
>>> : > Dawid
>>> : >
>>> : >
>>> : > On Wed, Sep 15, 2021 at 9:46 PM Robert Muir <rcmuir@gmail.com> wrote:
>>> : > >
>>> : > > But its irrelevant that is "valid" when virtually no tools match it.
>>> : > >
>>> : > > In other words, I'd agree with you if the "jar" command had some
>>> : > > ability to read these manifests and print stuff to stdout, e.g. if
>>> : > > there was ANY interop at all here.
>>> : > >
>>> : > > But there isn't. So IMO it makes no sense to cause confusion and chaos
>>> : > > by adding an unnecessarily long git commit hash.
>>> : > >
>>> : > > On Wed, Sep 15, 2021 at 3:26 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>>> : > > >
>>> : > > >
>>> : > > > This is valid manifest line-breaking though... Can we read the manifest properly on the smoke tester side somehow (for example, run a Java process that reads and extracts the required attribute)? This way we wouldn't care about the implementation details of how manifest wraps the lines (or escapes characters).
>>> : > > >
>>> : > > > D.
>>> : > > >
>>> : > > > On Wed, Sep 15, 2021 at 8:46 PM Mike Drob <mdrob@mdrob.com> wrote:
>>> : > > >>
>>> : > > >> The benchmark jar has the info we need… sort of. When I built it, it has:
>>> : > > >>
>>> : > > >> Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
>>> : > > >> 1 - mdrob - 2021-09-15 11:40:36
>>> : > > >>
>>> : > > >>
>>> : > > >> and it’s looking for Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
>>> : > > >>
>>> : > > >> Because 8.10 is a character longer than 8.9, we happen to wrap the last character of the git commit sha. From the manifest spec:
>>> : > > >>
>>> : > > >> No line may be longer than 72 bytes (not characters), in its UTF8-encoded form.
>>> : > > >> If a value would make the initial line longer than this, it should be continued
>>> : > > >> on extra lines (each starting with a single SPACE).
>>> : > > >>
>>> : > > >> And we were already teetering on the edge of that limit. We'll run into this problem again in a few years when we try to release version 10.0.0, so solving it now has practical benefits down the line.
>>> : > > >>
>>> : > > >> There's a few options that I can come up with -
>>> : > > >> 1. Use the short-hash when we generate the jar
>>> : > > >> 2. Use the short-hash when we check the contents in the smoke test
>>> : > > >> 3. Do some line join magic in the smoke test.
>>> : > > >>
>>> : > > >> I'm leaning towards number 1 as I feel that would still be unique enough for our needs, but would like to hear from others as well.
>>> : > > >>
>>> : > > >> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <thelabdude@gmail.com> wrote:
>>> : > > >>>
>>> : > > >>> can someone also please look into that benchmark jar issue?
>>> : > > >>>
>>> : > > >>> Sent from my iPhone
>>> : > > >>>
>>> : > > >>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <nhat.nguyen@elastic.co.invalid> wrote:
>>> : > > >>>
>>> : > > >>> ?
>>> : > > >>> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
>>> : > > >>>
>>> : > > >>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <mdrob@mdrob.com> wrote:
>>> : > > >>>>
>>> : > > >>>> I think since Tim is out on vacation, it's probably not too late. That looks like a good fix to have, do we know how long the bug has been present?
>>> : > > >>>>
>>> : > > >>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova <mayya.sharipova@elastic.co.invalid> wrote:
>>> : > > >>>>>
>>> : > > >>>>> Hello everyone,
>>> : > > >>>>> We have discovered a bug and fixed a bug in Lucene sort optimization (LUCENE-10106) and would like to merge it to Lucene 8.10 if it is not too late.
>>> : > > >>>>> I apologize for the inconvenience, the bug was discovered just yesterday.
>>> : > > >>>>>
>>> : > > >>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <thelabdude@apache.org> wrote:
>>> : > > >>>>>>
>>> : > > >>>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
>>> : > > >>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
>>> : > > >>>>>> unless someone else wants to pick up the RM duties before then?
>>> : > > >>>>>>
>>> : > > >>>>>> After failing the test suite at various places and other weirdness
>>> : > > >>>>>> like .asc files not getting created, I finally got to the smoke test
>>> : > > >>>>>> part, which is now failing with:
>>> : > > >>>>>>
>>> : > > >>>>>> File "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
>>> : > > >>>>>> line 176, in checkJARMetaData
>>> : > > >>>>>> raise RuntimeError('%s is missing "%s" inside its
>>> : > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
>>> : > > >>>>>> RuntimeError: JAR file
>>> : > > >>>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
>>> : > > >>>>>> is missing "Implementation-Version: 8.10.0
>>> : > > >>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
>>> : > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)
>>> : > > >>>>>>
>>> : > > >>>>>> FWIW, I verified that the other Lucene JAR files have this line in
>>> : > > >>>>>> them, such as core:
>>> : > > >>>>>>
>>> : > > >>>>>> Manifest-Version: 1.0
>>> : > > >>>>>> Ant-Version: Apache Ant 1.9.15
>>> : > > >>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
>>> : > > >>>>>> Extension-Name: org.apache.lucene
>>> : > > >>>>>> Specification-Title: Lucene Search Engine: core
>>> : > > >>>>>> Specification-Version: 8.10.0
>>> : > > >>>>>> Specification-Vendor: The Apache Software Foundation
>>> : > > >>>>>> Implementation-Title: org.apache.lucene
>>> : > > >>>>>> Implementation-Version: 8.10.0 ecf5c747e6df418dd05a18af327c20051f0584d
>>> : > > >>>>>> 7 - tjp - 2021-09-14 19:08:42
>>> : > > >>>>>> Implementation-Vendor: The Apache Software Foundation
>>> : > > >>>>>> X-Compile-Source-JDK: 8
>>> : > > >>>>>> X-Compile-Target-JDK: 8
>>> : > > >>>>>> Multi-Release: true
>>> : > > >>>>>>
>>> : > > >>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
>>> : > > >>>>>> <ichattopadhyaya@gmail.com> wrote:
>>> : > > >>>>>> >
>>> : > > >>>>>> > All the best, this is the worst step.
>>> : > > >>>>>> >
>>> : > > >>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, <thelabdude@gmail.com> wrote:
>>> : > > >>>>>> >>
>>> : > > >>>>>> >> Building RC1 now ... stay tuned.
>>> : > > >>>>>> >>
>>> : > > >>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter <thelabdude@gmail.com> wrote:
>>> : > > >>>>>> >> >
>>> : > > >>>>>> >> > Thanks for the update Mike!
>>> : > > >>>>>> >> >
>>> : > > >>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
>>> : > > >>>>>> >> > SOLR-15621, which looks like an easy win for the issue Cassandra
>>> : > > >>>>>> >> > reported on Slack earlier today.
>>> : > > >>>>>> >> >
>>> : > > >>>>>> >> > Cheers,
>>> : > > >>>>>> >> > Tim
>>> : > > >>>>>> >> >
>>> : > > >>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <mdrob@apache.org> wrote:
>>> : > > >>>>>> >> > >
>>> : > > >>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and benchmarking
>>> : > > >>>>>> >> > > both look pretty good, but I've got a few last unit tests that I need
>>> : > > >>>>>> >> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
>>> : > > >>>>>> >> > > sure to keep you updated though.
>>> : > > >>>>>> >> > >
>>> : > > >>>>>> >> > >
>>> : > > >>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter <thelabdude@gmail.com> wrote:
>>> : > > >>>>>> >> > > >
>>> : > > >>>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
>>> : > > >>>>>> >> > > > the schema designer. I haven't built the RC yet, so going to see if I
>>> : > > >>>>>> >> > > > can get this in today.
>>> : > > >>>>>> >> > > >
>>> : > > >>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <thelabdude@apache.org> wrote:
>>> : > > >>>>>> >> > > > >
>>> : > > >>>>>> >> > > > > NOTICE:
>>> : > > >>>>>> >> > > > >
>>> : > > >>>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on stable branch.
>>> : > > >>>>>> >> > > > >
>>> : > > >>>>>> >> > > > > Please observe the normal rules:
>>> : > > >>>>>> >> > > > >
>>> : > > >>>>>> >> > > > > * No new features may be committed to the branch.
>>> : > > >>>>>> >> > > > >
>>> : > > >>>>>> >> > > > > * Documentation patches, build patches and serious bug fixes may be
>>> : > > >>>>>> >> > > > > committed to the branch. However, you should submit all patches you
>>> : > > >>>>>> >> > > > > want to commit to Jira first to give others the chance to review
>>> : > > >>>>>> >> > > > > and possibly vote against the patch. Keep in mind that it is our
>>> : > > >>>>>> >> > > > > main intention to keep the branch as stable as possible.
>>> : > > >>>>>> >> > > > >
>>> : > > >>>>>> >> > > > > * All patches that are intended for the branch should first be committed
>>> : > > >>>>>> >> > > > > to the unstable branch, merged into the stable branch, and then into
>>> : > > >>>>>> >> > > > > the current release branch.
>>> : > > >>>>>> >> > > > >
>>> : > > >>>>>> >> > > > > * Normal unstable and stable branch development may continue as usual.
>>> : > > >>>>>> >> > > > > However, if you plan to commit a big change to the unstable branch
>>> : > > >>>>>> >> > > > > while the branch feature freeze is in effect, think twice: can't the
>>> : > > >>>>>> >> > > > > addition wait a couple more days? Merges of bug fixes into the branch
>>> : > > >>>>>> >> > > > > may become more difficult.
>>> : > > >>>>>> >> > > > >
>>> : > > >>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
>>> : > > >>>>>> >> > > > > a release candidate build.
>>> : > > >>>>>> >> > > > > ----
>>> : > > >>>>>> >> > > >
>>> : > > >>>>>> >> > > > ---------------------------------------------------------------------
>>> : > > >>>>>> >> > > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> : > > >>>>>> >> > > > For additional commands, e-mail: dev-help@lucene.apache.org
>>> : > > >>>>>> >> > > >
>>> : > > >>>>>> >> > >
>>> : > > >>>>>> >> > > ---------------------------------------------------------------------
>>> : > > >>>>>> >> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> : > > >>>>>> >> > > For additional commands, e-mail: dev-help@lucene.apache.org
>>> : > > >>>>>> >> > >
>>> : > > >>>>>> >>
>>> : > > >>>>>> >> ---------------------------------------------------------------------
>>> : > > >>>>>> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>> : > > >>>>>> >> For additional commands, e-mail: dev-help@solr.apache.org
>>> : > > >>>>>> >>
>>> : > > >>>>>>
>>> : > > >>>>>> ---------------------------------------------------------------------
>>> : > > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> : > > >>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>> : > > >>>>>>
>>> : > >
>>> : > > ---------------------------------------------------------------------
>>> : > > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>> : > > For additional commands, e-mail: dev-help@solr.apache.org
>>> : > >
>>> : >
>>> : > ---------------------------------------------------------------------
>>> : > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> : > For additional commands, e-mail: dev-help@lucene.apache.org
>>> : >
>>> :
>>> : ---------------------------------------------------------------------
>>> : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> : For additional commands, e-mail: dev-help@lucene.apache.org
>>> :
>>> :
>>>
>>> -Hoss
>>> http://www.lucidworks.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>> For additional commands, e-mail: dev-help@solr.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

1 2  View All