Mailing List Archive

SNAPSHOT builds
Anyone familiar with how the lucene snapshot builds
<http://repository.apache.org/content/groups/snapshots/org/apache/lucene/>
are generated on repository.apache.org? Are they auto published somehow or
manually published? Looks like the artifacts include timestamp and build
identifier but all the modules seem to be built at different times and are
not consistent with each other (e.g., lucene-core-9.0.0
<https://repository.apache.org/content/groups/snapshots/org/apache/lucene/lucene-core/9.0.0-SNAPSHOT/>
snapshot
was recently published, but lucene-spatial-9.0.0
<https://repository.apache.org/content/groups/snapshots/org/apache/lucene/lucene-spatial/9.0.0-SNAPSHOT/>
snapshot
was last built Dec 12, 2019).

Nicholas Knize, Ph.D., GISP
Principal Engineer - Search | Amazon
Apache Lucene PMC Member and Committer
nknize@apache.org
Re: SNAPSHOT builds [ In reply to ]
Also, lucene-spatial module's last release was 8.4. So I'm confused why
there are any artifacts past 8.4 let alone one built in 2019 for 9.0.0?
Seems like all the modules should be consistent on a consistent SHA? Is
this snapshot repository used for anything?

Nicholas Knize, Ph.D., GISP
Principal Engineer - Search | Amazon
Apache Lucene PMC Member and Committer
nknize@apache.org


On Thu, Jun 3, 2021 at 5:46 PM Nicholas Knize <nknize@gmail.com> wrote:

> Anyone familiar with how the lucene snapshot builds
> <http://repository.apache.org/content/groups/snapshots/org/apache/lucene/>
> are generated on repository.apache.org? Are they auto published somehow
> or manually published? Looks like the artifacts include timestamp and build
> identifier but all the modules seem to be built at different times and are
> not consistent with each other (e.g., lucene-core-9.0.0
> <https://repository.apache.org/content/groups/snapshots/org/apache/lucene/lucene-core/9.0.0-SNAPSHOT/> snapshot
> was recently published, but lucene-spatial-9.0.0
> <https://repository.apache.org/content/groups/snapshots/org/apache/lucene/lucene-spatial/9.0.0-SNAPSHOT/> snapshot
> was last built Dec 12, 2019).
>
> Nicholas Knize, Ph.D., GISP
> Principal Engineer - Search | Amazon
> Apache Lucene PMC Member and Committer
> nknize@apache.org
>
Re: SNAPSHOT builds [ In reply to ]
This is the list of maven artifacts published from Jenkins, Nick:

https://github.com/apache/lucene/blob/main/gradle/publishing/defaults-maven.gradle#L27-L57

If something is overlooked, it should be added there.

On Fri, Jun 4, 2021 at 1:00 AM Nicholas Knize <nknize@gmail.com> wrote:

> Also, lucene-spatial module's last release was 8.4. So I'm confused why
> there are any artifacts past 8.4 let alone one built in 2019 for 9.0.0?
> Seems like all the modules should be consistent on a consistent SHA? Is
> this snapshot repository used for anything?
>
> Nicholas Knize, Ph.D., GISP
> Principal Engineer - Search | Amazon
> Apache Lucene PMC Member and Committer
> nknize@apache.org
>
>
> On Thu, Jun 3, 2021 at 5:46 PM Nicholas Knize <nknize@gmail.com> wrote:
>
>> Anyone familiar with how the lucene snapshot builds
>> <http://repository.apache.org/content/groups/snapshots/org/apache/lucene/>
>> are generated on repository.apache.org? Are they auto published somehow
>> or manually published? Looks like the artifacts include timestamp and build
>> identifier but all the modules seem to be built at different times and are
>> not consistent with each other (e.g., lucene-core-9.0.0
>> <https://repository.apache.org/content/groups/snapshots/org/apache/lucene/lucene-core/9.0.0-SNAPSHOT/> snapshot
>> was recently published, but lucene-spatial-9.0.0
>> <https://repository.apache.org/content/groups/snapshots/org/apache/lucene/lucene-spatial/9.0.0-SNAPSHOT/> snapshot
>> was last built Dec 12, 2019).
>>
>> Nicholas Knize, Ph.D., GISP
>> Principal Engineer - Search | Amazon
>> Apache Lucene PMC Member and Committer
>> nknize@apache.org
>>
>
RE: SNAPSHOT builds [ In reply to ]
Hi Nicholas,



the Snapshots are generated nightly by Jenkins:

* Main: https://ci-builds.apache.org/job/Lucene/job/Lucene-Maven-Snapshots-main/
* 8.x: https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-Maven-8.x/



About your questions: The snapshots pile up, so there is relics of outdated modules that were deleted. E.g., lucene-spatial no longer exists and was replaced by lucene-spatial3d. Because of this you see some old artifacts that were published when it was still active. As no new artifacts are pushed, the last 10 old ones per version number stay there forever.



During 8.x there is a gap because the snapshot build was broken for about 1.5 years. So the last ones were 8.3.2. It was reactivated for 8.9.0 (see https://issues.apache.org/jira/browse/LUCENE-9836).



Snapshot version numbers are always slightly different. You wont use direct version numbers. If you want to refer to a snapshot artifact you would use the -SNAPSHOT version number, whoich downloads the latest one.



For Solr it’s a bit different now, we do not rely on snapshots anymore (too uncontrollable). There we build “real” (non-snapshot) prereleases which are located on some separate repository with version numbers after the Jenkins build number. The builds are manually triggered when Solr decides to upgrade Lucene, it is not automatic: https://ci-builds.apache.org/job/Solr/job/Lucene-prerelease-main/

Artifacts are here (one repository per “release”): https://nightlies.apache.org/solr/lucene-prereleases/ (latest is jenkins build 5)



Uwe



-----

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: uwe@thetaphi.de



From: Nicholas Knize <nknize@gmail.com>
Sent: Friday, June 4, 2021 1:00 AM
To: Lucene/Solr dev <dev@lucene.apache.org>
Subject: Re: SNAPSHOT builds



Also, lucene-spatial module's last release was 8.4. So I'm confused why there are any artifacts past 8.4 let alone one built in 2019 for 9.0.0? Seems like all the modules should be consistent on a consistent SHA? Is this snapshot repository used for anything?




Nicholas Knize, Ph.D., GISP
Principal Engineer - Search | Amazon
Apache Lucene PMC Member and Committer
<mailto:nknize@apache.org> nknize@apache.org





On Thu, Jun 3, 2021 at 5:46 PM Nicholas Knize <nknize@gmail.com <mailto:nknize@gmail.com> > wrote:

Anyone familiar with how the lucene snapshot builds <http://repository.apache.org/content/groups/snapshots/org/apache/lucene/> are generated on repository.apache.org <http://repository.apache.org> ? Are they auto published somehow or manually published? Looks like the artifacts include timestamp and build identifier but all the modules seem to be built at different times and are not consistent with each other (e.g., lucene-core-9.0.0 <https://repository.apache.org/content/groups/snapshots/org/apache/lucene/lucene-core/9.0.0-SNAPSHOT/> snapshot was recently published, but lucene-spatial-9.0.0 <https://repository.apache.org/content/groups/snapshots/org/apache/lucene/lucene-spatial/9.0.0-SNAPSHOT/> snapshot was last built Dec 12, 2019).



Nicholas Knize, Ph.D., GISP
Principal Engineer - Search | Amazon
Apache Lucene PMC Member and Committer
<mailto:nknize@apache.org> nknize@apache.org
Re: SNAPSHOT builds [ In reply to ]
Wonderful and useful information. Thank you Uwe and Dawid!

Nicholas Knize, Ph.D., GISP
Principal Engineer - Search | Amazon
Apache Lucene PMC Member and Committer
nknize@apache.org


On Fri, Jun 4, 2021 at 2:42 AM Uwe Schindler <uwe@thetaphi.de> wrote:

> Hi Nicholas,
>
>
>
> the Snapshots are generated nightly by Jenkins:
>
> - Main:
> https://ci-builds.apache.org/job/Lucene/job/Lucene-Maven-Snapshots-main/
> - 8.x:
> https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-Maven-8.x/
>
>
>
> About your questions: The snapshots pile up, so there is relics of
> outdated modules that were deleted. E.g., lucene-spatial no longer exists
> and was replaced by lucene-spatial3d. Because of this you see some old
> artifacts that were published when it was still active. As no new artifacts
> are pushed, the last 10 old ones per version number stay there forever.
>
>
>
> During 8.x there is a gap because the snapshot build was broken for about
> 1.5 years. So the last ones were 8.3.2. It was reactivated for 8.9.0 (see
> https://issues.apache.org/jira/browse/LUCENE-9836).
>
>
>
> Snapshot version numbers are always slightly different. You wont use
> direct version numbers. If you want to refer to a snapshot artifact you
> would use the -SNAPSHOT version number, whoich downloads the latest one.
>
>
>
> For Solr it’s a bit different now, we do not rely on snapshots anymore
> (too uncontrollable). There we build “real” (non-snapshot) prereleases
> which are located on some separate repository with version numbers after
> the Jenkins build number. The builds are manually triggered when Solr
> decides to upgrade Lucene, it is not automatic:
> https://ci-builds.apache.org/job/Solr/job/Lucene-prerelease-main/
>
> Artifacts are here (one repository per “release”):
> https://nightlies.apache.org/solr/lucene-prereleases/ (latest is jenkins
> build 5)
>
>
>
> Uwe
>
>
>
> -----
>
> Uwe Schindler
>
> Achterdiek 19, D-28357 Bremen
>
> https://www.thetaphi.de
>
> eMail: uwe@thetaphi.de
>
>
>
> *From:* Nicholas Knize <nknize@gmail.com>
> *Sent:* Friday, June 4, 2021 1:00 AM
> *To:* Lucene/Solr dev <dev@lucene.apache.org>
> *Subject:* Re: SNAPSHOT builds
>
>
>
> Also, lucene-spatial module's last release was 8.4. So I'm confused why
> there are any artifacts past 8.4 let alone one built in 2019 for 9.0.0?
> Seems like all the modules should be consistent on a consistent SHA? Is
> this snapshot repository used for anything?
>
>
> Nicholas Knize, Ph.D., GISP
> Principal Engineer - Search | Amazon
> Apache Lucene PMC Member and Committer
> nknize@apache.org
>
>
>
>
>
> On Thu, Jun 3, 2021 at 5:46 PM Nicholas Knize <nknize@gmail.com> wrote:
>
> Anyone familiar with how the lucene snapshot builds
> <http://repository.apache.org/content/groups/snapshots/org/apache/lucene/>
> are generated on repository.apache.org? Are they auto published somehow
> or manually published? Looks like the artifacts include timestamp and build
> identifier but all the modules seem to be built at different times and are
> not consistent with each other (e.g., lucene-core-9.0.0
> <https://repository.apache.org/content/groups/snapshots/org/apache/lucene/lucene-core/9.0.0-SNAPSHOT/> snapshot
> was recently published, but lucene-spatial-9.0.0
> <https://repository.apache.org/content/groups/snapshots/org/apache/lucene/lucene-spatial/9.0.0-SNAPSHOT/> snapshot
> was last built Dec 12, 2019).
>
>
>
> Nicholas Knize, Ph.D., GISP
> Principal Engineer - Search | Amazon
> Apache Lucene PMC Member and Committer
> nknize@apache.org
>
>