Mailing List Archive

Lucene Migration query
Hi Team,

Currently I am using Lucene 7.3, I want to upgrade to lucene 8.5.1. Should
I do reindexing in this case ?
Can I make use of backward codec jar without a reindex?


Thanks & Regards,
Adarsh Sunilkumar
RE: Lucene Migration query [ In reply to ]
Hi,

> Currently I am using Lucene 7.3, I want to upgrade to lucene 8.5.1. Should
> I do reindexing in this case ?

No, you don't need that.

> Can I make use of backward codec jar without a reindex?

Yes, just add the JAR file to your classpath and it can read the indexes. Updates written to the index will use the new codecs. To force a full upgrade (rewrite all segments), invoke the IndexUpgrader class either from your code or using the command line. But this is not needed, it just makes sure that you can get rid of the backwards-codecs jar.

Uwe


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene Migration query [ In reply to ]
I think running the upgrade tool would also be necessary to set you up for
the next upgrade, when 9.0 comes along.

On Fri, Nov 20, 2020, 4:25 AM Uwe Schindler <uwe@thetaphi.de> wrote:

> Hi,
>
> > Currently I am using Lucene 7.3, I want to upgrade to lucene 8.5.1.
> Should
> > I do reindexing in this case ?
>
> No, you don't need that.
>
> > Can I make use of backward codec jar without a reindex?
>
> Yes, just add the JAR file to your classpath and it can read the indexes.
> Updates written to the index will use the new codecs. To force a full
> upgrade (rewrite all segments), invoke the IndexUpgrader class either from
> your code or using the command line. But this is not needed, it just makes
> sure that you can get rid of the backwards-codecs jar.
>
> Uwe
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: Lucene Migration query [ In reply to ]
The IndexUpgraderTool does a forceMerge(1). If you have a large index,
that has its own problems, but will work. The threshold for the issues is
5G. See: https://lucidworks.com/post/solr-and-optimizing-your-index-take-ii/
I should emphasize that if you have a very large single segment as a
result, it’ll eventually shrink if it accumulated deleted (or updated) documents,
it’ll just require a bunch of I/O amortized over time.

IndexUpgraderTool will _not_ allow you to take an index originally created with
7x to be used in 9x. (Uwe, I’ve been telling people this for a long time, if I’ve
been lying please let me know!). Starting with Lucene 6, a version is written into
each segment. Upon merge, the lowest version stamp is preserved. Lucene
will refuse to open an index where _any_ segment has a version stamp X-2 or
older.

Best,
Erick

> On Nov 20, 2020, at 7:57 AM, Michael Sokolov <msokolov@gmail.com> wrote:
>
> I think running the upgrade tool would also be necessary to set you up for
> the next upgrade, when 9.0 comes along.
>
> On Fri, Nov 20, 2020, 4:25 AM Uwe Schindler <uwe@thetaphi.de> wrote:
>
>> Hi,
>>
>>> Currently I am using Lucene 7.3, I want to upgrade to lucene 8.5.1.
>> Should
>>> I do reindexing in this case ?
>>
>> No, you don't need that.
>>
>>> Can I make use of backward codec jar without a reindex?
>>
>> Yes, just add the JAR file to your classpath and it can read the indexes.
>> Updates written to the index will use the new codecs. To force a full
>> upgrade (rewrite all segments), invoke the IndexUpgrader class either from
>> your code or using the command line. But this is not needed, it just makes
>> sure that you can get rid of the backwards-codecs jar.
>>
>> Uwe
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene Migration query [ In reply to ]
Ah, sorry for the misdirection, thanks for the correction, Erick. That
does jibe with what I now remember having heard before. I guess we
reserve the right to create index data structures in the future for
which we did not save sufficient data in the past.

On Fri, Nov 20, 2020 at 9:15 AM Erick Erickson <erickerickson@gmail.com> wrote:
>
> The IndexUpgraderTool does a forceMerge(1). If you have a large index,
> that has its own problems, but will work. The threshold for the issues is
> 5G. See: https://lucidworks.com/post/solr-and-optimizing-your-index-take-ii/
> I should emphasize that if you have a very large single segment as a
> result, it’ll eventually shrink if it accumulated deleted (or updated) documents,
> it’ll just require a bunch of I/O amortized over time.
>
> IndexUpgraderTool will _not_ allow you to take an index originally created with
> 7x to be used in 9x. (Uwe, I’ve been telling people this for a long time, if I’ve
> been lying please let me know!). Starting with Lucene 6, a version is written into
> each segment. Upon merge, the lowest version stamp is preserved. Lucene
> will refuse to open an index where _any_ segment has a version stamp X-2 or
> older.
>
> Best,
> Erick
>
> > On Nov 20, 2020, at 7:57 AM, Michael Sokolov <msokolov@gmail.com> wrote:
> >
> > I think running the upgrade tool would also be necessary to set you up for
> > the next upgrade, when 9.0 comes along.
> >
> > On Fri, Nov 20, 2020, 4:25 AM Uwe Schindler <uwe@thetaphi.de> wrote:
> >
> >> Hi,
> >>
> >>> Currently I am using Lucene 7.3, I want to upgrade to lucene 8.5.1.
> >> Should
> >>> I do reindexing in this case ?
> >>
> >> No, you don't need that.
> >>
> >>> Can I make use of backward codec jar without a reindex?
> >>
> >> Yes, just add the JAR file to your classpath and it can read the indexes.
> >> Updates written to the index will use the new codecs. To force a full
> >> upgrade (rewrite all segments), invoke the IndexUpgrader class either from
> >> your code or using the command line. But this is not needed, it just makes
> >> sure that you can get rid of the backwards-codecs jar.
> >>
> >> Uwe
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Lucene Migration Query [ In reply to ]
Hi Team,

Currently I am using Lucene 7.3, I want to upgrade to lucene 8.5.1. Should
I do reindexing in this case ?
Can I make use of backward codec jar without a reindex?



Thanks & Regards,
Adarsh Sunilkumar.
Re: Lucene Migration Query [ In reply to ]
If you created your index with 7x, you don’t need to do anything, 8x will be able to operate with it. If you ever used 6x to index any docs you must reindex completely by deleting the entire index and starting over, or index to a new collection and use collection aliasing to seamlessly switch.

Best,
Erick

> On Nov 22, 2020, at 7:48 AM, Adarsh Sunilkumar <adarshsunilkumar55@gmail.com> wrote:
>
> Hi Team,
>
> Currently I am using Lucene 7.3, I want to upgrade to lucene 8.5.1. Should
> I do reindexing in this case ?
> Can I make use of backward codec jar without a reindex?
>
>
>
> Thanks & Regards,
> Adarsh Sunilkumar.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org