Rats, fat fingers and the stupid large touchpad on MBPs, which I dislike intensely. Anyway:
I was once again explaining that if you upgrade X->X+1->X+2 you need to reindex from scratch sometime at least with X+1 before upgrading to X+2, even if you re-index all documents with X+1 into the existing index when it occurred to me to wonder whether a new merge policy could make that process easier.
Would it work (we can talk about the details later) to create a merge policy that was, say, an offshoot of TMP that refused to merge any segments that were created with Lucene X-1? Or perhaps only merged segments with the _same_ marker? The idea here is that if I reindex all the docs in my index only allowing segments with the same marker to be merged, eventually all the segments created with X-1 could contain only deleted documents and magically disappear, leaving all my segments created with the current version marker. No need to start over.
Does this make any kind of sense? And TBD is whether the advice “create a new collection, reindex all your docs and use collection aliasing to switch” is sufficient (I realize I’m mixing Lucene and Solr here, but hey, I’m a Solr kind of guy).
Or am I behind the times and there’s something like this already? I suppose there’d be pesky situations like “the reindex process re-indexed all the docs except one and that one segment with that one doc prevents going to X+2”. I suppose we could get around that by some kind of command that said “delete every segment not with the most recent marker”, but now we’re getting into a lot of work whereas “create a new collection an alias” is sufficient, and it’s more effort than it’s worth..
Erick
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
I was once again explaining that if you upgrade X->X+1->X+2 you need to reindex from scratch sometime at least with X+1 before upgrading to X+2, even if you re-index all documents with X+1 into the existing index when it occurred to me to wonder whether a new merge policy could make that process easier.
Would it work (we can talk about the details later) to create a merge policy that was, say, an offshoot of TMP that refused to merge any segments that were created with Lucene X-1? Or perhaps only merged segments with the _same_ marker? The idea here is that if I reindex all the docs in my index only allowing segments with the same marker to be merged, eventually all the segments created with X-1 could contain only deleted documents and magically disappear, leaving all my segments created with the current version marker. No need to start over.
Does this make any kind of sense? And TBD is whether the advice “create a new collection, reindex all your docs and use collection aliasing to switch” is sufficient (I realize I’m mixing Lucene and Solr here, but hey, I’m a Solr kind of guy).
Or am I behind the times and there’s something like this already? I suppose there’d be pesky situations like “the reindex process re-indexed all the docs except one and that one segment with that one doc prevents going to X+2”. I suppose we could get around that by some kind of command that said “delete every segment not with the most recent marker”, but now we’re getting into a lot of work whereas “create a new collection an alias” is sufficient, and it’s more effort than it’s worth..
Erick
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org