Mailing List Archive

Re: Question for customize index segment search order
Hello, Wei.
Pardon for pinging you back to the Lucene field.
Here's the loop over segments
https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java#L674
So, presumably:
- custom searcher may loop segments out of order
- custom wrapper over index reader may yield list of child contexts in
reverse order
- some code around NTR commit may put recent segments in the beginning.
I'm not aware of any of these^ implementations, but it should be something
which is needed often.

On Fri, May 12, 2023 at 12:03?AM Wei <weiwang19@gmail.com> wrote:

> Hi ,
>
> We have a index that has multiple segments generated with continuous
> updates. There is always a large dominant segment after index rebuild,
> then many small segments are generated with continuous updates. At query
> time we apply early termination with EarlyTerminatingCollector
>
> https://github.com/apache/solr/blob/d9ddba3ac51ece953d762c796f62730e27629966/solr/core/src/java/org/apache/solr/search/EarlyTerminatingCollector.java
> ,
> which triggers EarlyTerminatingCollectorException in SolrIndexSearcher
>
> https://github.com/apache/solr/blob/d9ddba3ac51ece953d762c796f62730e27629966/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java#L281
> .
> We see a problem that the limit can be reached within the dominant segment
> alone (seems it is always traversed first) while documents with recent
> updates in the newer segments doesn't get a chance to be scored. Is it
> possible to customize the segment visiting order in Solr so that the latest
> generated segments are searched first? Any suggestion is appreciated.
>
> Thanks,
> Wei
>


--
Sincerely yours
Mikhail Khludnev
https://t.me/MUST_SEARCH
A caveat: Cyrillic!