Mailing List Archive

pagination with searchAfter
I’ve never used searchAfter before so looking for some tips and hints.

I understand that I need to maintain a server side cache with the relevant ScoreDocs, right?

The index is refreshed every couple of minutes. How will that affect the cached ScoreDocs?

I don’t mind too much having some inconsistencies but I want to make sure that following queries will still return reasonable results.



Thanks,

Erel
Re: pagination with searchAfter [ In reply to ]
You don't need a server-side cache as the searchAfter value has all
the information, it is just your "current position".
For example if you are sorting by ID and you return IDs 1,2,3,4,5, the
searchAfter value is basically 5.
So when you query the next time with that searchAfter=5, it skips over
any value <= to that, and returns 6,7,8.9,10.

So depending on what you are doing, refreshing the index might indeed
give some inconsistencies in pagination. But this is nothing new, you
may also see inconsistencies in pagination across refreshes with
ordinary searches that don't use searchAfter, too.

If you want to reduce these, have a look at SearcherLifetimeManager,
which provides a way to keep "older points in time" open for
consistency:
https://blog.mikemccandless.com/2011/11/searcherlifetimemanager-prevents-broken.html

On Sat, Sep 24, 2022 at 1:58 AM <erel@eth.gl> wrote:
>
> I’ve never used searchAfter before so looking for some tips and hints.
>
> I understand that I need to maintain a server side cache with the relevant ScoreDocs, right?
>
> The index is refreshed every couple of minutes. How will that affect the cached ScoreDocs?
>
> I don’t mind too much having some inconsistencies but I want to make sure that following queries will still return reasonable results.
>
>
>
> Thanks,
>
> Erel
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org