Mailing List Archive: Query Optimization in search/searchAfter

Query Optimization in search/searchAfter

Apr 11, 2024, 7:33 AM

Post #1 of 5 (39 views)

Hello,
I was wondering if a user-defined Query is optimized the same way in both
search/searchAfter provided the index stays the same (no CRUD takes place).

In searchAfter we pass in an "after" doc so I was wondering if that changes
how a query is optimized at all. By looking at the code, I'm thinking no
but was wondering if there were any other parameters here that I am not
aware of that would influence query optimization differently in
search/searchAfter. Thanks!

Re: Query Optimization in search/searchAfter [ In reply to ]

jpountz at gmail

Apr 12, 2024, 5:46 AM

Post #2 of 5 (38 views)

Permalink

Hello Puneeth,

When you pass an `after` doc, Lucene will filter out documents that compare
better than this `after` document if it can. See e.g. what LongComparator
does with its `topValue`, which is the value of the `after` doc.

On Thu, Apr 11, 2024 at 4:34?PM Puneeth Bikkumanla <pbikkumanla@gmail.com>
wrote:

> Hello,
> I was wondering if a user-defined Query is optimized the same way in both
> search/searchAfter provided the index stays the same (no CRUD takes place).
>
> In searchAfter we pass in an "after" doc so I was wondering if that changes
> how a query is optimized at all. By looking at the code, I'm thinking no
> but was wondering if there were any other parameters here that I am not
> aware of that would influence query optimization differently in
> search/searchAfter. Thanks!
>

--
Adrien

Re: Query Optimization in search/searchAfter [ In reply to ]

pbikkumanla at gmail

Apr 12, 2024, 6:36 AM

Post #3 of 5 (38 views)

Permalink

Hello,
Sorry I should have clarified what I meant by “optimized”. I am familiar
with the collector/comparators using the “after” doc to filter out
documents but I specifically was talking about the query rewriting phase.
Is the query rewritten differently in search vs searchAfter? Looking at the
code I think no but would just like to confirm if there are any edge cases
here.

On Fri, Apr 12, 2024 at 8:46 AM Adrien Grand <jpountz@gmail.com> wrote:

> Hello Puneeth,
>
> When you pass an `after` doc, Lucene will filter out documents that compare
> better than this `after` document if it can. See e.g. what LongComparator
> does with its `topValue`, which is the value of the `after` doc.
>
> On Thu, Apr 11, 2024 at 4:34?PM Puneeth Bikkumanla <pbikkumanla@gmail.com>
> wrote:
>
> > Hello,
> > I was wondering if a user-defined Query is optimized the same way in both
> > search/searchAfter provided the index stays the same (no CRUD takes
> place).
> >
> > In searchAfter we pass in an "after" doc so I was wondering if that
> changes
> > how a query is optimized at all. By looking at the code, I'm thinking no
> > but was wondering if there were any other parameters here that I am not
> > aware of that would influence query optimization differently in
> > search/searchAfter. Thanks!
> >
>
>
> --
> Adrien
>

Re: Query Optimization in search/searchAfter [ In reply to ]

jpountz at gmail

Apr 12, 2024, 6:48 AM

Post #4 of 5 (38 views)

Permalink

You are correct, query rewriting is not affected by the use of search vs.
searchAfter.

On Fri, Apr 12, 2024 at 3:37?PM Puneeth Bikkumanla <pbikkumanla@gmail.com>
wrote:

> Hello,
> Sorry I should have clarified what I meant by “optimized”. I am familiar
> with the collector/comparators using the “after” doc to filter out
> documents but I specifically was talking about the query rewriting phase.
> Is the query rewritten differently in search vs searchAfter? Looking at the
> code I think no but would just like to confirm if there are any edge cases
> here.
>
> On Fri, Apr 12, 2024 at 8:46 AM Adrien Grand <jpountz@gmail.com> wrote:
>
> > Hello Puneeth,
> >
> > When you pass an `after` doc, Lucene will filter out documents that
> compare
> > better than this `after` document if it can. See e.g. what LongComparator
> > does with its `topValue`, which is the value of the `after` doc.
> >
> > On Thu, Apr 11, 2024 at 4:34?PM Puneeth Bikkumanla <
> pbikkumanla@gmail.com>
> > wrote:
> >
> > > Hello,
> > > I was wondering if a user-defined Query is optimized the same way in
> both
> > > search/searchAfter provided the index stays the same (no CRUD takes
> > place).
> > >
> > > In searchAfter we pass in an "after" doc so I was wondering if that
> > changes
> > > how a query is optimized at all. By looking at the code, I'm thinking
> no
> > > but was wondering if there were any other parameters here that I am not
> > > aware of that would influence query optimization differently in
> > > search/searchAfter. Thanks!
> > >
> >
> >
> > --
> > Adrien
> >
>

--
Adrien

Re: Query Optimization in search/searchAfter [ In reply to ]

pbikkumanla at gmail

Apr 12, 2024, 7:15 AM

Post #5 of 5 (38 views)

Permalink

Thanks Adrien!

On Fri, Apr 12, 2024 at 9:49?AM Adrien Grand <jpountz@gmail.com> wrote:

> You are correct, query rewriting is not affected by the use of search vs.
> searchAfter.
>
> On Fri, Apr 12, 2024 at 3:37?PM Puneeth Bikkumanla <pbikkumanla@gmail.com>
> wrote:
>
> > Hello,
> > Sorry I should have clarified what I meant by “optimized”. I am familiar
> > with the collector/comparators using the “after” doc to filter out
> > documents but I specifically was talking about the query rewriting phase.
> > Is the query rewritten differently in search vs searchAfter? Looking at
> the
> > code I think no but would just like to confirm if there are any edge
> cases
> > here.
> >
> > On Fri, Apr 12, 2024 at 8:46 AM Adrien Grand <jpountz@gmail.com> wrote:
> >
> > > Hello Puneeth,
> > >
> > > When you pass an `after` doc, Lucene will filter out documents that
> > compare
> > > better than this `after` document if it can. See e.g. what
> LongComparator
> > > does with its `topValue`, which is the value of the `after` doc.
> > >
> > > On Thu, Apr 11, 2024 at 4:34?PM Puneeth Bikkumanla <
> > pbikkumanla@gmail.com>
> > > wrote:
> > >
> > > > Hello,
> > > > I was wondering if a user-defined Query is optimized the same way in
> > both
> > > > search/searchAfter provided the index stays the same (no CRUD takes
> > > place).
> > > >
> > > > In searchAfter we pass in an "after" doc so I was wondering if that
> > > changes
> > > > how a query is optimized at all. By looking at the code, I'm thinking
> > no
> > > > but was wondering if there were any other parameters here that I am
> not
> > > > aware of that would influence query optimization differently in
> > > > search/searchAfter. Thanks!
> > > >
> > >
> > >
> > > --
> > > Adrien
> > >
> >
>
>
> --
> Adrien
>