Mailing List Archive

Hi Haoyu,

I'm pretty sure (but not certain!) that query cache is smart enough to
recurse through the full query tree, and consider any of the whole queries
it finds during that recursion.

So e.g. a query like +A +B +(C D) would consider caching A, B, C D, or the
whole original +A +B +(C D) query.

But I'm not sure! Hopefully someone who knows more about query cache might
chime in.

Mike McCandless

http://blog.mikemccandless.com


On Mon, Feb 22, 2021 at 8:55 PM Haoyu Zhai <zhai7631@gmail.com> wrote:

> Hi folks,
> I'm trying to understand how QueryCache works and one question popped out
> of my head was that is QueryCache caching
> 1. the whole query that being submitted to IndexSearcher or
> 2. it will recurse into the query and selectively caching some of the
> clauses (especially for BooleanQuery)?
>
> From my observation it is the former case but I just want to double check
> in case I missed anything.
>
> Thanks
> Patrick
>

It does recurse indeed! To reuse Mike's example, in that case the cache
would consider caching:
- A,
- B,
- C,
- D,
- (C D),
- +A +B +(C D)

One weakness of this cache is that it doesn't consider caching subsets of
boolean queries (except single clauses). E.g. in the above example, it
would never consider caching +A +B even if the conjunction of these two
clauses occurs in many queries.

Le ven. 26 févr. 2021 à 20:03, Michael McCandless <lucene@mikemccandless.com>
a écrit :

> Hi Haoyu,
>
> I'm pretty sure (but not certain!) that query cache is smart enough to
> recurse through the full query tree, and consider any of the whole queries
> it finds during that recursion.
>
> So e.g. a query like +A +B +(C D) would consider caching A, B, C D, or the
> whole original +A +B +(C D) query.
>
> But I'm not sure! Hopefully someone who knows more about query cache
> might chime in.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Mon, Feb 22, 2021 at 8:55 PM Haoyu Zhai <zhai7631@gmail.com> wrote:
>
>> Hi folks,
>> I'm trying to understand how QueryCache works and one question popped out
>> of my head was that is QueryCache caching
>> 1. the whole query that being submitted to IndexSearcher or
>> 2. it will recurse into the query and selectively caching some of the
>> clauses (especially for BooleanQuery)?
>>
>> From my observation it is the former case but I just want to double check
>> in case I missed anything.
>>
>> Thanks
>> Patrick
>>
>

Thanks Mike and Adrien for confirming the behavior!
I checked again and debugged the unit case and found it is
IndexSearcher.createWeight will be recursively called when BooleanQuery is
creating weight (
https://github.com/apache/lucene-solr/blob/e88b3e9c204f907fdb41d6d0f40d685574acde97/lucene/core/src/java/org/apache/lucene/search/BooleanWeight.java#L59),
I missed this part when I previously checking the logic.

Best
Patrick

Adrien Grand <jpountz@gmail.com> ?2021?2?26??? ??1:02???

> It does recurse indeed! To reuse Mike's example, in that case the cache
> would consider caching:
> - A,
> - B,
> - C,
> - D,
> - (C D),
> - +A +B +(C D)
>
> One weakness of this cache is that it doesn't consider caching subsets of
> boolean queries (except single clauses). E.g. in the above example, it
> would never consider caching +A +B even if the conjunction of these two
> clauses occurs in many queries.
>
> Le ven. 26 févr. 2021 à 20:03, Michael McCandless <
> lucene@mikemccandless.com> a écrit :
>
>> Hi Haoyu,
>>
>> I'm pretty sure (but not certain!) that query cache is smart enough to
>> recurse through the full query tree, and consider any of the whole queries
>> it finds during that recursion.
>>
>> So e.g. a query like +A +B +(C D) would consider caching A, B, C D, or
>> the whole original +A +B +(C D) query.
>>
>> But I'm not sure! Hopefully someone who knows more about query cache
>> might chime in.
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>
>> On Mon, Feb 22, 2021 at 8:55 PM Haoyu Zhai <zhai7631@gmail.com> wrote:
>>
>>> Hi folks,
>>> I'm trying to understand how QueryCache works and one question
>>> popped out of my head was that is QueryCache caching
>>> 1. the whole query that being submitted to IndexSearcher or
>>> 2. it will recurse into the query and selectively caching some of the
>>> clauses (especially for BooleanQuery)?
>>>
>>> From my observation it is the former case but I just want to double
>>> check in case I missed anything.
>>>
>>> Thanks
>>> Patrick
>>>
>>