Mailing List Archive

Resizable LRUQueryCache
Hi,

We are looking at implementing a setter function to resize the
LRUQueryCache on demand and evict the least recently used queries. Please
share any thoughts or concerns around this subject.

Adithya
Re: Resizable LRUQueryCache [ In reply to ]
Why do you need to resize it? And if you need to resize it, would it be
good enough to drop the old cache and start using a new one?

On Thu, Mar 5, 2020 at 12:54 AM Aadithya C <aadithya.c@gmail.com> wrote:

> Hi,
>
> We are looking at implementing a setter function to resize the
> LRUQueryCache on demand and evict the least recently used queries. Please
> share any thoughts or concerns around this subject.
>
> Adithya
>


--
Adrien
Re: Resizable LRUQueryCache [ In reply to ]
In my personal opinion, there are a few advantages of resizing -


1) The size of the cache is unpredictable as there is a fixed(guesstimate)
accounting for the key size. With a resizable cache, we can potentially
cache heavier queries and exploratively resize the cache when faced with
memory pressure.


2) With a resizable cache, we can control memory allocation dynamically
based on the workload. For a large cache, dropping the entire cache when we
want to reallocate some memory seems like an excessive action.


3) The query cache effectiveness is very workload dependent. I have
observed the cache hit-rate can be in single digits for certain workloads
and memory can be effectively used elsewhere.


Adithya





On Thu, Mar 5, 2020 at 11:04 AM Adrien Grand <jpountz@gmail.com> wrote:

> Why do you need to resize it? And if you need to resize it, would it be
> good enough to drop the old cache and start using a new one?
>
> On Thu, Mar 5, 2020 at 12:54 AM Aadithya C <aadithya.c@gmail.com> wrote:
>
> > Hi,
> >
> > We are looking at implementing a setter function to resize the
> > LRUQueryCache on demand and evict the least recently used queries. Please
> > share any thoughts or concerns around this subject.
> >
> > Adithya
> >
>
>
> --
> Adrien
>
Re: Resizable LRUQueryCache [ In reply to ]
On Fri, Mar 6, 2020 at 1:04 AM Aadithya C <aadithya.c@gmail.com> wrote:
>
> In my personal opinion, there are a few advantages of resizing -
>
>
> 1) The size of the cache is unpredictable as there is a fixed(guesstimate)
> accounting for the key size. With a resizable cache, we can potentially
> cache heavier queries and exploratively resize the cache when faced with
> memory pressure.
>
>
> 2) With a resizable cache, we can control memory allocation dynamically
> based on the workload. For a large cache, dropping the entire cache when we
> want to reallocate some memory seems like an excessive action.
>
>
> 3) The query cache effectiveness is very workload dependent. I have
> observed the cache hit-rate can be in single digits for certain workloads
> and memory can be effectively used elsewhere.

When you say resizing, do you mean only increasing sizes or decreasing as well?

IMO, this would require defining a model for on demand eviction until
the cache reaches a target size (be mindful of what happens to
incoming caching requests in that period). A potentially large
downsizing request can lead to a large eviction time -- not sure what
impact it would have on query performance.

I would agree with Adrien's suggestion of using a new cache unless you
have a very compelling reason not to. Even then, I would be wary of
muddying LRUQueryCache's waters -- you might want to create your own
custom derivative of it?

Atri

--
Regards,

Atri
Apache Concerted

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Resizable LRUQueryCache [ In reply to ]
Hi Atri,

Thanks for pointing out potential performance issues for high QPS workloads
when downsizing. Depending on the size of the cache, it might make sense to
create a new one with the necessary entries. On the other hand, if you
consider cases where the process is under heavy memory pressure and
performance is already bad, resizing the cache will only help irrespective
of the workload. Additionally, we can also explore performance
optimizations for large evictions.
I can create a new derivative but I was wondering if this change can be
beneficial to other users and should be a part of lucene. One advantage of
being part of the LRUQueryCache is that this feature will evolve with other
cache changes.

Adithya

On Thu, Mar 5, 2020 at 7:17 PM Atri Sharma <atri@apache.org> wrote:

> On Fri, Mar 6, 2020 at 1:04 AM Aadithya C <aadithya.c@gmail.com> wrote:
> >
> > In my personal opinion, there are a few advantages of resizing -
> >
> >
> > 1) The size of the cache is unpredictable as there is a
> fixed(guesstimate)
> > accounting for the key size. With a resizable cache, we can potentially
> > cache heavier queries and exploratively resize the cache when faced with
> > memory pressure.
> >
> >
> > 2) With a resizable cache, we can control memory allocation dynamically
> > based on the workload. For a large cache, dropping the entire cache when
> we
> > want to reallocate some memory seems like an excessive action.
> >
> >
> > 3) The query cache effectiveness is very workload dependent. I have
> > observed the cache hit-rate can be in single digits for certain workloads
> > and memory can be effectively used elsewhere.
>
> When you say resizing, do you mean only increasing sizes or decreasing as
> well?
>
> IMO, this would require defining a model for on demand eviction until
> the cache reaches a target size (be mindful of what happens to
> incoming caching requests in that period). A potentially large
> downsizing request can lead to a large eviction time -- not sure what
> impact it would have on query performance.
>
> I would agree with Adrien's suggestion of using a new cache unless you
> have a very compelling reason not to. Even then, I would be wary of
> muddying LRUQueryCache's waters -- you might want to create your own
> custom derivative of it?
>
> Atri
>
> --
> Regards,
>
> Atri
> Apache Concerted
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: Resizable LRUQueryCache [ In reply to ]
Maybe start with your own cache implementation that implements a resize
method? The cache is pluggable through IndexSearcher.

Fully discarding the cache and swapping in a newly sized (empty) one could
also be jarring, so I can see the motivation for this method. Especially
for usages that are hosting many separate indices and want some control
over total heap usage.

Mike McCandless

http://blog.mikemccandless.com


On Tue, Mar 10, 2020 at 1:48 PM Aadithya C <aadithya.c@gmail.com> wrote:

> Hi Atri,
>
> Thanks for pointing out potential performance issues for high QPS workloads
> when downsizing. Depending on the size of the cache, it might make sense to
> create a new one with the necessary entries. On the other hand, if you
> consider cases where the process is under heavy memory pressure and
> performance is already bad, resizing the cache will only help irrespective
> of the workload. Additionally, we can also explore performance
> optimizations for large evictions.
> I can create a new derivative but I was wondering if this change can be
> beneficial to other users and should be a part of lucene. One advantage of
> being part of the LRUQueryCache is that this feature will evolve with other
> cache changes.
>
> Adithya
>
> On Thu, Mar 5, 2020 at 7:17 PM Atri Sharma <atri@apache.org> wrote:
>
> > On Fri, Mar 6, 2020 at 1:04 AM Aadithya C <aadithya.c@gmail.com> wrote:
> > >
> > > In my personal opinion, there are a few advantages of resizing -
> > >
> > >
> > > 1) The size of the cache is unpredictable as there is a
> > fixed(guesstimate)
> > > accounting for the key size. With a resizable cache, we can potentially
> > > cache heavier queries and exploratively resize the cache when faced
> with
> > > memory pressure.
> > >
> > >
> > > 2) With a resizable cache, we can control memory allocation dynamically
> > > based on the workload. For a large cache, dropping the entire cache
> when
> > we
> > > want to reallocate some memory seems like an excessive action.
> > >
> > >
> > > 3) The query cache effectiveness is very workload dependent. I have
> > > observed the cache hit-rate can be in single digits for certain
> workloads
> > > and memory can be effectively used elsewhere.
> >
> > When you say resizing, do you mean only increasing sizes or decreasing as
> > well?
> >
> > IMO, this would require defining a model for on demand eviction until
> > the cache reaches a target size (be mindful of what happens to
> > incoming caching requests in that period). A potentially large
> > downsizing request can lead to a large eviction time -- not sure what
> > impact it would have on query performance.
> >
> > I would agree with Adrien's suggestion of using a new cache unless you
> > have a very compelling reason not to. Even then, I would be wary of
> > muddying LRUQueryCache's waters -- you might want to create your own
> > custom derivative of it?
> >
> > Atri
> >
> > --
> > Regards,
> >
> > Atri
> > Apache Concerted
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>