Mailing List Archive

Re: Memory Usage?
For what it is worth I've got an index with 1.6 million
small entries, taking up 350Mb of disk space, and have
a long running server process handling search requests
against the index and memory usage is stable at 95Mb.
It has handled 425000+ mostly simple searches so far.
The index is optimized and the process reuses an IndexReader.



--
Ian.
ian.lea@blackwell.co.uk


Anders Nielsen wrote:
>
> funny...I was just about to write something along the same lines..
>
> I have 700.000 entries, in all about 1 gig of data. And when I search I have
> to allocate at least 150meg to the java-process or it'll run out of memory.
>
> Could someone (most likely Doug) outline how much memory we can expect is
> necessary when searching? (big-Oh notation would be nice)
>
> regards,
> Anders Nielsen
>
> -----Original Message-----
> From: Scott Ganyo [mailto:scott.ganyo@eTapestry.com]
> Sent: 8. november 2001 21:30
> To: Lucene-Users (E-mail)
> Subject: Memory Usage?
>
> We're having problems with memory allocation (and thus garbage collection)
> that seem to be directly attributable to our heavy use of Lucene indexes. I
> don't have a lot of data on this yet, but has anyone else seen this as an
> issue? Is there any way that Lucene could be tuned to use less memory
> during searches?
>
> Thanks,
> Scott

--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>
RE: Memory Usage? [ In reply to ]
funny...I was just about to write something along the same lines..

I have 700.000 entries, in all about 1 gig of data. And when I search I have
to allocate at least 150meg to the java-process or it'll run out of memory.

Could someone (most likely Doug) outline how much memory we can expect is
necessary when searching? (big-Oh notation would be nice)

regards,
Anders Nielsen

-----Original Message-----
From: Scott Ganyo [mailto:scott.ganyo@eTapestry.com]
Sent: 8. november 2001 21:30
To: Lucene-Users (E-mail)
Subject: Memory Usage?


We're having problems with memory allocation (and thus garbage collection)
that seem to be directly attributable to our heavy use of Lucene indexes. I
don't have a lot of data on this yet, but has anyone else seen this as an
issue? Is there any way that Lucene could be tuned to use less memory
during searches?

Thanks,
Scott


--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>
RE: Memory Usage? [ In reply to ]
oh, and I forgot to mention that the excessive memory-usage only appears
when I run very complex queries (BooleanQuery with about 30 or so terms and
a DateFilter)

I'm not complaining mind you, Lucene is still by far the best performing
full text indexing system I've been able to find.

regards,
Anders Nielsen

-----Original Message-----
From: Scott Ganyo [mailto:scott.ganyo@eTapestry.com]
Sent: 8. november 2001 21:30
To: Lucene-Users (E-mail)
Subject: Memory Usage?


We're having problems with memory allocation (and thus garbage collection)
that seem to be directly attributable to our heavy use of Lucene indexes. I
don't have a lot of data on this yet, but has anyone else seen this as an
issue? Is there any way that Lucene could be tuned to use less memory
during searches?

Thanks,
Scott


--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>