First of, thanks to Jagadesh Nandasamy who directed me to the right
direction.
It seems, that in my situation, more homogeneous indexes work better
than fewer heterogeneous indexes:
I have a dozen class that I'm indexing. They vary from two fields to
more than a dozen field per document (aka object). I went through
different indexing strategy with them (per class, per date, per root
class, ... ) to see how it goes. In any case, while trying to use my
stuff with rc4 I consolidated all my different class indexes into one
root class index to see if I could reduce my resources consumption. Less
indexes, less RandomAccessFile was the rational. Well, I was wrong. In
fact the exact opposite seems to hold true: more -homogeneous- indexes
use overall less RandomAccessFile than less -heterogeneous- indexes...
One of those -not so obvious- thing you have to learn the hard way I
guess... ;-)
In any case, I would like to thanks again Jagadesh for his insight. Also
thanks to Pier Fumagalli for pointing out "LSOF". A very handy tool
indeed.
As a final note, several people suggested to increase the number of file
descriptors per process with something like "ulimit"... From what I
learned today, I think it's a *bad* idea to have to change some system
parameters just because your/my app is written in such a way that it may
run out of some system resources. Your/my app has to fit in the system.
Hacking "ulimit" and/or other system parameters is just a quick patch
that will -at best- delay dealing with the real problem that's usually
one of design.
Just my two cents.
PA.
--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>
direction.
It seems, that in my situation, more homogeneous indexes work better
than fewer heterogeneous indexes:
I have a dozen class that I'm indexing. They vary from two fields to
more than a dozen field per document (aka object). I went through
different indexing strategy with them (per class, per date, per root
class, ... ) to see how it goes. In any case, while trying to use my
stuff with rc4 I consolidated all my different class indexes into one
root class index to see if I could reduce my resources consumption. Less
indexes, less RandomAccessFile was the rational. Well, I was wrong. In
fact the exact opposite seems to hold true: more -homogeneous- indexes
use overall less RandomAccessFile than less -heterogeneous- indexes...
One of those -not so obvious- thing you have to learn the hard way I
guess... ;-)
In any case, I would like to thanks again Jagadesh for his insight. Also
thanks to Pier Fumagalli for pointing out "LSOF". A very handy tool
indeed.
As a final note, several people suggested to increase the number of file
descriptors per process with something like "ulimit"... From what I
learned today, I think it's a *bad* idea to have to change some system
parameters just because your/my app is written in such a way that it may
run out of some system resources. Your/my app has to fit in the system.
Hacking "ulimit" and/or other system parameters is just a quick patch
that will -at best- delay dealing with the real problem that's usually
one of design.
Just my two cents.
PA.
--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>