Mailing List Archive

Disk I/O in Lucene
Hi, I am newbie to Lucene and have a question about
Lucene's disk I/O.

For JDK 1.4 has java.nio APIs, does Lucene use this
new I/O library to improve the disk I/O performance?
Are they any benchmark about the performance of
Lucene?

Thanks.

Alex

__________________________________________________
Do You Yahoo!?
Yahoo! Tax Center - online filing with TurboTax
http://taxes.yahoo.com/

--
To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>
Re: Disk I/O in Lucene [ In reply to ]
Hi,

Lucene does not use the new nio API's in 1.4 (in fact it is compatible back
to 1.1.8).

What kind of bench marks are you looking for? I am currently searching over
100K documents in about .015 seconds for a simple query on a Sun Netra T1
(450 Mhz).

If you have a real need for speed, Lucene also supports a RAM Directory to
store the index. I have not used this, but heard it's faster.

--Peter


On 4/9/02 2:41 PM, "Donglin Lu" <qlv@yahoo.com> wrote:

>
> Hi, I am newbie to Lucene and have a question about
> Lucene's disk I/O.
>
> For JDK 1.4 has java.nio APIs, does Lucene use this
> new I/O library to improve the disk I/O performance?
> Are they any benchmark about the performance of
> Lucene?
>
> Thanks.
>
> Alex
>
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Tax Center - online filing with TurboTax
> http://taxes.yahoo.com/
>
> --
> To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>
>
>


--
To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>
Re: Disk I/O in Lucene [ In reply to ]
It might also be an interesting project to implement an NIODirectory,
which would be just like the current FileDirectory, but would use the
nio API. Lucene's abstraction of a Directory should make this possible,
I think.

Dmitry.


Peter Carlson wrote:

>Hi,
>
>Lucene does not use the new nio API's in 1.4 (in fact it is compatible back
>to 1.1.8).
>
>What kind of bench marks are you looking for? I am currently searching over
>100K documents in about .015 seconds for a simple query on a Sun Netra T1
>(450 Mhz).
>
>If you have a real need for speed, Lucene also supports a RAM Directory to
>store the index. I have not used this, but heard it's faster.
>
>--Peter
>
>
>On 4/9/02 2:41 PM, "Donglin Lu" <qlv@yahoo.com> wrote:
>
>>Hi, I am newbie to Lucene and have a question about
>>Lucene's disk I/O.
>>
>>For JDK 1.4 has java.nio APIs, does Lucene use this
>>new I/O library to improve the disk I/O performance?
>>Are they any benchmark about the performance of
>>Lucene?
>>
>>Thanks.
>>
>>Alex
>>
>>__________________________________________________
>>Do You Yahoo!?
>>Yahoo! Tax Center - online filing with TurboTax
>>http://taxes.yahoo.com/
>>
>>--
>>To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
>>For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>
>>
>>
>
>
>--
>To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
>For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>
>
>




--
To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>
Re: Disk I/O in Lucene [ In reply to ]
Hi, Peter.

Thanks for the information. I am curious why Lucene
doesn't use the NIO API of JDK 1.4. It may improve the
performance.

I have not tried the RAM Directory yet.

Alex

----- Original Message -----
From: "Peter Carlson" <carlson@bookandhammer.com>
To: "Lucene Developers List"
<lucene-dev@jakarta.apache.org>; "Lucene Users List"
<lucene-user@jakarta.apache.org>
Sent: Tuesday, April 09, 2002 7:08 PM
Subject: Re: Disk I/O in Lucene


> Hi,
>
> Lucene does not use the new nio API's in 1.4 (in
fact it is compatible back
> to 1.1.8).
>
> What kind of bench marks are you looking for? I am
currently searching over
> 100K documents in about .015 seconds for a simple
query on a Sun Netra T1
> (450 Mhz).
>
> If you have a real need for speed, Lucene also
supports a RAM Directory to
> store the index. I have not used this, but heard
it's faster.
>
> --Peter
>
>
> On 4/9/02 2:41 PM, "Donglin Lu" <qlv@yahoo.com>
wrote:
>
> >
> > Hi, I am newbie to Lucene and have a question
about
> > Lucene's disk I/O.
> >
> > For JDK 1.4 has java.nio APIs, does Lucene use
this
> > new I/O library to improve the disk I/O
performance?
> > Are they any benchmark about the performance of
> > Lucene?
> >
> > Thanks.
> >
> > Alex
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Yahoo! Tax Center - online filing with TurboTax
> > http://taxes.yahoo.com/
> >
> > --
> > To unsubscribe, e-mail:
<mailto:lucene-dev-unsubscribe@jakarta.apache.org>
> > For additional commands, e-mail:
<mailto:lucene-dev-help@jakarta.apache.org>
> >
> >
>
>
> --
> To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>

__________________________________________________
Do You Yahoo!?
Yahoo! Tax Center - online filing with TurboTax
http://taxes.yahoo.com/

--
To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>
Re: Disk I/O in Lucene [ In reply to ]
--- Donglin Lu <qlv@yahoo.com> wrote:
>
> Hi, Peter.
>
> Thanks for the information. I am curious why Lucene
> doesn't use the NIO API of JDK 1.4. It may improve the
> performance.

Lucene was written before 1.4.
If you implement Directory using NIO please contribute the code if you
can.

Otis

> I have not tried the RAM Directory yet.
>
> Alex
>
> ----- Original Message -----
> From: "Peter Carlson" <carlson@bookandhammer.com>
> To: "Lucene Developers List"
> <lucene-dev@jakarta.apache.org>; "Lucene Users List"
> <lucene-user@jakarta.apache.org>
> Sent: Tuesday, April 09, 2002 7:08 PM
> Subject: Re: Disk I/O in Lucene
>
>
> > Hi,
> >
> > Lucene does not use the new nio API's in 1.4 (in
> fact it is compatible back
> > to 1.1.8).
> >
> > What kind of bench marks are you looking for? I am
> currently searching over
> > 100K documents in about .015 seconds for a simple
> query on a Sun Netra T1
> > (450 Mhz).
> >
> > If you have a real need for speed, Lucene also
> supports a RAM Directory to
> > store the index. I have not used this, but heard
> it's faster.
> >
> > --Peter
> >
> >
> > On 4/9/02 2:41 PM, "Donglin Lu" <qlv@yahoo.com>
> wrote:
> >
> > >
> > > Hi, I am newbie to Lucene and have a question
> about
> > > Lucene's disk I/O.
> > >
> > > For JDK 1.4 has java.nio APIs, does Lucene use
> this
> > > new I/O library to improve the disk I/O
> performance?
> > > Are they any benchmark about the performance of
> > > Lucene?
> > >
> > > Thanks.
> > >
> > > Alex
> > >
> > > __________________________________________________
> > > Do You Yahoo!?
> > > Yahoo! Tax Center - online filing with TurboTax
> > > http://taxes.yahoo.com/
> > >
> > > --
> > > To unsubscribe, e-mail:
> <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
> > > For additional commands, e-mail:
> <mailto:lucene-dev-help@jakarta.apache.org>
> > >
> > >
> >
> >
> > --
> > To unsubscribe, e-mail:
> <mailto:lucene-user-unsubscribe@jakarta.apache.org>
> > For additional commands, e-mail:
> <mailto:lucene-user-help@jakarta.apache.org>
>
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Tax Center - online filing with TurboTax
> http://taxes.yahoo.com/
>
> --
> To unsubscribe, e-mail:
> <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail:
> <mailto:lucene-dev-help@jakarta.apache.org>
>


__________________________________________________
Do You Yahoo!?
Yahoo! Tax Center - online filing with TurboTax
http://taxes.yahoo.com/

--
To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>
RE: Disk I/O in Lucene [ In reply to ]
Somewhat obvious caution: don't assume the NIO (well I guess you
mean memory mapped i/o) code will be faster - some isolation testing
I did in release candidates of jdk1.4 showed it to be *slower* than
normal file i/o. No doubt it should be faster.

Note: can't find bug report I submitted to javasoft but there are other
reports
that seem to indicate that this is fixed.

-----Original Message-----
From: Donglin Lu [mailto:qlv@yahoo.com]
Sent: Wednesday, April 10, 2002 12:23 AM
To: lucene-dev@jakarta.apache.org
Subject: Re: Disk I/O in Lucene



Hi, Peter.

Thanks for the information. I am curious why Lucene
doesn't use the NIO API of JDK 1.4. It may improve the
performance.

I have not tried the RAM Directory yet.

Alex

----- Original Message -----
From: "Peter Carlson" <carlson@bookandhammer.com>
To: "Lucene Developers List"
<lucene-dev@jakarta.apache.org>; "Lucene Users List"
<lucene-user@jakarta.apache.org>
Sent: Tuesday, April 09, 2002 7:08 PM
Subject: Re: Disk I/O in Lucene


> Hi,
>
> Lucene does not use the new nio API's in 1.4 (in
fact it is compatible back
> to 1.1.8).
>
> What kind of bench marks are you looking for? I am
currently searching over
> 100K documents in about .015 seconds for a simple
query on a Sun Netra T1
> (450 Mhz).
>
> If you have a real need for speed, Lucene also
supports a RAM Directory to
> store the index. I have not used this, but heard
it's faster.
>
> --Peter
>
>
> On 4/9/02 2:41 PM, "Donglin Lu" <qlv@yahoo.com>
wrote:
>
> >
> > Hi, I am newbie to Lucene and have a question
about
> > Lucene's disk I/O.
> >
> > For JDK 1.4 has java.nio APIs, does Lucene use
this
> > new I/O library to improve the disk I/O
performance?
> > Are they any benchmark about the performance of
> > Lucene?
> >
> > Thanks.
> >
> > Alex
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Yahoo! Tax Center - online filing with TurboTax
> > http://taxes.yahoo.com/
> >
> > --
> > To unsubscribe, e-mail:
<mailto:lucene-dev-unsubscribe@jakarta.apache.org>
> > For additional commands, e-mail:
<mailto:lucene-dev-help@jakarta.apache.org>
> >
> >
>
>
> --
> To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>

__________________________________________________
Do You Yahoo!?
Yahoo! Tax Center - online filing with TurboTax
http://taxes.yahoo.com/

--
To unsubscribe, e-mail:
<mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:lucene-dev-help@jakarta.apache.org>


--
To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>