Mailing List Archive

Hello Stefan,

--- Stefan Bergstrand <stefan.bergstrand@polopoly.com> wrote:
>
>
> When I recently mentioned that it would be nice to be able to search
> without locking the index, I got the answer "It's on the TODO-list".
>
> I'm glad that it has a good chance to be fixed, but to us it has
> become
> a really big problem.
>
> This is our case:
>
> The indexes are stored on a disk that is NFS-mounted to a number of
> load-balanced hosts, on which the searching is performed. The locking
> mechanism prevents us not only from mounting the disk read-only,
> which
> would have been very nice, but also from searching the same index
> concurrently. Thus, we get performance as if we had just one
> host. Which would be single-thread, too.
>
> So, I would like to know:
>
> * Does anyone know when the fix could made?

I don't have an answer to this. I don't know about others, but I do
Lucene only when I have extra time, which is here and there.

> * Is there a plan on how to fix it? (If there is, it would be better
> if I fixed it that way instead of figuring out a kludge of my own.)

Check the mailing list archive, look for terms like CD-ROM or
media/medium, an Italian guy posted a patch or maybe even whole .java
files a few (3+) months ago. This is what I would like to use to
eliminate this problem, but I haven't had time to take a look at his
contribution.
Actually, if you try it, could you please report how it went, whether
the code works as described, or if you had to modify it, etc.?
That would be helpful.

Thanks,
Otis


__________________________________________________
Do You Yahoo!?
Yahoo! - Official partner of 2002 FIFA World Cup
http://fifaworldcup.yahoo.com

--
To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>

Stefan,

I assume you mean disabling the lock file? This is pretty easy to do by
editing the source code. Just edit FSDirectory.java and replace the
current makeLock method with the following:

-------------------------------------
public final Lock makeLock(String name) {
final File lockFile = new File(directory, name);
return new Lock() {
public boolean obtain() throws IOException {
// DISABLE ALL LOCKS
return true;
//if (Constants.JAVA_1_1) return true; // locks disabled in
jdk 1.1
//return lockFile.createNewFile();
}
public void release() {
// DISABLE ALL LOCKS
return;
// if (Constants.JAVA_1_1) return; // locks disabled in
jdk 1.1
//lockFile.delete();
}
public String toString() {
return "Lock@" + lockFile;
}
};
}
--------------------------------------

The "real" way to fix this will be to make the use of lock files
configurable.

Regards,
Matt

> -----Original Message-----
> From: stefan@blues.intra.polopoly.com
> [mailto:stefan@blues.intra.polopoly.com] On Behalf Of Stefan
> Bergstrand
> Sent: Tuesday, June 11, 2002 10:40 AM
> To: Lucene Developer
> Subject: Locking problems
>
>
>
>
> When I recently mentioned that it would be nice to be able to
> search without locking the index, I got the answer "It's on
> the TODO-list".
>
> I'm glad that it has a good chance to be fixed, but to us it
> has become a really big problem.
>
> This is our case:
>
> The indexes are stored on a disk that is NFS-mounted to a
> number of load-balanced hosts, on which the searching is
> performed. The locking mechanism prevents us not only from
> mounting the disk read-only, which would have been very nice,
> but also from searching the same index concurrently. Thus, we
> get performance as if we had just one host. Which would be
> single-thread, too.
>
> So, I would like to know:
>
> * Does anyone know when the fix could made?
>
> * Is there a plan on how to fix it? (If there is, it would be better
> if I fixed it that way instead of figuring out a kludge of my own.)
>
> /Stefan B
>
>
> --
> ---------------------------
> Stefan Bergstrand
> Polopoly - Cultivating the information garden
> Kungsgatan 88, SE-112 27 Stockholm, SWEDEN
> Ph: +46 8 506 782 67
> Cell: +46 704 47 82 67
> stefan.bergstrand@polopoly.com, http://www.polopoly.com
>
> --
> To unsubscribe, e-mail:
> <mailto:lucene-dev-> unsubscribe@jakarta.apache.org>
> For
> additional commands,
> e-mail: <mailto:lucene-dev-help@jakarta.apache.org>
>


--
To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>

Ok, now I have fixed a way to turn locking off. Here is the modified
method in FSDirectory, and a helper method. Locking is turned off by
setting -DdisableLuceneLocks=true on the command line.

Thanks to Matt Tucker for pointing out the place where to modify.

/Stefan B


/** Construct a {@link Lock}.
* @param name the name of the lock file
*/
public final Lock makeLock(String name) {
final File lockFile = new File(directory, name);
return new Lock() {
public boolean obtain() throws IOException {

if (locksDisabled()){
return true;
}

if (Constants.JAVA_1_1)
return true; // locks disabled in jdk 1.1

return lockFile.createNewFile();
}
public void release() {

if (locksDisabled()){
return;
}

if (Constants.JAVA_1_1)
return; // locks disabled in jdk 1.1
lockFile.delete();
}

public String toString() {
return "Lock@" + lockFile;
}
};
}

protected final boolean locksDisabled(){

if (System.getProperty("disableLuceneLocks") == null){
return false;
} else {
return System.getProperty("disableLuceneLocks").equals("true");
}

}

--
---------------------------
Stefan Bergstrand
Polopoly - Cultivating the information garden
Kungsgatan 88, SE-112 27 Stockholm, SWEDEN
Ph: +46 8 506 782 67
Cell: +46 704 47 82 67
stefan.bergstrand@polopoly.com, http://www.polopoly.com

--
To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>

Stefan,

your locksDisabled() method is a reimplementation of functionality already
provided by the Java SDK:

Boolean.getBoolean(String name) does exactly what your method does:

Returns true if and only if the system property named by the argument exists
and is equal to the string "true".



-----Original Message-----
From: Stefan Bergstrand
To: Lucene Developer
Sent: 6/12/02 3:28 AM
Subject: Re: Locking problems


Ok, now I have fixed a way to turn locking off. Here is the modified
method in FSDirectory, and a helper method. Locking is turned off by
setting -DdisableLuceneLocks=true on the command line.

Thanks to Matt Tucker for pointing out the place where to modify.

/Stefan B


/** Construct a {@link Lock}.
* @param name the name of the lock file
*/
public final Lock makeLock(String name) {
final File lockFile = new File(directory, name);
return new Lock() {
public boolean obtain() throws IOException {

if (locksDisabled()){
return true;
}

if (Constants.JAVA_1_1)
return true; // locks disabled in jdk 1.1

return lockFile.createNewFile();
}
public void release() {

if (locksDisabled()){
return;
}

if (Constants.JAVA_1_1)
return; // locks disabled in jdk 1.1
lockFile.delete();
}

public String toString() {
return "Lock@" + lockFile;
}
};
}

protected final boolean locksDisabled(){

if (System.getProperty("disableLuceneLocks") == null){
return false;
} else {
return
System.getProperty("disableLuceneLocks").equals("true");
}

}

--
---------------------------
Stefan Bergstrand
Polopoly - Cultivating the information garden
Kungsgatan 88, SE-112 27 Stockholm, SWEDEN
Ph: +46 8 506 782 67
Cell: +46 704 47 82 67
stefan.bergstrand@polopoly.com, http://www.polopoly.com

--
To unsubscribe, e-mail:
<mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:lucene-dev-help@jakarta.apache.org>

--
To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>

I hasten to add that it's not terribly efficient to check this system
property every time lucene asks to obtain a lock. Better would be to assign
a boolean once and then check that in the obtain() method.

-----Original Message-----
From: Friedman, Eric
To: 'Stefan Bergstrand '; 'Lucene Developer '
Sent: 6/12/02 7:03 AM
Subject: RE: Locking problems

Stefan,

your locksDisabled() method is a reimplementation of functionality
already
provided by the Java SDK:

Boolean.getBoolean(String name) does exactly what your method does:

Returns true if and only if the system property named by the argument
exists
and is equal to the string "true".



-----Original Message-----
From: Stefan Bergstrand
To: Lucene Developer
Sent: 6/12/02 3:28 AM
Subject: Re: Locking problems


Ok, now I have fixed a way to turn locking off. Here is the modified
method in FSDirectory, and a helper method. Locking is turned off by
setting -DdisableLuceneLocks=true on the command line.

Thanks to Matt Tucker for pointing out the place where to modify.

/Stefan B


/** Construct a {@link Lock}.
* @param name the name of the lock file
*/
public final Lock makeLock(String name) {
final File lockFile = new File(directory, name);
return new Lock() {
public boolean obtain() throws IOException {

if (locksDisabled()){
return true;
}

if (Constants.JAVA_1_1)
return true; // locks disabled in jdk 1.1

return lockFile.createNewFile();
}
public void release() {

if (locksDisabled()){
return;
}

if (Constants.JAVA_1_1)
return; // locks disabled in jdk 1.1
lockFile.delete();
}

public String toString() {
return "Lock@" + lockFile;
}
};
}

protected final boolean locksDisabled(){

if (System.getProperty("disableLuceneLocks") == null){
return false;
} else {
return
System.getProperty("disableLuceneLocks").equals("true");
}

}

--
---------------------------
Stefan Bergstrand
Polopoly - Cultivating the information garden
Kungsgatan 88, SE-112 27 Stockholm, SWEDEN
Ph: +46 8 506 782 67
Cell: +46 704 47 82 67
stefan.bergstrand@polopoly.com, http://www.polopoly.com

--
To unsubscribe, e-mail:
<mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:lucene-dev-help@jakarta.apache.org>

--
To unsubscribe, e-mail:
<mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:lucene-dev-help@jakarta.apache.org>

--
To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>