Mailing List Archive: Attribute Search Bug

This program illustrates what may be a bug. It creates an index, a document
with two fields. The second field is the problem. I use the Field
constructor to make a field that is not stored, is indexed, not tokenized
(there is no factory method for this combination).

The program then queries on the second field and no hits are returned.

import java.io.File;
import java.io.IOException;
import java.io.OutputStream;
import java.util.*;

import org.apache.lucene.analysis.SimpleAnalyzer;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.document.*;
import org.apache.lucene.search.*;
import org.apache.lucene.queryParser.QueryParser;

public class Bug1 {
public static void main(String[] args) throws Exception {
String indexPath = "./bugdex";
String fieldName = "catno";
String fieldValue= "AB12-42 2X";
IndexWriter writer;

// An index is created by opening an IndexWriter with the
// create argument set to true.
writer = new IndexWriter(indexPath, null, true);
writer.close();

// now index a document
try {
writer = new IndexWriter(indexPath, new SimpleAnalyzer(), false);

Document doc = new Document();
// add id as unindexed - can't search, but it is stored
doc.add(Field.UnIndexed("id", "12345"));
doc.add(new Field(fieldName,fieldValue,false,true,false));
writer.addDocument(doc);
writer.close();

} catch (Exception e) {
e.printStackTrace();
}

// now make a query and search for document
Searcher searcher = new IndexSearcher(indexPath);
Query query = QueryParser.parse(fieldValue, fieldName,
new SimpleAnalyzer());
Hits hits = searcher.search(query);
for (int i=0; i<hits.length(); i++) {
System.out.println(
"id "+hits.doc(i).get("id") + "; Score: " + hits.score(i));
}
System.out.println("\nNumber of hits: "+ hits.length());
}
}

----- Original Message -----
From: Doug Cutting <DCutting@grandcentral.com>
To: 'Lucene Users List' <lucene-user@jakarta.apache.org>
Sent: Monday, November 26, 2001 2:42 PM
Subject: RE: Attribute Search

> > From: New, Cecil (GEAE) [mailto:cecil.new@ae.ge.com]
> >
> > this is exactly what I was doing. Store=false, index=true,
> > and token=false.
> >
> > It appeared to work ok, but searches *never* returned any hits.
> >
> > That's why I suspect it is a bug.
>
> If you think this is a bug, please submit a test case, as a simple class
> whose 'main()' method illustrates the problem.
>
> Doug
>
> --
> To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>
>

--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>

Thanks for the test case.

The problem is using SimpleAnalyzer when you parse your query.
SimpleAnalyzer uses LetterTokenizer, which turns your query for "AB12-42 2X"
into a two term query containing "ab" and "x". Since there are no instances
of the terms "ab" or "x" in your index, no documents are returned.

You either need to use a different Analyzer, one that knows that knows how
to tokenized the "catno" field, or don't use the query parser to construct
your query.

For example, if you instead construct your query as:
Query query = new TermQuery(new Term(fieldName, fieldValue));
then your document is found.

Note that you can combine such manually-constructed queries with queries
from the query parser by adding both to a BooleanQuery:
Query q2 = QueryParser.parse(....);
BooleanQuery finalQuery = new BooleanQuery();
finalQuery.add(query, true, false);
finalQuery.add(q2, true, false);
Hits hits = searcher.search(finalQuery);

Doug

> -----Original Message-----
> From: Cecil, Paula New [mailto:cnew@fuse.net]
> Sent: Wednesday, November 28, 2001 9:28 PM
> To: Lucene Users List
> Subject: Attribute Search Bug
>
>
> This program illustrates what may be a bug. It creates an
> index, a document
> with two fields. The second field is the problem. I use the Field
> constructor to make a field that is not stored, is indexed,
> not tokenized
> (there is no factory method for this combination).
>
> The program then queries on the second field and no hits are returned.
>
> import java.io.File;
> import java.io.IOException;
> import java.io.OutputStream;
> import java.util.*;
>
> import org.apache.lucene.analysis.SimpleAnalyzer;
> import org.apache.lucene.index.IndexWriter;
> import org.apache.lucene.document.*;
> import org.apache.lucene.search.*;
> import org.apache.lucene.queryParser.QueryParser;
>
> public class Bug1 {
> public static void main(String[] args) throws Exception {
> String indexPath = "./bugdex";
> String fieldName = "catno";
> String fieldValue= "AB12-42 2X";
> IndexWriter writer;
>
> // An index is created by opening an IndexWriter with the
> // create argument set to true.
> writer = new IndexWriter(indexPath, null, true);
> writer.close();
>
> // now index a document
> try {
> writer = new IndexWriter(indexPath, new
> SimpleAnalyzer(), false);
>
> Document doc = new Document();
> // add id as unindexed - can't search, but it is stored
> doc.add(Field.UnIndexed("id", "12345"));
> doc.add(new Field(fieldName,fieldValue,false,true,false));
> writer.addDocument(doc);
> writer.close();
>
> } catch (Exception e) {
> e.printStackTrace();
> }
>
> // now make a query and search for document
> Searcher searcher = new IndexSearcher(indexPath);
> Query query = QueryParser.parse(fieldValue, fieldName,
> new SimpleAnalyzer());
> Hits hits = searcher.search(query);
> for (int i=0; i<hits.length(); i++) {
> System.out.println(
> "id "+hits.doc(i).get("id") + "; Score: " + hits.score(i));
> }
> System.out.println("\nNumber of hits: "+ hits.length());
> }
> }
>
>
>
> ----- Original Message -----
> From: Doug Cutting <DCutting@grandcentral.com>
> To: 'Lucene Users List' <lucene-user@jakarta.apache.org>
> Sent: Monday, November 26, 2001 2:42 PM
> Subject: RE: Attribute Search
>
>
> > > From: New, Cecil (GEAE) [mailto:cecil.new@ae.ge.com]
> > >
> > > this is exactly what I was doing. Store=false, index=true,
> > > and token=false.
> > >
> > > It appeared to work ok, but searches *never* returned any hits.
> > >
> > > That's why I suspect it is a bug.
> >
> > If you think this is a bug, please submit a test case, as a
> simple class
> > whose 'main()' method illustrates the problem.
> >
> > Doug
> >
> > --
> > To unsubscribe, e-mail:
> <mailto:lucene-user-unsubscribe@jakarta.apache.org>
> > For additional commands, e-mail:
> <mailto:lucene-user-help@jakarta.apache.org>
> >
>
>
>
> --
> To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>

--
To unsubscribe, e-mail: <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>