Mailing List Archive

re: indexed v. non-indexed fields [was: Indexing part number]
This is because you have added some documents with "originType" as an
indexed field, and others with "originType" as an un-indexed field. Did you
change this in your code after your index was created? That is not
currently permitted.

As I think about it, I don't think this should be an error. Please try
making the following change to index/FieldInfos.java. Change the definition
of add(String,boolean) to:

private void add(String name, boolean isIndexed) {
FieldInfo fi = fieldInfo(name);
if (fi == null)
addInternal(name, isIndexed);
else if (isIndexed) // if isIndexed in any seg
fi.isIndexed = true; // isIndexed in merged
}

That should fix your problem, and I don't think it will cause any others.
Please send an email to lucene-dev@jakarta.apache.org telling whether this
works.

Doug

-----Original Message-----
From: Anshuman [mailto:amoni@verilytics.com]
Sent: Wednesday, October 03, 2001 10:14 PM
To: Doug Cutting; lucene-user@jakarta.apache.org
Subject: Re: Indexing part number


Hi !,

I have an incremental index and the present size is about 1.34GB. I am
getting following exception for the addition of new files in the index:

java.lang.IllegalStateException: field originType cannot be an indexed
field.
at
com.lucene.index.FieldInfos.add(java/com/lucene/index/FieldInfos.java:72)
at
com.lucene.index.FieldInfos.add(java/com/lucene/index/FieldInfos.java:63)
at
com.lucene.index.SegmentMerger.mergeFields(java/com/lucene/index/SegmentMerg
er.java:69)
at
com.lucene.index.SegmentMerger.merge(java/com/lucene/index/SegmentMerger.jav
a:53)
at
com.lucene.index.IndexWriter.mergeSegments(java/com/lucene/index/IndexWriter
.java:267)
at
com.lucene.index.IndexWriter.mergeSegments(java/com/lucene/index/IndexWriter
.java:241)
at
com.lucene.index.IndexWriter.maybeMergeSegments(java/com/lucene/index/IndexW
riter.java:230)
at
com.lucene.index.IndexWriter.addDocument(java/com/lucene/index/IndexWriter.j
ava:125)
.......

Can any one suggest me as to what can be the cause ? The originType is one
of the field on which I create index. Is it related to the size of the index
?

ASM
----- Original Message -----
From: Doug Cutting
To: 'jmbertinchamps@edpsa.com' ; lucene-user@jakarta.apache.org
Sent: Wednesday, October 03, 2001 4:25 AM
Subject: RE: Indexing part number


Try using org.apache.lucene.analysis.standard.StandardAnalyzer instead of
StopAnalyzer. This will index numbers, etc. Have a look at
StandardTokenizer.jj in the sources for details. If that grammar is not
quite right, copy it and compile your own tokenizer.

Doug

> -----Original Message-----
> From: Jean-Marc Bertinchamps [mailto:jmbertinchamps@edpsa.com]
> Sent: Tuesday, October 02, 2001 6:22 AM
> To: lucene-user@jakarta.apache.org
> Subject: Indexing part number
>
>
> Does anybody could let me know what should be changed in the
> "IndexFiles"
> demo to let met index and query "pure" digit part number.
> Currently only
> alphabetic query seem to work, digit and special characters
> (-, _, /, ...)
> are ignored.
>
> Thanks
>
> Jean-Marc Bertinchamps
>
>
Re: indexed v. non-indexed fields [was: Indexing part number] [ In reply to ]
Hi !

Thanks for the reply. Actually the exception was because of more than one process updating the incremental index, simultaneously. That caused multiple .fnm and segements file and confusion for the process updating the index.

ASM
----- Original Message -----
From: Doug Cutting
To: 'Anshuman' ; lucene-user@jakarta.apache.org
Sent: Thursday, October 04, 2001 8:55 PM
Subject: re: indexed v. non-indexed fields [was: Indexing part number]


This is because you have added some documents with "originType" as an
indexed field, and others with "originType" as an un-indexed field. Did you
change this in your code after your index was created? That is not
currently permitted.

As I think about it, I don't think this should be an error. Please try
making the following change to index/FieldInfos.java. Change the definition
of add(String,boolean) to:

private void add(String name, boolean isIndexed) {
FieldInfo fi = fieldInfo(name);
if (fi == null)
addInternal(name, isIndexed);
else if (isIndexed) // if isIndexed in any seg
fi.isIndexed = true; // isIndexed in merged
}

That should fix your problem, and I don't think it will cause any others.
Please send an email to lucene-dev@jakarta.apache.org telling whether this
works.

Doug

-----Original Message-----
From: Anshuman [mailto:amoni@verilytics.com]
Sent: Wednesday, October 03, 2001 10:14 PM
To: Doug Cutting; lucene-user@jakarta.apache.org
Subject: Re: Indexing part number


Hi !,

I have an incremental index and the present size is about 1.34GB. I am
getting following exception for the addition of new files in the index:

java.lang.IllegalStateException: field originType cannot be an indexed
field.
at
com.lucene.index.FieldInfos.add(java/com/lucene/index/FieldInfos.java:72)
at
com.lucene.index.FieldInfos.add(java/com/lucene/index/FieldInfos.java:63)
at
com.lucene.index.SegmentMerger.mergeFields(java/com/lucene/index/SegmentMerg
er.java:69)
at
com.lucene.index.SegmentMerger.merge(java/com/lucene/index/SegmentMerger.jav
a:53)
at
com.lucene.index.IndexWriter.mergeSegments(java/com/lucene/index/IndexWriter
.java:267)
at
com.lucene.index.IndexWriter.mergeSegments(java/com/lucene/index/IndexWriter
.java:241)
at
com.lucene.index.IndexWriter.maybeMergeSegments(java/com/lucene/index/IndexW
riter.java:230)
at
com.lucene.index.IndexWriter.addDocument(java/com/lucene/index/IndexWriter.j
ava:125)
.......

Can any one suggest me as to what can be the cause ? The originType is one
of the field on which I create index. Is it related to the size of the index
?

ASM
----- Original Message -----
From: Doug Cutting
To: 'jmbertinchamps@edpsa.com' ; lucene-user@jakarta.apache.org
Sent: Wednesday, October 03, 2001 4:25 AM
Subject: RE: Indexing part number


Try using org.apache.lucene.analysis.standard.StandardAnalyzer instead of
StopAnalyzer. This will index numbers, etc. Have a look at
StandardTokenizer.jj in the sources for details. If that grammar is not
quite right, copy it and compile your own tokenizer.

Doug

> -----Original Message-----
> From: Jean-Marc Bertinchamps [mailto:jmbertinchamps@edpsa.com]
> Sent: Tuesday, October 02, 2001 6:22 AM
> To: lucene-user@jakarta.apache.org
> Subject: Indexing part number
>
>
> Does anybody could let me know what should be changed in the
> "IndexFiles"
> demo to let met index and query "pure" digit part number.
> Currently only
> alphabetic query seem to work, digit and special characters
> (-, _, /, ...)
> are ignored.
>
> Thanks
>
> Jean-Marc Bertinchamps
>
>