Hi,
In our project for Lucene migration from 2.4.0 to 8.11.2, we need your suggestion to address a case.
With Lucene 2.4.0, we were using the kind of below code snippet.
With Lucene 8.11.2[Written snippet below], we need to extract the startOffset & endOffset value for further some calculation similar to Lucene 2.4.0.
Is there any easy way/API to extract the values from tokenStream?
//Lucene 2.4.0
===========================================================================
Token token;
TokenStream valueStream = analyzer.tokenStream(new StringReader(fieldValue), false,true);
while ((token = valueStream.next()) != null) {
int startOffset = token.startOffset();
int endOffset = token.endOffset();
//Do some calculation based on startOffset & endOffset
}
============================================================================
//Lucene 8.11.2
========================================================================
TokenStream valueStream = analyzer.tokenStream(field, new StringReader(fieldValue));
CharTermAttribute charValueTermAttribute = valueStream.addAttribute(CharTermAttribute.class);
while (valueStream.incrementToken()) {
String termValueText = charValueTermAttribute.toString();
//How to get startOffset & endOffset as like in Lucene 2.4
//Do some calculation based on startOffset & endOffset
}
Please let me know, if there is any further information is required from my side.
Regards
Rajib
In our project for Lucene migration from 2.4.0 to 8.11.2, we need your suggestion to address a case.
With Lucene 2.4.0, we were using the kind of below code snippet.
With Lucene 8.11.2[Written snippet below], we need to extract the startOffset & endOffset value for further some calculation similar to Lucene 2.4.0.
Is there any easy way/API to extract the values from tokenStream?
//Lucene 2.4.0
===========================================================================
Token token;
TokenStream valueStream = analyzer.tokenStream(new StringReader(fieldValue), false,true);
while ((token = valueStream.next()) != null) {
int startOffset = token.startOffset();
int endOffset = token.endOffset();
//Do some calculation based on startOffset & endOffset
}
============================================================================
//Lucene 8.11.2
========================================================================
TokenStream valueStream = analyzer.tokenStream(field, new StringReader(fieldValue));
CharTermAttribute charValueTermAttribute = valueStream.addAttribute(CharTermAttribute.class);
while (valueStream.incrementToken()) {
String termValueText = charValueTermAttribute.toString();
//How to get startOffset & endOffset as like in Lucene 2.4
//Do some calculation based on startOffset & endOffset
}
Please let me know, if there is any further information is required from my side.
Regards
Rajib