Mailing List Archive

Hi,

In our project for Lucene migration from 2.4.0 to 8.11.2, we need your suggestion to address a case.
With Lucene 2.4.0, we were using the kind of below code snippet.
With Lucene 8.11.2[Written snippet below], we need to extract the startOffset & endOffset value for further some calculation similar to Lucene 2.4.0.
Is there any easy way/API to extract the values from tokenStream?

//Lucene 2.4.0
===========================================================================
Token token;
TokenStream valueStream = analyzer.tokenStream(new StringReader(fieldValue), false,true);
while ((token = valueStream.next()) != null) {
int startOffset = token.startOffset();
int endOffset = token.endOffset();

//Do some calculation based on startOffset & endOffset
}
============================================================================


//Lucene 8.11.2
========================================================================
TokenStream valueStream = analyzer.tokenStream(field, new StringReader(fieldValue));
CharTermAttribute charValueTermAttribute = valueStream.addAttribute(CharTermAttribute.class);
while (valueStream.incrementToken()) {
String termValueText = charValueTermAttribute.toString();

//How to get startOffset & endOffset as like in Lucene 2.4

//Do some calculation based on startOffset & endOffset
}

Please let me know, if there is any further information is required from my side.

Regards
Rajib