Mailing List Archive: Lucene filter

Lucene filter

Dec 1, 2016, 10:30 PM

Post #1 of 6 (747 views)

Hi All,

I wanna use a filter into a query (BooleanClause.Occur.FILTER). For
example, my query is:

#repository:clinicaltrials +title:multipl

It looks like when I build the query from this String, the filter is not
working. In other words, the total hits in the first example below is
different from total hits in the second example below. Please, does anyone
know what wrong with this simple example?

Example 1:
String query = "#repository:clinicaltrials +title:multipl";
QueryParser qr = new QueryParser("", new StandardAnalyzer());
TopDocs hits = is.search(qr.parse(query), 1);

Example 2:
String[] fields = new String[]{"repository", "title"};
BooleanClause.Occur[] allflags = new
BooleanClause.Occur[]{BooleanClause.Occur.FILTER,
BooleanClause.Occur.MUST};
String[] query_text = new String[]{"clinicaltrials", "multipl"};
Query finalQuery = MultiFieldQueryParser.parse(query_text, fields,
allflags, new StandardAnalyzer());
TopDocs hits = is.search(finalQuery, 1);

thanks,

Best,
reda

Re: Lucene filter [ In reply to ]

mkhl at apache

Dec 1, 2016, 11:26 PM

Post #2 of 6 (742 views)

Permalink

Hello,

I don't think # is supported in query parsers, although it would be great.
So, far I saw them in only in toString().

On Fri, Dec 2, 2016 at 9:30 AM, Bouadjenek mohamed reda <
redateksystem@gmail.com> wrote:

> Hi All,
>
>
> I wanna use a filter into a query (BooleanClause.Occur.FILTER). For
> example, my query is:
>
> #repository:clinicaltrials +title:multipl
>
> It looks like when I build the query from this String, the filter is not
> working. In other words, the total hits in the first example below is
> different from total hits in the second example below. Please, does anyone
> know what wrong with this simple example?
>
> Example 1:
> String query = "#repository:clinicaltrials +title:multipl";
> QueryParser qr = new QueryParser("", new StandardAnalyzer());
> TopDocs hits = is.search(qr.parse(query), 1);
>
> Example 2:
> String[] fields = new String[]{"repository", "title"};
> BooleanClause.Occur[] allflags = new
> BooleanClause.Occur[]{BooleanClause.Occur.FILTER,
> BooleanClause.Occur.MUST};
> String[] query_text = new String[]{"clinicaltrials", "multipl"};
> Query finalQuery = MultiFieldQueryParser.parse(query_text, fields,
> allflags, new StandardAnalyzer());
> TopDocs hits = is.search(finalQuery, 1);
>
>
> thanks,
>
>
> Best,
> reda
>

--
Sincerely yours
Mikhail Khludnev

Re: Lucene filter [ In reply to ]

redateksystem at gmail

Dec 1, 2016, 11:32 PM

Post #3 of 6 (739 views)

Permalink

Hi Mikhail,

Do you have any suggestion to transform a string to a query object?
thanks,
reda

> On 2 Dec. 2016, at 18:26, Mikhail Khludnev <mkhl@apache.org> wrote:
>
> Hello,
>
> I don't think # is supported in query parsers, although it would be great.
> So, far I saw them in only in toString().
>
> On Fri, Dec 2, 2016 at 9:30 AM, Bouadjenek mohamed reda <
> redateksystem@gmail.com> wrote:
>
>> Hi All,
>>
>>
>> I wanna use a filter into a query (BooleanClause.Occur.FILTER). For
>> example, my query is:
>>
>> #repository:clinicaltrials +title:multipl
>>
>> It looks like when I build the query from this String, the filter is not
>> working. In other words, the total hits in the first example below is
>> different from total hits in the second example below. Please, does anyone
>> know what wrong with this simple example?
>>
>> Example 1:
>> String query = "#repository:clinicaltrials +title:multipl";
>> QueryParser qr = new QueryParser("", new StandardAnalyzer());
>> TopDocs hits = is.search(qr.parse(query), 1);
>>
>> Example 2:
>> String[] fields = new String[]{"repository", "title"};
>> BooleanClause.Occur[] allflags = new
>> BooleanClause.Occur[]{BooleanClause.Occur.FILTER,
>> BooleanClause.Occur.MUST};
>> String[] query_text = new String[]{"clinicaltrials", "multipl"};
>> Query finalQuery = MultiFieldQueryParser.parse(query_text, fields,
>> allflags, new StandardAnalyzer());
>> TopDocs hits = is.search(finalQuery, 1);
>>
>>
>> thanks,
>>
>>
>> Best,
>> reda
>>
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev

RE: Lucene filter [ In reply to ]

uwe at thetaphi

Dec 2, 2016, 12:38 AM

Post #4 of 6 (737 views)

Permalink

Hi,

You could use 2 query parsers, e.g., one for the user input and another one for the filters. Finally combine the 2 results into one query by combining them with an outer BooleanQuery. Having everything in one single string is quite uncommon for typical search application logic.

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Reda Kouba [mailto:redateksystem@gmail.com]
> Sent: Friday, December 2, 2016 8:33 AM
> To: general@lucene.apache.org
> Subject: Re: Lucene filter
>
> Hi Mikhail,
>
> Do you have any suggestion to transform a string to a query object?
> thanks,
> reda
>
>
> > On 2 Dec. 2016, at 18:26, Mikhail Khludnev <mkhl@apache.org> wrote:
> >
> > Hello,
> >
> > I don't think # is supported in query parsers, although it would be great.
> > So, far I saw them in only in toString().
> >
> > On Fri, Dec 2, 2016 at 9:30 AM, Bouadjenek mohamed reda <
> > redateksystem@gmail.com> wrote:
> >
> >> Hi All,
> >>
> >>
> >> I wanna use a filter into a query (BooleanClause.Occur.FILTER). For
> >> example, my query is:
> >>
> >> #repository:clinicaltrials +title:multipl
> >>
> >> It looks like when I build the query from this String, the filter is not
> >> working. In other words, the total hits in the first example below is
> >> different from total hits in the second example below. Please, does
> anyone
> >> know what wrong with this simple example?
> >>
> >> Example 1:
> >> String query = "#repository:clinicaltrials +title:multipl";
> >> QueryParser qr = new QueryParser("", new StandardAnalyzer());
> >> TopDocs hits = is.search(qr.parse(query), 1);
> >>
> >> Example 2:
> >> String[] fields = new String[]{"repository", "title"};
> >> BooleanClause.Occur[] allflags = new
> >> BooleanClause.Occur[]{BooleanClause.Occur.FILTER,
> >> BooleanClause.Occur.MUST};
> >> String[] query_text = new String[]{"clinicaltrials", "multipl"};
> >> Query finalQuery = MultiFieldQueryParser.parse(query_text, fields,
> >> allflags, new StandardAnalyzer());
> >> TopDocs hits = is.search(finalQuery, 1);
> >>
> >>
> >> thanks,
> >>
> >>
> >> Best,
> >> reda
> >>
> >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev

Re: Lucene filter [ In reply to ]

redateksystem at gmail

Dec 2, 2016, 12:41 AM

Post #5 of 6 (737 views)

Permalink

Hi Uwe,

Thank you for your suggestion. In fact, itâ€™s for an IR challenge. No intent to use my code for end-users.
best,
reda
> On 2 Dec. 2016, at 19:38, Uwe Schindler <uwe@thetaphi.de> wrote:
>
> Hi,
>
> You could use 2 query parsers, e.g., one for the user input and another one for the filters. Finally combine the 2 results into one query by combining them with an outer BooleanQuery. Having everything in one single string is quite uncommon for typical search application logic.
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>> -----Original Message-----
>> From: Reda Kouba [mailto:redateksystem@gmail.com]
>> Sent: Friday, December 2, 2016 8:33 AM
>> To: general@lucene.apache.org
>> Subject: Re: Lucene filter
>>
>> Hi Mikhail,
>>
>> Do you have any suggestion to transform a string to a query object?
>> thanks,
>> reda
>>
>>
>>> On 2 Dec. 2016, at 18:26, Mikhail Khludnev <mkhl@apache.org> wrote:
>>>
>>> Hello,
>>>
>>> I don't think # is supported in query parsers, although it would be great.
>>> So, far I saw them in only in toString().
>>>
>>> On Fri, Dec 2, 2016 at 9:30 AM, Bouadjenek mohamed reda <
>>> redateksystem@gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>>
>>>> I wanna use a filter into a query (BooleanClause.Occur.FILTER). For
>>>> example, my query is:
>>>>
>>>> #repository:clinicaltrials +title:multipl
>>>>
>>>> It looks like when I build the query from this String, the filter is not
>>>> working. In other words, the total hits in the first example below is
>>>> different from total hits in the second example below. Please, does
>> anyone
>>>> know what wrong with this simple example?
>>>>
>>>> Example 1:
>>>> String query = "#repository:clinicaltrials +title:multipl";
>>>> QueryParser qr = new QueryParser("", new StandardAnalyzer());
>>>> TopDocs hits = is.search(qr.parse(query), 1);
>>>>
>>>> Example 2:
>>>> String[] fields = new String[]{"repository", "title"};
>>>> BooleanClause.Occur[] allflags = new
>>>> BooleanClause.Occur[]{BooleanClause.Occur.FILTER,
>>>> BooleanClause.Occur.MUST};
>>>> String[] query_text = new String[]{"clinicaltrials", "multipl"};
>>>> Query finalQuery = MultiFieldQueryParser.parse(query_text, fields,
>>>> allflags, new StandardAnalyzer());
>>>> TopDocs hits = is.search(finalQuery, 1);
>>>>
>>>>
>>>> thanks,
>>>>
>>>>
>>>> Best,
>>>> reda
>>>>
>>>
>>>
>>>
>>> --
>>> Sincerely yours
>>> Mikhail Khludnev
>

Re: Lucene filter [ In reply to ]

mkhl at apache

Dec 2, 2016, 6:57 AM

Post #6 of 6 (737 views)

Permalink

Hi Reda,

You probably need to modify
org/apache/lucene/queryparser/classic/QueryParser.jj and QueryParserBase
and regenerate QueryParser.java probably.

or you can tweak flexible parser
org.apache.lucene.queryparser.flexible.precedence.processors.BooleanModifiersQueryNodeProcessor
org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser

As you see mileage is huge, and turns to a challenge itself.

On Fri, Dec 2, 2016 at 10:32 AM, Reda Kouba <redateksystem@gmail.com> wrote:

> Hi Mikhail,
>
> Do you have any suggestion to transform a string to a query object?
> thanks,
> reda
>
>
> > On 2 Dec. 2016, at 18:26, Mikhail Khludnev <mkhl@apache.org> wrote:
> >
> > Hello,
> >
> > I don't think # is supported in query parsers, although it would be
> great.
> > So, far I saw them in only in toString().
> >
> > On Fri, Dec 2, 2016 at 9:30 AM, Bouadjenek mohamed reda <
> > redateksystem@gmail.com> wrote:
> >
> >> Hi All,
> >>
> >>
> >> I wanna use a filter into a query (BooleanClause.Occur.FILTER). For
> >> example, my query is:
> >>
> >> #repository:clinicaltrials +title:multipl
> >>
> >> It looks like when I build the query from this String, the filter is not
> >> working. In other words, the total hits in the first example below is
> >> different from total hits in the second example below. Please, does
> anyone
> >> know what wrong with this simple example?
> >>
> >> Example 1:
> >> String query = "#repository:clinicaltrials +title:multipl";
> >> QueryParser qr = new QueryParser("", new StandardAnalyzer());
> >> TopDocs hits = is.search(qr.parse(query), 1);
> >>
> >> Example 2:
> >> String[] fields = new String[]{"repository", "title"};
> >> BooleanClause.Occur[] allflags = new
> >> BooleanClause.Occur[]{BooleanClause.Occur.FILTER,
> >> BooleanClause.Occur.MUST};
> >> String[] query_text = new String[]{"clinicaltrials", "multipl"};
> >> Query finalQuery = MultiFieldQueryParser.parse(query_text, fields,
> >> allflags, new StandardAnalyzer());
> >> TopDocs hits = is.search(finalQuery, 1);
> >>
> >>
> >> thanks,
> >>
> >>
> >> Best,
> >> reda
> >>
> >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
>
>

--
Sincerely yours
Mikhail Khludnev