Mailing List Archive

About custom score using Solr8/Lucene8
Hi,

i was trying to implement custom score inside my solr8 instance.
On google i found some examples, all focused on using CustomScoreProvider and CustomScoreQuery classes. This classes have been removed in Solr8/Lucene8
and, from Lucene 8 changeling, i saw a developer /contributor saying to use FunctionScoreQuery.

Googling around was unsuccessful as i found no example, so i have no clue from where to start from

I think i have a very basic need. Generate a score from field in documents using logic completely unrelated with query itself.

The only way available (until any clue…) is using org.apache.solr.search.ValueSourceParser


Is there any example, at least to understand how to start a minimal basic project?

Thanks
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Hello.

Have you checked the fourth item at
http://lucene.apache.org/core/8_0_0/MIGRATE.html ?

On Wed, May 8, 2019 at 5:46 PM Federico Pici <akirapix@libero.it.invalid>
wrote:

> Hi,
>
> i was trying to implement custom score inside my solr8 instance.
> On google i found some examples, all focused on using CustomScoreProvider
> and CustomScoreQuery classes. This classes have been removed in
> Solr8/Lucene8
> and, from Lucene 8 changeling, i saw a developer /contributor saying to
> use FunctionScoreQuery.
>
> Googling around was unsuccessful as i found no example, so i have no clue
> from where to start from
>
> I think i have a very basic need. Generate a score from field in
> documents using logic completely unrelated with query itself.
>
> The only way available (until any clue…) is using
> org.apache.solr.search.ValueSourceParser
>
>
> Is there any example, at least to understand how to start a minimal basic
> project?
>
> Thanks
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

--
Sincerely yours
Mikhail Khludnev
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Hello Mikhail,
yes i did. I read all the page at https://issues.apache.org/jira/browse/LUCENE-8099 <https://issues.apache.org/jira/browse/LUCENE-8099>

I still do not understand where/how to start to write a class to be embedded in solr8. All
available example since 2012 where written around CustomScoreQuery/CustomScoreProvider.

I really don’t know how to setup a minimal project to implement custom scoring in solr8/lucene8 using FunctionScoreQuery

> Il giorno 8 mag 2019, alle ore 22:15, Mikhail Khludnev <mkhl@apache.org> ha scritto:
>
> Hello.
>
> Have you checked the fourth item at
> http://lucene.apache.org/core/8_0_0/MIGRATE.html ?
>
> On Wed, May 8, 2019 at 5:46 PM Federico Pici <akirapix@libero.it.invalid>
> wrote:
>
>> Hi,
>>
>> i was trying to implement custom score inside my solr8 instance.
>> On google i found some examples, all focused on using CustomScoreProvider
>> and CustomScoreQuery classes. This classes have been removed in
>> Solr8/Lucene8
>> and, from Lucene 8 changeling, i saw a developer /contributor saying to
>> use FunctionScoreQuery.
>>
>> Googling around was unsuccessful as i found no example, so i have no clue
>> from where to start from
>>
>> I think i have a very basic need. Generate a score from field in
>> documents using logic completely unrelated with query itself.
>>
>> The only way available (until any clue…) is using
>> org.apache.solr.search.ValueSourceParser
>>
>>
>> Is there any example, at least to understand how to start a minimal basic
>> project?
>>
>> Thanks
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> --
> Sincerely yours
> Mikhail Khludnev
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Hi Federico,

Could you give more details on what is the score that you want to produce? how big is the collection?
Is the score affected by the query? if not, would be possible to store the score with the document?

Cheers,
Diego

From: java-user@lucene.apache.org At: 05/09/19 08:13:35To: java-user@lucene.apache.org
Subject: Re: About custom score using Solr8/Lucene8

Hello Mikhail,
yes i did. I read all the page at
https://issues.apache.org/jira/browse/LUCENE-8099
<https://issues.apache.org/jira/browse/LUCENE-8099>

I still do not understand where/how to start to write a class to be embedded in
solr8. All
available example since 2012 where written around
CustomScoreQuery/CustomScoreProvider.

I really don’t know how to setup a minimal project to implement custom scoring
in solr8/lucene8 using FunctionScoreQuery

> Il giorno 8 mag 2019, alle ore 22:15, Mikhail Khludnev <mkhl@apache.org> ha
scritto:
>
> Hello.
>
> Have you checked the fourth item at
> http://lucene.apache.org/core/8_0_0/MIGRATE.html ?
>
> On Wed, May 8, 2019 at 5:46 PM Federico Pici <akirapix@libero.it.invalid>
> wrote:
>
>> Hi,
>>
>> i was trying to implement custom score inside my solr8 instance.
>> On google i found some examples, all focused on using CustomScoreProvider
>> and CustomScoreQuery classes. This classes have been removed in
>> Solr8/Lucene8
>> and, from Lucene 8 changeling, i saw a developer /contributor saying to
>> use FunctionScoreQuery.
>>
>> Googling around was unsuccessful as i found no example, so i have no clue
>> from where to start from
>>
>> I think i have a very basic need. Generate a score from field in
>> documents using logic completely unrelated with query itself.
>>
>> The only way available (until any clue…) is using
>> org.apache.solr.search.ValueSourceParser
>>
>>
>> Is there any example, at least to understand how to start a minimal basic
>> project?
>>
>> Thanks
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> --
> Sincerely yours
> Mikhail Khludnev
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Well,

my index is basically a list of documents of users. so i could say i got a couple of millions of documents per user.

I need to produce a score that is unrelated with search term (something like “give me documents that are the most important to user”). I would
search for documents of a user without specifying a particular term. Documents that are the most important for a user are calculated with strategy based
on particular text analysis (we still do not have a strategy).

I evaluated the idea of storing a score for each document based on an external scoring strategy (a job that reindex each user’s document every night?), but
i do not know how could scale up (more user with many documents could make this strategy problematic in the long time), so the idea was to be able
to calculate this scoring (or other scoring strategy) trying to follow some best-practice (and as i can see using CustomScoreQuery is the most preferred way)

I was not able to try a solution because of Lucene8 missing CustomScoreQuery.
I made something very basic using ValueSourceParser (but is this a good practice?)



> Il giorno 9 mag 2019, alle ore 10:50, Diego Ceccarelli (BLOOMBERG/ LONDON) <dceccarelli4@bloomberg.net> ha scritto:
>
> Hi Federico,
>
> Could you give more details on what is the score that you want to produce? how big is the collection?
> Is the score affected by the query? if not, would be possible to store the score with the document?
>
> Cheers,
> Diego
>
> From: java-user@lucene.apache.org At: 05/09/19 08:13:35To: java-user@lucene.apache.org
> Subject: Re: About custom score using Solr8/Lucene8
>
> Hello Mikhail,
> yes i did. I read all the page at
> https://issues.apache.org/jira/browse/LUCENE-8099
> <https://issues.apache.org/jira/browse/LUCENE-8099>
>
> I still do not understand where/how to start to write a class to be embedded in
> solr8. All
> available example since 2012 where written around
> CustomScoreQuery/CustomScoreProvider.
>
> I really don’t know how to setup a minimal project to implement custom scoring
> in solr8/lucene8 using FunctionScoreQuery
>
>> Il giorno 8 mag 2019, alle ore 22:15, Mikhail Khludnev <mkhl@apache.org> ha
> scritto:
>>
>> Hello.
>>
>> Have you checked the fourth item at
>> http://lucene.apache.org/core/8_0_0/MIGRATE.html ?
>>
>> On Wed, May 8, 2019 at 5:46 PM Federico Pici <akirapix@libero.it.invalid>
>> wrote:
>>
>>> Hi,
>>>
>>> i was trying to implement custom score inside my solr8 instance.
>>> On google i found some examples, all focused on using CustomScoreProvider
>>> and CustomScoreQuery classes. This classes have been removed in
>>> Solr8/Lucene8
>>> and, from Lucene 8 changeling, i saw a developer /contributor saying to
>>> use FunctionScoreQuery.
>>>
>>> Googling around was unsuccessful as i found no example, so i have no clue
>>> from where to start from
>>>
>>> I think i have a very basic need. Generate a score from field in
>>> documents using logic completely unrelated with query itself.
>>>
>>> The only way available (until any clue…) is using
>>> org.apache.solr.search.ValueSourceParser
>>>
>>>
>>> Is there any example, at least to understand how to start a minimal basic
>>> project?
>>>
>>> Thanks
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> --
>> Sincerely yours
>> Mikhail Khludnev
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: About custom score using Solr8/Lucene8 [ In reply to ]
why would you need to reindex all the users documents everynight? is the score affected by the number of documents per user?

Also for scores, you might be able to use in-place updates:

https://lucene.apache.org/solr/guide/6_6/updating-parts-of-documents.html#UpdatingPartsofDocuments-In-PlaceUpdates

so no need to reindex all the document.


From: java-user@lucene.apache.org At: 05/09/19 11:07:37To: Diego Ceccarelli (BLOOMBERG/ LONDON ) , java-user@lucene.apache.org
Subject: Re: About custom score using Solr8/Lucene8

Well,

my index is basically a list of documents of users. so i could say i got a
couple of millions of documents per user.

I need to produce a score that is unrelated with search term (something like
“give me documents that are the most important to user”). I would
search for documents of a user without specifying a particular term. Documents
that are the most important for a user are calculated with strategy based
on particular text analysis (we still do not have a strategy).

I evaluated the idea of storing a score for each document based on an external
scoring strategy (a job that reindex each user’s document every night?), but
i do not know how could scale up (more user with many documents could make this
strategy problematic in the long time), so the idea was to be able
to calculate this scoring (or other scoring strategy) trying to follow some
best-practice (and as i can see using CustomScoreQuery is the most preferred
way)

I was not able to try a solution because of Lucene8 missing CustomScoreQuery.
I made something very basic using ValueSourceParser (but is this a good
practice?)


> Il giorno 9 mag 2019, alle ore 10:50, Diego Ceccarelli (BLOOMBERG/ LONDON)
<dceccarelli4@bloomberg.net> ha scritto:
>
> Hi Federico,
>
> Could you give more details on what is the score that you want to produce?
how big is the collection?
> Is the score affected by the query? if not, would be possible to store the
score with the document?
>
> Cheers,
> Diego
>
> From: java-user@lucene.apache.org At: 05/09/19 08:13:35To:
java-user@lucene.apache.org
> Subject: Re: About custom score using Solr8/Lucene8
>
> Hello Mikhail,
> yes i did. I read all the page at
> https://issues.apache.org/jira/browse/LUCENE-8099
> <https://issues.apache.org/jira/browse/LUCENE-8099>
>
> I still do not understand where/how to start to write a class to be embedded
in
> solr8. All
> available example since 2012 where written around
> CustomScoreQuery/CustomScoreProvider.
>
> I really don’t know how to setup a minimal project to implement custom
scoring
> in solr8/lucene8 using FunctionScoreQuery
>
>> Il giorno 8 mag 2019, alle ore 22:15, Mikhail Khludnev <mkhl@apache.org> ha
> scritto:
>>
>> Hello.
>>
>> Have you checked the fourth item at
>> http://lucene.apache.org/core/8_0_0/MIGRATE.html ?
>>
>> On Wed, May 8, 2019 at 5:46 PM Federico Pici <akirapix@libero.it.invalid>
>> wrote:
>>
>>> Hi,
>>>
>>> i was trying to implement custom score inside my solr8 instance.
>>> On google i found some examples, all focused on using CustomScoreProvider
>>> and CustomScoreQuery classes. This classes have been removed in
>>> Solr8/Lucene8
>>> and, from Lucene 8 changeling, i saw a developer /contributor saying to
>>> use FunctionScoreQuery.
>>>
>>> Googling around was unsuccessful as i found no example, so i have no clue
>>> from where to start from
>>>
>>> I think i have a very basic need. Generate a score from field in
>>> documents using logic completely unrelated with query itself.
>>>
>>> The only way available (until any clue…) is using
>>> org.apache.solr.search.ValueSourceParser
>>>
>>>
>>> Is there any example, at least to understand how to start a minimal basic
>>> project?
>>>
>>> Thanks
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> --
>> Sincerely yours
>> Mikhail Khludnev
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Yes this was an option, and it solves a part of the problem.

But let’s consider this use case:

for each user index we need to calculate some important informations based on the most relevant terms, the “starred” documents, the number of times the were opened by user (and other key factors based on documents usage).
So, maybe weekly, we should update this “global” factors that would affect each user’s document score. If this week i opened, for example, more documents of typeA and less of typeB, ALL documents of typeA would be more important
of all documents of typeB (this is just a strategy). So each time i recalculate “global” importance key factors i would need to update every user’s document to change its score.

In this case i would fear scaling up, because as users grow up (and as each user’s document number grow up) i would been updating more and more document.

That’s why i would like to separate this moments. I would calculate “global” key factors in a separate process, but i would like to use them at query time to generate different scoring strate




> Il giorno 9 mag 2019, alle ore 13:29, Diego Ceccarelli (BLOOMBERG/ LONDON) <dceccarelli4@bloomberg.net> ha scritto:
>
> why would you need to reindex all the users documents everynight? is the score affected by the number of documents per user?
>
> Also for scores, you might be able to use in-place updates:
>
> https://lucene.apache.org/solr/guide/6_6/updating-parts-of-documents.html#UpdatingPartsofDocuments-In-PlaceUpdates
>
> so no need to reindex all the document.
>
>
> From: java-user@lucene.apache.org At: 05/09/19 11:07:37To: Diego Ceccarelli (BLOOMBERG/ LONDON ) , java-user@lucene.apache.org
> Subject: Re: About custom score using Solr8/Lucene8
>
> Well,
>
> my index is basically a list of documents of users. so i could say i got a
> couple of millions of documents per user.
>
> I need to produce a score that is unrelated with search term (something like
> “give me documents that are the most important to user”). I would
> search for documents of a user without specifying a particular term. Documents
> that are the most important for a user are calculated with strategy based
> on particular text analysis (we still do not have a strategy).
>
> I evaluated the idea of storing a score for each document based on an external
> scoring strategy (a job that reindex each user’s document every night?), but
> i do not know how could scale up (more user with many documents could make this
> strategy problematic in the long time), so the idea was to be able
> to calculate this scoring (or other scoring strategy) trying to follow some
> best-practice (and as i can see using CustomScoreQuery is the most preferred
> way)
>
> I was not able to try a solution because of Lucene8 missing CustomScoreQuery.
> I made something very basic using ValueSourceParser (but is this a good
> practice?)
>
>
>> Il giorno 9 mag 2019, alle ore 10:50, Diego Ceccarelli (BLOOMBERG/ LONDON)
> <dceccarelli4@bloomberg.net> ha scritto:
>>
>> Hi Federico,
>>
>> Could you give more details on what is the score that you want to produce?
> how big is the collection?
>> Is the score affected by the query? if not, would be possible to store the
> score with the document?
>>
>> Cheers,
>> Diego
>>
>> From: java-user@lucene.apache.org At: 05/09/19 08:13:35To:
> java-user@lucene.apache.org
>> Subject: Re: About custom score using Solr8/Lucene8
>>
>> Hello Mikhail,
>> yes i did. I read all the page at
>> https://issues.apache.org/jira/browse/LUCENE-8099
>> <https://issues.apache.org/jira/browse/LUCENE-8099>
>>
>> I still do not understand where/how to start to write a class to be embedded
> in
>> solr8. All
>> available example since 2012 where written around
>> CustomScoreQuery/CustomScoreProvider.
>>
>> I really don’t know how to setup a minimal project to implement custom
> scoring
>> in solr8/lucene8 using FunctionScoreQuery
>>
>>> Il giorno 8 mag 2019, alle ore 22:15, Mikhail Khludnev <mkhl@apache.org> ha
>> scritto:
>>>
>>> Hello.
>>>
>>> Have you checked the fourth item at
>>> http://lucene.apache.org/core/8_0_0/MIGRATE.html ?
>>>
>>> On Wed, May 8, 2019 at 5:46 PM Federico Pici <akirapix@libero.it.invalid>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> i was trying to implement custom score inside my solr8 instance.
>>>> On google i found some examples, all focused on using CustomScoreProvider
>>>> and CustomScoreQuery classes. This classes have been removed in
>>>> Solr8/Lucene8
>>>> and, from Lucene 8 changeling, i saw a developer /contributor saying to
>>>> use FunctionScoreQuery.
>>>>
>>>> Googling around was unsuccessful as i found no example, so i have no clue
>>>> from where to start from
>>>>
>>>> I think i have a very basic need. Generate a score from field in
>>>> documents using logic completely unrelated with query itself.
>>>>
>>>> The only way available (until any clue…) is using
>>>> org.apache.solr.search.ValueSourceParser
>>>>
>>>>
>>>> Is there any example, at least to understand how to start a minimal basic
>>>> project?
>>>>
>>>> Thanks
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>
>>> --
>>> Sincerely yours
>>> Mikhail Khludnev
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: About custom score using Solr8/Lucene8 [ In reply to ]
What about external fields?
https://lucene.apache.org/solr/guide/6_6/working-with-external-files-and-processes.html
I admit I have not read in details all the thread, but from a superficial
reading, it could help.
Apologies if it's not helpful enough!

Cheers


On Thu, May 9, 2019 at 12:57 PM Federico Pici <akirapix@libero.it.invalid>
wrote:

> Yes this was an option, and it solves a part of the problem.
>
> But let’s consider this use case:
>
> for each user index we need to calculate some important informations based
> on the most relevant terms, the “starred” documents, the number of times
> the were opened by user (and other key factors based on documents usage).
> So, maybe weekly, we should update this “global” factors that would affect
> each user’s document score. If this week i opened, for example, more
> documents of typeA and less of typeB, ALL documents of typeA would be more
> important
> of all documents of typeB (this is just a strategy). So each time i
> recalculate “global” importance key factors i would need to update every
> user’s document to change its score.
>
> In this case i would fear scaling up, because as users grow up (and as
> each user’s document number grow up) i would been updating more and more
> document.
>
> That’s why i would like to separate this moments. I would calculate
> “global” key factors in a separate process, but i would like to use them at
> query time to generate different scoring strate
>
>
>
>
> > Il giorno 9 mag 2019, alle ore 13:29, Diego Ceccarelli (BLOOMBERG/
> LONDON) <dceccarelli4@bloomberg.net> ha scritto:
> >
> > why would you need to reindex all the users documents everynight? is the
> score affected by the number of documents per user?
> >
> > Also for scores, you might be able to use in-place updates:
> >
> >
> https://lucene.apache.org/solr/guide/6_6/updating-parts-of-documents.html#UpdatingPartsofDocuments-In-PlaceUpdates
> >
> > so no need to reindex all the document.
> >
> >
> > From: java-user@lucene.apache.org At: 05/09/19 11:07:37To: Diego
> Ceccarelli (BLOOMBERG/ LONDON ) , java-user@lucene.apache.org
> > Subject: Re: About custom score using Solr8/Lucene8
> >
> > Well,
> >
> > my index is basically a list of documents of users. so i could say i got
> a
> > couple of millions of documents per user.
> >
> > I need to produce a score that is unrelated with search term (something
> like
> > “give me documents that are the most important to user”). I would
> > search for documents of a user without specifying a particular term.
> Documents
> > that are the most important for a user are calculated with strategy
> based
> > on particular text analysis (we still do not have a strategy).
> >
> > I evaluated the idea of storing a score for each document based on an
> external
> > scoring strategy (a job that reindex each user’s document every night?),
> but
> > i do not know how could scale up (more user with many documents could
> make this
> > strategy problematic in the long time), so the idea was to be able
> > to calculate this scoring (or other scoring strategy) trying to follow
> some
> > best-practice (and as i can see using CustomScoreQuery is the most
> preferred
> > way)
> >
> > I was not able to try a solution because of Lucene8 missing
> CustomScoreQuery.
> > I made something very basic using ValueSourceParser (but is this a good
> > practice?)
> >
> >
> >> Il giorno 9 mag 2019, alle ore 10:50, Diego Ceccarelli (BLOOMBERG/
> LONDON)
> > <dceccarelli4@bloomberg.net> ha scritto:
> >>
> >> Hi Federico,
> >>
> >> Could you give more details on what is the score that you want to
> produce?
> > how big is the collection?
> >> Is the score affected by the query? if not, would be possible to store
> the
> > score with the document?
> >>
> >> Cheers,
> >> Diego
> >>
> >> From: java-user@lucene.apache.org At: 05/09/19 08:13:35To:
> > java-user@lucene.apache.org
> >> Subject: Re: About custom score using Solr8/Lucene8
> >>
> >> Hello Mikhail,
> >> yes i did. I read all the page at
> >> https://issues.apache.org/jira/browse/LUCENE-8099
> >> <https://issues.apache.org/jira/browse/LUCENE-8099>
> >>
> >> I still do not understand where/how to start to write a class to be
> embedded
> > in
> >> solr8. All
> >> available example since 2012 where written around
> >> CustomScoreQuery/CustomScoreProvider.
> >>
> >> I really don’t know how to setup a minimal project to implement custom
> > scoring
> >> in solr8/lucene8 using FunctionScoreQuery
> >>
> >>> Il giorno 8 mag 2019, alle ore 22:15, Mikhail Khludnev <
> mkhl@apache.org> ha
> >> scritto:
> >>>
> >>> Hello.
> >>>
> >>> Have you checked the fourth item at
> >>> http://lucene.apache.org/core/8_0_0/MIGRATE.html ?
> >>>
> >>> On Wed, May 8, 2019 at 5:46 PM Federico Pici
> <akirapix@libero.it.invalid>
> >>> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> i was trying to implement custom score inside my solr8 instance.
> >>>> On google i found some examples, all focused on using
> CustomScoreProvider
> >>>> and CustomScoreQuery classes. This classes have been removed in
> >>>> Solr8/Lucene8
> >>>> and, from Lucene 8 changeling, i saw a developer /contributor saying
> to
> >>>> use FunctionScoreQuery.
> >>>>
> >>>> Googling around was unsuccessful as i found no example, so i have no
> clue
> >>>> from where to start from
> >>>>
> >>>> I think i have a very basic need. Generate a score from field in
> >>>> documents using logic completely unrelated with query itself.
> >>>>
> >>>> The only way available (until any clue…) is using
> >>>> org.apache.solr.search.ValueSourceParser
> >>>>
> >>>>
> >>>> Is there any example, at least to understand how to start a minimal
> basic
> >>>> project?
> >>>>
> >>>> Thanks
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>>> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>>>
> >>>>
> >>>
> >>> --
> >>> Sincerely yours
> >>> Mikhail Khludnev
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

--
--------------------------

Benedetti Alessandro
Visiting card - http://about.me/alessandro_benedetti
Blog - http://alexbenedetti.blogspot.co.uk

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Diego it could be a nice workaround in case i was not able to perform custom scoring (or in case custom scoring function would generate a slow query)
Thanks

I still hope i will be able to implement a custom score function in some way

I was almost able to achieve such result with ValueSourceParser but i’m not sure is the best practice

> Il giorno 9 mag 2019, alle ore 16:02, Alessandro Benedetti <benedetti.alex85@gmail.com> ha scritto:
>
> What about external fields?
> https://lucene.apache.org/solr/guide/6_6/working-with-external-files-and-processes.html
> I admit I have not read in details all the thread, but from a superficial
> reading, it could help.
> Apologies if it's not helpful enough!
>
> Cheers
>
>
> On Thu, May 9, 2019 at 12:57 PM Federico Pici <akirapix@libero.it.invalid>
> wrote:
>
>> Yes this was an option, and it solves a part of the problem.
>>
>> But let’s consider this use case:
>>
>> for each user index we need to calculate some important informations based
>> on the most relevant terms, the “starred” documents, the number of times
>> the were opened by user (and other key factors based on documents usage).
>> So, maybe weekly, we should update this “global” factors that would affect
>> each user’s document score. If this week i opened, for example, more
>> documents of typeA and less of typeB, ALL documents of typeA would be more
>> important
>> of all documents of typeB (this is just a strategy). So each time i
>> recalculate “global” importance key factors i would need to update every
>> user’s document to change its score.
>>
>> In this case i would fear scaling up, because as users grow up (and as
>> each user’s document number grow up) i would been updating more and more
>> document.
>>
>> That’s why i would like to separate this moments. I would calculate
>> “global” key factors in a separate process, but i would like to use them at
>> query time to generate different scoring strate
>>
>>
>>
>>
>>> Il giorno 9 mag 2019, alle ore 13:29, Diego Ceccarelli (BLOOMBERG/
>> LONDON) <dceccarelli4@bloomberg.net> ha scritto:
>>>
>>> why would you need to reindex all the users documents everynight? is the
>> score affected by the number of documents per user?
>>>
>>> Also for scores, you might be able to use in-place updates:
>>>
>>>
>> https://lucene.apache.org/solr/guide/6_6/updating-parts-of-documents.html#UpdatingPartsofDocuments-In-PlaceUpdates
>>>
>>> so no need to reindex all the document.
>>>
>>>
>>> From: java-user@lucene.apache.org At: 05/09/19 11:07:37To: Diego
>> Ceccarelli (BLOOMBERG/ LONDON ) , java-user@lucene.apache.org
>>> Subject: Re: About custom score using Solr8/Lucene8
>>>
>>> Well,
>>>
>>> my index is basically a list of documents of users. so i could say i got
>> a
>>> couple of millions of documents per user.
>>>
>>> I need to produce a score that is unrelated with search term (something
>> like
>>> “give me documents that are the most important to user”). I would
>>> search for documents of a user without specifying a particular term.
>> Documents
>>> that are the most important for a user are calculated with strategy
>> based
>>> on particular text analysis (we still do not have a strategy).
>>>
>>> I evaluated the idea of storing a score for each document based on an
>> external
>>> scoring strategy (a job that reindex each user’s document every night?),
>> but
>>> i do not know how could scale up (more user with many documents could
>> make this
>>> strategy problematic in the long time), so the idea was to be able
>>> to calculate this scoring (or other scoring strategy) trying to follow
>> some
>>> best-practice (and as i can see using CustomScoreQuery is the most
>> preferred
>>> way)
>>>
>>> I was not able to try a solution because of Lucene8 missing
>> CustomScoreQuery.
>>> I made something very basic using ValueSourceParser (but is this a good
>>> practice?)
>>>
>>>
>>>> Il giorno 9 mag 2019, alle ore 10:50, Diego Ceccarelli (BLOOMBERG/
>> LONDON)
>>> <dceccarelli4@bloomberg.net> ha scritto:
>>>>
>>>> Hi Federico,
>>>>
>>>> Could you give more details on what is the score that you want to
>> produce?
>>> how big is the collection?
>>>> Is the score affected by the query? if not, would be possible to store
>> the
>>> score with the document?
>>>>
>>>> Cheers,
>>>> Diego
>>>>
>>>> From: java-user@lucene.apache.org At: 05/09/19 08:13:35To:
>>> java-user@lucene.apache.org
>>>> Subject: Re: About custom score using Solr8/Lucene8
>>>>
>>>> Hello Mikhail,
>>>> yes i did. I read all the page at
>>>> https://issues.apache.org/jira/browse/LUCENE-8099
>>>> <https://issues.apache.org/jira/browse/LUCENE-8099>
>>>>
>>>> I still do not understand where/how to start to write a class to be
>> embedded
>>> in
>>>> solr8. All
>>>> available example since 2012 where written around
>>>> CustomScoreQuery/CustomScoreProvider.
>>>>
>>>> I really don’t know how to setup a minimal project to implement custom
>>> scoring
>>>> in solr8/lucene8 using FunctionScoreQuery
>>>>
>>>>> Il giorno 8 mag 2019, alle ore 22:15, Mikhail Khludnev <
>> mkhl@apache.org> ha
>>>> scritto:
>>>>>
>>>>> Hello.
>>>>>
>>>>> Have you checked the fourth item at
>>>>> http://lucene.apache.org/core/8_0_0/MIGRATE.html ?
>>>>>
>>>>> On Wed, May 8, 2019 at 5:46 PM Federico Pici
>> <akirapix@libero.it.invalid>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> i was trying to implement custom score inside my solr8 instance.
>>>>>> On google i found some examples, all focused on using
>> CustomScoreProvider
>>>>>> and CustomScoreQuery classes. This classes have been removed in
>>>>>> Solr8/Lucene8
>>>>>> and, from Lucene 8 changeling, i saw a developer /contributor saying
>> to
>>>>>> use FunctionScoreQuery.
>>>>>>
>>>>>> Googling around was unsuccessful as i found no example, so i have no
>> clue
>>>>>> from where to start from
>>>>>>
>>>>>> I think i have a very basic need. Generate a score from field in
>>>>>> documents using logic completely unrelated with query itself.
>>>>>>
>>>>>> The only way available (until any clue…) is using
>>>>>> org.apache.solr.search.ValueSourceParser
>>>>>>
>>>>>>
>>>>>> Is there any example, at least to understand how to start a minimal
>> basic
>>>>>> project?
>>>>>>
>>>>>> Thanks
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Sincerely yours
>>>>> Mikhail Khludnev
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card - http://about.me/alessandro_benedetti
> Blog - http://alexbenedetti.blogspot.co.uk
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Have you seen the rerank plugin [1] ? maybe you could apply your custom scoring function only to the top documents?

[1] https://lucene.apache.org/solr/6_6_0//solr-core/org/apache/solr/search/ReRankQParserPlugin.html

From: java-user@lucene.apache.org At: 05/09/19 16:24:07To: java-user@lucene.apache.org
Subject: Re: About custom score using Solr8/Lucene8

Diego it could be a nice workaround in case i was not able to perform custom
scoring (or in case custom scoring function would generate a slow query)
Thanks

I still hope i will be able to implement a custom score function in some way

I was almost able to achieve such result with ValueSourceParser but i’m not
sure is the best practice

> Il giorno 9 mag 2019, alle ore 16:02, Alessandro Benedetti
<benedetti.alex85@gmail.com> ha scritto:
>
> What about external fields?
>
https://lucene.apache.org/solr/guide/6_6/working-with-external-files-and-process
es.html
> I admit I have not read in details all the thread, but from a superficial
> reading, it could help.
> Apologies if it's not helpful enough!
>
> Cheers
>
>
> On Thu, May 9, 2019 at 12:57 PM Federico Pici <akirapix@libero.it.invalid>
> wrote:
>
>> Yes this was an option, and it solves a part of the problem.
>>
>> But let’s consider this use case:
>>
>> for each user index we need to calculate some important informations based
>> on the most relevant terms, the “starred” documents, the number of times
>> the were opened by user (and other key factors based on documents usage).
>> So, maybe weekly, we should update this “global” factors that would affect
>> each user’s document score. If this week i opened, for example, more
>> documents of typeA and less of typeB, ALL documents of typeA would be more
>> important
>> of all documents of typeB (this is just a strategy). So each time i
>> recalculate “global” importance key factors i would need to update every
>> user’s document to change its score.
>>
>> In this case i would fear scaling up, because as users grow up (and as
>> each user’s document number grow up) i would been updating more and more
>> document.
>>
>> That’s why i would like to separate this moments. I would calculate
>> “global” key factors in a separate process, but i would like to use them at
>> query time to generate different scoring strate
>>
>>
>>
>>
>>> Il giorno 9 mag 2019, alle ore 13:29, Diego Ceccarelli (BLOOMBERG/
>> LONDON) <dceccarelli4@bloomberg.net> ha scritto:
>>>
>>> why would you need to reindex all the users documents everynight? is the
>> score affected by the number of documents per user?
>>>
>>> Also for scores, you might be able to use in-place updates:
>>>
>>>
>>
https://lucene.apache.org/solr/guide/6_6/updating-parts-of-documents.html#Updati
ngPartsofDocuments-In-PlaceUpdates
>>>
>>> so no need to reindex all the document.
>>>
>>>
>>> From: java-user@lucene.apache.org At: 05/09/19 11:07:37To: Diego
>> Ceccarelli (BLOOMBERG/ LONDON ) , java-user@lucene.apache.org
>>> Subject: Re: About custom score using Solr8/Lucene8
>>>
>>> Well,
>>>
>>> my index is basically a list of documents of users. so i could say i got
>> a
>>> couple of millions of documents per user.
>>>
>>> I need to produce a score that is unrelated with search term (something
>> like
>>> “give me documents that are the most important to user”). I would
>>> search for documents of a user without specifying a particular term.
>> Documents
>>> that are the most important for a user are calculated with strategy
>> based
>>> on particular text analysis (we still do not have a strategy).
>>>
>>> I evaluated the idea of storing a score for each document based on an
>> external
>>> scoring strategy (a job that reindex each user’s document every night?),
>> but
>>> i do not know how could scale up (more user with many documents could
>> make this
>>> strategy problematic in the long time), so the idea was to be able
>>> to calculate this scoring (or other scoring strategy) trying to follow
>> some
>>> best-practice (and as i can see using CustomScoreQuery is the most
>> preferred
>>> way)
>>>
>>> I was not able to try a solution because of Lucene8 missing
>> CustomScoreQuery.
>>> I made something very basic using ValueSourceParser (but is this a good
>>> practice?)
>>>
>>>
>>>> Il giorno 9 mag 2019, alle ore 10:50, Diego Ceccarelli (BLOOMBERG/
>> LONDON)
>>> <dceccarelli4@bloomberg.net> ha scritto:
>>>>
>>>> Hi Federico,
>>>>
>>>> Could you give more details on what is the score that you want to
>> produce?
>>> how big is the collection?
>>>> Is the score affected by the query? if not, would be possible to store
>> the
>>> score with the document?
>>>>
>>>> Cheers,
>>>> Diego
>>>>
>>>> From: java-user@lucene.apache.org At: 05/09/19 08:13:35To:
>>> java-user@lucene.apache.org
>>>> Subject: Re: About custom score using Solr8/Lucene8
>>>>
>>>> Hello Mikhail,
>>>> yes i did. I read all the page at
>>>> https://issues.apache.org/jira/browse/LUCENE-8099
>>>> <https://issues.apache.org/jira/browse/LUCENE-8099>
>>>>
>>>> I still do not understand where/how to start to write a class to be
>> embedded
>>> in
>>>> solr8. All
>>>> available example since 2012 where written around
>>>> CustomScoreQuery/CustomScoreProvider.
>>>>
>>>> I really don’t know how to setup a minimal project to implement custom
>>> scoring
>>>> in solr8/lucene8 using FunctionScoreQuery
>>>>
>>>>> Il giorno 8 mag 2019, alle ore 22:15, Mikhail Khludnev <
>> mkhl@apache.org> ha
>>>> scritto:
>>>>>
>>>>> Hello.
>>>>>
>>>>> Have you checked the fourth item at
>>>>> http://lucene.apache.org/core/8_0_0/MIGRATE.html ?
>>>>>
>>>>> On Wed, May 8, 2019 at 5:46 PM Federico Pici
>> <akirapix@libero.it.invalid>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> i was trying to implement custom score inside my solr8 instance.
>>>>>> On google i found some examples, all focused on using
>> CustomScoreProvider
>>>>>> and CustomScoreQuery classes. This classes have been removed in
>>>>>> Solr8/Lucene8
>>>>>> and, from Lucene 8 changeling, i saw a developer /contributor saying
>> to
>>>>>> use FunctionScoreQuery.
>>>>>>
>>>>>> Googling around was unsuccessful as i found no example, so i have no
>> clue
>>>>>> from where to start from
>>>>>>
>>>>>> I think i have a very basic need. Generate a score from field in
>>>>>> documents using logic completely unrelated with query itself.
>>>>>>
>>>>>> The only way available (until any clue…) is using
>>>>>> org.apache.solr.search.ValueSourceParser
>>>>>>
>>>>>>
>>>>>> Is there any example, at least to understand how to start a minimal
>> basic
>>>>>> project?
>>>>>>
>>>>>> Thanks
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Sincerely yours
>>>>> Mikhail Khludnev
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card - http://about.me/alessandro_benedetti
> Blog - http://alexbenedetti.blogspot.co.uk
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Yes i saw it. But it allows me to give a different ranking (and a different order) for the top N documents that match a search.

We would like to give documents an order in absence of a query.

to be more precise if do something like /solr/collection_1/select/?q=*.*

i normally wouldn’t get a score.

Using my (hopefully) custom score i would be able to give documents a “value” or “score” (based on factors that are outside search terms).

For example with a /solr/collection_1/select/?q=*.*
first documents could be the ones a user viewed most and that contains a certain quantity of information(so not strictly related to a search)

My strategy must be applied to all documents (and not the first one matched by a query and re-ranked)


> Il giorno 9 mag 2019, alle ore 17:40, Diego Ceccarelli (BLOOMBERG/ LONDON) <dceccarelli4@bloomberg.net> ha scritto:
>
> Have you seen the rerank plugin [1] ? maybe you could apply your custom scoring function only to the top documents?
>
> [1] https://lucene.apache.org/solr/6_6_0//solr-core/org/apache/solr/search/ReRankQParserPlugin.html
>
> From: java-user@lucene.apache.org At: 05/09/19 16:24:07To: java-user@lucene.apache.org
> Subject: Re: About custom score using Solr8/Lucene8
>
> Diego it could be a nice workaround in case i was not able to perform custom
> scoring (or in case custom scoring function would generate a slow query)
> Thanks
>
> I still hope i will be able to implement a custom score function in some way
>
> I was almost able to achieve such result with ValueSourceParser but i’m not
> sure is the best practice
>
>> Il giorno 9 mag 2019, alle ore 16:02, Alessandro Benedetti
> <benedetti.alex85@gmail.com> ha scritto:
>>
>> What about external fields?
>>
> https://lucene.apache.org/solr/guide/6_6/working-with-external-files-and-process
> es.html
>> I admit I have not read in details all the thread, but from a superficial
>> reading, it could help.
>> Apologies if it's not helpful enough!
>>
>> Cheers
>>
>>
>> On Thu, May 9, 2019 at 12:57 PM Federico Pici <akirapix@libero.it.invalid>
>> wrote:
>>
>>> Yes this was an option, and it solves a part of the problem.
>>>
>>> But let’s consider this use case:
>>>
>>> for each user index we need to calculate some important informations based
>>> on the most relevant terms, the “starred” documents, the number of times
>>> the were opened by user (and other key factors based on documents usage).
>>> So, maybe weekly, we should update this “global” factors that would affect
>>> each user’s document score. If this week i opened, for example, more
>>> documents of typeA and less of typeB, ALL documents of typeA would be more
>>> important
>>> of all documents of typeB (this is just a strategy). So each time i
>>> recalculate “global” importance key factors i would need to update every
>>> user’s document to change its score.
>>>
>>> In this case i would fear scaling up, because as users grow up (and as
>>> each user’s document number grow up) i would been updating more and more
>>> document.
>>>
>>> That’s why i would like to separate this moments. I would calculate
>>> “global” key factors in a separate process, but i would like to use them at
>>> query time to generate different scoring strate
>>>
>>>
>>>
>>>
>>>> Il giorno 9 mag 2019, alle ore 13:29, Diego Ceccarelli (BLOOMBERG/
>>> LONDON) <dceccarelli4@bloomberg.net> ha scritto:
>>>>
>>>> why would you need to reindex all the users documents everynight? is the
>>> score affected by the number of documents per user?
>>>>
>>>> Also for scores, you might be able to use in-place updates:
>>>>
>>>>
>>>
> https://lucene.apache.org/solr/guide/6_6/updating-parts-of-documents.html#Updati
> ngPartsofDocuments-In-PlaceUpdates
>>>>
>>>> so no need to reindex all the document.
>>>>
>>>>
>>>> From: java-user@lucene.apache.org At: 05/09/19 11:07:37To: Diego
>>> Ceccarelli (BLOOMBERG/ LONDON ) , java-user@lucene.apache.org
>>>> Subject: Re: About custom score using Solr8/Lucene8
>>>>
>>>> Well,
>>>>
>>>> my index is basically a list of documents of users. so i could say i got
>>> a
>>>> couple of millions of documents per user.
>>>>
>>>> I need to produce a score that is unrelated with search term (something
>>> like
>>>> “give me documents that are the most important to user”). I would
>>>> search for documents of a user without specifying a particular term.
>>> Documents
>>>> that are the most important for a user are calculated with strategy
>>> based
>>>> on particular text analysis (we still do not have a strategy).
>>>>
>>>> I evaluated the idea of storing a score for each document based on an
>>> external
>>>> scoring strategy (a job that reindex each user’s document every night?),
>>> but
>>>> i do not know how could scale up (more user with many documents could
>>> make this
>>>> strategy problematic in the long time), so the idea was to be able
>>>> to calculate this scoring (or other scoring strategy) trying to follow
>>> some
>>>> best-practice (and as i can see using CustomScoreQuery is the most
>>> preferred
>>>> way)
>>>>
>>>> I was not able to try a solution because of Lucene8 missing
>>> CustomScoreQuery.
>>>> I made something very basic using ValueSourceParser (but is this a good
>>>> practice?)
>>>>
>>>>
>>>>> Il giorno 9 mag 2019, alle ore 10:50, Diego Ceccarelli (BLOOMBERG/
>>> LONDON)
>>>> <dceccarelli4@bloomberg.net> ha scritto:
>>>>>
>>>>> Hi Federico,
>>>>>
>>>>> Could you give more details on what is the score that you want to
>>> produce?
>>>> how big is the collection?
>>>>> Is the score affected by the query? if not, would be possible to store
>>> the
>>>> score with the document?
>>>>>
>>>>> Cheers,
>>>>> Diego
>>>>>
>>>>> From: java-user@lucene.apache.org At: 05/09/19 08:13:35To:
>>>> java-user@lucene.apache.org
>>>>> Subject: Re: About custom score using Solr8/Lucene8
>>>>>
>>>>> Hello Mikhail,
>>>>> yes i did. I read all the page at
>>>>> https://issues.apache.org/jira/browse/LUCENE-8099
>>>>> <https://issues.apache.org/jira/browse/LUCENE-8099>
>>>>>
>>>>> I still do not understand where/how to start to write a class to be
>>> embedded
>>>> in
>>>>> solr8. All
>>>>> available example since 2012 where written around
>>>>> CustomScoreQuery/CustomScoreProvider.
>>>>>
>>>>> I really don’t know how to setup a minimal project to implement custom
>>>> scoring
>>>>> in solr8/lucene8 using FunctionScoreQuery
>>>>>
>>>>>> Il giorno 8 mag 2019, alle ore 22:15, Mikhail Khludnev <
>>> mkhl@apache.org> ha
>>>>> scritto:
>>>>>>
>>>>>> Hello.
>>>>>>
>>>>>> Have you checked the fourth item at
>>>>>> http://lucene.apache.org/core/8_0_0/MIGRATE.html ?
>>>>>>
>>>>>> On Wed, May 8, 2019 at 5:46 PM Federico Pici
>>> <akirapix@libero.it.invalid>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> i was trying to implement custom score inside my solr8 instance.
>>>>>>> On google i found some examples, all focused on using
>>> CustomScoreProvider
>>>>>>> and CustomScoreQuery classes. This classes have been removed in
>>>>>>> Solr8/Lucene8
>>>>>>> and, from Lucene 8 changeling, i saw a developer /contributor saying
>>> to
>>>>>>> use FunctionScoreQuery.
>>>>>>>
>>>>>>> Googling around was unsuccessful as i found no example, so i have no
>>> clue
>>>>>>> from where to start from
>>>>>>>
>>>>>>> I think i have a very basic need. Generate a score from field in
>>>>>>> documents using logic completely unrelated with query itself.
>>>>>>>
>>>>>>> The only way available (until any clue…) is using
>>>>>>> org.apache.solr.search.ValueSourceParser
>>>>>>>
>>>>>>>
>>>>>>> Is there any example, at least to understand how to start a minimal
>>> basic
>>>>>>> project?
>>>>>>>
>>>>>>> Thanks
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Sincerely yours
>>>>>> Mikhail Khludnev
>>>>>
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> --
>> --------------------------
>>
>> Benedetti Alessandro
>> Visiting card - http://about.me/alessandro_benedetti
>> Blog - http://alexbenedetti.blogspot.co.uk
>>
>> "Tyger, tyger burning bright
>> In the forests of the night,
>> What immortal hand or eye
>> Could frame thy fearful symmetry?"
>>
>> William Blake - Songs of Experience -1794 England
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: About custom score using Solr8/Lucene8 [ In reply to ]
@Federico Pici, did you figure out on how to produce customized score in Solr
8?



--
Sent from: http://lucene.472066.n3.nabble.com/Lucene-Java-Users-f532864.html

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Again, @Federico Pici or anybody, did you figure out how to
port CustomScoreQuery in Solr8?

On Tue, Jul 23, 2019 at 1:05 AM Xiaofei <me@xiaofei.ca> wrote:

> @Federico Pici, did you figure out on how to produce customized score in
> Solr
> 8?
>
>
>
> --
> Sent from:
> http://lucene.472066.n3.nabble.com/Lucene-Java-Users-f532864.html
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

--
Vincenzo D'Amore
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Vincenzo, are you talking about https://lucene.apache.org/core/7_4_0/queries/org/apache/lucene/queries/CustomScoreQuery.html ?

can you plug it in using a QueryParserPlugin?


From: java-user@lucene.apache.org At: 07/02/20 07:58:06To: java-user@lucene.apache.org
Subject: Re: About custom score using Solr8/Lucene8

Again, @Federico Pici or anybody, did you figure out how to
port CustomScoreQuery in Solr8?

On Tue, Jul 23, 2019 at 1:05 AM Xiaofei <me@xiaofei.ca> wrote:

> @Federico Pici, did you figure out on how to produce customized score in
> Solr
> 8?
>
>
>
> --
> Sent from:
> http://lucene.472066.n3.nabble.com/Lucene-Java-Users-f532864.html
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

--
Vincenzo D'Amore
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Hi, Vincenzo.

Have you tried to implement DoubleValuesSource ?

On Thu, Jul 2, 2020 at 9:58 AM Vincenzo D'Amore <v.damore@gmail.com> wrote:

> Again, @Federico Pici or anybody, did you figure out how to
> port CustomScoreQuery in Solr8?
>
> On Tue, Jul 23, 2019 at 1:05 AM Xiaofei <me@xiaofei.ca> wrote:
>
> > @Federico Pici, did you figure out on how to produce customized score in
> > Solr
> > 8?
> >
> >
> >
> > --
> > Sent from:
> > http://lucene.472066.n3.nabble.com/Lucene-Java-Users-f532864.html
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
> --
> Vincenzo D'Amore
>


--
Sincerely yours
Mikhail Khludnev
Re: About custom score using Solr8/Lucene8 [ In reply to ]
On Thu, Jul 2, 2020 at 10:32 AM Diego Ceccarelli (BLOOMBERG/ LONDON) <
dceccarelli4@bloomberg.net> wrote:

> Vincenzo, are you talking about
> https://lucene.apache.org/core/7_4_0/queries/org/apache/lucene/queries/CustomScoreQuery.html
> ?
>
>
Yep. I was just trying to figure it out how to port a class I have that
extends CustomScoreQuery to solr 8


> can you plug it in using a QueryParserPlugin?
>
> Yep, it's what I'm doing... Now I'm trying to move the code inside the Old
CustomScoreQuery->CustomScoreProvider.customScore()

I see the customScore() is executed for each document.

The documentation suggests to use FunctionScoreQuery and a
DoubleValuesSource.

So now I'm trying to create a custom DoubleValuesSource based on the old
class that extends CustomScoreQuery.
And inside my QueryParserPlugin, I have a Query that returns:

CustomDoubleValuesSource custom = new CustomDoubleValuesSource();
return new FunctionScoreQuery(query, custom);

If I'm doing right, the problem now is how to implement DoubleValuesSource

--
Vincenzo D'Amore
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Hi Mikhail, I was just trying to understand how to extend
DoubleValuesSource class, now I'm looking around to find an inspiring
example...

On Thu, Jul 2, 2020 at 12:55 PM Mikhail Khludnev <mkhl@apache.org> wrote:

> Hi, Vincenzo.
>
> Have you tried to implement DoubleValuesSource ?
>
> On Thu, Jul 2, 2020 at 9:58 AM Vincenzo D'Amore <v.damore@gmail.com>
> wrote:
>
> > Again, @Federico Pici or anybody, did you figure out how to
> > port CustomScoreQuery in Solr8?
> >
> > On Tue, Jul 23, 2019 at 1:05 AM Xiaofei <me@xiaofei.ca> wrote:
> >
> > > @Federico Pici, did you figure out on how to produce customized score
> in
> > > Solr
> > > 8?
> > >
> > >
> > >
> > > --
> > > Sent from:
> > > http://lucene.472066.n3.nabble.com/Lucene-Java-Users-f532864.html
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > >
> > >
> >
> > --
> > Vincenzo D'Amore
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>


--
Vincenzo D'Amore
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Hi all, I did few steps forward but still struggling in how read the field
value inside my custom DoubleValuesSource

final CustomValuesSource valuesSource = new
CustomValuesSource(data, req.getSchema().getField(field));
return FunctionScoreQuery.boostByValue(query,
valuesSource);

CustomValuesSource extends DoubleValuesSource

But, if I did right, I'm struggling with the getValues code.

public DoubleValues getValues(LeafReaderContext ctx, DoubleValues scores)
throws IOException {

The field I have to read is a binary field, and I can't find an example how
to read a binary field from LeafReaderContext

Any help appreciated.

Best regards,
Vincenzo

On Thu, Jul 2, 2020 at 1:19 PM Vincenzo D'Amore <v.damore@gmail.com> wrote:

> Hi Mikhail, I was just trying to understand how to extend
> DoubleValuesSource class, now I'm looking around to find an inspiring
> example...
>
> On Thu, Jul 2, 2020 at 12:55 PM Mikhail Khludnev <mkhl@apache.org> wrote:
>
>> Hi, Vincenzo.
>>
>> Have you tried to implement DoubleValuesSource ?
>>
>> On Thu, Jul 2, 2020 at 9:58 AM Vincenzo D'Amore <v.damore@gmail.com>
>> wrote:
>>
>> > Again, @Federico Pici or anybody, did you figure out how to
>> > port CustomScoreQuery in Solr8?
>> >
>> > On Tue, Jul 23, 2019 at 1:05 AM Xiaofei <me@xiaofei.ca> wrote:
>> >
>> > > @Federico Pici, did you figure out on how to produce customized score
>> in
>> > > Solr
>> > > 8?
>> > >
>> > >
>> > >
>> > > --
>> > > Sent from:
>> > > http://lucene.472066.n3.nabble.com/Lucene-Java-Users-f532864.html
>> > >
>> > > ---------------------------------------------------------------------
>> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> > > For additional commands, e-mail: java-user-help@lucene.apache.org
>> > >
>> > >
>> >
>> > --
>> > Vincenzo D'Amore
>> >
>>
>>
>> --
>> Sincerely yours
>> Mikhail Khludnev
>>
>
>
> --
> Vincenzo D'Amore
>
>

--
Vincenzo D'Amore
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Hi all,

Finally I have a custom DoubleValuesSource that gives the expected results,
but I'm a little worried about the lack of documentation.

When you extend DoubleValuesSource there are a number of methods to write,
for some of them it is not clear what they do and why they need to be
implemented.
Here I've listed the mandatory methods:

public abstract DoubleValues getValues(LeafReaderContext var1,
DoubleValues var2) throws IOException;
public abstract boolean needsScores()
public abstract DoubleValuesSource rewrite(IndexSearcher var1) throws
IOException;
public boolean isCacheable(LeafReaderContext ctx);
public abstract int hashCode();
public abstract boolean equals(Object var1);

for some of them I could imagine why (hashCode() or equals()) but what
about the others?
As said, I wrote an implementation of getValues that returns the expected
results (I've compared the results with the old version), but for many
methods I've just mimed (copied) the code found in other implementations.
So why does needsScores() always return false, how to implement
correctly isCacheable() ?
Anyone could write a short description of these methods and how they
have to be implemented?

Best regards,
Vincenzo

On Sat, Jul 4, 2020 at 3:29 AM Vincenzo D'Amore <v.damore@gmail.com> wrote:

> Hi all, I did few steps forward but still struggling in how read the field
> value inside my custom DoubleValuesSource
>
> final CustomValuesSource valuesSource = new
> CustomValuesSource(data, req.getSchema().getField(field));
> return FunctionScoreQuery.boostByValue(query,
> valuesSource);
>
> CustomValuesSource extends DoubleValuesSource
>
> But, if I did right, I'm struggling with the getValues code.
>
> public DoubleValues getValues(LeafReaderContext ctx, DoubleValues scores)
> throws IOException {
>
> The field I have to read is a binary field, and I can't find an example
> how to read a binary field from LeafReaderContext
>
> Any help appreciated.
>
> Best regards,
> Vincenzo
>
> On Thu, Jul 2, 2020 at 1:19 PM Vincenzo D'Amore <v.damore@gmail.com>
> wrote:
>
>> Hi Mikhail, I was just trying to understand how to extend
>> DoubleValuesSource class, now I'm looking around to find an inspiring
>> example...
>>
>> On Thu, Jul 2, 2020 at 12:55 PM Mikhail Khludnev <mkhl@apache.org> wrote:
>>
>>> Hi, Vincenzo.
>>>
>>> Have you tried to implement DoubleValuesSource ?
>>>
>>> On Thu, Jul 2, 2020 at 9:58 AM Vincenzo D'Amore <v.damore@gmail.com>
>>> wrote:
>>>
>>> > Again, @Federico Pici or anybody, did you figure out how to
>>> > port CustomScoreQuery in Solr8?
>>> >
>>> > On Tue, Jul 23, 2019 at 1:05 AM Xiaofei <me@xiaofei.ca> wrote:
>>> >
>>> > > @Federico Pici, did you figure out on how to produce customized
>>> score in
>>> > > Solr
>>> > > 8?
>>> > >
>>> > >
>>> > >
>>> > > --
>>> > > Sent from:
>>> > > http://lucene.472066.n3.nabble.com/Lucene-Java-Users-f532864.html
>>> > >
>>> > > ---------------------------------------------------------------------
>>> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> > > For additional commands, e-mail: java-user-help@lucene.apache.org
>>> > >
>>> > >
>>> >
>>> > --
>>> > Vincenzo D'Amore
>>> >
>>>
>>>
>>> --
>>> Sincerely yours
>>> Mikhail Khludnev
>>>
>>
>>
>> --
>> Vincenzo D'Amore
>>
>>
>
> --
> Vincenzo D'Amore
>
>

--
Vincenzo D'Amore
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Did you read the DoubleValuesSource javadocs, and find they weren't enough?

On Sun, Jul 5, 2020 at 7:54 AM Vincenzo D'Amore <v.damore@gmail.com> wrote:
>
> Hi all,
>
> Finally I have a custom DoubleValuesSource that gives the expected results,
> but I'm a little worried about the lack of documentation.
>
> When you extend DoubleValuesSource there are a number of methods to write,
> for some of them it is not clear what they do and why they need to be
> implemented.
> Here I've listed the mandatory methods:
>
> public abstract DoubleValues getValues(LeafReaderContext var1,
> DoubleValues var2) throws IOException;
> public abstract boolean needsScores()
> public abstract DoubleValuesSource rewrite(IndexSearcher var1) throws
> IOException;
> public boolean isCacheable(LeafReaderContext ctx);
> public abstract int hashCode();
> public abstract boolean equals(Object var1);
>
> for some of them I could imagine why (hashCode() or equals()) but what
> about the others?
> As said, I wrote an implementation of getValues that returns the expected
> results (I've compared the results with the old version), but for many
> methods I've just mimed (copied) the code found in other implementations.
> So why does needsScores() always return false, how to implement
> correctly isCacheable() ?
> Anyone could write a short description of these methods and how they
> have to be implemented?
>
> Best regards,
> Vincenzo
>
> On Sat, Jul 4, 2020 at 3:29 AM Vincenzo D'Amore <v.damore@gmail.com> wrote:
>
> > Hi all, I did few steps forward but still struggling in how read the field
> > value inside my custom DoubleValuesSource
> >
> > final CustomValuesSource valuesSource = new
> > CustomValuesSource(data, req.getSchema().getField(field));
> > return FunctionScoreQuery.boostByValue(query,
> > valuesSource);
> >
> > CustomValuesSource extends DoubleValuesSource
> >
> > But, if I did right, I'm struggling with the getValues code.
> >
> > public DoubleValues getValues(LeafReaderContext ctx, DoubleValues scores)
> > throws IOException {
> >
> > The field I have to read is a binary field, and I can't find an example
> > how to read a binary field from LeafReaderContext
> >
> > Any help appreciated.
> >
> > Best regards,
> > Vincenzo
> >
> > On Thu, Jul 2, 2020 at 1:19 PM Vincenzo D'Amore <v.damore@gmail.com>
> > wrote:
> >
> >> Hi Mikhail, I was just trying to understand how to extend
> >> DoubleValuesSource class, now I'm looking around to find an inspiring
> >> example...
> >>
> >> On Thu, Jul 2, 2020 at 12:55 PM Mikhail Khludnev <mkhl@apache.org> wrote:
> >>
> >>> Hi, Vincenzo.
> >>>
> >>> Have you tried to implement DoubleValuesSource ?
> >>>
> >>> On Thu, Jul 2, 2020 at 9:58 AM Vincenzo D'Amore <v.damore@gmail.com>
> >>> wrote:
> >>>
> >>> > Again, @Federico Pici or anybody, did you figure out how to
> >>> > port CustomScoreQuery in Solr8?
> >>> >
> >>> > On Tue, Jul 23, 2019 at 1:05 AM Xiaofei <me@xiaofei.ca> wrote:
> >>> >
> >>> > > @Federico Pici, did you figure out on how to produce customized
> >>> score in
> >>> > > Solr
> >>> > > 8?
> >>> > >
> >>> > >
> >>> > >
> >>> > > --
> >>> > > Sent from:
> >>> > > http://lucene.472066.n3.nabble.com/Lucene-Java-Users-f532864.html
> >>> > >
> >>> > > ---------------------------------------------------------------------
> >>> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>> > > For additional commands, e-mail: java-user-help@lucene.apache.org
> >>> > >
> >>> > >
> >>> >
> >>> > --
> >>> > Vincenzo D'Amore
> >>> >
> >>>
> >>>
> >>> --
> >>> Sincerely yours
> >>> Mikhail Khludnev
> >>>
> >>
> >>
> >> --
> >> Vincenzo D'Amore
> >>
> >>
> >
> > --
> > Vincenzo D'Amore
> >
> >
>
> --
> Vincenzo D'Amore

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Hi Michael, thanks for answering my questions.
Yes, I read, but I think that it not is enough.

For make the things clearer, this is taken from the javadocs:

abstract boolean needsScores() - Return true if document scores are needed
to calculate values

So, I thought return true, because yes, I need to calculate the scores for
my custom implementations.
Anyway, I should remember that the wrong way always seems more reasonable
:) , and googling around for:

"boolean needsScores" "DoubleValuesSource" site:github.com

I found that when there is explicit code many implementations returns
directly: false.

What does this mean? why and when should I return true or false?


On Mon, Jul 6, 2020 at 2:50 PM Michael Sokolov <msokolov@gmail.com> wrote:

> Did you read the DoubleValuesSource javadocs, and find they weren't enough?
>
> On Sun, Jul 5, 2020 at 7:54 AM Vincenzo D'Amore <v.damore@gmail.com>
> wrote:
> >
> > Hi all,
> >
> > Finally I have a custom DoubleValuesSource that gives the expected
> results,
> > but I'm a little worried about the lack of documentation.
> >
> > When you extend DoubleValuesSource there are a number of methods to
> write,
> > for some of them it is not clear what they do and why they need to be
> > implemented.
> > Here I've listed the mandatory methods:
> >
> > public abstract DoubleValues getValues(LeafReaderContext var1,
> > DoubleValues var2) throws IOException;
> > public abstract boolean needsScores()
> > public abstract DoubleValuesSource rewrite(IndexSearcher var1) throws
> > IOException;
> > public boolean isCacheable(LeafReaderContext ctx);
> > public abstract int hashCode();
> > public abstract boolean equals(Object var1);
> >
> > for some of them I could imagine why (hashCode() or equals()) but what
> > about the others?
> > As said, I wrote an implementation of getValues that returns the expected
> > results (I've compared the results with the old version), but for many
> > methods I've just mimed (copied) the code found in other implementations.
> > So why does needsScores() always return false, how to implement
> > correctly isCacheable() ?
> > Anyone could write a short description of these methods and how they
> > have to be implemented?
> >
> > Best regards,
> > Vincenzo
> >
> > On Sat, Jul 4, 2020 at 3:29 AM Vincenzo D'Amore <v.damore@gmail.com>
> wrote:
> >
> > > Hi all, I did few steps forward but still struggling in how read the
> field
> > > value inside my custom DoubleValuesSource
> > >
> > > final CustomValuesSource valuesSource = new
> > > CustomValuesSource(data, req.getSchema().getField(field));
> > > return FunctionScoreQuery.boostByValue(query,
> > > valuesSource);
> > >
> > > CustomValuesSource extends DoubleValuesSource
> > >
> > > But, if I did right, I'm struggling with the getValues code.
> > >
> > > public DoubleValues getValues(LeafReaderContext ctx, DoubleValues
> scores)
> > > throws IOException {
> > >
> > > The field I have to read is a binary field, and I can't find an example
> > > how to read a binary field from LeafReaderContext
> > >
> > > Any help appreciated.
> > >
> > > Best regards,
> > > Vincenzo
> > >
> > > On Thu, Jul 2, 2020 at 1:19 PM Vincenzo D'Amore <v.damore@gmail.com>
> > > wrote:
> > >
> > >> Hi Mikhail, I was just trying to understand how to extend
> > >> DoubleValuesSource class, now I'm looking around to find an inspiring
> > >> example...
> > >>
> > >> On Thu, Jul 2, 2020 at 12:55 PM Mikhail Khludnev <mkhl@apache.org>
> wrote:
> > >>
> > >>> Hi, Vincenzo.
> > >>>
> > >>> Have you tried to implement DoubleValuesSource ?
> > >>>
> > >>> On Thu, Jul 2, 2020 at 9:58 AM Vincenzo D'Amore <v.damore@gmail.com>
> > >>> wrote:
> > >>>
> > >>> > Again, @Federico Pici or anybody, did you figure out how to
> > >>> > port CustomScoreQuery in Solr8?
> > >>> >
> > >>> > On Tue, Jul 23, 2019 at 1:05 AM Xiaofei <me@xiaofei.ca> wrote:
> > >>> >
> > >>> > > @Federico Pici, did you figure out on how to produce customized
> > >>> score in
> > >>> > > Solr
> > >>> > > 8?
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> > > --
> > >>> > > Sent from:
> > >>> > >
> http://lucene.472066.n3.nabble.com/Lucene-Java-Users-f532864.html
> > >>> > >
> > >>> > >
> ---------------------------------------------------------------------
> > >>> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > >>> > > For additional commands, e-mail:
> java-user-help@lucene.apache.org
> > >>> > >
> > >>> > >
> > >>> >
> > >>> > --
> > >>> > Vincenzo D'Amore
> > >>> >
> > >>>
> > >>>
> > >>> --
> > >>> Sincerely yours
> > >>> Mikhail Khludnev
> > >>>
> > >>
> > >>
> > >> --
> > >> Vincenzo D'Amore
> > >>
> > >>
> > >
> > > --
> > > Vincenzo D'Amore
> > >
> > >
> >
> > --
> > Vincenzo D'Amore
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

--
Vincenzo D'Amore
Re: About custom score using Solr8/Lucene8 [ In reply to ]
That controls whether getValues(LeafReaderContext ctx, DoubleValues
scores) gets a null scores parameter or not. You should say true only
if you need the text relevance scores computed by the Query's Scorer.

On Mon, Jul 6, 2020 at 10:22 AM Vincenzo D'Amore <v.damore@gmail.com> wrote:
>
> Hi Michael, thanks for answering my questions.
> Yes, I read, but I think that it not is enough.
>
> For make the things clearer, this is taken from the javadocs:
>
> abstract boolean needsScores() - Return true if document scores are needed
> to calculate values
>
> So, I thought return true, because yes, I need to calculate the scores for
> my custom implementations.
> Anyway, I should remember that the wrong way always seems more reasonable
> :) , and googling around for:
>
> "boolean needsScores" "DoubleValuesSource" site:github.com
>
> I found that when there is explicit code many implementations returns
> directly: false.
>
> What does this mean? why and when should I return true or false?
>
>
> On Mon, Jul 6, 2020 at 2:50 PM Michael Sokolov <msokolov@gmail.com> wrote:
>
> > Did you read the DoubleValuesSource javadocs, and find they weren't enough?
> >
> > On Sun, Jul 5, 2020 at 7:54 AM Vincenzo D'Amore <v.damore@gmail.com>
> > wrote:
> > >
> > > Hi all,
> > >
> > > Finally I have a custom DoubleValuesSource that gives the expected
> > results,
> > > but I'm a little worried about the lack of documentation.
> > >
> > > When you extend DoubleValuesSource there are a number of methods to
> > write,
> > > for some of them it is not clear what they do and why they need to be
> > > implemented.
> > > Here I've listed the mandatory methods:
> > >
> > > public abstract DoubleValues getValues(LeafReaderContext var1,
> > > DoubleValues var2) throws IOException;
> > > public abstract boolean needsScores()
> > > public abstract DoubleValuesSource rewrite(IndexSearcher var1) throws
> > > IOException;
> > > public boolean isCacheable(LeafReaderContext ctx);
> > > public abstract int hashCode();
> > > public abstract boolean equals(Object var1);
> > >
> > > for some of them I could imagine why (hashCode() or equals()) but what
> > > about the others?
> > > As said, I wrote an implementation of getValues that returns the expected
> > > results (I've compared the results with the old version), but for many
> > > methods I've just mimed (copied) the code found in other implementations.
> > > So why does needsScores() always return false, how to implement
> > > correctly isCacheable() ?
> > > Anyone could write a short description of these methods and how they
> > > have to be implemented?
> > >
> > > Best regards,
> > > Vincenzo
> > >
> > > On Sat, Jul 4, 2020 at 3:29 AM Vincenzo D'Amore <v.damore@gmail.com>
> > wrote:
> > >
> > > > Hi all, I did few steps forward but still struggling in how read the
> > field
> > > > value inside my custom DoubleValuesSource
> > > >
> > > > final CustomValuesSource valuesSource = new
> > > > CustomValuesSource(data, req.getSchema().getField(field));
> > > > return FunctionScoreQuery.boostByValue(query,
> > > > valuesSource);
> > > >
> > > > CustomValuesSource extends DoubleValuesSource
> > > >
> > > > But, if I did right, I'm struggling with the getValues code.
> > > >
> > > > public DoubleValues getValues(LeafReaderContext ctx, DoubleValues
> > scores)
> > > > throws IOException {
> > > >
> > > > The field I have to read is a binary field, and I can't find an example
> > > > how to read a binary field from LeafReaderContext
> > > >
> > > > Any help appreciated.
> > > >
> > > > Best regards,
> > > > Vincenzo
> > > >
> > > > On Thu, Jul 2, 2020 at 1:19 PM Vincenzo D'Amore <v.damore@gmail.com>
> > > > wrote:
> > > >
> > > >> Hi Mikhail, I was just trying to understand how to extend
> > > >> DoubleValuesSource class, now I'm looking around to find an inspiring
> > > >> example...
> > > >>
> > > >> On Thu, Jul 2, 2020 at 12:55 PM Mikhail Khludnev <mkhl@apache.org>
> > wrote:
> > > >>
> > > >>> Hi, Vincenzo.
> > > >>>
> > > >>> Have you tried to implement DoubleValuesSource ?
> > > >>>
> > > >>> On Thu, Jul 2, 2020 at 9:58 AM Vincenzo D'Amore <v.damore@gmail.com>
> > > >>> wrote:
> > > >>>
> > > >>> > Again, @Federico Pici or anybody, did you figure out how to
> > > >>> > port CustomScoreQuery in Solr8?
> > > >>> >
> > > >>> > On Tue, Jul 23, 2019 at 1:05 AM Xiaofei <me@xiaofei.ca> wrote:
> > > >>> >
> > > >>> > > @Federico Pici, did you figure out on how to produce customized
> > > >>> score in
> > > >>> > > Solr
> > > >>> > > 8?
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > --
> > > >>> > > Sent from:
> > > >>> > >
> > http://lucene.472066.n3.nabble.com/Lucene-Java-Users-f532864.html
> > > >>> > >
> > > >>> > >
> > ---------------------------------------------------------------------
> > > >>> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > >>> > > For additional commands, e-mail:
> > java-user-help@lucene.apache.org
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>> > --
> > > >>> > Vincenzo D'Amore
> > > >>> >
> > > >>>
> > > >>>
> > > >>> --
> > > >>> Sincerely yours
> > > >>> Mikhail Khludnev
> > > >>>
> > > >>
> > > >>
> > > >> --
> > > >> Vincenzo D'Amore
> > > >>
> > > >>
> > > >
> > > > --
> > > > Vincenzo D'Amore
> > > >
> > > >
> > >
> > > --
> > > Vincenzo D'Amore
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
> --
> Vincenzo D'Amore

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: About custom score using Solr8/Lucene8 [ In reply to ]
Thanks Michael,
if I understand correctly the DoubleValuesSource is stateful.
When getValues is called if scores is null an internal state true/false is
saved.
This state should be returned by needsScores method.
Is this correct?

As far as I understood, the same thing happens to the DoubleValue object
returned by the getValues() method.
DoubleValue object has an internal state which depends from advanceExact()
method.
In other words, DoubleValue.doubleValue() returns a value produced after
the call of advanceExact(int docId).

Now the problem I have is understanding if DoubleValuesSource has to be
stateful or has to store an state that should be available later.
Or, on the other hand, understand if there is an order in the methods calls
(first getValues then needsScores, first advanceExact then doubleValue).
Don't you agree?


On Mon, Jul 6, 2020 at 4:58 PM Michael Sokolov <msokolov@gmail.com> wrote:

> That controls whether getValues(LeafReaderContext ctx, DoubleValues
> scores) gets a null scores parameter or not. You should say true only
> if you need the text relevance scores computed by the Query's Scorer.
>
> On Mon, Jul 6, 2020 at 10:22 AM Vincenzo D'Amore <v.damore@gmail.com>
> wrote:
> >
> > Hi Michael, thanks for answering my questions.
> > Yes, I read, but I think that it not is enough.
> >
> > For make the things clearer, this is taken from the javadocs:
> >
> > abstract boolean needsScores() - Return true if document scores are
> needed
> > to calculate values
> >
> > So, I thought return true, because yes, I need to calculate the scores
> for
> > my custom implementations.
> > Anyway, I should remember that the wrong way always seems more reasonable
> > :) , and googling around for:
> >
> > "boolean needsScores" "DoubleValuesSource" site:github.com
> >
> > I found that when there is explicit code many implementations returns
> > directly: false.
> >
> > What does this mean? why and when should I return true or false?
> >
> >
> > On Mon, Jul 6, 2020 at 2:50 PM Michael Sokolov <msokolov@gmail.com>
> wrote:
> >
> > > Did you read the DoubleValuesSource javadocs, and find they weren't
> enough?
> > >
> > > On Sun, Jul 5, 2020 at 7:54 AM Vincenzo D'Amore <v.damore@gmail.com>
> > > wrote:
> > > >
> > > > Hi all,
> > > >
> > > > Finally I have a custom DoubleValuesSource that gives the expected
> > > results,
> > > > but I'm a little worried about the lack of documentation.
> > > >
> > > > When you extend DoubleValuesSource there are a number of methods to
> > > write,
> > > > for some of them it is not clear what they do and why they need to be
> > > > implemented.
> > > > Here I've listed the mandatory methods:
> > > >
> > > > public abstract DoubleValues getValues(LeafReaderContext var1,
> > > > DoubleValues var2) throws IOException;
> > > > public abstract boolean needsScores()
> > > > public abstract DoubleValuesSource rewrite(IndexSearcher var1)
> throws
> > > > IOException;
> > > > public boolean isCacheable(LeafReaderContext ctx);
> > > > public abstract int hashCode();
> > > > public abstract boolean equals(Object var1);
> > > >
> > > > for some of them I could imagine why (hashCode() or equals()) but
> what
> > > > about the others?
> > > > As said, I wrote an implementation of getValues that returns the
> expected
> > > > results (I've compared the results with the old version), but for
> many
> > > > methods I've just mimed (copied) the code found in other
> implementations.
> > > > So why does needsScores() always return false, how to implement
> > > > correctly isCacheable() ?
> > > > Anyone could write a short description of these methods and how they
> > > > have to be implemented?
> > > >
> > > > Best regards,
> > > > Vincenzo
> > > >
> > > > On Sat, Jul 4, 2020 at 3:29 AM Vincenzo D'Amore <v.damore@gmail.com>
> > > wrote:
> > > >
> > > > > Hi all, I did few steps forward but still struggling in how read
> the
> > > field
> > > > > value inside my custom DoubleValuesSource
> > > > >
> > > > > final CustomValuesSource valuesSource = new
> > > > > CustomValuesSource(data, req.getSchema().getField(field));
> > > > > return FunctionScoreQuery.boostByValue(query,
> > > > > valuesSource);
> > > > >
> > > > > CustomValuesSource extends DoubleValuesSource
> > > > >
> > > > > But, if I did right, I'm struggling with the getValues code.
> > > > >
> > > > > public DoubleValues getValues(LeafReaderContext ctx, DoubleValues
> > > scores)
> > > > > throws IOException {
> > > > >
> > > > > The field I have to read is a binary field, and I can't find an
> example
> > > > > how to read a binary field from LeafReaderContext
> > > > >
> > > > > Any help appreciated.
> > > > >
> > > > > Best regards,
> > > > > Vincenzo
> > > > >
> > > > > On Thu, Jul 2, 2020 at 1:19 PM Vincenzo D'Amore <
> v.damore@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> Hi Mikhail, I was just trying to understand how to extend
> > > > >> DoubleValuesSource class, now I'm looking around to find an
> inspiring
> > > > >> example...
> > > > >>
> > > > >> On Thu, Jul 2, 2020 at 12:55 PM Mikhail Khludnev <mkhl@apache.org
> >
> > > wrote:
> > > > >>
> > > > >>> Hi, Vincenzo.
> > > > >>>
> > > > >>> Have you tried to implement DoubleValuesSource ?
> > > > >>>
> > > > >>> On Thu, Jul 2, 2020 at 9:58 AM Vincenzo D'Amore <
> v.damore@gmail.com>
> > > > >>> wrote:
> > > > >>>
> > > > >>> > Again, @Federico Pici or anybody, did you figure out how to
> > > > >>> > port CustomScoreQuery in Solr8?
> > > > >>> >
> > > > >>> > On Tue, Jul 23, 2019 at 1:05 AM Xiaofei <me@xiaofei.ca> wrote:
> > > > >>> >
> > > > >>> > > @Federico Pici, did you figure out on how to produce
> customized
> > > > >>> score in
> > > > >>> > > Solr
> > > > >>> > > 8?
> > > > >>> > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > --
> > > > >>> > > Sent from:
> > > > >>> > >
> > > http://lucene.472066.n3.nabble.com/Lucene-Java-Users-f532864.html
> > > > >>> > >
> > > > >>> > >
> > > ---------------------------------------------------------------------
> > > > >>> > > To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> > > > >>> > > For additional commands, e-mail:
> > > java-user-help@lucene.apache.org
> > > > >>> > >
> > > > >>> > >
> > > > >>> >
> > > > >>> > --
> > > > >>> > Vincenzo D'Amore
> > > > >>> >
> > > > >>>
> > > > >>>
> > > > >>> --
> > > > >>> Sincerely yours
> > > > >>> Mikhail Khludnev
> > > > >>>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Vincenzo D'Amore
> > > > >>
> > > > >>
> > > > >
> > > > > --
> > > > > Vincenzo D'Amore
> > > > >
> > > > >
> > > >
> > > > --
> > > > Vincenzo D'Amore
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > >
> > >
> >
> > --
> > Vincenzo D'Amore
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

--
Vincenzo D'Amore
Re: About custom score using Solr8/Lucene8 [ In reply to ]
You should think of DoubleValuesSource as a factory for DoubleValues.
Usually a factory will be immutable - you set it up and then it
produces per-leaf DoubleValues. So I don't really understand what
you're saying about state there. Regarding the DoubleValues, which is
an iterator, yes it definitely has state: it has to keep track of
which document it is positioned on (if any). I think the order of
operations is as you listed them.


On Tue, Jul 7, 2020 at 1:11 PM Vincenzo D'Amore <v.damore@gmail.com> wrote:
>
> Thanks Michael,
> if I understand correctly the DoubleValuesSource is stateful.
> When getValues is called if scores is null an internal state true/false is
> saved.
> This state should be returned by needsScores method.
> Is this correct?
>
> As far as I understood, the same thing happens to the DoubleValue object
> returned by the getValues() method.
> DoubleValue object has an internal state which depends from advanceExact()
> method.
> In other words, DoubleValue.doubleValue() returns a value produced after
> the call of advanceExact(int docId).
>
> Now the problem I have is understanding if DoubleValuesSource has to be
> stateful or has to store an state that should be available later.
> Or, on the other hand, understand if there is an order in the methods calls
> (first getValues then needsScores, first advanceExact then doubleValue).
> Don't you agree?
>
>
> On Mon, Jul 6, 2020 at 4:58 PM Michael Sokolov <msokolov@gmail.com> wrote:
>
> > That controls whether getValues(LeafReaderContext ctx, DoubleValues
> > scores) gets a null scores parameter or not. You should say true only
> > if you need the text relevance scores computed by the Query's Scorer.
> >
> > On Mon, Jul 6, 2020 at 10:22 AM Vincenzo D'Amore <v.damore@gmail.com>
> > wrote:
> > >
> > > Hi Michael, thanks for answering my questions.
> > > Yes, I read, but I think that it not is enough.
> > >
> > > For make the things clearer, this is taken from the javadocs:
> > >
> > > abstract boolean needsScores() - Return true if document scores are
> > needed
> > > to calculate values
> > >
> > > So, I thought return true, because yes, I need to calculate the scores
> > for
> > > my custom implementations.
> > > Anyway, I should remember that the wrong way always seems more reasonable
> > > :) , and googling around for:
> > >
> > > "boolean needsScores" "DoubleValuesSource" site:github.com
> > >
> > > I found that when there is explicit code many implementations returns
> > > directly: false.
> > >
> > > What does this mean? why and when should I return true or false?
> > >
> > >
> > > On Mon, Jul 6, 2020 at 2:50 PM Michael Sokolov <msokolov@gmail.com>
> > wrote:
> > >
> > > > Did you read the DoubleValuesSource javadocs, and find they weren't
> > enough?
> > > >
> > > > On Sun, Jul 5, 2020 at 7:54 AM Vincenzo D'Amore <v.damore@gmail.com>
> > > > wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > Finally I have a custom DoubleValuesSource that gives the expected
> > > > results,
> > > > > but I'm a little worried about the lack of documentation.
> > > > >
> > > > > When you extend DoubleValuesSource there are a number of methods to
> > > > write,
> > > > > for some of them it is not clear what they do and why they need to be
> > > > > implemented.
> > > > > Here I've listed the mandatory methods:
> > > > >
> > > > > public abstract DoubleValues getValues(LeafReaderContext var1,
> > > > > DoubleValues var2) throws IOException;
> > > > > public abstract boolean needsScores()
> > > > > public abstract DoubleValuesSource rewrite(IndexSearcher var1)
> > throws
> > > > > IOException;
> > > > > public boolean isCacheable(LeafReaderContext ctx);
> > > > > public abstract int hashCode();
> > > > > public abstract boolean equals(Object var1);
> > > > >
> > > > > for some of them I could imagine why (hashCode() or equals()) but
> > what
> > > > > about the others?
> > > > > As said, I wrote an implementation of getValues that returns the
> > expected
> > > > > results (I've compared the results with the old version), but for
> > many
> > > > > methods I've just mimed (copied) the code found in other
> > implementations.
> > > > > So why does needsScores() always return false, how to implement
> > > > > correctly isCacheable() ?
> > > > > Anyone could write a short description of these methods and how they
> > > > > have to be implemented?
> > > > >
> > > > > Best regards,
> > > > > Vincenzo
> > > > >
> > > > > On Sat, Jul 4, 2020 at 3:29 AM Vincenzo D'Amore <v.damore@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Hi all, I did few steps forward but still struggling in how read
> > the
> > > > field
> > > > > > value inside my custom DoubleValuesSource
> > > > > >
> > > > > > final CustomValuesSource valuesSource = new
> > > > > > CustomValuesSource(data, req.getSchema().getField(field));
> > > > > > return FunctionScoreQuery.boostByValue(query,
> > > > > > valuesSource);
> > > > > >
> > > > > > CustomValuesSource extends DoubleValuesSource
> > > > > >
> > > > > > But, if I did right, I'm struggling with the getValues code.
> > > > > >
> > > > > > public DoubleValues getValues(LeafReaderContext ctx, DoubleValues
> > > > scores)
> > > > > > throws IOException {
> > > > > >
> > > > > > The field I have to read is a binary field, and I can't find an
> > example
> > > > > > how to read a binary field from LeafReaderContext
> > > > > >
> > > > > > Any help appreciated.
> > > > > >
> > > > > > Best regards,
> > > > > > Vincenzo
> > > > > >
> > > > > > On Thu, Jul 2, 2020 at 1:19 PM Vincenzo D'Amore <
> > v.damore@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Mikhail, I was just trying to understand how to extend
> > > > > >> DoubleValuesSource class, now I'm looking around to find an
> > inspiring
> > > > > >> example...
> > > > > >>
> > > > > >> On Thu, Jul 2, 2020 at 12:55 PM Mikhail Khludnev <mkhl@apache.org
> > >
> > > > wrote:
> > > > > >>
> > > > > >>> Hi, Vincenzo.
> > > > > >>>
> > > > > >>> Have you tried to implement DoubleValuesSource ?
> > > > > >>>
> > > > > >>> On Thu, Jul 2, 2020 at 9:58 AM Vincenzo D'Amore <
> > v.damore@gmail.com>
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>> > Again, @Federico Pici or anybody, did you figure out how to
> > > > > >>> > port CustomScoreQuery in Solr8?
> > > > > >>> >
> > > > > >>> > On Tue, Jul 23, 2019 at 1:05 AM Xiaofei <me@xiaofei.ca> wrote:
> > > > > >>> >
> > > > > >>> > > @Federico Pici, did you figure out on how to produce
> > customized
> > > > > >>> score in
> > > > > >>> > > Solr
> > > > > >>> > > 8?
> > > > > >>> > >
> > > > > >>> > >
> > > > > >>> > >
> > > > > >>> > > --
> > > > > >>> > > Sent from:
> > > > > >>> > >
> > > > http://lucene.472066.n3.nabble.com/Lucene-Java-Users-f532864.html
> > > > > >>> > >
> > > > > >>> > >
> > > > ---------------------------------------------------------------------
> > > > > >>> > > To unsubscribe, e-mail:
> > java-user-unsubscribe@lucene.apache.org
> > > > > >>> > > For additional commands, e-mail:
> > > > java-user-help@lucene.apache.org
> > > > > >>> > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>> > --
> > > > > >>> > Vincenzo D'Amore
> > > > > >>> >
> > > > > >>>
> > > > > >>>
> > > > > >>> --
> > > > > >>> Sincerely yours
> > > > > >>> Mikhail Khludnev
> > > > > >>>
> > > > > >>
> > > > > >>
> > > > > >> --
> > > > > >> Vincenzo D'Amore
> > > > > >>
> > > > > >>
> > > > > >
> > > > > > --
> > > > > > Vincenzo D'Amore
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > Vincenzo D'Amore
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > > >
> > > >
> > >
> > > --
> > > Vincenzo D'Amore
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
> --
> Vincenzo D'Amore

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org