+1 to Tomas' proposal. Created SOLR-14907
<https://issues.apache.org/jira/browse/SOLR-14907> to track the effort.
- Houston
On Wed, Sep 30, 2020 at 12:26 PM Tomás Fernández Löbbe <
tomasflobbe@gmail.com> wrote:
> > Let's support the single file upload feature
> +1, but let this behave exactly as a zip file with a single file in it
> (regarding trusted/untrusted). We just need to change the configset handler
> to be able to handle non-zip files, and have a way to "locate" that file
> inside the configset (in case it needs to go somewhere other than the root).
>
> On Wed, Sep 30, 2020 at 8:45 AM Eric Pugh <epugh@opensourceconnections.com>
> wrote:
>
>> I think that me in “violent agreement” with you. Let’s understand the
>> Annotations approach that we have, or pick something that is commonly used
>> like JAX-RS / Jersey.
>>
>>
>>
>> On Sep 30, 2020, at 11:41 AM, Timothy Potter <thelabdude@gmail.com>
>> wrote:
>>
>> I'm sorry, I don't understand what you mean by "make it a single pattern
>> (the annotations?)" Eric?
>>
>> To me, the pattern is well established in the Java world: JAX-RS (with
>> Jersey as the underlying impl. which has nice integration with Jetty). But
>> when I suggested porting the code that uses restlet to JAX-RS / Jersey,
>> Ishan said that wasn't necessary and is already supported with some
>> Annotations ... I have no idea what that means and need more info about
>> what is already in place. Short of that, replacing restlet with JAX-RS /
>> Jersey looks like a trivial amount of work to me (and I'm happy to take it
>> on).
>>
>> Tim
>>
>> On Wed, Sep 30, 2020 at 9:36 AM Eric Pugh <
>> epugh@opensourceconnections.com> wrote:
>>
>>> The use case of “I want to update something via a API” is I think pretty
>>> common, and it would be nice to make it a single pattern (the annotations?)
>>> with lots of examples/developer docs for the next person.
>>>
>>>
>>>
>>> On Sep 30, 2020, at 11:04 AM, Timothy Potter <thelabdude@gmail.com>
>>> wrote:
>>>
>>> I started looking into removing Managed Resources in master and wanted
>>> to mention that the LTR contrib also relies on this framework
>>> (ManagedModelStore and ManagedFeatureStore, see:
>>> https://lucene.apache.org/solr/guide/8_6/learning-to-rank.html#uploading-a-model).
>>> I only mention this b/c it's been said several times in this thread that
>>> nobody uses this feature and it's only for editing config/schema like
>>> synonyms. Afaik, LTR is a broadly used feature of Solr so now I'm not so
>>> bullish on removing the ability to manage dynamic resources using a REST
>>> like API. I agree that changing resources like the synonym set could be
>>> replaced with configSet updates but I don't see how to replace the RESTful
>>> model / feature store API w/o something like Managed Resources?
>>>
>>> From where I sit, I think we should just remove the use of restlet in
>>> the implementation but keep the API for Solr 9 (master).
>>>
>>> @Ishan ~ you mentioned there is a way to get REST API like behavior w/o
>>> using JAX-RS / Jersey ... something about annotations? Can you point me to
>>> some example code of how that is done please?
>>>
>>> Cheers,
>>> Tim
>>>
>>> On Wed, Sep 30, 2020 at 8:29 AM David Smiley <dsmiley@apache.org> wrote:
>>>
>>>> These resources are fundamentally a part of the configSet and can (in
>>>> general) affect query results and thus flushing caches (via a reload) is
>>>> appropriate.
>>>>
>>>> ~ David Smiley
>>>> Apache Lucene/Solr Search Developer
>>>> http://www.linkedin.com/in/davidwsmiley
>>>>
>>>>
>>>> On Wed, Sep 30, 2020 at 9:06 AM Noble Paul <noble.paul@gmail.com>
>>>> wrote:
>>>>
>>>>> Well, I believe we should have a mechanism to upload a single file to
>>>>> a configset.
>>>>>
>>>>> > A single file configset upload would require the user to reload the
>>>>> collection, so it isn't better than managed resources.
>>>>>
>>>>> This is not true
>>>>>
>>>>> Only config/schema file changes result in core reload.
>>>>>
>>>>> On Wed, Sep 30, 2020 at 10:23 PM David Smiley <dsmiley@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > Definitely don't remove in 8.x!
>>>>> >
>>>>> > > A single file configset upload would require the user to reload
>>>>> the collection, so it isn't better than managed resources.
>>>>> >
>>>>> > Do you view that as a substantial point in favor of
>>>>> managed-resources? I view that as a trivial matter, and one I prefer to
>>>>> automagic and potentially premature reload if there are additional edits to
>>>>> be done (e.g. query-elevation or other word lists).
>>>>> >
>>>>> > ~ David Smiley
>>>>> > Apache Lucene/Solr Search Developer
>>>>> > http://www.linkedin.com/in/davidwsmiley
>>>>> >
>>>>> >
>>>>> > On Wed, Sep 30, 2020 at 5:46 AM Ishan Chattopadhyaya <
>>>>> ichattopadhyaya@gmail.com> wrote:
>>>>> >>
>>>>> >> > * Nobody knows how it works. It's unsupported
>>>>> >> It is supported and documented:
>>>>> https://lucene.apache.org/solr/guide/8_6/managed-resources.html
>>>>> >>
>>>>> >> > * RESTlet dependency
>>>>> >> > * Cannot be secured using standard permissions
>>>>> >> > * It's extremely complex for the functionality it offers.
>>>>> >>
>>>>> >> I agree. Whatever alternative we build should address these, before
>>>>> we consider removing managed resources.
>>>>> >>
>>>>> >> On Wed, Sep 30, 2020 at 2:52 PM Ishan Chattopadhyaya <
>>>>> ichattopadhyaya@gmail.com> wrote:
>>>>> >>>
>>>>> >>> The managed resources is the only reasonable way to upload
>>>>> synonyms on the fly for users today. A single file configset upload would
>>>>> require the user to reload the collection, so it isn't better than managed
>>>>> resources. I would not recommend we remove the functionality without first
>>>>> building a suitable alternative. I agree that the feature isn't built using
>>>>> proper framework or proper APIs, but it is a feature that works well.
>>>>> >>>
>>>>> >>> Usually, I support throwing features out even without existence of
>>>>> an alternative, but I do that for non essential features. In my mind,
>>>>> ability to manage synonyms elegantly is an essential feature for a search
>>>>> engine.
>>>>> >>>
>>>>> >>> On Wed, 30 Sep, 2020, 2:44 pm Uwe Schindler, <uwe@thetaphi.de>
>>>>> wrote:
>>>>> >>>>
>>>>> >>>> Please don't do this.
>>>>> >>>>
>>>>> >>>> In short: remove restlet stuff from master. Pull requests on
>>>>> master are executed with Gradle on GitHub hardware.
>>>>> >>>>
>>>>> >>>> Ivy stuff in 8.x is built in more or less persistent servers and
>>>>> there is no issue.
>>>>> >>>>
>>>>> >>>> What's the problem?
>>>>> >>>>
>>>>> >>>> Uwe
>>>>> >>>>
>>>>> >>>> Am September 30, 2020 8:59:06 AM UTC schrieb Ishan Chattopadhyaya
>>>>> <ichattopadhyaya@gmail.com>:
>>>>> >>>>>
>>>>> >>>>> Can we discuss this with ASF and get an exception for this?
>>>>> >>>>>
>>>>> >>>>> On Wed, 30 Sep, 2020, 11:57 am Dawid Weiss, <
>>>>> dawid.weiss@gmail.com> wrote:
>>>>> >>>>>>
>>>>> >>>>>> We can't have or redistribute binaries in ASL sources - that's
>>>>> my understanding.
>>>>> >>>>>>
>>>>> >>>>>> Dawid
>>>>> >>>>>>
>>>>> >>>>>> On Tue, Sep 29, 2020 at 10:02 PM Ishan Chattopadhyaya
>>>>> >>>>>> <ichattopadhyaya@gmail.com> wrote:
>>>>> >>>>>> >
>>>>> >>>>>> > Can we pull in the jar inside our codebase?
>>>>> >>>>>> >
>>>>> >>>>>> > On Wed, 30 Sep, 2020, 1:19 am Dawid Weiss, <
>>>>> dawid.weiss@gmail.com> wrote:
>>>>> >>>>>> >>
>>>>> >>>>>> >>
>>>>> >>>>>> >> We can upgrade if it doesn't break anything... which I can't
>>>>> guarantee. ;)
>>>>> >>>>>> >>
>>>>> >>>>>> >> Dawid
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> >>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> >>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>> >>>>>>
>>>>> >>>>
>>>>> >>>> --
>>>>> >>>> Uwe Schindler
>>>>> >>>> Achterdiek 19, 28357 Bremen
>>>>> >>>> https://www.thetaphi.de
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> -----------------------------------------------------
>>>>> Noble Paul
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>
>>>>>
>>> _______________________
>>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
>>> | http://www.opensourceconnections.com | My Free/Busy
>>> <http://tinyurl.com/eric-cal>
>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>>> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>>> This e-mail and all contents, including attachments, is considered to be
>>> Company Confidential unless explicitly stated otherwise, regardless
>>> of whether attachments are marked as such.
>>>
>>>
>> _______________________
>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
>> | http://www.opensourceconnections.com | My Free/Busy
>> <http://tinyurl.com/eric-cal>
>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>> This e-mail and all contents, including attachments, is considered to be
>> Company Confidential unless explicitly stated otherwise, regardless
>> of whether attachments are marked as such.
>>
>>
<https://issues.apache.org/jira/browse/SOLR-14907> to track the effort.
- Houston
On Wed, Sep 30, 2020 at 12:26 PM Tomás Fernández Löbbe <
tomasflobbe@gmail.com> wrote:
> > Let's support the single file upload feature
> +1, but let this behave exactly as a zip file with a single file in it
> (regarding trusted/untrusted). We just need to change the configset handler
> to be able to handle non-zip files, and have a way to "locate" that file
> inside the configset (in case it needs to go somewhere other than the root).
>
> On Wed, Sep 30, 2020 at 8:45 AM Eric Pugh <epugh@opensourceconnections.com>
> wrote:
>
>> I think that me in “violent agreement” with you. Let’s understand the
>> Annotations approach that we have, or pick something that is commonly used
>> like JAX-RS / Jersey.
>>
>>
>>
>> On Sep 30, 2020, at 11:41 AM, Timothy Potter <thelabdude@gmail.com>
>> wrote:
>>
>> I'm sorry, I don't understand what you mean by "make it a single pattern
>> (the annotations?)" Eric?
>>
>> To me, the pattern is well established in the Java world: JAX-RS (with
>> Jersey as the underlying impl. which has nice integration with Jetty). But
>> when I suggested porting the code that uses restlet to JAX-RS / Jersey,
>> Ishan said that wasn't necessary and is already supported with some
>> Annotations ... I have no idea what that means and need more info about
>> what is already in place. Short of that, replacing restlet with JAX-RS /
>> Jersey looks like a trivial amount of work to me (and I'm happy to take it
>> on).
>>
>> Tim
>>
>> On Wed, Sep 30, 2020 at 9:36 AM Eric Pugh <
>> epugh@opensourceconnections.com> wrote:
>>
>>> The use case of “I want to update something via a API” is I think pretty
>>> common, and it would be nice to make it a single pattern (the annotations?)
>>> with lots of examples/developer docs for the next person.
>>>
>>>
>>>
>>> On Sep 30, 2020, at 11:04 AM, Timothy Potter <thelabdude@gmail.com>
>>> wrote:
>>>
>>> I started looking into removing Managed Resources in master and wanted
>>> to mention that the LTR contrib also relies on this framework
>>> (ManagedModelStore and ManagedFeatureStore, see:
>>> https://lucene.apache.org/solr/guide/8_6/learning-to-rank.html#uploading-a-model).
>>> I only mention this b/c it's been said several times in this thread that
>>> nobody uses this feature and it's only for editing config/schema like
>>> synonyms. Afaik, LTR is a broadly used feature of Solr so now I'm not so
>>> bullish on removing the ability to manage dynamic resources using a REST
>>> like API. I agree that changing resources like the synonym set could be
>>> replaced with configSet updates but I don't see how to replace the RESTful
>>> model / feature store API w/o something like Managed Resources?
>>>
>>> From where I sit, I think we should just remove the use of restlet in
>>> the implementation but keep the API for Solr 9 (master).
>>>
>>> @Ishan ~ you mentioned there is a way to get REST API like behavior w/o
>>> using JAX-RS / Jersey ... something about annotations? Can you point me to
>>> some example code of how that is done please?
>>>
>>> Cheers,
>>> Tim
>>>
>>> On Wed, Sep 30, 2020 at 8:29 AM David Smiley <dsmiley@apache.org> wrote:
>>>
>>>> These resources are fundamentally a part of the configSet and can (in
>>>> general) affect query results and thus flushing caches (via a reload) is
>>>> appropriate.
>>>>
>>>> ~ David Smiley
>>>> Apache Lucene/Solr Search Developer
>>>> http://www.linkedin.com/in/davidwsmiley
>>>>
>>>>
>>>> On Wed, Sep 30, 2020 at 9:06 AM Noble Paul <noble.paul@gmail.com>
>>>> wrote:
>>>>
>>>>> Well, I believe we should have a mechanism to upload a single file to
>>>>> a configset.
>>>>>
>>>>> > A single file configset upload would require the user to reload the
>>>>> collection, so it isn't better than managed resources.
>>>>>
>>>>> This is not true
>>>>>
>>>>> Only config/schema file changes result in core reload.
>>>>>
>>>>> On Wed, Sep 30, 2020 at 10:23 PM David Smiley <dsmiley@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > Definitely don't remove in 8.x!
>>>>> >
>>>>> > > A single file configset upload would require the user to reload
>>>>> the collection, so it isn't better than managed resources.
>>>>> >
>>>>> > Do you view that as a substantial point in favor of
>>>>> managed-resources? I view that as a trivial matter, and one I prefer to
>>>>> automagic and potentially premature reload if there are additional edits to
>>>>> be done (e.g. query-elevation or other word lists).
>>>>> >
>>>>> > ~ David Smiley
>>>>> > Apache Lucene/Solr Search Developer
>>>>> > http://www.linkedin.com/in/davidwsmiley
>>>>> >
>>>>> >
>>>>> > On Wed, Sep 30, 2020 at 5:46 AM Ishan Chattopadhyaya <
>>>>> ichattopadhyaya@gmail.com> wrote:
>>>>> >>
>>>>> >> > * Nobody knows how it works. It's unsupported
>>>>> >> It is supported and documented:
>>>>> https://lucene.apache.org/solr/guide/8_6/managed-resources.html
>>>>> >>
>>>>> >> > * RESTlet dependency
>>>>> >> > * Cannot be secured using standard permissions
>>>>> >> > * It's extremely complex for the functionality it offers.
>>>>> >>
>>>>> >> I agree. Whatever alternative we build should address these, before
>>>>> we consider removing managed resources.
>>>>> >>
>>>>> >> On Wed, Sep 30, 2020 at 2:52 PM Ishan Chattopadhyaya <
>>>>> ichattopadhyaya@gmail.com> wrote:
>>>>> >>>
>>>>> >>> The managed resources is the only reasonable way to upload
>>>>> synonyms on the fly for users today. A single file configset upload would
>>>>> require the user to reload the collection, so it isn't better than managed
>>>>> resources. I would not recommend we remove the functionality without first
>>>>> building a suitable alternative. I agree that the feature isn't built using
>>>>> proper framework or proper APIs, but it is a feature that works well.
>>>>> >>>
>>>>> >>> Usually, I support throwing features out even without existence of
>>>>> an alternative, but I do that for non essential features. In my mind,
>>>>> ability to manage synonyms elegantly is an essential feature for a search
>>>>> engine.
>>>>> >>>
>>>>> >>> On Wed, 30 Sep, 2020, 2:44 pm Uwe Schindler, <uwe@thetaphi.de>
>>>>> wrote:
>>>>> >>>>
>>>>> >>>> Please don't do this.
>>>>> >>>>
>>>>> >>>> In short: remove restlet stuff from master. Pull requests on
>>>>> master are executed with Gradle on GitHub hardware.
>>>>> >>>>
>>>>> >>>> Ivy stuff in 8.x is built in more or less persistent servers and
>>>>> there is no issue.
>>>>> >>>>
>>>>> >>>> What's the problem?
>>>>> >>>>
>>>>> >>>> Uwe
>>>>> >>>>
>>>>> >>>> Am September 30, 2020 8:59:06 AM UTC schrieb Ishan Chattopadhyaya
>>>>> <ichattopadhyaya@gmail.com>:
>>>>> >>>>>
>>>>> >>>>> Can we discuss this with ASF and get an exception for this?
>>>>> >>>>>
>>>>> >>>>> On Wed, 30 Sep, 2020, 11:57 am Dawid Weiss, <
>>>>> dawid.weiss@gmail.com> wrote:
>>>>> >>>>>>
>>>>> >>>>>> We can't have or redistribute binaries in ASL sources - that's
>>>>> my understanding.
>>>>> >>>>>>
>>>>> >>>>>> Dawid
>>>>> >>>>>>
>>>>> >>>>>> On Tue, Sep 29, 2020 at 10:02 PM Ishan Chattopadhyaya
>>>>> >>>>>> <ichattopadhyaya@gmail.com> wrote:
>>>>> >>>>>> >
>>>>> >>>>>> > Can we pull in the jar inside our codebase?
>>>>> >>>>>> >
>>>>> >>>>>> > On Wed, 30 Sep, 2020, 1:19 am Dawid Weiss, <
>>>>> dawid.weiss@gmail.com> wrote:
>>>>> >>>>>> >>
>>>>> >>>>>> >>
>>>>> >>>>>> >> We can upgrade if it doesn't break anything... which I can't
>>>>> guarantee. ;)
>>>>> >>>>>> >>
>>>>> >>>>>> >> Dawid
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> >>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> >>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>> >>>>>>
>>>>> >>>>
>>>>> >>>> --
>>>>> >>>> Uwe Schindler
>>>>> >>>> Achterdiek 19, 28357 Bremen
>>>>> >>>> https://www.thetaphi.de
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> -----------------------------------------------------
>>>>> Noble Paul
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>
>>>>>
>>> _______________________
>>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
>>> | http://www.opensourceconnections.com | My Free/Busy
>>> <http://tinyurl.com/eric-cal>
>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>>> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>>> This e-mail and all contents, including attachments, is considered to be
>>> Company Confidential unless explicitly stated otherwise, regardless
>>> of whether attachments are marked as such.
>>>
>>>
>> _______________________
>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
>> | http://www.opensourceconnections.com | My Free/Busy
>> <http://tinyurl.com/eric-cal>
>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>> This e-mail and all contents, including attachments, is considered to be
>> Company Confidential unless explicitly stated otherwise, regardless
>> of whether attachments are marked as such.
>>
>>