Mailing List Archive: Filestore, subsume userfiles requirement

Filestore, subsume userfiles requirement

Sep 25, 2020, 5:32 AM

Post #1 of 13 (1284 views)

I'm looking to see that we can deprecate "userfiles" and remove for 9.0

Solr has a "userfiles" directory under Solr home that Jason added in some
issue relating to streaming expressions accessing a local file. I bet only
a few of you have even heard of it. I think the "File Store" that came
along with the package manager obsoletes "userfiles". If you have not
heard of the file store either, I wouldn't be surprised -- it's new and
it's name was changed from "package store" last minute, since it's general
purpose, with "packages" being a directory at the root of the file store
for packages. It's documented as the "package store" (should be renamed)
on the package manager internals doc:
https://lucene.apache.org/solr/guide/8_6/package-manager-internals.html#package-store
However IMO it's worthy of its own doc page as it's a very useful new
component of the Solr platform. It can store "user files" (hence
obsoleting the userfiles dir), ML models, or basically any file that's too
large to put in ZK. I'd be nice if SolrResourceLoader could resolve
resources from it -- an issue for another day. That would be another
avenue of use separate from the configSet. You can already upload single
files to the file store :-)

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley

Re: Filestore, subsume userfiles requirement [ In reply to ]

arafalov at gmail

Sep 25, 2020, 5:41 AM

Post #2 of 13 (1284 views)

Permalink

Would that file store also be under solr.home? Because if it is and
the user can upload core.property into it as well as other things that
core discovery will then load bypassing security....

Regards,
Alex.

On Fri, 25 Sep 2020 at 08:32, David Smiley <dsmiley@apache.org> wrote:
>
> I'm looking to see that we can deprecate "userfiles" and remove for 9.0
>
> Solr has a "userfiles" directory under Solr home that Jason added in some issue relating to streaming expressions accessing a local file. I bet only a few of you have even heard of it. I think the "File Store" that came along with the package manager obsoletes "userfiles". If you have not heard of the file store either, I wouldn't be surprised -- it's new and it's name was changed from "package store" last minute, since it's general purpose, with "packages" being a directory at the root of the file store for packages. It's documented as the "package store" (should be renamed) on the package manager internals doc: https://lucene.apache.org/solr/guide/8_6/package-manager-internals.html#package-store
> However IMO it's worthy of its own doc page as it's a very useful new component of the Solr platform. It can store "user files" (hence obsoleting the userfiles dir), ML models, or basically any file that's too large to put in ZK. I'd be nice if SolrResourceLoader could resolve resources from it -- an issue for another day. That would be another avenue of use separate from the configSet. You can already upload single files to the file store :-)
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: Filestore, subsume userfiles requirement [ In reply to ]

ichattopadhyaya at gmail

Sep 25, 2020, 5:47 AM

Post #3 of 13 (1284 views)

Permalink

+1

We can even obsolete the blob store on favour of the package store (once it
is renamed to file store). We built the package store as a general purpose
distributed file store. Since we were running fast with the package
manager, we didn't advertise it for all of its capabilities as a generic
file store. We didn't want to distract ourselves then, and didn't want to
release it without a full security audit. Now is a good time to leverage
the potential of the package store as a filestore for all our needs.

On Fri, 25 Sep, 2020, 6:02 pm David Smiley, <dsmiley@apache.org> wrote:

> I'm looking to see that we can deprecate "userfiles" and remove for 9.0
>
> Solr has a "userfiles" directory under Solr home that Jason added in some
> issue relating to streaming expressions accessing a local file. I bet only
> a few of you have even heard of it. I think the "File Store" that came
> along with the package manager obsoletes "userfiles". If you have not
> heard of the file store either, I wouldn't be surprised -- it's new and
> it's name was changed from "package store" last minute, since it's general
> purpose, with "packages" being a directory at the root of the file store
> for packages. It's documented as the "package store" (should be renamed)
> on the package manager internals doc:
> https://lucene.apache.org/solr/guide/8_6/package-manager-internals.html#package-store
> However IMO it's worthy of its own doc page as it's a very useful new
> component of the Solr platform. It can store "user files" (hence
> obsoleting the userfiles dir), ML models, or basically any file that's too
> large to put in ZK. I'd be nice if SolrResourceLoader could resolve
> resources from it -- an issue for another day. That would be another
> avenue of use separate from the configSet. You can already upload single
> files to the file store :-)
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>

Re: Filestore, subsume userfiles requirement [ In reply to ]

dsmiley at apache

Sep 25, 2020, 5:47 AM

Post #4 of 13 (1284 views)

Permalink

Yes. And I think it's high time that coreRootDirectory default to
<solrhome>/something (e.g. "cores")

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley

On Fri, Sep 25, 2020 at 8:42 AM Alexandre Rafalovitch <arafalov@gmail.com>
wrote:

> Would that file store also be under solr.home? Because if it is and
> the user can upload core.property into it as well as other things that
> core discovery will then load bypassing security....
>
> Regards,
> Alex.
>
> On Fri, 25 Sep 2020 at 08:32, David Smiley <dsmiley@apache.org> wrote:
> >
> > I'm looking to see that we can deprecate "userfiles" and remove for 9.0
> >
> > Solr has a "userfiles" directory under Solr home that Jason added in
> some issue relating to streaming expressions accessing a local file. I bet
> only a few of you have even heard of it. I think the "File Store" that
> came along with the package manager obsoletes "userfiles". If you have not
> heard of the file store either, I wouldn't be surprised -- it's new and
> it's name was changed from "package store" last minute, since it's general
> purpose, with "packages" being a directory at the root of the file store
> for packages. It's documented as the "package store" (should be renamed)
> on the package manager internals doc:
> https://lucene.apache.org/solr/guide/8_6/package-manager-internals.html#package-store
> > However IMO it's worthy of its own doc page as it's a very useful new
> component of the Solr platform. It can store "user files" (hence
> obsoleting the userfiles dir), ML models, or basically any file that's too
> large to put in ZK. I'd be nice if SolrResourceLoader could resolve
> resources from it -- an issue for another day. That would be another
> avenue of use separate from the configSet. You can already upload single
> files to the file store :-)
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

Re: Filestore, subsume userfiles requirement [ In reply to ]

epugh at opensourceconnections

Sep 25, 2020, 6:38 AM

Post #5 of 13 (1284 views)

Permalink

+1. I’ve found the user files really useful when doing things with streaming, but it’s also awkward to reach to put files into.

> On Sep 25, 2020, at 8:47 AM, David Smiley <dsmiley@apache.org> wrote:
>
> Yes. And I think it's high time that coreRootDirectory default to <solrhome>/something (e.g. "cores")
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley <http://www.linkedin.com/in/davidwsmiley>
>
> On Fri, Sep 25, 2020 at 8:42 AM Alexandre Rafalovitch <arafalov@gmail.com <mailto:arafalov@gmail.com>> wrote:
> Would that file store also be under solr.home? Because if it is and
> the user can upload core.property into it as well as other things that
> core discovery will then load bypassing security....
>
> Regards,
> Alex.
>
> On Fri, 25 Sep 2020 at 08:32, David Smiley <dsmiley@apache.org <mailto:dsmiley@apache.org>> wrote:
> >
> > I'm looking to see that we can deprecate "userfiles" and remove for 9.0
> >
> > Solr has a "userfiles" directory under Solr home that Jason added in some issue relating to streaming expressions accessing a local file. I bet only a few of you have even heard of it. I think the "File Store" that came along with the package manager obsoletes "userfiles". If you have not heard of the file store either, I wouldn't be surprised -- it's new and it's name was changed from "package store" last minute, since it's general purpose, with "packages" being a directory at the root of the file store for packages. It's documented as the "package store" (should be renamed) on the package manager internals doc: https://lucene.apache.org/solr/guide/8_6/package-manager-internals.html#package-store <https://lucene.apache.org/solr/guide/8_6/package-manager-internals.html#package-store>
> > However IMO it's worthy of its own doc page as it's a very useful new component of the Solr platform. It can store "user files" (hence obsoleting the userfiles dir), ML models, or basically any file that's too large to put in ZK. I'd be nice if SolrResourceLoader could resolve resources from it -- an issue for another day. That would be another avenue of use separate from the configSet. You can already upload single files to the file store :-)
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley <http://www.linkedin.com/in/davidwsmiley>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
> For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>
>

_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.

Re: Filestore, subsume userfiles requirement [ In reply to ]

gerlowskija at gmail

Sep 25, 2020, 7:20 AM

Post #6 of 13 (1284 views)

Permalink

I don't know much about the new package/file-store, but it does sound
like a good replacement for 'userfiles' (which Eric is right- is
pretty awkward to work with).

My only concerns with using the filestore would be around its
availability. Is it enabled by default (or will it be in an upcoming
release)? Does it work in standalone as well as SolrCloud? If the
answer to those questions are both "yes", then it makes sense to me as
a replacement on the face of things.

Jason

On Fri, Sep 25, 2020 at 9:39 AM Eric Pugh
<epugh@opensourceconnections.com> wrote:
>
> +1. I’ve found the user files really useful when doing things with streaming, but it’s also awkward to reach to put files into.
>
>
> On Sep 25, 2020, at 8:47 AM, David Smiley <dsmiley@apache.org> wrote:
>
> Yes. And I think it's high time that coreRootDirectory default to <solrhome>/something (e.g. "cores")
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Fri, Sep 25, 2020 at 8:42 AM Alexandre Rafalovitch <arafalov@gmail.com> wrote:
>>
>> Would that file store also be under solr.home? Because if it is and
>> the user can upload core.property into it as well as other things that
>> core discovery will then load bypassing security....
>>
>> Regards,
>> Alex.
>>
>> On Fri, 25 Sep 2020 at 08:32, David Smiley <dsmiley@apache.org> wrote:
>> >
>> > I'm looking to see that we can deprecate "userfiles" and remove for 9.0
>> >
>> > Solr has a "userfiles" directory under Solr home that Jason added in some issue relating to streaming expressions accessing a local file. I bet only a few of you have even heard of it. I think the "File Store" that came along with the package manager obsoletes "userfiles". If you have not heard of the file store either, I wouldn't be surprised -- it's new and it's name was changed from "package store" last minute, since it's general purpose, with "packages" being a directory at the root of the file store for packages. It's documented as the "package store" (should be renamed) on the package manager internals doc: https://lucene.apache.org/solr/guide/8_6/package-manager-internals.html#package-store
>> > However IMO it's worthy of its own doc page as it's a very useful new component of the Solr platform. It can store "user files" (hence obsoleting the userfiles dir), ML models, or basically any file that's too large to put in ZK. I'd be nice if SolrResourceLoader could resolve resources from it -- an issue for another day. That would be another avenue of use separate from the configSet. You can already upload single files to the file store :-)
>> >
>> > ~ David Smiley
>> > Apache Lucene/Solr Search Developer
>> > http://www.linkedin.com/in/davidwsmiley
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: Filestore, subsume userfiles requirement [ In reply to ]

ichattopadhyaya at gmail

Sep 25, 2020, 8:00 AM

Post #7 of 13 (1284 views)

Permalink

Standalone isn't supported.

We need to transition away from the standalone mode and get rid of it
completely.

On Fri, 25 Sep, 2020, 7:50 pm Jason Gerlowski, <gerlowskija@gmail.com>
wrote:

> I don't know much about the new package/file-store, but it does sound
> like a good replacement for 'userfiles' (which Eric is right- is
> pretty awkward to work with).
>
> My only concerns with using the filestore would be around its
> availability. Is it enabled by default (or will it be in an upcoming
> release)? Does it work in standalone as well as SolrCloud? If the
> answer to those questions are both "yes", then it makes sense to me as
> a replacement on the face of things.
>
> Jason
>
> On Fri, Sep 25, 2020 at 9:39 AM Eric Pugh
> <epugh@opensourceconnections.com> wrote:
> >
> > +1. I’ve found the user files really useful when doing things with
> streaming, but it’s also awkward to reach to put files into.
> >
> >
> > On Sep 25, 2020, at 8:47 AM, David Smiley <dsmiley@apache.org> wrote:
> >
> > Yes. And I think it's high time that coreRootDirectory default to
> <solrhome>/something (e.g. "cores")
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > On Fri, Sep 25, 2020 at 8:42 AM Alexandre Rafalovitch <
> arafalov@gmail.com> wrote:
> >>
> >> Would that file store also be under solr.home? Because if it is and
> >> the user can upload core.property into it as well as other things that
> >> core discovery will then load bypassing security....
> >>
> >> Regards,
> >> Alex.
> >>
> >> On Fri, 25 Sep 2020 at 08:32, David Smiley <dsmiley@apache.org> wrote:
> >> >
> >> > I'm looking to see that we can deprecate "userfiles" and remove for
> 9.0
> >> >
> >> > Solr has a "userfiles" directory under Solr home that Jason added in
> some issue relating to streaming expressions accessing a local file. I bet
> only a few of you have even heard of it. I think the "File Store" that
> came along with the package manager obsoletes "userfiles". If you have not
> heard of the file store either, I wouldn't be surprised -- it's new and
> it's name was changed from "package store" last minute, since it's general
> purpose, with "packages" being a directory at the root of the file store
> for packages. It's documented as the "package store" (should be renamed)
> on the package manager internals doc:
> https://lucene.apache.org/solr/guide/8_6/package-manager-internals.html#package-store
> >> > However IMO it's worthy of its own doc page as it's a very useful new
> component of the Solr platform. It can store "user files" (hence
> obsoleting the userfiles dir), ML models, or basically any file that's too
> large to put in ZK. I'd be nice if SolrResourceLoader could resolve
> resources from it -- an issue for another day. That would be another
> avenue of use separate from the configSet. You can already upload single
> files to the file store :-)
> >> >
> >> > ~ David Smiley
> >> > Apache Lucene/Solr Search Developer
> >> > http://www.linkedin.com/in/davidwsmiley
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: dev-help@lucene.apache.org
> >>
> >
> > _______________________
> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com | My Free/Busy
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> > This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

Re: Filestore, subsume userfiles requirement [ In reply to ]

dsmiley at apache

Sep 25, 2020, 8:24 AM

Post #8 of 13 (1284 views)

Permalink

The "userfiles" thing is merely a directory. There's no CRUD API on it.
Even if the CRUD API of the file store isn't active in standalone -- I
think that doesn't matter. You're left with the <solrhome>/filestore/ dir
to put there what you want in standalone mode.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley

On Fri, Sep 25, 2020 at 11:00 AM Ishan Chattopadhyaya <
ichattopadhyaya@gmail.com> wrote:

> Standalone isn't supported.
>
> We need to transition away from the standalone mode and get rid of it
> completely.
>
> On Fri, 25 Sep, 2020, 7:50 pm Jason Gerlowski, <gerlowskija@gmail.com>
> wrote:
>
>> I don't know much about the new package/file-store, but it does sound
>> like a good replacement for 'userfiles' (which Eric is right- is
>> pretty awkward to work with).
>>
>> My only concerns with using the filestore would be around its
>> availability. Is it enabled by default (or will it be in an upcoming
>> release)? Does it work in standalone as well as SolrCloud? If the
>> answer to those questions are both "yes", then it makes sense to me as
>> a replacement on the face of things.
>>
>> Jason
>>
>> On Fri, Sep 25, 2020 at 9:39 AM Eric Pugh
>> <epugh@opensourceconnections.com> wrote:
>> >
>> > +1. I’ve found the user files really useful when doing things with
>> streaming, but it’s also awkward to reach to put files into.
>> >
>> >
>> > On Sep 25, 2020, at 8:47 AM, David Smiley <dsmiley@apache.org> wrote:
>> >
>> > Yes. And I think it's high time that coreRootDirectory default to
>> <solrhome>/something (e.g. "cores")
>> >
>> > ~ David Smiley
>> > Apache Lucene/Solr Search Developer
>> > http://www.linkedin.com/in/davidwsmiley
>> >
>> >
>> > On Fri, Sep 25, 2020 at 8:42 AM Alexandre Rafalovitch <
>> arafalov@gmail.com> wrote:
>> >>
>> >> Would that file store also be under solr.home? Because if it is and
>> >> the user can upload core.property into it as well as other things that
>> >> core discovery will then load bypassing security....
>> >>
>> >> Regards,
>> >> Alex.
>> >>
>> >> On Fri, 25 Sep 2020 at 08:32, David Smiley <dsmiley@apache.org> wrote:
>> >> >
>> >> > I'm looking to see that we can deprecate "userfiles" and remove for
>> 9.0
>> >> >
>> >> > Solr has a "userfiles" directory under Solr home that Jason added in
>> some issue relating to streaming expressions accessing a local file. I bet
>> only a few of you have even heard of it. I think the "File Store" that
>> came along with the package manager obsoletes "userfiles". If you have not
>> heard of the file store either, I wouldn't be surprised -- it's new and
>> it's name was changed from "package store" last minute, since it's general
>> purpose, with "packages" being a directory at the root of the file store
>> for packages. It's documented as the "package store" (should be renamed)
>> on the package manager internals doc:
>> https://lucene.apache.org/solr/guide/8_6/package-manager-internals.html#package-store
>> >> > However IMO it's worthy of its own doc page as it's a very useful
>> new component of the Solr platform. It can store "user files" (hence
>> obsoleting the userfiles dir), ML models, or basically any file that's too
>> large to put in ZK. I'd be nice if SolrResourceLoader could resolve
>> resources from it -- an issue for another day. That would be another
>> avenue of use separate from the configSet. You can already upload single
>> files to the file store :-)
>> >> >
>> >> > ~ David Smiley
>> >> > Apache Lucene/Solr Search Developer
>> >> > http://www.linkedin.com/in/davidwsmiley
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >> For additional commands, e-mail: dev-help@lucene.apache.org
>> >>
>> >
>> > _______________________
>> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467
>> | http://www.opensourceconnections.com | My Free/Busy
>> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>> > This e-mail and all contents, including attachments, is considered to
>> be Company Confidential unless explicitly stated otherwise, regardless of
>> whether attachments are marked as such.
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>

Re: Filestore, subsume userfiles requirement [ In reply to ]

gerlowskija at gmail

Sep 25, 2020, 8:42 AM

Post #9 of 13 (1284 views)

Permalink

> You're left with the <solrhome>/filestore/ dir to put there what you want in standalone mode.
That's fine by me as that's effectively what "userfiles" provides now.
The main difference I imagine is that the 'filestore' directory won't
exist at all in standalone? In that case standalone users will have to
know where to create the dir if they want to reference 'filestore'
files in their streaming expressions. But that's not all that much
more to ask I guess.

I'm curious - is the filestore unimplemented in standalone because no
one prioritized it, or is there a specific technical blocker?

On Fri, Sep 25, 2020 at 11:24 AM David Smiley <dsmiley@apache.org> wrote:
>
> The "userfiles" thing is merely a directory. There's no CRUD API on it. Even if the CRUD API of the file store isn't active in standalone -- I think that doesn't matter. You're left with the <solrhome>/filestore/ dir to put there what you want in standalone mode.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Fri, Sep 25, 2020 at 11:00 AM Ishan Chattopadhyaya <ichattopadhyaya@gmail.com> wrote:
>>
>> Standalone isn't supported.
>>
>> We need to transition away from the standalone mode and get rid of it completely.
>>
>> On Fri, 25 Sep, 2020, 7:50 pm Jason Gerlowski, <gerlowskija@gmail.com> wrote:
>>>
>>> I don't know much about the new package/file-store, but it does sound
>>> like a good replacement for 'userfiles' (which Eric is right- is
>>> pretty awkward to work with).
>>>
>>> My only concerns with using the filestore would be around its
>>> availability. Is it enabled by default (or will it be in an upcoming
>>> release)? Does it work in standalone as well as SolrCloud? If the
>>> answer to those questions are both "yes", then it makes sense to me as
>>> a replacement on the face of things.
>>>
>>> Jason
>>>
>>> On Fri, Sep 25, 2020 at 9:39 AM Eric Pugh
>>> <epugh@opensourceconnections.com> wrote:
>>> >
>>> > +1. I’ve found the user files really useful when doing things with streaming, but it’s also awkward to reach to put files into.
>>> >
>>> >
>>> > On Sep 25, 2020, at 8:47 AM, David Smiley <dsmiley@apache.org> wrote:
>>> >
>>> > Yes. And I think it's high time that coreRootDirectory default to <solrhome>/something (e.g. "cores")
>>> >
>>> > ~ David Smiley
>>> > Apache Lucene/Solr Search Developer
>>> > http://www.linkedin.com/in/davidwsmiley
>>> >
>>> >
>>> > On Fri, Sep 25, 2020 at 8:42 AM Alexandre Rafalovitch <arafalov@gmail.com> wrote:
>>> >>
>>> >> Would that file store also be under solr.home? Because if it is and
>>> >> the user can upload core.property into it as well as other things that
>>> >> core discovery will then load bypassing security....
>>> >>
>>> >> Regards,
>>> >> Alex.
>>> >>
>>> >> On Fri, 25 Sep 2020 at 08:32, David Smiley <dsmiley@apache.org> wrote:
>>> >> >
>>> >> > I'm looking to see that we can deprecate "userfiles" and remove for 9.0
>>> >> >
>>> >> > Solr has a "userfiles" directory under Solr home that Jason added in some issue relating to streaming expressions accessing a local file. I bet only a few of you have even heard of it. I think the "File Store" that came along with the package manager obsoletes "userfiles". If you have not heard of the file store either, I wouldn't be surprised -- it's new and it's name was changed from "package store" last minute, since it's general purpose, with "packages" being a directory at the root of the file store for packages. It's documented as the "package store" (should be renamed) on the package manager internals doc: https://lucene.apache.org/solr/guide/8_6/package-manager-internals.html#package-store
>>> >> > However IMO it's worthy of its own doc page as it's a very useful new component of the Solr platform. It can store "user files" (hence obsoleting the userfiles dir), ML models, or basically any file that's too large to put in ZK. I'd be nice if SolrResourceLoader could resolve resources from it -- an issue for another day. That would be another avenue of use separate from the configSet. You can already upload single files to the file store :-)
>>> >> >
>>> >> > ~ David Smiley
>>> >> > Apache Lucene/Solr Search Developer
>>> >> > http://www.linkedin.com/in/davidwsmiley
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> >> For additional commands, e-mail: dev-help@lucene.apache.org
>>> >>
>>> >
>>> > _______________________
>>> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy
>>> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>>> > This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: Filestore, subsume userfiles requirement [ In reply to ]

arafalov at gmail

Sep 25, 2020, 8:45 AM

Post #10 of 13 (1284 views)

Permalink

It is not just a directory. There is a security feature that only
allows to read from several file locations (configurable in solr.xml).
Userfiles is part of default list, so whatever replacement is there
will need to be explicitly-named as well.

Regards,
Alex.

On Fri, 25 Sep 2020 at 11:43, Jason Gerlowski <gerlowskija@gmail.com> wrote:
>
> > You're left with the <solrhome>/filestore/ dir to put there what you want in standalone mode.
> That's fine by me as that's effectively what "userfiles" provides now.
> The main difference I imagine is that the 'filestore' directory won't
> exist at all in standalone? In that case standalone users will have to
> know where to create the dir if they want to reference 'filestore'
> files in their streaming expressions. But that's not all that much
> more to ask I guess.
>
> I'm curious - is the filestore unimplemented in standalone because no
> one prioritized it, or is there a specific technical blocker?
>
> On Fri, Sep 25, 2020 at 11:24 AM David Smiley <dsmiley@apache.org> wrote:
> >
> > The "userfiles" thing is merely a directory. There's no CRUD API on it. Even if the CRUD API of the file store isn't active in standalone -- I think that doesn't matter. You're left with the <solrhome>/filestore/ dir to put there what you want in standalone mode.
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > On Fri, Sep 25, 2020 at 11:00 AM Ishan Chattopadhyaya <ichattopadhyaya@gmail.com> wrote:
> >>
> >> Standalone isn't supported.
> >>
> >> We need to transition away from the standalone mode and get rid of it completely.
> >>
> >> On Fri, 25 Sep, 2020, 7:50 pm Jason Gerlowski, <gerlowskija@gmail.com> wrote:
> >>>
> >>> I don't know much about the new package/file-store, but it does sound
> >>> like a good replacement for 'userfiles' (which Eric is right- is
> >>> pretty awkward to work with).
> >>>
> >>> My only concerns with using the filestore would be around its
> >>> availability. Is it enabled by default (or will it be in an upcoming
> >>> release)? Does it work in standalone as well as SolrCloud? If the
> >>> answer to those questions are both "yes", then it makes sense to me as
> >>> a replacement on the face of things.
> >>>
> >>> Jason
> >>>
> >>> On Fri, Sep 25, 2020 at 9:39 AM Eric Pugh
> >>> <epugh@opensourceconnections.com> wrote:
> >>> >
> >>> > +1. I’ve found the user files really useful when doing things with streaming, but it’s also awkward to reach to put files into.
> >>> >
> >>> >
> >>> > On Sep 25, 2020, at 8:47 AM, David Smiley <dsmiley@apache.org> wrote:
> >>> >
> >>> > Yes. And I think it's high time that coreRootDirectory default to <solrhome>/something (e.g. "cores")
> >>> >
> >>> > ~ David Smiley
> >>> > Apache Lucene/Solr Search Developer
> >>> > http://www.linkedin.com/in/davidwsmiley
> >>> >
> >>> >
> >>> > On Fri, Sep 25, 2020 at 8:42 AM Alexandre Rafalovitch <arafalov@gmail.com> wrote:
> >>> >>
> >>> >> Would that file store also be under solr.home? Because if it is and
> >>> >> the user can upload core.property into it as well as other things that
> >>> >> core discovery will then load bypassing security....
> >>> >>
> >>> >> Regards,
> >>> >> Alex.
> >>> >>
> >>> >> On Fri, 25 Sep 2020 at 08:32, David Smiley <dsmiley@apache.org> wrote:
> >>> >> >
> >>> >> > I'm looking to see that we can deprecate "userfiles" and remove for 9.0
> >>> >> >
> >>> >> > Solr has a "userfiles" directory under Solr home that Jason added in some issue relating to streaming expressions accessing a local file. I bet only a few of you have even heard of it. I think the "File Store" that came along with the package manager obsoletes "userfiles". If you have not heard of the file store either, I wouldn't be surprised -- it's new and it's name was changed from "package store" last minute, since it's general purpose, with "packages" being a directory at the root of the file store for packages. It's documented as the "package store" (should be renamed) on the package manager internals doc: https://lucene.apache.org/solr/guide/8_6/package-manager-internals.html#package-store
> >>> >> > However IMO it's worthy of its own doc page as it's a very useful new component of the Solr platform. It can store "user files" (hence obsoleting the userfiles dir), ML models, or basically any file that's too large to put in ZK. I'd be nice if SolrResourceLoader could resolve resources from it -- an issue for another day. That would be another avenue of use separate from the configSet. You can already upload single files to the file store :-)
> >>> >> >
> >>> >> > ~ David Smiley
> >>> >> > Apache Lucene/Solr Search Developer
> >>> >> > http://www.linkedin.com/in/davidwsmiley
> >>> >>
> >>> >> ---------------------------------------------------------------------
> >>> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>> >> For additional commands, e-mail: dev-help@lucene.apache.org
> >>> >>
> >>> >
> >>> > _______________________
> >>> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy
> >>> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> >>> > This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
> >>> >
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>> For additional commands, e-mail: dev-help@lucene.apache.org
> >>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: Filestore, subsume userfiles requirement [ In reply to ]

epugh at opensourceconnections

Sep 25, 2020, 8:53 AM

Post #11 of 13 (1284 views)

Permalink

Can you do streaming expression in standalone solr? I was under the impression that it required SolrCloud.

> On Sep 25, 2020, at 11:45 AM, Alexandre Rafalovitch <arafalov@gmail.com> wrote:
>
> It is not just a directory. There is a security feature that only
> allows to read from several file locations (configurable in solr.xml).
> Userfiles is part of default list, so whatever replacement is there
> will need to be explicitly-named as well.
>
> Regards,
> Alex.
>
> On Fri, 25 Sep 2020 at 11:43, Jason Gerlowski <gerlowskija@gmail.com> wrote:
>>
>>> You're left with the <solrhome>/filestore/ dir to put there what you want in standalone mode.
>> That's fine by me as that's effectively what "userfiles" provides now.
>> The main difference I imagine is that the 'filestore' directory won't
>> exist at all in standalone? In that case standalone users will have to
>> know where to create the dir if they want to reference 'filestore'
>> files in their streaming expressions. But that's not all that much
>> more to ask I guess.
>>
>> I'm curious - is the filestore unimplemented in standalone because no
>> one prioritized it, or is there a specific technical blocker?
>>
>> On Fri, Sep 25, 2020 at 11:24 AM David Smiley <dsmiley@apache.org> wrote:
>>>
>>> The "userfiles" thing is merely a directory. There's no CRUD API on it. Even if the CRUD API of the file store isn't active in standalone -- I think that doesn't matter. You're left with the <solrhome>/filestore/ dir to put there what you want in standalone mode.
>>>
>>> ~ David Smiley
>>> Apache Lucene/Solr Search Developer
>>> http://www.linkedin.com/in/davidwsmiley
>>>
>>>
>>> On Fri, Sep 25, 2020 at 11:00 AM Ishan Chattopadhyaya <ichattopadhyaya@gmail.com> wrote:
>>>>
>>>> Standalone isn't supported.
>>>>
>>>> We need to transition away from the standalone mode and get rid of it completely.
>>>>
>>>> On Fri, 25 Sep, 2020, 7:50 pm Jason Gerlowski, <gerlowskija@gmail.com> wrote:
>>>>>
>>>>> I don't know much about the new package/file-store, but it does sound
>>>>> like a good replacement for 'userfiles' (which Eric is right- is
>>>>> pretty awkward to work with).
>>>>>
>>>>> My only concerns with using the filestore would be around its
>>>>> availability. Is it enabled by default (or will it be in an upcoming
>>>>> release)? Does it work in standalone as well as SolrCloud? If the
>>>>> answer to those questions are both "yes", then it makes sense to me as
>>>>> a replacement on the face of things.
>>>>>
>>>>> Jason
>>>>>
>>>>> On Fri, Sep 25, 2020 at 9:39 AM Eric Pugh
>>>>> <epugh@opensourceconnections.com> wrote:
>>>>>>
>>>>>> +1. I’ve found the user files really useful when doing things with streaming, but it’s also awkward to reach to put files into.
>>>>>>
>>>>>>
>>>>>> On Sep 25, 2020, at 8:47 AM, David Smiley <dsmiley@apache.org> wrote:
>>>>>>
>>>>>> Yes. And I think it's high time that coreRootDirectory default to <solrhome>/something (e.g. "cores")
>>>>>>
>>>>>> ~ David Smiley
>>>>>> Apache Lucene/Solr Search Developer
>>>>>> http://www.linkedin.com/in/davidwsmiley
>>>>>>
>>>>>>
>>>>>> On Fri, Sep 25, 2020 at 8:42 AM Alexandre Rafalovitch <arafalov@gmail.com> wrote:
>>>>>>>
>>>>>>> Would that file store also be under solr.home? Because if it is and
>>>>>>> the user can upload core.property into it as well as other things that
>>>>>>> core discovery will then load bypassing security....
>>>>>>>
>>>>>>> Regards,
>>>>>>> Alex.
>>>>>>>
>>>>>>> On Fri, 25 Sep 2020 at 08:32, David Smiley <dsmiley@apache.org> wrote:
>>>>>>>>
>>>>>>>> I'm looking to see that we can deprecate "userfiles" and remove for 9.0
>>>>>>>>
>>>>>>>> Solr has a "userfiles" directory under Solr home that Jason added in some issue relating to streaming expressions accessing a local file. I bet only a few of you have even heard of it. I think the "File Store" that came along with the package manager obsoletes "userfiles". If you have not heard of the file store either, I wouldn't be surprised -- it's new and it's name was changed from "package store" last minute, since it's general purpose, with "packages" being a directory at the root of the file store for packages. It's documented as the "package store" (should be renamed) on the package manager internals doc: https://lucene.apache.org/solr/guide/8_6/package-manager-internals.html#package-store
>>>>>>>> However IMO it's worthy of its own doc page as it's a very useful new component of the Solr platform. It can store "user files" (hence obsoleting the userfiles dir), ML models, or basically any file that's too large to put in ZK. I'd be nice if SolrResourceLoader could resolve resources from it -- an issue for another day. That would be another avenue of use separate from the configSet. You can already upload single files to the file store :-)
>>>>>>>>
>>>>>>>> ~ David Smiley
>>>>>>>> Apache Lucene/Solr Search Developer
>>>>>>>> http://www.linkedin.com/in/davidwsmiley
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>>>
>>>>>>
>>>>>> _______________________
>>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy
>>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>>>>>> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.

Re: Filestore, subsume userfiles requirement [ In reply to ]

gerlowskija at gmail

Sep 25, 2020, 9:02 AM

Post #12 of 13 (1284 views)

Permalink

I think some expressions are SolrCloud specific, but there are
expressions that work in standalone

e.g. https://paste.apache.org/zm3sw and https://paste.apache.org/vs35m

On Fri, Sep 25, 2020 at 11:53 AM Eric Pugh
<epugh@opensourceconnections.com> wrote:
>
> Can you do streaming expression in standalone solr? I was under the impression that it required SolrCloud.
>
>
> On Sep 25, 2020, at 11:45 AM, Alexandre Rafalovitch <arafalov@gmail.com> wrote:
>
> It is not just a directory. There is a security feature that only
> allows to read from several file locations (configurable in solr.xml).
> Userfiles is part of default list, so whatever replacement is there
> will need to be explicitly-named as well.
>
> Regards,
> Alex.
>
> On Fri, 25 Sep 2020 at 11:43, Jason Gerlowski <gerlowskija@gmail.com> wrote:
>
>
> You're left with the <solrhome>/filestore/ dir to put there what you want in standalone mode.
>
> That's fine by me as that's effectively what "userfiles" provides now.
> The main difference I imagine is that the 'filestore' directory won't
> exist at all in standalone? In that case standalone users will have to
> know where to create the dir if they want to reference 'filestore'
> files in their streaming expressions. But that's not all that much
> more to ask I guess.
>
> I'm curious - is the filestore unimplemented in standalone because no
> one prioritized it, or is there a specific technical blocker?
>
> On Fri, Sep 25, 2020 at 11:24 AM David Smiley <dsmiley@apache.org> wrote:
>
>
> The "userfiles" thing is merely a directory. There's no CRUD API on it. Even if the CRUD API of the file store isn't active in standalone -- I think that doesn't matter. You're left with the <solrhome>/filestore/ dir to put there what you want in standalone mode.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Fri, Sep 25, 2020 at 11:00 AM Ishan Chattopadhyaya <ichattopadhyaya@gmail.com> wrote:
>
>
> Standalone isn't supported.
>
> We need to transition away from the standalone mode and get rid of it completely.
>
> On Fri, 25 Sep, 2020, 7:50 pm Jason Gerlowski, <gerlowskija@gmail.com> wrote:
>
>
> I don't know much about the new package/file-store, but it does sound
> like a good replacement for 'userfiles' (which Eric is right- is
> pretty awkward to work with).
>
> My only concerns with using the filestore would be around its
> availability. Is it enabled by default (or will it be in an upcoming
> release)? Does it work in standalone as well as SolrCloud? If the
> answer to those questions are both "yes", then it makes sense to me as
> a replacement on the face of things.
>
> Jason
>
> On Fri, Sep 25, 2020 at 9:39 AM Eric Pugh
> <epugh@opensourceconnections.com> wrote:
>
>
> +1. I’ve found the user files really useful when doing things with streaming, but it’s also awkward to reach to put files into.
>
>
> On Sep 25, 2020, at 8:47 AM, David Smiley <dsmiley@apache.org> wrote:
>
> Yes. And I think it's high time that coreRootDirectory default to <solrhome>/something (e.g. "cores")
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Fri, Sep 25, 2020 at 8:42 AM Alexandre Rafalovitch <arafalov@gmail.com> wrote:
>
>
> Would that file store also be under solr.home? Because if it is and
> the user can upload core.property into it as well as other things that
> core discovery will then load bypassing security....
>
> Regards,
> Alex.
>
> On Fri, 25 Sep 2020 at 08:32, David Smiley <dsmiley@apache.org> wrote:
>
>
> I'm looking to see that we can deprecate "userfiles" and remove for 9.0
>
> Solr has a "userfiles" directory under Solr home that Jason added in some issue relating to streaming expressions accessing a local file. I bet only a few of you have even heard of it. I think the "File Store" that came along with the package manager obsoletes "userfiles". If you have not heard of the file store either, I wouldn't be surprised -- it's new and it's name was changed from "package store" last minute, since it's general purpose, with "packages" being a directory at the root of the file store for packages. It's documented as the "package store" (should be renamed) on the package manager internals doc: https://lucene.apache.org/solr/guide/8_6/package-manager-internals.html#package-store
> However IMO it's worthy of its own doc page as it's a very useful new component of the Solr platform. It can store "user files" (hence obsoleting the userfiles dir), ML models, or basically any file that's too large to put in ZK. I'd be nice if SolrResourceLoader could resolve resources from it -- an issue for another day. That would be another avenue of use separate from the configSet. You can already upload single files to the file store :-)
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: Filestore, subsume userfiles requirement [ In reply to ]

arafalov at gmail

Sep 25, 2020, 9:06 AM

Post #13 of 13 (1284 views)

Permalink

We are talking specifically about 'cat', I think. Nothing else
accesses that directory:
https://lucene.apache.org/solr/guide/8_6/stream-source-reference.html#cat

Seems standalone-ready. Very file-centric in fact. Makes me wonder if
it is SolrCloud ready....

Regards,
Alex.

On Fri, 25 Sep 2020 at 12:03, Jason Gerlowski <gerlowskija@gmail.com> wrote:
>
> I think some expressions are SolrCloud specific, but there are
> expressions that work in standalone
>
> e.g. https://paste.apache.org/zm3sw and https://paste.apache.org/vs35m
>
> On Fri, Sep 25, 2020 at 11:53 AM Eric Pugh
> <epugh@opensourceconnections.com> wrote:
> >
> > Can you do streaming expression in standalone solr? I was under the impression that it required SolrCloud.
> >
> >
> > On Sep 25, 2020, at 11:45 AM, Alexandre Rafalovitch <arafalov@gmail.com> wrote:
> >
> > It is not just a directory. There is a security feature that only
> > allows to read from several file locations (configurable in solr.xml).
> > Userfiles is part of default list, so whatever replacement is there
> > will need to be explicitly-named as well.
> >
> > Regards,
> > Alex.
> >
> > On Fri, 25 Sep 2020 at 11:43, Jason Gerlowski <gerlowskija@gmail.com> wrote:
> >
> >
> > You're left with the <solrhome>/filestore/ dir to put there what you want in standalone mode.
> >
> > That's fine by me as that's effectively what "userfiles" provides now.
> > The main difference I imagine is that the 'filestore' directory won't
> > exist at all in standalone? In that case standalone users will have to
> > know where to create the dir if they want to reference 'filestore'
> > files in their streaming expressions. But that's not all that much
> > more to ask I guess.
> >
> > I'm curious - is the filestore unimplemented in standalone because no
> > one prioritized it, or is there a specific technical blocker?
> >
> > On Fri, Sep 25, 2020 at 11:24 AM David Smiley <dsmiley@apache.org> wrote:
> >
> >
> > The "userfiles" thing is merely a directory. There's no CRUD API on it. Even if the CRUD API of the file store isn't active in standalone -- I think that doesn't matter. You're left with the <solrhome>/filestore/ dir to put there what you want in standalone mode.
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > On Fri, Sep 25, 2020 at 11:00 AM Ishan Chattopadhyaya <ichattopadhyaya@gmail.com> wrote:
> >
> >
> > Standalone isn't supported.
> >
> > We need to transition away from the standalone mode and get rid of it completely.
> >
> > On Fri, 25 Sep, 2020, 7:50 pm Jason Gerlowski, <gerlowskija@gmail.com> wrote:
> >
> >
> > I don't know much about the new package/file-store, but it does sound
> > like a good replacement for 'userfiles' (which Eric is right- is
> > pretty awkward to work with).
> >
> > My only concerns with using the filestore would be around its
> > availability. Is it enabled by default (or will it be in an upcoming
> > release)? Does it work in standalone as well as SolrCloud? If the
> > answer to those questions are both "yes", then it makes sense to me as
> > a replacement on the face of things.
> >
> > Jason
> >
> > On Fri, Sep 25, 2020 at 9:39 AM Eric Pugh
> > <epugh@opensourceconnections.com> wrote:
> >
> >
> > +1. I’ve found the user files really useful when doing things with streaming, but it’s also awkward to reach to put files into.
> >
> >
> > On Sep 25, 2020, at 8:47 AM, David Smiley <dsmiley@apache.org> wrote:
> >
> > Yes. And I think it's high time that coreRootDirectory default to <solrhome>/something (e.g. "cores")
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > On Fri, Sep 25, 2020 at 8:42 AM Alexandre Rafalovitch <arafalov@gmail.com> wrote:
> >
> >
> > Would that file store also be under solr.home? Because if it is and
> > the user can upload core.property into it as well as other things that
> > core discovery will then load bypassing security....
> >
> > Regards,
> > Alex.
> >
> > On Fri, 25 Sep 2020 at 08:32, David Smiley <dsmiley@apache.org> wrote:
> >
> >
> > I'm looking to see that we can deprecate "userfiles" and remove for 9.0
> >
> > Solr has a "userfiles" directory under Solr home that Jason added in some issue relating to streaming expressions accessing a local file. I bet only a few of you have even heard of it. I think the "File Store" that came along with the package manager obsoletes "userfiles". If you have not heard of the file store either, I wouldn't be surprised -- it's new and it's name was changed from "package store" last minute, since it's general purpose, with "packages" being a directory at the root of the file store for packages. It's documented as the "package store" (should be renamed) on the package manager internals doc: https://lucene.apache.org/solr/guide/8_6/package-manager-internals.html#package-store
> > However IMO it's worthy of its own doc page as it's a very useful new component of the Solr platform. It can store "user files" (hence obsoleting the userfiles dir), ML models, or basically any file that's too large to put in ZK. I'd be nice if SolrResourceLoader could resolve resources from it -- an issue for another day. That would be another avenue of use separate from the configSet. You can already upload single files to the file store :-)
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
> >
> > _______________________
> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> > This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
> >
> > _______________________
> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> > This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org