Mailing List Archive

Re: Solr with HDFS configuration example running in production/dev
Hi

I would not go down this road. What is the use case? Is this really the solution?

Go read all the relevant docs and configuration provided by Cloudera/HortonWorks and everything else related to SOLR and HDFS.

I am not inclined to help you down a road you do not want to travel. There be dragons!

Andy

> On 20 Aug 2020, at 07:25, Prashant Jyoti <jtprashant@gmail.com> wrote:
>
> ?Hi Joe,
> These are the errors I am running into:
>
> org.apache.solr.common.SolrException: Error CREATEing SolrCore
> 'newcollsolr2_shard1_replica_n1': Unable to create core
> [newcollsolr2_shard1_replica_n1] Caused by: Illegal char <:> at index 4:
> hdfs://
> hn1-pjhado.tvbhpqtgh3judk1e5ihrx2k21d.tx.internal.cloudapp.net:8020/user/solr-data/newcollsolr2/core_node3/data\
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1256)
> at
> org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
> at
> org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
> at
> org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
> at
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)
> at
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1596)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
> at
> org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at org.eclipse.jetty.server.Server.handle(Server.java:500)
> at
> org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
> at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.solr.common.SolrException: Unable to create core
> [newcollsolr2_shard1_replica_n1]
> at
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1327)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)
> ... 47 more
> Caused by: org.apache.solr.common.SolrException: Illegal char <:> at index
> 4: hdfs://
> hn1-pjhado.tvbhpqtgh3judk1e5ihrx2k21d.tx.internal.cloudapp.net:8020/user/solr-data/newcollsolr2/core_node3/data\
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1072)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)
> at
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
> ... 48 more
> Caused by: java.nio.file.InvalidPathException: Illegal char <:> at index 4:
> hdfs://
> hn1-pjhado.tvbhpqtgh3judk1e5ihrx2k21d.tx.internal.cloudapp.net:8020/user/solr-data/newcollsolr2/core_node3/data\
> at sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
> at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
> at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
> at sun.nio.fs.WindowsPath.parse(WindowsPath.java:94)
> at sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:255)
> at sun.nio.fs.AbstractPath.resolve(AbstractPath.java:53)
> at org.apache.solr.core.SolrCore.initUpdateLogDir(SolrCore.java:1380)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:958)
> ... 50 more
>
> Attaching it as a file too. Thanks!
>
>> On Wed, Aug 19, 2020 at 9:37 PM Joe Obernberger <
>> joseph.obernberger@gmail.com> wrote:
>>
>> Your exception didn't come across - can you paste it in?
>>
>> -Joe
>>
>>> On 8/19/2020 10:50 AM, Prashant Jyoti wrote:
>>> You're right Andrew. Even I read about that. But there's a use case for
>>> which we want to configure the said case.
>>>
>>> Are you also aware of what feature we are moving towards instead of HDFS?
>>> Will you be able to help me with the error that I'm running into?
>>>
>>> Thanks in advance!
>>>
>>>
>>> On Wed, 19 Aug, 2020, 5:24 pm Andrew MacKay, <Andrew.MacKay@superna.net>
>>> wrote:
>>>
>>>> I believe HDFS support is being deprecated in Solr. Not sure you want
>> to
>>>> continue configuration if support will disappear.
>>>>
>>>> On Wed, Aug 19, 2020 at 7:52 AM Prashant Jyoti <jtprashant@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>> Hope you are healthy and safe.
>>>>>
>>>>> Need some help with HDFS configuration.
>>>>>
>>>>> Could anybody of you share an example of the configuration with which
>> you
>>>>> are running Solr with HDFS in any of your production/dev environments?
>>>>> I am interested in the parts of SolrConfig.xml / Solr.in.cmd/sh which
>> you
>>>>> may have modified. Obviously with the security parts obfuscated.
>>>>>
>>>>> I am stuck at an error and unable to move ahead. Attaching the
>> exception
>>>>> log if anyone is interested to look at the error.
>>>>>
>>>>> Thanks!
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Prashant.
>>>>>
>>>> --
>>>> CONFIDENTIALITY NOTICE: The information contained in this email is
>>>> privileged and confidential and intended only for the use of the
>>>> individual
>>>> or entity to whom it is addressed. If you receive this message in
>> error,
>>>> please notify the sender immediately at 613-729-1100 and destroy the
>>>> original message and all copies. Thank you.
>>>>
>>>
>>
>
>
> --
> Regards,
> Prashant.