Mailing List Archive

No segments* file found in HdfsDirectory
This must be a newbie question but I’ve tried just about everything found from google search and from the documentation but to no avail.
When trying to put index in HDFS, my attempt to create cores always fail with "no segments* file found in HdfsDirectory"

For example:

~/solr-5.2.1/server/solr/configsets> solr create_core -c robert -d hdfs_basic

Creating new core 'robert' using command:
http://localhost:8983/solr/admin/cores?action=CREATE&name=robert&instanceDir=robert

Failed to create core 'robert' due to: Error CREATEing SolrCore 'robert': Unable to create core [robert] Caused by: no segments* file found in HdfsDirectory@dafcec5f lockFactory=org.apache.solr.store.hdfs.HdfsLockFactory@5e3b9ecc: files: [write.lock]

In configsets/hdfs_basic/conf/solrconfig.xml I have the following:

<directoryFactory name="DirectoryFactory" class="solr.HdfsDirectoryFactory">
<str name="solr.hdfs.home">hdfs://wrnidphmn0002:9000/user/tzz2c94/solr</str>
<bool name="solr.hdfs.blockcache.enabled">true</bool>
<int name="solr.hdfs.blockcache.slab.count">1</int>
<bool name="solr.hdfs.blockcache.direct.memory.allocation">true</bool>
<int name="solr.hdfs.blockcache.blocksperbank">16384</int>
<bool name="solr.hdfs.blockcache.read.enabled">true</bool>
<bool name="solr.hdfs.blockcache.write.enabled">true/bool>
<bool name="solr.hdfs.nrtcachingdirectory.enable">true</bool>
<int name="solr.hdfs.nrtcachingdirectory.maxmergesizemb">16</int>
<int name="solr.hdfs.nrtcachingdirectory.maxcachedmb">192</int>
</directoryFactory>
<lockType>hdfs</lockType>

I've tested that the command is indeed reading this config file by the crude test of putting in a bad classname and seeing the appropriate error message.

I've also tried the alternative command line method,
solr start -Dsolr.directoryFactory=HdfsDirectoryFactory -Dsolr.lock.type=hdfs -Dsolr.hdfs.home=hdfs://wrnidphmn0002:9000/user/tzz2c94/solr
and with
solr start -Dsolr.directoryFactory=HdfsDirectoryFactory -Dsolr.lock.type=hdfs -Dsolr.data.dir=hdfs://wrnidphmn0002:9000/user/tzz2c94/solr/data -Dsolr.updatelog=hdfs://wrnidphmn0002:9000/user/$USER/solr/upd

I've also tried using cloud mode and from suggestions found from a google search, setting unlockOnStartup but neither helped.

I'm out of things to try. Can anybody help?



Nothing in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included in this message.

Confidentiality Note: This message is intended only for the person or entity to which it is addressed. It may contain confidential and/or privileged material. Any review, transmission, dissemination or other use, or taking of any action in reliance upon this message by persons or entities other than the intended recipient is prohibited and may be unlawful. If you received this message in error, please contact the sender and delete it from your computer.
Re: No segments* file found in HdfsDirectory [ In reply to ]
The Solr user list is the correct list for questions like this.

Sometimes you can end up in a situation where the root index folders exist
in HDFS, but with no files in them. You may need to manually remove them
and try again.

- Mark

On Sat, Aug 8, 2015 at 3:00 PM Robert B Hamilton <robert.hamilton@gm.com>
wrote:

> This must be a newbie question but I’ve tried just about everything found
> from google search and from the documentation but to no avail.
> When trying to put index in HDFS, my attempt to create cores always fail
> with "no segments* file found in HdfsDirectory"
>
> For example:
>
> ~/solr-5.2.1/server/solr/configsets> solr create_core -c robert -d
> hdfs_basic
>
> Creating new core 'robert' using command:
>
> http://localhost:8983/solr/admin/cores?action=CREATE&name=robert&instanceDir=robert
>
> Failed to create core 'robert' due to: Error CREATEing SolrCore 'robert':
> Unable to create core [robert] Caused by: no segments* file found in
> HdfsDirectory@dafcec5f
> lockFactory=org.apache.solr.store.hdfs.HdfsLockFactory@5e3b9ecc: files:
> [write.lock]
>
> In configsets/hdfs_basic/conf/solrconfig.xml I have the following:
>
> <directoryFactory name="DirectoryFactory"
> class="solr.HdfsDirectoryFactory">
> <str
> name="solr.hdfs.home">hdfs://wrnidphmn0002:9000/user/tzz2c94/solr</str>
> <bool name="solr.hdfs.blockcache.enabled">true</bool>
> <int name="solr.hdfs.blockcache.slab.count">1</int>
> <bool
> name="solr.hdfs.blockcache.direct.memory.allocation">true</bool>
> <int name="solr.hdfs.blockcache.blocksperbank">16384</int>
> <bool name="solr.hdfs.blockcache.read.enabled">true</bool>
> <bool name="solr.hdfs.blockcache.write.enabled">true/bool>
> <bool name="solr.hdfs.nrtcachingdirectory.enable">true</bool>
> <int name="solr.hdfs.nrtcachingdirectory.maxmergesizemb">16</int>
> <int name="solr.hdfs.nrtcachingdirectory.maxcachedmb">192</int>
> </directoryFactory>
> <lockType>hdfs</lockType>
>
> I've tested that the command is indeed reading this config file by the
> crude test of putting in a bad classname and seeing the appropriate error
> message.
>
> I've also tried the alternative command line method,
> solr start -Dsolr.directoryFactory=HdfsDirectoryFactory
> -Dsolr.lock.type=hdfs
> -Dsolr.hdfs.home=hdfs://wrnidphmn0002:9000/user/tzz2c94/solr
> and with
> solr start -Dsolr.directoryFactory=HdfsDirectoryFactory
> -Dsolr.lock.type=hdfs
> -Dsolr.data.dir=hdfs://wrnidphmn0002:9000/user/tzz2c94/solr/data
> -Dsolr.updatelog=hdfs://wrnidphmn0002:9000/user/$USER/solr/upd
>
> I've also tried using cloud mode and from suggestions found from a google
> search, setting unlockOnStartup but neither helped.
>
> I'm out of things to try. Can anybody help?
>
>
>
> Nothing in this message is intended to constitute an electronic signature
> unless a specific statement to the contrary is included in this message.
>
> Confidentiality Note: This message is intended only for the person or
> entity to which it is addressed. It may contain confidential and/or
> privileged material. Any review, transmission, dissemination or other use,
> or taking of any action in reliance upon this message by persons or
> entities other than the intended recipient is prohibited and may be
> unlawful. If you received this message in error, please contact the sender
> and delete it from your computer.
>
--
- Mark
about.me/markrmiller