So it’s been a while since I’ve brought up SolrCloud sickness. Plenty to
navigate and figure out in the meantime. Given the constraints of life,
there was a point I wanted to give and share some insight into what I could
see. But it quickly became clear that was not a great plan - just what I
was left with because I could not dedicate or plan beyond a small or very
long time range. It turns out that telling someone, take a peak through
these key holes, can you see what I see? 10-100x better!? The skys the
limit? Don’t you see it?! What? The bathrooms? Yeah yeah, no the bathrooms
are being demolished, look here through the key hole. More units tests and
modules you say? Exaggeration confusion? Yeah yeah, never mind.
None the less, you do not tear down what you don’t have a plan to address.
And you cannot start anew until you have acknowledged and accepted the
past. Well, who knows, I have my own codes. What’s a man/woman without a
code.
And so, while maybe I’m always wrong or unintelligent, I’ve only ever
expressed a truth I feel I could and can defend, if the chips started
falling.
And at this point, I’d like to say that SolrCloud might be sic.
There is some ongoing work and plenty that will be happening for some time.
But I have not tried to create a collection or many collections other than
mostly around the small numbers that tests do (with the exception of 100
collections, created over time, sometimes with Ishans stress system). A
couple to a few collections. A handful of shards. A handful of replicas.
The other day I figured I’d modify a test just to see what I’d be dealing
with when I get some fun times soon.
I used a single 512mb test jvm, nightly settings and fired up a 12x12
collection on 4 jetty Solr runners. 144 SolrCores. It essentially just
started up and returned. Shit, is that broken? The test is green? No way.
Ok.
I would have bet 0 dollars on that first run.
Yesterday, okay, I’m feeling good, let’s roll the dice. I fire up 24 shards
and 24 replicas for each. I’m not even optimistic, I mean this is cold and
blind at these numbers. And after what looks like a nice start, things
quickly deteriorate. Ok, ok, looks like maybe a limit adjust and maybe I’m
asking a little much of 512mb of Ram. So I just bump it to 2 gig for some
play headroom.
Bam. 6 seconds. Fully green and active cluster. Sic.
That’s two runs, so yeah, I’ve got a future play date scheduled for more.
But, one, that’s indicative of a ridiculous amount of Solr code and
behavior. It’s a load bearing beam of action. But also, I’ve got similar
sicness buried all over the place.
So yeah, the system can rock. Yonik’s design sense was not the monster
after all, surprise ending. And interested parties could and will help take
it forward so very much further. Don’t mortgage the house yet, an Apache
release of my play time is not happening next week. But with the same shock
and horror and surety that I said SolrCloud looks sick, I revise to
SolrCloud looks sic. And it wants a second shot, and by god it will have it.
--
- Mark
http://about.me/markrmiller
navigate and figure out in the meantime. Given the constraints of life,
there was a point I wanted to give and share some insight into what I could
see. But it quickly became clear that was not a great plan - just what I
was left with because I could not dedicate or plan beyond a small or very
long time range. It turns out that telling someone, take a peak through
these key holes, can you see what I see? 10-100x better!? The skys the
limit? Don’t you see it?! What? The bathrooms? Yeah yeah, no the bathrooms
are being demolished, look here through the key hole. More units tests and
modules you say? Exaggeration confusion? Yeah yeah, never mind.
None the less, you do not tear down what you don’t have a plan to address.
And you cannot start anew until you have acknowledged and accepted the
past. Well, who knows, I have my own codes. What’s a man/woman without a
code.
And so, while maybe I’m always wrong or unintelligent, I’ve only ever
expressed a truth I feel I could and can defend, if the chips started
falling.
And at this point, I’d like to say that SolrCloud might be sic.
There is some ongoing work and plenty that will be happening for some time.
But I have not tried to create a collection or many collections other than
mostly around the small numbers that tests do (with the exception of 100
collections, created over time, sometimes with Ishans stress system). A
couple to a few collections. A handful of shards. A handful of replicas.
The other day I figured I’d modify a test just to see what I’d be dealing
with when I get some fun times soon.
I used a single 512mb test jvm, nightly settings and fired up a 12x12
collection on 4 jetty Solr runners. 144 SolrCores. It essentially just
started up and returned. Shit, is that broken? The test is green? No way.
Ok.
I would have bet 0 dollars on that first run.
Yesterday, okay, I’m feeling good, let’s roll the dice. I fire up 24 shards
and 24 replicas for each. I’m not even optimistic, I mean this is cold and
blind at these numbers. And after what looks like a nice start, things
quickly deteriorate. Ok, ok, looks like maybe a limit adjust and maybe I’m
asking a little much of 512mb of Ram. So I just bump it to 2 gig for some
play headroom.
Bam. 6 seconds. Fully green and active cluster. Sic.
That’s two runs, so yeah, I’ve got a future play date scheduled for more.
But, one, that’s indicative of a ridiculous amount of Solr code and
behavior. It’s a load bearing beam of action. But also, I’ve got similar
sicness buried all over the place.
So yeah, the system can rock. Yonik’s design sense was not the monster
after all, surprise ending. And interested parties could and will help take
it forward so very much further. Don’t mortgage the house yet, an Apache
release of my play time is not happening next week. But with the same shock
and horror and surety that I said SolrCloud looks sick, I revise to
SolrCloud looks sic. And it wants a second shot, and by god it will have it.
--
- Mark
http://about.me/markrmiller