So I've never really been in the position of having the time or tools to
easily and efficiently pound on Lucene updates - always lacked one of the
ingredients, and having a beer while a bit stocked up on both, I've been
doing some light hammering.
Not likely some high priority item, when I say efficient, I mean lots
coming in fast with minimal context switching or outside blocking and
locking - not that I'm in some reactive / back pressure bulldozer. But I
thought it might be interesting to someone, because I think I've seen hints
of it at a lesser scale when pointing any fingers towards Lucene was beyond
the effort or value.
So just a drop, don't mind a reply of "seems to hurt when I hit you" "Well
don't hit me so hard".
Somewhat older code, looks like maybe the issue could still hang out.
Seems to be the applylock in FrozenUpdates. I don't recall all the paths
coming in on it (I can easily reproduce), but it takes a few things
occurring around the same time, and the result ranges from massive slow
down that works through to essentially deadlock.
Seen a lot of this kind of behavior before, hard breaks to deadlock but
often hints of progress unwinding, so I gave a fair option on the lock a
try and seemed to address it for me without a noticeable penalty. I didn't
do rigorous testing and came at it from one angle. But took me to smooth
pounding from the off and on misery runs.
Also, unrelated, but since I saw someone struggling with it recently, I
might as well mention, I think there may be a few SPI related poor static
initializers in CharFilterFactory, TokenFilterFactory, TokenizerFactory -
don't quote me on the classes, but I believe a nice static holder pattern
there solves some fairly easy to trigger deadlock issues. Worked well for
me and I later found a similar class that already had this pattern with a
note about the deadlock doom impetus.
Sorry to pound, back pressure aint easy and "reactive" gets me about as
excited as Perl.
- Mark
easily and efficiently pound on Lucene updates - always lacked one of the
ingredients, and having a beer while a bit stocked up on both, I've been
doing some light hammering.
Not likely some high priority item, when I say efficient, I mean lots
coming in fast with minimal context switching or outside blocking and
locking - not that I'm in some reactive / back pressure bulldozer. But I
thought it might be interesting to someone, because I think I've seen hints
of it at a lesser scale when pointing any fingers towards Lucene was beyond
the effort or value.
So just a drop, don't mind a reply of "seems to hurt when I hit you" "Well
don't hit me so hard".
Somewhat older code, looks like maybe the issue could still hang out.
Seems to be the applylock in FrozenUpdates. I don't recall all the paths
coming in on it (I can easily reproduce), but it takes a few things
occurring around the same time, and the result ranges from massive slow
down that works through to essentially deadlock.
Seen a lot of this kind of behavior before, hard breaks to deadlock but
often hints of progress unwinding, so I gave a fair option on the lock a
try and seemed to address it for me without a noticeable penalty. I didn't
do rigorous testing and came at it from one angle. But took me to smooth
pounding from the off and on misery runs.
Also, unrelated, but since I saw someone struggling with it recently, I
might as well mention, I think there may be a few SPI related poor static
initializers in CharFilterFactory, TokenFilterFactory, TokenizerFactory -
don't quote me on the classes, but I believe a nice static holder pattern
there solves some fairly easy to trigger deadlock issues. Worked well for
me and I later found a similar class that already had this pattern with a
note about the deadlock doom impetus.
Sorry to pound, back pressure aint easy and "reactive" gets me about as
excited as Perl.
- Mark