Mailing List Archive

Our test failure rate is unacceptable.
http://fucit.org/solr-jenkins-reports/failure-report.html

HdfsAutoAddReplicasTest failing 100% of the time.
TestPackages.classMethod failing 100% of the time
3-4 AutoAddReplicas tests failing 98% of the time.

Is anyone looking at these? I realize the code base is changing a lot, and some temporary instability is to be expected. What I’d like is for some indication that people are actively addressing these.

Erick
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Our test failure rate is unacceptable. [ In reply to ]
IMO if a temporary instability is to be introduced deliberately, it should
be published on the list. If it’s inadvertently added, we either fix it
within an hour or so or revert the offending commit.

On Fri, 18 Sep 2020 at 20:26, Erick Erickson <erickerickson@gmail.com>
wrote:

> http://fucit.org/solr-jenkins-reports/failure-report.html
>
>
>
> HdfsAutoAddReplicasTest failing 100% of the time.
>
> TestPackages.classMethod failing 100% of the time
>
> 3-4 AutoAddReplicas tests failing 98% of the time.
>
>
>
> Is anyone looking at these? I realize the code base is changing a lot, and
> some temporary instability is to be expected. What I’d like is for some
> indication that people are actively addressing these.
>
>
>
> Erick
>
> ---------------------------------------------------------------------
>
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
>
> --
Regards,

Atri
Apache Concerted
Re: Our test failure rate is unacceptable. [ In reply to ]
bq. IMO if a temporary instability is to be introduced deliberately, it should be published on the list

Actually, I disagree. Having anything in the tests that fail 100% of the time is just unacceptable since it becomes a barrier for everyone else. AFAIK, if the problem can be identified to a particular push, I have no problems with that push being unilaterally rolled back.

The exception for me is when the problem is addressed immediately, I’ve certainly been the source of that kind of problem, as have others.

What I take great exception to is the fact that some of these tests have been failing 100% of the time for the last seven days! If it’s the case that the full test suite was never run before the push that’s another discussion. Yeah, it takes a long time but…

Erick

> On Sep 18, 2020, at 11:28 AM, Atri Sharma <atri@apache.org> wrote:
>
> IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit.
>
> On Fri, 18 Sep 2020 at 20:26, Erick Erickson <erickerickson@gmail.com> wrote:
> http://fucit.org/solr-jenkins-reports/failure-report.html
>
>
>
> HdfsAutoAddReplicasTest failing 100% of the time.
>
> TestPackages.classMethod failing 100% of the time
>
> 3-4 AutoAddReplicas tests failing 98% of the time.
>
>
>
> Is anyone looking at these? I realize the code base is changing a lot, and some temporary instability is to be expected. What I’d like is for some indication that people are actively addressing these.
>
>
>
> Erick
>
> ---------------------------------------------------------------------
>
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
>
> --
> Regards,
>
> Atri
> Apache Concerted


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Our test failure rate is unacceptable. [ In reply to ]
HdfsAutoAddReplicasTest is just wrapping AutoAddReplicas tests with HDFS
last I checked. There haven't been any HDFS changes lately that I've seen.
So what changed in regards to AutoAddReplicas? Based on
http://fucit.org/solr-jenkins-reports/history-trend-of-recent-failures.html#series/org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testAddCollectionPropAfterCreation
it looks like somewhere between 9/7 and 9/14.

Kevin Risden


On Fri, Sep 18, 2020 at 11:28 AM Atri Sharma <atri@apache.org> wrote:

> IMO if a temporary instability is to be introduced deliberately, it should
> be published on the list. If it’s inadvertently added, we either fix it
> within an hour or so or revert the offending commit.
>
> On Fri, 18 Sep 2020 at 20:26, Erick Erickson <erickerickson@gmail.com>
> wrote:
>
>> http://fucit.org/solr-jenkins-reports/failure-report.html
>>
>>
>>
>> HdfsAutoAddReplicasTest failing 100% of the time.
>>
>> TestPackages.classMethod failing 100% of the time
>>
>> 3-4 AutoAddReplicas tests failing 98% of the time.
>>
>>
>>
>> Is anyone looking at these? I realize the code base is changing a lot,
>> and some temporary instability is to be expected. What I’d like is for some
>> indication that people are actively addressing these.
>>
>>
>>
>> Erick
>>
>> ---------------------------------------------------------------------
>>
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>>
>> --
> Regards,
>
> Atri
> Apache Concerted
>
Re: Our test failure rate is unacceptable. [ In reply to ]
When I said temporary, I meant 3-4 hours. Definitely not more than that.

IMO we should just roll back offending commits if they are easily
identifiable. I agree with you — we all have been guilty of breaking builds
(mea culpa as well). The bad part here is the longevity of the failures.


On Fri, 18 Sep 2020 at 21:05, Erick Erickson <erickerickson@gmail.com>
wrote:

> bq. IMO if a temporary instability is to be introduced deliberately, it
> should be published on the list
>
>
>
> Actually, I disagree. Having anything in the tests that fail 100% of the
> time is just unacceptable since it becomes a barrier for everyone else.
> AFAIK, if the problem can be identified to a particular push, I have no
> problems with that push being unilaterally rolled back.
>
>
>
> The exception for me is when the problem is addressed immediately, I’ve
> certainly been the source of that kind of problem, as have others.
>
>
>
> What I take great exception to is the fact that some of these tests have
> been failing 100% of the time for the last seven days! If it’s the case
> that the full test suite was never run before the push that’s another
> discussion. Yeah, it takes a long time but…
>
>
>
> Erick
>
>
>
> > On Sep 18, 2020, at 11:28 AM, Atri Sharma <atri@apache.org> wrote:
>
> >
>
> > IMO if a temporary instability is to be introduced deliberately, it
> should be published on the list. If it’s inadvertently added, we either fix
> it within an hour or so or revert the offending commit.
>
> >
>
> > On Fri, 18 Sep 2020 at 20:26, Erick Erickson <erickerickson@gmail.com>
> wrote:
>
> > http://fucit.org/solr-jenkins-reports/failure-report.html
>
> >
>
> >
>
> >
>
> > HdfsAutoAddReplicasTest failing 100% of the time.
>
> >
>
> > TestPackages.classMethod failing 100% of the time
>
> >
>
> > 3-4 AutoAddReplicas tests failing 98% of the time.
>
> >
>
> >
>
> >
>
> > Is anyone looking at these? I realize the code base is changing a lot,
> and some temporary instability is to be expected. What I’d like is for some
> indication that people are actively addressing these.
>
> >
>
> >
>
> >
>
> > Erick
>
> >
>
> > ---------------------------------------------------------------------
>
> >
>
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>
> >
>
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
> >
>
> >
>
> >
>
> > --
>
> > Regards,
>
> >
>
> > Atri
>
> > Apache Concerted
>
>
>
>
>
> ---------------------------------------------------------------------
>
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
>
> --
Regards,

Atri
Apache Concerted
Re: Our test failure rate is unacceptable. [ In reply to ]
I believe these failures are associated to
https://issues.apache.org/jira/browse/SOLR-14151

• FAILED: org.apache.solr.pkg.TestPackages.classMethod
• FAILED:
org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest.testAdd2Fields
• FAILED:
org.apache.solr.schema.ManagedSchemaRoundRobinCloudTest.testAddFieldsRoundRobin

> IMO if a temporary instability is to be introduced deliberately, it
should be published on the list. If it’s inadvertently added, we either fix
it within an hour or so or revert the offending commit
I don't want to set specific time frames, but sometimes it's obviously too
much time.

On Fri, Sep 18, 2020 at 8:48 AM Atri Sharma <atri@apache.org> wrote:

> When I said temporary, I meant 3-4 hours. Definitely not more than that.
>
> IMO we should just roll back offending commits if they are easily
> identifiable. I agree with you — we all have been guilty of breaking builds
> (mea culpa as well). The bad part here is the longevity of the failures.
>
>
> On Fri, 18 Sep 2020 at 21:05, Erick Erickson <erickerickson@gmail.com>
> wrote:
>
>> bq. IMO if a temporary instability is to be introduced deliberately, it
>> should be published on the list
>>
>>
>>
>> Actually, I disagree. Having anything in the tests that fail 100% of the
>> time is just unacceptable since it becomes a barrier for everyone else.
>> AFAIK, if the problem can be identified to a particular push, I have no
>> problems with that push being unilaterally rolled back.
>>
>>
>>
>> The exception for me is when the problem is addressed immediately, I’ve
>> certainly been the source of that kind of problem, as have others.
>>
>>
>>
>> What I take great exception to is the fact that some of these tests have
>> been failing 100% of the time for the last seven days! If it’s the case
>> that the full test suite was never run before the push that’s another
>> discussion. Yeah, it takes a long time but…
>>
>>
>>
>> Erick
>>
>>
>>
>> > On Sep 18, 2020, at 11:28 AM, Atri Sharma <atri@apache.org> wrote:
>>
>> >
>>
>> > IMO if a temporary instability is to be introduced deliberately, it
>> should be published on the list. If it’s inadvertently added, we either fix
>> it within an hour or so or revert the offending commit.
>>
>> >
>>
>> > On Fri, 18 Sep 2020 at 20:26, Erick Erickson <erickerickson@gmail.com>
>> wrote:
>>
>> > http://fucit.org/solr-jenkins-reports/failure-report.html
>>
>> >
>>
>> >
>>
>> >
>>
>> > HdfsAutoAddReplicasTest failing 100% of the time.
>>
>> >
>>
>> > TestPackages.classMethod failing 100% of the time
>>
>> >
>>
>> > 3-4 AutoAddReplicas tests failing 98% of the time.
>>
>> >
>>
>> >
>>
>> >
>>
>> > Is anyone looking at these? I realize the code base is changing a lot,
>> and some temporary instability is to be expected. What I’d like is for some
>> indication that people are actively addressing these.
>>
>> >
>>
>> >
>>
>> >
>>
>> > Erick
>>
>> >
>>
>> > ---------------------------------------------------------------------
>>
>> >
>>
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>
>> >
>>
>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>
>> >
>>
>> >
>>
>> >
>>
>> > --
>>
>> > Regards,
>>
>> >
>>
>> > Atri
>>
>> > Apache Concerted
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>>
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>>
>> --
> Regards,
>
> Atri
> Apache Concerted
>
Re: Our test failure rate is unacceptable. [ In reply to ]
> If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit

> I don't want to set specific time frames,

To play Devil's Advocate here: why wait even an hour to revert a 100%
test failure? Reverts are usually trivial to do, unblock others
immediately, and don't interfere with the fix process at all.
Remembering the times I've broken the build myself, reverts even seem
preferable from that position - reverting up front takes all the
time-pressure off of getting out a fix. Why work under the gun when
you don't have to?

On Fri, Sep 18, 2020 at 1:14 PM Tomás Fernández Löbbe
<tomasflobbe@gmail.com> wrote:
>
> I believe these failures are associated to https://issues.apache.org/jira/browse/SOLR-14151
>
> • FAILED: org.apache.solr.pkg.TestPackages.classMethod
> • FAILED: org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest.testAdd2Fields
> • FAILED: org.apache.solr.schema.ManagedSchemaRoundRobinCloudTest.testAddFieldsRoundRobin
>
> > IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit
> I don't want to set specific time frames, but sometimes it's obviously too much time.
>
> On Fri, Sep 18, 2020 at 8:48 AM Atri Sharma <atri@apache.org> wrote:
>>
>> When I said temporary, I meant 3-4 hours. Definitely not more than that.
>>
>> IMO we should just roll back offending commits if they are easily identifiable. I agree with you — we all have been guilty of breaking builds (mea culpa as well). The bad part here is the longevity of the failures.
>>
>>
>> On Fri, 18 Sep 2020 at 21:05, Erick Erickson <erickerickson@gmail.com> wrote:
>>>
>>> bq. IMO if a temporary instability is to be introduced deliberately, it should be published on the list
>>>
>>>
>>>
>>> Actually, I disagree. Having anything in the tests that fail 100% of the time is just unacceptable since it becomes a barrier for everyone else. AFAIK, if the problem can be identified to a particular push, I have no problems with that push being unilaterally rolled back.
>>>
>>>
>>>
>>> The exception for me is when the problem is addressed immediately, I’ve certainly been the source of that kind of problem, as have others.
>>>
>>>
>>>
>>> What I take great exception to is the fact that some of these tests have been failing 100% of the time for the last seven days! If it’s the case that the full test suite was never run before the push that’s another discussion. Yeah, it takes a long time but…
>>>
>>>
>>>
>>> Erick
>>>
>>>
>>>
>>> > On Sep 18, 2020, at 11:28 AM, Atri Sharma <atri@apache.org> wrote:
>>>
>>> >
>>>
>>> > IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit.
>>>
>>> >
>>>
>>> > On Fri, 18 Sep 2020 at 20:26, Erick Erickson <erickerickson@gmail.com> wrote:
>>>
>>> > http://fucit.org/solr-jenkins-reports/failure-report.html
>>>
>>> >
>>>
>>> >
>>>
>>> >
>>>
>>> > HdfsAutoAddReplicasTest failing 100% of the time.
>>>
>>> >
>>>
>>> > TestPackages.classMethod failing 100% of the time
>>>
>>> >
>>>
>>> > 3-4 AutoAddReplicas tests failing 98% of the time.
>>>
>>> >
>>>
>>> >
>>>
>>> >
>>>
>>> > Is anyone looking at these? I realize the code base is changing a lot, and some temporary instability is to be expected. What I’d like is for some indication that people are actively addressing these.
>>>
>>> >
>>>
>>> >
>>>
>>> >
>>>
>>> > Erick
>>>
>>> >
>>>
>>> > ---------------------------------------------------------------------
>>>
>>> >
>>>
>>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>
>>> >
>>>
>>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>> >
>>>
>>> >
>>>
>>> >
>>>
>>> > --
>>>
>>> > Regards,
>>>
>>> >
>>>
>>> > Atri
>>>
>>> > Apache Concerted
>>>
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>>
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
>>>
>> --
>> Regards,
>>
>> Atri
>> Apache Concerted

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Our test failure rate is unacceptable. [ In reply to ]
Sometimes Jenkins may take hours to take your commit, may fail in the
middle of your night, may not fail consistently, etc. That's why I don't
think giving specific timeframes makes sense, but yes, as soon as you
notice it's failing, it's either fix immediately or revert IMO.

On Fri, Sep 18, 2020 at 12:03 PM Jason Gerlowski <gerlowskija@gmail.com>
wrote:

> > If it’s inadvertently added, we either fix it within an hour or so or
> revert the offending commit
>
> > I don't want to set specific time frames,
>
> To play Devil's Advocate here: why wait even an hour to revert a 100%
> test failure? Reverts are usually trivial to do, unblock others
> immediately, and don't interfere with the fix process at all.
> Remembering the times I've broken the build myself, reverts even seem
> preferable from that position - reverting up front takes all the
> time-pressure off of getting out a fix. Why work under the gun when
> you don't have to?
>
> On Fri, Sep 18, 2020 at 1:14 PM Tomás Fernández Löbbe
> <tomasflobbe@gmail.com> wrote:
> >
> > I believe these failures are associated to
> https://issues.apache.org/jira/browse/SOLR-14151
> >
> > • FAILED: org.apache.solr.pkg.TestPackages.classMethod
> > • FAILED:
> org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest.testAdd2Fields
> > • FAILED:
> org.apache.solr.schema.ManagedSchemaRoundRobinCloudTest.testAddFieldsRoundRobin
> >
> > > IMO if a temporary instability is to be introduced deliberately, it
> should be published on the list. If it’s inadvertently added, we either fix
> it within an hour or so or revert the offending commit
> > I don't want to set specific time frames, but sometimes it's obviously
> too much time.
> >
> > On Fri, Sep 18, 2020 at 8:48 AM Atri Sharma <atri@apache.org> wrote:
> >>
> >> When I said temporary, I meant 3-4 hours. Definitely not more than that.
> >>
> >> IMO we should just roll back offending commits if they are easily
> identifiable. I agree with you — we all have been guilty of breaking builds
> (mea culpa as well). The bad part here is the longevity of the failures.
> >>
> >>
> >> On Fri, 18 Sep 2020 at 21:05, Erick Erickson <erickerickson@gmail.com>
> wrote:
> >>>
> >>> bq. IMO if a temporary instability is to be introduced deliberately,
> it should be published on the list
> >>>
> >>>
> >>>
> >>> Actually, I disagree. Having anything in the tests that fail 100% of
> the time is just unacceptable since it becomes a barrier for everyone else.
> AFAIK, if the problem can be identified to a particular push, I have no
> problems with that push being unilaterally rolled back.
> >>>
> >>>
> >>>
> >>> The exception for me is when the problem is addressed immediately,
> I’ve certainly been the source of that kind of problem, as have others.
> >>>
> >>>
> >>>
> >>> What I take great exception to is the fact that some of these tests
> have been failing 100% of the time for the last seven days! If it’s the
> case that the full test suite was never run before the push that’s another
> discussion. Yeah, it takes a long time but…
> >>>
> >>>
> >>>
> >>> Erick
> >>>
> >>>
> >>>
> >>> > On Sep 18, 2020, at 11:28 AM, Atri Sharma <atri@apache.org> wrote:
> >>>
> >>> >
> >>>
> >>> > IMO if a temporary instability is to be introduced deliberately, it
> should be published on the list. If it’s inadvertently added, we either fix
> it within an hour or so or revert the offending commit.
> >>>
> >>> >
> >>>
> >>> > On Fri, 18 Sep 2020 at 20:26, Erick Erickson <
> erickerickson@gmail.com> wrote:
> >>>
> >>> > http://fucit.org/solr-jenkins-reports/failure-report.html
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> > HdfsAutoAddReplicasTest failing 100% of the time.
> >>>
> >>> >
> >>>
> >>> > TestPackages.classMethod failing 100% of the time
> >>>
> >>> >
> >>>
> >>> > 3-4 AutoAddReplicas tests failing 98% of the time.
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> > Is anyone looking at these? I realize the code base is changing a
> lot, and some temporary instability is to be expected. What I’d like is for
> some indication that people are actively addressing these.
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> > Erick
> >>>
> >>> >
> >>>
> >>> > ---------------------------------------------------------------------
> >>>
> >>> >
> >>>
> >>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>>
> >>> >
> >>>
> >>> > For additional commands, e-mail: dev-help@lucene.apache.org
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> > --
> >>>
> >>> > Regards,
> >>>
> >>> >
> >>>
> >>> > Atri
> >>>
> >>> > Apache Concerted
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>>
> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>>
> >>> For additional commands, e-mail: dev-help@lucene.apache.org
> >>>
> >>>
> >>>
> >> --
> >> Regards,
> >>
> >> Atri
> >> Apache Concerted
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: Our test failure rate is unacceptable. [ In reply to ]
Jason:

I wouldn’t object to reverting immediately. It’s a courtesy to give someone a chance to fix it themselves rather than unilaterally revert is all.

> On Sep 18, 2020, at 3:05 PM, Tomás Fernández Löbbe <tomasflobbe@gmail.com> wrote:
>
> Sometimes Jenkins may take hours to take your commit, may fail in the middle of your night, may not fail consistently, etc. That's why I don't think giving specific timeframes makes sense, but yes, as soon as you notice it's failing, it's either fix immediately or revert IMO.
>
> On Fri, Sep 18, 2020 at 12:03 PM Jason Gerlowski <gerlowskija@gmail.com> wrote:
> > If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit
>
> > I don't want to set specific time frames,
>
> To play Devil's Advocate here: why wait even an hour to revert a 100%
> test failure? Reverts are usually trivial to do, unblock others
> immediately, and don't interfere with the fix process at all.
> Remembering the times I've broken the build myself, reverts even seem
> preferable from that position - reverting up front takes all the
> time-pressure off of getting out a fix. Why work under the gun when
> you don't have to?
>
> On Fri, Sep 18, 2020 at 1:14 PM Tomás Fernández Löbbe
> <tomasflobbe@gmail.com> wrote:
> >
> > I believe these failures are associated to https://issues.apache.org/jira/browse/SOLR-14151
> >
> > • FAILED: org.apache.solr.pkg.TestPackages.classMethod
> > • FAILED: org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest.testAdd2Fields
> > • FAILED: org.apache.solr.schema.ManagedSchemaRoundRobinCloudTest.testAddFieldsRoundRobin
> >
> > > IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit
> > I don't want to set specific time frames, but sometimes it's obviously too much time.
> >
> > On Fri, Sep 18, 2020 at 8:48 AM Atri Sharma <atri@apache.org> wrote:
> >>
> >> When I said temporary, I meant 3-4 hours. Definitely not more than that.
> >>
> >> IMO we should just roll back offending commits if they are easily identifiable. I agree with you — we all have been guilty of breaking builds (mea culpa as well). The bad part here is the longevity of the failures.
> >>
> >>
> >> On Fri, 18 Sep 2020 at 21:05, Erick Erickson <erickerickson@gmail.com> wrote:
> >>>
> >>> bq. IMO if a temporary instability is to be introduced deliberately, it should be published on the list
> >>>
> >>>
> >>>
> >>> Actually, I disagree. Having anything in the tests that fail 100% of the time is just unacceptable since it becomes a barrier for everyone else. AFAIK, if the problem can be identified to a particular push, I have no problems with that push being unilaterally rolled back.
> >>>
> >>>
> >>>
> >>> The exception for me is when the problem is addressed immediately, I’ve certainly been the source of that kind of problem, as have others.
> >>>
> >>>
> >>>
> >>> What I take great exception to is the fact that some of these tests have been failing 100% of the time for the last seven days! If it’s the case that the full test suite was never run before the push that’s another discussion. Yeah, it takes a long time but…
> >>>
> >>>
> >>>
> >>> Erick
> >>>
> >>>
> >>>
> >>> > On Sep 18, 2020, at 11:28 AM, Atri Sharma <atri@apache.org> wrote:
> >>>
> >>> >
> >>>
> >>> > IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit.
> >>>
> >>> >
> >>>
> >>> > On Fri, 18 Sep 2020 at 20:26, Erick Erickson <erickerickson@gmail.com> wrote:
> >>>
> >>> > http://fucit.org/solr-jenkins-reports/failure-report.html
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> > HdfsAutoAddReplicasTest failing 100% of the time.
> >>>
> >>> >
> >>>
> >>> > TestPackages.classMethod failing 100% of the time
> >>>
> >>> >
> >>>
> >>> > 3-4 AutoAddReplicas tests failing 98% of the time.
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> > Is anyone looking at these? I realize the code base is changing a lot, and some temporary instability is to be expected. What I’d like is for some indication that people are actively addressing these.
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> > Erick
> >>>
> >>> >
> >>>
> >>> > ---------------------------------------------------------------------
> >>>
> >>> >
> >>>
> >>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>>
> >>> >
> >>>
> >>> > For additional commands, e-mail: dev-help@lucene.apache.org
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> >
> >>>
> >>> > --
> >>>
> >>> > Regards,
> >>>
> >>> >
> >>>
> >>> > Atri
> >>>
> >>> > Apache Concerted
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>>
> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>>
> >>> For additional commands, e-mail: dev-help@lucene.apache.org
> >>>
> >>>
> >>>
> >> --
> >> Regards,
> >>
> >> Atri
> >> Apache Concerted
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Our test failure rate is unacceptable. [ In reply to ]
I don't think it is along the Apache way to revert somebody's commit
without an explicit permission to do so... Not that I would personally
mind if somebody did it for me.

On Fri, Sep 18, 2020 at 9:06 PM Tomás Fernández Löbbe
<tomasflobbe@gmail.com> wrote:
>
> Sometimes Jenkins may take hours to take your commit, may fail in the middle of your night, may not fail consistently, etc. That's why I don't think giving specific timeframes makes sense, but yes, as soon as you notice it's failing, it's either fix immediately or revert IMO.
>
> On Fri, Sep 18, 2020 at 12:03 PM Jason Gerlowski <gerlowskija@gmail.com> wrote:
>>
>> > If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit
>>
>> > I don't want to set specific time frames,
>>
>> To play Devil's Advocate here: why wait even an hour to revert a 100%
>> test failure? Reverts are usually trivial to do, unblock others
>> immediately, and don't interfere with the fix process at all.
>> Remembering the times I've broken the build myself, reverts even seem
>> preferable from that position - reverting up front takes all the
>> time-pressure off of getting out a fix. Why work under the gun when
>> you don't have to?
>>
>> On Fri, Sep 18, 2020 at 1:14 PM Tomás Fernández Löbbe
>> <tomasflobbe@gmail.com> wrote:
>> >
>> > I believe these failures are associated to https://issues.apache.org/jira/browse/SOLR-14151
>> >
>> > • FAILED: org.apache.solr.pkg.TestPackages.classMethod
>> > • FAILED: org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest.testAdd2Fields
>> > • FAILED: org.apache.solr.schema.ManagedSchemaRoundRobinCloudTest.testAddFieldsRoundRobin
>> >
>> > > IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit
>> > I don't want to set specific time frames, but sometimes it's obviously too much time.
>> >
>> > On Fri, Sep 18, 2020 at 8:48 AM Atri Sharma <atri@apache.org> wrote:
>> >>
>> >> When I said temporary, I meant 3-4 hours. Definitely not more than that.
>> >>
>> >> IMO we should just roll back offending commits if they are easily identifiable. I agree with you — we all have been guilty of breaking builds (mea culpa as well). The bad part here is the longevity of the failures.
>> >>
>> >>
>> >> On Fri, 18 Sep 2020 at 21:05, Erick Erickson <erickerickson@gmail.com> wrote:
>> >>>
>> >>> bq. IMO if a temporary instability is to be introduced deliberately, it should be published on the list
>> >>>
>> >>>
>> >>>
>> >>> Actually, I disagree. Having anything in the tests that fail 100% of the time is just unacceptable since it becomes a barrier for everyone else. AFAIK, if the problem can be identified to a particular push, I have no problems with that push being unilaterally rolled back.
>> >>>
>> >>>
>> >>>
>> >>> The exception for me is when the problem is addressed immediately, I’ve certainly been the source of that kind of problem, as have others.
>> >>>
>> >>>
>> >>>
>> >>> What I take great exception to is the fact that some of these tests have been failing 100% of the time for the last seven days! If it’s the case that the full test suite was never run before the push that’s another discussion. Yeah, it takes a long time but…
>> >>>
>> >>>
>> >>>
>> >>> Erick
>> >>>
>> >>>
>> >>>
>> >>> > On Sep 18, 2020, at 11:28 AM, Atri Sharma <atri@apache.org> wrote:
>> >>>
>> >>> >
>> >>>
>> >>> > IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit.
>> >>>
>> >>> >
>> >>>
>> >>> > On Fri, 18 Sep 2020 at 20:26, Erick Erickson <erickerickson@gmail.com> wrote:
>> >>>
>> >>> > http://fucit.org/solr-jenkins-reports/failure-report.html
>> >>>
>> >>> >
>> >>>
>> >>> >
>> >>>
>> >>> >
>> >>>
>> >>> > HdfsAutoAddReplicasTest failing 100% of the time.
>> >>>
>> >>> >
>> >>>
>> >>> > TestPackages.classMethod failing 100% of the time
>> >>>
>> >>> >
>> >>>
>> >>> > 3-4 AutoAddReplicas tests failing 98% of the time.
>> >>>
>> >>> >
>> >>>
>> >>> >
>> >>>
>> >>> >
>> >>>
>> >>> > Is anyone looking at these? I realize the code base is changing a lot, and some temporary instability is to be expected. What I’d like is for some indication that people are actively addressing these.
>> >>>
>> >>> >
>> >>>
>> >>> >
>> >>>
>> >>> >
>> >>>
>> >>> > Erick
>> >>>
>> >>> >
>> >>>
>> >>> > ---------------------------------------------------------------------
>> >>>
>> >>> >
>> >>>
>> >>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >>>
>> >>> >
>> >>>
>> >>> > For additional commands, e-mail: dev-help@lucene.apache.org
>> >>>
>> >>> >
>> >>>
>> >>> >
>> >>>
>> >>> >
>> >>>
>> >>> > --
>> >>>
>> >>> > Regards,
>> >>>
>> >>> >
>> >>>
>> >>> > Atri
>> >>>
>> >>> > Apache Concerted
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>>
>> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >>>
>> >>> For additional commands, e-mail: dev-help@lucene.apache.org
>> >>>
>> >>>
>> >>>
>> >> --
>> >> Regards,
>> >>
>> >> Atri
>> >> Apache Concerted
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Our test failure rate is unacceptable. [ In reply to ]
I thought we were talking about reverting your own commits, not someone
else’s...

On Fri, Sep 18, 2020 at 12:31 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:

> I don't think it is along the Apache way to revert somebody's commit
>
> without an explicit permission to do so... Not that I would personally
>
> mind if somebody did it for me.
>
>
>
> On Fri, Sep 18, 2020 at 9:06 PM Tomás Fernández Löbbe
>
> <tomasflobbe@gmail.com> wrote:
>
> >
>
> > Sometimes Jenkins may take hours to take your commit, may fail in the
> middle of your night, may not fail consistently, etc. That's why I don't
> think giving specific timeframes makes sense, but yes, as soon as you
> notice it's failing, it's either fix immediately or revert IMO.
>
> >
>
> > On Fri, Sep 18, 2020 at 12:03 PM Jason Gerlowski <gerlowskija@gmail.com>
> wrote:
>
> >>
>
> >> > If it’s inadvertently added, we either fix it within an hour or so or
> revert the offending commit
>
> >>
>
> >> > I don't want to set specific time frames,
>
> >>
>
> >> To play Devil's Advocate here: why wait even an hour to revert a 100%
>
> >> test failure? Reverts are usually trivial to do, unblock others
>
> >> immediately, and don't interfere with the fix process at all.
>
> >> Remembering the times I've broken the build myself, reverts even seem
>
> >> preferable from that position - reverting up front takes all the
>
> >> time-pressure off of getting out a fix. Why work under the gun when
>
> >> you don't have to?
>
> >>
>
> >> On Fri, Sep 18, 2020 at 1:14 PM Tomás Fernández Löbbe
>
> >> <tomasflobbe@gmail.com> wrote:
>
> >> >
>
> >> > I believe these failures are associated to
> https://issues.apache.org/jira/browse/SOLR-14151
>
> >> >
>
> >> > • FAILED: org.apache.solr.pkg.TestPackages.classMethod
>
> >> > • FAILED:
> org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest.testAdd2Fields
>
> >> > • FAILED:
> org.apache.solr.schema.ManagedSchemaRoundRobinCloudTest.testAddFieldsRoundRobin
>
> >> >
>
> >> > > IMO if a temporary instability is to be introduced deliberately, it
> should be published on the list. If it’s inadvertently added, we either fix
> it within an hour or so or revert the offending commit
>
> >> > I don't want to set specific time frames, but sometimes it's
> obviously too much time.
>
> >> >
>
> >> > On Fri, Sep 18, 2020 at 8:48 AM Atri Sharma <atri@apache.org> wrote:
>
> >> >>
>
> >> >> When I said temporary, I meant 3-4 hours. Definitely not more than
> that.
>
> >> >>
>
> >> >> IMO we should just roll back offending commits if they are easily
> identifiable. I agree with you — we all have been guilty of breaking builds
> (mea culpa as well). The bad part here is the longevity of the failures.
>
> >> >>
>
> >> >>
>
> >> >> On Fri, 18 Sep 2020 at 21:05, Erick Erickson <
> erickerickson@gmail.com> wrote:
>
> >> >>>
>
> >> >>> bq. IMO if a temporary instability is to be introduced
> deliberately, it should be published on the list
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> Actually, I disagree. Having anything in the tests that fail 100%
> of the time is just unacceptable since it becomes a barrier for everyone
> else. AFAIK, if the problem can be identified to a particular push, I have
> no problems with that push being unilaterally rolled back.
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> The exception for me is when the problem is addressed immediately,
> I’ve certainly been the source of that kind of problem, as have others.
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> What I take great exception to is the fact that some of these tests
> have been failing 100% of the time for the last seven days! If it’s the
> case that the full test suite was never run before the push that’s another
> discussion. Yeah, it takes a long time but…
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> Erick
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> > On Sep 18, 2020, at 11:28 AM, Atri Sharma <atri@apache.org>
> wrote:
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> > IMO if a temporary instability is to be introduced deliberately,
> it should be published on the list. If it’s inadvertently added, we either
> fix it within an hour or so or revert the offending commit.
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> > On Fri, 18 Sep 2020 at 20:26, Erick Erickson <
> erickerickson@gmail.com> wrote:
>
> >> >>>
>
> >> >>> > http://fucit.org/solr-jenkins-reports/failure-report.html
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> > HdfsAutoAddReplicasTest failing 100% of the time.
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> > TestPackages.classMethod failing 100% of the time
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> > 3-4 AutoAddReplicas tests failing 98% of the time.
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> > Is anyone looking at these? I realize the code base is changing a
> lot, and some temporary instability is to be expected. What I’d like is for
> some indication that people are actively addressing these.
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> > Erick
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> >
> ---------------------------------------------------------------------
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> > For additional commands, e-mail: dev-help@lucene.apache.org
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> > --
>
> >> >>>
>
> >> >>> > Regards,
>
> >> >>>
>
> >> >>> >
>
> >> >>>
>
> >> >>> > Atri
>
> >> >>>
>
> >> >>> > Apache Concerted
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>>
> ---------------------------------------------------------------------
>
> >> >>>
>
> >> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>
> >> >>>
>
> >> >>> For additional commands, e-mail: dev-help@lucene.apache.org
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >> --
>
> >> >> Regards,
>
> >> >>
>
> >> >> Atri
>
> >> >> Apache Concerted
>
> >>
>
> >> ---------------------------------------------------------------------
>
> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>
> >> For additional commands, e-mail: dev-help@lucene.apache.org
>
> >>
>
>
>
> ---------------------------------------------------------------------
>
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
>
>
Re: Our test failure rate is unacceptable. [ In reply to ]
> I don't think it is along the Apache way to revert somebody's commit without an explicit permission to do so
Interesting, I made the Devil's Advocate argument above with the
Apache Way specifically in mind.

"Community over Code" comes up most often in terms of navigating
interpersonal conflict and fostering contributors; that's valid and
important. But broken builds cause concrete "Community harm" as well.
100%-fails slow down every developer working on Solr for whatever
length of time the project is in that state. Established committers,
first-PR contributors, Github forks, everyone. Leaving 100%-fails out
there while waiting for a committer to respond or fix an issue
prolongs that period: slowing down development and turning off new
potential contributors all the while. So I think there's a concrete
Apache Way argument for reverting early.

Obviously the revert has to be done diplomatically or it risks
alienating committers and undermining the other Apache Way benefits.
But that's a question of execution not of approach. There are
tactful, inoffensive ways to roll back a change without implying
negligence, impugning skill-sets, etc. It's also not a concern
that's specific to reverts - any JIRA comment or dev-list discussion
pointing out issues runs into that.

All that said, this is a Devil's Advocate argument I'm making here. I
have no plans to go around reverting other's commits; I was just
curious where others were on this in case it came up again later.

Best,

Jason

On Fri, Sep 18, 2020 at 3:59 PM Tomás Fernández Löbbe
<tomasflobbe@gmail.com> wrote:
>
> I thought we were talking about reverting your own commits, not someone else’s...
>
> On Fri, Sep 18, 2020 at 12:31 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>>
>> I don't think it is along the Apache way to revert somebody's commit
>>
>> without an explicit permission to do so... Not that I would personally
>>
>> mind if somebody did it for me.
>>
>>
>>
>> On Fri, Sep 18, 2020 at 9:06 PM Tomás Fernández Löbbe
>>
>> <tomasflobbe@gmail.com> wrote:
>>
>> >
>>
>> > Sometimes Jenkins may take hours to take your commit, may fail in the middle of your night, may not fail consistently, etc. That's why I don't think giving specific timeframes makes sense, but yes, as soon as you notice it's failing, it's either fix immediately or revert IMO.
>>
>> >
>>
>> > On Fri, Sep 18, 2020 at 12:03 PM Jason Gerlowski <gerlowskija@gmail.com> wrote:
>>
>> >>
>>
>> >> > If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit
>>
>> >>
>>
>> >> > I don't want to set specific time frames,
>>
>> >>
>>
>> >> To play Devil's Advocate here: why wait even an hour to revert a 100%
>>
>> >> test failure? Reverts are usually trivial to do, unblock others
>>
>> >> immediately, and don't interfere with the fix process at all.
>>
>> >> Remembering the times I've broken the build myself, reverts even seem
>>
>> >> preferable from that position - reverting up front takes all the
>>
>> >> time-pressure off of getting out a fix. Why work under the gun when
>>
>> >> you don't have to?
>>
>> >>
>>
>> >> On Fri, Sep 18, 2020 at 1:14 PM Tomás Fernández Löbbe
>>
>> >> <tomasflobbe@gmail.com> wrote:
>>
>> >> >
>>
>> >> > I believe these failures are associated to https://issues.apache.org/jira/browse/SOLR-14151
>>
>> >> >
>>
>> >> > • FAILED: org.apache.solr.pkg.TestPackages.classMethod
>>
>> >> > • FAILED: org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest.testAdd2Fields
>>
>> >> > • FAILED: org.apache.solr.schema.ManagedSchemaRoundRobinCloudTest.testAddFieldsRoundRobin
>>
>> >> >
>>
>> >> > > IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit
>>
>> >> > I don't want to set specific time frames, but sometimes it's obviously too much time.
>>
>> >> >
>>
>> >> > On Fri, Sep 18, 2020 at 8:48 AM Atri Sharma <atri@apache.org> wrote:
>>
>> >> >>
>>
>> >> >> When I said temporary, I meant 3-4 hours. Definitely not more than that.
>>
>> >> >>
>>
>> >> >> IMO we should just roll back offending commits if they are easily identifiable. I agree with you — we all have been guilty of breaking builds (mea culpa as well). The bad part here is the longevity of the failures.
>>
>> >> >>
>>
>> >> >>
>>
>> >> >> On Fri, 18 Sep 2020 at 21:05, Erick Erickson <erickerickson@gmail.com> wrote:
>>
>> >> >>>
>>
>> >> >>> bq. IMO if a temporary instability is to be introduced deliberately, it should be published on the list
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>> Actually, I disagree. Having anything in the tests that fail 100% of the time is just unacceptable since it becomes a barrier for everyone else. AFAIK, if the problem can be identified to a particular push, I have no problems with that push being unilaterally rolled back.
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>> The exception for me is when the problem is addressed immediately, I’ve certainly been the source of that kind of problem, as have others.
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>> What I take great exception to is the fact that some of these tests have been failing 100% of the time for the last seven days! If it’s the case that the full test suite was never run before the push that’s another discussion. Yeah, it takes a long time but…
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>> Erick
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>> > On Sep 18, 2020, at 11:28 AM, Atri Sharma <atri@apache.org> wrote:
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> > IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit.
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> > On Fri, 18 Sep 2020 at 20:26, Erick Erickson <erickerickson@gmail.com> wrote:
>>
>> >> >>>
>>
>> >> >>> > http://fucit.org/solr-jenkins-reports/failure-report.html
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> > HdfsAutoAddReplicasTest failing 100% of the time.
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> > TestPackages.classMethod failing 100% of the time
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> > 3-4 AutoAddReplicas tests failing 98% of the time.
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> > Is anyone looking at these? I realize the code base is changing a lot, and some temporary instability is to be expected. What I’d like is for some indication that people are actively addressing these.
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> > Erick
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> > ---------------------------------------------------------------------
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> > --
>>
>> >> >>>
>>
>> >> >>> > Regards,
>>
>> >> >>>
>>
>> >> >>> >
>>
>> >> >>>
>>
>> >> >>> > Atri
>>
>> >> >>>
>>
>> >> >>> > Apache Concerted
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>> ---------------------------------------------------------------------
>>
>> >> >>>
>>
>> >> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>
>> >> >>>
>>
>> >> >>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >>>
>>
>> >> >> --
>>
>> >> >> Regards,
>>
>> >> >>
>>
>> >> >> Atri
>>
>> >> >> Apache Concerted
>>
>> >>
>>
>> >> ---------------------------------------------------------------------
>>
>> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>
>> >> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>> >>
>>
>>
>>
>> ---------------------------------------------------------------------
>>
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Our test failure rate is unacceptable. [ In reply to ]
I shall revert the changes and work on a solution

On Sat, Sep 19, 2020, 6:54 AM Jason Gerlowski <gerlowskija@gmail.com> wrote:

> > I don't think it is along the Apache way to revert somebody's commit
> without an explicit permission to do so
> Interesting, I made the Devil's Advocate argument above with the
> Apache Way specifically in mind.
>
> "Community over Code" comes up most often in terms of navigating
> interpersonal conflict and fostering contributors; that's valid and
> important. But broken builds cause concrete "Community harm" as well.
> 100%-fails slow down every developer working on Solr for whatever
> length of time the project is in that state. Established committers,
> first-PR contributors, Github forks, everyone. Leaving 100%-fails out
> there while waiting for a committer to respond or fix an issue
> prolongs that period: slowing down development and turning off new
> potential contributors all the while. So I think there's a concrete
> Apache Way argument for reverting early.
>
> Obviously the revert has to be done diplomatically or it risks
> alienating committers and undermining the other Apache Way benefits.
> But that's a question of execution not of approach. There are
> tactful, inoffensive ways to roll back a change without implying
> negligence, impugning skill-sets, etc. It's also not a concern
> that's specific to reverts - any JIRA comment or dev-list discussion
> pointing out issues runs into that.
>
> All that said, this is a Devil's Advocate argument I'm making here. I
> have no plans to go around reverting other's commits; I was just
> curious where others were on this in case it came up again later.
>
> Best,
>
> Jason
>
> On Fri, Sep 18, 2020 at 3:59 PM Tomás Fernández Löbbe
> <tomasflobbe@gmail.com> wrote:
> >
> > I thought we were talking about reverting your own commits, not someone
> else’s...
> >
> > On Fri, Sep 18, 2020 at 12:31 PM Dawid Weiss <dawid.weiss@gmail.com>
> wrote:
> >>
> >> I don't think it is along the Apache way to revert somebody's commit
> >>
> >> without an explicit permission to do so... Not that I would personally
> >>
> >> mind if somebody did it for me.
> >>
> >>
> >>
> >> On Fri, Sep 18, 2020 at 9:06 PM Tomás Fernández Löbbe
> >>
> >> <tomasflobbe@gmail.com> wrote:
> >>
> >> >
> >>
> >> > Sometimes Jenkins may take hours to take your commit, may fail in the
> middle of your night, may not fail consistently, etc. That's why I don't
> think giving specific timeframes makes sense, but yes, as soon as you
> notice it's failing, it's either fix immediately or revert IMO.
> >>
> >> >
> >>
> >> > On Fri, Sep 18, 2020 at 12:03 PM Jason Gerlowski <
> gerlowskija@gmail.com> wrote:
> >>
> >> >>
> >>
> >> >> > If it’s inadvertently added, we either fix it within an hour or so
> or revert the offending commit
> >>
> >> >>
> >>
> >> >> > I don't want to set specific time frames,
> >>
> >> >>
> >>
> >> >> To play Devil's Advocate here: why wait even an hour to revert a 100%
> >>
> >> >> test failure? Reverts are usually trivial to do, unblock others
> >>
> >> >> immediately, and don't interfere with the fix process at all.
> >>
> >> >> Remembering the times I've broken the build myself, reverts even seem
> >>
> >> >> preferable from that position - reverting up front takes all the
> >>
> >> >> time-pressure off of getting out a fix. Why work under the gun when
> >>
> >> >> you don't have to?
> >>
> >> >>
> >>
> >> >> On Fri, Sep 18, 2020 at 1:14 PM Tomás Fernández Löbbe
> >>
> >> >> <tomasflobbe@gmail.com> wrote:
> >>
> >> >> >
> >>
> >> >> > I believe these failures are associated to
> https://issues.apache.org/jira/browse/SOLR-14151
> >>
> >> >> >
> >>
> >> >> > • FAILED: org.apache.solr.pkg.TestPackages.classMethod
> >>
> >> >> > • FAILED:
> org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest.testAdd2Fields
> >>
> >> >> > • FAILED:
> org.apache.solr.schema.ManagedSchemaRoundRobinCloudTest.testAddFieldsRoundRobin
> >>
> >> >> >
> >>
> >> >> > > IMO if a temporary instability is to be introduced deliberately,
> it should be published on the list. If it’s inadvertently added, we either
> fix it within an hour or so or revert the offending commit
> >>
> >> >> > I don't want to set specific time frames, but sometimes it's
> obviously too much time.
> >>
> >> >> >
> >>
> >> >> > On Fri, Sep 18, 2020 at 8:48 AM Atri Sharma <atri@apache.org>
> wrote:
> >>
> >> >> >>
> >>
> >> >> >> When I said temporary, I meant 3-4 hours. Definitely not more
> than that.
> >>
> >> >> >>
> >>
> >> >> >> IMO we should just roll back offending commits if they are easily
> identifiable. I agree with you — we all have been guilty of breaking builds
> (mea culpa as well). The bad part here is the longevity of the failures.
> >>
> >> >> >>
> >>
> >> >> >>
> >>
> >> >> >> On Fri, 18 Sep 2020 at 21:05, Erick Erickson <
> erickerickson@gmail.com> wrote:
> >>
> >> >> >>>
> >>
> >> >> >>> bq. IMO if a temporary instability is to be introduced
> deliberately, it should be published on the list
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>> Actually, I disagree. Having anything in the tests that fail
> 100% of the time is just unacceptable since it becomes a barrier for
> everyone else. AFAIK, if the problem can be identified to a particular
> push, I have no problems with that push being unilaterally rolled back.
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>> The exception for me is when the problem is addressed
> immediately, I’ve certainly been the source of that kind of problem, as
> have others.
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>> What I take great exception to is the fact that some of these
> tests have been failing 100% of the time for the last seven days! If it’s
> the case that the full test suite was never run before the push that’s
> another discussion. Yeah, it takes a long time but…
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>> Erick
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>> > On Sep 18, 2020, at 11:28 AM, Atri Sharma <atri@apache.org>
> wrote:
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > IMO if a temporary instability is to be introduced
> deliberately, it should be published on the list. If it’s inadvertently
> added, we either fix it within an hour or so or revert the offending commit.
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > On Fri, 18 Sep 2020 at 20:26, Erick Erickson <
> erickerickson@gmail.com> wrote:
> >>
> >> >> >>>
> >>
> >> >> >>> > http://fucit.org/solr-jenkins-reports/failure-report.html
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > HdfsAutoAddReplicasTest failing 100% of the time.
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > TestPackages.classMethod failing 100% of the time
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > 3-4 AutoAddReplicas tests failing 98% of the time.
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > Is anyone looking at these? I realize the code base is
> changing a lot, and some temporary instability is to be expected. What I’d
> like is for some indication that people are actively addressing these.
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > Erick
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> ---------------------------------------------------------------------
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > For additional commands, e-mail: dev-help@lucene.apache.org
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > --
> >>
> >> >> >>>
> >>
> >> >> >>> > Regards,
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > Atri
> >>
> >> >> >>>
> >>
> >> >> >>> > Apache Concerted
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> ---------------------------------------------------------------------
> >>
> >> >> >>>
> >>
> >> >> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>
> >> >> >>>
> >>
> >> >> >>> For additional commands, e-mail: dev-help@lucene.apache.org
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >> --
> >>
> >> >> >> Regards,
> >>
> >> >> >>
> >>
> >> >> >> Atri
> >>
> >> >> >> Apache Concerted
> >>
> >> >>
> >>
> >> >> ---------------------------------------------------------------------
> >>
> >> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>
> >> >> For additional commands, e-mail: dev-help@lucene.apache.org
> >>
> >> >>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >>
> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>
> >> For additional commands, e-mail: dev-help@lucene.apache.org
> >>
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: Our test failure rate is unacceptable. [ In reply to ]
I’ll try and help with testing this feature more, as I have a specific package that needs this feature.

We are somewhat stuck in a weird time, as we’re doing some great stuff, like the packages, to make core Solr foot print smaller, which means we need to add more complexity to core Solr, yet at the same time, we don’t have the (hopefully!) cleaner structure that is being worked on in the reference_impl_dev to properly support the complexity.

Don’t get discouraged!

> On Sep 18, 2020, at 11:21 PM, Noble Paul <noble.paul@gmail.com <mailto:noble.paul@gmail.com>> wrote:
>
> I shall revert the changes and work on a solution
>
> On Sat, Sep 19, 2020, 6:54 AM Jason Gerlowski <gerlowskija@gmail.com <mailto:gerlowskija@gmail.com>> wrote:
> > I don't think it is along the Apache way to revert somebody's commit without an explicit permission to do so
> Interesting, I made the Devil's Advocate argument above with the
> Apache Way specifically in mind.
>
> "Community over Code" comes up most often in terms of navigating
> interpersonal conflict and fostering contributors; that's valid and
> important. But broken builds cause concrete "Community harm" as well.
> 100%-fails slow down every developer working on Solr for whatever
> length of time the project is in that state. Established committers,
> first-PR contributors, Github forks, everyone. Leaving 100%-fails out
> there while waiting for a committer to respond or fix an issue
> prolongs that period: slowing down development and turning off new
> potential contributors all the while. So I think there's a concrete
> Apache Way argument for reverting early.
>
> Obviously the revert has to be done diplomatically or it risks
> alienating committers and undermining the other Apache Way benefits.
> But that's a question of execution not of approach. There are
> tactful, inoffensive ways to roll back a change without implying
> negligence, impugning skill-sets, etc. It's also not a concern
> that's specific to reverts - any JIRA comment or dev-list discussion
> pointing out issues runs into that.
>
> All that said, this is a Devil's Advocate argument I'm making here. I
> have no plans to go around reverting other's commits; I was just
> curious where others were on this in case it came up again later.
>
> Best,
>
> Jason
>
> On Fri, Sep 18, 2020 at 3:59 PM Tomás Fernández Löbbe
> <tomasflobbe@gmail.com <mailto:tomasflobbe@gmail.com>> wrote:
> >
> > I thought we were talking about reverting your own commits, not someone else’s...
> >
> > On Fri, Sep 18, 2020 at 12:31 PM Dawid Weiss <dawid.weiss@gmail.com <mailto:dawid.weiss@gmail.com>> wrote:
> >>
> >> I don't think it is along the Apache way to revert somebody's commit
> >>
> >> without an explicit permission to do so... Not that I would personally
> >>
> >> mind if somebody did it for me.
> >>
> >>
> >>
> >> On Fri, Sep 18, 2020 at 9:06 PM Tomás Fernández Löbbe
> >>
> >> <tomasflobbe@gmail.com <mailto:tomasflobbe@gmail.com>> wrote:
> >>
> >> >
> >>
> >> > Sometimes Jenkins may take hours to take your commit, may fail in the middle of your night, may not fail consistently, etc. That's why I don't think giving specific timeframes makes sense, but yes, as soon as you notice it's failing, it's either fix immediately or revert IMO.
> >>
> >> >
> >>
> >> > On Fri, Sep 18, 2020 at 12:03 PM Jason Gerlowski <gerlowskija@gmail.com <mailto:gerlowskija@gmail.com>> wrote:
> >>
> >> >>
> >>
> >> >> > If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit
> >>
> >> >>
> >>
> >> >> > I don't want to set specific time frames,
> >>
> >> >>
> >>
> >> >> To play Devil's Advocate here: why wait even an hour to revert a 100%
> >>
> >> >> test failure? Reverts are usually trivial to do, unblock others
> >>
> >> >> immediately, and don't interfere with the fix process at all.
> >>
> >> >> Remembering the times I've broken the build myself, reverts even seem
> >>
> >> >> preferable from that position - reverting up front takes all the
> >>
> >> >> time-pressure off of getting out a fix. Why work under the gun when
> >>
> >> >> you don't have to?
> >>
> >> >>
> >>
> >> >> On Fri, Sep 18, 2020 at 1:14 PM Tomás Fernández Löbbe
> >>
> >> >> <tomasflobbe@gmail.com <mailto:tomasflobbe@gmail.com>> wrote:
> >>
> >> >> >
> >>
> >> >> > I believe these failures are associated to https://issues.apache.org/jira/browse/SOLR-14151 <https://issues.apache.org/jira/browse/SOLR-14151>
> >>
> >> >> >
> >>
> >> >> > • FAILED: org.apache.solr.pkg.TestPackages.classMethod
> >>
> >> >> > • FAILED: org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest.testAdd2Fields
> >>
> >> >> > • FAILED: org.apache.solr.schema.ManagedSchemaRoundRobinCloudTest.testAddFieldsRoundRobin
> >>
> >> >> >
> >>
> >> >> > > IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit
> >>
> >> >> > I don't want to set specific time frames, but sometimes it's obviously too much time.
> >>
> >> >> >
> >>
> >> >> > On Fri, Sep 18, 2020 at 8:48 AM Atri Sharma <atri@apache.org <mailto:atri@apache.org>> wrote:
> >>
> >> >> >>
> >>
> >> >> >> When I said temporary, I meant 3-4 hours. Definitely not more than that.
> >>
> >> >> >>
> >>
> >> >> >> IMO we should just roll back offending commits if they are easily identifiable. I agree with you — we all have been guilty of breaking builds (mea culpa as well). The bad part here is the longevity of the failures.
> >>
> >> >> >>
> >>
> >> >> >>
> >>
> >> >> >> On Fri, 18 Sep 2020 at 21:05, Erick Erickson <erickerickson@gmail.com <mailto:erickerickson@gmail.com>> wrote:
> >>
> >> >> >>>
> >>
> >> >> >>> bq. IMO if a temporary instability is to be introduced deliberately, it should be published on the list
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>> Actually, I disagree. Having anything in the tests that fail 100% of the time is just unacceptable since it becomes a barrier for everyone else. AFAIK, if the problem can be identified to a particular push, I have no problems with that push being unilaterally rolled back.
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>> The exception for me is when the problem is addressed immediately, I’ve certainly been the source of that kind of problem, as have others.
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>> What I take great exception to is the fact that some of these tests have been failing 100% of the time for the last seven days! If it’s the case that the full test suite was never run before the push that’s another discussion. Yeah, it takes a long time but…
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>> Erick
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>> > On Sep 18, 2020, at 11:28 AM, Atri Sharma <atri@apache.org <mailto:atri@apache.org>> wrote:
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit.
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > On Fri, 18 Sep 2020 at 20:26, Erick Erickson <erickerickson@gmail.com <mailto:erickerickson@gmail.com>> wrote:
> >>
> >> >> >>>
> >>
> >> >> >>> > http://fucit.org/solr-jenkins-reports/failure-report.html <http://fucit.org/solr-jenkins-reports/failure-report.html>
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > HdfsAutoAddReplicasTest failing 100% of the time.
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > TestPackages.classMethod failing 100% of the time
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > 3-4 AutoAddReplicas tests failing 98% of the time.
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > Is anyone looking at these? I realize the code base is changing a lot, and some temporary instability is to be expected. What I’d like is for some indication that people are actively addressing these.
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > Erick
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > ---------------------------------------------------------------------
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > --
> >>
> >> >> >>>
> >>
> >> >> >>> > Regards,
> >>
> >> >> >>>
> >>
> >> >> >>> >
> >>
> >> >> >>>
> >>
> >> >> >>> > Atri
> >>
> >> >> >>>
> >>
> >> >> >>> > Apache Concerted
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>> ---------------------------------------------------------------------
> >>
> >> >> >>>
> >>
> >> >> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
> >>
> >> >> >>>
> >>
> >> >> >>> For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >>>
> >>
> >> >> >> --
> >>
> >> >> >> Regards,
> >>
> >> >> >>
> >>
> >> >> >> Atri
> >>
> >> >> >> Apache Concerted
> >>
> >> >>
> >>
> >> >> ---------------------------------------------------------------------
> >>
> >> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
> >>
> >> >> For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>
> >>
> >> >>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >>
> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
> >>
> >> For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>
> >>
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
> For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>
>

_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
Re: Our test failure rate is unacceptable. [ In reply to ]
This seems to be a reproducing seed, at least 2/2

ant test -Dtestcase=TestPackages -Dtests.seed=C29471044D369FD3 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=et-EE -Dtests.timezone=Europe/Mariehamn -Dtests.asserts=true -Dtests.file.encoding=UTF-8

> On Sep 19, 2020, at 6:40 AM, Eric Pugh <epugh@opensourceconnections.com> wrote:
>
> I’ll try and help with testing this feature more, as I have a specific package that needs this feature.
>
> We are somewhat stuck in a weird time, as we’re doing some great stuff, like the packages, to make core Solr foot print smaller, which means we need to add more complexity to core Solr, yet at the same time, we don’t have the (hopefully!) cleaner structure that is being worked on in the reference_impl_dev to properly support the complexity.
>
> Don’t get discouraged!
>
>> On Sep 18, 2020, at 11:21 PM, Noble Paul <noble.paul@gmail.com> wrote:
>>
>> I shall revert the changes and work on a solution
>>
>> On Sat, Sep 19, 2020, 6:54 AM Jason Gerlowski <gerlowskija@gmail.com> wrote:
>> > I don't think it is along the Apache way to revert somebody's commit without an explicit permission to do so
>> Interesting, I made the Devil's Advocate argument above with the
>> Apache Way specifically in mind.
>>
>> "Community over Code" comes up most often in terms of navigating
>> interpersonal conflict and fostering contributors; that's valid and
>> important. But broken builds cause concrete "Community harm" as well.
>> 100%-fails slow down every developer working on Solr for whatever
>> length of time the project is in that state. Established committers,
>> first-PR contributors, Github forks, everyone. Leaving 100%-fails out
>> there while waiting for a committer to respond or fix an issue
>> prolongs that period: slowing down development and turning off new
>> potential contributors all the while. So I think there's a concrete
>> Apache Way argument for reverting early.
>>
>> Obviously the revert has to be done diplomatically or it risks
>> alienating committers and undermining the other Apache Way benefits.
>> But that's a question of execution not of approach. There are
>> tactful, inoffensive ways to roll back a change without implying
>> negligence, impugning skill-sets, etc. It's also not a concern
>> that's specific to reverts - any JIRA comment or dev-list discussion
>> pointing out issues runs into that.
>>
>> All that said, this is a Devil's Advocate argument I'm making here. I
>> have no plans to go around reverting other's commits; I was just
>> curious where others were on this in case it came up again later.
>>
>> Best,
>>
>> Jason
>>
>> On Fri, Sep 18, 2020 at 3:59 PM Tomás Fernández Löbbe
>> <tomasflobbe@gmail.com> wrote:
>> >
>> > I thought we were talking about reverting your own commits, not someone else’s...
>> >
>> > On Fri, Sep 18, 2020 at 12:31 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
>> >>
>> >> I don't think it is along the Apache way to revert somebody's commit
>> >>
>> >> without an explicit permission to do so... Not that I would personally
>> >>
>> >> mind if somebody did it for me.
>> >>
>> >>
>> >>
>> >> On Fri, Sep 18, 2020 at 9:06 PM Tomás Fernández Löbbe
>> >>
>> >> <tomasflobbe@gmail.com> wrote:
>> >>
>> >> >
>> >>
>> >> > Sometimes Jenkins may take hours to take your commit, may fail in the middle of your night, may not fail consistently, etc. That's why I don't think giving specific timeframes makes sense, but yes, as soon as you notice it's failing, it's either fix immediately or revert IMO.
>> >>
>> >> >
>> >>
>> >> > On Fri, Sep 18, 2020 at 12:03 PM Jason Gerlowski <gerlowskija@gmail.com> wrote:
>> >>
>> >> >>
>> >>
>> >> >> > If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit
>> >>
>> >> >>
>> >>
>> >> >> > I don't want to set specific time frames,
>> >>
>> >> >>
>> >>
>> >> >> To play Devil's Advocate here: why wait even an hour to revert a 100%
>> >>
>> >> >> test failure? Reverts are usually trivial to do, unblock others
>> >>
>> >> >> immediately, and don't interfere with the fix process at all.
>> >>
>> >> >> Remembering the times I've broken the build myself, reverts even seem
>> >>
>> >> >> preferable from that position - reverting up front takes all the
>> >>
>> >> >> time-pressure off of getting out a fix. Why work under the gun when
>> >>
>> >> >> you don't have to?
>> >>
>> >> >>
>> >>
>> >> >> On Fri, Sep 18, 2020 at 1:14 PM Tomás Fernández Löbbe
>> >>
>> >> >> <tomasflobbe@gmail.com> wrote:
>> >>
>> >> >> >
>> >>
>> >> >> > I believe these failures are associated to https://issues.apache.org/jira/browse/SOLR-14151
>> >>
>> >> >> >
>> >>
>> >> >> > • FAILED: org.apache.solr.pkg.TestPackages.classMethod
>> >>
>> >> >> > • FAILED: org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest.testAdd2Fields
>> >>
>> >> >> > • FAILED: org.apache.solr.schema.ManagedSchemaRoundRobinCloudTest.testAddFieldsRoundRobin
>> >>
>> >> >> >
>> >>
>> >> >> > > IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit
>> >>
>> >> >> > I don't want to set specific time frames, but sometimes it's obviously too much time.
>> >>
>> >> >> >
>> >>
>> >> >> > On Fri, Sep 18, 2020 at 8:48 AM Atri Sharma <atri@apache.org> wrote:
>> >>
>> >> >> >>
>> >>
>> >> >> >> When I said temporary, I meant 3-4 hours. Definitely not more than that.
>> >>
>> >> >> >>
>> >>
>> >> >> >> IMO we should just roll back offending commits if they are easily identifiable. I agree with you — we all have been guilty of breaking builds (mea culpa as well). The bad part here is the longevity of the failures.
>> >>
>> >> >> >>
>> >>
>> >> >> >>
>> >>
>> >> >> >> On Fri, 18 Sep 2020 at 21:05, Erick Erickson <erickerickson@gmail.com> wrote:
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> bq. IMO if a temporary instability is to be introduced deliberately, it should be published on the list
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> Actually, I disagree. Having anything in the tests that fail 100% of the time is just unacceptable since it becomes a barrier for everyone else. AFAIK, if the problem can be identified to a particular push, I have no problems with that push being unilaterally rolled back.
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> The exception for me is when the problem is addressed immediately, I’ve certainly been the source of that kind of problem, as have others.
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> What I take great exception to is the fact that some of these tests have been failing 100% of the time for the last seven days! If it’s the case that the full test suite was never run before the push that’s another discussion. Yeah, it takes a long time but…
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> Erick
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > On Sep 18, 2020, at 11:28 AM, Atri Sharma <atri@apache.org> wrote:
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit.
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > On Fri, 18 Sep 2020 at 20:26, Erick Erickson <erickerickson@gmail.com> wrote:
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > http://fucit.org/solr-jenkins-reports/failure-report.html
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > HdfsAutoAddReplicasTest failing 100% of the time.
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > TestPackages.classMethod failing 100% of the time
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > 3-4 AutoAddReplicas tests failing 98% of the time.
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > Is anyone looking at these? I realize the code base is changing a lot, and some temporary instability is to be expected. What I’d like is for some indication that people are actively addressing these.
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > Erick
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > ---------------------------------------------------------------------
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > For additional commands, e-mail: dev-help@lucene.apache.org
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > --
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > Regards,
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> >
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > Atri
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> > Apache Concerted
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> ---------------------------------------------------------------------
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >>
>> >> >> >>>
>> >>
>> >> >> >>> For additional commands, e-mail: dev-help@lucene.apache.org
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >>>
>> >>
>> >> >> >> --
>> >>
>> >> >> >> Regards,
>> >>
>> >> >> >>
>> >>
>> >> >> >> Atri
>> >>
>> >> >> >> Apache Concerted
>> >>
>> >> >>
>> >>
>> >> >> ---------------------------------------------------------------------
>> >>
>> >> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >>
>> >> >> For additional commands, e-mail: dev-help@lucene.apache.org
>> >>
>> >> >>
>> >>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >>
>> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >>
>> >> For additional commands, e-mail: dev-help@lucene.apache.org
>> >>
>> >>
>> >>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Our test failure rate is unacceptable. [ In reply to ]
It passes for me all the time Erick

Can you please test with the branch
https://github.com/apache/lucene-solr/tree/jira/solr14879


On Sun, Sep 20, 2020 at 7:14 AM Erick Erickson <erickerickson@gmail.com> wrote:
>
> This seems to be a reproducing seed, at least 2/2
>
> ant test -Dtestcase=TestPackages -Dtests.seed=C29471044D369FD3 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=et-EE -Dtests.timezone=Europe/Mariehamn -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>
> > On Sep 19, 2020, at 6:40 AM, Eric Pugh <epugh@opensourceconnections.com> wrote:
> >
> > I’ll try and help with testing this feature more, as I have a specific package that needs this feature.
> >
> > We are somewhat stuck in a weird time, as we’re doing some great stuff, like the packages, to make core Solr foot print smaller, which means we need to add more complexity to core Solr, yet at the same time, we don’t have the (hopefully!) cleaner structure that is being worked on in the reference_impl_dev to properly support the complexity.
> >
> > Don’t get discouraged!
> >
> >> On Sep 18, 2020, at 11:21 PM, Noble Paul <noble.paul@gmail.com> wrote:
> >>
> >> I shall revert the changes and work on a solution
> >>
> >> On Sat, Sep 19, 2020, 6:54 AM Jason Gerlowski <gerlowskija@gmail.com> wrote:
> >> > I don't think it is along the Apache way to revert somebody's commit without an explicit permission to do so
> >> Interesting, I made the Devil's Advocate argument above with the
> >> Apache Way specifically in mind.
> >>
> >> "Community over Code" comes up most often in terms of navigating
> >> interpersonal conflict and fostering contributors; that's valid and
> >> important. But broken builds cause concrete "Community harm" as well.
> >> 100%-fails slow down every developer working on Solr for whatever
> >> length of time the project is in that state. Established committers,
> >> first-PR contributors, Github forks, everyone. Leaving 100%-fails out
> >> there while waiting for a committer to respond or fix an issue
> >> prolongs that period: slowing down development and turning off new
> >> potential contributors all the while. So I think there's a concrete
> >> Apache Way argument for reverting early.
> >>
> >> Obviously the revert has to be done diplomatically or it risks
> >> alienating committers and undermining the other Apache Way benefits.
> >> But that's a question of execution not of approach. There are
> >> tactful, inoffensive ways to roll back a change without implying
> >> negligence, impugning skill-sets, etc. It's also not a concern
> >> that's specific to reverts - any JIRA comment or dev-list discussion
> >> pointing out issues runs into that.
> >>
> >> All that said, this is a Devil's Advocate argument I'm making here. I
> >> have no plans to go around reverting other's commits; I was just
> >> curious where others were on this in case it came up again later.
> >>
> >> Best,
> >>
> >> Jason
> >>
> >> On Fri, Sep 18, 2020 at 3:59 PM Tomás Fernández Löbbe
> >> <tomasflobbe@gmail.com> wrote:
> >> >
> >> > I thought we were talking about reverting your own commits, not someone else’s...
> >> >
> >> > On Fri, Sep 18, 2020 at 12:31 PM Dawid Weiss <dawid.weiss@gmail.com> wrote:
> >> >>
> >> >> I don't think it is along the Apache way to revert somebody's commit
> >> >>
> >> >> without an explicit permission to do so... Not that I would personally
> >> >>
> >> >> mind if somebody did it for me.
> >> >>
> >> >>
> >> >>
> >> >> On Fri, Sep 18, 2020 at 9:06 PM Tomás Fernández Löbbe
> >> >>
> >> >> <tomasflobbe@gmail.com> wrote:
> >> >>
> >> >> >
> >> >>
> >> >> > Sometimes Jenkins may take hours to take your commit, may fail in the middle of your night, may not fail consistently, etc. That's why I don't think giving specific timeframes makes sense, but yes, as soon as you notice it's failing, it's either fix immediately or revert IMO.
> >> >>
> >> >> >
> >> >>
> >> >> > On Fri, Sep 18, 2020 at 12:03 PM Jason Gerlowski <gerlowskija@gmail.com> wrote:
> >> >>
> >> >> >>
> >> >>
> >> >> >> > If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit
> >> >>
> >> >> >>
> >> >>
> >> >> >> > I don't want to set specific time frames,
> >> >>
> >> >> >>
> >> >>
> >> >> >> To play Devil's Advocate here: why wait even an hour to revert a 100%
> >> >>
> >> >> >> test failure? Reverts are usually trivial to do, unblock others
> >> >>
> >> >> >> immediately, and don't interfere with the fix process at all.
> >> >>
> >> >> >> Remembering the times I've broken the build myself, reverts even seem
> >> >>
> >> >> >> preferable from that position - reverting up front takes all the
> >> >>
> >> >> >> time-pressure off of getting out a fix. Why work under the gun when
> >> >>
> >> >> >> you don't have to?
> >> >>
> >> >> >>
> >> >>
> >> >> >> On Fri, Sep 18, 2020 at 1:14 PM Tomás Fernández Löbbe
> >> >>
> >> >> >> <tomasflobbe@gmail.com> wrote:
> >> >>
> >> >> >> >
> >> >>
> >> >> >> > I believe these failures are associated to https://issues.apache.org/jira/browse/SOLR-14151
> >> >>
> >> >> >> >
> >> >>
> >> >> >> > • FAILED: org.apache.solr.pkg.TestPackages.classMethod
> >> >>
> >> >> >> > • FAILED: org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest.testAdd2Fields
> >> >>
> >> >> >> > • FAILED: org.apache.solr.schema.ManagedSchemaRoundRobinCloudTest.testAddFieldsRoundRobin
> >> >>
> >> >> >> >
> >> >>
> >> >> >> > > IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit
> >> >>
> >> >> >> > I don't want to set specific time frames, but sometimes it's obviously too much time.
> >> >>
> >> >> >> >
> >> >>
> >> >> >> > On Fri, Sep 18, 2020 at 8:48 AM Atri Sharma <atri@apache.org> wrote:
> >> >>
> >> >> >> >>
> >> >>
> >> >> >> >> When I said temporary, I meant 3-4 hours. Definitely not more than that.
> >> >>
> >> >> >> >>
> >> >>
> >> >> >> >> IMO we should just roll back offending commits if they are easily identifiable. I agree with you — we all have been guilty of breaking builds (mea culpa as well). The bad part here is the longevity of the failures.
> >> >>
> >> >> >> >>
> >> >>
> >> >> >> >>
> >> >>
> >> >> >> >> On Fri, 18 Sep 2020 at 21:05, Erick Erickson <erickerickson@gmail.com> wrote:
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> bq. IMO if a temporary instability is to be introduced deliberately, it should be published on the list
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> Actually, I disagree. Having anything in the tests that fail 100% of the time is just unacceptable since it becomes a barrier for everyone else. AFAIK, if the problem can be identified to a particular push, I have no problems with that push being unilaterally rolled back.
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> The exception for me is when the problem is addressed immediately, I’ve certainly been the source of that kind of problem, as have others.
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> What I take great exception to is the fact that some of these tests have been failing 100% of the time for the last seven days! If it’s the case that the full test suite was never run before the push that’s another discussion. Yeah, it takes a long time but…
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> Erick
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > On Sep 18, 2020, at 11:28 AM, Atri Sharma <atri@apache.org> wrote:
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > IMO if a temporary instability is to be introduced deliberately, it should be published on the list. If it’s inadvertently added, we either fix it within an hour or so or revert the offending commit.
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > On Fri, 18 Sep 2020 at 20:26, Erick Erickson <erickerickson@gmail.com> wrote:
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > http://fucit.org/solr-jenkins-reports/failure-report.html
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > HdfsAutoAddReplicasTest failing 100% of the time.
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > TestPackages.classMethod failing 100% of the time
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > 3-4 AutoAddReplicas tests failing 98% of the time.
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > Is anyone looking at these? I realize the code base is changing a lot, and some temporary instability is to be expected. What I’d like is for some indication that people are actively addressing these.
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > Erick
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > ---------------------------------------------------------------------
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > For additional commands, e-mail: dev-help@lucene.apache.org
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > --
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > Regards,
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> >
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > Atri
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> > Apache Concerted
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> ---------------------------------------------------------------------
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>> For additional commands, e-mail: dev-help@lucene.apache.org
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >>>
> >> >>
> >> >> >> >> --
> >> >>
> >> >> >> >> Regards,
> >> >>
> >> >> >> >>
> >> >>
> >> >> >> >> Atri
> >> >>
> >> >> >> >> Apache Concerted
> >> >>
> >> >> >>
> >> >>
> >> >> >> ---------------------------------------------------------------------
> >> >>
> >> >> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> >>
> >> >> >> For additional commands, e-mail: dev-help@lucene.apache.org
> >> >>
> >> >> >>
> >> >>
> >> >>
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >>
> >> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> >>
> >> >> For additional commands, e-mail: dev-help@lucene.apache.org
> >> >>
> >> >>
> >> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: dev-help@lucene.apache.org
> >>
> >
> > _______________________
> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> > This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>


--
-----------------------------------------------------
Noble Paul

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Our test failure rate is unacceptable. [ In reply to ]
Can’t seem to get that to fail on master. Note the reproducible bit was 8x.

> On Sep 19, 2020, at 5:13 PM, Erick Erickson <erickerickson@gmail.com> wrote:
>
> ant test -Dtestcase=TestPackages -Dtests.seed=C29471044D369FD3 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=et-EE -Dtests.timezone=Europe/Mariehamn -Dtests.asserts=true -Dtests.file.encoding=UTF-8


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Our test failure rate is unacceptable. [ In reply to ]
I tried in 8x of course. The ant command is gone from master anyway

On Mon, Sep 21, 2020 at 1:32 AM Erick Erickson <erickerickson@gmail.com> wrote:
>
> Can’t seem to get that to fail on master. Note the reproducible bit was 8x.
>
> > On Sep 19, 2020, at 5:13 PM, Erick Erickson <erickerickson@gmail.com> wrote:
> >
> > ant test -Dtestcase=TestPackages -Dtests.seed=C29471044D369FD3 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=et-EE -Dtests.timezone=Europe/Mariehamn -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>


--
-----------------------------------------------------
Noble Paul

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org