Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Lucene>
  3. Java-Dev
BadApple report, but please read the first bit
erickerickson at gmail
Aug 3, 2020, 4:26 AM
Post #1 of 6 (318 views)
Permalink
There are several tests that are causing a lot of noise:

SharedFSAutoReplicaFailoverTest is failing 90%+ of the time.
TestBulkSchemaConcurrent 31%
StressHdfsTest 16%
SchemaApiFailureTest 13.88%

I encourage people to look at: http://fucit.org/solr-jenkins-reports/failure-report.html and see if anything looks like it is affected by recent work. TestBulkSchemaConcurrent has been failing off and on for a long time, but the failure rate picked up dramatically in the last couple of weeks. Ditto SchemaApiFailureTest.

Do we even care about Hdfs? Are we deprecating it or not?

Holding relatively steady otherwise:

Raw fail count by week totals, most recent week first (corresponds to bits):
Week: 0 had 82 failures
Week: 1 had 94 failures
Week: 2 had 502 failures
Week: 3 had 19 failures


********Failures in Hoss' reports for the last 4 rollups.

There were 562 unannotated tests that failed in Hoss' rollups. Ordered by the date I downloaded the rollup file, newest->oldest. See above for the dates the files were collected
These tests were NOT BadApple'd or AwaitsFix'd

Failures in the last 4 reports..
Report Pct runs fails test
0123 0.3 1271 8 RollingRestartTest.test
0123 93.3 41 36 SharedFSAutoReplicaFailoverTest.test
0123 3.5 627 16 TestCircuitBreaker.testBuildingMemoryPressure
0123 1.0 627 8 TestCircuitBreaker.testResponseWithCBTiming
0123 5.8 1483 79 TestContainerPlugin.testApiFromPackage
0123 2.3 1335 23 TestDistributedGrouping.test
********************************************


Full report:

Attached Files:

Re: BadApple report, but please read the first bit [ In reply to ]
dsmiley at apache
Aug 11, 2020, 11:06 PM
Post #2 of 6 (314 views)
Permalink
Do we have any long term (aka "longitudinal") pass/fail rates for tests?

SharedFSAutoReplicaFailoverTest in particular is kinda-sorta tied to HDFS,
and that's going away to a plug-in for 9.0. The shared file system notion
isn't well supported in SolrCloud, I think.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Mon, Aug 3, 2020 at 7:26 AM Erick Erickson <erickerickson@gmail.com>
wrote:

> There are several tests that are causing a lot of noise:
>
> SharedFSAutoReplicaFailoverTest is failing 90%+ of the time.
> TestBulkSchemaConcurrent 31%
> StressHdfsTest 16%
> SchemaApiFailureTest 13.88%
>
> I encourage people to look at:
> http://fucit.org/solr-jenkins-reports/failure-report.html and see if
> anything looks like it is affected by recent work. TestBulkSchemaConcurrent
> has been failing off and on for a long time, but the failure rate picked up
> dramatically in the last couple of weeks. Ditto SchemaApiFailureTest.
>
> Do we even care about Hdfs? Are we deprecating it or not?
>
> Holding relatively steady otherwise:
>
> Raw fail count by week totals, most recent week first (corresponds to
> bits):
> Week: 0 had 82 failures
> Week: 1 had 94 failures
> Week: 2 had 502 failures
> Week: 3 had 19 failures
>
>
> ********Failures in Hoss' reports for the last 4 rollups.
>
> There were 562 unannotated tests that failed in Hoss' rollups. Ordered by
> the date I downloaded the rollup file, newest->oldest. See above for the
> dates the files were collected
> These tests were NOT BadApple'd or AwaitsFix'd
>
> Failures in the last 4 reports..
> Report Pct runs fails test
> 0123 0.3 1271 8 RollingRestartTest.test
> 0123 93.3 41 36 SharedFSAutoReplicaFailoverTest.test
> 0123 3.5 627 16
> TestCircuitBreaker.testBuildingMemoryPressure
> 0123 1.0 627 8
> TestCircuitBreaker.testResponseWithCBTiming
> 0123 5.8 1483 79 TestContainerPlugin.testApiFromPackage
> 0123 2.3 1335 23 TestDistributedGrouping.test
> ********************************************
>
>
> Full report:
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
Re: BadApple report, but please read the first bit [ In reply to ]
erickerickson at gmail
Aug 12, 2020, 6:03 AM
Post #3 of 6 (314 views)
Permalink
I have the weekly rollups (with a few gaps) going back to about April 2018, but nothing’s been done to try to make them generally available. Each BadApple report has rates for the last 4 weeks in the attached file, just below "Failures over the last 4 weeks, but not every week. Ordered most-recent first:”



> On Aug 12, 2020, at 2:06 AM, David Smiley <dsmiley@apache.org> wrote:
>
> Do we have any long term (aka "longitudinal") pass/fail rates for tests?
>
> SharedFSAutoReplicaFailoverTest in particular is kinda-sorta tied to HDFS, and that's going away to a plug-in for 9.0. The shared file system notion isn't well supported in SolrCloud, I think.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Mon, Aug 3, 2020 at 7:26 AM Erick Erickson <erickerickson@gmail.com> wrote:
> There are several tests that are causing a lot of noise:
>
> SharedFSAutoReplicaFailoverTest is failing 90%+ of the time.
> TestBulkSchemaConcurrent 31%
> StressHdfsTest 16%
> SchemaApiFailureTest 13.88%
>
> I encourage people to look at: http://fucit.org/solr-jenkins-reports/failure-report.html and see if anything looks like it is affected by recent work. TestBulkSchemaConcurrent has been failing off and on for a long time, but the failure rate picked up dramatically in the last couple of weeks. Ditto SchemaApiFailureTest.
>
> Do we even care about Hdfs? Are we deprecating it or not?
>
> Holding relatively steady otherwise:
>
> Raw fail count by week totals, most recent week first (corresponds to bits):
> Week: 0 had 82 failures
> Week: 1 had 94 failures
> Week: 2 had 502 failures
> Week: 3 had 19 failures
>
>
> ********Failures in Hoss' reports for the last 4 rollups.
>
> There were 562 unannotated tests that failed in Hoss' rollups. Ordered by the date I downloaded the rollup file, newest->oldest. See above for the dates the files were collected
> These tests were NOT BadApple'd or AwaitsFix'd
>
> Failures in the last 4 reports..
> Report Pct runs fails test
> 0123 0.3 1271 8 RollingRestartTest.test
> 0123 93.3 41 36 SharedFSAutoReplicaFailoverTest.test
> 0123 3.5 627 16 TestCircuitBreaker.testBuildingMemoryPressure
> 0123 1.0 627 8 TestCircuitBreaker.testResponseWithCBTiming
> 0123 5.8 1483 79 TestContainerPlugin.testApiFromPackage
> 0123 2.3 1335 23 TestDistributedGrouping.test
> ********************************************
>
>
> Full report:
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: BadApple report, but please read the first bit [ In reply to ]
krisden at apache
Aug 12, 2020, 6:07 AM
Post #4 of 6 (314 views)
Permalink
http://fucit.org/solr-jenkins-reports/history-trend-of-recent-failures.html#series/org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test

David for that specific test you asked the failures are recent with as far
as I know no change to HDFS stuff. Starting June/July failing regularly.

Kevin Risden



On Wed, Aug 12, 2020 at 9:03 AM Erick Erickson <erickerickson@gmail.com>
wrote:

> I have the weekly rollups (with a few gaps) going back to about April
> 2018, but nothing’s been done to try to make them generally available. Each
> BadApple report has rates for the last 4 weeks in the attached file, just
> below "Failures over the last 4 weeks, but not every week. Ordered
> most-recent first:”
>
>
>
> > On Aug 12, 2020, at 2:06 AM, David Smiley <dsmiley@apache.org> wrote:
> >
> > Do we have any long term (aka "longitudinal") pass/fail rates for tests?
> >
> > SharedFSAutoReplicaFailoverTest in particular is kinda-sorta tied to
> HDFS, and that's going away to a plug-in for 9.0. The shared file system
> notion isn't well supported in SolrCloud, I think.
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > On Mon, Aug 3, 2020 at 7:26 AM Erick Erickson <erickerickson@gmail.com>
> wrote:
> > There are several tests that are causing a lot of noise:
> >
> > SharedFSAutoReplicaFailoverTest is failing 90%+ of the time.
> > TestBulkSchemaConcurrent 31%
> > StressHdfsTest 16%
> > SchemaApiFailureTest 13.88%
> >
> > I encourage people to look at:
> http://fucit.org/solr-jenkins-reports/failure-report.html and see if
> anything looks like it is affected by recent work. TestBulkSchemaConcurrent
> has been failing off and on for a long time, but the failure rate picked up
> dramatically in the last couple of weeks. Ditto SchemaApiFailureTest.
> >
> > Do we even care about Hdfs? Are we deprecating it or not?
> >
> > Holding relatively steady otherwise:
> >
> > Raw fail count by week totals, most recent week first (corresponds to
> bits):
> > Week: 0 had 82 failures
> > Week: 1 had 94 failures
> > Week: 2 had 502 failures
> > Week: 3 had 19 failures
> >
> >
> > ********Failures in Hoss' reports for the last 4 rollups.
> >
> > There were 562 unannotated tests that failed in Hoss' rollups. Ordered
> by the date I downloaded the rollup file, newest->oldest. See above for the
> dates the files were collected
> > These tests were NOT BadApple'd or AwaitsFix'd
> >
> > Failures in the last 4 reports..
> > Report Pct runs fails test
> > 0123 0.3 1271 8 RollingRestartTest.test
> > 0123 93.3 41 36 SharedFSAutoReplicaFailoverTest.test
> > 0123 3.5 627 16
> TestCircuitBreaker.testBuildingMemoryPressure
> > 0123 1.0 627 8
> TestCircuitBreaker.testResponseWithCBTiming
> > 0123 5.8 1483 79
> TestContainerPlugin.testApiFromPackage
> > 0123 2.3 1335 23 TestDistributedGrouping.test
> > ********************************************
> >
> >
> > Full report:
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: BadApple report, but please read the first bit [ In reply to ]
erickerickson at gmail
Aug 12, 2020, 6:14 AM
Post #5 of 6 (314 views)
Permalink
Didn’t think at first (only one cup of coffee). Here’s the Emails that test appears in, the formatting is poor…

After that is the raw data from Hoss’ rollups that might be easier to ingest.

I have 1.3G of this kind of historical data, I’ve had vague thoughts about putting it someplace accessible to others but haven’t done anything with it.

I suppose, wrapped around this, is the entire question of how much value it’ll have depending on what happens with Mark’s reference impl...

“Suite” fails are things like object tracker failures.

e-mail-2018-03-26.txt:SharedFSAutoReplicaFailoverTest.java
e-mail-2018-04-02.txt:SharedFSAutoReplicaFailoverTest.java
e-mail-2018-04-09.txt:SharedFSAutoReplicaFailoverTest.java
e-mail-2018-04-16.txt:SharedFSAutoReplicaFailoverTest.java
e-mail-2018-04-30.txt:SharedFSAutoReplicaFailoverTest.java
e-mail-2018-05-21.txt:SharedFSAutoReplicaFailoverTest.java
e-mail-2018-06-11.txt: SharedFSAutoReplicaFailoverTest.test
e-mail-2018-06-11.txt: 3 100.0 2 2 SharedFSAutoReplicaFailoverTest(suite)
e-mail-2018-06-11.txt:SharedFSAutoReplicaFailoverTest.test
e-mail-2018-06-18.txt: 0 100.0 2 2 SharedFSAutoReplicaFailoverTest(suite)
e-mail-2018-06-25.txt: 01 74.1 29 22 SharedFSAutoReplicaFailoverTest(suite)
e-mail-2018-06-25.txt: 0 5.9 34 2 SharedFSAutoReplicaFailoverTest.test
e-mail-2018-07-02.txt: 012 74.1 56 42 SharedFSAutoReplicaFailoverTest(suite)
e-mail-2018-07-02.txt: 01 5.1 73 4 SharedFSAutoReplicaFailoverTest.test
e-mail-2018-07-09.txt: 0123 74.1 83 62 SharedFSAutoReplicaFailoverTest(suite)
e-mail-2018-07-09.txt: 012 2.3 117 5 SharedFSAutoReplicaFailoverTest.test
e-mail-2018-07-16.txt: 0123 74.1 108 80 SharedFSAutoReplicaFailoverTest(suite)
e-mail-2018-07-16.txt: 0123 17.6 151 11 SharedFSAutoReplicaFailoverTest.test
e-mail-2018-07-23.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2018-07-23.txt:SharedFSAutoReplicaFailoverTest.test
e-mail-2018-07-30.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2018-07-30.txt:SharedFSAutoReplicaFailoverTest.test
e-mail-2018-08-06.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2018-08-06.txt:SharedFSAutoReplicaFailoverTest.test
e-mail-2018-08-14.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2018-08-14.txt:SharedFSAutoReplicaFailoverTest.test
e-mail-2018-08-20.txt: SharedFSAutoReplicaFailoverTest.test
e-mail-2018-08-20.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2018-08-20.txt:SharedFSAutoReplicaFailoverTest.test
e-mail-2018-08-27.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2018-09-03.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2018-09-10.txt: 0 20.0 5 1 SharedFSAutoReplicaFailoverTest.test
e-mail-2018-09-10.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2018-09-18.txt: 01 33.3 8 2 SharedFSAutoReplicaFailoverTest.test
e-mail-2018-09-18.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2018-10-08.txt: 3 33.3 3 1 SharedFSAutoReplicaFailoverTest.test
e-mail-2018-10-08.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2018-12-24.txt: 0 3 33.3 6 2 SharedFSAutoReplicaFailoverTest.test
e-mail-2018-12-24.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-01-08.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-01-15.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-02-12.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-02-18.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-03-04.txt: 1 33.3 3 1 SharedFSAutoReplicaFailoverTest.test
e-mail-2019-03-04.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-03-11.txt: 2 33.3 3 1 SharedFSAutoReplicaFailoverTest.test
e-mail-2019-03-11.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-03-18.txt: 3 33.3 3 1 SharedFSAutoReplicaFailoverTest.test
e-mail-2019-03-18.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-03-25.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-04-01.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-04-08.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-04-15.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-04-22.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-05-06.txt: 0 100.0 1 1 SharedFSAutoReplicaFailoverTest.test
e-mail-2019-05-06.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-05-20.txt: 2 100.0 1 1 SharedFSAutoReplicaFailoverTest.test
e-mail-2019-05-20.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-06-03.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-06-10.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-06-17.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-06-24.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-07-01.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-07-29.txt: 1 18.2 11 2 SharedFSAutoReplicaFailoverTest.test
e-mail-2019-07-29.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-08-05.txt: 2 18.2 11 2 SharedFSAutoReplicaFailoverTest.test
e-mail-2019-08-05.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-08-12.txt: 3 18.2 11 2 SharedFSAutoReplicaFailoverTest.test
e-mail-2019-08-12.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-08-19.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-09-16.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-10-28.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-11-04.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-11-11.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-11-18.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-11-25.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-12-02.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-12-09.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-12-23.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2019-12-30.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-01-06.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-01-13.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-01-20.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-01-27.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-02-03.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-02-10.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-03-16.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-03-24.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-03-30.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-04-06.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-04-13.txt: 0 80.0 10 8 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-04-13.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-04-20.txt: 01 50.0 16 11 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-04-20.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-04-27.txt: 012 80.0 21 15 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-04-27.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-05-04.txt: 123 80.0 21 15 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-05-04.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-05-11.txt: 23 80.0 11 7 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-05-11.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-05-18.txt: 3 80.0 5 4 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-05-18.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-05-25.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-06-01.txt: 0 57.1 7 4 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-06-01.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-06-08.txt: 1 57.1 7 4 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-06-08.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-06-15.txt: 0 2 50.0 9 5 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-06-15.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-06-22.txt: 1 3 50.0 9 5 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-06-22.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-06-29.txt: 0 2 90.0 12 10 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-06-29.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-07-06.txt: 01 3 100.0 16 14 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-07-06.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-07-13.txt: 012 100.0 18 17 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-07-13.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-07-20.txt: 0123 57.1 25 21 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-07-20.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-07-27.txt: 0123 93.3 30 26 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-07-27.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-08-03.txt: 0123 93.3 41 36 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-08-03.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite
e-mail-2020-08-10.txt: 0123 100.0 38 33 SharedFSAutoReplicaFailoverTest.test
e-mail-2020-08-10.txt:SharedFSAutoReplicaFailoverTest.SharedFSAutoReplicaFailoverTest suite


Hoss’ weekly rollups:
HOSS-2018-03-26.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","14.2857142857143","7","1"
HOSS-2018-04-16.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","100","2","2"
HOSS-2018-04-25.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","80","5","4"
HOSS-2018-05-21.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","100","2","2"
HOSS-2018-06-18.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","100","2","2"
HOSS-2018-06-25.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","74.0740740740741","27","20"
HOSS-2018-06-25.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","5.88235294117647","34","2"
HOSS-2018-07-02.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","74.0740740740741","27","20"
HOSS-2018-07-02.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","5.12820512820513","39","2"
HOSS-2018-07-09.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","74.0740740740741","27","20"
HOSS-2018-07-09.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","2.27272727272727","44","1"
HOSS-2018-07-16.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","74.0740740740741","27","20"
HOSS-2018-07-16.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","17.6470588235294","34","6"
HOSS-2018-07-23.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","62.5","16","10"
HOSS-2018-07-23.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","4.34782608695652","23","1"
HOSS-2018-08-06.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","66.6666666666667","3","2"
HOSS-2018-08-13.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","100","2","2"
HOSS-2018-09-10.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","20","5","1"
HOSS-2018-09-10.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","50","4","2"
HOSS-2018-09-18.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","66.6666666666667","3","2"
HOSS-2018-09-18.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","33.3333333333333","3","1"
HOSS-2018-10-02.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","100","4","4"
HOSS-2018-12-03.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","100","3","3"
HOSS-2018-12-03.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","33.3333333333333","3","1"
HOSS-2018-12-10.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","150","2","3"
HOSS-2018-12-24.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","33.3333333333333","3","1"
HOSS-2019-01-28.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","60","5","3"
HOSS-2019-02-25.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","33.3333333333333","3","1"
HOSS-2019-05-06.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","100","1","1"
HOSS-2019-06-17.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","100","3","3"
HOSS-2019-07-08.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","18.1818181818182","11","2"
HOSS-2019-12-23.csv:"true","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","","100","3","3"
HOSS-2020-04-13.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","80","10","8"
HOSS-2020-04-20.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","50","6","3"
HOSS-2020-04-27.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","80","5","4"
HOSS-2020-06-01.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","57.1428571428571","7","4"
HOSS-2020-06-15.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","50","2","1"
HOSS-2020-06-29.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","90","10","9"
HOSS-2020-07-06.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","100","4","4"
HOSS-2020-07-13.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","100","4","4"
HOSS-2020-07-20.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","57.1428571428571","7","4"
HOSS-2020-07-27.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","93.3333333333333","15","14"
HOSS-2020-08-03.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","93.3333333333333","15","14"
HOSS-2020-08-10.csv:"false","org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest","test","100","1","1"


> On Aug 12, 2020, at 9:03 AM, Erick Erickson <erickerickson@gmail.com> wrote:
>
>> SharedFSAutoReplicaFailoverTest


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: BadApple report, but please read the first bit [ In reply to ]
dsmiley at apache
Aug 13, 2020, 9:22 PM
Post #6 of 6 (311 views)
Permalink
Thanks Kevin; clearly I missed the link to that which I can now see at
fucit.

I was worried I may have worked on something that could have perturbed this
recent issue but no -- I don't think so.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Wed, Aug 12, 2020 at 9:08 AM Kevin Risden <krisden@apache.org> wrote:

>
> http://fucit.org/solr-jenkins-reports/history-trend-of-recent-failures.html#series/org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
>
> David for that specific test you asked the failures are recent with as far
> as I know no change to HDFS stuff. Starting June/July failing regularly.
>
> Kevin Risden
>
>
>
> On Wed, Aug 12, 2020 at 9:03 AM Erick Erickson <erickerickson@gmail.com>
> wrote:
>
>> I have the weekly rollups (with a few gaps) going back to about April
>> 2018, but nothing’s been done to try to make them generally available. Each
>> BadApple report has rates for the last 4 weeks in the attached file, just
>> below "Failures over the last 4 weeks, but not every week. Ordered
>> most-recent first:”
>>
>>
>>
>> > On Aug 12, 2020, at 2:06 AM, David Smiley <dsmiley@apache.org> wrote:
>> >
>> > Do we have any long term (aka "longitudinal") pass/fail rates for tests?
>> >
>> > SharedFSAutoReplicaFailoverTest in particular is kinda-sorta tied to
>> HDFS, and that's going away to a plug-in for 9.0. The shared file system
>> notion isn't well supported in SolrCloud, I think.
>> >
>> > ~ David Smiley
>> > Apache Lucene/Solr Search Developer
>> > http://www.linkedin.com/in/davidwsmiley
>> >
>> >
>> > On Mon, Aug 3, 2020 at 7:26 AM Erick Erickson <erickerickson@gmail.com>
>> wrote:
>> > There are several tests that are causing a lot of noise:
>> >
>> > SharedFSAutoReplicaFailoverTest is failing 90%+ of the time.
>> > TestBulkSchemaConcurrent 31%
>> > StressHdfsTest 16%
>> > SchemaApiFailureTest 13.88%
>> >
>> > I encourage people to look at:
>> http://fucit.org/solr-jenkins-reports/failure-report.html and see if
>> anything looks like it is affected by recent work. TestBulkSchemaConcurrent
>> has been failing off and on for a long time, but the failure rate picked up
>> dramatically in the last couple of weeks. Ditto SchemaApiFailureTest.
>> >
>> > Do we even care about Hdfs? Are we deprecating it or not?
>> >
>> > Holding relatively steady otherwise:
>> >
>> > Raw fail count by week totals, most recent week first (corresponds to
>> bits):
>> > Week: 0 had 82 failures
>> > Week: 1 had 94 failures
>> > Week: 2 had 502 failures
>> > Week: 3 had 19 failures
>> >
>> >
>> > ********Failures in Hoss' reports for the last 4 rollups.
>> >
>> > There were 562 unannotated tests that failed in Hoss' rollups. Ordered
>> by the date I downloaded the rollup file, newest->oldest. See above for the
>> dates the files were collected
>> > These tests were NOT BadApple'd or AwaitsFix'd
>> >
>> > Failures in the last 4 reports..
>> > Report Pct runs fails test
>> > 0123 0.3 1271 8 RollingRestartTest.test
>> > 0123 93.3 41 36
>> SharedFSAutoReplicaFailoverTest.test
>> > 0123 3.5 627 16
>> TestCircuitBreaker.testBuildingMemoryPressure
>> > 0123 1.0 627 8
>> TestCircuitBreaker.testResponseWithCBTiming
>> > 0123 5.8 1483 79
>> TestContainerPlugin.testApiFromPackage
>> > 0123 2.3 1335 23 TestDistributedGrouping.test
>> > ********************************************
>> >
>> >
>> > Full report:
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal