Mailing List Archive: Re: [master] b6d0ff8 Merge tests and reshuffle things

Re: [master] b6d0ff8 Merge tests and reshuffle things

Jan 27, 2017, 12:11 PM

Post #1 of 4 (897 views)

On Fri, Jan 27, 2017 at 7:37 PM, Federico G. Schwindt <fgsch@lodoss.net> wrote:
>
> commit b6d0ff84fabfce8bffad426c02421eb891ef6a7b
> Author: Federico G. Schwindt <fgsch@lodoss.net>
> Date: Fri Jan 27 18:35:22 2017 +0000
>
> Merge tests and reshuffle things

Instead of shuffling things *right after* I stabilized them we could
have discussed this. It now fails even more frequently. And yes I
know my fix would hide errors (not actual test failures) but at least
it was better than noise considering the hard-coded ports.

I suggest a revert of this commit and then a reshuffle without the
merge of r1813. I'm aware it is covering varnishd usage but it's also
a regression test so I can live with "have same address" being its own
test. For some systems vtest is currently reporting 100% of failures
caused by c3.vtc for 9a652c7.

Dridi

_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev

Re: [master] b6d0ff8 Merge tests and reshuffle things [ In reply to ]

dridi at varni

Jan 27, 2017, 12:14 PM

Post #2 of 4 (897 views)

Permalink

On Fri, Jan 27, 2017 at 9:11 PM, Dridi Boukelmoune <dridi@varni.sh> wrote:
> On Fri, Jan 27, 2017 at 7:37 PM, Federico G. Schwindt <fgsch@lodoss.net> wrote:
>>
>> commit b6d0ff84fabfce8bffad426c02421eb891ef6a7b
>> Author: Federico G. Schwindt <fgsch@lodoss.net>
>> Date: Fri Jan 27 18:35:22 2017 +0000
>>
>> Merge tests and reshuffle things
>
> Instead of shuffling things *right after* I stabilized them we could
> have discussed this. It now fails even more frequently. And yes I
> know my fix would hide errors (not actual test failures) but at least
> it was better than noise considering the hard-coded ports.
>
> I suggest a revert of this commit and then a reshuffle without the
> merge of r1813. I'm aware it is covering varnishd usage but it's also
> a regression test so I can live with "have same address" being its own
> test. For some systems vtest is currently reporting 100% of failures
> caused by c3.vtc for 9a652c7.

I see you've fixed it again while I was writing this message.

_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev

Re: [master] b6d0ff8 Merge tests and reshuffle things [ In reply to ]

fgsch at lodoss

Jan 27, 2017, 12:21 PM

Post #3 of 4 (897 views)

Permalink

Not happy about hiding the error but it will do it for now.

The tests are not using a hardcoded port anymore and should fail with the
expected error now.
Why this is a problem in sunos is beyond to me at the moment.

On Fri, Jan 27, 2017 at 8:14 PM, Dridi Boukelmoune <dridi@varni.sh> wrote:

> On Fri, Jan 27, 2017 at 9:11 PM, Dridi Boukelmoune <dridi@varni.sh> wrote:
> > On Fri, Jan 27, 2017 at 7:37 PM, Federico G. Schwindt <fgsch@lodoss.net>
> wrote:
> >>
> >> commit b6d0ff84fabfce8bffad426c02421eb891ef6a7b
> >> Author: Federico G. Schwindt <fgsch@lodoss.net>
> >> Date: Fri Jan 27 18:35:22 2017 +0000
> >>
> >> Merge tests and reshuffle things
> >
> > Instead of shuffling things *right after* I stabilized them we could
> > have discussed this. It now fails even more frequently. And yes I
> > know my fix would hide errors (not actual test failures) but at least
> > it was better than noise considering the hard-coded ports.
> >
> > I suggest a revert of this commit and then a reshuffle without the
> > merge of r1813. I'm aware it is covering varnishd usage but it's also
> > a regression test so I can live with "have same address" being its own
> > test. For some systems vtest is currently reporting 100% of failures
> > caused by c3.vtc for 9a652c7.
>
> I see you've fixed it again while I was writing this message.
>

Re: [master] b6d0ff8 Merge tests and reshuffle things [ In reply to ]

dridi at varni

Jan 27, 2017, 4:19 PM

Post #4 of 4 (894 views)

Permalink

On Fri, Jan 27, 2017 at 9:21 PM, Federico Schwindt <fgsch@lodoss.net> wrote:
> Not happy about hiding the error but it will do it for now.
>
> The tests are not using a hardcoded port anymore and should fail with the
> expected error now.

Your test case is misleading because it pretends to be safe, but either
${v1_port} is bound when you're using it the test can't [1] possibly pass,
or it's available for anyone to bind and you run into the same
concurrency problem again.

> Why this is a problem in sunos is beyond to me at the moment.

SunOS, FreeBSD, and my Linux box. I tried to get rid of the hard-coded
port too but didn't manage, so I kept it as close as the original test.

This is very simple: varnishtest doesn't offer enough isolation to safely
run this test, especially with concurrent jobs.

Dridi

[1] Except maybe with SO_REUSEPORT, but... can of worms?

_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev