Mailing List Archive

Paul,

Just restating things I've said here before :

- I don't the issue is backlog

- I do think the issue is that in order for
organizations/companies/individuals to invest in an open source project
that there needs to be process transparency, predicable releases, and a
deterministic way to get submissions/patches into a release.

- I frankly don't see how the current model scales or satisfies this.

- A bunch of the community worked together to come up with a proposed
new process and wanted to try it out -- which, from my perspective, you
vetoed.

I think these leaves the ball in your court.

Lou


On 9/30/2016 6:35 AM, Paul Jakma wrote:
> On Wed, 21 Sep 2016, Lou Berger wrote:
>
>> I hope the the last two releases have convinced you that we need to do
>> something different WRT patch integration.
> Convinced in what sense? Also, the implication is that you feel I am
> against changing /anything/ - which is not at all the case.
>
> High level objectives of getting patches reviewed, decided on and
> integrated/pushed-back quickly, efficiently, transparently, reliably?
> Who could disagree with that?
>
> Determining how to change things to improve on delivery of those goals,
> by teasing out the strands of the problem, sorting the objective from
> the subject (and where possible finding ways to transform as much of the
> subjective into the objective as possible), and determining as
> analytically as possible how proposed chnages affect the different
> constraints (at least some of which are in tension): Great.
>
> Throwing everything up in the air, changing everything at the same time
> (from communication methods, to constitution) - less keen on that.
>
> ----
>
> And there were big changes when I started full-time again. Vincent
> started with the batched, systematic, review process of the backlog in
> patchwork. I continued on with that, changed some logging aspects from
> email to git.
>
> Is that process ideal for a non-backlogged, steady-state? Perhaps not.
>
> However, the problem then was backlog. And optimising integrator
> throughput - while not regressing back to more ad-hoc much less
> systematic working - to get the backlog done seemed to be a high
> priority. That hopefully really is close to done now, with the final CI
> obstacle found, and we can close off a large swathe of old stuff and
> release.
>
> It could have gone faster perhaps, with more co-operation and fewer
> other distractions. But hey.
>
> regards,


_______________________________________________
Quagga-dev mailing list
Quagga-dev@lists.quagga.net
https://lists.quagga.net/mailman/listinfo/quagga-dev

Hi Lou,

On Fri, 30 Sep 2016, Lou Berger wrote:

> Paul,
>
> Just restating things I've said here before :
>
> - I don't the issue is backlog

It definitely was an issue. People were complaining about how long stuf
had been waiting. I'm not imagining that, pretty sure.

More generally, there are issue_s_. I don't think there is any /one/
issue, the solving of which there is a magic bullet for.

> - I do think the issue is that in order for
> organizations/companies/individuals to invest in an open source
> project that there needs to be process transparency, predicable
> releases, and a deterministic way to get submissions/patches into a
> release.

process transparency, regular releases, deterministic process for
submissions, yes, all good. Though "to get submissions/patches into a
release" is assuming a bit too much.

> - I frankly don't see how the current model scales or satisfies this.

Well, based on queue length it has objectively done better than what was
there before. As the queue length has gone from increasing to
decreasing, and from years of stuff waiting for action on the review to
less than a years worth of stuff (assuming Martin's full protocol tests
pass and we get a release out RSN).

Is it the best model for a steady state? I don't know. Are there further
improvements to make, no doubt.

However, it seemed the best way to get stuff done on the backlog, in a
fairly transparent and systematic way, given the lack of willing
resources.

> - A bunch of the community worked together to come up with a proposed
> new process and wanted to try it out -- which, from my perspective,
> you vetoed.

Cause it wanted to change _everything_:

- The ethos and constitution of Quagga

From one where the onus is on submitters to get submissions to a
standard high enough in terms of (as applicable) design documentation
and advance planning, code organisation and style, and testing, so
that at least no one else strongly enough to object;

To one where alliances of contributors, potentially some with next to
_no prior involvement_ in Quagga, could band together to shove stuff
through, and where reviewers with objections would have only days to
try rally others to their view. To the extent I'm mistaken on that, it
was to the extent this was under-specified (however, the intent of
majority voting for stuff to go in was clearly there).

I.e., code going in based on the 51% with the /lowest/ standards, and
a more political and gameable process.

I think the biggest gulf between the proposers of the mega-changes and
myself was in this area.

- The integration process was actually _gaining_ overhead and
bureaucracy from a day to day process.

Patches to require explicit 2nd ACKs - that's overhead. IME with
Quagga, most stuff doesn't need ACKs. People will object to
objectionable stuff, and they'll let it pass otherwise. A "Revisions
needed" model for those minority of patches is more efficient than
a "ack for every patch".

- The proposed integration process was more a contributors wish list
than a readily implementable integration process.

Patches to go be applied within days, by who? How?

I've written before about lessons learned in the past from Quagga, on
patches can more easily fall through the cracks when it isn't clear
who is responsible for applying. Also "days" - there are a number of
problems with that, in various dimensions (see also above on ethos).

The contributors view of the process is only half the story. It also
has to work for and be efficient for the integrator(s). See also
prior. If you increase the overheads for integrators, then things
likely get worse.

- Changing the communication tools from email to video conferencing.

Real-time comms is nice for forming opinions, but it's ungood for
definitive decision making comms. Quagga, as with many open-source
projects, is distributed across the world, and it impossible to suit
everyone (and people in global orgs may already have calendar
competition for time-slots that suit multiple parts of the globe).

AV also needs reliable bandwidth, not always a guarantee. (E.g., right
now I don't have that).

Also, I much prefer email for deliberative stuff.

- Multiple branches and releases, with unclear relationships between
them.

Overall, the proposals were vague on many important things. On some of
the more concrete points, they seemed to be objectively adding overheads
(rather than decreasing), and changing the ethos of the project to one
that I don't like (cause, I've done the "shove everything in" thing
already - in the early days of Quagga; and it lead to quality issues
that took a _lot_ of years to fix; so I don't want to go back there).

So, let's change stuff for the better. There's a lot of scope for more
automation, and moving more of the work to the contributor side:

- some kind of auto-integrating head, to accumulate stuff from the list
in a linear order and do the CI pass/fail.

I'm less keen on having GitHub in the middle though.

- work out how to fit review onto that. I'd go with a batched pipe-line
for this. Basically, kicking out the CI-pass patches that have
other nits / review issues, and punting them with comments back to the
submitter.

- releases: I'd almost be tempted to make that a cron job

Identify specific issues, or specific potential improvements, and lets
evaluate them - bearing in mind many of the goals and constraints are in
tension.

Further, those tensions can be dynamic in nature. Optimising to reduce
one tension, may over time increase another tension. Oscillating hard
over time, bouncing between tensions, may not be good either...

regards,
--
Paul Jakma | paul@jakma.org | @pjakma | Key ID: 0xD86BF79464A2FF6A
Fortune:
Children begin by loving their parents. After a time they judge them.
Rarely, if ever, do they forgive them.
-- Oscar Wilde

_______________________________________________
Quagga-dev mailing list
Quagga-dev@lists.quagga.net
https://lists.quagga.net/mailman/listinfo/quagga-dev

Paul,

So where do you want to go with this? Perhaps, propose specific
changes to the community (based on current docs, or last round of
proposed) and see if there's buy-in?

Really, it comes down to how you plan to do things differently for the
next release.

Lou

On 10/3/2016 5:44 AM, Paul Jakma wrote:
> Hi Lou,
>
> On Fri, 30 Sep 2016, Lou Berger wrote:
>
>> Paul,
>>
>> Just restating things I've said here before :
>>
>> - I don't the issue is backlog
> It definitely was an issue. People were complaining about how long stuf
> had been waiting. I'm not imagining that, pretty sure.
>
> More generally, there are issue_s_. I don't think there is any /one/
> issue, the solving of which there is a magic bullet for.
>
>> - I do think the issue is that in order for
>> organizations/companies/individuals to invest in an open source
>> project that there needs to be process transparency, predicable
>> releases, and a deterministic way to get submissions/patches into a
>> release.
> process transparency, regular releases, deterministic process for
> submissions, yes, all good. Though "to get submissions/patches into a
> release" is assuming a bit too much.
>
>> - I frankly don't see how the current model scales or satisfies this.
> Well, based on queue length it has objectively done better than what was
> there before. As the queue length has gone from increasing to
> decreasing, and from years of stuff waiting for action on the review to
> less than a years worth of stuff (assuming Martin's full protocol tests
> pass and we get a release out RSN).
>
> Is it the best model for a steady state? I don't know. Are there further
> improvements to make, no doubt.
>
> However, it seemed the best way to get stuff done on the backlog, in a
> fairly transparent and systematic way, given the lack of willing
> resources.
>
>> - A bunch of the community worked together to come up with a proposed
>> new process and wanted to try it out -- which, from my perspective,
>> you vetoed.
> Cause it wanted to change _everything_:
>
> - The ethos and constitution of Quagga
>
> From one where the onus is on submitters to get submissions to a
> standard high enough in terms of (as applicable) design documentation
> and advance planning, code organisation and style, and testing, so
> that at least no one else strongly enough to object;
>
> To one where alliances of contributors, potentially some with next to
> _no prior involvement_ in Quagga, could band together to shove stuff
> through, and where reviewers with objections would have only days to
> try rally others to their view. To the extent I'm mistaken on that, it
> was to the extent this was under-specified (however, the intent of
> majority voting for stuff to go in was clearly there).
>
> I.e., code going in based on the 51% with the /lowest/ standards, and
> a more political and gameable process.
>
> I think the biggest gulf between the proposers of the mega-changes and
> myself was in this area.
>
> - The integration process was actually _gaining_ overhead and
> bureaucracy from a day to day process.
>
> Patches to require explicit 2nd ACKs - that's overhead. IME with
> Quagga, most stuff doesn't need ACKs. People will object to
> objectionable stuff, and they'll let it pass otherwise. A "Revisions
> needed" model for those minority of patches is more efficient than
> a "ack for every patch".
>
> - The proposed integration process was more a contributors wish list
> than a readily implementable integration process.
>
> Patches to go be applied within days, by who? How?
>
> I've written before about lessons learned in the past from Quagga, on
> patches can more easily fall through the cracks when it isn't clear
> who is responsible for applying. Also "days" - there are a number of
> problems with that, in various dimensions (see also above on ethos).
>
> The contributors view of the process is only half the story. It also
> has to work for and be efficient for the integrator(s). See also
> prior. If you increase the overheads for integrators, then things
> likely get worse.
>
> - Changing the communication tools from email to video conferencing.
>
> Real-time comms is nice for forming opinions, but it's ungood for
> definitive decision making comms. Quagga, as with many open-source
> projects, is distributed across the world, and it impossible to suit
> everyone (and people in global orgs may already have calendar
> competition for time-slots that suit multiple parts of the globe).
>
> AV also needs reliable bandwidth, not always a guarantee. (E.g., right
> now I don't have that).
>
> Also, I much prefer email for deliberative stuff.
>
> - Multiple branches and releases, with unclear relationships between
> them.
>
> Overall, the proposals were vague on many important things. On some of
> the more concrete points, they seemed to be objectively adding overheads
> (rather than decreasing), and changing the ethos of the project to one
> that I don't like (cause, I've done the "shove everything in" thing
> already - in the early days of Quagga; and it lead to quality issues
> that took a _lot_ of years to fix; so I don't want to go back there).
>
> So, let's change stuff for the better. There's a lot of scope for more
> automation, and moving more of the work to the contributor side:
>
> - some kind of auto-integrating head, to accumulate stuff from the list
> in a linear order and do the CI pass/fail.
>
> I'm less keen on having GitHub in the middle though.
>
> - work out how to fit review onto that. I'd go with a batched pipe-line
> for this. Basically, kicking out the CI-pass patches that have
> other nits / review issues, and punting them with comments back to the
> submitter.
>
> - releases: I'd almost be tempted to make that a cron job
>
> Identify specific issues, or specific potential improvements, and lets
> evaluate them - bearing in mind many of the goals and constraints are in
> tension.
>
> Further, those tensions can be dynamic in nature. Optimising to reduce
> one tension, may over time increase another tension. Oscillating hard
> over time, bouncing between tensions, may not be good either...
>
> regards,


_______________________________________________
Quagga-dev mailing list
Quagga-dev@lists.quagga.net
https://lists.quagga.net/mailman/listinfo/quagga-dev