Hi,
People can have opinions, however they can not have their own facts. If
people say things in public about the past of Quagga which are blatantly
(and surely wilfully) misleading I will provide facts. To that end:
- The email based process was _OpenSourceRouting's_, _never_ my
preferred way of working. See quoted email below:
"Ultimately, I think email is good for discussions, but terrible for
holding state. I would prefer we use git to track state."
Bugzilla bug or RFE as a canonical place-mark/tracking-point for a
contribution (there are a number of tools to integrate bugzilla with
git), or GitLab, is what I was advocating.
- I tried my damnedest to get certain people to agree to reforms of
governance (again, see email below - just one of many), but they would
never engage. They just rejected proposals outright, with minimal
discussion, (e.g. the below and attached) which surely were reasonable
places to /start/ a discussion.
(And which later actions of theirs would appear to confirm were not at
all unreasonable concerns).
I tried, for at least 1.5 years, to work with people who were quibbling
about things, to find /some/ kind of reasonable solution. I did my best
to listen, to propose reasonable solutions. They _refused_ to engage
productively.
Then I was given ultimatums.
Including on silly stuff like use of Google Hangouts (Google, whatever
one's own preference, is problematic for certain people; some because of
principle, others because of location, as GOOG properties are blocked in
fairly non-trivial parts of the world, including ones I sometimes spend
non-trivial chunks of time in), or real-time conf. calls for final
decision making (having experience of working in environments where one
must schedule calls with various people across the world, real-time
calendar scheduling is a very difficult task, which doesn't scale up -
and theory says so too) - not even on these things would people budge.
When pretty much all the public rationales for motivation are
counter-factual or at odds with the later actions of people, one must
ask what rationales could possibly be left.
Trying to pin the problems of:
- Low activity
- Unfit for purpose method of tracking contributions
- Blockage of changes to governance
on Quagga.net is _dishonest_, when these were issues (to the extent they
were issues) which _OpenSourceRouting_ - who are _no longer with
Quagga.net_ - were responsible for.
regards,
--
Paul Jakma | paul@jakma.org | @pjakma | Key ID: 0xD86BF79464A2FF6A
Fortune:
The Law of the Letter:
The best way to inspire fresh thoughts is to seal the envelope.
---------- Forwarded message ----------
Date: Tue, 20 Jan 2015 17:54:03 +0000 (GMT)
From: Paul Jakma <paul@jakma.org>
To: Martin Winter <mwinter@opensourcerouting.org>
Subject: processes
Hi,
So, to pick up the processes thread, interrupted by my fall and break(s).
Obviously, the issue is finding the right state machine and accompanying
workflow to act as the process for submitting, tracking, reviewing and
accepting contributions. On the one hand, that state machine has to fit the
software engineering and social goals and needs of the project. On the other
hand, that state machine has fit in to a workflow constrained by the tools
available, to come up with something workable. Finally, we want that workflow
to be as low overhead as possible. We need a goldilocks level of process, not
too much, not too little but just right. The little process we have today is
probably no longer sufficient for the number of people and interests.
What's the abstract flow for contributions to Quagga?
1. There is the general universe of patches to Quagga
2. Some of these are offered for inclusion, or come to the attention of
the maintainers and thus need review
3. Some have had review but have concerns that need to be addressed.
3a. The most important concerns are often the most abstract: does the
contribution push Quagga in the right direction in terms of
architecture, does it improve or at least maintain the cohesiveness of
design, etc.?
This level of review is about the fundamentals of the contribution. It
can be quite subjective, and it may be hard for a submitter to address
concerns without major reworking. As it is at an abstract level, in
the ideal case the contributor would go through this review /before/
doing the bulk of detailed coding work. Potentially this review could
be sought /before/ doing any coding.
3b. Other concerns can be at the implementation level. Nits in the code or
arrangement of files, lack of documentation, formatting, or
presentation of the contribution, etc. The contributor should address
these.
4. Next, at least some people will want some level of testing of the
contribution. Unit tests, conformance tests, etc.
So there seems to be obvious states and general process here for proposed
changes:
* Pending proposals
* Review states:
** Architecture review
** Implementation/change review
* Accepted:
** Testing
** Mature
I've done a diagramme, see abstract-workflow.png (original source is the .dia).
That's pretty much the rough process we go through, but we're not very clear
about the states, and we don't give clear feedback about state transitions.
The "Timeout" transition is particularly vague today.
It seems to me we could easily be more transparent and explicit about our
process. This would help contributors, and it help the maintainers.
E.g. at present it's very possible that a contributor is fuming, waiting for
someone to act on his patch, while it's completely off the radar of the
maintainers. Being explicit about timeouts would at least clear that up for
everyone.
Further, there's an opportunity to involve the community more clearly and
explicitly in the architectural review stuff. We can create an explicit "Review
Committee" and have it be fairly open. That's an easy way to make it explicitly
clear to community members that they're involved.
Next, it would be good to be explicit about the results of reviews. E.g. if one
reviewer things something is good and another not. I think we need explicit
votes, at least for anything contentious. This would avoid the problem of
unclear resolutions, where a proposal hasn't been rejected by everyone and
hasn't been accepted by everyone either and it just unclear what the status is.
There's been a couple of cases recently, e.g.
So I would propose we start a "Quagga Architecture Review Committee". See next
section.
On the testing side, I think Quagga.net master should be "Testing" portion of
the above, not the mature portion. I think everyone is agreed that really.
Architecture Review Committee:
--------------------
I think we need a "Quagga Architecture Review Committee" (QARC) to review
things. The rough, high-level flow would be as in the attached
architecture-review-detail.png diagramme (with .dia original source).
Firstly, all new proposed changes *must* go through architecture review.
For simple stuff, this could be a fast-track "No architecture concerns"
approval. This should be a quick & lightweight process, and probably the most
commonly used.
For full reviews we might demand the submitter provide some level of discussion
of the architectural concerns and why their proposal is best.
Submitters could indicate in changes whether they think they need a full review
or are suitable for fast-track. With either the justification for the
fast-track, or else the full set of materials to back up why their proposal is
the right architecture. Other pending changes could be triaged by Quagga
maintainers for this otherwise.
Fast-track proposals can be kicked to full review if needs be.
Note that the later implentation review can effectively kick any proposal back
to full review by raising major concerns. Including any proposal that had
previously had full review waived and had a fast-track architecture review
approval.
Full review (or implementation review) can result in major concerns being
raised, which would require the submitter to address them to the reviewers
satisfaction.
Major concerns kick a proposal back to pending, where it will have to start the
process again. A reject would pretty final for the proposal, in that form.
The architecture review need not be done by maintainers. We could have a
committee from the wider community - the "ARC". Anyone interested could ask to
join this committee, and be accepted by a vote of the ARC, say.
Fast-track approvals could be given by any ARC member, as long as they are not
associated with the submitter. Full ARC reviews might require a majority
(simple or super), with fewer than X NACKs, say.
This process is loosely based on the system used at Sun, which I thought worked
quite well. Having a defined ARC would be an easy way to get wider community
involvement and help ensure there is broad consensus on features - rather than
having to have the maintainers "sense" the wind. Everyone has an opinion on
architecture, and being on the ARC would give some recognition to people. Being
on the ARC wouldn't mean having to commit to time-consuming code reviews, or
mean that we have to open final commit access to a wide set of people.
A big benefit is that the ARC review does *not* require code. It would just be
a text document. This would allow us to encourage people to get ideas reviewed
*before* they spend lots of time coding, and potentially save wasted effort.
They can have the assurance of a Full approval before they start implementing.
It would be *really* good to encourage people to avail of this before starting
big projects, to help avoid integration issues, even disappointment later.
I have attached a rough, initial, probably incomplete "charter" for this QARC,
see quagga-arc-charter.txt which sets out the rules, and a suggested template
that contributors might use to submit proposals for "full reviews", see
arc-template.txt.
Fast-tracks wouldn't need such a template.
Of course, such documents might need further evolution. Probably best to get a
QARC running and let it do that though.
Implementation review:
----------------------
This would be concerned with the nit-picky details for a proposed git commit.
Code issues, commit messages, etc.
This would basically be the review process we have now, where anyone can raise
issues on list, and where final approval comes from the maintainers. So the
"Iplmentation review commmittee" here I guess consists of "public" and "core"
members. Anyone can give advisory reviews, and the "IRcore" / "maintainers" can
make binding decisions.
A changeset could be kicked back to ARC by raising major concerns. This would
generally be for changes implementation review uncovers issues that mean a full
ARC review is required. Typically this would be for:
* Fast-track approved change with some issue that wasn't realised or
disclosed in the initial submission.
* Full-review approved change, where the implementation review uncovers
major issues that had not been disclosed to or realised by the full ARC
review
In very rare cases an implementation review might raise a major concern for an
issue that had already been explicitly considered and approved by the ARC. This
would put the implementation reviewers at odds with the ARC, which would be an
unfortunate situation. This could be avoided by having those reviewers sit on
the ARC, and the ARC having veto votes.
In the long-run, it would be nice to formalise the implementation review
process a bit more so that we could get it down to a checklist. Again, at some
point we could open aspects of this to a wider community "committee".
I.e. to start off the "Implementation RC" is basically the maintainers working
to ad-hoc-ish processes, as per today, but we should try formalise that too in
time (as formal as needs be, but no more!).
Acceptance:
-----------
When things have both ARC and IRcore approval they can be accepted. Changes
then need testing. Testing can be some combination of unit tests, conformance
tests, test networks, availability for public testing, etc. After some
combination of these (e.g. "No problem reports in X time"), stuff may be viewed
as "mature".
I would suggest Quagga.net master should correspond to the "Testing" state. I
think the "Mature" state is, largely, not something that Quagga.net can
usefully specify, as different people can have very different requirements for
this.
Tools:
------
In terms of the tools, we have git, email and patchwork. Patchwork can track
diffs in some emails and assign/remember states for the diff/contribution,
based on special email tags, if people remember. Git can also maintain states
for diffs, by using different branches. Patchwork has a web interface and a
terminal client, but you can only submit patches or add reviews/comments via
the mailing list.
Ultimately, I think email is good for discussions, but terrible for holding
state. I would prefer we use git to track state.
E.g. I think we should use git branches to track incoming patches. So the
"Pending" state should be a "pending/" namespace in git, not a list email in
patchwork. It should require a positive act of triage for a patch to end up
there, so that it is explicit to everyone, including the contributor whether or
not the patch has been picked up.
We might also need some kind of "rejected/" namespace in git - but it needs a
better name. ;)
For proposals and QARC review state, I'd prefer files and directories in a git
repo.
But we can discuss tooling further.
If that seems sane, the next step is to discuss this more widely.
regards,
--
Paul Jakma paul@jakma.org @pjakma Key ID: 64A2FF6A
Fortune:
heavy, adj.:
Seduced by the chocolate side of the force.
People can have opinions, however they can not have their own facts. If
people say things in public about the past of Quagga which are blatantly
(and surely wilfully) misleading I will provide facts. To that end:
- The email based process was _OpenSourceRouting's_, _never_ my
preferred way of working. See quoted email below:
"Ultimately, I think email is good for discussions, but terrible for
holding state. I would prefer we use git to track state."
Bugzilla bug or RFE as a canonical place-mark/tracking-point for a
contribution (there are a number of tools to integrate bugzilla with
git), or GitLab, is what I was advocating.
- I tried my damnedest to get certain people to agree to reforms of
governance (again, see email below - just one of many), but they would
never engage. They just rejected proposals outright, with minimal
discussion, (e.g. the below and attached) which surely were reasonable
places to /start/ a discussion.
(And which later actions of theirs would appear to confirm were not at
all unreasonable concerns).
I tried, for at least 1.5 years, to work with people who were quibbling
about things, to find /some/ kind of reasonable solution. I did my best
to listen, to propose reasonable solutions. They _refused_ to engage
productively.
Then I was given ultimatums.
Including on silly stuff like use of Google Hangouts (Google, whatever
one's own preference, is problematic for certain people; some because of
principle, others because of location, as GOOG properties are blocked in
fairly non-trivial parts of the world, including ones I sometimes spend
non-trivial chunks of time in), or real-time conf. calls for final
decision making (having experience of working in environments where one
must schedule calls with various people across the world, real-time
calendar scheduling is a very difficult task, which doesn't scale up -
and theory says so too) - not even on these things would people budge.
When pretty much all the public rationales for motivation are
counter-factual or at odds with the later actions of people, one must
ask what rationales could possibly be left.
Trying to pin the problems of:
- Low activity
- Unfit for purpose method of tracking contributions
- Blockage of changes to governance
on Quagga.net is _dishonest_, when these were issues (to the extent they
were issues) which _OpenSourceRouting_ - who are _no longer with
Quagga.net_ - were responsible for.
regards,
--
Paul Jakma | paul@jakma.org | @pjakma | Key ID: 0xD86BF79464A2FF6A
Fortune:
The Law of the Letter:
The best way to inspire fresh thoughts is to seal the envelope.
---------- Forwarded message ----------
Date: Tue, 20 Jan 2015 17:54:03 +0000 (GMT)
From: Paul Jakma <paul@jakma.org>
To: Martin Winter <mwinter@opensourcerouting.org>
Subject: processes
Hi,
So, to pick up the processes thread, interrupted by my fall and break(s).
Obviously, the issue is finding the right state machine and accompanying
workflow to act as the process for submitting, tracking, reviewing and
accepting contributions. On the one hand, that state machine has to fit the
software engineering and social goals and needs of the project. On the other
hand, that state machine has fit in to a workflow constrained by the tools
available, to come up with something workable. Finally, we want that workflow
to be as low overhead as possible. We need a goldilocks level of process, not
too much, not too little but just right. The little process we have today is
probably no longer sufficient for the number of people and interests.
What's the abstract flow for contributions to Quagga?
1. There is the general universe of patches to Quagga
2. Some of these are offered for inclusion, or come to the attention of
the maintainers and thus need review
3. Some have had review but have concerns that need to be addressed.
3a. The most important concerns are often the most abstract: does the
contribution push Quagga in the right direction in terms of
architecture, does it improve or at least maintain the cohesiveness of
design, etc.?
This level of review is about the fundamentals of the contribution. It
can be quite subjective, and it may be hard for a submitter to address
concerns without major reworking. As it is at an abstract level, in
the ideal case the contributor would go through this review /before/
doing the bulk of detailed coding work. Potentially this review could
be sought /before/ doing any coding.
3b. Other concerns can be at the implementation level. Nits in the code or
arrangement of files, lack of documentation, formatting, or
presentation of the contribution, etc. The contributor should address
these.
4. Next, at least some people will want some level of testing of the
contribution. Unit tests, conformance tests, etc.
So there seems to be obvious states and general process here for proposed
changes:
* Pending proposals
* Review states:
** Architecture review
** Implementation/change review
* Accepted:
** Testing
** Mature
I've done a diagramme, see abstract-workflow.png (original source is the .dia).
That's pretty much the rough process we go through, but we're not very clear
about the states, and we don't give clear feedback about state transitions.
The "Timeout" transition is particularly vague today.
It seems to me we could easily be more transparent and explicit about our
process. This would help contributors, and it help the maintainers.
E.g. at present it's very possible that a contributor is fuming, waiting for
someone to act on his patch, while it's completely off the radar of the
maintainers. Being explicit about timeouts would at least clear that up for
everyone.
Further, there's an opportunity to involve the community more clearly and
explicitly in the architectural review stuff. We can create an explicit "Review
Committee" and have it be fairly open. That's an easy way to make it explicitly
clear to community members that they're involved.
Next, it would be good to be explicit about the results of reviews. E.g. if one
reviewer things something is good and another not. I think we need explicit
votes, at least for anything contentious. This would avoid the problem of
unclear resolutions, where a proposal hasn't been rejected by everyone and
hasn't been accepted by everyone either and it just unclear what the status is.
There's been a couple of cases recently, e.g.
So I would propose we start a "Quagga Architecture Review Committee". See next
section.
On the testing side, I think Quagga.net master should be "Testing" portion of
the above, not the mature portion. I think everyone is agreed that really.
Architecture Review Committee:
--------------------
I think we need a "Quagga Architecture Review Committee" (QARC) to review
things. The rough, high-level flow would be as in the attached
architecture-review-detail.png diagramme (with .dia original source).
Firstly, all new proposed changes *must* go through architecture review.
For simple stuff, this could be a fast-track "No architecture concerns"
approval. This should be a quick & lightweight process, and probably the most
commonly used.
For full reviews we might demand the submitter provide some level of discussion
of the architectural concerns and why their proposal is best.
Submitters could indicate in changes whether they think they need a full review
or are suitable for fast-track. With either the justification for the
fast-track, or else the full set of materials to back up why their proposal is
the right architecture. Other pending changes could be triaged by Quagga
maintainers for this otherwise.
Fast-track proposals can be kicked to full review if needs be.
Note that the later implentation review can effectively kick any proposal back
to full review by raising major concerns. Including any proposal that had
previously had full review waived and had a fast-track architecture review
approval.
Full review (or implementation review) can result in major concerns being
raised, which would require the submitter to address them to the reviewers
satisfaction.
Major concerns kick a proposal back to pending, where it will have to start the
process again. A reject would pretty final for the proposal, in that form.
The architecture review need not be done by maintainers. We could have a
committee from the wider community - the "ARC". Anyone interested could ask to
join this committee, and be accepted by a vote of the ARC, say.
Fast-track approvals could be given by any ARC member, as long as they are not
associated with the submitter. Full ARC reviews might require a majority
(simple or super), with fewer than X NACKs, say.
This process is loosely based on the system used at Sun, which I thought worked
quite well. Having a defined ARC would be an easy way to get wider community
involvement and help ensure there is broad consensus on features - rather than
having to have the maintainers "sense" the wind. Everyone has an opinion on
architecture, and being on the ARC would give some recognition to people. Being
on the ARC wouldn't mean having to commit to time-consuming code reviews, or
mean that we have to open final commit access to a wide set of people.
A big benefit is that the ARC review does *not* require code. It would just be
a text document. This would allow us to encourage people to get ideas reviewed
*before* they spend lots of time coding, and potentially save wasted effort.
They can have the assurance of a Full approval before they start implementing.
It would be *really* good to encourage people to avail of this before starting
big projects, to help avoid integration issues, even disappointment later.
I have attached a rough, initial, probably incomplete "charter" for this QARC,
see quagga-arc-charter.txt which sets out the rules, and a suggested template
that contributors might use to submit proposals for "full reviews", see
arc-template.txt.
Fast-tracks wouldn't need such a template.
Of course, such documents might need further evolution. Probably best to get a
QARC running and let it do that though.
Implementation review:
----------------------
This would be concerned with the nit-picky details for a proposed git commit.
Code issues, commit messages, etc.
This would basically be the review process we have now, where anyone can raise
issues on list, and where final approval comes from the maintainers. So the
"Iplmentation review commmittee" here I guess consists of "public" and "core"
members. Anyone can give advisory reviews, and the "IRcore" / "maintainers" can
make binding decisions.
A changeset could be kicked back to ARC by raising major concerns. This would
generally be for changes implementation review uncovers issues that mean a full
ARC review is required. Typically this would be for:
* Fast-track approved change with some issue that wasn't realised or
disclosed in the initial submission.
* Full-review approved change, where the implementation review uncovers
major issues that had not been disclosed to or realised by the full ARC
review
In very rare cases an implementation review might raise a major concern for an
issue that had already been explicitly considered and approved by the ARC. This
would put the implementation reviewers at odds with the ARC, which would be an
unfortunate situation. This could be avoided by having those reviewers sit on
the ARC, and the ARC having veto votes.
In the long-run, it would be nice to formalise the implementation review
process a bit more so that we could get it down to a checklist. Again, at some
point we could open aspects of this to a wider community "committee".
I.e. to start off the "Implementation RC" is basically the maintainers working
to ad-hoc-ish processes, as per today, but we should try formalise that too in
time (as formal as needs be, but no more!).
Acceptance:
-----------
When things have both ARC and IRcore approval they can be accepted. Changes
then need testing. Testing can be some combination of unit tests, conformance
tests, test networks, availability for public testing, etc. After some
combination of these (e.g. "No problem reports in X time"), stuff may be viewed
as "mature".
I would suggest Quagga.net master should correspond to the "Testing" state. I
think the "Mature" state is, largely, not something that Quagga.net can
usefully specify, as different people can have very different requirements for
this.
Tools:
------
In terms of the tools, we have git, email and patchwork. Patchwork can track
diffs in some emails and assign/remember states for the diff/contribution,
based on special email tags, if people remember. Git can also maintain states
for diffs, by using different branches. Patchwork has a web interface and a
terminal client, but you can only submit patches or add reviews/comments via
the mailing list.
Ultimately, I think email is good for discussions, but terrible for holding
state. I would prefer we use git to track state.
E.g. I think we should use git branches to track incoming patches. So the
"Pending" state should be a "pending/" namespace in git, not a list email in
patchwork. It should require a positive act of triage for a patch to end up
there, so that it is explicit to everyone, including the contributor whether or
not the patch has been picked up.
We might also need some kind of "rejected/" namespace in git - but it needs a
better name. ;)
For proposals and QARC review state, I'd prefer files and directories in a git
repo.
But we can discuss tooling further.
If that seems sane, the next step is to discuss this more widely.
regards,
--
Paul Jakma paul@jakma.org @pjakma Key ID: 64A2FF6A
Fortune:
heavy, adj.:
Seduced by the chocolate side of the force.
Attached Files:
-
abstract-workflow.dia (2.84 KB)
-
architecture-review-detail.dia (2.50 KB)
-
quagga-arc-charter.txt (7.94 KB)
-
arc-template.txt (2.34 KB)
-
abstract-workflow.png (25.3 KB)
-
architecture-review-detail.png (20.6 KB)