Mailing List Archive

Ideas for improving the contribution experience
(Context: Continuing to prepare for the core dev sprint next week. Since
the sprint is near, *I'd greatly appreciate any quick comments, feedback
and ideas!*)

Following up my collection of past beginning contributor experiences, I've
collected these experiences in a dedicated GitHub repo[1] and written a
(subjective!) summary of main themes that I recognize in the stories, which
I've also included in the repo[2].

A "TL;DR" bullet list of those main themes:
* Slow/no responsiveness
* Long, slow process
* Hard to find where to contribute
* Mentorship helps a lot, but is scarce
* A lot to learn to get started
* It's intimidating

More specifically, something that has come up often is that maintaining
momentum for new contributors is crucial for them to become long-term
contributors. Most often, this comes up in relation to the first two
points: Suggestions or PRs are completely receive no attention at all
("ignored") or stop receiving attention at some point ("lost to the void").
Unfortunately, the probability of this is pretty high for any issue/PR, so
for a new contributor this is almost guaranteed to happen while working on
one of their first few contributions. I've seen this happen many times, and
have found that I have to personally follow promising contributors' work to
ensure that this doesn't happen to them. I've also seen contributors learn
to actively seek out core devs when these situations arise, which is often
a successful tactic, but shouldn't be necessary so often.

Now, this is in large part a result of the fact that us core devs are not a
very large group, made up almost entirely of volunteers working on this in
their spare time. Last I checked, the total amount of paid development time
dedicated to developing Python is less than 3 full-time (i.e. ~100 hours a
week).

The situation being problematic is clear enough that the PSF had concrete
plans to hire paid developers to review issues and PRs. However, those
plans have been put on hold indefinitely, since the PSF's funding has
shrunk dramatically since the COVID-19 outbreak (no PyCon!).

So, what can be done? Besides raising more funds (see a note on this
below), I think we can find ways to reduce how often issues/PRs become
"stalled". Here are some ideas:

1. *Generate reminders for reviewers when an issue or PR becomes "stalled'
due to them.* Personally, I've found that both b.p.o. and GitHub make it
relatively hard to remember to follow up on all of the many issues/PRs
you've taken part in reviewing. It takes considerable attention and
discipline to do so consistently, and reminders like these would have
helped me. Many (many!) times, all it took to get an issue/PR moving
forward (or closed) was a simple "ping?" comment.

2. *Generate reminders for contributors when an issue or PR becomes
"stalled" due to them.* Similar to the above, but I consider these separate.

3. *Advertise something like a "2-for-1" standing offer for reviews.* This
would give contributors an "official", acceptable way to get attention for
their issue/PR, other than "begging" for attention on a mailing list. There
are good ways for new contributors to be of significant help despite being
new to the project, such as checking whether old bugs are still relevant,
searching for duplicate issues, or applying old patches to the current code
and creating a PR. (This would be similar to Martin v. Löwis's 5-for-1
offer in 2012[3], which had little success but lead to some interesting
followup discussion[4]).

4. *Encourage core devs to dedicate some of their time to working through
issues/PRs which are "ignored" or "stalled".* This would require first
generating reliable lists of issues/PRs in such states. This could be in
various forms, such as predefined GitHub/b.p.o. queries, a dedicated
web-page, a periodic message similar to b.p.o.'s "weekly summary" email, or
dedicated tags/labels for issues/PRs. (Perhaps prioritize "stalled" over
"ignored".)

- Tal Einat


[1]: https://github.com/taleinat/python-contribution-feedback
[2]:
https://github.com/taleinat/python-contribution-feedback/blob/master/Takeaways%20-%20October%202020.md
[3]:
https://mail.python.org/archives/list/python-dev@python.org/message/7DLUN4Y7P77BSDW5YRWQQGVB3KVOY2M3/
[4]:
https://mail.python.org/archives/list/python-dev@python.org/thread/N4MMHXXOAL77UJLO264PCTCRJLAIKBC6/#N4MMHXXOAL77UJLO264PCTCRJLAIKBC6
Re: Ideas for improving the contribution experience [ In reply to ]
I think the other way to help is to really lean into automation so
reviewing is even lighterweight than it is now. Now this can be as simple
as to remind people when they need to regenerate a file like 'configure'
via a status check, simply telling people when their PR failed a status
check, or go as far as try to automate PEP 7/8 stuff.

Or another way to put it is to try and use automation to help make PRs more
self-service so that there's less required from a human being to do.

On Fri, Oct 16, 2020 at 3:31 PM Tal Einat <taleinat@gmail.com> wrote:

> (Context: Continuing to prepare for the core dev sprint next week. Since
> the sprint is near, *I'd greatly appreciate any quick comments, feedback
> and ideas!*)
>
> Following up my collection of past beginning contributor experiences, I've
> collected these experiences in a dedicated GitHub repo[1] and written a
> (subjective!) summary of main themes that I recognize in the stories, which
> I've also included in the repo[2].
>
> A "TL;DR" bullet list of those main themes:
> * Slow/no responsiveness
> * Long, slow process
> * Hard to find where to contribute
> * Mentorship helps a lot, but is scarce
> * A lot to learn to get started
> * It's intimidating
>
> More specifically, something that has come up often is that maintaining
> momentum for new contributors is crucial for them to become long-term
> contributors. Most often, this comes up in relation to the first two
> points: Suggestions or PRs are completely receive no attention at all
> ("ignored") or stop receiving attention at some point ("lost to the void").
> Unfortunately, the probability of this is pretty high for any issue/PR, so
> for a new contributor this is almost guaranteed to happen while working on
> one of their first few contributions. I've seen this happen many times, and
> have found that I have to personally follow promising contributors' work to
> ensure that this doesn't happen to them. I've also seen contributors learn
> to actively seek out core devs when these situations arise, which is often
> a successful tactic, but shouldn't be necessary so often.
>
> Now, this is in large part a result of the fact that us core devs are not
> a very large group, made up almost entirely of volunteers working on this
> in their spare time. Last I checked, the total amount of paid development
> time dedicated to developing Python is less than 3 full-time (i.e. ~100
> hours a week).
>
> The situation being problematic is clear enough that the PSF had concrete
> plans to hire paid developers to review issues and PRs. However, those
> plans have been put on hold indefinitely, since the PSF's funding has
> shrunk dramatically since the COVID-19 outbreak (no PyCon!).
>
> So, what can be done? Besides raising more funds (see a note on this
> below), I think we can find ways to reduce how often issues/PRs become
> "stalled". Here are some ideas:
>
> 1. *Generate reminders for reviewers when an issue or PR becomes
> "stalled' due to them.* Personally, I've found that both b.p.o. and
> GitHub make it relatively hard to remember to follow up on all of the many
> issues/PRs you've taken part in reviewing. It takes considerable attention
> and discipline to do so consistently, and reminders like these would have
> helped me. Many (many!) times, all it took to get an issue/PR moving
> forward (or closed) was a simple "ping?" comment.
>
> 2. *Generate reminders for contributors when an issue or PR becomes
> "stalled" due to them.* Similar to the above, but I consider these
> separate.
>
> 3. *Advertise something like a "2-for-1" standing offer for reviews.*
> This would give contributors an "official", acceptable way to get attention
> for their issue/PR, other than "begging" for attention on a mailing list.
> There are good ways for new contributors to be of significant help despite
> being new to the project, such as checking whether old bugs are still
> relevant, searching for duplicate issues, or applying old patches to the
> current code and creating a PR. (This would be similar to Martin v. Löwis's
> 5-for-1 offer in 2012[3], which had little success but lead to some
> interesting followup discussion[4]).
>
> 4. *Encourage core devs to dedicate some of their time to working through
> issues/PRs which are "ignored" or "stalled".* This would require first
> generating reliable lists of issues/PRs in such states. This could be in
> various forms, such as predefined GitHub/b.p.o. queries, a dedicated
> web-page, a periodic message similar to b.p.o.'s "weekly summary" email, or
> dedicated tags/labels for issues/PRs. (Perhaps prioritize "stalled" over
> "ignored".)
>
> - Tal Einat
>
>
> [1]: https://github.com/taleinat/python-contribution-feedback
> [2]:
> https://github.com/taleinat/python-contribution-feedback/blob/master/Takeaways%20-%20October%202020.md
> [3]:
> https://mail.python.org/archives/list/python-dev@python.org/message/7DLUN4Y7P77BSDW5YRWQQGVB3KVOY2M3/
> [4]:
> https://mail.python.org/archives/list/python-dev@python.org/thread/N4MMHXXOAL77UJLO264PCTCRJLAIKBC6/#N4MMHXXOAL77UJLO264PCTCRJLAIKBC6
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/AYSE65DUJFMFY5F7GJZSZDD252UWQENE/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
Re: Ideas for improving the contribution experience [ In reply to ]
On Mon, Oct 19, 2020 at 9:24 PM Brett Cannon <brett@python.org> wrote:

> I think the other way to help is to really lean into automation so
> reviewing is even lighterweight than it is now. Now this can be as simple
> as to remind people when they need to regenerate a file like 'configure'
> via a status check, simply telling people when their PR failed a status
> check, or go as far as try to automate PEP 7/8 stuff.
>
> Or another way to put it is to try and use automation to help make PRs
> more self-service so that there's less required from a human being to do.
>

A huge +1 from me!

Besides efficiency, I like that kind of automation because it makes our
checking of these things more consistent and official. Another reason is
that it allows contributors to be more independent in their work, rather
than waiting for someone else to explain what they have to do.

I've avoided mentioning code style until now since I remembered it being a
bit of a touchy subject a few years ago when I recall it coming up.
Personally I think that the benefits of being able to automate styling and
style checking far outweigh the negatives that were brought up at the time.
Brett, if you'd like to open that can of worms, I'd be willing to help.

- Tal
Re: Ideas for improving the contribution experience [ In reply to ]
On Mon., Oct. 19, 2020, 12:33 Tal Einat, <taleinat@gmail.com> wrote:

> On Mon, Oct 19, 2020 at 9:24 PM Brett Cannon <brett@python.org> wrote:
>
>> I think the other way to help is to really lean into automation so
>> reviewing is even lighterweight than it is now. Now this can be as simple
>> as to remind people when they need to regenerate a file like 'configure'
>> via a status check, simply telling people when their PR failed a status
>> check, or go as far as try to automate PEP 7/8 stuff.
>>
>> Or another way to put it is to try and use automation to help make PRs
>> more self-service so that there's less required from a human being to do.
>>
>
> A huge +1 from me!
>
> Besides efficiency, I like that kind of automation because it makes our
> checking of these things more consistent and official. Another reason is
> that it allows contributors to be more independent in their work, rather
> than waiting for someone else to explain what they have to do.
>
> I've avoided mentioning code style until now since I remembered it being a
> bit of a touchy subject a few years ago when I recall it coming up.
> Personally I think that the benefits of being able to automate styling and
> style checking far outweigh the negatives that were brought up at the time.
> Brett, if you'd like to open that can of worms, I'd be willing to help.
>


I have too many cans open right now to pop open another one. ????

-Brett


> - Tal
>
Re: Ideas for improving the contribution experience [ In reply to ]
Hi Tal,

Thanks for driving such research, it's really interesting. Sadly, I'm
not surprised by the results, since I'm already aware of these issues.

IMHO the most promising way to get more review is to create groups
where developers work on the same topic and review each other.

Also, mentoring is great! I advice core developers to delegate their
"boring tasks" to people willing to help. Many contributors are eager
to take these new responsibilities. Delegating shortly gives more time
to work on other topics where rare skills are needed.


> * Hard to find where to contribute

I believe that https://devguide.python.org/ could be enhanced to
better explain this part. Here are my notes on this question:

"Contribute to CPython: where should I start?"
https://cpython-core-tutorial.readthedocs.io/en/latest/where_should_i_start.html


> * Mentorship helps a lot, but is scarce

Over the last 2 years, I mentored different contributors, some of them
have been promoted as core developers, some of them moved to another
project, some of them continue to contribute as contributors.
Mentoring really accelerates the training to learn the most critical
things that a developer must know about the Python development
workflow, but also to build a trust relationship.

I'm trying to communicate about mentorship and my success stories, but
I chose to not communicate about people that I'm mentoring since it
would put too much pressure on them. It's perfectly fine if a mentee
doesn't become a core dev.

The difficult part is to stop doing simple tasks and delegate them.
It's difficult since mentoring does take more time in the short term,
reviews require a lot of back-and-forth exchanges to reach the
expected code quality. It's hard to foresee the benefit in the long
term.

In the past, I complained a lot about failing buildbots, the
maintenance burden of the servers and analyzing every single failure.
Thanks to mentoring, Pablo decided to help and I'm now really happy to
distribute the workload with him!


For one year, I'm working on https://bugs.python.org/issue1635741
which is a giant project (issue open for 13 years, tons of sub-issues,
tons of commits). The idea is to modernize the code base of the 118 C
extension modules of the Python standard library: convert static types
to heap types, and convert extensions to the multi-phase
initialization API (PEP 489).

Required changes are complex and must be reviewed carefully. Also, a
part of this work is boring since it's always the same things (repeat
the same patterns). Dong-hee Na, Hai Shi, Mohamed Koubaa are
collaborating with me, and we made significant progress in one year.
When one of them writes a PR, I ask the others to review it.

Since we are a small task force working on the same topic for one
year, slowly everyone is able to spot the common mistakes. For
example, forget to DECREF the type in the dealloc function when a
static type becomes a heap type. I trust them and so it puts less
pressure on my shoulders when I have to merge a PR. I also spend less
time on reviewing, and no time on writing changes since others are
writing these changes!

IMO this collaboration is a success since the scope is limited, it's
possible to work on a single file per PR (or even write multiple PRs
per file). While changes are complex, once you understand what should
be done, it's mostly a simple task of replacing a pattern with another
(ok, it's a little bit more complex than that in practice ;-)).


Le sam. 17 oct. 2020 à 00:30, Tal Einat <taleinat@gmail.com> a écrit :
> 1. Generate reminders for reviewers (...)

The root issue is the number of reviewers. Right now, it's mostly core
devs who review changes. I'm not sure that forcing core developers
will solve the root issue.


> 3. Advertise something like a "2-for-1" standing offer for reviews.

Right. That sounds like a better long term solution.

We need to encourage contributors to review pull requests written by
other contributors in similar areas.

As I wrote, when I read a review from a contributor, even if they are
a complete stranger to me, I'm more confident in reviewing a PR, since
I expect that the first review layers have been checked.


> 4. Encourage core devs to dedicate some of their time to working through issues/PRs which are "ignored" or "stalled".

I ignore many areas of Python on purpose. I don't have skills in these
areas or simply no interest.

I have a different idea. What about organizing weeks of review where
we focus on a small set of topics. For example, organize an "IDLE"
week where we invite everybody to review IDLE issues, triage IDLE
bugs, or maybe write new PRs. To be efficient, we need core developers
involved in the loop to merge PRs, core devs expert in this topic.
Otherwise, we cannot reduce the number of open PRs.

I propose a whole week since reviews always need many back-and-forth.
Such week might be the opportunity for more information discussions on
a dedicated chat where people can ask any question, especially dumb
questions.

I'm a volunteer to help for such a week, but don't expect me to be
available more than 1h per day, and only from monday to friday (so 5
days, 1h per day). Another idea would be to open a video call where I
share my screen to share what I'm doing, but I'm not sure if it's
interesting :-)

Example of topics:

* Review bpo-1635741 PRs and write new PRs (need C skills)
* Investigate test_asyncio random failures on buildbots

Sometimes, such collaboration doesn't produce any "value" immediately,
but it always helps people to meet each other and feel less shy to ask
questions and reach out to other people.

What do you think?

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LQKMCMN4ZWXOONZCPKVMY4ZYV6EIOL2Y/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: Ideas for improving the contribution experience [ In reply to ]
On Sat, Oct 17, 2020 at 7:31 AM Tal Einat <taleinat@gmail.com> wrote:
>
>
> 4. Encourage core devs to dedicate some of their time to working through issues/PRs which are "ignored" or "stalled". This would require first generating reliable lists of issues/PRs in such states. This could be in various forms, such as predefined GitHub/b.p.o. queries, a dedicated web-page, a periodic message similar to b.p.o.'s "weekly summary" email, or dedicated tags/labels for issues/PRs. (Perhaps prioritize "stalled" over "ignored".)
>

I looked random pull requests during the core sprint and I have merged
dozen pull requests.
But it is difficult to find any pull requests I can do anything. See this list:

open + awaiting merge: https://github.com/python/cpython/labels/awaiting%20merge
open + awaiting core review:
https://github.com/python/cpython/labels/awaiting%20merge

* Some PR is "owned" by some core developer already. We just need to
wait them. (Although assignee is not set in most case)
* Some PR is under discussion in b.p.o. issue.
* Some PR requires expertise at some areas.

etc...

Significant effort is required to find PRs I can help. My ideas:

* Recommend to set "assignee" more aggressive.
* "Assign me" when you will review/work on it later.
* Check "assign to me" when you have time.
* Feel free to un-assign.
* Add "waiting who's help" in the issue title.
* E.g. "needs compiler experts", "xml", "http", "mail", etc...

It will help to find PRs I can help in my spare time.

Regards,



--
Inada Naoki <songofacandy@gmail.com>
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/U66XTG6YWPWRQZMFVRQJ3PPLPWWCEEKR/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: Ideas for improving the contribution experience [ In reply to ]
I was impressed by the good ideas put forward, however, some what not full solutions.
My position: I have free time now and I want to contribute to open source/freeware. But I don’t have great software in depth experience, however, I’m willing to learn.
I was think of contributing to Python but I can't seem to see exactly where I can contribute, except for the obvious place: docs. I have persevered reading the bug list, emails and I find this helpful after discovering https://cpython-core-tutorial.readthedocs.io/en/latest/where_should_i_start.html
Where I will see if I can progress, but I don’t have great hopes.

What I see is that CPython is written at the highest level of software development, and found my own experience is difficult to get started, because the bar is so high. It seems that you are having difficulty recruiting enough/good developers?
So if you are honest about wanting to get new people to help then you need make an investment to make it happen.
My suggestion: create the developers by creating group(s) of mentors from across the whole team and actively train newcomers under a formal plan. The format/program of course must suit the mentors and mentee. This will mean less time for contributing to Python development but it will provide for a steady stream of people to add to the team, possibly saving time later on, or adding further content.

Regards,
Mike McLeod
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RZ3G2DCO3J4LLX6N2GECF3ZPFKQVLAP5/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: Ideas for improving the contribution experience [ In reply to ]
> My suggestion: create the developers by creating group(s) of
> mentors

Do you know about

https://mail.python.org/mailman3/lists/core-mentorship.python.org/

and if you do, why isn't that what you suggest?
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EC4OGSZGEX3C46LCDMBIHBGTYQJBZK7Y/
Code of Conduct: http://python.org/psf/codeofconduct/
Re: Ideas for improving the contribution experience [ In reply to ]
On 10/16/20 3:29 PM, Tal Einat wrote:
> (Context: Continuing to prepare for the core dev sprint next week.
> Since the sprint is near, *I'd greatly appreciate any quick comments,
> feedback and ideas!*)
>
> Following up my collection of past beginning contributor experiences,
> I've collected these experiences in a dedicated GitHub repo[1] and
> written a (subjective!) summary of main themes that I recognize in the
> stories, which I've also included in the repo[2].
>
> A "TL;DR" bullet list of those main themes:
> * Slow/no responsiveness
> * Long, slow process
> * Hard to find where to contribute
> * Mentorship helps a lot, but is scarce
> * A lot to learn to get started
> * It's intimidating
>
> More specifically, something that has come up often is that
> maintaining momentum for new contributors is crucial for them to
> become long-term contributors. Most often, this comes up in relation
> to the first two points: Suggestions or PRs are completely receive no
> attention at all ("ignored") or stop receiving attention at some point
> ("lost to the void"). Unfortunately, the probability of this is pretty
> high for any issue/PR, so for a new contributor this is almost
> guaranteed to happen while working on one of their first few
> contributions. I've seen this happen many times, and have found that I
> have to personally follow promising contributors' work to ensure that
> this doesn't happen to them. I've also seen contributors learn to
> actively seek out core devs when these situations arise, which is
> often a successful tactic, but shouldn't be necessary so often.
>
> Now, this is in large part a result of the fact that us core devs are
> not a very large group, made up almost entirely of volunteers working
> on this in their spare time. Last I checked, the total amount of paid
> development time dedicated to developing Python is less than 3
> full-time (i.e. ~100 hours a week).
>
> The situation being problematic is clear enough that the PSF had
> concrete plans to hire paid developers to review issues and PRs.
> However, those plans have been put on hold indefinitely, since the
> PSF's funding has shrunk dramatically since the COVID-19 outbreak (no
> PyCon!).
>
> So, what can be done? Besides raising more funds (see a note on this
> below), I think we can find ways to reduce how often issues/PRs become
> "stalled". Here are some ideas:
>
> 1. *Generate reminders for reviewers when an issue or PR becomes
> "stalled' due to them.* Personally, I've found that both b.p.o. and
> GitHub make it relatively hard to remember to follow up on all of the
> many issues/PRs you've taken part in reviewing. It takes considerable
> attention and discipline to do so consistently, and reminders like
> these would have helped me. Many (many!) times, all it took to get an
> issue/PR moving forward (or closed) was a simple "ping?" comment.
>
> 2. *Generate reminders for contributors when an issue or PR becomes
> "stalled" due to them.* Similar to the above, but I consider these
> separate.
>
> 3. *Advertise something like a "2-for-1" standing offer for reviews.*
> This would give contributors an "official", acceptable way to get
> attention for their issue/PR, other than "begging" for attention on a
> mailing list. There are good ways for new contributors to be of
> significant help despite being new to the project, such as checking
> whether old bugs are still relevant, searching for duplicate issues,
> or applying old patches to the current code and creating a PR. (This
> would be similar to Martin v. Löwis's 5-for-1 offer in 2012[3], which
> had little success but lead to some interesting followup discussion[4]).
>
> 4. *Encourage core devs to dedicate some of their time to working
> through issues/PRs which are "ignored" or "stalled".* This would
> require first generating reliable lists of issues/PRs in such states.
> This could be in various forms, such as predefined GitHub/b.p.o.
> queries, a dedicated web-page, a periodic message similar to b.p.o.'s
> "weekly summary" email, or dedicated tags/labels for issues/PRs.
> (Perhaps prioritize "stalled" over "ignored".)
>
> - Tal Einat


Tal, this would be great.

Is it possible to get a ballpark idea on how long a PR should/would take
to be merged so that contributors know if they should ping or not?

-Alfred