Mailing List Archive

Proposal: new structure for MediaWiki RELEASE-NOTES
Hello.

TLDR: In MW core, RELEASE-NOTES is a cause of the majority of merge conflicts.
I propose to restructure it in a way that could let us do less manual conflict resolutions in Gerrit.

The problem

When working on MediaWiki core, all of us often need to update RELEASE-NOTES-XX file
in case some deprecations or breaking changes were made. While the change is under review,
other changes touching RELEASE-NOTES-XX are made, causing merge conflicts that need
to be resolved manually. Non insignificant portion of manual conflict resolutions are needed
exclusively due to RELEASE-NOTES.

This is not a surprise due to the structure of the notes file. It has several sections, with every
section being a list of individual notes. Since most of us only append lines to the end of the list,
it creates a contention.

The proposal

I propose to reorganize RELEASE-NOTES. We keep the sections, but within each section
we start the note with the name of the class that’s been touched, if it makes sense.

For example, instead of writing

* In AbstractBlock, the getTargetAndType() and getTarget() methods are
hard deprecated. Instead use getTargetName() and getTargetUserIdentity()
together with getType().

We would write

* AbstractBlock: the following method were hard-deprecated:
- getTarget() - use getTargetName() or getTargetUserIdentity()
- getTargetAndType() - use getType() with getTargetUserIdentity() or getTargetName()

After switching to a more standardized format, we sort individual notes alphabetically by class
name. Method names within messages need to be sorted alphabetically as well. This will make
an insertion point of new release notes more random, hopefully reducing the contention and
thus reducing the probability of a merge conflict. Additionally, I personally think this will look nicer
and will be easier to read for extension developers.

Not all the release notes can follow this pattern, and it’s ok. The point is not to be perfect, the
point is to try to be better.

Alternative solutions

We could write a custom merge driver for RELEASE-NOTES which always puts ‘ours’ before ’theirs’,
but I’m not sure the result will justify the investment.

Feedback

Please tell me what you think about this.
- Do you think it is a real problem?
- Do you think simply restructuring the notes with help?
- Do you think writing a custom merge driver is a way to go?
- Will you remember to follow the new structure if we agree on it even though it’s not really possible
to enforce with linters since not all the notes can follow this structure?
- Extension developers - do you care? Do you think alphabetically sorted RELEASE-NOTES will be
any easier for you to read?

Thank you.
Petr Pchelko.
Staff Software Engineer.
WMF Platform Engineering Team.
Re: Proposal: new structure for MediaWiki RELEASE-NOTES [ In reply to ]
> We could write a custom merge driver for RELEASE-NOTES which always puts
‘ours’ before ’theirs’,
but I’m not sure the result will justify the investment.

Didnt someone already do that? There used to be a ruby script floating
around for that.

On Monday, July 19, 2021, Petr Pchelko <ppchelko@wikimedia.org> wrote:

> Hello.
>
> TLDR: In MW core, RELEASE-NOTES is a cause of the majority of merge
> conflicts.
> I propose to restructure it in a way that could let us do less manual
> conflict resolutions in Gerrit.
>
> *The problem*
>
> When working on MediaWiki core, all of us often need to update
> RELEASE-NOTES-XX file
> in case some deprecations or breaking changes were made. While the change
> is under review,
> other changes touching RELEASE-NOTES-XX are made, causing merge conflicts
> that need
> to be resolved manually. Non insignificant portion of manual conflict
> resolutions are needed
> exclusively due to RELEASE-NOTES.
>
> This is not a surprise due to the structure of the notes file. It has
> several sections, with every
> section being a list of individual notes. Since most of us only append
> lines to the end of the list,
> it creates a contention.
>
> *The proposal*
>
> I propose to reorganize RELEASE-NOTES. We keep the sections, but within
> each section
> we start the note with the name of the class that’s been touched, if it
> makes sense.
>
> For example, instead of writing
>
> * In AbstractBlock, the getTargetAndType() and getTarget() methods are
> hard deprecated. Instead use getTargetName() and getTargetUserIdentity()
> together with getType().
>
> We would write
>
> * AbstractBlock: the following method were hard-deprecated:
> - getTarget() - use getTargetName() or getTargetUserIdentity()
> - getTargetAndType() - use getType() with getTargetUserIdentity() or
> getTargetName()
>
> After switching to a more standardized format, we sort individual notes
> alphabetically by class
> name. Method names within messages need to be sorted alphabetically as
> well. This will make
> an insertion point of new release notes more random, hopefully reducing
> the contention and
> thus reducing the probability of a merge conflict. Additionally, I
> personally think this will look nicer
> and will be easier to read for extension developers.
>
> Not all the release notes can follow this pattern, and it’s ok. The point
> is not to be perfect, the
> point is to try to be better.
>
> *Alternative solutions*
>
> We could write a custom merge driver for RELEASE-NOTES which always puts
> ‘ours’ before ’theirs’,
> but I’m not sure the result will justify the investment.
>
> *Feedback*
>
> Please tell me what you think about this.
> - Do you think it is a real problem?
> - Do you think simply restructuring the notes with help?
> - Do you think writing a custom merge driver is a way to go?
> - Will you remember to follow the new structure if we agree on it even
> though it’s not really possible
> to enforce with linters since not all the notes can follow this structure?
> - Extension developers - do you care? Do you think alphabetically sorted
> RELEASE-NOTES will be
> any easier for you to read?
>
> Thank you.
> Petr Pchelko.
> Staff Software Engineer.
> WMF Platform Engineering Team.
>
>
>
>
Re: Proposal: new structure for MediaWiki RELEASE-NOTES [ In reply to ]
On Tue, Jul 20, 2021 at 3:38 AM Petr Pchelko <ppchelko@wikimedia.org> wrote:

> *Alternative solutions*
>
> We could write a custom merge driver for RELEASE-NOTES which always puts
> ‘ours’ before ’theirs’,
> but I’m not sure the result will justify the investment.
>

Probably overkill for MediaWiki but I'd like to mention the way that Python
developers (CPython) manages their release notes, in case it might be
useful.
The TL;DR is that each change has a unique valid reStructuredText file in a
specific directory and then there is a tool to merge all changes when a
release is made.

The full process from a contributor point of view is described in [1].
The tool used to both generate the change files and merge them into a
release file is [2].

[1]
https://devguide.python.org/committing/#updating-news-and-what-s-new-in-python
[2] https://pypi.org/project/blurb/
Re: Proposal: new structure for MediaWiki RELEASE-NOTES [ In reply to ]
Le 20/07/2021 à 09:48, Riccardo Coccioli a écrit :
> On Tue, Jul 20, 2021 at 3:38 AM Petr Pchelko <ppchelko@wikimedia.org
> <mailto:ppchelko@wikimedia.org>> wrote:
>
> *Alternative solutions*
>
> We could write a custom merge driver for RELEASE-NOTES which
> always puts ‘ours’ before ’theirs’,
> but I’m not sure the result will justify the investment.
>
>
> Probably overkill for MediaWiki but I'd like to mention the way that
> Python developers (CPython) manages their release notes, in case it
> might be useful.
> The TL;DR is that each change has a unique valid reStructuredText file
> in a specific directory and then there is a tool to merge all changes
> when a release is made.
>
> The full process from a contributor point of view is described in [1].
> The tool used to both generate the change files and merge them into a
> release file is [2].
>
> [1]
> https://devguide.python.org/committing/#updating-news-and-what-s-new-in-python
> <https://devguide.python.org/committing/#updating-news-and-what-s-new-in-python>
> [2] https://pypi.org/project/blurb/ <https://pypi.org/project/blurb/>

OpenStack has a similar tool: reno. The doc has an overview of the
requirements: https://docs.openstack.org/reno/latest/user/design.html
<https://docs.openstack.org/reno/latest/user/design.html> and the usage
doc for quick glance:
https://docs.openstack.org/reno/latest/user/usage.html
<https://docs.openstack.org/reno/latest/user/usage.html>

There is an a 30 mins presentation
https://www.youtube.com/watch?v=tEOGJ_h0Lx0
<https://www.youtube.com/watch?v=tEOGJ_h0Lx0>

Short of having to introduce a Python tool to the developers, maybe the
PHP ecosystem has a similar tool?  Or we can reach out to other high
traffic projects and see how they are managing their changelog and maybe
forge a common tool.

Antoine "hashar" Musso
Re: Proposal: new structure for MediaWiki RELEASE-NOTES [ In reply to ]
Le 20/07/2021 à 06:06, Brian Wolff a écrit :
> > We could write a custom merge driver for RELEASE-NOTES which always
> puts ‘ours’ before ’theirs’,
> but I’m not sure the result will justify the investment.
>
> Didnt someone already do that? There used to be a ruby script floating
> around for that.

The tool is: https://github.com/MatmaRex/mediawikireleasenotes-driver
<https://github.com/MatmaRex/mediawikireleasenotes-driver> created in
2013. It leads me to some material we wrote at that time when the issue
got first identified:

* 2013 RFC
https://www.mediawiki.org/wiki/Requests_for_comment/Release_notes_automation
<https://www.mediawiki.org/wiki/Requests_for_comment/Release_notes_automation>

* 2018 task (still open) https://phabricator.wikimedia.org/T200392
<https://phabricator.wikimedia.org/T200392>

If we go with a merge driver, it would needs to be installed on CI
(zuul-merger does use cgit) and on Gerrit (which uses jGit).

Antoine "hashar" Musso
Re: Proposal: new structure for MediaWiki RELEASE-NOTES [ In reply to ]
On Tue, Jul 20, 2021 at 5:22 AM Antoine Musso <hashar@free.fr> wrote:
>
> Le 20/07/2021 à 09:48, Riccardo Coccioli a écrit :
>
> On Tue, Jul 20, 2021 at 3:38 AM Petr Pchelko <ppchelko@wikimedia.org> wrote:
>>
>> Alternative solutions
>>
>> We could write a custom merge driver for RELEASE-NOTES which always puts ‘ours’ before ’theirs’,
>> but I’m not sure the result will justify the investment.
>
>
> Probably overkill for MediaWiki but I'd like to mention the way that Python developers (CPython) manages their release notes, in case it might be useful.
> The TL;DR is that each change has a unique valid reStructuredText file in a specific directory and then there is a tool to merge all changes when a release is made.
>
> The full process from a contributor point of view is described in [1].
> The tool used to both generate the change files and merge them into a release file is [2].
>
> [1] https://devguide.python.org/committing/#updating-news-and-what-s-new-in-python
> [2] https://pypi.org/project/blurb/
>
> OpenStack has a similar tool: reno. The doc has an overview of the requirements: https://docs.openstack.org/reno/latest/user/design.html and the usage doc for quick glance: https://docs.openstack.org/reno/latest/user/usage.html
>
> There is an a 30 mins presentation https://www.youtube.com/watch?v=tEOGJ_h0Lx0
>
> Short of having to introduce a Python tool to the developers, maybe the PHP ecosystem has a similar tool? Or we can reach out to other high traffic projects and see how they are managing their changelog and maybe forge a common tool.

https://github.com/Automattic/jetpack-changelogger is a similar tool
to reno that is pure php and composer installable. Some of you may
recognize the primary author too if you check the git history there.

Bryan
--
Bryan Davis Technical Engagement Wikimedia Foundation
Principal Software Engineer Boise, ID USA
[[m:User:BDavis_(WMF)]] irc: bd808
_______________________________________________
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/