Mailing List Archive

RFC: Documenting changes in the CHANGES file
Reviewing our backport process I noticed that in many cases a clean merge via svn merge fails due to conflicts in CHANGES. While
these are easy to solve it puts IMHO unnecessary extra work on the backport process, both for reviewing and for actually doing the
backport. How about if we change the way we document changes the following way:

1. We create a changes-fragments directory (name to be determined) at the top level.
2. For each release we create a subdirectory such that we end up with the following structure:

changes-fragments/
2.4.41/
2.4.42/
2.4.43/
2.4.44/

3. Each directory contains the changes for each release and each change entry is a single file.
4. We have a script that builds our current CHANGES file from the content in changes-fragments directories with the help of
a template or at least some sort of header / footer that is static.
5. This script can be called either manually and we commit the resulting CHANGES file as we like just like the x-forms commits
for documentation plus this script is called by the release scripts from Daniel as part of the preparation of rolling a
release.


Regards

Rüdiger
Re: RFC: Documenting changes in the CHANGES file [ In reply to ]
On Fri, May 29, 2020 at 9:30 PM Ruediger Pluem <rpluem@apache.org> wrote:
>
> Reviewing our backport process I noticed that in many cases a clean merge via svn merge fails due to conflicts in CHANGES. While
> these are easy to solve it puts IMHO unnecessary extra work on the backport process, both for reviewing and for actually doing the
> backport. How about if we change the way we document changes the following way:
>
> 1. We create a changes-fragments directory (name to be determined) at the top level.
> 2. For each release we create a subdirectory such that we end up with the following structure:
>
> changes-fragments/
> 2.4.41/
> 2.4.42/
> 2.4.43/
> 2.4.44/
>
> 3. Each directory contains the changes for each release and each change entry is a single file.
> 4. We have a script that builds our current CHANGES file from the content in changes-fragments directories with the help of
> a template or at least some sort of header / footer that is static.
> 5. This script can be called either manually and we commit the resulting CHANGES file as we like just like the x-forms commits
> for documentation plus this script is called by the release scripts from Daniel as part of the preparation of rolling a
> release.

+1 from me, I don't volonteer for the scripts though :)

Regards;
Yann.
Re: RFC: Documenting changes in the CHANGES file [ In reply to ]
Works for me.

> On May 29, 2020, at 3:30 PM, Ruediger Pluem <rpluem@apache.org> wrote:
>
> Reviewing our backport process I noticed that in many cases a clean merge via svn merge fails due to conflicts in CHANGES. While
> these are easy to solve it puts IMHO unnecessary extra work on the backport process, both for reviewing and for actually doing the
> backport. How about if we change the way we document changes the following way:
>
> 1. We create a changes-fragments directory (name to be determined) at the top level.
> 2. For each release we create a subdirectory such that we end up with the following structure:
>
> changes-fragments/
> 2.4.41/
> 2.4.42/
> 2.4.43/
> 2.4.44/
>
> 3. Each directory contains the changes for each release and each change entry is a single file.
> 4. We have a script that builds our current CHANGES file from the content in changes-fragments directories with the help of
> a template or at least some sort of header / footer that is static.
> 5. This script can be called either manually and we commit the resulting CHANGES file as we like just like the x-forms commits
> for documentation plus this script is called by the release scripts from Daniel as part of the preparation of rolling a
> release.
>
>
> Regards
>
> Rüdiger
Re: RFC: Documenting changes in the CHANGES file [ In reply to ]
On 29 May 2020, at 21:30, Ruediger Pluem <rpluem@apache.org> wrote:

> Reviewing our backport process I noticed that in many cases a clean merge via svn merge fails due to conflicts in CHANGES. While
> these are easy to solve it puts IMHO unnecessary extra work on the backport process, both for reviewing and for actually doing the
> backport. How about if we change the way we document changes the following way:
>
> 1. We create a changes-fragments directory (name to be determined) at the top level.
> 2. For each release we create a subdirectory such that we end up with the following structure:
>
> changes-fragments/
> 2.4.41/
> 2.4.42/
> 2.4.43/
> 2.4.44/
>
> 3. Each directory contains the changes for each release and each change entry is a single file.
> 4. We have a script that builds our current CHANGES file from the content in changes-fragments directories with the help of
> a template or at least some sort of header / footer that is static.
> 5. This script can be called either manually and we commit the resulting CHANGES file as we like just like the x-forms commits
> for documentation plus this script is called by the release scripts from Daniel as part of the preparation of rolling a
> release.

I’m keen for a simpler version of this that doesn't create additional steps for people.

How about something like this:

1. We create a changes-fragments directory (name to be determined) at the top level.
2. The changes-fragments directory contains the changes for each release and each change entry is a single file.
3. We update the Makefiles so that if any files are found to exist in changes-fragments, those files are moved to the top of CHANGES and then deleted. For all builds except for backports, this will be a silent noop. We would need some kind of token to force it to be a non-noop in release branches.

What this means is - after every backport, which won’t conflict because changes is in a unique file as you describe above, the person performs a build to verify that the change is ok. That build automatically sorts out the CHANGES file, and the attempt to commit the backport updates changes.

To summarise, with the above, people put changes in changes-fragments directory, and that’s it. Everything else is automated as part of the normal build, no special scripts, no extra steps, nothing for humans to forget.

Regards,
Graham
Re: RFC: Documenting changes in the CHANGES file [ In reply to ]
On Fri, May 29, 2020 at 3:30 PM Ruediger Pluem <rpluem@apache.org> wrote:
>
> Reviewing our backport process I noticed that in many cases a clean merge via svn merge fails due to conflicts in CHANGES. While
> these are easy to solve it puts IMHO unnecessary extra work on the backport process, both for reviewing and for actually doing the
> backport. How about if we change the way we document changes the following way:
>
> 1. We create a changes-fragments directory (name to be determined) at the top level.
> 2. For each release we create a subdirectory such that we end up with the following structure:
>
> changes-fragments/
> 2.4.41/
> 2.4.42/
> 2.4.43/
> 2.4.44/
>
> 3. Each directory contains the changes for each release and each change entry is a single file.
> 4. We have a script that builds our current CHANGES file from the content in changes-fragments directories with the help of
> a template or at least some sort of header / footer that is static.
> 5. This script can be called either manually and we commit the resulting CHANGES file as we like just like the x-forms commits
> for documentation plus this script is called by the release scripts from Daniel as part of the preparation of rolling a
> release.

If we removed or generalized the banner at the top of each changes,
and added extra whitespace between entries, would svn merge push stuff
to the top any better?
I don't know if it would hunt for more non-whitespace context to try
to figure out how to merge.
Re: RFC: Documenting changes in the CHANGES file [ In reply to ]
On 6/1/2020 6:23 AM, Yann Ylavic wrote:
> On Fri, May 29, 2020 at 9:30 PM Ruediger Pluem <rpluem@apache.org> wrote:
>> Reviewing our backport process I noticed that in many cases a clean merge via svn merge fails due to conflicts in CHANGES. While
>> these are easy to solve it puts IMHO unnecessary extra work on the backport process, both for reviewing and for actually doing the
>> backport. How about if we change the way we document changes the following way:
>>
>> 1. We create a changes-fragments directory (name to be determined) at the top level.
>> 2. For each release we create a subdirectory such that we end up with the following structure:
>>
>> changes-fragments/
>> 2.4.41/
>> 2.4.42/
>> 2.4.43/
>> 2.4.44/
>>
>> 3. Each directory contains the changes for each release and each change entry is a single file.
>> 4. We have a script that builds our current CHANGES file from the content in changes-fragments directories with the help of
>> a template or at least some sort of header / footer that is static.
>> 5. This script can be called either manually and we commit the resulting CHANGES file as we like just like the x-forms commits
>> for documentation plus this script is called by the release scripts from Daniel as part of the preparation of rolling a
>> release.
> +1 from me, I don't volonteer for the scripts though :)
>
> Regards;
> Yann.

Hi, Yann;

I'm open to whatever... and don't mind writing or tweaking scripts once
we decide on an approach :-)

While we are discussing ideas in this neighborhood, one thing to keep in
mind is that during release of security fixes, sometimes there are items
added to CHANGES and sometimes CHANGES is modified to add CVE
information. There have been minor bumps in the road where these patches
don't always apply cleanly. So, if possible, it would be great to
consider. There may be nothing to do, though, since that happens waaaaay
after backport.

--
Daniel Ruggeri
Re: RFC: Documenting changes in the CHANGES file [ In reply to ]
> Am 02.06.2020 um 14:11 schrieb Daniel Ruggeri <daniel@bitnebula.com>:
>
> On 6/1/2020 6:23 AM, Yann Ylavic wrote:
>> On Fri, May 29, 2020 at 9:30 PM Ruediger Pluem <rpluem@apache.org>
>> wrote:
>>
>>> Reviewing our backport process I noticed that in many cases a clean merge via svn merge fails due to conflicts in CHANGES. While
>>> these are easy to solve it puts IMHO unnecessary extra work on the backport process, both for reviewing and for actually doing the
>>> backport. How about if we change the way we document changes the following way:
>>>
>>> 1. We create a changes-fragments directory (name to be determined) at the top level.
>>> 2. For each release we create a subdirectory such that we end up with the following structure:
>>>
>>> changes-fragments/
>>> 2.4.41/
>>> 2.4.42/
>>> 2.4.43/
>>> 2.4.44/
>>>
>>> 3. Each directory contains the changes for each release and each change entry is a single file.
>>> 4. We have a script that builds our current CHANGES file from the content in changes-fragments directories with the help of
>>> a template or at least some sort of header / footer that is static.
>>> 5. This script can be called either manually and we commit the resulting CHANGES file as we like just like the x-forms commits
>>> for documentation plus this script is called by the release scripts from Daniel as part of the preparation of rolling a
>>> release.
>>>
>> +1 from me, I don't volonteer for the scripts though :)
>>
>> Regards;
>> Yann.
>>
> Hi, Yann;
>
> I'm open to whatever... and don't mind writing or tweaking scripts once we decide on an approach :-)
>
> While we are discussing ideas in this neighborhood, one thing to keep in mind is that during release of security fixes, sometimes there are items added to CHANGES and sometimes CHANGES is modified to add CVE information. There have been minor bumps in the road where these patches don't always apply cleanly. So, if possible, it would be great to consider. There may be nothing to do, though, since that happens waaaaay after backport.
>


+1 from me as well. CHANGES is annoying atm, any automation appreciated.

Cheers, Stefan
Re: RFC: Documenting changes in the CHANGES file [ In reply to ]
On 6/2/20 2:17 PM, Stefan Eissing wrote:
>
>> Am 02.06.2020 um 14:11 schrieb Daniel Ruggeri <daniel@bitnebula.com>:
>>
>> On 6/1/2020 6:23 AM, Yann Ylavic wrote:
>>> On Fri, May 29, 2020 at 9:30 PM Ruediger Pluem <rpluem@apache.org>
>>> wrote:
>>>
>>>> Reviewing our backport process I noticed that in many cases a clean merge via svn merge fails due to conflicts in CHANGES. While
>>>> these are easy to solve it puts IMHO unnecessary extra work on the backport process, both for reviewing and for actually doing the
>>>> backport. How about if we change the way we document changes the following way:
>>>>
>>>> 1. We create a changes-fragments directory (name to be determined) at the top level.
>>>> 2. For each release we create a subdirectory such that we end up with the following structure:
>>>>
>>>> changes-fragments/
>>>> 2.4.41/
>>>> 2.4.42/
>>>> 2.4.43/
>>>> 2.4.44/
>>>>
>>>> 3. Each directory contains the changes for each release and each change entry is a single file.
>>>> 4. We have a script that builds our current CHANGES file from the content in changes-fragments directories with the help of
>>>> a template or at least some sort of header / footer that is static.
>>>> 5. This script can be called either manually and we commit the resulting CHANGES file as we like just like the x-forms commits
>>>> for documentation plus this script is called by the release scripts from Daniel as part of the preparation of rolling a
>>>> release.
>>>>
>>> +1 from me, I don't volonteer for the scripts though :)
>>>
>>> Regards;
>>> Yann.
>>>
>> Hi, Yann;
>>
>> I'm open to whatever... and don't mind writing or tweaking scripts once we decide on an approach :-)
>>
>> While we are discussing ideas in this neighborhood, one thing to keep in mind is that during release of security fixes, sometimes there are items added to CHANGES and sometimes CHANGES is modified to add CVE information. There have been minor bumps in the road where these patches don't always apply cleanly. So, if possible, it would be great to consider. There may be nothing to do, though, since that happens waaaaay after backport.
>>
>
>
> +1 from me as well. CHANGES is annoying atm, any automation appreciated.
>

Thanks for all the feedback. I try to work out something more detailed aka patch that we can discuss then.

Regards

Rüdiger
Re: RFC: Documenting changes in the CHANGES file [ In reply to ]
> On 1 Jun 2020, at 13:33, Graham Leggett <minfrin@sharp.fm> wrote:
>
> On 29 May 2020, at 21:30, Ruediger Pluem <rpluem@apache.org> wrote:
>
>> changes-fragments/
>> 2.4.41/
>> 2.4.42/
>> 2.4.43/
>> 2.4.44/

And a current/ as symlink?

> I’m keen for a simpler version of this that doesn't create additional steps for people.

Not sure that's simpler, though it too has potential. Delete-after-append seems tidier
than keeping forever-fragments.

I wonder if this could be hooked into SVN? Something like an @CHANGES tag in
a svn commit message?

--
Nick Kew
Re: RFC: Documenting changes in the CHANGES file [ In reply to ]
On 6/8/20 10:20 AM, Ruediger Pluem wrote:
>

>>
>
> Thanks for all the feedback. I try to work out something more detailed aka patch that we can discuss then.
>

Done as r1879822. Happy to get some feedback.

Regards

Rüdiger