Mailing List Archive

release scripts
Having a look at the release scripts and how they work.

I think we can use SVN and its branches/tags in a bit smarter way
and make things easier. As Christophe mentioned, he would like a
"--dry-run" option, because when things do not run smoothly to the
end, you have not only a local messed-up state but the SVN repro
itself is also changed without easy remedies.

I propose to do all work up to release testing in separate,
temporary branches that can simply be removed and restarted.
This would not require a rewrite of the release scripts, but
some re-ordering and branch name changes.

- checkout branches/2.4.x
- make sure change-entries are merged, commit
- test branch locally for health
- svn cp . branches/2.4.49-tmp
- svn switch to branches/2.4.49-tmp
- update versions, build docs, etc on that
- commit this into branches/2.4.49-tmp
- svn mv branches/2.4.49-tmp branches/2.4.49-candidate
- svn switch to branches/2.4.49-candidate
- create tar files and signature from -candidate
- make them available for testing
- announce release voting
#
# everything up to here can be reverted by "svn rm"
#
- switch to branches/2.4.x
- bump versions in branches/2.4.x to 2.4.50

- Stefan

-------------------------------------------------------------------------
Summary of what the release process involves and what each script does:
0. tag.sh
svn checkout $HEAD of branch
check version numbers
check SVN tag for version does not exist
check PGP/GPG key in httpd-dist/KEYS
update Copyright in docs/manual/style/xsl/common.xsl
checkout docs-build, build docs, remove docs-build checkout
run "update-changes" using Makfile.in
Set AP_SERVER_DEVBUILD_BOOLEAN to 0 in include/ap_release.h
svn commit all changes
Change local include/ap_release.h back
bump up patch version in CHANGES, STATUS, docs/manual/style/version.ent
before committing this, copy $HEAD of branch to tags/$version
svn commit the bump-up changes to branch
1a. release.sh
svn export $HEAD of branches/2.4.x
svn export of dependencies (APR)
apply patches (currently broken)
prepare CHANGES_<version>
invoke roll.sh
1b. roll.sh
run releasecheck.sh (checking libtool availability? for what?)
remove files not wanted in release (fixed list)
touch .y dependants (fixed list)
remove doc sources (*.xml *.xsl)
move srclib into -deps sibling directory
build tar balls
build checksum files (lots of alternatives if openssl is not found, do we need that?)
signing the files
substitute new version number in various files
2. voting
3. push.sh
checkout https://dist.apache.org/repos/dist/
remove old CHANGES_$major.minor
generate new CHANGES_$version, CHANGES_$major.minor
modify Announcement$major.minor
svn move generated tar, deps and CHANGES
svn commit
4. update bugzilla
5. announce.sh
lots of messing with dist and pmc to get security details in
some sanity checks
send mails
send more mails
svn commit changes to "$dist_base" "$branch_base" "$private_base" "$site_base"
Re: release scripts [ In reply to ]
On 9/1/21 12:08 PM, stefan@eissing.org wrote:
> Having a look at the release scripts and how they work.
>
> I think we can use SVN and its branches/tags in a bit smarter way
> and make things easier. As Christophe mentioned, he would like a
> "--dry-run" option, because when things do not run smoothly to the
> end, you have not only a local messed-up state but the SVN repro
> itself is also changed without easy remedies.
>
> I propose to do all work up to release testing in separate,
> temporary branches that can simply be removed and restarted.
> This would not require a rewrite of the release scripts, but
> some re-ordering and branch name changes.
>
> - checkout branches/2.4.x
> - make sure change-entries are merged, commit
> - test branch locally for health
> - svn cp . branches/2.4.49-tmp
> - svn switch to branches/2.4.49-tmp
> - update versions, build docs, etc on that
> - commit this into branches/2.4.49-tmp
> - svn mv branches/2.4.49-tmp branches/2.4.49-candidate
> - svn switch to branches/2.4.49-candidate
> - create tar files and signature from -candidate
> - make them available for testing
> - announce release voting
> #
> # everything up to here can be reverted by "svn rm"
> #
> - switch to branches/2.4.x
> - bump versions in branches/2.4.x to 2.4.50

Sounds good. One addition on this flow: I would like to see the build docs results finding their way back into the 2.4.x branch.
Probably by redoing build docs after switching back to branches 2.4 in the last step.

Regards

RĂ¼diger
Re: release scripts [ In reply to ]
> check PGP/GPG key in httpd-dist/KEYS

This reminds me, when I looked the other day you did not yet have an
entry in KEYS (svn co https://dist.apache.org/repos/dist/release/httpd
httpd-dist)
Re: release scripts [ In reply to ]
On Wed, Sep 1, 2021 at 6:08 AM stefan@eissing.org <stefan@eissing.org> wrote:
>
> Having a look at the release scripts and how they work.
>
> I think we can use SVN and its branches/tags in a bit smarter way
> and make things easier. As Christophe mentioned, he would like a
> "--dry-run" option, because when things do not run smoothly to the
> end, you have not only a local messed-up state but the SVN repro
> itself is also changed without easy remedies.

LGTM. Maybe even separate -tmp and -candidate branches aren't needed?
It's all scratch space for the RM.
Does the release branch get moved to a tag at the end?
Re: release scripts [ In reply to ]
> Am 01.09.2021 um 13:15 schrieb Eric Covener <covener@gmail.com>:
>
> On Wed, Sep 1, 2021 at 6:08 AM stefan@eissing.org <stefan@eissing.org> wrote:
>>
>> Having a look at the release scripts and how they work.
>>
>> I think we can use SVN and its branches/tags in a bit smarter way
>> and make things easier. As Christophe mentioned, he would like a
>> "--dry-run" option, because when things do not run smoothly to the
>> end, you have not only a local messed-up state but the SVN repro
>> itself is also changed without easy remedies.
>
> LGTM. Maybe even separate -tmp and -candidate branches aren't needed?

We can do without, yes. Seems more simple that way.

> It's all scratch space for the RM.
> Does the release branch get moved to a tag at the end?

Yes, that line was missing, I saw too late and did not
want to add another mail to my wall of text. So

> svn mv branches/2.4.49-candidate tags/2.4.49

would be the last thing at the voting.
Re: release scripts [ In reply to ]
OK, small hickup. The build shell for the manuals wants a JDK 1.8 or 11...
uhm...I need a time machine, or we can change the check there.

Since I am not experienced in our manual stuff, can someone knowledgable check this?

- Stefan

> Am 01.09.2021 um 13:27 schrieb stefan@eissing.org:
>
>
>
>> Am 01.09.2021 um 13:15 schrieb Eric Covener <covener@gmail.com>:
>>
>> On Wed, Sep 1, 2021 at 6:08 AM stefan@eissing.org <stefan@eissing.org> wrote:
>>>
>>> Having a look at the release scripts and how they work.
>>>
>>> I think we can use SVN and its branches/tags in a bit smarter way
>>> and make things easier. As Christophe mentioned, he would like a
>>> "--dry-run" option, because when things do not run smoothly to the
>>> end, you have not only a local messed-up state but the SVN repro
>>> itself is also changed without easy remedies.
>>
>> LGTM. Maybe even separate -tmp and -candidate branches aren't needed?
>
> We can do without, yes. Seems more simple that way.
>
>> It's all scratch space for the RM.
>> Does the release branch get moved to a tag at the end?
>
> Yes, that line was missing, I saw too late and did not
> want to add another mail to my wall of text. So
>
>> svn mv branches/2.4.49-candidate tags/2.4.49
>
> would be the last thing at the voting.
Re: release scripts [ In reply to ]
On Thu, Sep 2, 2021 at 8:21 AM stefan@eissing.org <stefan@eissing.org> wrote:
>
> OK, small hickup. The build shell for the manuals wants a JDK 1.8 or 11...
> uhm...I need a time machine, or we can change the check there.
>
> Since I am not experienced in our manual stuff, can someone knowledgable check this?

If it doesn't create a crazy diff from what's currently generated,
it's fine. Maybe time to strip out the check.
release scripts [ In reply to ]
After some learning experience (*cough*) I committed a new
version of the scripts, now in $DEV_TOOLS/release that go
so far as staging local changes to all repositories after
a successful vote.

I will test some more tomorrow and add the announce mailing
part. The real test will then come when actually doing a
release as testing is hard to cover all code paths otherwise.

I have silently sneaked in - as some fringe benefit to all
the pain - a '-rcN' suffix for vote candidates. Since we
will be able to cancel a release much later now without
modifying a source repository.

The -rcN tars for voting are named as such, but the directories
contained have the version without any suffix. So, a -rcN
tar can be renamed to the release tar without changes when
the vote has passed.


Cheers, Stefan
Re: release scripts [ In reply to ]
Awesome, great work Stefan, thanks a lot!

On Tue, Sep 7, 2021 at 4:22 PM stefan@eissing.org <stefan@eissing.org> wrote:
>
> After some learning experience (*cough*) I committed a new
> version of the scripts, now in $DEV_TOOLS/release that go
> so far as staging local changes to all repositories after
> a successful vote.
>
> I will test some more tomorrow and add the announce mailing
> part. The real test will then come when actually doing a
> release as testing is hard to cover all code paths otherwise.
>
> I have silently sneaked in - as some fringe benefit to all
> the pain - a '-rcN' suffix for vote candidates. Since we
> will be able to cancel a release much later now without
> modifying a source repository.
>
> The -rcN tars for voting are named as such, but the directories
> contained have the version without any suffix. So, a -rcN
> tar can be renamed to the release tar without changes when
> the vote has passed.
>
>
> Cheers, Stefan