Mailing List Archive: Publish PGP signed tarball without generated content?

Publish PGP signed tarball without generated content?

simon at josefsson

Apr 17, 2024, 1:35 AM

Post #1 of 9 (74 views)

Hi

What do you think about publishing PGP signed tarballs without the
generated files such as the ./configure script?

What I'm looking for is for some private key holder of the OpenSSH
portable release key to run

git checkout V_9_7_P1
git archive --prefix=openssh-portable-V_9_7_P1/ -o openssh-9.7p1-src.tar.gz HEAD
gpg --detach-sign --armor openssh-9.7p1-src.tar.gz

and then publish the resulting openssh-9.7p1-src.tar.gz and
openssh-9.7p1-src.tar.gz.asc files, preferably using a version of git
that leads to archives that are identical to what GitHub currently
publish.

The tarball would then be identical to what can (currently) be
downloaded from the GitHub release page, thereby also allowing easy
auditing of both GitHub download links.

git clone https://github.com/openssh/openssh-portable.git openssh-github
cd openssh-github
git checkout V_9_7_P1
git archive --prefix=openssh-portable-V_9_7_P1/ -o openssh-9.7p1-src.tar.gz HEAD
wget -nv https://github.com/openssh/openssh-portable/archive/refs/tags/V_9_7_P1.tar.gz
sha256sum openssh-9.7p1-src.tar.gz V_9_7_P1.tar.gz
=>
f0c22a08eeaa7dfbae3ba553031a8c7d5322e498216d99ad8074a076b28c6f90 openssh-9.7p1-src.tar.gz
f0c22a08eeaa7dfbae3ba553031a8c7d5322e498216d99ad8074a076b28c6f90 V_9_7_P1.tar.gz

The advantage with all this is that people can then build from a tarball
that corresponds to what's in the git repository, and not have to audit
the generated ./configure script and other files, or have to manually
figure out which files needs to be removed from the official release
tarball to get something that corresponds to the git repository.

Building from a 'git clone' after verifying PGP signature of the
V_9_7_P1 git tag does not lead to the same level of assurance: 1) the
git tag can be moved and re-signed at any time but tarballs are forever,
2) git tags covers a SHA1 commit identity and SHA1 is broken so this
verification does not necessarily prove that the file content correspond
to what was intended to be released. Any SHA-256 checksums of the git
tree is not part of the release announcements either, so it is not
possible to trace things back to the release information. For more
discussion of rationale, see also:

https://blog.josefsson.org/2024/04/13/reproducible-and-minimal-source-only-tarballs/

/Simon

Re: Publish PGP signed tarball without generated content? [ In reply to ]

Apr 17, 2024, 5:06 PM

Post #2 of 9 (74 views)

I think we're going to check in the autoconf-generated files on the
release branches instead.

On Wed, 17 Apr 2024, Simon Josefsson wrote:

> Hi
>
> What do you think about publishing PGP signed tarballs without the
> generated files such as the ./configure script?
>
> What I'm looking for is for some private key holder of the OpenSSH
> portable release key to run
>
> git checkout V_9_7_P1
> git archive --prefix=openssh-portable-V_9_7_P1/ -o openssh-9.7p1-src.tar.gz HEAD
> gpg --detach-sign --armor openssh-9.7p1-src.tar.gz
>
> and then publish the resulting openssh-9.7p1-src.tar.gz and
> openssh-9.7p1-src.tar.gz.asc files, preferably using a version of git
> that leads to archives that are identical to what GitHub currently
> publish.
>
> The tarball would then be identical to what can (currently) be
> downloaded from the GitHub release page, thereby also allowing easy
> auditing of both GitHub download links.
>
> git clone https://github.com/openssh/openssh-portable.git openssh-github
> cd openssh-github
> git checkout V_9_7_P1
> git archive --prefix=openssh-portable-V_9_7_P1/ -o openssh-9.7p1-src.tar.gz HEAD
> wget -nv https://github.com/openssh/openssh-portable/archive/refs/tags/V_9_7_P1.tar.gz
> sha256sum openssh-9.7p1-src.tar.gz V_9_7_P1.tar.gz
> =>
> f0c22a08eeaa7dfbae3ba553031a8c7d5322e498216d99ad8074a076b28c6f90 openssh-9.7p1-src.tar.gz
> f0c22a08eeaa7dfbae3ba553031a8c7d5322e498216d99ad8074a076b28c6f90 V_9_7_P1.tar.gz
>
> The advantage with all this is that people can then build from a tarball
> that corresponds to what's in the git repository, and not have to audit
> the generated ./configure script and other files, or have to manually
> figure out which files needs to be removed from the official release
> tarball to get something that corresponds to the git repository.
>
> Building from a 'git clone' after verifying PGP signature of the
> V_9_7_P1 git tag does not lead to the same level of assurance: 1) the
> git tag can be moved and re-signed at any time but tarballs are forever,
> 2) git tags covers a SHA1 commit identity and SHA1 is broken so this
> verification does not necessarily prove that the file content correspond
> to what was intended to be released. Any SHA-256 checksums of the git
> tree is not part of the release announcements either, so it is not
> possible to trace things back to the release information. For more
> discussion of rationale, see also:
>
> https://blog.josefsson.org/2024/04/13/reproducible-and-minimal-source-only-tarballs/
>
> /Simon
>
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev

Re: Publish PGP signed tarball without generated content? [ In reply to ]

simon at josefsson

Apr 17, 2024, 11:50 PM

Post #3 of 9 (74 views)

Damien Miller <djm@mindrot.org> writes:

> I think we're going to check in the autoconf-generated files on the
> release branches instead.

Ok that may also achieve the same goal of reproducible release tarballs
built from source code.

With that approach, the tarball depends on which autoconf version was
used by the release manager, and perhaps other things from the
environment.

Could you document how to re-generate the release tarball including
mentioning which autoconf version that you used?

That would probably be sufficient to allow people to reproduce the
release tarballs, and to allow people to audit that all generated files
in the tarball were generated from the corresponding source code.

/Simon

Re: Publish PGP signed tarball without generated content? [ In reply to ]

simon at josefsson

Apr 17, 2024, 11:50 PM

Post #4 of 9 (74 views)

Damien Miller <djm@mindrot.org> writes:

> I think we're going to check in the autoconf-generated files on the
> release branches instead.

Ok that may also achieve the same goal of reproducible release tarballs
built from source code.

With that approach, the tarball depends on which autoconf version was
used by the release manager, and perhaps other things from the
environment.

Could you document how to re-generate the release tarball including
mentioning which autoconf version that you used?

That would probably be sufficient to allow people to reproduce the
release tarballs, and to allow people to audit that all generated files
in the tarball were generated from the corresponding source code.

/Simon

Re: Publish PGP signed tarball without generated content? [ In reply to ]

vinschen at redhat

Apr 18, 2024, 12:51 AM

Post #5 of 9 (74 views)

On Apr 18 08:50, Simon Josefsson wrote:
> Damien Miller <djm@mindrot.org> writes:
>
> > I think we're going to check in the autoconf-generated files on the
> > release branches instead.
>
> Ok that may also achieve the same goal of reproducible release tarballs
> built from source code.
>
> With that approach, the tarball depends on which autoconf version was
> used by the release manager, and perhaps other things from the
> environment.
>
> Could you document how to re-generate the release tarball including
> mentioning which autoconf version that you used?

The autoconf version used to generate the files is always put in the
headers of the generated files.

Corinna

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev

Re: Publish PGP signed tarball without generated content? [ In reply to ]

vinschen at redhat

Apr 18, 2024, 12:51 AM

Post #6 of 9 (74 views)

On Apr 18 08:50, Simon Josefsson wrote:
> Damien Miller <djm@mindrot.org> writes:
>
> > I think we're going to check in the autoconf-generated files on the
> > release branches instead.
>
> Ok that may also achieve the same goal of reproducible release tarballs
> built from source code.
>
> With that approach, the tarball depends on which autoconf version was
> used by the release manager, and perhaps other things from the
> environment.
>
> Could you document how to re-generate the release tarball including
> mentioning which autoconf version that you used?

The autoconf version used to generate the files is always put in the
headers of the generated files.

Corinna

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev

Re: Publish PGP signed tarball without generated content? [ In reply to ]

stu at spacehopper

Apr 18, 2024, 2:33 AM

Post #7 of 9 (74 views)

On 2024/04/18 09:51, Corinna Vinschen wrote:
> On Apr 18 08:50, Simon Josefsson wrote:
> > Damien Miller <djm@mindrot.org> writes:
> >
> > > I think we're going to check in the autoconf-generated files on the
> > > release branches instead.
> >
> > Ok that may also achieve the same goal of reproducible release tarballs
> > built from source code.
> >
> > With that approach, the tarball depends on which autoconf version was
> > used by the release manager, and perhaps other things from the
> > environment.
> >
> > Could you document how to re-generate the release tarball including
> > mentioning which autoconf version that you used?
>
> The autoconf version used to generate the files is always put in the
> headers of the generated files.

What is ostensibly the same autoconf version can sometimes generate
different output, as some OS packages include patches to autoconf.
This is usually pretty obvious in a diff though.
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev

Re: Publish PGP signed tarball without generated content? [ In reply to ]

stu at spacehopper

Apr 18, 2024, 2:35 AM

Post #8 of 9 (74 views)

On 2024/04/18 10:06, Damien Miller wrote:
> I think we're going to check in the autoconf-generated files on the
> release branches instead.

That seems a sane approach.

> On Wed, 17 Apr 2024, Simon Josefsson wrote:
> > and then publish the resulting openssh-9.7p1-src.tar.gz and
> > openssh-9.7p1-src.tar.gz.asc files, preferably using a version of git
> > that leads to archives that are identical to what GitHub currently
> > publish.

More than git is involved in this - it also depends on versions of
things like tar and gzip. And github don't guarantee that these files
won't change.

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev

Re: Publish PGP signed tarball without generated content? [ In reply to ]

openssh at roumenpetrov

Apr 18, 2024, 10:43 AM

Post #9 of 9 (72 views)

Canonization .... of file content ?

?? 18.04.24 ?. ? 12:35 ?., Stuart Henderson ??????:
> On 2024/04/18 10:06, Damien Miller wrote:
>> I think we're going to check in the autoconf-generated files on the
>> release branches instead.
>
> That seems a sane approach.
>
>> On Wed, 17 Apr 2024, Simon Josefsson wrote:
>>> and then publish the resulting openssh-9.7p1-src.tar.gz and
>>> openssh-9.7p1-src.tar.gz.asc files, preferably using a version of git
>>> that leads to archives that are identical to what GitHub currently
>>> publish.
>
> More than git is involved in this - it also depends on versions of
> things like tar and gzip. And github don't guarantee that these files
> won't change.

Extracted content depends from OS. Some files should keep EOL in UNIX
style for instance shell scripts, other will be translated.

How to canonise file content? Canonical XML is only for xml. What about
other file types?

Request does not look valuable.

Regards,
Roumen

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev