Mailing List Archive

??, 2 ???. 2020 ?. ? 13:52, Ramon Fischer <Ramon_Fischer@hotmail.de>:
>
> I decided to use "EGIT_COMMIT" to let the ebuild pulling a certain commit.

And even that would not give the sense of security...

Just read in gentoo-dev [1]:
...unannounced serverside change by GitHub, which broke download of
tarballs by git-tree-hash, e.g. previously https://
api.github.com/repos/JuliaLang/MbedTLS.jl/tarball/
2d94286a9c2f52c63a16146bb86fd6cdfbf677c6 would give the tarball for
that tree- hash, while it now gives the tarball for master instead.

[1] - https://archives.gentoo.org/gentoo-dev/message/41d8c5457df392ed0309153651db5b3c

--
Best regards,
Alex

On Tue, Aug 4, 2020 at 6:57 PM Alexey Mishustin <shumkar@shumkar.ru> wrote:
>
> ??, 2 ???. 2020 ?. ? 13:52, Ramon Fischer <Ramon_Fischer@hotmail.de>:
> >
> > I decided to use "EGIT_COMMIT" to let the ebuild pulling a certain commit.
>
> And even that would not give the sense of security...
>
> Just read in gentoo-dev [1]:
> ...unannounced serverside change by GitHub, which broke download of
> tarballs by git-tree-hash, e.g. previously https://
> api.github.com/repos/JuliaLang/MbedTLS.jl/tarball/
> 2d94286a9c2f52c63a16146bb86fd6cdfbf677c6 would give the tarball for
> that tree- hash, while it now gives the tarball for master instead.
>

I'm pretty sure EGIT_COMMIT will fetch by commit ID using git, not
download a hash-labeled tarball, so I don't think this issue would
impact you if that is how you're fetching things.

If you did use a hash tarball with SRC_URI and a conventional
download, then emerge would still refuse to use the tarball if it
failed the manifest hash check, so it wouldn't be installing anything
you didn't want.

Generally this isn't going to immediately break anything used by the
Gentoo repo since 99% of this stuff will be mirrored, and the mirrors
check hashes too. So, when github breaks the download link the
mirrors will preserve their existing tarballs and refuse to replace
them with new ones that don't have a matching hash (I'm talking about
the actual hash of the file using multiple algorithms, not the hash in
the filename). When you fetch from a mirror you'll still get the
correct version of the file. If for some reason you can't reach any
mirrors then you would download the broken link from github and then
emerge would reject the file due to hash mismatch.

Still, unless github fixes this we'll probably have to fix a bunch of
links in the repositories - at least any based on hashes. I'm not
sure if this impacts tags. The SRC_URIs are still invalid and we
don't want to maintain that state as new mirrors won't be able to
retrieve the file, and we generally want a valid SRC_URI for
everything. Devs can always just upload the tarball to any random
webserver and change the URI to point to it. My guess though is that
everybody will want to give this a few days to see if github fixes
their links.

Really this could happen with any web hosting service - github is just
a really prominent one. Back in the day if sourceforge suddenly went
down a whole bunch of SRC_URIs would have broken too.

--
Rich

On 2020-08-04 19:36-0400 Rich Freeman <rich0@gentoo.org> wrote:

> On Tue, Aug 4, 2020 at 6:57 PM Alexey Mishustin <shumkar@shumkar.ru>
> wrote:
> >
> > ??, 2 ???. 2020 ?. ? 13:52, Ramon Fischer
> > <Ramon_Fischer@hotmail.de>:
> > >
> > > I decided to use "EGIT_COMMIT" to let the ebuild pulling a
> > > certain commit.
> >
> > And even that would not give the sense of security...
> >
> > Just read in gentoo-dev [1]:
> > ...unannounced serverside change by GitHub, which broke download of
> > tarballs by git-tree-hash, e.g. previously https://
> > api.github.com/repos/JuliaLang/MbedTLS.jl/tarball/
> > 2d94286a9c2f52c63a16146bb86fd6cdfbf677c6 would give the tarball for
> > that tree- hash, while it now gives the tarball for master instead.
> >

This seems to affect only api.github.com, packages in ::guru use
https://github.com/<REPO>/archive/<COMMIT>.tar.gz instead, which is not
affected (just checked with net-wireless/rtl8192eu-0_pre20200123).

> I'm pretty sure EGIT_COMMIT will fetch by commit ID using git, not
> download a hash-labeled tarball, so I don't think this issue would
> impact you if that is how you're fetching things.

Correct.

> […]
> Still, unless github fixes this we'll probably have to fix a bunch of
> links in the repositories - at least any based on hashes. I'm not
> sure if this impacts tags. The SRC_URIs are still invalid and we
> don't want to maintain that state as new mirrors won't be able to
> retrieve the file, and we generally want a valid SRC_URI for
> everything. Devs can always just upload the tarball to any random
> webserver and change the URI to point to it. My guess though is that
> everybody will want to give this a few days to see if github fixes
> their links.

A quick grep indicated that the only packages in ::gentoo using
api\.github\.com.*tarball are net-analyzer/tcpflow, dev-python/mypy,
dev-lang/julia and app-forensics/dfxml.

> Really this could happen with any web hosting service - github is just
> a really prominent one. Back in the day if sourceforge suddenly went
> down a whole bunch of SRC_URIs would have broken too.
>

On Tue, Aug 4, 2020 at 7:51 PM tastytea <tastytea+gentoo@tastytea.de> wrote:
>
> This seems to affect only api.github.com, packages in ::guru use
> https://github.com/<REPO>/archive/<COMMIT>.tar.gz instead, which is not
> affected (just checked with net-wireless/rtl8192eu-0_pre20200123).

Ah, didn't notice that. This is the more common approach for Gentoo
packages, if they use hashes at all. Usually tags are preferred.

And if upstream actually has an official source tarball that is what
gets used. The only reason anybody in Gentoo uses github links at all
is either because upstream uses it officially, or upstream doesn't
even bother to release source tarballs.

--
Rich