Mailing List Archive

On Fri, 2024-03-15 at 10:06 +0200, Jaco Kroon wrote:
> Hi All,
>
> I was messing with some storage related caching on some of our hosts
> this morning when I wondered about how much storage the gentoo mirrors
> were consuming.  I'm not too worried about the current storage, but I am
> noticing that the storage requirements are creeping quite a bit (as per
> attached), and if that growth rate continues it may become a problem
> *eventually*.
>
> Can this growth be explained?
>

I guess the simplest explanation is that software is growing larger,
and in the end we should be expecting to adding new packages faster than
removing dead ones. Add to that the grotesque inefficiency of modern
programming languages such as Go and Rust.

--
Best regards,
Micha? Górny

>I guess the simplest explanation is that software is growing larger,
>and in the end we should be expecting to adding new packages faster than
>removing dead ones. Add to that the grotesque inefficiency of modern
>programming languages such as Go and Rust.

Wouldn’t initiatives like rust-dev[0] help with that? I know that Debian is
also packaging Rust this way[1].

[0]: https://wiki.gentoo.org/wiki/Project:Rust/rust-dev
[1]: https://wiki.debian.org/Rust

Hoël

> Wouldn’t initiatives like rust-dev[0] help with that? I know that Debian
> is also packaging Rust this way[1].

I think this was tried long time ago in rust-overlay and failed at the
end because the dependency graph was incredibly big. In fact you can see
it on the wiki, this is larger than _the bigger_ Haskell packages.

> I guess the simplest explanation is that software is growing larger,

This is not only the case of Rust, but Go, JAVA and .NET and maybe some
other projects. Self-bootstrap anyone? :)

> Can this growth be explained?
> Is it expected to continue at this rate?

Graph is just showing the overall growth, if we associate distfiles to
packages we will get the answers.

W dniu 15.03.2024 o 16:40, Hoël Bézier pisze:
>> I guess the simplest explanation is that software is growing larger,
>> and in the end we should be expecting to adding new packages faster than
>> removing dead ones.  Add to that the grotesque inefficiency of modern
>> programming languages such as Go and Rust.
>
> Wouldn’t initiatives like rust-dev[0] help with that? I know that Debian
> is also packaging Rust this way[1].
>
> [0]: https://wiki.gentoo.org/wiki/Project:Rust/rust-dev
> [1]: https://wiki.debian.org/Rust
>
> Hoël

--
Have a great day!

~ Maciej XGQT Bar?

xgqt@gentoo.org
Gentoo Linux developer
(dotnet, emacs, math, ml, nim, scheme, sci)
https://wiki.gentoo.org/wiki/User:Xgqt
9B0A 4C5D 02A3 B43C 9D6F D6B1 14D7 4A1F 43A6 AC3C

Hi Jaco,

* we have more stages
* the binary packages have to go somewhere
* and, temporarily, things are duplicated due to the 17.x / 23.0 profile transition

The third point will eventually go away. However, I'm not sure how much it actually
contributes.

https://www.akhuettel.de/~huettel/plots/mirrors.php

If you look at the plots, the distfiles part is surprisingly large.
Binary packages (17.x and 23.0) and 17.x stages are under "releases".
The 23.0 stages for testing are under "experimental".

Lastly, I'm still working on an automated cleanup for outdated "small arches" binary
packages (i.e. not arm64 and amd64, these are cleaned automatically already).
This just wasn't a priority so far.

Hope this helps.
-a

Am Freitag, 15. M?rz 2024, 09:06:36 CET schrieb Jaco Kroon:
> Hi All,
>
> I was messing with some storage related caching on some of our hosts
> this morning when I wondered about how much storage the gentoo mirrors
> were consuming. I'm not too worried about the current storage, but I am
> noticing that the storage requirements are creeping quite a bit (as per
> attached), and if that growth rate continues it may become a problem
> *eventually*.
>
> Can this growth be explained?
>
> Is it expected to continue at this rate?
>
> Kind regards,
> Jaco
>


--
Andreas K. H?ttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, comrel, toolchain, base-system, perl, libreoffice)
https://wiki.gentoo.org/wiki/User:Dilfridge