Mailing List Archive

digest + manifest = new file format
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

This is basically a resubmission of Genone's file format merger [1].

The code to handle digests and manifests is currently being designed and
rewritten. As such we need two things from the developer community.
One is a decision of what the new format will entail. Two is a firm
agreement on the migration plan.

1. The file format is specified in Genone's original mail, but I'll go
over it again to save you the extra linkage ;)

Inside of the digest-manifest we have:
EBUILD Filename SIZE 1234567 MD5 A4FD085FF SHA-256 A7439FDB1
<type> <filename> <size-tag> <size> <hash-tag> <value> <hash-tag>
<value>....
Where type is the type of file (DIGEST, SRC_URI, EBUILD, PATCH, etc)

*Note, that the size-tag was added to make the human parsing easier.

2. If you didn't notice by now, this format breaks versions of portage
that don't support it ( ie 2.0.X ). This leads us to a few options as
users migrate to a version of portage that supports the new file format.

A. Write some code into the final stable version to do both kind of
formats and/or use the new format style but fall back to the old style (
since at release time the new style format probably won't be in the tree
yet ). Either way, the last release of 2.0.X should work with the new
file format in enough of a fashion to not break. This means that when
the old manifest + digests are combined in the tree, most users should
be ok even if they aren't upgraded to 2.1.X.

B. Have a migration time where both formats are in the tree. For a
while the tree will be larger, and many people will have issues with
this. However, old portage will use the old files, and new releases
will use the new files. Then after a period of however long ( 6 months?
) announce loudly beforehand that portage-2.0.X will no longer be
supported and pull the old digests/manifests out of the tree.

C. The Carpaski way ;)[2] Add support for the new format in 2.1,
while also adding support for current format. Force everyone to upgrade
to 2.1.X, while announcing that 2.0.X will not work in the future. When
we have confidence that most users are upgraded, pull the old format and
add the new format to the tree.

Problems with all of these include the same problems as the cascaded
profiles, some goofball doesn't upgrade for a year, syncs with new
digests...how does he get his portage upgraded? An upgrade path should
be provided and documented in this case.

The best route is probably some combination of the above. Pre-emptively
add support for the new format to stable, while announcing the death of
the old format after 2.1's release. When most users upgrade normally to
the latest 2.0.X series ( perhaps fearing the changes of 2.1.X ) they
will gain support for the new file format (or if not support, merely a
working portage instead of broken). We should catch the majority of
users who upgrade with either the last 2.0.X release or the new release
of 2.1.X.

Suggestions, criticisms, etc... welcome.

[1] http://marc.theaimsgroup.com/?l=gentoo-dev&m=109725383228494&w=2
[2] http://marc.theaimsgroup.com/?l=gentoo-dev&m=109725779909405&w=2
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQvLQuGzglR5RwbyYAQLykg//TbNRfPlUutmLIn09IzQ5nX1XR1RZNUVo
e4dgiOWbJz+dBbc4jU8NxrMlSp7tv1rMhSaHw+oqhtPFZGCBbTXIFEZkBgvq9H2L
AYO62Oht8xgfYgIl/KIJgWTJTmlbC2mLRMhceD83pfJKDhp6px435g+7oUyK/sPz
rxcDHn95veT+4WdkrwrGK8rGbAsWQ9NivyUySzfPyZN83/mc50XLd7x2cyHHKmkx
wwPORG8wdhWYFeLD16SZPyBmiVW1MkA1LIK95fDRV3j9wA2mqftZlSMR0lO76fJr
tgbrKewJKVvms4+MbFJwRJGL9R6mg4VstjFeNw6smZtTSUgXtGV6qJX4S3Zj+YQJ
37gpl/zCVVvmGEa8uN0gGRDnVHEyeAAHeZbeniukc6m3fn/6JvXeDqPRFGTwWorS
B4G2DyaoJCg0Sia5lwAYBqgnikKXOZ4Q3eD0fMP1kCwBZdTUG2CTV1hNvF6FfV/e
+V26rqI2R/rXmPnXmnipM1XsajT2GLnhqY1L7OUZfWF52QuqJI4sta7VybQ4lBny
cSFEmjJM+nHoCnTW90iYWhEcwNC7/F20LZEsin/uim0/rHhqhuLPyTZ6hkg3CauW
BeLfRfGC4NY00M/xamkVzPlnoMsIZzsX9W7WkpBgwgB4lljwzUmQ6CXPiQsu8tLC
SYKtXlGJhI8=
=At7Y
-----END PGP SIGNATURE-----
--
gentoo-dev@gentoo.org mailing list
Re: digest + manifest = new file format [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Whoopse, should have gone to gentoo-portage-dev, my bad ;)

Alec Warner wrote:
> This is basically a resubmission of Genone's file format merger [1].
>
> The code to handle digests and manifests is currently being designed and
> rewritten. As such we need two things from the developer community.
> One is a decision of what the new format will entail. Two is a firm
> agreement on the migration plan.
>
> 1. The file format is specified in Genone's original mail, but I'll go
> over it again to save you the extra linkage ;)
>
> Inside of the digest-manifest we have:
> EBUILD Filename SIZE 1234567 MD5 A4FD085FF SHA-256 A7439FDB1
> <type> <filename> <size-tag> <size> <hash-tag> <value> <hash-tag>
> <value>....
> Where type is the type of file (DIGEST, SRC_URI, EBUILD, PATCH, etc)
>
> *Note, that the size-tag was added to make the human parsing easier.
>
> 2. If you didn't notice by now, this format breaks versions of portage
> that don't support it ( ie 2.0.X ). This leads us to a few options as
> users migrate to a version of portage that supports the new file format.
>
> A. Write some code into the final stable version to do both kind of
> formats and/or use the new format style but fall back to the old style (
> since at release time the new style format probably won't be in the tree
> yet ). Either way, the last release of 2.0.X should work with the new
> file format in enough of a fashion to not break. This means that when
> the old manifest + digests are combined in the tree, most users should
> be ok even if they aren't upgraded to 2.1.X.
>
> B. Have a migration time where both formats are in the tree. For a
> while the tree will be larger, and many people will have issues with
> this. However, old portage will use the old files, and new releases
> will use the new files. Then after a period of however long ( 6 months?
> ) announce loudly beforehand that portage-2.0.X will no longer be
> supported and pull the old digests/manifests out of the tree.
>
> C. The Carpaski way ;)[2] Add support for the new format in 2.1,
> while also adding support for current format. Force everyone to upgrade
> to 2.1.X, while announcing that 2.0.X will not work in the future. When
> we have confidence that most users are upgraded, pull the old format and
> add the new format to the tree.
>
> Problems with all of these include the same problems as the cascaded
> profiles, some goofball doesn't upgrade for a year, syncs with new
> digests...how does he get his portage upgraded? An upgrade path should
> be provided and documented in this case.
>
> The best route is probably some combination of the above. Pre-emptively
> add support for the new format to stable, while announcing the death of
> the old format after 2.1's release. When most users upgrade normally to
> the latest 2.0.X series ( perhaps fearing the changes of 2.1.X ) they
> will gain support for the new file format (or if not support, merely a
> working portage instead of broken). We should catch the majority of
> users who upgrade with either the last 2.0.X release or the new release
> of 2.1.X.
>
> Suggestions, criticisms, etc... welcome.
>
> [1] http://marc.theaimsgroup.com/?l=gentoo-dev&m=109725383228494&w=2
> [2] http://marc.theaimsgroup.com/?l=gentoo-dev&m=109725779909405&w=2
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQvLSLmzglR5RwbyYAQJkEQ//aQFAjvEFJ1tRq988HOEyh/XxHlsPnksm
HAQCs8iSkLY6cI9IcRniTgkLuNwLYMqElEMACSWlTWCGmL5lrG/6XDkPaXDBeNLg
jyw1h5yCauQnH8THwzuLQ4RVg3pFsmIthrJyB55V4i/DDiAY6JGfs7xwSys1Poih
rUohcrWiI/gWr2UHzrlS4xtqO4iWw5kpUBAslDk8D+RZEjn1i5GcW/3J8GBS3RUF
TFKkzBf/ruhqN50Cpbo6Q0gEnvKcgtQSrYBlIyCdw+GpGqbalPVbGPVbozApwk6d
DQvLbMgZq7G8oM8vo8IL0cThdocNfWcNDAUrPadxzA6szzxxCiu3DiCVNLRoAhOy
2JCyGfTEY3KStX39NvCCcnkGTcP6XbGw3ummPJwLlNxP6x+KwCbHj5Hats+LDO71
dxOjh8+GeOCmafO89OmSD2W2l/zAdmU6KPUYY7OfUttrmbcTZehdTksQBZqlohRg
Coz4Fe6BxMn3dQ2C3wplALwRIikLsP95UwH7MSlAIFzrEdy7XPQydPKB/kjOk4e3
r7nrM/EOoW54QbhX8ItY+y3dEDJWNKJduax7uYXbz+tMH0J74oxiVswcu/LTgXFv
RkCJ4UqED7zvyQafaL5yTk8UYknI/oeo7x9siiqSJ90J7Y6m2GrX1i8vLoiOnyTf
+2gWdQCckT8=
=TOrq
-----END PGP SIGNATURE-----
--
gentoo-dev@gentoo.org mailing list
Re: digest + manifest = new file format [ In reply to ]
On 5/8/2005 4:36:40, Alec Warner (warnera6@egr.msu.edu) wrote:

> Problems with all of these include the same problems as the cascaded
> profiles, some goofball doesn't upgrade for a year, syncs with new
> digests...how does he get his portage upgraded? An upgrade path should
> be provided and documented in this case.

A simple way of doing this would be to provide an emergency portage
download; a tarball that one could unpack over the live filesystem that
puts a current stable portage in place. Since portage doesn't include
target-specific binaries it doesn't vary from one system to another so
one tarball is enuogh. It could be on an 'emergency recovery' page
available from www.gentoo.org.

Kev.

--
gentoo-dev@gentoo.org mailing list