Mailing List Archive

On 2021.09.21 18:22, Andreas K. Huettel wrote:
> So let's experiment with this... :) announcing:
> https://gentoo.osuosl.org/experimental/amd64/binpkg/default/linux/17.1/x86-64/
>
> More information can be found in a blog post, which will also be on
> planet.g.o
> soon:
> https://dilfridge.blogspot.com/2021/09/experimental-binary-gentoo-package.html
>
> Cheers -A
>
> --
> Andreas K. H?ttel
> dilfridge@gentoo.org
> Gentoo Linux developer
> (council, qa, toolchain, base-system, perl, libreoffice)
>


Andreas,

How is USE=bindist treated?
Its on for stage building and off in profiles.

--
Regards,

Roy Bamford
(Neddyseagoon) a member of
elections
gentoo-ops
forum-mods
arm64

> Andreas,
>
> How is USE=bindist treated?
> Its on for stage building and off in profiles.

USE=bindist is switched on, and in addition we have
ACCEPT_RESTRICT="* -bindist"

--
Andreas K. H?ttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, qa, toolchain, base-system, perl, libreoffice)

Nice job!

On Tue, Sep 21, 2021 at 2:25 PM Andreas K. Huettel <dilfridge@gentoo.org>
wrote:

> > Andreas,
> >
> > How is USE=bindist treated?
> > Its on for stage building and off in profiles.
>
> USE=bindist is switched on, and in addition we have
> ACCEPT_RESTRICT="* -bindist"
>
> --
> Andreas K. Hüttel
> dilfridge@gentoo.org
> Gentoo Linux developer
> (council, qa, toolchain, base-system, perl, libreoffice)
>

I have repos configs placed into /etc/portage/repos.conf with "rsync-type = git" fo all reposso I created binhost.cond file here instead of /etc/portage/ as mentioned in blog post. Then I tried to sync and got warnings:...!!! Section 'binhost' in repos.conf is missing location attribute
!!! Repository 'binhost' has sync-uri attribute, but is missing sync-type attribute
... So it seems it's dosn't sync with binhost index without location attribute. After setup location attribute to specific directory I got warning on sync trying:!!! Repository 'binhost' has sync-uri attribute, but is missing sync-type attribute
!!! Repository 'binhost' is missing masters attribute in '/usr/local/portage/binhost/metadata/layout.conf'
!!! Set 'masters = gentoo' in this file for future compatibility I tried to set up "rsync-type = rsync" or "webrsync" with "amsters = gentoo" but then I got error:!!! Repository 'binhost' is missing masters attribute in '/usr/local/portage/binhost/metadata/layout.conf'
!!! Set 'masters = gentoo' in this file for future compatibility So I created /usr/local/portage/binhost/metadata/layout.conf :masters = gentoo
auto-sync = false
thin-manifests = true

So then on sync I got:   The current sync-type attribute of repository 'gentoo' is not set to 'rsync' or 'webrsync':
   sync-type=git---- Is it currently git sync available for binhost?
        21.09.2021, 20:22, "Andreas K. Huettel" <dilfridge@gentoo.org>:

So let's experiment with this... :) announcing:
https://gentoo.osuosl.org/experimental/amd64/binpkg/default/linux/17.1/x86-64/"]https://gentoo.osuosl.org/experimental/amd64/binpkg/default/linux/17.1/x86-64/

More information can be found in a blog post, which will also be on planet.g.o
soon:
https://dilfridge.blogspot.com/2021/09/experimental-binary-gentoo-package.html"]https://dilfridge.blogspot.com/2021/09/experimental-binary-gentoo-package.html

Cheers -A
 --
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, qa, toolchain, base-system, perl, libreoffice)

On 22/09/2021 09:58, Torokhov Sergey wrote:
> I have repos configs placed into /etc/portage/repos.conf with
> "rsync-type = git" fo all repos
> so I created binhost.cond file here instead of /etc/portage/ as
> mentioned in blog post.

binhost.conf should be in binrepos.conf instead of repos.conf. See also:
https://wiki.gentoo.org/wiki/Binary_package_guide#Pulling_packages_from_a_binary_package_host

> Then I tried to sync and got warnings:
> ...
> !!! Section 'binhost' in repos.conf is missing location attribute
> !!! Repository 'binhost' has sync-uri attribute, but is missing
> sync-type attribute
> ...
> So it seems it's dosn't sync with binhost index without location attribute.
> After setup location attribute to specific directory I got warning on
> sync trying:
> !!! Repository 'binhost' has sync-uri attribute, but is missing
> sync-type attribute
> !!! Repository 'binhost' is missing masters attribute in
> '/usr/local/portage/binhost/metadata/layout.conf'
> !!! Set 'masters = gentoo' in this file for future compatibility
> I tried to set up "rsync-type = rsync" or "webrsync" with "amsters =
> gentoo" but then I got error:
> !!! Repository 'binhost' is missing masters attribute in
> '/usr/local/portage/binhost/metadata/layout.conf'
> !!! Set 'masters = gentoo' in this file for future compatibility
> So I created /usr/local/portage/binhost/metadata/layout.conf :
> masters = gentoo
> auto-sync = false
> thin-manifests = true
>
> So then on sync I got:
>    The current sync-type attribute of repository 'gentoo' is not set to
> 'rsync' or 'webrsync':
>    sync-type=git
> ----
> Is it currently git sync available for binhost?

A binhost is different from an ebuild repository, it is just a
collection of .xpak files so there is no git sync involved.

Sorry for previous html message. I tried to recend it as plaintext.

I have repos configs placed into /etc/portage/repos.conf with "rsync-type = git" fo all repos
so I created binhost.cond file here instead of /etc/portage/ as mentioned in blog post.

Then I tried to sync and got warnings:
...
!!! Section 'binhost' in repos.conf is missing location attribute
!!! Repository 'binhost' has sync-uri attribute, but is missing sync-type attribute
...

So it seems it's dosn't sync with binhost index without location attribute.

After setup location attribute to specific directory I got warning on sync trying:
!!! Repository 'binhost' has sync-uri attribute, but is missing sync-type attribute
!!! Repository 'binhost' is missing masters attribute in '/usr/local/portage/binhost/metadata/layout.conf'
!!! Set 'masters = gentoo' in this file for future compatibility

I tried to set up "rsync-type = rsync" or "webrsync" with "amsters = gentoo" but then I got error:
!!! Repository 'binhost' is missing masters attribute in '/usr/local/portage/binhost/metadata/layout.conf'
!!! Set 'masters = gentoo' in this file for future compatibility

So I created /usr/local/portage/binhost/metadata/layout.conf :
masters = gentoo
auto-sync = false
thin-manifests = true

So then on sync I got:
The current sync-type attribute of repository 'gentoo' is not set to 'rsync' or 'webrsync':
sync-type=git


Is it currently git sync available for binhost?

Am Mittwoch, 22. September 2021, 10:20:10 CEST schrieb Torokhov
Sergey:
> Sorry for previous html message. I tried to recend it as plaintext.
>
> I have repos configs placed into /etc/portage/repos.conf with
> "rsync-type = git" fo all repos so I created binhost.cond file here
> instead of /etc/portage/ as mentioned in blog post.

Nope. As the blog post says, you need to put that text into a *file*
with name -->>> /etc/portage/binrepos.conf <<<--- ...

--
Andreas K. H?ttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

Thanks. These settings are works. I just forgot to add USE="bindist" for tested dev-qt/qtwebengine package as mentioned early in mailing list.

Great! Now no need to compile qtwebengine for hours next time! :)

Thanks! I never used an external binhost repo so mislead myself.

Now I see that it's like a remote analogue of "/usr/portage/packages"
and tarballs from remote binhost are fetched on demand into "local binhost"
( /usr/portage/packages/<category> ) before installation with replacing early presented tarballs.


> binhost.conf should be in binrepos.conf instead of repos.conf. See also:
> https://wiki.gentoo.org/wiki/Binary_package_guide#Pulling_packages_from_a_binary_package_host


> A binhost is different from an ebuild repository, it is just a
> collection of .xpak files so there is no git sync involved.

Finally it happened!

I already planned to try to ask infra/council about sponsoring few servers for
build farm for "official gentoo binhosts" when I had enough time, but
fortunately, you've already did that.

It's very good news.

Btw, do you need any help with that?

I'd be very happy to help with that project.

Hi Vadim,

> Finally it happened!
> I already planned to try to ask infra/council about sponsoring few
> servers for build farm for "official gentoo binhosts" when I had
> enough time, but fortunately, you've already did that.
> It's very good news.

Thanks! Nice to see that this is appreciated :)

So far I'm only using "spare time" on the machine that builds the
releng stages (amd64, x86, m68k, riscv). So no need for a big server
farm.

> Btw, do you need any help with that?
> I'd be very happy to help with that project.

Sure! Feel free to add yourself to the Project:Binhost wiki page. I'll
ask for an alias and a channel soon.

The most useful steps now are only half related to actual building. I
barely know any python and am not very familiar with portage
internals... this is what in my opinion we'd need next:

1) a tool to manage and manipulate a binpkg/ directory tree
The main functions that I see needed are
* delete packages/versions that are not in the gentoo repository
anymore (xpak and in index file), maybe with some grace time
* merge xpak files built elsewhere into the directory (also in the
index file)
(imagine you have a second container that builds with same CFLAGS, but
with use settings for gnome, not plasma... or with updated
dependencies because of changes in gentoo.git... you want to merge the
trees for distribution without having duplicate builds)

2) binary package cryptographic signing and verification
Essentially we need to finish support for GLEP78; this is being worked
on in RinCat's pull request
https://github.com/gentoo/portage/pull/562
See also https://www.gentoo.org/glep/glep-0078.html

3) an easy way to figure out if a binary package repo is suitable for
a profile / arch / ... or not, and a standard for path names
This is not so important right now, and partially also already present
I guess.

The actual builder right now is very simple and wired up with a single
daily cron job; the mirrors are only updated manually by me until bug
813528 is handled.

Cheers
Andreas

--
Andreas K. H?ttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

On Thursday, September 23rd, 2021 at 10:11, Andreas K. Huettel <dilfridge@gentoo.org> wrote:
> 3. an easy way to figure out if a binary package repo is suitable for
> a profile / arch / ... or not, and a standard for path names
> This is not so important right now, and partially also already present
>

There is a PROFILE data in Packages index, but it is not useful if we are
going to mix multiple profiles together, like KDE/GNOME.

Also currently package manager does not care about the CFLAGS /
microarchitecture of the binary package, like from x86-64 to x86-64-v4.
Additional metadata and matches need to be added.

Thanks,
Sheng Yu

On Thu, Sep 23, 2021 at 7:12 AM Andreas K. Huettel <dilfridge@gentoo.org> wrote:
>
> Hi Vadim,
>
> > Finally it happened!
> > I already planned to try to ask infra/council about sponsoring few
> > servers for build farm for "official gentoo binhosts" when I had
> > enough time, but fortunately, you've already did that.
> > It's very good news.
>
> Thanks! Nice to see that this is appreciated :)
>
> So far I'm only using "spare time" on the machine that builds the
> releng stages (amd64, x86, m68k, riscv). So no need for a big server
> farm.
>
> > Btw, do you need any help with that?
> > I'd be very happy to help with that project.
>
> Sure! Feel free to add yourself to the Project:Binhost wiki page. I'll
> ask for an alias and a channel soon.
>
> The most useful steps now are only half related to actual building. I
> barely know any python and am not very familiar with portage
> internals... this is what in my opinion we'd need next:
>
> 1) a tool to manage and manipulate a binpkg/ directory tree
> The main functions that I see needed are
> * delete packages/versions that are not in the gentoo repository
> anymore (xpak and in index file), maybe with some grace time
> * merge xpak files built elsewhere into the directory (also in the
> index file)

eclean packages from gentoolkit does this exactly.

* Matt Turner schrieb am 23.09.21 um 09:28 Uhr:
> On Thu, Sep 23, 2021 at 7:12 AM Andreas K. Huettel <dilfridge@gentoo.org> wrote:
> >
> > Hi Vadim,
> >
> > > Finally it happened!
> > > I already planned to try to ask infra/council about sponsoring few
> > > servers for build farm for "official gentoo binhosts" when I had
> > > enough time, but fortunately, you've already did that.
> > > It's very good news.
> >
> > Thanks! Nice to see that this is appreciated :)
> >
> > So far I'm only using "spare time" on the machine that builds the
> > releng stages (amd64, x86, m68k, riscv). So no need for a big server
> > farm.
> >
> > > Btw, do you need any help with that?
> > > I'd be very happy to help with that project.
> >
> > Sure! Feel free to add yourself to the Project:Binhost wiki page. I'll
> > ask for an alias and a channel soon.
> >
> > The most useful steps now are only half related to actual building. I
> > barely know any python and am not very familiar with portage
> > internals... this is what in my opinion we'd need next:
> >
> > 1) a tool to manage and manipulate a binpkg/ directory tree
> > The main functions that I see needed are
> > * delete packages/versions that are not in the gentoo repository
> > anymore (xpak and in index file), maybe with some grace time
> > * merge xpak files built elsewhere into the directory (also in the
> > index file)
>
> eclean packages from gentoolkit does this exactly.

plus 'emaint binhost' for rebuilding the index after adding packages
manually :)

-Marc

--
0xCA3E7BF67F979BE5 - F7FB 78F7 7CC3 79F6 DF07
6E9E CA3E 7BF6 7F97 9BE5