Mailing List Archive

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Marc Blumentritt wrote:
> Hi,
>
> since some months I'm searching about informations, how Gentoo handles
> all packages which are needed for compilation and installation of source
> packages. Of course emerge handles all installations, but how does it
> handle the toolchain to not break it or system compatibility, if one
> major package of the toolchain is updated (like gcc)?
>
> I know that there is a gcc upgrade guide, but what I have in mind is a
> general discription of the toolchain, how Gentoo handles it through
> emerge and ebuilds, and what you have to think about, when you update
> toolchain packages. I think that Gentoo as a source based distribution
> needs something like this to explain to all users, which never heard of
> a toolchain before they switched to Gentoo (like me), what's all magic
> behind compiling all your packages on your own system and why this is so
> much better than using precompiled binarys like nearly everyone else.
>
> What to you all think?
> Marc Blumentritt

Well, on the Gentoo-specific side of things, you're probably looking for the
Gentoo Developer Handbook[1]. Or you might try project-specific[2] information
for whichever arch-specific toolchain you will be working with (they can be
funny beasts between different arches).

As far as things to think about when updating your toolchain, look in the forums
for threads by robmoss, a retired Gentoo toolchain maintainer. Also search for
early stage 1/3 guides (early 2005 and older) for more input from users and
developers about the toolchain upgrade/downgrade process.

I think the Gentoo base project folks might be able to give you some more good
links on related documentation available elsewhere, as they maintain gcc et al.
Anything else, like how autotools magic works (or doesn't), you should probably
check out the relevant gnu.org pages.

If you can put together a doc that's more Gentoo specific (ideally, less having
to do with Linux in general, IMHO), I'm sure it'd be welcomed. Otherwise, you
might consider submitting it to the knowledge base[3] for review.

Hope this helps.

[1] http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml
[2] http://www.gentoo.org/proj/en/index.xml?showlevel=3
[3] http://kbase.gentoo.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.4 (GNU/Linux)

iD8DBQFEx94rrsJQqN81j74RAqUgAKCZ2TPC4wOiTf2XB0ztMkgch40gxwCgtnQS
Lq2QWKhlY8isabKrFnFpKMc=
=eoJL
-----END PGP SIGNATURE-----
--
gentoo-doc@gentoo.org mailing list

Josh Saddler schrieb:
> Well, on the Gentoo-specific side of things, you're probably looking for the
> Gentoo Developer Handbook[1]. Or you might try project-specific[2] information
> for whichever arch-specific toolchain you will be working with (they can be
> funny beasts between different arches).
I looked at these links (have done before in case of the Devbook, but
could not find answers, only new places to ask my question.
The toolchain is arch-specific? I suspected this, but you confirmed it
and for me this is one more reason for a guide to the toolchain! Perhaps
it should be part of the Gentoo Handbook, since this is already arch
specific?

> As far as things to think about when updating your toolchain, look in the forums
> for threads by robmoss, a retired Gentoo toolchain maintainer. Also search for
> early stage 1/3 guides (early 2005 and older) for more input from users and
> developers about the toolchain upgrade/downgrade process.
I did two stage 1/3 installations and read a lot about the proposed
proper "rebuild your system" instructions: emerge -e system && emerge -e
system && emerge -e world && emerge -e world (I read also the
explanation from BobP, why he proposes this - circular deps and stuff).
robmoss was even adding a run of the bootstrap.sh before these
instructions in [1]. He does not explain why he would run this command
chain. His post is from 2004, so I do not now, if it is still a good
advice. I have read different statements, saying that this is absolutly
overkill and adds nothing in functionality, speed and stability (e.g.
here [2]), because gcc will build itself 3 times during one build
process (3-stage bootstrap, does it do this with every emerge or only
when you bootstrap?).

I would like to have something official about this AND some explanation
why to do it this way.

> I think the Gentoo base project folks might be able to give you some more good
> links on related documentation available elsewhere, as they maintain gcc et al.
> Anything else, like how autotools magic works (or doesn't), you should probably
> check out the relevant gnu.org pages.
Yes, I can try my luck on gnu.org, but since the toolchain is so central
for Gentoo (source based) and it's users (even if they do not know it),
I think a guide from Gentoo would be a good thing, because it could
point out everything special to Gentoo.

> If you can put together a doc that's more Gentoo specific (ideally, less having
> to do with Linux in general, IMHO), I'm sure it'd be welcomed. Otherwise, you
> might consider submitting it to the knowledge base[3] for review.
Thought already about submitting this to the knowledge base and read the
first posts on the kb mailing list, but I think this topic is to general
for kb. As far as I understood the kb is something like "You have this
error, than do this.". Correct me, if I'm wrong.


I would like to put together a doc for Gentoo, but I think I do not now
nearly enough for this task (that's the reason I asked for this guide in
the first place). I have thought about a structure, but I cannot fill it
with meaningful content:

0) Introduction
Binaries vs. scripts? Why binaries? What are libs and what is dynamic
linking vs. static linking (this is important for rebuilding your
system)? Why is Gentoo source based? Why is this so cool compared to
binary based?

1) What is the toolchain aka how to build binaries?
Generic
A group of programs, which create from source code binaries and install
them on your computer.
List of programs and a short description:
gcc
binutils
glibc
linux-headers
...

Put these programs in the correct order for the build process. If you
create a general binary, than first you use foo to do bar... Since this
is different from package to package, a "wrapper" would be cool and
tada, here comes portage (point to the next chapter).

Helper programs
Why are they needed

Arch-Specific
Why is the toolchain different from arch to arch?
List of arch specific programs

2) How does Gentoo handle the Toolchain?
Short discription of portage (ebuild (the script), emerge, ebuilds (the
build instructions), eclasses, USE-flags, ...). Perhaps something about
the new alternatives to emerge (or was it portage)?

What about compilation options (CHOST, CFLAGS and CXXFLAGS)? What to do,
if you want to change these? Not a complete list of options (that's
gnu.org for), but the general way, how Gentoo handles these, sane
compilation options. What to do, if you change these and why could this
be critical for your system? How to rebuild your system (twice? world?)?

3) How does Portage handle updates of Toolchain packages?
Difference between minor and major updates, is there a general way to
handle this or is it a special case from major update to major update,
so that you cannot give general instructions.

Should you rebuild your complete system (world) after a major update?
What do gain from this? What if I do not rebuild it? What to do, if you
do not need to rebuild world, but only some special packages (e.g. some
libs) to prevent breaking part of your system?

Major updates and changing of compilation options create similar
problems, yes or no? If yes, can they be handled in the same way?




Well, these are the questions I'm pondering. What do you all think?

I read the linux-from-scratch guide for some answers about toolchain
handling and building of binaries, but this just increased my hunger for
a Gentoo guide on this topic.

Regards,
Marc

[1] http://forums.gentoo.org/viewtopic-p-1368747.html#1368747
[2] http://www.mail-archive.com/gentoo-user%40lists.gentoo.org/msg36343.html
--
gentoo-doc@gentoo.org mailing list

On Wednesday 26 July 2006 17:27, Josh Saddler wrote:
> As far as things to think about when updating your toolchain, look in the
> forums for threads by robmoss, a retired Gentoo toolchain maintainer.

not to be picky, but he wasnt a toolchain maintainer ... he maintained some
base packages (like grub), but not the toolchain

not to say his guides arent good, information coming from him on the forum is
generally high quality stuff

> I think the Gentoo base project folks might be able to give you some more
> good links on related documentation available elsewhere, as they maintain
> gcc et al.

there is a base-system group who maintains most "system" type packages (things
like bootloaders, filesystem apps, etc...) and then there is a toolchain
group who manages just the compiler packages (binutils, glibc, gcc, etc...)

generally the core people in both groups are the same people though ;)

> Anything else, like how autotools magic works (or doesn't), you
> should probably check out the relevant gnu.org pages.

the devmanual goes into autotool details now ... but that's geared towards
helping Gentoo developers identify/fix common bugs/mistakes
-mike

On Friday 28 July 2006 04:30, Marc Blumentritt wrote:
> The toolchain is arch-specific?

that depends on what you mean by "arch-specific"

binutils/gcc/glibc all use the same source code from upstream/build system
regardless of arch, but the same version may not work correctly on all
architectures

> I suspected this, but you confirmed it
> and for me this is one more reason for a guide to the toolchain! Perhaps
> it should be part of the Gentoo Handbook, since this is already arch
> specific?

i dont really understand what it is you're trying to document or what you
think you're trying to document

> 0) Introduction
> Binaries vs. scripts? Why binaries?

this question is answered in other Gentoo documents ... normally referred to
as "why from source" or "source vs binaries"

> What are libs and what is dynamic
> linking vs. static linking (this is important for rebuilding your
> system)?

static vs dynamic libs is not related to Gentoo at all ... all operating
systems deal with this question

> Why is Gentoo source based? Why is this so cool compared to
> binary based?

same questions as what you posted first in this section

> 1) What is the toolchain aka how to build binaries?
> <snip>
> Put these programs in the correct order for the build process. If you
> <snip>

again, not Gentoo specific ... but might be useful i think

> Helper programs
> Why are they needed

a better question: *what* helper programs ?

> Arch-Specific
> Why is the toolchain different from arch to arch?

i dont understand this at all ... of course the toolchain will be different as
no cpu is the same so generating code for a powerpc cpu will not work on an
amd64

> List of arch specific programs

there are none

> 2) How does Gentoo handle the Toolchain?
> Short discription of portage (ebuild (the script), emerge, ebuilds (the
> build instructions), eclasses, USE-flags, ...). Perhaps something about
> the new alternatives to emerge (or was it portage)?

a high level view might be useful, but getting into the ebuild guts would not
cater to the general user at all ... plus we toolchain guys like to rewrite
things all the time and updating the docs for people outside of the toolchain
herd would be a waste of our time ... if someone felt like just doing that
though, that would be useful

> What about compilation options (CHOST, CFLAGS and CXXFLAGS)? What to do,
> if you want to change these? Not a complete list of options (that's
> gnu.org for), but the general way, how Gentoo handles these, sane
> compilation options. What to do, if you change these and why could this
> be critical for your system? How to rebuild your system (twice? world?)?

i think we already have a section touching on these topics

> 3) How does Portage handle updates of Toolchain packages?

people shouldnt care ... it should always "just work"
-mike

Mike Frysinger schrieb:
> On Friday 28 July 2006 04:30, Marc Blumentritt wrote:
>> I suspected this, but you confirmed it
>> and for me this is one more reason for a guide to the toolchain! Perhaps
>> it should be part of the Gentoo Handbook, since this is already arch
>> specific?
>
> i dont really understand what it is you're trying to document or what you
> think you're trying to document

I suggest to document the toolchain in two steps:
1.) Some general information about the toolchain. I switched to Gentoo,
because just from using it I learn a lot about Linux. The fisrt section
should be for learning. Before Gentoo I never heard about the toolchain,
because I just downloaded binaries and used them. Now I download the
source and build them, so I would like to now more about this process,
because I want to learn. I know a little bit about programming (mostly
scripting), but the first time I used a compiler was during my studies
on a windows machine, not really knowing, what this thing is doing (was
during my study). So this is not really a Gentoo topic, but Gentoo
provides very good docs and I hope to get the toolchain explained in one
of these wonderful Gentoo docs.

2.) After general information I put the Gentoo related stuff... (of
course, for learning, too, but in a more Gentooish way.)

>
>> 0) Introduction
>> Binaries vs. scripts? Why binaries?
>
> this question is answered in other Gentoo documents ... normally referred to
> as "why from source" or "source vs binaries"
I wanted to say here: A script is a list of commands, which can be
executed in some kind of runtime environment (like bash or python).
Syntax errors are found during runtime. Binaries are checked during
compilation, which translates lists of commands in computer language, so
that the computer can execute them much faster...

something like this. I did not want to compare source vs binary, yet.
>
>> What are libs and what is dynamic
>> linking vs. static linking (this is important for rebuilding your
>> system)?
>
> static vs dynamic libs is not related to Gentoo at all ... all operating
> systems deal with this question

I added this point, because of this scenario: I build a package foo,
which depends on lib bar dynamically. I rebuild foo with a newer version
of gcc but I do not rebuild lib bar. Does this break linkage? What with
a static build lib? Just wanted to point out a source of problems, which
I want to learn more about. You are of course right, that this is agian
general stuff.
>
>> Why is Gentoo source based? Why is this so cool compared to
>> binary based?
>
> same questions as what you posted first in this section
This is the part, where I wanted to compare source vs binary and you are
right, that this is already addressed in other docs, but since this is
closly connected to the toolchain, I thought to add it here. If it is
already written, than it is just copy/paste or a link to a doc.
>
>> 1) What is the toolchain aka how to build binaries?
>> <snip>
>> Put these programs in the correct order for the build process. If you
>> <snip>
>
> again, not Gentoo specific ... but might be useful i think

Yes, again learning part 1. And more important, which programs are
included in the toolchain? This is the heart of a source based distro
(in case of Gentoo Portage is the brain ;) ).

>
>> Helper programs
>> Why are they needed
>
> a better question: *what* helper programs ?

gcc depends on some programs (e.g. sed). This is what I termed helper
program. Which programs are required by the toolchain and why?

>
>> Arch-Specific
>> Why is the toolchain different from arch to arch?
>
> i dont understand this at all ... of course the toolchain will be different as
> no cpu is the same so generating code for a powerpc cpu will not work on an
> amd64
>
>> List of arch specific programs
>
> there are none
OK, so the toolchain packages (sources) of all arches are the same, but
of course the binaries are not.
>
>> 2) How does Gentoo handle the Toolchain?
>> Short discription of portage (ebuild (the script), emerge, ebuilds (the
>> build instructions), eclasses, USE-flags, ...). Perhaps something about
>> the new alternatives to emerge (or was it portage)?
>
> a high level view might be useful, but getting into the ebuild guts would not
> cater to the general user at all ... plus we toolchain guys like to rewrite
> things all the time and updating the docs for people outside of the toolchain
> herd would be a waste of our time ... if someone felt like just doing that
> though, that would be useful

Yes, I wanted a high level view here and not a replacement for the dev
handbook. Just telling the people: "Good news everybody! While handling
the toolchain on your own is quite complex, we give you this tool called
emerge, which is doing everything for you. It uses [...] And you can
configure all builds with these neat USE-flags!" This is of course
written about, but this is now the toolchain guide, so you have to write
about this.

There are also some USE-flags like "nptl", which are perhaps more
special than "gnome". They could be addressed here (or perhaps they
should be addressed in the knowledge base?).

>
>> What about compilation options (CHOST, CFLAGS and CXXFLAGS)? What to do,
>> if you want to change these? Not a complete list of options (that's
>> gnu.org for), but the general way, how Gentoo handles these, sane
>> compilation options. What to do, if you change these and why could this
>> be critical for your system? How to rebuild your system (twice? world?)?
>
> i think we already have a section touching on these topics

Um, yes, but the handbook just says "edit these and be carefull", while
the x86 Quick Install guide says "edit these, be carefull, and rebuild
your system twice to get full merrit (even if small) from these
changes". I'm looking for something final.

>
>> 3) How does Portage handle updates of Toolchain packages?
>
> people shouldnt care ... it should always "just work"

Yes, should, but I broke my system switching from gcc3.3 to gcc3.4. Of
course there is the gcc upgrade guide, which I should have read, but I
did not realize at that time, how problematic a toolchain upgrade can
be. Does this only concern gcc? Are there other packages, which could
break your system with major updates? I'm a little bit touchy (me cry
baby ;) ) since this and beside the wish to learn more about the
toolchain this topic is my most important reason for asking for a
toolchain guide: how to address correctly toolchain updates? Should I
rebuild the system? If yes, why and how (twice, ...?)? If no, why not?
Gain vs loss?

Regards
Marc
--
gentoo-doc@gentoo.org mailing list

a lot of this cant really be called "toolchain" ... it's general *nix stuff:
- basic build/compiler explanations
- binaries vs scripts
- shared vs static libraries
- how to compile/link applications
- general unix binaries
-mike

Mike Frysinger schrieb:
> a lot of this cant really be called "toolchain" ... it's general *nix stuff:
> - basic build/compiler explanations
> - binaries vs scripts
> - shared vs static libraries
> - how to compile/link applications
> - general unix binaries
> -mike

if you describe it like this, than you are right, it is too much for
just a toolchain guide. I think, I overshot my goal here, but I thought
a guide/introduction/whatever about compiling packages would be cool for
Gentoo, because this is so important for Gentoo. Of course any
distributions compiles it packages, since otherwise they could
distribute binaries, but with Gentoo you do it yourself (with much help
from the devs, who provide you with portage). It just feels a little bit
like building your own Linux, so knowing about what actually happens
would be great. Looking around the net certainly provides you with
information, but if Gentoo would provide it, it would be nicer. But I
can understand it, that this kind of guide does not have much priority...


Anyway, I would like to have some information/instructions on the
toolchain. I know you said, that emerge handles it, period. But a major
gcc update broke my system, and I think this can happen again with an
major update of every toolchain package - as far as I understand this
topic. And this is MY main problem. I just do not understand it enough,
but I would like to. Perhaps a guide is too much for this, so I should
ask at the user-list or forums? On the other hand, there are already a
lot of posts, containing the big questions: How to rebuild properly your
complete system? I think some official answer to this with an
explanation would be really appreciated not only from me, but from a lot
of Gentoo user.

Regards,
Marc

--
gentoo-doc@gentoo.org mailing list

I'd be interested in this type of guide. I'm trying to teach myself
Gentoo/*nix development and there isn't really a clear guide out
there(that I've found) to development.

I see no problem if the guide contains a lot of general information,
rather than lots of Gentoo specific information. The guide could also
be adapted in the future to include more Gentoo specific information,
if there was a need to.
--
gentoo-doc@gentoo.org mailing list

I'd be interested in this type of guide. I'm trying to teach myself
Gentoo/*nix development and there isn't really a clear guide out
there(that I've found) to development.

I see no problem if the guide contains a lot of general information,
rather than lots of Gentoo specific information. The guide could also
be adapted in the future to include more Gentoo specific information,
if there was a need to.
--
gentoo-doc@gentoo.org mailing list