Mailing List Archive

Licence compliance - capturing all source files used to make a build?
Hi, how do others handle open source licence compliance when building
some base system using gentoo?

In particular I guess simply capturing the ebuilds is not sufficient and
it's necessary to capture and distribute all the source and patch files
used to create a build. The emerge tool doesn't obviously give a way to
capture this stuff. I looked in the eclasses, particularly the epatch
file and I'm not clear that I can easily hook into that.

At the moment I'm using a bashrc file to grab everything from the build
directory. This seems reasonably robust for source files. However, for
patches I have considered creating a fake patch utility which would
record all the files it operates on. Any other suggestions? Perhaps
catalyst already has done something like that - not familiar with it though?

Whilst the above is largely targeting GPL type licences, are there other
things I should consider for other licences? Other things I need to
ensure I distribute for GPL? Any pointers to (simple) documentation on
how one can be a compliant open source citizen..?

Thanks

Ed W
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
Ed W wrote:
> Hi, how do others handle open source licence compliance when building
> some base system using gentoo?

Review the packages that get built, and adhere to their licenses. It
can be a fair bit of work.


> In particular I guess simply capturing the ebuilds is not sufficient
> and it's necessary to capture and distribute all the source and
> patch files used to create a build.

How do you build your system? If you use catalyst, the open source
gold standard citizen publishes spec files, snapshot, toolchain and
toolchain source.


> The emerge tool doesn't obviously give a way to capture this stuff.

First step is to analyze licenses. emerge does know the license for a
package, and it is available in /var/db/pkg/ after install.


> I looked in the eclasses, particularly the epatch file and I'm not
> clear that I can easily hook into that.

If you have patches which use a different license than the package
they modify then you have more work to do. Portage doesn't help here.
A good start would be to add record of all patches applied by emerge.
Indeed add it into the epatch command.


> Whilst the above is largely targeting GPL type licences, are there
> other things I should consider for other licences?

Yes. You obviously need to adhere to all licenses used by the
packages in your system. :)


> Other things I need to ensure I distribute for GPL?

Read the licenses, really.


> Any pointers to (simple) documentation on how one can be a
> compliant open source citizen..?

It's not simple. You have to learn the requirements of each license
and see if and how they allow themselves to be combined. There are
businesses doing exactly that. If you want to DIY I think you just
have to start by reading the licenses. You may or may not want an
IP lawyer sitting beside you while doing it.


//Peter
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
Hi

> It's not simple. You have to learn the requirements of each license
> and see if and how they allow themselves to be combined. There are
> businesses doing exactly that. If you want to DIY I think you just
> have to start by reading the licenses. You may or may not want an
> IP lawyer sitting beside you while doing it.

This is the kind of unhelpful answer that I can find plenty of examples
of through google...

Consider that all software comes with some kind of licence. Generally
if you ask a non opensource company about licensing costs then even the
sales droid can help you out. I do find it quite baffling that on
average if you question an opensource user then their answer on
licensing is that one should redirect the question to one of the most
expensive and opaque professions on earth... If your mate gave you that
answer in the pub when you asked what price for a beer you would
immediately cotton on that they don't really know and are bluffing...

The bit people seem to miss is that legal documents are for forcing
arbitration in the event of dispute - in the meantime people are
supposed to rub along in a cooperative manner. That many OSS advocates
seem to feel that employing expensive lawyers is the only way to talk to
them shows that they are probably missing the bigger picture...

On a more constructive note: I think I do understand the key terms of
the main software licences we use, from my understanding they are not
all that onerous. So can we perhaps move this topic onto tips,
suggestions and practical matters about moving forward? I'm not sure
that one of the most expensive type of lawyers is best employed talking
scripting tips?

> If you have patches which use a different license than the package
> they modify then you have more work to do. Portage doesn't help here.
> A good start would be to add record of all patches applied by emerge.
> Indeed add it into the epatch command.

OK, so this is what I asked the list. Please don't turn it back at me...

Firstly can we not assume that the patches in gentoo *are* in
compliance, otherwise gentoo's various packaged binaries would cause
Gentoo to be out of compliance? (I'm going to assume that human error
will cause at least some mistakes, but lets hope that just like Gentoo
isn't being sued right now, copyright holders are actually going to be
cooperative in fixing minor issues...!)


So, back to the problem: one of the bigger challenges seems to be how to
actually capture the absolute list of patches applied? Any
suggestions? I already suggested creating my own "patch" utility which
saves it's input - seems ugly - other suggestions?

I'm not using catalyst. Any tips from others on capturing, presenting,
managing and deploying GPL code?

Hoping for useful answers here rather than "talk to some really
expensive professional who knows nothing about programming".

Gentoo seems very attractive for building embedded system - however,
there seem to be some missing steps to help with deployment. I thought
that was ontopic for this list? Any tips from others who are building
things?


Cheers

Ed W
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
Ed W wrote:
>> It's not simple. You have to learn the requirements of each license
>> and see if and how they allow themselves to be combined. There are
>> businesses doing exactly that. If you want to DIY I think you just
>> have to start by reading the licenses. You may or may not want an
>> IP lawyer sitting beside you while doing it.
>
> This is the kind of unhelpful answer that I can find plenty of examples of
> through google...
>
> Consider that all software comes with some kind of licence. Generally if
> you ask a non opensource company about licensing costs then even the sales
> droid can help you out. I do find it quite baffling that on average if you
> question an opensource user then their answer on licensing is that one
> should redirect the question to one of the most expensive and opaque
> professions on earth...

That's not what I wrote, I wrote that *you* should read the licenses.

I wouldn't have mentioned an IP lawyer at all had it not been for the
fact that I know that you are in the US. :)

Lawyer or not depends on how much self education one is interested
in. I've given open source law advice to customers and have had some
contact guiding IP lawyers to better understanding. But it really is
a big field, and you weren't clear on if you already had a good
understanding of the requirements in the various licenses.


> If your mate gave you that answer in the pub when you asked what
> price for a beer you would immediately cotton on that they don't
> really know and are bluffing...

No bluff.


> The bit people seem to miss is that legal documents are for forcing
> arbitration in the event of dispute - in the meantime people are supposed
> to rub along in a cooperative manner. That many OSS advocates seem to feel
> that employing expensive lawyers is the only way to talk to them shows that
> they are probably missing the bigger picture...

Mh. A pretty picture, but the reason IP lawyers can live off open
source alone is that there are people who do not and do not want to
understand the licenses themselves. They may or may not desire to
behave well. If they do want to behave well, they may want IP lawyers
to not have to learn themselves, or because they don't trust
themselves or their understanding of the legal system. If they don't
want to behave well the IP lawyers will be working for the opposition
and have a job hunting them down. :)

I do not and have never advocated immediately talking to a lawyer
over first trying to understand licenses oneself.

That said, the legal systems of the world are such that it actually
doesn't matter what the own understanding is, the only thing that
matters is the opinions of the legal systems. And that's where the
lawyers have experience.

OTOH, I think that by now, there are enough documented cases that
allow also developers themselves to understand the issues if they
want to, and I very much encourage this.


> On a more constructive note: I think I do understand the key terms of the
> main software licences we use, from my understanding they are not all that
> onerous.

Sounds good. It is important to have gotten this right, since it is
what drives everything else.


> So can we perhaps move this topic onto tips, suggestions and
> practical matters about moving forward? I'm not sure that one of
> the most expensive type of lawyers is best employed talking
> scripting tips?

No, and I never said they were.


>> If you have patches which use a different license than the package
>> they modify then you have more work to do. Portage doesn't help here.
>> A good start would be to add record of all patches applied by emerge.
>> Indeed add it into the epatch command.
>
> OK, so this is what I asked the list. Please don't turn it back at me...
>
> Firstly can we not assume that the patches in gentoo *are* in compliance,
> otherwise gentoo's various packaged binaries would cause Gentoo to be out
> of compliance?

Hm, this last sentence is inconsistent, but I guess it should be
without the first "not" based on the rest you write.

If you dare assume that all patches use the same license as the
package they apply to, then I would say that it's actually easy
to do an inventory of the packages and licenses your system uses.

The package/license inventory is the first step.


> So, back to the problem: one of the bigger challenges seems to be how to
> actually capture the absolute list of patches applied? Any suggestions?

I think you will have to extend portage. You can have a look at PMS
(emerge app-doc/pms) to find what is currently possible. I think the
A environment variable (Table 12.1 page 46) is the closest to what
you want, but it does not seem to cover patches.


> I already suggested creating my own "patch" utility which saves it's
> input - seems ugly - other suggestions?

I gave it already in the last mail; you should modify the epatch
function to also do some bookkeeping.


> I'm not using catalyst.

Ok, then you have more manual work to do, because catalyst already
does some of the things required by e.g. GPL for you, or at least it
makes them easier to do.


> Any tips from others on capturing, presenting, managing and
> deploying GPL code?

This is a little like asking "Any tips from others on building a car
from scratch?" It's not a very practical question; it's much too
broad. I think it would be good to focus on something specific.


> Hoping for useful answers here rather than "talk to some really
> expensive professional who knows nothing about programming".

The programming involved is trivial IMO, although there are of course
pricey commercial products for software inventory and license
management out there. The difficult part is to understand the legal
requirements that create technical requirements for your distribution
of open source software.

That process obviously has nothing to do with programming, and if you
need legal advice it really is a good idea to talk to lawyers, not
programmers.

At the same time, I do advocate studying and discussing licenses.
They are not magical, I actually find them pretty straightforward.


> Gentoo seems very attractive for building embedded system - however, there
> seem to be some missing steps to help with deployment. I thought that was
> ontopic for this list? Any tips from others who are building things?

I use catalyst, and I control what gets deployed with custom ebuilds
and snapshots. The fewer packages in the final system the better;
less stuff to track.


//Peter
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
Hi

> I wouldn't have mentioned an IP lawyer at all had it not been for the
> fact that I know that you are in the US. :)

I'm in the UK

> I use catalyst, and I control what gets deployed with custom ebuilds
> and snapshots. The fewer packages in the final system the better;
> less stuff to track.

Whilst I guess it should be possible to tear apart catalyst and find out
how they do it, does anyone happen to know or have a heads up on the
code for catalyst? It must be a solved problem so I should think others
have solved this in various ways?

Thanks

Ed W
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
Hey,

Ed W wrote:
>> I wouldn't have mentioned an IP lawyer at all had it not been for the
>> fact that I know that you are in the US. :)
>
> I'm in the UK

Ha! Awesome. :) Sorry, must have mixed you up then!


>> I use catalyst, and I control what gets deployed with custom ebuilds
>> and snapshots. The fewer packages in the final system the better;
>> less stuff to track.
>
> Whilst I guess it should be possible to tear apart catalyst and find out
> how they do it, does anyone happen to know or have a heads up on the code
> for catalyst?

The catalyst code has no part in this, but it takes a portage snapshot
as one of it's inputs, and if you maintain a custom snapshot (with
only packages you need) then you know what gets used.


> It must be a solved problem so I should think others have solved
> this in various ways?

I'm not sure it is a solved problem. If you want a different solution
than basically maintaining your own portage snapshot then the easiest
way to track patches is (third time now) to add bookkeeping in the
epatch function.


//Peter
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
On 02/29/12 09:46, Ed W wrote:

> Whilst the above is largely targeting GPL type licences, are there other
> things I should consider for other licences? Other things I need to
> ensure I distribute for GPL? Any pointers to (simple) documentation on
> how one can be a compliant open source citizen..?

Ed,

It maybe worth the effort to ask your questions to other embedded lists
too, as my reading of all of these responses, makes me wonder, has not
someone else already discovered and publish a list at some point in time.

For example maybe at Linux From Scratch they advise on what softwares
and codes to use, depending on what you are building up. Or maybe
open embedded ?

It just seems like this question should be solved and already documented
somewhere? With dozens (hundreds) of commercial linux distros, surely
they list licenses for codes they include therein?

Maybe some research on what google has published on the licenses it
encounters via the Android packages? Some of the BSD embedded
projects might also be a good source of information.

hth,
James
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
wireless wrote:
> It just seems like this question should be solved and already documented
> somewhere? With dozens (hundreds) of commercial linux distros, surely
> they list licenses for codes they include therein?

The license question is easy to answer using what goes into /var/db/pkg.

The next step is obviously to make sure that you are in compliance.
Part of that means being able to sometimes provide source code for
the binaries you have distributed. If those binaries include patches
added by portage build scripts then it's not sufficient to provide
the upstream tarball.


//Peter
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
Hi

>> Whilst I guess it should be possible to tear apart catalyst and find out
>> how they do it, does anyone happen to know or have a heads up on the code
>> for catalyst?
> The catalyst code has no part in this, but it takes a portage snapshot
> as one of it's inputs, and if you maintain a custom snapshot (with
> only packages you need) then you know what gets used.
>

But not all the patches are in the portage tree? Trivial example might
be the kernel where the ebuild is tiny and references an http location
for the patches? My understanding is that for a GPL licence one should
provide a copy of these patches in the "code dump", not just an http
link? Is that your understanding?

So by implication it's not clear that catalyst does satisfy your GPL
requirements for distribution?

I suspect something more is probably happening, eg some of the linked
patches probably get included into the source download location and
probably you can pick them up there - however, there are now a LOT of
ways to fetching source and patches and it would be hard to be sure of
100% coverage?

Has someone done some actual probing on this? Peter what does catalyst
provide for say gcc/kernel sources in it's source output? All the patches?

Cheers

Ed W
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
Ed W wrote:
>>> Whilst I guess it should be possible to tear apart catalyst and find out
>>> how they do it, does anyone happen to know or have a heads up on the code
>>> for catalyst?
>>
>> The catalyst code has no part in this, but it takes a portage snapshot
>> as one of it's inputs, and if you maintain a custom snapshot (with
>> only packages you need) then you know what gets used.
>
> But not all the patches are in the portage tree? Trivial example might
> be the kernel where the ebuild is tiny and references an http location
> for the patches?

Then you would change the kernel ebuild in your snapshot, so that it
becomes self-contained.

For the specific example of the kernel you could of course just pick
vanilla-sources, but the issue is real.


> My understanding is that for a GPL licence one should provide a
> copy of these patches in the "code dump", not just an http link?
> Is that your understanding?

I think your understanding is incomplete, and I recommend that you
read through the license again.

There isn't just a single way to provide the source, but yes, if you
have downloaded and included a patch in your binary, then you have to
provide that patch yourself, because if you refer to someone else and
they stop providing the patch you would no longer be in compliance.


> So by implication it's not clear that catalyst does satisfy your GPL
> requirements for distribution?

I never say it did. I said that it helps with some things.


> I suspect something more is probably happening, eg some of the linked
> patches probably get included into the source download location and
> probably you can pick them up there - however, there are now a LOT of
> ways to fetching source and patches and it would be hard to be sure
> of 100% coverage?

Fourth time: Add bookkeeping into the epatch function.

Downloading is irrelevant, especially since sometimes many more
patches are downloaded than are actually applied.


> Has someone done some actual probing on this? Peter what does catalyst
> provide for say gcc/kernel sources in it's source output? All the patches?

It's the other way around:

You provide a snapshot to catalyst, and catalyst builds kernel from
that. You say what you want catalyst to build, and you create the
package.

You may end up doing more ebuild maintenance, but you likely want to
do just that anyway, in order to keep track of what actually goes
into your system.


//Peter
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
On Wednesday 29 February 2012 09:46:57 Ed W wrote:
> In particular I guess simply capturing the ebuilds is not sufficient and
> it's necessary to capture and distribute all the source and patch files
> used to create a build. The emerge tool doesn't obviously give a way to
> capture this stuff.

file a bug report to add a feature to do this ... something like "buildsrcpkg".
it'd automatically bundle up all the eclasses the pkg is using as well as all
of $CATEGORY/$PN/.

> At the moment I'm using a bashrc file to grab everything from the build
> directory. This seems reasonably robust for source files. However, for
> patches I have considered creating a fake patch utility which would
> record all the files it operates on. Any other suggestions? Perhaps
> catalyst already has done something like that - not familiar with it
> though?

if you capture all of the $PORTDIR/$CATEGORY/$PN/ and $A, then there should be
no need to manually hook into epatch to capture the patches. there's really
no other place these could come from.
-mike
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
Mike Frysinger wrote:
> if you capture all of the $PORTDIR/$CATEGORY/$PN/ and $A, then
> there should be no need to manually hook into epatch

The point of hooking into epatch would be to only have exactly those
patches which get applied. Some ebuilds come with a huge set of
patches, but only few may be applied depending on USE and version.
It's nice to have just the right ones.


//Peter
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
On 02.03.2012 15:35, Peter Stuge wrote:
> Mike Frysinger wrote:
>> if you capture all of the $PORTDIR/$CATEGORY/$PN/ and $A, then
>> there should be no need to manually hook into epatch
>
> The point of hooking into epatch would be to only have exactly those
> patches which get applied. Some ebuilds come with a huge set of
> patches, but only few may be applied depending on USE and version.
> It's nice to have just the right ones.

epatch is not the only necessary thing, some ebuilds do 'sed -i' on
files. I don't really know for autotools files.

An extension about Mike mind can be an automagic diff between all
SRC_URI freshly src_unpack(ed) and after install+distclean or after
src_prepare. Assuming that not any other code is modified during
src_(compile|install).
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
Hi

>> But not all the patches are in the portage tree? Trivial example might
>> be the kernel where the ebuild is tiny and references an http location
>> for the patches?
> Then you would change the kernel ebuild in your snapshot, so that it
> becomes self-contained.

That's clearly not a practical suggestion because there are many such
ebuilds with this behaviour and the suggestion to "rewrite all your
ebuilds" kind of defeats the benefit of using gentoo?


>> My understanding is that for a GPL licence one should provide a
>> copy of these patches in the "code dump", not just an http link?
>> Is that your understanding?
> I think your understanding is incomplete, and I recommend that you
> read through the license again.

?? Why all the stupid hints rather than just stating the answer!

Under what circumstances do you claim that it's not necessary to
actually supply the code for a patch which has been made to a GPL
licenced code base?? I think you are implying that it's satisfactory to
"supply" code by having a twisted and nested chain of source locations
for all the code, some of which may not be under my control? As you
hint, I then have the risk of servers outside of my control causing my
compliance failure. However, this is all moot because my whole question
is about accurately capturing all the upstream source so that I can
maintain my own cache?


I'm not sure why GPL seems to attract such special behaviour. In every
other industry one will usually provide both a legal licence and also a
non legal "summary of intent". For some reason the open source
advocates seem to excel in leaping on any minor misunderstanding of
their licensing agreements, but then enjoy confounding the situation
with "nah that's not it, but I can't give you any hints as to why I
*think* you are wrong...". Look it's just a straightforward licence -
we don't need to be lawyers to have a stab at complying with it and
generally helping with understanding it's nuances...

The big thing which annoys me is that one can comply with to the letter
of the GPL with a big code dump that, and lets be honest here, benefits
absolutely no one really (what do you do with a lump of undocumented and
obfuscated hacky code. There are several open letters on the internet
discussing this, but what you are looking for is people to get involved
with the *spirit* of working within the open source process and sharing
in a useful way, not just code dumping.

The piece we are discussing here is really the boring compliance piece
which personally I think is largely unhelpful, last chance saloon kind
of code dump. All the useful pieces of code I try to push upstream.
For sure the GPL provision at least means you get the code even if *I*
don't try and push it upstream and am uncooperative, but really, for the
vast majority of code, it's just boiler plate reproduction of stuff that
you would get from upstream if you needed it anyway...


>> So by implication it's not clear that catalyst does satisfy your GPL
>> requirements for distribution?
> I never say it did. I said that it helps with some things.

What "some things"? Previously I asked for help capturing the source
code tree and you implied that it would be correctly captured by
catalyst - however, now it seems to be becoming clear that catalyst
doesn't capture all the patches either? So we seem to be back to the
original question again and catalyst seem to be just a detour that
hasn't advanced us?

With that in mind if you are using only catalyst, how do *you* make sure
you are GPL compliant and provide all patches/sources, etc? (Not a
challenge, just genuinely trying to learn from how others are doing things?)



>> I suspect something more is probably happening, eg some of the linked
>> patches probably get included into the source download location and
>> probably you can pick them up there - however, there are now a LOT of
>> ways to fetching source and patches and it would be hard to be sure
>> of 100% coverage?
> Fourth time: Add bookkeeping into the epatch function.

No, it's not "fourth time". It was my idea in the original email!
However, patching portage is unsatisfactory in that it's fragile and
easily overwritten accidently. By all means if you have a way of
patching which is less fragile, eg if there is some way to patch the
eclass using some overlay in /usr/local/portage then I would be grateful
for *that* information.

you are just saying "do it" like having the idea is the easy bit!
Actually the implementation seems hacky to me. Wrapping the patch
utility seems more robust to me, but it's still not ideal...

> Downloading is irrelevant, especially since sometimes many more
> patches are downloaded than are actually applied.

I'm not sure I follow? My understanding is that we need to supply
patches that are applied, not just every patch to every ebuild - I think
we are agreeing on that?


> It's the other way around:
>
> You provide a snapshot to catalyst, and catalyst builds kernel from
> that. You say what you want catalyst to build, and you create the
> package.
>
> You may end up doing more ebuild maintenance, but you likely want to
> do just that anyway, in order to keep track of what actually goes
> into your system.

Hmm, that's a very superficial description of what is done, but I can
infer some of what you mean.

You might be saying that you figure out every ebuild that you need in
your solution, then manually patch them all to use source pulled down
from your own server, plus sync all the sources from gentoo to yoru own
server? However, this seems like a desperate amount of work?

You might be saying you just snapshot the gentoo portage tree, however,
I don't see how that helps you capture all the sources and patches
correctly?


Can you please clarify how you generate your portage snapshot for
catalyst and how you create your own offline snapshot of all sources
(including downloaded patches) - this is I think is what I'm looking to
learn?

Thanks

Ed W
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
On 02/03/2012 15:22, Bertrand Jacquin wrote:
> On 02.03.2012 15:35, Peter Stuge wrote:
>> Mike Frysinger wrote:
>>> if you capture all of the $PORTDIR/$CATEGORY/$PN/ and $A, then
>>> there should be no need to manually hook into epatch
>>
>> The point of hooking into epatch would be to only have exactly those
>> patches which get applied. Some ebuilds come with a huge set of
>> patches, but only few may be applied depending on USE and version.
>> It's nice to have just the right ones.
>
> epatch is not the only necessary thing, some ebuilds do 'sed -i' on
> files. I don't really know for autotools files.

Hmm, this is an interesting thought.

My instinct would be to consider this under the heading of "build
recipe" since it's arguably similar to what the makefile and other
pre-processors are doing. I don't disagree that someone could argue
this all kinds of ways, but I think you would have to be fairly bloody
minded to try for an infringement claim if the ebuild were provided
(since arguably the patch is there)?

It also occurs to me that it's safer to include all of $ FILESDIR, or at
least everything without a .patch extension, since there is also "cp
$FILESDIR/somefile $S" to watch for?

> An extension about Mike mind can be an automagic diff between all
> SRC_URI freshly src_unpack(ed) and after install+distclean or after
> src_prepare. Assuming that not any other code is modified during
> src_(compile|install).

A very simple solution is to diff the code - however, I claim this is
*incredibly* unhelpful to the whole notion of open source. If I were
the copyright holder then I would far rather have a cooperative, but
accidentally non compliant distributor who has genuinely made an effort,
than someone who just provides a massive code diff... (Yeah, probably
some corner case in that argument, but the point is that patches like
"fix segfault on touching some file" are infinitely more useful than a
massive diff...)


Thanks for the feedback

Ed W
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
On 02/03/2012 06:37, Mike Frysinger wrote:
> On Wednesday 29 February 2012 09:46:57 Ed W wrote:
>> In particular I guess simply capturing the ebuilds is not sufficient and
>> it's necessary to capture and distribute all the source and patch files
>> used to create a build. The emerge tool doesn't obviously give a way to
>> capture this stuff.
> file a bug report to add a feature to do this ... something like "buildsrcpkg".
> it'd automatically bundle up all the eclasses the pkg is using as well as all
> of $CATEGORY/$PN/.
>

Submitted

https://bugs.gentoo.org/show_bug.cgi?id=406811

Thanks

Ed W
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
* Ed W <lists@wildgooses.com> [120303 13:29]:
> Hi
>
[..]

> >> My understanding is that for a GPL licence one should provide a
> >> copy of these patches in the "code dump", not just an http link?
> >> Is that your understanding?
> > I think your understanding is incomplete, and I recommend that you
> > read through the license again.
>
> ?? Why all the stupid hints rather than just stating the answer!

I'm sure it's just frustration but your emails really make you sound
like an a-hole with a sense of entitlement.

They're legal licenses. As with anything involved with lawyers and the
legal system you really need to decide for yourself what needs to be
done (most people to be safe would contact a lawyer.)

If you're at a company releasing a product then the company most likely
has a legal dept or legal consultant. They certainly would here in the
US (I know you said you're not in the US so perhaps that's not the
case.)

Have you looked at http://www.gnu.org/licenses/gpl-faq.html?

I'm not a lawyer (by a far shot) but what's the problem with creating a
script that when run pulls the upstream files from
/usr/portage/distfiles, the files and ebuilds from /usr/portage for
whatever packages you have installed on whatever you're releasing?

If I were releasing commercial software I'd want all that on a local
mirror (in source control too) so that I could recreate any released
versions.

Todd
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
On Thursday 01 March 2012 14:05:36 Peter Stuge wrote:
> Ed W wrote:
> > My understanding is that for a GPL licence one should provide a
> > copy of these patches in the "code dump", not just an http link?
> > Is that your understanding?
>
> I think your understanding is incomplete, and I recommend that you
> read through the license again.
>
> There isn't just a single way to provide the source, but yes, if you
> have downloaded and included a patch in your binary, then you have to
> provide that patch yourself, because if you refer to someone else and
> they stop providing the patch you would no longer be in compliance.

Peter's understanding seems to match my own. pointing someone to a URL that
provides the source satisfies the GPL requirements. obviously if the linked
source is incomplete or outdated, that's another matter, but if the full
source is there, then the obligations have been met.

alternatively, you could create a bundle of all the sources and provide it
directly. companies tend to do this more because they modify the releases
directly rather than a cleaner build approach.
-mike
Re: Licence compliance - capturing all source files used to make a build? [ In reply to ]
Hi

> They're legal licenses. As with anything involved with lawyers and the
> legal system you really need to decide for yourself what needs to be
> done (most people to be safe would contact a lawyer.)

Thanks for your feedback - however, again you are mentioning lawyers.
My understanding is that even in the USA, lawyers aren't the best people
write build scripts?

The (main) question is a technical one, not a legal one. I think we are
all clear that the general provision is to supply all code to downstream
users. If you check my original question, I'm looking for technical
suggestions to wrap portage and ensure that all patches, source files,
etc are supplied in a neat and useful form (I'm trying to avoid the big
code dump and keep things as well documented as possible)

That said I'm grateful for the clarifications on interpretations of the
GPL such as with regards to linking to other sources of code. Thanks
(Mike/Peter?)

> If you're at a company releasing a product then the company most likely
> has a legal dept or legal consultant. They certainly would here in the
> US (I know you said you're not in the US so perhaps that's not the
> case.)

In the UK you would generally be quite a large company to have an
expensive person such as a lawyer on your book full time (or have some
special reason to be needing one every day...). Generally we just rent
them as they are needed. Same also for accountants, you only need them
once a year, so rent them rather than owning them full time...


> I'm not a lawyer (by a far shot) but what's the problem with creating a
> script that when run pulls the upstream files from
> /usr/portage/distfiles, the files and ebuilds from /usr/portage for
> whatever packages you have installed on whatever you're releasing?

Please see original question. Yes, this is basically what I'm trying to do

However, it's not quite as straightforward as you state. For a build
process you would need to clear distfiles at some point, then scrape it
later in order to infer what some package had used.

However, yes, you are on the right lines for the kind of thing I'm
looking for. Note that it's the corner cases which are important, hence
the reason for my question (I really don't want to learn about how many
lawyers every US company owns...)


> If I were releasing commercial software I'd want all that on a local
> mirror (in source control too) so that I could recreate any released
> versions.

Agreed. See - my question was sensible! I'm using git for all that
right now.

However, I think you are being a bit handwaving about the details. For
example, how would you handle this in practice, for example we need to
clean distfiles before we start building a fresh image otherwise we
don't know what is new, on the flip side we don't want to download some
existing file which is unchanged and already in source control? So a
kind of two level distfiles would be helpful...

At the moment I'm tackling it a slightly different way by grabbing $A
and the ./files dir during the build, via a bashrc script (basically as
suggested by Mike Frysinger). I think I'm coming to the conclusion that
this is the most complete solution, but obviously grabs additional
pieces that might not be used.

However, please, if anyone else has tackled this and has some notes then
please share? Some technical challenges have been exposed from some of
the answers so far.

Cheers

Ed W