Mailing List Archive

On 3/10/15, Frank Peters <frank.peters@comcast.net> wrote:
> Ffmpeg-2.6 fails to emerge with the following error:
>
> The following REQUIRED_USE flag constraints are unsatisfied:
> cpu_flags_x86_sse? ( cpu_flags_x86_mmxext )
>
>
> I can fix this by including "mmxext" in the CPU_FLAGS_X86
> variable of make.conf.
>
> But my processor is a Core i7 and has no mmxext as seen
> in /proc/cpuinfo:
>
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
> cmov
> pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp
> lm
> constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc
> aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm
> sse4_1 sse4_2 popcnt lahf_lm dtherm tpr_shadow vnmi flexpriority ept vpid
>
>
> What is going on? Is the ebuild file wrong?
>
> Looking at the ffmpeg ebuild directory, I see that only the
> ffmpeg-2.6.ebuild
> uses the X86_CPU_REQUIRED_USE variable. Maybe the maintainer got it wrong.
>
> Frank Peters
>
>
>
Did you use that new script for setting the flags? I installed an
update of ffmpeg today on my i7 980X with no issues. Not sure if it's
the same version as I am replying from my Kindle.

Mark

On Tue, 10 Mar 2015 16:49:01 -0700
Mark Knecht <markknecht@gmail.com> wrote:

>
> Did you use that new script for setting the flags? I installed an
> update of ffmpeg today on my i7 980X with no issues. Not sure if it's
> the same version as I am replying from my Kindle.
>

Do you mean that python script called "cpuinfo2cpuflags-x86."

If so, I ran it and it recommended that the mmxext flag should be
included in make,conf.

But I didn't believe it because I knew that the Core i7 does not
have mmxext (or mmx2). If it did, it would show up in /proc/cpuinfo.

However, I could be missing something someplace.

Frank Peters

On Tue, Mar 10, 2015 at 5:00 PM, Frank Peters <frank.peters@comcast.net> wrote:
> On Tue, 10 Mar 2015 16:49:01 -0700
> Mark Knecht <markknecht@gmail.com> wrote:
>
>>
>> Did you use that new script for setting the flags? I installed an
>> update of ffmpeg today on my i7 980X with no issues. Not sure if it's
>> the same version as I am replying from my Kindle.
>>
>
> Do you mean that python script called "cpuinfo2cpuflags-x86."
>
> If so, I ran it and it recommended that the mmxext flag should be
> included in make,conf.
>
> But I didn't believe it because I knew that the Core i7 does not
> have mmxext (or mmx2). If it did, it would show up in /proc/cpuinfo.
>
> However, I could be missing something someplace.
>
> Frank Peters
>
>

OK, so I got back on the machine and I do have it in make.conf here:

mark@c2RAID6 ~ $ cat /etc/portage/make.conf | grep CPU_FLAGS
#CPU_FLAGS_X86="mmx sse sse2 ssse3"
CPU_FLAGS_X86="aes mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3"
mark@c2RAID6 ~ $

and I guess like you I do not see it in /proc/cpuinfo:

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts
rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64
monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2
popcnt aes lahf_lm ida arat epb dtherm tpr_shadow vnmi flexpriority
ept vpid

It appears more study is needed. I've had no problems, as best I can
tell, with ffmpeg and these settings.

Cheers,
Mark

On Tue, 10 Mar 2015 17:17:11 -0700
Mark Knecht <markknecht@gmail.com> wrote:

>
> I've had no problems, as best I can
> tell, with ffmpeg and these settings.
>

Well, if the use of the mmxext flag leads to instructions in
the compiled code which the cpu does not have, then CRASH.

But the SSE instruction sets supersede MMX (and mmxext) and so the
mmxext flag is really unnecessary on Core i7 and many newer processors.
(Ffmpeg may have reasons for still using MMX but I don't know.)

Frank Peters

On Tue, Mar 10, 2015 at 5:27 PM, Frank Peters <frank.peters@comcast.net> wrote:
> On Tue, 10 Mar 2015 17:17:11 -0700
> Mark Knecht <markknecht@gmail.com> wrote:
>
>>
>> I've had no problems, as best I can
>> tell, with ffmpeg and these settings.
>>
>
> Well, if the use of the mmxext flag leads to instructions in
> the compiled code which the cpu does not have, then CRASH.
>
> But the SSE instruction sets supersede MMX (and mmxext) and so the
> mmxext flag is really unnecessary on Core i7 and many newer processors.
> (Ffmpeg may have reasons for still using MMX but I don't know.)
>
> Frank Peters

True, but I don't think it's likely as the old mmxext flags (AMD only
possibly?) became a subset of SSE. That said, report the bug and copy
us on the report. I'd be interested in hearing what the developers
say.

Cheers,
Mark

Mark Knecht posted on Tue, 10 Mar 2015 17:40:33 -0700 as excerpted:

> On Tue, Mar 10, 2015 at 5:27 PM, Frank Peters <frank.peters@comcast.net>
> wrote:
>> On Tue, 10 Mar 2015 17:17:11 -0700 Mark Knecht <markknecht@gmail.com>
>> wrote:
>>
>>
>>> I've had no problems, as best I can tell, with ffmpeg and these
>>> settings.
>>>
>>>
>> Well, if the use of the mmxext flag leads to instructions in the
>> compiled code which the cpu does not have, then CRASH.
>>
>> But the SSE instruction sets supersede MMX (and mmxext) and so the
>> mmxext flag is really unnecessary on Core i7 and many newer processors.
>> (Ffmpeg may have reasons for still using MMX but I don't know.)
>>
>> Frank Peters
>
> True, but I don't think it's likely as the old mmxext flags (AMD only
> possibly?) became a subset of SSE. That said, report the bug and copy us
> on the report. I'd be interested in hearing what the developers say.

AFAIK (and verified by wikipedia, see the links below), all amd64 aka
x86_64 CPUs ship with mmx/mmxext/sse/sse2. It was part of the base amd64
spec, and as such, mmxext should be considered available on any amd64
based machine.

More specifically, according to wikipedia, AMD's second generation 3DNow
(called various things, there was apparently never an official name) was
introduced in the first-gen Athlon CPUs. The combined 24 new
instructions were segregated into 5 new 3DNow instructions and 19 new MMX
instructions in tech manuals, and some pre-amd64 CPUs actually shipped
with the 5 additional 3DNow instructions only, while customer-focused
literature tended to blur the lines and refer to the 24 instructions
combined as enhanced 3DNow technology.

Meanwhile, the 19 new MMX instructions were a subset of Intel's SSE (v1)
instruction set. This SSE subset of 19 instructions is apparently what
mmxext actually refers to. Thus, anything with sse in the cpuflags by
definition has mmxext as well.

So you can actually verify your CPU mmxext safe using two different
methods. (1) It's in the amd64/x86_64 baseline definition, so if your
machine is amd64/X86_64, it's automatically mmxext. (2) SSE supersets
mmxext, so if sse appears in the cpuflags, it's automatically mmxext as
well.


https://en.wikipedia.org/wiki/3DNow!#3DNow_extensions

(In particular, see the 3DNow or MMX Extensions paragraph.)

https://en.wikipedia.org/wiki/X86-64#Architectural_features

(In particular, see the SSE Instructions paragraph.)


--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

On Wed, 11 Mar 2015 03:25:24 +0000 (UTC)
Duncan <1i5t5.duncan@cox.net> wrote:

>
> Meanwhile, the 19 new MMX instructions were a subset of Intel's SSE (v1)
> instruction set. This SSE subset of 19 instructions is apparently what
> mmxext actually refers to. Thus, anything with sse in the cpuflags by
> definition has mmxext as well.
>

These flags have to be maintained for backwards compatibility,
I suppose, even though mmxext no longer makes sense on newer
processors.

Also, why the need to even specify the string of "sse sse2 sse3 sse4_1
sse4_2 ssse3?" Each successive SSEX includes all previous instructions
so only the highest "X" for a given processor would be appropriate.

Frank Peters

Frank Peters posted on Wed, 11 Mar 2015 00:23:03 -0400 as excerpted:

> On Wed, 11 Mar 2015 03:25:24 +0000 (UTC)
> Duncan <1i5t5.duncan@cox.net> wrote:
>
>
>> Meanwhile, the 19 new MMX instructions were a subset of Intel's SSE
>> (v1)
>> instruction set. This SSE subset of 19 instructions is apparently what
>> mmxext actually refers to. Thus, anything with sse in the cpuflags by
>> definition has mmxext as well.
>>
>>
> These flags have to be maintained for backwards compatibility,
> I suppose, even though mmxext no longer makes sense on newer processors.
>
> Also, why the need to even specify the string of "sse sse2 sse3 sse4_1
> sse4_2 ssse3?" Each successive SSEX includes all previous instructions
> so only the highest "X" for a given processor would be appropriate.

I believe you're correct in general, in terms of keeping the strings
around for backward compatibility.

Tho I'm not sure you're /entirely/ correct on each sse generation exactly
supersetting the previous. In particular, I believe there's some set-
exclusions in the sse4 and ssse3 areas on specific CPUs. Additionally,
there's cases where particular cpu's technically implement the code, but
where those implementations aren't well optimized, such that if one cares
enough to do the research or to actually do profiled optimization
testing, certain instructions won't be used as they're actually slower
than the alternatives. If a package uses these instructions, then,
disabling the specific cpuflag for it when building that package can be a
viable option, even with later cpuflags enabled. Were cpuflags always
treated as supersets of the previous, that level of previous cpuflag
disabling wouldn't be possible.


Meanwhole, concerning mmxext specifically...

I believe the distinction with mmxext is that (as I mentioned in the
previous message) it never actually had an official name on its own. AMD
blurred the lines both on what it included and on the name itself, and
Intel apparently never adopted it in the first place, moving directly
from mmx to sse. And since sse WAS an official and widely adopted name
and included mmxext, mmxext basically got dropped by the wayside...
except for the fringe "extreme technophile" class that builds from
sources, who by definition are supposed to either know or be able to
lookup this sort of thing in short order.

IOW, basically, nobody else really cares. The masses get pre-built
binaries that either process things efficiently (using the appropriate
instructions but the masses don't know or care about that) or not, so
they don't care. And most of the developer masses don't care either,
since it never applied to Intel at all. Only a very few developers who
happened to have AMD machines in the slot between mmx only and full sse,
and knew they could make stuff like media encoding go faster with
appropriate hand-coded assembly using these instructions, thus having
that itch to scratch, and as a consequence, the very few gentooers and
others building from sources the apps these developers created, had real
reason to care.

Meanwhile, arguably, since sse is known to superset mmxext, the gentoo
ffmpeg maintainer got this wrong, and instead of setting up the required-
use like that, he should have simply auto-enabled mmxext if sse was
enabled. I imagine I might have coded it up like that, anyway, such that
the few people only having mmxext could enable it, while the others could
enable sse and not have to futz around with the mmxext corner case at all.

If one cared enough about it, I suppose one could file a bug to that
effect. However, now that the existing implementation is there and
configured correctly works, I don't know that it's worth bothering with.
And the maintainer may likewise consider it not worth his time...

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman