Mailing List Archive

strict-aliasing rules, don't break them
You may aware or not that there is a nice optimization (more effective
if you have some registers to spare) based on how you access memory
thought variables. Since it wasn't that effective and didn't break
anything it is enabled since -O2.

Problem: recent gcc are doing some quite smart tricks with aliasing and
as result they will break in subtle way your code if you use
strict-aliasing rules optimization on type puns.

Quick solution: enforce -fno-strict-aliasing as global cflag.

Side effect: you may lose a bit in performance, but better safe than
sorry: the first package showing issues was openssl[1] mismatching
hashes, guess how discovered that and how (hint: ssh was refusing logins...)

Long term solution:
1- check your new package for aliasing compliance, and if you have time
fix it in the code or in the makefile, if you haven't append
-fno-strict-aliasing to the cflags and maybe send a notice about it upstream

2- append -fno-strict-aliasing to every source known to have such issue.

I'll do 2 on all packages in the tree showing the issue if you think is
ok, arches not yet affected will be in the future.

lu

[1]http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14725


--

Luca Barbato

Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

--
gentoo-dev@gentoo.org mailing list
Re: strict-aliasing rules, don't break them [ In reply to ]
On Saturday 17 June 2006 12:17, Luca Barbato wrote:
> Long term solution:
The best long term solution would have been to fix the code, but actually I
didn't ever found a quick explanation of how to fix this kind of code...

--
Diego "Flameeyes" Pettenò - http://farragut.flameeyes.is-a-geek.org/
Gentoo/Alt lead, Gentoo/FreeBSD, Video, AMD64, Sound, PAM, KDE
Re: strict-aliasing rules, don't break them [ In reply to ]
Diego 'Flameeyes' Pettenò wrote:
> On Saturday 17 June 2006 12:17, Luca Barbato wrote:
>> Long term solution:
> The best long term solution would have been to fix the code, but actually I
> didn't ever found a quick explanation of how to fix this kind of code...
>

you can use unions or rewrite completely the line using it in another
way, in certain case the type pun is the quickest solution so it's
better to append -fno-strict-aliasing in the Makefile.

lu

--

Luca Barbato

Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

--
gentoo-dev@gentoo.org mailing list
Re: strict-aliasing rules, don't break them [ In reply to ]
On Saturday 17 June 2006 13:43, Luca Barbato wrote:
> you can use unions or rewrite completely the line using it in another
> way, in certain case the type pun is the quickest solution so it's
> better to append -fno-strict-aliasing in the Makefile.
Err give me an example of the line, a lot of strict aliasing breakage was due
to double pointers.

--
Diego "Flameeyes" Pettenò - http://farragut.flameeyes.is-a-geek.org/
Gentoo/Alt lead, Gentoo/FreeBSD, Video, AMD64, Sound, PAM, KDE
Re: strict-aliasing rules, don't break them [ In reply to ]
On Sat, Jun 17, 2006 at 01:57:33PM +0200, Diego 'Flameeyes' Pettenò wrote:
> On Saturday 17 June 2006 13:43, Luca Barbato wrote:
> > you can use unions or rewrite completely the line using it in another
> > way, in certain case the type pun is the quickest solution so it's
> > better to append -fno-strict-aliasing in the Makefile.
> Err give me an example of the line, a lot of strict aliasing breakage was due
> to double pointers.

Using your own example from
http://planet.gentoo.org/developers/flameeyes/2006/03/02/fixing_strict_aliasing_warnings

struct dl_node {
struct dl_node *next;
struct dl_node *prev;
};
struct dl_head {
struct dl_node *first;
struct dl_node *null;
struct dl_node *last;
};
and then accessing {first, null}, or {null, last} as a struct dl_node
the way to fix the code would be to rewrite the code to use arrays
of pointers instead of simply three pointer members. There is no
valid way to do what the code does using pointer members (that's simple
logic: it relies on the padding between first and null to be exactly the
same as that between null and last, which is a guarantee standard C
doesn't make), so fixing it requires quite a bit of work.

If you don't mind keeping the code invalid, but in such a way that GCC
will compile it as intended, you could do

struct dl_head {
union {
struct {
struct dl_node *first;
struct dl_node *null;
struct dl_node *last;
};
struct {
struct dl_node firstnode;
struct dl_node *dummy1;
};
struct {
struct dl_node *dummy2;
struct dl_node secondnode;
};
};
} h;

and then change
(struct dl_node *) &h->first
to
&h->firstnode

and similarly, change
(struct dl_node *) &h->null
to
&h->secondnode

Again, it's not valid, and it can still break if not handled with care
even with GCC.
--
gentoo-dev@gentoo.org mailing list
Re: strict-aliasing rules, don't break them [ In reply to ]
On Sat, 17 Jun 2006 15:20:52 +0200 Harald van Dijk <truedfx@gentoo.org>
wrote:
| and then accessing {first, null}, or {null, last} as a struct dl_node
| the way to fix the code would be to rewrite the code to use arrays
| of pointers instead of simply three pointer members. There is no
| valid way to do what the code does using pointer members (that's
| simple logic: it relies on the padding between first and null to be
| exactly the same as that between null and last, which is a guarantee
| standard C doesn't make), so fixing it requires quite a bit of work.

It can be done using nested structs.

--
Ciaran McCreesh
Mail : ciaran dot mccreesh at blueyonder.co.uk


--
gentoo-dev@gentoo.org mailing list
Re: strict-aliasing rules, don't break them [ In reply to ]
Easiest way to find these is to stick -Wall in global CFLAGS (or just
-Wstrict-aliasing if you want to be more specific), and grep the build
log for 'will break strict aliasing' (LC_ALL=C obviously). Such
issues should be filed upstream, I guess, to either get the Makefile to
forcibly set -fno-strict-aliasing or to rework the code.

FWIW here's a list of packages that show the warning, from my build logs
for packages I have installed:

app-antivirus/clamav-0.88.2
app-editors/vim-6.4
app-text/enscript-1.6.4-r2
app-text/ghostscript-esp-8.15.1_p20060430
dev-db/mysql-4.1.20
dev-java/gnu-classpath-0.90
dev-lang/nasm-0.98.39-r3
dev-lang/ocaml-3.08.4
dev-libs/cyrus-sasl-2.1.21-r2
dev-libs/glib-2.8.6
dev-libs/nspr-4.6.1-r2
dev-libs/nss-3.11-r1
dev-tcltk/itcl-3.3-r1
dev-util/valgrind-3.1.1
gnome-base/orbit-0.5.17-r1
kde-base/kdm-3.5.1
kde-base/kopete-3.5.1-r1
media-libs/faad2-2.0-r11
media-libs/fontconfig-2.2.3
media-libs/libsdl-1.2.9-r1
media-libs/xine-lib-1.1.2_pre20060328-r9
media-sound/audacious-0.2.3
media-video/vlc-0.8.5-r1
net-dialup/ppp-2.4.2-r15
net-fs/samba-3.0.22
net-irc/bitchx-1.1-r2
net-nds/openldap-2.3.24-r1
sys-apps/coreutils-5.94-r1
sys-apps/dbus-0.60-r4
sys-apps/shadow-4.0.15-r2
sys-devel/autogen-5.7.1
sys-devel/gcc-* (libjava)
sys-devel/gdb-6.3-r5
sys-fs/e2fsprogs-1.38-r1
sys-libs/slang-1.4.9-r2
x11-apps/xdm-1.0.4
x11-libs/gtk+-2.8.12
x11-libs/gtkglarea-1.2.3-r1
x11-libs/libXScrnSaver-1.1.0
x11-libs/libXaw-1.0.2
x11-libs/libXt-1.0.2
x11-wm/twm-1.0.1

Whether any of these actually trigger real problems or not I don't
know; but then probaly neither do the upstream developers...

--
Kevin F. Quinn
Re: strict-aliasing rules, don't break them [ In reply to ]
On Sat, Jun 17, 2006 at 03:32:36PM +0200, Kevin F. Quinn wrote:
> Easiest way to find these is to stick -Wall in global CFLAGS (or just
> -Wstrict-aliasing if you want to be more specific), and grep the build
> log for 'will break strict aliasing' (LC_ALL=C obviously).

That warning is given for valid code too. Please only add
-fno-strict-aliasing if you actually find a package misbehaves without
it, or if you have verified that there is indeed an aliasing violation
in the code.
--
gentoo-dev@gentoo.org mailing list
Re: strict-aliasing rules, don't break them [ In reply to ]
On Sat, Jun 17, 2006 at 02:28:28PM +0100, Ciaran McCreesh wrote:
> On Sat, 17 Jun 2006 15:20:52 +0200 Harald van Dijk <truedfx@gentoo.org>
> wrote:
> | and then accessing {first, null}, or {null, last} as a struct dl_node
> | the way to fix the code would be to rewrite the code to use arrays
> | of pointers instead of simply three pointer members. There is no
> | valid way to do what the code does using pointer members (that's
> | simple logic: it relies on the padding between first and null to be
> | exactly the same as that between null and last, which is a guarantee
> | standard C doesn't make), so fixing it requires quite a bit of work.
>
> It can be done using nested structs.

No, it can't: the two dl_nodes overlap.
--
gentoo-dev@gentoo.org mailing list
Re: strict-aliasing rules, don't break them [ In reply to ]
Harald van Dijk wrote:

> That warning is given for valid code too. Please only add
> -fno-strict-aliasing if you actually find a package misbehaves without
> it, or if you have verified that there is indeed an aliasing violation
> in the code.

Or just bug upstream to either avoid ugly code or just use
-fno-strict-aliasing when needed.

lu

--

Luca Barbato

Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

--
gentoo-dev@gentoo.org mailing list
Re: strict-aliasing rules, don't break them [ In reply to ]
On Saturday 17 June 2006 06:17, Luca Barbato wrote:
> Long term solution:
> 1- check your new package for aliasing compliance, and if you have time
> fix it in the code or in the makefile, if you haven't append
> -fno-strict-aliasing to the cflags and maybe send a notice about it
> upstream
>
> 2- append -fno-strict-aliasing to every source known to have such issue.

3 - include checking in portage
http://bugs.gentoo.org/111436
-mike
Re: strict-aliasing rules, don't break them [ In reply to ]
some more docs:
http://gcc.gnu.org/bugs.html#nonbugs_c
Casting does not work as expected when optimization is turned on
-mike
Re: strict-aliasing rules, don't break them [ In reply to ]
On Sat, Jun 17, 2006 at 01:15:58PM -0400, Mike Frysinger wrote:
> On Saturday 17 June 2006 06:17, Luca Barbato wrote:
> > Long term solution:
> > 1- check your new package for aliasing compliance, and if you have time
> > fix it in the code or in the makefile, if you haven't append
> > -fno-strict-aliasing to the cflags and maybe send a notice about it
> > upstream
> >
> > 2- append -fno-strict-aliasing to every source known to have such issue.
>
> 3 - include checking in portage
> http://bugs.gentoo.org/111436

Would this make builds fail by default, or would it provide a hook for
the user to allow builds to fail if specific conditions are matched ?
--
gentoo-dev@gentoo.org mailing list
Re: strict-aliasing rules, don't break them [ In reply to ]
On Saturday 17 June 2006 13:20, Mike Frysinger wrote:
> some more docs:
> http://gcc.gnu.org/bugs.html#nonbugs_c
> Casting does not work as expected when optimization is turned on

and a follow up link:
http://mail-index.netbsd.org/tech-kern/2003/08/11/0001.html
-mike
Re: strict-aliasing rules, don't break them [ In reply to ]
On Saturday 17 June 2006 13:28, Harald van Dijk wrote:
> On Sat, Jun 17, 2006 at 01:15:58PM -0400, Mike Frysinger wrote:
> > On Saturday 17 June 2006 06:17, Luca Barbato wrote:
> > > Long term solution:
> > > 1- check your new package for aliasing compliance, and if you have time
> > > fix it in the code or in the makefile, if you haven't append
> > > -fno-strict-aliasing to the cflags and maybe send a notice about it
> > > upstream
> > >
> > > 2- append -fno-strict-aliasing to every source known to have such
> > > issue.
> >
> > 3 - include checking in portage
> > http://bugs.gentoo.org/111436
>
> Would this make builds fail by default, or would it provide a hook for
> the user to allow builds to fail if specific conditions are matched ?

right now such behavior is not defined

i would lean on the FEATURES=stricter setting though to control warn/die
behavior
-mike
Re: strict-aliasing rules, don't break them [ In reply to ]
On Sat, Jun 17, 2006 at 01:44:34PM -0400, Mike Frysinger wrote:
> On Saturday 17 June 2006 13:28, Harald van Dijk wrote:
> > On Sat, Jun 17, 2006 at 01:15:58PM -0400, Mike Frysinger wrote:
> > > On Saturday 17 June 2006 06:17, Luca Barbato wrote:
> > > > Long term solution:
> > > > 1- check your new package for aliasing compliance, and if you have time
> > > > fix it in the code or in the makefile, if you haven't append
> > > > -fno-strict-aliasing to the cflags and maybe send a notice about it
> > > > upstream
> > > >
> > > > 2- append -fno-strict-aliasing to every source known to have such
> > > > issue.
> > >
> > > 3 - include checking in portage
> > > http://bugs.gentoo.org/111436
> >
> > Would this make builds fail by default, or would it provide a hook for
> > the user to allow builds to fail if specific conditions are matched ?
>
> right now such behavior is not defined
>
> i would lean on the FEATURES=stricter setting though to control warn/die
> behavior

One reason I asked was because you'd have to force LC_ALL=C to make that
work, unless you want to check for every translation of the warning.

Another, as said earlier already, is that the warning is given for valid
code too sometimes, but as long as it's possible to state in the ebuild
that the warnings are safe to ignore for that package, I guess that's
okay.
--
gentoo-dev@gentoo.org mailing list
Re: strict-aliasing rules, don't break them [ In reply to ]
On Sat, Jun 17, 2006 at 03:35:10PM +0200, Harald van D??k wrote:
> On Sat, Jun 17, 2006 at 03:32:36PM +0200, Kevin F. Quinn wrote:
> > Easiest way to find these is to stick -Wall in global CFLAGS (or just
> > -Wstrict-aliasing if you want to be more specific), and grep the build
> > log for 'will break strict aliasing' (LC_ALL=C obviously).

http://dev.gentoo.org/~pvdabeel/strict-aliasing.log

It seems that roughly 10% of the ebuilds tested with strict-aliasing
in cflags warn about type punned pointers breaking strict aliasing
rules. A few ebuilds did not make it through testing with strict
aliasing, but did with -fno-strict-aliasing. Those should be obvious
candidates for appending the fno-strict-aliasing flag (or an
attempt at fixing the violation by patching the code).

> That warning is given for valid code too. Please only add
> -fno-strict-aliasing if you actually find a package misbehaves without
> it, or if you have verified that there is indeed an aliasing violation
> in the code.

Right now we have -fno-strict-aliasing in the default cflags and are
considering auditing the tree for aliasing violations.

Pieter Van den Abeele

> --
> gentoo-dev@gentoo.org mailing list
--
gentoo-dev@gentoo.org mailing list
Re: strict-aliasing rules, don't break them [ In reply to ]
So, just for people to know..

On Saturday 17 June 2006 15:32, Kevin F. Quinn wrote:
> kde-base/kdm-3.5.1
Fixed in latest ~arch (and upstream)

> media-libs/faad2-2.0-r11
Fixed in latest ~arch

> media-libs/xine-lib-1.1.2_pre20060328-r9
Fixed in latest ~arch (and upstream)

> media-video/vlc-0.8.5-r1
Testing in progress, will be fixed for tomorrow.

--
Diego "Flameeyes" Pettenò - http://farragut.flameeyes.is-a-geek.org/
Gentoo/Alt lead, Gentoo/FreeBSD, Video, AMD64, Sound, PAM, KDE
Re: strict-aliasing rules, don't break them [ In reply to ]
2006-06-17, Harald van Dijk sanoi, jotta:

> [F]orce LC_ALL=C to make
> that work, unless you want to check for every translation of the
> warning.

You can also check against current locale’s translation of the warning
using gettext(1) or such to extract it.

--
Flammie, Gentoo Linux Documentation’s Finnish head translator
and FlameEyes’ bot <http://dev.gentoo.org/~flammie>.