ABI compatibility regression: Return values on x86

1 msg	gcc-4.1-20080107 is now available

ABI compatibility regression: Return values on x86
\ Andrew Haley (7 Jan 2008)
. \ H.J. Lu (7 Jan 2008)
. . \ Andrew Haley (8 Jan 2008)
. . . \ H.J. Lu (8 Jan 2008)
. . . . \ Andrew Haley (8 Jan 2008)
. . . . . \ H.J. Lu (8 Jan 2008)

2 msg	svn update: checksum's don't match
1 msg	'No such file error' when compiling powerpc-sof...
1 msg	Announcement: AspeCt-oriented C (ACC) version 0...
2 msg	MIPS/FORTRAN failures...
1 msg	Pilgerweg nach Assisi
2 msg	changing gcc
1 msg	gcc-4.3-20080104 is now available
2 msg	XML dump for GCC
1 msg	Re: [PATCH] Disallow inlining if static vars in...
2 msg	GCC 4.3.0 Status Report (2008-01-02)
3 msg	plugin help: Inserting a function call in gimpl...
1 msg	gcc-4.2-20080102 is now available
3 msg	PATCH: PR gas/5534: 'XXX PTR' isn't checked pro...
2 msg	Restricted or no run time in Ada
2 msg	Does GIMPLE tree comprise ARRAY_REF/ARRAY_REF_R...
2 msg	Gfortran annual report for 2008
3 msg	Re: What is a regression?
9 msg	Optimizations documentation

Subject:	ABI compatibility regression: Return values on x86
Group:	Gcc
From:	Andrew Haley
Date:	7 Jan 2008

gcc (x86) recently changed its behaviour when returning values shorter
than int. It used to sign extend, and now it doesn't.

short func2( short *size) { return *size; }

trunk:

func2:
pushl %ebp
movl %esp, %ebp
movl 8(%ebp), %eax
movzwl (%eax), %eax
popl %ebp
ret

gcc, all previous versions:

func2:
pushl %ebp
movl %esp, %ebp
movl 8(%ebp), %eax
movswl (%eax),%eax
leave
ret

This applies to both 32- and 64-bit gcc versions.

This ABI change was caused by

svn diff -r126479:126480 svn+ssh://gcc.gnu.org/svn/gcc/trunk

2007-07-09 Richard Guenther <rguenther>

* c-decl.c (start_function): Do not promote return type.

Index: gcc/c-decl.c
===================================================================
--- gcc/c-decl.c (revision 126479)
+++ gcc/c-decl.c (revision 126480)
@@ -6270,18 +6270,6 @@
declare_parm_level ();

restype = TREE_TYPE (TREE_TYPE (current_function_decl));
- /* Promote the value to int before returning it. */
- if (c_promoting_integer_type_p (restype))
- {
- /* It retains unsignedness if not really getting wider. */
- if (TYPE_UNSIGNED (restype)
- && (TYPE_PRECISION (restype)
- == TYPE_PRECISION (integer_type_node)))
- restype = unsigned_type_node;
- else
- restype = integer_type_node;
- }
-
resdecl = build_decl (RESULT_DECL, NULL_TREE, restype);
DECL_ARTIFICIAL (resdecl) = 1;
DECL_IGNORED_P (resdecl) = 1;

This is generic code; I don't think there was any intention to change
the x86 ABI.

The 32-bit psABI says

"A function that returns an integral or pointer value places its
result in register %eax.

"[ ... ] Functions pass all integer-valued arguments as words,
expanding or padding signed or unsigned bytes and halfwords as
needed."

It is not explicit that return values are handled in the same way as
incoming args, but IMO it is reasonable to assume so. In any case,
we'd have to have a very good reason to change the ABI at this stage.

Ian Taylor pointed out that any change to this wouldn't be visible to
gcc-generated code, which is true. This is why, I suppose. no-one
noticed it, despite the fact that it's an ABI change. However, it may
well break other languages that link to gcc. It certainly caused
libffi test failures, which is how we noticed it.

So, what now? Can we even agree about what the psABI actually says
about sign-extending result values? Was what we did before correct,
or what we do now? I don't believe that it doesn't matter.

Andrew.

--
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903