| Message ID | 20190215101427.9896-1-uwe@kleine-koenig.org (mailing list archive) |
|---|---|
| State | Superseded, archived |
| Headers | show |
| Series | [RFC] determine specifics using gcc -dumpmachine instead of uname -m | expand |
On Fri, Feb 15, 2019 at 11:14:27AM +0100, Uwe Kleine-König wrote: > uname -m is unreliable as the host architecture might not match the > target architecture. This can happen when cross compiling or (more > common) if you are running a 32 bit userspace on a 64 bit kernel. The > latter makes sparse fail to build on some machines of the Debian build > farm as for example some armhf builder run on arm64 kernels. > > This patch is only lightly tested and also misses details for mips* and > so marked as RFC. Hi, Thanks you for trying this. I think it would be better to discard everything after the first '-'. For example, one of the machine I use have 'gcc -dumpmachine' returning 'ppc64le-redhat-linux' and none of the pattern you've used will match any BSD. The uname-backoff will catch them but it's easy to avoid it. Please note that (as far as I can think) if you're on a platform A and cross-compile sparse for platform B and then use sparse (with o without cgcc) on this platform B, then everything should be OK. OTOH, I'm not sure I understand exactly what is done in this reproducibility test here. I suppose it's something like: on platform A, sparse is compiled natively then, still on platform A, sparse is used (with cgcc) with a cross-compiler for platform B. In this sort of situation, even if cgcc is made somehow 'cross-platform aware', I'm pretty sure there will be many others problems. Best regards, -- Luc
On 2/15/19 9:13 PM, Luc Van Oostenryck wrote: > On Fri, Feb 15, 2019 at 11:14:27AM +0100, Uwe Kleine-König wrote: >> uname -m is unreliable as the host architecture might not match the >> target architecture. This can happen when cross compiling or (more >> common) if you are running a 32 bit userspace on a 64 bit kernel. The >> latter makes sparse fail to build on some machines of the Debian build >> farm as for example some armhf builder run on arm64 kernels. >> >> This patch is only lightly tested and also misses details for mips* and >> so marked as RFC. > > Hi, > > Thanks you for trying this. > > I think it would be better to discard everything after the first '-'. This will not be enough. Consider for example arm-linux-gnueabi vs. arm-linux-gnueabihf. (I think this is the only offender in the official debian architectures according to my list I created to author my patch.) > For example, one of the machine I use have 'gcc -dumpmachine' > returning 'ppc64le-redhat-linux' and none of the pattern you've > used will match any BSD. The uname-backoff will catch them but > it's easy to avoid it. easy but wrong :-) > Please note that (as far as I can think) if you're on a platform A > and cross-compile sparse for platform B and then use sparse (with o > without cgcc) on this platform B, then everything should be OK. > OTOH, I'm not sure I understand exactly what is done in this > reproducibility test here. This is not really a cross-compile situation. It's just that the second builder isn't a native armhf machine, but it runs an arm64 kernel and the build is done in an armhf chroot. Similar for i686: That's a i686 chroot on an amd64 machine. The background here is that the Debian people will probably rely on this kind of setup because on real armhf hardware you cannot properly build software like libreoffice (IIRC), and the only way out to keep armhf supported in the long run is to be able to build packages on arm64 which has a bigger address space. > I suppose it's something like: > on platform A, sparse is compiled natively then, still on platform A, > sparse is used (with cgcc) with a cross-compiler for platform B. > In this sort of situation, even if cgcc is made somehow > 'cross-platform aware', I'm pretty sure there will be many others > problems. I noticed the naming to be wrong at least. Actually you want target_arch_specs instead of host_arch_specs. Similar for host_os_specs vs. target_os_specs. But using gcc -dumpmachine should do the right thing here, so my patch goes in the right direction I think. Best regards Uwe
On Fri, Feb 15, 2019 at 10:38:44PM +0100, Uwe Kleine-König wrote: > On 2/15/19 9:13 PM, Luc Van Oostenryck wrote: > > Please note that (as far as I can think) if you're on a platform A > > and cross-compile sparse for platform B and then use sparse (with o > > without cgcc) on this platform B, then everything should be OK. > > OTOH, I'm not sure I understand exactly what is done in this > > reproducibility test here. > > This is not really a cross-compile situation. It's just that the second > builder isn't a native armhf machine, but it runs an arm64 kernel and > the build is done in an armhf chroot. OK, and what is the -dumpmachine of the compiler used to compile sparse in this chroot? 'arm-linux-gnueabihf'? > > ... even if cgcc is made somehow > > 'cross-platform aware', I'm pretty sure there will be many others > > problems. > > ... But using gcc -dumpmachine should do the right > thing here, so my patch goes in the right direction I think. Yes, for sure using 'gcc -dumpmachine' won't give worst result than using 'uname -m'. What I meant was that here we have a problem with __ARM_PCS_VFP which must be set by default for arm-linux-gnueabihf. But this flag must also be set for arm-linux-gnueabi if used with -mfloat-abi=hard and must not be set for arm-linux-gnueabihf with -mfloat-abi=soft or -mfloat-abi=softfp. Each archs have several, sometimes dozen, other such sort of things where gcc's target (and otherconfiguration choices) and the options efectively used must be taken in account to give correct resuls. For ARM, a non-exhaustive list is: __APCS_32__, __ARMEL__ __ARM_32BIT_STATE, __ARM_ARCH, __ARM_ARCH_7A__, __ARM_ARCH_ISA_ARM, __ARM_ARCH_ISA_THUMB, __ARM_EABI__, __ARM_FEATURE_<feature>, __ARM_FP, __ARM_NEON_FP, __VFP_FP__, ... (and that's just for preprocessor things). MIPS is most probably the worst with all its ABI choices which can then be changed further with more options (like -mfp64, -mgp32). Best regards, -- Luc
On 2/16/19 12:36 PM, Luc Van Oostenryck wrote: > On Fri, Feb 15, 2019 at 10:38:44PM +0100, Uwe Kleine-König wrote: >> On 2/15/19 9:13 PM, Luc Van Oostenryck wrote: >>> Please note that (as far as I can think) if you're on a platform A >>> and cross-compile sparse for platform B and then use sparse (with o >>> without cgcc) on this platform B, then everything should be OK. >>> OTOH, I'm not sure I understand exactly what is done in this >>> reproducibility test here. >> >> This is not really a cross-compile situation. It's just that the second >> builder isn't a native armhf machine, but it runs an arm64 kernel and >> the build is done in an armhf chroot. > > OK, and what is the -dumpmachine of the compiler used to compile > sparse in this chroot? 'arm-linux-gnueabihf'? Right. On the arm64 porter box (uname -m -> aarch64) when entering an armhf chroot I get: $ uname -m armv8l So the problem with the reproducible stuff might only happen because uname is modified on purpose. >>> ... even if cgcc is made somehow >>> 'cross-platform aware', I'm pretty sure there will be many others >>> problems. >> >> ... But using gcc -dumpmachine should do the right >> thing here, so my patch goes in the right direction I think. > > Yes, for sure using 'gcc -dumpmachine' won't give worst result than > using 'uname -m'. What I meant was that here we have a problem with > __ARM_PCS_VFP which must be set by default for arm-linux-gnueabihf. > But this flag must also be set for arm-linux-gnueabi if used with > -mfloat-abi=hard and must not be set for arm-linux-gnueabihf with > -mfloat-abi=soft or -mfloat-abi=softfp. I wonder if gcc is happy with this. I just tried gcc -mfloat-abi=soft test.c -o test on an armhf system and it failed with /usr/include/arm-linux-gnueabihf/gnu/stubs.h:7:11: fatal error: gnu/stubs-soft.h: No such file or directory According to https://packages.debian.org/search?searchon=contents&keywords=stubs-soft.h&mode=path&suite=unstable&arch=any this file is provided by libc6-dev on armel, but I think even installing that won't make gcc happy as the path where the file is provided isn't considered. So I'd say it's ok if cgcc -mfloat-abi=soft fails on armhf, too. Making this work is a big quest I'd say. We'd need something like "properties" like endianess, pointer-size and float handling and then categorize the needed cpp-symbols to be active only when certain properties are "active". Best regards Uwe
On Sat, Feb 16, 2019 at 10:59:14PM +0100, Uwe Kleine-König wrote: > On 2/16/19 12:36 PM, Luc Van Oostenryck wrote: > > On Fri, Feb 15, 2019 at 10:38:44PM +0100, Uwe Kleine-König wrote: > >> On 2/15/19 9:13 PM, Luc Van Oostenryck wrote: > >>> Please note that (as far as I can think) if you're on a platform A > >>> and cross-compile sparse for platform B and then use sparse (with o > >>> without cgcc) on this platform B, then everything should be OK. > >>> OTOH, I'm not sure I understand exactly what is done in this > >>> reproducibility test here. > >> > >> This is not really a cross-compile situation. It's just that the second > >> builder isn't a native armhf machine, but it runs an arm64 kernel and > >> the build is done in an armhf chroot. > > > > OK, and what is the -dumpmachine of the compiler used to compile > > sparse in this chroot? 'arm-linux-gnueabihf'? > > Right. On the arm64 porter box (uname -m -> aarch64) when entering an > armhf chroot I get: > > $ uname -m > armv8l OK, thanks. > So the problem with the reproducible stuff might only happen because > uname is modified on purpose. > > >>> ... even if cgcc is made somehow > >>> 'cross-platform aware', I'm pretty sure there will be many others > >>> problems. > >> > >> ... But using gcc -dumpmachine should do the right > >> thing here, so my patch goes in the right direction I think. > > > > Yes, for sure using 'gcc -dumpmachine' won't give worst result than > > using 'uname -m'. What I meant was that here we have a problem with > > __ARM_PCS_VFP which must be set by default for arm-linux-gnueabihf. > > But this flag must also be set for arm-linux-gnueabi if used with > > -mfloat-abi=hard and must not be set for arm-linux-gnueabihf with > > -mfloat-abi=soft or -mfloat-abi=softfp. > > I wonder if gcc is happy with this. I just tried > > gcc -mfloat-abi=soft test.c -o test > > on an armhf system and it failed with > > /usr/include/arm-linux-gnueabihf/gnu/stubs.h:7:11: fatal error: > gnu/stubs-soft.h: No such file or directory I'm used to use this sort of flags for bare-metal/stand-alone stuff but yes, in order to work, you need to have the libs and everything else coherent. > According to > https://packages.debian.org/search?searchon=contents&keywords=stubs-soft.h&mode=path&suite=unstable&arch=any > this file is provided by libc6-dev on armel, but I think even installing > that won't make gcc happy as the path where the file is provided isn't > considered. > So I'd say it's ok if cgcc -mfloat-abi=soft fails on armhf, too. I don't really agree. In the setup you're using, yes 'gcc -mfloat-abi=soft' can't be used but in others ones it's working perfectly (I mean, even with a tool chain configured for armhf by default). > Making this work is a big quest I'd say. Yes, it's also an endless can of worms. Best regards, -- Luc
diff --git a/cgcc b/cgcc index 629124b9412e..176c14c35adb 100755 --- a/cgcc +++ b/cgcc @@ -339,28 +339,71 @@ sub add_specs { chomp $os; return &add_specs (lc $os); } elsif ($spec eq 'host_arch_specs') { - my $arch = `uname -m`; - chomp $arch; - if ($arch =~ /^(i.?86|athlon)$/i) { - return &add_specs ('i386'); - } elsif ($arch =~ /^(sun4u)$/i) { - return &add_specs ('sparc'); - } elsif ($arch =~ /^(x86_64)$/i) { - return &add_specs ('x86_64'); - } elsif ($arch =~ /^(ppc)$/i) { - return &add_specs ('ppc'); - } elsif ($arch =~ /^(ppc64)$/i) { - return &add_specs ('ppc64') . ' -mbig-endian -D_CALL_ELF=1'; - } elsif ($arch =~ /^(ppc64le)$/i) { - return &add_specs ('ppc64') . ' -mlittle-endian -D_CALL_ELF=2'; - } elsif ($arch =~ /^(s390x)$/i) { - return &add_specs ('s390x'); - } elsif ($arch =~ /^(sparc64)$/i) { - return &add_specs ('sparc64'); - } elsif ($arch =~ /^arm(?:v[78]l)?$/i) { - return &add_specs ('arm'); - } elsif ($arch =~ /^(aarch64)$/i) { - return &add_specs ('aarch64'); + chomp(my $gccmachine = `$cc -dumpmachine`); + + if ($gccmachine eq 'aarch64-linux-gnu') { + return (' -m64' . &float_types (1, 1, 36, [24, 8], [53, 11], [113, 15])); + + } elsif ($gccmachine eq 'arm-linux-gnueabi') { + return (' -m32' . &float_types (1, 1, 36, [24, 8], [53,11], [53, 11])); + + } elsif ($gccmachine eq 'arm-linux-gnueabihf') { + return (' -m32 -D__ARM_PCS_VFP=1' . &float_types (1, 1, 36, [24,8], [53,11], [53, 11])); + + } elsif ($gccmachine eq 'i686-linux-gnu') { + return &float_types (1, 1, 21, [24, 8], [53, 11], [64, 15]); + + } elsif ($gccmachine eq 'mips-linux-gnu') { + # ??? + + } elsif ($gccmachine eq 'mipsel-linux-gnu') { + # ??? + + } elsif ($gccmachine eq 'mips64el-linux-gnuabi64') { + # ??? + + } elsif ($gccmachine eq 'powerpc64le-linux-gnu') { + return (' -D_STRING_ARCH_unaligned=1 -m64 -mlittle-endian -D_CALL_ELF=2' . &float_types (1, 1, 21, [24,8], [53,11], [113,15])); + + } elsif ($gccmachine eq 's390x') { + + return (' -D_BIG_ENDIAN' . + &integer_types (8, 16, 32, $m64 ? 64 : 32, 64) . + &float_types (1, 1, 36, [24,8], [53,11], [113,15]) . + &define_size_t ("long unsigned int") . + ' -D__SIZEOF_POINTER__=' . ($m64 ? '8' : '4')); + + } elsif ($gccmachine eq 'x86_64-linux-gnu') { + return &float_types (1, 1, 33, [24, 8], [53, 11], [113, 15]); + + } else { + # fall back to uname -m to determine the specifics. Note this is + # unstable as we might run an i386 compiler on an amd64 box or even + # a hppa compiler on s390x. + + my $arch = `uname -m`; + chomp $arch; + if ($arch =~ /^(i.?86|athlon)$/i) { + return &add_specs ('i386'); + } elsif ($arch =~ /^(sun4u)$/i) { + return &add_specs ('sparc'); + } elsif ($arch =~ /^(x86_64)$/i) { + return &add_specs ('x86_64'); + } elsif ($arch =~ /^(ppc)$/i) { + return &add_specs ('ppc'); + } elsif ($arch =~ /^(ppc64)$/i) { + return &add_specs ('ppc64') . ' -mbig-endian -D_CALL_ELF=1'; + } elsif ($arch =~ /^(ppc64le)$/i) { + return &add_specs ('ppc64') . ' -mlittle-endian -D_CALL_ELF=2'; + } elsif ($arch =~ /^(s390x)$/i) { + return &add_specs ('s390x'); + } elsif ($arch =~ /^(sparc64)$/i) { + return &add_specs ('sparc64'); + } elsif ($arch =~ /^arm(?:v[78]l)?$/i) { + return &add_specs ('arm'); + } elsif ($arch =~ /^(aarch64)$/i) { + return &add_specs ('aarch64'); + } } } else { die "$0: invalid specs: $spec\n";
uname -m is unreliable as the host architecture might not match the target architecture. This can happen when cross compiling or (more common) if you are running a 32 bit userspace on a 64 bit kernel. The latter makes sparse fail to build on some machines of the Debian build farm as for example some armhf builder run on arm64 kernels. This patch is only lightly tested and also misses details for mips* and so marked as RFC. Signed-off-by: Uwe Kleine-König <uwe@kleine-koenig.org> --- cgcc | 87 +++++++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 65 insertions(+), 22 deletions(-)