compat/bswap.h: detect ARM64 when using MSVC

Commit Message

Daniel Gurney Nov. 7, 2020, 10:19 p.m. UTC

Signed-off-by: Daniel Gurney <[e-mail]>
---
 compat/bswap.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

brian m. carlson Nov. 7, 2020, 10:47 p.m. UTC | #1

On 2020-11-07 at 22:19:16, Daniel Gurney wrote:
> Signed-off-by: Daniel Gurney <[e-mail]>
> ---
>  compat/bswap.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/compat/bswap.h b/compat/bswap.h
> index c0bb744adc..512f6f4b99 100644
> --- a/compat/bswap.h
> +++ b/compat/bswap.h
> @@ -74,7 +74,7 @@ static inline uint64_t git_bswap64(uint64_t x)
>  }
>  #endif
>  
> -#elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64))
> +#elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64) || defined(_M_ARM64))
>  
>  #include <stdlib.h>
>  

I think this is fine as it is, but I have a question here: why, if we're
using MSVC, is that not sufficient to enable this?  In other words, why
can't this line simply be this:

  #elif defined(_MSC_VER)

As far as I know, Windows has always run on little-endian hardware.  It
looks like MSVC did run on the M68000 series and MIPS[0] at some point.
Are those really versions of MSVC we care about and think Git can
practically support, given the fact that we require so many C99
constructs that are not practically available in old versions of MSVC?
If not, wouldn't it make sense to simplify?

[0] Wikipedia does not specify the endiannesses supported by the MIPS
edition.

Daniel Gurney Nov. 7, 2020, 11:23 p.m. UTC | #2

> As far as I know, Windows has always run on little-endian hardware.

That's a fair point, and I doubt there'll be any new developments in
this regard. I'll send a new patch with the simplification you
suggested.

Johannes Schindelin Nov. 10, 2020, 1:58 p.m. UTC | #3

Hi brian & Daniel,

On Sat, 7 Nov 2020, brian m. carlson wrote:

> On 2020-11-07 at 22:19:16, Daniel Gurney wrote:
> > Signed-off-by: Daniel Gurney <[e-mail]>
> > ---
> >  compat/bswap.h | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/compat/bswap.h b/compat/bswap.h
> > index c0bb744adc..512f6f4b99 100644
> > --- a/compat/bswap.h
> > +++ b/compat/bswap.h
> > @@ -74,7 +74,7 @@ static inline uint64_t git_bswap64(uint64_t x)
> >  }
> >  #endif
> >
> > -#elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64))
> > +#elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64) || defined(_M_ARM64))
> >
> >  #include <stdlib.h>
> >
>
> I think this is fine as it is, but I have a question here: why, if we're
> using MSVC, is that not sufficient to enable this?  In other words, why
> can't this line simply be this:
>
>   #elif defined(_MSC_VER)

That is a good question. As far as I can see, this code path has been
introduced in 0fcabdeb52b (Use faster byte swapping when compiling with
MSVC, 2009-10-19). Then commit looks like this:

    Author: Sebastian Schuberth <[e-mail]>
    Date:   Mon Oct 19 18:37:05 2009 +0200
    Subject: Use faster byte swapping when compiling with MSVC

    When compiling with MSVC on x86-compatible, use an intrinsic for
    byte swapping. In contrast to the GCC path, we do not prefer
    inline assembly here as it is not supported for the x64 platform.

    Signed-off-by: Sebastian Schuberth <[e-mail]>
    Signed-off-by: Junio C Hamano <[e-mail]>

    diff --git a/compat/bswap.h b/compat/bswap.h
    index 5cc4acbfcce..279e0b48b15 100644
    --- a/compat/bswap.h
    +++ b/compat/bswap.h
    @@ -28,6 +28,16 @@ static inline uint32_t default_swab32(uint32_t val)
     	} \
     	__res; })

    +#elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64))
    +
    +#include <stdlib.h>
    +
    +#define bswap32(x) _byteswap_ulong(x)
    +
    +#endif
    +
    +#ifdef bswap32
    +
     #undef ntohl
     #undef htonl
     #define ntohl(x) bswap32(x)

I Cc:ed Sebastian, to confirm my hunch: Looking a bit above that hunk, I
see that this merely imitates the way things are done for GCC:

    #if defined(__GNUC__) && (defined(__i386__) || defined(__x86_64__))

    [...]

Now, let's have a slightly harder look at the patch under consideration:
what this does is to re-use the (known to be little-endian) code path of
x86 and x86_64 to define the `bswap32()` and `bswap64()` macros also for
ARM64.

Further down, the presence of these macros is taken for a sign that we
_are_ little-endian and therefore the `ntol()` family of functions isn't a
no-op, but has to swap the order, and that's what those macros are then
used for.

The biggest question now is: are we certain that `_M_ARM64` implies
little-endian?

I remember that ARM (the 32-bit variety, that is) has support for
switching endianness on the fly. Happily, MSVC talks specifically about
_not_ supporting that:
https://docs.microsoft.com/en-us/cpp/build/overview-of-arm-abi-conventions

Likewise, it says the same about ARM64 (mentioning that it would be much
harder to switch endianness there to begin with):
https://docs.microsoft.com/en-us/cpp/build/arm64-windows-abi-conventions

So does that make us confident that we can just add that `_M_ARM64` part?
Yes. Does it make me confident that we can just drop all of the
architecture-dependent conditions? No, it does not. There _were_ versions
of MSVC that could compile code for PowerPC, for example, which _is_
big-endian.

> As far as I know, Windows has always run on little-endian hardware.

I think that depends on your point of view... IIRC an early version of
Windows NT (or was it still VMS Plus?) ran on DEC Alpha, which I seem to
_vaguely_ remember was big-endian.

> It looks like MSVC did run on the M68000 series and MIPS[0] at some
> point. Are those really versions of MSVC we care about and think Git can
> practically support, given the fact that we require so many C99
> constructs that are not practically available in old versions of MSVC?
> If not, wouldn't it make sense to simplify?

Indeed, and it is more than just the C99 constructs that prevent us from
supporting other MSVC versions: we do not support compiling with MSVC out
of the box. Nowadays, we rely on CMake to generate the appropriate files
to allow us to build with MSVC because our Makefile-based approach very
much hardcodes GCC (or compatible) options.

>
> [0] Wikipedia does not specify the endiannesses supported by the MIPS
> edition.

I have another vague memory about MIPS (a wonderful SGI machine I had the
pleasure of banging my head against, for lack of Python support and Git
requiring Python back then) being big-endian, too.

Short version: while I managed to convince myself that _currently_ there
are no big-endian platforms that we can support via MSVC, I would like to
stay within the boundaries of caution and _not_ drop those `defined(_M_*)`
parts.

Ciao,
Dscho

Sebastian Schuberth Nov. 10, 2020, 4:44 p.m. UTC | #4

On Tue, Nov 10, 2020 at 2:58 PM Johannes Schindelin
<[e-mail]> wrote:

> > -#elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64))
> > +#elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64) || defined(_M_ARM64))

> I Cc:ed Sebastian, to confirm my hunch: Looking a bit above that hunk, I
> see that this merely imitates the way things are done for GCC:
>
>     #if defined(__GNUC__) && (defined(__i386__) || defined(__x86_64__))

I believe my intention was not necessarily to imitate the way things
are done for GCC, but to independently be on the safe side by checking
that this code is only used on x86-style / little-endian architecture.

> > As far as I know, Windows has always run on little-endian hardware.
>
> I think that depends on your point of view... IIRC an early version of
> Windows NT (or was it still VMS Plus?) ran on DEC Alpha, which I seem to
> _vaguely_ remember was big-endian.

IMO, strictly speaking, from a semantic point of view this is not
about which OS we are running on, but about which compiler is being
used. So the question here is: Can MSVC compile for a non-little
endian target platform (incl. things like cross-compilation)? And
AFAIK the answer is yes, it could in the past / still can nowadays.

> Short version: while I managed to convince myself that _currently_ there
> are no big-endian platforms that we can support via MSVC, I would like to
> stay within the boundaries of caution and _not_ drop those `defined(_M_*)`
> parts.

Same here, I'd prefer to keep these for explicitness, and for
consistency with the GCC check.

brian m. carlson Nov. 10, 2020, 11:38 p.m. UTC | #5

On 2020-11-10 at 13:58:21, Johannes Schindelin wrote:

> The biggest question now is: are we certain that `_M_ARM64` implies
> little-endian?

For Windows?  Yes.  I'm almost certain Windows has only supported
little-endian architectures, possibly with the exception of gaming
consoles.

> I remember that ARM (the 32-bit variety, that is) has support for
> switching endianness on the fly. Happily, MSVC talks specifically about
> _not_ supporting that:
> https://docs.microsoft.com/en-us/cpp/build/overview-of-arm-abi-conventions
> 
> Likewise, it says the same about ARM64 (mentioning that it would be much
> harder to switch endianness there to begin with):
> https://docs.microsoft.com/en-us/cpp/build/arm64-windows-abi-conventions
> 
> So does that make us confident that we can just add that `_M_ARM64` part?
> Yes. Does it make me confident that we can just drop all of the
> architecture-dependent conditions? No, it does not. There _were_ versions
> of MSVC that could compile code for PowerPC, for example, which _is_
> big-endian.

PowerPC can actually be either.  Most 64-bit PowerPC machines these days
are run as little endian, and Windows has always run it in little-endian
mode.  Macs ran it in big-endian mode.

> > As far as I know, Windows has always run on little-endian hardware.
> 
> I think that depends on your point of view... IIRC an early version of
> Windows NT (or was it still VMS Plus?) ran on DEC Alpha, which I seem to
> _vaguely_ remember was big-endian.

Alpha appears to have supported both, but as far as I know, both Windows
and Linux used it in little-endian mode.

> > [0] Wikipedia does not specify the endiannesses supported by the MIPS
> > edition.
> 
> I have another vague memory about MIPS (a wonderful SGI machine I had the
> pleasure of banging my head against, for lack of Python support and Git
> requiring Python back then) being big-endian, too.

Another architecture that supports both endiannesses.  Debian supports
both, but I believe Windows only supported the little-endian version.  I
have a small MIPS board that uses the little endian port for Debian.

> Short version: while I managed to convince myself that _currently_ there
> are no big-endian platforms that we can support via MSVC, I would like to
> stay within the boundaries of caution and _not_ drop those `defined(_M_*)`
> parts.

While I'm confident in my statements, you're the relevant subsystem
maintainer here, so I'm happy to defer to your judgment.  I think Junio
can just pick up the earlier patch version and we should be good to go,
since that patch seemed to meet everyone's needs.

Patch

diff --git a/compat/bswap.h b/compat/bswap.h
index c0bb744adc..512f6f4b99 100644
--- a/compat/bswap.h
+++ b/compat/bswap.h
@@ -74,7 +74,7 @@  static inline uint64_t git_bswap64(uint64_t x)
 }
 #endif
 
-#elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64))
+#elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64) || defined(_M_ARM64))
 
 #include <stdlib.h>