[v2] Makefile: use sha1collisiondetection by default on OSX and Darwin

Commit Message

Ævar Arnfjörð Bjarmason Dec. 15, 2022, 8:43 a.m. UTC

When the sha1collisiondetection library was added and made the default
in [1] the interaction with APPLE_COMMON_CRYPTO added in [2] and [3]
seems to have been missed. On modern OSX and Darwin we are able to use
Apple's CommonCrypto both for SHA-1, and as a generic (but partial)
OpenSSL replacement.

This left OSX and Darwin without protection against the SHAttered
attack when building Git in its default configuration.

Let's also use sha1collisiondetection on OSX, to do so we'll need to
split up the "APPLE_COMMON_CRYPTO" flag into that flag and a new
"APPLE_COMMON_CRYPTO_SHA1".

Because of this we can stop conflating whether we want to use Apple's
CommonCrypto at all, and whether we want to use it for SHA-1.  This
makes the CI recipe added in [4] simpler.

1. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17)
2. 4dcd7732db0 (Makefile: add support for Apple CommonCrypto facility, 2013-05-19)
3. 61067954ce1 (cache.h: eliminate SHA-1 deprecation warnings on Mac OS X, 2013-05-19)
4. 1ad5c3df35a (ci: use DC_SHA1=YesPlease on osx-clang job for CI,
   2022-10-20)

Signed-off-by: Ævar Arnfjörð Bjarmason <[e-mail]>
---
Range-diff against v1:
1:  1f4e39be97b ! 1:  3de7cdbd260 Makefile: use sha1collisiondetection by default on OSX and Darwin
    @@ Commit message
         split up the "APPLE_COMMON_CRYPTO" flag into that flag and a new
         "APPLE_COMMON_CRYPTO_SHA1".
     
    -    Let's also change the CI for "osx-clang" to test with the new
    -    APPLE_COMMON_CRYPTO_SHA1 knob ("osx-gcc" uses the new
    -    sha1collisiondetection default).
    -
    -    In practice this will spot issues like the one noted in [4], as
    -    testing with just two backends should be enough to spot unportable
    -    code. Ideally we'd have other CI jobs to test the various SHA-1
    -    combinations, but for now we have better CI coverage than before.
    +    Because of this we can stop conflating whether we want to use Apple's
    +    CommonCrypto at all, and whether we want to use it for SHA-1.  This
    +    makes the CI recipe added in [4] simpler.
     
         1. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17)
         2. 4dcd7732db0 (Makefile: add support for Apple CommonCrypto facility, 2013-05-19)
         3. 61067954ce1 (cache.h: eliminate SHA-1 deprecation warnings on Mac OS X, 2013-05-19)
    -    4. 32205655dc7 (fsmonitor OSX: compile with DC_SHA1=YesPlease, 2022-10-19)
    +    4. 1ad5c3df35a (ci: use DC_SHA1=YesPlease on osx-clang job for CI,
    +       2022-10-20)
     
         Signed-off-by: Ævar Arnfjörð Bjarmason <[e-mail]>
     
    @@ Makefile: include shared.mak
      #
     -# Define NO_APPLE_COMMON_CRYPTO on OSX to opt-out of using the
     -# "APPLE_COMMON_CRYPTO" backend for SHA-1, which is currently the
    --# default on that OS. We'll define NO_APPLE_COMMON_CRYPTO on Mac OS
    --# 10.4 or older ("Tiger", released in early 2005).
    +-# default on that OS. On macOS 01.4 (Tiger) or older,
    +-# NO_APPLE_COMMON_CRYPTO is defined by default.
     +# Define APPLE_COMMON_CRYPTO_SHA1 to use Apple's CommonCrypto for
     +# SHA-1.
      #
    - # === SHA-256 backend ===
    - #
    + # If don't enable any of the *_SHA1 settings in this section, Git will
    + # default to its built-in sha1collisiondetection library, which is a
     @@ Makefile: ifdef NO_POSIX_GOODIES
      	BASIC_CFLAGS += -DNO_POSIX_GOODIES
      endif
    @@ Makefile: ifdef BLK_SHA1
      else
     
      ## ci/lib.sh ##
    -@@ ci/lib.sh: macos-latest)
    - esac
    - 
    - case "$jobname" in
    -+osx-clang)
    -+	MAKEFLAGS="$MAKEFLAGS APPLE_COMMON_CRYPTO_SHA1=Yes"
    -+	;;
    - linux32)
    - 	CC=gcc
    +@@ ci/lib.sh: macos-*)
    + 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
    + 	else
    + 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
    +-		MAKEFLAGS="$MAKEFLAGS NO_APPLE_COMMON_CRYPTO=NoThanks"
    +-		MAKEFLAGS="$MAKEFLAGS NO_OPENSSL=NoThanks"
    ++		MAKEFLAGS="$MAKEFLAGS APPLE_COMMON_CRYPTO_SHA1=Yes"
    + 	fi
      	;;
    + esac

 Makefile  | 10 ++++------
 ci/lib.sh |  3 +--
 2 files changed, 5 insertions(+), 8 deletions(-)

Comments

Junio C Hamano Dec. 15, 2022, 9:17 p.m. UTC | #1

Ævar Arnfjörð Bjarmason  <[e-mail]> writes:

> Let's also use sha1collisiondetection on OSX, to do so we'll need to
> split up the "APPLE_COMMON_CRYPTO" flag into that flag and a new
> "APPLE_COMMON_CRYPTO_SHA1".

That's well reasoned

This leaves these in Makefile:

        # Define NO_APPLE_COMMON_CRYPTO if you are building on Darwin/Mac OS X
        # and do not want to use Apple's CommonCrypto library.  This allows you
        # to provide your own OpenSSL library, for example from MacPorts.

        ifndef NO_APPLE_COMMON_CRYPTO
                NO_OPENSSL = YesPlease
                APPLE_COMMON_CRYPTO = YesPlease
                COMPAT_CFLAGS += -DAPPLE_COMMON_CRYPTO
        endif

Because we only mention what it does to use NO_APPLE_COMMON_CRYPTO
here, without promising anything about what happens otherwise, we
can do this change without breaking any promises ;-)

However there is this bit:

        ifdef APPLE_COMMON_CRYPTO
                LIB_4_CRYPTO += -framework Security -framework CoreFoundation
        endif

So, if one says 

    make NO_APPLE_COMMON_CRYPTO=NoThanks APPLE_COMMON_CRYPTO_SHA1=YesPlease

presumably the end result will fail to link?  I _think_ that falls
into "if it hurts, don't do that", and 

 (1) automatically disabling the latter when the former is set would
     be more confusing than helpful.

 (2) explicitly detecting the situation and issue an error message
     from the Makefile might appear nicer, but if the linker does
     the failing with a messaage fine, that would be sufficient.

so in the end, the posted patch as-is should be fine, I think.

Patch

diff --git a/Makefile b/Makefile
index 0f7d7ab1fd2..db447d07383 100644
--- a/Makefile
+++ b/Makefile
@@ -511,10 +511,8 @@  include shared.mak
 # Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled
 # with git (in the block-sha1/ directory).
 #
-# Define NO_APPLE_COMMON_CRYPTO on OSX to opt-out of using the
-# "APPLE_COMMON_CRYPTO" backend for SHA-1, which is currently the
-# default on that OS. On macOS 01.4 (Tiger) or older,
-# NO_APPLE_COMMON_CRYPTO is defined by default.
+# Define APPLE_COMMON_CRYPTO_SHA1 to use Apple's CommonCrypto for
+# SHA-1.
 #
 # If don't enable any of the *_SHA1 settings in this section, Git will
 # default to its built-in sha1collisiondetection library, which is a
@@ -1911,7 +1909,7 @@  ifdef NO_POSIX_GOODIES
 	BASIC_CFLAGS += -DNO_POSIX_GOODIES
 endif
 
-ifdef APPLE_COMMON_CRYPTO
+ifdef APPLE_COMMON_CRYPTO_SHA1
 	# Apple CommonCrypto requires chunking
 	SHA1_MAX_BLOCK_SIZE = 1024L*1024L*1024L
 endif
@@ -1928,7 +1926,7 @@  ifdef BLK_SHA1
 	LIB_OBJS += block-sha1/sha1.o
 	BASIC_CFLAGS += -DSHA1_BLK
 else
-ifdef APPLE_COMMON_CRYPTO
+ifdef APPLE_COMMON_CRYPTO_SHA1
 	COMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL
 	BASIC_CFLAGS += -DSHA1_APPLE
 else
diff --git a/ci/lib.sh b/ci/lib.sh
index 706e3ba7e93..db7105e8a8d 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -258,8 +258,7 @@  macos-*)
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
 	else
 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
-		MAKEFLAGS="$MAKEFLAGS NO_APPLE_COMMON_CRYPTO=NoThanks"
-		MAKEFLAGS="$MAKEFLAGS NO_OPENSSL=NoThanks"
+		MAKEFLAGS="$MAKEFLAGS APPLE_COMMON_CRYPTO_SHA1=Yes"
 	fi
 	;;
 esac