From patchwork Wed Jul 14 17:23:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12377517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC121C07E9C for ; Wed, 14 Jul 2021 17:24:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8FB0D613C3 for ; Wed, 14 Jul 2021 17:24:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238844AbhGNR04 (ORCPT ); Wed, 14 Jul 2021 13:26:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45120 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237956AbhGNR0z (ORCPT ); Wed, 14 Jul 2021 13:26:55 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10411C061760 for ; Wed, 14 Jul 2021 10:24:03 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id k4so4159348wrc.8 for ; Wed, 14 Jul 2021 10:24:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wDEYwpC7L20UMoA/d8ufPq3KGReEptPhuSSLPWrnoSA=; b=Oj0YHXzAjyHMWaYVPLwYCtKJTCjko6jqrNwg2z/EB3z2lFM7P0vBy4kNNiezdkyfz0 k+JWQXFHLHcZjYHG4tQuVIbmVBDLuSvdW/93d2+gkncx+KxPSWjMQPKdaiR/FByMbOFs bcRgxN83OQoYnzN7hFoGUJOiIvTi+LXtQmB8ACVzxs3c1PifdtRDA9FxVNqk2UZmAPwV lxyRPs035Pve1oukkC92Io+F62NxTVJ6EX/QQKEqJPWiIQU+6AdqTLzSrR8IRedWty8R Qr5MKSlTNKTPQCQQeBpHGLlAHCAC/FjXaYSeEG/4JgN463ILbfrKE/2xavRmdyA89LNw pVDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wDEYwpC7L20UMoA/d8ufPq3KGReEptPhuSSLPWrnoSA=; b=Qe1qdxKjO/iLrjLl2n9VMv0PQnIg/RqwxhPGQ/0oqGD/dx/vA10iTfoIM3IYiKyMhz tsRHUxnFJlEimXt7tBWi5n5Ird5jNJVO1UOuT+O+ZT1rDHHVSyw3tHUW6Bm9LqQfssve MJj8wcoUm0NkdoqhzxgM1Wy6Zw0cG2hjCqZXd026XnmLNEcAxJxCFkP9OKSKfMTzEsI4 WNkORhhcntAGRgTuSU+wcRVjHB/Y1VnQvg72Vz3E0EmsIEeH3XCgJB5YPyif0Vbllh57 I4vUM2wVqJ4s8zfwrTscuM+0RUnDE8HL67d7iWX05f4eziUTPsk/lDTN/+5T+O1hDpnV CN7A== X-Gm-Message-State: AOAM532u03HiNbQIkb1AOQW+Ym/AZMlNhpM5osJV55DFUf0qiB9xx1rJ wZbw7XaiF4JElqY+c99rApsXjvtlQ5ejlnTy X-Google-Smtp-Source: ABdhPJxSqALrg2tiRntKog/+9BNwVbN3WNL2GkGVEN0itvJ9Dl5449FrfwNjTsdtAzhB36QXtVoLIg== X-Received: by 2002:a5d:6c6e:: with SMTP id r14mr14729289wrz.242.1626283441381; Wed, 14 Jul 2021 10:24:01 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id f5sm3823795wrg.67.2021.07.14.10.24.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Jul 2021 10:24:00 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Andrzej Hunt , =?utf-8?b?TMOpbmHDr2MgSHVhcmQ=?= , Derrick Stolee , Felipe Contreras , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v2 1/4] tests: add a test mode for SANITIZE=leak, run it in CI Date: Wed, 14 Jul 2021 19:23:51 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.853.g5a570c9bf9 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org While git can be compiled with SANITIZE=leak there has been no corresponding GIT_TEST_* mode for it, i.e. memory leaks have been fixed as one-offs without structured regression testing. This change add such a mode, we now have new linux-{clang,gcc}-sanitize-leak CI targets, these targets run the same tests as linux-{clang,gcc}, except that almost all of them are skipped. There is a whitelist of some tests that are OK in test-lib.sh, and individual tests can be opted-in by setting GIT_TEST_SANITIZE_LEAK=true before sourcing test-lib.sh. Within those individual test can be skipped with the "!SANITIZE_LEAK" prerequisite. See the updated t/README for more details. I'm using the GIT_TEST_SANITIZE_LEAK=true and !SANITIZE_LEAK pattern in a couple of tests whose memory leaks I'll fix in subsequent commits. I'm not being aggressive about opting in tests, it's not all tests that currently pass under SANITIZE=leak, just a small number of known-good tests. We can add more later as we fix leaks and grow more confident in this test mode. See the recent discussion at [1] about the lack of this sort of test mode, and 0e5bba53af (add UNLEAK annotation for reducing leak false positives, 2017-09-08) for the initial addition of SANITIZE=leak. See also 09595ab381 (Merge branch 'jk/leak-checkers', 2017-09-19), 7782066f67 (Merge branch 'jk/apache-lsan', 2019-05-19) and the recent 936e58851a (Merge branch 'ah/plugleaks', 2021-05-07) for some of the past history of "one-off" SANITIZE=leak (and more) fixes. When calling maybe_skip_all_sanitize_leak matching against "$TEST_NAME" instead of "$this_test" as other "match_pattern_list()" users do is intentional. I'd like to match things like "t13*config*" in subsequent commits. This part of the API isn't public, so we can freely change it in the future. 1. https://lore.kernel.org/git/87czsv2idy.fsf@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 6 ++++ Makefile | 5 ++++ ci/install-dependencies.sh | 4 +-- ci/lib.sh | 18 ++++++++---- ci/run-build-and-tests.sh | 4 +-- t/README | 16 ++++++++++ t/t5701-git-serve.sh | 2 +- t/test-lib.sh | 60 ++++++++++++++++++++++++++++++++++++++ 8 files changed, 105 insertions(+), 10 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 73856bafc9..752fe187f9 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -297,6 +297,12 @@ jobs: - jobname: linux-gcc-default cc: gcc pool: ubuntu-latest + - jobname: linux-clang-sanitize-leak + cc: clang + pool: ubuntu-latest + - jobname: linux-gcc-sanitize-leak + cc: gcc + pool: ubuntu-latest env: CC: ${{matrix.vector.cc}} jobname: ${{matrix.vector.jobname}} diff --git a/Makefile b/Makefile index 502e0c9a81..d4cad5136f 100644 --- a/Makefile +++ b/Makefile @@ -1216,6 +1216,9 @@ PTHREAD_CFLAGS = SPARSE_FLAGS ?= SP_EXTRA_FLAGS = -Wno-universal-initializer +# For informing GIT-BUILD-OPTIONS of the SANITIZE=leak target +SANITIZE_LEAK = + # For the 'coccicheck' target; setting SPATCH_BATCH_SIZE higher will # usually result in less CPU usage at the cost of higher peak memory. # Setting it to 0 will feed all files in a single spatch invocation. @@ -1260,6 +1263,7 @@ BASIC_CFLAGS += -DSHA1DC_FORCE_ALIGNED_ACCESS endif ifneq ($(filter leak,$(SANITIZERS)),) BASIC_CFLAGS += -DSUPPRESS_ANNOTATED_LEAKS +SANITIZE_LEAK = YesCompiledWithIt endif ifneq ($(filter address,$(SANITIZERS)),) NO_REGEX = NeededForASAN @@ -2793,6 +2797,7 @@ GIT-BUILD-OPTIONS: FORCE @echo NO_UNIX_SOCKETS=\''$(subst ','\'',$(subst ','\'',$(NO_UNIX_SOCKETS)))'\' >>$@+ @echo PAGER_ENV=\''$(subst ','\'',$(subst ','\'',$(PAGER_ENV)))'\' >>$@+ @echo DC_SHA1=\''$(subst ','\'',$(subst ','\'',$(DC_SHA1)))'\' >>$@+ + @echo SANITIZE_LEAK=\''$(subst ','\'',$(subst ','\'',$(SANITIZE_LEAK)))'\' >>$@+ @echo X=\'$(X)\' >>$@+ ifdef TEST_OUTPUT_DIRECTORY @echo TEST_OUTPUT_DIRECTORY=\''$(subst ','\'',$(subst ','\'',$(TEST_OUTPUT_DIRECTORY)))'\' >>$@+ diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 67852d0d37..8ac72d7246 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -12,13 +12,13 @@ UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl" case "$jobname" in -linux-clang|linux-gcc) +linux-clang|linux-gcc|linux-clang-sanitize-leak|linux-gcc-sanitize-leak) sudo apt-add-repository -y "ppa:ubuntu-toolchain-r/test" sudo apt-get -q update sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ $UBUNTU_COMMON_PKGS case "$jobname" in - linux-gcc) + linux-gcc|linux-gcc-sanitize-leak) sudo apt-get -q -y install gcc-8 ;; esac diff --git a/ci/lib.sh b/ci/lib.sh index 476c3f369f..bb02b5abf4 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -183,14 +183,16 @@ export GIT_TEST_CLONE_2GB=true export SKIP_DASHED_BUILT_INS=YesPlease case "$jobname" in -linux-clang|linux-gcc) - if [ "$jobname" = linux-gcc ] - then +linux-clang|linux-gcc|linux-clang-sanitize-leak|linux-gcc-sanitize-leak) + case "$jobname" in + linux-gcc|linux-gcc-sanitize-leak) export CC=gcc-8 MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python3" - else + ;; + *) MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python2" - fi + ;; + esac export GIT_TEST_HTTPD=true @@ -233,4 +235,10 @@ linux-musl) ;; esac +case "$jobname" in +linux-clang-sanitize-leak|linux-gcc-sanitize-leak) + export SANITIZE=leak + ;; +esac + MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index 3ce81ffee9..5fe047b5c6 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -12,7 +12,7 @@ esac make case "$jobname" in -linux-gcc) +linux-gcc|linux-gcc-sanitize-leak) export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main make test export GIT_TEST_SPLIT_INDEX=yes @@ -29,7 +29,7 @@ linux-gcc) export GIT_TEST_CHECKOUT_WORKERS=2 make test ;; -linux-clang) +linux-clang|linux-clang-sanitize-leak) export GIT_TEST_DEFAULT_HASH=sha1 make test export GIT_TEST_DEFAULT_HASH=sha256 diff --git a/t/README b/t/README index 1a2072b2c8..303d0be817 100644 --- a/t/README +++ b/t/README @@ -448,6 +448,22 @@ GIT_TEST_CHECKOUT_WORKERS= overrides the 'checkout.workers' setting to and 'checkout.thresholdForParallelism' to 0, forcing the execution of the parallel-checkout code. +GIT_TEST_SANITIZE_LEAK= will force the tests to run when git +is compiled with SANITIZE=leak (we pick it up via +../GIT-BUILD-OPTIONS). + +By default all tests are skipped when compiled with SANITIZE=leak, and +individual test scripts opt themselves in to leak testing by setting +GIT_TEST_SANITIZE_LEAK=true before sourcing test-lib.sh. Within those +tests use the SANITIZE_LEAK prerequisite to skip individiual tests +(i.e. test_expect_success !SANITIZE_LEAK [...]). + +So the GIT_TEST_SANITIZE_LEAK setting is different in behavior from +both other GIT_TEST_*=[true|false] settings, but more useful given how +SANITIZE=leak works & the state of the test suite. Manually setting +GIT_TEST_SANITIZE_LEAK=true is only useful during development when +finding and fixing memory leaks. + Naming Tests ------------ diff --git a/t/t5701-git-serve.sh b/t/t5701-git-serve.sh index 930721f053..d58efb0aa9 100755 --- a/t/t5701-git-serve.sh +++ b/t/t5701-git-serve.sh @@ -243,7 +243,7 @@ test_expect_success 'unexpected lines are not allowed in fetch request' ' # Test the basics of object-info # -test_expect_success 'basics of object-info' ' +test_expect_success !SANITIZE_LEAK 'basics of object-info' ' test-tool pkt-line pack >in <<-EOF && command=object-info object-format=$(test_oid algo) diff --git a/t/test-lib.sh b/t/test-lib.sh index 7036f83b33..9201510e16 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1353,6 +1353,40 @@ then exit 1 fi +# SANITIZE=leak test mode +sanitize_leak_true= +add_sanitize_leak_true () { + sanitize_leak_true="$sanitize_leak_true$1 " +} + +sanitize_leak_false= +add_sanitize_leak_false () { + sanitize_leak_false="$sanitize_leak_false$1 " +} + +sanitize_leak_opt_in_msg="opt-in with GIT_TEST_SANITIZE_LEAK=true" +maybe_skip_all_sanitize_leak () { + # Whitelist patterns + add_sanitize_leak_true 't000*' + add_sanitize_leak_true 't001*' + add_sanitize_leak_true 't006*' + + # Blacklist patterns (overrides whitelist) + add_sanitize_leak_false 't000[469]*' + add_sanitize_leak_false 't001[2459]*' + add_sanitize_leak_false 't006[0248]*' + + if match_pattern_list "$1" "$sanitize_leak_false" + then + skip_all="test $this_test on SANITIZE=leak blacklist, $sanitize_leak_opt_in_msg" + test_done + elif match_pattern_list "$1" "$sanitize_leak_true" + then + return 0 + fi + return 1 +} + # Are we running this test at all? remove_trash= this_test=${0##*/} @@ -1364,6 +1398,31 @@ then test_done fi +# Aggressively skip non-whitelisted tests when compiled with +# SANITIZE=leak +if test -n "$SANITIZE_LEAK" +then + if test -z "$GIT_TEST_SANITIZE_LEAK" && + maybe_skip_all_sanitize_leak "$TEST_NAME" + then + say_color info >&3 "test $this_test on SANITIZE=leak whitelist" + GIT_TEST_SANITIZE_LEAK=true + fi + + # We need to see it in "git env--helper" (via + # test_bool_env) + export GIT_TEST_SANITIZE_LEAK + + if ! test_bool_env GIT_TEST_SANITIZE_LEAK false + then + skip_all="skip all tests in $this_test under SANITIZE=leak, $sanitize_leak_opt_in_msg" + test_done + fi +elif test_bool_env GIT_TEST_SANITIZE_LEAK false +then + error "GIT_TEST_SANITIZE_LEAK=true has no effect except when compiled with SANITIZE=leak" +fi + # Last-minute variable setup HOME="$TRASH_DIRECTORY" GNUPGHOME="$HOME/gnupg-home-not-used" @@ -1516,6 +1575,7 @@ test -z "$NO_PYTHON" && test_set_prereq PYTHON test -n "$USE_LIBPCRE2" && test_set_prereq PCRE test -n "$USE_LIBPCRE2" && test_set_prereq LIBPCRE2 test -z "$NO_GETTEXT" && test_set_prereq GETTEXT +test -n "$SANITIZE_LEAK" && test_set_prereq SANITIZE_LEAK if test -z "$GIT_TEST_CHECK_CACHE_TREE" then From patchwork Wed Jul 14 17:23:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12377515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40FF8C11F67 for ; Wed, 14 Jul 2021 17:24:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 212CB613B5 for ; Wed, 14 Jul 2021 17:24:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238890AbhGNR05 (ORCPT ); Wed, 14 Jul 2021 13:26:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238701AbhGNR0z (ORCPT ); Wed, 14 Jul 2021 13:26:55 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D9C0FC06175F for ; Wed, 14 Jul 2021 10:24:03 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id d12so4132001wre.13 for ; Wed, 14 Jul 2021 10:24:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rxcziaB04QiHLrU8L8ABGkV2c60dK90wSvkdoe3u8WM=; b=UWwg74IZii4MoYQ2l8+Dn+3I77LXzKUVz4g3syiZEnhuLGP76PryaGUZvl7JMx6NZi M/YrF5kftx3UHpIdx2GenLMeuBvUebat2+8eF1c1jvK4O1+motW5muJgF36iN8wmBolQ Xc03FOcJAQVdOF+bcNNENQ+IlcS0Z3hIMJhZTaYKP+MsItKDYtqr25CQNHvfsZW2Kgwv 2sUquYH5YDq5BDHRtaPUlTbD5iynl+n6Y2Sxvysmh1IdyzYeevlmSM1xyn4i6y4YFIsD 1WimhX5HqGRHApry5h96aBahtXi1+vBzvVAOnlCLlIbTb/ZEO8Ansc7+q6gKCnC20TUn Dkbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rxcziaB04QiHLrU8L8ABGkV2c60dK90wSvkdoe3u8WM=; b=M6qiAcy1DDVKMBe0EsM8tZHTQiGb3J7N++f6sJGopANOZX4s9npNb6rPdVz4QH2bNh dBUFcIVXN9YwRPqGM2B8VuLBAZth39dMaAmmkjkbSpndNidFL3vqiFqfhg7ukWZimHLo vr3GNmyAc9ASLuklLQ2+pFRgSWdUO2/uD2co4CgPv58jt95+NnLnobsM1oXKw+ep5wBn A+ygdgZ33BV3MMkcWWfHb/vMHPOSorSU11tAKbQXTSQTgjuBqSeP2ODRevkaw8hUsqfO PNHpsfp+u5nvDE6/bKxGVwikP+EKpEwVpOrzA+gy4qDP30geAKJbtylfyxLXaZ5Qx7zv KceQ== X-Gm-Message-State: AOAM530jujMKxGSxJSJf/7kx0A06VCX7bbvF2F4MR0qW9XqEX8MEOsVQ EYTsv1McknsNUVlTD+xrhv7ifiQB0Z8Qr1BV X-Google-Smtp-Source: ABdhPJye74bNdprTPKjsIoDtqNITsK1MVpZnbSFGzup6ymX3M3h/vghEWHI833QnOT7x8855AdgYZw== X-Received: by 2002:a05:6000:1b02:: with SMTP id f2mr14295877wrz.315.1626283442197; Wed, 14 Jul 2021 10:24:02 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id f5sm3823795wrg.67.2021.07.14.10.24.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Jul 2021 10:24:01 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Andrzej Hunt , =?utf-8?b?TMOpbmHDr2MgSHVhcmQ=?= , Derrick Stolee , Felipe Contreras , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v2 2/4] SANITIZE tests: fix memory leaks in t13*config*, add to whitelist Date: Wed, 14 Jul 2021 19:23:52 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.853.g5a570c9bf9 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix a couple of trivial memory leaks introduced in 3efd0bedc6 (config: add conditional include, 2017-03-01) and my own 867ad08a26 (hooks: allow customizing where the hook directory is, 2016-05-04). In the latter case the "fix" is UNLEAK() on the global variable. This allows us to run all t13*config* tests under SANITIZE=leak. With this change we can now run almost the whole set of config.c tests (t13*config) under SANITIZE=leak, so let's do so, with a few exceptions: * The test added in ce81b1da23 (config: add new way to pass config via `--config-env`, 2021-01-12), it fails in GitHub CI, but passes for me locally. Let's just skip it for now. * Ditto the split_cmdline and "aliases of builtins" tests, the former required splitting up an existing test, there an issue with the test that would have also been revealed by skipping it. Signed-off-by: Ævar Arnfjörð Bjarmason --- config.c | 17 ++++++++++++----- t/t1300-config.sh | 16 ++++++++++------ t/test-lib.sh | 1 + 3 files changed, 23 insertions(+), 11 deletions(-) diff --git a/config.c b/config.c index f9c400ad30..38e132c0e2 100644 --- a/config.c +++ b/config.c @@ -138,8 +138,10 @@ static int handle_path_include(const char *path, struct config_include_data *inc return config_error_nonbool("include.path"); expanded = expand_user_path(path, 0); - if (!expanded) - return error(_("could not expand include path '%s'"), path); + if (!expanded) { + ret = error(_("could not expand include path '%s'"), path); + goto cleanup; + } path = expanded; /* @@ -149,8 +151,10 @@ static int handle_path_include(const char *path, struct config_include_data *inc if (!is_absolute_path(path)) { char *slash; - if (!cf || !cf->path) - return error(_("relative config includes must come from files")); + if (!cf || !cf->path) { + ret = error(_("relative config includes must come from files")); + goto cleanup; + } slash = find_last_dir_sep(cf->path); if (slash) @@ -168,6 +172,7 @@ static int handle_path_include(const char *path, struct config_include_data *inc ret = git_config_from_file(git_config_include, path, inc); inc->depth--; } +cleanup: strbuf_release(&buf); free(expanded); return ret; @@ -1331,8 +1336,10 @@ static int git_default_core_config(const char *var, const char *value, void *cb) if (!strcmp(var, "core.attributesfile")) return git_config_pathname(&git_attributes_file, var, value); - if (!strcmp(var, "core.hookspath")) + if (!strcmp(var, "core.hookspath")) { + UNLEAK(git_hooks_path); return git_config_pathname(&git_hooks_path, var, value); + } if (!strcmp(var, "core.bare")) { is_bare_repository_cfg = git_config_bool(var, value); diff --git a/t/t1300-config.sh b/t/t1300-config.sh index 9ff46f3b04..93ad0f4887 100755 --- a/t/t1300-config.sh +++ b/t/t1300-config.sh @@ -1050,12 +1050,16 @@ test_expect_success SYMLINKS 'symlink to nonexistent configuration' ' test_must_fail git config --file=linktolinktonada --list ' -test_expect_success 'check split_cmdline return' " - git config alias.split-cmdline-fix 'echo \"' && - test_must_fail git split-cmdline-fix && +test_expect_success 'setup check split_cmdline return' " echo foo > foo && git add foo && - git commit -m 'initial commit' && + git commit -m 'initial commit' +" + +test_expect_success !SANITIZE_LEAK 'check split_cmdline return' " + git config alias.split-cmdline-fix 'echo \"' && + test_must_fail git split-cmdline-fix && + git config branch.main.mergeoptions 'echo \"' && test_must_fail git merge main " @@ -1101,7 +1105,7 @@ test_expect_success 'key sanity-checking' ' git config foo."ba =z".bar false ' -test_expect_success 'git -c works with aliases of builtins' ' +test_expect_success !SANITIZE_LEAK 'git -c works with aliases of builtins' ' git config alias.checkconfig "-c foo.check=bar config foo.check" && echo bar >expect && git checkconfig >actual && @@ -1397,7 +1401,7 @@ test_expect_success 'git --config-env with missing value' ' grep "invalid config format: config" error ' -test_expect_success 'git --config-env fails with invalid parameters' ' +test_expect_success !SANITIZE_LEAK 'git --config-env fails with invalid parameters' ' test_must_fail git --config-env=foo.flag config --bool foo.flag 2>error && test_i18ngrep "invalid config format: foo.flag" error && test_must_fail git --config-env=foo.flag= config --bool foo.flag 2>error && diff --git a/t/test-lib.sh b/t/test-lib.sh index 9201510e16..98e20950c3 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1370,6 +1370,7 @@ maybe_skip_all_sanitize_leak () { add_sanitize_leak_true 't000*' add_sanitize_leak_true 't001*' add_sanitize_leak_true 't006*' + add_sanitize_leak_true 't13*config*' # Blacklist patterns (overrides whitelist) add_sanitize_leak_false 't000[469]*' From patchwork Wed Jul 14 17:23:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12377513 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B064C11F66 for ; Wed, 14 Jul 2021 17:24:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E9029613C1 for ; Wed, 14 Jul 2021 17:24:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238906AbhGNR06 (ORCPT ); Wed, 14 Jul 2021 13:26:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45128 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237956AbhGNR04 (ORCPT ); Wed, 14 Jul 2021 13:26:56 -0400 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5CCDC06175F for ; Wed, 14 Jul 2021 10:24:04 -0700 (PDT) Received: by mail-wm1-x332.google.com with SMTP id g12so2081945wme.2 for ; Wed, 14 Jul 2021 10:24:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rINXkdbXhlGhldlZ8EoFoo8l5VMWkSR/uY3izJeywqE=; b=i0huJwRwX4Z+XH/HyVCW6Lmrcj4k6CEViOcI16UvLDZfWjbOfJzdO6lnbksGLhEak2 PlLROLffy80tYjcvro6OzGionxd1SjGVgUFwybwMCXQLcfcyjtbEu7EpZtaSY7Q/kyND oR2OoPPZQLA855xbKKOZN1IDIyw6WJ908Mg3BnRSYgBChxjY0c+qz8hdf1wfNJVOzcXK Du6Yuf4+MiaO5olXGGipxpGX2XuOIEeIIEaJkd+/sWY1WC5Q0siJu8XmvqEoi8zbwGFW 1Gef3FoXjtBMb4Cy3E/BpHLeogzaOQp2kzDNriJM7bifbxTC35infXIsvzoh0a3Bt50V Ppmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rINXkdbXhlGhldlZ8EoFoo8l5VMWkSR/uY3izJeywqE=; b=DYmEOqkCgljWVG3aIL9GxdznlmqAmMdfVoNXMiTl5kt8HNP4kMfFYLrP0R8YNQ6zBk RCHIhP7Rnw2scVAH2+pHuQrQhAfIJlr8fKBh0AVOKaqCLXIUyph5AW9FH6qarNLt6Goc 8eOitHyg6Fz/hdxpiI4LuO14cXmW6nCfZaUlrLBX8WwUgDyaGKqbHijLLNZEALNbTeUa tlmorpJC4PNAjXaOGuNiwCzqd05UFemPJbnLHz/hMT1iUu0jhyvsCdK4gOm6tTPs2mR3 8OyEu9vuJqi/udzmheYXAehiutkb4Mm1LJAX9ZoLaU+0nPgjZZkaZhAa14ikQi/Zye+y IaTA== X-Gm-Message-State: AOAM533NNUYDyRGt+vKskpnpPV+CTLCrpk7xg6ESuOJw4nT+mkh4bRR7 8XpEgWKcuAOtlAOWZldlMDskYev9Rii3dDAv X-Google-Smtp-Source: ABdhPJxxNrmkg+L4pGzeCu+/ZerFKwRzz20E+w7vKa6vq5tP47342Y0Sdvdd47I41lNL9qmM1smF/g== X-Received: by 2002:a7b:c76b:: with SMTP id x11mr5415541wmk.79.1626283443087; Wed, 14 Jul 2021 10:24:03 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id f5sm3823795wrg.67.2021.07.14.10.24.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Jul 2021 10:24:02 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Andrzej Hunt , =?utf-8?b?TMOpbmHDr2MgSHVhcmQ=?= , Derrick Stolee , Felipe Contreras , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v2 3/4] SANITIZE tests: fix memory leaks in t5701*, add to whitelist Date: Wed, 14 Jul 2021 19:23:53 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.853.g5a570c9bf9 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix a memory leak in a2ba162cda (object-info: support for retrieving object info, 2021-04-20) which appears to have been based on a misunderstanding of how the pkt-line.c API works, there is no need to strdup() input to, it's just a printf()-like format function. This fixes a potentially large memory leak, since the number of OID lines the "object-info" call can be arbitrarily large (or a small one if the request is small). Signed-off-by: Ævar Arnfjörð Bjarmason --- protocol-caps.c | 5 +++-- t/t5701-git-serve.sh | 1 + 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/protocol-caps.c b/protocol-caps.c index 13a9e63a04..901b6795e4 100644 --- a/protocol-caps.c +++ b/protocol-caps.c @@ -69,9 +69,10 @@ static void send_info(struct repository *r, struct packet_writer *writer, } } - packet_writer_write(writer, "%s", - strbuf_detach(&send_buffer, NULL)); + packet_writer_write(writer, "%s", send_buffer.buf); + strbuf_reset(&send_buffer); } + strbuf_release(&send_buffer); } int cap_object_info(struct repository *r, struct strvec *keys, diff --git a/t/t5701-git-serve.sh b/t/t5701-git-serve.sh index d58efb0aa9..e2f4832adf 100755 --- a/t/t5701-git-serve.sh +++ b/t/t5701-git-serve.sh @@ -5,6 +5,7 @@ test_description='test protocol v2 server commands' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +GIT_TEST_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success 'test capability advertisement' ' From patchwork Wed Jul 14 17:23:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12377519 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9CCAC07E9A for ; Wed, 14 Jul 2021 17:24:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CFE18613C3 for ; Wed, 14 Jul 2021 17:24:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239405AbhGNR1C (ORCPT ); Wed, 14 Jul 2021 13:27:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237956AbhGNR06 (ORCPT ); Wed, 14 Jul 2021 13:26:58 -0400 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 988ABC06175F for ; Wed, 14 Jul 2021 10:24:05 -0700 (PDT) Received: by mail-wm1-x336.google.com with SMTP id l4-20020a05600c4f04b0290220f8455631so1938593wmq.1 for ; Wed, 14 Jul 2021 10:24:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rsjH7uH10VvgHyq1XZ15BNWgWFXLQi7LOGIAHd8ON1k=; b=MgNQ/4ezUdJhNuA2P2uPZqHdIVYp5lfh/QHMsgMP9fVUE2Nc2y7LoDDDQlEEMdf/64 to1P/aLvpJMOJID4K+eZ2bjuyJM75PuSEtKQVxlRp+Kpd13UwZ+8bYGIH3S7HU/Hs8fP iMLbemh60nGUM9VDw02jWPHOPv4CTeEidCrHObDu2bLJAdgikdNw7ftYKFCJnevEDIGQ 06wN6snqyXZ5+20HbByEfFC0ZGyGoU+aDj0e0LUTVtv93/JmO8FrM1c+yOAf6sYs95Hh PL4FlnYVRSwscTju2xfd98G1i+GjflCdC8fytLOBHl6a214ZUbxnE9eF1usBaGQEWGz2 O7MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rsjH7uH10VvgHyq1XZ15BNWgWFXLQi7LOGIAHd8ON1k=; b=TJY9mYyis3dhVCrHhVf2ujv13N7PXbofKcSW5dFlvdq90XZUaCc3DdT+IaQ/7ZhoZQ YyU3HHYVomhRFDv7mNOmBq9pqtcDxn2+C4BygMNazenhGqjAnzuksT8D1qwefNYYOLVg FWhfrn084DMTjgfgRynwidkfp4Q9Q184SPC+TqwOc6gRqIV62LTPK0gL3SpMKwHVDj/p uDvExq6ycCiIJ5/voxV9RIgbqVnBXa9vz1Vd10NicX9pGodcXZbGvJhK7Daggw0Z9V3y VnMQQShaVJSilyacObk+Sge4J42W1bo/FHorOy4JqnrC+KhbaXv7T+9+DPNSSlVsPk5E tHpw== X-Gm-Message-State: AOAM530e9sLWsfsc6xN9HSXqNB6W3R1iNRajJbUDKpL8tjfPuC8zdIdT xrjj0RWYxF+d+VVA1BL94Ik7jt1ipqKyLme6 X-Google-Smtp-Source: ABdhPJwFVNpaugd+GHKTkJxwBbL2SAvstwaA9iLwZOZibTBjAJclhx9ZnPRmjrRFMO8uuEd6qoETwg== X-Received: by 2002:a7b:cb92:: with SMTP id m18mr12472153wmi.29.1626283443956; Wed, 14 Jul 2021 10:24:03 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id f5sm3823795wrg.67.2021.07.14.10.24.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Jul 2021 10:24:03 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Andrzej Hunt , =?utf-8?b?TMOpbmHDr2MgSHVhcmQ=?= , Derrick Stolee , Felipe Contreras , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v2 4/4] SANITIZE tests: fix leak in mailmap.c Date: Wed, 14 Jul 2021 19:23:54 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.853.g5a570c9bf9 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Get closer to being able to run t4203-mailmap.sh by fixing a couple of memory leak in mailmap.c. In the free_mailmap_entry() code added in 0925ce4d49 (Add map_user() and clear_mailmap() to mailmap, 2009-02-08) the intent was clearly to clear the "me" structure, but while we freed parts of the mailmap_entry structure, we didn't free the structure itself. The same goes for the "mailmap_info" structure. Signed-off-by: Ævar Arnfjörð Bjarmason --- mailmap.c | 2 ++ t/t4203-mailmap.sh | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/mailmap.c b/mailmap.c index d1f7c0d272..e1c8736093 100644 --- a/mailmap.c +++ b/mailmap.c @@ -36,6 +36,7 @@ static void free_mailmap_info(void *p, const char *s) s, debug_str(mi->name), debug_str(mi->email)); free(mi->name); free(mi->email); + free(mi); } static void free_mailmap_entry(void *p, const char *s) @@ -51,6 +52,7 @@ static void free_mailmap_entry(void *p, const char *s) me->namemap.strdup_strings = 1; string_list_clear_func(&me->namemap, free_mailmap_info); + free(me); } /* diff --git a/t/t4203-mailmap.sh b/t/t4203-mailmap.sh index 0b2d21ec55..c7de4299cf 100755 --- a/t/t4203-mailmap.sh +++ b/t/t4203-mailmap.sh @@ -79,6 +79,12 @@ test_expect_success 'check-mailmap bogus contact --stdin' ' test_must_fail git check-mailmap --stdin bogus expect <<-EOF && $GIT_AUTHOR_NAME (1):