From patchwork Thu May 20 11:22:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270047 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7340C433B4 for ; Thu, 20 May 2021 12:25:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C6AB061261 for ; Thu, 20 May 2021 12:25:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234423AbhETM1D (ORCPT ); Thu, 20 May 2021 08:27:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55306 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232860AbhETM0p (ORCPT ); Thu, 20 May 2021 08:26:45 -0400 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C2D9C0438D8 for ; Thu, 20 May 2021 04:23:17 -0700 (PDT) Received: by mail-ej1-x630.google.com with SMTP id k14so21204504eji.2 for ; Thu, 20 May 2021 04:23:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=An/x+D4iuFeJXmT+0NRANkZQJPSij8xl4RgnQ1Uq8EM=; b=tANi5UOSstt/rjLm7/hFWyVKvyMu75l64ELMwLEenNsfA9KMRNqqfFO4RhEY2Rp0O8 IKcEyRsqj9/XRlSKe4Nrr3F9zRWoURen6//LRNdWTLzPv97DWWOmIxDNHjdo9puvQp0l vTXC1N3oO456Siri2aTWk/0nN+u9Pe+tUndCVF8nnUICNlnO2yBdsYb1dYll6gyXzL4V zg6hRfbyYW1Jnrsldp1SxFijqsANHO98TLxnquLwBPB25o2pAamXIRbxh1l4Lui+M4fz QkpBYb4oL/u+hxLArVoZzpZtf/661YFOjiWqeWufbXwRbjw6Z6hZGmt20u1jprVRGz7W EZJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=An/x+D4iuFeJXmT+0NRANkZQJPSij8xl4RgnQ1Uq8EM=; b=bwl5PB9oTPZpRvmOwylnmhmnLjD2d9BNuT+e1Fm0Gzvv5PCKFZaFdeuunDe6YYBKWR rVHFdUMSc09Re2JC/2IGT0GjKa2IIjCkPSVOo1bP1s+8dZXTzwc2tKb/e5WMMfxIsD5A AikZRO7gjFAoZ/sR0ln/eXgElzIiq56xjlmj993CukA1+Xc4ykChSpDMUAFYXqzbdexB OWKay1JrwBo6EykZzN/EmIfF7CqXc/JX9PIuCm+4bLRTZI/imQxZoJIlmyB65aoOYO6U GnmuhU5M37d7eTSpi63muNtWuDnpWlfrlIG9Keri5O9qrID1giBLPUCMKFBLpYYYnT7B FMcg== X-Gm-Message-State: AOAM533i1esOySWxov5bTzxL2kno+NU9cIWzEr6S8ClQ4v0xllFW55s+ 2Rxb9o6CRFgPDU9hNRPyZ799ZSBaYafNgw== X-Google-Smtp-Source: ABdhPJyapfuq7dl14+dldvmHqsvf4wy3vQBim3dZGLezuu1nn4O2fN3RvnE5d+fMRO7UPr8/h4bPlw== X-Received: by 2002:a17:906:1cc3:: with SMTP id i3mr4323004ejh.87.1621509795512; Thu, 20 May 2021 04:23:15 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:14 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 01/17] fsck tests: refactor one test to use a sub-repo Date: Thu, 20 May 2021 13:22:55 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Refactor one of the fsck tests to use a throwaway repository. It's a pervasive pattern in t1450-fsck.sh to spend a lot of effort on the teardown of a tests so we're not leaving corrupt content for the next test. We should instead simply use something like this test_create_repo pattern. It's both less verbose, and makes things easier to debug as a failing test can have their state left behind under -d without damaging the state for other tests. But let's punt on that general refactoring and just change this one test, I'm going to change it further in subsequent commits. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1450-fsck.sh | 34 ++++++++++++++++------------------ 1 file changed, 16 insertions(+), 18 deletions(-) diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh index 5071ac63a5..1563b35f88 100755 --- a/t/t1450-fsck.sh +++ b/t/t1450-fsck.sh @@ -48,24 +48,22 @@ remove_object () { rm "$(sha1_file "$1")" } -test_expect_success 'object with bad sha1' ' - sha=$(echo blob | git hash-object -w --stdin) && - old=$(test_oid_to_path "$sha") && - new=$(dirname $old)/$(test_oid ff_2) && - sha="$(dirname $new)$(basename $new)" && - mv .git/objects/$old .git/objects/$new && - test_when_finished "remove_object $sha" && - git update-index --add --cacheinfo 100644 $sha foo && - test_when_finished "git read-tree -u --reset HEAD" && - tree=$(git write-tree) && - test_when_finished "remove_object $tree" && - cmt=$(echo bogus | git commit-tree $tree) && - test_when_finished "remove_object $cmt" && - git update-ref refs/heads/bogus $cmt && - test_when_finished "git update-ref -d refs/heads/bogus" && - - test_must_fail git fsck 2>out && - test_i18ngrep "$sha.*corrupt" out +test_expect_success 'object with hash mismatch' ' + test_create_repo hash-mismatch && + ( + cd hash-mismatch && + oid=$(echo blob | git hash-object -w --stdin) && + old=$(test_oid_to_path "$oid") && + new=$(dirname $old)/$(test_oid ff_2) && + oid="$(dirname $new)$(basename $new)" && + mv .git/objects/$old .git/objects/$new && + git update-index --add --cacheinfo 100644 $oid foo && + tree=$(git write-tree) && + cmt=$(echo bogus | git commit-tree $tree) && + git update-ref refs/heads/bogus $cmt && + test_must_fail git fsck 2>out && + test_i18ngrep "$oid.*corrupt" out + ) ' test_expect_success 'branch pointing to non-commit' ' From patchwork Thu May 20 11:22:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270049 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14257C433ED for ; Thu, 20 May 2021 12:25:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E7F2161353 for ; Thu, 20 May 2021 12:25:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234705AbhETM1G (ORCPT ); Thu, 20 May 2021 08:27:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231601AbhETM0q (ORCPT ); Thu, 20 May 2021 08:26:46 -0400 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 34DBBC0438D9 for ; Thu, 20 May 2021 04:23:18 -0700 (PDT) Received: by mail-ej1-x633.google.com with SMTP id et19so17610029ejc.4 for ; Thu, 20 May 2021 04:23:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tlCHGDetKv+3INTN1nkBs7u/eHpRz5fS/FiQYeYL2U8=; b=Huj7yzF5TgLazz7405qcGc8Gp5XQ0b1omOreY9MKC9QS5evNA9WsiuN3V9rzy5NFXs 3RfnJS/vyt9z6kNMJk5BIZqQ1vNwxK6Cg0t7eNr2E/NEM0wdTniTp0Kr5lStjRNPqLZV kboMpJnud2hH7C3cZYFmU0rr4oJcKepIYxXaC8Vxn4OpUIBjRdW7qOlrA9unRvzmZk8y 0hux/n8TGfcO+4ylHhLle7UiiYpQ7bJSqPAQm5wvbDrq82g0aV0bqEWWUamZUH+nfZB0 yf3iRL5KKTuWLd/zraDzX+n0SE73nKEa8Y7tWGc4Y01+fWpBptOyfgqdhTFbbQDbBYi9 edBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tlCHGDetKv+3INTN1nkBs7u/eHpRz5fS/FiQYeYL2U8=; b=PqzCTE5zu0gBV556SALcPY2kOVuiDswSKfOyu8Bb4/e3kVljxBUG5WVG44ad4hDLsj NF6nBomyV6pX/fnR8LjykGwAQCXepmpac2DImBtqHBejKx6hT9YyGMNbK6h1aWbnlPm5 Tp6IQDZXkN13xmHO+dQYvoJDERvs/MgUAb3832I/sd/0pzunGVe6EZQVqE8C+K+7ybhH 96ShtWjozjf63in3sm9Ma7ynxqWcEYeKvZRdbGB6Pd/HDA5ISBaOZzgUXtBFCizGAAWS ux8hod4aoC02+IOxJYFEfdiLI5OP76JooeMs/+/HBbiHY8w079ClLi+7scBfSqlmsxK6 U6XA== X-Gm-Message-State: AOAM533MdvbwSx/9fa8T3i6g2dDj27PfDZ41e62F6wJM0fgZXoeR1pCn WMpjlO7Q2Xn0MhymC7COGV91vC+72gyLGw== X-Google-Smtp-Source: ABdhPJwVhz4OjAx63nqiyYgC3NN+VCitYXn+JPPRcZOTZg4xfubCMzhdUi4X2H88NuldnKBMUvWoEQ== X-Received: by 2002:a17:906:eb0c:: with SMTP id mb12mr4131788ejb.109.1621509796516; Thu, 20 May 2021 04:23:16 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:15 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 02/17] fsck tests: add test for fsck-ing an unknown type Date: Thu, 20 May 2021 13:22:56 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix a blindspot in the fsck tests by checking what we do when we encounter an unknown "garbage" type produced with hash-object's --literally option. This behavior needs to be improved, which'll be done in subsequent patches, but for now let's test for the current behavior. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1450-fsck.sh | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh index 1563b35f88..f36ec1e2f4 100755 --- a/t/t1450-fsck.sh +++ b/t/t1450-fsck.sh @@ -863,4 +863,16 @@ test_expect_success 'detect corrupt index file in fsck' ' test_i18ngrep "bad index file" errors ' +test_expect_success 'fsck hard errors on an invalid object type' ' + test_create_repo garbage-type && + empty_blob=$(git -C garbage-type hash-object --stdin -w -t blob err.expect <<-\EOF && + fatal: invalid object type + EOF + test_must_fail git -C garbage-type fsck >out.actual 2>err.actual && + test_cmp err.expect err.actual && + test_must_be_empty out.actual +' + test_done From patchwork Thu May 20 11:22:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38FCDC433B4 for ; Thu, 20 May 2021 12:25:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1E16E611ED for ; Thu, 20 May 2021 12:25:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235367AbhETM1P (ORCPT ); Thu, 20 May 2021 08:27:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54754 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232884AbhETM0s (ORCPT ); Thu, 20 May 2021 08:26:48 -0400 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 236BEC0438DA for ; Thu, 20 May 2021 04:23:19 -0700 (PDT) Received: by mail-ej1-x632.google.com with SMTP id lz27so24632143ejb.11 for ; Thu, 20 May 2021 04:23:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iithknjOnpC5bxEYiDsJLW3r9clveNKxwKwkHYWLDLc=; b=nH8UgIT26gnHplkKjVGcTfWkW5Xfdw074VXSel7gB9EhZ0EqtWQHJXt3hkFhSBPzZA auZxAeYnujQDiD3Fou1B3izj5tGvCgyZIBVtUBc3YdxXmFbOgoutyhxFBET1TKmWWnYW hLZR2H8c7XAAuS6MF13+Xagrs+fF6lpUjSyE2/aKveGDUJHMmJTGL1BCbKup/k1N1TUR 4qos2C+A1j726048m8tjx+5GmDtt/mL4O9axB/hnoi13YOYVySHs5uEz/ajpLuR2LKk4 Nxu96B/Yjep+OFyh9u/G1m9ZTTQsfltOt3z3j57ACUKruQ41xRpEeRLW/LjjvOo3XWY8 ZIdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iithknjOnpC5bxEYiDsJLW3r9clveNKxwKwkHYWLDLc=; b=N7MuN/z2Lm7n9DOLp/yMvyeTnwu0iuE2DJdOsuKDZpeNBFMqdR2lmWvLefiaArQNyb SIV9sat7B4rPoH5YyeA35sFU1B59vAMiJp9ESex53A7RoTMAd4djZgZ0hYOduF8KS94u 5uwZ4nG4PNPGLP0FyRWYl6iaTxEoNnlx5CmiDjFN+u+wEmj5Nno6BH7UA53VZ6dK7L4U QoCZQ+wlZIBEUHwhhtrRARjgdcl9eMMpiKh/bW+ptwf3YT8u6VyI72Nq2dJ6q8A0GAyZ 6xb7y1OJb8bR4bPh72H7xNQFFwqjhr64zz8thSYcp1UstKOo3C2JGj2TwBYND0nTciSm tcWg== X-Gm-Message-State: AOAM531mAIAKAoYKd+G9A5Ln5nu0upnR7hx4MTZjkfMsTtCJmOpY9og1 HQJabMX9nJa/2rnz9W885Vwrktd0c5VWfQ== X-Google-Smtp-Source: ABdhPJyhqaTPVsUHmO5x2kkc9Q2/NEqoLw1suKqrGXQgBGqH1R4JA8KdOw7mRcXD/hy7P5GVLZjcrw== X-Received: by 2002:a17:906:2596:: with SMTP id m22mr4231492ejb.175.1621509797517; Thu, 20 May 2021 04:23:17 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:16 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 03/17] cat-file tests: test for missing object with -t and -s Date: Thu, 20 May 2021 13:22:57 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Test for what happens when the -t and -s flags are asked to operate on a missing object, this extends tests added in 3e370f9faf0 (t1006: add tests for git cat-file --allow-unknown-type, 2015-05-03). The -t and -s flags are the only ones that can be combined with --allow-unknown-type, so let's test with and without that flag. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1006-cat-file.sh | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 5d2dc99b74..b71ef94329 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -315,6 +315,33 @@ test_expect_success '%(deltabase) reports packed delta bases' ' } ' +missing_oid=$(test_oid deadbeef) +test_expect_success 'error on type of missing object' ' + cat >expect.err <<-\EOF && + fatal: git cat-file: could not get object info + EOF + test_must_fail git cat-file -t $missing_oid >out 2>err && + test_must_be_empty out && + test_cmp expect.err err && + + test_must_fail git cat-file -t --allow-unknown-type $missing_oid >out 2>err && + test_must_be_empty out && + test_cmp expect.err err +' + +test_expect_success 'error on size of missing object' ' + cat >expect.err <<-\EOF && + fatal: git cat-file: could not get object info + EOF + test_must_fail git cat-file -s $missing_oid >out 2>err && + test_must_be_empty out && + test_cmp expect.err err && + + test_must_fail git cat-file -s --allow-unknown-type $missing_oid >out 2>err && + test_must_be_empty out && + test_cmp expect.err err +' + bogus_type="bogus" bogus_content="bogus" bogus_size=$(strlen "$bogus_content") From patchwork Thu May 20 11:22:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BF86C43460 for ; Thu, 20 May 2021 12:25:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7E506611ED for ; Thu, 20 May 2021 12:25:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235246AbhETM1N (ORCPT ); Thu, 20 May 2021 08:27:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232596AbhETM0r (ORCPT ); Thu, 20 May 2021 08:26:47 -0400 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF341C0438DB for ; Thu, 20 May 2021 04:23:19 -0700 (PDT) Received: by mail-ej1-x62a.google.com with SMTP id n2so24655062ejy.7 for ; Thu, 20 May 2021 04:23:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wwnOwso6Xzm0e73vrG/Ct+tR69VMpLcONZz8qyaKAaY=; b=NGoVNIwQSjhLPmCOy4hEtNFOvV2Z5fEHGw85E1lmAbq5fHe2qN9lTR7p1RJ8oscXGf tJOEa/Jcnt7wqdIGYGK2oSQAIjsyOQGyMk2ck5kktoWEt2Gft/v1F/lNMhWaSAwvTsGF MnJZgLYYlX4lqK3wuEoqq/o4u4evUg8hQhy2C9pzmbjnT2DwWk+YqZSEQi7WxSnRD8cF EB8UFLXHWzaAReCCVkxiujdI40LhL+wwX9cdTyvybNeka0caqNYT3gRA3XSVQt8Fzjze XKud00JTt45DgwAvjvgLE03wBhQqlbTboKUpuugX0XNEK8kD/w9yTIbG6q/J6B9rkquR Usqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wwnOwso6Xzm0e73vrG/Ct+tR69VMpLcONZz8qyaKAaY=; b=m0+ikr4HIjGtxq0Nvekp+++3gmGD6Q1I8C8AqVdjF9sz2KHJ+VLlgKR1HoHeyfCAhd P8vMCaSgHnYQsxAwKvaIxXI2JsXe7jb6NKsW3GDFjGWOsg61eLJG0X5YlviuIHOB1/Ao 20aW5aERnCrmvQAMh2Yj9RXcOYV2ix9RJun+DRHgf8HxMsI2cwq59eQWsgo0gXaGet5g ve+9e9RiLKvMJ4pqccangjV6mxtM35KzP+Wd0DDy1AP3kNPWG1pX94Q18H2Y1ck5GxkX yLY0xdomafT03cDq7HF/HFWIIE3usGTiGymdZvkirCBEdHLgag9GWBfcQ92jyNFvYASr hZaQ== X-Gm-Message-State: AOAM533zbepDdn8H1HsTphFwMzg4MCp+IyemBEWHH/bWxG5VHBgioa1L JBNzuz8ML69cSxT+6QY6hfp8bN3SNhk2eg== X-Google-Smtp-Source: ABdhPJw91ltmcy4htMRakG0KDaAvTBdtzWLmfRF0Egc4+Gq4SaVuQS0MVWY8M3YUj05+/Vog4L50+g== X-Received: by 2002:a17:907:7654:: with SMTP id kj20mr4207918ejc.439.1621509798205; Thu, 20 May 2021 04:23:18 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:17 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 04/17] cat-file tests: test that --allow-unknown-type isn't on by default Date: Thu, 20 May 2021 13:22:58 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix a blindspot in the tests added in the tests for the --allow-unknown-type feature, added in 39e4ae38804 (cat-file: teach cat-file a '--allow-unknown-type' option, 2015-05-03). Before this change all the tests would succeed if --allow-unknown-type was on by default, let's fix that by asserting that -t and -s die on a "garbage" type without --allow-unknown-type. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1006-cat-file.sh | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index b71ef94329..dc01d7c4a9 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -347,6 +347,20 @@ bogus_content="bogus" bogus_size=$(strlen "$bogus_content") bogus_sha1=$(echo_without_newline "$bogus_content" | git hash-object -t $bogus_type --literally -w --stdin) +test_expect_success 'die on broken object under -t and -s without --allow-unknown-type' ' + cat >err.expect <<-\EOF && + fatal: invalid object type + EOF + + test_must_fail git cat-file -t $bogus_sha1 >out.actual 2>err.actual && + test_cmp err.expect err.actual && + test_must_be_empty out.actual && + + test_must_fail git cat-file -s $bogus_sha1 >out.actual 2>err.actual && + test_cmp err.expect err.actual && + test_must_be_empty out.actual +' + test_expect_success "Type of broken object is correct" ' echo $bogus_type >expect && git cat-file -t --allow-unknown-type $bogus_sha1 >actual && @@ -363,6 +377,21 @@ bogus_content="bogus" bogus_size=$(strlen "$bogus_content") bogus_sha1=$(echo_without_newline "$bogus_content" | git hash-object -t $bogus_type --literally -w --stdin) +test_expect_success 'die on broken object with large type under -t and -s without --allow-unknown-type' ' + cat >err.expect <<-EOF && + error: unable to unpack $bogus_sha1 header + fatal: git cat-file: could not get object info + EOF + + test_must_fail git cat-file -t $bogus_sha1 >out.actual 2>err.actual && + test_cmp err.expect err.actual && + test_must_be_empty out.actual && + + test_must_fail git cat-file -s $bogus_sha1 >out.actual 2>err.actual && + test_cmp err.expect err.actual && + test_must_be_empty out.actual +' + test_expect_success "Type of broken object is correct when type is large" ' echo $bogus_type >expect && git cat-file -t --allow-unknown-type $bogus_sha1 >actual && From patchwork Thu May 20 11:22:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270055 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CCCBC433B4 for ; Thu, 20 May 2021 12:26:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0193A61261 for ; Thu, 20 May 2021 12:25:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236591AbhETM1U (ORCPT ); Thu, 20 May 2021 08:27:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55318 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233684AbhETM0u (ORCPT ); Thu, 20 May 2021 08:26:50 -0400 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96C94C0438DC for ; Thu, 20 May 2021 04:23:20 -0700 (PDT) Received: by mail-ej1-x630.google.com with SMTP id i7so6838179ejc.5 for ; Thu, 20 May 2021 04:23:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6KXuiYIOQlsjo6MK7lONu0XRCHCjrtRY9x15o0FhIpM=; b=Sj+yBDmFxm0CzBT1K+3/B6IX8QUjcACVGYP8Weha6c2dMKs9i4F50ZBLEUK4kPNrR5 i0ekBQ1EGCXU/cFCpNfi/dXgmpBsE0QriKjdxROQM9tMlX+gNYigQGgNiR5LwgOg+sqc HD/H5LljieM7wjaleFTwroOd3AVBzpAr/icfZ8IS+wg5QOMhwxVtMA9LWoYfhk9e+n/S nTsSfrsIZBsSG1EOpRvKSkiZSGFzRmO2nl/qENuD8wxtasiJhy5Ry8xJMV7UQznNaGgT quGEkXY9Vtnw2P9OPMX2JigYtIooSFG8Flt2kUNAJCOyO+X2gDONmLssB6ogranSY59Q UCeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6KXuiYIOQlsjo6MK7lONu0XRCHCjrtRY9x15o0FhIpM=; b=BKApENuQFaF5wCftWj0Ai389bDUegw8hIBI++PFi/Hw+g40+ckwgrlBLq2aM1XVSHe IThQ2cItRWaLyXhSmXGYcacyZMvjxeEowZXnr4vMK8XCuKtpPTBBrDw5AX8AhrECowMX R+k+WI04pQfN6UC3e9DGDshu58gtStSgUUCZhbWSAWiHtZBQN4Nx36TqRBmhjZsdkg5l AjwJ5zVblQsrmjA1QQ1QJ8n14oCkleROXkigX/40JG858+3Meb1JFi3E22GA2NQ1iAt6 hrPI5/FKPB50Zfg6n34Z9VzFDY9cDwIFpayIFH2PlJ1OF2Lpy83XmzBrfVmkx9g1QPZX N1tA== X-Gm-Message-State: AOAM530j/vxdaCIIHlszJTTZjWkIV6vpYSA38yQVgTru35MQ2DyiRfAY 7ovzG7wOcxJPd1crNaOY/8xbJ/ZdX6Sztw== X-Google-Smtp-Source: ABdhPJxKLuLZ7bZVpJPWtikxtbNVHL7F/gf69+anZSAkkJXil7vyVC5XAOtZUdmHG/vOCHZ5IibPEA== X-Received: by 2002:a17:907:20d7:: with SMTP id qq23mr4154450ejb.549.1621509798980; Thu, 20 May 2021 04:23:18 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:18 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 05/17] rev-list tests: test for behavior with invalid object types Date: Thu, 20 May 2021 13:22:59 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix a blindspot in the tests for the "rev-list --disk-usage" feature added in 16950f8384a (rev-list: add --disk-usage option for calculating disk usage, 2021-02-09) to test for what happens when it's asked to calculate the disk usage of invalid object types. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t6115-rev-list-du.sh | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/t/t6115-rev-list-du.sh b/t/t6115-rev-list-du.sh index b4aef32b71..edb2ed5584 100755 --- a/t/t6115-rev-list-du.sh +++ b/t/t6115-rev-list-du.sh @@ -48,4 +48,15 @@ check_du HEAD check_du --objects HEAD check_du --objects HEAD^..HEAD +test_expect_success 'setup garbage repository' ' + git clone --bare . garbage.git && + garbage_oid=$(git -C garbage.git hash-object -t garbage -w --stdin --literally garbage.git/refs/tags/garbage-tag && + test_must_fail git -C garbage.git rev-list --objects --all --disk-usage +' + test_done From patchwork Thu May 20 11:23:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 514BEC433ED for ; Thu, 20 May 2021 12:26:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3861B61261 for ; Thu, 20 May 2021 12:26:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237067AbhETM1V (ORCPT ); Thu, 20 May 2021 08:27:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54416 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232678AbhETM0u (ORCPT ); Thu, 20 May 2021 08:26:50 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6CD5AC0438EB for ; Thu, 20 May 2021 04:23:21 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id lz27so24632293ejb.11 for ; Thu, 20 May 2021 04:23:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ep6MHXX7MgySl2lRuw1QeCEkvT61VaNDLwOeB3SjQlU=; b=ZSIO2k/UGJjyT9Wog/Z3j7qdM+hZvQwuocjjFpGeEE+2/xkEVEpQwrW4KB1RcBKdLZ pbfcTzOakySJj/nInHIxQV3Inij0EplzJTDRjtta6CrO3XtI1sf3TdP3Vx+NLvfgeSAE ZIHQXtrUE5WreSSrZQM1CS+2ce4f8KD/AmDU1USlGj3xpzHzxL7FVVFZd7RQ3LEsYPd3 +gjbV+VtGRgdqPnUi2gPc83cD0DLryODUwdcpFoFaZih6LF4deIERHmPoV8t02JibUoJ l3tHRNDXUACFmKDatounpf44jHiq0uO8Y5TH8RsiMvIVrDuiKEODXwMfbrEU4CpIZrXo FaFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ep6MHXX7MgySl2lRuw1QeCEkvT61VaNDLwOeB3SjQlU=; b=ciyqOhSDTGklotlXMk7KAWYQSW3HrY3CJ6KgLb07pdNkNhsp2EY/aVK1IRX4cT4wjD CxHrNbQGlrdgFY/zItcdgrUvJb65nzclK8WU7fTGTJAK5ejmFgMCJoxLpXY1r2O3PkQN 9kxStuNp7sxIRw8VLfxA64r09Al5OF18/lFvHkgiqLgm+0dS1K2uS1BlSl60TADM02m2 P9Arw7tv6aawFmJb6ni3fVZ09Dg0C7XrRvmv8qISWU0OjSaQJt6GGBcQ+FlmHbyj0n7X tU6hcfT+qLT9wMBEKhsljnNUYCLm87d2WpPswgjzfoKJXhbrm8RVcmh5e8+9RNdgpnFV RdoQ== X-Gm-Message-State: AOAM5315N/ayyd7D1FT5JGSzCw2F5lURtyPKipIHk+X0fW5vGIMlFZER DRGve7WWWdp+1fqJi3tYPxSENUT0hJGQuw== X-Google-Smtp-Source: ABdhPJyts7c2lCDPLT5ef6rQKxBPXsCtlPvboT3/SrNBtdMyL1eYnh8ylbx70gIuiQ2m1Ggj09lPgg== X-Received: by 2002:a17:906:2f91:: with SMTP id w17mr4213003eji.443.1621509799770; Thu, 20 May 2021 04:23:19 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:19 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 06/17] cat-file tests: add corrupt loose object test Date: Thu, 20 May 2021 13:23:00 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix a blindspot in the tests for "cat-file" (and by proxy, the guts of object-file.c) by testing that when we can't decode a loose object with zlib we'll emit an error from zlib.c. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1006-cat-file.sh | 52 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index dc01d7c4a9..4a76ff024d 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -404,6 +404,58 @@ test_expect_success "Size of large broken object is correct when type is large" test_cmp expect actual ' +test_expect_success 'cat-file -t and -s on corrupt loose object' ' + git init --bare corrupt-loose.git && + ( + cd corrupt-loose.git && + + # Setup and create the empty blob and its path + empty_path=$(git rev-parse --git-path objects/$(test_oid_to_path "$EMPTY_BLOB")) && + git hash-object -w --stdin other.blob && + other_blob=$(git hash-object -w --stdin out.expect <<-EOF && + 0 + EOF + git cat-file -s "$EMPTY_BLOB" >out.actual 2>err.actual && + test_must_be_empty err.actual && + test_cmp out.expect out.actual && + + # Swap the two to corrupt the repository + mv -v "$other_path" "$empty_path" && + test_must_fail git fsck 2>err.fsck && + grep "hash mismatch" err.fsck && + + # confirm that cat-file is reading the new swapped-in + # blob... + cat >out.expect <<-EOF && + blob + EOF + git cat-file -t "$EMPTY_BLOB" >out.actual 2>err.actual && + test_must_be_empty err.actual && + test_cmp out.expect out.actual && + + # ... since it has a different size now. + cat >out.expect <<-EOF && + 6 + EOF + git cat-file -s "$EMPTY_BLOB" >out.actual 2>err.actual && + test_must_be_empty err.actual && + test_cmp out.expect out.actual && + + # So far "cat-file" has been happy to spew the found + # content out as-is. Try to make it zlib-invalid. + mv -v other.blob "$empty_path" && + test_must_fail git fsck 2>err.fsck && + grep "^error: inflate: data stream error (" err.fsck + ) +' + # Tests for git cat-file --follow-symlinks test_expect_success 'prep for symlink tests' ' echo_without_newline "$hello_content" >morx && From patchwork Thu May 20 11:23:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270059 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BC77C433ED for ; Thu, 20 May 2021 12:26:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4BFD1611ED for ; Thu, 20 May 2021 12:26:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237250AbhETM1Z (ORCPT ); Thu, 20 May 2021 08:27:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54710 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233706AbhETM0v (ORCPT ); Thu, 20 May 2021 08:26:51 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 56EA0C0438DD for ; Thu, 20 May 2021 04:23:22 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id z12so23222103ejw.0 for ; Thu, 20 May 2021 04:23:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eP4AEHzVzEbZOTA44KMAoTGCAlADhb68JBfO2bUIPSI=; b=bNq+fmOm+Nr+/NqLGGxpH8blE4cDktyItzD5cFIpWU0buEQJEe//RGNCpQxawX52fZ 5AkgS/+ewFY4eF5I/wcD0FQPgTQsFZwkuFt6WKDjRObuPK39wreBV3wEPVKgLG0CzWY4 I8F5Shp1jXrTnSjaSXU0yVOSRWiBnfZvCG9YSSKYKgsdV2nujmt2DvOr2Y5yw1L52iHu wwUB//ZCubJOLNp2+NwiUTnrdfWLxzY5z/KYClWs4DuXuOPYaYgmGQ0pWDPKICb5AR+Z OKNC9biQyLRqRJad9fwcgwGESwlXaeZH7R+QNNtuygWaFzyeedh8TcwFn8jNus1sg8aZ qX3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eP4AEHzVzEbZOTA44KMAoTGCAlADhb68JBfO2bUIPSI=; b=CGTyA0SZykcHvuXx3FEXDWEYPJpbTeQt/frtKxTThMIhoru7GMT1ddm3FtujnTNkXR FsqvvGggqPfnc5RAEo7yDoQ4HRfrr0IACEdE53nTc9qwz01PVLU9QBy8/IS+Rm2Cu+th 2pnc5KgIHptBcgJXubg67lKkHdoflY8KkF9bnujQ9ar2ItuuOqHs1j6nVa57PlrJv/+N 1Fk2CFn+mnes2kANm6o/zs1H98kCtYvGP+cfn4lPbXTubFJu1OUMOEEwX4sZoYfgcaKX ppQIQ1JD4TE2uUB5CVrR0VwhAEY7b5iJHm15Yz0pIXj1EoQLQV9WcFxHgYHm0x2fszdD OoHA== X-Gm-Message-State: AOAM5314b9aSirRasc3hkjexfTic9kmG3+qowEGy75dz19mKgSTMtJdR v6RYMgxvXNN3KoleJ/+Uwsl6tTag/7Kutw== X-Google-Smtp-Source: ABdhPJzQli/EDIUymKwISiFUNWOq68UukdW5m2htHSDuy2Y7WuM+vxU+uy68oVnX3l4d77yhIarW5A== X-Received: by 2002:a17:906:f256:: with SMTP id gy22mr4090750ejb.537.1621509800654; Thu, 20 May 2021 04:23:20 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:19 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 07/17] cat-file tests: test for current --allow-unknown-type behavior Date: Thu, 20 May 2021 13:23:01 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add more tests for the current --allow-unknown-type behavior. As noted in [1] I don't think much of this makes sense, but let's test for it as-is so we can see if the behavior changes in the future. 1. https://lore.kernel.org/git/87r1i4qf4h.fsf@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1006-cat-file.sh | 61 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 4a76ff024d..d3d3fd733a 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -361,6 +361,46 @@ test_expect_success 'die on broken object under -t and -s without --allow-unknow test_must_be_empty out.actual ' +test_expect_success '-e is OK with a broken object without --allow-unknown-type' ' + git cat-file -e $bogus_sha1 +' + +test_expect_success '-e can not be combined with --allow-unknown-type' ' + test_expect_code 128 git cat-file -e --allow-unknown-type $bogus_sha1 +' + +test_expect_success '-p cannot print a broken object even with --allow-unknown-type' ' + test_must_fail git cat-file -p $bogus_sha1 && + test_expect_code 128 git cat-file -p --allow-unknown-type $bogus_sha1 +' + +test_expect_success ' does not work with objects of broken types' ' + cat >err.expect <<-\EOF && + fatal: invalid object type "bogus" + EOF + test_must_fail git cat-file $bogus_type $bogus_sha1 2>err.actual && + test_cmp err.expect err.actual +' + +test_expect_success 'broken types combined with --batch and --batch-check' ' + echo $bogus_sha1 >bogus-oid && + + cat >err.expect <<-\EOF && + fatal: invalid object type + EOF + + test_must_fail git cat-file --batch err.actual && + test_cmp err.expect err.actual && + + test_must_fail git cat-file --batch-check err.actual && + test_cmp err.expect err.actual +' + +test_expect_success 'the --batch and --batch-check options do not combine with --allow-unknown-type' ' + test_expect_code 128 git cat-file --batch --allow-unknown-type expect && git cat-file -t --allow-unknown-type $bogus_sha1 >actual && @@ -372,6 +412,27 @@ test_expect_success "Size of broken object is correct" ' git cat-file -s --allow-unknown-type $bogus_sha1 >actual && test_cmp expect actual ' + +test_expect_success 'the --allow-unknown-type option does not consider replacement refs' ' + cat >expect <<-EOF && + $bogus_type + EOF + git cat-file -t --allow-unknown-type $bogus_sha1 >actual && + test_cmp expect actual && + + # Create it manually, as "git replace" will die on bogus + # types. + head=$(git rev-parse --verify HEAD) && + mkdir -p .git/refs/replace && + echo $head >.git/refs/replace/$bogus_sha1 && + + cat >expect <<-EOF && + commit + EOF + git cat-file -t --allow-unknown-type $bogus_sha1 >actual && + test_cmp expect actual +' + bogus_type="abcdefghijklmnopqrstuvwxyz1234679" bogus_content="bogus" bogus_size=$(strlen "$bogus_content") From patchwork Thu May 20 11:23:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCFA0C433B4 for ; Thu, 20 May 2021 12:26:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C37DD61261 for ; Thu, 20 May 2021 12:26:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238372AbhETM1c (ORCPT ); Thu, 20 May 2021 08:27:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54468 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233732AbhETM0w (ORCPT ); Thu, 20 May 2021 08:26:52 -0400 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DF54C0AF7B9 for ; Thu, 20 May 2021 04:23:23 -0700 (PDT) Received: by mail-ej1-x632.google.com with SMTP id et19so17610360ejc.4 for ; Thu, 20 May 2021 04:23:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2hx9NPV3qPtLVtey0KbAPnexlattqMqA/wEJ0v5UGEs=; b=jYy9DnHODH+4SQf+ePzMYaAWvTJ9wK6HS31ZudwIz2lLzF3qODEYySg4kltKiBH9RN 2Wjj3q3xsBHEiQvVRtxeN0s/iDbQF1vyugEl55AwFBMHQ/Rq89Gri2vOI0Jy4GEvJMqg PaJ3oEhrJFYyh8I85kHH953K3SLh9Dx27x5gYYJuyOvn39Ute/HOdd0E79iwcXAH0Jex CUTtWxpGOi8RYFkocmCl+U8vgrbmq7RPhzDpLmdeOC5lOXKuyvAGhXpi2J15cZ/wWrlC QWNJTUWKNhgSNXi4r3GUHNxUK6kDg6sLvY1psiJxw6yhd8sNT+na97L32jv/nPnGzbur d91Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2hx9NPV3qPtLVtey0KbAPnexlattqMqA/wEJ0v5UGEs=; b=DF+TVdOvNVIHqko3W1tfRPWYEsGwQlCPgLlX3936a512ECuG49WDNZ3BO3iQlC2R2c oe7wssq3veF+Jgnrsc7rs8JNmUjYJt9sVp720diwyjxspiWj0ogfMwAw/L3Ydag7zlTf E+C3tmSj1krDI6KtYEX6mdLRrAbI8HM/5YJvyh9K1d/firDMpJxeEA+8Hb+L2/lsX6Io qP8SZ23Ctd/GjEaFKOsDo9qNPpPGrUH0jXiypRjS0o0gHzUVBfxus7PCzS1jGsh3J0Zh G2lDZ+1QEsWLiH0dvWet2pskDPMdCos6dsxkK2pnvGDyQpBmWNyxnG6L/Ht9ghDY44Je +aPA== X-Gm-Message-State: AOAM5331B82MCe5GbZXcAbywRcHR0XgnCZ1a+L7fgxB3rzCeVl+ujMGy M5mlv0it4wggOXtaJ1M7qXf88BtOjYyVOg== X-Google-Smtp-Source: ABdhPJwCCm+K33yfujZ9ENFu2pY3dLisUufMolocJYKW00m6JwPhB1ROJJluJOR2jTa3V3kyqjU9+Q== X-Received: by 2002:a17:906:5d0c:: with SMTP id g12mr4125329ejt.447.1621509801435; Thu, 20 May 2021 04:23:21 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:20 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 08/17] cache.h: move object functions to object-store.h Date: Thu, 20 May 2021 13:23:02 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move the declaration of some ancient object functions added in e.g. c4483576b8d (Add "unpack_sha1_header()" helper function, 2005-06-01) from cache.h to object-store.h. This continues work started in cbd53a2193d (object-store: move object access functions to object-store.h, 2018-05-15). Signed-off-by: Ævar Arnfjörð Bjarmason --- cache.h | 10 ---------- object-store.h | 9 +++++++++ 2 files changed, 9 insertions(+), 10 deletions(-) diff --git a/cache.h b/cache.h index ba04ff8bd3..32ea1ea047 100644 --- a/cache.h +++ b/cache.h @@ -1302,16 +1302,6 @@ char *xdg_cache_home(const char *filename); int git_open_cloexec(const char *name, int flags); #define git_open(name) git_open_cloexec(name, O_RDONLY) -int unpack_loose_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz); -int parse_loose_header(const char *hdr, unsigned long *sizep); - -int check_object_signature(struct repository *r, const struct object_id *oid, - void *buf, unsigned long size, const char *type); - -int finalize_object_file(const char *tmpfile, const char *filename); - -/* Helper to check and "touch" a file */ -int check_and_freshen_file(const char *fn, int freshen); extern const signed char hexval_table[256]; static inline unsigned int hexval(unsigned char c) diff --git a/object-store.h b/object-store.h index ec32c23dcb..9117115a50 100644 --- a/object-store.h +++ b/object-store.h @@ -477,4 +477,13 @@ int for_each_object_in_pack(struct packed_git *p, int for_each_packed_object(each_packed_object_fn, void *, enum for_each_object_flags flags); +int unpack_loose_header(git_zstream *stream, unsigned char *map, + unsigned long mapsize, void *buffer, + unsigned long bufsiz); +int parse_loose_header(const char *hdr, unsigned long *sizep); +int check_object_signature(struct repository *r, const struct object_id *oid, + void *buf, unsigned long size, const char *type); +int finalize_object_file(const char *tmpfile, const char *filename); +int check_and_freshen_file(const char *fn, int freshen); + #endif /* OBJECT_STORE_H */ From patchwork Thu May 20 11:23:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270061 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3579CC43460 for ; Thu, 20 May 2021 12:26:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1695E611ED for ; Thu, 20 May 2021 12:26:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237359AbhETM11 (ORCPT ); Thu, 20 May 2021 08:27:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233096AbhETM0v (ORCPT ); Thu, 20 May 2021 08:26:51 -0400 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05724C0AF7BA for ; Thu, 20 May 2021 04:23:24 -0700 (PDT) Received: by mail-ed1-x535.google.com with SMTP id t15so18932390edr.11 for ; Thu, 20 May 2021 04:23:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=s+gFROMqipHl9DFeImSi02N9eb7dAGLr1iIMb1t87Hs=; b=GL9IchQxcQlj3IhAJKi5yIc2581meSnFf4B5JkU/577C83wiQP3XcoHzx+5Uxb+O2m VBI6dWSsbVYcvXYo3vvPkGmAxe1jdBGAXxIXaUwfxrg3I3XI6vp1fIfPcfQjkGop6Mwd E46QYnw1liHtTG6Cyo6wYN7ayWMQyqFcFwPDbaARg9dn7114bLiV7QydhF9VGodmd8Ig lm2rLvFGUribFkHy9UrK8UKuxpgy6AGkM48Z6s+YiU34raplWI3LTLCoL/8kDhLQfKrY x6/Ca7+gS3D/R0M8JQb6gh1sWykIqI47JVIvToSb8Tc/DrUqLmKKGT4nQ4Mn/lvuL/Ni aZSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=s+gFROMqipHl9DFeImSi02N9eb7dAGLr1iIMb1t87Hs=; b=XpXb5a6gjoHpp2ns1k3h+guT+goWlfQLB1ItwjmQ2TY6aoFFL7W2BPSpJfF5luqHKS jzmtvQj/zTRQbKzAilhROUkCAH2Ljh+kkJoimusTaxdotLpwNoSfsAtQefby0pAVLjEm CrOvk3iTYY2IKbXjgBMsiqCdbtcJs3fdW6/kADf63i5jwaJC7JcPKtI+b4CiWBllcN19 M0DK/r4hU0Fa4Dk6+WRPte9OEmX0ZdiW5Thz8pokz0D4ph3Q+NuUzJIXPNCdAQq72s0n h3hcjZSbKMm1+x+tUtlrtKTvMp19gqe5f9TEALF223z5vXbCs5ItAt12pGPsCHpT/J8J OPTA== X-Gm-Message-State: AOAM530oFJJtZjo8tJuAgWSwdPtrHcW8dvcemmPTDWX+8lly3uyYZwOG Qbbx+p50MvyfEjAjQ+6HDeXrs8OLWQnJgg== X-Google-Smtp-Source: ABdhPJz0zdjEaN/zyJn1P1yQXc5OvC6gahPM9k4MtnPq/QmsSsD+G5KuRfY3tPBvqUmdnzNf7Sb/gw== X-Received: by 2002:a05:6402:2317:: with SMTP id l23mr4478642eda.265.1621509802294; Thu, 20 May 2021 04:23:22 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:21 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 09/17] object-file.c: make parse_loose_header_extended() public Date: Thu, 20 May 2021 13:23:03 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Make the parse_loose_header_extended() function public and remove the parse_loose_header() wrapper. The only direct user of it outside of object-file.c itself was in streaming.c, that caller can simply pass the required "struct object-info *" instead. This change is being done in preparation for teaching read_loose_object() to accept a flag to pass to parse_loose_header(). It isn't strictly necessary for that change, we could simply use parse_loose_header_extended() there, but will leave the API in a better end state. Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 21 ++++++++------------- object-store.h | 3 ++- streaming.c | 5 ++++- 3 files changed, 14 insertions(+), 15 deletions(-) diff --git a/object-file.c b/object-file.c index f233b440b2..527f435381 100644 --- a/object-file.c +++ b/object-file.c @@ -1340,8 +1340,9 @@ static void *unpack_loose_rest(git_zstream *stream, * too permissive for what we want to check. So do an anal * object header parse by hand. */ -static int parse_loose_header_extended(const char *hdr, struct object_info *oi, - unsigned int flags) +int parse_loose_header(const char *hdr, + struct object_info *oi, + unsigned int flags) { const char *type_buf = hdr; unsigned long size; @@ -1401,14 +1402,6 @@ static int parse_loose_header_extended(const char *hdr, struct object_info *oi, return *hdr ? -1 : type; } -int parse_loose_header(const char *hdr, unsigned long *sizep) -{ - struct object_info oi = OBJECT_INFO_INIT; - - oi.sizep = sizep; - return parse_loose_header_extended(hdr, &oi, 0); -} - static int loose_object_info(struct repository *r, const struct object_id *oid, struct object_info *oi, int flags) @@ -1463,10 +1456,10 @@ static int loose_object_info(struct repository *r, if (status < 0) ; /* Do nothing */ else if (hdrbuf.len) { - if ((status = parse_loose_header_extended(hdrbuf.buf, oi, flags)) < 0) + if ((status = parse_loose_header(hdrbuf.buf, oi, flags)) < 0) status = error(_("unable to parse %s header with --allow-unknown-type"), oid_to_hex(oid)); - } else if ((status = parse_loose_header_extended(hdr, oi, flags)) < 0) + } else if ((status = parse_loose_header(hdr, oi, flags)) < 0) status = error(_("unable to parse %s header"), oid_to_hex(oid)); if (status >= 0 && oi->contentp) { @@ -2549,6 +2542,8 @@ int read_loose_object(const char *path, unsigned long mapsize; git_zstream stream; char hdr[MAX_HEADER_LEN]; + struct object_info oi = OBJECT_INFO_INIT; + oi.sizep = size; *contents = NULL; @@ -2563,7 +2558,7 @@ int read_loose_object(const char *path, goto out; } - *type = parse_loose_header(hdr, size); + *type = parse_loose_header(hdr, &oi, 0); if (*type < 0) { error(_("unable to parse header of %s"), path); git_inflate_end(&stream); diff --git a/object-store.h b/object-store.h index 9117115a50..d443964447 100644 --- a/object-store.h +++ b/object-store.h @@ -480,7 +480,8 @@ int for_each_packed_object(each_packed_object_fn, void *, int unpack_loose_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz); -int parse_loose_header(const char *hdr, unsigned long *sizep); +int parse_loose_header(const char *hdr, struct object_info *oi, + unsigned int flags); int check_object_signature(struct repository *r, const struct object_id *oid, void *buf, unsigned long size, const char *type); int finalize_object_file(const char *tmpfile, const char *filename); diff --git a/streaming.c b/streaming.c index 5f480ad50c..8beac62cbb 100644 --- a/streaming.c +++ b/streaming.c @@ -223,6 +223,9 @@ static int open_istream_loose(struct git_istream *st, struct repository *r, const struct object_id *oid, enum object_type *type) { + struct object_info oi = OBJECT_INFO_INIT; + oi.sizep = &st->size; + st->u.loose.mapped = map_loose_object(r, oid, &st->u.loose.mapsize); if (!st->u.loose.mapped) return -1; @@ -231,7 +234,7 @@ static int open_istream_loose(struct git_istream *st, struct repository *r, st->u.loose.mapsize, st->u.loose.hdr, sizeof(st->u.loose.hdr)) < 0) || - (parse_loose_header(st->u.loose.hdr, &st->size) < 0)) { + (parse_loose_header(st->u.loose.hdr, &oi, 0) < 0)) { git_inflate_end(&st->z); munmap(st->u.loose.mapped, st->u.loose.mapsize); return -1; From patchwork Thu May 20 11:23:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B31A4C433B4 for ; Thu, 20 May 2021 12:26:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9910A611ED for ; Thu, 20 May 2021 12:26:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237828AbhETM13 (ORCPT ); Thu, 20 May 2021 08:27:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233755AbhETM0w (ORCPT ); Thu, 20 May 2021 08:26:52 -0400 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 08D8CC0AF7BB for ; Thu, 20 May 2021 04:23:25 -0700 (PDT) Received: by mail-ej1-x62d.google.com with SMTP id k14so21205006eji.2 for ; Thu, 20 May 2021 04:23:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=t9/RvS8OpT98zShrRzRwb/Z5xFfD/l/sJ7ZJQVKBfKc=; b=mVbTZNkZEQVUNVWIxpAuCQ4if1S5H2SYxa2kX0qiykWpEpGrz4r6gGhG5FrjvtdBGS 9SdWK0F6PPMIQCVZ52/qvDmuQmDpdgjhNeff1148cTh3GWbv5m1r6dCFKR3cZgvF9UIs opoueuWy/1ACHlKzt9Rp0PYFMOhOHL7zXYW6d6gjDu/Kuf2OQU/enkGwN0lQYZLBmISg 7u4gjlFTfX0wZatfihs3vVwPL6dl6wSLjIUvlec1m1gYkFDdzrx6SdRLPSJOJCjGRWvN RKQ3bKjQ3nMhDsqTHaspCU2voppjWX5F8gzGCfMQ6HgM355tnm/0gX00tSKTFtidAP6Z +69g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t9/RvS8OpT98zShrRzRwb/Z5xFfD/l/sJ7ZJQVKBfKc=; b=Cgf1GC2lhPaZgt1u0lG5G+pdu9EsFj+n/0fucU6mx1qO5/I9hBZl5cUhpjsYwte87n c9DJIKeqw4YZ7ENevBpGtpuCAvqCJNV9YT/qDnzKEmsVqOxbwj+AcjOK7+I76WRoUgWl /en347Gxx94izDLtwCVeeZjNVnRqXIwB3l5MyGsfiJnaeum65KY1rWtpMZu3MUrZTEsM 214HDPfkSfc9U/JV8wmH8fy7TRGHbLkNf6GFJAjYEtU3TpNDngPd9jeB1CT5ZnN4PCvd ez1HQHwr7CoESw1YctSFY/Q4FyetfII31AUUd4B0xKW0CJWOZVeBEN9RVw7YiGNaIIMI qcDQ== X-Gm-Message-State: AOAM530pxFNUyG73kKxbvTn3b7ByV+hGQfp3qvdUT5IU513uDfmQaNRS jws/y3PtwbJsxKnQi7MJxzQJvxG4hBTw4Q== X-Google-Smtp-Source: ABdhPJx0HYtp31wNJzaHfunzVg2WTXyVgpJn+yJXPlaC/Vvz+ucPH/5/TtwbvNwGsmJlmDHEasTLhA== X-Received: by 2002:a17:906:d14b:: with SMTP id br11mr4241973ejb.490.1621509803319; Thu, 20 May 2021 04:23:23 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:22 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 10/17] object-file.c: add missing braces to loose_object_info() Date: Thu, 20 May 2021 13:23:04 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the formatting in loose_object_info() to conform with our usual coding style: When there are multiple arms to a conditional and some of them require braces, enclose even a single line block in braces for consistency -- Documentation/CodingGuidelines This formatting-only change makes a subsequent commit easier to read. Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/object-file.c b/object-file.c index 527f435381..115054389c 100644 --- a/object-file.c +++ b/object-file.c @@ -1450,17 +1450,20 @@ static int loose_object_info(struct repository *r, if (unpack_loose_header_to_strbuf(&stream, map, mapsize, hdr, sizeof(hdr), &hdrbuf) < 0) status = error(_("unable to unpack %s header with --allow-unknown-type"), oid_to_hex(oid)); - } else if (unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0) + } else if (unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0) { status = error(_("unable to unpack %s header"), oid_to_hex(oid)); - if (status < 0) - ; /* Do nothing */ - else if (hdrbuf.len) { + } + + if (status < 0) { + /* Do nothing */ + } else if (hdrbuf.len) { if ((status = parse_loose_header(hdrbuf.buf, oi, flags)) < 0) status = error(_("unable to parse %s header with --allow-unknown-type"), oid_to_hex(oid)); - } else if ((status = parse_loose_header(hdr, oi, flags)) < 0) + } else if ((status = parse_loose_header(hdr, oi, flags)) < 0) { status = error(_("unable to parse %s header"), oid_to_hex(oid)); + } if (status >= 0 && oi->contentp) { *oi->contentp = unpack_loose_rest(&stream, hdr, @@ -1469,8 +1472,9 @@ static int loose_object_info(struct repository *r, git_inflate_end(&stream); status = -1; } - } else + } else { git_inflate_end(&stream); + } munmap(map, mapsize); if (status && oi->typep) From patchwork Thu May 20 11:23:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A70A5C433ED for ; Thu, 20 May 2021 12:26:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8AD76611ED for ; Thu, 20 May 2021 12:26:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237441AbhETM12 (ORCPT ); Thu, 20 May 2021 08:27:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54560 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233046AbhETM0w (ORCPT ); Thu, 20 May 2021 08:26:52 -0400 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 343B3C0AF7BC for ; Thu, 20 May 2021 04:23:26 -0700 (PDT) Received: by mail-ed1-x531.google.com with SMTP id w12so11189036edx.1 for ; Thu, 20 May 2021 04:23:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tlbvbX2iLXerpoyY+8TzkQgj47ePtOp5uXVOwnSIBXA=; b=tlGXSCJ2UU9OkO+EbSSrIRT8v93CeN5YYBn9HKK0LOUVDBC/o9SyOxou9uhjETFttm dqYGlgH1tOj1IHzRWmv0puZ7snitGPQyEruO1VPNxJzS+9VLRggGD4bJGQOwTn9GZWLa Vk1RSiWQgp7fsRCEQLJSG1ytKTpAJrugN0v1NV+6MnpeHvdoDeqX8usF0gbcl7PicEwE tbCntoZYSBdNsntczcLCGXrB+ylwex5NYWNp4a0FJElNZzHLAHsH1HXKpjENkmoVIrk3 tq0PlEDKhL+jVNe1pc0GjXLswRYf2S9zqmTiy4EOcrQqQ+Pu/uJsefi4Lo3mJ5Zi5mnM f5Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tlbvbX2iLXerpoyY+8TzkQgj47ePtOp5uXVOwnSIBXA=; b=nrHQ/ZRzvPTw2hrOfJbcHRMmLNu4r4CULnG13qvdJCYljiNy2lUTXag9dmKjGEU2Ve eve0t+xN9CDyVMYM0NVxG1d3gc2MEQJ/BNHFHfz0XiOatsvJbq4EyyzoL4zKgB+FQb1O 04VjHyNyZDsHUPyts0vZhHIVD7b9oDkVxZHeGYVAdkHSCi16Ns8dYSxPrg9NbLPkPzhI dH8bRMlaUSshtgwUXMohdx4ryeKaPFT1tbFvyksHncL9H5HqlR6YUeHD+wd3+4rz3ViY Dm2+cFog7YEDQIbP2X+pxYeo+a5vkdP01MwZ04waA3usWS6ef+VjSYw4xudBDVlYG+fb 9Hnw== X-Gm-Message-State: AOAM530C2hP6b3EY8sgIgft12zWpBYPZNup0BCbj3NMXSo9wHHWV0zb2 bizHB2E013kC2Vcv/p+kGmw9EUhqx184Dw== X-Google-Smtp-Source: ABdhPJw8Yxuez4qoWv5n5oeo86dZs1Y++fX9KQu7p2roTxctH05+wUFqmyFfWrVfHUokarbyYF3l3g== X-Received: by 2002:aa7:d844:: with SMTP id f4mr4315417eds.203.1621509804105; Thu, 20 May 2021 04:23:24 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:23 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 11/17] object-file.c: stop dying in parse_loose_header() Date: Thu, 20 May 2021 13:23:05 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Start the libification of parse_loose_header() by making it return error codes and data instead of invoking die() by itself. For now we'll move the relevant die() call to loose_object_info() and read_loose_object() to keep this change smaller, but in subsequent commits we'll also libify those. The reason this makes sense is that with the refactoring of parse_loose_header_extended() in an earlier commit the public interface for parse_loose_header() no longer just accepts a "unsigned long *sizep". Rather it accepts a "struct object_info *", that structure will be populated with information about the object. It thus makes sense to further libify the interface so that it stops calling die() when it encounters OBJ_BAD, and instead rely on its callers to check the populated "oi->typep". This also allows us to simplify away the unpack_loose_header_to_strbuf() function added in 46f034483eb (sha1_file: support reading from a loose object of unknown type, 2015-05-03). Its code was mostly copy/pasted between it and both of unpack_loose_header() and unpack_loose_short_header(). We now have a single unpack_loose_header() function which accepts an optional "struct strbuf *" instead. I think the remaining unpack_loose_header() function could be further simplified, we're carrying some complexity just to be able to emit a garbage type longer than MAX_HEADER_LEN, we could alternatively just say "we found a garbage type ..." instead, but let's leave this in place for now. Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 105 ++++++++++++++++++++----------------------------- object-store.h | 25 ++++++++++-- streaming.c | 7 +++- 3 files changed, 70 insertions(+), 67 deletions(-) diff --git a/object-file.c b/object-file.c index 115054389c..d4bdf86657 100644 --- a/object-file.c +++ b/object-file.c @@ -1210,11 +1210,12 @@ void *map_loose_object(struct repository *r, return map_loose_object_1(r, NULL, oid, size); } -static int unpack_loose_short_header(git_zstream *stream, - unsigned char *map, unsigned long mapsize, - void *buffer, unsigned long bufsiz) +int unpack_loose_header(git_zstream *stream, + unsigned char *map, unsigned long mapsize, + void *buffer, unsigned long bufsiz, + struct strbuf *header) { - int ret; + int status; /* Get the data stream */ memset(stream, 0, sizeof(*stream)); @@ -1225,44 +1226,25 @@ static int unpack_loose_short_header(git_zstream *stream, git_inflate_init(stream); obj_read_unlock(); - ret = git_inflate(stream, 0); + status = git_inflate(stream, 0); obj_read_lock(); - - return ret; -} - -int unpack_loose_header(git_zstream *stream, - unsigned char *map, unsigned long mapsize, - void *buffer, unsigned long bufsiz) -{ - int status = unpack_loose_short_header(stream, map, mapsize, - buffer, bufsiz); - if (status < Z_OK) return status; - /* Make sure we have the terminating NUL */ - if (!memchr(buffer, '\0', stream->next_out - (unsigned char *)buffer)) - return -1; - return 0; -} - -static int unpack_loose_header_to_strbuf(git_zstream *stream, unsigned char *map, - unsigned long mapsize, void *buffer, - unsigned long bufsiz, struct strbuf *header) -{ - int status; - - status = unpack_loose_short_header(stream, map, mapsize, buffer, bufsiz); - if (status < Z_OK) - return -1; - /* * Check if entire header is unpacked in the first iteration. */ if (memchr(buffer, '\0', stream->next_out - (unsigned char *)buffer)) return 0; + /* + * We have a header longer than MAX_HEADER_LEN. We abort early + * unless under we're running as e.g. "cat-file + * --allow-unknown-type". + */ + if (!header) + return -1; + /* * buffer[0..bufsiz] was not large enough. Copy the partial * result out to header, and then append the result of further @@ -1340,9 +1322,7 @@ static void *unpack_loose_rest(git_zstream *stream, * too permissive for what we want to check. So do an anal * object header parse by hand. */ -int parse_loose_header(const char *hdr, - struct object_info *oi, - unsigned int flags) +int parse_loose_header(const char *hdr, struct object_info *oi) { const char *type_buf = hdr; unsigned long size; @@ -1364,15 +1344,6 @@ int parse_loose_header(const char *hdr, type = type_from_string_gently(type_buf, type_len, 1); if (oi->type_name) strbuf_add(oi->type_name, type_buf, type_len); - /* - * Set type to 0 if its an unknown object and - * we're obtaining the type using '--allow-unknown-type' - * option. - */ - if ((flags & OBJECT_INFO_ALLOW_UNKNOWN_TYPE) && (type < 0)) - type = 0; - else if (type < 0) - die(_("invalid object type")); if (oi->typep) *oi->typep = type; @@ -1399,7 +1370,14 @@ int parse_loose_header(const char *hdr, /* * The length must be followed by a zero byte */ - return *hdr ? -1 : type; + if (*hdr) + return -1; + + /* + * The format is valid, but the type may still be bogus. The + * Caller needs to check its oi->typep. + */ + return 0; } static int loose_object_info(struct repository *r, @@ -1410,9 +1388,12 @@ static int loose_object_info(struct repository *r, unsigned long mapsize; void *map; git_zstream stream; + int hdr_ret; char hdr[MAX_HEADER_LEN]; struct strbuf hdrbuf = STRBUF_INIT; unsigned long size_scratch; + enum object_type type_scratch; + int allow_unknown = flags & OBJECT_INFO_ALLOW_UNKNOWN_TYPE; if (oi->delta_base_oid) oidclr(oi->delta_base_oid); @@ -1443,27 +1424,23 @@ static int loose_object_info(struct repository *r, if (!oi->sizep) oi->sizep = &size_scratch; + if (!oi->typep) + oi->typep = &type_scratch; if (oi->disk_sizep) *oi->disk_sizep = mapsize; - if ((flags & OBJECT_INFO_ALLOW_UNKNOWN_TYPE)) { - if (unpack_loose_header_to_strbuf(&stream, map, mapsize, hdr, sizeof(hdr), &hdrbuf) < 0) - status = error(_("unable to unpack %s header with --allow-unknown-type"), - oid_to_hex(oid)); - } else if (unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0) { + + hdr_ret = unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr), + allow_unknown ? &hdrbuf : NULL); + if (hdr_ret < 0) { status = error(_("unable to unpack %s header"), oid_to_hex(oid)); } - - if (status < 0) { - /* Do nothing */ - } else if (hdrbuf.len) { - if ((status = parse_loose_header(hdrbuf.buf, oi, flags)) < 0) - status = error(_("unable to parse %s header with --allow-unknown-type"), - oid_to_hex(oid)); - } else if ((status = parse_loose_header(hdr, oi, flags)) < 0) { + if (!status && parse_loose_header(hdrbuf.len ? hdrbuf.buf : hdr, oi) < 0) { status = error(_("unable to parse %s header"), oid_to_hex(oid)); } + if (!allow_unknown && *oi->typep < 0) + die(_("invalid object type")); if (status >= 0 && oi->contentp) { *oi->contentp = unpack_loose_rest(&stream, hdr, @@ -1481,7 +1458,8 @@ static int loose_object_info(struct repository *r, *oi->typep = status; if (oi->sizep == &size_scratch) oi->sizep = NULL; - strbuf_release(&hdrbuf); + if (oi->typep == &type_scratch) + oi->typep = NULL; oi->whence = OI_LOOSE; return (status < 0) ? status : 0; } @@ -2547,6 +2525,7 @@ int read_loose_object(const char *path, git_zstream stream; char hdr[MAX_HEADER_LEN]; struct object_info oi = OBJECT_INFO_INIT; + oi.typep = type; oi.sizep = size; *contents = NULL; @@ -2557,17 +2536,19 @@ int read_loose_object(const char *path, goto out; } - if (unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0) { + if (unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr), + NULL) < 0) { error(_("unable to unpack header of %s"), path); goto out; } - *type = parse_loose_header(hdr, &oi, 0); - if (*type < 0) { + if (parse_loose_header(hdr, &oi) < 0) { error(_("unable to parse header of %s"), path); git_inflate_end(&stream); goto out; } + if (*type < 0) + die(_("invalid object type")); if (*type == OBJ_BLOB && *size > big_file_threshold) { if (check_stream_oid(&stream, hdr, *size, path, expected_oid) < 0) diff --git a/object-store.h b/object-store.h index d443964447..740edcac30 100644 --- a/object-store.h +++ b/object-store.h @@ -477,11 +477,30 @@ int for_each_object_in_pack(struct packed_git *p, int for_each_packed_object(each_packed_object_fn, void *, enum for_each_object_flags flags); +/** + * unpack_loose_header() initializes the data stream needed to unpack + * a loose object header. + * + * Returns 0 on success. Returns negative values on error. + * + * It will only parse up to MAX_HEADER_LEN bytes unless an optional + * "hdrbuf" argument is non-NULL. This is intended for use with + * OBJECT_INFO_ALLOW_UNKNOWN_TYPE to extract the bad type for (error) + * reporting. The full header will be extracted to "hdrbuf" for use + * with parse_loose_header(). + */ int unpack_loose_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, - unsigned long bufsiz); -int parse_loose_header(const char *hdr, struct object_info *oi, - unsigned int flags); + unsigned long bufsiz, struct strbuf *hdrbuf); + +/** + * parse_loose_header() parses the starting " \0" of an + * object. If it doesn't follow that format -1 is returned. To check + * the validity of the populate the "typep" in the "struct + * object_info". It will be OBJ_BAD if the object type is unknown. + */ +int parse_loose_header(const char *hdr, struct object_info *oi); + int check_object_signature(struct repository *r, const struct object_id *oid, void *buf, unsigned long size, const char *type); int finalize_object_file(const char *tmpfile, const char *filename); diff --git a/streaming.c b/streaming.c index 8beac62cbb..c3dc241d6a 100644 --- a/streaming.c +++ b/streaming.c @@ -225,6 +225,7 @@ static int open_istream_loose(struct git_istream *st, struct repository *r, { struct object_info oi = OBJECT_INFO_INIT; oi.sizep = &st->size; + oi.typep = type; st->u.loose.mapped = map_loose_object(r, oid, &st->u.loose.mapsize); if (!st->u.loose.mapped) @@ -233,8 +234,10 @@ static int open_istream_loose(struct git_istream *st, struct repository *r, st->u.loose.mapped, st->u.loose.mapsize, st->u.loose.hdr, - sizeof(st->u.loose.hdr)) < 0) || - (parse_loose_header(st->u.loose.hdr, &oi, 0) < 0)) { + sizeof(st->u.loose.hdr), + NULL) < 0) || + (parse_loose_header(st->u.loose.hdr, &oi) < 0) || + *type < 0) { git_inflate_end(&st->z); munmap(st->u.loose.mapped, st->u.loose.mapsize); return -1; From patchwork Thu May 20 11:23:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC7A0C433ED for ; Thu, 20 May 2021 12:26:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8EF91611ED for ; Thu, 20 May 2021 12:26:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233001AbhETM16 (ORCPT ); Thu, 20 May 2021 08:27:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55332 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233106AbhETM0x (ORCPT ); Thu, 20 May 2021 08:26:53 -0400 Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [IPv6:2a00:1450:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F82AC0AF7BD for ; Thu, 20 May 2021 04:23:26 -0700 (PDT) Received: by mail-ed1-x532.google.com with SMTP id w12so11189051edx.1 for ; Thu, 20 May 2021 04:23:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5s47HNmbDVPDWoUxUQJgwmK6VMW/ZEK0NUqt7d71NvQ=; b=KNCpE4KoErmoN5zMGAm7wx1fk2FRVhU+5XTRZ9I1eWsoYDyl6Nc00yoU4Lg2689tha gozo9tCxYP5jM8D7Hfiih858u/YyEOj66Uvd++A+LhmVM0zgjJcUeSZ31mDZ1oaCF/QL Wk915Fa85lUhxkyDlxbHoMZbrpJim5zBzgu4bOpy2yRvl8XV2ZeA+dhjEuFDFU7fCt/e Fp3FUYyGGfdW3xXvfRuapF4YmGx4yb/V4y5cabdUvXs82xkYvOGHwb4IB49BFGFe63v1 td5U/CndxNd6KFDbirkaeeT4QhKBXwsy6yzJokQRlnur8CmEJrSe4PYcIsaHKJ9oX2Hh cnIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5s47HNmbDVPDWoUxUQJgwmK6VMW/ZEK0NUqt7d71NvQ=; b=MZMwVKes1lu4SJlY6yHCb0NoEie8ZI6naVYPPIYo2YMgO34NcwKhvfTKkrQqSC9Xzk 2JQZYh1lTzPHSxKm/Mx5BRaGKyFcLdUY07wrieYMxl/dhJNcUzAexI0ZYz/cmHaEha6W /czyVVLAu2ZRWTvabi7t3jL8YwleJN3kXHu6RtduFrO3L6nVDtsca8FrmqNzUCNV6EYr xxL8xGEapFydUTwznJZjsL7/3aBLOX4LtSbaXtC9gEmUuZjoV/MNwKAg1FcnKeTgyPim OVgMzo8hl/pF7movgG1T0ZywUbBapvwGnqSwolYWCKng++fg5QTYFB8a5gJEUnjaLykV ENjQ== X-Gm-Message-State: AOAM532Nu2l51hXDjFj7357GFhF4GdP62UeUfbpGHlt3gfSj4UQGVT6/ zdB+swpvaOcwmtELz6EJtDj9Hm4jd/yIRQ== X-Google-Smtp-Source: ABdhPJzn0uqxGy4iH64N3XC2LgdS5Dif6j4b9sdJQaEbLh8eh4UIsSvGEWB9t5zm73KOTAi7CwMO+g== X-Received: by 2002:a05:6402:42cc:: with SMTP id i12mr4579669edc.272.1621509804835; Thu, 20 May 2021 04:23:24 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:24 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 12/17] object-file.c: return -2 on "header too long" in unpack_loose_header() Date: Thu, 20 May 2021 13:23:06 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Split up the return code for "header too long" from the generic negative return value unpack_loose_header() returns, and report via error() if we exceed MAX_HEADER_LEN. As a test added earlier in this series in t1006-cat-file.sh shows we'll correctly emit zlib errors from zlib.c already in this case, so we have no need to carry those return codes further down the stack. Let's instead just return -2 saying we ran into the MAX_HEADER_LEN limit, or other negative values for "unable to unpack header". Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 15 ++++++++++----- object-store.h | 6 ++++-- t/t1006-cat-file.sh | 2 +- 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/object-file.c b/object-file.c index d4bdf86657..7623ada1aa 100644 --- a/object-file.c +++ b/object-file.c @@ -1240,10 +1240,10 @@ int unpack_loose_header(git_zstream *stream, /* * We have a header longer than MAX_HEADER_LEN. We abort early * unless under we're running as e.g. "cat-file - * --allow-unknown-type". + * --allow-unknown-type". A -2 is "header too long" */ if (!header) - return -1; + return -2; /* * buffer[0..bufsiz] was not large enough. Copy the partial @@ -1264,7 +1264,7 @@ int unpack_loose_header(git_zstream *stream, stream->next_out = buffer; stream->avail_out = bufsiz; } while (status != Z_STREAM_END); - return -1; + return -2; } static void *unpack_loose_rest(git_zstream *stream, @@ -1433,9 +1433,14 @@ static int loose_object_info(struct repository *r, hdr_ret = unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr), allow_unknown ? &hdrbuf : NULL); if (hdr_ret < 0) { - status = error(_("unable to unpack %s header"), - oid_to_hex(oid)); + if (hdr_ret == -2) + status = error(_("header for %s too long, exceeds %d bytes"), + oid_to_hex(oid), MAX_HEADER_LEN); + else + status = error(_("unable to unpack %s header"), + oid_to_hex(oid)); } + if (!status && parse_loose_header(hdrbuf.len ? hdrbuf.buf : hdr, oi) < 0) { status = error(_("unable to parse %s header"), oid_to_hex(oid)); } diff --git a/object-store.h b/object-store.h index 740edcac30..9accb614fc 100644 --- a/object-store.h +++ b/object-store.h @@ -481,13 +481,15 @@ int for_each_packed_object(each_packed_object_fn, void *, * unpack_loose_header() initializes the data stream needed to unpack * a loose object header. * - * Returns 0 on success. Returns negative values on error. + * Returns 0 on success. Returns negative values on error. If the + * header exceeds MAX_HEADER_LEN -2 will be returned. * * It will only parse up to MAX_HEADER_LEN bytes unless an optional * "hdrbuf" argument is non-NULL. This is intended for use with * OBJECT_INFO_ALLOW_UNKNOWN_TYPE to extract the bad type for (error) * reporting. The full header will be extracted to "hdrbuf" for use - * with parse_loose_header(). + * with parse_loose_header(), -2 will still be returned from this + * function to indicate that the header was too long. */ int unpack_loose_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index d3d3fd733a..f12b06150e 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -440,7 +440,7 @@ bogus_sha1=$(echo_without_newline "$bogus_content" | git hash-object -t $bogus_t test_expect_success 'die on broken object with large type under -t and -s without --allow-unknown-type' ' cat >err.expect <<-EOF && - error: unable to unpack $bogus_sha1 header + error: header for $bogus_sha1 too long, exceeds 32 bytes fatal: git cat-file: could not get object info EOF From patchwork Thu May 20 11:23:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B6A4C433ED for ; Thu, 20 May 2021 12:26:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3F8D260C41 for ; Thu, 20 May 2021 12:26:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239683AbhETM1m (ORCPT ); Thu, 20 May 2021 08:27:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233436AbhETM0x (ORCPT ); Thu, 20 May 2021 08:26:53 -0400 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 597E1C0AF7BE for ; Thu, 20 May 2021 04:23:27 -0700 (PDT) Received: by mail-ej1-x630.google.com with SMTP id u21so24605453ejo.13 for ; Thu, 20 May 2021 04:23:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QXLTONZHnk/yMVPW/t0IVE7KOVDiMC9YyS7bMLZRtcM=; b=dn3ypN1I+JjmZ8qkrpiYUmpgBV10uYZFlNYK6cWH7OWFLQBOd/vPOwY0GSEFiFgO54 U7evNfTncxJ8Nk2cGKha97XkGnpODYvB46gp6V8MHtqJoFpnbG1YrgfEDc3Fms2LRyh+ rpTVbpS26SkXEq/49OMklCTBLXswLdCEou2ruDgcDLUbvyC6IY7HFItmtI4hS7HsEEXG jYvljwV63g7pfiS6TnQqcXbIq9Tmp+qLB3yq27TdDSOupPXyOhR9sBcDfXj5jHr4WfH9 tqUbpXCviJzrLqRDTKLoTwR3xLOO4AePYdfC6+qfrRrJN80VBjFaojGVNH71IjuDJtRl GTSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QXLTONZHnk/yMVPW/t0IVE7KOVDiMC9YyS7bMLZRtcM=; b=jwqw8huxkth+QE/5usxwkUiU+G55UQquipU6CqYGU6oqvZCVUQWzRN9tgzojfL6bJ3 EvG/OHc2ulC3e455gGHt38kn3YEAXX6VMBBdLvhVnkcOr720pv0ogJ0rsVw8+w/K7Kpi seQI1M7pi12+vKJ6Ksd9xTnRfXHsFy0upMC/0o+bPsAbPUHGoynv88Ugcz846XhvvQTi W6/Vo+LoAE2wMxADzMMZz+/oWhINXGB08FZxj73N77hRfDAVYIz03DRnRZCJSTvUodzi oDIlUGFU9dJbCe5jbOc6mFsXI5dfbEufOG42w9Gi0k3x7ij/7iMB14OF8OHuMohQ3nHf X9eQ== X-Gm-Message-State: AOAM533iuENsWR7+1P6x6ENasP73cZ9GgFwtDF4eM9M5HtGrpPyZ6ngb OtH5YNFlzc9iZ/kLcwfcCM8llOEViUt+0g== X-Google-Smtp-Source: ABdhPJw7NKT0qlqio7g1HlZPLztvA3nnWsl5IwiZ6lUfEIMFYSAK8oiCFXKCmYtH8WwzffRDr0+9UQ== X-Received: by 2002:a17:907:a06e:: with SMTP id ia14mr4302707ejc.17.1621509805648; Thu, 20 May 2021 04:23:25 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:25 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 13/17] object-file.c: return -1, not "status" from unpack_loose_header() Date: Thu, 20 May 2021 13:23:07 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Return a -1 when git_inflate() fails instead of whatever Z_* status we'd get from zlib.c. This makes no difference to any error we report, but makes it more obvious that we don't care about the specific zlib error codes here. See d21f8426907 (unpack_sha1_header(): detect malformed object header, 2016-09-25) for the commit that added the "return status" code. As far as I can tell there was never a real reason (e.g. different reporting) for carrying down the "status" as opposed to "-1". At the time that d21f8426907 was written there was a corresponding "ret < Z_OK" check right after the unpack_sha1_header() call (the "unpack_sha1_header()" function was later rename to our current "unpack_loose_header()"). However, that check was removed in c84a1f3ed4d (sha1_file: refactor read_object, 2017-06-21) without changing the corresponding return code. So let's do the minor cleanup of also changing this function to return a -1. Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/object-file.c b/object-file.c index 7623ada1aa..0de699de98 100644 --- a/object-file.c +++ b/object-file.c @@ -1229,7 +1229,7 @@ int unpack_loose_header(git_zstream *stream, status = git_inflate(stream, 0); obj_read_lock(); if (status < Z_OK) - return status; + return -1; /* * Check if entire header is unpacked in the first iteration. From patchwork Thu May 20 11:23:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270069 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8757C433ED for ; Thu, 20 May 2021 12:26:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9924261261 for ; Thu, 20 May 2021 12:26:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238455AbhETM1e (ORCPT ); Thu, 20 May 2021 08:27:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233355AbhETM0x (ORCPT ); Thu, 20 May 2021 08:26:53 -0400 Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2BBCBC0438DE for ; Thu, 20 May 2021 04:23:28 -0700 (PDT) Received: by mail-ed1-x530.google.com with SMTP id s6so18884249edu.10 for ; Thu, 20 May 2021 04:23:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CuOTGCbbZ35BhMC7NR7yODtXyWRRxsJJ1v4Bf/Ynmjw=; b=tKAkqWxq81N5kJBi8N+jbTtPllNEyHY6hKkyaOTRmyOafVZL6hcyYeg0e0VgaX+phB GVPRiYjn7rQPYyceZQYmPq9yoTlKICvLq6kVsNz7O6elvJZ3dH2EjdSKmAdt72YPEBh4 PsevXHamuWx36PbcX+7FNK3Lb/EH6AgdPdIJnyLU3a9ycm+R+lSkK4fjKSRmfkLBA8m4 JN+cVyFN12pQa2vZG2Hu9a5WcoEtQZV5lnTjOYm+xUzG9JsnoOOkjYzRNzTaMgSqqMDU gt49cFArK9WlkOU+BnK2CKiZBrN0SidqRml2x+CYDt2wb1RzHFn7Jp23yGg5Te9X66dP 2XJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CuOTGCbbZ35BhMC7NR7yODtXyWRRxsJJ1v4Bf/Ynmjw=; b=l8xGODlWxzLe3XpKyvhz+8/qaca7K3pUX/s7WUSfeK4NeTCrmzt7I4/VQXyDtbdpJ+ DSVpbot2lUpOpBZkBiqQEObjz9wxObRhdo/ACx/jRnYvzmYID765G+MqklhtBZzGNbsl LfmR67BvCe53yKgnWAc1DWf1dW4PwXTG9zOCWOK31JOGkKOJsNo07ietwC70ksu6i+xb ReaJy4nRND6PdCMpNq0sZzPBlgT2lgT+aAa5iGSWQfyFeRVv2wHEqtrUgekXuIqqfiRw ace7ftbzGm85vAWBZj75K5ItQ0Xo2BirWdIlxnI5f4D14H+cu9GutCkoqSpQEmRl1Qr1 JgKg== X-Gm-Message-State: AOAM531mEKvIm+sfcCyA656M+UYGUbeGsD1iossg9W4FO2wz081QVjeG +qy/2HS/m/0UyNzJJmaR7FbqdZEs5nRU0w== X-Google-Smtp-Source: ABdhPJyuOnQmf2BLDMPnXb0iO8N2khZxt0RQwC7GgV58sPKkq+M+7ulIuNwVlis/MzC78Z8dCFawXA== X-Received: by 2002:a05:6402:3594:: with SMTP id y20mr4501327edc.63.1621509806511; Thu, 20 May 2021 04:23:26 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:25 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 14/17] fsck: don't hard die on invalid object types Date: Thu, 20 May 2021 13:23:08 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the error fsck emits on invalid object types, such as: $ git hash-object --stdin -w -t garbage --literally From the very ungraceful error of: $ git fsck fatal: invalid object type $ To: $ git fsck error: hash mismatch for (expected ) error: : object corrupt or missing: [ the rest of the fsck output here, i.e. it didn't hard die ] We'll still exit with non-zero, but now we'll finish the rest of the traversal. The tests that's being added here asserts that we'll still complain about other fsck issues (e.g. an unrelated dangling blob). To do this we need to pass down the "OBJECT_INFO_ALLOW_UNKNOWN_TYPE" flag from read_loose_object() through to parse_loose_header(). Since the read_loose_object() function is only used in builtin/fsck.c we can simply change it. See f6371f92104 (sha1_file: add read_loose_object() function, 2017-01-13) for the introduction of read_loose_object(). Why are we complaining about a "hash mismatch" for an object of a type we don't know about? We shouldn't. This is the bare minimal change needed to not make fsck hard die on a repository that's been corrupted in this manner. In subsequent commits we'll teach fsck to recognize this particular type of corruption and emit a better error message. Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/fsck.c | 3 ++- object-file.c | 11 ++++++++--- object-store.h | 3 ++- t/t1450-fsck.sh | 14 +++++++------- 4 files changed, 19 insertions(+), 12 deletions(-) diff --git a/builtin/fsck.c b/builtin/fsck.c index 87a99b0108..38b515deb6 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -600,7 +600,8 @@ static int fsck_loose(const struct object_id *oid, const char *path, void *data) void *contents; int eaten; - if (read_loose_object(path, oid, &type, &size, &contents) < 0) { + if (read_loose_object(path, oid, &type, &size, &contents, + OBJECT_INFO_ALLOW_UNKNOWN_TYPE) < 0) { errors_found |= ERROR_OBJECT; error(_("%s: object corrupt or missing: %s"), oid_to_hex(oid), path); diff --git a/object-file.c b/object-file.c index 0de699de98..0e8a024eb3 100644 --- a/object-file.c +++ b/object-file.c @@ -2522,7 +2522,8 @@ int read_loose_object(const char *path, const struct object_id *expected_oid, enum object_type *type, unsigned long *size, - void **contents) + void **contents, + unsigned int oi_flags) { int ret = -1; void *map = NULL; @@ -2530,6 +2531,7 @@ int read_loose_object(const char *path, git_zstream stream; char hdr[MAX_HEADER_LEN]; struct object_info oi = OBJECT_INFO_INIT; + int allow_unknown = oi_flags & OBJECT_INFO_ALLOW_UNKNOWN_TYPE; oi.typep = type; oi.sizep = size; @@ -2552,8 +2554,11 @@ int read_loose_object(const char *path, git_inflate_end(&stream); goto out; } - if (*type < 0) - die(_("invalid object type")); + if (!allow_unknown && *type < 0) { + error(_("header for %s declares an unknown type"), path); + git_inflate_end(&stream); + goto out; + } if (*type == OBJ_BLOB && *size > big_file_threshold) { if (check_stream_oid(&stream, hdr, *size, path, expected_oid) < 0) diff --git a/object-store.h b/object-store.h index 9accb614fc..790e8b1798 100644 --- a/object-store.h +++ b/object-store.h @@ -245,7 +245,8 @@ int read_loose_object(const char *path, const struct object_id *expected_oid, enum object_type *type, unsigned long *size, - void **contents); + void **contents, + unsigned int oi_flags); /* Retry packed storage after checking packed and loose storage */ #define HAS_OBJECT_RECHECK_PACKED 1 diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh index f36ec1e2f4..e7e8decebb 100755 --- a/t/t1450-fsck.sh +++ b/t/t1450-fsck.sh @@ -863,16 +863,16 @@ test_expect_success 'detect corrupt index file in fsck' ' test_i18ngrep "bad index file" errors ' -test_expect_success 'fsck hard errors on an invalid object type' ' +test_expect_success 'fsck error and recovery on invalid object type' ' test_create_repo garbage-type && empty_blob=$(git -C garbage-type hash-object --stdin -w -t blob err.expect <<-\EOF && - fatal: invalid object type - EOF - test_must_fail git -C garbage-type fsck >out.actual 2>err.actual && - test_cmp err.expect err.actual && - test_must_be_empty out.actual + test_must_fail git -C garbage-type fsck >out 2>err && + grep -e "^error" -e "^fatal" err >errors && + test_line_count = 2 errors && + grep "error: hash mismatch for" err && + grep "$garbage_blob: object corrupt or missing:" err && + grep "dangling blob $empty_blob" out ' test_done From patchwork Thu May 20 11:23:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEBFBC433ED for ; Thu, 20 May 2021 12:26:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C0F99611ED for ; Thu, 20 May 2021 12:26:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239331AbhETM1h (ORCPT ); Thu, 20 May 2021 08:27:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232178AbhETM0x (ORCPT ); Thu, 20 May 2021 08:26:53 -0400 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 12E6DC0438DF for ; Thu, 20 May 2021 04:23:29 -0700 (PDT) Received: by mail-ej1-x62e.google.com with SMTP id lg14so24647960ejb.9 for ; Thu, 20 May 2021 04:23:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2hz6UWfo8NlIyUeLj9eB5EU1nny5r2LBbc2rHGJLb5s=; b=QbFj7ypd36RGbrYl7w64cU3WkLw1qDluUxd5bhdbfLC2xFtUynCa6MYPWaEuSmWHxC JKDoI5dPKbwfbMpP5UrxcxZLn9rlN95mURvCkCt5MIvrTSZEYwW19x+5z7oeEUNc0Pm/ Fobl9bG0TkjyFaWmyv+IKQyA6+kdQT2iR83ZNTnNTr/XQyRy10CvmBR1kQBQxdsJ7+PY HL11n+hjJvh7LJcXG8aY5nxxCU/OkUlPyu+UuGo+cBUim5BtCQjrWO7bRI6pmusc3tMb dvloqmlpUdwSOClkZWvb6Mk18hlBJVJAGP3sP+6ZvT/DwSOuJReeYj0sfn3KJLlr2UYR 7ZaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2hz6UWfo8NlIyUeLj9eB5EU1nny5r2LBbc2rHGJLb5s=; b=OuisQipfpx5r46jx4qm3lzAUHNTIvsp4NbbkYrlngV4Pi/TLfjlm7GuworFPL5b05Q YJxiUW2pGajAHudylbktf1qn6vI5FtXDdynkuWBUO/6xSl3nHrIYvyAdceAg34pNzIQr jUMMVQvAybwdd1PqICZnrTDyzcn7mUqBT0iEPDMSeYi0jpr8OPYzarTNs+5q6BupZ1H1 BPrxY8qIKpI9A5sxBsYcjUs2fY0H6hD6Ds0IQud3oqWPItvZTfhN3e+uGraQBVw7/F7s +g/g6L5CU+zef1tkHdmeh76Qe3XB3dVztpU2hHtMsVyI+PRLd+5KVdnZ9U3WS+dByita UX4A== X-Gm-Message-State: AOAM533pYlp8nZyPw9eeEEDPCKYDKlqtzGIYlBjNugT3KJfTMVkUJoul OoVJla3dxT4a+6ktHkwgI56285mGKAgq1Q== X-Google-Smtp-Source: ABdhPJwMtRRRHtUw0nu+8juP8CkdjwLWgPNITZnMKJuhLDWGxV5g5gAiSc9/ocuoTLlVijcK3lzJDQ== X-Received: by 2002:a17:906:1dd1:: with SMTP id v17mr4179870ejh.31.1621509807390; Thu, 20 May 2021 04:23:27 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:26 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 15/17] object-store.h: move read_loose_object() below 'struct object_info' Date: Thu, 20 May 2021 13:23:09 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move the declaration of read_loose_object() below "struct object_info". In the next commit we'll add a "struct object_info *" parameter to it, moving it will avoid a forward declaration of the struct. Signed-off-by: Ævar Arnfjörð Bjarmason --- object-store.h | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/object-store.h b/object-store.h index 790e8b1798..698a701d70 100644 --- a/object-store.h +++ b/object-store.h @@ -234,20 +234,6 @@ int pretend_object_file(void *, unsigned long, enum object_type, int force_object_loose(const struct object_id *oid, time_t mtime); -/* - * Open the loose object at path, check its hash, and return the contents, - * type, and size. If the object is a blob, then "contents" may return NULL, - * to allow streaming of large blobs. - * - * Returns 0 on success, negative on error (details may be written to stderr). - */ -int read_loose_object(const char *path, - const struct object_id *expected_oid, - enum object_type *type, - unsigned long *size, - void **contents, - unsigned int oi_flags); - /* Retry packed storage after checking packed and loose storage */ #define HAS_OBJECT_RECHECK_PACKED 1 @@ -388,6 +374,20 @@ int oid_object_info_extended(struct repository *r, const struct object_id *, struct object_info *, unsigned flags); +/* + * Open the loose object at path, check its hash, and return the contents, + * type, and size. If the object is a blob, then "contents" may return NULL, + * to allow streaming of large blobs. + * + * Returns 0 on success, negative on error (details may be written to stderr). + */ +int read_loose_object(const char *path, + const struct object_id *expected_oid, + enum object_type *type, + unsigned long *size, + void **contents, + unsigned int oi_flags); + /* * Iterate over the files in the loose-object parts of the object * directory "path", triggering the following callbacks: From patchwork Thu May 20 11:23:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21B2DC433B4 for ; Thu, 20 May 2021 12:26:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EFE6860200 for ; Thu, 20 May 2021 12:26:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231657AbhETM2B (ORCPT ); Thu, 20 May 2021 08:28:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233757AbhETM0x (ORCPT ); Thu, 20 May 2021 08:26:53 -0400 Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [IPv6:2a00:1450:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05528C0438E0 for ; Thu, 20 May 2021 04:23:30 -0700 (PDT) Received: by mail-ej1-x631.google.com with SMTP id s22so24456259ejv.12 for ; Thu, 20 May 2021 04:23:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5WoFbxzc6Mm6zb3PPvGLqk/s2Dpx8PSpK7WfJal56ls=; b=N/zQuGoxGNY6N4U6VBQaB0I9RGkWW1KxTxFWq4NQlveWSKfBZQq+hEAKQWOX8m0VOd i5Pxw6BcWJsZqEycE6erE1y7kNb21rIYMQDFVpL1PRh7QIqGchaOLbCtdgrMQy7snOcH vkcqwfDrK+hrJYyMIRo9ip/+dOodkcw0iWcl0w0DpXJ/2PQJDn4Uum00W5Pw1x3cYFTI RoqP8c1+Xd9K3Guk08/cid/ax7ALg/oH5LMi+tNnCKp3Ohl4jVxKedM2SOIU9Q4hTTNk rNegpb0rKcMpch0RbXp8jnwVvX+tLfF/PehviFye2wsMOqVsDSh/L05hTHbSnod3J8nW DznQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5WoFbxzc6Mm6zb3PPvGLqk/s2Dpx8PSpK7WfJal56ls=; b=aIYJZ5zZiqfQnjJB2SNq426bmBQHwhZAvScQsmSNitihTfQFqkU1xKr+TDyIMqbZ3h 6QfhLr4ccdaZNR7gZXUz1tBdJrY157HOqaOTh2RCP08eTvEi9hjXv5FUYdhL5Ky1550w Aw8od8iL6JOhF8k2UvGXJWWaW10nlzaRyjuM1ELpJWbNrSObptJRaM0Tb5KXbrxT5tsi 6EMX412x6PpYITS16bxdajxl+barQ+P1Xk9fI1Y7XFSByEzjeRaDJ/Zt6D8BChuc2Hr3 W5o/KHhjJazfKzKqq0f4OJHVhBXyRtX09ihNhNMjpDdol4upY6zDpudgv3TWCxuirciv 8lTg== X-Gm-Message-State: AOAM530dbmYe8a5GG5wacTMFLxYbxTo1CvsIxcfvBjJcna6tWxR9M/AI XzI3syx9Y/QDvjfNgFFfSM7R3dR21xPyVQ== X-Google-Smtp-Source: ABdhPJziEaMjA24Re49PyZsLoqprNL3VtbYzzbpKm2ammqrBylcz/nFzxyfSoA3iHf6nK+cRmXqgCA== X-Received: by 2002:a17:906:1cc3:: with SMTP id i3mr4323755ejh.87.1621509808266; Thu, 20 May 2021 04:23:28 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:27 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 16/17] fsck: report invalid types recorded in objects Date: Thu, 20 May 2021 13:23:10 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Continue the work in the preceding commit and improve the error on: $ git hash-object --stdin -w -t garbage --literally (expected ) error: : object corrupt or missing: [ other fsck output ] To instead emit: $ git fsck error: : object is of unknown type 'garbage': [ other fsck output ] The complaint about a "hash mismatch" was simply an emergent property of how we'd fall though from read_loose_object() into fsck_loose() when we didn't get the data we expected. Now we'll correctly note that the object type is invalid. Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/fsck.c | 22 ++++++++++++++++++---- object-file.c | 13 +++++-------- object-store.h | 4 ++-- t/t1450-fsck.sh | 24 +++++++++++++++++++++--- 4 files changed, 46 insertions(+), 17 deletions(-) diff --git a/builtin/fsck.c b/builtin/fsck.c index 38b515deb6..32f11dc1fe 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -599,12 +599,26 @@ static int fsck_loose(const struct object_id *oid, const char *path, void *data) unsigned long size; void *contents; int eaten; - - if (read_loose_object(path, oid, &type, &size, &contents, - OBJECT_INFO_ALLOW_UNKNOWN_TYPE) < 0) { - errors_found |= ERROR_OBJECT; + struct strbuf sb = STRBUF_INIT; + unsigned int oi_flags = OBJECT_INFO_ALLOW_UNKNOWN_TYPE; + struct object_info oi; + int found = 0; + oi.type_name = &sb; + oi.sizep = &size; + oi.typep = &type; + + if (read_loose_object(path, oid, &contents, &oi, oi_flags) < 0) { + found |= ERROR_OBJECT; error(_("%s: object corrupt or missing: %s"), oid_to_hex(oid), path); + } + if (type < 0) { + found |= ERROR_OBJECT; + error(_("%s: object is of unknown type '%s': %s"), + oid_to_hex(oid), sb.buf, path); + } + if (found) { + errors_found |= ERROR_OBJECT; return 0; /* keep checking other objects */ } diff --git a/object-file.c b/object-file.c index 0e8a024eb3..f24fa2fe4a 100644 --- a/object-file.c +++ b/object-file.c @@ -2520,9 +2520,8 @@ static int check_stream_oid(git_zstream *stream, int read_loose_object(const char *path, const struct object_id *expected_oid, - enum object_type *type, - unsigned long *size, void **contents, + struct object_info *oi, unsigned int oi_flags) { int ret = -1; @@ -2530,10 +2529,9 @@ int read_loose_object(const char *path, unsigned long mapsize; git_zstream stream; char hdr[MAX_HEADER_LEN]; - struct object_info oi = OBJECT_INFO_INIT; int allow_unknown = oi_flags & OBJECT_INFO_ALLOW_UNKNOWN_TYPE; - oi.typep = type; - oi.sizep = size; + enum object_type *type = oi->typep; + unsigned long *size = oi->sizep; *contents = NULL; @@ -2549,7 +2547,7 @@ int read_loose_object(const char *path, goto out; } - if (parse_loose_header(hdr, &oi) < 0) { + if (parse_loose_header(hdr, oi) < 0) { error(_("unable to parse header of %s"), path); git_inflate_end(&stream); goto out; @@ -2571,8 +2569,7 @@ int read_loose_object(const char *path, goto out; } if (check_object_signature(the_repository, expected_oid, - *contents, *size, - type_name(*type))) { + *contents, *size, oi->type_name->buf)) { error(_("hash mismatch for %s (expected %s)"), path, oid_to_hex(expected_oid)); free(*contents); diff --git a/object-store.h b/object-store.h index 698a701d70..ba6e5d76c0 100644 --- a/object-store.h +++ b/object-store.h @@ -376,6 +376,7 @@ int oid_object_info_extended(struct repository *r, /* * Open the loose object at path, check its hash, and return the contents, + * use the "oi" argument to assert things about the object, or e.g. populate its * type, and size. If the object is a blob, then "contents" may return NULL, * to allow streaming of large blobs. * @@ -383,9 +384,8 @@ int oid_object_info_extended(struct repository *r, */ int read_loose_object(const char *path, const struct object_id *expected_oid, - enum object_type *type, - unsigned long *size, void **contents, + struct object_info *oi, unsigned int oi_flags); /* diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh index e7e8decebb..bc541af2cf 100755 --- a/t/t1450-fsck.sh +++ b/t/t1450-fsck.sh @@ -66,6 +66,25 @@ test_expect_success 'object with hash mismatch' ' ) ' +test_expect_success 'object with hash and type mismatch' ' + test_create_repo hash-type-mismatch && + ( + cd hash-type-mismatch && + oid=$(echo blob | git hash-object -w --stdin -t garbage --literally) && + old=$(test_oid_to_path "$oid") && + new=$(dirname $old)/$(test_oid ff_2) && + oid="$(dirname $new)$(basename $new)" && + mv .git/objects/$old .git/objects/$new && + git update-index --add --cacheinfo 100644 $oid foo && + tree=$(git write-tree) && + cmt=$(echo bogus | git commit-tree $tree) && + git update-ref refs/heads/bogus $cmt && + test_must_fail git fsck 2>out && + grep "^error: hash mismatch for " out && + grep "^error: $oid: object is of unknown type '"'"'garbage'"'"'" out + ) +' + test_expect_success 'branch pointing to non-commit' ' git rev-parse HEAD^{tree} >.git/refs/heads/invalid && test_when_finished "git update-ref -d refs/heads/invalid" && @@ -869,9 +888,8 @@ test_expect_success 'fsck error and recovery on invalid object type' ' garbage_blob=$(git -C garbage-type hash-object --stdin -w -t garbage --literally out 2>err && grep -e "^error" -e "^fatal" err >errors && - test_line_count = 2 errors && - grep "error: hash mismatch for" err && - grep "$garbage_blob: object corrupt or missing:" err && + test_line_count = 1 errors && + grep "$garbage_blob: object is of unknown type '"'"'garbage'"'"':" err && grep "dangling blob $empty_blob" out ' From patchwork Thu May 20 11:23:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12270071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD786C433B4 for ; Thu, 20 May 2021 12:26:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BD2F4611ED for ; Thu, 20 May 2021 12:26:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238649AbhETM1f (ORCPT ); Thu, 20 May 2021 08:27:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54638 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233751AbhETM0x (ORCPT ); Thu, 20 May 2021 08:26:53 -0400 Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C07EC0438E1 for ; Thu, 20 May 2021 04:23:31 -0700 (PDT) Received: by mail-ed1-x52b.google.com with SMTP id j10so1403037edw.8 for ; Thu, 20 May 2021 04:23:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8sCnso70+NgPvXNoV5EF1/eaUMgOY8WIjfJc8vsWvTc=; b=QHermpUMXaac0qh/BdYB51xWRKNUExVKRfBl8QNj2ZjKlAT7XvYh2UAe92Y0pikgfE attkUCC/bQZveJ+5e1Jj6Z6QgJdXgT+6igQTzuwPfsN3guJXj/uARW2pfJZpPj2jPcGG Pa3lBnsJgG905FUgr9idf8CmKsaESVT8ZZa+PiAF0vJPFPZ5SF7KW8180OHuIunjGPQK zki+Gxw/swiSrWX/6DzycT2C/VV5Xd0KQnggPX8z27OGXuoWgoEHGPZnqytneiM58IBl a/AgjnWYxvHwkwXOnh6M4Xzxav7j6AcFAY8UKZ3CaF2GyPZOQDBMxutYuW8lrxq9hv+O PF0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8sCnso70+NgPvXNoV5EF1/eaUMgOY8WIjfJc8vsWvTc=; b=Ud+B63NhG3dTNQBVvkkf85erx5/bKaCZFwXuwuas2t9DFCJxfVqhkaKTlmwR/3ssBL Dg0zD9r9flM6csYi210ZdoPMvfE+IogIR4JK5507SOzF5+QjXI1XiGxIVkBEvnBaZBCq K+u6UqrpSE5EhbTv7rgeuyO7w8RIzVO95UG4TDgY3ZleHCqr6JE7K+hD+T9eIxQwL0MI eftVmQQ3aOSq86ZwG/YD0S0Z9LDGD/0pfdS88hrSH5BiV3JxJr0ECeZl9tsrHV9MZClM LsuhtY/TfMecqP6Pd7FST2d+Q0UlPiH5Ax92i08oY948DNWbTqLsyDuOpysSXsAJqij/ 612Q== X-Gm-Message-State: AOAM530IaP06FC/OX9ZdW+E/llMuIrdh8rOu4QC7jc/EYaFt75QYjAFh As+Xhsv3fxDqECIYLAQOTR/mqTJCC7fzSA== X-Google-Smtp-Source: ABdhPJyT/VdKZ/6K3U8sOSWEry2hOzdIvijVYGT3lTPlAU9A13QAuW8h+zaQH8DfRJLAtsuG8ov5Fg== X-Received: by 2002:aa7:d74a:: with SMTP id a10mr4296545eds.311.1621509809162; Thu, 20 May 2021 04:23:29 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id jy21sm1243273ejc.13.2021.05.20.04.23.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 04:23:28 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Johannes Sixt , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFz?= =?utf-8?b?b24=?= Subject: [PATCH v3 17/17] fsck: report invalid object type-path combinations Date: Thu, 20 May 2021 13:23:11 +0200 Message-Id: X-Mailer: git-send-email 2.32.0.rc0.406.g73369325f8d In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Improve the error that's emitted in cases where we find a loose object we parse, but which isn't at the location we expect it to be. Before this change we'd prefix the error with a not-a-OID derived from the path at which the object was found, due to an emergent behavior in how we'd end up with an "OID" in these codepaths. Now we'll instead say what object we hashed, and what path it was found at. Before this patch series e.g.: $ git hash-object --stdin -w -t blob --- builtin/fast-export.c | 2 +- builtin/fsck.c | 13 +++++++++---- builtin/index-pack.c | 2 +- builtin/mktag.c | 3 ++- object-file.c | 21 ++++++++++++--------- object-store.h | 4 +++- object.c | 4 ++-- pack-check.c | 3 ++- t/t1006-cat-file.sh | 2 +- t/t1450-fsck.sh | 8 +++++--- 10 files changed, 38 insertions(+), 24 deletions(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 3c20f164f0..48a3b6a7f8 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -312,7 +312,7 @@ static void export_blob(const struct object_id *oid) if (!buf) die("could not read blob %s", oid_to_hex(oid)); if (check_object_signature(the_repository, oid, buf, size, - type_name(type)) < 0) + type_name(type), NULL) < 0) die("oid mismatch in blob %s", oid_to_hex(oid)); object = parse_object_buffer(the_repository, oid, type, size, buf, &eaten); diff --git a/builtin/fsck.c b/builtin/fsck.c index 32f11dc1fe..96df1aadbf 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -602,20 +602,25 @@ static int fsck_loose(const struct object_id *oid, const char *path, void *data) struct strbuf sb = STRBUF_INIT; unsigned int oi_flags = OBJECT_INFO_ALLOW_UNKNOWN_TYPE; struct object_info oi; + struct object_id real_oid = *null_oid(); int found = 0; oi.type_name = &sb; oi.sizep = &size; oi.typep = &type; - if (read_loose_object(path, oid, &contents, &oi, oi_flags) < 0) { + if (read_loose_object(path, oid, &real_oid, &contents, &oi, oi_flags) < 0) { found |= ERROR_OBJECT; - error(_("%s: object corrupt or missing: %s"), - oid_to_hex(oid), path); + if (!oideq(&real_oid, oid)) + error(_("%s: hash-path mismatch, found at: %s"), + oid_to_hex(&real_oid), path); + else + error(_("%s: object corrupt or missing: %s"), + oid_to_hex(oid), path); } if (type < 0) { found |= ERROR_OBJECT; error(_("%s: object is of unknown type '%s': %s"), - oid_to_hex(oid), sb.buf, path); + oid_to_hex(&real_oid), sb.buf, path); } if (found) { errors_found |= ERROR_OBJECT; diff --git a/builtin/index-pack.c b/builtin/index-pack.c index 3fbc5d7077..bf860b6555 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -1421,7 +1421,7 @@ static void fix_unresolved_deltas(struct hashfile *f) if (check_object_signature(the_repository, &d->oid, data, size, - type_name(type))) + type_name(type), NULL)) die(_("local object %s is corrupt"), oid_to_hex(&d->oid)); /* diff --git a/builtin/mktag.c b/builtin/mktag.c index dddcccdd36..3b2dbbb37e 100644 --- a/builtin/mktag.c +++ b/builtin/mktag.c @@ -62,7 +62,8 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type) repl = lookup_replace_object(the_repository, tagged_oid); ret = check_object_signature(the_repository, repl, - buffer, size, type_name(*tagged_type)); + buffer, size, type_name(*tagged_type), + NULL); free(buffer); return ret; diff --git a/object-file.c b/object-file.c index f24fa2fe4a..bf9ac38d73 100644 --- a/object-file.c +++ b/object-file.c @@ -1039,9 +1039,11 @@ void *xmmap(void *start, size_t length, * the streaming interface and rehash it to do the same. */ int check_object_signature(struct repository *r, const struct object_id *oid, - void *map, unsigned long size, const char *type) + void *map, unsigned long size, const char *type, + struct object_id *real_oidp) { - struct object_id real_oid; + struct object_id tmp; + struct object_id *real_oid = real_oidp ? real_oidp : &tmp; enum object_type obj_type; struct git_istream *st; git_hash_ctx c; @@ -1049,8 +1051,8 @@ int check_object_signature(struct repository *r, const struct object_id *oid, int hdrlen; if (map) { - hash_object_file(r->hash_algo, map, size, type, &real_oid); - return !oideq(oid, &real_oid) ? -1 : 0; + hash_object_file(r->hash_algo, map, size, type, real_oid); + return !oideq(oid, real_oid) ? -1 : 0; } st = open_istream(r, oid, &obj_type, &size, NULL); @@ -1075,9 +1077,9 @@ int check_object_signature(struct repository *r, const struct object_id *oid, break; r->hash_algo->update_fn(&c, buf, readlen); } - r->hash_algo->final_oid_fn(&real_oid, &c); + r->hash_algo->final_oid_fn(real_oid, &c); close_istream(st); - return !oideq(oid, &real_oid) ? -1 : 0; + return !oideq(oid, real_oid) ? -1 : 0; } int git_open_cloexec(const char *name, int flags) @@ -2520,6 +2522,7 @@ static int check_stream_oid(git_zstream *stream, int read_loose_object(const char *path, const struct object_id *expected_oid, + struct object_id *real_oid, void **contents, struct object_info *oi, unsigned int oi_flags) @@ -2569,9 +2572,9 @@ int read_loose_object(const char *path, goto out; } if (check_object_signature(the_repository, expected_oid, - *contents, *size, oi->type_name->buf)) { - error(_("hash mismatch for %s (expected %s)"), path, - oid_to_hex(expected_oid)); + *contents, *size, oi->type_name->buf, real_oid)) { + if (oideq(real_oid, null_oid())) + BUG("should only get OID mismatch errors with mapped contents"); free(*contents); goto out; } diff --git a/object-store.h b/object-store.h index ba6e5d76c0..60b566a63b 100644 --- a/object-store.h +++ b/object-store.h @@ -384,6 +384,7 @@ int oid_object_info_extended(struct repository *r, */ int read_loose_object(const char *path, const struct object_id *expected_oid, + struct object_id *real_oid, void **contents, struct object_info *oi, unsigned int oi_flags); @@ -505,7 +506,8 @@ int unpack_loose_header(git_zstream *stream, unsigned char *map, int parse_loose_header(const char *hdr, struct object_info *oi); int check_object_signature(struct repository *r, const struct object_id *oid, - void *buf, unsigned long size, const char *type); + void *buf, unsigned long size, const char *type, + struct object_id *real_oidp); int finalize_object_file(const char *tmpfile, const char *filename); int check_and_freshen_file(const char *fn, int freshen); diff --git a/object.c b/object.c index 14188453c5..5467ead328 100644 --- a/object.c +++ b/object.c @@ -261,7 +261,7 @@ struct object *parse_object(struct repository *r, const struct object_id *oid) if ((obj && obj->type == OBJ_BLOB && repo_has_object_file(r, oid)) || (!obj && repo_has_object_file(r, oid) && oid_object_info(r, oid, NULL) == OBJ_BLOB)) { - if (check_object_signature(r, repl, NULL, 0, NULL) < 0) { + if (check_object_signature(r, repl, NULL, 0, NULL, NULL) < 0) { error(_("hash mismatch %s"), oid_to_hex(oid)); return NULL; } @@ -272,7 +272,7 @@ struct object *parse_object(struct repository *r, const struct object_id *oid) buffer = repo_read_object_file(r, oid, &type, &size); if (buffer) { if (check_object_signature(r, repl, buffer, size, - type_name(type)) < 0) { + type_name(type), NULL) < 0) { free(buffer); error(_("hash mismatch %s"), oid_to_hex(repl)); return NULL; diff --git a/pack-check.c b/pack-check.c index 4b089fe8ec..e6aa4442c9 100644 --- a/pack-check.c +++ b/pack-check.c @@ -142,7 +142,8 @@ static int verify_packfile(struct repository *r, err = error("cannot unpack %s from %s at offset %"PRIuMAX"", oid_to_hex(&oid), p->pack_name, (uintmax_t)entries[i].offset); - else if (check_object_signature(r, &oid, data, size, type_name(type))) + else if (check_object_signature(r, &oid, data, size, + type_name(type), NULL)) err = error("packed %s from %s is corrupt", oid_to_hex(&oid), p->pack_name); else if (fn) { diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index f12b06150e..0bb5ee97de 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -490,7 +490,7 @@ test_expect_success 'cat-file -t and -s on corrupt loose object' ' # Swap the two to corrupt the repository mv -v "$other_path" "$empty_path" && test_must_fail git fsck 2>err.fsck && - grep "hash mismatch" err.fsck && + grep "hash-path mismatch" err.fsck && # confirm that cat-file is reading the new swapped-in # blob... diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh index bc541af2cf..d76293c495 100755 --- a/t/t1450-fsck.sh +++ b/t/t1450-fsck.sh @@ -53,6 +53,7 @@ test_expect_success 'object with hash mismatch' ' ( cd hash-mismatch && oid=$(echo blob | git hash-object -w --stdin) && + oldoid=$oid && old=$(test_oid_to_path "$oid") && new=$(dirname $old)/$(test_oid ff_2) && oid="$(dirname $new)$(basename $new)" && @@ -62,7 +63,7 @@ test_expect_success 'object with hash mismatch' ' cmt=$(echo bogus | git commit-tree $tree) && git update-ref refs/heads/bogus $cmt && test_must_fail git fsck 2>out && - test_i18ngrep "$oid.*corrupt" out + grep "$oldoid: hash-path mismatch, found at: .*$new" out ) ' @@ -71,6 +72,7 @@ test_expect_success 'object with hash and type mismatch' ' ( cd hash-type-mismatch && oid=$(echo blob | git hash-object -w --stdin -t garbage --literally) && + oldoid=$oid && old=$(test_oid_to_path "$oid") && new=$(dirname $old)/$(test_oid ff_2) && oid="$(dirname $new)$(basename $new)" && @@ -80,8 +82,8 @@ test_expect_success 'object with hash and type mismatch' ' cmt=$(echo bogus | git commit-tree $tree) && git update-ref refs/heads/bogus $cmt && test_must_fail git fsck 2>out && - grep "^error: hash mismatch for " out && - grep "^error: $oid: object is of unknown type '"'"'garbage'"'"'" out + grep "^error: $oldoid: hash-path mismatch, found at: .*$new" out && + grep "^error: $oldoid: object is of unknown type '"'"'garbage'"'"'" out ) '