From patchwork Tue Sep 7 10:57:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4694C433EF for ; Tue, 7 Sep 2021 10:58:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C5A0460E52 for ; Tue, 7 Sep 2021 10:58:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343801AbhIGK7e (ORCPT ); Tue, 7 Sep 2021 06:59:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245383AbhIGK7c (ORCPT ); Tue, 7 Sep 2021 06:59:32 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 406F7C061575 for ; Tue, 7 Sep 2021 03:58:26 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id d6so13130386wrc.11 for ; Tue, 07 Sep 2021 03:58:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9/HWubv+axYRNPDzHdSSH0dhZV08i3gcrYKofoJlGdw=; b=j2YPvSxA3ooipiBd1y0VqHPSljnxT9blEQzC4LA5EyKefi0HsWPqzx9aAZh6X38vAY Xfl0z84+X1y7HjLd1upoPvc0bqTXUV3UGZ6x0MCv0QV1cm8OExt6smWb1r+YCus066fi apNyW9ZowBFKHerZF815nTxmmhrtf6eyKWoR3Ctn8iMCbviWtdDhgl8RN6nSyx8ZRnZ3 /kmjpDbkhEzSURD6tDxarJU6Y05xB1tsGGzKDVdjX4DIN6D1llHJYrwbJK6/FF8Kmr1/ F9GYPQ6Q31gBRHWnz+mAlBPHvb6M3Ocf9d9EzCsL9wE0yCXe1o3YQuB2l3xrCSZ3BAmF UiHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9/HWubv+axYRNPDzHdSSH0dhZV08i3gcrYKofoJlGdw=; b=EC5wm4qTQMIKurTUWcmQ90PBGcHpVdArB3hw6SHQzgytWjyjDxrbOcC1LqJ8XBToec ihhd5WlaeVXUuMmQ48k+g1he6AzmWkVscnQqev8VIrV5svqCwi4YKYc4b0yptbw3AAem uChuRUorndECPF24KcrsacCoibJF5N/YCQLw2Z0okxAzAmMS4PEjH9MtT/zsX0/IiVjg V11mqPwxxz3yenHa8kr17rvdSZIrScAeLFkREfnS+9fe353hhfT2xLRH2GU7Sy5Y95zi 3LqQpu+Dwyr/x8MtkMitGhCiH5gzuCO+iprEloVTWh+oe0lETJKLe53ZVOteTLiqXAIL L+UA== X-Gm-Message-State: AOAM530uZpWzI7rcA11jsUaDxvltfaF/cENUSYp+teM6vv7u9ctlPrdQ e9DWT9/fERnliY89KVTDlQB7elh7q80xYw== X-Google-Smtp-Source: ABdhPJzsFNr1jKkI0QYQgqx6VmyIz5C41h0SQ1u9mfuSJIyIJUg6M7bM6Ea2Hrtzr43q3N1u03pULQ== X-Received: by 2002:adf:9d47:: with SMTP id o7mr18458244wre.50.1631012304562; Tue, 07 Sep 2021 03:58:24 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:23 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 01/22] fsck tests: refactor one test to use a sub-repo Date: Tue, 7 Sep 2021 12:57:56 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Refactor one of the fsck tests to use a throwaway repository. It's a pervasive pattern in t1450-fsck.sh to spend a lot of effort on the teardown of a tests so we're not leaving corrupt content for the next test. We should instead simply use something like this test_create_repo pattern. It's both less verbose, and makes things easier to debug as a failing test can have their state left behind under -d without damaging the state for other tests. But let's punt on that general refactoring and just change this one test, I'm going to change it further in subsequent commits. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1450-fsck.sh | 34 ++++++++++++++++------------------ 1 file changed, 16 insertions(+), 18 deletions(-) diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh index 5071ac63a5b..7becab5ba1e 100755 --- a/t/t1450-fsck.sh +++ b/t/t1450-fsck.sh @@ -48,24 +48,22 @@ remove_object () { rm "$(sha1_file "$1")" } -test_expect_success 'object with bad sha1' ' - sha=$(echo blob | git hash-object -w --stdin) && - old=$(test_oid_to_path "$sha") && - new=$(dirname $old)/$(test_oid ff_2) && - sha="$(dirname $new)$(basename $new)" && - mv .git/objects/$old .git/objects/$new && - test_when_finished "remove_object $sha" && - git update-index --add --cacheinfo 100644 $sha foo && - test_when_finished "git read-tree -u --reset HEAD" && - tree=$(git write-tree) && - test_when_finished "remove_object $tree" && - cmt=$(echo bogus | git commit-tree $tree) && - test_when_finished "remove_object $cmt" && - git update-ref refs/heads/bogus $cmt && - test_when_finished "git update-ref -d refs/heads/bogus" && - - test_must_fail git fsck 2>out && - test_i18ngrep "$sha.*corrupt" out +test_expect_success 'object with hash mismatch' ' + git init --bare hash-mismatch && + ( + cd hash-mismatch && + oid=$(echo blob | git hash-object -w --stdin) && + old=$(test_oid_to_path "$oid") && + new=$(dirname $old)/$(test_oid ff_2) && + oid="$(dirname $new)$(basename $new)" && + mv objects/$old objects/$new && + git update-index --add --cacheinfo 100644 $oid foo && + tree=$(git write-tree) && + cmt=$(echo bogus | git commit-tree $tree) && + git update-ref refs/heads/bogus $cmt && + test_must_fail git fsck 2>out && + test_i18ngrep "$oid.*corrupt" out + ) ' test_expect_success 'branch pointing to non-commit' ' From patchwork Tue Sep 7 10:57:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D40BC433F5 for ; Tue, 7 Sep 2021 10:58:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6DDFB610F8 for ; Tue, 7 Sep 2021 10:58:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343805AbhIGK7f (ORCPT ); Tue, 7 Sep 2021 06:59:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50992 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343792AbhIGK7d (ORCPT ); Tue, 7 Sep 2021 06:59:33 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4137CC061575 for ; Tue, 7 Sep 2021 03:58:27 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id v10so13781197wrd.4 for ; Tue, 07 Sep 2021 03:58:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gHYIxajtkUyojn1F4BH6Y2hATQ5sJwhM+//9NYKMkaU=; b=Dzn+GeMHDPJGUDt+8C1GSDuSNSKfG1H1f62aStYlvwO5WeMlScKnPndglukeLsGH8u 6XpXJ+cc7al59uyxoV5HLRsZ1DyZ4RxRVUWRU48/JvsHlPQiD4FycckXGFBp8/E8XVtT P6y8B3MEpO4DZ3WnA6Vlh+hpmp/b4HPMCfder+z0A5yA/0ZJF67DUTcT64KOgJioa5bf lQElu8WPgPvDYeCLhV9XeIH0zadeHAzSyHGKyLGN1eo/aZPAFCNFdQS47b3fDAK2Oq9R UOUsKgqanDcp+ea4fMjtGEU4aQv3NlNNj695uKCeceUbIuctvrcRL77HrvbGVyLebVEQ /SRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gHYIxajtkUyojn1F4BH6Y2hATQ5sJwhM+//9NYKMkaU=; b=rQLHWmXBzfTfk6KWPh8tOqcipbThehV301MExPNJZriRGYDAFLumcrT9yHhPernzHY GywVI6ECX3I0+CeyEfw1wA1Kr1yz5tIUSqvczuhQbvlEW3ewRcwq95MnqMXdasIguNX7 qMHmxO6WVRXaEkMtaFIz9OEuWL3GVjQmQWgDbJtSX4u28hVXzSfZteNyF8Sfy5xmbP7j IiZ8g+3bHF7PKeLpj1G39NJy0fEXKYJtti4Hbuq40H6QAIoE/XQHhD9bRQzTfFTwvgI0 ZwI3SvUfb8MeW13dy/mSa4AYiwzTo4EAoMw2r25N5v2TnXb2rSFWbtItT1ZCh/L4mDrC zH0w== X-Gm-Message-State: AOAM532C7zVJw3jKypDGN3os+iAvh9ChD4tVVXCxShTFbKAlz28ZVmU7 ce/bFfngr77umZIco1eNk4Lbz/Onw9kPTA== X-Google-Smtp-Source: ABdhPJwoJBZTe0oro777mwRgj0fG896vG/widTiDkRD2fYaBPLD+HPStxuwcYnYb31Hkb8+pYYD/Zw== X-Received: by 2002:adf:fc43:: with SMTP id e3mr17764836wrs.436.1631012305625; Tue, 07 Sep 2021 03:58:25 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:24 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 02/22] fsck tests: add test for fsck-ing an unknown type Date: Tue, 7 Sep 2021 12:57:57 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix a blindspot in the fsck tests by checking what we do when we encounter an unknown "garbage" type produced with hash-object's --literally option. This behavior needs to be improved, which'll be done in subsequent patches, but for now let's test for the current behavior. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1450-fsck.sh | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh index 7becab5ba1e..f10d6f7b7e8 100755 --- a/t/t1450-fsck.sh +++ b/t/t1450-fsck.sh @@ -863,4 +863,16 @@ test_expect_success 'detect corrupt index file in fsck' ' test_i18ngrep "bad index file" errors ' +test_expect_success 'fsck hard errors on an invalid object type' ' + git init --bare garbage-type && + empty_blob=$(git -C garbage-type hash-object --stdin -w -t blob err.expect <<-\EOF && + fatal: invalid object type + EOF + test_must_fail git -C garbage-type fsck >out.actual 2>err.actual && + test_cmp err.expect err.actual && + test_must_be_empty out.actual +' + test_done From patchwork Tue Sep 7 10:57:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478297 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EECE5C433FE for ; Tue, 7 Sep 2021 10:58:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CC07160E52 for ; Tue, 7 Sep 2021 10:58:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343792AbhIGK7g (ORCPT ); Tue, 7 Sep 2021 06:59:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343804AbhIGK7e (ORCPT ); Tue, 7 Sep 2021 06:59:34 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 133C6C061757 for ; Tue, 7 Sep 2021 03:58:28 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id 192-20020a1c04c9000000b002f7a4ab0a49so1591179wme.0 for ; Tue, 07 Sep 2021 03:58:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Q00M4AtvtJU2iuLRc/A1xwyi5qwWrl9cgtEiDhBbJBY=; b=lDojtrYSSIO4TZQj7Ctfr3ai48Nw6OIEKp4KOe/Ad4Y/ayP5bUvEkFcC0Zalt1Y3g2 iKvN9OrTQt3bAdZFXPyxeI957WXBFJ+82NEEu7Va4Yn9gdpF/kEx8SGOARLHvxrxp750 GNTBClj+UAJL2vLzSqQaD0KniNqqK7veu5jBINaThlqZEhfnQAS82pndrNXngxQ5s+ov Pyw60CM1XuIzrpl6TAgovLk8saB0SqCsphBoad7MFXKjj0vU3gvY5pysp3ygylBYNoaw dx5bucRBV0gEjEWU1kV2Qf7B79qaIXKzd9IEGvPqgeZ5RWisSlmH1P+2hNZAofLDYyZa PvNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Q00M4AtvtJU2iuLRc/A1xwyi5qwWrl9cgtEiDhBbJBY=; b=gGC6NLi3dN/Dhoa6K2mcWktlLBRyWnC8xqJ3snqQWPsyIToMxVcqPpPXAZrXIIv/UI KBy6vd+q72gLAMe/8kH0nm2fE3hfh6cdukXbNlZFaPxBlVzsKQsOl1+EiXFB5xX9oO7R zvJqL2QlXCDO7J80VLOQSl+Jpc3xDYWveXmASfvyX+avj4BzL1T3oTlTkNXQO4oYyieL MDs4Q+TOXLvoxbZSUFx+kpbbhA9qw9EXGEV/dwLbMNz8QeOhapU4OP5FpeXxrQ+sr758 vOEuNuW622Cg3D6p630Gx6IvQ107/tuGO2uzu023LEBs16mAypyT6TSM/l4TM4CSdZKR JPSg== X-Gm-Message-State: AOAM5310r7jhVu5GVwq15ryUhAdgHAB53tc10AAyzeJ81WKJuuLo1PpU sVzEVlwWdV17YFgY/ImhsMxtBHFhr/VyOw== X-Google-Smtp-Source: ABdhPJw+1tVE7UBr7+v6jnnM/D8sdghkNCbkEK4oW+Ti6bZFdl6XQSZiB3DWKsf/ULTscDHM0F8t7Q== X-Received: by 2002:a1c:c911:: with SMTP id f17mr3265763wmb.190.1631012306414; Tue, 07 Sep 2021 03:58:26 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:25 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 03/22] cat-file tests: test for missing object with -t and -s Date: Tue, 7 Sep 2021 12:57:58 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Test for what happens when the -t and -s flags are asked to operate on a missing object, this extends tests added in 3e370f9faf0 (t1006: add tests for git cat-file --allow-unknown-type, 2015-05-03). The -t and -s flags are the only ones that can be combined with --allow-unknown-type, so let's test with and without that flag. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1006-cat-file.sh | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 18b3779ccb6..3a7b138fe4e 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -315,6 +315,33 @@ test_expect_success '%(deltabase) reports packed delta bases' ' } ' +missing_oid=$(test_oid deadbeef) +test_expect_success 'error on type of missing object' ' + cat >expect.err <<-\EOF && + fatal: git cat-file: could not get object info + EOF + test_must_fail git cat-file -t $missing_oid >out 2>err && + test_must_be_empty out && + test_cmp expect.err err && + + test_must_fail git cat-file -t --allow-unknown-type $missing_oid >out 2>err && + test_must_be_empty out && + test_cmp expect.err err +' + +test_expect_success 'error on size of missing object' ' + cat >expect.err <<-\EOF && + fatal: git cat-file: could not get object info + EOF + test_must_fail git cat-file -s $missing_oid >out 2>err && + test_must_be_empty out && + test_cmp expect.err err && + + test_must_fail git cat-file -s --allow-unknown-type $missing_oid >out 2>err && + test_must_be_empty out && + test_cmp expect.err err +' + bogus_type="bogus" bogus_content="bogus" bogus_size=$(strlen "$bogus_content") From patchwork Tue Sep 7 10:57:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3DAFC433EF for ; Tue, 7 Sep 2021 10:58:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AB41760E52 for ; Tue, 7 Sep 2021 10:58:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343831AbhIGK7i (ORCPT ); Tue, 7 Sep 2021 06:59:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343798AbhIGK7f (ORCPT ); Tue, 7 Sep 2021 06:59:35 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F063BC061575 for ; Tue, 7 Sep 2021 03:58:28 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id s24so908052wmh.4 for ; Tue, 07 Sep 2021 03:58:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fZ8ux2fJFm1ThSxAdzfsS9LGMbQ1RnQhrszlzbvc+gE=; b=ojfX/oMqfzJ3+NcUAEZ7rrjPlbmCktq5+NjR5xO88XUsCvL6ieUkjveor0tj+pTJZJ IGubBN6UvVC9ClRlXjqe+jvY4mrdVTH4xSq0Oin7tU5uYupG+3zN/Sm81dfxAEm6RWTY 13I4/EiipTgUx7tQWv4IG9vn6Z/qrT6rHcJhaM/Imyr981GsMWUzkBp+2zkXtcLwlXqz 4QycFnxqzABPEfTs31opKcvlOBaZrSe8KVgM+kLQQLHBU1O/Xtz6dVQ7VZwRzYNMwj+2 539RUILUpOSuhQpcHoDtjxPbIAOUK92IeFippTbgOc5eIRBnIj7yNOTM3PCwEB8LIiLg 9RRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fZ8ux2fJFm1ThSxAdzfsS9LGMbQ1RnQhrszlzbvc+gE=; b=gDhJw5glDjchL7CFLNRk4bnUEvpf+BDIgM61cYcZkRmi0TF+lyc40fnSxWMygbX7k2 kn1vLMJo8SBGEMC2cCM/QcueqGrEolYRIHGEmacQaREREnaQHIVfoZ7MkyWrtvUyhO22 0x7AVtrHvcWeDxggl8e3Dcg+/nDnbfLUAqBykjVoB2yA9olOfDY2CzNdjE8PU9uqRuMo b2+tNwpap6wWbYQSLXhjftH1mMr76CIXqM0aGjh/ALNd4R9rk1jqjrMDoG9nXZ9lN8eU YWNh3i79gnIvCNpzMSesWEatUhxEjAn/eWikoE4BvKLv/acUl6HWQaq8LL5nyETWxyA4 14SQ== X-Gm-Message-State: AOAM530Z0Ci6o6aAUl41oKZrZuIOFGxRQli0VG3BM+c/pT4zekJ/cixg eRaFjjFQNwt5J9sZBV/IS18t0IViwYIWWQ== X-Google-Smtp-Source: ABdhPJw69qGNvI3EnoZ0EVRlhnPwAKpsUdi7CC+MML2yl/OuO6hdfEbxUU9eughXKt0OoVoAt4vblw== X-Received: by 2002:a05:600c:221a:: with SMTP id z26mr3306894wml.142.1631012307337; Tue, 07 Sep 2021 03:58:27 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:26 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 04/22] cat-file tests: test that --allow-unknown-type isn't on by default Date: Tue, 7 Sep 2021 12:57:59 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix a blindspot in the tests for the --allow-unknown-type feature added in 39e4ae38804 (cat-file: teach cat-file a '--allow-unknown-type' option, 2015-05-03). We should check that --allow-unknown-type isn't on by default. Before this change all the tests would succeed if --allow-unknown-type was on by default, let's fix that by asserting that -t and -s die on a "garbage" type without --allow-unknown-type. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1006-cat-file.sh | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 3a7b138fe4e..5e05ea0861e 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -347,6 +347,20 @@ bogus_content="bogus" bogus_size=$(strlen "$bogus_content") bogus_sha1=$(echo_without_newline "$bogus_content" | git hash-object -t $bogus_type --literally -w --stdin) +test_expect_success 'die on broken object under -t and -s without --allow-unknown-type' ' + cat >err.expect <<-\EOF && + fatal: invalid object type + EOF + + test_must_fail git cat-file -t $bogus_sha1 >out.actual 2>err.actual && + test_cmp err.expect err.actual && + test_must_be_empty out.actual && + + test_must_fail git cat-file -s $bogus_sha1 >out.actual 2>err.actual && + test_cmp err.expect err.actual && + test_must_be_empty out.actual +' + test_expect_success "Type of broken object is correct" ' echo $bogus_type >expect && git cat-file -t --allow-unknown-type $bogus_sha1 >actual && @@ -363,6 +377,21 @@ bogus_content="bogus" bogus_size=$(strlen "$bogus_content") bogus_sha1=$(echo_without_newline "$bogus_content" | git hash-object -t $bogus_type --literally -w --stdin) +test_expect_success 'die on broken object with large type under -t and -s without --allow-unknown-type' ' + cat >err.expect <<-EOF && + error: unable to unpack $bogus_sha1 header + fatal: git cat-file: could not get object info + EOF + + test_must_fail git cat-file -t $bogus_sha1 >out.actual 2>err.actual && + test_cmp err.expect err.actual && + test_must_be_empty out.actual && + + test_must_fail git cat-file -s $bogus_sha1 >out.actual 2>err.actual && + test_cmp err.expect err.actual && + test_must_be_empty out.actual +' + test_expect_success "Type of broken object is correct when type is large" ' echo $bogus_type >expect && git cat-file -t --allow-unknown-type $bogus_sha1 >actual && From patchwork Tue Sep 7 10:58:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4EF3FC433EF for ; Tue, 7 Sep 2021 10:58:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 328AD60F6D for ; Tue, 7 Sep 2021 10:58:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343882AbhIGK7s (ORCPT ); Tue, 7 Sep 2021 06:59:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343816AbhIGK7g (ORCPT ); Tue, 7 Sep 2021 06:59:36 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB089C06175F for ; Tue, 7 Sep 2021 03:58:29 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id b6so13775494wrh.10 for ; Tue, 07 Sep 2021 03:58:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aZ5/7jrpsvS9HxZBssXcFY7qHBSYN3VAo6Q72TRhCYY=; b=D5vE9rWoHKay37Po61u5B2mG20QCjfMDu0EeaRJ4LZZzqHfb3GkOLVm8iBlw5DqIvT zNsR9wIWT00yVGL1nu5IvjtmXy2jHN0F4VEVHCdPwe7bDb6k1wqd1U+4xssIE9xjPNWA LQ19spKP9CyeHmorP0E7vZyak1Eesl18XL23Lx8/h0tG2QuOYV2/KyohnFCnttSTl3wW CjR4goc2zSWe9NwsG0sXxadebpUI5XdAKK+MGIWi9RhfkNPgKqi7powFNDJh7Lh/hh/+ rztZwlwBTx7IkzduUVCYWndCQ7r2zVnDw9RGFp8pVmEVJwyM0sEZtIxPij6mYGdOLqSM A+9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aZ5/7jrpsvS9HxZBssXcFY7qHBSYN3VAo6Q72TRhCYY=; b=HhW/zYRJDVeNQKMIHHUsz78fhd8JhirChZpZTkAY5NUNTmJy9j4/5GwvSpEK7p8RKz ACVUr+XhIINRAgKWiNntaa7ySzr77gdbJuvSVXzbC419FPbdVHUcuBUykE7JFq5cDCD2 OlzNf4VNA15YHzOB3fiCyd+HCSC5TBMO+2JqDyYThDI3nNJq5lzMr55y4Vw27uAsj6Jh aJer1veTjzZhdUxmItNO6V/z7gMuc5Z7PPW3jYzOvtvceXFfdvQkRaUwNHEvZbjUjJG4 YxBpR9TJiUsWCpNeX8MqvCjgeUZGpVBl3vImlMP1x8TA5/GS5gHKK1BaEE3l1Jb/H6cX /1Zw== X-Gm-Message-State: AOAM533LtMd7mrG5D+GVBhF9aUoQm3sbnnaUD2bHPz0zDsFl3PSU/V74 5voGCHCN16Lcsy7bQsH+kPtAgJepLl+TJw== X-Google-Smtp-Source: ABdhPJx8CGLI5nEtTuLOPcrjhfYGI67HvZ+qjcVLknzmIYpcYOzYH7ehot0xlqi4ybzI3k7esgf27Q== X-Received: by 2002:adf:fb8f:: with SMTP id a15mr18328781wrr.92.1631012308279; Tue, 07 Sep 2021 03:58:28 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:27 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 05/22] rev-list tests: test for behavior with invalid object types Date: Tue, 7 Sep 2021 12:58:00 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix a blindspot in the tests for the "rev-list --disk-usage" feature added in 16950f8384a (rev-list: add --disk-usage option for calculating disk usage, 2021-02-09) to test for what happens when it's asked to calculate the disk usage of invalid object types. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t6115-rev-list-du.sh | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/t/t6115-rev-list-du.sh b/t/t6115-rev-list-du.sh index b4aef32b713..edb2ed55846 100755 --- a/t/t6115-rev-list-du.sh +++ b/t/t6115-rev-list-du.sh @@ -48,4 +48,15 @@ check_du HEAD check_du --objects HEAD check_du --objects HEAD^..HEAD +test_expect_success 'setup garbage repository' ' + git clone --bare . garbage.git && + garbage_oid=$(git -C garbage.git hash-object -t garbage -w --stdin --literally garbage.git/refs/tags/garbage-tag && + test_must_fail git -C garbage.git rev-list --objects --all --disk-usage +' + test_done From patchwork Tue Sep 7 10:58:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478303 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AE2CC433FE for ; Tue, 7 Sep 2021 10:58:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0CF60610F8 for ; Tue, 7 Sep 2021 10:58:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343883AbhIGK7t (ORCPT ); Tue, 7 Sep 2021 06:59:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343819AbhIGK7h (ORCPT ); Tue, 7 Sep 2021 06:59:37 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 01409C061757 for ; Tue, 7 Sep 2021 03:58:31 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id q11so13767928wrr.9 for ; Tue, 07 Sep 2021 03:58:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GbKE2r5K2HvEU8KOTr9OLuep7B3kuRwcbVqAKUCbf5A=; b=h/PiuN+9pre6bsFdRDdGORxG5uHIhz6B3x9Ar2ilGZdk9rHpAuUA15qbJjTPHkuGyP CF/P96KlGN6vNJvJ8p6n1rJiQbvimy17dffL4JHpk/HjNZCTAqfX0cCqNNAkI+4McGIi Qs256NfRZKKT+CTYF6kcbu8UxEqaJ+poiATLq209o/dB5bz4MMOGlOeoH2YNOtyAzzjs IGYt2U/W87T4k8UIEYtmoAqtfFZD1dQfz3s8192TK7Ox8F25na0E9VLuvhEvo2jejxVS hfbsHayc6no37oXMh2nlk/f2SlZIk144WO8d1BY+QpyvG8q3IZwWkwpiwTvbr0xOnTQ3 +Ydw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GbKE2r5K2HvEU8KOTr9OLuep7B3kuRwcbVqAKUCbf5A=; b=clSLWFfkVoNwcAFLzQRStfwzgzJjZUinnjpPUXAM9Df95pfALLPIGn88I9b06jo4hp EpIfspQycrB6C8WVE6D+0dkL1lKeE7SVux1Wlsqo0ErOgXJs6yv5C9RVXZna4QR6rxXT mUw58uu48Stm57rH+DL/asbiC3EhwwsrUU90xYAeRRQ3WIkPd+su6Nvs/OZfaniQl8C6 QGIBG+A6N5mJXbp3k00XzvbGaTtYKKA0wtGHU48JuT7Rx6rKfQpQrNQeh/tlk4T2GNV3 7Ivu5rWMvxfbewQFhJpov8CnigNr0XFdfBussrvH3KSzhv/OLBIQZ1GSIKLORNSquzTY 9wRw== X-Gm-Message-State: AOAM5319ZPF6iKRUDhoLSzstNY4C/ZROjCpGOL46xAB4Pk35i6pHX2+1 R5xKINzvckeln1p4mkpaLkpHUtl7aHJB6g== X-Google-Smtp-Source: ABdhPJxZhUrZVBc/NuuCw1+/+Hmrxmnm0Ia9J9NB89SGgHlEdX3SrN31HSCjShJt9yyWFXaTJYHtig== X-Received: by 2002:adf:e68b:: with SMTP id r11mr18224271wrm.394.1631012309348; Tue, 07 Sep 2021 03:58:29 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:28 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 06/22] cat-file tests: add corrupt loose object test Date: Tue, 7 Sep 2021 12:58:01 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix a blindspot in the tests for "cat-file" (and by proxy, the guts of object-file.c) by testing that when we can't decode a loose object with zlib we'll emit an error from zlib.c. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1006-cat-file.sh | 52 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 5e05ea0861e..8f3516db188 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -404,6 +404,58 @@ test_expect_success "Size of large broken object is correct when type is large" test_cmp expect actual ' +test_expect_success 'cat-file -t and -s on corrupt loose object' ' + git init --bare corrupt-loose.git && + ( + cd corrupt-loose.git && + + # Setup and create the empty blob and its path + empty_path=$(git rev-parse --git-path objects/$(test_oid_to_path "$EMPTY_BLOB")) && + git hash-object -w --stdin other.blob && + other_blob=$(git hash-object -w --stdin out.expect <<-EOF && + 0 + EOF + git cat-file -s "$EMPTY_BLOB" >out.actual 2>err.actual && + test_must_be_empty err.actual && + test_cmp out.expect out.actual && + + # Swap the two to corrupt the repository + mv -f "$other_path" "$empty_path" && + test_must_fail git fsck 2>err.fsck && + grep "hash mismatch" err.fsck && + + # confirm that cat-file is reading the new swapped-in + # blob... + cat >out.expect <<-EOF && + blob + EOF + git cat-file -t "$EMPTY_BLOB" >out.actual 2>err.actual && + test_must_be_empty err.actual && + test_cmp out.expect out.actual && + + # ... since it has a different size now. + cat >out.expect <<-EOF && + 6 + EOF + git cat-file -s "$EMPTY_BLOB" >out.actual 2>err.actual && + test_must_be_empty err.actual && + test_cmp out.expect out.actual && + + # So far "cat-file" has been happy to spew the found + # content out as-is. Try to make it zlib-invalid. + mv -f other.blob "$empty_path" && + test_must_fail git fsck 2>err.fsck && + grep "^error: inflate: data stream error (" err.fsck + ) +' + # Tests for git cat-file --follow-symlinks test_expect_success 'prep for symlink tests' ' echo_without_newline "$hello_content" >morx && From patchwork Tue Sep 7 10:58:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478305 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94A8DC433F5 for ; Tue, 7 Sep 2021 10:58:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7E4ED60E52 for ; Tue, 7 Sep 2021 10:58:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343856AbhIGK7u (ORCPT ); Tue, 7 Sep 2021 06:59:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343848AbhIGK7m (ORCPT ); Tue, 7 Sep 2021 06:59:42 -0400 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 12195C0613C1 for ; Tue, 7 Sep 2021 03:58:32 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id g135so6501856wme.5 for ; Tue, 07 Sep 2021 03:58:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mcQHzjc/F8Zn+ZhPENdKj+31A4m8CQ2KxoXa2PuB48k=; b=gD3hmScaMlhB12aLRD6MvIbmpxHGsk3hNaZrDawuKsbIP2F8RG7D7gE6OvTI3RZ6n7 ZRHXUyr3PU8W0uBehWyHmOpW6RZ5GjFlpjxaH0qAcrzIaXYw05HeEvOhmsHyOtcxwEzQ Z8dCklxY0q0aVZTfR3/Ff8pDJUHwsvtXOZ/O/pAESl0nO6LpcarXIMXMRU7/axYRXlRr kXTikomNj+OLIxnN6ZhLiHT/qjSSR1SNXyCC9MWgjyE+6Y7UfXKUims3GBCpRY/ywE5C 2KgkZVqUP52eYnC12fbAdqH4vB7kiYnMPZ7uIM03ZhZIMUlvHpm3lrU96Db2xBGSu6n8 zWwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mcQHzjc/F8Zn+ZhPENdKj+31A4m8CQ2KxoXa2PuB48k=; b=JFy2Ngl1QyOSNrJ7MAoSIBrSOwVpXj5l6zTtp9FoCNczrXGfNyOetOLh0NkCs5SK/5 842kihuDL4MDkmptYeRyb3GQKNvWUSl/gdCzohwYw9TRdBdorcO+AyLd/8riW5ngFDr+ QcQsBSczNYst2pBlIQKuyO+wEcIk4k9mXKuLi5AnunyY58YdS5gU/xsQX8OvHOCx+LN8 pIWHrUivz1UwlSurz583f6ozeYGpTTT7N2WFZBWwYTyTiYuKvTqkGhNaUh3ykz/QZKuh 0Av8/iEnEAUakUqpm3YdWuFBESmMf/jDhBmXfu+7IrcqBvOdpi6UyYAc9bKk3DJ/Md28 hryw== X-Gm-Message-State: AOAM531MS46x+DwwzgVxw8yBdHQEYRqkmq5IbDHBT9+Kdny3zWBfbjvq sL2sMM4k6yj18qRFELZv0BBc9F0J60IVKw== X-Google-Smtp-Source: ABdhPJyVwlgPTdXT2aMfc/RgZ1lVmwJ/8bfXvTxaaXfWurjPLPQ6DK+wVQ6+h4BLWV1x9nfof3JJEQ== X-Received: by 2002:a1c:4d10:: with SMTP id o16mr3339898wmh.60.1631012310417; Tue, 07 Sep 2021 03:58:30 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:29 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 07/22] cat-file tests: test for current --allow-unknown-type behavior Date: Tue, 7 Sep 2021 12:58:02 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add more tests for the current --allow-unknown-type behavior. As noted in [1] I don't think much of this makes sense, but let's test for it as-is so we can see if the behavior changes in the future. 1. https://lore.kernel.org/git/87r1i4qf4h.fsf@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1006-cat-file.sh | 61 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 8f3516db188..98729f1edfc 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -361,6 +361,46 @@ test_expect_success 'die on broken object under -t and -s without --allow-unknow test_must_be_empty out.actual ' +test_expect_success '-e is OK with a broken object without --allow-unknown-type' ' + git cat-file -e $bogus_sha1 +' + +test_expect_success '-e can not be combined with --allow-unknown-type' ' + test_expect_code 128 git cat-file -e --allow-unknown-type $bogus_sha1 +' + +test_expect_success '-p cannot print a broken object even with --allow-unknown-type' ' + test_must_fail git cat-file -p $bogus_sha1 && + test_expect_code 128 git cat-file -p --allow-unknown-type $bogus_sha1 +' + +test_expect_success ' does not work with objects of broken types' ' + cat >err.expect <<-\EOF && + fatal: invalid object type "bogus" + EOF + test_must_fail git cat-file $bogus_type $bogus_sha1 2>err.actual && + test_cmp err.expect err.actual +' + +test_expect_success 'broken types combined with --batch and --batch-check' ' + echo $bogus_sha1 >bogus-oid && + + cat >err.expect <<-\EOF && + fatal: invalid object type + EOF + + test_must_fail git cat-file --batch err.actual && + test_cmp err.expect err.actual && + + test_must_fail git cat-file --batch-check err.actual && + test_cmp err.expect err.actual +' + +test_expect_success 'the --batch and --batch-check options do not combine with --allow-unknown-type' ' + test_expect_code 128 git cat-file --batch --allow-unknown-type expect && git cat-file -t --allow-unknown-type $bogus_sha1 >actual && @@ -372,6 +412,27 @@ test_expect_success "Size of broken object is correct" ' git cat-file -s --allow-unknown-type $bogus_sha1 >actual && test_cmp expect actual ' + +test_expect_success 'the --allow-unknown-type option does not consider replacement refs' ' + cat >expect <<-EOF && + $bogus_type + EOF + git cat-file -t --allow-unknown-type $bogus_sha1 >actual && + test_cmp expect actual && + + # Create it manually, as "git replace" will die on bogus + # types. + head=$(git rev-parse --verify HEAD) && + mkdir -p .git/refs/replace && + echo $head >.git/refs/replace/$bogus_sha1 && + + cat >expect <<-EOF && + commit + EOF + git cat-file -t --allow-unknown-type $bogus_sha1 >actual && + test_cmp expect actual +' + bogus_type="abcdefghijklmnopqrstuvwxyz1234679" bogus_content="bogus" bogus_size=$(strlen "$bogus_content") From patchwork Tue Sep 7 10:58:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23E6EC433F5 for ; Tue, 7 Sep 2021 10:58:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0CFF060F6D for ; Tue, 7 Sep 2021 10:58:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343874AbhIGK7v (ORCPT ); Tue, 7 Sep 2021 06:59:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343818AbhIGK7r (ORCPT ); Tue, 7 Sep 2021 06:59:47 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 314A6C0617AE for ; Tue, 7 Sep 2021 03:58:33 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id t18so3018699wrb.0 for ; Tue, 07 Sep 2021 03:58:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=APCotmmvl/QuUQPDEWeqTOs86uPhFRGzdbGzhIi4Mok=; b=SyhaYBAKFN0jj8jpGz78vWhMfZ+nWYIG//TR6sW19AvaXc8kdPa32YvG6LV8Mm0G71 rC2FEYXntzfvWDDov3W5zTkTolfvuVyx4LeEWjvj8VQ81PSYA03zmvdOwgeZDAOs3PA3 AY63BzyZ/sRrURclJ1/y0RH5Lb91l2fH0W183a0S5EGHuKNK7RunCprDHMyE5siCMa5e BraRTAH8KZIALYu+RiEt76CcrS3gqsBHhtmJZTYtpk6troiS4a7KzuLXhmwsK3KRC/Qr YHEoDxfBEzAlBA5EOzC2LTiecedx160EnfFLXgHnJRMf6v1Ig70Nl0Z3k5F+e006z/eH JCOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=APCotmmvl/QuUQPDEWeqTOs86uPhFRGzdbGzhIi4Mok=; b=n9hIcJ1dWv4h7eUczj5/OATRPpWK8TTs1aYAzQkE+Zbq03dnt64gPRR1TptT2XaNti CyObwS0/cfJ8zQ7c9RjsXtcFN4Urlyb4ZDy0ypMrIFyExmxARnJddW5r9kb+/eLpJolf WqWagnUEyFh1WOXTzM8BD4lp0s6dZQDleJd2uTNDzQuj7kqscO3ONDz+JfBGt7dH5W5T Hm70hlkhaY9XS3VmtwqPpoBpRwO0QymSRmPGP/Id3aF6AhQSVMg+jipSaVeEj1dUuyTH tNhXLKBMDeG7pQUG6MU1zpEZo3q51uKe9AsrQo0xzSfCZVBjKIxPgpDGUXOYjdJPmpnT z6Ew== X-Gm-Message-State: AOAM530St8bGP9wD6pZm4Xty0NrWsKZuXMJwCL0uRCJla8EWnmptSiIC YCOS8RAN4OzJCcypFl+9eZAmWrjdDvtH+g== X-Google-Smtp-Source: ABdhPJwurOIxFUA/yZ7Fo7BfFWI156dJMwKwUsPh2Ld1vFerWj69/56XZgnt62wjCUZTkcwF5HzgCg== X-Received: by 2002:adf:ea4d:: with SMTP id j13mr17970987wrn.86.1631012311500; Tue, 07 Sep 2021 03:58:31 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:30 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 08/22] object-file.c: don't set "typep" when returning non-zero Date: Tue, 7 Sep 2021 12:58:03 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When the loose_object_info() function returns an error stop faking up the "oi->typep" to OBJ_BAD. Let the return value of the function itself suffice. This code cleanup simplifies subsequent changes. That we set this at all is a relic from the past. Before 052fe5eaca9 (sha1_loose_object_info: make type lookup optional, 2013-07-12) we would always return the type_from_string(type) via the parse_sha1_header() function, or -1 (i.e. OBJ_BAD) if we couldn't parse it. Then in a combination of 46f034483eb (sha1_file: support reading from a loose object of unknown type, 2015-05-03) and b3ea7dd32d6 (sha1_loose_object_info: handle errors from unpack_sha1_rest, 2017-10-05) our API drifted even further towards conflating the two again. Having read the code paths involved carefully I think this is OK. We are just about to return -1, and we have only one caller: do_oid_object_info_extended(). That function will in turn go on to return -1 when we return -1 here. This might be introducing a subtle bug where a caller of oid_object_info_extended() would inspect its "typep" and expect a meaningful value if the function returned -1. Such a problem would not occur for its simpler oid_object_info() sister function. That one always returns the "enum object_type", which in the case of -1 would be the OBJ_BAD. Having read the code for all the callers of these functions I don't believe any such bug is being introduced here, and in any case we'd likely already have such a bug for the "sizep" member (although blindly checking "typep" first would be a more common case). Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/object-file.c b/object-file.c index a8be8994814..bda3497d5ca 100644 --- a/object-file.c +++ b/object-file.c @@ -1503,8 +1503,6 @@ static int loose_object_info(struct repository *r, git_inflate_end(&stream); munmap(map, mapsize); - if (status && oi->typep) - *oi->typep = status; if (oi->sizep == &size_scratch) oi->sizep = NULL; strbuf_release(&hdrbuf); From patchwork Tue Sep 7 10:58:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8353C433EF for ; Tue, 7 Sep 2021 10:58:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C09D5610F8 for ; Tue, 7 Sep 2021 10:58:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343956AbhIGK7x (ORCPT ); Tue, 7 Sep 2021 06:59:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343819AbhIGK7t (ORCPT ); Tue, 7 Sep 2021 06:59:49 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0310DC0617AF for ; Tue, 7 Sep 2021 03:58:33 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id m9so13797774wrb.1 for ; Tue, 07 Sep 2021 03:58:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=b50x62CCPDWTvLOp8Jd/k0/8MoLuR0GKk/efBGc78v0=; b=AlBrzbb5n4DDNUQq1wzlVDH/MhheV7dhrdqKobx+7pxat64SLf4OizpH4DPJIPKaYd FJRVKR0dNfurZ5m0KOvBgYBjsxqyRnzmjc9RLAoyOuwpbgWog4vpt5mwz8/NCBuY7EJN NTcKiR4EnYLmzCS9gYdqWZ+MSi54YwtaZwOR3eyCJ2s7pfYIMvswwfeNBlSJ7/0dpss/ xH4C6rxKQlOXaymmaiqdo/Xxh8+iYujexcBfn/xYpP9nFv0va25o95TI1ivE7i66ywZP l3VhxYywTrrmYVHwMK97AU2mHHwMCskw1WHgboeUBjzXAgvQ9tZgiOUaYIIb7ojkzGW/ fxvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=b50x62CCPDWTvLOp8Jd/k0/8MoLuR0GKk/efBGc78v0=; b=aOBLxvrxav5EdwXqA4fCdjjdzKBIJhd2Lh7Vjoexmh6sh1aFgYT/JVtMRJX7vu58S5 Tn96EXzJdMTpyIXY29pFwauatoP14N6fP1abQMZjcRdE2VatNy/k60Ib651jKXR6IFca X5oBjR0WlsA2FT3wO8Ps3/iXMb9JpuyjNMV4Gp2OU2HlzwLjGLsgFRMAD8ZcnjwaO7U9 5MT6kcxitV2EOB6ETJPFyxsIie2ds6avtRX7a6OgF45If8R5JblFUhdwv2Jf5fQj42+k 4vWq9wKKYKg+NzPqBx+mu0A5VdPsxCgWpEMQYksx0A0MqQiuRgdXnp4S6LIkp4LeT9rZ ixbQ== X-Gm-Message-State: AOAM533FBy0SuljTDDODNSB+8U1E8fG5zdqO2dKV+bKzxJ0LK66Qe8Ne N4/BPfqm0DgHpsIlbrjXsZM5O2kpFmOFtQ== X-Google-Smtp-Source: ABdhPJyp3pT0EjvFT2w+zO1i+d3fKz6HVS/cZ2eoGqA+UBQ0dCcutO8FmCRnVKhF+PrKDtnd/kqyTQ== X-Received: by 2002:adf:efc2:: with SMTP id i2mr18277911wrp.94.1631012312314; Tue, 07 Sep 2021 03:58:32 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:31 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 09/22] cache.h: move object functions to object-store.h Date: Tue, 7 Sep 2021 12:58:04 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move the declaration of some ancient object functions added in e.g. c4483576b8d (Add "unpack_sha1_header()" helper function, 2005-06-01) from cache.h to object-store.h. This continues work started in cbd53a2193d (object-store: move object access functions to object-store.h, 2018-05-15). Signed-off-by: Ævar Arnfjörð Bjarmason --- cache.h | 10 ---------- object-store.h | 9 +++++++++ 2 files changed, 9 insertions(+), 10 deletions(-) diff --git a/cache.h b/cache.h index d23de693680..11a04a93436 100644 --- a/cache.h +++ b/cache.h @@ -1313,16 +1313,6 @@ char *xdg_cache_home(const char *filename); int git_open_cloexec(const char *name, int flags); #define git_open(name) git_open_cloexec(name, O_RDONLY) -int unpack_loose_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz); -int parse_loose_header(const char *hdr, unsigned long *sizep); - -int check_object_signature(struct repository *r, const struct object_id *oid, - void *buf, unsigned long size, const char *type); - -int finalize_object_file(const char *tmpfile, const char *filename); - -/* Helper to check and "touch" a file */ -int check_and_freshen_file(const char *fn, int freshen); extern const signed char hexval_table[256]; static inline unsigned int hexval(unsigned char c) diff --git a/object-store.h b/object-store.h index d24915ced1b..eb4876ec983 100644 --- a/object-store.h +++ b/object-store.h @@ -485,4 +485,13 @@ int for_each_object_in_pack(struct packed_git *p, int for_each_packed_object(each_packed_object_fn, void *, enum for_each_object_flags flags); +int unpack_loose_header(git_zstream *stream, unsigned char *map, + unsigned long mapsize, void *buffer, + unsigned long bufsiz); +int parse_loose_header(const char *hdr, unsigned long *sizep); +int check_object_signature(struct repository *r, const struct object_id *oid, + void *buf, unsigned long size, const char *type); +int finalize_object_file(const char *tmpfile, const char *filename); +int check_and_freshen_file(const char *fn, int freshen); + #endif /* OBJECT_STORE_H */ From patchwork Tue Sep 7 10:58:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3D98C433F5 for ; Tue, 7 Sep 2021 10:58:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9CBBB60F6D for ; Tue, 7 Sep 2021 10:58:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343964AbhIGK7y (ORCPT ); Tue, 7 Sep 2021 06:59:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51086 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343847AbhIGK7t (ORCPT ); Tue, 7 Sep 2021 06:59:49 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A10DC0613A4 for ; Tue, 7 Sep 2021 03:58:35 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id v10so13781799wrd.4 for ; Tue, 07 Sep 2021 03:58:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AMg+jZCaKmvEZeNcC+mhr5NOAfyJxbsOGU6Ug1z/QG4=; b=DPDI/vyQrELGfItk/1KXZpyxu/lw3PgB8anJZAk+KMhbzIyUAV59ZySpiBywHdJiIr YqiQW6YMybiFiFMSpMChqR2WvQ8Nc7OP7az6XpLND5u0fJLxFW2YbG4NKpi4D2qpKh4T ZW5WcAwiDyRKa5oBWksQMX0mnag42/wpKp+LIch6ihBnqkPh863jFULHVlezkHMJshRd JfmhGCmR+8EvuqEqELIqUC65y1sCUVM2m6veE4/gk94qhp/8lqhCEiJJ2Q5CaoFslryf qRY/8IPVGq2SiTz8FyBNulTiiAtSvp7hjongWTyWbDGNpO4zInc4x0diSy1GfcBMHHLi SGDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AMg+jZCaKmvEZeNcC+mhr5NOAfyJxbsOGU6Ug1z/QG4=; b=H6uSUv1NOC7/O8F2s3OgwGK5IWBnudFelnjGTb0qUHBukCltYhCSKvAgNY55d8+/Jf ee1bj+ITseibevMmF7HOJcqjtW+gY+R5P2d70i0Gls+J2qlF3gEY0iFWrk0jV3KYMAOL QwYTGf/I4DG2KMbZShgHLP3ykJciAI1wBrBZo+U7HfIH9951oR9RCPfihiiHgbzlk0SW BAaIsiB32/6mZvSw61mlhxaKGycgD3RPinWgWG77ak0DcjiUadxy2NHMEvipOIRkIlzy 9j9Rqcy2uJpnXi1Cce3TR7pwLWgejDzStFI0N263cXmQlJKCOUp9h8sABCAG7Azu/Bb6 v91g== X-Gm-Message-State: AOAM53369qE97apg2qrxIk80ZfOoGZfO5/VP3WX8h1r0tBNEvb4HZnBW Y/f03hcxigtb+KDpi/xfjlnBBs2OC8CimA== X-Google-Smtp-Source: ABdhPJyLcmUDhNyP+DM0SeGF32+/N01s91zQGxO3rKSHidS93Z2+sHqyAdr2oMa9I2uHS0pTz23K3w== X-Received: by 2002:adf:b741:: with SMTP id n1mr18148484wre.120.1631012313279; Tue, 07 Sep 2021 03:58:33 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:32 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 10/22] object-file.c: make parse_loose_header_extended() public Date: Tue, 7 Sep 2021 12:58:05 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Make the parse_loose_header_extended() function public and remove the parse_loose_header() wrapper. The only direct user of it outside of object-file.c itself was in streaming.c, that caller can simply pass the required "struct object-info *" instead. This change is being done in preparation for teaching read_loose_object() to accept a flag to pass to parse_loose_header(). It isn't strictly necessary for that change, we could simply use parse_loose_header_extended() there, but will leave the API in a better end state. Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 21 ++++++++------------- object-store.h | 3 ++- streaming.c | 5 ++++- 3 files changed, 14 insertions(+), 15 deletions(-) diff --git a/object-file.c b/object-file.c index bda3497d5ca..7a47af68bd8 100644 --- a/object-file.c +++ b/object-file.c @@ -1363,8 +1363,9 @@ static void *unpack_loose_rest(git_zstream *stream, * too permissive for what we want to check. So do an anal * object header parse by hand. */ -static int parse_loose_header_extended(const char *hdr, struct object_info *oi, - unsigned int flags) +int parse_loose_header(const char *hdr, + struct object_info *oi, + unsigned int flags) { const char *type_buf = hdr; unsigned long size; @@ -1424,14 +1425,6 @@ static int parse_loose_header_extended(const char *hdr, struct object_info *oi, return *hdr ? -1 : type; } -int parse_loose_header(const char *hdr, unsigned long *sizep) -{ - struct object_info oi = OBJECT_INFO_INIT; - - oi.sizep = sizep; - return parse_loose_header_extended(hdr, &oi, 0); -} - static int loose_object_info(struct repository *r, const struct object_id *oid, struct object_info *oi, int flags) @@ -1486,10 +1479,10 @@ static int loose_object_info(struct repository *r, if (status < 0) ; /* Do nothing */ else if (hdrbuf.len) { - if ((status = parse_loose_header_extended(hdrbuf.buf, oi, flags)) < 0) + if ((status = parse_loose_header(hdrbuf.buf, oi, flags)) < 0) status = error(_("unable to parse %s header with --allow-unknown-type"), oid_to_hex(oid)); - } else if ((status = parse_loose_header_extended(hdr, oi, flags)) < 0) + } else if ((status = parse_loose_header(hdr, oi, flags)) < 0) status = error(_("unable to parse %s header"), oid_to_hex(oid)); if (status >= 0 && oi->contentp) { @@ -2573,6 +2566,8 @@ int read_loose_object(const char *path, unsigned long mapsize; git_zstream stream; char hdr[MAX_HEADER_LEN]; + struct object_info oi = OBJECT_INFO_INIT; + oi.sizep = size; *contents = NULL; @@ -2587,7 +2582,7 @@ int read_loose_object(const char *path, goto out; } - *type = parse_loose_header(hdr, size); + *type = parse_loose_header(hdr, &oi, 0); if (*type < 0) { error(_("unable to parse header of %s"), path); git_inflate_end(&stream); diff --git a/object-store.h b/object-store.h index eb4876ec983..25e641a606f 100644 --- a/object-store.h +++ b/object-store.h @@ -488,7 +488,8 @@ int for_each_packed_object(each_packed_object_fn, void *, int unpack_loose_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz); -int parse_loose_header(const char *hdr, unsigned long *sizep); +int parse_loose_header(const char *hdr, struct object_info *oi, + unsigned int flags); int check_object_signature(struct repository *r, const struct object_id *oid, void *buf, unsigned long size, const char *type); int finalize_object_file(const char *tmpfile, const char *filename); diff --git a/streaming.c b/streaming.c index 5f480ad50c4..8beac62cbb7 100644 --- a/streaming.c +++ b/streaming.c @@ -223,6 +223,9 @@ static int open_istream_loose(struct git_istream *st, struct repository *r, const struct object_id *oid, enum object_type *type) { + struct object_info oi = OBJECT_INFO_INIT; + oi.sizep = &st->size; + st->u.loose.mapped = map_loose_object(r, oid, &st->u.loose.mapsize); if (!st->u.loose.mapped) return -1; @@ -231,7 +234,7 @@ static int open_istream_loose(struct git_istream *st, struct repository *r, st->u.loose.mapsize, st->u.loose.hdr, sizeof(st->u.loose.hdr)) < 0) || - (parse_loose_header(st->u.loose.hdr, &st->size) < 0)) { + (parse_loose_header(st->u.loose.hdr, &oi, 0) < 0)) { git_inflate_end(&st->z); munmap(st->u.loose.mapped, st->u.loose.mapsize); return -1; From patchwork Tue Sep 7 10:58:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478313 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C213EC433F5 for ; Tue, 7 Sep 2021 10:58:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AB9B360F6D for ; Tue, 7 Sep 2021 10:58:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343968AbhIGK7z (ORCPT ); Tue, 7 Sep 2021 06:59:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51088 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343844AbhIGK7t (ORCPT ); Tue, 7 Sep 2021 06:59:49 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10DEBC0612E7 for ; Tue, 7 Sep 2021 03:58:36 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id v20-20020a1cf714000000b002e71f4d2026so1560672wmh.1 for ; Tue, 07 Sep 2021 03:58:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5P3kh/X9HhslBfcGEKe9JwBUL1cFukw7pG36VBtTsBo=; b=S4d2CcMdhGf/NE7TCAl8da06FOuTNZgDRe5AgM0iUtkZAS20BzazvbFdepCYYr2sVT NsDUegI7xfvs1BON4nyB49WjK6JtFLDDxI5ikiYM6Z8JFJDJJAy1jrBFsowfmdbs5HbO QkkFS7tO7RoTSVuKTZueW+FFvuDe30Nu6n3BuyyTFSv6q9TPQcb6QtWIP7DCqGNOw+CC ZWS6Z8OmH2LUYIKsQbUtXsy1YAbwmfHony+j82USRgFPkbmkXMrIvu8OX0TYJf/+UBr8 yzUeU2p/VzRtcy6Oi9PKQ9iRwT1qMLvMs1ptP2gQDGoyLzvl45X/G0eAlM9BqviZJdV2 lFXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5P3kh/X9HhslBfcGEKe9JwBUL1cFukw7pG36VBtTsBo=; b=DIoaHNAN/w3wVSK15PMBCMhG3ShCkuJiaFzDeksdktKFvb8tZxTugosP4PZLhdvH4i V8tHPOXMWxJoZuMpwq8Chxsfvvv/nhSij751ZefglIEW0f3KSP9XLLY+v3JRPBry3zA0 j8BY9oWAQFs00toDpQPa8Xmy2Tp+4XEpF39KxPnPFQBkfSmJvBzTs5tcw26fkMuazUsG TVZfBFq0LNfYNMOgPBYPvCpX4lZhPsL76Fr3Rkn59bM4STMRuFxXlE/N7f5l9voBE5tB A30c/6XMcrQUDad8SRUWl7WlQnQ394hhWlWdt+2wmr0crcZ4UV1TXufCNfHhyL9/lVsg BDcA== X-Gm-Message-State: AOAM533gXLKUVhbo9Pc+E5om7dQVXAWOut0krXFz29/on6Zz29Vban5w XnyQ2Euc+XJkJWGA/kgzgzLSU0LwDTVYjA== X-Google-Smtp-Source: ABdhPJylC07Tvya4jc/Vdaq0RPMerdoPY79wUbtYoeT8dwGVnl1/+B2pCpwLL15yQZfHDS0ptOI2fQ== X-Received: by 2002:a1c:c911:: with SMTP id f17mr3266276wmb.190.1631012314325; Tue, 07 Sep 2021 03:58:34 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:33 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 11/22] object-file.c: add missing braces to loose_object_info() Date: Tue, 7 Sep 2021 12:58:06 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the formatting in loose_object_info() to conform with our usual coding style: When there are multiple arms to a conditional and some of them require braces, enclose even a single line block in braces for consistency -- Documentation/CodingGuidelines This formatting-only change makes a subsequent commit easier to read. Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/object-file.c b/object-file.c index 7a47af68bd8..878a4298c9b 100644 --- a/object-file.c +++ b/object-file.c @@ -1473,17 +1473,20 @@ static int loose_object_info(struct repository *r, if (unpack_loose_header_to_strbuf(&stream, map, mapsize, hdr, sizeof(hdr), &hdrbuf) < 0) status = error(_("unable to unpack %s header with --allow-unknown-type"), oid_to_hex(oid)); - } else if (unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0) + } else if (unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0) { status = error(_("unable to unpack %s header"), oid_to_hex(oid)); - if (status < 0) - ; /* Do nothing */ - else if (hdrbuf.len) { + } + + if (status < 0) { + /* Do nothing */ + } else if (hdrbuf.len) { if ((status = parse_loose_header(hdrbuf.buf, oi, flags)) < 0) status = error(_("unable to parse %s header with --allow-unknown-type"), oid_to_hex(oid)); - } else if ((status = parse_loose_header(hdr, oi, flags)) < 0) + } else if ((status = parse_loose_header(hdr, oi, flags)) < 0) { status = error(_("unable to parse %s header"), oid_to_hex(oid)); + } if (status >= 0 && oi->contentp) { *oi->contentp = unpack_loose_rest(&stream, hdr, @@ -1492,8 +1495,9 @@ static int loose_object_info(struct repository *r, git_inflate_end(&stream); status = -1; } - } else + } else { git_inflate_end(&stream); + } munmap(map, mapsize); if (oi->sizep == &size_scratch) From patchwork Tue Sep 7 10:58:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BEA0C433EF for ; Tue, 7 Sep 2021 10:58:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6459760F6D for ; Tue, 7 Sep 2021 10:58:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343977AbhIGK75 (ORCPT ); Tue, 7 Sep 2021 06:59:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51090 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343885AbhIGK7u (ORCPT ); Tue, 7 Sep 2021 06:59:50 -0400 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7D70C061292 for ; Tue, 7 Sep 2021 03:58:37 -0700 (PDT) Received: by mail-wm1-x333.google.com with SMTP id y132so864966wmc.1 for ; Tue, 07 Sep 2021 03:58:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sWFQ5/BD6J33/tYVQ1i8TEJpH5JWz6Iu4PvPfTFeTFM=; b=k40fAp786HS9RaA7gKlpnkOKfzlb7t48rBYSOCnuPw2OZzUs7WO2WysJIqHBdwFTJf VQ9b4DAulVQrlZVTZ83t5cwiOA2s2J4pVjDR2No10FWG6Y5NpIL0D1O2y7W7Atz3pHNu MY8AoXqN6SVMo1h2/m9ry1ccCiKU9EUniEdqHisCQ7exalYzQkLUiN/AwR6roj0ledzN l17etwfCjMdXNjALmVE0h/+yxvXYfUBlqJALMYehmsH3l7yGBBKtK11UvpFPi5txk8AE wCiulReiNt8xNfbJdHL7+B+c5G5LvTiDdhxypOlDar1psFY4pugCe8+yED8QvKL59qCB 42tA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sWFQ5/BD6J33/tYVQ1i8TEJpH5JWz6Iu4PvPfTFeTFM=; b=CHPUmvcANM4dlMNsBnfTzLQ6P/KmO4q2uMqTYnxo7EVql6LU3dMQKeD8PUV0Vay1EH 5YqokViKREN8Upo0mGHP8ntAyZ/hipOvrwibOg3X58hqhlBqEhWMnwnuJI1yoMsJWYLQ 7D5euymFYrSwYJmhjUpgDP+v3xc3FCzeLc6ZINamME75MhUOCCeQdXtuKYCQaW9O8/Zq QBaMHGrrSnj/m3x0GA6qySb4+Q5eaKOih7eernQL8BAUlB5OIBJqXEJXvvzA+/d/rP35 Sp9qdvXvfwyZecJCOfUgU5C7jCnrxxELKbkBFu3CaHhV19NjzQoOgBEk3dPtg7ko1itM ncVQ== X-Gm-Message-State: AOAM533BFkYI9BKJN5d2KuLh5C0RkbW54hZVS/+o+AnEF7Ix3K0EnRbJ PnRNRI4ixmkWje+Xcyy0GvRctXObDraljA== X-Google-Smtp-Source: ABdhPJxM11OHup+Qixr+PuLqFbMN0zxORzW+48fEMdxiUNHLzDLZO39gysm0AlDgEgd4rGV+hfOlww== X-Received: by 2002:a05:600c:221a:: with SMTP id z26mr3307435wml.142.1631012315433; Tue, 07 Sep 2021 03:58:35 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:34 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 12/22] object-file.c: simplify unpack_loose_short_header() Date: Tue, 7 Sep 2021 12:58:07 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Combine the unpack_loose_short_header(), unpack_loose_header_to_strbuf() and unpack_loose_header() functions into one. The unpack_loose_header_to_strbuf() function was added in 46f034483eb (sha1_file: support reading from a loose object of unknown type, 2015-05-03). Its code was mostly copy/pasted between it and both of unpack_loose_header() and unpack_loose_short_header(). We now have a single unpack_loose_header() function which accepts an optional "struct strbuf *" instead. I think the remaining unpack_loose_header() function could be further simplified, we're carrying some complexity just to be able to emit a garbage type longer than MAX_HEADER_LEN, we could alternatively just say "we found a garbage type ..." instead. But let's leave the current behavior in place for now. Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 60 ++++++++++++++++++-------------------------------- object-store.h | 14 +++++++++++- streaming.c | 3 ++- 3 files changed, 37 insertions(+), 40 deletions(-) diff --git a/object-file.c b/object-file.c index 878a4298c9b..2dd4cdd1ae0 100644 --- a/object-file.c +++ b/object-file.c @@ -1233,11 +1233,12 @@ void *map_loose_object(struct repository *r, return map_loose_object_1(r, NULL, oid, size); } -static int unpack_loose_short_header(git_zstream *stream, - unsigned char *map, unsigned long mapsize, - void *buffer, unsigned long bufsiz) +int unpack_loose_header(git_zstream *stream, + unsigned char *map, unsigned long mapsize, + void *buffer, unsigned long bufsiz, + struct strbuf *header) { - int ret; + int status; /* Get the data stream */ memset(stream, 0, sizeof(*stream)); @@ -1248,44 +1249,25 @@ static int unpack_loose_short_header(git_zstream *stream, git_inflate_init(stream); obj_read_unlock(); - ret = git_inflate(stream, 0); + status = git_inflate(stream, 0); obj_read_lock(); - - return ret; -} - -int unpack_loose_header(git_zstream *stream, - unsigned char *map, unsigned long mapsize, - void *buffer, unsigned long bufsiz) -{ - int status = unpack_loose_short_header(stream, map, mapsize, - buffer, bufsiz); - if (status < Z_OK) return status; - /* Make sure we have the terminating NUL */ - if (!memchr(buffer, '\0', stream->next_out - (unsigned char *)buffer)) - return -1; - return 0; -} - -static int unpack_loose_header_to_strbuf(git_zstream *stream, unsigned char *map, - unsigned long mapsize, void *buffer, - unsigned long bufsiz, struct strbuf *header) -{ - int status; - - status = unpack_loose_short_header(stream, map, mapsize, buffer, bufsiz); - if (status < Z_OK) - return -1; - /* * Check if entire header is unpacked in the first iteration. */ if (memchr(buffer, '\0', stream->next_out - (unsigned char *)buffer)) return 0; + /* + * We have a header longer than MAX_HEADER_LEN. The "header" + * here is only non-NULL when we run "cat-file + * --allow-unknown-type". + */ + if (!header) + return -1; + /* * buffer[0..bufsiz] was not large enough. Copy the partial * result out to header, and then append the result of further @@ -1433,9 +1415,11 @@ static int loose_object_info(struct repository *r, unsigned long mapsize; void *map; git_zstream stream; + int hdr_ret; char hdr[MAX_HEADER_LEN]; struct strbuf hdrbuf = STRBUF_INIT; unsigned long size_scratch; + int allow_unknown = flags & OBJECT_INFO_ALLOW_UNKNOWN_TYPE; if (oi->delta_base_oid) oidclr(oi->delta_base_oid); @@ -1469,11 +1453,10 @@ static int loose_object_info(struct repository *r, if (oi->disk_sizep) *oi->disk_sizep = mapsize; - if ((flags & OBJECT_INFO_ALLOW_UNKNOWN_TYPE)) { - if (unpack_loose_header_to_strbuf(&stream, map, mapsize, hdr, sizeof(hdr), &hdrbuf) < 0) - status = error(_("unable to unpack %s header with --allow-unknown-type"), - oid_to_hex(oid)); - } else if (unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0) { + + hdr_ret = unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr), + allow_unknown ? &hdrbuf : NULL); + if (hdr_ret < 0) { status = error(_("unable to unpack %s header"), oid_to_hex(oid)); } @@ -2581,7 +2564,8 @@ int read_loose_object(const char *path, goto out; } - if (unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0) { + if (unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr), + NULL) < 0) { error(_("unable to unpack header of %s"), path); goto out; } diff --git a/object-store.h b/object-store.h index 25e641a606f..4064710ae29 100644 --- a/object-store.h +++ b/object-store.h @@ -485,9 +485,21 @@ int for_each_object_in_pack(struct packed_git *p, int for_each_packed_object(each_packed_object_fn, void *, enum for_each_object_flags flags); +/** + * unpack_loose_header() initializes the data stream needed to unpack + * a loose object header. + * + * Returns 0 on success. Returns negative values on error. + * + * It will only parse up to MAX_HEADER_LEN bytes unless an optional + * "hdrbuf" argument is non-NULL. This is intended for use with + * OBJECT_INFO_ALLOW_UNKNOWN_TYPE to extract the bad type for (error) + * reporting. The full header will be extracted to "hdrbuf" for use + * with parse_loose_header(). + */ int unpack_loose_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, - unsigned long bufsiz); + unsigned long bufsiz, struct strbuf *hdrbuf); int parse_loose_header(const char *hdr, struct object_info *oi, unsigned int flags); int check_object_signature(struct repository *r, const struct object_id *oid, diff --git a/streaming.c b/streaming.c index 8beac62cbb7..cb3c3cf6ff6 100644 --- a/streaming.c +++ b/streaming.c @@ -233,7 +233,8 @@ static int open_istream_loose(struct git_istream *st, struct repository *r, st->u.loose.mapped, st->u.loose.mapsize, st->u.loose.hdr, - sizeof(st->u.loose.hdr)) < 0) || + sizeof(st->u.loose.hdr), + NULL) < 0) || (parse_loose_header(st->u.loose.hdr, &oi, 0) < 0)) { git_inflate_end(&st->z); munmap(st->u.loose.mapped, st->u.loose.mapsize); From patchwork Tue Sep 7 10:58:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAFF6C433F5 for ; Tue, 7 Sep 2021 10:58:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AEBEC60F6D for ; Tue, 7 Sep 2021 10:58:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344007AbhIGLAA (ORCPT ); Tue, 7 Sep 2021 07:00:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343888AbhIGK7u (ORCPT ); Tue, 7 Sep 2021 06:59:50 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 363C4C06129E for ; Tue, 7 Sep 2021 03:58:38 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id c8-20020a7bc008000000b002e6e462e95fso1472525wmb.2 for ; Tue, 07 Sep 2021 03:58:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FKA2gN+AKTbQHqM2l0D4ZakKrU9BY5OaxyIn9EApBzo=; b=Jodm8SGbRYS7lecVlhlCRKcXeoWLvoJF7MEJQf4inlUcAZ9igX3EnHj8VHP5EKPsOu Sm2R8DZtNgx7/VR9hDMk0LCAzG5/lQPHQr05VSxTgQomVWWNMfacRqWbTLXQeyLMKUBu ZDAPiwBEd8QqAnW2/QWp9sMz/RQZo78swfNOwGYbXXRDXy5ceOkdeQTfk4xm0OSDaD3l hYfCBJs3G+fyPvGKyIWwLWYh2+CsNM4FxNFR/PKZz7jnofPqqatmCf+DHScoAVtT2i1S mUPsbcfXlRrYhqPJB1XSrBM3N3Crl/8U0QYCOcvxnKNlZCPeKqMFclMYKXCufhrH60Jk Zrsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FKA2gN+AKTbQHqM2l0D4ZakKrU9BY5OaxyIn9EApBzo=; b=CBquZfQBo+PePk3fxoyHIH2YAEnAfHg/KluBzPAPBqDPDL2XvgLNTx0CywfXopxJ1v K/2tdVVRlCtBZOuWiJrAbSi5xNqemaOU2O1zYSPxjFSNRVsoFTPCMTS2r6M9e5eM+JzQ cqnSKfDWuy4Cr6/40qTdlYwJ3JPlswuXG8OlHIfa9fFY3WZ95eFA8c4s+8V+/zorL9C5 iml8yYTTmzAhgRTdm6HJr1App6X2tVCn5Qc4d/he9U28LhMJ90O0VQSoIpTy6hti71ZR khNFvSHoCu6T5EUOZnp8gc4yFwQakPFdf7CDZQLnG+CWnCAHyVSkUA5PpfBFY57mcgWA 3Ltg== X-Gm-Message-State: AOAM530kxK21loi/xZCCs/6PWdPiAs1X1XDtBj9psaTC6nbutBqQha5w 4OYEpolrhjndEOu1DhDfuNlb1ogJbkIZLA== X-Google-Smtp-Source: ABdhPJyGrHxJLavo5y7RE2Vg2B9CICewL6vubMqs9fb9R8c6CgCd7iMcUseHgRxdmt/IXszWy+VeUA== X-Received: by 2002:a1c:9d50:: with SMTP id g77mr3312881wme.67.1631012316382; Tue, 07 Sep 2021 03:58:36 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:35 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 13/22] object-file.c: split up ternary in parse_loose_header() Date: Tue, 7 Sep 2021 12:58:08 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org This minor formatting change serves to make a subsequent patch easier to read. Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/object-file.c b/object-file.c index 2dd4cdd1ae0..7c6a865a6c0 100644 --- a/object-file.c +++ b/object-file.c @@ -1404,7 +1404,10 @@ int parse_loose_header(const char *hdr, /* * The length must be followed by a zero byte */ - return *hdr ? -1 : type; + if (*hdr) + return -1; + + return type; } static int loose_object_info(struct repository *r, From patchwork Tue Sep 7 10:58:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4D83C433EF for ; Tue, 7 Sep 2021 10:58:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A144060F6D for ; Tue, 7 Sep 2021 10:58:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343986AbhIGK76 (ORCPT ); Tue, 7 Sep 2021 06:59:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343889AbhIGK7u (ORCPT ); Tue, 7 Sep 2021 06:59:50 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C195C06129F for ; Tue, 7 Sep 2021 03:58:39 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id u19-20020a7bc053000000b002f8d045b2caso820297wmc.1 for ; Tue, 07 Sep 2021 03:58:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mKPGQcEdxSHXpNCwGotG/qh6jyDtSZlEEym+uPKc4nk=; b=eHMXFnjzQACNgnFqrJdtAtO738FuV39q81ht4Hlu30Wn1Gh59cjpkLYsn7j9A/aR6x 3w6QRFybfjnpfhIBeTXvlWb6AUehiNu8jn3gyOzZ5Qe9DKkhMqpawfwhza7wQwar4DRc GxDWOiuzcpmguXL6lG7IogyEWwtCWgMHlWULphmxrEgwZhXDMHtx8n1nKrnJAIkjxKfS kZfETfirB+vhrQOAd7M6tKwFcnAl1muj/aV2h4EGH68Bf4iBM/DAM20JYE1LtFnZNDRR vSrV8+///J4d99MqsDTxbhBO5k/uBwnbCC1kKNG0hsxhc0t9dEGZL9XSMjuWUG6CnR/O qF4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mKPGQcEdxSHXpNCwGotG/qh6jyDtSZlEEym+uPKc4nk=; b=lJi4qm9fPHJGQ1uPfVgCcj3uSMuZDeBMBOX56TJHZx0kpEGBYgzba1VDeRS5VjXPgt gyyU4eIz1Bd3KVFQyEuRDqk9mWcPkhYmIwb8wMGC3LyIQ73QWvwTuaROXhPm30oPMof7 VVsvUH/r5OUxbY+UbKlYKoTDLsDtW4y5btMEQOcgvnd71tVQPlCjuY4EovHMQGkB5GU3 9eRM0xzX5lJHAa8UaHQzsuC0ZXzhv+SxxLyucGqfh8V0MIDDygWwuizOkxK6VL8VM1aX keFR8zk2dZD5lEb3AV528UL3kdgTi4aJtdYXR4bVtzBuy36UTS8Uc6rrIyBvyTc5/GUZ l1Uw== X-Gm-Message-State: AOAM530K1M2OEPg1syJKeEgpwfRnY9f+I0Y9vjCps7tCA4Rs1CzVZQwM A1MXNY8sT1GUVFipTKLnlbomeLVUWoZnmQ== X-Google-Smtp-Source: ABdhPJzk+uVEpkaJH99MuS9FF63zzu2tUvTOg1uQZn89VK2STyAZchEKtgtaDTbm09dHFGtaHumeTw== X-Received: by 2002:a05:600c:3542:: with SMTP id i2mr3265972wmq.170.1631012317333; Tue, 07 Sep 2021 03:58:37 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:36 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 14/22] object-file.c: stop dying in parse_loose_header() Date: Tue, 7 Sep 2021 12:58:09 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Start the libification of parse_loose_header() by making it return error codes and data instead of invoking die() by itself. For now we'll move the relevant die() call to loose_object_info() and read_loose_object() to keep this change smaller, but in subsequent commits we'll also libify those. Since the refactoring of parse_loose_header_extended() into parse_loose_header() in an earlier commit, its interface accepts a "unsigned long *sizep". Rather it accepts a "struct object_info *", that structure will be populated with information about the object. It thus makes sense to further libify the interface so that it stops calling die() when it encounters OBJ_BAD, and instead rely on its callers to check the populated "oi->typep". Because of this we don't need to pass in the "unsigned int flags" which we used for OBJECT_INFO_ALLOW_UNKNOWN_TYPE, we can instead do that check in loose_object_info(). This also refactors some confusing control flow around the "status" variable. In some cases we set it to the return value of "error()", i.e. -1, and later checked if "status < 0" was true. In another case added in c84a1f3ed4d (sha1_file: refactor read_object, 2017-06-21) (but the behavior pre-dated that) we did checks of "status >= 0", because at that point "status" had become the return value of parse_loose_header(). I.e. a non-negative "enum object_type" (unless we -1, aka. OBJ_BAD). Now that parse_loose_header() will return 0 on success instead of the type (which it'll stick into the "struct object_info") we don't need to conflate these two cases in its callers. Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 53 ++++++++++++++++++++++++++------------------------ object-store.h | 13 +++++++++++-- streaming.c | 4 +++- 3 files changed, 42 insertions(+), 28 deletions(-) diff --git a/object-file.c b/object-file.c index 7c6a865a6c0..d656960422d 100644 --- a/object-file.c +++ b/object-file.c @@ -1345,9 +1345,7 @@ static void *unpack_loose_rest(git_zstream *stream, * too permissive for what we want to check. So do an anal * object header parse by hand. */ -int parse_loose_header(const char *hdr, - struct object_info *oi, - unsigned int flags) +int parse_loose_header(const char *hdr, struct object_info *oi) { const char *type_buf = hdr; unsigned long size; @@ -1369,15 +1367,6 @@ int parse_loose_header(const char *hdr, type = type_from_string_gently(type_buf, type_len, 1); if (oi->type_name) strbuf_add(oi->type_name, type_buf, type_len); - /* - * Set type to 0 if its an unknown object and - * we're obtaining the type using '--allow-unknown-type' - * option. - */ - if ((flags & OBJECT_INFO_ALLOW_UNKNOWN_TYPE) && (type < 0)) - type = 0; - else if (type < 0) - die(_("invalid object type")); if (oi->typep) *oi->typep = type; @@ -1407,7 +1396,11 @@ int parse_loose_header(const char *hdr, if (*hdr) return -1; - return type; + /* + * The format is valid, but the type may still be bogus. The + * Caller needs to check its oi->typep. + */ + return 0; } static int loose_object_info(struct repository *r, @@ -1422,6 +1415,8 @@ static int loose_object_info(struct repository *r, char hdr[MAX_HEADER_LEN]; struct strbuf hdrbuf = STRBUF_INIT; unsigned long size_scratch; + enum object_type type_scratch; + int parsed_header = 0; int allow_unknown = flags & OBJECT_INFO_ALLOW_UNKNOWN_TYPE; if (oi->delta_base_oid) @@ -1453,6 +1448,8 @@ static int loose_object_info(struct repository *r, if (!oi->sizep) oi->sizep = &size_scratch; + if (!oi->typep) + oi->typep = &type_scratch; if (oi->disk_sizep) *oi->disk_sizep = mapsize; @@ -1463,18 +1460,20 @@ static int loose_object_info(struct repository *r, status = error(_("unable to unpack %s header"), oid_to_hex(oid)); } - - if (status < 0) { - /* Do nothing */ - } else if (hdrbuf.len) { - if ((status = parse_loose_header(hdrbuf.buf, oi, flags)) < 0) - status = error(_("unable to parse %s header with --allow-unknown-type"), - oid_to_hex(oid)); - } else if ((status = parse_loose_header(hdr, oi, flags)) < 0) { - status = error(_("unable to parse %s header"), oid_to_hex(oid)); + if (!status) { + if (!parse_loose_header(hdrbuf.len ? hdrbuf.buf : hdr, oi)) + /* + * oi->{sizep,typep} are meaningless unless + * parse_loose_header() returns >= 0. + */ + parsed_header = 1; + else + status = error(_("unable to parse %s header"), oid_to_hex(oid)); } + if (!allow_unknown && parsed_header && *oi->typep < 0) + die(_("invalid object type")); - if (status >= 0 && oi->contentp) { + if (parsed_header && oi->contentp) { *oi->contentp = unpack_loose_rest(&stream, hdr, *oi->sizep, oid); if (!*oi->contentp) { @@ -1489,6 +1488,8 @@ static int loose_object_info(struct repository *r, if (oi->sizep == &size_scratch) oi->sizep = NULL; strbuf_release(&hdrbuf); + if (oi->typep == &type_scratch) + oi->typep = NULL; oi->whence = OI_LOOSE; return (status < 0) ? status : 0; } @@ -2557,6 +2558,7 @@ int read_loose_object(const char *path, git_zstream stream; char hdr[MAX_HEADER_LEN]; struct object_info oi = OBJECT_INFO_INIT; + oi.typep = type; oi.sizep = size; *contents = NULL; @@ -2573,12 +2575,13 @@ int read_loose_object(const char *path, goto out; } - *type = parse_loose_header(hdr, &oi, 0); - if (*type < 0) { + if (parse_loose_header(hdr, &oi) < 0) { error(_("unable to parse header of %s"), path); git_inflate_end(&stream); goto out; } + if (*type < 0) + die(_("invalid object type")); if (*type == OBJ_BLOB && *size > big_file_threshold) { if (check_stream_oid(&stream, hdr, *size, path, expected_oid) < 0) diff --git a/object-store.h b/object-store.h index 4064710ae29..584bf5556af 100644 --- a/object-store.h +++ b/object-store.h @@ -500,8 +500,17 @@ int for_each_packed_object(each_packed_object_fn, void *, int unpack_loose_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz, struct strbuf *hdrbuf); -int parse_loose_header(const char *hdr, struct object_info *oi, - unsigned int flags); + +/** + * parse_loose_header() parses the starting " \0" of an + * object. If it doesn't follow that format -1 is returned. To check + * the validity of the populate the "typep" in the "struct + * object_info". It will be OBJ_BAD if the object type is unknown. The + * parsed can be retrieved via "oi->sizep", and from there + * passed to unpack_loose_rest(). + */ +int parse_loose_header(const char *hdr, struct object_info *oi); + int check_object_signature(struct repository *r, const struct object_id *oid, void *buf, unsigned long size, const char *type); int finalize_object_file(const char *tmpfile, const char *filename); diff --git a/streaming.c b/streaming.c index cb3c3cf6ff6..c3dc241d6a5 100644 --- a/streaming.c +++ b/streaming.c @@ -225,6 +225,7 @@ static int open_istream_loose(struct git_istream *st, struct repository *r, { struct object_info oi = OBJECT_INFO_INIT; oi.sizep = &st->size; + oi.typep = type; st->u.loose.mapped = map_loose_object(r, oid, &st->u.loose.mapsize); if (!st->u.loose.mapped) @@ -235,7 +236,8 @@ static int open_istream_loose(struct git_istream *st, struct repository *r, st->u.loose.hdr, sizeof(st->u.loose.hdr), NULL) < 0) || - (parse_loose_header(st->u.loose.hdr, &oi, 0) < 0)) { + (parse_loose_header(st->u.loose.hdr, &oi) < 0) || + *type < 0) { git_inflate_end(&st->z); munmap(st->u.loose.mapped, st->u.loose.mapsize); return -1; From patchwork Tue Sep 7 10:58:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B11BAC433EF for ; Tue, 7 Sep 2021 10:59:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 99B1660F6D for ; Tue, 7 Sep 2021 10:59:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343921AbhIGLAF (ORCPT ); Tue, 7 Sep 2021 07:00:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343896AbhIGK7u (ORCPT ); Tue, 7 Sep 2021 06:59:50 -0400 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 00DA9C0612A5 for ; Tue, 7 Sep 2021 03:58:39 -0700 (PDT) Received: by mail-wm1-x332.google.com with SMTP id k5-20020a05600c1c8500b002f76c42214bso1462490wms.3 for ; Tue, 07 Sep 2021 03:58:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L+Fg1O4GBlWOOuoTj+YWVJcfh8FfRPMY1UToyXwX/vY=; b=nnEuUyVZ4pek3VXCj1r2ydHq0A5Q7yGMH5pqMo2KkPoh6BdCC8ZcMlqpxFIpG76w5u YcqpC8ij3SheQF8gDbsWHCbskERIS23fmsi2cby/vm1QJglyoBDmWayNfUSdD0KFdMlN TbA6gz/MEP8VOlwWDBddTHvFuDT7Oh+KuouQcm+/1Gn2yeeChOm4R8OQbh1DM+TiQ8bb rDsLr0NoR8L5YbJ4eZDhy5elb/garVipG+i3PxhMyMCiXK65HAPfyoBAq/JhloycwZHw M0hTFtwnh2MGzsJtcThpfD+nK0CVnN/xpMoEnJ7Ht9m9404HxHBSbeYyMt8AzFHTgZ2A gZsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L+Fg1O4GBlWOOuoTj+YWVJcfh8FfRPMY1UToyXwX/vY=; b=NYiyWo54jBXClK3yZ/NwO4HHfAEQXrg/IRq1Qm5EADBVtae6a4SjFfzVJTsGfzVCyc /5TjloS0bg8Rnjx48EElbv9CHnbJn6ZWsHz1sDCef/X93O8IRJ+HSfXUWfdWjdlpaloB 7LM8aF29ddw9GUHIMxfdXDpNAFAGEyG/nZiKwfvbw83RJe3Zl7azBQpqGkXLbWup/D+z 0MpmJS2T1lYxYSQpgxUjFxt4ysvYJxC4Ce4M4jJg3iE4I4+ylvYU6sVSkZVmwstkvgIl y79zN2M8mnjjhWJ39Py7i9E9A3eTQzj2QjxeoP4Gh3QS1kur1Wb1bkfuNS9JTeaLokDX jTRA== X-Gm-Message-State: AOAM530hpsQMBHOGhRz32qCVd8hpQyXIuMPFHBFauRDPipW9FVjYjI2S +nUD0cBcJoz8KhKJlJJECte193tjBtY99Q== X-Google-Smtp-Source: ABdhPJzimwIXTrxVjE8YuHAbhLxgL/rBPb2njGSar7NswPYKtYa8/dOls42EUlcOQjbsQzbx1pTkrA== X-Received: by 2002:a1c:730a:: with SMTP id d10mr3348571wmb.59.1631012318182; Tue, 07 Sep 2021 03:58:38 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:37 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 15/22] object-file.c: guard against future bugs in loose_object_info() Date: Tue, 7 Sep 2021 12:58:10 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org An earlier version of the preceding commit had a subtle bug where our "type_scratch" (later assigned to "oi->typep") would be uninitialized and used in the "!allow_unknown" case, at which point it would contain a nonsensical value if we'd failed to call parse_loose_header(). The preceding commit introduced "parsed_header" variable to check for this case, but I think we can do better, let's carry a "oi_header" variable initially set to NULL, and only set it to "oi" once we're past parse_loose_header(). This is functionally the same thing, but hopefully makes it even more obvious in the future that we must not access the "typep" and "sizep" (or "type_name") unless parse_loose_header() succeeds, but that accessing other fields set earlier (such as the "disk_sizep" set earlier) is OK. Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/object-file.c b/object-file.c index d656960422d..ae6a37ab5fb 100644 --- a/object-file.c +++ b/object-file.c @@ -1416,7 +1416,7 @@ static int loose_object_info(struct repository *r, struct strbuf hdrbuf = STRBUF_INIT; unsigned long size_scratch; enum object_type type_scratch; - int parsed_header = 0; + struct object_info *oi_header = NULL; int allow_unknown = flags & OBJECT_INFO_ALLOW_UNKNOWN_TYPE; if (oi->delta_base_oid) @@ -1464,18 +1464,20 @@ static int loose_object_info(struct repository *r, if (!parse_loose_header(hdrbuf.len ? hdrbuf.buf : hdr, oi)) /* * oi->{sizep,typep} are meaningless unless - * parse_loose_header() returns >= 0. + * parse_loose_header() returns >= 0. Let's + * access them as "oi_header" (just an alias + * for "oi") below to make that intent clear. */ - parsed_header = 1; + oi_header = oi; else status = error(_("unable to parse %s header"), oid_to_hex(oid)); } - if (!allow_unknown && parsed_header && *oi->typep < 0) + if (!allow_unknown && oi_header && *oi_header->typep < 0) die(_("invalid object type")); - if (parsed_header && oi->contentp) { + if (oi_header && oi->contentp) { *oi->contentp = unpack_loose_rest(&stream, hdr, - *oi->sizep, oid); + *oi_header->sizep, oid); if (!*oi->contentp) { git_inflate_end(&stream); status = -1; From patchwork Tue Sep 7 10:58:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82F87C433FE for ; Tue, 7 Sep 2021 10:58:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 65E2660E52 for ; Tue, 7 Sep 2021 10:58:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343888AbhIGLAC (ORCPT ); Tue, 7 Sep 2021 07:00:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51064 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343895AbhIGK7u (ORCPT ); Tue, 7 Sep 2021 06:59:50 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B31C4C061575 for ; Tue, 7 Sep 2021 03:58:40 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id z4so13752071wrr.6 for ; Tue, 07 Sep 2021 03:58:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zglVV0rGpFX+1s8lU+HeNKkRQ0b2n0f1I1s89JyqSDM=; b=Kf+r9o6YPosjfp8Xi2aX14oiJU9DCGyC26MfBa7sZqHh+jLRtvhS2f1hFLrFMbYztn r2db6fjqCKv0gwZtfPstuort2RGDGFQJD6VjuLy/Elwl1QLyyv2Ocak8kQisYoF2VzO4 +ot/UA7eZMD/NXxC3IMa7w20nlCsSJGwGYt35mNZ8XutPHRbH/JsPBQNIuv+3kRRkE/n T8Ok+IG/ZbhKVmidS9cQMIZlhXCphPIClNQZqtDK4IF78e+vP1jOHuoFSQtdVs/mvMq6 4yQU/ULrs7k3gfO6iXc/MckVMVZdl60nQf1y/VoFc3+XqbYA+MBe5VooWdcqrPxSxJLU ZXBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zglVV0rGpFX+1s8lU+HeNKkRQ0b2n0f1I1s89JyqSDM=; b=dIyJrAUT+5oJ9eowq57KJKpryQ9rsmD7ejaw8HLUcCh/yb0J68aPQEMxpUg6CbEsie tLVYLy4jnSiZ3CTvpkbHSC2ior/hzxfES8BIc81fAlyDHnK/H7F0+Gyn/klHo4U/Euls ed+DbLXoXgZE8FQ8XnbtSOKq+gtaN/x/xeUwhbXxm0T6VQqw9AGyv7f3OdJRR4mHnJex OwU+w11FEvTr5buCdkkccEzA0DdqgXy6bJ7NVk6S0LJkLgOagZF6LRdPslOF+2f7w0J6 irMVdF0GZdoFStyszzIia22PiedTm6xOMkFjsbcum883b4jLpw5dNFmwYHcsJhZZ9mVk h2Iw== X-Gm-Message-State: AOAM532IbvGM0mcJLbXqzuVcO3f7ejMueKYKC72YRHcPaEYjS3tMb+bH xwJ20QQXX6s3SXcliQxsIPLPJznLCji5ww== X-Google-Smtp-Source: ABdhPJzFsspG09FIDiEuSKtvhetLWOtJKcS+IVqRRwT6NIsumQFW+LeM84UqjkZV92dnYFG0CkBcyw== X-Received: by 2002:adf:e0cd:: with SMTP id m13mr18118501wri.137.1631012319109; Tue, 07 Sep 2021 03:58:39 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:38 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 16/22] object-file.c: return -1, not "status" from unpack_loose_header() Date: Tue, 7 Sep 2021 12:58:11 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Return a -1 when git_inflate() fails instead of whatever Z_* status we'd get from zlib.c. This makes no difference to any error we report, but makes it more obvious that we don't care about the specific zlib error codes here. See d21f8426907 (unpack_sha1_header(): detect malformed object header, 2016-09-25) for the commit that added the "return status" code. As far as I can tell there was never a real reason (e.g. different reporting) for carrying down the "status" as opposed to "-1". At the time that d21f8426907 was written there was a corresponding "ret < Z_OK" check right after the unpack_sha1_header() call (the "unpack_sha1_header()" function was later rename to our current "unpack_loose_header()"). However, that check was removed in c84a1f3ed4d (sha1_file: refactor read_object, 2017-06-21) without changing the corresponding return code. So let's do the minor cleanup of also changing this function to return a -1. Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/object-file.c b/object-file.c index ae6a37ab5fb..11df4485147 100644 --- a/object-file.c +++ b/object-file.c @@ -1252,7 +1252,7 @@ int unpack_loose_header(git_zstream *stream, status = git_inflate(stream, 0); obj_read_lock(); if (status < Z_OK) - return status; + return -1; /* * Check if entire header is unpacked in the first iteration. From patchwork Tue Sep 7 10:58:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2AC3C433EF for ; Tue, 7 Sep 2021 10:58:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 99AB1610FE for ; Tue, 7 Sep 2021 10:58:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344021AbhIGLAD (ORCPT ); Tue, 7 Sep 2021 07:00:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51100 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343855AbhIGK7u (ORCPT ); Tue, 7 Sep 2021 06:59:50 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 909B0C06175F for ; Tue, 7 Sep 2021 03:58:41 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id u26-20020a05600c441a00b002f66b2d8603so1458435wmn.4 for ; Tue, 07 Sep 2021 03:58:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=M0Xz7uqRYV/uB+NfjsY69EBlGmMi3A//UBWiDVbV2CY=; b=T7/HcxEDeBqSwgpKvWEFLmB54VRmAEcNGlyrUlUF2B0iYlhGf0uXDeMdIGnZbkiyws 008XldWBmbY46WKB/yjC9gMfjtuqmq3wdNrKh+zygNk7wMB8zE4sjHUAEKwBFlt95eIi S2/YPVsZFD7JvBGWm4NTHuEGRqa07nI8KFiblGvapSQ/dodNre87VWfG+l9aqnIK+A++ EidKV+sIPsmxsrP9ayaUZg0PqHny9ax6F9u3P1oNru77aJ7DfKS84L/Ny0BfVVzat4bH JRy0VA+hiPAn/yj21e1jzk1Oq5IPk1/mFyBlU0gSMcpKLXL1CEMvRgubCYh8mU/4rPu6 pynQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=M0Xz7uqRYV/uB+NfjsY69EBlGmMi3A//UBWiDVbV2CY=; b=bcvyoku2VixlAg8ukt1RMmAF9lKIaU2KRVfWuhSqrKM5bbwgtNHyP8UW+pGgxIRxcC 7QEXBh/5zapgodwoLXv2RUegwqpL000H7kupHgRl06fQa5Z4Gz700CAOp2Abh/Sci3eM 1KQ5EV58kg5qvLo+f4L5HJmPOJQk+5yRkhQoUOjft664csBEIt3BbrZgtsqoafxL9mN2 gC/OttSDvByXiH01ft1zQIsNoDKIh+JhgU0h35BI4v47BD/oXZxvNVAu9L1We2wXJV9a pZeUAULf9+lZ5rQiRbkGwHXcxbXUrzmOxF0eDpFv1QL4mT+O9l9SRH0efLfCME7gNZOi WYzg== X-Gm-Message-State: AOAM532CLlG688Do2pGqs4HmAB3VsGO7O+1hTg0HDuREvXrgYcbQ12Wt 9CJraAHl21yLxz2TwJDPVse+LFgV0+ah2g== X-Google-Smtp-Source: ABdhPJyVfeRX3VD0vqV02UJA1n4vU3nm9MzBTSD4oAaWu8R7/Duzwzc7h5W2stw+bAP4kJrLyYOTgQ== X-Received: by 2002:a1c:7515:: with SMTP id o21mr3310603wmc.150.1631012319898; Tue, 07 Sep 2021 03:58:39 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:39 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 17/22] object-file.c: return -2 on "header too long" in unpack_loose_header() Date: Tue, 7 Sep 2021 12:58:12 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Split up the return code for "header too long" from the generic negative return value unpack_loose_header() returns, and report via error() if we exceed MAX_HEADER_LEN. As a test added earlier in this series in t1006-cat-file.sh shows we'll correctly emit zlib errors from zlib.c already in this case, so we have no need to carry those return codes further down the stack. Let's instead just return -2 saying we ran into the MAX_HEADER_LEN limit, or other negative values for "unable to unpack header". Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 16 +++++++++++++--- object-store.h | 6 ++++-- t/t1006-cat-file.sh | 2 +- 3 files changed, 18 insertions(+), 6 deletions(-) diff --git a/object-file.c b/object-file.c index 11df4485147..0cb5287d3ef 100644 --- a/object-file.c +++ b/object-file.c @@ -1266,7 +1266,7 @@ int unpack_loose_header(git_zstream *stream, * --allow-unknown-type". */ if (!header) - return -1; + return -2; /* * buffer[0..bufsiz] was not large enough. Copy the partial @@ -1287,7 +1287,7 @@ int unpack_loose_header(git_zstream *stream, stream->next_out = buffer; stream->avail_out = bufsiz; } while (status != Z_STREAM_END); - return -1; + return -2; } static void *unpack_loose_rest(git_zstream *stream, @@ -1456,9 +1456,19 @@ static int loose_object_info(struct repository *r, hdr_ret = unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr), allow_unknown ? &hdrbuf : NULL); - if (hdr_ret < 0) { + switch (hdr_ret) { + case 0: + break; + case -1: status = error(_("unable to unpack %s header"), oid_to_hex(oid)); + break; + case -2: + status = error(_("header for %s too long, exceeds %d bytes"), + oid_to_hex(oid), MAX_HEADER_LEN); + break; + default: + BUG("unknown hdr_ret value %d", hdr_ret); } if (!status) { if (!parse_loose_header(hdrbuf.len ? hdrbuf.buf : hdr, oi)) diff --git a/object-store.h b/object-store.h index 584bf5556af..e896b813f24 100644 --- a/object-store.h +++ b/object-store.h @@ -489,13 +489,15 @@ int for_each_packed_object(each_packed_object_fn, void *, * unpack_loose_header() initializes the data stream needed to unpack * a loose object header. * - * Returns 0 on success. Returns negative values on error. + * Returns 0 on success. Returns negative values on error. If the + * header exceeds MAX_HEADER_LEN -2 will be returned. * * It will only parse up to MAX_HEADER_LEN bytes unless an optional * "hdrbuf" argument is non-NULL. This is intended for use with * OBJECT_INFO_ALLOW_UNKNOWN_TYPE to extract the bad type for (error) * reporting. The full header will be extracted to "hdrbuf" for use - * with parse_loose_header(). + * with parse_loose_header(), -2 will still be returned from this + * function to indicate that the header was too long. */ int unpack_loose_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 98729f1edfc..43a9f4e7f0c 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -440,7 +440,7 @@ bogus_sha1=$(echo_without_newline "$bogus_content" | git hash-object -t $bogus_t test_expect_success 'die on broken object with large type under -t and -s without --allow-unknown-type' ' cat >err.expect <<-EOF && - error: unable to unpack $bogus_sha1 header + error: header for $bogus_sha1 too long, exceeds 32 bytes fatal: git cat-file: could not get object info EOF From patchwork Tue Sep 7 10:58:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478327 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D473DC433F5 for ; Tue, 7 Sep 2021 10:59:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BF17860E52 for ; Tue, 7 Sep 2021 10:59:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343941AbhIGLAH (ORCPT ); Tue, 7 Sep 2021 07:00:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51066 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343848AbhIGK7u (ORCPT ); Tue, 7 Sep 2021 06:59:50 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A6A5C0612A6 for ; Tue, 7 Sep 2021 03:58:42 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id i3so6502567wmq.3 for ; Tue, 07 Sep 2021 03:58:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=k/byc6j7Bpgd+6guaK79Q+lG8HWsTzOjCAUjuk1pnlg=; b=Xsbd4PLCA5kdZQg7xzxo5+NpDOia9Ak+G1IqENUX51w77tMT03nuaIIblvPj8OFoge AQzOHuaGJORN8V09E97qC0L5LBgT8xwB+z80ILJn9O9oq7zUatmWH8kArc/hNhPtRP0N r/sc7baqZwqAaDuMvK8BAZ0sUieuBm/LFmVIBn06nllde4TOBFCY3kgqZOmxaPh+Fz8Y gkFSU5oyijmWMCutrJY2vWdVLSLk+TrXPykYqRz9nGNB0CN1+oBlBsh/JNvBaC/rBQGG fPJZy/6eZ3GOGQHejaoygpbetPIo9DqiBIf8K5JXLTfpDH3kI7fLe+qQYcansT1hvE3b XkQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=k/byc6j7Bpgd+6guaK79Q+lG8HWsTzOjCAUjuk1pnlg=; b=tskhBs7SLMcaY7Q/oSRi6vfguPuWe4MpSAl7qiVCrzGXhc1wCKdvJxsAcxVg9bX7SK 4T+uaiF2FRPTOUqhIkmndQAHqAAo6CRoo2AbccZVrhPxp4076qSEhe3e6/5J1OI3rfD+ i7PjXKORt37iJLYOLhc/jYjS9RVuOw166AjEXB0MryyzHSVTlQWHdGVRi5rzIpRzjuvI 2W04tT1l+Df/IbhGF2qlFNdXlPzHuQ54iemIhX5KTBiS7f9Xe4XMvwlMVAu1WzP3UPCj WPf+LmvYkxzbIABOswrhz3AcCv8OoYejjqxvIB6hieaGQPiW6WYxsrYji9iwuYQUZFb9 ZD6w== X-Gm-Message-State: AOAM531+aLYRNUueA4ltruR3t6P8AX2zIGt9VvcwJIlVKcutWWkhdCZk lyDrgTN4jW3RKpNtzVgM9xaCVXrvfPdKZQ== X-Google-Smtp-Source: ABdhPJykgtbgpssOwqLxjx3WXjQdJYd95CYiNKhPZJn0/PrUEO5JwHd0kKmoI/VKey6+02sTEcJlFg== X-Received: by 2002:a05:600c:2114:: with SMTP id u20mr3362904wml.65.1631012320750; Tue, 07 Sep 2021 03:58:40 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:40 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 18/22] object-file.c: use "enum" return type for unpack_loose_header() Date: Tue, 7 Sep 2021 12:58:13 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In the preceding commits we changed and documented unpack_loose_header() from return any negative value or zero, to only -2, -1 or 0. Let's instead add an "enum unpack_loose_header_result" type and use it, and have the compiler assert that we're exhaustively covering all return values. This gets rid of the need for having a "default" BUG() case in loose_object_info(). I'm on the fence about whether this is more readable or worth it, but since it was suggested in [1] to do this let's go for it. 1. https://lore.kernel.org/git/20210527175433.2673306-1-jonathantanmy@google.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- object-file.c | 20 ++++++++++---------- object-store.h | 27 ++++++++++++++++++++------- streaming.c | 27 ++++++++++++++++----------- 3 files changed, 46 insertions(+), 28 deletions(-) diff --git a/object-file.c b/object-file.c index 0cb5287d3ef..9484c7ce2be 100644 --- a/object-file.c +++ b/object-file.c @@ -1233,10 +1233,12 @@ void *map_loose_object(struct repository *r, return map_loose_object_1(r, NULL, oid, size); } -int unpack_loose_header(git_zstream *stream, - unsigned char *map, unsigned long mapsize, - void *buffer, unsigned long bufsiz, - struct strbuf *header) +enum unpack_loose_header_result unpack_loose_header(git_zstream *stream, + unsigned char *map, + unsigned long mapsize, + void *buffer, + unsigned long bufsiz, + struct strbuf *header) { int status; @@ -1411,7 +1413,7 @@ static int loose_object_info(struct repository *r, unsigned long mapsize; void *map; git_zstream stream; - int hdr_ret; + enum unpack_loose_header_result hdr_ret; char hdr[MAX_HEADER_LEN]; struct strbuf hdrbuf = STRBUF_INIT; unsigned long size_scratch; @@ -1457,18 +1459,16 @@ static int loose_object_info(struct repository *r, hdr_ret = unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr), allow_unknown ? &hdrbuf : NULL); switch (hdr_ret) { - case 0: + case UNPACK_LOOSE_HEADER_RESULT_OK: break; - case -1: + case UNPACK_LOOSE_HEADER_RESULT_BAD: status = error(_("unable to unpack %s header"), oid_to_hex(oid)); break; - case -2: + case UNPACK_LOOSE_HEADER_RESULT_BAD_TOO_LONG: status = error(_("header for %s too long, exceeds %d bytes"), oid_to_hex(oid), MAX_HEADER_LEN); break; - default: - BUG("unknown hdr_ret value %d", hdr_ret); } if (!status) { if (!parse_loose_header(hdrbuf.len ? hdrbuf.buf : hdr, oi)) diff --git a/object-store.h b/object-store.h index e896b813f24..ac55b02f15a 100644 --- a/object-store.h +++ b/object-store.h @@ -485,23 +485,36 @@ int for_each_object_in_pack(struct packed_git *p, int for_each_packed_object(each_packed_object_fn, void *, enum for_each_object_flags flags); +enum unpack_loose_header_result { + UNPACK_LOOSE_HEADER_RESULT_BAD_TOO_LONG = -2, + UNPACK_LOOSE_HEADER_RESULT_BAD = -1, + UNPACK_LOOSE_HEADER_RESULT_OK, + +}; + /** * unpack_loose_header() initializes the data stream needed to unpack * a loose object header. * - * Returns 0 on success. Returns negative values on error. If the - * header exceeds MAX_HEADER_LEN -2 will be returned. + * Returns UNPACK_LOOSE_HEADER_RESULT_OK on success. Returns + * UNPACK_LOOSE_HEADER_RESULT_BAD values on error, or if the header + * exceeds MAX_HEADER_LEN UNPACK_LOOSE_HEADER_RESULT_BAD_TOO_LONG will + * be returned. * * It will only parse up to MAX_HEADER_LEN bytes unless an optional * "hdrbuf" argument is non-NULL. This is intended for use with * OBJECT_INFO_ALLOW_UNKNOWN_TYPE to extract the bad type for (error) * reporting. The full header will be extracted to "hdrbuf" for use - * with parse_loose_header(), -2 will still be returned from this - * function to indicate that the header was too long. + * with parse_loose_header(), UNPACK_LOOSE_HEADER_RESULT_BAD_TOO_LONG + * will still be returned from this function to indicate that the + * header was too long. */ -int unpack_loose_header(git_zstream *stream, unsigned char *map, - unsigned long mapsize, void *buffer, - unsigned long bufsiz, struct strbuf *hdrbuf); +enum unpack_loose_header_result unpack_loose_header(git_zstream *stream, + unsigned char *map, + unsigned long mapsize, + void *buffer, + unsigned long bufsiz, + struct strbuf *hdrbuf); /** * parse_loose_header() parses the starting " \0" of an diff --git a/streaming.c b/streaming.c index c3dc241d6a5..3e5045c004d 100644 --- a/streaming.c +++ b/streaming.c @@ -224,24 +224,25 @@ static int open_istream_loose(struct git_istream *st, struct repository *r, enum object_type *type) { struct object_info oi = OBJECT_INFO_INIT; + enum unpack_loose_header_result hdr_ret; oi.sizep = &st->size; oi.typep = type; st->u.loose.mapped = map_loose_object(r, oid, &st->u.loose.mapsize); if (!st->u.loose.mapped) return -1; - if ((unpack_loose_header(&st->z, - st->u.loose.mapped, - st->u.loose.mapsize, - st->u.loose.hdr, - sizeof(st->u.loose.hdr), - NULL) < 0) || - (parse_loose_header(st->u.loose.hdr, &oi) < 0) || - *type < 0) { - git_inflate_end(&st->z); - munmap(st->u.loose.mapped, st->u.loose.mapsize); - return -1; + hdr_ret = unpack_loose_header(&st->z, st->u.loose.mapped, + st->u.loose.mapsize, st->u.loose.hdr, + sizeof(st->u.loose.hdr), NULL); + switch (hdr_ret) { + case UNPACK_LOOSE_HEADER_RESULT_OK: + break; + case UNPACK_LOOSE_HEADER_RESULT_BAD: + case UNPACK_LOOSE_HEADER_RESULT_BAD_TOO_LONG: + goto error; } + if (parse_loose_header(st->u.loose.hdr, &oi) < 0 || *type < 0) + goto error; st->u.loose.hdr_used = strlen(st->u.loose.hdr) + 1; st->u.loose.hdr_avail = st->z.total_out; @@ -250,6 +251,10 @@ static int open_istream_loose(struct git_istream *st, struct repository *r, st->read = read_istream_loose; return 0; +error: + git_inflate_end(&st->z); + munmap(st->u.loose.mapped, st->u.loose.mapsize); + return -1; } From patchwork Tue Sep 7 10:58:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59F1FC433FE for ; Tue, 7 Sep 2021 10:59:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3EE9960F6D for ; Tue, 7 Sep 2021 10:59:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344044AbhIGLAI (ORCPT ); Tue, 7 Sep 2021 07:00:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343861AbhIGK7u (ORCPT ); Tue, 7 Sep 2021 06:59:50 -0400 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7E88CC0612A7 for ; Tue, 7 Sep 2021 03:58:43 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id y132so865164wmc.1 for ; Tue, 07 Sep 2021 03:58:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aDLy6U5KSPfcD6GccCttA0W3sVX4odpy2+oNHvBuyWs=; b=MHFj1pVKRiCM7VP8Cy6MLlX4AzCxvKU5ja/CEUTRfT0zM7aTnwkMU3Zs/7UaFhL3z7 atWhzzffPbajVndF7tNNdGAWAHbarMDadJb8E/XukFkANvUqUotHxa9qhqTgk6F5q0bP 6Pnbc+UY8EmeRc7WuWACXycxyjpg0faqfcrEhzww1eBEkWWhoDY5mhxNAROe6kQnlF6A yAIdZT/Soc5HwlHfanbY3+OoJoVM4gNtM5gAnZJ4qac3N+OB5gWuT5IR9Bl2pvocCmfr q4hhzETt+5AcnyZaGwYvnsP7dHg1taCwOLRjR28wKzyhdIEUs7CUAexCuM78N0HtI2K2 MSOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aDLy6U5KSPfcD6GccCttA0W3sVX4odpy2+oNHvBuyWs=; b=iwfO0dp7ZfuD9tZI0FIi78FCYt0+2C5LUdGc6h3XH+9qy472aP+u9TrwlTfMkKgISL oJ5ggl/C/memJRsZi9zQ/PdIvKkl5iW/kG4BvJtOuNIwe2yIFaUnuXYxIkiyzDKhXzE2 aG7KTzKu3Fa4JTtVW6yy0eyKZaLh3SpYaVT+OOFGzaM4r+GATgUlFQ985Qq9+jdS16/M Vk9mcxqilLC7qxmgqYco3JafglujtMWjEkPs2pvc+8a9zQct6/gWjzMboAvBIdqAjkjB TA0ocpNUk7TPHK5zRa4gMrSCusgIZjpGgYNF6nlEBB8HYp+23ivDC/g42J/pJWC0GL5+ v5Bw== X-Gm-Message-State: AOAM533WtbdrbIl7M0RrZRPOZJakZMXdqQ4vk1ek+akIyJBbRkH8OIy6 Et8NKtrT1I3DmsSTZR5nY1lEXcCrJ1m9IA== X-Google-Smtp-Source: ABdhPJwT9OgapFW+YjPnAsoksr4zX6a/MAwMPZYq9bdTM0gYTCL74M066rGQwz273+MpPVEHpw83nw== X-Received: by 2002:a1c:c911:: with SMTP id f17mr3266792wmb.190.1631012321812; Tue, 07 Sep 2021 03:58:41 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:41 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 19/22] fsck: don't hard die on invalid object types Date: Tue, 7 Sep 2021 12:58:14 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the error fsck emits on invalid object types, such as: $ git hash-object --stdin -w -t garbage --literally From the very ungraceful error of: $ git fsck fatal: invalid object type $ To: $ git fsck error: hash mismatch for (expected ) error: : object corrupt or missing: [ the rest of the fsck output here, i.e. it didn't hard die ] We'll still exit with non-zero, but now we'll finish the rest of the traversal. The tests that's being added here asserts that we'll still complain about other fsck issues (e.g. an unrelated dangling blob). To do this we need to pass down the "OBJECT_INFO_ALLOW_UNKNOWN_TYPE" flag from read_loose_object() through to parse_loose_header(). Since the read_loose_object() function is only used in builtin/fsck.c we can simply change it. See f6371f92104 (sha1_file: add read_loose_object() function, 2017-01-13) for the introduction of read_loose_object(). Why are we complaining about a "hash mismatch" for an object of a type we don't know about? We shouldn't. This is the bare minimal change needed to not make fsck hard die on a repository that's been corrupted in this manner. In subsequent commits we'll teach fsck to recognize this particular type of corruption and emit a better error message. Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/fsck.c | 3 ++- object-file.c | 11 ++++++++--- object-store.h | 3 ++- t/t1450-fsck.sh | 14 +++++++------- 4 files changed, 19 insertions(+), 12 deletions(-) diff --git a/builtin/fsck.c b/builtin/fsck.c index b42b6fe21f7..082dadd5629 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -601,7 +601,8 @@ static int fsck_loose(const struct object_id *oid, const char *path, void *data) void *contents; int eaten; - if (read_loose_object(path, oid, &type, &size, &contents) < 0) { + if (read_loose_object(path, oid, &type, &size, &contents, + OBJECT_INFO_ALLOW_UNKNOWN_TYPE) < 0) { errors_found |= ERROR_OBJECT; error(_("%s: object corrupt or missing: %s"), oid_to_hex(oid), path); diff --git a/object-file.c b/object-file.c index 9484c7ce2be..0e6937fad73 100644 --- a/object-file.c +++ b/object-file.c @@ -2562,7 +2562,8 @@ int read_loose_object(const char *path, const struct object_id *expected_oid, enum object_type *type, unsigned long *size, - void **contents) + void **contents, + unsigned int oi_flags) { int ret = -1; void *map = NULL; @@ -2570,6 +2571,7 @@ int read_loose_object(const char *path, git_zstream stream; char hdr[MAX_HEADER_LEN]; struct object_info oi = OBJECT_INFO_INIT; + int allow_unknown = oi_flags & OBJECT_INFO_ALLOW_UNKNOWN_TYPE; oi.typep = type; oi.sizep = size; @@ -2592,8 +2594,11 @@ int read_loose_object(const char *path, git_inflate_end(&stream); goto out; } - if (*type < 0) - die(_("invalid object type")); + if (!allow_unknown && *type < 0) { + error(_("header for %s declares an unknown type"), path); + git_inflate_end(&stream); + goto out; + } if (*type == OBJ_BLOB && *size > big_file_threshold) { if (check_stream_oid(&stream, hdr, *size, path, expected_oid) < 0) diff --git a/object-store.h b/object-store.h index ac55b02f15a..c268662f5ba 100644 --- a/object-store.h +++ b/object-store.h @@ -253,7 +253,8 @@ int read_loose_object(const char *path, const struct object_id *expected_oid, enum object_type *type, unsigned long *size, - void **contents); + void **contents, + unsigned int oi_flags); /* Retry packed storage after checking packed and loose storage */ #define HAS_OBJECT_RECHECK_PACKED 1 diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh index f10d6f7b7e8..d8303db9709 100755 --- a/t/t1450-fsck.sh +++ b/t/t1450-fsck.sh @@ -863,16 +863,16 @@ test_expect_success 'detect corrupt index file in fsck' ' test_i18ngrep "bad index file" errors ' -test_expect_success 'fsck hard errors on an invalid object type' ' +test_expect_success 'fsck error and recovery on invalid object type' ' git init --bare garbage-type && empty_blob=$(git -C garbage-type hash-object --stdin -w -t blob err.expect <<-\EOF && - fatal: invalid object type - EOF - test_must_fail git -C garbage-type fsck >out.actual 2>err.actual && - test_cmp err.expect err.actual && - test_must_be_empty out.actual + test_must_fail git -C garbage-type fsck >out 2>err && + grep -e "^error" -e "^fatal" err >errors && + test_line_count = 2 errors && + grep "error: hash mismatch for" err && + grep "$garbage_blob: object corrupt or missing:" err && + grep "dangling blob $empty_blob" out ' test_done From patchwork Tue Sep 7 10:58:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478331 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8072C433F5 for ; Tue, 7 Sep 2021 10:59:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A4E0E60E52 for ; Tue, 7 Sep 2021 10:59:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344064AbhIGLAK (ORCPT ); Tue, 7 Sep 2021 07:00:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343833AbhIGK7v (ORCPT ); Tue, 7 Sep 2021 06:59:51 -0400 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 653ACC0613CF for ; Tue, 7 Sep 2021 03:58:44 -0700 (PDT) Received: by mail-wm1-x336.google.com with SMTP id m25-20020a7bcb99000000b002e751bcb5dbso1845172wmi.5 for ; Tue, 07 Sep 2021 03:58:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=n+lhjppXbOj7Funr9CGKj854dERQmQ2rDWls2a+sJyo=; b=f008ZwMobY8C90TGtznxqVvCba/heP45DPcFBEiJulufdL9eZOdUcwviqwi9QqDSUI +xmT1dw2x4LInMCZHopgWKRbCy9nPzt/wMiKxPtAhRUjVD8/uG0ZWNBysr2XMoF8Rv9F YXTHLPQ2LOKbKbuIpG4zo0pkzkJuwX35HQjuQrZMqkXwKZnKl1X+UjMyWDwtyikMkzvr vvCZFdoZMC1jxaAfJl1DG26ZGsCTnt7k5kdGqr2nmpQThTkmoEXFsMm8gIxU8HolUYLv Cy0JXpITEWi6BBXm9bMd8M/yOaw6qo0tysk6OGYpri7hTdJpd1nPARO/CbPRbfININxq gPXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=n+lhjppXbOj7Funr9CGKj854dERQmQ2rDWls2a+sJyo=; b=Fmr5nV3XAdLlS8MDbqIiBPuEkh6XMP3ssboI3qAwthnxQ5XGhIYjJ4WU3mKSuEvP4h 8chXPRJzFm9bEw6JfZHf+qTDL/c3o4pT2ZDTJSI6XUXlmO/f324BuwF80zqx73srDIjj k5jNDBobjJqcWZ87+iDlFvXatCjqczbncUlAe4BeFoRsJ1Rf8/JbvblJdrsGMAlL9lfP LdQd71i/YdKalWIyv+XXmfZQ5vhUhNnrSu9h/EtJlmVZjePSchejGus0jtqybVuYZ9W5 DtcJuiCKKjtUYB5gnrxnPvKd3Ksfp6cPbtALihhVXAZ10n6+4zUP/Y9YE0DEltuIPgQh pUEg== X-Gm-Message-State: AOAM533yg4ZeamRGEUTQKyTrTp9OPNUyMnr8iZ6ozqfVA95z+zZLETQs yOm0mhtTXXYjcTAUJHqw4xEDqbbp+5yshg== X-Google-Smtp-Source: ABdhPJzYNa1O5UT89lwt5HRyO3tswYm+USH59b/78vwbI3eBQKiNq2q59IoYaeP/ye/tfU1o7NIh8A== X-Received: by 2002:a05:600c:2f90:: with SMTP id t16mr3249635wmn.136.1631012322779; Tue, 07 Sep 2021 03:58:42 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:42 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 20/22] object-store.h: move read_loose_object() below 'struct object_info' Date: Tue, 7 Sep 2021 12:58:15 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move the declaration of read_loose_object() below "struct object_info". In the next commit we'll add a "struct object_info *" parameter to it, moving it will avoid a forward declaration of the struct. Signed-off-by: Ævar Arnfjörð Bjarmason --- object-store.h | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/object-store.h b/object-store.h index c268662f5ba..dc638335e7d 100644 --- a/object-store.h +++ b/object-store.h @@ -242,20 +242,6 @@ int pretend_object_file(void *, unsigned long, enum object_type, int force_object_loose(const struct object_id *oid, time_t mtime); -/* - * Open the loose object at path, check its hash, and return the contents, - * type, and size. If the object is a blob, then "contents" may return NULL, - * to allow streaming of large blobs. - * - * Returns 0 on success, negative on error (details may be written to stderr). - */ -int read_loose_object(const char *path, - const struct object_id *expected_oid, - enum object_type *type, - unsigned long *size, - void **contents, - unsigned int oi_flags); - /* Retry packed storage after checking packed and loose storage */ #define HAS_OBJECT_RECHECK_PACKED 1 @@ -396,6 +382,20 @@ int oid_object_info_extended(struct repository *r, const struct object_id *, struct object_info *, unsigned flags); +/* + * Open the loose object at path, check its hash, and return the contents, + * type, and size. If the object is a blob, then "contents" may return NULL, + * to allow streaming of large blobs. + * + * Returns 0 on success, negative on error (details may be written to stderr). + */ +int read_loose_object(const char *path, + const struct object_id *expected_oid, + enum object_type *type, + unsigned long *size, + void **contents, + unsigned int oi_flags); + /* * Iterate over the files in the loose-object parts of the object * directory "path", triggering the following callbacks: From patchwork Tue Sep 7 10:58:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478333 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E18E4C433EF for ; Tue, 7 Sep 2021 10:59:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C3118610FE for ; Tue, 7 Sep 2021 10:59:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343729AbhIGLAN (ORCPT ); Tue, 7 Sep 2021 07:00:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51058 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343924AbhIGK7w (ORCPT ); Tue, 7 Sep 2021 06:59:52 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B77C5C0617AD for ; Tue, 7 Sep 2021 03:58:45 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id i3so6502699wmq.3 for ; Tue, 07 Sep 2021 03:58:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=b9T4WSYs3xOrsCqjo4p19wvv6PfuLLE3X8q7P1KXmO8=; b=hf1nAQ3grQI1m/PUFtDUu8FRDYDhUpDFEfAsl/G/Jdf4MXXktCRNVIv+pAiBRBTFZg coHv6pG6Q+y/xgKEOhDW589+PpTRXwQzsTZ8byCH2WgK1VwwKTdFHFJ9FTCMBvG4abb6 /HuqpEAuP2uOyK7fWfXYOpBlAr7mMEqYPzarTiCnl1LX1ry67e3IxJzMiwHZuj7wwY83 AOvm8h7CFke1mLFZiLS7MMuHqxA6R71xpn5soO2nOJ4mQ4hzJ0kGz9aYVnc9Lw7BlDr8 c6eCUEsMo0A+uF1Kvu611LwLn1OxBsvq1wa51f6Y+VYVs2rRo+4OaWbA3ZJg5es3iiFb M8Vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=b9T4WSYs3xOrsCqjo4p19wvv6PfuLLE3X8q7P1KXmO8=; b=E0BbsjQ9JD/PByvFJzlTJO1AUjkwAEUE+d2foDN4o2FFCMABjRMggwLTVvkaYPbleT n9KZdWLE5L3P883q/oc/NAGBKChcNTpI4KgkkARzFXsGrN1ZJeOkqdUDRTjrbKOY4WXh xdbSdp4oW1nKM9d/jmyahGSrASoEk0JvqCtAcWN9bc4O7ziWIA9yZaiQMvI0JCQxEd+s OjE5oIuMNl/qeOAbYTgOeGfyztoVyTchwPKg9IeZJeckuP6vb6Lmaox0muMhUTkHq8aI Bh6Jc5dI85fUAKaplNwdoUh+ICYuTcjn3E++fLAxFylqbeAlz8h5vdnXTlaVJLbBbzVi DEOQ== X-Gm-Message-State: AOAM5303SNh7+hxqzwUKQmgpUHOYPrqVcogUqjvyek99tDw1AQ7GyJ3u JP8pPif43ehqHXz/EOC5HsDdqAvJcRdmdg== X-Google-Smtp-Source: ABdhPJzNyojkN2kyygE1K1PVHchlCTvxGEkXYVlX3hn9BwNXD5NgHNXWEvkjNzlERyNzG4gNd9H1bg== X-Received: by 2002:a1c:791a:: with SMTP id l26mr3283302wme.100.1631012324057; Tue, 07 Sep 2021 03:58:44 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:43 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 21/22] fsck: report invalid types recorded in objects Date: Tue, 7 Sep 2021 12:58:16 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Continue the work in the preceding commit and improve the error on: $ git hash-object --stdin -w -t garbage --literally (expected ) error: : object corrupt or missing: [ other fsck output ] To instead emit: $ git fsck error: : object is of unknown type 'garbage': [ other fsck output ] The complaint about a "hash mismatch" was simply an emergent property of how we'd fall though from read_loose_object() into fsck_loose() when we didn't get the data we expected. Now we'll correctly note that the object type is invalid. Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/fsck.c | 22 ++++++++++++++++++---- object-file.c | 13 +++++-------- object-store.h | 4 ++-- t/t1450-fsck.sh | 24 +++++++++++++++++++++--- 4 files changed, 46 insertions(+), 17 deletions(-) diff --git a/builtin/fsck.c b/builtin/fsck.c index 082dadd5629..07af0434db6 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -600,12 +600,26 @@ static int fsck_loose(const struct object_id *oid, const char *path, void *data) unsigned long size; void *contents; int eaten; - - if (read_loose_object(path, oid, &type, &size, &contents, - OBJECT_INFO_ALLOW_UNKNOWN_TYPE) < 0) { - errors_found |= ERROR_OBJECT; + struct strbuf sb = STRBUF_INIT; + unsigned int oi_flags = OBJECT_INFO_ALLOW_UNKNOWN_TYPE; + struct object_info oi; + int found = 0; + oi.type_name = &sb; + oi.sizep = &size; + oi.typep = &type; + + if (read_loose_object(path, oid, &contents, &oi, oi_flags) < 0) { + found |= ERROR_OBJECT; error(_("%s: object corrupt or missing: %s"), oid_to_hex(oid), path); + } + if (type < 0) { + found |= ERROR_OBJECT; + error(_("%s: object is of unknown type '%s': %s"), + oid_to_hex(oid), sb.buf, path); + } + if (found) { + errors_found |= ERROR_OBJECT; return 0; /* keep checking other objects */ } diff --git a/object-file.c b/object-file.c index 0e6937fad73..f4850ba62b4 100644 --- a/object-file.c +++ b/object-file.c @@ -2560,9 +2560,8 @@ static int check_stream_oid(git_zstream *stream, int read_loose_object(const char *path, const struct object_id *expected_oid, - enum object_type *type, - unsigned long *size, void **contents, + struct object_info *oi, unsigned int oi_flags) { int ret = -1; @@ -2570,10 +2569,9 @@ int read_loose_object(const char *path, unsigned long mapsize; git_zstream stream; char hdr[MAX_HEADER_LEN]; - struct object_info oi = OBJECT_INFO_INIT; int allow_unknown = oi_flags & OBJECT_INFO_ALLOW_UNKNOWN_TYPE; - oi.typep = type; - oi.sizep = size; + enum object_type *type = oi->typep; + unsigned long *size = oi->sizep; *contents = NULL; @@ -2589,7 +2587,7 @@ int read_loose_object(const char *path, goto out; } - if (parse_loose_header(hdr, &oi) < 0) { + if (parse_loose_header(hdr, oi) < 0) { error(_("unable to parse header of %s"), path); git_inflate_end(&stream); goto out; @@ -2611,8 +2609,7 @@ int read_loose_object(const char *path, goto out; } if (check_object_signature(the_repository, expected_oid, - *contents, *size, - type_name(*type))) { + *contents, *size, oi->type_name->buf)) { error(_("hash mismatch for %s (expected %s)"), path, oid_to_hex(expected_oid)); free(*contents); diff --git a/object-store.h b/object-store.h index dc638335e7d..f3045148b89 100644 --- a/object-store.h +++ b/object-store.h @@ -384,6 +384,7 @@ int oid_object_info_extended(struct repository *r, /* * Open the loose object at path, check its hash, and return the contents, + * use the "oi" argument to assert things about the object, or e.g. populate its * type, and size. If the object is a blob, then "contents" may return NULL, * to allow streaming of large blobs. * @@ -391,9 +392,8 @@ int oid_object_info_extended(struct repository *r, */ int read_loose_object(const char *path, const struct object_id *expected_oid, - enum object_type *type, - unsigned long *size, void **contents, + struct object_info *oi, unsigned int oi_flags); /* diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh index d8303db9709..da2658155c7 100755 --- a/t/t1450-fsck.sh +++ b/t/t1450-fsck.sh @@ -66,6 +66,25 @@ test_expect_success 'object with hash mismatch' ' ) ' +test_expect_success 'object with hash and type mismatch' ' + git init --bare hash-type-mismatch && + ( + cd hash-type-mismatch && + oid=$(echo blob | git hash-object -w --stdin -t garbage --literally) && + old=$(test_oid_to_path "$oid") && + new=$(dirname $old)/$(test_oid ff_2) && + oid="$(dirname $new)$(basename $new)" && + mv objects/$old objects/$new && + git update-index --add --cacheinfo 100644 $oid foo && + tree=$(git write-tree) && + cmt=$(echo bogus | git commit-tree $tree) && + git update-ref refs/heads/bogus $cmt && + test_must_fail git fsck 2>out && + grep "^error: hash mismatch for " out && + grep "^error: $oid: object is of unknown type '"'"'garbage'"'"'" out + ) +' + test_expect_success 'branch pointing to non-commit' ' git rev-parse HEAD^{tree} >.git/refs/heads/invalid && test_when_finished "git update-ref -d refs/heads/invalid" && @@ -869,9 +888,8 @@ test_expect_success 'fsck error and recovery on invalid object type' ' garbage_blob=$(git -C garbage-type hash-object --stdin -w -t garbage --literally out 2>err && grep -e "^error" -e "^fatal" err >errors && - test_line_count = 2 errors && - grep "error: hash mismatch for" err && - grep "$garbage_blob: object corrupt or missing:" err && + test_line_count = 1 errors && + grep "$garbage_blob: object is of unknown type '"'"'garbage'"'"':" err && grep "dangling blob $empty_blob" out ' From patchwork Tue Sep 7 10:58:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12478335 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CA25C433FE for ; Tue, 7 Sep 2021 10:59:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EE32E60F6D for ; Tue, 7 Sep 2021 10:59:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343732AbhIGLAP (ORCPT ); Tue, 7 Sep 2021 07:00:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343944AbhIGK7x (ORCPT ); Tue, 7 Sep 2021 06:59:53 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 19A5AC0612E7 for ; Tue, 7 Sep 2021 03:58:46 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id u16so13779488wrn.5 for ; Tue, 07 Sep 2021 03:58:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lTMZsWmcbbPQ1jY9Fd2Vlkg9IlIs1cKyWXs/9KBZ1yw=; b=g/KhQFQ9HR04wU4LGFkRF+TxtjJgGrta1RArNTc/sK+qfSBNhQhiCQ1tASvqeqZcpI m//+P3iCskQsoUOEWPgYeiYRb/TnxfDuFtovFsNnZxzdBaVnlCTemWlFoWQrMk1yjouQ L6efVSJbhPonf5ZTgOwRmipvYaA2tfqq/7PHUgvVHaOvouWNIFlYEXco55+E4t2+2FyJ rr5AuOaOnYD5ra8NMcgmiUvk9qhUTwcAJjOTQ+kUm16bL/psrAjVbaH3Ub2YLHVRyGoz gb+jSuYXavrNVPIBaZvdfVsQ5yDo3gqfjOJ+Ars+t0j9hGj/fEMEWJZUU48F/+/3X7hI jjQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lTMZsWmcbbPQ1jY9Fd2Vlkg9IlIs1cKyWXs/9KBZ1yw=; b=bpxRwIYaRi3h33JroGfjFt4vDlWRKTRY/Rh45DKeR3nS+hJVtH4FlUJiyjrWW84I34 wXB1hcZoJRv0CeHJjzCzbLFhH3YNzKPPrkuJuZT4nBZ245aH+UD4szBIvuY1IsQUPNva WFGyx7RPP9uH33TK2T8t/1o6RBI2BzIAFL/ZguyjxTWKgZ37zpzb2X9/n1zpBUF2zAaE eVn1YuCCtZUW+BX5qnht5Pm6O9csud0mYhzm0Ll6VUlTHyxWJShG+HxCaHmR7rszNEGJ fV/a9fqa3PB45f6+CQYawhskbVXMzgzYN36xDcJKBPaIvqnrjK1yVGfFsKhifRJ1qM4G Sy7g== X-Gm-Message-State: AOAM533bHG+prwg+6bjWNSp65vb8KkRZZuIVzJndH/TT+4zWR6X58xnF erENs3dvDIp2UakPBo2uQcY3NEf2IdyDVg== X-Google-Smtp-Source: ABdhPJxGn2E9tuV3i6HNp0CGzuugtuTpEWgLDzf33SVgiZCzenfvBHqguYOJAo2BuaFbUdAdS7FaVg== X-Received: by 2002:adf:efc2:: with SMTP id i2mr18278885wrp.94.1631012325172; Tue, 07 Sep 2021 03:58:45 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id l16sm8255602wrh.44.2021.09.07.03.58.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Sep 2021 03:58:44 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Jonathan Tan , Andrei Rybak , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 22/22] fsck: report invalid object type-path combinations Date: Tue, 7 Sep 2021 12:58:17 +0200 Message-Id: X-Mailer: git-send-email 2.33.0.815.g21c7aaf6073 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Improve the error that's emitted in cases where we find a loose object we parse, but which isn't at the location we expect it to be. Before this change we'd prefix the error with a not-a-OID derived from the path at which the object was found, due to an emergent behavior in how we'd end up with an "OID" in these codepaths. Now we'll instead say what object we hashed, and what path it was found at. Before this patch series e.g.: $ git hash-object --stdin -w -t blob --- builtin/fast-export.c | 2 +- builtin/fsck.c | 13 +++++++++---- builtin/index-pack.c | 2 +- builtin/mktag.c | 3 ++- object-file.c | 21 ++++++++++++--------- object-store.h | 4 +++- object.c | 4 ++-- pack-check.c | 3 ++- t/t1006-cat-file.sh | 2 +- t/t1450-fsck.sh | 8 +++++--- 10 files changed, 38 insertions(+), 24 deletions(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 3c20f164f0f..48a3b6a7f8f 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -312,7 +312,7 @@ static void export_blob(const struct object_id *oid) if (!buf) die("could not read blob %s", oid_to_hex(oid)); if (check_object_signature(the_repository, oid, buf, size, - type_name(type)) < 0) + type_name(type), NULL) < 0) die("oid mismatch in blob %s", oid_to_hex(oid)); object = parse_object_buffer(the_repository, oid, type, size, buf, &eaten); diff --git a/builtin/fsck.c b/builtin/fsck.c index 07af0434db6..158b9dac9b3 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -603,20 +603,25 @@ static int fsck_loose(const struct object_id *oid, const char *path, void *data) struct strbuf sb = STRBUF_INIT; unsigned int oi_flags = OBJECT_INFO_ALLOW_UNKNOWN_TYPE; struct object_info oi; + struct object_id real_oid = *null_oid(); int found = 0; oi.type_name = &sb; oi.sizep = &size; oi.typep = &type; - if (read_loose_object(path, oid, &contents, &oi, oi_flags) < 0) { + if (read_loose_object(path, oid, &real_oid, &contents, &oi, oi_flags) < 0) { found |= ERROR_OBJECT; - error(_("%s: object corrupt or missing: %s"), - oid_to_hex(oid), path); + if (!oideq(&real_oid, oid)) + error(_("%s: hash-path mismatch, found at: %s"), + oid_to_hex(&real_oid), path); + else + error(_("%s: object corrupt or missing: %s"), + oid_to_hex(oid), path); } if (type < 0) { found |= ERROR_OBJECT; error(_("%s: object is of unknown type '%s': %s"), - oid_to_hex(oid), sb.buf, path); + oid_to_hex(&real_oid), sb.buf, path); } if (found) { errors_found |= ERROR_OBJECT; diff --git a/builtin/index-pack.c b/builtin/index-pack.c index 8336466865c..9f540e0236a 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -1419,7 +1419,7 @@ static void fix_unresolved_deltas(struct hashfile *f) if (check_object_signature(the_repository, &d->oid, data, size, - type_name(type))) + type_name(type), NULL)) die(_("local object %s is corrupt"), oid_to_hex(&d->oid)); /* diff --git a/builtin/mktag.c b/builtin/mktag.c index dddcccdd368..3b2dbbb37e6 100644 --- a/builtin/mktag.c +++ b/builtin/mktag.c @@ -62,7 +62,8 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type) repl = lookup_replace_object(the_repository, tagged_oid); ret = check_object_signature(the_repository, repl, - buffer, size, type_name(*tagged_type)); + buffer, size, type_name(*tagged_type), + NULL); free(buffer); return ret; diff --git a/object-file.c b/object-file.c index f4850ba62b4..07b3e4d9b4b 100644 --- a/object-file.c +++ b/object-file.c @@ -1062,9 +1062,11 @@ void *xmmap(void *start, size_t length, * the streaming interface and rehash it to do the same. */ int check_object_signature(struct repository *r, const struct object_id *oid, - void *map, unsigned long size, const char *type) + void *map, unsigned long size, const char *type, + struct object_id *real_oidp) { - struct object_id real_oid; + struct object_id tmp; + struct object_id *real_oid = real_oidp ? real_oidp : &tmp; enum object_type obj_type; struct git_istream *st; git_hash_ctx c; @@ -1072,8 +1074,8 @@ int check_object_signature(struct repository *r, const struct object_id *oid, int hdrlen; if (map) { - hash_object_file(r->hash_algo, map, size, type, &real_oid); - return !oideq(oid, &real_oid) ? -1 : 0; + hash_object_file(r->hash_algo, map, size, type, real_oid); + return !oideq(oid, real_oid) ? -1 : 0; } st = open_istream(r, oid, &obj_type, &size, NULL); @@ -1098,9 +1100,9 @@ int check_object_signature(struct repository *r, const struct object_id *oid, break; r->hash_algo->update_fn(&c, buf, readlen); } - r->hash_algo->final_oid_fn(&real_oid, &c); + r->hash_algo->final_oid_fn(real_oid, &c); close_istream(st); - return !oideq(oid, &real_oid) ? -1 : 0; + return !oideq(oid, real_oid) ? -1 : 0; } int git_open_cloexec(const char *name, int flags) @@ -2560,6 +2562,7 @@ static int check_stream_oid(git_zstream *stream, int read_loose_object(const char *path, const struct object_id *expected_oid, + struct object_id *real_oid, void **contents, struct object_info *oi, unsigned int oi_flags) @@ -2609,9 +2612,9 @@ int read_loose_object(const char *path, goto out; } if (check_object_signature(the_repository, expected_oid, - *contents, *size, oi->type_name->buf)) { - error(_("hash mismatch for %s (expected %s)"), path, - oid_to_hex(expected_oid)); + *contents, *size, oi->type_name->buf, real_oid)) { + if (oideq(real_oid, null_oid())) + BUG("should only get OID mismatch errors with mapped contents"); free(*contents); goto out; } diff --git a/object-store.h b/object-store.h index f3045148b89..3c4ada23f5d 100644 --- a/object-store.h +++ b/object-store.h @@ -392,6 +392,7 @@ int oid_object_info_extended(struct repository *r, */ int read_loose_object(const char *path, const struct object_id *expected_oid, + struct object_id *real_oid, void **contents, struct object_info *oi, unsigned int oi_flags); @@ -528,7 +529,8 @@ enum unpack_loose_header_result unpack_loose_header(git_zstream *stream, int parse_loose_header(const char *hdr, struct object_info *oi); int check_object_signature(struct repository *r, const struct object_id *oid, - void *buf, unsigned long size, const char *type); + void *buf, unsigned long size, const char *type, + struct object_id *real_oidp); int finalize_object_file(const char *tmpfile, const char *filename); int check_and_freshen_file(const char *fn, int freshen); diff --git a/object.c b/object.c index 4e85955a941..23a24e678a8 100644 --- a/object.c +++ b/object.c @@ -279,7 +279,7 @@ struct object *parse_object(struct repository *r, const struct object_id *oid) if ((obj && obj->type == OBJ_BLOB && repo_has_object_file(r, oid)) || (!obj && repo_has_object_file(r, oid) && oid_object_info(r, oid, NULL) == OBJ_BLOB)) { - if (check_object_signature(r, repl, NULL, 0, NULL) < 0) { + if (check_object_signature(r, repl, NULL, 0, NULL, NULL) < 0) { error(_("hash mismatch %s"), oid_to_hex(oid)); return NULL; } @@ -290,7 +290,7 @@ struct object *parse_object(struct repository *r, const struct object_id *oid) buffer = repo_read_object_file(r, oid, &type, &size); if (buffer) { if (check_object_signature(r, repl, buffer, size, - type_name(type)) < 0) { + type_name(type), NULL) < 0) { free(buffer); error(_("hash mismatch %s"), oid_to_hex(repl)); return NULL; diff --git a/pack-check.c b/pack-check.c index c8e560d71ab..3f418e3a6af 100644 --- a/pack-check.c +++ b/pack-check.c @@ -142,7 +142,8 @@ static int verify_packfile(struct repository *r, err = error("cannot unpack %s from %s at offset %"PRIuMAX"", oid_to_hex(&oid), p->pack_name, (uintmax_t)entries[i].offset); - else if (check_object_signature(r, &oid, data, size, type_name(type))) + else if (check_object_signature(r, &oid, data, size, + type_name(type), NULL)) err = error("packed %s from %s is corrupt", oid_to_hex(&oid), p->pack_name); else if (fn) { diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 43a9f4e7f0c..39fe11bc92c 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -490,7 +490,7 @@ test_expect_success 'cat-file -t and -s on corrupt loose object' ' # Swap the two to corrupt the repository mv -f "$other_path" "$empty_path" && test_must_fail git fsck 2>err.fsck && - grep "hash mismatch" err.fsck && + grep "hash-path mismatch" err.fsck && # confirm that cat-file is reading the new swapped-in # blob... diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh index da2658155c7..7d0d57564b5 100755 --- a/t/t1450-fsck.sh +++ b/t/t1450-fsck.sh @@ -53,6 +53,7 @@ test_expect_success 'object with hash mismatch' ' ( cd hash-mismatch && oid=$(echo blob | git hash-object -w --stdin) && + oldoid=$oid && old=$(test_oid_to_path "$oid") && new=$(dirname $old)/$(test_oid ff_2) && oid="$(dirname $new)$(basename $new)" && @@ -62,7 +63,7 @@ test_expect_success 'object with hash mismatch' ' cmt=$(echo bogus | git commit-tree $tree) && git update-ref refs/heads/bogus $cmt && test_must_fail git fsck 2>out && - test_i18ngrep "$oid.*corrupt" out + grep "$oldoid: hash-path mismatch, found at: .*$new" out ) ' @@ -71,6 +72,7 @@ test_expect_success 'object with hash and type mismatch' ' ( cd hash-type-mismatch && oid=$(echo blob | git hash-object -w --stdin -t garbage --literally) && + oldoid=$oid && old=$(test_oid_to_path "$oid") && new=$(dirname $old)/$(test_oid ff_2) && oid="$(dirname $new)$(basename $new)" && @@ -80,8 +82,8 @@ test_expect_success 'object with hash and type mismatch' ' cmt=$(echo bogus | git commit-tree $tree) && git update-ref refs/heads/bogus $cmt && test_must_fail git fsck 2>out && - grep "^error: hash mismatch for " out && - grep "^error: $oid: object is of unknown type '"'"'garbage'"'"'" out + grep "^error: $oldoid: hash-path mismatch, found at: .*$new" out && + grep "^error: $oldoid: object is of unknown type '"'"'garbage'"'"'" out ) '