From patchwork Fri Apr 5 03:37:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 10886851 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B9D371390 for ; Fri, 5 Apr 2019 03:37:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A1E7E28AF3 for ; Fri, 5 Apr 2019 03:37:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 960D028ADA; Fri, 5 Apr 2019 03:37:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2AB1B28ADA for ; Fri, 5 Apr 2019 03:37:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729887AbfDEDho (ORCPT ); Thu, 4 Apr 2019 23:37:44 -0400 Received: from mail-pl1-f194.google.com ([209.85.214.194]:44520 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729631AbfDEDho (ORCPT ); Thu, 4 Apr 2019 23:37:44 -0400 Received: by mail-pl1-f194.google.com with SMTP id g12so2241411pll.11 for ; Thu, 04 Apr 2019 20:37:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=QHITU3saHrz6FKalxeyHDbDq0zATq6MwsUSWFfoZ3Fs=; b=yPly15uiTvRfAIcYFf4yHFtKFJFLMr7b6raIvSyNw8lMBNwkpYVist3uG7AZObyOEG jhq1pWcHvoHtCwrj0Dwb6Hvsl5mOivB270e5JvxhzOuMkWk9MmjDcZ+m6uRrVv/7jXl3 HF8r/9+e2hUINuiqyEH6WSWP5rtVWt9rQGg+7RvkmkUEVuCe+9j7rra8Rgb5yu+iWsW+ xc62ozWuYgzV/+bOOQTPObD9WOF/pTlc8V3lCqu7vcjadFB6KcUfwJ10RTjQM+aICgbt QQGb5F0lVxsHXJvCXW6riUkn1MNekrDvCGnY7eFICPvnlb0jBCQ21scUuN6O5VMdbQve COVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=QHITU3saHrz6FKalxeyHDbDq0zATq6MwsUSWFfoZ3Fs=; b=PQOlo4PafPu1o0Vj4tYbUkMCfLE6hYTq3jHiHsxG+MSdICXP+wbxlXZoqaX3Qf/7h4 hjfoyXkqDAkMNKGyzLURZQ7ObF4zdJJSIOa/5pdqiuIiEdB59e0OjtVhgdqvHvGUk+a+ z/npJKkUiYhM3WBTix5hatJwFsYVkStzEFp1UJzLo4vmvN4xZNMu9BYGrFvX3guRu8DN ArfoVdsqGkY+j2b0FN4l0z6KuiOm8TMfvm9FkpghzlX7m8FWxg+UR2vN3p+Tf7HL5tmp q5u/ORrbZZ0Yz4EOcRKobaCNoQV4aE4otFg6hRnSOEc2RHSh96G54rmmahPqFXNekFpf KPtg== X-Gm-Message-State: APjAAAU/ot/14LkKOJUjJ94cQS8i4cbedUi1KXMhCQX0YdnBwuM1IBKt Tv1N+y9rnb6lBam1zFabRaU8EDYX83VNfA== X-Google-Smtp-Source: APXvYqw7Phj92JS/ifXP69+BdLsl6GR9012VX0n/joOR+yr/QtIOlyyEynrhtwosQ+CrP9CPS2av0g== X-Received: by 2002:a17:902:2bc9:: with SMTP id l67mr10655437plb.102.1554435463416; Thu, 04 Apr 2019 20:37:43 -0700 (PDT) Received: from localhost ([2601:602:9200:32b0:9900:4016:ceb4:1574]) by smtp.gmail.com with ESMTPSA id l69sm27661879pga.73.2019.04.04.20.37.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Apr 2019 20:37:42 -0700 (PDT) Date: Thu, 4 Apr 2019 20:37:42 -0700 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, gitster@pobox.com Subject: [PATCH 1/7] t: move 'hex2oct' into test-lib-functions.sh Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The helper 'hex2oct' is used to convert base-16 encoded data into a base-8 binary form, and is useful for preparing data for commands that accept input in a binary format, such as 'git hash-object', via 'printf'. This helper is defined identically in three separate places throughout 't'. Move the definition to test-lib-function.sh, so that it can be used in new test suites, and its definition is not redundant. This will likewise make our job easier in the subsequent commit, which also uses 'hex2oct'. Signed-off-by: Taylor Blau --- t/t1007-hash-object.sh | 4 ---- t/t1450-fsck.sh | 4 ---- t/t5601-clone.sh | 4 ---- t/test-lib-functions.sh | 6 ++++++ 4 files changed, 6 insertions(+), 12 deletions(-) diff --git a/t/t1007-hash-object.sh b/t/t1007-hash-object.sh index a37753047e..7099d33508 100755 --- a/t/t1007-hash-object.sh +++ b/t/t1007-hash-object.sh @@ -199,10 +199,6 @@ test_expect_success 'too-short tree' ' test_i18ngrep "too-short tree object" err ' -hex2oct() { - perl -ne 'printf "\\%03o", hex for /../g' -} - test_expect_success 'malformed mode in tree' ' hex_sha1=$(echo foo | git hash-object --stdin -w) && bin_sha1=$(echo $hex_sha1 | hex2oct) && diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh index 49f08d5b9c..0f268a3664 100755 --- a/t/t1450-fsck.sh +++ b/t/t1450-fsck.sh @@ -256,10 +256,6 @@ test_expect_success 'unparseable tree object' ' test_i18ngrep ! "fatal: empty filename in tree entry" out ' -hex2oct() { - perl -ne 'printf "\\%03o", hex for /../g' -} - test_expect_success 'tree entry with type mismatch' ' test_when_finished "remove_object \$blob" && test_when_finished "remove_object \$tree" && diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh index d6948cbdab..3f49943010 100755 --- a/t/t5601-clone.sh +++ b/t/t5601-clone.sh @@ -611,10 +611,6 @@ test_expect_success 'GIT_TRACE_PACKFILE produces a usable pack' ' git -C replay.git index-pack -v --stdin X-Patchwork-Id: 10886853 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 566681390 for ; Fri, 5 Apr 2019 03:37:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E06928ADA for ; Fri, 5 Apr 2019 03:37:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 32A1128AFC; Fri, 5 Apr 2019 03:37:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8E6D928ADA for ; Fri, 5 Apr 2019 03:37:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729908AbfDEDhr (ORCPT ); Thu, 4 Apr 2019 23:37:47 -0400 Received: from mail-pl1-f196.google.com ([209.85.214.196]:37953 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729631AbfDEDhr (ORCPT ); Thu, 4 Apr 2019 23:37:47 -0400 Received: by mail-pl1-f196.google.com with SMTP id g37so2252187plb.5 for ; Thu, 04 Apr 2019 20:37:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=0tTQ10WZWd33DovXUawoSSM0sld+DtFlKe8MHueC55s=; b=kaldXoIF5UrsUVVoQymqLiZPtgafJdVpZjMXhy49CG9f17G0SE6bRg++d6Y5EJpqy2 PPsr3k46FaNTV7Ym63fJDxWQmTcii9fjgMBZNV8NAimSrM5sB8vm4qufE6VbIQTtKGrH itzTqujYypu/5TOZiAlQDCZSgeTr1bIRVG4MgnYv579nx3rY4td+elTZa2CJ9g87C8HS VgxZhhufwF3/dSEvdYOOd2CBVP9mdZAr6NRDSLIcE32mseF/zr6HjfahaYL3UWWVN7js Gy8h+J3/+oXM75XhDPHCqQJH1SgSPlXFiSzipCzp7SMzc5OgXss/jfdiw7D7K8xWq6pz s4Ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=0tTQ10WZWd33DovXUawoSSM0sld+DtFlKe8MHueC55s=; b=GjalJRQ2fU4fHXeajTgaOka3j7YdCEW1QwLwgGPK2M5G+zAXImGfTv41/vmnNd6gXY 5escsOYr4o6ChP31brnhGxnCjc39zQgGtCev3nayAwUw0oGovxE/f9bzF+vLPWcv+SAB ZHF2edhtxrS7t46pfqoQ8oIMFiOOFDW7ouI3jzuonwnF0kK5mv8/Y1U2OceedIYmT+KY klBJBuo/C3TIjLzdScHsUL6OU6KWp+/Hc52RkrLce8oQ5qPAtVRXiIeGUlJbQytF+rEX naJ6bvjboX2D0xQ2aBWCGRMYhWDSVqLToCPy6xb754w9jqLGyVk64zFxWEe13du12Gqw MsKQ== X-Gm-Message-State: APjAAAU9knPMGAHrw3cKzspsRUm9ozvRlmwf4uCzPVMqNsiN7NJIZzCE 2UK7S/b80v7Co9SgxKc3eQBlYRF6XC7JIw== X-Google-Smtp-Source: APXvYqzvvKyTIHeBwM+hRVaNq7g/cN/4w+XbNgOItLKajamnuuhljmauPeIfEqMYlPbkBteXvQvzLg== X-Received: by 2002:a17:902:421:: with SMTP id 30mr10049165ple.142.1554435465597; Thu, 04 Apr 2019 20:37:45 -0700 (PDT) Received: from localhost ([2601:602:9200:32b0:9900:4016:ceb4:1574]) by smtp.gmail.com with ESMTPSA id t24sm21250069pfe.110.2019.04.04.20.37.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Apr 2019 20:37:45 -0700 (PDT) Date: Thu, 4 Apr 2019 20:37:44 -0700 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, gitster@pobox.com Subject: [PATCH 2/7] t: introduce tests for unexpected object types Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Call an object's type "unexpected" when the actual type of an object does not match Git's contextual expectation. For example, a tree entry whose mode differs from the object's actual type, or a commit's parent which is not another commit, and so on. This can manifest itself in various unfortunate ways, including Git SIGSEGV-ing under specific conditions. Consider the following example: Git traverses a blob (say, via `git rev-list`), and then tries to read out a tree-entry which lists that object as something other than a blob. In this case, `lookup_blob()` will return NULL, and the subsequent dereference will result in a SIGSEGV. Introduce tests that present objects of "unexpected" type in the above fashion to 'git rev-list'. Mark as failures the combinations that are already broken (i.e., they exhibit the segfault described above). In the cases that are not broken (i.e., they have NULL-ness checks or similar), mark these as expecting success. Let A be the object referenced with an unexpected type, and B be the object doing the referencing. Do the following: - test 'git rev-list --objects A B'. This causes A to be "cached", and presents the above scenario. Likewise, if we have a tree entry that claims to be a tree (for example) but points to another object type (say, a blob), there are two ways we might find out: - when we call lookup_tree(), we might find that we've already seen the object referenced as another type, in which case we'd get NULL - we call lookup_tree() successfully, but when we try to read the object, we find out it's something else. We should check that we behave sensibly in both cases (especially because it is easy for a malicious actor to provoke one case or the other). Co-authored-by: Jeff King Signed-off-by: Taylor Blau Signed-off-by: Jeff King --- t/t6102-rev-list-unexpected-objects.sh | 123 +++++++++++++++++++++++++ 1 file changed, 123 insertions(+) create mode 100755 t/t6102-rev-list-unexpected-objects.sh diff --git a/t/t6102-rev-list-unexpected-objects.sh b/t/t6102-rev-list-unexpected-objects.sh new file mode 100755 index 0000000000..472b08528a --- /dev/null +++ b/t/t6102-rev-list-unexpected-objects.sh @@ -0,0 +1,123 @@ +#!/bin/sh + +test_description='git rev-list should handle unexpected object types' + +. ./test-lib.sh + +test_expect_success 'setup well-formed objects' ' + blob="$(printf "foo" | git hash-object -w --stdin)" && + tree="$(printf "100644 blob $blob\tfoo" | git mktree)" && + commit="$(git commit-tree $tree -m "first commit")" +' + +test_expect_success 'setup unexpected non-blob entry' ' + printf "100644 foo\0$(echo $tree | hex2oct)" >broken-tree && + broken_tree="$(git hash-object -w --literally -t tree broken-tree)" +' + +test_expect_failure 'traverse unexpected non-blob entry (lone)' ' + test_must_fail git rev-list --objects $broken_tree +' + +test_expect_failure 'traverse unexpected non-blob entry (seen)' ' + test_must_fail git rev-list --objects $tree $broken_tree +' + +test_expect_success 'setup unexpected non-tree entry' ' + printf "40000 foo\0$(echo $blob | hex2oct)" >broken-tree && + broken_tree="$(git hash-object -w --literally -t tree broken-tree)" +' + +test_expect_failure 'traverse unexpected non-tree entry (lone)' ' + test_must_fail git rev-list --objects $broken_tree +' + +test_expect_failure 'traverse unexpected non-tree entry (seen)' ' + test_must_fail git rev-list --objects $blob $broken_tree >output 2>&1 +' + +test_expect_success 'setup unexpected non-commit parent' ' + git cat-file commit $commit | + perl -lpe "/^author/ && print q(parent $blob)" \ + >broken-commit && + broken_commit="$(git hash-object -w --literally -t commit \ + broken-commit)" +' + +test_expect_success 'traverse unexpected non-commit parent (lone)' ' + test_must_fail git rev-list --objects $broken_commit >output 2>&1 && + test_i18ngrep "not a commit" output +' + +test_expect_success 'traverse unexpected non-commit parent (seen)' ' + test_must_fail git rev-list --objects $commit $broken_commit \ + >output 2>&1 && + test_i18ngrep "not a commit" output +' + +test_expect_success 'setup unexpected non-tree root' ' + git cat-file commit $commit | + sed -e "s/$tree/$blob/" >broken-commit && + broken_commit="$(git hash-object -w --literally -t commit \ + broken-commit)" +' + +test_expect_failure 'traverse unexpected non-tree root (lone)' ' + test_must_fail git rev-list --objects $broken_commit +' + +test_expect_failure 'traverse unexpected non-tree root (seen)' ' + test_must_fail git rev-list --objects $blob $broken_commit +' + +test_expect_success 'setup unexpected non-commit tag' ' + git tag -a -m "tagged commit" tag $commit && + test_when_finished "git tag -d tag" && + git cat-file -p tag | sed -e "s/$commit/$blob/" >broken-tag && + tag=$(git hash-object -w --literally -t tag broken-tag) +' + +test_expect_success 'traverse unexpected non-commit tag (lone)' ' + test_must_fail git rev-list --objects $tag +' + +test_expect_success 'traverse unexpected non-commit tag (seen)' ' + test_must_fail git rev-list --objects $blob $tag >output 2>&1 && + test_i18ngrep "not a commit" output +' + +test_expect_success 'setup unexpected non-tree tag' ' + git tag -a -m "tagged tree" tag $tree && + test_when_finished "git tag -d tag" && + git cat-file -p tag | + sed -e "s/$tree/$blob/" >broken-tag && + tag=$(git hash-object -w --literally -t tag broken-tag) +' + +test_expect_success 'traverse unexpected non-tree tag (lone)' ' + test_must_fail git rev-list --objects $tag +' + +test_expect_success 'traverse unexpected non-tree tag (seen)' ' + test_must_fail git rev-list --objects $blob $tag >output 2>&1 && + test_i18ngrep "not a tree" output +' + +test_expect_success 'setup unexpected non-blob tag' ' + git tag -a -m "tagged blob" tag $blob && + test_when_finished "git tag -d tag" && + git cat-file -p tag | + sed -e "s/$blob/$commit/" >broken-tag && + tag=$(git hash-object -w --literally -t tag broken-tag) +' + +test_expect_failure 'traverse unexpected non-blob tag (lone)' ' + test_must_fail git rev-list --objects $tag +' + +test_expect_success 'traverse unexpected non-blob tag (seen)' ' + test_must_fail git rev-list --objects $commit $tag >output 2>&1 && + test_i18ngrep "not a blob" output +' + +test_done From patchwork Fri Apr 5 03:37:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 10886855 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D3F4C1390 for ; Fri, 5 Apr 2019 03:37:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BC78A28ADA for ; Fri, 5 Apr 2019 03:37:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B0C7528AFC; Fri, 5 Apr 2019 03:37:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 59F0728ADA for ; Fri, 5 Apr 2019 03:37:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729931AbfDEDht (ORCPT ); Thu, 4 Apr 2019 23:37:49 -0400 Received: from mail-pg1-f196.google.com ([209.85.215.196]:36534 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729631AbfDEDhs (ORCPT ); Thu, 4 Apr 2019 23:37:48 -0400 Received: by mail-pg1-f196.google.com with SMTP id 85so2306478pgc.3 for ; Thu, 04 Apr 2019 20:37:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=AxVTQ4DnUgP3W/0YNxbBl/tAE1nkMLyDaJJJllUySFY=; b=XkuNJCvg7NmQuJ9PvBX/QTjQQWpKwro/mGAuqwf1HT9ozsw4psQgL2C4qwM94KWQPL NNfWwHEirqTv+e3K8QiUFoXWJOAFDhkWGCUSVf7TpzLvZCjQi7lHdFJvMNidW4EH5iL/ EGq77OxPf6/EeOENEVnBcs5lzPRMwv32TyILeqS2Bc/8155JMoHx+8riRxTUqQiSt2LB 3Rnq71i9JI2uYz2LWZQq2wozrvPUv/n8XY5v+96aTfCHXddIEH0VHJ1QGjINZjem+qFP RDYw8sptmWWZTYHgCpSFQdYCbdm0sONMejL5leWw+ErG3zGAIPNcXT9zHU/yqaTjkRsU hzWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=AxVTQ4DnUgP3W/0YNxbBl/tAE1nkMLyDaJJJllUySFY=; b=imCQOqBusUyEjLVEfsm/ILOHjRTfBXCpe2j3+8mOnAqOYnYXIExjfkSCPO4ueGzJTA 0ML4dTLfSxS0BS+OJ1G+bjIJAbVxuS5+tJ3as0rYihWN2CuZmegvFQEywRCUVobfk6WK Agb2v15zjRngUHdW7kIhaLgsxMyG3qdlEqDO8/wvENybMoPDfLgt7L2yMJPMNVHVip9K mGvATuwkAawMBfP7JlI/7PZBN2xk6hilRN+Eq5GPNnRBDAEhnFfRL8AM2CpOZwG08Vlr 474X4h35F6RH/oOaaDNOSCCrSd81QhtHpFpY50LpbLCgxpJeZ3pLi7GvjiA4MEIcASsW JGYg== X-Gm-Message-State: APjAAAV6D0yze5IZQfOctxMuGe8Ex3r9ai1pyiiTjBud6l/cBQ6nqx4+ AmjGvAFflBFmyxoM0VzgCVBB7wZV/WairQ== X-Google-Smtp-Source: APXvYqzG8rVlfX9oP0DuN7UPL6c602gKWG8HBGIczwliOq01bxeIhsmSQz7lD6TJ9qq5aCOiQA7Cbg== X-Received: by 2002:a63:1912:: with SMTP id z18mr9686967pgl.115.1554435467628; Thu, 04 Apr 2019 20:37:47 -0700 (PDT) Received: from localhost ([2601:602:9200:32b0:9900:4016:ceb4:1574]) by smtp.gmail.com with ESMTPSA id l63sm30487829pfc.89.2019.04.04.20.37.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Apr 2019 20:37:47 -0700 (PDT) Date: Thu, 4 Apr 2019 20:37:46 -0700 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, gitster@pobox.com Subject: [PATCH 3/7] list-objects.c: handle unexpected non-blob entries Message-ID: <75f20ee26def257aa45dd31cd016ee76f81dcb83.1554435033.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Fix one of the cases described in the previous commit where a tree-entry that is promised to a blob is in fact a non-blob. When 'lookup_blob()' returns NULL, it is because Git has cached the requested object as a non-blob. In this case, prevent a SIGSEGV by 'die()'-ing immediately before attempting to dereference the result. Signed-off-by: Taylor Blau --- list-objects.c | 5 +++++ t/t6102-rev-list-unexpected-objects.sh | 5 +++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/list-objects.c b/list-objects.c index dc77361e11..ea04bbdee6 100644 --- a/list-objects.c +++ b/list-objects.c @@ -133,6 +133,11 @@ static void process_tree_contents(struct traversal_context *ctx, base, entry.path); else { struct blob *b = lookup_blob(ctx->revs->repo, &entry.oid); + if (!b) { + die(_("entry '%s' in tree %s has blob mode, " + "but is not a blob"), + entry.path, oid_to_hex(&tree->object.oid)); + } b->object.flags |= NOT_USER_GIVEN; process_blob(ctx, b, base, entry.path); } diff --git a/t/t6102-rev-list-unexpected-objects.sh b/t/t6102-rev-list-unexpected-objects.sh index 472b08528a..76fe9be30f 100755 --- a/t/t6102-rev-list-unexpected-objects.sh +++ b/t/t6102-rev-list-unexpected-objects.sh @@ -19,8 +19,9 @@ test_expect_failure 'traverse unexpected non-blob entry (lone)' ' test_must_fail git rev-list --objects $broken_tree ' -test_expect_failure 'traverse unexpected non-blob entry (seen)' ' - test_must_fail git rev-list --objects $tree $broken_tree +test_expect_success 'traverse unexpected non-blob entry (seen)' ' + test_must_fail git rev-list --objects $tree $broken_tree >output 2>&1 && + test_i18ngrep "is not a blob" output ' test_expect_success 'setup unexpected non-tree entry' ' From patchwork Fri Apr 5 03:37:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 10886857 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C952017E1 for ; Fri, 5 Apr 2019 03:37:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B23FE28ADA for ; Fri, 5 Apr 2019 03:37:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A6A1528AFC; Fri, 5 Apr 2019 03:37:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4D27228ADA for ; Fri, 5 Apr 2019 03:37:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729934AbfDEDhv (ORCPT ); Thu, 4 Apr 2019 23:37:51 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:45422 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729631AbfDEDhu (ORCPT ); Thu, 4 Apr 2019 23:37:50 -0400 Received: by mail-pl1-f195.google.com with SMTP id bf11so2239572plb.12 for ; Thu, 04 Apr 2019 20:37:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=eGWxjGhlNFg5TpJtevAfYnV2v0oecDO4SqpwhnHhpFE=; b=MtP0fknwN7OiSrvqs74ahIKPzyytAyFgqAP6n4pzEPCBqFiL8YN+v0ZPxdxjwGdG0S vkBTQ2Vbr+AvztFObOcjj6nwqB5Q9yi19H3vFeVkURldAeYiR82Y2cH26ZEhx1/QSirj 8hnjRx43+ApOqNJSNt6lRG1PtrPgi+nJBEbGPjlK6CguflPm8ZmGSUS9ZJQtiAdwhHdO c3wGRf7t4NFwmQvpnfBJFjawMK+k6wuUPPXw3x212QKqSToBQ243rgPBy/69q2CZJTOn bQmqOPrkCebSxSA62ynPD2O9jm4FkupM/e6NDdI/J9nmRRQj4fpSD8g/mA/vFxKqXeUY 00YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=eGWxjGhlNFg5TpJtevAfYnV2v0oecDO4SqpwhnHhpFE=; b=s6SjuQT6F8dy4XdGOxC4U+0H/GBr1gqxm8POtLP+E0xPoRkeHs8oam7/fbPZwNaqTw yQESkjymWdCqqf4IOKBv0dO6iesYqrXI+l+EsNJhMBa0EBIuKQqKPpyH1k0yiRD4oMoP xD5BJsrfWYbX/mdr7bKdGoHjmtC6fjwvYST8v7Of7IZANfjoO0GLceTsaOFp33OdcLuy sF/l2mbxhuqx1v6PznotoXYhtUcRQUOGvwnjA4x8bnucnkbqIYylaWBTG5UYa0uXrV01 srhNtLru0+MKFin+bU+9oTknvuQZXJ6Pd7whdDYJS1HhLNNIGP7k0E3GVqsZ4DqwoAi3 rPnA== X-Gm-Message-State: APjAAAVihwLvHfG3uwT5EHQXMG4dbU3YjekMbOC5MdPQvzQXaYSDKXtK QDy/TZh3L/z0vkx61qn0n0DcGr009+7crA== X-Google-Smtp-Source: APXvYqwZuAmF6srajO1/Aws9aSPZ23goSgNiYllXJVxYFJTu0sWTtM5sl06ytKelQo3RwhPRjqVWZA== X-Received: by 2002:a17:902:b484:: with SMTP id y4mr10358248plr.88.1554435469925; Thu, 04 Apr 2019 20:37:49 -0700 (PDT) Received: from localhost ([2601:602:9200:32b0:9900:4016:ceb4:1574]) by smtp.gmail.com with ESMTPSA id o5sm68988463pfa.135.2019.04.04.20.37.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Apr 2019 20:37:49 -0700 (PDT) Date: Thu, 4 Apr 2019 20:37:48 -0700 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, gitster@pobox.com Subject: [PATCH 4/7] list-objects.c: handle unexpected non-tree entries Message-ID: <6d59461830b89fab5eabae1c1b883888bd9d9fca.1554435033.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Apply similar treatment as the previous commit for non-tree entries, too. Signed-off-by: Taylor Blau --- list-objects.c | 5 +++++ t/t6102-rev-list-unexpected-objects.sh | 5 +++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/list-objects.c b/list-objects.c index ea04bbdee6..bb7e61ef4b 100644 --- a/list-objects.c +++ b/list-objects.c @@ -125,6 +125,11 @@ static void process_tree_contents(struct traversal_context *ctx, if (S_ISDIR(entry.mode)) { struct tree *t = lookup_tree(ctx->revs->repo, &entry.oid); + if (!t) { + die(_("entry '%s' in tree %s has tree mode, " + "but is not a tree"), + entry.path, oid_to_hex(&tree->object.oid)); + } t->object.flags |= NOT_USER_GIVEN; process_tree(ctx, t, base, entry.path); } diff --git a/t/t6102-rev-list-unexpected-objects.sh b/t/t6102-rev-list-unexpected-objects.sh index 76fe9be30f..30976385a8 100755 --- a/t/t6102-rev-list-unexpected-objects.sh +++ b/t/t6102-rev-list-unexpected-objects.sh @@ -33,8 +33,9 @@ test_expect_failure 'traverse unexpected non-tree entry (lone)' ' test_must_fail git rev-list --objects $broken_tree ' -test_expect_failure 'traverse unexpected non-tree entry (seen)' ' - test_must_fail git rev-list --objects $blob $broken_tree >output 2>&1 +test_expect_success 'traverse unexpected non-tree entry (seen)' ' + test_must_fail git rev-list --objects $blob $broken_tree >output 2>&1 && + test_i18ngrep "is not a tree" output ' test_expect_success 'setup unexpected non-commit parent' ' From patchwork Fri Apr 5 03:37:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 10886859 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B908917E1 for ; Fri, 5 Apr 2019 03:37:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9C48728AF3 for ; Fri, 5 Apr 2019 03:37:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8D34528ADA; Fri, 5 Apr 2019 03:37:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 37CD028ADA for ; Fri, 5 Apr 2019 03:37:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729944AbfDEDhy (ORCPT ); Thu, 4 Apr 2019 23:37:54 -0400 Received: from mail-pf1-f194.google.com ([209.85.210.194]:46168 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729941AbfDEDhx (ORCPT ); Thu, 4 Apr 2019 23:37:53 -0400 Received: by mail-pf1-f194.google.com with SMTP id 9so2462126pfj.13 for ; Thu, 04 Apr 2019 20:37:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=SrdbbYJLYN1A9NpG7aHq8VQP3FpKbnoaDhrpoWW65iU=; b=YNnOX/gSLNTKl0WSu/XsWP+eeSzQUA6YFnUMVlXilOCCmZnxegJANQVPwHks1dNd0D k3tQ3CEMMCgNgkrwVOYcTQAta29FnaE78mbY5N0FYLWVPEopDEaLpoKDiMuGaPHnjspY Bj+wBHIVztCpLusBqUum+pLMCeFAvEKfVRjNW1DYhyM+/YAgdzaFvoaw1B8uMWwqSPYA CoANcI20evUxbY46Y//P8TNZr8eAd7pFGc/RAGImLsxsXDXexnrqSnN3rXQ7kKnzqBkZ pjsnyhe+5AvpGrdfSAeYKUKDvTkcZHAAPM3d/2j/wxGfCHq5tJwLtD4UgrG6UPpphrx4 GxuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=SrdbbYJLYN1A9NpG7aHq8VQP3FpKbnoaDhrpoWW65iU=; b=HH3eq59wjDBdznArwau6sEP46r2+3cjAvlVeQ/qn4iY0IhsuiUQpdD+khdH+9yS6Mb wznax74yC85x+PpqP1VYPTze1xQp8PtEOdxSk1UNReGkldpob23p2ufDkcYU2gq3MtIK 5fZCsjwRiD4CFfAUAaSk/znL9B/bPyL19+7zoScHelvLiAoMV+jRTax94GYc8tr7+hAs 0Wyh1eIs6E7j6Zg0Vsy6u2RJUu8PUrBUUBWEcCXTFSBbvPcuFctTdvTSiuDwnzaHROSn SmH/1cG5RKiaqrABGuiQzlnS+XSEiH8LPaeCv11XvLy/OgZwjGGkeoq6df0YxSiPZqZ3 q34g== X-Gm-Message-State: APjAAAVNMomBZDHWOHnDU3umoj2y7DAroPmOljM5NaI6mz6pc9lgpagY duMfFk/pEw0+2N90wiGSDSmRbK9UP/KUMw== X-Google-Smtp-Source: APXvYqzE7VgiFOQ89st3pGqwxIsaRbaKbs9NZ1ET4M22ax2VuggnD+l7KSm42/ZYIaO4Sv5dOEXh1A== X-Received: by 2002:a63:7152:: with SMTP id b18mr9567360pgn.186.1554435472845; Thu, 04 Apr 2019 20:37:52 -0700 (PDT) Received: from localhost ([2601:602:9200:32b0:9900:4016:ceb4:1574]) by smtp.gmail.com with ESMTPSA id c17sm29720117pfd.76.2019.04.04.20.37.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Apr 2019 20:37:52 -0700 (PDT) Date: Thu, 4 Apr 2019 20:37:50 -0700 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, gitster@pobox.com Subject: [PATCH 5/7] get_commit_tree(): return NULL for broken tree Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jeff King Return NULL from 'get_commit_tree()' when a commit's root tree is corrupt, doesn't exist, or points to an object which is not a tree. In [1], this situation became a BUG(), but it can certainly occur in cases which are not a bug in Git, for e.g., if a caller manually crafts a commit whose tree is corrupt in any of the above ways. Note that the expect_failure test in t6102 triggers this BUG(), but we can't flip it to expect_success yet. Solving this problem actually reveals a second bug. [1]: 7b8a21dba1 (commit-graph: lazy-load trees for commits, 2018-04-06) Co-authored-by: Taylor Blau Signed-off-by: Taylor Blau Signed-off-by: Jeff King --- commit.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/commit.c b/commit.c index a5333c7ac6..e2cde566a9 100644 --- a/commit.c +++ b/commit.c @@ -345,10 +345,10 @@ struct tree *get_commit_tree(const struct commit *commit) if (commit->maybe_tree || !commit->object.parsed) return commit->maybe_tree; - if (commit->graph_pos == COMMIT_NOT_FROM_GRAPH) - BUG("commit has NULL tree, but was not loaded from commit-graph"); + if (commit->graph_pos != COMMIT_NOT_FROM_GRAPH) + return get_commit_tree_in_graph(the_repository, commit); - return get_commit_tree_in_graph(the_repository, commit); + return NULL; } struct object_id *get_commit_tree_oid(const struct commit *commit) From patchwork Fri Apr 5 03:37:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 10886861 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9577B1390 for ; Fri, 5 Apr 2019 03:37:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7C83228ADA for ; Fri, 5 Apr 2019 03:37:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7026D28AFC; Fri, 5 Apr 2019 03:37:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EE70A28ADA for ; Fri, 5 Apr 2019 03:37:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729958AbfDEDh5 (ORCPT ); Thu, 4 Apr 2019 23:37:57 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:39810 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729941AbfDEDh4 (ORCPT ); Thu, 4 Apr 2019 23:37:56 -0400 Received: by mail-pf1-f196.google.com with SMTP id i17so2483907pfo.6 for ; Thu, 04 Apr 2019 20:37:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=jKy0qoTPDs6MceJ1/QoSplmrd/fS2qAbAaAih34meOQ=; b=gG2a4P5Z8MzcgJ5dDh2o71JMkgflFoY1QvKSwb6qXiePsA/u23+s5IAJgNWvqO8dPI mnt1xW/bjxXUPpZft9A+zYggRAc99i7B4IM8p7lOagmy+StqdPC/8sb4cPrh8llPumn2 zvT/OdbfOA/E6l7XdDZ06CAlbuDBbuvf6X51Z8IQUPXxGL7o3YL9ZgpwnxDSPgPUd1NM HQoJu1NGnuhW6xRZjgP650H7huVJlvETj6311U/KYjD1lDt0dojWAKFamUGLC1E5hhyC /NtrCB70oe+2zrS4WIrrNAssvo1jTbkWSN249bQu46SLRk0p1pIwpqWg7rvMPsWfjuOp Q03Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=jKy0qoTPDs6MceJ1/QoSplmrd/fS2qAbAaAih34meOQ=; b=KMLh6T9D+BieRnyilimLXTFGh7l3ZSc6g8Hr9VjdHuf9JjRRtV23h5DzjG9gXf6T/o xTX8CW3KGwnLcFXDGDKX+UdJqYumaE4+wrhX+fRgqSFnBci0+OVhtgzDSabZtLEGQ/qA vfAuMiIwZA+KhCX+QW5w68weNFXfXTVOyZ/qe97U0eOTQ7frTxNV8i8ZMCpmqC4ENPmj 4m4w5ax4RxMrxBWqGuA5OqU/xVHJbwQncmxdo3zgmPXDbWTWHOfkpVcp7em4oog80y6q 5pjre2/CDPWKzr+gKHi40IJvFeSwDHv3QthIN5j0aeGF8QpaUa7iuID6RsAfV2DZejS+ Lh0w== X-Gm-Message-State: APjAAAWcu0cTnDxKcGwR8EKkdOC1bUj9u9018LGqp9C2K5LdPUybgZ8g k/ydIWqbyNQY3bEyXWrq7hjuiaBzZ3Ga6g== X-Google-Smtp-Source: APXvYqw1e+8CkBcpBEK83Y1MiDzzaoGvYs2t4flxfkaz3QN44jrWReBNx9KIr35AgXIEUvmhk85yGA== X-Received: by 2002:a62:5144:: with SMTP id f65mr10039034pfb.13.1554435475239; Thu, 04 Apr 2019 20:37:55 -0700 (PDT) Received: from localhost ([2601:602:9200:32b0:9900:4016:ceb4:1574]) by smtp.gmail.com with ESMTPSA id p17sm27330433pfn.157.2019.04.04.20.37.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Apr 2019 20:37:54 -0700 (PDT) Date: Thu, 4 Apr 2019 20:37:54 -0700 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, gitster@pobox.com Subject: [PATCH 6/7] rev-list: let traversal die when --missing is not in use Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jeff King Commit 7c0fe330d5 (rev-list: handle missing tree objects properly, 2018-10-05) taught the traversal machinery used by git-rev-list to ignore missing trees, so that rev-list could handle them itself. However, it does so only by checking via oid_object_info_extended() that the object exists at all. This can miss several classes of errors that were previously detected by rev-list: - type mismatches (e.g., we expected a tree but got a blob) - failure to read the object data (e.g., due to bitrot on disk) This is especially important because we use "rev-list --objects" as our connectivity check to admit new objects to the repository, and it will now miss these cases (though the bitrot one is less important here, because we'd typically have just hashed and stored the object). There are a few options to fix this: 1. we could check these properties in rev-list when we do the existence check. This is probably too expensive in practice (perhaps even for a type check, but definitely for checking the whole content again, which implies loading each object into memory twice). 2. teach the traversal machinery to differentiate between a missing object, and one that could not be loaded as expected. This probably wouldn't be too hard to detect type mismatches, but detecting bitrot versus a truly missing object would require deep changes to the object-loading code. 3. have the traversal machinery communicate the failure to the caller, so that it can decide how to proceed without re-evaluting the object itself. Of those, I think (3) is probably the best path forward. However, this patch does none of them. In the name of expediently fixing the regression to a normal "rev-list --objects" that we use for connectivity checks, this simply restores the pre-7c0fe330d5 behavior of having the traversal die as soon as it fails to load a tree (when --missing is set to MA_ERROR, which is the default). Note that we can't get rid of the object-existence check in finish_object(), because this also handles blobs (which are not otherwise checked at all by the traversal code). Signed-off-by: Jeff King --- builtin/rev-list.c | 4 +++- t/t6102-rev-list-unexpected-objects.sh | 4 ++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/builtin/rev-list.c b/builtin/rev-list.c index 425a5774db..9f31837d30 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -379,7 +379,6 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) repo_init_revisions(the_repository, &revs, prefix); revs.abbrev = DEFAULT_ABBREV; revs.commit_format = CMIT_FMT_UNSPECIFIED; - revs.do_not_die_on_missing_tree = 1; /* * Scan the argument list before invoking setup_revisions(), so that we @@ -409,6 +408,9 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) } } + if (arg_missing_action) + revs.do_not_die_on_missing_tree = 1; + argc = setup_revisions(argc, argv, &revs, &s_r_opt); memset(&info, 0, sizeof(info)); diff --git a/t/t6102-rev-list-unexpected-objects.sh b/t/t6102-rev-list-unexpected-objects.sh index 30976385a8..c8d4b31f8f 100755 --- a/t/t6102-rev-list-unexpected-objects.sh +++ b/t/t6102-rev-list-unexpected-objects.sh @@ -29,7 +29,7 @@ test_expect_success 'setup unexpected non-tree entry' ' broken_tree="$(git hash-object -w --literally -t tree broken-tree)" ' -test_expect_failure 'traverse unexpected non-tree entry (lone)' ' +test_expect_success 'traverse unexpected non-tree entry (lone)' ' test_must_fail git rev-list --objects $broken_tree ' @@ -64,7 +64,7 @@ test_expect_success 'setup unexpected non-tree root' ' broken-commit)" ' -test_expect_failure 'traverse unexpected non-tree root (lone)' ' +test_expect_success 'traverse unexpected non-tree root (lone)' ' test_must_fail git rev-list --objects $broken_commit ' From patchwork Fri Apr 5 03:37:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 10886863 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A4CA1390 for ; Fri, 5 Apr 2019 03:38:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 409FF28ADA for ; Fri, 5 Apr 2019 03:38:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3499028AFC; Fri, 5 Apr 2019 03:38:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CE40B28ADA for ; Fri, 5 Apr 2019 03:37:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729977AbfDEDh6 (ORCPT ); Thu, 4 Apr 2019 23:37:58 -0400 Received: from mail-pg1-f171.google.com ([209.85.215.171]:41924 "EHLO mail-pg1-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729941AbfDEDh6 (ORCPT ); Thu, 4 Apr 2019 23:37:58 -0400 Received: by mail-pg1-f171.google.com with SMTP id f6so2296967pgs.8 for ; Thu, 04 Apr 2019 20:37:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=nIQRy5oKpfovtvtUNRuuXxdSyxPv41Jq/FlYM5wt0wg=; b=J4ubGEJM8y5NNw04OapfimFzPmIhI54w6ro1nN0Dh9G+E44u/2yJaINkgl77bDknEH YkYsBGM1rGgp4y6pTB6V0N+z9olY1LvV70OxMZ4nbRt16oW+88RdMa3KkVkBxKdTVK3n Dhtjf/TR1xmOCzdV2HQA0TRG7cTQUzHcFuiqUFeaN+6EtnzTqsQIeWoKOeFtHzyhGo4n +dlqD23/kkxe2yRE8SnfZfLnZi6vMRugmkCAQS913oPSYZl0GgN6Stzj3TlQfH2XvLzp VCkNrFg0FzC7hRtSnXQ6DeM87q3zb28MVJ6EuEnvViQyIhBwY0venJPd/KmAdzZnyDzS mj+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=nIQRy5oKpfovtvtUNRuuXxdSyxPv41Jq/FlYM5wt0wg=; b=OWOjFJrojnJDzslCJh8bDPpHPMvOv06mHxVB6XuOTJsSHBiIbL9AmYj8NhLwZ2JNCn COa0TnAvdemJt14UEwV9DPdjBPJIaia0CjCiKpnNG5L99CIQm1oP2/gwnDaDvAFcSNdQ qzo7rG3tCNNa0ivFlVApwhjduWpaviNvNuZSieBZUQgLI0+P8ZwQXCzRFSkhX+GQDnt6 RIZmvnfSQ3b7dK0DyhYlCIBvh9F+CeeDKEnwAWb7L7t6FujdxbIUiy54lsVq1aIvOcsK R72Zd2kRoNofy2i7Y6E75Xaql3d1ZWnQrbSnjzz9NGpl/lKjLy8O4GQsa7RKV15ueGzR 2q8g== X-Gm-Message-State: APjAAAXq58VNrl1USqGjtF07/AmPNIdkKaWdmuVQwFbXlFpi6mIlI3Nq 6rc6bxkackqE2+adLUV83b4y9L1HBmlu/A== X-Google-Smtp-Source: APXvYqz5HlFVGowQewrKQKVcxQrEMvkEOmoURsmtHDRWTWelACvJ5SoO4AUkfDXQI5Rerb8SH9Y5mg== X-Received: by 2002:a63:1003:: with SMTP id f3mr9309364pgl.227.1554435477331; Thu, 04 Apr 2019 20:37:57 -0700 (PDT) Received: from localhost ([2601:602:9200:32b0:9900:4016:ceb4:1574]) by smtp.gmail.com with ESMTPSA id e21sm29004683pfd.177.2019.04.04.20.37.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Apr 2019 20:37:56 -0700 (PDT) Date: Thu, 4 Apr 2019 20:37:55 -0700 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, gitster@pobox.com Subject: [PATCH 7/7] rev-list: detect broken root trees Message-ID: <358da99528daba2c9d34430fe0d262b6b5d23009.1554435033.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jeff King When the traversal machinery sees a commit without a root tree, it assumes that the tree was part of a BOUNDARY commit, and quietly ignores the tree. But it could also be caused by a commit whose root tree is broken or missing. Instead, let's die() when we see a NULL root tree. We can differentiate it from the BOUNDARY case by seeing if the commit was actually parsed. This covers that case, plus future-proofs us against any others where we might try to show an unparsed commit. Signed-off-by: Jeff King --- list-objects.c | 3 +++ t/t6102-rev-list-unexpected-objects.sh | 6 ++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/list-objects.c b/list-objects.c index bb7e61ef4b..b5651ddd5b 100644 --- a/list-objects.c +++ b/list-objects.c @@ -374,6 +374,9 @@ static void do_traverse(struct traversal_context *ctx) struct tree *tree = get_commit_tree(commit); tree->object.flags |= NOT_USER_GIVEN; add_pending_tree(ctx->revs, tree); + } else if (commit->object.parsed) { + die(_("unable to load root tree for commit %s"), + oid_to_hex(&commit->object.oid)); } ctx->show_commit(commit, ctx->show_data); diff --git a/t/t6102-rev-list-unexpected-objects.sh b/t/t6102-rev-list-unexpected-objects.sh index c8d4b31f8f..98f5cffbb6 100755 --- a/t/t6102-rev-list-unexpected-objects.sh +++ b/t/t6102-rev-list-unexpected-objects.sh @@ -68,8 +68,10 @@ test_expect_success 'traverse unexpected non-tree root (lone)' ' test_must_fail git rev-list --objects $broken_commit ' -test_expect_failure 'traverse unexpected non-tree root (seen)' ' - test_must_fail git rev-list --objects $blob $broken_commit +test_expect_success 'traverse unexpected non-tree root (seen)' ' + test_must_fail git rev-list --objects $blob $broken_commit \ + >output 2>&1 && + test_i18ngrep "not a tree" output ' test_expect_success 'setup unexpected non-commit tag' '