From patchwork Thu Aug 29 22:38:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jon Simons X-Patchwork-Id: 11122555 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A33751399 for ; Thu, 29 Aug 2019 22:46:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7FE53233FF for ; Thu, 29 Aug 2019 22:46:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=g001.emailsrvr.com header.i=@g001.emailsrvr.com header.b="DFFncFuc" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728761AbfH2WqT (ORCPT ); Thu, 29 Aug 2019 18:46:19 -0400 Received: from smtp123.ord1d.emailsrvr.com ([184.106.54.123]:35486 "EHLO smtp123.ord1d.emailsrvr.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728749AbfH2WqT (ORCPT ); Thu, 29 Aug 2019 18:46:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=g001.emailsrvr.com; s=20190322-9u7zjiwi; t=1567118300; bh=SWjQ5b8JutVPiwHd9KmLqAVpd4x7J9NFb2o5FhlmmwY=; h=From:To:Subject:Date:From; b=DFFncFucCaQ9AGCQbyboX9vVFheWRRFeG+o/KFMbgINjKWzgaqu3/NH7F/iAvpJZ5 NdAhktALMMWEOunsZ8iX2Ic8cuz/0hnHeguk6PtK7ogY1W3D/O9lUKp8iPuXfv53sk Ts6b+mynjB8LOxbNfocsY3K+nG6MoR39dAV4UVI8= X-Auth-ID: jon@jonsimons.org Received: by smtp24.relay.ord1d.emailsrvr.com (Authenticated sender: jon-AT-jonsimons.org) with ESMTPSA id B34A8A0252; Thu, 29 Aug 2019 18:38:19 -0400 (EDT) X-Sender-Id: jon@jonsimons.org Received: from localhost.localdomain (c-73-223-68-105.hsd1.ca.comcast.net [73.223.68.105]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256) by 0.0.0.0:465 (trex/5.7.12); Thu, 29 Aug 2019 18:38:20 -0400 From: Jon Simons To: jon@jonsimons.org, git@vger.kernel.org Cc: me@ttaylorr.com, peff@peff.net, sunshine@sunshineco.com, stolee@gmail.com Subject: [PATCH v2 1/2] list-objects-filter: only parse sparse OID when 'have_git_dir' Date: Thu, 29 Aug 2019 18:38:10 -0400 Message-Id: <20190829223811.12072-2-jon@jonsimons.org> X-Mailer: git-send-email 2.23.0.37.g745f681289.dirty In-Reply-To: <20190829223811.12072-1-jon@jonsimons.org> References: <20190829223811.12072-1-jon@jonsimons.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix a bug in partial cloning with sparse filters by ensuring to check for 'have_git_dir' before attempting to resolve the sparse filter OID. Otherwise the client will trigger: BUG: refs.c:1851: attempting to get main_ref_store outside of repository when attempting to git clone with a sparse filter. Note that this fix is the minimal one which avoids the BUG and allows for the clone to complete successfully: There is an open question as to whether there should be any attempt to resolve the OID provided by the client in this context, as a filter for the clone to be used on the remote side. For cases where local and remote OID resolutions differ, resolving on the client side could be considered a bug. For now, the minimal approach here is used to unblock further testing for partial clones with sparse filters, while a more invasive fix could make sense to pursue as a future direction. t5616 is updated to demonstrate the change. Signed-off-by: Jon Simons --- list-objects-filter-options.c | 3 ++- t/t5616-partial-clone.sh | 21 +++++++++++++++++++++ 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index 1cb20c659c..aaba312edb 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -71,7 +71,8 @@ static int gently_parse_list_objects_filter( * command, but DO NOT complain if we don't have the blob or * ref locally. */ - if (!get_oid_with_context(the_repository, v0, GET_OID_BLOB, + if (have_git_dir() && + !get_oid_with_context(the_repository, v0, GET_OID_BLOB, &sparse_oid, &oc)) filter_options->sparse_oid_value = oiddup(&sparse_oid); filter_options->choice = LOFC_SPARSE_OID; diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh index 565254558f..0c6f365bf2 100755 --- a/t/t5616-partial-clone.sh +++ b/t/t5616-partial-clone.sh @@ -241,6 +241,27 @@ test_expect_success 'fetch what is specified on CLI even if already promised' ' ! grep "?$(cat blob)" missing_after ' +test_expect_success 'setup src repo for sparse filter' ' + git init sparse-src && + git -C sparse-src config --local uploadpack.allowfilter 1 && + git -C sparse-src config --local uploadpack.allowanysha1inwant 1 && + for n in 1 2 3 4 + do + test_commit -C sparse-src "this-is-file-$n" file.$n.txt || return 1 + done && + test_write_lines /file1.txt /file3.txt >sparse-src/odd-files && + test_write_lines /file2.txt /file4.txt >sparse-src/even-files && + test_write_lines /* >sparse-src/all-files && + git -C sparse-src add odd-files even-files all-files && + git -C sparse-src commit -m "some sparse checkout files" +' + +test_expect_success 'partial clone with sparse filter succeeds' ' + git clone --no-local --no-checkout --filter=sparse:oid=master:all-files sparse-src pc-all && + git clone --no-local --no-checkout --filter=sparse:oid=master:even-files sparse-src pc-even && + git clone --no-local --no-checkout --filter=sparse:oid=master:odd-files sparse-src pc-odd +' + . "$TEST_DIRECTORY"/lib-httpd.sh start_httpd From patchwork Thu Aug 29 22:38:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jon Simons X-Patchwork-Id: 11122557 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 83BCC14E5 for ; Thu, 29 Aug 2019 22:46:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5FC7B233FF for ; Thu, 29 Aug 2019 22:46:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=g001.emailsrvr.com header.i=@g001.emailsrvr.com header.b="S9hgj4PU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728756AbfH2WqT (ORCPT ); Thu, 29 Aug 2019 18:46:19 -0400 Received: from smtp124.ord1d.emailsrvr.com ([184.106.54.124]:33061 "EHLO smtp124.ord1d.emailsrvr.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728272AbfH2WqS (ORCPT ); Thu, 29 Aug 2019 18:46:18 -0400 X-Greylist: delayed 478 seconds by postgrey-1.27 at vger.kernel.org; Thu, 29 Aug 2019 18:46:17 EDT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=g001.emailsrvr.com; s=20190322-9u7zjiwi; t=1567118302; bh=dIvzEvx1AoPh8FTDfDSwmiD4gAAuFOrubryc5M4bD+U=; h=From:To:Subject:Date:From; b=S9hgj4PUYVpLu8f6wSXXxLqeY6bf5qnDJ+Wst15n32oEvx6nRu1SmX10ocXcRpZA3 YkURFf9hpj4EylbBfhEG50IfaP2iNhcSY3uY4Fae+63QMtqlVGUSgYKS0agZlI6vkc +k/yutBuRR8yhpnTDWF7qOC+v1qFTAiVCZ4jiQ7Q= X-Auth-ID: jon@jonsimons.org Received: by smtp24.relay.ord1d.emailsrvr.com (Authenticated sender: jon-AT-jonsimons.org) with ESMTPSA id 6A409A025C; Thu, 29 Aug 2019 18:38:21 -0400 (EDT) X-Sender-Id: jon@jonsimons.org Received: from localhost.localdomain (c-73-223-68-105.hsd1.ca.comcast.net [73.223.68.105]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256) by 0.0.0.0:465 (trex/5.7.12); Thu, 29 Aug 2019 18:38:22 -0400 From: Jon Simons To: jon@jonsimons.org, git@vger.kernel.org Cc: me@ttaylorr.com, peff@peff.net, sunshine@sunshineco.com, stolee@gmail.com Subject: [PATCH v2 2/2] list-objects-filter: handle unresolved sparse filter OID Date: Thu, 29 Aug 2019 18:38:11 -0400 Message-Id: <20190829223811.12072-3-jon@jonsimons.org> X-Mailer: git-send-email 2.23.0.37.g745f681289.dirty In-Reply-To: <20190829223811.12072-1-jon@jonsimons.org> References: <20190829223811.12072-1-jon@jonsimons.org> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Handle a potential NULL 'sparse_oid_value' when attempting to load sparse filter exclusions by blob, to avoid segfaulting later during 'add_excludes_from_blob_to_list'. While here, uniquify the errors emitted to distinguish between the case that a given OID is NULL due to an earlier failure to resolve it, and when an OID resolves but parsing the sparse filter spec fails. t5616 is updated to demonstrate the change. Co-authored-by: Jeff King Signed-off-by: Jeff King Signed-off-by: Jon Simons --- list-objects-filter.c | 6 +++++- t/t5616-partial-clone.sh | 7 +++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/list-objects-filter.c b/list-objects-filter.c index 36e1f774bc..252fae5d4e 100644 --- a/list-objects-filter.c +++ b/list-objects-filter.c @@ -464,9 +464,13 @@ static void *filter_sparse_oid__init( { struct filter_sparse_data *d = xcalloc(1, sizeof(*d)); d->omits = omitted; + if (!filter_options->sparse_oid_value) + die(_("unable to read sparse filter specification from %s"), + filter_options->filter_spec); if (add_excludes_from_blob_to_list(filter_options->sparse_oid_value, NULL, 0, &d->el) < 0) - die("could not load filter specification"); + die(_("unable to parse sparse filter data in %s"), + oid_to_hex(filter_options->sparse_oid_value)); ALLOC_GROW(d->array_frame, d->nr + 1, d->alloc); d->array_frame[d->nr].defval = 0; /* default to include */ diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh index 0c6f365bf2..495998b7fb 100755 --- a/t/t5616-partial-clone.sh +++ b/t/t5616-partial-clone.sh @@ -262,6 +262,13 @@ test_expect_success 'partial clone with sparse filter succeeds' ' git clone --no-local --no-checkout --filter=sparse:oid=master:odd-files sparse-src pc-odd ' +test_expect_success 'partial clone with unresolvable sparse filter fails cleanly' ' + test_must_fail git clone --no-local --no-checkout --filter=sparse:oid=master:sparse-filter sparse-src sc1 2>err && + test_i18ngrep "unable to read sparse filter specification from sparse:oid=master:sparse-filter" err && + test_must_fail git clone --no-local --no-checkout --filter=sparse:oid=master sparse-src sc2 2>err && + test_i18ngrep "unable to parse sparse filter data in $(git -C sparse-src rev-parse master)" err +' + . "$TEST_DIRECTORY"/lib-httpd.sh start_httpd