From patchwork Wed Mar 9 16:01:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12775219 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 815AEC433F5 for ; Wed, 9 Mar 2022 16:02:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233737AbiCIQD1 (ORCPT ); Wed, 9 Mar 2022 11:03:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234248AbiCIQCy (ORCPT ); Wed, 9 Mar 2022 11:02:54 -0500 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21D7A17F699 for ; Wed, 9 Mar 2022 08:01:48 -0800 (PST) Received: by mail-wm1-x331.google.com with SMTP id k29-20020a05600c1c9d00b003817fdc0f00so1748655wms.4 for ; Wed, 09 Mar 2022 08:01:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=7/Vet349YwWXOD88+eWNxKjf9Jhn0cuBQkzK8diKGoU=; b=q4hpUf0Q47d1OmEAvIkW8tVQ15X+I7Sj3u2DURYD5Q6kT9GGFDiOPWBq8PiTkitvn7 6n7UGDJEm3l68bf05mjSX6wRKZJlkfoF/KSAOtpGIJT4+CdDWRwa1bSo8Sn5EmfPtOd0 rGgSPWk8i4yhU50/1BQ1JuQWyxJCpe8ZcZo2pbMMzmruwcpu6KEv7B278Y2FfeMWE99T /hWrnNDcZQ4waaqISyM2tFqyFkanCpE0s2Ll2IJ4o8oBTfK2FrS7jC1mclVP+bvNh7Bl dG4ERmRhzxMjrYOa7+IKrXAbPxgoNo4gc1IHSQy5rBU6dM+5CGU566Cy1KB1mkZ8WI0Y CGog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=7/Vet349YwWXOD88+eWNxKjf9Jhn0cuBQkzK8diKGoU=; b=IFFkA7rctMetKCBT2cAfaCAh0bRXumVoorL4hwENmhhY3YVObof0kwQhYw4T2qzyae CiAIPUG1r8VRX0YG2A6y4cVFuJa6aB8TrB2bIUFJ961juVWkprwC9hLGs7JvdrSLgWGd 1oafF6oZOignpkEa0C82ASnrIWk9GIHAF7V8Wyg7VAZT9HJGKYZyrPj5dJ7w8WB3V6Dk yasVOnwuExEP/s/+eZTVecj33WHal7SYhfm7QZPR3uox5XgYbVBcpKffDK2c3+K7I5Qn GwFgmnzI2TF+6QsxITOoRAfXRiikxv4HpvI3PP+YIl7kqrSe6RAgtGVvbA1RYWPp1yDY q7Pg== X-Gm-Message-State: AOAM532mF2mvrYmQvFmn+ySF+pfDFlgqKAsfBmCVzJoPplRto+qy8bSI SD3P9Oe2cjcM5QNtGlYqfepsamtY1cM= X-Google-Smtp-Source: ABdhPJzdPfSodIppS9fant2JbYMG+912kU4KDPnrsZGNIe2J3vZ/AEFPP40MJNknu46FOLelzOfu4Q== X-Received: by 2002:a7b:c253:0:b0:389:bbf4:f431 with SMTP id b19-20020a7bc253000000b00389bbf4f431mr8108679wmj.65.1646841706911; Wed, 09 Mar 2022 08:01:46 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p18-20020adfba92000000b001e4ae791663sm2121056wrg.62.2022.03.09.08.01.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Mar 2022 08:01:46 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 09 Mar 2022 16:01:31 +0000 Subject: [PATCH v4 01/13] index-pack: document and test the --promisor option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Jeff Hostetler , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The --promisor option of 'git index-pack' was created in 88e2f9e (introduce fetch-object: fetch one promisor object, 2017-12-05) but was untested. It is currently unused within the Git codebase, but that will change in an upcoming change to 'git bundle unbundle' when there is a filter capability. For now, add documentation about the option and add a test to ensure it is working as expected. Signed-off-by: Derrick Stolee --- Documentation/git-index-pack.txt | 8 ++++++++ t/t5300-pack-object.sh | 4 +++- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/Documentation/git-index-pack.txt b/Documentation/git-index-pack.txt index 1f1e3592251..4e71c256ecb 100644 --- a/Documentation/git-index-pack.txt +++ b/Documentation/git-index-pack.txt @@ -122,6 +122,14 @@ This option cannot be used with --stdin. + include::object-format-disclaimer.txt[] +--promisor[=]:: + Before committing the pack-index, create a .promisor file for this + pack. Particularly helpful when writing a promisor pack with --fix-thin + since the name of the pack is not final until the pack has been fully + written. If a `` is provided, then that content will be + written to the .promisor file for future reference. See + link:technical/partial-clone.html[partial clone] for more information. + NOTES ----- diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh index 2fd845187e7..a11d61206ad 100755 --- a/t/t5300-pack-object.sh +++ b/t/t5300-pack-object.sh @@ -315,8 +315,10 @@ test_expect_success \ git index-pack -o tmp.idx test-3.pack && cmp tmp.idx test-1-${packname_1}.idx && - git index-pack test-3.pack && + git index-pack --promisor=message test-3.pack && cmp test-3.idx test-1-${packname_1}.idx && + echo message >expect && + test_cmp expect test-3.promisor && cat test-2-${packname_2}.pack >test-3.pack && git index-pack -o tmp.idx test-2-${packname_2}.pack && From patchwork Wed Mar 9 16:01:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12775225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9FB7C4332F for ; Wed, 9 Mar 2022 16:02:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230311AbiCIQDo (ORCPT ); Wed, 9 Mar 2022 11:03:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234247AbiCIQCy (ORCPT ); Wed, 9 Mar 2022 11:02:54 -0500 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4759B17F6A4 for ; Wed, 9 Mar 2022 08:01:50 -0800 (PST) Received: by mail-wm1-x332.google.com with SMTP id k29-20020a05600c1c9d00b003817fdc0f00so1748732wms.4 for ; Wed, 09 Mar 2022 08:01:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=nhoz//lgtAOsg7zwaxkPxCWbsvzxKtOa6k0srOiAd7k=; b=LhTaBZdbwVrU2gIYkbkDjY5U3yaHC7VhXZy4mJHnspL33FmqPUHuW53yNo9oFYu6b7 5melvROP31/E28vhs4nt6mafFCJyDTyKazG44bxH8xdGxhkQIKN2xZMoQQ+DmWsbJIpi 6pWwHrXBj3sne6bFHMASKvwrrIzrwwWpGnmngvCf8QF1HjyrghgqKx+wf+p6jz9h2CQC QwoFRvIPd5Vvp6vNRAYCAvzaVxxqQzTCISQbEGKGxjDuwDgLGTEQA/9olyh/lhTs/3nh emfLD2pLXb8h0BSU6VthrE3kN7v693O9lEBD7DIUkNXSxtUvdOpvmjxzSnKLXDHIDLZd uaIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=nhoz//lgtAOsg7zwaxkPxCWbsvzxKtOa6k0srOiAd7k=; b=uZkgFMpTGJfY8hWT8vTeJ68im1ocJZj2FdEGz7xMPUOXeb9dk0unGRrJSo57oh36Vr AqjTnoAzcQMRsK6SfaIhtzkjue41C+tkVHegOHRXloqSohVaxoATuwcSMhLqGb59kxbb U31JTcMgG/w+93iYX3wXJtCdsYhGtIhyPWDTAtDigM8cUV+OcP5mWzF7beeI7leZ+eI/ 7Xn8mpfpFB4MKjStGDgIxyTvSMcH7lDcG/gIm5GxrjZIZ4gw4gZJb03CTdPmv1UsTzQy MdTSlxAtNFmEiqYzavRsd3OmG8ATuD0SBNO0r+jSiOJ+0R0VisTlh/HuPmyKPIjiHO5B 3Fyg== X-Gm-Message-State: AOAM530qm6N1V1rQ6yeFiB8DxUkhd9PhhqIKVDpLqvGLpdC65OfhL7r1 unDPR3kcNEYa/dYlPt2VPMD6mg0WwIc= X-Google-Smtp-Source: ABdhPJxZRi1Oa/DmwspucF4pTCSxylN3mo+jhalPzu4ZkpeV9DhPWbWBJwvym1XuS3JKgrZOxKte6g== X-Received: by 2002:a7b:c04e:0:b0:380:ead9:254a with SMTP id u14-20020a7bc04e000000b00380ead9254amr8086702wmc.58.1646841708055; Wed, 09 Mar 2022 08:01:48 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r187-20020a1c44c4000000b0038377fb18f8sm3121207wma.5.2022.03.09.08.01.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Mar 2022 08:01:47 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 09 Mar 2022 16:01:32 +0000 Subject: [PATCH v4 02/13] list-objects-filter-options: create copy helper Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Jeff Hostetler , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee As we add more embedded members with type 'struct list_objects_filter_options', it will be important to easily perform a deep copy across multiple such structs. Create list_objects_filter_copy() to satisfy this need. This method is recursive to match the recursive nature of the struct. Signed-off-by: Derrick Stolee --- list-objects-filter-options.c | 19 +++++++++++++++++++ list-objects-filter-options.h | 4 ++++ 2 files changed, 23 insertions(+) diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index fd8d59f653a..449d53af69f 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -415,3 +415,22 @@ void partial_clone_get_default_filter_spec( &errbuf); strbuf_release(&errbuf); } + +void list_objects_filter_copy( + struct list_objects_filter_options *dest, + const struct list_objects_filter_options *src) +{ + int i; + struct string_list_item *item; + + /* Copy everything. We will overwrite the pointers shortly. */ + memcpy(dest, src, sizeof(struct list_objects_filter_options)); + + string_list_init_dup(&dest->filter_spec); + for_each_string_list_item(item, &src->filter_spec) + string_list_append(&dest->filter_spec, item->string); + + ALLOC_ARRAY(dest->sub, dest->sub_alloc); + for (i = 0; i < src->sub_nr; i++) + list_objects_filter_copy(&dest->sub[i], &src->sub[i]); +} diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h index da5b6737e27..425c38cae9d 100644 --- a/list-objects-filter-options.h +++ b/list-objects-filter-options.h @@ -132,4 +132,8 @@ void partial_clone_get_default_filter_spec( struct list_objects_filter_options *filter_options, const char *remote); +void list_objects_filter_copy( + struct list_objects_filter_options *dest, + const struct list_objects_filter_options *src); + #endif /* LIST_OBJECTS_FILTER_OPTIONS_H */ From patchwork Wed Mar 9 16:01:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12775218 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E1E1C4321E for ; Wed, 9 Mar 2022 16:02:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232943AbiCIQDZ (ORCPT ); Wed, 9 Mar 2022 11:03:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40688 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234258AbiCIQCy (ORCPT ); Wed, 9 Mar 2022 11:02:54 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7E6FE17F6A6 for ; Wed, 9 Mar 2022 08:01:51 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id l10so1669320wmb.0 for ; Wed, 09 Mar 2022 08:01:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=pltF+US/IVTKHV5tEnt42SamcuMBECOoZ121AD99wBA=; b=MDbZniDFT1GEqOypZqyZhFLcfKb/l1eXzODQLOeFaxl1A+0CtRwLByxzPjjEtPwYrO ClKz/nTxZBPqWjUSvutTzxsWg/qI9Nc4yglSCkhsjniaGK341aiXTjQKtvz5upmPvACi RYdjZhl2XJr2FxO3SvN8ZA7m1lhbmRstW+7GngRQQEmMU+9gqvqHX6ktElBI1KKONOJt yegbGvr+Wio4ElLw+y3KbiSJLkC/NuTKXbiDUD0+qa8pLzK7NwcfOBi4OrMSfvsSo5tX DLoCh9RswaPLPJLLKwMvOe/RkOKsJ6rEvndOy97I+GyY1ZcD3a8VUqn9lYYfOlwZsbgo WPow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=pltF+US/IVTKHV5tEnt42SamcuMBECOoZ121AD99wBA=; b=2R8eaQwqPk47EhsDW38RyozCyGrxmJH593PWZXDNLH+whmjIDDZGpc7Twk203Guh/A VZZPaMlsFsHgE2SilUrbtdXWETZXmU1jy2cIB7S0apLG115xHWHwETPjly1Ra5BOUaRr xKuKOrmYl+ndvZY04dh3UEOd9JYqplMavhNfOecIY11CXw8mTOt8AtJyuK5uFY5CTIZC /ZsYj6sffNRszbqgF866gf9ICWpL0sQn5RFdDPB8L1dYMD3ZS9kl/HkOn0M6/LHyl0nb rrU6ozauorTxGdNLtAh9YIvKtIQT+5CxAtAJX3YAb2ExGUrQe2Yx32X8fTq3Ck1gw8Py 6QrA== X-Gm-Message-State: AOAM531v+4F6iD5M726TL/wnn0lRI1UkJXO/8pv9T7410jcawxUidEw7 XdDLjrjPJFmHWZDSkoOWdwfILhAEcvc= X-Google-Smtp-Source: ABdhPJwefZyqvltdQz0dPoxCQ0e6dl2birMZU7q5a0KvZ/dRUMY1eDrpzOmCqUdl2eMSdPDwj6b/sw== X-Received: by 2002:a05:600c:1c09:b0:389:d0a8:d3b7 with SMTP id j9-20020a05600c1c0900b00389d0a8d3b7mr50653wms.92.1646841709675; Wed, 09 Mar 2022 08:01:49 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h10-20020adf9cca000000b001f0236ba918sm1997660wre.35.2022.03.09.08.01.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Mar 2022 08:01:48 -0800 (PST) Message-Id: <4ac09ddbfaa3f8aad2b77c7b250f6953a3ff9f24.1646841703.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 09 Mar 2022 16:01:33 +0000 Subject: [PATCH v4 03/13] revision: put object filter into struct rev_info Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Jeff Hostetler , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Placing a 'struct list_objects_filter_options' within 'struct rev_info' will assist making some bookkeeping around object filters in the future. For now, let's use this new member to remove a static global instance of the struct from builtin/rev-list.c. Signed-off-by: Derrick Stolee --- builtin/rev-list.c | 26 ++++++++++++-------------- revision.h | 7 +++++++ 2 files changed, 19 insertions(+), 14 deletions(-) diff --git a/builtin/rev-list.c b/builtin/rev-list.c index 777558e9b06..1beb578cc51 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -62,7 +62,6 @@ static const char rev_list_usage[] = static struct progress *progress; static unsigned progress_counter; -static struct list_objects_filter_options filter_options; static struct oidset omitted_objects; static int arg_print_omitted; /* print objects omitted by filter */ @@ -400,7 +399,6 @@ static inline int parse_missing_action_value(const char *value) } static int try_bitmap_count(struct rev_info *revs, - struct list_objects_filter_options *filter, int filter_provided_objects) { uint32_t commit_count = 0, @@ -436,7 +434,8 @@ static int try_bitmap_count(struct rev_info *revs, */ max_count = revs->max_count; - bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects); + bitmap_git = prepare_bitmap_walk(revs, &revs->filter, + filter_provided_objects); if (!bitmap_git) return -1; @@ -453,7 +452,6 @@ static int try_bitmap_count(struct rev_info *revs, } static int try_bitmap_traversal(struct rev_info *revs, - struct list_objects_filter_options *filter, int filter_provided_objects) { struct bitmap_index *bitmap_git; @@ -465,7 +463,8 @@ static int try_bitmap_traversal(struct rev_info *revs, if (revs->max_count >= 0) return -1; - bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects); + bitmap_git = prepare_bitmap_walk(revs, &revs->filter, + filter_provided_objects); if (!bitmap_git) return -1; @@ -475,7 +474,6 @@ static int try_bitmap_traversal(struct rev_info *revs, } static int try_bitmap_disk_usage(struct rev_info *revs, - struct list_objects_filter_options *filter, int filter_provided_objects) { struct bitmap_index *bitmap_git; @@ -483,7 +481,7 @@ static int try_bitmap_disk_usage(struct rev_info *revs, if (!show_disk_usage) return -1; - bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects); + bitmap_git = prepare_bitmap_walk(revs, &revs->filter, filter_provided_objects); if (!bitmap_git) return -1; @@ -597,13 +595,13 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) } if (skip_prefix(arg, ("--" CL_ARG__FILTER "="), &arg)) { - parse_list_objects_filter(&filter_options, arg); - if (filter_options.choice && !revs.blob_objects) + parse_list_objects_filter(&revs.filter, arg); + if (revs.filter.choice && !revs.blob_objects) die(_("object filtering requires --objects")); continue; } if (!strcmp(arg, ("--no-" CL_ARG__FILTER))) { - list_objects_filter_set_no_filter(&filter_options); + list_objects_filter_set_no_filter(&revs.filter); continue; } if (!strcmp(arg, "--filter-provided-objects")) { @@ -688,11 +686,11 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) progress = start_delayed_progress(show_progress, 0); if (use_bitmap_index) { - if (!try_bitmap_count(&revs, &filter_options, filter_provided_objects)) + if (!try_bitmap_count(&revs, filter_provided_objects)) return 0; - if (!try_bitmap_disk_usage(&revs, &filter_options, filter_provided_objects)) + if (!try_bitmap_disk_usage(&revs, filter_provided_objects)) return 0; - if (!try_bitmap_traversal(&revs, &filter_options, filter_provided_objects)) + if (!try_bitmap_traversal(&revs, filter_provided_objects)) return 0; } @@ -733,7 +731,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) oidset_init(&missing_objects, DEFAULT_OIDSET_SIZE); traverse_commit_list_filtered( - &filter_options, &revs, show_commit, show_object, &info, + &revs.filter, &revs, show_commit, show_object, &info, (arg_print_omitted ? &omitted_objects : NULL)); if (arg_print_omitted) { diff --git a/revision.h b/revision.h index 3c58c18c63a..b1669a8cc33 100644 --- a/revision.h +++ b/revision.h @@ -8,6 +8,7 @@ #include "pretty.h" #include "diff.h" #include "commit-slab-decl.h" +#include "list-objects-filter-options.h" /** * The revision walking API offers functions to build a list of revisions @@ -94,6 +95,12 @@ struct rev_info { /* The end-points specified by the end user */ struct rev_cmdline_info cmdline; + /* + * Object filter options. No filtering is specified + * if and only if filter.choice is zero. + */ + struct list_objects_filter_options filter; + /* excluding from --branches, --refs, etc. expansion */ struct string_list *ref_excludes; From patchwork Wed Mar 9 16:01:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12775222 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BD22C433EF for ; Wed, 9 Mar 2022 16:02:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234282AbiCIQDf (ORCPT ); Wed, 9 Mar 2022 11:03:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234252AbiCIQCy (ORCPT ); Wed, 9 Mar 2022 11:02:54 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9260917F6B5 for ; Wed, 9 Mar 2022 08:01:52 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id r6so3345282wrr.2 for ; Wed, 09 Mar 2022 08:01:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=qvpX/0xqAMbcZk3Asv2E/I9bjNoKDUuKRq3KG+b3yEw=; b=TrUGb/rucz6KEpUE44KzlxheQldvDzDCvOYL8f2fQ+OaIyCh2YUK0Pqi26A9EBL5qJ XqfXD6Ewq5EcNNLO8YlKAUNU1eKPco+Ug4RfRGSrXoEIlFWkhgwZh0QoTd1S0Tm4SHuN Q3xS9KYxMZDGEPUk8tttX83pwBOLdVgZL+bM0+Y9eCaMA99Xn9s4ov8Rw2193/vJP9lW ttmwjFyhwIkysGZxrkRLMq7aoasARuA2cybtoznZ2XHQ/dF81Vr9yjkf6Dro6wkTSWsU xKkJx/J+3Rwfx+gaz+HEzWMngZnoa0EjY1CX1k+ShMFaiy+RjCuzKdSxdWKZXudsxQhH CvFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=qvpX/0xqAMbcZk3Asv2E/I9bjNoKDUuKRq3KG+b3yEw=; b=SFIs9UodIb9fM57ozsF2LfaHDxjG4z5vJ1mayLv4seaoPTBkeQBC5KP45pr4BrOihf m4TkMwfnisKPjmEFzYAnLCll7Eme9cuiQ6dQ7IqEfgUl6noGFQxcnVVx8WqnkJd9a4IX lLUo3Ydcgci7xVVqttd+afeAJmwhViCBEBzKA0xZzWxWewAPGaZU+cXdHmc/ja6X3ZJk jUx93pW8aPcF+/k8ZNwUUlM+o9pZcc4wFgs1omM2uYTElMwgk8Q9Klo0RIJW5kXjDf9L 6FDeeDbEpW/kTGnCb+JG0cDkyp6dx6zsO07PBEtMLE6IXQDGFoimz96EWHTshzCHyspQ fN9A== X-Gm-Message-State: AOAM531kjruPhiYGBlV1DPBuCMW2zkS+5znrP2p6l3x7ZtHb7Alfb9fk 81MYYxywqVF3S3H66Hz7LNYDM1U29QA= X-Google-Smtp-Source: ABdhPJyRM+NqCg2nI3hNl8oDk47hVMH4VRv74HVtooi3/YI5Hsgg+8O369ot9w8A8suSVMr38vIBcQ== X-Received: by 2002:adf:fa8c:0:b0:1f0:47ef:a04 with SMTP id h12-20020adffa8c000000b001f047ef0a04mr238254wrr.398.1646841710520; Wed, 09 Mar 2022 08:01:50 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id bg20-20020a05600c3c9400b0037fa5c422c8sm5949334wmb.48.2022.03.09.08.01.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Mar 2022 08:01:50 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 09 Mar 2022 16:01:34 +0000 Subject: [PATCH v4 04/13] pack-objects: use rev.filter when possible Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Jeff Hostetler , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee In builtin/pack-objects.c, we use a 'filter_options' global to populate the --filter= argument. The previous change created a pointer to a filter option in 'struct rev_info', so we can use that pointer here as a start to simplifying some usage of object filters. Signed-off-by: Derrick Stolee --- builtin/pack-objects.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index ba2006f2212..e5b7d015d7d 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -3651,7 +3651,7 @@ static int pack_options_allow_reuse(void) static int get_object_list_from_bitmap(struct rev_info *revs) { - if (!(bitmap_git = prepare_bitmap_walk(revs, &filter_options, 0))) + if (!(bitmap_git = prepare_bitmap_walk(revs, &revs->filter, 0))) return -1; if (pack_options_allow_reuse() && @@ -3727,6 +3727,7 @@ static void get_object_list(int ac, const char **av) repo_init_revisions(the_repository, &revs, NULL); save_commit_buffer = 0; setup_revisions(ac, av, &revs, &s_r_opt); + list_objects_filter_copy(&revs.filter, &filter_options); /* make sure shallows are read */ is_repository_shallow(the_repository); @@ -3777,7 +3778,7 @@ static void get_object_list(int ac, const char **av) if (!fn_show_object) fn_show_object = show_object; - traverse_commit_list_filtered(&filter_options, &revs, + traverse_commit_list_filtered(&revs.filter, &revs, show_commit, fn_show_object, NULL, NULL); From patchwork Wed Mar 9 16:01:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12775224 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C88CC433F5 for ; Wed, 9 Mar 2022 16:02:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234295AbiCIQDm (ORCPT ); Wed, 9 Mar 2022 11:03:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234263AbiCIQCz (ORCPT ); Wed, 9 Mar 2022 11:02:55 -0500 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48F2217F6BC for ; Wed, 9 Mar 2022 08:01:53 -0800 (PST) Received: by mail-wm1-x329.google.com with SMTP id r187-20020a1c2bc4000000b003810e6b192aso1770126wmr.1 for ; Wed, 09 Mar 2022 08:01:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ywH7XXe2/CGU4iPM53RQJuTtpc/9HPR86S2+Kpmyxlk=; b=aGaG5IbwEhYd6kd1/DK/OUo5h3hdHIyms2gM9jiNtPlgD7Uv62kOdhWLaeaEtu84gE V8/ttksbOegjz1qYv9t25V2e45nkUiMOr6gfVXMmSGLwFmPis9io9T+xHfSgW1r0j92m MFGGXp1P8gMyownP4EA5pibHZsYSNr0DR0NOT+o4r1hj6aZWj8UTw++k1LTwG6yW4+vR C7fAuToyETp74iFdLjd0iqOXz9BJPxqF8b/RwbdOUsY9/qdZEpckc5nYYRhOdisiXlaU A6fqCbUofhwot3tcquFwf8NJYI1Cgc2pNb1+jpeaiAshp6F87UI3ahbR4eHDfBY2o46A Ro4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ywH7XXe2/CGU4iPM53RQJuTtpc/9HPR86S2+Kpmyxlk=; b=neULsh7m+d7E9tpS6h/yNDzTXM8CsgDkAlOC4wabICbSNUGScU4iSU98luof0EI0bp WhSfmputgcZkVPNwI+KlMvKhnklIFkvoZfYcrhib8q7NV50XkmZTaRblo7F0RrGACxmX Anf1xx7P1Wxnot2qCraNHVcQf5crVa14lQHWJ+oSGJ7I/IpuMk8oVsio7VMRDmtI21My F0g2UdzrRiir5ixIcCYmHgmF38AUSmkmnzwUISdCRTYSIPy1C3PtdrX8Sn+ntMjXssc7 qCJv7GchiT38bM0Im5gjMLFkNX83x3wg+Aj7rWdfJvZcdg7xMWlFl9Trt6JqpW2FfTRd BfuA== X-Gm-Message-State: AOAM532yCwzZKCDjC6St2LiK6iZNfG2+xMV6h3fd7cuZYc4o9p+vZV6g Dkj5VGy47R5wlkEmp6hNA84TP+kpnl0= X-Google-Smtp-Source: ABdhPJyZvzKcMkkNE9Jzmzaeip4fxNc1dnhQ8NDMRxw6VoMtzbfC1e8/2+AhXvmdBJkX3m/4lTXZaw== X-Received: by 2002:a05:600c:214a:b0:385:9c42:848 with SMTP id v10-20020a05600c214a00b003859c420848mr42881wml.176.1646841711627; Wed, 09 Mar 2022 08:01:51 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i74-20020adf90d0000000b0020373ba7beesm3042643wri.0.2022.03.09.08.01.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Mar 2022 08:01:51 -0800 (PST) Message-Id: <346baa78ec515b5cd808f4fbd677632073b7ddd1.1646841703.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 09 Mar 2022 16:01:35 +0000 Subject: [PATCH v4 05/13] pack-bitmap: drop filter in prepare_bitmap_walk() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Jeff Hostetler , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Now that all consumers of prepare_bitmap_walk() have populated the 'filter' member of 'struct rev_info', we can drop that extra parameter from the method and access it directly from the 'struct rev_info'. Signed-off-by: Derrick Stolee --- builtin/pack-objects.c | 2 +- builtin/rev-list.c | 8 +++----- pack-bitmap.c | 20 +++++++++----------- pack-bitmap.h | 2 -- reachable.c | 2 +- 5 files changed, 14 insertions(+), 20 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index e5b7d015d7d..bafce542778 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -3651,7 +3651,7 @@ static int pack_options_allow_reuse(void) static int get_object_list_from_bitmap(struct rev_info *revs) { - if (!(bitmap_git = prepare_bitmap_walk(revs, &revs->filter, 0))) + if (!(bitmap_git = prepare_bitmap_walk(revs, 0))) return -1; if (pack_options_allow_reuse() && diff --git a/builtin/rev-list.c b/builtin/rev-list.c index 1beb578cc51..ab7558bd66a 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -434,8 +434,7 @@ static int try_bitmap_count(struct rev_info *revs, */ max_count = revs->max_count; - bitmap_git = prepare_bitmap_walk(revs, &revs->filter, - filter_provided_objects); + bitmap_git = prepare_bitmap_walk(revs, filter_provided_objects); if (!bitmap_git) return -1; @@ -463,8 +462,7 @@ static int try_bitmap_traversal(struct rev_info *revs, if (revs->max_count >= 0) return -1; - bitmap_git = prepare_bitmap_walk(revs, &revs->filter, - filter_provided_objects); + bitmap_git = prepare_bitmap_walk(revs, filter_provided_objects); if (!bitmap_git) return -1; @@ -481,7 +479,7 @@ static int try_bitmap_disk_usage(struct rev_info *revs, if (!show_disk_usage) return -1; - bitmap_git = prepare_bitmap_walk(revs, &revs->filter, filter_provided_objects); + bitmap_git = prepare_bitmap_walk(revs, filter_provided_objects); if (!bitmap_git) return -1; diff --git a/pack-bitmap.c b/pack-bitmap.c index 9c666cdb8bd..37fa4905796 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -739,8 +739,7 @@ static int add_commit_to_bitmap(struct bitmap_index *bitmap_git, static struct bitmap *find_objects(struct bitmap_index *bitmap_git, struct rev_info *revs, struct object_list *roots, - struct bitmap *seen, - struct list_objects_filter_options *filter) + struct bitmap *seen) { struct bitmap *base = NULL; int needs_walk = 0; @@ -823,7 +822,7 @@ static struct bitmap *find_objects(struct bitmap_index *bitmap_git, show_data.bitmap_git = bitmap_git; show_data.base = base; - traverse_commit_list_filtered(filter, revs, + traverse_commit_list_filtered(&revs->filter, revs, show_commit, show_object, &show_data, NULL); @@ -1219,7 +1218,6 @@ static int can_filter_bitmap(struct list_objects_filter_options *filter) } struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, - struct list_objects_filter_options *filter, int filter_provided_objects) { unsigned int i; @@ -1240,7 +1238,7 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, if (revs->prune) return NULL; - if (!can_filter_bitmap(filter)) + if (!can_filter_bitmap(&revs->filter)) return NULL; /* try to open a bitmapped pack, but don't parse it yet @@ -1297,8 +1295,7 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, if (haves) { revs->ignore_missing_links = 1; - haves_bitmap = find_objects(bitmap_git, revs, haves, NULL, - filter); + haves_bitmap = find_objects(bitmap_git, revs, haves, NULL); reset_revision_walk(); revs->ignore_missing_links = 0; @@ -1306,8 +1303,7 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, BUG("failed to perform bitmap walk"); } - wants_bitmap = find_objects(bitmap_git, revs, wants, haves_bitmap, - filter); + wants_bitmap = find_objects(bitmap_git, revs, wants, haves_bitmap); if (!wants_bitmap) BUG("failed to perform bitmap walk"); @@ -1315,8 +1311,10 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, if (haves_bitmap) bitmap_and_not(wants_bitmap, haves_bitmap); - filter_bitmap(bitmap_git, (filter && filter_provided_objects) ? NULL : wants, - wants_bitmap, filter); + filter_bitmap(bitmap_git, + (revs->filter.choice && filter_provided_objects) ? NULL : wants, + wants_bitmap, + &revs->filter); bitmap_git->result = wants_bitmap; bitmap_git->haves = haves_bitmap; diff --git a/pack-bitmap.h b/pack-bitmap.h index 19a63fa1abc..3d3ddd77345 100644 --- a/pack-bitmap.h +++ b/pack-bitmap.h @@ -10,7 +10,6 @@ struct commit; struct repository; struct rev_info; -struct list_objects_filter_options; static const char BITMAP_IDX_SIGNATURE[] = {'B', 'I', 'T', 'M'}; @@ -54,7 +53,6 @@ void test_bitmap_walk(struct rev_info *revs); int test_bitmap_commits(struct repository *r); int test_bitmap_hashes(struct repository *r); struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, - struct list_objects_filter_options *filter, int filter_provided_objects); uint32_t midx_preferred_pack(struct bitmap_index *bitmap_git); int reuse_partial_packfile_from_bitmap(struct bitmap_index *, diff --git a/reachable.c b/reachable.c index 84e3d0d75ed..b9f4ad886ef 100644 --- a/reachable.c +++ b/reachable.c @@ -205,7 +205,7 @@ void mark_reachable_objects(struct rev_info *revs, int mark_reflog, cp.progress = progress; cp.count = 0; - bitmap_git = prepare_bitmap_walk(revs, NULL, 0); + bitmap_git = prepare_bitmap_walk(revs, 0); if (bitmap_git) { traverse_bitmap_commit_list(bitmap_git, revs, mark_object_seen); free_bitmap_index(bitmap_git); From patchwork Wed Mar 9 16:01:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12775217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29FA5C43219 for ; Wed, 9 Mar 2022 16:02:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231627AbiCIQDW (ORCPT ); Wed, 9 Mar 2022 11:03:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40768 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234275AbiCIQC4 (ORCPT ); Wed, 9 Mar 2022 11:02:56 -0500 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8135E17AEFD for ; Wed, 9 Mar 2022 08:01:54 -0800 (PST) Received: by mail-wr1-x429.google.com with SMTP id t11so3777834wrm.5 for ; Wed, 09 Mar 2022 08:01:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=SkhrWJVmdGpCICyM+CSYFhQSLcZUmg6mmBvJFAffoaU=; b=aUylwLurY8xoTeejrVWre2ubCwby31NWA2FRqTi3gZ6WDnmdFaGT9Xm3eTR1SnDgtD jlbVHr1is4tiT/dSgSXfbfAYgI7z9YB9Mcuwofaixw+oehn3NTrXf296fyJOQTcMD3JB /Kze16c9Ev6aNVIxAPR1pq3jl6K1RhgTMr59X4hzxWh/tnzXjazG/NfV/Jwg0+YIFSUq Bv+TGWa5pFJcen8H+AhU+8a0W5gUm4gfUukJcE6tjBbX56gtVP+pl5X7qZnzM4v8cgUv BCdDpcl/WuxLBe7h8rDWPfg9dyxTCWc1yyqAGnCiUxCeteOYvpDRlih4k0aKdNRXHQMd 0OYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=SkhrWJVmdGpCICyM+CSYFhQSLcZUmg6mmBvJFAffoaU=; b=IEhnphRoXJ9BXxjUP1dw6YJ2Ai5Wql2sSmGTNdNqktNpmgh8Y0u0hI3stpGKkraCTQ AIUdnTYzLbnygoumri6T+wE+/FzyopLhjQkyUsi4+oX692VVbi/ESkkIj2ZHB+uiM9aC Z4AancH4MMQRVRmnA+BDXDzi+M/QJa53Z4B5dMEeNo67wE+rMgXEBM1RUHdLlEYPLAU/ YLtEv+s7eqKXhqX/OUox9cI7vCvtBfjbGgJZYEBZrZ5c3jT3mp7Xrmv25/2bXYos0p2C CFufqhD1sjS1nNYD2l2/2k0DCFf3l8nieM9Y5deeAMZOsvS+4DQ/6vAv5Ip7dUYCxYPm MoGA== X-Gm-Message-State: AOAM531FdPF9yjg+afxvNiNZdI0Xp9pj0qmOhu+2lt9f1qdtqrySQSMC lg8/StPnVagogD1CIYz1py7M38CjERE= X-Google-Smtp-Source: ABdhPJzho6/w+On9q8bz+jXhZeCg/Tmxh/5QPa7h37EyAIoqKNsJxl+RWbVXrPXBaHrj4vSGrTp2OQ== X-Received: by 2002:adf:c792:0:b0:1f0:769:9ef3 with SMTP id l18-20020adfc792000000b001f007699ef3mr236295wrg.336.1646841712564; Wed, 09 Mar 2022 08:01:52 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y4-20020adff144000000b001f022290737sm1962437wro.6.2022.03.09.08.01.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Mar 2022 08:01:52 -0800 (PST) Message-Id: <34afea8fcd620e8dd1e95f3e2918b47e226fff23.1646841704.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 09 Mar 2022 16:01:36 +0000 Subject: [PATCH v4 06/13] list-objects: consolidate traverse_commit_list[_filtered] Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Jeff Hostetler , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Now that all consumers of traverse_commit_list_filtered() populate the 'filter' member of 'struct rev_info', we can drop that parameter from the method prototype to simplify things. In addition, the only thing different now between traverse_commit_list_filtered() and traverse_commit_list() is the presence of the 'omitted' parameter, which is only non-NULL for one caller. We can consolidate these two methods by having one call the other and use the simpler form everywhere the 'omitted' parameter would be NULL. Signed-off-by: Derrick Stolee --- builtin/pack-objects.c | 6 +++--- builtin/rev-list.c | 2 +- list-objects.c | 34 ++++++++++++---------------------- list-objects.h | 12 ++++++++++-- pack-bitmap.c | 6 +++--- 5 files changed, 29 insertions(+), 31 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index bafce542778..b18724e32a3 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -3778,9 +3778,9 @@ static void get_object_list(int ac, const char **av) if (!fn_show_object) fn_show_object = show_object; - traverse_commit_list_filtered(&revs.filter, &revs, - show_commit, fn_show_object, NULL, - NULL); + traverse_commit_list(&revs, + show_commit, fn_show_object, + NULL); if (unpack_unreachable_expiration) { revs.ignore_missing_links = 1; diff --git a/builtin/rev-list.c b/builtin/rev-list.c index ab7558bd66a..ec433cb6d37 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -729,7 +729,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) oidset_init(&missing_objects, DEFAULT_OIDSET_SIZE); traverse_commit_list_filtered( - &revs.filter, &revs, show_commit, show_object, &info, + &revs, show_commit, show_object, &info, (arg_print_omitted ? &omitted_objects : NULL)); if (arg_print_omitted) { diff --git a/list-objects.c b/list-objects.c index 2f623f82115..117f734398c 100644 --- a/list-objects.c +++ b/list-objects.c @@ -416,35 +416,25 @@ static void do_traverse(struct traversal_context *ctx) strbuf_release(&csp); } -void traverse_commit_list(struct rev_info *revs, - show_commit_fn show_commit, - show_object_fn show_object, - void *show_data) -{ - struct traversal_context ctx; - ctx.revs = revs; - ctx.show_commit = show_commit; - ctx.show_object = show_object; - ctx.show_data = show_data; - ctx.filter = NULL; - do_traverse(&ctx); -} - void traverse_commit_list_filtered( - struct list_objects_filter_options *filter_options, struct rev_info *revs, show_commit_fn show_commit, show_object_fn show_object, void *show_data, struct oidset *omitted) { - struct traversal_context ctx; + struct traversal_context ctx = { + .revs = revs, + .show_object = show_object, + .show_commit = show_commit, + .show_data = show_data, + }; + + if (revs->filter.choice) + ctx.filter = list_objects_filter__init(omitted, &revs->filter); - ctx.revs = revs; - ctx.show_object = show_object; - ctx.show_commit = show_commit; - ctx.show_data = show_data; - ctx.filter = list_objects_filter__init(omitted, filter_options); do_traverse(&ctx); - list_objects_filter__free(ctx.filter); + + if (ctx.filter) + list_objects_filter__free(ctx.filter); } diff --git a/list-objects.h b/list-objects.h index a952680e466..9eaf4de8449 100644 --- a/list-objects.h +++ b/list-objects.h @@ -7,7 +7,6 @@ struct rev_info; typedef void (*show_commit_fn)(struct commit *, void *); typedef void (*show_object_fn)(struct object *, const char *, void *); -void traverse_commit_list(struct rev_info *, show_commit_fn, show_object_fn, void *); typedef void (*show_edge_fn)(struct commit *); void mark_edges_uninteresting(struct rev_info *revs, @@ -18,11 +17,20 @@ struct oidset; struct list_objects_filter_options; void traverse_commit_list_filtered( - struct list_objects_filter_options *filter_options, struct rev_info *revs, show_commit_fn show_commit, show_object_fn show_object, void *show_data, struct oidset *omitted); +static inline void traverse_commit_list( + struct rev_info *revs, + show_commit_fn show_commit, + show_object_fn show_object, + void *show_data) +{ + traverse_commit_list_filtered(revs, show_commit, + show_object, show_data, NULL); +} + #endif /* LIST_OBJECTS_H */ diff --git a/pack-bitmap.c b/pack-bitmap.c index 37fa4905796..97909d48da3 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -822,9 +822,9 @@ static struct bitmap *find_objects(struct bitmap_index *bitmap_git, show_data.bitmap_git = bitmap_git; show_data.base = base; - traverse_commit_list_filtered(&revs->filter, revs, - show_commit, show_object, - &show_data, NULL); + traverse_commit_list(revs, + show_commit, show_object, + &show_data); revs->include_check = NULL; revs->include_check_obj = NULL; From patchwork Wed Mar 9 16:01:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12775220 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04F3BC433F5 for ; Wed, 9 Mar 2022 16:02:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234274AbiCIQDc (ORCPT ); Wed, 9 Mar 2022 11:03:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40694 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234277AbiCIQC5 (ORCPT ); Wed, 9 Mar 2022 11:02:57 -0500 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 563B717AEF0 for ; Wed, 9 Mar 2022 08:01:55 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id r6so3345596wrr.2 for ; Wed, 09 Mar 2022 08:01:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=RxrhDEZAKd1TxOQXR/q4fcpzONcBLxkiP9hpg49quRw=; b=NUrmalYNvUU3TrT68QNW/28Fq53KeyPduwk2IFQNo0zyYyAgfS/duaNwNebIXIGFwd PLwl2xSDn8X72eOlf48RFuWz76tIDc0Uh21kAXHcM72R8hVb2m83rbeLjR/+vQmZOgxr A1pbwDE5dObx6ZAbzp68g8M9CZF2RvHFv+3L9474HrVO7iZiJ00mlAZZvz3BN1RRt/6T SHN9s9oiIfRqAl4afaDha979TQSjOGhDDOAPClWAvIy1Emcj21eAtqhXMzqvQoVrhMY4 E2SN1lTX8VmdHiVbZuwacDnNEK8/vyEFk3migbaZNwzpKMxbAlonBmnKPMqduOrkxdby LZvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=RxrhDEZAKd1TxOQXR/q4fcpzONcBLxkiP9hpg49quRw=; b=KdoJROyUTRSBn1mYppzZHAB99dK473MxJuv854BMHrlX58dy27/c85Qw3i9o4JY7Am eNgOXA4Vav2mecVQ3l0ZGMFQ7QAP5lsDtYiRhVuPb85eO4DbsjzVIaeozDPBdt1tD+AC 1YXX4PQzscHQxcSLymewADgYtty6f9DHKJNig+XgYdS56hjDdRfJn3ilW0iQtoYR8Cnk CKcDodiUc60/gW4OrjFz0lTmTUnRymGEOlxu12XL2CCgki0dLheyEQeYm0TLdERjL3F1 cWuhPlEyHf1kKGPOrEZGUGgPEnwcollhzR4xt3Nrw+fBXHTlXHkX4Ulssn7O5Y8Nro3U 52Rw== X-Gm-Message-State: AOAM530vNIYKGtuHtVtsR0S8z+DdVembiSGi08tW+ZPH7VGzG3WBDeGV +nBFM7Pmh4pKwFo8oCRIzeNZsKQf/G4= X-Google-Smtp-Source: ABdhPJy/mSEQw/gxK3z/Ru0HoSIabRrEa6FjW6Z6+qOhtw0eFLKKWFjfjcrC5pgTYxmSj2TlYfQW2Q== X-Received: by 2002:a5d:5850:0:b0:203:76f7:5421 with SMTP id i16-20020a5d5850000000b0020376f75421mr254609wrf.114.1646841713629; Wed, 09 Mar 2022 08:01:53 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r17-20020a05600c35d100b00389d75e2a61sm152087wmq.28.2022.03.09.08.01.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Mar 2022 08:01:53 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 09 Mar 2022 16:01:37 +0000 Subject: [PATCH v4 07/13] MyFirstObjectWalk: update recommended usage Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Jeff Hostetler , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The previous change consolidated traverse_commit_list() and traverse_commit_list_filtered(). This allows us to simplify the recommended usage in MyFirstObjectWalk.txt to use this new set of values. While here, add some clarification on the difference between the two methods. Signed-off-by: Derrick Stolee --- Documentation/MyFirstObjectWalk.txt | 44 +++++++++++------------------ 1 file changed, 16 insertions(+), 28 deletions(-) diff --git a/Documentation/MyFirstObjectWalk.txt b/Documentation/MyFirstObjectWalk.txt index ca267941f3e..8d9e85566e6 100644 --- a/Documentation/MyFirstObjectWalk.txt +++ b/Documentation/MyFirstObjectWalk.txt @@ -522,24 +522,25 @@ function shows that the all-object walk is being performed by `traverse_commit_list()` or `traverse_commit_list_filtered()`. Those two functions reside in `list-objects.c`; examining the source shows that, despite the name, these functions traverse all kinds of objects. Let's have a look at -the arguments to `traverse_commit_list_filtered()`, which are a superset of the -arguments to the unfiltered version. +the arguments to `traverse_commit_list()`. -- `struct list_objects_filter_options *filter_options`: This is a struct which - stores a filter-spec as outlined in `Documentation/rev-list-options.txt`. -- `struct rev_info *revs`: This is the `rev_info` used for the walk. +- `struct rev_info *revs`: This is the `rev_info` used for the walk. If + its `filter` member is not `NULL`, then `filter` contains information for + how to filter the object list. - `show_commit_fn show_commit`: A callback which will be used to handle each individual commit object. - `show_object_fn show_object`: A callback which will be used to handle each non-commit object (so each blob, tree, or tag). - `void *show_data`: A context buffer which is passed in turn to `show_commit` and `show_object`. + +In addition, `traverse_commit_list_filtered()` has an additional paramter: + - `struct oidset *omitted`: A linked-list of object IDs which the provided filter caused to be omitted. -It looks like this `traverse_commit_list_filtered()` uses callbacks we provide -instead of needing us to call it repeatedly ourselves. Cool! Let's add the -callbacks first. +It looks like these methods use callbacks we provide instead of needing us +to call it repeatedly ourselves. Cool! Let's add the callbacks first. For the sake of this tutorial, we'll simply keep track of how many of each kind of object we find. At file scope in `builtin/walken.c` add the following @@ -712,20 +713,9 @@ help understand. In our case, that means we omit trees and blobs not directly referenced by `HEAD` or `HEAD`'s history, because we begin the walk with only `HEAD` in the `pending` list.) -First, we'll need to `#include "list-objects-filter-options.h"` and set up the -`struct list_objects_filter_options` at the top of the function. - ----- -static void walken_object_walk(struct rev_info *rev) -{ - struct list_objects_filter_options filter_options = { 0 }; - - ... ----- - For now, we are not going to track the omitted objects, so we'll replace those parameters with `NULL`. For the sake of simplicity, we'll add a simple -build-time branch to use our filter or not. Replace the line calling +build-time branch to use our filter or not. Preface the line calling `traverse_commit_list()` with the following, which will remind us which kind of walk we've just performed: @@ -733,19 +723,17 @@ walk we've just performed: if (0) { /* Unfiltered: */ trace_printf(_("Unfiltered object walk.\n")); - traverse_commit_list(rev, walken_show_commit, - walken_show_object, NULL); } else { trace_printf( _("Filtered object walk with filterspec 'tree:1'.\n")); - parse_list_objects_filter(&filter_options, "tree:1"); - - traverse_commit_list_filtered(&filter_options, rev, - walken_show_commit, walken_show_object, NULL, NULL); + CALLOC_ARRAY(rev->filter, 1); + parse_list_objects_filter(rev->filter, "tree:1"); } + traverse_commit_list(rev, walken_show_commit, + walken_show_object, NULL); ---- -`struct list_objects_filter_options` is usually built directly from a command +The `rev->filter` member is usually built directly from a command line argument, so the module provides an easy way to build one from a string. Even though we aren't taking user input right now, we can still build one with a hardcoded string using `parse_list_objects_filter()`. @@ -784,7 +772,7 @@ object: ---- ... - traverse_commit_list_filtered(&filter_options, rev, + traverse_commit_list_filtered(rev, walken_show_commit, walken_show_object, NULL, &omitted); ... From patchwork Wed Mar 9 16:01:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12775223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9E29C433EF for ; Wed, 9 Mar 2022 16:02:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234252AbiCIQDi (ORCPT ); Wed, 9 Mar 2022 11:03:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234309AbiCIQDA (ORCPT ); Wed, 9 Mar 2022 11:03:00 -0500 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DAB0A179A3E for ; Wed, 9 Mar 2022 08:02:00 -0800 (PST) Received: by mail-wm1-x329.google.com with SMTP id h127-20020a1c2185000000b00389d0a5c511so318508wmh.5 for ; Wed, 09 Mar 2022 08:02:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=3AlRxdtCHh8EZhdx7bYcK7Q5bS3qwIH9q69ubfXY1Gg=; b=EBoAjyh9/aqdKjFqKc1mVxW/l+IdloS+E82SeZZhDvzH7b3sj2BMlUiv+IhhDS6ses xMrfP4bv4YpCHa47oPzJQmvK9tWdi+nZJNxEc8TiPmkpisvZ3sdkT4MRxBwt3iz6mVkm LlT3IaiKeXZGducOumbHRVOGuD22moo3kRb3NwGx8Q+ZwUcRFvAqT/vqrAHFeBdS1rnY vwFPjgB1GVzfAaaE89twKw5+SeR8pSUV0l4umKXijq7l/kII9ueLvxEojIC7lFbfjzgH wg6myM/0HuG3w39iY/uldx5LzmHkWbBgaILPiW8N3vSkD8N6QDM1O0ihGO8YvhAov0xV 0lJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=3AlRxdtCHh8EZhdx7bYcK7Q5bS3qwIH9q69ubfXY1Gg=; b=FZlEnXxb0LRygt9i6lCFIBjFhXhV0tU9/wLXCF2ZQXyl/mPEdpxbNFw/xKAlRzfBDN B5Ox8nLQl4rbHFGigwkV4O32v2vGT8cLEwISD1o1VKCkRTlZWY19SW7Ma2d+iGosdvN7 F5O4DllNTIwDDrfKIvjvlK55XUWwMnv9qdXeDREYr1FQy1Xp1R8wizxglglMTHn7iXIJ BwxoCAosLncn35x7aH6VVOWSNEaae1i5YcgF3NbfIAqjFyijb70hBy8YinOLca7PSUJ8 4fQdvtupKfTdxOkuyefsTJOThCiDBwBqZjARRe3d3HVsbkxuB/m62IHkNKrpEP9T9cmm ozvg== X-Gm-Message-State: AOAM532M3ph0fBCe4du1ClKMc2sMkA4uc1c3VoNAPT6oJ7z/QBNQaXvv vz7eCnrnkLr9UNduKqFtinpo8T/0D6s= X-Google-Smtp-Source: ABdhPJxhVbdcFe6WSTnNJcaQBQtS6KoL+VWCubH4ivGa6ebDK5w9c9PqwvngUa2RHFdRlb8AkxrTjQ== X-Received: by 2002:a1c:7919:0:b0:389:79c5:fe83 with SMTP id l25-20020a1c7919000000b0038979c5fe83mr7986051wme.173.1646841714831; Wed, 09 Mar 2022 08:01:54 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u23-20020a7bcb17000000b0037bdfa1665asm7139284wmj.18.2022.03.09.08.01.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Mar 2022 08:01:54 -0800 (PST) Message-Id: <7287aaec598149a05fd459bc250a8b93c1f119fe.1646841704.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 09 Mar 2022 16:01:38 +0000 Subject: [PATCH v4 08/13] list-objects: handle NULL function pointers MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Jeff Hostetler , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?= If a caller to traverse_commit_list() specifies the options for the --objects flag but does not specify a show_object function pointer, the result is a segfault. This is currently visible by running 'git bundle create --objects HEAD'. We could fix this problem by supplying a no-op callback in builtin/bundle.c, but that only solves the problem for one builtin, leaving this segfault open for other callers. Replace all callers of the show_commit and show_object function pointers in list-objects.c to call helper functions show_commit() and show_object() which check that the given context has non-NULL functions before passing the necessary data. One extra benefit is that it reduces duplication due to passing ctx->show_data to every caller. Test that this segfault no longer occurs for 'git bundle'. Co-authored-by: Derrick Stolee Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Derrick Stolee --- bundle.c | 2 ++ list-objects.c | 27 ++++++++++++++++++++++----- t/t6020-bundle-misc.sh | 12 ++++++++++++ 3 files changed, 36 insertions(+), 5 deletions(-) diff --git a/bundle.c b/bundle.c index a0bb687b0f4..7ba60a573d7 100644 --- a/bundle.c +++ b/bundle.c @@ -544,6 +544,8 @@ int create_bundle(struct repository *r, const char *path, die("revision walk setup failed"); bpi.fd = bundle_fd; bpi.pending = &revs_copy.pending; + + revs.blob_objects = revs.tree_objects = 0; traverse_commit_list(&revs, write_bundle_prerequisites, NULL, &bpi); object_array_remove_duplicates(&revs_copy.pending); diff --git a/list-objects.c b/list-objects.c index 117f734398c..250d9de41cb 100644 --- a/list-objects.c +++ b/list-objects.c @@ -21,6 +21,23 @@ struct traversal_context { struct filter *filter; }; +static void show_commit(struct traversal_context *ctx, + struct commit *commit) +{ + if (!ctx->show_commit) + return; + ctx->show_commit(commit, ctx->show_data); +} + +static void show_object(struct traversal_context *ctx, + struct object *object, + const char *name) +{ + if (!ctx->show_object) + return; + ctx->show_object(object, name, ctx->show_data); +} + static void process_blob(struct traversal_context *ctx, struct blob *blob, struct strbuf *path, @@ -60,7 +77,7 @@ static void process_blob(struct traversal_context *ctx, if (r & LOFR_MARK_SEEN) obj->flags |= SEEN; if (r & LOFR_DO_SHOW) - ctx->show_object(obj, path->buf, ctx->show_data); + show_object(ctx, obj, path->buf); strbuf_setlen(path, pathlen); } @@ -194,7 +211,7 @@ static void process_tree(struct traversal_context *ctx, if (r & LOFR_MARK_SEEN) obj->flags |= SEEN; if (r & LOFR_DO_SHOW) - ctx->show_object(obj, base->buf, ctx->show_data); + show_object(ctx, obj, base->buf); if (base->len) strbuf_addch(base, '/'); @@ -210,7 +227,7 @@ static void process_tree(struct traversal_context *ctx, if (r & LOFR_MARK_SEEN) obj->flags |= SEEN; if (r & LOFR_DO_SHOW) - ctx->show_object(obj, base->buf, ctx->show_data); + show_object(ctx, obj, base->buf); strbuf_setlen(base, baselen); free_tree_buffer(tree); @@ -228,7 +245,7 @@ static void process_tag(struct traversal_context *ctx, if (r & LOFR_MARK_SEEN) tag->object.flags |= SEEN; if (r & LOFR_DO_SHOW) - ctx->show_object(&tag->object, name, ctx->show_data); + show_object(ctx, &tag->object, name); } static void mark_edge_parents_uninteresting(struct commit *commit, @@ -402,7 +419,7 @@ static void do_traverse(struct traversal_context *ctx) if (r & LOFR_MARK_SEEN) commit->object.flags |= SEEN; if (r & LOFR_DO_SHOW) - ctx->show_commit(commit, ctx->show_data); + show_commit(ctx, commit); if (ctx->revs->tree_blobs_in_commit_order) /* diff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh index b13e8a52a93..df5ff561fa5 100755 --- a/t/t6020-bundle-misc.sh +++ b/t/t6020-bundle-misc.sh @@ -475,4 +475,16 @@ test_expect_success 'clone from bundle' ' test_cmp expect actual ' +test_expect_success 'unfiltered bundle with --objects' ' + git bundle create all-objects.bdl \ + --all --objects && + git bundle create all.bdl \ + --all && + + # Compare the headers of these files. + sed -n -e "/^$/q" -e "p" all.bdl >expect && + sed -n -e "/^$/q" -e "p" all-objects.bdl >actual && + test_cmp expect actual +' + test_done From patchwork Wed Mar 9 16:01:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12775216 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE0EFC433F5 for ; Wed, 9 Mar 2022 16:02:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234154AbiCIQDS (ORCPT ); Wed, 9 Mar 2022 11:03:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40804 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234288AbiCIQC6 (ORCPT ); Wed, 9 Mar 2022 11:02:58 -0500 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79D02172E77 for ; Wed, 9 Mar 2022 08:01:57 -0800 (PST) Received: by mail-wr1-x42c.google.com with SMTP id p9so3741375wra.12 for ; Wed, 09 Mar 2022 08:01:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=x21zZXWoDS7htTo9KeOXoy/R1/9D5U1HzpQi0NNkuy0=; b=Mpzt3WoJWZ56waP/u8sy+QST7ZkSzMldE2tcwG6nGtdVdTHRVKh41vPsE1v3TzfzMm 6Gmj5395HNKAtqB3TWuutd2VepBrh9Rs+3hpNLKMpHqkqsz2kCZBIYjADkCh881DY6T3 N21iGY97XtiCeiMAUaT2zA1Q1j77y+EiuhiQ633MrmxFcdot/PkPGlZAQ1E85to/V2Vq 9pULxi4zpY0LVWjVdpV1c/Yb9ZxbA13hfgbU1yRCJjulyauUqYt0sPgoiBXtIN0g7pRT Y6/YxlmKU6hilx0t7MFa67eyAdBxYWIcPYmBT3KEgh7oqmGpHkmKUUuqPLPWERQlrmy2 iiNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=x21zZXWoDS7htTo9KeOXoy/R1/9D5U1HzpQi0NNkuy0=; b=ZtL4dizN3IVqZEkcs7F9Mdv7i6WP8xuujX9e7IcXznSc4uhh+mSsVauFZ+8nqbDWus /IbIhcsBbxNRQ9L7/qDg1chPuquQfQTZ3wq5KoSfW7NpvtzN50EFuIXUEo+ATRw5B1Z1 GKQMMBzQmDS1/aK51kWlBp+QOR9obZNt96x3SetvrQDTjnpq7z1bv1GaT/ilEUtFJfeq O8kuBvfmOZfhDzsfNRJsrODpNSapDOwdkO9u9eATUj94QVmF3IGuUkoNCqP/mFOAPHDQ goIPkte5iMM/C1YmAnEvlVlnY1JGTFqobQtS9+qGlTkGUXXDJnzsP4LGE5rjsckHOYKQ SbUQ== X-Gm-Message-State: AOAM532dc3dwZ+59dGmqCFL5FQOwVrwnhNEqNu8tRciYvi7TC2k5a93w EMV6j8QIceFRr3tjNMAWxL6Rev6O1P4= X-Google-Smtp-Source: ABdhPJx3Oh/UU4YbXCUX8cqLblKRHY/dWZKczwylbh4hR+2xJtmkoX3UNiIFaQsTP+m794zP7vbdLw== X-Received: by 2002:adf:f947:0:b0:1ef:9b55:7cba with SMTP id q7-20020adff947000000b001ef9b557cbamr261711wrr.142.1646841715732; Wed, 09 Mar 2022 08:01:55 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i8-20020a7bc948000000b003898dfd7990sm2369863wml.29.2022.03.09.08.01.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Mar 2022 08:01:55 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 09 Mar 2022 16:01:39 +0000 Subject: [PATCH v4 09/13] bundle: parse filter capability Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Jeff Hostetler , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The v3 bundle format has capabilities, allowing newer versions of Git to create bundles with newer features. Older versions that do not understand these new capabilities will fail with a helpful warning. Create a new capability allowing Git to understand that the contained pack-file is filtered according to some object filter. Typically, this filter will be "blob:none" for a blobless partial clone. This change teaches Git to parse this capability, place its value in the bundle header, and demonstrate this understanding by adding a message to 'git bundle verify'. Since we will use gently_parse_list_objects_filter() outside of list-objects-filter-options.c, make it an external method and move its API documentation to before its declaration. Signed-off-by: Derrick Stolee --- Documentation/git-bundle.txt | 7 +++++-- Documentation/technical/bundle-format.txt | 11 ++++++++--- bundle.c | 15 ++++++++++++++- bundle.h | 2 ++ list-objects-filter-options.c | 17 +---------------- list-objects-filter-options.h | 20 ++++++++++++++++++++ 6 files changed, 50 insertions(+), 22 deletions(-) diff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt index 72ab8139052..ac4c4352aae 100644 --- a/Documentation/git-bundle.txt +++ b/Documentation/git-bundle.txt @@ -75,8 +75,11 @@ verify :: cleanly to the current repository. This includes checks on the bundle format itself as well as checking that the prerequisite commits exist and are fully linked in the current repository. - 'git bundle' prints a list of missing commits, if any, and exits - with a non-zero status. + Information about additional capabilities, such as "object filter", + is printed. See "Capabilities" in link:technical/bundle-format.html + for more information. Finally, 'git bundle' prints a list of + missing commits, if any. The exit code is zero for success, but + will be nonzero if the bundle file is invalid. list-heads :: Lists the references defined in the bundle. If followed by a diff --git a/Documentation/technical/bundle-format.txt b/Documentation/technical/bundle-format.txt index bac558d049a..b9be8644cf5 100644 --- a/Documentation/technical/bundle-format.txt +++ b/Documentation/technical/bundle-format.txt @@ -71,6 +71,11 @@ and the Git bundle v2 format cannot represent a shallow clone repository. == Capabilities Because there is no opportunity for negotiation, unknown capabilities cause 'git -bundle' to abort. The only known capability is `object-format`, which specifies -the hash algorithm in use, and can take the same values as the -`extensions.objectFormat` configuration value. +bundle' to abort. + +* `object-format` specifies the hash algorithm in use, and can take the same + values as the `extensions.objectFormat` configuration value. + +* `filter` specifies an object filter as in the `--filter` option in + linkgit:git-rev-list[1]. The resulting pack-file must be marked as a + `.promisor` pack-file after it is unbundled. diff --git a/bundle.c b/bundle.c index 7ba60a573d7..41e75efab9a 100644 --- a/bundle.c +++ b/bundle.c @@ -11,7 +11,7 @@ #include "run-command.h" #include "refs.h" #include "strvec.h" - +#include "list-objects-filter-options.h" static const char v2_bundle_signature[] = "# v2 git bundle\n"; static const char v3_bundle_signature[] = "# v3 git bundle\n"; @@ -33,6 +33,7 @@ void bundle_header_release(struct bundle_header *header) { string_list_clear(&header->prerequisites, 1); string_list_clear(&header->references, 1); + list_objects_filter_release(&header->filter); } static int parse_capability(struct bundle_header *header, const char *capability) @@ -45,6 +46,10 @@ static int parse_capability(struct bundle_header *header, const char *capability header->hash_algo = &hash_algos[algo]; return 0; } + if (skip_prefix(capability, "filter=", &arg)) { + parse_list_objects_filter(&header->filter, arg); + return 0; + } return error(_("unknown capability '%s'"), capability); } @@ -220,6 +225,8 @@ int verify_bundle(struct repository *r, req_nr = revs.pending.nr; setup_revisions(2, argv, &revs, NULL); + list_objects_filter_copy(&revs.filter, &header->filter); + if (prepare_revision_walk(&revs)) die(_("revision walk setup failed")); @@ -259,6 +266,12 @@ int verify_bundle(struct repository *r, r->nr), r->nr); list_refs(r, 0, NULL); + + if (header->filter.choice) { + printf_ln("The bundle uses this filter: %s", + list_objects_filter_spec(&header->filter)); + } + r = &header->prerequisites; if (!r->nr) { printf_ln(_("The bundle records a complete history.")); diff --git a/bundle.h b/bundle.h index 06009fe6b1f..7fef2108f43 100644 --- a/bundle.h +++ b/bundle.h @@ -4,12 +4,14 @@ #include "strvec.h" #include "cache.h" #include "string-list.h" +#include "list-objects-filter-options.h" struct bundle_header { unsigned version; struct string_list prerequisites; struct string_list references; const struct git_hash_algo *hash_algo; + struct list_objects_filter_options filter; }; #define BUNDLE_HEADER_INIT \ diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index 449d53af69f..f02d8df1422 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -40,22 +40,7 @@ const char *list_object_filter_config_name(enum list_objects_filter_choice c) BUG("list_object_filter_config_name: invalid argument '%d'", c); } -/* - * Parse value of the argument to the "filter" keyword. - * On the command line this looks like: - * --filter= - * and in the pack protocol as: - * "filter" SP - * - * The filter keyword will be used by many commands. - * See Documentation/rev-list-options.txt for allowed values for . - * - * Capture the given arg as the "filter_spec". This can be forwarded to - * subordinate commands when necessary (although it's better to pass it through - * expand_list_objects_filter_spec() first). We also "intern" the arg for the - * convenience of the current command. - */ -static int gently_parse_list_objects_filter( +int gently_parse_list_objects_filter( struct list_objects_filter_options *filter_options, const char *arg, struct strbuf *errbuf) diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h index 425c38cae9d..2eb6c983949 100644 --- a/list-objects-filter-options.h +++ b/list-objects-filter-options.h @@ -72,6 +72,26 @@ struct list_objects_filter_options { /* Normalized command line arguments */ #define CL_ARG__FILTER "filter" +/* + * Parse value of the argument to the "filter" keyword. + * On the command line this looks like: + * --filter= + * and in the pack protocol as: + * "filter" SP + * + * The filter keyword will be used by many commands. + * See Documentation/rev-list-options.txt for allowed values for . + * + * Capture the given arg as the "filter_spec". This can be forwarded to + * subordinate commands when necessary (although it's better to pass it through + * expand_list_objects_filter_spec() first). We also "intern" the arg for the + * convenience of the current command. + */ +int gently_parse_list_objects_filter( + struct list_objects_filter_options *filter_options, + const char *arg, + struct strbuf *errbuf); + void list_objects_filter_die_if_populated( struct list_objects_filter_options *filter_options); From patchwork Wed Mar 9 16:01:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12775226 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDEBEC433EF for ; Wed, 9 Mar 2022 16:02:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234230AbiCIQDq (ORCPT ); Wed, 9 Mar 2022 11:03:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234290AbiCIQC6 (ORCPT ); Wed, 9 Mar 2022 11:02:58 -0500 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6CC7217585A for ; Wed, 9 Mar 2022 08:01:58 -0800 (PST) Received: by mail-wr1-x430.google.com with SMTP id p9so3741510wra.12 for ; Wed, 09 Mar 2022 08:01:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=11Uo0gVQ9fME59Fqhjf9u776SRTbt2UmEbraPBNP+XE=; b=oJZITbxbSASCqjoeRKPqk3LkDGf/nJeM+v9fKogJr6ee2NROZzp+M5QhM5r4lcU0B0 z9xguPAMRm1xZhVde4LliQ+LOypFWNdjOBU9uiRAPaWnNctPb4lCTBKlaQG/nRSwTpGd dt84WZloFolLSZsGceFHil3KZpGszCPgOokJ5zGt+dPgARh2kpiOez6EHr4WgsRBIsvc 3/Y/JL2gG+R/VJYFV+WxDGOUVYXnzR9ilH00+eEBNfFg2PLsUppFDIReOn6xVCJe8EoU KvMsjPThwy4wQ5fV66s/zksyWLs8EikyiBY1DkK4jr4eL0aXajhjuwrva2Qk14nJqoef auVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=11Uo0gVQ9fME59Fqhjf9u776SRTbt2UmEbraPBNP+XE=; b=6MbdBlI9olJ+L0cqclHXCgPrLY8pmY6UAWvZY/D+sbMh+QXinkmam5p3WSqEbdpWjq Qb7P5CIp3dIxxG5L7y2sCkqpzTku+GsNw6c0eaoldYd/gMz/cIc5MS5DZeJEsBfUCiMd 0x5NDEjJqU0iYX06LGmXCautF6JgitTSZW7+DzccmxGiKmR/bwdeD/DE1MdxNB5zfh8w ecxfpo1S2xUPtxsEYqTm8XuU3iyiiw7YCnC0dHlII56fqJZjOc21x5nUUg4OrEGP/7XG zGHT4n+OGXcoby7ejOxNnKz3jdzTNxReUvCh19c15JIznh5MPYuo1pD/H4Ktlv9XOSGj 3jYQ== X-Gm-Message-State: AOAM531BMaC4s6dlS9Hf7A1olcrunBAq04gI9iSKQb5Jy1pImV/Vi/WQ olHbP+kD6qoExPMJW+X81QwVVj62k2g= X-Google-Smtp-Source: ABdhPJzrGqJA8c3QXE9r0ib+UhvM0OyJNSr/FzMuSMIyAbTkhUfJnl2YDvmzGdarSgnrcsWPsm1bjw== X-Received: by 2002:adf:9d93:0:b0:1f0:639f:3bc2 with SMTP id p19-20020adf9d93000000b001f0639f3bc2mr218897wre.203.1646841716820; Wed, 09 Mar 2022 08:01:56 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id b3-20020a5d4d83000000b001f1d72a6f97sm1943993wru.50.2022.03.09.08.01.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Mar 2022 08:01:56 -0800 (PST) Message-Id: <05d7322fdfcd6abb1f3a160405121e4c7974ebc4.1646841704.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 09 Mar 2022 16:01:40 +0000 Subject: [PATCH v4 10/13] rev-list: move --filter parsing into revision.c Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Jeff Hostetler , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Now that 'struct rev_info' has a 'filter' member and most consumers of object filtering are using that member instead of an external struct, move the parsing of the '--filter' option out of builtin/rev-list.c and into revision.c. This use within handle_revision_pseudo_opt() allows us to find the option within setup_revisions() if the arguments are passed directly. In the case of a command such as 'git blame', the arguments are first scanned and checked with parse_revision_opt(), which complains about the option, so 'git blame --filter=blob:none ' does not become valid with this change. Some commands, such as 'git diff' gain this option without having it make an effect. And 'git diff --objects' was already possible, but does not actually make sense in that builtin. The key addition that is coming is 'git bundle create --filter=' so we can create bundles containing promisor packs. More work is required to make them fully functional, but that will follow. Signed-off-by: Derrick Stolee --- builtin/rev-list.c | 11 ----------- revision.c | 7 +++++++ 2 files changed, 7 insertions(+), 11 deletions(-) diff --git a/builtin/rev-list.c b/builtin/rev-list.c index ec433cb6d37..640828149c5 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -591,17 +591,6 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) show_progress = arg; continue; } - - if (skip_prefix(arg, ("--" CL_ARG__FILTER "="), &arg)) { - parse_list_objects_filter(&revs.filter, arg); - if (revs.filter.choice && !revs.blob_objects) - die(_("object filtering requires --objects")); - continue; - } - if (!strcmp(arg, ("--no-" CL_ARG__FILTER))) { - list_objects_filter_set_no_filter(&revs.filter); - continue; - } if (!strcmp(arg, "--filter-provided-objects")) { filter_provided_objects = 1; continue; diff --git a/revision.c b/revision.c index ad4286fbdde..15efe23c40f 100644 --- a/revision.c +++ b/revision.c @@ -32,6 +32,7 @@ #include "utf8.h" #include "bloom.h" #include "json-writer.h" +#include "list-objects-filter-options.h" volatile show_early_output_fn_t show_early_output; @@ -2669,6 +2670,10 @@ static int handle_revision_pseudo_opt(struct rev_info *revs, revs->no_walk = 0; } else if (!strcmp(arg, "--single-worktree")) { revs->single_worktree = 1; + } else if (skip_prefix(arg, ("--" CL_ARG__FILTER "="), &arg)) { + parse_list_objects_filter(&revs->filter, arg); + } else if (!strcmp(arg, ("--no-" CL_ARG__FILTER))) { + list_objects_filter_set_no_filter(&revs->filter); } else { return 0; } @@ -2872,6 +2877,8 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s die("cannot combine --walk-reflogs with history-limiting options"); if (revs->rewrite_parents && revs->children.name) die(_("options '%s' and '%s' cannot be used together"), "--parents", "--children"); + if (revs->filter.choice && !revs->blob_objects) + die(_("object filtering requires --objects")); /* * Limitations on the graph functionality From patchwork Wed Mar 9 16:01:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12775221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D7A2C4332F for ; Wed, 9 Mar 2022 16:02:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234237AbiCIQDe (ORCPT ); Wed, 9 Mar 2022 11:03:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234302AbiCIQC7 (ORCPT ); Wed, 9 Mar 2022 11:02:59 -0500 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7020F1795DE for ; Wed, 9 Mar 2022 08:01:59 -0800 (PST) Received: by mail-wm1-x32b.google.com with SMTP id v2-20020a7bcb42000000b0037b9d960079so3735950wmj.0 for ; Wed, 09 Mar 2022 08:01:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=wtq4sy4sjg9ijxSnEbwHlFT5qYc6N4W4kaL9UO5zixo=; b=LVthuGICN2ngnO5HXMvgLcIEg80Q9I6Wh4d/V8vaHXtQMehEWNG+8O43Zj2SiSo/8B dlhiaxVWdkUsQ3rlQcxLqAsiegw2iTDgxYHYniWBXj4cTMCTc4JTY95z6kjpX5OCtNZn DugRN7FS9IpnLNrLQ1olr34GEShIoMVM7/j6Edu0gevArx6vc1I7WJH2T7TbudN4/v5J toARHKRzqTOHnq1arcRSB+Irt6vs9gEv15/PU/JW/vDzJMCUtvNQYdLslNfhiLyolclU vspElfV0WnCqklD+H5JBY1NCtutH7OKRNCkGkT5xdmbMU9mtF4wprvnTZQQ2z6S4O5GA imvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=wtq4sy4sjg9ijxSnEbwHlFT5qYc6N4W4kaL9UO5zixo=; b=hl9LxC26NiNz3LCHlp+K9pYOzWmeyfTQZZSoljs+O79af3G9k438Y5XFB4OvOTCfID xNWk24lRErb4xKwHzdvjjACdIeCR08eTq37WiNtOsb2RUdEPYImkZR/hBPBP/haaWYki qPyRft1ZpEUtuTgRXxvhQpqasD/REKaWfHDsL0o2kqPTFSwOYThkMRIiPEcmOt4raJMC cbM+CPtehLuFroZTBygSmBgM+umqz53NVP0Ys+j27tNNS5s2mVJ+hDFU8CrXo7rn+0H5 IhzExSqlIq4La+tH00ucB0wlda1RGtMphyNR0TQMOvxXDKFGcPSrQ13DFLmgR47XvWxc kj0Q== X-Gm-Message-State: AOAM530yFF5S5DWQIz0p2Hk3zuBOcJFUxG5to8/C2rnrBEo5IPvrBWeH nm9Kbtirkgr45ogHhLJ6fGLflFP7dTg= X-Google-Smtp-Source: ABdhPJyoP65g/4WzdgsXDeKZvnDzJq+kiKMmcglDfNedzn5P6PlDu5Ye5yZgswUoslRY8X37CL31ww== X-Received: by 2002:a05:600c:3b13:b0:389:cf43:eaf5 with SMTP id m19-20020a05600c3b1300b00389cf43eaf5mr41139wms.198.1646841717792; Wed, 09 Mar 2022 08:01:57 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g6-20020a05600c4ec600b0037bf934bca3sm6366611wmq.17.2022.03.09.08.01.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Mar 2022 08:01:57 -0800 (PST) Message-Id: <7435095bbc997ee8646bdfd804b1e820ffa80100.1646841704.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 09 Mar 2022 16:01:41 +0000 Subject: [PATCH v4 11/13] bundle: create filtered bundles Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Jeff Hostetler , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee A previous change allowed Git to parse bundles with the 'filter' capability. Now, teach Git to create bundles with this option. Some rearranging of code is required to get the option parsing in the correct spot. There are now two reasons why we might need capabilities (a new hash algorithm or an object filter) so that is pulled out into a place where we can check both at the same time. The --filter option is parsed as part of setup_revisions(), but it expected the --objects flag, too. That flag is somewhat implied by 'git bundle' because it creates a pack-file walking objects, but there is also a walk that walks the revision range expecting only commits. Make this parsing work by setting 'revs.tree_objects' and 'revs.blob_objects' before the call to setup_revisions(). Signed-off-by: Derrick Stolee --- bundle.c | 53 +++++++++++++++++++++++++++++++++--------- t/t6020-bundle-misc.sh | 48 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 90 insertions(+), 11 deletions(-) diff --git a/bundle.c b/bundle.c index 41e75efab9a..9370a6e307c 100644 --- a/bundle.c +++ b/bundle.c @@ -332,6 +332,9 @@ static int write_pack_data(int bundle_fd, struct rev_info *revs, struct strvec * "--stdout", "--thin", "--delta-base-offset", NULL); strvec_pushv(&pack_objects.args, pack_options->v); + if (revs->filter.choice) + strvec_pushf(&pack_objects.args, "--filter=%s", + list_objects_filter_spec(&revs->filter)); pack_objects.in = -1; pack_objects.out = bundle_fd; pack_objects.git_cmd = 1; @@ -499,10 +502,37 @@ int create_bundle(struct repository *r, const char *path, int bundle_to_stdout; int ref_count = 0; struct rev_info revs, revs_copy; - int min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3; + int min_version = 2; struct bundle_prerequisites_info bpi; int i; + /* init revs to list objects for pack-objects later */ + save_commit_buffer = 0; + repo_init_revisions(r, &revs, NULL); + + /* + * Pre-initialize the '--objects' flag so we can parse a + * --filter option successfully. + */ + revs.tree_objects = revs.blob_objects = 1; + + argc = setup_revisions(argc, argv, &revs, NULL); + + /* + * Reasons to require version 3: + * + * 1. @object-format is required because our hash algorithm is not + * SHA1. + * 2. @filter is required because we parsed an object filter. + */ + if (the_hash_algo != &hash_algos[GIT_HASH_SHA1] || revs.filter.choice) + min_version = 3; + + if (argc > 1) { + error(_("unrecognized argument: %s"), argv[1]); + goto err; + } + bundle_to_stdout = !strcmp(path, "-"); if (bundle_to_stdout) bundle_fd = 1; @@ -525,17 +555,14 @@ int create_bundle(struct repository *r, const char *path, write_or_die(bundle_fd, capability, strlen(capability)); write_or_die(bundle_fd, the_hash_algo->name, strlen(the_hash_algo->name)); write_or_die(bundle_fd, "\n", 1); - } - - /* init revs to list objects for pack-objects later */ - save_commit_buffer = 0; - repo_init_revisions(r, &revs, NULL); - argc = setup_revisions(argc, argv, &revs, NULL); - - if (argc > 1) { - error(_("unrecognized argument: %s"), argv[1]); - goto err; + if (revs.filter.choice) { + const char *value = expand_list_objects_filter_spec(&revs.filter); + capability = "@filter="; + write_or_die(bundle_fd, capability, strlen(capability)); + write_or_die(bundle_fd, value, strlen(value)); + write_or_die(bundle_fd, "\n", 1); + } } /* save revs.pending in revs_copy for later use */ @@ -558,6 +585,10 @@ int create_bundle(struct repository *r, const char *path, bpi.fd = bundle_fd; bpi.pending = &revs_copy.pending; + /* + * Remove any object walking here. We only care about commits and + * tags here. The revs_copy has the right instances of these values. + */ revs.blob_objects = revs.tree_objects = 0; traverse_commit_list(&revs, write_bundle_prerequisites, NULL, &bpi); object_array_remove_duplicates(&revs_copy.pending); diff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh index df5ff561fa5..6e97c044ee7 100755 --- a/t/t6020-bundle-misc.sh +++ b/t/t6020-bundle-misc.sh @@ -487,4 +487,52 @@ test_expect_success 'unfiltered bundle with --objects' ' test_cmp expect actual ' +for filter in "blob:none" "tree:0" "tree:1" "blob:limit=100" +do + test_expect_success "filtered bundle: $filter" ' + test_when_finished rm -rf .git/objects/pack cloned unbundled && + git bundle create partial.bdl \ + --all \ + --filter=$filter && + + git bundle verify partial.bdl >unfiltered && + make_user_friendly_and_stable_output actual && + + cat >expect <<-EOF && + The bundle contains these 10 refs: + refs/heads/main + refs/heads/release + refs/heads/topic/1 + refs/heads/topic/2 + refs/pull/1/head + refs/pull/2/head + refs/tags/v1 + refs/tags/v2 + refs/tags/v3 + HEAD + The bundle uses this filter: $filter + The bundle records a complete history. + EOF + test_cmp expect actual && + + test_config uploadpack.allowfilter 1 && + test_config uploadpack.allowanysha1inwant 1 && + git clone --no-local --filter=$filter --bare "file://$(pwd)" cloned && + + git init unbundled && + git -C unbundled bundle unbundle ../partial.bdl >ref-list.txt && + + # Count the same number of reachable objects. + reflist=$(git for-each-ref --format="%(objectname)") && + git rev-list --objects --filter=$filter --missing=allow-any \ + $reflist >expect && + for repo in cloned unbundled + do + git -C $repo rev-list --objects --missing=allow-any \ + $reflist >actual && + test_cmp expect actual || return 1 + done + ' +done + test_done From patchwork Wed Mar 9 16:01:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12775228 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54BB1C433EF for ; Wed, 9 Mar 2022 16:02:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234219AbiCIQDy (ORCPT ); Wed, 9 Mar 2022 11:03:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40844 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234301AbiCIQC7 (ORCPT ); Wed, 9 Mar 2022 11:02:59 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8741179A28 for ; Wed, 9 Mar 2022 08:02:00 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id k24so3774549wrd.7 for ; Wed, 09 Mar 2022 08:02:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=guBdmIvrxmYIKLajG644S8L8MvB8iHuPF96zFc6SXQc=; b=UWJ2K17smdcCtEOY+yovbiqCPTYUZl+t+lmGE733VRi3/Np/F/T+vPpFaoCqXLs4DC xoF+bRSuEcm0rhjCEgX3JoqBP6pnlgqNB9geybiYGqYS1YI+78OlwpCWhGJxXq/U40Ga Gi8XZZOnMAZw5sJDLjcApLJiOL3KG3QIPCR/A/yq13NVGbwJaKw7EkiSPIB6OGMRkZEa zxktx91jUJxFTRiyMPkEvQHcGFJDBKfc+tH4UjfzvcR8gN3a0rIBB1XJMoouvme+E+Aj uBrrNGxnfSHCjs/7J5JaRQS9Fxfir7/qKqELJhoSVBoOorGAmJ/+nywliVDp6qsqDhhV R2VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=guBdmIvrxmYIKLajG644S8L8MvB8iHuPF96zFc6SXQc=; b=OsDlTgbH3HJrBmazuWAPDidFXYExM/8nIuesd84zwNZk0rPSHnJ3sipvQkj3vBzlEw NOXB8QqfXt8o88QYTPX8+iXGgtRiU21GDCz5qXfwOW2en8QiLw2t50ozs583XIWWf7bv 6ln6hZjN758vZKd15ZqYvQ9y4AarZa1fgOKd2UN35+LuEPRYHuoOg5NHrXatkawLg8kO 6mx/exPRYACvhJmLWqA/TPshXWATByshCW+M3b7Yxdp8fTT4d5kr0wriAbOulKm0CFkW jvFoKtZp0mDIDIJcjaFX1ptlXiivVdfog/RU0yVBjl5BCRKV/XG22yzijy5fp9S+e/eA ubBA== X-Gm-Message-State: AOAM5333nUJoYYTzfhJz0j7qsFZnGfuPhP1UzmF2MnuXu0jocaZdKu23 ut2sZu2KY0iX847lxBRrU96TXu7VA9c= X-Google-Smtp-Source: ABdhPJyaKzmtLvqQRHcc2yCqXp61+SOh8cux4SlYIia6qkf5JSc/nkW5/JYzPyu7ScO/cKcdt6hW4g== X-Received: by 2002:a5d:64cc:0:b0:203:781b:6a1 with SMTP id f12-20020a5d64cc000000b00203781b06a1mr231309wri.205.1646841718696; Wed, 09 Mar 2022 08:01:58 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e20-20020adfa454000000b001f01a14dce8sm2035697wra.97.2022.03.09.08.01.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Mar 2022 08:01:58 -0800 (PST) Message-Id: <77a62156332160cd3346d604d342cfab5b9001d4.1646841704.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 09 Mar 2022 16:01:42 +0000 Subject: [PATCH v4 12/13] bundle: unbundle promisor packs Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Jeff Hostetler , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee In order to have a valid pack-file after unbundling a bundle that has the 'filter' capability, we need to generate a .promisor file. The bundle does not promise _where_ the objects can be found, but we can expect that these bundles will be unbundled in repositories with appropriate promisor remotes that can find those missing objects. Use the 'git index-pack --promisor=' option to create this .promisor file. Add "from-bundle" as the message to help anyone diagnose issues with these promisor packs. Signed-off-by: Derrick Stolee --- bundle.c | 4 ++++ t/t6020-bundle-misc.sh | 2 ++ 2 files changed, 6 insertions(+) diff --git a/bundle.c b/bundle.c index 9370a6e307c..56681c21131 100644 --- a/bundle.c +++ b/bundle.c @@ -620,6 +620,10 @@ int unbundle(struct repository *r, struct bundle_header *header, struct child_process ip = CHILD_PROCESS_INIT; strvec_pushl(&ip.args, "index-pack", "--fix-thin", "--stdin", NULL); + /* If there is a filter, then we need to create the promisor pack. */ + if (header->filter.choice) + strvec_push(&ip.args, "--promisor=from-bundle"); + if (extra_index_pack_args) { strvec_pushv(&ip.args, extra_index_pack_args->v); strvec_clear(extra_index_pack_args); diff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh index 6e97c044ee7..7c6db670221 100755 --- a/t/t6020-bundle-misc.sh +++ b/t/t6020-bundle-misc.sh @@ -521,6 +521,8 @@ do git init unbundled && git -C unbundled bundle unbundle ../partial.bdl >ref-list.txt && + ls unbundled/.git/objects/pack/pack-*.promisor >promisor && + test_line_count = 1 promisor && # Count the same number of reachable objects. reflist=$(git for-each-ref --format="%(objectname)") && From patchwork Wed Mar 9 16:01:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12775229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 385B1C433EF for ; Wed, 9 Mar 2022 16:03:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234206AbiCIQD6 (ORCPT ); Wed, 9 Mar 2022 11:03:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40870 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234315AbiCIQDA (ORCPT ); Wed, 9 Mar 2022 11:03:00 -0500 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A71F417AED0 for ; Wed, 9 Mar 2022 08:02:01 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id e24so3748146wrc.10 for ; Wed, 09 Mar 2022 08:02:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=hdWHJKUC6IBcvDLdZTYSt1g97GEbtJoKDnx+sfvbRiU=; b=i8BZ/3KlyNg1JJuQB4n96R/k1PT3DVsSoZM8jXKZD68NdPsjKDJus8N8q5UgNx/u9W /OGIcd/W6JMxubgLEhnEaLUEHr22DcRpouYtcQRoM+dykeu7oZC5lt8rSkMaCEuCegvI 4gpKJPL+imURWXshLwlsOl0j566QYNAfYd/hABJiEIKmCcmnlQksUEi/8jf/Ac8qPdjl v3VHOSMrG+WLRzULh+HSV1M0XLZ7rbecpAldoqCjaQCrwZUW+qiysc6QREnbcf9PEMSy K0jSeCY8v36RpucIl2tJtuJ1eUuZuwFindwUGvVXWHr5xY8sXUQJpWOrjgB04cazX/2O WBCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=hdWHJKUC6IBcvDLdZTYSt1g97GEbtJoKDnx+sfvbRiU=; b=eSGL4MC6cX9Y9D+CNPAx37qSg/lY19zSD1XQiRvYeZ1MPp+YH7x6YFzIGC/FlRWeuK 1/+YSL3Np3eKsC//524IKNBRSCU4vyr2X3K2d0gR40RM5bL8lEuZftdFUtJzct/TDhBi dLoX+Khlw9o6bexlG00aLupGjG/dAaGGav9YEI1iJ1wJEiYpUzlZj4k5i8Ujuzj2hJ2w DRkQpdRaCGddZLC+g6ZfIdafIzvGWuevV2oOcyWq1OMCtCyX1DYGJiJRpFW632UNkI+F QftegKTHo5b3ETEt/YVOG0tR7+3F6YPUgFc+0StCfVr3C3x4q9fcWbeGgQtAToSPYXqE /r9Q== X-Gm-Message-State: AOAM533Tl6Mb83cfr18xfihTl0Br/V1zTcJlRa3pxeNin8mEuFoWstAN oZgzFpWiEzzheWlTtzKDx+dGvraWXqE= X-Google-Smtp-Source: ABdhPJy1QNjuHrK9eTaUilfDEOJG4QXzbf4ZjeKRMJoT6Qw1uZ43wsIJ7D/CcE1H71ztBjYOtma2iw== X-Received: by 2002:adf:bc09:0:b0:1f0:2483:48b with SMTP id s9-20020adfbc09000000b001f02483048bmr267066wrg.118.1646841720097; Wed, 09 Mar 2022 08:02:00 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y4-20020adff144000000b001f022290737sm1962697wro.6.2022.03.09.08.01.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Mar 2022 08:01:59 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 09 Mar 2022 16:01:43 +0000 Subject: [PATCH v4 13/13] clone: fail gracefully when cloning filtered bundle Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Jeff Hostetler , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Users can create a new repository using 'git clone '. The new "@filter" capability for bundles means that we can generate a bundle that does not contain all reachable objects, even if the header has no negative commit OIDs. It is feasible to think that we could make a filtered bundle work with the command git clone --filter=$filter --bare or possibly replacing --bare with --no-checkout. However, this requires having some repository-global config that specifies the specified object filter and notifies Git about the existence of promisor pack-files. Without a remote, that is currently impossible. As a stop-gap, parse the bundle header during 'git clone' and die() with a helpful error message instead of the current behavior of failing due to "missing objects". Most of the existing logic for handling bundle clones actually happens in fetch-pack.c, but that logic is the same as if the user specified 'git fetch ', so we want to avoid failing to fetch a filtered bundle when in an existing repository that has the proper config set up for at least one remote. Carefully comment around the test that this is not the desired long-term behavior of 'git clone' in this case, but instead that we need to do more work before that is possible. Signed-off-by: Derrick Stolee --- builtin/clone.c | 13 +++++++++++++ t/t6020-bundle-misc.sh | 12 ++++++++++++ 2 files changed, 25 insertions(+) diff --git a/builtin/clone.c b/builtin/clone.c index 9c29093b352..e57504c2aa8 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -33,6 +33,7 @@ #include "packfile.h" #include "list-objects-filter-options.h" #include "hook.h" +#include "bundle.h" /* * Overall FIXMEs: @@ -1138,6 +1139,18 @@ int cmd_clone(int argc, const char **argv, const char *prefix) warning(_("--local is ignored")); transport->cloning = 1; + if (is_bundle) { + struct bundle_header header = BUNDLE_HEADER_INIT; + int fd = read_bundle_header(path, &header); + int has_filter = header.filter.choice != LOFC_DISABLED; + + if (fd > 0) + close(fd); + bundle_header_release(&header); + if (has_filter) + die(_("cannot clone from filtered bundle")); + } + transport_set_option(transport, TRANS_OPT_KEEP, "yes"); if (reject_shallow) diff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh index 7c6db670221..ed95d195427 100755 --- a/t/t6020-bundle-misc.sh +++ b/t/t6020-bundle-misc.sh @@ -537,4 +537,16 @@ do ' done +# NEEDSWORK: 'git clone --bare' should be able to clone from a filtered +# bundle, but that requires a change to promisor/filter config options. +# For now, we fail gracefully with a helpful error. This behavior can be +# changed in the future to succeed as much as possible. +test_expect_success 'cloning from filtered bundle has useful error' ' + git bundle create partial.bdl \ + --all \ + --filter=blob:none && + test_must_fail git clone --bare partial.bdl partial 2>err && + grep "cannot clone from filtered bundle" err +' + test_done