From patchwork Wed Feb 23 17:55:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12757248 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E4FBC433FE for ; Wed, 23 Feb 2022 17:55:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243578AbiBWRzp (ORCPT ); Wed, 23 Feb 2022 12:55:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234110AbiBWRzo (ORCPT ); Wed, 23 Feb 2022 12:55:44 -0500 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7FE265BC for ; Wed, 23 Feb 2022 09:55:15 -0800 (PST) Received: by mail-wr1-x434.google.com with SMTP id d17so6076262wrc.9 for ; Wed, 23 Feb 2022 09:55:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=7/Vet349YwWXOD88+eWNxKjf9Jhn0cuBQkzK8diKGoU=; b=P29g+cBobG2E39g70chUs6flKta/3ZKJE0FVm2WkWIAyUZWjTuccGtivSWIhk4ZIze 1aeaRuGvizN/+9uRAwXDXNbcqWTc0jGy/qVvMkLKe53NGPG5NfFYA1mI4VVoA4DCBZid jBX7p48mHd3PxFJWzliKm/sL64im1jPZIG8oOJohASqstSQX3LQ20b2YehHtD9+uK4kq 8NyCbLtBVdW1oZN4JL5UNHHP13hR6d6XBg7zg+gVwWMn01pmd/l5UXlQP3bPVSHlUAY0 +9dj+ODmu4KuvVtNTD3685HcOI5Rv7cpjRl3yiDM48dqrOjAGHJf5AdjYWORsl4iJ6J0 fQ/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=7/Vet349YwWXOD88+eWNxKjf9Jhn0cuBQkzK8diKGoU=; b=KFeoxWKRvxcxdLKHgSy4KWdcx+933zpgUWCg+ULD2dXH3tYI7QW/2KJ0vEiyWHzrMP AB2lCVIxCSFTabAp0yySRt1nHqLWuW/5DTxPAy+5rUB1poW61Hdtp/7TuPyposbSXW3t UlLMzr3UdcE2iWlLYX6qZsFShpqJYpAGuyfsPBvH5Dsv0FIJCTod0o+BS4EWeJgf5zWN STCyCo+VnalHSmcMoboU7PyF3ckScznj/njexgfuIjKzc+JS8PSdmeIcZqgGHa+ikGVx R5SGI1x8+aHmDjibYGqDoztvX6b2iOhjIvm0G0lAyVjRRtK2S0EhIclDdPfmkPkdlvWT XpRw== X-Gm-Message-State: AOAM533b3AYGs4TZvq7UOqaMtKOFIb1iF/DVufDoHY04HAr/yc26iYUe VAgE0rbKe5l03wp+Bsiu9Yr/X4H8yGk= X-Google-Smtp-Source: ABdhPJyzEb7AOb1Xq6Iibpiwml7lsMVjk8g9/NxrxXiklUYkzeH9V7jyVwbX4SiiskbSuuXONLbvvg== X-Received: by 2002:a5d:47ae:0:b0:1e8:e479:fa8e with SMTP id 14-20020a5d47ae000000b001e8e479fa8emr557719wrb.169.1645638914005; Wed, 23 Feb 2022 09:55:14 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j9sm276747wmc.5.2022.02.23.09.55.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Feb 2022 09:55:13 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 23 Feb 2022 17:55:01 +0000 Subject: [PATCH 01/11] index-pack: document and test the --promisor option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The --promisor option of 'git index-pack' was created in 88e2f9e (introduce fetch-object: fetch one promisor object, 2017-12-05) but was untested. It is currently unused within the Git codebase, but that will change in an upcoming change to 'git bundle unbundle' when there is a filter capability. For now, add documentation about the option and add a test to ensure it is working as expected. Signed-off-by: Derrick Stolee --- Documentation/git-index-pack.txt | 8 ++++++++ t/t5300-pack-object.sh | 4 +++- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/Documentation/git-index-pack.txt b/Documentation/git-index-pack.txt index 1f1e3592251..4e71c256ecb 100644 --- a/Documentation/git-index-pack.txt +++ b/Documentation/git-index-pack.txt @@ -122,6 +122,14 @@ This option cannot be used with --stdin. + include::object-format-disclaimer.txt[] +--promisor[=]:: + Before committing the pack-index, create a .promisor file for this + pack. Particularly helpful when writing a promisor pack with --fix-thin + since the name of the pack is not final until the pack has been fully + written. If a `` is provided, then that content will be + written to the .promisor file for future reference. See + link:technical/partial-clone.html[partial clone] for more information. + NOTES ----- diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh index 2fd845187e7..a11d61206ad 100755 --- a/t/t5300-pack-object.sh +++ b/t/t5300-pack-object.sh @@ -315,8 +315,10 @@ test_expect_success \ git index-pack -o tmp.idx test-3.pack && cmp tmp.idx test-1-${packname_1}.idx && - git index-pack test-3.pack && + git index-pack --promisor=message test-3.pack && cmp test-3.idx test-1-${packname_1}.idx && + echo message >expect && + test_cmp expect test-3.promisor && cat test-2-${packname_2}.pack >test-3.pack && git index-pack -o tmp.idx test-2-${packname_2}.pack && From patchwork Wed Feb 23 17:55:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12757249 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 750E4C433EF for ; Wed, 23 Feb 2022 17:55:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243589AbiBWRzu (ORCPT ); Wed, 23 Feb 2022 12:55:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243577AbiBWRzp (ORCPT ); Wed, 23 Feb 2022 12:55:45 -0500 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1BF572736 for ; Wed, 23 Feb 2022 09:55:17 -0800 (PST) Received: by mail-wr1-x432.google.com with SMTP id o4so4154942wrf.3 for ; Wed, 23 Feb 2022 09:55:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=sclOhVe4EV0U75UTToLOlsrVExWmGOlFRPY6iEbDWEo=; b=P7/Ldv6SIGiLL/RkC2BsSyBD8I53ombjUL2gk3i6bdVFABaXCRcprclDkvYCVW2rTM bQjOiOe0AJTwDYzPrp7FCSEXyjbd2ur1G+DJsI21N2XjIIaRgCnUKFoWS8uWAVp7c0Od K2Qu7b7Nbx6hUItPjhwDUX8mflvwrPLhGc8iL5QHaOPBB9BMxoBXT2W9XO74dMx7ijhv M19ZYJM+s5FXcGLB3oOP6GFJiNqoQNhvLtTVzt9eFHZE+XZdOprnZd9AJL6GMOgJYLa7 pZ65WX/IYgQ4hyv5lbXU6z5myupO/NKGTdO3JxgmeHz6IfdVpji5Xx28OnJZXwMl1QRh esaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=sclOhVe4EV0U75UTToLOlsrVExWmGOlFRPY6iEbDWEo=; b=jqcmSkSeizWtFI0Vuq2I5jmrTrsum5DvuAXYorMtgRpbTB2EjnW5cm3XXL28vbV+0V iIVf54DxZhFg7uCcGthdFpomYmdcEOKWiNn5TvYnNegO9I+8/ALb8NFxXUActRylovUM PGN0u/HNYFIoiixmbDj94p4R13OWVe8XAzhzCq71M/Zj/03YSSmaqS6SetGD9C1Za2X5 qzre9eQqFHhOvn2nbnbsdZa+1n1LspPdM6wgqXLZJV8wnreCbf0p1bvS+SQJKz4P1Hur lw+aT97Vq8FTe5KS5v3vfhxuCwz4EFmREZYGsP7qYp1KUuET5E4+X5cLrrheoevgik6o 5DSA== X-Gm-Message-State: AOAM532gRlLaKmnH7afzs8qNJv2r5HyGF/ud9iq/ZJ+7u2CIk0cI+P3k pW7fbO3JUrLufM0DkGkTlHG9ouxKAQ0= X-Google-Smtp-Source: ABdhPJx+8dniDZ1+YE/x2pCysORcPY3ilVJTygA7paGBkVKg1pLPVfK678pR7FA2Xr+GQJsZwhS8Nw== X-Received: by 2002:a5d:6da7:0:b0:1ea:78a4:8e00 with SMTP id u7-20020a5d6da7000000b001ea78a48e00mr555265wrs.129.1645638915315; Wed, 23 Feb 2022 09:55:15 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e6sm211506wrn.67.2022.02.23.09.55.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Feb 2022 09:55:14 -0800 (PST) Message-Id: <3a88c99d9bc765bf4728fe0f0df1eed86adace0e.1645638911.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 23 Feb 2022 17:55:02 +0000 Subject: [PATCH 02/11] revision: put object filter into struct rev_info Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Placing a 'struct list_objects_filter_options' pointer within 'struct rev_info' will assist making some bookkeeping around object filters in the future. For now, let's use this new member to remove a static global instance of the struct from builtin/rev-list.c. Signed-off-by: Derrick Stolee --- builtin/rev-list.c | 30 ++++++++++++++++-------------- revision.h | 4 ++++ 2 files changed, 20 insertions(+), 14 deletions(-) diff --git a/builtin/rev-list.c b/builtin/rev-list.c index 777558e9b06..6f2b91d304e 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -62,7 +62,6 @@ static const char rev_list_usage[] = static struct progress *progress; static unsigned progress_counter; -static struct list_objects_filter_options filter_options; static struct oidset omitted_objects; static int arg_print_omitted; /* print objects omitted by filter */ @@ -400,7 +399,6 @@ static inline int parse_missing_action_value(const char *value) } static int try_bitmap_count(struct rev_info *revs, - struct list_objects_filter_options *filter, int filter_provided_objects) { uint32_t commit_count = 0, @@ -436,7 +434,8 @@ static int try_bitmap_count(struct rev_info *revs, */ max_count = revs->max_count; - bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects); + bitmap_git = prepare_bitmap_walk(revs, revs->filter, + filter_provided_objects); if (!bitmap_git) return -1; @@ -453,7 +452,6 @@ static int try_bitmap_count(struct rev_info *revs, } static int try_bitmap_traversal(struct rev_info *revs, - struct list_objects_filter_options *filter, int filter_provided_objects) { struct bitmap_index *bitmap_git; @@ -465,7 +463,8 @@ static int try_bitmap_traversal(struct rev_info *revs, if (revs->max_count >= 0) return -1; - bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects); + bitmap_git = prepare_bitmap_walk(revs, revs->filter, + filter_provided_objects); if (!bitmap_git) return -1; @@ -475,7 +474,6 @@ static int try_bitmap_traversal(struct rev_info *revs, } static int try_bitmap_disk_usage(struct rev_info *revs, - struct list_objects_filter_options *filter, int filter_provided_objects) { struct bitmap_index *bitmap_git; @@ -483,7 +481,7 @@ static int try_bitmap_disk_usage(struct rev_info *revs, if (!show_disk_usage) return -1; - bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects); + bitmap_git = prepare_bitmap_walk(revs, revs->filter, filter_provided_objects); if (!bitmap_git) return -1; @@ -597,13 +595,17 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) } if (skip_prefix(arg, ("--" CL_ARG__FILTER "="), &arg)) { - parse_list_objects_filter(&filter_options, arg); - if (filter_options.choice && !revs.blob_objects) + if (!revs.filter) + CALLOC_ARRAY(revs.filter, 1); + parse_list_objects_filter(revs.filter, arg); + if (revs.filter->choice && !revs.blob_objects) die(_("object filtering requires --objects")); continue; } if (!strcmp(arg, ("--no-" CL_ARG__FILTER))) { - list_objects_filter_set_no_filter(&filter_options); + if (!revs.filter) + CALLOC_ARRAY(revs.filter, 1); + list_objects_filter_set_no_filter(revs.filter); continue; } if (!strcmp(arg, "--filter-provided-objects")) { @@ -688,11 +690,11 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) progress = start_delayed_progress(show_progress, 0); if (use_bitmap_index) { - if (!try_bitmap_count(&revs, &filter_options, filter_provided_objects)) + if (!try_bitmap_count(&revs, filter_provided_objects)) return 0; - if (!try_bitmap_disk_usage(&revs, &filter_options, filter_provided_objects)) + if (!try_bitmap_disk_usage(&revs, filter_provided_objects)) return 0; - if (!try_bitmap_traversal(&revs, &filter_options, filter_provided_objects)) + if (!try_bitmap_traversal(&revs, filter_provided_objects)) return 0; } @@ -733,7 +735,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) oidset_init(&missing_objects, DEFAULT_OIDSET_SIZE); traverse_commit_list_filtered( - &filter_options, &revs, show_commit, show_object, &info, + revs.filter, &revs, show_commit, show_object, &info, (arg_print_omitted ? &omitted_objects : NULL)); if (arg_print_omitted) { diff --git a/revision.h b/revision.h index 3c58c18c63a..1ddb73ab82e 100644 --- a/revision.h +++ b/revision.h @@ -81,6 +81,7 @@ struct rev_cmdline_info { struct oidset; struct topo_walk_info; +struct list_objects_filter_options; struct rev_info { /* Starting list */ @@ -94,6 +95,9 @@ struct rev_info { /* The end-points specified by the end user */ struct rev_cmdline_info cmdline; + /* Object filter options. NULL for no filtering. */ + struct list_objects_filter_options *filter; + /* excluding from --branches, --refs, etc. expansion */ struct string_list *ref_excludes; From patchwork Wed Feb 23 17:55:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12757250 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56E88C433F5 for ; Wed, 23 Feb 2022 17:55:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243580AbiBWRzv (ORCPT ); Wed, 23 Feb 2022 12:55:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243579AbiBWRzq (ORCPT ); Wed, 23 Feb 2022 12:55:46 -0500 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E694A63A0 for ; Wed, 23 Feb 2022 09:55:17 -0800 (PST) Received: by mail-wr1-x429.google.com with SMTP id j22so14838370wrb.13 for ; Wed, 23 Feb 2022 09:55:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=h62JVzb/uF3I+8SLeb9g0QKm9zTbW6d6LFRo3OfnaqQ=; b=aLvLcyOrw3dVD4fgmLkC5kEweHhK05RJtFOkkcwLPxLdKxVRV1HPgSH2/8lcFEPRYi rYCov8YsG+zrvIjnBVQviCwPzCnkHKZ0/RczVuRI4UqxQMv2t5yil0VhrHSwoKm/75cL XpSvQc2PZlFS9HnSkU5kJELhE53cVnX4eZXZbQOmewecds1rDI3rEBR8Y9jGsh7Leyn1 FbYQS1yApQpue+Nwyf40eBfg8MoWaVEs2lrPdBLPHalkuY5wwHImiF5HFxiY37BD/uMH Vyb3eaejh2HZo9I9TNE6YAW+//gklYaxko0gY0R1RCVsJ0NilefZoJGkP0nu0ksiB/m5 9dWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=h62JVzb/uF3I+8SLeb9g0QKm9zTbW6d6LFRo3OfnaqQ=; b=ryRbHFbLJH7NGO+Vf2A7d/jyLjzFNDoJPGGRlJpnWrpZ++QbtwFOh6xJPOb6Vpqp1E YiudSxpqHNpUaoOUKWISppTBRqSIjOgitZ/pacVquEiQAOG+xM/ZfTl8jIo5HCTxgpGN c+Fv3zwRuL3WA1QR8ESd6OUexLVe4zRaoJVDKegPiYj7VZejfFsltT6OGX0EQiYAA5t3 gS7yJRK+nt3hiltcFzkrVBEh73093dw4Aa0+4jqqekVv2BH9HixiCfoQ2PeOvd6YBFgw RfXiN1hDvuJfEjvg1hb0TfGVOJZa7EOVohpcu2u6RzRVhD6dWjA8Twau0IqeuwPVqsRA K/0Q== X-Gm-Message-State: AOAM5338s3HK3PeI6TtrRUZ1DlZwfcOcRWWZZPmRHnRgg6mdxc6kwHzQ MJLcm3Y6D+THkJdZhhusuZ0jTJvHL+w= X-Google-Smtp-Source: ABdhPJyK2tJpKQ92t2somcQfAp+jGq/H4eu0OhPk8gtwX1OOhd6T3KTz1CBXcgh4dmYlgAaTOSEjug== X-Received: by 2002:a05:6000:1547:b0:1ea:7d56:83e8 with SMTP id 7-20020a056000154700b001ea7d5683e8mr605433wry.404.1645638916437; Wed, 23 Feb 2022 09:55:16 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j12sm240765wrs.1.2022.02.23.09.55.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Feb 2022 09:55:15 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 23 Feb 2022 17:55:03 +0000 Subject: [PATCH 03/11] pack-objects: use rev.filter when possible Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee In builtin/pack-objects.c, we use a 'filter_options' global to populate the --filter= argument. The previous change created a pointer to a filter option in 'struct rev_info', so we can use that pointer here as a start to simplifying some usage of object filters. Signed-off-by: Derrick Stolee --- builtin/pack-objects.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index ba2006f2212..256d9b1798f 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -3651,7 +3651,7 @@ static int pack_options_allow_reuse(void) static int get_object_list_from_bitmap(struct rev_info *revs) { - if (!(bitmap_git = prepare_bitmap_walk(revs, &filter_options, 0))) + if (!(bitmap_git = prepare_bitmap_walk(revs, revs->filter, 0))) return -1; if (pack_options_allow_reuse() && @@ -3727,6 +3727,7 @@ static void get_object_list(int ac, const char **av) repo_init_revisions(the_repository, &revs, NULL); save_commit_buffer = 0; setup_revisions(ac, av, &revs, &s_r_opt); + revs.filter = &filter_options; /* make sure shallows are read */ is_repository_shallow(the_repository); @@ -3777,7 +3778,7 @@ static void get_object_list(int ac, const char **av) if (!fn_show_object) fn_show_object = show_object; - traverse_commit_list_filtered(&filter_options, &revs, + traverse_commit_list_filtered(revs.filter, &revs, show_commit, fn_show_object, NULL, NULL); From patchwork Wed Feb 23 17:55:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12757252 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3EDBC433FE for ; Wed, 23 Feb 2022 17:55:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243587AbiBWRzw (ORCPT ); Wed, 23 Feb 2022 12:55:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49116 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235401AbiBWRzt (ORCPT ); Wed, 23 Feb 2022 12:55:49 -0500 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE96365EC for ; Wed, 23 Feb 2022 09:55:18 -0800 (PST) Received: by mail-wm1-x333.google.com with SMTP id c18-20020a7bc852000000b003806ce86c6dso2044578wml.5 for ; Wed, 23 Feb 2022 09:55:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=q8+BEG3NwnoERyq+lPMXDfKtOelfHmetpKek56q/MYw=; b=gMri/RKKRgjqJBzDyo5vIsK0gxxPmBZUaWLiXJl1gawiCPbysh7aqmnerc1idrdzmZ EYVKwGeWz+pguvicrbh6o9QCo3/Dso/DS0ALIlE//SA1/jQHhSeWMVybUjMV5Tz1+Gtl As8cyRv1kElZvJJ+sEaV4VqD44agMv5YD0BHSmrngP0bbYlBBUvFho/972TnwZUaemcX k6Qyp81EKQ22RmdvJLCWM90L9ynqmI0yME8hvrefLB4mysxEOLEUGtIJ4grdQ9JYJBDh UE+gPk9r2adjVZGh1tb+SBii0ghKpENi/GdfkKZPEKwstT0rPpuvNu5vVYV/l5d+FiNd ucIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=q8+BEG3NwnoERyq+lPMXDfKtOelfHmetpKek56q/MYw=; b=T915l5dAfwjUfGYja6Djg1c37LRg8wKIeytXeYoIWubp+iTExm5IOL01He/8B+Hzrn gh41+ioynqdBpYsPjAb7wDzVm5ygrfP1INqKDzCofdBN8QX7CXmAcJH+nVE/iM1maIYy rZJgSyklGGT/dpMpk3bQPehnKn/YwnWdpGtxDUvZFJYU28uX7rbdURS7AkiH/Q0A14JB rkpK+TACV0kb3+/5+FuT7AandCvs9Am+Zf/yLPaSTE8I9ZOBQYmKVZNZDT2JMfkoqny1 BlRFwJbC87G7Mc6+O+bp9mcGBqZsExxRllW5Hpldura8Y9G2WM+sIGLcB1ckpTyzVhxP H4KA== X-Gm-Message-State: AOAM531IVaDlEjQJJIQ7+1xlPeCYmcL9QmM3vT+okqFxdK1NXKyGk2Hx ig2klNG3BpBaeXG4YiMcRHRYpbjW12I= X-Google-Smtp-Source: ABdhPJy1p44IByKm2u+whb+yBtxCn8vec+Ko2jGmy7Ph75upDOXg1ld8qrJ+tmnEDR8L2fUku2ADOw== X-Received: by 2002:a05:600c:4b92:b0:380:e46c:c35 with SMTP id e18-20020a05600c4b9200b00380e46c0c35mr662536wmp.15.1645638917150; Wed, 23 Feb 2022 09:55:17 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l38-20020a05600c1d2600b0037e9090fb1esm422040wms.24.2022.02.23.09.55.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Feb 2022 09:55:16 -0800 (PST) Message-Id: <888774f6f28b291c928041a32c3df360cee13d10.1645638911.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 23 Feb 2022 17:55:04 +0000 Subject: [PATCH 04/11] pack-bitmap: drop filter in prepare_bitmap_walk() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Now that all consumers of prepare_bitmap_walk() have populated the 'filter' member of 'struct rev_info', we can drop that extra parameter from the method and access it directly from the 'struct rev_info'. Signed-off-by: Derrick Stolee --- builtin/pack-objects.c | 2 +- builtin/rev-list.c | 8 +++----- pack-bitmap.c | 20 +++++++++----------- pack-bitmap.h | 2 -- reachable.c | 2 +- 5 files changed, 14 insertions(+), 20 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 256d9b1798f..57f2cf49696 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -3651,7 +3651,7 @@ static int pack_options_allow_reuse(void) static int get_object_list_from_bitmap(struct rev_info *revs) { - if (!(bitmap_git = prepare_bitmap_walk(revs, revs->filter, 0))) + if (!(bitmap_git = prepare_bitmap_walk(revs, 0))) return -1; if (pack_options_allow_reuse() && diff --git a/builtin/rev-list.c b/builtin/rev-list.c index 6f2b91d304e..556e78aebb9 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -434,8 +434,7 @@ static int try_bitmap_count(struct rev_info *revs, */ max_count = revs->max_count; - bitmap_git = prepare_bitmap_walk(revs, revs->filter, - filter_provided_objects); + bitmap_git = prepare_bitmap_walk(revs, filter_provided_objects); if (!bitmap_git) return -1; @@ -463,8 +462,7 @@ static int try_bitmap_traversal(struct rev_info *revs, if (revs->max_count >= 0) return -1; - bitmap_git = prepare_bitmap_walk(revs, revs->filter, - filter_provided_objects); + bitmap_git = prepare_bitmap_walk(revs, filter_provided_objects); if (!bitmap_git) return -1; @@ -481,7 +479,7 @@ static int try_bitmap_disk_usage(struct rev_info *revs, if (!show_disk_usage) return -1; - bitmap_git = prepare_bitmap_walk(revs, revs->filter, filter_provided_objects); + bitmap_git = prepare_bitmap_walk(revs, filter_provided_objects); if (!bitmap_git) return -1; diff --git a/pack-bitmap.c b/pack-bitmap.c index 9c666cdb8bd..613f2797cdf 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -739,8 +739,7 @@ static int add_commit_to_bitmap(struct bitmap_index *bitmap_git, static struct bitmap *find_objects(struct bitmap_index *bitmap_git, struct rev_info *revs, struct object_list *roots, - struct bitmap *seen, - struct list_objects_filter_options *filter) + struct bitmap *seen) { struct bitmap *base = NULL; int needs_walk = 0; @@ -823,7 +822,7 @@ static struct bitmap *find_objects(struct bitmap_index *bitmap_git, show_data.bitmap_git = bitmap_git; show_data.base = base; - traverse_commit_list_filtered(filter, revs, + traverse_commit_list_filtered(revs->filter, revs, show_commit, show_object, &show_data, NULL); @@ -1219,7 +1218,6 @@ static int can_filter_bitmap(struct list_objects_filter_options *filter) } struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, - struct list_objects_filter_options *filter, int filter_provided_objects) { unsigned int i; @@ -1240,7 +1238,7 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, if (revs->prune) return NULL; - if (!can_filter_bitmap(filter)) + if (!can_filter_bitmap(revs->filter)) return NULL; /* try to open a bitmapped pack, but don't parse it yet @@ -1297,8 +1295,7 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, if (haves) { revs->ignore_missing_links = 1; - haves_bitmap = find_objects(bitmap_git, revs, haves, NULL, - filter); + haves_bitmap = find_objects(bitmap_git, revs, haves, NULL); reset_revision_walk(); revs->ignore_missing_links = 0; @@ -1306,8 +1303,7 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, BUG("failed to perform bitmap walk"); } - wants_bitmap = find_objects(bitmap_git, revs, wants, haves_bitmap, - filter); + wants_bitmap = find_objects(bitmap_git, revs, wants, haves_bitmap); if (!wants_bitmap) BUG("failed to perform bitmap walk"); @@ -1315,8 +1311,10 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, if (haves_bitmap) bitmap_and_not(wants_bitmap, haves_bitmap); - filter_bitmap(bitmap_git, (filter && filter_provided_objects) ? NULL : wants, - wants_bitmap, filter); + filter_bitmap(bitmap_git, + (revs->filter && filter_provided_objects) ? NULL : wants, + wants_bitmap, + revs->filter); bitmap_git->result = wants_bitmap; bitmap_git->haves = haves_bitmap; diff --git a/pack-bitmap.h b/pack-bitmap.h index 19a63fa1abc..3d3ddd77345 100644 --- a/pack-bitmap.h +++ b/pack-bitmap.h @@ -10,7 +10,6 @@ struct commit; struct repository; struct rev_info; -struct list_objects_filter_options; static const char BITMAP_IDX_SIGNATURE[] = {'B', 'I', 'T', 'M'}; @@ -54,7 +53,6 @@ void test_bitmap_walk(struct rev_info *revs); int test_bitmap_commits(struct repository *r); int test_bitmap_hashes(struct repository *r); struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, - struct list_objects_filter_options *filter, int filter_provided_objects); uint32_t midx_preferred_pack(struct bitmap_index *bitmap_git); int reuse_partial_packfile_from_bitmap(struct bitmap_index *, diff --git a/reachable.c b/reachable.c index 84e3d0d75ed..b9f4ad886ef 100644 --- a/reachable.c +++ b/reachable.c @@ -205,7 +205,7 @@ void mark_reachable_objects(struct rev_info *revs, int mark_reflog, cp.progress = progress; cp.count = 0; - bitmap_git = prepare_bitmap_walk(revs, NULL, 0); + bitmap_git = prepare_bitmap_walk(revs, 0); if (bitmap_git) { traverse_bitmap_commit_list(bitmap_git, revs, mark_object_seen); free_bitmap_index(bitmap_git); From patchwork Wed Feb 23 17:55:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12757251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3ED0EC4332F for ; Wed, 23 Feb 2022 17:55:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243595AbiBWRzw (ORCPT ); Wed, 23 Feb 2022 12:55:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49120 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243583AbiBWRzt (ORCPT ); Wed, 23 Feb 2022 12:55:49 -0500 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89D8F62F7 for ; Wed, 23 Feb 2022 09:55:19 -0800 (PST) Received: by mail-wm1-x329.google.com with SMTP id c192so14248729wma.4 for ; Wed, 23 Feb 2022 09:55:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=FheJfNt4rgBk9mCbt6/7lxWi/jD4IcqMQZxKGewAEyY=; b=ZDZZ4QoFRy50Nbw9eEpFwKehGiW72vRC/VvXBg7nkEkYeT0+4uQkSoatbnJYE5opJO ut/cyt6/eh7i1RvQZiUblFcByw83q25oP1gM8cq/JzAw8n7vuUqqkhqdtrTdqREh/oGe JAOtlYfVKNK+8/oLCj8ZEbrFWU7MTP8CBLcsyXzMF4arVC4shb8b+MGtC1IPcEDGrrNy BEIc5YyHpPNoSkXPH2HtWn/Qw0HvUrQnuIbhq/9H4+zPaxzm+mwMqbaJT7m7OHMNtFQC p1euw4j6Dd3VNOy7pn6zGeMwZQJabCbXP2m9HZGS0jo3oXgUF/qtkUL6eLEvEbaiN35B Vl/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=FheJfNt4rgBk9mCbt6/7lxWi/jD4IcqMQZxKGewAEyY=; b=VbwpnnMI0RNO3/lxO4vcCKbEAGtbJo/cC/gEpAVmHzPAvr4VRj0ZYA5XM4QfwiPOVN 4wUu1+0Gacx99gm+uMBiiT7koB2eG/QdE18OqV1XK9ggjTmc5xS9rY+W6Xa7f73fbGrC t1dSGkC/M76yEQNxS3EmK75/g1FIC8uMh/Yn2CveciNI6Bqh2EE1W2T+BRqv1FN5tMql 6nvroK0QvzD2ttddFbsf+/Qwh8bxrfJrjzYOvIqFcKK0kZQlTga+30kE6JVb84gQA1Bm XXmJVIbtgmkgOEU0kNMAr7iJJf1dhskTGvlCsPVes3d3ce8lNAhKeyEMXRzFvzpi2+u4 cneg== X-Gm-Message-State: AOAM530264tZZF0GEX310BbRyGm29b6P21MmGdOPFM9e851nIpctPFhQ jqVrmeXee0XWezMHt4b21WqCFOQIYGg= X-Google-Smtp-Source: ABdhPJxpmQS9BGCctTSfRJtBUqSVfKNOZ//bfj/FEZlXf4sy+UlxBMGiI6CXOVW3qZVUeTXd9mvJZA== X-Received: by 2002:a7b:cd87:0:b0:37b:b8a8:2d28 with SMTP id y7-20020a7bcd87000000b0037bb8a82d28mr645947wmj.176.1645638918004; Wed, 23 Feb 2022 09:55:18 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m21-20020a05600c4f5500b0037d342db78asm387534wmq.35.2022.02.23.09.55.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Feb 2022 09:55:17 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 23 Feb 2022 17:55:05 +0000 Subject: [PATCH 05/11] list-objects: consolidate traverse_commit_list[_filtered] Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Now that all consumers of traverse_commit_list_filtered() populate the 'filter' member of 'struct rev_info', we can drop that parameter from the method prototype to simplify things. In addition, the only thing different now between traverse_commit_list_filtered() and traverse_commit_list() is the presence of the 'omitted' parameter, which is only non-NULL for one caller. We can consolidate these two methods by having one call the other and use the simpler form everywhere the 'omitted' paramter would be NULL. Signed-off-by: Derrick Stolee --- builtin/pack-objects.c | 6 +++--- builtin/rev-list.c | 2 +- list-objects.c | 25 ++++++++----------------- list-objects.h | 12 ++++++++++-- pack-bitmap.c | 6 +++--- 5 files changed, 25 insertions(+), 26 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 57f2cf49696..0432ae1e499 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -3778,9 +3778,9 @@ static void get_object_list(int ac, const char **av) if (!fn_show_object) fn_show_object = show_object; - traverse_commit_list_filtered(revs.filter, &revs, - show_commit, fn_show_object, NULL, - NULL); + traverse_commit_list(&revs, + show_commit, fn_show_object, + NULL); if (unpack_unreachable_expiration) { revs.ignore_missing_links = 1; diff --git a/builtin/rev-list.c b/builtin/rev-list.c index 556e78aebb9..3ab727817fd 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -733,7 +733,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) oidset_init(&missing_objects, DEFAULT_OIDSET_SIZE); traverse_commit_list_filtered( - revs.filter, &revs, show_commit, show_object, &info, + &revs, show_commit, show_object, &info, (arg_print_omitted ? &omitted_objects : NULL)); if (arg_print_omitted) { diff --git a/list-objects.c b/list-objects.c index 2f623f82115..9422625b39e 100644 --- a/list-objects.c +++ b/list-objects.c @@ -416,22 +416,7 @@ static void do_traverse(struct traversal_context *ctx) strbuf_release(&csp); } -void traverse_commit_list(struct rev_info *revs, - show_commit_fn show_commit, - show_object_fn show_object, - void *show_data) -{ - struct traversal_context ctx; - ctx.revs = revs; - ctx.show_commit = show_commit; - ctx.show_object = show_object; - ctx.show_data = show_data; - ctx.filter = NULL; - do_traverse(&ctx); -} - void traverse_commit_list_filtered( - struct list_objects_filter_options *filter_options, struct rev_info *revs, show_commit_fn show_commit, show_object_fn show_object, @@ -444,7 +429,13 @@ void traverse_commit_list_filtered( ctx.show_object = show_object; ctx.show_commit = show_commit; ctx.show_data = show_data; - ctx.filter = list_objects_filter__init(omitted, filter_options); + if (revs->filter) + ctx.filter = list_objects_filter__init(omitted, revs->filter); + else + ctx.filter = NULL; + do_traverse(&ctx); - list_objects_filter__free(ctx.filter); + + if (ctx.filter) + list_objects_filter__free(ctx.filter); } diff --git a/list-objects.h b/list-objects.h index a952680e466..9eaf4de8449 100644 --- a/list-objects.h +++ b/list-objects.h @@ -7,7 +7,6 @@ struct rev_info; typedef void (*show_commit_fn)(struct commit *, void *); typedef void (*show_object_fn)(struct object *, const char *, void *); -void traverse_commit_list(struct rev_info *, show_commit_fn, show_object_fn, void *); typedef void (*show_edge_fn)(struct commit *); void mark_edges_uninteresting(struct rev_info *revs, @@ -18,11 +17,20 @@ struct oidset; struct list_objects_filter_options; void traverse_commit_list_filtered( - struct list_objects_filter_options *filter_options, struct rev_info *revs, show_commit_fn show_commit, show_object_fn show_object, void *show_data, struct oidset *omitted); +static inline void traverse_commit_list( + struct rev_info *revs, + show_commit_fn show_commit, + show_object_fn show_object, + void *show_data) +{ + traverse_commit_list_filtered(revs, show_commit, + show_object, show_data, NULL); +} + #endif /* LIST_OBJECTS_H */ diff --git a/pack-bitmap.c b/pack-bitmap.c index 613f2797cdf..cbefaedbf43 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -822,9 +822,9 @@ static struct bitmap *find_objects(struct bitmap_index *bitmap_git, show_data.bitmap_git = bitmap_git; show_data.base = base; - traverse_commit_list_filtered(revs->filter, revs, - show_commit, show_object, - &show_data, NULL); + traverse_commit_list(revs, + show_commit, show_object, + &show_data); revs->include_check = NULL; revs->include_check_obj = NULL; From patchwork Wed Feb 23 17:55:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12757253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33D3EC43217 for ; Wed, 23 Feb 2022 17:55:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243597AbiBWRzx (ORCPT ); Wed, 23 Feb 2022 12:55:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243584AbiBWRzt (ORCPT ); Wed, 23 Feb 2022 12:55:49 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6CED47676 for ; Wed, 23 Feb 2022 09:55:20 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id y5so3789706wmi.0 for ; Wed, 23 Feb 2022 09:55:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=nGmhClemBjCXUaD+XiZ/cSIN9riLpu6JFE1FglUlAQg=; b=lkWujp2eUs+W/Vcv0miS8EuhaA2E3QFHVj7q6RWALJ+Jl18z7UQ3P2iogMA1PojnF7 xDpI5t6hioZSsLyOmZRxYamM4TzPA+d1BPvagbgEWIv0mNjm24W51RW+S/i5tcMDgpJ2 VO7wMB95feuzw0rmNGcCmRX4Pj2dJ+a4nfpDu7hDjf6/i/C6DLCNgLhtpE2MOw6F3vNe lo/iFOTrNfVyJuCY216sMDkuScnc6Rj8DDV0sOnPE21LvbFD1kk0hHvDzPgo4bKnHLp8 +AKCzaPBV/JzZc5uNbkWktdiignEM1yZ5/3t/vYoj/xP+AVSRHzL+PKhRu8V7oG7Hrwx XbHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=nGmhClemBjCXUaD+XiZ/cSIN9riLpu6JFE1FglUlAQg=; b=ApHso5fEZ2gys0ckydnVBGsG6/bnakkcFlNHZ7DouON1j1HknVsjAutGlfEmJIzX34 R8k8NNvpokVxg8sln08R1xetrBb+TcdtwFl8fcJXtyCNbBbzBvKmkJanj5INVkedreGB s72BiTAMYmGTA2O5CKAoKoRt2Lu+IbArZGIpEtyj13NCgMCfZMCMpGIVhQQiPJmQZTGu mo/JL9qzfAnq54YSgyxfLBu1hLGtqGWCoHqJce/A1/iuGTz3Mqz1Otbfu2A3YBnqY4e0 UHGYobgrm9TqoXdDyrH/2GO7+7E7MOZ+FS0rSsPL3iVETpcD/ssJHAQqHn3e8MyEKVNr HRMQ== X-Gm-Message-State: AOAM532Xgm1jJVbxSrW7Mmahp7TLCAdVZhWmlbe+38q6VBTcFx84dd07 P032MS+ucUm+c9MypTGmmpRJtV05CIA= X-Google-Smtp-Source: ABdhPJz4irxormOzt/w5Ogf0skQPjPPXy26QOGcFQnir3oiP4qPVjJW1URQ+M8GFl63nlPhTWy0a0A== X-Received: by 2002:a05:600c:2319:b0:380:e369:83ea with SMTP id 25-20020a05600c231900b00380e36983eamr652403wmo.45.1645638918744; Wed, 23 Feb 2022 09:55:18 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g6sm194451wrq.97.2022.02.23.09.55.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Feb 2022 09:55:18 -0800 (PST) Message-Id: <355c503157ad02e6106179c2dc7228bdf63a6228.1645638911.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 23 Feb 2022 17:55:06 +0000 Subject: [PATCH 06/11] MyFirstObjectWalk: update recommended usage Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The previous change consolidated traverse_commit_list() and traverse_commit_list_filtered(). This allows us to simplify the recommended usage in MyFirstObjectWalk.txt to use this new set of values. While here, add some clarification on the difference between the two methods. Signed-off-by: Derrick Stolee --- Documentation/MyFirstObjectWalk.txt | 44 +++++++++++------------------ 1 file changed, 16 insertions(+), 28 deletions(-) diff --git a/Documentation/MyFirstObjectWalk.txt b/Documentation/MyFirstObjectWalk.txt index ca267941f3e..8ec83185b8a 100644 --- a/Documentation/MyFirstObjectWalk.txt +++ b/Documentation/MyFirstObjectWalk.txt @@ -522,24 +522,25 @@ function shows that the all-object walk is being performed by `traverse_commit_list()` or `traverse_commit_list_filtered()`. Those two functions reside in `list-objects.c`; examining the source shows that, despite the name, these functions traverse all kinds of objects. Let's have a look at -the arguments to `traverse_commit_list_filtered()`, which are a superset of the -arguments to the unfiltered version. +the arguments to `traverse_commit_list()`. -- `struct list_objects_filter_options *filter_options`: This is a struct which - stores a filter-spec as outlined in `Documentation/rev-list-options.txt`. -- `struct rev_info *revs`: This is the `rev_info` used for the walk. +- `struct rev_info *revs`: This is the `rev_info` used for the walk. It + includes a `filter` member which contains information for how to filter + the object list. - `show_commit_fn show_commit`: A callback which will be used to handle each individual commit object. - `show_object_fn show_object`: A callback which will be used to handle each non-commit object (so each blob, tree, or tag). - `void *show_data`: A context buffer which is passed in turn to `show_commit` and `show_object`. + +In addition, `traverse_commit_list_filtered()` has an additional paramter: + - `struct oidset *omitted`: A linked-list of object IDs which the provided filter caused to be omitted. -It looks like this `traverse_commit_list_filtered()` uses callbacks we provide -instead of needing us to call it repeatedly ourselves. Cool! Let's add the -callbacks first. +It looks like these methods use callbacks we provide instead of needing us +to call it repeatedly ourselves. Cool! Let's add the callbacks first. For the sake of this tutorial, we'll simply keep track of how many of each kind of object we find. At file scope in `builtin/walken.c` add the following @@ -712,20 +713,9 @@ help understand. In our case, that means we omit trees and blobs not directly referenced by `HEAD` or `HEAD`'s history, because we begin the walk with only `HEAD` in the `pending` list.) -First, we'll need to `#include "list-objects-filter-options.h"` and set up the -`struct list_objects_filter_options` at the top of the function. - ----- -static void walken_object_walk(struct rev_info *rev) -{ - struct list_objects_filter_options filter_options = { 0 }; - - ... ----- - For now, we are not going to track the omitted objects, so we'll replace those parameters with `NULL`. For the sake of simplicity, we'll add a simple -build-time branch to use our filter or not. Replace the line calling +build-time branch to use our filter or not. Preface the line calling `traverse_commit_list()` with the following, which will remind us which kind of walk we've just performed: @@ -733,19 +723,17 @@ walk we've just performed: if (0) { /* Unfiltered: */ trace_printf(_("Unfiltered object walk.\n")); - traverse_commit_list(rev, walken_show_commit, - walken_show_object, NULL); } else { trace_printf( _("Filtered object walk with filterspec 'tree:1'.\n")); - parse_list_objects_filter(&filter_options, "tree:1"); - - traverse_commit_list_filtered(&filter_options, rev, - walken_show_commit, walken_show_object, NULL, NULL); + CALLOC_ARRAY(rev->filter, 1); + parse_list_objects_filter(rev->filter, "tree:1"); } + traverse_commit_list(rev, walken_show_commit, + walken_show_object, NULL); ---- -`struct list_objects_filter_options` is usually built directly from a command +The `rev->filter` member is usually built directly from a command line argument, so the module provides an easy way to build one from a string. Even though we aren't taking user input right now, we can still build one with a hardcoded string using `parse_list_objects_filter()`. @@ -784,7 +772,7 @@ object: ---- ... - traverse_commit_list_filtered(&filter_options, rev, + traverse_commit_list_filtered(rev, walken_show_commit, walken_show_object, NULL, &omitted); ... From patchwork Wed Feb 23 17:55:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12757255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C57DEC4332F for ; Wed, 23 Feb 2022 17:55:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243603AbiBWRzz (ORCPT ); Wed, 23 Feb 2022 12:55:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243590AbiBWRzt (ORCPT ); Wed, 23 Feb 2022 12:55:49 -0500 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 520BCA190 for ; Wed, 23 Feb 2022 09:55:21 -0800 (PST) Received: by mail-wr1-x436.google.com with SMTP id s1so14518723wrg.10 for ; Wed, 23 Feb 2022 09:55:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=1GVaBXimqBTY5hmpJCclNff/xDZkhFYtw17lQu0BLKo=; b=DVYAuW6Zz8Rl6oHlquxD4ZWRVupEF53LplHhncDumh+BRCU0OrKOy2nW50RGvWRLjj +G8Flc+6yzvEZypqGhgCWsqOkNcW9+a458N8s+PNO1fP4dg1cVkGkxXKoXJ6MDlhGJIT Yn1ZV0FgtK8ZwM/DBKlfnGzNEnW/oR6dHRs/NK9W5pYo7jqhFW4vmJF8m5wHcMzLDkkA xeh0vz1ApWe/JSfNav0fhOnotS6J7FjOXGQ9gv7jFAWi2/yJVCfZSs77lop7HIAFcsDi zyiwqkhF7AFoVahmO02hdo9Vl3Vw+ACd23DQfInCipdvAoAeQPXrFUoh6ENwivUe8DeZ CVGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=1GVaBXimqBTY5hmpJCclNff/xDZkhFYtw17lQu0BLKo=; b=FnsIhCSlhJFnymvgJ8X7KZSl7AmiFrxxzdd0IunF9PU+VSyVFfi5XXqO5AJw+vTZY/ MFmY6jHtadPFmv1S6enup8k8LU0SrcohnxEj+feWf4ndC6C1kBSmckgS60pxgbdqi6Qa HLnGSzK5aKKK16FednhW+aSe3J+ksBOEDQqhJkSd/VUQx1iwaaO6dqQHdkjBWZFQ/Xf2 /fcofdcD+wkz+Wx/xLNVpf0B370YAxZqXQ85tHBQzs9f04tfQNM80W0AI3RGoNBSjF3/ MTXy1fKfmLcn/Zspsejvnm4dqw4htRMWXYvt8ZQk6Ou25cdUrsxZRsAYO+sDP0ZFVTxZ hMIg== X-Gm-Message-State: AOAM531a8QjIrVKH0gLvXT2V+nnS66REGuZ8F4jhfB6aW9Vbx/0QHJW8 voidayeRIktCdeLfgklLPSlHXDI9D7s= X-Google-Smtp-Source: ABdhPJxX363JoeRB21RxU+p1qvZl5PVGbapt/tZ7k1BHp0PCDRwJj/lAnOsqhCxG6f4FCjY3AaPV2g== X-Received: by 2002:a5d:4ecd:0:b0:1ea:910b:7641 with SMTP id s13-20020a5d4ecd000000b001ea910b7641mr561885wrv.103.1645638919741; Wed, 23 Feb 2022 09:55:19 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j27sm221559wrd.32.2022.02.23.09.55.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Feb 2022 09:55:19 -0800 (PST) Message-Id: <1476a9495c53a165e6971afe75205889524fe7ca.1645638911.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 23 Feb 2022 17:55:07 +0000 Subject: [PATCH 07/11] bundle: safely handle --objects option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Since 'git bundle' uses setup_revisions() to specify the object walk, some options do not make sense to include during the pack-objects child process. Further, these options are used for a call to traverse_commit_list() which would then require a callback which is currently NULL. By populating the callback we prevent a segfault in the case of adding the --objects flag. This is really a redundant statement because the bundles are constructing a pack-file containing all objects in the discovered commit range. Adding --objects to a 'git bundle' command might cause a slower command, but at least it will not have a hard failure when the user supplies this option. We can also disable walking trees and blobs in advance of this walk. Signed-off-by: Derrick Stolee --- bundle.c | 10 +++++++++- t/t6020-bundle-misc.sh | 12 ++++++++++++ 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/bundle.c b/bundle.c index a0bb687b0f4..dc56db9a50a 100644 --- a/bundle.c +++ b/bundle.c @@ -451,6 +451,12 @@ struct bundle_prerequisites_info { int fd; }; + +static void ignore_object(struct object *obj, const char *v, void *data) +{ + /* Do nothing. */ +} + static void write_bundle_prerequisites(struct commit *commit, void *data) { struct bundle_prerequisites_info *bpi = data; @@ -544,7 +550,9 @@ int create_bundle(struct repository *r, const char *path, die("revision walk setup failed"); bpi.fd = bundle_fd; bpi.pending = &revs_copy.pending; - traverse_commit_list(&revs, write_bundle_prerequisites, NULL, &bpi); + + revs.blob_objects = revs.tree_objects = 0; + traverse_commit_list(&revs, write_bundle_prerequisites, ignore_object, &bpi); object_array_remove_duplicates(&revs_copy.pending); /* write bundle refs */ diff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh index b13e8a52a93..6522401617d 100755 --- a/t/t6020-bundle-misc.sh +++ b/t/t6020-bundle-misc.sh @@ -475,4 +475,16 @@ test_expect_success 'clone from bundle' ' test_cmp expect actual ' +test_expect_success 'unfiltered bundle with --objects' ' + git bundle create all-objects.bdl \ + --all --objects && + git bundle create all.bdl \ + --all && + + # Compare the headers of these files. + head -11 all.bdl >expect && + head -11 all-objects.bdl >actual && + test_cmp expect actual +' + test_done From patchwork Wed Feb 23 17:55:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12757254 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59229C433EF for ; Wed, 23 Feb 2022 17:55:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243601AbiBWRzy (ORCPT ); Wed, 23 Feb 2022 12:55:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243582AbiBWRzu (ORCPT ); Wed, 23 Feb 2022 12:55:50 -0500 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A03AAE5D for ; Wed, 23 Feb 2022 09:55:22 -0800 (PST) Received: by mail-wm1-x336.google.com with SMTP id y5so3789748wmi.0 for ; Wed, 23 Feb 2022 09:55:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=2uvoAv8r7xSnCh6eMu60yhWRWmcE78KKGIGJIpmlJQA=; b=UQvH4+otMB6/0OegciWEapNksKXhMYY4G2FoDwsi0bpE5CjDTXl2SL+rmcg8iM5O6S JGK12UU1MfmXHmAxAGbu/Qxya1atKan7m5edIYRyXJal7b1quhAbbgKT175zAE5iH7kC QGXj8ulUdoV06sIM3wrtdM8d05uaqLgvVA4nOiEQ0v/r0rTJd1CkNVwWxqvjZCDUnSYD q61e+En69OVdrkUGVsoxiT3UdsC6FotReRA6dkO3C5AlUNCRCa+1kIU7BZJxH9R6E5K+ PLDnsEtFVKdKweiVhhc/IB3g5/RVyMfoZ8SUHYJ3vAsa43cM2QUX32vZTy1AnT/wkmFZ AM8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=2uvoAv8r7xSnCh6eMu60yhWRWmcE78KKGIGJIpmlJQA=; b=IUQMp9YmKKoHw4uwCHfgDFrqhHUrIj1H0N0sWZleL+sQuOVPdD1WEcpFF8iTzz5DW9 OhkKYsj9LCu2xVzLEpSbwoebzMcIQtiN8c4J12y+eGQH1708Rk2/6C5WlzUVw5wEleTa VmDZH72gJsLvJcRXVnhXLfe2iROnab9JI/56Utf46yycDEOlMH/p4/Ss3RiPplS9s6y8 Zt7eh5ZoroyeBSgY0BUDsMId/9p/VMmW9zMl59inZ3+baq4hUq8tfRp5l9jcCh1S4+tD oUIaWNeM8MnTjDLEygE67Bh170uGqpP/xOLLNydH21G0qhIXud8aimDXdUu5kkSRKy+1 n/DQ== X-Gm-Message-State: AOAM53233F8I8gcenq2KUOWdTKUHwS+DQklV9sTbJsha3XF/yg5TohgJ zcDR1ro7w68BF19WJKRoLelQHS7+VAA= X-Google-Smtp-Source: ABdhPJy7jdpw7e471nHarHKaKqJSswqK3Kb1xvI0oFIaOa/ykHjryCUgTUKGgEqLZbw6/Sf6tu77vA== X-Received: by 2002:a7b:cb83:0:b0:37e:bc50:3c6b with SMTP id m3-20020a7bcb83000000b0037ebc503c6bmr8394754wmi.67.1645638920636; Wed, 23 Feb 2022 09:55:20 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e3sm174450wrr.94.2022.02.23.09.55.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Feb 2022 09:55:20 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 23 Feb 2022 17:55:08 +0000 Subject: [PATCH 08/11] bundle: parse filter capability Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The v3 bundle format has capabilities, allowing newer versions of Git to create bundles with newer features. Older versions that do not understand these new capabilities will fail with a helpful warning. Create a new capability allowing Git to understand that the contained pack-file is filtered according to some object filter. Typically, this filter will be "blob:none" for a blobless partial clone. This change teaches Git to parse this capability, place its value in the bundle header, and demonstrate this understanding by adding a message to 'git bundle verify'. Signed-off-by: Derrick Stolee --- bundle.c | 17 ++++++++++++++++- bundle.h | 3 +++ list-objects-filter-options.c | 2 +- list-objects-filter-options.h | 5 +++++ 4 files changed, 25 insertions(+), 2 deletions(-) diff --git a/bundle.c b/bundle.c index dc56db9a50a..2afced4d991 100644 --- a/bundle.c +++ b/bundle.c @@ -11,7 +11,7 @@ #include "run-command.h" #include "refs.h" #include "strvec.h" - +#include "list-objects-filter-options.h" static const char v2_bundle_signature[] = "# v2 git bundle\n"; static const char v3_bundle_signature[] = "# v3 git bundle\n"; @@ -33,6 +33,8 @@ void bundle_header_release(struct bundle_header *header) { string_list_clear(&header->prerequisites, 1); string_list_clear(&header->references, 1); + list_objects_filter_release(header->filter); + free(header->filter); } static int parse_capability(struct bundle_header *header, const char *capability) @@ -45,6 +47,11 @@ static int parse_capability(struct bundle_header *header, const char *capability header->hash_algo = &hash_algos[algo]; return 0; } + if (skip_prefix(capability, "filter=", &arg)) { + CALLOC_ARRAY(header->filter, 1); + parse_list_objects_filter(header->filter, arg); + return 0; + } return error(_("unknown capability '%s'"), capability); } @@ -220,6 +227,8 @@ int verify_bundle(struct repository *r, req_nr = revs.pending.nr; setup_revisions(2, argv, &revs, NULL); + revs.filter = header->filter; + if (prepare_revision_walk(&revs)) die(_("revision walk setup failed")); @@ -259,6 +268,12 @@ int verify_bundle(struct repository *r, r->nr), r->nr); list_refs(r, 0, NULL); + + if (header->filter) { + printf_ln("The bundle uses this filter: %s", + list_objects_filter_spec(header->filter)); + } + r = &header->prerequisites; if (!r->nr) { printf_ln(_("The bundle records a complete history.")); diff --git a/bundle.h b/bundle.h index 06009fe6b1f..eb026153d56 100644 --- a/bundle.h +++ b/bundle.h @@ -5,11 +5,14 @@ #include "cache.h" #include "string-list.h" +struct list_objects_filter_options; + struct bundle_header { unsigned version; struct string_list prerequisites; struct string_list references; const struct git_hash_algo *hash_algo; + struct list_objects_filter_options *filter; }; #define BUNDLE_HEADER_INIT \ diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index fd8d59f653a..b9d10770e4f 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -55,7 +55,7 @@ const char *list_object_filter_config_name(enum list_objects_filter_choice c) * expand_list_objects_filter_spec() first). We also "intern" the arg for the * convenience of the current command. */ -static int gently_parse_list_objects_filter( +int gently_parse_list_objects_filter( struct list_objects_filter_options *filter_options, const char *arg, struct strbuf *errbuf) diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h index da5b6737e27..347a99c28cf 100644 --- a/list-objects-filter-options.h +++ b/list-objects-filter-options.h @@ -72,6 +72,11 @@ struct list_objects_filter_options { /* Normalized command line arguments */ #define CL_ARG__FILTER "filter" +int gently_parse_list_objects_filter( + struct list_objects_filter_options *filter_options, + const char *arg, + struct strbuf *errbuf); + void list_objects_filter_die_if_populated( struct list_objects_filter_options *filter_options); From patchwork Wed Feb 23 17:55:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12757256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67E5EC433F5 for ; Wed, 23 Feb 2022 17:55:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243605AbiBWRz6 (ORCPT ); Wed, 23 Feb 2022 12:55:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49158 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243585AbiBWRzv (ORCPT ); Wed, 23 Feb 2022 12:55:51 -0500 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0660762F7 for ; Wed, 23 Feb 2022 09:55:23 -0800 (PST) Received: by mail-wm1-x336.google.com with SMTP id n25-20020a05600c3b9900b00380f41e51e6so1863249wms.2 for ; Wed, 23 Feb 2022 09:55:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=banR2/YPRedUlMEFqceTdv5AzXLOH8Pv1nIbKygHUss=; b=YkOU9nof693tJHtqH8BEYrEOacDjNnsMVa6/YaRMsfPZMwYd05hWbChGltvoP3RSvC 3qA4n5RCXOKeBlbLZVcjKbca4bSWURhROaokW/jUFCbF1A+kPruI69l/45lJtE1DYC3E ddBAFVVfcOXjUKbY/1U/KmTeEnWDOUYLzdoNaWbE4gXs2FTVW8YVChciYMBxj6Ma/g3w 69WJGVhJGs3qe2Zo2ll9s7mT21PmX6zPOqxJtpyE6dkmNJD0cn2gsz8RUiDCLMyqIw+J ACINqDgNK9ZlY9YK7xupbL0VYFxUcjqGXdt9U8tWt9B3I9pghbEcrhocliy6wJ6F6iE+ /vWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=banR2/YPRedUlMEFqceTdv5AzXLOH8Pv1nIbKygHUss=; b=u6Ev/cjhxGFmu264Li8d3RrleIhFESaLcHN+O4XRqqlfgxDtdZfsAK2kvUkU9QInU4 il/xLdKq4Ui5pkMljAjOhX0OVBr+pD1yydXrGQ6jtCmJQOkPMEdvVB+/kv45yeYK58s5 jejwp7RFvQ/Cg6t+CTeY4doFArq8SxuPtHMgejG5vY5pYVIZsrIhDZ9frROUpDujSojk 33zPb4Pfuu4AiRVrXE6Wc51BoAa1LoV+8J//UO0pD1qYGj/yLFBI4oVJjLVqlaoxicGb Upx5e0xcXKghogVJR1SgTJg82IA6xRqVps2/VbluJkNQjzonQOydT58dvE6GPVRghFx+ 93pQ== X-Gm-Message-State: AOAM530Sc3d1GHj/7LV4/mNEXOIVKqB6De79CiiKj5XYKNU/aCnFMTrF zsi6tAKt4PIWgK5OPrb645j004HW3DQ= X-Google-Smtp-Source: ABdhPJwuNL3Dckv4GjzOQGrUdYwHcI0EZRp8xnjKTA1O2DoehXN14rrSWcUVH1T2o+YfVyPG1li8dg== X-Received: by 2002:a1c:4d09:0:b0:380:e340:bfba with SMTP id o9-20020a1c4d09000000b00380e340bfbamr6262039wmh.80.1645638921500; Wed, 23 Feb 2022 09:55:21 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o10sm255825wrc.98.2022.02.23.09.55.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Feb 2022 09:55:21 -0800 (PST) Message-Id: <22c4fe9d4bc84bb7c9e6cd3c837a1d18647ad6ee.1645638911.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 23 Feb 2022 17:55:09 +0000 Subject: [PATCH 09/11] rev-list: move --filter parsing into revision.c Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Now that 'struct rev_info' has a 'filter' member and most consumers of object filtering are using that member instead of an external struct, move the parsing of the '--filter' option out of builtin/rev-list.c and into revision.c. This use within handle_revision_pseudo_opt() allows us to find the option within setup_revisions() if the arguments are passed directly. In the case of a command such as 'git blame', the arguments are first scanned and checked with parse_revision_opt(), which complains about the option, so 'git blame --filter=blob:none ' does not become valid with this change. Some commands, such as 'git diff' gain this option without having it make an effect. And 'git diff --objects' was already possible, but does not actually make sense in that builtin. The key addition that is coming is 'git bundle create --filter=' so we can create bundles containing promisor packs. More work is required to make them fully functional, but that will follow. Signed-off-by: Derrick Stolee --- builtin/rev-list.c | 15 --------------- revision.c | 11 +++++++++++ 2 files changed, 11 insertions(+), 15 deletions(-) diff --git a/builtin/rev-list.c b/builtin/rev-list.c index 3ab727817fd..640828149c5 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -591,21 +591,6 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) show_progress = arg; continue; } - - if (skip_prefix(arg, ("--" CL_ARG__FILTER "="), &arg)) { - if (!revs.filter) - CALLOC_ARRAY(revs.filter, 1); - parse_list_objects_filter(revs.filter, arg); - if (revs.filter->choice && !revs.blob_objects) - die(_("object filtering requires --objects")); - continue; - } - if (!strcmp(arg, ("--no-" CL_ARG__FILTER))) { - if (!revs.filter) - CALLOC_ARRAY(revs.filter, 1); - list_objects_filter_set_no_filter(revs.filter); - continue; - } if (!strcmp(arg, "--filter-provided-objects")) { filter_provided_objects = 1; continue; diff --git a/revision.c b/revision.c index ad4286fbdde..1d612c1c102 100644 --- a/revision.c +++ b/revision.c @@ -32,6 +32,7 @@ #include "utf8.h" #include "bloom.h" #include "json-writer.h" +#include "list-objects-filter-options.h" volatile show_early_output_fn_t show_early_output; @@ -2669,6 +2670,14 @@ static int handle_revision_pseudo_opt(struct rev_info *revs, revs->no_walk = 0; } else if (!strcmp(arg, "--single-worktree")) { revs->single_worktree = 1; + } else if (skip_prefix(arg, ("--" CL_ARG__FILTER "="), &arg)) { + if (!revs->filter) + CALLOC_ARRAY(revs->filter, 1); + parse_list_objects_filter(revs->filter, arg); + } else if (!strcmp(arg, ("--no-" CL_ARG__FILTER))) { + if (!revs->filter) + CALLOC_ARRAY(revs->filter, 1); + list_objects_filter_set_no_filter(revs->filter); } else { return 0; } @@ -2872,6 +2881,8 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s die("cannot combine --walk-reflogs with history-limiting options"); if (revs->rewrite_parents && revs->children.name) die(_("options '%s' and '%s' cannot be used together"), "--parents", "--children"); + if (revs->filter && revs->filter->choice && !revs->blob_objects) + die(_("object filtering requires --objects")); /* * Limitations on the graph functionality From patchwork Wed Feb 23 17:55:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12757257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 144B1C433F5 for ; Wed, 23 Feb 2022 17:55:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243609AbiBWR4E (ORCPT ); Wed, 23 Feb 2022 12:56:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49208 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243593AbiBWRzw (ORCPT ); Wed, 23 Feb 2022 12:55:52 -0500 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F86E2736 for ; Wed, 23 Feb 2022 09:55:24 -0800 (PST) Received: by mail-wr1-x42a.google.com with SMTP id j22so14838840wrb.13 for ; Wed, 23 Feb 2022 09:55:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=5mG5vvumPmhw2DkdF6cHCjGmFvwG0XBh/NgIZByVDf4=; b=YgQNa1lAUn6PFhNHjMjhqdID/xq2607oxQwEV/8GduSjrt7t8GLnp3ymYWf92JzBqZ LdD6Ps0tB3Z2Jm6xQhsHZ4Dkuacy0ilosnkDnvYUDQFbIQx4w+wWEq/HV04I5c1aFiK2 kucpKQou8cWmbjJo+wK4FlSwFx7B8J2hht2q1P97yXOmuIk6MeAYegB6z7qzoM64bLCW XTltV8GHSM7K0/hgLCi/zNZGKGkSCleXmSLWZD8XFlQEdTMb5Ubh/+Mpy3IiFK5ueARs hh27LFxQLo5G8YHc0Ci5I1Tm2xkT9obg9rVaqO1A+pxbirP61tD/NhmV6nFqz1W24yhn LGWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=5mG5vvumPmhw2DkdF6cHCjGmFvwG0XBh/NgIZByVDf4=; b=UEvrH+4uw+Yw4EUxVR5a6SLMvWkmVigDMQRnzSbrKYZLWythkQdGLi0U0Qru/cgx5U EQDP2dbpXU4YMaerOioqXzjBuI719aVi0J3bAvC9sGbe4NloVKEd1Ued27vHR0s5D6id nCwOr3sa/ERKCTymV52cDUOTzovr3voPSOYo1jPsuCL48yUY/keZepsIO7MK7sC5J6m8 a7L1/ui4mE6aWOWt+Oxaf3/ARZkjq9kW6k+8EAJiE1OqG88JXwzc9WciZdplctLPgOEP RulsMq887B5FyuZ5N6J/L6n9Z9jrsajRkOGuAPQogqtscl0NNXxLFeNzlJF4qcLIDiX3 GfSw== X-Gm-Message-State: AOAM532XTz4h5HXK6ErpW5drjdyaELpratn00PqUBvKDcOMA4AZ6l3QA BgtE7CEZVweppyUpJfnMVGSSCeEilcE= X-Google-Smtp-Source: ABdhPJz9I+IdvaR1fVWxULjZqTH+roo2tEmX//sv5Y7QmUSUqAnbUVjRd0qDnJfEX/AuEYg8pUJz4A== X-Received: by 2002:a5d:550d:0:b0:1ed:c155:6c2a with SMTP id b13-20020a5d550d000000b001edc1556c2amr581475wrv.470.1645638922473; Wed, 23 Feb 2022 09:55:22 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u15sm223825wrn.48.2022.02.23.09.55.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Feb 2022 09:55:21 -0800 (PST) Message-Id: <5393e74708dfd38e5596d9e877a491e6ed8dda24.1645638911.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 23 Feb 2022 17:55:10 +0000 Subject: [PATCH 10/11] bundle: create filtered bundles Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee A previous change allowed Git to parse bundles with the 'filter' capability. Now, teach Git to create bundles with this option. Some rearranging of code is required to get the option parsing in the correct spot. There are now two reasons why we might need capabilities (a new hash algorithm or an object filter) so that is pulled out into a place where we can check both at the same time. The --filter option is parsed as part of setup_revisions(), but it expected the --objects flag, too. That flag is somewhat implied by 'git bundle' because it creates a pack-file walking objects, but there is also a walk that walks the revision range expecting only commits. Make this parsing work by setting 'revs.tree_objects' and 'revs.blob_objects' before the call to setup_revisions(). Signed-off-by: Derrick Stolee --- bundle.c | 56 +++++++++++++++++++++++++++++++++--------- t/t6020-bundle-misc.sh | 30 ++++++++++++++++++++++ 2 files changed, 75 insertions(+), 11 deletions(-) diff --git a/bundle.c b/bundle.c index 2afced4d991..e284ef63062 100644 --- a/bundle.c +++ b/bundle.c @@ -334,6 +334,9 @@ static int write_pack_data(int bundle_fd, struct rev_info *revs, struct strvec * "--stdout", "--thin", "--delta-base-offset", NULL); strvec_pushv(&pack_objects.args, pack_options->v); + if (revs->filter) + strvec_pushf(&pack_objects.args, "--filter=%s", + list_objects_filter_spec(revs->filter)); pack_objects.in = -1; pack_objects.out = bundle_fd; pack_objects.git_cmd = 1; @@ -507,10 +510,38 @@ int create_bundle(struct repository *r, const char *path, int bundle_to_stdout; int ref_count = 0; struct rev_info revs, revs_copy; - int min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3; + int min_version = 2; struct bundle_prerequisites_info bpi; int i; + /* init revs to list objects for pack-objects later */ + save_commit_buffer = 0; + repo_init_revisions(r, &revs, NULL); + + /* + * Pre-initialize the '--objects' flag so we can parse a + * --filter option successfully. + */ + revs.tree_objects = revs.blob_objects = 1; + + argc = setup_revisions(argc, argv, &revs, NULL); + + /* + * Reasons to require version 3: + * + * 1. @object-format is required because our hash algorithm is not + * SHA1. + * 2. @filter is required because we parsed an object filter. + */ + if (the_hash_algo != &hash_algos[GIT_HASH_SHA1] || + revs.filter) + min_version = 3; + + if (argc > 1) { + error(_("unrecognized argument: %s"), argv[1]); + goto err; + } + bundle_to_stdout = !strcmp(path, "-"); if (bundle_to_stdout) bundle_fd = 1; @@ -533,17 +564,14 @@ int create_bundle(struct repository *r, const char *path, write_or_die(bundle_fd, capability, strlen(capability)); write_or_die(bundle_fd, the_hash_algo->name, strlen(the_hash_algo->name)); write_or_die(bundle_fd, "\n", 1); - } - - /* init revs to list objects for pack-objects later */ - save_commit_buffer = 0; - repo_init_revisions(r, &revs, NULL); - argc = setup_revisions(argc, argv, &revs, NULL); - - if (argc > 1) { - error(_("unrecognized argument: %s"), argv[1]); - goto err; + if (revs.filter) { + const char *value = expand_list_objects_filter_spec(revs.filter); + capability = "@filter="; + write_or_die(bundle_fd, capability, strlen(capability)); + write_or_die(bundle_fd, value, strlen(value)); + write_or_die(bundle_fd, "\n", 1); + } } /* save revs.pending in revs_copy for later use */ @@ -566,6 +594,12 @@ int create_bundle(struct repository *r, const char *path, bpi.fd = bundle_fd; bpi.pending = &revs_copy.pending; + /* + * Nullify the filter here, and any object walking. We only care + * about commits and tags here. The revs_copy has the right + * instances of these values. + */ + revs.filter = NULL; revs.blob_objects = revs.tree_objects = 0; traverse_commit_list(&revs, write_bundle_prerequisites, ignore_object, &bpi); object_array_remove_duplicates(&revs_copy.pending); diff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh index 6522401617d..39cfefafb65 100755 --- a/t/t6020-bundle-misc.sh +++ b/t/t6020-bundle-misc.sh @@ -487,4 +487,34 @@ test_expect_success 'unfiltered bundle with --objects' ' test_cmp expect actual ' +for filter in "blob:none" "tree:0" "tree:1" "blob:limit=100" +do + test_expect_success 'filtered bundle: $filter' ' + test_when_finished rm -rf .git/objects/pack && + git bundle create partial.bdl \ + --all \ + --filter=$filter && + + git bundle verify partial.bdl >unfiltered && + make_user_friendly_and_stable_output actual && + + cat >expect <<-EOF && + The bundle contains these 10 refs: + refs/heads/main + refs/heads/release + refs/heads/topic/1 + refs/heads/topic/2 + refs/pull/1/head + refs/pull/2/head + refs/tags/v1 + refs/tags/v2 + refs/tags/v3 + HEAD + The bundle uses this filter: $filter + The bundle records a complete history. + EOF + test_cmp expect actual + ' +done + test_done From patchwork Wed Feb 23 17:55:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12757258 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACC44C433F5 for ; Wed, 23 Feb 2022 17:55:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243610AbiBWR4M (ORCPT ); Wed, 23 Feb 2022 12:56:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49212 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243594AbiBWRzw (ORCPT ); Wed, 23 Feb 2022 12:55:52 -0500 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD9DB65EE for ; Wed, 23 Feb 2022 09:55:24 -0800 (PST) Received: by mail-wr1-x436.google.com with SMTP id o4so4155450wrf.3 for ; Wed, 23 Feb 2022 09:55:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=E/dL4dbXmnevF9IPOBnoODuzwSNqZWPE+6v50BBklS0=; b=K8NOgDuc2ZvOFoTU2vAhyG3KgOjPewKePXKmLNjzMDiPaS4V44U5HgrMkb3OkhIjZW 7PWPEp1LcbI11cnAVCNJ7JHU9pxEawS3g8FAPQR3c/FaXA7Js7vpIDv1HB4MFupQLono CWys2yKvU1mOUK3cv5JERIN73DQqJ8jN2K5/7rZlIweb2oVp/Isbkkb+K2NkANPkArNy yQc8t4v0ioyCO9oUURaCqWiZCyzvxoKZi8LQJ3U/Q03lW3Zas+R1/xm8Ya+MyIk6dKGi drAOljU+FGnfgAPMc4mAcIE6SeKWzyzFPU3Ss0T2uRJVFMUlIUBl0Pe1Sgte/CWEqOXu knpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=E/dL4dbXmnevF9IPOBnoODuzwSNqZWPE+6v50BBklS0=; b=CE3a9yrydNTUm6S5KaYdDR2vVMTOTfnYocdNKAceD2BcQLNsmCpBx0/1RbV+zx2ivv 0m9LP8zGLbN+HCjpiqW2oxJJETXYnEW+jYLx9MHjGHthdysP1Ay0hJQGeU4wBMMzOrQf oZOrtTwAynNo3XgbVs5QePsnXpMXMGrVP2InNwPMXTGL8s0B8sO5vuIVsJifg1ZaJtk9 XQZHM3uJ9UTkC4t85BwLnz3DRXmlUgR5Np4nxfmCZuqEL3FQEieutY72GeWCP1IMBVHL hTV07+LLReG4F2YGCP0uh15RIDag3gIg7DEhtCdjhwMEEF6urf6Hj/Lf395B8r09S4dw txpw== X-Gm-Message-State: AOAM531Oo7vN01v/uRbthUkPr66LRxNg1QPns0JHMqWZek13dlVSJSHU E0CAf1yLf3kKecqQNAJIozAJMi+OOEU= X-Google-Smtp-Source: ABdhPJwoFqO9zERov9rXVZQQFjmJbRmtWB+XZVtPRjdirMmaVl1+IXGIbYum46yQQC97tfyFDgQgUw== X-Received: by 2002:adf:f303:0:b0:1e7:aeab:ac6a with SMTP id i3-20020adff303000000b001e7aeabac6amr604105wro.40.1645638923262; Wed, 23 Feb 2022 09:55:23 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e20sm246424wre.90.2022.02.23.09.55.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Feb 2022 09:55:22 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 23 Feb 2022 17:55:11 +0000 Subject: [PATCH 11/11] bundle: unbundle promisor packs Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com, zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee In order to have a valid pack-file after unbundling a bundle that has the 'filter' capability, we need to generate a .promisor file. The bundle does not promise _where_ the objects can be found, but we can expect that these bundles will be unbundled in repositories with appropriate promisor remotes that can find those missing objects. Use the 'git index-pack --promisor=' option to create this .promisor file. Add "from-bundle" as the message to help anyone diagnose issues with these promisor packs. Signed-off-by: Derrick Stolee --- bundle.c | 4 ++++ t/t6020-bundle-misc.sh | 8 +++++++- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/bundle.c b/bundle.c index e284ef63062..3d97de40ef0 100644 --- a/bundle.c +++ b/bundle.c @@ -631,6 +631,10 @@ int unbundle(struct repository *r, struct bundle_header *header, struct child_process ip = CHILD_PROCESS_INIT; strvec_pushl(&ip.args, "index-pack", "--fix-thin", "--stdin", NULL); + /* If there is a filter, then we need to create the promisor pack. */ + if (header->filter) + strvec_push(&ip.args, "--promisor=from-bundle"); + if (extra_index_pack_args) { strvec_pushv(&ip.args, extra_index_pack_args->v); strvec_clear(extra_index_pack_args); diff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh index 39cfefafb65..344af34db1e 100755 --- a/t/t6020-bundle-misc.sh +++ b/t/t6020-bundle-misc.sh @@ -513,7 +513,13 @@ do The bundle uses this filter: $filter The bundle records a complete history. EOF - test_cmp expect actual + test_cmp expect actual && + + # This creates the first pack-file in the + # .git/objects/pack directory. Look for a .promisor. + git bundle unbundle partial.bdl && + ls .git/objects/pack/pack-*.promisor >promisor && + test_line_count = 1 promisor ' done