From patchwork Mon Apr 12 13:37:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12198075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B5C0C433ED for ; Mon, 12 Apr 2021 13:37:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 35B5A61350 for ; Mon, 12 Apr 2021 13:37:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241897AbhDLNhm (ORCPT ); Mon, 12 Apr 2021 09:37:42 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:51499 "EHLO wout4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239686AbhDLNhl (ORCPT ); Mon, 12 Apr 2021 09:37:41 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 0106316D9; Mon, 12 Apr 2021 09:37:22 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 12 Apr 2021 09:37:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=wJt1DRqH9VzoI6RnSk3LY07ur99 w3JGZaC/dRpYS1sQ=; b=nVGrt1PWgbVvCaUvM4qGb1Z91K69hkZ9EWKJ80OtEnV 0CAzdBPdyJYtrVr3NkkarE2YdN5JEnZCHEHhsfYxqjgJy+LXd6C1yGeKwjzmM6sO 1ns4WAItv3wds+hzJxgiHDvrNvkqPfTZOverg8xzLtY8dFbBrhE53sbeKMEwASJB q4SpZgFUGvMvhgoEEmAVOxkMPW8BKM4HTiUPXd90KqVY6oadQeq+mbKgSRCT+OMh mofQmoioQJgx6QzM/zb8s0iAzfQG1XetEwrG/wRNRbPdSZZVKV+p+YofXyNvj+ne zA/MikNnyZgsUj8ZjnXxFlFUNnAuN34HpBU4aJ6WDVw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=wJt1DR qH9VzoI6RnSk3LY07ur99w3JGZaC/dRpYS1sQ=; b=Z7UYrlZpCOQSFvV8vP8xPx o3piZed3Qhz1uCNZRt9YWGScSfV8RGx9zqHqpFD/t6UAyOO/mysmf+VIM47yhMwg Fyn3iMGhifyJwCPi9Vn0fGKB4cpRjXTcHyZS2o/WxX62TQkZD6SDdcWRZSXCTg+f 6QrzuQNZLPyB/G3ZMnvFlrdQ16ltMrA+CSikxat63jU/z8p7UnjtvVJtOD1etvEN n++dzK5FI0g/JQ0K9ak6UR5CJCXqG/wyqZ/mnjKj8Ufmy/9np7d85uuFePNnmrVk V0U8zVWW8wUiZ5nphKEjzGc5sJ69L1DUNOYrGbyxf7UdTHPWN6RHzJzZ4RIgJnDw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekjedgieejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeekrdehhedrfeehrdeftdenucevlhhushhtvghrufhiiigvpedtnecurfgrrh grmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-078-055-035-030.78.55.pool.telefonica.de [78.55.35.30]) by mail.messagingengine.com (Postfix) with ESMTPA id DFCBB24005A; Mon, 12 Apr 2021 09:37:21 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 59de14f7 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 12 Apr 2021 13:37:21 +0000 (UTC) Date: Mon, 12 Apr 2021 15:37:20 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley Subject: [PATCH v4 1/8] uploadpack.txt: document implication of `uploadpackfilter.allow` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When `uploadpackfilter.allow` is set to `true`, it means that filters are enabled by default except in the case where a filter is explicitly disabled via `uploadpackilter..allow`. This option will not only enable the currently supported set of filters, but also any filters which get added in the future. As such, an admin which wants to have tight control over which filters are allowed and which aren't probably shouldn't ever set `uploadpackfilter.allow=true`. Amend the documentation to make the ramifications more explicit so that admins are aware of this. Signed-off-by: Patrick Steinhardt --- Documentation/config/uploadpack.txt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Documentation/config/uploadpack.txt b/Documentation/config/uploadpack.txt index b0d761282c..6729a072ea 100644 --- a/Documentation/config/uploadpack.txt +++ b/Documentation/config/uploadpack.txt @@ -59,7 +59,8 @@ uploadpack.allowFilter:: uploadpackfilter.allow:: Provides a default value for unspecified object filters (see: the - below configuration variable). + below configuration variable). If set to `true`, this will also + enable all filters which get added in the future. Defaults to `true`. uploadpackfilter..allow:: From patchwork Mon Apr 12 13:37:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12198079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9B67C433B4 for ; Mon, 12 Apr 2021 13:37:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9F33A6128B for ; Mon, 12 Apr 2021 13:37:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241917AbhDLNhr (ORCPT ); Mon, 12 Apr 2021 09:37:47 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:47185 "EHLO wout4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238958AbhDLNhr (ORCPT ); Mon, 12 Apr 2021 09:37:47 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 1B2AD16C4; Mon, 12 Apr 2021 09:37:29 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 12 Apr 2021 09:37:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=VFpuqbz3Zw59ClajNNjnme+eE/S U2XVEgl2fT1fpl1Y=; b=r+ad1dvzKJtARd40kbo3hzbj98lUWrwSRLcgsYt8+BM qMWHqCKcS/U1Cyi5zuCpSvvrou/bWQqfmBmIUMffDJKgRnMsMrBOx53hewDeC4/u 8mwDEj3ffQvqLi4YUHTd7bwaP+0bDtVSUqj9Id18ZXaS7OcPVLqVa4uz7G57xH3X TdPKoPA0+fNxUcMdUgV/Qt3kIHc9I6wpA0R+qY5mXQT61azBinJx5p6lyHaKQspn tlhmx22rZ3+N5KrMqMHpahP9lzppVIOEO4odl4PP+39Tr5suYTuhHmN0YxZzRuD+ gaJXHYpywVB/ucL2fcIWtVaFC9f/ZXEDAk0D7vejiCQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=VFpuqb z3Zw59ClajNNjnme+eE/SU2XVEgl2fT1fpl1Y=; b=LBd/UHyXyFh1DXC34lxfZo BjLCLxZKPg2p2QW0tNJodpiXsIgaLevGI9U3zZKSauxBbH65/SJEUAErmUYGtW4c oU6iKxET2vGssjPypmx9oY/Auub3/0/v/kzSsdmwOeCxm9lHpsBkE7ONtZ54Nxpw WM4T7TN0FATMaGcFPR5Zc3AfvSbNi/IdD0ySvlYR1EnNOWkY3/QG+y40ZdIdahfK 9T/AOyMSHiM7df0fcfC/hZ4mi2rTbB2doL7ni6b1/QpJUTNVtKeq2Jl3KTB7Ath7 L7cjIQ7ZbmLyXwr9VzPaCmDjCty/I40SD91i6COVSCcrk0deRjz/VKzQByz4YIAg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekjedgieejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeekrdehhedrfeehrdeftdenucevlhhushhtvghrufhiiigvpedunecurfgrrh grmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-078-055-035-030.78.55.pool.telefonica.de [78.55.35.30]) by mail.messagingengine.com (Postfix) with ESMTPA id D5477240054; Mon, 12 Apr 2021 09:37:27 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id b2fc9031 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 12 Apr 2021 13:37:27 +0000 (UTC) Date: Mon, 12 Apr 2021 15:37:26 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley Subject: [PATCH v4 2/8] revision: mark commit parents as NOT_USER_GIVEN Message-ID: <46c1952405173eca40706ed7be3b939b210ad2d5.1618234575.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The NOT_USER_GIVEN flag of an object marks whether a flag was explicitly provided by the user or not. The most important use case for this is when filtering objects: only objects that were not explicitly requested will get filtered. The flag is currently only set for blobs and trees, which has been fine given that there are no filters for tags or commits currently. We're about to extend filtering capabilities to add object type filter though, which requires us to set up the NOT_USER_GIVEN flag correctly -- if it's not set, the object wouldn't get filtered at all. Mark unseen commit parents as NOT_USER_GIVEN when processing parents. Like this, explicitly provided parents stay user-given and thus unfiltered, while parents which get loaded as part of the graph walk can be filtered. This commit shouldn't have any user-visible impact yet as there is no logic to filter commits yet. Signed-off-by: Patrick Steinhardt --- revision.c | 4 ++-- revision.h | 3 --- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/revision.c b/revision.c index 553c0faa9b..fd34c75e23 100644 --- a/revision.c +++ b/revision.c @@ -1123,7 +1123,7 @@ static int process_parents(struct rev_info *revs, struct commit *commit, mark_parents_uninteresting(p); if (p->object.flags & SEEN) continue; - p->object.flags |= SEEN; + p->object.flags |= (SEEN | NOT_USER_GIVEN); if (list) commit_list_insert_by_date(p, list); if (queue) @@ -1165,7 +1165,7 @@ static int process_parents(struct rev_info *revs, struct commit *commit, } p->object.flags |= left_flag; if (!(p->object.flags & SEEN)) { - p->object.flags |= SEEN; + p->object.flags |= (SEEN | NOT_USER_GIVEN); if (list) commit_list_insert_by_date(p, list); if (queue) diff --git a/revision.h b/revision.h index a24f72dcd1..93aa012f51 100644 --- a/revision.h +++ b/revision.h @@ -44,9 +44,6 @@ /* * Indicates object was reached by traversal. i.e. not given by user on * command-line or stdin. - * NEEDSWORK: NOT_USER_GIVEN doesn't apply to commits because we only support - * filtering trees and blobs, but it may be useful to support filtering commits - * in the future. */ #define NOT_USER_GIVEN (1u<<25) #define TRACK_LINEAR (1u<<26) From patchwork Mon Apr 12 13:37:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12198081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D363C433ED for ; Mon, 12 Apr 2021 13:37:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DDA096128C for ; Mon, 12 Apr 2021 13:37:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241962AbhDLNhz (ORCPT ); Mon, 12 Apr 2021 09:37:55 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:50367 "EHLO wout4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241957AbhDLNhw (ORCPT ); Mon, 12 Apr 2021 09:37:52 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id ED6531681; Mon, 12 Apr 2021 09:37:33 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 12 Apr 2021 09:37:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=NylQZS62uBrE3ZBQTRLMOEeAaC7 haJOlaQ328Y77S2w=; b=CJhwHFw6Skmqvj9zv09/RdGt/4BFs9YZwKr8tZ3rAGg FJkpeiE315U6Gjuk/tUcYXiChsCJdMuNGTn0mfUXO9vcSB6X3EpNplIGJMFEXr4j zoljdbczugQ3cVnHosx7QspCsq8e3GjKcU664/dDe7HhwgQAd2x1tPpX6Bl4auuz AB8RQEdZ+6Kmtchkoz7hvEtcrcEGcR+NJAPjEvygh57mZ/jibHFAZd/D0yWNEN5W WTtWLzyJlviDV34JnQbUm9/lrSHnZroUJ4Fsq0mWtnGCHlMfUF7wgl2yoJFAbvOC XMrHxqXdtRpI6rqDQXBIazqbXorsYudmEroIO0ThTrA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=NylQZS 62uBrE3ZBQTRLMOEeAaC7haJOlaQ328Y77S2w=; b=Yt/BALkIefaXNYgONy0ppy XOhIX0zFMtPrTsf1K/ZZz4bMXRIlh4qY4OCmciM4GRqb9KOobEWWneIm5b9tq2vq CL5I0f6JSF1lWxNl4vySa8psojJQ60RW9C9OgLVGDWHKuSdC3PltEd9ekHAwG4Ps zYihn4dOU9NJxeKK3CghgNhBHoq7knfFW/yL5FmTxdzOQ1RUywVZpu7VocNYG2Mj 60AiFhEM+BGjOwe+XSe7YUN42BGV5jTxZ1bAxsuzGG5yO0UjnGxpCF8ezTBH/+/5 xyGfrA3e7JRsuEi8Ttf3HpJRON4aKuGZBvl7S2BOJmKEHvrfRyBaOqip2bzKPjjQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekjedgieejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeekrdehhedrfeehrdeftdenucevlhhushhtvghrufhiiigvpedvnecurfgrrh grmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-078-055-035-030.78.55.pool.telefonica.de [78.55.35.30]) by mail.messagingengine.com (Postfix) with ESMTPA id D4EC324005B; Mon, 12 Apr 2021 09:37:32 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 1a8f906d (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 12 Apr 2021 13:37:31 +0000 (UTC) Date: Mon, 12 Apr 2021 15:37:30 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley Subject: [PATCH v4 3/8] list-objects: move tag processing into its own function Message-ID: <3d792f63390da79a6d225de51bdfc651234cf893.1618234575.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move processing of tags into its own function to make the logic easier to extend when we're going to implement filtering for tags. No change in behaviour is expected from this commit. Signed-off-by: Patrick Steinhardt --- list-objects.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/list-objects.c b/list-objects.c index e19589baa0..a5a60301cb 100644 --- a/list-objects.c +++ b/list-objects.c @@ -213,6 +213,14 @@ static void process_tree(struct traversal_context *ctx, free_tree_buffer(tree); } +static void process_tag(struct traversal_context *ctx, + struct tag *tag, + const char *name) +{ + tag->object.flags |= SEEN; + ctx->show_object(&tag->object, name, ctx->show_data); +} + static void mark_edge_parents_uninteresting(struct commit *commit, struct rev_info *revs, show_edge_fn show_edge) @@ -334,8 +342,7 @@ static void traverse_trees_and_blobs(struct traversal_context *ctx, if (obj->flags & (UNINTERESTING | SEEN)) continue; if (obj->type == OBJ_TAG) { - obj->flags |= SEEN; - ctx->show_object(obj, name, ctx->show_data); + process_tag(ctx, (struct tag *)obj, name); continue; } if (!path) From patchwork Mon Apr 12 13:37:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12198083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9469FC433B4 for ; Mon, 12 Apr 2021 13:37:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6B2016128C for ; Mon, 12 Apr 2021 13:37:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241983AbhDLNh4 (ORCPT ); Mon, 12 Apr 2021 09:37:56 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:49315 "EHLO wout4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241957AbhDLNh4 (ORCPT ); Mon, 12 Apr 2021 09:37:56 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id E321812AB; Mon, 12 Apr 2021 09:37:37 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 12 Apr 2021 09:37:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=ZE5wN4DRpMTdyalxf8j2MdLRigI zrYChMs0BTznzP68=; b=dyVzsHHqDtFnYAky7moRUgDBdrM9zxfto0PrBGMw+iC imMisr8LPKUHyXFNcFBBVOO7tLtY0CRwmnJyDdqNBCSDtChQOXGGa0cshz/YHC/W Eri4CLSxBZARfGr1TNz2Uw8krArnKg4nBfbmmTlDZMkpiW4FEhc6hyvmRNY45Sgq aN6hc38+7707PVlCGA5Epp2YmTlL4cYBTGflIb8wUvdzwubF/bPUCjXAK9yS3LRm 3cbTNEuaH5kvoFArGjdmWlFDuPq3Z/ls8IuyrSdEXXEubrXzJ5wr+/AwTPWzhzLV t2UpOr21f6QzIQIHokpSLP20ngqmX1XBnrOzwnsXY3w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=ZE5wN4 DRpMTdyalxf8j2MdLRigIzrYChMs0BTznzP68=; b=u/CTJcbKiPq+Us8Lf0SooU nIoEckXr1/b5zOTfC3X6GTwMsxHfm1tN0gjT1GpoRZZphXUqN1uikQKNlAsR9VcN jcu63ML59SlkOdsKVS+E6P6sYp/L8bcmCXs0yLWkdfbu7C/5cD2KZKptH5GpO7Kq UcYFk1QDlNlHSS9s0h16a68u8yGqYCgRq28AQZ/2Ec8WgxYM6AEhBGga5PLRH+k7 IQIhK4qAi+IvwbUjGmpIvS/QtLwAWknOYXPK2Vazq1RMUBgpFLLUeASl8wSb06b9 ZiHbh5yRszvExVAXO6bx0KM2U8MeZpYnhB+U3ieG8j3jfYe7/j066AR6LjgMJuHA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekjedgieejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeekrdehhedrfeehrdeftdenucevlhhushhtvghrufhiiigvpeefnecurfgrrh grmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-078-055-035-030.78.55.pool.telefonica.de [78.55.35.30]) by mail.messagingengine.com (Postfix) with ESMTPA id C5CC924005A; Mon, 12 Apr 2021 09:37:36 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id f41c2c57 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 12 Apr 2021 13:37:36 +0000 (UTC) Date: Mon, 12 Apr 2021 15:37:35 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley Subject: [PATCH v4 4/8] list-objects: support filtering by tag and commit Message-ID: <674da0f9accf64c1d0506013cd216ba17849abc2.1618234575.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Object filters currently only support filtering blobs or trees based on some criteria. This commit lays the foundation to also allow filtering of tags and commits. No change in behaviour is expected from this commit given that there are no filters yet for those object types. Signed-off-by: Patrick Steinhardt --- list-objects-filter.c | 40 ++++++++++++++++++++++++++++++++++++++++ list-objects-filter.h | 2 ++ list-objects.c | 23 ++++++++++++++++++++--- 3 files changed, 62 insertions(+), 3 deletions(-) diff --git a/list-objects-filter.c b/list-objects-filter.c index 39e2f15333..0ebfa52966 100644 --- a/list-objects-filter.c +++ b/list-objects-filter.c @@ -82,6 +82,16 @@ static enum list_objects_filter_result filter_blobs_none( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); /* always include all tree objects */ @@ -173,6 +183,16 @@ static enum list_objects_filter_result filter_trees_depth( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_END_TREE: assert(obj->type == OBJ_TREE); filter_data->current_depth--; @@ -267,6 +287,16 @@ static enum list_objects_filter_result filter_blobs_limit( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); /* always include all tree objects */ @@ -371,6 +401,16 @@ static enum list_objects_filter_result filter_sparse( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); dtype = DT_DIR; diff --git a/list-objects-filter.h b/list-objects-filter.h index cfd784e203..9e98814111 100644 --- a/list-objects-filter.h +++ b/list-objects-filter.h @@ -55,6 +55,8 @@ enum list_objects_filter_result { }; enum list_objects_filter_situation { + LOFS_COMMIT, + LOFS_TAG, LOFS_BEGIN_TREE, LOFS_END_TREE, LOFS_BLOB diff --git a/list-objects.c b/list-objects.c index a5a60301cb..7f404677d5 100644 --- a/list-objects.c +++ b/list-objects.c @@ -217,8 +217,15 @@ static void process_tag(struct traversal_context *ctx, struct tag *tag, const char *name) { - tag->object.flags |= SEEN; - ctx->show_object(&tag->object, name, ctx->show_data); + enum list_objects_filter_result r; + + r = list_objects_filter__filter_object(ctx->revs->repo, LOFS_TAG, + &tag->object, NULL, NULL, + ctx->filter); + if (r & LOFR_MARK_SEEN) + tag->object.flags |= SEEN; + if (r & LOFR_DO_SHOW) + ctx->show_object(&tag->object, name, ctx->show_data); } static void mark_edge_parents_uninteresting(struct commit *commit, @@ -368,6 +375,12 @@ static void do_traverse(struct traversal_context *ctx) strbuf_init(&csp, PATH_MAX); while ((commit = get_revision(ctx->revs)) != NULL) { + enum list_objects_filter_result r; + + r = list_objects_filter__filter_object(ctx->revs->repo, + LOFS_COMMIT, &commit->object, + NULL, NULL, ctx->filter); + /* * an uninteresting boundary commit may not have its tree * parsed yet, but we are not going to show them anyway @@ -382,7 +395,11 @@ static void do_traverse(struct traversal_context *ctx) die(_("unable to load root tree for commit %s"), oid_to_hex(&commit->object.oid)); } - ctx->show_commit(commit, ctx->show_data); + + if (r & LOFR_MARK_SEEN) + commit->object.flags |= SEEN; + if (r & LOFR_DO_SHOW) + ctx->show_commit(commit, ctx->show_data); if (ctx->revs->tree_blobs_in_commit_order) /* From patchwork Mon Apr 12 13:37:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12198085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C352BC433ED for ; Mon, 12 Apr 2021 13:37:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9DF376128B for ; Mon, 12 Apr 2021 13:37:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241989AbhDLNiA (ORCPT ); Mon, 12 Apr 2021 09:38:00 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:36981 "EHLO wout4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241957AbhDLNiA (ORCPT ); Mon, 12 Apr 2021 09:38:00 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 4357216DD; Mon, 12 Apr 2021 09:37:42 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Mon, 12 Apr 2021 09:37:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=ukoXWfDNgrvQb0BJQeLOroSCrfV 7T4aJta/6VjicEss=; b=CF1fluiKTbx8fTHuyyTdOIoyngW/aXFBKzjNpEOcMvU +Q1l36vmMa1OEwwoYgmGVPP4qk0BtjyjRESaPPTc9OYgYHqaM6ZtVC9FKsGuIv/P JiJgckrAOrvmqsO3rsps/MxCbzKskUzzZKOnjHHhDttfYTvVB6ZCdjg8eqbwc7bg p6dmZygwohCVffGIEq3ad6SxWIXQdQ952sriq7Dmk+RKY+BfDnz3fL6priG/a4ei y7pCaHYIUeu+23+ro+IMS1G/8YItCdBvqpVLI0r7TbJ2fwwkIzj8LUyng6UXToAo as176CRChiMeX7GvWEZ1vz3bRC6wryzHpb0+M5X0WYQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=ukoXWf DNgrvQb0BJQeLOroSCrfV7T4aJta/6VjicEss=; b=nKiIB6DYRpPZydi0TQktoA rFg9n/vv20IAy6sfzZMqF8NEHSsILVy66vg/35cdzc6hPb/ZRqEIeO8SkgBVgk3l Xn7RRKKax//YRYh7qdulMcGvH0bU+css1NsxqLGb4SWkIR9ZvaCMltgfQPy+OGpl PnJAJdve4DWdHFZG8PeMafIeCMrnQM8rWND6jrTIb0h86AULDZ8NFkUCMSiliPYj aXjDL5Ceg0x0PjmAKy055MVtD7grJA0wxAnpw0yKahXYE2Vc1QhUFjS2j5YEu2JE lQ3Z5aMpX3CoBSKFeXxensCxAcqSPwqPbXnpQoCevbts9D1jVC0GtCpPD7eb3iwA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekjedgieejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeekrdehhedrfeehrdeftdenucevlhhushhtvghrufhiiigvpedtnecurfgrrh grmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-078-055-035-030.78.55.pool.telefonica.de [78.55.35.30]) by mail.messagingengine.com (Postfix) with ESMTPA id BB46924005A; Mon, 12 Apr 2021 09:37:40 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id aee93e16 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 12 Apr 2021 13:37:40 +0000 (UTC) Date: Mon, 12 Apr 2021 15:37:39 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley Subject: [PATCH v4 5/8] list-objects: implement object type filter Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org While it already is possible to filter objects by some criteria in git-rev-list(1), it is not yet possible to filter out only a specific type of objects. This makes some filters less useful. The `blob:limit` filter for example filters blobs such that only those which are smaller than the given limit are returned. But it is unfit to ask only for these smallish blobs, given that git-rev-list(1) will continue to print tags, commits and trees. Now that we have the infrastructure in place to also filter tags and commits, we can improve this situation by implementing a new filter which selects objects based on their type. Above query can thus trivially be implemented with the following command: $ git rev-list --objects --filter=object:type=blob \ --filter=blob:limit=200 Furthermore, this filter allows to optimize for certain other cases: if for example only tags or commits have been selected, there is no need to walk down trees. The new filter is not yet supported in bitmaps. This is going to be implemented in a subsequent commit. Signed-off-by: Patrick Steinhardt --- Documentation/config/uploadpack.txt | 6 +-- Documentation/rev-list-options.txt | 3 ++ list-objects-filter-options.c | 14 ++++++ list-objects-filter-options.h | 2 + list-objects-filter.c | 76 +++++++++++++++++++++++++++++ t/t6112-rev-list-filters-objects.sh | 48 ++++++++++++++++++ 6 files changed, 146 insertions(+), 3 deletions(-) diff --git a/Documentation/config/uploadpack.txt b/Documentation/config/uploadpack.txt index 6729a072ea..32fad5bbe8 100644 --- a/Documentation/config/uploadpack.txt +++ b/Documentation/config/uploadpack.txt @@ -66,9 +66,9 @@ uploadpackfilter.allow:: uploadpackfilter..allow:: Explicitly allow or ban the object filter corresponding to ``, where `` may be one of: `blob:none`, - `blob:limit`, `tree`, `sparse:oid`, or `combine`. If using - combined filters, both `combine` and all of the nested filter - kinds must be allowed. Defaults to `uploadpackfilter.allow`. + `blob:limit`, `object:type`, `tree`, `sparse:oid`, or `combine`. + If using combined filters, both `combine` and all of the nested + filter kinds must be allowed. Defaults to `uploadpackfilter.allow`. uploadpackfilter.tree.maxDepth:: Only allow `--filter=tree:` when `` is no more than the value of diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index b1c8f86c6e..3afa8fffbd 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -892,6 +892,9 @@ or units. n may be zero. The suffixes k, m, and g can be used to name units in KiB, MiB, or GiB. For example, 'blob:limit=1k' is the same as 'blob:limit=1024'. + +The form '--filter=object:type=(tag|commit|tree|blob)' omits all objects +which are not of the requested type. ++ The form '--filter=sparse:oid=' uses a sparse-checkout specification contained in the blob (or blob-expression) '' to omit blobs that would not be not required for a sparse checkout on diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index d2d1c81caf..bb6f6577d5 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -29,6 +29,8 @@ const char *list_object_filter_config_name(enum list_objects_filter_choice c) return "tree"; case LOFC_SPARSE_OID: return "sparse:oid"; + case LOFC_OBJECT_TYPE: + return "object:type"; case LOFC_COMBINE: return "combine"; case LOFC__COUNT: @@ -97,6 +99,18 @@ static int gently_parse_list_objects_filter( } return 1; + } else if (skip_prefix(arg, "object:type=", &v0)) { + int type = type_from_string_gently(v0, -1, 1); + if (type < 0) { + strbuf_addstr(errbuf, _("expected 'object:type='")); + return 1; + } + + filter_options->object_type = type; + filter_options->choice = LOFC_OBJECT_TYPE; + + return 0; + } else if (skip_prefix(arg, "combine:", &v0)) { return parse_combine_filter(filter_options, v0, errbuf); diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h index 01767c3c96..4d0d0588cc 100644 --- a/list-objects-filter-options.h +++ b/list-objects-filter-options.h @@ -13,6 +13,7 @@ enum list_objects_filter_choice { LOFC_BLOB_LIMIT, LOFC_TREE_DEPTH, LOFC_SPARSE_OID, + LOFC_OBJECT_TYPE, LOFC_COMBINE, LOFC__COUNT /* must be last */ }; @@ -54,6 +55,7 @@ struct list_objects_filter_options { char *sparse_oid_name; unsigned long blob_limit_value; unsigned long tree_exclude_depth; + enum object_type object_type; /* LOFC_COMBINE values */ diff --git a/list-objects-filter.c b/list-objects-filter.c index 0ebfa52966..1c1ee3d1bb 100644 --- a/list-objects-filter.c +++ b/list-objects-filter.c @@ -545,6 +545,81 @@ static void filter_sparse_oid__init( filter->free_fn = filter_sparse_free; } +/* + * A filter for list-objects to omit large blobs. + * And to OPTIONALLY collect a list of the omitted OIDs. + */ +struct filter_object_type_data { + enum object_type object_type; +}; + +static enum list_objects_filter_result filter_object_type( + struct repository *r, + enum list_objects_filter_situation filter_situation, + struct object *obj, + const char *pathname, + const char *filename, + struct oidset *omits, + void *filter_data_) +{ + struct filter_object_type_data *filter_data = filter_data_; + + switch (filter_situation) { + default: + BUG("unknown filter_situation: %d", filter_situation); + + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + if (filter_data->object_type == OBJ_TAG) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + return LOFR_MARK_SEEN; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + if (filter_data->object_type == OBJ_COMMIT) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + return LOFR_MARK_SEEN; + + case LOFS_BEGIN_TREE: + assert(obj->type == OBJ_TREE); + + /* + * If we only want to show commits or tags, then there is no + * need to walk down trees. + */ + if (filter_data->object_type == OBJ_COMMIT || + filter_data->object_type == OBJ_TAG) + return LOFR_SKIP_TREE; + + if (filter_data->object_type == OBJ_TREE) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + return LOFR_MARK_SEEN; + + case LOFS_BLOB: + assert(obj->type == OBJ_BLOB); + + if (filter_data->object_type == OBJ_BLOB) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + return LOFR_MARK_SEEN; + + case LOFS_END_TREE: + return LOFR_ZERO; + } +} + +static void filter_object_type__init( + struct list_objects_filter_options *filter_options, + struct filter *filter) +{ + struct filter_object_type_data *d = xcalloc(1, sizeof(*d)); + d->object_type = filter_options->object_type; + + filter->filter_data = d; + filter->filter_object_fn = filter_object_type; + filter->free_fn = free; +} + /* A filter which only shows objects shown by all sub-filters. */ struct combine_filter_data { struct subfilter *sub; @@ -691,6 +766,7 @@ static filter_init_fn s_filters[] = { filter_blobs_limit__init, filter_trees_depth__init, filter_sparse_oid__init, + filter_object_type__init, filter_combine__init, }; diff --git a/t/t6112-rev-list-filters-objects.sh b/t/t6112-rev-list-filters-objects.sh index 31457d13b9..c79ec04060 100755 --- a/t/t6112-rev-list-filters-objects.sh +++ b/t/t6112-rev-list-filters-objects.sh @@ -159,6 +159,54 @@ test_expect_success 'verify blob:limit=1m' ' test_must_be_empty observed ' +# Test object:type= filter. + +test_expect_success 'setup object-type' ' + git init object-type && + echo contents >object-type/blob && + git -C object-type add blob && + git -C object-type commit -m commit-message && + git -C object-type tag tag -m tag-message +' + +test_expect_success 'verify object:type= fails with invalid type' ' + test_must_fail git -C object-type rev-list --objects --filter=object:type= HEAD && + test_must_fail git -C object-type rev-list --objects --filter=object:type=invalid HEAD +' + +test_expect_success 'verify object:type=blob prints blob and commit' ' + ( + git -C object-type rev-parse HEAD && + printf "%s blob\n" $(git -C object-type rev-parse HEAD:blob) + ) >expected && + git -C object-type rev-list --objects --filter=object:type=blob HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tree prints tree and commit' ' + ( + git -C object-type rev-parse HEAD && + printf "%s \n" $(git -C object-type rev-parse HEAD^{tree}) + ) >expected && + git -C object-type rev-list --objects --filter=object:type=tree HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=commit prints commit' ' + git -C object-type rev-parse HEAD >expected && + git -C object-type rev-list --objects --filter=object:type=commit HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tag prints tag' ' + ( + git -C object-type rev-parse HEAD && + printf "%s tag\n" $(git -C object-type rev-parse tag) + ) >expected && + git -C object-type rev-list --objects --filter=object:type=tag tag >actual && + test_cmp expected actual +' + # Test sparse:path= filter. # !!!! # NOTE: sparse:path filter support has been dropped for security reasons, From patchwork Mon Apr 12 13:37:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12198087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA67DC433B4 for ; Mon, 12 Apr 2021 13:37:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B93E36128B for ; Mon, 12 Apr 2021 13:37:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241992AbhDLNiF (ORCPT ); Mon, 12 Apr 2021 09:38:05 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:54535 "EHLO wout4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241957AbhDLNiE (ORCPT ); Mon, 12 Apr 2021 09:38:04 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id E720D16D9; Mon, 12 Apr 2021 09:37:45 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 12 Apr 2021 09:37:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=MfGzchl8ejzqUJAct36DPIA/bMj 0w1zRuT/YKuASZ0o=; b=Be+hke3495Yjo6NaNKSYzO7tAs4tu+B8m8aK597Nq2U r2ooHRgIksh7TFCWsms7d/4LhQDA0/SOuNixxuT9i/I/xfqh6LC4Ru7fKH3MQbNB 3tQfNEyqy+uqkK4oOtphfEhHfUFCUd3nWmtjcJVlaGJMwT1Cgru6DC8oJ8wHs8Rv NhaLk2s4PjXIuCNXwWI138ZD0MH7vBFY6RqoZd7kOAc0IhWwd8OJFL91uBJ0FvRX LsPa9pvoSSmSdIA+hsLyEN5nrcxUwIVKLB55u+0XUhujlu5ekU1OeqZnW+5Tsh0N MqXCrwTX97BfSurpF6/MWWjSysza4egX3eVbMxDgqxQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=MfGzch l8ejzqUJAct36DPIA/bMj0w1zRuT/YKuASZ0o=; b=bLKpvDYY+UpegRPVbUj8Re WEQeycX1Cs2SgCOpx7BUXHfqouUoJ2EaCfH1SWYBvy0Ucva2jI5QYsKvqGS+O0n1 OZT3YtQMCuWh99GotJTPzlk4ofEHy8GLqL/4pkxVWJh6+qGp2SVG9ZAhpctLfZXX QLDvijF9Nive2tJ9w0VcdX7ISdT2MBdbKdnKwXcjV2fuCbVVG4f8w89ldotY3lKm H68/W0MXUsX0/cchjGdlM7tNTEpACToLCgp38HrgjeYqAhZ6rCZRXjnhi/khZkdd 6TSClggwZw6ItufHNWBegVI1ukDZPhbWwvWShh3z2epU/+9TQ+gxVHSi9KA3wLmg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekjedgieejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeekrdehhedrfeehrdeftdenucevlhhushhtvghrufhiiigvpeegnecurfgrrh grmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-078-055-035-030.78.55.pool.telefonica.de [78.55.35.30]) by mail.messagingengine.com (Postfix) with ESMTPA id AD03B24005A; Mon, 12 Apr 2021 09:37:44 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id b36a959b (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 12 Apr 2021 13:37:44 +0000 (UTC) Date: Mon, 12 Apr 2021 15:37:43 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley Subject: [PATCH v4 6/8] pack-bitmap: implement object type filter Message-ID: <17c9f66bbcff1f949caaf928102fbab7b96685d6.1618234575.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The preceding commit has added a new object filter for git-rev-list(1) which allows to filter objects by type. Implement the equivalent filter for packfile bitmaps so that we can answer these queries fast. Signed-off-by: Patrick Steinhardt --- pack-bitmap.c | 29 ++++++++++++++++++++++++++--- t/t6113-rev-list-bitmap-filters.sh | 25 ++++++++++++++++++++++++- 2 files changed, 50 insertions(+), 4 deletions(-) diff --git a/pack-bitmap.c b/pack-bitmap.c index b4513f8672..cd3f5c433e 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -779,9 +779,6 @@ static void filter_bitmap_exclude_type(struct bitmap_index *bitmap_git, eword_t mask; uint32_t i; - if (type != OBJ_BLOB && type != OBJ_TREE) - BUG("filter_bitmap_exclude_type: unsupported type '%d'", type); - /* * The non-bitmap version of this filter never removes * objects which the other side specifically asked for, @@ -911,6 +908,24 @@ static void filter_bitmap_tree_depth(struct bitmap_index *bitmap_git, OBJ_BLOB); } +static void filter_bitmap_object_type(struct bitmap_index *bitmap_git, + struct object_list *tip_objects, + struct bitmap *to_filter, + enum object_type object_type) +{ + if (object_type < OBJ_COMMIT || object_type > OBJ_TAG) + BUG("filter_bitmap_object_type given invalid object"); + + if (object_type != OBJ_TAG) + filter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_TAG); + if (object_type != OBJ_COMMIT) + filter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_COMMIT); + if (object_type != OBJ_TREE) + filter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_TREE); + if (object_type != OBJ_BLOB) + filter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_BLOB); +} + static int filter_bitmap(struct bitmap_index *bitmap_git, struct object_list *tip_objects, struct bitmap *to_filter, @@ -943,6 +958,14 @@ static int filter_bitmap(struct bitmap_index *bitmap_git, return 0; } + if (filter->choice == LOFC_OBJECT_TYPE) { + if (bitmap_git) + filter_bitmap_object_type(bitmap_git, tip_objects, + to_filter, + filter->object_type); + return 0; + } + /* filter choice not handled */ return -1; } diff --git a/t/t6113-rev-list-bitmap-filters.sh b/t/t6113-rev-list-bitmap-filters.sh index 3f889949ca..fb66735ac8 100755 --- a/t/t6113-rev-list-bitmap-filters.sh +++ b/t/t6113-rev-list-bitmap-filters.sh @@ -10,7 +10,8 @@ test_expect_success 'set up bitmapped repo' ' test_commit much-larger-blob-one && git repack -adb && test_commit two && - test_commit much-larger-blob-two + test_commit much-larger-blob-two && + git tag tag ' test_expect_success 'filters fallback to non-bitmap traversal' ' @@ -75,4 +76,26 @@ test_expect_success 'tree:1 filter' ' test_cmp expect actual ' +test_expect_success 'object:type filter' ' + git rev-list --objects --filter=object:type=tag tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=tag tag >actual && + test_cmp expect actual && + + git rev-list --objects --filter=object:type=commit tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=commit tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter=object:type=tree tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=tree tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual +' + test_done From patchwork Mon Apr 12 13:37:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12198089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 904B4C433ED for ; Mon, 12 Apr 2021 13:37:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 60C1C6128B for ; Mon, 12 Apr 2021 13:37:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241996AbhDLNiK (ORCPT ); Mon, 12 Apr 2021 09:38:10 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:54165 "EHLO wout4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241957AbhDLNiK (ORCPT ); Mon, 12 Apr 2021 09:38:10 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id E033412AB; Mon, 12 Apr 2021 09:37:51 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 12 Apr 2021 09:37:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=I8n7sQ+mPuObR10YIz+RH6luhj5 g+WhDmy7MZBfo8Vo=; b=i5Njnj05AX6ZZfrUr8nGlHU5R3roe3V+p49znaTYF1h aU3Tz6JvjxkmMRM0AE3hRNWyzyOpkPXFa3/EgFXcPBoOSu8LMSwX8MoGVzzYUPHB Bo84axi8TeECfEQSfmjwHsQrsHdQA5cu6q6MPgg+oQhADSFYo6UFZxh6qew9hbzI Bwm0XcP8Y2wepuEqL0rlDyqONcND2na+8X3ahcB36zPLWDI3ywbLEYPH2EtPnoWh pa5Fj8iVnIvpv+QKUk5ENM1BVkXcWGe/UnvkZad0bUOBdjDsxzRDkqhuEGyPRkY/ Hs9xiroQhLXw6IgjTZoIRol9KLOFfZ5qdA/ioxnHYww== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=I8n7sQ +mPuObR10YIz+RH6luhj5g+WhDmy7MZBfo8Vo=; b=gL2EhMhTcN9hN4FI0868Na wu7uU4tfkHA9mUJnINHd0PKAAF9cWQAhT6HTppaGbbZJ9bdr+eROWTUd8VeV3/35 OZgTZ+b4+UutbRX+300CKI6wcbJ3NJiItFj2p9NywmM+itb/ldUOzRfEeJRDXaza rb1GBo73xXJgEKL+3/z3RYjZgauefcNfgvDNMqpqX6enZ64G89KCEZ8lTu3vX1h0 P24FVKwNZhNcGrRxf8gVdHJ5DzW/7+cv6UrlrU1fk+ylt3HvuG1FIvsqbjPuEM92 3In5F9ECOJVoIDY3sIX/F9eadPhGmIMrCL826+zOr0JzusOq+2dTN+lYqA/4MVsw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekjedgieejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeekrdehhedrfeehrdeftdenucevlhhushhtvghrufhiiigvpeehnecurfgrrh grmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-078-055-035-030.78.55.pool.telefonica.de [78.55.35.30]) by mail.messagingengine.com (Postfix) with ESMTPA id AD272240066; Mon, 12 Apr 2021 09:37:50 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 4193a035 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 12 Apr 2021 13:37:49 +0000 (UTC) Date: Mon, 12 Apr 2021 15:37:48 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley Subject: [PATCH v4 7/8] pack-bitmap: implement combined filter Message-ID: <759ac54bb2551097d62ffeac77e6e698eeae96ec.1618234575.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When the user has multiple objects filters specified, then this is internally represented by having a "combined" filter. These combined filters aren't yet supported by bitmap indices and can thus not be accelerated. Fix this by implementing support for these combined filters. The implementation is quite trivial: when there's a combined filter, we simply recurse into `filter_bitmap()` for all of the sub-filters. Signed-off-by: Patrick Steinhardt --- pack-bitmap.c | 10 ++++++++++ t/t6113-rev-list-bitmap-filters.sh | 7 +++++++ 2 files changed, 17 insertions(+) diff --git a/pack-bitmap.c b/pack-bitmap.c index cd3f5c433e..7ce3ede7e4 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -966,6 +966,16 @@ static int filter_bitmap(struct bitmap_index *bitmap_git, return 0; } + if (filter->choice == LOFC_COMBINE) { + int i; + for (i = 0; i < filter->sub_nr; i++) { + if (filter_bitmap(bitmap_git, tip_objects, to_filter, + &filter->sub[i]) < 0) + return -1; + } + return 0; + } + /* filter choice not handled */ return -1; } diff --git a/t/t6113-rev-list-bitmap-filters.sh b/t/t6113-rev-list-bitmap-filters.sh index fb66735ac8..cb9db7df6f 100755 --- a/t/t6113-rev-list-bitmap-filters.sh +++ b/t/t6113-rev-list-bitmap-filters.sh @@ -98,4 +98,11 @@ test_expect_success 'object:type filter' ' test_bitmap_traversal expect actual ' +test_expect_success 'combine filter' ' + git rev-list --objects --filter=blob:limit=1000 --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=blob:limit=1000 --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual +' + test_done From patchwork Mon Apr 12 13:37:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12198091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14DE6C433ED for ; Mon, 12 Apr 2021 13:37:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CC5DA61288 for ; Mon, 12 Apr 2021 13:37:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241999AbhDLNiO (ORCPT ); Mon, 12 Apr 2021 09:38:14 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:53171 "EHLO wout4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238495AbhDLNiO (ORCPT ); Mon, 12 Apr 2021 09:38:14 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id CD2541681; Mon, 12 Apr 2021 09:37:55 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 12 Apr 2021 09:37:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=vy7YMdPGC8CN57IrKcdSgw31SeG K9niymFaU1RPC5XU=; b=ka1bzhi4iL+N1lzHZ32o1mRV+D3wINTRnjb4NQziOUw RC5G46lLugd2bLbQl4Mjp0/M/92qPGVYwomfWwwuSp40jVI82QzsXvgO/zkN+GEH HQmho31geMTy6AmUdLElo6XMDYbLmSIzc5ZTPLJwGP6sp8PcMfLGN/Zin26ogPZw R7b6jzmlCUoT+YGKn7hFPmQzBYGrlMYQdZsC/nm4CLjUa30jnlPVgUcVCpWxyY9Y wexo+vH57d52zbrtHw3557WXL7iR4FP+byC1Y9P3rwGDU6IGCLbNqi3tWi3NjswE SbEooVbElwuI1H+gfFeqeuFCchtJM0y9IBMeQOBir8Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=vy7YMd PGC8CN57IrKcdSgw31SeGK9niymFaU1RPC5XU=; b=k8mCAkoTmUwlAMRnXAlMmz riHi7DmoPwQPbNph+XMDuCFbUPdsU41FAQltKYd+T+4zQGRq72COfGtDe9fAkIS2 xRtvefSphKxxCpovTpr7K9Gofj09Ve3FOsCDDWKaYzWx0gtQ6Rm0WKZJdU0ZA3vg jLo4dX+o0wPzFvzX0LP2YcI7WqX8qw0+hd4N3Ww52gy7CpHvjj4m8KvHjNgdG6D4 hPVhejaZU7tB9C52pas2bAN+PCuowG8FkyhbJpDSie3oYjs9Q8yh36LWCYQOOaFN V++PalpZHEyao2KVhqRjnvvkvMXiy4U3Z1Sj43pA3r051rwo/YBnfI0dXZVhSipw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekjedgieejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpefgfeejuefgheegueeljeffteefuedthfdvvedugfevvdffleduueeiiefgvedtheen ucffohhmrghinhepphgvnhguihhnghdrnhhrnecukfhppeejkedrheehrdefhedrfedtne cuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphhssehp khhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-078-055-035-030.78.55.pool.telefonica.de [78.55.35.30]) by mail.messagingengine.com (Postfix) with ESMTPA id 9F23C24005A; Mon, 12 Apr 2021 09:37:54 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 93775e50 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 12 Apr 2021 13:37:54 +0000 (UTC) Date: Mon, 12 Apr 2021 15:37:53 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley Subject: [PATCH v4 8/8] rev-list: allow filtering of provided items Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When providing an object filter, it is currently impossible to also filter provided items. E.g. when executing `git rev-list HEAD` , the commit this reference points to will be treated as user-provided and is thus excluded from the filtering mechanism. This makes it harder than necessary to properly use the new `--filter=object:type` filter given that even if the user wants to only see blobs, he'll still see commits of provided references. Improve this by introducing a new `--filter-provided-objects` option to the git-rev-parse(1) command. If given, then all user-provided references will be subject to filtering. Signed-off-by: Patrick Steinhardt --- Documentation/rev-list-options.txt | 5 ++++ builtin/pack-objects.c | 2 +- builtin/rev-list.c | 36 +++++++++++++++++++++-------- pack-bitmap.c | 6 +++-- pack-bitmap.h | 3 ++- reachable.c | 2 +- t/t6112-rev-list-filters-objects.sh | 28 ++++++++++++++++++++++ t/t6113-rev-list-bitmap-filters.sh | 36 +++++++++++++++++++++++++++++ 8 files changed, 104 insertions(+), 14 deletions(-) diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index 3afa8fffbd..5bf2a85f69 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -933,6 +933,11 @@ equivalent. --no-filter:: Turn off any previous `--filter=` argument. +--filter-provided-objects:: + Filter the list of explicitly provided objects, which would otherwise + always be printed even if they did not match any of the filters. Only + useful with `--filter=`. + --filter-print-omitted:: Only useful with `--filter=`; prints a list of the objects omitted by the filter. Object IDs are prefixed with a ``~'' character. diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 525c2d8552..2f2026dc87 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -3516,7 +3516,7 @@ static int pack_options_allow_reuse(void) static int get_object_list_from_bitmap(struct rev_info *revs) { - if (!(bitmap_git = prepare_bitmap_walk(revs, &filter_options))) + if (!(bitmap_git = prepare_bitmap_walk(revs, &filter_options, 0))) return -1; if (pack_options_allow_reuse() && diff --git a/builtin/rev-list.c b/builtin/rev-list.c index b4d8ea0a35..7677b1af5a 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -398,7 +398,8 @@ static inline int parse_missing_action_value(const char *value) } static int try_bitmap_count(struct rev_info *revs, - struct list_objects_filter_options *filter) + struct list_objects_filter_options *filter, + int filter_provided_objects) { uint32_t commit_count = 0, tag_count = 0, @@ -433,7 +434,7 @@ static int try_bitmap_count(struct rev_info *revs, */ max_count = revs->max_count; - bitmap_git = prepare_bitmap_walk(revs, filter); + bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects); if (!bitmap_git) return -1; @@ -450,7 +451,8 @@ static int try_bitmap_count(struct rev_info *revs, } static int try_bitmap_traversal(struct rev_info *revs, - struct list_objects_filter_options *filter) + struct list_objects_filter_options *filter, + int filter_provided_objects) { struct bitmap_index *bitmap_git; @@ -461,7 +463,7 @@ static int try_bitmap_traversal(struct rev_info *revs, if (revs->max_count >= 0) return -1; - bitmap_git = prepare_bitmap_walk(revs, filter); + bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects); if (!bitmap_git) return -1; @@ -471,14 +473,15 @@ static int try_bitmap_traversal(struct rev_info *revs, } static int try_bitmap_disk_usage(struct rev_info *revs, - struct list_objects_filter_options *filter) + struct list_objects_filter_options *filter, + int filter_provided_objects) { struct bitmap_index *bitmap_git; if (!show_disk_usage) return -1; - bitmap_git = prepare_bitmap_walk(revs, filter); + bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects); if (!bitmap_git) return -1; @@ -499,6 +502,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) int bisect_show_vars = 0; int bisect_find_all = 0; int use_bitmap_index = 0; + int filter_provided_objects = 0; const char *show_progress = NULL; if (argc == 2 && !strcmp(argv[1], "-h")) @@ -599,6 +603,10 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) list_objects_filter_set_no_filter(&filter_options); continue; } + if (!strcmp(arg, "--filter-provided-objects")) { + filter_provided_objects = 1; + continue; + } if (!strcmp(arg, "--filter-print-omitted")) { arg_print_omitted = 1; continue; @@ -665,11 +673,11 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) progress = start_delayed_progress(show_progress, 0); if (use_bitmap_index) { - if (!try_bitmap_count(&revs, &filter_options)) + if (!try_bitmap_count(&revs, &filter_options, filter_provided_objects)) return 0; - if (!try_bitmap_disk_usage(&revs, &filter_options)) + if (!try_bitmap_disk_usage(&revs, &filter_options, filter_provided_objects)) return 0; - if (!try_bitmap_traversal(&revs, &filter_options)) + if (!try_bitmap_traversal(&revs, &filter_options, filter_provided_objects)) return 0; } @@ -694,6 +702,16 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) return show_bisect_vars(&info, reaches, all); } + if (filter_provided_objects) { + struct commit_list *c; + for (i = 0; i < revs.pending.nr; i++) { + struct object_array_entry *pending = revs.pending.objects + i; + pending->item->flags |= NOT_USER_GIVEN; + } + for (c = revs.commits; c; c = c->next) + c->item->object.flags |= NOT_USER_GIVEN; + } + if (arg_print_omitted) oidset_init(&omitted_objects, DEFAULT_OIDSET_SIZE); if (arg_missing_action == MA_PRINT) diff --git a/pack-bitmap.c b/pack-bitmap.c index 7ce3ede7e4..6b790a834b 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -986,7 +986,8 @@ static int can_filter_bitmap(struct list_objects_filter_options *filter) } struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, - struct list_objects_filter_options *filter) + struct list_objects_filter_options *filter, + int filter_provided_objects) { unsigned int i; @@ -1081,7 +1082,8 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, if (haves_bitmap) bitmap_and_not(wants_bitmap, haves_bitmap); - filter_bitmap(bitmap_git, wants, wants_bitmap, filter); + filter_bitmap(bitmap_git, (filter && filter_provided_objects) ? NULL : wants, + wants_bitmap, filter); bitmap_git->result = wants_bitmap; bitmap_git->haves = haves_bitmap; diff --git a/pack-bitmap.h b/pack-bitmap.h index 36d99930d8..bb45217d3b 100644 --- a/pack-bitmap.h +++ b/pack-bitmap.h @@ -50,7 +50,8 @@ void traverse_bitmap_commit_list(struct bitmap_index *, show_reachable_fn show_reachable); void test_bitmap_walk(struct rev_info *revs); struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, - struct list_objects_filter_options *filter); + struct list_objects_filter_options *filter, + int filter_provided_objects); int reuse_partial_packfile_from_bitmap(struct bitmap_index *, struct packed_git **packfile, uint32_t *entries, diff --git a/reachable.c b/reachable.c index 77a60c70a5..fc833cae43 100644 --- a/reachable.c +++ b/reachable.c @@ -223,7 +223,7 @@ void mark_reachable_objects(struct rev_info *revs, int mark_reflog, cp.progress = progress; cp.count = 0; - bitmap_git = prepare_bitmap_walk(revs, NULL); + bitmap_git = prepare_bitmap_walk(revs, NULL, 0); if (bitmap_git) { traverse_bitmap_commit_list(bitmap_git, revs, mark_object_seen); free_bitmap_index(bitmap_git); diff --git a/t/t6112-rev-list-filters-objects.sh b/t/t6112-rev-list-filters-objects.sh index c79ec04060..de751b65b4 100755 --- a/t/t6112-rev-list-filters-objects.sh +++ b/t/t6112-rev-list-filters-objects.sh @@ -207,6 +207,34 @@ test_expect_success 'verify object:type=tag prints tag' ' test_cmp expected actual ' +test_expect_success 'verify object:type=blob prints only blob with --filter-provided-objects' ' + printf "%s blob\n" $(git -C object-type rev-parse HEAD:blob) >expected && + git -C object-type rev-list --objects \ + --filter=object:type=blob --filter-provided-objects HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tree prints only tree with --filter-provided-objects' ' + printf "%s \n" $(git -C object-type rev-parse HEAD^{tree}) >expected && + git -C object-type rev-list --objects \ + --filter=object:type=tree HEAD --filter-provided-objects >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=commit prints only commit with --filter-provided-objects' ' + git -C object-type rev-parse HEAD >expected && + git -C object-type rev-list --objects \ + --filter=object:type=commit --filter-provided-objects HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tag prints only tag with --filter-provided-objects' ' + printf "%s tag\n" $(git -C object-type rev-parse tag) >expected && + git -C object-type rev-list --objects \ + --filter=object:type=tag --filter-provided-objects tag >actual && + test_cmp expected actual +' + # Test sparse:path= filter. # !!!! # NOTE: sparse:path filter support has been dropped for security reasons, diff --git a/t/t6113-rev-list-bitmap-filters.sh b/t/t6113-rev-list-bitmap-filters.sh index cb9db7df6f..4d8e09167e 100755 --- a/t/t6113-rev-list-bitmap-filters.sh +++ b/t/t6113-rev-list-bitmap-filters.sh @@ -98,6 +98,28 @@ test_expect_success 'object:type filter' ' test_bitmap_traversal expect actual ' +test_expect_success 'object:type filter with --filter-provided-objects' ' + git rev-list --objects --filter-provided-objects --filter=object:type=tag tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided-objects --filter=object:type=tag tag >actual && + test_cmp expect actual && + + git rev-list --objects --filter-provided-objects --filter=object:type=commit tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided-objects --filter=object:type=commit tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter-provided-objects --filter=object:type=tree tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided-objects --filter=object:type=tree tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter-provided-objects --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided-objects --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual +' + test_expect_success 'combine filter' ' git rev-list --objects --filter=blob:limit=1000 --filter=object:type=blob tag >expect && git rev-list --use-bitmap-index \ @@ -105,4 +127,18 @@ test_expect_success 'combine filter' ' test_bitmap_traversal expect actual ' +test_expect_success 'combine filter with --filter-provided-objects' ' + git rev-list --objects --filter-provided-objects --filter=blob:limit=1000 --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided-objects --filter=blob:limit=1000 --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual && + + git cat-file --batch-check="%(objecttype) %(objectsize)" objects && + while read objecttype objectsize + do + test "$objecttype" = blob || return 1 + test "$objectsize" -le 1000 || return 1 + done