From patchwork Fri Apr 9 11:27:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12193835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40B54C433B4 for ; Fri, 9 Apr 2021 11:27:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E5ABE61177 for ; Fri, 9 Apr 2021 11:27:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233533AbhDIL2I (ORCPT ); Fri, 9 Apr 2021 07:28:08 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:43561 "EHLO out3-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233530AbhDIL2I (ORCPT ); Fri, 9 Apr 2021 07:28:08 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 42F115C004A; Fri, 9 Apr 2021 07:27:55 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Fri, 09 Apr 2021 07:27:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=vUkJjMbAaKnoAL2luhpkldKlHQF JyUo2e/y+BcVSD24=; b=HHWm52cZxR9EKS5yQ34C+yiNuiLbndCUCOzeqnp/glx 0NDvr6WtnWl9peOoB/UpvLjG4I/NHN/Ghmid+zCHaZiA0JG3aaHXhPBlnBXlce72 9tQ9stWRl64IIERBx4TxxCRxFNO7FqWR+pwM12Fs7JvSNDLHrWYMTl/r+LsamZCA Pr9sfroGPZHIdI+ipvMsXPgT/99xFkDQwvYPtsYKccpYNIZpjRxNCL1J+VxZytvu 6QaXo/IObycBydfKBjxSSamzTOFRXzDzHhohWocTvIAmrF753j6xO0g1b5u4Ujge OOu93imqYh2SZpW8rPQOqJqRwJyUU4lt/SezAt/53wA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=vUkJjM bAaKnoAL2luhpkldKlHQFJyUo2e/y+BcVSD24=; b=XND9Il4m07XP5u3cOPocfH 1jIhMJSbS3tcSOq7if8yPP6U/o2RG00MGljY/i3wJIOW6o4IXAgNc09yFgw+dIQE mNjA5qTcFBKGJWeLTYOXLDzB4PxtwmcO4YzqxeYljTOGo3+tdMNMQm4sZVNoQowT DWEgsHW3ywkZrgZae82b5ooKUZf2ccbFyoBG446Be5Atfx+yaHisDFmX8S/AY25Y i7dhSgmNF6Vu9yaSDc3AY+eTF/IHMt5NVvc9AVXrvhBqSiJS7HeXC+GYy3N+z9Dp lCtqbmNkwwCWRdCsPauztWhGbqeEtGsibVOoD5zPmj3Ub6TN4iKim/1CmPlCH7LQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekuddggeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeejrdduledurdekrdduieelnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf08a9.dyn.telefonica.de [77.191.8.169]) by mail.messagingengine.com (Postfix) with ESMTPA id 56188240067; Fri, 9 Apr 2021 07:27:54 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 050ef418 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 9 Apr 2021 11:27:54 +0000 (UTC) Date: Fri, 9 Apr 2021 13:27:52 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau Subject: [PATCH v3 1/8] uploadpack.txt: document implication of `uploadpackfilter.allow` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When `uploadpackfilter.allow` is set to `true`, it means that filters are enabled by default except in the case where a filter is explicitly disabled via `uploadpackilter..allow`. This option will not only enable the currently supported set of filters, but also any filters which get added in the future. As such, an admin which wants to have tight control over which filters are allowed and which aren't probably shouldn't ever set `uploadpackfilter.allow=true`. Amend the documentation to make the ramifications more explicit so that admins are aware of this. Signed-off-by: Patrick Steinhardt --- Documentation/config/uploadpack.txt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Documentation/config/uploadpack.txt b/Documentation/config/uploadpack.txt index b0d761282c..6729a072ea 100644 --- a/Documentation/config/uploadpack.txt +++ b/Documentation/config/uploadpack.txt @@ -59,7 +59,8 @@ uploadpack.allowFilter:: uploadpackfilter.allow:: Provides a default value for unspecified object filters (see: the - below configuration variable). + below configuration variable). If set to `true`, this will also + enable all filters which get added in the future. Defaults to `true`. uploadpackfilter..allow:: From patchwork Fri Apr 9 11:27:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12193837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86C79C433ED for ; Fri, 9 Apr 2021 11:28:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 536106105A for ; Fri, 9 Apr 2021 11:28:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233564AbhDIL2P (ORCPT ); Fri, 9 Apr 2021 07:28:15 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:37711 "EHLO out3-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231599AbhDIL2N (ORCPT ); Fri, 9 Apr 2021 07:28:13 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id D73755C0093; Fri, 9 Apr 2021 07:28:00 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Fri, 09 Apr 2021 07:28:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=3HFQtrD5QIMa3ooX/VKrKiz7YJ1 rk8ybu00BZsvO5Nk=; b=JMJS5uwqmz+wJDSBW7X0I04TNTdLD+4jbHTomvQNbpw CffkQ3xnrYo/YMSSNxZI5If89s/i67E/FUx7cxcXPT0H2IL4fzsfbcH4q1LeJCb2 NFf7KuoLL3nLydLatpuiPyzzRP7Fi+9Lr8nJQ5KjN5/7TyOdf/1TXfkeSvDSScNY R3HQGFP0zsAsHrBemUYsoRIEuR9IRewwtdrmo92b9SDMEGtynBQlDa+iottcvaNo a/srVep3U0W876IHPX7umOn9Ydk9JwA6rMvWeOVwmpYYrzMH8ZNykkWIDdgeK73c w45NRnWzwdvqLXcGq5afKV+lo376btxcVZNzTrajynA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=3HFQtr D5QIMa3ooX/VKrKiz7YJ1rk8ybu00BZsvO5Nk=; b=esi1r0aXa80k6KiL2DvJAn 3WjWtSNYzTimovDJLE7+YOI/vvwrXKcZqmBv4dHG8ULTVJ4X7Ec4IrrRkwlGp4Pv qrnH5rBNaeD+40DwI4+YX4ddM43t48SHorJjE8XZqm2xmUKcJLVGgQLbDhJ0gFUI h5ZyXsZ/ZgnyppDRWKAu0pWLVEtN2FbozeIPEQAmYWEgJ33ZsDszXuIN+dVvTW+a 1STMUL1mwnQ+LF+VW2Ka8cdf6yDb+9Urymv06ORBxvunqreVD8C1VadBR4qmO89B 8U7YZ2w98CO4j72gxMQO+HPGPQwdbeYkFxhnsvCQqLNXqFbgrSjqga8maajqXH+w == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekuddggeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeejrdduledurdekrdduieelnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf08a9.dyn.telefonica.de [77.191.8.169]) by mail.messagingengine.com (Postfix) with ESMTPA id 35247240065; Fri, 9 Apr 2021 07:28:00 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 5ba95a97 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 9 Apr 2021 11:27:59 +0000 (UTC) Date: Fri, 9 Apr 2021 13:27:57 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau Subject: [PATCH v3 2/8] revision: mark commit parents as NOT_USER_GIVEN Message-ID: <46c1952405173eca40706ed7be3b939b210ad2d5.1617967252.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The NOT_USER_GIVEN flag of an object marks whether a flag was explicitly provided by the user or not. The most important use case for this is when filtering objects: only objects that were not explicitly requested will get filtered. The flag is currently only set for blobs and trees, which has been fine given that there are no filters for tags or commits currently. We're about to extend filtering capabilities to add object type filter though, which requires us to set up the NOT_USER_GIVEN flag correctly -- if it's not set, the object wouldn't get filtered at all. Mark unseen commit parents as NOT_USER_GIVEN when processing parents. Like this, explicitly provided parents stay user-given and thus unfiltered, while parents which get loaded as part of the graph walk can be filtered. This commit shouldn't have any user-visible impact yet as there is no logic to filter commits yet. Signed-off-by: Patrick Steinhardt --- revision.c | 4 ++-- revision.h | 3 --- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/revision.c b/revision.c index 553c0faa9b..fd34c75e23 100644 --- a/revision.c +++ b/revision.c @@ -1123,7 +1123,7 @@ static int process_parents(struct rev_info *revs, struct commit *commit, mark_parents_uninteresting(p); if (p->object.flags & SEEN) continue; - p->object.flags |= SEEN; + p->object.flags |= (SEEN | NOT_USER_GIVEN); if (list) commit_list_insert_by_date(p, list); if (queue) @@ -1165,7 +1165,7 @@ static int process_parents(struct rev_info *revs, struct commit *commit, } p->object.flags |= left_flag; if (!(p->object.flags & SEEN)) { - p->object.flags |= SEEN; + p->object.flags |= (SEEN | NOT_USER_GIVEN); if (list) commit_list_insert_by_date(p, list); if (queue) diff --git a/revision.h b/revision.h index a24f72dcd1..93aa012f51 100644 --- a/revision.h +++ b/revision.h @@ -44,9 +44,6 @@ /* * Indicates object was reached by traversal. i.e. not given by user on * command-line or stdin. - * NEEDSWORK: NOT_USER_GIVEN doesn't apply to commits because we only support - * filtering trees and blobs, but it may be useful to support filtering commits - * in the future. */ #define NOT_USER_GIVEN (1u<<25) #define TRACK_LINEAR (1u<<26) From patchwork Fri Apr 9 11:28:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12193839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D92F7C433ED for ; Fri, 9 Apr 2021 11:28:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A81C9610F7 for ; Fri, 9 Apr 2021 11:28:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233591AbhDIL2T (ORCPT ); Fri, 9 Apr 2021 07:28:19 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:40529 "EHLO out3-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233569AbhDIL2T (ORCPT ); Fri, 9 Apr 2021 07:28:19 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 1227E5C0093; Fri, 9 Apr 2021 07:28:06 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Fri, 09 Apr 2021 07:28:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=0Ib9k7zztJ5PEPwoceqdhosrSXz QoKBTCD3p7JL90Ag=; b=OGNShehybGVvs4VgxxObjF2ybSlOPOGyQvcrLkFgjgn kerHwRCb+8xep7i+kIuALesS69cEf1tnoqh/l0YohylIf6QugYrkzOUYkSPEZqtL 2LEd3nlDVXteb/lpr8QHp6BAHVnyQ11A+1TP+lbqKdLoLc4btd7NL2HruqPdM/lW XbEN0f94IpeQ5qHe9qQ0SA+EW5eLBQgPzeGgSwonDx/gAaF/KyGhc4v8sGaM5uyL aUy59rIz8NMdzXdvWiouIvd8sy0cYa/Nnz4g56SorVBgJRRddegi8wlTOArGxmTz 3VgCKMrCOGyzmfXsYrXO0hpGLZ0Ox/uvYnSFd/Boa6g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=0Ib9k7 zztJ5PEPwoceqdhosrSXzQoKBTCD3p7JL90Ag=; b=gAHiZQT6Iby6bdYiAjywm/ EkBMhSYoclLbhH9i441McNCSXpXt3y6PTtLvenC3ZZIj9tzcDaUJRjOaDl9xtDir Syac21+HtQ2bKT0qekdSuUHazZ1b2qVpY5PLmDZZCHZz6WpyKlLe5/T/pdN/FZHo Bug8lMiIZcbGPq09uAPR2foZpE8GzLqU9Jh+tCu10qlzPQ4xQ2C3AJAA7VL7XRTU QTKo45Zu20Yx9RfG48znPzFNLyxUhoFyPEZI83SvISr7or+YJqgKJfhJcczjXZ0G nLay4U1RXfryxLXYq26/8v39UV7w2666L9qFmbSxWUV0Dl7ZZJJ2XXgYqGpJVhMA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekuddggeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeejrdduledurdekrdduieelnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf08a9.dyn.telefonica.de [77.191.8.169]) by mail.messagingengine.com (Postfix) with ESMTPA id 0FC9324005C; Fri, 9 Apr 2021 07:28:04 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 0721b4c1 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 9 Apr 2021 11:28:03 +0000 (UTC) Date: Fri, 9 Apr 2021 13:28:02 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau Subject: [PATCH v3 3/8] list-objects: move tag processing into its own function Message-ID: <3d792f63390da79a6d225de51bdfc651234cf893.1617967252.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move processing of tags into its own function to make the logic easier to extend when we're going to implement filtering for tags. No change in behaviour is expected from this commit. Signed-off-by: Patrick Steinhardt --- list-objects.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/list-objects.c b/list-objects.c index e19589baa0..a5a60301cb 100644 --- a/list-objects.c +++ b/list-objects.c @@ -213,6 +213,14 @@ static void process_tree(struct traversal_context *ctx, free_tree_buffer(tree); } +static void process_tag(struct traversal_context *ctx, + struct tag *tag, + const char *name) +{ + tag->object.flags |= SEEN; + ctx->show_object(&tag->object, name, ctx->show_data); +} + static void mark_edge_parents_uninteresting(struct commit *commit, struct rev_info *revs, show_edge_fn show_edge) @@ -334,8 +342,7 @@ static void traverse_trees_and_blobs(struct traversal_context *ctx, if (obj->flags & (UNINTERESTING | SEEN)) continue; if (obj->type == OBJ_TAG) { - obj->flags |= SEEN; - ctx->show_object(obj, name, ctx->show_data); + process_tag(ctx, (struct tag *)obj, name); continue; } if (!path) From patchwork Fri Apr 9 11:28:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12193841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A900C433ED for ; Fri, 9 Apr 2021 11:28:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DFCBE610F7 for ; Fri, 9 Apr 2021 11:28:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233607AbhDIL2X (ORCPT ); Fri, 9 Apr 2021 07:28:23 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:45531 "EHLO out3-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233603AbhDIL2W (ORCPT ); Fri, 9 Apr 2021 07:28:22 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 9DE4F5C004A; Fri, 9 Apr 2021 07:28:09 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Fri, 09 Apr 2021 07:28:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=5ZWe+vwa0vFxasYpYbMMxErvrte lhGpg0TXOI5MnJZE=; b=E+qF+5kTq0R6oaWyHjvVQ7G0+2TcDyHJANUlw3/tDVI 4LExR0D+tkmKYly8XNu+Wqj056GJMHrqvApDQhbQpwSbUsT2MrK+QkXpAR7we+Hi pEgOlewFLVGTMCwIgteI2pZ3BgNbMNrwBdtsiX+HLpmXVSRj9xIqakJGIOnPjiP7 1mvrnx90egsm2jKcOTeWeZC6ftDWzf6LmJm5RDtaX+WfK17fXgQp7VodYz40x8iO 0lIiuQn9MWX/rCJQVKo9w2LzUUUD6W6lDqJTQaQUFURjp0f1829qqCXOCMVFo9ZL WZMY34LjESZQDXCT/RSbHPyWN6elRg/LVV5eoNh0r1Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=5ZWe+v wa0vFxasYpYbMMxErvrtelhGpg0TXOI5MnJZE=; b=hV73CZFc482QQ42poPZokL Pj/ytBFa2zpDcpjVVsYUvXd5Gf07NbzMKTjzvRKKqlH2+OzNx+0BXNMJWhhyFpKK qw+CycbUV0hex9HIbT/u09DqdzxblXUjIciTHy2SYoDYCXRrapEKmmX7EX9YH9h2 ZVnDBKtDttxWgWXr+LetpWZteuomyFg4LBO++kc6A+9jsUYyUq14tzAjQCl5RBJa sFaslN6CA3psWzlJmmn6gpN4bKfyvRR2mJeUcp4WthxZc1FyIu7y5UF2eVA+e6Mi twEp9+9TqPZMv5OY90goPQHnQGMlQMS1AgmXMBAAK54OBNnU6L5XqUuyhhm5PJEg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekuddggeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeejrdduledurdekrdduieelnecuvehluhhsthgvrhfuihiivgepudenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf08a9.dyn.telefonica.de [77.191.8.169]) by mail.messagingengine.com (Postfix) with ESMTPA id DC5BE240067; Fri, 9 Apr 2021 07:28:08 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id a089389a (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 9 Apr 2021 11:28:08 +0000 (UTC) Date: Fri, 9 Apr 2021 13:28:07 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau Subject: [PATCH v3 4/8] list-objects: support filtering by tag and commit Message-ID: <80193d6ba38254aab3dd2cda39cabf12986fd939.1617967252.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Object filters currently only support filtering blobs or trees based on some criteria. This commit lays the foundation to also allow filtering of tags and commits. No change in behaviour is expected from this commit given that there are no filters yet for those object types. Signed-off-by: Patrick Steinhardt --- list-objects-filter.c | 40 ++++++++++++++++++++++++++++++++++++++++ list-objects-filter.h | 2 ++ list-objects.c | 22 +++++++++++++++++++--- 3 files changed, 61 insertions(+), 3 deletions(-) diff --git a/list-objects-filter.c b/list-objects-filter.c index 39e2f15333..0ebfa52966 100644 --- a/list-objects-filter.c +++ b/list-objects-filter.c @@ -82,6 +82,16 @@ static enum list_objects_filter_result filter_blobs_none( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); /* always include all tree objects */ @@ -173,6 +183,16 @@ static enum list_objects_filter_result filter_trees_depth( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_END_TREE: assert(obj->type == OBJ_TREE); filter_data->current_depth--; @@ -267,6 +287,16 @@ static enum list_objects_filter_result filter_blobs_limit( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); /* always include all tree objects */ @@ -371,6 +401,16 @@ static enum list_objects_filter_result filter_sparse( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); dtype = DT_DIR; diff --git a/list-objects-filter.h b/list-objects-filter.h index cfd784e203..9e98814111 100644 --- a/list-objects-filter.h +++ b/list-objects-filter.h @@ -55,6 +55,8 @@ enum list_objects_filter_result { }; enum list_objects_filter_situation { + LOFS_COMMIT, + LOFS_TAG, LOFS_BEGIN_TREE, LOFS_END_TREE, LOFS_BLOB diff --git a/list-objects.c b/list-objects.c index a5a60301cb..0c524a81ac 100644 --- a/list-objects.c +++ b/list-objects.c @@ -217,8 +217,14 @@ static void process_tag(struct traversal_context *ctx, struct tag *tag, const char *name) { - tag->object.flags |= SEEN; - ctx->show_object(&tag->object, name, ctx->show_data); + enum list_objects_filter_result r; + + r = list_objects_filter__filter_object(ctx->revs->repo, LOFS_TAG, + &tag->object, "", 0, ctx->filter); + if (r & LOFR_MARK_SEEN) + tag->object.flags |= SEEN; + if (r & LOFR_DO_SHOW) + ctx->show_object(&tag->object, name, ctx->show_data); } static void mark_edge_parents_uninteresting(struct commit *commit, @@ -368,6 +374,12 @@ static void do_traverse(struct traversal_context *ctx) strbuf_init(&csp, PATH_MAX); while ((commit = get_revision(ctx->revs)) != NULL) { + enum list_objects_filter_result r; + + r = list_objects_filter__filter_object(ctx->revs->repo, + LOFS_COMMIT, &commit->object, + NULL, NULL, ctx->filter); + /* * an uninteresting boundary commit may not have its tree * parsed yet, but we are not going to show them anyway @@ -382,7 +394,11 @@ static void do_traverse(struct traversal_context *ctx) die(_("unable to load root tree for commit %s"), oid_to_hex(&commit->object.oid)); } - ctx->show_commit(commit, ctx->show_data); + + if (r & LOFR_MARK_SEEN) + commit->object.flags |= SEEN; + if (r & LOFR_DO_SHOW) + ctx->show_commit(commit, ctx->show_data); if (ctx->revs->tree_blobs_in_commit_order) /* From patchwork Fri Apr 9 11:28:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12193843 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CEBDC433B4 for ; Fri, 9 Apr 2021 11:28:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1A0B6610A7 for ; Fri, 9 Apr 2021 11:28:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233569AbhDIL22 (ORCPT ); Fri, 9 Apr 2021 07:28:28 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:49877 "EHLO out3-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233603AbhDIL21 (ORCPT ); Fri, 9 Apr 2021 07:28:27 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 7A5A75C004A; Fri, 9 Apr 2021 07:28:14 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Fri, 09 Apr 2021 07:28:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=NaE5sJtg/a8SZ+Bk4Oi1ZUve2NY 4FXWVwwjyP9hwQdc=; b=h6KTQNb/EDssBXUuoFxhCbMpIDCTrt0zS3ZL0ynk7bq R/bjsvzVLGtPvXBA4/rhlw+BJEYWlpes+OQ7q2FJF7oBzmglTeFUuBkro3bbqrR7 eaVMuz9IkWpZwpupbX+MXkVQ4ixGQdz+pnk6qEKkO3gfdFmzgf9d7g3r/rI39ke0 m0ILF1ran7xBqOlIRm3Hpn1yQjPxKqHRRZZxiBFvqA4k883cot8nw0gJUT8KtDbX YkORUEJgm75NoKIDuQI4o+P2lU+HxTzFfOrY3ZQkbFojx2Fi5geWK5pc70RLQECL PiAB1YSj+EwQ5ur66tZHfqdGIz07G/nOdHKVTlQDLtQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=NaE5sJ tg/a8SZ+Bk4Oi1ZUve2NY4FXWVwwjyP9hwQdc=; b=mctvTlRADMErvjGRlxp7Fd g4wL4Q8LTTWMIew2yoVS48rb/g5sb+8/eBcDN4Jprh8te9tv73+U9TWcUHNh2avi t52jeOT5eA+S/a9tV/VjZBh11a5mJZWQ5S4GQVx0RMQiPUlSYBO0YsgmA8WvgZ/m Jv2LwpLcS0x07WNrFK6yio2jTzgISKOZ3oi/yEP87VFdESaDJNqVQiqlgbczmdeb O34WdX6YzSrssrKeVsBkX9o2COC6byg4Gx1RyFdZMR58Yy7dI8jqsOuA3ch+yuve UGoiGMKbQnc2nWNJdVE52TbkUk07/B0apMJgI2DDVp3szT3srPAHOLd3eecSym3A == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekuddggeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeejrdduledurdekrdduieelnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf08a9.dyn.telefonica.de [77.191.8.169]) by mail.messagingengine.com (Postfix) with ESMTPA id CDB7D24005C; Fri, 9 Apr 2021 07:28:13 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 399ed0e1 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 9 Apr 2021 11:28:12 +0000 (UTC) Date: Fri, 9 Apr 2021 13:28:11 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau Subject: [PATCH v3 5/8] list-objects: implement object type filter Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org While it already is possible to filter objects by some criteria in git-rev-list(1), it is not yet possible to filter out only a specific type of objects. This makes some filters less useful. The `blob:limit` filter for example filters blobs such that only those which are smaller than the given limit are returned. But it is unfit to ask only for these smallish blobs, given that git-rev-list(1) will continue to print tags, commits and trees. Now that we have the infrastructure in place to also filter tags and commits, we can improve this situation by implementing a new filter which selects objects based on their type. Above query can thus trivially be implemented with the following command: $ git rev-list --objects --filter=object:type=blob \ --filter=blob:limit=200 Furthermore, this filter allows to optimize for certain other cases: if for example only tags or commits have been selected, there is no need to walk down trees. The new filter is not yet supported in bitmaps. This is going to be implemented in a subsequent commit. Signed-off-by: Patrick Steinhardt --- Documentation/config/uploadpack.txt | 6 +-- Documentation/rev-list-options.txt | 3 ++ list-objects-filter-options.c | 14 ++++++ list-objects-filter-options.h | 2 + list-objects-filter.c | 76 +++++++++++++++++++++++++++++ t/t6112-rev-list-filters-objects.sh | 48 ++++++++++++++++++ 6 files changed, 146 insertions(+), 3 deletions(-) diff --git a/Documentation/config/uploadpack.txt b/Documentation/config/uploadpack.txt index 6729a072ea..32fad5bbe8 100644 --- a/Documentation/config/uploadpack.txt +++ b/Documentation/config/uploadpack.txt @@ -66,9 +66,9 @@ uploadpackfilter.allow:: uploadpackfilter..allow:: Explicitly allow or ban the object filter corresponding to ``, where `` may be one of: `blob:none`, - `blob:limit`, `tree`, `sparse:oid`, or `combine`. If using - combined filters, both `combine` and all of the nested filter - kinds must be allowed. Defaults to `uploadpackfilter.allow`. + `blob:limit`, `object:type`, `tree`, `sparse:oid`, or `combine`. + If using combined filters, both `combine` and all of the nested + filter kinds must be allowed. Defaults to `uploadpackfilter.allow`. uploadpackfilter.tree.maxDepth:: Only allow `--filter=tree:` when `` is no more than the value of diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index b1c8f86c6e..3afa8fffbd 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -892,6 +892,9 @@ or units. n may be zero. The suffixes k, m, and g can be used to name units in KiB, MiB, or GiB. For example, 'blob:limit=1k' is the same as 'blob:limit=1024'. + +The form '--filter=object:type=(tag|commit|tree|blob)' omits all objects +which are not of the requested type. ++ The form '--filter=sparse:oid=' uses a sparse-checkout specification contained in the blob (or blob-expression) '' to omit blobs that would not be not required for a sparse checkout on diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index d2d1c81caf..bb6f6577d5 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -29,6 +29,8 @@ const char *list_object_filter_config_name(enum list_objects_filter_choice c) return "tree"; case LOFC_SPARSE_OID: return "sparse:oid"; + case LOFC_OBJECT_TYPE: + return "object:type"; case LOFC_COMBINE: return "combine"; case LOFC__COUNT: @@ -97,6 +99,18 @@ static int gently_parse_list_objects_filter( } return 1; + } else if (skip_prefix(arg, "object:type=", &v0)) { + int type = type_from_string_gently(v0, -1, 1); + if (type < 0) { + strbuf_addstr(errbuf, _("expected 'object:type='")); + return 1; + } + + filter_options->object_type = type; + filter_options->choice = LOFC_OBJECT_TYPE; + + return 0; + } else if (skip_prefix(arg, "combine:", &v0)) { return parse_combine_filter(filter_options, v0, errbuf); diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h index 01767c3c96..4d0d0588cc 100644 --- a/list-objects-filter-options.h +++ b/list-objects-filter-options.h @@ -13,6 +13,7 @@ enum list_objects_filter_choice { LOFC_BLOB_LIMIT, LOFC_TREE_DEPTH, LOFC_SPARSE_OID, + LOFC_OBJECT_TYPE, LOFC_COMBINE, LOFC__COUNT /* must be last */ }; @@ -54,6 +55,7 @@ struct list_objects_filter_options { char *sparse_oid_name; unsigned long blob_limit_value; unsigned long tree_exclude_depth; + enum object_type object_type; /* LOFC_COMBINE values */ diff --git a/list-objects-filter.c b/list-objects-filter.c index 0ebfa52966..1c1ee3d1bb 100644 --- a/list-objects-filter.c +++ b/list-objects-filter.c @@ -545,6 +545,81 @@ static void filter_sparse_oid__init( filter->free_fn = filter_sparse_free; } +/* + * A filter for list-objects to omit large blobs. + * And to OPTIONALLY collect a list of the omitted OIDs. + */ +struct filter_object_type_data { + enum object_type object_type; +}; + +static enum list_objects_filter_result filter_object_type( + struct repository *r, + enum list_objects_filter_situation filter_situation, + struct object *obj, + const char *pathname, + const char *filename, + struct oidset *omits, + void *filter_data_) +{ + struct filter_object_type_data *filter_data = filter_data_; + + switch (filter_situation) { + default: + BUG("unknown filter_situation: %d", filter_situation); + + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + if (filter_data->object_type == OBJ_TAG) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + return LOFR_MARK_SEEN; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + if (filter_data->object_type == OBJ_COMMIT) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + return LOFR_MARK_SEEN; + + case LOFS_BEGIN_TREE: + assert(obj->type == OBJ_TREE); + + /* + * If we only want to show commits or tags, then there is no + * need to walk down trees. + */ + if (filter_data->object_type == OBJ_COMMIT || + filter_data->object_type == OBJ_TAG) + return LOFR_SKIP_TREE; + + if (filter_data->object_type == OBJ_TREE) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + return LOFR_MARK_SEEN; + + case LOFS_BLOB: + assert(obj->type == OBJ_BLOB); + + if (filter_data->object_type == OBJ_BLOB) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + return LOFR_MARK_SEEN; + + case LOFS_END_TREE: + return LOFR_ZERO; + } +} + +static void filter_object_type__init( + struct list_objects_filter_options *filter_options, + struct filter *filter) +{ + struct filter_object_type_data *d = xcalloc(1, sizeof(*d)); + d->object_type = filter_options->object_type; + + filter->filter_data = d; + filter->filter_object_fn = filter_object_type; + filter->free_fn = free; +} + /* A filter which only shows objects shown by all sub-filters. */ struct combine_filter_data { struct subfilter *sub; @@ -691,6 +766,7 @@ static filter_init_fn s_filters[] = { filter_blobs_limit__init, filter_trees_depth__init, filter_sparse_oid__init, + filter_object_type__init, filter_combine__init, }; diff --git a/t/t6112-rev-list-filters-objects.sh b/t/t6112-rev-list-filters-objects.sh index 31457d13b9..c79ec04060 100755 --- a/t/t6112-rev-list-filters-objects.sh +++ b/t/t6112-rev-list-filters-objects.sh @@ -159,6 +159,54 @@ test_expect_success 'verify blob:limit=1m' ' test_must_be_empty observed ' +# Test object:type= filter. + +test_expect_success 'setup object-type' ' + git init object-type && + echo contents >object-type/blob && + git -C object-type add blob && + git -C object-type commit -m commit-message && + git -C object-type tag tag -m tag-message +' + +test_expect_success 'verify object:type= fails with invalid type' ' + test_must_fail git -C object-type rev-list --objects --filter=object:type= HEAD && + test_must_fail git -C object-type rev-list --objects --filter=object:type=invalid HEAD +' + +test_expect_success 'verify object:type=blob prints blob and commit' ' + ( + git -C object-type rev-parse HEAD && + printf "%s blob\n" $(git -C object-type rev-parse HEAD:blob) + ) >expected && + git -C object-type rev-list --objects --filter=object:type=blob HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tree prints tree and commit' ' + ( + git -C object-type rev-parse HEAD && + printf "%s \n" $(git -C object-type rev-parse HEAD^{tree}) + ) >expected && + git -C object-type rev-list --objects --filter=object:type=tree HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=commit prints commit' ' + git -C object-type rev-parse HEAD >expected && + git -C object-type rev-list --objects --filter=object:type=commit HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tag prints tag' ' + ( + git -C object-type rev-parse HEAD && + printf "%s tag\n" $(git -C object-type rev-parse tag) + ) >expected && + git -C object-type rev-list --objects --filter=object:type=tag tag >actual && + test_cmp expected actual +' + # Test sparse:path= filter. # !!!! # NOTE: sparse:path filter support has been dropped for security reasons, From patchwork Fri Apr 9 11:28:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12193845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 220AFC433ED for ; Fri, 9 Apr 2021 11:28:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F04E761108 for ; Fri, 9 Apr 2021 11:28:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233505AbhDIL2d (ORCPT ); Fri, 9 Apr 2021 07:28:33 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:55429 "EHLO out3-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233629AbhDIL2c (ORCPT ); Fri, 9 Apr 2021 07:28:32 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 75B6C5C00B5; Fri, 9 Apr 2021 07:28:19 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Fri, 09 Apr 2021 07:28:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=57YpcIAe/DWtOj5HCNhv4xd/9yD 2lFb51gVa2UMeasU=; b=sAk8Ab4vqdyg8EzmEQDW7i0gH2hgX75S3VX7V8iECbG FQrd0PZyTAhgrY+Jvz0CUR0yltfPrP3XvDb4A1nTYqprPthQ0aEgBqIRsh17F5Lp D6m315pWYvgdSi6f2JbSMeKKFitKDsthkCdMVhHs0953RRFWuyTjKFq1Ny3O5ZeJ G+lcSDQuaL7VulpDF6uBLz7Aa3v8dU+o3ZqvcoOvFU0VpenCweejUEMdknh5gLeB 4prqzTOOeWma4+n6OUkLolzwxRSLMVON+NLlNCpcn/Ks80IBdakEFg7UVfehfUjG Wq97y6rqFqf+nqvOZQxp08oA65FKpxtZ5Uj1dj3sJIw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=57YpcI Ae/DWtOj5HCNhv4xd/9yD2lFb51gVa2UMeasU=; b=EMq9N6efmCmQIls+yDZRPB rLL2aFGY10qUzQAo92UzreMRElcnc76+vuFUXcvaY2IywrgtFBGY8S9wMKiqdcPw hTr4QllLIb9S4pubbrGfCZdYZja5ZF4M18UDwy4KCp5BbWWt/9SIq9i3NKb6FOkg eIKcoWQQgrzKkYKb5BdHCuM8DMN8ijcps585vnLAl/SEvYRUxPX5voBttTL1J95+ Dhmn/rPRALtCV/TpyFo62BMMYRd0Y7hoIfD32ReRgYU9WlC1bBkF1pppSt/CR4rR ZZS0VRtUsv3pdP7dFYx+++cEhDl5PfJjD+4bUTQ8aWCh/iQjjNwjOB0BVO0qldNw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekuddggeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeejrdduledurdekrdduieelnecuvehluhhsthgvrhfuihiivgepudenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf08a9.dyn.telefonica.de [77.191.8.169]) by mail.messagingengine.com (Postfix) with ESMTPA id A6713240065; Fri, 9 Apr 2021 07:28:18 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 3ff22a19 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 9 Apr 2021 11:28:17 +0000 (UTC) Date: Fri, 9 Apr 2021 13:28:16 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau Subject: [PATCH v3 6/8] pack-bitmap: implement object type filter Message-ID: <46d4450d38231d1af0b17c4855b9671ad3446aef.1617967252.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The preceding commit has added a new object filter for git-rev-list(1) which allows to filter objects by type. Implement the equivalent filter for packfile bitmaps so that we can answer these queries fast. Signed-off-by: Patrick Steinhardt --- pack-bitmap.c | 29 ++++++++++++++++++++++++++--- t/t6113-rev-list-bitmap-filters.sh | 25 ++++++++++++++++++++++++- 2 files changed, 50 insertions(+), 4 deletions(-) diff --git a/pack-bitmap.c b/pack-bitmap.c index b4513f8672..cd3f5c433e 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -779,9 +779,6 @@ static void filter_bitmap_exclude_type(struct bitmap_index *bitmap_git, eword_t mask; uint32_t i; - if (type != OBJ_BLOB && type != OBJ_TREE) - BUG("filter_bitmap_exclude_type: unsupported type '%d'", type); - /* * The non-bitmap version of this filter never removes * objects which the other side specifically asked for, @@ -911,6 +908,24 @@ static void filter_bitmap_tree_depth(struct bitmap_index *bitmap_git, OBJ_BLOB); } +static void filter_bitmap_object_type(struct bitmap_index *bitmap_git, + struct object_list *tip_objects, + struct bitmap *to_filter, + enum object_type object_type) +{ + if (object_type < OBJ_COMMIT || object_type > OBJ_TAG) + BUG("filter_bitmap_object_type given invalid object"); + + if (object_type != OBJ_TAG) + filter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_TAG); + if (object_type != OBJ_COMMIT) + filter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_COMMIT); + if (object_type != OBJ_TREE) + filter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_TREE); + if (object_type != OBJ_BLOB) + filter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_BLOB); +} + static int filter_bitmap(struct bitmap_index *bitmap_git, struct object_list *tip_objects, struct bitmap *to_filter, @@ -943,6 +958,14 @@ static int filter_bitmap(struct bitmap_index *bitmap_git, return 0; } + if (filter->choice == LOFC_OBJECT_TYPE) { + if (bitmap_git) + filter_bitmap_object_type(bitmap_git, tip_objects, + to_filter, + filter->object_type); + return 0; + } + /* filter choice not handled */ return -1; } diff --git a/t/t6113-rev-list-bitmap-filters.sh b/t/t6113-rev-list-bitmap-filters.sh index 3f889949ca..fb66735ac8 100755 --- a/t/t6113-rev-list-bitmap-filters.sh +++ b/t/t6113-rev-list-bitmap-filters.sh @@ -10,7 +10,8 @@ test_expect_success 'set up bitmapped repo' ' test_commit much-larger-blob-one && git repack -adb && test_commit two && - test_commit much-larger-blob-two + test_commit much-larger-blob-two && + git tag tag ' test_expect_success 'filters fallback to non-bitmap traversal' ' @@ -75,4 +76,26 @@ test_expect_success 'tree:1 filter' ' test_cmp expect actual ' +test_expect_success 'object:type filter' ' + git rev-list --objects --filter=object:type=tag tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=tag tag >actual && + test_cmp expect actual && + + git rev-list --objects --filter=object:type=commit tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=commit tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter=object:type=tree tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=tree tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual +' + test_done From patchwork Fri Apr 9 11:28:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12193847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79CA6C433ED for ; Fri, 9 Apr 2021 11:28:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 53A9C610F7 for ; Fri, 9 Apr 2021 11:28:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233633AbhDIL2i (ORCPT ); Fri, 9 Apr 2021 07:28:38 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:41925 "EHLO out3-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233603AbhDIL2g (ORCPT ); Fri, 9 Apr 2021 07:28:36 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 5ADF75C0097; Fri, 9 Apr 2021 07:28:23 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Fri, 09 Apr 2021 07:28:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=7QVbyRXVMdH4GypcTjyJOGafoQr l6wqozT3DJiP2AIU=; b=x3f8aRCklopQpoHKUTfR1m+9vpsMqv9YYi9npiqYF45 wCz1JZALbH4pjBl2UGSj9c7gqX3YuMbAc+fNUJVnTQ0UYbk+s/qbHJ7r7RIbkp5T K2VNQrnNBEwpTWJ/r613QIHUQ5QLAm8G7CEz3PmM0R35m4eOcMOBOJYIYtPpmYe0 mdMqVdX/dKAljUcRyp4CAT5xU3+MrK0E/vPdDo6IqPhW3JgsKQ8PsaOfGFsaNZ4N ExwMGt0n6k2qTqJ6stPdvoF4e9Y6Lznu3rGbp6dJULaVq7wDFhG+iaqvo+xyGTF3 H2nn+VBPfuWv1M/e7fgufzu7rUEwSHsHr7gh1wPFDsA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=7QVbyR XVMdH4GypcTjyJOGafoQrl6wqozT3DJiP2AIU=; b=GKjMkVcWQ9EM/cKPoapQYS jSs96t0mF/JNnx4eREeN9O9oVJEN4PVOUdTc3n1qLpME6Qcd0rsjvTO5/ZOWMPqB G58kJzw14vd5YQAcCPSl3xElMcXr40QziuhCtgoyfO4/pXge+5AeWl8xYF9U2cNg 0OPpKU66L8+dcI5/rnH7DY7eG05kH97n4WtVF28Z4tgA5cU2kHw96G3M8+RODILC id9pyja6jg7LIfUrjG7LGTXkLqAua65xpTPrUOpwCAQTFktbC8Bk7DLOaww2W96f aEvPEDkN9MSoOBBtaQRGshgVaVheSn1QEVwYi6IKDzt3dcSfHTXO3jpllyTiId0A == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekuddggeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeejrdduledurdekrdduieelnecuvehluhhsthgvrhfuihiivgepvdenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf08a9.dyn.telefonica.de [77.191.8.169]) by mail.messagingengine.com (Postfix) with ESMTPA id A629C240068; Fri, 9 Apr 2021 07:28:22 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id f1f23e66 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 9 Apr 2021 11:28:22 +0000 (UTC) Date: Fri, 9 Apr 2021 13:28:21 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau Subject: [PATCH v3 7/8] pack-bitmap: implement combined filter Message-ID: <06a376399bcce0caed993fd2e1a9cff5a57ce502.1617967252.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When the user has multiple objects filters specified, then this is internally represented by having a "combined" filter. These combined filters aren't yet supported by bitmap indices and can thus not be accelerated. Fix this by implementing support for these combined filters. The implementation is quite trivial: when there's a combined filter, we simply recurse into `filter_bitmap()` for all of the sub-filters. Signed-off-by: Patrick Steinhardt --- pack-bitmap.c | 41 +++++++++++++++++++++++++++--- t/t6113-rev-list-bitmap-filters.sh | 7 +++++ 2 files changed, 44 insertions(+), 4 deletions(-) diff --git a/pack-bitmap.c b/pack-bitmap.c index cd3f5c433e..4385f15828 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -926,6 +926,29 @@ static void filter_bitmap_object_type(struct bitmap_index *bitmap_git, filter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_BLOB); } +static int filter_supported(struct list_objects_filter_options *filter) +{ + int i; + + switch (filter->choice) { + case LOFC_BLOB_NONE: + case LOFC_BLOB_LIMIT: + case LOFC_OBJECT_TYPE: + return 1; + case LOFC_TREE_DEPTH: + if (filter->tree_exclude_depth == 0) + return 1; + return 0; + case LOFC_COMBINE: + for (i = 0; i < filter->sub_nr; i++) + if (!filter_supported(&filter->sub[i])) + return 0; + return 1; + default: + return 0; + } +} + static int filter_bitmap(struct bitmap_index *bitmap_git, struct object_list *tip_objects, struct bitmap *to_filter, @@ -933,6 +956,8 @@ static int filter_bitmap(struct bitmap_index *bitmap_git, { if (!filter || filter->choice == LOFC_DISABLED) return 0; + if (!filter_supported(filter)) + return -1; if (filter->choice == LOFC_BLOB_NONE) { if (bitmap_git) @@ -949,8 +974,7 @@ static int filter_bitmap(struct bitmap_index *bitmap_git, return 0; } - if (filter->choice == LOFC_TREE_DEPTH && - filter->tree_exclude_depth == 0) { + if (filter->choice == LOFC_TREE_DEPTH) { if (bitmap_git) filter_bitmap_tree_depth(bitmap_git, tip_objects, to_filter, @@ -966,8 +990,17 @@ static int filter_bitmap(struct bitmap_index *bitmap_git, return 0; } - /* filter choice not handled */ - return -1; + if (filter->choice == LOFC_COMBINE) { + int i; + for (i = 0; i < filter->sub_nr; i++) { + if (filter_bitmap(bitmap_git, tip_objects, to_filter, + &filter->sub[i]) < 0) + return -1; + } + return 0; + } + + BUG("unsupported filter choice"); } static int can_filter_bitmap(struct list_objects_filter_options *filter) diff --git a/t/t6113-rev-list-bitmap-filters.sh b/t/t6113-rev-list-bitmap-filters.sh index fb66735ac8..cb9db7df6f 100755 --- a/t/t6113-rev-list-bitmap-filters.sh +++ b/t/t6113-rev-list-bitmap-filters.sh @@ -98,4 +98,11 @@ test_expect_success 'object:type filter' ' test_bitmap_traversal expect actual ' +test_expect_success 'combine filter' ' + git rev-list --objects --filter=blob:limit=1000 --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=blob:limit=1000 --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual +' + test_done From patchwork Fri Apr 9 11:28:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12193849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36A60C433ED for ; Fri, 9 Apr 2021 11:28:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0DE7B610A7 for ; Fri, 9 Apr 2021 11:28:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231819AbhDIL2o (ORCPT ); Fri, 9 Apr 2021 07:28:44 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:60141 "EHLO out3-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233747AbhDIL2l (ORCPT ); Fri, 9 Apr 2021 07:28:41 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 514AF5C0037; Fri, 9 Apr 2021 07:28:28 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Fri, 09 Apr 2021 07:28:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=EHDIHRhopAsgVBFyC4Fl6hSKF8/ RqVzcpWs2ki5tH0Y=; b=I16hgE3WbGm7Xk/wpoI4UUGDpT1229r08mL2TyEEmEe jOgULrG5+iXmX6XfajkrGxr5N9A+T66XggA1AoIdGMBqYlUCiWb13Fy0w2Ia+N9K 4/KeiwZSu1T4v0X1QnzTngJp0AoVhpX3A1/P7LwvhNYlRNSWrARm1IYNivcCQ57l SKzynlQIMuifhOOA04GKqYIBGVEKz90xg5QLdYmzSO/HGHSxmbXEfnEOpwlOzNLK deBzDsNQPEh8GwLkz2JIWQsOYRQPLv2WDRDXL7HNpHrwux2oy+Pqz7fAfJPJUbFr 7b/ektZ8rkSDNy9SK9buc3QH45C4sJP1r6RGQHHbs9Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=EHDIHR hopAsgVBFyC4Fl6hSKF8/RqVzcpWs2ki5tH0Y=; b=CbvXAPS1SzEsXfGnvBvjQh xpEbkb9OVHTVoIyThLwqHSpC8Op8ujLzUz0cOhN72+1z7qyv/Sn4f7/O/mYWzPW0 ZS0hjNASfghwFWiJvYmr3v3Ddeukc6pOqfPYwUsmxe8nnqlXdXyuEzy2akNTEfYB i1zOnNCe+Y50cIXiqURUfAFowMs2xSf2F1rXbXqgaK2rMZ9ioVRIfanjuSqmIDdp SK/L7ZL655+CNXRybLSW/6R2lrUuM8Wmw8Mtk+ajCiuDCNx4de56MjSO5VfNcgZE u+oawR8KJeq8AQOMXcrc3HVhGMfr6WUAhcpRIZt9pdgi2lCzM4pvrn+UtjevRdCw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekuddggeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpefgfeejuefgheegueeljeffteefuedthfdvvedugfevvdffleduueeiiefgvedtheen ucffohhmrghinhepphgvnhguihhnghdrnhhrnecukfhppeejjedrudeluddrkedrudeile enucevlhhushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhrohhmpehpshes phhkshdrihhm X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf08a9.dyn.telefonica.de [77.191.8.169]) by mail.messagingengine.com (Postfix) with ESMTPA id 8AD7E240067; Fri, 9 Apr 2021 07:28:27 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id f6696864 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 9 Apr 2021 11:28:26 +0000 (UTC) Date: Fri, 9 Apr 2021 13:28:25 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau Subject: [PATCH v3 8/8] rev-list: allow filtering of provided items Message-ID: <796606f32b3ffc286a3157312d00ee0ee3e5600c.1617967252.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When providing an object filter, it is currently impossible to also filter provided items. E.g. when executing `git rev-list HEAD` , the commit this reference points to will be treated as user-provided and is thus excluded from the filtering mechanism. This makes it harder than necessary to properly use the new `--filter=object:type` filter given that even if the user wants to only see blobs, he'll still see commits of provided references. Improve this by introducing a new `--filter-provided` option to the git-rev-parse(1) command. If given, then all user-provided references will be subject to filtering. Signed-off-by: Patrick Steinhardt --- Documentation/rev-list-options.txt | 5 ++++ builtin/pack-objects.c | 2 +- builtin/rev-list.c | 36 +++++++++++++++++++++-------- pack-bitmap.c | 6 +++-- pack-bitmap.h | 3 ++- reachable.c | 2 +- t/t6112-rev-list-filters-objects.sh | 28 ++++++++++++++++++++++ t/t6113-rev-list-bitmap-filters.sh | 36 +++++++++++++++++++++++++++++ 8 files changed, 104 insertions(+), 14 deletions(-) diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index 3afa8fffbd..7fa18fc6e6 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -933,6 +933,11 @@ equivalent. --no-filter:: Turn off any previous `--filter=` argument. +--filter-provided-revisions:: + Filter the list of explicitly provided revisions, which would otherwise + always be printed even if they did not match any of the filters. Only + useful with `--filter=`. + --filter-print-omitted:: Only useful with `--filter=`; prints a list of the objects omitted by the filter. Object IDs are prefixed with a ``~'' character. diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 525c2d8552..2f2026dc87 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -3516,7 +3516,7 @@ static int pack_options_allow_reuse(void) static int get_object_list_from_bitmap(struct rev_info *revs) { - if (!(bitmap_git = prepare_bitmap_walk(revs, &filter_options))) + if (!(bitmap_git = prepare_bitmap_walk(revs, &filter_options, 0))) return -1; if (pack_options_allow_reuse() && diff --git a/builtin/rev-list.c b/builtin/rev-list.c index b4d8ea0a35..13f0ff3f8d 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -398,7 +398,8 @@ static inline int parse_missing_action_value(const char *value) } static int try_bitmap_count(struct rev_info *revs, - struct list_objects_filter_options *filter) + struct list_objects_filter_options *filter, + int filter_provided_revs) { uint32_t commit_count = 0, tag_count = 0, @@ -433,7 +434,7 @@ static int try_bitmap_count(struct rev_info *revs, */ max_count = revs->max_count; - bitmap_git = prepare_bitmap_walk(revs, filter); + bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_revs); if (!bitmap_git) return -1; @@ -450,7 +451,8 @@ static int try_bitmap_count(struct rev_info *revs, } static int try_bitmap_traversal(struct rev_info *revs, - struct list_objects_filter_options *filter) + struct list_objects_filter_options *filter, + int filter_provided_revs) { struct bitmap_index *bitmap_git; @@ -461,7 +463,7 @@ static int try_bitmap_traversal(struct rev_info *revs, if (revs->max_count >= 0) return -1; - bitmap_git = prepare_bitmap_walk(revs, filter); + bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_revs); if (!bitmap_git) return -1; @@ -471,14 +473,15 @@ static int try_bitmap_traversal(struct rev_info *revs, } static int try_bitmap_disk_usage(struct rev_info *revs, - struct list_objects_filter_options *filter) + struct list_objects_filter_options *filter, + int filter_provided_revs) { struct bitmap_index *bitmap_git; if (!show_disk_usage) return -1; - bitmap_git = prepare_bitmap_walk(revs, filter); + bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_revs); if (!bitmap_git) return -1; @@ -499,6 +502,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) int bisect_show_vars = 0; int bisect_find_all = 0; int use_bitmap_index = 0; + int filter_provided_revs = 0; const char *show_progress = NULL; if (argc == 2 && !strcmp(argv[1], "-h")) @@ -599,6 +603,10 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) list_objects_filter_set_no_filter(&filter_options); continue; } + if (!strcmp(arg, "--filter-provided-revisions")) { + filter_provided_revs = 1; + continue; + } if (!strcmp(arg, "--filter-print-omitted")) { arg_print_omitted = 1; continue; @@ -665,11 +673,11 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) progress = start_delayed_progress(show_progress, 0); if (use_bitmap_index) { - if (!try_bitmap_count(&revs, &filter_options)) + if (!try_bitmap_count(&revs, &filter_options, filter_provided_revs)) return 0; - if (!try_bitmap_disk_usage(&revs, &filter_options)) + if (!try_bitmap_disk_usage(&revs, &filter_options, filter_provided_revs)) return 0; - if (!try_bitmap_traversal(&revs, &filter_options)) + if (!try_bitmap_traversal(&revs, &filter_options, filter_provided_revs)) return 0; } @@ -694,6 +702,16 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) return show_bisect_vars(&info, reaches, all); } + if (filter_provided_revs) { + struct commit_list *c; + for (i = 0; i < revs.pending.nr; i++) { + struct object_array_entry *pending = revs.pending.objects + i; + pending->item->flags |= NOT_USER_GIVEN; + } + for (c = revs.commits; c; c = c->next) + c->item->object.flags |= NOT_USER_GIVEN; + } + if (arg_print_omitted) oidset_init(&omitted_objects, DEFAULT_OIDSET_SIZE); if (arg_missing_action == MA_PRINT) diff --git a/pack-bitmap.c b/pack-bitmap.c index 4385f15828..0576a19a28 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -1009,7 +1009,8 @@ static int can_filter_bitmap(struct list_objects_filter_options *filter) } struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, - struct list_objects_filter_options *filter) + struct list_objects_filter_options *filter, + int filter_provided_revs) { unsigned int i; @@ -1104,7 +1105,8 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, if (haves_bitmap) bitmap_and_not(wants_bitmap, haves_bitmap); - filter_bitmap(bitmap_git, wants, wants_bitmap, filter); + filter_bitmap(bitmap_git, (filter && filter_provided_revs) ? NULL : wants, + wants_bitmap, filter); bitmap_git->result = wants_bitmap; bitmap_git->haves = haves_bitmap; diff --git a/pack-bitmap.h b/pack-bitmap.h index 36d99930d8..5d8ae3b590 100644 --- a/pack-bitmap.h +++ b/pack-bitmap.h @@ -50,7 +50,8 @@ void traverse_bitmap_commit_list(struct bitmap_index *, show_reachable_fn show_reachable); void test_bitmap_walk(struct rev_info *revs); struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, - struct list_objects_filter_options *filter); + struct list_objects_filter_options *filter, + int filter_provided_revs); int reuse_partial_packfile_from_bitmap(struct bitmap_index *, struct packed_git **packfile, uint32_t *entries, diff --git a/reachable.c b/reachable.c index 77a60c70a5..fc833cae43 100644 --- a/reachable.c +++ b/reachable.c @@ -223,7 +223,7 @@ void mark_reachable_objects(struct rev_info *revs, int mark_reflog, cp.progress = progress; cp.count = 0; - bitmap_git = prepare_bitmap_walk(revs, NULL); + bitmap_git = prepare_bitmap_walk(revs, NULL, 0); if (bitmap_git) { traverse_bitmap_commit_list(bitmap_git, revs, mark_object_seen); free_bitmap_index(bitmap_git); diff --git a/t/t6112-rev-list-filters-objects.sh b/t/t6112-rev-list-filters-objects.sh index c79ec04060..47c558ab0e 100755 --- a/t/t6112-rev-list-filters-objects.sh +++ b/t/t6112-rev-list-filters-objects.sh @@ -207,6 +207,34 @@ test_expect_success 'verify object:type=tag prints tag' ' test_cmp expected actual ' +test_expect_success 'verify object:type=blob prints only blob with --filter-provided' ' + printf "%s blob\n" $(git -C object-type rev-parse HEAD:blob) >expected && + git -C object-type rev-list --objects \ + --filter=object:type=blob --filter-provided HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tree prints only tree with --filter-provided' ' + printf "%s \n" $(git -C object-type rev-parse HEAD^{tree}) >expected && + git -C object-type rev-list --objects \ + --filter=object:type=tree HEAD --filter-provided >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=commit prints only commit with --filter-provided' ' + git -C object-type rev-parse HEAD >expected && + git -C object-type rev-list --objects \ + --filter=object:type=commit --filter-provided HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tag prints only tag with --filter-provided' ' + printf "%s tag\n" $(git -C object-type rev-parse tag) >expected && + git -C object-type rev-list --objects \ + --filter=object:type=tag --filter-provided tag >actual && + test_cmp expected actual +' + # Test sparse:path= filter. # !!!! # NOTE: sparse:path filter support has been dropped for security reasons, diff --git a/t/t6113-rev-list-bitmap-filters.sh b/t/t6113-rev-list-bitmap-filters.sh index cb9db7df6f..9053ac5059 100755 --- a/t/t6113-rev-list-bitmap-filters.sh +++ b/t/t6113-rev-list-bitmap-filters.sh @@ -98,6 +98,28 @@ test_expect_success 'object:type filter' ' test_bitmap_traversal expect actual ' +test_expect_success 'object:type filter with --filter-provided' ' + git rev-list --objects --filter-provided --filter=object:type=tag tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided --filter=object:type=tag tag >actual && + test_cmp expect actual && + + git rev-list --objects --filter-provided --filter=object:type=commit tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided --filter=object:type=commit tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter-provided --filter=object:type=tree tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided --filter=object:type=tree tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter-provided --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual +' + test_expect_success 'combine filter' ' git rev-list --objects --filter=blob:limit=1000 --filter=object:type=blob tag >expect && git rev-list --use-bitmap-index \ @@ -105,4 +127,18 @@ test_expect_success 'combine filter' ' test_bitmap_traversal expect actual ' +test_expect_success 'combine filter with --filter-provided' ' + git rev-list --objects --filter-provided --filter=blob:limit=1000 --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided --filter=blob:limit=1000 --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual && + + git cat-file --batch-check="%(objecttype) %(objectsize)" objects && + while read objecttype objectsize + do + test "$objecttype" = blob || return 1 + test "$objectsize" -le 1000 || return 1 + done