From patchwork Mon Apr 19 11:46:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12211649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FB78C433ED for ; Mon, 19 Apr 2021 11:46:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 57E6361090 for ; Mon, 19 Apr 2021 11:46:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238751AbhDSLrO (ORCPT ); Mon, 19 Apr 2021 07:47:14 -0400 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:39387 "EHLO wout1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238856AbhDSLrJ (ORCPT ); Mon, 19 Apr 2021 07:47:09 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id A8505B01; Mon, 19 Apr 2021 07:46:39 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 19 Apr 2021 07:46:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=K4JgZnJdYfycPHrP9xfCggxiDsh 7q22Nb6DqqR/ypmA=; b=nqlR5KcRcFvE2H0O5hXlQ17J0Z0KQ2S0HxYWFIl/rAA 8D+QXVlSrD0hie0j7Mp2nvuvNQ5YBS1/iaSFHX4ofUnUlzOB+r/z1oF6Esyh1XdD glRO5QH6diETHPMAx0WODW6vRHTKtUwMtvI2/E22VA+rTAIOvA8S2OSpFGfHWJmq ZsgYqWLe2AntPAxFxNdLSUmsWvqeOwXpm4yQLKzQg+d+RxA/jLdT5JlQtTPGwvPP ZvLrXUvq8wlnmciSiTTHPSU5i49iREWP7eRVe81aRdEpo94CqVb4+IL2gXu2Xi78 WwtCfgLdoNQGL1nxdiBz4qsQhHg5Vn5TcOwF2BoaL9w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=K4JgZn JdYfycPHrP9xfCggxiDsh7q22Nb6DqqR/ypmA=; b=Ad6eqbep6gkyi8zDYJO/xJ hFb5Av0s0HD4cd3wlgpnNIZnixX5uA0de2tDAzsC/pdPtfT/HJ0IVxLPBY6Aljj2 W4x7UpJKWH89LXXP6PFw56b+OjavXamVPqHtbv4QJbTYW8tTurjaFV8zj20g9hY+ 7NBQqpsZustTg+SQOmP9ivbLCtXpmGRaM7I+GOrhvyy9oJxuOc60ifJ+B2Nlb2/Q yh3toQn6GHMqJFokvANMSiXa+b9EbcM08abUbz1idGULN+6ybt7jjrLwEDo5kTnN Bn6E3kS/IhfupEkJRpVf+9hQe+Yol10KEjtLejGbQnQ8z5Kvw3vPrynrmGVkg7cQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvddtgedggeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeejrdduledurdeikedrkedtnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf4450.dyn.telefonica.de [77.191.68.80]) by mail.messagingengine.com (Postfix) with ESMTPA id 5E213108006B; Mon, 19 Apr 2021 07:46:38 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 7887ba6d (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 19 Apr 2021 11:46:37 +0000 (UTC) Date: Mon, 19 Apr 2021 13:46:36 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley , Ramsay Jones , =?iso-8859-1?q?=C6var_Arnfj=F6r?= =?iso-8859-1?q?=F0?= Bjarmason Subject: [PATCH v5 1/8] uploadpack.txt: document implication of `uploadpackfilter.allow` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When `uploadpackfilter.allow` is set to `true`, it means that filters are enabled by default except in the case where a filter is explicitly disabled via `uploadpackilter..allow`. This option will not only enable the currently supported set of filters, but also any filters which get added in the future. As such, an admin which wants to have tight control over which filters are allowed and which aren't probably shouldn't ever set `uploadpackfilter.allow=true`. Amend the documentation to make the ramifications more explicit so that admins are aware of this. Signed-off-by: Patrick Steinhardt --- Documentation/config/uploadpack.txt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Documentation/config/uploadpack.txt b/Documentation/config/uploadpack.txt index b0d761282c..6729a072ea 100644 --- a/Documentation/config/uploadpack.txt +++ b/Documentation/config/uploadpack.txt @@ -59,7 +59,8 @@ uploadpack.allowFilter:: uploadpackfilter.allow:: Provides a default value for unspecified object filters (see: the - below configuration variable). + below configuration variable). If set to `true`, this will also + enable all filters which get added in the future. Defaults to `true`. uploadpackfilter..allow:: From patchwork Mon Apr 19 11:46:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12211651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE556C433B4 for ; Mon, 19 Apr 2021 11:46:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 91DE26108B for ; Mon, 19 Apr 2021 11:46:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238869AbhDSLrR (ORCPT ); Mon, 19 Apr 2021 07:47:17 -0400 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:45009 "EHLO wout1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238870AbhDSLrP (ORCPT ); Mon, 19 Apr 2021 07:47:15 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id D06E3E1B; Mon, 19 Apr 2021 07:46:44 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 19 Apr 2021 07:46:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=eIi1YD+80dWcV6ejewL8Hpbj5EU wA9aIPMo/vh1KK4E=; b=HVDWJhFJD75m8v7hAXjGE6KYQCTXGMK62yYo7ojktWP DyLQi5mTkxPFJbcjttej7B3tVzukh+O0hBZYzFufhGBADZL3FNTb0i375IeNyHKi kD+r3Tc1JP4uv7NvGL91hfUkhrP4CeXVvtn51lgBU1XSr5HovKlV6LwFYI9niwgl 8VyhPkyotSTchOJrn1AfasXeWQxp4gxBM7CWCJ0G0PF13ZSwOZcHynO1HZSn0o14 I523OmtZ8f++vXKDNRi+GeeMlSWV71PPEQADRwSe4SAta9QSCZgNU/EZq/hicr46 4ioiUndoAbY62JVLOv2Y9wtp6s6ho9qkWzfMPPM9Y6A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=eIi1YD +80dWcV6ejewL8Hpbj5EUwA9aIPMo/vh1KK4E=; b=Hq0NTXIwD6qoibfANCFCVM fTq0tAmnnHeTYP9hQRy4TOg4cFTj1kz5Fo2R+CQ+ASWfoDjeFvJ5B10d6+TwxUgn O8170dZK406MLTvhhqx3zT2d17BpXniP6fOr9TCNdC07j733hCp9yo42dLu42FAV Ok30PDsdZvTsx+AdIoIygqOks4Au0aKgA8ycNvnTi/QGs28SF9GJn8DuUgBDmwwc 74gphc4Z0Fbe4tJU2bFZI8ILlkeFpKlpDxPrTcZ9Av/vhT2Hq+1+AYBcgND+gQWd ebTu9xLCMZLY5U/Go0Vey/e/R9ALtgFqatUrDH3oe1xI3Bd45Qjnc+5So+1GhW/g == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvddtgedggeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeejrdduledurdeikedrkedtnecuvehluhhsthgvrhfuihiivgepvdenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf4450.dyn.telefonica.de [77.191.68.80]) by mail.messagingengine.com (Postfix) with ESMTPA id 650F11080064; Mon, 19 Apr 2021 07:46:43 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 900dae38 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 19 Apr 2021 11:46:42 +0000 (UTC) Date: Mon, 19 Apr 2021 13:46:41 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley , Ramsay Jones , =?iso-8859-1?q?=C6var_Arnfj=F6r?= =?iso-8859-1?q?=F0?= Bjarmason Subject: [PATCH v5 2/8] revision: mark commit parents as NOT_USER_GIVEN Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The NOT_USER_GIVEN flag of an object marks whether a flag was explicitly provided by the user or not. The most important use case for this is when filtering objects: only objects that were not explicitly requested will get filtered. The flag is currently only set for blobs and trees, which has been fine given that there are no filters for tags or commits currently. We're about to extend filtering capabilities to add object type filter though, which requires us to set up the NOT_USER_GIVEN flag correctly -- if it's not set, the object wouldn't get filtered at all. Mark unseen commit parents as NOT_USER_GIVEN when processing parents. Like this, explicitly provided parents stay user-given and thus unfiltered, while parents which get loaded as part of the graph walk can be filtered. This commit shouldn't have any user-visible impact yet as there is no logic to filter commits yet. Signed-off-by: Patrick Steinhardt --- revision.c | 4 ++-- revision.h | 3 --- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/revision.c b/revision.c index 553c0faa9b..fd34c75e23 100644 --- a/revision.c +++ b/revision.c @@ -1123,7 +1123,7 @@ static int process_parents(struct rev_info *revs, struct commit *commit, mark_parents_uninteresting(p); if (p->object.flags & SEEN) continue; - p->object.flags |= SEEN; + p->object.flags |= (SEEN | NOT_USER_GIVEN); if (list) commit_list_insert_by_date(p, list); if (queue) @@ -1165,7 +1165,7 @@ static int process_parents(struct rev_info *revs, struct commit *commit, } p->object.flags |= left_flag; if (!(p->object.flags & SEEN)) { - p->object.flags |= SEEN; + p->object.flags |= (SEEN | NOT_USER_GIVEN); if (list) commit_list_insert_by_date(p, list); if (queue) diff --git a/revision.h b/revision.h index a24f72dcd1..93aa012f51 100644 --- a/revision.h +++ b/revision.h @@ -44,9 +44,6 @@ /* * Indicates object was reached by traversal. i.e. not given by user on * command-line or stdin. - * NEEDSWORK: NOT_USER_GIVEN doesn't apply to commits because we only support - * filtering trees and blobs, but it may be useful to support filtering commits - * in the future. */ #define NOT_USER_GIVEN (1u<<25) #define TRACK_LINEAR (1u<<26) From patchwork Mon Apr 19 11:46:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12211653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88A43C433ED for ; Mon, 19 Apr 2021 11:46:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 68F1E6108B for ; Mon, 19 Apr 2021 11:46:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238882AbhDSLrU (ORCPT ); Mon, 19 Apr 2021 07:47:20 -0400 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:56101 "EHLO wout1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238853AbhDSLrT (ORCPT ); Mon, 19 Apr 2021 07:47:19 -0400 Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 59D92E11; Mon, 19 Apr 2021 07:46:49 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Mon, 19 Apr 2021 07:46:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=sDDJEG8WaKOB8DgW61tKtMmQ7f3 ifwVRtzy5Mdd2fzI=; b=A6KF7FIGubILHBwh24BeAMLKxtSfSLbvX09tSthzS9q 39RZ5Akq7FN4n1RLEF0qif3s6PLpyORZqFYqz0rkc2Ubg0cY4VD8AeGewEGhOZun viPmTqr/iKYLXa1FAH9YIJqrc2l6no0mZy/9+jRaRhiAsH1qYgKypS6VAphRzDiX Av97yYgeT3CsMkqIuD7xw31lwy6pQgsD9rCr84XSEVDfaPodGQ6mfInvT4PnXkra U87Q7+EGcXaEeB0fe13YuJxadOrucSFRAB6PUpz+4mJFbApwTJ+tyNQQx59upitB xfV7Px9hBtcjaBrkUp/yKFv6QCAIKB9SDRtOp6ZFzrQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=sDDJEG 8WaKOB8DgW61tKtMmQ7f3ifwVRtzy5Mdd2fzI=; b=j3pVNHci4hmrkUyJq1T6l7 h3+jI5cV/VllPJiscFatmB+i1e95F5OVpLL5TifzP8domzIWjYuO9D+ikiEM8aHq d0du7czBhLumqXTQoRdYFzLCb6e3XukS9UkFnZaPHouXstt9RYfujTgYn20cHzV7 m3o5tiCui3sB2j46ifbgKx9sOF7ffmehWINwTy6cUbBwZeOwgJXacAcxZ8hAl3j0 V6M95idQbUo4PZ7ksVyGmjHbIks0uFd4asaIkLG1Y+Iuo2DIdkJx8RlYG30nxZWU Q+lC1QAzdnvovPOSEub/G5fw8HMrz6ZOgIUfpEIvzFsx/7KlkNXvRjvhkT3xkHPA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvddtgedggeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeejrdduledurdeikedrkedtnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf4450.dyn.telefonica.de [77.191.68.80]) by mail.messagingengine.com (Postfix) with ESMTPA id 8CDE51080063; Mon, 19 Apr 2021 07:46:47 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id de706fe1 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 19 Apr 2021 11:46:46 +0000 (UTC) Date: Mon, 19 Apr 2021 13:46:45 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley , Ramsay Jones , =?iso-8859-1?q?=C6var_Arnfj=F6r?= =?iso-8859-1?q?=F0?= Bjarmason Subject: [PATCH v5 3/8] list-objects: move tag processing into its own function Message-ID: <21d7f06d0a94fc8b4f48f90007adb30fae18aa28.1618832437.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move processing of tags into its own function to make the logic easier to extend when we're going to implement filtering for tags. No change in behaviour is expected from this commit. Signed-off-by: Patrick Steinhardt --- list-objects.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/list-objects.c b/list-objects.c index e19589baa0..a5a60301cb 100644 --- a/list-objects.c +++ b/list-objects.c @@ -213,6 +213,14 @@ static void process_tree(struct traversal_context *ctx, free_tree_buffer(tree); } +static void process_tag(struct traversal_context *ctx, + struct tag *tag, + const char *name) +{ + tag->object.flags |= SEEN; + ctx->show_object(&tag->object, name, ctx->show_data); +} + static void mark_edge_parents_uninteresting(struct commit *commit, struct rev_info *revs, show_edge_fn show_edge) @@ -334,8 +342,7 @@ static void traverse_trees_and_blobs(struct traversal_context *ctx, if (obj->flags & (UNINTERESTING | SEEN)) continue; if (obj->type == OBJ_TAG) { - obj->flags |= SEEN; - ctx->show_object(obj, name, ctx->show_data); + process_tag(ctx, (struct tag *)obj, name); continue; } if (!path) From patchwork Mon Apr 19 11:46:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12211655 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B79F9C433ED for ; Mon, 19 Apr 2021 11:46:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8E21D61090 for ; Mon, 19 Apr 2021 11:46:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238784AbhDSLr0 (ORCPT ); Mon, 19 Apr 2021 07:47:26 -0400 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:59027 "EHLO wout1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238861AbhDSLrX (ORCPT ); Mon, 19 Apr 2021 07:47:23 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 16C83B01; Mon, 19 Apr 2021 07:46:53 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 19 Apr 2021 07:46:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=lDFhB8knGj7NUMQ1WyvQdPhHz2k XBXLsT+eQboid8WI=; b=cXzFBBQMQUFTe2RzvRSoHRMjEgdjd2tKZRYQA+VRNiZ exCjMan4rerMspceCzY3s1ue2Rejt8gzZRHQJHbz9IFzNZYMFWU/wde4ORzLTlos oIMHvZDxa+x5OnU3Iw38MEowRBJijfYFqSrONfNEoXOLi8f0tSI0VRPFaXuer34g 8o1aDn4l5DH8xl8I6CH1L2JoO9mC3HEKuN7rSWmBtw+ssMXoDYDTUrRgxTQBJmFc Kp0dFyT1ktDKCD/99yUK3z5L5sYun+l9ZzSGXLl9BadGlwCx+ZVEDiV/ofPJ+D9x EeNxjfDFs2z5u7nRRBjTPoZ1eOVHELAvuE3jcLABYRQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=lDFhB8 knGj7NUMQ1WyvQdPhHz2kXBXLsT+eQboid8WI=; b=vM1owIpEBJ13du+4ugqBy6 AOF70ohJrDp2qcY74O/8eU5NH67ZKOS1QY1kfwKfPG/LGrw4HT2DJRUPZ7lXIi9C dyrUTaD53IYi1sUEJarebBMhqANa3g0rDB046anxIARsF4+BoKIrwt6FX8XcXCZP WfPLMit1ykLsW61xkSFZBFb+4jlZUllI8g321aLUZxY6zX7H3VsN2oge6e81fj+T 2AaXUMLx1EDEBk8zq84dRpp7BAF3mtb5Hs78n/8VYNgjfjfev2lb1D15FfhfTzvV 8GsiuEdEcne0O6EYdfRF4n1sCCGGQ+UX37+Oz32u1z5oQvMEbbxijgNVhhgWQUFw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvddtgedggeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeejrdduledurdeikedrkedtnecuvehluhhsthgvrhfuihiivgepfeenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf4450.dyn.telefonica.de [77.191.68.80]) by mail.messagingengine.com (Postfix) with ESMTPA id AA35C1080063; Mon, 19 Apr 2021 07:46:51 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 85d40870 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 19 Apr 2021 11:46:50 +0000 (UTC) Date: Mon, 19 Apr 2021 13:46:49 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley , Ramsay Jones , =?iso-8859-1?q?=C6var_Arnfj=F6r?= =?iso-8859-1?q?=F0?= Bjarmason Subject: [PATCH v5 4/8] list-objects: support filtering by tag and commit Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Object filters currently only support filtering blobs or trees based on some criteria. This commit lays the foundation to also allow filtering of tags and commits. No change in behaviour is expected from this commit given that there are no filters yet for those object types. Signed-off-by: Patrick Steinhardt --- list-objects-filter.c | 40 ++++++++++++++++++++++++++++++++++++++++ list-objects-filter.h | 2 ++ list-objects.c | 23 ++++++++++++++++++++--- 3 files changed, 62 insertions(+), 3 deletions(-) diff --git a/list-objects-filter.c b/list-objects-filter.c index 39e2f15333..0ebfa52966 100644 --- a/list-objects-filter.c +++ b/list-objects-filter.c @@ -82,6 +82,16 @@ static enum list_objects_filter_result filter_blobs_none( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); /* always include all tree objects */ @@ -173,6 +183,16 @@ static enum list_objects_filter_result filter_trees_depth( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_END_TREE: assert(obj->type == OBJ_TREE); filter_data->current_depth--; @@ -267,6 +287,16 @@ static enum list_objects_filter_result filter_blobs_limit( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); /* always include all tree objects */ @@ -371,6 +401,16 @@ static enum list_objects_filter_result filter_sparse( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); dtype = DT_DIR; diff --git a/list-objects-filter.h b/list-objects-filter.h index cfd784e203..9e98814111 100644 --- a/list-objects-filter.h +++ b/list-objects-filter.h @@ -55,6 +55,8 @@ enum list_objects_filter_result { }; enum list_objects_filter_situation { + LOFS_COMMIT, + LOFS_TAG, LOFS_BEGIN_TREE, LOFS_END_TREE, LOFS_BLOB diff --git a/list-objects.c b/list-objects.c index a5a60301cb..7f404677d5 100644 --- a/list-objects.c +++ b/list-objects.c @@ -217,8 +217,15 @@ static void process_tag(struct traversal_context *ctx, struct tag *tag, const char *name) { - tag->object.flags |= SEEN; - ctx->show_object(&tag->object, name, ctx->show_data); + enum list_objects_filter_result r; + + r = list_objects_filter__filter_object(ctx->revs->repo, LOFS_TAG, + &tag->object, NULL, NULL, + ctx->filter); + if (r & LOFR_MARK_SEEN) + tag->object.flags |= SEEN; + if (r & LOFR_DO_SHOW) + ctx->show_object(&tag->object, name, ctx->show_data); } static void mark_edge_parents_uninteresting(struct commit *commit, @@ -368,6 +375,12 @@ static void do_traverse(struct traversal_context *ctx) strbuf_init(&csp, PATH_MAX); while ((commit = get_revision(ctx->revs)) != NULL) { + enum list_objects_filter_result r; + + r = list_objects_filter__filter_object(ctx->revs->repo, + LOFS_COMMIT, &commit->object, + NULL, NULL, ctx->filter); + /* * an uninteresting boundary commit may not have its tree * parsed yet, but we are not going to show them anyway @@ -382,7 +395,11 @@ static void do_traverse(struct traversal_context *ctx) die(_("unable to load root tree for commit %s"), oid_to_hex(&commit->object.oid)); } - ctx->show_commit(commit, ctx->show_data); + + if (r & LOFR_MARK_SEEN) + commit->object.flags |= SEEN; + if (r & LOFR_DO_SHOW) + ctx->show_commit(commit, ctx->show_data); if (ctx->revs->tree_blobs_in_commit_order) /* From patchwork Mon Apr 19 11:46:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12211657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED33DC433B4 for ; Mon, 19 Apr 2021 11:47:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CF0EB6108B for ; Mon, 19 Apr 2021 11:47:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238893AbhDSLri (ORCPT ); Mon, 19 Apr 2021 07:47:38 -0400 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:34057 "EHLO wout1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238853AbhDSLr1 (ORCPT ); Mon, 19 Apr 2021 07:47:27 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 3863DB06; Mon, 19 Apr 2021 07:46:57 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 19 Apr 2021 07:46:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=R266wXLwAjRjLm20wd/Q33iCQA0 eNuZQRbNZzhoZCdk=; b=xuU9A4nHCc4kCKIJNGewEVy3f69/rZBVv4LJdY7sqzA Hkrogvhc6/Nw3yYpm5+dR7wOD3CnaNtjRzFtUgDgeQLuvhqg4F+beSsgbpubE9m4 0xRi61xzehvDF+Sv/66gkQwDgZ7c1sDxFVnulB30TydbAxHTwQ/xvOzFJmjcuI59 CnhPdW4ouosBcuoemPC23Af1/lHqyHOauTFFPPDCxb/cGu4OkGzabhVi7ElbuBYZ IjW9O9/Hacau1ZGnVi1rpM1spihJKBM0hKoFGOUIZFNmusHtAImDdIE+2ntWA2Xh Mu6M2zWwB7sJkF5RkjerJVYcAkHHW1Uh33swsStq5bw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=R266wX LwAjRjLm20wd/Q33iCQA0eNuZQRbNZzhoZCdk=; b=sk+Rf68ip5jIc3vJC4cTWg IpFmawcNabISvksXMg+nFzTIEsfoB/WUi17/V6EudgHGE+IV9DIdNF2GLz8BZQd0 0JBZE89tgC6OH/SHqD2oUfhw2oq8/cvUg1UBOIDUTHAo5ZRmDKJAVW6Fc97b810B 9ruJlRbUAa48EDu87arzInYbW1zb5VAGtX191MGoztjI4IbL42fyK6HHhQ9muVHr d6KDR9Xf/5KMKbY5JJvUdAgHKlWcP2+aq2qX77jvr4vXcfb3INgAMEN9Tx4R9nLH 98WH/lLPgvqLKUvWdE175KDrKk3s6vblqCpL07nRmAFpZaFInefDyEzS1EW1l0+w == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvddtgedggeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeejrdduledurdeikedrkedtnecuvehluhhsthgvrhfuihiivgepgeenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf4450.dyn.telefonica.de [77.191.68.80]) by mail.messagingengine.com (Postfix) with ESMTPA id C7D731080064; Mon, 19 Apr 2021 07:46:55 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id dd629e51 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 19 Apr 2021 11:46:55 +0000 (UTC) Date: Mon, 19 Apr 2021 13:46:53 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley , Ramsay Jones , =?iso-8859-1?q?=C6var_Arnfj=F6r?= =?iso-8859-1?q?=F0?= Bjarmason Subject: [PATCH v5 5/8] list-objects: implement object type filter Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org While it already is possible to filter objects by some criteria in git-rev-list(1), it is not yet possible to filter out only a specific type of objects. This makes some filters less useful. The `blob:limit` filter for example filters blobs such that only those which are smaller than the given limit are returned. But it is unfit to ask only for these smallish blobs, given that git-rev-list(1) will continue to print tags, commits and trees. Now that we have the infrastructure in place to also filter tags and commits, we can improve this situation by implementing a new filter which selects objects based on their type. Above query can thus trivially be implemented with the following command: $ git rev-list --objects --filter=object:type=blob \ --filter=blob:limit=200 Furthermore, this filter allows to optimize for certain other cases: if for example only tags or commits have been selected, there is no need to walk down trees. The new filter is not yet supported in bitmaps. This is going to be implemented in a subsequent commit. Signed-off-by: Patrick Steinhardt --- Documentation/config/uploadpack.txt | 6 +-- Documentation/rev-list-options.txt | 3 ++ list-objects-filter-options.c | 15 ++++++ list-objects-filter-options.h | 3 ++ list-objects-filter.c | 76 +++++++++++++++++++++++++++++ t/t6112-rev-list-filters-objects.sh | 44 +++++++++++++++++ 6 files changed, 144 insertions(+), 3 deletions(-) diff --git a/Documentation/config/uploadpack.txt b/Documentation/config/uploadpack.txt index 6729a072ea..32fad5bbe8 100644 --- a/Documentation/config/uploadpack.txt +++ b/Documentation/config/uploadpack.txt @@ -66,9 +66,9 @@ uploadpackfilter.allow:: uploadpackfilter..allow:: Explicitly allow or ban the object filter corresponding to ``, where `` may be one of: `blob:none`, - `blob:limit`, `tree`, `sparse:oid`, or `combine`. If using - combined filters, both `combine` and all of the nested filter - kinds must be allowed. Defaults to `uploadpackfilter.allow`. + `blob:limit`, `object:type`, `tree`, `sparse:oid`, or `combine`. + If using combined filters, both `combine` and all of the nested + filter kinds must be allowed. Defaults to `uploadpackfilter.allow`. uploadpackfilter.tree.maxDepth:: Only allow `--filter=tree:` when `` is no more than the value of diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index b1c8f86c6e..3afa8fffbd 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -892,6 +892,9 @@ or units. n may be zero. The suffixes k, m, and g can be used to name units in KiB, MiB, or GiB. For example, 'blob:limit=1k' is the same as 'blob:limit=1024'. + +The form '--filter=object:type=(tag|commit|tree|blob)' omits all objects +which are not of the requested type. ++ The form '--filter=sparse:oid=' uses a sparse-checkout specification contained in the blob (or blob-expression) '' to omit blobs that would not be not required for a sparse checkout on diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index d2d1c81caf..96a605c8ad 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -29,6 +29,8 @@ const char *list_object_filter_config_name(enum list_objects_filter_choice c) return "tree"; case LOFC_SPARSE_OID: return "sparse:oid"; + case LOFC_OBJECT_TYPE: + return "object:type"; case LOFC_COMBINE: return "combine"; case LOFC__COUNT: @@ -97,6 +99,19 @@ static int gently_parse_list_objects_filter( } return 1; + } else if (skip_prefix(arg, "object:type=", &v0)) { + int type = type_from_string_gently(v0, strlen(v0), 1); + if (type < 0) { + strbuf_addf(errbuf, _("'%s' for 'object:type=' is" + "not a valid object type"), v0); + return 1; + } + + filter_options->object_type = type; + filter_options->choice = LOFC_OBJECT_TYPE; + + return 0; + } else if (skip_prefix(arg, "combine:", &v0)) { return parse_combine_filter(filter_options, v0, errbuf); diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h index 01767c3c96..da5b6737e2 100644 --- a/list-objects-filter-options.h +++ b/list-objects-filter-options.h @@ -1,6 +1,7 @@ #ifndef LIST_OBJECTS_FILTER_OPTIONS_H #define LIST_OBJECTS_FILTER_OPTIONS_H +#include "cache.h" #include "parse-options.h" #include "string-list.h" @@ -13,6 +14,7 @@ enum list_objects_filter_choice { LOFC_BLOB_LIMIT, LOFC_TREE_DEPTH, LOFC_SPARSE_OID, + LOFC_OBJECT_TYPE, LOFC_COMBINE, LOFC__COUNT /* must be last */ }; @@ -54,6 +56,7 @@ struct list_objects_filter_options { char *sparse_oid_name; unsigned long blob_limit_value; unsigned long tree_exclude_depth; + enum object_type object_type; /* LOFC_COMBINE values */ diff --git a/list-objects-filter.c b/list-objects-filter.c index 0ebfa52966..1c1ee3d1bb 100644 --- a/list-objects-filter.c +++ b/list-objects-filter.c @@ -545,6 +545,81 @@ static void filter_sparse_oid__init( filter->free_fn = filter_sparse_free; } +/* + * A filter for list-objects to omit large blobs. + * And to OPTIONALLY collect a list of the omitted OIDs. + */ +struct filter_object_type_data { + enum object_type object_type; +}; + +static enum list_objects_filter_result filter_object_type( + struct repository *r, + enum list_objects_filter_situation filter_situation, + struct object *obj, + const char *pathname, + const char *filename, + struct oidset *omits, + void *filter_data_) +{ + struct filter_object_type_data *filter_data = filter_data_; + + switch (filter_situation) { + default: + BUG("unknown filter_situation: %d", filter_situation); + + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + if (filter_data->object_type == OBJ_TAG) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + return LOFR_MARK_SEEN; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + if (filter_data->object_type == OBJ_COMMIT) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + return LOFR_MARK_SEEN; + + case LOFS_BEGIN_TREE: + assert(obj->type == OBJ_TREE); + + /* + * If we only want to show commits or tags, then there is no + * need to walk down trees. + */ + if (filter_data->object_type == OBJ_COMMIT || + filter_data->object_type == OBJ_TAG) + return LOFR_SKIP_TREE; + + if (filter_data->object_type == OBJ_TREE) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + return LOFR_MARK_SEEN; + + case LOFS_BLOB: + assert(obj->type == OBJ_BLOB); + + if (filter_data->object_type == OBJ_BLOB) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + return LOFR_MARK_SEEN; + + case LOFS_END_TREE: + return LOFR_ZERO; + } +} + +static void filter_object_type__init( + struct list_objects_filter_options *filter_options, + struct filter *filter) +{ + struct filter_object_type_data *d = xcalloc(1, sizeof(*d)); + d->object_type = filter_options->object_type; + + filter->filter_data = d; + filter->filter_object_fn = filter_object_type; + filter->free_fn = free; +} + /* A filter which only shows objects shown by all sub-filters. */ struct combine_filter_data { struct subfilter *sub; @@ -691,6 +766,7 @@ static filter_init_fn s_filters[] = { filter_blobs_limit__init, filter_trees_depth__init, filter_sparse_oid__init, + filter_object_type__init, filter_combine__init, }; diff --git a/t/t6112-rev-list-filters-objects.sh b/t/t6112-rev-list-filters-objects.sh index 31457d13b9..c810a62807 100755 --- a/t/t6112-rev-list-filters-objects.sh +++ b/t/t6112-rev-list-filters-objects.sh @@ -159,6 +159,50 @@ test_expect_success 'verify blob:limit=1m' ' test_must_be_empty observed ' +# Test object:type= filter. + +test_expect_success 'setup object-type' ' + test_create_repo object-type && + test_commit --no-tag -C object-type message blob && + git -C object-type tag tag -m tag-message +' + +test_expect_success 'verify object:type= fails with invalid type' ' + test_must_fail git -C object-type rev-list --objects --filter=object:type= HEAD && + test_must_fail git -C object-type rev-list --objects --filter=object:type=invalid HEAD +' + +test_expect_success 'verify object:type=blob prints blob and commit' ' + git -C object-type rev-parse HEAD >expected && + printf "%s blob\n" $(git -C object-type rev-parse HEAD:blob) >>expected && + git -C object-type rev-list --objects --filter=object:type=blob HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tree prints tree and commit' ' + ( + git -C object-type rev-parse HEAD && + printf "%s \n" $(git -C object-type rev-parse HEAD^{tree}) + ) >expected && + git -C object-type rev-list --objects --filter=object:type=tree HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=commit prints commit' ' + git -C object-type rev-parse HEAD >expected && + git -C object-type rev-list --objects --filter=object:type=commit HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tag prints tag' ' + ( + git -C object-type rev-parse HEAD && + printf "%s tag\n" $(git -C object-type rev-parse tag) + ) >expected && + git -C object-type rev-list --objects --filter=object:type=tag tag >actual && + test_cmp expected actual +' + # Test sparse:path= filter. # !!!! # NOTE: sparse:path filter support has been dropped for security reasons, From patchwork Mon Apr 19 11:46:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12211661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3153DC433ED for ; Mon, 19 Apr 2021 11:47:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1027B61165 for ; Mon, 19 Apr 2021 11:47:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238896AbhDSLrx (ORCPT ); Mon, 19 Apr 2021 07:47:53 -0400 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:55079 "EHLO wout1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238892AbhDSLrb (ORCPT ); Mon, 19 Apr 2021 07:47:31 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 586A8E0B; Mon, 19 Apr 2021 07:47:01 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 19 Apr 2021 07:47:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=51tka3J7eTCF+fhMUrHcNg991fP 7d5aT18nIRw5XwJo=; b=isFOTT+qZXbSHNc85ckjBMsKowDfUi4oNKTd4e9CB6M h74gio+npB3jQbUbUkfLtxx38pDNCoDmYPoTdE0C7BO0o9/BXz44yg8NP74GAeye Iy1w5GDg4O9Wvj3unNshTxt8xjatCiez4Yl7nQu+2Ao7R6ZPdHIwQtCMUL/Zn7ft fHrTAMpeVMai7hBZNbrxuOzTFknjLiwY1dB3V54RdfQxdR3BGOjdtTIUTaQ47aC+ mSAaUwQP3W2j5AJIZlj2Hhd8kzXseZgbPYSrR1VE6aSeB0szjjempuVE+GkCAEAT CdMv5SrHz1kPUDj4kaOS89FE3vFnnNBVJTSDCyHW1Rg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=51tka3 J7eTCF+fhMUrHcNg991fP7d5aT18nIRw5XwJo=; b=bkLj9luyGKOcTThhrR5IE3 xqRNt5MpjTRm0reEDp3pn+DGRhCyP2N8qFuErJAE5qEhHVI6EgBniP8gUVrbRY8b dTCVT0HMp+fOXiVQ9wAy3L69CStJ1oAnYP/K1nFKnD1bqFAI6QBnISiED3wkb8yJ 2bU+SQnFdxut3XW7ennTNtUU59QYEcHwLBdRE1eUQjJJ+1HJEuJj4Gonk1IZC3J1 lf8/XnxqmS9WNixqyot7HWbIgzXFhc0aZwNNfXiLPxtMUfzLbO21B7ereshGmI/U zDVrmp6OJj8oI99aZweg0S0hHr1AM6SEV2NpAXiD5P2u1qlkgdmBT+yzTGrjQquQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvddtgedggeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeejrdduledurdeikedrkedtnecuvehluhhsthgvrhfuihiivgepheenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf4450.dyn.telefonica.de [77.191.68.80]) by mail.messagingengine.com (Postfix) with ESMTPA id E45A4108005C; Mon, 19 Apr 2021 07:46:59 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 0434d6bd (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 19 Apr 2021 11:46:59 +0000 (UTC) Date: Mon, 19 Apr 2021 13:46:58 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley , Ramsay Jones , =?iso-8859-1?q?=C6var_Arnfj=F6r?= =?iso-8859-1?q?=F0?= Bjarmason Subject: [PATCH v5 6/8] pack-bitmap: implement object type filter Message-ID: <90c80c1efad7974d5f1d87970c0c7918c06808b5.1618832437.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The preceding commit has added a new object filter for git-rev-list(1) which allows to filter objects by type. Implement the equivalent filter for packfile bitmaps so that we can answer these queries fast. Signed-off-by: Patrick Steinhardt --- pack-bitmap.c | 29 ++++++++++++++++++++++++++--- t/t6113-rev-list-bitmap-filters.sh | 25 ++++++++++++++++++++++++- 2 files changed, 50 insertions(+), 4 deletions(-) diff --git a/pack-bitmap.c b/pack-bitmap.c index 3ed15431cd..e419f4206b 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -780,9 +780,6 @@ static void filter_bitmap_exclude_type(struct bitmap_index *bitmap_git, eword_t mask; uint32_t i; - if (type != OBJ_BLOB && type != OBJ_TREE) - BUG("filter_bitmap_exclude_type: unsupported type '%d'", type); - /* * The non-bitmap version of this filter never removes * objects which the other side specifically asked for, @@ -912,6 +909,24 @@ static void filter_bitmap_tree_depth(struct bitmap_index *bitmap_git, OBJ_BLOB); } +static void filter_bitmap_object_type(struct bitmap_index *bitmap_git, + struct object_list *tip_objects, + struct bitmap *to_filter, + enum object_type object_type) +{ + if (object_type < OBJ_COMMIT || object_type > OBJ_TAG) + BUG("filter_bitmap_object_type given invalid object"); + + if (object_type != OBJ_TAG) + filter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_TAG); + if (object_type != OBJ_COMMIT) + filter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_COMMIT); + if (object_type != OBJ_TREE) + filter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_TREE); + if (object_type != OBJ_BLOB) + filter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, OBJ_BLOB); +} + static int filter_bitmap(struct bitmap_index *bitmap_git, struct object_list *tip_objects, struct bitmap *to_filter, @@ -944,6 +959,14 @@ static int filter_bitmap(struct bitmap_index *bitmap_git, return 0; } + if (filter->choice == LOFC_OBJECT_TYPE) { + if (bitmap_git) + filter_bitmap_object_type(bitmap_git, tip_objects, + to_filter, + filter->object_type); + return 0; + } + /* filter choice not handled */ return -1; } diff --git a/t/t6113-rev-list-bitmap-filters.sh b/t/t6113-rev-list-bitmap-filters.sh index 3f889949ca..fb66735ac8 100755 --- a/t/t6113-rev-list-bitmap-filters.sh +++ b/t/t6113-rev-list-bitmap-filters.sh @@ -10,7 +10,8 @@ test_expect_success 'set up bitmapped repo' ' test_commit much-larger-blob-one && git repack -adb && test_commit two && - test_commit much-larger-blob-two + test_commit much-larger-blob-two && + git tag tag ' test_expect_success 'filters fallback to non-bitmap traversal' ' @@ -75,4 +76,26 @@ test_expect_success 'tree:1 filter' ' test_cmp expect actual ' +test_expect_success 'object:type filter' ' + git rev-list --objects --filter=object:type=tag tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=tag tag >actual && + test_cmp expect actual && + + git rev-list --objects --filter=object:type=commit tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=commit tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter=object:type=tree tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=tree tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual +' + test_done From patchwork Mon Apr 19 11:47:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12211659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C444EC433ED for ; Mon, 19 Apr 2021 11:47:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A48336108B for ; Mon, 19 Apr 2021 11:47:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238567AbhDSLru (ORCPT ); Mon, 19 Apr 2021 07:47:50 -0400 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:53653 "EHLO wout1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238896AbhDSLrf (ORCPT ); Mon, 19 Apr 2021 07:47:35 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 68606B3B; Mon, 19 Apr 2021 07:47:05 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 19 Apr 2021 07:47:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=D4pdkB7ro/GuZ3+Lm7yBiYSpBDg Lu4sKUkYFh6OGCyc=; b=yZw0pAMEQOirDzzquLGrxBjThXnAQ2hG3kJt70mKi5N 65Gs4oBDRFPWKDg3434c4cHll4m+z+AJwTLT536Fs4mEC+Czis79W37Op61Twv9F NGJFqeV0GIFHMrhdsXt7GNZ4oHGRwUHLaVk2+4vsZSsaazHUiFTURZNswpJrMqxz bYZmK1YRbB5aju7Xue14Pbj4tcbcIoRqinan3uro/iUqahw/UL9t0fv4d4HXwxNY 6ktgtDcS70aEom4lnFy1KEU8sTLA+f4aRdK2zCFJZ+xfDOSBO/o7xPVjI/gZeZen NZjBxTkXlhLMltP7fyfiItqBl6BR6lQcWhtOA47hOPA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=D4pdkB 7ro/GuZ3+Lm7yBiYSpBDgLu4sKUkYFh6OGCyc=; b=FlN3uC8ptzsnjervNvGt7g GEICDWewJrw7MSWBQrHQzr0QBg/CV748XkKhpc69OQIfb/+2gLXpin+eo/9raCDn lGA/wojBCyycPrd+wObQJWOzsLyNjaOOON/1xsvrMVTZakdIihY9SW0MAaK606md Z0U69RlQEAUopVKsXGjH/bXO330dyWxRRO/X/F+o15K8AS8NqS6/LcDXPKab4mIK mGMn7YMsgAzDpVSis4zk0yFoMRkFydqAjw5EEwvNLjSqfpKHQI8ZqBUBLPlNl9w9 YlLoYllsY2X33Q8shcmLJNPxeycQCHOmw69bv6wVIsO9B566wjC9SG5Gw5zdYbKw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvddtgedggeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepjeejrdduledurdeikedrkedtnecuvehluhhsthgvrhfuihiivgepheenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf4450.dyn.telefonica.de [77.191.68.80]) by mail.messagingengine.com (Postfix) with ESMTPA id 1461D108005B; Mon, 19 Apr 2021 07:47:03 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id dc7c0fad (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 19 Apr 2021 11:47:03 +0000 (UTC) Date: Mon, 19 Apr 2021 13:47:02 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley , Ramsay Jones , =?iso-8859-1?q?=C6var_Arnfj=F6r?= =?iso-8859-1?q?=F0?= Bjarmason Subject: [PATCH v5 7/8] pack-bitmap: implement combined filter Message-ID: <9726e698617fa6a5a4d41283400e6a72e08baefb.1618832437.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When the user has multiple objects filters specified, then this is internally represented by having a "combined" filter. These combined filters aren't yet supported by bitmap indices and can thus not be accelerated. Fix this by implementing support for these combined filters. The implementation is quite trivial: when there's a combined filter, we simply recurse into `filter_bitmap()` for all of the sub-filters. Signed-off-by: Patrick Steinhardt --- pack-bitmap.c | 10 ++++++++++ t/t6113-rev-list-bitmap-filters.sh | 7 +++++++ 2 files changed, 17 insertions(+) diff --git a/pack-bitmap.c b/pack-bitmap.c index e419f4206b..53f35d41c6 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -967,6 +967,16 @@ static int filter_bitmap(struct bitmap_index *bitmap_git, return 0; } + if (filter->choice == LOFC_COMBINE) { + int i; + for (i = 0; i < filter->sub_nr; i++) { + if (filter_bitmap(bitmap_git, tip_objects, to_filter, + &filter->sub[i]) < 0) + return -1; + } + return 0; + } + /* filter choice not handled */ return -1; } diff --git a/t/t6113-rev-list-bitmap-filters.sh b/t/t6113-rev-list-bitmap-filters.sh index fb66735ac8..cb9db7df6f 100755 --- a/t/t6113-rev-list-bitmap-filters.sh +++ b/t/t6113-rev-list-bitmap-filters.sh @@ -98,4 +98,11 @@ test_expect_success 'object:type filter' ' test_bitmap_traversal expect actual ' +test_expect_success 'combine filter' ' + git rev-list --objects --filter=blob:limit=1000 --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=blob:limit=1000 --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual +' + test_done From patchwork Mon Apr 19 11:47:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12211669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B43BCC433B4 for ; Mon, 19 Apr 2021 11:48:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 87C1261090 for ; Mon, 19 Apr 2021 11:48:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233565AbhDSLsu (ORCPT ); Mon, 19 Apr 2021 07:48:50 -0400 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:44791 "EHLO wout1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238897AbhDSLrk (ORCPT ); Mon, 19 Apr 2021 07:47:40 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 92DF7B3E; Mon, 19 Apr 2021 07:47:10 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 19 Apr 2021 07:47:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=QDQqnEJRCUullGzgaqMtGjPc0tL bYM1jQACRjC1qdT8=; b=ckz3Ca0uHZfUHFNeKGCLt9oHsvL0GRAwHlxn6ccMYdK qOsVBNiJYAS6ot/qIZvmIxiNsB+VIyCFsEalrqeDUsOMUsulEyGwby7Ma8sLqvwG MqaCKyO1319qfyehTZ19TmppHiTDwl8ty07IhK3KiNsp0lB5zKjtPhccifYjHG+5 XxLieiyiJMfehEK+9Teqpf4eMZv4N89wqdgF1nRARimjDWqjfAcf43SuwnCjqPUX uMSoZDyWe79qOJpVG8tPVfzCru01hU9Rw5NOK2FcFoGmk3okMkgoEYW0zO08hO7U TT9BBc0Jkl95sfcj/WFHcwNAVxmuhkwE324Nb2F2roA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=QDQqnE JRCUullGzgaqMtGjPc0tLbYM1jQACRjC1qdT8=; b=ceeoMwNSCXOoh/a6HLjjKL Z8AMGeQq1Gc+cjFnlntAfxVF9bP7SRqN/9RuD7lRmj/7E/3ymsrEfiaUbo1ueUiW 9G4lpgCg9BsppUKToUYZOY8c1JAj/XNS1juCivqolBBvL3C6HJ17ujAbD6aJAugd CU+fplK6Ex7OaidD7ifEZnFm82Gap7tocJxnKgXyNe+XEwkXFN6I4PHtF2Bhp/EM AzxfOzEsyMs7eUFbtUXm9r1asKGuECV1AAyvhyjDQ8k1zBZuqY921Ok6FvVALlbq p3k7N11xNq152sqFAST9Jr2v26sjy6O/LzxHYM0fTy7Ff4rvhiddc8BIZqGlxGEQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvddtgedggeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpefgfeejuefgheegueeljeffteefuedthfdvvedugfevvdffleduueeiiefgvedtheen ucffohhmrghinhepphgvnhguihhnghdrnhhrnecukfhppeejjedrudeluddrieekrdektd enucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpshes phhkshdrihhm X-ME-Proxy: Received: from vm-mail.pks.im (x4dbf4450.dyn.telefonica.de [77.191.68.80]) by mail.messagingengine.com (Postfix) with ESMTPA id 3200D108005B; Mon, 19 Apr 2021 07:47:09 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 02542775 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 19 Apr 2021 11:47:08 +0000 (UTC) Date: Mon, 19 Apr 2021 13:47:06 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Christian Couder , Taylor Blau , Philip Oakley , Ramsay Jones , =?iso-8859-1?q?=C6var_Arnfj=F6r?= =?iso-8859-1?q?=F0?= Bjarmason Subject: [PATCH v5 8/8] rev-list: allow filtering of provided items Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When providing an object filter, it is currently impossible to also filter provided items. E.g. when executing `git rev-list HEAD` , the commit this reference points to will be treated as user-provided and is thus excluded from the filtering mechanism. This makes it harder than necessary to properly use the new `--filter=object:type` filter given that even if the user wants to only see blobs, he'll still see commits of provided references. Improve this by introducing a new `--filter-provided-objects` option to the git-rev-parse(1) command. If given, then all user-provided references will be subject to filtering. Signed-off-by: Patrick Steinhardt --- Documentation/rev-list-options.txt | 5 ++++ builtin/pack-objects.c | 2 +- builtin/rev-list.c | 36 +++++++++++++++++++++-------- pack-bitmap.c | 6 +++-- pack-bitmap.h | 3 ++- reachable.c | 2 +- t/t6112-rev-list-filters-objects.sh | 28 ++++++++++++++++++++++ t/t6113-rev-list-bitmap-filters.sh | 36 +++++++++++++++++++++++++++++ 8 files changed, 104 insertions(+), 14 deletions(-) diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index 3afa8fffbd..5bf2a85f69 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -933,6 +933,11 @@ equivalent. --no-filter:: Turn off any previous `--filter=` argument. +--filter-provided-objects:: + Filter the list of explicitly provided objects, which would otherwise + always be printed even if they did not match any of the filters. Only + useful with `--filter=`. + --filter-print-omitted:: Only useful with `--filter=`; prints a list of the objects omitted by the filter. Object IDs are prefixed with a ``~'' character. diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index a1e33d7507..c89e996f0d 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -3516,7 +3516,7 @@ static int pack_options_allow_reuse(void) static int get_object_list_from_bitmap(struct rev_info *revs) { - if (!(bitmap_git = prepare_bitmap_walk(revs, &filter_options))) + if (!(bitmap_git = prepare_bitmap_walk(revs, &filter_options, 0))) return -1; if (pack_options_allow_reuse() && diff --git a/builtin/rev-list.c b/builtin/rev-list.c index b4d8ea0a35..7677b1af5a 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -398,7 +398,8 @@ static inline int parse_missing_action_value(const char *value) } static int try_bitmap_count(struct rev_info *revs, - struct list_objects_filter_options *filter) + struct list_objects_filter_options *filter, + int filter_provided_objects) { uint32_t commit_count = 0, tag_count = 0, @@ -433,7 +434,7 @@ static int try_bitmap_count(struct rev_info *revs, */ max_count = revs->max_count; - bitmap_git = prepare_bitmap_walk(revs, filter); + bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects); if (!bitmap_git) return -1; @@ -450,7 +451,8 @@ static int try_bitmap_count(struct rev_info *revs, } static int try_bitmap_traversal(struct rev_info *revs, - struct list_objects_filter_options *filter) + struct list_objects_filter_options *filter, + int filter_provided_objects) { struct bitmap_index *bitmap_git; @@ -461,7 +463,7 @@ static int try_bitmap_traversal(struct rev_info *revs, if (revs->max_count >= 0) return -1; - bitmap_git = prepare_bitmap_walk(revs, filter); + bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects); if (!bitmap_git) return -1; @@ -471,14 +473,15 @@ static int try_bitmap_traversal(struct rev_info *revs, } static int try_bitmap_disk_usage(struct rev_info *revs, - struct list_objects_filter_options *filter) + struct list_objects_filter_options *filter, + int filter_provided_objects) { struct bitmap_index *bitmap_git; if (!show_disk_usage) return -1; - bitmap_git = prepare_bitmap_walk(revs, filter); + bitmap_git = prepare_bitmap_walk(revs, filter, filter_provided_objects); if (!bitmap_git) return -1; @@ -499,6 +502,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) int bisect_show_vars = 0; int bisect_find_all = 0; int use_bitmap_index = 0; + int filter_provided_objects = 0; const char *show_progress = NULL; if (argc == 2 && !strcmp(argv[1], "-h")) @@ -599,6 +603,10 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) list_objects_filter_set_no_filter(&filter_options); continue; } + if (!strcmp(arg, "--filter-provided-objects")) { + filter_provided_objects = 1; + continue; + } if (!strcmp(arg, "--filter-print-omitted")) { arg_print_omitted = 1; continue; @@ -665,11 +673,11 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) progress = start_delayed_progress(show_progress, 0); if (use_bitmap_index) { - if (!try_bitmap_count(&revs, &filter_options)) + if (!try_bitmap_count(&revs, &filter_options, filter_provided_objects)) return 0; - if (!try_bitmap_disk_usage(&revs, &filter_options)) + if (!try_bitmap_disk_usage(&revs, &filter_options, filter_provided_objects)) return 0; - if (!try_bitmap_traversal(&revs, &filter_options)) + if (!try_bitmap_traversal(&revs, &filter_options, filter_provided_objects)) return 0; } @@ -694,6 +702,16 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) return show_bisect_vars(&info, reaches, all); } + if (filter_provided_objects) { + struct commit_list *c; + for (i = 0; i < revs.pending.nr; i++) { + struct object_array_entry *pending = revs.pending.objects + i; + pending->item->flags |= NOT_USER_GIVEN; + } + for (c = revs.commits; c; c = c->next) + c->item->object.flags |= NOT_USER_GIVEN; + } + if (arg_print_omitted) oidset_init(&omitted_objects, DEFAULT_OIDSET_SIZE); if (arg_missing_action == MA_PRINT) diff --git a/pack-bitmap.c b/pack-bitmap.c index 53f35d41c6..6efea70ae7 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -987,7 +987,8 @@ static int can_filter_bitmap(struct list_objects_filter_options *filter) } struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, - struct list_objects_filter_options *filter) + struct list_objects_filter_options *filter, + int filter_provided_objects) { unsigned int i; @@ -1082,7 +1083,8 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, if (haves_bitmap) bitmap_and_not(wants_bitmap, haves_bitmap); - filter_bitmap(bitmap_git, wants, wants_bitmap, filter); + filter_bitmap(bitmap_git, (filter && filter_provided_objects) ? NULL : wants, + wants_bitmap, filter); bitmap_git->result = wants_bitmap; bitmap_git->haves = haves_bitmap; diff --git a/pack-bitmap.h b/pack-bitmap.h index 78f2b3ff79..99d733eb26 100644 --- a/pack-bitmap.h +++ b/pack-bitmap.h @@ -52,7 +52,8 @@ void traverse_bitmap_commit_list(struct bitmap_index *, void test_bitmap_walk(struct rev_info *revs); int test_bitmap_commits(struct repository *r); struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, - struct list_objects_filter_options *filter); + struct list_objects_filter_options *filter, + int filter_provided_objects); int reuse_partial_packfile_from_bitmap(struct bitmap_index *, struct packed_git **packfile, uint32_t *entries, diff --git a/reachable.c b/reachable.c index 77a60c70a5..fc833cae43 100644 --- a/reachable.c +++ b/reachable.c @@ -223,7 +223,7 @@ void mark_reachable_objects(struct rev_info *revs, int mark_reflog, cp.progress = progress; cp.count = 0; - bitmap_git = prepare_bitmap_walk(revs, NULL); + bitmap_git = prepare_bitmap_walk(revs, NULL, 0); if (bitmap_git) { traverse_bitmap_commit_list(bitmap_git, revs, mark_object_seen); free_bitmap_index(bitmap_git); diff --git a/t/t6112-rev-list-filters-objects.sh b/t/t6112-rev-list-filters-objects.sh index c810a62807..4ade105db3 100755 --- a/t/t6112-rev-list-filters-objects.sh +++ b/t/t6112-rev-list-filters-objects.sh @@ -203,6 +203,34 @@ test_expect_success 'verify object:type=tag prints tag' ' test_cmp expected actual ' +test_expect_success 'verify object:type=blob prints only blob with --filter-provided-objects' ' + printf "%s blob\n" $(git -C object-type rev-parse HEAD:blob) >expected && + git -C object-type rev-list --objects \ + --filter=object:type=blob --filter-provided-objects HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tree prints only tree with --filter-provided-objects' ' + printf "%s \n" $(git -C object-type rev-parse HEAD^{tree}) >expected && + git -C object-type rev-list --objects \ + --filter=object:type=tree HEAD --filter-provided-objects >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=commit prints only commit with --filter-provided-objects' ' + git -C object-type rev-parse HEAD >expected && + git -C object-type rev-list --objects \ + --filter=object:type=commit --filter-provided-objects HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tag prints only tag with --filter-provided-objects' ' + printf "%s tag\n" $(git -C object-type rev-parse tag) >expected && + git -C object-type rev-list --objects \ + --filter=object:type=tag --filter-provided-objects tag >actual && + test_cmp expected actual +' + # Test sparse:path= filter. # !!!! # NOTE: sparse:path filter support has been dropped for security reasons, diff --git a/t/t6113-rev-list-bitmap-filters.sh b/t/t6113-rev-list-bitmap-filters.sh index cb9db7df6f..4d8e09167e 100755 --- a/t/t6113-rev-list-bitmap-filters.sh +++ b/t/t6113-rev-list-bitmap-filters.sh @@ -98,6 +98,28 @@ test_expect_success 'object:type filter' ' test_bitmap_traversal expect actual ' +test_expect_success 'object:type filter with --filter-provided-objects' ' + git rev-list --objects --filter-provided-objects --filter=object:type=tag tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided-objects --filter=object:type=tag tag >actual && + test_cmp expect actual && + + git rev-list --objects --filter-provided-objects --filter=object:type=commit tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided-objects --filter=object:type=commit tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter-provided-objects --filter=object:type=tree tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided-objects --filter=object:type=tree tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter-provided-objects --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided-objects --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual +' + test_expect_success 'combine filter' ' git rev-list --objects --filter=blob:limit=1000 --filter=object:type=blob tag >expect && git rev-list --use-bitmap-index \ @@ -105,4 +127,18 @@ test_expect_success 'combine filter' ' test_bitmap_traversal expect actual ' +test_expect_success 'combine filter with --filter-provided-objects' ' + git rev-list --objects --filter-provided-objects --filter=blob:limit=1000 --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided-objects --filter=blob:limit=1000 --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual && + + git cat-file --batch-check="%(objecttype) %(objectsize)" objects && + while read objecttype objectsize + do + test "$objecttype" = blob || return 1 + test "$objectsize" -le 1000 || return 1 + done