From patchwork Wed Jun 7 10:40:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270346 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 589F1C7EE23 for ; Wed, 7 Jun 2023 10:41:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239446AbjFGKlB (ORCPT ); Wed, 7 Jun 2023 06:41:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52848 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236749AbjFGKk6 (ORCPT ); Wed, 7 Jun 2023 06:40:58 -0400 Received: from mail-yw1-x1135.google.com (mail-yw1-x1135.google.com [IPv6:2607:f8b0:4864:20::1135]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 951CA1712 for ; Wed, 7 Jun 2023 03:40:57 -0700 (PDT) Received: by mail-yw1-x1135.google.com with SMTP id 00721157ae682-566586b180fso79906127b3.0 for ; Wed, 07 Jun 2023 03:40:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134456; x=1688726456; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=mxm4wZcZ0x73i7IXJKnida+6znYDlPCjYJcCLbVAnXM=; b=n8mKsPtnuWfpEQEvt5+kWHE6Z4fyW3kEYCp/cA6+7iV/LtfHK9+6smeCwDcHMiJDfy pBp8J3WbGIGKaKZQ2v9slCy02zsuiEGDs2PSG6RCSng9rJNfYmtEIwOIsa2bL8Z2/zbO lM09DZ5BtMElXbeo62ltA39SXHQ+Yjy/nPW1Al9NtXyQ+iyE+s4pyBt7/YWj0vNSwyjQ CZOScO6MXzNpYOMQHr7gFZaqJ78sTs2i7PgEbK/QQBLKMobSV3Mx+o27e5e93+WhENQM 0250a0Mz7AzSi7w9aLSll1trlrevhLHPkcSI9qFpfaivsWw+VeSIyTB7JswftN92MFse LHfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134456; x=1688726456; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=mxm4wZcZ0x73i7IXJKnida+6znYDlPCjYJcCLbVAnXM=; b=l9aftXOo3KGw06MLry/pDa9Wond10IHyvPXSs6QdzUxRxxZf4U9fpcJ7wsZK2J6Xk0 pdtHV4vSUrtK2l9sNXUth+xd1wJaf+WWKldogbQNrfXOvFlc1EJW0GeL4m45cMgKnI0v xypUHPdTSKzPdCA8IrB+JiIFgCWlOp0UUkLQwZdqiqPy/HTfYdME+xWRqLa3SYMGI54+ gHaYublv400gS+sI+DjIrGS7Brranjw0AqIrz8oub6MMGiW/Y1tholQjkzg8QEJj6465 t7IlN5NeBMLijy6g007pZrfjxDMzprtviaX3jHn3UMtUEzhyWjHQWp48amPt2yYpD+N9 knxA== X-Gm-Message-State: AC+VfDx+ar+kKZrd6ipXx0TXCef+ONzXlglQN8HBiSaA1jBY5d6zdkjz SSUn6K7slsLqDc4EcKWvXEOebK0s83E4gtWTtupWj64N X-Google-Smtp-Source: ACHHUZ7TiaWqNIjk6V9R/YMfmPRIszxTn6r9ScGzemx2aDQBtRPgaxsBqn5y12E6VV2N9B9oQlQz8A== X-Received: by 2002:a81:6d4a:0:b0:565:798b:949e with SMTP id i71-20020a816d4a000000b00565798b949emr6437716ywc.20.1686134456634; Wed, 07 Jun 2023 03:40:56 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id y184-20020a0defc1000000b0056944c0b016sm959258ywe.76.2023.06.07.03.40.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:40:56 -0700 (PDT) Date: Wed, 7 Jun 2023 06:40:53 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 01/16] refs.c: rename `ref_filter` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King The refs machinery has its own implementation of a `ref_filter` (used by `for-each-ref`), which is distinct from the `ref-filler.h` API (also used by `for-each-ref`, among other things). Rename the one within refs.c to more clearly indicate its purpose. Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- refs.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/refs.c b/refs.c index d2a98e1c21..b9b77d2eff 100644 --- a/refs.c +++ b/refs.c @@ -375,8 +375,8 @@ char *resolve_refdup(const char *refname, int resolve_flags, oid, flags); } -/* The argument to filter_refs */ -struct ref_filter { +/* The argument to for_each_filter_refs */ +struct for_each_ref_filter { const char *pattern; const char *prefix; each_ref_fn *fn; @@ -409,10 +409,11 @@ int ref_exists(const char *refname) return refs_ref_exists(get_main_ref_store(the_repository), refname); } -static int filter_refs(const char *refname, const struct object_id *oid, - int flags, void *data) +static int for_each_filter_refs(const char *refname, + const struct object_id *oid, + int flags, void *data) { - struct ref_filter *filter = (struct ref_filter *)data; + struct for_each_ref_filter *filter = data; if (wildmatch(filter->pattern, refname, 0)) return 0; @@ -569,7 +570,7 @@ int for_each_glob_ref_in(each_ref_fn fn, const char *pattern, const char *prefix, void *cb_data) { struct strbuf real_pattern = STRBUF_INIT; - struct ref_filter filter; + struct for_each_ref_filter filter; int ret; if (!prefix && !starts_with(pattern, "refs/")) @@ -589,7 +590,7 @@ int for_each_glob_ref_in(each_ref_fn fn, const char *pattern, filter.prefix = prefix; filter.fn = fn; filter.cb_data = cb_data; - ret = for_each_ref(filter_refs, &filter); + ret = for_each_ref(for_each_filter_refs, &filter); strbuf_release(&real_pattern); return ret; From patchwork Wed Jun 7 10:40:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52087C77B7A for ; Wed, 7 Jun 2023 10:41:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239500AbjFGKlI (ORCPT ); Wed, 7 Jun 2023 06:41:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239278AbjFGKlF (ORCPT ); Wed, 7 Jun 2023 06:41:05 -0400 Received: from mail-yb1-xb2c.google.com (mail-yb1-xb2c.google.com [IPv6:2607:f8b0:4864:20::b2c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B3E9E1BE3 for ; Wed, 7 Jun 2023 03:41:02 -0700 (PDT) Received: by mail-yb1-xb2c.google.com with SMTP id 3f1490d57ef6-bb1f7c5495dso8143983276.3 for ; Wed, 07 Jun 2023 03:41:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134462; x=1688726462; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=jX0N8JFZbxglHUJIDDeEixEf5mA1yVKgPVZm5bHwtyo=; b=AuMrgRw5MhFMTwO5hPcD76fL2J7peUnqWUed7sq41tQX+OSiGgNMqT5Y0dmdUSy+Nx FhMnhWuc63QOzVUVZSq3qu9gjy8rjM+uWfVnvAr8edRAubfkseuhoszxUQbbjOA7hPgP /3KLakbJB2Z58TSEgE6mrkuGL6tvPDkXFb89MeS0r56OSe4E+o2FJbjd82NIfFO+rwNE +qF9xKeSoO8waE3EPZxjuX3rNlXcGas9pJG6/sp6nGyeZiCBHR3SYzv9iA+9uaFVDvZT 7jfhcLGJby29I7CbT1/MO/tfxvi2OlQ3IrJwb1rRo3lPH/wluRU8WBZE2BbOYAmHPTK2 ipMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134462; x=1688726462; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=jX0N8JFZbxglHUJIDDeEixEf5mA1yVKgPVZm5bHwtyo=; b=JELx+9FVU9eJsRt7d0js16f3UFdSp5o+Vi7mASjlO0FpU3/nXOdQXbLUs3u9GGL4v5 4XQQo6Ys08byl3oG5kidTE7o0Suw9GVKqja39024EfE02Pzxde9qJtctRxqAFCthAYdL l9j4KoJUpq9iRwFkPr6T7QWIeRYDsESyxVzbOh/QYjecxIW2RfucoeKOMf40MXUEfUxi hCi66E5pMLmj46F3My8Ullc6ZTqbCFqnEys4POazVuwvDUGLPmWjHs728HNr9QQ4g9oD mQYkXUjZlK0yE3SWX0xYEK06ZNw6p9uFLTxgzgbtbzRFNZo1gsA4ly8s9b+GZ3Ut8b5C cL6Q== X-Gm-Message-State: AC+VfDyeAKf+cXTLXq18dwtw9hTUZFqmCptKWCOQXzLSUg9zBBBO2+pW nx88dqFXSpjPlYTwLNxjqxnLkFz1Htf+QNis5Sc0t8nZ X-Google-Smtp-Source: ACHHUZ7p9q9GJNksYmgUDwYNIz32XpQ/HdvJM334tFFRao8eou8nmNUrgyQnqWUmaMfx7Pv9tiGaow== X-Received: by 2002:a25:4cc3:0:b0:bac:26d2:53dd with SMTP id z186-20020a254cc3000000b00bac26d253ddmr4117285yba.43.1686134461599; Wed, 07 Jun 2023 03:41:01 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id p184-20020a25d8c1000000b00bb188e29af3sm3762986ybg.22.2023.06.07.03.41.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:41:01 -0700 (PDT) Date: Wed, 7 Jun 2023 06:40:58 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 02/16] ref-filter.h: provide `REF_FILTER_INIT` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King Provide a sane initialization value for `struct ref_filter`, which in a subsequent patch will be used to initialize a new field. In the meantime, fix a case in test-reach.c where its `ref_filter` is not even zero-initialized. Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- builtin/branch.c | 3 +-- builtin/for-each-ref.c | 3 +-- builtin/tag.c | 3 +-- ref-filter.h | 3 +++ t/helper/test-reach.c | 2 +- 5 files changed, 7 insertions(+), 7 deletions(-) diff --git a/builtin/branch.c b/builtin/branch.c index e6c2655af6..7891dec361 100644 --- a/builtin/branch.c +++ b/builtin/branch.c @@ -707,7 +707,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix) int reflog = 0, quiet = 0, icase = 0, force = 0, recurse_submodules_explicit = 0; enum branch_track track; - struct ref_filter filter; + struct ref_filter filter = REF_FILTER_INIT; static struct ref_sorting *sorting; struct string_list sorting_options = STRING_LIST_INIT_DUP; struct ref_format format = REF_FORMAT_INIT; @@ -765,7 +765,6 @@ int cmd_branch(int argc, const char **argv, const char *prefix) setup_ref_filter_porcelain_msg(); - memset(&filter, 0, sizeof(filter)); filter.kind = FILTER_REFS_BRANCHES; filter.abbrev = -1; diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c index 695fc8f4a5..99ccb73518 100644 --- a/builtin/for-each-ref.c +++ b/builtin/for-each-ref.c @@ -24,7 +24,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) struct string_list sorting_options = STRING_LIST_INIT_DUP; int maxcount = 0, icase = 0, omit_empty = 0; struct ref_array array; - struct ref_filter filter; + struct ref_filter filter = REF_FILTER_INIT; struct ref_format format = REF_FORMAT_INIT; struct strbuf output = STRBUF_INIT; struct strbuf err = STRBUF_INIT; @@ -61,7 +61,6 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) }; memset(&array, 0, sizeof(array)); - memset(&filter, 0, sizeof(filter)); format.format = "%(objectname) %(objecttype)\t%(refname)"; diff --git a/builtin/tag.c b/builtin/tag.c index 1850a6a6fd..6b41bb7374 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -443,7 +443,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix) struct msg_arg msg = { .buf = STRBUF_INIT }; struct ref_transaction *transaction; struct strbuf err = STRBUF_INIT; - struct ref_filter filter; + struct ref_filter filter = REF_FILTER_INIT; struct ref_sorting *sorting; struct string_list sorting_options = STRING_LIST_INIT_DUP; struct ref_format format = REF_FORMAT_INIT; @@ -501,7 +501,6 @@ int cmd_tag(int argc, const char **argv, const char *prefix) git_config(git_tag_config, &sorting_options); memset(&opt, 0, sizeof(opt)); - memset(&filter, 0, sizeof(filter)); filter.lines = -1; opt.sign = -1; diff --git a/ref-filter.h b/ref-filter.h index 430701cfb7..a920f73b29 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -92,6 +92,9 @@ struct ref_format { struct string_list bases; }; +#define REF_FILTER_INIT { \ + .points_at = OID_ARRAY_INIT, \ +} #define REF_FORMAT_INIT { \ .use_color = -1, \ .bases = STRING_LIST_INIT_DUP, \ diff --git a/t/helper/test-reach.c b/t/helper/test-reach.c index 5b6f217441..ef58f10c2d 100644 --- a/t/helper/test-reach.c +++ b/t/helper/test-reach.c @@ -139,7 +139,7 @@ int cmd__reach(int ac, const char **av) printf("%s(X,_,_,0,0):%d\n", av[1], can_all_from_reach_with_flag(&X_obj, 2, 4, 0, 0)); } else if (!strcmp(av[1], "commit_contains")) { - struct ref_filter filter; + struct ref_filter filter = REF_FILTER_INIT; struct contains_cache cache; init_contains_cache(&cache); From patchwork Wed Jun 7 10:41:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270348 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE0DAC77B7A for ; Wed, 7 Jun 2023 10:41:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239516AbjFGKlO (ORCPT ); Wed, 7 Jun 2023 06:41:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52980 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239569AbjFGKlJ (ORCPT ); Wed, 7 Jun 2023 06:41:09 -0400 Received: from mail-yw1-x1134.google.com (mail-yw1-x1134.google.com [IPv6:2607:f8b0:4864:20::1134]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A427A1BFB for ; Wed, 7 Jun 2023 03:41:07 -0700 (PDT) Received: by mail-yw1-x1134.google.com with SMTP id 00721157ae682-565aa2cc428so66703287b3.1 for ; Wed, 07 Jun 2023 03:41:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134466; x=1688726466; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Idaw3A+j8Lr4A1hR/sp3wu4pqRNzzh+DRmE9v8IG7NY=; b=hP7HTzx/5mexgggf+jR9A+b6yuHpV3nt0DCJyG5C4tt1bEh6H67kIfz15CCCkwe81r 3SiUfYZK9AqapgNJJc3UKwnyAR84Bv2GOG1n653BA9cbzoGMYF1Nj2X/Rp5rhDLvEHp9 YBwFdX331yWZMYY9abU4aHTD0IadI8Dc5T5lnaY7tnZs38JJn3otZttvqfWh5Kaa9yfV rspOxrgRBmro6oRCBxkxS9zw1Cl314CkDsWPqCuSTGziK2YNb6sf/M5Evj0cFWbjvVOp TyCHE9/r+PtSWBGztbSwG8P70Uge7rmptlugfCcmCguzOgPJ1/ZGBdu8uzgPE+7w5pQQ EQ7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134466; x=1688726466; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Idaw3A+j8Lr4A1hR/sp3wu4pqRNzzh+DRmE9v8IG7NY=; b=hYb+2X+pvFT8R8MHp7N7OGbmGWJjYZEPBlihrWgqej42TNOIIjoEz8r7ihQx57yN8w L+Nwzpy0YLKaq+zvHlq46mmKs/wWsh26t9PHB6ZYS9+FnJWQC6wzhVWtVgzNK0zTIvY+ o4uEUiPKYiW0VCFRKrRH3a5BZYD+Q3eI1KDCMWrkYndyIDlpNTZkSgXCK0jei0wnl1W2 /1XEXRU9kP/nNJ8mpqZ9sNexUmXB8MczCntUhjNcRAsaWfbm3UdowkVNJs0t0VcwylpD VNYVahp5BYCMGgB/EW3S1hPhcm4GvVK8B4skHjsD4BeJjBmoybxkPJ7VC+3FbYjXGRxe k1rg== X-Gm-Message-State: AC+VfDwyzxSUklhyvpmhofTzDTg4f8t+U/S4NDqJkFSonde2wATaXC2T LU4DAnXnu4Rrd5kAEHJRLUQopEsMzLBhEa9EwKYb8c3k X-Google-Smtp-Source: ACHHUZ4ve+KJBuBLcZuD3f0oZnwof00YzkTOFfmVTyAHbSwq/qW5K5RsKBct04mhUTT8+Z9VXF0NxA== X-Received: by 2002:a0d:df90:0:b0:55a:6100:c0e6 with SMTP id i138-20020a0ddf90000000b0055a6100c0e6mr5347852ywe.47.1686134466582; Wed, 07 Jun 2023 03:41:06 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id t197-20020a8183ce000000b005619cfb1b88sm4720965ywf.52.2023.06.07.03.41.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:41:06 -0700 (PDT) Date: Wed, 7 Jun 2023 06:41:03 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 03/16] ref-filter: clear reachable list pointers after freeing Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King In reach_filter(), we pop all commits from the reachable lists, leaving them empty. But because we're operating on a list pointer that was passed by value, the original filter.reachable_from pointer is left dangling. Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- ref-filter.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 4991cd4f7a..048d277cbf 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2418,13 +2418,13 @@ void ref_array_clear(struct ref_array *array) #define EXCLUDE_REACHED 0 #define INCLUDE_REACHED 1 static void reach_filter(struct ref_array *array, - struct commit_list *check_reachable, + struct commit_list **check_reachable, int include_reached) { int i, old_nr; struct commit **to_clear; - if (!check_reachable) + if (!*check_reachable) return; CALLOC_ARRAY(to_clear, array->nr); @@ -2434,7 +2434,7 @@ static void reach_filter(struct ref_array *array, } tips_reachable_from_bases(the_repository, - check_reachable, + *check_reachable, to_clear, array->nr, UNINTERESTING); @@ -2455,8 +2455,8 @@ static void reach_filter(struct ref_array *array, clear_commit_marks_many(old_nr, to_clear, ALL_REV_FLAGS); - while (check_reachable) { - struct commit *merge_commit = pop_commit(&check_reachable); + while (*check_reachable) { + struct commit *merge_commit = pop_commit(check_reachable); clear_commit_marks(merge_commit, ALL_REV_FLAGS); } @@ -2553,8 +2553,8 @@ int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int clear_contains_cache(&ref_cbdata.no_contains_cache); /* Filters that need revision walking */ - reach_filter(array, filter->reachable_from, INCLUDE_REACHED); - reach_filter(array, filter->unreachable_from, EXCLUDE_REACHED); + reach_filter(array, &filter->reachable_from, INCLUDE_REACHED); + reach_filter(array, &filter->unreachable_from, EXCLUDE_REACHED); save_commit_buffer = save_commit_buffer_orig; return ret; From patchwork Wed Jun 7 10:41:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEAD8C77B7A for ; Wed, 7 Jun 2023 10:41:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239548AbjFGKlT (ORCPT ); Wed, 7 Jun 2023 06:41:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239522AbjFGKlO (ORCPT ); Wed, 7 Jun 2023 06:41:14 -0400 Received: from mail-yb1-xb2f.google.com (mail-yb1-xb2f.google.com [IPv6:2607:f8b0:4864:20::b2f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA9FB1BDA for ; Wed, 7 Jun 2023 03:41:12 -0700 (PDT) Received: by mail-yb1-xb2f.google.com with SMTP id 3f1490d57ef6-bacfb7acdb7so8670489276.0 for ; Wed, 07 Jun 2023 03:41:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134472; x=1688726472; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=YHfcvs8EUcosgL3leePNnuehqaeKMvNIAHrpFnBigo4=; b=YiudMEcOaK10ZZUCI7qK+x65kcAG7ev8RsYWxUdkKt8icVfn6PbwjNAQOwrazpzG9I uQxLk2ETclON2sZGSNgjfi6VIUmgOQHey/nbLMqIfrHd9XSucjSzBEekcKOXk3Xo9tz8 7Obwd3p4VAxm4/2XOBksIVlsIiE3fZ3LWNCRgL+nOiCRzYDIJPvijNPsKYQkRMsYcWmm hPSvwcP9bcZcVclweq4tPpywbHLgCBSo7JCD+RTZk6CshD+i8VmzOlTh0J8dPmqTf1v1 37v+LMYrSMGJbp6smGY8DcAfhenzdQM0jtidiPHvHQUmHg9C/lOtEwvRaVEAeWssVhjT JKhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134472; x=1688726472; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=YHfcvs8EUcosgL3leePNnuehqaeKMvNIAHrpFnBigo4=; b=RRtu9rpY+c9t7w429yzDuzbriNeHminS69PrNBweBW1+Ivo3x/MNtS/Mf3lCPalftz +m51x56hYmS12gnQqUVSOLSwEuRSTXISzbVGBsuDyTjozQeQG7pWWPgQ/848by4qI7WR u4Q0qqqYB7wlak8sdqBnK5CGTfdVEZsST2jsccq+JSMLLr7V7dY2f6xCZcn5i0VwBY/x iDg+bh6PnWbDZUeP/ia/Ga3tp6RiyV2Ye0bQrk94sXXbI8VGqAn28Hb2F8yzegEgI2Ne hWPYeEbeeqrrsD4ATveLFfgZyccHjnE1RSVeTau2XNoX6lz9m7LiXe6CZQMgruuakL3t 0V7A== X-Gm-Message-State: AC+VfDxWt/ag/FZEsiX/bBpYF7Cvnl/oAbqLX8rgJSn6hLbnIgTJTULt gd0KSFC6ornDOKu5HLLhx+l23BbYI81y36AA8ETIykA0 X-Google-Smtp-Source: ACHHUZ74hQ3EbmUv+6oXgET+N/3O4Udf/gW6+PbB1WVdWiCrHBdW1khcQKqFlo4p3H7L5f0LjRjw8Q== X-Received: by 2002:a25:d7c5:0:b0:b9e:d97:e34b with SMTP id o188-20020a25d7c5000000b00b9e0d97e34bmr5693325ybg.50.1686134471431; Wed, 07 Jun 2023 03:41:11 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id k84-20020a252457000000b00bb131cd635bsm3798851ybk.8.2023.06.07.03.41.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:41:11 -0700 (PDT) Date: Wed, 7 Jun 2023 06:41:08 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 04/16] ref-filter: add `ref_filter_clear()` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King We did not bother to clean up at all in `git branch` or `git tag`, and `git for-each-ref` only cleans up a couple of members. Add and call `ref_filter_clear()` when cleaning up a `struct ref_filter`. Running this patch (without any test changes) indicates a couple of now leak-free tests. This was found by running: $ make SANITIZE=leak $ make -C t GIT_TEST_PASSING_SANITIZE_LEAK=check GIT_TEST_OPTS=--immediate (Note that the `reachable_from` and `unreachable_from` lists should be cleaned as they are used. So this is just covering any case where we might bail before running the reachability check.) Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- builtin/branch.c | 1 + builtin/for-each-ref.c | 3 +-- builtin/tag.c | 1 + ref-filter.c | 16 ++++++++++++++++ ref-filter.h | 3 +++ t/t0041-usage.sh | 1 + t/t3402-rebase-merge.sh | 1 + 7 files changed, 24 insertions(+), 2 deletions(-) diff --git a/builtin/branch.c b/builtin/branch.c index 7891dec361..07ee874617 100644 --- a/builtin/branch.c +++ b/builtin/branch.c @@ -858,6 +858,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix) print_columns(&output, colopts, NULL); string_list_clear(&output, 0); ref_sorting_release(sorting); + ref_filter_clear(&filter); return 0; } else if (edit_description) { const char *branch_name; diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c index 99ccb73518..c01fa6fefe 100644 --- a/builtin/for-each-ref.c +++ b/builtin/for-each-ref.c @@ -120,8 +120,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) strbuf_release(&err); strbuf_release(&output); ref_array_clear(&array); - free_commit_list(filter.with_commit); - free_commit_list(filter.no_commit); + ref_filter_clear(&filter); ref_sorting_release(sorting); strvec_clear(&vec); return 0; diff --git a/builtin/tag.c b/builtin/tag.c index 6b41bb7374..aab5e693fe 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -645,6 +645,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix) cleanup: ref_sorting_release(sorting); + ref_filter_clear(&filter); strbuf_release(&buf); strbuf_release(&ref); strbuf_release(&reflog_msg); diff --git a/ref-filter.c b/ref-filter.c index 048d277cbf..d32f426898 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2866,3 +2866,19 @@ int parse_opt_merge_filter(const struct option *opt, const char *arg, int unset) return 0; } + +void ref_filter_init(struct ref_filter *filter) +{ + struct ref_filter blank = REF_FILTER_INIT; + memcpy(filter, &blank, sizeof(blank)); +} + +void ref_filter_clear(struct ref_filter *filter) +{ + oid_array_clear(&filter->points_at); + free_commit_list(filter->with_commit); + free_commit_list(filter->no_commit); + free_commit_list(filter->reachable_from); + free_commit_list(filter->unreachable_from); + ref_filter_init(filter); +} diff --git a/ref-filter.h b/ref-filter.h index a920f73b29..160b807224 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -170,4 +170,7 @@ void filter_ahead_behind(struct repository *r, struct ref_format *format, struct ref_array *array); +void ref_filter_init(struct ref_filter *filter); +void ref_filter_clear(struct ref_filter *filter); + #endif /* REF_FILTER_H */ diff --git a/t/t0041-usage.sh b/t/t0041-usage.sh index c4fc34eb18..9ea974b0c6 100755 --- a/t/t0041-usage.sh +++ b/t/t0041-usage.sh @@ -5,6 +5,7 @@ test_description='Test commands behavior when given invalid argument value' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success 'setup ' ' diff --git a/t/t3402-rebase-merge.sh b/t/t3402-rebase-merge.sh index 79b0640c00..e9e03ca4b5 100755 --- a/t/t3402-rebase-merge.sh +++ b/t/t3402-rebase-merge.sh @@ -8,6 +8,7 @@ test_description='git rebase --merge test' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh T="A quick brown fox From patchwork Wed Jun 7 10:41:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270350 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3956CC7EE2E for ; Wed, 7 Jun 2023 10:41:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239513AbjFGKlV (ORCPT ); Wed, 7 Jun 2023 06:41:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53144 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239555AbjFGKlS (ORCPT ); Wed, 7 Jun 2023 06:41:18 -0400 Received: from mail-yw1-x1129.google.com (mail-yw1-x1129.google.com [IPv6:2607:f8b0:4864:20::1129]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6271C1BEA for ; Wed, 7 Jun 2023 03:41:17 -0700 (PDT) Received: by mail-yw1-x1129.google.com with SMTP id 00721157ae682-565c9109167so66716857b3.2 for ; Wed, 07 Jun 2023 03:41:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134476; x=1688726476; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=0Z9oMo7gji4/HCNUlzvLTKEiXc1PfBMlilRXhV63s74=; b=IJdC+LKHyAep+HUJZKzwxqZZYxrifQTnjc2zYgD2OnmyKkOFdwKoCHolDmzN6tjXeo tEODGdBnxQYvfu4Szp7Bs2PmN6JJ7CWdZeHs/RSN+eXPxOKghEP5DS+hhDkVzoYixVxD VrnJz8elizVR2M/NJ7BHmFPE5FyoX5pzCFS9kYo1ST2ITmN/sekbpICz0tR1LenLm+E+ BuUmIuQPIyoQilNtzSAPs81p3YAoKtT4LOWvfoadBdf1HkPvAX/Web5GfLcoLI4nB4GJ 0djX5N9m3rRBTjG1vpJ226RONetTX3WEjzZs3xQV67tGApOFJL+OerOq6zJa62P9Usyy CN3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134476; x=1688726476; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=0Z9oMo7gji4/HCNUlzvLTKEiXc1PfBMlilRXhV63s74=; b=VRgEWdbs/WtxlyM+qt2Jc06lTfcB2OUSe7Tc6zuXl1RrI6ci9QNiRAzmcIYWbbh4zR 7XwUNAkQqtjy9Kb3019ikHwmjRcE3Vt+MM+4FQDaqdz2OAPxQ8cRAs5jqlCeCoMaFoIC bipjNBJajUdpUTFTjseAkopjvn0RO0IWuj/3jZDXvKiFVovDEzIE7XhX7ZzpfjN/rh17 VojDHiyu+BUTPBFbmIKFLpFlm2TctrV7jko+X4SHV5kPoLZOZezpzt03e7yr2dguvnN2 dFCXAAslgAMjYdk77ZTYFgXUkbXveKMm3jDSyDjc3iQ9JPn3/z1bKOhCa6ip8/6KKEU5 6doQ== X-Gm-Message-State: AC+VfDzNk0UXq5Y7bm5upVgz8JmOzDAbeRzjNrRLgmM4uYEtse1Dg93u mRLBKu0/sldGwEed+/Cm5BaflaMXe9F8ennCgp0SY6vy X-Google-Smtp-Source: ACHHUZ4O7Z94OMMl34PaznJKFW4rC0/glpsGJzVW4X3FF+tBWJt7HsasoHxsnhxj3g2wLXJ6n4g65Q== X-Received: by 2002:a0d:cb47:0:b0:565:c96b:f526 with SMTP id n68-20020a0dcb47000000b00565c96bf526mr5526431ywd.19.1686134476403; Wed, 07 Jun 2023 03:41:16 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id p17-20020a25b511000000b00b9e5b28dbd8sm3771833ybj.17.2023.06.07.03.41.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:41:16 -0700 (PDT) Date: Wed, 7 Jun 2023 06:41:13 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 05/16] ref-filter.c: parameterize match functions over patterns Message-ID: <1988ca4c0a79065a1893259da8abe63919e05cf3.1686134440.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King `match_pattern()` and `match_name_as_path()` both take a `struct ref_filter *`, and then store a stack variable `patterns` pointing at `filter->patterns`. The subsequent patch will add a new array of patterns to match over (the excluded patterns, via a new `git for-each-ref --exclude` option), treating the return value of these functions differently depending on which patterns are being used to match. Tweak `match_pattern()` and `match_name_as_path()` to take an array of patterns to prepare for passing either in. Once we start passing either in, `match_pattern()` will have little to do with a particular `struct ref_filter *` instance. To clarify this, drop it from the argument list, and replace it with the only bit of the `ref_filter` that we care about (`filter->ignore_case`). Co-authored-by: Taylor Blau Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- ref-filter.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index d32f426898..6d91c7cb0d 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2104,12 +2104,12 @@ static int get_ref_atom_value(struct ref_array_item *ref, int atom, * matches a pattern "refs/heads/mas") or a wildcard (e.g. the same ref * matches "refs/heads/mas*", too). */ -static int match_pattern(const struct ref_filter *filter, const char *refname) +static int match_pattern(const char **patterns, const char *refname, + const int ignore_case) { - const char **patterns = filter->name_patterns; unsigned flags = 0; - if (filter->ignore_case) + if (ignore_case) flags |= WM_CASEFOLD; /* @@ -2134,9 +2134,10 @@ static int match_pattern(const struct ref_filter *filter, const char *refname) * matches a pattern "refs/heads/" but not "refs/heads/m") or a * wildcard (e.g. the same ref matches "refs/heads/m*", too). */ -static int match_name_as_path(const struct ref_filter *filter, const char *refname) +static int match_name_as_path(const struct ref_filter *filter, + const char **pattern, + const char *refname) { - const char **pattern = filter->name_patterns; int namelen = strlen(refname); unsigned flags = WM_PATHNAME; @@ -2165,8 +2166,9 @@ static int filter_pattern_match(struct ref_filter *filter, const char *refname) if (!*filter->name_patterns) return 1; /* No pattern always matches */ if (filter->match_as_path) - return match_name_as_path(filter, refname); - return match_pattern(filter, refname); + return match_name_as_path(filter, filter->name_patterns, refname); + return match_pattern(filter->name_patterns, refname, + filter->ignore_case); } /* From patchwork Wed Jun 7 10:41:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DDDDC77B7A for ; Wed, 7 Jun 2023 10:41:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239592AbjFGKlc (ORCPT ); Wed, 7 Jun 2023 06:41:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53310 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235774AbjFGKl2 (ORCPT ); Wed, 7 Jun 2023 06:41:28 -0400 Received: from mail-yw1-x1133.google.com (mail-yw1-x1133.google.com [IPv6:2607:f8b0:4864:20::1133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96E191BF1 for ; Wed, 7 Jun 2023 03:41:22 -0700 (PDT) Received: by mail-yw1-x1133.google.com with SMTP id 00721157ae682-565a022ef06so89217577b3.3 for ; Wed, 07 Jun 2023 03:41:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134481; x=1688726481; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=pxUhHxgCb3k8K3Mm4mImHmG5APfQpZgBAyrHM4+IEvg=; b=oXyyXDET6akHsqfVUFlUPdJZ3dw5ttem7YOFzz06DOc+D3WUa1f6cunuGni7jzf3VM buN7QxXrY9VMlBTshStsdtwCL3MozHDr2w1gsNEx5mlK8qdikFn7GFx8/HEJomE5/kQs 9M1rEJFGx0JtdyjAe5nSsxusks4e4TV6i3S2G2BI6goHPhMJ/MZLVp1l7nQKJs40sxS7 qNjkkQdypsCr1Sfx84FGDbTIgGh9ok4j/mRDtRwVRIGARPWPZjhDs7pxg+YchXpidX7n FqDgSQJdWyuhMP6+Vf25DkbCNiqfgsgY3vsHUKH/aHy6eNJ3oQyRUEzQGH7vYMuJutd5 ZiCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134481; x=1688726481; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=pxUhHxgCb3k8K3Mm4mImHmG5APfQpZgBAyrHM4+IEvg=; b=IqY+dVtj26U6Nabdi3S3lNR99hLZ+ouLB3xU0yxf9hW84bEM8564ODZ8S/JhgSS128 9RTmMcs2INbTrWd+hdGvWgYtK5StLyJhoY7jquECF2wbdDOIetsZox0DytiA6ZHKilPw t1+ZRAVOcWuZrQNUrjbzw/DGXzlJjuMGZ7fqKEDBGpphpKQe6L2PE+SORL9Xj8/eeFE0 XsCNRXn69aC8zKKu1mYMCHny6LQSfXz7aQFWAcZapYzGd69H2qq9xWFLpYx+v5hxZRRJ f++chwtJB9T98o+qdrpPg0T4ukXqU0RYvy/g4Ce8bMYpYC7/ErCv8O1aD2PKCC+3abyJ Cceg== X-Gm-Message-State: AC+VfDy/G6HYOimC7f3Zv07hIGXP0NA0AmiLB7vT1jexU/DREKdVYoCD LT/fDSeASQwaQt1S3kZ2DFYHoTNUAnzQIVTvgSKARdkm X-Google-Smtp-Source: ACHHUZ5ldFyM3uext0e98xmH3SaRAXMBhVsTEI6sx5rl2shGvcKS6RjjdlAUCtRWfCFV9BwQ2NAqRw== X-Received: by 2002:a0d:e291:0:b0:569:ff01:41b8 with SMTP id l139-20020a0de291000000b00569ff0141b8mr3955963ywe.15.1686134481442; Wed, 07 Jun 2023 03:41:21 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id w205-20020a817bd6000000b005569567aac1sm4633473ywc.106.2023.06.07.03.41.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:41:21 -0700 (PDT) Date: Wed, 7 Jun 2023 06:41:18 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 06/16] builtin/for-each-ref.c: add `--exclude` option Message-ID: <60d85aa4ad10cf57c7694e6a07d6bf2d8f6984eb.1686134440.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When using `for-each-ref`, it is sometimes convenient for the caller to be able to exclude certain parts of the references. For example, if there are many `refs/__hidden__/*` references, the caller may want to emit all references *except* the hidden ones. Currently, the only way to do this is to post-process the output, like: $ git for-each-ref --format='%(refname)' | grep -v '^refs/hidden/' Which is do-able, but requires processing a potentially large quantity of references. Teach `git for-each-ref` a new `--exclude=` option, which excludes references from the results if they match one or more excluded patterns. This patch provides a naive implementation where the `ref_filter` still sees all references (including ones that it will discard) and is left to check whether each reference matches any excluded pattern(s) before emitting them. By culling out references we know the caller doesn't care about, we can avoid allocating memory for their storage, as well as spending time sorting the output (among other things). Even the naive implementation provides a significant speed-up on a modified copy of linux.git (that has a hidden ref pointing at each commit): $ hyperfine \ 'git.compile for-each-ref --format="%(objectname) %(refname)" | grep -vE "[0-9a-f]{40} refs/pull/"' \ 'git.compile for-each-ref --format="%(objectname) %(refname)" --exclude refs/pull/' Benchmark 1: git.compile for-each-ref --format="%(objectname) %(refname)" | grep -vE "[0-9a-f]{40} refs/pull/" Time (mean ± σ): 820.1 ms ± 2.0 ms [User: 703.7 ms, System: 152.0 ms] Range (min … max): 817.7 ms … 823.3 ms 10 runs Benchmark 2: git.compile for-each-ref --format="%(objectname) %(refname)" --exclude refs/pull/ Time (mean ± σ): 106.6 ms ± 1.1 ms [User: 99.4 ms, System: 7.1 ms] Range (min … max): 104.7 ms … 109.1 ms 27 runs Summary 'git.compile for-each-ref --format="%(objectname) %(refname)" --exclude refs/pull/' ran 7.69 ± 0.08 times faster than 'git.compile for-each-ref --format="%(objectname) %(refname)" | grep -vE "[0-9a-f]{40} refs/pull/"' Subsequent patches will improve on this by avoiding visiting excluded sections of the `packed-refs` file in certain cases. Co-authored-by: Jeff King Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- Documentation/git-for-each-ref.txt | 6 +++++ builtin/for-each-ref.c | 1 + ref-filter.c | 13 +++++++++++ ref-filter.h | 6 +++++ t/t6300-for-each-ref.sh | 35 ++++++++++++++++++++++++++++++ 5 files changed, 61 insertions(+) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index 1e215d4e73..5743eb5def 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -14,6 +14,7 @@ SYNOPSIS [--points-at=] [--merged[=]] [--no-merged[=]] [--contains[=]] [--no-contains[=]] + [--exclude= ...] DESCRIPTION ----------- @@ -102,6 +103,11 @@ OPTIONS Do not print a newline after formatted refs where the format expands to the empty string. +--exclude=:: + If one or more patterns are given, only refs which do not match + any excluded pattern(s) are shown. Matching is done using the + same rules as `` above. + FIELD NAMES ----------- diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c index c01fa6fefe..3384987428 100644 --- a/builtin/for-each-ref.c +++ b/builtin/for-each-ref.c @@ -47,6 +47,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) OPT_INTEGER( 0 , "count", &maxcount, N_("show only matched refs")), OPT_STRING( 0 , "format", &format.format, N_("format"), N_("format to use for the output")), OPT__COLOR(&format.use_color, N_("respect format colors")), + OPT_REF_FILTER_EXCLUDE(&filter), OPT_REF_SORT(&sorting_options), OPT_CALLBACK(0, "points-at", &filter.points_at, N_("object"), N_("print only refs which points at the given object"), diff --git a/ref-filter.c b/ref-filter.c index 6d91c7cb0d..d44418efb7 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2171,6 +2171,15 @@ static int filter_pattern_match(struct ref_filter *filter, const char *refname) filter->ignore_case); } +static int filter_exclude_match(struct ref_filter *filter, const char *refname) +{ + if (!filter->exclude.nr) + return 0; + if (filter->match_as_path) + return match_name_as_path(filter, filter->exclude.v, refname); + return match_pattern(filter->exclude.v, refname, filter->ignore_case); +} + /* * This is the same as for_each_fullref_in(), but it tries to iterate * only over the patterns we'll care about. Note that it _doesn't_ do a full @@ -2338,6 +2347,9 @@ static int ref_filter_handler(const char *refname, const struct object_id *oid, if (!filter_pattern_match(filter, refname)) return 0; + if (filter_exclude_match(filter, refname)) + return 0; + if (filter->points_at.nr && !match_points_at(&filter->points_at, oid, refname)) return 0; @@ -2877,6 +2889,7 @@ void ref_filter_init(struct ref_filter *filter) void ref_filter_clear(struct ref_filter *filter) { + strvec_clear(&filter->exclude); oid_array_clear(&filter->points_at); free_commit_list(filter->with_commit); free_commit_list(filter->no_commit); diff --git a/ref-filter.h b/ref-filter.h index 160b807224..1524bc463a 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -6,6 +6,7 @@ #include "refs.h" #include "commit.h" #include "string-list.h" +#include "strvec.h" /* Quoting styles */ #define QUOTE_NONE 0 @@ -59,6 +60,7 @@ struct ref_array { struct ref_filter { const char **name_patterns; + struct strvec exclude; struct oid_array points_at; struct commit_list *with_commit; struct commit_list *no_commit; @@ -94,6 +96,7 @@ struct ref_format { #define REF_FILTER_INIT { \ .points_at = OID_ARRAY_INIT, \ + .exclude = STRVEC_INIT, \ } #define REF_FORMAT_INIT { \ .use_color = -1, \ @@ -112,6 +115,9 @@ struct ref_format { #define OPT_REF_SORT(var) \ OPT_STRING_LIST(0, "sort", (var), \ N_("key"), N_("field name to sort on")) +#define OPT_REF_FILTER_EXCLUDE(var) \ + OPT_STRVEC(0, "exclude", &(var)->exclude, \ + N_("pattern"), N_("exclude refs which match pattern")) /* * API for filtering a set of refs. Based on the type of refs the user diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 5c00607608..7e8d578522 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -447,6 +447,41 @@ test_expect_success 'exercise glob patterns with prefixes' ' test_cmp expected actual ' +cat >expected <<\EOF +refs/tags/bar +refs/tags/baz +refs/tags/testtag +EOF + +test_expect_success 'exercise patterns with prefix exclusions' ' + for tag in foo/one foo/two foo/three bar baz + do + git tag "$tag" || return 1 + done && + test_when_finished "git tag -d foo/one foo/two foo/three bar baz" && + git for-each-ref --format="%(refname)" \ + refs/tags/ --exclude=refs/tags/foo >actual && + test_cmp expected actual +' + +cat >expected <<\EOF +refs/tags/bar +refs/tags/baz +refs/tags/foo/one +refs/tags/testtag +EOF + +test_expect_success 'exercise patterns with pattern exclusions' ' + for tag in foo/one foo/two foo/three bar baz + do + git tag "$tag" || return 1 + done && + test_when_finished "git tag -d foo/one foo/two foo/three bar baz" && + git for-each-ref --format="%(refname)" \ + refs/tags/ --exclude="refs/tags/foo/t*" >actual && + test_cmp expected actual +' + cat >expected <<\EOF 'refs/heads/main' 'refs/remotes/origin/main' From patchwork Wed Jun 7 10:41:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270352 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 385C5C7EE23 for ; Wed, 7 Jun 2023 10:41:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235774AbjFGKlk (ORCPT ); Wed, 7 Jun 2023 06:41:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53310 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239130AbjFGKlb (ORCPT ); Wed, 7 Jun 2023 06:41:31 -0400 Received: from mail-yb1-xb29.google.com (mail-yb1-xb29.google.com [IPv6:2607:f8b0:4864:20::b29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8BE6A1BE2 for ; Wed, 7 Jun 2023 03:41:27 -0700 (PDT) Received: by mail-yb1-xb29.google.com with SMTP id 3f1490d57ef6-bad97da58adso9249839276.1 for ; Wed, 07 Jun 2023 03:41:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134486; x=1688726486; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=mC+dGnqGCQkw17BM9sVbyIYk2JERYIIcH2HsRZQNbFA=; b=lg4R02W1q10c3lN4Pnt2SN0BlibHoZZROuan6wc7siZmCt/pKtkGb5apdDfOOYxvSC tFt3/xPQo20ZJkT2Ml20G+/k2r9VHgQC/34+uDEP1rdqqEoKwS9Q7USFU91pU814HhF+ NW1HZ79Fcx5R9aNvpmJEf9QB5f4W2X2F1ntcWXKQo6L33eCKyhgwzxzed97dqfXc+1iU +D3DDk/Oxa0Mv03nWQb51kmHV0RzCZblv8ZCb+on7okdMyoAGm+IW1OuXtrGEuXjIzC6 Aw92hxMCXl/GKQpdtidDa6L9RvRgU86J8qMM0ZE+BUck7krhsx9oauXQ5ozP0oMBrbaA PbHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134486; x=1688726486; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=mC+dGnqGCQkw17BM9sVbyIYk2JERYIIcH2HsRZQNbFA=; b=J3bcNB2Cwvwt/lJAp83EJSTUPBiswd8a8+ziQFrKvQL1iPDZ4TTkjwoNmtXOKbAM1/ yzKTHp7ydseITjF7bsXf7PuV6LgBWp4B9x00T0wD1kx3k9BYG1ZgLC7zgi4Wb86mPVWv 0gPqdSuWvSK3LpeZx0kwlR/MaX+KvX0D6234wPYetz+m6FlY50EPm4PLbE03OaLxlSJO FfjfaRDiBv75XJnntqO47xRi2AFe96V85u1zn25zMYPglRQ42k89JGJG0fLcDhrvBuC/ rKL+tDekXxKNOC6W4IGel9PMYu5JhN51C2Xa0A7cFu0d4utaq85+xDhSqDEZcbDEETmn st3A== X-Gm-Message-State: AC+VfDwlw2qgTKFICC35CE/aDNU8LaaREN8ND2MGkATd8y83GlC4a5Sz h9i2HWHZKZPoj2sEJb0TtmMRsAmOUVT+0Bi1gi84BAjN X-Google-Smtp-Source: ACHHUZ5wB8XLxQXGooaNAEd36ez88nMncBPvYvUs41yoca0iCu2cRDXcl49SEdfH3a8IRFQvPmFeqA== X-Received: by 2002:a25:84c9:0:b0:ba9:6b90:e551 with SMTP id x9-20020a2584c9000000b00ba96b90e551mr6856824ybm.50.1686134486526; Wed, 07 Jun 2023 03:41:26 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id l197-20020a2525ce000000b00ba87bc06fe5sm3786916ybl.52.2023.06.07.03.41.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:41:26 -0700 (PDT) Date: Wed, 7 Jun 2023 06:41:23 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 07/16] refs: plumb `exclude_patterns` argument throughout Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The subsequent patch will want to access an optional `excluded_patterns` array within `refs/packed-backend.c` that will cull out certain references matching any of the given patterns on a best-effort basis. To do so, the refs subsystem needs to be updated to pass this value across a number of different locations. Prepare for a future patch by introducing this plumbing now, passing NULLs at top-level APIs in order to make that patch less noisy and more easily readable. Signed-off-by: Taylor Blau --- ls-refs.c | 2 +- ref-filter.c | 5 +++-- refs.c | 32 +++++++++++++++++++------------- refs.h | 8 +++++++- refs/debug.c | 5 +++-- refs/files-backend.c | 5 +++-- refs/packed-backend.c | 5 +++-- refs/refs-internal.h | 7 ++++--- revision.c | 2 +- 9 files changed, 44 insertions(+), 27 deletions(-) diff --git a/ls-refs.c b/ls-refs.c index f385938b64..6f490b2d9c 100644 --- a/ls-refs.c +++ b/ls-refs.c @@ -193,7 +193,7 @@ int ls_refs(struct repository *r, struct packet_reader *request) strvec_push(&data.prefixes, ""); refs_for_each_fullref_in_prefixes(get_main_ref_store(r), get_git_namespace(), data.prefixes.v, - send_ref, &data); + NULL, send_ref, &data); packet_fflush(stdout); strvec_clear(&data.prefixes); strbuf_release(&data.buf); diff --git a/ref-filter.c b/ref-filter.c index d44418efb7..717c3c4bcf 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2209,12 +2209,13 @@ static int for_each_fullref_in_pattern(struct ref_filter *filter, if (!filter->name_patterns[0]) { /* no patterns; we have to look at everything */ - return for_each_fullref_in("", cb, cb_data); + return refs_for_each_fullref_in(get_main_ref_store(the_repository), + "", NULL, cb, cb_data); } return refs_for_each_fullref_in_prefixes(get_main_ref_store(the_repository), NULL, filter->name_patterns, - cb, cb_data); + NULL, cb, cb_data); } /* diff --git a/refs.c b/refs.c index b9b77d2eff..538bde644e 100644 --- a/refs.c +++ b/refs.c @@ -1526,7 +1526,9 @@ int head_ref(each_ref_fn fn, void *cb_data) struct ref_iterator *refs_ref_iterator_begin( struct ref_store *refs, - const char *prefix, int trim, + const char *prefix, + const char **exclude_patterns, + int trim, enum do_for_each_ref_flags flags) { struct ref_iterator *iter; @@ -1542,8 +1544,7 @@ struct ref_iterator *refs_ref_iterator_begin( } } - iter = refs->be->iterator_begin(refs, prefix, flags); - + iter = refs->be->iterator_begin(refs, prefix, exclude_patterns, flags); /* * `iterator_begin()` already takes care of prefix, but we * might need to do some trimming: @@ -1577,7 +1578,7 @@ static int do_for_each_repo_ref(struct repository *r, const char *prefix, if (!refs) return 0; - iter = refs_ref_iterator_begin(refs, prefix, trim, flags); + iter = refs_ref_iterator_begin(refs, prefix, NULL, trim, flags); return do_for_each_repo_ref_iterator(r, iter, fn, cb_data); } @@ -1599,6 +1600,7 @@ static int do_for_each_ref_helper(struct repository *r, } static int do_for_each_ref(struct ref_store *refs, const char *prefix, + const char **exclude_patterns, each_ref_fn fn, int trim, enum do_for_each_ref_flags flags, void *cb_data) { @@ -1608,7 +1610,8 @@ static int do_for_each_ref(struct ref_store *refs, const char *prefix, if (!refs) return 0; - iter = refs_ref_iterator_begin(refs, prefix, trim, flags); + iter = refs_ref_iterator_begin(refs, prefix, exclude_patterns, trim, + flags); return do_for_each_repo_ref_iterator(the_repository, iter, do_for_each_ref_helper, &hp); @@ -1616,7 +1619,7 @@ static int do_for_each_ref(struct ref_store *refs, const char *prefix, int refs_for_each_ref(struct ref_store *refs, each_ref_fn fn, void *cb_data) { - return do_for_each_ref(refs, "", fn, 0, 0, cb_data); + return do_for_each_ref(refs, "", NULL, fn, 0, 0, cb_data); } int for_each_ref(each_ref_fn fn, void *cb_data) @@ -1627,7 +1630,7 @@ int for_each_ref(each_ref_fn fn, void *cb_data) int refs_for_each_ref_in(struct ref_store *refs, const char *prefix, each_ref_fn fn, void *cb_data) { - return do_for_each_ref(refs, prefix, fn, strlen(prefix), 0, cb_data); + return do_for_each_ref(refs, prefix, NULL, fn, strlen(prefix), 0, cb_data); } int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data) @@ -1638,13 +1641,14 @@ int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data) int for_each_fullref_in(const char *prefix, each_ref_fn fn, void *cb_data) { return do_for_each_ref(get_main_ref_store(the_repository), - prefix, fn, 0, 0, cb_data); + prefix, NULL, fn, 0, 0, cb_data); } int refs_for_each_fullref_in(struct ref_store *refs, const char *prefix, + const char **exclude_patterns, each_ref_fn fn, void *cb_data) { - return do_for_each_ref(refs, prefix, fn, 0, 0, cb_data); + return do_for_each_ref(refs, prefix, exclude_patterns, fn, 0, 0, cb_data); } int for_each_replace_ref(struct repository *r, each_repo_ref_fn fn, void *cb_data) @@ -1661,14 +1665,14 @@ int for_each_namespaced_ref(each_ref_fn fn, void *cb_data) int ret; strbuf_addf(&buf, "%srefs/", get_git_namespace()); ret = do_for_each_ref(get_main_ref_store(the_repository), - buf.buf, fn, 0, 0, cb_data); + buf.buf, NULL, fn, 0, 0, cb_data); strbuf_release(&buf); return ret; } int refs_for_each_rawref(struct ref_store *refs, each_ref_fn fn, void *cb_data) { - return do_for_each_ref(refs, "", fn, 0, + return do_for_each_ref(refs, "", NULL, fn, 0, DO_FOR_EACH_INCLUDE_BROKEN, cb_data); } @@ -1738,6 +1742,7 @@ static void find_longest_prefixes(struct string_list *out, int refs_for_each_fullref_in_prefixes(struct ref_store *ref_store, const char *namespace, const char **patterns, + const char **exclude_patterns, each_ref_fn fn, void *cb_data) { struct string_list prefixes = STRING_LIST_INIT_DUP; @@ -1753,7 +1758,8 @@ int refs_for_each_fullref_in_prefixes(struct ref_store *ref_store, for_each_string_list_item(prefix, &prefixes) { strbuf_addstr(&buf, prefix->string); - ret = refs_for_each_fullref_in(ref_store, buf.buf, fn, cb_data); + ret = refs_for_each_fullref_in(ref_store, buf.buf, + exclude_patterns, fn, cb_data); if (ret) break; strbuf_setlen(&buf, namespace_len); @@ -2408,7 +2414,7 @@ int refs_verify_refname_available(struct ref_store *refs, strbuf_addstr(&dirname, refname + dirname.len); strbuf_addch(&dirname, '/'); - iter = refs_ref_iterator_begin(refs, dirname.buf, 0, + iter = refs_ref_iterator_begin(refs, dirname.buf, NULL, 0, DO_FOR_EACH_INCLUDE_BROKEN); while ((ok = ref_iterator_advance(iter)) == ITER_OK) { if (skip && diff --git a/refs.h b/refs.h index 123cfa4424..d672d636cf 100644 --- a/refs.h +++ b/refs.h @@ -338,6 +338,7 @@ int for_each_ref(each_ref_fn fn, void *cb_data); int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data); int refs_for_each_fullref_in(struct ref_store *refs, const char *prefix, + const char **exclude_patterns, each_ref_fn fn, void *cb_data); int for_each_fullref_in(const char *prefix, each_ref_fn fn, void *cb_data); @@ -345,10 +346,15 @@ int for_each_fullref_in(const char *prefix, each_ref_fn fn, void *cb_data); * iterate all refs in "patterns" by partitioning patterns into disjoint sets * and iterating the longest-common prefix of each set. * + * references matching any pattern in "exclude_patterns" are omitted from the + * result set on a best-effort basis. + * * callers should be prepared to ignore references that they did not ask for. */ int refs_for_each_fullref_in_prefixes(struct ref_store *refs, - const char *namespace, const char **patterns, + const char *namespace, + const char **patterns, + const char **exclude_patterns, each_ref_fn fn, void *cb_data); /** diff --git a/refs/debug.c b/refs/debug.c index 6f11e6de46..328f894177 100644 --- a/refs/debug.c +++ b/refs/debug.c @@ -229,11 +229,12 @@ static struct ref_iterator_vtable debug_ref_iterator_vtable = { static struct ref_iterator * debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix, - unsigned int flags) + const char **exclude_patterns, unsigned int flags) { struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store; struct ref_iterator *res = - drefs->refs->be->iterator_begin(drefs->refs, prefix, flags); + drefs->refs->be->iterator_begin(drefs->refs, prefix, + exclude_patterns, flags); struct debug_ref_iterator *diter = xcalloc(1, sizeof(*diter)); base_ref_iterator_init(&diter->base, &debug_ref_iterator_vtable, 1); diter->iter = res; diff --git a/refs/files-backend.c b/refs/files-backend.c index bca7b851c5..3bc3c57c05 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -829,7 +829,8 @@ static struct ref_iterator_vtable files_ref_iterator_vtable = { static struct ref_iterator *files_ref_iterator_begin( struct ref_store *ref_store, - const char *prefix, unsigned int flags) + const char *prefix, const char **exclude_patterns, + unsigned int flags) { struct files_ref_store *refs; struct ref_iterator *loose_iter, *packed_iter, *overlay_iter; @@ -874,7 +875,7 @@ static struct ref_iterator *files_ref_iterator_begin( * the packed and loose references. */ packed_iter = refs_ref_iterator_begin( - refs->packed_ref_store, prefix, 0, + refs->packed_ref_store, prefix, exclude_patterns, 0, DO_FOR_EACH_INCLUDE_BROKEN); overlay_iter = overlay_ref_iterator_begin(loose_iter, packed_iter); diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 5b412a133b..176bd3905b 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -924,7 +924,8 @@ static struct ref_iterator_vtable packed_ref_iterator_vtable = { static struct ref_iterator *packed_ref_iterator_begin( struct ref_store *ref_store, - const char *prefix, unsigned int flags) + const char *prefix, const char **exclude_patterns, + unsigned int flags) { struct packed_ref_store *refs; struct snapshot *snapshot; @@ -1149,7 +1150,7 @@ static int write_with_updates(struct packed_ref_store *refs, * list of refs is exhausted, set iter to NULL. When the list * of updates is exhausted, leave i set to updates->nr. */ - iter = packed_ref_iterator_begin(&refs->base, "", + iter = packed_ref_iterator_begin(&refs->base, "", NULL, DO_FOR_EACH_INCLUDE_BROKEN); if ((ok = ref_iterator_advance(iter)) != ITER_OK) iter = NULL; diff --git a/refs/refs-internal.h b/refs/refs-internal.h index a85d113123..28a11b9d61 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -367,8 +367,8 @@ int is_empty_ref_iterator(struct ref_iterator *ref_iterator); */ struct ref_iterator *refs_ref_iterator_begin( struct ref_store *refs, - const char *prefix, int trim, - enum do_for_each_ref_flags flags); + const char *prefix, const char **exclude_patterns, + int trim, enum do_for_each_ref_flags flags); /* * A callback function used to instruct merge_ref_iterator how to @@ -570,7 +570,8 @@ typedef int copy_ref_fn(struct ref_store *ref_store, */ typedef struct ref_iterator *ref_iterator_begin_fn( struct ref_store *ref_store, - const char *prefix, unsigned int flags); + const char *prefix, const char **exclude_patterns, + unsigned int flags); /* reflog functions */ diff --git a/revision.c b/revision.c index b33cc1d106..89953592f9 100644 --- a/revision.c +++ b/revision.c @@ -2670,7 +2670,7 @@ static int for_each_bisect_ref(struct ref_store *refs, each_ref_fn fn, struct strbuf bisect_refs = STRBUF_INIT; int status; strbuf_addf(&bisect_refs, "refs/bisect/%s", term); - status = refs_for_each_fullref_in(refs, bisect_refs.buf, fn, cb_data); + status = refs_for_each_fullref_in(refs, bisect_refs.buf, NULL, fn, cb_data); strbuf_release(&bisect_refs); return status; } From patchwork Wed Jun 7 10:41:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D97FAC7EE2E for ; Wed, 7 Jun 2023 10:41:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239555AbjFGKlm (ORCPT ); Wed, 7 Jun 2023 06:41:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239614AbjFGKle (ORCPT ); Wed, 7 Jun 2023 06:41:34 -0400 Received: from mail-yw1-x112d.google.com (mail-yw1-x112d.google.com [IPv6:2607:f8b0:4864:20::112d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7B3791BE8 for ; Wed, 7 Jun 2023 03:41:32 -0700 (PDT) Received: by mail-yw1-x112d.google.com with SMTP id 00721157ae682-565ee3d14c2so79801107b3.2 for ; Wed, 07 Jun 2023 03:41:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134491; x=1688726491; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=xLAuiGphArCHQszyrtInzCh2h7W2YrDAbAzRk8wtp14=; b=5GURjp5XAbNLDxclprkchMmZ9Lqh1euPYJiC+4wIMrxKGLAeWhSp9zw6w2SP4kmUOl FTRI0S8jNmlQG9jhwj7Y1zzpdGxWglQ9gcMVdFjsLdnxJ+euWyULFZkPIPTWLp8cK6v/ g93BfdAd7AS3EzsTx8gfx93K/y5QubY4mc9XBvMxd/fnHxnI6iHast+2DkrweGoGQOI2 Ge0MFI38WSmsGX5w0PgSOoFs9ynuEjN/kMIgtORgINTfJWhx4LVXqy6Qi77pS5VMkjvq bh3z8w9JVD/+kkpZFuHxvd4IbjyfgjoQkb+M3ixYJxzVZy8DzV3U0+cvXMGr04JMEkYW gdtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134491; x=1688726491; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=xLAuiGphArCHQszyrtInzCh2h7W2YrDAbAzRk8wtp14=; b=SxBmzHMNTNofFFSCenv6kJFEViB5o2k4raA8/sANMRul+KAcW8UYzP/+HPFZKso/Cd 5K/5oNu1flL+cApfIvO6X3a0BHZ2+twA0AuMLrhytsffV4AMLQi6dMWGuITJoLZ5JaM9 GXfD0C3rj1FcKUmut7YFRKRnA2xdMPUDjYX1uVkF8BZ+6dLV0CeeSCUKEFRfNtiuuRAO K+yu3NGO8SInn0DS9fH8+OlkZEISknlXVcn9Wfpv8XXI2WcuxkqD0rBb5mduw8ftPhMY YEKU+mcdWNduntotdNPPnd5QG2glhjZTA0V4QYKHfohyz3BdujoPwm8o9BSohf/3Sawd ooPg== X-Gm-Message-State: AC+VfDwDctGN7afsf1nr9JRXWN5FgIjMkNfyQMrjRa++AxMBnIRxNXxo BOUW3dherGMchKiRsoGgInjBjP9I9GuD51Hy5pcqZIIn X-Google-Smtp-Source: ACHHUZ7p/tg87tS5boR0bdBYw50maOd9pKZil4n4UDA8PpUHdj3+FecpBCBJX1x4bU30QOCKdeY8LQ== X-Received: by 2002:a0d:cd42:0:b0:55d:ba47:5359 with SMTP id p63-20020a0dcd42000000b0055dba475359mr6051892ywd.31.1686134491590; Wed, 07 Jun 2023 03:41:31 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id h66-20020a0dde45000000b0055d7f00d4f7sm4658313ywe.22.2023.06.07.03.41.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:41:31 -0700 (PDT) Date: Wed, 7 Jun 2023 06:41:28 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 08/16] refs/packed-backend.c: refactor `find_reference_location()` Message-ID: <9cab5e06996b370e106c965b0e6b92384a744f31.1686134440.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The function `find_reference_location()` is used to perform a binary search-like function over the contents of a repository's `$GIT_DIR/packed-refs` file. The search it implements is unlike a standard binary search in that the records it searches over are not of a fixed width, so the comparison must locate the end of a record before comparing it. Extract the core routine of `find_reference_location()` in order to implement a function in the following patch which will find the first location in the `packed-refs` file that *doesn't* match the given pattern. The behavior of `find_reference_location()` is unchanged. Co-authored-by: Jeff King Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- refs/packed-backend.c | 38 ++++++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 176bd3905b..33639f73e1 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -527,22 +527,8 @@ static int load_contents(struct snapshot *snapshot) return 1; } -/* - * Find the place in `snapshot->buf` where the start of the record for - * `refname` starts. If `mustexist` is true and the reference doesn't - * exist, then return NULL. If `mustexist` is false and the reference - * doesn't exist, then return the point where that reference would be - * inserted, or `snapshot->eof` (which might be NULL) if it would be - * inserted at the end of the file. In the latter mode, `refname` - * doesn't have to be a proper reference name; for example, one could - * search for "refs/replace/" to find the start of any replace - * references. - * - * The record is sought using a binary search, so `snapshot->buf` must - * be sorted. - */ -static const char *find_reference_location(struct snapshot *snapshot, - const char *refname, int mustexist) +static const char *find_reference_location_1(struct snapshot *snapshot, + const char *refname, int mustexist) { /* * This is not *quite* a garden-variety binary search, because @@ -588,6 +574,26 @@ static const char *find_reference_location(struct snapshot *snapshot, return lo; } +/* + * Find the place in `snapshot->buf` where the start of the record for + * `refname` starts. If `mustexist` is true and the reference doesn't + * exist, then return NULL. If `mustexist` is false and the reference + * doesn't exist, then return the point where that reference would be + * inserted, or `snapshot->eof` (which might be NULL) if it would be + * inserted at the end of the file. In the latter mode, `refname` + * doesn't have to be a proper reference name; for example, one could + * search for "refs/replace/" to find the start of any replace + * references. + * + * The record is sought using a binary search, so `snapshot->buf` must + * be sorted. + */ +static const char *find_reference_location(struct snapshot *snapshot, + const char *refname, int mustexist) +{ + return find_reference_location_1(snapshot, refname, mustexist); +} + /* * Create a newly-allocated `snapshot` of the `packed-refs` file in * its current state and return it. The return value will already have From patchwork Wed Jun 7 10:41:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270354 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 366B9C7EE23 for ; Wed, 7 Jun 2023 10:41:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239569AbjFGKlu (ORCPT ); Wed, 7 Jun 2023 06:41:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53356 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239480AbjFGKlk (ORCPT ); Wed, 7 Jun 2023 06:41:40 -0400 Received: from mail-yw1-x112e.google.com (mail-yw1-x112e.google.com [IPv6:2607:f8b0:4864:20::112e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABB941BF5 for ; Wed, 7 Jun 2023 03:41:37 -0700 (PDT) Received: by mail-yw1-x112e.google.com with SMTP id 00721157ae682-569f25e735fso18672797b3.0 for ; Wed, 07 Jun 2023 03:41:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134497; x=1688726497; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=qzK/yzmSRUZF5Au/9OkfKE7/BDmJFQOHsbsOs0Kwyno=; b=Xg03zzr3t37aPkegRfvTSosKqBjyLTOXYbjefzpl6NC47oYwRMOjrn5VmJw9jLIV6g 2CDmJJZYJf1IXZIQn7O3VLnNtFYW93RsU2g9km7PIbed11soqCjy7VC8UFLoKQ7t+RlK aFv8XSFevkAC4RgjMX1jMfLfmD4sRTdb/34ljmvr2jINt+N5AOb4hNaqd1R8G6xwn7vJ vRZVub8UhQsN100012/0UhPEw9wpIu66t8jh6hoPewaR7fo5qS6w3dDicyiCgyhXRKv5 OBCMOZG1diwWVlqsb4pgOpy19+ad+CPCHJZaWs3qpGL3DOw+WDuvH5OBCeMJDU6gSElL P7IA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134497; x=1688726497; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=qzK/yzmSRUZF5Au/9OkfKE7/BDmJFQOHsbsOs0Kwyno=; b=XQ7P0E+2m8TdaeyAu+2jVVF2ron/xQxhtfJ2clrEY3rdpJ7w9jldrQnr1h2a2BFvKa qpyuRiWft2H1mzVSzjxkyEHcI+rB7YecSctYUd8rUM6VeebBzYA4/Vn0dVWyccYa0P3F D5b6vzf7a5qUE99BF83RxvbeLHmWO6h0KORDVBAKuLoBJ7/Sov7lv7klPo7k6X7PPEbd f85y3rch0cGZpgzGqsZHUd54FLmPRf9MVj4lxkSBfM0UF8enI1WU2O9mdo/BqR6W46TP 5Hq+ulBDB2YPlqlRUAnPDcayNnWh08zGaqyb+5YcGUTN3MnlQUaoAHIeVEQYpmxzQDd5 K0Xw== X-Gm-Message-State: AC+VfDwElPzFrrgfy+ksgoaGRcMwYRb89ocm4FFbXArW0M7ce+9EOpyq PLQvqyOrr+FenlVBrIUN/3ll6gTXIbFoWRNnfeH5etNg X-Google-Smtp-Source: ACHHUZ70JcmhUNGcOiCpUntP29qWIVRFnrQc2ceto5tZhjYEOlhEESYQadCb4r2lRZVqJuKHYL6c6w== X-Received: by 2002:a0d:dbd5:0:b0:559:e180:2197 with SMTP id d204-20020a0ddbd5000000b00559e1802197mr5951864ywe.21.1686134496613; Wed, 07 Jun 2023 03:41:36 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id t21-20020a0dea15000000b0056872195267sm4695804ywe.129.2023.06.07.03.41.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:41:36 -0700 (PDT) Date: Wed, 7 Jun 2023 06:41:33 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 09/16] refs/packed-backend.c: implement jump lists to avoid excluded pattern(s) Message-ID: <8066858bf59386eeec68f0f1e4247eeebb6482f7.1686134440.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When iterating through the `packed-refs` file in order to answer a query like: $ git for-each-ref --exclude=refs/__hidden__ it would be useful to avoid walking over all of the entries in `refs/__hidden__/*` when possible, since we know that the ref-filter code is going to throw them away anyways. In certain circumstances, doing so is possible. The algorithm for doing so is as follows: - For each excluded pattern, find the first record that matches it, and the first record that *doesn't* match it (i.e. the location you'd next want to consider when excluding that pattern). - Sort the set of excluded regions from the previous step in ascending order of the first location within the `packed-refs` file that matches. - Clean up the results from the previous step: discard empty regions, and combine adjacent regions. Then when iterating through the `packed-refs` file, if `iter->pos` is ever contained in one of the regions from the previous steps, advance `iter->pos` past the end of that region, and continue enumeration. Note that we only perform this optimization when none of the excluded pattern(s) have special meta-characters in them. For a pattern like "refs/foo[ac]", the excluded regions ("refs/fooa", "refs/fooc", and everything underneath them) are not connected. A future implementation that handles this case may split the character class (pretending as if two patterns were excluded: "refs/fooa", and "refs/fooc"). There are a few other gotchas worth considering. First, note that the jump list is sorted, so once we jump past a region, we can avoid considering it (or any regions preceding it) again. The member `jump_pos` is used to track the first next-possible region to jump through. Second, note that the exclusion list is best-effort, since we do not handle loose references, and because of the meta-character issue above. In repositories with a large number of hidden references, the speed-up can be significant. Tests here are done with a copy of linux.git with a reference "refs/pull/N" pointing at every commit, as in: $ git rev-list HEAD | awk '{ print "create refs/pull/" NR " " $0 }' | git update-ref --stdin $ git pack-refs --all , it is significantly faster to have `for-each-ref` jump over the excluded references, as opposed to filtering them out after the fact: $ hyperfine \ 'git for-each-ref --format="%(objectname) %(refname)" | grep -vE "^[0-9a-f]{40} refs/pull/"' \ 'git.compile for-each-ref --format="%(objectname) %(refname)" --exclude="refs/pull"' Benchmark 1: git for-each-ref --format="%(objectname) %(refname)" | grep -vE "^[0-9a-f]{40} refs/pull/" Time (mean ± σ): 802.7 ms ± 2.1 ms [User: 691.6 ms, System: 147.0 ms] Range (min … max): 800.0 ms … 807.7 ms 10 runs Benchmark 2: git.compile for-each-ref --format="%(objectname) %(refname)" --exclude="refs/pull" Time (mean ± σ): 4.7 ms ± 0.3 ms [User: 0.7 ms, System: 4.0 ms] Range (min … max): 4.3 ms … 6.7 ms 422 runs Summary 'git.compile for-each-ref --format="%(objectname) %(refname)" --exclude="refs/pull"' ran 172.03 ± 9.60 times faster than 'git for-each-ref --format="%(objectname) %(refname)" | grep -vE "^[0-9a-f]{40} refs/pull/"' Using the jump list is fairly straightforward (see the changes to `refs/packed-backend.c::next_record()`), but constructing the list is not. To ensure that the construction is correct, add a new suite of tests in t1419 covering various corner cases (overlapping regions, partially overlapping regions, adjacent regions, etc.). Co-authored-by: Jeff King Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- ref-filter.c | 5 +- refs/packed-backend.c | 166 ++++++++++++++++++++++++++++++++++++-- t/helper/test-ref-store.c | 10 +++ t/t1419-exclude-refs.sh | 101 +++++++++++++++++++++++ 4 files changed, 274 insertions(+), 8 deletions(-) create mode 100755 t/t1419-exclude-refs.sh diff --git a/ref-filter.c b/ref-filter.c index 717c3c4bcf..ddc7f5204f 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2210,12 +2210,13 @@ static int for_each_fullref_in_pattern(struct ref_filter *filter, if (!filter->name_patterns[0]) { /* no patterns; we have to look at everything */ return refs_for_each_fullref_in(get_main_ref_store(the_repository), - "", NULL, cb, cb_data); + "", filter->exclude.v, cb, cb_data); } return refs_for_each_fullref_in_prefixes(get_main_ref_store(the_repository), NULL, filter->name_patterns, - NULL, cb, cb_data); + filter->exclude.v, + cb, cb_data); } /* diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 33639f73e1..67327e579c 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -303,7 +303,8 @@ static int cmp_packed_ref_records(const void *v1, const void *v2) * Compare a snapshot record at `rec` to the specified NUL-terminated * refname. */ -static int cmp_record_to_refname(const char *rec, const char *refname) +static int cmp_record_to_refname(const char *rec, const char *refname, + int start) { const char *r1 = rec + the_hash_algo->hexsz + 1; const char *r2 = refname; @@ -312,7 +313,7 @@ static int cmp_record_to_refname(const char *rec, const char *refname) if (*r1 == '\n') return *r2 ? -1 : 0; if (!*r2) - return 1; + return start ? 1 : -1; if (*r1 != *r2) return (unsigned char)*r1 < (unsigned char)*r2 ? -1 : +1; r1++; @@ -528,7 +529,8 @@ static int load_contents(struct snapshot *snapshot) } static const char *find_reference_location_1(struct snapshot *snapshot, - const char *refname, int mustexist) + const char *refname, int mustexist, + int start) { /* * This is not *quite* a garden-variety binary search, because @@ -558,7 +560,7 @@ static const char *find_reference_location_1(struct snapshot *snapshot, mid = lo + (hi - lo) / 2; rec = find_start_of_record(lo, mid); - cmp = cmp_record_to_refname(rec, refname); + cmp = cmp_record_to_refname(rec, refname, start); if (cmp < 0) { lo = find_end_of_record(mid, hi); } else if (cmp > 0) { @@ -591,7 +593,22 @@ static const char *find_reference_location_1(struct snapshot *snapshot, static const char *find_reference_location(struct snapshot *snapshot, const char *refname, int mustexist) { - return find_reference_location_1(snapshot, refname, mustexist); + return find_reference_location_1(snapshot, refname, mustexist, 1); +} + +/* + * Find the place in `snapshot->buf` after the end of the record for + * `refname`. In other words, find the location of first thing *after* + * `refname`. + * + * Other semantics are identical to the ones in + * `find_reference_location()`. + */ +static const char *find_reference_location_end(struct snapshot *snapshot, + const char *refname, + int mustexist) +{ + return find_reference_location_1(snapshot, refname, mustexist, 0); } /* @@ -785,6 +802,13 @@ struct packed_ref_iterator { /* The end of the part of the buffer that will be iterated over: */ const char *eof; + struct jump_list_entry { + const char *start; + const char *end; + } *jump; + size_t jump_nr, jump_alloc; + size_t jump_pos; + /* Scratch space for current values: */ struct object_id oid, peeled; struct strbuf refname_buf; @@ -802,14 +826,34 @@ struct packed_ref_iterator { */ static int next_record(struct packed_ref_iterator *iter) { - const char *p = iter->pos, *eol; + const char *p, *eol; strbuf_reset(&iter->refname_buf); + /* + * If iter->pos is contained within a skipped region, jump past + * it. + * + * Note that each skipped region is considered at most once, + * since they are ordered based on their starting position. + */ + while (iter->jump_pos < iter->jump_nr) { + struct jump_list_entry *curr = &iter->jump[iter->jump_pos]; + if (iter->pos < curr->start) + break; /* not to the next jump yet */ + + iter->jump_pos++; + if (iter->pos < curr->end) { + iter->pos = curr->end; + break; + } + } + if (iter->pos == iter->eof) return ITER_DONE; iter->base.flags = REF_ISPACKED; + p = iter->pos; if (iter->eof - p < the_hash_algo->hexsz + 2 || parse_oid_hex(p, &iter->oid, &p) || @@ -917,6 +961,7 @@ static int packed_ref_iterator_abort(struct ref_iterator *ref_iterator) int ok = ITER_DONE; strbuf_release(&iter->refname_buf); + free(iter->jump); release_snapshot(iter->snapshot); base_ref_iterator_free(ref_iterator); return ok; @@ -928,6 +973,112 @@ static struct ref_iterator_vtable packed_ref_iterator_vtable = { .abort = packed_ref_iterator_abort }; +static int jump_list_entry_cmp(const void *va, const void *vb) +{ + const struct jump_list_entry *a = va; + const struct jump_list_entry *b = vb; + + if (a->start < b->start) + return -1; + if (a->start > b->start) + return 1; + return 0; +} + +static int has_glob_special(const char *str) +{ + const char *p; + for (p = str; *p; p++) { + if (is_glob_special(*p)) + return 1; + } + return 0; +} + +static const char *ptr_max(const char *x, const char *y) +{ + if (x > y) + return x; + return y; +} + +static void populate_excluded_jump_list(struct packed_ref_iterator *iter, + struct snapshot *snapshot, + const char **excluded_patterns) +{ + size_t i, j; + const char **pattern; + struct jump_list_entry *last_disjoint; + + if (!excluded_patterns) + return; + + for (pattern = excluded_patterns; *pattern; pattern++) { + struct jump_list_entry *e; + + /* + * We can't feed any excludes with globs in them to the + * refs machinery. It only understands prefix matching. + * We likewise can't even feed the string leading up to + * the first meta-character, as something like "foo[a]" + * should not exclude "foobar" (but the prefix "foo" + * would match that and mark it for exclusion). + */ + if (has_glob_special(*pattern)) + continue; + + ALLOC_GROW(iter->jump, iter->jump_nr + 1, iter->jump_alloc); + + e = &iter->jump[iter->jump_nr++]; + e->start = find_reference_location(snapshot, *pattern, 0); + e->end = find_reference_location_end(snapshot, *pattern, 0); + } + + if (!iter->jump_nr) { + /* + * Every entry in exclude_patterns has a meta-character, + * nothing to do here. + */ + return; + } + + QSORT(iter->jump, iter->jump_nr, jump_list_entry_cmp); + + /* + * As an optimization, merge adjacent entries in the jump list + * to jump forwards as far as possible when entering a skipped + * region. + * + * For example, if we have two skipped regions: + * + * [[A, B], [B, C]] + * + * we want to combine that into a single entry jumping from A to + * C. + */ + last_disjoint = iter->jump; + + for (i = 1, j = 1; i < iter->jump_nr; i++) { + struct jump_list_entry *ours = &iter->jump[i]; + + if (ours->start == ours->end) { + /* ignore empty regions (no matching entries) */ + continue; + } else if (ours->start <= last_disjoint->end) { + /* overlapping regions extend the previous one */ + last_disjoint->end = ptr_max(last_disjoint->end, ours->end); + } else { + /* otherwise, insert a new region */ + iter->jump[j++] = *ours; + last_disjoint = ours; + + } + } + + iter->jump_nr = j; + iter->jump_pos = 0; +} + static struct ref_iterator *packed_ref_iterator_begin( struct ref_store *ref_store, const char *prefix, const char **exclude_patterns, @@ -963,6 +1114,9 @@ static struct ref_iterator *packed_ref_iterator_begin( ref_iterator = &iter->base; base_ref_iterator_init(ref_iterator, &packed_ref_iterator_vtable, 1); + if (exclude_patterns) + populate_excluded_jump_list(iter, snapshot, exclude_patterns); + iter->snapshot = snapshot; acquire_snapshot(snapshot); diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c index 6d8f844e9c..2bff003f7c 100644 --- a/t/helper/test-ref-store.c +++ b/t/helper/test-ref-store.c @@ -175,6 +175,15 @@ static int cmd_for_each_ref(struct ref_store *refs, const char **argv) return refs_for_each_ref_in(refs, prefix, each_ref, NULL); } +static int cmd_for_each_ref__exclude(struct ref_store *refs, const char **argv) +{ + const char *prefix = notnull(*argv++, "prefix"); + const char **exclude_patterns = argv; + + return refs_for_each_fullref_in(refs, prefix, exclude_patterns, each_ref, + NULL); +} + static int cmd_resolve_ref(struct ref_store *refs, const char **argv) { struct object_id oid = *null_oid(); @@ -307,6 +316,7 @@ static struct command commands[] = { { "delete-refs", cmd_delete_refs }, { "rename-ref", cmd_rename_ref }, { "for-each-ref", cmd_for_each_ref }, + { "for-each-ref--exclude", cmd_for_each_ref__exclude }, { "resolve-ref", cmd_resolve_ref }, { "verify-ref", cmd_verify_ref }, { "for-each-reflog", cmd_for_each_reflog }, diff --git a/t/t1419-exclude-refs.sh b/t/t1419-exclude-refs.sh new file mode 100755 index 0000000000..bc534c8ea1 --- /dev/null +++ b/t/t1419-exclude-refs.sh @@ -0,0 +1,101 @@ +#!/bin/sh + +test_description='test exclude_patterns functionality in main ref store' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + +TEST_PASSES_SANITIZE_LEAK=true +. ./test-lib.sh + +for_each_ref__exclude () { + test-tool ref-store main for-each-ref--exclude "$@" >actual.raw + cut -d ' ' -f 2 actual.raw +} + +for_each_ref () { + git for-each-ref --format='%(refname)' "$@" +} + +test_expect_success 'setup' ' + test_commit --no-tag base && + base="$(git rev-parse HEAD)" && + + for name in foo bar baz quux + do + for i in 1 2 3 + do + echo "create refs/heads/$name/$i $base" || return 1 + done || return 1 + done >in && + echo "delete refs/heads/main" >>in && + + git update-ref --stdin actual && + for_each_ref refs/heads/bar refs/heads/baz refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'excluded region at beginning' ' + for_each_ref__exclude refs/heads refs/heads/bar >actual && + for_each_ref refs/heads/baz refs/heads/foo refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'excluded region at end' ' + for_each_ref__exclude refs/heads refs/heads/quux >actual && + for_each_ref refs/heads/foo refs/heads/bar refs/heads/baz >expect && + + test_cmp expect actual +' + +test_expect_success 'disjoint excluded regions' ' + for_each_ref__exclude refs/heads refs/heads/bar refs/heads/quux >actual && + for_each_ref refs/heads/baz refs/heads/foo >expect && + + test_cmp expect actual +' + +test_expect_success 'adjacent, non-overlapping excluded regions' ' + for_each_ref__exclude refs/heads refs/heads/bar refs/heads/baz >actual && + for_each_ref refs/heads/foo refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'overlapping excluded regions' ' + for_each_ref__exclude refs/heads refs/heads/ba refs/heads/baz >actual && + for_each_ref refs/heads/foo refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'several overlapping excluded regions' ' + for_each_ref__exclude refs/heads \ + refs/heads/bar refs/heads/baz refs/heads/foo >actual && + for_each_ref refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'non-matching excluded section' ' + for_each_ref__exclude refs/heads refs/heads/does/not/exist >actual && + for_each_ref >expect && + + test_cmp expect actual +' + +test_expect_success 'meta-characters are discarded' ' + for_each_ref__exclude refs/heads "refs/heads/ba*" >actual && + for_each_ref >expect && + + test_cmp expect actual +' + +test_done From patchwork Wed Jun 7 10:41:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E32C8C7EE23 for ; Wed, 7 Jun 2023 10:42:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239696AbjFGKmC (ORCPT ); Wed, 7 Jun 2023 06:42:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53564 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239699AbjFGKlr (ORCPT ); Wed, 7 Jun 2023 06:41:47 -0400 Received: from mail-yw1-x112a.google.com (mail-yw1-x112a.google.com [IPv6:2607:f8b0:4864:20::112a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C9C71BFD for ; Wed, 7 Jun 2023 03:41:46 -0700 (PDT) Received: by mail-yw1-x112a.google.com with SMTP id 00721157ae682-568900c331aso82222747b3.3 for ; Wed, 07 Jun 2023 03:41:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134505; x=1688726505; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=45LklVVipxjXTGAhiAAdWYFD4sw7+OHceu2D4E2NAvE=; b=4icinca/O9no73GLMMZ3ic1LE3fP1isz+aGa0Uw1yBdNhGtuSBoBlh2fU2aHjdSDcb AJ+23/qoBwTRM8sns7OW9e7R32ErGJITdBEVoVUWsHDoMTBuNVHLHsgJG5+EVE6aXp8V Sl4mk7TzEsrqPs+M/EQ+QzKyN2DV6LgdXGEwQXc0MQuB8zmwN0fJH1qjvDH9VErXCYtq ldwAHF4U05K6z4ybPgScFNID3U1j0jVRd69nayKE2OZjjI8X8Jo0qWSzrjUkV0ylCOsa 3bxGMtwfzkZfgRp+tqR3aZBDw/t/gOTMfPVR0EPlGuefhYDwO+jcmiFq2ce6QLI+CzPM kyzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134505; x=1688726505; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=45LklVVipxjXTGAhiAAdWYFD4sw7+OHceu2D4E2NAvE=; b=aczCIXuUhvTkOIi7kka8lgUxfA9FDlS5F6giPYZjtFNHBQ3DQLg3a+93aJsmq7Rb0s HqsUFjHdQULfSXQkJuiFZub9W1itvMXBs3MAPsim1aUa3TyRMh0MClZfWuIn3SQ8w1Bx Q6SEDLWcT7/bU3QaOihHnTUrur9ih8PeN+FXWD3ByUPOPAUQb0kRBUHyxdN01ejw1ni2 lUcUq4pV+qQjpvuCRb9albW8JkNFB4uw5otj5o4h5eONwhFzY3DnQ3pX/ipayy4vgXJr 2ZExc4Tz0lIwvUw9A9WoWDO5SkmCvTvAKeQ8XA92E+0zTXuFZ4MZ7m5UbyxTDsYlttGf P7Cg== X-Gm-Message-State: AC+VfDw4smPcMfBOUzi/dlOVpONC7KkEpLy0H/IWif0WNvQj2HkKqFnT DYP6E1yfMWirE/W7CNSIDTfyEvCzvO/BtWFSnHZUR7Ag X-Google-Smtp-Source: ACHHUZ6zFWTMSdOAM3E1pGrVLH9Yt/1F7lnIsmTdaq7w0Td2Sy9hpbGbP2+N0xZ8dq4sPsEkA97DUQ== X-Received: by 2002:a0d:df09:0:b0:565:cf40:238e with SMTP id i9-20020a0ddf09000000b00565cf40238emr5483534ywe.15.1686134505351; Wed, 07 Jun 2023 03:41:45 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id i2-20020a81d502000000b0054c0f3fd3ddsm4668906ywj.30.2023.06.07.03.41.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:41:45 -0700 (PDT) Date: Wed, 7 Jun 2023 06:41:42 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 10/16] refs/packed-backend.c: add trace2 counters for jump list Message-ID: <3c045076a95fab9d050a312114c9aca712eb7141.1686134440.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The previous commit added low-level tests to ensure that the packed-refs iterator did not enumerate excluded sections of the refspace. However, there was no guarantee that these sections weren't being visited, only that they were being suppressed from the output. To harden these tests, add a trace2 counter which tracks the number of regions skipped by the packed-refs iterator, and assert on its value. Suggested-by: Derrick Stolee Signed-off-by: Taylor Blau --- refs/packed-backend.c | 2 ++ t/t1419-exclude-refs.sh | 59 ++++++++++++++++++++++++++++------------- trace2.h | 2 ++ trace2/tr2_ctr.c | 5 ++++ 4 files changed, 49 insertions(+), 19 deletions(-) diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 67327e579c..7ba9fa2bb8 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -12,6 +12,7 @@ #include "../chdir-notify.h" #include "../wrapper.h" #include "../write-or-die.h" +#include "../trace2.h" enum mmap_strategy { /* @@ -845,6 +846,7 @@ static int next_record(struct packed_ref_iterator *iter) iter->jump_pos++; if (iter->pos < curr->end) { iter->pos = curr->end; + trace2_counter_add(TRACE2_COUNTER_ID_PACKED_REFS_JUMPS, 1); break; } } diff --git a/t/t1419-exclude-refs.sh b/t/t1419-exclude-refs.sh index bc534c8ea1..350a7d2587 100755 --- a/t/t1419-exclude-refs.sh +++ b/t/t1419-exclude-refs.sh @@ -9,7 +9,8 @@ TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh for_each_ref__exclude () { - test-tool ref-store main for-each-ref--exclude "$@" >actual.raw + GIT_TRACE2_PERF=1 test-tool ref-store main \ + for-each-ref--exclude "$@" >actual.raw cut -d ' ' -f 2 actual.raw } @@ -17,6 +18,17 @@ for_each_ref () { git for-each-ref --format='%(refname)' "$@" } +assert_jumps () { + local nr="$1" + local trace="$2" + + grep -q "name:jumps_made value:$nr" $trace +} + +assert_no_jumps () { + ! assert_jumps ".*" "$1" +} + test_expect_success 'setup' ' test_commit --no-tag base && base="$(git rev-parse HEAD)" && @@ -35,67 +47,76 @@ test_expect_success 'setup' ' ' test_expect_success 'excluded region in middle' ' - for_each_ref__exclude refs/heads refs/heads/foo >actual && + for_each_ref__exclude refs/heads refs/heads/foo >actual 2>perf && for_each_ref refs/heads/bar refs/heads/baz refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'excluded region at beginning' ' - for_each_ref__exclude refs/heads refs/heads/bar >actual && + for_each_ref__exclude refs/heads refs/heads/bar >actual 2>perf && for_each_ref refs/heads/baz refs/heads/foo refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'excluded region at end' ' - for_each_ref__exclude refs/heads refs/heads/quux >actual && + for_each_ref__exclude refs/heads refs/heads/quux >actual 2>perf && for_each_ref refs/heads/foo refs/heads/bar refs/heads/baz >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'disjoint excluded regions' ' - for_each_ref__exclude refs/heads refs/heads/bar refs/heads/quux >actual && + for_each_ref__exclude refs/heads refs/heads/bar refs/heads/quux >actual 2>perf && for_each_ref refs/heads/baz refs/heads/foo >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 2 perf ' test_expect_success 'adjacent, non-overlapping excluded regions' ' - for_each_ref__exclude refs/heads refs/heads/bar refs/heads/baz >actual && + for_each_ref__exclude refs/heads refs/heads/bar refs/heads/baz >actual 2>perf && for_each_ref refs/heads/foo refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'overlapping excluded regions' ' - for_each_ref__exclude refs/heads refs/heads/ba refs/heads/baz >actual && + for_each_ref__exclude refs/heads refs/heads/ba refs/heads/baz >actual 2>perf && for_each_ref refs/heads/foo refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'several overlapping excluded regions' ' for_each_ref__exclude refs/heads \ - refs/heads/bar refs/heads/baz refs/heads/foo >actual && + refs/heads/bar refs/heads/baz refs/heads/foo >actual 2>perf && for_each_ref refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'non-matching excluded section' ' - for_each_ref__exclude refs/heads refs/heads/does/not/exist >actual && + for_each_ref__exclude refs/heads refs/heads/does/not/exist >actual 2>perf && for_each_ref >expect && - test_cmp expect actual + test_cmp expect actual && + assert_no_jumps ' test_expect_success 'meta-characters are discarded' ' - for_each_ref__exclude refs/heads "refs/heads/ba*" >actual && + for_each_ref__exclude refs/heads "refs/heads/ba*" >actual 2>perf && for_each_ref >expect && - test_cmp expect actual + test_cmp expect actual && + assert_no_jumps ' test_done diff --git a/trace2.h b/trace2.h index 4ced30c0db..9452e291f5 100644 --- a/trace2.h +++ b/trace2.h @@ -551,6 +551,8 @@ enum trace2_counter_id { TRACE2_COUNTER_ID_TEST1 = 0, /* emits summary event only */ TRACE2_COUNTER_ID_TEST2, /* emits summary and thread events */ + TRACE2_COUNTER_ID_PACKED_REFS_JUMPS, /* counts number of jumps */ + /* Add additional counter definitions before here. */ TRACE2_NUMBER_OF_COUNTERS }; diff --git a/trace2/tr2_ctr.c b/trace2/tr2_ctr.c index b342d3b1a3..50570d0165 100644 --- a/trace2/tr2_ctr.c +++ b/trace2/tr2_ctr.c @@ -27,6 +27,11 @@ static struct tr2_counter_metadata tr2_counter_metadata[TRACE2_NUMBER_OF_COUNTER .name = "test2", .want_per_thread_events = 1, }, + [TRACE2_COUNTER_ID_PACKED_REFS_JUMPS] = { + .category = "packed-refs", + .name = "jumps_made", + .want_per_thread_events = 0, + }, /* Add additional metadata before here. */ }; From patchwork Wed Jun 7 10:41:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270356 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B7D1C7EE23 for ; Wed, 7 Jun 2023 10:42:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239700AbjFGKmE (ORCPT ); Wed, 7 Jun 2023 06:42:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239596AbjFGKlx (ORCPT ); Wed, 7 Jun 2023 06:41:53 -0400 Received: from mail-yw1-x1133.google.com (mail-yw1-x1133.google.com [IPv6:2607:f8b0:4864:20::1133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 23380AA for ; Wed, 7 Jun 2023 03:41:51 -0700 (PDT) Received: by mail-yw1-x1133.google.com with SMTP id 00721157ae682-565ba53f434so80021497b3.3 for ; Wed, 07 Jun 2023 03:41:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134510; x=1688726510; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ZxNxpSZm9oCdX8mEdMdkkC305eB3R6MykgDz8PVoIOE=; b=M7Zyu6HnZmSlkNy+sLwbWepCLPC/nkclX5d9GtDbplfMbImEFgLn+FiFWHGDBypIRu pFnlanBD+gjGTEXjlHQhCGyp98FzLhSxz3BEg6SzElIecavk1s9G3f7T3Jd6hU6d4U3o gFa0FRcRbBln52/kHFoKk7zXlL2hw6Yc0WUq8DCPs8c+qaNr7t17kdGEE2YDMGpSNItV OEfNRMj+5kMeMF2gbA8jfBsN76cOnVSygEJAtBRhyJWABHPchjfrT+JraJVJWVktkIBe o4xkSrfgVi9k7xl3cAkyQArLaYqs/mwy7d2dmXRIYuKXue6V7UEo54XC/SVtYd5s5hKy W/PA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134510; x=1688726510; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=ZxNxpSZm9oCdX8mEdMdkkC305eB3R6MykgDz8PVoIOE=; b=cDR5c3WIX/4x2u9e8/yvPnLnOzS1xNJ0ETevMd8Mq3sBpwvfw9rtYlNxEL91+kOIuK +lCaxo+MPZirYEFDp7l1FoYKQjR7glr6N/fJdvfOC97/agesrr3uvmnTHB46GFU54lUH WoT77ueElJcKhUk1gp3Rs5Nf2iB0BMBS7gvwh5x0rBQqkOishTztX9J8u+QoacCztW55 P6sRksVXMGkOhi989P0Lk17DtYEZOA5VErkudrYfbU/oOoOaNyqPK9jYKwS/10mo1D/l vMK0APH1bFK39zBO0vF8n3fBXN9sqEX3HlUjqMFSVYqJsQj1ewvzYBB1rk0CIbwdQ+Vr m/3A== X-Gm-Message-State: AC+VfDyIjdzSl69C+BYQeTEmnc0BTo+yD9vyXONV7so3fSJozURAOlgP ATmpy9TtxoewAvKBjLia/qWgcRrl8DUS66IUnNhUfAF7 X-Google-Smtp-Source: ACHHUZ7NqNIfRHJUW9p1EXveGT2VTXpT5FcWlsla24xPNg+jwbpsUwnX0XqsoRiOcc3epugIIa4zGg== X-Received: by 2002:a0d:ebd2:0:b0:565:cf47:7331 with SMTP id u201-20020a0debd2000000b00565cf477331mr7474469ywe.2.1686134510206; Wed, 07 Jun 2023 03:41:50 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id w205-20020a817bd6000000b005569567aac1sm4633642ywc.106.2023.06.07.03.41.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:41:49 -0700 (PDT) Date: Wed, 7 Jun 2023 06:41:47 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 11/16] revision.h: store hidden refs in a `strvec` Message-ID: <0ff542eaefd5142056b484901a3c22e2a590b07e.1686134440.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In subsequent commits, it will be convenient to have a 'const char **' of hidden refs (matching `transfer.hiderefs`, `uploadpack.hideRefs`, etc.), instead of a `string_list`. Convert spots throughout the tree that store the list of hidden refs from a `string_list` to a `strvec`. Note that in `parse_hide_refs_config()` there is an ugly const-cast used to avoid an extra copy of each value before trimming any trailing slash characters. This could instead be written as: ref = xstrdup(value); len = strlen(ref); while (len && ref[len - 1] == '/') ref[--len] = '\0'; strvec_push(hide_refs, ref); free(ref); but the double-copy (once when calling `xstrdup()`, and another via `strvec_push()`) is wasteful. Signed-off-by: Taylor Blau --- builtin/receive-pack.c | 4 ++-- ls-refs.c | 6 +++--- refs.c | 11 ++++++----- refs.h | 4 ++-- revision.c | 2 +- revision.h | 5 +++-- upload-pack.c | 10 +++++----- 7 files changed, 22 insertions(+), 20 deletions(-) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 1a31a58367..1a8472eddc 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -90,7 +90,7 @@ static struct object_id push_cert_oid; static struct signature_check sigcheck; static const char *push_cert_nonce; static const char *cert_nonce_seed; -static struct string_list hidden_refs = STRING_LIST_INIT_DUP; +static struct strvec hidden_refs = STRVEC_INIT; static const char *NONCE_UNSOLICITED = "UNSOLICITED"; static const char *NONCE_BAD = "BAD"; @@ -2619,7 +2619,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix) packet_flush(1); oid_array_clear(&shallow); oid_array_clear(&ref); - string_list_clear(&hidden_refs, 0); + strvec_clear(&hidden_refs); free((void *)push_cert_nonce); return 0; } diff --git a/ls-refs.c b/ls-refs.c index 6f490b2d9c..8c3181d051 100644 --- a/ls-refs.c +++ b/ls-refs.c @@ -72,7 +72,7 @@ struct ls_refs_data { unsigned symrefs; struct strvec prefixes; struct strbuf buf; - struct string_list hidden_refs; + struct strvec hidden_refs; unsigned unborn : 1; }; @@ -155,7 +155,7 @@ int ls_refs(struct repository *r, struct packet_reader *request) memset(&data, 0, sizeof(data)); strvec_init(&data.prefixes); strbuf_init(&data.buf, 0); - string_list_init_dup(&data.hidden_refs); + strvec_init(&data.hidden_refs); git_config(ls_refs_config, &data); @@ -197,7 +197,7 @@ int ls_refs(struct repository *r, struct packet_reader *request) packet_fflush(stdout); strvec_clear(&data.prefixes); strbuf_release(&data.buf); - string_list_clear(&data.hidden_refs, 0); + strvec_clear(&data.hidden_refs); return 0; } diff --git a/refs.c b/refs.c index 538bde644e..ec4d5b9101 100644 --- a/refs.c +++ b/refs.c @@ -1427,7 +1427,7 @@ char *shorten_unambiguous_ref(const char *refname, int strict) } int parse_hide_refs_config(const char *var, const char *value, const char *section, - struct string_list *hide_refs) + struct strvec *hide_refs) { const char *key; if (!strcmp("transfer.hiderefs", var) || @@ -1438,22 +1438,23 @@ int parse_hide_refs_config(const char *var, const char *value, const char *secti if (!value) return config_error_nonbool(var); - ref = xstrdup(value); + + /* drop const to remove trailing '/' characters */ + ref = (char *)strvec_push(hide_refs, value); len = strlen(ref); while (len && ref[len - 1] == '/') ref[--len] = '\0'; - string_list_append_nodup(hide_refs, ref); } return 0; } int ref_is_hidden(const char *refname, const char *refname_full, - const struct string_list *hide_refs) + const struct strvec *hide_refs) { int i; for (i = hide_refs->nr - 1; i >= 0; i--) { - const char *match = hide_refs->items[i].string; + const char *match = hide_refs->v[i]; const char *subject; int neg = 0; const char *p; diff --git a/refs.h b/refs.h index d672d636cf..a7751a1fc9 100644 --- a/refs.h +++ b/refs.h @@ -810,7 +810,7 @@ int update_ref(const char *msg, const char *refname, unsigned int flags, enum action_on_err onerr); int parse_hide_refs_config(const char *var, const char *value, const char *, - struct string_list *); + struct strvec *); /* * Check whether a ref is hidden. If no namespace is set, both the first and @@ -820,7 +820,7 @@ int parse_hide_refs_config(const char *var, const char *value, const char *, * the ref is outside that namespace, the first parameter is NULL. The second * parameter always points to the full ref name. */ -int ref_is_hidden(const char *, const char *, const struct string_list *); +int ref_is_hidden(const char *, const char *, const struct strvec *); /* Is this a per-worktree ref living in the refs/ namespace? */ int is_per_worktree_ref(const char *refname); diff --git a/revision.c b/revision.c index 89953592f9..7c9367a266 100644 --- a/revision.c +++ b/revision.c @@ -1558,7 +1558,7 @@ void init_ref_exclusions(struct ref_exclusions *exclusions) void clear_ref_exclusions(struct ref_exclusions *exclusions) { string_list_clear(&exclusions->excluded_refs, 0); - string_list_clear(&exclusions->hidden_refs, 0); + strvec_clear(&exclusions->hidden_refs); exclusions->hidden_refs_configured = 0; } diff --git a/revision.h b/revision.h index 31828748dc..94f035fa22 100644 --- a/revision.h +++ b/revision.h @@ -10,6 +10,7 @@ #include "decorate.h" #include "ident.h" #include "list-objects-filter-options.h" +#include "strvec.h" /** * The revision walking API offers functions to build a list of revisions @@ -95,7 +96,7 @@ struct ref_exclusions { * Hidden refs is a list of patterns that is to be hidden via * `ref_is_hidden()`. */ - struct string_list hidden_refs; + struct strvec hidden_refs; /* * Indicates whether hidden refs have been configured. This is to @@ -110,7 +111,7 @@ struct ref_exclusions { */ #define REF_EXCLUSIONS_INIT { \ .excluded_refs = STRING_LIST_INIT_DUP, \ - .hidden_refs = STRING_LIST_INIT_DUP, \ + .hidden_refs = STRVEC_INIT, \ } struct oidset; diff --git a/upload-pack.c b/upload-pack.c index d3312006a3..1a213ed775 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -69,7 +69,7 @@ struct upload_pack_data { struct object_array have_obj; struct oid_array haves; /* v2 only */ struct string_list wanted_refs; /* v2 only */ - struct string_list hidden_refs; + struct strvec hidden_refs; struct object_array shallows; struct string_list deepen_not; @@ -127,7 +127,7 @@ static void upload_pack_data_init(struct upload_pack_data *data) { struct string_list symref = STRING_LIST_INIT_DUP; struct string_list wanted_refs = STRING_LIST_INIT_DUP; - struct string_list hidden_refs = STRING_LIST_INIT_DUP; + struct strvec hidden_refs = STRVEC_INIT; struct object_array want_obj = OBJECT_ARRAY_INIT; struct object_array have_obj = OBJECT_ARRAY_INIT; struct oid_array haves = OID_ARRAY_INIT; @@ -162,7 +162,7 @@ static void upload_pack_data_clear(struct upload_pack_data *data) { string_list_clear(&data->symref, 1); string_list_clear(&data->wanted_refs, 1); - string_list_clear(&data->hidden_refs, 0); + strvec_clear(&data->hidden_refs); object_array_clear(&data->want_obj); object_array_clear(&data->have_obj); oid_array_clear(&data->haves); @@ -1170,7 +1170,7 @@ static void receive_needs(struct upload_pack_data *data, /* return non-zero if the ref is hidden, otherwise 0 */ static int mark_our_ref(const char *refname, const char *refname_full, - const struct object_id *oid, const struct string_list *hidden_refs) + const struct object_id *oid, const struct strvec *hidden_refs) { struct object *o = lookup_unknown_object(the_repository, oid); @@ -1465,7 +1465,7 @@ static int parse_want(struct packet_writer *writer, const char *line, static int parse_want_ref(struct packet_writer *writer, const char *line, struct string_list *wanted_refs, - struct string_list *hidden_refs, + struct strvec *hidden_refs, struct object_array *want_obj) { const char *refname_nons; From patchwork Wed Jun 7 10:41:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9060C7EE23 for ; Wed, 7 Jun 2023 10:42:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239773AbjFGKmI (ORCPT ); Wed, 7 Jun 2023 06:42:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239672AbjFGKl6 (ORCPT ); Wed, 7 Jun 2023 06:41:58 -0400 Received: from mail-yw1-x1131.google.com (mail-yw1-x1131.google.com [IPv6:2607:f8b0:4864:20::1131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 738911BEC for ; Wed, 7 Jun 2023 03:41:56 -0700 (PDT) Received: by mail-yw1-x1131.google.com with SMTP id 00721157ae682-565aa2cc428so66709877b3.1 for ; Wed, 07 Jun 2023 03:41:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134515; x=1688726515; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=bq3ESmIcam7O5vlh8zzQ2Y/N+YL92aHyGi81jqxafR4=; b=4um+ukqteyLuNQEXer8fMbAFn8uASvcpHrD7Yb+Yk/7HyLCs06lsGO4HzrizdRyV5F XMmFrRSJQgseIlY4SePaQWz5Tv9FJZiHCMSI8tj16Kbp/4+DGkP5Gx2/WwkM94qq1RMb WRncxpsM6gRL0r0cy3+0Qv56KyGw6NxqjTUIcAhmGGEYGjTQ96BPJUMiUyGHgzb8yPWN zkV+gMhQLXGEHgOBHlElEXe/EY6zoXHcBUE34Mfqg81+36KeJlS95LYoq83NPSX9jSLg T5ryEYZM+PB9gDLIb0RcC6VRFbUYnlR+XIRm2rJoob03v2tU5fEwPK8iNduVs7+P8bkG dbFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134515; x=1688726515; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=bq3ESmIcam7O5vlh8zzQ2Y/N+YL92aHyGi81jqxafR4=; b=M91z1uSs6i61KzCEf7dU44xndZIWlyRjAzNQlByhuSaFD8cIUR2unEUfRIhrZZWxE2 /hGa0RxR05R1n8oV2KWMTVM3koYK+MQ0dDlF1YgSBDNyAHl1KzqqhqOSyVBdxKvFCpeJ L2RuIQ501jioRPYEh1+uXUH36h4DTbfV4mR1z+5usuwKi6ScmO6rAk5JxbICidoHANtz JfniTJ/POVoodUAGPNN/9syW2ks/lg5t+jO4PJKsMwUGMZNGw6hJVQ1f11xun22Ml58u or7nXk5KbecRZpt8l0hCPE0MIn+nW4S+YxJXJRX4F3qSb7WAdn8NX9o9Aj+IlrWLn4nM /Mzg== X-Gm-Message-State: AC+VfDwgfYFdJMNjSBkDYDKtryCfDAfpW/cbYIzs98n978Uj14XYG8Vg eDX4ygNpkVN8sjgCppqo0q1Zv0LC020xAnjDpUNoH6dJ X-Google-Smtp-Source: ACHHUZ7VHmhy22/Om0iOG+hQnhR48RVPYoISf3aSL5sVnUGHRbR5gXCGGmfoEwW/e06GA6W08+0cNw== X-Received: by 2002:a81:5b0a:0:b0:561:b00d:1e12 with SMTP id p10-20020a815b0a000000b00561b00d1e12mr5114991ywb.29.1686134515195; Wed, 07 Jun 2023 03:41:55 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id w187-20020a8186c4000000b00567679ef3a6sm4608914ywf.109.2023.06.07.03.41.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:41:54 -0700 (PDT) Date: Wed, 7 Jun 2023 06:41:52 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 12/16] refs/packed-backend.c: ignore complicated hidden refs rules Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In subsequent commits, we'll teach `receive-pack` and `upload-pack` to use the new skip-list feature in the packed-refs iterator by ignoring references which are mentioned via its respective hideRefs lists. However, the packed-ref skip lists cannot handle un-hiding rules (that begin with '!'), or namespace comparisons (that begin with '^'). Detect and avoid these cases by falling back to the normal enumeration without a skip list when such patterns exist. Signed-off-by: Taylor Blau --- refs/packed-backend.c | 19 +++++++++++++++++++ t/t1419-exclude-refs.sh | 9 +++++++++ 2 files changed, 28 insertions(+) diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 7ba9fa2bb8..9ea6c07866 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -1015,6 +1015,25 @@ static void populate_excluded_jump_list(struct packed_ref_iterator *iter, if (!excluded_patterns) return; + for (pattern = excluded_patterns; *pattern; pattern++) { + /* + * We also can't feed any excludes from hidden refs + * config sections, since later rules may override + * previous ones. For example, with rules "refs/foo" and + * "!refs/foo/bar", we should show "refs/foo/bar" (and + * everything underneath it), but the earlier exclusion + * would cause us to skip all of "refs/foo". We likewise + * don't implement the namespace stripping required for + * '^' rules. + * + * Both are possible to do, but complicated, so avoid + * populating the jump list at all if we see either of + * these patterns. + */ + if (**pattern == '!' || **pattern == '^') + return; + } + for (pattern = excluded_patterns; *pattern; pattern++) { struct jump_list_entry *e; diff --git a/t/t1419-exclude-refs.sh b/t/t1419-exclude-refs.sh index 350a7d2587..0e91e2f399 100755 --- a/t/t1419-exclude-refs.sh +++ b/t/t1419-exclude-refs.sh @@ -119,4 +119,13 @@ test_expect_success 'meta-characters are discarded' ' assert_no_jumps ' +test_expect_success 'complex hidden ref rules are discarded' ' + for_each_ref__exclude refs/heads refs/heads/foo "!refs/heads/foo/1" \ + >actual 2>perf && + for_each_ref >expect && + + test_cmp expect actual && + assert_no_jumps +' + test_done From patchwork Wed Jun 7 10:41:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270358 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21C8AC7EE23 for ; Wed, 7 Jun 2023 10:42:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239810AbjFGKmM (ORCPT ); Wed, 7 Jun 2023 06:42:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53658 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239645AbjFGKmD (ORCPT ); Wed, 7 Jun 2023 06:42:03 -0400 Received: from mail-yw1-x1132.google.com (mail-yw1-x1132.google.com [IPv6:2607:f8b0:4864:20::1132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2EBA31BFD for ; Wed, 7 Jun 2023 03:42:01 -0700 (PDT) Received: by mail-yw1-x1132.google.com with SMTP id 00721157ae682-565cfe4ece7so80374947b3.2 for ; Wed, 07 Jun 2023 03:42:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134520; x=1688726520; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=tGbCNYkwoJB1yHBWHL4MDXOa1qaRRZdalmL1mP4pZ18=; b=CkI9YeLvoVEeKQO3oqa/1od/xRBszMBtMG4JXrhxD1kicECZyPGjXVKBz+ZX40vWFd AK/b/qC7oQtFBlDKVR6WX0lRvdsHvINmGC98nxy77aWUI3VjdlpPlSBJ2tYmmIGCRJUX lE+e6KTmbVOGfrqWu6cQhrbrB/eXlOtnzfufMZRYvhJKniga/uJjSBMVUD7sMLi1l0D/ mP9MExWlgcEAvz7EnmChiqj+KXWyfxiAgHPVRX4UnMcFdB2CD4Vaf8hABNkYYEEjxlPw O1NnvKaTWtmOOYH04upINnhLNp3Vx6u2iPuztVXQaUIxvnPVitMxZ5Md/+FKeJwHGPa1 TBoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134520; x=1688726520; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=tGbCNYkwoJB1yHBWHL4MDXOa1qaRRZdalmL1mP4pZ18=; b=hObtv1A5mlv7O4mp2gC01nISJS95KAEnmH+ZkZ5+1Io/+xvUPQMg5gQPT4LkyeXem8 xtd/H2RfWaB5SturQ8YljN4I8341Lv5xmPlLM3y/T1LTH9R+VX6PMz3dYEVv1cYJQkRg lC2Ky81NXJyY/CWhSblHxavr800gisPBs6b4xqAqvnzxBKceriRfT0qlysDQjVVJfth8 LgHQ3+AT+xrUeSsasMOAOi4LNYzwYJKOZbJSLIQ+jz/zkqSS9scK4e4eoHPXinxsBC2v 21Kym2nQuR3qsFrppO0S1sIIniBxy2McHFV1cJ42kZ6DM3HcIX0yjMKjWm7BA4a6TJOr 5rcA== X-Gm-Message-State: AC+VfDzQe8rUX+4j5QCRK3+SUrtUH79gBlwf2VCPkZxsfQg6016NU9ho k4PMOyk+6XYry1sLIZm3pGbaJwCkSoAVo9nZouDvGIhq X-Google-Smtp-Source: ACHHUZ73mwPowwvgH3ZzA1+FfzalGutJ2Ktrz/F8GHO0WRftQmQm0tFi7K1jQi1rXIym+Lp25cc/aQ== X-Received: by 2002:a81:7245:0:b0:568:8fa1:7a3 with SMTP id n66-20020a817245000000b005688fa107a3mr5597049ywc.5.1686134520270; Wed, 07 Jun 2023 03:42:00 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id w11-20020a81a20b000000b0054f9e7fed7asm4711926ywg.137.2023.06.07.03.41.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:42:00 -0700 (PDT) Date: Wed, 7 Jun 2023 06:41:57 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 13/16] refs.h: let `for_each_namespaced_ref()` take excluded patterns Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The following commit will want to call `for_each_namespaced_ref()` with a list of excluded patterns. We could introduce a variant of that function, say, `for_each_namespaced_ref_exclude()` which takes the extra parameter, and reimplement the original function in terms of that. But all but one caller (in `http-backend.c`) will supply the new parameter, so add the new parameter to `for_each_namespaced_ref()` itself instead of introducing a new function. For now, supply NULL for the list of excluded patterns at all callers to avoid changing behavior, which we will do in the subsequent commit. Signed-off-by: Taylor Blau --- http-backend.c | 2 +- refs.c | 5 +++-- refs.h | 3 ++- upload-pack.c | 6 +++--- 4 files changed, 9 insertions(+), 7 deletions(-) diff --git a/http-backend.c b/http-backend.c index ac146d85c5..ad500683c8 100644 --- a/http-backend.c +++ b/http-backend.c @@ -559,7 +559,7 @@ static void get_info_refs(struct strbuf *hdr, char *arg UNUSED) } else { select_getanyfile(hdr); - for_each_namespaced_ref(show_text_ref, &buf); + for_each_namespaced_ref(NULL, show_text_ref, &buf); send_strbuf(hdr, "text/plain", &buf); } strbuf_release(&buf); diff --git a/refs.c b/refs.c index ec4d5b9101..95a7db9563 100644 --- a/refs.c +++ b/refs.c @@ -1660,13 +1660,14 @@ int for_each_replace_ref(struct repository *r, each_repo_ref_fn fn, void *cb_dat DO_FOR_EACH_INCLUDE_BROKEN, cb_data); } -int for_each_namespaced_ref(each_ref_fn fn, void *cb_data) +int for_each_namespaced_ref(const char **exclude_patterns, + each_ref_fn fn, void *cb_data) { struct strbuf buf = STRBUF_INIT; int ret; strbuf_addf(&buf, "%srefs/", get_git_namespace()); ret = do_for_each_ref(get_main_ref_store(the_repository), - buf.buf, NULL, fn, 0, 0, cb_data); + buf.buf, exclude_patterns, fn, 0, 0, cb_data); strbuf_release(&buf); return ret; } diff --git a/refs.h b/refs.h index a7751a1fc9..f23626beca 100644 --- a/refs.h +++ b/refs.h @@ -372,7 +372,8 @@ int for_each_glob_ref_in(each_ref_fn fn, const char *pattern, const char *prefix, void *cb_data); int head_ref_namespaced(each_ref_fn fn, void *cb_data); -int for_each_namespaced_ref(each_ref_fn fn, void *cb_data); +int for_each_namespaced_ref(const char **exclude_patterns, + each_ref_fn fn, void *cb_data); /* can be used to learn about broken ref and symref */ int refs_for_each_rawref(struct ref_store *refs, each_ref_fn fn, void *cb_data); diff --git a/upload-pack.c b/upload-pack.c index 1a213ed775..99d216938c 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -855,7 +855,7 @@ static void deepen(struct upload_pack_data *data, int depth) * marked with OUR_REF. */ head_ref_namespaced(check_ref, data); - for_each_namespaced_ref(check_ref, data); + for_each_namespaced_ref(NULL, check_ref, data); get_reachable_list(data, &reachable_shallows); result = get_shallow_commits(&reachable_shallows, @@ -1386,7 +1386,7 @@ void upload_pack(const int advertise_refs, const int stateless_rpc, if (advertise_refs) data.no_done = 1; head_ref_namespaced(send_ref, &data); - for_each_namespaced_ref(send_ref, &data); + for_each_namespaced_ref(NULL, send_ref, &data); if (!data.sent_capabilities) { const char *refname = "capabilities^{}"; write_v0_ref(&data, refname, refname, null_oid()); @@ -1400,7 +1400,7 @@ void upload_pack(const int advertise_refs, const int stateless_rpc, packet_flush(1); } else { head_ref_namespaced(check_ref, &data); - for_each_namespaced_ref(check_ref, &data); + for_each_namespaced_ref(NULL, check_ref, &data); } if (!advertise_refs) { From patchwork Wed Jun 7 10:42:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4879CC77B7A for ; Wed, 7 Jun 2023 10:42:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234168AbjFGKm3 (ORCPT ); Wed, 7 Jun 2023 06:42:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54054 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239638AbjFGKmM (ORCPT ); Wed, 7 Jun 2023 06:42:12 -0400 Received: from mail-yw1-x112f.google.com (mail-yw1-x112f.google.com [IPv6:2607:f8b0:4864:20::112f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AC5F71FD5 for ; Wed, 7 Jun 2023 03:42:06 -0700 (PDT) Received: by mail-yw1-x112f.google.com with SMTP id 00721157ae682-5689335d2b6so76381847b3.3 for ; Wed, 07 Jun 2023 03:42:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134525; x=1688726525; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=8o+emfCOUVDPDpu7vdkZVw2FhYfkyaTLjACxTMDB1N0=; b=MO4V8nxZ9LHSQK6AEDhO3rbvFys304p+Rd2AHgvfwmrdMoIde9nBdYJjPM/XHgPQOs tDuRgmoE2v4BXYo1+KKaTA6MQUegVoXc+uVhVBmIGvCbb9+jZ2zZcyHOeSuKtNx/nR7Y nIBrKH44tU8IoxY/LqdQPLrLRYq6BEp6f/P8BwRH9I/Qe7ADg1kxuDwsTYhi8PwHulUT KM3Q1Pt5tA2VcIhLEGq3+kt8aphNw8cxp11D8rPAdkILizpSh0wucJeevFS6aP3fmgUf ZzZFoJHSoJlMWIqgKRE7tpX4hPUrFUyDeuPaf1kZAubCAocWtTjkWrFxTHqpXXMQz0jq WFDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134525; x=1688726525; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8o+emfCOUVDPDpu7vdkZVw2FhYfkyaTLjACxTMDB1N0=; b=ZubPXJ+7p0H61TN3aU0AKVfSgowiIoWU6J8VkWl7sNXb3ocVHblRnKtYf9479vG7h0 fqaXi6fWXkFjKdwT6OdozIAlGgHCF0nSvZf99qv2qGoMDyNbu6f6N3andKfrJFpN1kn8 BKfC1f6VRA6c1dDGCIXlfYm24TEF1ftNAdbJbBbGs+Wcm06JXkEWsyZCy3ypEOGihWPx 63FDdXj19okrPgcvfZVl2i1vAURFOyGf5yuNUHO+q9jG5aIVMfNDBjJroyuBJ1CQ+I2K 4dYD1Ktp1GBwnowRHAri07C+YGsTlSILnG+v9LOhGbbZC1y42smukazmX+tGt3sFEcOp zj+Q== X-Gm-Message-State: AC+VfDwfTwRZ4FaXmhzAa8aImc5uT+k1obohpMJALMHb8x+ZeEQGEFHX oudg0TKo+UFV/HZ00w5qx/jfm4D2N2n5Bw9HfpomVe7G X-Google-Smtp-Source: ACHHUZ6Q/O3nLCtZsf61mdtDtO7LgBVNOj/VFepKMhw5SgNMc9mAsffGy8AHNKeStFFrk09dXgEaSg== X-Received: by 2002:a0d:e296:0:b0:565:c7dd:95b9 with SMTP id l144-20020a0de296000000b00565c7dd95b9mr6066312ywe.12.1686134525098; Wed, 07 Jun 2023 03:42:05 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id m5-20020a81d245000000b005694b0ae1c6sm4667506ywl.126.2023.06.07.03.42.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:42:04 -0700 (PDT) Date: Wed, 7 Jun 2023 06:42:02 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 14/16] builtin/receive-pack.c: avoid enumerating hidden references Message-ID: <1db10b76eaa4054fb0c065fe4e15449f07e3d1c5.1686134440.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Now that `refs_for_each_fullref_in()` has the ability to avoid enumerating references matching certain pattern(s), use that to avoid visiting hidden refs when constructing the ref advertisement via receive-pack. Note that since this exclusion is best-effort, we still need `show_ref_cb()` to check whether or not each reference is hidden or not before including it in the advertisement. As was the case when applying this same optimization to `upload-pack`, `receive-pack`'s reference advertisement phase can proceed much quicker by avoiding enumerating references that will not be part of the advertisement. (Below, we're still using linux.git with one hidden refs/pull/N ref per commit): $ hyperfine -L v ,.compile 'git{v} -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git' Benchmark 1: git -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git Time (mean ± σ): 89.1 ms ± 1.7 ms [User: 82.0 ms, System: 7.0 ms] Range (min … max): 87.7 ms … 95.5 ms 31 runs Benchmark 2: git.compile -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git Time (mean ± σ): 4.5 ms ± 0.2 ms [User: 0.5 ms, System: 3.9 ms] Range (min … max): 4.1 ms … 5.6 ms 508 runs Summary 'git.compile -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git' ran 20.00 ± 1.05 times faster than 'git -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git' Signed-off-by: Taylor Blau --- builtin/receive-pack.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 1a8472eddc..bd5bcc375f 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -337,7 +337,8 @@ static void write_head_info(void) { static struct oidset seen = OIDSET_INIT; - for_each_ref(show_ref_cb, &seen); + refs_for_each_fullref_in(get_main_ref_store(the_repository), "", + hidden_refs.v, show_ref_cb, &seen); for_each_alternate_ref(show_one_alternate_ref, &seen); oidset_clear(&seen); if (!sent_capabilities) From patchwork Wed Jun 7 10:42:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270360 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79BD5C7EE23 for ; Wed, 7 Jun 2023 10:42:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239763AbjFGKmh (ORCPT ); Wed, 7 Jun 2023 06:42:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53958 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239710AbjFGKmY (ORCPT ); Wed, 7 Jun 2023 06:42:24 -0400 Received: from mail-yw1-x112c.google.com (mail-yw1-x112c.google.com [IPv6:2607:f8b0:4864:20::112c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 22FEF1BF1 for ; Wed, 7 Jun 2023 03:42:11 -0700 (PDT) Received: by mail-yw1-x112c.google.com with SMTP id 00721157ae682-565bd368e19so76372147b3.1 for ; Wed, 07 Jun 2023 03:42:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134530; x=1688726530; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=CfxZiFyK3h2tCysbyRnTB46yuTEa8xu8x3OV/PpqJQk=; b=lt7Zr0lSZUh6cveu/cZwXoVmFhNMkyUvjJ7gGNcUGK38MjN2WrOEsLXSnkuJQtyudH Vu72yrE/qFmb9NDhv5JUXP7bclDdaj5WkMs4AUN/4W5vpPQI0IkSqZPUHbrcci71wbkk Ki/aXpRaq3nEhTzKnN6cA8xujTIJcE+0moPkX1v4iilZmoA92DfRdfNk4Lth2b0kTLo4 6+VKbsWPtSck9/Xjkx4L2BcC3c8k0IYa88gcv2hF+qwvS1G3OPxIyFOa+4q/GCHkkj9d W1p9Azajw3LaBiL0TD99lzN95PmXpN4t3yQsYKRICuY0J5ORrPI2G3HsWuGPjusfCKga mnPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134530; x=1688726530; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=CfxZiFyK3h2tCysbyRnTB46yuTEa8xu8x3OV/PpqJQk=; b=RvH1BLt5Y2z+11L+CZCua4ClyD9WA6AtMaRvMuhgNlDtgbHOBjUR4lFYkEyyWwcXZ8 wVcRFneXt7Qkk07064dCqHbrqhGO1tkXUNso9VV1jXYw07+rliqkJlKby8jvweuVcvTK 1re0eQasitkn6wdwNqXpC/LdalbPJesflCw6y4uZxrCAa7YlX/FmyPXE141zDpbpIQc6 b+Dc6xBZnFH713X3nHDmUxqsv2bkFhzacXebT/Zc+Zk4lRuHQZydGTICEJTXJQWeMQ/j +/8CnLMYHyzDbsr9lS/gtstYOIylZ23e7vqhFQOyw+o1P2bp7fjRYRD7iSWAn8r+LiyV 1fpA== X-Gm-Message-State: AC+VfDxDFFkLbXmdI4n5wWmuUeVNJXlCMj1MJmhhiODcaoZgAz12YRmn 6gXgMD2/+RgaAHhAP0GAZ4gpdWmXIh7WuMFyJNDFD+Ft X-Google-Smtp-Source: ACHHUZ63TmvZ1An+C68lbFE2gfsWOxhUdmAIkfk5RubgCr/57WiwHhHZuBegv5voHLNX5THsqIY/7w== X-Received: by 2002:a81:6006:0:b0:568:ada0:f9ea with SMTP id u6-20020a816006000000b00568ada0f9eamr5814854ywb.49.1686134530019; Wed, 07 Jun 2023 03:42:10 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id x186-20020a817cc3000000b00545a081849esm4753549ywc.46.2023.06.07.03.42.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:42:09 -0700 (PDT) Date: Wed, 7 Jun 2023 06:42:07 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 15/16] upload-pack.c: avoid enumerating hidden refs where possible Message-ID: <014243ebe6586313b38fe0eb2a57941b27131423.1686134440.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a similar fashion as a previous commit, teach `upload-pack` to avoid enumerating hidden references where possible. Note, however, that there are certain cases where cannot avoid enumerating even hidden references, in particular when either of: - `uploadpack.allowTipSHA1InWant`, or - `uploadpack.allowReachableSHA1InWant` are set, corresponding to `ALLOW_TIP_SHA1` and `ALLOW_REACHABLE_SHA1`, respectively. When either of these bits are set, upload-pack's `is_our_ref()` function needs to consider the `HIDDEN_REF` bit of the referent's object flags. So we must visit all references, including the hidden ones, in order to mark their referents with the `HIDDEN_REF` bit. When neither `ALLOW_TIP_SHA1` nor `ALLOW_REACHABLE_SHA1` are set, the `is_our_ref()` function considers only the `OUR_REF` bit, and not the `HIDDEN_REF` one. `OUR_REF` is applied via `mark_our_ref()`, and only to objects at the tips of non-hidden references, so we do not need to visit hidden references in this case. When neither of those bits are set, `upload-pack` can potentially avoid enumerating a large number of references. In the same example as a previous commit (linux.git with one hidden reference per commit, "refs/pull/N"): $ printf 0000 >in $ hyperfine --warmup=1 \ 'git -c transfer.hideRefs=refs/pull upload-pack . --- upload-pack.c | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/upload-pack.c b/upload-pack.c index 99d216938c..366a101d8d 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -602,11 +602,32 @@ static int get_common_commits(struct upload_pack_data *data, } } +static int allow_hidden_refs(enum allow_uor allow_uor) +{ + return allow_uor & (ALLOW_TIP_SHA1 | ALLOW_REACHABLE_SHA1); +} + +static void for_each_namespaced_ref_1(each_ref_fn fn, + struct upload_pack_data *data) +{ + /* + * If `data->allow_uor` allows fetching hidden refs, we need to + * mark all references (including hidden ones), to check in + * `is_our_ref()` below. + * + * Otherwise, we only care about whether each reference's object + * has the OUR_REF bit set or not, so do not need to visit + * hidden references. + */ + if (allow_hidden_refs(data->allow_uor)) + for_each_namespaced_ref(NULL, fn, data); + else + for_each_namespaced_ref(data->hidden_refs.v, fn, data); +} + static int is_our_ref(struct object *o, enum allow_uor allow_uor) { - int allow_hidden_ref = (allow_uor & - (ALLOW_TIP_SHA1 | ALLOW_REACHABLE_SHA1)); - return o->flags & ((allow_hidden_ref ? HIDDEN_REF : 0) | OUR_REF); + return o->flags & ((allow_hidden_refs(allow_uor) ? HIDDEN_REF : 0) | OUR_REF); } /* @@ -855,7 +876,7 @@ static void deepen(struct upload_pack_data *data, int depth) * marked with OUR_REF. */ head_ref_namespaced(check_ref, data); - for_each_namespaced_ref(NULL, check_ref, data); + for_each_namespaced_ref_1(check_ref, data); get_reachable_list(data, &reachable_shallows); result = get_shallow_commits(&reachable_shallows, @@ -1386,7 +1407,7 @@ void upload_pack(const int advertise_refs, const int stateless_rpc, if (advertise_refs) data.no_done = 1; head_ref_namespaced(send_ref, &data); - for_each_namespaced_ref(NULL, send_ref, &data); + for_each_namespaced_ref_1(send_ref, &data); if (!data.sent_capabilities) { const char *refname = "capabilities^{}"; write_v0_ref(&data, refname, refname, null_oid()); @@ -1400,7 +1421,7 @@ void upload_pack(const int advertise_refs, const int stateless_rpc, packet_flush(1); } else { head_ref_namespaced(check_ref, &data); - for_each_namespaced_ref(NULL, check_ref, &data); + for_each_namespaced_ref_1(check_ref, &data); } if (!advertise_refs) { From patchwork Wed Jun 7 10:42:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13270361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D86AAC77B7A for ; Wed, 7 Jun 2023 10:42:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239806AbjFGKmn (ORCPT ); Wed, 7 Jun 2023 06:42:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53706 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239480AbjFGKm0 (ORCPT ); Wed, 7 Jun 2023 06:42:26 -0400 Received: from mail-yw1-x1136.google.com (mail-yw1-x1136.google.com [IPv6:2607:f8b0:4864:20::1136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC83D2115 for ; Wed, 7 Jun 2023 03:42:15 -0700 (PDT) Received: by mail-yw1-x1136.google.com with SMTP id 00721157ae682-565c7399afaso80001717b3.1 for ; Wed, 07 Jun 2023 03:42:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1686134535; x=1688726535; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=oFHCUunFc1C2OxzI27ntXEVCAIvtN6LZX1fBwmQ6qqw=; b=XBd2A1NR/xr9eQXEyztRFZoNX9JypO9LLb6xRIU6eQdLptxo/YdjIZzDCb4xAd3alY Se79bQwUy3R2tKqykWq415iRU44mB0QS82r8iJrPnNol+kOg5kbpvTUGcnfmoVD4bA9s GjUUNhdtwT6ZocpIz7oj3PX2yyXzO2gY9urESppCG29OhRNfB48oC7MxK/Gc/lk/PIQ3 2I+ezjMM+eIwdEknI4O91UIOSlmDAnWYLNuHCEqmuZALPUxAG3Ncv51FZg7+DSL7WqqL JRdn8W04cPKec7jv9tegm/f3clzZjvDZ9gxd0VqOuNu9SGjDZ428RIop9hG6u/liji+d pkUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686134535; x=1688726535; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oFHCUunFc1C2OxzI27ntXEVCAIvtN6LZX1fBwmQ6qqw=; b=ieM8wHRtibZYszlzXPikqoYCdjl6c+ebN1pEmCBoWSAYxcikX9g7hCTh0N7TiWznKO QJE3+XtJcp5hRe8wmiIYrBmCRJaXc+l5wbghCVFvBTVfbCy70G5dziBOdLwDG8/9EW+D nPOkZNI99DhqFdvDZd06HKyBlQqPxcm70zamsigCHE9VdpHxa6DHYA6DIPCeAQ+27NJo W81ecVGVKUpfeApGl6ftPXa7WV1hIx4JbpTvwdXgeP2OVfxHx9rC0HBPkx6mIJExOcrk rn7YVTFszUbk4C0EVePUXwQ4U/GqduDUzmfl6hLnLOldh5iFis70TVW2E9B+DChnVtd+ zBHQ== X-Gm-Message-State: AC+VfDzL9fjM1/azPkhVfd0aiMvq6aRI1MX9OKB6/7iW6+AfHasC3OuO oZ/mFp/PJhss8Btk6L/7L2/s5euobO95WtKdJaU2ydCo X-Google-Smtp-Source: ACHHUZ4eWZ9swHUCeOOLcdFZ1qKMwB+B8AquPAIGZ22zXkkgPFSdgmqZW+VNtmcYeXXyBUBPSS+CjQ== X-Received: by 2002:a0d:cb07:0:b0:55a:5ce4:aff2 with SMTP id n7-20020a0dcb07000000b0055a5ce4aff2mr5268762ywd.39.1686134534925; Wed, 07 Jun 2023 03:42:14 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id d14-20020a81d34e000000b0055a931afe48sm4721739ywl.8.2023.06.07.03.42.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jun 2023 03:42:14 -0700 (PDT) Date: Wed, 7 Jun 2023 06:42:11 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v3 16/16] ls-refs.c: avoid enumerating hidden refs where possible Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a similar fashion as in previous commits, teach `ls-refs` to avoid enumerating hidden references where possible. As before, this is linux.git with one hidden reference per commit. $ hyperfine -L v ,.compile 'git{v} -c protocol.version=2 ls-remote .' Benchmark 1: git -c protocol.version=2 ls-remote . Time (mean ± σ): 89.8 ms ± 0.6 ms [User: 84.3 ms, System: 5.7 ms] Range (min … max): 88.8 ms … 91.3 ms 32 runs Benchmark 2: git.compile -c protocol.version=2 ls-remote . Time (mean ± σ): 6.5 ms ± 0.1 ms [User: 2.4 ms, System: 4.3 ms] Range (min … max): 6.2 ms … 8.3 ms 397 runs Summary 'git.compile -c protocol.version=2 ls-remote .' ran 13.85 ± 0.33 times faster than 'git -c protocol.version=2 ls-remote .' Signed-off-by: Taylor Blau --- ls-refs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ls-refs.c b/ls-refs.c index 8c3181d051..c9a723ba89 100644 --- a/ls-refs.c +++ b/ls-refs.c @@ -193,7 +193,7 @@ int ls_refs(struct repository *r, struct packet_reader *request) strvec_push(&data.prefixes, ""); refs_for_each_fullref_in_prefixes(get_main_ref_store(r), get_git_namespace(), data.prefixes.v, - NULL, send_ref, &data); + data.hidden_refs.v, send_ref, &data); packet_fflush(stdout); strvec_clear(&data.prefixes); strbuf_release(&data.buf);