From patchwork Wed Jan 20 16:53:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12032891 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D092FC433DB for ; Wed, 20 Jan 2021 16:55:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A194223358 for ; Wed, 20 Jan 2021 16:55:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2403956AbhATQzC (ORCPT ); Wed, 20 Jan 2021 11:55:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56072 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403868AbhATQyo (ORCPT ); Wed, 20 Jan 2021 11:54:44 -0500 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57298C061757 for ; Wed, 20 Jan 2021 08:54:03 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id q7so1833879wre.13 for ; Wed, 20 Jan 2021 08:54:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=/QDPQ+/9yTiAK2nGZzt9251CqQ8MsbzHs5AF++igo8I=; b=PfMSOSOQYDk20zXv2FCe2fxxd7xEG/3m47Bdv4FOGyiqes8PVFiaI03xhxEeOStR7F N0OQWOkRZP3OPkAOLaAKYXr8bY51zJFit77zQFwHe7Zlj0lXjF1J5a+u+9nIsYAnyvxe oPFga9qP1LVmak62ld1km+1ss3NH2Wk9OMpqyAvyYw12VX0RRFINSB0OEbZJTJcFzNxS NZfJdqw9bBOHelq4R9DQ26jyd5PMNHsdTr++lMzMxdOuU/mE3e98Hy3QTH2e/Vbh24oL 1DnNHSTcyHFhv1v1DnzpA/fxU0n6pSV1RKo993SNeGfEejGWU5py7G+PrUpf23/TvjME XbiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=/QDPQ+/9yTiAK2nGZzt9251CqQ8MsbzHs5AF++igo8I=; b=D4SPURhvWkqG63hKBi6Ef/Uo4MZnT3F6OI7dk+rvuIv8z5GcF6eOVx/GPmFnQSsgv3 sgvL/v24PIT/tGUSDfsvOuQ7A39GgDJ9QUuger0z9isGbLTHJMnUfnYDXTCuXxiNgC69 tdvzT9TSXqisHlSyjlBCW2YDiMmGm3qEU1XXY1Izvdi0gZTK9ghUnGNcGFUE2IxosRUc H0ZhPMVmRukHVWWV53BvGkwmaY3PMsfxoDnzVtHRaPytIqRscGknrHwaRNb6kSl364Vg LFGy+3vm4a3jY7uSqIzQkoBCeZJbZrLQAH6iDdv7IOXCutQ8x+u/mpGGBaGi8lz4gYbO zDQA== X-Gm-Message-State: AOAM5327IAw83l/W3qYFCaGwOE04ekQ6v5tOm8MfBYfwn97JAX/qt89n SQ1JTB6bAhHeVj5oGkOnmdYYV2KbLKs= X-Google-Smtp-Source: ABdhPJyXBjA+Q4ByR5wGfqGvDWSou7E3fCXDaJpy+lhqWMDngyWrz9u7Wdla96QcBH969NBmBiAfYg== X-Received: by 2002:adf:e8c5:: with SMTP id k5mr10587478wrn.242.1611161641855; Wed, 20 Jan 2021 08:54:01 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m82sm5159538wmf.29.2021.01.20.08.54.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Jan 2021 08:54:01 -0800 (PST) Message-Id: <0bccfd34ae5924aef0432fd6727debb75c052da5.1611161639.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 20 Jan 2021 16:53:51 +0000 Subject: [PATCH 1/9] cache-tree: clean up cache_tree_update() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Make the method safer by allocating a cache_tree member for the given index_state if it is not already present. Also drop local variables that are used exactly once and can be found directly from the 'istate' parameter. Signed-off-by: Derrick Stolee --- cache-tree.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/cache-tree.c b/cache-tree.c index 3f1a8d4f1b7..c1e49901c17 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -436,16 +436,20 @@ static int update_one(struct cache_tree *it, int cache_tree_update(struct index_state *istate, int flags) { - struct cache_tree *it = istate->cache_tree; - struct cache_entry **cache = istate->cache; - int entries = istate->cache_nr; - int skip, i = verify_cache(cache, entries, flags); + int skip, i; + + i = verify_cache(istate->cache, istate->cache_nr, flags); if (i) return i; + + if (!istate->cache_tree) + istate->cache_tree = cache_tree(); + trace_performance_enter(); trace2_region_enter("cache_tree", "update", the_repository); - i = update_one(it, cache, entries, "", 0, &skip, flags); + i = update_one(istate->cache_tree, istate->cache, istate->cache_nr, + "", 0, &skip, flags); trace2_region_leave("cache_tree", "update", the_repository); trace_performance_leave("cache_tree_update"); if (i < 0) From patchwork Wed Jan 20 16:53:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12032921 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5CF60C433E6 for ; Wed, 20 Jan 2021 16:57:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3015C233EB for ; Wed, 20 Jan 2021 16:57:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391544AbhATQ5U (ORCPT ); Wed, 20 Jan 2021 11:57:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56076 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403908AbhATQyo (ORCPT ); Wed, 20 Jan 2021 11:54:44 -0500 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41450C0613C1 for ; Wed, 20 Jan 2021 08:54:04 -0800 (PST) Received: by mail-wm1-x330.google.com with SMTP id y187so3442638wmd.3 for ; Wed, 20 Jan 2021 08:54:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=11GVRZ1uX92OrDukLEJw+fr6Pd0hb/0vgREeqfoK+sw=; b=a91G3NAezRC2Qy8gcxJeWTJrmJXouRLbdxvooMUaKSVD5owYmcUq7G3UwklIK7RMIU PrBzaxo3KaUmSzvN6o19sj/NylNhXnr5kq1j88bjj0oybDYpPTDh17llmAlduivzTKwu DjByrGFDGWCKLdpP5+qyJpgGYP8/M2Kxwm0irgmZx+nkCnEEp45MKLWSrzARIzG2cFgC 3FBqJh2+/dQfUfEI343oejkgf/W8H1V56lZU8Lf9I35fbGVnrmUmXwlu/gmWOslJytg0 lyKu1qtFSdDRljlebG6lAirpxdzyQyIbdqGnMbDV08lAv1C6oA61SlOHy3eDpKAoDWIO e7RQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=11GVRZ1uX92OrDukLEJw+fr6Pd0hb/0vgREeqfoK+sw=; b=RdHOh/osrMfpx68wWWvxICuaaQzv0gdMa1k7O8AcC4llk+33k8Eo0RZ70eYXZYnC8k T5v1n2/JLwmmfdpiZojftAqUtBLfC5yK9YaBggSuPGgNLuJLD3GiZZHULq9p5ZipL4Nn dHpSrN4I0M28JX6sPjrFRX6iJuXRvdEXkkPe+1fasoSUd5dCJdTZ8uGvejOAbw1jqWhN GziW1/HYyU98HgMAhf97deVBZITjJhojIuEyDYd1qbOdx1Tx/aTJ5pAXftl70H9oO7e0 suizEqmLTKjg196nUqW41QD1Nyro6Fz412xqcztCldLwmkEYZl77e/qP/Gayb6c5dh3H NsIw== X-Gm-Message-State: AOAM533yeINNgEcEZO+ocOlY3Rvlmygc9EqXq4b7asSYOnzlEAFQrIPZ gYIZuN7So/bfvli7vlMY5Iig29s6jJ0= X-Google-Smtp-Source: ABdhPJwniv2h5kWpvqtKIduC4m+nfOLEMmdlcenr9KKzrx9sE7GRLtxqlH83i2qC0GNPant3cFFqUQ== X-Received: by 2002:a1c:620a:: with SMTP id w10mr5042058wmb.167.1611161642809; Wed, 20 Jan 2021 08:54:02 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j59sm5362771wrj.13.2021.01.20.08.54.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Jan 2021 08:54:02 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 20 Jan 2021 16:53:52 +0000 Subject: [PATCH 2/9] cache-tree: extract subtree_pos() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee This method will be helpful to use outside of cache-tree.c in a later feature. The implementation is subtle due to subtree_name_cmp() sorting by length and then lexicographically. Signed-off-by: Derrick Stolee --- cache-tree.c | 6 +++--- cache-tree.h | 2 ++ 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/cache-tree.c b/cache-tree.c index c1e49901c17..2b130dd5e19 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -45,7 +45,7 @@ static int subtree_name_cmp(const char *one, int onelen, return memcmp(one, two, onelen); } -static int subtree_pos(struct cache_tree *it, const char *path, int pathlen) +int cache_tree_subtree_pos(struct cache_tree *it, const char *path, int pathlen) { struct cache_tree_sub **down = it->down; int lo, hi; @@ -72,7 +72,7 @@ static struct cache_tree_sub *find_subtree(struct cache_tree *it, int create) { struct cache_tree_sub *down; - int pos = subtree_pos(it, path, pathlen); + int pos = cache_tree_subtree_pos(it, path, pathlen); if (0 <= pos) return it->down[pos]; if (!create) @@ -123,7 +123,7 @@ static int do_invalidate_path(struct cache_tree *it, const char *path) it->entry_count = -1; if (!*slash) { int pos; - pos = subtree_pos(it, path, namelen); + pos = cache_tree_subtree_pos(it, path, namelen); if (0 <= pos) { cache_tree_free(&it->down[pos]->cache_tree); free(it->down[pos]); diff --git a/cache-tree.h b/cache-tree.h index 639bfa5340e..8efeccebfc9 100644 --- a/cache-tree.h +++ b/cache-tree.h @@ -27,6 +27,8 @@ void cache_tree_free(struct cache_tree **); void cache_tree_invalidate_path(struct index_state *, const char *); struct cache_tree_sub *cache_tree_sub(struct cache_tree *, const char *); +int cache_tree_subtree_pos(struct cache_tree *it, const char *path, int pathlen); + void cache_tree_write(struct strbuf *, struct cache_tree *root); struct cache_tree *cache_tree_read(const char *buffer, unsigned long size); From patchwork Wed Jan 20 16:53:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12032893 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAFC2C433E0 for ; Wed, 20 Jan 2021 16:55:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A050523358 for ; Wed, 20 Jan 2021 16:55:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2403924AbhATQzN (ORCPT ); Wed, 20 Jan 2021 11:55:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403913AbhATQyp (ORCPT ); Wed, 20 Jan 2021 11:54:45 -0500 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D3E6C0613CF for ; Wed, 20 Jan 2021 08:54:05 -0800 (PST) Received: by mail-wr1-x42a.google.com with SMTP id a12so23741140wrv.8 for ; Wed, 20 Jan 2021 08:54:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=3tMIyV1e5H7KibNfv8wSK28+QZ3PH8wQIwxwCCv8pUg=; b=h0fNgGhKlyAu0s9wZVwkgSt1FokDrb2Yh1Nd5crgRHgCXNl19m4YNsGBDClL1VGkDk FsWcmZjTOE99xKLafM3VMEkv/+CKKq+L4sUeGGqXhCMLeindW0MDF0Zk+NwJGQmenOH6 QhoX59ZTayoP7vDzxJkDXwtiNmI1On057tev4tmFhIHGlfWGsx9oaApg4UEWsMySvSlk yuy8x2ye/ib9StvJeDfPH3/QHq6185uiNYtXk8xRkaOvxtZZd9zt2vwYL9vfRyxMMaTS 4+DEPSbI/4oMeKFdTA2DxVw0sqmagqt3mCJyqVU55KYT5hdiGlN1Lm7BouLljVNSMNgY e3Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=3tMIyV1e5H7KibNfv8wSK28+QZ3PH8wQIwxwCCv8pUg=; b=PkZeLIbqnhJdthpuhvvGNecQAagXtYYxFX+UMWJYWxll33WhKlBMA3w4yx1EmPcpUL hXbYzohpiNB2KslHeKK75Mj/5mxjVIbAU0Zf23/m8dLKt1GzKKHOQkH0EiRJDjM+w0s3 o8HhnpuiZ3AaKR0vnFCxP7McuZgJwxIxfqwrk3PkxwT1wKpMsK2BaMYSZXEFG6nmSS2d mmOMp78wWz+Y64yG3oiLSwuJUQcFMbCeJAmF2roCMJTdnF5liVbLw/33xO87wcRsVEkW du1kNkuWe3H0ByzwTQ+E5GGp3MzuoVT8A5C1pfbcvU9TU0UHmUmFEYTGmh4TpzCydClA LGow== X-Gm-Message-State: AOAM530sdnKHk8cfTqSG3RwxYa8fFvI3t69MUmg3Hc6Da27PnvByP9Yd LmMLE+DULVsGBTISC8EAhLw1Bdwlv1o= X-Google-Smtp-Source: ABdhPJxOkho8uHZof8L4XQnZAdaK0o09rJAUbtH1IEzOGidPFdwaw/rRgtAXx0783SaIcbe/lj6Nsg== X-Received: by 2002:adf:f684:: with SMTP id v4mr10398931wrp.387.1611161643690; Wed, 20 Jan 2021 08:54:03 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n12sm5776714wrg.76.2021.01.20.08.54.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Jan 2021 08:54:03 -0800 (PST) Message-Id: <838922de2e9756c00f4a159b2b8722ae4d28b011.1611161639.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 20 Jan 2021 16:53:53 +0000 Subject: [PATCH 3/9] fsmonitor: de-duplicate BUG()s around dirty bits Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The index has an fsmonitor_dirty bitmap that records which index entries are "dirty" based on the response from the FSMonitor. If this bitmap ever grows larger than the index, then there was an error in how it was constructed, and it was probably a developer's bug. There are several BUG() statements that are very similar, so replace these uses with a simpler assert_index_minimum(). Since there is one caller that uses a custom 'pos' value instead of the bit_size member, we cannot simplify it too much. However, the error string is identical in each, so this simplifies things. The end result is that the code is simpler to read while also preserving these assertions for developers in the FSMonitor space. Signed-off-by: Derrick Stolee --- fsmonitor.c | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/fsmonitor.c b/fsmonitor.c index ca031c3abb8..52a50a9545a 100644 --- a/fsmonitor.c +++ b/fsmonitor.c @@ -13,14 +13,19 @@ struct trace_key trace_fsmonitor = TRACE_KEY_INIT(FSMONITOR); +static void assert_index_minimum(struct index_state *istate, size_t pos) +{ + if (pos > istate->cache_nr) + BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" > %u)", + (uintmax_t)pos, istate->cache_nr); +} + static void fsmonitor_ewah_callback(size_t pos, void *is) { struct index_state *istate = (struct index_state *)is; struct cache_entry *ce; - if (pos >= istate->cache_nr) - BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" >= %u)", - (uintmax_t)pos, istate->cache_nr); + assert_index_minimum(istate, pos); ce = istate->cache[pos]; ce->ce_flags &= ~CE_FSMONITOR_VALID; @@ -82,10 +87,8 @@ int read_fsmonitor_extension(struct index_state *istate, const void *data, } istate->fsmonitor_dirty = fsmonitor_dirty; - if (!istate->split_index && - istate->fsmonitor_dirty->bit_size > istate->cache_nr) - BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" > %u)", - (uintmax_t)istate->fsmonitor_dirty->bit_size, istate->cache_nr); + if (!istate->split_index) + assert_index_minimum(istate, istate->fsmonitor_dirty->bit_size); trace_printf_key(&trace_fsmonitor, "read fsmonitor extension successful"); return 0; @@ -110,10 +113,8 @@ void write_fsmonitor_extension(struct strbuf *sb, struct index_state *istate) uint32_t ewah_size = 0; int fixup = 0; - if (!istate->split_index && - istate->fsmonitor_dirty->bit_size > istate->cache_nr) - BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" > %u)", - (uintmax_t)istate->fsmonitor_dirty->bit_size, istate->cache_nr); + if (!istate->split_index) + assert_index_minimum(istate, istate->fsmonitor_dirty->bit_size); put_be32(&hdr_version, INDEX_EXTENSION_VERSION2); strbuf_add(sb, &hdr_version, sizeof(uint32_t)); @@ -335,9 +336,7 @@ void tweak_fsmonitor(struct index_state *istate) } /* Mark all previously saved entries as dirty */ - if (istate->fsmonitor_dirty->bit_size > istate->cache_nr) - BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" > %u)", - (uintmax_t)istate->fsmonitor_dirty->bit_size, istate->cache_nr); + assert_index_minimum(istate, istate->fsmonitor_dirty->bit_size); ewah_each_bit(istate->fsmonitor_dirty, fsmonitor_ewah_callback, istate); refresh_fsmonitor(istate); From patchwork Wed Jan 20 16:53:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12032895 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8185C433E0 for ; Wed, 20 Jan 2021 16:56:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 788BA23358 for ; Wed, 20 Jan 2021 16:56:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731111AbhATQzb (ORCPT ); Wed, 20 Jan 2021 11:55:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403947AbhATQyq (ORCPT ); Wed, 20 Jan 2021 11:54:46 -0500 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DABF2C0613D3 for ; Wed, 20 Jan 2021 08:54:05 -0800 (PST) Received: by mail-wm1-x332.google.com with SMTP id y187so3442732wmd.3 for ; Wed, 20 Jan 2021 08:54:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ZV0JWSbubCzVhkJnD425A62mw4d+BJkeIo6hw/50c5A=; b=O52Nc/EIkPMykFCd587M7tRBn4T79aoOqEouo90H5JoRxnUAOB78TTl+1eWLxeSmIy pkFxU+xXQgl2F0lrVgtlhv/7hZnP1n5hAJ6a1G2M7xBOySc3vKlv6+0EpR7h71BFFV4e 0QNARSf7d05ScfhG+tA2cfRStYpvZFAFfDUfSNsmJv6JMlUlSzdoO6UcPwlhv2Iy/VkF nn2XL1zTBRAeh79Q8pRzx+ch79mfBGaZJPS7CoI4X79PDExUtjBg05gr7uh9wdQSmkBC RyYeT0fluLIBYNi1eDrPjdUSHIbttX3A1+Ey62jR+ixnWN8STRs0DWBKcmLNXlBv6bXw c1tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ZV0JWSbubCzVhkJnD425A62mw4d+BJkeIo6hw/50c5A=; b=Rab8IXqiDtiY86Ta96Kj37m6PmFckwAXO6Fm2JiETU+fIhCSqX2z6QvvMvTvIwCdWf Clc/UxPCH0lKL9g7IkqoJAyXUcm1dKouJ/2tZnjGtRpYhKKYFevllqeJ0E4scLyogN69 dLQT/29H40sX0A8BjX8FHeBXbwUIBlqAVQAuvA7s4vWIQzGEXz6c6RM7yCRGbp4Ie1Rz edIckHla0ggz5x1lSpjPT8S7Od0CWTKBf+RvUriwx38k4WYi+tzXbvMFItjAXSLSBBtd Fur3+fUqNqkM0jNfyeOgkjg/Jx9Nq/0biqxuWdO4W8UFU9STrCpiAjx0WRQHO7hdWq5t dDqA== X-Gm-Message-State: AOAM531YvTMI9Y1UOJOtMtWUMoO/I8KD0geo25hBqYtYonV24/+WRBIp CkQ91x1gBKfOEdgRQLL1h0s8OiFzwdo= X-Google-Smtp-Source: ABdhPJw0WvgE7uizpka5T/PtQtmHt2r+hC2Lh6DJCizpH8ZTvMABWaJz7aT8TKdXHDRWWRE57b4PFw== X-Received: by 2002:a1c:e1d4:: with SMTP id y203mr3863893wmg.50.1611161644523; Wed, 20 Jan 2021 08:54:04 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d7sm4495421wmb.47.2021.01.20.08.54.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Jan 2021 08:54:04 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 20 Jan 2021 16:53:54 +0000 Subject: [PATCH 4/9] repository: add repo reference to index_state Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee It will be helpful to add behavior to index opertations that might trigger an object lookup. Since each index belongs to a specific repository, add a 'repo' pointer to struct index_state that allows access to this repository. This will prevent future changes from needing to pass an additional 'struct repository *repo' parameter and instead rely only on the 'struct index_state *istate' parameter. Signed-off-by: Derrick Stolee --- cache.h | 1 + repository.c | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/cache.h b/cache.h index 71097657489..f9c7a603841 100644 --- a/cache.h +++ b/cache.h @@ -328,6 +328,7 @@ struct index_state { struct ewah_bitmap *fsmonitor_dirty; struct mem_pool *ce_mem_pool; struct progress *progress; + struct repository *repo; }; /* Name hashing */ diff --git a/repository.c b/repository.c index a4174ddb062..67a4c1da2d9 100644 --- a/repository.c +++ b/repository.c @@ -264,6 +264,10 @@ int repo_read_index(struct repository *repo) if (!repo->index) repo->index = xcalloc(1, sizeof(*repo->index)); + /* Complete the double-reference */ + if (!repo->index->repo) + repo->index->repo = repo; + return read_index_from(repo->index, repo->index_file, repo->gitdir); } From patchwork Wed Jan 20 16:53:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12032897 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D66BBC433E0 for ; Wed, 20 Jan 2021 16:56:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9B77B23358 for ; Wed, 20 Jan 2021 16:56:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404083AbhATQzq (ORCPT ); Wed, 20 Jan 2021 11:55:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56236 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404073AbhATQzZ (ORCPT ); Wed, 20 Jan 2021 11:55:25 -0500 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC00CC0613D6 for ; Wed, 20 Jan 2021 08:54:06 -0800 (PST) Received: by mail-wr1-x42e.google.com with SMTP id b5so1012669wrr.10 for ; Wed, 20 Jan 2021 08:54:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=3LC/1eestJjR+EOcwKHsjfdOZknum6kvDyZ7fg9I1AA=; b=IylzHAtr/pdLB2jWyg0Lz2dNVhgjiVyl0JbBlBFs5+CKDmkDkyhG4pJwRRdnn4QwOZ wFoMOZX0tRTRAHoLtc1npqNp244BdHtaeJYfk9Gd4eaiOfO0O4unGSa7Fn4Ik/8Rm3JN 4iGJgUcwJaNdvk3J2JImcFIFWCBGuIpwD2Ozx+kNr2reb4KGs+6Z4Nv6lGSd4p0ez+/0 S0KWVGk24TUZhm8Tzezg8stLKNE9rtuRskbzCWncVKZxsImyWLKtwqvU6flngicLKtDn xHcNLn4BwI3wFTh+LAjpbsbS4wmHDUy6aN1oZov5boZiETti0MgryHRXuCs7CSSEa8zc qMmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=3LC/1eestJjR+EOcwKHsjfdOZknum6kvDyZ7fg9I1AA=; b=AcfDS0annJSqHLLa3BtrEXoDBpqnHxZGfHx2oV3IsgJGXuhP2h4hM3b7RyxqS17jQl usKtivgqWh+YtWvSmdA77rj51hOjEOUe54temgzB6j+HnoE6/n+F85yKXOVwP2LXqFix v8CKz8T9BOI4LSOI2coLCNLEwOAays99Zu15ttWO9KMTrN6J/CU5UqahiZyksFtOwkZq svIMrDCCaxQclTU5eyy553rLCwxHO3yrMtCMA/7hE7te/2vWOMQ3GEBwmBXoOFiBiimy zaFrL7zdY5RmTBuvzwR4IpGvEPy393Qs7EgHw2WjEYkIgWIy/a2abmvbTZjgMHaU20dJ zdlA== X-Gm-Message-State: AOAM530ic2WpBU8FmhAsSx3fZj6xaV2/AWMAxscVTjM2YnDnCn9I121K VqH/mKsne8JBMbnA0aneoDhME249ECs= X-Google-Smtp-Source: ABdhPJwstZI5GEu8ezfU3HGGAhGmQOKbbWRg9YPzlwTey3YQJPxapk6hJ2HiKK3o/73LLAI/3yRzjg== X-Received: by 2002:adf:e704:: with SMTP id c4mr10245022wrm.355.1611161645440; Wed, 20 Jan 2021 08:54:05 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s4sm5053881wrt.85.2021.01.20.08.54.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Jan 2021 08:54:04 -0800 (PST) Message-Id: <3ba4b35f09cf3dc313787cdec6856286fe97d8a8.1611161639.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 20 Jan 2021 16:53:55 +0000 Subject: [PATCH 5/9] name-hash: use trace2 regions for init Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The lazy_init_name_hash() populates a hashset with all filenames and another with all directories represented in the index. This is run only if we need to use the hashsets to check for existence or case-folding renames. Place trace2 regions where there is already a performance trace. Signed-off-by: Derrick Stolee --- name-hash.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/name-hash.c b/name-hash.c index 5d3c7b12c18..4e03fac9bb1 100644 --- a/name-hash.c +++ b/name-hash.c @@ -7,6 +7,7 @@ */ #include "cache.h" #include "thread-utils.h" +#include "trace2.h" struct dir_entry { struct hashmap_entry ent; @@ -577,6 +578,7 @@ static void lazy_init_name_hash(struct index_state *istate) if (istate->name_hash_initialized) return; trace_performance_enter(); + trace2_region_enter("index", "name-hash-init", istate->repo); hashmap_init(&istate->name_hash, cache_entry_cmp, NULL, istate->cache_nr); hashmap_init(&istate->dir_hash, dir_entry_cmp, NULL, istate->cache_nr); @@ -597,6 +599,7 @@ static void lazy_init_name_hash(struct index_state *istate) } istate->name_hash_initialized = 1; + trace2_region_leave("index", "name-hash-init", istate->repo); trace_performance_leave("initialize name hash"); } From patchwork Wed Jan 20 16:53:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12032913 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 069F8C433E6 for ; Wed, 20 Jan 2021 16:56:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CC7AE23358 for ; Wed, 20 Jan 2021 16:56:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391172AbhATQ4k (ORCPT ); Wed, 20 Jan 2021 11:56:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404074AbhATQzZ (ORCPT ); Wed, 20 Jan 2021 11:55:25 -0500 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4458C0613ED for ; Wed, 20 Jan 2021 08:54:07 -0800 (PST) Received: by mail-wr1-x432.google.com with SMTP id a9so20318585wrt.5 for ; Wed, 20 Jan 2021 08:54:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=i6bsGcr92iaw2mbWyW6jBWtwOPrpstDD1S5j/W6nyDk=; b=V6J0VdCg0SdbYsVY/74XrPGltkO6yn2R4aA+fElsY1fyARaAdhk1CConUuSJdkptpE FVfbeHlHNpbrmgMqt5hFsMcG1yxX0sc4ciYrr9+Xi+iPv41VSC6F6/N/p7ReHZTZL8f5 aeHaG37L+U4TooLPT/uA7Ki9SZwAcqahn4ENPUUqk1me2LBY/BIkB9ByctcGyYG6irHv ateqlLJzhTnP7Lb7e0tivWJx/Sx1zSM0gANQogl05jzzpK9Z5A8ooIFPU7540Mvlv9L8 Zbt3JIQPBxy2S3VyJl3zdO6i3spdYfjp+1wKpHFi9/DzRuzvQ1qp5xWIAnfWTdoZ6htY FPMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=i6bsGcr92iaw2mbWyW6jBWtwOPrpstDD1S5j/W6nyDk=; b=JWNvvlazwmhNFmLNPXrzlqAS83wWDZLTROzFscpWS4vNSERt6pzEv8rTv1hO2yty8Q Td/LRLLQIfoZUARk610X+uvx/PA4ypLQFBHkOm/Sx4qbcipan0T4u1bleTIAMa3/WhzV 4HPgzY0VvPZEcL0Yc0jkQF7fGGHt2Ly/KP+iTS5KmOFNyey3sfg61UoN9b1QzFhAgn+2 s2uu6dILfcfR1I6OxddXMDwtqVbLeYkPaFcz1YcjyU8k+OEZpuyXJ2IufJiFnck3C2cX wB0nx52ddkJ9NpIirRrGvnnpL+m9/2mRjqw1ohdlmbcJjczyuyp4XT7/zHOVqlWX/1Ms JQzw== X-Gm-Message-State: AOAM533OC/X9MlupGYW/ZRW/1QatzrhZA+XYOnd1RKvJehr5q1IYKAmy 551ua/37iDKCWRl2f+o6zj8FeBkGpTA= X-Google-Smtp-Source: ABdhPJxPdCeAiw+33U+iC4+YRDkP9Mjez8u2qHIwI+nCnwONDl+o82lezByBWtDdiCFSahklXtiVtQ== X-Received: by 2002:adf:fbc5:: with SMTP id d5mr10198899wrs.82.1611161646365; Wed, 20 Jan 2021 08:54:06 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m2sm4729750wml.34.2021.01.20.08.54.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Jan 2021 08:54:05 -0800 (PST) Message-Id: <64358ec7ea2b3df4a8f1099452579a7568996921.1611161639.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 20 Jan 2021 16:53:56 +0000 Subject: [PATCH 6/9] sparse-checkout: load sparse-checkout patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee A future feature will want to load the sparse-checkout patterns into a pattern_list, but the current mechanism to do so is a bit complicated. This is made difficult due to needing to find the sparse-checkout file in different ways throughout the codebase. The logic implemented in the new get_sparse_checkout_patterns() was duplicated in populate_from_existing_patterns() in unpack-trees.c. Use the new method instead, keeping the logic around handling the struct unpack_trees_options. The callers to get_sparse_checkout_filename() in builtin/sparse-checkout.c manipulate the sparse-checkout file directly, so it is not appropriate to replace logic in that file with get_sparse_checkout_patterns(). Signed-off-by: Derrick Stolee --- builtin/sparse-checkout.c | 5 ----- dir.c | 17 +++++++++++++++++ dir.h | 2 ++ unpack-trees.c | 6 +----- 4 files changed, 20 insertions(+), 10 deletions(-) diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index e3140db2a0a..2306a9ad98e 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -22,11 +22,6 @@ static char const * const builtin_sparse_checkout_usage[] = { NULL }; -static char *get_sparse_checkout_filename(void) -{ - return git_pathdup("info/sparse-checkout"); -} - static void write_patterns_to_file(FILE *fp, struct pattern_list *pl) { int i; diff --git a/dir.c b/dir.c index d637461da5c..d153a63bbd1 100644 --- a/dir.c +++ b/dir.c @@ -2998,6 +2998,23 @@ void setup_standard_excludes(struct dir_struct *dir) } } +char *get_sparse_checkout_filename(void) +{ + return git_pathdup("info/sparse-checkout"); +} + +int get_sparse_checkout_patterns(struct pattern_list *pl) +{ + int res; + char *sparse_filename = get_sparse_checkout_filename(); + + pl->use_cone_patterns = core_sparse_checkout_cone; + res = add_patterns_from_file_to_list(sparse_filename, "", 0, pl, NULL); + + free(sparse_filename); + return res; +} + int remove_path(const char *name) { char *slash; diff --git a/dir.h b/dir.h index a3c40dec516..facfae47402 100644 --- a/dir.h +++ b/dir.h @@ -448,6 +448,8 @@ int is_empty_dir(const char *dir); void setup_standard_excludes(struct dir_struct *dir); +char *get_sparse_checkout_filename(void); +int get_sparse_checkout_patterns(struct pattern_list *pl); /* Constants for remove_dir_recursively: */ diff --git a/unpack-trees.c b/unpack-trees.c index af6e9b9c2fd..837b8bb42fb 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -1549,14 +1549,10 @@ static void mark_new_skip_worktree(struct pattern_list *pl, static void populate_from_existing_patterns(struct unpack_trees_options *o, struct pattern_list *pl) { - char *sparse = git_pathdup("info/sparse-checkout"); - - pl->use_cone_patterns = core_sparse_checkout_cone; - if (add_patterns_from_file_to_list(sparse, "", 0, pl, NULL) < 0) + if (get_sparse_checkout_patterns(pl) < 0) o->skip_sparse_checkout = 1; else o->pl = pl; - free(sparse); } From patchwork Wed Jan 20 16:53:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12032915 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28B38C433E6 for ; Wed, 20 Jan 2021 16:57:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EF329233E2 for ; Wed, 20 Jan 2021 16:57:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391387AbhATQ4t (ORCPT ); Wed, 20 Jan 2021 11:56:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56244 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404076AbhATQz1 (ORCPT ); Wed, 20 Jan 2021 11:55:27 -0500 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9319BC061786 for ; Wed, 20 Jan 2021 08:54:08 -0800 (PST) Received: by mail-wr1-x433.google.com with SMTP id 7so16513968wrz.0 for ; Wed, 20 Jan 2021 08:54:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=nNuAXab2vPlq3R+IaBIQjsjYiHa8LdjqA99dbgGHa1g=; b=Ljx3q2wyCnYRvha0sINwvfScRgherWCcDB+tSOmsnRigCWvEbE0iV9zd68PcxegLY4 2FG8TJT6HSnSv53eHCTB3h+CDYClmY6IUMP3nUNh0bwSVU5WvZ62LonlebRzpENjxhnc al0QdF+EmqkbuQBM6GgDQlsqvo5iTS5WTBDukL37hCoD4H05ASLUf5uaWUF2Ods31VE1 5Iz5gwEj2A5BhshyAgZnWuWlXuVRbrb5uFGxNNzri3Xevqt1SG3o8M910tML88RrtY6Z JzsPivyVJQ79kaLttBWTOMl0jNAihoIsgJlwsRsytInXgDdEWf6vjkuLpCfjTofAjae7 01Qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=nNuAXab2vPlq3R+IaBIQjsjYiHa8LdjqA99dbgGHa1g=; b=RYcM6LWUZgCVqGyzghEibymRCGZpXC+TiWYYtNNE/yxBz1aZW119v2NuoJCB85KDYs rpli3Dn9Kc4dpnwgnvz274/T2Xqu362KqHK3VZ3U2SRMaKgsBZox4esdtVoyka7I/7QY JIj9Flwbv/UWDVwe8MQvyMDPSxvXg6y5reRa01iXnSIYJkAC1Ay8310TTR//2qgAGPYt XgquBSMzbblSXbjmcsA3Mc/PARlzOEc8AEu7BB0ZzpyYCUtW6vaVPEByMXniY3LNpSrY nUY/tUOJzaLFmVNpuIF5IgwqKQZd2lcg1Abb9yeYULNWqygyYyIhwSoWEl/FI0F9YQbU nTqQ== X-Gm-Message-State: AOAM530wz3/0y/cUjP5La9W+T7kpBPvCmNdE8bzeyWqDYLVvEv9sc+hx lla2efVhFeWgV9fNLWAlBA2S+Tu93nE= X-Google-Smtp-Source: ABdhPJzlyFJqlnyqQImnsmylu5vPMOla+N7A4or3ox/X2gZ3IxupumUYyjWyueQuDTAAHA1tHEZYeg== X-Received: by 2002:adf:fbd2:: with SMTP id d18mr10414272wrs.222.1611161647232; Wed, 20 Jan 2021 08:54:07 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id a24sm4591339wmj.17.2021.01.20.08.54.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Jan 2021 08:54:06 -0800 (PST) Message-Id: <91344f5108c835a48b2460b9de75c489045b9cce.1611161639.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 20 Jan 2021 16:53:57 +0000 Subject: [PATCH 7/9] sparse-checkout: hold pattern list in index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee As we modify the sparse-checkout definition, we perform index operations on a pattern_list that only exists in-memory. This allows easy backing out in case the index update fails. However, if the index write itself cares about the sparse-checkout pattern set, we need access to that in-memory copy. Place a pointer to a 'struct pattern_list' in the index so we can access this on-demand. Signed-off-by: Derrick Stolee --- builtin/sparse-checkout.c | 17 ++++++++++------- cache.h | 2 ++ 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index 2306a9ad98e..e00b82af727 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -110,6 +110,8 @@ static int update_working_directory(struct pattern_list *pl) if (is_index_unborn(r->index)) return UPDATE_SPARSITY_SUCCESS; + r->index->sparse_checkout_patterns = pl; + memset(&o, 0, sizeof(o)); o.verbose_update = isatty(2); o.update = 1; @@ -138,6 +140,7 @@ static int update_working_directory(struct pattern_list *pl) else rollback_lock_file(&lock_file); + r->index->sparse_checkout_patterns = NULL; return result; } @@ -517,19 +520,18 @@ static int modify_pattern_list(int argc, const char **argv, enum modify_type m) { int result; int changed_config = 0; - struct pattern_list pl; - memset(&pl, 0, sizeof(pl)); + struct pattern_list *pl = xcalloc(1, sizeof(*pl)); switch (m) { case ADD: if (core_sparse_checkout_cone) - add_patterns_cone_mode(argc, argv, &pl); + add_patterns_cone_mode(argc, argv, pl); else - add_patterns_literal(argc, argv, &pl); + add_patterns_literal(argc, argv, pl); break; case REPLACE: - add_patterns_from_input(&pl, argc, argv); + add_patterns_from_input(pl, argc, argv); break; } @@ -539,12 +541,13 @@ static int modify_pattern_list(int argc, const char **argv, enum modify_type m) changed_config = 1; } - result = write_patterns_and_update(&pl); + result = write_patterns_and_update(pl); if (result && changed_config) set_config(MODE_NO_PATTERNS); - clear_pattern_list(&pl); + clear_pattern_list(pl); + free(pl); return result; } diff --git a/cache.h b/cache.h index f9c7a603841..bf4ec3de4b0 100644 --- a/cache.h +++ b/cache.h @@ -305,6 +305,7 @@ static inline unsigned int canon_mode(unsigned int mode) struct split_index; struct untracked_cache; struct progress; +struct pattern_list; struct index_state { struct cache_entry **cache; @@ -329,6 +330,7 @@ struct index_state { struct mem_pool *ce_mem_pool; struct progress *progress; struct repository *repo; + struct pattern_list *sparse_checkout_patterns; }; /* Name hashing */ From patchwork Wed Jan 20 16:53:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12032917 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 449B0C433E9 for ; Wed, 20 Jan 2021 16:57:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0FD5B233F6 for ; Wed, 20 Jan 2021 16:57:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391592AbhATQ5B (ORCPT ); Wed, 20 Jan 2021 11:57:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56246 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404077AbhATQz1 (ORCPT ); Wed, 20 Jan 2021 11:55:27 -0500 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F57CC061793 for ; Wed, 20 Jan 2021 08:54:09 -0800 (PST) Received: by mail-wr1-x432.google.com with SMTP id a1so2462180wrq.6 for ; Wed, 20 Jan 2021 08:54:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=g5rYqLWoWL2WCFNwvHUDqPp3kEwXKkEJegs8hgz8btI=; b=SEWO7wK84/SBeLS0loo6nDxNPfVqJko54rPtLBkRNCRL+DLA7Y9SrD8CCZU+MfmZl7 ibu/ONw5wgRuz1Mc9wsIKTTZV/EYR4gfYsCSxh2pXtfmLl4yaJOr1l/B8DGy7vYxF+Ll GN+b6fvZHJ/VBmX737DRbMaq5qsS6EWtc1tto3X1aw/lSmdkl+Y7RAuRAWH3sJ9pymQK N36ODdwzOvXTGdWYVcqw4KgS+fN+b8ypp4dSko5duL8iXVDzYd83bdvtbxB9FgcRL2+M p+8Yn3ZOUHgFv6R07DUhnbI72cACp6JZWXZ3RcEYIMRmf+p9tPQHENL8WHXKa0Qnlo1M vTQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=g5rYqLWoWL2WCFNwvHUDqPp3kEwXKkEJegs8hgz8btI=; b=jLDkgAS5PwXJHejGDBhn/zAvvKlWr/HSSvcPq9vEwKBi3QWFtwXCTU6+KqY4KQtBY+ X8o8CcU/NxiGJJxomYIyZlJfSRALmIH+inbgx5QJfeS/tqm+xcYcRm02Dt/JsANDY9h3 7NQfEPI4/YQTv7XBUc+dRnEDr0/YuG9MYJ3Bcfbdj0ucl81X4rKqQbKEVc6MEMduR8L/ nWFD2n/7rWaI584CLzfHVYmDgn84iEbV9GTQQ30guoArmBsO8lWi/XxI9JMCgR8SCJoG FP7glPveWPINcnsrbKybCjxcxGQ9sRAVkJYiRRQ9l+Dmn02sosJeg7tG6XgrVKZzjI4+ hQWA== X-Gm-Message-State: AOAM532ZyqPEYs3BL1s5R8rcivmxi1QfBI3cT4nNLLkgU3MGYtTt9RYJ VQ8KJRL7A9nFHfki1TAUATF0M0R6hFc= X-Google-Smtp-Source: ABdhPJy8+4R1jFIMFeTXKcq38FVYbcAK+bjEhE0NFFYGRRagsGov6IlQ4zpctb3oKVI1qZgr22xvtg== X-Received: by 2002:a5d:6cc9:: with SMTP id c9mr9995416wrc.411.1611161647991; Wed, 20 Jan 2021 08:54:07 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h9sm5229274wre.24.2021.01.20.08.54.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Jan 2021 08:54:07 -0800 (PST) Message-Id: <8326a9b5320e1e774caef568fcce2bfd2ec13cb1.1611161639.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 20 Jan 2021 16:53:58 +0000 Subject: [PATCH 8/9] test-lib: test_region looks for trace2 regions Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Most test cases can verify Git's behavior using input/output expectations or changes to the .git directory. However, sometimes we want to check that Git did or did not run a certain section of code. This is particularly important for performance-only features that we want to ensure have been enabled in certain cases. Add a new 'test_region' function that checks if a trace2 region was entered and left in a given trace2 event log. There is one existing test (t0500-progress-display.sh) that performs this check already, so use the helper function instead. More uses will be added in a later change. t6423-merge-rename-directories.sh also greps for region_enter lines, but it verifies the number of such lines, which is not the same as an existence check. Signed-off-by: Derrick Stolee --- t/t0500-progress-display.sh | 3 +-- t/test-lib-functions.sh | 40 +++++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 2 deletions(-) diff --git a/t/t0500-progress-display.sh b/t/t0500-progress-display.sh index 1ed1df351cb..c461b89dfaf 100755 --- a/t/t0500-progress-display.sh +++ b/t/t0500-progress-display.sh @@ -303,8 +303,7 @@ test_expect_success 'progress generates traces' ' "Working hard" stderr && # t0212/parse_events.perl intentionally omits regions and data. - grep -e "region_enter" -e "\"category\":\"progress\"" trace.event && - grep -e "region_leave" -e "\"category\":\"progress\"" trace.event && + test_region category progress trace.event && grep "\"key\":\"total_objects\",\"value\":\"40\"" trace.event && grep "\"key\":\"total_bytes\",\"value\":\"409600\"" trace.event ' diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh index 999982fe4a9..c878db93013 100644 --- a/t/test-lib-functions.sh +++ b/t/test-lib-functions.sh @@ -1655,3 +1655,43 @@ test_subcommand () { grep "\[$expr\]" fi } + +# Check that the given command was invoked as part of the +# trace2-format trace on stdin. +# +# test_region [!]