From patchwork Sat Jan 23 19:58:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12041873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32AE6C433DB for ; Sat, 23 Jan 2021 19:59:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 047E822EBE for ; Sat, 23 Jan 2021 19:59:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726284AbhAWT7j (ORCPT ); Sat, 23 Jan 2021 14:59:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725922AbhAWT7E (ORCPT ); Sat, 23 Jan 2021 14:59:04 -0500 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C0659C0613D6 for ; Sat, 23 Jan 2021 11:58:23 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id 6so8375959wri.3 for ; Sat, 23 Jan 2021 11:58:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=VNwHDx/owv/iw0AULzqRDsjmoZuBpRiDxJMwBI85Ozc=; b=rn1JIF5mlYAb1p7LqO8EmQ2WuNegHFc/WqTci/6x5VK3VS1JbU4hXNYL4XJCm8UmOQ qsPN5BnAT8QYYHpho7cPSiz7g7U5GUoYdXfEK0u/BnxQFWRjjDwsahIqpAFYVsb5cd+p STkvCYFpmCAnF/Wsfqa0yBMsqiNEaf6+Gu/9tSSL2iu3t3TbMKhnfL76HFsLZvfMR9YL JJxHmJTZa+4b8R9Fqcd8cLscPB35tLRxmgmcY3IxHsepfHcFSUQP9g9EuzMGcWhlnjQp UZBvvWD3bOCrR/AiUoRbP1xWKDkw+9kNeWr2GTGFAS0cFEGU2krR8ioi5X8CK5x8rIwA CKBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=VNwHDx/owv/iw0AULzqRDsjmoZuBpRiDxJMwBI85Ozc=; b=bs9HI2X0nFjN5QD6aGGaTw7Tr5yn0Xrgx8lDmte5hkLmmzRNZ2qo40GfL1AxX0wnUu wAnwr/v0hBOw4pXI21NdHZkGaeVw9ZCQ2Jx0YNMFG4LNOpemK0uA8niDreDBVp8l+24n fP0suGv9iNg2gimOrdKhtc3gr8xb9Kb/iVnv2lmpqUPUaMR8hnt87SpKsT4h8yPqso8V Afr5OU8SQSrqGb+NrZOyfzJ/xnvIAVRL0HmqCGR1Kk7N8j0mCg5QnpIJIXU5qoS4B9z3 lxmExak2lbvZVaGrAXGwNbd9n2NqVviu6oyy5h9kRllypy8ZYaG71ZmfQPtL1mhQW0MK uOeg== X-Gm-Message-State: AOAM531zpnOrnijkuAt7B8qcOYRF3jN9KZq9Za1yjYxE4dJJHhURA+Sx 3kKUglMPUAJ2+wZ2KrbouCQ+6S7nNNc= X-Google-Smtp-Source: ABdhPJxKFO4YxLP6wfZjgh4pUBvX2grXcLfGlAnuxk8hlEYNDgf8HAQIHk+V+SZgIyeYaGsECo2eKQ== X-Received: by 2002:a5d:4987:: with SMTP id r7mr10451169wrq.352.1611431902113; Sat, 23 Jan 2021 11:58:22 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id b4sm4663879wrn.12.2021.01.23.11.58.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Jan 2021 11:58:21 -0800 (PST) Message-Id: In-Reply-To: References: Date: Sat, 23 Jan 2021 19:58:11 +0000 Subject: [PATCH v3 1/9] cache-tree: clean up cache_tree_update() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , gitster@pobox.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Make the method safer by allocating a cache_tree member for the given index_state if it is not already present. This is preferrable to a BUG() statement or returning with an error because future callers will want to populate an empty cache-tree using this method. Callers can also remove their conditional allocations of cache_tree. Also drop local variables that can be found directly from the 'istate' parameter. Signed-off-by: Derrick Stolee --- builtin/checkout.c | 3 --- cache-tree.c | 17 +++++++++-------- sequencer.c | 3 --- unpack-trees.c | 2 -- 4 files changed, 9 insertions(+), 16 deletions(-) diff --git a/builtin/checkout.c b/builtin/checkout.c index c9ba23c2794..2d6550bc3c8 100644 --- a/builtin/checkout.c +++ b/builtin/checkout.c @@ -821,9 +821,6 @@ static int merge_working_tree(const struct checkout_opts *opts, } } - if (!active_cache_tree) - active_cache_tree = cache_tree(); - if (!cache_tree_fully_valid(active_cache_tree)) cache_tree_update(&the_index, WRITE_TREE_SILENT | WRITE_TREE_REPAIR); diff --git a/cache-tree.c b/cache-tree.c index 3f1a8d4f1b7..60b6aefbf51 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -436,16 +436,20 @@ static int update_one(struct cache_tree *it, int cache_tree_update(struct index_state *istate, int flags) { - struct cache_tree *it = istate->cache_tree; - struct cache_entry **cache = istate->cache; - int entries = istate->cache_nr; - int skip, i = verify_cache(cache, entries, flags); + int skip, i; + + i = verify_cache(istate->cache, istate->cache_nr, flags); if (i) return i; + + if (!istate->cache_tree) + istate->cache_tree = cache_tree(); + trace_performance_enter(); trace2_region_enter("cache_tree", "update", the_repository); - i = update_one(it, cache, entries, "", 0, &skip, flags); + i = update_one(istate->cache_tree, istate->cache, istate->cache_nr, + "", 0, &skip, flags); trace2_region_leave("cache_tree", "update", the_repository); trace_performance_leave("cache_tree_update"); if (i < 0) @@ -635,9 +639,6 @@ static int write_index_as_tree_internal(struct object_id *oid, cache_tree_valid = 0; } - if (!index_state->cache_tree) - index_state->cache_tree = cache_tree(); - if (!cache_tree_valid && cache_tree_update(index_state, flags) < 0) return WRITE_TREE_UNMERGED_INDEX; diff --git a/sequencer.c b/sequencer.c index 8909a467700..aa3e4c81cf0 100644 --- a/sequencer.c +++ b/sequencer.c @@ -679,9 +679,6 @@ static int do_recursive_merge(struct repository *r, static struct object_id *get_cache_tree_oid(struct index_state *istate) { - if (!istate->cache_tree) - istate->cache_tree = cache_tree(); - if (!cache_tree_fully_valid(istate->cache_tree)) if (cache_tree_update(istate, 0)) { error(_("unable to update cache tree")); diff --git a/unpack-trees.c b/unpack-trees.c index af6e9b9c2fd..a810b79657e 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -1726,8 +1726,6 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options if (!ret) { if (git_env_bool("GIT_TEST_CHECK_CACHE_TREE", 0)) cache_tree_verify(the_repository, &o->result); - if (!o->result.cache_tree) - o->result.cache_tree = cache_tree(); if (!cache_tree_fully_valid(o->result.cache_tree)) cache_tree_update(&o->result, WRITE_TREE_SILENT | From patchwork Sat Jan 23 19:58:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12041869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA697C433DB for ; Sat, 23 Jan 2021 19:59:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B8A872311B for ; Sat, 23 Jan 2021 19:59:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726249AbhAWT7S (ORCPT ); Sat, 23 Jan 2021 14:59:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726021AbhAWT7E (ORCPT ); Sat, 23 Jan 2021 14:59:04 -0500 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58177C061786 for ; Sat, 23 Jan 2021 11:58:24 -0800 (PST) Received: by mail-wr1-x434.google.com with SMTP id d16so7697302wro.11 for ; Sat, 23 Jan 2021 11:58:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=+A0SHikiMuu/wPZEMUwCv2+0OZd1VZkq05UUFRdHlUA=; b=c0ztjakr8he5ZPexu4be7JR8drIz1LvQ9Z1O33bgFw1NOiRAVb6I/fP33wl8CVPtbU 4QdHUM6A/Sk55xXo5rNs0Tl/ggUjffEIcQbngT5EYZz8az8sRm6UzDUg5XRUH+EorUS3 RBGQK6kB9bJccmrPNriGrI+puwOvACLOSz0wbeT2HrWYUeXPLAN4AUYBm2gZqCx6i5dK V/m2kVIttOw+EHFy8JKFnug0o7/F/FVRP9e9lfUAjk2qrIPjRcgHu7lUlw8RTsBZaUTu RElCLwg11uhG9ddHZEBNMDgBKItrrSaS78+y4KxvPinaHlNDFj5djcX/yRCMoRf9T7uB orlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=+A0SHikiMuu/wPZEMUwCv2+0OZd1VZkq05UUFRdHlUA=; b=Msc+PwTqBvBrym11e/WU/jPPGUMvTRjQ7qxSIxO8nWP9hHYLIGe66xUVmj63FZ2jhE 6Mh1/qQ4xXj09DpTCtXN9eDzi8Z8M9pDxpdSHvpijsallj1tleUAGQwlYK042VtNpphV sjc73WCj11xm60cxXXLU5I3OHUbyEI1Hs9De/koYdIhyqvkk3ztCJVuEsziL7y+zX8iL adAtIirabBqurkxRbsfIJFadbA6oN+OEV0AuUHLGbuPTbgFpEzpGpNnCUoaQZF7NdbOP 9xSz8Aiv5C0sWck//Bb/Qpj7+RGY6irEfwlPGJCpQ28xS0cr5ax3vJMN+r33EJmXYuwI cxfw== X-Gm-Message-State: AOAM533HpZOA0MIVWU9wLBMkqlV9C0DjqwYS3Zl/pG2q/bppEk8AOjff b44WouJIBAfMXKCAxCV15nU/lCobwKo= X-Google-Smtp-Source: ABdhPJzmvbCTw0C8Hxy7y7+G4DXaDVSVW1d3FDSKFSfFeLjS/jXX0a4X+L+k+zn4IZkvdaIIliRGmQ== X-Received: by 2002:adf:81e4:: with SMTP id 91mr10484393wra.161.1611431902985; Sat, 23 Jan 2021 11:58:22 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id b4sm4663933wrn.12.2021.01.23.11.58.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Jan 2021 11:58:22 -0800 (PST) Message-Id: <1b8b56800948339c0e0387555698bdfdc80a19ad.1611431900.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sat, 23 Jan 2021 19:58:12 +0000 Subject: [PATCH v3 2/9] cache-tree: simplify verify_cache() prototype Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , gitster@pobox.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The verify_cache() method takes an array of cache entries and a count, but these are always provided directly from a struct index_state. Use a pointer to the full structure instead. There is a subtle point when istate->cache_nr is zero that subtracting one will underflow. This triggers a failure in t0000-basic.sh, among others. Use "i + 1 < istate->cache_nr" to avoid these strange comparisons. Convert i to be unsigned as well, which also removes the potential signed overflow in the unlikely case that cache_nr is over 2.1 billion entries. The 'funny' variable has a maximum value of 11, so making it unsigned does not change anything of importance. Signed-off-by: Derrick Stolee --- cache-tree.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/cache-tree.c b/cache-tree.c index 60b6aefbf51..acac6d58c37 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -151,16 +151,15 @@ void cache_tree_invalidate_path(struct index_state *istate, const char *path) istate->cache_changed |= CACHE_TREE_CHANGED; } -static int verify_cache(struct cache_entry **cache, - int entries, int flags) +static int verify_cache(struct index_state *istate, int flags) { - int i, funny; + unsigned i, funny; int silent = flags & WRITE_TREE_SILENT; /* Verify that the tree is merged */ funny = 0; - for (i = 0; i < entries; i++) { - const struct cache_entry *ce = cache[i]; + for (i = 0; i < istate->cache_nr; i++) { + const struct cache_entry *ce = istate->cache[i]; if (ce_stage(ce)) { if (silent) return -1; @@ -180,13 +179,13 @@ static int verify_cache(struct cache_entry **cache, * stage 0 entries. */ funny = 0; - for (i = 0; i < entries - 1; i++) { + for (i = 0; i + 1 < istate->cache_nr; i++) { /* path/file always comes after path because of the way * the cache is sorted. Also path can appear only once, * which means conflicting one would immediately follow. */ - const struct cache_entry *this_ce = cache[i]; - const struct cache_entry *next_ce = cache[i + 1]; + const struct cache_entry *this_ce = istate->cache[i]; + const struct cache_entry *next_ce = istate->cache[i + 1]; const char *this_name = this_ce->name; const char *next_name = next_ce->name; int this_len = ce_namelen(this_ce); @@ -438,7 +437,7 @@ int cache_tree_update(struct index_state *istate, int flags) { int skip, i; - i = verify_cache(istate->cache, istate->cache_nr, flags); + i = verify_cache(istate, flags); if (i) return i; From patchwork Sat Jan 23 19:58:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12041867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFA3BC433DB for ; Sat, 23 Jan 2021 19:59:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 79A5B22D2B for ; Sat, 23 Jan 2021 19:59:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726223AbhAWT7R (ORCPT ); Sat, 23 Jan 2021 14:59:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726197AbhAWT7F (ORCPT ); Sat, 23 Jan 2021 14:59:05 -0500 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 31646C061788 for ; Sat, 23 Jan 2021 11:58:25 -0800 (PST) Received: by mail-wm1-x32e.google.com with SMTP id y187so7312105wmd.3 for ; Sat, 23 Jan 2021 11:58:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=bh8yh3+iaQ4iXXap8pGPFY/q1QZLWVT+vydoYX5XmtQ=; b=rIYM8Sj1xmEI9gizswqiR/akQNACjvzbfprhTNdSAnCKn04N90sUhLJfSsxzKbutra Aq8s2L3o3K3GGjy7oW/IAxZnR8XxTE98+PU6Ie/exPsqR7i9J4iyLq65UO7q4/StRwjq 3b7mplEl221IIdInScHZNgKQK6j+j0/sHJQrQFgMotZ+StmgvJHqFfIZDBpVTIWo3Bm/ rydTfBRix3dKyR6xtQ0HXtjA4tNeiuU3Y3OH4G7auPIpqP9eBoDaqLZqlo+RuboyclL1 34Dyb7SmWrPbT0VqTbJ8OrW57Dughq0696pxwnEVq/Dn09zGZiUaA3/mYdbUEcyIws5B FCLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=bh8yh3+iaQ4iXXap8pGPFY/q1QZLWVT+vydoYX5XmtQ=; b=MkkfAqZIzzVD6SpPSZnWRQt0Gd42WCwnx79tfX53jWrswJjuUBVCc8skXp3CCuELPx 1UVJQA6ieYd9ygF/CBw3b13OovHmPzRimzGQDoo79Y8yiSM6fFZMOXcZVjRXjl58QuZj kpkVBFjA6qgES17h1t6ZP5p/tUxsIv1/wOIW/rL7K8a9DRU3fTYlKCRv2OqhuibHStBs hX5tO1B666G/QZtrZDjE7qfyOaqOW9lw3sULW2ypE0q456j5U0Ny3VafWd+8qgnk32xJ tIC8W9Vj/JPUyHZLLCU8UrFYOVslqBQ3bI+2N6Zw/5k2nubbQN0/P+5pqDgl8zRpiv1X HZ0Q== X-Gm-Message-State: AOAM531akU9+VlGyaUJUVn80mNC1k7ZVAIv5VfgWEVrCH63qHK5LqwTy 9PqkiZK8syYbZ/lkRGGiVX14msLGw/o= X-Google-Smtp-Source: ABdhPJzwd4NPxNGJZ9zqYVteJ9XVjYb3seUEJumdfQIGXICxliN8hXh3SZz3bOBI98K28v/HsEwROQ== X-Received: by 2002:a1c:b7d6:: with SMTP id h205mr9392825wmf.182.1611431903791; Sat, 23 Jan 2021 11:58:23 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x1sm5691023wmi.14.2021.01.23.11.58.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Jan 2021 11:58:23 -0800 (PST) Message-Id: <314b6b34f759c3a0e0d361bc08b254960a4d28e4.1611431900.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sat, 23 Jan 2021 19:58:13 +0000 Subject: [PATCH v3 3/9] cache-tree: extract subtree_pos() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , gitster@pobox.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee This method will be helpful to use outside of cache-tree.c in a later feature. The implementation is subtle due to subtree_name_cmp() sorting by length and then lexicographically. Signed-off-by: Derrick Stolee --- cache-tree.c | 6 +++--- cache-tree.h | 2 ++ 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/cache-tree.c b/cache-tree.c index acac6d58c37..2fb483d3c08 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -45,7 +45,7 @@ static int subtree_name_cmp(const char *one, int onelen, return memcmp(one, two, onelen); } -static int subtree_pos(struct cache_tree *it, const char *path, int pathlen) +int cache_tree_subtree_pos(struct cache_tree *it, const char *path, int pathlen) { struct cache_tree_sub **down = it->down; int lo, hi; @@ -72,7 +72,7 @@ static struct cache_tree_sub *find_subtree(struct cache_tree *it, int create) { struct cache_tree_sub *down; - int pos = subtree_pos(it, path, pathlen); + int pos = cache_tree_subtree_pos(it, path, pathlen); if (0 <= pos) return it->down[pos]; if (!create) @@ -123,7 +123,7 @@ static int do_invalidate_path(struct cache_tree *it, const char *path) it->entry_count = -1; if (!*slash) { int pos; - pos = subtree_pos(it, path, namelen); + pos = cache_tree_subtree_pos(it, path, namelen); if (0 <= pos) { cache_tree_free(&it->down[pos]->cache_tree); free(it->down[pos]); diff --git a/cache-tree.h b/cache-tree.h index 639bfa5340e..8efeccebfc9 100644 --- a/cache-tree.h +++ b/cache-tree.h @@ -27,6 +27,8 @@ void cache_tree_free(struct cache_tree **); void cache_tree_invalidate_path(struct index_state *, const char *); struct cache_tree_sub *cache_tree_sub(struct cache_tree *, const char *); +int cache_tree_subtree_pos(struct cache_tree *it, const char *path, int pathlen); + void cache_tree_write(struct strbuf *, struct cache_tree *root); struct cache_tree *cache_tree_read(const char *buffer, unsigned long size); From patchwork Sat Jan 23 19:58:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12041871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CF70C433E0 for ; Sat, 23 Jan 2021 19:59:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5925F23139 for ; Sat, 23 Jan 2021 19:59:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726264AbhAWT7g (ORCPT ); Sat, 23 Jan 2021 14:59:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726215AbhAWT7G (ORCPT ); Sat, 23 Jan 2021 14:59:06 -0500 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D50BC06178B for ; Sat, 23 Jan 2021 11:58:26 -0800 (PST) Received: by mail-wr1-x435.google.com with SMTP id 7so8376951wrz.0 for ; Sat, 23 Jan 2021 11:58:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ohpT1dyG20gefcxLzQYcOZmK85UKoGo+762jzBHebhM=; b=GSs1jB2zzAo3ixnyPhrl3fFKucunFeitlmxxns08wCF/IRms9yJOnXjyPBU+hCkzp3 0KfwauBlPTpE5tKiOBS6sN2jjrIGhel6D9eNJ9Oqjq5oSMON23mNayix2t3p6Ujlrimu 7x86X+ESfM91G7tT0efRUaMUncRN+VTsxF868yPP/JrTBhAa7KvL+ZejkyKZ4f2ZsTZD Xymz1T9PZZ/HGymcUcXZcPgpW8RTM7gdU1mLdudOkb4kfTy8e+mCFb9Jw9n9c8EDLckb 8OWKCCrvXpP2qAv34MzAw9NmDehui2pPcKrBBjv0Vz9YsaYo8HdrICbVch82vaaTA4MJ jJOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ohpT1dyG20gefcxLzQYcOZmK85UKoGo+762jzBHebhM=; b=Oq9TpZlD92N79rh97oH3BB3VOxbiJqpTz0KmB6AWXgQRL7lUGRkUjqmeYXAWF968r4 SKe5MYPOSyBi15F7A+lOOekAGuGdM4Z0e+6uOMHZKtpfp+R9HLxBFkDrFMVcQpoYMGCJ OrLu6GsSFljBPUTshACUOEWLI1cGCsaYv3TRO/l7JmJ9SL8hmDZIKLB33BgcPkFNBIBQ lY3l704x+cHzMLhxDBWBlioaquxclpXbr6hvAjFj9sj7AyrOLE+Czw4uTD4g+h/U+hoJ cnTEhIl7OvuXYld1T+WQhY31iwqiGoPakg2D0TyCYrEwCjp2AxfK+HowpEk3GSCs6ABl e7tQ== X-Gm-Message-State: AOAM533jWLl4lVd9hMF1ofT8/ij27+81zBzDmyJWoWBBt2DMoDR3YAD9 rV4INJYxVCmziIlK1T7kqijLqkJKsFY= X-Google-Smtp-Source: ABdhPJz+RG1++iKUaBQJfF7Lu62Zrvz5bRYX5gf2hGdJgwUJAOk4HydkMOPOd4+vGlHvivTh2Lh7GA== X-Received: by 2002:adf:bc4b:: with SMTP id a11mr189014wrh.260.1611431904702; Sat, 23 Jan 2021 11:58:24 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p9sm16617552wrj.11.2021.01.23.11.58.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Jan 2021 11:58:24 -0800 (PST) Message-Id: <4e688d25f8c7949dd3edbdb47b924450d9a50796.1611431900.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sat, 23 Jan 2021 19:58:14 +0000 Subject: [PATCH v3 4/9] fsmonitor: de-duplicate BUG()s around dirty bits Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , gitster@pobox.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The index has an fsmonitor_dirty bitmap that records which index entries are "dirty" based on the response from the FSMonitor. If this bitmap ever grows larger than the index, then there was an error in how it was constructed, and it was probably a developer's bug. There are several BUG() statements that are very similar, so replace these uses with a simpler assert_index_minimum(). Since there is one caller that uses a custom 'pos' value instead of the bit_size member, we cannot simplify it too much. However, the error string is identical in each, so this simplifies things. Be sure to add one when checking if a position if valid, since the minimum is a bound on the expected size. The end result is that the code is simpler to read while also preserving these assertions for developers in the FSMonitor space. Signed-off-by: Derrick Stolee --- fsmonitor.c | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/fsmonitor.c b/fsmonitor.c index ca031c3abb8..fe9e9d7baf4 100644 --- a/fsmonitor.c +++ b/fsmonitor.c @@ -13,14 +13,19 @@ struct trace_key trace_fsmonitor = TRACE_KEY_INIT(FSMONITOR); +static void assert_index_minimum(struct index_state *istate, size_t pos) +{ + if (pos > istate->cache_nr) + BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" > %u)", + (uintmax_t)pos, istate->cache_nr); +} + static void fsmonitor_ewah_callback(size_t pos, void *is) { struct index_state *istate = (struct index_state *)is; struct cache_entry *ce; - if (pos >= istate->cache_nr) - BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" >= %u)", - (uintmax_t)pos, istate->cache_nr); + assert_index_minimum(istate, pos + 1); ce = istate->cache[pos]; ce->ce_flags &= ~CE_FSMONITOR_VALID; @@ -82,10 +87,8 @@ int read_fsmonitor_extension(struct index_state *istate, const void *data, } istate->fsmonitor_dirty = fsmonitor_dirty; - if (!istate->split_index && - istate->fsmonitor_dirty->bit_size > istate->cache_nr) - BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" > %u)", - (uintmax_t)istate->fsmonitor_dirty->bit_size, istate->cache_nr); + if (!istate->split_index) + assert_index_minimum(istate, istate->fsmonitor_dirty->bit_size); trace_printf_key(&trace_fsmonitor, "read fsmonitor extension successful"); return 0; @@ -110,10 +113,8 @@ void write_fsmonitor_extension(struct strbuf *sb, struct index_state *istate) uint32_t ewah_size = 0; int fixup = 0; - if (!istate->split_index && - istate->fsmonitor_dirty->bit_size > istate->cache_nr) - BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" > %u)", - (uintmax_t)istate->fsmonitor_dirty->bit_size, istate->cache_nr); + if (!istate->split_index) + assert_index_minimum(istate, istate->fsmonitor_dirty->bit_size); put_be32(&hdr_version, INDEX_EXTENSION_VERSION2); strbuf_add(sb, &hdr_version, sizeof(uint32_t)); @@ -335,9 +336,7 @@ void tweak_fsmonitor(struct index_state *istate) } /* Mark all previously saved entries as dirty */ - if (istate->fsmonitor_dirty->bit_size > istate->cache_nr) - BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" > %u)", - (uintmax_t)istate->fsmonitor_dirty->bit_size, istate->cache_nr); + assert_index_minimum(istate, istate->fsmonitor_dirty->bit_size); ewah_each_bit(istate->fsmonitor_dirty, fsmonitor_ewah_callback, istate); refresh_fsmonitor(istate); From patchwork Sat Jan 23 19:58:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12041875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC89EC433DB for ; Sat, 23 Jan 2021 20:00:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 819A122D50 for ; Sat, 23 Jan 2021 20:00:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726316AbhAWT74 (ORCPT ); Sat, 23 Jan 2021 14:59:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36662 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725922AbhAWT7o (ORCPT ); Sat, 23 Jan 2021 14:59:44 -0500 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C8496C061793 for ; Sat, 23 Jan 2021 11:58:26 -0800 (PST) Received: by mail-wm1-x333.google.com with SMTP id c128so7305208wme.2 for ; Sat, 23 Jan 2021 11:58:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=TuqSkycyIw6sa7UCYR8334RJzyjQdAuVGemdSXJzD5o=; b=kUI1i3UraOnKAPPdYY2vXknL584TyleXwbpE6OT/o0smuYuo0oR6Bl3Djk9tA8CZSf hqfhfEhIjLxEzwmww2wIKQyV2r7X1TFTK4Z6qcO80BHtf8CEqDgc91CBj1unx24X08R7 EzD03LJjBHHcx713SHpPj5boAwLWHaD0LjFge+CBqXmbZxmzsMQz1XUV3qsNTJ5TYSMA Tr8ENzd0fg3UEIMEfddrcZeBBsf6fM7UI/TKvLkaNEGihy4DjA+Vj9iqwkm/qeXR4bE7 e0hF+q1mpgXaBga8YKo6BpxgK9mhB7GydzNFicHbbAETSSXkhtxcK1R1ONnXld9ld2Iq GxCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=TuqSkycyIw6sa7UCYR8334RJzyjQdAuVGemdSXJzD5o=; b=tS7T7q8jEPEaYzoE7MgwGNAIo+yzu0Ro3DGMWX3CR2D2p2/9ngbGl2QV10OmtNV/PX MSoc3DXkvEttbFCwtAOji6HbzoR0I/bY76hREfjBN1dlNEi8kLWS130sM9j+TBFhkElT YShzHbukZ1fjyrKZzBX9oTA1dQv6mE8eTSTo18N1zNU5Y8pSU3zVS9WMhFrAACUpYB+E 104RDeJW6awBwlDSPNBEL7A8+QqQXbeAepGL22DbUbDcERfZvkJEuh/RrFgIxC2oL+Rh X5VM80ThFF8g40mw2AD/bCtbljLdU/+CBNjvHvysGRE6wVmbGO3UwNRot+2LfdW5VwvR CcGA== X-Gm-Message-State: AOAM530XiCSDfF6qGhOmCgyKDy+x4DiO7u4oAbFU4FSmoErqebiVYws+ /rdKMav0MX24z5d1k7kAxXpABqw7k10= X-Google-Smtp-Source: ABdhPJyqwJmWHt3KbuD47+NIVGvh53lK3PnMbT8kNQjEYEyXM0cn4EZiPeQnYp8cgMyDMFcIDrbpUw== X-Received: by 2002:a7b:c0d8:: with SMTP id s24mr7110045wmh.4.1611431905452; Sat, 23 Jan 2021 11:58:25 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h187sm16385239wmf.30.2021.01.23.11.58.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Jan 2021 11:58:25 -0800 (PST) Message-Id: <6373997e05c04f57e441355eac94ca03c634e3fb.1611431900.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sat, 23 Jan 2021 19:58:15 +0000 Subject: [PATCH v3 5/9] repository: add repo reference to index_state Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , gitster@pobox.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee It will be helpful to add behavior to index operations that might trigger an object lookup. Since each index belongs to a specific repository, add a 'repo' pointer to struct index_state that allows access to this repository. Add a BUG() statement if the repo already has an index, and the index already has a repo, but somehow the index points to a different repo. This will prevent future changes from needing to pass an additional 'struct repository *repo' parameter and instead rely only on the 'struct index_state *istate' parameter. Signed-off-by: Derrick Stolee --- cache.h | 1 + repository.c | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/cache.h b/cache.h index 71097657489..f9c7a603841 100644 --- a/cache.h +++ b/cache.h @@ -328,6 +328,7 @@ struct index_state { struct ewah_bitmap *fsmonitor_dirty; struct mem_pool *ce_mem_pool; struct progress *progress; + struct repository *repo; }; /* Name hashing */ diff --git a/repository.c b/repository.c index a4174ddb062..c98298acd01 100644 --- a/repository.c +++ b/repository.c @@ -264,6 +264,12 @@ int repo_read_index(struct repository *repo) if (!repo->index) repo->index = xcalloc(1, sizeof(*repo->index)); + /* Complete the double-reference */ + if (!repo->index->repo) + repo->index->repo = repo; + else if (repo->index->repo != repo) + BUG("repo's index should point back at itself"); + return read_index_from(repo->index, repo->index_file, repo->gitdir); } From patchwork Sat Jan 23 19:58:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12041877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75E5FC433E0 for ; Sat, 23 Jan 2021 20:00:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3098222DBF for ; Sat, 23 Jan 2021 20:00:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726324AbhAWUAB (ORCPT ); Sat, 23 Jan 2021 15:00:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36664 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726288AbhAWT7o (ORCPT ); Sat, 23 Jan 2021 14:59:44 -0500 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A9D27C061794 for ; Sat, 23 Jan 2021 11:58:27 -0800 (PST) Received: by mail-wr1-x430.google.com with SMTP id p15so1723367wrq.8 for ; Sat, 23 Jan 2021 11:58:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=3LC/1eestJjR+EOcwKHsjfdOZknum6kvDyZ7fg9I1AA=; b=gHakX2qOFeghf0gXhOSzQvUSZCVKsOnoX0xwdstX5fif3FDAcKrtkTCP/a9LKw7RuK YncpV3gI267Fuhjnod4+OxXRk05Hky9l+8/zYzbw7KtwV6Ry10/ITvBbZ1FkKtzsX5AL r6R9cqZxk77nY4zZwzmxbTunGBnpgVMzTptHUz59mfZXVjQyVmkqop2q1ELeojdRD4Cx AFd+LeCKI/VnwJvZTKg+/wmQVLRbMAsQ6lnGkF0/OithL+uJMyuVRfEBZXzy7RA1nI6v p5XDOxbaZakkZlW+Ba8IFI56A6asaYmCbkFj4pPqmNlyWweoddCN+Pw6OB8/1NNQ7LMr LtRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=3LC/1eestJjR+EOcwKHsjfdOZknum6kvDyZ7fg9I1AA=; b=O0lHR19913jtF/N9qy/5Ll5XoExNKLqLUFa6wZdSz7v/814eLxfb1lmU9/v+NxWwCD /u059yMSBX+u00J83VZPBcyMpFsJFDcoFXyypT5WNeNPaVPSXMLNppmsjx67xh/9igYq 3rQXh1e3oQZLHpAbkNy/iySuW4JzzpvajSXVQrUxw6IGRoIziAzuf0M4rHcQVzeIZ5gI rA3yBG9JtpE9KkCBobSD0m1C3yXKV+qaR+Zr3Ij5+m+W+iumtpZk1lDSqW3Xiwz+j7dM wh4uNR60QW5kTPjoYmgXbKUVRLvhyTz+w/HMsYayveSSbq0Xrk7lYqLEFSPxjGL9+f1w JPmQ== X-Gm-Message-State: AOAM5300gy9euRg29KHymciQ7YrlsIJ/lD56UXVbUscj2PKUzCJfe4zd /X3wchoQO8ikTN1xJq8sGtW+KFWfDhg= X-Google-Smtp-Source: ABdhPJw0JDSnsdYmAsVHi4J2f0FGLlensBwkejldPLxpD21eKBz8edpd85rXp2Mf6B4ZZQe5NjCo0g== X-Received: by 2002:a5d:53c3:: with SMTP id a3mr226869wrw.43.1611431906280; Sat, 23 Jan 2021 11:58:26 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p18sm15625221wmc.31.2021.01.23.11.58.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Jan 2021 11:58:25 -0800 (PST) Message-Id: <9b545d7dbecc6f071e58787e9d4339f59ac4e972.1611431900.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sat, 23 Jan 2021 19:58:16 +0000 Subject: [PATCH v3 6/9] name-hash: use trace2 regions for init Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , gitster@pobox.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The lazy_init_name_hash() populates a hashset with all filenames and another with all directories represented in the index. This is run only if we need to use the hashsets to check for existence or case-folding renames. Place trace2 regions where there is already a performance trace. Signed-off-by: Derrick Stolee --- name-hash.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/name-hash.c b/name-hash.c index 5d3c7b12c18..4e03fac9bb1 100644 --- a/name-hash.c +++ b/name-hash.c @@ -7,6 +7,7 @@ */ #include "cache.h" #include "thread-utils.h" +#include "trace2.h" struct dir_entry { struct hashmap_entry ent; @@ -577,6 +578,7 @@ static void lazy_init_name_hash(struct index_state *istate) if (istate->name_hash_initialized) return; trace_performance_enter(); + trace2_region_enter("index", "name-hash-init", istate->repo); hashmap_init(&istate->name_hash, cache_entry_cmp, NULL, istate->cache_nr); hashmap_init(&istate->dir_hash, dir_entry_cmp, NULL, istate->cache_nr); @@ -597,6 +599,7 @@ static void lazy_init_name_hash(struct index_state *istate) } istate->name_hash_initialized = 1; + trace2_region_leave("index", "name-hash-init", istate->repo); trace_performance_leave("initialize name hash"); } From patchwork Sat Jan 23 19:58:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12041881 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2054C433DB for ; Sat, 23 Jan 2021 20:01:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AD1A922D2B for ; Sat, 23 Jan 2021 20:01:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726405AbhAWUAi (ORCPT ); Sat, 23 Jan 2021 15:00:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726298AbhAWT7w (ORCPT ); Sat, 23 Jan 2021 14:59:52 -0500 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6B3A3C061797 for ; Sat, 23 Jan 2021 11:58:28 -0800 (PST) Received: by mail-wr1-x434.google.com with SMTP id c12so8368116wrc.7 for ; Sat, 23 Jan 2021 11:58:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=4enHWQFO5nB7n6onrNC7/5mXxlAlvnYX7+cK2osA/Ao=; b=ZN6yYa426IMhJt0XW8iyS1FkBZXmvNqyTN9j47hm9x7EHP+Pq7NGCNcm99MbbwXe7l EYgFUfR1VzLhmHLG46+DasOmFzriyIyGfPhehlFP4MCXN7vnXTaseGELxNL7OU2wCKyk nsX22RMy02acQ0/B5ae9q8cMm+ejzn7Cg2x2ypcIyKWpTCjjn3svBlsTkowoT3MfyJUN hwGLTWbW6wYizwzuYq0W9WSk40N+C7Xie421RbFcUhrJ8ppVTCtcGcfL/msfWopXHRmf KPV+EgtoVr6GO2dfMygcyTx271VjDCe7yEzV2J9SrU2wtl0+qPzDsvRzlcECDiqnllfl ifAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=4enHWQFO5nB7n6onrNC7/5mXxlAlvnYX7+cK2osA/Ao=; b=Xc115/qLe9S6UutpdMwcRMYkzHtsMiATmucpwT1FXEY/XW/Ir0pEmtGBvq/eh9hX5c /K3a2havaX5QVdUUvDZPW4V/dc3EtMq3ZclqxOnaWQmCqmkk86EHWYS5Vo/631vtlNUq 1Df/Cti2vt/Tk/qVAEb6rG0yWVSDRL3E1YEwqOM7mDRZTBC9bRHNnivJRlj12J+GasjK WoMuawiWy+su1BlVAlI7CTqUql2eQkimTS3GnqV9arLRr7Aee0aGheKgQzOmYYkWXrj7 tXkIlrToxSJTXXYmMA/fRsaYhsX67A7JQRbnu6krReJsNaRxoflXiEwXvta0p+SGYTZG YaoQ== X-Gm-Message-State: AOAM5322/liHH3UpgnxttftVJJyQgyu23G+o3wbVTGmqJjErDfp6BIxe cyQhiQfXhaSz6VgMGc76fjKtaFiuptw= X-Google-Smtp-Source: ABdhPJyB/QxByBfKd3rZu1IJOgJrIaOcvYF1M06DZoDxk3lXzH/X/iJAL2T7IVar4u2DlXljljOl8A== X-Received: by 2002:a05:6000:1803:: with SMTP id m3mr10099803wrh.34.1611431907067; Sat, 23 Jan 2021 11:58:27 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o17sm10655686wrp.34.2021.01.23.11.58.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Jan 2021 11:58:26 -0800 (PST) Message-Id: <554cc7647e63008e1e2cf2ae3813412e16cfcce7.1611431900.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sat, 23 Jan 2021 19:58:17 +0000 Subject: [PATCH v3 7/9] sparse-checkout: load sparse-checkout patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , gitster@pobox.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee A future feature will want to load the sparse-checkout patterns into a pattern_list, but the current mechanism to do so is a bit complicated. This is made difficult due to needing to find the sparse-checkout file in different ways throughout the codebase. The logic implemented in the new get_sparse_checkout_patterns() was duplicated in populate_from_existing_patterns() in unpack-trees.c. Use the new method instead, keeping the logic around handling the struct unpack_trees_options. The callers to get_sparse_checkout_filename() in builtin/sparse-checkout.c manipulate the sparse-checkout file directly, so it is not appropriate to replace logic in that file with get_sparse_checkout_patterns(). Signed-off-by: Derrick Stolee --- builtin/sparse-checkout.c | 5 ----- dir.c | 17 +++++++++++++++++ dir.h | 2 ++ unpack-trees.c | 6 +----- 4 files changed, 20 insertions(+), 10 deletions(-) diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index e3140db2a0a..2306a9ad98e 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -22,11 +22,6 @@ static char const * const builtin_sparse_checkout_usage[] = { NULL }; -static char *get_sparse_checkout_filename(void) -{ - return git_pathdup("info/sparse-checkout"); -} - static void write_patterns_to_file(FILE *fp, struct pattern_list *pl) { int i; diff --git a/dir.c b/dir.c index d637461da5c..d153a63bbd1 100644 --- a/dir.c +++ b/dir.c @@ -2998,6 +2998,23 @@ void setup_standard_excludes(struct dir_struct *dir) } } +char *get_sparse_checkout_filename(void) +{ + return git_pathdup("info/sparse-checkout"); +} + +int get_sparse_checkout_patterns(struct pattern_list *pl) +{ + int res; + char *sparse_filename = get_sparse_checkout_filename(); + + pl->use_cone_patterns = core_sparse_checkout_cone; + res = add_patterns_from_file_to_list(sparse_filename, "", 0, pl, NULL); + + free(sparse_filename); + return res; +} + int remove_path(const char *name) { char *slash; diff --git a/dir.h b/dir.h index a3c40dec516..facfae47402 100644 --- a/dir.h +++ b/dir.h @@ -448,6 +448,8 @@ int is_empty_dir(const char *dir); void setup_standard_excludes(struct dir_struct *dir); +char *get_sparse_checkout_filename(void); +int get_sparse_checkout_patterns(struct pattern_list *pl); /* Constants for remove_dir_recursively: */ diff --git a/unpack-trees.c b/unpack-trees.c index a810b79657e..f5f668f532d 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -1549,14 +1549,10 @@ static void mark_new_skip_worktree(struct pattern_list *pl, static void populate_from_existing_patterns(struct unpack_trees_options *o, struct pattern_list *pl) { - char *sparse = git_pathdup("info/sparse-checkout"); - - pl->use_cone_patterns = core_sparse_checkout_cone; - if (add_patterns_from_file_to_list(sparse, "", 0, pl, NULL) < 0) + if (get_sparse_checkout_patterns(pl) < 0) o->skip_sparse_checkout = 1; else o->pl = pl; - free(sparse); } From patchwork Sat Jan 23 19:58:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12041879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4609DC433DB for ; Sat, 23 Jan 2021 20:00:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E51AF22DBF for ; Sat, 23 Jan 2021 20:00:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726374AbhAWUA1 (ORCPT ); Sat, 23 Jan 2021 15:00:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36698 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726301AbhAWT7w (ORCPT ); Sat, 23 Jan 2021 14:59:52 -0500 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4FCB0C0617A7 for ; Sat, 23 Jan 2021 11:58:29 -0800 (PST) Received: by mail-wr1-x436.google.com with SMTP id a9so8362668wrt.5 for ; Sat, 23 Jan 2021 11:58:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=K0JqViei2fw4QavZNgFk2sBZuOssyj5IlQ9XTxSxVVY=; b=SFFDR3WzDI2SvZE75DHQIwbxmeHwF6KKJ/nqGW/Eo+NhNyTHeSXScbbNuE/3ubibux 82FyU3ZX8MnFMy5Ld3geVxazs1nRS5K7KaAx+pjtrCD/EBUiJd+Tn8RUBFBl25Vav/cA RCDKeXpFkOJYYM98Xik8bP1c/dmDCetPA1DWrEDFq5MTrWwuXAI0ypsOshQQpYcFp141 J2lDJ0SDVMaXcwSJr8OujoslcXgEip8fPJGpFX5WWoI2pQ12rD34TxGNDTp0JPAjzU+j rCT3kynN/mtmm9NF2bZPuLrpTV8kkWTsEPJl7Utsw7iYtFYi92/vJKvEbkKlqpg9iuq/ cY/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=K0JqViei2fw4QavZNgFk2sBZuOssyj5IlQ9XTxSxVVY=; b=cxCLM5Auyzlyrsjgvtaeiq9N79Fadn1WaGaPCZt1knxO7ecLVWaWMl9lNUzqf+1cLM vZx5LTv5wCDuj1NnUk4vmGkDPbqiwuIC6zwqht/PbpVimo5CL855BZuZJzkoFbM9tZ3g WwxHeDnHiVL5nuy5AIncsRmPaPo5qVs91XKVy2K49EMgvmCJ8oJe6EhhE0rDZQLCUuRo vHxllo7lUmqnSmPYYL97Rgl1iMJ0HgZuyenOAsBe/i9TxnQWlkNkuYDXfnBB3DJZ630t Cjh51LAFjhmSSEKVMen6f6GHOmWR0aHo+nP1J4nG1VZVWl6KnXTnuvvIirSkxl6pU9rY 2B/Q== X-Gm-Message-State: AOAM5338B0RVp7xHrsiZ15wkBWa1LHEjktXUykUhJ0gfHSAaLjuBFVh1 Ega4cl155BIFHHszqEJKdCuSLgmk4BY= X-Google-Smtp-Source: ABdhPJx/XLPKmqDDdKv2CJaYiJSGzU8DaI29FNHWHbKeE3BrDNevsARgGaDkVe2ERXhvcuQ3hQOCSw== X-Received: by 2002:adf:f6c9:: with SMTP id y9mr10433304wrp.121.1611431907905; Sat, 23 Jan 2021 11:58:27 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u6sm11077731wro.75.2021.01.23.11.58.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Jan 2021 11:58:27 -0800 (PST) Message-Id: In-Reply-To: References: Date: Sat, 23 Jan 2021 19:58:18 +0000 Subject: [PATCH v3 8/9] test-lib: test_region looks for trace2 regions Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, Derrick Stolee , gitster@pobox.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Most test cases can verify Git's behavior using input/output expectations or changes to the .git directory. However, sometimes we want to check that Git did or did not run a certain section of code. This is particularly important for performance-only features that we want to ensure have been enabled in certain cases. Add a new 'test_region' function that checks if a trace2 region was entered and left in a given trace2 event log. There is one existing test (t0500-progress-display.sh) that performs this check already, so use the helper function instead. Note that this changes the expectations slightly. The old test (incorrectly) used two patterns for the 'grep' invocation, but this performs an OR of the patterns, not an AND. This means that as long as one region_enter event was logged, the test would succeed, even if it was not due to the progress category. More uses will be added in a later change. t6423-merge-rename-directories.sh also greps for region_enter lines, but it verifies the number of such lines, which is not the same as an existence check. Signed-off-by: Derrick Stolee Signed-off-by: Derrick Stolee --- t/t0500-progress-display.sh | 3 +-- t/test-lib-functions.sh | 42 +++++++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+), 2 deletions(-) diff --git a/t/t0500-progress-display.sh b/t/t0500-progress-display.sh index 1ed1df351cb..84cce345e7d 100755 --- a/t/t0500-progress-display.sh +++ b/t/t0500-progress-display.sh @@ -303,8 +303,7 @@ test_expect_success 'progress generates traces' ' "Working hard" stderr && # t0212/parse_events.perl intentionally omits regions and data. - grep -e "region_enter" -e "\"category\":\"progress\"" trace.event && - grep -e "region_leave" -e "\"category\":\"progress\"" trace.event && + test_region progress "Working hard" trace.event && grep "\"key\":\"total_objects\",\"value\":\"40\"" trace.event && grep "\"key\":\"total_bytes\",\"value\":\"409600\"" trace.event ' diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh index 999982fe4a9..3f425deba18 100644 --- a/t/test-lib-functions.sh +++ b/t/test-lib-functions.sh @@ -1655,3 +1655,45 @@ test_subcommand () { grep "\[$expr\]" fi } + +# Check that the given command was invoked as part of the +# trace2-format trace on stdin. +# +# test_region [!]