From patchwork Thu Jul 25 02:23:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kazuhiro Kato via GitGitGadget X-Patchwork-Id: 11057817 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6BAAD112C for ; Thu, 25 Jul 2019 02:23:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5014C286C1 for ; Thu, 25 Jul 2019 02:23:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 43C1C28891; Thu, 25 Jul 2019 02:23:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 768D928978 for ; Thu, 25 Jul 2019 02:23:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388086AbfGYCXS (ORCPT ); Wed, 24 Jul 2019 22:23:18 -0400 Received: from mail-wm1-f65.google.com ([209.85.128.65]:54058 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387891AbfGYCXS (ORCPT ); Wed, 24 Jul 2019 22:23:18 -0400 Received: by mail-wm1-f65.google.com with SMTP id x15so43477757wmj.3 for ; Wed, 24 Jul 2019 19:23:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=QTdg6VwW3R+WGE2Fh7CSuYzkFulUqcJMSNMpAwx7qD4=; b=Yxr5HTvesgfaIb5KgC6L6lYMfUKoeXPLOJJn8hRUDvfcFgzkglYXRKmnJQZ+LppR9N MqP93Wat4sxmlaNjDh5bLpJ8jhq62usfyIqeCTaUgzSL5qI1jQ4bmaUdvixliUWn5NJK tubY/K07QNuAJROFzrqUPt8MxtZNUBPtvzWczxcAdmWgLoEqoGa2U/0PCqxHRyYfmiw7 8KA+1i2KxCix5/087tAYwyiRKfdWeiDqheDU+I83WRT7T5wDM3/TePSoWBiCG20e6s0y qeUbSNaH4Uoq1cf8NFRkLw+IL5N3SScsYAAHcmsUqFasSY9UETtf2b5uf08XMtFOGuiA SLrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=QTdg6VwW3R+WGE2Fh7CSuYzkFulUqcJMSNMpAwx7qD4=; b=RELgDqCCPM9xAaElJ2kWn8DH08TehJkph9IkLV8BPSNL6D+nOqlBS2AiLmxg6rIuMn 4oPes92CXbtAyor55CbG7ViPvjSCmaVizFZ6rmdb6nyLD5sCjULSrdeSn2NAUrpjNnll 9hQDxT1fDfyqFUylHA0X22hk2sxRf9uliJ+3kJFSgJ1yS+G/AjqUouMoXsbh38xEK2Hu XmrNHiGC3ENez5rXONmmaNC0pHE+f6Qih6mvUKMuhvVfBUfIPpyRk0El9Bv07hYa4BoB dYHdUz92b0SBFsq+Z7Gr4p3P9pV7v9zeTgTa/ZNIh0cKIOlpRC5k9BfX66HOr0QjtsYp B2TA== X-Gm-Message-State: APjAAAUCMO8uQDItK61Z+dh0Uqc+rXqtZgYgKk5WKs64h0z8iLMdnVIe tUYj4n6tAZhxryVtMCnpFm3eM/vu X-Google-Smtp-Source: APXvYqyItxFr7c69s7Cq5dKKU0XSF2GSOsY/8998Poy+/oOv9enK45EBbeqcQspUCvLV0nCZy4KVDw== X-Received: by 2002:a1c:6a17:: with SMTP id f23mr14965860wmc.91.1564021395439; Wed, 24 Jul 2019 19:23:15 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e19sm64571807wra.71.2019.07.24.19.23.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Jul 2019 19:23:15 -0700 (PDT) Date: Wed, 24 Jul 2019 19:23:15 -0700 (PDT) X-Google-Original-Date: Thu, 25 Jul 2019 02:23:08 GMT Message-Id: <597ab7d621983b5ad35006a2530aa70b452e895f.1564021392.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 1/5] repo-settings: consolidate some config settings Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes.Schindelin@gmx.de, peff@peff.net, jnareb@gmail.com, pclouds@gmail.com, carenas@gmail.com, avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee There are a few important config settings that are not loaded during git_default_config. These are instead loaded on-demand. Centralize these config options to a single scan, and store all of the values in a repo_settings struct. The values for each setting are initialized as negative to indicate "unset". This centralization will be particularly important in a later change to introduce "meta" config settings that change the defaults for these config settings. Signed-off-by: Derrick Stolee --- Makefile | 1 + builtin/gc.c | 13 ++++++------- builtin/pack-objects.c | 9 +++++---- commit-graph.c | 7 ++++--- read-cache.c | 12 +++++++----- repo-settings.c | 28 ++++++++++++++++++++++++++++ repo-settings.h | 17 +++++++++++++++++ repository.h | 5 +++++ 8 files changed, 73 insertions(+), 19 deletions(-) create mode 100644 repo-settings.c create mode 100644 repo-settings.h diff --git a/Makefile b/Makefile index 11ccea4071..032fe9b5f9 100644 --- a/Makefile +++ b/Makefile @@ -964,6 +964,7 @@ LIB_OBJS += refspec.o LIB_OBJS += ref-filter.o LIB_OBJS += remote.o LIB_OBJS += replace-object.o +LIB_OBJS += repo-settings.o LIB_OBJS += repository.o LIB_OBJS += rerere.o LIB_OBJS += resolve-undo.o diff --git a/builtin/gc.c b/builtin/gc.c index c18efadda5..5e365049aa 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -27,6 +27,7 @@ #include "pack-objects.h" #include "blob.h" #include "tree.h" +#include "repo-settings.h" #define FAILED_RUN "failed to run %s" @@ -41,7 +42,6 @@ static int aggressive_depth = 50; static int aggressive_window = 250; static int gc_auto_threshold = 6700; static int gc_auto_pack_limit = 50; -static int gc_write_commit_graph; static int detach_auto = 1; static timestamp_t gc_log_expire_time; static const char *gc_log_expire = "1.day.ago"; @@ -148,7 +148,6 @@ static void gc_config(void) git_config_get_int("gc.aggressivedepth", &aggressive_depth); git_config_get_int("gc.auto", &gc_auto_threshold); git_config_get_int("gc.autopacklimit", &gc_auto_pack_limit); - git_config_get_bool("gc.writecommitgraph", &gc_write_commit_graph); git_config_get_bool("gc.autodetach", &detach_auto); git_config_get_expiry("gc.pruneexpire", &prune_expire); git_config_get_expiry("gc.worktreepruneexpire", &prune_worktrees_expire); @@ -685,11 +684,11 @@ int cmd_gc(int argc, const char **argv, const char *prefix) clean_pack_garbage(); } - if (gc_write_commit_graph && - write_commit_graph_reachable(get_object_directory(), - !quiet && !daemonized ? COMMIT_GRAPH_PROGRESS : 0, - NULL)) - return 1; + prepare_repo_settings(the_repository); + if (the_repository->settings.gc_write_commit_graph == 1) + write_commit_graph_reachable(get_object_directory(), + !quiet && !daemonized ? COMMIT_GRAPH_PROGRESS : 0, + NULL); if (auto_gc && too_many_loose_objects()) warning(_("There are too many unreachable loose objects; " diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 267c562b1f..20bcd57033 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -34,6 +34,7 @@ #include "dir.h" #include "midx.h" #include "trace2.h" +#include "repo-settings.h" #define IN_PACK(obj) oe_in_pack(&to_pack, obj) #define SIZE(obj) oe_size(&to_pack, obj) @@ -2709,10 +2710,6 @@ static int git_pack_config(const char *k, const char *v, void *cb) use_bitmap_index_default = git_config_bool(k, v); return 0; } - if (!strcmp(k, "pack.usesparse")) { - sparse = git_config_bool(k, v); - return 0; - } if (!strcmp(k, "pack.threads")) { delta_search_threads = git_config_int(k, v); if (delta_search_threads < 0) @@ -3332,6 +3329,10 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) read_replace_refs = 0; sparse = git_env_bool("GIT_TEST_PACK_SPARSE", 0); + prepare_repo_settings(the_repository); + if (!sparse && the_repository->settings.pack_use_sparse != -1) + sparse = the_repository->settings.pack_use_sparse; + reset_pack_idx_option(&pack_idx_opts); git_config(git_pack_config, NULL); diff --git a/commit-graph.c b/commit-graph.c index b3c4de79b6..0c2dadab9b 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -16,6 +16,7 @@ #include "hashmap.h" #include "replace-object.h" #include "progress.h" +#include "repo-settings.h" #define GRAPH_SIGNATURE 0x43475048 /* "CGPH" */ #define GRAPH_CHUNKID_OIDFANOUT 0x4f494446 /* "OIDF" */ @@ -466,7 +467,6 @@ static void prepare_commit_graph_one(struct repository *r, const char *obj_dir) static int prepare_commit_graph(struct repository *r) { struct object_directory *odb; - int config_value; if (git_env_bool(GIT_TEST_COMMIT_GRAPH_DIE_ON_LOAD, 0)) die("dying as requested by the '%s' variable on commit-graph load!", @@ -476,9 +476,10 @@ static int prepare_commit_graph(struct repository *r) return !!r->objects->commit_graph; r->objects->commit_graph_attempted = 1; + prepare_repo_settings(r); + if (!git_env_bool(GIT_TEST_COMMIT_GRAPH, 0) && - (repo_config_get_bool(r, "core.commitgraph", &config_value) || - !config_value)) + r->settings.core_commit_graph != 1) /* * This repository is not configured to use commit graphs, so * do not load one. (But report commit_graph_attempted anyway diff --git a/read-cache.c b/read-cache.c index c701f7f8b8..bda665a25a 100644 --- a/read-cache.c +++ b/read-cache.c @@ -25,6 +25,7 @@ #include "fsmonitor.h" #include "thread-utils.h" #include "progress.h" +#include "repo-settings.h" /* Mask for the name length in ce_flags in the on-disk index */ @@ -1599,16 +1600,17 @@ struct cache_entry *refresh_cache_entry(struct index_state *istate, #define INDEX_FORMAT_DEFAULT 3 -static unsigned int get_index_format_default(void) +static unsigned int get_index_format_default(struct repository *r) { char *envversion = getenv("GIT_INDEX_VERSION"); char *endp; - int value; unsigned int version = INDEX_FORMAT_DEFAULT; if (!envversion) { - if (!git_config_get_int("index.version", &value)) - version = value; + prepare_repo_settings(r); + + if (r->settings.index_version >= 0) + version = r->settings.index_version; if (version < INDEX_FORMAT_LB || INDEX_FORMAT_UB < version) { warning(_("index.version set, but the value is invalid.\n" "Using version %i"), INDEX_FORMAT_DEFAULT); @@ -2765,7 +2767,7 @@ static int do_write_index(struct index_state *istate, struct tempfile *tempfile, } if (!istate->version) { - istate->version = get_index_format_default(); + istate->version = get_index_format_default(the_repository); if (git_env_bool("GIT_TEST_SPLIT_INDEX", 0)) init_split_index(istate); } diff --git a/repo-settings.c b/repo-settings.c new file mode 100644 index 0000000000..2c33810cc6 --- /dev/null +++ b/repo-settings.c @@ -0,0 +1,28 @@ +#include "cache.h" +#include "repository.h" +#include "config.h" +#include "repo-settings.h" + +void prepare_repo_settings(struct repository *r) +{ + int value; + + if (r->settings_initialized) + return; + + /* Defaults */ + memset(&r->settings, -1, sizeof(r->settings)); + + if (!repo_config_get_bool(r, "core.commitgraph", &value)) + r->settings.core_commit_graph = value; + if (!repo_config_get_bool(r, "gc.writecommitgraph", &value)) + r->settings.gc_write_commit_graph = value; + + if (!repo_config_get_bool(r, "index.version", &value)) + r->settings.index_version = value; + + if (!repo_config_get_bool(r, "pack.usesparse", &value)) + r->settings.pack_use_sparse = value; + + r->settings_initialized = 1; +} diff --git a/repo-settings.h b/repo-settings.h new file mode 100644 index 0000000000..89fb0159bf --- /dev/null +++ b/repo-settings.h @@ -0,0 +1,17 @@ +#ifndef REPO_SETTINGS_H +#define REPO_SETTINGS_H + +struct repo_settings { + int core_commit_graph; + int gc_write_commit_graph; + + int index_version; + + int pack_use_sparse; +}; + +struct repository; + +void prepare_repo_settings(struct repository *r); + +#endif /* REPO_SETTINGS_H */ diff --git a/repository.h b/repository.h index 4fb6a5885f..a817486825 100644 --- a/repository.h +++ b/repository.h @@ -2,8 +2,10 @@ #define REPOSITORY_H #include "path.h" +#include "repo-settings.h" struct config_set; +struct repo_settings; struct git_hash_algo; struct index_state; struct lock_file; @@ -72,6 +74,9 @@ struct repository { */ char *submodule_prefix; + int settings_initialized; + struct repo_settings settings; + /* Subsystems */ /* * Repository's config which contains key-value pairs from the usual From patchwork Thu Jul 25 02:23:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kazuhiro Kato via GitGitGadget X-Patchwork-Id: 11057819 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6FB6D746 for ; Thu, 25 Jul 2019 02:23:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 608DA208C2 for ; Thu, 25 Jul 2019 02:23:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 549C227CF3; Thu, 25 Jul 2019 02:23:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D876328891 for ; Thu, 25 Jul 2019 02:23:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388870AbfGYCXT (ORCPT ); Wed, 24 Jul 2019 22:23:19 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:38773 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387933AbfGYCXT (ORCPT ); Wed, 24 Jul 2019 22:23:19 -0400 Received: by mail-wr1-f68.google.com with SMTP id g17so48957273wrr.5 for ; Wed, 24 Jul 2019 19:23:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=6ZlBC78mwHouVI+oKBj3adbBX91NbubjtF0qo50X5oY=; b=fQSE71mIti5RnU3A085brbW0zimDzzvJd29Fa12CZLgP0IRZgmjAYGeO4CpZtG4aZR pu0ARYARMoKnffwy2bswLZJ3bD2jRCGftW7eMBQkNe1h7CrwZBzKvEjEE/wqUf2CCQWZ 5IhUAd4EOplcQ4k59wlCX1SC3voMYwI8DmVufZy7Y2UyXOSE7njX1bOFLtwD+NuPtC9+ zWLa0dYndK034oKZ3G1doN7H8MhPnleA2FgC8M39YOj7o045yJTme61iN0Og75zwviGJ 2HkTkzMR15BiPoRFkgGr+TyBVn7FujdaqPiIelXMvTEWzIUkmgAeLkCYfN8mbAeXubLJ 4DQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=6ZlBC78mwHouVI+oKBj3adbBX91NbubjtF0qo50X5oY=; b=jcNEA1VVQ6cnvablYlY8DKOPWP6MKDKPut8/KPXVwVHusaZHW05I8JqqUvCZV1UWdb 8KkXphAOYaF7mkP3chWZ0mfkVPZtdTIMraN09o/icOmiqbSolN0v2mzE+2TqFarFhOy7 m9/7bE0rHrrpiToqHm+o8LfxFpKcY/hBzVBeC9I24jf8wgcGhW59KIR+/ouFd1Ns73qX 0INEX+QAODAGMpL6UOFMBSVWedKEf+x5pEMv+k+Jz/XJtaOP394UosZCJQy9f1ZivD8l UkBub+QHD+UuKdPYuJClgUKyleew9eB73EbEifBXmRoezE4fyoIOP7BmFVxUUzZnCcfy eykg== X-Gm-Message-State: APjAAAVE4wcIIhKOB4DfvmvKAwr6MdZSSpbIYVRqDIxoK2ooPULWqY5i /9UvR5qaNos7XCcb7A2rep/JT148 X-Google-Smtp-Source: APXvYqy30GYBSF+m8J+zBoPc4rh3+Pc0xDbw+tn8ulswUf+BlzrRlnMII2ZmV+52W4X9s/8KYM6ZhQ== X-Received: by 2002:adf:f6d2:: with SMTP id y18mr15469592wrp.102.1564021396209; Wed, 24 Jul 2019 19:23:16 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p18sm48169376wrm.16.2019.07.24.19.23.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Jul 2019 19:23:15 -0700 (PDT) Date: Wed, 24 Jul 2019 19:23:15 -0700 (PDT) X-Google-Original-Date: Thu, 25 Jul 2019 02:23:09 GMT Message-Id: <86380c78321528221c6b0dd6520e714104dbcd18.1564021393.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 2/5] repo-settings: add feature.manyCommits setting Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes.Schindelin@gmx.de, peff@peff.net, jnareb@gmail.com, pclouds@gmail.com, carenas@gmail.com, avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee When a repo has many commits, it is helpful to write and read the commit-graph file. Future changes to Git may include new config settings that are beneficial in this scenario. Create the 'feature.manyCommits' config setting that changes the default values of 'core.commitGraph' and 'gc.writeCommitGraph' to true. Signed-off-by: Derrick Stolee --- Documentation/config.txt | 2 ++ Documentation/config/core.txt | 3 ++- Documentation/config/feature.txt | 15 +++++++++++++++ Documentation/config/gc.txt | 4 ++-- repo-settings.c | 7 +++++++ 5 files changed, 28 insertions(+), 3 deletions(-) create mode 100644 Documentation/config/feature.txt diff --git a/Documentation/config.txt b/Documentation/config.txt index e3f5bc3396..77f3b1486b 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -345,6 +345,8 @@ include::config/difftool.txt[] include::config/fastimport.txt[] +include::config/feature.txt[] + include::config/fetch.txt[] include::config/format.txt[] diff --git a/Documentation/config/core.txt b/Documentation/config/core.txt index 75538d27e7..d80162681a 100644 --- a/Documentation/config/core.txt +++ b/Documentation/config/core.txt @@ -577,7 +577,8 @@ the `GIT_NOTES_REF` environment variable. See linkgit:git-notes[1]. core.commitGraph:: If true, then git will read the commit-graph file (if it exists) - to parse the graph structure of commits. Defaults to false. See + to parse the graph structure of commits. Defaults to false, unless + `feature.manyCommits` is enabled. See linkgit:git-commit-graph[1] for more information. core.useReplaceRefs:: diff --git a/Documentation/config/feature.txt b/Documentation/config/feature.txt new file mode 100644 index 0000000000..f74314ae90 --- /dev/null +++ b/Documentation/config/feature.txt @@ -0,0 +1,15 @@ +feature.*:: + The config settings that start with `feature.` modify the defaults of + a group of other config settings. These groups are created by the Git + developer community as recommended defaults and are subject to change. + In particular, new config options may be added with different defaults. + +feature.manyCommits:: + Enable config options that optimize for repos with many commits. This + setting is recommended for repos with at least 100,000 commits. The + new default values are: ++ +* `core.commitGraph=true` enables reading the commit-graph file. ++ +* `gc.writeCommitGraph=true` enables writing the commit-graph file during +garbage collection. \ No newline at end of file diff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt index 02b92b18b5..31a5fc4f75 100644 --- a/Documentation/config/gc.txt +++ b/Documentation/config/gc.txt @@ -63,8 +63,8 @@ gc.writeCommitGraph:: If true, then gc will rewrite the commit-graph file when linkgit:git-gc[1] is run. When using `git gc --auto` the commit-graph will be updated if housekeeping is - required. Default is false. See linkgit:git-commit-graph[1] - for details. + required. Default is false, unless `feature.manyCommits` + is enabled. See linkgit:git-commit-graph[1] for details. gc.logExpiry:: If the file gc.log exists, then `git gc --auto` will print diff --git a/repo-settings.c b/repo-settings.c index 2c33810cc6..3face7d8b9 100644 --- a/repo-settings.c +++ b/repo-settings.c @@ -3,6 +3,8 @@ #include "config.h" #include "repo-settings.h" +#define UPDATE_DEFAULT(s,v) do { if (s == -1) { s = v; } } while(0) + void prepare_repo_settings(struct repository *r) { int value; @@ -24,5 +26,10 @@ void prepare_repo_settings(struct repository *r) if (!repo_config_get_bool(r, "pack.usesparse", &value)) r->settings.pack_use_sparse = value; + if (!repo_config_get_bool(r, "feature.manycommits", &value) && value) { + UPDATE_DEFAULT(r->settings.core_commit_graph, 1); + UPDATE_DEFAULT(r->settings.gc_write_commit_graph, 1); + } + r->settings_initialized = 1; } From patchwork Thu Jul 25 02:23:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kazuhiro Kato via GitGitGadget X-Patchwork-Id: 11057821 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 22029746 for ; Thu, 25 Jul 2019 02:23:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 11885286C1 for ; Thu, 25 Jul 2019 02:23:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 05A8928979; Thu, 25 Jul 2019 02:23:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 528432897B for ; Thu, 25 Jul 2019 02:23:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389030AbfGYCXW (ORCPT ); Wed, 24 Jul 2019 22:23:22 -0400 Received: from mail-wm1-f67.google.com ([209.85.128.67]:35721 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387955AbfGYCXT (ORCPT ); Wed, 24 Jul 2019 22:23:19 -0400 Received: by mail-wm1-f67.google.com with SMTP id l2so43215478wmg.0 for ; Wed, 24 Jul 2019 19:23:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=aGy3SlCJoVzMvuvx6qmq2C8aGYmxu6frNL3a/vFmo7c=; b=nF5pl4ISGEWvgmvV9/ZDIgLeNjOLjgSaHUoI9GVTEZDIlftPVJ725VKcLVNeRIgsC5 7qzLqnq7Av/aKIHKAOKn8AvMJFGvhVhnmx+ieyVYtrKs4SN+NnL8l5OdQh16yMLlgztd eyU0cmQXj8mVVmp86EalYZQcBimJCXBOyIuIdiKZsAsEbTX77XYpe5qI0XUvqKWEDfO+ CimQ4z6gMW1K+8LcGX35e6dU/GprzBIOpzuiT44Jq96cOwSfXKC5KxcRYqphRbAbY9p6 hu3OylJcONpax3y92DybpL2sTcL6LIRl1k6RMOgB3fWvg1q20NIOyIdA2aPtdl2Y4wwN RpEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=aGy3SlCJoVzMvuvx6qmq2C8aGYmxu6frNL3a/vFmo7c=; b=YB4C0UHzlGYBRrQpY4162blAee9dMkVrbe6HeAeD4Qn4ETkCIQQyJWzsVmDO7fvWVY Vd2Z8EMk4vtG8c5RtQW46tYW+j+YUecWEGszhPTABnq6LOIWWt6lhMameFaSGZZq7ln9 sE04lh9VqqwMVL8ZkhJVUPZ3tLW1Zme1a00JcLbth6bKrZSbimD7/0j60J1Z5u/mgHlI uX8vtwmqsDZLTIP1KWaAun1trfa/QLScAdwzIGapAPsRf6LGWG5DVsl643MYC/fMGTtJ pDIfIe8aVBf/+BcnD0r4JyGhmnzcx97EwKVjiCvG0FIv5f7zmK7+piZJ4PDoeRuHTCJp herg== X-Gm-Message-State: APjAAAXvWfs81tyvktOtfQqK/PEkBPfar0AWErid5DiPjRBIqKyUnPoZ konzs36awISjG2MyQmp98Wp5bfEj X-Google-Smtp-Source: APXvYqyCP2nQBgDp9w1kDe7/RhboK/xCk2kBquxW0dSIDV5odN+1eVfTbofnk/fMsg5Bfr79Gc0RWg== X-Received: by 2002:a1c:968c:: with SMTP id y134mr75207094wmd.75.1564021397015; Wed, 24 Jul 2019 19:23:17 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l25sm36626502wme.13.2019.07.24.19.23.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Jul 2019 19:23:16 -0700 (PDT) Date: Wed, 24 Jul 2019 19:23:16 -0700 (PDT) X-Google-Original-Date: Thu, 25 Jul 2019 02:23:10 GMT Message-Id: <49be7a73454b13b04270531eece87a2c6f6d3189.1564021393.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 3/5] repo-settings: parse core.untrackedCache Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes.Schindelin@gmx.de, peff@peff.net, jnareb@gmail.com, pclouds@gmail.com, carenas@gmail.com, avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee The core.untrackedCache config setting is slightly complicated, so clarify its use and centralize its parsing into the repo settings. The default value is "keep" (returned as -1), which persists the untracked cache if it exists. If the value is set as "false" (returned as 0), then remove the untracked cache if it exists. If the value is set as "true" (returned as 1), then write the untracked cache and persist it. Instead of relying on magic values of -1, 0, and 1, split these options into an enum. This allows the use of "-1" as a default value. After parsing the config options, if the value is unset we can initialize it to UNTRACKED_CACHE_KEEP. Signed-off-by: Derrick Stolee --- builtin/update-index.c | 7 +++++-- config.c | 24 ------------------------ read-cache.c | 19 +++++++++---------- repo-settings.c | 21 ++++++++++++++++++++- repo-settings.h | 8 ++++++++ 5 files changed, 42 insertions(+), 37 deletions(-) diff --git a/builtin/update-index.c b/builtin/update-index.c index dff2f4b837..5ced51c1ee 100644 --- a/builtin/update-index.c +++ b/builtin/update-index.c @@ -18,6 +18,7 @@ #include "dir.h" #include "split-index.h" #include "fsmonitor.h" +#include "repo-settings.h" /* * Default to not allowing changes to the list of files. The @@ -966,6 +967,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) struct parse_opt_ctx_t ctx; strbuf_getline_fn getline_fn; int parseopt_state = PARSE_OPT_UNKNOWN; + struct repository *r = the_repository; struct option options[] = { OPT_BIT('q', NULL, &refresh_args.flags, N_("continue refresh even when index needs update"), @@ -1180,11 +1182,12 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) remove_split_index(&the_index); } + prepare_repo_settings(r); switch (untracked_cache) { case UC_UNSPECIFIED: break; case UC_DISABLE: - if (git_config_get_untracked_cache() == 1) + if (r->settings.core_untracked_cache == UNTRACKED_CACHE_WRITE) warning(_("core.untrackedCache is set to true; " "remove or change it, if you really want to " "disable the untracked cache")); @@ -1196,7 +1199,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) return !test_if_untracked_cache_is_supported(); case UC_ENABLE: case UC_FORCE: - if (git_config_get_untracked_cache() == 0) + if (r->settings.core_untracked_cache == UNTRACKED_CACHE_REMOVE) warning(_("core.untrackedCache is set to false; " "remove or change it, if you really want to " "enable the untracked cache")); diff --git a/config.c b/config.c index faa57e436c..3241dbc54d 100644 --- a/config.c +++ b/config.c @@ -2277,30 +2277,6 @@ int git_config_get_expiry_in_days(const char *key, timestamp_t *expiry, timestam return -1; /* thing exists but cannot be parsed */ } -int git_config_get_untracked_cache(void) -{ - int val = -1; - const char *v; - - /* Hack for test programs like test-dump-untracked-cache */ - if (ignore_untracked_cache_config) - return -1; - - if (!git_config_get_maybe_bool("core.untrackedcache", &val)) - return val; - - if (!git_config_get_value("core.untrackedcache", &v)) { - if (!strcasecmp(v, "keep")) - return -1; - - error(_("unknown core.untrackedCache value '%s'; " - "using 'keep' default value"), v); - return -1; - } - - return -1; /* default value */ -} - int git_config_get_split_index(void) { int val; diff --git a/read-cache.c b/read-cache.c index bda665a25a..3761b9a171 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1846,18 +1846,17 @@ static void check_ce_order(struct index_state *istate) static void tweak_untracked_cache(struct index_state *istate) { - switch (git_config_get_untracked_cache()) { - case -1: /* keep: do nothing */ - break; - case 0: /* false */ + struct repository *r = the_repository; + + prepare_repo_settings(r); + + if (r->settings.core_untracked_cache == UNTRACKED_CACHE_REMOVE) { remove_untracked_cache(istate); - break; - case 1: /* true */ - add_untracked_cache(istate); - break; - default: /* unknown value: do nothing */ - break; + return; } + + if (r->settings.core_untracked_cache == UNTRACKED_CACHE_WRITE) + add_untracked_cache(istate); } static void tweak_split_index(struct index_state *istate) diff --git a/repo-settings.c b/repo-settings.c index 3face7d8b9..65b980403c 100644 --- a/repo-settings.c +++ b/repo-settings.c @@ -8,6 +8,7 @@ void prepare_repo_settings(struct repository *r) { int value; + char *strval; if (r->settings_initialized) return; @@ -22,14 +23,32 @@ void prepare_repo_settings(struct repository *r) if (!repo_config_get_bool(r, "index.version", &value)) r->settings.index_version = value; + if (!repo_config_get_maybe_bool(r, "core.untrackedcache", &value)) { + if (value == 0) + r->settings.core_untracked_cache = UNTRACKED_CACHE_REMOVE; + else + r->settings.core_untracked_cache = UNTRACKED_CACHE_WRITE; + } else if (!repo_config_get_string(r, "core.untrackedcache", &strval)) { + if (!strcasecmp(strval, "keep")) + r->settings.core_untracked_cache = UNTRACKED_CACHE_KEEP; + + free(strval); + } + if (!repo_config_get_bool(r, "pack.usesparse", &value)) r->settings.pack_use_sparse = value; - + if (!repo_config_get_bool(r, "feature.manycommits", &value) && value) { UPDATE_DEFAULT(r->settings.core_commit_graph, 1); UPDATE_DEFAULT(r->settings.gc_write_commit_graph, 1); } + /* Hack for test programs like test-dump-untracked-cache */ + if (ignore_untracked_cache_config) + r->settings.core_untracked_cache = UNTRACKED_CACHE_KEEP; + else + UPDATE_DEFAULT(r->settings.core_untracked_cache, UNTRACKED_CACHE_KEEP); + r->settings_initialized = 1; } diff --git a/repo-settings.h b/repo-settings.h index 89fb0159bf..e9b028e1a9 100644 --- a/repo-settings.h +++ b/repo-settings.h @@ -1,11 +1,19 @@ #ifndef REPO_SETTINGS_H #define REPO_SETTINGS_H +enum untracked_cache_setting { + UNTRACKED_CACHE_UNSET = -1, + UNTRACKED_CACHE_REMOVE = 0, + UNTRACKED_CACHE_KEEP = 1, + UNTRACKED_CACHE_WRITE = 2 +}; + struct repo_settings { int core_commit_graph; int gc_write_commit_graph; int index_version; + enum untracked_cache_setting core_untracked_cache; int pack_use_sparse; }; From patchwork Thu Jul 25 02:23:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kazuhiro Kato via GitGitGadget X-Patchwork-Id: 11057825 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 913BC13AC for ; Thu, 25 Jul 2019 02:23:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 82F1E27CF3 for ; Thu, 25 Jul 2019 02:23:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 77170288B9; Thu, 25 Jul 2019 02:23:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8058D28817 for ; Thu, 25 Jul 2019 02:23:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389034AbfGYCXX (ORCPT ); Wed, 24 Jul 2019 22:23:23 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:44947 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387891AbfGYCXT (ORCPT ); Wed, 24 Jul 2019 22:23:19 -0400 Received: by mail-wr1-f65.google.com with SMTP id p17so48914483wrf.11 for ; Wed, 24 Jul 2019 19:23:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=WMGvNSbFOrxNYHiyE4LFbFR9o+mRviwzKKwZMxcizxQ=; b=F2Bz/z7s2Q9/Xl71PFOdPwCKzfDdoqSeHJmszjorC2QvfalA5HqDrQL/8sglPIQguP 3o0rnaLeZtvoesDO7UqG7p4IoFlqriuo1fS+hk9a8wOfZTqINq+aaIuVKRMbwduINo9C 1Q5lE+rDsXyZWvJ4J/fAW1JWnz/R3jVJSIlIA2ROYdJA5Hu/pN5cCZmBPlKoIL8McNH1 EDsWYA3nlb1/zzTawemV847V7ygluwCxxo9aR7ot/onsNsQ6HVwzZ/7MurwZb8bWOESg t6d25ZXbmzOZ2ue+yHcj2e6srRLA3M8h8/Tu10bNAhunsB+Xl4IzwKCafhz+DdkeFoFb q9Aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=WMGvNSbFOrxNYHiyE4LFbFR9o+mRviwzKKwZMxcizxQ=; b=XFEEOelvHya5ovAMrJF1/dLjpvJnDgH4QINc+0Npdk9wMMAe8BKES81Z2I9JYdK0Op BQUa6/If7AgrBmlqV4tF4s3NT70p/A35pp9XbhyZesoDG24XZUdhCLBA44I75q47TjNI p3IVvZ6I9LTZTLw5RZolg85dRZ6KMSsHs+BNV3kqYNDnliWjWzOUsiJUJsxqsnObd9rz awXOcveYNrVwf8TGzBpy2EhA+994CH+2+w4yM4v5KYUtdFB1B4Rcd8idtt8CvovAPEaJ a/M3PqNd+V5FoB/44kB0m415R28RJTzYeLXtfm/agIwo750lvDOVk1RXBtBzPopa4MOS QFDA== X-Gm-Message-State: APjAAAW1ECLO8oPYwXBub89pnRQ14RkCDJtM6WLJXMYQUUW7afgO2tMY GCSEvLODsyUc2ORaKAaTO5tboaZ6 X-Google-Smtp-Source: APXvYqx0P9IW25Bh6CTwUOB9OlKzGgtU/WykD0BJbKpEii8krKjTSJgGxY16xy4wnCvb3IRJ66AYoA== X-Received: by 2002:adf:f246:: with SMTP id b6mr63572048wrp.92.1564021397776; Wed, 24 Jul 2019 19:23:17 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 91sm99347408wrp.3.2019.07.24.19.23.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Jul 2019 19:23:17 -0700 (PDT) Date: Wed, 24 Jul 2019 19:23:17 -0700 (PDT) X-Google-Original-Date: Thu, 25 Jul 2019 02:23:11 GMT Message-Id: <86a5a0c5895b4cd99a3d42469abe8142be7ac380.1564021393.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 4/5] repo-settings: create feature.manyFiles setting Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes.Schindelin@gmx.de, peff@peff.net, jnareb@gmail.com, pclouds@gmail.com, carenas@gmail.com, avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee The feature.manyFiles setting is suitable for repos with many files in the working directory. By setting index.version=4 and core.untrackedCache=true, commands such as 'git status' should improve. While adding this setting, modify the index version precedence tests to check how this setting overrides the default for index.version is unset. Signed-off-by: Derrick Stolee --- Documentation/config/core.txt | 4 +++- Documentation/config/feature.txt | 12 +++++++++++- Documentation/config/index.txt | 1 + repo-settings.c | 4 ++++ t/t1600-index.sh | 31 ++++++++++++++++++++++++++----- 5 files changed, 45 insertions(+), 7 deletions(-) diff --git a/Documentation/config/core.txt b/Documentation/config/core.txt index d80162681a..7a2a33bc8c 100644 --- a/Documentation/config/core.txt +++ b/Documentation/config/core.txt @@ -86,7 +86,9 @@ core.untrackedCache:: it will automatically be removed, if set to `false`. Before setting it to `true`, you should check that mtime is working properly on your system. - See linkgit:git-update-index[1]. `keep` by default. + See linkgit:git-update-index[1]. `keep` by default, unless + `feature.manyFiles` is enabled which sets this setting to + `true` by default. core.checkStat:: When missing or is set to `default`, many fields in the stat diff --git a/Documentation/config/feature.txt b/Documentation/config/feature.txt index f74314ae90..c2d9ef7473 100644 --- a/Documentation/config/feature.txt +++ b/Documentation/config/feature.txt @@ -12,4 +12,14 @@ feature.manyCommits:: * `core.commitGraph=true` enables reading the commit-graph file. + * `gc.writeCommitGraph=true` enables writing the commit-graph file during -garbage collection. \ No newline at end of file +garbage collection. + +feature.manyFiles:: + Enable config options that optimize for repos with many files in the + working directory. With many files, commands such as `git status` and + `git checkout` may be slow and these new defaults improve performance: ++ +* `index.version=4` enables path-prefix compression in the index. ++ +* `core.untrackedCache=true` enables the untracked cache. This setting assumes +that mtime is working on your machine. \ No newline at end of file diff --git a/Documentation/config/index.txt b/Documentation/config/index.txt index f181503041..7cb50b37e9 100644 --- a/Documentation/config/index.txt +++ b/Documentation/config/index.txt @@ -24,3 +24,4 @@ index.threads:: index.version:: Specify the version with which new index files should be initialized. This does not affect existing repositories. + If `feature.manyFiles` is enabled, then the default is 4. diff --git a/repo-settings.c b/repo-settings.c index 65b980403c..3a5cf33814 100644 --- a/repo-settings.c +++ b/repo-settings.c @@ -43,6 +43,10 @@ void prepare_repo_settings(struct repository *r) UPDATE_DEFAULT(r->settings.core_commit_graph, 1); UPDATE_DEFAULT(r->settings.gc_write_commit_graph, 1); } + if (!repo_config_get_bool(r, "feature.manyfiles", &value) && value) { + UPDATE_DEFAULT(r->settings.index_version, 4); + UPDATE_DEFAULT(r->settings.core_untracked_cache, UNTRACKED_CACHE_WRITE); + } /* Hack for test programs like test-dump-untracked-cache */ if (ignore_untracked_cache_config) diff --git a/t/t1600-index.sh b/t/t1600-index.sh index 42962ed7d4..c77721b580 100755 --- a/t/t1600-index.sh +++ b/t/t1600-index.sh @@ -59,17 +59,38 @@ test_expect_success 'out of bounds index.version issues warning' ' ) ' -test_expect_success 'GIT_INDEX_VERSION takes precedence over config' ' +test_index_version () { + INDEX_VERSION_CONFIG=$1 && + FEATURE_MANY_FILES=$2 && + ENV_VAR_VERSION=$3 + EXPECTED_OUTPUT_VERSION=$4 && ( rm -f .git/index && - GIT_INDEX_VERSION=4 && - export GIT_INDEX_VERSION && - git config --add index.version 2 && + rm -f .git/config && + if test "$INDEX_VERSION_CONFIG" -ne 0 + then + git config --add index.version $INDEX_VERSION_CONFIG + fi && + git config --add feature.manyFiles $FEATURE_MANY_FILES + if test "$ENV_VAR_VERSION" -ne 0 + then + GIT_INDEX_VERSION=$ENV_VAR_VERSION && + export GIT_INDEX_VERSION + else + unset GIT_INDEX_VERSION + fi && git add a 2>&1 && - echo 4 >expect && + echo $EXPECTED_OUTPUT_VERSION >expect && test-tool index-version <.git/index >actual && test_cmp expect actual ) +} + +test_expect_success 'index version config precedence' ' + test_index_version 2 false 4 4 && + test_index_version 2 true 0 2 && + test_index_version 0 true 0 4 && + test_index_version 0 true 2 2 ' test_done From patchwork Thu Jul 25 02:23:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kazuhiro Kato via GitGitGadget X-Patchwork-Id: 11057823 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 78E11746 for ; Thu, 25 Jul 2019 02:23:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 639D928979 for ; Thu, 25 Jul 2019 02:23:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 57C6B2897C; Thu, 25 Jul 2019 02:23:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F9892897B for ; Thu, 25 Jul 2019 02:23:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389037AbfGYCXX (ORCPT ); Wed, 24 Jul 2019 22:23:23 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:33894 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388139AbfGYCXV (ORCPT ); Wed, 24 Jul 2019 22:23:21 -0400 Received: by mail-wr1-f68.google.com with SMTP id 31so48994293wrm.1 for ; Wed, 24 Jul 2019 19:23:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ap/SUs9HcudhvxxU9tooEfSrAL5PHr5m8VIZLFfV3vE=; b=c6DaEc0RSWIVFUD/hAzoa0bRgSdzP1MXgDHOBHn+8/+JTat0VCMIp/wqjuD9papbRV Zp/05NGehh7U5kAWStbsT/B5zWjhMeqFwhwizQDpMuHc+wQmeqBE0Jx2LesMe4b3qs3g lFIih2NprNkiwRDJ/cCLvm1zicnKfhG7B9F0FIgEJR8BJuWJ06f3PlR2WS/qbSKASM31 TjcAffRlSedhJuZB1MVSPrGV4PeIg+QB2/o3r1gbcgGdfvHIvSRbJr4Rve88PgCClRyH bKMjyp7Sc+f+HT8PxDjD61gmMIMH4kGWUNFcw0GroitlNk6M+lw1kW2uvWc8DDLBzi5m uiHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ap/SUs9HcudhvxxU9tooEfSrAL5PHr5m8VIZLFfV3vE=; b=h2H5sHIXR35ivjStHZqXbrQ3v+boWyIqlewFILUomUC6thJ7jAqoJ+JX7IIzn5QV8l 82BwB5SIrgR8osGuXug+Y34kckzwYEiHEwakyzxcQcImdg7Rf9dHR/marG1nBJ8ElHlC fMXcLjPcfUa1RCHecYIz2RSuBRXBkxMDKA7sSmkOqSVwfyULDDvMvrpK2BIpSXfKnOXg nAB3fplXgBLbKLlARZmlCYZJ1WCHrOokUOHGP02xnlCXr27ZGTyqcgjOtvYJqrgtAZvQ hWTmc/xkm6jSwFXHimseimkDKfEiA7tgQ75LbBZQHcM8ALuTq+PtREBt79TfzB57bYZP hIrw== X-Gm-Message-State: APjAAAXf9tlK28SHlkSu5/eKYZ+G0EH/lvBwzH5MNujXqLUKDLlNw/Yx XRtslDNAhwnCLYzjKI+b+ZissIac X-Google-Smtp-Source: APXvYqwFmI0Vmf7+vlTaQgv3ogaONEtJJaLMgjGaASZ1wpDlserOgmQ/iGj9TDV3vOs0rZVeDtiP/Q== X-Received: by 2002:adf:b612:: with SMTP id f18mr80244405wre.97.1564021398509; Wed, 24 Jul 2019 19:23:18 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id a67sm50759262wmh.40.2019.07.24.19.23.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Jul 2019 19:23:18 -0700 (PDT) Date: Wed, 24 Jul 2019 19:23:18 -0700 (PDT) X-Google-Original-Date: Thu, 25 Jul 2019 02:23:12 GMT Message-Id: In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 5/5] repo-settings: create feature.experimental setting Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes.Schindelin@gmx.de, peff@peff.net, jnareb@gmail.com, pclouds@gmail.com, carenas@gmail.com, avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee The 'feature.experimental' setting includes config options that are not committed to become defaults, but could use additional testing. Update the following config settings to take new defaults, and to use the repo_settings struct if not already using them: * 'pack.useSparse=true' * 'merge.directoryRenames=true' * 'fetch.negotiationAlgorithm=skipping' In the case of fetch.negotiationAlgorithm, the existing logic would load the config option only when about to use the setting, so had a die() statement on an unknown string value. This is removed as now the config is parsed under prepare_repo_settings(). In general, this die() is probably misplaced and not valuable. A test was removed that checked this die() statement executed. Signed-off-by: Derrick Stolee --- Documentation/config/feature.txt | 17 +++++++++++++++++ Documentation/config/fetch.txt | 3 ++- Documentation/config/merge.txt | 3 ++- Documentation/config/pack.txt | 3 ++- builtin/am.c | 1 + fetch-negotiator.c | 26 ++++++++++++++------------ fetch-negotiator.h | 5 +++-- fetch-pack.c | 11 +++++------ merge-recursive.c | 15 ++++++--------- repo-settings.c | 20 ++++++++++++++++++++ repo-settings.h | 16 ++++++++++++++++ t/t5552-skipping-fetch-negotiator.sh | 23 ----------------------- 12 files changed, 88 insertions(+), 55 deletions(-) diff --git a/Documentation/config/feature.txt b/Documentation/config/feature.txt index c2d9ef7473..287621b1e8 100644 --- a/Documentation/config/feature.txt +++ b/Documentation/config/feature.txt @@ -4,6 +4,23 @@ feature.*:: developer community as recommended defaults and are subject to change. In particular, new config options may be added with different defaults. +feature.experimental:: + Enable config options that are new to Git, and are being considered for + future defaults. Config settings included here may be added or removed + with each release, including minor version updates. These settings may + have unintended interactions since they are so new. Please enable this + setting if you are interested in providing feedback on experimental + features. The new default values are: ++ +* `pack.useSparse=true` uses a new algorithm when constructing a pack-file +which can improve `git push` performance in repos with many files. ++ +* `merge.directoryRenames=true` uses a new algorithm for detecting renames by +using entire directories at a time instead of single files at a time. ++ +* `fetch.negotiationAlgorithm=skipping` may improve fetch negotiation times by +skipping more commits at a time, reducing the number of round trips. + feature.manyCommits:: Enable config options that optimize for repos with many commits. This setting is recommended for repos with at least 100,000 commits. The diff --git a/Documentation/config/fetch.txt b/Documentation/config/fetch.txt index ba890b5884..d402110638 100644 --- a/Documentation/config/fetch.txt +++ b/Documentation/config/fetch.txt @@ -59,7 +59,8 @@ fetch.negotiationAlgorithm:: effort to converge faster, but may result in a larger-than-necessary packfile; The default is "default" which instructs Git to use the default algorithm that never skips commits (unless the server has acknowledged it or one - of its descendants). + of its descendants). If `feature.experimental` is enabled, then this + setting defaults to "skipping". Unknown values will cause 'git fetch' to error out. + See also the `--negotiation-tip` option for linkgit:git-fetch[1]. diff --git a/Documentation/config/merge.txt b/Documentation/config/merge.txt index 6a313937f8..e8def2d63c 100644 --- a/Documentation/config/merge.txt +++ b/Documentation/config/merge.txt @@ -54,7 +54,8 @@ merge.directoryRenames:: moved into the new directory. If set to "conflict", a conflict will be reported for such paths. If merge.renames is false, merge.directoryRenames is ignored and treated as false. Defaults - to "conflict". + to "conflict" unless `feature.experimental` is enabled and the + default is "true". merge.renormalize:: Tell Git that canonical representation of files in the diff --git a/Documentation/config/pack.txt b/Documentation/config/pack.txt index 9cdcfa7324..1d66f0c992 100644 --- a/Documentation/config/pack.txt +++ b/Documentation/config/pack.txt @@ -112,7 +112,8 @@ pack.useSparse:: objects. This can have significant performance benefits when computing a pack to send a small change. However, it is possible that extra objects are added to the pack-file if the included - commits contain certain types of direct renames. + commits contain certain types of direct renames. Default is `false` + unless `feature.experimental` is enabled. pack.writeBitmaps (deprecated):: This is a deprecated synonym for `repack.writeBitmaps`. diff --git a/builtin/am.c b/builtin/am.c index 1aea657a7f..912c6155b8 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -34,6 +34,7 @@ #include "string-list.h" #include "packfile.h" #include "repository.h" +#include "repo-settings.h" /** * Returns the length of the first line of msg. diff --git a/fetch-negotiator.c b/fetch-negotiator.c index d6d685cba0..1a68be1a2e 100644 --- a/fetch-negotiator.c +++ b/fetch-negotiator.c @@ -2,19 +2,21 @@ #include "fetch-negotiator.h" #include "negotiator/default.h" #include "negotiator/skipping.h" +#include "repository.h" +#include "repo-settings.h" -void fetch_negotiator_init(struct fetch_negotiator *negotiator, - const char *algorithm) +void fetch_negotiator_init(struct repository *r, + struct fetch_negotiator *negotiator) { - if (algorithm) { - if (!strcmp(algorithm, "skipping")) { - skipping_negotiator_init(negotiator); - return; - } else if (!strcmp(algorithm, "default")) { - /* Fall through to default initialization */ - } else { - die("unknown fetch negotiation algorithm '%s'", algorithm); - } + prepare_repo_settings(r); + switch(r->settings.fetch_negotiation_algorithm) { + case FETCH_NEGOTIATION_SKIPPING: + skipping_negotiator_init(negotiator); + return; + + case FETCH_NEGOTIATION_DEFAULT: + default: + default_negotiator_init(negotiator); + return; } - default_negotiator_init(negotiator); } diff --git a/fetch-negotiator.h b/fetch-negotiator.h index 9e3967ce66..ea78868504 100644 --- a/fetch-negotiator.h +++ b/fetch-negotiator.h @@ -2,6 +2,7 @@ #define FETCH_NEGOTIATOR_H struct commit; +struct repository; /* * An object that supplies the information needed to negotiate the contents of @@ -52,7 +53,7 @@ struct fetch_negotiator { void *data; }; -void fetch_negotiator_init(struct fetch_negotiator *negotiator, - const char *algorithm); +void fetch_negotiator_init(struct repository *r, + struct fetch_negotiator *negotiator); #endif diff --git a/fetch-pack.c b/fetch-pack.c index 65be043f2a..d81f47c07b 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -36,7 +36,6 @@ static int agent_supported; static int server_supports_filtering; static struct lock_file shallow_lock; static const char *alternate_shallow_file; -static char *negotiation_algorithm; static struct strbuf fsck_msg_types = STRBUF_INIT; /* Remember to update object flag allocation in object.h */ @@ -892,12 +891,13 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args, struct shallow_info *si, char **pack_lockfile) { + struct repository *r = the_repository; struct ref *ref = copy_ref_list(orig_ref); struct object_id oid; const char *agent_feature; int agent_len; struct fetch_negotiator negotiator; - fetch_negotiator_init(&negotiator, negotiation_algorithm); + fetch_negotiator_init(r, &negotiator); sort_ref_list(&ref, ref_compare_name); QSORT(sought, nr_sought, cmp_ref_by_name); @@ -911,7 +911,7 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args, if (server_supports("shallow")) print_verbose(args, _("Server supports %s"), "shallow"); - else if (args->depth > 0 || is_repository_shallow(the_repository)) + else if (args->depth > 0 || is_repository_shallow(r)) die(_("Server does not support shallow clients")); if (args->depth > 0 || args->deepen_since || args->deepen_not) args->deepen = 1; @@ -1379,6 +1379,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args, struct shallow_info *si, char **pack_lockfile) { + struct repository *r = the_repository; struct ref *ref = copy_ref_list(orig_ref); enum fetch_state state = FETCH_CHECK_LOCAL; struct oidset common = OIDSET_INIT; @@ -1386,7 +1387,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args, int in_vain = 0; int haves_to_send = INITIAL_FLUSH; struct fetch_negotiator negotiator; - fetch_negotiator_init(&negotiator, negotiation_algorithm); + fetch_negotiator_init(r, &negotiator); packet_reader_init(&reader, fd[0], NULL, 0, PACKET_READ_CHOMP_NEWLINE | PACKET_READ_DIE_ON_ERR_PACKET); @@ -1505,8 +1506,6 @@ static void fetch_pack_config(void) git_config_get_bool("repack.usedeltabaseoffset", &prefer_ofs_delta); git_config_get_bool("fetch.fsckobjects", &fetch_fsck_objects); git_config_get_bool("transfer.fsckobjects", &transfer_fsck_objects); - git_config_get_string("fetch.negotiationalgorithm", - &negotiation_algorithm); git_config(fetch_pack_config_cb, NULL); } diff --git a/merge-recursive.c b/merge-recursive.c index 12300131fc..f39c31c9c4 100644 --- a/merge-recursive.c +++ b/merge-recursive.c @@ -28,6 +28,7 @@ #include "submodule.h" #include "revision.h" #include "commit-reach.h" +#include "repo-settings.h" struct path_hashmap_entry { struct hashmap_entry e; @@ -3662,15 +3663,6 @@ static void merge_recursive_config(struct merge_options *opt) opt->merge_detect_rename = git_config_rename("merge.renames", value); free(value); } - if (!git_config_get_string("merge.directoryrenames", &value)) { - int boolval = git_parse_maybe_bool(value); - if (0 <= boolval) { - opt->detect_directory_renames = boolval ? 2 : 0; - } else if (!strcasecmp(value, "conflict")) { - opt->detect_directory_renames = 1; - } /* avoid erroring on values from future versions of git */ - free(value); - } git_config(git_xmerge_config, NULL); } @@ -3688,6 +3680,11 @@ void init_merge_options(struct merge_options *opt, opt->diff_detect_rename = -1; opt->merge_detect_rename = -1; opt->detect_directory_renames = 1; + + prepare_repo_settings(repo); + if (repo->settings.merge_directory_renames >= 0) + opt->detect_directory_renames = repo->settings.merge_directory_renames; + merge_recursive_config(opt); merge_verbosity = getenv("GIT_MERGE_VERBOSITY"); if (merge_verbosity) diff --git a/repo-settings.c b/repo-settings.c index 3a5cf33814..bfb3288fda 100644 --- a/repo-settings.c +++ b/repo-settings.c @@ -35,6 +35,18 @@ void prepare_repo_settings(struct repository *r) free(strval); } + if (!repo_config_get_maybe_bool(r, "merge.directoryrenames", &value)) + r->settings.merge_directory_renames = value ? MERGE_DIRECTORY_RENAMES_TRUE : 0; + else if (!repo_config_get_string(r, "merge.directoryrenames", &strval)) { + if (!strcasecmp(strval, "conflict")) + r->settings.merge_directory_renames = MERGE_DIRECTORY_RENAMES_CONFLICT; + } + if (!repo_config_get_string(r, "fetch.negotiationalgorithm", &strval)) { + if (!strcasecmp(strval, "skipping")) + r->settings.fetch_negotiation_algorithm = FETCH_NEGOTIATION_SKIPPING; + else + r->settings.fetch_negotiation_algorithm = FETCH_NEGOTIATION_DEFAULT; + } if (!repo_config_get_bool(r, "pack.usesparse", &value)) r->settings.pack_use_sparse = value; @@ -47,6 +59,11 @@ void prepare_repo_settings(struct repository *r) UPDATE_DEFAULT(r->settings.index_version, 4); UPDATE_DEFAULT(r->settings.core_untracked_cache, UNTRACKED_CACHE_WRITE); } + if (!repo_config_get_bool(r, "feature.experimental", &value) && value) { + UPDATE_DEFAULT(r->settings.pack_use_sparse, 1); + UPDATE_DEFAULT(r->settings.merge_directory_renames, MERGE_DIRECTORY_RENAMES_TRUE); + UPDATE_DEFAULT(r->settings.fetch_negotiation_algorithm, FETCH_NEGOTIATION_SKIPPING); + } /* Hack for test programs like test-dump-untracked-cache */ if (ignore_untracked_cache_config) @@ -54,5 +71,8 @@ void prepare_repo_settings(struct repository *r) else UPDATE_DEFAULT(r->settings.core_untracked_cache, UNTRACKED_CACHE_KEEP); + UPDATE_DEFAULT(r->settings.merge_directory_renames, MERGE_DIRECTORY_RENAMES_CONFLICT); + UPDATE_DEFAULT(r->settings.fetch_negotiation_algorithm, FETCH_NEGOTIATION_DEFAULT); + r->settings_initialized = 1; } diff --git a/repo-settings.h b/repo-settings.h index e9b028e1a9..95d3392df2 100644 --- a/repo-settings.h +++ b/repo-settings.h @@ -8,6 +8,20 @@ enum untracked_cache_setting { UNTRACKED_CACHE_WRITE = 2 }; +enum merge_directory_renames_setting { + MERGE_DIRECTORY_RENAMES_UNSET = -1, + MERGE_DIRECTORY_RENAMES_NONE = 0, + MERGE_DIRECTORY_RENAMES_CONFLICT = 1, + MERGE_DIRECTORY_RENAMES_TRUE = 2, +}; + +enum fetch_negotiation_setting { + FETCH_NEGOTIATION_UNSET = -1, + FETCH_NEGOTIATION_NONE = 0, + FETCH_NEGOTIATION_DEFAULT = 1, + FETCH_NEGOTIATION_SKIPPING = 2, +}; + struct repo_settings { int core_commit_graph; int gc_write_commit_graph; @@ -16,6 +30,8 @@ struct repo_settings { enum untracked_cache_setting core_untracked_cache; int pack_use_sparse; + enum merge_directory_renames_setting merge_directory_renames; + enum fetch_negotiation_setting fetch_negotiation_algorithm; }; struct repository; diff --git a/t/t5552-skipping-fetch-negotiator.sh b/t/t5552-skipping-fetch-negotiator.sh index 8a14be51a1..f70cbcc9ca 100755 --- a/t/t5552-skipping-fetch-negotiator.sh +++ b/t/t5552-skipping-fetch-negotiator.sh @@ -60,29 +60,6 @@ test_expect_success 'commits with no parents are sent regardless of skip distanc have_not_sent c6 c4 c3 ' -test_expect_success 'unknown fetch.negotiationAlgorithm values error out' ' - rm -rf server client trace && - git init server && - test_commit -C server to_fetch && - - git init client && - test_commit -C client on_client && - git -C client checkout on_client && - - test_config -C client fetch.negotiationAlgorithm invalid && - test_must_fail git -C client fetch "$(pwd)/server" 2>err && - test_i18ngrep "unknown fetch negotiation algorithm" err && - - # Explicit "default" value - test_config -C client fetch.negotiationAlgorithm default && - git -C client -c fetch.negotiationAlgorithm=default fetch "$(pwd)/server" && - - # Implementation detail: If there is nothing to fetch, we will not error out - test_config -C client fetch.negotiationAlgorithm invalid && - git -C client fetch "$(pwd)/server" 2>err && - test_i18ngrep ! "unknown fetch negotiation algorithm" err -' - test_expect_success 'when two skips collide, favor the larger one' ' rm -rf server client trace && git init server &&