From patchwork Fri Mar 15 15:59:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10855091 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CB7C417E6 for ; Fri, 15 Mar 2019 16:00:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B65022AB05 for ; Fri, 15 Mar 2019 16:00:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B47A22AB02; Fri, 15 Mar 2019 16:00:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F3072AB0C for ; Fri, 15 Mar 2019 16:00:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729417AbfCOQAO (ORCPT ); Fri, 15 Mar 2019 12:00:14 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:40463 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727451AbfCOQAN (ORCPT ); Fri, 15 Mar 2019 12:00:13 -0400 Received: by mail-wr1-f65.google.com with SMTP id t5so10138060wri.7 for ; Fri, 15 Mar 2019 09:00:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Rlx085tm1Uq4Vf2N3ei8v+BJfaqhpZ1jcSkPOmHfrrM=; b=kDD60LtU0+16ThvUkyC/OFCEaen2lP5qvOtJjGnDR7semIxW2RmYz7YjhCoSslXVTl 7L23GCQ20fVPawljiUw+MjDNvtNQF1wVgikkBwPWxkL3+ltrSB6/cSp4HDzfGvbZB6SF ZLe8IBSLOMAOnkAGq5VJUxTdyOYgMpgxpQZfnIsmUKVtUc8709h/94vIETPyoxch2CCj bLoDq/PRYhAphVfWgfEgNuw9mIRW+DPLRHOHQjBB9eRv+wjo9SMYtkuQF7hhwPVM7S0N pQBi/Pr2o7VZ6CFZHssDmTLJneyb7BWL7UTHmM+v/0xrohz9UbvNpsRpiyMQwb4WAu3K 1FVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Rlx085tm1Uq4Vf2N3ei8v+BJfaqhpZ1jcSkPOmHfrrM=; b=gHBzk7jH9AJewKA4Y4fEFwGY/4L0fsgyVzGbFeluDYw6bzLedDJEQVpTrBaD4kPbKJ L5bUYowHkuBmOp6ZHAEDH3o0zBdcecz+UZly51vyO3mlSWJhf40CGhBdhKiPocHHJyO1 bo9z6FQtGeYIM3nExq0H9ywVk/0YzjMPs6gKt5QkT+H2APXsJJZBWY0/ebNOfdrgSgow NnRUJ2YfmrTrOxmSxXvHOntW71VPHETMGmfztZGeW1TEBUTU54E3lImk4/zwvb+lIEPH WhcLOKQ4O7CkLLuifke6ItmPnMUYoh4VfvNhUYDsPoQermcK7xoARoqcoxcP6/lbLosw xO/g== X-Gm-Message-State: APjAAAWUof36l4vw0JhnOBWnJUVxxY/yHyn2I0UBhd2qVm9s93nrvtEd GxPwgksSzLLLJ6AJyyZCgEvGVJQHFwg= X-Google-Smtp-Source: APXvYqyPhYP8lz1L08BTpqsvGXziyKne9KcDpofpXTl9tuWKcTAKq/aRL5AuZf9ioyWY4SRFo4/UAA== X-Received: by 2002:adf:ffc2:: with SMTP id x2mr2809914wrs.157.1552665611460; Fri, 15 Mar 2019 09:00:11 -0700 (PDT) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id a18sm467766wmm.14.2019.03.15.09.00.10 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 15 Mar 2019 09:00:10 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Jeff King , Michael Haggerty , Stefan Beller , Jonathan Nieder , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 1/8] gc: remove redundant check for gc_auto_threshold Date: Fri, 15 Mar 2019 16:59:52 +0100 Message-Id: <20190315155959.12390-2-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.360.g471c308f928 In-Reply-To: <20190314123439.4347-1-avarab@gmail.com> References: <20190314123439.4347-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Checking gc_auto_threshold in too_many_loose_objects() was added in 17815501a8 ("git-gc --auto: run "repack -A -d -l" as necessary.", 2007-09-17) when need_to_gc() itself was also reliant on gc_auto_pack_limit before its early return: gc_auto_threshold <= 0 && gc_auto_pack_limit <= 0 When that check was simplified to just checking "gc_auto_threshold <= 0" in b14d255ba8 ("builtin-gc.c: allow disabling all auto-gc'ing by assigning 0 to gc.auto", 2008-03-19) this unreachable code should have been removed. We only call too_many_loose_objects() from within need_to_gc() itself, which will return if this condition holds, and in cmd_gc() which will return before ever getting to "auto_gc && too_many_loose_objects()" if "auto_gc && !need_to_gc()" is true earlier in the function. Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/gc.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/builtin/gc.c b/builtin/gc.c index 020f725acc4..8c2312681ce 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -157,9 +157,6 @@ static int too_many_loose_objects(void) int num_loose = 0; int needed = 0; - if (gc_auto_threshold <= 0) - return 0; - dir = opendir(git_path("objects/17")); if (!dir) return 0; From patchwork Fri Mar 15 15:59:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10855093 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5EE321515 for ; Fri, 15 Mar 2019 16:00:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 47CE72AB0E for ; Fri, 15 Mar 2019 16:00:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 45EF42AB1C; Fri, 15 Mar 2019 16:00:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C95902AB1F for ; Fri, 15 Mar 2019 16:00:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729427AbfCOQAQ (ORCPT ); Fri, 15 Mar 2019 12:00:16 -0400 Received: from mail-wm1-f42.google.com ([209.85.128.42]:35910 "EHLO mail-wm1-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727451AbfCOQAQ (ORCPT ); Fri, 15 Mar 2019 12:00:16 -0400 Received: by mail-wm1-f42.google.com with SMTP id e16so6626134wme.1 for ; Fri, 15 Mar 2019 09:00:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=O2CFz7V9Qay5xspCJr+iwzVrnflA4NTgAyx9dGsuJlo=; b=J4j0gBB4FRDXV1rcZOcISvHSzwMZy72yCJcoAcQd8LFhZHIzrPpEKJ1QoXaR9x4uYD zfKvrq5TSi9P0GlbftwC2t9be2dpD6K0e5EsV4J+qh736iKO4REX3c2p2bep+UjDKMOV AeEdy+PwEj3ZvFBZzrHYQDa1lp4Ei825mKdPzckTxQ7VYqekPdoaLqgsBPKSXWwSyT2i JH4DYI3G0fMGmrshKrK15JKOuSJtslLUC8TUkNggndwbtFiKPrTLydRIufdU90y+Ug66 BcRhrOPqrhHGdQSgrm3HVvcB21eiTeB3Jrz9Gy+XPiFG6SrH7Yo7DrnxCazei5v5ntDd 2N/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=O2CFz7V9Qay5xspCJr+iwzVrnflA4NTgAyx9dGsuJlo=; b=Kkh3W9xRewFs7t8jLngCNBPnU3fPrbqfQd38BAcRjeyudlqv4fpl7FczC+S7wPSH/S YAV4Js35d+9iJmYcHt4nwCliay1OooRCVDAYNjtBNAW0trUR7wgoGNI93PzAycph3ce6 ZgR9HksgSuI41RGWafT5fWq/h3jwtl0hsYELcrONiDCWU89vE8MjVg7gbRGGJxtHDCvE ray7T2Em2uZ55qmKsfM9ZsjlhySeUVvtVIc5FMo1QMYxW/QsDgEBRKTp0pVu69eFf8Tu NLotWtr7XhXaZxskNoN+sutrB4rupxPl8xRq47xIEXdPrCrRrOR6M2wE3VGNjjHhbZ0T bJbw== X-Gm-Message-State: APjAAAWd1wncqoJfQftXSyatLyuRtXjDwYSqr35tIojE9aPB7BdJ1/eJ 3zz216Zk1IZ8vi0Pl0q9IMm+z59b+UU= X-Google-Smtp-Source: APXvYqyFsobwbXJaHwNeri2XnsQxyJinqYtIfYxpXEDUZU0CSj+uqkNKbr5WesN2bYV5aMwzBufjEQ== X-Received: by 2002:a05:600c:22d9:: with SMTP id 25mr2512857wmg.59.1552665613346; Fri, 15 Mar 2019 09:00:13 -0700 (PDT) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id a18sm467766wmm.14.2019.03.15.09.00.12 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 15 Mar 2019 09:00:12 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Jeff King , Michael Haggerty , Stefan Beller , Jonathan Nieder , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 2/8] gc: convert to using the_hash_algo Date: Fri, 15 Mar 2019 16:59:53 +0100 Message-Id: <20190315155959.12390-3-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.360.g471c308f928 In-Reply-To: <20190314123439.4347-1-avarab@gmail.com> References: <20190314123439.4347-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP There's been a lot of changing of the hardcoded "40" values to the_hash_algo->hexsz, but we've so far missed this one where we hardcoded 38 for the loose object file length. This is because a SHA-1 like abcde[...] gets turned into objects/ab/cde[...]. There's no reason to suppose the same won't be the case for SHA-256, and reading between the lines in hash-function-transition.txt the format is planned to be the same. In the future we may want to further modify this code for the hash function transition. There's a potential pathological case here where we'll only consider the loose objects for the currently active hash, but objects for that hash will share a directory storage with the other hash. Thus we could theoretically have e.g. 1k SHA-1 loose objects, and 1 million SHA-256 objects. Then not notice that we need to pack them because we're currently using SHA-1, even though our FS may be straining under the stress of such humongous directories. So assuming that "gc" eventually learns to pack up both SHA-1 and SHA-256 objects regardless of what the current the_hash_algo is, perhaps this check should be changed to consider all files in objects/17/ matching [0-9a-f] 38 or 62 characters in length (i.e. both SHA-1 and SHA-256). But none of that is something we need to worry about now, and supporting both 38 and 62 characters depending on "the_hash_algo" removes another case of SHA-1 hardcoding. As noted in [1] I'm making no effort to somehow remove the hardcoding for "2" as in "use the first two hexdigits for the directory name". There's no indication that that'll ever change, and somehow generalizing it here would be a drop in the ocean, so there's no point in doing that. It also couldn't be done without coming up with some generalized version of the magical "objects/17" directory. See [2] for a discussion of that directory. 1. https://public-inbox.org/git/874l84ber7.fsf@evledraar.gmail.com/ 2. https://public-inbox.org/git/87k1mta9x5.fsf@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/gc.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/builtin/gc.c b/builtin/gc.c index 8c2312681ce..733bd7bdf46 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -156,6 +156,7 @@ static int too_many_loose_objects(void) int auto_threshold; int num_loose = 0; int needed = 0; + const unsigned hexsz_loose = the_hash_algo->hexsz - 2; dir = opendir(git_path("objects/17")); if (!dir) @@ -163,8 +164,8 @@ static int too_many_loose_objects(void) auto_threshold = DIV_ROUND_UP(gc_auto_threshold, 256); while ((ent = readdir(dir)) != NULL) { - if (strspn(ent->d_name, "0123456789abcdef") != 38 || - ent->d_name[38] != '\0') + if (strspn(ent->d_name, "0123456789abcdef") != hexsz_loose || + ent->d_name[hexsz_loose] != '\0') continue; if (++num_loose > auto_threshold) { needed = 1; From patchwork Fri Mar 15 15:59:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10855095 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D14F1515 for ; Fri, 15 Mar 2019 16:00:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 77C3A2AB08 for ; Fri, 15 Mar 2019 16:00:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6B11E2AB11; Fri, 15 Mar 2019 16:00:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0BC7C2AB1D for ; Fri, 15 Mar 2019 16:00:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729433AbfCOQAT (ORCPT ); Fri, 15 Mar 2019 12:00:19 -0400 Received: from mail-wm1-f41.google.com ([209.85.128.41]:37023 "EHLO mail-wm1-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729406AbfCOQAR (ORCPT ); Fri, 15 Mar 2019 12:00:17 -0400 Received: by mail-wm1-f41.google.com with SMTP id x10so6541914wmg.2 for ; Fri, 15 Mar 2019 09:00:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UVDmBL8A1keR5cUjodPXRbcLV0Y5DAksOI6dDmLUqkI=; b=bdQm1RXqbiMPHpz8BhN6d1Bcn48gAo39r4Bkq0u7Q++VEpT6kXoKeuRxlg99cdMEjO HgGgsCvWw10qq0atgF2kNer3WewNV7UhnCRUQWI15SLtkSfQySQamm4X2PkfTv4aP6A4 2MFUaHSQiRDDqjR8aH7yXgSia5x3QUVm4Oamr0ejTB8UvL5C8EwFvaqQaLRZawVfCLiQ IFBQ76IDKyywzFA+EwqM2cZf5GjdKBaYY2xfMhRzm1w+kbmBAAitpH/worM9uN4fcxCF 6bfO1QgkH/awmjWIyrQ6O23a+cbsKfuJminVRLDNvvChgXzuEsQ/6M8c/6LAKBQfAXfe VoHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UVDmBL8A1keR5cUjodPXRbcLV0Y5DAksOI6dDmLUqkI=; b=ffL/U+bWy4NuhbLkpfvjoNvs1CvCHCEBCT060MJBGGWMlA8Bl6nRHx9OcQ9gn0ZeE/ 3rCFlt5qTkHWsM4iFojNQL3LtBcFQ5+zjA72wbLVCH9kauxUtW0QM9xQ1zSoo+8g+Kz4 AoYbD5qXxA/q58dCd+27RB/HN67DxLPQ/Ey4wR7mUZY+VJhACzXvlLo4jJl7k0zr0gS2 mw6pZ76iHBZFYVxVK2hy6RnsYb8qZdMH117mrSsX4jyfG1C0PRhO8j2J4CJ1sDQvwKFe 3IfLmkUXoyoavROOk57Hbc93oCQ+jxk6cHJU1aFhPNhhXToWAuwTTmSjGP/FaeqiSRYe U8YA== X-Gm-Message-State: APjAAAWLxuO+MllhtceTh7Rt8Nvb9qhOF2gv9eEO6Zx81y7x0NvGgw8M gu9qG23niQJdV6uFY9ZnlXOYgUw0+QA= X-Google-Smtp-Source: APXvYqwWOrvWVIVsDw/IDNpbqSVPNWkCFit0WFQVI31H7TXnL2grn2eQvX4mbYgx77YW16YlDnA/Dw== X-Received: by 2002:a1c:2dc9:: with SMTP id t192mr2555025wmt.119.1552665615015; Fri, 15 Mar 2019 09:00:15 -0700 (PDT) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id a18sm467766wmm.14.2019.03.15.09.00.13 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 15 Mar 2019 09:00:13 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Jeff King , Michael Haggerty , Stefan Beller , Jonathan Nieder , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 3/8] gc: refactor a "call me once" pattern Date: Fri, 15 Mar 2019 16:59:54 +0100 Message-Id: <20190315155959.12390-4-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.360.g471c308f928 In-Reply-To: <20190314123439.4347-1-avarab@gmail.com> References: <20190314123439.4347-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Change an idiom we're using to ensure that gc_before_repack() only does work once (see 62aad1849f ("gc --auto: do not lock refs in the background", 2014-05-25)) to be more obvious. Nothing except this function cares about the "pack_refs" and "prune_reflogs" variables, so let's not leave the reader wondering if they're being zero'd out for later use somewhere else. Signed-off-by: Jeff King Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/gc.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/builtin/gc.c b/builtin/gc.c index 733bd7bdf46..ae716a00d4a 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -489,14 +489,20 @@ static int report_last_gc_error(void) static void gc_before_repack(void) { + /* + * We may be called twice, as both the pre- and + * post-daemonized phases will call us, but running these + * commands more than once is pointless and wasteful. + */ + static int done = 0; + if (done++) + return; + if (pack_refs && run_command_v_opt(pack_refs_cmd.argv, RUN_GIT_CMD)) die(FAILED_RUN, pack_refs_cmd.argv[0]); if (prune_reflogs && run_command_v_opt(reflog.argv, RUN_GIT_CMD)) die(FAILED_RUN, reflog.argv[0]); - - pack_refs = 0; - prune_reflogs = 0; } int cmd_gc(int argc, const char **argv, const char *prefix) From patchwork Fri Mar 15 15:59:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10855097 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A2E371575 for ; Fri, 15 Mar 2019 16:00:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DC922AB0C for ; Fri, 15 Mar 2019 16:00:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8B0A02AB19; Fri, 15 Mar 2019 16:00:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3B3B72AB12 for ; Fri, 15 Mar 2019 16:00:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729439AbfCOQAT (ORCPT ); Fri, 15 Mar 2019 12:00:19 -0400 Received: from mail-wr1-f46.google.com ([209.85.221.46]:39266 "EHLO mail-wr1-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727451AbfCOQAS (ORCPT ); Fri, 15 Mar 2019 12:00:18 -0400 Received: by mail-wr1-f46.google.com with SMTP id p8so10135226wrq.6 for ; Fri, 15 Mar 2019 09:00:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IFjkF2LYNPVVbotv74/vDyvYfDtYa/Y3+GF/BTRGOVw=; b=KGZHEnJP6P1JAXdHtI1M/zPQSpx6aaN/sVHiPkuKJSXOG/LUCzfsu9wS47+BDmdcSg 7503t4QvHb5QFmr+PyVLTI/WloYuG78CcPaDyvzY8rvhddRI/j0E4gSMHt4C/kj3CCcm ZMFrK7LSUPeM0AbyiC5n9OTCxsPZB4bwNlUrpBI8zROMtIYGD5XQl77oA0HuPQFU7HkO CE+LnAge5m9N2mchnwG+QJ5xyqcE/px5iDQnTo9QOwRAel0Jmot2dIevvp2eSE9D/96d 0oB0J85U4PycGOJiesXEdIA9JSS81THARstXAtRElzQiX0JCtN9a9Qut4vPwg0H0go/r L5ZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IFjkF2LYNPVVbotv74/vDyvYfDtYa/Y3+GF/BTRGOVw=; b=OpjWZJWULHniENhMTriQC3ufsqIZXOOtXt5Rh6NUu3QvzFknXbwW1Fx2y3HMAVAnFy KntUq8r3Uh+UNMq3cO/MBehz1WI/ANI0+UjPuEDSXHMlC14hnvbn7yQa6UosWgYvrKyD tq0MlB6T0HoGRVB1EqmFQjHVareK09aqbASnlEZ1S/dMzhAg44GMeB80SPVbw6/6799s PzyCFDzkm5zDtJKf694buWx7uorRulAMKpoQppNU9dUlOydRP8pmdqHr3I+RIXGR9NRl 4cUM2wTnEJWTJospTfLUjmH5r0gFY5KpxrrbeEvQuXjtv+BYncvKcb0RY+qOARkfSVm2 5gCQ== X-Gm-Message-State: APjAAAXTtTgdlVbDhEyPSXjavM+b6DQtS5pPx5YGOzgADFUNSykwtP5k JkmTi4h+qci5sA8cVYVribuamSJZA70= X-Google-Smtp-Source: APXvYqx1gt2vfNCVcIJP8C2UfyLMZFST3qd21xcbq9dr2utFudJfbHfFAaniqb+z16xBdkfYQf/jig== X-Received: by 2002:a5d:538a:: with SMTP id d10mr2940606wrv.121.1552665616433; Fri, 15 Mar 2019 09:00:16 -0700 (PDT) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id a18sm467766wmm.14.2019.03.15.09.00.15 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 15 Mar 2019 09:00:15 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Jeff King , Michael Haggerty , Stefan Beller , Jonathan Nieder , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 4/8] reflog tests: make use of "test_config" idiom Date: Fri, 15 Mar 2019 16:59:55 +0100 Message-Id: <20190315155959.12390-5-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.360.g471c308f928 In-Reply-To: <20190314123439.4347-1-avarab@gmail.com> References: <20190314123439.4347-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Change a couple of tests that weren't using the helper to use it. This makes the trailing "--unset" unnecessary. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1410-reflog.sh | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/t/t1410-reflog.sh b/t/t1410-reflog.sh index ae8a448e343..42f5ac9ed95 100755 --- a/t/t1410-reflog.sh +++ b/t/t1410-reflog.sh @@ -232,24 +232,21 @@ test_expect_success '--expire=never' ' ' test_expect_success 'gc.reflogexpire=never' ' + test_config gc.reflogexpire never && + test_config gc.reflogexpireunreachable never && - git config gc.reflogexpire never && - git config gc.reflogexpireunreachable never && git reflog expire --verbose --all && git reflog refs/heads/master >output && test_line_count = 4 output ' test_expect_success 'gc.reflogexpire=false' ' + test_config gc.reflogexpire false && + test_config gc.reflogexpireunreachable false && - git config gc.reflogexpire false && - git config gc.reflogexpireunreachable false && git reflog expire --verbose --all && git reflog refs/heads/master >output && - test_line_count = 4 output && - - git config --unset gc.reflogexpire && - git config --unset gc.reflogexpireunreachable + test_line_count = 4 output ' From patchwork Fri Mar 15 15:59:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10855099 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A015F1575 for ; Fri, 15 Mar 2019 16:00:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 86E162AB02 for ; Fri, 15 Mar 2019 16:00:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 852972AB1D; Fri, 15 Mar 2019 16:00:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1686A2AB02 for ; Fri, 15 Mar 2019 16:00:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729443AbfCOQAV (ORCPT ); Fri, 15 Mar 2019 12:00:21 -0400 Received: from mail-wr1-f47.google.com ([209.85.221.47]:44460 "EHLO mail-wr1-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729406AbfCOQAU (ORCPT ); Fri, 15 Mar 2019 12:00:20 -0400 Received: by mail-wr1-f47.google.com with SMTP id w2so10095497wrt.11 for ; Fri, 15 Mar 2019 09:00:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Sxg3A3VeqUoOLalA8W9tVQdvu/dmzw/RhmnRwu5BHC4=; b=VmE02Zxsds/IlxpkK+G2KVJPKMVtVA3OCSqvBjnxxadaHxqLjwNdxSYmsjV2BVXfMb bGkN3IzwyYhGHEcDH7n0+36V7fFaEuXHCg0tXUn42oGJw3KKev16ayWGyOT0tlwNpPlk bHRuSGOw+J9QrQi+XA/VEJEaKfaA0pA3uRvbwZcgJB7Nrz/yq0uWpGIuF3sEWTFlam4d 1TXh984Y90k4WR5D+V6OTX87ptCycoZJd154lWl6qGmmwtUS3WQHKjD0iFZz76TJfhD4 /W8wbfXMi73YqMdNcCpK7fEKS3DFQ4AAF/zbx4FfMsB7x5addTnbUdhCPGaTTtXohybR /o3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Sxg3A3VeqUoOLalA8W9tVQdvu/dmzw/RhmnRwu5BHC4=; b=B+YH2fe0y+eUERSJIiNM7oq4ui4uhGQC62R0qQZ1oBPaVD9QCMg0No/h+8iUpqRgjN WsHz8LWyMYKaH36iskiDaeleTc9NpvanPgZE84wRYQmDH4R41AkhpNRVfgOcdyhaxNQk i2MkJYwPDvKHS3HDDSJK6n1wSBLTfgnYh2zp3m0iSJOsFey0pIn+/ss95t4mS4aLIGec lncbpXKMLD+nECfm4EdLfW/QhA0yqCiD71JdLfPh6nJ/8hh2WnELqFWnnqgZOIisl1pD EKrPx+MOqd4Gts6Ba7m2Oa7zicbPkRUJf20Z6uJKvllpsykM8Z6M8W3cPhBJ2ZVItuDx dmaA== X-Gm-Message-State: APjAAAW1CRLLMiBohgDhw0edV0Z7HN8CB6NkXw9NSYXIPyupa7z729Tt 5ppZjQpzbI5YI8Ufe3hLyI/5mSe6aC8= X-Google-Smtp-Source: APXvYqwY+FX3T8snPIxRuHuR6AfwXYE6BVNeWUeJ3H2SwQvVIVjG48j30hrORJAKrgAbnn0oaY408w== X-Received: by 2002:adf:f68d:: with SMTP id v13mr1510960wrp.6.1552665617644; Fri, 15 Mar 2019 09:00:17 -0700 (PDT) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id a18sm467766wmm.14.2019.03.15.09.00.16 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 15 Mar 2019 09:00:16 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Jeff King , Michael Haggerty , Stefan Beller , Jonathan Nieder , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 5/8] reflog tests: test for the "points nowhere" warning Date: Fri, 15 Mar 2019 16:59:56 +0100 Message-Id: <20190315155959.12390-6-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.360.g471c308f928 In-Reply-To: <20190314123439.4347-1-avarab@gmail.com> References: <20190314123439.4347-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The "git reflog expire" command when given an unknown reference has since 4264dc15e1 ("git reflog expire", 2006-12-19) when this command was implemented emit an error, but this has never been tested for. Let's test for it, also under gc.reflogExpire{Unreachable,}=never in case a future change is tempted to take shortcuts in the presence of such config. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1410-reflog.sh | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/t/t1410-reflog.sh b/t/t1410-reflog.sh index 42f5ac9ed95..e8f8ac97856 100755 --- a/t/t1410-reflog.sh +++ b/t/t1410-reflog.sh @@ -250,6 +250,16 @@ test_expect_success 'gc.reflogexpire=false' ' ' +test_expect_success 'git reflog expire unknown reference' ' + test_config gc.reflogexpire never && + test_config gc.reflogexpireunreachable never && + + test_must_fail git reflog expire master@{123} 2>stderr && + test_i18ngrep "points nowhere" stderr && + test_must_fail git reflog expire does-not-exist 2>stderr && + test_i18ngrep "points nowhere" stderr +' + test_expect_success 'checkout should not delete log for packed ref' ' test $(git reflog master | wc -l) = 4 && git branch foo && From patchwork Fri Mar 15 15:59:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10855101 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5DF5E1515 for ; Fri, 15 Mar 2019 16:00:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 474A22AB12 for ; Fri, 15 Mar 2019 16:00:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 45AE82AB27; Fri, 15 Mar 2019 16:00:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CE28A2AB12 for ; Fri, 15 Mar 2019 16:00:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729445AbfCOQAW (ORCPT ); Fri, 15 Mar 2019 12:00:22 -0400 Received: from mail-wr1-f66.google.com ([209.85.221.66]:34015 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729440AbfCOQAV (ORCPT ); Fri, 15 Mar 2019 12:00:21 -0400 Received: by mail-wr1-f66.google.com with SMTP id k1so9633341wre.1 for ; Fri, 15 Mar 2019 09:00:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iNDIe43hB1TkJe54aKqtbwinyUGuaYGoBTvx1wB0Lpo=; b=leOamYuok3w0pb83Dmr1xuIOXiC3O7E4otNel8U7UfQK0VpVlinRIDkl3y9H1f/mXq GYjaFsPVyT4Aq0QG2EiLtHvfzxk9x0Is9hoo2CgxibB8sd2cd4EHb6niNZY+5ig3YZaA rd1abMUcCuPtWb5KVzN3yswcV58Lwnlw2jR/lyn1rIA59Uwf35exD62mp2+CB0qwhmLw mxCLWPhEs9HyiLRnWXGMZLA7QjnCc4dvEwKb49rhpfkJIGyM7peWJRjn86aF+8huU8Fs 8qmlfqwXvfG4vYc23W7jUPlzgO1mUyVjcBiVE69IiSRDjUXPcVpio2wl0vyVI1LyDQhc F99g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iNDIe43hB1TkJe54aKqtbwinyUGuaYGoBTvx1wB0Lpo=; b=UMrFUvgnwaW/WwIdR/eLqT5L54j7TZoRH1HbrTPmOtma1dylIHU+1YONgaBKufU6tT hD648p575PTa6znStPyBMcMYx6E0UjEdti/7kO4OXGbm6tEAzYAmtzSGbxjhCN19vE1r eAIg0RMmX26wLeJKPVmzhSHsVEeCK3/ryc8dsgbwEjfQ+L884uilCJFZ+9MB6Ymz7utk T/mX9FNDAYR+gV2reNiVerjLSRpdpf8DXD9XLazz3iU4UKzSPAtDknq45ynTOOjoDO3s i7tntTbzr6B4sqm0561Q7tC0QjX/wGgTkY6UWFuojpuIQlzkHVegf/abFE0t4c+rdPbM aAow== X-Gm-Message-State: APjAAAWfKl5PTt2mTL2+eG76mBqVkI7ikAeUn/i9onpnjqdTwz4tv1li 88/s5BnJkpncKADokwv3pCDFyoOqi90= X-Google-Smtp-Source: APXvYqxv7no4P1X8ZpypbAoKR9//rJVrgBUH+T9Bmb9JcMRazkl0LNsGB31V6TUsYv4gOB9sJxCrsQ== X-Received: by 2002:adf:9cc3:: with SMTP id h3mr2867852wre.47.1552665619145; Fri, 15 Mar 2019 09:00:19 -0700 (PDT) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id a18sm467766wmm.14.2019.03.15.09.00.17 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 15 Mar 2019 09:00:17 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Jeff King , Michael Haggerty , Stefan Beller , Jonathan Nieder , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 6/8] reflog tests: assert lack of early exit with expiry="never" Date: Fri, 15 Mar 2019 16:59:57 +0100 Message-Id: <20190315155959.12390-7-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.360.g471c308f928 In-Reply-To: <20190314123439.4347-1-avarab@gmail.com> References: <20190314123439.4347-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When gc.reflogExpire and gc.reflogExpireUnreachable are set to "never" and --stale-fix isn't in effect (covered by the first part of the "if" statement being modified here) we *could* exit early without pointlessly looping over all the reflogs. However, as an earlier change to add a test for the "points nowhere" warning shows even in such a mode we might want to print out a warning. So while it's conceivable to implement this, I don't think it's worth it. It's going to be too easy to inadvertently add some flag that'll make the expiry happen anyway, and even with "never" we'd like to see all the lines we're going to keep. So let's assert that we're going to loop over all the references even when this configuration is in effect. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1410-reflog.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/t/t1410-reflog.sh b/t/t1410-reflog.sh index e8f8ac97856..79f731db37c 100755 --- a/t/t1410-reflog.sh +++ b/t/t1410-reflog.sh @@ -235,7 +235,9 @@ test_expect_success 'gc.reflogexpire=never' ' test_config gc.reflogexpire never && test_config gc.reflogexpireunreachable never && - git reflog expire --verbose --all && + git reflog expire --verbose --all >output && + test_line_count = 9 output && + git reflog refs/heads/master >output && test_line_count = 4 output ' From patchwork Fri Mar 15 15:59:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10855105 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 857871575 for ; Fri, 15 Mar 2019 16:00:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6FF132AB25 for ; Fri, 15 Mar 2019 16:00:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6DC872AB09; Fri, 15 Mar 2019 16:00:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C28002AB38 for ; Fri, 15 Mar 2019 16:00:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729449AbfCOQA0 (ORCPT ); Fri, 15 Mar 2019 12:00:26 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:35268 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727451AbfCOQAX (ORCPT ); Fri, 15 Mar 2019 12:00:23 -0400 Received: by mail-wm1-f68.google.com with SMTP id y15so6638704wma.0 for ; Fri, 15 Mar 2019 09:00:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4ebkw8BRdYgl0xvfh6d7pSyKczD01KbunU+1N6IDEEQ=; b=pDBp037ZeV5VIE91ar+aPXoxDVp45XU4hCLPTxrLW13sj0LjNyvXYN6zmckAZxkSNB gokVkO84DSJSqDdbw+FLJIZH2XtR5EjIKbREsfn1kjvt18tQBfSrQ+GGg/UM8MuyDm2R sWTy8GEgS+fM73yUGujz4X27ro/2g1cSVgh2CXwJwCq2vuBPwpve90T8/pjeWOMfmiGY WwiTnvpdihqjXvuODpEflOk11rCujc/4MtJTqDyhCSRHUmFD5fm5qDnePbYbe/1rt8C9 tF116HxcBCQqCfs+oB4OYshONq3YjzCteOzvAWPosvfpKXCOta6GQzLsch3PIWnag3uj b2Fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4ebkw8BRdYgl0xvfh6d7pSyKczD01KbunU+1N6IDEEQ=; b=m/FHTBZTyFrmcHtEo55X8ElHwxm1eTV7Sgwg0qcd3JTkyHHqSdqS3oRw6htjBxPxzf fRv5G4O1Ybek4Z9I8+jM3yiGDtBjAU5DqY44OHKHNvJyDhnIUWiJDe1o6qNQ3mEh2F+x uImg1UYx2GVSgE87BKyilDAAnDcrO0TH17ekXMvAt5iruWthQmaStHkWSx7EyNrPY/Pp Ta37qEptI6wzDPwII2nx/bZqLrOXl7PF6dLRuC/SK98zbR4eJFY+KV5xj8r7W+pmQ2+y m9MPD6CcimidcHoMGUl00wrq9WfB+qppuCHvwT9qyGwlHAKlnc1YVMC/V0Zo7zNpF0xk KSxQ== X-Gm-Message-State: APjAAAUA1Zxq0SHiZYj5li3qKY7d5ZW14QqIp1XSiS65hu18ZWuuQ9Fo vctj1QSJ+ZeBAb2O3yvlydXl4XsuRxE= X-Google-Smtp-Source: APXvYqxgB8iJLR3FGUKdHFZJDvju3DblV6tuKyVF56S3YhIV+YBWxGbOodVHf25+uYri4yKbWQUArQ== X-Received: by 2002:a05:600c:224f:: with SMTP id a15mr2589345wmm.2.1552665620207; Fri, 15 Mar 2019 09:00:20 -0700 (PDT) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id a18sm467766wmm.14.2019.03.15.09.00.19 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 15 Mar 2019 09:00:19 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Jeff King , Michael Haggerty , Stefan Beller , Jonathan Nieder , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 7/8] gc: handle & check gc.reflogExpire config Date: Fri, 15 Mar 2019 16:59:58 +0100 Message-Id: <20190315155959.12390-8-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.360.g471c308f928 In-Reply-To: <20190314123439.4347-1-avarab@gmail.com> References: <20190314123439.4347-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Don't redundantly run "git reflog expire --all" when gc.reflogExpire and gc.reflogExpireUnreachable are set to "never", and die immediately if those configuration valuer are bad. As an earlier "assert lack of early exit" change to the tests for "git reflog expire" shows, an early check of gc.reflogExpire{Unreachable,} isn't wanted in general for "git reflog expire", but it makes sense for "gc" because: 1) Similarly to 8ab5aa4bd8 ("parseopt: handle malformed --expire arguments more nicely", 2018-04-21) we'll now die early if the config variables are set to invalid values. We run "pack-refs" before "reflog expire", which can take a while, only to then die on an invalid gc.reflogExpire{Unreachable,} configuration. 2) Not invoking the command at all means it won't show up in trace output, which makes what's going on more obvious when the two are set to "never". 3) As a later change documents we lock the refs when looping over the refs to expire, even in cases where we end up doing nothing due to this config. For the reasons noted in the earlier "assert lack of early exit" change I don't think it's worth it to bend over backwards in "git reflog expire" itself to carefully detect if we'll really do nothing given the combination of all its possible options and skip that locking, but that's easy to detect here in "gc" where we'll only run "reflog expire" in a relatively simple mode. Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/gc.c | 17 +++++++++++++++++ t/t6500-gc.sh | 19 +++++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/builtin/gc.c b/builtin/gc.c index ae716a00d4a..8943bcc300d 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -116,6 +116,19 @@ static void process_log_file_on_signal(int signo) raise(signo); } +static int gc_config_is_timestamp_never(const char *var) +{ + const char *value; + timestamp_t expire; + + if (!git_config_get_value(var, &value) && value) { + if (parse_expiry_date(value, &expire)) + die(_("failed to parse '%s' value '%s'"), var, value); + return expire == 0; + } + return 0; +} + static void gc_config(void) { const char *value; @@ -127,6 +140,10 @@ static void gc_config(void) pack_refs = git_config_bool("gc.packrefs", value); } + if (gc_config_is_timestamp_never("gc.reflogexpire") && + gc_config_is_timestamp_never("gc.reflogexpireunreachable")) + prune_reflogs = 0; + git_config_get_int("gc.aggressivewindow", &aggressive_window); git_config_get_int("gc.aggressivedepth", &aggressive_depth); git_config_get_int("gc.auto", &gc_auto_threshold); diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh index 4684d06552a..7411bf7fecd 100755 --- a/t/t6500-gc.sh +++ b/t/t6500-gc.sh @@ -120,6 +120,25 @@ test_expect_success 'gc --quiet' ' test_must_be_empty stderr ' +test_expect_success 'gc.reflogExpire{Unreachable,}=never skips "expire" via "gc"' ' + test_config gc.reflogExpire never && + test_config gc.reflogExpireUnreachable never && + + GIT_TRACE=$(pwd)/trace.out git gc && + + # Check that git-pack-refs is run as a sanity check (done via + # gc_before_repack()) but that git-expire is not. + grep -E "^trace: (built-in|exec|run_command): git pack-refs --" trace.out && + ! grep -E "^trace: (built-in|exec|run_command): git reflog expire --" trace.out +' + +test_expect_success 'one of gc.reflogExpire{Unreachable,}=never does not skip "expire" via "gc"' ' + >trace.out && + test_config gc.reflogExpire never && + GIT_TRACE=$(pwd)/trace.out git gc && + grep -E "^trace: (built-in|exec|run_command): git reflog expire --" trace.out +' + run_and_wait_for_auto_gc () { # We read stdout from gc for the side effect of waiting until the # background gc process exits, closing its fd 9. Furthermore, the From patchwork Fri Mar 15 15:59:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10855103 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BA6D41575 for ; Fri, 15 Mar 2019 16:00:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A0B672AB09 for ; Fri, 15 Mar 2019 16:00:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 950ED2AB19; Fri, 15 Mar 2019 16:00:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0FE062AB24 for ; Fri, 15 Mar 2019 16:00:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729451AbfCOQA0 (ORCPT ); Fri, 15 Mar 2019 12:00:26 -0400 Received: from mail-wm1-f44.google.com ([209.85.128.44]:39007 "EHLO mail-wm1-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729440AbfCOQAY (ORCPT ); Fri, 15 Mar 2019 12:00:24 -0400 Received: by mail-wm1-f44.google.com with SMTP id t124so6530747wma.4 for ; Fri, 15 Mar 2019 09:00:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Z+rLvUzHSP2IZk76mXTgDM2KCWQ+BW4ue1TD0Qiwpq8=; b=FdVzpoE73TnLLag58pFQh5yd6awzlk/W4cNfFj/bT43X4ACc0l+zjdwoKpeOFSOzer 5cH6G5zaraw40JGUDlJ0lH1TmKL8BjKEG+jBgKiubYjhyfcq5E7/17fjMzvP+8LIU74k CXbXYr8Z3CeApuLC2FNW2/Nj6m9hD4Qywi+X+LC3FD6w7cTlQqFx+YyI+GdAn9ATPra1 fK2YrNPWwLfpQeUUwuiw6VK6IQF3QKL/r19dLHBCE+BDsRAzeIAOOQfW+fJVST3/m0Op eikaPh4sLkgx1HIbM7ZRXKgl8kVWWSh7Ljj/WTw9hlnNsPTkdBUjyv9DU3UCHLaIDNeV xLiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Z+rLvUzHSP2IZk76mXTgDM2KCWQ+BW4ue1TD0Qiwpq8=; b=RWrM12MagQlYeytr7FXD99gqLbS7rLl74TGPtyjlg4c61TdLkvd2xw8ZuEqaIDr8Pv MTVIsBRWVROhp6sw7uSnLgnd7FsuIZ2lpxGhxHhWsnbZvUWAO6i2QRXpVM7dH0rn2naM xkgW5TfP2MJXBnLYFc6yLri77ZmmELlUXcUQd0cKSe8JmevI3Z6cLFUkhLhTc7gd+H6w GZxROIdriA5FPwcP9zV3i/Kri3C+DyFMLaCY6CO07E2F92B/Eh3qC2bPu6OLbE2mEQJR 4lHaF+0H5vrOl2P0Ij7jBEcQE2gkwyT4nFIGU1LV5LuQTXG85XcSiVOwNViP4XjRMnkC C1BA== X-Gm-Message-State: APjAAAUFEgkrYrjTIBIlWpQCzRNASy4ocRmFPoqZGx6DmqHgOCErFVtv XnpsYca/RNnHpO4caD46FvODXGTSGM4= X-Google-Smtp-Source: APXvYqxG1WZSiPGrIUkcivYcRVzC+Jh7ao26V/TCIpRyAI6OIJ80r5+LSw8HDxIOwqnplh04F9F8NQ== X-Received: by 2002:a1c:ed0b:: with SMTP id l11mr2763231wmh.38.1552665621745; Fri, 15 Mar 2019 09:00:21 -0700 (PDT) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id a18sm467766wmm.14.2019.03.15.09.00.20 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 15 Mar 2019 09:00:20 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Jeff King , Michael Haggerty , Stefan Beller , Jonathan Nieder , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 8/8] reflog expire: don't assert the OID when locking refs Date: Fri, 15 Mar 2019 16:59:59 +0100 Message-Id: <20190315155959.12390-9-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.360.g471c308f928 In-Reply-To: <20190314123439.4347-1-avarab@gmail.com> References: <20190314123439.4347-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP During reflog expiry, the cmd_reflog_expire() function first iterates over all reflogs in logs/*, and then one-by-one acquires the lock for each one to expire its reflog by getting a *.lock file on the corresponding loose ref[1] (even if the actual ref is packed). This lock is needed, but what isn't needed is locking the loose ref as a function of the OID we found from that first iteration. By the time we get around to re-visiting the reference some of the OIDs may have changed. Thus the verify_lock() function called by the lock_ref_oid_basic() function being changed here would fail with e.g. "ref '%s' is at %s but expected %s" if the repository was being updated concurrent to the "reflog expire". By not passing the OID to it we'll try to lock the reference regardless of it last known OID. Locking as a function of the OID would make "reflog expire" exit with a non-zero exit status under such contention, which in turn meant that a "gc" command (which expires reflogs before forking to the background) would encounter a hard error. This behavior of considering the OID when locking has been here ever since "reflog expire" was initially implemented in 4264dc15e1 ("git reflog expire", 2006-12-19). As seen in that simpler initial version of the code we subsequently use the OID to inform the expiry (and still do), but never needed to use it to lock the reference associated with the reflog. By locking the reference without considering what OID we last saw it at, we won't encounter user-visible contention to the extent that core.filesRefLockTimeout mitigates it. See 4ff0f01cb7 ("refs: retry acquiring reference locks for 100ms", 2017-08-21). Unfortunately this sort of probabilistic contention is hard to turn into a test. I've tested this by running the following three subshells in concurrent terminals: ( cd /tmp && rm -rf git && git init git && cd git && while true do head -c 10 /dev/urandom | hexdump >out && git add out && git commit -m"out" done ) ( cd /tmp && rm -rf git-clone && git clone file:///tmp/git git-clone && cd git-clone && while git pull do date done ) ( cd /tmp/git-clone && while git reflog expire --all do date done ) Before this change the "reflog expire" would fail really quickly with a "but expected" error. After this change both the "pull" and "reflog expire" will run for a while, but eventually fail because I get unlucky with core.filesRefLockTimeout (the "reflog expire" is in a really tight loop). That can be resolved by being more generous with higher values of core.filesRefLockTimeout than the 100ms default. As noted in the commentary being added here we also need to handle the case of references being racily deleted, that can be tested by adding this to the above: ( cd /tmp/git-clone && while git branch topic master && git branch -D topic do date done ) We could change lock_ref_oid_basic() to always pass down RESOLVE_REF_READING to refs_resolve_ref_unsafe() and then files_reflog_expire() to detect the "is it deleted?" state. But let's not bother, in the event of such a race we're going to redundantly create a lock on the deleted reference, and shortly afterwards handle that case and others with the refs_reflog_exists() check. 1. https://public-inbox.org/git/54857871.5090805@alum.mit.edu/ Signed-off-by: Ævar Arnfjörð Bjarmason --- refs/files-backend.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/refs/files-backend.c b/refs/files-backend.c index ef053f716c3..c7ed1792b3b 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -3036,8 +3036,14 @@ static int files_reflog_expire(struct ref_store *ref_store, * The reflog file is locked by holding the lock on the * reference itself, plus we might need to update the * reference if --updateref was specified: + * + * We don't pass down the oid here because we'd like to be + * tolerant to the OID of the ref having changed, and to + * gracefully handle the case where it's been deleted (see oid + * -> mustexist -> RESOLVE_REF_READING in + * lock_ref_oid_basic()) ... */ - lock = lock_ref_oid_basic(refs, refname, oid, + lock = lock_ref_oid_basic(refs, refname, NULL, NULL, NULL, REF_NO_DEREF, &type, &err); if (!lock) { @@ -3045,6 +3051,13 @@ static int files_reflog_expire(struct ref_store *ref_store, strbuf_release(&err); return -1; } + /* + * When refs are deleted their reflog is deleted before the + * loose ref is deleted. This catches that case, i.e. when + * racing against a ref deletion lock_ref_oid_basic() will + * have acquired a lock on the now-deleted ref, but here's + * where we find out it has no reflog anymore. + */ if (!refs_reflog_exists(ref_store, refname)) { unlock_ref(lock); return 0;