From patchwork Wed Sep 8 22:17:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12481873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0EF8C433EF for ; Wed, 8 Sep 2021 22:17:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B170260F5B for ; Wed, 8 Sep 2021 22:17:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235602AbhIHWSb (ORCPT ); Wed, 8 Sep 2021 18:18:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53266 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235450AbhIHWSa (ORCPT ); Wed, 8 Sep 2021 18:18:30 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A752CC06175F for ; Wed, 8 Sep 2021 15:17:21 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id x6so5427817wrv.13 for ; Wed, 08 Sep 2021 15:17:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=/dfmxcYqlIsu/UwkRYNcSZeZirrrF6otX0QawEmFTYM=; b=rlgwuSPgztx3CGFQX1o540LAl06HU3YrBLbY7G2KGQrifRcz4jpWaEOahjsVGBnCOd 9eRRdby2J+OgZ+EzHmIONkZuwqtJEhNXT43JnFC6mHWWCZK5LX0Uwf16XbIIvoAeZ+iJ 4Rubjx2ntDmNamnLaV9YoudbVrknshuW2f6dmQXtwxJe4VaOHzqw857YK7CiqUKItkkp tAmY4jXZVAs0Rjl3IPEJKiBeapTPpRLkl8cg4j7CVUxHgHAXJKBFYZuWD+EcmPkdUepl 36wdU4AoQ6Zxm/Drf19sNnoi2ebq/6ShoPojAqZ/8pZCUD52oOBYUfjHslncKQ83mq6x V8kA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=/dfmxcYqlIsu/UwkRYNcSZeZirrrF6otX0QawEmFTYM=; b=hYzVf7/G5ovq2sGBGlju0806Ur+r0nAey99T/UbUaEWw7b053GcM2bWJhzEIAsMyOv vD3rHbUn/pzi3uBYYm8gApwOtsXiHWPS8gMmNYctZy9rxoaYYmAu95xHQl+fowrHBwnM I2N5dqzXOtSbUz8EQIPM0Hb444sxakEZ3QAqAYbtuV8ryAphZV4MgwNI/SrQwYOdESP6 BcvkxdtmoClpFZ/5teOA31BpOxfCkE4vzSri3CCEKb2of4iwSYgMo3dQHb6uZTB4akL+ zgm1za9nRK9VIcQCbSu/SxpSseTSA3gcBqlqeTRHBwuBIhTE+0WaAcXKrfLHVBvWpT4Y 2vpA== X-Gm-Message-State: AOAM530zZ0sWP/TB91BM8QwFBYaPhj/UxNc8hoaUPgStHl3LwA01lUDP e6iSJcspUEFymW9QISYjMgzESZV+OQp2hqf4 X-Google-Smtp-Source: ABdhPJxHR37txFP8aupqY2lj/Sef1TFcbnfUC/Vx897Z1nY7+N2zUy49T71/i178pLyjrfCzWu0vPA== X-Received: by 2002:adf:c18d:: with SMTP id x13mr526060wre.380.1631139440015; Wed, 08 Sep 2021 15:17:20 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id v10sm380197wrg.15.2021.09.08.15.17.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 15:17:19 -0700 (PDT) Date: Wed, 8 Sep 2021 18:17:17 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, avarab@gmail.com, gitster@pobox.com Subject: [PATCH v2 1/3] pack-write.c: rename `.idx` files into place last Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org We treat the presence of an `.idx` file as the indicator of whether or not it's safe to use a packfile. But `finish_tmp_packfile()` (which is responsible for writing the pack and moving the temporary versions of all of its auxiliary files into place) is inconsistent about the write order. But the `.rev` file is moved into place after the `.idx`, so it's possible for a process to open a pack in between the two (i.e., while the `.idx` file is in place but the `.rev` file is not) and mistakenly fall back to generating the pack's reverse index in memory. Though racy, this amounts to an unnecessary slow-down at worst, and doesn't affect the correctness of the resulting reverse index. Close this race by moving the .rev file into place before moving the .idx file into place. While we're here, only call strbuf_setlen() if we actually modified the buffer by bringing it inside of the same if-statement that calls rename(). Signed-off-by: Taylor Blau --- pack-write.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/pack-write.c b/pack-write.c index f1fc3ecafa..277c60165e 100644 --- a/pack-write.c +++ b/pack-write.c @@ -490,18 +490,18 @@ void finish_tmp_packfile(struct strbuf *name_buffer, strbuf_setlen(name_buffer, basename_len); - strbuf_addf(name_buffer, "%s.idx", hash_to_hex(hash)); - if (rename(idx_tmp_name, name_buffer->buf)) - die_errno("unable to rename temporary index file"); - - strbuf_setlen(name_buffer, basename_len); - if (rev_tmp_name) { strbuf_addf(name_buffer, "%s.rev", hash_to_hex(hash)); if (rename(rev_tmp_name, name_buffer->buf)) die_errno("unable to rename temporary reverse-index file"); + + strbuf_setlen(name_buffer, basename_len); } + strbuf_addf(name_buffer, "%s.idx", hash_to_hex(hash)); + if (rename(idx_tmp_name, name_buffer->buf)) + die_errno("unable to rename temporary index file"); + strbuf_setlen(name_buffer, basename_len); free((void *)idx_tmp_name); From patchwork Wed Sep 8 22:17:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12481875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 487C0C433EF for ; Wed, 8 Sep 2021 22:17:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2F7196113A for ; Wed, 8 Sep 2021 22:17:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235802AbhIHWSd (ORCPT ); Wed, 8 Sep 2021 18:18:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235450AbhIHWSb (ORCPT ); Wed, 8 Sep 2021 18:18:31 -0400 Received: from mail-io1-xd2a.google.com (mail-io1-xd2a.google.com [IPv6:2607:f8b0:4864:20::d2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7D45CC061575 for ; Wed, 8 Sep 2021 15:17:23 -0700 (PDT) Received: by mail-io1-xd2a.google.com with SMTP id a22so5308664iok.12 for ; Wed, 08 Sep 2021 15:17:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=1VxA8eA24M/VaCbj2GyZd23biCP9gqVmYOjFWdRdvVM=; b=mz+EHinvHbYG/amcEeJlHbYWsthhj4FLLbI1UMBET8LdQHtrU2iMlH2YNF253jOdKO xmaX7HzmdmOmaFZLRitvR1JXYbEKaGbDYcrN8YVSO0OVRr1L8zTs2NK7o+9PXCzAZPmB yvJwpeWqXQlxjBGN4fJmBldx97qP5PhjwIme+FEMby1ciIQjFozzqpiU9m7RLemfF7gG bKJ+MvHBk2pH1onKNYcAVoErSSqOMVD9seScCtz4G0/cvBvYu8z0jhEFPs+9+z5z7Y02 yQArCsJ9rdYpJ8kz+VrrGw+tsVVkMs5oMgly/BQAsn+0+51Q2/5mAc4TSJuP8SCfnaA3 XSjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=1VxA8eA24M/VaCbj2GyZd23biCP9gqVmYOjFWdRdvVM=; b=ACWRSfBcY9dWVXWBrtdYv8uboFD2mW4Ugcl6tIiT1egACmcvYImuXoCF4+rJEZXY1a qgWY30SaRGDVROSzixjeWfFOmp1lMMj8G96t/MoYphhAYYQsaYmLcBXAY/gY45AQe6/T Vsl0qpG/jJLL+qeBW+R7Xl4eqyVxv8noUP6O/APG3BuwLpgpqlQOKne3SGnKDspzLwBJ yRYi4EMokKqtAQdmZxJRvWRMkiTitRn95JdruNon5WprKs/TDhe+Y9KiJ1CvpDTxxrj0 aceJy7vq7RxGbj0BE46DFvWwCpn04r2gF996eBZEN0cBM/D75ASm+nmyFZI4hTbDlpwS H6PQ== X-Gm-Message-State: AOAM5323DsjbDXLgDXBeRaErN8Z3vQSfxCl1GGjXZVdy28WEAhV3SakR QPv2uecqUNSibmv1nxIcVoxumcAIQP+Lo2NJ X-Google-Smtp-Source: ABdhPJxvXHikYWfgnoZ+tuFN0ydTSP25iFYU3rLFFxgKkpcvO7yGOKkstEZ1AEOW2+XF1O84I5rfGQ== X-Received: by 2002:a02:5442:: with SMTP id t63mr570430jaa.7.1631139442836; Wed, 08 Sep 2021 15:17:22 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id f13sm203346ilk.45.2021.09.08.15.17.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 15:17:22 -0700 (PDT) Date: Wed, 8 Sep 2021 18:17:21 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, avarab@gmail.com, gitster@pobox.com Subject: [PATCH v2 2/3] builtin/repack.c: move `.idx` files into place last Message-ID: <925f9ada2adf4712f273140ce4da0ce49e5e6c06.1631139433.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a similar spirit as the previous patch, fix the identical problem from `git repack` (which invokes `pack-objects` with a temporary location for output, and then moves the files into their final locations itself). Signed-off-by: Taylor Blau --- builtin/repack.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/builtin/repack.c b/builtin/repack.c index 5f9bc74adc..c3e4771609 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -208,10 +208,10 @@ static struct { unsigned optional:1; } exts[] = { {".pack"}, - {".idx"}, {".rev", 1}, {".bitmap", 1}, {".promisor", 1}, + {".idx"}, }; static unsigned populate_pack_exts(char *name) From patchwork Wed Sep 8 22:17:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12481877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE41CC433FE for ; Wed, 8 Sep 2021 22:17:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BD9C161157 for ; Wed, 8 Sep 2021 22:17:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235896AbhIHWSf (ORCPT ); Wed, 8 Sep 2021 18:18:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235647AbhIHWSe (ORCPT ); Wed, 8 Sep 2021 18:18:34 -0400 Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A65CC061575 for ; Wed, 8 Sep 2021 15:17:26 -0700 (PDT) Received: by mail-io1-xd2e.google.com with SMTP id a15so5323029iot.2 for ; Wed, 08 Sep 2021 15:17:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=BGDSvYvc150kaOrOZO2v4d3Xihe6LQmxzAeQFCvv2DA=; b=Ri0uyWnpJ+3nywioqCRNQfV1BON0V5Ss3rGcZ2G/wH9fbCX0ErLnAIPYr9ZFMk7IAK adD649g5qGG42dcYmMc2YwPl8OpGAfeSb4AhIdQl3ljYIBqITxGVi8MHperd8sxz3/RO IPGbxhS1rD/JUhn9yIQaOmgkvLAYwMBWuUf/UKOJMiJPXIQwxCc5A4IPytDmiPUMrx9A 2Y6/DjHR+Htlml4EipelMj+k7WzQperMsFICiDZ16NqZYPXvLpQ9xZuxpVVFyqFDFeh2 uxwVLpI8CW47hiOzTA9Gt7cd3+X7ZlCB2xMo7t6JgIIdvZX4Pi50secIQde4bgO5M0KR LTGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=BGDSvYvc150kaOrOZO2v4d3Xihe6LQmxzAeQFCvv2DA=; b=eZNSHtYiGXi6vBpsCjpQDr8Z6M1zlCAIDQLsk6gFK5bPpY57XY+0aj/vDjZJI8QQgZ zM+3FbG8l0eq2Tthhf6sAveFZCycTJXIANoxndCxg/dzaIp0K0k7A11xtdiiTgwY45ff ReaYX+ZwcD1fWBzKYXp11Q9Bavl924l3cJrIav3UeWFwhPza3LPudSBGvYak6BlqbY2J tvKrQ7JBrZfKCzK1xGBcDBAgx9Z/ay7HZepl0wlA7UtQ+OXGiX7EQ1o0YM5FPPuQ93Tl RaIFH8Q7WMjLzDiTLawkfsBtJDFx1SPmTCtKIGpZJE1+76WMeT+HS72YHtcNz5LBjj1h 7GiA== X-Gm-Message-State: AOAM531KCdWvw2JwzTo9Xlb34BRr9Dtlx8PGJBqXy31AgFIUxCsSo1+h t637jaAFuDy8l1gBUolHWQuxsAxn8Y8U8NEc X-Google-Smtp-Source: ABdhPJzfBdKJGwu87ySfLvyXwhKQIo4R6RBGjUF42MyQa4UuF+R7sY0sl+WwJUy+OCJia6+ztdylDw== X-Received: by 2002:a6b:fc17:: with SMTP id r23mr405860ioh.89.1631139445440; Wed, 08 Sep 2021 15:17:25 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id e1sm187005ils.76.2021.09.08.15.17.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 15:17:25 -0700 (PDT) Date: Wed, 8 Sep 2021 18:17:24 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, avarab@gmail.com, gitster@pobox.com Subject: [PATCH v2 3/3] builtin/index-pack.c: move `.idx` files into place last Message-ID: <460b7b915157503df2ebef61f6863ed259586385.1631139433.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a similar spirit as the previous patch, fix the identical problem in `git index-pack`. Signed-off-by: Taylor Blau --- builtin/index-pack.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/builtin/index-pack.c b/builtin/index-pack.c index 8336466865..6ca47c5c03 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -1517,14 +1517,6 @@ static void final(const char *final_pack_name, const char *curr_pack_name, } else if (from_stdin) chmod(final_pack_name, 0444); - if (final_index_name != curr_index_name) { - if (!final_index_name) - final_index_name = odb_pack_name(&index_name, hash, "idx"); - if (finalize_object_file(curr_index_name, final_index_name)) - die(_("cannot store index file")); - } else - chmod(final_index_name, 0444); - if (curr_rev_index_name) { if (final_rev_index_name != curr_rev_index_name) { if (!final_rev_index_name) @@ -1535,6 +1527,14 @@ static void final(const char *final_pack_name, const char *curr_pack_name, chmod(final_rev_index_name, 0444); } + if (final_index_name != curr_index_name) { + if (!final_index_name) + final_index_name = odb_pack_name(&index_name, hash, "idx"); + if (finalize_object_file(curr_index_name, final_index_name)) + die(_("cannot store index file")); + } else + chmod(final_index_name, 0444); + if (do_fsck_object) { struct packed_git *p; p = add_packed_git(final_index_name, strlen(final_index_name), 0);