From patchwork Thu Sep 9 03:24:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12482365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3466CC433F5 for ; Thu, 9 Sep 2021 03:24:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0D88761131 for ; Thu, 9 Sep 2021 03:24:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350119AbhIID0D (ORCPT ); Wed, 8 Sep 2021 23:26:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350080AbhIID0A (ORCPT ); Wed, 8 Sep 2021 23:26:00 -0400 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8DF0DC061757 for ; Wed, 8 Sep 2021 20:24:51 -0700 (PDT) Received: by mail-wm1-x333.google.com with SMTP id u15-20020a05600c19cf00b002f6445b8f55so347728wmq.0 for ; Wed, 08 Sep 2021 20:24:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=jVjmHPs9Vzn+FL0BItXKX43L71zgap2RPxsgO9/F3J4=; b=UaWYTo8HE3QjcfLp28+zpPCdTYQoC77vffeuHV5z7ZeTj02DRn9rXamu6lxMggSIGw xkJrMmctWIh0TA0Qw+gt2nmw8wWu3m6YzTXItxL1Uwzfvr2jJwEZU2n3kX2hxjbISF4U ebUdaR79QX+0wf+L8wSjrfLcwIMt54melJQnhpZYb7zLJ9I2e2w3+uKs6/47FnIPzaa7 kXbmvU/Qd+WTWcn5Ic1h/9e+WW5YtjJESD4sUbLsquFoyz0zHAi7cpVOQDJBnVJwyJ3a jK6BhaydA9oXyCnm+o9OM4KAxGUMgwJknuglWv2mM46A7vCLFrI3UAYimb0BN4fP6/RO 7OoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=jVjmHPs9Vzn+FL0BItXKX43L71zgap2RPxsgO9/F3J4=; b=tDVqZ2wCRwKZjzeJbNaiaLuddNYml4or5gHXreupe8o52JYAr+i7sUTKw4htulQr6U N2ayg6TdAhq1fZhHd5Z6/NGmIO/ioVqKKOs2NufC9M751fRnvIu4on36MkZsPNqMUFOu dZgOhx3gAEghJSYabGJknrxHZZa7afTcBsMDOB2iTSJuPC+eMuGMwdUwjxz8Y8g/psx3 IQ3why5lZzLSWIJSrdasdZe5BIRh65CS6hOr88svStrykCvUaXPYZTP3gb1O/fQwKCJs dSote0xuljRhX6CQeFMpXzSTh1P1fRaJRhpTaPFj8gkwv7+5MLGU175Gclf0DdSSgB5w rDrA== X-Gm-Message-State: AOAM530UFeNlG7jcOeidNDlPTjWkXD3ugOaYFN47yVbmXV2LzXeZW9ue NaZ+/lzWz0h407a6GKNoUHAVBdhTlacpmhBY X-Google-Smtp-Source: ABdhPJwH7Sz/Hs6kPXWXrS5nZxHQYPmWh7YK3lRuHDTrxMJSPT+MzlPmOMQRmx8Zpjm5qQLG25/Lzw== X-Received: by 2002:a05:600c:3588:: with SMTP id p8mr597301wmq.20.1631157889959; Wed, 08 Sep 2021 20:24:49 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id n66sm324249wmn.2.2021.09.08.20.24.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 20:24:49 -0700 (PDT) Date: Wed, 8 Sep 2021 23:24:47 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com Subject: [PATCH 1/9] pack.h: line-wrap the definition of finish_tmp_packfile() Message-ID: <0b07aa4947caff53878a9000e2866f98146e2ccd.1631157880.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Ævar Arnfjörð Bjarmason Line-wrap the definition of finish_tmp_packfile() to make subsequent changes easier to read. See 0e990530ae (finish_tmp_packfile(): a helper function, 2011-10-28) for the commit that introduced this overly long line. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Taylor Blau --- pack.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/pack.h b/pack.h index fa13954526..1c17254c0a 100644 --- a/pack.h +++ b/pack.h @@ -110,6 +110,11 @@ int encode_in_pack_object_header(unsigned char *hdr, int hdr_len, int read_pack_header(int fd, struct pack_header *); struct hashfile *create_tmp_packfile(char **pack_tmp_name); -void finish_tmp_packfile(struct strbuf *name_buffer, const char *pack_tmp_name, struct pack_idx_entry **written_list, uint32_t nr_written, struct pack_idx_option *pack_idx_opts, unsigned char sha1[]); +void finish_tmp_packfile(struct strbuf *name_buffer, + const char *pack_tmp_name, + struct pack_idx_entry **written_list, + uint32_t nr_written, + struct pack_idx_option *pack_idx_opts, + unsigned char sha1[]); #endif From patchwork Thu Sep 9 03:24:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12482367 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44A0EC433EF for ; Thu, 9 Sep 2021 03:25:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 22F9F61167 for ; Thu, 9 Sep 2021 03:25:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350098AbhIID0H (ORCPT ); Wed, 8 Sep 2021 23:26:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36992 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350097AbhIID0C (ORCPT ); Wed, 8 Sep 2021 23:26:02 -0400 Received: from mail-il1-x130.google.com (mail-il1-x130.google.com [IPv6:2607:f8b0:4864:20::130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4358C0613CF for ; Wed, 8 Sep 2021 20:24:53 -0700 (PDT) Received: by mail-il1-x130.google.com with SMTP id z2so545479iln.0 for ; Wed, 08 Sep 2021 20:24:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=zRAm7wO8omCTI9xB/lMODQfTrHfmwinB1IrBpjhDdCU=; b=HsQWsDPv4QZf6auBYlO5MfQXidMIdIagsmnNOPLvWw8OzGvm2YIwzf6L2p/GKRAm+1 SIdB84zaKhqp/wpJvKLgWZx4ooyxtMVzvmgT1RRT40dNN+By0KmL3YVrYaRH+IqxezHf YmDhTMadYuGs3lRF8D9KWwZG1u+8n2SChDP9CoWixGPgYm468qZU7bBlHa3/Gp4uE5X6 jKA9GFXFTqjIGXRet1eO173FkSg44BKdwGvoiZwQyPVK/mM9IbB5pMkYBW3viG5xkiDd cAkuYC0yB4s/yi+A9SBy+W4LXOWPISuQlGM1EW/oZZLvQuIwVt5ynTdju8MEgR9M+DLN ZO1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=zRAm7wO8omCTI9xB/lMODQfTrHfmwinB1IrBpjhDdCU=; b=mGPYroNeSfgIlV+DeqJ2iBGZZZ8K+lh+I+nwdW1KpKwHeHeb+A3kcIBfC5DDLqPsy0 /BnFzc9gfaewetiSuyniGBRUzzTcLb3mB2cWFuEy2ewFpGAWl39p3LyuZzMdT4fnIZ+t QmrtKPMWMnlVNsm/WTyvmV/yKQTh5T8+wbRyzI/xEb6LkN/hCE/La6Q4dVlc34+tdl1D wG1mYzKITGzmqeIMkHRlPPmobMdeGU+U752fBxVMSXeErqO+ONc1Gk6C+Lqvy/pR1Mdi jL//vb/USPs3kCaL8ts8YwwM2B2Q1zFryXdkstc7oT/0nFbyn7d4B7RdhciQNkHtteyi R1oA== X-Gm-Message-State: AOAM532LRCtASwnbweBAf3E5t5UwXGkueEx7ImsO8zKccJpCjzTCRJ+J wpJs9JVD8cPj0CBq+A9VJSuPhRmWRbGCAHbg X-Google-Smtp-Source: ABdhPJwFQ0U5RHJUQu577e0exSuzO0ivC88a4Y1WynE9VveXe082SG8D2Ea75Wk4odwsnntKhxLLoA== X-Received: by 2002:a05:6e02:ef4:: with SMTP id j20mr706967ilk.294.1631157892878; Wed, 08 Sep 2021 20:24:52 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id s18sm307275iov.53.2021.09.08.20.24.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 20:24:52 -0700 (PDT) Date: Wed, 8 Sep 2021 23:24:51 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com Subject: [PATCH 2/9] bulk-checkin.c: store checksum directly Message-ID: <20b35ce0505d7bcc84b6fd68a33e0b6b1afa31e6.1631157880.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org finish_bulk_checkin() stores the checksum from finalize_hashfile() by writing to the `hash` member of `struct object_id`, but that hash has nothing to do with an object id (it's just a convenient location that happens to be sized correctly). Store the hash directly in an unsigned char array. This behaves the same as writing to the `hash` member, but makes the intent clearer (and avoids allocating an extra four bytes for the `algo` member of `struct object_id`). Signed-off-by: Taylor Blau --- bulk-checkin.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/bulk-checkin.c b/bulk-checkin.c index b023d9959a..6283bc8bd9 100644 --- a/bulk-checkin.c +++ b/bulk-checkin.c @@ -25,7 +25,7 @@ static struct bulk_checkin_state { static void finish_bulk_checkin(struct bulk_checkin_state *state) { - struct object_id oid; + unsigned char hash[GIT_MAX_RAWSZ]; struct strbuf packname = STRBUF_INIT; int i; @@ -37,11 +37,11 @@ static void finish_bulk_checkin(struct bulk_checkin_state *state) unlink(state->pack_tmp_name); goto clear_exit; } else if (state->nr_written == 1) { - finalize_hashfile(state->f, oid.hash, CSUM_HASH_IN_STREAM | CSUM_FSYNC | CSUM_CLOSE); + finalize_hashfile(state->f, hash, CSUM_HASH_IN_STREAM | CSUM_FSYNC | CSUM_CLOSE); } else { - int fd = finalize_hashfile(state->f, oid.hash, 0); - fixup_pack_header_footer(fd, oid.hash, state->pack_tmp_name, - state->nr_written, oid.hash, + int fd = finalize_hashfile(state->f, hash, 0); + fixup_pack_header_footer(fd, hash, state->pack_tmp_name, + state->nr_written, hash, state->offset); close(fd); } @@ -49,7 +49,7 @@ static void finish_bulk_checkin(struct bulk_checkin_state *state) strbuf_addf(&packname, "%s/pack/pack-", get_object_directory()); finish_tmp_packfile(&packname, state->pack_tmp_name, state->written, state->nr_written, - &state->pack_idx_opts, oid.hash); + &state->pack_idx_opts, hash); for (i = 0; i < state->nr_written; i++) free(state->written[i]); From patchwork Thu Sep 9 03:24:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12482369 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADA71C433FE for ; Thu, 9 Sep 2021 03:25:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 86D0861178 for ; Thu, 9 Sep 2021 03:25:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350099AbhIID0J (ORCPT ); Wed, 8 Sep 2021 23:26:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350109AbhIID0I (ORCPT ); Wed, 8 Sep 2021 23:26:08 -0400 Received: from mail-il1-x133.google.com (mail-il1-x133.google.com [IPv6:2607:f8b0:4864:20::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52DA9C061575 for ; Wed, 8 Sep 2021 20:24:57 -0700 (PDT) Received: by mail-il1-x133.google.com with SMTP id i13so511273ilm.4 for ; Wed, 08 Sep 2021 20:24:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=snPumuC91c2nde8TXJfK77FvS3+EXY1MFjkDnno6nlU=; b=Y8n0M3thBruf/7NdAGn+bW572ryMQBTto1Wzy6YS0XCZO1T+n3pW9EsaBLSk7L7+CI IHf6mka+zwcTx+QpmtlXSg3ZrQUvC+ppirs8qTtd1Kq4th1jov2L1/3tKztPuoZBWP0i HyZpA60k+RiytAWBll+lmEJ/hosa1ty73jFGE4xDz8Gonarh4YCnAgGS9EkUWGgOPQ64 2OvYe/YGzXMW91+IZv7wgI61TzlI/ikmS+3AR9Noeg2AZIcdk/pP+Vz2aC3PLaud2dQs 0+7ydgRnZqzLyPyr+g7aJWwqGYijoXKHGCBrEAiBeE/QtF1O0+LBZOdu8fRfA6sfXqul vxxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=snPumuC91c2nde8TXJfK77FvS3+EXY1MFjkDnno6nlU=; b=wRyv7jrqKNdx1M273TaDpSH1cB4PBUINXQ3QIMG8g0/f1ldgGoZwK3geT7guNoWKOf B2O62ScE7y1BZAmh7vp0AHA4agj8AUb+ovo5P9p4ofBNuH7i061Gh+hzDlpSx3lxjcZm qivLAMxXR/FwED4j/VD9IGqcOnitNIMMZolHSoqGIpPWnezY8p+CZ/Dn1PkSjYgs1Y0O yYYTBPth/bCTV2RKT4oVBUrZbjvtHIlAdCd8LgXq0gGc8DJqwR9k+o4fH1GBRJLnJJoz fpCRGS2FuRScHyEKhEQs4OIaTxhoqToHlDZf7ANPXNvZDk2SvDtDIDU8E9skBOnl4c92 gI8A== X-Gm-Message-State: AOAM532C5MiGOevHSSppdMTQuLi/gsNM/8TZbffBoA5rfYKuqoH8e7fA kWjfLYftELKK1TGuF/2eac0eNdsnJyed2wbI X-Google-Smtp-Source: ABdhPJy2Ql4oUR63U+swXyFL/Uyh+5gyzouwm8nczqPtodh5gynnMRFNH24YIhHL0TRKLtgzpBNu5g== X-Received: by 2002:a92:c56d:: with SMTP id b13mr681331ilj.317.1631157896617; Wed, 08 Sep 2021 20:24:56 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id a21sm266123iot.43.2021.09.08.20.24.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 20:24:56 -0700 (PDT) Date: Wed, 8 Sep 2021 23:24:55 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com Subject: [PATCH 3/9] pack-write: refactor renaming in finish_tmp_packfile() Message-ID: <35052ef494dbc55119614f3e22742d8d814b21b1.1631157880.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Ævar Arnfjörð Bjarmason Refactor the renaming in finish_tmp_packfile() into a helper function. The callers are now expected to pass a "name_buffer" ending in "pack-OID." instead of the previous "pack-", we then append "pack", "idx" or "rev" to it. By doing the strbuf_setlen() in rename_tmp_packfile() we reuse the buffer and avoid the repeated allocations we'd get if that function had its own temporary "struct strbuf". This approach of reusing the buffer does make the last user in pack-object.c's write_pack_file() slightly awkward, since we needlessly do a strbuf_setlen() before calling strbuf_release() for consistency. In subsequent changes we'll move that bitmap writing code around, so let's not skip the strbuf_setlen() now. The previous strbuf_reset() idiom originated with 5889271114a (finish_tmp_packfile():use strbuf for pathname construction, 2014-03-03), which in turn was a minimal adjustment of pre-strbuf code added in 0e990530ae (finish_tmp_packfile(): a helper function, 2011-10-28). Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Taylor Blau --- builtin/pack-objects.c | 7 +++++-- bulk-checkin.c | 3 ++- pack-write.c | 37 ++++++++++++++++--------------------- 3 files changed, 23 insertions(+), 24 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index df49f656b9..2a105c8d6e 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -1237,7 +1237,8 @@ static void write_pack_file(void) warning_errno(_("failed utime() on %s"), pack_tmp_name); } - strbuf_addf(&tmpname, "%s-", base_name); + strbuf_addf(&tmpname, "%s-%s.", base_name, + hash_to_hex(hash)); if (write_bitmap_index) { bitmap_writer_set_checksum(hash); @@ -1250,8 +1251,9 @@ static void write_pack_file(void) &pack_idx_opts, hash); if (write_bitmap_index) { - strbuf_addf(&tmpname, "%s.bitmap", hash_to_hex(hash)); + size_t tmpname_len = tmpname.len; + strbuf_addstr(&tmpname, "bitmap"); stop_progress(&progress_state); bitmap_writer_show_progress(progress); @@ -1260,6 +1262,7 @@ static void write_pack_file(void) bitmap_writer_finish(written_list, nr_written, tmpname.buf, write_bitmap_options); write_bitmap_index = 0; + strbuf_setlen(&tmpname, tmpname_len); } strbuf_release(&tmpname); diff --git a/bulk-checkin.c b/bulk-checkin.c index 6283bc8bd9..c19d471f0b 100644 --- a/bulk-checkin.c +++ b/bulk-checkin.c @@ -46,7 +46,8 @@ static void finish_bulk_checkin(struct bulk_checkin_state *state) close(fd); } - strbuf_addf(&packname, "%s/pack/pack-", get_object_directory()); + strbuf_addf(&packname, "%s/pack/pack-%s.", get_object_directory(), + hash_to_hex(hash)); finish_tmp_packfile(&packname, state->pack_tmp_name, state->written, state->nr_written, &state->pack_idx_opts, hash); diff --git a/pack-write.c b/pack-write.c index 2767b78619..95b063be94 100644 --- a/pack-write.c +++ b/pack-write.c @@ -458,6 +458,18 @@ struct hashfile *create_tmp_packfile(char **pack_tmp_name) return hashfd(fd, *pack_tmp_name); } +static void rename_tmp_packfile(struct strbuf *nb, const char *source, + const char *ext) +{ + size_t nb_len = nb->len; + + strbuf_addstr(nb, ext); + if (rename(source, nb->buf)) + die_errno("unable to rename temporary '*.%s' file to '%s'", + ext, nb->buf); + strbuf_setlen(nb, nb_len); +} + void finish_tmp_packfile(struct strbuf *name_buffer, const char *pack_tmp_name, struct pack_idx_entry **written_list, @@ -466,7 +478,6 @@ void finish_tmp_packfile(struct strbuf *name_buffer, unsigned char hash[]) { const char *idx_tmp_name, *rev_tmp_name = NULL; - int basename_len = name_buffer->len; if (adjust_shared_perm(pack_tmp_name)) die_errno("unable to make temporary pack file readable"); @@ -479,26 +490,10 @@ void finish_tmp_packfile(struct strbuf *name_buffer, rev_tmp_name = write_rev_file(NULL, written_list, nr_written, hash, pack_idx_opts->flags); - strbuf_addf(name_buffer, "%s.pack", hash_to_hex(hash)); - - if (rename(pack_tmp_name, name_buffer->buf)) - die_errno("unable to rename temporary pack file"); - - strbuf_setlen(name_buffer, basename_len); - - strbuf_addf(name_buffer, "%s.idx", hash_to_hex(hash)); - if (rename(idx_tmp_name, name_buffer->buf)) - die_errno("unable to rename temporary index file"); - - strbuf_setlen(name_buffer, basename_len); - - if (rev_tmp_name) { - strbuf_addf(name_buffer, "%s.rev", hash_to_hex(hash)); - if (rename(rev_tmp_name, name_buffer->buf)) - die_errno("unable to rename temporary reverse-index file"); - } - - strbuf_setlen(name_buffer, basename_len); + rename_tmp_packfile(name_buffer, pack_tmp_name, "pack"); + rename_tmp_packfile(name_buffer, idx_tmp_name, "idx"); + if (rev_tmp_name) + rename_tmp_packfile(name_buffer, rev_tmp_name, "rev"); free((void *)idx_tmp_name); } From patchwork Thu Sep 9 03:24:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12482371 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3D7FC433F5 for ; Thu, 9 Sep 2021 03:25:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C7CF661178 for ; Thu, 9 Sep 2021 03:25:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243655AbhIID0W (ORCPT ); Wed, 8 Sep 2021 23:26:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37048 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347223AbhIID0M (ORCPT ); Wed, 8 Sep 2021 23:26:12 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E24DC0613C1 for ; Wed, 8 Sep 2021 20:25:03 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id e26so300162wmk.2 for ; Wed, 08 Sep 2021 20:25:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=0GH/X5HCgQtgghJYJJMgKfzAuaoeMDV0lg7f3kpEJZI=; b=0vkigo6Ch1eSll1m/5RLZ3LxqyhfCMM+RII/bDBoBiZYrvsEmRByPV2St4Xz/DYaw6 YViOGVvoEF11ENhEzEFidkoYnaK2u4K3SfdQMPnvz/jRyu7gmdMIZYlFUb0tvNXgQD7j gshyyLwzvuWYxYElHITC+491p391qhRhwunZ0LSTgvrGFJTYnppnAeGexu9iG+/8qgE1 FVaWU1eh39eoxVaFlRVRODca/NafnAJ8GA1p8hdykpax6xxARTv9ejVy8YHSKyAY+wzg F1+6E349WkGateyqo+E4/aWAvQGWjV5eyzTLtfAmeT4GrI0hHaIyGv2g2XfMaqOXQHcy ovqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=0GH/X5HCgQtgghJYJJMgKfzAuaoeMDV0lg7f3kpEJZI=; b=sZQUB+hnJVj4+Gxf6zjwAytTV8kccoDUb1pOxhAS36LcmpIgjudNqYqaK5Tnjih7Td yFzMCIC7zBOWCgr3jeg6GllLbb7ln8fPLhOXBuop12XpL9e2bUEkdjLWDP9GZwwQ86v8 DbPvjRLiw8K9a+AefYonrQDUzM+dA+jgy85JVOMAIkHsfiu5WLOqzjg9FsLELVMQCTmo rHGeT1axR72epap9dMtNl6fu3TmOnSX0c4xUpVklEzQOuWKZJ6JrefgsNdDDwQ4s34Ra msSqLzlyoQrkpgsTlJc+vG276VV1DFCmD7DUUFAZ5rCNHijAcQyxWHQuLGX5QWlUZYNG h/dg== X-Gm-Message-State: AOAM533xm9Rf9eXoRg096RqOBwwfOP8i2ILZqcq0qOlFZmIIzOHm0XeI uFpI3zvP/P/o8cff7GzkF+snwbxQPf+x9Rog X-Google-Smtp-Source: ABdhPJzr30/L7Di1TY3h7gj4TJFVCDH9kZGtJvrhAh+CMLZqbeCTBjqcFHtiEP9lgYvGLg4L9152RQ== X-Received: by 2002:a1c:cc16:: with SMTP id h22mr549996wmb.117.1631157901616; Wed, 08 Sep 2021 20:25:01 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id n5sm332670wmd.29.2021.09.08.20.25.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 20:25:01 -0700 (PDT) Date: Wed, 8 Sep 2021 23:24:59 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com Subject: [PATCH 4/9] pack-write.c: rename `.idx` files after `*.rev` Message-ID: <0fb2c25f5ad8bfdccd653f760b1c4beeb05273e7.1631157880.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org We treat the presence of an `.idx` file as the indicator of whether or not it's safe to use a packfile. But `finish_tmp_packfile()` (which is responsible for writing the pack and moving the temporary versions of all of its auxiliary files into place) is inconsistent about the write order. Specifically, it moves the `.rev` file into place after the `.idx`, leaving open the possibility to open a pack which looks "ready" (because the `.idx` file exists and is readable) but appears momentarily to not have a `.rev` file. This causes Git to fall back to generating the pack's reverse index in memory. Though racy, this amounts to an unnecessary slow-down at worst, and doesn't affect the correctness of the resulting reverse index. Close this race by moving the .rev file into place before moving the .idx file into place. This still leaves the issue of `.idx` files being renamed into place before the auxiliary `.bitmap` file is renamed when in pack-object.c's write_pack_file() "write_bitmap_index" is true. That race will be addressed in subsequent commits. Signed-off-by: Taylor Blau Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Taylor Blau --- pack-write.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pack-write.c b/pack-write.c index 95b063be94..077710090e 100644 --- a/pack-write.c +++ b/pack-write.c @@ -491,9 +491,9 @@ void finish_tmp_packfile(struct strbuf *name_buffer, pack_idx_opts->flags); rename_tmp_packfile(name_buffer, pack_tmp_name, "pack"); - rename_tmp_packfile(name_buffer, idx_tmp_name, "idx"); if (rev_tmp_name) rename_tmp_packfile(name_buffer, rev_tmp_name, "rev"); + rename_tmp_packfile(name_buffer, idx_tmp_name, "idx"); free((void *)idx_tmp_name); } From patchwork Thu Sep 9 03:25:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12482373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB6F2C433F5 for ; Thu, 9 Sep 2021 03:25:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CC03061167 for ; Thu, 9 Sep 2021 03:25:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350195AbhIID0f (ORCPT ); Wed, 8 Sep 2021 23:26:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37100 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350167AbhIID0V (ORCPT ); Wed, 8 Sep 2021 23:26:21 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5957CC0613A3 for ; Wed, 8 Sep 2021 20:25:07 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id z4so363436wrr.6 for ; Wed, 08 Sep 2021 20:25:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=7nqV9WfFtPETODmBhucsE1hvbSzAfz7L/410X/EGp9c=; b=S1nTrpNQUJvWnkBXv6l3W4e97FP6qbReSl5DcEeRRShgPpmDzbXtdu5NQnuOlLF76v KC0hmDCTuNuDpquAMkrintsVc36xLSeqB+xaroUtqeY9/x1Ngh3CZmD2TVh+U0Qzb1Nh yQIXRl/60JNiIYlxMkSGdbUr8lhKLITTTQkwMEp8Y/38hJEn55jsGeUWg6Ze4Gm+JEWx CyCp/UC77hyOlc3sE2BipOSEw/FtmNX+MS5cKCriMO9gfj0yXdggP7F7Fad08OQzyTyG o1q/nr7XppVG0OHCKtEe7RlaC9WPo5yujboNJvm36KX+60hYGYuIkeGquaZsvyrpy6tU itFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=7nqV9WfFtPETODmBhucsE1hvbSzAfz7L/410X/EGp9c=; b=YrNYd0qVfWK3oGrYjFd463g+afaejuq7k/xVFoDnpNUm/IqI2lJryOOgEl2UOGZW/N qbvVUaP8gzjbfFROPXyqcWxksCg0F0Dlugq6oZsPZnDwg2obvwLG/1NTy8CuxhuJY5r4 QBdHaTC5oNAusu8KDmZInzo2Kw3CO1RZD+GU4UZaTY2UccqU9OTO/o494ykA2BwmKad8 vPkSm8iY8pow9zf+MpfrG2j6uOAzkUYrhEiRwyT0DVGuldcszAFzwKsTFczIseZa3209 AjXqp3/o0A29CYnJLMziWLBri9o1lJmORQ4R1Awk1zQPdsDbnFKLogqGvUVfDiiqm5Ri u2bA== X-Gm-Message-State: AOAM532KYlTFtd2fM5+vwv78a7aU/l1B9dgiOsXmJ1D0i+TiYmvFNafw aTC9UTYkm2dcfD4Rf0flB0jNQg/r56YuowJ2 X-Google-Smtp-Source: ABdhPJxkSbBA4KlXBa9kA01wDeGxdrjef1H0GkofpO76b2/VVMffbp8kAK3h6XSH1DI25bLGeUElEA== X-Received: by 2002:adf:b785:: with SMTP id s5mr822221wre.30.1631157905693; Wed, 08 Sep 2021 20:25:05 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id i5sm395027wrc.86.2021.09.08.20.25.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 20:25:05 -0700 (PDT) Date: Wed, 8 Sep 2021 23:25:03 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com Subject: [PATCH 5/9] builtin/repack.c: move `.idx` files into place last Message-ID: <3b10a97ec0e7c9e672904e6415909a1b8cea872e.1631157880.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a similar spirit as the previous patch, fix the identical problem from `git repack` (which invokes `pack-objects` with a temporary location for output, and then moves the files into their final locations itself). Signed-off-by: Taylor Blau Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Taylor Blau --- builtin/repack.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/builtin/repack.c b/builtin/repack.c index 5f9bc74adc..c3e4771609 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -208,10 +208,10 @@ static struct { unsigned optional:1; } exts[] = { {".pack"}, - {".idx"}, {".rev", 1}, {".bitmap", 1}, {".promisor", 1}, + {".idx"}, }; static unsigned populate_pack_exts(char *name) From patchwork Thu Sep 9 03:25:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12482375 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BFE1C433F5 for ; Thu, 9 Sep 2021 03:25:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E8C1A61178 for ; Thu, 9 Sep 2021 03:25:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350199AbhIID0j (ORCPT ); Wed, 8 Sep 2021 23:26:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348438AbhIID0Z (ORCPT ); Wed, 8 Sep 2021 23:26:25 -0400 Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F2B0C06129E for ; Wed, 8 Sep 2021 20:25:11 -0700 (PDT) Received: by mail-io1-xd2e.google.com with SMTP id b10so511796ioq.9 for ; Wed, 08 Sep 2021 20:25:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=aG4vFDRSstWIxGq9i2r2LsI+fZ6ttAvueBHHOcVtyLo=; b=xCFDLR1fImwxb7xnExiUGHv+B0TIUa5x95O13b76mijXdkp16dHs+9n1/DF41nhDPy wy+hX/qGXjON7YgByqh1IhikuEa+mC8g/FAGDQG2E9jDKogWwRW5qre1ktyh2QVl0erN n0ljuQUb5UxekeOFno79Xnl5lxBtDQZqyeNTAEPW67JYQjBopJprPp3e1V3sm2NyapqS /bl9ZC1KxRrmQFHWxwom9R2Gj0AI6Vjhj7gakyxzwmUrnsvwzQ50tEvjBFc77/fsbO2Y bUBsOKZCiEcbsFokveCzXx0hcT1Kh70EtmoDU96kZs6qjfyxRrIAZC39K+jbCwr/UAyD SJow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=aG4vFDRSstWIxGq9i2r2LsI+fZ6ttAvueBHHOcVtyLo=; b=4pD+MkTaSWG0mMkXbaCjKn5DnlFVtQ/e7MpS+YgIjFbR75AXnAj3T8q+h6XB0n4ZUR BuNOUw56bTFHrcJxzVFHk3A1TPqUkWxrzIZHVwXE49G07yng5EE4bK/1FJoNOQxD9JdA Jis+rtBKePJaLjkSCyUH+HVkivPcsEcRUAlznJ7W0JT8XI4Tpi//JNRzRJpgMmYeLunD yzXhRy2EhFMpOnm/LG7vrGJUox9wKHMCNuce66A6QX6BDH761msuIfz7gIxlQjaWyIDn 6b1NylnMxvoetrZMssJuM3QSa3T0H2wtdHrERhBQA/0IM+yfdvIgla1VkQmZiugImGM6 gklQ== X-Gm-Message-State: AOAM5307Z2LOktqGhEfqrhC4/zhFvlJ06hRIR1Wuu6DSOkOWh2J1hi6U RRHPcjydgfCSU8HofX3I5cn3cvRclwYGDCzf X-Google-Smtp-Source: ABdhPJx77SbtsnyxAHYw+8GNK1SncYRsA+D95JVv9UhQxV0eo/dMAEnBEvtXOvSSHkOOy10R3+OtRQ== X-Received: by 2002:a02:ad17:: with SMTP id s23mr841449jan.135.1631157910466; Wed, 08 Sep 2021 20:25:10 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id p135sm305443iod.26.2021.09.08.20.25.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 20:25:10 -0700 (PDT) Date: Wed, 8 Sep 2021 23:25:09 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com Subject: [PATCH 6/9] index-pack: refactor renaming in final() Message-ID: <3c9b515907ecb632faf73ce8db83efed1493d1f1.1631157880.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Ævar Arnfjörð Bjarmason Refactor the renaming in final() into a helper function, this is similar in spirit to a preceding refactoring of finish_tmp_packfile() in pack-write.c. Before e37d0b8730b (builtin/index-pack.c: write reverse indexes, 2021-01-25) it probably wasn't worth it to have this sort of helper, due to the differing "else if" case for "pack" files v.s. "idx" files. But since we've got "rev" as well now, let's do the renaming via a helper, this is both a net decrease in lines, and improves the readability, since we can easily see at a glance that the logic for writing these three types of files is exactly the same, aside from the obviously differing cases of "*final_xyz_name" being NULL, and "else_chmod_if" being different. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Taylor Blau --- builtin/index-pack.c | 48 +++++++++++++++++++++----------------------- 1 file changed, 23 insertions(+), 25 deletions(-) diff --git a/builtin/index-pack.c b/builtin/index-pack.c index 6cc4890217..cd4e85f5bb 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -1477,6 +1477,22 @@ static void write_special_file(const char *suffix, const char *msg, strbuf_release(&name_buf); } +static void rename_tmp_packfile(const char **final_xyz_name, + const char *curr_xyz_name, + struct strbuf *xyz_name, unsigned char *hash, + const char *ext, int else_chmod_if) +{ + if (*final_xyz_name != curr_xyz_name) { + if (!*final_xyz_name) + *final_xyz_name = odb_pack_name(xyz_name, hash, ext); + if (finalize_object_file(curr_xyz_name, *final_xyz_name)) + die(_("unable to rename temporary '*.%s' file to '%s"), + ext, *final_xyz_name); + } else if (else_chmod_if) { + chmod(*final_xyz_name, 0444); + } +} + static void final(const char *final_pack_name, const char *curr_pack_name, const char *final_index_name, const char *curr_index_name, const char *final_rev_index_name, const char *curr_rev_index_name, @@ -1505,31 +1521,13 @@ static void final(const char *final_pack_name, const char *curr_pack_name, write_special_file("promisor", promisor_msg, final_pack_name, hash, NULL); - if (final_pack_name != curr_pack_name) { - if (!final_pack_name) - final_pack_name = odb_pack_name(&pack_name, hash, "pack"); - if (finalize_object_file(curr_pack_name, final_pack_name)) - die(_("cannot store pack file")); - } else if (from_stdin) - chmod(final_pack_name, 0444); - - if (final_index_name != curr_index_name) { - if (!final_index_name) - final_index_name = odb_pack_name(&index_name, hash, "idx"); - if (finalize_object_file(curr_index_name, final_index_name)) - die(_("cannot store index file")); - } else - chmod(final_index_name, 0444); - - if (curr_rev_index_name) { - if (final_rev_index_name != curr_rev_index_name) { - if (!final_rev_index_name) - final_rev_index_name = odb_pack_name(&rev_index_name, hash, "rev"); - if (finalize_object_file(curr_rev_index_name, final_rev_index_name)) - die(_("cannot store reverse index file")); - } else - chmod(final_rev_index_name, 0444); - } + rename_tmp_packfile(&final_pack_name, curr_pack_name, &pack_name, + hash, "pack", from_stdin); + rename_tmp_packfile(&final_index_name, curr_index_name, &index_name, + hash, "idx", 1); + if (curr_rev_index_name) + rename_tmp_packfile(&final_rev_index_name, curr_rev_index_name, + &rev_index_name, hash, "rev", 1); if (do_fsck_object) { struct packed_git *p; From patchwork Thu Sep 9 03:25:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12482379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26C1CC433F5 for ; Thu, 9 Sep 2021 03:26:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0208960F93 for ; Thu, 9 Sep 2021 03:26:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350127AbhIID1T (ORCPT ); Wed, 8 Sep 2021 23:27:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37094 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350142AbhIID0c (ORCPT ); Wed, 8 Sep 2021 23:26:32 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 555AEC0613D9 for ; Wed, 8 Sep 2021 20:25:17 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id l7-20020a1c2507000000b002e6be5d86b3so322201wml.3 for ; Wed, 08 Sep 2021 20:25:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=I/jSGotHph8kLDRIU5d1Y/68nmelh25acg/xGCG6l3g=; b=A8ESGdAXvslJYmqPRRnzvU8JZJS4Y+WX4s37eRb2wHnYtxQilV76sN5QVb+Qkt+g6Q sYOe6AhIuo/QyvicX8WvkNkXt2hZLApD40NZJkNpXqsY45o8Xi/M8LyHAa3bm3lhJIbl g8rsEUe0yBVZeNmKDz22ETgTwAgCrvoCYa5O0DBXwSWnJoTmGfTgQ8z6F/GBGpZBUJEz haBYfkuz/4IwYvpPcaLQTb3C14bSlCVWFJVrBuT7b/VwOdw4BISm9W8dVapletsWyjmU EU9VZGkPNQmNWi4nST4d9Ld9qqgxTYiQiQvQ+mUjZpD1mrXHqKkA6WaUx9pn1Ld5hVi/ kgDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=I/jSGotHph8kLDRIU5d1Y/68nmelh25acg/xGCG6l3g=; b=JnegoR2HTqPx4IIuqsW3OL4GVnLsyJFA1rX0QJs3/Kd0/12wSob8ywbb36u4+gM9Rd xNWi6gXr26jZV5e0yyDGsufm9pPGza1cmeH9h/bbo1MItsVpe3pRUfsGyW5d2f0bdzRM y1diM4jjREegWYbKuQaMAEx3EX9whLspsMxrpsls/h+07INpPUZrYGJgFFHvQiCOyuuc WZzboL6SrFcJEyYqylSRbZ84e9kp72IESihkPyt9w1oCMaVdnTZ3P2Qt/mXlFqR0Vwdu YUa85LSHCbZfcjd5nj9hjWQpxYbb+QjRxKd5QUObbE3E0lWVYiQnPpYzzY/DVjv51i82 c6Xw== X-Gm-Message-State: AOAM533/LTUCGn6w4aCnBJ9Ql/g6EmGc7j0a0dD0T4Zbsuks7U6tXyq8 soSTUzJsbt6jB2kWOTKZrS8+W3t0yXwZNn7A X-Google-Smtp-Source: ABdhPJwS2+3GYTne0x6IG2LU4BQOrr5S16G9mEmmR8tF8BjHFfYEWg4ruPTkco3JTXNXlpO2DUCFuw== X-Received: by 2002:a7b:c451:: with SMTP id l17mr612572wmi.74.1631157915681; Wed, 08 Sep 2021 20:25:15 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id c7sm310317wmq.13.2021.09.08.20.25.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 20:25:15 -0700 (PDT) Date: Wed, 8 Sep 2021 23:25:13 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com Subject: [PATCH 7/9] builtin/index-pack.c: move `.idx` files into place last Message-ID: <8d67a715014b9daa9a6ca3a77be647e733324522.1631157880.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a similar spirit as preceding patches to `git repack` and `git pack-objects`, fix the identical problem in `git index-pack`. Signed-off-by: Taylor Blau Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Taylor Blau --- builtin/index-pack.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/builtin/index-pack.c b/builtin/index-pack.c index cd4e85f5bb..bf294d9083 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -1523,11 +1523,11 @@ static void final(const char *final_pack_name, const char *curr_pack_name, rename_tmp_packfile(&final_pack_name, curr_pack_name, &pack_name, hash, "pack", from_stdin); - rename_tmp_packfile(&final_index_name, curr_index_name, &index_name, - hash, "idx", 1); if (curr_rev_index_name) rename_tmp_packfile(&final_rev_index_name, curr_rev_index_name, &rev_index_name, hash, "rev", 1); + rename_tmp_packfile(&final_index_name, curr_index_name, &index_name, + hash, "idx", 1); if (do_fsck_object) { struct packed_git *p; From patchwork Thu Sep 9 03:25:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12482377 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE6E8C433EF for ; Thu, 9 Sep 2021 03:26:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CEE5B61059 for ; Thu, 9 Sep 2021 03:26:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350053AbhIID1S (ORCPT ); Wed, 8 Sep 2021 23:27:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350147AbhIID0c (ORCPT ); Wed, 8 Sep 2021 23:26:32 -0400 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 87041C061796 for ; Wed, 8 Sep 2021 20:25:19 -0700 (PDT) Received: by mail-io1-xd2b.google.com with SMTP id b10so512095ioq.9 for ; Wed, 08 Sep 2021 20:25:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=mDeGGO4oyqxo9L8YtU/mBXVY0SatXK3Wj/QeNSit4PM=; b=NPo5ERliiCI0OGHl9m8LdTLlx7atXapKXRwTE3n2yfTYD3YaPX7T/o0EnYFQ7sZpTg t03p8fso1a21HTwMB3Nw3miTNxnpaOWeslHXXlH77euykN+YaB2+oi/AMA8hexHWloSs PtHcJhxVa3clZtT0SYFtP5VBUCRtN1cGiYJHAF4PS4y4SHiwsgis3OaZPfm/bJWueIlS 7DkEgqqIyHKVXw4a349yDdza/vmN8u62jpeGi5LyNGc/LglMKa5sPbhJqjJZ9HBTh81K OWC8JYgYeff3WfKFcFXaN6grBQ5+SOKDiMqHb9oM12LcxDMQhL/eMHVKsbIZICncy1AN 58fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=mDeGGO4oyqxo9L8YtU/mBXVY0SatXK3Wj/QeNSit4PM=; b=rKyYVG8nDr/MSDpEEC8Yt2hDA45NUaLNRQTe3de3MuQgSyOiNfPfTLpoX7w/QwvMlN VzqD1lK6F7ccrOVSuU1IKuuMgFFD+zyPLt1fKZBdKq0Yf5fNHQm3WFbX8qmjxyxfdlQ+ /eHDmHQ5PW5RZwamJ/gCI9SsFGBy9pJRP896WZLWBzkQmrhzEfDwPucju7TRhVfscCoP y7MvaYhp3EkVP9Y2po3dUGl8ZCZj39RdWqAdq+L3vEdrDuuaFRBueFk2NCec62m/FD/F ATAascYHZkuXTRI9hTlNqSbBJfS/5k6pJf05zPKaw8CP0FQVDYmWFOMQfMO6vQNyvF6C rNAQ== X-Gm-Message-State: AOAM530piItwwhS+brRRZqQxuSEfKtJuFqu+N+y7Dx4CCm0ds/ZCwggm EJ5wMrs2LEaR/RN//eRKLG7t5r/JEOrpe3sR X-Google-Smtp-Source: ABdhPJwQyWInBPQKdgQ5JWZ/x++qVQOnexw9cIYwKbizQRvW9NsBs/vwsYBGP3de2kdn4s6yu6DzRg== X-Received: by 2002:a05:6602:22d4:: with SMTP id e20mr740632ioe.203.1631157918854; Wed, 08 Sep 2021 20:25:18 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id e1sm278527ils.76.2021.09.08.20.25.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 20:25:18 -0700 (PDT) Date: Wed, 8 Sep 2021 23:25:17 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com Subject: [PATCH 8/9] pack-write: split up finish_tmp_packfile() function Message-ID: <5c553229b00c5d09c74a0947d1cbc9bd5cc880e4.1631157880.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Ævar Arnfjörð Bjarmason Split up the finish_tmp_packfile() function and use the split-up version in pack-objects.c in preparation for moving the step of renaming the *.idx file later as part of a function change. Since the only other caller of finish_tmp_packfile() was in bulk-checkin.c, and it won't be needing a change to its *.idx renaming, provide a thin wrapper for the old function as a static function in that file. If other callers end up needing the simpler version it could be moved back to "pack-write.c" and "pack.h". Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Taylor Blau --- builtin/pack-objects.c | 7 +++++-- bulk-checkin.c | 16 ++++++++++++++++ pack-write.c | 22 +++++++++++++--------- pack.h | 7 +++++-- 4 files changed, 39 insertions(+), 13 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 2a105c8d6e..944134b6f2 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -1217,6 +1217,7 @@ static void write_pack_file(void) if (!pack_to_stdout) { struct stat st; struct strbuf tmpname = STRBUF_INIT; + char *idx_tmp_name = NULL; /* * Packs are runtime accessed in their mtime @@ -1246,9 +1247,10 @@ static void write_pack_file(void) &to_pack, written_list, nr_written); } - finish_tmp_packfile(&tmpname, pack_tmp_name, + stage_tmp_packfiles(&tmpname, pack_tmp_name, written_list, nr_written, - &pack_idx_opts, hash); + &pack_idx_opts, hash, &idx_tmp_name); + rename_tmp_packfile_idx(&tmpname, &idx_tmp_name); if (write_bitmap_index) { size_t tmpname_len = tmpname.len; @@ -1265,6 +1267,7 @@ static void write_pack_file(void) strbuf_setlen(&tmpname, tmpname_len); } + free(idx_tmp_name); strbuf_release(&tmpname); free(pack_tmp_name); puts(hash_to_hex(hash)); diff --git a/bulk-checkin.c b/bulk-checkin.c index c19d471f0b..8785b2ac80 100644 --- a/bulk-checkin.c +++ b/bulk-checkin.c @@ -23,6 +23,22 @@ static struct bulk_checkin_state { uint32_t nr_written; } state; +static void finish_tmp_packfile(struct strbuf *basename, + const char *pack_tmp_name, + struct pack_idx_entry **written_list, + uint32_t nr_written, + struct pack_idx_option *pack_idx_opts, + unsigned char hash[]) +{ + char *idx_tmp_name = NULL; + + stage_tmp_packfiles(basename, pack_tmp_name, written_list, nr_written, + pack_idx_opts, hash, &idx_tmp_name); + rename_tmp_packfile_idx(basename, &idx_tmp_name); + + free(idx_tmp_name); +} + static void finish_bulk_checkin(struct bulk_checkin_state *state) { unsigned char hash[GIT_MAX_RAWSZ]; diff --git a/pack-write.c b/pack-write.c index 077710090e..d40d2ddd01 100644 --- a/pack-write.c +++ b/pack-write.c @@ -470,21 +470,28 @@ static void rename_tmp_packfile(struct strbuf *nb, const char *source, strbuf_setlen(nb, nb_len); } -void finish_tmp_packfile(struct strbuf *name_buffer, +void rename_tmp_packfile_idx(struct strbuf *name_buffer, + char **idx_tmp_name) +{ + rename_tmp_packfile(name_buffer, *idx_tmp_name, "idx"); +} + +void stage_tmp_packfiles(struct strbuf *name_buffer, const char *pack_tmp_name, struct pack_idx_entry **written_list, uint32_t nr_written, struct pack_idx_option *pack_idx_opts, - unsigned char hash[]) + unsigned char hash[], + char **idx_tmp_name) { - const char *idx_tmp_name, *rev_tmp_name = NULL; + const char *rev_tmp_name = NULL; if (adjust_shared_perm(pack_tmp_name)) die_errno("unable to make temporary pack file readable"); - idx_tmp_name = write_idx_file(NULL, written_list, nr_written, - pack_idx_opts, hash); - if (adjust_shared_perm(idx_tmp_name)) + *idx_tmp_name = (char *)write_idx_file(NULL, written_list, nr_written, + pack_idx_opts, hash); + if (adjust_shared_perm(*idx_tmp_name)) die_errno("unable to make temporary index file readable"); rev_tmp_name = write_rev_file(NULL, written_list, nr_written, hash, @@ -493,9 +500,6 @@ void finish_tmp_packfile(struct strbuf *name_buffer, rename_tmp_packfile(name_buffer, pack_tmp_name, "pack"); if (rev_tmp_name) rename_tmp_packfile(name_buffer, rev_tmp_name, "rev"); - rename_tmp_packfile(name_buffer, idx_tmp_name, "idx"); - - free((void *)idx_tmp_name); } void write_promisor_file(const char *promisor_name, struct ref **sought, int nr_sought) diff --git a/pack.h b/pack.h index 1c17254c0a..b22bfc4a18 100644 --- a/pack.h +++ b/pack.h @@ -110,11 +110,14 @@ int encode_in_pack_object_header(unsigned char *hdr, int hdr_len, int read_pack_header(int fd, struct pack_header *); struct hashfile *create_tmp_packfile(char **pack_tmp_name); -void finish_tmp_packfile(struct strbuf *name_buffer, +void stage_tmp_packfiles(struct strbuf *name_buffer, const char *pack_tmp_name, struct pack_idx_entry **written_list, uint32_t nr_written, struct pack_idx_option *pack_idx_opts, - unsigned char sha1[]); + unsigned char hash[], + char **idx_tmp_name); +void rename_tmp_packfile_idx(struct strbuf *basename, + char **idx_tmp_name); #endif From patchwork Thu Sep 9 03:25:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12482381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7850C433EF for ; Thu, 9 Sep 2021 03:26:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C750160F92 for ; Thu, 9 Sep 2021 03:26:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350159AbhIID1U (ORCPT ); Wed, 8 Sep 2021 23:27:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37100 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350162AbhIID0e (ORCPT ); Wed, 8 Sep 2021 23:26:34 -0400 Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35F5AC0617A8 for ; Wed, 8 Sep 2021 20:25:24 -0700 (PDT) Received: by mail-io1-xd35.google.com with SMTP id q3so532108iot.3 for ; Wed, 08 Sep 2021 20:25:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=ajkVj0Qyvw/bQUPoafTQ+yX+kdQZvrArM4W/UBNO2dA=; b=JBrhKeDy4v6ffTEu+qUTtJzbiM4Osaqg+AjCUUdvacinuTDek07LXl54U1y0Roj87Z S9Ekzah7VguNkzIg+S39dd1CM665/BKu53X33TyDEMHGiFWulVsi3H1vD6xiaJI3QQOX 1adIxiMdpoMj7WtKQy1toGSr9OWOt0Qi5QaWFukYLofeunw9yy8TT9g4/dtpfHK61EYP DIPkZHY9C76iyMwRtjUr3e+9JT+kF2jMRVyWMQ+CCll4VAIuNIJlNmdUbYLJZufRy2+b SRS+RvQAHgUbYtHf6MVj/Sb43/Bxeo1EPJPa6eYpG1HRlNLGZ/OmAfgpk0MHomjigLmp GVng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=ajkVj0Qyvw/bQUPoafTQ+yX+kdQZvrArM4W/UBNO2dA=; b=4FGR3oYganQbtXrYLN77EmNGbQ3H6RQbtdeZLx0W+uiVZ27oPP2dqFns3duvd6GFe+ tIGfA/W/Yruo1tHS59gbVZ2ggF6OfYVPmIYWSOPXyg+Az4DBsN+RYIrAAtVqj/zU1DLI pAFNVyMLA+Z3BkeRb3rXDMx/o8uGdFnm682SXYrHIwkaDoxI8m9WDMU4XBWbSMBw5ZKu 9IGXrBrZmqpAG/oCOYphtLQdYxHU/qn65z6ZpSpdQhBdSZfc9d/TZT2D77HNm58TMa1y gSAcvMrcfXI5m2DVeHx3iE/djIhsvxMBOQReo8AYFoUVwfpS/pZekscdF0Eygmv03fQG ppvQ== X-Gm-Message-State: AOAM5308StSNIzE3J+28GTp7P+K+0gGdJTlEDxiT7c6EIJEXUodnX5iV S5DRHkKP+m4GJKLX0u55gRVU8a23j6rVdrQQ X-Google-Smtp-Source: ABdhPJxljJWfWK3JJf855VsjTXfP/BZ5yiAnKnCESCpzj0zCXP9S9znflWmKE1Pw1F/tJ/n+mXufRA== X-Received: by 2002:a5e:9901:: with SMTP id t1mr792240ioj.92.1631157923522; Wed, 08 Sep 2021 20:25:23 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id o15sm296187ilo.73.2021.09.08.20.25.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 20:25:23 -0700 (PDT) Date: Wed, 8 Sep 2021 23:25:22 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: avarab@gmail.com, gitster@pobox.com Subject: [PATCH 9/9] pack-objects: rename .idx files into place after .bitmap files Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Ævar Arnfjörð Bjarmason In preceding commits the race of renaming .idx files in place before .rev files and other auxiliary files was fixed in pack-write.c's finish_tmp_packfile(), builtin/repack.c's "struct exts", and builtin/index-pack.c's final(). As noted in the change to pack-write.c we left in place the issue of writing *.bitmap files after the *.idx, let's fix that issue. See 7cc8f971085 (pack-objects: implement bitmap writing, 2013-12-21) for commentary at the time when *.bitmap was implemented about how those files are written out, nothing in that commit contradicts what's being done here. Note that this commit and preceding ones only close any race condition with *.idx files being written before their auxiliary files if we're optimistic about our lack of fsync()-ing in this are not tripping us over. See the thread at [1] for a rabbit hole of various discussions about filesystem races in the face of doing and not doing fsync() (and if doing fsync(), not doing it properly). In particular, in this case of writing to ".git/objects/pack" we only write and fsync() the individual files, but if we wanted to guarantee that the metadata update was seen in that way by concurrent processes we'd need to fsync() on the "fd" of the containing directory. That concern is probably more theoretical than not, modern OS's tend to be more on the forgiving side than the overly pedantic side of implementing POSIX FS semantics. 1. https://lore.kernel.org/git/8735qgkvv1.fsf@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Taylor Blau --- builtin/pack-objects.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 944134b6f2..a01767a384 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -1250,7 +1250,6 @@ static void write_pack_file(void) stage_tmp_packfiles(&tmpname, pack_tmp_name, written_list, nr_written, &pack_idx_opts, hash, &idx_tmp_name); - rename_tmp_packfile_idx(&tmpname, &idx_tmp_name); if (write_bitmap_index) { size_t tmpname_len = tmpname.len; @@ -1267,6 +1266,8 @@ static void write_pack_file(void) strbuf_setlen(&tmpname, tmpname_len); } + rename_tmp_packfile_idx(&tmpname, &idx_tmp_name); + free(idx_tmp_name); strbuf_release(&tmpname); free(pack_tmp_name);