From patchwork Thu Mar 9 13:09:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13167385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3051C61DA4 for ; Thu, 9 Mar 2023 13:09:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230104AbjCINJh (ORCPT ); Thu, 9 Mar 2023 08:09:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229965AbjCINJf (ORCPT ); Thu, 9 Mar 2023 08:09:35 -0500 Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1242B2449F for ; Thu, 9 Mar 2023 05:09:33 -0800 (PST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 906025C011B for ; Thu, 9 Mar 2023 08:09:31 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Thu, 09 Mar 2023 08:09:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :message-id:mime-version:reply-to:sender:subject:subject:to:to; s=fm1; t=1678367371; x=1678453771; bh=XNnIbWFj7uXrJPNbh97LFvKJZ AAYggJuWibPI81hKVg=; b=YCLmWiAPkofT7tkM+GESV5ynrTzgMsErC8T57wa3Z 44JTlk0cU1B5DurmmuR+0+lqyPxlQoRb7mMIWkju9JMuFr4em4C8IRjXSKV9U7hK 430IlEOuN1kbXxSvqr4Su7OrUjQCj0eeOW/IW+dB5/u5mJ17BwiIsknyOEmUntKh ahNVv+4Fm/ZIgtYHjefeeIhwDWLLDiS89Yjt3udtYNs93Q0MKqrIr6O0Qk3nXNxV wSpth1XhhxicQPTiPMtRX+AW2z18jU4y0pXAU/3FPQod/4pUbqaAjKsX7DB8TgQq Q5kkwFpdXRkO58YNiXqhN7UehwRIdtdwkhbHjI6shPRUg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:message-id :mime-version:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1678367371; x=1678453771; bh=XNnIbWFj7uXrJPNbh97LFvKJZAAYggJuWib PI81hKVg=; b=l3ufzN/ynBAt61IEn0WO6dmApx5cKnmMLn5o+4Iw+TKwI3qKVrC N+PUylziHFbiYs/gSwzg3gDXeEF451i9+BrgNl8EHTq78WJLcpPh/5pWmjKtugYZ +3PZnsTC5erQQo0HJhdPASpt0Ro0R4foBJsPTSFzaOq5wUiLErz3sswvhrWIhp1d FbPuN5qyXUBM8mj44IGkqaQ69y168nskM+Ll0+16hhl6pcZqqlNnWFlbkaVL1UGQ TXQVe+tOSkoQB6Tv7Bfnxn5JSITXiyz7ZqjPdV8a8Cq1bp8RtuR30kDxzBXEEwOA 9ibKCLthdvBdiyaS1rOm0rjKVRGIl8tTAxg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdduiedggeejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkgggtugesghdtreertd dtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhs rdhimheqnecuggftrfgrthhtvghrnhepjeeifedvueelfffgjeduffdvgefhiefgjefgvd dvfeduvefffeevfffhgfekieffnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghm pehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Thu, 9 Mar 2023 08:09:30 -0500 (EST) Received: by pks.im (OpenSMTPD) with ESMTPSA id 081bda30 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Thu, 9 Mar 2023 13:09:17 +0000 (UTC) Date: Thu, 9 Mar 2023 14:09:23 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH] receive-pack: fix stale packfile locks when dying Message-ID: MIME-Version: 1.0 Content-Disposition: inline Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When accepting a packfile in git-receive-pack(1), we feed that packfile into git-index-pack(1) to generate the packfile index. As the packfile would often only contain unreachable objects until the references have been updated, concurrently running garbage collection might be tempted to delete the packfile right away and thus cause corruption. To fix this, we ask git-index-pack(1) to create a `.keep` file before moving the packfile into place, which is getting deleted again once all of the reference updates have been processed. Now in production systems we have observed that those `.keep` files are sometimes not getting deleted as expected, where the result is that repositories tend to grow packfiles that are never deleted over time. This seems to be caused by a race when git-receive-pack(1) is killed after we have migrated the kept packfile from the quarantine directory into the main object database. While this race window is typically small it can be extended for example by installing a `proc-receive` hook. Fix this race by installing an atexit(3P) handler that unlinks the keep file. Signed-off-by: Patrick Steinhardt --- builtin/receive-pack.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index cd5c7a28ef..0a6030d775 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -2186,6 +2186,12 @@ static const char *parse_pack_header(struct pack_header *hdr) static const char *pack_lockfile; +static void unlink_pack_lockfile(void) +{ + if (pack_lockfile) + unlink(pack_lockfile); +} + static void push_header_arg(struct strvec *args, struct pack_header *hdr) { strvec_pushf(args, "--pack_header=%"PRIu32",%"PRIu32, @@ -2281,6 +2287,7 @@ static const char *unpack(int err_fd, struct shallow_info *si) if (status) return "index-pack fork failed"; pack_lockfile = index_pack_lockfile(child.out, NULL); + atexit(unlink_pack_lockfile); close(child.out); status = finish_command(&child); if (status)