From patchwork Mon Mar 4 11:10:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13580437 Received: from wfhigh1-smtp.messagingengine.com (wfhigh1-smtp.messagingengine.com [64.147.123.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7ED8137179 for ; Mon, 4 Mar 2024 11:10:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709550651; cv=none; b=bhByJfZa+XtGGypMN78IgU5LoaXikS48IZB7SGRI5PjEvBGjY8yhKwQIyd1gHmouLFnKt1wGtbILCPI2QkwNa791DVunu0fGDZTZ0Y1L1N8BVXxS8F/HfX3VcNJs7k32T0EO9ddsgZuy1491TKTktnfWtwV8GIvK8KwSm7sCJQE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709550651; c=relaxed/simple; bh=NZpvJQ4ohwnVFW3VV95NsRiKutRgvJc0Dc+RF1XErpc=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Il6Ie5dWym/XKMDDFY02ZFPL4Nqaqut+ugJVYVNmv8PwA3juEsyArL3tHFoJsmU9PS19K6ukricx3RuCroDW/IPvwOupktSPDiExGV4EOgHt39iV/dEcjOEv5r3xEn8tPX0fVpBIfzvNstdXvGNF7CzYVrpbBeORWC1SR3M/6Ns= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=D5SFk/WH; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=RfQnwQIk; arc=none smtp.client-ip=64.147.123.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="D5SFk/WH"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="RfQnwQIk" Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfhigh.west.internal (Postfix) with ESMTP id A164E18000CB for ; Mon, 4 Mar 2024 06:10:48 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute7.internal (MEProxy); Mon, 04 Mar 2024 06:10:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1709550648; x=1709637048; bh=rMFmd8gHhc ml/ZrtHZOV59LEdgAMtl7tUQro5dcHmoo=; b=D5SFk/WHN+LOvm/4qZ0/nZVLKR 2ERm1ETQUCRywdcnDemdQ779NVoQrKswwfaAnkHrc2LSXCxd8ZuzW0Gh8GxOu5FI KS7UubammhA7+iZeeGPMX2t6F5fXpBaDET7d+hK5E2rKnkdqFUgpuOI0A79etJor pyWrUuLPcMuM/LOQXYHZ2jPFOPaQt4j663+ko0OrGPrA8U8Cv55m/zV6+ZTAoqF3 LdoP2K49Em9QOjlyqTfH93kLoMn/hSMD+BBpEEvKMjFA3kPYmNRWtUCS173oL6Q1 FbehlYZ6rwM0piYSWUqi+V7f/7zg6HBfJeMvk8Acs8k0F9/6jUpE7R9Xf+tw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1709550648; x=1709637048; bh=rMFmd8gHhcml/ZrtHZOV59LEdgAM tl7tUQro5dcHmoo=; b=RfQnwQIkKfKN3fJ5F0cCD0+nVCiOdXZP/9OVDmsfHDfR x+BuyFzitqmQEWxjTXBmsPlgeCSS/yxzDbDiNtXcOsS8pvaMC2YVHB4x5womAZIa v3Fmko8ulOHJRjaff7qgO58XJob0pXkVhqtWo4GOZZzbRwTI5NtjHsfZ93Hv3UCR 48K9op4w+3/plzEv57mcN2vJX8dqp0jgwmQZkCi69pfoJqNOoh/Ba4m/Rg6UQiJF 4SL1UtR+nvN1b6I8OTJNF7d+eygBMwjZY1HHg1lknjThSxeQeMfAQOy22MnsWxKQ P5DDYy+o/0kpgjNyiCGzOipvqEDli7+UcX3+qkyNqQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrheejgddvgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtderre dttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhk shdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtdfhvd ejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr mhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Mon, 4 Mar 2024 06:10:47 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id a10b4545 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Mon, 4 Mar 2024 11:06:22 +0000 (UTC) Date: Mon, 4 Mar 2024 12:10:45 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 1/4] lockfile: report when rollback fails Message-ID: <1acaa9ca1a53e3d8e875fed7ce961d0b5426688f.1709549619.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We do not report to the caller when rolling back a lockfile fails, which will be needed by the reftable compaction logic in a subsequent commit. It also cannot really report on all errors because the function calls `delete_tempfile()`, which doesn't return an error either. Refactor the code so that both `delete_tempfile()` and `rollback_lock_file()` return an error code. Signed-off-by: Patrick Steinhardt --- lockfile.h | 4 ++-- tempfile.c | 21 +++++++++++++-------- tempfile.h | 2 +- 3 files changed, 16 insertions(+), 11 deletions(-) diff --git a/lockfile.h b/lockfile.h index 90af4e66b2..4ed570d3f7 100644 --- a/lockfile.h +++ b/lockfile.h @@ -323,9 +323,9 @@ static inline int commit_lock_file_to(struct lock_file *lk, const char *path) * for a `lock_file` object that has already been committed or rolled * back. */ -static inline void rollback_lock_file(struct lock_file *lk) +static inline int rollback_lock_file(struct lock_file *lk) { - delete_tempfile(&lk->tempfile); + return delete_tempfile(&lk->tempfile); } #endif /* LOCKFILE_H */ diff --git a/tempfile.c b/tempfile.c index ecdebf1afb..ed88cf8431 100644 --- a/tempfile.c +++ b/tempfile.c @@ -50,15 +50,17 @@ static VOLATILE_LIST_HEAD(tempfile_list); -static void remove_template_directory(struct tempfile *tempfile, +static int remove_template_directory(struct tempfile *tempfile, int in_signal_handler) { if (tempfile->directory) { if (in_signal_handler) - rmdir(tempfile->directory); + return rmdir(tempfile->directory); else - rmdir_or_warn(tempfile->directory); + return rmdir_or_warn(tempfile->directory); } + + return 0; } static void remove_tempfiles(int in_signal_handler) @@ -353,16 +355,19 @@ int rename_tempfile(struct tempfile **tempfile_p, const char *path) return 0; } -void delete_tempfile(struct tempfile **tempfile_p) +int delete_tempfile(struct tempfile **tempfile_p) { struct tempfile *tempfile = *tempfile_p; + int err = 0; if (!is_tempfile_active(tempfile)) - return; + return 0; - close_tempfile_gently(tempfile); - unlink_or_warn(tempfile->filename.buf); - remove_template_directory(tempfile, 0); + err |= close_tempfile_gently(tempfile); + err |= unlink_or_warn(tempfile->filename.buf); + err |= remove_template_directory(tempfile, 0); deactivate_tempfile(tempfile); *tempfile_p = NULL; + + return err ? -1 : 0; } diff --git a/tempfile.h b/tempfile.h index d0413af733..2d2ae5b657 100644 --- a/tempfile.h +++ b/tempfile.h @@ -269,7 +269,7 @@ int reopen_tempfile(struct tempfile *tempfile); * `delete_tempfile()` for a `tempfile` object that has already been * deleted or renamed. */ -void delete_tempfile(struct tempfile **tempfile_p); +int delete_tempfile(struct tempfile **tempfile_p); /* * Close the file descriptor and/or file pointer if they are still From patchwork Mon Mar 4 11:10:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13580438 Received: from wfout1-smtp.messagingengine.com (wfout1-smtp.messagingengine.com [64.147.123.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 21B9F3838B for ; Mon, 4 Mar 2024 11:10:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709550656; cv=none; b=nVRldvdMEEXRPcOK86I1Wcja8RQJawCzTgmEvqwoShhX927e6J6n39n/8dmmlgjSIkJLpj/1bxEO8rNUYK/jbbhoDaeDsM3VD6a/tVE0zyVRCS2mjwsRKlg/k3ah8I/1Q6+/efLCViOiNVzZYJx9VfsWGxMtTZUGQQNZ1vToN4A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709550656; c=relaxed/simple; bh=fIz8TCCjFB1EQc1EOUJXiv88tkU/WVoNl4/RVzGl2ds=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=LNM0wqwkUPqfmTxuCj5Qohbqr+dB7HNAoYy1A78Mow0kWYdu+CQ4uq+ZiTPVs80wyKJjgqBEXBAwrqZg0dgHWp1MYeLpIncXVy6EloMjJtN0rYD+lbHkezp+ktGYANyl0kefw5t9CLN/VbxyoMik78M8fDy4JXD6650sbbDwcjk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=XlgeAxmP; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=eC7HjTbP; arc=none smtp.client-ip=64.147.123.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="XlgeAxmP"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="eC7HjTbP" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfout.west.internal (Postfix) with ESMTP id 4AB381C0009C for ; Mon, 4 Mar 2024 06:10:54 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 04 Mar 2024 06:10:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1709550653; x=1709637053; bh=6SPLdU7tvj YYRAlufn0tMJULprNxta/hFSHlaVCY2aU=; b=XlgeAxmP1UHrRx/oNkNU0uzPmI Pk8Dy8gjIjU03CxJItH2HZaWm3BH3+VFrXYXUDsA/ZGgXNps8myO0B13MIs8ketG 4kO+XQhqhCduhSar9c+06PQ+iWD959kIKwxSZHlgl+JB9iCSnPa0xqnwlYCYtJw8 cDTc65W0AxtvjuY5bFtHiWzVe1q5Lvz5q9InwgJTP6hfchIxaU4jprKJGBD+pI8N 8DNNK2eEFNTbwusVbgCpGyJY30tDlyGtga/tp2U6/ptUeuNFisD9fY0nOsEp+sZC zUp7ZvlbYBIil+DfJS10TyBzEdZ2ycYzXBN6LkcA4G4/zRPntsVHl9HTLMGQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1709550653; x=1709637053; bh=6SPLdU7tvjYYRAlufn0tMJULprNx ta/hFSHlaVCY2aU=; b=eC7HjTbPVBn4cLcyGfOUbsK125mB7hXqkxyfrAu+vSFt DRz9zaSwufhiUw6t21FJ3RnCGcSwFse6RwJ+mXnTe1r+jKtQqxekeJpuLoEOxbqM wVp1DlrUpSCaJ/O7wDOOmsvTTsKLX5Gd1zeUtIsEOA9S8fMYKtdd8y64wilbBoNL gCiXiU3ucPgT76NNslFaMcJFlO5mkjVjmY5hgu9UzFngyOir2yq5pA6iwdaako9L iKYGRQDQS4RDcSZGQfoAwnOZTAFCoUGy6r3W0XD3InVd+zvBhqya3WRf4/whsFfa cbJhHIzz+80mW2NPjgO37dIHqViHE2P7xYk295pgew== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrheejgddvgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtderre dttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhk shdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtdfhvd ejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr mhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Mon, 4 Mar 2024 06:10:53 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 4c068f74 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Mon, 4 Mar 2024 11:06:28 +0000 (UTC) Date: Mon, 4 Mar 2024 12:10:51 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 2/4] reftable/stack: register new tables as tempfiles Message-ID: <02bf41d419efd00e510a89a405e1b046b166ba20.1709549619.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We do not register new tables which we're about to add to the stack with the tempfile API. Those tables will thus not be deleted in case Git gets killed. Refactor the code to register tables as tempfiles. Signed-off-by: Patrick Steinhardt --- reftable/stack.c | 29 ++++++++++++----------------- 1 file changed, 12 insertions(+), 17 deletions(-) diff --git a/reftable/stack.c b/reftable/stack.c index b64e55648a..81544fbfa0 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -737,8 +737,9 @@ int reftable_addition_add(struct reftable_addition *add, struct strbuf tab_file_name = STRBUF_INIT; struct strbuf next_name = STRBUF_INIT; struct reftable_writer *wr = NULL; + struct tempfile *tab_file = NULL; int err = 0; - int tab_fd = 0; + int tab_fd; strbuf_reset(&next_name); format_name(&next_name, add->next_update_index, add->next_update_index); @@ -746,17 +747,20 @@ int reftable_addition_add(struct reftable_addition *add, stack_filename(&temp_tab_file_name, add->stack, next_name.buf); strbuf_addstr(&temp_tab_file_name, ".temp.XXXXXX"); - tab_fd = mkstemp(temp_tab_file_name.buf); - if (tab_fd < 0) { + tab_file = mks_tempfile(temp_tab_file_name.buf); + if (!tab_file) { err = REFTABLE_IO_ERROR; goto done; } if (add->stack->config.default_permissions) { - if (chmod(temp_tab_file_name.buf, add->stack->config.default_permissions)) { + if (chmod(get_tempfile_path(tab_file), + add->stack->config.default_permissions)) { err = REFTABLE_IO_ERROR; goto done; } } + tab_fd = get_tempfile_fd(tab_file); + wr = reftable_new_writer(reftable_fd_write, reftable_fd_flush, &tab_fd, &add->stack->config); err = write_table(wr, arg); @@ -771,14 +775,13 @@ int reftable_addition_add(struct reftable_addition *add, if (err < 0) goto done; - err = close(tab_fd); - tab_fd = 0; + err = close_tempfile_gently(tab_file); if (err < 0) { err = REFTABLE_IO_ERROR; goto done; } - err = stack_check_addition(add->stack, temp_tab_file_name.buf); + err = stack_check_addition(add->stack, get_tempfile_path(tab_file)); if (err < 0) goto done; @@ -789,14 +792,13 @@ int reftable_addition_add(struct reftable_addition *add, format_name(&next_name, wr->min_update_index, wr->max_update_index); strbuf_addstr(&next_name, ".ref"); - stack_filename(&tab_file_name, add->stack, next_name.buf); /* On windows, this relies on rand() picking a unique destination name. Maybe we should do retry loop as well? */ - err = rename(temp_tab_file_name.buf, tab_file_name.buf); + err = rename_tempfile(&tab_file, tab_file_name.buf); if (err < 0) { err = REFTABLE_IO_ERROR; goto done; @@ -806,14 +808,7 @@ int reftable_addition_add(struct reftable_addition *add, add->new_tables_cap); add->new_tables[add->new_tables_len++] = strbuf_detach(&next_name, NULL); done: - if (tab_fd > 0) { - close(tab_fd); - tab_fd = 0; - } - if (temp_tab_file_name.len > 0) { - unlink(temp_tab_file_name.buf); - } - + delete_tempfile(&tab_file); strbuf_release(&temp_tab_file_name); strbuf_release(&tab_file_name); strbuf_release(&next_name); From patchwork Mon Mar 4 11:10:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13580439 Received: from wfhigh1-smtp.messagingengine.com (wfhigh1-smtp.messagingengine.com [64.147.123.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B355338DF5 for ; Mon, 4 Mar 2024 11:10:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709550661; cv=none; b=RXSL3vx9N3R30cawBv6jKKMJi0zARfBG/vWrY5dqLut+HkBBVxs7jYx0aVtdfpWW1sMvJPXZ2Gq01Vej9pOgYAMujGHiYbvA/tafCW0kJtXeIpBPEIJlBK+XrcGEBPDE+L/MercH7yIDY8nSCHtvV+W0earM7g1lJA29oTqFL8M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709550661; c=relaxed/simple; bh=0+RGmJ9URKT3RmLAlGpE7xbVjcrJ+IKKtw1dkemQ6aY=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=WB+gUNta4stNyZaCuvwMnbODasyHnK/mgLFSMqMTPCSm4P/9EFScTNj2O9gZhv458SE3J7EC5kw61ffP+vBwlNZJPTZVIbCz+jm0f6IH9x/6HaTJ1xJkre94z9AoFU0nKXsmeSIvI+1bFt9bDfz/0yT2bhz5BpfAyTDnyTi+UT0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=UND4sFKt; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=jiSySMaR; arc=none smtp.client-ip=64.147.123.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="UND4sFKt"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="jiSySMaR" Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfhigh.west.internal (Postfix) with ESMTP id EA85218000CC for ; Mon, 4 Mar 2024 06:10:58 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute7.internal (MEProxy); Mon, 04 Mar 2024 06:10:59 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1709550658; x=1709637058; bh=4eD4/nUXB5 Vi2wD4z8TBHizHXifrBG8YvaPxLXS4xNc=; b=UND4sFKt5aRzXVhrPgMoCzE6yS T9rMXBxxMrHGCCVOZ0A8gq35vvn/Sv+NVCpscXZHAv/EllP5yYKfOh++1AA1Bsc4 /qog0XCeeWkkXP5vKi/MaKdDjYIBpPVqQvKkW1fXj0QmXjuWh/RfwJA3Q0x9v3CF z7FIV53xAc3ousEc8HKjC8QwAaa1XlJ4kwtLKzgF8/JVzJDCAfbEhsX5e50U1Q/4 2zhvgPAHS3LcowD1+hLNxV/SlQALnp1G1ArG/q/Ehqex+olXD52MDxEsmpdk3k4y xz2uio32RLHOFYix/9OlCA9JXtq7RG+Wpzn0cdAz66TtP6npARQ43fzOhftQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1709550658; x=1709637058; bh=4eD4/nUXB5Vi2wD4z8TBHizHXifr BG8YvaPxLXS4xNc=; b=jiSySMaRI+HBOxYtFtw4Heuk4M2HhJvIF4E1RQjgDrXG iduZElGcgy0QxpWeNnm95diOod5CuXZtpCwNcJImnY6FzbK1Do7ziYJXgYPO0M+8 mzwN4QKSr4D4emofnZ1++bwMLDBYQPePuSHifY7mVbQIQxvtaXt5zKzuQyKKJag9 J/rVOSQQhxlpigz/R8D5uzMHlNGGXFxjhVIJV1QpEPuO8c4x1GD+l0CA77m5hKnx re15uFfYxZ3Ml01tqLnv77LJXTvqX0QZ7saaie8piTA9r0uUp4vEcHDzR6OkpYeN mrQWk/KqnOxQ092h9k6NN5h2mCQHTupZwT5avvB9yw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrheejgddvgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtderre dttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhk shdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtdfhvd ejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedunecurfgrrhgr mhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Mon, 4 Mar 2024 06:10:57 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 6d2f64b7 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Mon, 4 Mar 2024 11:06:32 +0000 (UTC) Date: Mon, 4 Mar 2024 12:10:55 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 3/4] reftable/stack: register lockfiles during compaction Message-ID: <45b5c3167fbfd64d8d1e14ed55bae94cb9cba28b.1709549619.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We do not register any of the locks we acquire when compacting the reftable stack via our lockfiles interfaces. These locks will thus not be released when Git gets killed. Refactor the code to register locks as lockfiles. Signed-off-by: Patrick Steinhardt --- reftable/stack.c | 255 ++++++++++++++++++++++------------------------ reftable/system.h | 2 + 2 files changed, 123 insertions(+), 134 deletions(-) diff --git a/reftable/stack.c b/reftable/stack.c index 81544fbfa0..977336b7d5 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -978,212 +978,199 @@ static int stack_compact_range(struct reftable_stack *st, size_t first, size_t last, struct reftable_log_expiry_config *expiry) { - char **delete_on_success = NULL, **subtable_locks = NULL, **listp = NULL; - struct strbuf temp_tab_file_name = STRBUF_INIT; + struct strbuf tables_list_buf = STRBUF_INIT; + struct strbuf new_table_temp_path = STRBUF_INIT; struct strbuf new_table_name = STRBUF_INIT; - struct strbuf lock_file_name = STRBUF_INIT; - struct strbuf ref_list_contents = STRBUF_INIT; struct strbuf new_table_path = STRBUF_INIT; - size_t i, j, compact_count; - int err = 0; - int have_lock = 0; - int lock_file_fd = -1; - int is_empty_table = 0; + struct strbuf table_name = STRBUF_INIT; + struct lock_file tables_list_lock = LOCK_INIT; + struct lock_file *table_locks = NULL; + int is_empty_table = 0, err = 0; + size_t i; if (first > last || (!expiry && first == last)) { err = 0; goto done; } - compact_count = last - first + 1; - REFTABLE_CALLOC_ARRAY(delete_on_success, compact_count + 1); - REFTABLE_CALLOC_ARRAY(subtable_locks, compact_count + 1); - st->stats.attempts++; - strbuf_reset(&lock_file_name); - strbuf_addstr(&lock_file_name, st->list_file); - strbuf_addstr(&lock_file_name, ".lock"); - - lock_file_fd = - open(lock_file_name.buf, O_EXCL | O_CREAT | O_WRONLY, 0666); - if (lock_file_fd < 0) { - if (errno == EEXIST) { + /* + * Hold the lock so that we can read "tables.list" and lock all tables + * which are part of the user-specified range. + */ + err = hold_lock_file_for_update(&tables_list_lock, st->list_file, + LOCK_NO_DEREF); + if (err < 0) { + if (errno == EEXIST) err = 1; - } else { + else err = REFTABLE_IO_ERROR; - } goto done; } - /* Don't want to write to the lock for now. */ - close(lock_file_fd); - lock_file_fd = -1; - have_lock = 1; err = stack_uptodate(st); - if (err != 0) + if (err) goto done; - for (i = first, j = 0; i <= last; i++) { - struct strbuf subtab_file_name = STRBUF_INIT; - struct strbuf subtab_lock = STRBUF_INIT; - int sublock_file_fd = -1; - - stack_filename(&subtab_file_name, st, - reader_name(st->readers[i])); - - strbuf_reset(&subtab_lock); - strbuf_addbuf(&subtab_lock, &subtab_file_name); - strbuf_addstr(&subtab_lock, ".lock"); + /* + * Lock all tables in the user-provided range. This is the slice of our + * stack which we'll compact. + */ + REFTABLE_CALLOC_ARRAY(table_locks, last - first + 1); + for (i = first; i <= last; i++) { + stack_filename(&table_name, st, reader_name(st->readers[i])); - sublock_file_fd = open(subtab_lock.buf, - O_EXCL | O_CREAT | O_WRONLY, 0666); - if (sublock_file_fd >= 0) { - close(sublock_file_fd); - } else if (sublock_file_fd < 0) { - if (errno == EEXIST) { + err = hold_lock_file_for_update(&table_locks[i - first], + table_name.buf, LOCK_NO_DEREF); + if (err < 0) { + if (errno == EEXIST) err = 1; - } else { + else err = REFTABLE_IO_ERROR; - } + goto done; } - subtable_locks[j] = subtab_lock.buf; - delete_on_success[j] = subtab_file_name.buf; - j++; - - if (err != 0) + /* + * We need to close the lockfiles as we might otherwise easily + * run into file descriptor exhaustion when we compress a lot + * of tables. + */ + err = close_lock_file_gently(&table_locks[i - first]); + if (err < 0) { + err = REFTABLE_IO_ERROR; goto done; + } } - err = unlink(lock_file_name.buf); - if (err < 0) + /* + * We have locked all tables in our range and can thus release the + * "tables.list" lock while compacting the locked tables. This allows + * concurrent updates to the stack to proceed. + */ + err = rollback_lock_file(&tables_list_lock); + if (err < 0) { + err = REFTABLE_IO_ERROR; goto done; - have_lock = 0; - - err = stack_compact_locked(st, first, last, &temp_tab_file_name, - expiry); - /* Compaction + tombstones can create an empty table out of non-empty - * tables. */ - is_empty_table = (err == REFTABLE_EMPTY_TABLE_ERROR); - if (is_empty_table) { - err = 0; } - if (err < 0) - goto done; - lock_file_fd = - open(lock_file_name.buf, O_EXCL | O_CREAT | O_WRONLY, 0666); - if (lock_file_fd < 0) { - if (errno == EEXIST) { + /* + * Compact the now-locked tables into a new table. Note that compacting + * these tables may end up with an empty new table in case tombstones + * end up cancelling out all refs in that range. + */ + err = stack_compact_locked(st, first, last, &new_table_temp_path, expiry); + if (err < 0) { + if (err != REFTABLE_EMPTY_TABLE_ERROR) + goto done; + is_empty_table = 1; + } + + /* + * Now that we have written the new, compacted table we need to re-lock + * "tables.list". We'll then replace the compacted range of tables with + * the new table. + */ + err = hold_lock_file_for_update(&tables_list_lock, st->list_file, + LOCK_NO_DEREF); + if (err < 0) { + if (errno == EEXIST) err = 1; - } else { + else err = REFTABLE_IO_ERROR; - } goto done; } - have_lock = 1; + if (st->config.default_permissions) { - if (chmod(lock_file_name.buf, st->config.default_permissions) < 0) { + if (chmod(get_lock_file_path(&tables_list_lock), + st->config.default_permissions) < 0) { err = REFTABLE_IO_ERROR; goto done; } } - format_name(&new_table_name, st->readers[first]->min_update_index, - st->readers[last]->max_update_index); - strbuf_addstr(&new_table_name, ".ref"); - - stack_filename(&new_table_path, st, new_table_name.buf); - + /* + * If the resulting compacted table is not empty, then we need to move + * it into place now. + */ if (!is_empty_table) { - /* retry? */ - err = rename(temp_tab_file_name.buf, new_table_path.buf); + format_name(&new_table_name, st->readers[first]->min_update_index, + st->readers[last]->max_update_index); + strbuf_addstr(&new_table_name, ".ref"); + stack_filename(&new_table_path, st, new_table_name.buf); + + err = rename(new_table_temp_path.buf, new_table_path.buf); if (err < 0) { err = REFTABLE_IO_ERROR; goto done; } } - for (i = 0; i < first; i++) { - strbuf_addstr(&ref_list_contents, st->readers[i]->name); - strbuf_addstr(&ref_list_contents, "\n"); - } - if (!is_empty_table) { - strbuf_addbuf(&ref_list_contents, &new_table_name); - strbuf_addstr(&ref_list_contents, "\n"); - } - for (i = last + 1; i < st->merged->stack_len; i++) { - strbuf_addstr(&ref_list_contents, st->readers[i]->name); - strbuf_addstr(&ref_list_contents, "\n"); - } - - err = write_in_full(lock_file_fd, ref_list_contents.buf, ref_list_contents.len); - if (err < 0) { - err = REFTABLE_IO_ERROR; - unlink(new_table_path.buf); - goto done; - } - - err = fsync_component(FSYNC_COMPONENT_REFERENCE, lock_file_fd); + /* + * Write the new "tables.list" contents with the compacted table we + * have just written. In case the compacted table became empty we + * simply skip writing it. + */ + for (i = 0; i < first; i++) + strbuf_addf(&tables_list_buf, "%s\n", st->readers[i]->name); + if (!is_empty_table) + strbuf_addf(&tables_list_buf, "%s\n", new_table_name.buf); + for (i = last + 1; i < st->merged->stack_len; i++) + strbuf_addf(&tables_list_buf, "%s\n", st->readers[i]->name); + + err = write_in_full(get_lock_file_fd(&tables_list_lock), + tables_list_buf.buf, tables_list_buf.len); if (err < 0) { err = REFTABLE_IO_ERROR; unlink(new_table_path.buf); goto done; } - err = close(lock_file_fd); - lock_file_fd = -1; + err = fsync_component(FSYNC_COMPONENT_REFERENCE, get_lock_file_fd(&tables_list_lock)); if (err < 0) { err = REFTABLE_IO_ERROR; unlink(new_table_path.buf); goto done; } - err = rename(lock_file_name.buf, st->list_file); + err = commit_lock_file(&tables_list_lock); if (err < 0) { err = REFTABLE_IO_ERROR; unlink(new_table_path.buf); goto done; } - have_lock = 0; - /* Reload the stack before deleting. On windows, we can only delete the - files after we closed them. - */ + /* + * Reload the stack before deleting the compacted tables. We can only + * delete the files after we closed them on Windows, so this needs to + * happen first. + */ err = reftable_stack_reload_maybe_reuse(st, first < last); + if (err < 0) + goto done; - listp = delete_on_success; - while (*listp) { - if (strcmp(*listp, new_table_path.buf)) { - unlink(*listp); - } - listp++; + /* + * Delete the old tables. They may still be in use by concurrent + * readers, so it is expected that unlinking tables may fail. + */ + for (i = first; i <= last; i++) { + struct lock_file *table_lock = &table_locks[i - first]; + char *table_path = get_locked_file_path(table_lock); + unlink(table_path); + free(table_path); } done: - free_names(delete_on_success); + rollback_lock_file(&tables_list_lock); + for (i = first; table_locks && i <= last; i++) + rollback_lock_file(&table_locks[i - first]); + reftable_free(table_locks); - if (subtable_locks) { - listp = subtable_locks; - while (*listp) { - unlink(*listp); - listp++; - } - free_names(subtable_locks); - } - if (lock_file_fd >= 0) { - close(lock_file_fd); - lock_file_fd = -1; - } - if (have_lock) { - unlink(lock_file_name.buf); - } strbuf_release(&new_table_name); strbuf_release(&new_table_path); - strbuf_release(&ref_list_contents); - strbuf_release(&temp_tab_file_name); - strbuf_release(&lock_file_name); + strbuf_release(&new_table_temp_path); + strbuf_release(&tables_list_buf); + strbuf_release(&table_name); return err; } diff --git a/reftable/system.h b/reftable/system.h index 6b74a81514..5d8b6dede5 100644 --- a/reftable/system.h +++ b/reftable/system.h @@ -12,7 +12,9 @@ license that can be found in the LICENSE file or at /* This header glues the reftable library to the rest of Git */ #include "git-compat-util.h" +#include "lockfile.h" #include "strbuf.h" +#include "tempfile.h" #include "hash-ll.h" /* hash ID, sizes.*/ #include "dir.h" /* remove_dir_recursively, for tests.*/ From patchwork Mon Mar 4 11:10:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13580440 Received: from wfout1-smtp.messagingengine.com (wfout1-smtp.messagingengine.com [64.147.123.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 558EC38DF5 for ; Mon, 4 Mar 2024 11:11:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709550667; cv=none; b=NwOYRRZh5NSXXD5XdUc3Mvjefo9EzKAlCmYzb1VNStbiHgnwnyUkwNYn41Pc/LCRGQfqkWzwBB5UmwQbH3J2pxGg/7bznMhbta86r/Jp6m9jUAfizm37RXSp0FlKnA8hrUtStnUOvUYu0u+KtPYlaD2+onteALH6K5U4mm47rfU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709550667; c=relaxed/simple; bh=wlBh0bdOl4ViHhEe8h5NncPShgIe1ofNP57aufIg5XI=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=FJplFkKAPxCX5q4HXlU+6H21OpaFOfT2odYhbilXMdk6z8Z84FSOi8+f1jVRhcUgXoW2beJ4jxghUIyWBcUmLg1C12wT12RkzNltOsAEthCefcLh/XC93jtxcGOUOKnVnc6u3oZKdVYLM70jjXf5IDj5Zyy9oeuVWeSwPuiielM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=iYDw1oR7; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=KPORKC+n; arc=none smtp.client-ip=64.147.123.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="iYDw1oR7"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="KPORKC+n" Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfout.west.internal (Postfix) with ESMTP id 8D80F1C00079 for ; Mon, 4 Mar 2024 06:11:02 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Mon, 04 Mar 2024 06:11:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1709550662; x=1709637062; bh=8e2edTElmQ EzH3yRnJcJDvPM90Dtc6xso8I9nLOMgOo=; b=iYDw1oR7lobKbDXQBqFKN77RTM 3abSyQS4LOh/yeAR8lm5jYTk+o9fpr28x5ANoluT6ZodfSC9DyMbHZPa0Vye8AZb 94uydUuDhfSMMyQnyDKrb0Pi/zd6b5rUhRjl9Pbtkba+J5RCo1t+EtNO2aHFkCeH YJYPF9NMj6vi7R5awHTDJ50hSfJTFZsG7F/SrX0g+nPA4xKYQVG3+S1fs8squ0uG zbiri1S6JWIHpl7y0cSYm/BdjwBOHhAf1PMBsumc3zIqOmKhakFbKt83aWOczrDn CH8pj3UFO0Bi9eRGiwyyBQ98+LTmVD5jmvgN5jWmYxvACCbKlaM+bIg/Rc0Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1709550662; x=1709637062; bh=8e2edTElmQEzH3yRnJcJDvPM90Dt c6xso8I9nLOMgOo=; b=KPORKC+nvPqTMNL69HtxMTkmzFRoTkGs2loxJbSngfyG mbbjZEaHnECA1q9E6WxHOVBOfkClmQVNKZwml5Dw78NqZ3CyTYTo+TQTdKgjAK1R 7HQ3eJn1XXnD6v4MbnDLwXAH2s95S+nDcxV6fH/4cBzMmzJrMwEZf4F0dSTEwGXd 5KDS0gVEe6qxQd6u1NMj9+E/7XNO99N8bxiRlIzd9a+qr+7uzTGLfCcdH2AFxIYS 6rNj1S7uvtkn4OJ2g/bRCXS+6TptSZJ2IlgK2kCr8uTx7jbGINH2T59iKkGfnQ/i io6RMTv3CLNhP06Ld5GJJhQI0+dsVN8g47aE5/TmuQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrheejgddvgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtderre dttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhk shdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtdfhvd ejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr mhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Mon, 4 Mar 2024 06:11:01 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id ad151f7a (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Mon, 4 Mar 2024 11:06:36 +0000 (UTC) Date: Mon, 4 Mar 2024 12:10:59 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 4/4] reftable/stack: register compacted tables as tempfiles Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We do not register tables resulting from stack compaction with the tempfile API. Those tables will thus not be deleted in case Git gets killed. Refactor the code to register compacted tables as tempfiles. Signed-off-by: Patrick Steinhardt --- reftable/stack.c | 53 +++++++++++++++++++++++++++--------------------- 1 file changed, 30 insertions(+), 23 deletions(-) diff --git a/reftable/stack.c b/reftable/stack.c index 977336b7d5..40129da16c 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -827,51 +827,57 @@ uint64_t reftable_stack_next_update_index(struct reftable_stack *st) static int stack_compact_locked(struct reftable_stack *st, size_t first, size_t last, - struct strbuf *temp_tab, - struct reftable_log_expiry_config *config) + struct reftable_log_expiry_config *config, + struct tempfile **temp_table_out) { struct strbuf next_name = STRBUF_INIT; - int tab_fd = -1; + struct strbuf table_path = STRBUF_INIT; struct reftable_writer *wr = NULL; + struct tempfile *temp_table; + int temp_table_fd; int err = 0; format_name(&next_name, reftable_reader_min_update_index(st->readers[first]), reftable_reader_max_update_index(st->readers[last])); + stack_filename(&table_path, st, next_name.buf); + strbuf_addstr(&table_path, ".temp.XXXXXX"); - stack_filename(temp_tab, st, next_name.buf); - strbuf_addstr(temp_tab, ".temp.XXXXXX"); + temp_table = mks_tempfile(table_path.buf); + if (!temp_table) { + err = REFTABLE_IO_ERROR; + goto done; + } + temp_table_fd = get_tempfile_fd(temp_table); - tab_fd = mkstemp(temp_tab->buf); if (st->config.default_permissions && - chmod(temp_tab->buf, st->config.default_permissions) < 0) { + chmod(get_tempfile_path(temp_table), st->config.default_permissions) < 0) { err = REFTABLE_IO_ERROR; goto done; } - wr = reftable_new_writer(reftable_fd_write, reftable_fd_flush, &tab_fd, &st->config); - + wr = reftable_new_writer(reftable_fd_write, reftable_fd_flush, + &temp_table_fd, &st->config); err = stack_write_compact(st, wr, first, last, config); if (err < 0) goto done; + err = reftable_writer_close(wr); if (err < 0) goto done; - err = close(tab_fd); - tab_fd = 0; + err = close_tempfile_gently(temp_table); + if (err < 0) + goto done; + + *temp_table_out = temp_table; + temp_table = NULL; done: + delete_tempfile(&temp_table); reftable_writer_free(wr); - if (tab_fd > 0) { - close(tab_fd); - tab_fd = 0; - } - if (err != 0 && temp_tab->len > 0) { - unlink(temp_tab->buf); - strbuf_release(temp_tab); - } strbuf_release(&next_name); + strbuf_release(&table_path); return err; } @@ -979,12 +985,12 @@ static int stack_compact_range(struct reftable_stack *st, struct reftable_log_expiry_config *expiry) { struct strbuf tables_list_buf = STRBUF_INIT; - struct strbuf new_table_temp_path = STRBUF_INIT; struct strbuf new_table_name = STRBUF_INIT; struct strbuf new_table_path = STRBUF_INIT; struct strbuf table_name = STRBUF_INIT; struct lock_file tables_list_lock = LOCK_INIT; struct lock_file *table_locks = NULL; + struct tempfile *new_table = NULL; int is_empty_table = 0, err = 0; size_t i; @@ -1059,7 +1065,7 @@ static int stack_compact_range(struct reftable_stack *st, * these tables may end up with an empty new table in case tombstones * end up cancelling out all refs in that range. */ - err = stack_compact_locked(st, first, last, &new_table_temp_path, expiry); + err = stack_compact_locked(st, first, last, expiry, &new_table); if (err < 0) { if (err != REFTABLE_EMPTY_TABLE_ERROR) goto done; @@ -1099,7 +1105,7 @@ static int stack_compact_range(struct reftable_stack *st, strbuf_addstr(&new_table_name, ".ref"); stack_filename(&new_table_path, st, new_table_name.buf); - err = rename(new_table_temp_path.buf, new_table_path.buf); + err = rename_tempfile(&new_table, new_table_path.buf); if (err < 0) { err = REFTABLE_IO_ERROR; goto done; @@ -1166,9 +1172,10 @@ static int stack_compact_range(struct reftable_stack *st, rollback_lock_file(&table_locks[i - first]); reftable_free(table_locks); + delete_tempfile(&new_table); strbuf_release(&new_table_name); strbuf_release(&new_table_path); - strbuf_release(&new_table_temp_path); + strbuf_release(&tables_list_buf); strbuf_release(&table_name); return err;