From patchwork Mon Mar 4 11:10:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13580440 Received: from wfout1-smtp.messagingengine.com (wfout1-smtp.messagingengine.com [64.147.123.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 558EC38DF5 for ; Mon, 4 Mar 2024 11:11:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709550667; cv=none; b=NwOYRRZh5NSXXD5XdUc3Mvjefo9EzKAlCmYzb1VNStbiHgnwnyUkwNYn41Pc/LCRGQfqkWzwBB5UmwQbH3J2pxGg/7bznMhbta86r/Jp6m9jUAfizm37RXSp0FlKnA8hrUtStnUOvUYu0u+KtPYlaD2+onteALH6K5U4mm47rfU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709550667; c=relaxed/simple; bh=wlBh0bdOl4ViHhEe8h5NncPShgIe1ofNP57aufIg5XI=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=FJplFkKAPxCX5q4HXlU+6H21OpaFOfT2odYhbilXMdk6z8Z84FSOi8+f1jVRhcUgXoW2beJ4jxghUIyWBcUmLg1C12wT12RkzNltOsAEthCefcLh/XC93jtxcGOUOKnVnc6u3oZKdVYLM70jjXf5IDj5Zyy9oeuVWeSwPuiielM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=iYDw1oR7; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=KPORKC+n; arc=none smtp.client-ip=64.147.123.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="iYDw1oR7"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="KPORKC+n" Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfout.west.internal (Postfix) with ESMTP id 8D80F1C00079 for ; Mon, 4 Mar 2024 06:11:02 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Mon, 04 Mar 2024 06:11:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1709550662; x=1709637062; bh=8e2edTElmQ EzH3yRnJcJDvPM90Dtc6xso8I9nLOMgOo=; b=iYDw1oR7lobKbDXQBqFKN77RTM 3abSyQS4LOh/yeAR8lm5jYTk+o9fpr28x5ANoluT6ZodfSC9DyMbHZPa0Vye8AZb 94uydUuDhfSMMyQnyDKrb0Pi/zd6b5rUhRjl9Pbtkba+J5RCo1t+EtNO2aHFkCeH YJYPF9NMj6vi7R5awHTDJ50hSfJTFZsG7F/SrX0g+nPA4xKYQVG3+S1fs8squ0uG zbiri1S6JWIHpl7y0cSYm/BdjwBOHhAf1PMBsumc3zIqOmKhakFbKt83aWOczrDn CH8pj3UFO0Bi9eRGiwyyBQ98+LTmVD5jmvgN5jWmYxvACCbKlaM+bIg/Rc0Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1709550662; x=1709637062; bh=8e2edTElmQEzH3yRnJcJDvPM90Dt c6xso8I9nLOMgOo=; b=KPORKC+nvPqTMNL69HtxMTkmzFRoTkGs2loxJbSngfyG mbbjZEaHnECA1q9E6WxHOVBOfkClmQVNKZwml5Dw78NqZ3CyTYTo+TQTdKgjAK1R 7HQ3eJn1XXnD6v4MbnDLwXAH2s95S+nDcxV6fH/4cBzMmzJrMwEZf4F0dSTEwGXd 5KDS0gVEe6qxQd6u1NMj9+E/7XNO99N8bxiRlIzd9a+qr+7uzTGLfCcdH2AFxIYS 6rNj1S7uvtkn4OJ2g/bRCXS+6TptSZJ2IlgK2kCr8uTx7jbGINH2T59iKkGfnQ/i io6RMTv3CLNhP06Ld5GJJhQI0+dsVN8g47aE5/TmuQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrheejgddvgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtderre dttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhk shdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtdfhvd ejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr mhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Mon, 4 Mar 2024 06:11:01 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id ad151f7a (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Mon, 4 Mar 2024 11:06:36 +0000 (UTC) Date: Mon, 4 Mar 2024 12:10:59 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 4/4] reftable/stack: register compacted tables as tempfiles Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We do not register tables resulting from stack compaction with the tempfile API. Those tables will thus not be deleted in case Git gets killed. Refactor the code to register compacted tables as tempfiles. Signed-off-by: Patrick Steinhardt --- reftable/stack.c | 53 +++++++++++++++++++++++++++--------------------- 1 file changed, 30 insertions(+), 23 deletions(-) diff --git a/reftable/stack.c b/reftable/stack.c index 977336b7d5..40129da16c 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -827,51 +827,57 @@ uint64_t reftable_stack_next_update_index(struct reftable_stack *st) static int stack_compact_locked(struct reftable_stack *st, size_t first, size_t last, - struct strbuf *temp_tab, - struct reftable_log_expiry_config *config) + struct reftable_log_expiry_config *config, + struct tempfile **temp_table_out) { struct strbuf next_name = STRBUF_INIT; - int tab_fd = -1; + struct strbuf table_path = STRBUF_INIT; struct reftable_writer *wr = NULL; + struct tempfile *temp_table; + int temp_table_fd; int err = 0; format_name(&next_name, reftable_reader_min_update_index(st->readers[first]), reftable_reader_max_update_index(st->readers[last])); + stack_filename(&table_path, st, next_name.buf); + strbuf_addstr(&table_path, ".temp.XXXXXX"); - stack_filename(temp_tab, st, next_name.buf); - strbuf_addstr(temp_tab, ".temp.XXXXXX"); + temp_table = mks_tempfile(table_path.buf); + if (!temp_table) { + err = REFTABLE_IO_ERROR; + goto done; + } + temp_table_fd = get_tempfile_fd(temp_table); - tab_fd = mkstemp(temp_tab->buf); if (st->config.default_permissions && - chmod(temp_tab->buf, st->config.default_permissions) < 0) { + chmod(get_tempfile_path(temp_table), st->config.default_permissions) < 0) { err = REFTABLE_IO_ERROR; goto done; } - wr = reftable_new_writer(reftable_fd_write, reftable_fd_flush, &tab_fd, &st->config); - + wr = reftable_new_writer(reftable_fd_write, reftable_fd_flush, + &temp_table_fd, &st->config); err = stack_write_compact(st, wr, first, last, config); if (err < 0) goto done; + err = reftable_writer_close(wr); if (err < 0) goto done; - err = close(tab_fd); - tab_fd = 0; + err = close_tempfile_gently(temp_table); + if (err < 0) + goto done; + + *temp_table_out = temp_table; + temp_table = NULL; done: + delete_tempfile(&temp_table); reftable_writer_free(wr); - if (tab_fd > 0) { - close(tab_fd); - tab_fd = 0; - } - if (err != 0 && temp_tab->len > 0) { - unlink(temp_tab->buf); - strbuf_release(temp_tab); - } strbuf_release(&next_name); + strbuf_release(&table_path); return err; } @@ -979,12 +985,12 @@ static int stack_compact_range(struct reftable_stack *st, struct reftable_log_expiry_config *expiry) { struct strbuf tables_list_buf = STRBUF_INIT; - struct strbuf new_table_temp_path = STRBUF_INIT; struct strbuf new_table_name = STRBUF_INIT; struct strbuf new_table_path = STRBUF_INIT; struct strbuf table_name = STRBUF_INIT; struct lock_file tables_list_lock = LOCK_INIT; struct lock_file *table_locks = NULL; + struct tempfile *new_table = NULL; int is_empty_table = 0, err = 0; size_t i; @@ -1059,7 +1065,7 @@ static int stack_compact_range(struct reftable_stack *st, * these tables may end up with an empty new table in case tombstones * end up cancelling out all refs in that range. */ - err = stack_compact_locked(st, first, last, &new_table_temp_path, expiry); + err = stack_compact_locked(st, first, last, expiry, &new_table); if (err < 0) { if (err != REFTABLE_EMPTY_TABLE_ERROR) goto done; @@ -1099,7 +1105,7 @@ static int stack_compact_range(struct reftable_stack *st, strbuf_addstr(&new_table_name, ".ref"); stack_filename(&new_table_path, st, new_table_name.buf); - err = rename(new_table_temp_path.buf, new_table_path.buf); + err = rename_tempfile(&new_table, new_table_path.buf); if (err < 0) { err = REFTABLE_IO_ERROR; goto done; @@ -1166,9 +1172,10 @@ static int stack_compact_range(struct reftable_stack *st, rollback_lock_file(&table_locks[i - first]); reftable_free(table_locks); + delete_tempfile(&new_table); strbuf_release(&new_table_name); strbuf_release(&new_table_path); - strbuf_release(&new_table_temp_path); + strbuf_release(&tables_list_buf); strbuf_release(&table_name); return err;