From patchwork Wed Jul 31 14:14:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13748833 Received: from fout8-smtp.messagingengine.com (fout8-smtp.messagingengine.com [103.168.172.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8C0B1B375C for ; Wed, 31 Jul 2024 14:15:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.151 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722435302; cv=none; b=LVV4tcy2VOkTtWf/K84sXx4g0DkUnPpt44Iu060FbJzNNd+GSu1qlPofykgdxz+JYNOyZSELkjxfAaPSAWS1UH321WWMkGvy7EJk3XAeI7CBW4nrwnlYr0LACHYq47rTiOkjlILJVG3RBRVzk7ril177+Lght4SkeOaq8l2Rkv4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722435302; c=relaxed/simple; bh=ygkhdS004+btHxYp9hjaisI/UZJm7p/hK9WjtY/NvZk=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=UqE1aOza1H/2v18IjZKFcymMN8YQNyKRITY2UTe4DwcKU9IAb9FxSppjTcso8gjzpNhxq5pIB0Ym933k4FcgNgHYDrIhjZJahUZeQm3/f1xKLA1rVK0oVAJayHUxgnQZ5yHXt2Bw5nPYkr9p3FVFqxW96gfo0K9qXA7Uzg9WWjc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=IJjHuX/2; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=BifOLTg6; arc=none smtp.client-ip=103.168.172.151 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="IJjHuX/2"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="BifOLTg6" Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfout.nyi.internal (Postfix) with ESMTP id 07D041381E13 for ; Wed, 31 Jul 2024 10:15:00 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Wed, 31 Jul 2024 10:15:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1722435300; x=1722521700; bh=Si9etyY1Cm I533199jI1jEP50fsbX2gipMnoIxbjPR0=; b=IJjHuX/2iUB4y4hqtgP8OuhFFl B4YXHL7QC/15eqERi1SZNGu/qfmdQsONlnsvRYVvy/57FUkQ8utlovrHTOdIbKli fc2pXPMh/7aLZoPvkURWj+EsT7u8yFgNpDrccGTePyOsA4L6SEj91L+eQiqRdZ3s hQq1L7nIq2XLPSD45ZwROWlNn+VmHb1CRaL1AwJyT66zXJpU9xwIgySEDgCrFzn6 bXRepbRa1njx1e2wSmuthMXrw0HpupSGyAe1+TzIczaDsNw6aDcghbKKxfpw3EKS fRHYQ5d9+MZDo5rfSBuDhqbWYYTZqvqj+LiQ56ZmiINax0n0fQ7Avy55f4/A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1722435300; x=1722521700; bh=Si9etyY1CmI533199jI1jEP50fsb X2gipMnoIxbjPR0=; b=BifOLTg6kdc1tTXPOeKImM9eWlntrUKFJmvQBF/YCqFf iEWVBYG9ZL/yZ6ROrxmUxO7HSs8XVmoJN3SjBa8BZC9bGxuGeu/4eMyT7sibc0cZ ca0zxVXz9hbbjgvIEhMBNHc+unmRSv+4LOgsRGJWugToomy4/I7f8yDfBpUuKUCC 8k1vD+jZz9oVj6MlhfB49Rh5ydb+k+BZLbdLYk9KpuHYYANQrEEu3Xlf4iNo95Qn z5fuicrwNIC2Mm65azT/QJTigvWmicqrCClULfrLe4n+4icOiM7AlrQiiszYBv5+ nUH0Cxozs5Tnexg+sqQA3ItyMIoV4DTcyKtMLzrWfA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrjeeigdejhecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtderre dttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhk shdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtdfhvd ejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr mhepmhgrihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopedt X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 31 Jul 2024 10:14:59 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 971f2713 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 31 Jul 2024 14:13:29 +0000 (UTC) Date: Wed, 31 Jul 2024 16:14:57 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 2/8] reftable/stack: test compaction with already-locked tables Message-ID: <123fb9d80eecbd3690280991e0415cbb718b7202.1722435214.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We're lacking test coverage for compacting tables when some of the tables that we are about to compact are locked. Add two tests that exercise this, one for auto-compaction and one for full compaction. Signed-off-by: Patrick Steinhardt --- reftable/stack_test.c | 103 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) diff --git a/reftable/stack_test.c b/reftable/stack_test.c index e3c11e6a6e..04526c6604 100644 --- a/reftable/stack_test.c +++ b/reftable/stack_test.c @@ -863,6 +863,58 @@ static void test_reftable_stack_auto_compaction(void) clear_dir(dir); } +static void test_reftable_stack_auto_compaction_with_locked_tables(void) +{ + struct reftable_write_options opts = { + .disable_auto_compact = 1, + }; + struct reftable_stack *st = NULL; + struct strbuf buf = STRBUF_INIT; + char *dir = get_tmp_dir(__LINE__); + int err; + + err = reftable_new_stack(&st, dir, &opts); + EXPECT_ERR(err); + + for (size_t i = 0; i < 5; i++) { + struct reftable_ref_record ref = { + .update_index = reftable_stack_next_update_index(st), + .value_type = REFTABLE_REF_VAL1, + .value.val1 = { i }, + }; + + strbuf_reset(&buf); + strbuf_addf(&buf, "refs/heads/branch-%04" PRIuMAX, (uintmax_t) i); + ref.refname = buf.buf; + + err = reftable_stack_add(st, &write_test_ref, &ref); + EXPECT_ERR(err); + } + EXPECT(st->merged->stack_len == 5); + + /* + * Given that all tables we have written should be roughly the same + * size, we expect that auto-compaction will want to compact all of the + * tables. Locking any of the tables will keep it from doing so. + */ + strbuf_reset(&buf); + strbuf_addf(&buf, "%s/%s.lock", dir, st->readers[2]->name); + write_file_buf(buf.buf, "", 0); + + /* + * Ideally, we'd handle the situation where any of the tables is locked + * gracefully. We don't (yet) do this though and thus fail. + */ + err = reftable_stack_auto_compact(st); + EXPECT(err == REFTABLE_LOCK_ERROR); + EXPECT(st->stats.failures == 1); + EXPECT(st->merged->stack_len == 5); + + reftable_stack_destroy(st); + strbuf_release(&buf); + clear_dir(dir); +} + static void test_reftable_stack_add_performs_auto_compaction(void) { struct reftable_write_options opts = { 0 }; @@ -911,6 +963,55 @@ static void test_reftable_stack_add_performs_auto_compaction(void) clear_dir(dir); } +static void test_reftable_stack_compaction_with_locked_tables(void) +{ + struct reftable_write_options opts = { + .disable_auto_compact = 1, + }; + struct reftable_stack *st = NULL; + struct strbuf buf = STRBUF_INIT; + char *dir = get_tmp_dir(__LINE__); + int err; + + err = reftable_new_stack(&st, dir, &opts); + EXPECT_ERR(err); + + for (size_t i = 0; i < 3; i++) { + struct reftable_ref_record ref = { + .update_index = reftable_stack_next_update_index(st), + .value_type = REFTABLE_REF_VAL1, + .value.val1 = { i }, + }; + + strbuf_reset(&buf); + strbuf_addf(&buf, "refs/heads/branch-%04" PRIuMAX, (uintmax_t) i); + ref.refname = buf.buf; + + err = reftable_stack_add(st, &write_test_ref, &ref); + EXPECT_ERR(err); + } + EXPECT(st->merged->stack_len == 3); + + /* Lock one of the tables that we're about to compact. */ + strbuf_reset(&buf); + strbuf_addf(&buf, "%s/%s.lock", dir, st->readers[1]->name); + write_file_buf(buf.buf, "", 0); + + /* + * Compaction is expected to fail given that we were not able to + * compact all tables. + */ + err = reftable_stack_compact_all(st, NULL); + EXPECT(err == REFTABLE_LOCK_ERROR); + /* TODO: this is wrong, we should get notified about the failure. */ + EXPECT(st->stats.failures == 0); + EXPECT(st->merged->stack_len == 3); + + reftable_stack_destroy(st); + strbuf_release(&buf); + clear_dir(dir); +} + static void test_reftable_stack_compaction_concurrent(void) { struct reftable_write_options opts = { 0 }; @@ -1016,9 +1117,11 @@ int stack_test_main(int argc, const char *argv[]) RUN_TEST(test_reftable_stack_add); RUN_TEST(test_reftable_stack_add_one); RUN_TEST(test_reftable_stack_auto_compaction); + RUN_TEST(test_reftable_stack_auto_compaction_with_locked_tables); RUN_TEST(test_reftable_stack_add_performs_auto_compaction); RUN_TEST(test_reftable_stack_compaction_concurrent); RUN_TEST(test_reftable_stack_compaction_concurrent_clean); + RUN_TEST(test_reftable_stack_compaction_with_locked_tables); RUN_TEST(test_reftable_stack_hash_id); RUN_TEST(test_reftable_stack_lock_failure); RUN_TEST(test_reftable_stack_log_normalize);