From patchwork Sat Jun 15 00:29:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 13699193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A735C2BA12 for ; Sat, 15 Jun 2024 00:37:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D42836B015E; Fri, 14 Jun 2024 20:31:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3F4A36B0162; Fri, 14 Jun 2024 20:31:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B74316B015A; Fri, 14 Jun 2024 20:31:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B3F966B025B for ; Fri, 14 Jun 2024 20:29:43 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 612ED1208DE for ; Sat, 15 Jun 2024 00:29:43 +0000 (UTC) X-FDA: 82231239846.19.017317C Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf16.hostedemail.com (Postfix) with ESMTP id BA5D5180009 for ; Sat, 15 Jun 2024 00:29:41 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=BoLCf2GR; spf=none (imf16.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718411379; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=l0z7gTm6DMRVo2mw+7Jc7hKaTrGpQ1Z5iRpAq1N5ksU=; b=1h+UlLHGtxt3Bx3V4YyW/Yzr7bbrZ/tGnQGJq5BNQKEf/zAbHoY+Ls7LB4zj8SAFfMwHN8 WYPdAvNWIM4hJuqkznlrlJTdlmDPKheGTYjwvHWoUXXyrd9BBYxf51b+dEwxi9UgvXcSHw 2u68OudUtjoJPYiPPKouzTv+vsk3Vxc= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=BoLCf2GR; spf=none (imf16.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718411379; a=rsa-sha256; cv=none; b=EItsko1NEnvzWrw/tQioBOwnZIYPVhtP4RW4F/UjoKQJK5+ILduQQ80bg7IcDOVsFbzW5F fOTjLpN1ArFvZAwPaXof5nnvDCKEtlxlLLxCuiSP1X0/h1tgXXMeKDehInXkAmN6vOuPhC 5y6pxD/FgZ/YjFjk/Sn84foEUFS6Mfc= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=l0z7gTm6DMRVo2mw+7Jc7hKaTrGpQ1Z5iRpAq1N5ksU=; b=BoLCf2GRTuS3TDDx+N0sqtHOuE obVfJenx066vHCWHo0iL99RY43disTNPjvUf5s3usD1XMVffXlTQTleYtPmVtxQfsrFMNG6Ue11I1 lndSHHge44PxADFi/ZwWpBdN0slBgQ9j28SAmWLRdAc3tnNnShfaMZbGTdOXNtSmf3QD9QN5IMhwQ dq/NDGbnjn4O52QDb+7dQVwmQ/tr+RQMwhdHRTA0spSbPIMNCG5Ky7zNdKIEHxhEsZPNlZq7VMsbt zadWqoTxH8hz+KK+Qd+rcq320hEV20LvXY9/QNZPu/YkuoDtBxdKUjx/CKFhG+wGjGDvciuxznIoB G0KJhXmA==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1sIHIV-00000004KkM-3zHu; Sat, 15 Jun 2024 00:29:35 +0000 From: Luis Chamberlain To: patches@lists.linux.dev, fstests@vger.kernel.org Cc: linux-xfs@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, akpm@linux-foundation.org, ziy@nvidia.com, vbabka@suse.cz, seanjc@google.com, willy@infradead.org, david@redhat.com, hughd@google.com, linmiaohe@huawei.com, muchun.song@linux.dev, osalvador@suse.de, p.raghav@samsung.com, da.gomez@samsung.com, hare@suse.de, john.g.garry@oracle.com, mcgrof@kernel.org, "Darrick J . Wong" Subject: [PATCH v2 3/5] fstests: add fsstress + compaction test Date: Fri, 14 Jun 2024 17:29:32 -0700 Message-ID: <20240615002935.1033031-4-mcgrof@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240615002935.1033031-1-mcgrof@kernel.org> References: <20240615002935.1033031-1-mcgrof@kernel.org> MIME-Version: 1.0 X-Stat-Signature: ga5g51gc6zgoe18km1qpdcsmuuqwsbbw X-Rspam-User: X-Rspamd-Queue-Id: BA5D5180009 X-Rspamd-Server: rspam02 X-HE-Tag: 1718411381-547103 X-HE-Meta: U2FsdGVkX19cyTyOzPKBg7pOYLgiNkWyKv7nzJJVWkM11NetgH755poPaaqgvwMa+/57uSNNbUwGbeL9+vq3KK5K5+cO2KGGZuctVJ/JpwbT8qybXJr4s2W7zYiNSCjqy9mqvrpL2sstkNcoYmb/xPQ+Nk4vLSl2So7kA21CwtMGBXXvMb0Y1uKxjZ8oIQhRFq1XBX7Rgl4VnnEyfBRkLtA0g+qWfwilDHZ/NjClr2VmBG11g8IqxqRqd7iqCD21AArkxmAAPFDwE+duu3pteJ6XuGICjpoZbMpMHJ6x96ZrZj8e37zx7H9L5dnAOwjJ/blu7oQ8Mst7WhUYg/0vTp8eVruOYnZkVfPC+SZniQ7T7/a9mty+8fWuOaFIn8vvWdrhA6q1jeGDXuwq5Qr4ReD/3hgz3GdoW63ZpRfG9B983Y5feTLcy390xtkC0TXPdrNoiI697mcnST9F6dADME7FEKmUHelinZSOsynbnx+FTR7ZA1r5YeUW8eC7P5boiTs49aIGuipVpOKgehgshFUTtDOFe3StmB386/ZUrcSqXeO2EiGegvQSS1VTKnjt35ybT2+rw6iHkxstvLz0dF5qxX6jAKAuWVseYuQifWTOBS8SfQf4T/92W+PJjEu8T3A031S2iFopS5ZWxmrGpm8AXt/UPrukg7abXHk8EXmMsKPmEvXnx6BKuR5PLLAVTO4gfVIA6SZ2VdIi1SxVBj0CJ9bTPJBtVXHf2mPg3Fu4uyq/JuOcGz9SIcnGIPm0K6VjMduydCWY+j5aojQOlZh26j2y2I4c/JQX7EJ4ky7tqrFWnj0xhBpiRlttWPOXreemEEKh2jvD8e1jy56Lq5FlpW1xDOYNLSj2xLZbvanI00iFvjTj/I+2w0kONe/91EjuZKG4WpyUyaTjM4OChPg+UeHzTqYjcbSd6hyEyIOghEMY5fyb6aEnaFCTGQ1n0a2jNJH4CVuGN+4tvu7 Q0YwJ4aK wZzuYbC/q+DmuEHzM91dQ3GJ5K/bIoKXh2zhU+vxcgfgtlaH1pFq2Cp0RU5mXF3Eu3wyjASIIRFwx0wAiPKPBvKh1NbOliLICj6ex6mw605V/+AJtveh13RFD0Fu/Vwx9KlIVPo6d77VzlDAoaohcN0jIUNjfbsIBRBBJKJrznhBDfZzNrpNmwVfgeCjiWMTLbZRGyCvbHvHE2XptEcyw3iM8HzvRiHpuyS+EtfzKphnN4xHcDzvg7/Yxp4o+T0xrEWYoRWSl6p9qhO6RQGeqq4hAQhj7OJPxBDQGX5daT2pCkW67xBk0iCSw/ZSCclj9ssQXL6AT6RvI62TFm3CCNgfMdn8ToIfYGeEOP4gye1JV68F3ubMql9zsMjPW1K4mfHU1XEiyvcZRwLHQzdcEt/LXZRjz4dfTBAkzarB3D1pfTfTZUuYsm+GiNb8LNOJt+b+YHVBvomHxgfvWgCML5Owgut1ggBiLpPgN X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Running compaction while we run fsstress can crash older kernels as per korg#218227 [0], the fix for that [0] has been posted [1] that patch was merged on v6.9-rc6 fixed by commit d99e3140a4d3 ("mm: turn folio_test_hugetlb into a PageType"). This test reproduces that crash right away. But we have more work to do ... Even on v6.10-rc2 where this kernel commit is already merged we can still deadlock when running fsstress and at the same time triggering compaction, this is a new issue being reported now through this patch, but this patch also serves as a reproducer with a high confidence. At least for XFS running this test ~ 44 times will deadlock. If you enable CONFIG_PROVE_LOCKING with the defaults you will end up with a complaint about increasing MAX_LOCKDEP_CHAIN_HLOCKS [1], if you adjust that you then end up with a few soft lockup complaints and some possible deadlock candidates to evaluate [2]. Provide a simple reproducer and pave the way so we keep on testing this. [0] https://bugzilla.kernel.org/show_bug.cgi?id=218227 [1] https://gist.github.com/mcgrof/824913b645892214effeb1631df75072 [2] https://gist.github.com/mcgrof/926e183d21c5c4c55d74ec90197bd77a Reviewed-by: Darrick J. Wong Signed-off-by: Luis Chamberlain --- common/rc | 7 +++++ tests/generic/750 | 63 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/750.out | 2 ++ 3 files changed, 72 insertions(+) create mode 100755 tests/generic/750 create mode 100644 tests/generic/750.out diff --git a/common/rc b/common/rc index e812a2f7cc67..18ad25662d5c 100644 --- a/common/rc +++ b/common/rc @@ -151,6 +151,13 @@ _require_hugepages() _notrun "Kernel does not report huge page size" } +# Requires CONFIG_COMPACTION +_require_vm_compaction() +{ + if [ ! -f /proc/sys/vm/compact_memory ]; then + _notrun "Need compaction enabled CONFIG_COMPACTION=y" + fi +} # Get hugepagesize in bytes _get_hugepagesize() { diff --git a/tests/generic/750 b/tests/generic/750 new file mode 100755 index 000000000000..3057937d7176 --- /dev/null +++ b/tests/generic/750 @@ -0,0 +1,63 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024 Luis Chamberlain. All Rights Reserved. +# +# FS QA Test 750 +# +# fsstress + memory compaction test +# +. ./common/preamble +_begin_fstest auto rw long_rw stress soak smoketest + +_cleanup() +{ + cd / + rm -f $runfile + rm -f $tmp.* + kill -9 $trigger_compaction_pid > /dev/null 2>&1 + $KILLALL_PROG -9 fsstress > /dev/null 2>&1 + + wait > /dev/null 2>&1 +} + +# Import common functions. + +# real QA test starts here + +_supported_fs generic + +_require_scratch +_require_vm_compaction +_require_command "$KILLALL_PROG" "killall" + +# We still deadlock with this test on v6.10-rc2, we need more work. +# but the below makes things better. +_fixed_by_git_commit kernel d99e3140a4d3 \ + "mm: turn folio_test_hugetlb into a PageType" + +echo "Silence is golden" + +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +nr_cpus=$((LOAD_FACTOR * 4)) +nr_ops=$((25000 * nr_cpus * TIME_FACTOR)) +fsstress_args=(-w -d $SCRATCH_MNT -n $nr_ops -p $nr_cpus) +test -n "$SOAK_DURATION" && fsstress_args+=(--duration="$SOAK_DURATION") + +# start a background trigger for memory compaction +runfile="$tmp.compaction" +touch $runfile +while [ -e $runfile ]; do + echo 1 > /proc/sys/vm/compact_memory + sleep 5 +done & +trigger_compaction_pid=$! + +$FSSTRESS_PROG $FSSTRESS_AVOID "${fsstress_args[@]}" >> $seqres.full + +rm -f $runfile +wait > /dev/null 2>&1 + +status=0 +exit diff --git a/tests/generic/750.out b/tests/generic/750.out new file mode 100644 index 000000000000..bd79507b632e --- /dev/null +++ b/tests/generic/750.out @@ -0,0 +1,2 @@ +QA output created by 750 +Silence is golden