From patchwork Mon Apr 29 05:00:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 13646284 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39A9BC04FFE for ; Mon, 29 Apr 2024 05:00:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 56DE66B007B; Mon, 29 Apr 2024 01:00:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 51DE66B0083; Mon, 29 Apr 2024 01:00:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E4D66B0085; Mon, 29 Apr 2024 01:00:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 1A7736B007B for ; Mon, 29 Apr 2024 01:00:32 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 75FA2A17EE for ; Mon, 29 Apr 2024 05:00:31 +0000 (UTC) X-FDA: 82061368662.01.0D9FFB6 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf04.hostedemail.com (Postfix) with ESMTP id 7CB374000B for ; Mon, 29 Apr 2024 05:00:28 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=2feX+zPn; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=none (imf04.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714366830; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=0up5w6U/SD43DrHq7xXf8Ll9aXDDKmogMJKfJTux2Uw=; b=rPkW0hlkycTxyI1/hWn3q8ta81V0+ruxck9rTBrygvErPnl6H7MEEd2bXPr3Bp172FBu08 wJwlfA2yOOyTqJhsYyqe/lSIIVnTr5liLYcKxcIMNAvsp0dq7LdFglZfkMKMpb9MsW5vBX mVvNlxSPU59fjpc+UKhgOoEonPQ4aUE= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=2feX+zPn; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=none (imf04.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714366830; a=rsa-sha256; cv=none; b=3psC1h6r5DxGt7ziBzsYe49KNM0l6K3IyKUDVHMQAAdfJ/DAhacQa8wScU53G8hqwlGHTJ FeFxb4ZJHpt7zc1SqGX1slvZrp/38Y+bTtJ69XmQH8eurf7zKJ48K08B/arx9lNe91q4Sy DVa9Jzw5zpKMaqx96qojubcJ3jS5Vug= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=0up5w6U/SD43DrHq7xXf8Ll9aXDDKmogMJKfJTux2Uw=; b=2feX+zPnyDaIDIoXqrS5icBciX 8dUhFHXFjBk8vww8VOa9IFnUM/TRQd6u9UuaEf2fVDuxXMlo3Dyt0VvEtmuOL9fBE7LrLxPKNHns/ nlGdysdqRTBOAQ86LMKL2qGXpH7jv8lj/53KyHIb/egMByJETfFwyva6cBQ8BzTqahL1sA3/npSWl QDXb8A4NaKS2+eMYce0pnLuvRs/jZgbJnhBNTdzxceVHynaSJlfHl2S40/y8Ktg39OkYJ3UdQ1wEC xYL3zg8S27AYAB7RRL5ROLxzAyChs5ZtxvvH6pkpHZZqECV81YCfvbyMnHG3ux5A3X+LeADo8mnfs MHUxw7YQ==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1s1J7n-00000001TGv-1acJ; Mon, 29 Apr 2024 05:00:23 +0000 From: Luis Chamberlain To: fstests@vger.kernel.org Cc: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, akpm@linux-foundation.org, ziy@nvidia.com, vbabka@suse.cz, seanjc@google.com, willy@infradead.org, p.raghav@samsung.com, da.gomez@samsung.com, hare@suse.de, john.g.garry@oracle.com, linux-xfs@vger.kernel.org, patches@lists.linux.dev, Luis Chamberlain Subject: [RFC v2] fstests: add stress truncation + writeback + compaction split test Date: Sun, 28 Apr 2024 22:00:22 -0700 Message-ID: <20240429050022.350818-1-mcgrof@kernel.org> X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Stat-Signature: 1zyfbjgp3ydx6rtifobutnrpxtsgp6ec X-Rspam-User: X-Rspamd-Queue-Id: 7CB374000B X-HE-Tag: 1714366828-481967 X-HE-Meta: U2FsdGVkX18/qz/FJZdrpoDxoQxKjPLBkgYJHlDhylc1lZYaPsU1WNT3sMnOrBK9qh2T/uVwNHShy6rBfUukkq/kJXnw3Ff/l8ZivET0PPSvYXxzUDcMKUQfgaCluj3v951/j1bQyy0aRcXeqeOYXNhuQRMlZPDBmsQZiGIb9ZRzwlUtRbnXUJtKm8TpOyrbFzzBRsur/9xZt1xd/oZQqeB0GgLwoqrU7Ie7Rz+w9wY65wDqnIqRA75/D7KtT63m4qn7IGH6mDft1LrcXU2ARoen09GVsAkp0JYMY9RtB9aZYq+DrT06PUDyg5qpDLSkE1brr3qJrFLBrzQ12h3epblurWGX7MxDLantrRTGDxIoz/hbw/JOF8bZzjDTNPqxoNdMn0dhrYWmOfAl/Vl3wLCx1MJ3TjzFcFTBBSgJHR9glh6Zo+DG0dIr8dC1HZUJGplpc278EusfNrNIGIMHF+1e0Q1HnGlYdfqM5fybx0xnzPShafXrp1ncUKHwWyiMlXm3/h9rAK2NoY6JFdMB02nji/b9roaq96Jsk5WHSgTp06Pkxi7DEbgp/ouE3XVv2wG7zx6SnE601hz4d7G3tGNhAt78mFSqOhfheATVgUgZYz4GOqw4W9v/6pQni4ZV57o3cIgk9DA+J2+Ek/M/z6UZRGb4Xh3krJc31SPqU7RjsTLyzFvXV4oGjGY/dbQFkwaLLJolPHdSZrLpvU5+38YuusBRJAyfJG9ZgpiZvXSIBnAgtrGfDM1VAze6FgF4iGDTv4uTfp5Q4slz4Xq8VgjYikbE+ne2oRhNlW2mEv32sA9j5sUqA/BjJ2w+e9vjOgzXG62OVCbPrLMtsrHnTqnBby8K1oHX6eLFutClqHQ7m5IITuRKB7mPGyXhHAW5mLnLivsDRdxbN3B3M8BwaBQmpn3qSy/tU0JeU4lVjgDS7W8Fct0u2S5rWsZTYrVKmdHQ22U2BSyKsQCaY5/ Av3VcdgU EmwozeYEyFhUDDIB83taSlZrFsI5ThFuZ1SM73GZkWiJMLZ+KuqVLAJQUrwgHfFCuUhj+IRTQnj6rETu3T3qNaBh0T5gZvv9yqdI6LLEZDTaQGWJWPqLq7Mg7eF4bPei9xcJWkt/oUi3xZcnr2xFOy/RpzQvegRR21hdiYrBkk5jXV9hBfPAS7j3IKtHl5rDVIiuSEpSuWIGK+NBIP2qW/9Sl/bmwX17l/5ux4kx9tbdDZnPUlcAgf6AOm45W7ZcmOjbicaCudXLq5Dvn5ObiTP9lFVS4xvDzeZwJ8ZgrHGDpe31zsYQqSw0BueXblXnPUkg1u9Tut9s4XXRKlmCRXkawUR37brL8V49K/a5j09SDofFVEU+FZBUWoHrYwSkdjMiDoz1eBlkwv4l836pDFHWaGRbfHnl5mKsDr1TcIMOT/5bBO26F1i1G9XN49Fg09JuZShN+kNF49QRdGvSGmvSy2sS2Wb+dPam5H0u73LAAJZdYtsYAorprowPwesOXO0tw X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Stress test folio splits by using the debugfs interface to a target a new smaller folio order while running compaction at the same time. This is dangerous at the moment as its using a debugfs API which requires two out of tree fixes [0] [1] which have already been posted but not yet merged. With these debugfs patches applied this test can now be used to reproduce an issue which was only possible to reproduce by running generic/447 twice with min order: https://gist.github.com/mcgrof/d12f586ec6ebe32b2472b5d634c397df This is designed to try to exacerbate races with folio splits incurred by truncation and race that with compaction and writeback. This only creates a crash with min order enabled, so for example with a 16k block sized XFS test profile. This also begs the question if something like MADV_NOHUGEPAGE might be desirable from userspace, so to enable userspace to request splits when possible. If inspecting more closely, you'll want to enable on your kernel boot: dyndbg='file mm/huge_memory.c +p' Since we want to race large folio splits we also augment the full test output log $seqres.full with the test specific number of successful splits from vmstat thp_split_page and thp_split_page_failed. [0] https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/commit/?h=20240424-lbs&id=80f6df5037fd0ad560526af45bd7f4d779fe03f6 [1] https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/commit/?h=20240424-lbs&id=38f6fac5b4283ea48b1876fc56728f062168f8c3 Signed-off-by: Luis Chamberlain --- For now at laest to allow people to more easily reproduce the crash we're discussing here: https://lkml.kernel.org/r/Zi8aYA92pvjDY7d5@bombadil.infradead.org I can clean this up based on Zorro's feedback after this. Posting this RFCv2 so to enable folks to more easily reproduce the issue and also the debugfs issue that this uses. common/rc | 20 ++++++++ tests/generic/745 | 115 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/745.out | 2 + 3 files changed, 137 insertions(+) create mode 100755 tests/generic/745 create mode 100644 tests/generic/745.out diff --git a/common/rc b/common/rc index d4432f5ce259..1eefb53aa84b 100644 --- a/common/rc +++ b/common/rc @@ -127,6 +127,26 @@ _require_compaction() _notrun "Need compaction enabled CONFIG_COMPACTION=y" fi } + +# Requires CONFIG_DEBUGFS and truncation knobs +SPLIT_DEBUGFS="/sys/kernel/debug/split_huge_pages" +_require_split_debugfs() +{ + if [ ! -f $SPLIT_DEBUGFS ]; then + _notrun "Needs CONFIG_DEBUGFS and split_huge_pages" + fi +} + +_split_huge_pages_file_full() +{ + local file=$1 + local offset="0x0" + local len=$(printf "%x" $(stat --format='%s' $file)) + local order="0" + local split_cmd="$file,$offset,0x${len},$order" + echo $split_cmd > $SPLIT_DEBUGFS +} + # Get hugepagesize in bytes _get_hugepagesize() { diff --git a/tests/generic/745 b/tests/generic/745 new file mode 100755 index 000000000000..0c67bd990a2f --- /dev/null +++ b/tests/generic/745 @@ -0,0 +1,115 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2024 Luis Chamberlain. All Rights Reserved. +# +# FS QA Test No. 734 +# +# stress truncation + writeback + compaction +# +# This aims at trying to reproduce a difficult to reproduce bug found with +# min order. The root cause lies in compaction racing with truncation on +# min order: +# +# https://gist.github.com/mcgrof/d12f586ec6ebe32b2472b5d634c397df +# +# If you're enabling this and want to check underneath the hood you may want to +# enable: +# +# dyndbg='file mm/huge_memory.c +p' +# +# We want to increase the rate of successful truncations + compaction racing, +# so we want to increase the value of thp_split_page in $seqres.full. +# +# Our general goal here is to race with folio truncation + writeback and +# compaction. + +. ./common/preamble + +# This is dangerous_fuzzers fow now until we get the debugfs interface +# this uses fixed. Patches for that have been posted but still under +# review. +_begin_fstest long_rw stress soak smoketest dangerous_fuzzers + +# Override the default cleanup function. +_cleanup() +{ + cd / + rm -f $tmp.* + $KILLALL_PROG -9 fsstress > /dev/null 2>&1 +} + +# Import common functions. +. ./common/filter + +# real QA test starts here +_supported_fs generic +_require_test +_require_scratch +_require_split_debugfs +_require_compaction +_require_command "$KILLALL_PROG" "killall" + +echo "Silence is golden" + +_scratch_mkfs >>$seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +nr_cpus=$((LOAD_FACTOR * 4)) +nr_ops=$((25000 * nr_cpus * TIME_FACTOR)) + +fsstress_args=(-w -d $SCRATCH_MNT/test -n $nr_ops -p $nr_cpus) + +# used to let our loops know when to stop +runfile="$tmp.keep.running.loop" +touch $runfile + +# The background ops are out of bounds, the goal is to race with fsstress. + +# Force folio split if possible, this seems to be screaming for MADV_NOHUGEPAGE +# for large folios. +while [ -e $runfile ]; do + for i in $(find $SCRATCH_MNT/test \( -type f \) 2>/dev/null); do + _split_huge_pages_file_full $i >/dev/null 2>&1 + done + sleep 2 +done & +split_huge_pages_files_pid=$! + +while [ -e $runfile ]; do + echo 1 > /proc/sys/vm/compact_memory + sleep 10 +done & +compaction_pid=$! + +blocksize=$(_get_file_block_size $SCRATCH_MNT) +export XFS_DIO_MIN=$((blocksize * 2)) + +test -n "$SOAK_DURATION" && fsstress_args+=(--duration="$SOAK_DURATION") + +split_count_before=0 +split_count_failed_before=0 + +if grep -q thp_split_page /proc/vmstat; then + split_count_before=$(grep ^thp_split_page /proc/vmstat | head -1 | awk '{print $2}') + split_count_failed_before=$(grep ^thp_split_page_failed /proc/vmstat | head -1 | awk '{print $2}') +else + echo "no thp_split_page in /proc/vmstat" >> /proc/vmstat +fi + +$FSSTRESS_PROG $FSSTRESS_AVOID "${fsstress_args[@]}" >> $seqres.full + +rm -f $runfile +wait > /dev/null 2>&1 + +if grep -q thp_split_page /proc/vmstat; then + split_count_after=$(grep ^thp_split_page /proc/vmstat | head -1 | awk '{print $2}') + split_count_failed_after=$(grep ^thp_split_page_failed /proc/vmstat | head -1 | awk '{print $2}') + thp_split_page=$((split_count_after - split_count_before)) + thp_split_page_failed=$((split_count_failed_after - split_count_failed_before)) + + echo "vmstat thp_split_page: $thp_split_page" >> $seqres.full + echo "vmstat thp_split_page_failed: $thp_split_page_failed" >> $seqres.full +fi + +status=0 +exit diff --git a/tests/generic/745.out b/tests/generic/745.out new file mode 100644 index 000000000000..fce6b7f5489d --- /dev/null +++ b/tests/generic/745.out @@ -0,0 +1,2 @@ +QA output created by 745 +Silence is golden