From patchwork Tue Jan 10 22:49:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Chinner X-Patchwork-Id: 13095743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 377F8C61DB3 for ; Tue, 10 Jan 2023 22:51:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235150AbjAJWvD (ORCPT ); Tue, 10 Jan 2023 17:51:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52904 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235482AbjAJWuE (ORCPT ); Tue, 10 Jan 2023 17:50:04 -0500 Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90A2233D64 for ; Tue, 10 Jan 2023 14:49:13 -0800 (PST) Received: by mail-pg1-x529.google.com with SMTP id 78so9283272pgb.8 for ; Tue, 10 Jan 2023 14:49:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=QDgXaYw8HrZpdgJMgIOJ+aNMO11VMGqUpEb6tyMz958=; b=WwdyG/YXXATWaLbB5eUxS27gZbfzh13XceaEe8T+pWu8maVh1xKjKIQvrwN0OVh1Kn wBlgM25MOdADk/xkGR0p1O878I3Yisi4D1xGaYhW7gVhaBbIBlRUSAq7lsjOxsxt0dE0 T144Povbs4q5+fzFhBoltt27UrzTHF/TipAVplPQOdXcaKb4oZvoO8ME7BAoxWXweFiv dcy1ACg82ZJwDblgPjbkfoIy8PEIqnuhh1x90Ho3C9ERJOLbiQAIftdnB9PvmS8ALf7L IqHIDYIXmwp46UIRH1aAyGZ+uln+sedc3471yx3ReiCE0xdaXTfzIJDtYuE7ebG5JzVT tdfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QDgXaYw8HrZpdgJMgIOJ+aNMO11VMGqUpEb6tyMz958=; b=IeKllrWH6c67M+B22dIQ5V8Oxh6hARaGQ/GW+fy3VThNwl5fCXnz2f9tQAjXlDt0fG ewu2K7EB9HuL9MzAoHfErBEalw8pi6/SroFyhHzFpxY5z1Xi8wNaZIqOQNoTdgH5Kk1B Y+Z8UAUlttgsNMmMLC84vw5MjFVaUJq8yZ8IL7Opi+jWv9nRU6TNnlu68BAhVzhUn+zx kAgDK9Ej2ojIPCmQ/u2t/ZvXK1dxxKTmZhbsm1P9Xm0OSJihU93oqEhmKFoZHWuQjt+3 n9jwRAceK5whPW3T4d35rkXR1zBcDYqvTzJKtjAqJGZEhPa9KyFkJnQZI8oYySZFFLAR 0u3A== X-Gm-Message-State: AFqh2koZla9Q/cNEtqIaUNXMc99VkYuNcARXW/eyl6vOlJQWgEEkwf2u axibjx8XNDULw/gYHqeovVGg9k8jTClHyCzV X-Google-Smtp-Source: AMrXdXs2CWJgOyrZOTUtQX6MCV+QYemQ1F2VHoZLmdyht5TIfcdXI/sfJBtJ6gk0s0B14zaI7yQUYQ== X-Received: by 2002:aa7:9526:0:b0:587:405a:421c with SMTP id c6-20020aa79526000000b00587405a421cmr13041641pfp.7.1673390952980; Tue, 10 Jan 2023 14:49:12 -0800 (PST) Received: from dread.disaster.area (pa49-186-146-207.pa.vic.optusnet.com.au. [49.186.146.207]) by smtp.gmail.com with ESMTPSA id y24-20020aa79438000000b00574ebfdc721sm8905024pfo.16.2023.01.10.14.49.11 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Jan 2023 14:49:11 -0800 (PST) Received: from [192.168.253.23] (helo=devoid.disaster.area) by dread.disaster.area with esmtp (Exim 4.92.3) (envelope-from ) id 1pFNQa-001Yl6-Ba for fstests@vger.kernel.org; Wed, 11 Jan 2023 09:49:08 +1100 Received: from dave by devoid.disaster.area with local (Exim 4.96) (envelope-from ) id 1pFNQa-004unL-17 for fstests@vger.kernel.org; Wed, 11 Jan 2023 09:49:08 +1100 From: Dave Chinner To: fstests@vger.kernel.org Subject: [PATCH 1/3] populate: fix horrible performance due to excessive forking Date: Wed, 11 Jan 2023 09:49:04 +1100 Message-Id: <20230110224906.1171483-2-david@fromorbit.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230110224906.1171483-1-david@fromorbit.com> References: <20230110224906.1171483-1-david@fromorbit.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Dave Chinner xfs/155 is taking close on 4 minutes to populate the filesystem, and most of that is because the populate functions are coded without consideration of performance. Most of the operations can be executed in parallel as the operate on separate files or in separate directories. Creating a zero length file in a shell script can be very fast if we do the creation within the shell, but running touch, xfs_io or some other process to create the file is extremely slow - performance is limited by the process creation/destruction rate, not the filesystem create rate. Same goes for unlinking files. We can use 'echo -n > $file' to create or truncate an existing file to zero length from within the shell. This is much, much faster than calling touch. For removing lots of files, there is no shell built in to do this without forking, but we can easily build a file list and pipe it to 'xargs rm -f' to execute rm with as many files as possible in one execution. Doing this removes approximately 50,000 process creat/destroy cycles to populate the filesystem, reducing system time from ~200s to ~35s to populate the filesystem. Along with running operations in parallel, this brings the population time down from ~235s to less than 45s. The long tail of that 45s runtime time is the btree format attribute tree create. That executes setfattr a very large number of times, taking 44s to run and consuming 36s of system time mostly just creating and destroying thousands of setfattr process contexts. There's no easy shell coding solution to that issue, so that's for another rainy day. Signed-off-by: Dave Chinner --- common/populate | 179 ++++++++++++++++++++++++++++-------------------- 1 file changed, 104 insertions(+), 75 deletions(-) diff --git a/common/populate b/common/populate index 44b4af166..9b60fa5c1 100644 --- a/common/populate +++ b/common/populate @@ -52,23 +52,64 @@ __populate_fragment_file() { test -f "${fname}" && $here/src/punch-alternating "${fname}" } -# Create a large directory -__populate_create_dir() { - name="$1" - nr="$2" - missing="$3" +# Create a specified number of files or until the maximum extent count is +# reached. If the extent count is reached, return the number of files created. +# This is optimised for speed - do not add anything that executes a separate +# process in every loop as this will slow it down by a factor of at least 5. +__populate_create_nfiles() { + local name="$1" + local nr="$2" + local max_nextents="$3" + local d=0 mkdir -p "${name}" - seq 0 "${nr}" | while read d; do - creat=mkdir - test "$((d % 20))" -eq 0 && creat=touch - $creat "${name}/$(printf "%.08d" "$d")" + for d in `seq 0 "${nr}"`; do + local fname="" + printf -v fname "${name}/%.08d" "$d" + + if [ "$((d % 20))" -eq 0 ]; then + mkdir ${fname} + else + echo -n > ${fname} + fi + + if [ "${max_nextents}" -eq 0 ]; then + continue + fi + if [ "$((d % 40))" -ne 0 ]; then + continue + fi + + local nextents="$(_xfs_get_fsxattr nextents $name)" + if [ "${nextents}" -gt "${max_nextents}" ]; then + echo ${d} + break + fi done +} + +# remove every second file in the given directory. This is optimised for speed - +# do not add anything that executes a separate process in each loop as this will +# slow it down by at least factor of 10. +__populate_remove_nfiles() { + local name="$1" + local nr="$2" + local d=1 + + for d in `seq 1 2 "${nr}"`; do + printf "${name}/%.08d " "$d" + done | xargs rm -f +} +# Create a large directory +__populate_create_dir() { + local name="$1" + local nr="$2" + local missing="$3" + + __populate_create_nfiles "${name}" "${nr}" 0 test -z "${missing}" && return - seq 1 2 "${nr}" | while read d; do - rm -rf "${name}/$(printf "%.08d" "$d")" - done + __populate_remove_nfiles "${name}" "${nr}" } # Create a large directory and ensure that it's a btree format @@ -82,31 +123,18 @@ __populate_xfs_create_btree_dir() { # watch for when the extent count exceeds the space after the # inode core. local max_nextents="$(((isize - icore_size) / 16))" - local nr=0 - - mkdir -p "${name}" - while true; do - local creat=mkdir - test "$((nr % 20))" -eq 0 && creat=touch - $creat "${name}/$(printf "%.08d" "$nr")" - if [ "$((nr % 40))" -eq 0 ]; then - local nextents="$(_xfs_get_fsxattr nextents $name)" - [ $nextents -gt $max_nextents ] && break - fi - nr=$((nr+1)) - done + local nr=100000 + nr=$(__populate_create_nfiles "${name}" "${nr}" "${max_nextents}") test -z "${missing}" && return - seq 1 2 "${nr}" | while read d; do - rm -rf "${name}/$(printf "%.08d" "$d")" - done + __populate_remove_nfiles "${name}" "${nr}" } # Add a bunch of attrs to a file __populate_create_attr() { - name="$1" - nr="$2" - missing="$3" + local name="$1" + local nr="$2" + local missing="$3" touch "${name}" seq 0 "${nr}" | while read d; do @@ -121,17 +149,18 @@ __populate_create_attr() { # Fill up some percentage of the remaining free space __populate_fill_fs() { - dir="$1" - pct="$2" + local dir="$1" + local pct="$2" + local nr=0 test -z "${pct}" && pct=60 mkdir -p "${dir}/test/1" cp -pRdu "${dir}"/S_IFREG* "${dir}/test/1/" - SRC_SZ="$(du -ks "${dir}/test/1" | cut -f 1)" - FS_SZ="$(( $(stat -f "${dir}" -c '%a * %S') / 1024 ))" + local SRC_SZ="$(du -ks "${dir}/test/1" | cut -f 1)" + local FS_SZ="$(( $(stat -f "${dir}" -c '%a * %S') / 1024 ))" - NR="$(( (FS_SZ * ${pct} / 100) / SRC_SZ ))" + local NR="$(( (FS_SZ * ${pct} / 100) / SRC_SZ ))" echo "FILL FS" echo "src_sz $SRC_SZ fs_sz $FS_SZ nr $NR" @@ -220,45 +249,45 @@ _scratch_xfs_populate() { # Data: # Fill up the root inode chunk - echo "+ fill root ino chunk" + ( echo "+ fill root ino chunk" seq 1 64 | while read f; do - $XFS_IO_PROG -f -c "truncate 0" "${SCRATCH_MNT}/dummy${f}" - done + echo -n > "${SCRATCH_MNT}/dummy${f}" + done ) & # Regular files # - FMT_EXTENTS echo "+ extents file" - __populate_create_file $blksz "${SCRATCH_MNT}/S_IFREG.FMT_EXTENTS" + __populate_create_file $blksz "${SCRATCH_MNT}/S_IFREG.FMT_EXTENTS" & # - FMT_BTREE echo "+ btree extents file" nr="$((blksz * 2 / 16))" - __populate_create_file $((blksz * nr)) "${SCRATCH_MNT}/S_IFREG.FMT_BTREE" + __populate_create_file $((blksz * nr)) "${SCRATCH_MNT}/S_IFREG.FMT_BTREE" & # Directories # - INLINE - echo "+ inline dir" - __populate_create_dir "${SCRATCH_MNT}/S_IFDIR.FMT_INLINE" 1 + echo "+ inline dir" + __populate_create_dir "${SCRATCH_MNT}/S_IFDIR.FMT_INLINE" 1 "" & # - BLOCK echo "+ block dir" - __populate_create_dir "${SCRATCH_MNT}/S_IFDIR.FMT_BLOCK" "$((dblksz / 40))" + __populate_create_dir "${SCRATCH_MNT}/S_IFDIR.FMT_BLOCK" "$((dblksz / 40))" "" & # - LEAF echo "+ leaf dir" - __populate_create_dir "${SCRATCH_MNT}/S_IFDIR.FMT_LEAF" "$((dblksz / 12))" + __populate_create_dir "${SCRATCH_MNT}/S_IFDIR.FMT_LEAF" "$((dblksz / 12))" "" & # - LEAFN echo "+ leafn dir" - __populate_create_dir "${SCRATCH_MNT}/S_IFDIR.FMT_LEAFN" "$(( ((dblksz - leaf_hdr_size) / 8) - 3 ))" + __populate_create_dir "${SCRATCH_MNT}/S_IFDIR.FMT_LEAFN" "$(( ((dblksz - leaf_hdr_size) / 8) - 3 ))" "" & # - NODE echo "+ node dir" - __populate_create_dir "${SCRATCH_MNT}/S_IFDIR.FMT_NODE" "$((16 * dblksz / 40))" true + __populate_create_dir "${SCRATCH_MNT}/S_IFDIR.FMT_NODE" "$((16 * dblksz / 40))" true & # - BTREE echo "+ btree dir" - __populate_xfs_create_btree_dir "${SCRATCH_MNT}/S_IFDIR.FMT_BTREE" "$isize" true + __populate_xfs_create_btree_dir "${SCRATCH_MNT}/S_IFDIR.FMT_BTREE" "$isize" true & # Symlinks # - FMT_LOCAL @@ -280,20 +309,20 @@ _scratch_xfs_populate() { # Attribute formats # LOCAL - echo "+ local attr" - __populate_create_attr "${SCRATCH_MNT}/ATTR.FMT_LOCAL" 1 + echo "+ local attr" + __populate_create_attr "${SCRATCH_MNT}/ATTR.FMT_LOCAL" 1 "" & # LEAF - echo "+ leaf attr" - __populate_create_attr "${SCRATCH_MNT}/ATTR.FMT_LEAF" "$((blksz / 40))" + echo "+ leaf attr" + __populate_create_attr "${SCRATCH_MNT}/ATTR.FMT_LEAF" "$((blksz / 40))" "" & # NODE echo "+ node attr" - __populate_create_attr "${SCRATCH_MNT}/ATTR.FMT_NODE" "$((8 * blksz / 40))" + __populate_create_attr "${SCRATCH_MNT}/ATTR.FMT_NODE" "$((8 * blksz / 40))" "" & # BTREE echo "+ btree attr" - __populate_create_attr "${SCRATCH_MNT}/ATTR.FMT_BTREE" "$((64 * blksz / 40))" true + __populate_create_attr "${SCRATCH_MNT}/ATTR.FMT_BTREE" "$((64 * blksz / 40))" true & # trusted namespace touch ${SCRATCH_MNT}/ATTR.TRUSTED @@ -321,68 +350,68 @@ _scratch_xfs_populate() { rm -rf "${SCRATCH_MNT}/attrvalfile" # Make an unused inode - echo "+ empty file" + ( echo "+ empty file" touch "${SCRATCH_MNT}/unused" $XFS_IO_PROG -f -c 'fsync' "${SCRATCH_MNT}/unused" - rm -rf "${SCRATCH_MNT}/unused" + rm -rf "${SCRATCH_MNT}/unused" ) & # Free space btree echo "+ freesp btree" nr="$((blksz * 2 / 8))" - __populate_create_file $((blksz * nr)) "${SCRATCH_MNT}/BNOBT" + __populate_create_file $((blksz * nr)) "${SCRATCH_MNT}/BNOBT" & # Inode btree - echo "+ inobt btree" + ( echo "+ inobt btree" local ino_per_rec=64 local rec_per_btblock=16 local nr="$(( 2 * (blksz / rec_per_btblock) * ino_per_rec ))" local dir="${SCRATCH_MNT}/INOBT" - mkdir -p "${dir}" - seq 0 "${nr}" | while read f; do - touch "${dir}/${f}" - done - - seq 0 2 "${nr}" | while read f; do - rm -f "${dir}/${f}" - done + __populate_create_dir "${SCRATCH_MNT}/INOBT" "${nr}" true + ) & # Reverse-mapping btree is_rmapbt="$(_xfs_has_feature "$SCRATCH_MNT" rmapbt -v)" if [ $is_rmapbt -gt 0 ]; then - echo "+ rmapbt btree" + ( echo "+ rmapbt btree" nr="$((blksz * 2 / 24))" __populate_create_file $((blksz * nr)) "${SCRATCH_MNT}/RMAPBT" + ) & fi # Realtime Reverse-mapping btree is_rt="$(_xfs_get_rtextents "$SCRATCH_MNT")" if [ $is_rmapbt -gt 0 ] && [ $is_rt -gt 0 ]; then - echo "+ rtrmapbt btree" + ( echo "+ rtrmapbt btree" nr="$((blksz * 2 / 32))" $XFS_IO_PROG -R -f -c 'truncate 0' "${SCRATCH_MNT}/RTRMAPBT" __populate_create_file $((blksz * nr)) "${SCRATCH_MNT}/RTRMAPBT" + ) & fi # Reference-count btree is_reflink="$(_xfs_has_feature "$SCRATCH_MNT" reflink -v)" if [ $is_reflink -gt 0 ]; then - echo "+ reflink btree" + ( echo "+ reflink btree" nr="$((blksz * 2 / 12))" __populate_create_file $((blksz * nr)) "${SCRATCH_MNT}/REFCOUNTBT" cp --reflink=always "${SCRATCH_MNT}/REFCOUNTBT" "${SCRATCH_MNT}/REFCOUNTBT2" + ) & fi # Copy some real files (xfs tests, I guess...) echo "+ real files" test $fill -ne 0 && __populate_fill_fs "${SCRATCH_MNT}" 5 - # Make sure we get all the fragmentation we asked for - __populate_fragment_file "${SCRATCH_MNT}/S_IFREG.FMT_BTREE" - __populate_fragment_file "${SCRATCH_MNT}/BNOBT" - __populate_fragment_file "${SCRATCH_MNT}/RMAPBT" - __populate_fragment_file "${SCRATCH_MNT}/RTRMAPBT" - __populate_fragment_file "${SCRATCH_MNT}/REFCOUNTBT" + # Wait for all file creation to complete before we start fragmenting + # the files as needed. + wait + __populate_fragment_file "${SCRATCH_MNT}/S_IFREG.FMT_BTREE" & + __populate_fragment_file "${SCRATCH_MNT}/BNOBT" & + __populate_fragment_file "${SCRATCH_MNT}/RMAPBT" & + __populate_fragment_file "${SCRATCH_MNT}/RTRMAPBT" & + __populate_fragment_file "${SCRATCH_MNT}/REFCOUNTBT" & + wait umount "${SCRATCH_MNT}" } From patchwork Tue Jan 10 22:49:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Chinner X-Patchwork-Id: 13095741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 048B5C46467 for ; Tue, 10 Jan 2023 22:51:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234837AbjAJWvA (ORCPT ); Tue, 10 Jan 2023 17:51:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235460AbjAJWt7 (ORCPT ); Tue, 10 Jan 2023 17:49:59 -0500 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1926D32EB5 for ; Tue, 10 Jan 2023 14:49:13 -0800 (PST) Received: by mail-pf1-x433.google.com with SMTP id a30so10030320pfr.6 for ; Tue, 10 Jan 2023 14:49:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=Wx72QwF6cg/7thZ1yaywDGiRxV5C6xJMmfXnQKTufAU=; b=Of1t6szKqmpprN8W0WA0OBH3NUa3wKy4YzIaRxr753y8bCh2q35Y7QmB2AZ2+0IHv4 jfJMaxitRm0OJcFGYW4YffVIyjGnPqPYm+MiY8+Dr6cTD5s0ryXtMyLNiKu+O0dMViGT pqHzy1gmO3TFZyB7jcUXudDHKZU0HXBeiTVCGlBzmUNEcr3qoS9K0uICe8gnSvQzHrs3 /Qi3ouw3k9fwWY3BsxIKxD3JvnMulgeDT+hKvyYJxRliiJBo5HFnegfpeSww6DliVIdj HvPUoB2PE8BpgaORSw4oGIlvP52qGb+xLTDtBKjN86TCfDXLX1O1ASM1r01nhixphPhE 3TVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Wx72QwF6cg/7thZ1yaywDGiRxV5C6xJMmfXnQKTufAU=; b=7RtVa6AO1sUssHXvrIlIYoqAReC+8jH2XPLBHkzZr/2ENYphe21DLHGEfKI7AJ2azx ShT98WIQVLsO+csf9I2xWiZoulww2L/Eh6gW95LGXBepaqgYiuwKp0vLij0pf2+7BfR5 y8QcrcpkBXph7ZoqOiKPS5DUldh1kGljiLWNz/fwMJ89bAINFS/TANHk3uwf25NTiDHW uIfdfz2KAbPDexc97bdhr4raEEd6kXmBOk123PkJuud8w8ezYEHt4daV79iqd4uaK064 V2RQ/WRAkRHv+yZcB7/5WukkXdOxi9RIyvdIT3qZ4Y5r9XVPyifTCTUVpmRCk1uJa7j5 d6vQ== X-Gm-Message-State: AFqh2kp5PWh+Pxo6VgFvqSkGnmi5sAVr+Shg+Z1EU5I9ZJ0I27Xth2mM qwjwh8aKo+XImGbUaRBH6q/c5gZEnZ6kb6Cc X-Google-Smtp-Source: AMrXdXvTNqMnz9GlmPCmlNJM1rDjVFa4VEB0ybHw7lknkpFl7Nqz2MRgXXMIvBj8VOcS2C97uxeWTA== X-Received: by 2002:a05:6a00:4486:b0:589:85ed:4119 with SMTP id cu6-20020a056a00448600b0058985ed4119mr115041pfb.32.1673390952613; Tue, 10 Jan 2023 14:49:12 -0800 (PST) Received: from dread.disaster.area (pa49-186-146-207.pa.vic.optusnet.com.au. [49.186.146.207]) by smtp.gmail.com with ESMTPSA id b12-20020aa78ecc000000b00581a156b920sm8567225pfr.132.2023.01.10.14.49.11 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Jan 2023 14:49:11 -0800 (PST) Received: from [192.168.253.23] (helo=devoid.disaster.area) by dread.disaster.area with esmtp (Exim 4.92.3) (envelope-from ) id 1pFNQa-001Yl8-CS for fstests@vger.kernel.org; Wed, 11 Jan 2023 09:49:08 +1100 Received: from dave by devoid.disaster.area with local (Exim 4.96) (envelope-from ) id 1pFNQa-004unP-1C for fstests@vger.kernel.org; Wed, 11 Jan 2023 09:49:08 +1100 From: Dave Chinner To: fstests@vger.kernel.org Subject: [PATCH 2/3] populate: ensure btree directories are created reliably Date: Wed, 11 Jan 2023 09:49:05 +1100 Message-Id: <20230110224906.1171483-3-david@fromorbit.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230110224906.1171483-1-david@fromorbit.com> References: <20230110224906.1171483-1-david@fromorbit.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Dave Chinner The population function creates an XFS btree format directory by polling the extent count of the inode and creating new dirents until the extent count goes over the limit that pushes it into btree format. It then removes every second dirent to create empty space in the directory data to ensure that operations like metadump with obfuscation can check that they don't leak stale data from deleted dirents. Whilst this does not result in directory data blocks being freed, it does not take into account the fact that the dabtree index has half the entries removed from it and that can result in btree nodes merging and extents being freed. This causes the extent count to go down, and the inode is converted back into extent form. The population checks then fail because it should be in btree form. Fix this by counting the number of directory data extents rather than the total number of extents in the data fork. We can do this simply by using xfs_bmap and counting the number of extents returned as it does not report extents beyond EOF (which is where the dabtree is located). As the number of data blocks does not change with the dirent removal algorithm used, this will ensure that the inode data fork remains in btree format. Signed-off-by: Dave Chinner Reviewed-by: Darrick J. Wong Reviewed-by: Gao Xiang --- common/populate | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/common/populate b/common/populate index 9b60fa5c1..7b5b16fb8 100644 --- a/common/populate +++ b/common/populate @@ -80,8 +80,11 @@ __populate_create_nfiles() { continue fi - local nextents="$(_xfs_get_fsxattr nextents $name)" - if [ "${nextents}" -gt "${max_nextents}" ]; then + # Extent count checks use data blocks only to avoid the removal + # step from removing dabtree index blocks and reducing the + # number of extents below the required threshold. + local nextents="$(xfs_bmap ${name} |grep -v hole | wc -l)" + if [ "$((nextents - 1))" -gt "${max_nextents}" ]; then echo ${d} break fi From patchwork Tue Jan 10 22:49:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Chinner X-Patchwork-Id: 13095744 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 278EFC677F1 for ; Tue, 10 Jan 2023 22:51:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235025AbjAJWvC (ORCPT ); Tue, 10 Jan 2023 17:51:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235301AbjAJWt6 (ORCPT ); Tue, 10 Jan 2023 17:49:58 -0500 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C11B365370 for ; Tue, 10 Jan 2023 14:49:12 -0800 (PST) Received: by mail-pj1-x1034.google.com with SMTP id v23so13963583pju.3 for ; Tue, 10 Jan 2023 14:49:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=7GywMSbUNbObTd9MZjb+pI5tfXCpGm+fRZqmLUIidfc=; b=BO3cwDnenTXl19qPyLLGUKubATFGbOzGrV5nxUhFO/eGnMbgvNVgerc2L+e/n7LARw 7u4GX8wRYSoOElDTQkeiRc9yTM0khDcpZRfCE+mQnRLJELfyxoGozeYGZHzKDTNV+svu q0SmweSQLMgdkmm8k3/ec+88oSnGJOGyNphkeMGbeN1n9Uh/g+hroz3i3Rggq8Fw1wl7 zn24nkJhkr96ann0OfF1KBAkyMVnYDJIiFZ5Io9mWcvjpahy7WPyKyoMC0WlLfBaQ5ct 2ICIVb/MNr+ySzgxK4ar6iesn1iALC/60avxZAWSPHOflHOhZ8xM6u8E/EmxKjCGTHn/ rvSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7GywMSbUNbObTd9MZjb+pI5tfXCpGm+fRZqmLUIidfc=; b=hBBwcu9jj6DwZuKFmVj5IgiCMiC8SyZJJbRjmIPnOsbywthFiUEQHUl1N3iAtRse6p 0IADiMHTgJJNknwA8jm2zS5Nzxocuy/4Fdn+c1WdWkpUH3AvXvLz2qIhn33W7GTEoq1T Z05HSUfuj8cv1a6BMUOtYWj57b8vLKbeBD7RcHhbCuy3oY+qyul16bOD2hAedW9qg91O xZu6/J/7mott7rUEtNcWC0oouXHVC2+hB4vChIKDpWXENsjhriptrvinbhVSEib42eZt JlR8CsP8+wNdd6gl6yuRDXrCMwTYJbnkUTC27AiAGb9xYHzIbnFrk/4yUpmUuKaGmMtC ilqw== X-Gm-Message-State: AFqh2kpnQczgru9Qw0roeCY6ofAv0WUMRhq5aLTyuNsefOsX4X+ONWuW wMKKbwNabLlJigyRN7lHUZZjq9zoca819ZF6 X-Google-Smtp-Source: AMrXdXu+x8eekwrYaWSA+M+EGD8ZF8UPBui19cAQSv0LLvtkaY7ZVqqoU3fr3mxdgkRGTr6OnlJL1Q== X-Received: by 2002:a17:902:e3d5:b0:192:6a20:3cb1 with SMTP id r21-20020a170902e3d500b001926a203cb1mr49145715ple.51.1673390952281; Tue, 10 Jan 2023 14:49:12 -0800 (PST) Received: from dread.disaster.area (pa49-186-146-207.pa.vic.optusnet.com.au. [49.186.146.207]) by smtp.gmail.com with ESMTPSA id w4-20020a170902e88400b0018999a3dd7esm8641705plg.28.2023.01.10.14.49.11 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Jan 2023 14:49:11 -0800 (PST) Received: from [192.168.253.23] (helo=devoid.disaster.area) by dread.disaster.area with esmtp (Exim 4.92.3) (envelope-from ) id 1pFNQa-001YlA-DM for fstests@vger.kernel.org; Wed, 11 Jan 2023 09:49:08 +1100 Received: from dave by devoid.disaster.area with local (Exim 4.96) (envelope-from ) id 1pFNQa-004unT-1I for fstests@vger.kernel.org; Wed, 11 Jan 2023 09:49:08 +1100 From: Dave Chinner To: fstests@vger.kernel.org Subject: [PATCH 3/3] xfs/294: performance is unreasonably slow Date: Wed, 11 Jan 2023 09:49:06 +1100 Message-Id: <20230110224906.1171483-4-david@fromorbit.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230110224906.1171483-1-david@fromorbit.com> References: <20230110224906.1171483-1-david@fromorbit.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Dave Chinner This creates a bunch of files in a dir, then deletes 97% of them attempting to leave 1 allocated inode per inode chunk so that they aren't freed. Performance is badly limited by task creation and destruction for each inode created. Fix this by using "echo -n > file" rather than touch so that the shell creates the empty files without needing to fork/exec a separate task for each creation. This reduces runtime from 45s down to 15s. Also add more debug with inode counts and internal superblock counter information for determining why this test may ENOSPC on the final creation loop. Signed-off-by: Dave Chinner Reviewed-by: David Disseldorp Reviewed-by: Zorro Lang --- tests/xfs/294 | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/tests/xfs/294 b/tests/xfs/294 index d381e2c85..1ce0d1cc5 100755 --- a/tests/xfs/294 +++ b/tests/xfs/294 @@ -28,6 +28,13 @@ _require_test_program "punch-alternating" _require_xfs_io_command "falloc" _require_xfs_io_command "fpunch" +dump_freespace() +{ + df $SCRATCH_MNT + df -i $SCRATCH_MNT + $XFS_IO_PROG -rc "statfs -c" $SCRATCH_MNT +} + # We want to mkfs with a very specific geometry MKFS_OPTIONS="" _scratch_mkfs "-d size=512m -n size=8192 -i size=1024" >> $seqres.full 2>&1 \ @@ -37,7 +44,7 @@ _scratch_mount # Make a ton of mostly-empty inode clusters so we can always # make more inodes mkdir $SCRATCH_MNT/tmp -for I in `seq 1 10000`; do touch $SCRATCH_MNT/tmp/$I; done +for I in `seq 1 10000`; do echo -n > $SCRATCH_MNT/tmp/$I; done # These mostly-empty clusters will live here: mkdir $SCRATCH_MNT/clusters @@ -50,7 +57,7 @@ rm -rf $SCRATCH_MNT/tmp mkdir $SCRATCH_MNT/testdir # roughly 20 chars per file for I in `seq 1 100`; do - touch $SCRATCH_MNT/testdir/12345678901234567890$I; + echo -n > $SCRATCH_MNT/testdir/12345678901234567890$I; done # File to fragment: @@ -63,7 +70,7 @@ space=$(stat -f -c '%f * %S * 95 / 100' $SCRATCH_MNT | $BC_PROG) $XFS_IO_PROG -f -c "falloc 0 $space" $SCRATCH_MNT/fillfile || _fail "Could not allocate space" -df -h $SCRATCH_MNT >> $seqres.full 2>&1 +dump_freespace >> $seqres.full 2>&1 # Fill remaining space; let this run to failure dd if=/dev/zero of=$SCRATCH_MNT/spacefile1 oflag=direct >> $seqres.full 2>&1 @@ -75,12 +82,16 @@ $here/src/punch-alternating $SCRATCH_MNT/fragfile >> $seqres.full 2>&1 # (and then some for good measure) dd conv=fsync if=/dev/zero of=$SCRATCH_MNT/spacefile2 bs=1M count=64 >> $seqres.full 2>&1 +dump_freespace >> $seqres.full 2>&1 + # Now populate the directory so that it must allocate these # fragmented blocks for I in `seq 1 1400`; do - touch $SCRATCH_MNT/testdir/12345678901234567890$I; + echo -n > $SCRATCH_MNT/testdir/12345678901234567890$I; done +dump_freespace >> $seqres.full 2>&1 + # Now traverse that ugly thing! find $SCRATCH_MNT/testdir | sort | _filter_scratch | md5sum