From patchwork Tue Jan 14 21:40:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13939543 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00B00209F4C for ; Tue, 14 Jan 2025 21:40:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736890839; cv=none; b=YClM4LlCmxtSkFbMlbrhsMHvI75b9w5wkY9oghEeTUsh2lzsbqtjmHMmjVMbmE15/oZT5t8vbmS7y9q0SCVKTzbqdSTt1s6ONirW9z3cWTRBCqGd6oRzHquTwsFszqfw31tbjMnTmhh3IFfWyVECEJAcAJ8WiHGwthC/R5TvZJo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736890839; c=relaxed/simple; bh=1YkTVmxcrDWYflgE3xalJKcOXiRcQiDlna/L6L2vvQI=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MbhtgQC2mW0fxCJYgACGTKMJOHCIi/il3JURfPAyuEgh8C9BVZ4mqKlYdl0kjnF6v5nDyHcTq7uk3xGvsHDLjiz3z8cIVPyaH+QhsDh/3OhBqOmCfDviuz9EVMPolWvhAY55iQqGuIyaA74pE8jHfJ0rVufeOWrrXySsD8i1Ek8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lETZSB1m; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lETZSB1m" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 89B0BC4CEDD; Tue, 14 Jan 2025 21:40:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736890838; bh=1YkTVmxcrDWYflgE3xalJKcOXiRcQiDlna/L6L2vvQI=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=lETZSB1m3ugXLd+nyXgJen3+Om+/8gySxN9sx6nXacOSDildcQkjQQIYfW/z6kClr h00NoPqkrbHKyiF+40VmVb6/BoDYuhP4QS3bkm3TR6pjpSS1MsIYNcg3fQkcBPEJBW G6k6eG09HBD6MX4Vvd7TISlms1wnNj+s6yHJsbbbTw/xjmQvxj3k/m1XZ6c7jDPSWI OX7jiF7N/JmuJ3pD8m2VgGySHxPLuxrXcXbJTT/Ei54H1qCMEL3V2gvs5fd/j3f7G/ LzjpCFjFvbG30xoLCpEopsVu0U3LXAlbtGGmX036RVeBRWg4XPiyTEYK4nmPTEYR9E 3YWEuVpiHo53g== Date: Tue, 14 Jan 2025 13:40:38 -0800 Subject: [PATCH 1/5] xfs_db: improve error message when unknown btree type given to btheight From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, hch@lst.de Message-ID: <173689081897.3476119.13308060628733838470.stgit@frogsfrogsfrogs> In-Reply-To: <173689081879.3476119.15344563789813181160.stgit@frogsfrogsfrogs> References: <173689081879.3476119.15344563789813181160.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong I found accidentally that if you do this (note 'rmap', not 'rmapbt'): xfs_db /dev/sda -c 'btheight -n 100 rmap' The program spits back "Numerical result out of range". That's the result of it failing to match "rmap" against a known btree type, and falling back to parsing the string as if it were a btree geometry description. Improve this a little by checking that there's at least one semicolon in the string so that the error message improves to: "rmap: expected a btree geometry specification" Fixes: cb1e69c564c1e0 ("xfs_db: add a function to compute btree geometry") Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- db/btheight.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/db/btheight.c b/db/btheight.c index 6643489c82c4c9..98165b522e4f6f 100644 --- a/db/btheight.c +++ b/db/btheight.c @@ -145,6 +145,12 @@ construct_records_per_block( } } + p = strchr(tag, ':'); + if (!p) { + fprintf(stderr, _("%s: expected a btree geometry specification.\n"), tag); + return -1; + } + toktag = strdup(tag); ret = -1; From patchwork Tue Jan 14 21:40:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13939544 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCDB3209F4C for ; Tue, 14 Jan 2025 21:40:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736890854; cv=none; b=ne1u+pt7baPIlWlvOocV4RuvoF8s6WQ6DY+dBDW0Ey8iWS/51fw+Pjavkg7cVOJKJdDZuZJfFfWBHQTjYMRDfJKwXraiuH9b8Wljx1VNA8dB7brC6HWpKyXD/JVRhCwzFpWGNYlr0G8KMicP5gx+TO4/XQfOO4p8IAKXUdq4uX4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736890854; c=relaxed/simple; bh=/PFYCQPoSzWSiyKoSq952MkoCdedEyMzfiyWq5CdKBo=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=i8Jv9Dt8PiEqoTy0lbmMcDvsdbRKuBg1A+rlnHqOl9krCnBzrETNFA4OLA2H2SV3gXn4SZ7BOg+BLTRLyYKvr1+Z7wZdFEAFKhKgDopGjsCkSUQXbfB1rdooqWS/pwK2mZQWyCzu/hO+am/hqH6Io4NXqi2P7vF4LfVHvdMzK0M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XiwdqT3J; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XiwdqT3J" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 463A2C4CEDD; Tue, 14 Jan 2025 21:40:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736890854; bh=/PFYCQPoSzWSiyKoSq952MkoCdedEyMzfiyWq5CdKBo=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=XiwdqT3JykveRA2319LyMmVcTY8zD5Oq14DhGlaOX3bpZhZ8qK+AyGW1fJeOtHmYg dUN54e6QAFDWZWtmfNfOw5ZAWFh4TulAA4P2M2/herr802pSSQttqlOFSR09yXOW2E xe/sdEcdf/BlaiqdXdlvmz7y5p46UhjarGwU9NANmlB269RiE7CxZ98zkjB6H0XWD/ IOTEO3yBKnzI7V+ioAYTwi9XqBK3cqkvE8oCgvQuE/8JTzJMQdlRbOeIQ8aC31P0Ep yaC0d02s5I7KJWMytDrjHrS/UCWkCR7cASg/GKTHTOJZY7H5MUd7fBhhT1QILngH67 KN0w+p9/CL//w== Date: Tue, 14 Jan 2025 13:40:53 -0800 Subject: [PATCH 2/5] mkfs: fix parsing of value-less -d/-l concurrency cli option From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, hch@lst.de Message-ID: <173689081910.3476119.11332577729920649286.stgit@frogsfrogsfrogs> In-Reply-To: <173689081879.3476119.15344563789813181160.stgit@frogsfrogsfrogs> References: <173689081879.3476119.15344563789813181160.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong It's supposed to be possible to specify the -d concurrency option with no value in order to get mkfs calculate the agcount from the number of CPUs. Unfortunately I forgot to handle that case (optarg is null) so mkfs crashes instead. Fix that. Fixes: 9338bc8b1bf073 ("mkfs: allow sizing allocation groups for concurrency") Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- mkfs/xfs_mkfs.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c index 956cc295489342..deaac2044b94dd 100644 --- a/mkfs/xfs_mkfs.c +++ b/mkfs/xfs_mkfs.c @@ -1722,7 +1722,7 @@ set_data_concurrency( * "nr_cpus" or "1" means set the concurrency level to the CPU count. * If this cannot be determined, fall back to the default AG geometry. */ - if (!strcmp(value, "nr_cpus")) + if (!value || !strcmp(value, "nr_cpus")) optnum = 1; else optnum = getnum(value, opts, subopt); @@ -1867,7 +1867,7 @@ set_log_concurrency( * "nr_cpus" or 1 means set the concurrency level to the CPU count. If * this cannot be determined, fall back to the default computation. */ - if (!strcmp(value, "nr_cpus")) + if (!value || !strcmp(value, "nr_cpus")) optnum = 1; else optnum = getnum(value, opts, subopt); From patchwork Tue Jan 14 21:41:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13939545 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4904021322A for ; Tue, 14 Jan 2025 21:41:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736890870; cv=none; b=QfB4Ma0t4KnhlCtyZhxArJAuK9o1Q0+pgOk+p9tDEzzcvmk+6LwxhABav+/9HDQRJ/yKxMiKGHqK6tuRg/1/2w2/YFf20kAIWDtYseVU4JOuBtwzseoYDFx+6J4kJrqEZJdMTmGGubOxkMKTfYmICxlkwA5FNtDfkmNxULDw4FQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736890870; c=relaxed/simple; bh=HhIakbguhjSvzIyjUQWXNglItuB2Ko8xFViSBjMk/g8=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=oWuUl9E4/TCDRqfxiQvDkNy/RjWeIlqpmpRCkxWKiy5RHVmaif1av3D4yXZKlzgeD8ryWuVPe/2ayZcbVHZJT3sSMRxUEG+R5AMGB1t2YNOl0JXoPIaUa0OJAtZF2V3G+4abC4RlJ2husUWRbAhBN0cQIQRaDvQoF75Bz7jGWhg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HKB/qIaO; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HKB/qIaO" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F0B6FC4CEDD; Tue, 14 Jan 2025 21:41:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736890870; bh=HhIakbguhjSvzIyjUQWXNglItuB2Ko8xFViSBjMk/g8=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=HKB/qIaODyje5V4+CKGd1ERt7bv3LByfhYIkH3x/P+I9eTbiKPXu6RoXJ0FHvDfZo XaSqEQbN/cK2nVMPLnOw/SE8E6n93DeFI3pRJffeSwrY32t/5prlyhAryJ+iIomBlM Al1DYbLVyrQv2NvpnxWzuM15RzYXVLDsr7uea7GFhVZDbTvrGgBcsMmkF10LtfJvFY iLklQ28R9l7sshpVte0gJFdvKrPb/Rs5HvXlfLuc0r13cYepWiYn/w4QtfW+JwChLs d653GQM4FHe8+WDv3My/GqUwo/RVEDgKmdOOz7QHYcoyqGPoOjFd2ytADoNFujTw30 fCKIoJGlY86Jw== Date: Tue, 14 Jan 2025 13:41:09 -0800 Subject: [PATCH 3/5] m4: fix statx override selection if /usr/include doesn't define it From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, hch@lst.de Message-ID: <173689081926.3476119.12874616172883806766.stgit@frogsfrogsfrogs> In-Reply-To: <173689081879.3476119.15344563789813181160.stgit@frogsfrogsfrogs> References: <173689081879.3476119.15344563789813181160.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong If the system headers (aka the ones in /usr/include) do not define struct statx at all, we need to use our internal override. The m4 code doesn't handle this admittedly corner case, but let's fix it for anyone trying to build new xfsprogs on a decade-old distribution. Fixes: 409477af604f46 ("xfs_io: add support for atomic write statx fields") Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- m4/package_libcdev.m4 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/m4/package_libcdev.m4 b/m4/package_libcdev.m4 index 6db1177350b643..4ef7e8f67a3ba6 100644 --- a/m4/package_libcdev.m4 +++ b/m4/package_libcdev.m4 @@ -112,7 +112,7 @@ AC_DEFUN([AC_NEED_INTERNAL_STATX], need_internal_statx=yes, [#include ] ) - ],, + ],need_internal_statx=yes, [#include ] ) AC_SUBST(need_internal_statx) From patchwork Tue Jan 14 21:41:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13939546 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 250FD146590 for ; Tue, 14 Jan 2025 21:41:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736890888; cv=none; b=Ph+hAvdDUKHV//yYoNHZVswb3J8s/fNIViaP8f4Ljkol4T3MqssCmmuQxdIWIKgJWDLvWfqJWr97MbqZ+epn/92eMHL9yyPId/CqZ0JJ4CnENcPQ8Qy/e626FIpyrkUt4cdDO8JuoZF35p/l3N4zjTUSXf4dWTMbZhGRo0u91ow= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736890888; c=relaxed/simple; bh=vgzVjP0HKubT4A9f+9QvYz0mcrngRluDZKdQZ9TwvH4=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=utBuMV4+qv5Bl6aRPlMfBVPPPdWWv60z8Ij+zlv+4GkGoCYcFJ82KL4Cy0Iu24h3aV9BOEaVhPSyrmKozsuGLzYpe+hzZQ+xvVt4IJXDY1zSm/aR9qc1wV9U7UJkqHDgdXKAPP7u76yNxn1DhqFXgrOVkOFs8JhN1gRayeNuV9Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kwwcLCg6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kwwcLCg6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 90337C4CEDD; Tue, 14 Jan 2025 21:41:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736890885; bh=vgzVjP0HKubT4A9f+9QvYz0mcrngRluDZKdQZ9TwvH4=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=kwwcLCg65uU8L3H+0TEpZdrlomZRd0NKIccNo7X7IhKS8DmKDkrpYN5kyO3Vt2uaE tdaCp5jmsctkiWQ1E2gRdK/rfj2ZQN9Ze2S8DK4s4nyAhoImS8l72o9P4MP9exDt4N E8FJVSfZs6fq0dU49FOQrQKSlz15kR9xWb5wADO9s/iZZcmbTRGgfZDHcgGbNizFx2 3TyYVImilrm+QaUuS1GQ5u8VgTwb+UQ9T7BdpcD2GijLZgZkH5bXS8tDNy4hFoT5BE BV67zRMnEE0cR44zoVkVB0HsUkMD0b454bZLk5Sb20vqxFmd9XxgpAharOtOgFJ4dS 3iKGoEoK91QWQ== Date: Tue, 14 Jan 2025 13:41:25 -0800 Subject: [PATCH 4/5] build: initialize stack variables to zero by default From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, hch@lst.de Message-ID: <173689081941.3476119.6143322419702468919.stgit@frogsfrogsfrogs> In-Reply-To: <173689081879.3476119.15344563789813181160.stgit@frogsfrogsfrogs> References: <173689081879.3476119.15344563789813181160.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Newer versions of gcc and clang can include the ability to zero stack variables by default. Let's enable it so that we (a) reduce the risk of writing stack contents to disk somewhere and (b) try to reduce unpredictable program behavior based on random stack contents. The kernel added this 6 years ago, so I think it's mature enough for xfsprogs. Signed-off-by: "Darrick J. Wong" --- configure.ac | 1 + include/builddefs.in | 2 +- m4/package_sanitizer.m4 | 14 ++++++++++++++ 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index 224d1d3930bf2f..90ef7925a925d0 100644 --- a/configure.ac +++ b/configure.ac @@ -177,6 +177,7 @@ AC_CONFIG_SYSTEMD_SYSTEM_UNIT_DIR AC_CONFIG_CROND_DIR AC_CONFIG_UDEV_RULE_DIR AC_HAVE_BLKID_TOPO +AC_HAVE_TRIVIAL_AUTO_VAR_INIT if test "$enable_ubsan" = "yes" || test "$enable_ubsan" = "probe"; then AC_PACKAGE_CHECK_UBSAN diff --git a/include/builddefs.in b/include/builddefs.in index ac43b6412c8cbb..82840ec7fd3adb 100644 --- a/include/builddefs.in +++ b/include/builddefs.in @@ -146,7 +146,7 @@ ifeq ($(HAVE_LIBURCU_ATOMIC64),yes) PCFLAGS += -DHAVE_LIBURCU_ATOMIC64 endif -SANITIZER_CFLAGS += @addrsan_cflags@ @threadsan_cflags@ @ubsan_cflags@ +SANITIZER_CFLAGS += @addrsan_cflags@ @threadsan_cflags@ @ubsan_cflags@ @autovar_init_cflags@ SANITIZER_LDFLAGS += @addrsan_ldflags@ @threadsan_ldflags@ @ubsan_ldflags@ # Use special ar/ranlib wrappers if we have lto diff --git a/m4/package_sanitizer.m4 b/m4/package_sanitizer.m4 index 41b729906a27ba..6488f7ebce2f50 100644 --- a/m4/package_sanitizer.m4 +++ b/m4/package_sanitizer.m4 @@ -57,3 +57,17 @@ AC_DEFUN([AC_PACKAGE_CHECK_THREADSAN], AC_SUBST(threadsan_cflags) AC_SUBST(threadsan_ldflags) ]) + +# Check if we have -ftrivial-auto-var-init=zero +AC_DEFUN([AC_HAVE_TRIVIAL_AUTO_VAR_INIT], + [ AC_MSG_CHECKING([if C compiler supports zeroing automatic vars]) + OLD_CFLAGS="$CFLAGS" + TEST_CFLAGS="-ftrivial-auto-var-init=zero" + CFLAGS="$CFLAGS $TEST_CFLAGS" + AC_LINK_IFELSE([AC_LANG_PROGRAM([])], + [AC_MSG_RESULT([yes])] + [autovar_init_cflags=$TEST_CFLAGS], + [AC_MSG_RESULT([no])]) + CFLAGS="${OLD_CFLAGS}" + AC_SUBST(autovar_init_cflags) + ]) From patchwork Tue Jan 14 21:41:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13939547 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9BE6C146590 for ; Tue, 14 Jan 2025 21:41:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736890902; cv=none; b=BJbmCMw9ifDyKQS7F9dDPX86jtcu3+LvUsoceaTcdzPdcMQrFZcEzH/TKkz5Nrv1Qqrt2LuBNTcZ9AQbUVVY2Yi0KiKvwj6GvOjDuE7Rq00YakHX7AY7+HQIH29jHEpmxGvJCeKc6nKqrSE5oe70m80XBeQuFhORz90e5I1N4iA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736890902; c=relaxed/simple; bh=oEESqqLibXrBoToFC6oJ31G9N7N0ZcZnCJmilDX4qvw=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Z+qUDsUvs4YITLeehCNvJTjvdfIrdJPXgmKmVFEVDIeXDpVOHL4w4Kv/l+BmmSySg3wEX4BLbgooEE69rkaSj/UAl15u+X5RWNLcn19sQbOi3wpyzAtyJL7KiWUB4UGroYub6k9tMJCfx4JPd8sBqjCAQ51Bvutx5R2w22yYcvw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Squz5NUv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Squz5NUv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 30A09C4CEE3; Tue, 14 Jan 2025 21:41:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736890901; bh=oEESqqLibXrBoToFC6oJ31G9N7N0ZcZnCJmilDX4qvw=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=Squz5NUvIUlc1fgAP0W+bNTvJNTrzukxsacCHnlfmsQGBeXRVtqgj+XsAi/BeLKSu f18SKnADV+HtiGPbLyvOV+oRgWA+FT8Jle0y0h431MY8+QvDhcG/gDS1xr71Qk8Rzp Ee/Ac6QkN8RBR3TF6fJV/EN2j9R6EdCmZTbEeHjVX39mHThHfIXBWERq4ei8EcBSC9 ShJW3WGD5YDzhuqXBFd68KNjF0iOuZyqKBmRoiLn9UwvjpCyP9A9zgkJaOwcdiQz/t IR1d0PgpkvoyxpsZme03ZMayPSFAIl34rzqHylMFZQdYi0AD8wqjkAN3X1JRUfMiqb SjH1Wroh7lZyw== Date: Tue, 14 Jan 2025 13:41:40 -0800 Subject: [PATCH 5/5] mkfs: allow sizing realtime allocation groups for concurrency From: "Darrick J. Wong" To: aalbersh@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, hch@lst.de Message-ID: <173689081956.3476119.7466311188976179968.stgit@frogsfrogsfrogs> In-Reply-To: <173689081879.3476119.15344563789813181160.stgit@frogsfrogsfrogs> References: <173689081879.3476119.15344563789813181160.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Add a -r concurrency= option to mkfs so that sysadmins can configure the filesystem so that there are enough rtgroups that the specified number of threads can (in theory) can find an uncontended rtgroup from which to allocate space. This has the exact same purpose as the -d concurrency switch that was added for the data device. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- man/man8/mkfs.xfs.8.in | 28 ++++++++++ mkfs/xfs_mkfs.c | 140 +++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 165 insertions(+), 3 deletions(-) diff --git a/man/man8/mkfs.xfs.8.in b/man/man8/mkfs.xfs.8.in index 32361cf973fcf8..37e3a88e7ac777 100644 --- a/man/man8/mkfs.xfs.8.in +++ b/man/man8/mkfs.xfs.8.in @@ -1220,6 +1220,34 @@ .SH OPTIONS and .B rgsize suboptions are mutually exclusive. +.TP +.BI concurrency= value +Create enough realtime allocation groups to handle the desired level of +concurrency. +The goal of this calculation scheme is to set the number of rtgroups to an +integer multiple of the number of writer threads desired, to minimize +contention of rtgroup locks. +This scheme will neither create fewer rtgroups than would be created by the +default configuration, nor will it create rtgroups smaller than 4GB. +This option is not compatible with the +.B rgcount +or +.B rgsize +options. +The magic value +.I nr_cpus +or +.I 1 +or no value at all will set this parameter to the number of active processors +in the system. +If the kernel advertises that the realtime device is a non-mechanical storage +device, +.B mkfs.xfs +will use this new geometry calculation scheme. +The magic value of +.I 0 +forces use of the older rtgroups geometry calculations that is used for +mechanical storage. .RE .PP .PD 0 diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c index deaac2044b94dd..073e79ac58303c 100644 --- a/mkfs/xfs_mkfs.c +++ b/mkfs/xfs_mkfs.c @@ -134,6 +134,7 @@ enum { R_NOALIGN, R_RGCOUNT, R_RGSIZE, + R_CONCURRENCY, R_MAX_OPTS, }; @@ -737,6 +738,7 @@ static struct opt_params ropts = { [R_NOALIGN] = "noalign", [R_RGCOUNT] = "rgcount", [R_RGSIZE] = "rgsize", + [R_CONCURRENCY] = "concurrency", [R_MAX_OPTS] = NULL, }, .subopt_params = { @@ -778,6 +780,7 @@ static struct opt_params ropts = { }, { .index = R_RGCOUNT, .conflicts = { { &ropts, R_RGSIZE }, + { &ropts, R_CONCURRENCY }, { NULL, LAST_CONFLICT } }, .minval = 1, .maxval = XFS_MAX_RGNUMBER, @@ -785,12 +788,22 @@ static struct opt_params ropts = { }, { .index = R_RGSIZE, .conflicts = { { &ropts, R_RGCOUNT }, + { &ropts, R_CONCURRENCY }, { NULL, LAST_CONFLICT } }, .convert = true, .minval = 0, .maxval = (unsigned long long)XFS_MAX_RGBLOCKS << XFS_MAX_BLOCKSIZE_LOG, .defaultval = SUBOPT_NEEDS_VAL, }, + { .index = R_CONCURRENCY, + .conflicts = { { &ropts, R_RGCOUNT }, + { &ropts, R_RGSIZE }, + { NULL, LAST_CONFLICT } }, + .convert = true, + .minval = 0, + .maxval = INT_MAX, + .defaultval = 1, + }, }, }; @@ -1034,6 +1047,7 @@ struct cli_params { int proto_slashes_are_spaces; int data_concurrency; int log_concurrency; + int rtvol_concurrency; /* parameters where 0 is not a valid value */ int64_t agcount; @@ -1157,7 +1171,8 @@ usage( void ) /* no-op info only */ [-N]\n\ /* prototype file */ [-p fname]\n\ /* quiet */ [-q]\n\ -/* realtime subvol */ [-r extsize=num,size=num,rtdev=xxx,rgcount=n,rgsize=n]\n\ +/* realtime subvol */ [-r extsize=num,size=num,rtdev=xxx,rgcount=n,rgsize=n,\n\ + concurrency=num]\n\ /* sectorsize */ [-s size=num]\n\ /* version */ [-V]\n\ devicename\n\ @@ -2071,6 +2086,31 @@ proto_opts_parser( return 0; } +static void +set_rtvol_concurrency( + struct opt_params *opts, + int subopt, + struct cli_params *cli, + const char *value) +{ + long long optnum; + + /* + * "nr_cpus" or "1" means set the concurrency level to the CPU count. + * If this cannot be determined, fall back to the default rtgroup + * geometry. + */ + if (!value || !strcmp(value, "nr_cpus")) + optnum = 1; + else + optnum = getnum(value, opts, subopt); + + if (optnum == 1) + cli->rtvol_concurrency = nr_cpus(); + else + cli->rtvol_concurrency = optnum; +} + static int rtdev_opts_parser( struct opt_params *opts, @@ -2101,6 +2141,9 @@ rtdev_opts_parser( case R_RGSIZE: cli->rgsize = getstr(value, opts, subopt); break; + case R_CONCURRENCY: + set_rtvol_concurrency(opts, subopt, cli, value); + break; default: return -EINVAL; } @@ -3740,10 +3783,97 @@ _("realtime group size (%llu) not at all congruent with extent size (%llu)\n"), return 0; } +static bool +rtdev_is_solidstate( + struct libxfs_init *xi) +{ + unsigned short rotational = 1; + int error; + + error = ioctl(xi->rt.fd, BLKROTATIONAL, &rotational); + if (error) + return false; + + return rotational == 0; +} + +static void +calc_concurrency_rtgroup_geometry( + struct mkfs_params *cfg, + struct cli_params *cli, + struct libxfs_init *xi) +{ + uint64_t try_rgsize; + uint64_t def_rgsize; + uint64_t def_rgcount; + int nr_threads = cli->rtvol_concurrency; + int try_threads; + + if (is_power_of_2(cfg->rtextblocks)) + def_rgsize = calc_rgsize_extsize_power(cfg); + else + def_rgsize = calc_rgsize_extsize_nonpower(cfg); + def_rgcount = howmany(cfg->rtblocks, def_rgsize); + try_rgsize = def_rgsize; + + /* + * If the caller doesn't have a particular concurrency level in mind, + * set it to the number of CPUs in the system. + */ + if (nr_threads < 0) + nr_threads = nr_cpus(); + + /* + * Don't create fewer rtgroups than what we would create with the + * default geometry calculation. + */ + if (!nr_threads || nr_threads < def_rgcount) + goto out; + + /* + * Let's try matching the number of rtgroups to the number of CPUs. If + * the proposed geometry results in rtgroups smaller than 4GB, reduce + * the rtgroup count until we have 4GB rtgroups. Don't let the thread + * count go below the default geometry calculation. + */ + try_threads = nr_threads; + try_rgsize = cfg->rtblocks / try_threads; + if (try_rgsize < GIGABYTES(4, cfg->blocklog)) { + do { + try_threads--; + if (try_threads <= def_rgcount) { + try_rgsize = def_rgsize; + goto out; + } + + try_rgsize = cfg->rtblocks / try_threads; + } while (try_rgsize < GIGABYTES(4, cfg->blocklog)); + goto out; + } + + /* + * For large filesystems we try to ensure that the rtgroup count is a + * multiple of the desired thread count. Specifically, if the proposed + * rtgroup size is larger than both the maximum rtgroup size and the + * rtgroup size we would have gotten with the defaults, add the thread + * count to the rtgroup count until we get an rtgroup size below both + * of those factors. + */ + while (try_rgsize > XFS_MAX_RGBLOCKS && try_rgsize > def_rgsize) { + try_threads += nr_threads; + try_rgsize = cfg->dblocks / try_threads; + } + +out: + cfg->rgsize = try_rgsize; + cfg->rgcount = howmany(cfg->rtblocks, cfg->rgsize); +} + static void calculate_rtgroup_geometry( struct mkfs_params *cfg, - struct cli_params *cli) + struct cli_params *cli, + struct libxfs_init *xi) { if (!cli->sb_feat.metadir) { cfg->rgcount = 0; @@ -3783,6 +3913,9 @@ _("rgsize (%s) not a multiple of fs blk size (%d)\n"), /* too small even for a single group */ cfg->rgsize = cfg->rtblocks; cfg->rgcount = 0; + } else if (cli->rtvol_concurrency > 0 || + (cli->data_concurrency == -1 && rtdev_is_solidstate(xi))) { + calc_concurrency_rtgroup_geometry(cfg, cli, xi); } else if (is_power_of_2(cfg->rtextblocks)) { cfg->rgsize = calc_rgsize_extsize_power(cfg); cfg->rgcount = cfg->rtblocks / cfg->rgsize + @@ -4890,6 +5023,7 @@ main( .is_supported = 1, .data_concurrency = -1, /* auto detect non-mechanical storage */ .log_concurrency = -1, /* auto detect non-mechanical ddev */ + .rtvol_concurrency = -1, /* auto detect non-mechanical rtdev */ .autofsck = FSPROP_AUTOFSCK_UNSET, }; struct mkfs_params cfg = {}; @@ -5077,7 +5211,7 @@ main( */ calculate_initial_ag_geometry(&cfg, &cli, &xi); align_ag_geometry(&cfg); - calculate_rtgroup_geometry(&cfg, &cli); + calculate_rtgroup_geometry(&cfg, &cli, &xi); calculate_imaxpct(&cfg, &cli);