From patchwork Wed Sep 27 17:46:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Burkov X-Patchwork-Id: 13401456 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74733E82CA1 for ; Wed, 27 Sep 2023 17:45:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229464AbjI0Rp5 (ORCPT ); Wed, 27 Sep 2023 13:45:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49278 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229458AbjI0Rp4 (ORCPT ); Wed, 27 Sep 2023 13:45:56 -0400 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B42A311D for ; Wed, 27 Sep 2023 10:45:54 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 063EB32009AE; Wed, 27 Sep 2023 13:45:53 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Wed, 27 Sep 2023 13:45:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bur.io; h=cc :content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm3; t=1695836753; x= 1695923153; bh=yFWx4Cwk0irAiOCFjO+5YbO/UedhaVkn1He3LKXHkN4=; b=G GtYeAwvEKFPcdPs+lp0b5Qy1k09VHUqokjwL4btIyG8ygWepTTiNNSPyWsIBitOD P3QG6UWW0M3xIMbUDuWLfDr8eOxQ6vDgSXLX/atpzRRmTM71GBj6n8nXHJRwim7S lAd69wozRIOHK/YO29QzfCnUJQgjRm1S6bApigrLvznSO+I2DFUQ+On3u9HlvwYa hotrFcLQrZwPj8pb/cD7lzlbDLWCsJNLokfNWxCfPI9349lraU8BViJbMHXAkZjG OleX9HJ/W52PcSuUsf7kwKLw9cAhoVAvBuAOa0T7X7i77DSC5aPfVeIJStHuV9ay jfbT6o13sbxFrkMJYcZTA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1695836753; x=1695923153; bh=y FWx4Cwk0irAiOCFjO+5YbO/UedhaVkn1He3LKXHkN4=; b=h47dPcX6I2iBmFxmE r4dlU27wEnY0d4MxFFBdkGVITT8tkZ9McF3VPehdb0hSZJ78o4huaMd2uVseFBJh JtWtEFeLsN8Cd9sjIx7xHS1nx64G6U58VDj2lnlRuHYi4KsEiLTzlwreurIA3LYw JGu3/WB9Hrc2w1S686o8LTQvzrJZEhli+hNDDM4OyOwcMlpxUOxSJgWXYeI1OyxX qmoid6ArWSPI97TFaOsswxJWqaC9ZaUrsGentWwZLt7g0oyYgiY8RxcyBMsyTuA7 X8L+vhLXtCb5sbN5waKD7a3ai9sBfdCNYyLY4OdghfLzEtqGsIk1rmBGtM+4dsGd ISYUw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvjedrtdeggdehgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpeeuohhrihhsuceuuhhrkhhovhcuoegsohhrihhssegsuhhrrdhi oheqnecuggftrfgrthhtvghrnhepieeuffeuvdeiueejhfehiefgkeevudejjeejffevvd ehtddufeeihfekgeeuheelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehm rghilhhfrhhomhepsghorhhishessghurhdrihho X-ME-Proxy: Feedback-ID: i083147f8:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 27 Sep 2023 13:45:52 -0400 (EDT) From: Boris Burkov To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v3 1/8] btrfs-progs: document squotas Date: Wed, 27 Sep 2023 10:46:42 -0700 Message-ID: X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Document the new options in btrfs quota and mkfs.btrfs. Also, add a section to the long form qgroups document about squotas. Signed-off-by: Boris Burkov --- Documentation/btrfs-quota.rst | 7 +++- Documentation/ch-quota-intro.rst | 59 ++++++++++++++++++++++++++++++++ Documentation/mkfs.btrfs.rst | 6 ++++ 3 files changed, 71 insertions(+), 1 deletion(-) diff --git a/Documentation/btrfs-quota.rst b/Documentation/btrfs-quota.rst index 04d53ef53..2103130fd 100644 --- a/Documentation/btrfs-quota.rst +++ b/Documentation/btrfs-quota.rst @@ -47,9 +47,14 @@ SUBCOMMAND disable Disable subvolume quota support for a filesystem. -enable +enable [options] Enable subvolume quota support for a filesystem. + ``Options`` + + -s|--simple + use simple quotas (squotas) instead of qgroups + rescan [options] Trash all qgroup numbers and scan the metadata again with the current config. diff --git a/Documentation/ch-quota-intro.rst b/Documentation/ch-quota-intro.rst index 1c1228f86..094a510cd 100644 --- a/Documentation/ch-quota-intro.rst +++ b/Documentation/ch-quota-intro.rst @@ -188,3 +188,62 @@ but some snapshots for backup purposes are being created by the system. The user's snapshots should be accounted to the user, not the system. The solution is similar to the one from section *Accounting snapshots to the user*, but do not assign system snapshots to user's qgroup. + +Simple Quotas (squotas) +^^^^^^^^^^^^^^^^^^^^^^^ + +As detailed in this document, qgroups can handle many complex extent sharing +and unsharing scenarios while maintaining an accurate count of exclusive and +shared usage. However, this flexibility comes at a cost: many of the +computations are global, in the sense that we must count up the number of trees +referring to an extent after its references change. This can slow down +transaction commits and lead to unacceptable latencies, especially in cases +where snapshots scale up. + +To work around this limitation of qgroups, btrfs also supports a second set of +quota semantics: simple quotas or squotas. Squotas fully share the qgroups API +and hierarchical model, but do not track shared vs. exclusive usage. Instead, +they account all extents to the subvolume that first allocated it. With a bit +of new bookkeeping, this allows all accounting decisions to be local to the +allocation or freeing operation that deals with the extents themselves, and +fully avoids the complex and costly back-reference resolutions. + +``Example`` + +To illustrate the difference between squotas and qgroups, consider the following +basic example assuming a nodesize of 16KiB. + +1. create subvolume 256 +2. rack up 1GiB of data and metadata usage in 256 +3. snapshot 256, creating subvolume 257 +4. CoW 512MiB of the data and metadata in 257 +5. delete everything in 256 + +At each step, qgroups would have the following accounting: +1. 0/256: 16KiB excl 0 shared +2. 0/256: 1GiB excl 0 shared +3. 0/256: 0 excl 1GiB shared; 0/257: 0 excl 1GiB shared +4. 0/256: 512MiB excl 512MiB shared; 0/257: 512MiB excl 512MiB shared +5. 0/256: 16KiB excl 0 shared; 0/257: 1GiB excl 0 shared + +Whereas under squotas, the accounting would look like: +1. 0/256: 16KiB excl 16KiB shared +2. 0/256: 1GiB excl 1GiB shared +3. 0/256: 1GiB excl 1GiB shared; 0/257: 16KiB excl 16KiB shared +4. 0/256: 1GiB excl 1GiB shared; 0/257: 512MiB excl 512MiB shared +5. 0/256: 512MiB excl 512MiB shared; 0/257: 512MiB excl 512MiB shared + +Note that since the original snapshotted 512MiB are still referenced by 257, +they cannot be freed from 256, even after 256 is emptied, or even deleted. + +``Summary`` + +If you want some of power and flexibility of quotas for tracking and limiting +subvolume usage, but want to avoid the performance penalty of accurately +tracking extent ownership lifecycles, then squotas can be a useful option. + +Furthermore, squotas is targeted at use cases where the original extent is +immutable, like image snapshotting for container startup, in which case we avoid +these awkward scenarios where a subvolume is empty or deleted but still has +significant extents accounted to it. However, as long as you are aware of the +accounting semantics, they can handle mutable original extents. diff --git a/Documentation/mkfs.btrfs.rst b/Documentation/mkfs.btrfs.rst index 1fca7448b..da8e708ae 100644 --- a/Documentation/mkfs.btrfs.rst +++ b/Documentation/mkfs.btrfs.rst @@ -326,6 +326,12 @@ block-group-tree enabled at *mkfs* time is possible, see :doc:`btrfstune`. Online conversion is not possible. +squota + (kernel support since X.X) + + Enable simple quota support (squotas). This is an alternative to qgroups with + a smaller performance impact but no notion of shared vs. exclusive usage. + .. _mkfs-section-profiles: BLOCK GROUPS, CHUNKS, RAID From patchwork Wed Sep 27 17:46:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Burkov X-Patchwork-Id: 13401457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36B29E82CBD for ; Wed, 27 Sep 2023 17:46:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229476AbjI0Rp7 (ORCPT ); Wed, 27 Sep 2023 13:45:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229473AbjI0Rp6 (ORCPT ); Wed, 27 Sep 2023 13:45:58 -0400 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8536BF5 for ; Wed, 27 Sep 2023 10:45:57 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id AA7D6320094C; Wed, 27 Sep 2023 13:45:56 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Wed, 27 Sep 2023 13:45:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bur.io; h=cc :content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm3; t=1695836756; x= 1695923156; bh=GQZL2z8JNLswgRqQXPROAzG4h6fcUbinjS35PqXh9MA=; b=u +hhppwIE3i1wWCcdYU4UBEs2lab4sh0OpaPraepSR+EIbkjjytuKaXoTjWfumkBs Ua05YG7lOf2+MbyWUfPQIYW+ZDQ/J2wpzwNDUt9sgv9UKCxs101sGWBUANK+hhQ1 kLI09ZJpeObEvzAg4EOIcr1qTTfmy2HIu1R3F/IrsueM6jlXgzwXEgzG/aFcJue5 P5H9APoTF4PEJb0PD2Nv8PsRAUalb+VltB1CEUao3xUYG8V+dHHv0yYmca4iNWSk TaHFPURU5zU/24eZs3l1To8Yb0ZgIdC570BkwNECew6LoYs8Y/bvrLiOZeBtKwfq /cHWl/OxJuSxjUsYczngw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1695836756; x=1695923156; bh=G QZL2z8JNLswgRqQXPROAzG4h6fcUbinjS35PqXh9MA=; b=bmmOQsWeyB00TfAKI Z6Vdt10r+zj9p7Bw3fZBGQclzt++xI8/tUdtbzjmXFtaG6reUl+rDji2WKShNxlP PMLhEs1qkDglrnJJgqXK0yx76oDZAP5pYXfAOWGMmJ1Beg96kEz7p3zPE/XtkWyT 6XnSPnkP6uMlehApmcwGYf0Tn0fZhmc7R4wuxlzy6lfEXQiPPebindSaNn1ydcq9 CgudoNKCYZx5I9OEKkW+kgogkOByy0BSY6FzmVFXy8AtR3IJlb7CZyyLyIqfDPUJ 6dLuvEDlTukzIgMO24/6sq7qGJYOonPf19SuTyPbFiqUgoiwNI+tMUdF7d99tZ00 KHAVA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvjedrtdeggdehgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpeeuohhrihhsuceuuhhrkhhovhcuoegsohhrihhssegsuhhrrdhi oheqnecuggftrfgrthhtvghrnhepieeuffeuvdeiueejhfehiefgkeevudejjeejffevvd ehtddufeeihfekgeeuheelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehm rghilhhfrhhomhepsghorhhishessghurhdrihho X-ME-Proxy: Feedback-ID: i083147f8:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 27 Sep 2023 13:45:55 -0400 (EDT) From: Boris Burkov To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v3 2/8] btrfs-progs: simple quotas kernel definitions Date: Wed, 27 Sep 2023 10:46:43 -0700 Message-ID: <88ca4f96ddc75dcb52f15b9b175bb31f6f6a331f.1695836680.git.boris@bur.io> X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Copy over structs, accessors, and constants for simple quotas Signed-off-by: Boris Burkov Reviewed-by: Josef Bacik --- kernel-shared/accessors.h | 9 +++++++++ kernel-shared/ctree.h | 6 ++++-- kernel-shared/uapi/btrfs.h | 1 + kernel-shared/uapi/btrfs_tree.h | 16 +++++++++++++++- 4 files changed, 29 insertions(+), 3 deletions(-) diff --git a/kernel-shared/accessors.h b/kernel-shared/accessors.h index 539c20d09..ab8c2d337 100644 --- a/kernel-shared/accessors.h +++ b/kernel-shared/accessors.h @@ -379,9 +379,13 @@ static inline u32 btrfs_extent_inline_ref_size(int type) if (type == BTRFS_EXTENT_DATA_REF_KEY) return sizeof(struct btrfs_extent_data_ref) + offsetof(struct btrfs_extent_inline_ref, offset); + if (type == BTRFS_EXTENT_OWNER_REF_KEY) + return sizeof(struct btrfs_extent_inline_ref); return 0; } +BTRFS_SETGET_FUNCS(extent_owner_ref_root_id, struct btrfs_extent_owner_ref, root_id, 64); + /* struct btrfs_node */ BTRFS_SETGET_FUNCS(key_blockptr, struct btrfs_key_ptr, blockptr, 64); BTRFS_SETGET_FUNCS(key_generation, struct btrfs_key_ptr, generation, 64); @@ -979,6 +983,9 @@ BTRFS_SETGET_FUNCS(qgroup_status_flags, struct btrfs_qgroup_status_item, flags, 64); BTRFS_SETGET_FUNCS(qgroup_status_rescan, struct btrfs_qgroup_status_item, rescan, 64); +BTRFS_SETGET_FUNCS(qgroup_status_enable_gen, struct btrfs_qgroup_status_item, + enable_gen, 64); + BTRFS_SETGET_STACK_FUNCS(stack_qgroup_status_generation, struct btrfs_qgroup_status_item, generation, 64); BTRFS_SETGET_STACK_FUNCS(stack_qgroup_status_version, @@ -987,6 +994,8 @@ BTRFS_SETGET_STACK_FUNCS(stack_qgroup_status_flags, struct btrfs_qgroup_status_item, flags, 64); BTRFS_SETGET_STACK_FUNCS(stack_qgroup_status_rescan, struct btrfs_qgroup_status_item, rescan, 64); +BTRFS_SETGET_STACK_FUNCS(stack_qgroup_status_enable_gen, + struct btrfs_qgroup_status_item, enable_gen, 64); /* btrfs_qgroup_info_item */ BTRFS_SETGET_FUNCS(qgroup_info_generation, struct btrfs_qgroup_info_item, diff --git a/kernel-shared/ctree.h b/kernel-shared/ctree.h index 59533879b..bdfebd665 100644 --- a/kernel-shared/ctree.h +++ b/kernel-shared/ctree.h @@ -103,7 +103,8 @@ static inline u32 __BTRFS_LEAF_DATA_SIZE(u32 nodesize) BTRFS_FEATURE_INCOMPAT_RAID1C34 | \ BTRFS_FEATURE_INCOMPAT_METADATA_UUID | \ BTRFS_FEATURE_INCOMPAT_ZONED | \ - BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2) + BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2 | \ + BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA) #else #define BTRFS_FEATURE_INCOMPAT_SUPP \ (BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF | \ @@ -118,7 +119,8 @@ static inline u32 __BTRFS_LEAF_DATA_SIZE(u32 nodesize) BTRFS_FEATURE_INCOMPAT_NO_HOLES | \ BTRFS_FEATURE_INCOMPAT_RAID1C34 | \ BTRFS_FEATURE_INCOMPAT_METADATA_UUID | \ - BTRFS_FEATURE_INCOMPAT_ZONED) + BTRFS_FEATURE_INCOMPAT_ZONED | \ + BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA) #endif /* diff --git a/kernel-shared/uapi/btrfs.h b/kernel-shared/uapi/btrfs.h index 85b04f89a..7e0078a5d 100644 --- a/kernel-shared/uapi/btrfs.h +++ b/kernel-shared/uapi/btrfs.h @@ -356,6 +356,7 @@ _static_assert(sizeof(struct btrfs_ioctl_fs_info_args) == 1024); #define BTRFS_FEATURE_INCOMPAT_RAID1C34 (1ULL << 11) #define BTRFS_FEATURE_INCOMPAT_ZONED (1ULL << 12) #define BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2 (1ULL << 13) +#define BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA (1ULL << 16) struct btrfs_ioctl_feature_flags { __u64 compat_flags; diff --git a/kernel-shared/uapi/btrfs_tree.h b/kernel-shared/uapi/btrfs_tree.h index ad555e705..29cf2f3d3 100644 --- a/kernel-shared/uapi/btrfs_tree.h +++ b/kernel-shared/uapi/btrfs_tree.h @@ -227,6 +227,8 @@ #define BTRFS_SHARED_DATA_REF_KEY 184 +#define BTRFS_EXTENT_OWNER_REF_KEY 188 + /* * block groups give us hints into the extent allocation trees. Which * blocks are free etc etc @@ -783,6 +785,10 @@ struct btrfs_shared_data_ref { __le32 count; } __attribute__ ((__packed__)); +struct btrfs_extent_owner_ref { + __le64 root_id; +} __attribute__ ((__packed__)); + struct btrfs_extent_inline_ref { __u8 type; __le64 offset; @@ -1199,10 +1205,12 @@ static inline __u16 btrfs_qgroup_level(__u64 qgroupid) * Turning qouta off and on again makes it inconsistent, too. */ #define BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT (1ULL << 2) +#define BTRFS_QGROUP_STATUS_FLAG_SIMPLE_MODE (1ULL << 3) #define BTRFS_QGROUP_STATUS_FLAGS_MASK (BTRFS_QGROUP_STATUS_FLAG_ON | \ BTRFS_QGROUP_STATUS_FLAG_RESCAN | \ - BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT) + BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT | \ + BTRFS_QGROUP_STATUS_FLAG_SIMPLE_MODE) #define BTRFS_QGROUP_STATUS_VERSION 1 @@ -1224,6 +1232,12 @@ struct btrfs_qgroup_status_item { * of the scan. It contains a logical address */ __le64 rescan; + + /* + * Used by simple quotas to ignore old extent deletions + * Present iff incompat flag SIMPLE_QUOTA is set + */ + __le64 enable_gen; } __attribute__ ((__packed__)); struct btrfs_qgroup_info_item { From patchwork Wed Sep 27 17:46:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Burkov X-Patchwork-Id: 13401458 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 129E1E82CBB for ; Wed, 27 Sep 2023 17:46:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229530AbjI0RqD (ORCPT ); Wed, 27 Sep 2023 13:46:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49324 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229473AbjI0RqC (ORCPT ); Wed, 27 Sep 2023 13:46:02 -0400 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F998F3 for ; Wed, 27 Sep 2023 10:46:00 -0700 (PDT) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id BFC003200930; Wed, 27 Sep 2023 13:45:59 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Wed, 27 Sep 2023 13:46:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bur.io; h=cc :content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm3; t=1695836759; x= 1695923159; bh=OBPy6RP/cUpyBOQtJlvx2CyC9ypyqFkqScJcquhrODI=; b=V t1TpoeamWKEkJsSSfA0U7oNjXYLO/x8endlrBGe9+t905/8Hm4SmZPUJB3qWI1Rd Qh9ILgRwJ+/JLBOffgbc3FSfrP8b4jE++93qI86lQcMBvGUIoPMAmIXTAR+TYi0h UXWazI+BSX/tbIMbKivCsGlxgNEP8h0E2Gap+8cbAvTm0a92uDKoD15UyR1k4EnZ TqfWhgXbxGzFMrqO5NZV7CCR5ZDTl/SMB1y0KTHgALxGnrT1vcgczTel1l7en9ZM 73f9WQmZA4Fqz27Ym1Fnd63PyOeSGitjtjjXNyxgX4i160yjVXYyOcKr2xj7Md5V p3Tpr0bmwb5xAiWI0+sFg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1695836759; x=1695923159; bh=O BPy6RP/cUpyBOQtJlvx2CyC9ypyqFkqScJcquhrODI=; b=G9HXf5utIwVBb8fDk EIA+pxYota7tsYteQ95mAKxxDSMF9XHFNMtsuhn4TQ2ARliKcasSlbvSPKgjA3hU t+oDj5rydFtO9IFqUgT2greZ9J2XCRfHgAINmNZERxN66v0iLBfjd4ahrBdXPBEW ghIzRCWcxQJYLqIbv1Q5I1iW9Czyw45NTIUV40ZJNWBWp2n/p45XKeWhE83kNjMp BgpMFI8SPK4tLcZqGWKOP0BXMomGagWRZomrt1K1zdBYeQqrkN3KRIwypzJrNeNS D2ciCzOst42JDyjwPaGmsbrMSXvTiTRL1jOSmbkk2Mzxy2eogfXi66OuF2PG78kB yjFFw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvjedrtdeggdehfecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpeeuohhrihhsuceuuhhrkhhovhcuoegsohhrihhssegsuhhrrdhi oheqnecuggftrfgrthhtvghrnhepieeuffeuvdeiueejhfehiefgkeevudejjeejffevvd ehtddufeeihfekgeeuheelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehm rghilhhfrhhomhepsghorhhishessghurhdrihho X-ME-Proxy: Feedback-ID: i083147f8:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 27 Sep 2023 13:45:58 -0400 (EDT) From: Boris Burkov To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v3 3/8] btrfs-progs: simple quotas dump commands Date: Wed, 27 Sep 2023 10:46:44 -0700 Message-ID: X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Add support to btrfs inspect-internal dump-super and dump-tree for the new structures and feature flags introduced by simple quotas Signed-off-by: Boris Burkov Reviewed-by: Josef Bacik --- kernel-shared/print-tree.c | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/kernel-shared/print-tree.c b/kernel-shared/print-tree.c index d7ffeccd1..666664868 100644 --- a/kernel-shared/print-tree.c +++ b/kernel-shared/print-tree.c @@ -509,6 +509,10 @@ void print_extent_item(struct extent_buffer *eb, int slot, int metadata) (unsigned long long)offset, btrfs_shared_data_ref_count(eb, sref)); break; + case BTRFS_EXTENT_OWNER_REF_KEY: + printf("\t\textent owner root %llu\n", + (unsigned long long)offset); + break; default: return; } @@ -661,6 +665,7 @@ void print_key_type(FILE *stream, u64 objectid, u8 type) [BTRFS_EXTENT_DATA_REF_KEY] = "EXTENT_DATA_REF", [BTRFS_SHARED_DATA_REF_KEY] = "SHARED_DATA_REF", [BTRFS_EXTENT_REF_V0_KEY] = "EXTENT_REF_V0", + [BTRFS_EXTENT_OWNER_REF_KEY] = "EXTENT_OWNER_REF", [BTRFS_CSUM_ITEM_KEY] = "CSUM_ITEM", [BTRFS_EXTENT_CSUM_KEY] = "EXTENT_CSUM", [BTRFS_EXTENT_DATA_KEY] = "EXTENT_DATA", @@ -1042,6 +1047,17 @@ static void print_shared_data_ref(struct extent_buffer *eb, int slot) btrfs_shared_data_ref_count(eb, sref)); } +static void print_extent_owner_ref(struct extent_buffer *eb, int slot) +{ + struct btrfs_extent_owner_ref *oref; + u64 root_id; + + oref = btrfs_item_ptr(eb, slot, struct btrfs_extent_owner_ref); + root_id = btrfs_extent_owner_ref_root_id(eb, oref); + + printf("\t\textent owner root %llu\n", root_id); +} + static void print_free_space_info(struct extent_buffer *eb, int slot) { struct btrfs_free_space_info *free_info; @@ -1083,11 +1099,16 @@ static void print_qgroup_status(struct extent_buffer *eb, int slot) memset(flags_str, 0, sizeof(flags_str)); qgroup_flags_to_str(btrfs_qgroup_status_flags(eb, qg_status), flags_str); - printf("\t\tversion %llu generation %llu flags %s scan %llu\n", + printf("\t\tversion %llu generation %llu flags %s scan %llu", (unsigned long long)btrfs_qgroup_status_version(eb, qg_status), (unsigned long long)btrfs_qgroup_status_generation(eb, qg_status), flags_str, (unsigned long long)btrfs_qgroup_status_rescan(eb, qg_status)); + if (btrfs_fs_incompat(eb->fs_info, SIMPLE_QUOTA)) + printf(" enable_gen %llu\n", + (unsigned long long)btrfs_qgroup_status_enable_gen(eb, qg_status)); + else + printf("\n"); } static void print_qgroup_info(struct extent_buffer *eb, int slot) @@ -1407,6 +1428,9 @@ void btrfs_print_leaf(struct extent_buffer *eb, unsigned int mode) case BTRFS_SHARED_DATA_REF_KEY: print_shared_data_ref(eb, i); break; + case BTRFS_EXTENT_OWNER_REF_KEY: + print_extent_owner_ref(eb, i); + break; case BTRFS_EXTENT_REF_V0_KEY: printf("\t\textent ref v0 (deprecated)\n"); break; @@ -1708,6 +1732,7 @@ static struct readable_flag_entry incompat_flags_array[] = { DEF_INCOMPAT_FLAG_ENTRY(RAID1C34), DEF_INCOMPAT_FLAG_ENTRY(ZONED), DEF_INCOMPAT_FLAG_ENTRY(EXTENT_TREE_V2), + DEF_INCOMPAT_FLAG_ENTRY(SIMPLE_QUOTA), }; static const int incompat_flags_num = sizeof(incompat_flags_array) / sizeof(struct readable_flag_entry); From patchwork Wed Sep 27 17:46:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Burkov X-Patchwork-Id: 13401459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89DA7E82CA1 for ; Wed, 27 Sep 2023 17:46:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229502AbjI0RqG (ORCPT ); Wed, 27 Sep 2023 13:46:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49358 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229473AbjI0RqE (ORCPT ); Wed, 27 Sep 2023 13:46:04 -0400 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 251CBA1 for ; Wed, 27 Sep 2023 10:46:03 -0700 (PDT) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 698FB320076F; Wed, 27 Sep 2023 13:46:02 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Wed, 27 Sep 2023 13:46:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bur.io; h=cc :content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm3; t=1695836761; x= 1695923161; bh=6MxL23bwYDa0zlTDrjiGXhMZGayLe0OmXy3Yv4y6HZc=; b=L L5rIKHdwVa8iLpgTkOFLBywlMbCfl6x66iNoIjK3xfxxn22V8VqTe3iQgfOEgYvJ GU8V3BrOGfxo4XHLNBvtg0ZqR9CgHeSV+jbSTxsJLs1slOzLocmU2OO/1PZZvIen IRXANhdOYcXaXW7uLBzs/fB5xWk1NZXUoL/UKO+2pmu4LdLynxXNBVKIpAL58fCn HJTR0R7q3HiS+PqEgA3n8RtN2api2xdl9KoZaomy+/6gTe1Y5lFlqqKg7qdBHPSS tn/DgId0Fn3Jxa1slPLiSQfDtWfc9+4laCpmRLSxyloqYXIRhCV/KTCMct3hQ73U rV2EpELac/2XBLqhEzCxA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1695836761; x=1695923161; bh=6 MxL23bwYDa0zlTDrjiGXhMZGayLe0OmXy3Yv4y6HZc=; b=iozCR57n5port7dnj mmPcf5S13g0AH+HHFF8m5e5qS4PVf3CmKjnJIyEhzmfGZMeVKbqsMHXl8cN524rF Q/9ges+fzMkbEVgdbm8GCF5eyKBfgpOq7sXhhLNAPHUehtyOpCnbAYglHNU+d6vf s/3ZFP0VH6wgAkgkRt40k56oHzwxgY4IhaKpAle1pqSidy7mXXIdQbyEvQM+cBxr yPK2i94lZpVKURIDkOaeYQa0JkVmTVsruo/cavM+OzYRhqWjI0X2pHC/g/bZLNyq O5hkYmO8wvdgahRFzp+B1iQz71aC5hnMU4dkBfSMqgKMM+yzwi8zw+vjaqDx3+Gh LFGIA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvjedrtdeggdehgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpeeuohhrihhsuceuuhhrkhhovhcuoegsohhrihhssegsuhhrrdhi oheqnecuggftrfgrthhtvghrnhepieeuffeuvdeiueejhfehiefgkeevudejjeejffevvd ehtddufeeihfekgeeuheelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehm rghilhhfrhhomhepsghorhhishessghurhdrihho X-ME-Proxy: Feedback-ID: i083147f8:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 27 Sep 2023 13:46:01 -0400 (EDT) From: Boris Burkov To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v3 4/8] btrfs-progs: simple quotas fsck Date: Wed, 27 Sep 2023 10:46:45 -0700 Message-ID: <772bfef2ddac0c967c11297957dd78b09aa516aa.1695836680.git.boris@bur.io> X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Add simple quotas checks to btrfs check. Like the kernel feature, these checks bypass most of the backref walking in the qgroups check. Instead, they enforce the invariant behind the design of simple quotas by scanning the extent tree and determining the owner of each extent: Data: reading the owner ref inline item Metadata: reading the tree block and reading its btrfs_header's owner This gives us the expected count from squotas which we check against the on-disk state of the qgroup items. Signed-off-by: Boris Burkov --- check/main.c | 2 ++ check/qgroup-verify.c | 79 +++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 78 insertions(+), 3 deletions(-) diff --git a/check/main.c b/check/main.c index d387eb25d..c3cfd199e 100644 --- a/check/main.c +++ b/check/main.c @@ -5674,6 +5674,8 @@ static int process_extent_item(struct btrfs_root *root, btrfs_shared_data_ref_count(eb, sref), gen, 0, num_bytes); break; + case BTRFS_EXTENT_OWNER_REF_KEY: + break; default: fprintf(stderr, "corrupt extent record: key [%llu,%u,%llu]\n", diff --git a/check/qgroup-verify.c b/check/qgroup-verify.c index 1a62009b8..c95e6f806 100644 --- a/check/qgroup-verify.c +++ b/check/qgroup-verify.c @@ -85,6 +85,8 @@ static struct counts_tree { unsigned int num_groups; unsigned int rescan_running:1; unsigned int qgroup_inconsist:1; + unsigned int simple:1; + u64 enable_gen; u64 scan_progress; } counts = { .root = RB_ROOT }; @@ -915,14 +917,18 @@ static int add_qgroup_relation(u64 memberid, u64 parentid) return 0; } -static void read_qgroup_status(struct extent_buffer *eb, int slot, - struct counts_tree *counts) +static void read_qgroup_status(struct btrfs_fs_info *info, + struct extent_buffer *eb, + int slot, struct counts_tree *counts) { struct btrfs_qgroup_status_item *status_item; u64 flags; status_item = btrfs_item_ptr(eb, slot, struct btrfs_qgroup_status_item); flags = btrfs_qgroup_status_flags(eb, status_item); + + if (counts->simple == 1) + counts->enable_gen = btrfs_qgroup_status_enable_gen(eb, status_item); /* * Since qgroup_inconsist/rescan_running is just one bit, * assign value directly won't work. @@ -948,6 +954,8 @@ static int load_quota_info(struct btrfs_fs_info *info) int i, nr; int search_relations = 0; + if (btrfs_fs_incompat(info, SIMPLE_QUOTA)) + counts.simple = 1; loop: /* * Do 2 passes, the first allocates group counts and reads status @@ -990,7 +998,7 @@ loop: } if (key.type == BTRFS_QGROUP_STATUS_KEY) { - read_qgroup_status(leaf, i, &counts); + read_qgroup_status(info, leaf, i, &counts); continue; } @@ -1038,6 +1046,51 @@ out: return ret; } +static int simple_quota_account_extent(struct btrfs_fs_info *info, + struct extent_buffer *leaf, + struct btrfs_key *key, + struct btrfs_extent_item *ei, + struct btrfs_extent_inline_ref *iref, + u64 bytenr, u64 num_bytes, int meta_item) +{ + u64 generation; + int type; + u64 root; + struct ulist *roots = ulist_alloc(0); + int ret; + struct extent_buffer *node_eb; + u64 extent_root; + + generation = btrfs_extent_generation(leaf, ei); + if (generation < counts.enable_gen) + return 0; + + type = btrfs_extent_inline_ref_type(leaf, iref); + if (!meta_item) { + if (type == BTRFS_EXTENT_OWNER_REF_KEY) { + struct btrfs_extent_owner_ref *oref = (struct btrfs_extent_owner_ref *)(&iref->offset); + root = btrfs_extent_owner_ref_root_id(leaf, oref); + } else { + return 0; + } + } else { + extent_root = btrfs_root_id(btrfs_extent_root(info, key->objectid)); + node_eb = read_tree_block(info, key->objectid, extent_root, 0, 0, NULL); + if (!extent_buffer_uptodate(node_eb)) + return -EIO; + root = btrfs_header_owner(node_eb); + free_extent_buffer(node_eb); + } + + if (!is_fstree(root)) + return 0; + + ulist_add(roots, root, 0, 0); + ret = account_one_extent(roots, bytenr, num_bytes); + ulist_free(roots); + return ret; +} + static int add_inline_refs(struct btrfs_fs_info *info, struct extent_buffer *ei_leaf, int slot, u64 bytenr, u64 num_bytes, int meta_item) @@ -1045,6 +1098,7 @@ static int add_inline_refs(struct btrfs_fs_info *info, struct btrfs_extent_item *ei; struct btrfs_extent_inline_ref *iref; struct btrfs_extent_data_ref *dref; + struct btrfs_key key; u64 flags, root_obj, offset, parent; u32 item_size = btrfs_item_size(ei_leaf, slot); int type; @@ -1052,6 +1106,7 @@ static int add_inline_refs(struct btrfs_fs_info *info, unsigned long ptr; ei = btrfs_item_ptr(ei_leaf, slot, struct btrfs_extent_item); + btrfs_item_key_to_cpu(ei_leaf, &key, slot); flags = btrfs_extent_flags(ei_leaf, ei); if (flags & BTRFS_EXTENT_FLAG_TREE_BLOCK && !meta_item) { @@ -1062,6 +1117,15 @@ static int add_inline_refs(struct btrfs_fs_info *info, iref = (struct btrfs_extent_inline_ref *)(ei + 1); } + if (counts.simple) { + int ret = simple_quota_account_extent(info, ei_leaf, &key, ei, iref, + bytenr, num_bytes, meta_item); + + if (ret) + error("simple quota account extent error: %d", ret); + return ret; + } + ptr = (unsigned long)iref; end = (unsigned long)ei + item_size; while (ptr < end) { @@ -1083,6 +1147,7 @@ static int add_inline_refs(struct btrfs_fs_info *info, parent = offset; break; default: + error("unexpected iref type %d", type); return 1; } @@ -1445,6 +1510,13 @@ int qgroup_verify_all(struct btrfs_fs_info *info) goto out; } } + /* + * As in the kernel, simple qgroup accounting is done locally per extent, + * so we don't need to resolve backrefs to find which subvol an extent + * is accounted to. + */ + if (counts.simple) + goto check; ret = map_implied_refs(info); if (ret) { @@ -1454,6 +1526,7 @@ int qgroup_verify_all(struct btrfs_fs_info *info) ret = account_all_refs(1, 0); +check: /* * Do the correctness check here, so for callers who don't want * verbose report can skip calling report_qgroups() From patchwork Wed Sep 27 17:46:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Burkov X-Patchwork-Id: 13401460 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FF9CE82CBC for ; Wed, 27 Sep 2023 17:46:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229547AbjI0RqI (ORCPT ); Wed, 27 Sep 2023 13:46:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50034 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229473AbjI0RqH (ORCPT ); Wed, 27 Sep 2023 13:46:07 -0400 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C3FABA1 for ; Wed, 27 Sep 2023 10:46:05 -0700 (PDT) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 0CC573200906; Wed, 27 Sep 2023 13:46:04 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Wed, 27 Sep 2023 13:46:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bur.io; h=cc :content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm3; t=1695836764; x= 1695923164; bh=nfacq4SjVU+70Uf1Je2tz9agFZq5yx7t4DA/p7b/ZNk=; b=V Qs63xAKJTrvLxFosRT7cAJdz7jb9SlxOQETWE+dbTHrbGK2oBQH3QfLtjyh+u+9L n7Q13jiUdwQ8eUjK9fSzd6AzyKeJ3BddjqFaU4oh41365bS+KwV2aWNWFXTWAy1T v9ZXdF/kveaTqA1hDZlbXJsxievIHjVp5ycr04VS10CDoiy6BSMVrX3+1p0OBA0z 6ojVY6IMPARJl4jdDRmg9s2+1K+r9Fn7NfKd4T6iK+7w1fuYTVjsGt5DaySzdUxx nSeO1RjeFPEVMsSRTAYYmJ9NfpAgZKp/iw9Ox96/jVH4NraK3ahleB0Ue7Xo9zff ZG2EnpSbM8uWZzyh1XuJA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1695836764; x=1695923164; bh=n facq4SjVU+70Uf1Je2tz9agFZq5yx7t4DA/p7b/ZNk=; b=eA+pQUUnVCTxzSf+S wsauE4gCACVNZnbyuz2UKyAw0HVMN2uSdhGEalHzE8pqhB4AiXxbxg07TSbkd52Z fZjR0TlSLa17DRaorSmwHVuUNASUUKTTpxRrIcEhY8YPhHcTax9e0SEcr8n7BoaV oLsmERLaygIiRV+00Yof/p/bR7YU7hM+rHGFrArXe4R0O9eppSvZ6L/I+3oYpLdN pQmAhI+0aeXnd1Mi96CA+E+dGHeMRZCC9i2Ys6iKYXzFkd3X8dUxR23HYTc2C8gg VKTn8dE0vTHhdSgqwrFztd+tiTs3sq9tDbUHHzK+Ob/nm5KqKuyVEFrWJxFeE6yI e08Dw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvjedrtdeggdehfecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpeeuohhrihhsuceuuhhrkhhovhcuoegsohhrihhssegsuhhrrdhi oheqnecuggftrfgrthhtvghrnhepieeuffeuvdeiueejhfehiefgkeevudejjeejffevvd ehtddufeeihfekgeeuheelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehm rghilhhfrhhomhepsghorhhishessghurhdrihho X-ME-Proxy: Feedback-ID: i083147f8:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 27 Sep 2023 13:46:04 -0400 (EDT) From: Boris Burkov To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v3 5/8] btrfs-progs: simple quotas mkfs Date: Wed, 27 Sep 2023 10:46:46 -0700 Message-ID: <3b69a29059b051962e556f4ce3aa53e4d00466a2.1695836680.git.boris@bur.io> X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Add the ability to enable simple quotas from mkfs with '-O squota' There is some complication around handling enable gen while still counting the root node of an fs. To handle this, employ a hack of doing a no-op write on the root node to bump its generation up above that of the qgroup enable generation, which results in counting it properly. Signed-off-by: Boris Burkov Reviewed-by: Josef Bacik --- check/qgroup-verify.c | 7 +++-- common/fsfeatures.c | 9 ++++++ mkfs/main.c | 64 +++++++++++++++++++++++++++++++++++++++---- 3 files changed, 72 insertions(+), 8 deletions(-) diff --git a/check/qgroup-verify.c b/check/qgroup-verify.c index c95e6f806..1cf01d2d0 100644 --- a/check/qgroup-verify.c +++ b/check/qgroup-verify.c @@ -1695,6 +1695,8 @@ static int repair_qgroup_status(struct btrfs_fs_info *info, bool silent) struct btrfs_path path; struct btrfs_key key; struct btrfs_qgroup_status_item *status_item; + bool simple = btrfs_fs_incompat(info, SIMPLE_QUOTA); + int flags = BTRFS_QGROUP_STATUS_FLAG_ON; if (!silent) printf("Repair qgroup status item\n"); @@ -1717,8 +1719,9 @@ static int repair_qgroup_status(struct btrfs_fs_info *info, bool silent) status_item = btrfs_item_ptr(path.nodes[0], path.slots[0], struct btrfs_qgroup_status_item); - btrfs_set_qgroup_status_flags(path.nodes[0], status_item, - BTRFS_QGROUP_STATUS_FLAG_ON); + if (simple) + flags |= BTRFS_QGROUP_STATUS_FLAG_SIMPLE_MODE; + btrfs_set_qgroup_status_flags(path.nodes[0], status_item, flags); btrfs_set_qgroup_status_rescan(path.nodes[0], status_item, 0); btrfs_set_qgroup_status_generation(path.nodes[0], status_item, trans->transid); diff --git a/common/fsfeatures.c b/common/fsfeatures.c index 1c993b594..6bf87937b 100644 --- a/common/fsfeatures.c +++ b/common/fsfeatures.c @@ -109,6 +109,15 @@ static const struct btrfs_feature mkfs_features[] = { VERSION_NULL(default), .desc = "quota support (qgroups)" }, + { + .name = "squota", + .incompat_flag = BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA, + .sysfs_name = "squota", + VERSION_TO_STRING2(compat, 6,5), + VERSION_NULL(safe), + VERSION_NULL(default), + .desc = "squota support (simple qgroups)" + }, { .name = "extref", .incompat_flag = BTRFS_FEATURE_INCOMPAT_EXTENDED_IREF, diff --git a/mkfs/main.c b/mkfs/main.c index 1c5d668e1..5bb4d1a21 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -59,6 +59,8 @@ #include "mkfs/common.h" #include "mkfs/rootdir.h" +#include "libbtrfs/ctree.h" + struct mkfs_allocation { u64 data; u64 metadata; @@ -882,6 +884,37 @@ static int insert_qgroup_items(struct btrfs_trans_handle *trans, return ret; } +static int touch_root_subvol(struct btrfs_fs_info *fs_info) +{ + struct btrfs_trans_handle *trans; + struct btrfs_key key = { + .objectid = BTRFS_FIRST_FREE_OBJECTID, + .type = BTRFS_INODE_ITEM_KEY, + .offset = 0, + }; + struct extent_buffer *leaf; + int slot; + struct btrfs_path path; + int ret; + + trans = btrfs_start_transaction(fs_info->fs_root, 1); + btrfs_init_path(&path); + ret = btrfs_search_slot(trans, fs_info->fs_root, &key, &path, 0, 1); + if (ret) + goto fail; + leaf = path.nodes[0]; + slot = path.slots[0]; + btrfs_item_key_to_cpu(leaf, &key, slot); + btrfs_mark_buffer_dirty(leaf); + btrfs_commit_transaction(trans, fs_info->fs_root); + btrfs_release_path(&path); + return 0; +fail: + btrfs_abort_transaction(trans, ret); + btrfs_release_path(&path); + return ret; +} + static int setup_quota_root(struct btrfs_fs_info *fs_info) { struct btrfs_trans_handle *trans; @@ -890,8 +923,11 @@ static int setup_quota_root(struct btrfs_fs_info *fs_info) struct btrfs_path path; struct btrfs_key key; int qgroup_repaired = 0; + bool simple = btrfs_fs_incompat(fs_info, SIMPLE_QUOTA); + int flags; int ret; + /* One to modify tree root, one for quota root */ trans = btrfs_start_transaction(fs_info->tree_root, 2); if (IS_ERR(trans)) { @@ -921,13 +957,19 @@ static int setup_quota_root(struct btrfs_fs_info *fs_info) qsi = btrfs_item_ptr(path.nodes[0], path.slots[0], struct btrfs_qgroup_status_item); - btrfs_set_qgroup_status_generation(path.nodes[0], qsi, 0); + btrfs_set_qgroup_status_generation(path.nodes[0], qsi, trans->transid); btrfs_set_qgroup_status_rescan(path.nodes[0], qsi, 0); + flags = BTRFS_QGROUP_STATUS_FLAG_ON; + if (simple) { + btrfs_set_qgroup_status_enable_gen(path.nodes[0], qsi, trans->transid); + flags |= BTRFS_QGROUP_STATUS_FLAG_SIMPLE_MODE; + } + else { + flags |= BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT; + } - /* Mark current status info inconsistent, and fix it later */ - btrfs_set_qgroup_status_flags(path.nodes[0], qsi, - BTRFS_QGROUP_STATUS_FLAG_ON | - BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT); + btrfs_set_qgroup_status_version(path.nodes[0], qsi, 1); + btrfs_set_qgroup_status_flags(path.nodes[0], qsi, flags); btrfs_release_path(&path); /* Currently mkfs will only create one subvolume */ @@ -944,6 +986,15 @@ static int setup_quota_root(struct btrfs_fs_info *fs_info) return ret; } + /* Hack to count the default subvol metadata by dirtying it */ + if (simple) { + ret = touch_root_subvol(fs_info); + if (ret) { + error("failed to touch root dir for simple quota accounting %d (%m)", ret); + goto fail; + } + } + /* * Qgroup is setup but with wrong info, use qgroup-verify * infrastructure to repair them. (Just acts as offline rescan) @@ -1743,7 +1794,8 @@ raid_groups: } } - if (features.runtime_flags & BTRFS_FEATURE_RUNTIME_QUOTA) { + if (features.runtime_flags & BTRFS_FEATURE_RUNTIME_QUOTA || + features.incompat_flags & BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA) { ret = setup_quota_root(fs_info); if (ret < 0) { error("failed to initialize quota: %d (%m)", ret); From patchwork Wed Sep 27 17:46:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Burkov X-Patchwork-Id: 13401461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A00FE82CBB for ; Wed, 27 Sep 2023 17:46:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229552AbjI0RqL (ORCPT ); Wed, 27 Sep 2023 13:46:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229473AbjI0RqK (ORCPT ); Wed, 27 Sep 2023 13:46:10 -0400 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F156A1 for ; Wed, 27 Sep 2023 10:46:08 -0700 (PDT) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 959923200906; Wed, 27 Sep 2023 13:46:07 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Wed, 27 Sep 2023 13:46:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bur.io; h=cc :content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm3; t=1695836767; x= 1695923167; bh=GCpiqOET8qxMOhlnmiHhXbbYxN2P5Tnq6TO5JNIhLrE=; b=q p2d9ebYb4FeOot8+L6s19+UGbsT7+4BcirjddD4qQfL36rakdk34UXdUcpodEWS9 doqSgRI/oEib3Q9n8QOz+HFr8EZPBdDCe5vrmseZQYMO+HxlikBj3uksIgiH+R7S YpMIKHIq3/gS9VtS9rwhpwiSeaJVglRuzIILQ625sO1HWWhA3lnMK+lq9hQ//gSB qCNaYn+rMhw0o3R/JhDHNHfRdrkxhDqqLmKABLMaRu8aZlcb/S9IOlBSq+lD5udW ptUwZo+n/Y4xB7kXQyEKG/9SPSp1MMLVA/CI42fBz4zp1HfRD+NqxnZttPNF1AYT ZfnkRGGDKyOuwOPQmsy7g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1695836767; x=1695923167; bh=G CpiqOET8qxMOhlnmiHhXbbYxN2P5Tnq6TO5JNIhLrE=; b=MG0j0R9UruiYSpq5m vpx1n9Q3aLkKlEScTNEtFb1SCwYy2a1C1WhlAF897uAotj5fqtrBrcl5k5n7atjA ZviB2nbsD2aFCLvi/AqF8YKV4C4+kLLLX9yO900YB1mcLpmH/NGxlE42XlMkLK+Y tXURofad/deJuIUJAihN5MNBlzRVtMKEaBwAT8ruNU4HuxotIostCXO2bY8PvM/+ HutA53FWQ6jCu3OSfTFvQalytV/h6Koa968FwfVwGt+yRGtE+uL9lTxLeqAuRMkt 8raMedo8vdBRs0LkbCkLS80CilTh5YQBCV4dXqOfk9oN/rx0ZsH0fkpoOx6zVTDz PiDkw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvjedrtdeggdehfecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpeeuohhrihhsuceuuhhrkhhovhcuoegsohhrihhssegsuhhrrdhi oheqnecuggftrfgrthhtvghrnhepieeuffeuvdeiueejhfehiefgkeevudejjeejffevvd ehtddufeeihfekgeeuheelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehm rghilhhfrhhomhepsghorhhishessghurhdrihho X-ME-Proxy: Feedback-ID: i083147f8:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 27 Sep 2023 13:46:06 -0400 (EDT) From: Boris Burkov To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v3 6/8] btrfs-progs: simple quotas btrfstune Date: Wed, 27 Sep 2023 10:46:47 -0700 Message-ID: <4c3729d6ecf4537e77865da4b044453ae3ef2d8a.1695836680.git.boris@bur.io> X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Add the ability to enable simple quotas on an existing file system at rest with btrfstune. This is similar to the functionality in mkfs, except it must also find all the roots for which it must create qgroups. Note that this *does not* retroactively compute usage for existing extents as that is impossible for data. This is consistent with the behavior of the live enable ioctl. Signed-off-by: Boris Burkov Reviewed-by: Josef Bacik --- Makefile | 2 +- tune/main.c | 18 +++++- tune/quota.c | 172 +++++++++++++++++++++++++++++++++++++++++++++++++++ tune/tune.h | 3 + 4 files changed, 193 insertions(+), 2 deletions(-) create mode 100644 tune/quota.c diff --git a/Makefile b/Makefile index 5dfd3c01d..7419475dd 100644 --- a/Makefile +++ b/Makefile @@ -265,7 +265,7 @@ mkfs_objects = mkfs/main.o mkfs/common.o mkfs/rootdir.o image_objects = image/main.o image/sanitize.o image/image-create.o image/common.o \ image/image-restore.o tune_objects = tune/main.o tune/seeding.o tune/change-uuid.o tune/change-metadata-uuid.o \ - tune/convert-bgt.o tune/change-csum.o check/clear-cache.o + tune/convert-bgt.o tune/change-csum.o tune/quota.o check/clear-cache.o all_objects = $(objects) $(cmds_objects) $(libbtrfs_objects) $(convert_objects) \ $(mkfs_objects) $(image_objects) $(tune_objects) $(libbtrfsutil_objects) diff --git a/tune/main.c b/tune/main.c index 63539b991..895e4da03 100644 --- a/tune/main.c +++ b/tune/main.c @@ -103,6 +103,7 @@ static const char * const tune_usage[] = { OPTLINE("-x", "enable skinny metadata extent refs (mkfs: skinny-metadata)"), OPTLINE("-n", "enable no-holes feature (mkfs: no-holes, more efficient sparse file representation)"), OPTLINE("-S <0|1>", "set/unset seeding status of a device"), + OPTLINE("-q", "enable simple quotas on the file system. (mkfs: squota)"), OPTLINE("--convert-to-block-group-tree", "convert filesystem to track block groups in " "the separate block-group-tree instead of extent tree (sets the incompat bit)"), OPTLINE("--convert-from-block-group-tree", @@ -153,6 +154,9 @@ enum btrfstune_group_enum { */ LEGACY, + /* Qgroup options */ + QGROUP, + BTRFSTUNE_NR_GROUPS, }; @@ -189,6 +193,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[]) char *new_fsid_str = NULL; int ret; u64 super_flags = 0; + int quota = 0; int fd = -1; btrfs_config_init(); @@ -211,7 +216,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[]) #endif { NULL, 0, NULL, 0 } }; - int c = getopt_long(argc, argv, "S:rxfuU:nmM:", long_options, NULL); + int c = getopt_long(argc, argv, "S:rxqfuU:nmM:", long_options, NULL); if (c < 0) break; @@ -229,6 +234,10 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[]) super_flags |= BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA; btrfstune_cmd_groups[LEGACY] = true; break; + case 'q': + quota = 1; + btrfstune_cmd_groups[QGROUP] = true; + break; case 'n': super_flags |= BTRFS_FEATURE_INCOMPAT_NO_HOLES; btrfstune_cmd_groups[LEGACY] = true; @@ -499,6 +508,13 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[]) ret = change_uuid(fs_info, new_fsid_str); goto out; } + + if (quota) { + ret = enable_quota(root->fs_info, true); + if (ret) + goto out; + } + out: if (ret < 0) { fs_info->readonly = 1; diff --git a/tune/quota.c b/tune/quota.c new file mode 100644 index 000000000..949cfc409 --- /dev/null +++ b/tune/quota.c @@ -0,0 +1,172 @@ +#include + +#include "common/messages.h" +#include "kernel-shared/ctree.h" +#include "kernel-shared/disk-io.h" +#include "kernel-shared/transaction.h" +#include "kernel-shared/uapi/btrfs_tree.h" +#include "tune/tune.h" + +static int create_qgroup(struct btrfs_fs_info *fs_info, + struct btrfs_trans_handle *trans, + u64 qgroupid) +{ + struct btrfs_path path; + struct btrfs_root *quota_root = fs_info->quota_root; + struct btrfs_key key; + int ret; + + if (qgroupid >> BTRFS_QGROUP_LEVEL_SHIFT) { + error("qgroup level other than 0 is not supported yet"); + return -ENOTTY; + } + + key.objectid = 0; + key.type = BTRFS_QGROUP_INFO_KEY; + key.offset = qgroupid; + + btrfs_init_path(&path); + ret = btrfs_insert_empty_item(trans, quota_root, &path, &key, + sizeof(struct btrfs_qgroup_info_item)); + btrfs_release_path(&path); + if (ret < 0) + return ret; + + key.objectid = 0; + key.type = BTRFS_QGROUP_LIMIT_KEY; + key.offset = qgroupid; + ret = btrfs_insert_empty_item(trans, quota_root, &path, &key, + sizeof(struct btrfs_qgroup_limit_item)); + btrfs_release_path(&path); + + printf("created qgroup for %llu\n", qgroupid); + return ret; +} + +static int create_qgroups(struct btrfs_fs_info *fs_info, + struct btrfs_trans_handle *trans) +{ + struct btrfs_key key = { + .objectid = 0, + .type = BTRFS_ROOT_REF_KEY, + .offset = 0, + }; + struct btrfs_path path; + struct extent_buffer *leaf; + int slot; + struct btrfs_root *tree_root = fs_info->tree_root; + int ret; + + + ret = create_qgroup(fs_info, trans, BTRFS_FS_TREE_OBJECTID); + if (ret) + goto out; + + btrfs_init_path(&path); + ret = btrfs_search_slot_for_read(tree_root, &key, &path, 1, 0); + if (ret) + goto out; + + while (1) { + slot = path.slots[0]; + leaf = path.nodes[0]; + btrfs_item_key_to_cpu(leaf, &key, slot); + if (key.type == BTRFS_ROOT_REF_KEY) { + ret = create_qgroup(fs_info, trans, key.offset); + if (ret) + goto out; + } + ret = btrfs_next_item(tree_root, &path); + if (ret < 0) { + error("failed to advance to next item"); + goto out; + } + if (ret) + break; + } + +out: + btrfs_release_path(&path); + return ret; +} + +int enable_quota(struct btrfs_fs_info *fs_info, bool simple) +{ + struct btrfs_super_block *sb = fs_info->super_copy; + struct btrfs_trans_handle *trans; + int super_flags = btrfs_super_incompat_flags(sb); + struct btrfs_qgroup_status_item *qsi; + struct btrfs_root *quota_root; + struct btrfs_path path; + struct btrfs_key key; + int flags; + int ret; + + trans = btrfs_start_transaction(fs_info->tree_root, 2); + if (IS_ERR(trans)) { + ret = PTR_ERR(trans); + errno = -ret; + error_msg(ERROR_MSG_START_TRANS, "%m"); + return ret; + } + + ret = btrfs_create_root(trans, fs_info, BTRFS_QUOTA_TREE_OBJECTID); + if (ret < 0) { + error("failed to create quota root: %d (%m)", ret); + goto fail; + } + quota_root = fs_info->quota_root; + + /* Create the qgroup status item */ + key.objectid = 0; + key.type = BTRFS_QGROUP_STATUS_KEY; + key.offset = 0; + + btrfs_init_path(&path); + ret = btrfs_insert_empty_item(trans, quota_root, &path, &key, + sizeof(*qsi)); + if (ret < 0) { + error("failed to insert qgroup status item: %d (%m)", ret); + goto fail; + } + + qsi = btrfs_item_ptr(path.nodes[0], path.slots[0], + struct btrfs_qgroup_status_item); + btrfs_set_qgroup_status_generation(path.nodes[0], qsi, trans->transid); + btrfs_set_qgroup_status_rescan(path.nodes[0], qsi, 0); + flags = BTRFS_QGROUP_STATUS_FLAG_ON; + if (simple) { + btrfs_set_qgroup_status_enable_gen(path.nodes[0], qsi, trans->transid); + flags |= BTRFS_QGROUP_STATUS_FLAG_SIMPLE_MODE; + } else { + flags |= BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT; + } + + btrfs_set_qgroup_status_version(path.nodes[0], qsi, 1); + btrfs_set_qgroup_status_flags(path.nodes[0], qsi, flags); + btrfs_release_path(&path); + + /* Create the qgroup items */ + ret = create_qgroups(fs_info, trans); + if (ret < 0) { + error("failed to create qgroup items for subvols %d (%m)", ret); + goto fail; + } + + /* Set squota incompat flag */ + if (simple) { + super_flags |= BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA; + btrfs_set_super_incompat_flags(sb, super_flags); + } + + ret = btrfs_commit_transaction(trans, fs_info->tree_root); + if (ret < 0) { + errno = -ret; + error_msg(ERROR_MSG_COMMIT_TRANS, "%m"); + return ret; + } + return ret; +fail: + btrfs_abort_transaction(trans, ret); + return ret; +} diff --git a/tune/tune.h b/tune/tune.h index 0ef249d89..cbf33b2e7 100644 --- a/tune/tune.h +++ b/tune/tune.h @@ -33,4 +33,7 @@ int convert_to_bg_tree(struct btrfs_fs_info *fs_info); int convert_to_extent_tree(struct btrfs_fs_info *fs_info); int btrfs_change_csum_type(struct btrfs_fs_info *fs_info, u16 new_csum_type); + +int enable_quota(struct btrfs_fs_info *fs_info, bool simple); + #endif From patchwork Wed Sep 27 17:46:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Burkov X-Patchwork-Id: 13401462 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EC6FE82CA1 for ; Wed, 27 Sep 2023 17:46:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229565AbjI0RqN (ORCPT ); Wed, 27 Sep 2023 13:46:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50064 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229473AbjI0RqM (ORCPT ); Wed, 27 Sep 2023 13:46:12 -0400 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 07A831A7 for ; Wed, 27 Sep 2023 10:46:11 -0700 (PDT) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 4A1453200A31; Wed, 27 Sep 2023 13:46:10 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Wed, 27 Sep 2023 13:46:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bur.io; h=cc :content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm3; t=1695836769; x= 1695923169; bh=EAJzeNrebYA4fz8wK/r/eveCHdguJ+mKDlTqNvzszqw=; b=B iE/HyrvSk7rdOxRt3rjM4qV3ixRoFqAn0LX6A0LGsb1IWpFRgP2rVMEbcHgAmkT1 5NfxVKzSVkfYP481jDbJdXGZe1RJmeK3bxpfUkAFoL+3f6CDK5OvqvbYJz3jvE6c yMzIgAfDC1/Tk1QhqEGBANFuY9W/tF6G71Qnt3mCuGbpZsKgFyevzHxibA4oL85B t45ara6B4174DpIp4LDyfY28tEqvx2ae44SsIaVc1mRAQL8+ecGFThg89RfITrfO kbj5HGZQKCPHdTxjXvVktJHExnysLELMDbhwHERvplWgrMebvqgIuiPDwZwBlYiO 5SQ0OoypdXYcUFipR7h/A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1695836769; x=1695923169; bh=E AJzeNrebYA4fz8wK/r/eveCHdguJ+mKDlTqNvzszqw=; b=ZGjW0WjvoReljf/Qf +FJIHav0meCqMJMTMqzmBTvnBZxmmquwWFb0vZmr3jPsFmtsHVvNG0eQas9FgRBU Aj4QAAzmdmVvaP03L8aydXTacq9vC8gNTH0e75Kfns/KdhFJUwXAROZde6bnYicW nqR4UhbrMNxaxekpBWcMela1PfZ5KJLsuc/YKTz66oxSgYKOV865wkAIlfuyaUif h1i3G4KxyzVPx7OrVeLSANIKWZj/KqvSt/lhA1fPGLHdmihk6AlUrUHe0VyeeXPV /zUW+LoL8qD7ZR5HFKPWdoppV9uaUPF4h5m8Tlf/P9KXPrdgIS3ZbjDWe2ZdctGa nhh9A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvjedrtdeggdehgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpeeuohhrihhsuceuuhhrkhhovhcuoegsohhrihhssegsuhhrrdhi oheqnecuggftrfgrthhtvghrnhepieeuffeuvdeiueejhfehiefgkeevudejjeejffevvd ehtddufeeihfekgeeuheelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehm rghilhhfrhhomhepsghorhhishessghurhdrihho X-ME-Proxy: Feedback-ID: i083147f8:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 27 Sep 2023 13:46:09 -0400 (EDT) From: Boris Burkov To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v3 7/8] btrfs-progs: simple quotas enable cmd Date: Wed, 27 Sep 2023 10:46:48 -0700 Message-ID: X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Add a --simple flag to btrfs quota enable. If set, this enables simple quotas instead of full qgroups by using the new ioctl command value. Signed-off-by: Boris Burkov --- cmds/quota.c | 39 ++++++++++++++++++++++++++++++-------- kernel-shared/uapi/btrfs.h | 2 ++ 2 files changed, 33 insertions(+), 8 deletions(-) diff --git a/cmds/quota.c b/cmds/quota.c index cd874f9ed..a7d032a03 100644 --- a/cmds/quota.c +++ b/cmds/quota.c @@ -34,17 +34,13 @@ static const char * const quota_cmd_group_usage[] = { NULL }; -static int quota_ctl(int cmd, int argc, char **argv) +static int quota_ctl(int cmd, char *path) { int ret = 0; int fd; - char *path = argv[1]; struct btrfs_ioctl_quota_ctl_args args; DIR *dirstream = NULL; - if (check_argc_exact(argc, 2)) - return -1; - memset(&args, 0, sizeof(args)); args.cmd = cmd; @@ -67,16 +63,40 @@ static const char * const cmd_quota_enable_usage[] = { "Any data already present on the filesystem will not count towards", "the space usage numbers. It is recommended to enable quota for a", "filesystem before writing any data to it.", + "", + "-s|--simple simple qgroups account ownership by extent lifetime rather than backref walks", NULL }; static int cmd_quota_enable(const struct cmd_struct *cmd, int argc, char **argv) { int ret; + int ctl_cmd = BTRFS_QUOTA_CTL_ENABLE; - clean_args_no_options(cmd, argc, argv); + optind = 0; + while (1) { + static const struct option long_options[] = { + {"simple", no_argument, NULL, 's'}, + {NULL, 0, NULL, 0} + }; + int c; - ret = quota_ctl(BTRFS_QUOTA_CTL_ENABLE, argc, argv); + c = getopt_long(argc, argv, "s", long_options, NULL); + if (c < 0) + break; + + switch (c) { + case 's': + ctl_cmd = BTRFS_QUOTA_CTL_ENABLE_SIMPLE_QUOTA; + break; + default: + usage_unknown_option(cmd, argv); + } + } + if (check_argc_exact(argc - optind, 1)) + return -1; + + ret = quota_ctl(ctl_cmd, argv[optind]); if (ret < 0) usage(cmd, 1); @@ -97,7 +117,10 @@ static int cmd_quota_disable(const struct cmd_struct *cmd, clean_args_no_options(cmd, argc, argv); - ret = quota_ctl(BTRFS_QUOTA_CTL_DISABLE, argc, argv); + if (check_argc_exact(argc, 2)) + return -1; + + ret = quota_ctl(BTRFS_QUOTA_CTL_DISABLE, argv[1]); if (ret < 0) usage(cmd, 1); diff --git a/kernel-shared/uapi/btrfs.h b/kernel-shared/uapi/btrfs.h index 7e0078a5d..11ffd54d4 100644 --- a/kernel-shared/uapi/btrfs.h +++ b/kernel-shared/uapi/btrfs.h @@ -786,9 +786,11 @@ struct btrfs_ioctl_get_dev_stats { }; _static_assert(sizeof(struct btrfs_ioctl_get_dev_stats) == 1032); +/* cmd values */ #define BTRFS_QUOTA_CTL_ENABLE 1 #define BTRFS_QUOTA_CTL_DISABLE 2 #define BTRFS_QUOTA_CTL_RESCAN__NOTUSED 3 +#define BTRFS_QUOTA_CTL_ENABLE_SIMPLE_QUOTA 4 struct btrfs_ioctl_quota_ctl_args { __u64 cmd; __u64 status; From patchwork Wed Sep 27 17:46:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Burkov X-Patchwork-Id: 13401463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBE9FE82CBB for ; Wed, 27 Sep 2023 17:46:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229570AbjI0RqP (ORCPT ); Wed, 27 Sep 2023 13:46:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50072 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229473AbjI0RqO (ORCPT ); Wed, 27 Sep 2023 13:46:14 -0400 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A7B81E5 for ; Wed, 27 Sep 2023 10:46:13 -0700 (PDT) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id EADD83200906; Wed, 27 Sep 2023 13:46:12 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Wed, 27 Sep 2023 13:46:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bur.io; h=cc :content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm3; t=1695836772; x= 1695923172; bh=juJf4adaoKEs10sartKuic/L0xAj1EhjuXtIUizOzC0=; b=H WVqo1bc2zEGUn1Fpu9iSuokuTZr3+y54QOZt0SjAtAJR4+rklGYgO5TGrjdcaFN5 1KqkiBBFIKYZqpG+ENr0guRjTwriXxlA0TbZj2KesalizFMQQWgX2hQI2URxD0ko XWiKKDJcafMWHTSoE2JTDTyRh9HLo1gB9FU/jRNBDDVWCjljEhvh7/M4AF7BbMd7 OTUUp1qsvF2UAj6yjuBio/yP6zJuhqAkOb5hPnwFB3cpFungVleOQtWxZ+OXXcOu +/YjjFaOiTi5wi2EcmCxrDK+Sl4FtYdOdXvYUQ2j3L9SJxxKE3puESEjUoNR4bri zPQabRk0huWzUEHgVRTxA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1695836772; x=1695923172; bh=j uJf4adaoKEs10sartKuic/L0xAj1EhjuXtIUizOzC0=; b=gZdc8fXaZ3l/mLd0/ WS77Zp5PYmh/8T2jDR9es0fxeLLeqPo1XHKpTQI2V8r/sRytLHX7cxkVj7/tszWK NhY5dbwCiRIyU6d5p3e7W2xM9dUyr5w3ZEcFtUhh8Ac4gsYRx7FlQjVvQaE6MjoC Fbf1YGVE5QW3LZiYNPRRlaUQivr6z/XnlqVmNJHUY7aPk9+CT4GPnFS4aXud8SXY 3SmHDUm8U50ybMzCvgkS5oQyDLvW0QkiQfqh9ZEj/U82y4T91iGZCM8boBSoSZ/A 4EK8N6d/727qqrmkLCUTk/G0qsKJam6OTz4flbA5Rnf3HoyDkLPGhuLM/5ffOl0v ljSpw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvjedrtdeggdehgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpeeuohhrihhsuceuuhhrkhhovhcuoegsohhrihhssegsuhhrrdhi oheqnecuggftrfgrthhtvghrnhepieeuffeuvdeiueejhfehiefgkeevudejjeejffevvd ehtddufeeihfekgeeuheelnecuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehm rghilhhfrhhomhepsghorhhishessghurhdrihho X-ME-Proxy: Feedback-ID: i083147f8:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 27 Sep 2023 13:46:11 -0400 (EDT) From: Boris Burkov To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v3 8/8] btrfs-progs: tree-checker: handle owner ref items Date: Wed, 27 Sep 2023 10:46:49 -0700 Message-ID: X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Add the new OWNER_REF inline items to the tree-checker extent item checking code. We could somehow validate the root id for being a valid fstree id, but just skipping it seems fine as well. Signed-off-by: Boris Burkov Reviewed-by: Josef Bacik --- kernel-shared/tree-checker.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel-shared/tree-checker.c b/kernel-shared/tree-checker.c index 107975891..2f834cf33 100644 --- a/kernel-shared/tree-checker.c +++ b/kernel-shared/tree-checker.c @@ -1477,6 +1477,8 @@ static int check_extent_item(struct extent_buffer *leaf, } inline_refs += btrfs_shared_data_ref_count(leaf, sref); break; + case BTRFS_EXTENT_OWNER_REF_KEY: + break; default: extent_err(leaf, slot, "unknown inline ref type: %u", inline_type);