From patchwork Mon Feb 26 09:49:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pankaj Raghav (Samsung)" X-Patchwork-Id: 13571760 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6A65C48BF6 for ; Mon, 26 Feb 2024 09:49:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2CFBD94001C; Mon, 26 Feb 2024 04:49:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 233EB940014; Mon, 26 Feb 2024 04:49:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0D23D94001C; Mon, 26 Feb 2024 04:49:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id EEF6D940014 for ; Mon, 26 Feb 2024 04:49:55 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id BCB1814071A for ; Mon, 26 Feb 2024 09:49:55 +0000 (UTC) X-FDA: 81833483550.30.D6CEB42 Received: from mout-p-101.mailbox.org (mout-p-101.mailbox.org [80.241.56.151]) by imf18.hostedemail.com (Postfix) with ESMTP id D4A1C1C0005 for ; Mon, 26 Feb 2024 09:49:53 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=pankajraghav.com header.s=MBO0001 header.b=eHLA+ySb; dmarc=none; spf=pass (imf18.hostedemail.com: domain of me@pankajraghav.com designates 80.241.56.151 as permitted sender) smtp.mailfrom=me@pankajraghav.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708940994; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1cELEozXOYCd1Af2ZXfddDuv3Odm1CWuSBqyYDa15DQ=; b=vMF+YwiaTkO0YD5cFdhmEk0zhmt+i6VrgltlhMosxF9pgufekBGYkzdbFm41mnMSxxI2S+ Aoh5PInQS6feB7GQgPIoQOAv/QU+6MBHs4l0oBOC6vpCINvbx1va3hvdj34HQlLTxjdTzM TBUCe75C/kyjDall52P9d02ymPOlDD0= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=pankajraghav.com header.s=MBO0001 header.b=eHLA+ySb; dmarc=none; spf=pass (imf18.hostedemail.com: domain of me@pankajraghav.com designates 80.241.56.151 as permitted sender) smtp.mailfrom=me@pankajraghav.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708940994; a=rsa-sha256; cv=none; b=NtgpEv+Nv6khnDhD+QuddrP7NbP6K1GQm8gx1TMWiFV4RTy6iwv7CSxyH9KF19LE6+1Rtp sBI6H/T/Hmjo+s6tlwE28xpiXmEtWvKkxHEF/cgho3yOw6MzXgayZCBo0XtWUjLnuz3tkP nQL/CIRP6gB+Fk0xkFgsJ8p3U0aiNw4= Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-101.mailbox.org (Postfix) with ESMTPS id 4Tjwnp4YR9z9sT0; Mon, 26 Feb 2024 10:49:50 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pankajraghav.com; s=MBO0001; t=1708940990; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1cELEozXOYCd1Af2ZXfddDuv3Odm1CWuSBqyYDa15DQ=; b=eHLA+ySb1gvVydfMakrzGixCW9UqE/O4JF8zTriGRt4EDHU0TwUfMWELwyC5HYIs9xEiwj PSpYl8DwDjWmjrhrpuSj6HhJyy76xEQLrXC5Rsk0BbNYX+RmkQI9K44Olivj0jne+3sQac 4nuD6fjSNjzNBIVfAOfRRhfuF9BFJzUIfwivK9VD2XPDve+6gVvsj+1n1e4WQHZZQs1W1l COmR49AyYJbAs9aj9ATG4yeZ2R7yePNoEOh43ZOKUc+W6jmytZn1KldvUs2mLDOOS/9DzP DTgbs5SMLFY/quRvVtUH8K1aeNtyJEUaGgmz646rVVQ6nQfrat517xMfy1yhAQ== From: "Pankaj Raghav (Samsung)" To: linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, david@fromorbit.com, chandan.babu@oracle.com, akpm@linux-foundation.org, mcgrof@kernel.org, ziy@nvidia.com, hare@suse.de, djwong@kernel.org, gost.dev@samsung.com, linux-mm@kvack.org, willy@infradead.org, Pankaj Raghav Subject: [PATCH 02/13] fs: Allow fine-grained control of folio sizes Date: Mon, 26 Feb 2024 10:49:25 +0100 Message-ID: <20240226094936.2677493-3-kernel@pankajraghav.com> In-Reply-To: <20240226094936.2677493-1-kernel@pankajraghav.com> References: <20240226094936.2677493-1-kernel@pankajraghav.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: xtarpjyqocy8phtcfkbgm3anjt7etras X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: D4A1C1C0005 X-HE-Tag: 1708940993-552269 X-HE-Meta: U2FsdGVkX18FNQLH0dTvHH9DkiZEygs6aUBAiCQxCVdyrzLwzOAspx5AyGDiZvKBRgbSViaRMsJrmI9f+cZvxIibPc3qILK8xMmuE9jtql9w3oI7GOel95gVz6eWF2jzNbwsZSKqzpT0FVpWtz7mO1Cp/ytd/59COAdHZCfkzJ1aSYt4B2vci/6fEd7qkX/wPmLUbOgGEl7aFI4GyQBpptT5JV2WjMirDLsonPPIxwxYS7bPsJuFDTVvq8hY7qJpQQ1nXT1rDHgmyeF8MSA5n56dxfU6i0SU5q6sPLF4JZyaF0LD46FH18FgFGxYlq6y5lCM4xw265w4kDw6WTJ7nK7lBAJvbkV1BJQxXNElxP7HT+6q0/HZOIP347r+3I8eWaXfzSFww7MafLzkH3Kv32+xNQb5Q5gzUrK2H/UMgNIKZIPRJ4N2EUMAYW4egYuW9alVRbfTunVTF5HJ0id9SPBmjyDn+VsDaaXrGEFiL00r2to0WBkEVpdYdNN2pypOGoMi0pM1D8OxLKw3CedGuIXsMia5x+xZBZR6667bXL/kEg6qb5+a5pjj0bb2ZyHzhkqv7/6qwrpKP/JRQ5NHGu51tnJrYBQKaTyHEJKAvFidvBD1B30FskaOixMrIsoWSgV1wduQ9x3L0Kstk5131oM2PTr4nrKjG0194+7jGpJg70qPcYS3Oml1RmeMlrTuUfAgrDTtg+wSBCEzYZIDa6UWrDI6AZ6SWlPoSXoQv8NtjdGnlHmHMkibx2QlNHGr7NQVl4+rDb/glwKfucdhC7wlu6f3p3NynklNht28yBqxJbYbBVrUt8RVA7UVxDAtxRoEXKEJn0ZYWQWKLfCW5Oo1XZxz3sduZF8KIhfNDok/tTNPXt+tcFRc5RmgejI4dn5iifWwjGUrDXQawxNOFqGtq9oowPv5LzSQVA3sU8px41nG4uPn49IfzUn0LL3lS7EpbzyNJl/iVHlqj5P MwfztLhT sdNglmIykafavMgF4QM691FJdj5yY4GYkExRhEMfS3/Gh/LEPU0q2Xhm8CgEQL9Y+3x5XOdlOnOsdrAaWsqUaVVPxCmD5OqVpbiAJIUqmGLU94dVoJZ9a+9K92eVEfIPzuM1BjJ9U/J4dkSx9N3Nlx/Z4RiniP2xaY4CcQihCj/GZt8G2l70/kRDL2FH1b9CwtqnyN8buexHGQQbIgSVbpO7caXF0eIRMcAQNBtnZLgt/AREzT84Hnue7NguppNRRp/j2OwjAsWwRXqvLubwbsE8jAm3SeQVsLGT5XOnfYA1nA0NX+Pzp566vXkv5BxEwq0HDSBWG/mD5c2xUBApLidQrlAFveuL3dVRZix8NoZDxJvVGA/HUGHqnT28VQSxlhBGk6Mr6ESuySn+4Zpg35Pf9LsVjwySuuaMe6lwkAVnGLaxrLv1XkstiLig3gVMVjiCgkMSsubwXVVY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Matthew Wilcox (Oracle)" Some filesystems want to be able to ensure that folios that are added to the page cache are at least a certain size. Add mapping_set_folio_min_order() to allow this level of control. Signed-off-by: Matthew Wilcox (Oracle) Co-developed-by: Pankaj Raghav Signed-off-by: Pankaj Raghav Signed-off-by: Luis Chamberlain --- include/linux/pagemap.h | 100 ++++++++++++++++++++++++++++++++-------- 1 file changed, 80 insertions(+), 20 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 2df35e65557d..fc8eb9c94e9c 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -202,13 +202,18 @@ enum mapping_flags { AS_EXITING = 4, /* final truncate in progress */ /* writeback related tags are not used */ AS_NO_WRITEBACK_TAGS = 5, - AS_LARGE_FOLIO_SUPPORT = 6, - AS_RELEASE_ALWAYS, /* Call ->release_folio(), even if no private data */ - AS_STABLE_WRITES, /* must wait for writeback before modifying + AS_RELEASE_ALWAYS = 6, /* Call ->release_folio(), even if no private data */ + AS_STABLE_WRITES = 7, /* must wait for writeback before modifying folio contents */ - AS_UNMOVABLE, /* The mapping cannot be moved, ever */ + AS_FOLIO_ORDER_MIN = 8, + AS_FOLIO_ORDER_MAX = 13, /* Bit 8-17 are used for FOLIO_ORDER */ + AS_UNMOVABLE = 18, /* The mapping cannot be moved, ever */ }; +#define AS_FOLIO_ORDER_MIN_MASK 0x00001f00 +#define AS_FOLIO_ORDER_MAX_MASK 0x0003e000 +#define AS_FOLIO_ORDER_MASK (AS_FOLIO_ORDER_MIN_MASK | AS_FOLIO_ORDER_MAX_MASK) + /** * mapping_set_error - record a writeback error in the address_space * @mapping: the mapping in which an error should be set @@ -344,9 +349,47 @@ static inline void mapping_set_gfp_mask(struct address_space *m, gfp_t mask) m->gfp_mask = mask; } +/* + * There are some parts of the kernel which assume that PMD entries + * are exactly HPAGE_PMD_ORDER. Those should be fixed, but until then, + * limit the maximum allocation order to PMD size. I'm not aware of any + * assumptions about maximum order if THP are disabled, but 8 seems like + * a good order (that's 1MB if you're using 4kB pages) + */ +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#define MAX_PAGECACHE_ORDER HPAGE_PMD_ORDER +#else +#define MAX_PAGECACHE_ORDER 8 +#endif + +/* + * mapping_set_folio_min_order() - Set the minimum folio order + * @mapping: The address_space. + * @min: Minimum folio order (between 0-MAX_PAGECACHE_ORDER inclusive). + * + * The filesystem should call this function in its inode constructor to + * indicate which base size of folio the VFS can use to cache the contents + * of the file. This should only be used if the filesystem needs special + * handling of folio sizes (ie there is something the core cannot know). + * Do not tune it based on, eg, i_size. + * + * Context: This should not be called while the inode is active as it + * is non-atomic. + */ +static inline void mapping_set_folio_min_order(struct address_space *mapping, + unsigned int min) +{ + if (min > MAX_PAGECACHE_ORDER) + min = MAX_PAGECACHE_ORDER; + + mapping->flags = (mapping->flags & ~AS_FOLIO_ORDER_MASK) | + (min << AS_FOLIO_ORDER_MIN) | + (MAX_PAGECACHE_ORDER << AS_FOLIO_ORDER_MAX); +} + /** * mapping_set_large_folios() - Indicate the file supports large folios. - * @mapping: The file. + * @mapping: The address_space. * * The filesystem should call this function in its inode constructor to * indicate that the VFS can use large folios to cache the contents of @@ -357,7 +400,37 @@ static inline void mapping_set_gfp_mask(struct address_space *m, gfp_t mask) */ static inline void mapping_set_large_folios(struct address_space *mapping) { - __set_bit(AS_LARGE_FOLIO_SUPPORT, &mapping->flags); + mapping_set_folio_min_order(mapping, 0); +} + +static inline unsigned int mapping_max_folio_order(struct address_space *mapping) +{ + return (mapping->flags & AS_FOLIO_ORDER_MAX_MASK) >> AS_FOLIO_ORDER_MAX; +} + +static inline unsigned int mapping_min_folio_order(struct address_space *mapping) +{ + return (mapping->flags & AS_FOLIO_ORDER_MIN_MASK) >> AS_FOLIO_ORDER_MIN; +} + +static inline unsigned long mapping_min_folio_nrpages(struct address_space *mapping) +{ + return 1UL << mapping_min_folio_order(mapping); +} + +/** + * mapping_align_start_index() - Align starting index based on the min + * folio order of the page cache. + * @mapping: The address_space. + * + * Ensure the index used is aligned to the minimum folio order when adding + * new folios to the page cache by rounding down to the nearest minimum + * folio number of pages. + */ +static inline pgoff_t mapping_align_start_index(struct address_space *mapping, + pgoff_t index) +{ + return round_down(index, mapping_min_folio_nrpages(mapping)); } /* @@ -367,7 +440,7 @@ static inline void mapping_set_large_folios(struct address_space *mapping) static inline bool mapping_large_folio_support(struct address_space *mapping) { return IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && - test_bit(AS_LARGE_FOLIO_SUPPORT, &mapping->flags); + (mapping_max_folio_order(mapping) > 0); } static inline int filemap_nr_thps(struct address_space *mapping) @@ -528,19 +601,6 @@ static inline void *detach_page_private(struct page *page) return folio_detach_private(page_folio(page)); } -/* - * There are some parts of the kernel which assume that PMD entries - * are exactly HPAGE_PMD_ORDER. Those should be fixed, but until then, - * limit the maximum allocation order to PMD size. I'm not aware of any - * assumptions about maximum order if THP are disabled, but 8 seems like - * a good order (that's 1MB if you're using 4kB pages) - */ -#ifdef CONFIG_TRANSPARENT_HUGEPAGE -#define MAX_PAGECACHE_ORDER HPAGE_PMD_ORDER -#else -#define MAX_PAGECACHE_ORDER 8 -#endif - #ifdef CONFIG_NUMA struct folio *filemap_alloc_folio(gfp_t gfp, unsigned int order); #else