From patchwork Fri Feb 21 22:38:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 13986409 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6599BC021B5 for ; Fri, 21 Feb 2025 22:38:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E68A6280017; Fri, 21 Feb 2025 17:38:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9969428001D; Fri, 21 Feb 2025 17:38:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 42BB228001C; Fri, 21 Feb 2025 17:38:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B7BC4280019 for ; Fri, 21 Feb 2025 17:38:33 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 74DEE1208E3 for ; Fri, 21 Feb 2025 22:38:33 +0000 (UTC) X-FDA: 83145417306.16.EEAC85D Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf15.hostedemail.com (Postfix) with ESMTP id F1F45A0004 for ; Fri, 21 Feb 2025 22:38:30 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=T3tx0wBJ; spf=none (imf15.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=quarantine) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740177511; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fpTh1KbPClQQAibvuLbxjJCGXlYIrXiRedMs+nLQf5A=; b=51Q2M4TMSWWrqL5Vcc+X3pwUC7xy+9TKSPbnlTo71ymLUHumD2d3DhGl2WBI5LCu1s5fEL mLHqrKrl/qu3piupVsdtEzF8SdVx3kgmfK/eH7l9+ntDigi+n7CCGA8FYVP8tVxlFdGZzU 9jxgMB56wpxWwc8ye2LmYA0j/YW4edI= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=T3tx0wBJ; spf=none (imf15.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=quarantine) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740177511; a=rsa-sha256; cv=none; b=BXr/6xcBruRIMihojTJMeYSlf8tX9ftJ2GPNwwxZD9xq3zlmZvz/Y7f6K5l962O1Q/uQPm uo64WGXgfL5IeN+pyhe5ZkquWK3kTZr2Q/l5iBhEsRQ/v/vwrhFS4hhjqnpAdpKb2Gl2Bi XPtSga7ludthB0wHfQybVgJr6en+OzU= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=fpTh1KbPClQQAibvuLbxjJCGXlYIrXiRedMs+nLQf5A=; b=T3tx0wBJhUa4LW3rSL9hlLLjCm /nfyI88jHLYZ/es2eDKg6NggJ3wwhw7XRGc9qVH/+wHe3uGIMtnKEORNzbYCn/KloX86/WmPANPs7 7CeCqTCrLZgF0FO242DBCTVmLUCHi3DOrJhys6RXzmVCZmf+rEuujTDpTM5m58KVPJnnpbo6LE/4a Qe3Y7/gGuc3cdgPUzM5kfvJjrBoYZp7pdh+sLmLUqr99v01fYrGwUDVyGYkDmPhIBjj0UVmnNqksP k0wOoQC3TfvK+cpGqKrB7zkGYtnNc3C6jcBC20z2UscC1nM0zc6MAjDRzH/oiBZ034AeNwaC5coGX CDikMjWw==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tlbf7-000000073D2-3GTV; Fri, 21 Feb 2025 22:38:25 +0000 From: Luis Chamberlain To: brauner@kernel.org, akpm@linux-foundation.org, hare@suse.de, willy@infradead.org, dave@stgolabs.net, david@fromorbit.com, djwong@kernel.org, kbusch@kernel.org Cc: john.g.garry@oracle.com, hch@lst.de, ritesh.list@gmail.com, linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, gost.dev@samsung.com, p.raghav@samsung.com, da.gomez@samsung.com, kernel@pankajraghav.com, mcgrof@kernel.org Subject: [PATCH v3 2/8] fs/buffer: remove batching from async read Date: Fri, 21 Feb 2025 14:38:17 -0800 Message-ID: <20250221223823.1680616-3-mcgrof@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250221223823.1680616-1-mcgrof@kernel.org> References: <20250221223823.1680616-1-mcgrof@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: F1F45A0004 X-Stat-Signature: d1nqym7teupyo14mgx3iwkbf8fjy4gu8 X-Rspamd-Pre-Result: action=add header; module=dmarc; Action set by DMARC X-Rspam: Yes X-HE-Tag: 1740177510-990448 X-HE-Meta: U2FsdGVkX19Ant6K0JwtlO+g41KZvO+NCKKez58+2xuAY7TWdrHRGY51EzGM0Pok9M0Q+7HWgchoySLd5RjhHPIzrJf9yglP/BsS3R7w68uaYfU15oWVQ000X3gPGvReCSeKCxbkQwe6LqTSb5j+U0k/QxXG7mJx9f3HdA2TzmOo9usQ9Du9pRkHkWyc0EhHgy/5E9pXISr8ke16+d3Y8lxlY5ye3zG9t+dk37WHKc9u2aR8Nc3Gebo+dXfrqSYtqccA8plmOkrJVqA4Q0OsTIB2GA5TZE6KzIoHm6q5iW6ycQUW6Pn9fKtSaAGzJ6TktWCXVvy5+M5po8hd6jFoKNpQDNKpIKjvecslHvupgcmqCEuesvyEHp7VZLSGmoCi49+K3IgNzhMpNnwz5qrXHIWcbrR0MsVWNXjwz/ozckse3Fxo6qnwQgFAnYA5pk5kaFy+eHQoTL2fx6Xu0oIVpLMAt6VrfZoZxORLzVpYTlV4Vq/GqtHrw8AzYsJeYmUK3u5lboX3lGFh3lXhHPFc2oh3wDN6wgNR5rVaRuzKIuk55uHTfhx25qIr7OIQD7Vcq+TgsnSuW83xEudYw3/ShkV+Yxmbvb3suff4lIJSusivLn/OivibdK0J+aQJ6bVxdTdAORc1myNoNR7PA/b7CEAxQeB8sgT4ilddzvhAsySUUfAyrKqpjRoTue+A2tfiB6EJFHrh5wF0ZoRhkMNe8hMZHpmxYveJypgxvUPobAqpyxQ7TfpoNn1eS9s6wN3JuToozgMYm02ZZVVKhABnZZAhD8zf+zyevW7UOgsq7ZrudFq/um99Fr97yLvQjyIhJ4UcRZD9JPzUffFLLC8T5D1lHFFSXB9LfjUGs6YBRtMKKRHpbuabZhLAX2NaWm4FTC1yp/ffo8fVYDkY6q4xO5nDJkZOb0OBCHLgYCzJq0pb8Gx3FWajivjov/QB4emo2cWkTgXz5ZZX/w2P7Fs 3LyJNnMt tuBJiWv7Tl8OwVQLfPUncZwfrx1cbPz7QMyJVsAYWHWNrPcN5yEC5Ssem4/BEKXaCwyvBfvzwfDYe9yZRsRM5gSRo92kFZNYmbVsG9eDd6rnu3Octqi2Na0iS+fs1Em4857OYcoptTUjomC+NnEWiQkCnNmMvVX7E7NFFmtnTETjUIxyN9zm7eOcI/Dcp4pT+x/+csM1T7qqja2Rj9Tl8qKihHVyuyZtRPrZc8utv9zpWFXyF95vUXALHbG0hp6R8lzarPQmtMQXK/jdn9vRkScpbWhppfifib/8xCUMQwoVvYHGt6yJnumam/nxYqkNj0OBu+srbMYRvvhWvXL8LjQ2lyfBokqvmdbsmH4/T3F6m3wZScFnvSCz0vtJqVNcLF1d6JiGS5wOJMdzaTpphAGw6fBR33vvy39CIQI1SFN8z9ZbXNqre14VvbMJUmPB9FgWsQGPN+xStCBLXOk+s2jS+JAn+2N/ZhKhm+OMfMabpYMj+s1FsLLbgeNOMzwqWAhlbhpN79GSlCYWO+CsRH90jYH2mGKQlp/zDKvvZ0zlpRu7Amo/jondvCA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Matthew Wilcox block_read_full_folio() currently puts all !uptodate buffers into an array allocated on the stack, then iterates over it twice, first locking the buffers and then submitting them for read. We want to remove this array because it occupies too much stack space on configurations with a larger PAGE_SIZE (eg 512 bytes with 8 byte pointers and a 64KiB PAGE_SIZE). We cannot simply submit buffer heads as we find them as the completion handler needs to be able to tell when all reads are finished, so it can end the folio read. So we keep one buffer in reserve (using the 'prev' variable) until the end of the function. Reviewed-by: Hannes Reinecke Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: Luis Chamberlain --- fs/buffer.c | 51 +++++++++++++++++++++------------------------------ 1 file changed, 21 insertions(+), 30 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index b99560e8a142..167fa3e33566 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -2361,9 +2361,8 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) { struct inode *inode = folio->mapping->host; sector_t iblock, lblock; - struct buffer_head *bh, *head, *arr[MAX_BUF_PER_PAGE]; + struct buffer_head *bh, *head, *prev = NULL; size_t blocksize; - int nr, i; int fully_mapped = 1; bool page_error = false; loff_t limit = i_size_read(inode); @@ -2380,7 +2379,6 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) iblock = div_u64(folio_pos(folio), blocksize); lblock = div_u64(limit + blocksize - 1, blocksize); bh = head; - nr = 0; do { if (buffer_uptodate(bh)) @@ -2410,40 +2408,33 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) if (buffer_uptodate(bh)) continue; } - arr[nr++] = bh; + + lock_buffer(bh); + if (buffer_uptodate(bh)) { + unlock_buffer(bh); + continue; + } + + mark_buffer_async_read(bh); + if (prev) + submit_bh(REQ_OP_READ, prev); + prev = bh; } while (iblock++, (bh = bh->b_this_page) != head); if (fully_mapped) folio_set_mappedtodisk(folio); - if (!nr) { - /* - * All buffers are uptodate or get_block() returned an - * error when trying to map them - we can finish the read. - */ - folio_end_read(folio, !page_error); - return 0; - } - - /* Stage two: lock the buffers */ - for (i = 0; i < nr; i++) { - bh = arr[i]; - lock_buffer(bh); - mark_buffer_async_read(bh); - } - /* - * Stage 3: start the IO. Check for uptodateness - * inside the buffer lock in case another process reading - * the underlying blockdev brought it uptodate (the sct fix). + * All buffers are uptodate or get_block() returned an error + * when trying to map them - we must finish the read because + * end_buffer_async_read() will never be called on any buffer + * in this folio. */ - for (i = 0; i < nr; i++) { - bh = arr[i]; - if (buffer_uptodate(bh)) - end_buffer_async_read(bh, 1); - else - submit_bh(REQ_OP_READ, bh); - } + if (prev) + submit_bh(REQ_OP_READ, prev); + else + folio_end_read(folio, !page_error); + return 0; } EXPORT_SYMBOL(block_read_full_folio);