From patchwork Tue Feb 4 23:12:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 13960169 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F265C02196 for ; Tue, 4 Feb 2025 23:12:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D03028000F; Tue, 4 Feb 2025 18:12:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7735E28000E; Tue, 4 Feb 2025 18:12:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A596280010; Tue, 4 Feb 2025 18:12:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id DB13B28000D for ; Tue, 4 Feb 2025 18:12:18 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 96BB7B0589 for ; Tue, 4 Feb 2025 23:12:18 +0000 (UTC) X-FDA: 83083812756.22.E802ACD Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf17.hostedemail.com (Postfix) with ESMTP id E94C74000F for ; Tue, 4 Feb 2025 23:12:16 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=bLk3usnU; spf=none (imf17.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=quarantine) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738710737; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AJ1b0u4x2rstuiVDqeFM5lSxnt8lCHKW5w+bHMF0FyA=; b=zLvVIUKseYVos4Hi9g6rDuoARRBMCEexTPKGRVGKoAiiX7ABAZnSdE9wCt8UGFTTdLDVTW 0KtY9tji5Vxze6dftsSH8Wo2YfnmDp/f81BjvCfEqbcs4CdmzzpHK2jSuAjRyWNTSAl5Yb VLkN4esb7vKb1nC+HUWCjr0TkqNISHo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738710737; a=rsa-sha256; cv=none; b=OGnBDvJW0a4o4rD0RqrHeLjX4pRiaA+/JZctOzALrwBay5hnQbqxUT+JvP/KMJyZNHtCTT LIfLqBpZaFkmEeRCqoQG4yXZ6yk3kdcxmO6ZpSUiyb2vtdJFW94RTVVW1gPG5D2buOv1Bn 8+6JGg4aDRWpFFCbwyrMslyHmV3C7L0= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=bLk3usnU; spf=none (imf17.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=quarantine) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=AJ1b0u4x2rstuiVDqeFM5lSxnt8lCHKW5w+bHMF0FyA=; b=bLk3usnU7zD694D9cQlqCOIrGl Zxo2SpJ8CmrEnmdcSP9tTWq9KazLoLEJXfKmVyoWXHcZIIzXjNuhivNzLCD1GkFL9/Kj3aqWK86pS EzI+jZTF6PaJ5TBjJC3z7QZZAz2ATQJKMVfE/PH34cMBCPJjo9fAfzGGhOS+pylbC0D2dzUf77w23 FtQT5QLp0D9jcPgK6T8gfY7FY55KtdBlH01vUDo8v2ESxod5Zu+CxKgNXCTNQ820pC9tUpMAuZ/od qo3+wT5PtysGvfbIUhMxWAYLub35Ep2ZoLFFj5hMU+FkI2ef676zc86QDgnz2KnEuAZTgNELMIAL0 nK7kF1sA==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tfS5T-00000001nhO-1EZy; Tue, 04 Feb 2025 23:12:11 +0000 From: Luis Chamberlain To: hare@suse.de, willy@infradead.org, dave@stgolabs.net, david@fromorbit.com, djwong@kernel.org, kbusch@kernel.org Cc: john.g.garry@oracle.com, hch@lst.de, ritesh.list@gmail.com, linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, gost.dev@samsung.com, p.raghav@samsung.com, da.gomez@samsung.com, kernel@pankajraghav.com, mcgrof@kernel.org Subject: [PATCH v2 2/8] fs/buffer: remove batching from async read Date: Tue, 4 Feb 2025 15:12:03 -0800 Message-ID: <20250204231209.429356-3-mcgrof@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250204231209.429356-1-mcgrof@kernel.org> References: <20250204231209.429356-1-mcgrof@kernel.org> MIME-Version: 1.0 X-Stat-Signature: xuyokhydnrzduorb5a6oy7ocntf441ew X-Rspamd-Pre-Result: action=add header; module=dmarc; Action set by DMARC X-Rspam-User: X-Rspamd-Queue-Id: E94C74000F X-Rspamd-Server: rspam03 X-Rspam: Yes X-HE-Tag: 1738710736-758591 X-HE-Meta: U2FsdGVkX1/LyQPVFT8Mhno+Wr6gI2cOM1vYKrpvAV1i1LCgJIJ2nQdqeU6Dh8FKl5UigZgsm4+0Jxl0zErEKe2A61yzCbscvVTQSslo41UElruPpUnS3eNcWwRaFtxToFslaMftg3nTroLaCaaWIdXLkD3dz2E360PmqRFGDIzSV/rDpY85DZY6pYwgaoakYJPNErNb/Pf5r26owZ0kp0LdknUiTok1bG+d7krxmNPthTb4uF3Z3RtlWTnVMRgbF5/aLI0Z0jm97Ht2akLn/J4HZ1SGh7MK5cEjrJB/HXU80jOFr7rhoQImPyu6Qvxf28OH8cMPbsJiiUtpRvJ74ME46RVxB+Fkb6Hdzc/ea5pGZFWAT2CVxpkjfUZcCMD+XmXpvPrqBxcSkH/+Sqj8Tzlz0fTNQkYNikp/VL0Aj4H6C1SW7Y/CagJAjbD6xCX2AwvKU4V7Hy4kg7Q62K264neJy9AWpuQvhY/HQCZ9R1vWJzFkf6dWRSjuhrJ46O9WJLKDibRuFAlHLMuNRV4zJq0mRopUQ849jslEp9JJCdEUsUNsS27qzc9kr3so1nXlfeCaayF7Pv8QkxcvscH3BUXrAImPHeP9Zl9+udif4D1792s6NJ/YM1QqXASUykKnArXpDdxIH44OdaLG1CLD+Bg6cG9DBOj889qVee59VD71emrngr1VsQpt5vkdIKQjCls0x2jadgPni3RnMD26O3rYFDBsjL3BTfUxsJhOyzyDKkSrgeH8O383z0kbEVIvvdKiu/b+jWOC3qN04EYiNlBotvTCCsO6vUnZYkfL+gxWl++KWZLnb+X9HXSiXuhRVTYivz2ANNCu7b+RwKLHwRRI2Ev44krGjIrBeJCuuiXh3uc2UpPtx6ozmdTCZ9oUho1B8V6/H8ghVbQLVD9+99dxTqd1z7pqZuJt2ZJ0j6seUfj2xVk8LiZGN4xQICTOODUeN7nNh+uT+Svp5TP sN4UU9x2 YnKwBQeazjhmZ+loCfsi0GzbR2KIEn3ftOENy7QnTwErbopPx9iLiY3dXzDB9AeS6FPbaNTJhWwtX8he+VKWyw5Mg/3tfOXwaxVo41Z/3xh+rEDXsqTDRzHMsVKK4ZJ0Rj+mK7QjpRBD4Gx6KgZNeglf9u925YAQsdtU4EMmPYlrohG01lfA2CjSlbaPsdN/tHWljHfanT9oyWMKJkRyvEEIr91GzTqtP9hKLBbyEO7nj6r5mf9wPrhGB0blyzLO1cGKfr3Vc26N1VSZiqiLIGpWWtOoQ0jeYm8wre0sALkQd/BDyCabbVellVsPu7dKWG7A1KQ0gzguVQoUufO2tV0VPH1Nk/oGji+HNVBFSsDmlf/faoKkiiTA6G83q8T0nSUBhyFtFicQjMXIdo3nNn4CuMmc0tx8aDVbqcZ5lbBPlM64Jcb90iiFwqPIjhGFI3TNCuqSUHsPfrWcFAwBNRvg650nVW7URpsX2+ZpEm2kvws7V0kg7mdBSUdDRS4aT4FS5sjR4TXh96/tAsFNFDXkL/EDx6qaIn1q6bavAt/3FlGXfSAesgWO9Gg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Matthew Wilcox The current implementation of a folio async read in block_read_full_folio() first batches all buffer-heads which need IOs issued for by putting them on an array of max size MAX_BUF_PER_PAGE. After collection it locks the batched buffer-heads and finally submits the pending reads. On systems with CPUs where the system page size is quite larger like Hexagon with 256 KiB this batching can lead stack growth warnings so we want to avoid that. Note the use of folio_end_read() through block_read_full_folio(), its used either when the folio is determined to be fully uptodate and no pending read is needed, an IO error happened on get_block(), or an out of bound read raced against batching collection to make our required reads uptodate. We can simplify this logic considerably and remove the stack growth issues of MAX_BUF_PER_PAGE by just replacing the batched logic with one which only issues IO for the previous buffer-head keeping in mind we'll always have one buffer-head (the current one) on the folio with an async flag, this will prevent any calls to folio_end_read(). So we accomplish two things with this: o Avoid large stacks arrays with MAX_BUF_PER_PAGE o Make the need for folio_end_read() explicit and easier to read Suggested-by: Matthew Wilcox Signed-off-by: Luis Chamberlain --- fs/buffer.c | 51 +++++++++++++++++++++------------------------------ 1 file changed, 21 insertions(+), 30 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index b99560e8a142..167fa3e33566 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -2361,9 +2361,8 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) { struct inode *inode = folio->mapping->host; sector_t iblock, lblock; - struct buffer_head *bh, *head, *arr[MAX_BUF_PER_PAGE]; + struct buffer_head *bh, *head, *prev = NULL; size_t blocksize; - int nr, i; int fully_mapped = 1; bool page_error = false; loff_t limit = i_size_read(inode); @@ -2380,7 +2379,6 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) iblock = div_u64(folio_pos(folio), blocksize); lblock = div_u64(limit + blocksize - 1, blocksize); bh = head; - nr = 0; do { if (buffer_uptodate(bh)) @@ -2410,40 +2408,33 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) if (buffer_uptodate(bh)) continue; } - arr[nr++] = bh; + + lock_buffer(bh); + if (buffer_uptodate(bh)) { + unlock_buffer(bh); + continue; + } + + mark_buffer_async_read(bh); + if (prev) + submit_bh(REQ_OP_READ, prev); + prev = bh; } while (iblock++, (bh = bh->b_this_page) != head); if (fully_mapped) folio_set_mappedtodisk(folio); - if (!nr) { - /* - * All buffers are uptodate or get_block() returned an - * error when trying to map them - we can finish the read. - */ - folio_end_read(folio, !page_error); - return 0; - } - - /* Stage two: lock the buffers */ - for (i = 0; i < nr; i++) { - bh = arr[i]; - lock_buffer(bh); - mark_buffer_async_read(bh); - } - /* - * Stage 3: start the IO. Check for uptodateness - * inside the buffer lock in case another process reading - * the underlying blockdev brought it uptodate (the sct fix). + * All buffers are uptodate or get_block() returned an error + * when trying to map them - we must finish the read because + * end_buffer_async_read() will never be called on any buffer + * in this folio. */ - for (i = 0; i < nr; i++) { - bh = arr[i]; - if (buffer_uptodate(bh)) - end_buffer_async_read(bh, 1); - else - submit_bh(REQ_OP_READ, bh); - } + if (prev) + submit_bh(REQ_OP_READ, prev); + else + folio_end_read(folio, !page_error); + return 0; } EXPORT_SYMBOL(block_read_full_folio);