From patchwork Fri Aug 11 16:15:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13351085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC8C7C41513 for ; Fri, 11 Aug 2023 16:15:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235181AbjHKQPq (ORCPT ); Fri, 11 Aug 2023 12:15:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58212 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234275AbjHKQPm (ORCPT ); Fri, 11 Aug 2023 12:15:42 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57D5C273E; Fri, 11 Aug 2023 09:15:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=mGLxLmLqoMtqKqObPpHvI58V/LNR6g3q9ebkb1KeSj4=; b=qovPmxo6vngNv6zu0MhFQMQVHu JY/vfqtADdsSKY48Foze9fiVOUY3EU3/VEqkPES+ALe5cZIxWHqsapi6R9IED0OuShRmmW4PkvOvp Kf9j7PWrHUhPDOVezbqHIXon6GD+UAvTOtuFgvJYqqm6j82cL+oz2fq3uk98sN8msmz1cEPUO0nBc zHl+CIGgWFQ3wfAIA1IGVYBGERRuhswJjYLxSpu5X8MnF/+wQBbrAKNCpJW/KYa3RTVNdUCPDreXm eXsPvj/6KL/F6q8Q0rrBFNWrCr8VxSY4taIg3tqe/hzCTW3OTtexFfCjslQP/k4qYAB8oSEwV+k4D P7Kr/r0w==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qUUnT-0027kW-0A; Fri, 11 Aug 2023 16:15:31 +0000 From: "Matthew Wilcox (Oracle)" To: akpm@linux-foundation.org Cc: "Matthew Wilcox (Oracle)" , Hui Zhu , linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH 1/3] buffer: Pass GFP flags to folio_alloc_buffers() Date: Fri, 11 Aug 2023 17:15:26 +0100 Message-Id: <20230811161528.506437-2-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230811161528.506437-1-willy@infradead.org> References: <20230811161528.506437-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Instead of creating entirely new flags, inherit them from grow_dev_page(). The other callers create the same flags that this function used to create. Signed-off-by: Matthew Wilcox (Oracle) --- fs/buffer.c | 17 +++++++++-------- include/linux/buffer_head.h | 2 +- 2 files changed, 10 insertions(+), 9 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index bd091329026c..7326acc29541 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -920,16 +920,12 @@ int remove_inode_buffers(struct inode *inode) * which may not fail from ordinary buffer allocations. */ struct buffer_head *folio_alloc_buffers(struct folio *folio, unsigned long size, - bool retry) + gfp_t gfp) { struct buffer_head *bh, *head; - gfp_t gfp = GFP_NOFS | __GFP_ACCOUNT; long offset; struct mem_cgroup *memcg, *old_memcg; - if (retry) - gfp |= __GFP_NOFAIL; - /* The folio lock pins the memcg */ memcg = folio_memcg(folio); old_memcg = set_active_memcg(memcg); @@ -972,7 +968,11 @@ EXPORT_SYMBOL_GPL(folio_alloc_buffers); struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size, bool retry) { - return folio_alloc_buffers(page_folio(page), size, retry); + gfp_t gfp = GFP_NOFS | __GFP_ACCOUNT; + if (retry) + gfp |= __GFP_NOFAIL; + + return folio_alloc_buffers(page_folio(page), size, gfp); } EXPORT_SYMBOL_GPL(alloc_page_buffers); @@ -1074,7 +1074,7 @@ grow_dev_page(struct block_device *bdev, sector_t block, goto failed; } - bh = folio_alloc_buffers(folio, size, true); + bh = folio_alloc_buffers(folio, size, gfp_mask); /* * Link the folio to the buffers and initialise them. Take the @@ -1665,8 +1665,9 @@ void folio_create_empty_buffers(struct folio *folio, unsigned long blocksize, unsigned long b_state) { struct buffer_head *bh, *head, *tail; + gfp_t gfp = GFP_NOFS | __GFP_ACCOUNT | __GFP_NOFAIL; - head = folio_alloc_buffers(folio, blocksize, true); + head = folio_alloc_buffers(folio, blocksize, gfp); bh = head; do { bh->b_state |= b_state; diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index 6cb3e9af78c9..d17efb8b7976 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -200,7 +200,7 @@ void folio_set_bh(struct buffer_head *bh, struct folio *folio, unsigned long offset); bool try_to_free_buffers(struct folio *); struct buffer_head *folio_alloc_buffers(struct folio *folio, unsigned long size, - bool retry); + gfp_t gfp); struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size, bool retry); void create_empty_buffers(struct page *, unsigned long, From patchwork Fri Aug 11 16:15:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13351086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9E8EC0015E for ; Fri, 11 Aug 2023 16:15:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235132AbjHKQPw (ORCPT ); Fri, 11 Aug 2023 12:15:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235157AbjHKQPq (ORCPT ); Fri, 11 Aug 2023 12:15:46 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C582271E; Fri, 11 Aug 2023 09:15:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=vnCMJDHvrSrLNSZkcpfQR1Q0wFl30okMiQEpgzdx9ds=; b=USTOcRLnPKT38Z7UkKIopQ+evQ +U6mA6hybkvUu3vUq2cTEdn6mIS9VGPYYF5fvVdHBxkigAjcbpMI6X8tJgNnCvtGtRkqMAaalNA7E 7nCFXGIl+jVu2RnbwmlODeDHQRBqw7M+VjNfBXRCqj4M8gbC+p4HY7E0t51umne9y9jiTyfLCm3S8 +hekBh7O12rkJsxZ6bsasm1Ctz+kl1ZP539DmEfEStQ8zAATDW6HRh0rbDT04l8moQE/EtKCqX56b k4sk6igMuKis+SGUSMjhkbcpM1DOf4xXwvVlQ7FkQQZQN4y8eXOedNeQHu8xkEi5NvkJ24x43q1hz 2x+TJIsQ==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qUUnT-0027kY-2q; Fri, 11 Aug 2023 16:15:31 +0000 From: "Matthew Wilcox (Oracle)" To: akpm@linux-foundation.org Cc: "Matthew Wilcox (Oracle)" , Hui Zhu , linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH 2/3] buffer: Hoist GFP flags from grow_dev_page() to __getblk_gfp() Date: Fri, 11 Aug 2023 17:15:27 +0100 Message-Id: <20230811161528.506437-3-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230811161528.506437-1-willy@infradead.org> References: <20230811161528.506437-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org grow_dev_page() is only called by grow_buffers(). grow_buffers() is only called by __getblk_slow() and __getblk_slow() is only called from __getblk_gfp(), so it is safe to move the GFP flags setting all the way up. With that done, add a new bdev_getblk() entry point that leaves the GFP flags the way the caller specified them. Signed-off-by: Matthew Wilcox (Oracle) --- fs/buffer.c | 60 ++++++++++++++++++++++++------------- include/linux/buffer_head.h | 2 ++ 2 files changed, 41 insertions(+), 21 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index 7326acc29541..122b7d16befb 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -1048,20 +1048,11 @@ grow_dev_page(struct block_device *bdev, sector_t block, struct buffer_head *bh; sector_t end_block; int ret = 0; - gfp_t gfp_mask; - - gfp_mask = mapping_gfp_constraint(inode->i_mapping, ~__GFP_FS) | gfp; - - /* - * XXX: __getblk_slow() can not really deal with failure and - * will endlessly loop on improvised global reclaim. Prefer - * looping in the allocator rather than here, at least that - * code knows what it's doing. - */ - gfp_mask |= __GFP_NOFAIL; folio = __filemap_get_folio(inode->i_mapping, index, - FGP_LOCK | FGP_ACCESSED | FGP_CREAT, gfp_mask); + FGP_LOCK | FGP_ACCESSED | FGP_CREAT, gfp); + if (IS_ERR(folio)) + return PTR_ERR(folio); bh = folio_buffers(folio); if (bh) { @@ -1074,7 +1065,9 @@ grow_dev_page(struct block_device *bdev, sector_t block, goto failed; } - bh = folio_alloc_buffers(folio, size, gfp_mask); + bh = folio_alloc_buffers(folio, size, gfp); + if (!bh) + goto failed; /* * Link the folio to the buffers and initialise them. Take the @@ -1426,24 +1419,49 @@ __find_get_block(struct block_device *bdev, sector_t block, unsigned size) } EXPORT_SYMBOL(__find_get_block); +/** + * bdev_getblk - Get a buffer_head in a block device's buffer cache. + * @bdev: The block device. + * @block: The block number. + * @size: The size of buffer_heads for this @bdev. + * @gfp: The memory allocation flags to use. + * + * In contrast to __getblk_gfp(), the @gfp flags must be all of the flags; + * they are not augmented with the mapping's GFP flags. + * + * Return: The buffer head, or NULL if memory could not be allocated. + */ +struct buffer_head *bdev_getblk(struct block_device *bdev, sector_t block, + unsigned size, gfp_t gfp) +{ + struct buffer_head *bh = __find_get_block(bdev, block, size); + + might_alloc(gfp); + if (bh) + return bh; + + return __getblk_slow(bdev, block, size, gfp); +} +EXPORT_SYMBOL(bdev_getblk); + /* * __getblk_gfp() will locate (and, if necessary, create) the buffer_head * which corresponds to the passed block_device, block and size. The * returned buffer has its reference count incremented. - * - * __getblk_gfp() will lock up the machine if grow_dev_page's - * try_to_free_buffers() attempt is failing. FIXME, perhaps? */ struct buffer_head * __getblk_gfp(struct block_device *bdev, sector_t block, unsigned size, gfp_t gfp) { - struct buffer_head *bh = __find_get_block(bdev, block, size); + gfp |= mapping_gfp_constraint(bdev->bd_inode->i_mapping, ~__GFP_FS); - might_sleep(); - if (bh == NULL) - bh = __getblk_slow(bdev, block, size, gfp); - return bh; + /* + * Prefer looping in the allocator rather than here, at least that + * code knows what it's doing. + */ + gfp |= __GFP_NOFAIL; + + return bdev_getblk(bdev, block, size, gfp); } EXPORT_SYMBOL(__getblk_gfp); diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index d17efb8b7976..01110db9213c 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -233,6 +233,8 @@ void __wait_on_buffer(struct buffer_head *); wait_queue_head_t *bh_waitq_head(struct buffer_head *bh); struct buffer_head *__find_get_block(struct block_device *bdev, sector_t block, unsigned size); +struct buffer_head *bdev_getblk(struct block_device *bdev, sector_t block, + unsigned size, gfp_t gfp); struct buffer_head *__getblk_gfp(struct block_device *bdev, sector_t block, unsigned size, gfp_t gfp); void __brelse(struct buffer_head *); From patchwork Fri Aug 11 16:15:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13351084 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABB63C001DE for ; Fri, 11 Aug 2023 16:15:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234961AbjHKQPn (ORCPT ); Fri, 11 Aug 2023 12:15:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58184 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232046AbjHKQPl (ORCPT ); Fri, 11 Aug 2023 12:15:41 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 554542694; Fri, 11 Aug 2023 09:15:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=TtnsL4i1WDoruwjihR7ho5E4T0YVgEIZCD8hWU4Po38=; b=Hk9/6EzKfW6Tas2SkqzCYnGlqY iIwVxlAicMgGum0LcucsvrJ2hXGVeC9GSLJLx//IoSjl3dd9G6O7DBlNCtMxUMj6x/TrYlCy7IApx FApal6jU0O0dDTmmHkH4Wb8E+lFP25+jLGpCJxAJ6Ow8Yd/A3rTxDVOxkUCeGQVgwOf1Y0rE3958G 2p8lgRV2Ivbq/xKQWVvp0zEj1RfF2kLrGz+2UZ/iTda59lqiAUidk+cpPLDgU2H43VLJWtN7oIy/X KdTvxVjwfJDUz58OWvcffBaXV+LzLGfns3lTAPPMIN1ZxVBmFhKV5aczuafCdnlYq7bQ/7wOEIQJS MjmNlqPg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qUUnT-0027ka-6Y; Fri, 11 Aug 2023 16:15:31 +0000 From: "Matthew Wilcox (Oracle)" To: akpm@linux-foundation.org Cc: "Matthew Wilcox (Oracle)" , Hui Zhu , linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH 3/3] ext4: Use bdev_getblk() to avoid memory reclaim in readahead path Date: Fri, 11 Aug 2023 17:15:28 +0100 Message-Id: <20230811161528.506437-4-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230811161528.506437-1-willy@infradead.org> References: <20230811161528.506437-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org sb_getblk_gfp adds __GFP_NOFAIL, which is unnecessary for readahead; we're quite comfortable with the possibility that we may not get a bh back. Switch to bdev_getblk() which does not include __GFP_NOFAIL. Signed-off-by: Matthew Wilcox (Oracle) Reported-by: Hui Zhu Link: https://lore.kernel.org/linux-fsdevel/20230811035705.3296-1-teawaterz@linux.alibaba.com/ --- fs/ext4/super.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/ext4/super.c b/fs/ext4/super.c index c94ebf704616..48524314be97 100644 --- a/fs/ext4/super.c +++ b/fs/ext4/super.c @@ -254,7 +254,8 @@ struct buffer_head *ext4_sb_bread_unmovable(struct super_block *sb, void ext4_sb_breadahead_unmovable(struct super_block *sb, sector_t block) { - struct buffer_head *bh = sb_getblk_gfp(sb, block, 0); + struct buffer_head *bh = bdev_getblk(sb->s_bdev, block, + sb->s_blocksize, GFP_NOWAIT); if (likely(bh)) { if (trylock_buffer(bh)) From patchwork Fri Aug 11 19:23:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13351342 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53397C001DE for ; Fri, 11 Aug 2023 19:23:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234617AbjHKTXT (ORCPT ); Fri, 11 Aug 2023 15:23:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51774 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229959AbjHKTXS (ORCPT ); Fri, 11 Aug 2023 15:23:18 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 280C0120; Fri, 11 Aug 2023 12:23:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=SCjfE0ihVcOKQWLIcU8GEL15xW0kjppT5rqoPuDlnqU=; b=GzoTSGKJWneXfmwmwJytS9bRLm yuwJ6wSD5oilzUyqu9EYqjwALMw3krzY9689khfPDLvDqU8cdWrTWkcwzU6+/ghc4sp/5ZTrrwke6 I8EaRWP3PJKgKNw0BDEur913Z0/1OAWthjSBwj0jcCPHN3elDWqiVmt2129ugispTm+GTPjdMsN/A gE16a1FOese/e1VWbqNfd1Qk+jmCXtehmwtNnyAsRaULoNOwBRux8lAvxptPOaRxMtAzvpscsIMDz o1lMBoysZ53LM9o1znL/Beq4BjsS+wyFdk65021lvKyjORnouttYzMgwZihSqfsZpsNZsfQ/Nm9SM 2+oJAZzw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qUXj7-002vvW-41; Fri, 11 Aug 2023 19:23:13 +0000 Date: Fri, 11 Aug 2023 20:23:13 +0100 From: Matthew Wilcox To: akpm@linux-foundation.org Cc: Hui Zhu , linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH 4/3] buffer: Use bdev_getblk() in __breadahead() Message-ID: References: <20230811161528.506437-1-willy@infradead.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230811161528.506437-1-willy@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org It occurs to me that this would also be useful and I'll include it in the next version: diff --git a/fs/buffer.c b/fs/buffer.c index 122b7d16befb..b551a5b1196b 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -1470,7 +1470,9 @@ EXPORT_SYMBOL(__getblk_gfp); */ void __breadahead(struct block_device *bdev, sector_t block, unsigned size) { - struct buffer_head *bh = __getblk(bdev, block, size); + struct buffer_head *bh = bdev_getblk(bdev, block, size, + GFP_NOWAIT | __GFP_MOVABLE | __GFP_ACCOUNT); + if (likely(bh)) { bh_readahead(bh, REQ_RAHEAD); brelse(bh);