From patchwork Thu Dec 22 15:02:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13080066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 947EEC4332F for ; Thu, 22 Dec 2022 15:02:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 31C14940008; Thu, 22 Dec 2022 10:02:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2CBF8900002; Thu, 22 Dec 2022 10:02:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 16CBA940008; Thu, 22 Dec 2022 10:02:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 067A6900002 for ; Thu, 22 Dec 2022 10:02:16 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 9A5E9120E66 for ; Thu, 22 Dec 2022 15:02:15 +0000 (UTC) X-FDA: 80270257830.18.8A3FCFB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf13.hostedemail.com (Postfix) with ESMTP id 779C62000C for ; Thu, 22 Dec 2022 15:02:13 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=c9q8463B; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf13.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671721333; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=K2+I4bpb9SdoO0DVAXNZibfBlFbEjwAQot6E33TLUuc=; b=XEB1RUqzTD1J4npMOoBgdnxah85doEGw221P3MOVtRHxxeYd/UjdCvlVe8GjywGpz5oIZw MQf36HqmR/Q+rOFAH3ae2alzGBX539RrYZZZUV3n6hBrcxis6oFL1VNeboBMOfK6+N/Mlj op7S1XJ2/ePwTzUj3citLwmPV1wjuSA= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=c9q8463B; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf13.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671721333; a=rsa-sha256; cv=none; b=WBpYtv+OY0UTQUbGLkuuGDMpFElkNqH/2dxRKiFhXV6wuK9vA1Pqp496AEXpPhDJgO/LSh gpDJyYZQl/lcAj+0kFqgRAgyBUHiDGxhYghQ8JDSfMrFRHiXL9wDziKpWPV8dyAcigfs8M SeYc3G4wH7agoKveE3DoQCPTm/3KKqg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1671721332; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K2+I4bpb9SdoO0DVAXNZibfBlFbEjwAQot6E33TLUuc=; b=c9q8463Bu6U8VrFeDo4c4ntsIDk3ioHKm7F6c/0cHHxqOxNw6BZ2eXC5vDOcGY+8WJ41bi Epg4TZllK0+wm8PZJ7fZqrIJUoknyat8H9xUdfB2xROw3CWIG0OWLq+GL+oPQDmmZuuI7E HVpO4PBdFrQsIGRWbQvS0ftaZhf/ZTQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-216-4cOOowuzO0aCUZLDTERhxA-1; Thu, 22 Dec 2022 10:02:08 -0500 X-MC-Unique: 4cOOowuzO0aCUZLDTERhxA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 112398533DC; Thu, 22 Dec 2022 15:02:06 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.96]) by smtp.corp.redhat.com (Postfix) with ESMTP id 67A0B40C2064; Thu, 22 Dec 2022 15:02:03 +0000 (UTC) Organization: Red Hat UK Ltd. Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SI4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 3798903 Subject: [PATCH v5 1/3] mm: Merge folio_has_private()/filemap_release_folio() call pairs From: David Howells To: Matthew Wilcox Cc: Rohith Surabattula , Linus Torvalds , Steve French , Shyam Prasad N , Rohith Surabattula , Dave Wysochanski , Dominique Martinet , Ilya Dryomov , Theodore Ts'o , Andreas Dilger , linux-cachefs@redhat.com, linux-cifs@vger.kernel.org, linux-afs@lists.infradead.org, v9fs-developer@lists.sourceforge.net, ceph-devel@vger.kernel.org, linux-nfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, dhowells@redhat.com, Linus Torvalds , Jeff Layton , linux-afs@lists.infradead.org, linux-nfs@vger.kernel.org, linux-cifs@vger.kernel.org, ceph-devel@vger.kernel.org, v9fs-developer@lists.sourceforge.net, linux-erofs@lists.ozlabs.org, linux-ext4@lists.ozlabs.org, linux-cachefs@redhat.com, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Date: Thu, 22 Dec 2022 15:02:02 +0000 Message-ID: <167172132272.2334525.13617516784050484518.stgit@warthog.procyon.org.uk> In-Reply-To: <167172131368.2334525.8569808925687731937.stgit@warthog.procyon.org.uk> References: <167172131368.2334525.8569808925687731937.stgit@warthog.procyon.org.uk> User-Agent: StGit/1.5 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Rspamd-Queue-Id: 779C62000C X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: 4dutykd6xokpzs4pn1rsa6w5f3rtbcud X-HE-Tag: 1671721333-452854 X-HE-Meta: U2FsdGVkX1/Dbk52hRh8H/c+mECZQXobzYLb+AItomNS80LNcczuDFCX8hiNUXHhwpWtymxoAKHs/AcaSzxGcn22+SWMiAHhVxYY4oy7tPU4W5/9GJtafdc4KuUdf1U7IWFz/fNjwFM9tx5Lmw+L68XpJbrDEN6AHTeBMVFEYi41C7lOPuK6484R8llPVIwKLYScrN1qbRyfx+7lYX0xIlxNfOaM1KSYA2ZV2YYeLaFKNVDE4MZfjds9TyajjTjDfH7hFhTYz+Xr0TElJJa09q1bs9mrIR1iTTi0Z8429O/qTwcZdCcNe6T98oI9KUDgMATjOaJPFk1cnGFlmvbgoLGoBFZ7qqOia8TmDpa8gKO7Z1Wyo3D2Y0ZQro79wMsbNAyvbtiGDom/vEG/wRF7FX2Qq32WbxMjRjpgLcX2//+9kHIbSTjZ+Q5oIlplWzcYii0DonxVMeTL3Ne3BZ+4kds86WfEuwPcsvVVFP3yKGyRpusQhUPgMjsiDcEH90nfUL21IzUIwfZbitXdMTB1O3z73uw9tZHPwUgrE9MAqv+T5HPa0+Y3VJyA1A2j/AqOKlrMtIRXof16Mn2wkrBftdvcV7PZRi3J2qIaR3YqgNSH0MbXm4K2Nhkz7d/brLODBDu4rL2h+GcDhwKuD9eB4A+j+3miesj6DxacVLShJjHSRNgQbfdzlXLnkYRgtKMg4RdURdXNcUVAlEMb3caz+9BCr46YX7RAI+gHMAhbb5EzDxRoZ7ucqBFUIiok7/JOwjSIh8Cg/PdlYqMSRpLx4enh4ntSP6R8Ozn0/++8RKE5j9rskqu/5PlDTvC5u+U/iIabC3Vlusev3nGoLOhzcSAlY53S4Hz3fZO43uzMuGGeoWAJh7gcaShIQK0PQG6N0iIcg5YzEYAlNJeNCL3aQgwATfIdZdBFYjdpliVUNw/Hk4x9j6wJC37EFSCOQq4InoA6tEPO07NpdCMU8Vo TpEDPzfZ KvXs0k9xN0/IwiH5dz8pfX1BI7Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make filemap_release_folio() check folio_has_private(). Then, in most cases, where a call to folio_has_private() is immediately followed by a call to filemap_release_folio(), we can get rid of the test in the pair. The same is done to page_has_private()/try_to_release_page() call pairs. There are a couple of sites in mm/vscan.c that this can't so easily be done. In shrink_folio_list(), there are actually three cases (something different is done for incompletely invalidated buffers), but filemap_release_folio() elides two of them. In shrink_active_list(), we don't have have the folio lock yet, so the check allows us to avoid locking the page unnecessarily. A wrapper function to check if a folio needs release is provided for those places that still need to do it in the mm/ directory. This will acquire additional parts to the condition in a future patch. After this, the only remaining caller of folio_has_private() outside of mm/ is a check in fuse. Changes: ======== ver #5) - Rebased on linus/master. try_to_release_page() has now been entirely replaced by filemap_release_folio(), barring one comment. - Cleaned up some pairs in ext4. ver #4) - Split from fscache fix. - Moved folio_needs_release() to mm/internal.h and removed open-coded version from filemap_release_folio(). ver #3) - Fixed mapping_clear_release_always() to use clear_bit() not set_bit(). - Moved a '&&' to the correct line. ver #2) - Rewrote entirely according to Willy's suggestion[1]. Reported-by: Rohith Surabattula Suggested-by: Matthew Wilcox Signed-off-by: David Howells cc: Matthew Wilcox cc: Linus Torvalds cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Dave Wysochanski cc: Dominique Martinet cc: Ilya Dryomov cc: "Theodore Ts'o" cc: Andreas Dilger cc: linux-cachefs@redhat.com cc: linux-cifs@vger.kernel.org cc: linux-afs@lists.infradead.org cc: v9fs-developer@lists.sourceforge.net cc: ceph-devel@vger.kernel.org cc: linux-nfs@vger.kernel.org cc: linux-ext4@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/Yk9V/03wgdYi65Lb@casper.infradead.org/ [1] Link: https://lore.kernel.org/r/164928630577.457102.8519251179327601178.stgit@warthog.procyon.org.uk/ # v1 Link: https://lore.kernel.org/r/166844174069.1124521.10890506360974169994.stgit@warthog.procyon.org.uk/ # v2 Link: https://lore.kernel.org/r/166869495238.3720468.4878151409085146764.stgit@warthog.procyon.org.uk/ # v3 Link: https://lore.kernel.org/r/1459152.1669208550@warthog.procyon.org.uk/ # v3 also Link: https://lore.kernel.org/r/166924371591.1772793.13893659228628027575.stgit@warthog.procyon.org.uk/ # v4 --- fs/ext4/move_extent.c | 12 ++++-------- fs/splice.c | 3 +-- mm/filemap.c | 2 ++ mm/huge_memory.c | 3 +-- mm/internal.h | 8 ++++++++ mm/khugepaged.c | 3 +-- mm/memory-failure.c | 8 +++----- mm/migrate.c | 3 +-- mm/truncate.c | 6 ++---- mm/vmscan.c | 8 ++++---- 10 files changed, 27 insertions(+), 29 deletions(-) diff --git a/fs/ext4/move_extent.c b/fs/ext4/move_extent.c index 8dbb87edf24c..dedc9d445f24 100644 --- a/fs/ext4/move_extent.c +++ b/fs/ext4/move_extent.c @@ -339,10 +339,8 @@ move_extent_per_page(struct file *o_filp, struct inode *donor_inode, ext4_double_up_write_data_sem(orig_inode, donor_inode); goto data_copy; } - if ((folio_has_private(folio[0]) && - !filemap_release_folio(folio[0], 0)) || - (folio_has_private(folio[1]) && - !filemap_release_folio(folio[1], 0))) { + if (!filemap_release_folio(folio[0], 0) || + !filemap_release_folio(folio[1], 0)) { *err = -EBUSY; goto drop_data_sem; } @@ -361,10 +359,8 @@ move_extent_per_page(struct file *o_filp, struct inode *donor_inode, /* At this point all buffers in range are uptodate, old mapping layout * is no longer required, try to drop it now. */ - if ((folio_has_private(folio[0]) && - !filemap_release_folio(folio[0], 0)) || - (folio_has_private(folio[1]) && - !filemap_release_folio(folio[1], 0))) { + if (!filemap_release_folio(folio[0], 0) || + !filemap_release_folio(folio[1], 0)) { *err = -EBUSY; goto unlock_folios; } diff --git a/fs/splice.c b/fs/splice.c index 5969b7a1d353..e69eddaf9d7c 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -65,8 +65,7 @@ static bool page_cache_pipe_buf_try_steal(struct pipe_inode_info *pipe, */ folio_wait_writeback(folio); - if (folio_has_private(folio) && - !filemap_release_folio(folio, GFP_KERNEL)) + if (!filemap_release_folio(folio, GFP_KERNEL)) goto out_unlock; /* diff --git a/mm/filemap.c b/mm/filemap.c index c4d4ace9cc70..344146c170b0 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3960,6 +3960,8 @@ bool filemap_release_folio(struct folio *folio, gfp_t gfp) struct address_space * const mapping = folio->mapping; BUG_ON(!folio_test_locked(folio)); + if (!folio_needs_release(folio)) + return true; if (folio_test_writeback(folio)) return false; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index abe6cfd92ffa..8490c42dedb3 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2702,8 +2702,7 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) gfp = current_gfp_context(mapping_gfp_mask(mapping) & GFP_RECLAIM_MASK); - if (folio_test_private(folio) && - !filemap_release_folio(folio, gfp)) { + if (!filemap_release_folio(folio, gfp)) { ret = -EBUSY; goto out; } diff --git a/mm/internal.h b/mm/internal.h index bcf75a8b032d..c4c8e58e1d12 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -163,6 +163,14 @@ static inline void set_page_refcounted(struct page *page) set_page_count(page, 1); } +/* + * Return true if a folio needs ->release_folio() calling upon it. + */ +static inline bool folio_needs_release(struct folio *folio) +{ + return folio_has_private(folio); +} + extern unsigned long highest_memmap_pfn; /* diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 5cb401aa2b9d..80157dd79570 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1926,8 +1926,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, goto out_unlock; } - if (folio_has_private(folio) && - !filemap_release_folio(folio, GFP_KERNEL)) { + if (!filemap_release_folio(folio, GFP_KERNEL)) { result = SCAN_PAGE_HAS_PRIVATE; folio_putback_lru(folio); goto out_unlock; diff --git a/mm/memory-failure.c b/mm/memory-failure.c index c77a9e37e27e..a4f809c11ae9 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -843,14 +843,12 @@ static int truncate_error_page(struct page *p, unsigned long pfn, struct folio *folio = page_folio(p); int err = mapping->a_ops->error_remove_page(mapping, p); - if (err != 0) { + if (err != 0) pr_info("%#lx: Failed to punch page: %d\n", pfn, err); - } else if (folio_has_private(folio) && - !filemap_release_folio(folio, GFP_NOIO)) { + else if (!filemap_release_folio(folio, GFP_NOIO)) pr_info("%#lx: failed to release buffers\n", pfn); - } else { + else ret = MF_RECOVERED; - } } else { /* * If the file system doesn't support it just invalidate diff --git a/mm/migrate.c b/mm/migrate.c index a4d3fc65085f..db867bb80128 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -915,8 +915,7 @@ static int fallback_migrate_folio(struct address_space *mapping, * Buffers may be managed in a filesystem specific way. * We must have no buffers or drop them. */ - if (folio_test_private(src) && - !filemap_release_folio(src, GFP_KERNEL)) + if (!filemap_release_folio(src, GFP_KERNEL)) return mode == MIGRATE_SYNC ? -EAGAIN : -EBUSY; return migrate_folio(mapping, dst, src, mode); diff --git a/mm/truncate.c b/mm/truncate.c index 7b4ea4c4a46b..8378aabb5294 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -19,7 +19,6 @@ #include #include #include -#include /* grr. try_to_release_page */ #include #include #include "internal.h" @@ -276,7 +275,7 @@ static long mapping_evict_folio(struct address_space *mapping, if (folio_ref_count(folio) > folio_nr_pages(folio) + folio_has_private(folio) + 1) return 0; - if (folio_has_private(folio) && !filemap_release_folio(folio, 0)) + if (!filemap_release_folio(folio, 0)) return 0; return remove_mapping(mapping, folio); @@ -573,8 +572,7 @@ static int invalidate_complete_folio2(struct address_space *mapping, if (folio->mapping != mapping) return 0; - if (folio_has_private(folio) && - !filemap_release_folio(folio, GFP_KERNEL)) + if (!filemap_release_folio(folio, GFP_KERNEL)) return 0; spin_lock(&mapping->host->i_lock); diff --git a/mm/vmscan.c b/mm/vmscan.c index bd6637fcd8f9..bded71961143 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1996,7 +1996,7 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, * (refcount == 1) it can be freed. Otherwise, leave * the folio on the LRU so it is swappable. */ - if (folio_has_private(folio)) { + if (folio_needs_release(folio)) { if (!filemap_release_folio(folio, sc->gfp_mask)) goto activate_locked; if (!mapping && folio_ref_count(folio) == 1) { @@ -2641,9 +2641,9 @@ static void shrink_active_list(unsigned long nr_to_scan, } if (unlikely(buffer_heads_over_limit)) { - if (folio_test_private(folio) && folio_trylock(folio)) { - if (folio_test_private(folio)) - filemap_release_folio(folio, 0); + if (folio_needs_release(folio) && + folio_trylock(folio)) { + filemap_release_folio(folio, 0); folio_unlock(folio); } } From patchwork Thu Dec 22 15:02:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13080067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D9CAC4332F for ; Thu, 22 Dec 2022 15:02:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 084D9940009; Thu, 22 Dec 2022 10:02:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 03396900002; Thu, 22 Dec 2022 10:02:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E1743940009; Thu, 22 Dec 2022 10:02:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D28F1900002 for ; Thu, 22 Dec 2022 10:02:31 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 64AA9407AA for ; Thu, 22 Dec 2022 15:02:31 +0000 (UTC) X-FDA: 80270258502.14.F84E029 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf07.hostedemail.com (Postfix) with ESMTP id 493D74000D for ; Thu, 22 Dec 2022 15:02:27 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=G81Ravvn; spf=pass (imf07.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671721348; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GGA96kB5DfYk4dcqOCNOCnjdGba2ksAIa634gghn6lw=; b=H/zLFbLLanJCTJX8NUAh513FbSw+mYtVmNm5E3Xkt8ZPkuTYJiLJZHIxXIvdh0LVv//WXj YHHHKTYzGL1gmSxyPEfO4nF//JQmHQrjyIeQMBdwBRcyV55AP3uR39UHgaxAnJs7fPYc54 TM9hz5iujyYj7laVrETT/hRLcVvO6TY= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=G81Ravvn; spf=pass (imf07.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671721348; a=rsa-sha256; cv=none; b=2lzZ6YzUX/45I3sV7NlIliLu6P8zJH2y0sDvKF5H/cMhLbKlF3S+rWkUKlzEo52Z0p06Ky tp31XXiCAbKHpXqlO0oyxPXJ2rnIoB3yeqhTW+We9hCxG6NEoLkG2W2LrQuSfsOnysDdPe 4n7TVl0KKfs5ovlGsvB8VNwGWmezWPg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1671721347; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GGA96kB5DfYk4dcqOCNOCnjdGba2ksAIa634gghn6lw=; b=G81RavvnM66tYrIQAuqKQgn+xcB3TBuBAWAfNTaZO1/20Sy7Ya6kaTPNDqTYcpVpzBI7up 90ID3pG6M6cSD2EOMfDwIdEWbk4BQnMCIUCcAJmXx5mNXo69PmrtBkki3mOVNp/EGaYcbi CWjohzc+IuxEpOFhZKQWkylBdQs2SWY= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-391-KQGAhYXSPRu5DHU9hS8EeA-1; Thu, 22 Dec 2022 10:02:25 -0500 X-MC-Unique: KQGAhYXSPRu5DHU9hS8EeA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7581E3C0F683; Thu, 22 Dec 2022 15:02:24 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.96]) by smtp.corp.redhat.com (Postfix) with ESMTP id E186814152F4; Thu, 22 Dec 2022 15:02:11 +0000 (UTC) Organization: Red Hat UK Ltd. Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SI4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 3798903 Subject: [PATCH v5 2/3] mm, netfs, fscache: Stop read optimisation when folio removed from pagecache From: David Howells To: Matthew Wilcox Cc: Rohith Surabattula , Linus Torvalds , Steve French , Shyam Prasad N , Rohith Surabattula , Dave Wysochanski , Dominique Martinet , Ilya Dryomov , linux-cachefs@redhat.com, linux-cifs@vger.kernel.org, linux-afs@lists.infradead.org, v9fs-developer@lists.sourceforge.net, ceph-devel@vger.kernel.org, linux-nfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, dhowells@redhat.com, Linus Torvalds , Jeff Layton , linux-afs@lists.infradead.org, linux-nfs@vger.kernel.org, linux-cifs@vger.kernel.org, ceph-devel@vger.kernel.org, v9fs-developer@lists.sourceforge.net, linux-erofs@lists.ozlabs.org, linux-ext4@lists.ozlabs.org, linux-cachefs@redhat.com, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Date: Thu, 22 Dec 2022 15:02:11 +0000 Message-ID: <167172133121.2334525.2608800018126833569.stgit@warthog.procyon.org.uk> In-Reply-To: <167172131368.2334525.8569808925687731937.stgit@warthog.procyon.org.uk> References: <167172131368.2334525.8569808925687731937.stgit@warthog.procyon.org.uk> User-Agent: StGit/1.5 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 493D74000D X-Rspam-User: X-Stat-Signature: xn5b7mq3g7deh8eofk14xytbxsfk5ne9 X-HE-Tag: 1671721347-885471 X-HE-Meta: U2FsdGVkX18sDdVMFxIZxb+cMrIQkNZnDyer/i3sLt8cmpKsZu+Xtq6ZlJlApV11Lk6ZqpgaZXZxOtSqYRuIoNkn+yrHSLcnL38gIC1bCoCLpN9J2mvU2CqKOtydZD5hb8i1lu4uK82tnKCV3adKD3rjY+VV+VARZC3zfBQZJpGfhIauahNMuZU5GbaC+Wcz5UC1SpQ7efUpXqoAOI49Et9WdbpxhoQnPtqmcJbjm0U24wag37bmYfqP5fzTz1S1yN6MPbjPM4hrKN2Nq+YojIRknF5loEWI8ImjncKyrkJJfPBVs4F05pw0BFqNV2d6SkM0FzmJ37Po94F1DIlJmbULEQ290vbaMFBglFLrPlekUZgRwWExXvxqpzY/reEyzeEvyywoRrWRWc+63qcI3sf90m/VtH4TtZ1lQJa/Yz1Llc0FIQbE9esZm4PEeQ5KYIOB5CONqB0lZTH65P9aX/EKFvbhGpmDF9WlGaNrQXv+FLW7Zl6LhwMCc6FIzOIyt780dYk2wp+Wl4X0jArtIq4n99wW4/pXjyYTKu6bJOK9uunWmZFt0/BoILX17o8t9pXObhPV8247Rkzm8DxNIP6CjTpotK3Wn+JMq8dsOT2NrH4C07FJFuPmxTyxIh2/1JWMoTtAwK1rhcxVUlQGDJK9LpdlNMEXeZ0kIZr11Dn4rZiSMbtx/Kz4WLNoGtw+7QixQdLXW7UyeCNY559yjeo3ziVwvqzdwoHMmtQExfmgckR49+Bq2/4aMZB/xaip3X2gISSNJImhbxXLctYmEcD12M/6StujdAlMZTwwREuDbRiyAF4lmG4XOFzuQqu9H6/MjTZXjroZhIUjUYtRKo+TpLeKcZga+3v+90dFfP6JLDKsvTu7DIDZMvviFM83JwCik5bxI6hG7RFqk3hg6xvf1pQw57FXvRCEVsFgUPyv6EQPvZyk1R+yEOCGxUOY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Fscache has an optimisation by which reads from the cache are skipped until we know that (a) there's data there to be read and (b) that data isn't entirely covered by pages resident in the netfs pagecache. This is done with two flags manipulated by fscache_note_page_release(): if (... test_bit(FSCACHE_COOKIE_HAVE_DATA, &cookie->flags) && test_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags)) clear_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags); where the NO_DATA_TO_READ flag causes cachefiles_prepare_read() to indicate that netfslib should download from the server or clear the page instead. The fscache_note_page_release() function is intended to be called from ->releasepage() - but that only gets called if PG_private or PG_private_2 is set - and currently the former is at the discretion of the network filesystem and the latter is only set whilst a page is being written to the cache, so sometimes we miss clearing the optimisation. Fix this by following Willy's suggestion[1] and adding an address_space flag, AS_RELEASE_ALWAYS, that causes filemap_release_folio() to always call ->release_folio() if it's set, even if PG_private or PG_private_2 aren't set. Note that this would require folio_test_private() and page_has_private() to become more complicated. To avoid that, in the places[*] where these are used to conditionalise calls to filemap_release_folio() and try_to_release_page(), the tests are removed the those functions just jumped to unconditionally and the test is performed there. [*] There are some exceptions in vmscan.c where the check guards more than just a call to the releaser. I've added a function, folio_needs_release() to wrap all the checks for that. AS_RELEASE_ALWAYS should be set if a non-NULL cookie is obtained from fscache and cleared in ->evict_inode() before truncate_inode_pages_final() is called. Additionally, the FSCACHE_COOKIE_NO_DATA_TO_READ flag needs to be cleared and the optimisation cancelled if a cachefiles object already contains data when we open it. Changes: ======== ver #4) - Split out merging of folio_has_private()/filemap_release_folio() call pairs into a preceding patch. - Don't need to clear AS_RELEASE_ALWAYS in ->evict_inode(). ver #3) - Fixed mapping_clear_release_always() to use clear_bit() not set_bit(). - Moved a '&&' to the correct line. ver #2) - Rewrote entirely according to Willy's suggestion[1]. Reported-by: Rohith Surabattula Suggested-by: Matthew Wilcox Signed-off-by: David Howells cc: Matthew Wilcox cc: Linus Torvalds cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Dave Wysochanski cc: Dominique Martinet cc: Ilya Dryomov cc: linux-cachefs@redhat.com cc: linux-cifs@vger.kernel.org cc: linux-afs@lists.infradead.org cc: v9fs-developer@lists.sourceforge.net cc: ceph-devel@vger.kernel.org cc: linux-nfs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/Yk9V/03wgdYi65Lb@casper.infradead.org/ [1] Link: https://lore.kernel.org/r/164928630577.457102.8519251179327601178.stgit@warthog.procyon.org.uk/ # v1 Link: https://lore.kernel.org/r/166844174069.1124521.10890506360974169994.stgit@warthog.procyon.org.uk/ # v2 Link: https://lore.kernel.org/r/166869495238.3720468.4878151409085146764.stgit@warthog.procyon.org.uk/ # v3 Link: https://lore.kernel.org/r/1459152.1669208550@warthog.procyon.org.uk/ # v3 also Link: https://lore.kernel.org/r/166924372614.1772793.3804564782036202222.stgit@warthog.procyon.org.uk/ # v4 --- fs/9p/cache.c | 2 ++ fs/afs/internal.h | 2 ++ fs/cachefiles/namei.c | 2 ++ fs/ceph/cache.c | 2 ++ fs/cifs/fscache.c | 2 ++ include/linux/pagemap.h | 16 ++++++++++++++++ mm/internal.h | 5 ++++- 7 files changed, 30 insertions(+), 1 deletion(-) diff --git a/fs/9p/cache.c b/fs/9p/cache.c index cebba4eaa0b5..12c0ae29f185 100644 --- a/fs/9p/cache.c +++ b/fs/9p/cache.c @@ -68,6 +68,8 @@ void v9fs_cache_inode_get_cookie(struct inode *inode) &path, sizeof(path), &version, sizeof(version), i_size_read(&v9inode->netfs.inode)); + if (v9inode->netfs.cache) + mapping_set_release_always(inode->i_mapping); p9_debug(P9_DEBUG_FSC, "inode %p get cookie %p\n", inode, v9fs_inode_cookie(v9inode)); diff --git a/fs/afs/internal.h b/fs/afs/internal.h index 9ba7b68375c9..8e4afaeb6bff 100644 --- a/fs/afs/internal.h +++ b/fs/afs/internal.h @@ -680,6 +680,8 @@ static inline void afs_vnode_set_cache(struct afs_vnode *vnode, { #ifdef CONFIG_AFS_FSCACHE vnode->netfs.cache = cookie; + if (cookie) + mapping_set_release_always(vnode->netfs.inode.i_mapping); #endif } diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c index 03ca8f2f657a..50b2ee163af6 100644 --- a/fs/cachefiles/namei.c +++ b/fs/cachefiles/namei.c @@ -584,6 +584,8 @@ static bool cachefiles_open_file(struct cachefiles_object *object, if (ret < 0) goto check_failed; + clear_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &object->cookie->flags); + object->file = file; /* Always update the atime on an object we've just looked up (this is diff --git a/fs/ceph/cache.c b/fs/ceph/cache.c index 177d8e8d73fe..de1dee46d3df 100644 --- a/fs/ceph/cache.c +++ b/fs/ceph/cache.c @@ -36,6 +36,8 @@ void ceph_fscache_register_inode_cookie(struct inode *inode) &ci->i_vino, sizeof(ci->i_vino), &ci->i_version, sizeof(ci->i_version), i_size_read(inode)); + if (ci->netfs.cache) + mapping_set_release_always(inode->i_mapping); } void ceph_fscache_unregister_inode_cookie(struct ceph_inode_info *ci) diff --git a/fs/cifs/fscache.c b/fs/cifs/fscache.c index f6f3a6b75601..79e9665dfc90 100644 --- a/fs/cifs/fscache.c +++ b/fs/cifs/fscache.c @@ -108,6 +108,8 @@ void cifs_fscache_get_inode_cookie(struct inode *inode) &cifsi->uniqueid, sizeof(cifsi->uniqueid), &cd, sizeof(cd), i_size_read(&cifsi->netfs.inode)); + if (cifsi->netfs.cache) + mapping_set_release_always(inode->i_mapping); } void cifs_fscache_unuse_inode_cookie(struct inode *inode, bool update) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 29e1f9e76eb6..a0d433e0addd 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -199,6 +199,7 @@ enum mapping_flags { /* writeback related tags are not used */ AS_NO_WRITEBACK_TAGS = 5, AS_LARGE_FOLIO_SUPPORT = 6, + AS_RELEASE_ALWAYS, /* Call ->release_folio(), even if no private data */ }; /** @@ -269,6 +270,21 @@ static inline int mapping_use_writeback_tags(struct address_space *mapping) return !test_bit(AS_NO_WRITEBACK_TAGS, &mapping->flags); } +static inline bool mapping_release_always(const struct address_space *mapping) +{ + return test_bit(AS_RELEASE_ALWAYS, &mapping->flags); +} + +static inline void mapping_set_release_always(struct address_space *mapping) +{ + set_bit(AS_RELEASE_ALWAYS, &mapping->flags); +} + +static inline void mapping_clear_release_always(struct address_space *mapping) +{ + clear_bit(AS_RELEASE_ALWAYS, &mapping->flags); +} + static inline gfp_t mapping_gfp_mask(struct address_space * mapping) { return mapping->gfp_mask; diff --git a/mm/internal.h b/mm/internal.h index c4c8e58e1d12..5421ce8661fa 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -168,7 +168,10 @@ static inline void set_page_refcounted(struct page *page) */ static inline bool folio_needs_release(struct folio *folio) { - return folio_has_private(folio); + struct address_space *mapping = folio->mapping; + + return folio_has_private(folio) || + (mapping && mapping_release_always(mapping)); } extern unsigned long highest_memmap_pfn; From patchwork Thu Dec 22 15:02:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13080068 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5ACFC4167B for ; Thu, 22 Dec 2022 15:02:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2F57194000A; Thu, 22 Dec 2022 10:02:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2A45F900002; Thu, 22 Dec 2022 10:02:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 145E994000A; Thu, 22 Dec 2022 10:02:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 03DB0900002 for ; Thu, 22 Dec 2022 10:02:44 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id B632D120E71 for ; Thu, 22 Dec 2022 15:02:43 +0000 (UTC) X-FDA: 80270259006.07.11349A8 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf09.hostedemail.com (Postfix) with ESMTP id 107ED140011 for ; Thu, 22 Dec 2022 15:02:39 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XLRwGmDZ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf09.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671721360; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=75w1pfVKB1Ro0XqkbXmr8eWxoXfk22o+qXL/5b9cucw=; b=DLN3D5VJ56fZ5Lx8FznQYnKUG9ekEq1h1vDNYQZFW1qS2+Aw4y8fpFTxMgyQ1L19ecKiLa sPmqk4U1TuOyp4br3qBWIpnMyg7MtW9q7GP9PUcE9b2hUh297p4fvV0VQWiNKRqHBcFwEY 6IV9uCJXHFlFyrETr5ZHG7ow7Sr2S2g= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XLRwGmDZ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf09.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671721360; a=rsa-sha256; cv=none; b=LEHRSntKkROhhR27hnfcruKZFF3OFbj7mpJ0e9zK4wtudpOE005+XzroH2dMQWcVebwl0T OrtW3GBqlIWGZfcSiJ0XkGxLDAqaG0CRjj1vlgaP7M9XPSOlu9r8Xmfs6XCAGEYdOT3ebI +/w7tehglqgwjxptC50Y5H6OKa9/CfM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1671721359; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=75w1pfVKB1Ro0XqkbXmr8eWxoXfk22o+qXL/5b9cucw=; b=XLRwGmDZVV85fRvVvsUASDYl7VFxM593UFrwdOCx6aZ/dDknwEiIuagJK9gGqzIefbEJdK ggWKfvvBeo4hlBDyrp4qdzreAzufQab2QaG5zoIczq13PotfA5MiOpkPrlB5QXZhvMTCp4 9PX8e9U26OF55MDxspHf6SLGT1znpmI= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-290-HaKXfIxlMWuof_yakpjAKg-1; Thu, 22 Dec 2022 10:02:34 -0500 X-MC-Unique: HaKXfIxlMWuof_yakpjAKg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A1C872803D7E; Thu, 22 Dec 2022 15:02:32 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.96]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4BBDC2166B29; Thu, 22 Dec 2022 15:02:30 +0000 (UTC) Organization: Red Hat UK Ltd. Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SI4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 3798903 Subject: [PATCH v5 3/3] mm: Make filemap_release_folio() better inform shrink_folio_list() From: David Howells To: Matthew Wilcox Cc: Linus Torvalds , Steve French , Shyam Prasad N , Rohith Surabattula , Dave Wysochanski , Dominique Martinet , Ilya Dryomov , linux-cachefs@redhat.com, linux-cifs@vger.kernel.org, linux-afs@lists.infradead.org, v9fs-developer@lists.sourceforge.net, ceph-devel@vger.kernel.org, linux-nfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, dhowells@redhat.com, Linus Torvalds , Jeff Layton , linux-afs@lists.infradead.org, linux-nfs@vger.kernel.org, linux-cifs@vger.kernel.org, ceph-devel@vger.kernel.org, v9fs-developer@lists.sourceforge.net, linux-erofs@lists.ozlabs.org, linux-ext4@lists.ozlabs.org, linux-cachefs@redhat.com, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Date: Thu, 22 Dec 2022 15:02:29 +0000 Message-ID: <167172134962.2334525.570622889806603086.stgit@warthog.procyon.org.uk> In-Reply-To: <167172131368.2334525.8569808925687731937.stgit@warthog.procyon.org.uk> References: <167172131368.2334525.8569808925687731937.stgit@warthog.procyon.org.uk> User-Agent: StGit/1.5 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 107ED140011 X-Stat-Signature: b1zr7i6rcgjksxjatognd54mitjp6bp3 X-HE-Tag: 1671721359-277108 X-HE-Meta: U2FsdGVkX19f5HdHhzB8229qeGRoB9ZwdvkTP2/4kDcBCrdTdVdZQmo+WbOdCBeo2qFhCOBO4aK8o5v1Iec2fVCBKhRZrmcaP3LqKAnBuA4u5kMt1VbjTNA9PwVvQFKhJaGMxkKq/gbzlXFhU6HOem/zbuFYQeADJFS1Xcxo2VwURntfcCNmr8frNGqVRVBenEFxB0Z48mQsDmmwMeszf87L/sPQ+7L8X8oc78WQXw1RC5IHrdex5zTz86w6e+R/Y68Us+dRgSS/Ps1BQ1DSauewMOkIYJ9HfN9Uj3fXhaqyGKydpQjX8oTg0gFt0kgDlJPJgzdngT2e8VjdeVijzSgtpB5OtKqJYqJus7+qj2UPtm+3Qy9nUu9sWPJvnlTuppo9QO3pxyS9Qt2xqmYjN02qO5nQorH+J0eK7Se/s5L2KqKq8WF16gHArZPd/yCUCE+VgS7fJGf7vhyHEQIoldq2ItmxK+A+tT6gaXUQ/eHjBy70Xcm11POPL6GeyXQmcrNfiljSucKQBIdJx02nA7L5s3ddztn6SlBg2ofUr4+ijQC+R8aEWN+Ea9MMaaMJDdRRnRE1qQNq+bxM1gOMePbVlU7MWbw5pRedNNAmA3z/HvXZSvd+nGEA248kXeEHnwoD5n0sth8GdwaWmecM+MnCWkCZmhGEFTp+kQ6r+E1wPXA2fFK7PWLQTBySzwvQletWAfNRWUnJNZJ62/A/Ot79LdYu7a42y+HL/trkF9RSfsbJPPQcn2G+ynJUZzWz3jrvq3l06QKAobCkp2DH3duURrsDTzpIjr0hf8x9edw3J8Zlm+jXdaKSWkurpMyqpZia4j+OZNWnTQsUl1DDaOVObS4iHwjDMqLDWE6ySpKXp20fPNh4MU8/PLNexmMJAXIClS9KvGhFfcy++sSkfL0i6+6krKmkNj4Zuku7lAy705ZAS1/+LWEkl38PObKBh2LGfIGy83IkEnzzVvO GyGf2S9t MlVGbVlDtzzVeLnha6Mxz5sIhCw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make filemap_release_folio() return one of three values: (0) FILEMAP_CANT_RELEASE_FOLIO Couldn't release the folio's private data, so the folio can't itself be released. (1) FILEMAP_RELEASED_FOLIO The private data on the folio was released and the folio can be released. (2) FILEMAP_FOLIO_HAD_NO_PRIVATE There was no private data on the folio and the folio can be released. The first must be zero so that existing tests of !filemap_release_folio() continue to work as expected; similarly the other two must both be non-zero so that existing tests of filemap_release_folio() continue to work as expected. Using this, make shrink_folio_list() choose which of three cases to follow based on the return from filemap_release_folio() rather than testing the folio's private bit itself. Signed-off-by: David Howells cc: Matthew Wilcox cc: Linus Torvalds cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Dave Wysochanski cc: Dominique Martinet cc: Ilya Dryomov cc: linux-cachefs@redhat.com cc: linux-cifs@vger.kernel.org cc: linux-afs@lists.infradead.org cc: v9fs-developer@lists.sourceforge.net cc: ceph-devel@vger.kernel.org cc: linux-nfs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/1459152.1669208550@warthog.procyon.org.uk/ # v3 Link: https://lore.kernel.org/r/166924373637.1772793.2622483388224911574.stgit@warthog.procyon.org.uk/ # v4 --- include/linux/pagemap.h | 7 ++++++- mm/filemap.c | 20 ++++++++++++++------ mm/vmscan.c | 29 +++++++++++++++-------------- 3 files changed, 35 insertions(+), 21 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index a0d433e0addd..cd00fb3b524b 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -1121,7 +1121,12 @@ void __filemap_remove_folio(struct folio *folio, void *shadow); void replace_page_cache_folio(struct folio *old, struct folio *new); void delete_from_page_cache_batch(struct address_space *mapping, struct folio_batch *fbatch); -bool filemap_release_folio(struct folio *folio, gfp_t gfp); +enum filemap_released_folio { + FILEMAP_CANT_RELEASE_FOLIO = 0, /* (This must be 0) Release failed */ + FILEMAP_RELEASED_FOLIO = 1, /* Folio's private data released */ + FILEMAP_FOLIO_HAD_NO_PRIVATE = 2, /* Folio had no private data */ +}; +enum filemap_released_folio filemap_release_folio(struct folio *folio, gfp_t gfp); loff_t mapping_seek_hole_data(struct address_space *, loff_t start, loff_t end, int whence); diff --git a/mm/filemap.c b/mm/filemap.c index 344146c170b0..217ca847773a 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3953,20 +3953,28 @@ EXPORT_SYMBOL(generic_file_write_iter); * this page (__GFP_IO), and whether the call may block * (__GFP_RECLAIM & __GFP_FS). * - * Return: %true if the release was successful, otherwise %false. + * Return: %FILEMAP_RELEASED_FOLIO if the release was successful, + * %FILEMAP_CANT_RELEASE_FOLIO if the private data couldn't be released and + * %FILEMAP_FOLIO_HAD_NO_PRIVATE if there was no private data. */ -bool filemap_release_folio(struct folio *folio, gfp_t gfp) +enum filemap_released_folio filemap_release_folio(struct folio *folio, + gfp_t gfp) { struct address_space * const mapping = folio->mapping; + bool released; BUG_ON(!folio_test_locked(folio)); if (!folio_needs_release(folio)) - return true; + return FILEMAP_FOLIO_HAD_NO_PRIVATE; if (folio_test_writeback(folio)) - return false; + return FILEMAP_CANT_RELEASE_FOLIO; if (mapping && mapping->a_ops->release_folio) - return mapping->a_ops->release_folio(folio, gfp); - return try_to_free_buffers(folio); + released = mapping->a_ops->release_folio(folio, gfp); + else + released = try_to_free_buffers(folio); + + return released ? + FILEMAP_RELEASED_FOLIO : FILEMAP_CANT_RELEASE_FOLIO; } EXPORT_SYMBOL(filemap_release_folio); diff --git a/mm/vmscan.c b/mm/vmscan.c index bded71961143..b1e5ca348223 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1996,25 +1996,26 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, * (refcount == 1) it can be freed. Otherwise, leave * the folio on the LRU so it is swappable. */ - if (folio_needs_release(folio)) { - if (!filemap_release_folio(folio, sc->gfp_mask)) - goto activate_locked; + switch (filemap_release_folio(folio, sc->gfp_mask)) { + case FILEMAP_CANT_RELEASE_FOLIO: + goto activate_locked; + case FILEMAP_RELEASED_FOLIO: if (!mapping && folio_ref_count(folio) == 1) { folio_unlock(folio); if (folio_put_testzero(folio)) goto free_it; - else { - /* - * rare race with speculative reference. - * the speculative reference will free - * this folio shortly, so we may - * increment nr_reclaimed here (and - * leave it off the LRU). - */ - nr_reclaimed += nr_pages; - continue; - } + /* + * rare race with speculative reference. the + * speculative reference will free this folio + * shortly, so we may increment nr_reclaimed + * here (and leave it off the LRU). + */ + nr_reclaimed += nr_pages; + continue; } + break; + case FILEMAP_FOLIO_HAD_NO_PRIVATE: + break; } if (folio_test_anon(folio) && !folio_test_swapbacked(folio)) {