From patchwork Wed Apr 5 20:59:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna Schumaker X-Patchwork-Id: 13202541 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93044C76188 for ; Wed, 5 Apr 2023 20:59:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231440AbjDEU70 (ORCPT ); Wed, 5 Apr 2023 16:59:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229484AbjDEU7X (ORCPT ); Wed, 5 Apr 2023 16:59:23 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B98DC5583 for ; Wed, 5 Apr 2023 13:59:21 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 443FF6363B for ; Wed, 5 Apr 2023 20:59:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5F633C433D2; Wed, 5 Apr 2023 20:59:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1680728360; bh=zVLjjOW7dxBEZSWUTdqq/WPt6w2yYT/c1QCizLHO+f4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cMizTafECGwnTJqZ+HSDmBjS9Z5ago4tlopxHVZ/eMRjM+lpRUwHQ23Q6DmzM0sE3 VYJy1itlhcxEnYMDDRsaXq3K8TOaPw/Xf8gG2kos20rEtkd+wbMzlUm9MdmVWPie0q 7pkSPtZUvn1zEz8jghK+X3M2bn/Bn+kC1MH28L+ZCPjmXhauTMPGrl2bY2CMD9n3XT FtdKi/rheNuheH6XqSbngrMF/jOw31kQMzlhae5fBKAuhW7+CR0w0ebZGH0+j0rf87 /b3xKiTfXzX0/uCnSUF6HGc8487Qoryop78yw/oxsfC4CPYKLXn6ujv9eaQTCLGfL2 nitFHzEkMZ3Jg== From: Anna Schumaker To: linux-nfs@vger.kernel.org, trond.myklebust@hammerspace.com Cc: anna@kernel.org Subject: [PATCH 1/2] NFS: Convert the readdir array-of-pages into an array-of-folios Date: Wed, 5 Apr 2023 16:59:17 -0400 Message-Id: <20230405205918.228394-2-anna@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230405205918.228394-1-anna@kernel.org> References: <20230405205918.228394-1-anna@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Anna Schumaker This patch only converts the actual array, but doesn't touch the individual nfs_cache_array pages and related functions (that will be done in the next patch). I also adjust the names of the fields in the nfs_readdir_descriptor to say "folio" instead of "page". Signed-off-by: Anna Schumaker --- fs/nfs/dir.c | 129 ++++++++++++++++++++++++++------------------------- 1 file changed, 65 insertions(+), 64 deletions(-) diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index 6fbcbb8d6587..9edb0584c6d1 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -154,10 +154,10 @@ struct nfs_cache_array { struct nfs_readdir_descriptor { struct file *file; - struct page *page; + struct folio *folio; struct dir_context *ctx; - pgoff_t page_index; - pgoff_t page_index_max; + pgoff_t folio_index; + pgoff_t folio_index_max; u64 dir_cookie; u64 last_cookie; loff_t current_index; @@ -312,8 +312,8 @@ static int nfs_readdir_array_can_expand(struct nfs_cache_array *array) } static int nfs_readdir_page_array_append(struct page *page, - const struct nfs_entry *entry, - u64 *cookie) + const struct nfs_entry *entry, + u64 *cookie) { struct nfs_cache_array *array; struct nfs_cache_array_entry *cache_entry; @@ -485,7 +485,7 @@ static void nfs_readdir_seek_next_array(struct nfs_cache_array *array, desc->last_cookie = array->last_cookie; desc->current_index += array->size; desc->cache_entry_index = 0; - desc->page_index++; + desc->folio_index++; } else desc->last_cookie = nfs_readdir_array_index_cookie(array); } @@ -494,7 +494,7 @@ static void nfs_readdir_rewind_search(struct nfs_readdir_descriptor *desc) { desc->current_index = 0; desc->last_cookie = 0; - desc->page_index = 0; + desc->folio_index = 0; } static int nfs_readdir_search_for_pos(struct nfs_cache_array *array, @@ -568,7 +568,7 @@ static int nfs_readdir_search_array(struct nfs_readdir_descriptor *desc) struct nfs_cache_array *array; int status; - array = kmap_local_page(desc->page); + array = kmap_local_folio(desc->folio, 0); if (desc->dir_cookie == 0) status = nfs_readdir_search_for_pos(array, desc); @@ -819,16 +819,17 @@ static int nfs_readdir_entry_decode(struct nfs_readdir_descriptor *desc, } /* Perform conversion from xdr to cache array */ -static int nfs_readdir_page_filler(struct nfs_readdir_descriptor *desc, - struct nfs_entry *entry, - struct page **xdr_pages, unsigned int buflen, - struct page **arrays, size_t narrays, - u64 change_attr) +static int nfs_readdir_folio_filler(struct nfs_readdir_descriptor *desc, + struct nfs_entry *entry, + struct page **xdr_pages, unsigned int buflen, + struct folio **arrays, size_t narrays, + u64 change_attr) { struct address_space *mapping = desc->file->f_mapping; + struct folio *folio = *arrays; struct xdr_stream stream; + struct page *scratch, *new; struct xdr_buf buf; - struct page *scratch, *new, *page = *arrays; u64 cookie; int status; @@ -844,36 +845,36 @@ static int nfs_readdir_page_filler(struct nfs_readdir_descriptor *desc, if (status != 0) break; - status = nfs_readdir_page_array_append(page, entry, &cookie); + status = nfs_readdir_page_array_append(folio_page(folio, 0), entry, &cookie); if (status != -ENOSPC) continue; - if (page->mapping != mapping) { + if (folio->mapping != mapping) { if (!--narrays) break; new = nfs_readdir_page_array_alloc(cookie, GFP_KERNEL); if (!new) break; arrays++; - *arrays = page = new; + *arrays = folio = page_folio(new); } else { new = nfs_readdir_page_get_next(mapping, cookie, change_attr); if (!new) break; - if (page != *arrays) - nfs_readdir_page_unlock_and_put(page); - page = new; + if (folio != *arrays) + nfs_readdir_page_unlock_and_put(folio_page(folio, 0)); + folio = page_folio(new); } - desc->page_index_max++; - status = nfs_readdir_page_array_append(page, entry, &cookie); + desc->folio_index_max++; + status = nfs_readdir_page_array_append(folio_page(folio, 0), entry, &cookie); } while (!status && !entry->eof); switch (status) { case -EBADCOOKIE: if (!entry->eof) break; - nfs_readdir_page_set_eof(page); + nfs_readdir_page_set_eof(folio_page(folio, 0)); fallthrough; case -EAGAIN: status = 0; @@ -886,8 +887,8 @@ static int nfs_readdir_page_filler(struct nfs_readdir_descriptor *desc, ; } - if (page != *arrays) - nfs_readdir_page_unlock_and_put(page); + if (folio != *arrays) + nfs_readdir_page_unlock_and_put(folio_page(folio, 0)); put_page(scratch); return status; @@ -927,11 +928,11 @@ static struct page **nfs_readdir_alloc_pages(size_t npages) static int nfs_readdir_xdr_to_array(struct nfs_readdir_descriptor *desc, __be32 *verf_arg, __be32 *verf_res, - struct page **arrays, size_t narrays) + struct folio **arrays, size_t narrays) { u64 change_attr; struct page **pages; - struct page *page = *arrays; + struct folio *folio = *arrays; struct nfs_entry *entry; size_t array_size; struct inode *inode = file_inode(desc->file); @@ -942,7 +943,7 @@ static int nfs_readdir_xdr_to_array(struct nfs_readdir_descriptor *desc, entry = kzalloc(sizeof(*entry), GFP_KERNEL); if (!entry) return -ENOMEM; - entry->cookie = nfs_readdir_page_last_cookie(page); + entry->cookie = nfs_readdir_page_last_cookie(folio_page(folio, 0)); entry->fh = nfs_alloc_fhandle(); entry->fattr = nfs_alloc_fattr_with_label(NFS_SERVER(inode)); entry->server = NFS_SERVER(inode); @@ -962,10 +963,10 @@ static int nfs_readdir_xdr_to_array(struct nfs_readdir_descriptor *desc, pglen = status; if (pglen != 0) - status = nfs_readdir_page_filler(desc, entry, pages, pglen, - arrays, narrays, change_attr); + status = nfs_readdir_folio_filler(desc, entry, pages, pglen, + arrays, narrays, change_attr); else - nfs_readdir_page_set_eof(page); + nfs_readdir_page_set_eof(folio_page(folio, 0)); desc->buffer_fills++; free_pages: @@ -977,21 +978,21 @@ static int nfs_readdir_xdr_to_array(struct nfs_readdir_descriptor *desc, return status; } -static void nfs_readdir_page_put(struct nfs_readdir_descriptor *desc) +static void nfs_readdir_folio_put(struct nfs_readdir_descriptor *desc) { - put_page(desc->page); - desc->page = NULL; + folio_put(desc->folio); + desc->folio = NULL; } static void -nfs_readdir_page_unlock_and_put_cached(struct nfs_readdir_descriptor *desc) +nfs_readdir_folio_unlock_and_put_cached(struct nfs_readdir_descriptor *desc) { - unlock_page(desc->page); - nfs_readdir_page_put(desc); + folio_unlock(desc->folio); + nfs_readdir_folio_put(desc); } -static struct page * -nfs_readdir_page_get_cached(struct nfs_readdir_descriptor *desc) +static struct folio * +nfs_readdir_folio_get_cached(struct nfs_readdir_descriptor *desc) { struct address_space *mapping = desc->file->f_mapping; u64 change_attr = inode_peek_iversion_raw(mapping->host); @@ -1003,7 +1004,7 @@ nfs_readdir_page_get_cached(struct nfs_readdir_descriptor *desc) return NULL; if (desc->clear_cache && !nfs_readdir_page_needs_filling(page)) nfs_readdir_page_reinit_array(page, cookie, change_attr); - return page; + return page_folio(page); } /* @@ -1017,21 +1018,21 @@ static int find_and_lock_cache_page(struct nfs_readdir_descriptor *desc) __be32 verf[NFS_DIR_VERIFIER_SIZE]; int res; - desc->page = nfs_readdir_page_get_cached(desc); - if (!desc->page) + desc->folio = nfs_readdir_folio_get_cached(desc); + if (!desc->folio) return -ENOMEM; - if (nfs_readdir_page_needs_filling(desc->page)) { + if (nfs_readdir_page_needs_filling(folio_page(desc->folio, 0))) { /* Grow the dtsize if we had to go back for more pages */ - if (desc->page_index == desc->page_index_max) + if (desc->folio_index == desc->folio_index_max) nfs_grow_dtsize(desc); - desc->page_index_max = desc->page_index; + desc->folio_index_max = desc->folio_index; trace_nfs_readdir_cache_fill(desc->file, nfsi->cookieverf, desc->last_cookie, - desc->page->index, desc->dtsize); + desc->folio->index, desc->dtsize); res = nfs_readdir_xdr_to_array(desc, nfsi->cookieverf, verf, - &desc->page, 1); + &desc->folio, 1); if (res < 0) { - nfs_readdir_page_unlock_and_put_cached(desc); + nfs_readdir_folio_unlock_and_put_cached(desc); trace_nfs_readdir_cache_fill_done(inode, res); if (res == -EBADCOOKIE || res == -ENOTSYNC) { invalidate_inode_pages2(desc->file->f_mapping); @@ -1059,7 +1060,7 @@ static int find_and_lock_cache_page(struct nfs_readdir_descriptor *desc) res = nfs_readdir_search_array(desc); if (res == 0) return 0; - nfs_readdir_page_unlock_and_put_cached(desc); + nfs_readdir_folio_unlock_and_put_cached(desc); return res; } @@ -1087,7 +1088,7 @@ static void nfs_do_filldir(struct nfs_readdir_descriptor *desc, unsigned int i; bool first_emit = !desc->dir_cookie; - array = kmap_local_page(desc->page); + array = kmap_local_folio(desc->folio, 0); for (i = desc->cache_entry_index; i < array->size; i++) { struct nfs_cache_array_entry *ent; @@ -1136,7 +1137,7 @@ static void nfs_do_filldir(struct nfs_readdir_descriptor *desc, */ static int uncached_readdir(struct nfs_readdir_descriptor *desc) { - struct page **arrays; + struct folio **arrays; size_t i, sz = 512; __be32 verf[NFS_DIR_VERIFIER_SIZE]; int status = -ENOMEM; @@ -1147,14 +1148,14 @@ static int uncached_readdir(struct nfs_readdir_descriptor *desc) arrays = kcalloc(sz, sizeof(*arrays), GFP_KERNEL); if (!arrays) goto out; - arrays[0] = nfs_readdir_page_array_alloc(desc->dir_cookie, GFP_KERNEL); + arrays[0] = page_folio(nfs_readdir_page_array_alloc(desc->dir_cookie, GFP_KERNEL)); if (!arrays[0]) goto out; - desc->page_index = 0; + desc->folio_index = 0; desc->cache_entry_index = 0; desc->last_cookie = desc->dir_cookie; - desc->page_index_max = 0; + desc->folio_index_max = 0; trace_nfs_readdir_uncached(desc->file, desc->verf, desc->last_cookie, -1, desc->dtsize); @@ -1166,10 +1167,10 @@ static int uncached_readdir(struct nfs_readdir_descriptor *desc) } for (i = 0; !desc->eob && i < sz && arrays[i]; i++) { - desc->page = arrays[i]; + desc->folio = arrays[i]; nfs_do_filldir(desc, verf); } - desc->page = NULL; + desc->folio = NULL; /* * Grow the dtsize if we have to go back for more pages, @@ -1179,16 +1180,16 @@ static int uncached_readdir(struct nfs_readdir_descriptor *desc) if (!desc->eob) nfs_grow_dtsize(desc); else if (desc->buffer_fills == 1 && - i < (desc->page_index_max >> 1)) + i < (desc->folio_index_max >> 1)) nfs_shrink_dtsize(desc); } out_free: for (i = 0; i < sz && arrays[i]; i++) - nfs_readdir_page_array_free(arrays[i]); + nfs_readdir_page_array_free(folio_page(arrays[i], 0)); out: if (!nfs_readdir_use_cookie(desc->file)) nfs_readdir_rewind_search(desc); - desc->page_index_max = -1; + desc->folio_index_max = -1; kfree(arrays); dfprintk(DIRCACHE, "NFS: %s: returns %d\n", __func__, status); return status; @@ -1240,11 +1241,11 @@ static int nfs_readdir(struct file *file, struct dir_context *ctx) goto out; desc->file = file; desc->ctx = ctx; - desc->page_index_max = -1; + desc->folio_index_max = -1; spin_lock(&file->f_lock); desc->dir_cookie = dir_ctx->dir_cookie; - desc->page_index = dir_ctx->page_index; + desc->folio_index = dir_ctx->page_index; desc->last_cookie = dir_ctx->last_cookie; desc->attr_gencount = dir_ctx->attr_gencount; desc->eof = dir_ctx->eof; @@ -1291,8 +1292,8 @@ static int nfs_readdir(struct file *file, struct dir_context *ctx) break; nfs_do_filldir(desc, nfsi->cookieverf); - nfs_readdir_page_unlock_and_put_cached(desc); - if (desc->page_index == desc->page_index_max) + nfs_readdir_folio_unlock_and_put_cached(desc); + if (desc->folio_index == desc->folio_index_max) desc->clear_cache = force_clear; } while (!desc->eob && !desc->eof); @@ -1300,7 +1301,7 @@ static int nfs_readdir(struct file *file, struct dir_context *ctx) dir_ctx->dir_cookie = desc->dir_cookie; dir_ctx->last_cookie = desc->last_cookie; dir_ctx->attr_gencount = desc->attr_gencount; - dir_ctx->page_index = desc->page_index; + dir_ctx->page_index = desc->folio_index; dir_ctx->force_clear = force_clear; dir_ctx->eof = desc->eof; dir_ctx->dtsize = desc->dtsize; From patchwork Wed Apr 5 20:59:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna Schumaker X-Patchwork-Id: 13202542 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC295C761AF for ; Wed, 5 Apr 2023 20:59:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229484AbjDEU71 (ORCPT ); Wed, 5 Apr 2023 16:59:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50310 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229969AbjDEU7Y (ORCPT ); Wed, 5 Apr 2023 16:59:24 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C97946B0 for ; Wed, 5 Apr 2023 13:59:22 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CEB1A62AAE for ; Wed, 5 Apr 2023 20:59:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DF507C433EF; Wed, 5 Apr 2023 20:59:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1680728361; bh=8QaW/kA7+fFuKxFnrs2AehaA00VFh7bJjhws3eC2FFI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oQbQ03uE1hJmPCyazdSxFlqULzgI43Oq+jovMJAfZhq0J83k3+gjQF4HfB8qzytIN mghLme80HCMjX8/GfEifKSulgZAg6siel6hALpBYJ7x2EoEaJLX63btQGCk65RQjqX n8m7ucdCeQN8GmniX2ou+vkTfe/pputHHgnPx1HYghgZGG6AfUbra0p63IvEO7vm+R HeujemBa5wqoJTCUPdX16EK67cdlgvi/mn8am7amQxL3lO+xKeSZN//2p/l+KwuYD8 rQac0b2F9+57XeIT8yibf5z6ISAMJ5gAKu44HyDPOxjeuFqBYJ8UJJuKbhHEXmgrsC IVBRkUjt/+bWQ== From: Anna Schumaker To: linux-nfs@vger.kernel.org, trond.myklebust@hammerspace.com Cc: anna@kernel.org Subject: [PATCH 2/2] NFS: Convert readdir page array functions to use a folio Date: Wed, 5 Apr 2023 16:59:18 -0400 Message-Id: <20230405205918.228394-3-anna@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230405205918.228394-1-anna@kernel.org> References: <20230405205918.228394-1-anna@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Anna Schumaker Signed-off-by: Anna Schumaker --- fs/nfs/dir.c | 201 +++++++++++++++++++++++++-------------------------- 1 file changed, 99 insertions(+), 102 deletions(-) diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index 9edb0584c6d1..8257de6dba45 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -55,7 +55,7 @@ static int nfs_closedir(struct inode *, struct file *); static int nfs_readdir(struct file *, struct dir_context *); static int nfs_fsync_dir(struct file *, loff_t, loff_t, int); static loff_t nfs_llseek_dir(struct file *, loff_t, int); -static void nfs_readdir_free_folio(struct folio *); +static void nfs_readdir_clear_array(struct folio *); const struct file_operations nfs_dir_operations = { .llseek = nfs_llseek_dir, @@ -67,7 +67,7 @@ const struct file_operations nfs_dir_operations = { }; const struct address_space_operations nfs_dir_aops = { - .free_folio = nfs_readdir_free_folio, + .free_folio = nfs_readdir_clear_array, }; #define NFS_INIT_DTSIZE PAGE_SIZE @@ -146,8 +146,8 @@ struct nfs_cache_array { u64 change_attr; u64 last_cookie; unsigned int size; - unsigned char page_full : 1, - page_is_eof : 1, + unsigned char folio_full : 1, + folio_is_eof : 1, cookies_are_ordered : 1; struct nfs_cache_array_entry array[]; }; @@ -198,17 +198,17 @@ static void nfs_grow_dtsize(struct nfs_readdir_descriptor *desc) nfs_set_dtsize(desc, desc->dtsize << 1); } -static void nfs_readdir_page_init_array(struct page *page, u64 last_cookie, - u64 change_attr) +static void nfs_readdir_folio_init_array(struct folio *folio, u64 last_cookie, + u64 change_attr) { struct nfs_cache_array *array; - array = kmap_local_page(page); + array = kmap_local_folio(folio, 0); array->change_attr = change_attr; array->last_cookie = last_cookie; array->size = 0; - array->page_full = 0; - array->page_is_eof = 0; + array->folio_full = 0; + array->folio_is_eof = 0; array->cookies_are_ordered = 1; kunmap_local(array); } @@ -216,44 +216,39 @@ static void nfs_readdir_page_init_array(struct page *page, u64 last_cookie, /* * we are freeing strings created by nfs_add_to_readdir_array() */ -static void nfs_readdir_clear_array(struct page *page) +static void nfs_readdir_clear_array(struct folio *folio) { struct nfs_cache_array *array; unsigned int i; - array = kmap_local_page(page); + array = kmap_local_folio(folio, 0); for (i = 0; i < array->size; i++) kfree(array->array[i].name); array->size = 0; kunmap_local(array); } -static void nfs_readdir_free_folio(struct folio *folio) +static void nfs_readdir_folio_reinit_array(struct folio *folio, u64 last_cookie, + u64 change_attr) { - nfs_readdir_clear_array(&folio->page); + nfs_readdir_clear_array(folio); + nfs_readdir_folio_init_array(folio, last_cookie, change_attr); } -static void nfs_readdir_page_reinit_array(struct page *page, u64 last_cookie, - u64 change_attr) +static struct folio * +nfs_readdir_folio_array_alloc(u64 last_cookie, gfp_t gfp_flags) { - nfs_readdir_clear_array(page); - nfs_readdir_page_init_array(page, last_cookie, change_attr); + struct folio *folio = folio_alloc(gfp_flags, 0); + if (folio) + nfs_readdir_folio_init_array(folio, last_cookie, 0); + return folio; } -static struct page * -nfs_readdir_page_array_alloc(u64 last_cookie, gfp_t gfp_flags) +static void nfs_readdir_folio_array_free(struct folio *folio) { - struct page *page = alloc_page(gfp_flags); - if (page) - nfs_readdir_page_init_array(page, last_cookie, 0); - return page; -} - -static void nfs_readdir_page_array_free(struct page *page) -{ - if (page) { - nfs_readdir_clear_array(page); - put_page(page); + if (folio) { + nfs_readdir_clear_array(folio); + folio_put(folio); } } @@ -264,13 +259,13 @@ static u64 nfs_readdir_array_index_cookie(struct nfs_cache_array *array) static void nfs_readdir_array_set_eof(struct nfs_cache_array *array) { - array->page_is_eof = 1; - array->page_full = 1; + array->folio_is_eof = 1; + array->folio_full = 1; } static bool nfs_readdir_array_is_full(struct nfs_cache_array *array) { - return array->page_full; + return array->folio_full; } /* @@ -302,16 +297,16 @@ static size_t nfs_readdir_array_maxentries(void) */ static int nfs_readdir_array_can_expand(struct nfs_cache_array *array) { - if (array->page_full) + if (array->folio_full) return -ENOSPC; if (array->size == nfs_readdir_array_maxentries()) { - array->page_full = 1; + array->folio_full = 1; return -ENOSPC; } return 0; } -static int nfs_readdir_page_array_append(struct page *page, +static int nfs_readdir_folio_array_append(struct folio *folio, const struct nfs_entry *entry, u64 *cookie) { @@ -322,7 +317,7 @@ static int nfs_readdir_page_array_append(struct page *page, name = nfs_readdir_copy_name(entry->name, entry->len); - array = kmap_atomic(page); + array = kmap_atomic(folio_page(folio, 0)); if (!name) goto out; ret = nfs_readdir_array_can_expand(array); @@ -361,17 +356,17 @@ static int nfs_readdir_page_array_append(struct page *page, * 127 readdir entries for a typical 64-bit system, that works out to a * cache of ~ 33 million entries per directory. */ -static pgoff_t nfs_readdir_page_cookie_hash(u64 cookie) +static pgoff_t nfs_readdir_folio_cookie_hash(u64 cookie) { if (cookie == 0) return 0; return hash_64(cookie, 18); } -static bool nfs_readdir_page_validate(struct page *page, u64 last_cookie, - u64 change_attr) +static bool nfs_readdir_folio_validate(struct folio *folio, u64 last_cookie, + u64 change_attr) { - struct nfs_cache_array *array = kmap_local_page(page); + struct nfs_cache_array *array = kmap_local_folio(folio, 0); int ret = true; if (array->change_attr != change_attr) @@ -382,81 +377,83 @@ static bool nfs_readdir_page_validate(struct page *page, u64 last_cookie, return ret; } -static void nfs_readdir_page_unlock_and_put(struct page *page) +static void nfs_readdir_folio_unlock_and_put(struct folio *folio) { - unlock_page(page); - put_page(page); + folio_unlock(folio); + folio_put(folio); } -static void nfs_readdir_page_init_and_validate(struct page *page, u64 cookie, - u64 change_attr) +static void nfs_readdir_folio_init_and_validate(struct folio *folio, u64 cookie, + u64 change_attr) { - if (PageUptodate(page)) { - if (nfs_readdir_page_validate(page, cookie, change_attr)) + if (folio_test_uptodate(folio)) { + if (nfs_readdir_folio_validate(folio, cookie, change_attr)) return; - nfs_readdir_clear_array(page); + nfs_readdir_clear_array(folio); } - nfs_readdir_page_init_array(page, cookie, change_attr); - SetPageUptodate(page); + nfs_readdir_folio_init_array(folio, cookie, change_attr); + folio_mark_uptodate(folio); } -static struct page *nfs_readdir_page_get_locked(struct address_space *mapping, - u64 cookie, u64 change_attr) +static struct folio *nfs_readdir_folio_get_locked(struct address_space *mapping, + u64 cookie, u64 change_attr) { - pgoff_t index = nfs_readdir_page_cookie_hash(cookie); - struct page *page; + pgoff_t index = nfs_readdir_folio_cookie_hash(cookie); + struct folio *folio; - page = grab_cache_page(mapping, index); - if (!page) + folio = filemap_grab_folio(mapping, index); + if (!folio) return NULL; - nfs_readdir_page_init_and_validate(page, cookie, change_attr); - return page; + nfs_readdir_folio_init_and_validate(folio, cookie, change_attr); + return folio; } -static u64 nfs_readdir_page_last_cookie(struct page *page) +static u64 nfs_readdir_folio_last_cookie(struct folio *folio) { struct nfs_cache_array *array; u64 ret; - array = kmap_local_page(page); + array = kmap_local_folio(folio, 0); ret = array->last_cookie; kunmap_local(array); return ret; } -static bool nfs_readdir_page_needs_filling(struct page *page) +static bool nfs_readdir_folio_needs_filling(struct folio *folio) { struct nfs_cache_array *array; bool ret; - array = kmap_local_page(page); + array = kmap_local_folio(folio, 0); ret = !nfs_readdir_array_is_full(array); kunmap_local(array); return ret; } -static void nfs_readdir_page_set_eof(struct page *page) +static void nfs_readdir_folio_set_eof(struct folio *folio) { struct nfs_cache_array *array; - array = kmap_local_page(page); + array = kmap_local_folio(folio, 0); nfs_readdir_array_set_eof(array); kunmap_local(array); } -static struct page *nfs_readdir_page_get_next(struct address_space *mapping, - u64 cookie, u64 change_attr) +static struct folio *nfs_readdir_folio_get_next(struct address_space *mapping, + u64 cookie, u64 change_attr) { - pgoff_t index = nfs_readdir_page_cookie_hash(cookie); - struct page *page; + pgoff_t index = nfs_readdir_folio_cookie_hash(cookie); + struct folio *folio; - page = grab_cache_page_nowait(mapping, index); - if (!page) + folio = __filemap_get_folio(mapping, index, + FGP_LOCK|FGP_CREAT|FGP_NOFS|FGP_NOWAIT, + mapping_gfp_mask(mapping)); + if (!folio) return NULL; - nfs_readdir_page_init_and_validate(page, cookie, change_attr); - if (nfs_readdir_page_last_cookie(page) != cookie) - nfs_readdir_page_reinit_array(page, cookie, change_attr); - return page; + nfs_readdir_folio_init_and_validate(folio, cookie, change_attr); + if (nfs_readdir_folio_last_cookie(folio) != cookie) + nfs_readdir_folio_reinit_array(folio, cookie, change_attr); + return folio; } static inline @@ -481,7 +478,7 @@ bool nfs_readdir_use_cookie(const struct file *filp) static void nfs_readdir_seek_next_array(struct nfs_cache_array *array, struct nfs_readdir_descriptor *desc) { - if (array->page_full) { + if (array->folio_full) { desc->last_cookie = array->last_cookie; desc->current_index += array->size; desc->cache_entry_index = 0; @@ -506,7 +503,7 @@ static int nfs_readdir_search_for_pos(struct nfs_cache_array *array, if (diff < 0) goto out_eof; if (diff >= array->size) { - if (array->page_is_eof) + if (array->folio_is_eof) goto out_eof; nfs_readdir_seek_next_array(array, desc); return -EAGAIN; @@ -554,7 +551,7 @@ static int nfs_readdir_search_for_cookie(struct nfs_cache_array *array, } } check_eof: - if (array->page_is_eof) { + if (array->folio_is_eof) { status = -EBADCOOKIE; if (desc->dir_cookie == array->last_cookie) desc->eof = true; @@ -826,9 +823,9 @@ static int nfs_readdir_folio_filler(struct nfs_readdir_descriptor *desc, u64 change_attr) { struct address_space *mapping = desc->file->f_mapping; - struct folio *folio = *arrays; + struct folio *new, *folio = *arrays; struct xdr_stream stream; - struct page *scratch, *new; + struct page *scratch; struct xdr_buf buf; u64 cookie; int status; @@ -845,36 +842,36 @@ static int nfs_readdir_folio_filler(struct nfs_readdir_descriptor *desc, if (status != 0) break; - status = nfs_readdir_page_array_append(folio_page(folio, 0), entry, &cookie); + status = nfs_readdir_folio_array_append(folio, entry, &cookie); if (status != -ENOSPC) continue; if (folio->mapping != mapping) { if (!--narrays) break; - new = nfs_readdir_page_array_alloc(cookie, GFP_KERNEL); + new = nfs_readdir_folio_array_alloc(cookie, GFP_KERNEL); if (!new) break; arrays++; - *arrays = folio = page_folio(new); + *arrays = folio = new; } else { - new = nfs_readdir_page_get_next(mapping, cookie, - change_attr); + new = nfs_readdir_folio_get_next(mapping, cookie, + change_attr); if (!new) break; if (folio != *arrays) - nfs_readdir_page_unlock_and_put(folio_page(folio, 0)); - folio = page_folio(new); + nfs_readdir_folio_unlock_and_put(folio); + folio = new; } desc->folio_index_max++; - status = nfs_readdir_page_array_append(folio_page(folio, 0), entry, &cookie); + status = nfs_readdir_folio_array_append(folio, entry, &cookie); } while (!status && !entry->eof); switch (status) { case -EBADCOOKIE: if (!entry->eof) break; - nfs_readdir_page_set_eof(folio_page(folio, 0)); + nfs_readdir_folio_set_eof(folio); fallthrough; case -EAGAIN: status = 0; @@ -888,7 +885,7 @@ static int nfs_readdir_folio_filler(struct nfs_readdir_descriptor *desc, } if (folio != *arrays) - nfs_readdir_page_unlock_and_put(folio_page(folio, 0)); + nfs_readdir_folio_unlock_and_put(folio); put_page(scratch); return status; @@ -943,7 +940,7 @@ static int nfs_readdir_xdr_to_array(struct nfs_readdir_descriptor *desc, entry = kzalloc(sizeof(*entry), GFP_KERNEL); if (!entry) return -ENOMEM; - entry->cookie = nfs_readdir_page_last_cookie(folio_page(folio, 0)); + entry->cookie = nfs_readdir_folio_last_cookie(folio); entry->fh = nfs_alloc_fhandle(); entry->fattr = nfs_alloc_fattr_with_label(NFS_SERVER(inode)); entry->server = NFS_SERVER(inode); @@ -966,7 +963,7 @@ static int nfs_readdir_xdr_to_array(struct nfs_readdir_descriptor *desc, status = nfs_readdir_folio_filler(desc, entry, pages, pglen, arrays, narrays, change_attr); else - nfs_readdir_page_set_eof(folio_page(folio, 0)); + nfs_readdir_folio_set_eof(folio); desc->buffer_fills++; free_pages: @@ -997,14 +994,14 @@ nfs_readdir_folio_get_cached(struct nfs_readdir_descriptor *desc) struct address_space *mapping = desc->file->f_mapping; u64 change_attr = inode_peek_iversion_raw(mapping->host); u64 cookie = desc->last_cookie; - struct page *page; + struct folio *folio; - page = nfs_readdir_page_get_locked(mapping, cookie, change_attr); - if (!page) + folio = nfs_readdir_folio_get_locked(mapping, cookie, change_attr); + if (!folio) return NULL; - if (desc->clear_cache && !nfs_readdir_page_needs_filling(page)) - nfs_readdir_page_reinit_array(page, cookie, change_attr); - return page_folio(page); + if (desc->clear_cache && !nfs_readdir_folio_needs_filling(folio)) + nfs_readdir_folio_reinit_array(folio, cookie, change_attr); + return folio; } /* @@ -1021,7 +1018,7 @@ static int find_and_lock_cache_page(struct nfs_readdir_descriptor *desc) desc->folio = nfs_readdir_folio_get_cached(desc); if (!desc->folio) return -ENOMEM; - if (nfs_readdir_page_needs_filling(folio_page(desc->folio, 0))) { + if (nfs_readdir_folio_needs_filling(desc->folio)) { /* Grow the dtsize if we had to go back for more pages */ if (desc->folio_index == desc->folio_index_max) nfs_grow_dtsize(desc); @@ -1115,7 +1112,7 @@ static void nfs_do_filldir(struct nfs_readdir_descriptor *desc, break; } } - if (array->page_is_eof) + if (array->folio_is_eof) desc->eof = !desc->eob; kunmap_local(array); @@ -1148,7 +1145,7 @@ static int uncached_readdir(struct nfs_readdir_descriptor *desc) arrays = kcalloc(sz, sizeof(*arrays), GFP_KERNEL); if (!arrays) goto out; - arrays[0] = page_folio(nfs_readdir_page_array_alloc(desc->dir_cookie, GFP_KERNEL)); + arrays[0] = nfs_readdir_folio_array_alloc(desc->dir_cookie, GFP_KERNEL); if (!arrays[0]) goto out; @@ -1185,7 +1182,7 @@ static int uncached_readdir(struct nfs_readdir_descriptor *desc) } out_free: for (i = 0; i < sz && arrays[i]; i++) - nfs_readdir_page_array_free(folio_page(arrays[i], 0)); + nfs_readdir_folio_array_free(arrays[i]); out: if (!nfs_readdir_use_cookie(desc->file)) nfs_readdir_rewind_search(desc);