From patchwork Mon Aug 8 19:33:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12939057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96AB9C25B0C for ; Mon, 8 Aug 2022 19:35:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D10A5940012; Mon, 8 Aug 2022 15:35:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C4B5C940007; Mon, 8 Aug 2022 15:35:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93E9A940013; Mon, 8 Aug 2022 15:35:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5CC91940007 for ; Mon, 8 Aug 2022 15:35:19 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2E28EAB3BE for ; Mon, 8 Aug 2022 19:35:19 +0000 (UTC) X-FDA: 79777429158.03.A7E4033 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf21.hostedemail.com (Postfix) with ESMTP id B75341C005A for ; Mon, 8 Aug 2022 19:35:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=6cHy2CgAmT8QbiJcz9fEbNPRrvcgWUDZeC1E/6gPUr8=; b=JVtd4uChBzXwrhrGzyBkoGveSd fHHis0Db7zAnYwh9OyrPpUVtbnW/MZRc7HoePmXc06xGXfeng0GPEF7MlxTKllU0toMAqVg2WEekW tJXzg4GI62VbOP5L8rgjWiSiheergTTgc0CUFOJ5k0PC9H15wrMtu8dE1N87cFtHfKiXvb26JxUdl rpYP4RIHMeWPq7YKrRvWCnafOEXtK1LdDVJF27RgcIlrfpI/v73uUbHWR4WfyL1K1sVFyW7hMdHR6 SqTQC8IqxDWa1LjSZJfn5MZLVwz1cnSqsDvvnYdQA4TxfRN6TdQwPtpOeGwrLjIsdL1vpML71yF5S 6S9ljnCw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1oL8Ww-00EAvg-CT; Mon, 08 Aug 2022 19:35:17 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hughd@google.com Subject: [PATCH 20/59] shmem: Convert shmem_mfill_atomic_pte() to use a folio Date: Mon, 8 Aug 2022 20:33:48 +0100 Message-Id: <20220808193430.3378317-21-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20220808193430.3378317-1-willy@infradead.org> References: <20220808193430.3378317-1-willy@infradead.org> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=JVtd4uCh; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659987318; a=rsa-sha256; cv=none; b=OJLrQJ/CzTG5THFRY7N4ubtDhRw3AYJODrwJgoJKWYBmsxuzsXaXeafEHQxpEoQa1022eB oHMr83G17SLv+vejn1SQIO25gJ+36r33S1khy6u5jB82Is4FlU8doCXE4Y1woQM6I48Eof UTsn8NXPuC0TE1Rf2b+AQAXokSjhCdc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659987318; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6cHy2CgAmT8QbiJcz9fEbNPRrvcgWUDZeC1E/6gPUr8=; b=nG6vACgjkC8hK741kC/7SS/t1lqDTaO8kK2R52iHloUSY7zZoDFQABol8GZdVnL8g9y05B FwT4TL/J6nTnmxkk4t4SoMfuCu8u2HbccIs+Fs8Tn9FeHKMFO4NCyl8654ewrp+fdOyfBL ukPQanZqPF6l0cK96MiM4bqX361nxN8= X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: B75341C005A X-Rspam-User: Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=JVtd4uCh; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none X-Stat-Signature: txmaf589zr5johhc58tmth89idph3c3s X-HE-Tag: 1659987318-594460 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Assert that this is a single-page folio as there are several assumptions in here that it's exactly PAGE_SIZE bytes large. Saves several calls to compound_head() and removes the last caller of shmem_alloc_page(). Signed-off-by: Matthew Wilcox (Oracle) --- mm/shmem.c | 45 +++++++++++++++++++-------------------------- 1 file changed, 19 insertions(+), 26 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 4693edb33648..e7fd1dfb2895 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2353,12 +2353,6 @@ static struct inode *shmem_get_inode(struct super_block *sb, struct inode *dir, } #ifdef CONFIG_USERFAULTFD -static struct page *shmem_alloc_page(gfp_t gfp, - struct shmem_inode_info *info, pgoff_t index) -{ - return &shmem_alloc_folio(gfp, info, index)->page; -} - int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, struct vm_area_struct *dst_vma, @@ -2374,7 +2368,6 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, pgoff_t pgoff = linear_page_index(dst_vma, dst_addr); void *page_kaddr; struct folio *folio; - struct page *page; int ret; pgoff_t max_off; @@ -2393,53 +2386,53 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, if (!*pagep) { ret = -ENOMEM; - page = shmem_alloc_page(gfp, info, pgoff); - if (!page) + folio = shmem_alloc_folio(gfp, info, pgoff); + if (!folio) goto out_unacct_blocks; if (!zeropage) { /* COPY */ - page_kaddr = kmap_atomic(page); + page_kaddr = kmap_local_folio(folio, 0); ret = copy_from_user(page_kaddr, (const void __user *)src_addr, PAGE_SIZE); - kunmap_atomic(page_kaddr); + kunmap_local(page_kaddr); /* fallback to copy_from_user outside mmap_lock */ if (unlikely(ret)) { - *pagep = page; + *pagep = &folio->page; ret = -ENOENT; /* don't free the page */ goto out_unacct_blocks; } - flush_dcache_page(page); + flush_dcache_folio(folio); } else { /* ZEROPAGE */ - clear_user_highpage(page, dst_addr); + clear_user_highpage(&folio->page, dst_addr); } } else { - page = *pagep; + folio = page_folio(*pagep); + VM_BUG_ON_FOLIO(folio_test_large(folio), folio); *pagep = NULL; } - VM_BUG_ON(PageLocked(page)); - VM_BUG_ON(PageSwapBacked(page)); - __SetPageLocked(page); - __SetPageSwapBacked(page); - __SetPageUptodate(page); + VM_BUG_ON(folio_test_locked(folio)); + VM_BUG_ON(folio_test_swapbacked(folio)); + __folio_set_locked(folio); + __folio_set_swapbacked(folio); + __folio_mark_uptodate(folio); ret = -EFAULT; max_off = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE); if (unlikely(pgoff >= max_off)) goto out_release; - folio = page_folio(page); ret = shmem_add_to_page_cache(folio, mapping, pgoff, NULL, gfp & GFP_RECLAIM_MASK, dst_mm); if (ret) goto out_release; ret = mfill_atomic_install_pte(dst_mm, dst_pmd, dst_vma, dst_addr, - page, true, wp_copy); + &folio->page, true, wp_copy); if (ret) goto out_delete_from_cache; @@ -2449,13 +2442,13 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, shmem_recalc_inode(inode); spin_unlock_irq(&info->lock); - unlock_page(page); + folio_unlock(folio); return 0; out_delete_from_cache: - delete_from_page_cache(page); + filemap_remove_folio(folio); out_release: - unlock_page(page); - put_page(page); + folio_unlock(folio); + folio_put(folio); out_unacct_blocks: shmem_inode_unacct_blocks(inode, 1); return ret;