From patchwork Mon Aug 8 19:34:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12939078 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18CB0C00140 for ; Mon, 8 Aug 2022 19:35:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9185C6B0073; Mon, 8 Aug 2022 15:35:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 879F06B0074; Mon, 8 Aug 2022 15:35:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60589940022; Mon, 8 Aug 2022 15:35:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 270C8940018 for ; Mon, 8 Aug 2022 15:35:40 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id E61F6C019A for ; Mon, 8 Aug 2022 19:35:39 +0000 (UTC) X-FDA: 79777429998.26.F99FDC2 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf21.hostedemail.com (Postfix) with ESMTP id 929051C005A for ; Mon, 8 Aug 2022 19:35:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=9mOmmVtZpFIWWo3nfTdSfePzyVXKHrnvdlc0DSY2URw=; b=ulk5fJbGld4YTUhYPsK819aM/u deJFxCZtd1dX1BFAElWhl5ceiliGwft3tZfNoj5jxlHh3pIRUub9zpG7T9WFVNmw3kgn5DOYZA8KW C7pMZBr34zIx9pLrRxOWnWXpCH+mZGcExmXx4sg95xv14DYgHU+zS4a14gJDWwJeMLLyobZEOgwBt 1yOQwz7RiloU0kUaYaiF6tBmlBx/b465+PDv+ODeHZQveRVnWU6p+CP8n+jLkUpZYiMu3jJuB+ET5 5S1qsuMa6oIBGmzcO/sgQ0j/Ycb846XBa0hk7KcyvhZDr/AOe1JqvlYicZxTyGBnK/bF0Z0zpEtIp 7lg6CzSA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1oL8XK-00EB0M-0n; Mon, 08 Aug 2022 19:35:38 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hughd@google.com Subject: [PATCH 46/59] madvise: Convert madvise_free_pte_range() to use a folio Date: Mon, 8 Aug 2022 20:34:14 +0100 Message-Id: <20220808193430.3378317-47-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20220808193430.3378317-1-willy@infradead.org> References: <20220808193430.3378317-1-willy@infradead.org> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659987339; a=rsa-sha256; cv=none; b=Z+zW1lJwyOHwCh2d6ykRQ4s5+TmxGdrsYyK/7STxI1ksk1n+r4zGHh/jfEAmS/AA+pwkAN rTmwsiFJcCoNBX7GDzpLesXYHroHyKCq3OI4rlI3q3hF4GwsAlZrQGW8HCjUWq6co0GDG0 IY18tGjk5+FHjrAEEhE/kqMURFZOn7o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659987339; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9mOmmVtZpFIWWo3nfTdSfePzyVXKHrnvdlc0DSY2URw=; b=KjQ1Y5CMmtkn+Fepp1pvWfyLPVkdFQIiztk8EQ2oIJef5TM2+s26Kq3Aqie1lJfS0eDqVW 5SsZK/0iRUqvN5zy2TRgGW6I0PcpHvH6LSaEr3ttYrWR/UWzBqyiPSD0G0iDjYJiwtFF4B aJuWPro5YJvZypqcIbw3Rl2DkTy6x8M= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ulk5fJbG; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none X-Stat-Signature: bazihtk5kpiarngozxpwer9firn8ub5r X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 929051C005A Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ulk5fJbG; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none X-HE-Tag: 1659987339-529066 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Saves a lot of calls to compound_head(). Signed-off-by: Matthew Wilcox (Oracle) --- mm/madvise.c | 49 +++++++++++++++++++++++++------------------------ 1 file changed, 25 insertions(+), 24 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 5f0f0948a50e..05de79d269e8 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -597,6 +597,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, struct vm_area_struct *vma = walk->vma; spinlock_t *ptl; pte_t *orig_pte, *pte, ptent; + struct folio *folio; struct page *page; int nr_swap = 0; unsigned long next; @@ -641,56 +642,56 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, page = vm_normal_page(vma, addr, ptent); if (!page || is_zone_device_page(page)) continue; + folio = page_folio(page); /* - * If pmd isn't transhuge but the page is THP and + * If pmd isn't transhuge but the folio is large and * is owned by only this process, split it and * deactivate all pages. */ - if (PageTransCompound(page)) { - if (page_mapcount(page) != 1) + if (folio_test_large(folio)) { + if (folio_mapcount(folio) != 1) goto out; - get_page(page); - if (!trylock_page(page)) { - put_page(page); + folio_get(folio); + if (!folio_trylock(folio)) { + folio_put(folio); goto out; } pte_unmap_unlock(orig_pte, ptl); - if (split_huge_page(page)) { - unlock_page(page); - put_page(page); + if (split_folio(folio)) { + folio_unlock(folio); + folio_put(folio); orig_pte = pte_offset_map_lock(mm, pmd, addr, &ptl); goto out; } - unlock_page(page); - put_page(page); + folio_unlock(folio); + folio_put(folio); orig_pte = pte = pte_offset_map_lock(mm, pmd, addr, &ptl); pte--; addr -= PAGE_SIZE; continue; } - VM_BUG_ON_PAGE(PageTransCompound(page), page); - - if (PageSwapCache(page) || PageDirty(page)) { - if (!trylock_page(page)) + if (folio_test_swapcache(folio) || folio_test_dirty(folio)) { + if (!folio_trylock(folio)) continue; /* - * If page is shared with others, we couldn't clear - * PG_dirty of the page. + * If folio is shared with others, we mustn't clear + * the folio's dirty flag. */ - if (page_mapcount(page) != 1) { - unlock_page(page); + if (folio_mapcount(folio) != 1) { + folio_unlock(folio); continue; } - if (PageSwapCache(page) && !try_to_free_swap(page)) { - unlock_page(page); + if (folio_test_swapcache(folio) && + !folio_free_swap(folio)) { + folio_unlock(folio); continue; } - ClearPageDirty(page); - unlock_page(page); + folio_clear_dirty(folio); + folio_unlock(folio); } if (pte_young(ptent) || pte_dirty(ptent)) { @@ -708,7 +709,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, set_pte_at(mm, addr, pte, ptent); tlb_remove_tlb_entry(tlb, pte, addr); } - mark_page_lazyfree(page); + mark_page_lazyfree(&folio->page); } out: if (nr_swap) {