From patchwork Sun Nov 19 19:47:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13460659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D35A1C5AE5B for ; Sun, 19 Nov 2023 19:48:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 619C76B0359; Sun, 19 Nov 2023 14:48:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5C9306B035A; Sun, 19 Nov 2023 14:48:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41D4C6B035B; Sun, 19 Nov 2023 14:48:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2B09E6B0359 for ; Sun, 19 Nov 2023 14:48:16 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 09074A0285 for ; Sun, 19 Nov 2023 19:48:16 +0000 (UTC) X-FDA: 81475740192.02.B2C5895 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by imf06.hostedemail.com (Postfix) with ESMTP id 27A5418000E for ; Sun, 19 Nov 2023 19:48:13 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=GTJamWqN; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf06.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700423294; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NgugyapInVThiGa6OPAw9VMpbPbUYFH6tRNCt397cNY=; b=lzj7lFznIzsAINxp+a07qo9thtJGDQ1QQpVdTJLM3YOYVUai0nd/homGtp9TUSPfGIhkad GsmvT2n3Jrln4f8sCuX1ASCU5efJwI7puZHSeU6kY4hsTJzzbzo4tl8DWGhAXEwDJgqVF3 YNaLEYbo+f4phy4WUqqc+hV806JgSWM= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=GTJamWqN; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf06.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700423294; a=rsa-sha256; cv=none; b=hMl8oyv9YVdsLhoVrO0VMtCdFEA+idvxX/uDasyHLMW+5d3KJM26t+ktOBJK67ByPEAnT/ Iq7bvKIx2gj42a1cvpEmxoSGltiiHkqh58MgjLXSkd27cDCaJ2AepFwQCVv8BbJxkY4EFZ fOzXSFE5jMYfCPjhEEoJZzF1Y2uB5X4= Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-6c4eb5fda3cso3858592b3a.2 for ; Sun, 19 Nov 2023 11:48:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423292; x=1701028092; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=NgugyapInVThiGa6OPAw9VMpbPbUYFH6tRNCt397cNY=; b=GTJamWqNUrvC2o3Ho8JT7uShdEFicHjh0r3Rsun75h1WNtEmpWyiyd7w+lDOisxb8A gXmeCS/NNXK57PAbI+eq+hP9rj6LCL1ZSrX+WGHtqZ1KwdJC/hjqFx676ryv29u4kPfk RGlMagQwzAZloJZreicE45Ou5Mq3YOy1QcsJqdL+bOGZ7JFceGbzhb/FG2R9OBZUgM/B yE99KDFPPplaXTAyr/WWBYCu9TDHj0GywY8G/0ACkp1SEBaH5KIK4fugR964/uMuZPyq lz1RGpE4iWGzxnawC9NkzYRbnzTZEWfUPgJTEBKfO99Yw/q1hS1gOX+ZbyX0m634MSdm rqFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423292; x=1701028092; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=NgugyapInVThiGa6OPAw9VMpbPbUYFH6tRNCt397cNY=; b=VZtP1679tH3s1RpqqV1tbzCAxyniCxWnZBWnZNPk7sjE5zUTruZVh2oj6RmXowROtg OI2p3Qy+pSGrEgFIaY30Gqo1pPYo/tjFLz7dIUopiQXwMA2ufdjJkQlgh87rpD5tSfYc 0nNnPYL0Dx2XomfTAMXlwDkn/ga3agUcwXenNxcJ5V9npQ62uomybVvXIWELMy7TPmsE azI7IkUjCjJeA1h1MGCiQIZynkhZcpKek9hsAlKbswr+HMLkoXtwX3AHW9tx7g/qbLg6 jgNN/IuKrH4khqlXXvD2n0+LGannpJiHijSScZ9b0/I1D2G0Xuv9s6Zbkkyv/FUfv/Zh H7ZA== X-Gm-Message-State: AOJu0Yz3RaZ1BEbKKPnSrpc8VPRhg3PZDKKlMvy1PGShPRdmI76q+s9Z 3zsJLyCY4/34umbUCKe0/C+JORiIETgNERSg X-Google-Smtp-Source: AGHT+IEheipTHeYRZdBVyN0mV2ARTS7XP+zRN1OT8UJpXXSOat4LuPJfFp26youNgVdlUjeMC3A1hA== X-Received: by 2002:a05:6a20:e104:b0:18a:181b:146b with SMTP id kr4-20020a056a20e10400b0018a181b146bmr5052162pzb.29.1700423291675; Sun, 19 Nov 2023 11:48:11 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.08 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:11 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 03/24] mm/swap: move no readahead swapin code to a stand alone helper Date: Mon, 20 Nov 2023 03:47:19 +0800 Message-ID: <20231119194740.94101-4-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 27A5418000E X-Stat-Signature: a8uoc37i7mbmrztittgbc9bfjrcku4ep X-HE-Tag: 1700423293-398884 X-HE-Meta: U2FsdGVkX197q0RKbNIPZUlF9QxDSRHwtXtVdWn39LoB+ZZhOVyWgjxBmQv3kP2cnAUmpZD9YMVXppunrYBx8CaCr8LNA28yo8M51/ASdIqzFK5zKpS9JimBJm2eCIt6XC3J1TbHcBbhFXhrfvUCM1IRlBvV2fCyXj+wGGqQjD4gA4gb6076hl0c/gFLGZ9OREyx64q+ocbYZc3ztvokO5ataJaaW/FI+Xn8kfDmIkeDVjrjgq1kr0hi5AJ+y5f9yKQxkMCnYTh9hJV3w+b4TfeaufrOOy/AxJFDI1i6sKemfF3SMlCrlgYFvYVHoxyXfFqrgcwI/bLddGCe8jmHeze0r9cjtV5OA6XQJHpR8hTOu5nxnieCEshn6FpegyAGjgrSeKgLTdrPmK6oxpboRCabHApv5vYIAz+c9ZbD3a817SNFQDrIu/y5SbTFZ71KIQ4GnMrAk2gDN06x4/KvMDXmOMVdAO9xaqk0f8vDvSJ+rzq/niPzOH1t/Ifq/goSr80XjSAkdNeSKY6pJAD0Srq/9RRItMqcMiISzw552ujmI18fTSNX/BIlFn2nvZSHoQoKrvcnbg2U5vLTh1kBDwK3IHu4AdoFnOQ2K+h5h5s/+TMB/42knl/ZNsmZ8NmEUlIeEB2HEZW9stjpmkp8jvwNIYPKRb1oVIn3ct3XuyqPTnZmV5YvgpIcAyRpN8MguwdHZdAuAma/Xc8qhFtbLx0KBeIgWdwGpuob6gX6TUd6/kxthm5zTAEmhblVEIQMCOnP4vakIGLQJ/KSreXdRu+eC9nWEYpceMiIFoXjSJkUsXeRvgQnFeibuvUA3DtkSv6XuBEmlHQCoDWkKZ9J0se2Iri4HQtKVujhaprsryfBPFBuIhBA+fCmPE8KhB/EKjzJI2W+/j5b6B72R1/WpTJVOFUPV233YrMSNJh8VaP26iWOCkMXwXLuhUej2lDKXod2wGUwE1k7i/KYlbW SqkPcT5b lsk7iJiMR0k8oi8GVFw2boLkAA0OUTM9TbazlenaYXVDMdfGTyleJkqgundlBn81Yd0YtfvrICpqy16SXfu33QYga6u8BpoRlRszKLrLv0rE7DTTQX6J0E7J28CilqWc/B18GC7ymAXjcIb+4xU5srrOiOl1hYRI6vjWXfw2HCvZ4KTmB6PmqvhepchdDiRTfGE4wzK0WamK9GdHpQAJTotKTC4mKTxsAPV3n2QgwfwCHX8HaZ5sOK1jGCkZujtcArtg6xIEXAaR6eIeYDhZMF4hYOyj4U6g4E5I1u85+xPIFkHaaHXN6t9RA+5/aPl3VHYbyw+Ko2QysgddaphFoZCE//AZ+ut6LtBflB9t4isshiyKHNYy0RNUGX8kq/nQ0KD9Qhiut+TYQyZxnPWb9/6mdGAADG0JfF4/eD3MMTefi/sld+NsHYmEPJqXyu+hlQbPjeARduP3cYOB9IhY2Ut7EVZ3v2uhQLnb1pJ7yJjaiacIoekPbTkmTs/xM/3kuYUKiWbFMyffioI78TpM/ykrtBVqewGFpcGqEikA7rPKTHTOox/5vhY3ZS5JQXzEIouchDMYI8cHgEEFaitLhXREmK64HWiQJsxA0DkvNBc9kbLGNcI53T+IQab7XqQfsX2Ov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song No feature change, simply move the routine to a standalone function to be used later. The error path handling is copied from the "out_page" label, to make the code change minimized for easier reviewing. Signed-off-by: Kairui Song --- mm/memory.c | 33 +++++---------------------------- mm/swap.h | 2 ++ mm/swap_state.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 55 insertions(+), 28 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 70ffa867b1be..fba4a5229163 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3794,7 +3794,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) swp_entry_t entry; pte_t pte; vm_fault_t ret = 0; - void *shadow = NULL; if (!pte_unmap_same(vmf)) goto out; @@ -3858,33 +3857,11 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (!folio) { if (data_race(si->flags & SWP_SYNCHRONOUS_IO) && __swap_count(entry) == 1) { - /* skip swapcache */ - folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, - vma, vmf->address, false); - if (folio) { - __folio_set_locked(folio); - __folio_set_swapbacked(folio); - - if (mem_cgroup_swapin_charge_folio(folio, - vma->vm_mm, GFP_KERNEL, - entry)) { - ret = VM_FAULT_OOM; - goto out_page; - } - mem_cgroup_swapin_uncharge_swap(entry); - - shadow = get_shadow_from_swap_cache(entry); - if (shadow) - workingset_refault(folio, shadow); - - folio_add_lru(folio); - page = &folio->page; - - /* To provide entry to swap_readpage() */ - folio->swap = entry; - swap_readpage(page, true, NULL); - folio->private = NULL; - } + /* skip swapcache and readahead */ + page = swapin_no_readahead(entry, GFP_HIGHUSER_MOVABLE, + vmf); + if (page) + folio = page_folio(page); } else { page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf); diff --git a/mm/swap.h b/mm/swap.h index 73c332ee4d91..ea4be4791394 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -56,6 +56,8 @@ struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, struct mempolicy *mpol, pgoff_t ilx); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, struct vm_fault *vmf); +struct page *swapin_no_readahead(swp_entry_t entry, gfp_t flag, + struct vm_fault *vmf); static inline unsigned int folio_swap_flags(struct folio *folio) { diff --git a/mm/swap_state.c b/mm/swap_state.c index 85d9e5806a6a..ac4fa404eaa7 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -853,6 +853,54 @@ static struct page *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, return page; } +/** + * swapin_no_readahead - swap in pages skipping swap cache and readahead + * @entry: swap entry of this memory + * @gfp_mask: memory allocation flags + * @vmf: fault information + * + * Returns the struct page for entry and addr after the swap entry is read + * in. + */ +struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, + struct vm_fault *vmf) +{ + struct vm_area_struct *vma = vmf->vma; + struct page *page = NULL; + struct folio *folio; + void *shadow = NULL; + + folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, + vma, vmf->address, false); + if (folio) { + __folio_set_locked(folio); + __folio_set_swapbacked(folio); + + if (mem_cgroup_swapin_charge_folio(folio, + vma->vm_mm, GFP_KERNEL, + entry)) { + folio_unlock(folio); + folio_put(folio); + return NULL; + } + mem_cgroup_swapin_uncharge_swap(entry); + + shadow = get_shadow_from_swap_cache(entry); + if (shadow) + workingset_refault(folio, shadow); + + folio_add_lru(folio); + + /* To provide entry to swap_readpage() */ + folio->swap = entry; + page = &folio->page; + swap_readpage(page, true, NULL); + folio->private = NULL; + } + + return page; +} + /** * swapin_readahead - swap in pages in hope we need them soon * @entry: swap entry of this memory