From patchwork Tue Oct 22 19:24:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13846075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFDAECDD0D8 for ; Tue, 22 Oct 2024 19:29:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 71A006B0082; Tue, 22 Oct 2024 15:29:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6CA316B0083; Tue, 22 Oct 2024 15:29:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56A5B6B0085; Tue, 22 Oct 2024 15:29:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3BC1A6B0082 for ; Tue, 22 Oct 2024 15:29:51 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 0FB25AAEC8 for ; Tue, 22 Oct 2024 19:29:18 +0000 (UTC) X-FDA: 82702227468.19.AC84342 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf20.hostedemail.com (Postfix) with ESMTP id 188731C0019 for ; Tue, 22 Oct 2024 19:29:28 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=jFUo6ukh; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf20.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729625222; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/mi3lQ6T07AhjXuuW+YLZdNW4Sf4NNrHXlGjygFJG4M=; b=OqVcYjHI0Q+iFjNFswZHF2xCZyb9tVu2tSZmcMXfkJQjQYMo22h83H4e7RXNpBC3xHt6WP P2UOO2+6xd+pnZYfvn9zCL0mnfdmKjkodoUA33Vqurg6f72+EvB7OAc1mCSVPf466RsLcT eVEadPqx6NvDGd745/Pfq/5+eAnMibE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729625222; a=rsa-sha256; cv=none; b=flE1PcyTi12NvSQkVQCz/m7pET9XX9uOL80JzVATDrIG4MbY0NoCqtTgYV7UGtOIjnyTap jhqgh7xWnI8oC/7BasdBcClObymCY9tswKmJOLNXen6d/Z0Tr4OVpFduxfNqh0eIzj8nE7 uA9fG48Nwx5YBkkrrgLs0X+Ij0f2fEk= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=jFUo6ukh; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf20.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=ryncsn@gmail.com Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-20cdb889222so58305635ad.3 for ; Tue, 22 Oct 2024 12:29:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729625387; x=1730230187; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=/mi3lQ6T07AhjXuuW+YLZdNW4Sf4NNrHXlGjygFJG4M=; b=jFUo6ukhnGKhLhCzrmw9NlwzE7E4qEjqzD9OEZZnfMAptuVOdeSJMHZShl93/2yx3/ padQYB1ND0U6G8oFXWpb5MG8x/VtjZ57OkqnStep27b6/SE6DpTffUWbrpP45mTcBCpK 8gC8YWIYne7GCCj6F3TDJpR63RZWmSgvwq/hbK6+WOB027SmUnkgnJQtGr52Mif0LKHQ 2Qlq1qP3NabTPi/PlWMZ7PVlF4Rjhx7+hLXHX2a60cPRTHI4WKNd/l/cBaGKqrUPW8jN pwf1NVYGh1yJVRC5sUNvlUWA5FHATHlrykh5i1aJu8vuY9XaIc7dwY/r/l//tJKWIj1x EASg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729625387; x=1730230187; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=/mi3lQ6T07AhjXuuW+YLZdNW4Sf4NNrHXlGjygFJG4M=; b=SlBCbOfOKvIR5dD4itLT593z74gtxFX4PrHbq4t8skKa8P5wz6gNFdDPqWXSp/1bsM 04sJ9vy6kc9DJeE45uY44Dh2/p09BhuMDxshk70ajfFvsA23xCXsXXT14VqWJNJ0eGbE TbcsETYkKEvmwmCnSHdmBbks0pXI4Y3TncU4nLvLhMfy6vMRV/vM9Hq6NcFP0MAy21Kp bdVcpgf5Sj9aAjNqolWVV07jreZVN7beX/n+d/85IOkd0Wa3BukBSJcdRJX6uodzFRJ7 CX6tWanBpVzlkRiiO877hQjMhOa2dChA1C6DZM2gU1c6aiK16ili0WjRx7Gr4YoSqrlK 1bIg== X-Gm-Message-State: AOJu0YwIfppK0V948K8R1UKAn59pswzjAXppiy447L/2yVscSYZUz8Wb Fqo7BZ5St4+E5lLo6Yv5BQDr8CHPUKCxeyxhYcg2Ivaf+dYr29LlHvZETL9x7HQ= X-Google-Smtp-Source: AGHT+IHYM0AqePAeT91a9awvzQB5jl0p+WmH3l4Mu0GnxO1LklkU59Sg3M8q1ydkMmTY3kKflEy/Lg== X-Received: by 2002:a17:903:2447:b0:20c:ce1f:13bd with SMTP id d9443c01a7336-20fa9e2488dmr2805955ad.18.1729625387386; Tue, 22 Oct 2024 12:29:47 -0700 (PDT) Received: from KASONG-MC4.tencent.com ([106.37.123.36]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20e7f0d9f05sm45895305ad.186.2024.10.22.12.29.44 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 22 Oct 2024 12:29:46 -0700 (PDT) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Barry Song , Ryan Roberts , Hugh Dickins , Yosry Ahmed , "Huang, Ying" , Tim Chen , Nhat Pham , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 01/13] mm, swap: minor clean up for swap entry allocation Date: Wed, 23 Oct 2024 03:24:39 +0800 Message-ID: <20241022192451.38138-2-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241022192451.38138-1-ryncsn@gmail.com> References: <20241022192451.38138-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 188731C0019 X-Stat-Signature: ry4pe5h6jw4x68kot1ghbtc6t1eb8o37 X-Rspam-User: X-HE-Tag: 1729625368-783445 X-HE-Meta: U2FsdGVkX1/p3+6Rtx1Cz4KXPq75e/W2CwbvoyY/T4U2uLfaJd0fmsUiPZcVp5+hwSr8T3mQG97lnAPB2tmZERqxIABSqL7oxGhz04qWL+kMXbF0mJjHIw5TkMdSGZxXHyWTARS6XkK6oG9ZvK6eV93ucpWCmbEGK+bzl8tEEPDk+0iRYVQ91wOk2SYn+Mfb5hGuzxlHpbVIn7smTHRR7ol2jXm+36qoQMUuG/d6bp/EfwAAsG3jdqVlvv6z/9f/Cr2JaKj/CF7/p47J1ZcsyhesZWaIWdYn0tUC70N5A3B8botsqRVj15cEB4KqklDrgYJ7ECtJ1hBn63mdviqKhRTrV+TA937VUYvI8yyFV3jUyJI1nw+fxOI/foPoGi1kEacOIsQkblBU27h74jdBX9ye4Iq/r9FicO1xjAk24vWbS9KucXzAuVBxxbpP65plotwHneB8GFx659WlkBKnaUKH6nXYPwdhUh1TZOpwZ6sxtdsLbmlcYhIB/9nysYo5P6EG6fQHFfIUHgHQB3xGPlizZ1csUm+LLbvwceWCSb2xty8pCUsVf1QOH57hdJKPhk7SbAjNh2UMf2UIASbYbxgg8OiKclFrhb3mt2Hr9rfR2RSkfRlm+QmAZqlWMAquXdhVDIt9QncZ1OUIiqC1rrz3uK1MD8U81ar/4OkHprloRauiUb1jiLovd9Z9i4AACuxXNm04i5eckcxo3qg7089YicSlO07qUoSGvfWmTryMyT/y1vLoyDBthZsElAK2rl1hD7ekkSG3L93aJlGKApx99sMUO6xBwhdJtmMo+J2QnVa2lN6d4WuVJkZ8M7OLCMQ9rqFvX++7gtdurhhEPv5m5HJIHfrVvBwTMw9RReqRMocD3DfWpBFDq6dX13N/EINDMDL4V3okopaANv6/nld5z6VoR3ep8J0FJErgWpgKCMEXuvGcGnHIXeC4zhLYmTRuaLyAzU07bKd7RMw zjst0JPA dioK4q0/aORDLTciY8xe7NEVZwXjloWVdyivmRo/As6/KqvMlLw0pQ1k0fVUGKZMsIFUtayhVHmVW4WoPGAlVVG2C6EY5/UOu/A93+DHL7pOmWM0jPi7brhwAf534x5FkIfOzFD3Xj0OEPRMEcOvM8pG8AQROjbKIrvTVDNbTi9BkJwKIEBKq3ydzNylLDD96wlAJXEkke/eRZZTblN5DgtgIQIToUhpkdHS2G3aRjpc6W8qm+Z9enu8HsTmG04KPOo4PFNoMcIHJrLQHX7aIaFmC5HOYEMR0FTS9280MDzStlQVEZWzGCRJGscNjPYfmRZknBfF8Gd5fEGXYt3n72hDqigHNo1QTnyxZWthOVX1u9CGA2nPcD+5Xx41J1+r5KV9wwbfXM6Z+5VSDIZHibmBEAAmkiUxylLbOyIb0o84vmWuskuxdHULzRVUVn0FpTqw0QXnTwApxPplQfl32pHe87E0Euh2UsGcZ84h8tjY9VZV9+4XGvr4SO4GOSKiijkv/xR05ZWgmbJQlVt2pZQ/Ysn0xnAxEJRmTrn8DNAsqjVSl2L0gLjNdcw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Direct reclaim can skip the whole folio after reclaimed a set of folio based slots. Also simplify the code for allocation, reduce indention. Signed-off-by: Kairui Song --- mm/swapfile.c | 59 +++++++++++++++++++++++++-------------------------- 1 file changed, 29 insertions(+), 30 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index 46bd4b1a3c07..1128cea95c47 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -604,23 +604,28 @@ static bool cluster_reclaim_range(struct swap_info_struct *si, unsigned long start, unsigned long end) { unsigned char *map = si->swap_map; - unsigned long offset; + unsigned long offset = start; + int nr_reclaim; spin_unlock(&ci->lock); spin_unlock(&si->lock); - for (offset = start; offset < end; offset++) { + do { switch (READ_ONCE(map[offset])) { case 0: - continue; + offset++; + break; case SWAP_HAS_CACHE: - if (__try_to_reclaim_swap(si, offset, TTRS_ANYWAY | TTRS_DIRECT) > 0) - continue; - goto out; + nr_reclaim = __try_to_reclaim_swap(si, offset, TTRS_ANYWAY | TTRS_DIRECT); + if (nr_reclaim > 0) + offset += nr_reclaim; + else + goto out; + break; default: goto out; } - } + } while (offset < end); out: spin_lock(&si->lock); spin_lock(&ci->lock); @@ -826,35 +831,30 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si, int o &found, order, usage); frags++; if (found) - break; + goto done; } - if (!found) { + /* + * Nonfull clusters are moved to frag tail if we reached + * here, count them too, don't over scan the frag list. + */ + while (frags < si->frag_cluster_nr[order]) { + ci = list_first_entry(&si->frag_clusters[order], + struct swap_cluster_info, list); /* - * Nonfull clusters are moved to frag tail if we reached - * here, count them too, don't over scan the frag list. + * Rotate the frag list to iterate, they were all failing + * high order allocation or moved here due to per-CPU usage, + * this help keeping usable cluster ahead. */ - while (frags < si->frag_cluster_nr[order]) { - ci = list_first_entry(&si->frag_clusters[order], - struct swap_cluster_info, list); - /* - * Rotate the frag list to iterate, they were all failing - * high order allocation or moved here due to per-CPU usage, - * this help keeping usable cluster ahead. - */ - list_move_tail(&ci->list, &si->frag_clusters[order]); - offset = alloc_swap_scan_cluster(si, cluster_offset(si, ci), - &found, order, usage); - frags++; - if (found) - break; - } + list_move_tail(&ci->list, &si->frag_clusters[order]); + offset = alloc_swap_scan_cluster(si, cluster_offset(si, ci), + &found, order, usage); + frags++; + if (found) + goto done; } } - if (found) - goto done; - if (!list_empty(&si->discard_clusters)) { /* * we don't have free cluster but have some clusters in @@ -892,7 +892,6 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si, int o goto done; } } - done: cluster->next[order] = offset; return found;