From patchwork Mon Jan 13 17:57:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13937867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AD04C02183 for ; Mon, 13 Jan 2025 17:59:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EBBF76B0088; Mon, 13 Jan 2025 12:59:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E1B366B008C; Mon, 13 Jan 2025 12:59:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C94FD6B0095; Mon, 13 Jan 2025 12:59:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id AB1106B0088 for ; Mon, 13 Jan 2025 12:59:57 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 660091A024D for ; Mon, 13 Jan 2025 17:59:57 +0000 (UTC) X-FDA: 83003192034.03.85A7315 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf20.hostedemail.com (Postfix) with ESMTP id 7A54F1C000A for ; Mon, 13 Jan 2025 17:59:55 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=N+vVmHjL; spf=pass (imf20.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736791195; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3bA8B6i+M3WoYB0+zo3p6m39muwwLLEWjZCQybnFwG0=; b=hQ1P0V/ovUMLh0RcMrYcTeZVHqYpOsRV15VULnniknnanCnbOYbhVe4VvKlOo5IbvMDyXD YPlqtYh9IKpTpyLhOzaTQahhJKr+K06AedLeNku9cukf8U60JBTAjBiK9bchWaQLpm2IoJ HnRZhvOGFgeS9RTdTj7rc4htjfaMpsQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736791195; a=rsa-sha256; cv=none; b=xOAmTXP/7tZibx5rMKtw8f2U68jZeuinR7allX1zj4izektBBxoFG/neigA4+gz+AZWls0 NKMQl6gMrh3sHUZQ8PjtG8Gp77i8J4PK3uUpODx26gsEBq4fR2IbDRGmChrg2rf+idut+T QO3l1W+eFB9Fw+8xXz8Bpwe7BoNlcyo= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=N+vVmHjL; spf=pass (imf20.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-21675fd60feso104510675ad.2 for ; Mon, 13 Jan 2025 09:59:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736791194; x=1737395994; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=3bA8B6i+M3WoYB0+zo3p6m39muwwLLEWjZCQybnFwG0=; b=N+vVmHjLB369Wcbvnv5OmMOt/usY9MozFsk5xbtjFPwg+UG7rETyu4qqxdHPwXSAze D3TbJ5I1IO2itJRU7tiAUdwUe9a0nlMT61KV3Hu7b14/BxCtA2hbkCFbVMH+ej5Ti6uL efL6xk4qms0134cu7korO2JPKtgE6MnWd15/9GjYk677ccATkfaM9bhc8nTUc/kSSp7n /VltbFEqCLGr5V78fQePgs/vUxBsSIV4KwWYOAf6E7TL0zzOZ1Honw1i+Vp5zFA7xXsB hvn+RXGelom403fcyEb+HJ8gaQ9+ErF3J4mYtnw37kfySIksijzh7Hc4I7yF55PISGE1 0oMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736791194; x=1737395994; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=3bA8B6i+M3WoYB0+zo3p6m39muwwLLEWjZCQybnFwG0=; b=gkqE019NJTYK02Nl580IOtZpBcXyjQqs/XbtESZLLOfHx1gpNDunuMZNw2ajfM+0/z Gx1xJmpwjBCuMK1mNB2qdYhdojd8BbScDrCByp4ZiZvR+cd6gz63ewO2i975M1Moam/u qcKNmC+Pi4lcsJI3+Pjzuwz59VXwRoBH5geu08LiE7HcyIJvAoSrM+vjGJStwdwkzs9f b5v8PoakKyfTZb9RSa4UzhLoKGQ0lyrWSpdcuy5b2dXb32Hm8x/uKm6L2Lhjoe49Xt/+ 5jTdnRxmsxpWMW1Y9pOmBgcLzLRfPiCszYqUqMSkMVoDpLVFXKzXGUULuellJBZcRqQU GUqw== X-Gm-Message-State: AOJu0YzpsM9Jw83DnvpKu8rYh40wdHipcT69L6PWiDqKrYYrMXekVbzk kaEMHSO6p59U2kSD4KyjrQEOvR+uQWsVL2466co9fMaNQhSXPq+jA6Gz/VHYdTI= X-Gm-Gg: ASbGnct7kdZ/I+/A0wEU3oUPtXasK2ji098EnkDuRuizXXeSsPDVJHBKb1k/SWXzGRU Tqjzmbv3LU8roXUXhCMA50PYM+SrcW7KhYtfq9Kiq3Uwggx3dXGFlZW/+nBkt2xN80qOmJr54PB Bc+p5cjKWo5T1I0LLCupr6lDoupNOh0dWG4GXGzNd0DMeFoICKdGyfkuR+3Bpe1GksoEV3dOp06 UHgug4VQt8ff1CoxzbRGcPAk5XOg2Cveq+p0p2xyzci5hbBJLsr/mHQaHfetYXUyF+tVuGEWj7M 5g== X-Google-Smtp-Source: AGHT+IHG9J+kj0NwuV4dCCUaflW/hQEMT5mplvb1rxDF2wEH2kzs02TDJ+5TN5Jg0cDR5NohlVyXfw== X-Received: by 2002:a17:903:41c4:b0:212:5786:7bb6 with SMTP id d9443c01a7336-21a83f469b8mr301683785ad.3.1736791193793; Mon, 13 Jan 2025 09:59:53 -0800 (PST) Received: from KASONG-MC4.tencent.com ([115.171.41.132]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21a9f21aba7sm57023635ad.113.2025.01.13.09.59.50 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 13 Jan 2025 09:59:53 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Barry Song , Ryan Roberts , Hugh Dickins , Yosry Ahmed , "Huang, Ying" , Baoquan He , Nhat Pham , Johannes Weiner , Kalesh Singh , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH v4 01/13] mm, swap: minor clean up for swap entry allocation Date: Tue, 14 Jan 2025 01:57:20 +0800 Message-ID: <20250113175732.48099-2-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250113175732.48099-1-ryncsn@gmail.com> References: <20250113175732.48099-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 7A54F1C000A X-Stat-Signature: ezmkifdkuu4qe9kjcub1igx39nay9pjx X-Rspam-User: X-HE-Tag: 1736791195-83405 X-HE-Meta: U2FsdGVkX18FIDoimbH2z0xUbCjQAsSIYx3jYeceYV5jclyaQ8VTF4rIV13xnyOKV9GoU7j+IKg03wfzoIo9LkKj+whbmxMHQ/uXa+1fYxC7dp3oRhbukLVQKE39wmKFQkZfbLGmtQ1X4bF5dEoOg3YjRbkKvTnRZCGle8cpDSNYVtu7kW5g/uY3UQx9gbzDLTPxF6BocfDZZimU0fNj+Dx+RQKFoX/W3SuqkAhhvMEQC9ulF4E7Jp6UJ+yk0lAKIrKincoEDUkSCh3MSPOpZhlnlbB2c2ioO9FgziLjflBGYNTdvimXlWNtDaPleho0OeHHOQ28Kg7ri7uJoYseIk7rmalI60mSr5AmRv18rIyYWMO2xx3RWUDn2glF88wM+iEH254orYp6zUEZpJ62X1ySo17+61yEsBt4D8QQLqBmmJK5YDawTpsBtnAdsR1nA2eMQGz+KsrT0CKiCrr8c6ScW2j94xa+vzzV/z6LnMbuswVfBlNo+jkwBhC7cyN8llqgzPjFBg7DmIfHRHT/cmdEjbrh1KVKiFEB/5vxbO3cysX7Mz1D2M+OMm4hM/nhJQurwI3U8WES/hF5h+i49pLmEMqqP2WBcCCh8iTiUxUU3NUzKYAy+PLIQmy9WKZCPD/62v+Sup1qoWSdYtS393xBaVEzFr3h9RcCSiLkG6WedcFfHbB07QMUywb/eRcBe/ctdqC6iWRq2ZXqVfw3oWop+XgwJE/33NHj9H/BY4Hvxud/zqQiO2IzZWb35WycfKErl68oRNu1eYsXT9R6+CykWPTvBjU5ZCgu5vIxsfIEt5Yfgkl+g9V5484SncSn281oiSN5oPkOB2ZYwrSpjI4my8FipR299N1tmwQ8CHl+/ZoQst7PEIqL+i1EZ1r5tPcKOgWW+YVFKujmR6lT9RxN5Rtq1O3Sg55NwL9f1tsuGpCmto303RJWLej2YeBkG6r0ea9AXzCwvCW5Snj j4+HL3No W+J0CwdT1Jlrwj4BP9ipPRmbyPextBl5HKsPDkUBLB1p+TFNR/tc+rSJSkG3lTZXOJ1dIlvBBHdTev15xGOLAuuLGXbcpy4k9hrJ5NC4rYQBBnmqsTa5sB6ZeAfeq6SQAlf98jJi43BZbm7PnbBboB/MthaS9ET7rCDnr0PX+LGWwloSZwrG0bJPmvP2hy6tion9OmT6VvCpBma2dDhV5ur1Zh+4qKF7JnVTULl5q/st5ZI72NwxC/Cnonhe92ZXjjh1Hp5yVHAfbkDkXLG0QhnzRJeLhwyqMNvjIQMejeNMAlIlGYaKP0v7SWwIYjhjsXq8CjzJiJJx9QHF31EMbilFIDIuVsW9yoBbcBUY9kUOyASfSfD/aFhQeT0pEIO2hjbeHK9osMA0To3DpHMAfhFxoscEydxHiLWeOaX3oXF/QF/zPtoN3eRED5sKExr3Ve7qMnQOJncCGZ+Fyo28RBXLJ2D4dusqBA1Oo4IuDhdjN9JJBykNYTaSk1ecaYgiTgrQz9hkMk9rPmi1w7lSLq2OhHXOf6w2s/0xUUmtubX5+WUA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Direct reclaim can skip the whole folio after reclaimed a set of folio based slots. Also simplify the code for allocation, reduce indention. Signed-off-by: Kairui Song Reviewed-by: Baoquan He --- mm/swapfile.c | 59 +++++++++++++++++++++++++-------------------------- 1 file changed, 29 insertions(+), 30 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index b0a9071cfe1d..f8002f110104 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -604,23 +604,28 @@ static bool cluster_reclaim_range(struct swap_info_struct *si, unsigned long start, unsigned long end) { unsigned char *map = si->swap_map; - unsigned long offset; + unsigned long offset = start; + int nr_reclaim; spin_unlock(&ci->lock); spin_unlock(&si->lock); - for (offset = start; offset < end; offset++) { + do { switch (READ_ONCE(map[offset])) { case 0: - continue; + offset++; + break; case SWAP_HAS_CACHE: - if (__try_to_reclaim_swap(si, offset, TTRS_ANYWAY | TTRS_DIRECT) > 0) - continue; - goto out; + nr_reclaim = __try_to_reclaim_swap(si, offset, TTRS_ANYWAY | TTRS_DIRECT); + if (nr_reclaim > 0) + offset += nr_reclaim; + else + goto out; + break; default: goto out; } - } + } while (offset < end); out: spin_lock(&si->lock); spin_lock(&ci->lock); @@ -838,35 +843,30 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si, int o &found, order, usage); frags++; if (found) - break; + goto done; } - if (!found) { + /* + * Nonfull clusters are moved to frag tail if we reached + * here, count them too, don't over scan the frag list. + */ + while (frags < si->frag_cluster_nr[order]) { + ci = list_first_entry(&si->frag_clusters[order], + struct swap_cluster_info, list); /* - * Nonfull clusters are moved to frag tail if we reached - * here, count them too, don't over scan the frag list. + * Rotate the frag list to iterate, they were all failing + * high order allocation or moved here due to per-CPU usage, + * this help keeping usable cluster ahead. */ - while (frags < si->frag_cluster_nr[order]) { - ci = list_first_entry(&si->frag_clusters[order], - struct swap_cluster_info, list); - /* - * Rotate the frag list to iterate, they were all failing - * high order allocation or moved here due to per-CPU usage, - * this help keeping usable cluster ahead. - */ - list_move_tail(&ci->list, &si->frag_clusters[order]); - offset = alloc_swap_scan_cluster(si, cluster_offset(si, ci), - &found, order, usage); - frags++; - if (found) - break; - } + list_move_tail(&ci->list, &si->frag_clusters[order]); + offset = alloc_swap_scan_cluster(si, cluster_offset(si, ci), + &found, order, usage); + frags++; + if (found) + goto done; } } - if (found) - goto done; - if (!list_empty(&si->discard_clusters)) { /* * we don't have free cluster but have some clusters in @@ -904,7 +904,6 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si, int o goto done; } } - done: cluster->next[order] = offset; return found;