From patchwork Tue Dec 24 14:38:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13920196 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D16ABE7718D for ; Tue, 24 Dec 2024 14:40:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6F3D76B009D; Tue, 24 Dec 2024 09:40:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 654046B009E; Tue, 24 Dec 2024 09:40:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 481776B009F; Tue, 24 Dec 2024 09:40:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 1F0B56B009D for ; Tue, 24 Dec 2024 09:40:06 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id CE2201A1681 for ; Tue, 24 Dec 2024 14:40:05 +0000 (UTC) X-FDA: 82930110564.19.5BB089E Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf14.hostedemail.com (Postfix) with ESMTP id 6B70510001B for ; Tue, 24 Dec 2024 14:39:20 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=g1jRSSFy; spf=pass (imf14.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735051177; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=x1wPQexfBO4aVb9VNyBG+dw97Ap+sARYAr0a6bgjO1Q=; b=F2fobEwtpE0wNQvWYBsW+hiadg09Ck7WiYnnvjx+8Phz+ChjSI0bTXLoRvSFpjdZZteWC9 rYBzBpGQeSv+elKj3LsUeNJyFfLHclxvvTuGfvOBPndSNxAtz3V4JANyeiAWJ38FQjo67O gFZEo65rVolCHdlV1Vwlwcvc+rIrlkk= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=g1jRSSFy; spf=pass (imf14.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735051177; a=rsa-sha256; cv=none; b=bc3I5n2EPL64RLlusdN5zV5s2qC4CAvlpEg6/rQjGPSCYaUf6qxWMEhirMCkxMr2g3kUsF Hls4MihS12Xb8yD3pl+v2uO7pPRjMoCMxu1JL7dL6/NxHwV+cxpRruXl2IK+uArvrKGR/H VzMXGwJ5OWxs5cjOM7mzhRxH03DbACk= Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-2166f1e589cso62911245ad.3 for ; Tue, 24 Dec 2024 06:40:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735051202; x=1735656002; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=x1wPQexfBO4aVb9VNyBG+dw97Ap+sARYAr0a6bgjO1Q=; b=g1jRSSFydq4C4DvUSS9SQ71JJjEYrVUkKOXWt1HNZiI+ZpzlCDWb2g6lyfJWhw1nOs clKiRBwedJvRHtQThylYt7PsRd3KetxZgcW2vwDRVH4UGytZJQ2CgxgJulU+r7d9+8hV qXNO4z7YOx8taQZYQJzlPFJc3XFpJtZqpkFeJhMTtIruXJbg1hs2MDTMhUWNt6Z89ZqZ 9cdxJ2yopeL2jM3sSp9DxeSc62y4mjOkTxiw7+Yhdjnjt6T02tTgEYmiDEGp/GKbrbO8 iCdegaPX8lDQCo+1dpRX1g5TKePCcmO+nlovdDDo6VomAmIf06ZJWQQJP1Wkf11f3YlV 1W9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735051202; x=1735656002; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=x1wPQexfBO4aVb9VNyBG+dw97Ap+sARYAr0a6bgjO1Q=; b=P5SKruLyMW2QjFrqhWDrcif4vMFatl9iT80SNhhfyWJJgze6sGzREE5ni6PaB8TIfO r/hAKevyRfUjAEJB4pb+/ZEEympsbf9Di1D8QLwe59PfdwbBv0pNAdVOF94ENVB6cXFu r9Vp/SpIoOKuOa5a+7Ql3d0XibX+YrIURed3rOHGruXWmyrR7iNRxFCqYaSCx95hjq2u bayXCKzf50BT+dbgiTL49YbpZxo6r1cYCfHhHBzp7jOBh7tOA9vsspFoh13zJLAC7fW1 U2UB2iC6lVlu/SvRuk+UskAOQ/Fs0TTrd2NKcM9TQtcvgwHNC/F5Na+VbWy/yUTJFNrz v3Cg== X-Gm-Message-State: AOJu0Yxf+LqiN+sQYiD2bPepCwSMGm09p6yCgY3fDUQj4N+2jp03uwPA bCuHkLCto7TLq0zK7SpiTT5jM/W/HRIFw+46tQ7AUiOa4DgfHKLmvrbe5Yd9Gfk= X-Gm-Gg: ASbGncsEjhhj0y7Wgt584UoFbgcpaTEPWeixm/uarykEgcGtJhwwF5ctqBKKo9sTMJF nmshiV+TZwoQrwMPoC908OmiEUNyn5w0M8mIimW+9dOaqLC6VYEPmRHkiEK30GlaqUIsZXMl9Al AZXAUVS3pdytTqUr5a4FVJRWT5IfYAn447k8aJ2P5efoXcnIhPGSP7b5BGb7oY93bHjnWMkk+HD 1SYbXN/+IUfgBHlD9CaBK6LKWzIGXrJgNyaDxcTAWtMfAhFSpA0fv8RQDEOD5MkNVYRNAv4VsLr ZA== X-Google-Smtp-Source: AGHT+IEysBcICmtcsLy08l3mcesaPslPW6JfCL1D4WCO+ovHstzHh+/6NlLWVBwW4F+V5pf421KQkA== X-Received: by 2002:a17:903:1105:b0:215:a3fb:b4d6 with SMTP id d9443c01a7336-219e6e8c5a0mr250562295ad.8.1735051201813; Tue, 24 Dec 2024 06:40:01 -0800 (PST) Received: from KASONG-MC4.tencent.com ([115.171.41.189]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc9cdec1sm90598735ad.136.2024.12.24.06.39.58 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 24 Dec 2024 06:40:01 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Barry Song , Ryan Roberts , Hugh Dickins , Yosry Ahmed , "Huang, Ying" , Nhat Pham , Johannes Weiner , Kalesh Singh , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH v2 08/13] mm, swap: use an enum to define all cluster flags and wrap flags changes Date: Tue, 24 Dec 2024 22:38:06 +0800 Message-ID: <20241224143811.33462-9-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241224143811.33462-1-ryncsn@gmail.com> References: <20241224143811.33462-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Stat-Signature: aqajpe71cseh8kin519dh5js15wf8jpn X-Rspamd-Queue-Id: 6B70510001B X-Rspam-User: X-HE-Tag: 1735051160-601289 X-HE-Meta: U2FsdGVkX1/SmMCzNbQjfOnqT6JHaOCb4QDQHvaFkyhs4RPEr8nTre2Nwx4oLWPKzs0t444pq31c34PGJD8RZo3HwvJMPOTmUyXCUp+EsmERIeUcZyVbbGwPfUKWUJQSIOwggAjuSAajbgo0FcbWmKOltb0R1xOhJONjPLMpz7v/dzZtlVd5U6R+wCoe03DiZUNKD0taNwDGai39/ddFltBeE3f0YmoGEuRBDGKaM+uFvZmSncsDlbeX7y5YEgfnUMhgwE0FhaJUEhd4+GNRmVT32x4p0O7uiNCuDEsvFB2XrorHM/gR+vlDvhPX1cX+nLWEmU++F6tCy4GWWVKwf27s5AaTwg+3QSQihp4ujz37DPtYhKeLoRur07HgIAORAkHjncsoCDXIAE0yi5OCScG6Sc8y8dKu2a7SwM4sty6mhwoUemjMSvV7P9MU8wxGMWSEMDdmY6NlWDB7scUYjNJTblK7f5NGLFoGGR4tPVPLqXOjy2VqFuvmssJZkTe+Ak0xy+rcOhi96xoklRqSAAl1w9MgcHjSpJ/VSVOj0EignCqHpLSsiBGb54UGGwBsmPk97sEKFdqqy5aog5+t/jFWbavhqG5qUoHWh1U/un9yu1Fu0g+DVGuMRMNCQWSlKkk+I7xrO9yeYD8syyfovmg79h48cK1vD5YggkTDQIo5VCQwf7xoQvUOpFXs4WFGN8HGI6Vj/zRO7drAGpREcQDrZgYHHAo242IVGgSvy7v/ooHHk33J/2r2w+Mnfne2NlIsxwpr94s7xtvv30iLDkhYvT2qRR9ecV27NxsALjBtHimD4FkKGaHvhBTbUVMhf83cyX832LeRD8YsQT1pV+HP/CfqQVbdb1b7nTJ2JMcQeR+wTe3Rjyecdbz6xM2EYunVODwKRoS/AFhmRF755dYghC2iC2UNnviXtAPkVabdijINZ7USAYLwJCHugid1Ww/rptEapRiRvDz3AG/ sf16sl/5 wYXzByrvLb+HjtLs9VvWMiFbcEh3rSF/YBwh93q8xKoEjThNx9yTfl+KNf/nsvPCrZ1Xzjke2bEaX+l8nVOwhQezzhQk1r2CtjE2swBvA5VmVG9XAPX/dnXM0MuxY5+npjy7IgCZ83x2DYhrpSWl9+1BvvC3mIVzpz5uUEBY1JfY5nuuvYu2Sg4Ap6rkbtE3g8jS9Rhr6w+fvTmZtC570z5oqlph3fKtcAb1KvRe2SpbIP5YXgTYaWEtz6iS8Hy82t+oOkaOQptNcFVbK5CxVANOPjvUjbLiSCGlSwvQY2RHknHe2GFkYd2DrWiZb3Tl+07xJx0yImXXoSjHSQfBNYXDIr89dkhwUDvbUwac2ikGyqzAK0sBgWouPN7iv3IKsT9QBgCdutJXLLLqprUiTCNNcoh3DzP2FerzV75e5oN2Hhfji1Kp47RGIVqesheSFaq+JskS1ubgrTWLT5/Yt/dXZuIyTcDqyz+x3RHn0HSagzT5gjRsUgRTjDhgkyLC82hvbic0yMCs/kAC+wVUvKK4NN9clmvMgGyGotlLhNVUfP9M= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000860, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Currently, we are only using flags to indicate which list the cluster is on. Using one bit for each list type might be a waste, as the list type grows, we will consume too many bits. Additionally, the current mixed usage of '&' and '==' is a bit confusing. Make it clean by using an enum to define all possible cluster statuses. Only an off-list cluster will have the NONE (0) flag. And use a wrapper to annotate and sanitize all flag settings and list movements. Suggested-by: Chris Li Signed-off-by: Kairui Song --- include/linux/swap.h | 17 +++++++--- mm/swapfile.c | 75 +++++++++++++++++++++++--------------------- 2 files changed, 52 insertions(+), 40 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 02120f1005d5..339d7f0192ff 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -257,10 +257,19 @@ struct swap_cluster_info { u8 order; struct list_head list; }; -#define CLUSTER_FLAG_FREE 1 /* This cluster is free */ -#define CLUSTER_FLAG_NONFULL 2 /* This cluster is on nonfull list */ -#define CLUSTER_FLAG_FRAG 4 /* This cluster is on nonfull list */ -#define CLUSTER_FLAG_FULL 8 /* This cluster is on full list */ + +/* All on-list cluster must have a non-zero flag. */ +enum swap_cluster_flags { + CLUSTER_FLAG_NONE = 0, /* For temporary off-list cluster */ + CLUSTER_FLAG_FREE, + CLUSTER_FLAG_NONFULL, + CLUSTER_FLAG_FRAG, + /* Clusters with flags above are allocatable */ + CLUSTER_FLAG_USABLE = CLUSTER_FLAG_FRAG, + CLUSTER_FLAG_FULL, + CLUSTER_FLAG_DISCARD, + CLUSTER_FLAG_MAX, +}; /* * The first page in the swap file is the swap header, which is always marked diff --git a/mm/swapfile.c b/mm/swapfile.c index 0abff343f5f0..be2c719a51bb 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -403,7 +403,7 @@ static void discard_swap_cluster(struct swap_info_struct *si, static inline bool cluster_is_free(struct swap_cluster_info *info) { - return info->flags & CLUSTER_FLAG_FREE; + return info->flags == CLUSTER_FLAG_FREE; } static inline unsigned int cluster_index(struct swap_info_struct *si, @@ -434,6 +434,27 @@ static inline void unlock_cluster(struct swap_cluster_info *ci) spin_unlock(&ci->lock); } +static void cluster_move(struct swap_info_struct *si, + struct swap_cluster_info *ci, struct list_head *list, + enum swap_cluster_flags new_flags) +{ + VM_WARN_ON(ci->flags == new_flags); + BUILD_BUG_ON(1 << sizeof(ci->flags) * BITS_PER_BYTE < CLUSTER_FLAG_MAX); + + if (ci->flags == CLUSTER_FLAG_NONE) { + list_add_tail(&ci->list, list); + } else { + if (ci->flags == CLUSTER_FLAG_FRAG) { + VM_WARN_ON(!si->frag_cluster_nr[ci->order]); + si->frag_cluster_nr[ci->order]--; + } + list_move_tail(&ci->list, list); + } + ci->flags = new_flags; + if (new_flags == CLUSTER_FLAG_FRAG) + si->frag_cluster_nr[ci->order]++; +} + /* Add a cluster to discard list and schedule it to do discard */ static void swap_cluster_schedule_discard(struct swap_info_struct *si, struct swap_cluster_info *ci) @@ -447,10 +468,8 @@ static void swap_cluster_schedule_discard(struct swap_info_struct *si, */ memset(si->swap_map + idx * SWAPFILE_CLUSTER, SWAP_MAP_BAD, SWAPFILE_CLUSTER); - - VM_BUG_ON(ci->flags & CLUSTER_FLAG_FREE); - list_move_tail(&ci->list, &si->discard_clusters); - ci->flags = 0; + VM_BUG_ON(ci->flags == CLUSTER_FLAG_FREE); + cluster_move(si, ci, &si->discard_clusters, CLUSTER_FLAG_DISCARD); schedule_work(&si->discard_work); } @@ -458,12 +477,7 @@ static void __free_cluster(struct swap_info_struct *si, struct swap_cluster_info { lockdep_assert_held(&si->lock); lockdep_assert_held(&ci->lock); - - if (ci->flags) - list_move_tail(&ci->list, &si->free_clusters); - else - list_add_tail(&ci->list, &si->free_clusters); - ci->flags = CLUSTER_FLAG_FREE; + cluster_move(si, ci, &si->free_clusters, CLUSTER_FLAG_FREE); ci->order = 0; } @@ -479,6 +493,8 @@ static void swap_do_scheduled_discard(struct swap_info_struct *si) while (!list_empty(&si->discard_clusters)) { ci = list_first_entry(&si->discard_clusters, struct swap_cluster_info, list); list_del(&ci->list); + /* Must clear flag when taking a cluster off-list */ + ci->flags = CLUSTER_FLAG_NONE; idx = cluster_index(si, ci); spin_unlock(&si->lock); @@ -519,9 +535,6 @@ static void free_cluster(struct swap_info_struct *si, struct swap_cluster_info * lockdep_assert_held(&si->lock); lockdep_assert_held(&ci->lock); - if (ci->flags & CLUSTER_FLAG_FRAG) - si->frag_cluster_nr[ci->order]--; - /* * If the swap is discardable, prepare discard the cluster * instead of free it immediately. The cluster will be freed @@ -573,13 +586,9 @@ static void dec_cluster_info_page(struct swap_info_struct *si, return; } - if (!(ci->flags & CLUSTER_FLAG_NONFULL)) { - VM_BUG_ON(ci->flags & CLUSTER_FLAG_FREE); - if (ci->flags & CLUSTER_FLAG_FRAG) - si->frag_cluster_nr[ci->order]--; - list_move_tail(&ci->list, &si->nonfull_clusters[ci->order]); - ci->flags = CLUSTER_FLAG_NONFULL; - } + if (ci->flags != CLUSTER_FLAG_NONFULL) + cluster_move(si, ci, &si->nonfull_clusters[ci->order], + CLUSTER_FLAG_NONFULL); } static bool cluster_reclaim_range(struct swap_info_struct *si, @@ -663,11 +672,13 @@ static bool cluster_alloc_range(struct swap_info_struct *si, struct swap_cluster if (!(si->flags & SWP_WRITEOK)) return false; + VM_BUG_ON(ci->flags == CLUSTER_FLAG_NONE); + VM_BUG_ON(ci->flags > CLUSTER_FLAG_USABLE); + if (cluster_is_free(ci)) { - if (nr_pages < SWAPFILE_CLUSTER) { - list_move_tail(&ci->list, &si->nonfull_clusters[order]); - ci->flags = CLUSTER_FLAG_NONFULL; - } + if (nr_pages < SWAPFILE_CLUSTER) + cluster_move(si, ci, &si->nonfull_clusters[order], + CLUSTER_FLAG_NONFULL); ci->order = order; } @@ -675,14 +686,8 @@ static bool cluster_alloc_range(struct swap_info_struct *si, struct swap_cluster swap_range_alloc(si, nr_pages); ci->count += nr_pages; - if (ci->count == SWAPFILE_CLUSTER) { - VM_BUG_ON(!(ci->flags & - (CLUSTER_FLAG_FREE | CLUSTER_FLAG_NONFULL | CLUSTER_FLAG_FRAG))); - if (ci->flags & CLUSTER_FLAG_FRAG) - si->frag_cluster_nr[ci->order]--; - list_move_tail(&ci->list, &si->full_clusters); - ci->flags = CLUSTER_FLAG_FULL; - } + if (ci->count == SWAPFILE_CLUSTER) + cluster_move(si, ci, &si->full_clusters, CLUSTER_FLAG_FULL); return true; } @@ -821,9 +826,7 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si, int o while (!list_empty(&si->nonfull_clusters[order])) { ci = list_first_entry(&si->nonfull_clusters[order], struct swap_cluster_info, list); - list_move_tail(&ci->list, &si->frag_clusters[order]); - ci->flags = CLUSTER_FLAG_FRAG; - si->frag_cluster_nr[order]++; + cluster_move(si, ci, &si->frag_clusters[order], CLUSTER_FLAG_FRAG); offset = alloc_swap_scan_cluster(si, cluster_offset(si, ci), &found, order, usage); frags++;