From patchwork Tue Oct 22 19:24:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13846082 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DCBDCDD0D7 for ; Tue, 22 Oct 2024 19:30:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BFE5A6B009C; Tue, 22 Oct 2024 15:30:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BAAB86B009D; Tue, 22 Oct 2024 15:30:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A244F6B009E; Tue, 22 Oct 2024 15:30:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 78C3F6B009C for ; Tue, 22 Oct 2024 15:30:17 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id DBE6DA0469 for ; Tue, 22 Oct 2024 19:29:46 +0000 (UTC) X-FDA: 82702228770.30.2F572FD Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf02.hostedemail.com (Postfix) with ESMTP id 0974E8000C for ; Tue, 22 Oct 2024 19:29:43 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=fcJVvfoa; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729625364; a=rsa-sha256; cv=none; b=DYbxw4VvcJk2bbobEPXYIKi0ktSRufRlaVBX0v9px79Gkgcj1C3KlhbPxeW3vI2xUz254/ Wh8IoqYy0cY7VLP6six1ni5igIGUklsFcZz+qAKJlwjEOhKnyBo1KEn+dP/hMBN2t2I7hb nTwxcagyAMYfyo56h3FkexGx6vIf6jM= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=fcJVvfoa; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729625364; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C0u3UHnFQamHvHumiMWfS42vAHn/bzcqM1FVrGPT7rA=; b=gH9qyo48d6gg7+SBSqO0FfCxnDmB4hubA7qh8YBVvUbp9VyXTygD/zrSAKchMdss4+w2Us NC8uTBrFH8x26DIu9TNH5cR5DLYrf6zYiotpj63QoXQ0unZZUTmQdrFjFR9Kjt7rXxYUSt +o4JGdbeQGx9I2rYRO21XlUwPR3uutw= Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-20ceb8bd22fso51544595ad.3 for ; Tue, 22 Oct 2024 12:30:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729625413; x=1730230213; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=C0u3UHnFQamHvHumiMWfS42vAHn/bzcqM1FVrGPT7rA=; b=fcJVvfoaQRhoI/x3hY6PgCjnlwbMvoHsgvDTy1xRCjylYtbVFvWRCcvvmehLt2hZrm ua6nAFmUVXNQPNFBJInYDUDMx/GB5BgvkmxGgkNZ13hJA0djlI+tZfBCOq1Yh0FB5JWe 9vImoyC/nfLuhf2oMgMlY8ZqLPiCUAdyW7WY4dp8hEmmon7AU12D3zApW41NCnqpA0aR sSyLFVDWMUXoxcggQLZqFZpyq1xsGUvl9N0C960mXeZNgXP/fdAb8OHl7WWBLibQY4uv uV+J6vCHcXejJ+JTuHoh3dWI4D5gRahTvv6UTVON2IhL95o6zq8+gWuRMhbAGat6FUrt L0QQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729625413; x=1730230213; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=C0u3UHnFQamHvHumiMWfS42vAHn/bzcqM1FVrGPT7rA=; b=gffRlnoEvTfdk/QghNM8EUrVwCIY56NfVooBjYtEc3k+gndSd+lsVEhS7ZZoyffArP n2EpVUHQGgmWb9+IsaJmj/q/aVNXjRmxmlkWDapGSm4cTPqahTxHIlM/rtzVyGb8KOi5 ienUxKExNiob7uQ7l8pHh8wiaw/kbLqIBbJGblfcnI+VAJA+uwUFuc+ulWFJb1wecbLD 3caPzPXhN+BZWEcs1rqbjBLu39eU9ggAYLWyNPiZn3yFlhJ1rzhU5bHsDKkKjF9vwH8a gRo0peaqG0Zlt92Sv9Iq5YPjvd0+kw0lDTuNjmqdihbBifnv47Viu3OhjS7eYXf7V5L5 n58g== X-Gm-Message-State: AOJu0YyK3OMCoV3RRV/DX0T0P6mWnJ6AsRqfUIvV54e81DxxTeWswhTX tseFg8hT2K4Ke2DPiuPRXesPVo00TA3Wj3S0zdpckcpQUmNmr8PXsFqIfuOuqDQ= X-Google-Smtp-Source: AGHT+IGcRlQHedKQaqyDIKAfCIehFDARjRb4U2itQQ2f0zTHaoGQCyPSUA4jWG4iyNeB5YMGTc2MgA== X-Received: by 2002:a17:902:db12:b0:20c:e875:12b5 with SMTP id d9443c01a7336-20fab3276b7mr2519545ad.59.1729625413398; Tue, 22 Oct 2024 12:30:13 -0700 (PDT) Received: from KASONG-MC4.tencent.com ([106.37.123.36]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20e7f0d9f05sm45895305ad.186.2024.10.22.12.30.09 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 22 Oct 2024 12:30:12 -0700 (PDT) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Barry Song , Ryan Roberts , Hugh Dickins , Yosry Ahmed , "Huang, Ying" , Tim Chen , Nhat Pham , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 08/13] mm, swap: use an enum to define all cluster flags and wrap flags changes Date: Wed, 23 Oct 2024 03:24:46 +0800 Message-ID: <20241022192451.38138-9-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241022192451.38138-1-ryncsn@gmail.com> References: <20241022192451.38138-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 0974E8000C X-Rspamd-Server: rspam01 X-Stat-Signature: uo1zjt7oeinzmpgcbnqawjkuu9nmdnxt X-HE-Tag: 1729625383-911352 X-HE-Meta: U2FsdGVkX1/oREuE73ag47fqseRYuumYYrGguP6awgZ5B8gCvEjUgZapbmYCYn/s/yxGGDltPoB+XHLlDzBDqVsQFL3raEI2Iq7VZqJsSuVSpLeHjh++BMgw3RuH75oOcLeHS4FZ6P8/k4T2fJSWOQrtWJ4GvoATIQMSAdFGLAhy2pFcV9er/Pp6S/xW6LsDuAX/opSledkuses0iJY/0wdIuHVyVzGKeGXjDW0MLzlFPOGjP2JwsOn3aMppFZWWLJIK+Cr0jW2p/ZyzigY+MYxyW8FrwkeMbu5/98xQFswSqVKIsvoKjXHBEOtwOVa7ndYjKwfBC3sCpBp84a1x5xE0p9Nxo02P9WOv/cSzBgWrUC8rE+KjYTeavGZGkc3ts87rTJ0QNQJp7CCp6vjyl/8TYFfxQsq7pYtDIDjg7NG+pLOYfoWDegBBM6gXqjEXErf9QQ0dKx8OGz+kvMTP397yhxf56vW8FV+3629nAWSt2Mt1fo49qR5KfZy4aplFIkM/b30FWvgzcG8BBgWQJ01qkGpq1YU095nBfWQJt08QVjsjYMStpJdXjkkK/n6hRJPFrlNmrZawOYR8fJbiprG7kpKtUfF2IdvVgthVgKaYzPBUHmfiJcRP6L50KDp+vNyIEDSu/QjQOcUQ9h9FtWFrE61x9UTMQiM8n3TtpPnyHZBxVvviF7Ll0UIVTWXGPmDcle6igflDM5nT3xBr2xLBOFo4y2WFdnDVqTt/Mw3jbgAMu72hlBX5Ka8GQX5H7ZS8y+ynWYK8iT9tC0De7VZ/6Qp4tGd5JQJEBRGQGktB//Vcf0HebSDyQ8c8YqIBW6z1WvS+/l5gcTMQuz6A/0MKV1FkeTcXFXqHZiHRPx85vnhHQfLlVzykIgpG+hbS00HE6eITST9F2A8pdAhCw9k6JNyCyecHZenLaSvQq5QaWMiX/qQQiORlUr5ebn8pYOK++M4WkTc+OY+Kl4x wudTbMOR l7Aioj9WbSFQ0dWY19jEVTSQDIZOEstLgS8MqvUPBoVDbeC1gmejpn1bOWj4dumMZ/GVEOIbXEEKt3LX2zrpo0SdwJNjo6gqiifYwsq31lqV//VZK9VU4WH13QO/r7U8iFy1ldEgqzcoMtyOtBSvv6IrcYom1atZSYXzRak1XGHBxvCTxFft6c7q2XaOGmqfrSvUilRQ1dwqqLirKN7PEgmJf+VRrqjIoPLIvTOCOkV665/nT0FsLPVKa7I1Np0viYu8/akNpKDVBnlZVSzYK9u+jGAYJUPW6rS69/coIVE3tGW5HM8JTZ9EXXERuU4K76TkClOrrDxxAfRcRZZA/6IM46xvv9IQaVmFctBaLB0qP3vyh+kJH3xNYvhcwu/92y2EJSSs8BFfq9++lOrA+3qZcX5olUaUY1jOir0qeNU4pgj2cc852snlaDy3o7tkz8zLs0/I5qIw3G9uHQHvUVtxQwVbG/wvE08nMmnhS2FOW2kzX8FeLSqNhm6R+bs8faoNqrQpC94oDon/D6pTHOd1yPcN3kyNvippZmssi4NQiUAM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Currently we are only using flags to indicate which list the cluster is on, using one bit for each list type might be a waste as the list type grows we will consume too much bits. And current the mixed usage of "&" and "==" is a bit confusing. Make it clean by using an enum to define all possible cluster status, only an off-list cluster will have the NONE (0) flag. And use a wrapper to annotate and sanitize all flag setting and list movement. Suggested-by: Chris Li Signed-off-by: Kairui Song --- include/linux/swap.h | 17 +++++++--- mm/swapfile.c | 76 +++++++++++++++++++++++--------------------- 2 files changed, 53 insertions(+), 40 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 1651174959c8..75fc2da1767d 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -256,10 +256,19 @@ struct swap_cluster_info { u8 order; struct list_head list; }; -#define CLUSTER_FLAG_FREE 1 /* This cluster is free */ -#define CLUSTER_FLAG_NONFULL 2 /* This cluster is on nonfull list */ -#define CLUSTER_FLAG_FRAG 4 /* This cluster is on nonfull list */ -#define CLUSTER_FLAG_FULL 8 /* This cluster is on full list */ + +/* + * All on-list cluster must have a non-zero flag. + */ +enum swap_cluster_flags { + CLUSTER_FLAG_NONE = 0, /* For temporary off-list cluster */ + CLUSTER_FLAG_FREE, + CLUSTER_FLAG_NONFULL, + CLUSTER_FLAG_FRAG, + CLUSTER_FLAG_FULL, + CLUSTER_FLAG_DISCARD, + CLUSTER_FLAG_MAX, +}; /* * The first page in the swap file is the swap header, which is always marked diff --git a/mm/swapfile.c b/mm/swapfile.c index d6b6e71ccc19..96d8012b003c 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -402,7 +402,7 @@ static void discard_swap_cluster(struct swap_info_struct *si, static inline bool cluster_is_free(struct swap_cluster_info *info) { - return info->flags & CLUSTER_FLAG_FREE; + return info->flags == CLUSTER_FLAG_FREE; } static inline unsigned int cluster_index(struct swap_info_struct *si, @@ -433,6 +433,27 @@ static inline void unlock_cluster(struct swap_cluster_info *ci) spin_unlock(&ci->lock); } +static void cluster_move(struct swap_info_struct *si, + struct swap_cluster_info *ci, struct list_head *list, + enum swap_cluster_flags new_flags) +{ + VM_WARN_ON(ci->flags == new_flags); + BUILD_BUG_ON(1 << sizeof(ci->flags) * BITS_PER_BYTE < CLUSTER_FLAG_MAX); + + if (ci->flags == CLUSTER_FLAG_NONE) { + list_add_tail(&ci->list, list); + } else { + if (ci->flags == CLUSTER_FLAG_FRAG) { + VM_WARN_ON(!si->frag_cluster_nr[ci->order]); + si->frag_cluster_nr[ci->order]--; + } + list_move_tail(&ci->list, list); + } + ci->flags = new_flags; + if (new_flags == CLUSTER_FLAG_FRAG) + si->frag_cluster_nr[ci->order]++; +} + /* Add a cluster to discard list and schedule it to do discard */ static void swap_cluster_schedule_discard(struct swap_info_struct *si, struct swap_cluster_info *ci) @@ -446,10 +467,8 @@ static void swap_cluster_schedule_discard(struct swap_info_struct *si, */ memset(si->swap_map + idx * SWAPFILE_CLUSTER, SWAP_MAP_BAD, SWAPFILE_CLUSTER); - - VM_BUG_ON(ci->flags & CLUSTER_FLAG_FREE); - list_move_tail(&ci->list, &si->discard_clusters); - ci->flags = 0; + VM_BUG_ON(ci->flags == CLUSTER_FLAG_FREE); + cluster_move(si, ci, &si->discard_clusters, CLUSTER_FLAG_DISCARD); schedule_work(&si->discard_work); } @@ -457,12 +476,7 @@ static void __free_cluster(struct swap_info_struct *si, struct swap_cluster_info { lockdep_assert_held(&si->lock); lockdep_assert_held(&ci->lock); - - if (ci->flags) - list_move_tail(&ci->list, &si->free_clusters); - else - list_add_tail(&ci->list, &si->free_clusters); - ci->flags = CLUSTER_FLAG_FREE; + cluster_move(si, ci, &si->free_clusters, CLUSTER_FLAG_FREE); ci->order = 0; } @@ -478,6 +492,8 @@ static void swap_do_scheduled_discard(struct swap_info_struct *si) while (!list_empty(&si->discard_clusters)) { ci = list_first_entry(&si->discard_clusters, struct swap_cluster_info, list); list_del(&ci->list); + /* Must clear flag when taking a cluster off-list */ + ci->flags = CLUSTER_FLAG_NONE; idx = cluster_index(si, ci); spin_unlock(&si->lock); @@ -518,9 +534,6 @@ static void free_cluster(struct swap_info_struct *si, struct swap_cluster_info * lockdep_assert_held(&si->lock); lockdep_assert_held(&ci->lock); - if (ci->flags & CLUSTER_FLAG_FRAG) - si->frag_cluster_nr[ci->order]--; - /* * If the swap is discardable, prepare discard the cluster * instead of free it immediately. The cluster will be freed @@ -572,13 +585,9 @@ static void dec_cluster_info_page(struct swap_info_struct *si, return; } - if (!(ci->flags & CLUSTER_FLAG_NONFULL)) { - VM_BUG_ON(ci->flags & CLUSTER_FLAG_FREE); - if (ci->flags & CLUSTER_FLAG_FRAG) - si->frag_cluster_nr[ci->order]--; - list_move_tail(&ci->list, &si->nonfull_clusters[ci->order]); - ci->flags = CLUSTER_FLAG_NONFULL; - } + if (ci->flags != CLUSTER_FLAG_NONFULL) + cluster_move(si, ci, &si->nonfull_clusters[ci->order], + CLUSTER_FLAG_NONFULL); } static bool cluster_reclaim_range(struct swap_info_struct *si, @@ -657,11 +666,14 @@ static void cluster_alloc_range(struct swap_info_struct *si, struct swap_cluster { unsigned int nr_pages = 1 << order; + VM_BUG_ON(ci->flags != CLUSTER_FLAG_FREE && + ci->flags != CLUSTER_FLAG_NONFULL && + ci->flags != CLUSTER_FLAG_FRAG); + if (cluster_is_free(ci)) { - if (nr_pages < SWAPFILE_CLUSTER) { - list_move_tail(&ci->list, &si->nonfull_clusters[order]); - ci->flags = CLUSTER_FLAG_NONFULL; - } + if (nr_pages < SWAPFILE_CLUSTER) + cluster_move(si, ci, &si->nonfull_clusters[order], + CLUSTER_FLAG_NONFULL); ci->order = order; } @@ -669,14 +681,8 @@ static void cluster_alloc_range(struct swap_info_struct *si, struct swap_cluster swap_range_alloc(si, nr_pages); ci->count += nr_pages; - if (ci->count == SWAPFILE_CLUSTER) { - VM_BUG_ON(!(ci->flags & - (CLUSTER_FLAG_FREE | CLUSTER_FLAG_NONFULL | CLUSTER_FLAG_FRAG))); - if (ci->flags & CLUSTER_FLAG_FRAG) - si->frag_cluster_nr[ci->order]--; - list_move_tail(&ci->list, &si->full_clusters); - ci->flags = CLUSTER_FLAG_FULL; - } + if (ci->count == SWAPFILE_CLUSTER) + cluster_move(si, ci, &si->full_clusters, CLUSTER_FLAG_FULL); } static unsigned int alloc_swap_scan_cluster(struct swap_info_struct *si, unsigned long offset, @@ -806,9 +812,7 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si, int o while (!list_empty(&si->nonfull_clusters[order])) { ci = list_first_entry(&si->nonfull_clusters[order], struct swap_cluster_info, list); - list_move_tail(&ci->list, &si->frag_clusters[order]); - ci->flags = CLUSTER_FLAG_FRAG; - si->frag_cluster_nr[order]++; + cluster_move(si, ci, &si->frag_clusters[order], CLUSTER_FLAG_FRAG); offset = alloc_swap_scan_cluster(si, cluster_offset(si, ci), &found, order, usage); frags++;