From patchwork Wed Oct 2 01:20:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13819172 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A65BCF318A for ; Wed, 2 Oct 2024 01:20:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 82BBE680048; Tue, 1 Oct 2024 21:20:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7DA57680046; Tue, 1 Oct 2024 21:20:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B88F440168; Tue, 1 Oct 2024 21:20:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 24E22680046 for ; Tue, 1 Oct 2024 21:20:47 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BBDFC1C7407 for ; Wed, 2 Oct 2024 01:20:46 +0000 (UTC) X-FDA: 82626907692.21.1FFAEFD Received: from mail-yb1-f169.google.com (mail-yb1-f169.google.com [209.85.219.169]) by imf20.hostedemail.com (Postfix) with ESMTP id DE2D21C0002 for ; Wed, 2 Oct 2024 01:20:44 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HDJ7UH2B; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf20.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.219.169 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727831941; a=rsa-sha256; cv=none; b=FdkUOFtEK5qwuzbTBeDVnUfoCZkLxkjUupC3Iu2mC0R1MYQYSdcNGHiwIcK71AX2QsNCeU ZFuq6cYwcL7d6O4B3ASVKUcoK8XaQofc2vFQ6yluOQtkuSnLmAXRpP349Stoy8pdKCPEl7 uPZ5mIobqc4KNdcIqmTGN/hMU0KFeVY= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HDJ7UH2B; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf20.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.219.169 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727831941; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=utFR9zmlqHgm/5JvxynF3o1HACmZNmPmIIYRg2mm7i0=; b=79PM05RUbHeb5zip9dM8jfby1iwIeWAdhEbKgOg9Z6rYyKAd+WYfLVidDputkFMQaynwk8 VnHcSEsYvMCZv2mdz4uGN9CkzyLlP1zPN0OKnhqn38yshbDFLz6LLxrEmwLmZ+3MXa84lG QsKumovk+c+LdrJtD341HVI89fSAIv4= Received: by mail-yb1-f169.google.com with SMTP id 3f1490d57ef6-dff1ccdc17bso6209932276.0 for ; Tue, 01 Oct 2024 18:20:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727832044; x=1728436844; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=utFR9zmlqHgm/5JvxynF3o1HACmZNmPmIIYRg2mm7i0=; b=HDJ7UH2BNvBofe0BikSwx+b72gKejNKIWu46RWnt/cJQTotlqC+wYx89wqsIQaOHsx vjTrQF1r0r2HBUGEg4nyHjGUD9HM77sR+P4PIMdeKG/UFnxBfWEhh8ShgXFEWNHQMteM /EblMc1wGvR+1xL3WrktueOcDjyrh67Rf5NaD2NNIqjO4yXXTV/fd+FC05VFZxoLLohK 46AGWuycUoVSU7Wj/sDoCD7SI/1sLwUdv1x2COXKaBzvwlvP6K29zOAcJfQ99QJiBxnw ty32YQrr2CGM3V1bhXBRgy6iVw+yWhivMg13ERH9nlDyGyGjeDde6TMt1XoBondAe83d 0ZAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727832044; x=1728436844; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=utFR9zmlqHgm/5JvxynF3o1HACmZNmPmIIYRg2mm7i0=; b=BAK04dc9GtlEQNTqXP2CJ9cg+jyGS5mIj4kSeGhh0cd8RsdUZklB/4BQmbdHevg528 +g57X8z6XMkViL1IQug4iqMeTB7wm4OeJBzX0W3hMbmLOiDoMoEr9e/JgMTPZkebkItp MSGTMMBEG+nL+HGIVeZtH3aZUDyALGxioBAmE/ZvqRkZrltLezmcWm6G2MzZIhble+nv eX8rLgZhv4CFtp8R/RA0p0Kzv65NrN1qxyEp060ouMK1jI2eo7nz9R7NfSX8wgZpz5xn l9JZqzhbLYuEs4mPSRUx33tSFOZbAGiQnmVrZ14gHjho8a8SqkLYl8/TVYaJmm36kC8V /eRA== X-Forwarded-Encrypted: i=1; AJvYcCUcR/hcLVpXUiuK4RO0sgd2Id/C8cPDfhq/2aT9o5nWCdwPw0a4//mY40spCfbiT+EObN8s8q9AnA==@kvack.org X-Gm-Message-State: AOJu0Yz0yDZjJT8VWglIxxfvBtfWU1SPZtqvi8iYPoNbFAvROK6W8pzr kLuxYxsN1bJ0/7/KB59OUNGKi0MbCLX/w9vMrHXDaGyxoi8pa+wL X-Google-Smtp-Source: AGHT+IHtucbPLaNpcrLB+B6VLBX8j1Ni5DiJX6AZ+B2nx3ZvFau3S4sggAqq8GnraKziEGfIygLpSQ== X-Received: by 2002:a05:690c:2a41:b0:6db:db51:c02d with SMTP id 00721157ae682-6e2a2e088demr14041587b3.25.1727832043859; Tue, 01 Oct 2024 18:20:43 -0700 (PDT) Received: from localhost (fwdproxy-nha-116.fbsv.net. [2a03:2880:25ff:74::face:b00c]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6e24536992dsm22244397b3.96.2024.10.01.18.20.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Oct 2024 18:20:43 -0700 (PDT) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, yosryahmed@google.com, hughd@google.com, shakeel.butt@linux.dev, ryan.roberts@arm.com, ying.huang@intel.com, chrisl@kernel.org, david@redhat.com, kasong@tencent.com, willy@infradead.org, viro@zeniv.linux.org.uk, baohua@kernel.org, chengming.zhou@linux.dev, v-songbaohua@oppo.com, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/1] swap: shmem: remove SWAP_MAP_SHMEM Date: Tue, 1 Oct 2024 18:20:42 -0700 Message-ID: <20241002012042.2753174-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241002012042.2753174-1-nphamcs@gmail.com> References: <20241002012042.2753174-1-nphamcs@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: DE2D21C0002 X-Stat-Signature: zegwdj171y5drh6x74mss1tm99g8waog X-Rspam-User: X-HE-Tag: 1727832044-754485 X-HE-Meta: U2FsdGVkX1+BaLZvjb53YouJjWLStgWxG6JMmF+NlusKKyT42Zo/E8tKfN0IcBdFc6Sas6u7rOts7iDSJ75cccKu+CN+jU0iEbdcpCqhp4dGahXjiIWHVC3QxbStQCKgVjYu6Y1bBNbG8qlB+A4N5Gj+oVyhaq2XZx3++HfkWu7IEhG3/BKifA5UWRcwu0/Hi5Pxma0gi1pMmcXyRf8axQTUNVaO7mdlTOcIgf/mBbFU6xw6TJsnpQ/P1aYO0WloNicOd1luQ36QBOvFsQ0ywNVLaLJsSRCi0Laf455wRHTh2s4GBkDk5+AxU0PMc2q/ebmfA/8ITclBHUTp4KSvonH0DMJM1N7fBxF8qSuae09Et3GqS2789kN8O1eNCFauFS+SJEePS7aY+yqipmd3mRF/G95UgWUI7xawTuKqmCxXt7S5otWH6sXiiAfTjIZWdRMoGVwnmI92hFgnbNqORk4vP1iksjqi3s3aLnXz5geTEkYae4Z/GIrtVZnHR4tPZfGlU4/PttMeNS70m6XQyhcf74zuavQB1BxCL/PvMt6H8cJoWMSmXS6KIO2IflO7EhtorsrskyccTTcXg88hYzjP2m1QzShV4uknngTkS7/qLTyLmfh0RClR3NzYsKasntrABRvoyP9wL2Wnov3fQOMBBya7L9t960GbFjY/kEH3ez03Rw4kJDukl4biytg0z05j0dRMMU5vEVzRVf312arhE6OtUOjCKXIfHTRU0MRhAMlBW0fZ19jpCoSMtD7zF4bj7QsMkzWhFRo/uEugT6/kvvI4JvyL5fq4DsWL7bD4gTgMm2JylRQ7Z5BVOf6fsdFxH7rnjIf4i/aESfSxk9u2dqHo6xfN3AWcZ8gMlRyyfk0X2EF9OHlRnmDFCNw+utDVAqbrcaC1bvUFkynWb1Q4EknUUCdkR7beTdG9lcQlp9Pq6V+KlU3NGC+ufqYoFoRklSRhlcJjcggCvSc f815pQIr naoounPYIDbmdLYGb+FdgYo0ExEejTSqOoaJH8R689hgM9yy9Pmvja/ZuqPLJ0IZ0uiH5cArVjPGE+k7H0+5RJwDOPpm0UymzFQUs2xdUc2n3Ro93roamV6tUbW5LBVhUZOZbbT4fNY4lpVHEYtulg8W5vbub4bbGe8WJ0ZJh2RjOg2Qff4GhBAkZZYkCCLkX4C8n6euAhqtCcaQ7mgj6BcP9WtFoNheZpoXoGGYkDSkhQca+ifYqec2FxGQWe15vmmecNm9bzra16baF0B36XLQ5+Sn6Ba8e0udN5G8DJF0ce1YihPo/QotqudJXSmN/45U3/z0MwBUF0xNPJSCqhsb/TinTqOyfaxyGUIwIdtqGm9h9F4+wK938parN3fBHhFBhG0VcYIaIXelkqFglgr56zD3O6iTU/EXQvBhXuHh5SFox+6S9sx7jbltZeiDhmUWt X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The SWAP_MAP_SHMEM state was introduced in the commit aaa468653b4a ("swap_info: note SWAP_MAP_SHMEM"), to quickly determine if a swap entry belongs to shmem during swapoff. However, swapoff has since been rewritten in the commit b56a2d8af914 ("mm: rid swapoff of quadratic complexity"). Now having swap count == SWAP_MAP_SHMEM value is basically the same as having swap count == 1, and swap_shmem_alloc() behaves analogously to swap_duplicate(). The only difference of note is that swap_shmem_alloc() does not check for -ENOMEM returned from __swap_duplicate(), but it is OK because shmem never re-duplicates any swap entry it owns. This will stil be safe if we use (batched) swap_duplicate() instead. This commit adds swap_duplicate_nr(), the batched variant of swap_duplicate(), and removes the SWAP_MAP_SHMEM state and the associated swap_shmem_alloc() helper to simplify the state machine (both mentally and in terms of actual code). We will also have an extra state/special value that can be repurposed (for swap entries that never gets re-duplicated). Signed-off-by: Nhat Pham --- include/linux/swap.h | 16 ++++++++-------- mm/shmem.c | 2 +- mm/swapfile.c | 41 +++++++++++++++++++++-------------------- 3 files changed, 30 insertions(+), 29 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index ca533b478c21..017f3c03ff7a 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -232,7 +232,6 @@ enum { /* Special value in first swap_map */ #define SWAP_MAP_MAX 0x3e /* Max count */ #define SWAP_MAP_BAD 0x3f /* Note page is bad */ -#define SWAP_MAP_SHMEM 0xbf /* Owned by shmem/tmpfs */ /* Special value in each swap_map continuation */ #define SWAP_CONT_MAX 0x7f /* Max count */ @@ -482,8 +481,7 @@ void put_swap_folio(struct folio *folio, swp_entry_t entry); extern swp_entry_t get_swap_page_of_type(int); extern int get_swap_pages(int n, swp_entry_t swp_entries[], int order); extern int add_swap_count_continuation(swp_entry_t, gfp_t); -extern void swap_shmem_alloc(swp_entry_t, int); -extern int swap_duplicate(swp_entry_t); +extern int swap_duplicate_nr(swp_entry_t, int); extern int swapcache_prepare(swp_entry_t entry, int nr); extern void swap_free_nr(swp_entry_t entry, int nr_pages); extern void swapcache_free_entries(swp_entry_t *entries, int n); @@ -549,11 +547,7 @@ static inline int add_swap_count_continuation(swp_entry_t swp, gfp_t gfp_mask) return 0; } -static inline void swap_shmem_alloc(swp_entry_t swp, int nr) -{ -} - -static inline int swap_duplicate(swp_entry_t swp) +static inline int swap_duplicate_nr(swp_entry_t swp, int nr) { return 0; } @@ -606,6 +600,12 @@ static inline int add_swap_extent(struct swap_info_struct *sis, } #endif /* CONFIG_SWAP */ +static inline int swap_duplicate(swp_entry_t entry) +{ + return swap_duplicate_nr(entry, 1); +} + + static inline void free_swap_and_cache(swp_entry_t entry) { free_swap_and_cache_nr(entry, 1); diff --git a/mm/shmem.c b/mm/shmem.c index 0613421e09e7..e3f72f99be32 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1561,7 +1561,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN, NULL) == 0) { shmem_recalc_inode(inode, 0, nr_pages); - swap_shmem_alloc(swap, nr_pages); + swap_duplicate_nr(swap, nr_pages); shmem_delete_from_page_cache(folio, swp_to_radix_entry(swap)); mutex_unlock(&shmem_swaplist_mutex); diff --git a/mm/swapfile.c b/mm/swapfile.c index 0cded32414a1..9bb94e618914 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1381,12 +1381,6 @@ static unsigned char __swap_entry_free_locked(struct swap_info_struct *si, if (usage == SWAP_HAS_CACHE) { VM_BUG_ON(!has_cache); has_cache = 0; - } else if (count == SWAP_MAP_SHMEM) { - /* - * Or we could insist on shmem.c using a special - * swap_shmem_free() and free_shmem_swap_and_cache()... - */ - count = 0; } else if ((count & ~COUNT_CONTINUED) <= SWAP_MAP_MAX) { if (count == COUNT_CONTINUED) { if (swap_count_continued(si, offset, count)) @@ -3626,7 +3620,6 @@ static int __swap_duplicate(swp_entry_t entry, unsigned char usage, int nr) offset = swp_offset(entry); VM_WARN_ON(nr > SWAPFILE_CLUSTER - offset % SWAPFILE_CLUSTER); - VM_WARN_ON(usage == 1 && nr > 1); ci = lock_cluster_or_swap_info(si, offset); err = 0; @@ -3652,6 +3645,13 @@ static int __swap_duplicate(swp_entry_t entry, unsigned char usage, int nr) err = -EEXIST; } else if ((count & ~COUNT_CONTINUED) > SWAP_MAP_MAX) { err = -EINVAL; + } else { + /* + * The only swap_duplicate_nr() caller that passes nr > 1 is shmem, + * who never re-duplicates any swap entry it owns. So this should + * not happen. + */ + VM_WARN_ON(nr > 1 && (count & ~COUNT_CONTINUED) == SWAP_MAP_MAX); } if (err) @@ -3686,27 +3686,28 @@ static int __swap_duplicate(swp_entry_t entry, unsigned char usage, int nr) return err; } -/* - * Help swapoff by noting that swap entry belongs to shmem/tmpfs - * (in which case its reference count is never incremented). - */ -void swap_shmem_alloc(swp_entry_t entry, int nr) -{ - __swap_duplicate(entry, SWAP_MAP_SHMEM, nr); -} - -/* - * Increase reference count of swap entry by 1. +/** + * swap_duplicate_nr() - Increase reference count of nr contiguous swap entries + * by 1. + * + * @entry: first swap entry from which we want to increase the refcount. + * @nr: Number of entries in range. + * * Returns 0 for success, or -ENOMEM if a swap_count_continuation is required * but could not be atomically allocated. Returns 0, just as if it succeeded, * if __swap_duplicate() fails for another reason (-EINVAL or -ENOENT), which * might occur if a page table entry has got corrupted. + * + * Note that we are currently not handling the case where nr > 1 and we need to + * add swap count continuation. This is OK, because no such user exists - shmem + * is the only user that can pass nr > 1, and it never re-duplicates any swap + * entry it owns. */ -int swap_duplicate(swp_entry_t entry) +int swap_duplicate_nr(swp_entry_t entry, int nr) { int err = 0; - while (!err && __swap_duplicate(entry, 1, 1) == -ENOMEM) + while (!err && __swap_duplicate(entry, 1, nr) == -ENOMEM) err = add_swap_count_continuation(entry, GFP_ATOMIC); return err; }