From patchwork Wed Jan 24 04:51:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13528534 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D669AC47E49 for ; Wed, 24 Jan 2024 04:51:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 70DBC6B007B; Tue, 23 Jan 2024 23:51:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6BECE6B007D; Tue, 23 Jan 2024 23:51:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 49B4E6B007E; Tue, 23 Jan 2024 23:51:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 3DC626B007B for ; Tue, 23 Jan 2024 23:51:20 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 18CC980B2E for ; Wed, 24 Jan 2024 04:51:20 +0000 (UTC) X-FDA: 81712980720.09.60C052A Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf07.hostedemail.com (Postfix) with ESMTP id 6715540008 for ; Wed, 24 Jan 2024 04:51:18 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=yPqzwVN9; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of 3RZewZQoKCGogWaZgIPUMLOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--yosryahmed.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3RZewZQoKCGogWaZgIPUMLOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--yosryahmed.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706071878; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tZ4zMgobYN5FtUkXrR87lWOCtHQq0Np8Yg1aQ6/7PDQ=; b=F0nNJylirW5decg2gW34b4znt8GyK9P9oyoDNFNcb9/C/VvmZNQybPdju9nL5/JSTjW1JM wo5f4eYe/CPt51NXW2bTmpzBmH48uHYgBbsF/ihN/S8HcL8SfmvhQ54nlBaMDN5pm8GNAL dvZqzQL/6U04weD3l9u2TDB6EV9HB9g= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=yPqzwVN9; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of 3RZewZQoKCGogWaZgIPUMLOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--yosryahmed.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3RZewZQoKCGogWaZgIPUMLOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--yosryahmed.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706071878; a=rsa-sha256; cv=none; b=lpmheXc5Sw+Lrz79YtnwKJ7mPvos84kJcnz7ZsYplHertGTEQxxw57FSwBdk6KH8zUTEo9 +pULp2AawCxysiGq72L+1m7I/dK5nP6m0P8Pp3NhRQonNJhRFQfWQmFqeITujVm3fhiQ0J RUUWn7mYmozuen0+RmTk63v4NSWW6h0= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-5ffa9b3659cso64736047b3.1 for ; Tue, 23 Jan 2024 20:51:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706071877; x=1706676677; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=tZ4zMgobYN5FtUkXrR87lWOCtHQq0Np8Yg1aQ6/7PDQ=; b=yPqzwVN9ngt1HCwRre5vzHS/FbHEDwwv3PNZPWe/cEyjXb6OgBig+c6OQaugq6Go3A 0HOY0wn5fg9sWuRZtmUuLDj198lym97ZJD9NinW7mL7IIb1mC6PTD/5ue7QGrYkEwIeS b6nZfLBKHKmaMxRwa/aYN5WP2Xl5R8PJ9HfEufSJa1E5saaBDIHmDlKcA0oE57+SmocI klVkGP8eGT2ZdJGkGndH9sQkxLYyNx/Bpsn7yRdMqrRcdy8K3/kWtQAD24FwIu7YJfg4 QdpJX8N8/f2giuB0wHo3f3SutQAvkCp+OKzE9VfHkLAj+jhGukMbO5XJk3tQxC/fqtrL COJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706071877; x=1706676677; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tZ4zMgobYN5FtUkXrR87lWOCtHQq0Np8Yg1aQ6/7PDQ=; b=HtSRUIfyVmbrM1C2OUaJCEi59ccWwWMdYxdUtyOxdKRGKj9riHCm1c37mdKE3w8YmW V9wNLtNSLIotLdQ+Yg5InYcb3FRGiVWhh5CiSvZ4hBys8FW2tWuE96fCdp0CNG01zb+q qqRNLF7IG5wPovzdaYoJq4bFLLnz82ZMpmMsNAHBgLAKZWWyiWyC+uBL6kGlFFTSpikA bM4cf5GHFe9ODwUAEbMDzabxPviz+pQ2Ik6qPQMt70EL8IJ+0oaRie3s78hsV/5mXEME bWAAYUexu81VaFVXH69ZgaRQaljdUPrIM0yJdEg9pFmRqZ3cA/21z/RpCPjcttjnLEhX 5P0A== X-Gm-Message-State: AOJu0YwxW19FdfLh1H4AxAdQ7FqOSzesUSgKPRESwMsh+2wkcpGncg8A CZFfJMInOPedMW2xnKlwCInpKLj3p6G/9W89VqZaIAbnABywWC2jtMBQxyOYfi7tUv8WmIuR0AG 34ST0ooLi5kVPnPEabw== X-Google-Smtp-Source: AGHT+IH4dlOAA/aNVlbOqFDsv6qz2rL4n/3QH5h9ZkNH6atvYVWvPw72AEqTtaPY+eysFAbiENcG3HUSpBnk9xox X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a81:9806:0:b0:5fa:4896:7114 with SMTP id p6-20020a819806000000b005fa48967114mr148615ywg.5.1706071877573; Tue, 23 Jan 2024 20:51:17 -0800 (PST) Date: Wed, 24 Jan 2024 04:51:11 +0000 In-Reply-To: <20240124045113.415378-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20240124045113.415378-1-yosryahmed@google.com> X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240124045113.415378-2-yosryahmed@google.com> Subject: [PATCH v2 1/2] mm: swap: enforce updating inuse_pages at the end of swap_range_free() From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Nhat Pham , Chris Li , Chengming Zhou , Huang Ying , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed X-Rspamd-Queue-Id: 6715540008 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: umwjn4fzk4pat1z3qa3xekaijawfp53t X-HE-Tag: 1706071878-655618 X-HE-Meta: U2FsdGVkX1+mfA8IcFI1DuFRFJDxrR1AYJkzvlG6LyY/Vhs+cZla/vVr83yuOC0b2iZspZ0nwPvG0qkG8q/9W5i5CGSPyRUZqthnkDjfU7nyXslM0aNjfpF2De7CF/+3lSsBbypRPxAkJlpiWZtzHFzJkn45rZnPBqfMEinEJGPGPowdSjTyEISxdJCB0swyk+8Pl14dERjEm7RGr3RhBTGV8H7NepB13sJCReT/WJJQ8JTCCPNyEPAO96ciGrDhVcSM/9J4Zu2QR2/Jp4M/s+zVQZpeQdSpfW04PkB6+Di5Z/KDF768SULc7IpnT2wyCUI3oCxjs4Lq7gQod51sloAE1rs+Ogy6t15igHDDJ4A2kLlPbAM3GGmtsZD9wNWUtZoOjkUnHmwN8NucrVXmzBQ8gtYWkc1GUun/Y4PHWzwOuqqEkmXTkS4pOhnrOXZkNqlcXHAioGzq3B8mj8I4HeFHp2ydwn9puDlZ22BjJEDfiO7ygZd46gmp1doXk8AkDX6/hhWWkJu5zZ2/Fm1jk+LTxFsGAJyhUgFcDlR3cSM4DJzjca3cyhygOHn/g2Hu77N21ir8LqjY8f51QPEfoT+oF5dB8fjB3+0mE1J54wBp8xKwdcomaMrUorPQKOy7oy1OjKw/Wb4QrDuj0TLMQevxAXQdL4UaqtFoAJmTmvkoN30XGZVgVcxjc4IpgIg3wtEmHJT0Jz4j02HpUzybfYHHYf4FrB0Gj+fXeWgTCD+XrYJu3yHnQHgdLwyVdr3FZ60M+s7qtXzAIJ2GGOw3pwikvwMLUXCRDM2jGDBgYSLvkxyKg8aYnuKIa3zykJXE+GzZNq3fgyM5Zb9eUmCFMyIDGnKDNZSd0cidkgwKh3MHu0eqp/8rmylziMh3LQ/ya7o6Hom3IGYqoWXB1P3HvJzHiZvu1PtU9LK9mSD1zXYLwiSrWmUieVBgghDXrb2KbGgriaBmfFuDsPy3b4q iEAepmyD iU1ewdteMrwpocMDQ/GxdFZMMGddwTTS43mgcPE2jNrhfSl2Q7QotqPPfGJO+LV1Vibei+0B41G0f4GBfIcyQx65QBzNXhRUvu8PAHDzz7EB+M2D9yOpm5VNh54Qr0SSaDBmnAqycU8mb1LfEuqqdFXB+H9CTiw4aIrrONAMz70DDwUHY8MF7IYt6AOyQo8A6uXhzaZRtG/kGBcnYATLxdobWd/KaadIfifOn8cA8FHj+m0JH+Zw+3fV+JXAoiFqRChqe3W6upf7QqJd1CRGldpjAiRE6A33+cgCNpIUO6ZknWfjvEsrGS8sVdTY8UB9OU9dgPUnCwEmYNlTXXs9MUxgHaT9kuL3R8h1YtXKQT2DWazn7XAlOrP/lE09yoND3h1LnMybeIF4yX7ALwp/QrRCbkIljnYbMPYuPCz646Gkpmk5ox/naU4E8/nFoiubOYQa388in+8EHWA/1AT60r/5gT6RWmLqP6YAN5t3kIWr+xn0g94HbyU6ATpHKkfQpCrU0QA2Avikk2tTHSXvNzVVREN8OeK/fUKOvwPUOTEe2xaXIFP/RxXfRBRWvZgqWAXTk4OqNGeJ7f64IZBDAgCTbW1QAQ9Cl//tiT2sdz4LmL/Ya+R0nB7I9SgmaL/VZ8RrRvzcq33klggXHc2HBr6uU3P6B1FgNAthmbYFMiJV99tBFibEVzObfkQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In swap_range_free(), we update inuse_pages then do some cleanups (arch invalidation, zswap invalidation, swap cache cleanups, etc). During swapoff, try_to_unuse() checks that inuse_pages is 0 to make sure all swap entries are freed. Make sure we only update inuse_pages after we are done with the cleanups in swap_range_free(), and use the proper memory barriers to enforce it. This makes sure that code following try_to_unuse() can safely assume that swap_range_free() ran for all entries in thr swapfile (e.g. swap cache cleanup, zswap_swapoff()). In practice, this currently isn't a problem because swap_range_free() is called with the swap info lock held, and the swapoff code happens to spin for that after try_to_unuse(). However, this seems fragile and unintentional, so make it more relable and future-proof. This also facilitates a following simplification of zswap_swapoff(). Signed-off-by: Yosry Ahmed Reviewed-by: "Huang, Ying" --- mm/swapfile.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index b11b6057d8b5f..0580bb3e34d77 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -737,8 +737,6 @@ static void swap_range_free(struct swap_info_struct *si, unsigned long offset, if (was_full && (si->flags & SWP_WRITEOK)) add_to_avail_list(si); } - atomic_long_add(nr_entries, &nr_swap_pages); - WRITE_ONCE(si->inuse_pages, si->inuse_pages - nr_entries); if (si->flags & SWP_BLKDEV) swap_slot_free_notify = si->bdev->bd_disk->fops->swap_slot_free_notify; @@ -752,6 +750,14 @@ static void swap_range_free(struct swap_info_struct *si, unsigned long offset, offset++; } clear_shadow_from_swap_cache(si->type, begin, end); + + /* + * Make sure that try_to_unuse() observes si->inuse_pages reaching 0 + * only after the above cleanups are done. + */ + smp_wmb(); + atomic_long_add(nr_entries, &nr_swap_pages); + WRITE_ONCE(si->inuse_pages, si->inuse_pages - nr_entries); } static void set_cluster_next(struct swap_info_struct *si, unsigned long next) @@ -2049,7 +2055,7 @@ static int try_to_unuse(unsigned int type) unsigned int i; if (!READ_ONCE(si->inuse_pages)) - return 0; + goto success; retry: retval = shmem_unuse(type); @@ -2130,6 +2136,12 @@ static int try_to_unuse(unsigned int type) return -EINTR; } +success: + /* + * Make sure that further cleanups after try_to_unuse() returns happen + * after swap_range_free() reduces si->inuse_pages to 0. + */ + smp_mb(); return 0; }