From patchwork Mon Oct 24 20:48:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13018121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D86D1C38A2D for ; Mon, 24 Oct 2022 20:48:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D86A94000A; Mon, 24 Oct 2022 16:48:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6616D940007; Mon, 24 Oct 2022 16:48:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4DBA094000A; Mon, 24 Oct 2022 16:48:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3FC36940007 for ; Mon, 24 Oct 2022 16:48:48 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 0327F120658 for ; Mon, 24 Oct 2022 20:48:47 +0000 (UTC) X-FDA: 80057031936.12.98EF80F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf10.hostedemail.com (Postfix) with ESMTP id 82735C0037 for ; Mon, 24 Oct 2022 20:48:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666644527; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DtjgD11+A1aHTX9ijK54hqLSilN13SGt8oWF8m3PTwE=; b=g+kXHmVdZmYgqDCs7ecssQ5k8XOpC7ISftPliqqsVL8otvfQaurMySN2qXMaRDNLsPb/iE P/dXP7P3Y4h9pzbgj120+2CguahZyvxMv7gCIxj+4tCN3YivC1JB+WR/jsyDC2u/uGXd6O UZv2yYzr52TV+Kyhl/UETuyHdR1WePI= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-590-RRbvioxaMm21INrJ06RBHQ-1; Mon, 24 Oct 2022 16:48:45 -0400 X-MC-Unique: RRbvioxaMm21INrJ06RBHQ-1 Received: by mail-qv1-f71.google.com with SMTP id c6-20020ad44306000000b004bb8352cb4cso1118516qvs.14 for ; Mon, 24 Oct 2022 13:48:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DtjgD11+A1aHTX9ijK54hqLSilN13SGt8oWF8m3PTwE=; b=thF+uw3xIReOjx2yPSfFcGENI8pIRy4d61R1aazc+4yA+hLy1ExAX3Elj2FyKERNaC T7SpvWfp/EdrTa1L4J37HZRS9OyjPLZxMDwPyh9VSLuCKWnDrTMy5zIRVb5jkl5Mhp2E tySUwXTju7cmk/1mtUIam9JQ/ZnojGxPTeExldn+ABX/aaguwlq6qWIdHasJuoPzbMzA 9H/nzo4jGExLgNPSdD2jPKXhYbX+8FEJehg0XOjGlp9fJalRWQR0RLospYYTAiLLagWO 8XJA4FOgYcVlpWEt2eTSQk1IbSGWOIu+V99qIw9Fnnjb+YHlS6CJyc6Nn8rKJFqfIMHf N5NA== X-Gm-Message-State: ACrzQf0ElbZvXDCt3n0JOEpVUwvtT7DH2bwvAT3K46SDublo+2wNxQyE pmZ3nGSRl3ATwx7/HzO7pi3+Y6zQvPESEBrbtaKU6Zy+dNNPAHhieaLKVNn9hJ4nYKpOmPXmR0r GAJRZr35k8kQ= X-Received: by 2002:ac8:5c82:0:b0:39c:d508:f1a8 with SMTP id r2-20020ac85c82000000b0039cd508f1a8mr28841672qta.142.1666644524963; Mon, 24 Oct 2022 13:48:44 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4oC3D5ScenLf0AO4sZWnMdtFGxOo0lGq1CGbaHFx86JYXFD9MW/8lK3Zu73fLeqWmyNTJMwg== X-Received: by 2002:a05:620a:12ec:b0:6ee:9e71:190 with SMTP id f12-20020a05620a12ec00b006ee9e710190mr23862434qkl.527.1666644514574; Mon, 24 Oct 2022 13:48:34 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id d13-20020ac8544d000000b0038d9555b580sm498531qtq.44.2022.10.24.13.48.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 13:48:34 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Huang Ying , peterx@redhat.com, Andrew Morton , HORIGUCHI NAOYA , Miaohe Lin , David Hildenbrand , Andrea Arcangeli Subject: [PATCH RFC 1/2] mm: Always compile in pte markers Date: Mon, 24 Oct 2022 16:48:29 -0400 Message-Id: <20221024204830.1342169-2-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221024204830.1342169-1-peterx@redhat.com> References: <20221024204830.1342169-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-type: text/plain ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666644527; a=rsa-sha256; cv=none; b=Yni7z2IkuWbr9GVYoiANUsot8uD5IfvzbrUqU9q4uWJcfSqsM91x3zquDpX/lO4Jy0++b1 1PBpvLcgF4dokJLoGvNMPSFHDwjuiilS0UBz8ezy5TV4LN/Plj+ZQNc28gUXQQgftTpS1b 8iKiDTXLTRSlkDHf+LSl4/NLl1LWR4g= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=g+kXHmVd; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf10.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666644527; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DtjgD11+A1aHTX9ijK54hqLSilN13SGt8oWF8m3PTwE=; b=AEmwm0KGK/d37/SzkHLss4tgudvZOOu5tVsEFLN1fVm5Bn2z8C2O6WWCsSXgLJgeXoaKPv 7ecZ/Pbl+GB07pWeTWxlVN/jf3xV62ukEZULwGvQPEojWF+RQbe1mLoh/C0Uc4D4WuSsR0 e8KVji6DfvpywvlEy8YSbBYDv+KDlws= X-Rspamd-Queue-Id: 82735C0037 Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=g+kXHmVd; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf10.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com X-Rspamd-Server: rspam12 X-Rspam-User: X-Stat-Signature: t8uc7fnc7up5wauycdcsgzfijc9cs4uh X-HE-Tag: 1666644527-383581 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The PTE markers code is tiny and now it's enabled for most of the distributions. It's fine to keep it as-is, but to make a broader use of it (e.g. replacing read error swap entry) it needs to be there always otherwise we need special code path to take care of !PTE_MARKER case. It'll be easier just make pte marker always exist. Use this chance to extend its usage to anonymous too by simply touching up some of the old comments, because it'll be used for anonymous pages in the follow up patches. Signed-off-by: Peter Xu Reviewed-by: Miaohe Lin --- include/linux/swap.h | 10 +++------- include/linux/swapops.h | 31 ------------------------------- mm/Kconfig | 7 ------- mm/memory.c | 7 +++---- 4 files changed, 6 insertions(+), 49 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 369d7799205d..211aeca9bfa7 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -60,17 +60,13 @@ static inline int current_is_kswapd(void) SWP_MIGRATION_NUM + SWP_DEVICE_NUM + \ SWP_PTE_MARKER_NUM) /* - * PTE markers are used to persist information onto PTEs that are mapped with - * file-backed memories. As its name "PTE" hints, it should only be applied to - * the leaves of pgtables. + * PTE markers are used to persist information onto PTEs that otherwise + * should be a none pte. As its name "PTE" hints, it should only be + * applied to the leaves of pgtables. */ -#ifdef CONFIG_PTE_MARKER #define SWP_PTE_MARKER_NUM 1 #define SWP_PTE_MARKER (MAX_SWAPFILES + SWP_HWPOISON_NUM + \ SWP_MIGRATION_NUM + SWP_DEVICE_NUM) -#else -#define SWP_PTE_MARKER_NUM 0 -#endif /* * Unaddressable device memory support. See include/linux/hmm.h and diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 86b95ccb81bb..41a6e7b23415 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -412,8 +412,6 @@ typedef unsigned long pte_marker; #define PTE_MARKER_UFFD_WP BIT(0) #define PTE_MARKER_MASK (PTE_MARKER_UFFD_WP) -#ifdef CONFIG_PTE_MARKER - static inline swp_entry_t make_pte_marker_entry(pte_marker marker) { return swp_entry(SWP_PTE_MARKER, marker); @@ -434,32 +432,6 @@ static inline bool is_pte_marker(pte_t pte) return is_swap_pte(pte) && is_pte_marker_entry(pte_to_swp_entry(pte)); } -#else /* CONFIG_PTE_MARKER */ - -static inline swp_entry_t make_pte_marker_entry(pte_marker marker) -{ - /* This should never be called if !CONFIG_PTE_MARKER */ - WARN_ON_ONCE(1); - return swp_entry(0, 0); -} - -static inline bool is_pte_marker_entry(swp_entry_t entry) -{ - return false; -} - -static inline pte_marker pte_marker_get(swp_entry_t entry) -{ - return 0; -} - -static inline bool is_pte_marker(pte_t pte) -{ - return false; -} - -#endif /* CONFIG_PTE_MARKER */ - static inline pte_t make_pte_marker(pte_marker marker) { return swp_entry_to_pte(make_pte_marker_entry(marker)); @@ -477,9 +449,6 @@ static inline pte_t make_pte_marker(pte_marker marker) * memory, kernel-only memory (including when the system is during-boot), * non-ram based generic file-system. It's fine to be used even there, but the * extra pte marker check will be pure overhead. - * - * For systems configured with !CONFIG_PTE_MARKER this will be automatically - * optimized to pte_none(). */ static inline int pte_none_mostly(pte_t pte) { diff --git a/mm/Kconfig b/mm/Kconfig index 57e1d8c5b505..4b28800d9be1 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1107,17 +1107,10 @@ config HAVE_ARCH_USERFAULTFD_MINOR help Arch has userfaultfd minor fault support -config PTE_MARKER - bool - - help - Allows to create marker PTEs for file-backed memory. - config PTE_MARKER_UFFD_WP bool "Userfaultfd write protection support for shmem/hugetlbfs" default y depends on HAVE_ARCH_USERFAULTFD_WP - select PTE_MARKER help Allows to create marker PTEs for userfaultfd write protection diff --git a/mm/memory.c b/mm/memory.c index 25b12d1a7db0..71114f05a68c 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3699,11 +3699,10 @@ static vm_fault_t handle_pte_marker(struct vm_fault *vmf) unsigned long marker = pte_marker_get(entry); /* - * PTE markers should always be with file-backed memories, and the - * marker should never be empty. If anything weird happened, the best - * thing to do is to kill the process along with its mm. + * PTE markers should never be empty. If anything weird happened, + * the best thing to do is to kill the process along with its mm. */ - if (WARN_ON_ONCE(vma_is_anonymous(vmf->vma) || !marker)) + if (WARN_ON_ONCE(!marker)) return VM_FAULT_SIGBUS; if (pte_marker_entry_uffd_wp(entry)) From patchwork Mon Oct 24 20:48:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13018120 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD5D5C67871 for ; Mon, 24 Oct 2022 20:48:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6B11E940009; Mon, 24 Oct 2022 16:48:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 63A50940007; Mon, 24 Oct 2022 16:48:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4B3FB940009; Mon, 24 Oct 2022 16:48:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3C9F1940007 for ; Mon, 24 Oct 2022 16:48:41 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BFC021C5C13 for ; Mon, 24 Oct 2022 20:48:40 +0000 (UTC) X-FDA: 80057031600.02.A4BB7A8 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf17.hostedemail.com (Postfix) with ESMTP id 587CA40022 for ; Mon, 24 Oct 2022 20:48:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666644519; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vZ6z2mx7oNlRshrZTg5WSyqHxrl+GaeTDPKlsjXuMmM=; b=D03B9mKqDXrKNVpI3Xz49cyBqNHSzstJgj+eRjVpF070WNk6IdQd5V9rFhq1YRC5KsElzm sUYWpWBrMu2woDoSB2WQzVl31JbmXSxYK/7RwCNs2eA2d++USmxhax9Rs8o3oapkgxIY8J Z3uqIxOOgf+VLePxUJS+p4tCdrVtYik= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-17-SrqDoVBfNBeFwgYonaZqaA-1; Mon, 24 Oct 2022 16:48:36 -0400 X-MC-Unique: SrqDoVBfNBeFwgYonaZqaA-1 Received: by mail-qk1-f197.google.com with SMTP id u6-20020a05620a430600b006e47fa02576so9971437qko.22 for ; Mon, 24 Oct 2022 13:48:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vZ6z2mx7oNlRshrZTg5WSyqHxrl+GaeTDPKlsjXuMmM=; b=k2Sh6LbPANBnBEvp1ZJgVNSey2BWHxtUpyLWke2hrRoLg/JWkOF0PGqZ5dtyvxv1j7 L1/njNOt8oNC+35CWv5ZCeLqRyRHuIuC64ZPjhu1aPZiiYasfMYJUzz+P1EAd4HZPdC8 vTDsx9UwcA8VLhIpWW0YYTA2dmSykaKf+OM8UpZvhJEoUHromL34l+0+MzG745dcq648 nrCMMFpBgct4d+/s6ujSwz9yI5GnxNij4E+9GfLh3aaJjQrjhLJPigUVGEFnVKyFhepy CyTN3j9B9x4lR5pf5oBEd5qjoyEqPMK+yK2OhZQgg2CvMqyJW4Pal4OjQyhXeeWjxoxS f7Bw== X-Gm-Message-State: ACrzQf3u+ajHyO1VnGzjrw729/vLgpugnPRKUB7vBSx2eWJp5QHM8Xm6 EFL0aSOj0aq4JfosNQwiKCwmDfkb55IelpcYcAGMgYm226J4D3o2X5kMxE71AsuKuEhcivxnSWD ZvDBUAjM2vvE= X-Received: by 2002:a05:6214:b6c:b0:4bb:6747:3089 with SMTP id ey12-20020a0562140b6c00b004bb67473089mr7896756qvb.41.1666644516143; Mon, 24 Oct 2022 13:48:36 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6TOJBrQCGAvT7s3wRbvuxazhcJmkdpvIzFrO2mB8REtMPUwzf7bUiQBnGEcyrDxmWpKMjIKA== X-Received: by 2002:a05:6214:b6c:b0:4bb:6747:3089 with SMTP id ey12-20020a0562140b6c00b004bb67473089mr7896733qvb.41.1666644515915; Mon, 24 Oct 2022 13:48:35 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id d13-20020ac8544d000000b0038d9555b580sm498531qtq.44.2022.10.24.13.48.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 13:48:35 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Huang Ying , peterx@redhat.com, Andrew Morton , HORIGUCHI NAOYA , Miaohe Lin , David Hildenbrand , Andrea Arcangeli Subject: [PATCH RFC 2/2] mm: Use pte markers for swap errors Date: Mon, 24 Oct 2022 16:48:30 -0400 Message-Id: <20221024204830.1342169-3-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221024204830.1342169-1-peterx@redhat.com> References: <20221024204830.1342169-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-type: text/plain ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666644520; a=rsa-sha256; cv=none; b=MkB2eJkZ5AGKCgntEutsifvCJRlz+9cjCyWHcPcXJo2zhjQ7rcBWr6rEVVcPAOmm3f0u1s VFz/sA9/6lCVBnP9CnOYOuVRfyYqY2EhrC10+Iq3kp2ZLZg3FhcMY9EzT9f4w69hTLcpRF lJlexUFHt+D51a01yk/phRs01YhRZwQ= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=D03B9mKq; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf17.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666644520; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vZ6z2mx7oNlRshrZTg5WSyqHxrl+GaeTDPKlsjXuMmM=; b=eU+WsP4cfNe5vURn1Yvy1+nVWZVBTENK6hrqb2eGCY8qUMeZCEANGpYFWmYZY1q6k4IgGs bnet6oK3+EMfkRsOn/F+wsyGnVCPJB4amhJTX24qCLSmb26oQPebC7FQEE+jQeihQVxkpB hARQPQAab+V3Rt7FlVHw/YpN92m1KGk= X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 587CA40022 X-Rspam-User: Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=D03B9mKq; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf17.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com X-Stat-Signature: 5ucmsf13hzfz7q9tq177xir4oq9bk9n3 X-HE-Tag: 1666644520-306778 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: PTE markers are ideal mechanism for things like SWP_SWAPIN_ERROR. Using a whole swap entry type for this purpose can be an overkill, especially if we already have PTE markers. Define a new bit for swapin error and replace it with pte markers. Then we can safely drop SWP_SWAPIN_ERROR and give one device slot back to swap. We used to have SWP_SWAPIN_ERROR taking the page pfn as part of the swap entry, but it's never used. Neither do I see how it can be useful because normally the swapin failure should not be caused by a bad page but bad swap device. Drop it alongside. Signed-off-by: Peter Xu Reviewed-by: Miaohe Lin --- include/linux/swap.h | 6 +----- include/linux/swapops.h | 26 ++++++++++++++------------ mm/memory.c | 6 ++++-- mm/shmem.c | 2 +- mm/swapfile.c | 2 +- 5 files changed, 21 insertions(+), 21 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 211aeca9bfa7..fec6647a289a 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -55,10 +55,6 @@ static inline int current_is_kswapd(void) * actions on faults. */ -#define SWP_SWAPIN_ERROR_NUM 1 -#define SWP_SWAPIN_ERROR (MAX_SWAPFILES + SWP_HWPOISON_NUM + \ - SWP_MIGRATION_NUM + SWP_DEVICE_NUM + \ - SWP_PTE_MARKER_NUM) /* * PTE markers are used to persist information onto PTEs that otherwise * should be a none pte. As its name "PTE" hints, it should only be @@ -121,7 +117,7 @@ static inline int current_is_kswapd(void) #define MAX_SWAPFILES \ ((1 << MAX_SWAPFILES_SHIFT) - SWP_DEVICE_NUM - \ SWP_MIGRATION_NUM - SWP_HWPOISON_NUM - \ - SWP_PTE_MARKER_NUM - SWP_SWAPIN_ERROR_NUM) + SWP_PTE_MARKER_NUM) /* * Magic header for a swap area. The first part of the union is diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 41a6e7b23415..dd087b7287b0 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -162,16 +162,6 @@ static inline void *swp_to_radix_entry(swp_entry_t entry) return xa_mk_value(entry.val); } -static inline swp_entry_t make_swapin_error_entry(struct page *page) -{ - return swp_entry(SWP_SWAPIN_ERROR, page_to_pfn(page)); -} - -static inline int is_swapin_error_entry(swp_entry_t entry) -{ - return swp_type(entry) == SWP_SWAPIN_ERROR; -} - #if IS_ENABLED(CONFIG_DEVICE_PRIVATE) static inline swp_entry_t make_readable_device_private_entry(pgoff_t offset) { @@ -409,8 +399,9 @@ static inline bool is_migration_entry_dirty(swp_entry_t entry) typedef unsigned long pte_marker; -#define PTE_MARKER_UFFD_WP BIT(0) -#define PTE_MARKER_MASK (PTE_MARKER_UFFD_WP) +#define PTE_MARKER_UFFD_WP BIT(0) +#define PTE_MARKER_SWAP_ERROR BIT(1) +#define PTE_MARKER_MASK (BIT(2) - 1) static inline swp_entry_t make_pte_marker_entry(pte_marker marker) { @@ -437,6 +428,17 @@ static inline pte_t make_pte_marker(pte_marker marker) return swp_entry_to_pte(make_pte_marker_entry(marker)); } +static inline swp_entry_t make_swapin_error_entry(void) +{ + return make_pte_marker_entry(PTE_MARKER_SWAP_ERROR); +} + +static inline int is_swapin_error_entry(swp_entry_t entry) +{ + return is_pte_marker_entry(entry) && + (pte_marker_get(entry) & PTE_MARKER_SWAP_ERROR); +} + /* * This is a special version to check pte_none() just to cover the case when * the pte is a pte marker. It existed because in many cases the pte marker diff --git a/mm/memory.c b/mm/memory.c index 71114f05a68c..cf602c3854c3 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3705,6 +3705,10 @@ static vm_fault_t handle_pte_marker(struct vm_fault *vmf) if (WARN_ON_ONCE(!marker)) return VM_FAULT_SIGBUS; + /* Higher priority than uffd-wp when data corrupted */ + if (marker & PTE_MARKER_SWAP_ERROR) + return VM_FAULT_SIGBUS; + if (pte_marker_entry_uffd_wp(entry)) return pte_marker_handle_uffd_wp(vmf); @@ -3764,8 +3768,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) put_page(vmf->page); } else if (is_hwpoison_entry(entry)) { ret = VM_FAULT_HWPOISON; - } else if (is_swapin_error_entry(entry)) { - ret = VM_FAULT_SIGBUS; } else if (is_pte_marker_entry(entry)) { ret = handle_pte_marker(vmf); } else { diff --git a/mm/shmem.c b/mm/shmem.c index c5f7a6668416..47dc595e98a3 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1682,7 +1682,7 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, swp_entry_t swapin_error; void *old; - swapin_error = make_swapin_error_entry(&folio->page); + swapin_error = make_swapin_error_entry(); old = xa_cmpxchg_irq(&mapping->i_pages, index, swp_to_radix_entry(swap), swp_to_radix_entry(swapin_error), 0); diff --git a/mm/swapfile.c b/mm/swapfile.c index 5fc1237a9f21..13f2569a6719 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1781,7 +1781,7 @@ static int unuse_pte(struct vm_area_struct *vma, pmd_t *pmd, pte_t pteval; dec_mm_counter(vma->vm_mm, MM_SWAPENTS); - pteval = swp_entry_to_pte(make_swapin_error_entry(page)); + pteval = swp_entry_to_pte(make_swapin_error_entry()); set_pte_at(vma->vm_mm, addr, pte, pteval); swap_free(entry); ret = 0;