From patchwork Fri Jan 13 17:10:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13101212 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C2EDC67871 for ; Fri, 13 Jan 2023 17:13:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 19DBE8E0005; Fri, 13 Jan 2023 12:13:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 100208E0001; Fri, 13 Jan 2023 12:13:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E94258E0005; Fri, 13 Jan 2023 12:13:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D62978E0001 for ; Fri, 13 Jan 2023 12:13:12 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C01E9A0C89 for ; Fri, 13 Jan 2023 17:13:12 +0000 (UTC) X-FDA: 80350421424.30.4F74C10 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf20.hostedemail.com (Postfix) with ESMTP id 34F981C0018 for ; Fri, 13 Jan 2023 17:13:11 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=jNsSR61e; spf=pass (imf20.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673629991; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=e/tSLKnjXWongAfZcI54y+AzPoBqzDq2scVqv8imtys=; b=yolIx21euAv95qj+oJR2Ys3d+52vn8YY5nbHluWxZ5VqMhvc8keE27Sf2p/2VNM81JRXDP LO4F9eQq4bQLiScbfGnbW6uouL+0AAucC3Esqf1UlL1yDMZNQi/8nQFXbe7lsTnjFhBqEA lp3KOyagYInZuQUCYZWVdJg6n3A0Fak= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=jNsSR61e; spf=pass (imf20.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673629991; a=rsa-sha256; cv=none; b=QybXVSj7lqt6PqRAJzeem4/pYXnUpNc8rSqH2xHhrbdVIqrCWrnTW3JWf1TgRnIaCvHdsH TVWKeDufF0VAnPdcQIDAfWO/4E2rU6aSdWO1qqxkBIYbaVwBUQRYqD1/shBxBi+2dNOl/Q Xqbbg+LpwxC3SKzgi3X61kvCwzlwDN0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1673629990; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e/tSLKnjXWongAfZcI54y+AzPoBqzDq2scVqv8imtys=; b=jNsSR61e8sjMFTnwvQDACTt5L4ymAb51sXMPOBd/yCva84+Q7WcETUfEUDHCkK3QHPM1XU KUoYOuUj9FAVQxsLkoFcypeF0jt7SLjn3pL4RtblaCwU/LIOO/qiJUuHtGu3XSuY9Lx9uK m39h70na3TKwl7Uh8zBCMBugnuZ9/Ko= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-384-HNQxNCQiN5-TTLiX3HPNWw-1; Fri, 13 Jan 2023 12:13:03 -0500 X-MC-Unique: HNQxNCQiN5-TTLiX3HPNWw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2CFD61869B70; Fri, 13 Jan 2023 17:13:02 +0000 (UTC) Received: from t480s.fritz.box (unknown [10.39.193.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2BFB940C2064; Fri, 13 Jan 2023 17:12:55 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: Andrew Morton , Hugh Dickins , John Hubbard , Jason Gunthorpe , Mike Rapoport , Yang Shi , Vlastimil Babka , Nadav Amit , Andrea Arcangeli , Peter Xu , linux-mm@kvack.org, x86@kernel.org, linux-alpha@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-um@lists.infradead.org, linux-xtensa@linux-xtensa.org, David Hildenbrand , Paul Walmsley , Palmer Dabbelt , Albert Ou Subject: [PATCH mm-unstable v1 19/26] riscv/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE Date: Fri, 13 Jan 2023 18:10:19 +0100 Message-Id: <20230113171026.582290-20-david@redhat.com> In-Reply-To: <20230113171026.582290-1-david@redhat.com> References: <20230113171026.582290-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 34F981C0018 X-Stat-Signature: 5x9r7zypuf7s77pmrs1mwephdmwhg9ki X-Rspam-User: X-HE-Tag: 1673629991-362475 X-HE-Meta: U2FsdGVkX1+39WmopbNV+HZs0DIxCgXZca6Scu0w2JR56gKMUmWVAfwOIUDlkbglf760a1Vs4P7diaKbfV+qsBRxZU58MiEWmt2XM3/dWom7XuXsfJPREqj7IneFlUT6ZOe3yPiGTMqOPj3w1NiqMSs2REqQObh39SSA/4ikkB73Mon3i/l9RAaDagLlULX3kc1BZ5O4Z/9qqAw/OPu9DYIx5dB4Et+K5H6YAl7E2QkMe6aezRcR4bykdDBybTVeKEbLLV99OfgK+DXmE116t1lhKyqaAJV7lbYuguxFmmEgovnISz/WhxzgZc2dWTVTXsGG6cmGhi5EqXtpdMgiYHvYimCbG+Davjwpk64MqN9tZyQl1eGzYmICsEuA1gK8uZEqZ1Px2s8i0io0CW/hJ8dAOUJ/gZqPmlG4QL9Zwq/a5iWGmNXIJd9CYIbOnai+WaWPecRiViWZUFXGIxQsmWFWTKDRDu9r6RwS4JAGXvP00o9Lp6O077nLJSeGatOz/EB1zh9z8BKTZtVjZjlAV34AtauIuIQkAPbVFdIrqn4ExaAeA3JSABNGGh+5FZnCnVBwia4A9CjLWcMks9AWVhOXDobfbZk6ktfUKFW6ZtVkeRGpmM1IowspUdKKPuM4Lw39qizWIWmWWlMnqPtCskleHUdYrAE38abCqPhXCXNXCpEBHA62ek1Eq7NyEPtLb4citeiCM+mGmNtLxFYoBR31Y7eLIrkYlOKlG+1uTWu9cy2WnJXgSgB7DBcF+9qBqzVZqd/GKl3JWIDdWNfMwZv5AJdaJhIzw51/W8GO1lVzPRRZO+wR2xjcrb0EpVu5Vn+Ly7ntGfdEuEoxBfZJUjekV8bmpuyZIzxQvPalZlWrF4+VtbkED94fP1CLKxWSQ14wLNsFP2Pl3OYBUZPGQKRNLoFvl1OfKubWhr7H+3EPYK+wNOdp+Xe30ljIM11+K5oX5UowmK6YvqOYu8F 25l1snbZ qJUGh2+7b6Yir6ff7u9Z+uoi5yv22YCr6OA6p X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by stealing one bit from the offset. This reduces the maximum swap space per file: on 32bit to 16 GiB (was 32 GiB). Note that this bit does not conflict with swap PMDs and could also be used in swap PMD context later. While at it, mask the type in __swp_entry(). Cc: Paul Walmsley Cc: Palmer Dabbelt Cc: Albert Ou Signed-off-by: David Hildenbrand Acked-by: Palmer Dabbelt Reviewed-by: Palmer Dabbelt --- arch/riscv/include/asm/pgtable-bits.h | 3 +++ arch/riscv/include/asm/pgtable.h | 29 ++++++++++++++++++++++----- 2 files changed, 27 insertions(+), 5 deletions(-) diff --git a/arch/riscv/include/asm/pgtable-bits.h b/arch/riscv/include/asm/pgtable-bits.h index b9e13a8fe2b7..f896708e8331 100644 --- a/arch/riscv/include/asm/pgtable-bits.h +++ b/arch/riscv/include/asm/pgtable-bits.h @@ -27,6 +27,9 @@ */ #define _PAGE_PROT_NONE _PAGE_GLOBAL +/* Used for swap PTEs only. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_ACCESSED + #define _PAGE_PFN_SHIFT 10 /* diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 4eba9a98d0e3..03a4728db039 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -724,16 +724,18 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma, #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ /* - * Encode and decode a swap entry + * Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that + * are !pte_none() && !pte_present(). * * Format of swap PTE: * bit 0: _PAGE_PRESENT (zero) * bit 1 to 3: _PAGE_LEAF (zero) * bit 5: _PAGE_PROT_NONE (zero) - * bits 6 to 10: swap type - * bits 10 to XLEN-1: swap offset + * bit 6: exclusive marker + * bits 7 to 11: swap type + * bits 11 to XLEN-1: swap offset */ -#define __SWP_TYPE_SHIFT 6 +#define __SWP_TYPE_SHIFT 7 #define __SWP_TYPE_BITS 5 #define __SWP_TYPE_MASK ((1UL << __SWP_TYPE_BITS) - 1) #define __SWP_OFFSET_SHIFT (__SWP_TYPE_BITS + __SWP_TYPE_SHIFT) @@ -744,11 +746,28 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma, #define __swp_type(x) (((x).val >> __SWP_TYPE_SHIFT) & __SWP_TYPE_MASK) #define __swp_offset(x) ((x).val >> __SWP_OFFSET_SHIFT) #define __swp_entry(type, offset) ((swp_entry_t) \ - { ((type) << __SWP_TYPE_SHIFT) | ((offset) << __SWP_OFFSET_SHIFT) }) + { (((type) & __SWP_TYPE_MASK) << __SWP_TYPE_SHIFT) | \ + ((offset) << __SWP_OFFSET_SHIFT) }) #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive(pte_t pte) +{ + return pte_val(pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive(pte_t pte) +{ + return __pte(pte_val(pte) | _PAGE_SWP_EXCLUSIVE); +} + +static inline pte_t pte_swp_clear_exclusive(pte_t pte) +{ + return __pte(pte_val(pte) & ~_PAGE_SWP_EXCLUSIVE); +} + #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION #define __pmd_to_swp_entry(pmd) ((swp_entry_t) { pmd_val(pmd) }) #define __swp_entry_to_pmd(swp) __pmd((swp).val)