From patchwork Tue Jan 23 14:53:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13527553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 433ACC47DDF for ; Tue, 23 Jan 2024 14:57:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ZRoEcxiGpR0qFZdyu4yJFQUNIOq5vBIO6YRM8PnanmY=; b=dZ0tMbpX3mNZtO6BOGsw3gEl3Y H9KUIoVe04wXmgItcFJ4Q8PcrA7FiPmc4cWWEmzYytx/pVgwhSFjXEvLY9Oazdw0wG9oyMKYRx/fQ /GgksNhJ7N2WIx3KnzRxLX+8kbGT8JCsFX0QYo30dxS64P+BOd9xGOTMmR6WO7S/WKVHxdhdvWkYk dVct2EJ0NTFNUFE4cUF+5DZvtGx7Nz6ES9KH9Bd29+/wBUqAmNXWYmXhDNBJl8ZIt0CrAhY6DnK4C fk/tN1srkm04GY8geWgrCW1Fy1hwQQvL6u868BH3g/jepf52g4CG1lpfpY0LQhDRpyg178AnVbzyJ lS+BGF0w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rSICf-00Gumn-3B; Tue, 23 Jan 2024 14:56:41 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rSIBq-00GuMK-1Y for linux-arm-kernel@bombadil.infradead.org; Tue, 23 Jan 2024 14:55:50 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=Lg92dN/Mg5BeIohbx0CLPVEaI8ExVr2RHnwHR1fA1wc=; b=Z9vsnml6pyjtgZPxmf2VHl9Tyh q0JeF/qq3rZd/jZiiQIHtn273ow1bfxUzzRwOCRXzftKlw0hTnsHfOND8FLjfyxf9PeiC/b7prcrZ WzdQfiygAo8alWohVvUZ+XEl9AJZKWOaffSyxYoXeGAog6jvuWsemnnlH9y9sVMR40odN4neznc4+ 6n5I5SF2clYnfhI9VaGaTl3oYweW3oNKY27QCFatAKS7ecWQ9wmDsGFf/AHSDAcxitWoJ+rFoRYid E/CXGvsv18a16ABoNZ8QiEgCx4zESja5F2diE70LUhDteRpuhKkv06hLA+O+EIfy9jrlvbbc6/Stt TFeTRfTA==; Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by desiato.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rSIBn-00000003vWu-0wgI for linux-arm-kernel@lists.infradead.org; Tue, 23 Jan 2024 14:55:49 +0000 Received: by mail-wr1-x449.google.com with SMTP id ffacd0b85a97d-33941f7e299so461640f8f.0 for ; Tue, 23 Jan 2024 06:55:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706021745; x=1706626545; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Lg92dN/Mg5BeIohbx0CLPVEaI8ExVr2RHnwHR1fA1wc=; b=Azgq20WUPJlQ0nMGGLvtO1fIQUatszvsA9j4SgV4q0C6BFa3ZIE5MEED7cV2jKhdkE 2MxfiJya4+aMKNYw6h89PgSU4vu7EiIkOMr/5QL8o4Fu8E+Gwjbc5c8EnHy3Mynz+4WX YCA+ZlNyJTciXQAauCzvzF02Z2GDqrbBN++jh4Ln1hKOdkm8x9UxPqglTtBwXJ6hxbRA ESgYxXr+o9OsoZrgstUYZhMBs2bbcKMMZZ7x55JvnGpVdPy8g2ndDuLqEkppEIdjinI4 bDPVoFO5gCX4Ak0zS3dKmDQAjOXW13hytj9aou0wX/acdxfPuBZakSOMDnp0Wf3dV+ZQ OORA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706021745; x=1706626545; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Lg92dN/Mg5BeIohbx0CLPVEaI8ExVr2RHnwHR1fA1wc=; b=s7xmAMkF6dkWntTwo8VzuPmaC/r3GaRmLGFAmowlMGGg4aFRYtLX2rm/JMb24elSkd vpus1REACgL6P59spMmGSCDfSLXJx1KDGUpCwTfmvUVEpvaP76FuNHQ6570MWi8mmwMQ dHY4t9ct8/Kbjku1zkadaWtyQwvu0L/NTAwARp6Wd8s3+RlRQtm7wHxOaZW6uta/dSdl uhmxEEeXTtEErzeJgJZYTLKTZgH7hzb2o+3/LJWGhZkWqoaLU23JW+s9w222iadmlyz2 +QiKpWeZ4+L9Gs0NnR4e4HSkz1yP2ErhXCcStwvs5EhlVz0R+VVqE1vGtXw/I2VyogBA G1xA== X-Gm-Message-State: AOJu0YzPMNydPOJAtrRTBt9lzh/rLmrZDzNa0iRHYws2L3ZTiUUFBmih VusBmwIqZxtV++DBpAjr8BqufdbyWkLf+hyXKjS6GqNVEZ9W5zO2uNAX4Psuwbux0w+x1OCUMkK kODIl5LkDOWFyt14jRsyW44P9kwMruIK3Dh0/fVMBDK0OoIpeTr1KsRsmYq/4gVQX4I0TylXPgZ yvNA1B8aoi4coq503rb2ZweOJn0Vv02rv0TYTHubdv X-Google-Smtp-Source: AGHT+IHIMpVBHxRfh8l7ePsmWtunUKhYFUPAoC/1gLPv8BwfMD/cacdM/Fm9FDTwuy/d9fd0qYHr7JFe X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:6000:1e12:b0:339:2a23:8e69 with SMTP id bj18-20020a0560001e1200b003392a238e69mr16226wrb.5.1706021745273; Tue, 23 Jan 2024 06:55:45 -0800 (PST) Date: Tue, 23 Jan 2024 15:53:35 +0100 In-Reply-To: <20240123145258.1462979-52-ardb+git@google.com> Mime-Version: 1.0 References: <20240123145258.1462979-52-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3974; i=ardb@kernel.org; h=from:subject; bh=gecEIJDd8zMAkroaF8QQO7Ni/qDxVGJunsWUBre+5z8=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXX9pU8v768L3mhcxzJ1j7wm897tJ+3z2HRDp7lLegrdP uX8+eKTjlIWBjEOBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCRFd8YGb5dyuR2KmT7UVIv pWW8/bo1w8qDb9+asYqzaKa9Ucq81c/wP/L/bb6SsPOl0eXM/u6nPNT2CHw59avjlt+GLV5Bjcm cvAA= X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240123145258.1462979-88-ardb+git@google.com> Subject: [PATCH v7 36/50] arm64: mm: Wire up TCR.DS bit to PTE shareability fields From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland , Ryan Roberts , Anshuman Khandual , Kees Cook X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240123_145547_426977_AE8E0C6D X-CRM114-Status: GOOD ( 15.96 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel When LPA2 is enabled, bits 8 and 9 of page and block descriptors become part of the output address instead of carrying shareability attributes for the region in question. So avoid setting these bits if TCR.DS == 1, which means LPA2 is enabled. Signed-off-by: Ard Biesheuvel --- arch/arm64/Kconfig | 4 ++++ arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/include/asm/pgtable-prot.h | 16 ++++++++++++++-- arch/arm64/mm/mmap.c | 4 ++++ arch/arm64/mm/proc.S | 2 ++ 5 files changed, 25 insertions(+), 2 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index aa7c1d435139..8c2c36fffcf5 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1377,6 +1377,10 @@ config ARM64_PA_BITS default 48 if ARM64_PA_BITS_48 default 52 if ARM64_PA_BITS_52 +config ARM64_LPA2 + def_bool y + depends on ARM64_PA_BITS_52 && !ARM64_64K_PAGES + choice prompt "Endianness" default CPU_LITTLE_ENDIAN diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index e4944d517c99..b770f98fc0b5 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -284,6 +284,7 @@ #define TCR_E0PD1 (UL(1) << 56) #define TCR_TCMA0 (UL(1) << 57) #define TCR_TCMA1 (UL(1) << 58) +#define TCR_DS (UL(1) << 59) /* * TTBR. diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h index 63ced9ccec21..dd9ee67d1d87 100644 --- a/arch/arm64/include/asm/pgtable-prot.h +++ b/arch/arm64/include/asm/pgtable-prot.h @@ -30,8 +30,8 @@ #define _PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) #define _PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S) -#define PROT_DEFAULT (_PROT_DEFAULT | PTE_MAYBE_NG) -#define PROT_SECT_DEFAULT (_PROT_SECT_DEFAULT | PMD_MAYBE_NG) +#define PROT_DEFAULT (PTE_TYPE_PAGE | PTE_MAYBE_NG | PTE_MAYBE_SHARED | PTE_AF) +#define PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_MAYBE_NG | PMD_MAYBE_SHARED | PMD_SECT_AF) #define PROT_DEVICE_nGnRnE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_DEVICE_nGnRnE)) #define PROT_DEVICE_nGnRE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_DEVICE_nGnRE)) @@ -67,7 +67,19 @@ extern bool arm64_use_ng_mappings; #define PTE_MAYBE_NG (arm64_use_ng_mappings ? PTE_NG : 0) #define PMD_MAYBE_NG (arm64_use_ng_mappings ? PMD_SECT_NG : 0) +#ifndef CONFIG_ARM64_LPA2 #define lpa2_is_enabled() false +#define PTE_MAYBE_SHARED PTE_SHARED +#define PMD_MAYBE_SHARED PMD_SECT_S +#else +static inline bool __pure lpa2_is_enabled(void) +{ + return read_tcr() & TCR_DS; +} + +#define PTE_MAYBE_SHARED (lpa2_is_enabled() ? 0 : PTE_SHARED) +#define PMD_MAYBE_SHARED (lpa2_is_enabled() ? 0 : PMD_SECT_S) +#endif /* * If we have userspace only BTI we don't want to mark kernel pages diff --git a/arch/arm64/mm/mmap.c b/arch/arm64/mm/mmap.c index 645fe60d000f..642bdf908b22 100644 --- a/arch/arm64/mm/mmap.c +++ b/arch/arm64/mm/mmap.c @@ -73,6 +73,10 @@ static int __init adjust_protection_map(void) protection_map[VM_EXEC | VM_SHARED] = PAGE_EXECONLY; } + if (lpa2_is_enabled()) + for (int i = 0; i < ARRAY_SIZE(protection_map); i++) + pgprot_val(protection_map[i]) &= ~PTE_SHARED; + return 0; } arch_initcall(adjust_protection_map); diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 6e1b2bc41a9f..7c46f8cfd6ae 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -466,6 +466,7 @@ alternative_else_nop_endif */ #define PTE_MAYBE_NG 0 +#define PTE_MAYBE_SHARED 0 mov_q x0, PIE_E0 msr REG_PIRE0_EL1, x0 @@ -473,6 +474,7 @@ alternative_else_nop_endif msr REG_PIR_EL1, x0 #undef PTE_MAYBE_NG +#undef PTE_MAYBE_SHARED mov x0, TCR2_EL1x_PIE msr REG_TCR2_EL1, x0