From patchwork Thu Mar 4 17:11:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12116605 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 270DCC433E6 for ; Thu, 4 Mar 2021 17:15:18 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B62EA64F59 for ; Thu, 4 Mar 2021 17:15:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B62EA64F59 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=vlk4N+EMm+DEkvVaVl2Wvz96fQiML+h7zIVPLEcm7+c=; b=ikG6IQ6RXqP5QFjHQmL798dnS yjYUVt7URuZlDt7XltkZFWphzWcMVIk/zsyBUhRaIKUtMn9YuDqnGJJg6vvzGq0/giVAuaELhixGR AyoB50HuMB628QVzqjfhjIZK61oeLBbsXYkElEqV4ZJKVSk28HtKlQzX/DH2s50gbqusPH3vhSPqd nCA/xGHyMBDuRmj9/B3BWMdecpXWK+F7xmxp7xi/37NP0cMSoGxQ2b7wTq8gdTpOFyZN6moHf5X4B 944mM3Ep7iXcU2jsVwxM5BPj40jBWF1dKLzubgv9cTeLtZNpgqOg3rI40mED5oOAiJdPqpmEegkWO oomIGX7ag==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHrXI-009O8h-5O; Thu, 04 Mar 2021 17:13:16 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHrWu-009O3t-HQ for linux-arm-kernel@lists.infradead.org; Thu, 04 Mar 2021 17:12:55 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 5293064F55; Thu, 4 Mar 2021 17:12:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1614877967; bh=H4Na2ynW4w01AlcXOXf+7Kg2aX6CuzmAPdxg2C9L2pA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Qo+VxH1bH3i+0ix7GfUSfDff4Gspe6Zqa3gKF+6xmKSOtkDCln041PmhvRUwS5RDl TnKsuy969mkcpRKd+FohqU57cDEdp1gr7pFkA+Oq0zFCTBb7xdODd8S60ozDOQcFz8 RtR3z5PjVOdNZsFnf/BRxY028hSxCuFCWRPeCXZRTBw5vb4zAGZQ9WblHcK7B1BErY qLvhF5n6LJLTYLtdoVTah51N4X+LqOKye5YJcfdFLDJPZ5aKg/IEwBeeWncL37xX/H jySPsikKOky/GH24SbKlv7E95u3MBUDDVR4h60DhaUhRVikfQ/vQDuyJq+4rh0jupo 0gL9q9ajg99CQ== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , maz@kernel.org, catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, anshuman.khandual@arm.com, qperret@google.com, kernel-team@android.com Subject: [PATCH 1/2] arm64: mm: add missing P4D definitions and use them consistently Date: Thu, 4 Mar 2021 18:11:44 +0100 Message-Id: <20210304171145.12281-2-ardb@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210304171145.12281-1-ardb@kernel.org> References: <20210304171145.12281-1-ardb@kernel.org> MIME-Version: 1.0 X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Even though level 0, 1 and 2 descriptors share the same attribute encodings, let's be a bit more consistent about using the right one at the right level. So add new macros for level 0/P4D definitions, and clean up some inconsistencies involving these macros. Signed-off-by: Ard Biesheuvel Acked-by: Mark Rutland --- arch/arm64/include/asm/pgtable-hwdef.h | 9 +++++++++ arch/arm64/mm/mmu.c | 6 +++--- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index 42442a0ae2ab..e64e77a345b2 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -94,6 +94,15 @@ /* * Hardware page table definitions. * + * Level 0 descriptor (P4D). + */ +#define P4D_TYPE_TABLE (_AT(p4dval_t, 3) << 0) +#define P4D_TABLE_BIT (_AT(p4dval_t, 1) << 1) +#define P4D_TYPE_MASK (_AT(p4dval_t, 3) << 0) +#define P4D_TYPE_SECT (_AT(p4dval_t, 1) << 0) +#define P4D_SECT_RDONLY (_AT(p4dval_t, 1) << 7) /* AP[2] */ + +/* * Level 1 descriptor (PUD). */ #define PUD_TYPE_TABLE (_AT(pudval_t, 3) << 0) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 3802cfbdd20d..029091474042 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -309,7 +309,7 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, phys_addr_t pud_phys; BUG_ON(!pgtable_alloc); pud_phys = pgtable_alloc(PUD_SHIFT); - __p4d_populate(p4dp, pud_phys, PUD_TYPE_TABLE); + __p4d_populate(p4dp, pud_phys, P4D_TYPE_TABLE); p4d = READ_ONCE(*p4dp); } BUG_ON(p4d_bad(p4d)); @@ -1209,11 +1209,11 @@ void __init early_fixmap_init(void) pudp = pud_offset_kimg(p4dp, addr); } else { if (p4d_none(p4d)) - __p4d_populate(p4dp, __pa_symbol(bm_pud), PUD_TYPE_TABLE); + __p4d_populate(p4dp, __pa_symbol(bm_pud), P4D_TYPE_TABLE); pudp = fixmap_pud(addr); } if (pud_none(READ_ONCE(*pudp))) - __pud_populate(pudp, __pa_symbol(bm_pmd), PMD_TYPE_TABLE); + __pud_populate(pudp, __pa_symbol(bm_pmd), PUD_TYPE_TABLE); pmdp = fixmap_pmd(addr); __pmd_populate(pmdp, __pa_symbol(bm_pte), PMD_TYPE_TABLE); From patchwork Thu Mar 4 17:11:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12116603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53FC9C433DB for ; Thu, 4 Mar 2021 17:15:16 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E799764F57 for ; Thu, 4 Mar 2021 17:15:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E799764F57 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=mUqWe5mEm5KMEmecwwCQEyckklDgCsYq+8Qd6iOPako=; b=j2hW6V6EM90jUOzj3ejFks7Je FJvWI65U8uF4gilzRFybA9FQiH40/Itawvz1gR+zHM22A6aMiAbxGvKTeWnamxEwPmpg2dzEa3EdV Y4o+EQf4Sb22RZp4Z/3dwvW0z2hHYXckOh7qlzJu1EObi0TT/garZcxDJ5EPc+V+EOL+aoZSI4UwV TI+vTxvkzMS+B38a8+eTfqYAi38nq81egEwHKKwI1NWFJdIR5QFE1SDlyx3njUqvWFUorGiElor/+ zEXN4C5pIRZ0YQ4vjzyCh3PQQWm99sTW9kHbZKFI87R8vAny6vir7YJgSfkgAl5IN62u/khf3YXvD eZtA7sgLw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHrX7-009O6p-Jv; Thu, 04 Mar 2021 17:13:05 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHrWs-009O4J-Qs for linux-arm-kernel@lists.infradead.org; Thu, 04 Mar 2021 17:12:54 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 8808664F56; Thu, 4 Mar 2021 17:12:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1614877969; bh=6JtBaJlHGmt7Lw5WPl9b8IObo4pDx4RlgEvVfimMIhI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Yw1oclL9y6hqeDq55IimRwvxvWkt2PzxokBQJdVUrTFEgMNA+WMcB/62OqZXZhCuj n5Qd22XbMnXMGL8I65VRfDCVbxbP4Ogfbnydq4UWl60/Z880H7YS3+cfQRzGpxmgtn uUWQYA6S+o3gcs33IAabJxuwAchtkMagao+wYOm+/hNUQUWv9WlNJcc+eKsdrgYXeH YQY/ec/asBIITdpTiCJ7Usd9DOm/iTvV1cy9TDffOwicO3qwQhqJjlFPbhdBxalJ9y NV7M3qPIxMp/MO917x91l0hkY3ZvsAjJsUUQ5nAa4xuBJddJoXKigNZvlRR1QS36xF TUJFWayrbWnXA== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , maz@kernel.org, catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, anshuman.khandual@arm.com, qperret@google.com, kernel-team@android.com Subject: [PATCH 2/2] arm64: mm: use XN table mapping attributes for the linear region Date: Thu, 4 Mar 2021 18:11:45 +0100 Message-Id: <20210304171145.12281-3-ardb@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210304171145.12281-1-ardb@kernel.org> References: <20210304171145.12281-1-ardb@kernel.org> MIME-Version: 1.0 X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The way the arm64 kernel virtual address space is constructed guarantees that swapper PGD entries are never shared between the linear region on the one hand, and the vmalloc region on the other, which is where all kernel text, module text and BPF text mappings reside. This means that mappings in the linear region (which never require executable permissions) never share any table entries at any level with mappings that do require executable permissions, and so we can set the table-level PXN/UXN attributes for all table entries that are created while setting up mappings in the linear region. Since swapper's PGD level page table is mapped r/o itself, this adds another layer of robustness to the way the kernel manages its own page tables. Signed-off-by: Ard Biesheuvel Acked-by: Mark Rutland --- arch/arm64/include/asm/pgtable-hwdef.h | 3 +++ arch/arm64/mm/mmu.c | 27 +++++++++++++++----- 2 files changed, 23 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index e64e77a345b2..b6f50abd63d7 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -101,6 +101,7 @@ #define P4D_TYPE_MASK (_AT(p4dval_t, 3) << 0) #define P4D_TYPE_SECT (_AT(p4dval_t, 1) << 0) #define P4D_SECT_RDONLY (_AT(p4dval_t, 1) << 7) /* AP[2] */ +#define P4D_TABLE_XN (_AT(p4dval_t, 3) << 59) /* UXN:PXN */ /* * Level 1 descriptor (PUD). @@ -110,6 +111,7 @@ #define PUD_TYPE_MASK (_AT(pudval_t, 3) << 0) #define PUD_TYPE_SECT (_AT(pudval_t, 1) << 0) #define PUD_SECT_RDONLY (_AT(pudval_t, 1) << 7) /* AP[2] */ +#define PUD_TABLE_XN (_AT(pudval_t, 3) << 59) /* UXN:PXN */ /* * Level 2 descriptor (PMD). @@ -131,6 +133,7 @@ #define PMD_SECT_CONT (_AT(pmdval_t, 1) << 52) #define PMD_SECT_PXN (_AT(pmdval_t, 1) << 53) #define PMD_SECT_UXN (_AT(pmdval_t, 1) << 54) +#define PMD_TABLE_XN (_AT(pmdval_t, 3) << 59) /* UXN:PXN */ /* * AttrIndx[2:0] encoding (mapping attributes defined in the MAIR* registers). diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 029091474042..ff5cbca403be 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -39,6 +39,7 @@ #define NO_BLOCK_MAPPINGS BIT(0) #define NO_CONT_MAPPINGS BIT(1) +#define NO_EXEC_MAPPINGS BIT(2) u64 idmap_t0sz = TCR_T0SZ(VA_BITS); u64 idmap_ptrs_per_pgd = PTRS_PER_PGD; @@ -185,10 +186,14 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, BUG_ON(pmd_sect(pmd)); if (pmd_none(pmd)) { + pmdval_t pmdval = PMD_TYPE_TABLE; phys_addr_t pte_phys; + + if (flags & NO_EXEC_MAPPINGS) + pmdval |= PMD_TABLE_XN; BUG_ON(!pgtable_alloc); pte_phys = pgtable_alloc(PAGE_SHIFT); - __pmd_populate(pmdp, pte_phys, PMD_TYPE_TABLE); + __pmd_populate(pmdp, pte_phys, pmdval); pmd = READ_ONCE(*pmdp); } BUG_ON(pmd_bad(pmd)); @@ -259,10 +264,14 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, */ BUG_ON(pud_sect(pud)); if (pud_none(pud)) { + pudval_t pudval = PUD_TYPE_TABLE; phys_addr_t pmd_phys; + + if (flags & NO_EXEC_MAPPINGS) + pudval |= PUD_TABLE_XN; BUG_ON(!pgtable_alloc); pmd_phys = pgtable_alloc(PMD_SHIFT); - __pud_populate(pudp, pmd_phys, PUD_TYPE_TABLE); + __pud_populate(pudp, pmd_phys, pudval); pud = READ_ONCE(*pudp); } BUG_ON(pud_bad(pud)); @@ -306,10 +315,14 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, p4d_t p4d = READ_ONCE(*p4dp); if (p4d_none(p4d)) { + p4dval_t p4dval = P4D_TYPE_TABLE; phys_addr_t pud_phys; + + if (flags & NO_EXEC_MAPPINGS) + p4dval |= P4D_TABLE_XN; BUG_ON(!pgtable_alloc); pud_phys = pgtable_alloc(PUD_SHIFT); - __p4d_populate(p4dp, pud_phys, P4D_TYPE_TABLE); + __p4d_populate(p4dp, pud_phys, p4dval); p4d = READ_ONCE(*p4dp); } BUG_ON(p4d_bad(p4d)); @@ -489,11 +502,11 @@ static void __init map_mem(pgd_t *pgdp) phys_addr_t kernel_start = __pa_symbol(_stext); phys_addr_t kernel_end = __pa_symbol(__init_begin); phys_addr_t start, end; - int flags = 0; + int flags = NO_EXEC_MAPPINGS; u64 i; if (rodata_full || crash_mem_map || debug_pagealloc_enabled()) - flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; + flags |= NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; /* * Take care not to create a writable alias for the @@ -1462,7 +1475,7 @@ struct range arch_get_mappable_range(void) int arch_add_memory(int nid, u64 start, u64 size, struct mhp_params *params) { - int ret, flags = 0; + int ret, flags = NO_EXEC_MAPPINGS; VM_BUG_ON(!mhp_range_allowed(start, size, true)); @@ -1472,7 +1485,7 @@ int arch_add_memory(int nid, u64 start, u64 size, */ if (rodata_full || debug_pagealloc_enabled() || IS_ENABLED(CONFIG_KFENCE)) - flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; + flags |= NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; __create_pgd_mapping(swapper_pg_dir, start, __phys_to_virt(start), size, params->pgprot, __pgd_pgtable_alloc,