From patchwork Wed Oct 17 16:34:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Capper X-Patchwork-Id: 10645899 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 82C29157A for ; Wed, 17 Oct 2018 16:35:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 715BB28478 for ; Wed, 17 Oct 2018 16:35:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 650AC286FC; Wed, 17 Oct 2018 16:35:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA3A428478 for ; Wed, 17 Oct 2018 16:35:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AD5FC6B0269; Wed, 17 Oct 2018 12:35:25 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A83DF6B026A; Wed, 17 Oct 2018 12:35:25 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 973E46B026B; Wed, 17 Oct 2018 12:35:25 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-oi1-f198.google.com (mail-oi1-f198.google.com [209.85.167.198]) by kanga.kvack.org (Postfix) with ESMTP id 6CA7C6B0269 for ; Wed, 17 Oct 2018 12:35:25 -0400 (EDT) Received: by mail-oi1-f198.google.com with SMTP id u205-v6so2039130oie.5 for ; Wed, 17 Oct 2018 09:35:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=he7mJPA42CmiW0otUgvDEJIlYTU7T3zkFrzC6QrbR8A=; b=RQ1+iAUuV3YHlB+yGJP3T+Nu5KTInRvzBit+y9IjIfwUP6DaIVVahz4g0f44aVanAe P3qj7wOvwdLDaECwAoA2g6iVt+i0VuS3rRdJXZaJ3df30dcxNBZ7NwDpt5gAostBWA3A MkVaxkef76hRMzurN3CRbUEGOWMQBoRHd/wtJGkb+se39zmLXyPhAgiixuSp2eR/DojU tS422svXE7qV90OQZb8NyyLZDTGW4kadQXH3nonrRINWIdw2uqqoCv0t/DC2NMGQDn+L sdRQCCJg92PkvFBzuR+z8Okk2KNYSjtNWFk1Gj+EDxbX8VH6I+63hnxHFq5/IBqqJ6ig Ty6g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com X-Gm-Message-State: ABuFfoiEqbcbRgQoEQPFsqQsKh+nH4+NfRiBAEef5Y+yFJLrsoNuK3i1 hOHeRcPfrDR5Q1gfT8XHmsQN2p50fYRr2VDtVSyWU7x5kyeDGopgEgElYfxoWC1JivzCKo91xVh 9BZc6i6/kZaeJhK0233DdSj4wQ92IlK7drQ0/yHiFFdjtBYDocW9gmmr3QpFjgM7cqg== X-Received: by 2002:a9d:faa:: with SMTP id d39mr17579812otd.277.1539794125183; Wed, 17 Oct 2018 09:35:25 -0700 (PDT) X-Google-Smtp-Source: ACcGV61+/Y4lWCnc6t585DSCtD3v0JZphlSyXXaNYJoWzplH+yaQb5sNI7XbYqNAMunupu6zrYIb X-Received: by 2002:a9d:faa:: with SMTP id d39mr17579772otd.277.1539794124357; Wed, 17 Oct 2018 09:35:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539794124; cv=none; d=google.com; s=arc-20160816; b=NeFozbYv3EK9jFS42of5OWAlB0uDziQgH++y18cz7OG4NGMusIyUTlrTKfwDKHRnRC 7QWcVRTiA4+lyc8v7eoBvZkM42/vSWvxgsfZZiws0JCCzskbFGmsTjylFfOm4uMoA0dx UHfQ6vhxz1e/HQxL/hqbMJzZ9xPZXFZQA5YR1r9NjiA8IvPfU5diVJNItKQIGeMQi6im PPceqRETLxJip17pbaLiLODDruTolUbDKzH7MBTLNJAtjpNVjToKMkx/jB8TgBUl6aIH fgQ+1drqJ4Q1Ok6KRJzWuSkUm+N8x0DY0rw2FbgQkkiRpXIPDI+JqPQQsKA8/lEc97jn +V+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=he7mJPA42CmiW0otUgvDEJIlYTU7T3zkFrzC6QrbR8A=; b=vl838N4wQMV84DlNexLVRf490QVQ/1h+PR/vo7zlSL2BM8UAyVn/NKuFKrEYYj+QGN +KEbvJEoQZRsxUZ0TztZ66wrQ1UrMX6puJA9TCEe8BC0UO3Ctkbp9ZNebuafPYirss5N NtZve5gxXDPcDshKdlsoG5KmycTPJmJSmKiuejFWL4fm6uIXsDZltOUmP1ID8U5R0n+X 5JYpZhCWXuot9kjbBsnizEyihcmq0aBifMfZKJOfgUxlP/x++QLphAm4XQft5ZlGbk0I X/prgwKX1OVSdifS2MTvn0QwRoaDpd5b08PA/p2p0JwBZe14sRILTr6LsWjvdz2P88GK Vuqg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id q185-v6si8023443oia.57.2018.10.17.09.35.24 for ; Wed, 17 Oct 2018 09:35:24 -0700 (PDT) Received-SPF: pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A4839EBD; Wed, 17 Oct 2018 09:35:23 -0700 (PDT) Received: from capper-debian.emea.arm.com (unknown [10.32.102.177]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 674D13F71A; Wed, 17 Oct 2018 09:35:22 -0700 (PDT) From: Steve Capper To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, ard.biesheuvel@linaro.org, jcm@redhat.com, Steve Capper Subject: [PATCH V2 1/4] mm: mmap: Allow for "high" userspace addresses Date: Wed, 17 Oct 2018 17:34:56 +0100 Message-Id: <20181017163459.20175-2-steve.capper@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181017163459.20175-1-steve.capper@arm.com> References: <20181017163459.20175-1-steve.capper@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch adds support for "high" userspace addresses that are optionally supported on the system and have to be requested via a hint mechanism ("high" addr parameter to mmap). Rather than duplicate the arch_get_unmapped_* stock implementations, this patch instead introduces two architectural helper macros and applies them to arch_get_unmapped_*: arch_get_mmap_end(addr) - get mmap upper limit depending on addr hint arch_get_mmap_base(addr, base) - get mmap_base depending on addr hint If these macros are not defined in architectural code then they default to (TASK_SIZE) and (base) so should not introduce any behavioural changes to architectures that do not define them. Signed-off-by: Steve Capper --- Changed in V2, these alterations are made to mm/mmap.c rather than copied over to arch/arm64 and modified there. --- mm/mmap.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 5f2b2b184c60..396b8ae12783 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2042,6 +2042,15 @@ unsigned long unmapped_area_topdown(struct vm_unmapped_area_info *info) return gap_end; } + +#ifndef arch_get_mmap_end +#define arch_get_mmap_end(addr) (TASK_SIZE) +#endif + +#ifndef arch_get_mmap_base +#define arch_get_mmap_base(addr, base) (base) +#endif + /* Get an address range which is currently unmapped. * For shmat() with addr=0. * @@ -2061,8 +2070,9 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr, struct mm_struct *mm = current->mm; struct vm_area_struct *vma, *prev; struct vm_unmapped_area_info info; + const unsigned long mmap_end = arch_get_mmap_end(addr); - if (len > TASK_SIZE - mmap_min_addr) + if (len > mmap_end - mmap_min_addr) return -ENOMEM; if (flags & MAP_FIXED) @@ -2071,7 +2081,7 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr, if (addr) { addr = PAGE_ALIGN(addr); vma = find_vma_prev(mm, addr, &prev); - if (TASK_SIZE - len >= addr && addr >= mmap_min_addr && + if (mmap_end - len >= addr && addr >= mmap_min_addr && (!vma || addr + len <= vm_start_gap(vma)) && (!prev || addr >= vm_end_gap(prev))) return addr; @@ -2080,7 +2090,7 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr, info.flags = 0; info.length = len; info.low_limit = mm->mmap_base; - info.high_limit = TASK_SIZE; + info.high_limit = mmap_end; info.align_mask = 0; return vm_unmapped_area(&info); } @@ -2100,9 +2110,10 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, struct mm_struct *mm = current->mm; unsigned long addr = addr0; struct vm_unmapped_area_info info; + const unsigned long mmap_end = arch_get_mmap_end(addr); /* requested length too big for entire address space */ - if (len > TASK_SIZE - mmap_min_addr) + if (len > mmap_end - mmap_min_addr) return -ENOMEM; if (flags & MAP_FIXED) @@ -2112,7 +2123,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, if (addr) { addr = PAGE_ALIGN(addr); vma = find_vma_prev(mm, addr, &prev); - if (TASK_SIZE - len >= addr && addr >= mmap_min_addr && + if (mmap_end - len >= addr && addr >= mmap_min_addr && (!vma || addr + len <= vm_start_gap(vma)) && (!prev || addr >= vm_end_gap(prev))) return addr; @@ -2121,7 +2132,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, info.flags = VM_UNMAPPED_AREA_TOPDOWN; info.length = len; info.low_limit = max(PAGE_SIZE, mmap_min_addr); - info.high_limit = mm->mmap_base; + info.high_limit = arch_get_mmap_base(addr, mm->mmap_base); info.align_mask = 0; addr = vm_unmapped_area(&info); @@ -2135,7 +2146,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, VM_BUG_ON(addr != -ENOMEM); info.flags = 0; info.low_limit = TASK_UNMAPPED_BASE; - info.high_limit = TASK_SIZE; + info.high_limit = mmap_end; addr = vm_unmapped_area(&info); } From patchwork Wed Oct 17 16:34:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Capper X-Patchwork-Id: 10645901 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EC46F157A for ; Wed, 17 Oct 2018 16:35:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA5691FF82 for ; Wed, 17 Oct 2018 16:35:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CE6BF2851E; Wed, 17 Oct 2018 16:35:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C4A41FF82 for ; Wed, 17 Oct 2018 16:35:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72CBF6B026B; Wed, 17 Oct 2018 12:35:28 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6DC236B026C; Wed, 17 Oct 2018 12:35:28 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5CAA16B026D; Wed, 17 Oct 2018 12:35:28 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ot1-f70.google.com (mail-ot1-f70.google.com [209.85.210.70]) by kanga.kvack.org (Postfix) with ESMTP id 2EEFE6B026B for ; Wed, 17 Oct 2018 12:35:28 -0400 (EDT) Received: by mail-ot1-f70.google.com with SMTP id s2so20060560ote.13 for ; Wed, 17 Oct 2018 09:35:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=7S8chC9yT3/FjpFMfT4/JB6spqdASyIZ+/YqjjnY9kU=; b=BoxQuqnr8NYQzPnEzOanK+bsE5ZDydVsro0TNCCjfNOmX+6KfpdEwoNZICnrVcbMiO z1ZLYvz2LWNFWXEjxA0n/yUXrUcwmPG4agKA+xCGqa74iysmjveWvvu+5W+q343pnNdD +o5spEE4An9j5Cg9kBFPiPJ5Gu0xL7zGe4GSaNgQFqetiaOdwGHkV7+yFgi/So6T+m88 NPdgcfHz02cAMd0N0jnrTtcoRaWL+0Guh+oIMePXE6//+gl8ZIuQyBWaoFJWvkmxV946 f5TvUolLcvqfCCP79XbfpPvNG/+dveSISVlzq20GmG/IuzuJXIS4yo+sgpghMOh7JJN7 5e5A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com X-Gm-Message-State: ABuFfogUTlyUKc2NOuo/6THN1cQS5+TSvTFFB8nms6UTtZe7jglzZd9z W8+m6YPNwU/vI+i9aLk3UHSOckpWOOkpa2PKyzVkcPlx7DP9XYOJvyS3cQHqImwyDAds8nRHh5K HV8Th0gTFau9b/1T8SHI9bH8D3iw++BtsDSUPOvbAfyYUiIv6HFRLBM8Zsbv/C5nToA== X-Received: by 2002:a9d:f07:: with SMTP id 7mr17881058ott.170.1539794127916; Wed, 17 Oct 2018 09:35:27 -0700 (PDT) X-Google-Smtp-Source: ACcGV621vdeLJ/5k0uR4ebFU6L0InePGrPivCXqc2GIGhtb0o2c4ENDAm09OXD5jQkSu/GhC1jKL X-Received: by 2002:a9d:f07:: with SMTP id 7mr17881027ott.170.1539794127022; Wed, 17 Oct 2018 09:35:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539794126; cv=none; d=google.com; s=arc-20160816; b=G5al2mSKD4NBSRerupa+ee03uwh+9jGWxtkidwmOB/pEFVKaBGlssP2Ewrhg/Sd3Oo KYrnm0Sd9PhhfAM9IYLLe4BfVbyoeRUtR1lW8+ZF0j5raaaFC5MN7vCpWKN9gDyX3/TL cHGJiTjax0Gtm1Z0cKt23uyvKC+56Wxxmpc0ic+m99bKuBJuwEOyaXfoaDGGu8Yibn98 6gU6KucZXVMeqypkWLvNIYTZ1meCg/I9B0HHLcWuNo2hzX43c0I72Fnjdnmklt3pYJNj 2rXaKYrHlZAUcqTl8S8bp771zB1OFlUYRbjZNmHZRAkvpc5Ge/RU7JQnygFNWbKzpUK8 /aqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=7S8chC9yT3/FjpFMfT4/JB6spqdASyIZ+/YqjjnY9kU=; b=L0Dnf7XmZYihxTM2LwpdkNhRTPErs5EOpzbfcEYNIBgHOTb0ct078RJpcT+fk9BkXU pCS/UsrBbK1kHgxFevAz7xNWvjI6XrMVHmBz6BWP3AtkaiVcxE6/7pggYi0Z8ja5Ntau 7llCAx4+keTpPYckZcj+HmGSS9dCn2hVGVe5RJw2T2kjJuJJFyILBmvD06mv6CW22tm/ kovYd48o5n8yCP5bbxat5GMPDbDXunn45ZIGwi/W44jClxK0JWCC0Qn27yclXmZo7DOv Pt4WWR9Mk6yHKd+sX0a3JMTNMW+3DLS8j+XBUACjlqRpslTA5jxpa5yd6TCAjx44Ikrr oTDg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id w67-v6si8471795oig.136.2018.10.17.09.35.26 for ; Wed, 17 Oct 2018 09:35:26 -0700 (PDT) Received-SPF: pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3EEB11650; Wed, 17 Oct 2018 09:35:26 -0700 (PDT) Received: from capper-debian.emea.arm.com (unknown [10.32.102.177]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C6E283F71A; Wed, 17 Oct 2018 09:35:24 -0700 (PDT) From: Steve Capper To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, ard.biesheuvel@linaro.org, jcm@redhat.com, Steve Capper Subject: [PATCH V2 2/4] arm64: mm: Introduce DEFAULT_MAP_WINDOW Date: Wed, 17 Oct 2018 17:34:57 +0100 Message-Id: <20181017163459.20175-3-steve.capper@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181017163459.20175-1-steve.capper@arm.com> References: <20181017163459.20175-1-steve.capper@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP We wish to introduce a 52-bit virtual address space for userspace but maintain compatibility with software that assumes the maximum VA space size is 48 bit. In order to achieve this, on 52-bit VA systems, we make mmap behave as if it were running on a 48-bit VA system (unless userspace explicitly requests a VA where addr[51:48] != 0). On a system running a 52-bit userspace we need TASK_SIZE to represent the 52-bit limit as it is used in various places to distinguish between kernelspace and userspace addresses. Thus we need a new limit for mmap, stack, ELF loader and EFI (which uses TTBR0) to represent the non-extended VA space. This patch introduces DEFAULT_MAP_WINDOW and DEFAULT_MAP_WINDOW_64 and switches the appropriate logic to use that instead of TASK_SIZE. Signed-off-by: Steve Capper --- arch/arm64/include/asm/elf.h | 2 +- arch/arm64/include/asm/processor.h | 9 +++++++-- drivers/firmware/efi/arm-runtime.c | 2 +- drivers/firmware/efi/libstub/arm-stub.c | 2 +- 4 files changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h index 433b9554c6a1..bc9bd9e77d9d 100644 --- a/arch/arm64/include/asm/elf.h +++ b/arch/arm64/include/asm/elf.h @@ -117,7 +117,7 @@ * 64-bit, this is above 4GB to leave the entire 32-bit address * space open for things that want to use the area for 32-bit pointers. */ -#define ELF_ET_DYN_BASE (2 * TASK_SIZE_64 / 3) +#define ELF_ET_DYN_BASE (2 * DEFAULT_MAP_WINDOW_64 / 3) #ifndef __ASSEMBLY__ diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 79657ad91397..46c9d9ff028c 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -26,6 +26,8 @@ #ifndef __ASSEMBLY__ +#define DEFAULT_MAP_WINDOW_64 (UL(1) << VA_BITS) + /* * Default implementation of macro that returns current * instruction pointer ("program counter"). @@ -58,13 +60,16 @@ TASK_SIZE_32 : TASK_SIZE_64) #define TASK_SIZE_OF(tsk) (test_tsk_thread_flag(tsk, TIF_32BIT) ? \ TASK_SIZE_32 : TASK_SIZE_64) +#define DEFAULT_MAP_WINDOW (test_tsk_thread_flag(tsk, TIF_32BIT) ? \ + TASK_SIZE_32 : DEFAULT_MAP_WINDOW_64) #else #define TASK_SIZE TASK_SIZE_64 +#define DEFAULT_MAP_WINDOW DEFAULT_MAP_WINDOW_64 #endif /* CONFIG_COMPAT */ -#define TASK_UNMAPPED_BASE (PAGE_ALIGN(TASK_SIZE / 4)) +#define TASK_UNMAPPED_BASE (PAGE_ALIGN(DEFAULT_MAP_WINDOW / 4)) +#define STACK_TOP_MAX DEFAULT_MAP_WINDOW_64 -#define STACK_TOP_MAX TASK_SIZE_64 #ifdef CONFIG_COMPAT #define AARCH32_VECTORS_BASE 0xffff0000 #define STACK_TOP (test_thread_flag(TIF_32BIT) ? \ diff --git a/drivers/firmware/efi/arm-runtime.c b/drivers/firmware/efi/arm-runtime.c index 922cfb813109..952cec5b611a 100644 --- a/drivers/firmware/efi/arm-runtime.c +++ b/drivers/firmware/efi/arm-runtime.c @@ -38,7 +38,7 @@ static struct ptdump_info efi_ptdump_info = { .mm = &efi_mm, .markers = (struct addr_marker[]){ { 0, "UEFI runtime start" }, - { TASK_SIZE_64, "UEFI runtime end" } + { DEFAULT_MAP_WINDOW_64, "UEFI runtime end" } }, .base_addr = 0, }; diff --git a/drivers/firmware/efi/libstub/arm-stub.c b/drivers/firmware/efi/libstub/arm-stub.c index 6920033de6d4..ac297c20ab1e 100644 --- a/drivers/firmware/efi/libstub/arm-stub.c +++ b/drivers/firmware/efi/libstub/arm-stub.c @@ -33,7 +33,7 @@ #define EFI_RT_VIRTUAL_SIZE SZ_512M #ifdef CONFIG_ARM64 -# define EFI_RT_VIRTUAL_LIMIT TASK_SIZE_64 +# define EFI_RT_VIRTUAL_LIMIT DEFAULT_MAP_WINDOW_64 #else # define EFI_RT_VIRTUAL_LIMIT TASK_SIZE #endif From patchwork Wed Oct 17 16:34:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Capper X-Patchwork-Id: 10645903 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A00AD13B0 for ; Wed, 17 Oct 2018 16:35:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 908201FF82 for ; Wed, 17 Oct 2018 16:35:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 827B72851E; Wed, 17 Oct 2018 16:35:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A7B61FF82 for ; Wed, 17 Oct 2018 16:35:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 248FB6B026D; Wed, 17 Oct 2018 12:35:31 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1F9D36B026E; Wed, 17 Oct 2018 12:35:31 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09AF26B026F; Wed, 17 Oct 2018 12:35:31 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-oi1-f197.google.com (mail-oi1-f197.google.com [209.85.167.197]) by kanga.kvack.org (Postfix) with ESMTP id D84286B026D for ; Wed, 17 Oct 2018 12:35:30 -0400 (EDT) Received: by mail-oi1-f197.google.com with SMTP id u205-v6so2039596oie.5 for ; Wed, 17 Oct 2018 09:35:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=bFMoRGJYhwlJofsu69UJCc+41ttAzbYKc6tl0AmOdOY=; b=hhKkCn008xz/TWpvFkgmZTDwot88NHR0VqnwuN+UM7+2spSrjhl9WPrRN9VnnrMhuS 9ckfKNxSKXA9csDqQDac7RmTJPRlBS5/vwfoNzqJMRu01gd5I/tMXWgCG4zkOfCK4bcD f9KZGNwBWj3xEJgygPg2f5mHstB28G4lF85r9EganXuSWUdMDqMl8YThUFPXPsB3Lhhb 1w1iZM8k65ZQbXoE4H4w+84T0PXwU/monPRJ51Oe0VO6hwU9WhUAW5aAlqUXY1QJC/Em JdpLU1e2socBeItCynH3NxtCRgcuBG7X5IfWagodtyfuTIAzExF/cHskhNFLRn8xJ1B8 fxVg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com X-Gm-Message-State: ABuFfog0vP413kN/4fD1bGeWFQ+G6S8ds/c92tWAEcvlnPyHu/GD8eJS tcwWiZMe5VoEYrlE8CufWabpyXDUk7tpyiRVg1w0SCQtOXi06Z77JeySgROx4yQMNeXB7cp/FeI Ey8chDXVObsunBXcO7vonpC35ubY5JupasVXOHdb1NsFQ+6oGX9kki2m1NNs5qetp0w== X-Received: by 2002:a9d:6014:: with SMTP id h20mr7193507otj.349.1539794130664; Wed, 17 Oct 2018 09:35:30 -0700 (PDT) X-Google-Smtp-Source: ACcGV63CBgEp9H/OgtL9TjvmK3KG4+WODJHMkpFoh9LOWWTMH+ARq+O35B5ifZgFX768ew7sXM27 X-Received: by 2002:a9d:6014:: with SMTP id h20mr7193467otj.349.1539794129604; Wed, 17 Oct 2018 09:35:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539794129; cv=none; d=google.com; s=arc-20160816; b=yB95aGo7OUCDwFXXlIlbBcD3/xAr77CM4rqMZwhnrfZ1zW3AWsB58aELd255o77wCU XZiK72CF9oh/rou2PPl0R/S1R6NZbUE0jhht4BgJlo55kZxTeQtKMer4d4ceTpjx3/pp YRzCaHDQOrs2I4eZTP13F43M2goSOxOcloL0NBU/u5QeBzOnvJPhqN6YhonBn30uiJ6B s+dXxzM9H5dHKqAopA01lKRucFOBUS1fJp48XWm691PCv8ddL3Bc/ZxWk0ugP5xLEgrn rng2XX4ZMYikuvJfpkOeCUxuoGs0bDAclIxe0fms1ntgEj6TaZ2bvbr9mjvzyenO/DU/ NgKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=bFMoRGJYhwlJofsu69UJCc+41ttAzbYKc6tl0AmOdOY=; b=g7gjlli9bUIUwg5gosQDY4AQ+EanlxgPxGpHuAoJcZXXBdApaEbXIFd2Y03GDqNmHH 7Mv3rF0Z/Mt6vmknwz8zoXl3uIEMVKp5Hg6i0fdaEl8JFeXeKno33Aul1SXgIabh9Kui udQwZ+8nEGZnkO4nV4gwnK4gmj3Xw3eoJaAILdQsc0kcYtKHQdsO3X8ey5twG4U/0deI uh2J5jOvfrKVMu949S1TSBzf3D9W8xNq+7FDVPZ6fDtB6lYWlbsc4y5CE4Pn+Lj2oMO0 ti3kNLnIG1K5hfwMZXzadMFlx8DriytNpfl+IfQOsuFPgvEy9CPbLlbHII//Svd7JPcW jkvQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id s83-v6si8716712oie.222.2018.10.17.09.35.29 for ; Wed, 17 Oct 2018 09:35:29 -0700 (PDT) Received-SPF: pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 15E93165C; Wed, 17 Oct 2018 09:35:29 -0700 (PDT) Received: from capper-debian.emea.arm.com (unknown [10.32.102.177]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9FB083F71A; Wed, 17 Oct 2018 09:35:27 -0700 (PDT) From: Steve Capper To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, ard.biesheuvel@linaro.org, jcm@redhat.com, Steve Capper Subject: [PATCH V2 3/4] arm64: mm: Define arch_get_mmap_end, arch_get_mmap_base Date: Wed, 17 Oct 2018 17:34:58 +0100 Message-Id: <20181017163459.20175-4-steve.capper@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181017163459.20175-1-steve.capper@arm.com> References: <20181017163459.20175-1-steve.capper@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Now that we have DEFAULT_MAP_WINDOW defined, we can arch_get_mmap_end and arch_get_mmap_base helpers to allow for high addresses in mmap. Signed-off-by: Steve Capper --- arch/arm64/include/asm/processor.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 46c9d9ff028c..5afc0c5eb1cb 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -78,6 +78,13 @@ #define STACK_TOP STACK_TOP_MAX #endif /* CONFIG_COMPAT */ +#define arch_get_mmap_end(addr) ((addr > DEFAULT_MAP_WINDOW) ? TASK_SIZE :\ + DEFAULT_MAP_WINDOW) + +#define arch_get_mmap_base(addr, base) ((addr > DEFAULT_MAP_WINDOW) ? \ + base + TASK_SIZE - DEFAULT_MAP_WINDOW :\ + base) + extern phys_addr_t arm64_dma_phys_limit; #define ARCH_LOW_ADDRESS_LIMIT (arm64_dma_phys_limit - 1) From patchwork Wed Oct 17 16:34:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Capper X-Patchwork-Id: 10645907 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 130AD157A for ; Wed, 17 Oct 2018 16:35:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0336E1FF82 for ; Wed, 17 Oct 2018 16:35:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EB7942851E; Wed, 17 Oct 2018 16:35:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 300DF1FF82 for ; Wed, 17 Oct 2018 16:35:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 04E2C6B026F; Wed, 17 Oct 2018 12:35:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F406A6B0270; Wed, 17 Oct 2018 12:35:33 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E56AE6B0271; Wed, 17 Oct 2018 12:35:33 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-oi1-f198.google.com (mail-oi1-f198.google.com [209.85.167.198]) by kanga.kvack.org (Postfix) with ESMTP id B84B06B026F for ; Wed, 17 Oct 2018 12:35:33 -0400 (EDT) Received: by mail-oi1-f198.google.com with SMTP id f62-v6so18558366oia.2 for ; Wed, 17 Oct 2018 09:35:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=oC3rbFUCZOyTUFdsYcbkYk3Mf36Gfx7DAcVRyM7qQY4=; b=r/lx4GfrIzDAJGWooZ+L0NTJJyP3hYPmtQ6Qv6xcB26/vExk00XcnpaZebnxwklk4J zdpIVNaVRrIc49dFZ1qVF3MSXQNoFy8hEaaPrM8SQt4Dn0mOhIiy6Tf6LhUB4HG8201j 7WMR6Ks3PJU9SHODKPkzc5YerykTyDbgiyqwUQE/DFAaMzMlqr8mKfmpEJis1X3SuHA1 rfN/WGmoMGKLswa27BOw6muh8/bvdrlodF3NQ2Ui0AxYHP+zsI+BOM4MfOxnl9LZ257L KtS1jML1zipxv9XlUKAWvUrT2fsiucivC7HbiIA8KIroD54xJbwkPmN4zAW+z16f0sfg Qo5g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com X-Gm-Message-State: ABuFfogwsyHOq7rNb11+MhjYBDP9z0oi8fhuXipq/Mjp9uF7gajYAt6o SPXgepcKFHWJl8LoTa003TsksHLtraM5hIv6ER/hUk9NosmWA4ZBIY+o5XUdYFf3mTASZBMraz2 KFqFQrbIFcnR8rOUKynk0909iIZ5Me/yzIrmHAVWbyUyf1zpFm5U89z9jzOyhBvwuIw== X-Received: by 2002:aca:3884:: with SMTP id f126-v6mr14680716oia.249.1539794133440; Wed, 17 Oct 2018 09:35:33 -0700 (PDT) X-Google-Smtp-Source: ACcGV62ZC62lkLCZ9hNpp6PTo0yRtFdOTkVqZyhLJWfUl/9X3VxcVx1bduin2URlBvHd6naRZ7QR X-Received: by 2002:aca:3884:: with SMTP id f126-v6mr14680679oia.249.1539794132388; Wed, 17 Oct 2018 09:35:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539794132; cv=none; d=google.com; s=arc-20160816; b=ZoLqEKRuQIQkYCqPKLxRjxCA0fRkQOgqYA3mWP2jnFx78j0ZygzYlDhfAM/p62XQBf zibMT7XWUWyPC6C0ZdJwgK8ciSs94dapOPXrSg3eAp5y6V82pUsFb4ogcT74ytOprR2L lcNWdiSu8xlTGluLi3alJ7adnFH0+aCm/0DiJizeWQpf4iYUjst8w2irc/pxiPFZD1c8 s3SrDRtXgXL7h/b3nCGC4VnGqLXVbUH3WlKcn+uUkDtOcJsTebQn4PT9HuDLr3jJ+pAM Nqoha73cpxprTglEpAOcEDf8E+hVPvPatPaBJsHzjyd33u+cIh7duivnd8NCV5QrtydH ChLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=oC3rbFUCZOyTUFdsYcbkYk3Mf36Gfx7DAcVRyM7qQY4=; b=IySpFnF5CCljAetoyNHtohXFXQJ6sPzDXuwNrgAdNSSw6832hv0WyONCinMsXzO+S8 hJEGgbG9maAj4jQc+YzlCdVutu8XvA2/F+emEZvDUFyY9Vgf0bhoiCJUTXZ5gWhZT/Rr 6pXi7rn4uVj7C8jP3GcT9JF3xLmOCkaacs5SvcllbWySIdE4yyq/pGNrhkzCcMBNd7f+ oyQIUzyuu66h9rJi1UFfdB4JipB0Nm1nKp1DUHTROd0w3LwXTiVuhNpf75N8sWkxWI61 SDOYyedRgg1UO9i6gHsR/V9Jk9wiYfhqLU0zlcAPVbH5FaOFRUxWVyvAPM2WcEufPOgt MdQA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id r18si8716199ote.203.2018.10.17.09.35.32 for ; Wed, 17 Oct 2018 09:35:32 -0700 (PDT) Received-SPF: pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B1BDE15AB; Wed, 17 Oct 2018 09:35:31 -0700 (PDT) Received: from capper-debian.emea.arm.com (unknown [10.32.102.177]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 4EA7D3F71A; Wed, 17 Oct 2018 09:35:30 -0700 (PDT) From: Steve Capper To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, ard.biesheuvel@linaro.org, jcm@redhat.com, Steve Capper Subject: [PATCH V2 4/4] arm64: mm: introduce 52-bit userspace support Date: Wed, 17 Oct 2018 17:34:59 +0100 Message-Id: <20181017163459.20175-5-steve.capper@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181017163459.20175-1-steve.capper@arm.com> References: <20181017163459.20175-1-steve.capper@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP On arm64 there is optional support for a 52-bit virtual address space. To exploit this one has to be running with a 64KB page size and be running on hardware that supports this. For an arm64 kernel supporting a 48 bit VA with a 64KB page size, a few changes are needed to support a 52-bit userspace: * TCR_EL1.T0SZ needs to be 12 instead of 16, * pgd_offset needs to work with a different PTRS_PER_PGD, * PGD_SIZE needs to be increased, * TASK_SIZE needs to reflect the new size. This patch implements the above when the support for 52-bit VAs is detected at early boot time. On arm64 userspace addresses translation is controlled by TTBR0_EL1. As well as userspace, TTBR0_EL1 controls: * The identity mapping, * EFI runtime code. It is possible to run a kernel with an identity mapping that has a larger VA size than userspace (and for this case __cpu_set_tcr_t0sz() would set TCR_EL1.T0SZ as appropriate). However, when the conditions for 52-bit userspace are met; it is possible to keep TCR_EL1.T0SZ fixed at 12. Thus in this patch, the TCR_EL1.T0SZ size changing logic is disabled. Signed-off-by: Steve Capper --- Changed in V2: variable USER_DS limit dropped, this simplifies the patch considerably. We check for userspace mappings with init_mm rather than looking at the virtual address. --- arch/arm64/Kconfig | 4 ++++ arch/arm64/include/asm/assembler.h | 7 +++---- arch/arm64/include/asm/mmu_context.h | 3 +++ arch/arm64/include/asm/pgalloc.h | 4 ++++ arch/arm64/include/asm/pgtable.h | 16 +++++++++++++--- arch/arm64/include/asm/processor.h | 13 ++++++++----- arch/arm64/kernel/head.S | 13 +++++++++++++ arch/arm64/mm/fault.c | 2 +- arch/arm64/mm/mmu.c | 1 + arch/arm64/mm/proc.S | 10 +++++++++- 10 files changed, 59 insertions(+), 14 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 1b1a0e95c751..ce6ebba101de 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -691,6 +691,10 @@ config ARM64_PA_BITS_52 endchoice +config ARM64_52BIT_VA + def_bool y + depends on ARM64_VA_BITS_48 && ARM64_64K_PAGES + config ARM64_PA_BITS int default 48 if ARM64_PA_BITS_48 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 0bcc98dbba56..8c8ed20beca9 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -343,11 +343,10 @@ alternative_endif .endm /* - * tcr_set_idmap_t0sz - update TCR.T0SZ so that we can load the ID map + * tcr_set_t0sz - update TCR.T0SZ so that we can load the ID map */ - .macro tcr_set_idmap_t0sz, valreg, tmpreg - ldr_l \tmpreg, idmap_t0sz - bfi \valreg, \tmpreg, #TCR_T0SZ_OFFSET, #TCR_TxSZ_WIDTH + .macro tcr_set_t0sz, valreg, t0sz + bfi \valreg, \t0sz, #TCR_T0SZ_OFFSET, #TCR_TxSZ_WIDTH .endm /* diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index 39ec0b8a689e..1b7408e8d794 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -72,6 +72,9 @@ extern u64 idmap_ptrs_per_pgd; static inline bool __cpu_uses_extended_idmap(void) { + if (IS_ENABLED(CONFIG_ARM64_52BIT_VA)) + return false; + return unlikely(idmap_t0sz != TCR_T0SZ(VA_BITS)); } diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h index 2e05bcd944c8..56c3ccabeffe 100644 --- a/arch/arm64/include/asm/pgalloc.h +++ b/arch/arm64/include/asm/pgalloc.h @@ -27,7 +27,11 @@ #define check_pgt_cache() do { } while (0) #define PGALLOC_GFP (GFP_KERNEL | __GFP_ZERO) +#ifdef CONFIG_ARM64_52BIT_VA +#define PGD_SIZE ((1 << (52 - PGDIR_SHIFT)) * sizeof(pgd_t)) +#else #define PGD_SIZE (PTRS_PER_PGD * sizeof(pgd_t)) +#endif #if CONFIG_PGTABLE_LEVELS > 2 diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 1bdeca8918a6..b18df27b06e0 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -577,11 +577,21 @@ static inline phys_addr_t pgd_page_paddr(pgd_t pgd) #define pgd_ERROR(pgd) __pgd_error(__FILE__, __LINE__, pgd_val(pgd)) /* to find an entry in a page-table-directory */ -#define pgd_index(addr) (((addr) >> PGDIR_SHIFT) & (PTRS_PER_PGD - 1)) +#define pgd_index(addr, ptrs) (((addr) >> PGDIR_SHIFT) & ((ptrs) - 1)) +#define _pgd_offset_raw(pgd, addr, ptrs) ((pgd) + pgd_index(addr, ptrs)) +#define pgd_offset_raw(pgd, addr) (_pgd_offset_raw(pgd, addr, PTRS_PER_PGD)) -#define pgd_offset_raw(pgd, addr) ((pgd) + pgd_index(addr)) +static inline pgd_t *pgd_offset(const struct mm_struct *mm, unsigned long addr) +{ + pgd_t *ret; + + if (IS_ENABLED(CONFIG_ARM64_52BIT_VA) && (mm != &init_mm)) + ret = _pgd_offset_raw(mm->pgd, addr, 1ULL << (vabits_user - PGDIR_SHIFT)); + else + ret = pgd_offset_raw(mm->pgd, addr); -#define pgd_offset(mm, addr) (pgd_offset_raw((mm)->pgd, (addr))) + return ret; +} /* to find an entry in a kernel page-table-directory */ #define pgd_offset_k(addr) pgd_offset(&init_mm, addr) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 5afc0c5eb1cb..886392ac38ff 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -19,13 +19,16 @@ #ifndef __ASM_PROCESSOR_H #define __ASM_PROCESSOR_H -#define TASK_SIZE_64 (UL(1) << VA_BITS) - -#define KERNEL_DS UL(-1) -#define USER_DS (TASK_SIZE_64 - 1) - +#define KERNEL_DS UL(-1) +#ifdef CONFIG_ARM64_52BIT_VA +#define USER_DS ((UL(1) << 52) - 1) +#else +#define USER_DS ((UL(1) << VA_BITS) - 1) +#endif /* CONFIG_ARM64_52IT_VA */ #ifndef __ASSEMBLY__ +extern u64 vabits_user; +#define TASK_SIZE_64 (UL(1) << vabits_user) #define DEFAULT_MAP_WINDOW_64 (UL(1) << VA_BITS) /* diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index b0853069702f..796d119f2848 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -316,6 +316,19 @@ __create_page_tables: adrp x0, idmap_pg_dir adrp x3, __idmap_text_start // __pa(__idmap_text_start) +#ifdef CONFIG_ARM64_52BIT_VA + mrs_s x6, SYS_ID_AA64MMFR2_EL1 + and x6, x6, #(0xf << ID_AA64MMFR2_LVA_SHIFT) + mov x5, #52 + cbnz x6, 1f +#endif + mov x5, #VA_BITS +1: + adr_l x6, vabits_user + str x5, [x6] + dmb sy + dc ivac, x6 // Invalidate potentially stale cache line + /* * VA_BITS may be too small to allow for an ID mapping to be created * that covers system RAM if that is located sufficiently high in the diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 50b30ff30de4..cbf14bc14aa3 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -153,7 +153,7 @@ void show_pte(unsigned long addr) pr_alert("%s pgtable: %luk pages, %u-bit VAs, pgdp = %p\n", mm == &init_mm ? "swapper" : "user", PAGE_SIZE / SZ_1K, - VA_BITS, mm->pgd); + mm == &init_mm ? VA_BITS : (int) vabits_user, mm->pgd); pgdp = pgd_offset(mm, addr); pgd = READ_ONCE(*pgdp); pr_alert("[%016lx] pgd=%016llx", addr, pgd_val(pgd)); diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 8080c9f489c3..2aa9ea9bd2c2 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -52,6 +52,7 @@ u64 idmap_t0sz = TCR_T0SZ(VA_BITS); u64 idmap_ptrs_per_pgd = PTRS_PER_PGD; +u64 vabits_user __ro_after_init; u64 kimage_voffset __ro_after_init; EXPORT_SYMBOL(kimage_voffset); diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 03646e6a2ef4..bd01506a2555 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -441,7 +441,15 @@ ENTRY(__cpu_setup) ldr x10, =TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ TCR_TBI0 | TCR_A1 - tcr_set_idmap_t0sz x10, x9 + +#ifdef CONFIG_ARM64_52BIT_VA + ldr_l x9, vabits_user + sub x9, xzr, x9 + add x9, x9, #64 +#else + ldr_l x9, idmap_t0sz +#endif + tcr_set_t0sz x10, x9 /* * Set the IPS bits in TCR_EL1.