From patchwork Wed May 11 02:27:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A. Shutemov" X-Patchwork-Id: 12845714 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A795C4332F for ; Wed, 11 May 2022 02:29:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4678F8D0009; Tue, 10 May 2022 22:29:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3569C8D0006; Tue, 10 May 2022 22:29:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F29FA8D000A; Tue, 10 May 2022 22:29:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B0D918D0006 for ; Tue, 10 May 2022 22:29:46 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 8931821ECA for ; Wed, 11 May 2022 02:29:46 +0000 (UTC) X-FDA: 79451881572.04.0B7AC6B Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by imf01.hostedemail.com (Postfix) with ESMTP id 4C9A840003 for ; Wed, 11 May 2022 02:29:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652236185; x=1683772185; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Gd+nZWXT2Vj5sId1tBKnAzjGgf10Zdvpv1KhXHK183k=; b=VPid2hcyJ6Jb6XBWXu7YpqlU4hzhpJYfTWGlWQaskucPYnKCJMNyQ2Fu SyOSvBSZKUJt9AazIHs3nJbIjUTdQIsERG2jI2fKCFuQ/L4q7mjH2B8kL KbAZlU3ncadIbUPOkf7fccuDl928b56dngFI75PS/TudDc3R+us3zJr/T u7fgiJUD7mqIN0rHM4HHgcD5EY8diXMKwOvG3AQG+9cKuphOfabV+GwpW qBpBmgZl2sqhsaKPnejd3oy4QR5JBwWpcFtm4xZ34iW//bFBqPekQhQbl ggBkcBEBchuqD7vLaLkHtR3FnQ6cB4rKh6Zwgca2yOvh0S6LRZq3Zi6/j w==; X-IronPort-AV: E=McAfee;i="6400,9594,10343"; a="269695081" X-IronPort-AV: E=Sophos;i="5.91,215,1647327600"; d="scan'208";a="269695081" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 19:29:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,215,1647327600"; d="scan'208";a="670166557" Received: from black.fi.intel.com ([10.237.72.28]) by fmsmga002.fm.intel.com with ESMTP; 10 May 2022 19:29:40 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 69FD1646; Wed, 11 May 2022 05:28:01 +0300 (EEST) From: "Kirill A. Shutemov" To: Dave Hansen , Andy Lutomirski , Peter Zijlstra Cc: x86@kernel.org, Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , "H . J . Lu" , Andi Kleen , Rick Edgecombe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [RFCv2 09/10] x86/mm: Add userspace API to enable Linear Address Masking Date: Wed, 11 May 2022 05:27:50 +0300 Message-Id: <20220511022751.65540-11-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220511022751.65540-1-kirill.shutemov@linux.intel.com> References: <20220511022751.65540-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 4C9A840003 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=VPid2hcy; spf=none (imf01.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 192.55.52.115) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com X-Rspam-User: X-Stat-Signature: wryphsf5bad4znhj915ium4zgt6u4rwf X-HE-Tag: 1652236174-788169 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow to enable Linear Address Masking via ARCH_THREAD_FEATURE_ENABLE arch_prctl(2). Signed-off-by: Kirill A. Shutemov --- arch/x86/kernel/process.c | 21 +++++++++++++++- arch/x86/kernel/process.h | 2 ++ arch/x86/kernel/process_64.c | 46 ++++++++++++++++++++++++++++++++++++ 3 files changed, 68 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index cb8fc28f2eae..911c24321312 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -46,6 +46,8 @@ #include #include #include +#include +#include #include "process.h" @@ -992,7 +994,9 @@ unsigned long __get_wchan(struct task_struct *p) static long thread_feature_prctl(struct task_struct *task, int option, unsigned long features) { - const unsigned long known_features = 0; + const unsigned long known_features = + X86_THREAD_LAM_U48 | + X86_THREAD_LAM_U57; if (features & ~known_features) return -EINVAL; @@ -1013,8 +1017,23 @@ static long thread_feature_prctl(struct task_struct *task, int option, /* Handle ARCH_THREAD_FEATURE_ENABLE */ + if (features & (X86_THREAD_LAM_U48 | X86_THREAD_LAM_U57)) { + long ret; + + /* LAM is only available in long mode */ + if (in_32bit_syscall()) + return -EINVAL; + + ret = enable_lam(task, features); + if (ret) + return ret; + } + task->thread.features |= features; out: + /* Update CR3 to get LAM active */ + switch_mm(task->mm, task->mm, task); + return task->thread.features; } diff --git a/arch/x86/kernel/process.h b/arch/x86/kernel/process.h index 76b547b83232..b8fa0e599c6e 100644 --- a/arch/x86/kernel/process.h +++ b/arch/x86/kernel/process.h @@ -4,6 +4,8 @@ #include +long enable_lam(struct task_struct *task, unsigned long features); + void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p); /* diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index e459253649be..a25c51da7005 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -729,6 +729,52 @@ void set_personality_ia32(bool x32) } EXPORT_SYMBOL_GPL(set_personality_ia32); +static bool lam_u48_allowed(void) +{ + struct mm_struct *mm = current->mm; + + if (!full_va_allowed(mm)) + return true; + + return find_vma(mm, DEFAULT_MAP_WINDOW) == NULL; +} + +long enable_lam(struct task_struct *task, unsigned long features) +{ + features |= task->thread.features; + + /* LAM_U48 and LAM_U57 are mutually exclusive */ + if ((features & X86_THREAD_LAM_U48) && (features & X86_THREAD_LAM_U57)) + return -EINVAL; + + if (!cpu_feature_enabled(X86_FEATURE_LAM)) + return -ENXIO; + + if (mmap_write_lock_killable(task->mm)) + return -EINTR; + + if ((features & X86_THREAD_LAM_U48) && !lam_u48_allowed()) { + mmap_write_unlock(task->mm); + return -EINVAL; + } + + /* + * Record the most permissive (allowing the widest tags) LAM + * mode to the mm context. It determinates if a mappings above + * 47 bit is allowed for the process. + * + * The mode is also used by a kernel thread when it does work + * on behalf of the process (like async I/O, io_uring, etc.) + */ + if (features & X86_THREAD_LAM_U48) + current->mm->context.lam = LAM_U48; + else if (current->mm->context.lam == LAM_NONE) + current->mm->context.lam = LAM_U57; + + mmap_write_unlock(task->mm); + return 0; +} + #ifdef CONFIG_CHECKPOINT_RESTORE static long prctl_map_vdso(const struct vdso_image *image, unsigned long addr) {