From patchwork Fri Feb 5 15:16:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A. Shutemov" X-Patchwork-Id: 12070331 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59C2AC433E0 for ; Fri, 5 Feb 2021 15:17:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DE5DD650E7 for ; Fri, 5 Feb 2021 15:16:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DE5DD650E7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 24B5C6B007B; Fri, 5 Feb 2021 10:16:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1AC4C6B0078; Fri, 5 Feb 2021 10:16:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EA5D26B007E; Fri, 5 Feb 2021 10:16:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0025.hostedemail.com [216.40.44.25]) by kanga.kvack.org (Postfix) with ESMTP id B92006B007B for ; Fri, 5 Feb 2021 10:16:51 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 827C4181AEF1A for ; Fri, 5 Feb 2021 15:16:51 +0000 (UTC) X-FDA: 77784566622.19.quill78_4f0bd20275e5 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id 331871AD1B7 for ; Fri, 5 Feb 2021 15:16:51 +0000 (UTC) X-HE-Tag: quill78_4f0bd20275e5 X-Filterd-Recvd-Size: 4387 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Fri, 5 Feb 2021 15:16:50 +0000 (UTC) IronPort-SDR: lvMOKTIkLFWAUw4EQbsE+I7eVLDTmN+FO3mZnWqF1AtSAmuFuEgOO4YUXhRbRk8EcHoZ2sZZT3 vMAWplf1Z1/Q== X-IronPort-AV: E=McAfee;i="6000,8403,9885"; a="169120215" X-IronPort-AV: E=Sophos;i="5.81,155,1610438400"; d="scan'208";a="169120215" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2021 07:16:49 -0800 IronPort-SDR: q7Q3i9vozj8sDWJ29m98QW/kh6uPo5gdVnuiVSBEnlnVm0Sa/QFVXswC/aPjxw1Gq46ZBttVQF AQTL4mzUhNvg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,155,1610438400"; d="scan'208";a="358274761" Received: from black.fi.intel.com ([10.237.72.28]) by fmsmga007.fm.intel.com with ESMTP; 05 Feb 2021 07:16:45 -0800 Received: by black.fi.intel.com (Postfix, from userid 1000) id 08F43397; Fri, 5 Feb 2021 17:16:41 +0200 (EET) From: "Kirill A. Shutemov" To: Dave Hansen , Andy Lutomirski , Peter Zijlstra Cc: x86@kernel.org, Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , "H . J . Lu" , Andi Kleen , linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [RFC 7/9] x86/mm: Handle tagged memory accesses from kernel threads Date: Fri, 5 Feb 2021 18:16:28 +0300 Message-Id: <20210205151631.43511-9-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210205151631.43511-1-kirill.shutemov@linux.intel.com> References: <20210205151631.43511-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When a kernel thread performs memory access on behalf of a process (like in async I/O, io_uring, etc.) it has to respect tagging setup of the process as user addresses can include tags. Normally, LAM setup is per-thread and recorded in thread flags, but for this use case we also track LAM setup per-mm. mm->context.lam would record LAM that allows the most tag bits among the threads of the mm. The info used by switch_mm_irqs_off() to construct CR3 if the task is kernel thread. Thread flags of the kernel thread get updated according to mm->context.lam. It allows untagged_addr() to work correctly. Signed-off-by: Kirill A. Shutemov --- arch/x86/include/asm/mmu.h | 1 + arch/x86/mm/tlb.c | 28 ++++++++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/arch/x86/include/asm/mmu.h b/arch/x86/include/asm/mmu.h index 9257667d13c5..fb05d6a11538 100644 --- a/arch/x86/include/asm/mmu.h +++ b/arch/x86/include/asm/mmu.h @@ -35,6 +35,7 @@ typedef struct { #ifdef CONFIG_X86_64 /* True if mm supports a task running in 32 bit compatibility mode. */ unsigned short ia32_compat; + u8 lam; #endif struct mutex lock; diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 138d4748aa97..1f9749da12e4 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -176,6 +176,34 @@ static u8 gen_lam(struct task_struct *tsk, struct mm_struct *mm) if (!tsk) return LAM_NONE; + if (tsk->flags & PF_KTHREAD) { + /* + * For kernel thread use the most permissive LAM + * used by the mm. It's required to handle kernel thread + * memory accesses on behalf of a process. + * + * Adjust thread flags accodringly, so untagged_addr() would + * work correctly. + */ + switch (mm->context.lam) { + case LAM_NONE: + clear_thread_flag(TIF_LAM_U48); + clear_thread_flag(TIF_LAM_U57); + return LAM_NONE; + case LAM_U57: + clear_thread_flag(TIF_LAM_U48); + set_thread_flag(TIF_LAM_U57); + return LAM_U57; + case LAM_U48: + set_thread_flag(TIF_LAM_U48); + clear_thread_flag(TIF_LAM_U57); + return LAM_U48; + default: + WARN_ON_ONCE(1); + return LAM_NONE; + } + } + if (test_ti_thread_flag(ti, TIF_LAM_U57)) return LAM_U57; if (test_ti_thread_flag(ti, TIF_LAM_U48))