From patchwork Fri Jul 12 17:00:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brendan Jackman X-Patchwork-Id: 13731984 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AF09C3DA45 for ; Fri, 12 Jul 2024 17:00:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9D4E06B008C; Fri, 12 Jul 2024 13:00:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 95D9E6B0092; Fri, 12 Jul 2024 13:00:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7FE2F6B0093; Fri, 12 Jul 2024 13:00:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 6220C6B008C for ; Fri, 12 Jul 2024 13:00:58 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1B98540C14 for ; Fri, 12 Jul 2024 17:00:58 +0000 (UTC) X-FDA: 82331715396.14.A36BA1E Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf14.hostedemail.com (Postfix) with ESMTP id D951F10001C for ; Fri, 12 Jul 2024 17:00:54 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=dGtyBpi6; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of 3RGGRZggKCIculnvxlymrzzrwp.nzxwty58-xxv6lnv.z2r@flex--jackmanb.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3RGGRZggKCIculnvxlymrzzrwp.nzxwty58-xxv6lnv.z2r@flex--jackmanb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720803629; a=rsa-sha256; cv=none; b=lgt4DHH46FdGtV+E0aqcvm6lAK2bpbwZ2OW1rucfEzag8MGgbdPcXb7PSR3wh2eHY37b3U 1GBPJK6dotxfR+vS1v5Kdp4W21HSwWXSoYMwi+cfaScUUFR3QNs+OTYHZv8B7KfrloCYGr VXJprhxLxZkdU/gWjuh07ZkuK16NAyI= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=dGtyBpi6; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of 3RGGRZggKCIculnvxlymrzzrwp.nzxwty58-xxv6lnv.z2r@flex--jackmanb.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=3RGGRZggKCIculnvxlymrzzrwp.nzxwty58-xxv6lnv.z2r@flex--jackmanb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720803629; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XYkIsXZqE6yHwQwkHX4TOc/fbGXEyWGzvfr2StuEGo8=; b=jtpRk3h9Lm4n05sF82JdAEHxFEQw2A7WPp0zIhbiDnhK4uWnkPkvFBsk5fIBoMkPtdt9XI 15tvwRTGX14dgLThdjyiuZl/6UAR26dOy8X3ZLAOx6+JxlORLkEVtqWr0QO4nT3DmYfkJ4 eLyDkY9dRtYpKtJPQPDC3Bcs+xqkTiU= Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-4266d0183feso13489825e9.0 for ; Fri, 12 Jul 2024 10:00:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1720803653; x=1721408453; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=XYkIsXZqE6yHwQwkHX4TOc/fbGXEyWGzvfr2StuEGo8=; b=dGtyBpi6X5fsiGjvAv2BtAHZr6GBl8gAOu6J0KRqNw8sTWrfn2Xi+ZalUYZycnUlPf bPTj4DiDIeMlwKbJrdQ4P4rrfB45FE5jJBnHQjnS1A+tN5vqjUaSPdU/JRI9iS7Apmi1 piiroVAB8jTkX80dHqefNPKEpmtB+GeqP0/DB2E5WAc3tugTnP6Fn3uEhqtFh8ohdtK+ 0jEB03+HwUSaPK8xdJ50e/uKRKnvqetM3t8UHCHxwS7Z5ciHqQ/QLtvRkyydxmnid58g IHL+13v2dEmzWO+0Z6aNjYfdRDKuGBh31ns3P2GgFsjvovfd+cNW/7S3Dw9M4FIFGbVk j/HQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720803653; x=1721408453; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XYkIsXZqE6yHwQwkHX4TOc/fbGXEyWGzvfr2StuEGo8=; b=s6vWJcXFD9GUOZzk7l9KUKY0KD76nhyEygcXcwkZ7T4IdP/d9Kd5BuHBdU/bsiszdr lXfY2tFJ6p6Vht2P0ZXCalvLVf5GLywLyCyGNCd99vO6GxHOZiDGq9EQ65BNn6QJR+5g vKLPa0YeQePIxEmvbBzj5a9HDXvm22IUkNCPXQ5kV7IQxsRA6wZI+4wjVte0vXNndxxZ HqgtgOpfKRm0w4ZQkyqRH6SyE4r8JKWwTFfRMJj2KS0qss5ZXYixKGLsYkel+ZCwk/Ha AAS3pdQtAcVn3N1EAlfASvmUoS4g1bnWGtwmuG1Z0jaRsrx1l+y2NimEf+En+N6OzUtW bfNg== X-Forwarded-Encrypted: i=1; AJvYcCUds+3fO1mmLha7GKTEozXv5edS0TLFJ2JrjMoYVZf3rxhgKydrR/b2mcAp69SiH++cu4BKBdkZ9yqEdX2pSVHkySc= X-Gm-Message-State: AOJu0Yw1wUHJL961tn/PF8kC7qCm5VX+T+DBFXSNDxUIgNhDhpYUOzPI mLTVa0I5oFykCZA2vEUA2SHRUrItgs5hF5DWiBsiFmttTULSLpbZkZJZo5p5OcD60Vm7FTKrmSk J0ccOTSzCKA== X-Google-Smtp-Source: AGHT+IHv51LtvzZcsv+Q35b3qDQSC+/JaCm1f27HCjANRBhvdVtpbFdUnaeYYxG55HOkAMdxTZXXEQFhJkEhkg== X-Received: from beeg.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:11db]) (user=jackmanb job=sendgmr) by 2002:a05:600c:35c6:b0:424:a4ac:561b with SMTP id 5b1f17b1804b1-426708fda54mr866675e9.7.1720803652922; Fri, 12 Jul 2024 10:00:52 -0700 (PDT) Date: Fri, 12 Jul 2024 17:00:19 +0000 In-Reply-To: <20240712-asi-rfc-24-v1-0-144b319a40d8@google.com> Mime-Version: 1.0 References: <20240712-asi-rfc-24-v1-0-144b319a40d8@google.com> X-Mailer: b4 0.14-dev Message-ID: <20240712-asi-rfc-24-v1-1-144b319a40d8@google.com> Subject: [PATCH 01/26] mm: asi: Make some utility functions noinstr compatible From: Brendan Jackman To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Sean Christopherson , Paolo Bonzini , Alexandre Chartre , Liran Alon , Jan Setje-Eilers , Catalin Marinas , Will Deacon , Mark Rutland , Andrew Morton , Mel Gorman , Lorenzo Stoakes , David Hildenbrand , Vlastimil Babka , Michal Hocko , Khalid Aziz , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Valentin Schneider , Paul Turner , Reiji Watanabe , Junaid Shahid , Ofir Weisse , Yosry Ahmed , Patrick Bellasi , KP Singh , Alexandra Sandulescu , Matteo Rizzo , Jann Horn Cc: x86@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kvm@vger.kernel.org, Brendan Jackman X-Rspamd-Queue-Id: D951F10001C X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: re8whbcmb3tssdj5434qqnwsa68fgc8g X-HE-Tag: 1720803654-43709 X-HE-Meta: U2FsdGVkX1/vIGIcqevuy82gIyY5GAi6VDsaL5fn5WTHgjaj4QvVnNUAnf6XvfngNWLJ9cd9C0mryNxmUQumDBzDB8/EHSHeEM2W6YkB1ftZD6JuDHvMjKsUOCQDQ33b35DPih1q9VHRETibQj4A2y6AogM1Ir9aPwAkPu3WVv03/Q+c3Zkkaiv1BZchB1liqwoE3wo4CRi7kFMLcJLSt+SGXdt3bf0wHDwl+5zot8JQAYv/75Z+Gy8k6whGhjkJvtl1YbO+wMQq8O3/q/FZHixfcXsiI6KUH9AMk/UV5sk48ipBWnadAKV7NAYaU6Zw185L7lW6mg+LVeRcGoR8HSJSDVEjs69P3oDY2UlKwr6QHBTmAfB5+nbOM/2fLGoRwHKVc3UTUPWo6qnUbJfdkCFw2fakP7VcAI3A40B/ccV1QbSNtGe9g8L0Xe8HySYJ/KxfD/fqTNB8FSB9OTJlM6M3nGeuidqlMAXqvWlzFDfTqYpGlDalNCne5Zqs6liyRD787eBDuo7eNdp6ati+XFtSYdjq5H4qFwk7crATcVZBud8q5ee0mBPyU4q2VNbpl53T4r+xr7qvcX5ZcVoRRGbbCJdlv6rVHmCEfqrFNbp/+b7JCiSuFkNYdCYgpmF+vNQLWizXrSEGyCZs084yGdT5+DwwVLjOI8AAHRkWgd9fTwg6TKGbDEdfRdldkE5GcG/vaJ/aC3rEu717LRAyJ+vUVRsU6BXiU6D1kyb4Y1E5mE7/HbLcB08wiffeAtDGRaq1WOhEXEEwvbQT0UyZykHA/HE2BZclfFn6R0UMkWatthZ/kZZzamGkYNH5P/h+/oy7Umdc7WDL5fKk5MOWSR22MyiX49wQcP4aIvB8oD+cBI6a+PDqvogH1o7kYuLNKQZxj8yMY3Jr5pUbuUFqh3M8/b8nQRsfA1uAY5GLRedBA6bb025LKySQBksCgSTSjrTbaqXkujQ3fo3snE7 Hp0KvZZg c3STW3JbwArv9Neu8IJGjXqgDSoV68jfOZ716jJ3VC3ko0IjgaFItCmkrRc/SzPlOFsRWGdnkn4Xs8rSGY8H7Lt9r3nKQ+0+uNfqqsdUv+DzYh3rLz2tnnXKmipZCloWPsj47nY/VuP3ZUuBMwV1CKCBisZKbaWi87FwiJ4VbEs0IK+B+aBrK/47Bri4VbYc9WT5GiAKJ4lM7eF2ZkPee9y1beeYdRFowjm+5Erg/jQ+mz3bJRCJz+pBlibl+bnNo6FJjA6Z8mc9obpWzn0x2BjoOuAmrBeIB7feGiUGAN9m1gOQpvDuEAGU+ZXeKJYreGePIQUuyBEIccpwJaxy7eWIJyBdEZk085RnTHj4YJ/HuByfEivFgfuJcVbTg+yYkULQ7wO0s+kTeO2tTfRMQD/KhTle/Z54+kMAGGnwTSfenFdKYhOhdn+Nmu/nrZKNNU2lGLK/YCpvVPIxnl1cnE6DbOLUBjj2XHmmv7375nmKwek/1Mbjz44SvfRPr0gdDuWcQpq6f7YQt/4FE/2qkVJ4cxHdLAIU02iHO X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Junaid Shahid Some existing utility functions would need to be called from a noinstr context in the later patches. So mark these as either noinstr or __always_inline. Signed-off-by: Junaid Shahid Signed-off-by: Brendan Jackman --- arch/x86/include/asm/processor.h | 2 +- arch/x86/include/asm/special_insns.h | 8 ++++---- arch/x86/mm/tlb.c | 8 ++++---- include/linux/compiler_types.h | 8 ++++++++ 4 files changed, 17 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h index 78e51b0d6433d..dc45d622eae4e 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -206,7 +206,7 @@ void print_cpu_msr(struct cpuinfo_x86 *); /* * Friendlier CR3 helpers. */ -static inline unsigned long read_cr3_pa(void) +static __always_inline unsigned long read_cr3_pa(void) { return __read_cr3() & CR3_ADDR_MASK; } diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/special_insns.h index 2e9fc5c400cdc..c63433dc04d34 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -42,14 +42,14 @@ static __always_inline void native_write_cr2(unsigned long val) asm volatile("mov %0,%%cr2": : "r" (val) : "memory"); } -static inline unsigned long __native_read_cr3(void) +static __always_inline unsigned long __native_read_cr3(void) { unsigned long val; asm volatile("mov %%cr3,%0\n\t" : "=r" (val) : __FORCE_ORDER); return val; } -static inline void native_write_cr3(unsigned long val) +static __always_inline void native_write_cr3(unsigned long val) { asm volatile("mov %0,%%cr3": : "r" (val) : "memory"); } @@ -153,12 +153,12 @@ static __always_inline void write_cr2(unsigned long x) * Careful! CR3 contains more than just an address. You probably want * read_cr3_pa() instead. */ -static inline unsigned long __read_cr3(void) +static __always_inline unsigned long __read_cr3(void) { return __native_read_cr3(); } -static inline void write_cr3(unsigned long x) +static __always_inline void write_cr3(unsigned long x) { native_write_cr3(x); } diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 44ac64f3a047c..6ca18ac9058b6 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -110,7 +110,7 @@ /* * Given @asid, compute kPCID */ -static inline u16 kern_pcid(u16 asid) +static inline_or_noinstr u16 kern_pcid(u16 asid) { VM_WARN_ON_ONCE(asid > MAX_ASID_AVAILABLE); @@ -155,9 +155,9 @@ static inline u16 user_pcid(u16 asid) return ret; } -static inline unsigned long build_cr3(pgd_t *pgd, u16 asid, unsigned long lam) +static inline_or_noinstr unsigned long build_cr3(pgd_t *pgd, u16 asid, unsigned long lam) { - unsigned long cr3 = __sme_pa(pgd) | lam; + unsigned long cr3 = __sme_pa_nodebug(pgd) | lam; if (static_cpu_has(X86_FEATURE_PCID)) { VM_WARN_ON_ONCE(asid > MAX_ASID_AVAILABLE); @@ -1087,7 +1087,7 @@ void flush_tlb_kernel_range(unsigned long start, unsigned long end) * It's intended to be used for code like KVM that sneakily changes CR3 * and needs to restore it. It needs to be used very carefully. */ -unsigned long __get_current_cr3_fast(void) +inline_or_noinstr unsigned long __get_current_cr3_fast(void) { unsigned long cr3 = build_cr3(this_cpu_read(cpu_tlbstate.loaded_mm)->pgd, diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h index 8f8236317d5b1..955497335832c 100644 --- a/include/linux/compiler_types.h +++ b/include/linux/compiler_types.h @@ -320,6 +320,14 @@ struct ftrace_likely_data { */ #define __cpuidle __noinstr_section(".cpuidle.text") +/* + * Can be used for functions which themselves are not strictly noinstr, but + * may be called from noinstr code. + */ +#define inline_or_noinstr \ + inline notrace __attribute((__section__(".noinstr.text"))) \ + __no_kcsan __no_sanitize_address __no_sanitize_coverage + #endif /* __KERNEL__ */ #endif /* __ASSEMBLY__ */