From patchwork Thu Mar 7 13:39:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13585599 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31007C54E4A for ; Thu, 7 Mar 2024 13:39:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A90B46B017F; Thu, 7 Mar 2024 08:39:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A40D96B0181; Thu, 7 Mar 2024 08:39:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8E15A6B0182; Thu, 7 Mar 2024 08:39:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 772446B017F for ; Thu, 7 Mar 2024 08:39:23 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1A4811C1394 for ; Thu, 7 Mar 2024 13:39:23 +0000 (UTC) X-FDA: 81870349806.01.DF1F922 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf09.hostedemail.com (Postfix) with ESMTP id 7C44B14000E for ; Thu, 7 Mar 2024 13:39:21 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=AghDYDNt; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of 3iMPpZQoKCOslbfelNUZRQTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--yosryahmed.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3iMPpZQoKCOslbfelNUZRQTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--yosryahmed.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709818761; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XhRtgDTyBtYE/TCEGVA0Jcde7nXTTPmxFYt2TI6fczU=; b=0hW5aLEL2ws1bkTVbvdcGWCZt6gEjnIVnmYSUaSqbrqOvI1KV9MvMJPGqMj3CEvQ6H6Dq3 nF63brbcN3/c8Mr4avT8SQqfS7i58pXzXZA2oO1Kd30sOJbzjVvSEBMYMbM7R+Mw0/eL2L dw37pVIJ++cdGvFEqhqekGfznZVDrGk= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=AghDYDNt; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of 3iMPpZQoKCOslbfelNUZRQTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--yosryahmed.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3iMPpZQoKCOslbfelNUZRQTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--yosryahmed.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709818761; a=rsa-sha256; cv=none; b=ENb0t0HydmTJJP3QFaMZdV7M0kQGZjcNWnuEL030FDnvHXpg1or1kW8nRXVlVU1OIZ5isZ 5xwVS2djdYmxPuSgFlW1xOew1iyz8KjE00qV9jFjELmP+vF/ebg0kVfY7y5cDjS5Q9vw9m HsnAGFCIBX2HI3T/m1/KF2knrhXDZFc= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dc6b26845cdso1290915276.3 for ; Thu, 07 Mar 2024 05:39:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709818760; x=1710423560; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=XhRtgDTyBtYE/TCEGVA0Jcde7nXTTPmxFYt2TI6fczU=; b=AghDYDNtXYjnuJ5HJgcNINERosIm83dD+SjqG7/BWRHCsLsTHfsEB0C+osPfHeeHJ7 r4LfSWs7xgdA1eJ/j9DmUlak7+HeyfQWDZYicUdqO5nJ/DBdP+3ePFlA60+P42yA3Pt9 Z1QgMhe/Dp9jA+L9CaB68P9g4S9rmYZ8wKVQ1RfUwtxdt/wVFoyko4Sw3s5cVYiqn3fZ w85j6APTdSiHanoSpJiniggsVSEJuBk8TYDy6YHzDm43hdGgc1FdVxSGhJCanaIp5p7n 2FW7Z96yYUpMLTR1tdM2Qzj28TVu8a5DCU/LAJhYmHzNrugR1G+KaPheBQWsDIhSrsxv zmXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709818760; x=1710423560; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XhRtgDTyBtYE/TCEGVA0Jcde7nXTTPmxFYt2TI6fczU=; b=U1xKWjYK9U+baDD/15CIQVRenQRFRymnZkElVGsp3dxYQe29FEQWauLCEMYW7w0uli b7ZJn/dU5uiG/tFvyVjs5AfyszPvqGdXLGK7I2bFcKeLuk+eTZ4KkkxvwlrZhyvVkQwC 0rbqvCguV+1hOTDAIjOmckRwEcZjv4DwF2V47i24eKMpFeUCsKu3iRwsySM0GnYVDg6y i9Y3fkndGrcpWRxOWFqylkGsA5t22+Q6opvoJrvkJkHi7rr26rP8JaOKJxH3EAky9sq5 57nezTJG8atMWoe/+Pjptfs1xhzvtJv+np/FmpI7EclA8nY3p7n1GL6CVYuv/DFJERg6 01jg== X-Forwarded-Encrypted: i=1; AJvYcCXucvxiTCB7RYQ0mVz9vIT6iK5o+7d2L8i5LfNl/Lv3g2MqvM8a/PqDo/Bu9Z4fp+2BK2r9OB8WFY4ihgexGzty3ec= X-Gm-Message-State: AOJu0YzbkgB+S39o7IXXPEzcxHlvVzMtFE0Ty+LNxNkV4eeDBujIDaTp lVxWCOc1tth5js6xtq6qwXiy91V6bmhfI3J3DlenLplUWD+QCZgqPp471O3kVhGK/Wry+mOJy4h GqjMYyVXAgVy0bKJysQ== X-Google-Smtp-Source: AGHT+IFRqFk4oamlQows6KoGWBh8WrwoJ8kNbWyjPiHy+1cnclmprEIfJ2rYatkNe3gWdYcqZi/b1yYDg6XF3ZIA X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a25:ab0e:0:b0:dc6:519b:5425 with SMTP id u14-20020a25ab0e000000b00dc6519b5425mr4354682ybi.11.1709818760421; Thu, 07 Mar 2024 05:39:20 -0800 (PST) Date: Thu, 7 Mar 2024 13:39:14 +0000 In-Reply-To: <20240307133916.3782068-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20240307133916.3782068-1-yosryahmed@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240307133916.3782068-2-yosryahmed@google.com> Subject: [RFC PATCH 1/3] x86/mm: fix LAM cr3 mask inconsistency during context switch From: Yosry Ahmed To: Andrew Morton Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Andy Lutomirski , "Kirill A. Shutemov" , x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed X-Rspamd-Queue-Id: 7C44B14000E X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: zxb14tt8ndphirekx78dnqogd3dmbsmg X-HE-Tag: 1709818761-542933 X-HE-Meta: U2FsdGVkX18luMrbFXVSe8kRUFoNqdZqBOLVBJZTI5ldDqVK124Fr2FLSkTvxbvYdVPiEQ7wa7QT61e24j44Do8gWwugrvTYyJ1tzIByS1tQ7uUMhKURNlqmOxSJH4JMhYN7BvAM3f0OCUSpR+0HHxQknKBnl4jLFfBgbN2xYpvIM9Lu19NMjOl7F4MLu2VhZJPZHsK4KP91TdvueNP3Y6br39cWECXdhLRT8+ZYiOGdvFfWjnbJZnVmqIGMWbYC7f25YCDMmfhAD9fARvZXFJWTP3vGS+B+Yx9Fw5oXSIIh34snOxmswFckqbE/R5e96BP+R84XWFiEcGPsXxnbMPUkbowXffbaT9DFxxEk5nQgstJAvYxPd9Gm0ZSZiLJfnP3tl4oI1ndF7Z299ZmeWrV+I/D3tQL8LsE03jAEwBjShTLRp4XrL+4L7HYTSZzlnTKZA2yg3knzi7g/2IxIaOcBf/Sgc/LWxw/k9B9pil1kuoURYSSJY5MoTl2DkOy2/HMe2NNvTFhBAprYXjiZ3Uyj41dG4rN4Us0+ZyP8aID5SJfun6arauDJNej59oXCmS/CBah5/GNAhpfaZaar6vroYme71rkg9y66ajmYJp+0FKeqzjlLrs9jo1UlFqEDEXAIrLw9gzCbGBeLyioEje1h4C3TKM/nAYpKNSSuc9iuGeAMqgg5EbSS85Gge8Z9RPe9MvFArzeHwj2x7DAO+6XAGx3bPojYRJxxdvHIrfNATxPFa4Gry0rsKPQOwhUFrM9tajOSBKWJJ1yUyHQo3aWmkq6/ZqKyWIRtIwMGrFVvLm2DwHYyL/Xw54gnfiHsXlWNp+K39Nc4UPlj1eIo1dtBhkV07nRGINlS5gfIL3tGt+ZHwYL89rY6xFdzgiSDNP3BbKK+wzeuAFp3rh6zzxCA4Bn8H1Be46BpavduUwjh+m2zC+3YD6bvj5T9uetdryDDMg3T+Q2N+BC08sQ F856tTOu 67jLnaHM608syDXfsxa7TsfoPovgI2EoF0PwpEYGpTO8MsahnpftnpB+aVGClZL+Dqb35CtNNVnaddZTXMzKpGM1QgSQgWq0lLg3DeBDWzIZ0svVsGt9z9LLc20zAODl2lPp9lQwDhNv2SChCHNJWMXETiPTCM14nPYPUZPUbsLjR00/gIarH8Osfe4iZnEMGHVJBvUEmrcMZCYO+e49XMqhHNgDcLXSEMSXi2vU/nW2t5fUEPzIGLeWa+TurL900LgpLVO7UfwxbCAvMijBdBr6CRgYTtpTUufZB2YYhMLcXWMuYANBitVYE05slVTSoTK3v5EorImwlRqAC7p7lwS/6j8GxTWyzAoQQKn1sMm6rMbmh2B1oG+PO/E/Y4vjhG5yt X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In switch_mm_irqs_off(), we read the 'mm->context.lam_cr3_mask' into 'new_lam', which is later passed to load_new_mm_cr3(). However, there is a call to set_tlbstate_lam_mode() in between which will read 'mm->context.lam_cr3_mask' again and set 'cpu_tlbstate.lam' accordingly. If we race with another thread updating 'mm->context.lam_cr3_mask', the value in 'cpu_tlbstate.lam' could end up being different from CR3. Fix the problem by updating set_tlbstate_lam_mode() to return the LAM mask that was set to 'cpu_tlbstate.lam', and use that mask in switch_mm_irqs_off() when writing CR3. Use READ_ONCE to make sure we read the mask once and use it consistenly. No practical problems have been observed from this, but it's a recipe for future problems (e.g. debug warnings in switch_mm_irqs_off() or __get_current_cr3_fast() could fire). It is unlikely that this can cause any real issues since only a single-threaded process can update its own LAM mask, so the race here could happen when context switching between kthreads using a borrowed MM. In this case, it's unlikely that LAM is important. If it is, then we would need to ensure all CPUs using the mm are updated before returning to userspace when LAM is enabled -- but we don't do that. While we are at it, remove the misguiding comment that states that 'new_lam' may not match tlbstate_lam_cr3_mask() if a race occurs. This can happen without a race, a different thread could have just enabled LAM since the last context switch on the current CPU. Replace it with a hopefully clearer comment above set_tlbstate_lam_mode(). Signed-off-by: Yosry Ahmed --- arch/x86/include/asm/tlbflush.h | 11 +++++++---- arch/x86/mm/tlb.c | 17 ++++++++--------- 2 files changed, 15 insertions(+), 13 deletions(-) diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index 25726893c6f4d..a4ddb20f84fe7 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -399,11 +399,13 @@ static inline u64 tlbstate_lam_cr3_mask(void) return lam << X86_CR3_LAM_U57_BIT; } -static inline void set_tlbstate_lam_mode(struct mm_struct *mm) +static inline unsigned long set_tlbstate_lam_mode(struct mm_struct *mm) { - this_cpu_write(cpu_tlbstate.lam, - mm->context.lam_cr3_mask >> X86_CR3_LAM_U57_BIT); + unsigned long lam = READ_ONCE(mm->context.lam_cr3_mask); + + this_cpu_write(cpu_tlbstate.lam, lam >> X86_CR3_LAM_U57_BIT); this_cpu_write(tlbstate_untag_mask, mm->context.untag_mask); + return lam; } #else @@ -413,8 +415,9 @@ static inline u64 tlbstate_lam_cr3_mask(void) return 0; } -static inline void set_tlbstate_lam_mode(struct mm_struct *mm) +static inline unsigned long set_tlbstate_lam_mode(struct mm_struct *mm) { + return 0; } #endif #endif /* !MODULE */ diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 51f9f56941058..2975d3f89a5de 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -503,9 +503,9 @@ void switch_mm_irqs_off(struct mm_struct *unused, struct mm_struct *next, { struct mm_struct *prev = this_cpu_read(cpu_tlbstate.loaded_mm); u16 prev_asid = this_cpu_read(cpu_tlbstate.loaded_mm_asid); - unsigned long new_lam = mm_lam_cr3_mask(next); bool was_lazy = this_cpu_read(cpu_tlbstate_shared.is_lazy); unsigned cpu = smp_processor_id(); + unsigned long new_lam; u64 next_tlb_gen; bool need_flush; u16 new_asid; @@ -561,11 +561,6 @@ void switch_mm_irqs_off(struct mm_struct *unused, struct mm_struct *next, VM_WARN_ON(this_cpu_read(cpu_tlbstate.ctxs[prev_asid].ctx_id) != next->context.ctx_id); - /* - * If this races with another thread that enables lam, 'new_lam' - * might not match tlbstate_lam_cr3_mask(). - */ - /* * Even in lazy TLB mode, the CPU should stay set in the * mm_cpumask. The TLB shootdown code can figure out from @@ -633,7 +628,12 @@ void switch_mm_irqs_off(struct mm_struct *unused, struct mm_struct *next, barrier(); } - set_tlbstate_lam_mode(next); + /* + * Even if we are not actually switching mm's, another thread could have + * updated mm->context.lam_cr3_mask. Make sure tlbstate_lam_cr3_mask() + * and the loaded CR3 use the up-to-date mask. + */ + new_lam = set_tlbstate_lam_mode(next); if (need_flush) { this_cpu_write(cpu_tlbstate.ctxs[new_asid].ctx_id, next->context.ctx_id); this_cpu_write(cpu_tlbstate.ctxs[new_asid].tlb_gen, next_tlb_gen); @@ -705,7 +705,6 @@ void initialize_tlbstate_and_flush(void) /* LAM expected to be disabled */ WARN_ON(cr3 & (X86_CR3_LAM_U48 | X86_CR3_LAM_U57)); - WARN_ON(mm_lam_cr3_mask(mm)); /* * Assert that CR4.PCIDE is set if needed. (CR4.PCIDE initialization @@ -724,7 +723,7 @@ void initialize_tlbstate_and_flush(void) this_cpu_write(cpu_tlbstate.next_asid, 1); this_cpu_write(cpu_tlbstate.ctxs[0].ctx_id, mm->context.ctx_id); this_cpu_write(cpu_tlbstate.ctxs[0].tlb_gen, tlb_gen); - set_tlbstate_lam_mode(mm); + WARN_ON(set_tlbstate_lam_mode(mm)); for (i = 1; i < TLB_NR_DYN_ASIDS; i++) this_cpu_write(cpu_tlbstate.ctxs[i].ctx_id, 0); From patchwork Thu Mar 7 13:39:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13585600 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A8D2C54E49 for ; Thu, 7 Mar 2024 13:39:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C8C046B0181; Thu, 7 Mar 2024 08:39:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C3DB16B0183; Thu, 7 Mar 2024 08:39:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F3616B0184; Thu, 7 Mar 2024 08:39:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 89C4A6B0181 for ; Thu, 7 Mar 2024 08:39:25 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 544BDA1816 for ; Thu, 7 Mar 2024 13:39:25 +0000 (UTC) X-FDA: 81870349890.15.55CE6AA Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf04.hostedemail.com (Postfix) with ESMTP id 4F48A40017 for ; Thu, 7 Mar 2024 13:39:23 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=kXVyWS8H; spf=pass (imf04.hostedemail.com: domain of 3isPpZQoKCO0ndhgnPWbTSVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--yosryahmed.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3isPpZQoKCO0ndhgnPWbTSVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709818763; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MPmHpgEfpH39wv9P3N7+5I+Np9ZrrLnfHGs5cKPvN1A=; b=ALs/ruDTFPlpNMPhSaYCozzasCGdW9xpO+476DC2faxHqwqtXODTpKpyq0dDlGkjeKYQGG Outpk8gYcV9blmqfJnZJzNVoH/4I66/WP80Ivk1L6wFY7BvOPw7EPrGF8YMzoZPoGMtYee 6sRhePNtICfAPlV6b+8djWN5hvObmS8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709818763; a=rsa-sha256; cv=none; b=RwZ74Sbg4TCYjAHF8OQGX/IUCoKFxHC3rjI4JBrJfLKatYyGV2jhT+je/q6z/9ZlFLhXY9 sap5mDXxNOSLUPTF+j6GmLJplEiuvBB8hpNQIpZ8+jAAVEsjZH+P6QJOgJIq59bOH4Vn2r +HqFafR8/12XWBnrixk30+XyfSenUMs= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=kXVyWS8H; spf=pass (imf04.hostedemail.com: domain of 3isPpZQoKCO0ndhgnPWbTSVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--yosryahmed.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3isPpZQoKCO0ndhgnPWbTSVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dc74ac7d015so1160516276.0 for ; Thu, 07 Mar 2024 05:39:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709818762; x=1710423562; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=MPmHpgEfpH39wv9P3N7+5I+Np9ZrrLnfHGs5cKPvN1A=; b=kXVyWS8HUj1PF4hL+o0KEuQyn4iyyeL0SpU7Pxjf8Rzj90lIrIksHYHhuZa5pWyVdD cdFevNzJH2/vzpBoLuyjYqStzL3BIi/p+yeaSzl3r0i8Dhy1SNueSiE4IFpd9FKw+Ilg JfHelsINLVyRVAkTieiu2CcPlPTrKUSZFYhwL8DNCRaqpWsESkrft5dDTyPr3qfV4q9l 6Lf/FDOKe9uaP4aCeEwz70uPcbiTqXae+zLqx/sfo3SBog9d+wpwk9Mvy4QYcoSEBEuu A2T5lBDEkCzibWpGAlr8SDtoQX8Zxyi//DvDpTlxGZ004BUHRs9ufkSbn/1E3CblA5wi dBAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709818762; x=1710423562; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=MPmHpgEfpH39wv9P3N7+5I+Np9ZrrLnfHGs5cKPvN1A=; b=FIgHFm55k0g9bvMGBMXQRTcs6Pzo9A/1PHhv41dVoLSXrPEQfR2NhGA/RKsqldDxwa Fkv4azEa7coTuHM2z2jhgo+l+fSpUA97IU8YNyoonNT5EFYYuNuxm8lQ1/uhbCovlyBO Q2LlKHuefKuEuUW6Kkr0jQeRQBonnRNXt6Hn9vPJXldx3nywvjvuYNVxyc04O9Iq32gG xW7hs4U7pk0a9i4BzZI98xDNrt7zAcUx/n9SphZi4+mQDxaeITlXRGDkFotFXJGoM3ka 8YdU8ursOp+QrLPljcooxvme7Rdn7yh3KKn3+otRV45LgA3UE6Dj+OtXRwv4Cg2jIUeL rm2w== X-Forwarded-Encrypted: i=1; AJvYcCVJ8vEVoJqZGJIrCya3Oxf71GjYOeeVeIwbe6QdH9YFzFEmwWETApG/hXK0/GzdQMo4tgkvKTI40PjWWFYed9bSWyI= X-Gm-Message-State: AOJu0YxWtQvaU5h2lh96+ZfSSDY/v8XHmkj/r/nAcEpCY5UnOKMMUJAP xlqdnxR7r4ulG3dSQlJcxHGkiIXjVMIoilAt3OhKfCZ4ZLxu9oMqWsxj8R27DoatROcZp/lhxcs O0sIy4p9lRW9IqCXoSQ== X-Google-Smtp-Source: AGHT+IERNwqp0BbMH+HdatOkbB3X7SeaUZ0qNtYllODJdaNR0SQYYHmwI9BfJm6ZZYu3OtMlHLptB1kSZCzrhOZk X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a05:6902:2183:b0:dcb:fb69:eadc with SMTP id dl3-20020a056902218300b00dcbfb69eadcmr781549ybb.6.1709818762290; Thu, 07 Mar 2024 05:39:22 -0800 (PST) Date: Thu, 7 Mar 2024 13:39:15 +0000 In-Reply-To: <20240307133916.3782068-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20240307133916.3782068-1-yosryahmed@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240307133916.3782068-3-yosryahmed@google.com> Subject: [RFC PATCH 2/3] x86/mm: make sure LAM is up-to-date during context switching From: Yosry Ahmed To: Andrew Morton Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Andy Lutomirski , "Kirill A. Shutemov" , x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed X-Stat-Signature: ox7t866pxe9wh6d4rq6hn65c7311m8u4 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 4F48A40017 X-Rspam-User: X-HE-Tag: 1709818763-533278 X-HE-Meta: U2FsdGVkX186QPnL1tbs3PlgFRGswh+sPMZ7cijjbM1bb7IWe+X5DTxAleu7ttLbSv30fIrqhF263isbE+9D7WRjIMha2eaTJK7o2fWuZ3SYOPRtkYLhZwDpehw5lqze5nj6LXi/Flz1vTucw9b0NDAZfJpJWK2DQYO3cvY5LJNDIxZNuTPGhC7Ip6oOTYbFdEuM5nl7GxARyh1uIGRYBnU2qisEw/HGMyXfs0pYnaX+CyA/yhskIegYsQi5LLc8cFMDC1l985KYcfifOl/GD5X+DEqLLc9i9G0yrHI+4FznSh/gWUgvOxcuK1i+uXREoGLwcO/vtZT05MPYJIGS0HIO/c+ASYcsLpifA1KlzT6j4ZHHIMxgiW9YoXfgEBah8fAbN1AZ99znX5QRAE9r2sdQSYk5ivdqQ+oL7VpSj3Q8T/yQgbV7I5CURYyY/Nv15YHmaH0wUMhrsPlWrPZDn64fFIgHGseJy3oLG4+OmyZQi/WcsMk/r2gDFnMtHmtOnmbddW8sFCCgiZH1NGVn/14YVpIu2ugL85ZxYJdj1JM+CqFr/or15vmJqw5TnX1q3QTeJ66MNNpX5wrlr9IHxcZCRoieDLYrcNHv5OXau38he2IYFx1FMxtsi3cd0CG7s3U3ghH05mG5LmcHhL7nPdgzgGx4iEojA5/+uip6PIh00S1b1iqiv8j0Vjzf+VTN9iO4Ue8J5DZlGedqmYGHdfMHFuLN4kHDiv6k1asw+HdWMITQsvL3GALM3Sy6+c01319r6vH19IVijWiwQeCNHvzJXKe66+83P0rgYqPUUZxLmOt9zLy5DAGWT8Qt43E71T0iuHq4YfZyK6AToEqbiN6CMABmLCiSofp2Bk4+mnoo//CxQvKJsUrYSdsj8Hn8snNJCIu3Vui2D31gCPtUn9FqXYSd6rVjsWJa8n7vhkyNzeGNFRm2dy9WVd7Ieuw1dfi9zEkrh4XF1juyBX/ 9pW8CqcW 9IYpxF8ieJIaNJt7odXovO6TN276qj2KrUuDUF9tnm5rhroA2HCpEfowbLZKueYBqsdVCJncXS1S7fFNn4od7KJMTbQZW7sK6TiGxrFDBkH+2q6Yhlwhv41pmYsIdAJZXrWx5+o45fyENIZy/Hofh7GQk2baZOPbOz5x6uCQ7hulnzB6LNYbFSEc5giRA80fkSXsd+EQ3rgwl+Jn29DdMCPtnpBcykWw+j9L7qVkZ8hz9QjFQXXHoTM0VZxBXK2fFXay0N8dsD48sm87vneBmDC8oBaJyVKiiNHFEgA+oIrD6rl8EybGg96Es0zgezC03ehpn1hTvfFswMpJbdE/w7dtKCI25l3Qlaw8k3apa9+RaZ30KmXurV1LXmeH1HCk0mU57Zijkd15CJoxhXcq/t1g8Dwm/W6Rk+vmZ4BdEsnYvJQ1YTbnROIdgMyOMAXfQADH31VX4CANgN7WnjusWhnC+J66EQ/IjsVzN1TlCalxWHK1IGWXg83/PA11fFfjjkapva/ZePqgLFXv5L7tNeSYcdNNcTfkZnFhomvLFLsk6GbVpHZx3+VgbJ4NGRfMkEYeEeQ4XZTvbnz/ud56iAn1QzmoiCVIusbOUXiV1y5OIZgBE1jxaTkCg3WDZVTPFPfEc6LOHEe7/+hk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: During context switching, if we are not switching to new mm and no TLB flush is needed, we do not write CR3. However, it is possible that a user thread enables LAM while a kthread is running on a different CPU with the old LAM CR3 mask. If the kthread context switches into any thread of that user process, it may not write CR3 with the new LAM mask, which would cause the user thread to run with a misconfigured CR3 that disables LAM on the CPU. Fix this by making sure we write a new CR3 if LAM is not up-to-date. No problems were observed in practice, this was found by code inspection. Not that it is possible that mm->context.lam_cr3_mask changes throughout switch_mm_irqs_off(). But since LAM can only be enabled by a single-threaded process on its own behalf, in that case we cannot be switching to a user thread in that same process, we can only be switching to another kthread using the borrowed mm or a different user process, which should be fine. Fixes: 82721d8b25d7 ("x86/mm: Handle LAM on context switch") Signed-off-by: Yosry Ahmed --- arch/x86/mm/tlb.c | 50 ++++++++++++++++++++++++++++------------------- 1 file changed, 30 insertions(+), 20 deletions(-) diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 2975d3f89a5de..3610c23499085 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -503,11 +503,12 @@ void switch_mm_irqs_off(struct mm_struct *unused, struct mm_struct *next, { struct mm_struct *prev = this_cpu_read(cpu_tlbstate.loaded_mm); u16 prev_asid = this_cpu_read(cpu_tlbstate.loaded_mm_asid); + u64 cpu_tlb_gen = this_cpu_read(cpu_tlbstate.ctxs[prev_asid].tlb_gen); bool was_lazy = this_cpu_read(cpu_tlbstate_shared.is_lazy); + bool need_flush = false, need_lam_update = false; unsigned cpu = smp_processor_id(); unsigned long new_lam; u64 next_tlb_gen; - bool need_flush; u16 new_asid; /* We don't want flush_tlb_func() to run concurrently with us. */ @@ -570,32 +571,41 @@ void switch_mm_irqs_off(struct mm_struct *unused, struct mm_struct *next, !cpumask_test_cpu(cpu, mm_cpumask(next)))) cpumask_set_cpu(cpu, mm_cpumask(next)); + /* + * tlbstate_lam_cr3_mask() may be outdated if a different thread + * has enabled LAM while we were borrowing its mm on this CPU. + * Make sure we update CR3 in case we are switching to another + * thread in that process. + */ + if (tlbstate_lam_cr3_mask() != mm_lam_cr3_mask(next)) + need_lam_update = true; + /* * If the CPU is not in lazy TLB mode, we are just switching * from one thread in a process to another thread in the same * process. No TLB flush required. */ - if (!was_lazy) - return; + if (was_lazy) { + /* + * Read the tlb_gen to check whether a flush is needed. + * If the TLB is up to date, just use it. The barrier + * synchronizes with the tlb_gen increment in the TLB + * shootdown code. + */ + smp_mb(); + next_tlb_gen = atomic64_read(&next->context.tlb_gen); + if (cpu_tlb_gen < next_tlb_gen) { + /* + * TLB contents went out of date while we were + * in lazy mode. + */ + new_asid = prev_asid; + need_flush = true; + } + } - /* - * Read the tlb_gen to check whether a flush is needed. - * If the TLB is up to date, just use it. - * The barrier synchronizes with the tlb_gen increment in - * the TLB shootdown code. - */ - smp_mb(); - next_tlb_gen = atomic64_read(&next->context.tlb_gen); - if (this_cpu_read(cpu_tlbstate.ctxs[prev_asid].tlb_gen) == - next_tlb_gen) + if (!need_flush && !need_lam_update) return; - - /* - * TLB contents went out of date while we were in lazy - * mode. Fall through to the TLB switching code below. - */ - new_asid = prev_asid; - need_flush = true; } else { /* * Apply process to process speculation vulnerability From patchwork Thu Mar 7 13:39:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13585601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E76EC54798 for ; Thu, 7 Mar 2024 13:39:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5FCFF6B0183; Thu, 7 Mar 2024 08:39:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 49F116B0185; Thu, 7 Mar 2024 08:39:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20A996B0186; Thu, 7 Mar 2024 08:39:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id F3C3D6B0183 for ; Thu, 7 Mar 2024 08:39:26 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D3A9C1C1342 for ; Thu, 7 Mar 2024 13:39:26 +0000 (UTC) X-FDA: 81870349932.30.DA40824 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf22.hostedemail.com (Postfix) with ESMTP id 1783DC0012 for ; Thu, 7 Mar 2024 13:39:24 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Kp2YoqoH; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 3jMPpZQoKCO8pfjipRYdVUXffXcV.TfdcZelo-ddbmRTb.fiX@flex--yosryahmed.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3jMPpZQoKCO8pfjipRYdVUXffXcV.TfdcZelo-ddbmRTb.fiX@flex--yosryahmed.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709818765; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Y+p7lwWPK4A+HxnQYYFVMxpxfoiERHBbI1OMhXMl4tc=; b=0l9AZDyuJ6/usXjjW+0Gedx+Y7gvFMCQjVHycuXF43k/seuU8tBXZ0BjQWe1c80m60nBNq kmwehHToKeRbW94EjpJvrluaL843LDAuswpq1We0h1UJ8okvdiIgZ5pqjW/3VQUWOsilCu OElkwyryDwI9KxFHuJw3iO2vVYG0dF4= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Kp2YoqoH; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 3jMPpZQoKCO8pfjipRYdVUXffXcV.TfdcZelo-ddbmRTb.fiX@flex--yosryahmed.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3jMPpZQoKCO8pfjipRYdVUXffXcV.TfdcZelo-ddbmRTb.fiX@flex--yosryahmed.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709818765; a=rsa-sha256; cv=none; b=E8MvenQcMMRCNqOz5lB05m5P7XGFcOOwnmOkvwNiyNsZXZcg9+aPd6WWHpuCesWwuivu1Y 00Dk6diOdcdru+M8NBhQxtE68oQqkW27f0Rsjs972weKjHZqD83f/lZblHEYb6Lw8fWXgF bH7oYzA5qj0Hc9j4kEg8dtiUTTU/4mI= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dc693399655so1648059276.1 for ; Thu, 07 Mar 2024 05:39:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709818764; x=1710423564; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Y+p7lwWPK4A+HxnQYYFVMxpxfoiERHBbI1OMhXMl4tc=; b=Kp2YoqoH6W94NuhDCHGZOl8a5bI4sc18VEODCPOO+jhk0OA319YRPLL34W0xHs+cJz mFR0f2MMlWEMgsjaYg/V8M4GPZTy1yyaexBkxxBlkkG7xFUux+l9RcrfgEHWu90ze7Jy 7oRld7ePeA4hYpcxIWIW3/IPPs3pGKSWVX4LJiIAOWtWrRuYWRBKuYUJLY1i0UlRvjb0 WY4s2XPaVppmFvRCelWG8l03kHauLN4XxUnHFM06r+QyeVijdZnkLupFdcSZ18ey+hkM 3RXcMYn92GJqBJ/6+YoDBhrXqDS3jh20AmOLvto9IbR9NQx32eM6K53flq+heErzqK8O 1UUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709818764; x=1710423564; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Y+p7lwWPK4A+HxnQYYFVMxpxfoiERHBbI1OMhXMl4tc=; b=wPog8i0mPZnjekjiDeExSg6tLL0Su7bud6dhr3r3a1T9WGYgO7AQYd99ibuypr8Hau rztnsmNtvHDox3uQ/H3Gx9KOLZsT7auZDBiBS9qKOuv2QZcAcVcABhU37KcJs3YLHCdN pi0iKAaJnbbZPgP3+8GR/mu7UJFZ0LAXSL2PjVsPUa9wnj1hbr36Xu6ZD+0rBVurgT3E 883iD7X96F0+VnLfUqoV2XKr3h1cFhwxidJyvrikqWIJ8GN1LGm7H9K6cjulOHHPliQp Bmn+EPceFjXRh8SH3F0KGvS3e6Yr6D4oi8zFSY6CMaWY80XTUfsM5VKtTtqLfwVauzS1 aO0g== X-Forwarded-Encrypted: i=1; AJvYcCVbVQEMoGE2Q0Gth+Z79izM2cZYI74WJBM2ilBQ600M41vudSimTWVXbuIbrjFZWmVQN1qS9YVoGC7DyyCLBQherdY= X-Gm-Message-State: AOJu0Yz4F/dHeDNntQJwqAJIn6EIoiAvk7wB/LMt/TnP4KffYE+dnK3S fzaOqf8rVHbLrpDoCuSnL1PZ/pWrslD1f6f6veA3vOndz6QWnOiffoqP/mQf9L9gIq3M6hCULpa zAHFaFYdfWYMi6ei2wg== X-Google-Smtp-Source: AGHT+IERxGCOOgtmKs7Ul3o8yNM4OeKiruullY93iXEMk4G4tg4OgAxxdrUv6OMg9noyhQpJciR8PQuPntNjh18q X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a05:6902:1243:b0:dcf:b5b8:f825 with SMTP id t3-20020a056902124300b00dcfb5b8f825mr4599717ybu.0.1709818764114; Thu, 07 Mar 2024 05:39:24 -0800 (PST) Date: Thu, 7 Mar 2024 13:39:16 +0000 In-Reply-To: <20240307133916.3782068-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20240307133916.3782068-1-yosryahmed@google.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog Message-ID: <20240307133916.3782068-4-yosryahmed@google.com> Subject: [RFC PATCH 3/3] x86/mm: cleanup prctl_enable_tagged_addr() nr_bits error checking From: Yosry Ahmed To: Andrew Morton Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Andy Lutomirski , "Kirill A. Shutemov" , x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed X-Rspamd-Queue-Id: 1783DC0012 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 6nhpodr3p34wui8797gu6yqskry73zwf X-HE-Tag: 1709818764-832947 X-HE-Meta: U2FsdGVkX1+gu47jX6OH+zJ9HiaYE8Y7KuMfICj4iaN7yCJEhoIae5iTxEiI1ErPlPHRxWe5PGHordc8jTzisjIxBgfEaYd456BV81ywLOphtO3azRjEo7yjYUYeXQwTrcr8rCe7CfVAVBXs1RSCHeEYFYnjlUWAN53sH6sQiUXkbA24+d2tozmmlByzMLcKc2ao8AW59LByWiU5NRDntt7e4Jk3Pf916aWv3TgQF9JQmBZkaKeGF5FRbFI2xxRQ8enfPznwGq60RpmDJBY9dFjcqDslQvsfXXPQNVYo59lLOtJojZ6K+9uDqtvPT1Cb+AEM1OJ70vBdLcc8h5UDR5e7Oi2IE2T79Fl2YILwfvcR4I8tIY7MMcc3ncwK7RWQd4rLLnWn1fZZBdGJy6Y8oJ8NHNLkHcysWtA7mm4QlIpZnf6ESWVs3X0Jl0QV1OT3ox/M4zVFyo5ZswgO0S/7QK4Jsr0G1nhaLG0UfxtnS4pppJlPuDX3vMM8e2O7kcQtlFPaGR88ajvJhv3ERKZzi7/LSkXvw5xPCKr6HUUDvQ7rkuo7uoo8NyaO8wgqgRxm2GH6HFFRu2q0WQsTvsSB4SAIRgcHl57bd3gj//7ap92umxMRkQH028mZuN6yzBvVbF3pp28W7FAcQ1T/UsLlTy1ZqQXG7mLSaMoMomREjVnKDHe/RWZGMxklBTYmyauOvOKTeggul5/YIqj1HFZUU0+4XR9KETBPEeLf8f0NwJY+1gldRMfSfn279GtBlSIe0WquE4t+59FZ5huutrSuTDF3sj90ZLVA9eQDdi5OOUG9CIAfM98sHQGlXbufWe+RBKWtyvstj/sk9aB5Dce9KonHB/w/BdJ1btVD9VTa7jlJBeVaaou0ZGrro72bYZKqn8pY1bPC2SOlJlcPEwrtz9povtdl+tUSJX23vxWaMuEN6LeDJQTB4C4LJtAel351PwwwpIe7/ijW2nSCEsM raHH8Cz+ Px6AAu8tJC0BnhO+R0JhNE9OQ0Xc8MA8Y1lAuoAvscucVV2CfVW67K9UGMCaN1aVbIJ8JTFWBakNlCFr87bvoQQHpz4ih1fgqrwkaEzntOXsl+3gLNPJna6H2cDGAcz7MsyRuGPnDy6tENqF77WpZFUtGcE7XvQ9gC1k6ODxTi0928jtEbMfPXfI5qbSvWAiwr2Q3G4eEJGMhjkNgUtGOgBUND7PE0sRDtLszqmPXYxzF06qQ6vwBf1p/XTyWwBP/FF/piP1d0zRW0c5+/1PwLDwV9Be/OK2uKy00gS4UnFJ5RA/Hk/U+2JiPwvMmabUX3K+La0VYSfsb8ZeWNHmPZ8wwkOubRRtpZ3/PspnYiwZJXZ+OuEfb2xIvCg/zc7v8r+wu0VBgProb4o+z4viRoSh4zRwGgrzizFno9wXu/B5GgiZi7XikI4lNpCEOQR52nU2/a9Sm4NFS3h/YDbxoBpS20yMxyxhGN6vkO5tuJylCPpgyCwleSPsRcJi6rn/vxqvgInDv0/uko8q3+N4+51ozB+OXiMS5qLe8zvpUu2tKumObQ164R/3Mr8aPocQO2NLOGNIAnU0Eu33efAHrFlOWqlNDHscCV5yQR1nRjlXAOLMd8NBtriWof+RYD8++h36qHoJPuBJjUsGO/fi1BGCbHIDVYt+h31I6baW+WMtZtfeqd/TwTbP8JA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In prctl_enable_tagged_addr(), we check that nr_bits is in the correct range, but we do so in a twisted if/else block where the correct case is sandwiched between two error cases doing exactly the same thing. Simplify the if condition and pull the correct case outside with the rest of the success code path. Signed-off-by: Yosry Ahmed --- arch/x86/kernel/process_64.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index 33b268747bb7b..3f381906bbe1d 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -771,17 +771,13 @@ static int prctl_enable_tagged_addr(struct mm_struct *mm, unsigned long nr_bits) return -EBUSY; } - if (!nr_bits) { - mmap_write_unlock(mm); - return -EINVAL; - } else if (nr_bits <= LAM_U57_BITS) { - mm->context.lam_cr3_mask = X86_CR3_LAM_U57; - mm->context.untag_mask = ~GENMASK(62, 57); - } else { + if (!nr_bits || nr_bits > LAM_U57_BITS) { mmap_write_unlock(mm); return -EINVAL; } + mm->context.lam_cr3_mask = X86_CR3_LAM_U57; + mm->context.untag_mask = ~GENMASK(62, 57); write_cr3(__read_cr3() | mm->context.lam_cr3_mask); set_tlbstate_lam_mode(mm); set_bit(MM_CONTEXT_LOCK_LAM, &mm->context.flags);