From patchwork Mon Jul 18 12:02:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nadav Amit X-Patchwork-Id: 12921688 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6C85CCA481 for ; Mon, 18 Jul 2022 19:37:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 428B26B0080; Mon, 18 Jul 2022 15:37:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 363186B0081; Mon, 18 Jul 2022 15:37:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 166206B0082; Mon, 18 Jul 2022 15:37:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id F36866B0080 for ; Mon, 18 Jul 2022 15:37:21 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D2759208B0 for ; Mon, 18 Jul 2022 19:37:21 +0000 (UTC) X-FDA: 79701229482.11.EBD5885 Received: from relay5.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by imf02.hostedemail.com (Postfix) with ESMTP id 875FD80043 for ; Mon, 18 Jul 2022 19:37:21 +0000 (UTC) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 3C1F0208B0 for ; Mon, 18 Jul 2022 19:37:21 +0000 (UTC) X-FDA: 79701229482.29.2F78B5B Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf07.hostedemail.com (Postfix) with ESMTP id ABCD94005B for ; Mon, 18 Jul 2022 19:37:20 +0000 (UTC) Received: by mail-pf1-f171.google.com with SMTP id 70so11573785pfx.1 for ; Mon, 18 Jul 2022 12:37:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sYv+0w7UbdJfCd/9qiOzPSdo3DPlp98QBnk6mmrNOzw=; b=LfD4eNC9SIqr9cYWB25Xbr4WiNNia1nr+3GwzKavBUssVyFOxkqXDhxuEsCKotgHGX IWop0WCMjmuoVKLDBbwwaKBQDtdmqkvwVj7sFVhuA7x/GT8cqq96HiJcsEWIREVlkZkC XkrXrhlO0AHfhi7Ka0pK/Bk6TGPHIcLO4v6aQWeY6gTl5lS1/EBV6g/dpM3ldLkVpRs4 P/DRFwB289y/gdLxuvwxWRrdWSS/ANky/BYuh5pHXytyibxUmY1neqRo3cZIvwKFVK6F YK0uUoEexTXqrgimYNcJK1bFS70JivrQyx9FnwmH/rpHZPac+0k2ux4Obajgr/ZrEpGI ztog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sYv+0w7UbdJfCd/9qiOzPSdo3DPlp98QBnk6mmrNOzw=; b=ZA+RbUAx3hUg+3Suqs2XmzGf3lYQuKpUkrmZ3YHFyGX8bezA2A8L88YeTGdZJfZq7N dixbd8F/7Pm4snlp0JMpbaNCln9Y4oI+ahZhZUbcf1ZGZWA1pq0yUHqid/+K3QGZRVOR zF+ZCHbFqoTTVCe08A2USb/OxK+TeBNoniBthI1ZpN5Rv3KuTXePaTUDTHn8Y4mNNBdi kYW4wQWtDvGZesAOVkx8owaDwPLP4lDfI+256D5ft+8U4r1dkqsvMSRaz+SO3Hrh0exd AfU7EK+0rrJ7Ym0YvPq4+0GhU51fP3KBDoJVs/WKX3h/8PlvC4cgwM0pmLy6E/6uITQB zwug== X-Gm-Message-State: AJIora9A+hGsa8XN6lj+q7NmFZA0KHIeSPfPTR7XOoivLE6076AwjZ3x Lxg1Jba9nzErs/c7IPO1W2Dr6NXKxhj96A== X-Google-Smtp-Source: AGRyM1vupv7tu2tOFZ1Su8p2ph2+On2wlRIi/LDIoBmyKSXGXyJd3hN8YQju5I/uoHMU9RGeKryGig== X-Received: by 2002:a63:2684:0:b0:415:18d8:78dd with SMTP id m126-20020a632684000000b0041518d878ddmr26454387pgm.33.1658173039290; Mon, 18 Jul 2022 12:37:19 -0700 (PDT) Received: from sc2-haas01-esx0118.eng.vmware.com ([66.170.99.1]) by smtp.gmail.com with ESMTPSA id q6-20020a170902a3c600b0016bc4a6ce28sm9907887plb.98.2022.07.18.12.37.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 12:37:18 -0700 (PDT) From: Nadav Amit X-Google-Original-From: Nadav Amit To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , Mike Rapoport , Axel Rasmussen , Nadav Amit , Andrea Arcangeli , Andrew Cooper , Andy Lutomirski , Dave Hansen , David Hildenbrand , Peter Xu , Peter Zijlstra , Thomas Gleixner , Will Deacon , Yu Zhao , Nick Piggin Subject: [RFC PATCH 11/14] x86/mm: use relaxed TLB flushes when protection is removed Date: Mon, 18 Jul 2022 05:02:09 -0700 Message-Id: <20220718120212.3180-12-namit@vmware.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220718120212.3180-1-namit@vmware.com> References: <20220718120212.3180-1-namit@vmware.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1658173041; a=rsa-sha256; cv=none; b=IP0jhPYthAKoolBIK1G2Qm/d1hp8D6gG0Wi9khQxeSvgLmU9ErrPmc1PS2vyoaQ1+2a5re DuDON6vAQKucpkHyeKFXemX1secpQvV8p20rYjCx9U7TSTOqf177Tk4jzTntQ1MSjMWj08 HGY5tFhvfPkYIFltFMN4n7ucnlUA1UA= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=LfD4eNC9; dmarc=pass (policy=none) header.from=gmail.com; spf=none (imf02.hostedemail.com: domain of MAILER-DAEMON@hostedemail.com has no SPF policy when checking 216.40.44.12) smtp.mailfrom=MAILER-DAEMON@hostedemail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1658173041; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sYv+0w7UbdJfCd/9qiOzPSdo3DPlp98QBnk6mmrNOzw=; b=wbJDoPWRIJR+NkF7T3kE3OcQVCxTKqNBPGYhBFo7yJ6/PWyvSv1gb4I+hZExZxC2qErQff PWF0kMrYYd8RMg35Rc1gYCKgDfHa5lN/fuT10Lvvi+b3T1HfFYPz7kUY3OlDkK9ttbVPtp sEssSipx0SdPR2BLFxGNskGNkQJVJyM= X-HE-Tag-Orig: 1658173040-117575 X-Stat-Signature: dnbdxze65imc9519jjzmas6yydijcuzq X-Rspamd-Queue-Id: 875FD80043 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=LfD4eNC9; dmarc=pass (policy=none) header.from=gmail.com; spf=none (imf02.hostedemail.com: domain of MAILER-DAEMON@hostedemail.com has no SPF policy when checking 216.40.44.12) smtp.mailfrom=MAILER-DAEMON@hostedemail.com X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1658173041-170570 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Nadav Amit When checking x86 PTE flags to determine whether a TLB flush is needed, determine whether a relaxed TLB flush is sufficient. If protection is added (NX removed or W added), indicate that a relaxed TLB flush would suffice. Cc: Andrea Arcangeli Cc: Andrew Cooper Cc: Andrew Morton Cc: Andy Lutomirski Cc: Dave Hansen Cc: David Hildenbrand Cc: Peter Xu Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Will Deacon Cc: Yu Zhao Cc: Nick Piggin Signed-off-by: Nadav Amit --- arch/x86/include/asm/tlbflush.h | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index 230cd1d24fe6..4f98735ab07a 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -271,18 +271,23 @@ static inline enum pte_flush_type pte_flags_flush_type(unsigned long oldflags, * dirty/access bit if needed without a fault. */ const pteval_t flush_on_clear = _PAGE_DIRTY | _PAGE_PRESENT | - _PAGE_ACCESSED; + _PAGE_ACCESSED | _PAGE_RW; + const pteval_t flush_on_set = _PAGE_NX; + const pteval_t flush_on_set_relaxed = _PAGE_RW; + const pteval_t flush_on_clear_relaxed = _PAGE_NX; const pteval_t software_flags = _PAGE_SOFTW1 | _PAGE_SOFTW2 | _PAGE_SOFTW3 | _PAGE_SOFTW4; - const pteval_t flush_on_change = _PAGE_RW | _PAGE_USER | _PAGE_PWT | + const pteval_t flush_on_change = _PAGE_USER | _PAGE_PWT | _PAGE_PCD | _PAGE_PSE | _PAGE_GLOBAL | _PAGE_PAT | _PAGE_PAT_LARGE | _PAGE_PKEY_BIT0 | _PAGE_PKEY_BIT1 | - _PAGE_PKEY_BIT2 | _PAGE_PKEY_BIT3 | _PAGE_NX; + _PAGE_PKEY_BIT2 | _PAGE_PKEY_BIT3; unsigned long diff = oldflags ^ newflags; BUILD_BUG_ON(flush_on_clear & software_flags); BUILD_BUG_ON(flush_on_clear & flush_on_change); BUILD_BUG_ON(flush_on_change & software_flags); + BUILD_BUG_ON(flush_on_change & flush_on_clear_relaxed); + BUILD_BUG_ON(flush_on_change & flush_on_set_relaxed); /* Ignore software flags */ diff &= ~software_flags; @@ -301,9 +306,16 @@ static inline enum pte_flush_type pte_flags_flush_type(unsigned long oldflags, if (diff & flush_on_change) return PTE_FLUSH_STRICT; + if (diff & oldflags & flush_on_clear_relaxed) + return PTE_FLUSH_RELAXED; + + if (diff & newflags & flush_on_set_relaxed) + return PTE_FLUSH_RELAXED; + /* Ensure there are no flags that were left behind */ if (IS_ENABLED(CONFIG_DEBUG_VM) && - (diff & ~(flush_on_clear | software_flags | flush_on_change))) { + (diff & ~(flush_on_clear | flush_on_set | + software_flags | flush_on_change))) { VM_WARN_ON_ONCE(1); return PTE_FLUSH_STRICT; }