From patchwork Mon Jul 1 08:46:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13717672 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96DFCC2BD09 for ; Mon, 1 Jul 2024 08:48:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E9F246B00AE; Mon, 1 Jul 2024 04:48:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E4F166B00AF; Mon, 1 Jul 2024 04:48:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C2EBC6B00B0; Mon, 1 Jul 2024 04:48:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A0A6F6B00AE for ; Mon, 1 Jul 2024 04:48:33 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 56C7A41A2C for ; Mon, 1 Jul 2024 08:48:33 +0000 (UTC) X-FDA: 82290557706.10.6C50A08 Received: from mail-oa1-f52.google.com (mail-oa1-f52.google.com [209.85.160.52]) by imf01.hostedemail.com (Postfix) with ESMTP id 83C6340018 for ; Mon, 1 Jul 2024 08:48:31 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Hr64vZ9P; spf=none (imf01.hostedemail.com: domain of zhengqi.arch@bytedance.com has no SPF policy when checking 209.85.160.52) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719823700; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BSdXPp6cua9HpA6rLebdl04hcrKD7F+dglilKwP4NdI=; b=coB1sor1rK6RxxblQvTLIr7w9s7IWr//tVc0Dl+SK/UHodYKEYD0la1BL/oVdEU9fhwuQW EZLEYJJhb6mPuImlRu9TGR64pZs1pGERuV/2E+V2eeKdkA9e3gE6NgTaCqe5veaWmzPo3y qT3mRod0hKdgpLaEbN4EV9U4LtJMiw0= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Hr64vZ9P; spf=none (imf01.hostedemail.com: domain of zhengqi.arch@bytedance.com has no SPF policy when checking 209.85.160.52) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719823700; a=rsa-sha256; cv=none; b=FaGK0+8ycaXSM5mBZsY0eSR9nwHHgElN6UY3uxaE4xMtuobw2A5nduSEeU3xUoTEBBYpTd RNtmdDGuUPF9F4Z2dh4XmkEq11UdsWQ/JCl0kcaQmcX/Ou2Voml87Z86PlT1KFFxRDu4Di 6DlwxnDV+zr2vyz+EKV3OzR84wWwK6M= Received: by mail-oa1-f52.google.com with SMTP id 586e51a60fabf-25cd2b51fd3so351838fac.3 for ; Mon, 01 Jul 2024 01:48:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1719823710; x=1720428510; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BSdXPp6cua9HpA6rLebdl04hcrKD7F+dglilKwP4NdI=; b=Hr64vZ9PGR/7Y3MvgZ3byBZqg/xjp5/YlZ89/kd0pRe0BCzIocnCZeYxCN61eluMvv F+CWMtATdOJ+wyke65H58qts6DJ1sCwfs1wlqb9g7ry8vkrzsWiCm+85tZjlauJBmDuF 3TbP4yQcxV7GiSh0jeu9uy6jwshv8+XTZz9H8j5XS8un05XW3HotJQwTSuroSBDq/UUV MoLPoEClxFEbxDP/uq9ntZdqyLXTVR6XhusJNy+b6lP8WQ7aRD2WSi+ZuqzoLwp7CFJf I9NHkM2Az01UVW1IKsn75D/KdXBTL3xyiJGJeJd50ZjdUJJbUEba2qYr08WfxtU7gqle kTcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719823710; x=1720428510; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BSdXPp6cua9HpA6rLebdl04hcrKD7F+dglilKwP4NdI=; b=pYs3k/Zs1ftvbPohf0zvWcwC+K0sa8+TzJIod6X5iZD3Z8IzB88inqzxoJzBB1XhXc VtBpXNk6rySH1gZjj7JEBVzI6BfJlne6d0Drk1gB3QVmh+rMlOoARqzxVFDYUbZsA/MS V/1gmJIPnM47fNgJyY1jNFm1e4amqqF1uQ25l/COCFjgPL3zgNRfX5cG7+AvRUgpNdbJ sETQqGHVJW/O1nptFB2/avtxmbhAALz7HJQ3kno15/TwnoKk60KYt2ej1DpStRQmgcaG xX4P+rj5D3eIXYZST35PBI+usjk0fuNKEasd8FuAt4X701yGHpyCSniy2XZqqTnFQgLI 4TIg== X-Gm-Message-State: AOJu0YxEGm5OfgM49D32NP2oFq3VMwCLVb5edPzJ18RZg3BEcQyLMkSr hG9y8C9VO5AKJ+wtzJwPmjUHTdpsGzgMtzTZJfHHUNvaJxJPaDPMTtZSdwVPw1s= X-Google-Smtp-Source: AGHT+IFfafy76qxnP3FNZsq5qulnNjrKrfxWqb4/wPoL/DGlK5/mwzGRfDiOgExztR4uBC55G6FnbA== X-Received: by 2002:a05:6871:7806:b0:254:ecbd:1815 with SMTP id 586e51a60fabf-25db3709c91mr5161423fac.5.1719823710530; Mon, 01 Jul 2024 01:48:30 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.241]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70804a7e7e0sm5932374b3a.204.2024.07.01.01.48.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Jul 2024 01:48:30 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, mgorman@suse.de, muchun.song@linux.dev, akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Qi Zheng Subject: [RFC PATCH 6/7] x86: mm: define arch_flush_tlb_before_set_huge_page Date: Mon, 1 Jul 2024 16:46:47 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 83C6340018 X-Stat-Signature: 81nki8t46uzfkqs5zihxbs4nttafbuzs X-Rspam-User: X-HE-Tag: 1719823711-397342 X-HE-Meta: U2FsdGVkX1/6Pq34E4J6pydwHZnNEN9OiR8yX4fx9bCmolI1CeCSR4Gjm/jrBabc4I5JrGWoc8Ub9KxgjIInbnccO4fS1XKiTyDAIKctD5bdUM3ElLzpClkViEC7whzfR7VZ0MsBlBdG+1M9A1TwRJqssKgBiTu7jfU5XOEckMCfP+3l/FjoIozrcGdsDDwsS4MNFZzeSzH97NrwIyOLKIpLs0r28YSe77ngehWrRzEOx2gyJhAbe/zbHk97N6MRjSjMdmZ2aDarLqelAfWuPeFZLA12vmbwyfJDBC4NhVZU3L73ZVQ7wEGeTeQ31Ymsg+Hc+xWrHS9jx5HOx1f58icul3baO0VAA8xpSzZPmw95GPRc+7g8+kZ2E/1hWWgZSfYqWxjeIc7SMitg6wufOtLwpUsscJS7kIqLFhx0nup80WD5FRuvAuW8N4zsLLEwJJETA8laLRyu/If54vCDn15Tc1wFMHB3St8/3p3O/I0aRxH3d0RdTyGIR9OjwPkyFlw9oWP/42Sh7THeD0FdHPIqMwUmJJmuyHYl0I0CTh79EHLMQ3FSU1u4sBL+xixpzZNK301NWct30ff5spkglZtrYX2DwcYkA7/OUZAoOeZe7FT8lBiXDTAGN0AbpGmPpFIdwAyXf7QsDwzpI/XJG3VrHte7XTL4viJ63r0R1u8UeWNlMBw7wonW8IAcoIqWtBMYhDKSAx25SgZFrhMxg7EcMGcFa1I+ZGoAWKldLHvkyojSN4zjHt6nF3ZKU64vwcnMJJ89m0ewp0RWDlAsfryKZBRUcwGLUF0/qTP0Y4K3RjT0W1+kYexR3ActjRj5YZOcKv5jxQrdza6qNL5mHk2YNe6El6EvW9mMpYJni8wsykvBhYPXvC8GPnor7QmUtMpQbhb0LU89pRb3d0n5st+U6d+HRMq7VQjKU+XCu8JLc+hi2CWJjzcKgT4Tghcj5ndcq145oyAue9vNR7U 5MjeuzvS zy6ZV86fdqscBGAwAV/wY/e77IBdO51QHRNSV8uvBq5Kupf/FacGbHr4rzMIFgu0MNTJJjsjjYL+mNmC0L1yncsSo+K13BHNs257PfMLlnfmhgfxxUH+3Eqsc8zFGP9CoNEoTeVhQcNpKWr+21oF2fW2dB1AnzhFvqhKuWdcJw18zYNmtMI5oFxegXbZN6P+ZGvNtRpPvlGOewaEHjMi3D/maEpDYoEthweSWtAI113zaNwjf8YwnJ64YFJ16g7uiIbyzsSviNoj4AlcYcrMNE7OeI1za/cAf1IN74fR3NMN/2hubRXpjOzYL7bmEPcndK9LU9Uye7ZzgMSp6X90YBwDEOvjs4yJG+azE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When we use mmu_gather to batch flush tlb and free PTE pages, the TLB is not flushed before pmd lock is unlocked. This may result in the following two situations: 1) Userland can trigger page fault and fill a huge page, which will cause the existence of small size TLB and huge TLB for the same address. 2) Userland can also trigger page fault and fill a PTE page, which will cause the existence of two small size TLBs, but the PTE page they map are different. According to Intel's TLB Application note (317080), some CPUs of x86 do not allow the 1) case, so define arch_flush_tlb_before_set_huge_page to detect and fix this issue. Signed-off-by: Qi Zheng --- arch/x86/include/asm/pgtable.h | 6 ++++++ arch/x86/mm/pgtable.c | 13 +++++++++++++ 2 files changed, 19 insertions(+) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index e39311a89bf4..f93d964ab6a3 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1668,6 +1668,12 @@ void arch_check_zapped_pte(struct vm_area_struct *vma, pte_t pte); #define arch_check_zapped_pmd arch_check_zapped_pmd void arch_check_zapped_pmd(struct vm_area_struct *vma, pmd_t pmd); +#ifdef CONFIG_PT_RECLAIM +#define arch_flush_tlb_before_set_huge_page arch_flush_tlb_before_set_huge_page +void arch_flush_tlb_before_set_huge_page(struct mm_struct *mm, + unsigned long addr); +#endif + #ifdef CONFIG_XEN_PV #define arch_has_hw_nonleaf_pmd_young arch_has_hw_nonleaf_pmd_young static inline bool arch_has_hw_nonleaf_pmd_young(void) diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index cd5bf2157611..d037f7425f82 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -926,3 +926,16 @@ void arch_check_zapped_pmd(struct vm_area_struct *vma, pmd_t pmd) VM_WARN_ON_ONCE(!(vma->vm_flags & VM_SHADOW_STACK) && pmd_shstk(pmd)); } + +#ifdef CONFIG_PT_RECLAIM +void arch_flush_tlb_before_set_huge_page(struct mm_struct *mm, + unsigned long addr) +{ + if (atomic_read(&mm->tlb_flush_pending)) { + unsigned long start = ALIGN_DOWN(addr, PMD_SIZE); + unsigned long end = start + PMD_SIZE; + + flush_tlb_mm_range(mm, start, end, PAGE_SHIFT, false); + } +} +#endif /* CONFIG_PT_RECLAIM */