From patchwork Tue Aug 6 02:21:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13754346 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1D2A9C3DA4A for ; Tue, 6 Aug 2024 02:22:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To:From: Subject:Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=yV7dY4XDiIXGgrOmDcJWCpFmJrDLDUqsW20nfMTrSyE=; b=ZulJdGHWZCpESJMX0azlQxvHpP yTK63572orHNRAjwU3uPFJfghgw/6VKgnHyzpUdeZt/vj0j5TRT3vIcjJXKUFcYohxD9NjzR4SLGi 0ftrXLNsHl2w5RRV1u6726MfUaahQms4DQFrPpk7a9UbH8CZ7l36tlgdxbGQITVCAYa2BdXBu9WPk NE3/us6Ivs7n5IKrAgmIC74is2Bvn3TbSSvJWo27RmhVScdm+CtCf2z7GfUXQjrcG1UJz4oP78cG1 zmE4GogV1+SZ3SB5cZARDT9RzNsi/iGOIhzObaJ6Xln+oVfX9uMa8VNEykK67Z6REVOrpGV9YYBYR ybnlzVRw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sb9qC-00000000G8u-1tbo; Tue, 06 Aug 2024 02:22:24 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sb9pD-00000000FpI-3pD3 for linux-arm-kernel@lists.infradead.org; Tue, 06 Aug 2024 02:21:25 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6698f11853aso6016547b3.0 for ; Mon, 05 Aug 2024 19:21:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1722910882; x=1723515682; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=yV7dY4XDiIXGgrOmDcJWCpFmJrDLDUqsW20nfMTrSyE=; b=essDp1899C1IpCLN9DdILJns69KrxP0I08dJHjaiSew2jrjYW3URb9yeXCbdsQF4jK CAIdbSGezlqqwh3Pwoov3E2acrfDowhVLU19Uv60lTtbfq9KcLF8OF8WKiNkbixwTbbN pqSkghqS3eAXMZTWopnJcVM6PvjiMNkTOqj/ed/Ye/k2i5gbnpHQVCQLbYj8PXXpdyk4 8dNy3+ubYMGXIBWAKiutgrR05z52mrkLS/cZ+MfYH0V9Sr+8DTnAIX0y+oBRmaxDDAR0 9JK7VrLMx25S3i3Zqcqvi9CkbKbEdyp63BrHDFMLNU6eDWIp8EjGj/w76S8i22RfF/2S bQ5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722910882; x=1723515682; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yV7dY4XDiIXGgrOmDcJWCpFmJrDLDUqsW20nfMTrSyE=; b=hoGQw1H1n7Z4VtpWaFdhdmNdfPqyAJVr8GREw5Ia5xMidq1SpB5d1CRYwJ2Shc74lj Seguv6xEsB7JvnmLNPps2rnoSKcHCN+NmfCRTgP9YzCxY/odE19U0mAhYVd2dNRQlLVl qjSfDQZGI2fzQn0Eqq40hwKOcWkTeKv06e47VmmLHKYoqcs+aYEdFxI3wwnd0Zr+vcW/ ygjEHw9bjzwHzSjpfaN07NbILPzdo7bRPsp80BAbRbYfIIRMdBHQCShLtMS9yQRjMkPP VoSA8CaNCoiTBKtcI3gm0a88giEytA5LRrJI8jw5ZQ9PPWitpR6+ICN4OCaB+SFFnlLl vaMg== X-Forwarded-Encrypted: i=1; AJvYcCWR7gF9k7DbcHwkSYlexwqMwqnIC+kaQ03QI5JvN7xvA7NdOMp1QIwHjjDQu+3YvAn0Iw5nJkIhgcyApwMUX0toDJPLotIC6fRD3DYLay6gLG01Udo= X-Gm-Message-State: AOJu0Ywmj/rGVZFe772Y1otQrglLeqC6LDeNQW211y20RSUH7kj8chTj FaJn0RYOq2xitygkz9aHVuYda6s5/IFIrOeW2O0A726CwaFE/5N8JXwGDpk7Zjj+o9PNhDGpW/j CnQ== X-Google-Smtp-Source: AGHT+IGSY6zrX4Qs7JrUctkrSdbtisepSykseFX5DdXlWxT8TG2QUbuQRw2CO7r/i/yRAh7feSGbV18ovH8= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:261c:802b:6b55:e09c]) (user=yuzhao job=sendgmr) by 2002:a05:6902:2b84:b0:e0b:cce3:45c7 with SMTP id 3f1490d57ef6-e0bde3ef075mr24855276.9.1722910881783; Mon, 05 Aug 2024 19:21:21 -0700 (PDT) Date: Mon, 5 Aug 2024 20:21:11 -0600 In-Reply-To: <20240806022114.3320543-1-yuzhao@google.com> Mime-Version: 1.0 References: <20240806022114.3320543-1-yuzhao@google.com> X-Mailer: git-send-email 2.46.0.rc2.264.g509ed76dc8-goog Message-ID: <20240806022114.3320543-2-yuzhao@google.com> Subject: [RFC PATCH 1/4] mm: HVO: introduce helper function to update and flush pgtable From: Yu Zhao To: Catalin Marinas , Will Deacon Cc: Andrew Morton , David Rientjes , Douglas Anderson , Frank van der Linden , Mark Rutland , Muchun Song , Nanyong Sun , Yang Shi , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Yu Zhao X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240805_192124_333120_6339F6F0 X-CRM114-Status: GOOD ( 16.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Nanyong Sun Add pmd/pte update and tlb flush helper function to update page table. This refactoring patch is designed to facilitate each architecture to implement its own special logic in preparation for the arm64 architecture to follow the necessary break-before-make sequence when updating page tables. Signed-off-by: Nanyong Sun Reviewed-by: Muchun Song Signed-off-by: Yu Zhao --- mm/hugetlb_vmemmap.c | 55 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 829112b0a914..2dd92e58f304 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -46,6 +46,37 @@ struct vmemmap_remap_walk { unsigned long flags; }; +#ifndef vmemmap_update_pmd +static inline void vmemmap_update_pmd(unsigned long addr, + pmd_t *pmdp, pte_t *ptep) +{ + pmd_populate_kernel(&init_mm, pmdp, ptep); +} +#endif + +#ifndef vmemmap_update_pte +static inline void vmemmap_update_pte(unsigned long addr, + pte_t *ptep, pte_t pte) +{ + set_pte_at(&init_mm, addr, ptep, pte); +} +#endif + +#ifndef vmemmap_flush_tlb_all +static inline void vmemmap_flush_tlb_all(void) +{ + flush_tlb_all(); +} +#endif + +#ifndef vmemmap_flush_tlb_range +static inline void vmemmap_flush_tlb_range(unsigned long start, + unsigned long end) +{ + flush_tlb_kernel_range(start, end); +} +#endif + static int vmemmap_split_pmd(pmd_t *pmd, struct page *head, unsigned long start, struct vmemmap_remap_walk *walk) { @@ -81,9 +112,9 @@ static int vmemmap_split_pmd(pmd_t *pmd, struct page *head, unsigned long start, /* Make pte visible before pmd. See comment in pmd_install(). */ smp_wmb(); - pmd_populate_kernel(&init_mm, pmd, pgtable); + vmemmap_update_pmd(start, pmd, pgtable); if (!(walk->flags & VMEMMAP_SPLIT_NO_TLB_FLUSH)) - flush_tlb_kernel_range(start, start + PMD_SIZE); + vmemmap_flush_tlb_range(start, start + PMD_SIZE); } else { pte_free_kernel(&init_mm, pgtable); } @@ -171,7 +202,7 @@ static int vmemmap_remap_range(unsigned long start, unsigned long end, return ret; if (walk->remap_pte && !(walk->flags & VMEMMAP_REMAP_NO_TLB_FLUSH)) - flush_tlb_kernel_range(start, end); + vmemmap_flush_tlb_range(start, end); return 0; } @@ -220,15 +251,15 @@ static void vmemmap_remap_pte(pte_t *pte, unsigned long addr, /* * Makes sure that preceding stores to the page contents from - * vmemmap_remap_free() become visible before the set_pte_at() - * write. + * vmemmap_remap_free() become visible before the + * vmemmap_update_pte() write. */ smp_wmb(); } entry = mk_pte(walk->reuse_page, pgprot); list_add(&page->lru, walk->vmemmap_pages); - set_pte_at(&init_mm, addr, pte, entry); + vmemmap_update_pte(addr, pte, entry); } /* @@ -267,10 +298,10 @@ static void vmemmap_restore_pte(pte_t *pte, unsigned long addr, /* * Makes sure that preceding stores to the page contents become visible - * before the set_pte_at() write. + * before the vmemmap_update_pte() write. */ smp_wmb(); - set_pte_at(&init_mm, addr, pte, mk_pte(page, pgprot)); + vmemmap_update_pte(addr, pte, mk_pte(page, pgprot)); } /** @@ -536,7 +567,7 @@ long hugetlb_vmemmap_restore_folios(const struct hstate *h, } if (restored) - flush_tlb_all(); + vmemmap_flush_tlb_all(); if (!ret) ret = restored; return ret; @@ -664,7 +695,7 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_l break; } - flush_tlb_all(); + vmemmap_flush_tlb_all(); /* avoid writes from page_ref_add_unless() while folding vmemmap */ synchronize_rcu(); @@ -684,7 +715,7 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_l * allowing more vmemmap remaps to occur. */ if (ret == -ENOMEM && !list_empty(&vmemmap_pages)) { - flush_tlb_all(); + vmemmap_flush_tlb_all(); free_vmemmap_page_list(&vmemmap_pages); INIT_LIST_HEAD(&vmemmap_pages); __hugetlb_vmemmap_optimize_folio(h, folio, &vmemmap_pages, @@ -692,7 +723,7 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_l } } - flush_tlb_all(); + vmemmap_flush_tlb_all(); free_vmemmap_page_list(&vmemmap_pages); } From patchwork Tue Aug 6 02:21:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13754347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 11516C3DA4A for ; Tue, 6 Aug 2024 02:23:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To:From: Subject:Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=O2mshx39uv2zkvLi95oIpi0tXJ9/QGv1zrpIuDCw18w=; b=DSuZ46VL0uWalPFuu8tz2+jMa3 8vacJXnFX+ClKCH21TTkDDfq+FbRoZ0Hb0/0z+BrIlszRGWQFZOzULn0DUfEGQzcFZUQiSL73tyD2 tTY0piZQcrTzjCf18TccYclkAZN+Y4HkW0QHDL6XLaihfo4Df2+oeqEPwA2dvojHyXzs2nBK3tlde 5+jM4EW4ej1Tw/L2Oi502xriEcL+M6d4jAUTcRLXaW5bpIZEMPE5ao4Fm94BN8kal3gaot/Ga7P+F toXWjCngTf7swEBWUm0f7s7Zh3Om7/+GuDLvdNAZ0XJAoAPOrSwpw7zL7lDdVbLbhNeYsmZuyMYRK imPTYsQA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sb9qh-00000000GIT-1Jrl; Tue, 06 Aug 2024 02:22:55 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sb9pF-00000000Fq8-47cN for linux-arm-kernel@lists.infradead.org; Tue, 06 Aug 2024 02:21:27 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-68d1d966ef7so4039907b3.1 for ; Mon, 05 Aug 2024 19:21:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1722910884; x=1723515684; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=O2mshx39uv2zkvLi95oIpi0tXJ9/QGv1zrpIuDCw18w=; b=Aj16ar9x5HcIZnlgM8e9fct6DYFVeRTQ9+T+BK/3fX+xg+XbBay1sjcr0ubG7YRhiv XXd2Kz6yvtAXYxiZH8kvPRR2zN/zpK1HiuKd5NYIzNOSBzHqT6igyHjLWaufyRzvEi7B aKcqWfAvsoWRUC5W88McluugGOm7UmLYZy5npNGLzVPYI1+RK52o6qzXvFHVCrQF0/D9 5afFpT1EYPv0R0SDd8wxVPH4JRj1KRzC49iTbzxGP0vhoTIn0BUFkxbTTM261LDkZHgE sGPI2HvmKk1nZryNhGTS01msl9pDYvTGfAKyG5S7ncG5rOQdVtpHCU5McMuYCUi+gm8M E9VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722910884; x=1723515684; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=O2mshx39uv2zkvLi95oIpi0tXJ9/QGv1zrpIuDCw18w=; b=xLhAEoViRr1Cxs9rrOo6yyBz417vl2lTCHX5bse0DGr+nJZWXayaL+L1sIUccZgLSY R5EJxp+QP9GlBWwwi5B7hTCPS4A/P7124QDod5FhhGnJVOSa3uweyLZDx9hq9Mif2uLm f39Z3fXwRgBi/M1mAnuRwAe/xubK/yEVjDVxgeiySzMo9UxGWkO8VCLb8u6yytrD3oIy mPqn6SXjdOjXBYIz8pEFmRx5wK7S0knFD94U/gKdINOARIMlzjhsKE3DE1Anw98OUfxw /A1N5B0RFWbnAsddsXTQHdzCvetbA+wm2AB90BDPbGPsWdG4Se5REWH/jJPqUR2FZ2Th cW8A== X-Forwarded-Encrypted: i=1; AJvYcCUli15eXjWrpxgHiHdxJ50vAUKSfolwN+q+E60zw0U7DdT4r1BFGGBy0qIv9yjlFdVw2C/R1XQ3ist632pm7fqlrWC6Pp02arBFSLT2dAYw5E0tAEE= X-Gm-Message-State: AOJu0YxJpoQGKHR3gYnP+0YVm6iTPsYPUfD8Nw+L8lXGqCOZyMa8SNl8 oRmh7sPH8YyKfCW2UDFl+qQ/IAkrsipM+v8/5yP8yuE7w4bilcD3mNFko4iFDFKXkGBHBup4VLX tOQ== X-Google-Smtp-Source: AGHT+IG70ROskvb4vGAVk7HFhpn0VwxmpNbq5+lHHNoIlP/q1TqLemnSQjn2n7kwU3OW0QixZ2DVJNXlB8A= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:261c:802b:6b55:e09c]) (user=yuzhao job=sendgmr) by 2002:a05:690c:288:b0:673:b39a:9291 with SMTP id 00721157ae682-6895ffbd30dmr4163187b3.3.1722910884345; Mon, 05 Aug 2024 19:21:24 -0700 (PDT) Date: Mon, 5 Aug 2024 20:21:12 -0600 In-Reply-To: <20240806022114.3320543-1-yuzhao@google.com> Mime-Version: 1.0 References: <20240806022114.3320543-1-yuzhao@google.com> X-Mailer: git-send-email 2.46.0.rc2.264.g509ed76dc8-goog Message-ID: <20240806022114.3320543-3-yuzhao@google.com> Subject: [RFC PATCH 2/4] arm64: use IPIs to pause/resume remote CPUs From: Yu Zhao To: Catalin Marinas , Will Deacon Cc: Andrew Morton , David Rientjes , Douglas Anderson , Frank van der Linden , Mark Rutland , Muchun Song , Nanyong Sun , Yang Shi , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240805_192126_063656_C7B7A086 X-CRM114-Status: GOOD ( 16.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Use pseudo-NMI IPIs to pause remote CPUs for a short period of time, and then reliably resume them when the local CPU exits critical sections that preclude the execution of remote CPUs. A typical example of such critical sections is BBM on kernel PTEs. HugeTLB Vmemmap Optimization (HVO) on arm64 was disabled by commit 060a2c92d1b6 ("arm64: mm: hugetlb: Disable HUGETLB_PAGE_OPTIMIZE_VMEMMAP") due to the folllowing reason: This is deemed UNPREDICTABLE by the Arm architecture without a break-before-make sequence (make the PTE invalid, TLBI, write the new valid PTE). However, such sequence is not possible since the vmemmap may be concurrently accessed by the kernel. Supporting BBM on kernel PTEs is one of the approaches that can potentially make arm64 support HVO. Signed-off-by: Yu Zhao --- arch/arm64/include/asm/smp.h | 3 + arch/arm64/kernel/smp.c | 110 +++++++++++++++++++++++++++++++++++ 2 files changed, 113 insertions(+) diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h index 2510eec026f7..cffb0cfed961 100644 --- a/arch/arm64/include/asm/smp.h +++ b/arch/arm64/include/asm/smp.h @@ -133,6 +133,9 @@ bool cpus_are_stuck_in_kernel(void); extern void crash_smp_send_stop(void); extern bool smp_crash_stop_failed(void); +void pause_remote_cpus(void); +void resume_remote_cpus(void); + #endif /* ifndef __ASSEMBLY__ */ #endif /* ifndef __ASM_SMP_H */ diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 5e18fbcee9a2..aa80266e5c9d 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -68,16 +68,25 @@ enum ipi_msg_type { IPI_RESCHEDULE, IPI_CALL_FUNC, IPI_CPU_STOP, + IPI_CPU_PAUSE, +#ifdef CONFIG_KEXEC_CORE IPI_CPU_CRASH_STOP, +#endif +#ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST IPI_TIMER, +#endif +#ifdef CONFIG_IRQ_WORK IPI_IRQ_WORK, +#endif NR_IPI, /* * Any enum >= NR_IPI and < MAX_IPI is special and not tracable * with trace_ipi_* */ IPI_CPU_BACKTRACE = NR_IPI, +#ifdef CONFIG_KGDB IPI_KGDB_ROUNDUP, +#endif MAX_IPI }; @@ -821,11 +830,20 @@ static const char *ipi_types[MAX_IPI] __tracepoint_string = { [IPI_RESCHEDULE] = "Rescheduling interrupts", [IPI_CALL_FUNC] = "Function call interrupts", [IPI_CPU_STOP] = "CPU stop interrupts", + [IPI_CPU_PAUSE] = "CPU pause interrupts", +#ifdef CONFIG_KEXEC_CORE [IPI_CPU_CRASH_STOP] = "CPU stop (for crash dump) interrupts", +#endif +#ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST [IPI_TIMER] = "Timer broadcast interrupts", +#endif +#ifdef CONFIG_IRQ_WORK [IPI_IRQ_WORK] = "IRQ work interrupts", +#endif [IPI_CPU_BACKTRACE] = "CPU backtrace interrupts", +#ifdef CONFIG_KGDB [IPI_KGDB_ROUNDUP] = "KGDB roundup interrupts", +#endif }; static void smp_cross_call(const struct cpumask *target, unsigned int ipinr); @@ -884,6 +902,85 @@ void __noreturn panic_smp_self_stop(void) local_cpu_stop(); } +static DEFINE_SPINLOCK(cpu_pause_lock); +static cpumask_t paused_cpus; +static cpumask_t resumed_cpus; + +static void pause_local_cpu(void) +{ + int cpu = smp_processor_id(); + + cpumask_clear_cpu(cpu, &resumed_cpus); + /* + * Paired with pause_remote_cpus() to confirm that this CPU not only + * will be paused but also can be reliably resumed. + */ + smp_wmb(); + cpumask_set_cpu(cpu, &paused_cpus); + /* A typical example for sleep and wake-up functions. */ + smp_mb(); + while (!cpumask_test_cpu(cpu, &resumed_cpus)) { + wfe(); + barrier(); + } + barrier(); + cpumask_clear_cpu(cpu, &paused_cpus); +} + +void pause_remote_cpus(void) +{ + cpumask_t cpus_to_pause; + + lockdep_assert_cpus_held(); + lockdep_assert_preemption_disabled(); + + cpumask_copy(&cpus_to_pause, cpu_online_mask); + cpumask_clear_cpu(smp_processor_id(), &cpus_to_pause); + + spin_lock(&cpu_pause_lock); + + WARN_ON_ONCE(!cpumask_empty(&paused_cpus)); + + smp_cross_call(&cpus_to_pause, IPI_CPU_PAUSE); + + while (!cpumask_equal(&cpus_to_pause, &paused_cpus)) { + cpu_relax(); + barrier(); + } + /* + * Paired with pause_local_cpu() to confirm that all CPUs not only will + * be paused but also can be reliably resumed. + */ + smp_rmb(); + WARN_ON_ONCE(cpumask_intersects(&cpus_to_pause, &resumed_cpus)); + + spin_unlock(&cpu_pause_lock); +} + +void resume_remote_cpus(void) +{ + cpumask_t cpus_to_resume; + + lockdep_assert_cpus_held(); + lockdep_assert_preemption_disabled(); + + cpumask_copy(&cpus_to_resume, cpu_online_mask); + cpumask_clear_cpu(smp_processor_id(), &cpus_to_resume); + + spin_lock(&cpu_pause_lock); + + cpumask_setall(&resumed_cpus); + /* A typical example for sleep and wake-up functions. */ + smp_mb(); + while (cpumask_intersects(&cpus_to_resume, &paused_cpus)) { + sev(); + cpu_relax(); + barrier(); + } + + spin_unlock(&cpu_pause_lock); +} + #ifdef CONFIG_KEXEC_CORE static atomic_t waiting_for_crash_ipi = ATOMIC_INIT(0); #endif @@ -963,6 +1060,11 @@ static void do_handle_IPI(int ipinr) local_cpu_stop(); break; + case IPI_CPU_PAUSE: + pause_local_cpu(); + break; + +#ifdef CONFIG_KEXEC_CORE case IPI_CPU_CRASH_STOP: if (IS_ENABLED(CONFIG_KEXEC_CORE)) { ipi_cpu_crash_stop(cpu, get_irq_regs()); @@ -970,6 +1072,7 @@ static void do_handle_IPI(int ipinr) unreachable(); } break; +#endif #ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST case IPI_TIMER: @@ -991,9 +1094,11 @@ static void do_handle_IPI(int ipinr) nmi_cpu_backtrace(get_irq_regs()); break; +#ifdef CONFIG_KGDB case IPI_KGDB_ROUNDUP: kgdb_nmicallback(cpu, get_irq_regs()); break; +#endif default: pr_crit("CPU%u: Unknown IPI message 0x%x\n", cpu, ipinr); @@ -1023,9 +1128,14 @@ static bool ipi_should_be_nmi(enum ipi_msg_type ipi) switch (ipi) { case IPI_CPU_STOP: + case IPI_CPU_PAUSE: +#ifdef CONFIG_KEXEC_CORE case IPI_CPU_CRASH_STOP: +#endif case IPI_CPU_BACKTRACE: +#ifdef CONFIG_KGDB case IPI_KGDB_ROUNDUP: +#endif return true; default: return false; From patchwork Tue Aug 6 02:21:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13754348 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6BE10C3DA7F for ; Tue, 6 Aug 2024 02:23:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To:From: Subject:Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Dz9+L9Nas5YdR+3aU4+X/tR5OCfJ0AKY3iAQMLv0S5k=; b=h8is+p2+mtnI1YKP27lv9g0Lbh FuVrKQzlbxXh+iSd4RkGphilUHfoZ4sVaNDzj7seGMYEluwc5SyIlLoewgNfhvnNr0Dq/rduxyV4a l2zylWlCJJtwqU1aclpvvSbHrU5oa8jdP9EGGMgHdV4VFnuelWdV1KK5occqSwr1Sa1p93Hid6G3O 9fhxtRtaJdDvNs/rcBJRaBU67YwfHHH7zRZ93aXOZvS5Ewxv0SXW1NLbt+KkAE6xNjG6RP3NP33R2 aVu2NPKamhCFluNUB/gIUalOPJmGkY4oynTueCrF1QBrtWOB1Wn+OUKxpphIlVqS/cMWyWKZLHuUE 0Gutbhig==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sb9rD-00000000GTc-0SQn; Tue, 06 Aug 2024 02:23:27 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sb9pI-00000000Frl-0LFH for linux-arm-kernel@lists.infradead.org; Tue, 06 Aug 2024 02:21:29 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-690404fd34eso534347b3.1 for ; Mon, 05 Aug 2024 19:21:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1722910887; x=1723515687; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Dz9+L9Nas5YdR+3aU4+X/tR5OCfJ0AKY3iAQMLv0S5k=; b=Gq+2RmgMtvlGQbsq8iUoOYnmxg4t4svvO/dIqo7ZRZ8uv7ncXj8uaqnFA8tes38r5v BvxQche43tUzoTK1p3bZxVbqKpwJ9SVdRyJyMTSFslMIt7JKFVVVoY9dK4zCpIKYoATw LYwpZH5KPEvMU/zeOcDnHyuxrcTQF6CzPsKNOw8nb9A+rEclR7q62nhV8Y+7OufugEoe ihbWPCGJw67I9W7sMjVfuI2ZEHGYgvHEG18ZMmVOiRN/+wGAJjRKtXIxdtGgNe1f0eEn VrqCUGmyv86aSVztMGImSotWVkzCMalEYfRa28zoT+rL6y+JbkfRdDP0nCvjCVteyFqM 7giA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722910887; x=1723515687; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Dz9+L9Nas5YdR+3aU4+X/tR5OCfJ0AKY3iAQMLv0S5k=; b=Nfmd8jKvxR+OKK/cASanSAIU+a1GNA/UnhbkD6UhK77ar5Wssto3bGR2GiaIygbXpX f4NHrqORLJHyTieNGbVjKJ+QHsxY+3kVNvLxetWACkAY5ebkubt5X2Scz2/HPcH8a3Li YNLPUtjW2I3glM96lGeykDV2q6DDOSvUvd3tGaDLbQeRDxYoxSTOzkxCyimZ/aFhCQhk 1HD+4eclvTL1zVrC6CKnOHMHb7OpwMaQuU+FcjAtoDMp0OFi8btz31d0DUtKI9UW9t/G 20BlW/pMD2vYQygktt5IibZT4EMOmtav8XwUa469S+JRuxZrzYCdLq0wD1cJCUBxx+Js e6bA== X-Forwarded-Encrypted: i=1; AJvYcCUxMBadRcMMuQZ9hjMyvGAq6ndt4q5+QzX4gRAchdKkV87qONk0CyJ5mkaM54AeqUUL/NrS2s4L1xT486TeJiy/KfHhBBdp8Vl6TFORDHyghpfMtak= X-Gm-Message-State: AOJu0Yzi76DWn9fHzX1mBlCt6dj+o8XN9G3Kch6QVTOWwoBdpoyrcjJ2 nJ6vHPlevowk3bB2779pHVC3z+7rs20g4PvBsW1qPQR9R64F20EngcnbF02iHks2IvZ66ZPU7PR 20g== X-Google-Smtp-Source: AGHT+IE5Ot4MCUehL2BALtzt0Nbbacrl8HCjMPwdvfemN8nL8qevC8oWRD8zmA2efNk4jGDFZ//D943lM9s= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:261c:802b:6b55:e09c]) (user=yuzhao job=sendgmr) by 2002:a05:690c:113:b0:648:3f93:61f2 with SMTP id 00721157ae682-689641a414emr5295867b3.6.1722910886865; Mon, 05 Aug 2024 19:21:26 -0700 (PDT) Date: Mon, 5 Aug 2024 20:21:13 -0600 In-Reply-To: <20240806022114.3320543-1-yuzhao@google.com> Mime-Version: 1.0 References: <20240806022114.3320543-1-yuzhao@google.com> X-Mailer: git-send-email 2.46.0.rc2.264.g509ed76dc8-goog Message-ID: <20240806022114.3320543-4-yuzhao@google.com> Subject: [RFC PATCH 3/4] arm64: pause remote CPUs to update vmemmap From: Yu Zhao To: Catalin Marinas , Will Deacon Cc: Andrew Morton , David Rientjes , Douglas Anderson , Frank van der Linden , Mark Rutland , Muchun Song , Nanyong Sun , Yang Shi , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240805_192128_232114_1A9155D3 X-CRM114-Status: GOOD ( 12.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Pause remote CPUs so that the local CPU can follow the proper BBM sequence to safely update the vmemmap mapping `struct page` areas. While updating the vmemmap, it is guaranteed that neither the local CPU nor the remote ones will access the `struct page` area being updated, and therefore they will not trigger kernel PFs. Signed-off-by: Yu Zhao --- arch/arm64/include/asm/pgalloc.h | 55 ++++++++++++++++++++++++++++++++ mm/hugetlb_vmemmap.c | 14 ++++++++ 2 files changed, 69 insertions(+) diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h index 8ff5f2a2579e..1af1aa34a351 100644 --- a/arch/arm64/include/asm/pgalloc.h +++ b/arch/arm64/include/asm/pgalloc.h @@ -12,6 +12,7 @@ #include #include #include +#include #define __HAVE_ARCH_PGD_FREE #define __HAVE_ARCH_PUD_FREE @@ -137,4 +138,58 @@ pmd_populate(struct mm_struct *mm, pmd_t *pmdp, pgtable_t ptep) __pmd_populate(pmdp, page_to_phys(ptep), PMD_TYPE_TABLE | PMD_TABLE_PXN); } +#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP + +#define vmemmap_update_lock vmemmap_update_lock +static inline void vmemmap_update_lock(void) +{ + cpus_read_lock(); +} + +#define vmemmap_update_unlock vmemmap_update_unlock +static inline void vmemmap_update_unlock(void) +{ + cpus_read_unlock(); +} + +#define vmemmap_update_pte vmemmap_update_pte +static inline void vmemmap_update_pte(unsigned long addr, pte_t *ptep, pte_t pte) +{ + preempt_disable(); + pause_remote_cpus(); + + pte_clear(&init_mm, addr, ptep); + flush_tlb_kernel_range(addr, addr + PAGE_SIZE); + set_pte_at(&init_mm, addr, ptep, pte); + + resume_remote_cpus(); + preempt_enable(); +} + +#define vmemmap_update_pmd vmemmap_update_pmd +static inline void vmemmap_update_pmd(unsigned long addr, pmd_t *pmdp, pte_t *ptep) +{ + preempt_disable(); + pause_remote_cpus(); + + pmd_clear(pmdp); + flush_tlb_kernel_range(addr, addr + PMD_SIZE); + pmd_populate_kernel(&init_mm, pmdp, ptep); + + resume_remote_cpus(); + preempt_enable(); +} + +#define vmemmap_flush_tlb_all vmemmap_flush_tlb_all +static inline void vmemmap_flush_tlb_all(void) +{ +} + +#define vmemmap_flush_tlb_range vmemmap_flush_tlb_range +static inline void vmemmap_flush_tlb_range(unsigned long start, unsigned long end) +{ +} + +#endif /* CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP */ + #endif diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 2dd92e58f304..893c73493d9c 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -46,6 +46,18 @@ struct vmemmap_remap_walk { unsigned long flags; }; +#ifndef vmemmap_update_lock +static void vmemmap_update_lock(void) +{ +} +#endif + +#ifndef vmemmap_update_unlock +static void vmemmap_update_unlock(void) +{ +} +#endif + #ifndef vmemmap_update_pmd static inline void vmemmap_update_pmd(unsigned long addr, pmd_t *pmdp, pte_t *ptep) @@ -194,10 +206,12 @@ static int vmemmap_remap_range(unsigned long start, unsigned long end, VM_BUG_ON(!PAGE_ALIGNED(start | end)); + vmemmap_update_lock(); mmap_read_lock(&init_mm); ret = walk_page_range_novma(&init_mm, start, end, &vmemmap_remap_ops, NULL, walk); mmap_read_unlock(&init_mm); + vmemmap_update_unlock(); if (ret) return ret; From patchwork Tue Aug 6 02:21:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13754357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 14CCAC3DA7F for ; Tue, 6 Aug 2024 02:24:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To:From: Subject:Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=klWRv7jv+zR9aL4I7OIDRLjBsHfroErcwBIt7k9CqK8=; b=AZnQduJsybQAChZV1sKZ4qdtHr 5j12zolG4s14+xknwpGwxNGJPvRqbmst1r4oMsycYCI4k2E97HHoVM8kbIB+avgk/1Xenz6xOrH/C Xt8Squc+C3+neFCSuSN2DygvaoOl4Q6gZN0YYt/Wd5TGmNRJdjIVbLuUAKmD+Kn0FY3tTCBGF1G5c +2y7QjRFDwwYietRgXSWAY6eipIx3slOqpczbuywGl2JaIrlUmMkv2bWMG+jiT5ED73WyxVmCUArI KDd0KaeC3yTGyuVTlf1FyqI+toqU9cruYrz+11t6v8AaKVchcc9Zx3RCRDb4Y623UOYbIj43SeIb0 N9HhzYpw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sb9ri-00000000Gda-3ABe; Tue, 06 Aug 2024 02:23:58 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sb9pK-00000000FtN-3sY6 for linux-arm-kernel@lists.infradead.org; Tue, 06 Aug 2024 02:21:32 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e0be2fa8f68so320940276.3 for ; Mon, 05 Aug 2024 19:21:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1722910889; x=1723515689; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=klWRv7jv+zR9aL4I7OIDRLjBsHfroErcwBIt7k9CqK8=; b=GjqFFstGwQSzfyeRj4JBe3bhbKj3/FfsF0I0yiZ3R4+GUUAjZbBU5cW8TmxW/COp+i K/xSwpkN0ywi63iDfTm2quJdGWKoghQIe1EALg9rFO+P3RrHqUaGiyq5twHyFgkN11W6 UOEde3dxKqOYr5BlAuS7IE3eADRyyYmCD9V3HQVqCgiWcM34zzCuaUUK6ylKppPBUFme tUA5EOcotN3JH+oiHoGqEbxeojRmUL0qG2l5oaIaz0/JW+MCIsX1V8XM7Sg233ikB8Yb sqOL07vfujSVzOHfEwN3+DfYLbpIZvXbFgd1Z8PP5MA7iwesADee9laMEkdtQacuCqz6 qw9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722910889; x=1723515689; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=klWRv7jv+zR9aL4I7OIDRLjBsHfroErcwBIt7k9CqK8=; b=XvQcqEoscxhq5htKBWyO7wypWS6OC3/HcEzJBLmoh5ieVBC7q2v74Ppnykb5n7CkuC pFdqFipp6EAU1g9D48KtDo0jSp1Im4O7V9BBB0lrQnZ2L/ofJ/tHENav5CyvF43zLfl+ Z45sHH+50iTRQF1Zg+AwM4LkqfyRxLwaw5FOnhG0vWnYUmmIiEWBj8zF0apfrAyj357S /WQlHu/eCYzPK/bszCinYYHv4YkCK8PlzgG5jjLEWXU8x0uvobZB/rSNrBf0Hw/SvZml Y+1Udd6T4uvDCPuaYywvCUcsqlaAvDkibREmpN+Q9cFa2GVGCIskDuIjo7IWTJMeA31+ LWuA== X-Forwarded-Encrypted: i=1; AJvYcCXGvXWRrNezZcKaoYDlLfuSm/lJgT3OHyBD/wXuVNPI7s0c41xHKC8SOXrEuOOhB0EGbZXMSmoodP0Y6V30wKHikMwihn0PzK5fBmoRcgLIjc1Fl7c= X-Gm-Message-State: AOJu0YzcWW4YGTKbTMy3yGMWnox2chJ3z1lXpjt4sMFxSbhgQM1ogD0X i/yW1K85yVuXD5u0r/c0Lb7sdyXcbjP6grUwO1dGbrhi1hCMvJlENL+6A1NuEoFy0fXwKswtGn7 BbA== X-Google-Smtp-Source: AGHT+IGBeIkNUsFUwrlBhpMmppS6FEO5hkHGCy4Rnvwll+98qZLKAoOlt7677TqFz3cp98ge6KT5O3g57gw= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:261c:802b:6b55:e09c]) (user=yuzhao job=sendgmr) by 2002:a05:6902:2b90:b0:e05:eccb:95dc with SMTP id 3f1490d57ef6-e0bde283222mr64471276.6.1722910889235; Mon, 05 Aug 2024 19:21:29 -0700 (PDT) Date: Mon, 5 Aug 2024 20:21:14 -0600 In-Reply-To: <20240806022114.3320543-1-yuzhao@google.com> Mime-Version: 1.0 References: <20240806022114.3320543-1-yuzhao@google.com> X-Mailer: git-send-email 2.46.0.rc2.264.g509ed76dc8-goog Message-ID: <20240806022114.3320543-5-yuzhao@google.com> Subject: [RFC PATCH 4/4] arm64: mm: Re-enable OPTIMIZE_HUGETLB_VMEMMAP From: Yu Zhao To: Catalin Marinas , Will Deacon Cc: Andrew Morton , David Rientjes , Douglas Anderson , Frank van der Linden , Mark Rutland , Muchun Song , Nanyong Sun , Yang Shi , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Yu Zhao X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240805_192130_988821_2CA231A7 X-CRM114-Status: UNSURE ( 9.68 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Nanyong Sun Now update of vmemmap page table can follow the rule of break-before-make safely for arm64 architecture, re-enable HVO on arm64. Signed-off-by: Nanyong Sun Reviewed-by: Muchun Song Signed-off-by: Yu Zhao --- arch/arm64/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a2f8ff354ca6..25ff026cdaf5 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -105,6 +105,7 @@ config ARM64 select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_HUGE_PMD_SHARE if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36) + select ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP select ARCH_WANT_LD_ORPHAN_WARN select ARCH_WANTS_EXECMEM_LATE if EXECMEM select ARCH_WANTS_NO_INSTR