From patchwork Wed Mar 27 04:49:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605546 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D25DC47DD9 for ; Wed, 27 Mar 2024 04:50:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 200386B0093; Wed, 27 Mar 2024 00:50:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 187A86B0095; Wed, 27 Mar 2024 00:50:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0517C6B0096; Wed, 27 Mar 2024 00:50:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E7A496B0093 for ; Wed, 27 Mar 2024 00:50:45 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BB8481A0546 for ; Wed, 27 Mar 2024 04:50:45 +0000 (UTC) X-FDA: 81941593650.30.D39E78E Received: from mail-ot1-f45.google.com (mail-ot1-f45.google.com [209.85.210.45]) by imf07.hostedemail.com (Postfix) with ESMTP id D259140010 for ; Wed, 27 Mar 2024 04:50:43 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=LJeD13l6; spf=pass (imf07.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.45 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515043; a=rsa-sha256; cv=none; b=bk/hWrC6aKXmuFx1WJADFx0wKKuUQh1aX+U2vPpaT7/+0MjZs1heaaRypS+b8Jm7Sii8ZA 18D6s1vkpQSho3T2wNpIRo4hJwajU3KhKTefqgq/nYCoCCuWdHMjFq1EjzF/UZp/nwbGP1 sa9Vd+DF+GmrSZNPCjuQdEd116FoQ/c= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=LJeD13l6; spf=pass (imf07.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.45 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515043; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Tmp/TOSalTk1E9Om53lqcGFTwB3eFzfKle7hmp+7bao=; b=3lwhRQHSZ439j8fRaBzYX7chKQkNubv/WzLuImOOqSb84SiQw5F9+8W1Xkxiu9bicUyyZP 62SgKVOY3/ctBmrwR7JcKa57x6wKjQDJWSet/lbC7hILToFINGVCsmMBkXkIqS96LM7wo6 DzzIrbsoIC5qv73VT/UlXeSTwa024Is= Received: by mail-ot1-f45.google.com with SMTP id 46e09a7af769-6e67d42422aso3794906a34.0 for ; Tue, 26 Mar 2024 21:50:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515043; x=1712119843; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Tmp/TOSalTk1E9Om53lqcGFTwB3eFzfKle7hmp+7bao=; b=LJeD13l6RhF2OWdBw874ogDgs1M6kSSndaSou18TmG8ShX8pz7PiPsrjQjandjV/Dq swp0yjwvdpz5yT/t0m9A3ipch9QFBO36bmH3bMQ6vrbK/03bI9ndtGk3HcUt+CUGv2F2 7FFKqJCQRBDHHM/T6hGv9/mOPoVkPgbhHCOkHeH7baMa0jQdm20o3cT5qixLB3KncO6R PG0Bd4H2+08iRagRdEibGAjJKUPpi3OQCHZgRT6N1kLDhytEbGEbN9wGxtGTlWJo2azF 6+3OTjq2j+exaotoU3IlkQ1vnmih6x7czf/DANzDU7O5zUMCifOaKTwv5MLVR5XFANRO a/8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515043; x=1712119843; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Tmp/TOSalTk1E9Om53lqcGFTwB3eFzfKle7hmp+7bao=; b=M2gnYum+Zw0FttnZ1nVv0aLd3OfkjLwSMqoxPir2ShYANUGDB93zeHljbcpd6CxxhN qwiUKsGzEW/nZZAs1TRqneyldJehy5bPxM3Z0O4AlJSfRlGyQkFusZs6wiZ0Gz4PGbgR d8QVbL1WrqT5YuL2U1VvNxv8BkfCw0JrGQ866qHcFaOENW93VBNGTKUwORs9A3+ocXfs cQW8gF8esTh5nhLe/QRXCaZx95lO+g7XTqJNHCpem1F49oDUCpHKcYETnlFgvLP0F7aS iVo07tehoi9AYxRMIgBnCtbArQSNsoxsio3EV00Q6jMRSU3I1aLYp/Hxl6uQAvkP4jo4 In5g== X-Forwarded-Encrypted: i=1; AJvYcCU1fwXuCyLEQCnDaP5HCPmE/DZKeFcy3VFhDPwM6K0y87kh3aH7BcpAyLFNRLMZuUMXvWuB9OwnfuDoghCQ25AhqN8= X-Gm-Message-State: AOJu0YwiVol+F0RLMbU6fhJLKo/ntPVdjmUdI8wyVRy8H4dGiZMQa9ba /sIEQVdzM9FaRHh7CwTemKImZp0MoziEb3FpsYnEeo3Ynast8RltpMPw6fbSfNM= X-Google-Smtp-Source: AGHT+IFl3z8IwkcLePPXROsXhoQYUv/uXEkbIlJCNYH90sGzXXkCB+mcXoO2kI0LDgwvd8XvzM/1fA== X-Received: by 2002:a05:6830:1bd5:b0:6e6:99da:8944 with SMTP id v21-20020a0568301bd500b006e699da8944mr3176474ota.26.1711515043004; Tue, 26 Mar 2024 21:50:43 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:50:42 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 02/13] riscv: Factor out page table TLB synchronization Date: Tue, 26 Mar 2024 21:49:43 -0700 Message-ID: <20240327045035.368512-3-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: D259140010 X-Stat-Signature: uchf37qxjjqmph7rk3iiauj8xkxrtq37 X-Rspam-User: X-HE-Tag: 1711515043-385632 X-HE-Meta: U2FsdGVkX1+WfjEvmkvu81HjoK+cmZAIossAwMttkc0g73K1KABNDosusz5VIy1+rAWFLwxWc25v4R6Y6Tn+smjTNkSXjJ0dTL7N/OlQYsbKYDAhIexmGmRB44fTY1D56Hq01OzWCyriYBYPGLbtfa540xouU8fCaNX8O4c7fTp1b9KD27tkwHLmSA73PUKnNyvSsflzXDUgqrfKFgLS4z1d3DV3KTsk2Efsgl1PPMfgDVfgZy5rRMs75KrTjetNFZxWjvanEWa8sQS8QDrdpbhXuWEeNyLgigo5nmgGh/GXzj1E7HyBBLzWqQgT80KBRG6fl/BdEWGGSRupRXvBemg9I4IqQNKHdFQG/QUZd+p4EdWi4foKD0cUnzfUkHzJKHc+kgYWNjn6I2ejiPb3zNN2/GSEW+tQ9EL5UYh5nmMh01LdkX38jPahu41TdVD245ffRDs38ZOj+LqkwL+j+4EWNR/pHg8vUqxMWobxLLcYUj2u9U5r4FhxJ1Y1R48g/0QjrWZtGbMCkNqfxCuv+/D9kGFCtGHdsjZcGetg8ePPlsHb6PIAI7EtVcVUOYMB65vCvZ2IsRiboj6qyN8B4W4izJEjHvWnfVCQzPz8GfjedYmOvQpMJbENirubPrhgWtgjmE73EogklmTpZSc7EhmQwPT/qgIPWahI67FYys8B46YCdAOgVbhQgaOh9ZrHUddzNnGzRlJMhVNabW4Z/9ammDrlT6cEirM4AgduLp5yzxu8+AARabB1XekePTXSNVeBAcGObmAZdZbtXEx/8AkbUTXJ/emqo4J1gFsZr06iOgOd5Jtxu459HSIV2m++MjDW+f/tI6FOANvub+d5J4f4Ni9IyrqpTCEKs/rqZtoNUcgNnvHbTxGMY3vWOdG8FJV64YTqM+Tv+aV+8YtPw822TThNdBtGYFEhYlQS5ommthlJeIFK16AH0lXYKnaTQRP4fuq+xhnA62yOe34 RoBghqGM 1dF7tHSIs9m0CXcvsYCsuymvbHKvpDh3SmfFHaKBJdECOOlTzbea8WwStx17QZg5qDTCt5Fk6yxuDsu8y8nsBveaUFAknoVTH97xA7kP/6adgKB6o59j5hSgHxz3GWnT4cxoP/3ge7uECdEt67FXHlAk37KBHt31+oESJ6mSDVOwa3f2iXqJn5+Ikt7E9PA26cDqUZv2O3/+DQBx9Mpp7m3/AIsmPaNM1kOxbu90HvIcC5rMJecVqDPLcHK9ulzFOzJTH4AZCqL3A1QLf8IhKQZJ0x4PZ4jjaR6N41wtiJKexsr7oemx097npFvfiKMy0WWbMqqAmDkVQdlCbt5lo56KHWngBvrFJCaiX4I0yUSFOG3BBfKTOKTKMXnpvnGa2utSTOLGTvFSIAqQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000008, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The logic is the same for all page table levels. See commit 69be3fb111e7 ("riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU"). Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti --- Changes in v6: - Move riscv_tlb_remove_ptdesc() definition to fix 32-bit build Changes in v5: - New patch for v5 arch/riscv/include/asm/pgalloc.h | 31 +++++++++++++------------------ 1 file changed, 13 insertions(+), 18 deletions(-) diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h index deaf971253a2..b34587da8882 100644 --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -15,6 +15,14 @@ #define __HAVE_ARCH_PUD_FREE #include +static inline void riscv_tlb_remove_ptdesc(struct mmu_gather *tlb, void *pt) +{ + if (riscv_use_ipi_for_rfence()) + tlb_remove_page_ptdesc(tlb, pt); + else + tlb_remove_ptdesc(tlb, pt); +} + static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, pte_t *pte) { @@ -102,10 +110,7 @@ static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pud, struct ptdesc *ptdesc = virt_to_ptdesc(pud); pagetable_pud_dtor(ptdesc); - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, ptdesc); - else - tlb_remove_ptdesc(tlb, ptdesc); + riscv_tlb_remove_ptdesc(tlb, ptdesc); } } @@ -139,12 +144,8 @@ static inline void p4d_free(struct mm_struct *mm, p4d_t *p4d) static inline void __p4d_free_tlb(struct mmu_gather *tlb, p4d_t *p4d, unsigned long addr) { - if (pgtable_l5_enabled) { - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, virt_to_ptdesc(p4d)); - else - tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); - } + if (pgtable_l5_enabled) + riscv_tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); } #endif /* __PAGETABLE_PMD_FOLDED */ @@ -176,10 +177,7 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, struct ptdesc *ptdesc = virt_to_ptdesc(pmd); pagetable_pmd_dtor(ptdesc); - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, ptdesc); - else - tlb_remove_ptdesc(tlb, ptdesc); + riscv_tlb_remove_ptdesc(tlb, ptdesc); } #endif /* __PAGETABLE_PMD_FOLDED */ @@ -190,10 +188,7 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, struct ptdesc *ptdesc = page_ptdesc(pte); pagetable_pte_dtor(ptdesc); - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, ptdesc); - else - tlb_remove_ptdesc(tlb, ptdesc); + riscv_tlb_remove_ptdesc(tlb, ptdesc); } #endif /* CONFIG_MMU */