From patchwork Sat Oct 28 23:12:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13439613 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC773C4167D for ; Sat, 28 Oct 2023 23:13:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3406E6B02D3; Sat, 28 Oct 2023 19:13:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C5886B02D7; Sat, 28 Oct 2023 19:13:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0A48D6B02D9; Sat, 28 Oct 2023 19:13:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E266D6B02D3 for ; Sat, 28 Oct 2023 19:13:51 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id B1E03807EE for ; Sat, 28 Oct 2023 23:13:51 +0000 (UTC) X-FDA: 81396424662.06.320044D Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf01.hostedemail.com (Postfix) with ESMTP id CE78940007 for ; Sat, 28 Oct 2023 23:13:49 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=lHjyCIKO; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf01.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698534829; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7iD5kVeuyzksz9qFDj4nk3gJEMMhm0SiL+jEgfo9gAo=; b=1JD7hTVnRgAhHqMoT/+GA+tWgJIfkYzDZo2v5Bio4qv3nxKxm5I4cBWZBosZbRoNZ4/xtm PYUNzDiQEpzpGBre/3iGHYXtgy4iMpB5/qa69gF3o5HIOo8Qg5nq/x78fCF/VbXyUvjEVj bww9TUNsKg8b+Aw3AY6RO+7oG1WYQMM= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=lHjyCIKO; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf01.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698534829; a=rsa-sha256; cv=none; b=gfF5iAgm03fed6FPcErRufdAEFBA1iN70zn69evwwRCzb0rjyYwV/T+oEXTcKP9PxaMA/p VTNFAmRMc+/AiiC81E/e4c2A7BHtw4LG4WOS+9VO+v3yiuLuBi5+HI1mkwPeEzhm+9/ZzB V10SUXuKuxihUdLYTi/77w57vFrgPh8= Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1cc30bf9e22so6051065ad.1 for ; Sat, 28 Oct 2023 16:13:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1698534829; x=1699139629; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7iD5kVeuyzksz9qFDj4nk3gJEMMhm0SiL+jEgfo9gAo=; b=lHjyCIKOt9C6qR0SC2oC7Frie1OQ3GZoXy/OZELdawDgcdbKecl9CoUhDfjaQhJYVv 5mcdXaiH5GP3L7UOIiv0lYf+KQQS9LWMciJHyAcZxc4xKSlUmsStOm8Qgn1QphwNDcPl V9YBXeC+J86xHel3WolHGIJh2RU91k0t/49Y9sUgkY5opON2g57f1JxKgjTYKitfA0YR Ea1TFxmd5NyeyQ3PHwq/drSpsPn3phLVUg8v7gKbHVVDMxGDKUT/FScwB4N6juHrItp0 k4Jh9SPDEwRnIDJIsJRJym+MZLVwV5jzYBhAyYYT6IIvTFgcPrF0M3y+oByYqgqo7y3+ RQow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698534829; x=1699139629; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7iD5kVeuyzksz9qFDj4nk3gJEMMhm0SiL+jEgfo9gAo=; b=U2wOvpz4/j2S3GZFfcV+MGAZwxnhtqqB3A/j77h50qcJkSFWguHuV/U4MhA3OuFInC qTWiB5GV+aYa0WXxdwNgWePOiICvTXc8bSQ1YcMvfMbysf5lAwlL8K9ZXe8bXOWFTVnU 9UC2IQ0ZpduwbU4qDjrheAKVAFbFY4GILx5ggc9EV8TLdNOztRxEk2I2wK8N4gRLZhGP rs8xiH8seNePtq5Y/OD9H5gA6VRueJNuJDR2GvkW4YLPjvf/yeKShwU/1PkH+RsVJMO2 Y1bupeDl6JaYQnOgsz35TNJmOyz+FPNqpwnY9f76ACKBpw5IRtvbhIA8FzcYk2qhga7z BXYA== X-Gm-Message-State: AOJu0YzWH3xvwZP3GFMIaMGB5aXNmWbLndZaDMCjy+HFgbnFGhrb4fi7 9WGR6EPypnGv5JynLSGk49fMzg== X-Google-Smtp-Source: AGHT+IGZfjM6oWIducpDDQ7hM02X3ekD2Mrbs3+cVpCJdtVQrXLcSNbo2i119+LD0HVTSm0uB+XkgQ== X-Received: by 2002:a17:903:11d1:b0:1cc:f41:8f82 with SMTP id q17-20020a17090311d100b001cc0f418f82mr8777913plh.16.1698534828754; Sat, 28 Oct 2023 16:13:48 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id u17-20020a17090341d100b001b8622c1ad2sm3679345ple.130.2023.10.28.16.13.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 28 Oct 2023 16:13:48 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , Alexandre Ghiti , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Samuel Holland Subject: [PATCH v2 05/11] riscv: mm: Combine the SMP and UP TLB flush code Date: Sat, 28 Oct 2023 16:12:03 -0700 Message-ID: <20231028231339.3116618-6-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231028231339.3116618-1-samuel.holland@sifive.com> References: <20231028231339.3116618-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: CE78940007 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: rujej8fpp4os97f386ec8z4b83jp35gm X-HE-Tag: 1698534829-568356 X-HE-Meta: U2FsdGVkX1/q7R7Svie/VHsWv5PEV8Kwix+e7+24RY4CojoFZ3PfdAl9mfbHAejKOL6jUdVTi+jIRppHvrsCWhsNqvTiJrWsaVbOYHoFh8syrPenQWLBfOe7IcQBAbCNm14tJPkrRrXd+xIBnOHISW6XtnBk2RAFbTvALW4e1jHRf6xdttUsvqjZiKAx9IqE/1n28rM466RWYRV7h1fLUulVm5yLlOubnHxj5cMPmBmzaiWroEfvI5lVeOvBmOFx9fm/m1MomGHrUkRm1ZbbMt1oaUepytSHqhYjX4FuCt30W5GWbf9O67CLm8k75JyYxgpmsFhrKr6HP5mkDSCwiUYUvFmxLJCxahg9OgDtDL+Mbr/sHcqfSmsObBj+q8jt55jFRhQzAfBf6ekTIQ4hzErXQF5arZTQCjwokOOAX4MENZ/7Qlxr7t3Uxr4iaRx7xL4tHCyNQySQA41nLSLAukwCpkrRyJ7WQqlTh1wWtkgXNnqcolGiX68BfTnvcrQq0JrZp9jIeZPYJmf20mqsHywL2SiVG0H4T5O7HqfUJSmwfwGLFTAEKL4NPVLaZyOUjxneelzPNfRK+N0Q2OWXvelpvkbUCtU3UZc1XVVsINeH5PwTZ2hovwAAkHJ5Qc6ntURdO8vbN1z3O+92E5fYvjqTLvoOHAaC4FbnMkoAIxPZz/hhJFpdfo2KJEH96yb9SpKG54k6duZD9en9sQhjqcsO+pF94aZdk/ETmCBXargKEPYRIBZkpz+ruwK4IQsm7zu8zsEB/lY4JFfECK6MMLgIwQ4XspJYIy9dc1XAhqFtrjt0foDq2XS0CBHJAVq3lGriwzghQaUG90B0QN8eyAbEp7Yk5zP+CBm0rgxLL6T8nh9WMvLbfMVFp1WYb0q0enNMcJUIwEYvOO6pp8v8jWY+a93YKMO6FYEplduydJwh76LzNEUOjOW8UtVdCQc5g2qkeHJ8iFk2xzF36JS TwrF9p/N pdJO7iuRykg70sUipA2i31poTcYwv+9IBE26cVgT0+lG7ounGvPAustEmeui6z/enaxTvGT2Wa6rGgyHSo8FH7EqrbnS7Ub7JxHHcFL1avbFsVxF3aBopVvwq3L3/SMMu8TRtxiQcCp8l3Ml3T+FNmFqSE8XQZlfBnA7sP8eJlTZPRHN4a3dQte3o8Hdetc/JPhK8ZX57wIycr4TsV1Gfui8NpNQzLZFiw1YB+pPHYkZlly0Zyb6siVbvs9mQxRJqtWLOlo5NuDo8UsWs1sLFXEdA0A2fThUG8QUh2vfR8syVApnEtmiRtaFWQohUBqf5RVc5behAth8rTwyK44jcxEM/CLW1GiLVgB9hB5tPcGVjl4fKK5k7IzfJtxnwSozcCEkWMPy+vaTwqbA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In SMP configurations, all TLB flushing narrower than flush_tlb_all() goes through __flush_tlb_range(). Do the same in UP configurations. This allows UP configurations to take advantage of recent improvements to the code in tlbflush.c, such as support for huge pages and flushing multiple-page ranges. Signed-off-by: Samuel Holland --- Changes in v2: - Move the SMP/UP merge earlier in the series to avoid build issues - Make a copy of __flush_tlb_range() instead of adding ifdefs inside - local_flush_tlb_all() is the only function used on !MMU (smpboot.c) arch/riscv/include/asm/tlbflush.h | 33 +++++++------------------------ arch/riscv/mm/Makefile | 5 +---- arch/riscv/mm/tlbflush.c | 13 ++++++++++++ 3 files changed, 21 insertions(+), 30 deletions(-) diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 8f3418c5f172..317a1811aa51 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -27,13 +27,12 @@ static inline void local_flush_tlb_page(unsigned long addr) { ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); } -#else /* CONFIG_MMU */ -#define local_flush_tlb_all() do { } while (0) -#define local_flush_tlb_page(addr) do { } while (0) -#endif /* CONFIG_MMU */ -#if defined(CONFIG_SMP) && defined(CONFIG_MMU) +#ifdef CONFIG_SMP void flush_tlb_all(void); +#else +#define flush_tlb_all() local_flush_tlb_all() +#endif void flush_tlb_mm(struct mm_struct *mm); void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, unsigned long end, unsigned int page_size); @@ -46,26 +45,8 @@ void flush_tlb_kernel_range(unsigned long start, unsigned long end); void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); #endif -#else /* CONFIG_SMP && CONFIG_MMU */ - -#define flush_tlb_all() local_flush_tlb_all() -#define flush_tlb_page(vma, addr) local_flush_tlb_page(addr) - -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - local_flush_tlb_all(); -} - -/* Flush a range of kernel pages */ -static inline void flush_tlb_kernel_range(unsigned long start, - unsigned long end) -{ - local_flush_tlb_all(); -} - -#define flush_tlb_mm(mm) flush_tlb_all() -#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() -#endif /* !CONFIG_SMP || !CONFIG_MMU */ +#else /* CONFIG_MMU */ +#define local_flush_tlb_all() do { } while (0) +#endif /* CONFIG_MMU */ #endif /* _ASM_RISCV_TLBFLUSH_H */ diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 9c454f90fd3d..64f901674e35 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -13,15 +13,12 @@ endif KCOV_INSTRUMENT_init.o := n obj-y += init.o -obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o +obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o tlbflush.o obj-y += cacheflush.o obj-y += context.o obj-y += pgtable.o obj-y += pmem.o -ifeq ($(CONFIG_MMU),y) -obj-$(CONFIG_SMP) += tlbflush.o -endif obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_KASAN) += kasan_init.o diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index e6659d7368b3..22d7ed5abf8e 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -66,6 +66,7 @@ static inline void local_flush_tlb_range_asid(unsigned long start, local_flush_tlb_range_threshold_asid(start, size, stride, asid); } +#ifdef CONFIG_SMP static void __ipi_flush_tlb_all(void *info) { local_flush_tlb_all(); @@ -138,6 +139,18 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, if (mm) put_cpu(); } +#else +static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, + unsigned long size, unsigned long stride) +{ + unsigned long asid = FLUSH_TLB_NO_ASID; + + if (mm && static_branch_unlikely(&use_asid_allocator)) + asid = atomic_long_read(&mm->context.id) & asid_mask; + + local_flush_tlb_range_asid(start, size, stride, asid); +} +#endif void flush_tlb_mm(struct mm_struct *mm) {