From patchwork Wed Aug 21 14:58:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11107211 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EAD32112C for ; Wed, 21 Aug 2019 14:58:47 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C85EF22DA7 for ; Wed, 21 Aug 2019 14:58:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Sdd/qESj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C85EF22DA7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=FVokYJnOsreeAG3adsx/3rWUuYZ51bv7tsJnNYRlJ4o=; b=Sdd/qESjYpC9C2 nQLaHpUqJ6l5pe9aMODt9SxFxKtxBu9YDtzZx5sZo6Accax6CJKxtGTkYaKqcxbiPeBxFqCF+X/ET cVI7UhGiEOTdByQUB2oe1IZkdRmp5178UhMZ7MeFg2doNYCoHv7ARL0HqKo4eg3iu+mryoaIMSlFE AZFZCokiudufAT3Q30M6KLtxwDA584XvNtSu/2jRGrbb77Aw5huu2p7OQL/9hKv7y6HI9PAfNs5N6 T+oPNf9mGabB6nZDg26gtR0cTEMnpcda5U/jrn+Xe0kkkyKGjDs4a3cUDX15TEmLqiEH12UrHcQN+ 0sRpe65qMbDsrxZIyqiw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1i0S4R-00064U-AO; Wed, 21 Aug 2019 14:58:43 +0000 Received: from [182.171.81.139] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92 #3 (Red Hat Linux)) id 1i0S4P-00062A-3P; Wed, 21 Aug 2019 14:58:41 +0000 From: Christoph Hellwig To: Palmer Dabbelt , Paul Walmsley Subject: [PATCH 1/6] riscv: refactor the IPI code Date: Wed, 21 Aug 2019 23:58:32 +0900 Message-Id: <20190821145837.3686-2-hch@lst.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190821145837.3686-1-hch@lst.de> References: <20190821145837.3686-1-hch@lst.de> MIME-Version: 1.0 X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atish Patra , linux-riscv@lists.infradead.org Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org This prepares for adding native non-SBI IPI code. Signed-off-by: Christoph Hellwig Reviewed-by: Atish Patra --- arch/riscv/kernel/smp.c | 55 +++++++++++++++++++++++------------------ 1 file changed, 31 insertions(+), 24 deletions(-) diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c index 5a9834503a2f..8cd730239613 100644 --- a/arch/riscv/kernel/smp.c +++ b/arch/riscv/kernel/smp.c @@ -78,13 +78,38 @@ static void ipi_stop(void) wait_for_interrupt(); } +static void send_ipi_mask(const struct cpumask *mask, enum ipi_message_type op) +{ + int cpuid, hartid; + struct cpumask hartid_mask; + + cpumask_clear(&hartid_mask); + mb(); + for_each_cpu(cpuid, mask) { + set_bit(op, &ipi_data[cpuid].bits); + hartid = cpuid_to_hartid_map(cpuid); + cpumask_set_cpu(hartid, &hartid_mask); + } + mb(); + sbi_send_ipi(cpumask_bits(&hartid_mask)); +} + +static void send_ipi_single(int cpu, enum ipi_message_type op) +{ + send_ipi_mask(cpumask_of(cpu), op); +} + +static inline void clear_ipi(void) +{ + csr_clear(CSR_SIP, SIE_SSIE); +} + void riscv_software_interrupt(void) { unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits; unsigned long *stats = ipi_data[smp_processor_id()].stats; - /* Clear pending IPI */ - csr_clear(CSR_SIP, SIE_SSIE); + clear_ipi(); while (true) { unsigned long ops; @@ -118,23 +143,6 @@ void riscv_software_interrupt(void) } } -static void -send_ipi_message(const struct cpumask *to_whom, enum ipi_message_type operation) -{ - int cpuid, hartid; - struct cpumask hartid_mask; - - cpumask_clear(&hartid_mask); - mb(); - for_each_cpu(cpuid, to_whom) { - set_bit(operation, &ipi_data[cpuid].bits); - hartid = cpuid_to_hartid_map(cpuid); - cpumask_set_cpu(hartid, &hartid_mask); - } - mb(); - sbi_send_ipi(cpumask_bits(&hartid_mask)); -} - static const char * const ipi_names[] = { [IPI_RESCHEDULE] = "Rescheduling interrupts", [IPI_CALL_FUNC] = "Function call interrupts", @@ -156,12 +164,12 @@ void show_ipi_stats(struct seq_file *p, int prec) void arch_send_call_function_ipi_mask(struct cpumask *mask) { - send_ipi_message(mask, IPI_CALL_FUNC); + send_ipi_mask(mask, IPI_CALL_FUNC); } void arch_send_call_function_single_ipi(int cpu) { - send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC); + send_ipi_single(cpu, IPI_CALL_FUNC); } void smp_send_stop(void) @@ -176,7 +184,7 @@ void smp_send_stop(void) if (system_state <= SYSTEM_RUNNING) pr_crit("SMP: stopping secondary CPUs\n"); - send_ipi_message(&mask, IPI_CPU_STOP); + send_ipi_mask(&mask, IPI_CPU_STOP); } /* Wait up to one second for other CPUs to stop */ @@ -191,6 +199,5 @@ void smp_send_stop(void) void smp_send_reschedule(int cpu) { - send_ipi_message(cpumask_of(cpu), IPI_RESCHEDULE); + send_ipi_single(cpu, IPI_RESCHEDULE); } - From patchwork Wed Aug 21 14:58:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11107213 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A2E8A112C for ; Wed, 21 Aug 2019 14:58:49 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 80EA322DA7 for ; Wed, 21 Aug 2019 14:58:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="X/Q1NwJo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 80EA322DA7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wpTez8OKOR5TWlP/PUWdXXZAyEeuHfUNaQ/MOmkjK38=; b=X/Q1NwJoJheQKm 4/6Z9BK4s2A0EDhvbSNgEyqNQeu0YeYUnS/RmBc04oaMLz+RqvLtihVovugFyyfAEFfKNryZeGHrM VMOqjqKn4RYESurToG8/gHfIl9Yt6+7WK3oF2fbLTo9lmYdojjav/WcRjlM/hJV/YrfjxlmYBGAnu YenLRKfTTC0w+IeanYY2ndelNCBLo7S0U7uMWtH0NjOhXFcq3TODNpNHFnEg7k5WOXpbY62EVMnCy dD8K09jAXVz9XqOhjx6OPVE5WXlSXs0OX65WRaIq634P4oXeCQV6Iri/1z4syz1i1UJ5xX9WNo8kK Wr7QmOxdbfwuqXhhCnHA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1i0S4T-00067T-0P; Wed, 21 Aug 2019 14:58:45 +0000 Received: from [182.171.81.139] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92 #3 (Red Hat Linux)) id 1i0S4Q-00063D-Iu; Wed, 21 Aug 2019 14:58:42 +0000 From: Christoph Hellwig To: Palmer Dabbelt , Paul Walmsley Subject: [PATCH 2/6] riscv: cleanup send_ipi_mask Date: Wed, 21 Aug 2019 23:58:33 +0900 Message-Id: <20190821145837.3686-3-hch@lst.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190821145837.3686-1-hch@lst.de> References: <20190821145837.3686-1-hch@lst.de> MIME-Version: 1.0 X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atish Patra , linux-riscv@lists.infradead.org Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Use the special barriers for atomic bitops to make the intention a little more clear, and use riscv_cpuid_to_hartid_mask instead of open coding it. Signed-off-by: Christoph Hellwig Reviewed-by: Atish Patra --- arch/riscv/kernel/smp.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c index 8cd730239613..2e21669aa068 100644 --- a/arch/riscv/kernel/smp.c +++ b/arch/riscv/kernel/smp.c @@ -80,17 +80,15 @@ static void ipi_stop(void) static void send_ipi_mask(const struct cpumask *mask, enum ipi_message_type op) { - int cpuid, hartid; struct cpumask hartid_mask; + int cpu; - cpumask_clear(&hartid_mask); - mb(); - for_each_cpu(cpuid, mask) { - set_bit(op, &ipi_data[cpuid].bits); - hartid = cpuid_to_hartid_map(cpuid); - cpumask_set_cpu(hartid, &hartid_mask); - } - mb(); + smp_mb__before_atomic(); + for_each_cpu(cpu, mask) + set_bit(op, &ipi_data[cpu].bits); + smp_mb__after_atomic(); + + riscv_cpuid_to_hartid_mask(mask, &hartid_mask); sbi_send_ipi(cpumask_bits(&hartid_mask)); } From patchwork Wed Aug 21 14:58:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11107215 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 56D191399 for ; Wed, 21 Aug 2019 14:58:52 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 355B622DA7 for ; Wed, 21 Aug 2019 14:58:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="OV6/NLmu" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 355B622DA7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Qc77Nmwm/zTTefqBKDMzklGwuosq5Dp6p3gVdJDnBLM=; b=OV6/NLmuRQ2Etb x8dgia/FZ++OHiU3BvYGmtbTkeY9ZM1dhLHyY2eY2yeNdAk5HIfpz5mIv/CLBWWzm2HxzoIRjZ4rR 4WXWVhiDpNkkVWF6ZMr6jX5C2XayESeI11CIQ3/YJSkTO0imgDx4sL2cu1P3C09qw72U2TRLQxzGV yNMy+IacuJbZZK/yvK/i3TMoGWVhgxDtgxLnr4olu8jzguwR6o+BU+u6ISzVZBBkVCrn43WfduYd8 K4xmsI3MFvqlX41XQsqXLBuCKNTDLis6clYWFAbsWbWQCy4METK5kcvDSJVe3V0UNJ4MUt7kAA5P5 l4KssDSyIB/dwP3oQrLQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1i0S4V-0006Aa-2E; Wed, 21 Aug 2019 14:58:47 +0000 Received: from [182.171.81.139] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92 #3 (Red Hat Linux)) id 1i0S4S-00064G-36; Wed, 21 Aug 2019 14:58:44 +0000 From: Christoph Hellwig To: Palmer Dabbelt , Paul Walmsley Subject: [PATCH 3/6] riscv: optimize send_ipi_single Date: Wed, 21 Aug 2019 23:58:34 +0900 Message-Id: <20190821145837.3686-4-hch@lst.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190821145837.3686-1-hch@lst.de> References: <20190821145837.3686-1-hch@lst.de> MIME-Version: 1.0 X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atish Patra , linux-riscv@lists.infradead.org Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Don't go through send_ipi_mask, but just set the op bit an then pass a simple generate hartid mask directly to sbi_send_ipi. Signed-off-by: Christoph Hellwig Reviewed-by: Atish Patra Signed-off-by: Christoph Hellwig Reviewed-by: Atish Patra Signed-off-by: Paul Walmsley --- arch/riscv/kernel/smp.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c index 2e21669aa068..a3715d621f60 100644 --- a/arch/riscv/kernel/smp.c +++ b/arch/riscv/kernel/smp.c @@ -94,7 +94,13 @@ static void send_ipi_mask(const struct cpumask *mask, enum ipi_message_type op) static void send_ipi_single(int cpu, enum ipi_message_type op) { - send_ipi_mask(cpumask_of(cpu), op); + int hartid = cpuid_to_hartid_map(cpu); + + smp_mb__before_atomic(); + set_bit(op, &ipi_data[cpu].bits); + smp_mb__after_atomic(); + + sbi_send_ipi(cpumask_bits(cpumask_of(hartid))); } static inline void clear_ipi(void) From patchwork Wed Aug 21 14:58:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11107217 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DBDCF1399 for ; Wed, 21 Aug 2019 14:58:53 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BA84D22DA7 for ; Wed, 21 Aug 2019 14:58:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="NMJ0LrRt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BA84D22DA7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=g5PoOAvp/bwRWPZbsNkUQ8MXm02rVeUONgX7yMx0iD0=; b=NMJ0LrRtHM8gJR vWuDK6pd1HGqu3YhQ5MDes3tZx2Qh6N1CzdES5xqFmtU5oLcbQGeeRawyxe5n9jXHzAtWyUoA1aEn Z6HIfk5ajTXISPojLjHVD2n3A231D3AFqSslyqyJqXphmQoAEtwl7zWm0IkF7uoyNdUM90kUgiBr0 HURXZqEsOzoCKbvYp74n+yduD5Y8iGFoLafvHJxL6UMR6d8JLpgWhHK/tAgAH6mY4W4XsK777C9Vf c76rh3gaBmmk5eb5IQTOKrTeZlG3i6jeOc6JlM9zIOlEAGhLjOQNTMG3dSrBsPNN5B+Omsi9SeHtW YO1bFtlHTLpNmq5qN9nw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1i0S4X-0006Df-9s; Wed, 21 Aug 2019 14:58:49 +0000 Received: from [182.171.81.139] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92 #3 (Red Hat Linux)) id 1i0S4T-00067m-SV; Wed, 21 Aug 2019 14:58:46 +0000 From: Christoph Hellwig To: Palmer Dabbelt , Paul Walmsley Subject: [PATCH 4/6] riscv: cleanup riscv_cpuid_to_hartid_mask Date: Wed, 21 Aug 2019 23:58:35 +0900 Message-Id: <20190821145837.3686-5-hch@lst.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190821145837.3686-1-hch@lst.de> References: <20190821145837.3686-1-hch@lst.de> MIME-Version: 1.0 X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atish Patra , linux-riscv@lists.infradead.org Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Move the initial clearing of the mask from the callers to riscv_cpuid_to_hartid_mask, and remove the unused !CONFIG_SMP stub. Signed-off-by: Christoph Hellwig Reviewed-by: Atish Patra --- arch/riscv/include/asm/smp.h | 6 ------ arch/riscv/include/asm/tlbflush.h | 1 - arch/riscv/kernel/smp.c | 1 + arch/riscv/mm/cacheflush.c | 1 - 4 files changed, 1 insertion(+), 8 deletions(-) diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h index c6ed4d691def..a83451d73a4e 100644 --- a/arch/riscv/include/asm/smp.h +++ b/arch/riscv/include/asm/smp.h @@ -61,11 +61,5 @@ static inline unsigned long cpuid_to_hartid_map(int cpu) return boot_cpu_hartid; } -static inline void riscv_cpuid_to_hartid_mask(const struct cpumask *in, - struct cpumask *out) -{ - cpumask_set_cpu(cpuid_to_hartid_map(0), out); -} - #endif /* CONFIG_SMP */ #endif /* _ASM_RISCV_SMP_H */ diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 4d9bbe8438bf..df31fe2ed09c 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -47,7 +47,6 @@ static inline void remote_sfence_vma(struct cpumask *cmask, unsigned long start, { struct cpumask hmask; - cpumask_clear(&hmask); riscv_cpuid_to_hartid_mask(cmask, &hmask); sbi_remote_sfence_vma(hmask.bits, start, size); } diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c index a3715d621f60..3836760d7aaf 100644 --- a/arch/riscv/kernel/smp.c +++ b/arch/riscv/kernel/smp.c @@ -56,6 +56,7 @@ void riscv_cpuid_to_hartid_mask(const struct cpumask *in, struct cpumask *out) { int cpu; + cpumask_clear(out); for_each_cpu(cpu, in) cpumask_set_cpu(cpuid_to_hartid_map(cpu), out); } diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index 9ebcff8ba263..3f15938dec89 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -47,7 +47,6 @@ void flush_icache_mm(struct mm_struct *mm, bool local) cpumask_andnot(&others, mm_cpumask(mm), cpumask_of(cpu)); local |= cpumask_empty(&others); if (mm != current->active_mm || !local) { - cpumask_clear(&hmask); riscv_cpuid_to_hartid_mask(&others, &hmask); sbi_remote_fence_i(hmask.bits); } else { From patchwork Wed Aug 21 14:58:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11107219 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 060AB112C for ; Wed, 21 Aug 2019 14:58:55 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D8A5A22DA7 for ; Wed, 21 Aug 2019 14:58:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="HnZC5uJS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D8A5A22DA7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+K+fEvVnTgTRpU3xnvsafZFmhzXxRBD/8ygfYmqeqjg=; b=HnZC5uJS38HAsz B6k6xTijCTOzJTo6BHTeAHl1w/xIecHerRa77orj+hWosqbgXxKsH55RvgdoHUDZNl/VoyIt2GmfL aFzYjEuO6t/8vwd/xxicw2ZzwJnRPmbYdyk9Me0yCKktlO64ImbrJB6AT3jthY0XBFYtu8uq4uBBQ 1n4kopbNxb/draTmbTfPG/N7UFJZSTU2Jx13JCaezZ54XPRqf+6Jr6TDry7ElrDy9janXPVUSBYvr HK71+pmTJFSQMy1vrKEg/prpI0+1qMGatzsEPsgOHXnddK3OMviEXXGGVs9Y1nft1GaC6G4GYLJUo DCSnUmsgWKpAD24zYWqQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1i0S4Z-0006HG-Fv; Wed, 21 Aug 2019 14:58:51 +0000 Received: from [182.171.81.139] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92 #3 (Red Hat Linux)) id 1i0S4V-0006A3-Er; Wed, 21 Aug 2019 14:58:47 +0000 From: Christoph Hellwig To: Palmer Dabbelt , Paul Walmsley Subject: [PATCH 5/6] riscv: don't use the rdtime(h) pseudo-instructions Date: Wed, 21 Aug 2019 23:58:36 +0900 Message-Id: <20190821145837.3686-6-hch@lst.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190821145837.3686-1-hch@lst.de> References: <20190821145837.3686-1-hch@lst.de> MIME-Version: 1.0 X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atish Patra , linux-riscv@lists.infradead.org Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org If we just use the CSRs that these map to directly the code is simpler and doesn't require extra inline assembly code. Also fix up the top-level comment in timer-riscv.c to not talk about the cycle count or mention details of the clocksource interface, of which this file is just a consumer. Signed-off-by: Christoph Hellwig Reviewed-by: Atish Patra --- arch/riscv/include/asm/timex.h | 44 +++++++++++++++---------------- drivers/clocksource/timer-riscv.c | 17 +++--------- 2 files changed, 25 insertions(+), 36 deletions(-) diff --git a/arch/riscv/include/asm/timex.h b/arch/riscv/include/asm/timex.h index 6a703ec9d796..c7ef131b9e4c 100644 --- a/arch/riscv/include/asm/timex.h +++ b/arch/riscv/include/asm/timex.h @@ -6,43 +6,41 @@ #ifndef _ASM_RISCV_TIMEX_H #define _ASM_RISCV_TIMEX_H -#include +#include typedef unsigned long cycles_t; -static inline cycles_t get_cycles_inline(void) +static inline cycles_t get_cycles(void) { - cycles_t n; - - __asm__ __volatile__ ( - "rdtime %0" - : "=r" (n)); - return n; + return csr_read(CSR_TIME); } -#define get_cycles get_cycles_inline +#define get_cycles get_cycles #ifdef CONFIG_64BIT -static inline uint64_t get_cycles64(void) +static inline u64 get_cycles64(void) +{ + return get_cycles(); +} +#else /* CONFIG_64BIT */ +static inline u32 get_cycles_hi(void) { - return get_cycles(); + return csr_read(CSR_TIMEH); } -#else -static inline uint64_t get_cycles64(void) + +static inline u64 get_cycles64(void) { - u32 lo, hi, tmp; - __asm__ __volatile__ ( - "1:\n" - "rdtimeh %0\n" - "rdtime %1\n" - "rdtimeh %2\n" - "bne %0, %2, 1b" - : "=&r" (hi), "=&r" (lo), "=&r" (tmp)); + u32 hi, lo; + + do { + hi = get_cycles_hi(); + lo = get_cycles(); + } while (hi != get_cycles_hi()); + return ((u64)hi << 32) | lo; } -#endif +#endif /* CONFIG_64BIT */ #define ARCH_HAS_READ_CURRENT_TIMER - static inline int read_current_timer(unsigned long *timer_val) { *timer_val = get_cycles(); diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-riscv.c index 09e031176bc6..470c7ef02ea4 100644 --- a/drivers/clocksource/timer-riscv.c +++ b/drivers/clocksource/timer-riscv.c @@ -2,6 +2,10 @@ /* * Copyright (C) 2012 Regents of the University of California * Copyright (C) 2017 SiFive + * + * All RISC-V systems have a timer attached to every hart. These timers can be + * read from the "time" and "timeh" CSRs, and can use the SBI to setup + * events. */ #include #include @@ -12,19 +16,6 @@ #include #include -/* - * All RISC-V systems have a timer attached to every hart. These timers can be - * read by the 'rdcycle' pseudo instruction, and can use the SBI to setup - * events. In order to abstract the architecture-specific timer reading and - * setting functions away from the clock event insertion code, we provide - * function pointers to the clockevent subsystem that perform two basic - * operations: rdtime() reads the timer on the current CPU, and - * next_event(delta) sets the next timer event to 'delta' cycles in the future. - * As the timers are inherently a per-cpu resource, these callbacks perform - * operations on the current hart. There is guaranteed to be exactly one timer - * per hart on all RISC-V systems. - */ - static int riscv_clock_next_event(unsigned long delta, struct clock_event_device *ce) { From patchwork Wed Aug 21 14:58:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11107221 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 22B6A112C for ; Wed, 21 Aug 2019 14:58:58 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 00D4022DA7 for ; Wed, 21 Aug 2019 14:58:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="fZPt3Ai/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 00D4022DA7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=mWaGzVLDd20ZdJa+eEHkNFjfz5HIze3wLLY91e0Iacc=; b=fZPt3Ai/BafFk0 6lJN7vGrhsaeQIFNKkzFmKP0OWq3wlepRhIjRDlt36Bz3PNN+rPEPCojmy/S6T86Kbwpm4g+dVrZK wZD8ZYFtvw4oU1Ahesx6+DTpazCUVkqmZABtRZ0gWaHqOLw0X71AJucim0lhsWVtYlw+ntdSk3oWg ujasJiPcVZ+76bwQT8nYP0mkwZXqxwCVgaAHxgPKoJb5tV33Jyit4y5NDoXkCIIPkv3/4XiMHjwsy +x4XqtwlpFkXD/6O6VGLSe8vcEHPletrgcoV/zNN3zYaVyObbig03qtSlXBB5LEOIGEC4cSD/CC17 DfrzXkZE3p6J5P/xifsw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1i0S4b-0006KJ-K5; Wed, 21 Aug 2019 14:58:53 +0000 Received: from [182.171.81.139] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92 #3 (Red Hat Linux)) id 1i0S4W-0006CV-Vz; Wed, 21 Aug 2019 14:58:49 +0000 From: Christoph Hellwig To: Palmer Dabbelt , Paul Walmsley Subject: [PATCH 6/6] riscv: move the TLB flush logic out of line Date: Wed, 21 Aug 2019 23:58:37 +0900 Message-Id: <20190821145837.3686-7-hch@lst.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190821145837.3686-1-hch@lst.de> References: <20190821145837.3686-1-hch@lst.de> MIME-Version: 1.0 X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atish Patra , linux-riscv@lists.infradead.org Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org The TLB flush logic is going to become more complex. Start moving it out of line. Signed-off-by: Christoph Hellwig Reviewed-by: Atish Patra Signed-off-by: Christoph Hellwig Reviewed-by: Atish Patra Signed-off-by: Paul Walmsley --- arch/riscv/include/asm/tlbflush.h | 37 ++++++------------------------- arch/riscv/mm/Makefile | 3 +++ arch/riscv/mm/tlbflush.c | 35 +++++++++++++++++++++++++++++ 3 files changed, 45 insertions(+), 30 deletions(-) create mode 100644 arch/riscv/mm/tlbflush.c diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index df31fe2ed09c..075a784c66c5 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -25,8 +25,13 @@ static inline void local_flush_tlb_page(unsigned long addr) __asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory"); } -#ifndef CONFIG_SMP - +#ifdef CONFIG_SMP +void flush_tlb_all(void); +void flush_tlb_mm(struct mm_struct *mm); +void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr); +void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, + unsigned long end); +#else /* CONFIG_SMP */ #define flush_tlb_all() local_flush_tlb_all() #define flush_tlb_page(vma, addr) local_flush_tlb_page(addr) @@ -37,34 +42,6 @@ static inline void flush_tlb_range(struct vm_area_struct *vma, } #define flush_tlb_mm(mm) flush_tlb_all() - -#else /* CONFIG_SMP */ - -#include - -static inline void remote_sfence_vma(struct cpumask *cmask, unsigned long start, - unsigned long size) -{ - struct cpumask hmask; - - riscv_cpuid_to_hartid_mask(cmask, &hmask); - sbi_remote_sfence_vma(hmask.bits, start, size); -} - -#define flush_tlb_all() sbi_remote_sfence_vma(NULL, 0, -1) - -#define flush_tlb_range(vma, start, end) \ - remote_sfence_vma(mm_cpumask((vma)->vm_mm), start, (end) - (start)) - -static inline void flush_tlb_page(struct vm_area_struct *vma, - unsigned long addr) -{ - flush_tlb_range(vma, addr, addr + PAGE_SIZE); -} - -#define flush_tlb_mm(mm) \ - remote_sfence_vma(mm_cpumask(mm), 0, -1) - #endif /* CONFIG_SMP */ /* Flush a range of kernel pages */ diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 74055e1d6f21..9d9a17335686 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -13,4 +13,7 @@ obj-y += cacheflush.o obj-y += context.o obj-y += sifive_l2_cache.o +ifeq ($(CONFIG_MMU),y) +obj-$(CONFIG_SMP) += tlbflush.o +endif obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c new file mode 100644 index 000000000000..df93b26f1b9d --- /dev/null +++ b/arch/riscv/mm/tlbflush.c @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include + +void flush_tlb_all(void) +{ + sbi_remote_sfence_vma(NULL, 0, -1); +} + +static void __sbi_tlb_flush_range(struct cpumask *cmask, unsigned long start, + unsigned long size) +{ + struct cpumask hmask; + + riscv_cpuid_to_hartid_mask(cmask, &hmask); + sbi_remote_sfence_vma(hmask.bits, start, size); +} + +void flush_tlb_mm(struct mm_struct *mm) +{ + __sbi_tlb_flush_range(mm_cpumask(mm), 0, -1); +} + +void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr) +{ + __sbi_tlb_flush_range(mm_cpumask(vma->vm_mm), addr, PAGE_SIZE); +} + +void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, + unsigned long end) +{ + __sbi_tlb_flush_range(mm_cpumask(vma->vm_mm), start, end - start); +}