From patchwork Thu Dec 17 13:39:29 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pratyush Anand X-Patchwork-Id: 7873571 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 5DED99F387 for ; Thu, 17 Dec 2015 13:42:46 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 487B12042A for ; Thu, 17 Dec 2015 13:42:45 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3E0FE201E4 for ; Thu, 17 Dec 2015 13:42:44 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1a9Yo1-0000Zy-QR; Thu, 17 Dec 2015 13:41:17 +0000 Received: from mx1.redhat.com ([209.132.183.28]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1a9Yny-0000PV-IV for linux-arm-kernel@lists.infradead.org; Thu, 17 Dec 2015 13:41:15 +0000 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 8DEE6691; Thu, 17 Dec 2015 13:40:53 +0000 (UTC) Received: from localhost (vpn-55-92.rdu2.redhat.com [10.10.55.92]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id tBHDepmO008894; Thu, 17 Dec 2015 08:40:52 -0500 From: Pratyush Anand To: linux-arm-kernel@lists.infradead.org Subject: [PATCH RFC] ARM64: Add cpu hotplug for device tree parking method Date: Thu, 17 Dec 2015 19:09:29 +0530 Message-Id: <49d1ca9f7ca09707cdefbebe6f0163c376b1eaea.1450358510.git.panand@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20151217_054114_687085_BD3BD30D X-CRM114-Status: GOOD ( 27.37 ) X-Spam-Score: -6.9 (------) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, Pratyush Anand , lorenzo.pieralisi@arm.com, Ard Biesheuvel , Marc Zyngier , catalin.marinas@arm.com, Sudeep Holla , will.deacon@arm.com, open list , "David A. Long" , Paul Walmsley , msalter@redhat.com, Sandeepa Prabhu , Hanjun Guo MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch has been implemented for CPU hotplug support for device tree spin-table based parking method. While cpu is offlined, cpu_die is called and when it is brought online cpu_boot is called. So, cpu_boot must wake secondary and release pen, otherwise dynamic cpu offline/online will not work. Signed-off-by: Pratyush Anand --- Hi, Actually this patch is using some infrastructure from Geoff's kexec-v12.3. But I am sending this patch for your review and feedback in advance. This patch is needed for kexec and cpu hotplug to work on a system with device tree spin-table method. Have tested this patch with kexec and also with cpu offline from sys interface. # lscpu Architecture: aarch64 Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Thread(s) per core: 1 Core(s) per socket: 2 Socket(s): 4 # echo 0 > /sys/bus/cpu/devices/cpu3/online # lscpu Architecture: aarch64 Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-2,4-7 Off-line CPU(s) list: 3 Thread(s) per core: 1 Core(s) per socket: 1 Socket(s): 4 # echo 1 > /sys/bus/cpu/devices/cpu3/online # lscpu Architecture: aarch64 Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Thread(s) per core: 1 Core(s) per socket: 2 Socket(s): 4 cpu-park infrastructure of this patch can further be shared by ACPI parking protocol support for providing CPU hotplug support. ~Pratyush arch/arm64/kernel/Makefile | 2 +- arch/arm64/kernel/cpu-park.S | 54 ++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/cpu-park.h | 25 ++++++++++++++++++ arch/arm64/kernel/smp_spin_table.c | 40 +++++++++++++++++++++++----- 4 files changed, 113 insertions(+), 8 deletions(-) create mode 100644 arch/arm64/kernel/cpu-park.S create mode 100644 arch/arm64/kernel/cpu-park.h diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile index a08b0545bffa..f229f3d4b455 100644 --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -17,7 +17,7 @@ arm64-obj-y := debug-monitors.o entry.o irq.o fpsimd.o \ hyp-stub.o psci.o psci-call.o cpu_ops.o insn.o \ return_address.o cpuinfo.o cpu_errata.o \ cpufeature.o alternative.o cacheinfo.o \ - smp.o smp_spin_table.o topology.o + smp.o smp_spin_table.o cpu-park.o topology.o extra-$(CONFIG_EFI) := efi-entry.o diff --git a/arch/arm64/kernel/cpu-park.S b/arch/arm64/kernel/cpu-park.S new file mode 100644 index 000000000000..7e80ecf24f28 --- /dev/null +++ b/arch/arm64/kernel/cpu-park.S @@ -0,0 +1,54 @@ +/* + * cpu park routines + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ + +#include +#include +#include +#include + +.text +.pushsection .idmap.text, "ax" + +/* + * __cpu_park(el2_switch, park_address) - Helper for cpu_park + * + * @el2_switch: Flag to indicate a swich to EL2 is needed, passed to cpu_park. + * @park_address - where cpu will keep on looking for address to jump + * + * Put the CPU into the wfe and check for valid none zero secondary address + * at parked address when a event is received. If secondary address is + * valid then jump to it. + */ + +ENTRY(__cpu_park) + /* Clear sctlr_el1 flags. */ + mrs x12, sctlr_el1 + ldr x13, =SCTLR_ELx_FLAGS + bic x12, x12, x13 + msr sctlr_el1, x12 + isb +1: + wfe + ldr x3, [x1] // get entry address + cmp x3, #0 + b.eq 1b + + mov x2, 0 + str x2, [x1] + + cbz x0, 2f // el2_switch? + + mov x0, x3 // entry + hvc #HVC_CALL_FUNC // no return + +2: + ret x3 + +ENDPROC(__cpu_park) + +.popsection diff --git a/arch/arm64/kernel/cpu-park.h b/arch/arm64/kernel/cpu-park.h new file mode 100644 index 000000000000..356438d21360 --- /dev/null +++ b/arch/arm64/kernel/cpu-park.h @@ -0,0 +1,25 @@ +/* + * cpu park routines + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ + +#ifndef _ARM64_CPU_PARK_H +#define _ARM64_CPU_PARK_H + +#include + +void __cpu_park(unsigned long el2_switch, unsigned long park_address); + +static inline void __noreturn cpu_park(unsigned long el2_switch, + unsigned long park_address) +{ + typeof(__cpu_park) *park_fn; + park_fn = (void *)virt_to_phys(__cpu_park); + park_fn(el2_switch, park_address); + unreachable(); +} + +#endif diff --git a/arch/arm64/kernel/smp_spin_table.c b/arch/arm64/kernel/smp_spin_table.c index aef3605a8c47..9411b9f59f9e 100644 --- a/arch/arm64/kernel/smp_spin_table.c +++ b/arch/arm64/kernel/smp_spin_table.c @@ -26,8 +26,11 @@ #include #include #include +#include #include +#include "cpu-park.h" + extern void secondary_holding_pen(void); volatile unsigned long secondary_holding_pen_release = INVALID_HWID; @@ -73,11 +76,16 @@ static int smp_spin_table_cpu_init(unsigned int cpu) static int smp_spin_table_cpu_prepare(unsigned int cpu) { - __le64 __iomem *release_addr; - if (!cpu_release_addr[cpu]) return -ENODEV; + return 0; +} + +static int smp_spin_table_cpu_boot(unsigned int cpu) +{ + __le64 __iomem *release_addr; + /* * The cpu-release-addr may or may not be inside the linear mapping. * As ioremap_cache will either give us a new mapping or reuse the @@ -107,11 +115,6 @@ static int smp_spin_table_cpu_prepare(unsigned int cpu) iounmap(release_addr); - return 0; -} - -static int smp_spin_table_cpu_boot(unsigned int cpu) -{ /* * Update the pen release flag. */ @@ -125,9 +128,32 @@ static int smp_spin_table_cpu_boot(unsigned int cpu) return 0; } +#ifdef CONFIG_HOTPLUG_CPU +static int smp_spin_table_cpu_disable(unsigned int cpu) +{ + if (!cpu_release_addr[cpu]) + return -EOPNOTSUPP; + + return 0; +} + +static void smp_spin_table_cpu_die(unsigned int cpu) +{ + setup_mm_for_reboot(); + cpu_park(in_crash_kexec ? 0 : is_hyp_mode_available(), + cpu_release_addr[cpu]); + + pr_crit("unable to power off CPU%u\n", cpu); +} +#endif + const struct cpu_operations smp_spin_table_ops = { .name = "spin-table", .cpu_init = smp_spin_table_cpu_init, .cpu_prepare = smp_spin_table_cpu_prepare, .cpu_boot = smp_spin_table_cpu_boot, +#ifdef CONFIG_HOTPLUG_CPU + .cpu_disable = smp_spin_table_cpu_disable, + .cpu_die = smp_spin_table_cpu_die, +#endif };