From patchwork Tue Aug 8 16:46:15 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 9888819 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9EC15601EB for ; Tue, 8 Aug 2017 16:51:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8FC382876C for ; Tue, 8 Aug 2017 16:51:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 84637289C2; Tue, 8 Aug 2017 16:51:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A04A42876C for ; Tue, 8 Aug 2017 16:51:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ue4VFX0M0JAbyT+6SW0yHHDamvnJy6pEfmftXYulK7E=; b=c8TXWebEVdVQvI5+6HaVXmx46q ASmvj7CVH1aPgjISMcHcCRgKGSIF86vj4fRadXAh7Jq98rWk6kTrTgn99SgMyt38yLPJTwbDGQEII kB3H9tXWT6MIuet0hDbd5u6P+VnAcKaum7Bx8S7LpxExis2Jbhmsi+gPh8KRFrD9ZTg/s9THhJ94M GzwH4m8nHtnnzpeMwqBxoKzGdB4nvYRoH+jthCXBgm2esSbgyWf2V6RHoDqnt2mElTI1aw64nauyr SyZgEgLhUcceA2w9JEqM4jspMTGMFYD0u8/6BvvNQAfEip5SESFKCgGPBn9narss4oQiE61u/sVTF m0+Tu0dg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1df7j6-0005lq-Bv; Tue, 08 Aug 2017 16:51:28 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1df7g4-0002Hj-FC for linux-arm-kernel@lists.infradead.org; Tue, 08 Aug 2017 16:48:29 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6D23F1610; Tue, 8 Aug 2017 09:48:04 -0700 (PDT) Received: from melchizedek.cambridge.arm.com (melchizedek.cambridge.arm.com [10.1.207.55]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7E6A33F540; Tue, 8 Aug 2017 09:48:02 -0700 (PDT) From: James Morse To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v2 10/11] firmware: arm_sdei: add support for CPU private events Date: Tue, 8 Aug 2017 17:46:15 +0100 Message-Id: <20170808164616.25949-11-james.morse@arm.com> X-Mailer: git-send-email 2.13.3 In-Reply-To: <20170808164616.25949-1-james.morse@arm.com> References: <20170808164616.25949-1-james.morse@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170808_094820_913092_7FD107FA X-CRM114-Status: GOOD ( 19.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , devicetree@vger.kernel.org, Marc Zyngier , Catalin Marinas , Will Deacon , Rob Herring , James Morse , Loc Ho , kvmarm@lists.cs.columbia.edu, Christoffer Dall MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Private SDE events are per-cpu, and need to be registered and enabled on each CPU. Hide this detail from the caller by adapting our {,un}register and {en,dis}able calls to send an IPI to each CPU if the event is private. CPU private events are unregistered when the CPU is powered-off, and re-registered when the CPU is brought back online. This saves bringing secondary cores back online to call private_reset() on shutdown, kexec and resume from hibernate. Signed-off-by: James Morse --- Changes since v1: * Made unregister on cpu-down print an error and continue as returning an error from here is ignored. drivers/firmware/arm_sdei.c | 241 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 227 insertions(+), 14 deletions(-) diff --git a/drivers/firmware/arm_sdei.c b/drivers/firmware/arm_sdei.c index abe032560663..67f941246711 100644 --- a/drivers/firmware/arm_sdei.c +++ b/drivers/firmware/arm_sdei.c @@ -20,9 +20,11 @@ #include #include +#include #include #include #include +#include #include #include #include @@ -65,12 +67,49 @@ struct sdei_event { u64 priority; /* This pointer is handed to firmware as the event argument. */ - struct sdei_registered_event *registered; + union { + /* Shared events */ + struct sdei_registered_event *registered; + + /* CPU private events */ + struct sdei_registered_event __percpu *private_registered; + }; }; static LIST_HEAD(sdei_events); static DEFINE_SPINLOCK(sdei_events_lock); +/* When frozen, cpu-hotplug notifiers shouldn't unregister/re-register events */ +static bool frozen; + +/* Private events are registered/enabled via IPI passing one of these */ +struct sdei_crosscall_args { + struct sdei_event *event; + atomic_t errors; + int first_error; +}; + +#define CROSSCALL_INIT(arg, event) (arg.event = event, \ + arg.first_error = 0, \ + atomic_set(&arg.errors, 0)) + +static inline int sdei_do_cross_call(void *fn, struct sdei_event *event) +{ + struct sdei_crosscall_args arg; + + CROSSCALL_INIT(arg, event); + on_each_cpu(fn, &arg, true); + + return arg.first_error; +} + +static inline void +sdei_cross_call_return(struct sdei_crosscall_args *arg, int err) +{ + if (err && (atomic_inc_return(&arg->errors) == 1)) + arg->first_error = err; +} + static int sdei_to_linux_errno(unsigned long sdei_err) { switch (sdei_err) { @@ -210,6 +249,25 @@ static struct sdei_event *sdei_event_create(u32 event_num, reg->callback = cb; reg->callback_arg = cb_arg; event->registered = reg; + } else { + int cpu; + struct sdei_registered_event __percpu *regs; + + regs = alloc_percpu(struct sdei_registered_event); + if (!regs) { + kfree(event); + return ERR_PTR(-ENOMEM); + } + + for_each_possible_cpu(cpu) { + reg = per_cpu_ptr(regs, cpu); + + reg->event_num = event->event_num; + reg->callback = cb; + reg->callback_arg = cb_arg; + } + + event->private_registered = regs; } if (sdei_event_find(event_num)) { @@ -231,6 +289,8 @@ static void sdei_event_destroy(struct sdei_event *event) if (event->type == SDEI_EVENT_TYPE_SHARED) kfree(event->registered); + else + free_percpu(event->private_registered); kfree(event); } @@ -261,11 +321,6 @@ static void _ipi_mask_cpu(void *ignored) sdei_mask_local_cpu(); } -static int sdei_cpuhp_down(unsigned int ignored) -{ - return sdei_mask_local_cpu(); -} - int sdei_unmask_local_cpu(void) { int err; @@ -287,11 +342,6 @@ static void _ipi_unmask_cpu(void *ignored) sdei_unmask_local_cpu(); } -static int sdei_cpuhp_up(unsigned int ignored) -{ - return sdei_unmask_local_cpu(); -} - static void _ipi_private_reset(void *ignored) { int err; @@ -342,6 +392,16 @@ static int sdei_api_event_enable(u32 event_num) 0, NULL); } +static void _ipi_event_enable(void *data) +{ + int err; + struct sdei_crosscall_args *arg = data; + + err = sdei_api_event_enable(arg->event->event_num); + + sdei_cross_call_return(arg, err); +} + int sdei_event_enable(u32 event_num) { int err = -EINVAL; @@ -353,6 +413,8 @@ int sdei_event_enable(u32 event_num) err = -ENOENT; else if (event->type == SDEI_EVENT_TYPE_SHARED) err = sdei_api_event_enable(event->event_num); + else + err = sdei_do_cross_call(_ipi_event_enable, event); spin_unlock(&sdei_events_lock); return err; @@ -365,6 +427,16 @@ static int sdei_api_event_disable(u32 event_num) 0, 0, NULL); } +static void _ipi_event_disable(void *data) +{ + int err; + struct sdei_crosscall_args *arg = data; + + err = sdei_api_event_disable(arg->event->event_num); + + sdei_cross_call_return(arg, err); +} + int sdei_event_disable(u32 event_num) { int err = -EINVAL; @@ -376,6 +448,9 @@ int sdei_event_disable(u32 event_num) err = -ENOENT; else if (event->type == SDEI_EVENT_TYPE_SHARED) err = sdei_api_event_disable(event->event_num); + else + err = sdei_do_cross_call(_ipi_event_disable, event); + spin_unlock(&sdei_events_lock); return err; @@ -388,6 +463,27 @@ static int sdei_api_event_unregister(u32 event_num) 0, 0, 0, NULL); } +static void _local_event_unregister(void *data) +{ + int err; + u64 result; + struct sdei_registered_event *reg; + struct sdei_crosscall_args *arg = data; + + WARN_ON_ONCE(preemptible()); + lockdep_assert_held(&sdei_events_lock); + + reg = per_cpu_ptr(arg->event->private_registered, smp_processor_id()); + err = sdei_api_event_status(reg->event_num, &result); + if (!err) { + reg->was_enabled = !!(result & BIT(SDEI_EVENT_STATUS_ENABLED)); + + err = sdei_api_event_unregister(arg->event->event_num); + } + + sdei_cross_call_return(arg, err); +} + static int _sdei_event_unregister(struct sdei_event *event) { int err; @@ -408,7 +504,7 @@ static int _sdei_event_unregister(struct sdei_event *event) return sdei_api_event_unregister(event->event_num); } - return -EINVAL; + return sdei_do_cross_call(_local_event_unregister, event); } int sdei_event_unregister(u32 event_num) @@ -467,15 +563,41 @@ static int sdei_api_event_register(u32 event_num, unsigned long entry_point, flags, affinity, NULL); } +static void _local_event_register(void *data) +{ + int err; + struct sdei_registered_event *reg; + struct sdei_crosscall_args *arg = data; + + WARN_ON(preemptible()); + lockdep_assert_held(&sdei_events_lock); + + reg = per_cpu_ptr(arg->event->private_registered, smp_processor_id()); + err = sdei_api_event_register(arg->event->event_num, sdei_entry_point, + reg, 0, 0); + + sdei_cross_call_return(arg, err); +} + static int _sdei_event_register(struct sdei_event *event) { + int err; + if (event->type == SDEI_EVENT_TYPE_SHARED) return sdei_api_event_register(event->event_num, sdei_entry_point, event->registered, SDEI_EVENT_REGISTER_RM_ANY, 0); - return -EINVAL; + get_online_cpus(); + + err = sdei_do_cross_call(_local_event_register, event); + if (err) + sdei_do_cross_call(_local_event_unregister, event); + + put_online_cpus(); + + return err; } int sdei_event_register(u32 event_num, sdei_event_callback *cb, void *arg, @@ -515,6 +637,22 @@ int sdei_event_register(u32 event_num, sdei_event_callback *cb, void *arg, } EXPORT_SYMBOL(sdei_event_register); +static void _local_event_reenable(void *data) +{ + int err = 0; + struct sdei_registered_event *reg; + struct sdei_crosscall_args *arg = data; + + WARN_ON_ONCE(preemptible()); + lockdep_assert_held(&sdei_events_lock); + + reg = per_cpu_ptr(arg->event->private_registered, smp_processor_id()); + if (reg->was_enabled) + err = sdei_api_event_enable(arg->event->event_num); + + sdei_cross_call_return(arg, err); +} + static int sdei_reregister_event(struct sdei_event *event) { int err; @@ -531,6 +669,8 @@ static int sdei_reregister_event(struct sdei_event *event) if (event->type == SDEI_EVENT_TYPE_SHARED) { if (event->registered->was_enabled) err = sdei_api_event_enable(event->event_num); + } else { + err = sdei_do_cross_call(_local_event_reenable, event); } if (err) @@ -555,6 +695,68 @@ static int sdei_reregister_events(void) return err; } +static int sdei_cpuhp_down(unsigned int cpu) +{ + struct sdei_event *event; + struct sdei_crosscall_args arg; + + if (frozen) { + /* All events unregistered */ + return sdei_mask_local_cpu(); + } + + /* un-register private events */ + spin_lock(&sdei_events_lock); + list_for_each_entry(event, &sdei_events, list) { + if (event->type == SDEI_EVENT_TYPE_SHARED) + continue; + + CROSSCALL_INIT(arg, event); + /* call the cross-call function locally... */ + _local_event_unregister(&arg); + if (arg.first_error) + pr_err("Failed to unregister event %u: %d\n", + event->event_num, arg.first_error); + } + spin_unlock(&sdei_events_lock); + + return sdei_mask_local_cpu(); +} + +static int sdei_cpuhp_up(unsigned int cpu) +{ + struct sdei_event *event; + struct sdei_crosscall_args arg; + + if (frozen) { + /* Events will be re-registered when we thaw. */ + return sdei_unmask_local_cpu(); + } + + /* re-register/enable private events */ + spin_lock(&sdei_events_lock); + list_for_each_entry(event, &sdei_events, list) { + if (event->type == SDEI_EVENT_TYPE_SHARED) + continue; + + CROSSCALL_INIT(arg, event); + /* call the cross-call function locally... */ + _local_event_register(&arg); + if (arg.first_error) + pr_err("Failed to re-register event %u: %d\n", + event->event_num, arg.first_error); + + CROSSCALL_INIT(arg, event); + _local_event_reenable(&arg); + if (arg.first_error) + pr_err("Failed to re-enable event %u: %d\n", + event->event_num, arg.first_error); + } + spin_unlock(&sdei_events_lock); + + return sdei_unmask_local_cpu(); +} + /* When entering idle, mask/unmask events for this cpu */ static int sdei_pm_notifier(struct notifier_block *nb, unsigned long action, void *data) @@ -607,6 +809,7 @@ static int sdei_device_freeze(struct device *dev) { int err; + frozen = true; err = sdei_event_unregister_all(); if (err) return err; @@ -616,9 +819,13 @@ static int sdei_device_freeze(struct device *dev) static int sdei_device_thaw(struct device *dev) { + int err; + sdei_device_resume(dev); - return sdei_reregister_events(); + err = sdei_reregister_events(); + frozen = false; + return err; } static int sdei_device_restore(struct device *dev) @@ -650,6 +857,12 @@ static int sdei_reboot_notifier(struct notifier_block *nb, unsigned long action, sdei_platform_reset(); + /* + * There is now no point trying to unregister private events if we go on + * to take CPUs offline. + */ + frozen = true; + return NOTIFY_OK; }