From patchwork Fri Mar 4 06:54:21 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bharata B Rao X-Patchwork-Id: 8499611 Return-Path: X-Original-To: patchwork-qemu-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 829F99F659 for ; Fri, 4 Mar 2016 07:00:30 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id B68222010B for ; Fri, 4 Mar 2016 07:00:29 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B5EC220108 for ; Fri, 4 Mar 2016 07:00:28 +0000 (UTC) Received: from localhost ([::1]:39162 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1abjiu-0002xQ-8x for patchwork-qemu-devel@patchwork.kernel.org; Fri, 04 Mar 2016 02:00:28 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57448) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1abjdk-0002Jr-Ho for qemu-devel@nongnu.org; Fri, 04 Mar 2016 01:55:09 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1abjdg-0005c5-CO for qemu-devel@nongnu.org; Fri, 04 Mar 2016 01:55:08 -0500 Received: from e28smtp06.in.ibm.com ([125.16.236.6]:35894) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1abjdf-0005bT-ND for qemu-devel@nongnu.org; Fri, 04 Mar 2016 01:55:04 -0500 Received: from localhost by e28smtp06.in.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 4 Mar 2016 12:25:01 +0530 Received: from d28relay04.in.ibm.com (9.184.220.61) by e28smtp06.in.ibm.com (192.168.1.136) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Fri, 4 Mar 2016 12:24:57 +0530 X-IBM-Helo: d28relay04.in.ibm.com X-IBM-MailFrom: bharata@linux.vnet.ibm.com X-IBM-RcptTo: qemu-ppc@nongnu.org;qemu-devel@nongnu.org Received: from d28av03.in.ibm.com (d28av03.in.ibm.com [9.184.220.65]) by d28relay04.in.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id u246sjFk62980218; Fri, 4 Mar 2016 12:24:45 +0530 Received: from d28av03.in.ibm.com (localhost [127.0.0.1]) by d28av03.in.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id u246snYi030597; Fri, 4 Mar 2016 12:24:51 +0530 Received: from bharata.in.ibm.com ([9.124.212.244]) by d28av03.in.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id u246sQDa029500; Fri, 4 Mar 2016 12:24:46 +0530 From: Bharata B Rao To: qemu-devel@nongnu.org Date: Fri, 4 Mar 2016 12:24:21 +0530 Message-Id: <1457074461-14285-11-git-send-email-bharata@linux.vnet.ibm.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1457074461-14285-1-git-send-email-bharata@linux.vnet.ibm.com> References: <1457074461-14285-1-git-send-email-bharata@linux.vnet.ibm.com> X-TM-AS-MML: disable x-cbid: 16030406-0021-0000-0000-00000A6F3F19 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 125.16.236.6 Cc: mjrosato@linux.vnet.ibm.com, thuth@redhat.com, pkrempa@redhat.com, ehabkost@redhat.com, aik@ozlabs.ru, Bharata B Rao , armbru@redhat.com, agraf@suse.de, borntraeger@de.ibm.com, qemu-ppc@nongnu.org, pbonzini@redhat.com, imammedo@redhat.com, mdroth@linux.vnet.ibm.com, afaerber@suse.de, david@gibson.dropbear.id.au Subject: [Qemu-devel] [RFC PATCH v1 10/10] spapr: CPU hot unplug support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Remove the CPU core device by removing the underlying CPU thread devices. Hot removal of CPU for sPAPR guests is supported by sending the hot unplug notification to the guest via EPOW interrupt. Release the vCPU object after CPU hot unplug so that vCPU fd can be parked and reused. Signed-off-by: Bharata B Rao --- hw/ppc/spapr.c | 16 ++++++++ hw/ppc/spapr_cpu_core.c | 84 +++++++++++++++++++++++++++++++++++++++++ include/hw/ppc/spapr.h | 1 + include/hw/ppc/spapr_cpu_core.h | 11 ++++++ 4 files changed, 112 insertions(+) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 6c4ac50..a42f8c0 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2379,11 +2379,27 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev, } } +void spapr_cpu_destroy(PowerPCCPU *cpu) +{ + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine()); + + xics_cpu_destroy(spapr->icp, cpu); + qemu_unregister_reset(spapr_cpu_reset, cpu); +} + static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp) { + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine()); + if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { error_setg(errp, "Memory hot unplug not supported by sPAPR"); + } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) { + if (!smc->dr_cpu_enabled) { + error_setg(errp, "CPU hot unplug not supported on this machine"); + return; + } + spapr_core_unplug(hotplug_dev, dev, errp); } } diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c index 4c233d7..5156eb3 100644 --- a/hw/ppc/spapr_cpu_core.c +++ b/hw/ppc/spapr_cpu_core.c @@ -73,6 +73,90 @@ void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev, } } +static void spapr_cpu_core_cleanup(struct sPAPRCPUUnplugList *unplug_list) +{ + sPAPRCPUUnplug *unplug, *next; + Object *cpu; + + QLIST_FOREACH_SAFE(unplug, unplug_list, node, next) { + cpu = unplug->cpu; + object_unparent(cpu); + QLIST_REMOVE(unplug, node); + g_free(unplug); + } +} + +static void spapr_add_cpu_to_unplug_list(Object *cpu, + struct sPAPRCPUUnplugList *unplug_list) +{ + sPAPRCPUUnplug *unplug = g_malloc(sizeof(*unplug)); + + unplug->cpu = cpu; + QLIST_INSERT_HEAD(unplug_list, unplug, node); +} + +static int spapr_cpu_release(Object *obj, void *opaque) +{ + DeviceState *dev = DEVICE(obj); + CPUState *cs = CPU(dev); + PowerPCCPU *cpu = POWERPC_CPU(cs); + struct sPAPRCPUUnplugList *unplug_list = opaque; + + spapr_cpu_destroy(cpu); + cpu_remove_sync(cs); + + /* + * We are still walking the core object's children list, and + * hence can't cleanup this CPU thread object just yet. Put + * it on a list for later removal. + */ + spapr_add_cpu_to_unplug_list(obj, unplug_list); + return 0; +} + +static void spapr_core_release(DeviceState *dev, void *opaque) +{ + struct sPAPRCPUUnplugList unplug_list; + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine()); + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev)); + char *slot = object_property_get_str(OBJECT(dev), CPU_CORE_SLOT_PROP, + &error_fatal); + + QLIST_INIT(&unplug_list); + object_child_foreach(OBJECT(dev), spapr_cpu_release, &unplug_list); + spapr_cpu_core_cleanup(&unplug_list); + + /* Unset the link from machine object to this core */ + object_property_set_link(OBJECT(spapr), NULL, slot, NULL); + g_free(slot); + + g_free(core->threads); + object_unparent(OBJECT(dev)); +} + +void spapr_core_unplug(HotplugHandler *hotplug_dev, DeviceState *dev, + Error **errp) +{ + sPAPRCPUCore *core = SPAPR_CPU_CORE(OBJECT(dev)); + PowerPCCPU *cpu = &core->threads[0]; + int id = ppc_get_vcpu_dt_id(cpu); + sPAPRDRConnector *drc = + spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, id); + sPAPRDRConnectorClass *drck; + Error *local_err = NULL; + + g_assert(drc); + + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc); + drck->detach(drc, dev, spapr_core_release, NULL, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + + spapr_hotplug_req_remove_by_index(drc); +} + static int spapr_cpu_core_realize_child(Object *child, void *opaque) { Error **errp = opaque; diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index d1a0af8..8fab27b 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -593,6 +593,7 @@ void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp); void spapr_cpu_reset(void *opaque); void *spapr_populate_hotplug_cpu_dt(DeviceState *dev, CPUState *cs, int *fdt_offset, sPAPRMachineState *spapr); +void spapr_cpu_destroy(PowerPCCPU *cpu); /* rtas-configure-connector state */ struct sPAPRConfigureConnectorState { diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h index 9879398..4638f96 100644 --- a/include/hw/ppc/spapr_cpu_core.h +++ b/include/hw/ppc/spapr_cpu_core.h @@ -28,4 +28,15 @@ typedef struct sPAPRCPUCore { void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp); +void spapr_core_unplug(HotplugHandler *hotplug_dev, DeviceState *dev, + Error **errp); + +/* List to store unplugged CPU objects for cleanup during unplug */ +typedef struct sPAPRCPUUnplug { + Object *cpu; + QLIST_ENTRY(sPAPRCPUUnplug) node; +} sPAPRCPUUnplug; + +QLIST_HEAD(sPAPRCPUUnplugList, sPAPRCPUUnplug); + #endif