From patchwork Thu Feb 11 22:52:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 12084287 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE81DC433E0 for ; Thu, 11 Feb 2021 22:55:42 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4ECD264E42 for ; Thu, 11 Feb 2021 22:55:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4ECD264E42 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:38516 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lAKs9-0002mZ-8y for qemu-devel@archiver.kernel.org; Thu, 11 Feb 2021 17:55:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41248) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lAKph-00014H-3p; Thu, 11 Feb 2021 17:53:09 -0500 Received: from mail-qk1-x72f.google.com ([2607:f8b0:4864:20::72f]:37854) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lAKpa-0002Uo-8r; Thu, 11 Feb 2021 17:53:08 -0500 Received: by mail-qk1-x72f.google.com with SMTP id s77so7055986qke.4; Thu, 11 Feb 2021 14:53:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5U9+y/dSkI4cN7W0ikuvTZLCpAHezOk1LxCyFVMnoqM=; b=e2RsSizrN0VC6NOo0NaAIAWSka4Qf98qa0Y/LGjbixhDYAnmD7uJcFQzZ1hZksL59l Du+VTd0ewRGxEq+4wxe7R/AakNCpzVJEZ2Oi1S7uo+Ta1WxRcod/QMrHNiaOTh3b0EwN aq/9pxxVwlO4k/Z/9m8PsS5JlHFn296bwLUsmuEYMkuGh7x7O6pCTQ4f0pxbs0lEo87M G3pkIrIPdniH0LYdBbUfWHEA2hk3SzNsk/7alPJ7lDtbkgOkP9yGHY00TvAeDX78TNxF XiP4vlRug21qUba+FYoMfyX6Mf7B/hc8J+0oZjmgSgMIkmmWx5kq3ztGu2MyDBtttQRx a43A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5U9+y/dSkI4cN7W0ikuvTZLCpAHezOk1LxCyFVMnoqM=; b=B0hrbR4ECakgIChvC/GkFSbAi/+3eR5j4jkb3TvGGXB4QBk5fWQTdNPu7qgXhPLmMa SRNOGP9au+GJkOIsIipEZQ+VBGUmFLsglr/QCDOmSc6fLFWespze1JE+eA08OcP1641W jSuGUXAeCPjBt7TpnZVaimbPOg7X/vN+i6jH9Mui0EQfMhjIZ5GVQNaxWlnR+l9jJzzJ wiEuUBZgINX+6290oE1vOK0zj2c4XGBoHlg4k3t4xtGuZDjjpZbey8ik2U1YWb8toYAY ekPTU/Tf95cOC4Q2IteYo82wL8JK5IHNX0R4vS0bGOUPDr2ubnAZ+tzd+ZR2EcyqPRDV Yvyw== X-Gm-Message-State: AOAM533Qp9XGEbkTlX4TaJpT+gFVvAcCqa9j9pDRNPuBACn4+mu2BzeJ mOFkZmhcNI9i4BxKFGorC0XuEimWnAuwow== X-Google-Smtp-Source: ABdhPJw0d+p8pQVAIbF+bnhXa7HnjLMS9uW0w1AxhgdkXbf2CmG1IFDht0S1jtU5qg7E1TTi2tQInA== X-Received: by 2002:ae9:dc87:: with SMTP id q129mr50162qkf.297.1613083980726; Thu, 11 Feb 2021 14:53:00 -0800 (PST) Received: from rekt.ibmuc.com ([2804:431:c7c6:6e83:c920:e8d3:a2df:bf3]) by smtp.gmail.com with ESMTPSA id x49sm4662227qth.95.2021.02.11.14.52.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Feb 2021 14:53:00 -0800 (PST) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [PATCH v3 1/7] spapr_drc.c: do not call spapr_drc_detach() in drc_isolate_logical() Date: Thu, 11 Feb 2021 19:52:40 -0300 Message-Id: <20210211225246.17315-2-danielhb413@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210211225246.17315-1-danielhb413@gmail.com> References: <20210211225246.17315-1-danielhb413@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::72f; envelope-from=danielhb413@gmail.com; helo=mail-qk1-x72f.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" drc_isolate_logical() is used to move the DRC from the "Configured" to the "Available" state, erroring out if the DRC is in the unexpected "Unisolate" state and doing nothing (with RTAS_OUT_SUCCESS) if the DRC is already in "Available" or in "Unusable" state. When moving from "Configured" to "Available", the DRC is moved to the LOGICAL_AVAILABLE state, a drc->unplug_requested check is done and, if true, spapr_drc_detach() is called. What spapr_drc_detach() does then is: - set drc->unplug_requested to true. In fact, this is the only place where unplug_request is set to true; - does nothing else if drc->state != drck->empty_state. If the DRC state is equal to drck->empty_state, spapr_drc_release() is called. For logical DRCs, drck->empty_state = LOGICAL_UNUSABLE. In short, calling spapr_drc_detach() in drc_isolate_logical() does nothing. It'll set unplug_request to true again ('again' since it was already true - otherwise the function wouldn't be called), and will return without calling spapr_drc_release() because the DRC is not in LOGICAL_UNUSABLE, since drc_isolate_logical() just moved it to LOGICAL_AVAILABLE. The only place where the logical DRC is released is when called from drc_set_unusable(), when it is moved to the "Unusable" state. As it should, according to PAPR. Even though calling spapr_drc_detach() in drc_isolate_logical() is benign, removing it will avoid further thought about the matter. So let's go ahead and do that. As a note, this logic was introduced in commit bbf5c878ab76. Since then, the DRC handling code was refactored and enhanced, and PAPR itself went through some changes in the DRC area as well. It is expected that some assumptions we had back then are now deprecated. Signed-off-by: Daniel Henrique Barboza Reviewed-by: Greg Kurz --- hw/ppc/spapr_drc.c | 13 ------------- 1 file changed, 13 deletions(-) diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c index 8571d5bafe..84bd3c881f 100644 --- a/hw/ppc/spapr_drc.c +++ b/hw/ppc/spapr_drc.c @@ -132,19 +132,6 @@ static uint32_t drc_isolate_logical(SpaprDrc *drc) drc->state = SPAPR_DRC_STATE_LOGICAL_AVAILABLE; - /* if we're awaiting release, but still in an unconfigured state, - * it's likely the guest is still in the process of configuring - * the device and is transitioning the devices to an ISOLATED - * state as a part of that process. so we only complete the - * removal when this transition happens for a device in a - * configured state, as suggested by the state diagram from PAPR+ - * 2.7, 13.4 - */ - if (drc->unplug_requested) { - uint32_t drc_index = spapr_drc_index(drc); - trace_spapr_drc_set_isolation_state_finalizing(drc_index); - spapr_drc_detach(drc); - } return RTAS_OUT_SUCCESS; } From patchwork Thu Feb 11 22:52:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 12084319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13330C433E6 for ; Thu, 11 Feb 2021 22:58:26 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 808CF64E3D for ; Thu, 11 Feb 2021 22:58:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 808CF64E3D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:46760 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lAKum-0006pM-AE for qemu-devel@archiver.kernel.org; Thu, 11 Feb 2021 17:58:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41254) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lAKph-00014q-Jw; Thu, 11 Feb 2021 17:53:09 -0500 Received: from mail-qt1-x82c.google.com ([2607:f8b0:4864:20::82c]:34106) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lAKpc-0002Vk-Au; Thu, 11 Feb 2021 17:53:09 -0500 Received: by mail-qt1-x82c.google.com with SMTP id c1so5482148qtc.1; Thu, 11 Feb 2021 14:53:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tCoYYHBKR+fFO7m++NZSpiMZ5sCIJgLWAYFJvHuEtbM=; b=YKJGVbBQMaUqRL3DGBeYL2qMOGSm7rhba0KFKrmAIVtRmgZwSFzbtf9FWaYVzMrunU 08Ibwb+IjM3zzc7+26wJ1cjM+WSwHYyMyXz1eR7qfNRUgHAnbKMw0Mz8kQTxOqycd7xF rtb4EtD+X/9txKWX9AMMTb+2+S0yqqPxS3k50S3Y6aoMWypOO2wCq7nWeDIYBCMYHyoG usgnnrHSzb/ytssZP2E+C9v/OqV5YpNB1vb9HdLT8E74TV3CVz28gn/9Kj8QQQ5P3RHk EifpZs4v77Qca2dsYlsQ/DfB3VjYRXnQcT4ugJ9HaefZqkMVyFhRmS8E+JNQ+qtDw4mi LQIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tCoYYHBKR+fFO7m++NZSpiMZ5sCIJgLWAYFJvHuEtbM=; b=M1DSWpWrgVVtXGFW7GHUWcDGZAYvyUlanHMn5Ayq2RWVn2aLOZMKh7fJzifE8qlYVS rSiYXxkvdTj5iCcUo5xYDJcIjOVFv+NoWjBjxc3q3ytNLLQnwY/G6Y1PKo138T20qo9f MjsYvD7o/+3TXkyxR0JZF3tXCVeBIpiI6v6+cf/grMbi+1J8aJjCoZbZOwhmFpoEV4f1 0oC+c2SelKZ7gKkb+ehKylZzKhLAN2NsZk3/vb0wmvba8VDtzD/FVl08dAuKx0BShvBC cs4b0AW5CcNUcqOD0btTksy+URXGkYIl/Ht6BGGqoHmQiOx3+114Uzn90m8xL0/AHiy9 SEJw== X-Gm-Message-State: AOAM5306XiIz6vTGd6Nm83rcGzchrXKUdmsUbTC5gMoCXlwi0+qUeKPp fo/GoGiumFoq+ks0t0axolfixAYGT2k2iA== X-Google-Smtp-Source: ABdhPJwFSWef5fomf+AIjyA9WiiaeBV+PI75OKZFxe7l/qMrahkBtz5PBscUm3FeWwmoXRsY/7pBdQ== X-Received: by 2002:a05:622a:7:: with SMTP id x7mr9554700qtw.257.1613083982884; Thu, 11 Feb 2021 14:53:02 -0800 (PST) Received: from rekt.ibmuc.com ([2804:431:c7c6:6e83:c920:e8d3:a2df:bf3]) by smtp.gmail.com with ESMTPSA id x49sm4662227qth.95.2021.02.11.14.53.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Feb 2021 14:53:02 -0800 (PST) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [PATCH v3 2/7] spapr_pci.c: simplify spapr_pci_unplug_request() function handling Date: Thu, 11 Feb 2021 19:52:41 -0300 Message-Id: <20210211225246.17315-3-danielhb413@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210211225246.17315-1-danielhb413@gmail.com> References: <20210211225246.17315-1-danielhb413@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::82c; envelope-from=danielhb413@gmail.com; helo=mail-qt1-x82c.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" When hotunplugging a PCI function we'll branch out the logic in two cases, function zero and non-zero. If non-zero, we'll call spapr_drc_detach() and nothing else. If it's function zero, we'll loop it once between all the functions in the slot to call spapr_drc_detach() on them, and afterwards we'll do another backwards loop where we'll signal the event to the guest. We can simplify this logic. We can ignore all the DRC handling for non-zero functions, since we'll end up doing that regardless when unplugging function zero. And for function zero, everything can be done in a single loop, since tt doesn't matter if we end up marking the function DRCs as unplug pending in backwards order or not, as long as we call spapr_drc_detach() before issuing the hotunplug event to the guest. This will also avoid a possible scenario where the user starts to hotunplug the slot, starting with a non-zero function, and then delays/forgets to hotunplug function zero afterwards. This would keep the function DRC marked as unplug requested indefinitely. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr_pci.c | 44 ++++++++++++++++---------------------------- 1 file changed, 16 insertions(+), 28 deletions(-) diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c index f1c7479816..1791d98a49 100644 --- a/hw/ppc/spapr_pci.c +++ b/hw/ppc/spapr_pci.c @@ -1709,38 +1709,26 @@ static void spapr_pci_unplug_request(HotplugHandler *plug_handler, return; } - /* ensure any other present functions are pending unplug */ - if (PCI_FUNC(pdev->devfn) == 0) { - for (i = 1; i < 8; i++) { - func_drc = drc_from_devfn(phb, chassis, PCI_DEVFN(slotnr, i)); - func_drck = SPAPR_DR_CONNECTOR_GET_CLASS(func_drc); - state = func_drck->dr_entity_sense(func_drc); - if (state == SPAPR_DR_ENTITY_SENSE_PRESENT - && !spapr_drc_unplug_requested(func_drc)) { - /* - * Attempting to remove function 0 of a multifunction - * device will will cascade into removing all child - * functions, even if their unplug weren't requested - * beforehand. - */ - spapr_drc_detach(func_drc); - } - } + /* + * The hotunplug itself will occur when unplugging function 0, + * regardless of marking any other functions DRCs as pending + * unplug beforehand (since 02a1536eee33). + */ + if (PCI_FUNC(pdev->devfn) != 0) { + return; } - spapr_drc_detach(drc); + for (i = 7; i >= 0; i--) { + func_drc = drc_from_devfn(phb, chassis, PCI_DEVFN(slotnr, i)); + func_drck = SPAPR_DR_CONNECTOR_GET_CLASS(func_drc); + state = func_drck->dr_entity_sense(func_drc); - /* if this isn't func 0, defer unplug event. otherwise signal removal - * for all present functions - */ - if (PCI_FUNC(pdev->devfn) == 0) { - for (i = 7; i >= 0; i--) { - func_drc = drc_from_devfn(phb, chassis, PCI_DEVFN(slotnr, i)); - func_drck = SPAPR_DR_CONNECTOR_GET_CLASS(func_drc); - state = func_drck->dr_entity_sense(func_drc); - if (state == SPAPR_DR_ENTITY_SENSE_PRESENT) { - spapr_hotplug_req_remove_by_index(func_drc); + if (state == SPAPR_DR_ENTITY_SENSE_PRESENT) { + /* Mark the DRC as requested unplug if needed. */ + if (!spapr_drc_unplug_requested(func_drc)) { + spapr_drc_detach(func_drc); } + spapr_hotplug_req_remove_by_index(func_drc); } } } From patchwork Thu Feb 11 22:52:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 12084347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47206C433DB for ; Thu, 11 Feb 2021 23:02:33 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CD62A64DD5 for ; Thu, 11 Feb 2021 23:02:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CD62A64DD5 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:54326 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lAKyl-0002TU-St for qemu-devel@archiver.kernel.org; Thu, 11 Feb 2021 18:02:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41272) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lAKpi-00015y-IV; Thu, 11 Feb 2021 17:53:10 -0500 Received: from mail-qv1-xf2c.google.com ([2607:f8b0:4864:20::f2c]:38793) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lAKpe-0002WR-RM; Thu, 11 Feb 2021 17:53:10 -0500 Received: by mail-qv1-xf2c.google.com with SMTP id p12so3394164qvv.5; Thu, 11 Feb 2021 14:53:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7fu0/1f7dpBfw92H2jE4xhexKk4zLuJqVoIdhtO1AYI=; b=TZRd+Dj+OQNhG+7udC0J6A2aUB425FaEuNkALvHtw/C88dW3hnqTeprMgZRD6PmYES x6TUAaTreNMvusgS0A1+gqlJB4aSvDLNfycFdYeSldVSZIuxEKuqhIfwV6MKWn8j/Yff QJnJiFGUPBU7spHSLwA1Ig7jztORR9eGPf9d52+46yYaPxqQj5a++8nYYzdhjCMoXQGF /CUBx+dLM4vf2NS/R95Q4TJIVg5dpktaBb6EivvQ6K6OSCa5avvkl8PAQ/n3q0v92qwx rEuIJ7BMmMgMrfjPcgEOZIvG009eG29D65i6PrIOR7fwI3fsOR7jCclqf3lrgwd+Jin2 lNjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7fu0/1f7dpBfw92H2jE4xhexKk4zLuJqVoIdhtO1AYI=; b=uk8xzDvtgq4eFh0aYeehYvieITVZ3UDXaxiFwpZD1vI+UM+XkFhqfaxgH3iHQD4Weg P+X89yTY8z/TO7x+5QPxk5sa/2fAJr7eV7CmupMF2mpjkscGXxlwMJj3Iuh5uwj9pksk kQAQ335ovsana+cLwfX7w873GrCAMy9X9+GLhiR+/DcR7iCqVTvZ7OPbU0cayfoZ9PSu SQPF83m6irjFWFdRPVAOhD6KAvfA2oAPcNPocBRPNcaCRddagRW/dl76NVFk12nM0w3u Ta5uMVBpFXimGS+5NZ/mCUpIQVHvR/oAH1IgG0lRvFe3qgJdEoAjNLCf4nGULwvAHmXE m7SA== X-Gm-Message-State: AOAM531brrqrzL/EeW2Yyk9/bUy+VKEei2oblKFkp14zfGmS35AG56tW TxG3RZUJN9gEGIQGv8wj5r33NMgzQeGwUg== X-Google-Smtp-Source: ABdhPJxICUPl55awWLMqCuXw2Rv+DA2L4LkrzL/M1VU1W+HZ2271A1GuUEEH8L5QeMifH8ZfAZjuFw== X-Received: by 2002:ad4:4e14:: with SMTP id dl20mr100923qvb.51.1613083984935; Thu, 11 Feb 2021 14:53:04 -0800 (PST) Received: from rekt.ibmuc.com ([2804:431:c7c6:6e83:c920:e8d3:a2df:bf3]) by smtp.gmail.com with ESMTPSA id x49sm4662227qth.95.2021.02.11.14.53.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Feb 2021 14:53:04 -0800 (PST) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [PATCH v3 3/7] spapr_drc.c: use spapr_drc_release() in isolate_physical/set_unusable Date: Thu, 11 Feb 2021 19:52:42 -0300 Message-Id: <20210211225246.17315-4-danielhb413@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210211225246.17315-1-danielhb413@gmail.com> References: <20210211225246.17315-1-danielhb413@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f2c; envelope-from=danielhb413@gmail.com; helo=mail-qv1-xf2c.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" When moving a physical DRC to "Available", drc_isolate_physical() will move the DRC state to STATE_PHYSICAL_POWERON and, if the DRC is marked for unplug, call spapr_drc_detach(). For physical DRCs, drck->empty_state is STATE_PHYSICAL_POWERON, meaning that we're sure that spapr_drc_detach() will end up calling spapr_drc_release() in the end. Likewise, for logical DRCs, drc_set_unusable will move the DRC to "Unusable" state, setting drc->state to STATE_LOGICAL_UNUSABLE, which is the drck->empty_state for logical DRCs. spapr_drc_detach() will call spapr_drc_release() in this case as well. In both scenarios, spapr_drc_detach() is being used as a spapr_drc_release(), wrapper, where we also set unplug_requested (which is already true, otherwise spapr_drc_detach() wouldn't be called in the first place) and check if drc->state == drck->empty_state, which we also know it's guaranteed to be true because we just set it. Just use spapr_drc_release() in these functions to be clear of our intentions in both these functions. Signed-off-by: Daniel Henrique Barboza Reviewed-by: David Gibson Reviewed-by: Greg Kurz --- hw/ppc/spapr_drc.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c index 84bd3c881f..555a25517d 100644 --- a/hw/ppc/spapr_drc.c +++ b/hw/ppc/spapr_drc.c @@ -50,6 +50,20 @@ uint32_t spapr_drc_index(SpaprDrc *drc) | (drc->id & DRC_INDEX_ID_MASK); } +static void spapr_drc_release(SpaprDrc *drc) +{ + SpaprDrcClass *drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc); + + drck->release(drc->dev); + + drc->unplug_requested = false; + g_free(drc->fdt); + drc->fdt = NULL; + drc->fdt_start_offset = 0; + object_property_del(OBJECT(drc), "device"); + drc->dev = NULL; +} + static uint32_t drc_isolate_physical(SpaprDrc *drc) { switch (drc->state) { @@ -68,7 +82,7 @@ static uint32_t drc_isolate_physical(SpaprDrc *drc) if (drc->unplug_requested) { uint32_t drc_index = spapr_drc_index(drc); trace_spapr_drc_set_isolation_state_finalizing(drc_index); - spapr_drc_detach(drc); + spapr_drc_release(drc); } return RTAS_OUT_SUCCESS; @@ -209,7 +223,7 @@ static uint32_t drc_set_unusable(SpaprDrc *drc) if (drc->unplug_requested) { uint32_t drc_index = spapr_drc_index(drc); trace_spapr_drc_set_allocation_state_finalizing(drc_index); - spapr_drc_detach(drc); + spapr_drc_release(drc); } return RTAS_OUT_SUCCESS; @@ -372,20 +386,6 @@ void spapr_drc_attach(SpaprDrc *drc, DeviceState *d) NULL, 0); } -static void spapr_drc_release(SpaprDrc *drc) -{ - SpaprDrcClass *drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc); - - drck->release(drc->dev); - - drc->unplug_requested = false; - g_free(drc->fdt); - drc->fdt = NULL; - drc->fdt_start_offset = 0; - object_property_del(OBJECT(drc), "device"); - drc->dev = NULL; -} - void spapr_drc_detach(SpaprDrc *drc) { SpaprDrcClass *drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc); From patchwork Thu Feb 11 22:52:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 12084289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EA08C433DB for ; Thu, 11 Feb 2021 22:55:52 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DA48564E42 for ; Thu, 11 Feb 2021 22:55:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DA48564E42 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:38852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lAKsI-0002vD-SU for qemu-devel@archiver.kernel.org; Thu, 11 Feb 2021 17:55:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41274) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lAKpj-00016l-0l; Thu, 11 Feb 2021 17:53:11 -0500 Received: from mail-qv1-xf36.google.com ([2607:f8b0:4864:20::f36]:37712) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lAKpg-0002Xe-RO; Thu, 11 Feb 2021 17:53:10 -0500 Received: by mail-qv1-xf36.google.com with SMTP id ew18so3393132qvb.4; Thu, 11 Feb 2021 14:53:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RYIFfTGBNKmoeyTOoD9yPinF0YoDJNuKg0BWwvrwzR0=; b=Apf97MIsjo8QYHWO4dhExXERlN28VOy4NecEH7jG7cVJCtTaN1YXWiTGMB4WKT9Z1Y YuOZWR/ngv9z4DUdQ7rMieziDooj2CWt8jKCS7HqLI2o1CG3G0IGE4CKGAMRxDrmWvj4 ocspyX6QT7Yot6LpT7wP1hbap/cPHy6Bcjkd3CvFBPIqkXloZhGyvJTr5KfZuangdOIW CWwf82RFZL9E8EDAMhHZVWsMCzFc0m89ep5aUW/1h147peyZWquteXyesI42kE91T9Pn VHqbxrXbHFd1TAtkD1SWRu65hDL5X8C/SbLaKDwrocBqBJI3QgB8Mc1rHqeqc8tEbV6O hTPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RYIFfTGBNKmoeyTOoD9yPinF0YoDJNuKg0BWwvrwzR0=; b=CusumGR9VdukuMO0B/sTkoofNijqSKWuJerZjj7Px4QGGvuscNyLMF3vG3dumq9kVK omB5WqTiV7tt36yVfxbuOZlaPksB1ne36XaCx09b7eVLS+w7kz+jRxkJZko+LK4nGuZF SBZA9TZtRaYSArJV5T4QazrwdMvGTx6+uvVTrbWPVUi1E5wuG56+eIn5qOtiZydp+kX9 tM2olbyylmzi+q+2S7WgolJ4bF3RTyGmAK2DTOGJ9y5WUp7sDdgCVPTDmMpaOSsHoNnV 65mkslJP67eus2OSnFov8gM8jEB5FsxU+wGddxzjOurJ4i7NOff7rjAfJG/M/xvGABp/ a1iQ== X-Gm-Message-State: AOAM533+NYmz48FGLc4iBkrZWi2ToL40L57suCndtny6hKsmPDVbu3OW tnpVr4bS8Qb49hfVSxneUfj58V4iXCfP1w== X-Google-Smtp-Source: ABdhPJzx5lbdqiBgCl+TKCKgSlkFqw3j4h0eQFVuVqR9B39clO/Ze221780eaTcSg7uwlJ63ayESZg== X-Received: by 2002:a0c:abce:: with SMTP id k14mr188902qvb.23.1613083987418; Thu, 11 Feb 2021 14:53:07 -0800 (PST) Received: from rekt.ibmuc.com ([2804:431:c7c6:6e83:c920:e8d3:a2df:bf3]) by smtp.gmail.com with ESMTPSA id x49sm4662227qth.95.2021.02.11.14.53.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Feb 2021 14:53:07 -0800 (PST) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [PATCH v3 4/7] spapr: rename spapr_drc_detach() to spapr_drc_unplug_request() Date: Thu, 11 Feb 2021 19:52:43 -0300 Message-Id: <20210211225246.17315-5-danielhb413@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210211225246.17315-1-danielhb413@gmail.com> References: <20210211225246.17315-1-danielhb413@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f36; envelope-from=danielhb413@gmail.com; helo=mail-qv1-xf36.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" spapr_drc_detach() is not the best name for what the function does. The function does not detach the DRC, it makes an uncommited attempt to do it. It'll mark the DRC as pending unplug, via the 'unplug_request' flag, and only if the DRC state is drck->empty_state it will detach the DRC, via spapr_drc_release(). This is a contrast with its pair spapr_drc_attach(), where the function is indeed creating the DRC QOM object. If you know what spapr_drc_attach() does, you can be misled into thinking that spapr_drc_detach() is removing the DRC from QEMU internal state, which isn't true. The current role of this function is better described as a request for detach, since there's no guarantee that we're going to detach the DRC in the end. Rename the function to spapr_drc_unplug_request to reflect what is is doing. The initial idea was to change the name to spapr_drc_detach_request(), and later on change the unplug_request flag to detach_request. However, unplug_request is a migratable boolean for a long time now and renaming it is not worth the trouble. spapr_drc_unplug_request() setting drc->unplug_request is more natural than spapr_drc_detach_request setting drc->unplug_request. Signed-off-by: Daniel Henrique Barboza Reviewed-by: David Gibson Reviewed-by: Greg Kurz --- hw/ppc/spapr.c | 6 +++--- hw/ppc/spapr_drc.c | 4 ++-- hw/ppc/spapr_pci.c | 2 +- hw/ppc/trace-events | 2 +- include/hw/ppc/spapr_drc.h | 2 +- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 85fe65f894..b066df68cb 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -3654,7 +3654,7 @@ static void spapr_memory_unplug_request(HotplugHandler *hotplug_dev, addr / SPAPR_MEMORY_BLOCK_SIZE); g_assert(drc); - spapr_drc_detach(drc); + spapr_drc_unplug_request(drc); addr += SPAPR_MEMORY_BLOCK_SIZE; } @@ -3722,7 +3722,7 @@ void spapr_core_unplug_request(HotplugHandler *hotplug_dev, DeviceState *dev, g_assert(drc); if (!spapr_drc_unplug_requested(drc)) { - spapr_drc_detach(drc); + spapr_drc_unplug_request(drc); spapr_hotplug_req_remove_by_index(drc); } } @@ -3985,7 +3985,7 @@ static void spapr_phb_unplug_request(HotplugHandler *hotplug_dev, assert(drc); if (!spapr_drc_unplug_requested(drc)) { - spapr_drc_detach(drc); + spapr_drc_unplug_request(drc); spapr_hotplug_req_remove_by_index(drc); } } diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c index 555a25517d..67041fb212 100644 --- a/hw/ppc/spapr_drc.c +++ b/hw/ppc/spapr_drc.c @@ -386,11 +386,11 @@ void spapr_drc_attach(SpaprDrc *drc, DeviceState *d) NULL, 0); } -void spapr_drc_detach(SpaprDrc *drc) +void spapr_drc_unplug_request(SpaprDrc *drc) { SpaprDrcClass *drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc); - trace_spapr_drc_detach(spapr_drc_index(drc)); + trace_spapr_drc_unplug_request(spapr_drc_index(drc)); g_assert(drc->dev); diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c index 1791d98a49..9334ba5dbb 100644 --- a/hw/ppc/spapr_pci.c +++ b/hw/ppc/spapr_pci.c @@ -1726,7 +1726,7 @@ static void spapr_pci_unplug_request(HotplugHandler *plug_handler, if (state == SPAPR_DR_ENTITY_SENSE_PRESENT) { /* Mark the DRC as requested unplug if needed. */ if (!spapr_drc_unplug_requested(func_drc)) { - spapr_drc_detach(func_drc); + spapr_drc_unplug_request(func_drc); } spapr_hotplug_req_remove_by_index(func_drc); } diff --git a/hw/ppc/trace-events b/hw/ppc/trace-events index 1e91984526..b4bbfbb013 100644 --- a/hw/ppc/trace-events +++ b/hw/ppc/trace-events @@ -50,7 +50,7 @@ spapr_drc_set_allocation_state(uint32_t index, int state) "drc: 0x%"PRIx32", sta spapr_drc_set_allocation_state_finalizing(uint32_t index) "drc: 0x%"PRIx32 spapr_drc_set_configured(uint32_t index) "drc: 0x%"PRIx32 spapr_drc_attach(uint32_t index) "drc: 0x%"PRIx32 -spapr_drc_detach(uint32_t index) "drc: 0x%"PRIx32 +spapr_drc_unplug_request(uint32_t index) "drc: 0x%"PRIx32 spapr_drc_awaiting_quiesce(uint32_t index) "drc: 0x%"PRIx32 spapr_drc_reset(uint32_t index) "drc: 0x%"PRIx32 spapr_drc_realize(uint32_t index) "drc: 0x%"PRIx32 diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h index 8982927d5c..02a63b3666 100644 --- a/include/hw/ppc/spapr_drc.h +++ b/include/hw/ppc/spapr_drc.h @@ -243,7 +243,7 @@ int spapr_dt_drc(void *fdt, int offset, Object *owner, uint32_t drc_type_mask); * beforehand (eg. check drc->dev at pre-plug). */ void spapr_drc_attach(SpaprDrc *drc, DeviceState *d); -void spapr_drc_detach(SpaprDrc *drc); +void spapr_drc_unplug_request(SpaprDrc *drc); /* * Reset all DRCs, causing pending hot-plug/unplug requests to complete. From patchwork Thu Feb 11 22:52:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 12084291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 397FFC433E0 for ; Thu, 11 Feb 2021 22:56:01 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B759364DD7 for ; Thu, 11 Feb 2021 22:56:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B759364DD7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:39588 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lAKsR-0003Kf-OI for qemu-devel@archiver.kernel.org; Thu, 11 Feb 2021 17:55:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41286) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lAKpk-0001Az-V6; Thu, 11 Feb 2021 17:53:12 -0500 Received: from mail-qt1-x835.google.com ([2607:f8b0:4864:20::835]:46578) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lAKpi-0002Yy-UZ; Thu, 11 Feb 2021 17:53:12 -0500 Received: by mail-qt1-x835.google.com with SMTP id b24so5424207qtp.13; Thu, 11 Feb 2021 14:53:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CN3Shy73WNB4yDxYenWFbL8Lil+6IpUeKyEWeJ7qbsQ=; b=f5tll60DUN+r0wXNI45A3oKKxxQLVDWVT1giD5LAY/v3qIKtOejyMgH7BNfIgPybZs 6EZIcRQwXoEwjuzKJg7busx+rPF92LUpFwe93sDKIyt8UcO7kfz+4BNzpYFnp5x5AOUy ZSH80qT1eJU2i3t4Avy2cF8tAIsBS0hBmKCpZD4+0snxIcsjDf7kBHfND6F1XPwyMJSp bhHYXQuDj20i/kcbHqpvnxniO41jvDjAtdBEJ9L0Gx+zYGvAk+CM2taw+WVP5tiepxV2 Oko1rkKJWkR3yDMn2OtmaI+jWOU7OUgSCA4aIT3NXJ7MPfua0Hhb5/Kzy5bAbn26lqIS G9KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CN3Shy73WNB4yDxYenWFbL8Lil+6IpUeKyEWeJ7qbsQ=; b=j9cjaq7HMuE6D4hNEgppj/gkyOU7b8gohqzEVsh8ogyFi821KAJR9j1DJ+JtfxyeEv zX0fbwDCVmyFtD0K+BpiMde0Ec3t9KGETwdn3dYWq7teXKKDxB3yI/yY9f9y/8IhNv+v 7u3pixrmdSlAaqv6TIWcNzMXt7RsV8dtxEwH1BcTKH3ZR7nq4dmoNTk7EEJIYZBrhs8t sMbAfjApfjR/0LWb6CDs1X7xHhr//f7QBf5DHe6fDl2WJ//33X9cdIFDqhFkjuOvaxQl jQ6jBE7kDg1Q6hwrDFbeMwO2kpQQkiR9mJhzV5/JXMasL7VurjEYgWPb5TfU5FO1Fbqs HCOw== X-Gm-Message-State: AOAM532FNyBR4kbIFbGKCiOaqr0d7Fmrx2+zGu5s3lVbusgySknFsLuo hHvG+k72zUwWBWSnl4FjZmQ6DHxsM86oUw== X-Google-Smtp-Source: ABdhPJwEUi1psSTHJ5o9lhUh8cUeh/jA6DI7UmU3khkKmH76ON5PtRAXDYb+OKzkprd90Mwv6edFWw== X-Received: by 2002:a05:622a:547:: with SMTP id m7mr9822189qtx.143.1613083989512; Thu, 11 Feb 2021 14:53:09 -0800 (PST) Received: from rekt.ibmuc.com ([2804:431:c7c6:6e83:c920:e8d3:a2df:bf3]) by smtp.gmail.com with ESMTPSA id x49sm4662227qth.95.2021.02.11.14.53.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Feb 2021 14:53:09 -0800 (PST) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [PATCH v3 5/7] spapr_drc.c: introduce unplug_timeout_timer Date: Thu, 11 Feb 2021 19:52:44 -0300 Message-Id: <20210211225246.17315-6-danielhb413@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210211225246.17315-1-danielhb413@gmail.com> References: <20210211225246.17315-1-danielhb413@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::835; envelope-from=danielhb413@gmail.com; helo=mail-qt1-x835.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The LoPAR spec provides no way for the guest kernel to report failure of hotplug/hotunplug events. This wouldn't be bad if those operations were granted to always succeed, but that's far for the reality. What ends up happening is that, in the case of a failed hotunplug, regardless of whether it was a QEMU error or a guest misbehavior, the pSeries machine is retaining the unplug state of the device in the running guest. This state is cleanup in machine reset, where it is assumed that this state represents a device that is pending unplug, and the device is hotunpluged from the board. Until the reset occurs, any hotunplug operation of the same device is forbid because there is a pending unplug state. This behavior has at least one undesirable side effect. A long standing pending unplug state is, more often than not, the result of a hotunplug error. The user had to dealt with it, since retrying to unplug the device is noy allowed, and then in the machine reset we're removing the device from the guest. This means that we're failing the user twice - failed to hotunplug when asked, then hotunplugged without notice. Solutions to this problem range between trying to predict when the hotunplug will fail and forbid the operation from the QEMU layer, from opening up the IRQ queue to allow for multiple hotunplug attempts, from telling the users to 'reboot the machine if something goes wrong'. The first solution is flawed because we can't fully predict guest behavior from QEMU, the second solution is a trial and error remediation that counts on a hope that the unplug will eventually succeed, and the third is ... well. This patch introduces a crude, but effective solution to hotunplug errors in the pSeries machine. For each unplug done, we'll timeout after some time. If a certain amount of time passes, we'll cleanup the hotunplug state from the machine. During the timeout period, any unplug operations in the same device will still be blocked. After that, we'll assume that the guest failed the operation, and allow the user to try again. If the timeout is too short we'll prevent legitimate hotunplug situations to occur, so we'll need to overestimate the regular time an unplug operation takes to succeed to account that. The true solution for the hotunplug errors in the pSeries machines is a PAPR change to allow for the guest to warn the platform about it. For now, the work done in this timeout design can be used for the new PAPR 'abort hcall' in the future, given that for both cases we'll need code to cleanup the existing unplug states of the DRCs. At this moment we're adding the basic wiring of the timer into the DRC. Next patch will use the timer to timeout failed CPU hotunplugs. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr_drc.c | 36 ++++++++++++++++++++++++++++++++++++ include/hw/ppc/spapr_drc.h | 2 ++ 2 files changed, 38 insertions(+) diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c index 67041fb212..c88bb524c5 100644 --- a/hw/ppc/spapr_drc.c +++ b/hw/ppc/spapr_drc.c @@ -57,6 +57,8 @@ static void spapr_drc_release(SpaprDrc *drc) drck->release(drc->dev); drc->unplug_requested = false; + timer_del(drc->unplug_timeout_timer); + g_free(drc->fdt); drc->fdt = NULL; drc->fdt_start_offset = 0; @@ -453,6 +455,24 @@ static const VMStateDescription vmstate_spapr_drc_unplug_requested = { } }; +static bool spapr_drc_unplug_timeout_timer_needed(void *opaque) +{ + SpaprDrc *drc = opaque; + + return timer_pending(drc->unplug_timeout_timer); +} + +static const VMStateDescription vmstate_spapr_drc_unplug_timeout_timer = { + .name = "DRC unplug timeout timer", + .version_id = 1, + .minimum_version_id = 1, + .needed = spapr_drc_unplug_timeout_timer_needed, + .fields = (VMStateField[]) { + VMSTATE_TIMER_PTR(unplug_timeout_timer, SpaprDrc), + VMSTATE_END_OF_LIST() + } +}; + static bool spapr_drc_needed(void *opaque) { SpaprDrc *drc = opaque; @@ -486,10 +506,20 @@ static const VMStateDescription vmstate_spapr_drc = { }, .subsections = (const VMStateDescription * []) { &vmstate_spapr_drc_unplug_requested, + &vmstate_spapr_drc_unplug_timeout_timer, NULL } }; +static void drc_unplug_timeout_cb(void *opaque) +{ + SpaprDrc *drc = opaque; + + if (drc->unplug_requested) { + drc->unplug_requested = false; + } +} + static void drc_realize(DeviceState *d, Error **errp) { SpaprDrc *drc = SPAPR_DR_CONNECTOR(d); @@ -512,6 +542,11 @@ static void drc_realize(DeviceState *d, Error **errp) object_property_add_alias(root_container, link_name, drc->owner, child_name); g_free(link_name); + + drc->unplug_timeout_timer = timer_new_ms(QEMU_CLOCK_VIRTUAL, + drc_unplug_timeout_cb, + drc); + vmstate_register(VMSTATE_IF(drc), spapr_drc_index(drc), &vmstate_spapr_drc, drc); trace_spapr_drc_realize_complete(spapr_drc_index(drc)); @@ -529,6 +564,7 @@ static void drc_unrealize(DeviceState *d) name = g_strdup_printf("%x", spapr_drc_index(drc)); object_property_del(root_container, name); g_free(name); + timer_free(drc->unplug_timeout_timer); } SpaprDrc *spapr_dr_connector_new(Object *owner, const char *type, diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h index 02a63b3666..b2e6222d09 100644 --- a/include/hw/ppc/spapr_drc.h +++ b/include/hw/ppc/spapr_drc.h @@ -187,6 +187,8 @@ typedef struct SpaprDrc { bool unplug_requested; void *fdt; int fdt_start_offset; + + QEMUTimer *unplug_timeout_timer; } SpaprDrc; struct SpaprMachineState; From patchwork Thu Feb 11 22:52:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 12084317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99BB4C433DB for ; Thu, 11 Feb 2021 22:58:25 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CFD4F60201 for ; Thu, 11 Feb 2021 22:58:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CFD4F60201 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:46698 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lAKul-0006no-G7 for qemu-devel@archiver.kernel.org; Thu, 11 Feb 2021 17:58:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41298) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lAKpn-0001GR-K8; Thu, 11 Feb 2021 17:53:15 -0500 Received: from mail-qt1-x830.google.com ([2607:f8b0:4864:20::830]:35627) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lAKpl-0002a6-JI; Thu, 11 Feb 2021 17:53:15 -0500 Received: by mail-qt1-x830.google.com with SMTP id c5so5477497qth.2; Thu, 11 Feb 2021 14:53:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1KiWwBM/VhEhrLsSBhtT31I9cjpl8vl+2pwC/cMov20=; b=uI7lcBtM1ZdZqtauM7YZMNMGvrSDdckPXGBEsqhV6naJ84chQWpnFn760GiZv0rkol 68/eQZrlWv6zUzsblu+lObcfga5JPtIAi0K7w5Ckm7pI/jhUDtSBkIxMN/5bWvqF8faI ImNNtbxfS3DSwmJXHa0vP7gc32Z7EAMOOU3WwalHtF4ntjBQgSVF6jhn0JS4KTYF95bP cmCSRhd/ykrwjEU47ZFTyLB7I5ijAfPZ6AlOfNFDHmtzSvPNig5wsk6C1Di+6dExQJLL A24dzw4WKdM7t+XRmDDwrMtFTtopWtNeiTCZn/F/6b43Y6O+uMdSXJjPaU8E4iZWWut+ Tbrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1KiWwBM/VhEhrLsSBhtT31I9cjpl8vl+2pwC/cMov20=; b=H1qvC2afeLQiSzqZzwMwY6iwmNyqkD9ouRHmWSwueH37QRYZ2vw53LjB0awRLqUxoc m3iYqkcdW5OuE2bthh0Gr764uozLQWa4w7VK+SYIvbUc1GVOpJa1uNXcOLjBWsenVSzp jSuUGsXcpLfMcsgLpqroFtPkQwpGRwEzFBJyndZr83OXLPat2zrpx9o4OvYNf/bQGP3n oGjT+4ohuAPpfUDKf+LaEUSt66zCXrCPYiydwKIxDEUcP7s+1CMbgvvrKBe1F4+ru+Ny hnz71zFppZE9kBTvz+IhNfdkrDewcM9APhMjDvfKRu1oq1FrWLaUd7X4sahXSM820aNY QeYw== X-Gm-Message-State: AOAM530zkGn1A9dTr0qPV7L5l7GvvSCKwaO2OIrlOpnIXzJJG7TkJ2Xo ufxzqbY1K+/LN0RK3KMceV0Yn/W+KmEloA== X-Google-Smtp-Source: ABdhPJxIP438lD6Pyps3R/JggWD5vOU9KTKuxQ6tNL1TawDTUYw5UpesO7sVnI/mxmzutmuuTTjNIA== X-Received: by 2002:ac8:5e8d:: with SMTP id r13mr9499130qtx.265.1613083992085; Thu, 11 Feb 2021 14:53:12 -0800 (PST) Received: from rekt.ibmuc.com ([2804:431:c7c6:6e83:c920:e8d3:a2df:bf3]) by smtp.gmail.com with ESMTPSA id x49sm4662227qth.95.2021.02.11.14.53.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Feb 2021 14:53:11 -0800 (PST) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [PATCH v3 6/7] spapr_drc.c: add hotunplug timeout for CPUs Date: Thu, 11 Feb 2021 19:52:45 -0300 Message-Id: <20210211225246.17315-7-danielhb413@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210211225246.17315-1-danielhb413@gmail.com> References: <20210211225246.17315-1-danielhb413@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::830; envelope-from=danielhb413@gmail.com; helo=mail-qt1-x830.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xujun Ma , Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" There is a reliable way to make a CPU hotunplug fail in the pseries machine. Hotplug a CPU A, then offline all other CPUs inside the guest but A. When trying to hotunplug A the guest kernel will refuse to do it, because A is now the last online CPU of the guest. PAPR has no 'error callback' in this situation to report back to the platform, so the guest kernel will deny the unplug in silent and QEMU will never know what happened. The unplug pending state of A will remain until the guest is shutdown or rebooted. Previous attempts of fixing it (see [1] and [2]) were aimed at trying to mitigate the effects of the problem. In [1] we were trying to guess which guest CPUs were online to forbid hotunplug of the last online CPU in the QEMU layer, avoiding the scenario described above because QEMU is now failing in behalf of the guest. This is not robust because the last online CPU of the guest can change while we're in the middle of the unplug process, and our initial assumptions are now invalid. In [2] we were accepting that our unplug process is uncertain and the user should be allowed to spam the IRQ hotunplug queue of the guest in case the CPU hotunplug fails. This patch presents another alternative, using the timeout infrastructure introduced in the previous patch. CPU hotunplugs in the pSeries machine will now timeout after 15 seconds. This is a long time for a single CPU unplug to occur, regardless of guest load - although the user is *strongly* encouraged to *not* hotunplug devices from a guest under high load - and we can be sure that something went wrong if it takes longer than that for the guest to release the CPU (the same can't be said about memory hotunplug - more on that in the next patch). Timing out the unplug operation will reset the unplug state of the CPU and allow the user to try it again, regardless of the error situation that prevented the hotunplug to occur. Of all the not so pretty fixes/mitigations for CPU hotunplug errors in pSeries, timing out the operation is an admission that we have no control in the process, and must assume the worst case if the operation doesn't succeed in a sensible time frame. [1] https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg03353.html [2] https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg04400.html Reported-by: Xujun Ma Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1911414 Signed-off-by: Daniel Henrique Barboza Reviewed-by: David Gibson --- hw/ppc/spapr.c | 4 ++++ hw/ppc/spapr_drc.c | 17 +++++++++++++++++ include/hw/ppc/spapr_drc.h | 3 +++ 3 files changed, 24 insertions(+) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index b066df68cb..ecce8abf14 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -3724,6 +3724,10 @@ void spapr_core_unplug_request(HotplugHandler *hotplug_dev, DeviceState *dev, if (!spapr_drc_unplug_requested(drc)) { spapr_drc_unplug_request(drc); spapr_hotplug_req_remove_by_index(drc); + } else { + error_setg(errp, "core-id %d unplug is still pending, %d seconds " + "timeout remaining", + cc->core_id, spapr_drc_unplug_timeout_remaining_sec(drc)); } } diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c index c88bb524c5..c143bfb6d3 100644 --- a/hw/ppc/spapr_drc.c +++ b/hw/ppc/spapr_drc.c @@ -398,6 +398,12 @@ void spapr_drc_unplug_request(SpaprDrc *drc) drc->unplug_requested = true; + if (drck->unplug_timeout_seconds != 0) { + timer_mod(drc->unplug_timeout_timer, + qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL) + + drck->unplug_timeout_seconds * 1000); + } + if (drc->state != drck->empty_state) { trace_spapr_drc_awaiting_quiesce(spapr_drc_index(drc)); return; @@ -406,6 +412,16 @@ void spapr_drc_unplug_request(SpaprDrc *drc) spapr_drc_release(drc); } +int spapr_drc_unplug_timeout_remaining_sec(SpaprDrc *drc) +{ + if (drc->unplug_requested && timer_pending(drc->unplug_timeout_timer)) { + return (qemu_timeout_ns_to_ms(drc->unplug_timeout_timer->expire_time) - + qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL)) / 1000; + } + + return 0; +} + bool spapr_drc_reset(SpaprDrc *drc) { SpaprDrcClass *drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc); @@ -706,6 +722,7 @@ static void spapr_drc_cpu_class_init(ObjectClass *k, void *data) drck->drc_name_prefix = "CPU "; drck->release = spapr_core_release; drck->dt_populate = spapr_core_dt_populate; + drck->unplug_timeout_seconds = 15; } static void spapr_drc_pci_class_init(ObjectClass *k, void *data) diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h index b2e6222d09..26599c385a 100644 --- a/include/hw/ppc/spapr_drc.h +++ b/include/hw/ppc/spapr_drc.h @@ -211,6 +211,8 @@ typedef struct SpaprDrcClass { int (*dt_populate)(SpaprDrc *drc, struct SpaprMachineState *spapr, void *fdt, int *fdt_start_offset, Error **errp); + + int unplug_timeout_seconds; } SpaprDrcClass; typedef struct SpaprDrcPhysical { @@ -246,6 +248,7 @@ int spapr_dt_drc(void *fdt, int offset, Object *owner, uint32_t drc_type_mask); */ void spapr_drc_attach(SpaprDrc *drc, DeviceState *d); void spapr_drc_unplug_request(SpaprDrc *drc); +int spapr_drc_unplug_timeout_remaining_sec(SpaprDrc *drc); /* * Reset all DRCs, causing pending hot-plug/unplug requests to complete. From patchwork Thu Feb 11 22:52:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 12084321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE624C433DB for ; Thu, 11 Feb 2021 23:01:16 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3C41C64D9F for ; Thu, 11 Feb 2021 23:01:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3C41C64D9F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:51896 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lAKxX-0001Ek-2H for qemu-devel@archiver.kernel.org; Thu, 11 Feb 2021 18:01:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41310) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lAKpp-0001Kv-O2; Thu, 11 Feb 2021 17:53:17 -0500 Received: from mail-qk1-x72c.google.com ([2607:f8b0:4864:20::72c]:44801) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lAKpn-0002as-Tn; Thu, 11 Feb 2021 17:53:17 -0500 Received: by mail-qk1-x72c.google.com with SMTP id o193so7031777qke.11; Thu, 11 Feb 2021 14:53:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mQVR3gMmth97tU7SkDPcq7EDdl4ka2Unc+wDklUQu/Q=; b=mgwhebO7Itd5kMk5tBC/Nrb8p8MGEBhOIytSvYV+iJQ8HpoDdzqKAPzAIooDJM+vNR wx0WPFyJI2390NQFbR09JIbkfnfxYz/GE+KW8x6hivFjs9mhClxuz6oJ+J0tpLGVbMo8 pt9r1BZgxVuPVJ1F3aJd/ErwQzoLxqHdcaAW2YuzMRwJAxm8l/jFsdoGGhDPFIE/u1MW Imc+wGTLKbwy8nF2hCSXsWYa3vVZCbFKcSt4EGOW4jsWP2AOAewosW7B0IfPUKKi4Iph GzRl1gybGwL9DHrjqQD9Ni+vu4V9oypT/ehc5P8pCIczeSKmNAJemDs2Sq9j+GsFMQG0 PFsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mQVR3gMmth97tU7SkDPcq7EDdl4ka2Unc+wDklUQu/Q=; b=dPjhD0TDV/DX8cLHyMV1UwMsK9YrfkifLyfzZ+FAZQUIPNPB3UQQ8TZZrEGtQfbprY FHHRwiLHifvKCTTLXKMh/bqFJwm+bgFQ7MbZAUdmQ+0OQzNbOkAXoOq5Eb4qeW6A945R RJMMID6rVjChWz2w3iAq4tEsMfhQ1Hk7CFuIfjTyQkX0GtZiaoIHiB+vwkTwn+zPKn0W DuLQ5JXc/Mfif5mfZdS97OZOskxE8InKvmahjy+PaogsxVhl1xwu9hKZhbfoHFu7pKLW MXUYG6NjidEB5Xfznf1wT9y7CnV7b/vMIwVnH7qPHyRtxH2GYXvlRAe37WzbvXpohZvb H2tQ== X-Gm-Message-State: AOAM532E5l/3cqIOuPbLibu+7m9vhgAXL7xMLJ3CSZG38cPa7L5EZuwy Qh4B6RVOOi6Q3W3bI917RPkwLyhY26P/Iw== X-Google-Smtp-Source: ABdhPJx4DoEOsytkWvx/oI3urpLjss8wlBwuJ5bZRK4RBtr40028+kaCDMp9l6sHJeoD2XhPM/DYJw== X-Received: by 2002:a37:7ac3:: with SMTP id v186mr53801qkc.211.1613083994480; Thu, 11 Feb 2021 14:53:14 -0800 (PST) Received: from rekt.ibmuc.com ([2804:431:c7c6:6e83:c920:e8d3:a2df:bf3]) by smtp.gmail.com with ESMTPSA id x49sm4662227qth.95.2021.02.11.14.53.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Feb 2021 14:53:14 -0800 (PST) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [PATCH v3 7/7] spapr_drc.c: use DRC reconfiguration to cleanup DIMM unplug state Date: Thu, 11 Feb 2021 19:52:46 -0300 Message-Id: <20210211225246.17315-8-danielhb413@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210211225246.17315-1-danielhb413@gmail.com> References: <20210211225246.17315-1-danielhb413@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::72c; envelope-from=danielhb413@gmail.com; helo=mail-qk1-x72c.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Handling errors in memory hotunplug in the pSeries machine is more complex than any other device type, because there are all the complications that other devices has, and more. For instance, determining a timeout for a DIMM hotunplug must consider if it's a Hash-MMU or a Radix-MMU guest, because Hash guests takes longer to hotunplug DIMMs. The size of the DIMM is also a factor, given that longer DIMMs naturally takes longer to be hotunplugged from the kernel. And there's also the guest memory usage to be considered: if there's a process that is consuming memory that would be lost by the DIMM unplug, the kernel will postpone the unplug process until the process finishes, and then initiate the regular hotunplug process. The first two considerations are manageable, but the last one is a deal breaker. There is no sane way for the pSeries machine to determine the memory load in the guest when attempting a DIMM hotunplug - and even if there was a way, the guest can start using all the RAM in the middle of the unplug process and invalidate our previous assumptions - and in result we can't even begin to calculate a timeout for the operation. This means that we can't implement a viable timeout mechanism for memory unplug in pSeries. Going back to why we would consider an unplug timeout, the reason is that we can't know if the kernel is giving up the unplug. Turns out that, sometimes, we can. Consider a failed memory hotunplug attempt where the kernel will error out with the following message: 'pseries-hotplug-mem: Memory indexed-count-remove failed, adding any removed LMBs' This happens when there is a LMB that the kernel gave up in removing, and the LMBs marked for removal of the same DIMM are now being added back. This process happens in the pseries kernel in [1], dlpar_memory_remove_by_ic() into dlpar_add_lmb(), and after that update_lmb_associativity_index(). In this function, the kernel is configuring the LMB DRC connector again. Note that this is a valid usage in LOPAR, as stated in section "ibm,configure-connector RTAS Call": 'A subsequent sequence of calls to ibm,configure-connector with the same entry from the “ibm,drc-indexes” or “ibm,drc-info” property will restart the configuration of devices which were not completely configured.' We can use this kernel behavior in our favor. If a DRC connector reconfiguration for a LMB that we marked as unplug pending happens, this indicates that the kernel changed its mind about the unplug and is reasserting that it will keep using the DIMM. In this case, it's safe to assume that the whole DIMM unplug was cancelled. This patch hops into rtas_ibm_configure_connector() and, in the scenario described above, clear the unplug state for the DIMM device. This will not solve all the problems we still have with memory unplug, but it will cover this case where the kernel reconfigures LMBs after a failed unplug. We are a bit more resilient, without using an unreliable timeout, and we didn't make the remaining error cases any worse. [1] arch/powerpc/platforms/pseries/hotplug-memory.c Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 30 ++++++++++++++++++++++++++++++ hw/ppc/spapr_drc.c | 14 ++++++++++++++ include/hw/ppc/spapr.h | 2 ++ 3 files changed, 46 insertions(+) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index ecce8abf14..4bcded4a1a 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -3575,6 +3575,36 @@ static SpaprDimmState *spapr_recover_pending_dimm_state(SpaprMachineState *ms, return spapr_pending_dimm_unplugs_add(ms, avail_lmbs, dimm); } +void spapr_clear_pending_dimm_unplug_state(SpaprMachineState *spapr, + PCDIMMDevice *dimm) +{ + SpaprDimmState *ds = spapr_pending_dimm_unplugs_find(spapr, dimm); + SpaprDrc *drc; + uint32_t nr_lmbs; + uint64_t size, addr_start, addr; + int i; + + if (ds) { + spapr_pending_dimm_unplugs_remove(spapr, ds); + } + + size = memory_device_get_region_size(MEMORY_DEVICE(dimm), &error_abort); + nr_lmbs = size / SPAPR_MEMORY_BLOCK_SIZE; + + addr_start = object_property_get_uint(OBJECT(dimm), PC_DIMM_ADDR_PROP, + &error_abort); + + addr = addr_start; + for (i = 0; i < nr_lmbs; i++) { + drc = spapr_drc_by_id(TYPE_SPAPR_DRC_LMB, + addr / SPAPR_MEMORY_BLOCK_SIZE); + g_assert(drc); + + drc->unplug_requested = false; + addr += SPAPR_MEMORY_BLOCK_SIZE; + } +} + /* Callback to be called during DRC release. */ void spapr_lmb_release(DeviceState *dev) { diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c index c143bfb6d3..eae941233a 100644 --- a/hw/ppc/spapr_drc.c +++ b/hw/ppc/spapr_drc.c @@ -1230,6 +1230,20 @@ static void rtas_ibm_configure_connector(PowerPCCPU *cpu, drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc); + /* + * This indicates that the kernel is reconfiguring a LMB due to + * a failed hotunplug. Clear the pending unplug state for the whole + * DIMM. + */ + if (spapr_drc_type(drc) == SPAPR_DR_CONNECTOR_TYPE_LMB && + drc->unplug_requested) { + + /* This really shouldn't happen in this point, but ... */ + g_assert(drc->dev); + + spapr_clear_pending_dimm_unplug_state(spapr, PC_DIMM(drc->dev)); + } + if (!drc->fdt) { void *fdt; int fdt_size; diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index ccbeeca1de..5bcc8f3bb8 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -847,6 +847,8 @@ int spapr_hpt_shift_for_ramsize(uint64_t ramsize); int spapr_reallocate_hpt(SpaprMachineState *spapr, int shift, Error **errp); void spapr_clear_pending_events(SpaprMachineState *spapr); void spapr_clear_pending_hotplug_events(SpaprMachineState *spapr); +void spapr_clear_pending_dimm_unplug_state(SpaprMachineState *spapr, + PCDIMMDevice *dimm); int spapr_max_server_number(SpaprMachineState *spapr); void spapr_store_hpte(PowerPCCPU *cpu, hwaddr ptex, uint64_t pte0, uint64_t pte1);