From patchwork Wed Jun 19 15:11:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krishna chaitanya chundru X-Patchwork-Id: 13704078 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 11BB67CF3E; Wed, 19 Jun 2024 15:11:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718809903; cv=none; b=PTL7oxPq8ScpQhd9cTyiX3F5gPeVc9rO7NPinym7/EHN8h5t2JQueidk6FYL/9+4dn+MMT+aiHc2oAZPW+YDKhCoRALX7ROUYhZz8fDObhyLi69HYFcV750lA3hOkpo1LTr1eYsK+6gNdMYrrDwlXAlZBts8n5nkve62LwNRyUc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718809903; c=relaxed/simple; bh=BwFdyEvpd+9lurlelU7UZbK7AzW5tmWU+tlySeF4IU0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-ID:References: In-Reply-To:To:CC; b=oi4vc1hb54gQ1mN8aJffkzNxoCbKwDwcglPsuE3IDoRlj+ZiGXPCJqWzo/lv6J+yE1PREB+PmGq6JRTwsAAcEecLN9XmxPT7iHIoT30h5bO2THiqKIzaG2Wm/JtsL92dK33sZASmuhboeh/W2d4Q7w4zx1YFFjjQuEqRRA1mjBU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com; spf=pass smtp.mailfrom=quicinc.com; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b=bCTuRkT9; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="bCTuRkT9" Received: from pps.filterd (m0279863.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45JA4K0b008449; Wed, 19 Jun 2024 15:11:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= 2A96CeR18taUJ/BoX6eHRUzdun9qVfzd3Ui2mbXbqkA=; b=bCTuRkT95KA9bt+o KBmzRgQQzNhRyKuzuTYzORVVWL5ZyaubX8ev7gqjRJ2HXhStzZIQ4t76/Zh59Fec YOOnFs/o3KEDsM2Ti+cxJhPODtgRadZxx3+NYqvYkEJJP1/evdL27pODcepORjYi YLmO22egrWKopZ9iFo2Dl+/b9wL5rwtQVeqrN3F5cJJtq2Vpqp5A3au2bMPxchKA 06IEVwjlqxVMHHbH57J9OamhWloFWhxQ6wtJ+yt/VPllV7XQaylZQTna8pFEDpiU 73RCLOeAJcSfOtGrBdWPF0px+8u7/g7bh/RybFez/GqTZp0IR2Yo2SqTkhwuDVfn CrG6eA== Received: from nalasppmta03.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3yuj9tj3sb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 19 Jun 2024 15:11:30 +0000 (GMT) Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA03.qualcomm.com (8.17.1.19/8.17.1.19) with ESMTPS id 45JFBTGF002062 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 19 Jun 2024 15:11:29 GMT Received: from hu-krichai-hyd.qualcomm.com (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.9; Wed, 19 Jun 2024 08:11:24 -0700 From: Krishna chaitanya chundru Date: Wed, 19 Jun 2024 20:41:10 +0530 Subject: [PATCH v15 1/4] PCI: qcom: Add ICC bandwidth vote for CPU to PCIe path Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20240619-opp_support-v15-1-aa769a2173a3@quicinc.com> References: <20240619-opp_support-v15-0-aa769a2173a3@quicinc.com> In-Reply-To: <20240619-opp_support-v15-0-aa769a2173a3@quicinc.com> To: Manivannan Sadhasivam , "Lorenzo Pieralisi" , =?utf-8?q?Krzysztof_Wilczy?= =?utf-8?q?=C5=84ski?= , Rob Herring , Bjorn Helgaas , Krzysztof Kozlowski , Conor Dooley , Bjorn Andersson CC: , , , , , , , Krishna chaitanya chundru , Bryan O'Donoghue X-Mailer: b4 0.13-dev-83828 X-Developer-Signature: v=1; a=ed25519-sha256; t=1718809879; l=4515; i=quic_krichai@quicinc.com; s=20230907; h=from:subject:message-id; bh=BwFdyEvpd+9lurlelU7UZbK7AzW5tmWU+tlySeF4IU0=; b=9OhcO1fcwg5KV/51kvafPor+tip0AYx5Gi+pwtvgVcOUfSQz1epOGd3xNLiNebnkKHdXA1JtQ tljiCrdcYR6DH/6OUAni7m1gPFKDRa7E9snSzR9Og80XmSYHWOp2A11 X-Developer-Key: i=quic_krichai@quicinc.com; a=ed25519; pk=10CL2pdAKFyzyOHbfSWHCD0X0my7CXxj8gJScmn1FAg= X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: lqsylGZ3434l9zF52grHQmJ3Ouaclf7e X-Proofpoint-ORIG-GUID: lqsylGZ3434l9zF52grHQmJ3Ouaclf7e X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-19_02,2024-06-19_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015 bulkscore=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=999 mlxscore=0 priorityscore=1501 spamscore=0 phishscore=0 suspectscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2405170001 definitions=main-2406190114 To access the host controller registers of the host controller and the endpoint BAR/config space, the CPU-PCIe ICC (interconnect) path should be voted otherwise it may lead to NoC (Network on chip) timeout. We are surviving because of other driver voting for this path. As there is less access on this path compared to PCIe to mem path add minimum vote i.e 1KBps bandwidth always which is sufficient enough to keep the path active and is recommended by HW team. During S2RAM (Suspend-to-RAM), the DBI access can happen very late (while disabling the boot CPU). So do not disable the CPU-PCIe interconnect path during S2RAM as that may lead to NoC error. Reviewed-by: Bryan O'Donoghue Reviewed-by: Manivannan Sadhasivam Signed-off-by: Krishna chaitanya chundru --- drivers/pci/controller/dwc/pcie-qcom.c | 45 +++++++++++++++++++++++++++++++--- 1 file changed, 41 insertions(+), 4 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c index 5f9f0ff19baa..ff1d891c8b9a 100644 --- a/drivers/pci/controller/dwc/pcie-qcom.c +++ b/drivers/pci/controller/dwc/pcie-qcom.c @@ -253,6 +253,7 @@ struct qcom_pcie { struct phy *phy; struct gpio_desc *reset; struct icc_path *icc_mem; + struct icc_path *icc_cpu; const struct qcom_pcie_cfg *cfg; struct dentry *debugfs; bool suspended; @@ -1369,6 +1370,9 @@ static int qcom_pcie_icc_init(struct qcom_pcie *pcie) if (IS_ERR(pcie->icc_mem)) return PTR_ERR(pcie->icc_mem); + pcie->icc_cpu = devm_of_icc_get(pci->dev, "cpu-pcie"); + if (IS_ERR(pcie->icc_cpu)) + return PTR_ERR(pcie->icc_cpu); /* * Some Qualcomm platforms require interconnect bandwidth constraints * to be set before enabling interconnect clocks. @@ -1378,11 +1382,25 @@ static int qcom_pcie_icc_init(struct qcom_pcie *pcie) */ ret = icc_set_bw(pcie->icc_mem, 0, QCOM_PCIE_LINK_SPEED_TO_BW(1)); if (ret) { - dev_err(pci->dev, "failed to set interconnect bandwidth: %d\n", + dev_err(pci->dev, "Failed to set bandwidth for PCIe-MEM interconnect path: %d\n", ret); return ret; } + /* + * Since the CPU-PCIe path is only used for activities like register + * access of the host controller and endpoint Config/BAR space access, + * HW team has recommended to use a minimal bandwidth of 1KBps just to + * keep the path active. + */ + ret = icc_set_bw(pcie->icc_cpu, 0, kBps_to_icc(1)); + if (ret) { + dev_err(pci->dev, "Failed to set bandwidth for CPU-PCIe interconnect path: %d\n", + ret); + icc_set_bw(pcie->icc_mem, 0, 0); + return ret; + } + return 0; } @@ -1408,7 +1426,7 @@ static void qcom_pcie_icc_update(struct qcom_pcie *pcie) ret = icc_set_bw(pcie->icc_mem, 0, width * QCOM_PCIE_LINK_SPEED_TO_BW(speed)); if (ret) { - dev_err(pci->dev, "failed to set interconnect bandwidth: %d\n", + dev_err(pci->dev, "Failed to set bandwidth for PCIe-MEM interconnect path: %d\n", ret); } } @@ -1570,7 +1588,7 @@ static int qcom_pcie_suspend_noirq(struct device *dev) */ ret = icc_set_bw(pcie->icc_mem, 0, kBps_to_icc(1)); if (ret) { - dev_err(dev, "Failed to set interconnect bandwidth: %d\n", ret); + dev_err(dev, "Failed to set bandwidth for PCIe-MEM interconnect path: %d\n", ret); return ret; } @@ -1594,7 +1612,18 @@ static int qcom_pcie_suspend_noirq(struct device *dev) pcie->suspended = true; } - return 0; + /* + * Only disable CPU-PCIe interconnect path if the suspend is non-S2RAM. + * Because on some platforms, DBI access can happen very late during the + * S2RAM and a non-active CPU-PCIe interconnect path may lead to NoC + * error. + */ + if (pm_suspend_target_state != PM_SUSPEND_MEM) { + ret = icc_disable(pcie->icc_cpu); + if (ret) + dev_err(dev, "Failed to disable CPU-PCIe interconnect path: %d\n", ret); + } + return ret; } static int qcom_pcie_resume_noirq(struct device *dev) @@ -1602,6 +1631,14 @@ static int qcom_pcie_resume_noirq(struct device *dev) struct qcom_pcie *pcie = dev_get_drvdata(dev); int ret; + if (pm_suspend_target_state != PM_SUSPEND_MEM) { + ret = icc_enable(pcie->icc_cpu); + if (ret) { + dev_err(dev, "Failed to enable CPU-PCIe interconnect path: %d\n", ret); + return ret; + } + } + if (pcie->suspended) { ret = qcom_pcie_host_init(&pcie->pci->pp); if (ret) From patchwork Wed Jun 19 15:11:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krishna chaitanya chundru X-Patchwork-Id: 13704080 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8943E13B7AF; Wed, 19 Jun 2024 15:11:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718809907; cv=none; b=bzRqiAQTQHCCRuewdp7EjHdhBsdFYE+jZ7PyFmJ+0M/9kmQI9On1b+uDvwRhlTsMLzGE2M3Oe4fTTxbtBZ45oeK1rkIDzhVf2YoNCZnWrTLuQo4+GU2zxpVtgp7+zLGbnY72SiKtfgDui/x+ySNQ3dhkQG+S6F6OABTGJsjAaXw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718809907; c=relaxed/simple; bh=FG21nJMzDgQVBPiWDAMRoSDJCoh5DoD48CK6U3Qv5f8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-ID:References: In-Reply-To:To:CC; b=tNvgM2IhH8qZjbFSnK207IPNeVKB/Eh/RFfvpx34YtnI9SUH7Yk+NX/dowKsuHZwa/BpCSRSEYZL3wuGSkYnWLqv9w/G68AeAPWOwY66RbJQ7g3wRmPit08mgygyDEDQwD5pXHd/VwxHhjZN99/NwKieJTkndOHCFYWe6dpSAB0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com; spf=pass smtp.mailfrom=quicinc.com; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b=j69YxypE; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="j69YxypE" Received: from pps.filterd (m0279863.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45JA4K0c008449; Wed, 19 Jun 2024 15:11:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= ljGBYLQFC3HKRQ9g+LrdPl9BYHhm2n7q5flQqnt1vP8=; b=j69YxypENRZfG8/A FQyVz6Yu9OCw0ABnqwRH+CC8mH62iSren2rVTVMp4zHCshS8CTKUmr4KE6mmcgxg tuj4xtk8Id7mpN8wn9EhQ3lRn94vTJIwvVcLnNutda9KUpuwMTEBubR818ewsWEe sjzv6Nsn74CefV2fF5SYRcbRKwtEF5T92cYfnp51jQCJOFCiFr5pVgP5JDuE94/i tYaxkGPmzackvl9RHVK9Qq52ngHvxI0udTmnKChSovSzWYjvcxATBfVHn5VZtLB2 H6nP1ywtNJPMuTpmETcaNwLn3Ka3uAmnLdr/jJk8/eXcJHTdbyQ5PPPGc1g2vuip N4fYvw== Received: from nalasppmta05.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3yuj9tj3sj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 19 Jun 2024 15:11:35 +0000 (GMT) Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA05.qualcomm.com (8.17.1.19/8.17.1.19) with ESMTPS id 45JFBYoi000962 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 19 Jun 2024 15:11:34 GMT Received: from hu-krichai-hyd.qualcomm.com (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.9; Wed, 19 Jun 2024 08:11:29 -0700 From: Krishna chaitanya chundru Date: Wed, 19 Jun 2024 20:41:11 +0530 Subject: [PATCH v15 2/4] dt-bindings: pci: qcom: Add OPP table Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20240619-opp_support-v15-2-aa769a2173a3@quicinc.com> References: <20240619-opp_support-v15-0-aa769a2173a3@quicinc.com> In-Reply-To: <20240619-opp_support-v15-0-aa769a2173a3@quicinc.com> To: Manivannan Sadhasivam , "Lorenzo Pieralisi" , =?utf-8?q?Krzysztof_Wilczy?= =?utf-8?q?=C5=84ski?= , Rob Herring , Bjorn Helgaas , Krzysztof Kozlowski , Conor Dooley , Bjorn Andersson CC: , , , , , , , Krishna chaitanya chundru , Krzysztof Kozlowski X-Mailer: b4 0.13-dev-83828 X-Developer-Signature: v=1; a=ed25519-sha256; t=1718809879; l=1082; i=quic_krichai@quicinc.com; s=20230907; h=from:subject:message-id; bh=FG21nJMzDgQVBPiWDAMRoSDJCoh5DoD48CK6U3Qv5f8=; b=HXDb9w+C2MkmeK9090SIvN1UU5kTiVo2tZQKmILJNxaC456hKAN899AjH7vbj6+bCUPBUtEIO Y0vpG3m+ZQrDnoIws2vBuXKQl3QKk4Ouw3lAiZBBZ55YziZ/obwLCPy X-Developer-Key: i=quic_krichai@quicinc.com; a=ed25519; pk=10CL2pdAKFyzyOHbfSWHCD0X0my7CXxj8gJScmn1FAg= X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: bpOKeSQKapEGiPuBqUIvSvJoRSpQe7yY X-Proofpoint-ORIG-GUID: bpOKeSQKapEGiPuBqUIvSvJoRSpQe7yY X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-19_02,2024-06-19_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015 bulkscore=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=999 mlxscore=0 priorityscore=1501 spamscore=0 phishscore=0 suspectscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2405170001 definitions=main-2406190114 PCIe needs to choose the appropriate performance state of RPMh power domain based on the PCIe gen speed. Adding the Operating Performance Points table allows to adjust power domain performance state and ICC peak bw, depending on the PCIe data rate and link width. Reviewed-by: Krzysztof Kozlowski Reviewed-by: Manivannan Sadhasivam Signed-off-by: Krishna chaitanya chundru --- Documentation/devicetree/bindings/pci/qcom,pcie-sm8450.yaml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Documentation/devicetree/bindings/pci/qcom,pcie-sm8450.yaml b/Documentation/devicetree/bindings/pci/qcom,pcie-sm8450.yaml index 1496d6993ab4..d8c0afaa4b19 100644 --- a/Documentation/devicetree/bindings/pci/qcom,pcie-sm8450.yaml +++ b/Documentation/devicetree/bindings/pci/qcom,pcie-sm8450.yaml @@ -69,6 +69,10 @@ properties: - const: msi6 - const: msi7 + operating-points-v2: true + opp-table: + type: object + resets: maxItems: 1 From patchwork Wed Jun 19 15:11:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krishna chaitanya chundru X-Patchwork-Id: 13704079 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4AE1150987; Wed, 19 Jun 2024 15:11:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718809907; cv=none; b=iASLm4dNeHLMnV348R244Wk3gf9V9z8YGaVBPiThm2Hs0rzl6Zql6gWWtAvMnFOSugkOrii1tGT7CwzH6ODtIHVSPk1REYnMzqu6ukS4Lsbl60N+fgt/Do3l5a/3ZFqoOIgxaoBBNuDfkFxWnx+WSi9CG+wVZ7tl6mQozHYn7HQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718809907; c=relaxed/simple; bh=Mk7mCVVVDh1Xk2NfmSJF0yNI6F09ea5QL/FfkH/3fTU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-ID:References: In-Reply-To:To:CC; b=czOjTy1HbFM6TtMwdaJqtk2ekK9fNuz2bQ94V/LLUL0WmRS0SaxTAhd/mwA9wF2SOaPKiQxsm7LRbV/gDDhjc6dBGKbhHkkGTrPClu0I3qyAJmooOswIT1EZmLA5J2SANJM2fl5J7LdrOFaXwn2tr3QE2uCQn5n0JFfbQHwglUQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com; spf=pass smtp.mailfrom=quicinc.com; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b=hJ0hdxWM; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="hJ0hdxWM" Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45J9ZKh6010307; Wed, 19 Jun 2024 15:11:40 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= CElOfEt70W5yFw1ZjNylR6hJxHJ3zLgfMT5NQlcwzYw=; b=hJ0hdxWM6vF1pe9M jTPYI8syQJW2C0AbmHE8EdaO/R3NPddu2xNveN0nQucT72TocX7L8IFT9DTxsnxZ 0/TjsErqK9pM8Gbi/Kzf5RYmuNdc9lbk3uwsRZj1LuqPrDDdMgn4ot8Qlf0moc3Q GjeYGN/YAvDGgZPbbKgkdKoiIVeDcKVzJvrrQsxNO9rDPYtU4S9fXqUzGqMKCdBH T90XZn3BHRxYnvzSuXhwIqiBDkG+4HXuVaGHSq5nqhnmPZDhcf2eKveigPlB7M4T 5wk2JxDYQSlZt144MLN+0PYIveqq9M3ZD7bPEnXEX3jVGQK+oBWt/eSaYLoTb37M 18+iag== Received: from nalasppmta01.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3yuja2j33c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 19 Jun 2024 15:11:40 +0000 (GMT) Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA01.qualcomm.com (8.17.1.19/8.17.1.19) with ESMTPS id 45JFBdlq026929 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 19 Jun 2024 15:11:39 GMT Received: from hu-krichai-hyd.qualcomm.com (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.9; Wed, 19 Jun 2024 08:11:34 -0700 From: Krishna chaitanya chundru Date: Wed, 19 Jun 2024 20:41:12 +0530 Subject: [PATCH v15 3/4] PCI: Bring the PCIe speed to MBps logic to new pcie_dev_speed_mbps() Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20240619-opp_support-v15-3-aa769a2173a3@quicinc.com> References: <20240619-opp_support-v15-0-aa769a2173a3@quicinc.com> In-Reply-To: <20240619-opp_support-v15-0-aa769a2173a3@quicinc.com> To: Manivannan Sadhasivam , "Lorenzo Pieralisi" , =?utf-8?q?Krzysztof_Wilczy?= =?utf-8?q?=C5=84ski?= , Rob Herring , Bjorn Helgaas , Krzysztof Kozlowski , Conor Dooley , Bjorn Andersson CC: , , , , , , , Krishna chaitanya chundru X-Mailer: b4 0.13-dev-83828 X-Developer-Signature: v=1; a=ed25519-sha256; t=1718809879; l=2073; i=quic_krichai@quicinc.com; s=20230907; h=from:subject:message-id; bh=Mk7mCVVVDh1Xk2NfmSJF0yNI6F09ea5QL/FfkH/3fTU=; b=G7rE3HUGC8v8+1cowhRRwYMUa7RWWMZlL+S03rFAj42SIDoTk36VFalev8CZZvbQ40M89IyJY 1pzu9m1Q9aMD3Tr55ZzCRjKoSwpC52b4fvaEThXls3J3hHUYPyWpQC6 X-Developer-Key: i=quic_krichai@quicinc.com; a=ed25519; pk=10CL2pdAKFyzyOHbfSWHCD0X0my7CXxj8gJScmn1FAg= X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: TgLuhs-y25xocSny1ig-qkruhuGWuLE7 X-Proofpoint-ORIG-GUID: TgLuhs-y25xocSny1ig-qkruhuGWuLE7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-19_02,2024-06-19_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 mlxscore=0 spamscore=0 adultscore=0 priorityscore=1501 suspectscore=0 mlxlogscore=948 malwarescore=0 phishscore=0 impostorscore=0 bulkscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2405170001 definitions=main-2406190114 Bring the switch case in pcie_link_speed_mbps() to new function to the header file so that it can be used in other places like in controller driver. Reviewed-by: Manivannan Sadhasivam Acked-by: Bjorn Helgaas Signed-off-by: Krishna chaitanya chundru --- drivers/pci/pci.c | 19 +------------------ drivers/pci/pci.h | 22 ++++++++++++++++++++++ 2 files changed, 23 insertions(+), 18 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index d2c388761ba9..5defa9384112 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -6027,24 +6027,7 @@ int pcie_link_speed_mbps(struct pci_dev *pdev) if (err) return err; - switch (to_pcie_link_speed(lnksta)) { - case PCIE_SPEED_2_5GT: - return 2500; - case PCIE_SPEED_5_0GT: - return 5000; - case PCIE_SPEED_8_0GT: - return 8000; - case PCIE_SPEED_16_0GT: - return 16000; - case PCIE_SPEED_32_0GT: - return 32000; - case PCIE_SPEED_64_0GT: - return 64000; - default: - break; - } - - return -EINVAL; + return pcie_dev_speed_mbps(to_pcie_link_speed(lnksta)); } EXPORT_SYMBOL(pcie_link_speed_mbps); diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 1b021579f26a..92dcc3702079 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -333,6 +333,28 @@ void pci_bus_put(struct pci_bus *bus); (speed) == PCIE_SPEED_2_5GT ? 2500*8/10 : \ 0) +static inline int pcie_dev_speed_mbps(enum pci_bus_speed speed) +{ + switch (speed) { + case PCIE_SPEED_2_5GT: + return 2500; + case PCIE_SPEED_5_0GT: + return 5000; + case PCIE_SPEED_8_0GT: + return 8000; + case PCIE_SPEED_16_0GT: + return 16000; + case PCIE_SPEED_32_0GT: + return 32000; + case PCIE_SPEED_64_0GT: + return 64000; + default: + break; + } + + return -EINVAL; +} + const char *pci_speed_string(enum pci_bus_speed speed); enum pci_bus_speed pcie_get_speed_cap(struct pci_dev *dev); enum pcie_link_width pcie_get_width_cap(struct pci_dev *dev); From patchwork Wed Jun 19 15:11:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krishna chaitanya chundru X-Patchwork-Id: 13704081 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 025AA143865; Wed, 19 Jun 2024 15:11:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718809919; cv=none; b=uMlvknR6wjBCOOS4a/LMwomxi7ZR/6w7358Lk9NuAPWVYYljS2BGRZHw/F8ZHyomXYCmInkiB6VPfRcQegLdbiXPpOejpEKFibzNsmqE5YZT8Cd5gdqMV6xy5Ngct4n8xXvXOnp08pM8Q7Z+Uxe6lfd69PfN6ekHlysEWSwpdl8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718809919; c=relaxed/simple; bh=BYVRKzfXBlXgDsIeh+DrronahvhHCjFkoWzYxDAk7vo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-ID:References: In-Reply-To:To:CC; b=KB+73rPFR2gkFBu1h8Ymq1TRQY/zC50IDqxMXcrv0InYvQNzlejaxx0fEsPtf9OY+y7gZSxOAwSN7PiS7DgfuQIRxTWvusanutZSv5/hqCoIRpk3aGqzO4z4QjUIGfpQOitDI1+wGRf6bgj0p7EUtqWHJ0B31JUIdWLuWuCICuk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com; spf=pass smtp.mailfrom=quicinc.com; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b=Ei5asQwp; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="Ei5asQwp" Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 45J9iYN0006366; Wed, 19 Jun 2024 15:11:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= 9Se0+N+BRJGDwyaVTI7sVgqfoiejlaZi69Y8Q01wbMA=; b=Ei5asQwpNT/vw6AW PQEpFexUEDg/HV3w6rfB2McH+V/mVoPENos+XizPqdNI19lF9fmeSEHlCfKfZHLv SUy5rtZwQrW+P7kIMQeE3DscE9nMs3UE2+zZwUSm4z2wXTnV/K+8KBVuKykzgmBo n8FnPbN11kZQMD+WrAIB8gjzFM0v7EzvihjqpVEtoTqgNJ10+j70wziCLg4bCC3B ZLFe4Vak9hWZiEbCtXvCv5fATfKpHoQxBC9soXWaRItsG5lyivDpxaB75rqsW2sg UKkppVw78zYzC4ysDHs4S+L1kWVhHUPaMW2H6wrhV1dmuxVPvtMFMkSMtMTLUL/8 wFNxEw== Received: from nalasppmta01.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3yuj9u23ww-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 19 Jun 2024 15:11:45 +0000 (GMT) Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA01.qualcomm.com (8.17.1.19/8.17.1.19) with ESMTPS id 45JFBikK027380 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 19 Jun 2024 15:11:44 GMT Received: from hu-krichai-hyd.qualcomm.com (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.9; Wed, 19 Jun 2024 08:11:39 -0700 From: Krishna chaitanya chundru Date: Wed, 19 Jun 2024 20:41:13 +0530 Subject: [PATCH v15 4/4] PCI: qcom: Add OPP support to scale performance Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20240619-opp_support-v15-4-aa769a2173a3@quicinc.com> References: <20240619-opp_support-v15-0-aa769a2173a3@quicinc.com> In-Reply-To: <20240619-opp_support-v15-0-aa769a2173a3@quicinc.com> To: Manivannan Sadhasivam , "Lorenzo Pieralisi" , =?utf-8?q?Krzysztof_Wilczy?= =?utf-8?q?=C5=84ski?= , Rob Herring , Bjorn Helgaas , Krzysztof Kozlowski , Conor Dooley , Bjorn Andersson CC: , , , , , , , Krishna chaitanya chundru X-Mailer: b4 0.13-dev-83828 X-Developer-Signature: v=1; a=ed25519-sha256; t=1718809879; l=6802; i=quic_krichai@quicinc.com; s=20230907; h=from:subject:message-id; bh=BYVRKzfXBlXgDsIeh+DrronahvhHCjFkoWzYxDAk7vo=; b=xfSrAdLwl5e6ynaIZ8MN64QeNJZK4QpCSLdIQl4n2LAdN8loS4sEu+xEtHFsfGKpf8fegPBXj Tiwt2h1sptSDFi6XdUJ43CpSRHnw9rEKCx6E6cg2wINJpI17rskTp4i X-Developer-Key: i=quic_krichai@quicinc.com; a=ed25519; pk=10CL2pdAKFyzyOHbfSWHCD0X0my7CXxj8gJScmn1FAg= X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: wsb0iITbWDuTAA7HR84Zn3fvvRfHBxfA X-Proofpoint-ORIG-GUID: wsb0iITbWDuTAA7HR84Zn3fvvRfHBxfA X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-06-19_02,2024-06-19_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 adultscore=0 mlxlogscore=999 clxscore=1015 mlxscore=0 phishscore=0 bulkscore=0 impostorscore=0 spamscore=0 suspectscore=0 malwarescore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2405170001 definitions=main-2406190114 QCOM Resource Power Manager-hardened (RPMh) is a hardware block which maintains hardware state of a regulator by performing max aggregation of the requests made by all of the clients. PCIe controller can operate on different RPMh performance state of power domain based on the speed of the link. And this performance state varies from target to target, like some controllers support GEN3 in NOM (Nominal) voltage corner, while some other supports GEN3 in low SVS (static voltage scaling). The SoC can be more power efficient if we scale the performance state based on the aggregate PCIe link bandwidth. Add Operating Performance Points (OPP) support to vote for RPMh state based on the aggregate link bandwidth. OPP can handle ICC bw voting also, so move ICC bw voting through OPP framework if OPP entries are present. As we are moving ICC voting as part of OPP, don't initialize ICC if OPP is supported. Before PCIe link is initialized vote for highest OPP in the OPP table, so that we are voting for maximum voltage corner for the link to come up in maximum supported speed. Reviewed-by: Manivannan Sadhasivam Signed-off-by: Krishna chaitanya chundru --- drivers/pci/controller/dwc/pcie-qcom.c | 97 +++++++++++++++++++++++++++------- 1 file changed, 78 insertions(+), 19 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c index ff1d891c8b9a..26405fcfa499 100644 --- a/drivers/pci/controller/dwc/pcie-qcom.c +++ b/drivers/pci/controller/dwc/pcie-qcom.c @@ -18,9 +18,11 @@ #include #include #include +#include #include #include #include +#include #include #include #include @@ -29,6 +31,7 @@ #include #include #include +#include #include "../../pci.h" #include "pcie-designware.h" @@ -1404,15 +1407,13 @@ static int qcom_pcie_icc_init(struct qcom_pcie *pcie) return 0; } -static void qcom_pcie_icc_update(struct qcom_pcie *pcie) +static void qcom_pcie_icc_opp_update(struct qcom_pcie *pcie) { + u32 offset, status, width, speed; struct dw_pcie *pci = pcie->pci; - u32 offset, status; - int speed, width; - int ret; - - if (!pcie->icc_mem) - return; + unsigned long freq_kbps; + struct dev_pm_opp *opp; + int ret, freq_mbps; offset = dw_pcie_find_capability(pci, PCI_CAP_ID_EXP); status = readw(pci->dbi_base + offset + PCI_EXP_LNKSTA); @@ -1424,10 +1425,26 @@ static void qcom_pcie_icc_update(struct qcom_pcie *pcie) speed = FIELD_GET(PCI_EXP_LNKSTA_CLS, status); width = FIELD_GET(PCI_EXP_LNKSTA_NLW, status); - ret = icc_set_bw(pcie->icc_mem, 0, width * QCOM_PCIE_LINK_SPEED_TO_BW(speed)); - if (ret) { - dev_err(pci->dev, "Failed to set bandwidth for PCIe-MEM interconnect path: %d\n", - ret); + if (pcie->icc_mem) { + ret = icc_set_bw(pcie->icc_mem, 0, width * QCOM_PCIE_LINK_SPEED_TO_BW(speed)); + if (ret) { + dev_err(pci->dev, "Failed to set bandwidth for PCIe-MEM interconnect path: %d\n", + ret); + } + } else { + freq_mbps = pcie_dev_speed_mbps(pcie_link_speed[speed]); + if (freq_mbps < 0) + return; + + freq_kbps = freq_mbps * KILO; + opp = dev_pm_opp_find_freq_exact(pci->dev, freq_kbps * width, true); + if (!IS_ERR(opp)) { + ret = dev_pm_opp_set_opp(pci->dev, opp); + if (ret) + dev_err(pci->dev, "Failed to set OPP for freq (%lu): %d\n", + freq_kbps * width, ret); + } + dev_pm_opp_put(opp); } } @@ -1471,7 +1488,9 @@ static void qcom_pcie_init_debugfs(struct qcom_pcie *pcie) static int qcom_pcie_probe(struct platform_device *pdev) { const struct qcom_pcie_cfg *pcie_cfg; + unsigned long max_freq = ULONG_MAX; struct device *dev = &pdev->dev; + struct dev_pm_opp *opp; struct qcom_pcie *pcie; struct dw_pcie_rp *pp; struct resource *res; @@ -1539,9 +1558,42 @@ static int qcom_pcie_probe(struct platform_device *pdev) goto err_pm_runtime_put; } - ret = qcom_pcie_icc_init(pcie); - if (ret) + /* OPP table is optional */ + ret = devm_pm_opp_of_add_table(dev); + if (ret && ret != -ENODEV) { + dev_err_probe(dev, ret, "Failed to add OPP table\n"); goto err_pm_runtime_put; + } + + /* + * Before PCIe link is initialized vote for highest OPP in the OPP table, + * so that we are voting for maximum voltage corner for the link to come up + * in maximum supported speed. At the end of the probe(), OPP will be + * updated using qcom_pcie_icc_opp_update(). + */ + if (!ret) { + opp = dev_pm_opp_find_freq_floor(dev, &max_freq); + if (IS_ERR(opp)) { + dev_err_probe(pci->dev, PTR_ERR(opp), + "Unable to find max freq OPP\n"); + goto err_pm_runtime_put; + } else { + ret = dev_pm_opp_set_opp(dev, opp); + } + + dev_pm_opp_put(opp); + if (ret) { + dev_err_probe(pci->dev, ret, + "Failed to set OPP for freq %lu\n", + max_freq); + goto err_pm_runtime_put; + } + } else { + /* Skip ICC init if OPP is supported as it is handled by OPP */ + ret = qcom_pcie_icc_init(pcie); + if (ret) + goto err_pm_runtime_put; + } ret = pcie->cfg->ops->get_resources(pcie); if (ret) @@ -1561,7 +1613,7 @@ static int qcom_pcie_probe(struct platform_device *pdev) goto err_phy_exit; } - qcom_pcie_icc_update(pcie); + qcom_pcie_icc_opp_update(pcie); if (pcie->mhi) qcom_pcie_init_debugfs(pcie); @@ -1586,10 +1638,14 @@ static int qcom_pcie_suspend_noirq(struct device *dev) * Set minimum bandwidth required to keep data path functional during * suspend. */ - ret = icc_set_bw(pcie->icc_mem, 0, kBps_to_icc(1)); - if (ret) { - dev_err(dev, "Failed to set bandwidth for PCIe-MEM interconnect path: %d\n", ret); - return ret; + if (pcie->icc_mem) { + ret = icc_set_bw(pcie->icc_mem, 0, kBps_to_icc(1)); + if (ret) { + dev_err(dev, + "Failed to set bandwidth for PCIe-MEM interconnect path: %d\n", + ret); + return ret; + } } /* @@ -1622,6 +1678,9 @@ static int qcom_pcie_suspend_noirq(struct device *dev) ret = icc_disable(pcie->icc_cpu); if (ret) dev_err(dev, "Failed to disable CPU-PCIe interconnect path: %d\n", ret); + + if (!pcie->icc_mem) + dev_pm_opp_set_opp(pcie->pci->dev, NULL); } return ret; } @@ -1647,7 +1706,7 @@ static int qcom_pcie_resume_noirq(struct device *dev) pcie->suspended = false; } - qcom_pcie_icc_update(pcie); + qcom_pcie_icc_opp_update(pcie); return 0; }