From patchwork Sun Feb 10 06:52:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Knut Omang X-Patchwork-Id: 10804653 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0DBE13B5 for ; Sun, 10 Feb 2019 06:55:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A5798298D7 for ; Sun, 10 Feb 2019 06:55:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7EFC02993F; Sun, 10 Feb 2019 06:55:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E5873298D7 for ; Sun, 10 Feb 2019 06:55:02 +0000 (UTC) Received: from localhost ([127.0.0.1]:55668 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gsj13-0007lr-Kc for patchwork-qemu-devel@patchwork.kernel.org; Sun, 10 Feb 2019 01:55:01 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51124) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gsizp-0006s4-AV for qemu-devel@nongnu.org; Sun, 10 Feb 2019 01:53:46 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gsizl-0000GB-GE for qemu-devel@nongnu.org; Sun, 10 Feb 2019 01:53:43 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:33670) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gsizh-00009o-E0 for qemu-devel@nongnu.org; Sun, 10 Feb 2019 01:53:38 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1A6rKHX145338; Sun, 10 Feb 2019 06:53:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=JhyUHyg20xm0hj08rhWqvd8JCx3+gFnVSaMgH7ND4+8=; b=brKgsG+8TpqALehkf28CL0YbF90d1zl1HNCnvBTjMRkuT74YJX6a3HNmU++4Eofy60PB 5ukVa8hVdUxNqyUw/a0OtsdWyQ3umwk5dUwwpUmNc2yAspgtw6Nv0XcQHWeTdUfRwpTM 8sBhdl9++aw9Dkwjs6X418962b0B91Vo3752AgasWAVn4tF+5nd5TQjc5Hk44rRnW4Hh 34BMAyHyUYVS5dt1fGvSRpnVEqBI7fVrruyAg/vAOkomEGrgJnxiFEFPzcgkmnMgxRNA v01+VIhkd/C3PYkg3i+Nor2Y1HznxTPBwhcOkD83Y2vrkOjL5yTg7RMXY0f0bq4xsWKy qg== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2qhre51t44-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 10 Feb 2019 06:53:20 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1A6rJYe008501 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 10 Feb 2019 06:53:19 GMT Received: from abhmp0018.oracle.com (abhmp0018.oracle.com [141.146.116.24]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1A6rJkp014414; Sun, 10 Feb 2019 06:53:19 GMT Received: from abi.no.oracle.com (/10.172.144.123) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Sun, 10 Feb 2019 06:53:18 +0000 From: Knut Omang To: qemu-devel@nongnu.org Date: Sun, 10 Feb 2019 07:52:59 +0100 Message-Id: <636bd37834daa1cc4c42462aeccd0de17aa370bf.1549779509.git-series.knut.omang@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9162 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902100052 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-Received-From: 141.146.126.79 Subject: [Qemu-devel] [PATCH v3 1/2] pcie: Add a simple PCIe ACS (Access Control Services) helper function X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Michael S . Tsirkin" , Tal Attaly , Knut Omang , Alex Williamson , Elijah Shakkour , Stefan Hajnoczi Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Add a helper function to add PCIe capability for Access Control Services (ACS) ACS support in the associated root port is a prerequisite to be able to do passthrough of individual functions of a device with VFIO without Alex Williamson's pcie_acs_override kernel patch or similar in the guest. Endpoints may also implement an ACS capability, but with limited features. Signed-off-by: Knut Omang Reviewed-by: Marcel Apfelbaum --- hw/pci/pcie.c | 29 +++++++++++++++++++++++++++++ include/hw/pci/pcie.h | 6 ++++++ include/hw/pci/pcie_regs.h | 4 ++++ 3 files changed, 39 insertions(+) diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c index 230478f..509632f 100644 --- a/hw/pci/pcie.c +++ b/hw/pci/pcie.c @@ -742,6 +742,14 @@ bool pcie_cap_is_arifwd_enabled(const PCIDevice *dev) PCI_EXP_DEVCTL2_ARI; } +/* Access Control Services (ACS) */ +void pcie_cap_acs_reset(PCIDevice *dev) +{ + if (dev->exp.acs_cap) { + pci_set_word(dev->config + dev->exp.acs_cap + PCI_ACS_CTRL, 0); + } +} + /************************************************************************** * pci express extended capability list management functions * uint16_t ext_cap_id (16 bit) @@ -906,3 +914,24 @@ void pcie_ats_init(PCIDevice *dev, uint16_t offset) pci_set_word(dev->wmask + dev->exp.ats_cap + PCI_ATS_CTRL, 0x800f); } + +/* ACS (Access Control Services) */ +void pcie_acs_init(PCIDevice *dev, uint16_t offset) +{ + bool is_pcie_slot = !!object_dynamic_cast(OBJECT(dev), TYPE_PCIE_SLOT); + uint16_t cap_bits = 0; + + pcie_add_capability(dev, PCI_EXT_CAP_ID_ACS, PCI_ACS_VER, offset, + PCI_ACS_SIZEOF); + dev->exp.acs_cap = offset; + + if (is_pcie_slot) { + /* Endpoints may also implement ACS, but these capabilities are */ + /* only valid for slots: */ + cap_bits = + PCI_ACS_SV | PCI_ACS_TB | PCI_ACS_RR | PCI_ACS_CR | PCI_ACS_UF; + } + + pci_set_word(dev->config + offset + PCI_ACS_CAP, cap_bits); + pci_set_word(dev->wmask + offset + PCI_ACS_CTRL, cap_bits); +} diff --git a/include/hw/pci/pcie.h b/include/hw/pci/pcie.h index 5b82a0d..4c40711 100644 --- a/include/hw/pci/pcie.h +++ b/include/hw/pci/pcie.h @@ -79,6 +79,9 @@ struct PCIExpressDevice { /* Offset of ATS capability in config space */ uint16_t ats_cap; + + /* ACS */ + uint16_t acs_cap; }; #define COMPAT_PROP_PCP "power_controller_present" @@ -116,6 +119,8 @@ void pcie_cap_flr_init(PCIDevice *dev); void pcie_cap_flr_write_config(PCIDevice *dev, uint32_t addr, uint32_t val, int len); +void pcie_cap_acs_reset(PCIDevice *dev); + /* ARI forwarding capability and control */ void pcie_cap_arifwd_init(PCIDevice *dev); void pcie_cap_arifwd_reset(PCIDevice *dev); @@ -129,6 +134,7 @@ void pcie_add_capability(PCIDevice *dev, void pcie_sync_bridge_lnk(PCIDevice *dev); void pcie_ari_init(PCIDevice *dev, uint16_t offset, uint16_t nextfn); +void pcie_acs_init(PCIDevice *dev, uint16_t offset); void pcie_dev_ser_num_init(PCIDevice *dev, uint16_t offset, uint64_t ser_num); void pcie_ats_init(PCIDevice *dev, uint16_t offset); diff --git a/include/hw/pci/pcie_regs.h b/include/hw/pci/pcie_regs.h index ad4e780..1db86b0 100644 --- a/include/hw/pci/pcie_regs.h +++ b/include/hw/pci/pcie_regs.h @@ -175,4 +175,8 @@ typedef enum PCIExpLinkWidth { PCI_ERR_COR_INTERNAL | \ PCI_ERR_COR_HL_OVERFLOW) +/* ACS */ +#define PCI_ACS_VER 0x1 +#define PCI_ACS_SIZEOF 8 + #endif /* QEMU_PCIE_REGS_H */ From patchwork Sun Feb 10 06:53:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Knut Omang X-Patchwork-Id: 10804655 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D261713B5 for ; Sun, 10 Feb 2019 06:55:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C3986298D7 for ; Sun, 10 Feb 2019 06:55:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B67E929950; Sun, 10 Feb 2019 06:55:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D3F34298D7 for ; Sun, 10 Feb 2019 06:55:06 +0000 (UTC) Received: from localhost ([127.0.0.1]:55677 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gsj18-0007nz-7C for patchwork-qemu-devel@patchwork.kernel.org; Sun, 10 Feb 2019 01:55:06 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51122) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gsizp-0006s2-AV for qemu-devel@nongnu.org; Sun, 10 Feb 2019 01:53:46 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gsizl-0000G9-GE for qemu-devel@nongnu.org; Sun, 10 Feb 2019 01:53:43 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:42708) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gsizh-00009z-E4 for qemu-devel@nongnu.org; Sun, 10 Feb 2019 01:53:38 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1A6nxtl133630; Sun, 10 Feb 2019 06:53:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=nv7Ns8pwn6X4htuJiwpWV98L81+MpaP9X/3575339Ws=; b=WjrRYZDFyIm+PNj2AXtCUFWPjXBOop9YkOJ8qTqDZsNsQPfVZueiQ0/TB+JJgUgTOTb0 EShKOPaYePiQouZPMiJYE1aDyjA6uFIZ8AmUzZGGY8wT7m2WGcksHeXuYajaJM/19fA0 Y6nPM/owI1YxwfNiq0gBywrdnvVfnoqI1hXa0+zwsX7XGYN59MoZO5Tu0qONjeNG19q7 t30hgd+fMZf/wwsnAhbEgDLKOXeh7i4VvdKGk4sQWDaaoTKERAM6zYTPLvxouIB3HI6H G6mCsebyqhMSxSDCmXvV2Z64MEfvYZrwpPtwSgIHEXZMaWc5LVcg083URrnqLO55XZez sw== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2130.oracle.com with ESMTP id 2qhrek1t2t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 10 Feb 2019 06:53:21 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1A6rLCe005464 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 10 Feb 2019 06:53:21 GMT Received: from abhmp0018.oracle.com (abhmp0018.oracle.com [141.146.116.24]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1A6rLW8002530; Sun, 10 Feb 2019 06:53:21 GMT Received: from abi.no.oracle.com (/10.172.144.123) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Sun, 10 Feb 2019 06:53:20 +0000 From: Knut Omang To: qemu-devel@nongnu.org Date: Sun, 10 Feb 2019 07:53:00 +0100 Message-Id: <04c99e3c0b56dcbfe494f668bbe11777adbadba9.1549779509.git-series.knut.omang@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9162 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902100052 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-Received-From: 156.151.31.86 Subject: [Qemu-devel] [PATCH v3 2/2] gen_pcie_root_port: Add ACS (Access Control Services) capability X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Michael S . Tsirkin" , Tal Attaly , Knut Omang , Alex Williamson , Elijah Shakkour , Stefan Hajnoczi Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Claim ACS support in the generic PCIe root port to allow passthrough of individual functions of a device to different guests (in a nested virt.setting) with VFIO. Without this patch, all functions of a device, such as all VFs of an SR/IOV device, will end up in the same IOMMU group. A similar situation occurs on Windows with Hyper-V. In the single function device case, it also has a small cosmetic benefit in that the root port itself is not grouped with the device. VFIO handles that situation in that binding rules only apply to endpoints, so it does not limit passthrough in those cases. Signed-off-by: Knut Omang Reviewed-by: Marcel Apfelbaum --- hw/pci-bridge/gen_pcie_root_port.c | 4 ++++ hw/pci-bridge/pcie_root_port.c | 4 ++++ include/hw/pci/pcie_port.h | 1 + 3 files changed, 9 insertions(+) diff --git a/hw/pci-bridge/gen_pcie_root_port.c b/hw/pci-bridge/gen_pcie_root_port.c index 9766edb..26bda73 100644 --- a/hw/pci-bridge/gen_pcie_root_port.c +++ b/hw/pci-bridge/gen_pcie_root_port.c @@ -20,6 +20,9 @@ OBJECT_CHECK(GenPCIERootPort, (obj), TYPE_GEN_PCIE_ROOT_PORT) #define GEN_PCIE_ROOT_PORT_AER_OFFSET 0x100 +#define GEN_PCIE_ROOT_PORT_ACS_OFFSET \ + (GEN_PCIE_ROOT_PORT_AER_OFFSET + PCI_ERR_SIZEOF) + #define GEN_PCIE_ROOT_PORT_MSIX_NR_VECTOR 1 typedef struct GenPCIERootPort { @@ -149,6 +152,7 @@ static void gen_rp_dev_class_init(ObjectClass *klass, void *data) rpc->interrupts_init = gen_rp_interrupts_init; rpc->interrupts_uninit = gen_rp_interrupts_uninit; rpc->aer_offset = GEN_PCIE_ROOT_PORT_AER_OFFSET; + rpc->acs_offset = GEN_PCIE_ROOT_PORT_ACS_OFFSET; } static const TypeInfo gen_rp_dev_info = { diff --git a/hw/pci-bridge/pcie_root_port.c b/hw/pci-bridge/pcie_root_port.c index 34ad767..a0b4cf7 100644 --- a/hw/pci-bridge/pcie_root_port.c +++ b/hw/pci-bridge/pcie_root_port.c @@ -47,6 +47,7 @@ static void rp_reset(DeviceState *qdev) pcie_cap_deverr_reset(d); pcie_cap_slot_reset(d); pcie_cap_arifwd_reset(d); + pcie_cap_acs_reset(d); pcie_aer_root_reset(d); pci_bridge_reset(qdev); pci_bridge_disable_base_limit(d); @@ -106,6 +107,9 @@ static void rp_realize(PCIDevice *d, Error **errp) pcie_aer_root_init(d); rp_aer_vector_update(d); + if (rpc->acs_offset) { + pcie_acs_init(d, rpc->acs_offset); + } return; err: diff --git a/include/hw/pci/pcie_port.h b/include/hw/pci/pcie_port.h index df242a0..09586f4 100644 --- a/include/hw/pci/pcie_port.h +++ b/include/hw/pci/pcie_port.h @@ -78,6 +78,7 @@ typedef struct PCIERootPortClass { int exp_offset; int aer_offset; int ssvid_offset; + int acs_offset; /* If nonzero, optional ACS capability offset */ int ssid; } PCIERootPortClass;