From patchwork Fri Sep 3 10:08:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12474057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91288C433EF for ; Fri, 3 Sep 2021 10:08:55 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4FC316108E for ; Fri, 3 Sep 2021 10:08:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 4FC316108E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.177970.323786 (Exim 4.92) (envelope-from ) id 1mM67m-0004U7-QR; Fri, 03 Sep 2021 10:08:42 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 177970.323786; Fri, 03 Sep 2021 10:08:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM67m-0004U0-My; Fri, 03 Sep 2021 10:08:42 +0000 Received: by outflank-mailman (input) for mailman id 177970; Fri, 03 Sep 2021 10:08:41 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM67l-0004Cr-9V for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:08:41 +0000 Received: from mail-ej1-x62d.google.com (unknown [2a00:1450:4864:20::62d]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id a207c60b-2f84-4a2e-8c87-e4da339ca8fd; Fri, 03 Sep 2021 10:08:36 +0000 (UTC) Received: by mail-ej1-x62d.google.com with SMTP id mf2so10990326ejb.9 for ; Fri, 03 Sep 2021 03:08:36 -0700 (PDT) Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua. [176.36.245.220]) by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Sep 2021 03:08:35 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a207c60b-2f84-4a2e-8c87-e4da339ca8fd DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZadtWS/HmdzZ6ZB6hMvTOb7/437UPtLCPee6STSCqfE=; b=YhHIT73vy+PVxx26h1vPYEo5XBj3ro3OZ8ijz9+VhFJjzLOVDwpyT4NP5euCOnWr6F sXNIdIdwveV+4JJNloPws1E3n6Qi3ChdN9OgIrXxwDb3quCHFdbymal/0yvXMNj9eLKy rpvYV/4QQUS3mNCN08ose74UFLipgvvzzaThClvTPt6m8yytTwArmYRLrDbwUdlCjlwa Us4A/LRhlNmNfGnBUb+ygPZcwZ46eX0hpKznDPYPRaQBm59/7pFQJhV/aYIUkAeTNmGW YCjjIQo1QPg+2ZCq+OnF2tXhNd9qFuj+CzZ8qJS8zb8SFxw36Od7qiX+H7XIm3NLw3Pg JKjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZadtWS/HmdzZ6ZB6hMvTOb7/437UPtLCPee6STSCqfE=; b=Uey5oSbYIwQXHH9NMZKn+7cI48afypAEdjXa55LJy1A+9R4MKMFUl49TIEpRjNybEB 13dtLALW21QIbsUBSq6WP1+gdfH3xLAKEV91ac/4OTv8HAiIpIg8TdXFNfcVyUP3iJic L63ds6uBWA3us0EY5o13sTUHj2DhQ0aZYPqDrylJqHO5A2cvntCJr+jYVCxcxs1fBnsy pTIUve6hfQIyPMNxP46vNTsYQZ4SixRVfuft2/YPEa1Oxsmt6rVFFIPIa2Uc6zeB+9/K +q0ig0nEPa2PYN6/vNRpqE/7Abwmo5hbDgrAZCYolMxnVPxLoPDHE+0V1sl8Adn15pkn QV1Q== X-Gm-Message-State: AOAM532yciggjHHlrkznHx4PQ140XKp6WbKSxiBfvBZDLj0Rv2g3sSnQ o7Nccq1vwXUrcAfJJz9Gr//naxT4rws= X-Google-Smtp-Source: ABdhPJwgZPoQj1ss4jKZJbBH8b+YlK07Jv0KEwHZKKBxDLUyLP9WFuc8uyLUXAIwk7Pj5zBg8EBp3Q== X-Received: by 2002:a17:907:362:: with SMTP id rs2mr3352436ejb.17.1630663715405; Fri, 03 Sep 2021 03:08:35 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH 1/9] vpci: Make vpci registers removal a dedicated function Date: Fri, 3 Sep 2021 13:08:23 +0300 Message-Id: <20210903100831.177748-2-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com> References: <20210903100831.177748-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko This is in preparation for dynamic assignment of the vpci register handlers depending on the domain: hwdom or guest. Signed-off-by: Oleksandr Andrushchenko --- xen/drivers/vpci/vpci.c | 7 ++++++- xen/include/xen/vpci.h | 2 ++ 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index cbd1bac7fc33..b05530f2a6b0 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -35,7 +35,7 @@ extern vpci_register_init_t *const __start_vpci_array[]; extern vpci_register_init_t *const __end_vpci_array[]; #define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array) -void vpci_remove_device(struct pci_dev *pdev) +void vpci_remove_device_registers(struct pci_dev *pdev) { spin_lock(&pdev->vpci->lock); while ( !list_empty(&pdev->vpci->handlers) ) @@ -48,6 +48,11 @@ void vpci_remove_device(struct pci_dev *pdev) xfree(r); } spin_unlock(&pdev->vpci->lock); +} + +void vpci_remove_device(struct pci_dev *pdev) +{ + vpci_remove_device_registers(pdev); xfree(pdev->vpci->msix); xfree(pdev->vpci->msi); xfree(pdev->vpci); diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index 9f5b5d52e159..b861f438cc78 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -28,6 +28,8 @@ int __must_check vpci_add_handlers(struct pci_dev *dev); /* Remove all handlers and free vpci related structures. */ void vpci_remove_device(struct pci_dev *pdev); +/* Remove all handlers for the device given. */ +void vpci_remove_device_registers(struct pci_dev *pdev); /* Add/remove a register handler. */ int __must_check vpci_add_register(struct vpci *vpci, From patchwork Fri Sep 3 10:08:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12474061 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53CC5C4332F for ; Fri, 3 Sep 2021 10:08:57 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F1BC96108E for ; Fri, 3 Sep 2021 10:08:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org F1BC96108E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.177971.323797 (Exim 4.92) (envelope-from ) id 1mM67s-0004p1-3u; Fri, 03 Sep 2021 10:08:48 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 177971.323797; Fri, 03 Sep 2021 10:08:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM67r-0004oq-VO; Fri, 03 Sep 2021 10:08:47 +0000 Received: by outflank-mailman (input) for mailman id 177971; Fri, 03 Sep 2021 10:08:46 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM67q-0004Cr-9p for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:08:46 +0000 Received: from mail-ej1-x62b.google.com (unknown [2a00:1450:4864:20::62b]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id f91edf82-e50f-4cf4-995d-7339dd9df84b; Fri, 03 Sep 2021 10:08:37 +0000 (UTC) Received: by mail-ej1-x62b.google.com with SMTP id ia27so10983815ejc.10 for ; Fri, 03 Sep 2021 03:08:37 -0700 (PDT) Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua. [176.36.245.220]) by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Sep 2021 03:08:36 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f91edf82-e50f-4cf4-995d-7339dd9df84b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EU6daQfIrHKFvAQi2eWLYJvBvTorOIHCZnBp6xeFP3g=; b=kcubAD1Yrxu5RXJEbu7bj0a07h2QblbRY+s8jSOmtT8ZkdpKKBQGCcX5WvzOVQxHbS 55eUezgwX9I0TbSWL9LToo0uUqvkN+I7fJZOt84M2creBFDahhPC94vgaafd+kHnigfs DNKtTUFK6Ibxfig4mz9B0SfhATEmqOExjB0cc+qPmEFCZiEZYlKxAY62tn4K5XHA2bm4 DluMJoVFnrYQN3CvNURCZGzNvdnDc+48eMedOJKcKFfHLc/He6w52N5HxvkUmrxcolud 3u9lfiOIhIVkADvUQQalsSwaH9feuKeguOXZ9unsAMQA/IpxrwZ8GvEpMu1xiLdcXTVR oSUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EU6daQfIrHKFvAQi2eWLYJvBvTorOIHCZnBp6xeFP3g=; b=idmU1+ovw6G5NrPkYssghrV2ChJh/LzPdwy3MG/ApEHrp5GoaavT+I5dNB60eSTktN HgWdygLn0apuTFRpUQpgLekox//f5VAGdeXKMigSq+atohC+MCeABhOcuKlCxi6UV3vr QYfBbn38/lB4r+arfq2OEJMJs3Livvfkgw1CZS6JIvZ8s8FWkaGuQTFhGQBtZuy7ClYT pmDLsu9mHGLUfcG2EYPwmQ+V7kYXqThdbxAtX8MaoTRBAdM4lkzhDqa1HtHdEcN35t97 Oleps0lYHqraC5hK8s3XUf/ZV0JXBRG+goSNiWJu6/SHn/izJln78+dkVr8J7akP0fuS voRQ== X-Gm-Message-State: AOAM531Ff1UH+vuQfiIvWYw/6W2PNoubK9ygdflTG4Or7V5FEJMBi9S/ Psl+wxbjX9KG29GVb75XzcQJgSln75o= X-Google-Smtp-Source: ABdhPJz6mPKOr3Bp3qDr/3iV61C9/zLAySQ2YnRt9O7RgvETUrpMoCcyTRoUbjFWHZd9dkA+8OI+fA== X-Received: by 2002:a17:906:1c8c:: with SMTP id g12mr3370387ejh.186.1630663716639; Fri, 03 Sep 2021 03:08:36 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH 2/9] vpci: Add hooks for PCI device assign/de-assign Date: Fri, 3 Sep 2021 13:08:24 +0300 Message-Id: <20210903100831.177748-3-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com> References: <20210903100831.177748-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko When a PCI device gets assigned/de-assigned some work on vPCI side needs to be done for that device. Introduce a pair of hooks so vPCI can handle that. Signed-off-by: Oleksandr Andrushchenko --- xen/drivers/passthrough/pci.c | 9 +++++++++ xen/drivers/vpci/vpci.c | 21 +++++++++++++++++++++ xen/include/xen/vpci.h | 16 ++++++++++++++++ 3 files changed, 46 insertions(+) diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index 25304dbe9956..deef986acbb4 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -864,6 +864,10 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus, if ( ret ) goto out; + ret = vpci_deassign_device(d, pdev); + if ( ret ) + goto out; + if ( pdev->domain == hardware_domain ) pdev->quarantine = false; @@ -1425,6 +1429,11 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag) rc = hd->platform_ops->assign_device(d, devfn, pci_to_dev(pdev), flag); } + if ( rc ) + goto done; + + rc = vpci_assign_device(d, pdev); + done: if ( rc ) printk(XENLOG_G_WARNING "%pd: assign (%pp) failed (%d)\n", diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index b05530f2a6b0..ee0ad63a3c12 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -86,6 +86,27 @@ int __hwdom_init vpci_add_handlers(struct pci_dev *pdev) return rc; } + +/* Notify vPCI that device is assigned to guest. */ +int vpci_assign_device(struct domain *d, struct pci_dev *dev) +{ + /* It only makes sense to assign for hwdom or guest domain. */ + if ( !has_vpci(d) || (d->domain_id >= DOMID_FIRST_RESERVED) ) + return 0; + + return 0; +} + +/* Notify vPCI that device is de-assigned from guest. */ +int vpci_deassign_device(struct domain *d, struct pci_dev *dev) +{ + /* It only makes sense to de-assign from hwdom or guest domain. */ + if ( !has_vpci(d) || (d->domain_id >= DOMID_FIRST_RESERVED) ) + return 0; + + return 0; +} + #endif /* __XEN__ */ static int vpci_register_cmp(const struct vpci_register *r1, diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index b861f438cc78..e7a1a09ab4c9 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -26,6 +26,12 @@ typedef int vpci_register_init_t(struct pci_dev *dev); /* Add vPCI handlers to device. */ int __must_check vpci_add_handlers(struct pci_dev *dev); +/* Notify vPCI that device is assigned to guest. */ +int __must_check vpci_assign_device(struct domain *d, struct pci_dev *dev); + +/* Notify vPCI that device is de-assigned from guest. */ +int __must_check vpci_deassign_device(struct domain *d, struct pci_dev *dev); + /* Remove all handlers and free vpci related structures. */ void vpci_remove_device(struct pci_dev *pdev); /* Remove all handlers for the device given. */ @@ -220,6 +226,16 @@ static inline int vpci_add_handlers(struct pci_dev *pdev) return 0; } +static inline int vpci_assign_device(struct domain *d, struct pci_dev *dev) +{ + return 0; +}; + +static inline int vpci_deassign_device(struct domain *d, struct pci_dev *dev) +{ + return 0; +}; + static inline void vpci_dump_msi(void) { } static inline uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, From patchwork Fri Sep 3 10:08:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12474063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76B3DC433EF for ; Fri, 3 Sep 2021 10:09:01 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 24CDB601FA for ; Fri, 3 Sep 2021 10:09:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 24CDB601FA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.177972.323808 (Exim 4.92) (envelope-from ) id 1mM67x-0005DG-De; Fri, 03 Sep 2021 10:08:53 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 177972.323808; Fri, 03 Sep 2021 10:08:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM67x-0005D5-9t; Fri, 03 Sep 2021 10:08:53 +0000 Received: by outflank-mailman (input) for mailman id 177972; Fri, 03 Sep 2021 10:08:51 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM67v-0004Cr-9x for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:08:51 +0000 Received: from mail-ej1-x635.google.com (unknown [2a00:1450:4864:20::635]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 89cdb7c8-25df-49a2-9db1-259e6a8e8cb4; Fri, 03 Sep 2021 10:08:38 +0000 (UTC) Received: by mail-ej1-x635.google.com with SMTP id me10so10971791ejb.11 for ; Fri, 03 Sep 2021 03:08:38 -0700 (PDT) Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua. [176.36.245.220]) by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Sep 2021 03:08:37 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 89cdb7c8-25df-49a2-9db1-259e6a8e8cb4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3lxrhjHMCKgFgAktbulKfLSKY9SXLgEbqYmkCK+IHbI=; b=Cv6kNVKYxIEVmbcaX8DIKNQHN8wBj6cryOD0sKuPJKv/+o8YHRTdPVgVqD77l1hCb6 vatzSz/C6yHUi7ZQZQ1JJlU9uCyOpq+GVxJBb9jeFDqhIQnX1cbkCtJinWZEU8eqJTQb jio/WmVPkE2k/zzaVXS7GL4dFAA2E0ouwzXpQCTsyFf2pUaDlvQFEaS78sYbu8YIPTXR U0FzaEvsPtV9sf174mXGL7Qw5/UuezecIBC+kpvG2EDzz17a/p2ZqvM6XZnGtCvqk3aL JIFwtBwu3xMljkBTPCWgfBV3fdb37XBeKMLOsqDZpBQznPnsEXb1R1MSjDFX+LUX9qeG nioA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3lxrhjHMCKgFgAktbulKfLSKY9SXLgEbqYmkCK+IHbI=; b=Ar3aSlPCK0GtA4sCgxbYDimL2lZEujfGD2CTEiPA4eS5hh2KiOr+ahOg/9X58VTFh+ v5lR3rXR1PGqZBPlv6NyAtoXOKwDm/GlvPQORD1cv9rpQng+6BEL/CrL/tjQaIaeOsLx RvUcwQiWdrc86goE79/rqqttWHovaGPNXawkzszYlLSZkzIyloeq+35lhBDgKREonL2E HY7eWCxYx2OJ1YAx3K2dVxroEccnwMuJyEQNYtGy25KnLRTzVJxrF6wWVINKeRkfq28i zk5netoP+L1mM+0e/Z1PeN9DzxSn+js6BPlupxFg73mQJCkXFXuJlL5oMNeuWRUsOoh5 FBug== X-Gm-Message-State: AOAM531FhXhUDauNFwiRD38UU1Lwv98u1OTOwNGaZvrk9+LAgVT1XWyj NZgM9J3TcaXszgvC0eIpF50lRxd6098= X-Google-Smtp-Source: ABdhPJydxNWuh4lL4oZ/Fi+pB8Xp26xfg9TIMj2hvhsDyPP/rb6TmJbYwxmJmWH3EGwe4uJeauyAdQ== X-Received: by 2002:a17:906:ff41:: with SMTP id zo1mr3148886ejb.525.1630663717693; Fri, 03 Sep 2021 03:08:37 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH 3/9] vpci/header: Move register assignments from init_bars Date: Fri, 3 Sep 2021 13:08:25 +0300 Message-Id: <20210903100831.177748-4-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com> References: <20210903100831.177748-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko This is in preparation for dynamic assignment of the vpci register handlers depending on the domain: hwdom or guest. Signed-off-by: Oleksandr Andrushchenko --- xen/drivers/vpci/header.c | 83 ++++++++++++++++++++++++++------------- 1 file changed, 56 insertions(+), 27 deletions(-) diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index f8cd55e7c024..31bca7a12942 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -445,6 +445,55 @@ static void rom_write(const struct pci_dev *pdev, unsigned int reg, rom->addr = val & PCI_ROM_ADDRESS_MASK; } +static int add_bar_handlers(struct pci_dev *pdev) +{ + unsigned int i; + struct vpci_header *header = &pdev->vpci->header; + struct vpci_bar *bars = header->bars; + int rc; + + /* Setup a handler for the command register. */ + rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND, + 2, header); + if ( rc ) + return rc; + + if ( pdev->ignore_bars ) + return 0; + + for ( i = 0; i < PCI_HEADER_NORMAL_NR_BARS + 1; i++ ) + { + if ( (bars[i].type == VPCI_BAR_IO) || (bars[i].type == VPCI_BAR_EMPTY) ) + continue; + + if ( bars[i].type == VPCI_BAR_ROM ) + { + unsigned int rom_reg; + uint8_t header_type = pci_conf_read8(pdev->sbdf, + PCI_HEADER_TYPE) & 0x7f; + if ( header_type == PCI_HEADER_TYPE_NORMAL ) + rom_reg = PCI_ROM_ADDRESS; + else + rom_reg = PCI_ROM_ADDRESS1; + rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write, + rom_reg, 4, &bars[i]); + if ( rc ) + return rc; + } + else + { + uint8_t reg = PCI_BASE_ADDRESS_0 + i * 4; + + /* This is either VPCI_BAR_MEM32 or VPCI_BAR_MEM64_{LO|HI}. */ + rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg, + 4, &bars[i]); + if ( rc ) + return rc; + } + } + return 0; +} + static int init_bars(struct pci_dev *pdev) { uint16_t cmd; @@ -470,14 +519,8 @@ static int init_bars(struct pci_dev *pdev) return -EOPNOTSUPP; } - /* Setup a handler for the command register. */ - rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND, - 2, header); - if ( rc ) - return rc; - if ( pdev->ignore_bars ) - return 0; + return add_bar_handlers(pdev); /* Disable memory decoding before sizing. */ cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND); @@ -492,14 +535,6 @@ static int init_bars(struct pci_dev *pdev) if ( i && bars[i - 1].type == VPCI_BAR_MEM64_LO ) { bars[i].type = VPCI_BAR_MEM64_HI; - rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg, - 4, &bars[i]); - if ( rc ) - { - pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd); - return rc; - } - continue; } @@ -532,14 +567,6 @@ static int init_bars(struct pci_dev *pdev) bars[i].addr = addr; bars[i].size = size; bars[i].prefetchable = val & PCI_BASE_ADDRESS_MEM_PREFETCH; - - rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg, 4, - &bars[i]); - if ( rc ) - { - pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd); - return rc; - } } /* Check expansion ROM. */ @@ -553,11 +580,13 @@ static int init_bars(struct pci_dev *pdev) rom->addr = addr; header->rom_enabled = pci_conf_read32(pdev->sbdf, rom_reg) & PCI_ROM_ADDRESS_ENABLE; + } - rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write, rom_reg, - 4, rom); - if ( rc ) - rom->type = VPCI_BAR_EMPTY; + rc = add_bar_handlers(pdev); + if ( rc ) + { + pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd); + return rc; } return (cmd & PCI_COMMAND_MEMORY) ? modify_bars(pdev, cmd, false) : 0; From patchwork Fri Sep 3 10:08:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12474065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FA86C433F5 for ; Fri, 3 Sep 2021 10:09:06 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 28CC5601FA for ; Fri, 3 Sep 2021 10:09:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 28CC5601FA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.177975.323819 (Exim 4.92) (envelope-from ) id 1mM681-0005gd-Ne; Fri, 03 Sep 2021 10:08:57 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 177975.323819; Fri, 03 Sep 2021 10:08:57 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM681-0005gQ-Jh; Fri, 03 Sep 2021 10:08:57 +0000 Received: by outflank-mailman (input) for mailman id 177975; Fri, 03 Sep 2021 10:08:56 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM680-0004Cr-AA for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:08:56 +0000 Received: from mail-ed1-x533.google.com (unknown [2a00:1450:4864:20::533]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id cfb4143e-4d10-43e8-a103-28cf681063e9; Fri, 03 Sep 2021 10:08:40 +0000 (UTC) Received: by mail-ed1-x533.google.com with SMTP id z19so7227660edi.9 for ; Fri, 03 Sep 2021 03:08:40 -0700 (PDT) Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua. [176.36.245.220]) by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Sep 2021 03:08:38 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cfb4143e-4d10-43e8-a103-28cf681063e9 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xpLp7pLNUQ/DHxUjEhr/yTY0NKqDW6A0T3pZsizfVtE=; b=aS6tQADQYeExZDhOABC9iBB8GoJc3IuoFCRob+Cctdwhth+nveel2KGxkARVYlU6Jd Qgnsw8axED8hMhxC2pYHQ9oNoSo/Olz1veZ4TZuqbKzQZivPkh6by4D4cHiKbYkHSK91 YRf3tKX+OFy/jc8fvxtX6h9mWPRbWAPECFFsxFE+A6QKglW5XRGmbuMg67a0TGkoDIJw /eSmzbrBWGArGKWd2yxvzNtdGv662yFyuBYuegN1QYkQSgbHkEqDsAUOzpcLfZcumjrq cZ2XfDNZRq17L5VjX+W5pDzGFNH7tT+468gnBCcwp09gPnFouOqA3oPEThxmheEzgx36 8JiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xpLp7pLNUQ/DHxUjEhr/yTY0NKqDW6A0T3pZsizfVtE=; b=hGnApIAP71TtOCLXUjUKPy4WIrMPkur5k+NPSUeWn7p/Bf8CxvczUWGnlwjKqpPQcP 9Y0eVWNEsj5402YA2Gbn//AJYvDDKoVXzV9QDF/Ps+iNmInd9cB0Cc+SOoFZS8/asqwT tTfsJeeSHJaC2dhpcFjiFNFectWoPuWWPGg0Z7aGKDHciQp/Z5hDb51wcF6vB7kI2N1r o0cFz+f9LWpM7xDisrSAzoVOdv0ooVvwsKGxRdZ2MfEA96Rzwy9NPLeVRX8dmbhQM0j6 ZYj5JkfP2reHEmbEYz3M/2N8AeaN+RpskyCOLs7iDYTcCtetuKAJNg2RtePNh6AlqiEw d1Mw== X-Gm-Message-State: AOAM5334/en9MEUJvPwecu2YmyDa6bOMJinsNafy5DPz5/8RswTn4lXw mt9V6eUOxVENPqJZI0u2GjZF5lc+2Bo= X-Google-Smtp-Source: ABdhPJwZyi4jY7+BYhHRUtcHJ71NsVbFBf7jRjXcdY+sWwfFplhPMgxjOeq7kjOI9krtIeEBDoMxMA== X-Received: by 2002:aa7:da8c:: with SMTP id q12mr3199500eds.380.1630663718806; Fri, 03 Sep 2021 03:08:38 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH 4/9] vpci/header: Add and remove register handlers dynamically Date: Fri, 3 Sep 2021 13:08:26 +0300 Message-Id: <20210903100831.177748-5-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com> References: <20210903100831.177748-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko Add relevant vpci register handlers when assigning PCI device to a domain and remove those when de-assigning. This allows having different handlers for different domains, e.g. hwdom and other guests. Use stubs for guest domains for now. Signed-off-by: Oleksandr Andrushchenko --- xen/drivers/vpci/header.c | 78 +++++++++++++++++++++++++++++++++++---- xen/drivers/vpci/vpci.c | 4 +- xen/include/xen/vpci.h | 4 ++ 3 files changed, 76 insertions(+), 10 deletions(-) diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index 31bca7a12942..5218b1af247e 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -397,6 +397,17 @@ static void bar_write(const struct pci_dev *pdev, unsigned int reg, pci_conf_write32(pdev->sbdf, reg, val); } +static void guest_bar_write(const struct pci_dev *pdev, unsigned int reg, + uint32_t val, void *data) +{ +} + +static uint32_t guest_bar_read(const struct pci_dev *pdev, unsigned int reg, + void *data) +{ + return 0xffffffff; +} + static void rom_write(const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data) { @@ -445,14 +456,25 @@ static void rom_write(const struct pci_dev *pdev, unsigned int reg, rom->addr = val & PCI_ROM_ADDRESS_MASK; } -static int add_bar_handlers(struct pci_dev *pdev) +static void guest_rom_write(const struct pci_dev *pdev, unsigned int reg, + uint32_t val, void *data) +{ +} + +static uint32_t guest_rom_read(const struct pci_dev *pdev, unsigned int reg, + void *data) +{ + return 0xffffffff; +} + +static int add_bar_handlers(struct pci_dev *pdev, bool is_hwdom) { unsigned int i; struct vpci_header *header = &pdev->vpci->header; struct vpci_bar *bars = header->bars; int rc; - /* Setup a handler for the command register. */ + /* Setup a handler for the command register: same for hwdom and guests. */ rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND, 2, header); if ( rc ) @@ -475,8 +497,13 @@ static int add_bar_handlers(struct pci_dev *pdev) rom_reg = PCI_ROM_ADDRESS; else rom_reg = PCI_ROM_ADDRESS1; - rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write, - rom_reg, 4, &bars[i]); + if ( is_hwdom ) + rc = vpci_add_register(pdev->vpci, vpci_hw_read32, rom_write, + rom_reg, 4, &bars[i]); + else + rc = vpci_add_register(pdev->vpci, + guest_rom_read, guest_rom_write, + rom_reg, 4, &bars[i]); if ( rc ) return rc; } @@ -485,8 +512,13 @@ static int add_bar_handlers(struct pci_dev *pdev) uint8_t reg = PCI_BASE_ADDRESS_0 + i * 4; /* This is either VPCI_BAR_MEM32 or VPCI_BAR_MEM64_{LO|HI}. */ - rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg, - 4, &bars[i]); + if ( is_hwdom ) + rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, + reg, 4, &bars[i]); + else + rc = vpci_add_register(pdev->vpci, + guest_bar_read, guest_bar_write, + reg, 4, &bars[i]); if ( rc ) return rc; } @@ -520,7 +552,7 @@ static int init_bars(struct pci_dev *pdev) } if ( pdev->ignore_bars ) - return add_bar_handlers(pdev); + return add_bar_handlers(pdev, true); /* Disable memory decoding before sizing. */ cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND); @@ -582,7 +614,7 @@ static int init_bars(struct pci_dev *pdev) PCI_ROM_ADDRESS_ENABLE; } - rc = add_bar_handlers(pdev); + rc = add_bar_handlers(pdev, true); if ( rc ) { pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd); @@ -593,6 +625,36 @@ static int init_bars(struct pci_dev *pdev) } REGISTER_VPCI_INIT(init_bars, VPCI_PRIORITY_MIDDLE); +int vpci_bar_add_handlers(const struct domain *d, struct pci_dev *pdev) +{ + int rc; + + /* Remove previously added registers. */ + vpci_remove_device_registers(pdev); + + /* It only makes sense to add registers for hwdom or guest domain. */ + if ( d->domain_id >= DOMID_FIRST_RESERVED ) + return 0; + + if ( is_hardware_domain(d) ) + rc = add_bar_handlers(pdev, true); + else + rc = add_bar_handlers(pdev, false); + + if ( rc ) + gprintk(XENLOG_ERR, + "%pp: failed to add BAR handlers for dom%d\n", &pdev->sbdf, + d->domain_id); + return rc; +} + +int vpci_bar_remove_handlers(const struct domain *d, struct pci_dev *pdev) +{ + /* Remove previously added registers. */ + vpci_remove_device_registers(pdev); + return 0; +} + /* * Local variables: * mode: C diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index ee0ad63a3c12..4530313f01e7 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -94,7 +94,7 @@ int vpci_assign_device(struct domain *d, struct pci_dev *dev) if ( !has_vpci(d) || (d->domain_id >= DOMID_FIRST_RESERVED) ) return 0; - return 0; + return vpci_bar_add_handlers(d, dev); } /* Notify vPCI that device is de-assigned from guest. */ @@ -104,7 +104,7 @@ int vpci_deassign_device(struct domain *d, struct pci_dev *dev) if ( !has_vpci(d) || (d->domain_id >= DOMID_FIRST_RESERVED) ) return 0; - return 0; + return vpci_bar_remove_handlers(d, dev); } #endif /* __XEN__ */ diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index e7a1a09ab4c9..4aa2941a1081 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -63,6 +63,10 @@ uint32_t vpci_hw_read32(const struct pci_dev *pdev, unsigned int reg, */ bool __must_check vpci_process_pending(struct vcpu *v); +/* Add/remove BAR handlers for a domain. */ +int vpci_bar_add_handlers(const struct domain *d, struct pci_dev *pdev); +int vpci_bar_remove_handlers(const struct domain *d, struct pci_dev *pdev); + struct vpci { /* List of vPCI handlers for a device. */ struct list_head handlers; From patchwork Fri Sep 3 10:08:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12474067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28101C433F5 for ; Fri, 3 Sep 2021 10:09:09 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D6A3F601FA for ; Fri, 3 Sep 2021 10:09:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D6A3F601FA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.177978.323830 (Exim 4.92) (envelope-from ) id 1mM686-0006Ar-8e; Fri, 03 Sep 2021 10:09:02 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 177978.323830; Fri, 03 Sep 2021 10:09:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM686-0006AD-3I; Fri, 03 Sep 2021 10:09:02 +0000 Received: by outflank-mailman (input) for mailman id 177978; Fri, 03 Sep 2021 10:09:01 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM685-0004Cr-AL for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:09:01 +0000 Received: from mail-ed1-x535.google.com (unknown [2a00:1450:4864:20::535]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 37d65df0-7563-4c10-bccc-ae1dcb106e2e; Fri, 03 Sep 2021 10:08:40 +0000 (UTC) Received: by mail-ed1-x535.google.com with SMTP id l6so7262298edb.7 for ; Fri, 03 Sep 2021 03:08:40 -0700 (PDT) Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua. [176.36.245.220]) by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Sep 2021 03:08:39 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 37d65df0-7563-4c10-bccc-ae1dcb106e2e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=s1iI/pfNat4kQKr5zc/XN1JzcSSy61j1M1hI6YBK6Xs=; b=gBoRCW32iLvtnLPg39hDXD7cmyijkqaXRSJbAXbKOPEXyhDl0dgAaIZEz7n/8UZi4w xyP3FtbSUjU/63qIdjQ0mfR/gRCcCNLbo5+ikGBVrFTa2CYIa89ObRAV5x0LVELdz3K0 gMdRG0CMUMNI1PoCqvx9gcF4dXo8MyFy60K9OjUcvHXSWX5viFQP5LDlfcUWYNyGXtIP zNLijBaZRzBMnCN3hJ3eHRipmudvgGiOpZcTXSfGXwt6qisNkUKhee+0WE34G69/e8mr HfTRJZRDqlnW++VApih2L91ZVfsGE+GAThkzBOdxFK99Io93rNBqo/1UlvVCckHjsX+9 O+Wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=s1iI/pfNat4kQKr5zc/XN1JzcSSy61j1M1hI6YBK6Xs=; b=ZH36fSUXi+4e9k9fED+FPhzapioOgKCe7C+6cxPN0UuaFxp4KN4lXQD87Uvbho9dbc ZZx86Su6a9Z8uVYtlcN4+kEjDJCgZ66LjCIPqAbzSjCFy51ucPxamBYSMFI7FJLj8TMS oFcpaTx+5DsPQW5xkXwvU8PHRN9REpyvveUCQtO8eWkMH1qjy05X2+/WO6c4SMpjXVFD zxI/g3d0pl6junt10RLbld6CQNUuUK2yZdwoOe+E11YUqcxfMLjHCIHlway3mXLBNnrk EKO+JALJdLx0ekrKEmzq1zohANVnfEMuKETO5Hrekci6wFI2BUOyMBmsV0fxqJSQNEnb rXUA== X-Gm-Message-State: AOAM5320yZ6ZaDujZgZ7muxVMPZlWCCMjj6PlaSzBYzth8romaK4UiCc VI2SoTl4QY8DO4p+o22aqXANV8JE758= X-Google-Smtp-Source: ABdhPJxrROUzQ4fz2kKiNwMn7jDNiM+KTMqFpM3OJ8BQuHdw/HwgIHYajSQwwLbnO5MBR/YkklXeKg== X-Received: by 2002:a05:6402:424c:: with SMTP id g12mr3162442edb.37.1630663719821; Fri, 03 Sep 2021 03:08:39 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH 5/9] vpci/header: Implement guest BAR register handlers Date: Fri, 3 Sep 2021 13:08:27 +0300 Message-Id: <20210903100831.177748-6-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com> References: <20210903100831.177748-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko Emulate guest BAR register values: this allows creating a guest view of the registers and emulates size and properties probe as it is done during PCI device enumeration by the guest. ROM BAR is only handled for the hardware domain and for guest domains there is a stub: at the moment PCI expansion ROM is x86 only, so it might not be used by other architectures without emulating x86. Other use-cases may include using that expansion ROM before Xen boots, hence no emulation is needed in Xen itself. Or when a guest wants to use the ROM code which seems to be rare. Signed-off-by: Oleksandr Andrushchenko --- xen/drivers/vpci/header.c | 69 +++++++++++++++++++++++++++++++++++++- xen/include/xen/pci_regs.h | 1 + xen/include/xen/vpci.h | 3 ++ 3 files changed, 72 insertions(+), 1 deletion(-) diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index 5218b1af247e..793f79ece831 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -400,12 +400,72 @@ static void bar_write(const struct pci_dev *pdev, unsigned int reg, static void guest_bar_write(const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data) { + struct vpci_bar *bar = data; + bool hi = false; + + if ( bar->type == VPCI_BAR_MEM64_HI ) + { + ASSERT(reg > PCI_BASE_ADDRESS_0); + bar--; + hi = true; + } + else + val &= PCI_BASE_ADDRESS_MEM_MASK; + bar->guest_addr &= ~(0xffffffffull << (hi ? 32 : 0)); + bar->guest_addr |= (uint64_t)val << (hi ? 32 : 0); } static uint32_t guest_bar_read(const struct pci_dev *pdev, unsigned int reg, void *data) { - return 0xffffffff; + struct vpci_bar *bar = data; + uint32_t val; + bool hi = false; + + switch ( bar->type ) + { + case VPCI_BAR_MEM64_HI: + ASSERT(reg > PCI_BASE_ADDRESS_0); + bar--; + hi = true; + /* fallthrough */ + case VPCI_BAR_MEM64_LO: + { + if ( hi ) + val = bar->guest_addr >> 32; + else + val = bar->guest_addr & 0xffffffff; + if ( (val & PCI_BASE_ADDRESS_MEM_MASK_32) == PCI_BASE_ADDRESS_MEM_MASK_32 ) + { + /* Guests detects BAR's properties and sizes. */ + if ( hi ) + val = bar->size >> 32; + else + val = 0xffffffff & ~(bar->size - 1); + } + if ( !hi ) + { + val |= PCI_BASE_ADDRESS_MEM_TYPE_64; + val |= bar->prefetchable ? PCI_BASE_ADDRESS_MEM_PREFETCH : 0; + } + bar->guest_addr &= ~(0xffffffffull << (hi ? 32 : 0)); + bar->guest_addr |= (uint64_t)val << (hi ? 32 : 0); + break; + } + case VPCI_BAR_MEM32: + { + val = bar->guest_addr; + if ( (val & PCI_BASE_ADDRESS_MEM_MASK_32) == PCI_BASE_ADDRESS_MEM_MASK_32 ) + val = 0xffffffff & ~(bar->size - 1); + val |= PCI_BASE_ADDRESS_MEM_TYPE_32; + val |= bar->prefetchable ? PCI_BASE_ADDRESS_MEM_PREFETCH : 0; + break; + } + default: + val = bar->guest_addr; + break; + } + return val; } static void rom_write(const struct pci_dev *pdev, unsigned int reg, @@ -522,6 +582,13 @@ static int add_bar_handlers(struct pci_dev *pdev, bool is_hwdom) if ( rc ) return rc; } + /* + * It is neither safe nor secure to initialize guest's view of the BARs + * with real values which are used by the hardware domain, so assign + * all zeros to guest's view of the BARs, so the guest can perform + * proper PCI device enumeration and assign BARs on its own. + */ + bars[i].guest_addr = 0; } return 0; } diff --git a/xen/include/xen/pci_regs.h b/xen/include/xen/pci_regs.h index cc4ee3b83e5c..038eb18c5357 100644 --- a/xen/include/xen/pci_regs.h +++ b/xen/include/xen/pci_regs.h @@ -103,6 +103,7 @@ #define PCI_BASE_ADDRESS_MEM_TYPE_64 0x04 /* 64 bit address */ #define PCI_BASE_ADDRESS_MEM_PREFETCH 0x08 /* prefetchable? */ #define PCI_BASE_ADDRESS_MEM_MASK (~0x0fUL) +#define PCI_BASE_ADDRESS_MEM_MASK_32 (~0x0fU) #define PCI_BASE_ADDRESS_IO_MASK (~0x03UL) /* bit 1 is reserved if address_space = 1 */ diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index 4aa2941a1081..db86b8e7fa3c 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -77,7 +77,10 @@ struct vpci { struct vpci_header { /* Information about the PCI BARs of this device. */ struct vpci_bar { + /* Physical view of the BAR. */ uint64_t addr; + /* Guest view of the BAR. */ + uint64_t guest_addr; uint64_t size; enum { VPCI_BAR_EMPTY, From patchwork Fri Sep 3 10:08:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12474069 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7106C433F5 for ; Fri, 3 Sep 2021 10:09:15 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7F1696108E for ; Fri, 3 Sep 2021 10:09:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7F1696108E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.177982.323841 (Exim 4.92) (envelope-from ) id 1mM68B-0006or-Lg; Fri, 03 Sep 2021 10:09:07 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 177982.323841; Fri, 03 Sep 2021 10:09:07 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM68B-0006od-HV; Fri, 03 Sep 2021 10:09:07 +0000 Received: by outflank-mailman (input) for mailman id 177982; Fri, 03 Sep 2021 10:09:06 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM68A-0004Cr-Ae for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:09:06 +0000 Received: from mail-ej1-x633.google.com (unknown [2a00:1450:4864:20::633]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 90692ff9-f747-4eed-8a0f-59677580be6d; Fri, 03 Sep 2021 10:08:42 +0000 (UTC) Received: by mail-ej1-x633.google.com with SMTP id a25so11014156ejv.6 for ; Fri, 03 Sep 2021 03:08:42 -0700 (PDT) Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua. [176.36.245.220]) by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Sep 2021 03:08:40 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 90692ff9-f747-4eed-8a0f-59677580be6d DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PguLDO+PDisBHZWiq5SMa+3ZkndR3NGesmKLiuatRS8=; b=gvhPjQoucEcq+9aAL21Wr1DWTsBBuSYkSRqt2KCkvaHEiq66AaoEH4VEGm0T6rw7an eNLQMtKsECkxAj541srNzEB+llBTYXQJjMGYBcFjVb7/pNGvYqRgNv42tI5IQJQm63rf oCVEBOpCac6Lbf0xITzzcrfyNtd9pcv5tkjAn5lmD5IZLszS7BGsr2ifeUWBpqzOd4Aq 8NhQxw3gLiIIWNZm1UGUBRrAl+24kJSXnaoRtcC4T59AZnJmalxLd/eSfBP5faMAjwiD tEzMHIJ4CDVHyHqvObg1+rtH46Gc+P/t/PtvGpW1Bds3YAdPHoG13TmR9RNbg+D2kVPZ +PQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PguLDO+PDisBHZWiq5SMa+3ZkndR3NGesmKLiuatRS8=; b=tSPiQuPMmxsUDFhHlkZ83e/4oPxp4CwtNiMnM4BG0dwQzT/QjwUjq9RHSXdsE4oN6b TJwRwUe5xrRO+fYOxxnryU0Xhf4flVRT/Y42KEhvCnFf29Mv3Oc9uq2u6zqEbRZSUKD7 H4GcNz5ja+bMQgaSSeOYE1UsVMd41V55vXufrVHgPyXSO8nW5vLcNV13MelEKm/RUfrj exwrq75nDzxsYPtfU68V/2K4hJs6w4RpeNfXpSely7PfVLY9ZXeHNukQ5HLOPoWLqcKA dcsT9jAJKx73nG8zLqjSQ8NElrSMrcyczltY/RyYq88mJjW2vH7qt1esatD8rT80YgYz tdsw== X-Gm-Message-State: AOAM533RE6MO9/iDYyOSdcsGsLCM9oRMQuPIHq7jlojE8mv5ZvMiwyCu E4m8LkZBVuXbcMgUqRmTR15VicZvF18= X-Google-Smtp-Source: ABdhPJyl+Tub9JIMSY/l/hoFxplCzEScXDX6jTh8LYN2njGF7bl9ybauJNv5gjjSPitXDu2IPjei+w== X-Received: by 2002:a17:906:4a8e:: with SMTP id x14mr3341520eju.389.1630663720860; Fri, 03 Sep 2021 03:08:40 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH 6/9] vpci/header: Handle p2m range sets per BAR Date: Fri, 3 Sep 2021 13:08:28 +0300 Message-Id: <20210903100831.177748-7-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com> References: <20210903100831.177748-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko Instead of handling a single range set, that contains all the memory regions of all the BARs and ROM, have them per BAR. This is in preparation of making non-identity mappings in p2m for the MMIOs/ROM. Signed-off-by: Oleksandr Andrushchenko --- xen/drivers/vpci/header.c | 172 ++++++++++++++++++++++++++------------ xen/include/xen/vpci.h | 3 +- 2 files changed, 122 insertions(+), 53 deletions(-) diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index 793f79ece831..7f54199a3894 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -131,49 +131,75 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd, bool vpci_process_pending(struct vcpu *v) { - if ( v->vpci.mem ) + if ( v->vpci.num_mem_ranges ) { struct map_data data = { .d = v->domain, .map = v->vpci.cmd & PCI_COMMAND_MEMORY, }; - int rc = rangeset_consume_ranges(v->vpci.mem, map_range, &data); + struct pci_dev *pdev = v->vpci.pdev; + struct vpci_header *header = &pdev->vpci->header; + unsigned int i; - if ( rc == -ERESTART ) - return true; + for ( i = 0; i < ARRAY_SIZE(header->bars); i++ ) + { + struct vpci_bar *bar = &header->bars[i]; + int rc; - spin_lock(&v->vpci.pdev->vpci->lock); - /* Disable memory decoding unconditionally on failure. */ - modify_decoding(v->vpci.pdev, - rc ? v->vpci.cmd & ~PCI_COMMAND_MEMORY : v->vpci.cmd, - !rc && v->vpci.rom_only); - spin_unlock(&v->vpci.pdev->vpci->lock); + if ( !bar->mem ) + continue; - rangeset_destroy(v->vpci.mem); - v->vpci.mem = NULL; - if ( rc ) - /* - * FIXME: in case of failure remove the device from the domain. - * Note that there might still be leftover mappings. While this is - * safe for Dom0, for DomUs the domain will likely need to be - * killed in order to avoid leaking stale p2m mappings on - * failure. - */ - vpci_remove_device(v->vpci.pdev); + rc = rangeset_consume_ranges(bar->mem, map_range, &data); + + if ( rc == -ERESTART ) + return true; + + spin_lock(&pdev->vpci->lock); + /* Disable memory decoding unconditionally on failure. */ + modify_decoding(pdev, + rc ? v->vpci.cmd & ~PCI_COMMAND_MEMORY : v->vpci.cmd, + !rc && v->vpci.rom_only); + spin_unlock(&pdev->vpci->lock); + + rangeset_destroy(bar->mem); + bar->mem = NULL; + v->vpci.num_mem_ranges--; + if ( rc ) + /* + * FIXME: in case of failure remove the device from the domain. + * Note that there might still be leftover mappings. While this is + * safe for Dom0, for DomUs the domain will likely need to be + * killed in order to avoid leaking stale p2m mappings on + * failure. + */ + vpci_remove_device(pdev); + } } return false; } static int __init apply_map(struct domain *d, const struct pci_dev *pdev, - struct rangeset *mem, uint16_t cmd) + uint16_t cmd) { struct map_data data = { .d = d, .map = true }; - int rc; + struct vpci_header *header = &pdev->vpci->header; + int rc = 0; + unsigned int i; + + for ( i = 0; i < ARRAY_SIZE(header->bars); i++ ) + { + struct vpci_bar *bar = &header->bars[i]; - while ( (rc = rangeset_consume_ranges(mem, map_range, &data)) == -ERESTART ) - process_pending_softirqs(); - rangeset_destroy(mem); + if ( !bar->mem ) + continue; + + while ( (rc = rangeset_consume_ranges(bar->mem, map_range, + &data)) == -ERESTART ) + process_pending_softirqs(); + rangeset_destroy(bar->mem); + bar->mem = NULL; + } if ( !rc ) modify_decoding(pdev, cmd, false); @@ -181,7 +207,7 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev, } static void defer_map(struct domain *d, struct pci_dev *pdev, - struct rangeset *mem, uint16_t cmd, bool rom_only) + uint16_t cmd, bool rom_only, uint8_t num_mem_ranges) { struct vcpu *curr = current; @@ -192,9 +218,9 @@ static void defer_map(struct domain *d, struct pci_dev *pdev, * started for the same device if the domain is not well-behaved. */ curr->vpci.pdev = pdev; - curr->vpci.mem = mem; curr->vpci.cmd = cmd; curr->vpci.rom_only = rom_only; + curr->vpci.num_mem_ranges = num_mem_ranges; /* * Raise a scheduler softirq in order to prevent the guest from resuming * execution with pending mapping operations, to trigger the invocation @@ -206,42 +232,47 @@ static void defer_map(struct domain *d, struct pci_dev *pdev, static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) { struct vpci_header *header = &pdev->vpci->header; - struct rangeset *mem = rangeset_new(NULL, NULL, 0); struct pci_dev *tmp, *dev = NULL; const struct vpci_msix *msix = pdev->vpci->msix; - unsigned int i; + unsigned int i, j; int rc; - - if ( !mem ) - return -ENOMEM; + uint8_t num_mem_ranges; /* - * Create a rangeset that represents the current device BARs memory region + * Create a rangeset per BAR that represents the current device memory region * and compare it against all the currently active BAR memory regions. If * an overlap is found, subtract it from the region to be mapped/unmapped. * - * First fill the rangeset with all the BARs of this device or with the ROM + * First fill the rangesets with all the BARs of this device or with the ROM * BAR only, depending on whether the guest is toggling the memory decode * bit of the command register, or the enable bit of the ROM BAR register. */ for ( i = 0; i < ARRAY_SIZE(header->bars); i++ ) { - const struct vpci_bar *bar = &header->bars[i]; + struct vpci_bar *bar = &header->bars[i]; unsigned long start = PFN_DOWN(bar->addr); unsigned long end = PFN_DOWN(bar->addr + bar->size - 1); + bar->mem = NULL; + if ( !MAPPABLE_BAR(bar) || (rom_only ? bar->type != VPCI_BAR_ROM : (bar->type == VPCI_BAR_ROM && !header->rom_enabled)) ) continue; - rc = rangeset_add_range(mem, start, end); + bar->mem = rangeset_new(NULL, NULL, 0); + if ( !bar->mem ) + { + rc = -ENOMEM; + goto fail; + } + + rc = rangeset_add_range(bar->mem, start, end); if ( rc ) { printk(XENLOG_G_WARNING "Failed to add [%lx, %lx]: %d\n", start, end, rc); - rangeset_destroy(mem); - return rc; + goto fail; } } @@ -252,14 +283,21 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) unsigned long end = PFN_DOWN(vmsix_table_addr(pdev->vpci, i) + vmsix_table_size(pdev->vpci, i) - 1); - rc = rangeset_remove_range(mem, start, end); - if ( rc ) + for ( j = 0; j < ARRAY_SIZE(header->bars); j++ ) { - printk(XENLOG_G_WARNING - "Failed to remove MSIX table [%lx, %lx]: %d\n", - start, end, rc); - rangeset_destroy(mem); - return rc; + const struct vpci_bar *bar = &header->bars[j]; + + if ( !bar->mem ) + continue; + + rc = rangeset_remove_range(bar->mem, start, end); + if ( rc ) + { + printk(XENLOG_G_WARNING + "Failed to remove MSIX table [%lx, %lx]: %d\n", + start, end, rc); + goto fail; + } } } @@ -291,7 +329,8 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) unsigned long start = PFN_DOWN(bar->addr); unsigned long end = PFN_DOWN(bar->addr + bar->size - 1); - if ( !bar->enabled || !rangeset_overlaps_range(mem, start, end) || + if ( !bar->enabled || + !rangeset_overlaps_range(bar->mem, start, end) || /* * If only the ROM enable bit is toggled check against other * BARs in the same device for overlaps, but not against the @@ -300,13 +339,12 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) (rom_only && tmp == pdev && bar->type == VPCI_BAR_ROM) ) continue; - rc = rangeset_remove_range(mem, start, end); + rc = rangeset_remove_range(bar->mem, start, end); if ( rc ) { printk(XENLOG_G_WARNING "Failed to remove [%lx, %lx]: %d\n", start, end, rc); - rangeset_destroy(mem); - return rc; + goto fail; } } } @@ -324,12 +362,42 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only) * will always be to establish mappings and process all the BARs. */ ASSERT((cmd & PCI_COMMAND_MEMORY) && !rom_only); - return apply_map(pdev->domain, pdev, mem, cmd); + return apply_map(pdev->domain, pdev, cmd); } - defer_map(dev->domain, dev, mem, cmd, rom_only); + /* Find out how many memory ranges has left after MSI and overlaps. */ + num_mem_ranges = 0; + for ( i = 0; i < ARRAY_SIZE(header->bars); i++ ) + { + struct vpci_bar *bar = &header->bars[i]; + + if ( !rangeset_is_empty(bar->mem) ) + num_mem_ranges++; + } + + /* + * There are cases when PCI device, root port for example, has neither + * memory space nor IO. In this case PCI command register write is + * missed resulting in the underlying PCI device not functional, so: + * - if there are no regions write the command register now + * - if there are regions then defer work and write later on + */ + if ( !num_mem_ranges ) + pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd); + else + defer_map(dev->domain, dev, cmd, rom_only, num_mem_ranges); return 0; + +fail: + for ( i = 0; i < ARRAY_SIZE(header->bars); i++ ) + { + struct vpci_bar *bar = &header->bars[i]; + + rangeset_destroy(bar->mem); + bar->mem = NULL; + } + return rc; } static void cmd_write(const struct pci_dev *pdev, unsigned int reg, diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index db86b8e7fa3c..a0cbdb4bf4fd 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -82,6 +82,7 @@ struct vpci { /* Guest view of the BAR. */ uint64_t guest_addr; uint64_t size; + struct rangeset *mem; enum { VPCI_BAR_EMPTY, VPCI_BAR_IO, @@ -156,9 +157,9 @@ struct vpci { struct vpci_vcpu { /* Per-vcpu structure to store state while {un}mapping of PCI BARs. */ - struct rangeset *mem; struct pci_dev *pdev; uint16_t cmd; + uint8_t num_mem_ranges; bool rom_only : 1; }; From patchwork Fri Sep 3 10:08:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12474071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 384BAC433F5 for ; Fri, 3 Sep 2021 10:09:20 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E56D66108E for ; Fri, 3 Sep 2021 10:09:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E56D66108E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.177986.323851 (Exim 4.92) (envelope-from ) id 1mM68G-0007KL-0K; Fri, 03 Sep 2021 10:09:12 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 177986.323851; Fri, 03 Sep 2021 10:09:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM68F-0007K7-Rz; Fri, 03 Sep 2021 10:09:11 +0000 Received: by outflank-mailman (input) for mailman id 177986; Fri, 03 Sep 2021 10:09:11 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM68F-0004Cr-Ae for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:09:11 +0000 Received: from mail-ej1-x635.google.com (unknown [2a00:1450:4864:20::635]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 574bafea-333f-4458-a85f-9d85f3dc68ab; Fri, 03 Sep 2021 10:08:43 +0000 (UTC) Received: by mail-ej1-x635.google.com with SMTP id a25so11014222ejv.6 for ; Fri, 03 Sep 2021 03:08:43 -0700 (PDT) Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua. [176.36.245.220]) by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Sep 2021 03:08:41 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 574bafea-333f-4458-a85f-9d85f3dc68ab DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EsmmWshqcRWEmqQF2VjxxyMLQRhse2D1gTlLi8+uzDk=; b=kYdmRslCs8NTALbXGc7+QzrWmSxEXO8+BDs+NVl9JKDEti73SNG8Bxa30Qe1HIM2o/ XlN/YA1k2fkV4slRcmp6oFpn1xBHxhY1jlkZ1Y/5EYq9OhYgZ58xAhwDlgV61QuuCjyy UZii3Li6tECiunnl+jSszF+5X1YlnUZ9HhEU5Y2gyKLjYGcWaXw8AL7xRlLjh8K9LC+M +4maoZpwsGzHA2FVcxyn/f9WCz+4CdQ3r6f4kt8mNoAuN+GCdEjFEkFTERnznC8Yl17X LuJl09dTX5S6Z5iWPREoG5DbkdNx09ZLnW0hEfhE+vSTnWXUawIhyV5Rq8wX2xSQGD2o HtZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EsmmWshqcRWEmqQF2VjxxyMLQRhse2D1gTlLi8+uzDk=; b=co2MPdxS2NYgbewPN0EUl+L5yydIua2nKlB3+oeYgTGZ0zPlgzdo1D56PN5tcyFvKJ 4vQHsr+e07uCJbEdCSGdtklVDouMTgTQdtdaYKPMBm3Zs+ScDOdognLMaFA+ASCCXylm TDll1E0DA0d4uH/K2f2ukCOmRNuR3TqlWHvowtNnngrGDANJ08NgjB1HOtpianM6R7XH xC2TrdoRkqloWnkxhmZl/ZTkaOY4WBWv40FlvE856y14PP0EDn1EyRNJI89itn6xiasQ pu+nMJUwSicPHDF0/5sHMtnmX3CQAVlJGz7/dVbGjBF/F6xPGUbelDowCcNNgTXRByq6 vXmg== X-Gm-Message-State: AOAM532pMRFyhTXXnLPAfz6ZVLbUkWDOV8kD/yEfGcqduJjOczZzFvcV 3kseNt3Fkh0dv4TZJ+euQ82rhIJlYDA= X-Google-Smtp-Source: ABdhPJx8SGIw/Qab2uy5z2TUM5bpuiWzlOzdu1ih68KEF3FIN3G+q7L3S9fsjy63kr+jyeFlOUjLhQ== X-Received: by 2002:a17:907:96a2:: with SMTP id hd34mr3270175ejc.113.1630663721926; Fri, 03 Sep 2021 03:08:41 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH 7/9] vpci/header: program p2m with guest BAR view Date: Fri, 3 Sep 2021 13:08:29 +0300 Message-Id: <20210903100831.177748-8-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com> References: <20210903100831.177748-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko Take into account guest's BAR view and program its p2m accordingly: gfn is guest's view of the BAR and mfn is the physical BAR value as set up by the host bridge in the hardware domain. This way hardware doamin sees physical BAR values and guest sees emulated ones. Signed-off-by: Oleksandr Andrushchenko --- xen/drivers/vpci/header.c | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index 7f54199a3894..7416ef1e1e06 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -30,6 +30,10 @@ struct map_data { struct domain *d; + /* Start address of the BAR as seen by the guest. */ + gfn_t start_gfn; + /* Physical start address of the BAR. */ + mfn_t start_mfn; bool map; }; @@ -37,12 +41,28 @@ static int map_range(unsigned long s, unsigned long e, void *data, unsigned long *c) { const struct map_data *map = data; + gfn_t start_gfn; int rc; for ( ; ; ) { unsigned long size = e - s + 1; + /* + * Any BAR may have holes in its memory we want to map, e.g. + * we don't want to map MSI regions which may be a part of that BAR, + * e.g. when a single BAR is used for both MMIO and MSI. + * In this case MSI regions are subtracted from the mapping, but + * map->start_gfn still points to the very beginning of the BAR. + * So if there is a hole present then we need to adjust start_gfn + * to reflect the fact of that substraction. + */ + start_gfn = gfn_add(map->start_gfn, s - mfn_x(map->start_mfn)); + + printk(XENLOG_G_DEBUG + "%smap [%lx, %lx] -> %#"PRI_gfn" for d%d\n", + map->map ? "" : "un", s, e, gfn_x(start_gfn), + map->d->domain_id); /* * ARM TODOs: * - On ARM whether the memory is prefetchable or not should be passed @@ -52,8 +72,10 @@ static int map_range(unsigned long s, unsigned long e, void *data, * - {un}map_mmio_regions doesn't support preemption. */ - rc = map->map ? map_mmio_regions(map->d, _gfn(s), size, _mfn(s)) - : unmap_mmio_regions(map->d, _gfn(s), size, _mfn(s)); + rc = map->map ? map_mmio_regions(map->d, start_gfn, + size, _mfn(s)) + : unmap_mmio_regions(map->d, start_gfn, + size, _mfn(s)); if ( rc == 0 ) { *c += size; @@ -69,6 +91,7 @@ static int map_range(unsigned long s, unsigned long e, void *data, ASSERT(rc < size); *c += rc; s += rc; + gfn_add(map->start_gfn, rc); if ( general_preempt_check() ) return -ERESTART; } @@ -149,6 +172,10 @@ bool vpci_process_pending(struct vcpu *v) if ( !bar->mem ) continue; + data.start_gfn = is_hardware_domain(v->vpci.pdev->domain) ? + _gfn(PFN_DOWN(bar->addr)) : + _gfn(PFN_DOWN(bar->guest_addr)); + data.start_mfn = _mfn(PFN_DOWN(bar->addr)); rc = rangeset_consume_ranges(bar->mem, map_range, &data); if ( rc == -ERESTART ) @@ -194,6 +221,10 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev, if ( !bar->mem ) continue; + data.start_gfn = is_hardware_domain(d) ? + _gfn(PFN_DOWN(bar->addr)) : + _gfn(PFN_DOWN(bar->guest_addr)); + data.start_mfn = _mfn(PFN_DOWN(bar->addr)); while ( (rc = rangeset_consume_ranges(bar->mem, map_range, &data)) == -ERESTART ) process_pending_softirqs(); From patchwork Fri Sep 3 10:08:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12474073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01AAEC433FE for ; Fri, 3 Sep 2021 10:09:24 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C40E6610A2 for ; Fri, 3 Sep 2021 10:09:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org C40E6610A2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.177992.323863 (Exim 4.92) (envelope-from ) id 1mM68L-0007xj-Il; Fri, 03 Sep 2021 10:09:17 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 177992.323863; Fri, 03 Sep 2021 10:09:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM68L-0007xZ-EE; Fri, 03 Sep 2021 10:09:17 +0000 Received: by outflank-mailman (input) for mailman id 177992; Fri, 03 Sep 2021 10:09:16 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM68K-0004Cr-An for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:09:16 +0000 Received: from mail-ej1-x635.google.com (unknown [2a00:1450:4864:20::635]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 81dc2936-e1af-40c8-8a27-a24992fd589f; Fri, 03 Sep 2021 10:08:44 +0000 (UTC) Received: by mail-ej1-x635.google.com with SMTP id mf2so10990948ejb.9 for ; Fri, 03 Sep 2021 03:08:43 -0700 (PDT) Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua. [176.36.245.220]) by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Sep 2021 03:08:42 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 81dc2936-e1af-40c8-8a27-a24992fd589f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9oZWyDoHrlYE4f9O9pDC/GUYZqq2tbn79VvRAgG5sU8=; b=G0v5SVgVhnp27fzhTFh/t3t6T3ZG3N3FQIFEBpPCV7HbEmNwdwHCsDna9Ka/KxV5FU DxTwEW485P0l8KGipmtwsR4PUeZba5wv3+Fu/ntpk5rL4UwSUtnBWGZYnBKUOM42NxuJ GFgt36WnRDm+PUd1QAGp4A2cQ2ZxqVUvShjebP00K+B1HTsHQQkU3uLmPWxgKL0AJPtb aznDETYoKDAewvu3eFK5aeZmP/KclS4BjcmZC+3sJHmxn1JpWg7lOPN/kWaaXmSgMb8e JqOVncQSXD/xA4F426v3mWQvQi+W2MGFum99JYmvYKVxmw6AeK1htZrltbHDiaSMYs2Q MMmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9oZWyDoHrlYE4f9O9pDC/GUYZqq2tbn79VvRAgG5sU8=; b=Ul5ha2HbBHeFKU56A9FmD89nbB41SnxF5p4NO3s0lTyS/i2jSY94clHHGBlMRvu5Pb R+FZdRi1AssRf/R98fuBSuQbzexSwpDKH5D6WeWoNYBhRsbKzAFeAQ2dCCHVS3kW48Yr oh0uzN+gzfLxTuVdhpGJruzP39PZlHEXUzNy8YJk85Mh4PVoRco/Nz+SUyOGOaCCJVj/ avBvwXftKBd0+zzkojEUvWHA+q9DBLrqWulwhrs7i+3wKSlOkmleWU4HyPUvssQCH+wi XcGMc3zMlannMfrlUBiW+YDS9TE9i+q50PJ0rEFPBNLxR91oYmCGOxEN986gTtmg2hya f8Kg== X-Gm-Message-State: AOAM530t0rSAulO2Tr5mbpKDyYH5bn482wOLwLHWZUUq6kRc6muezO5m b5RsjRZOtucJLIyYUMYBWvAf0AVt3G8= X-Google-Smtp-Source: ABdhPJwC0I+pj81VPq5Z8aniIDBjWrsyNx1aYeEbgRJViZP4d+n9MV4hh4STMk34EUBwp0NwdeCIrw== X-Received: by 2002:a17:906:748e:: with SMTP id e14mr3277798ejl.152.1630663722931; Fri, 03 Sep 2021 03:08:42 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH 8/9] vpci/header: Reset the command register when adding devices Date: Fri, 3 Sep 2021 13:08:30 +0300 Message-Id: <20210903100831.177748-9-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com> References: <20210903100831.177748-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko Reset the command register when passing through a PCI device: it is possible that when passing through a PCI device its memory decoding bits in the command register are already set. Thus, a guest OS may not write to the command register to update memory decoding, so guest mappings (guest's view of the BARs) are left not updated. Signed-off-by: Oleksandr Andrushchenko --- xen/drivers/vpci/header.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index 7416ef1e1e06..dac973368b1e 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -811,6 +811,16 @@ int vpci_bar_add_handlers(const struct domain *d, struct pci_dev *pdev) gprintk(XENLOG_ERR, "%pp: failed to add BAR handlers for dom%d\n", &pdev->sbdf, d->domain_id); + + /* + * Reset the command register: it is possible that when passing + * through a PCI device its memory decoding bits in the command + * register are already set. Thus, a guest OS may not write to the + * command register to update memory decoding, so guest mappings + * (guest's view of the BARs) are left not updated. + */ + pci_conf_write16(pdev->sbdf, PCI_COMMAND, 0); + return rc; } From patchwork Fri Sep 3 10:08:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12474075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37039C433F5 for ; Fri, 3 Sep 2021 10:09:30 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 081A8601FA for ; Fri, 3 Sep 2021 10:09:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 081A8601FA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.177997.323874 (Exim 4.92) (envelope-from ) id 1mM68Q-00006V-RH; Fri, 03 Sep 2021 10:09:22 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 177997.323874; Fri, 03 Sep 2021 10:09:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM68Q-00006O-OD; Fri, 03 Sep 2021 10:09:22 +0000 Received: by outflank-mailman (input) for mailman id 177997; Fri, 03 Sep 2021 10:09:21 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mM68P-0004Cr-Aw for xen-devel@lists.xenproject.org; Fri, 03 Sep 2021 10:09:21 +0000 Received: from mail-ej1-x62a.google.com (unknown [2a00:1450:4864:20::62a]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id cfd41a6e-29ed-4f0d-93f2-6a4adb13b1b1; Fri, 03 Sep 2021 10:08:45 +0000 (UTC) Received: by mail-ej1-x62a.google.com with SMTP id jg16so7877055ejc.1 for ; Fri, 03 Sep 2021 03:08:45 -0700 (PDT) Received: from a2klaptop.epam.com (host-176-36-245-220.b024.la.net.ua. [176.36.245.220]) by smtp.gmail.com with ESMTPSA id p5sm2427187eju.30.2021.09.03.03.08.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Sep 2021 03:08:43 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cfd41a6e-29ed-4f0d-93f2-6a4adb13b1b1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hMt74Ggq0zLuhF8LFxm6cOM3G4Wenq/IlhMP/O3Mgu8=; b=V1tudX9W4mPKN2zjexbubTP4JGF55U3CiWlqiVVH/aom0cJVJ9DzTQIKOt0DiEqdeB gwJsiJ7+KLjAsmewkiv3WjPx0NljKuCC6KIu8e1A/0EQb3tPwt+Iw9O6ttCu1nEbMen9 dPfIoN8NWovGilGVHpkSunNV38UJxPyiT5SWc9gLFIWuF7vISABGFOf/CkkqLy5U8eBT csUbFMOjSPYoczhbwliEbPFTcQ1pEyU+dcCSuBHQlOK/0Jns8dCF/muXfodFaHpq4rue NAmSsaWkuUYPr1nJ+O/wEdrM+g5lWOlPDEU6Ak9vRyYXvYs0OI0MpwfvQwziA7gVjDds pMxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hMt74Ggq0zLuhF8LFxm6cOM3G4Wenq/IlhMP/O3Mgu8=; b=rqp8P+819hb+Nc5KV1bfSgDPds2Qx4iqwmg8oFFbWh4ZqNIM6qVDcOCq7h6RawDZhd aEE62/tgVWnntBp55OnxsXazmB/MjUwRkpd4jY6duR1cEBqH3vVZuR+hJgW7BZsj4fER rmfXHct7z2ALSi7ZPhVn7iUoSjIX/debzlDvZ2RdKGyAlocz99WlZvuN7w0EVnGyLNw7 S0R1n7VoXVIWzLqA9KtzcLpg/U1QkTQR56MuN7H3uP7CY1Xekbfi1kTc/C+edWW7mscV HowfSkqgG+Tn62lUY4V9ElvWTpBuESqgoibah25qOZY/qr4FkS5erpQHUvtVo+fozXUP CLfA== X-Gm-Message-State: AOAM533PnJkO9S4kWSa0EgNTrODTxnqnnkoX83jVc6wYCeREriRFvzOK Ix1RSv1WQxCWpiQhuhKpoouQfIiBlUU= X-Google-Smtp-Source: ABdhPJypiSxpzvy084GcT4yZmHBhr6SMk1ebVuiTgJuY67izy5QHVzN9Q+8vXBtuc3hYUclLgUGWEQ== X-Received: by 2002:a17:906:e51:: with SMTP id q17mr3408604eji.76.1630663724006; Fri, 03 Sep 2021 03:08:44 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH 9/9] vpci/header: Use pdev's domain instead of vCPU Date: Fri, 3 Sep 2021 13:08:31 +0300 Message-Id: <20210903100831.177748-10-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210903100831.177748-1-andr2000@gmail.com> References: <20210903100831.177748-1-andr2000@gmail.com> MIME-Version: 1.0 From: Rahul Singh Fixes: 9c244fdef7e7 ("vpci: add header handlers") Signed-off-by: Rahul Singh Signed-off-by: Oleksandr Andrushchenko --- xen/drivers/vpci/header.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index dac973368b1e..688c69acbc23 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -157,7 +157,7 @@ bool vpci_process_pending(struct vcpu *v) if ( v->vpci.num_mem_ranges ) { struct map_data data = { - .d = v->domain, + .d = v->vpci.pdev->domain, .map = v->vpci.cmd & PCI_COMMAND_MEMORY, }; struct pci_dev *pdev = v->vpci.pdev;