From patchwork Fri Aug 2 15:35:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony PERARD X-Patchwork-Id: 11073787 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB9321399 for ; Fri, 2 Aug 2019 15:59:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CA83E28613 for ; Fri, 2 Aug 2019 15:59:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BE42828872; Fri, 2 Aug 2019 15:59:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CFB3228613 for ; Fri, 2 Aug 2019 15:59:06 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htZwk-0006G0-Pt; Fri, 02 Aug 2019 15:58:22 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htZwi-0006AN-6m for xen-devel@lists.xenproject.org; Fri, 02 Aug 2019 15:58:20 +0000 X-Inumbo-ID: 5784d0f8-b53e-11e9-8980-bc764e045a96 Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 5784d0f8-b53e-11e9-8980-bc764e045a96; Fri, 02 Aug 2019 15:58:17 +0000 (UTC) Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=anthony.perard@citrix.com; spf=Pass smtp.mailfrom=anthony.perard@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of anthony.perard@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="anthony.perard@citrix.com"; x-sender="anthony.perard@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa2.hc3370-68.iphmx.com: domain of anthony.perard@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="anthony.perard@citrix.com"; x-sender="anthony.perard@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ~all" Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="anthony.perard@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: M9oMceXj7vBExvf/ooJQ0YtkC+NKuIB339bDPGQuVersbVgQCMtzA1QTPMftsf3MpNCB9zf25v dom3nLaxRJNAHCL9AOJvRqmqPjAMKqf0ojox/pC4y8uDPb807toro67+0U8quIydV2QAZCd5Lx lLld6WsFfsxZIOtx4esqCkyiB0C8vRl6a+QFmchgcPNGP0GexwxQhJrdptH5YJ9OWgczEEbsy7 L+w9Azm2aYeDSOxCRYkh0jVHToUnzaLaaUO/T1S6RVxboHcPwnRiWC1nMThBO57JsWCmYF3fT8 qFg= X-SBRS: 2.7 X-MesageID: 3787841 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.64,338,1559534400"; d="scan'208";a="3787841" From: Anthony PERARD To: Date: Fri, 2 Aug 2019 16:35:59 +0100 Message-ID: <20190802153606.32061-29-anthony.perard@citrix.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190802153606.32061-1-anthony.perard@citrix.com> References: <20190802153606.32061-1-anthony.perard@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH 28/35] libxl_pci: Use ev_qmp in do_pci_add X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Anthony PERARD , Ian Jackson , Wei Liu Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch also replaces the use of libxl__wait_for_device_model_deprecated() by its equivalent without the need for a thread. Signed-off-by: Anthony PERARD Acked-by: Ian Jackson --- tools/libxl/libxl_internal.h | 1 - tools/libxl/libxl_pci.c | 288 ++++++++++++++++++++++++++++++++--- tools/libxl/libxl_qmp.c | 96 ------------ 3 files changed, 265 insertions(+), 120 deletions(-) diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index dd9934141f..277e322e09 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -1981,7 +1981,6 @@ typedef struct libxl__qmp_handler libxl__qmp_handler; */ _hidden libxl__qmp_handler *libxl__qmp_initialize(libxl__gc *gc, uint32_t domid); -_hidden int libxl__qmp_pci_add(libxl__gc *gc, int d, libxl_device_pci *pcidev); _hidden int libxl__qmp_pci_del(libxl__gc *gc, int domid, libxl_device_pci *pcidev); /* Resume hvm domain */ diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c index 503db6c260..3477f3aba6 100644 --- a/tools/libxl/libxl_pci.c +++ b/tools/libxl/libxl_pci.c @@ -23,6 +23,7 @@ #define PCI_BDF_VDEVFN "%04x:%02x:%02x.%01x@%02x" #define PCI_OPTIONS "msitranslate=%d,power_mgmt=%d" #define PCI_BDF_XSPATH "%04x-%02x-%02x-%01x" +#define PCI_PT_QDEV_ID "pci-pt-%02x_%02x.%01x" static unsigned int pcidev_encode_bdf(libxl_device_pci *pcidev) { @@ -991,33 +992,40 @@ typedef struct pci_add_state { void (*callback)(libxl__egc *, struct pci_add_state *, int rc); /* private to do_pci_add */ + libxl__xswait_state xswait; + libxl__ev_qmp qmp; + libxl__ev_time timeout; libxl_device_pci *pcidev; int pci_domid; } pci_add_state; +static void pci_add_qemu_trad_watch_state_cb(libxl__egc *egc, + libxl__xswait_state *xswa, int rc, const char *state); +static void pci_add_qmp_device_add(libxl__egc *, pci_add_state *); +static void pci_add_qmp_device_add_cb(libxl__egc *, + libxl__ev_qmp *, const libxl__json_object *, int rc); +static void pci_add_qmp_query_pci_cb(libxl__egc *, + libxl__ev_qmp *, const libxl__json_object *, int rc); +static void pci_add_timeout(libxl__egc *egc, libxl__ev_time *ev, + const struct timeval *requested_abs, int rc); +static void pci_add_dm_done(libxl__egc *, + pci_add_state *, int rc); + static void do_pci_add(libxl__egc *egc, libxl_domid domid, libxl_device_pci *pcidev, pci_add_state *pas) { STATE_AO_GC(pas->aodev->ao); - libxl_ctx *ctx = libxl__gc_owner(gc); libxl_domain_type type = libxl__domain_type(gc, domid); - char *sysfs_path; - FILE *f; - unsigned long long start, end, flags, size; - int irq, i, rc, hvm = 0; - uint32_t flag = XEN_DOMCTL_DEV_RDM_RELAXED; - uint32_t domainid = domid; - bool isstubdom = libxl_is_stubdom(ctx, domid, &domainid); - int r; - - /* Convenience aliases */ - bool starting = pas->starting; + int rc; /* init pci_add_state */ + libxl__xswait_init(&pas->xswait); + libxl__ev_qmp_init(&pas->qmp); pas->pcidev = pcidev; pas->pci_domid = domid; + libxl__ev_time_init(&pas->timeout); if (type == LIBXL_DOMAIN_TYPE_INVALID) { rc = ERROR_FAIL; @@ -1025,26 +1033,259 @@ static void do_pci_add(libxl__egc *egc, } if (type == LIBXL_DOMAIN_TYPE_HVM) { - hvm = 1; switch (libxl__device_model_version_running(gc, domid)) { case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL: - if (libxl__wait_for_device_model_deprecated(gc, domid, - "running", NULL, NULL, NULL) < 0) { - rc = ERROR_FAIL; - goto out; - } - rc = qemu_pci_add_xenstore(gc, domid, pcidev); - break; + pas->xswait.ao = ao; + pas->xswait.what = "Device Model"; + pas->xswait.path = DEVICE_MODEL_XS_PATH(gc, + libxl_get_stubdom_id(CTX, domid), domid, "/state"); + pas->xswait.timeout_ms = LIBXL_DEVICE_MODEL_START_TIMEOUT * 1000; + pas->xswait.callback = pci_add_qemu_trad_watch_state_cb; + rc = libxl__xswait_start(gc, &pas->xswait); + if (rc) goto out; + return; case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN: - rc = libxl__qmp_pci_add(gc, domid, pcidev); - break; + pci_add_qmp_device_add(egc, pas); /* must be last */ + return; default: rc = ERROR_INVAL; + break; } - if ( rc ) + } + + rc = 0; + +out: + pci_add_dm_done(egc, pas, rc); /* must be last */ +} + +static void pci_add_qemu_trad_watch_state_cb(libxl__egc *egc, + libxl__xswait_state *xswa, + int rc, + const char *state) +{ + pci_add_state *pas = CONTAINER_OF(xswa, *pas, xswait); + STATE_AO_GC(pas->aodev->ao); + + /* Convenience aliases */ + libxl_domid domid = pas->domid; + libxl_device_pci *pcidev = pas->pcidev; + + if (rc) { + if (rc == ERROR_TIMEDOUT) { + LOGD(ERROR, domid, "%s not ready", xswa->what); + } + goto out; + } + + if (!state) + return; + if (strcmp(state, "running")) + return; + + rc = qemu_pci_add_xenstore(gc, domid, pcidev); +out: + libxl__xswait_stop(gc, xswa); + pci_add_dm_done(egc, pas, rc); /* must be last */ +} + +static void pci_add_qmp_device_add(libxl__egc *egc, pci_add_state *pas) +{ + STATE_AO_GC(pas->aodev->ao); + libxl__json_object *args = NULL; + int rc; + + /* Convenience aliases */ + libxl_domid domid = pas->domid; + libxl_device_pci *pcidev = pas->pcidev; + libxl__ev_qmp *const qmp = &pas->qmp; + + rc = libxl__ev_time_register_rel(ao, &pas->timeout, + pci_add_timeout, + LIBXL_QMP_CMD_TIMEOUT * 1000); + if (rc) goto out; + + libxl__qmp_param_add_string(gc, &args, "driver", + "xen-pci-passthrough"); + QMP_PARAMETERS_SPRINTF(&args, "id", PCI_PT_QDEV_ID, + pcidev->bus, pcidev->dev, pcidev->func); + QMP_PARAMETERS_SPRINTF(&args, "hostaddr", + "%04x:%02x:%02x.%01x", pcidev->domain, + pcidev->bus, pcidev->dev, pcidev->func); + if (pcidev->vdevfn) { + QMP_PARAMETERS_SPRINTF(&args, "addr", "%x.%x", + PCI_SLOT(pcidev->vdevfn), + PCI_FUNC(pcidev->vdevfn)); + } + /* + * Version of QEMU prior to the XSA-131 fix did not support + * this property and were effectively always in permissive + * mode. The fix for XSA-131 switched the default to be + * restricted by default and added the permissive property. + * + * Therefore in order to support both old and new QEMU we only + * set the permissive flag if it is true. Users of older QEMU + * have no reason to set the flag so this is ok. + */ + if (pcidev->permissive) + libxl__qmp_param_add_bool(gc, &args, "permissive", true); + + qmp->ao = pas->aodev->ao; + qmp->domid = domid; + qmp->payload_fd = -1; + qmp->callback = pci_add_qmp_device_add_cb; + rc = libxl__ev_qmp_send(gc, qmp, "device_add", args); + if (rc) goto out; + return; + +out: + pci_add_dm_done(egc, pas, rc); /* must be last */ +} + +static void pci_add_qmp_device_add_cb(libxl__egc *egc, + libxl__ev_qmp *qmp, + const libxl__json_object *response, + int rc) +{ + EGC_GC; + pci_add_state *pas = CONTAINER_OF(qmp, *pas, qmp); + + if (rc) goto out; + + qmp->callback = pci_add_qmp_query_pci_cb; + rc = libxl__ev_qmp_send(gc, qmp, "query-pci", NULL); + if (rc) goto out; + return; + +out: + pci_add_dm_done(egc, pas, rc); /* must be last */ +} + +static void pci_add_qmp_query_pci_cb(libxl__egc *egc, + libxl__ev_qmp *qmp, + const libxl__json_object *response, + int rc) +{ + EGC_GC; + pci_add_state *pas = CONTAINER_OF(qmp, *pas, qmp); + const libxl__json_object *bus = NULL; + char *asked_id; + int i, j; + const libxl__json_object *devices = NULL; + const libxl__json_object *device = NULL; + const libxl__json_object *o = NULL; + const char *id = NULL; + int dev_slot, dev_func; + + /* Convenience aliases */ + libxl_device_pci *pcidev = pas->pcidev; + + if (rc) goto out; + + /* `query-pci' returns: + * [ + * {'bus': 'int', + * 'devices': [ + * {'bus': 'int', 'slot': 'int', 'function': 'int', + * 'class_info': 'PciDeviceClass', 'id': 'PciDeviceId', + * '*irq': 'int', 'qdev_id': 'str', + * '*pci_bridge': 'PciBridgeInfo', + * 'regions': ['PciMemoryRegion'] + * } + * ] + * } + * ] + * (See qemu.git/qapi/ for the struct that aren't detailed here) + */ + + asked_id = GCSPRINTF(PCI_PT_QDEV_ID, + pcidev->bus, pcidev->dev, pcidev->func); + + for (i = 0; (bus = libxl__json_array_get(response, i)); i++) { + devices = libxl__json_map_get("devices", bus, JSON_ARRAY); + if (!devices) { + rc = ERROR_QEMU_API; goto out; + } + + for (j = 0; (device = libxl__json_array_get(devices, j)); j++) { + o = libxl__json_map_get("qdev_id", device, JSON_STRING); + if (!o) { + rc = ERROR_QEMU_API; + goto out; + } + id = libxl__json_object_get_string(o); + if (!id || strcmp(asked_id, id)) + continue; + + o = libxl__json_map_get("slot", device, JSON_INTEGER); + if (!o) { + rc = ERROR_QEMU_API; + goto out; + } + dev_slot = libxl__json_object_get_integer(o); + o = libxl__json_map_get("function", device, JSON_INTEGER); + if (!o) { + rc = ERROR_QEMU_API; + goto out; + } + dev_func = libxl__json_object_get_integer(o); + + pcidev->vdevfn = PCI_DEVFN(dev_slot, dev_func); + + rc = 0; + goto out; + } } + rc = ERROR_FAIL; + LOGD(ERROR, qmp->domid, + "PCI device id '%s' wasn't found in QEMU's 'query-pci' response.", + asked_id); + +out: + if (rc == ERROR_QEMU_API) { + LOGD(ERROR, qmp->domid, + "Unexpected response to QMP cmd 'query-pci', received:\n%s", + JSON(response)); + } + pci_add_dm_done(egc, pas, rc); /* must be last */ +} + +static void pci_add_timeout(libxl__egc *egc, libxl__ev_time *ev, + const struct timeval *requested_abs, + int rc) +{ + pci_add_state *pas = CONTAINER_OF(ev, *pas, timeout); + + pci_add_dm_done(egc, pas, rc); +} + +static void pci_add_dm_done(libxl__egc *egc, + pci_add_state *pas, + int rc) +{ + STATE_AO_GC(pas->aodev->ao); + libxl_ctx *ctx = libxl__gc_owner(gc); + libxl_domid domid = pas->pci_domid; + char *sysfs_path; + FILE *f; + unsigned long long start, end, flags, size; + int irq, i; + int r; + uint32_t flag = XEN_DOMCTL_DEV_RDM_RELAXED; + uint32_t domainid = domid; + bool isstubdom = libxl_is_stubdom(ctx, domid, &domainid); + + /* Convenience aliases */ + bool starting = pas->starting; + libxl_device_pci *pcidev = pas->pcidev; + bool hvm = libxl__domain_type(gc, domid) == LIBXL_DOMAIN_TYPE_HVM; + + libxl__ev_qmp_dispose(gc, &pas->qmp); + + if (rc) goto out; + sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/resource", pcidev->domain, pcidev->bus, pcidev->dev, pcidev->func); f = fopen(sysfs_path, "r"); @@ -1145,6 +1386,7 @@ static void do_pci_add(libxl__egc *egc, else rc = 0; out: + libxl__ev_time_deregister(gc, &pas->timeout); pas->callback(egc, pas, rc); } diff --git a/tools/libxl/libxl_qmp.c b/tools/libxl/libxl_qmp.c index c78ef4637d..38ba63d5b9 100644 --- a/tools/libxl/libxl_qmp.c +++ b/tools/libxl/libxl_qmp.c @@ -736,54 +736,6 @@ void libxl__qmp_cleanup(libxl__gc *gc, uint32_t domid) } } -static int pci_add_callback(libxl__qmp_handler *qmp, - const libxl__json_object *response, void *opaque) -{ - libxl_device_pci *pcidev = opaque; - const libxl__json_object *bus = NULL; - GC_INIT(qmp->ctx); - int i, j, rc = -1; - char *asked_id = GCSPRINTF(PCI_PT_QDEV_ID, - pcidev->bus, pcidev->dev, pcidev->func); - - for (i = 0; (bus = libxl__json_array_get(response, i)); i++) { - const libxl__json_object *devices = NULL; - const libxl__json_object *device = NULL; - const libxl__json_object *o = NULL; - const char *id = NULL; - - devices = libxl__json_map_get("devices", bus, JSON_ARRAY); - - for (j = 0; (device = libxl__json_array_get(devices, j)); j++) { - o = libxl__json_map_get("qdev_id", device, JSON_STRING); - id = libxl__json_object_get_string(o); - - if (id && strcmp(asked_id, id) == 0) { - int dev_slot, dev_func; - - o = libxl__json_map_get("slot", device, JSON_INTEGER); - if (!o) - goto out; - dev_slot = libxl__json_object_get_integer(o); - o = libxl__json_map_get("function", device, JSON_INTEGER); - if (!o) - goto out; - dev_func = libxl__json_object_get_integer(o); - - pcidev->vdevfn = PCI_DEVFN(dev_slot, dev_func); - - rc = 0; - goto out; - } - } - } - - -out: - GC_FREE; - return rc; -} - static int pci_del_callback(libxl__qmp_handler *qmp, const libxl__json_object *response, void *opaque) { @@ -833,54 +785,6 @@ static int qmp_run_command(libxl__gc *gc, int domid, return rc; } -int libxl__qmp_pci_add(libxl__gc *gc, int domid, libxl_device_pci *pcidev) -{ - libxl__qmp_handler *qmp = NULL; - libxl__json_object *args = NULL; - char *hostaddr = NULL; - int rc = 0; - - qmp = libxl__qmp_initialize(gc, domid); - if (!qmp) - return -1; - - hostaddr = GCSPRINTF("%04x:%02x:%02x.%01x", pcidev->domain, - pcidev->bus, pcidev->dev, pcidev->func); - if (!hostaddr) - return -1; - - libxl__qmp_param_add_string(gc, &args, "driver", "xen-pci-passthrough"); - QMP_PARAMETERS_SPRINTF(&args, "id", PCI_PT_QDEV_ID, - pcidev->bus, pcidev->dev, pcidev->func); - libxl__qmp_param_add_string(gc, &args, "hostaddr", hostaddr); - if (pcidev->vdevfn) { - QMP_PARAMETERS_SPRINTF(&args, "addr", "%x.%x", - PCI_SLOT(pcidev->vdevfn), PCI_FUNC(pcidev->vdevfn)); - } - /* - * Version of QEMU prior to the XSA-131 fix did not support this - * property and were effectively always in permissive mode. The - * fix for XSA-131 switched the default to be restricted by - * default and added the permissive property. - * - * Therefore in order to support both old and new QEMU we only set - * the permissive flag if it is true. Users of older QEMU have no - * reason to set the flag so this is ok. - */ - if (pcidev->permissive) - libxl__qmp_param_add_bool(gc, &args, "permissive", true); - - rc = qmp_synchronous_send(qmp, "device_add", args, - NULL, NULL, qmp->timeout); - if (rc == 0) { - rc = qmp_synchronous_send(qmp, "query-pci", NULL, - pci_add_callback, pcidev, qmp->timeout); - } - - libxl__qmp_close(qmp); - return rc; -} - static int qmp_device_del(libxl__gc *gc, int domid, char *id) { libxl__json_object *args = NULL;