From patchwork Mon Aug 7 23:50:11 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Venu Busireddy X-Patchwork-Id: 9886297 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 137AA602CC for ; Mon, 7 Aug 2017 23:52:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0544028726 for ; Mon, 7 Aug 2017 23:52:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ED16628712; Mon, 7 Aug 2017 23:51:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6FFB128712 for ; Mon, 7 Aug 2017 23:51:59 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1derlz-0007nn-3A; Mon, 07 Aug 2017 23:49:23 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1derlx-0007nf-Qf for xen-devel@lists.xen.org; Mon, 07 Aug 2017 23:49:21 +0000 Received: from [193.109.254.147] by server-9.bemta-6.messagelabs.com id 1A/56-03406-18CF8895; Mon, 07 Aug 2017 23:49:21 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrLLMWRWlGSWpSXmKPExsXSO6nOVbfhT0e kwcsd0hZLPi5mcWD0OLr7N1MAYxRrZl5SfkUCa0b3xtesBXf1K+7PmM7ewLhStYuRi0NIYAKT xLZf1xghnF+MEs0nmlghnA2MEmvuXIfKTGSUaG9tBcpwcrAJGEgcPdwDViUi0MYosfrEZUaQh LBAuMTkVT1gNouAqsSXKafZQGxeAVuJy9t/gzVLCMhJLN1+nbmLkYODU8BO4uMTG5CwEFDJu9 3XGCFKDCU+b1zKPIGRdwEjwypGjeLUorLUIl0jc72kosz0jJLcxMwcXUMDM73c1OLixPTUnMS kYr3k/NxNjMCgYACCHYyL1wYeYpTkYFIS5eXc0hEpxJeUn1KZkVicEV9UmpNafIhRhoNDSYKX 8zdQTrAoNT21Ii0zBxieMGkJDh4lEd5fv4DSvMUFibnFmekQqVOMuhyvJvz/xiTEkpeflyolz vsDpEgApCijNA9uBCxWLjHKSgnzMgIdJcRTkFqUm1mCKv+KUZyDUUmYtx7kEp7MvBK4Ta+Ajm ACOuJNYivIESWJCCmpBsayN1rMj/qzr69rmXUiSP1DgLua3IzYf/+ZC1Yb7PThZt+S/P26LD+ 3cW6UwLvntgKn7jt61wZPjPeLvNXsZJn5Ijb9xHyB951Bps2MvKv0jI5vf2i6LPy4v1XhMqnu 9YIGn6J+v81nkWX71HnJ9vvFoFP3+2PqKi5kxU+uVtrZWLdN0tW6RImlOCPRUIu5qDgRANkUs BuQAgAA X-Env-Sender: venu.busireddy@oracle.com X-Msg-Ref: server-13.tower-27.messagelabs.com!1502149759!101226171!1 X-Originating-IP: [141.146.126.69] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTQxLjE0Ni4xMjYuNjkgPT4gMjc3MjE4\n X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 44241 invoked from network); 7 Aug 2017 23:49:20 -0000 Received: from aserp1040.oracle.com (HELO aserp1040.oracle.com) (141.146.126.69) by server-13.tower-27.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 7 Aug 2017 23:49:20 -0000 Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v77NnHSk032698 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 7 Aug 2017 23:49:17 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id v77NnHr3003359 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 7 Aug 2017 23:49:17 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id v77NnGSV008185; Mon, 7 Aug 2017 23:49:16 GMT Received: from ban25uut183.us.oracle.com (/10.153.74.183) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 07 Aug 2017 16:49:16 -0700 From: Venu Busireddy To: venu.busireddy@oracle.com, xen-devel@lists.xen.org, Ian Jackson , Wei Liu Date: Mon, 7 Aug 2017 18:50:11 -0500 Message-Id: <20170807235012.3804-2-venu.busireddy@oracle.com> X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170807235012.3804-1-venu.busireddy@oracle.com> References: <20170807235012.3804-1-venu.busireddy@oracle.com> X-Source-IP: userv0022.oracle.com [156.151.31.74] Subject: [Xen-devel] [PATCH v3 1/2] libxl: Implement the handler to handle unrecoverable AER errors X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Implement the callback function to handle unrecoverable AER errors, and also the public APIs that can be used to register/unregister the handler. When an AER error occurs, the handler will forcibly remove the erring PCIe device from the guest. Signed-off-by: Venu Busireddy --- tools/libxl/libxl.h | 14 +++++++ tools/libxl/libxl_event.h | 13 +++++++ tools/libxl/libxl_internal.h | 7 ++++ tools/libxl/libxl_pci.c | 90 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 124 insertions(+) diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index 7cf0f31..c5af0aa 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -1044,6 +1044,20 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src); */ #define LIBXL_HAVE_QED 1 +/* LIBXL_HAVE_REG_AER_EVENTS_HANDLER + * + * If it is defined, libxl has a library function called + * libxl_reg_aer_events_handler. + */ +#define LIBXL_HAVE_REG_AER_EVENTS_HANDLER 1 + +/* LIBXL_HAVE_UNREG_AER_EVENTS_HANDLER + * + * If it is defined, libxl has a library function called + * libxl_unreg_aer_events_handler. + */ +#define LIBXL_HAVE_UNREG_AER_EVENTS_HANDLER 1 + typedef char **libxl_string_list; void libxl_string_list_dispose(libxl_string_list *sl); int libxl_string_list_length(const libxl_string_list *sl); diff --git a/tools/libxl/libxl_event.h b/tools/libxl/libxl_event.h index 1ea789e..1aea906 100644 --- a/tools/libxl/libxl_event.h +++ b/tools/libxl/libxl_event.h @@ -184,6 +184,19 @@ void libxl_evdisable_domain_death(libxl_ctx *ctx, libxl_evgen_domain_death*); * may generate only a DEATH event. */ +typedef struct libxl__aer_watch libxl_aer_watch; +int libxl_reg_aer_events_handler(libxl_ctx *, uint32_t, libxl_aer_watch **) + LIBXL_EXTERNAL_CALLERS_ONLY; + /* + * Registers a handler to handle the occurrence of unrecoverable AER errors. + * This function depends on the calling application running the libxl's + * internal event loop. Toolstacks that do not use libxl's internal + * event loop must arrange to have their own event loop created and enter + * libxl (say, call libxl_event_wait()), to enable the event to be processed. + */ +void libxl_unreg_aer_events_handler(libxl_ctx *, uint32_t, libxl_aer_watch *) + LIBXL_EXTERNAL_CALLERS_ONLY; + typedef struct libxl__evgen_disk_eject libxl_evgen_disk_eject; int libxl_evenable_disk_eject(libxl_ctx *ctx, uint32_t domid, const char *vdev, libxl_ev_user, libxl_evgen_disk_eject **evgen_out); diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index afe6652..2b74286 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -352,6 +352,13 @@ struct libxl__ev_child { LIBXL_LIST_ENTRY(struct libxl__ev_child) entry; }; +/* + * Structure used for AER event handling. + */ +struct libxl__aer_watch { + uint32_t domid; + libxl__ev_xswatch watch; +}; /* * evgen structures, which are the state we use for generating diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c index 65ad5e5..feedf27 100644 --- a/tools/libxl/libxl_pci.c +++ b/tools/libxl/libxl_pci.c @@ -1678,6 +1678,96 @@ static int libxl_device_pci_compare(libxl_device_pci *d1, return COMPARE_PCI(d1, d2); } +static void aer_backend_watch_callback(libxl__egc *egc, + libxl__ev_xswatch *watch, + const char *watch_path, + const char *event_path) +{ + EGC_GC; + libxl_aer_watch *aer_ws = CONTAINER_OF(watch, *aer_ws, watch); + int rc; + uint32_t dom, bus, dev, fn; + uint32_t domid = aer_ws->domid; + char *p, *path; + const char *aerFailedSBDF; + libxl_device_pci pcidev; + + /* Extract the backend directory. */ + path = libxl__strdup(gc, event_path); + p = strrchr(path, '/'); + if ((p == NULL) || (strcmp(p, "/aerFailedSBDF") != 0)) + return; + /* Truncate the string so it points to the backend directory. */ + *p = '\0'; + + /* Fetch the value of the failed PCI device. */ + rc = libxl__xs_read_checked(gc, XBT_NULL, + GCSPRINTF("%s/aerFailedSBDF", path), &aerFailedSBDF); + if (rc || !aerFailedSBDF) + return; + sscanf(aerFailedSBDF, "%x:%x:%x.%x", &dom, &bus, &dev, &fn); + + libxl_device_pci_init(&pcidev); + pcidev_struct_fill(&pcidev, dom, bus, dev, fn, 0); + /* Forcibly remove the device from the guest */ + rc = libxl__device_pci_remove_common(gc, domid, &pcidev, 1); + if (rc) + LOGD(ERROR, domid, " libxl__device_pci_remove_common() failed, rc=x%x", + (unsigned int)rc); + + return; +} + +int libxl_reg_aer_events_handler(libxl_ctx *ctx, + uint32_t domid, + libxl_aer_watch **aer_ws_out) +{ + int rc = 0; + int dom0_domid; + char *be_path; + libxl_aer_watch *aer_ws = NULL; + GC_INIT(ctx); + + *aer_ws_out = NULL; + + rc = libxl__get_domid(gc, (uint32_t *)(&dom0_domid)); + if (rc) { + LOGD(ERROR, domid, " libxl__get_domid() failed, rc = %d", rc); + goto out; + } + + aer_ws = malloc(sizeof(libxl_aer_watch)); + if (!aer_ws) { + rc = ERROR_NOMEM; + goto out; + } + memset(aer_ws, 0, sizeof(libxl_aer_watch)); + + aer_ws->domid = domid; + be_path = GCSPRINTF("/local/domain/%d/backend/pci/%u/%d/%s", + dom0_domid, domid, dom0_domid, "aerFailedSBDF"); + rc = libxl__ev_xswatch_register(gc, &aer_ws->watch, + aer_backend_watch_callback, be_path); + *aer_ws_out = aer_ws; + +out: + GC_FREE; + return rc; +} + +void libxl_unreg_aer_events_handler(libxl_ctx *ctx, + uint32_t domid, + libxl_aer_watch *aer_ws) +{ + GC_INIT(ctx); + + libxl__ev_xswatch_deregister(gc, &aer_ws->watch); + + free(aer_ws); + GC_FREE; + return; +} + DEFINE_DEVICE_TYPE_STRUCT_X(pcidev, pci); /*