From patchwork Mon Aug 7 23:53:24 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Venu Busireddy X-Patchwork-Id: 9886319 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id BEE4C601EB for ; Mon, 7 Aug 2017 23:54:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B162E286F5 for ; Mon, 7 Aug 2017 23:54:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A5F522873F; Mon, 7 Aug 2017 23:54:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 96B9128726 for ; Mon, 7 Aug 2017 23:54:32 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1derp0-000091-8K; Mon, 07 Aug 2017 23:52:30 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1deroy-000078-5J for xen-devel@lists.xen.org; Mon, 07 Aug 2017 23:52:28 +0000 Received: from [85.158.137.68] by server-4.bemta-3.messagelabs.com id 34/34-04533-B3DF8895; Mon, 07 Aug 2017 23:52:27 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrLLMWRWlGSWpSXmKPExsUyZ7p8oK7V345 Ig/Y3OhZLPi5mcWD0OLr7N1MAYxRrZl5SfkUCa0b3xtesBXf1K+7PmM7ewLhStYuRi0NIYAKT xPY7x9khnD+MEk+OPGKCcNYzSkydtZMFwpnIKNF/eBtjFyMnB5uAgcTRwz2sIAkRgTZGidUnL oMlhAXCJSav6gGzWQRUJS5f3c0CYvMK2EocvnuNHcSWEJCTWLr9OjOIzSlgJzF5/1s2EFsIqG bL6hlMEDWGEp83LmWewMi7gJFhFaN6cWpRWWqRrpFeUlFmekZJbmJmjq6hgbFebmpxcWJ6ak5 iUrFecn7uJkZgUNQzMDDuYDzV7HyIUZKDSUmUl3NLR6QQX1J+SmVGYnFGfFFpTmrxIUYZDg4l CV6DP0A5waLU9NSKtMwcYHjCpCU4eJREeNVB0rzFBYm5xZnpEKlTjLocryb8/8YkxJKXn5cqJ c777zdQkQBIUUZpHtwIWKxcYpSVEuZlZGBgEOIpSC3KzSxBlX/FKM7BqCTMawqyiiczrwRu0y ugI5iAjniT2ApyREkiQkqqgVFXXChxxkTv5SvP72zdvr2pRPqg8sGHNdFnsjYIFUbFcOYZHGz Y9GLpxPk/i6K3fCq+usfv0dryN2e+Mx1jYbC709hfFhseuJfT+lKaXYSRzha9DZZ26RtecEge c5i9+8gi36Anx6RiGXxSReao72zdOsW3d2F9K89MgT614qMP2viuWCR+nqLEUpyRaKjFXFScC ADcISyckAIAAA== X-Env-Sender: venu.busireddy@oracle.com X-Msg-Ref: server-8.tower-31.messagelabs.com!1502149944!109066008!1 X-Originating-IP: [156.151.31.81] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTU2LjE1MS4zMS44MSA9PiAyODgzMzk=\n X-StarScan-Received: X-StarScan-Version: 9.4.25; banners=-,-,- X-VirusChecked: Checked Received: (qmail 43892 invoked from network); 7 Aug 2017 23:52:26 -0000 Received: from userp1040.oracle.com (HELO userp1040.oracle.com) (156.151.31.81) by server-8.tower-31.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 7 Aug 2017 23:52:26 -0000 Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v77NqKLU020080 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 7 Aug 2017 23:52:20 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id v77NqKZJ009045 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 7 Aug 2017 23:52:20 GMT Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id v77NqKCa026475; Mon, 7 Aug 2017 23:52:20 GMT Received: from ban25uut183.us.oracle.com (/10.153.74.183) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 07 Aug 2017 16:52:19 -0700 From: Venu Busireddy To: venu.busireddy@oracle.com, xen-devel@lists.xen.org, Ian Jackson , Wei Liu Date: Mon, 7 Aug 2017 18:53:24 -0500 Message-Id: <20170807235325.3894-2-venu.busireddy@oracle.com> X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170807235325.3894-1-venu.busireddy@oracle.com> References: <20170807235325.3894-1-venu.busireddy@oracle.com> X-Source-IP: userv0022.oracle.com [156.151.31.74] Subject: [Xen-devel] [PATCH v3 1/2] libxl: Implement the handler to handle unrecoverable AER errors X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Implement the callback function to handle unrecoverable AER errors, and also the public APIs that can be used to register/unregister the handler. When an AER error occurs, the handler will forcibly remove the erring PCIe device from the guest. Signed-off-by: Venu Busireddy --- tools/libxl/libxl.h | 14 +++++++ tools/libxl/libxl_event.h | 13 +++++++ tools/libxl/libxl_internal.h | 7 ++++ tools/libxl/libxl_pci.c | 90 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 124 insertions(+) diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index 7cf0f31..c5af0aa 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -1044,6 +1044,20 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src); */ #define LIBXL_HAVE_QED 1 +/* LIBXL_HAVE_REG_AER_EVENTS_HANDLER + * + * If it is defined, libxl has a library function called + * libxl_reg_aer_events_handler. + */ +#define LIBXL_HAVE_REG_AER_EVENTS_HANDLER 1 + +/* LIBXL_HAVE_UNREG_AER_EVENTS_HANDLER + * + * If it is defined, libxl has a library function called + * libxl_unreg_aer_events_handler. + */ +#define LIBXL_HAVE_UNREG_AER_EVENTS_HANDLER 1 + typedef char **libxl_string_list; void libxl_string_list_dispose(libxl_string_list *sl); int libxl_string_list_length(const libxl_string_list *sl); diff --git a/tools/libxl/libxl_event.h b/tools/libxl/libxl_event.h index 1ea789e..1aea906 100644 --- a/tools/libxl/libxl_event.h +++ b/tools/libxl/libxl_event.h @@ -184,6 +184,19 @@ void libxl_evdisable_domain_death(libxl_ctx *ctx, libxl_evgen_domain_death*); * may generate only a DEATH event. */ +typedef struct libxl__aer_watch libxl_aer_watch; +int libxl_reg_aer_events_handler(libxl_ctx *, uint32_t, libxl_aer_watch **) + LIBXL_EXTERNAL_CALLERS_ONLY; + /* + * Registers a handler to handle the occurrence of unrecoverable AER errors. + * This function depends on the calling application running the libxl's + * internal event loop. Toolstacks that do not use libxl's internal + * event loop must arrange to have their own event loop created and enter + * libxl (say, call libxl_event_wait()), to enable the event to be processed. + */ +void libxl_unreg_aer_events_handler(libxl_ctx *, uint32_t, libxl_aer_watch *) + LIBXL_EXTERNAL_CALLERS_ONLY; + typedef struct libxl__evgen_disk_eject libxl_evgen_disk_eject; int libxl_evenable_disk_eject(libxl_ctx *ctx, uint32_t domid, const char *vdev, libxl_ev_user, libxl_evgen_disk_eject **evgen_out); diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index afe6652..2b74286 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -352,6 +352,13 @@ struct libxl__ev_child { LIBXL_LIST_ENTRY(struct libxl__ev_child) entry; }; +/* + * Structure used for AER event handling. + */ +struct libxl__aer_watch { + uint32_t domid; + libxl__ev_xswatch watch; +}; /* * evgen structures, which are the state we use for generating diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c index 65ad5e5..feedf27 100644 --- a/tools/libxl/libxl_pci.c +++ b/tools/libxl/libxl_pci.c @@ -1678,6 +1678,96 @@ static int libxl_device_pci_compare(libxl_device_pci *d1, return COMPARE_PCI(d1, d2); } +static void aer_backend_watch_callback(libxl__egc *egc, + libxl__ev_xswatch *watch, + const char *watch_path, + const char *event_path) +{ + EGC_GC; + libxl_aer_watch *aer_ws = CONTAINER_OF(watch, *aer_ws, watch); + int rc; + uint32_t dom, bus, dev, fn; + uint32_t domid = aer_ws->domid; + char *p, *path; + const char *aerFailedSBDF; + libxl_device_pci pcidev; + + /* Extract the backend directory. */ + path = libxl__strdup(gc, event_path); + p = strrchr(path, '/'); + if ((p == NULL) || (strcmp(p, "/aerFailedSBDF") != 0)) + return; + /* Truncate the string so it points to the backend directory. */ + *p = '\0'; + + /* Fetch the value of the failed PCI device. */ + rc = libxl__xs_read_checked(gc, XBT_NULL, + GCSPRINTF("%s/aerFailedSBDF", path), &aerFailedSBDF); + if (rc || !aerFailedSBDF) + return; + sscanf(aerFailedSBDF, "%x:%x:%x.%x", &dom, &bus, &dev, &fn); + + libxl_device_pci_init(&pcidev); + pcidev_struct_fill(&pcidev, dom, bus, dev, fn, 0); + /* Forcibly remove the device from the guest */ + rc = libxl__device_pci_remove_common(gc, domid, &pcidev, 1); + if (rc) + LOGD(ERROR, domid, " libxl__device_pci_remove_common() failed, rc=x%x", + (unsigned int)rc); + + return; +} + +int libxl_reg_aer_events_handler(libxl_ctx *ctx, + uint32_t domid, + libxl_aer_watch **aer_ws_out) +{ + int rc = 0; + int dom0_domid; + char *be_path; + libxl_aer_watch *aer_ws = NULL; + GC_INIT(ctx); + + *aer_ws_out = NULL; + + rc = libxl__get_domid(gc, (uint32_t *)(&dom0_domid)); + if (rc) { + LOGD(ERROR, domid, " libxl__get_domid() failed, rc = %d", rc); + goto out; + } + + aer_ws = malloc(sizeof(libxl_aer_watch)); + if (!aer_ws) { + rc = ERROR_NOMEM; + goto out; + } + memset(aer_ws, 0, sizeof(libxl_aer_watch)); + + aer_ws->domid = domid; + be_path = GCSPRINTF("/local/domain/%d/backend/pci/%u/%d/%s", + dom0_domid, domid, dom0_domid, "aerFailedSBDF"); + rc = libxl__ev_xswatch_register(gc, &aer_ws->watch, + aer_backend_watch_callback, be_path); + *aer_ws_out = aer_ws; + +out: + GC_FREE; + return rc; +} + +void libxl_unreg_aer_events_handler(libxl_ctx *ctx, + uint32_t domid, + libxl_aer_watch *aer_ws) +{ + GC_INIT(ctx); + + libxl__ev_xswatch_deregister(gc, &aer_ws->watch); + + free(aer_ws); + GC_FREE; + return; +} + DEFINE_DEVICE_TYPE_STRUCT_X(pcidev, pci); /*