From patchwork Mon Aug 7 23:54:56 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Venu Busireddy X-Patchwork-Id: 9886325 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4D02460364 for ; Mon, 7 Aug 2017 23:56:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F1A7286E6 for ; Mon, 7 Aug 2017 23:56:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 33D1D28737; Mon, 7 Aug 2017 23:56:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EA377286E6 for ; Mon, 7 Aug 2017 23:56:13 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1derqa-0000T3-QZ; Mon, 07 Aug 2017 23:54:08 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1derqZ-0000Sc-Iq for xen-devel@lists.xen.org; Mon, 07 Aug 2017 23:54:07 +0000 Received: from [85.158.137.68] by server-13.bemta-3.messagelabs.com id 2F/DB-01862-E9DF8895; Mon, 07 Aug 2017 23:54:06 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmpnkeJIrShJLcpLzFFi42LpnVTnqjvvb0e kwbTDXBZLPi5mcWD0OLr7N1MAYxRrZl5SfkUCa0b3xtesBXf1K+7PmM7ewLhStYuRi0NIYAKT xNGzJ9i7GDmBnF+MEutbbSDsDYwSl77pQhRNZJQ4/GIlWBGbgIHE0cM9rCAJEYE2RonVJy4zg iSYBWok+r6tYAWxhQXCJSav6gGLswioSlz49wwszitgK7H89EVmEFtCQE5i6fbrYDangJ3E7V 03GSE220pc+9LNAlFjKPF541LmCYx8CxgZVjGqF6cWlaUW6ZroJRVlpmeU5CZm5ugaGhjr5aY WFyemp+YkJhXrJefnbmIEBgoDEOxgbPzidIhRkoNJSZSXc0tHpBBfUn5KZUZicUZ8UWlOavEh RhkODiUJXoM/QDnBotT01Iq0zBxgyMKkJTh4lER41UHSvMUFibnFmekQqVOMuhyvJvz/xiTEk peflyolzvvvN1CRAEhRRmke3AhY/FxilJUS5mUEOkqIpyC1KDezBFX+FaM4B6OSMO8+kCk8mX klcJteAR3BBHTEm8RWkCNKEhFSUg2M7v8YTkx0fmH8KeOqQOmEm6lzbHQZlCP27PvwUWT+woO dE4ru798dEelxg2e6E8uvhW38K83bN8b1Bq/eZ+rJocZ0mfl1Tdz6mKqpYRFthkKTdsxj3bOw b/fqR3F+Ef8+X1/w2fzUMY0JjdvbIpQj9s5l7upddDezyMrcd9XmqLw11qesJDU/KbEUZyQaa jEXFScCABKaO0OaAgAA X-Env-Sender: venu.busireddy@oracle.com X-Msg-Ref: server-7.tower-31.messagelabs.com!1502150044!101432797!1 X-Originating-IP: [141.146.126.69] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTQxLjE0Ni4xMjYuNjkgPT4gMjc3MjE4\n X-StarScan-Received: X-StarScan-Version: 9.4.25; banners=-,-,- X-VirusChecked: Checked Received: (qmail 63503 invoked from network); 7 Aug 2017 23:54:05 -0000 Received: from aserp1040.oracle.com (HELO aserp1040.oracle.com) (141.146.126.69) by server-7.tower-31.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 7 Aug 2017 23:54:05 -0000 Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v77Ns2bu004006 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 7 Aug 2017 23:54:03 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id v77Ns2uk011864 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 7 Aug 2017 23:54:02 GMT Received: from abhmp0012.oracle.com (abhmp0012.oracle.com [141.146.116.18]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id v77Ns2sC007319; Mon, 7 Aug 2017 23:54:02 GMT Received: from ban25uut183.us.oracle.com (/10.153.74.183) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 07 Aug 2017 16:54:01 -0700 From: Venu Busireddy To: venu.busireddy@oracle.com, xen-devel@lists.xen.org, Ian Jackson , Wei Liu Date: Mon, 7 Aug 2017 18:54:56 -0500 Message-Id: <20170807235457.3943-2-venu.busireddy@oracle.com> X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170807235457.3943-1-venu.busireddy@oracle.com> References: <20170807235457.3943-1-venu.busireddy@oracle.com> X-Source-IP: userv0022.oracle.com [156.151.31.74] Cc: Andrew Cooper , Jan Beulich Subject: [Xen-devel] [PATCH v3 1/2] libxl: Implement the handler to handle unrecoverable AER errors X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Implement the callback function to handle unrecoverable AER errors, and also the public APIs that can be used to register/unregister the handler. When an AER error occurs, the handler will forcibly remove the erring PCIe device from the guest. Signed-off-by: Venu Busireddy --- tools/libxl/libxl.h | 14 +++++++ tools/libxl/libxl_event.h | 13 +++++++ tools/libxl/libxl_internal.h | 7 ++++ tools/libxl/libxl_pci.c | 90 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 124 insertions(+) diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index 7cf0f31..c5af0aa 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -1044,6 +1044,20 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src); */ #define LIBXL_HAVE_QED 1 +/* LIBXL_HAVE_REG_AER_EVENTS_HANDLER + * + * If it is defined, libxl has a library function called + * libxl_reg_aer_events_handler. + */ +#define LIBXL_HAVE_REG_AER_EVENTS_HANDLER 1 + +/* LIBXL_HAVE_UNREG_AER_EVENTS_HANDLER + * + * If it is defined, libxl has a library function called + * libxl_unreg_aer_events_handler. + */ +#define LIBXL_HAVE_UNREG_AER_EVENTS_HANDLER 1 + typedef char **libxl_string_list; void libxl_string_list_dispose(libxl_string_list *sl); int libxl_string_list_length(const libxl_string_list *sl); diff --git a/tools/libxl/libxl_event.h b/tools/libxl/libxl_event.h index 1ea789e..1aea906 100644 --- a/tools/libxl/libxl_event.h +++ b/tools/libxl/libxl_event.h @@ -184,6 +184,19 @@ void libxl_evdisable_domain_death(libxl_ctx *ctx, libxl_evgen_domain_death*); * may generate only a DEATH event. */ +typedef struct libxl__aer_watch libxl_aer_watch; +int libxl_reg_aer_events_handler(libxl_ctx *, uint32_t, libxl_aer_watch **) + LIBXL_EXTERNAL_CALLERS_ONLY; + /* + * Registers a handler to handle the occurrence of unrecoverable AER errors. + * This function depends on the calling application running the libxl's + * internal event loop. Toolstacks that do not use libxl's internal + * event loop must arrange to have their own event loop created and enter + * libxl (say, call libxl_event_wait()), to enable the event to be processed. + */ +void libxl_unreg_aer_events_handler(libxl_ctx *, uint32_t, libxl_aer_watch *) + LIBXL_EXTERNAL_CALLERS_ONLY; + typedef struct libxl__evgen_disk_eject libxl_evgen_disk_eject; int libxl_evenable_disk_eject(libxl_ctx *ctx, uint32_t domid, const char *vdev, libxl_ev_user, libxl_evgen_disk_eject **evgen_out); diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index afe6652..2b74286 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -352,6 +352,13 @@ struct libxl__ev_child { LIBXL_LIST_ENTRY(struct libxl__ev_child) entry; }; +/* + * Structure used for AER event handling. + */ +struct libxl__aer_watch { + uint32_t domid; + libxl__ev_xswatch watch; +}; /* * evgen structures, which are the state we use for generating diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c index 65ad5e5..feedf27 100644 --- a/tools/libxl/libxl_pci.c +++ b/tools/libxl/libxl_pci.c @@ -1678,6 +1678,96 @@ static int libxl_device_pci_compare(libxl_device_pci *d1, return COMPARE_PCI(d1, d2); } +static void aer_backend_watch_callback(libxl__egc *egc, + libxl__ev_xswatch *watch, + const char *watch_path, + const char *event_path) +{ + EGC_GC; + libxl_aer_watch *aer_ws = CONTAINER_OF(watch, *aer_ws, watch); + int rc; + uint32_t dom, bus, dev, fn; + uint32_t domid = aer_ws->domid; + char *p, *path; + const char *aerFailedSBDF; + libxl_device_pci pcidev; + + /* Extract the backend directory. */ + path = libxl__strdup(gc, event_path); + p = strrchr(path, '/'); + if ((p == NULL) || (strcmp(p, "/aerFailedSBDF") != 0)) + return; + /* Truncate the string so it points to the backend directory. */ + *p = '\0'; + + /* Fetch the value of the failed PCI device. */ + rc = libxl__xs_read_checked(gc, XBT_NULL, + GCSPRINTF("%s/aerFailedSBDF", path), &aerFailedSBDF); + if (rc || !aerFailedSBDF) + return; + sscanf(aerFailedSBDF, "%x:%x:%x.%x", &dom, &bus, &dev, &fn); + + libxl_device_pci_init(&pcidev); + pcidev_struct_fill(&pcidev, dom, bus, dev, fn, 0); + /* Forcibly remove the device from the guest */ + rc = libxl__device_pci_remove_common(gc, domid, &pcidev, 1); + if (rc) + LOGD(ERROR, domid, " libxl__device_pci_remove_common() failed, rc=x%x", + (unsigned int)rc); + + return; +} + +int libxl_reg_aer_events_handler(libxl_ctx *ctx, + uint32_t domid, + libxl_aer_watch **aer_ws_out) +{ + int rc = 0; + int dom0_domid; + char *be_path; + libxl_aer_watch *aer_ws = NULL; + GC_INIT(ctx); + + *aer_ws_out = NULL; + + rc = libxl__get_domid(gc, (uint32_t *)(&dom0_domid)); + if (rc) { + LOGD(ERROR, domid, " libxl__get_domid() failed, rc = %d", rc); + goto out; + } + + aer_ws = malloc(sizeof(libxl_aer_watch)); + if (!aer_ws) { + rc = ERROR_NOMEM; + goto out; + } + memset(aer_ws, 0, sizeof(libxl_aer_watch)); + + aer_ws->domid = domid; + be_path = GCSPRINTF("/local/domain/%d/backend/pci/%u/%d/%s", + dom0_domid, domid, dom0_domid, "aerFailedSBDF"); + rc = libxl__ev_xswatch_register(gc, &aer_ws->watch, + aer_backend_watch_callback, be_path); + *aer_ws_out = aer_ws; + +out: + GC_FREE; + return rc; +} + +void libxl_unreg_aer_events_handler(libxl_ctx *ctx, + uint32_t domid, + libxl_aer_watch *aer_ws) +{ + GC_INIT(ctx); + + libxl__ev_xswatch_deregister(gc, &aer_ws->watch); + + free(aer_ws); + GC_FREE; + return; +} + DEFINE_DEVICE_TYPE_STRUCT_X(pcidev, pci); /*