From patchwork Tue Aug 2 03:57:54 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhou Jie X-Patchwork-Id: 9255077 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3E57A6077C for ; Tue, 2 Aug 2016 04:02:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2FB62284DC for ; Tue, 2 Aug 2016 04:02:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1FF88284ED; Tue, 2 Aug 2016 04:02:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 821B6284DC for ; Tue, 2 Aug 2016 04:02:58 +0000 (UTC) Received: from localhost ([::1]:53872 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bUQuv-000490-MI for patchwork-qemu-devel@patchwork.kernel.org; Tue, 02 Aug 2016 00:02:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50778) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bUQuS-0003zT-KS for qemu-devel@nongnu.org; Tue, 02 Aug 2016 00:02:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bUQuM-0008CF-VB for qemu-devel@nongnu.org; Tue, 02 Aug 2016 00:02:27 -0400 Received: from [59.151.112.132] (port=1742 helo=heian.cn.fujitsu.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bUQuJ-0008AC-La for qemu-devel@nongnu.org; Tue, 02 Aug 2016 00:02:22 -0400 X-IronPort-AV: E=Sophos;i="5.22,518,1449504000"; d="scan'208";a="9412302" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 02 Aug 2016 12:01:59 +0800 Received: from G08CNEXCHPEKD03.g08.fujitsu.local (unknown [10.167.33.85]) by cn.fujitsu.com (Postfix) with ESMTP id 0C1DD42B87FD; Tue, 2 Aug 2016 12:01:57 +0800 (CST) Received: from localhost.localdomain.localdomain (10.167.226.45) by G08CNEXCHPEKD03.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.279.2; Tue, 2 Aug 2016 12:01:55 +0800 From: Zhou Jie To: Date: Tue, 2 Aug 2016 11:57:54 +0800 Message-ID: <1470110274-14585-1-git-send-email-zhoujie2011@cn.fujitsu.com> X-Mailer: git-send-email 2.5.5 MIME-Version: 1.0 X-Originating-IP: [10.167.226.45] X-yoursite-MailScanner-ID: 0C1DD42B87FD.A9393 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: zhoujie2011@cn.fujitsu.com X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 59.151.112.132 Subject: [Qemu-devel] [PATCH v3] vfio : add aer process X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: izumi.taku@jp.fujitsu.com, fan.chen@easystack.cn, Zhou Jie , linux-kernel@vger.kernel.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP During aer err occurs and resume do following to protect device from being accessed. 1. Make config space read only. 2. Disable INTx/MSI Interrupt. 3. Do nothing for bar regions. Signed-off-by: Zhou Jie --- v2-v3: 1. Call init_completion() in vfio_pci_probe. 2. Call reinit_completion() in vfio_pci_aer_err_detected. 3. Remove unnecessary brackets. v1-v2: 1. Add aer process to vfio driver. drivers/vfio/pci/vfio_pci.c | 48 +++++++++++++++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci_private.h | 2 ++ include/uapi/linux/vfio.h | 2 ++ 3 files changed, 52 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index d624a52..4c246a1 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -648,6 +648,15 @@ static long vfio_pci_ioctl(void *device_data, struct vfio_pci_device *vdev = device_data; unsigned long minsz; + if (vdev->aer_error_in_progress && (cmd == VFIO_DEVICE_SET_IRQS || + cmd == VFIO_DEVICE_RESET || cmd == VFIO_DEVICE_PCI_HOT_RESET)) { + int ret; + ret = wait_for_completion_interruptible( + &vdev->aer_error_completion); + if (ret) + return ret; + } + if (cmd == VFIO_DEVICE_GET_INFO) { struct vfio_device_info info; @@ -664,6 +673,10 @@ static long vfio_pci_ioctl(void *device_data, if (vdev->reset_works) info.flags |= VFIO_DEVICE_FLAGS_RESET; + info.flags |= VFIO_DEVICE_FLAGS_AERPROCESS; + if (vdev->aer_error_in_progress) + info.flags |= VFIO_DEVICE_FLAGS_INAERPROCESS; + info.num_regions = VFIO_PCI_NUM_REGIONS + vdev->num_regions; info.num_irqs = VFIO_PCI_NUM_IRQS; @@ -1070,6 +1083,13 @@ static ssize_t vfio_pci_rw(void *device_data, char __user *buf, switch (index) { case VFIO_PCI_CONFIG_REGION_INDEX: + if (vdev->aer_error_in_progress && iswrite) { + int ret; + ret = wait_for_completion_interruptible( + &vdev->aer_error_completion); + if (ret) + return ret; + } return vfio_pci_config_rw(vdev, buf, count, ppos, iswrite); case VFIO_PCI_ROM_REGION_INDEX: @@ -1228,6 +1248,7 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) vdev->irq_type = VFIO_PCI_NUM_IRQS; mutex_init(&vdev->igate); spin_lock_init(&vdev->irqlock); + init_completion(&vdev->aer_error_completion); ret = vfio_add_group_dev(&pdev->dev, &vfio_pci_ops, vdev); if (ret) { @@ -1300,6 +1321,11 @@ static pci_ers_result_t vfio_pci_aer_err_detected(struct pci_dev *pdev, mutex_lock(&vdev->igate); + vdev->aer_error_in_progress = true; + reinit_completion(&vdev->aer_error_completion); + vfio_pci_set_irqs_ioctl(vdev, VFIO_IRQ_SET_DATA_NONE | + VFIO_IRQ_SET_ACTION_TRIGGER, + vdev->irq_type, 0, 0, NULL); if (vdev->err_trigger) eventfd_signal(vdev->err_trigger, 1); @@ -1310,8 +1336,30 @@ static pci_ers_result_t vfio_pci_aer_err_detected(struct pci_dev *pdev, return PCI_ERS_RESULT_CAN_RECOVER; } +static void vfio_pci_aer_resume(struct pci_dev *pdev) +{ + struct vfio_pci_device *vdev; + struct vfio_device *device; + + device = vfio_device_get_from_dev(&pdev->dev); + if (device == NULL) + return; + + vdev = vfio_device_data(device); + if (vdev == NULL) { + vfio_device_put(device); + return; + } + + vdev->aer_error_in_progress = false; + complete_all(&vdev->aer_error_completion); + + vfio_device_put(device); +} + static const struct pci_error_handlers vfio_err_handlers = { .error_detected = vfio_pci_aer_err_detected, + .resume = vfio_pci_aer_resume, }; static struct pci_driver vfio_pci_driver = { diff --git a/drivers/vfio/pci/vfio_pci_private.h b/drivers/vfio/pci/vfio_pci_private.h index 2128de8..7430d92 100644 --- a/drivers/vfio/pci/vfio_pci_private.h +++ b/drivers/vfio/pci/vfio_pci_private.h @@ -91,6 +91,8 @@ struct vfio_pci_device { bool has_vga; bool needs_reset; bool nointx; + bool aer_error_in_progress; + struct completion aer_error_completion; struct pci_saved_state *pci_saved_state; int refcnt; struct eventfd_ctx *err_trigger; diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 255a211..59b9cf6 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -198,6 +198,8 @@ struct vfio_device_info { #define VFIO_DEVICE_FLAGS_PCI (1 << 1) /* vfio-pci device */ #define VFIO_DEVICE_FLAGS_PLATFORM (1 << 2) /* vfio-platform device */ #define VFIO_DEVICE_FLAGS_AMBA (1 << 3) /* vfio-amba device */ +#define VFIO_DEVICE_FLAGS_AERPROCESS (1 << 4) /* support aer error progress */ +#define VFIO_DEVICE_FLAGS_INAERPROCESS (1 << 5)/* status in aer error progress */ __u32 num_regions; /* Max region index + 1 */ __u32 num_irqs; /* Max IRQ index + 1 */ };