From patchwork Wed Mar 27 03:45:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Parav Pandit X-Patchwork-Id: 10872463 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 833CD1390 for ; Wed, 27 Mar 2019 03:46:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6CEFA28A6F for ; Wed, 27 Mar 2019 03:46:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 60B3D28BB6; Wed, 27 Mar 2019 03:46:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D66F428B7F for ; Wed, 27 Mar 2019 03:45:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733004AbfC0Dp4 (ORCPT ); Tue, 26 Mar 2019 23:45:56 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40503 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732963AbfC0Dpz (ORCPT ); Tue, 26 Mar 2019 23:45:55 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from parav@mellanox.com) with ESMTPS (AES256-SHA encrypted); 27 Mar 2019 05:45:50 +0200 Received: from sw-mtx-036.mtx.labs.mlnx (sw-mtx-036.mtx.labs.mlnx [10.12.150.149]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x2R3jlG5031531; Wed, 27 Mar 2019 05:45:49 +0200 From: Parav Pandit To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, kwankhede@nvidia.com, alex.williamson@redhat.com Cc: cjia@nvidia.com, parav@mellanox.com Subject: [PATCHv1 1/7] vfio/mdev: Avoid release parent reference during error path Date: Tue, 26 Mar 2019 22:45:39 -0500 Message-Id: <1553658345-43995-2-git-send-email-parav@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1553658345-43995-1-git-send-email-parav@mellanox.com> References: <1553658345-43995-1-git-send-email-parav@mellanox.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP During mdev parent registration in mdev_register_device(), if parent device is duplicate, it releases the reference of existing parent device. This is incorrect. Existing parent device should not be touched. Fixes: 7b96953bc640 ("vfio: Mediated device Core driver") Reviewed By: Kirti Wankhede Reviewed-by: Maxim Levitsky Signed-off-by: Parav Pandit Reviewed-by: Cornelia Huck --- drivers/vfio/mdev/mdev_core.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c index b96fedc..1299d2e 100644 --- a/drivers/vfio/mdev/mdev_core.c +++ b/drivers/vfio/mdev/mdev_core.c @@ -181,6 +181,7 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops) /* Check for duplicate */ parent = __find_parent_device(dev); if (parent) { + parent = NULL; ret = -EEXIST; goto add_dev_err; } From patchwork Wed Mar 27 03:45:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Parav Pandit X-Patchwork-Id: 10872475 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EB0041390 for ; Wed, 27 Mar 2019 03:46:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D343728A6F for ; Wed, 27 Mar 2019 03:46:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C4B0B28B7F; Wed, 27 Mar 2019 03:46:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 74CCF28A6F for ; Wed, 27 Mar 2019 03:46:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732992AbfC0Dp4 (ORCPT ); Tue, 26 Mar 2019 23:45:56 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40506 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732961AbfC0Dpz (ORCPT ); Tue, 26 Mar 2019 23:45:55 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from parav@mellanox.com) with ESMTPS (AES256-SHA encrypted); 27 Mar 2019 05:45:52 +0200 Received: from sw-mtx-036.mtx.labs.mlnx (sw-mtx-036.mtx.labs.mlnx [10.12.150.149]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x2R3jlG6031531; Wed, 27 Mar 2019 05:45:51 +0200 From: Parav Pandit To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, kwankhede@nvidia.com, alex.williamson@redhat.com Cc: cjia@nvidia.com, parav@mellanox.com Subject: [PATCHv1 2/7] vfio/mdev: Removed unused kref Date: Tue, 26 Mar 2019 22:45:40 -0500 Message-Id: <1553658345-43995-3-git-send-email-parav@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1553658345-43995-1-git-send-email-parav@mellanox.com> References: <1553658345-43995-1-git-send-email-parav@mellanox.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Remove unused kref from the mdev_device structure. Fixes: 7b96953bc640 ("vfio: Mediated device Core driver") Reviewed By: Kirti Wankhede Reviewed-by: Maxim Levitsky Signed-off-by: Parav Pandit Reviewed-by: Cornelia Huck --- drivers/vfio/mdev/mdev_core.c | 1 - drivers/vfio/mdev/mdev_private.h | 1 - 2 files changed, 2 deletions(-) diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c index 1299d2e..00ca613 100644 --- a/drivers/vfio/mdev/mdev_core.c +++ b/drivers/vfio/mdev/mdev_core.c @@ -311,7 +311,6 @@ int mdev_device_create(struct kobject *kobj, mutex_unlock(&mdev_list_lock); mdev->parent = parent; - kref_init(&mdev->ref); mdev->dev.parent = dev; mdev->dev.bus = &mdev_bus_type; diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h index 379758c..ddcf9c7 100644 --- a/drivers/vfio/mdev/mdev_private.h +++ b/drivers/vfio/mdev/mdev_private.h @@ -30,7 +30,6 @@ struct mdev_device { struct mdev_parent *parent; guid_t uuid; void *driver_data; - struct kref ref; struct list_head next; struct kobject *type_kobj; bool active; From patchwork Wed Mar 27 03:45:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Parav Pandit X-Patchwork-Id: 10872477 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F3C651390 for ; Wed, 27 Mar 2019 03:46:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DFA0128A6F for ; Wed, 27 Mar 2019 03:46:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D3C9A28B7F; Wed, 27 Mar 2019 03:46:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7644128A6F for ; Wed, 27 Mar 2019 03:46:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732983AbfC0Dp4 (ORCPT ); Tue, 26 Mar 2019 23:45:56 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40509 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732971AbfC0Dpz (ORCPT ); Tue, 26 Mar 2019 23:45:55 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from parav@mellanox.com) with ESMTPS (AES256-SHA encrypted); 27 Mar 2019 05:45:54 +0200 Received: from sw-mtx-036.mtx.labs.mlnx (sw-mtx-036.mtx.labs.mlnx [10.12.150.149]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x2R3jlG7031531; Wed, 27 Mar 2019 05:45:52 +0200 From: Parav Pandit To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, kwankhede@nvidia.com, alex.williamson@redhat.com Cc: cjia@nvidia.com, parav@mellanox.com Subject: [PATCHv1 3/7] vfio/mdev: Drop redundant extern for exported symbols Date: Tue, 26 Mar 2019 22:45:41 -0500 Message-Id: <1553658345-43995-4-git-send-email-parav@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1553658345-43995-1-git-send-email-parav@mellanox.com> References: <1553658345-43995-1-git-send-email-parav@mellanox.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP There is no need use 'extern' for exported functions. Reviewed-by: Maxim Levitsky Signed-off-by: Parav Pandit Acked-by: Cornelia Huck --- include/linux/mdev.h | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/include/linux/mdev.h b/include/linux/mdev.h index d7aee90..4924d80 100644 --- a/include/linux/mdev.h +++ b/include/linux/mdev.h @@ -118,21 +118,20 @@ struct mdev_driver { #define to_mdev_driver(drv) container_of(drv, struct mdev_driver, driver) -extern void *mdev_get_drvdata(struct mdev_device *mdev); -extern void mdev_set_drvdata(struct mdev_device *mdev, void *data); -extern const guid_t *mdev_uuid(struct mdev_device *mdev); +void *mdev_get_drvdata(struct mdev_device *mdev); +void mdev_set_drvdata(struct mdev_device *mdev, void *data); +const guid_t *mdev_uuid(struct mdev_device *mdev); extern struct bus_type mdev_bus_type; -extern int mdev_register_device(struct device *dev, - const struct mdev_parent_ops *ops); -extern void mdev_unregister_device(struct device *dev); +int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops); +void mdev_unregister_device(struct device *dev); -extern int mdev_register_driver(struct mdev_driver *drv, struct module *owner); -extern void mdev_unregister_driver(struct mdev_driver *drv); +int mdev_register_driver(struct mdev_driver *drv, struct module *owner); +void mdev_unregister_driver(struct mdev_driver *drv); -extern struct device *mdev_parent_dev(struct mdev_device *mdev); -extern struct device *mdev_dev(struct mdev_device *mdev); -extern struct mdev_device *mdev_from_dev(struct device *dev); +struct device *mdev_parent_dev(struct mdev_device *mdev); +struct device *mdev_dev(struct mdev_device *mdev); +struct mdev_device *mdev_from_dev(struct device *dev); #endif /* MDEV_H */ From patchwork Wed Mar 27 03:45:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Parav Pandit X-Patchwork-Id: 10872471 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C5231390 for ; Wed, 27 Mar 2019 03:46:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6551328A6F for ; Wed, 27 Mar 2019 03:46:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 59A9928B7F; Wed, 27 Mar 2019 03:46:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 082E428A6F for ; Wed, 27 Mar 2019 03:46:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733060AbfC0DqB (ORCPT ); Tue, 26 Mar 2019 23:46:01 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40535 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1733027AbfC0DqA (ORCPT ); Tue, 26 Mar 2019 23:46:00 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from parav@mellanox.com) with ESMTPS (AES256-SHA encrypted); 27 Mar 2019 05:45:55 +0200 Received: from sw-mtx-036.mtx.labs.mlnx (sw-mtx-036.mtx.labs.mlnx [10.12.150.149]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x2R3jlG8031531; Wed, 27 Mar 2019 05:45:54 +0200 From: Parav Pandit To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, kwankhede@nvidia.com, alex.williamson@redhat.com Cc: cjia@nvidia.com, parav@mellanox.com Subject: [PATCHv1 4/7] vfio/mdev: Avoid masking error code to EBUSY Date: Tue, 26 Mar 2019 22:45:42 -0500 Message-Id: <1553658345-43995-5-git-send-email-parav@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1553658345-43995-1-git-send-email-parav@mellanox.com> References: <1553658345-43995-1-git-send-email-parav@mellanox.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Instead of masking return error to -EBUSY, return actual error returned by the driver. Reviewed-by: Maxim Levitsky Signed-off-by: Parav Pandit Reviewed-by: Cornelia Huck --- drivers/vfio/mdev/mdev_core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c index 00ca613..836d319 100644 --- a/drivers/vfio/mdev/mdev_core.c +++ b/drivers/vfio/mdev/mdev_core.c @@ -141,7 +141,7 @@ static int mdev_device_remove_ops(struct mdev_device *mdev, bool force_remove) */ ret = parent->ops->remove(mdev); if (ret && !force_remove) - return -EBUSY; + return ret; sysfs_remove_groups(&mdev->dev.kobj, parent->ops->mdev_attr_groups); return 0; From patchwork Wed Mar 27 03:45:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Parav Pandit X-Patchwork-Id: 10872473 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AA17A186E for ; Wed, 27 Mar 2019 03:46:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 95E1828A6F for ; Wed, 27 Mar 2019 03:46:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8A07E28B7F; Wed, 27 Mar 2019 03:46:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 35FB728A6F for ; Wed, 27 Mar 2019 03:46:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733162AbfC0DqV (ORCPT ); Tue, 26 Mar 2019 23:46:21 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40538 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1733035AbfC0DqA (ORCPT ); Tue, 26 Mar 2019 23:46:00 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from parav@mellanox.com) with ESMTPS (AES256-SHA encrypted); 27 Mar 2019 05:45:57 +0200 Received: from sw-mtx-036.mtx.labs.mlnx (sw-mtx-036.mtx.labs.mlnx [10.12.150.149]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x2R3jlG9031531; Wed, 27 Mar 2019 05:45:55 +0200 From: Parav Pandit To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, kwankhede@nvidia.com, alex.williamson@redhat.com Cc: cjia@nvidia.com, parav@mellanox.com Subject: [PATCHv1 5/7] vfio/mdev: Follow correct remove sequence Date: Tue, 26 Mar 2019 22:45:43 -0500 Message-Id: <1553658345-43995-6-git-send-email-parav@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1553658345-43995-1-git-send-email-parav@mellanox.com> References: <1553658345-43995-1-git-send-email-parav@mellanox.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP mdev_remove_sysfs_files() should follow exact mirror sequence of a create, similar to what is followed in error unwinding path of mdev_create_sysfs_files(). Fixes: 6a62c1dfb5c7 ("vfio/mdev: Re-order sysfs attribute creation") Reviewed-by: Maxim Levitsky Signed-off-by: Parav Pandit Reviewed-by: Cornelia Huck --- drivers/vfio/mdev/mdev_sysfs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/vfio/mdev/mdev_sysfs.c b/drivers/vfio/mdev/mdev_sysfs.c index 5193a0e..cbf94b8 100644 --- a/drivers/vfio/mdev/mdev_sysfs.c +++ b/drivers/vfio/mdev/mdev_sysfs.c @@ -280,7 +280,7 @@ int mdev_create_sysfs_files(struct device *dev, struct mdev_type *type) void mdev_remove_sysfs_files(struct device *dev, struct mdev_type *type) { + sysfs_remove_files(&dev->kobj, mdev_device_attrs); sysfs_remove_link(&dev->kobj, "mdev_type"); sysfs_remove_link(type->devices_kobj, dev_name(dev)); - sysfs_remove_files(&dev->kobj, mdev_device_attrs); } From patchwork Wed Mar 27 03:45:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Parav Pandit X-Patchwork-Id: 10872469 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CE2FA186E for ; Wed, 27 Mar 2019 03:46:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B860028A6F for ; Wed, 27 Mar 2019 03:46:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AC3F028B7F; Wed, 27 Mar 2019 03:46:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4B8FB28A6F for ; Wed, 27 Mar 2019 03:46:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733146AbfC0DqQ (ORCPT ); Tue, 26 Mar 2019 23:46:16 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40542 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1733037AbfC0DqB (ORCPT ); Tue, 26 Mar 2019 23:46:01 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from parav@mellanox.com) with ESMTPS (AES256-SHA encrypted); 27 Mar 2019 05:45:58 +0200 Received: from sw-mtx-036.mtx.labs.mlnx (sw-mtx-036.mtx.labs.mlnx [10.12.150.149]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x2R3jlGA031531; Wed, 27 Mar 2019 05:45:57 +0200 From: Parav Pandit To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, kwankhede@nvidia.com, alex.williamson@redhat.com Cc: cjia@nvidia.com, parav@mellanox.com Subject: [PATCHv1 6/7] vfio/mdev: Fix aborting mdev child device removal if one fails Date: Tue, 26 Mar 2019 22:45:44 -0500 Message-Id: <1553658345-43995-7-git-send-email-parav@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1553658345-43995-1-git-send-email-parav@mellanox.com> References: <1553658345-43995-1-git-send-email-parav@mellanox.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP device_for_each_child() stops executing callback function for remaining child devices, if callback hits an error. Each child mdev device is independent of each other. While unregistering parent device, mdev core must remove all child mdev devices. Therefore, mdev_device_remove_cb() always returns success so that device_for_each_child doesn't abort if one child removal hits error. While at it, improve remove and unregister functions for below simplicity. There isn't need to pass forced flag pointer during mdev parent removal which invokes mdev_device_remove(). So simplify the flow. mdev_device_remove() is called from two paths. 1. mdev_unregister_driver() mdev_device_remove_cb() mdev_device_remove() 2. remove_store() mdev_device_remove() When device is removed by user using remote_store(), device under removal is mdev device. When device is removed during parent device removal using generic child iterator, mdev check is already done using dev_is_mdev(). Hence, remove the unnecessary loop in mdev_device_remove(). Fixes: 7b96953bc640 ("vfio: Mediated device Core driver") Reviewed-by: Maxim Levitsky Signed-off-by: Parav Pandit --- drivers/vfio/mdev/mdev_core.c | 23 +++++------------------ 1 file changed, 5 insertions(+), 18 deletions(-) diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c index 836d319..aefcf34 100644 --- a/drivers/vfio/mdev/mdev_core.c +++ b/drivers/vfio/mdev/mdev_core.c @@ -149,10 +149,10 @@ static int mdev_device_remove_ops(struct mdev_device *mdev, bool force_remove) static int mdev_device_remove_cb(struct device *dev, void *data) { - if (!dev_is_mdev(dev)) - return 0; + if (dev_is_mdev(dev)) + mdev_device_remove(dev, true); - return mdev_device_remove(dev, data ? *(bool *)data : true); + return 0; } /* @@ -240,7 +240,6 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops) void mdev_unregister_device(struct device *dev) { struct mdev_parent *parent; - bool force_remove = true; mutex_lock(&parent_list_lock); parent = __find_parent_device(dev); @@ -254,8 +253,7 @@ void mdev_unregister_device(struct device *dev) list_del(&parent->next); class_compat_remove_link(mdev_bus_compat_class, dev, NULL); - device_for_each_child(dev, (void *)&force_remove, - mdev_device_remove_cb); + device_for_each_child(dev, NULL, mdev_device_remove_cb); parent_remove_sysfs_files(parent); @@ -348,24 +346,13 @@ int mdev_device_create(struct kobject *kobj, int mdev_device_remove(struct device *dev, bool force_remove) { - struct mdev_device *mdev, *tmp; + struct mdev_device *mdev; struct mdev_parent *parent; struct mdev_type *type; int ret; mdev = to_mdev_device(dev); - mutex_lock(&mdev_list_lock); - list_for_each_entry(tmp, &mdev_list, next) { - if (tmp == mdev) - break; - } - - if (tmp != mdev) { - mutex_unlock(&mdev_list_lock); - return -ENODEV; - } - if (!mdev->active) { mutex_unlock(&mdev_list_lock); return -EAGAIN; From patchwork Wed Mar 27 03:45:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Parav Pandit X-Patchwork-Id: 10872467 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6CC94186E for ; Wed, 27 Mar 2019 03:46:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 55E1928A8B for ; Wed, 27 Mar 2019 03:46:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4A29128B88; Wed, 27 Mar 2019 03:46:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 71AB428A8B for ; Wed, 27 Mar 2019 03:46:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733103AbfC0DqK (ORCPT ); Tue, 26 Mar 2019 23:46:10 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40563 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1733106AbfC0DqG (ORCPT ); Tue, 26 Mar 2019 23:46:06 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from parav@mellanox.com) with ESMTPS (AES256-SHA encrypted); 27 Mar 2019 05:46:00 +0200 Received: from sw-mtx-036.mtx.labs.mlnx (sw-mtx-036.mtx.labs.mlnx [10.12.150.149]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x2R3jlGB031531; Wed, 27 Mar 2019 05:45:59 +0200 From: Parav Pandit To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, kwankhede@nvidia.com, alex.williamson@redhat.com Cc: cjia@nvidia.com, parav@mellanox.com Subject: [PATCHv1 7/7] vfio/mdev: Fix race conditions with mdev device life cycle APIs Date: Tue, 26 Mar 2019 22:45:45 -0500 Message-Id: <1553658345-43995-8-git-send-email-parav@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1553658345-43995-1-git-send-email-parav@mellanox.com> References: <1553658345-43995-1-git-send-email-parav@mellanox.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Below race condition and call trace exist with current device life cycle sequence. 1. In following sequence, child devices created while removing mdev parent device can be left out, or it may lead to race of removing half initialized child mdev devices. issue-1: -------- cpu-0 cpu-1 ----- ----- mdev_unregister_device() device_for_each_child() mdev_device_remove_cb() mdev_device_remove() create_store() mdev_device_create() [...] device_register() parent_remove_sysfs_files() /* BUG: device added by cpu-0 * whose parent is getting removed. */ issue-2: -------- cpu-0 cpu-1 ----- ----- create_store() mdev_device_create() [...] device_register() [...] mdev_unregister_device() device_for_each_child() mdev_device_remove_cb() mdev_device_remove() mdev_create_sysfs_files() /* BUG: create is adding * sysfs files for a device * which is undergoing removal. */ parent_remove_sysfs_files() 2. Below crash is observed when user initiated remove is in progress and mdev_unregister_driver() completes parent unregistration. cpu-0 cpu-1 ----- ----- remove_store() mdev_device_remove() active = false; mdev_unregister_device() remove type [...] mdev_remove_ops() crashes. This is similar race like create() racing with mdev_unregister_device(). mtty mtty: MDEV: Registered iommu: Adding device 83b8f4f2-509f-382f-3c1e-e6bfe0fa1001 to group 57 vfio_mdev 83b8f4f2-509f-382f-3c1e-e6bfe0fa1001: MDEV: group_id = 57 mtty mtty: MDEV: Unregistering mtty_dev: Unloaded! BUG: unable to handle kernel paging request at ffffffffc027d668 PGD af9818067 P4D af9818067 PUD af981a067 PMD 8583c3067 PTE 0 Oops: 0000 [#1] SMP PTI CPU: 15 PID: 3517 Comm: bash Kdump: loaded Not tainted 5.0.0-rc7-vdevbus+ #2 Hardware name: Supermicro SYS-6028U-TR4+/X10DRU-i+, BIOS 2.0b 08/09/2016 RIP: 0010:mdev_device_remove_ops+0x1a/0x50 [mdev] Call Trace: mdev_device_remove+0xef/0x130 [mdev] remove_store+0x77/0xa0 [mdev] kernfs_fop_write+0x113/0x1a0 __vfs_write+0x33/0x1b0 ? rcu_read_lock_sched_held+0x64/0x70 ? rcu_sync_lockdep_assert+0x2a/0x50 ? __sb_start_write+0x121/0x1b0 ? vfs_write+0x17c/0x1b0 vfs_write+0xad/0x1b0 ? trace_hardirqs_on_thunk+0x1a/0x1c ksys_write+0x55/0xc0 do_syscall_64+0x5a/0x210 Therefore, mdev core is improved to overcome above issues. Wait for any ongoing mdev create() and remove() to finish before unregistering parent device using srcu. This continues to allow multiple create and remove to progress in parallel. At the same time guard parent removal while parent is being access by create() and remove callbacks. mdev_device_remove() is refactored to not block on srcu when device is removed as part of parent removal. Fixes: 7b96953bc640 ("vfio: Mediated device Core driver") Signed-off-by: Parav Pandit --- drivers/vfio/mdev/mdev_core.c | 83 ++++++++++++++++++++++++++++++++++------ drivers/vfio/mdev/mdev_private.h | 6 +++ 2 files changed, 77 insertions(+), 12 deletions(-) diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c index aefcf34..fa233c8 100644 --- a/drivers/vfio/mdev/mdev_core.c +++ b/drivers/vfio/mdev/mdev_core.c @@ -84,6 +84,7 @@ static void mdev_release_parent(struct kref *kref) ref); struct device *dev = parent->dev; + cleanup_srcu_struct(&parent->unreg_srcu); kfree(parent); put_device(dev); } @@ -147,10 +148,30 @@ static int mdev_device_remove_ops(struct mdev_device *mdev, bool force_remove) return 0; } +static int mdev_device_remove_common(struct mdev_device *mdev, + bool force_remove) +{ + struct mdev_type *type; + int ret; + + type = to_mdev_type(mdev->type_kobj); + + ret = mdev_device_remove_ops(mdev, force_remove); + if (ret && !force_remove) { + mutex_lock(&mdev_list_lock); + mdev->active = true; + mutex_unlock(&mdev_list_lock); + return ret; + } + mdev_remove_sysfs_files(&mdev->dev, type); + device_unregister(&mdev->dev); + return ret; +} + static int mdev_device_remove_cb(struct device *dev, void *data) { if (dev_is_mdev(dev)) - mdev_device_remove(dev, true); + mdev_device_remove_common(to_mdev_device(dev), true); return 0; } @@ -193,6 +214,7 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops) } kref_init(&parent->ref); + init_srcu_struct(&parent->unreg_srcu); parent->dev = dev; parent->ops = ops; @@ -213,6 +235,7 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops) if (ret) dev_warn(dev, "Failed to create compatibility class link\n"); + rcu_assign_pointer(parent->self, parent); list_add(&parent->next, &parent_list); mutex_unlock(&parent_list_lock); @@ -251,13 +274,31 @@ void mdev_unregister_device(struct device *dev) dev_info(dev, "MDEV: Unregistering\n"); list_del(&parent->next); + mutex_unlock(&parent_list_lock); + + /* + * Publish that this mdev parent is unregistering. So any new + * create/remove cannot start on this parent anymore by user. + */ + rcu_assign_pointer(parent->self, NULL); + + /* + * Wait for any active create() or remove() mdev ops on the parent + * to complete. + */ + synchronize_srcu(&parent->unreg_srcu); + + /* + * At this point it is confirmed that any pending user initiated + * create or remove callbacks accessing the parent are completed. + * It is safe to remove the parent now. + */ class_compat_remove_link(mdev_bus_compat_class, dev, NULL); device_for_each_child(dev, NULL, mdev_device_remove_cb); parent_remove_sysfs_files(parent); - mutex_unlock(&parent_list_lock); mdev_put_parent(parent); } EXPORT_SYMBOL(mdev_unregister_device); @@ -278,14 +319,24 @@ int mdev_device_create(struct kobject *kobj, struct device *dev, const guid_t *uuid) { int ret; + struct mdev_parent *valid_parent; struct mdev_device *mdev, *tmp; struct mdev_parent *parent; struct mdev_type *type = to_mdev_type(kobj); + int srcu_idx; parent = mdev_get_parent(type->parent); if (!parent) return -EINVAL; + srcu_idx = srcu_read_lock(&parent->unreg_srcu); + valid_parent = srcu_dereference(parent->self, &parent->unreg_srcu); + if (!valid_parent) { + /* parent is undergoing unregistration */ + ret = -ENODEV; + goto mdev_fail; + } + mutex_lock(&mdev_list_lock); /* Check for duplicate */ @@ -334,44 +385,52 @@ int mdev_device_create(struct kobject *kobj, mdev->type_kobj = kobj; mdev->active = true; dev_dbg(&mdev->dev, "MDEV: created\n"); + srcu_read_unlock(&parent->unreg_srcu, srcu_idx); return 0; create_fail: device_unregister(&mdev->dev); mdev_fail: + srcu_read_unlock(&parent->unreg_srcu, srcu_idx); mdev_put_parent(parent); return ret; } int mdev_device_remove(struct device *dev, bool force_remove) { + struct mdev_parent *valid_parent; struct mdev_device *mdev; struct mdev_parent *parent; - struct mdev_type *type; + int srcu_idx; int ret; mdev = to_mdev_device(dev); + parent = mdev->parent; + + srcu_idx = srcu_read_lock(&parent->unreg_srcu); + valid_parent = srcu_dereference(parent->self, &parent->unreg_srcu); + if (!valid_parent) { + srcu_read_unlock(&parent->unreg_srcu, srcu_idx); + /* parent is undergoing unregistration */ + return -ENODEV; + } + mutex_lock(&mdev_list_lock); if (!mdev->active) { mutex_unlock(&mdev_list_lock); + srcu_read_unlock(&parent->unreg_srcu, srcu_idx); return -EAGAIN; } mdev->active = false; mutex_unlock(&mdev_list_lock); - type = to_mdev_type(mdev->type_kobj); - parent = mdev->parent; - - ret = mdev_device_remove_ops(mdev, force_remove); - if (ret) { - mdev->active = true; + ret = mdev_device_remove_common(mdev, force_remove); + srcu_read_unlock(&parent->unreg_srcu, srcu_idx); + if (ret) return ret; - } - mdev_remove_sysfs_files(dev, type); - device_unregister(dev); mdev_put_parent(parent); return 0; diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h index ddcf9c7..b799978 100644 --- a/drivers/vfio/mdev/mdev_private.h +++ b/drivers/vfio/mdev/mdev_private.h @@ -23,6 +23,12 @@ struct mdev_parent { struct list_head next; struct kset *mdev_types_kset; struct list_head type_list; + /* + * Protects unregistration to wait until create/remove + * are completed. + */ + struct srcu_struct unreg_srcu; + struct mdev_parent __rcu *self; }; struct mdev_device {