From patchwork Mon Nov 18 06:17:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Wang X-Patchwork-Id: 11248943 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 54D151599 for ; Mon, 18 Nov 2019 06:20:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 229852071E for ; Mon, 18 Nov 2019 06:20:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="T6dO4bhU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726884AbfKRGUt (ORCPT ); Mon, 18 Nov 2019 01:20:49 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:27632 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726837AbfKRGUs (ORCPT ); Mon, 18 Nov 2019 01:20:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574058047; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=88z/p49H6YYm2ubEFhmY8J0avTXUGjoccixE43dsikM=; b=T6dO4bhUFS3jzpp99TN5+/wnUjLTSZJXv7hKeju4CzGAS/Hrj4/ZjQNyhrXz8yDo0F+7Xv IJp+7klKEHwM6Pb57b44xbCVSIe3eZj0Zl/PJzpYTeJd3TdMN0CmtyCYVvwf5tuF7LRO87 iAw1vsk4H8IlnPMHNYE8wB8PMxBGfsY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-336-p5-0HTuePVClVzqeAm3HMw-1; Mon, 18 Nov 2019 01:20:03 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A27DC6A6; Mon, 18 Nov 2019 06:19:59 +0000 (UTC) Received: from jason-ThinkPad-X1-Carbon-6th.redhat.com (ovpn-12-215.pek2.redhat.com [10.72.12.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1849760BF4; Mon, 18 Nov 2019 06:19:32 +0000 (UTC) From: Jason Wang To: kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, kwankhede@nvidia.com, alex.williamson@redhat.com, mst@redhat.com, tiwei.bie@intel.com Cc: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, cohuck@redhat.com, maxime.coquelin@redhat.com, cunming.liang@intel.com, zhihong.wang@intel.com, rob.miller@broadcom.com, xiao.w.wang@intel.com, haotian.wang@sifive.com, zhenyuw@linux.intel.com, zhi.a.wang@intel.com, jani.nikula@linux.intel.com, joonas.lahtinen@linux.intel.com, rodrigo.vivi@intel.com, airlied@linux.ie, daniel@ffwll.ch, farman@linux.ibm.com, pasic@linux.ibm.com, sebott@linux.ibm.com, oberpar@linux.ibm.com, heiko.carstens@de.ibm.com, gor@linux.ibm.com, borntraeger@de.ibm.com, akrowiak@linux.ibm.com, freude@linux.ibm.com, lingshan.zhu@intel.com, eperezma@redhat.com, lulu@redhat.com, parav@mellanox.com, christophe.de.dinechin@gmail.com, kevin.tian@intel.com, stefanha@redhat.com, rdunlap@infradead.org, hch@infradead.org, gregkh@linuxfoundation.org, jgg@mellanox.com, Jason Wang Subject: [PATCH V12 4/6] mdev: introduce mediated virtio bus Date: Mon, 18 Nov 2019 14:17:01 +0800 Message-Id: <20191118061703.8669-5-jasowang@redhat.com> In-Reply-To: <20191118061703.8669-1-jasowang@redhat.com> References: <20191118061703.8669-1-jasowang@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: p5-0HTuePVClVzqeAm3HMw-1 X-Mimecast-Spam-Score: 0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This patch implements a mediated virtio bus over mdev framework. This will be used by the future virtio-mdev and vhost-mdev on top to allow driver from either userspace or kernel to control the device which is capable of offloading virtio datapath. Signed-off-by: Jason Wang --- MAINTAINERS | 2 + drivers/mdev/Kconfig | 10 ++ drivers/mdev/Makefile | 2 + drivers/mdev/virtio.c | 126 +++++++++++++++++++++++ include/linux/mdev_virtio.h | 163 ++++++++++++++++++++++++++++++ include/linux/mod_devicetable.h | 8 ++ scripts/mod/devicetable-offsets.c | 3 + scripts/mod/file2alias.c | 12 +++ 8 files changed, 326 insertions(+) create mode 100644 drivers/mdev/virtio.c create mode 100644 include/linux/mdev_virtio.h diff --git a/MAINTAINERS b/MAINTAINERS index 829428d8a9f5..4c2e41641f7a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -17251,6 +17251,8 @@ F: include/linux/virtio*.h F: include/uapi/linux/virtio_*.h F: drivers/crypto/virtio/ F: mm/balloon_compaction.c +F: include/linux/mdev_virtio.h +F: drivers/mdev/virtio.c VIRTIO BLOCK AND SCSI DRIVERS M: "Michael S. Tsirkin" diff --git a/drivers/mdev/Kconfig b/drivers/mdev/Kconfig index 4561f2d4178f..cd84d4670552 100644 --- a/drivers/mdev/Kconfig +++ b/drivers/mdev/Kconfig @@ -17,3 +17,13 @@ config VFIO_MDEV more details. If you don't know what do here, say N. + +config MDEV_VIRTIO + tristate "Mediated VIRTIO bus" + depends on VIRTIO && MDEV + default n + help + Proivdes a mediated BUS for virtio. It could be used by + either kenrel driver or userspace driver. + + If you don't know what do here, say N. diff --git a/drivers/mdev/Makefile b/drivers/mdev/Makefile index 0b749e7f8ff4..eb14031c9944 100644 --- a/drivers/mdev/Makefile +++ b/drivers/mdev/Makefile @@ -1,5 +1,7 @@ mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o mdev_vfio-y := vfio.o +mdev_virtio-y := virtio.o obj-$(CONFIG_MDEV) += mdev.o obj-$(CONFIG_VFIO_MDEV) += mdev_vfio.o +obj-$(CONFIG_MDEV_VIRTIO) += mdev_virtio.o diff --git a/drivers/mdev/virtio.c b/drivers/mdev/virtio.c new file mode 100644 index 000000000000..25de329615c4 --- /dev/null +++ b/drivers/mdev/virtio.c @@ -0,0 +1,126 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Mediated VIRTIO bus + * + * Copyright (c) 2019, Red Hat. All rights reserved. + * Author: Jason Wang + */ + +#include +#include +#include +#include +#include +#include + +#include "mdev_private.h" + +#define DRIVER_VERSION "0.1" +#define DRIVER_AUTHOR "Jason Wang" +#define DRIVER_DESC "Mediated VIRTIO bus" + +struct bus_type mdev_virtio_bus_type; + +struct mdev_virtio_device { + struct mdev_device mdev; + const struct mdev_virtio_ops *ops; + u16 class_id; +}; + +#define to_mdev_virtio(mdev) container_of(mdev, \ + struct mdev_virtio_device, mdev) +#define to_mdev_virtio_drv(mdrv) container_of(mdrv, \ + struct mdev_virtio_driver, drv) + +static int mdev_virtio_match(struct device *dev, struct device_driver *drv) +{ + unsigned int i; + struct mdev_device *mdev = mdev_from_dev(dev, &mdev_virtio_bus_type); + struct mdev_virtio_device *mdev_virtio = to_mdev_virtio(mdev); + struct mdev_driver *mdrv = to_mdev_driver(drv); + struct mdev_virtio_driver *mdrv_virtio = to_mdev_virtio_drv(mdrv); + const struct mdev_virtio_class_id *ids = mdrv_virtio->id_table; + + if (!ids) + return 0; + + for (i = 0; ids[i].id; i++) + if (ids[i].id == mdev_virtio->class_id) + return 1; + return 0; +} + +static int mdev_virtio_uevent(struct device *dev, struct kobj_uevent_env *env) +{ + struct mdev_device *mdev = mdev_from_dev(dev, &mdev_virtio_bus_type); + struct mdev_virtio_device *mdev_virtio = to_mdev_virtio(mdev); + + return add_uevent_var(env, "MODALIAS=mdev_virtio:c%02X", + mdev_virtio->class_id); +} + +struct bus_type mdev_virtio_bus_type = { + .name = "mdev_virtio", + .probe = mdev_probe, + .remove = mdev_remove, + .match = mdev_virtio_match, + .uevent = mdev_virtio_uevent, +}; +EXPORT_SYMBOL(mdev_virtio_bus_type); + +void mdev_virtio_set_class_id(struct mdev_device *mdev, u16 class_id) +{ + struct mdev_virtio_device *mdev_virtio = to_mdev_virtio(mdev); + + mdev_virtio->class_id = class_id; +} +EXPORT_SYMBOL(mdev_virtio_set_class_id); + +int mdev_virtio_register_device(struct device *dev, + const struct mdev_parent_ops *ops) +{ + return mdev_register_device(dev, ops, &mdev_virtio_bus_type, + sizeof(struct mdev_virtio_device)); +} +EXPORT_SYMBOL(mdev_virtio_register_device); + +void mdev_virtio_unregister_device(struct device *dev) +{ + return mdev_unregister_device(dev); +} +EXPORT_SYMBOL(mdev_virtio_unregister_device); + +void mdev_virtio_set_ops(struct mdev_device *mdev, + const struct mdev_virtio_ops *ops) +{ + struct mdev_virtio_device *mdev_virtio = to_mdev_virtio(mdev); + + mdev_virtio->ops = ops; +} +EXPORT_SYMBOL(mdev_virtio_set_ops); + +const struct mdev_virtio_ops *mdev_virtio_get_ops(struct mdev_device *mdev) +{ + struct mdev_virtio_device *mdev_virtio = to_mdev_virtio(mdev); + + return mdev_virtio->ops; +} +EXPORT_SYMBOL(mdev_virtio_get_ops); + +static int __init mdev_init(void) +{ + return mdev_register_bus(&mdev_virtio_bus_type); +} + +static void __exit mdev_exit(void) +{ + mdev_unregister_bus(&mdev_virtio_bus_type); +} + +module_init(mdev_init) +module_exit(mdev_exit) + +MODULE_VERSION(DRIVER_VERSION); +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR(DRIVER_AUTHOR); +MODULE_DESCRIPTION(DRIVER_DESC); diff --git a/include/linux/mdev_virtio.h b/include/linux/mdev_virtio.h new file mode 100644 index 000000000000..ef2dbb6c383a --- /dev/null +++ b/include/linux/mdev_virtio.h @@ -0,0 +1,163 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * VIRTIO Mediated device definition + * + * Copyright (c) 2019, Red Hat. All rights reserved. + * Author: Jason Wang + */ + +#ifndef VIRTIO_MDEV_H +#define VIRTIO_MDEV_H + +#include +#include +#include + +extern struct bus_type mdev_virtio_bus_type; + +struct mdev_virtio_driver { + struct mdev_driver drv; + const struct mdev_virtio_class_id *id_table; +}; + +struct virtio_mdev_callback { + irqreturn_t (*callback)(void *data); + void *private; +}; + +/** + * struct mdev_virtio_device_ops - Structure to be registered for each + * mdev device to register the device for virtio/vhost drivers. + * + * The callbacks are mandatory unless explicitly mentioned. + * + * @set_vq_address: Set the address of virtqueue + * @mdev: mediated device + * @idx: virtqueue index + * @desc_area: address of desc area + * @driver_area: address of driver area + * @device_area: address of device area + * Returns integer: success (0) or error (< 0) + * @set_vq_num: Set the size of virtqueue + * @mdev: mediated device + * @idx: virtqueue index + * @num: the size of virtqueue + * @kick_vq: Kick the virtqueue + * @mdev: mediated device + * @idx: virtqueue index + * @set_vq_cb: Set the interrupt callback function for + * a virtqueue + * @mdev: mediated device + * @idx: virtqueue index + * @cb: virtio-mdev interrupt callback structure + * @set_vq_ready: Set ready status for a virtqueue + * @mdev: mediated device + * @idx: virtqueue index + * @ready: ready (true) not ready(false) + * @get_vq_ready: Get ready status for a virtqueue + * @mdev: mediated device + * @idx: virtqueue index + * Returns boolean: ready (true) or not (false) + * @set_vq_state: Set the state for a virtqueue + * @mdev: mediated device + * @idx: virtqueue index + * @state: virtqueue state (last_avail_idx) + * Returns integer: success (0) or error (< 0) + * @get_vq_state: Get the state for a virtqueue + * @mdev: mediated device + * @idx: virtqueue index + * Returns virtqueue state (last_avail_idx) + * @get_vq_align: Get the virtqueue align requirement + * for the device + * @mdev: mediated device + * Returns virtqueue algin requirement + * @get_features: Get virtio features supported by the device + * @mdev: mediated device + * Returns the virtio features support by the + * device + * @set_features: Set virtio features supported by the driver + * @mdev: mediated device + * @features: feature support by the driver + * Returns integer: success (0) or error (< 0) + * @set_config_cb: Set the config interrupt callback + * @mdev: mediated device + * @cb: virtio-mdev interrupt callback structure + * @get_vq_num_max: Get the max size of virtqueue + * @mdev: mediated device + * Returns u16: max size of virtqueue + * @get_device_id: Get virtio device id + * @mdev: mediated device + * Returns u32: virtio device id + * @get_vendor_id: Get id for the vendor that provides this device + * @mdev: mediated device + * Returns u32: virtio vendor id + * @get_status: Get the device status + * @mdev: mediated device + * Returns u8: virtio device status + * @set_status: Set the device status + * @mdev: mediated device + * @status: virtio device status + * @get_config: Read from device specific configuration space + * @mdev: mediated device + * @offset: offset from the beginning of + * configuration space + * @buf: buffer used to read to + * @len: the length to read from + * configration space + * @set_config: Write to device specific configuration space + * @mdev: mediated device + * @offset: offset from the beginning of + * configuration space + * @buf: buffer used to write from + * @len: the length to write to + * configration space + * @get_generation: Get device config generaton (optional) + * @mdev: mediated device + * Returns u32: device generation + */ +struct mdev_virtio_ops { + /* Virtqueue ops */ + int (*set_vq_address)(struct mdev_device *mdev, + u16 idx, u64 desc_area, u64 driver_area, + u64 device_area); + void (*set_vq_num)(struct mdev_device *mdev, u16 idx, u32 num); + void (*kick_vq)(struct mdev_device *mdev, u16 idx); + void (*set_vq_cb)(struct mdev_device *mdev, u16 idx, + struct virtio_mdev_callback *cb); + void (*set_vq_ready)(struct mdev_device *mdev, u16 idx, bool ready); + bool (*get_vq_ready)(struct mdev_device *mdev, u16 idx); + int (*set_vq_state)(struct mdev_device *mdev, u16 idx, u64 state); + u64 (*get_vq_state)(struct mdev_device *mdev, u16 idx); + + /* Device ops */ + u16 (*get_vq_align)(struct mdev_device *mdev); + u64 (*get_features)(struct mdev_device *mdev); + int (*set_features)(struct mdev_device *mdev, u64 features); + void (*set_config_cb)(struct mdev_device *mdev, + struct virtio_mdev_callback *cb); + u16 (*get_vq_num_max)(struct mdev_device *mdev); + u32 (*get_device_id)(struct mdev_device *mdev); + u32 (*get_vendor_id)(struct mdev_device *mdev); + u8 (*get_status)(struct mdev_device *mdev); + void (*set_status)(struct mdev_device *mdev, u8 status); + void (*get_config)(struct mdev_device *mdev, unsigned int offset, + void *buf, unsigned int len); + void (*set_config)(struct mdev_device *mdev, unsigned int offset, + const void *buf, unsigned int len); + u32 (*get_generation)(struct mdev_device *mdev); +}; + +int mdev_virtio_register_device(struct device *dev, + const struct mdev_parent_ops *ops); +void mdev_virtio_unregister_device(struct device *dev); +void mdev_virtio_set_ops(struct mdev_device *mdev, + const struct mdev_virtio_ops *ops); +const struct mdev_virtio_ops *mdev_virtio_get_ops(struct mdev_device *mdev); +void mdev_virtio_set_class_id(struct mdev_device *mdev, u16 class_id); + +static inline struct mdev_device *mdev_virtio_from_dev(struct device *dev) +{ + return mdev_from_dev(dev, &mdev_virtio_bus_type); +} + +#endif diff --git a/include/linux/mod_devicetable.h b/include/linux/mod_devicetable.h index 5714fd35a83c..59006c47ae8e 100644 --- a/include/linux/mod_devicetable.h +++ b/include/linux/mod_devicetable.h @@ -821,4 +821,12 @@ struct wmi_device_id { const void *context; }; +/** + * struct mdev_class_id - MDEV VIRTIO device class identifier + * @id: Used to identify a specific class of device, e.g vfio-mdev device. + */ +struct mdev_virtio_class_id { + __u16 id; +}; + #endif /* LINUX_MOD_DEVICETABLE_H */ diff --git a/scripts/mod/devicetable-offsets.c b/scripts/mod/devicetable-offsets.c index 054405b90ba4..178fd7c70812 100644 --- a/scripts/mod/devicetable-offsets.c +++ b/scripts/mod/devicetable-offsets.c @@ -231,5 +231,8 @@ int main(void) DEVID(wmi_device_id); DEVID_FIELD(wmi_device_id, guid_string); + DEVID(mdev_virtio_class_id); + DEVID_FIELD(mdev_virtio_class_id, id); + return 0; } diff --git a/scripts/mod/file2alias.c b/scripts/mod/file2alias.c index c91eba751804..1a9c1f591951 100644 --- a/scripts/mod/file2alias.c +++ b/scripts/mod/file2alias.c @@ -1335,6 +1335,17 @@ static int do_wmi_entry(const char *filename, void *symval, char *alias) return 1; } +/* looks like: "mdev_virtio:cN" */ +static int do_mdev_virtio_entry(const char *filename, void *symval, char *alias) +{ + DEF_FIELD(symval, mdev_virtio_class_id, id); + + sprintf(alias, "mdev_virtio:c%02X", id); + add_wildcard(alias); + return 1; +} + + /* Does namelen bytes of name exactly match the symbol? */ static bool sym_is(const char *name, unsigned namelen, const char *symbol) { @@ -1407,6 +1418,7 @@ static const struct devtable devtable[] = { {"typec", SIZE_typec_device_id, do_typec_entry}, {"tee", SIZE_tee_client_device_id, do_tee_entry}, {"wmi", SIZE_wmi_device_id, do_wmi_entry}, + {"mdev_virtio", SIZE_mdev_virtio_class_id, do_mdev_virtio_entry}, }; /* Create MODULE_ALIAS() statements.