diff mbox series

[v2,2/2] drivers: gpio: add virtio-gpio guest driver

Message ID 20201203191135.21576-2-info@metux.net (mailing list archive)
State New, archived
Headers show
Series [v2,1/2] drivers: gpio: put virtual gpio device into their own submenu | expand

Commit Message

Enrico Weigelt, metux IT consult Dec. 3, 2020, 7:11 p.m. UTC
Introducing new gpio driver for virtual GPIO devices via virtio.

The driver allows routing gpio control into VM guests, eg. brigding
virtual gpios to specific host gpios, or attaching simulators for
automatic application testing.

Changes v2:
    * fixed uapi header license
    * sorted include's
    * fixed formatting
    * fixed unneeded devm allocation - plain kzalloc/kfree is enough
    * fixed missing devm_kzalloc fail check
    * use devm_kcalloc() for array allocation
    * added virtio-gpio protocol specification

Signed-off-by: Enrico Weigelt, metux IT consult <info@metux.net>
---
 Documentation/gpio/virtio-gpio.rst | 176 ++++++++++++++++++++
 MAINTAINERS                        |   6 +
 drivers/gpio/Kconfig               |   9 +
 drivers/gpio/Makefile              |   1 +
 drivers/gpio/gpio-virtio.c         | 332 +++++++++++++++++++++++++++++++++++++
 include/uapi/linux/virtio_gpio.h   |  39 +++++
 include/uapi/linux/virtio_ids.h    |   1 +
 7 files changed, 564 insertions(+)
 create mode 100644 Documentation/gpio/virtio-gpio.rst
 create mode 100644 drivers/gpio/gpio-virtio.c
 create mode 100644 include/uapi/linux/virtio_gpio.h

Comments

Jason Wang Dec. 4, 2020, 3:35 a.m. UTC | #1
On 2020/12/4 上午3:11, Enrico Weigelt, metux IT consult wrote:
> Introducing new gpio driver for virtual GPIO devices via virtio.
>
> The driver allows routing gpio control into VM guests, eg. brigding
> virtual gpios to specific host gpios, or attaching simulators for
> automatic application testing.
>
> Changes v2:
>      * fixed uapi header license
>      * sorted include's
>      * fixed formatting
>      * fixed unneeded devm allocation - plain kzalloc/kfree is enough
>      * fixed missing devm_kzalloc fail check
>      * use devm_kcalloc() for array allocation
>      * added virtio-gpio protocol specification
>
> Signed-off-by: Enrico Weigelt, metux IT consult <info@metux.net>
> ---
>   Documentation/gpio/virtio-gpio.rst | 176 ++++++++++++++++++++
>   MAINTAINERS                        |   6 +
>   drivers/gpio/Kconfig               |   9 +
>   drivers/gpio/Makefile              |   1 +
>   drivers/gpio/gpio-virtio.c         | 332 +++++++++++++++++++++++++++++++++++++
>   include/uapi/linux/virtio_gpio.h   |  39 +++++
>   include/uapi/linux/virtio_ids.h    |   1 +
>   7 files changed, 564 insertions(+)
>   create mode 100644 Documentation/gpio/virtio-gpio.rst
>   create mode 100644 drivers/gpio/gpio-virtio.c
>   create mode 100644 include/uapi/linux/virtio_gpio.h
>
> diff --git a/Documentation/gpio/virtio-gpio.rst b/Documentation/gpio/virtio-gpio.rst
> new file mode 100644
> index 000000000000..04642be07b96
> --- /dev/null
> +++ b/Documentation/gpio/virtio-gpio.rst
> @@ -0,0 +1,176 @@
> +"""""""""""""""""
> +Virtio-GPIO protocol specification
> +"""""""""""""""""
> +...........
> +Specification for virtio-based virtiual GPIO devices
> +...........
> +


Is the plan to keep this doc synced with the one in the virtio 
specification?


> ++------------
> ++Version_ 1.0
> ++------------
> +
> +===================
> +General
> +===================
> +
> +The virtio-gpio protocol provides access to general purpose IO devices
> +to virtual machine guests. These virtualized GPIOs could be either provided
> +by some simulator (eg. virtual HIL), routed to some external device or
> +routed to real GPIOs on the host (eg. virtualized embedded applications).
> +
> +Instead of simulating some existing real GPIO chip within an VMM, this
> +protocol provides an hardware independent interface between host and guest
> +that solely relies on an active virtio connection (no matter which transport
> +actually used), no other buses or additional platform driver logic required.
> +
> +===================
> +Protocol layout
> +===================
> +
> +----------------------
> +Configuration space
> +----------------------
> +
> ++--------+----------+-------------------------------+
> +| Offset | Type     | Description                   |
> ++========+==========+===============================+
> +| 0x00   | uint8    | version                       |
> ++--------+----------+-------------------------------+
> +| 0x02   | uint16   | number of GPIO lines          |
> ++--------+----------+-------------------------------+
> +| 0x04   | uint32   | size of gpio name block       |
> ++--------+----------+-------------------------------+
> +| 0x20   | char[32] | device name (0-terminated)    |
> ++--------+----------+-------------------------------+
> +| 0x40   | char[]   | line names block              |
> ++--------+----------+-------------------------------+
> +


I think it's better to use u8 ot uint8_t here.Git grep told me the 
former is more popular under Documentation/.


> +- for version field currently only value 1 supported.
> +- the line names block holds a stream of zero-terminated strings,
> +  holding the individual line names.


I'm not sure but does this mean we don't have a fixed length of config 
space? Need to check whether it can bring any trouble to 
migration(compatibility).


> +- unspecified fields are reserved for future use and should be zero.
> +
> +------------------------
> +Virtqueues and messages:
> +------------------------
> +
> +- Queue #0: transmission from host to guest
> +- Queue #1: transmission from guest to host


Virtio became more a popular in the area without virtualization. So I 
think it's better to use "device/driver" instead of "host/guest" here.


> +
> +The queues transport messages of the struct virtio_gpio_event:
> +
> +Message format:
> +---------------
> +
> ++--------+----------+---------------+
> +| Offset | Type     | Description   |
> ++========+==========+===============+
> +| 0x00   | uint16   | event type    |
> ++--------+----------+---------------+
> +| 0x02   | uint16   | line id       |
> ++--------+----------+---------------+
> +| 0x04   | uint32   | value         |
> ++--------+----------+---------------+


Not a native speaker but event sounds like something driver read from 
device. Looking at the below lists, most of them except for 
VIRTIO_GPIO_EV_HOST_LEVEL looks more like a command.

Another question is, what's the benefit of unifying the message format 
of the two queues. E.g VIRTIO_GPIO_EV_HOST_LEVEL can only works fro rxq.


> +
> +Message types:
> +--------------
> +
> ++-------+---------------------------------------+-----------------------------+
> +| Code  | Symbol                                |                             |
> ++=======+=======================================+=============================+
> +| 0x01  | VIRTIO_GPIO_EV_GUEST_REQUEST          | request gpio line           |
> ++-------+---------------------------------------+-----------------------------+
> +| 0x02  | VIRTIO_GPIO_EV_GUEST_DIRECTION_INPUT  | set direction to input      |
> ++-------+---------------------------------------+-----------------------------+
> +| 0x03  | VIRTIO_GPIO_EV_GUEST_DIRECTION_OUTPUT | set direction to output     |
> ++-------+---------------------------------------+-----------------------------+
> +| 0x04  | VIRTIO_GPIO_EV_GUEST_GET_DIRECTION    | read current direction      |
> ++-------+---------------------------------------+-----------------------------+
> +| 0x05  | VIRTIO_GPIO_EV_GUEST_GET_VALUE        | read current level          |
> ++-------+---------------------------------------+-----------------------------+
> +| 0x06  | VIRTIO_GPIO_EV_GUEST_SET_VALUE        | set current (out) level     |
> ++-------+---------------------------------------+-----------------------------+
> +| 0x11  | VIRTIO_GPIO_EV_HOST_LEVEL             | state changed (host->guest) |
> ++-------+---------------------------------------+-----------------------------+
> +


Not familiar with GPIO but I wonder the value of a standalone 
VIRTIO_GPIO_EV_GUEST_DIRECTION_INPUT/OUTPUT. Can we simply imply them in 
SET/GET_VALUE?


> +----------------------
> +Data flow:
> +----------------------
> +
> +- all operations, except ``VIRTIO_GPIO_EV_HOST_LEVEL``, are guest-initiated
> +- host replies ``VIRTIO_GPIO_EV_HOST_LEVEL`` OR'ed to the ``type`` field
> +- ``VIRTIO_GPIO_EV_HOST_LEVEL`` is only sent asynchronically from host to guest
> +- in replies, a negative ``value`` field denotes an unix-style errno code


Virtio is in a different scope, so we need to define the error code on 
our own.

E.g for virtio-net we define:


#define VIRTIO_NET_OK     0
#define VIRTIO_NET_ERR    1


> +- valid direction values are:
> +  * 0 = output
> +  * 1 = input
> +- valid line state values are:
> +  * 0 = inactive
> +  * 1 = active
> +
> +VIRTIO_GPIO_EV_GUEST_REQUEST
> +----------------------------
> +
> +- notify the host that given line# is going to be used
> +- request:
> +  * ``line`` field: line number
> +  * ``value`` field: unused
> +- reply:
> +  * ``value`` field: errno code (0 = success)
> +
> +VIRTIO_GPIO_EV_GUEST_DIRECTION_INPUT
> +------------------------------------
> +
> +- set line line direction to input
> +- request:
> +  * ``line`` field: line number
> +  * ``value`` field: unused
> +- reply: value field holds errno
> +  * ``value`` field: errno code (0 = success)
> +
> +VIRTIO_GPIO_EV_GUEST_DIRECTION_OUTPUT
> +-------------------------------------
> +
> +- set line direction to output and given line state
> +- request:
> +  * ``line`` field: line number
> +  * ``value`` field: output state (0=inactive, 1=active)
> +- reply:
> +  * ``value`` field: holds errno
> +
> +VIRTIO_GPIO_EV_GUEST_GET_DIRECTION
> +----------------------------------
> +
> +- retrieve line direction
> +- request:
> +  * ``line`` field: line number
> +  * ``value`` field: unused
> +- reply:
> +  * ``value`` field: direction (0=output, 1=input) or errno code
> +
> +VIRTIO_GPIO_EV_GUEST_GET_VALUE
> +------------------------------
> +
> +- retrieve line state value
> +- request:
> +  * ``line`` field: line number
> +  * ``value`` field: unused
> +- reply:
> +  * ``value`` field: line state (0=inactive, 1=active) or errno code
> +
> +VIRTIO_GPIO_EV_GUEST_SET_VALUE
> +------------------------------
> +
> +- set line state value (output only)
> +- request:
> +  * ``line`` field: line number
> +  * ``value`` field: line state (0=inactive, 1=active)
> +- reply:
> +  * ``value`` field: new line state or errno code
> +
> +VIRTIO_GPIO_EV_HOST_LEVEL
> +-------------------------
> +
> +- async notification from host to gues: line state changed
> +- ``line`` field: line number
> +- ``value`` field: new line state (0=inactive, 1=active)
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 2daa6ee673f7..2b74e39275b3 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -18592,6 +18592,12 @@ F:	Documentation/filesystems/virtiofs.rst
>   F:	fs/fuse/virtio_fs.c
>   F:	include/uapi/linux/virtio_fs.h
>   
> +VIRTIO GPIO DRIVER
> +M:	Enrico Weigelt, metux IT consult <info@metux.net>
> +S:	Maintained
> +F:	drivers/gpio/gpio-virtio.c
> +F:	include/uapi/linux/virtio_gpio.h
> +
>   VIRTIO GPU DRIVER
>   M:	David Airlie <airlied@linux.ie>
>   M:	Gerd Hoffmann <kraxel@redhat.com>
> diff --git a/drivers/gpio/Kconfig b/drivers/gpio/Kconfig
> index 01619eb58396..7a33aa347dfb 100644
> --- a/drivers/gpio/Kconfig
> +++ b/drivers/gpio/Kconfig
> @@ -1615,6 +1615,15 @@ config GPIO_MOCKUP
>   	  tools/testing/selftests/gpio/gpio-mockup.sh. Reference the usage in
>   	  it.
>   
> +config GPIO_VIRTIO
> +	tristate "VirtIO GPIO support"
> +	depends on VIRTIO


Let's use select, since there's no prompt for VIRTIO and it doesn't have 
any dependencies.


> +	help
> +	  Say Y here to enable guest support for virtio-based GPIOs.
> +
> +	  These virtual GPIOs can be routed to real GPIOs or attached to
> +	  simulators on the host (qemu).


It's better to avoid talking host and qemu here for new virtio devices.


> +
>   endmenu
>   
>   endif
> diff --git a/drivers/gpio/Makefile b/drivers/gpio/Makefile
> index 09dada80ac34..2b12e75af123 100644
> --- a/drivers/gpio/Makefile
> +++ b/drivers/gpio/Makefile
> @@ -160,6 +160,7 @@ obj-$(CONFIG_GPIO_TWL4030)		+= gpio-twl4030.o
>   obj-$(CONFIG_GPIO_TWL6040)		+= gpio-twl6040.o
>   obj-$(CONFIG_GPIO_UCB1400)		+= gpio-ucb1400.o
>   obj-$(CONFIG_GPIO_UNIPHIER)		+= gpio-uniphier.o
> +obj-$(CONFIG_GPIO_VIRTIO)		+= gpio-virtio.o
>   obj-$(CONFIG_GPIO_VF610)		+= gpio-vf610.o
>   obj-$(CONFIG_GPIO_VIPERBOARD)		+= gpio-viperboard.o
>   obj-$(CONFIG_GPIO_VR41XX)		+= gpio-vr41xx.o
> diff --git a/drivers/gpio/gpio-virtio.c b/drivers/gpio/gpio-virtio.c
> new file mode 100644
> index 000000000000..f1ac47da26b6
> --- /dev/null
> +++ b/drivers/gpio/gpio-virtio.c
> @@ -0,0 +1,332 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +
> +/*
> + * GPIO driver for virtio-based virtual GPIOs
> + *
> + * Copyright (C) 2018 metux IT consult
> + * Author: Enrico Weigelt, metux IT consult <info@metux.net>
> + *
> + */
> +
> +#include <linux/err.h>
> +#include <linux/gpio/driver.h>
> +#include <linux/io.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/spinlock.h>
> +#include <linux/virtio_config.h>
> +#include <uapi/linux/virtio_ids.h>
> +#include <uapi/linux/virtio_gpio.h>
> +
> +struct virtio_gpio_priv {
> +	struct gpio_chip gc;
> +	spinlock_t vq_lock;
> +	spinlock_t op_lock;
> +	struct virtio_device *vdev;
> +	int num_gpios;
> +	char *name;
> +	struct virtqueue *vq_rx;
> +	struct virtqueue *vq_tx;
> +	struct virtio_gpio_event rcv_buf;
> +	struct virtio_gpio_event last;
> +	int irq_base;
> +	wait_queue_head_t waitq;
> +	unsigned long reply_wait;
> +};
> +
> +static void virtio_gpio_prepare_inbuf(struct virtio_gpio_priv *priv)
> +{
> +	struct scatterlist rcv_sg;
> +
> +	sg_init_one(&rcv_sg, &priv->rcv_buf, sizeof(priv->rcv_buf));
> +	virtqueue_add_inbuf(priv->vq_rx, &rcv_sg, 1, &priv->rcv_buf,
> +			    GFP_KERNEL);
> +	virtqueue_kick(priv->vq_rx);
> +}
> +
> +static int virtio_gpio_xmit(struct virtio_gpio_priv *priv, int type,
> +			    int pin, int value, struct virtio_gpio_event *ev)
> +{
> +	struct scatterlist sg[1];
> +	int ret;
> +	unsigned long flags;
> +
> +	WARN_ON(!ev);
> +
> +	ev->type = type;
> +	ev->pin = pin;
> +	ev->value = value;
> +
> +	sg_init_table(sg, 1);
> +	sg_set_buf(&sg[0], ev, sizeof(struct virtio_gpio_event));
> +
> +	spin_lock_irqsave(&priv->vq_lock, flags);
> +	ret = virtqueue_add_outbuf(priv->vq_tx, sg, ARRAY_SIZE(sg),
> +				   priv, GFP_KERNEL);
> +	if (ret < 0) {
> +		dev_err(&priv->vdev->dev,
> +			"virtqueue_add_outbuf() failed: %d\n", ret);
> +		goto out;


So except for the error log, the failure is silently ignored by the 
caller. Is this intended?


> +	}
> +	virtqueue_kick(priv->vq_tx);
> +
> +out:
> +	spin_unlock_irqrestore(&priv->vq_lock, flags);
> +	return 0;
> +}
> +
> +static inline void wakeup_event(struct virtio_gpio_priv *priv, int id)
> +{
> +	set_bit(id, &priv->reply_wait);
> +}
> +
> +static inline int check_event(struct virtio_gpio_priv *priv, int id)
> +{
> +	return test_bit(id, &priv->reply_wait);
> +}
> +
> +static inline void clear_event(struct virtio_gpio_priv *priv, int id)
> +{
> +	clear_bit(id, &priv->reply_wait);
> +}
> +
> +static int virtio_gpio_req(struct virtio_gpio_priv *priv, int type,
> +			   int pin, int value)
> +{
> +	struct virtio_gpio_event *ev
> +		= kzalloc(&priv->vdev->dev, sizeof(struct virtio_gpio_event),
> +			  GFP_KERNEL);
> +
> +	if (!ev)
> +		return -ENOMEM;
> +
> +	clear_event(priv, type);
> +	virtio_gpio_xmit(priv, type, pin, value, ev);
> +	wait_event_interruptible(priv->waitq, check_event(priv, type));


If I read the code correctly, this expects there will be at most a 
single type of event that can be processed at the same time. E.g can 
upper layer want to read from different lines in parallel? If yes, we 
need to deal with that.


> +
> +	kfree(&priv->vdev->dev, ev);
> +
> +	return priv->last.value;
> +}
> +
> +static int virtio_gpio_direction_input(struct gpio_chip *gc,
> +				       unsigned int pin)
> +{
> +	return virtio_gpio_req(gpiochip_get_data(gc),
> +			       VIRTIO_GPIO_EV_GUEST_DIRECTION_INPUT,
> +			       pin, 0);
> +}
> +
> +static int virtio_gpio_direction_output(struct gpio_chip *gc,
> +					unsigned int pin, int value)
> +{
> +	return virtio_gpio_req(gpiochip_get_data(gc),
> +			       VIRTIO_GPIO_EV_GUEST_DIRECTION_OUTPUT,
> +			       pin, value);
> +}
> +
> +static int virtio_gpio_get_direction(struct gpio_chip *gc, unsigned int pin)
> +{
> +	return virtio_gpio_req(gpiochip_get_data(gc),
> +			       VIRTIO_GPIO_EV_GUEST_GET_DIRECTION,
> +			       pin, 0);
> +}
> +
> +static void virtio_gpio_set(struct gpio_chip *gc,
> +			    unsigned int pin, int value)
> +{
> +	virtio_gpio_req(gpiochip_get_data(gc),
> +			VIRTIO_GPIO_EV_GUEST_SET_VALUE, pin, value);
> +}
> +
> +static int virtio_gpio_get(struct gpio_chip *gc,
> +			   unsigned int pin)
> +{
> +	return virtio_gpio_req(gpiochip_get_data(gc),
> +			       VIRTIO_GPIO_EV_GUEST_GET_VALUE, pin, 0);
> +}
> +
> +static int virtio_gpio_request(struct gpio_chip *gc,
> +			       unsigned int pin)
> +{
> +	return virtio_gpio_req(gpiochip_get_data(gc),
> +			       VIRTIO_GPIO_EV_GUEST_REQUEST, pin, 0);
> +}
> +
> +static void virtio_gpio_signal(struct virtio_gpio_priv *priv, int event,
> +			      int pin, int value)
> +{
> +	if (pin < priv->num_gpios)
> +		generic_handle_irq(priv->irq_base + pin);
> +}
> +
> +static void virtio_gpio_data_rx(struct virtqueue *vq)
> +{
> +	struct virtio_gpio_priv *priv = vq->vdev->priv;
> +	void *data;
> +	unsigned int len;
> +	struct virtio_gpio_event *ev;
> +
> +	data = virtqueue_get_buf(priv->vq_rx, &len);
> +	if (!data || !len) {
> +		dev_warn(&vq->vdev->dev, "RX received no data ! %d\n", len);
> +		return;
> +	}
> +
> +	ev = data;
> +	WARN_ON(data != &priv->rcv_buf);
> +
> +	memcpy(&priv->last, &priv->rcv_buf, sizeof(struct virtio_gpio_event));
> +
> +	switch (ev->type) {
> +	case VIRTIO_GPIO_EV_HOST_LEVEL:
> +		virtio_gpio_signal(priv, ev->type, ev->pin, ev->value);
> +		break;
> +	default:
> +		wakeup_event(priv, ev->type & ~VIRTIO_GPIO_EV_REPLY);


This looks suspicious, it looks to me what is done here is, consider we 
want to do VIRTIO_GPIO_EV_GUEST_SET_VALUE

1) put the event in txq, wait
2) the result is returned from rxq, wakeup

It looks to me this is racy since the device should be able to process a 
batch of descriptors and there's no guarantee that the descriptor is 
processed in order from the virtio level.

I wonder why not introduce two virtqueues:

1) command vq
2) event vq

All commands were sent via command vq and then device can write back to 
the command buffer as other virtio device did. Then there's no worries 
of batching or out of order completion.


> +		break;
> +	}
> +	virtio_gpio_prepare_inbuf(priv);


This assumes at most one event could be generated, is this how GPIO 
device expect to behave? I think level could change several times.


> +	wake_up_all(&priv->waitq);
> +}
> +
> +static int virtio_gpio_alloc_vq(struct virtio_gpio_priv *priv)
> +{
> +	struct virtqueue *vqs[2];
> +	vq_callback_t *cbs[] = {
> +		NULL,
> +		virtio_gpio_data_rx,
> +	};
> +	static const char * const names[] = { "in", "out", };
> +	int ret;
> +
> +	ret = virtio_find_vqs(priv->vdev, 2, vqs, cbs, names, NULL);
> +	if (ret) {
> +		dev_err(&priv->vdev->dev, "failed to alloc vqs: %d\n", ret);
> +		return ret;
> +	}
> +
> +	priv->vq_rx = vqs[0];
> +	priv->vq_tx = vqs[1];
> +
> +	virtio_gpio_prepare_inbuf(priv);
> +
> +	virtio_config_enable(priv->vdev);
> +	virtqueue_enable_cb(priv->vq_rx);
> +	virtio_device_ready(priv->vdev);
> +
> +	return 0;
> +}
> +
> +static int virtio_gpio_probe(struct virtio_device *vdev)
> +{
> +	struct virtio_gpio_priv *priv;
> +	struct virtio_gpio_config cf = {};
> +	char *name_buffer;
> +	const char **gpio_names = NULL;
> +	struct device *dev = &vdev->dev;
> +
> +	priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
> +	if (!priv)
> +		return -ENOMEM;


Is devres guaranteed to be enabled here?

Thanks
Enrico Weigelt, metux IT consult Dec. 4, 2020, 9:36 a.m. UTC | #2
On 04.12.20 04:35, Jason Wang wrote:

Hi,

> Is the plan to keep this doc synced with the one in the virtio
> specification?

Yes, of course. I'm still in progress of doing the beaurocratic stuff w/
virtio-tc folks (ID registration, ...) - yet have to see whether they
wanna add it to their spec documents ...

BTW: if you feel, sometings not good w/ the current spec, please raise
your voice now.

> I think it's better to use u8 ot uint8_t here.Git grep told me the
> former is more popular under Documentation/.

thx, I'll fix that

>> +- for version field currently only value 1 supported.
>> +- the line names block holds a stream of zero-terminated strings,
>> +  holding the individual line names.
> 
> I'm not sure but does this mean we don't have a fixed length of config
> space? Need to check whether it can bring any trouble to
> migration(compatibility).

Yes, it depends on how many gpio lines are present and how much space
their names take up.

A fixed size would either put unpleasent limits on the max number of
lines or waste a lot space when only few lines present.

Not that virtio-gpio is also meant for small embedded workloads running
under some hypervisor.

>> +- unspecified fields are reserved for future use and should be zero.
>> +
>> +------------------------
>> +Virtqueues and messages:
>> +------------------------
>> +
>> +- Queue #0: transmission from host to guest
>> +- Queue #1: transmission from guest to host
> 
> 
> Virtio became more a popular in the area without virtualization. So I
> think it's better to use "device/driver" instead of "host/guest" here.

Good point. But I'd prefer "cpu" instead of "driver" in that case.

> Not a native speaker but event sounds like something driver read from
> device. Looking at the below lists, most of them except for
> VIRTIO_GPIO_EV_HOST_LEVEL looks more like a command.

okay, shall I name it "message" ?

> Another question is, what's the benefit of unifying the message format
> of the two queues. E.g VIRTIO_GPIO_EV_HOST_LEVEL can only works fro rxq.

Simplicity. Those fields that aren't really relevant (eg. replies also
carry the line id), can just be ignored.

> Not familiar with GPIO but I wonder the value of a standalone
> VIRTIO_GPIO_EV_GUEST_DIRECTION_INPUT/OUTPUT. Can we simply imply them in
> SET/GET_VALUE?

Would introduce more complexity. Somewhere I'd have to fit in some extra
bit for differenciating between line state and line direction. The
direction tells whether the line currently acts as input or output. The
"value" (hmm, maybe I should rethink terminology here) is the current
line level (high/low or active/inactive).

>> +----------------------
>> +Data flow:
>> +----------------------
>> +
>> +- all operations, except ``VIRTIO_GPIO_EV_HOST_LEVEL``, are
>> guest-initiated
>> +- host replies ``VIRTIO_GPIO_EV_HOST_LEVEL`` OR'ed to the ``type`` field
>> +- ``VIRTIO_GPIO_EV_HOST_LEVEL`` is only sent asynchronically from
>> host to guest
>> +- in replies, a negative ``value`` field denotes an unix-style errno
>> code
> 
> 
> Virtio is in a different scope, so we need to define the error code on
> our own.
> 
> E.g for virtio-net we define:
> 
> 
> #define VIRTIO_NET_OK     0
> #define VIRTIO_NET_ERR    1

hmm, so I'd need to define all the error codes that possibly could happen ?

>>   +config GPIO_VIRTIO
>> +    tristate "VirtIO GPIO support"
>> +    depends on VIRTIO
> 
> 
> Let's use select, since there's no prompt for VIRTIO and it doesn't have
> any dependencies.

Ok. I just was under the impression that subsystems and busses should
not be select'ed, but depends on (eg. some time ago tried that w/ gpio
subsys and failed).

>> +    help
>> +      Say Y here to enable guest support for virtio-based GPIOs.
>> +
>> +      These virtual GPIOs can be routed to real GPIOs or attached to
>> +      simulators on the host (qemu).
> 
> 
> It's better to avoid talking host and qemu here for new virtio devices.

Ok, dropped that line.

>> +static int virtio_gpio_xmit(struct virtio_gpio_priv *priv, int type,
>> +                int pin, int value, struct virtio_gpio_event *ev)
>> +{
>> +    struct scatterlist sg[1];
>> +    int ret;
>> +    unsigned long flags;
>> +
>> +    WARN_ON(!ev);
>> +
>> +    ev->type = type;
>> +    ev->pin = pin;
>> +    ev->value = value;
>> +
>> +    sg_init_table(sg, 1);
>> +    sg_set_buf(&sg[0], ev, sizeof(struct virtio_gpio_event));
>> +
>> +    spin_lock_irqsave(&priv->vq_lock, flags);
>> +    ret = virtqueue_add_outbuf(priv->vq_tx, sg, ARRAY_SIZE(sg),
>> +                   priv, GFP_KERNEL);
>> +    if (ret < 0) {
>> +        dev_err(&priv->vdev->dev,
>> +            "virtqueue_add_outbuf() failed: %d\n", ret);
>> +        goto out;
> 
> 
> So except for the error log, the failure is silently ignored by the
> caller. Is this intended?

ups, I've forgotten the error handling in the caller. fixed in v3.

>> +static int virtio_gpio_req(struct virtio_gpio_priv *priv, int type,
>> +               int pin, int value)
>> +{
>> +    struct virtio_gpio_event *ev
>> +        = kzalloc(&priv->vdev->dev, sizeof(struct virtio_gpio_event),
>> +              GFP_KERNEL);
>> +
>> +    if (!ev)
>> +        return -ENOMEM;
>> +
>> +    clear_event(priv, type);
>> +    virtio_gpio_xmit(priv, type, pin, value, ev);
>> +    wait_event_interruptible(priv->waitq, check_event(priv, type));
> 
> 
> If I read the code correctly, this expects there will be at most a
> single type of event that can be processed at the same time. E.g can
> upper layer want to read from different lines in parallel? If yes, we
> need to deal with that.

@Linus @Bartosz: can that happen or does gpio subsys already serialize
requests ?

Initially, I tried to protect it by spinlock (so, only one request may
run at a time, other calls just wait until the first is finished), but
it crashed when gpio cdev registration calls into the driver (fetches
the status) while still in bootup.

Don't recall the exact error anymore, but something like an
inconsistency in the spinlock calls.

Did I just use the wrong type of lock ?

>> +static void virtio_gpio_data_rx(struct virtqueue *vq)
>> +{
>> +    struct virtio_gpio_priv *priv = vq->vdev->priv;
>> +    void *data;
>> +    unsigned int len;
>> +    struct virtio_gpio_event *ev;
>> +
>> +    data = virtqueue_get_buf(priv->vq_rx, &len);
>> +    if (!data || !len) {
>> +        dev_warn(&vq->vdev->dev, "RX received no data ! %d\n", len);
>> +        return;
>> +    }
>> +
>> +    ev = data;
>> +    WARN_ON(data != &priv->rcv_buf);
>> +
>> +    memcpy(&priv->last, &priv->rcv_buf, sizeof(struct
>> virtio_gpio_event));
>> +
>> +    switch (ev->type) {
>> +    case VIRTIO_GPIO_EV_HOST_LEVEL:
>> +        virtio_gpio_signal(priv, ev->type, ev->pin, ev->value);
>> +        break;
>> +    default:
>> +        wakeup_event(priv, ev->type & ~VIRTIO_GPIO_EV_REPLY);
> 
> 
> This looks suspicious, it looks to me what is done here is, consider we
> want to do VIRTIO_GPIO_EV_GUEST_SET_VALUE
> 
> 1) put the event in txq, wait
> 2) the result is returned from rxq, wakeup
> 
> It looks to me this is racy since the device should be able to process a
> batch of descriptors and there's no guarantee that the descriptor is
> processed in order from the virtio level.

Not sure whether we're on the same page, but:

VIRTIO_GPIO_EV_HOST_LEVEL is kinda interrupt - it tells cpu when the
input has changed level. We can receive this async event, it shouldn't
matter whether somebody else (another thread) is doing a regular call,
thus waiting for reply at the same time. The reply will be next in
queue.

What could go wrong here ?


> I wonder why not introduce two virtqueues:
> 
> 1) command vq
> 2) event vq
> 
> All commands were sent via command vq and then device can write back to
> the command buffer as other virtio device did. Then there's no worries
> of batching or out of order completion.

I've been under the impression that queues only work in only one
direction. (at least that's what my web research was telling).

Could you please give an example how bi-directional transmission within
the same queue could look like ?

>> +        break;
>> +    }
>> +    virtio_gpio_prepare_inbuf(priv);
> 
> 
> This assumes at most one event could be generated, is this how GPIO
> device expect to behave? I think level could change several times.

Should I add more buffers ?

Maybe add one new buffer per request and one new per received async
signal ?

>> +static int virtio_gpio_probe(struct virtio_device *vdev)
>> +{
>> +    struct virtio_gpio_priv *priv;
>> +    struct virtio_gpio_config cf = {};
>> +    char *name_buffer;
>> +    const char **gpio_names = NULL;
>> +    struct device *dev = &vdev->dev;
>> +
>> +    priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
>> +    if (!priv)
>> +        return -ENOMEM;
> 
> 
> Is devres guaranteed to be enabled here?

How should it not ? Could virtio probing so early that even devm
isn't working yet ?


--mtx
Enrico Weigelt, metux IT consult Dec. 5, 2020, 7:59 a.m. UTC | #3
On 04.12.20 04:35, Jason Wang wrote:

>> --- a/drivers/gpio/Kconfig
>> +++ b/drivers/gpio/Kconfig
>> @@ -1615,6 +1615,15 @@ config GPIO_MOCKUP
>>         tools/testing/selftests/gpio/gpio-mockup.sh. Reference the
>> usage in
>>         it.
>>   +config GPIO_VIRTIO
>> +    tristate "VirtIO GPIO support"
>> +    depends on VIRTIO
> 
> 
> Let's use select, since there's no prompt for VIRTIO and it doesn't have
> any dependencies.

whoops, it's not that simple:

make: Entering directory '/home/nekrad/src/apu2-dev/pkg/kernel.apu2.git'
make[1]: Entering directory
'/home/nekrad/src/dk/DistroKit/platform-x86_64/build-target/linux-5.8.9-build'
  GEN     Makefile
drivers/gpu/drm/Kconfig:74:error: recursive dependency detected!
drivers/gpu/drm/Kconfig:74:	symbol DRM_KMS_HELPER is selected by
DRM_VIRTIO_GPU
drivers/gpu/drm/virtio/Kconfig:2:	symbol DRM_VIRTIO_GPU depends on VIRTIO
drivers/virtio/Kconfig:2:	symbol VIRTIO is selected by GPIO_VIRTIO
drivers/gpio/Kconfig:1618:	symbol GPIO_VIRTIO depends on GPIOLIB
drivers/gpio/Kconfig:14:	symbol GPIOLIB is selected by I2C_MUX_LTC4306
drivers/i2c/muxes/Kconfig:47:	symbol I2C_MUX_LTC4306 depends on I2C
drivers/i2c/Kconfig:8:	symbol I2C is selected by FB_DDC
drivers/video/fbdev/Kconfig:63:	symbol FB_DDC depends on FB
drivers/video/fbdev/Kconfig:12:	symbol FB is selected by DRM_KMS_FB_HELPER
drivers/gpu/drm/Kconfig:80:	symbol DRM_KMS_FB_HELPER depends on
DRM_KMS_HELPER

Seems that we can only depend on or select some symbol - we run into
huge trouble if thats mixed. Just changed DRM_VIRTIO_GPU to just select
VIRIO instead of depending on it, and now it works.

I've posted another patch for fixing drivers/gpu/drm/virtio/Kconfig
to use 'select' instead of 'depends on'.
Michael S. Tsirkin Dec. 5, 2020, 7:32 p.m. UTC | #4
On Sat, Dec 05, 2020 at 08:59:55AM +0100, Enrico Weigelt, metux IT consult wrote:
> On 04.12.20 04:35, Jason Wang wrote:
> 
> >> --- a/drivers/gpio/Kconfig
> >> +++ b/drivers/gpio/Kconfig
> >> @@ -1615,6 +1615,15 @@ config GPIO_MOCKUP
> >>         tools/testing/selftests/gpio/gpio-mockup.sh. Reference the
> >> usage in
> >>         it.
> >>   +config GPIO_VIRTIO
> >> +    tristate "VirtIO GPIO support"
> >> +    depends on VIRTIO
> > 
> > 
> > Let's use select, since there's no prompt for VIRTIO and it doesn't have
> > any dependencies.
> 
> whoops, it's not that simple:
> 
> make: Entering directory '/home/nekrad/src/apu2-dev/pkg/kernel.apu2.git'
> make[1]: Entering directory
> '/home/nekrad/src/dk/DistroKit/platform-x86_64/build-target/linux-5.8.9-build'
>   GEN     Makefile
> drivers/gpu/drm/Kconfig:74:error: recursive dependency detected!
> drivers/gpu/drm/Kconfig:74:	symbol DRM_KMS_HELPER is selected by
> DRM_VIRTIO_GPU
> drivers/gpu/drm/virtio/Kconfig:2:	symbol DRM_VIRTIO_GPU depends on VIRTIO
> drivers/virtio/Kconfig:2:	symbol VIRTIO is selected by GPIO_VIRTIO
> drivers/gpio/Kconfig:1618:	symbol GPIO_VIRTIO depends on GPIOLIB
> drivers/gpio/Kconfig:14:	symbol GPIOLIB is selected by I2C_MUX_LTC4306
> drivers/i2c/muxes/Kconfig:47:	symbol I2C_MUX_LTC4306 depends on I2C
> drivers/i2c/Kconfig:8:	symbol I2C is selected by FB_DDC
> drivers/video/fbdev/Kconfig:63:	symbol FB_DDC depends on FB
> drivers/video/fbdev/Kconfig:12:	symbol FB is selected by DRM_KMS_FB_HELPER
> drivers/gpu/drm/Kconfig:80:	symbol DRM_KMS_FB_HELPER depends on
> DRM_KMS_HELPER
> 
> Seems that we can only depend on or select some symbol - we run into
> huge trouble if thats mixed. Just changed DRM_VIRTIO_GPU to just select
> VIRIO instead of depending on it, and now it works.
> 
> I've posted another patch for fixing drivers/gpu/drm/virtio/Kconfig
> to use 'select' instead of 'depends on'.

It seems a bit of a mess, at this point I'm not entirely sure when
should drivers select VIRTIO and when depend on it.

The text near it says:

# SPDX-License-Identifier: GPL-2.0-only
config VIRTIO
        tristate
        help
          This option is selected by any driver which implements the virtio
          bus, such as CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_MMIO, CONFIG_RPMSG
          or CONFIG_S390_GUEST.

Which seems clear enough and would indicate drivers for devices *behind*
the bus should not select VIRTIO and thus presumably should "depend on" it.
This is violated in virtio console and virtio fs drivers.

For console it says:

commit 9f30eb29c514589e16f2999ea070598583d1f6ec
Author: Michal Suchanek <msuchanek@suse.de>
Date:   Mon Aug 31 18:58:50 2020 +0200

    char: virtio: Select VIRTIO from VIRTIO_CONSOLE.
    
    Make it possible to have virtio console built-in when
    other virtio drivers are modular.
    
    Signed-off-by: Michal Suchanek <msuchanek@suse.de>
    Reviewed-by: Amit Shah <amit@kernel.org>
    Link: https://lore.kernel.org/r/20200831165850.26163-1-msuchanek@suse.de
    Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

which seems kind of bogus - why do we care about allowing a builtin
virtio console driver if the pci virtio bus driver is a module?
There won't be any devices on the bus to attach to ...

And for virtio fs it was like this from the beginning.

I am inclined to fix console and virtio fs to depend on VIRTIO:
select is harder to use correctly ...

Jason?


> -- 
> ---
> Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert
> werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren
> GPG/PGP-Schlüssel zu.
> ---
> Enrico Weigelt, metux IT consult
> Free software and Linux embedded engineering
> info@metux.net -- +49-151-27565287
Enrico Weigelt, metux IT consult Dec. 5, 2020, 8:05 p.m. UTC | #5
On 05.12.20 20:32, Michael S. Tsirkin wrote:

Hi,

> It seems a bit of a mess, at this point I'm not entirely sure when
> should drivers select VIRTIO and when depend on it.

if VIRTIO just enables something that could be seen as library
functions, then select should be right, IMHO.

> The text near it says:
> 
> # SPDX-License-Identifier: GPL-2.0-only
> config VIRTIO
>         tristate

oh, wait, doesn't have an menu text, so we can't even explicitly enable
it (not shown in menu) - only implicitly. Which means that some other
option must select it, in order to become availe at all, and in order
to make others depending on it becoming available.

IMHO, therefore select is the correct approach.


>         help
>           This option is selected by any driver which implements the virtio
>           bus, such as CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_MMIO, CONFIG_RPMSG
>           or CONFIG_S390_GUEST.
> 
> Which seems clear enough and would indicate drivers for devices *behind*
> the bus should not select VIRTIO and thus presumably should "depend on" it.
> This is violated in virtio console and virtio fs drivers.

See above: NAK. because it can't even be enabled directly (by the user).
If it wasn't meant otherwise, we'd have to add an menu text.

> For console it says:
> 
> commit 9f30eb29c514589e16f2999ea070598583d1f6ec
> Author: Michal Suchanek <msuchanek@suse.de>
> Date:   Mon Aug 31 18:58:50 2020 +0200
> 
>     char: virtio: Select VIRTIO from VIRTIO_CONSOLE.
>     
>     Make it possible to have virtio console built-in when
>     other virtio drivers are modular.
>     
>     Signed-off-by: Michal Suchanek <msuchanek@suse.de>
>     Reviewed-by: Amit Shah <amit@kernel.org>
>     Link: https://lore.kernel.org/r/20200831165850.26163-1-msuchanek@suse.de
>     Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> 
> which seems kind of bogus - why do we care about allowing a builtin
> virtio console driver if the pci virtio bus driver is a module?
> There won't be any devices on the bus to attach to ...

When using other transports ?
In my current project, eg. I'm using mmio - my kernel has pci completely
disabled.

> I am inclined to fix console and virtio fs to depend on VIRTIO:
> select is harder to use correctly ...

I don't thinkt that would be good - instead everybody should just select
VIRTIO, never depend on it (maybe depend on VIRTIO_MENU instead)


--mtx
Enrico Weigelt, metux IT consult Dec. 5, 2020, 8:15 p.m. UTC | #6
On 03.12.20 20:11, Enrico Weigelt, metux IT consult wrote:

Friends,

I've still got a problem w/ signal/irq handling:

The virtio-gpio device/host can raise a signal on line state change.
Kinda IRQ, but not actually running through real IRQs, instead by a
message running though queue. (hmm, kida MSI ? :o).

I've tried allocating an IRQ range and calling generic_handle_irq(),
but then I'm getting unhanled IRQ trap.

My hope was some gpio lib function for calling in when an line state
changes, that does all the magic (somebody listening on some gpio,
or gpio used as interrupt source), but the only thing I could find
was some helpers for gpio chips that have their own builtin
interrupt controller (VIRTIO_GPIO_EV_HOST_LEVEL).

Somehow feels that's not quite what I'm looking for.

Could anybody please give me more insights ?


--mtx
Jason Wang Dec. 7, 2020, 3:12 a.m. UTC | #7
On 2020/12/6 上午3:32, Michael S. Tsirkin wrote:
> On Sat, Dec 05, 2020 at 08:59:55AM +0100, Enrico Weigelt, metux IT consult wrote:
>> On 04.12.20 04:35, Jason Wang wrote:
>>
>>>> --- a/drivers/gpio/Kconfig
>>>> +++ b/drivers/gpio/Kconfig
>>>> @@ -1615,6 +1615,15 @@ config GPIO_MOCKUP
>>>>         tools/testing/selftests/gpio/gpio-mockup.sh. Reference the
>>>> usage in
>>>>         it.
>>>>   +config GPIO_VIRTIO
>>>> +    tristate "VirtIO GPIO support"
>>>> +    depends on VIRTIO
>>>
>>> Let's use select, since there's no prompt for VIRTIO and it doesn't have
>>> any dependencies.
>> whoops, it's not that simple:
>>
>> make: Entering directory '/home/nekrad/src/apu2-dev/pkg/kernel.apu2.git'
>> make[1]: Entering directory
>> '/home/nekrad/src/dk/DistroKit/platform-x86_64/build-target/linux-5.8.9-build'
>>    GEN     Makefile
>> drivers/gpu/drm/Kconfig:74:error: recursive dependency detected!
>> drivers/gpu/drm/Kconfig:74:	symbol DRM_KMS_HELPER is selected by
>> DRM_VIRTIO_GPU
>> drivers/gpu/drm/virtio/Kconfig:2:	symbol DRM_VIRTIO_GPU depends on VIRTIO
>> drivers/virtio/Kconfig:2:	symbol VIRTIO is selected by GPIO_VIRTIO
>> drivers/gpio/Kconfig:1618:	symbol GPIO_VIRTIO depends on GPIOLIB
>> drivers/gpio/Kconfig:14:	symbol GPIOLIB is selected by I2C_MUX_LTC4306
>> drivers/i2c/muxes/Kconfig:47:	symbol I2C_MUX_LTC4306 depends on I2C
>> drivers/i2c/Kconfig:8:	symbol I2C is selected by FB_DDC
>> drivers/video/fbdev/Kconfig:63:	symbol FB_DDC depends on FB
>> drivers/video/fbdev/Kconfig:12:	symbol FB is selected by DRM_KMS_FB_HELPER
>> drivers/gpu/drm/Kconfig:80:	symbol DRM_KMS_FB_HELPER depends on
>> DRM_KMS_HELPER
>>
>> Seems that we can only depend on or select some symbol - we run into
>> huge trouble if thats mixed. Just changed DRM_VIRTIO_GPU to just select
>> VIRIO instead of depending on it, and now it works.
>>
>> I've posted another patch for fixing drivers/gpu/drm/virtio/Kconfig
>> to use 'select' instead of 'depends on'.
> It seems a bit of a mess, at this point I'm not entirely sure when
> should drivers select VIRTIO and when depend on it.
>
> The text near it says:
>
> # SPDX-License-Identifier: GPL-2.0-only
> config VIRTIO
>          tristate
>          help
>            This option is selected by any driver which implements the virtio
>            bus, such as CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_MMIO, CONFIG_RPMSG
>            or CONFIG_S390_GUEST.
>
> Which seems clear enough and would indicate drivers for devices *behind*
> the bus should not select VIRTIO and thus presumably should "depend on" it.
> This is violated in virtio console and virtio fs drivers.
>
> For console it says:
>
> commit 9f30eb29c514589e16f2999ea070598583d1f6ec
> Author: Michal Suchanek <msuchanek@suse.de>
> Date:   Mon Aug 31 18:58:50 2020 +0200
>
>      char: virtio: Select VIRTIO from VIRTIO_CONSOLE.
>      
>      Make it possible to have virtio console built-in when
>      other virtio drivers are modular.
>      
>      Signed-off-by: Michal Suchanek <msuchanek@suse.de>
>      Reviewed-by: Amit Shah <amit@kernel.org>
>      Link: https://lore.kernel.org/r/20200831165850.26163-1-msuchanek@suse.de
>      Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>
> which seems kind of bogus - why do we care about allowing a builtin
> virtio console driver if the pci virtio bus driver is a module?
> There won't be any devices on the bus to attach to ...


For testing like switching bus from pci to MMIO?


> And for virtio fs it was like this from the beginning.
>
> I am inclined to fix console and virtio fs to depend on VIRTIO:
> select is harder to use correctly ...
>
> Jason?


I think it works, but we need a prompt for VIRTIO otherwise there's no 
way to enable it.

Thanks


>
>
>> -- 
>> ---
>> Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert
>> werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren
>> GPG/PGP-Schlüssel zu.
>> ---
>> Enrico Weigelt, metux IT consult
>> Free software and Linux embedded engineering
>> info@metux.net -- +49-151-27565287
Jason Wang Dec. 7, 2020, 3:16 a.m. UTC | #8
On 2020/12/6 上午4:05, Enrico Weigelt, metux IT consult wrote:
> On 05.12.20 20:32, Michael S. Tsirkin wrote:
>
> Hi,
>
>> It seems a bit of a mess, at this point I'm not entirely sure when
>> should drivers select VIRTIO and when depend on it.
> if VIRTIO just enables something that could be seen as library
> functions, then select should be right, IMHO.
>
>> The text near it says:
>>
>> # SPDX-License-Identifier: GPL-2.0-only
>> config VIRTIO
>>          tristate
> oh, wait, doesn't have an menu text, so we can't even explicitly enable
> it (not shown in menu) - only implicitly. Which means that some other
> option must select it, in order to become availe at all, and in order
> to make others depending on it becoming available.
>
> IMHO, therefore select is the correct approach.
>
>
>>          help
>>            This option is selected by any driver which implements the virtio
>>            bus, such as CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_MMIO, CONFIG_RPMSG
>>            or CONFIG_S390_GUEST.
>>
>> Which seems clear enough and would indicate drivers for devices *behind*
>> the bus should not select VIRTIO and thus presumably should "depend on" it.
>> This is violated in virtio console and virtio fs drivers.
> See above: NAK. because it can't even be enabled directly (by the user).
> If it wasn't meant otherwise, we'd have to add an menu text.
>
>> For console it says:
>>
>> commit 9f30eb29c514589e16f2999ea070598583d1f6ec
>> Author: Michal Suchanek <msuchanek@suse.de>
>> Date:   Mon Aug 31 18:58:50 2020 +0200
>>
>>      char: virtio: Select VIRTIO from VIRTIO_CONSOLE.
>>      
>>      Make it possible to have virtio console built-in when
>>      other virtio drivers are modular.
>>      
>>      Signed-off-by: Michal Suchanek <msuchanek@suse.de>
>>      Reviewed-by: Amit Shah <amit@kernel.org>
>>      Link: https://lore.kernel.org/r/20200831165850.26163-1-msuchanek@suse.de
>>      Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>>
>> which seems kind of bogus - why do we care about allowing a builtin
>> virtio console driver if the pci virtio bus driver is a module?
>> There won't be any devices on the bus to attach to ...
> When using other transports ?
> In my current project, eg. I'm using mmio - my kernel has pci completely
> disabled.
>
>> I am inclined to fix console and virtio fs to depend on VIRTIO:
>> select is harder to use correctly ...
> I don't thinkt that would be good - instead everybody should just select
> VIRTIO, never depend on it (maybe depend on VIRTIO_MENU instead)


I'm fine with either. Though I prefer to use select but it looks to me 
adding a prompt and use enable would be easier.

Thanks


>
>
> --mtx
>
Jason Wang Dec. 7, 2020, 3:48 a.m. UTC | #9
On 2020/12/4 下午5:36, Enrico Weigelt, metux IT consult wrote:
> On 04.12.20 04:35, Jason Wang wrote:
>
> Hi,
>
>> Is the plan to keep this doc synced with the one in the virtio
>> specification?
> Yes, of course. I'm still in progress of doing the beaurocratic stuff w/
> virtio-tc folks (ID registration, ...) - yet have to see whether they
> wanna add it to their spec documents ...
>
> BTW: if you feel, sometings not good w/ the current spec, please raise
> your voice now.


But, has the spec path posted?


>
>> I think it's better to use u8 ot uint8_t here.Git grep told me the
>> former is more popular under Documentation/.
> thx, I'll fix that
>
>>> +- for version field currently only value 1 supported.
>>> +- the line names block holds a stream of zero-terminated strings,
>>> +  holding the individual line names.
>> I'm not sure but does this mean we don't have a fixed length of config
>> space? Need to check whether it can bring any trouble to
>> migration(compatibility).
> Yes, it depends on how many gpio lines are present and how much space
> their names take up.
>
> A fixed size would either put unpleasent limits on the max number of
> lines or waste a lot space when only few lines present.
>
> Not that virtio-gpio is also meant for small embedded workloads running
> under some hypervisor.
>
>>> +- unspecified fields are reserved for future use and should be zero.
>>> +
>>> +------------------------
>>> +Virtqueues and messages:
>>> +------------------------
>>> +
>>> +- Queue #0: transmission from host to guest
>>> +- Queue #1: transmission from guest to host
>>
>> Virtio became more a popular in the area without virtualization. So I
>> think it's better to use "device/driver" instead of "host/guest" here.
> Good point. But I'd prefer "cpu" instead of "driver" in that case.
>
>> Not a native speaker but event sounds like something driver read from
>> device. Looking at the below lists, most of them except for
>> VIRTIO_GPIO_EV_HOST_LEVEL looks more like a command.
> okay, shall I name it "message" ?


It might be better.


>
>> Another question is, what's the benefit of unifying the message format
>> of the two queues. E.g VIRTIO_GPIO_EV_HOST_LEVEL can only works fro rxq.
> Simplicity. Those fields that aren't really relevant (eg. replies also
> carry the line id), can just be ignored.
>
>> Not familiar with GPIO but I wonder the value of a standalone
>> VIRTIO_GPIO_EV_GUEST_DIRECTION_INPUT/OUTPUT. Can we simply imply them in
>> SET/GET_VALUE?
> Would introduce more complexity. Somewhere I'd have to fit in some extra
> bit for differenciating between line state and line direction. The
> direction tells whether the line currently acts as input or output. The
> "value" (hmm, maybe I should rethink terminology here) is the current
> line level (high/low or active/inactive).


Ok.


>
>>> +----------------------
>>> +Data flow:
>>> +----------------------
>>> +
>>> +- all operations, except ``VIRTIO_GPIO_EV_HOST_LEVEL``, are
>>> guest-initiated
>>> +- host replies ``VIRTIO_GPIO_EV_HOST_LEVEL`` OR'ed to the ``type`` field
>>> +- ``VIRTIO_GPIO_EV_HOST_LEVEL`` is only sent asynchronically from
>>> host to guest
>>> +- in replies, a negative ``value`` field denotes an unix-style errno
>>> code
>>
>> Virtio is in a different scope, so we need to define the error code on
>> our own.
>>
>> E.g for virtio-net we define:
>>
>>
>> #define VIRTIO_NET_OK     0
>> #define VIRTIO_NET_ERR    1
> hmm, so I'd need to define all the error codes that possibly could happen ?


Yes, I think you need.


>
>>>    +config GPIO_VIRTIO
>>> +    tristate "VirtIO GPIO support"
>>> +    depends on VIRTIO
>>
>> Let's use select, since there's no prompt for VIRTIO and it doesn't have
>> any dependencies.
> Ok. I just was under the impression that subsystems and busses should
> not be select'ed, but depends on (eg. some time ago tried that w/ gpio
> subsys and failed).
>
>>> +    help
>>> +      Say Y here to enable guest support for virtio-based GPIOs.
>>> +
>>> +      These virtual GPIOs can be routed to real GPIOs or attached to
>>> +      simulators on the host (qemu).
>>
>> It's better to avoid talking host and qemu here for new virtio devices.
> Ok, dropped that line.
>
>>> +static int virtio_gpio_xmit(struct virtio_gpio_priv *priv, int type,
>>> +                int pin, int value, struct virtio_gpio_event *ev)
>>> +{
>>> +    struct scatterlist sg[1];
>>> +    int ret;
>>> +    unsigned long flags;
>>> +
>>> +    WARN_ON(!ev);
>>> +
>>> +    ev->type = type;
>>> +    ev->pin = pin;
>>> +    ev->value = value;
>>> +
>>> +    sg_init_table(sg, 1);
>>> +    sg_set_buf(&sg[0], ev, sizeof(struct virtio_gpio_event));
>>> +
>>> +    spin_lock_irqsave(&priv->vq_lock, flags);
>>> +    ret = virtqueue_add_outbuf(priv->vq_tx, sg, ARRAY_SIZE(sg),
>>> +                   priv, GFP_KERNEL);
>>> +    if (ret < 0) {
>>> +        dev_err(&priv->vdev->dev,
>>> +            "virtqueue_add_outbuf() failed: %d\n", ret);
>>> +        goto out;
>>
>> So except for the error log, the failure is silently ignored by the
>> caller. Is this intended?
> ups, I've forgotten the error handling in the caller. fixed in v3.
>
>>> +static int virtio_gpio_req(struct virtio_gpio_priv *priv, int type,
>>> +               int pin, int value)
>>> +{
>>> +    struct virtio_gpio_event *ev
>>> +        = kzalloc(&priv->vdev->dev, sizeof(struct virtio_gpio_event),
>>> +              GFP_KERNEL);
>>> +
>>> +    if (!ev)
>>> +        return -ENOMEM;
>>> +
>>> +    clear_event(priv, type);
>>> +    virtio_gpio_xmit(priv, type, pin, value, ev);
>>> +    wait_event_interruptible(priv->waitq, check_event(priv, type));
>>
>> If I read the code correctly, this expects there will be at most a
>> single type of event that can be processed at the same time. E.g can
>> upper layer want to read from different lines in parallel? If yes, we
>> need to deal with that.
> @Linus @Bartosz: can that happen or does gpio subsys already serialize
> requests ?
>
> Initially, I tried to protect it by spinlock (so, only one request may
> run at a time, other calls just wait until the first is finished), but
> it crashed when gpio cdev registration calls into the driver (fetches
> the status) while still in bootup.
>
> Don't recall the exact error anymore, but something like an
> inconsistency in the spinlock calls.
>
> Did I just use the wrong type of lock ?


I'm not sure since I am not familiar with GPIO. But a question is, if at 
most one request is allowed, I'm not sure virtio is the best choice here 
since we don't even need a queue(virtqueue) here.


>
>>> +static void virtio_gpio_data_rx(struct virtqueue *vq)
>>> +{
>>> +    struct virtio_gpio_priv *priv = vq->vdev->priv;
>>> +    void *data;
>>> +    unsigned int len;
>>> +    struct virtio_gpio_event *ev;
>>> +
>>> +    data = virtqueue_get_buf(priv->vq_rx, &len);
>>> +    if (!data || !len) {
>>> +        dev_warn(&vq->vdev->dev, "RX received no data ! %d\n", len);
>>> +        return;
>>> +    }
>>> +
>>> +    ev = data;
>>> +    WARN_ON(data != &priv->rcv_buf);
>>> +
>>> +    memcpy(&priv->last, &priv->rcv_buf, sizeof(struct
>>> virtio_gpio_event));
>>> +
>>> +    switch (ev->type) {
>>> +    case VIRTIO_GPIO_EV_HOST_LEVEL:
>>> +        virtio_gpio_signal(priv, ev->type, ev->pin, ev->value);
>>> +        break;
>>> +    default:
>>> +        wakeup_event(priv, ev->type & ~VIRTIO_GPIO_EV_REPLY);
>>
>> This looks suspicious, it looks to me what is done here is, consider we
>> want to do VIRTIO_GPIO_EV_GUEST_SET_VALUE
>>
>> 1) put the event in txq, wait
>> 2) the result is returned from rxq, wakeup
>>
>> It looks to me this is racy since the device should be able to process a
>> batch of descriptors and there's no guarantee that the descriptor is
>> processed in order from the virtio level.
> Not sure whether we're on the same page, but:
>
> VIRTIO_GPIO_EV_HOST_LEVEL is kinda interrupt - it tells cpu when the
> input has changed level. We can receive this async event, it shouldn't
> matter whether somebody else (another thread) is doing a regular call,
> thus waiting for reply at the same time. The reply will be next in
> queue.
>
> What could go wrong here ?


I think it's still about whether or not we need allow a batch of 
requests via a queue. Consider you've submitted two request A and B, and 
if B is done first, current code won't work. This is because, the reply 
is transported via rxq buffers not just reuse the txq buffer if I read 
the code correctly.


>
>
>> I wonder why not introduce two virtqueues:
>>
>> 1) command vq
>> 2) event vq
>>
>> All commands were sent via command vq and then device can write back to
>> the command buffer as other virtio device did. Then there's no worries
>> of batching or out of order completion.
> I've been under the impression that queues only work in only one
> direction. (at least that's what my web research was telling).
>
> Could you please give an example how bi-directional transmission within
> the same queue could look like ?


You can check how virtio-blk did this in:

https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-2500006


>
>>> +        break;
>>> +    }
>>> +    virtio_gpio_prepare_inbuf(priv);
>>
>> This assumes at most one event could be generated, is this how GPIO
>> device expect to behave? I think level could change several times.
> Should I add more buffers ?
>
> Maybe add one new buffer per request and one new per received async
> signal ?


It would be safe to fill the whole rxq and do the refill e.g when half 
of the queue is used.


>
>>> +static int virtio_gpio_probe(struct virtio_device *vdev)
>>> +{
>>> +    struct virtio_gpio_priv *priv;
>>> +    struct virtio_gpio_config cf = {};
>>> +    char *name_buffer;
>>> +    const char **gpio_names = NULL;
>>> +    struct device *dev = &vdev->dev;
>>> +
>>> +    priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
>>> +    if (!priv)
>>> +        return -ENOMEM;
>>
>> Is devres guaranteed to be enabled here?
> How should it not ? Could virtio probing so early that even devm
> isn't working yet ?


I think you are right, I misread the patch.

Thanks


>
>
> --mtx
>
Enrico Weigelt, metux IT consult Dec. 7, 2020, 9:33 a.m. UTC | #10
On 07.12.20 04:48, Jason Wang wrote:

Hi,

>>> Not a native speaker but event sounds like something driver read from
>>> device. Looking at the below lists, most of them except for
>>> VIRTIO_GPIO_EV_HOST_LEVEL looks more like a command.
>> okay, shall I name it "message" ?
> 
> 
> It might be better.

Okay, renamed to messages in v3.

>>> #define VIRTIO_NET_OK     0
>>> #define VIRTIO_NET_ERR    1
>> hmm, so I'd need to define all the error codes that possibly could
>> happen ?
> 
> 
> Yes, I think you need.

Okay, going to do it in the next version.

>>> If I read the code correctly, this expects there will be at most a
>>> single type of event that can be processed at the same time. E.g can
>>> upper layer want to read from different lines in parallel? If yes, we
>>> need to deal with that.
>> @Linus @Bartosz: can that happen or does gpio subsys already serialize
>> requests ?
>>
>> Initially, I tried to protect it by spinlock (so, only one request may
>> run at a time, other calls just wait until the first is finished), but
>> it crashed when gpio cdev registration calls into the driver (fetches
>> the status) while still in bootup.
>>
>> Don't recall the exact error anymore, but something like an
>> inconsistency in the spinlock calls.
>>
>> Did I just use the wrong type of lock ?
> 
> I'm not sure since I am not familiar with GPIO. But a question is, if at
> most one request is allowed, I'm not sure virtio is the best choice here
> since we don't even need a queue(virtqueue) here.

I guess, I should add locks to the gpio callback functions (where gpio
subsys calls in). That way, requests are requests are strictly ordered.
The locks didn't work in my previous attempts, but probably because I've
missed to set the can_sleep flag (now fixed in v3).

The gpio ops are already waiting for reply of the corresponding type, so
the only bad thing that could happen is the same operation being called
twice (when coming from different threads) and replies mixed up between
first and second one. OTOH I don't see much problem w/ that. This can be
fixed by adding a global lock.

> I think it's still about whether or not we need allow a batch of
> requests via a queue. Consider you've submitted two request A and B, and
> if B is done first, current code won't work. This is because, the reply
> is transported via rxq buffers not just reuse the txq buffer if I read
> the code correctly.

Meanwhile I've changed it to allocate a new rx buffer for the reply
(done right before the request is sent), so everything should be
processed in the order it had been sent. Assuming virtio keeps the
order of the buffers in the queues.

>> Could you please give an example how bi-directional transmission within
>> the same queue could look like ?
> 
> You can check how virtio-blk did this in:
> 
> https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-2500006

hmm, still don't see how the code would actually look like. (in qemu as
well as kernel). Just add the fetched inbuf as an outbuf (within the
same queue) ?

>> Maybe add one new buffer per request and one new per received async
>> signal ?
> 
> It would be safe to fill the whole rxq and do the refill e.g when half
> of the queue is used.

Okay, doing that now in v3: there's always at least one rx buffer,
and requests as well as the intr receiver always add a new one.
(they get removed on fetching, IMHO).


--mtx
Michael S. Tsirkin Dec. 7, 2020, 1:52 p.m. UTC | #11
On Sat, Dec 05, 2020 at 09:05:16PM +0100, Enrico Weigelt, metux IT consult wrote:
> On 05.12.20 20:32, Michael S. Tsirkin wrote:
> 
> Hi,
> 
> > It seems a bit of a mess, at this point I'm not entirely sure when
> > should drivers select VIRTIO and when depend on it.
> 
> if VIRTIO just enables something that could be seen as library
> functions, then select should be right, IMHO.
> 
> > The text near it says:
> > 
> > # SPDX-License-Identifier: GPL-2.0-only
> > config VIRTIO
> >         tristate
> 
> oh, wait, doesn't have an menu text, so we can't even explicitly enable
> it (not shown in menu) - only implicitly. Which means that some other
> option must select it, in order to become availe at all, and in order
> to make others depending on it becoming available.
> 
> IMHO, therefore select is the correct approach.
> 
> 
> >         help
> >           This option is selected by any driver which implements the virtio
> >           bus, such as CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_MMIO, CONFIG_RPMSG
> >           or CONFIG_S390_GUEST.
> > 
> > Which seems clear enough and would indicate drivers for devices *behind*
> > the bus should not select VIRTIO and thus presumably should "depend on" it.
> > This is violated in virtio console and virtio fs drivers.
> 
> See above: NAK. because it can't even be enabled directly (by the user).
> If it wasn't meant otherwise, we'd have to add an menu text.


The point is that user enables one of the bindings.
That in turn enables drivers. If we merely select VIRTIO
there's a chance user won't remember to select any bindings
and will be surprised not to see any devices.



> > For console it says:
> > 
> > commit 9f30eb29c514589e16f2999ea070598583d1f6ec
> > Author: Michal Suchanek <msuchanek@suse.de>
> > Date:   Mon Aug 31 18:58:50 2020 +0200
> > 
> >     char: virtio: Select VIRTIO from VIRTIO_CONSOLE.
> >     
> >     Make it possible to have virtio console built-in when
> >     other virtio drivers are modular.
> >     
> >     Signed-off-by: Michal Suchanek <msuchanek@suse.de>
> >     Reviewed-by: Amit Shah <amit@kernel.org>
> >     Link: https://lore.kernel.org/r/20200831165850.26163-1-msuchanek@suse.de
> >     Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > 
> > which seems kind of bogus - why do we care about allowing a builtin
> > virtio console driver if the pci virtio bus driver is a module?
> > There won't be any devices on the bus to attach to ...
> 
> When using other transports ?

Any transport selects VIRTIO so if you enable that, you get
VIRTIO and thus it's enough to depend on it.

> In my current project, eg. I'm using mmio - my kernel has pci completely
> disabled.
> 
> > I am inclined to fix console and virtio fs to depend on VIRTIO:
> > select is harder to use correctly ...
> 
> I don't thinkt that would be good - instead everybody should just select
> VIRTIO, never depend on it (maybe depend on VIRTIO_MENU instead)

GPU depends on VIRTIO and on VIRTIO_MENU ... which seems even messier
...

> 
> --mtx
> 
> -- 
> ---
> Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert
> werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren
> GPG/PGP-Schlüssel zu.
> ---
> Enrico Weigelt, metux IT consult
> Free software and Linux embedded engineering
> info@metux.net -- +49-151-27565287
Michael S. Tsirkin Dec. 7, 2020, 1:53 p.m. UTC | #12
On Mon, Dec 07, 2020 at 11:12:50AM +0800, Jason Wang wrote:
> 
> On 2020/12/6 上午3:32, Michael S. Tsirkin wrote:
> > On Sat, Dec 05, 2020 at 08:59:55AM +0100, Enrico Weigelt, metux IT consult wrote:
> > > On 04.12.20 04:35, Jason Wang wrote:
> > > 
> > > > > --- a/drivers/gpio/Kconfig
> > > > > +++ b/drivers/gpio/Kconfig
> > > > > @@ -1615,6 +1615,15 @@ config GPIO_MOCKUP
> > > > >         tools/testing/selftests/gpio/gpio-mockup.sh. Reference the
> > > > > usage in
> > > > >         it.
> > > > >   +config GPIO_VIRTIO
> > > > > +    tristate "VirtIO GPIO support"
> > > > > +    depends on VIRTIO
> > > > 
> > > > Let's use select, since there's no prompt for VIRTIO and it doesn't have
> > > > any dependencies.
> > > whoops, it's not that simple:
> > > 
> > > make: Entering directory '/home/nekrad/src/apu2-dev/pkg/kernel.apu2.git'
> > > make[1]: Entering directory
> > > '/home/nekrad/src/dk/DistroKit/platform-x86_64/build-target/linux-5.8.9-build'
> > >    GEN     Makefile
> > > drivers/gpu/drm/Kconfig:74:error: recursive dependency detected!
> > > drivers/gpu/drm/Kconfig:74:	symbol DRM_KMS_HELPER is selected by
> > > DRM_VIRTIO_GPU
> > > drivers/gpu/drm/virtio/Kconfig:2:	symbol DRM_VIRTIO_GPU depends on VIRTIO
> > > drivers/virtio/Kconfig:2:	symbol VIRTIO is selected by GPIO_VIRTIO
> > > drivers/gpio/Kconfig:1618:	symbol GPIO_VIRTIO depends on GPIOLIB
> > > drivers/gpio/Kconfig:14:	symbol GPIOLIB is selected by I2C_MUX_LTC4306
> > > drivers/i2c/muxes/Kconfig:47:	symbol I2C_MUX_LTC4306 depends on I2C
> > > drivers/i2c/Kconfig:8:	symbol I2C is selected by FB_DDC
> > > drivers/video/fbdev/Kconfig:63:	symbol FB_DDC depends on FB
> > > drivers/video/fbdev/Kconfig:12:	symbol FB is selected by DRM_KMS_FB_HELPER
> > > drivers/gpu/drm/Kconfig:80:	symbol DRM_KMS_FB_HELPER depends on
> > > DRM_KMS_HELPER
> > > 
> > > Seems that we can only depend on or select some symbol - we run into
> > > huge trouble if thats mixed. Just changed DRM_VIRTIO_GPU to just select
> > > VIRIO instead of depending on it, and now it works.
> > > 
> > > I've posted another patch for fixing drivers/gpu/drm/virtio/Kconfig
> > > to use 'select' instead of 'depends on'.
> > It seems a bit of a mess, at this point I'm not entirely sure when
> > should drivers select VIRTIO and when depend on it.
> > 
> > The text near it says:
> > 
> > # SPDX-License-Identifier: GPL-2.0-only
> > config VIRTIO
> >          tristate
> >          help
> >            This option is selected by any driver which implements the virtio
> >            bus, such as CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_MMIO, CONFIG_RPMSG
> >            or CONFIG_S390_GUEST.
> > 
> > Which seems clear enough and would indicate drivers for devices *behind*
> > the bus should not select VIRTIO and thus presumably should "depend on" it.
> > This is violated in virtio console and virtio fs drivers.
> > 
> > For console it says:
> > 
> > commit 9f30eb29c514589e16f2999ea070598583d1f6ec
> > Author: Michal Suchanek <msuchanek@suse.de>
> > Date:   Mon Aug 31 18:58:50 2020 +0200
> > 
> >      char: virtio: Select VIRTIO from VIRTIO_CONSOLE.
> >      Make it possible to have virtio console built-in when
> >      other virtio drivers are modular.
> >      Signed-off-by: Michal Suchanek <msuchanek@suse.de>
> >      Reviewed-by: Amit Shah <amit@kernel.org>
> >      Link: https://lore.kernel.org/r/20200831165850.26163-1-msuchanek@suse.de
> >      Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > 
> > which seems kind of bogus - why do we care about allowing a builtin
> > virtio console driver if the pci virtio bus driver is a module?
> > There won't be any devices on the bus to attach to ...
> 
> 
> For testing like switching bus from pci to MMIO?


Not sure I understand ... can you give an example?

> 
> > And for virtio fs it was like this from the beginning.
> > 
> > I am inclined to fix console and virtio fs to depend on VIRTIO:
> > select is harder to use correctly ...
> > 
> > Jason?
> 
> 
> I think it works, but we need a prompt for VIRTIO otherwise there's no way
> to enable it.
> 
> Thanks

That's even messier. No one needs VIRTIO core by itself - it's only used
by transports and drivers.

> 
> > 
> > 
> > > -- 
> > > ---
> > > Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert
> > > werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren
> > > GPG/PGP-Schlüssel zu.
> > > ---
> > > Enrico Weigelt, metux IT consult
> > > Free software and Linux embedded engineering
> > > info@metux.net -- +49-151-27565287
Enrico Weigelt, metux IT consult Dec. 7, 2020, 8:34 p.m. UTC | #13
On 07.12.20 14:52, Michael S. Tsirkin wrote:

>> See above: NAK. because it can't even be enabled directly (by the user).
>> If it wasn't meant otherwise, we'd have to add an menu text.
> 
> The point is that user enables one of the bindings.
> That in turn enables drivers. If we merely select VIRTIO
> there's a chance user won't remember to select any bindings
> and will be surprised not to see any devices.

Not sure what you mean by "bindings" ... transports ?

IMHO, transports and device drivers are entirely orthogonal. Both *use*
the core, but I don't think they shall only show up, after the core was
enabled explicitly. Any combination of transports is valid (having none
at all, of course, isn't actually useful).

>> When using other transports ?
> 
> Any transport selects VIRTIO so if you enable that, you get
> VIRTIO and thus it's enough to depend on it.

The combination of 'select VIRTIO' and 'depends on VIRTIO' is what
caused the recursive dependency. Chaning everything to 'select VIRTIO'
fixed that.

>> I don't thinkt that would be good - instead everybody should just select
>> VIRTIO, never depend on it (maybe depend on VIRTIO_MENU instead)
> 
> GPU depends on VIRTIO and on VIRTIO_MENU ... which seems even messier
> ...

See:
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg2404871.html


--mtx
Jason Wang Dec. 8, 2020, 2:36 a.m. UTC | #14
On 2020/12/7 下午9:53, Michael S. Tsirkin wrote:
> On Mon, Dec 07, 2020 at 11:12:50AM +0800, Jason Wang wrote:
>> On 2020/12/6 上午3:32, Michael S. Tsirkin wrote:
>>> On Sat, Dec 05, 2020 at 08:59:55AM +0100, Enrico Weigelt, metux IT consult wrote:
>>>> On 04.12.20 04:35, Jason Wang wrote:
>>>>
>>>>>> --- a/drivers/gpio/Kconfig
>>>>>> +++ b/drivers/gpio/Kconfig
>>>>>> @@ -1615,6 +1615,15 @@ config GPIO_MOCKUP
>>>>>>         tools/testing/selftests/gpio/gpio-mockup.sh. Reference the
>>>>>> usage in
>>>>>>         it.
>>>>>>   +config GPIO_VIRTIO
>>>>>> +    tristate "VirtIO GPIO support"
>>>>>> +    depends on VIRTIO
>>>>> Let's use select, since there's no prompt for VIRTIO and it doesn't have
>>>>> any dependencies.
>>>> whoops, it's not that simple:
>>>>
>>>> make: Entering directory '/home/nekrad/src/apu2-dev/pkg/kernel.apu2.git'
>>>> make[1]: Entering directory
>>>> '/home/nekrad/src/dk/DistroKit/platform-x86_64/build-target/linux-5.8.9-build'
>>>>     GEN     Makefile
>>>> drivers/gpu/drm/Kconfig:74:error: recursive dependency detected!
>>>> drivers/gpu/drm/Kconfig:74:	symbol DRM_KMS_HELPER is selected by
>>>> DRM_VIRTIO_GPU
>>>> drivers/gpu/drm/virtio/Kconfig:2:	symbol DRM_VIRTIO_GPU depends on VIRTIO
>>>> drivers/virtio/Kconfig:2:	symbol VIRTIO is selected by GPIO_VIRTIO
>>>> drivers/gpio/Kconfig:1618:	symbol GPIO_VIRTIO depends on GPIOLIB
>>>> drivers/gpio/Kconfig:14:	symbol GPIOLIB is selected by I2C_MUX_LTC4306
>>>> drivers/i2c/muxes/Kconfig:47:	symbol I2C_MUX_LTC4306 depends on I2C
>>>> drivers/i2c/Kconfig:8:	symbol I2C is selected by FB_DDC
>>>> drivers/video/fbdev/Kconfig:63:	symbol FB_DDC depends on FB
>>>> drivers/video/fbdev/Kconfig:12:	symbol FB is selected by DRM_KMS_FB_HELPER
>>>> drivers/gpu/drm/Kconfig:80:	symbol DRM_KMS_FB_HELPER depends on
>>>> DRM_KMS_HELPER
>>>>
>>>> Seems that we can only depend on or select some symbol - we run into
>>>> huge trouble if thats mixed. Just changed DRM_VIRTIO_GPU to just select
>>>> VIRIO instead of depending on it, and now it works.
>>>>
>>>> I've posted another patch for fixing drivers/gpu/drm/virtio/Kconfig
>>>> to use 'select' instead of 'depends on'.
>>> It seems a bit of a mess, at this point I'm not entirely sure when
>>> should drivers select VIRTIO and when depend on it.
>>>
>>> The text near it says:
>>>
>>> # SPDX-License-Identifier: GPL-2.0-only
>>> config VIRTIO
>>>           tristate
>>>           help
>>>             This option is selected by any driver which implements the virtio
>>>             bus, such as CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_MMIO, CONFIG_RPMSG
>>>             or CONFIG_S390_GUEST.
>>>
>>> Which seems clear enough and would indicate drivers for devices *behind*
>>> the bus should not select VIRTIO and thus presumably should "depend on" it.
>>> This is violated in virtio console and virtio fs drivers.
>>>
>>> For console it says:
>>>
>>> commit 9f30eb29c514589e16f2999ea070598583d1f6ec
>>> Author: Michal Suchanek <msuchanek@suse.de>
>>> Date:   Mon Aug 31 18:58:50 2020 +0200
>>>
>>>       char: virtio: Select VIRTIO from VIRTIO_CONSOLE.
>>>       Make it possible to have virtio console built-in when
>>>       other virtio drivers are modular.
>>>       Signed-off-by: Michal Suchanek <msuchanek@suse.de>
>>>       Reviewed-by: Amit Shah <amit@kernel.org>
>>>       Link: https://lore.kernel.org/r/20200831165850.26163-1-msuchanek@suse.de
>>>       Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>>>
>>> which seems kind of bogus - why do we care about allowing a builtin
>>> virtio console driver if the pci virtio bus driver is a module?
>>> There won't be any devices on the bus to attach to ...
>>
>> For testing like switching bus from pci to MMIO?
>
> Not sure I understand ... can you give an example?


E.g testing

modprobe -r virtio_mmio
modprobe virtio_pci

?


>
>>> And for virtio fs it was like this from the beginning.
>>>
>>> I am inclined to fix console and virtio fs to depend on VIRTIO:
>>> select is harder to use correctly ...
>>>
>>> Jason?
>>
>> I think it works, but we need a prompt for VIRTIO otherwise there's no way
>> to enable it.
>>
>> Thanks
> That's even messier. No one needs VIRTIO core by itself - it's only used
> by transports and drivers.


So we endup with two solutions (without a prompt):

1) using select, user may end up with driver without transport
2) using depends, user need to enable at least one transport

2) looks a little bit better I admit.

Thanks


>
>>>
>>>> -- 
>>>> ---
>>>> Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert
>>>> werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren
>>>> GPG/PGP-Schlüssel zu.
>>>> ---
>>>> Enrico Weigelt, metux IT consult
>>>> Free software and Linux embedded engineering
>>>> info@metux.net -- +49-151-27565287
Jason Wang Dec. 8, 2020, 2:49 a.m. UTC | #15
On 2020/12/7 下午5:33, Enrico Weigelt, metux IT consult wrote:
> On 07.12.20 04:48, Jason Wang wrote:
>
> Hi,
>
>>>> Not a native speaker but event sounds like something driver read from
>>>> device. Looking at the below lists, most of them except for
>>>> VIRTIO_GPIO_EV_HOST_LEVEL looks more like a command.
>>> okay, shall I name it "message" ?
>>
>> It might be better.
> Okay, renamed to messages in v3.
>
>>>> #define VIRTIO_NET_OK     0
>>>> #define VIRTIO_NET_ERR    1
>>> hmm, so I'd need to define all the error codes that possibly could
>>> happen ?
>>
>> Yes, I think you need.
> Okay, going to do it in the next version.
>
>>>> If I read the code correctly, this expects there will be at most a
>>>> single type of event that can be processed at the same time. E.g can
>>>> upper layer want to read from different lines in parallel? If yes, we
>>>> need to deal with that.
>>> @Linus @Bartosz: can that happen or does gpio subsys already serialize
>>> requests ?
>>>
>>> Initially, I tried to protect it by spinlock (so, only one request may
>>> run at a time, other calls just wait until the first is finished), but
>>> it crashed when gpio cdev registration calls into the driver (fetches
>>> the status) while still in bootup.
>>>
>>> Don't recall the exact error anymore, but something like an
>>> inconsistency in the spinlock calls.
>>>
>>> Did I just use the wrong type of lock ?
>> I'm not sure since I am not familiar with GPIO. But a question is, if at
>> most one request is allowed, I'm not sure virtio is the best choice here
>> since we don't even need a queue(virtqueue) here.
> I guess, I should add locks to the gpio callback functions (where gpio
> subsys calls in). That way, requests are requests are strictly ordered.
> The locks didn't work in my previous attempts, but probably because I've
> missed to set the can_sleep flag (now fixed in v3).
>
> The gpio ops are already waiting for reply of the corresponding type, so
> the only bad thing that could happen is the same operation being called
> twice (when coming from different threads) and replies mixed up between
> first and second one. OTOH I don't see much problem w/ that. This can be
> fixed by adding a global lock.
>
>> I think it's still about whether or not we need allow a batch of
>> requests via a queue. Consider you've submitted two request A and B, and
>> if B is done first, current code won't work. This is because, the reply
>> is transported via rxq buffers not just reuse the txq buffer if I read
>> the code correctly.
> Meanwhile I've changed it to allocate a new rx buffer for the reply
> (done right before the request is sent), so everything should be
> processed in the order it had been sent. Assuming virtio keeps the
> order of the buffers in the queues.


Unfortunately, there's no guarantee that virtio will keep the order or 
it needs to advertise VIRTIO_F_IN_ORDER. (see 2.6.9 in the virtio spec).

Btw, if possible, it's better to add a link to the userspace code here.


>
>>> Could you please give an example how bi-directional transmission within
>>> the same queue could look like ?
>> You can check how virtio-blk did this in:
>>
>> https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-2500006
> hmm, still don't see how the code would actually look like. (in qemu as
> well as kernel). Just add the fetched inbuf as an outbuf (within the
> same queue) ?


Yes, virtio allows adding IN buffers after OUT buffer through descriptor 
chaining.

Thanks


>
>>> Maybe add one new buffer per request and one new per received async
>>> signal ?
>> It would be safe to fill the whole rxq and do the refill e.g when half
>> of the queue is used.
> Okay, doing that now in v3: there's always at least one rx buffer,
> and requests as well as the intr receiver always add a new one.
> (they get removed on fetching, IMHO).
>
>
> --mtx
>
Enrico Weigelt, metux IT consult Dec. 8, 2020, 7:02 a.m. UTC | #16
On 08.12.20 03:36, Jason Wang wrote:

Hi,

> So we endup with two solutions (without a prompt):
> 
> 1) using select, user may end up with driver without transport

IMHO not an entirely unusual situation in other places of the kernel,
eg. one can enable USB devices, w/o having an usb host adapter enabled.

And even if some USB-HA driver is enabled, the actualy machine doesn't
necessarily have the corresponding device.

> 2) using depends, user need to enable at least one transport
> 
> 2) looks a little bit better I admit.

So, all virtio devices should depend on TRANSPORT_A || TRANSPORT_B ||
TRANSPORT_C || ... ? (and also change all these places if another
transport is added) ?

--mtx
Linus Walleij Dec. 8, 2020, 9:38 a.m. UTC | #17
On Sat, Dec 5, 2020 at 9:15 PM Enrico Weigelt, metux IT consult
<lkml@metux.net> wrote:

> The virtio-gpio device/host can raise a signal on line state change.
> Kinda IRQ, but not actually running through real IRQs, instead by a
> message running though queue. (hmm, kida MSI ? :o).
>
> I've tried allocating an IRQ range and calling generic_handle_irq(),
> but then I'm getting unhanled IRQ trap.

This is Bartosz territory, but the gpio-mockup.c driver will insert
IRQs into the system, he went and added really core stuff
into kernel/irq to make this happen. Notice that in Kconfig
it does:

select IRQ_SIM

Then this is used:
include/linux/irq_sim.h

This is intended for simulating IRQs and both GPIO and IIO use it.
I think this inserts IRQs from debugfs and I have no idea how
flexible that is.

If it is suitable for what you want to do I don't know but it's
virtio so...

Yours,
Linus Walleij
Michal Suchánek Dec. 8, 2020, 10:10 a.m. UTC | #18
Hello

On Sat, Dec 05, 2020 at 02:32:04PM -0500, Michael S. Tsirkin wrote:
> On Sat, Dec 05, 2020 at 08:59:55AM +0100, Enrico Weigelt, metux IT consult wrote:
> > On 04.12.20 04:35, Jason Wang wrote:
> > 
> > > 
> > > Let's use select, since there's no prompt for VIRTIO and it doesn't have
> > > any dependencies.
> > 
> > whoops, it's not that simple:
> > 
> > make: Entering directory '/home/nekrad/src/apu2-dev/pkg/kernel.apu2.git'
> > make[1]: Entering directory
> > '/home/nekrad/src/dk/DistroKit/platform-x86_64/build-target/linux-5.8.9-build'
> >   GEN     Makefile
> > drivers/gpu/drm/Kconfig:74:error: recursive dependency detected!
> > drivers/gpu/drm/Kconfig:74:	symbol DRM_KMS_HELPER is selected by
> > DRM_VIRTIO_GPU
> > drivers/gpu/drm/virtio/Kconfig:2:	symbol DRM_VIRTIO_GPU depends on VIRTIO
> > drivers/virtio/Kconfig:2:	symbol VIRTIO is selected by GPIO_VIRTIO
> > drivers/gpio/Kconfig:1618:	symbol GPIO_VIRTIO depends on GPIOLIB
> > drivers/gpio/Kconfig:14:	symbol GPIOLIB is selected by I2C_MUX_LTC4306
> > drivers/i2c/muxes/Kconfig:47:	symbol I2C_MUX_LTC4306 depends on I2C
> > drivers/i2c/Kconfig:8:	symbol I2C is selected by FB_DDC
> > drivers/video/fbdev/Kconfig:63:	symbol FB_DDC depends on FB
> > drivers/video/fbdev/Kconfig:12:	symbol FB is selected by DRM_KMS_FB_HELPER
> > drivers/gpu/drm/Kconfig:80:	symbol DRM_KMS_FB_HELPER depends on
> > DRM_KMS_HELPER
> > 
> > Seems that we can only depend on or select some symbol - we run into
> > huge trouble if thats mixed. Just changed DRM_VIRTIO_GPU to just select
> > VIRIO instead of depending on it, and now it works.
> > 
> > I've posted another patch for fixing drivers/gpu/drm/virtio/Kconfig
> > to use 'select' instead of 'depends on'.
> 
> It seems a bit of a mess, at this point I'm not entirely sure when
> should drivers select VIRTIO and when depend on it.
> 
> The text near it says:
> 
> # SPDX-License-Identifier: GPL-2.0-only
> config VIRTIO
>         tristate
>         help
>           This option is selected by any driver which implements the virtio
>           bus, such as CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_MMIO, CONFIG_RPMSG
>           or CONFIG_S390_GUEST.
> 
> Which seems clear enough and would indicate drivers for devices *behind*
> the bus should not select VIRTIO and thus presumably should "depend on" it.
> This is violated in virtio console and virtio fs drivers.
> 
> For console it says:
> 
> commit 9f30eb29c514589e16f2999ea070598583d1f6ec
> Author: Michal Suchanek <msuchanek@suse.de>
> Date:   Mon Aug 31 18:58:50 2020 +0200
> 
>     char: virtio: Select VIRTIO from VIRTIO_CONSOLE.
>     
>     Make it possible to have virtio console built-in when
>     other virtio drivers are modular.
>     
>     Signed-off-by: Michal Suchanek <msuchanek@suse.de>
>     Reviewed-by: Amit Shah <amit@kernel.org>
>     Link: https://lore.kernel.org/r/20200831165850.26163-1-msuchanek@suse.de
>     Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> 
> which seems kind of bogus - why do we care about allowing a builtin
> virtio console driver if the pci virtio bus driver is a module?
> There won't be any devices on the bus to attach to ...
The console driver provides early console which should initialize
without any transport. I have not tested it actually works but the code
seems to be there to support this use case.

Thanks

Michal
Enrico Weigelt, metux IT consult Dec. 8, 2020, 12:33 p.m. UTC | #19
On 08.12.20 11:10, Michal Suchánek wrote:

Hi,

> The console driver provides early console which should initialize
> without any transport. I have not tested it actually works but the code
> seems to be there to support this use case.

What does it do if it hasn't got any transport yet ?

Just eat the bits or buffer everything, until it gets a transportport
and sends out later ?

--mtx
Enrico Weigelt, metux IT consult Dec. 8, 2020, 2:04 p.m. UTC | #20
On 08.12.20 10:38, Linus Walleij wrote:

Hi,

> This is Bartosz territory, but the gpio-mockup.c driver will insert
> IRQs into the system, he went and added really core stuff
> into kernel/irq to make this happen. Notice that in Kconfig
> it does:
> 
> select IRQ_SIM
> 
> Then this is used:
> include/linux/irq_sim.h
> 
> This is intended for simulating IRQs and both GPIO and IIO use it.
> I think this inserts IRQs from debugfs and I have no idea how
> flexible that is.

Oh, thx.

It seems to implement a pseudo-irqchip driver. I've already though about
doing that, but didn't think its worth it, just for my driver alone.
I've implemented a few irq handling cb's directly the driver. But since
we already have it, I'll reconsider :)

BUT: this wasn't exactly my question :p

I've been looking for some more direct notification callback for gpio
consumers: here the consumer would register itself as a listener on
some gpio_desc and called back when something changes (with data what
exactly changed, eg. "gpio #3 input switched to high").

Seems we currently just have the indirect path via interrupts.

--mtx
Grygorii Strashko Dec. 8, 2020, 4:15 p.m. UTC | #21
On 08/12/2020 16:04, Enrico Weigelt, metux IT consult wrote:
> On 08.12.20 10:38, Linus Walleij wrote:
> 
> Hi,
> 
>> This is Bartosz territory, but the gpio-mockup.c driver will insert
>> IRQs into the system, he went and added really core stuff
>> into kernel/irq to make this happen. Notice that in Kconfig
>> it does:
>>
>> select IRQ_SIM
>>
>> Then this is used:
>> include/linux/irq_sim.h
>>
>> This is intended for simulating IRQs and both GPIO and IIO use it.
>> I think this inserts IRQs from debugfs and I have no idea how
>> flexible that is.
> 
> Oh, thx.
> 
> It seems to implement a pseudo-irqchip driver. I've already though about
> doing that, but didn't think its worth it, just for my driver alone.
> I've implemented a few irq handling cb's directly the driver. But since
> we already have it, I'll reconsider :)
> 
> BUT: this wasn't exactly my question :p
> 
> I've been looking for some more direct notification callback for gpio
> consumers: here the consumer would register itself as a listener on
> some gpio_desc and called back when something changes (with data what
> exactly changed, eg. "gpio #3 input switched to high").

But this is exactly why there is GPIO IRQs in the first place,
which are used to notify consumers.

More over most consumers doesn't know where the IRQ came from - on one HW it can be gpio,
while on another HW - direct interrupt controller line.
Linus Walleij Dec. 9, 2020, 8:51 a.m. UTC | #22
On Tue, Dec 8, 2020 at 3:07 PM Enrico Weigelt, metux IT consult
<lkml@metux.net> wrote:

> I've been looking for some more direct notification callback for gpio
> consumers: here the consumer would register itself as a listener on
> some gpio_desc and called back when something changes (with data what
> exactly changed, eg. "gpio #3 input switched to high").
>
> Seems we currently just have the indirect path via interrupts.

I don't know how indirect it is, it seems pretty direct to me. The subsystem
was designed in response to how the hardware in front of the developers
worked.

So far we have had:
- Cascaded interrupts
- Dedicated (hieararchical) interrupts
- Message Signalled Interrupts

And if you now bring something else to the show then it's not like the
subsystem was designed for some abstract quality such as
generic notification of events that occurred, all practical instances
have been around actual IRQs and that is why it is using
struct irq_chip.

What we need to understand is if your new usecase is an outlier
so it is simplest modeled by a "mock" irq_chip or we have to design
something new altogether like notifications on changes. I suspect
irq_chip would be best because all drivers using GPIOs for interrupts
are expecting interrupts, and it would be an enormous task to
change them all and really annoying to create a new mechanism
on the side.

Yours,
Linus Walleij
Jason Wang Dec. 9, 2020, 9:31 a.m. UTC | #23
On 2020/12/8 下午3:02, Enrico Weigelt, metux IT consult wrote:
> On 08.12.20 03:36, Jason Wang wrote:
>
> Hi,
>
>> So we endup with two solutions (without a prompt):
>>
>> 1) using select, user may end up with driver without transport
> IMHO not an entirely unusual situation in other places of the kernel,
> eg. one can enable USB devices, w/o having an usb host adapter enabled.
>
> And even if some USB-HA driver is enabled, the actualy machine doesn't
> necessarily have the corresponding device.


Ok, then select works for me.


>
>> 2) using depends, user need to enable at least one transport
>>
>> 2) looks a little bit better I admit.
> So, all virtio devices should depend on TRANSPORT_A || TRANSPORT_B ||
> TRANSPORT_C || ... ? (and also change all these places if another
> transport is added) ?


I think not. The idea is, if none of the transport (select VIRTIO) is 
enabled, user can not enable any virtio drivers (depends on VIRTIO).

Thanks


>
> --mtx
>
Enrico Weigelt, metux IT consult Dec. 9, 2020, 10:33 a.m. UTC | #24
On 09.12.20 10:31, Jason Wang wrote:

Hi,

>> And even if some USB-HA driver is enabled, the actualy machine doesn't
>> necessarily have the corresponding device.
> 
> Ok, then select works for me.

Great, so does everybody aggree on my patch ?
https://lore.kernel.org/lkml/20201204131221.2827-1-info@metux.net/

--mtx
Michal Suchánek Dec. 9, 2020, 10:34 a.m. UTC | #25
On Tue, Dec 08, 2020 at 01:33:02PM +0100, Enrico Weigelt, metux IT consult wrote:
> On 08.12.20 11:10, Michal Suchánek wrote:
> 
> Hi,
> 
> > The console driver provides early console which should initialize
> > without any transport. I have not tested it actually works but the code
> > seems to be there to support this use case.
> 
> What does it do if it hasn't got any transport yet ?
> 
> Just eat the bits or buffer everything, until it gets a transportport
> and sends out later ?

Why would it need the transport?

It's not like the data goes through an actual PCI bus, it is only used
for discovering and configuring the device, right?

Then if you do ad-hoc configuration of the device you do not need the
trasport at all.

Thanks

Michal
Arnd Bergmann Dec. 9, 2020, 11:19 a.m. UTC | #26
On Wed, Dec 9, 2020 at 9:51 AM Linus Walleij <linus.walleij@linaro.org> wrote:
> On Tue, Dec 8, 2020 at 3:07 PM Enrico Weigelt, metux IT consult <lkml@metux.net> wrote:

> What we need to understand is if your new usecase is an outlier
> so it is simplest modeled by a "mock" irq_chip or we have to design
> something new altogether like notifications on changes. I suspect
> irq_chip would be best because all drivers using GPIOs for interrupts
> are expecting interrupts, and it would be an enormous task to
> change them all and really annoying to create a new mechanism
> on the side.

I would expect the platform abstraction to actually be close enough
to a chained irqchip that it actually works: the notification should
come in via vring_interrupt(), which is a normal interrupt handler
that calls vq->vq.callback(), calling generic_handle_irq() (and
possibly chained_irq_enter()/chained_irq_exit() around it) like the
other gpio drivers do should just work here I think, and if it did
not, then I would expect this to be just a bug in the driver rather
than something missing in the gpio framework.

       Arnd
Linus Walleij Dec. 9, 2020, 12:53 p.m. UTC | #27
On Wed, Dec 9, 2020 at 12:19 PM Arnd Bergmann <arnd@kernel.org> wrote:
> On Wed, Dec 9, 2020 at 9:51 AM Linus Walleij <linus.walleij@linaro.org> wrote:
> > On Tue, Dec 8, 2020 at 3:07 PM Enrico Weigelt, metux IT consult <lkml@metux.net> wrote:
>
> > What we need to understand is if your new usecase is an outlier
> > so it is simplest modeled by a "mock" irq_chip or we have to design
> > something new altogether like notifications on changes. I suspect
> > irq_chip would be best because all drivers using GPIOs for interrupts
> > are expecting interrupts, and it would be an enormous task to
> > change them all and really annoying to create a new mechanism
> > on the side.
>
> I would expect the platform abstraction to actually be close enough
> to a chained irqchip that it actually works: the notification should
> come in via vring_interrupt(), which is a normal interrupt handler
> that calls vq->vq.callback(), calling generic_handle_irq() (and
> possibly chained_irq_enter()/chained_irq_exit() around it) like the
> other gpio drivers do should just work here I think, and if it did
> not, then I would expect this to be just a bug in the driver rather
> than something missing in the gpio framework.

Performance/latency-wise that would also be strongly encouraged.

Tglx isn't super-happy about the chained interrupts at times, as they
can create really nasty bugs, but a pure IRQ in fastpath of some
kinde is preferable and intuitive either way.

Yours,
Linus Walleij
Grygorii Strashko Dec. 9, 2020, 8:22 p.m. UTC | #28
On 09/12/2020 14:53, Linus Walleij wrote:
> On Wed, Dec 9, 2020 at 12:19 PM Arnd Bergmann <arnd@kernel.org> wrote:
>> On Wed, Dec 9, 2020 at 9:51 AM Linus Walleij <linus.walleij@linaro.org> wrote:
>>> On Tue, Dec 8, 2020 at 3:07 PM Enrico Weigelt, metux IT consult <lkml@metux.net> wrote:
>>
>>> What we need to understand is if your new usecase is an outlier
>>> so it is simplest modeled by a "mock" irq_chip or we have to design
>>> something new altogether like notifications on changes. I suspect
>>> irq_chip would be best because all drivers using GPIOs for interrupts
>>> are expecting interrupts, and it would be an enormous task to
>>> change them all and really annoying to create a new mechanism
>>> on the side.
>>
>> I would expect the platform abstraction to actually be close enough
>> to a chained irqchip that it actually works: the notification should
>> come in via vring_interrupt(), which is a normal interrupt handler
>> that calls vq->vq.callback(), calling generic_handle_irq() (and
>> possibly chained_irq_enter()/chained_irq_exit() around it) like the
>> other gpio drivers do should just work here I think, and if it did
>> not, then I would expect this to be just a bug in the driver rather
>> than something missing in the gpio framework.
> 
> Performance/latency-wise that would also be strongly encouraged.
> 
> Tglx isn't super-happy about the chained interrupts at times, as they
> can create really nasty bugs, but a pure IRQ in fastpath of some
> kinde is preferable and intuitive either way.

In my opinion the problem here is that proposed patch somehow describes Front end, but
says nothing about Backend and overall design.

What is expected to be virtualized? whole GPIO chip? or set of GPIOs from different GPIO chips?
Most often nobody want to give Guest access to the whole GPIO chip, so, most probably, smth. similar to
GPIO Aggregator will be needed.

How is situation going to be resolved that GPIO framework and IRQ framework are independent, but intersect, and
GPIO controller drivers have no idea who and how requesting GPIO IRQ - threaded vs non-threaded vs oneshot.
So, some information need to be transferred to Back end  - at minimum IRQ triggering type.

Overall, it might be better to start from pure gpio and leave GPIO IRQ aside.
Arnd Bergmann Dec. 9, 2020, 8:38 p.m. UTC | #29
On Wed, Dec 9, 2020 at 9:22 PM Grygorii Strashko
<grygorii.strashko@ti.com> wrote:
> On 09/12/2020 14:53, Linus Walleij wrote:
> > On Wed, Dec 9, 2020 at 12:19 PM Arnd Bergmann <arnd@kernel.org> wrote:
> >> On Wed, Dec 9, 2020 at 9:51 AM Linus Walleij <linus.walleij@linaro.org> wrote:
> >>> On Tue, Dec 8, 2020 at 3:07 PM Enrico Weigelt, metux IT consult <lkml@metux.net> wrote:
> >>
> >>> What we need to understand is if your new usecase is an outlier
> >>> so it is simplest modeled by a "mock" irq_chip or we have to design
> >>> something new altogether like notifications on changes. I suspect
> >>> irq_chip would be best because all drivers using GPIOs for interrupts
> >>> are expecting interrupts, and it would be an enormous task to
> >>> change them all and really annoying to create a new mechanism
> >>> on the side.
> >>
> >> I would expect the platform abstraction to actually be close enough
> >> to a chained irqchip that it actually works: the notification should
> >> come in via vring_interrupt(), which is a normal interrupt handler
> >> that calls vq->vq.callback(), calling generic_handle_irq() (and
> >> possibly chained_irq_enter()/chained_irq_exit() around it) like the
> >> other gpio drivers do should just work here I think, and if it did
> >> not, then I would expect this to be just a bug in the driver rather
> >> than something missing in the gpio framework.
> >
> > Performance/latency-wise that would also be strongly encouraged.
> >
> > Tglx isn't super-happy about the chained interrupts at times, as they
> > can create really nasty bugs, but a pure IRQ in fastpath of some
> > kinde is preferable and intuitive either way.
>
> In my opinion the problem here is that proposed patch somehow describes Front end, but
> says nothing about Backend and overall design.
>
> What is expected to be virtualized? whole GPIO chip? or set of GPIOs from different GPIO chips?
> Most often nobody want to give Guest access to the whole GPIO chip, so, most probably, smth. similar to
> GPIO Aggregator will be needed.

I would argue that it does not matter, the virtual GPIO chip could really
be anything. Certain functions such as a gpio based keyboard require
interrupts, so it sounds useful to make them work.

     Arnd
Grygorii Strashko Dec. 10, 2020, 1:32 p.m. UTC | #30
On 09/12/2020 22:38, Arnd Bergmann wrote:
> On Wed, Dec 9, 2020 at 9:22 PM Grygorii Strashko
> <grygorii.strashko@ti.com> wrote:
>> On 09/12/2020 14:53, Linus Walleij wrote:
>>> On Wed, Dec 9, 2020 at 12:19 PM Arnd Bergmann <arnd@kernel.org> wrote:
>>>> On Wed, Dec 9, 2020 at 9:51 AM Linus Walleij <linus.walleij@linaro.org> wrote:
>>>>> On Tue, Dec 8, 2020 at 3:07 PM Enrico Weigelt, metux IT consult <lkml@metux.net> wrote:
>>>>
>>>>> What we need to understand is if your new usecase is an outlier
>>>>> so it is simplest modeled by a "mock" irq_chip or we have to design
>>>>> something new altogether like notifications on changes. I suspect
>>>>> irq_chip would be best because all drivers using GPIOs for interrupts
>>>>> are expecting interrupts, and it would be an enormous task to
>>>>> change them all and really annoying to create a new mechanism
>>>>> on the side.
>>>>
>>>> I would expect the platform abstraction to actually be close enough
>>>> to a chained irqchip that it actually works: the notification should
>>>> come in via vring_interrupt(), which is a normal interrupt handler
>>>> that calls vq->vq.callback(), calling generic_handle_irq() (and
>>>> possibly chained_irq_enter()/chained_irq_exit() around it) like the
>>>> other gpio drivers do should just work here I think, and if it did
>>>> not, then I would expect this to be just a bug in the driver rather
>>>> than something missing in the gpio framework.
>>>
>>> Performance/latency-wise that would also be strongly encouraged.
>>>
>>> Tglx isn't super-happy about the chained interrupts at times, as they
>>> can create really nasty bugs, but a pure IRQ in fastpath of some
>>> kinde is preferable and intuitive either way.
>>
>> In my opinion the problem here is that proposed patch somehow describes Front end, but
>> says nothing about Backend and overall design.
>>
>> What is expected to be virtualized? whole GPIO chip? or set of GPIOs from different GPIO chips?
>> Most often nobody want to give Guest access to the whole GPIO chip, so, most probably, smth. similar to
>> GPIO Aggregator will be needed.
> 
> I would argue that it does not matter, the virtual GPIO chip could really
> be anything. Certain functions such as a gpio based keyboard require
> interrupts, so it sounds useful to make them work.

Agree, and my point was not to discard IRQ support, but solve problem step by step.
And existing Back end, in any form, may just help to understand virtio-gpio protocol spec and
identify missed pieces.

For example, should 'Configuration space' specify if IRQs are supported on not?
Alex Bennée April 13, 2021, 11:07 a.m. UTC | #31
"Enrico Weigelt, metux IT consult" <info@metux.net> writes:

> On 04.12.20 04:35, Jason Wang wrote:
>
> Hi,
>
>> Is the plan to keep this doc synced with the one in the virtio
>> specification?
>
> Yes, of course. I'm still in progress of doing the beaurocratic stuff w/
> virtio-tc folks (ID registration, ...) - yet have to see whether they
> wanna add it to their spec documents ...
>
> BTW: if you feel, sometings not good w/ the current spec, please raise
> your voice now.
>
>> I think it's better to use u8 ot uint8_t here.Git grep told me the
>> former is more popular under Documentation/.
>
> thx, I'll fix that
>
>>> +- for version field currently only value 1 supported.
>>> +- the line names block holds a stream of zero-terminated strings,
>>> +  holding the individual line names.
>> 
>> I'm not sure but does this mean we don't have a fixed length of config
>> space? Need to check whether it can bring any trouble to
>> migration(compatibility).
>
> Yes, it depends on how many gpio lines are present and how much space
> their names take up.
>
> A fixed size would either put unpleasent limits on the max number of
> lines or waste a lot space when only few lines present.
>
> Not that virtio-gpio is also meant for small embedded workloads running
> under some hypervisor.
>
>>> +- unspecified fields are reserved for future use and should be zero.
>>> +
>>> +------------------------
>>> +Virtqueues and messages:
>>> +------------------------
>>> +
>>> +- Queue #0: transmission from host to guest
>>> +- Queue #1: transmission from guest to host
>> 
>> 
>> Virtio became more a popular in the area without virtualization. So I
>> think it's better to use "device/driver" instead of "host/guest" here.
>
> Good point. But I'd prefer "cpu" instead of "driver" in that case.

I think you are going to tie yourself up in knots if you don't move this
to the OASIS spec. The reason being the VirtIO spec has definitions for
what a "Device" and a "Driver" is that are clear and unambiguous. The
upstream spec should be considered the canonical source of truth for any
implementation (Linux or otherwise).

By all means have the distilled documentation for the driver in the
kernel source tree but trying to upstream an implementation before
starting the definition in the standard is a little back to front IMHO*.

* that's not to say these things can't be done in parallel as the spec
  is reviewed and worked on and the kinks worked out but you want the
  final order of upstreaming to start with the spec.
Viresh Kumar May 24, 2021, 11:27 a.m. UTC | #32
On Fri, Dec 4, 2020 at 12:51 AM Enrico Weigelt, metux IT consult
<info@metux.net> wrote:
>
> Introducing new gpio driver for virtual GPIO devices via virtio.
>
> The driver allows routing gpio control into VM guests, eg. brigding
> virtual gpios to specific host gpios, or attaching simulators for
> automatic application testing.
>
> Changes v2:
>     * fixed uapi header license
>     * sorted include's
>     * fixed formatting
>     * fixed unneeded devm allocation - plain kzalloc/kfree is enough
>     * fixed missing devm_kzalloc fail check
>     * use devm_kcalloc() for array allocation
>     * added virtio-gpio protocol specification

Hi Enrico,

We (Linaro's Project Stratos
https://linaro.atlassian.net/wiki/spaces/STR/overview)
 are interested in this stuff. I was trying to look at the last status
of all this. Few
questions for you:

- Was the spec ever posted to virtio-dev list ? I thought that's the
very first step before
we merge the code.

- Any follow up on this patchset ?

Thanks. I will be happy to help otherwise and have cycles to work on
this if you need my help.

--
viresh
Enrico Weigelt, metux IT consult May 25, 2021, 12:59 p.m. UTC | #33
On 24.05.21 13:27, Viresh Kumar wrote:

Hi,


> We (Linaro's Project Stratos
> https://linaro.atlassian.net/wiki/spaces/STR/overview)
>   are interested in this stuff. I was trying to look at the last status
> of all this. Few
> questions for you:
> 
> - Was the spec ever posted to virtio-dev list ? I thought that's the
> very first step before
> we merge the code.

I had posted some spec quite some time ago, but it wasn't in the form
of patches against the .tex documentation files yet. It's been laying
aside for quite a while, since I've been busy w/ other things.


--mtx
Viresh Kumar May 26, 2021, 3:32 a.m. UTC | #34
On 25-05-21, 14:59, Enrico Weigelt, metux IT consult wrote:
> On 24.05.21 13:27, Viresh Kumar wrote:
> 
> Hi,
> 
> 
> > We (Linaro's Project Stratos
> > https://linaro.atlassian.net/wiki/spaces/STR/overview)
> >   are interested in this stuff. I was trying to look at the last status
> > of all this. Few
> > questions for you:
> > 
> > - Was the spec ever posted to virtio-dev list ? I thought that's the
> > very first step before
> > we merge the code.
> 
> I had posted some spec quite some time ago, but it wasn't in the form
> of patches against the .tex documentation files yet. It's been laying
> aside for quite a while, since I've been busy w/ other things.

Will you be fine if I take that up and restart the thread ?
Michael S. Tsirkin July 3, 2021, 8:05 a.m. UTC | #35
On Wed, May 26, 2021 at 09:02:06AM +0530, Viresh Kumar wrote:
> On 25-05-21, 14:59, Enrico Weigelt, metux IT consult wrote:
> > On 24.05.21 13:27, Viresh Kumar wrote:
> > 
> > Hi,
> > 
> > 
> > > We (Linaro's Project Stratos
> > > https://linaro.atlassian.net/wiki/spaces/STR/overview)
> > >   are interested in this stuff. I was trying to look at the last status
> > > of all this. Few
> > > questions for you:
> > > 
> > > - Was the spec ever posted to virtio-dev list ? I thought that's the
> > > very first step before
> > > we merge the code.
> > 
> > I had posted some spec quite some time ago, but it wasn't in the form
> > of patches against the .tex documentation files yet. It's been laying
> > aside for quite a while, since I've been busy w/ other things.
> 
> Will you be fine if I take that up and restart the thread ?

It's been a while - why not right?

> -- 
> viresh
Viresh Kumar July 5, 2021, 3:51 a.m. UTC | #36
On 03-07-21, 04:05, Michael S. Tsirkin wrote:
> On Wed, May 26, 2021 at 09:02:06AM +0530, Viresh Kumar wrote:
> > On 25-05-21, 14:59, Enrico Weigelt, metux IT consult wrote:
> > > On 24.05.21 13:27, Viresh Kumar wrote:
> > > 
> > > Hi,
> > > 
> > > 
> > > > We (Linaro's Project Stratos
> > > > https://linaro.atlassian.net/wiki/spaces/STR/overview)
> > > >   are interested in this stuff. I was trying to look at the last status
> > > > of all this. Few
> > > > questions for you:
> > > > 
> > > > - Was the spec ever posted to virtio-dev list ? I thought that's the
> > > > very first step before
> > > > we merge the code.
> > > 
> > > I had posted some spec quite some time ago, but it wasn't in the form
> > > of patches against the .tex documentation files yet. It's been laying
> > > aside for quite a while, since I've been busy w/ other things.
> > 
> > Will you be fine if I take that up and restart the thread ?
> 
> It's been a while - why not right?

Yeah, we went past that and here is the last version I posted.

https://lists.oasis-open.org/archives/virtio-comment/202106/msg00033.html

which I took back later on to let Enrico do it, as he wanted to.

And here is the last version from Enrico:

https://lists.oasis-open.org/archives/virtio-comment/202106/msg00048.html

Lets see how this goes in coming days.
diff mbox series

Patch

diff --git a/Documentation/gpio/virtio-gpio.rst b/Documentation/gpio/virtio-gpio.rst
new file mode 100644
index 000000000000..04642be07b96
--- /dev/null
+++ b/Documentation/gpio/virtio-gpio.rst
@@ -0,0 +1,176 @@ 
+"""""""""""""""""
+Virtio-GPIO protocol specification
+"""""""""""""""""
+...........
+Specification for virtio-based virtiual GPIO devices
+...........
+
++------------
++Version_ 1.0
++------------
+
+===================
+General
+===================
+
+The virtio-gpio protocol provides access to general purpose IO devices
+to virtual machine guests. These virtualized GPIOs could be either provided
+by some simulator (eg. virtual HIL), routed to some external device or
+routed to real GPIOs on the host (eg. virtualized embedded applications).
+
+Instead of simulating some existing real GPIO chip within an VMM, this
+protocol provides an hardware independent interface between host and guest
+that solely relies on an active virtio connection (no matter which transport
+actually used), no other buses or additional platform driver logic required.
+
+===================
+Protocol layout
+===================
+
+----------------------
+Configuration space
+----------------------
+
++--------+----------+-------------------------------+
+| Offset | Type     | Description                   |
++========+==========+===============================+
+| 0x00   | uint8    | version                       |
++--------+----------+-------------------------------+
+| 0x02   | uint16   | number of GPIO lines          |
++--------+----------+-------------------------------+
+| 0x04   | uint32   | size of gpio name block       |
++--------+----------+-------------------------------+
+| 0x20   | char[32] | device name (0-terminated)    |
++--------+----------+-------------------------------+
+| 0x40   | char[]   | line names block              |
++--------+----------+-------------------------------+
+
+- for version field currently only value 1 supported.
+- the line names block holds a stream of zero-terminated strings,
+  holding the individual line names.
+- unspecified fields are reserved for future use and should be zero.
+
+------------------------
+Virtqueues and messages:
+------------------------
+
+- Queue #0: transmission from host to guest
+- Queue #1: transmission from guest to host
+
+The queues transport messages of the struct virtio_gpio_event:
+
+Message format:
+---------------
+
++--------+----------+---------------+
+| Offset | Type     | Description   |
++========+==========+===============+
+| 0x00   | uint16   | event type    |
++--------+----------+---------------+
+| 0x02   | uint16   | line id       |
++--------+----------+---------------+
+| 0x04   | uint32   | value         |
++--------+----------+---------------+
+
+Message types:
+--------------
+
++-------+---------------------------------------+-----------------------------+
+| Code  | Symbol                                |                             |
++=======+=======================================+=============================+
+| 0x01  | VIRTIO_GPIO_EV_GUEST_REQUEST          | request gpio line           |
++-------+---------------------------------------+-----------------------------+
+| 0x02  | VIRTIO_GPIO_EV_GUEST_DIRECTION_INPUT  | set direction to input      |
++-------+---------------------------------------+-----------------------------+
+| 0x03  | VIRTIO_GPIO_EV_GUEST_DIRECTION_OUTPUT | set direction to output     |
++-------+---------------------------------------+-----------------------------+
+| 0x04  | VIRTIO_GPIO_EV_GUEST_GET_DIRECTION    | read current direction      |
++-------+---------------------------------------+-----------------------------+
+| 0x05  | VIRTIO_GPIO_EV_GUEST_GET_VALUE        | read current level          |
++-------+---------------------------------------+-----------------------------+
+| 0x06  | VIRTIO_GPIO_EV_GUEST_SET_VALUE        | set current (out) level     |
++-------+---------------------------------------+-----------------------------+
+| 0x11  | VIRTIO_GPIO_EV_HOST_LEVEL             | state changed (host->guest) |
++-------+---------------------------------------+-----------------------------+
+
+----------------------
+Data flow:
+----------------------
+
+- all operations, except ``VIRTIO_GPIO_EV_HOST_LEVEL``, are guest-initiated
+- host replies ``VIRTIO_GPIO_EV_HOST_LEVEL`` OR'ed to the ``type`` field
+- ``VIRTIO_GPIO_EV_HOST_LEVEL`` is only sent asynchronically from host to guest
+- in replies, a negative ``value`` field denotes an unix-style errno code
+- valid direction values are:
+  * 0 = output
+  * 1 = input
+- valid line state values are:
+  * 0 = inactive
+  * 1 = active
+
+VIRTIO_GPIO_EV_GUEST_REQUEST
+----------------------------
+
+- notify the host that given line# is going to be used
+- request:
+  * ``line`` field: line number
+  * ``value`` field: unused
+- reply:
+  * ``value`` field: errno code (0 = success)
+
+VIRTIO_GPIO_EV_GUEST_DIRECTION_INPUT
+------------------------------------
+
+- set line line direction to input
+- request:
+  * ``line`` field: line number
+  * ``value`` field: unused
+- reply: value field holds errno
+  * ``value`` field: errno code (0 = success)
+
+VIRTIO_GPIO_EV_GUEST_DIRECTION_OUTPUT
+-------------------------------------
+
+- set line direction to output and given line state
+- request:
+  * ``line`` field: line number
+  * ``value`` field: output state (0=inactive, 1=active)
+- reply:
+  * ``value`` field: holds errno
+
+VIRTIO_GPIO_EV_GUEST_GET_DIRECTION
+----------------------------------
+
+- retrieve line direction
+- request:
+  * ``line`` field: line number
+  * ``value`` field: unused
+- reply:
+  * ``value`` field: direction (0=output, 1=input) or errno code
+
+VIRTIO_GPIO_EV_GUEST_GET_VALUE
+------------------------------
+
+- retrieve line state value
+- request:
+  * ``line`` field: line number
+  * ``value`` field: unused
+- reply:
+  * ``value`` field: line state (0=inactive, 1=active) or errno code
+
+VIRTIO_GPIO_EV_GUEST_SET_VALUE
+------------------------------
+
+- set line state value (output only)
+- request:
+  * ``line`` field: line number
+  * ``value`` field: line state (0=inactive, 1=active)
+- reply:
+  * ``value`` field: new line state or errno code
+
+VIRTIO_GPIO_EV_HOST_LEVEL
+-------------------------
+
+- async notification from host to gues: line state changed
+- ``line`` field: line number
+- ``value`` field: new line state (0=inactive, 1=active)
diff --git a/MAINTAINERS b/MAINTAINERS
index 2daa6ee673f7..2b74e39275b3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18592,6 +18592,12 @@  F:	Documentation/filesystems/virtiofs.rst
 F:	fs/fuse/virtio_fs.c
 F:	include/uapi/linux/virtio_fs.h
 
+VIRTIO GPIO DRIVER
+M:	Enrico Weigelt, metux IT consult <info@metux.net>
+S:	Maintained
+F:	drivers/gpio/gpio-virtio.c
+F:	include/uapi/linux/virtio_gpio.h
+
 VIRTIO GPU DRIVER
 M:	David Airlie <airlied@linux.ie>
 M:	Gerd Hoffmann <kraxel@redhat.com>
diff --git a/drivers/gpio/Kconfig b/drivers/gpio/Kconfig
index 01619eb58396..7a33aa347dfb 100644
--- a/drivers/gpio/Kconfig
+++ b/drivers/gpio/Kconfig
@@ -1615,6 +1615,15 @@  config GPIO_MOCKUP
 	  tools/testing/selftests/gpio/gpio-mockup.sh. Reference the usage in
 	  it.
 
+config GPIO_VIRTIO
+	tristate "VirtIO GPIO support"
+	depends on VIRTIO
+	help
+	  Say Y here to enable guest support for virtio-based GPIOs.
+
+	  These virtual GPIOs can be routed to real GPIOs or attached to
+	  simulators on the host (qemu).
+
 endmenu
 
 endif
diff --git a/drivers/gpio/Makefile b/drivers/gpio/Makefile
index 09dada80ac34..2b12e75af123 100644
--- a/drivers/gpio/Makefile
+++ b/drivers/gpio/Makefile
@@ -160,6 +160,7 @@  obj-$(CONFIG_GPIO_TWL4030)		+= gpio-twl4030.o
 obj-$(CONFIG_GPIO_TWL6040)		+= gpio-twl6040.o
 obj-$(CONFIG_GPIO_UCB1400)		+= gpio-ucb1400.o
 obj-$(CONFIG_GPIO_UNIPHIER)		+= gpio-uniphier.o
+obj-$(CONFIG_GPIO_VIRTIO)		+= gpio-virtio.o
 obj-$(CONFIG_GPIO_VF610)		+= gpio-vf610.o
 obj-$(CONFIG_GPIO_VIPERBOARD)		+= gpio-viperboard.o
 obj-$(CONFIG_GPIO_VR41XX)		+= gpio-vr41xx.o
diff --git a/drivers/gpio/gpio-virtio.c b/drivers/gpio/gpio-virtio.c
new file mode 100644
index 000000000000..f1ac47da26b6
--- /dev/null
+++ b/drivers/gpio/gpio-virtio.c
@@ -0,0 +1,332 @@ 
+// SPDX-License-Identifier: GPL-2.0+
+
+/*
+ * GPIO driver for virtio-based virtual GPIOs
+ *
+ * Copyright (C) 2018 metux IT consult
+ * Author: Enrico Weigelt, metux IT consult <info@metux.net>
+ *
+ */
+
+#include <linux/err.h>
+#include <linux/gpio/driver.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/spinlock.h>
+#include <linux/virtio_config.h>
+#include <uapi/linux/virtio_ids.h>
+#include <uapi/linux/virtio_gpio.h>
+
+struct virtio_gpio_priv {
+	struct gpio_chip gc;
+	spinlock_t vq_lock;
+	spinlock_t op_lock;
+	struct virtio_device *vdev;
+	int num_gpios;
+	char *name;
+	struct virtqueue *vq_rx;
+	struct virtqueue *vq_tx;
+	struct virtio_gpio_event rcv_buf;
+	struct virtio_gpio_event last;
+	int irq_base;
+	wait_queue_head_t waitq;
+	unsigned long reply_wait;
+};
+
+static void virtio_gpio_prepare_inbuf(struct virtio_gpio_priv *priv)
+{
+	struct scatterlist rcv_sg;
+
+	sg_init_one(&rcv_sg, &priv->rcv_buf, sizeof(priv->rcv_buf));
+	virtqueue_add_inbuf(priv->vq_rx, &rcv_sg, 1, &priv->rcv_buf,
+			    GFP_KERNEL);
+	virtqueue_kick(priv->vq_rx);
+}
+
+static int virtio_gpio_xmit(struct virtio_gpio_priv *priv, int type,
+			    int pin, int value, struct virtio_gpio_event *ev)
+{
+	struct scatterlist sg[1];
+	int ret;
+	unsigned long flags;
+
+	WARN_ON(!ev);
+
+	ev->type = type;
+	ev->pin = pin;
+	ev->value = value;
+
+	sg_init_table(sg, 1);
+	sg_set_buf(&sg[0], ev, sizeof(struct virtio_gpio_event));
+
+	spin_lock_irqsave(&priv->vq_lock, flags);
+	ret = virtqueue_add_outbuf(priv->vq_tx, sg, ARRAY_SIZE(sg),
+				   priv, GFP_KERNEL);
+	if (ret < 0) {
+		dev_err(&priv->vdev->dev,
+			"virtqueue_add_outbuf() failed: %d\n", ret);
+		goto out;
+	}
+	virtqueue_kick(priv->vq_tx);
+
+out:
+	spin_unlock_irqrestore(&priv->vq_lock, flags);
+	return 0;
+}
+
+static inline void wakeup_event(struct virtio_gpio_priv *priv, int id)
+{
+	set_bit(id, &priv->reply_wait);
+}
+
+static inline int check_event(struct virtio_gpio_priv *priv, int id)
+{
+	return test_bit(id, &priv->reply_wait);
+}
+
+static inline void clear_event(struct virtio_gpio_priv *priv, int id)
+{
+	clear_bit(id, &priv->reply_wait);
+}
+
+static int virtio_gpio_req(struct virtio_gpio_priv *priv, int type,
+			   int pin, int value)
+{
+	struct virtio_gpio_event *ev
+		= kzalloc(&priv->vdev->dev, sizeof(struct virtio_gpio_event),
+			  GFP_KERNEL);
+
+	if (!ev)
+		return -ENOMEM;
+
+	clear_event(priv, type);
+	virtio_gpio_xmit(priv, type, pin, value, ev);
+	wait_event_interruptible(priv->waitq, check_event(priv, type));
+
+	kfree(&priv->vdev->dev, ev);
+
+	return priv->last.value;
+}
+
+static int virtio_gpio_direction_input(struct gpio_chip *gc,
+				       unsigned int pin)
+{
+	return virtio_gpio_req(gpiochip_get_data(gc),
+			       VIRTIO_GPIO_EV_GUEST_DIRECTION_INPUT,
+			       pin, 0);
+}
+
+static int virtio_gpio_direction_output(struct gpio_chip *gc,
+					unsigned int pin, int value)
+{
+	return virtio_gpio_req(gpiochip_get_data(gc),
+			       VIRTIO_GPIO_EV_GUEST_DIRECTION_OUTPUT,
+			       pin, value);
+}
+
+static int virtio_gpio_get_direction(struct gpio_chip *gc, unsigned int pin)
+{
+	return virtio_gpio_req(gpiochip_get_data(gc),
+			       VIRTIO_GPIO_EV_GUEST_GET_DIRECTION,
+			       pin, 0);
+}
+
+static void virtio_gpio_set(struct gpio_chip *gc,
+			    unsigned int pin, int value)
+{
+	virtio_gpio_req(gpiochip_get_data(gc),
+			VIRTIO_GPIO_EV_GUEST_SET_VALUE, pin, value);
+}
+
+static int virtio_gpio_get(struct gpio_chip *gc,
+			   unsigned int pin)
+{
+	return virtio_gpio_req(gpiochip_get_data(gc),
+			       VIRTIO_GPIO_EV_GUEST_GET_VALUE, pin, 0);
+}
+
+static int virtio_gpio_request(struct gpio_chip *gc,
+			       unsigned int pin)
+{
+	return virtio_gpio_req(gpiochip_get_data(gc),
+			       VIRTIO_GPIO_EV_GUEST_REQUEST, pin, 0);
+}
+
+static void virtio_gpio_signal(struct virtio_gpio_priv *priv, int event,
+			      int pin, int value)
+{
+	if (pin < priv->num_gpios)
+		generic_handle_irq(priv->irq_base + pin);
+}
+
+static void virtio_gpio_data_rx(struct virtqueue *vq)
+{
+	struct virtio_gpio_priv *priv = vq->vdev->priv;
+	void *data;
+	unsigned int len;
+	struct virtio_gpio_event *ev;
+
+	data = virtqueue_get_buf(priv->vq_rx, &len);
+	if (!data || !len) {
+		dev_warn(&vq->vdev->dev, "RX received no data ! %d\n", len);
+		return;
+	}
+
+	ev = data;
+	WARN_ON(data != &priv->rcv_buf);
+
+	memcpy(&priv->last, &priv->rcv_buf, sizeof(struct virtio_gpio_event));
+
+	switch (ev->type) {
+	case VIRTIO_GPIO_EV_HOST_LEVEL:
+		virtio_gpio_signal(priv, ev->type, ev->pin, ev->value);
+		break;
+	default:
+		wakeup_event(priv, ev->type & ~VIRTIO_GPIO_EV_REPLY);
+		break;
+	}
+	virtio_gpio_prepare_inbuf(priv);
+	wake_up_all(&priv->waitq);
+}
+
+static int virtio_gpio_alloc_vq(struct virtio_gpio_priv *priv)
+{
+	struct virtqueue *vqs[2];
+	vq_callback_t *cbs[] = {
+		NULL,
+		virtio_gpio_data_rx,
+	};
+	static const char * const names[] = { "in", "out", };
+	int ret;
+
+	ret = virtio_find_vqs(priv->vdev, 2, vqs, cbs, names, NULL);
+	if (ret) {
+		dev_err(&priv->vdev->dev, "failed to alloc vqs: %d\n", ret);
+		return ret;
+	}
+
+	priv->vq_rx = vqs[0];
+	priv->vq_tx = vqs[1];
+
+	virtio_gpio_prepare_inbuf(priv);
+
+	virtio_config_enable(priv->vdev);
+	virtqueue_enable_cb(priv->vq_rx);
+	virtio_device_ready(priv->vdev);
+
+	return 0;
+}
+
+static int virtio_gpio_probe(struct virtio_device *vdev)
+{
+	struct virtio_gpio_priv *priv;
+	struct virtio_gpio_config cf = {};
+	char *name_buffer;
+	const char **gpio_names = NULL;
+	struct device *dev = &vdev->dev;
+
+	priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+	if (!priv)
+		return -ENOMEM;
+
+	priv->name = devm_kzalloc(dev, sizeof(cf.name)+1, GFP_KERNEL);
+	if (!priv->name)
+		return -ENOMEM;
+
+	spin_lock_init(&priv->vq_lock);
+	spin_lock_init(&priv->op_lock);
+
+	virtio_cread(vdev, struct virtio_gpio_config, version, &cf.version);
+	virtio_cread(vdev, struct virtio_gpio_config, num_gpios,
+		     &cf.num_gpios);
+	virtio_cread(vdev, struct virtio_gpio_config, names_size,
+		     &cf.names_size);
+	virtio_cread_bytes(vdev, offsetof(struct virtio_gpio_config, name),
+			   priv->name, sizeof(cf.name));
+
+	if (cf.version != 1) {
+		dev_err(dev, "unsupported interface version %d\n", cf.version);
+		return -EINVAL;
+	}
+
+	priv->num_gpios = cf.num_gpios;
+
+	if (cf.names_size) {
+		char *bufwalk;
+		int idx = 0;
+
+		name_buffer = devm_kzalloc(&vdev->dev, cf.names_size,
+					   GFP_KERNEL)+1;
+		virtio_cread_bytes(vdev, sizeof(struct virtio_gpio_config),
+				   name_buffer, cf.names_size);
+		name_buffer[cf.names_size] = 0;
+
+		gpio_names = devm_kcalloc(dev, priv->num_gpios, sizeof(char *),
+					  GFP_KERNEL);
+		bufwalk = name_buffer;
+
+		while (idx < priv->num_gpios &&
+		       bufwalk < (name_buffer+cf.names_size)) {
+			gpio_names[idx] = (strlen(bufwalk) ? bufwalk : NULL);
+			bufwalk += strlen(bufwalk)+1;
+			idx++;
+		}
+	}
+
+	priv->gc.owner			= THIS_MODULE;
+	priv->gc.parent			= dev;
+	priv->gc.label			= (priv->name[0] ? priv->name
+							 : dev_name(dev));
+	priv->gc.ngpio			= priv->num_gpios;
+	priv->gc.names			= gpio_names;
+	priv->gc.base			= -1;
+	priv->gc.request		= virtio_gpio_request;
+	priv->gc.direction_input	= virtio_gpio_direction_input;
+	priv->gc.direction_output	= virtio_gpio_direction_output;
+	priv->gc.get_direction		= virtio_gpio_get_direction;
+	priv->gc.get			= virtio_gpio_get;
+	priv->gc.set			= virtio_gpio_set;
+
+	priv->vdev			= vdev;
+	vdev->priv = priv;
+
+	priv->irq_base = devm_irq_alloc_descs(dev, -1, 0, priv->num_gpios,
+					      NUMA_NO_NODE);
+	if (priv->irq_base < 0) {
+		dev_err(&vdev->dev, "failed to alloc irqs\n");
+		priv->irq_base = -1;
+		return -ENOMEM;
+	}
+
+	init_waitqueue_head(&priv->waitq);
+
+	priv->reply_wait = 0;
+
+	virtio_gpio_alloc_vq(priv);
+
+	return devm_gpiochip_add_data(dev, &priv->gc, priv);
+}
+
+static void virtio_gpio_remove(struct virtio_device *vdev)
+{
+}
+
+static const struct virtio_device_id id_table[] = {
+	{ VIRTIO_ID_GPIO, VIRTIO_DEV_ANY_ID },
+	{ 0 },
+};
+
+static struct virtio_driver virtio_gpio_driver = {
+	.driver.name	= KBUILD_MODNAME,
+	.driver.owner	= THIS_MODULE,
+	.id_table	= id_table,
+	.probe		= virtio_gpio_probe,
+	.remove		= virtio_gpio_remove,
+};
+
+module_virtio_driver(virtio_gpio_driver);
+
+MODULE_AUTHOR("Enrico Weigelt, metux IT consult <info@metux.net>");
+MODULE_DESCRIPTION("VirtIO GPIO driver");
+MODULE_LICENSE("GPL");
diff --git a/include/uapi/linux/virtio_gpio.h b/include/uapi/linux/virtio_gpio.h
new file mode 100644
index 000000000000..438b49f04505
--- /dev/null
+++ b/include/uapi/linux/virtio_gpio.h
@@ -0,0 +1,39 @@ 
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+
+#ifndef _LINUX_VIRTIO_GPIO_H
+#define _LINUX_VIRTIO_GPIO_H
+
+#include <linux/types.h>
+
+enum virtio_gpio_event_type {
+	// requests from quest to host
+	VIRTIO_GPIO_EV_GUEST_REQUEST		= 0x01,	// ->request()
+	VIRTIO_GPIO_EV_GUEST_DIRECTION_INPUT	= 0x02,	// ->direction_input()
+	VIRTIO_GPIO_EV_GUEST_DIRECTION_OUTPUT	= 0x03,	// ->direction_output()
+	VIRTIO_GPIO_EV_GUEST_GET_DIRECTION	= 0x04,	// ->get_direction()
+	VIRTIO_GPIO_EV_GUEST_GET_VALUE		= 0x05,	// ->get_value()
+	VIRTIO_GPIO_EV_GUEST_SET_VALUE		= 0x06,	// ->set_value()
+
+	// messages from host to guest
+	VIRTIO_GPIO_EV_HOST_LEVEL		= 0x11,	// gpio state changed
+
+	/* mask bit set on host->guest reply */
+	VIRTIO_GPIO_EV_REPLY			= 0xF000,
+};
+
+struct virtio_gpio_config {
+	__u8    version;
+	__u8    reserved0;
+	__u16   num_gpios;
+	__u32   names_size;
+	__u8    reserved1[24];
+	__u8    name[32];
+};
+
+struct virtio_gpio_event {
+	__le16 type;
+	__le16 pin;
+	__le32 value;
+};
+
+#endif /* _LINUX_VIRTIO_GPIO_H */
diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h
index b052355ac7a3..85772c0bcb4b 100644
--- a/include/uapi/linux/virtio_ids.h
+++ b/include/uapi/linux/virtio_ids.h
@@ -48,5 +48,6 @@ 
 #define VIRTIO_ID_FS           26 /* virtio filesystem */
 #define VIRTIO_ID_PMEM         27 /* virtio pmem */
 #define VIRTIO_ID_MAC80211_HWSIM 29 /* virtio mac80211-hwsim */
+#define VIRTIO_ID_GPIO           30 /* virtio GPIO */
 
 #endif /* _LINUX_VIRTIO_IDS_H */