From patchwork Fri Jan 29 17:24:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vikas Gupta X-Patchwork-Id: 12056213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MIME_HEADER_CTYPE_ONLY, SPF_HELO_NONE,SPF_PASS,T_TVD_MIME_NO_HEADERS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 285C7C433DB for ; Fri, 29 Jan 2021 17:25:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D80DD64E02 for ; Fri, 29 Jan 2021 17:25:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232492AbhA2RZt (ORCPT ); Fri, 29 Jan 2021 12:25:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50338 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232418AbhA2RZV (ORCPT ); Fri, 29 Jan 2021 12:25:21 -0500 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B411FC06174A for ; Fri, 29 Jan 2021 09:24:41 -0800 (PST) Received: by mail-pf1-x444.google.com with SMTP id y205so6635991pfc.5 for ; Fri, 29 Jan 2021 09:24:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=POtj3/lfdVAatOz+3UWgLmFqHV2ZKe94zk3hNOwyyoY=; b=RO8yD/jw7byCI2jWRY2k6IC0NgHPqt5aBtis+tMqm09Q/jbmT/BfmZevxZxSXm7xCH iZRLvUqmmvTHdDDdRQgBwXTRVJfgoSe5TdNdisT98Xss6zgtVikwV58fAYTazdP8uvMn vupXIy2hY7yQQzx+7Wc0thD/+nExU9w9mjGWw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=POtj3/lfdVAatOz+3UWgLmFqHV2ZKe94zk3hNOwyyoY=; b=ZyFh/l2Q0ePcGuvaTLSprGkOgx+Zbsc5Hxsnp7HPk3UFuFg7ckEyV9OasHFc5UhO01 ZM6wOiwxB5jdoy+7NXWb3YejyB8BqtrgFwke/ZH15yCbfT7hdZeAnWSihPd++7Qh2Qjd pb+9QC87s2nb73V3v4Au5WqFUPRKzNew0lvm9Gr8Btys6skWhjr/i/i1kFGl+nZl/3OM pVSkp0WGc0oE1YojCHrqTgeihmXWNf18aW70jIzMzGxaZlBMpkoQl4OCQ9XjlbRznUQi rRpg/dUl2TBQOHm9BuE8z/oQMmB8ZqBkFgTj7KwxkyvaRYIxovwCBLsXuPTS7ozN8Wyc YzLQ== X-Gm-Message-State: AOAM531izhsYNh8dHVOkg7nS4UsXyKq1oncaz/pL7UJb3aaQLYvaKiS+ scermWBPreqn/fMJuC6/EV01fg== X-Google-Smtp-Source: ABdhPJz1xHFhalJwgQpRsOh6aBRd1jc4JPmpAvYMGE0iehFeBEbbof5H6cBxTO+KDZYRinaQkBk3ZQ== X-Received: by 2002:a62:5881:0:b029:1c9:d72d:34d5 with SMTP id m123-20020a6258810000b02901c9d72d34d5mr5365191pfb.68.1611941080824; Fri, 29 Jan 2021 09:24:40 -0800 (PST) Received: from rahul_yocto_ubuntu18.ibn.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id w11sm9739016pge.28.2021.01.29.09.24.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jan 2021 09:24:40 -0800 (PST) From: Vikas Gupta To: eric.auger@redhat.com, alex.williamson@redhat.com, cohuck@redhat.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: vikram.prakash@broadcom.com, srinath.mannam@broadcom.com, ashwin.kamath@broadcom.com, zachary.schroff@broadcom.com, manish.kurup@broadcom.com, Vikas Gupta Subject: [RFC v4 1/3] vfio/platform: add support for msi Date: Fri, 29 Jan 2021 22:54:19 +0530 Message-Id: <20210129172421.43299-2-vikas.gupta@broadcom.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210129172421.43299-1-vikas.gupta@broadcom.com> References: <20201214174514.22006-1-vikas.gupta@broadcom.com> <20210129172421.43299-1-vikas.gupta@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org MSI support for platform devices. MSI is added as a single 'index' with 'count' as the number of MSI(s) supported by the devices. Signed-off-by: Vikas Gupta --- drivers/vfio/platform/Kconfig | 1 + drivers/vfio/platform/vfio_platform_common.c | 95 ++++++- drivers/vfio/platform/vfio_platform_irq.c | 253 ++++++++++++++++-- drivers/vfio/platform/vfio_platform_private.h | 29 ++ include/uapi/linux/vfio.h | 24 ++ 5 files changed, 373 insertions(+), 29 deletions(-) diff --git a/drivers/vfio/platform/Kconfig b/drivers/vfio/platform/Kconfig index dc1a3c44f2c6..d4bbc9f27763 100644 --- a/drivers/vfio/platform/Kconfig +++ b/drivers/vfio/platform/Kconfig @@ -3,6 +3,7 @@ config VFIO_PLATFORM tristate "VFIO support for platform devices" depends on VFIO && EVENTFD && (ARM || ARM64) select VFIO_VIRQFD + select GENERIC_MSI_IRQ_DOMAIN help Support for platform devices with VFIO. This is required to make use of platform devices present on the system using the VFIO diff --git a/drivers/vfio/platform/vfio_platform_common.c b/drivers/vfio/platform/vfio_platform_common.c index fb4b385191f2..f2b1f0c3bfcc 100644 --- a/drivers/vfio/platform/vfio_platform_common.c +++ b/drivers/vfio/platform/vfio_platform_common.c @@ -16,6 +16,7 @@ #include #include #include +#include #include "vfio_platform_private.h" @@ -28,23 +29,22 @@ static LIST_HEAD(reset_list); static DEFINE_MUTEX(driver_lock); -static vfio_platform_reset_fn_t vfio_platform_lookup_reset(const char *compat, - struct module **module) +static void vfio_platform_lookup_reset(const char *compat, + struct module **module, + struct vfio_platform_reset_node **node) { struct vfio_platform_reset_node *iter; - vfio_platform_reset_fn_t reset_fn = NULL; mutex_lock(&driver_lock); list_for_each_entry(iter, &reset_list, link) { if (!strcmp(iter->compat, compat) && try_module_get(iter->owner)) { *module = iter->owner; - reset_fn = iter->of_reset; + *node = iter; break; } } mutex_unlock(&driver_lock); - return reset_fn; } static int vfio_platform_acpi_probe(struct vfio_platform_device *vdev, @@ -112,15 +112,23 @@ static bool vfio_platform_has_reset(struct vfio_platform_device *vdev) static int vfio_platform_get_reset(struct vfio_platform_device *vdev) { + struct vfio_platform_reset_node *node = NULL; + if (VFIO_PLATFORM_IS_ACPI(vdev)) return vfio_platform_acpi_has_reset(vdev) ? 0 : -ENOENT; - vdev->of_reset = vfio_platform_lookup_reset(vdev->compat, - &vdev->reset_module); - if (!vdev->of_reset) { + vfio_platform_lookup_reset(vdev->compat, &vdev->reset_module, + &node); + if (!node) { request_module("vfio-reset:%s", vdev->compat); - vdev->of_reset = vfio_platform_lookup_reset(vdev->compat, - &vdev->reset_module); + vfio_platform_lookup_reset(vdev->compat, &vdev->reset_module, + &node); + } + + if (node) { + vdev->of_reset = node->of_reset; + vdev->of_get_msi = node->of_get_msi; + vdev->of_msi_write = node->of_msi_write; } return vdev->of_reset ? 0 : -ENOENT; @@ -343,9 +351,16 @@ static long vfio_platform_ioctl(void *device_data, } else if (cmd == VFIO_DEVICE_GET_IRQ_INFO) { struct vfio_irq_info info; + struct vfio_info_cap caps = { .buf = NULL, .size = 0 }; + int ext_irq_index = vdev->num_irqs - vdev->num_ext_irqs; + unsigned long capsz; + u32 index; minsz = offsetofend(struct vfio_irq_info, count); + /* For backward compatibility, cannot require this */ + capsz = offsetofend(struct vfio_irq_info, cap_offset); + if (copy_from_user(&info, (void __user *)arg, minsz)) return -EFAULT; @@ -355,8 +370,53 @@ static long vfio_platform_ioctl(void *device_data, if (info.index >= vdev->num_irqs) return -EINVAL; - info.flags = vdev->irqs[info.index].flags; - info.count = vdev->irqs[info.index].count; + if (info.argsz >= capsz) + minsz = capsz; + + index = info.index; + + info.flags = vdev->irqs[index].flags; + info.count = vdev->irqs[index].count; + + if (ext_irq_index - index == VFIO_EXT_IRQ_MSI) { + struct vfio_irq_info_cap_type cap_type = { + .header.id = VFIO_IRQ_INFO_CAP_TYPE, + .header.version = 1 }; + struct vfio_platform_irq *irq; + int ret; + + index = array_index_nospec(index, + vdev->num_irqs); + irq = &vdev->irqs[index]; + + cap_type.type = irq->type; + cap_type.subtype = irq->subtype; + + ret = vfio_info_add_capability(&caps, + &cap_type.header, + sizeof(cap_type)); + if (ret) + return ret; + } + + if (caps.size) { + info.flags |= VFIO_IRQ_INFO_FLAG_CAPS; + if (info.argsz < sizeof(info) + caps.size) { + info.argsz = sizeof(info) + caps.size; + info.cap_offset = 0; + } else { + vfio_info_cap_shift(&caps, sizeof(info)); + if (copy_to_user((void __user *)arg + + sizeof(info), caps.buf, + caps.size)) { + kfree(caps.buf); + return -EFAULT; + } + info.cap_offset = sizeof(info); + } + + kfree(caps.buf); + } return copy_to_user((void __user *)arg, &info, minsz) ? -EFAULT : 0; @@ -365,6 +425,7 @@ static long vfio_platform_ioctl(void *device_data, struct vfio_irq_set hdr; u8 *data = NULL; int ret = 0; + int max; size_t data_size = 0; minsz = offsetofend(struct vfio_irq_set, count); @@ -372,8 +433,14 @@ static long vfio_platform_ioctl(void *device_data, if (copy_from_user(&hdr, (void __user *)arg, minsz)) return -EFAULT; - ret = vfio_set_irqs_validate_and_prepare(&hdr, vdev->num_irqs, - vdev->num_irqs, &data_size); + if (hdr.index >= vdev->num_irqs) + return -EINVAL; + + max = vdev->irqs[hdr.index].count; + + ret = vfio_set_irqs_validate_and_prepare(&hdr, max, + vdev->num_irqs, + &data_size); if (ret) return ret; diff --git a/drivers/vfio/platform/vfio_platform_irq.c b/drivers/vfio/platform/vfio_platform_irq.c index c5b09ec0a3c9..db60240d27ca 100644 --- a/drivers/vfio/platform/vfio_platform_irq.c +++ b/drivers/vfio/platform/vfio_platform_irq.c @@ -8,10 +8,12 @@ #include #include +#include #include #include #include #include +#include #include "vfio_platform_private.h" @@ -253,6 +255,186 @@ static int vfio_platform_set_irq_trigger(struct vfio_platform_device *vdev, return 0; } +/* MSI/MSIX */ +static irqreturn_t vfio_msihandler(int irq, void *arg) +{ + struct eventfd_ctx *trigger = arg; + + eventfd_signal(trigger, 1); + return IRQ_HANDLED; +} + +static void msi_write(struct msi_desc *desc, struct msi_msg *msg) +{ + struct device *dev = msi_desc_to_dev(desc); + struct vfio_device *device = dev_get_drvdata(dev); + struct vfio_platform_device *vdev = (struct vfio_platform_device *) + vfio_device_data(device); + + vdev->of_msi_write(vdev, desc, msg); +} + +static int vfio_msi_enable(struct vfio_platform_device *vdev, + struct vfio_platform_irq *irq, int nvec) +{ + int ret; + int msi_idx = 0; + struct msi_desc *desc; + struct device *dev = vdev->device; + + irq->ctx = kcalloc(nvec, sizeof(struct vfio_irq_ctx), GFP_KERNEL); + if (!irq->ctx) + return -ENOMEM; + + /* Allocate platform MSIs */ + ret = platform_msi_domain_alloc_irqs(dev, nvec, msi_write); + if (ret < 0) { + kfree(irq->ctx); + return ret; + } + + for_each_msi_entry(desc, dev) { + irq->ctx[msi_idx].hwirq = desc->irq; + msi_idx++; + } + + irq->num_ctx = nvec; + irq->config_msi = 1; + + return 0; +} + +static int vfio_msi_set_vector_signal(struct vfio_platform_irq *irq, + int vector, int fd) +{ + struct eventfd_ctx *trigger; + int irq_num, ret; + + if (vector < 0 || vector >= irq->num_ctx) + return -EINVAL; + + irq_num = irq->ctx[vector].hwirq; + + if (irq->ctx[vector].trigger) { + free_irq(irq_num, irq->ctx[vector].trigger); + kfree(irq->ctx[vector].name); + eventfd_ctx_put(irq->ctx[vector].trigger); + irq->ctx[vector].trigger = NULL; + } + + if (fd < 0) + return 0; + + irq->ctx[vector].name = kasprintf(GFP_KERNEL, + "vfio-msi[%d]", vector); + if (!irq->ctx[vector].name) + return -ENOMEM; + + trigger = eventfd_ctx_fdget(fd); + if (IS_ERR(trigger)) { + kfree(irq->ctx[vector].name); + return PTR_ERR(trigger); + } + + ret = request_irq(irq_num, vfio_msihandler, 0, + irq->ctx[vector].name, trigger); + if (ret) { + kfree(irq->ctx[vector].name); + eventfd_ctx_put(trigger); + return ret; + } + + irq->ctx[vector].trigger = trigger; + + return 0; +} + +static int vfio_msi_set_block(struct vfio_platform_irq *irq, unsigned int start, + unsigned int count, int32_t *fds) +{ + int i, j, ret = 0; + + if (start >= irq->num_ctx || start + count > irq->num_ctx) + return -EINVAL; + + for (i = 0, j = start; i < count && !ret; i++, j++) { + int fd = fds ? fds[i] : -1; + + ret = vfio_msi_set_vector_signal(irq, j, fd); + } + + if (ret) { + for (--j; j >= (int)start; j--) + vfio_msi_set_vector_signal(irq, j, -1); + } + + return ret; +} + +static void vfio_msi_disable(struct vfio_platform_device *vdev, + struct vfio_platform_irq *irq) +{ + struct device *dev = vdev->device; + + vfio_msi_set_block(irq, 0, irq->num_ctx, NULL); + + platform_msi_domain_free_irqs(dev); + + irq->config_msi = 0; + irq->num_ctx = 0; + + kfree(irq->ctx); +} + +static int vfio_set_msi_trigger(struct vfio_platform_device *vdev, + unsigned int index, unsigned int start, + unsigned int count, uint32_t flags, void *data) +{ + int i; + struct vfio_platform_irq *irq = &vdev->irqs[index]; + + if (start + count > irq->count) + return -EINVAL; + + if (!count && (flags & VFIO_IRQ_SET_DATA_NONE)) { + vfio_msi_disable(vdev, irq); + return 0; + } + + if (flags & VFIO_IRQ_SET_DATA_EVENTFD) { + s32 *fds = data; + int ret; + + if (irq->config_msi) + return vfio_msi_set_block(irq, start, count, + fds); + ret = vfio_msi_enable(vdev, irq, start + count); + if (ret) + return ret; + + ret = vfio_msi_set_block(irq, start, count, fds); + if (ret) + vfio_msi_disable(vdev, irq); + + return ret; + } + + for (i = start; i < start + count; i++) { + if (!irq->ctx[i].trigger) + continue; + if (flags & VFIO_IRQ_SET_DATA_NONE) { + eventfd_signal(irq->ctx[i].trigger, 1); + } else if (flags & VFIO_IRQ_SET_DATA_BOOL) { + u8 *bools = data; + + if (bools[i - start]) + eventfd_signal(irq->ctx[i].trigger, 1); + } + } + + return 0; +} + int vfio_platform_set_irqs_ioctl(struct vfio_platform_device *vdev, uint32_t flags, unsigned index, unsigned start, unsigned count, void *data) @@ -261,16 +443,29 @@ int vfio_platform_set_irqs_ioctl(struct vfio_platform_device *vdev, unsigned start, unsigned count, uint32_t flags, void *data) = NULL; - switch (flags & VFIO_IRQ_SET_ACTION_TYPE_MASK) { - case VFIO_IRQ_SET_ACTION_MASK: - func = vfio_platform_set_irq_mask; - break; - case VFIO_IRQ_SET_ACTION_UNMASK: - func = vfio_platform_set_irq_unmask; - break; - case VFIO_IRQ_SET_ACTION_TRIGGER: - func = vfio_platform_set_irq_trigger; - break; + struct vfio_platform_irq *irq = &vdev->irqs[index]; + + if (irq->type == VFIO_IRQ_TYPE_MSI) { + switch (flags & VFIO_IRQ_SET_ACTION_TYPE_MASK) { + case VFIO_IRQ_SET_ACTION_MASK: + case VFIO_IRQ_SET_ACTION_UNMASK: + break; + case VFIO_IRQ_SET_ACTION_TRIGGER: + func = vfio_set_msi_trigger; + break; + } + } else { + switch (flags & VFIO_IRQ_SET_ACTION_TYPE_MASK) { + case VFIO_IRQ_SET_ACTION_MASK: + func = vfio_platform_set_irq_mask; + break; + case VFIO_IRQ_SET_ACTION_UNMASK: + func = vfio_platform_set_irq_unmask; + break; + case VFIO_IRQ_SET_ACTION_TRIGGER: + func = vfio_platform_set_irq_trigger; + break; + } } if (!func) @@ -281,12 +476,23 @@ int vfio_platform_set_irqs_ioctl(struct vfio_platform_device *vdev, int vfio_platform_irq_init(struct vfio_platform_device *vdev) { - int cnt = 0, i; + int i; + int cnt = 0; + int num_irqs = 0; + int msi_cnt = 0; while (vdev->get_irq(vdev, cnt) >= 0) cnt++; - vdev->irqs = kcalloc(cnt, sizeof(struct vfio_platform_irq), GFP_KERNEL); + num_irqs = cnt; + + if (vdev->of_get_msi) { + msi_cnt = vdev->of_get_msi(vdev); + num_irqs++; + } + + vdev->irqs = kcalloc(num_irqs, sizeof(struct vfio_platform_irq), + GFP_KERNEL); if (!vdev->irqs) return -ENOMEM; @@ -309,7 +515,19 @@ int vfio_platform_irq_init(struct vfio_platform_device *vdev) vdev->irqs[i].masked = false; } - vdev->num_irqs = cnt; + /* + * MSI block is added at last index and it is an ext irq + */ + if (msi_cnt > 0) { + vdev->irqs[i].flags = VFIO_IRQ_INFO_EVENTFD; + vdev->irqs[i].count = msi_cnt; + vdev->irqs[i].hwirq = 0; + vdev->irqs[i].masked = false; + vdev->irqs[i].type = VFIO_IRQ_TYPE_MSI; + vdev->num_ext_irqs = 1; + } + + vdev->num_irqs = num_irqs; return 0; err: @@ -321,8 +539,13 @@ void vfio_platform_irq_cleanup(struct vfio_platform_device *vdev) { int i; - for (i = 0; i < vdev->num_irqs; i++) - vfio_set_trigger(vdev, i, -1, NULL); + for (i = 0; i < vdev->num_irqs; i++) { + if (vdev->irqs[i].type == VFIO_IRQ_TYPE_MSI) + vfio_set_msi_trigger(vdev, i, 0, 0, + VFIO_IRQ_SET_DATA_NONE, NULL); + else + vfio_set_trigger(vdev, i, -1, NULL); + } vdev->num_irqs = 0; kfree(vdev->irqs); diff --git a/drivers/vfio/platform/vfio_platform_private.h b/drivers/vfio/platform/vfio_platform_private.h index 289089910643..b8dd892aec53 100644 --- a/drivers/vfio/platform/vfio_platform_private.h +++ b/drivers/vfio/platform/vfio_platform_private.h @@ -9,6 +9,7 @@ #include #include +#include #define VFIO_PLATFORM_OFFSET_SHIFT 40 #define VFIO_PLATFORM_OFFSET_MASK (((u64)(1) << VFIO_PLATFORM_OFFSET_SHIFT) - 1) @@ -19,9 +20,21 @@ #define VFIO_PLATFORM_INDEX_TO_OFFSET(index) \ ((u64)(index) << VFIO_PLATFORM_OFFSET_SHIFT) +/* IRQ index for MSI in ext IRQs */ +#define VFIO_EXT_IRQ_MSI 0 + +struct vfio_irq_ctx { + int hwirq; + char *name; + struct msi_msg msg; + struct eventfd_ctx *trigger; +}; + struct vfio_platform_irq { u32 flags; u32 count; + int num_ctx; + struct vfio_irq_ctx *ctx; int hwirq; char *name; struct eventfd_ctx *trigger; @@ -29,6 +42,11 @@ struct vfio_platform_irq { spinlock_t lock; struct virqfd *unmask; struct virqfd *mask; + + /* for extended irqs */ + u32 type; + u32 subtype; + int config_msi; }; struct vfio_platform_region { @@ -46,6 +64,7 @@ struct vfio_platform_device { u32 num_regions; struct vfio_platform_irq *irqs; u32 num_irqs; + int num_ext_irqs; int refcnt; struct mutex igate; struct module *parent_module; @@ -65,17 +84,27 @@ struct vfio_platform_device { (*get_resource)(struct vfio_platform_device *vdev, int i); int (*get_irq)(struct vfio_platform_device *vdev, int i); int (*of_reset)(struct vfio_platform_device *vdev); + u32 (*of_get_msi)(struct vfio_platform_device *vdev); + void (*of_msi_write)(struct vfio_platform_device *vdev, + struct msi_desc *desc, + struct msi_msg *msg); bool reset_required; }; typedef int (*vfio_platform_reset_fn_t)(struct vfio_platform_device *vdev); +typedef u32 (*vfio_platform_get_msi_fn_t)(struct vfio_platform_device *vdev); +typedef void (*vfio_platform_msi_write_fn_t)(struct vfio_platform_device *vdev, + struct msi_desc *desc, + struct msi_msg *msg); struct vfio_platform_reset_node { struct list_head link; char *compat; struct module *owner; vfio_platform_reset_fn_t of_reset; + vfio_platform_get_msi_fn_t of_get_msi; + vfio_platform_msi_write_fn_t of_msi_write; }; extern int vfio_platform_probe_common(struct vfio_platform_device *vdev, diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index d1812777139f..8e2c0131781d 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -697,11 +697,35 @@ struct vfio_irq_info { #define VFIO_IRQ_INFO_MASKABLE (1 << 1) #define VFIO_IRQ_INFO_AUTOMASKED (1 << 2) #define VFIO_IRQ_INFO_NORESIZE (1 << 3) +#define VFIO_IRQ_INFO_FLAG_CAPS (1 << 4) /* Info supports caps */ __u32 index; /* IRQ index */ __u32 count; /* Number of IRQs within this index */ + __u32 cap_offset; /* Offset within info struct of first cap */ }; #define VFIO_DEVICE_GET_IRQ_INFO _IO(VFIO_TYPE, VFIO_BASE + 9) +/* + * The irq type capability allows IRQs unique to a specific device or + * class of devices to be exposed. + * + * The structures below define version 1 of this capability. + */ +#define VFIO_IRQ_INFO_CAP_TYPE 3 + +struct vfio_irq_info_cap_type { + struct vfio_info_cap_header header; + __u32 type; /* global per bus driver */ + __u32 subtype; /* type specific */ +}; + +/* + * List of IRQ types, global per bus driver. + * If you introduce a new type, please add it here. + */ + +/* Non PCI devices having MSI(s) support */ +#define VFIO_IRQ_TYPE_MSI (1) + /** * VFIO_DEVICE_SET_IRQS - _IOW(VFIO_TYPE, VFIO_BASE + 10, struct vfio_irq_set) * From patchwork Fri Jan 29 17:24:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vikas Gupta X-Patchwork-Id: 12056223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MIME_HEADER_CTYPE_ONLY, SPF_HELO_NONE,SPF_PASS,T_TVD_MIME_NO_HEADERS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29F56C433E0 for ; Fri, 29 Jan 2021 17:29:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E5E6B64E02 for ; Fri, 29 Jan 2021 17:29:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231771AbhA2R3T (ORCPT ); Fri, 29 Jan 2021 12:29:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232504AbhA2R0C (ORCPT ); Fri, 29 Jan 2021 12:26:02 -0500 Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 983EBC0613ED for ; Fri, 29 Jan 2021 09:24:46 -0800 (PST) Received: by mail-pf1-x432.google.com with SMTP id 11so6645078pfu.4 for ; Fri, 29 Jan 2021 09:24:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=jQbUxL3FA6J+e4wy+qw0LR9aNP1mzHH69rLqi6B6fcw=; b=TtD9et2uN1uYZ1rtwl60JfeeGMB8qqWIR6FXTfV7UlhGtgtVFOzAsnCfXECxfrRBpt z4POnX5QR5PyuplCQ7wuP40oziFGbH4yMRGVyki3oOVG3/CaZN1nngB8c6yXUnS7CzGF dZqR2uUu/kxjfDiHkbmEZrozzqIPat3b1y+To= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=jQbUxL3FA6J+e4wy+qw0LR9aNP1mzHH69rLqi6B6fcw=; b=CKf26QVL6t2Yjcbu9aB2UZFh+MpodfEPCfS6ea7wtfiAgQXLq9pXIElVS/j3F/LJoe imbYkGTxN15Fzkim4gJWTg0CkgnopeyCYvIqls1YwvRltBZW0GIyOaGpO3KfNbLodPQm 7pBvYKHDTDxPTNxIvQ/2jb9ETI38C544Qbr8vfUG5Q9y8/Wa4F6awfm0hHcXrUHk8qfu 7oCGObwTWfwnNGRjikszS+UlTxIpvLGynJYsCg8lToAWyfibIe1r+c/IKB2f+Zwkcdmv c710xra6sHyvAGNsWFhVgG0eDN95hXM005ZJlUmAfIZCu1LKmjxsjvzt9xZTqDk9er1x hFCg== X-Gm-Message-State: AOAM533HHBIZX94ly67iFMgE+MDT7GeaU2XtxbQG15OvtQgwVT1BR7sE 6X8VIXaGvtmZ/aHz/s/spzNqiQ== X-Google-Smtp-Source: ABdhPJxfZvPlz/Mdm7FfhBLGQEahiV24WzFbjIj5+M/5E81eMQ9nMbhphAPOj3ycEDHvICydJwcoJA== X-Received: by 2002:a63:724a:: with SMTP id c10mr5951280pgn.124.1611941085889; Fri, 29 Jan 2021 09:24:45 -0800 (PST) Received: from rahul_yocto_ubuntu18.ibn.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id w11sm9739016pge.28.2021.01.29.09.24.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jan 2021 09:24:45 -0800 (PST) From: Vikas Gupta To: eric.auger@redhat.com, alex.williamson@redhat.com, cohuck@redhat.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: vikram.prakash@broadcom.com, srinath.mannam@broadcom.com, ashwin.kamath@broadcom.com, zachary.schroff@broadcom.com, manish.kurup@broadcom.com, Vikas Gupta Subject: [RFC v4 2/3] vfio/platform: change cleanup order Date: Fri, 29 Jan 2021 22:54:20 +0530 Message-Id: <20210129172421.43299-3-vikas.gupta@broadcom.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210129172421.43299-1-vikas.gupta@broadcom.com> References: <20201214174514.22006-1-vikas.gupta@broadcom.com> <20210129172421.43299-1-vikas.gupta@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org In the case of msi, vendor specific msi module may require region access to handle msi cleanup so we need to cleanup region after irq cleanup only. Signed-off-by: Vikas Gupta Acked-by: Eric Auger --- drivers/vfio/platform/vfio_platform_common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/vfio/platform/vfio_platform_common.c b/drivers/vfio/platform/vfio_platform_common.c index f2b1f0c3bfcc..1cc040e3ed1f 100644 --- a/drivers/vfio/platform/vfio_platform_common.c +++ b/drivers/vfio/platform/vfio_platform_common.c @@ -243,8 +243,8 @@ static void vfio_platform_release(void *device_data) WARN_ON(1); } pm_runtime_put(vdev->device); - vfio_platform_regions_cleanup(vdev); vfio_platform_irq_cleanup(vdev); + vfio_platform_regions_cleanup(vdev); } mutex_unlock(&driver_lock); From patchwork Fri Jan 29 17:24:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vikas Gupta X-Patchwork-Id: 12056225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MIME_HEADER_CTYPE_ONLY, SPF_HELO_NONE,SPF_PASS,T_TVD_MIME_NO_HEADERS,UNWANTED_LANGUAGE_BODY, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DB5EC433E6 for ; Fri, 29 Jan 2021 17:30:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E013F6024A for ; Fri, 29 Jan 2021 17:30:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231335AbhA2R3S (ORCPT ); Fri, 29 Jan 2021 12:29:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232519AbhA2R0J (ORCPT ); Fri, 29 Jan 2021 12:26:09 -0500 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D0FF4C061788 for ; Fri, 29 Jan 2021 09:24:51 -0800 (PST) Received: by mail-pj1-x1030.google.com with SMTP id cq1so6134948pjb.4 for ; Fri, 29 Jan 2021 09:24:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=9JwHZKN7IuEP8JQcuIlNN08rrng3LGs5w6sOoVE/E+Q=; b=fZMYkrxx+esCtt7HB5VW5fxx4/G39fYPJ9hpRMASHAUbb9M+2EdAcfeQW9NUKTCvfE nPKYtNCy5faqqSrL2TOTZbzBiPy/T6JCzWPjjyFBsBDFMfFiWECy6gNT/+MLZLFcC0Em 7tUXd8m6662u2ywhbEGuNbCP1CLcAO0MMi3Z0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9JwHZKN7IuEP8JQcuIlNN08rrng3LGs5w6sOoVE/E+Q=; b=pAD1PbJ0jhEVGgZSzQ617b7tlNgSXe7ZpewISbzTpF1mAA8jezW2g+xgfVR0iumd/o 1d1RHqqDV8kFC/CjEITVN0wCiet4CwHZs+Q1sPu0ewMHZHPZ7V3lORheo+RgDzqiOmv3 AH/NjkBZ+4jh5xxvSMOwBcPvxdzS7J65LRAXzdLeWehFO8NvGwPJVLAnUgzMaVh582CY U3ByNl4yRozGlfBp3PdznwMQnO8q//3MWqm103vVJ9ndI+VS6D0RBM98xj4LzFcf+O5w D7rNlzzOCBs+RFEVIGbAuWeNjSID2Qiyewyh4xZSLHksemV6NwMeDID3ZQbONQa7DQVO v/Dw== X-Gm-Message-State: AOAM533mhWoICLSVeIwvkopsC8J9qLrZyA3tzM2lFQpQvKER6VyRcVfD Y0xMsHknxvBPTiSv1BTrproJWA== X-Google-Smtp-Source: ABdhPJw6NDJK8xRAfGxWmP5w7msd3QKMC9ZPlpS3UWjmJg+UxOkcQtIOKx6B76K4zzzDgSpi+yI9Rw== X-Received: by 2002:a17:90a:f2c6:: with SMTP id gt6mr5414892pjb.35.1611941091086; Fri, 29 Jan 2021 09:24:51 -0800 (PST) Received: from rahul_yocto_ubuntu18.ibn.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id w11sm9739016pge.28.2021.01.29.09.24.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jan 2021 09:24:50 -0800 (PST) From: Vikas Gupta To: eric.auger@redhat.com, alex.williamson@redhat.com, cohuck@redhat.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: vikram.prakash@broadcom.com, srinath.mannam@broadcom.com, ashwin.kamath@broadcom.com, zachary.schroff@broadcom.com, manish.kurup@broadcom.com, Vikas Gupta Subject: [RFC v4 3/3] vfio: platform: reset: add msi support Date: Fri, 29 Jan 2021 22:54:21 +0530 Message-Id: <20210129172421.43299-4-vikas.gupta@broadcom.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210129172421.43299-1-vikas.gupta@broadcom.com> References: <20201214174514.22006-1-vikas.gupta@broadcom.com> <20210129172421.43299-1-vikas.gupta@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add msi support for Broadcom FlexRm device. Signed-off-by: Vikas Gupta --- .../platform/reset/vfio_platform_bcmflexrm.c | 72 ++++++++++++++++++- 1 file changed, 70 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/platform/reset/vfio_platform_bcmflexrm.c b/drivers/vfio/platform/reset/vfio_platform_bcmflexrm.c index 96064ef8f629..6ca4ca12575b 100644 --- a/drivers/vfio/platform/reset/vfio_platform_bcmflexrm.c +++ b/drivers/vfio/platform/reset/vfio_platform_bcmflexrm.c @@ -21,7 +21,9 @@ #include #include #include +#include #include +#include #include "../vfio_platform_private.h" @@ -33,6 +35,9 @@ #define RING_VER 0x000 #define RING_CONTROL 0x034 #define RING_FLUSH_DONE 0x038 +#define RING_MSI_ADDR_LS 0x03c +#define RING_MSI_ADDR_MS 0x040 +#define RING_MSI_DATA_VALUE 0x064 /* Register RING_CONTROL fields */ #define CONTROL_FLUSH_SHIFT 5 @@ -105,8 +110,71 @@ static int vfio_platform_bcmflexrm_reset(struct vfio_platform_device *vdev) return ret; } -module_vfio_reset_handler("brcm,iproc-flexrm-mbox", - vfio_platform_bcmflexrm_reset); +static u32 bcm_num_msi(struct vfio_platform_device *vdev) +{ + struct vfio_platform_region *reg = &vdev->regions[0]; + + return (reg->size / RING_REGS_SIZE); +} + +static void bcm_write_msi(struct vfio_platform_device *vdev, + struct msi_desc *desc, + struct msi_msg *msg) +{ + int i; + int hwirq = -1; + int msi_src; + void __iomem *ring; + struct vfio_platform_region *reg = &vdev->regions[0]; + + if (!reg) + return; + + for (i = 0; i < vdev->num_irqs; i++) + if (vdev->irqs[i].type == VFIO_IRQ_TYPE_MSI) + hwirq = vdev->irqs[i].ctx[0].hwirq; + + if (hwirq < 0) + return; + + msi_src = desc->irq - hwirq; + + if (!reg->ioaddr) { + reg->ioaddr = ioremap(reg->addr, reg->size); + if (!reg->ioaddr) + return; + } + + ring = reg->ioaddr + msi_src * RING_REGS_SIZE; + + writel_relaxed(msg->address_lo, ring + RING_MSI_ADDR_LS); + writel_relaxed(msg->address_hi, ring + RING_MSI_ADDR_MS); + writel_relaxed(msg->data, ring + RING_MSI_DATA_VALUE); +} + +static struct vfio_platform_reset_node vfio_platform_bcmflexrm_reset_node = { + .owner = THIS_MODULE, + .compat = "brcm,iproc-flexrm-mbox", + .of_reset = vfio_platform_bcmflexrm_reset, + .of_get_msi = bcm_num_msi, + .of_msi_write = bcm_write_msi +}; + +static int __init vfio_platform_bcmflexrm_reset_module_init(void) +{ + __vfio_platform_register_reset(&vfio_platform_bcmflexrm_reset_node); + + return 0; +} + +static void __exit vfio_platform_bcmflexrm_reset_module_exit(void) +{ + vfio_platform_unregister_reset("brcm,iproc-flexrm-mbox", + vfio_platform_bcmflexrm_reset); +} + +module_init(vfio_platform_bcmflexrm_reset_module_init); +module_exit(vfio_platform_bcmflexrm_reset_module_exit); MODULE_LICENSE("GPL v2"); MODULE_AUTHOR("Anup Patel ");