From patchwork Fri Feb 2 04:56:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542057 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A3C5F9C2; Fri, 2 Feb 2024 04:57:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849850; cv=none; b=Z86BOmk7lpup2uSX5wPL8LBSC4bxbJCyo5FqkahwnouEbqThfS8OGn5V9E9bkWIcMwQtuElOQ9EH9YOLnpptMNh+2HOnKoOM4/XWo7XriFdIJlqiJ9+fpWAiVD9tdEt7tmsa1HQXip+Sleoix0dzRmfquTaWapLkNuyOYdz4TWw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849850; c=relaxed/simple; bh=5y4NGXUar57YI25p8xEfMr33IxH4G1iGgFkp7ARQYPM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=duFzRpA0gV3GcRdLKxkPZRlrY6Eh1BfA963R0fIY1O5HI7GLQfoKgFFhzd9IGeWDoBxCuETJ0Z8FrsKDoPzX8qbc4YO8SgtF4b3vr06VhzjZ7UZK+jfWnpn/NIhN1v1eR6dk8xhRiBzfNR1pDpj0uvvf0pjr6clRczUgKj4n8Aw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=X06nBpSC; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="X06nBpSC" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849847; x=1738385847; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5y4NGXUar57YI25p8xEfMr33IxH4G1iGgFkp7ARQYPM=; b=X06nBpSCcFJa/ccHO+TvH10Yy0jQjTtonVm1Fml/qBWPNDz5EdLqlzGL bEEdf7vgqBZATalA4T466LR0WiyDQYDxGZERujWoWih+ZndAL58QM0HFr JHEA8itFJVAJPNgwbJl2ZPMAdte4dLYu8ZRh64ORySZ/kNCULm8pW9pGu 2QqEZP5av4atOEKfyuEMffnnV6veA4H+tgmHncsnfwE7haAbhopbPmKmd 8hHQR3rRygpKupnCOGVpCA6RIIz7RyWsr1l/2zuKG0mOgnEhVkuYGMrul dSaS7ZJ3j+1nWtH9U+1Gys8CwxiYwmsM+xa+3J69KjZ5p13eqRmHeaEEu w==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615791" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615791" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339759" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339759" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:24 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 01/17] vfio/pci: Use unsigned int instead of unsigned Date: Thu, 1 Feb 2024 20:56:55 -0800 Message-Id: <81103b9a7f0448bbd8740389fbb77f78c150620d.1706849424.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 checkpatch.pl warns about usage of bare unsigned. Change unsigned to unsigned int to eliminate checkpatch.pl warnings about existing code that are encountered by changes to the interrupt management code at or near lines using bare unsigned. Signed-off-by: Reinette Chatre --- Note to maintainers: Originally formed part of the IMS submission below, but is not specific to IMS. https://lore.kernel.org/lkml/cover.1696609476.git.reinette.chatre@intel.com drivers/vfio/pci/vfio_pci_intrs.c | 42 ++++++++++++++++++------------- 1 file changed, 24 insertions(+), 18 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 237beac83809..f9280e32972a 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -503,8 +503,9 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, return ret; } -static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned start, - unsigned count, int32_t *fds, bool msix) +static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, + unsigned int start, unsigned int count, + int32_t *fds, bool msix) { unsigned int i, j; int ret = 0; @@ -553,8 +554,9 @@ static void vfio_msi_disable(struct vfio_pci_core_device *vdev, bool msix) * IOCTL support */ static int vfio_pci_set_intx_unmask(struct vfio_pci_core_device *vdev, - unsigned index, unsigned start, - unsigned count, uint32_t flags, void *data) + unsigned int index, unsigned int start, + unsigned int count, uint32_t flags, + void *data) { if (!is_intx(vdev) || start != 0 || count != 1) return -EINVAL; @@ -584,8 +586,8 @@ static int vfio_pci_set_intx_unmask(struct vfio_pci_core_device *vdev, } static int vfio_pci_set_intx_mask(struct vfio_pci_core_device *vdev, - unsigned index, unsigned start, - unsigned count, uint32_t flags, void *data) + unsigned int index, unsigned int start, + unsigned int count, uint32_t flags, void *data) { if (!is_intx(vdev) || start != 0 || count != 1) return -EINVAL; @@ -604,8 +606,9 @@ static int vfio_pci_set_intx_mask(struct vfio_pci_core_device *vdev, } static int vfio_pci_set_intx_trigger(struct vfio_pci_core_device *vdev, - unsigned index, unsigned start, - unsigned count, uint32_t flags, void *data) + unsigned int index, unsigned int start, + unsigned int count, uint32_t flags, + void *data) { if (is_intx(vdev) && !count && (flags & VFIO_IRQ_SET_DATA_NONE)) { vfio_intx_disable(vdev); @@ -647,8 +650,9 @@ static int vfio_pci_set_intx_trigger(struct vfio_pci_core_device *vdev, } static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, - unsigned index, unsigned start, - unsigned count, uint32_t flags, void *data) + unsigned int index, unsigned int start, + unsigned int count, uint32_t flags, + void *data) { struct vfio_pci_irq_ctx *ctx; unsigned int i; @@ -755,8 +759,9 @@ static int vfio_pci_set_ctx_trigger_single(struct eventfd_ctx **ctx, } static int vfio_pci_set_err_trigger(struct vfio_pci_core_device *vdev, - unsigned index, unsigned start, - unsigned count, uint32_t flags, void *data) + unsigned int index, unsigned int start, + unsigned int count, uint32_t flags, + void *data) { if (index != VFIO_PCI_ERR_IRQ_INDEX || start != 0 || count > 1) return -EINVAL; @@ -766,8 +771,9 @@ static int vfio_pci_set_err_trigger(struct vfio_pci_core_device *vdev, } static int vfio_pci_set_req_trigger(struct vfio_pci_core_device *vdev, - unsigned index, unsigned start, - unsigned count, uint32_t flags, void *data) + unsigned int index, unsigned int start, + unsigned int count, uint32_t flags, + void *data) { if (index != VFIO_PCI_REQ_IRQ_INDEX || start != 0 || count > 1) return -EINVAL; @@ -777,11 +783,11 @@ static int vfio_pci_set_req_trigger(struct vfio_pci_core_device *vdev, } int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device *vdev, uint32_t flags, - unsigned index, unsigned start, unsigned count, - void *data) + unsigned int index, unsigned int start, + unsigned int count, void *data) { - int (*func)(struct vfio_pci_core_device *vdev, unsigned index, - unsigned start, unsigned count, uint32_t flags, + int (*func)(struct vfio_pci_core_device *vdev, unsigned int index, + unsigned int start, unsigned int count, uint32_t flags, void *data) = NULL; switch (index) { From patchwork Fri Feb 2 04:56:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542056 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF400D521; Fri, 2 Feb 2024 04:57:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849849; cv=none; b=VO2QUAlF36Jv4ye7lFf98SHVJJGBY7BC6UqMYN1GOa8O3u5MdL+capowuBqgenrdHhUUZZIWD22GS5sfHVQxKQFhBZOXLuLHsPS4h8nn54v+wZQzJP72qnfXPPywvemVw44N/xEVT2dQOZVnJj4o6l9cccCX4JnL/Mpr1AbDLR8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849849; c=relaxed/simple; bh=SRyuqWoDw8f/ODrjmB+YtXDl8kO/6S8q7xI8rCGmDRA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZGOs7zrR9nAFlUVynv3fv+eVjns6afjM09h2RyxXU1/qV8ZYnfrwCYgkU+M9JwVn2rI5EtM1sgumbrHvTfYjRtEwW0Rb+HZcU+YD7p7x44kGAt/Ldd2XYdlqLXQ8IUISmrCa8FUCn6DCFu/0zeLpHqogN9xk0aUw7dtR2AkQD5k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=BW6dVVNj; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="BW6dVVNj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849848; x=1738385848; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SRyuqWoDw8f/ODrjmB+YtXDl8kO/6S8q7xI8rCGmDRA=; b=BW6dVVNjLZUGEz26OkuVhAvAiR4prWmvehVUbmWtDzh81Vr4YZZOPdo3 f+vzGbgcs7pn21B6K6NnWe7vsclh48Ectsq8P+0MMJkkAKFz6EVwkXToL GcYk3NNhKdCkf2m5xpSV/8m40fbEcoF26/H6nElV1yiFuJQIwLcKbPytn +fBUKug3axMxD7TuPC8vgdrVryEhNJGZ09ZtNX2tIgMS6Yx4mYx5tpzJi uwxWGgs1bgUIVA0lbNvVeHXmuQlpOmoy6LN935WFO81y8OLm4ip/7m4py SBZR7hpWbAj1TjSRqj79vh9HAULvaRU352+YaE8j1fcC4h9OMTL60Zasp w==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615796" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615796" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339761" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339761" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:24 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 02/17] vfio/pci: Remove duplicate check from vfio_pci_set_ctx_trigger_single() wrappers Date: Thu, 1 Feb 2024 20:56:56 -0800 Message-Id: <1429dd8025b6807e4d6951a18c813b94bbc52c0e.1706849424.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Both wrappers of vfio_pci_set_ctx_trigger_single() perform the same parameter checks before calling vfio_pci_set_ctx_trigger_single(). vfio_pci_set_ctx_trigger_single() only handles a single interrupt that is expected to start from 0. Move parameter checks into vfio_pci_set_ctx_trigger_single() instead of duplicated in its callers. Signed-off-by: Reinette Chatre --- drivers/vfio/pci/vfio_pci_intrs.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index f9280e32972a..69ab11863282 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -704,9 +704,13 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, } static int vfio_pci_set_ctx_trigger_single(struct eventfd_ctx **ctx, + unsigned int start, unsigned int count, uint32_t flags, void *data) { + if (start != 0 || count > 1) + return -EINVAL; + /* DATA_NONE/DATA_BOOL enables loopback testing */ if (flags & VFIO_IRQ_SET_DATA_NONE) { if (*ctx) { @@ -763,10 +767,10 @@ static int vfio_pci_set_err_trigger(struct vfio_pci_core_device *vdev, unsigned int count, uint32_t flags, void *data) { - if (index != VFIO_PCI_ERR_IRQ_INDEX || start != 0 || count > 1) + if (index != VFIO_PCI_ERR_IRQ_INDEX) return -EINVAL; - return vfio_pci_set_ctx_trigger_single(&vdev->err_trigger, + return vfio_pci_set_ctx_trigger_single(&vdev->err_trigger, start, count, flags, data); } @@ -775,10 +779,10 @@ static int vfio_pci_set_req_trigger(struct vfio_pci_core_device *vdev, unsigned int count, uint32_t flags, void *data) { - if (index != VFIO_PCI_REQ_IRQ_INDEX || start != 0 || count > 1) + if (index != VFIO_PCI_REQ_IRQ_INDEX) return -EINVAL; - return vfio_pci_set_ctx_trigger_single(&vdev->req_trigger, + return vfio_pci_set_ctx_trigger_single(&vdev->req_trigger, start, count, flags, data); } From patchwork Fri Feb 2 04:56:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542060 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1FFD5F9D3; Fri, 2 Feb 2024 04:57:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849852; cv=none; b=JxwZGNAoVRJhItTuBv7cDAsS79WbwDA4t5oHg/cOwTId0q7KuY4XEOwgy3nDmDuKBoF+8BramyhicS2YQL0cLmWpZ8vW9NeeFGy17tT7c+G4j41iOz/FN4Fr6pDNzeAHektWgK9EbuoO/ha1/sq0Dy8JwkNv4beIWN5HzVcbZu4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849852; c=relaxed/simple; bh=JWucdCztDGT+G4ZMaH9FIiM5mhLJXuyhplUEEPj9ZAU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=XwWFMUW6MUCTqWsUJQJXSgEW+NdtNWlvuOCvci3ZpHJoPaLrcZtIQDdBAmpgRK7AoijATCSkfuwhY1vLtgye44OlcEJ3JkOwuWgsldLpL/tYbjGT2u4smd0wAZeTtYou8hWBrZypuTsh85EKcDd4mofXdXDrqrjX5nb/yI0IMgo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=G33JQGfS; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="G33JQGfS" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849849; x=1738385849; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JWucdCztDGT+G4ZMaH9FIiM5mhLJXuyhplUEEPj9ZAU=; b=G33JQGfSAFNAwEemVxQvWRWYBcXBnJJLkOwEseI2KZD9lBjS16Z/+AGe BWRleeN8X7Q8juOMluJBTzAHRtioGCVfMT3HSO+C+PfiWZNvkOXqJddHB fHG2dyMZDmThLfSjhAe9oQwkNr2jUfo3LxCi4au5AQYxva1qEL1ye8NrI dXPfduk/fKDFEs1zVEiI1ZvIbhQGH6+v+Yb5khw71jX4vGKvrRU/mjHM+ cjVaHA8aLuY9QP9AHin2HkLvxgrZIFNsSnjA5pWs9ACLahZHy9ejak/ii eBXmdiF3pOSgygadSLVNm2pU4ZvBiYmcMNDP+U5VQyZlpw91t22hj1PSo Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615802" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615802" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339765" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339765" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:24 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 03/17] vfio/pci: Consistently acquire mutex for interrupt management Date: Thu, 1 Feb 2024 20:56:57 -0800 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 vfio_pci_set_irqs_ioctl() is the entrypoint for interrupt management via the VFIO_DEVICE_SET_IRQS ioctl(). The igate mutex is obtained before calling vfio_pci_set_irqs_ioctl() for management of all interrupt types to protect against concurrent changes to the eventfds associated with device request notification and error interrupts. The igate mutex is not acquired consistently. The mutex is always (for all interrupt types) acquired from within vfio_pci_ioctl_set_irqs() before calling vfio_pci_set_irqs_ioctl(), but vfio_pci_set_irqs_ioctl() is called via vfio_pci_core_disable() without the mutex held. The latter is expected to be correct if the code flow can be guaranteed that the provided interrupt type is not a device request notification or error interrupt. Move igate mutex acquire and release into vfio_pci_set_irqs_ioctl() to make the locking consistent irrespective of interrupt type. This is one step closer to contain the interrupt management locking internals within the interrupt management code so that the VFIO PCI core can trigger management of the eventfds associated with device request notification and error interrupts without needing to access and manipulate VFIO interrupt management locks and data. Signed-off-by: Reinette Chatre --- Note to maintainers: Originally formed part of the IMS submission below, but is not specific to IMS. https://lore.kernel.org/lkml/cover.1696609476.git.reinette.chatre@intel.com drivers/vfio/pci/vfio_pci_core.c | 3 --- drivers/vfio/pci/vfio_pci_intrs.c | 10 ++++++++-- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index 1cbc990d42e0..d2847ca2f0cb 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1214,12 +1214,9 @@ static int vfio_pci_ioctl_set_irqs(struct vfio_pci_core_device *vdev, return PTR_ERR(data); } - mutex_lock(&vdev->igate); - ret = vfio_pci_set_irqs_ioctl(vdev, hdr.flags, hdr.index, hdr.start, hdr.count, data); - mutex_unlock(&vdev->igate); kfree(data); return ret; diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 69ab11863282..97a3bb22b186 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -793,7 +793,9 @@ int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device *vdev, uint32_t flags, int (*func)(struct vfio_pci_core_device *vdev, unsigned int index, unsigned int start, unsigned int count, uint32_t flags, void *data) = NULL; + int ret = -ENOTTY; + mutex_lock(&vdev->igate); switch (index) { case VFIO_PCI_INTX_IRQ_INDEX: switch (flags & VFIO_IRQ_SET_ACTION_TYPE_MASK) { @@ -838,7 +840,11 @@ int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device *vdev, uint32_t flags, } if (!func) - return -ENOTTY; + goto out_unlock; + + ret = func(vdev, index, start, count, flags, data); +out_unlock: + mutex_unlock(&vdev->igate); + return ret; - return func(vdev, index, start, count, flags, data); } From patchwork Fri Feb 2 04:56:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542058 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A363EF9F8; Fri, 2 Feb 2024 04:57:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849851; cv=none; b=b2ZO2bxSKVwf9DRrV9SYLcON1M65j43hbfU3NtpigX59fxDhPoheE1AzkomBpy701WZiDyeoiHZh1csgnn7SRYiQOsHPEF8dFf1M9oTtNDiXiNeXjC7Q6Uj7PhxNY/79XY3FW5+vHjy0mcra1yzIeUu1R+8zGqlkfIXtCqv+Xe4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849851; c=relaxed/simple; bh=HP2huTcvvO7TUh3mgU3qMuAjpGy887H4YuFiJWxsEUU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uGlqab8hO/7FgYmAf92K+MrrjTbPS9ue0fXd+05fwvU1jwMrH1VDIWHO2m7EFV2GbRGMLoXizJBYKNriODYTeA5l1lw4o8zl0wDPWGkNARE6A9+xWGyZu7kryGLIS43uGIXw7Ez52MKAx7UFGU5xDc1pcdbKOKSvk++kTvckX+Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=hNdMs8MS; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="hNdMs8MS" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849850; x=1738385850; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HP2huTcvvO7TUh3mgU3qMuAjpGy887H4YuFiJWxsEUU=; b=hNdMs8MS3e/bBEIeiIY5VHWX6P4SRBeKIi6QUjuZ/EDca/go53DmWCdi 3uyvbRwp3dNWR8onnBl4grBBVSeNsBIdcg8ekFNzS6c5lOzipRQtXsnwb xEwfJc1dGvgxB5XyLOMsJ5kFr/KBOkBbBw/PpvZxuvwOBDgQYl6vbxHBC Q7OtnPnpaxrN4yguZzsehQ7onLNlD6Xg6GxWmcYi6Ls4ovQuCLb+mMi7e k/nHKpu/zhwmABdhTtXd2RsEqy/Vlyl3Iv6PMZ7HX/pHOoTXAHCIn0U0g RjZt5jcz5xNTX3a6XCOisrc+vWarDxqQUKOpql1+11/gyaot8zKldBLaQ Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615808" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615808" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339768" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339768" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:24 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 04/17] vfio/pci: Remove duplicate interrupt management from core VFIO PCI Date: Thu, 1 Feb 2024 20:56:58 -0800 Message-Id: <93dfe93e8fc53e83ac585ed3a891cefde32b3016.1706849424.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The eventfds associated with device request notification and error IRQ are managed by the VFIO PCI interrupt management code contained in vfio_pci_intrs.c. The VFIO PCI interrupt management code supports acquiring, releasing, as well as signaling of these eventfd. The VFIO PCI core code duplicates the signaling of device request notification and error interrupts and does so by acquiring the lock and accessing data that should ideally be contained within the VFIO PCI interrupt management code. Do not duplicate but instead call existing VFIO PCI interrupt management code to signal device request notification and error interrupts. Signed-off-by: Reinette Chatre --- drivers/vfio/pci/vfio_pci_core.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index d2847ca2f0cb..de097174e455 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -700,16 +700,12 @@ void vfio_pci_core_close_device(struct vfio_device *core_vdev) #endif vfio_pci_core_disable(vdev); - mutex_lock(&vdev->igate); - if (vdev->err_trigger) { - eventfd_ctx_put(vdev->err_trigger); - vdev->err_trigger = NULL; - } - if (vdev->req_trigger) { - eventfd_ctx_put(vdev->req_trigger); - vdev->req_trigger = NULL; - } - mutex_unlock(&vdev->igate); + vfio_pci_set_irqs_ioctl(vdev, VFIO_IRQ_SET_DATA_NONE | + VFIO_IRQ_SET_ACTION_TRIGGER, + VFIO_PCI_ERR_IRQ_INDEX, 0, 0, NULL); + vfio_pci_set_irqs_ioctl(vdev, VFIO_IRQ_SET_DATA_NONE | + VFIO_IRQ_SET_ACTION_TRIGGER, + VFIO_PCI_REQ_IRQ_INDEX, 0, 0, NULL); } EXPORT_SYMBOL_GPL(vfio_pci_core_close_device); From patchwork Fri Feb 2 04:56:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542059 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2DC91101EC; Fri, 2 Feb 2024 04:57:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849851; cv=none; b=frFarHnJmAT9RLRerrCWX/GYRDf611wcwST3EnGFPZjUJrIqc33/100eXSks3mof5qX56M1cYpjZxKSMlSP+JVWovtADfyasKmb9no5YX9LmJT6Uj6actaOGngsAUm3qP7M5Fy4eVNET3XnMnuMWH+z5gogeVsNPrXMz21TVN9A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849851; c=relaxed/simple; bh=wjShxdkTqGwADOgBt3EJHJs5FYJXIg/tseoEaV8uAgM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=h+LiooV3yfqwWDWfXPczHc26MoycSQt5xqEigxgDQQZE/fdWD0WPa7xWrDOwfCgEzplFDh1vnC0svFqHLAM/DkElpeJV1j0hFg8mduk0CICx46MyhU/2/Liwp7eCUaOQCqQtD89uZmHTzIft/fB+L9P0GT/B1i17nbPe41NzHBc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=hWDHKiky; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="hWDHKiky" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849850; x=1738385850; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wjShxdkTqGwADOgBt3EJHJs5FYJXIg/tseoEaV8uAgM=; b=hWDHKikybu6AUycg4O0oPYwBVNvSHN0JWa0LTlEpkXFJFC2UwlIWk7B4 4CJYR0u7q4+d4dN8UITwRaahaSVNtbEDldb9m44QCv/9miCWlwiAtlI6Y lPPaqACjaFOGPavV4f4X3XU0DjGY8W0KxwJFm70kWiE5a5vcAnZM3vRDh I0rLwf/Qf6DpPUJ/Lbb8NocBUS3DvgCUhDg2OJeoZm+3l0YyY5033mZbR 4/6izxhOJGiJ4JR+V7znizMzu/+y5K11OuJMufKsJuZ2tlnyaGsiwyZZi 5YIDviDDO0EOTgwlFTaUf3o5kDBkc+Pc3qiQyglmNeztWiF4qG4jpivTs g==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615813" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615813" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339771" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339771" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:24 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 05/17] vfio/pci: Limit eventfd signaling to interrupt management code Date: Thu, 1 Feb 2024 20:56:59 -0800 Message-Id: <54ba34d884de30f479bbe3844c4a342980760d79.1706849424.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 VFIO PCI interrupt management has full support for configuring, signaling, and clearing the eventfds associated with device request notification and error interrupts. Core VFIO PCI duplicates eventfd signaling while also accessing data that should be contained within the interrupt management code. Remove the duplicate eventfd signaling from core VFIO PCI and rely on VFIO PCI interrupt management for eventfd signaling. Signed-off-by: Reinette Chatre --- drivers/vfio/pci/vfio_pci_core.c | 30 +++++++++++++----------------- 1 file changed, 13 insertions(+), 17 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index de097174e455..a8235b9c1a81 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1868,21 +1868,20 @@ void vfio_pci_core_request(struct vfio_device *core_vdev, unsigned int count) struct vfio_pci_core_device *vdev = container_of(core_vdev, struct vfio_pci_core_device, vdev); struct pci_dev *pdev = vdev->pdev; + int ret; - mutex_lock(&vdev->igate); + ret = vfio_pci_set_irqs_ioctl(vdev, VFIO_IRQ_SET_DATA_NONE | + VFIO_IRQ_SET_ACTION_TRIGGER, + VFIO_PCI_REQ_IRQ_INDEX, 0, 1, NULL); - if (vdev->req_trigger) { - if (!(count % 10)) - pci_notice_ratelimited(pdev, - "Relaying device request to user (#%u)\n", - count); - eventfd_signal(vdev->req_trigger); - } else if (count == 0) { + if (!ret && !(count % 10)) { + pci_notice_ratelimited(pdev, + "Relaying device request to user (#%u)\n", + count); + } else if (ret && count == 0) { pci_warn(pdev, - "No device request channel registered, blocked until released by user\n"); + "No device request channel registered, blocked until released by user\n"); } - - mutex_unlock(&vdev->igate); } EXPORT_SYMBOL_GPL(vfio_pci_core_request); @@ -2292,12 +2291,9 @@ pci_ers_result_t vfio_pci_core_aer_err_detected(struct pci_dev *pdev, { struct vfio_pci_core_device *vdev = dev_get_drvdata(&pdev->dev); - mutex_lock(&vdev->igate); - - if (vdev->err_trigger) - eventfd_signal(vdev->err_trigger); - - mutex_unlock(&vdev->igate); + vfio_pci_set_irqs_ioctl(vdev, VFIO_IRQ_SET_DATA_NONE | + VFIO_IRQ_SET_ACTION_TRIGGER, + VFIO_PCI_ERR_IRQ_INDEX, 0, 1, NULL); return PCI_ERS_RESULT_CAN_RECOVER; } From patchwork Fri Feb 2 04:57:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542061 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 68E901643E; Fri, 2 Feb 2024 04:57:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849853; cv=none; b=JcsLsY10AahD2sPIsHteSytZfREDzEd1N9doEPr6tpFcmzLFpNXa69Y3/3lwGbRiDGUHm+wn1RkZKGVcVP/T8CcBdfQtFEuiEEbkKEnXYFZ4XM0weUCriVAdzXaYY8bn+PHjWFk0iOEHiVXdU/sdGbk2cOjCIJZCiQcXZ5f5cpU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849853; c=relaxed/simple; bh=Kxk7o9A62R7KtL/giSvWj2QXePnCiz/Qp5YtBYfbf9E=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=J5A/dqRO35Hgkqoa0RvraIpJg8r0xSSrD+iwpf71kqexISTVR60Zcid1U+kq30OP4D244YC8WzOLUeKSZNFk7rRWVVvusqi8P6ddBVLpaQUPnDNFaNrBt0BNiN5coYrSCEK4AR6jIoNF8VXc4snhAZ1y+a8tyE4ks2U+83BmP/o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=b17HEZKp; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="b17HEZKp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849852; x=1738385852; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Kxk7o9A62R7KtL/giSvWj2QXePnCiz/Qp5YtBYfbf9E=; b=b17HEZKprk1chPeVsxf0dJpuHjnLK5y8AEqpnQcTfM6r/o2gqE8NAUB8 mEOUR5C0xShwXwqYFveMUtajxdj5N6+JA2+n9kgi2+SWQ4BDmv0fn6o0S YFmR+B2Y9UtkP5rZxtZIBCFNbTihiJu0Zk+VFEDOIX0hicWrxMigDx829 L9s5BtVq537tJ1U9wlGpSaeQ0LFsPW8l5ch3zN7d1gLet6RWbTDRp8L8m SqEICcPwLZc9AUQRBbXth6UdjNUiQs/rfw2gmQbFwd1QJI9kC3gHxWZJb 0zpHdwZCniiWYgJZLwmP6PDJKiLwkhZIJOowyPO2Qx31ZKlbLLWVgt+YN Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615818" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615818" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339774" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339774" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:24 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 06/17] vfio/pci: Remove interrupt index interpretation from wrappers Date: Thu, 1 Feb 2024 20:57:00 -0800 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 vfio_pci_set_intx_trigger() and vfio_pci_set_msi_trigger() have similar enough flows that can be converged into one shared flow instead of duplicated. To share code between management of interrupt types it is necessary that the type of the interrupt is only interpreted by the code specific to the interrupt type being managed. Remove interrupt type interpretation from what will eventually be shared code (vfio_pci_set_msi_trigger(), vfio_msi_set_block()) by pushing this interpretation to be within the interrupt type specific code (vfio_msi_enable(), (temporary) vfio_msi_set_vector_signal()). Signed-off-by: Reinette Chatre --- Note to maintainers: Originally formed part of the IMS submission below, but is not specific to IMS. https://lore.kernel.org/lkml/cover.1696609476.git.reinette.chatre@intel.com drivers/vfio/pci/vfio_pci_intrs.c | 38 +++++++++++++++++-------------- 1 file changed, 21 insertions(+), 17 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 97a3bb22b186..31f73c70fcd2 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -346,16 +346,19 @@ static irqreturn_t vfio_msihandler(int irq, void *arg) return IRQ_HANDLED; } -static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, bool msix) +static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, + unsigned int index) { struct pci_dev *pdev = vdev->pdev; - unsigned int flag = msix ? PCI_IRQ_MSIX : PCI_IRQ_MSI; + unsigned int flag; int ret; u16 cmd; if (!is_irq_none(vdev)) return -EINVAL; + flag = (index == VFIO_PCI_MSIX_IRQ_INDEX) ? PCI_IRQ_MSIX : PCI_IRQ_MSI; + /* return the number of supported vectors if we can't get all: */ cmd = vfio_pci_memory_lock_and_enable(vdev); ret = pci_alloc_irq_vectors(pdev, 1, nvec, flag); @@ -367,10 +370,9 @@ static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, bool msi } vfio_pci_memory_unlock_and_restore(vdev, cmd); - vdev->irq_type = msix ? VFIO_PCI_MSIX_IRQ_INDEX : - VFIO_PCI_MSI_IRQ_INDEX; + vdev->irq_type = index; - if (!msix) { + if (index == VFIO_PCI_MSI_IRQ_INDEX) { /* * Compute the virtual hardware field for max msi vectors - * it is the log base 2 of the number of vectors. @@ -413,8 +415,10 @@ static int vfio_msi_alloc_irq(struct vfio_pci_core_device *vdev, } static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, - unsigned int vector, int fd, bool msix) + unsigned int vector, int fd, + unsigned int index) { + bool msix = (index == VFIO_PCI_MSIX_IRQ_INDEX) ? true : false; struct pci_dev *pdev = vdev->pdev; struct vfio_pci_irq_ctx *ctx; struct eventfd_ctx *trigger; @@ -505,25 +509,26 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned int start, unsigned int count, - int32_t *fds, bool msix) + int32_t *fds, unsigned int index) { unsigned int i, j; int ret = 0; for (i = 0, j = start; i < count && !ret; i++, j++) { int fd = fds ? fds[i] : -1; - ret = vfio_msi_set_vector_signal(vdev, j, fd, msix); + ret = vfio_msi_set_vector_signal(vdev, j, fd, index); } if (ret) { for (i = start; i < j; i++) - vfio_msi_set_vector_signal(vdev, i, -1, msix); + vfio_msi_set_vector_signal(vdev, i, -1, index); } return ret; } -static void vfio_msi_disable(struct vfio_pci_core_device *vdev, bool msix) +static void vfio_msi_disable(struct vfio_pci_core_device *vdev, + unsigned int index) { struct pci_dev *pdev = vdev->pdev; struct vfio_pci_irq_ctx *ctx; @@ -533,7 +538,7 @@ static void vfio_msi_disable(struct vfio_pci_core_device *vdev, bool msix) xa_for_each(&vdev->ctx, i, ctx) { vfio_virqfd_disable(&ctx->unmask); vfio_virqfd_disable(&ctx->mask); - vfio_msi_set_vector_signal(vdev, i, -1, msix); + vfio_msi_set_vector_signal(vdev, i, -1, index); } cmd = vfio_pci_memory_lock_and_enable(vdev); @@ -656,10 +661,9 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, { struct vfio_pci_irq_ctx *ctx; unsigned int i; - bool msix = (index == VFIO_PCI_MSIX_IRQ_INDEX) ? true : false; if (irq_is(vdev, index) && !count && (flags & VFIO_IRQ_SET_DATA_NONE)) { - vfio_msi_disable(vdev, msix); + vfio_msi_disable(vdev, index); return 0; } @@ -672,15 +676,15 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, if (vdev->irq_type == index) return vfio_msi_set_block(vdev, start, count, - fds, msix); + fds, index); - ret = vfio_msi_enable(vdev, start + count, msix); + ret = vfio_msi_enable(vdev, start + count, index); if (ret) return ret; - ret = vfio_msi_set_block(vdev, start, count, fds, msix); + ret = vfio_msi_set_block(vdev, start, count, fds, index); if (ret) - vfio_msi_disable(vdev, msix); + vfio_msi_disable(vdev, index); return ret; } From patchwork Fri Feb 2 04:57:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542062 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E682F168DF; Fri, 2 Feb 2024 04:57:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849853; cv=none; b=CUbSfhFaD4jPfU9GFpLNbxUwazdczBoXSOXw5yZRNoDBDnQF+6qUHygWALO4PAPOenaTtw8zMHQWjHRJxUtdHaq1AZoQKvwh09n0+EbwAavE7AHF9pG5rrWUJ1NN1kmiJfIrnG6MAtzuU62HoZQ2UUcoZJtzHJfvlFDXZ1uHRFQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849853; c=relaxed/simple; bh=Y8mGWLOXkQHarvJTOtXDGSt3QFUWy8OXLPs0vhIlLNk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZCUP3myB891NUQXLHdQcT4ubcJcRJsT9/pY++GgCz801GP9SDVC0cLBVyA/hXZ7eWuHMPbeAavta4V3R/9A41Ay8tX8fYBDS45brrYTK0Zf4JoZokzecEvTAmSt5C8SurvBqddkNgAbziOsTJ73fracdCOmHhyvZKYczMvuC1LE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=WC4Zj/1A; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="WC4Zj/1A" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849852; x=1738385852; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Y8mGWLOXkQHarvJTOtXDGSt3QFUWy8OXLPs0vhIlLNk=; b=WC4Zj/1A+LpV9zvH+RjiNYlbHjnJMsGLoNT1EWi6IewmvNpGg/I1K9W+ lDC5Gqsfr34JsDMMLjvxwhvWxnNpuGMZHCyI8EXCiY1uZN3tuFsGtvp9s D0GDKb6LDMrFrZpm52UeUcPIFIqBF+98wKdC/1YI+oiDX85TTvu2Yg2qY OrMp8Hp6TtwyDk0bq4ZtWGVw80g4B2Baoo0pmyoXQBlb8YB7ybvjhauIX 05eyUsFIpxGJx+4upiP242AZtqbb11vn1f6mtjzi8xGNnlXLaL104gzOa 3td5kmIvqP0h3+TEEpZHzeQvpVNsFAKtmsKrpqqMqWDfHzOdwAzbu/EX+ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615823" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615823" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339776" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339776" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:24 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 07/17] vfio/pci: Preserve per-interrupt contexts Date: Thu, 1 Feb 2024 20:57:01 -0800 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 MSI and MSI-X interrupt management for PCI passthrough devices create a new per-interrupt context every time an interrupt is allocated, freeing it when the interrupt is freed. The per-interrupt context contains the properties of a particular interrupt. Without a property that persists across interrupt allocation and free it is acceptable to always create a new per-interrupt context. INTx interrupt context has a "masked" property that persists across allocation and free and thus preserves its interrupt context across interrupt allocation and free calls. MSI and MSI-X interrupts already remain allocated across interrupt allocation and free requests, additionally maintaining the individual interrupt context is a reflection of this existing behavior and matches INTx behavior so that more code can be shared. An additional benefit is that maintaining interrupt context supports a potential future use case of emulated interrupts, where the "is this interrupt emulated" is a property that needs to persist across allocation and free requests. Persistent interrupt contexts means that existence of per-interrupt context no longer implies a valid trigger, pointers to freed memory should be cleared, and a new per-interrupt context cannot be assumed needing allocation when an interrupt is allocated. Signed-off-by: Reinette Chatre --- Note to maintainers: This addition originally formed part of the IMS work below that mostly ignored INTx. This work focuses on INTx, MSI, MSI-X where this addition is relevant. https://lore.kernel.org/lkml/cover.1696609476.git.reinette.chatre@intel.com drivers/vfio/pci/vfio_pci_intrs.c | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 31f73c70fcd2..7ca2b983b66e 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -427,7 +427,7 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, ctx = vfio_irq_ctx_get(vdev, vector); - if (ctx) { + if (ctx && ctx->trigger) { irq_bypass_unregister_producer(&ctx->producer); irq = pci_irq_vector(pdev, vector); cmd = vfio_pci_memory_lock_and_enable(vdev); @@ -435,8 +435,9 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, vfio_pci_memory_unlock_and_restore(vdev, cmd); /* Interrupt stays allocated, will be freed at MSI-X disable. */ kfree(ctx->name); + ctx->name = NULL; eventfd_ctx_put(ctx->trigger); - vfio_irq_ctx_free(vdev, ctx, vector); + ctx->trigger = NULL; } if (fd < 0) @@ -449,16 +450,17 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, return irq; } - ctx = vfio_irq_ctx_alloc(vdev, vector); - if (!ctx) - return -ENOMEM; + /* Per-interrupt context remain allocated. */ + if (!ctx) { + ctx = vfio_irq_ctx_alloc(vdev, vector); + if (!ctx) + return -ENOMEM; + } ctx->name = kasprintf(GFP_KERNEL_ACCOUNT, "vfio-msi%s[%d](%s)", msix ? "x" : "", vector, pci_name(pdev)); - if (!ctx->name) { - ret = -ENOMEM; - goto out_free_ctx; - } + if (!ctx->name) + return -ENOMEM; trigger = eventfd_ctx_fdget(fd); if (IS_ERR(trigger)) { @@ -502,8 +504,7 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, eventfd_ctx_put(trigger); out_free_name: kfree(ctx->name); -out_free_ctx: - vfio_irq_ctx_free(vdev, ctx, vector); + ctx->name = NULL; return ret; } @@ -539,6 +540,7 @@ static void vfio_msi_disable(struct vfio_pci_core_device *vdev, vfio_virqfd_disable(&ctx->unmask); vfio_virqfd_disable(&ctx->mask); vfio_msi_set_vector_signal(vdev, i, -1, index); + vfio_irq_ctx_free(vdev, ctx, i); } cmd = vfio_pci_memory_lock_and_enable(vdev); @@ -694,7 +696,7 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, for (i = start; i < start + count; i++) { ctx = vfio_irq_ctx_get(vdev, i); - if (!ctx) + if (!ctx || !ctx->trigger) continue; if (flags & VFIO_IRQ_SET_DATA_NONE) { eventfd_signal(ctx->trigger); From patchwork Fri Feb 2 04:57:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542064 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6909817589; Fri, 2 Feb 2024 04:57:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849855; cv=none; b=kZKviJM1ssHtqE0QOYkSjRoux4YvURB6kmXCN40FezkUawvwDgO35skyZOF5OL0+EAJbNMrtEEOX1ErHmRh4UaoxrnM1QwdfzjXeyB+kwC22ZibSzG01kyRWpvH7sFjIIZrwPoCX6mt8omCYDrlCAm5kXVrohZuTBpSan90lj04= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849855; c=relaxed/simple; bh=hPTVpULSZEGp+qlRQ7yydOeKGAslemxYjjTwBe0NPao=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=T0x4YZ9TaOiAQtHPCulHJQbKIELkN0hIP+YtniNyjj9LUZUx/Ql4er5GQIqLWYD/ngXTfmN4rMtlRhudq9CtMbfL1SzBuzQpQ/+r5bCK5QKLhMuQMT3YPm/FC9QyGmhcEnNTMOFFHgmY3mAeRq8OIsGTBnJ6MdjZJelgCm2KExY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=C1deUzSI; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="C1deUzSI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849854; x=1738385854; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hPTVpULSZEGp+qlRQ7yydOeKGAslemxYjjTwBe0NPao=; b=C1deUzSI5rrujqcoF2dIGoBuWU1jD/6bAhBMdd/Eu15ipU1sopLF73KI uG4A1ldnxqnG6Tvhdb5nGOqgUnzqx+rmRQ8ZeIxwkSsIA9lyHEP2/HD2H ovHHri4FeZ29vhvyNZaY/mGUn/7PbqXjCfRyrw2xoGLDEQUtU8OscwBvj zuC4D5RL6RT/2EFLD8mWL9KS4l1EURmZpFFJJRIYyxrS+PHKc1ThxlH0e vVCaC3XKm8VYcmdW1yPH2dZHDjL/Fr/5SHHMH4B2whLlqlskUe+dxinsI 52IagxZcYw1ptS/5d9SG1/L2LFHi5lovXSTxbpCmMZ6Ntk2qurKol/ryr w==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615834" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615834" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339780" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339780" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:24 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 08/17] vfio/pci: Extract MSI/MSI-X specific code from common flow Date: Thu, 1 Feb 2024 20:57:02 -0800 Message-Id: <79aec9a35a494dcd11be059021a3c4c3f4f74b40.1706849424.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 vfio_intx_set_signal() and vfio_msi_set_signal() have similar code flow mixed with actions specific to the interrupt type. Code that is similar between MSI, MSI-X, and INTx management can be shared instead of duplicated. Start by replacing the MSI/MSI-X specific code within vfio_msi_set_signal() with functions. These functions that are specific to the management of MSI/MSI-X can later be called from a shared flow. Signed-off-by: Reinette Chatre --- Note to maintainers: This is similar to "vfio/pci: Separate frontend and backend code during interrupt enable/disable" that was submitted as part of IMS changes, but is not specific to IMS. https://lore.kernel.org/lkml/cover.1696609476.git.reinette.chatre@intel.com drivers/vfio/pci/vfio_pci_intrs.c | 134 ++++++++++++++++++++---------- 1 file changed, 89 insertions(+), 45 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 7ca2b983b66e..70f2382c9c0c 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -414,26 +414,99 @@ static int vfio_msi_alloc_irq(struct vfio_pci_core_device *vdev, return map.index < 0 ? map.index : map.virq; } +static void vfio_msi_free_interrupt(struct vfio_pci_core_device *vdev, + struct vfio_pci_irq_ctx *ctx, + unsigned int vector) +{ + struct pci_dev *pdev = vdev->pdev; + int irq; + u16 cmd; + + irq = pci_irq_vector(pdev, vector); + cmd = vfio_pci_memory_lock_and_enable(vdev); + free_irq(irq, ctx->trigger); + vfio_pci_memory_unlock_and_restore(vdev, cmd); + /*Interrupt stays allocated, will be freed at MSI/MSI-X disable. */ +} + +static int vfio_msi_request_interrupt(struct vfio_pci_core_device *vdev, + struct vfio_pci_irq_ctx *ctx, + unsigned int vector, unsigned int index) +{ + bool msix = (index == VFIO_PCI_MSIX_IRQ_INDEX) ? true : false; + int irq, ret; + u16 cmd; + + irq = vfio_msi_alloc_irq(vdev, vector, msix); + if (irq < 0) + return irq; + + /* + * If the vector was previously allocated, refresh the on-device + * message data before enabling in case it had been cleared or + * corrupted (e.g. due to backdoor resets) since writing. + */ + cmd = vfio_pci_memory_lock_and_enable(vdev); + if (msix) { + struct msi_msg msg; + + get_cached_msi_msg(irq, &msg); + pci_write_msi_msg(irq, &msg); + } + + ret = request_irq(irq, vfio_msihandler, 0, ctx->name, ctx->trigger); + vfio_pci_memory_unlock_and_restore(vdev, cmd); + + return ret; +} + +static char *vfio_msi_device_name(struct vfio_pci_core_device *vdev, + unsigned int vector, unsigned int index) +{ + struct pci_dev *pdev = vdev->pdev; + + return kasprintf(GFP_KERNEL_ACCOUNT, "vfio-msi%s[%d](%s)", + index == VFIO_PCI_MSIX_IRQ_INDEX ? "x" : "", + vector, pci_name(pdev)); +} + +static void vfio_msi_register_producer(struct vfio_pci_core_device *vdev, + struct vfio_pci_irq_ctx *ctx, + unsigned int vector) +{ + struct pci_dev *pdev = vdev->pdev; + int ret; + + ctx->producer.token = ctx->trigger; + ctx->producer.irq = pci_irq_vector(pdev, vector); + ret = irq_bypass_register_producer(&ctx->producer); + if (unlikely(ret)) { + dev_info(&pdev->dev, + "irq bypass producer (token %p) registration fails: %d\n", + ctx->producer.token, ret); + ctx->producer.token = NULL; + ctx->producer.irq = 0; + } +} + +static void vfio_msi_unregister_producer(struct vfio_pci_irq_ctx *ctx) +{ + irq_bypass_unregister_producer(&ctx->producer); +} + static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, unsigned int vector, int fd, unsigned int index) { - bool msix = (index == VFIO_PCI_MSIX_IRQ_INDEX) ? true : false; - struct pci_dev *pdev = vdev->pdev; struct vfio_pci_irq_ctx *ctx; struct eventfd_ctx *trigger; - int irq = -EINVAL, ret; - u16 cmd; + int ret; ctx = vfio_irq_ctx_get(vdev, vector); if (ctx && ctx->trigger) { - irq_bypass_unregister_producer(&ctx->producer); - irq = pci_irq_vector(pdev, vector); - cmd = vfio_pci_memory_lock_and_enable(vdev); - free_irq(irq, ctx->trigger); - vfio_pci_memory_unlock_and_restore(vdev, cmd); - /* Interrupt stays allocated, will be freed at MSI-X disable. */ + vfio_msi_unregister_producer(ctx); + vfio_msi_free_interrupt(vdev, ctx, vector); kfree(ctx->name); ctx->name = NULL; eventfd_ctx_put(ctx->trigger); @@ -443,13 +516,6 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, if (fd < 0) return 0; - if (irq == -EINVAL) { - /* Interrupt stays allocated, will be freed at MSI-X disable. */ - irq = vfio_msi_alloc_irq(vdev, vector, msix); - if (irq < 0) - return irq; - } - /* Per-interrupt context remain allocated. */ if (!ctx) { ctx = vfio_irq_ctx_alloc(vdev, vector); @@ -457,8 +523,7 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, return -ENOMEM; } - ctx->name = kasprintf(GFP_KERNEL_ACCOUNT, "vfio-msi%s[%d](%s)", - msix ? "x" : "", vector, pci_name(pdev)); + ctx->name = vfio_msi_device_name(vdev, vector, index); if (!ctx->name) return -ENOMEM; @@ -468,40 +533,19 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, goto out_free_name; } - /* - * If the vector was previously allocated, refresh the on-device - * message data before enabling in case it had been cleared or - * corrupted (e.g. due to backdoor resets) since writing. - */ - cmd = vfio_pci_memory_lock_and_enable(vdev); - if (msix) { - struct msi_msg msg; - - get_cached_msi_msg(irq, &msg); - pci_write_msi_msg(irq, &msg); - } + ctx->trigger = trigger; - ret = request_irq(irq, vfio_msihandler, 0, ctx->name, trigger); - vfio_pci_memory_unlock_and_restore(vdev, cmd); + ret = vfio_msi_request_interrupt(vdev, ctx, vector, index); if (ret) goto out_put_eventfd_ctx; - ctx->producer.token = trigger; - ctx->producer.irq = irq; - ret = irq_bypass_register_producer(&ctx->producer); - if (unlikely(ret)) { - dev_info(&pdev->dev, - "irq bypass producer (token %p) registration fails: %d\n", - ctx->producer.token, ret); - - ctx->producer.token = NULL; - } - ctx->trigger = trigger; + vfio_msi_register_producer(vdev, ctx, vector); return 0; out_put_eventfd_ctx: - eventfd_ctx_put(trigger); + eventfd_ctx_put(ctx->trigger); + ctx->trigger = NULL; out_free_name: kfree(ctx->name); ctx->name = NULL; From patchwork Fri Feb 2 04:57:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542063 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E8B1171BB; Fri, 2 Feb 2024 04:57:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849854; cv=none; b=syFgmW23vMRWb1xYVVdBC0PFnyHSjNHhqRBDaQTck0Rb0IAIVubMND1Wahu32ZMmrg5QovuU3tpa0BlTZEhkMFZUNhAzY4aw1FPx1eybKnt4g1If611f3v8Y92pLts6A6VrW65jisri0umEoIvw98lAyRMe/jL4MWUew39e6DT8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849854; c=relaxed/simple; bh=JgRfCrK7Y34Oc8Goi+1GvDvltDbKKC8kZ84KFNDHKXM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eOo035wwKnfEXt2FlpExUTYgkn9HrzRdgwGTX8Jaxt972JML7411UuZ5XjY82PA1voDYUO1BfRXBSPP2pIUhf36jQDdo/Ytp0UJk4O59JSUtbqqS3N0ACOwYkqv92p4dnniWvxIF37VrTGlpS95TOI7zIsaUCBRty5JJXYGO9LQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=jFZ7ofY3; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="jFZ7ofY3" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849852; x=1738385852; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JgRfCrK7Y34Oc8Goi+1GvDvltDbKKC8kZ84KFNDHKXM=; b=jFZ7ofY3rBygnljUqyFOkJzq526CJJ/nxKV/1nxuu5nohx55IcTAe08K sWuSQKPeS0fanu3/a0HwdFMGCoec66NQbOlmoAimFjFCo3Vp7VIMZ7dBL fCwCS+iuT5OwD8HJ4k5P1cla+1FYfYr+9ZWm01/+bubXtrsmeZvNduptx terVCoMDWfhOIwyw08dF00WK/rjxJ2zMLxKiqEfqjmwJudswOANaPsyh2 8ujoXArVthdljtki8DxiNztTpXMN93kWUlF8pDQ7kDLl22+6OHahTtDYr FaPTg9rue0cLz9tcJsmXo8I+oiygsfi8QGU0udKYGRCVNGeSXbjQUHprL g==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615828" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615828" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339783" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339783" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:25 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 09/17] vfio/pci: Converge similar code flows Date: Thu, 1 Feb 2024 20:57:03 -0800 Message-Id: <250808eb11a206075c0b92f2dae23a924f73c390.1706849424.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 vfio_intx_set_signal() and vfio_msi_set_vector_signal() have similar code flows that can be shared. Modify vfio_intx_set_signal()'s signature to resemble that of vfio_msi_set_vector_signal() by also providing the vector and interrupt type as parameters. For vfio_intx_set_signal() these two additional parameters are redundant and unused since the vector is always 0 and the interrupt type is always INTx. Adding these parameters make it possible to refactor vfio_intx_set_signal() to call INTx specific functions with the same signature as the MSI/MSI-X specific functions in preparation for these interrupt type specific functions to be called by shared code. Signed-off-by: Reinette Chatre --- drivers/vfio/pci/vfio_pci_intrs.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 70f2382c9c0c..98b099f58b2b 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -260,7 +260,9 @@ static int vfio_intx_enable(struct vfio_pci_core_device *vdev) return 0; } -static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, int fd) +static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, + unsigned int vector, int fd, + unsigned int index) { struct pci_dev *pdev = vdev->pdev; unsigned long irqflags = IRQF_SHARED; @@ -330,7 +332,7 @@ static void vfio_intx_disable(struct vfio_pci_core_device *vdev) vfio_virqfd_disable(&ctx->unmask); vfio_virqfd_disable(&ctx->mask); } - vfio_intx_set_signal(vdev, -1); + vfio_intx_set_signal(vdev, 0, -1, VFIO_PCI_INTX_IRQ_INDEX); vdev->irq_type = VFIO_PCI_NUM_IRQS; vfio_irq_ctx_free(vdev, ctx, 0); } @@ -674,13 +676,13 @@ static int vfio_pci_set_intx_trigger(struct vfio_pci_core_device *vdev, int ret; if (is_intx(vdev)) - return vfio_intx_set_signal(vdev, fd); + return vfio_intx_set_signal(vdev, start, fd, index); ret = vfio_intx_enable(vdev); if (ret) return ret; - ret = vfio_intx_set_signal(vdev, fd); + ret = vfio_intx_set_signal(vdev, start, fd, index); if (ret) vfio_intx_disable(vdev); From patchwork Fri Feb 2 04:57:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542065 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA8E417983; Fri, 2 Feb 2024 04:57:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849856; cv=none; b=RabVAZE/C8yf7XPAd+KndCYXo5c7P/F9L0irhmg9J6Kgf3MmiKKsXNBsnbGGwjLKU9arG/z+p3Fg+B8hZV9Evx1yYEVkzO+XwVgWCb7//GuyLhVqgezhdaFYSvcek435Z1aLMAQ4tDl0u4RFbBGukz8dLkOtxzQtlsKiECPPhJ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849856; c=relaxed/simple; bh=HLRJyna+Zk3jXF4APHd23iO3eqvDn9o+9jiToz4XQtQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=L/rZGJ9k9XNBOCxb4oq1RWF6em9XXXBJ8HhBvmh0Z5tpDPRR+fu6tJiTBQQTWC0orhLViMWEzyeoWyZ387/dbLGl5KJd5/7HhQvZa7lfMeZ6cHRDzWR0hwFz0T0I+nq8H1m1vE1bgPSCcAeaugz2HwuBQt4hhgdqnHR30FaEDC8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=bZzF8kbn; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="bZzF8kbn" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849854; x=1738385854; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HLRJyna+Zk3jXF4APHd23iO3eqvDn9o+9jiToz4XQtQ=; b=bZzF8kbnFfD6ePrnb1FsH6pXAI2YzADRBwBZA0v9mcDNsQWeEYUNmnm5 e95AfmeLenQqQ2WoirBQESi2gtAwOMIReql3Ln1oRW6GGbDr9q2zEhGsW SGXtIrXtAFqkAuqaPpIkLH68xCt//WHX7OYbY41gPG+51inJD1Z/4TsTn cf+J13FesttL1viAD/QnFaOxJJLV+NXIaWcIgp1GQI0ZxqYpmSsLVNEY6 1lSmXPT6/RSN89A2ZjZ4gcuTlNnRiiVZbVws5i/ZciGzTEpUjcMfGio2W H0I6iQPROaqdX8rzy76iUvB4Btz0srAjajHXQuspTNYE4HkcRLCRHWrsA g==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615840" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615840" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339786" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339786" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:25 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 10/17] vfio/pci: Extract INTx specific code from vfio_intx_set_signal() Date: Thu, 1 Feb 2024 20:57:04 -0800 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 vfio_intx_set_signal() and vfio_msi_set_vector_signal() use the same code flow for INTx, MSI, and MSI-X interrupt management. Extract the INTx specific code from vfio_intx_set_signal() to leave behind the same flow as vfio_msi_set_vector_signal(), ready for sharing. Signed-off-by: Reinette Chatre --- drivers/vfio/pci/vfio_pci_intrs.c | 69 ++++++++++++++++++++++--------- 1 file changed, 49 insertions(+), 20 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 98b099f58b2b..6eef4e2d7c13 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -260,15 +260,58 @@ static int vfio_intx_enable(struct vfio_pci_core_device *vdev) return 0; } +static void vfio_intx_free_interrupt(struct vfio_pci_core_device *vdev, + struct vfio_pci_irq_ctx *ctx, + unsigned int vector) +{ + struct pci_dev *pdev = vdev->pdev; + + free_irq(pdev->irq, vdev); +} + +static int vfio_intx_request_interrupt(struct vfio_pci_core_device *vdev, + struct vfio_pci_irq_ctx *ctx, + unsigned int vector, unsigned int index) +{ + unsigned long irqflags = IRQF_SHARED; + struct pci_dev *pdev = vdev->pdev; + unsigned long flags; + int ret; + + if (!vdev->pci_2_3) + irqflags = 0; + + ret = request_irq(pdev->irq, vfio_intx_handler, irqflags, + ctx->name, vdev); + if (ret) + return ret; + + /* + * INTx disable will stick across the new irq setup, + * disable_irq won't. + */ + spin_lock_irqsave(&vdev->irqlock, flags); + if (!vdev->pci_2_3 && ctx->masked) + disable_irq_nosync(pdev->irq); + spin_unlock_irqrestore(&vdev->irqlock, flags); + + return 0; +} + +static char *vfio_intx_device_name(struct vfio_pci_core_device *vdev, + unsigned int vector, unsigned int index) +{ + struct pci_dev *pdev = vdev->pdev; + + return kasprintf(GFP_KERNEL_ACCOUNT, "vfio-intx(%s)", pci_name(pdev)); +} + static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, unsigned int vector, int fd, unsigned int index) { - struct pci_dev *pdev = vdev->pdev; - unsigned long irqflags = IRQF_SHARED; struct vfio_pci_irq_ctx *ctx; struct eventfd_ctx *trigger; - unsigned long flags; int ret; ctx = vfio_irq_ctx_get(vdev, 0); @@ -276,7 +319,7 @@ static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, return -EINVAL; if (ctx->trigger) { - free_irq(pdev->irq, vdev); + vfio_intx_free_interrupt(vdev, ctx, vector); kfree(ctx->name); eventfd_ctx_put(ctx->trigger); ctx->trigger = NULL; @@ -285,8 +328,7 @@ static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, if (fd < 0) /* Disable only */ return 0; - ctx->name = kasprintf(GFP_KERNEL_ACCOUNT, "vfio-intx(%s)", - pci_name(pdev)); + ctx->name = vfio_intx_device_name(vdev, vector, index); if (!ctx->name) return -ENOMEM; @@ -298,11 +340,7 @@ static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, ctx->trigger = trigger; - if (!vdev->pci_2_3) - irqflags = 0; - - ret = request_irq(pdev->irq, vfio_intx_handler, - irqflags, ctx->name, vdev); + ret = vfio_intx_request_interrupt(vdev, ctx, vector, index); if (ret) { ctx->trigger = NULL; kfree(ctx->name); @@ -310,15 +348,6 @@ static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, return ret; } - /* - * INTx disable will stick across the new irq setup, - * disable_irq won't. - */ - spin_lock_irqsave(&vdev->irqlock, flags); - if (!vdev->pci_2_3 && ctx->masked) - disable_irq_nosync(pdev->irq); - spin_unlock_irqrestore(&vdev->irqlock, flags); - return 0; } From patchwork Fri Feb 2 04:57:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542066 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 515D0179B7; Fri, 2 Feb 2024 04:57:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849856; cv=none; b=lxeWo8LsegXjZv1ulID0YLmfMzFWY7o7V/QPrIM3b35u3v8GUhE13qisy5oV/Wterg1+FyV5y7oQTaM53rN/PoOMLNNgcded2T7EJgcXu4K5H7nEbo0DsD9EUec+TcuHaZ/aY+83fmD+QcBErNBQtwk7dHvBtsa4xl5h0DiS1G0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849856; c=relaxed/simple; bh=i1ZuVfKb6uZ0Dp+3vSDXltmIsz2QMq5Yyj+6aoiLHUY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Fd2PKC+6swxOwSZFNR7m4tWHdCYThMFNG109HHMiGorhIVK9SdatH//cTZFuJpJQflGe/X+GmkG7mLpuuG40exlcw/XiMev4sn1F1Vn1zV/tclxcNXSybaoUFl9eyNiwcm4MpNit0WMFqbtgtJD0jiaMLM23435JpaghyMMSqwk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=dalA8Zbf; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="dalA8Zbf" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849854; x=1738385854; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=i1ZuVfKb6uZ0Dp+3vSDXltmIsz2QMq5Yyj+6aoiLHUY=; b=dalA8Zbf82p2Sq03WFvifSQojFcgyf91WvqmKPKvKSP16WfL1vYsPixX ROJkVtXcxd07xgVRkFEG3Kh8aghjKmCUpt4UK/sUJNE1MHYt32mPiZ3ld zToiuuIagDVIrdJkX2mF+88kRTOg/DpHhBOXqDK1BuegdPnxPPFJYvgpb AHjaHZqiTy/iqj+mftwANGf7RCM95DOub8UoI0AG1bxffqkjk9XHf69Ol ULkfkoj3KPqdmTN5qoFR++AYSdMxnzvKyYoM4wBr00Ii1LsmHeyX+jNqs G50Gsko6dXTxjHz4b5GwLWqeIELMo4iq2SoiwbQdS+9eTFygZisIZjGp2 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615848" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615848" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339789" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339789" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:25 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 11/17] vfio/pci: Perform MSI/MSI-X interrupt management via callbacks Date: Thu, 1 Feb 2024 20:57:05 -0800 Message-Id: <4e04371e86722f8e2d867fe813a67c10f48222af.1706849424.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 vfio_msi_set_vector_signal() is specific to MSI and MSI-X interrupt management but its flow is the same as vfio_intx_set_signal() that manages the INTx interrupts. Replace the MSI and MSI-X specific calls with callbacks in preparation for vfio_msi_set_vector_signal() to manage INTx interrupts also. In preparation for support of INTx only the IRQ bypass code is made optional. Signed-off-by: Reinette Chatre --- Note to maintainers: This change resembles "vfio/pci: Replace backend specific calls with callbacks" that formed part of the IMS submission, but is not specific to IMS. https://lore.kernel.org/lkml/cover.1696609476.git.reinette.chatre@intel.com drivers/vfio/pci/vfio_pci_intrs.c | 44 +++++++++++++++++++++++++++---- 1 file changed, 39 insertions(+), 5 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 6eef4e2d7c13..07dc388c4513 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -31,6 +31,21 @@ struct vfio_pci_irq_ctx { struct irq_bypass_producer producer; }; +struct vfio_pci_intr_ops { + int (*request_interrupt)(struct vfio_pci_core_device *vdev, + struct vfio_pci_irq_ctx *ctx, + unsigned int vector, unsigned int index); + void (*free_interrupt)(struct vfio_pci_core_device *vdev, + struct vfio_pci_irq_ctx *ctx, + unsigned int vector); + char *(*device_name)(struct vfio_pci_core_device *vdev, + unsigned int vector, unsigned int index); + void (*register_producer)(struct vfio_pci_core_device *vdev, + struct vfio_pci_irq_ctx *ctx, + unsigned int vector); + void (*unregister_producer)(struct vfio_pci_irq_ctx *ctx); +}; + static bool irq_is(struct vfio_pci_core_device *vdev, int type) { return vdev->irq_type == type; @@ -525,6 +540,23 @@ static void vfio_msi_unregister_producer(struct vfio_pci_irq_ctx *ctx) irq_bypass_unregister_producer(&ctx->producer); } +static struct vfio_pci_intr_ops intr_ops[] = { + [VFIO_PCI_MSI_IRQ_INDEX] = { + .request_interrupt = vfio_msi_request_interrupt, + .free_interrupt = vfio_msi_free_interrupt, + .device_name = vfio_msi_device_name, + .register_producer = vfio_msi_register_producer, + .unregister_producer = vfio_msi_unregister_producer, + }, + [VFIO_PCI_MSIX_IRQ_INDEX] = { + .request_interrupt = vfio_msi_request_interrupt, + .free_interrupt = vfio_msi_free_interrupt, + .device_name = vfio_msi_device_name, + .register_producer = vfio_msi_register_producer, + .unregister_producer = vfio_msi_unregister_producer, + }, +}; + static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, unsigned int vector, int fd, unsigned int index) @@ -536,8 +568,9 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, ctx = vfio_irq_ctx_get(vdev, vector); if (ctx && ctx->trigger) { - vfio_msi_unregister_producer(ctx); - vfio_msi_free_interrupt(vdev, ctx, vector); + if (intr_ops[index].unregister_producer) + intr_ops[index].unregister_producer(ctx); + intr_ops[index].free_interrupt(vdev, ctx, vector); kfree(ctx->name); ctx->name = NULL; eventfd_ctx_put(ctx->trigger); @@ -554,7 +587,7 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, return -ENOMEM; } - ctx->name = vfio_msi_device_name(vdev, vector, index); + ctx->name = intr_ops[index].device_name(vdev, vector, index); if (!ctx->name) return -ENOMEM; @@ -566,11 +599,12 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, ctx->trigger = trigger; - ret = vfio_msi_request_interrupt(vdev, ctx, vector, index); + ret = intr_ops[index].request_interrupt(vdev, ctx, vector, index); if (ret) goto out_put_eventfd_ctx; - vfio_msi_register_producer(vdev, ctx, vector); + if (intr_ops[index].register_producer) + intr_ops[index].register_producer(vdev, ctx, vector); return 0; From patchwork Fri Feb 2 04:57:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542067 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9195418041; Fri, 2 Feb 2024 04:57:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849857; cv=none; b=rcz6hQj7h8OM7DbcHTx+sjC5XFGN3nwCYeOgBLiL99b/1pEbB2pdnYOE1nYQirfYGcFZSAFRdaqvwtRX5yH7IOM5pvAeH2ljhWK+ZR5LDv8OZkvUJrDJQruYN8CcGcACoUyEtqADoONbSxTPtQhkUZyCIVlObwjnw+w7bssfmp8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849857; c=relaxed/simple; bh=BJ/iiqpnrpLGQqepNldAyRvzSYt7cnd2BvmP/2lcET0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fixUsKsK5oQV6mr5/DwOUb1ynkkz9TGw6b95j2wpFzGigIVq8xpPQ5gRvYS+fEF/Zp2ORnuArN5lgiN+2okn8MXY5V7rBrgyGMd00ECApnNPwkEeka9OQllcjCK/uZtjmA7d1uTILYghM8J8aj+vzGl9c6nYND/KtGgF9AdJKkY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=RCFKid6y; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="RCFKid6y" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849856; x=1738385856; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BJ/iiqpnrpLGQqepNldAyRvzSYt7cnd2BvmP/2lcET0=; b=RCFKid6yYwTVoGcBw1kh22Wld3UKLnA7bJB62o1/ztSGHhf9W9Pxe3f1 kuVQhL+OSwKOO4JDCo1R6VhD+zwu1fQWhSd9nxveurjDcaE4VEl5KVuKT CLP9W3nK4R83+v713bkXtvJVG5Tg0MrAw/ZvJCvxulzoufLclloSetw19 f9HgWT+7VAa8YknDDaxMXY6dRR0l5W6HTuAjNEDAHJCAuW2C9M7/vD388 m5xUnfSogPI/QRDKGtG9LZgUD9eeMqAxyjWdF+g4r8KyriY+f8zsuqaH2 bSVKfNR1ljh6T6YuMz7dk09vFsEbEiLwAFwXBIrN5rm40vJmCNSnDg//u A==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615850" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615850" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339792" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339792" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:25 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 12/17] vfio/pci: Remove msi term from generic code Date: Thu, 1 Feb 2024 20:57:06 -0800 Message-Id: <0550572e64505df6ecff0b08f1eca869a79f6acf.1706849424.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 vfio_msi_set_vector_signal() and by extension vfio_msi_set_block() are no longer specific to MSI and MSI-X interrupts. Change the name of these functions in preparation for them to be used for management of INTx interrupts. Signed-off-by: Reinette Chatre --- drivers/vfio/pci/vfio_pci_intrs.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 07dc388c4513..7f9dc81cb97f 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -557,7 +557,7 @@ static struct vfio_pci_intr_ops intr_ops[] = { }, }; -static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, +static int vfio_irq_set_vector_signal(struct vfio_pci_core_device *vdev, unsigned int vector, int fd, unsigned int index) { @@ -617,7 +617,7 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, return ret; } -static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, +static int vfio_irq_set_block(struct vfio_pci_core_device *vdev, unsigned int start, unsigned int count, int32_t *fds, unsigned int index) { @@ -626,12 +626,12 @@ static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, for (i = 0, j = start; i < count && !ret; i++, j++) { int fd = fds ? fds[i] : -1; - ret = vfio_msi_set_vector_signal(vdev, j, fd, index); + ret = vfio_irq_set_vector_signal(vdev, j, fd, index); } if (ret) { for (i = start; i < j; i++) - vfio_msi_set_vector_signal(vdev, i, -1, index); + vfio_irq_set_vector_signal(vdev, i, -1, index); } return ret; @@ -648,7 +648,7 @@ static void vfio_msi_disable(struct vfio_pci_core_device *vdev, xa_for_each(&vdev->ctx, i, ctx) { vfio_virqfd_disable(&ctx->unmask); vfio_virqfd_disable(&ctx->mask); - vfio_msi_set_vector_signal(vdev, i, -1, index); + vfio_irq_set_vector_signal(vdev, i, -1, index); vfio_irq_ctx_free(vdev, ctx, i); } @@ -786,14 +786,14 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, int ret; if (vdev->irq_type == index) - return vfio_msi_set_block(vdev, start, count, + return vfio_irq_set_block(vdev, start, count, fds, index); ret = vfio_msi_enable(vdev, start + count, index); if (ret) return ret; - ret = vfio_msi_set_block(vdev, start, count, fds, index); + ret = vfio_irq_set_block(vdev, start, count, fds, index); if (ret) vfio_msi_disable(vdev, index); From patchwork Fri Feb 2 04:57:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542069 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3216A182D4; Fri, 2 Feb 2024 04:57:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849859; cv=none; b=qiflQxr7kNAkOKh+lSr1wtZaO9cOrKxInGD7xDSRYIifYMUJRPoJoHfX2lRvu9l8NTFNXW4DZ15vgaC/LwJlsw7DAlqhs7bQBdw4K1VHsZRDjym/0j4QJCC2SuXhAuyZHW+H371b2sP+1CDg+3fMuCQTBGJycu9CPwON2VUZQg0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849859; c=relaxed/simple; bh=j7fvY/2gZZFN9ycFck7Q2a4mMpEwf8X0nASlEstU/wQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ijdcj7KlvoyMKgmkXw9RNdv/vG0x9kABH5vZDdSsY+044DdkZFaFMSNAEzDeNr1YzsDSFOKb7oo4RaW3Qe6fyYo6oWqXhWrhB9Ig9BT7JBZqFZP+81k4BsiEd6pie/iWBncyCQ3b7mz97CBOyvKLBa0zNai0K6wcDOegMlel3po= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=TRPBhh8h; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="TRPBhh8h" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849856; x=1738385856; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=j7fvY/2gZZFN9ycFck7Q2a4mMpEwf8X0nASlEstU/wQ=; b=TRPBhh8hVWlGV3pVkqhREqPlwYQoEnaoYL57eSL30nFAo//s+AfSb5Un h6z9VeDM44LX7ct0u7KKB35V0pUjAKojZ6ssRcKzzyYi+0UwSHEf9+OXH WhCds5Fi1nVYQ/o56Tv6CFelvyPH0TWgMgEXVsVBtm21wASFiGmlDE3RQ Wry9JsrCWQDr+yDo69SPJETMGWNZKKgXr95Wm6UbrSizmKA5Nj48xVPk1 kCBG+CYf+oiqNNQ8beymRHW9+9GqAogK44A6lahm3FUrtWga1i1Jn7C+R iyfLIbegFFzLxMEWmv2057ucC43+L+IAo7iIaxY8cySA60GpvqNl/Deso w==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615858" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615858" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339794" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339794" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:25 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 13/17] vfio/pci: Remove vfio_intx_set_signal() Date: Thu, 1 Feb 2024 20:57:07 -0800 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The interrupt management flow of vfio_intx_set_signal() is available from the now generic vfio_irq_set_signal(). Initialize the INTx specific management ops so that vfio_irq_set_signal() can be used to manage INTx interrupts and point all existing INTx specific interrupt management calls to vfio_irq_set_signal(). Use vfio_irq_set_block() within vfio_pci_set_intx_trigger() to highlight its similarities with vfio_pci_set_msi_trigger() for the next stage of uniting the interrupt management code. vfio_pci_set_intx_trigger() ensures that start == 0 and count == 1 before vfio_irq_set_block() is called so the loop within it is essentially a direct call to vfio_irq_set_vector_signal(). Signed-off-by: Reinette Chatre --- drivers/vfio/pci/vfio_pci_intrs.c | 62 +++++++------------------------ 1 file changed, 13 insertions(+), 49 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 7f9dc81cb97f..d7c2cd739d74 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -46,6 +46,10 @@ struct vfio_pci_intr_ops { void (*unregister_producer)(struct vfio_pci_irq_ctx *ctx); }; +static int vfio_irq_set_vector_signal(struct vfio_pci_core_device *vdev, + unsigned int vector, int fd, + unsigned int index); + static bool irq_is(struct vfio_pci_core_device *vdev, int type) { return vdev->irq_type == type; @@ -321,51 +325,6 @@ static char *vfio_intx_device_name(struct vfio_pci_core_device *vdev, return kasprintf(GFP_KERNEL_ACCOUNT, "vfio-intx(%s)", pci_name(pdev)); } -static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, - unsigned int vector, int fd, - unsigned int index) -{ - struct vfio_pci_irq_ctx *ctx; - struct eventfd_ctx *trigger; - int ret; - - ctx = vfio_irq_ctx_get(vdev, 0); - if (WARN_ON_ONCE(!ctx)) - return -EINVAL; - - if (ctx->trigger) { - vfio_intx_free_interrupt(vdev, ctx, vector); - kfree(ctx->name); - eventfd_ctx_put(ctx->trigger); - ctx->trigger = NULL; - } - - if (fd < 0) /* Disable only */ - return 0; - - ctx->name = vfio_intx_device_name(vdev, vector, index); - if (!ctx->name) - return -ENOMEM; - - trigger = eventfd_ctx_fdget(fd); - if (IS_ERR(trigger)) { - kfree(ctx->name); - return PTR_ERR(trigger); - } - - ctx->trigger = trigger; - - ret = vfio_intx_request_interrupt(vdev, ctx, vector, index); - if (ret) { - ctx->trigger = NULL; - kfree(ctx->name); - eventfd_ctx_put(trigger); - return ret; - } - - return 0; -} - static void vfio_intx_disable(struct vfio_pci_core_device *vdev) { struct vfio_pci_irq_ctx *ctx; @@ -376,7 +335,7 @@ static void vfio_intx_disable(struct vfio_pci_core_device *vdev) vfio_virqfd_disable(&ctx->unmask); vfio_virqfd_disable(&ctx->mask); } - vfio_intx_set_signal(vdev, 0, -1, VFIO_PCI_INTX_IRQ_INDEX); + vfio_irq_set_vector_signal(vdev, 0, -1, VFIO_PCI_INTX_IRQ_INDEX); vdev->irq_type = VFIO_PCI_NUM_IRQS; vfio_irq_ctx_free(vdev, ctx, 0); } @@ -541,6 +500,11 @@ static void vfio_msi_unregister_producer(struct vfio_pci_irq_ctx *ctx) } static struct vfio_pci_intr_ops intr_ops[] = { + [VFIO_PCI_INTX_IRQ_INDEX] = { + .request_interrupt = vfio_intx_request_interrupt, + .free_interrupt = vfio_intx_free_interrupt, + .device_name = vfio_intx_device_name, + }, [VFIO_PCI_MSI_IRQ_INDEX] = { .request_interrupt = vfio_msi_request_interrupt, .free_interrupt = vfio_msi_free_interrupt, @@ -735,17 +699,17 @@ static int vfio_pci_set_intx_trigger(struct vfio_pci_core_device *vdev, return -EINVAL; if (flags & VFIO_IRQ_SET_DATA_EVENTFD) { - int32_t fd = *(int32_t *)data; + int32_t *fds = data; int ret; if (is_intx(vdev)) - return vfio_intx_set_signal(vdev, start, fd, index); + return vfio_irq_set_block(vdev, start, count, fds, index); ret = vfio_intx_enable(vdev); if (ret) return ret; - ret = vfio_intx_set_signal(vdev, start, fd, index); + ret = vfio_irq_set_block(vdev, start, count, fds, index); if (ret) vfio_intx_disable(vdev); From patchwork Fri Feb 2 04:57:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542068 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41EC618624; Fri, 2 Feb 2024 04:57:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849858; cv=none; b=QyCxGrk+fBd6N2pMgNcvi17owKZmtZkU+gnpgYQSZ/Yzm4wh96DDdJAwytS9G6/JYlQZNrBczX8NuDDF/GkZmOgOZeksaIjsfwtV486IMakQmcQ9HzMf3dimo4XFIq5cNoCORQ9qIGxEnlrvVLDLqWxB8rfSbEMIwO0k/PIUmLw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849858; c=relaxed/simple; bh=QAj3VSDW4vnn+85iNztQ9s3NyXx5nCvtMRrW9/sf7+4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=HWkFRrw6VoeB6VWtLHOrEwKDKvZgskeutYlvEovIIHmgMeY1UneSJajkClnMTN8bO0SZexT0PqkOYrU5yx2xYRhN+dYkrcy/9l64fgLy+X74Ec+dox3oBodGO9mfiZOh8MkOrPGU3xVK5sDacWn7JLBLdcNL4ewbLtKwXThUk/U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=aIaAUh6M; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="aIaAUh6M" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849856; x=1738385856; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QAj3VSDW4vnn+85iNztQ9s3NyXx5nCvtMRrW9/sf7+4=; b=aIaAUh6MXBBKLpWpRk3Va8xm9/sQUBNqoX0OxgOO9TqtbxSC6mJzmZt4 Lg5Grropsi1eCTx4oYLEDYaek4K282UtnbOQxfH+yqs9qFNkm2WoBF+ap s3cRkUFU+hgQasbNe5cqSx4NHFkwNTrW82155OBPzXkdLykwEV3u4uAWv Wqkis985A2AE/rwv1lkRag8drbTlYBBR8rMj6k1BJzPhe7juIALgOzRiQ Yb7W/53py2KEivNlqquzUl8SSwWUnEKshTvljXYM7k/hihIP6YKaw/bRm EOxodCJpfSw3sE0a605iT1qTYDhi4PAhnZ9ihd8YqJdrDwEPNbJPa2dQe Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615861" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615861" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339798" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339798" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:25 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 14/17] vfio/pci: Add utility to trigger INTx eventfd knowing interrupt context Date: Thu, 1 Feb 2024 20:57:08 -0800 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 vfio_send_intx_eventfd() is available to signal the eventfd associated with the INTx interrupt. It does so by first obtaining the vector's interrupt context and then signaling the eventfd. The interrupt context may already be available before vfio_send_intx_eventfd() is called, making the additional query unnecessary. Introduce vfio_send_intx_eventfd_ctx() that can be used to signal the eventfd associated with the INTx interrupt when the interrupt context is already known and use it instead of vfio_send_intx_eventfd() in the one instance where the interrupt context is already known. Replace usage of vfio_send_intx_eventfd() within vfio_pci_set_intx_trigger() with a new snippet that results in the same functionality while mirroring the flow vfio_pci_set_msi_trigger() as a preparatory step to merge vfio_pci_set_msi_trigger() and vfio_pci_set_intx_trigger(). The new snippet is marked as "temporary" until the flows are merged. Signed-off-by: Reinette Chatre --- drivers/vfio/pci/vfio_pci_intrs.c | 32 ++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index d7c2cd739d74..37065623d286 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -103,6 +103,13 @@ vfio_irq_ctx_alloc(struct vfio_pci_core_device *vdev, unsigned long index) /* * INTx */ +static void vfio_send_intx_eventfd_ctx(struct vfio_pci_core_device *vdev, + struct vfio_pci_irq_ctx *ctx) +{ + if (likely(is_intx(vdev) && !vdev->virq_disabled)) + eventfd_signal(ctx->trigger); +} + static void vfio_send_intx_eventfd(void *opaque, void *unused) { struct vfio_pci_core_device *vdev = opaque; @@ -245,7 +252,7 @@ static irqreturn_t vfio_intx_handler(int irq, void *dev_id) spin_unlock_irqrestore(&vdev->irqlock, flags); if (ret == IRQ_HANDLED) - vfio_send_intx_eventfd(vdev, NULL); + vfio_send_intx_eventfd_ctx(vdev, ctx); return ret; } @@ -690,6 +697,9 @@ static int vfio_pci_set_intx_trigger(struct vfio_pci_core_device *vdev, unsigned int count, uint32_t flags, void *data) { + struct vfio_pci_irq_ctx *ctx; + unsigned int i; + if (is_intx(vdev) && !count && (flags & VFIO_IRQ_SET_DATA_NONE)) { vfio_intx_disable(vdev); return 0; @@ -719,13 +729,21 @@ static int vfio_pci_set_intx_trigger(struct vfio_pci_core_device *vdev, if (!is_intx(vdev)) return -EINVAL; - if (flags & VFIO_IRQ_SET_DATA_NONE) { - vfio_send_intx_eventfd(vdev, NULL); - } else if (flags & VFIO_IRQ_SET_DATA_BOOL) { - uint8_t trigger = *(uint8_t *)data; - if (trigger) - vfio_send_intx_eventfd(vdev, NULL); + /* temporary */ + for (i = start; i < start + count; i++) { + ctx = vfio_irq_ctx_get(vdev, i); + if (!ctx || !ctx->trigger) + continue; + if (flags & VFIO_IRQ_SET_DATA_NONE) { + vfio_send_intx_eventfd_ctx(vdev, ctx); + } else if (flags & VFIO_IRQ_SET_DATA_BOOL) { + uint8_t *bools = data; + + if (bools[i - start]) + vfio_send_intx_eventfd_ctx(vdev, ctx); + } } + return 0; } From patchwork Fri Feb 2 04:57:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542070 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1081199C7; Fri, 2 Feb 2024 04:57:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849859; cv=none; b=Txy4UqsYPqzjUiTTEoNnm+YFJy2tvB+YdUOEcvg0UUDZnhEDHDQfnQOjoUeqereswXbqA6MYzZK7CwiUJ5qMw7wfOvRZlbLITI4IZqrHrIzoVrMA3XKGKMrtq+l3CgVYAEK+NpoAI9grxkWzT9D+wPySujsU6xapQPYM9D44xvU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849859; c=relaxed/simple; bh=UPkU4CgiMaVZ/F/r+r3Za8ed6ay66IJYSHKlYHufPks=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=j2iomrYfDzplSmc79ibCkHiw1dfjY9QyEp1YpIJG6Jjuyiu6FqYzKUWqwYxEyPYpo/r7F7dGrpWhQ41inZB4owRF2ODBwHUMFg3u4NxjBTSsLcs2MxNVYsqHPg7imCMgs26GK6yMvTP/4umfZAOnjY7e7hDHWjJ2Ov8MkukaOl8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=T0w3GHBm; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="T0w3GHBm" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849858; x=1738385858; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UPkU4CgiMaVZ/F/r+r3Za8ed6ay66IJYSHKlYHufPks=; b=T0w3GHBm46+6MvQbdSXdnZ7OTOnlxg64zn0FSGiOt7m67z5m4fO3tKGI E2xD9Os5j2oFs7n6yCQmqyqdtUETeIg3gySlphB8eN5PW8u7bK9rMqFvt NSeUKzcd8Nk+kDVjFUYEJ/f1I13tkMECHftAcII/6kVwa08dR4t0A7T9/ BovkDpAW3t4yyHLEnthfAmhj2GDSWgsWo6NWq/xFaXv7nOIxzSgoENc8v OyZErmmUuux8BSqbgxvaqlDAEFFTD4LXvNf3hMjDShpAbOWytnIlSDEPN 9qINJYIK+PZi0H63NNZF7igfNkMZ6Lu7SIkWjTk1xRqgXEm0bTZrvm22a g==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615868" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615868" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339801" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339801" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:25 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 15/17] vfio/pci: Let enable and disable of interrupt types use same signature Date: Thu, 1 Feb 2024 20:57:09 -0800 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 vfio_pci_set_intx_trigger() and vfio_pci_set_msi_trigger() have flows that can be shared. For INTx, MSI, and MSI-X interrupt management to share the same enable/disable flow the interrupt specific enable and disable functions should have the same signatures. Let vfio_intx_enable() and vfio_msi_enable() use the same parameters by passing "start" and "count" to these functions instead of letting the (what will eventually be) common code interpret these values. Providing "start" and "count" to vfio_intx_enable() enables the INTx specific check of these parameters to move into the INTx specific vfio_intx_enable(). Similarly, providing "start" and "count" to vfio_msi_enable() enables the MSI/MSI-X specific code to initialize number of vectors needed. The shared MSI/MSI-X code needs the interrupt index. Provide the interrupt index (clearly marked as unused) to the INTx code to use the same signatures. With interrupt type specific code using the same parameters it is possible to have common code that calls the enable/disable code for different interrupt types. Signed-off-by: Reinette Chatre --- drivers/vfio/pci/vfio_pci_intrs.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 37065623d286..9217fea3f636 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -257,13 +257,18 @@ static irqreturn_t vfio_intx_handler(int irq, void *dev_id) return ret; } -static int vfio_intx_enable(struct vfio_pci_core_device *vdev) +static int vfio_intx_enable(struct vfio_pci_core_device *vdev, + unsigned int start, unsigned int count, + unsigned int __always_unused index) { struct vfio_pci_irq_ctx *ctx; if (!is_irq_none(vdev)) return -EINVAL; + if (start != 0 || count != 1) + return -EINVAL; + if (!vdev->pdev->irq) return -ENODEV; @@ -332,7 +337,8 @@ static char *vfio_intx_device_name(struct vfio_pci_core_device *vdev, return kasprintf(GFP_KERNEL_ACCOUNT, "vfio-intx(%s)", pci_name(pdev)); } -static void vfio_intx_disable(struct vfio_pci_core_device *vdev) +static void vfio_intx_disable(struct vfio_pci_core_device *vdev, + unsigned int __always_unused index) { struct vfio_pci_irq_ctx *ctx; @@ -358,17 +364,20 @@ static irqreturn_t vfio_msihandler(int irq, void *arg) return IRQ_HANDLED; } -static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, +static int vfio_msi_enable(struct vfio_pci_core_device *vdev, + unsigned int start, unsigned int count, unsigned int index) { struct pci_dev *pdev = vdev->pdev; unsigned int flag; - int ret; + int ret, nvec; u16 cmd; if (!is_irq_none(vdev)) return -EINVAL; + nvec = start + count; + flag = (index == VFIO_PCI_MSIX_IRQ_INDEX) ? PCI_IRQ_MSIX : PCI_IRQ_MSI; /* return the number of supported vectors if we can't get all: */ @@ -701,11 +710,11 @@ static int vfio_pci_set_intx_trigger(struct vfio_pci_core_device *vdev, unsigned int i; if (is_intx(vdev) && !count && (flags & VFIO_IRQ_SET_DATA_NONE)) { - vfio_intx_disable(vdev); + vfio_intx_disable(vdev, index); return 0; } - if (!(is_intx(vdev) || is_irq_none(vdev)) || start != 0 || count != 1) + if (!(is_intx(vdev) || is_irq_none(vdev))) return -EINVAL; if (flags & VFIO_IRQ_SET_DATA_EVENTFD) { @@ -715,13 +724,13 @@ static int vfio_pci_set_intx_trigger(struct vfio_pci_core_device *vdev, if (is_intx(vdev)) return vfio_irq_set_block(vdev, start, count, fds, index); - ret = vfio_intx_enable(vdev); + ret = vfio_intx_enable(vdev, start, count, index); if (ret) return ret; ret = vfio_irq_set_block(vdev, start, count, fds, index); if (ret) - vfio_intx_disable(vdev); + vfio_intx_disable(vdev, index); return ret; } @@ -771,7 +780,7 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, return vfio_irq_set_block(vdev, start, count, fds, index); - ret = vfio_msi_enable(vdev, start + count, index); + ret = vfio_msi_enable(vdev, start, count, index); if (ret) return ret; From patchwork Fri Feb 2 04:57:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542071 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5955B1758D; Fri, 2 Feb 2024 04:57:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849860; cv=none; b=A+B8Vf44CAXbSdUt3VBr39Z09IUQc2BdaHpfU7ub+C/b6phDdUKvyXLCu2c2vK5L1aRJjOJuIuRZny5bJ2MFX7kB8CuhhrfgRcT5zuy1DBQGGU7FZPlBKaJ3GSGU0XHqVzsXfoixs/k1A6+Jl4pWhsnQYRyNsTCQzAyR99FdWpA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849860; c=relaxed/simple; bh=hJ099FlQtbdXqo2+oz8/ORBdCodnOxTdeN2C2GWn1FE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iKHYw9W3VxZuTzp4A1xbwL5aHx7rahnUgsgx+RzVtZhdKmv45EWMRm1sRB3few4mi+HElVv5iBoL6OOkkK8AFVp+tpHnU2k0NH1C1mFfMFdJs3m9OFBkIDP3qS7X0zlCS6YR3GgGIOM1KYnVy6o+oRqcXOhmC/DvlMA/0Lfed8w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=HnSP4nyM; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="HnSP4nyM" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849858; x=1738385858; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hJ099FlQtbdXqo2+oz8/ORBdCodnOxTdeN2C2GWn1FE=; b=HnSP4nyMS6sVrE6l4bOm1KcNKXsfD73JTXxRMl8U7y2wtC7H2mSCFekR ecLLfq3bIGDMzVI19nt8Xm/AlA9lOgpRICNLEupxfKhxb1TUBxGXg+UA7 E6XZTpd3AdF01CW8yp+Q56k6PnyA5jkyUjjvOH2Wv2qU6eHMJ3lb7HO+z N/P7YXQ5Hb1Iuornjr2w/sYS3uZ5ECblSnZpo0tPnXune1EYPs75HOfXv UgL+2zG31Wt+cDyxWt9RVKaJ7ujZB5A0XXvY5UXheoO+wxRrA6MLc+EHN SLe1f58gkA5PdF6YNjHM1Dzv6YDiciwbQSp1P/hETtcFRhqLYENp7NnLI g==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615870" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615870" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339803" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339803" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:25 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 16/17] vfio/pci: Move vfio_msi_disable() to be with other MSI/MSI-X management code Date: Thu, 1 Feb 2024 20:57:10 -0800 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The interrupt management code is mostly organized in four sections: shared code (interrupt type checking and interrupt context management), INTx management code, MSI/MSI-X management code, and IOCTL support. vfio_msi_disable() is separate from the other MSI/MSI-X code. This may have been required because vfio_msi_disable() relies on vfio_irq_set_vector_signal() within the IOCTL support. Since vfio_irq_set_vector_signal() is declared earlier it is not required for MSI/MSI-X management code to be mixed with the IOCTL support. Move vfio_msi_disable() to be located with all the other MSI/MSI-X management code. This move makes it simpler to initialize the interrupt management callbacks with vfio_msi_disable() so that it can be provided to the IOCTL support code. Signed-off-by: Reinette Chatre --- drivers/vfio/pci/vfio_pci_intrs.c | 58 +++++++++++++++---------------- 1 file changed, 29 insertions(+), 29 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 9217fea3f636..daa84a317f40 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -404,6 +404,35 @@ static int vfio_msi_enable(struct vfio_pci_core_device *vdev, return 0; } +static void vfio_msi_disable(struct vfio_pci_core_device *vdev, + unsigned int index) +{ + struct pci_dev *pdev = vdev->pdev; + struct vfio_pci_irq_ctx *ctx; + unsigned long i; + u16 cmd; + + xa_for_each(&vdev->ctx, i, ctx) { + vfio_virqfd_disable(&ctx->unmask); + vfio_virqfd_disable(&ctx->mask); + vfio_irq_set_vector_signal(vdev, i, -1, index); + vfio_irq_ctx_free(vdev, ctx, i); + } + + cmd = vfio_pci_memory_lock_and_enable(vdev); + pci_free_irq_vectors(pdev); + vfio_pci_memory_unlock_and_restore(vdev, cmd); + + /* + * Both disable paths above use pci_intx_for_msi() to clear DisINTx + * via their shutdown paths. Restore for NoINTx devices. + */ + if (vdev->nointx) + pci_intx(pdev, 0); + + vdev->irq_type = VFIO_PCI_NUM_IRQS; +} + /* * vfio_msi_alloc_irq() returns the Linux IRQ number of an MSI or MSI-X device * interrupt vector. If a Linux IRQ number is not available then a new @@ -617,35 +646,6 @@ static int vfio_irq_set_block(struct vfio_pci_core_device *vdev, return ret; } -static void vfio_msi_disable(struct vfio_pci_core_device *vdev, - unsigned int index) -{ - struct pci_dev *pdev = vdev->pdev; - struct vfio_pci_irq_ctx *ctx; - unsigned long i; - u16 cmd; - - xa_for_each(&vdev->ctx, i, ctx) { - vfio_virqfd_disable(&ctx->unmask); - vfio_virqfd_disable(&ctx->mask); - vfio_irq_set_vector_signal(vdev, i, -1, index); - vfio_irq_ctx_free(vdev, ctx, i); - } - - cmd = vfio_pci_memory_lock_and_enable(vdev); - pci_free_irq_vectors(pdev); - vfio_pci_memory_unlock_and_restore(vdev, cmd); - - /* - * Both disable paths above use pci_intx_for_msi() to clear DisINTx - * via their shutdown paths. Restore for NoINTx devices. - */ - if (vdev->nointx) - pci_intx(pdev, 0); - - vdev->irq_type = VFIO_PCI_NUM_IRQS; -} - /* * IOCTL support */ From patchwork Fri Feb 2 04:57:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13542072 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23F3A1B299; Fri, 2 Feb 2024 04:57:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849861; cv=none; b=cjYuL2NXN0qPIelN2C6Exxuk/ucP7aBFfjLnw608uYhCQTdWsz2ihTNQJi3y4CI7Un/6YndIW3ZtEBGsbdME+a8JziSIX9Xyd67Xs0v++a08/GWN2Zm3t9+fsgMji32F419Hn4w0V40+fTtwr9twngkt1q3gHpzoDYA86cnhqgo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706849861; c=relaxed/simple; bh=4QFGx+82iSOQD5Uy7w9iqIN8hz1HoOJM57vM7BEXPkc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=oT/Ddg0Npb6DrhmZQqZVy/tuC3nvM9Sua9qjQxh9FiskNKVkhhf8vXqhcd7VjOVcXbnjwLThNmuklNM5s+3QdJBGE4+8wIvHmku/tlbJgo0KlJhPLjJFQoZMGdIfVXkPP8yze15lozziz1OIgFkKWKW2SpiH3XbHGXZmY2gaHG4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Iy8CILgn; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Iy8CILgn" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706849859; x=1738385859; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=4QFGx+82iSOQD5Uy7w9iqIN8hz1HoOJM57vM7BEXPkc=; b=Iy8CILgnJZ1cQcQ52vjAEEuKGn3rwbhWJG2pc9idgTpLbkNBO9K29kZG 4V89/Lf3zUJKw5XZMOLAdyWv1SsPOefjHKwt7Sz0RdatupE2N/eciA7Bz ktJqopQUd0HGNneZVM2fDOEhMCsIRlmnKoCRS74MA8VTfzwcgVPbCNnRu jYvZJZJSiYstvG0mkzrIr8k7WUXgmdUJTe0pl1qop3+P+rbnjJivsg+9E dWsZVne9+65x/PbTl/WZtIjt5u7yQog5Za4iwoIV4jNkb7aQ/7AVo+Chy KyZCDsa1+xx8T+elmMN0JOLxpnu/AZiMqAXNwzPbX1upNwaocDdSoCf7u g==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17615872" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="17615872" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="912339806" X-IronPort-AV: E=Sophos;i="6.05,237,1701158400"; d="scan'208";a="912339806" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2024 20:57:25 -0800 From: Reinette Chatre To: jgg@nvidia.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, alex.williamson@redhat.com Cc: kvm@vger.kernel.org, dave.jiang@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, linux-kernel@vger.kernel.org, patches@lists.linux.dev Subject: [PATCH 17/17] vfio/pci: Remove duplicate interrupt management flow Date: Thu, 1 Feb 2024 20:57:11 -0800 Message-Id: <6ec901daffab4170d9740c7d066becd079925d53.1706849424.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 vfio_pci_set_intx_trigger() and vfio_pci_set_trigger() have the same flow that calls interrupt type (INTx, MSI, MSI-X) specific functions. Create callbacks for the interrupt type specific code that can be called by the shared code so that only one of these functions are needed. Rename the final generic function shared by all interrupt types vfio_pci_set_trigger(). Relocate the "IOCTL support" marker to correctly mark the now generic code. Signed-off-by: Reinette Chatre --- drivers/vfio/pci/vfio_pci_intrs.c | 104 ++++++++++-------------------- 1 file changed, 35 insertions(+), 69 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index daa84a317f40..a5b337cfae60 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -32,6 +32,12 @@ struct vfio_pci_irq_ctx { }; struct vfio_pci_intr_ops { + int (*enable)(struct vfio_pci_core_device *vdev, unsigned int start, + unsigned int count, unsigned int index); + void (*disable)(struct vfio_pci_core_device *vdev, + unsigned int index); + void (*send_eventfd)(struct vfio_pci_core_device *vdev, + struct vfio_pci_irq_ctx *ctx); int (*request_interrupt)(struct vfio_pci_core_device *vdev, struct vfio_pci_irq_ctx *ctx, unsigned int vector, unsigned int index); @@ -356,6 +362,12 @@ static void vfio_intx_disable(struct vfio_pci_core_device *vdev, /* * MSI/MSI-X */ +static void vfio_send_msi_eventfd(struct vfio_pci_core_device *vdev, + struct vfio_pci_irq_ctx *ctx) +{ + eventfd_signal(ctx->trigger); +} + static irqreturn_t vfio_msihandler(int irq, void *arg) { struct eventfd_ctx *trigger = arg; @@ -544,13 +556,22 @@ static void vfio_msi_unregister_producer(struct vfio_pci_irq_ctx *ctx) irq_bypass_unregister_producer(&ctx->producer); } +/* + * IOCTL support + */ static struct vfio_pci_intr_ops intr_ops[] = { [VFIO_PCI_INTX_IRQ_INDEX] = { + .enable = vfio_intx_enable, + .disable = vfio_intx_disable, + .send_eventfd = vfio_send_intx_eventfd_ctx, .request_interrupt = vfio_intx_request_interrupt, .free_interrupt = vfio_intx_free_interrupt, .device_name = vfio_intx_device_name, }, [VFIO_PCI_MSI_IRQ_INDEX] = { + .enable = vfio_msi_enable, + .disable = vfio_msi_disable, + .send_eventfd = vfio_send_msi_eventfd, .request_interrupt = vfio_msi_request_interrupt, .free_interrupt = vfio_msi_free_interrupt, .device_name = vfio_msi_device_name, @@ -558,6 +579,9 @@ static struct vfio_pci_intr_ops intr_ops[] = { .unregister_producer = vfio_msi_unregister_producer, }, [VFIO_PCI_MSIX_IRQ_INDEX] = { + .enable = vfio_msi_enable, + .disable = vfio_msi_disable, + .send_eventfd = vfio_send_msi_eventfd, .request_interrupt = vfio_msi_request_interrupt, .free_interrupt = vfio_msi_free_interrupt, .device_name = vfio_msi_device_name, @@ -646,9 +670,6 @@ static int vfio_irq_set_block(struct vfio_pci_core_device *vdev, return ret; } -/* - * IOCTL support - */ static int vfio_pci_set_intx_unmask(struct vfio_pci_core_device *vdev, unsigned int index, unsigned int start, unsigned int count, uint32_t flags, @@ -701,71 +722,16 @@ static int vfio_pci_set_intx_mask(struct vfio_pci_core_device *vdev, return 0; } -static int vfio_pci_set_intx_trigger(struct vfio_pci_core_device *vdev, - unsigned int index, unsigned int start, - unsigned int count, uint32_t flags, - void *data) -{ - struct vfio_pci_irq_ctx *ctx; - unsigned int i; - - if (is_intx(vdev) && !count && (flags & VFIO_IRQ_SET_DATA_NONE)) { - vfio_intx_disable(vdev, index); - return 0; - } - - if (!(is_intx(vdev) || is_irq_none(vdev))) - return -EINVAL; - - if (flags & VFIO_IRQ_SET_DATA_EVENTFD) { - int32_t *fds = data; - int ret; - - if (is_intx(vdev)) - return vfio_irq_set_block(vdev, start, count, fds, index); - - ret = vfio_intx_enable(vdev, start, count, index); - if (ret) - return ret; - - ret = vfio_irq_set_block(vdev, start, count, fds, index); - if (ret) - vfio_intx_disable(vdev, index); - - return ret; - } - - if (!is_intx(vdev)) - return -EINVAL; - - /* temporary */ - for (i = start; i < start + count; i++) { - ctx = vfio_irq_ctx_get(vdev, i); - if (!ctx || !ctx->trigger) - continue; - if (flags & VFIO_IRQ_SET_DATA_NONE) { - vfio_send_intx_eventfd_ctx(vdev, ctx); - } else if (flags & VFIO_IRQ_SET_DATA_BOOL) { - uint8_t *bools = data; - - if (bools[i - start]) - vfio_send_intx_eventfd_ctx(vdev, ctx); - } - } - - return 0; -} - -static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, - unsigned int index, unsigned int start, - unsigned int count, uint32_t flags, - void *data) +static int vfio_pci_set_trigger(struct vfio_pci_core_device *vdev, + unsigned int index, unsigned int start, + unsigned int count, uint32_t flags, + void *data) { struct vfio_pci_irq_ctx *ctx; unsigned int i; if (irq_is(vdev, index) && !count && (flags & VFIO_IRQ_SET_DATA_NONE)) { - vfio_msi_disable(vdev, index); + intr_ops[index].disable(vdev, index); return 0; } @@ -780,13 +746,13 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, return vfio_irq_set_block(vdev, start, count, fds, index); - ret = vfio_msi_enable(vdev, start, count, index); + ret = intr_ops[index].enable(vdev, start, count, index); if (ret) return ret; ret = vfio_irq_set_block(vdev, start, count, fds, index); if (ret) - vfio_msi_disable(vdev, index); + intr_ops[index].disable(vdev, index); return ret; } @@ -799,11 +765,11 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, if (!ctx || !ctx->trigger) continue; if (flags & VFIO_IRQ_SET_DATA_NONE) { - eventfd_signal(ctx->trigger); + intr_ops[index].send_eventfd(vdev, ctx); } else if (flags & VFIO_IRQ_SET_DATA_BOOL) { uint8_t *bools = data; if (bools[i - start]) - eventfd_signal(ctx->trigger); + intr_ops[index].send_eventfd(vdev, ctx); } } return 0; @@ -912,7 +878,7 @@ int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device *vdev, uint32_t flags, func = vfio_pci_set_intx_unmask; break; case VFIO_IRQ_SET_ACTION_TRIGGER: - func = vfio_pci_set_intx_trigger; + func = vfio_pci_set_trigger; break; } break; @@ -924,7 +890,7 @@ int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device *vdev, uint32_t flags, /* XXX Need masking support exported */ break; case VFIO_IRQ_SET_ACTION_TRIGGER: - func = vfio_pci_set_msi_trigger; + func = vfio_pci_set_trigger; break; } break;