From patchwork Fri Sep 13 21:46:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13804110 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2144A6BFD4 for ; Fri, 13 Sep 2024 21:47:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264038; cv=none; b=hoWfAuDtN4piTjxXpAIubz4vszS/fWEaZZlfXt+xxtfhC5/nko1yVrx95rzP3PJfydeNEZX/sDCtv/Spb5Z01H3AMUPKmYbeweSr7FDnReIp7R05C02OshQM4hNVFl4xqXCTng0HHBeazszSuVJPpjqvEQPR3OlfQh2CQ6SSf/g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264038; c=relaxed/simple; bh=LXaWD9KIdKV1BX3OScWS0sAEKTLeBqsNcj+9a5HDpLQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pJlVzloA9G9mb2vx5PksrQDYtKGQo7ECJGWsWH4pVRqkgVfr4m00tYPZesVEIz7YTCU5cfDZqa2U8qsTzkEC8vh7xtS7w9wDJnauO9C7pwzFujCn4hSnb8+ROWcU0lQ0NrLVddQhAPZK7BRJhBIiROv814rvmolZDUrlZ+K+xic= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=Pg7hnVNN; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Pg7hnVNN" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B4AE91083; Fri, 13 Sep 2024 23:45:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263948; bh=LXaWD9KIdKV1BX3OScWS0sAEKTLeBqsNcj+9a5HDpLQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Pg7hnVNN6moh8dJ/g3OHytTz1tJjorOe+/o0fU3TrCilAdFRJi+2LvsnUmzmmuQZE Vi3caaSV2FMtgGyoFdR30X3ccNAfNBrlhtJQTugPdCKXIfg6LfQTJgBjJwN0AFDECJ clT5HUDTx9NnTIYWNiefOfoMn9auf5a7GNL8EsKw= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 01/10] media: media-entity: Introduce media_entity_context Date: Fri, 13 Sep 2024 23:46:46 +0200 Message-ID: <20240913214657.1502838-2-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Introduce the 'struct media_entity_context' type, which serves for reference counting and introduce two new media entity operations to allow drivers to allocate and free a media entity context. The newly introduced type will be used as a base type for the device context types (video_device_context and v4l2_subdevice_context) that will be introduced in the next patches. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-entity.c | 46 +++++++++++++++++ include/media/media-entity.h | 99 ++++++++++++++++++++++++++++++++++++ 2 files changed, 145 insertions(+) diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 951b79ca125c..8ce72d72bc1d 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -1660,3 +1660,49 @@ struct media_link *__media_entity_next_link(struct media_entity *entity, return NULL; } EXPORT_SYMBOL_GPL(__media_entity_next_link); + +static void media_entity_release_context(struct kref *refcount) +{ + struct media_entity_context *ctx = + container_of(refcount, struct media_entity_context, refcount); + + ctx->entity->ops->destroy_context(ctx); +} + +struct media_entity_context * +media_entity_context_get(struct media_entity_context *ctx) +{ + if (!ctx) + return ERR_PTR(-EINVAL); + + kref_get(&ctx->refcount); + + return ctx; +} +EXPORT_SYMBOL_GPL(media_entity_context_get); + +void media_entity_context_put(struct media_entity_context *ctx) +{ + if (!ctx) + return; + + kref_put(&ctx->refcount, media_entity_release_context); +} +EXPORT_SYMBOL_GPL(media_entity_context_put); + +void media_entity_init_context(struct media_entity *entity, + struct media_entity_context *ctx) +{ + if (!ctx) + return; + + ctx->entity = entity; + kref_init(&ctx->refcount); + INIT_LIST_HEAD(&ctx->list); +} +EXPORT_SYMBOL_GPL(media_entity_init_context); + +void media_entity_cleanup_context(struct media_entity_context *ctx) +{ +} +EXPORT_SYMBOL_GPL(media_entity_cleanup_context); diff --git a/include/media/media-entity.h b/include/media/media-entity.h index 4d95893c8984..002a326de9b9 100644 --- a/include/media/media-entity.h +++ b/include/media/media-entity.h @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -248,6 +249,37 @@ struct media_pad { struct media_pipeline *pipe; }; +/** + * struct media_entity_context - A media entity execution context + * @mdev_context: The media device context this media entity is bound to. + * The field is initialized when the entity is bound to a media + * device context. + * @entity: The media entity this context belongs to + * @refcount: The kref reference counter + * list: The list entry to link the entity context in the media device context + * + * This type represent the 'base class' used to implement execution context for + * video device contexts and subdevice contexts. Those types embedds an instance + * of 'struct media_entity_context' as their first member, allowing the MC core + * to implement type polymorphism and handle video device and subdevice contexts + * transparently. + * + * The main function of this type is to provide reference counting for the + * 'dervived' device context types. The video device and subdevice core + * populates the 'context_release' function pointer that implement specific + * clean-up operations, similar to what a 'virtual destructor' would do in C++. + * + * Drivers are not expected to use this type directly, but only the MC core + * will. + */ +struct media_device_context; +struct media_entity_context { + struct media_device_context *mdev_context; + struct media_entity *entity; + struct kref refcount; + struct list_head list; +}; + /** * struct media_entity_operations - Media entity operations * @get_fwnode_pad: Return the pad number based on a fwnode endpoint or @@ -269,6 +301,15 @@ struct media_pad { * media_entity_has_pad_interdep(). * Optional: If the operation isn't implemented all pads * will be considered as interdependent. + * @alloc_context: Allocate a media entity context. Drivers are allowed to + * sub-class the entity context type by defining a driver + * specific type that embeds an instance of either a + * video_device_context or subdevice_context as first + * member, and allocate the size of a driver-specific type + * in the implementation of this operation. Returns 0 for + * success, or an error code < 0 otherwise. + * @destroy_context: Release a media entity context previously allocated by + * the driver. * * .. note:: * @@ -284,6 +325,9 @@ struct media_entity_operations { int (*link_validate)(struct media_link *link); bool (*has_pad_interdep)(struct media_entity *entity, unsigned int pad0, unsigned int pad1); + int (*alloc_context)(struct media_entity *entity, + struct media_entity_context **context); + void (*destroy_context)(struct media_entity_context *context); }; /** @@ -1452,3 +1496,58 @@ struct media_link *__media_entity_next_link(struct media_entity *entity, MEDIA_LNK_FL_DATA_LINK)) #endif + +/** + * media_entity_context_get - Increase the media entity context reference count + * and return a reference to it + * + * @ctx: the media entity context + * + * Increase the media entity context reference count. The reference count + * is increased by the V4L2 core when: + * + * - a new context is allocated when bounding a media entity to a media device + * context (by kref_init()) + * - the media pipeline the context is part of starts streaming + * + * The entity context gets automatically decreased by the V4L2 core when: + * - a context is unbound + * - the pipeline stops streaming + */ +struct media_entity_context * +media_entity_context_get(struct media_entity_context *ctx); + +/** + * media_entity_context_put - Decrease the media entity context reference count + * + * @ctx: the media entity context + * + * Decrease the media entity context reference count. The reference count + * is decreased by the V4L2 core when: + * - the file handle the context is associated with is closed + * - the media pipeline the context is part of is stopped + */ +void media_entity_context_put(struct media_entity_context *ctx); + +/** + * media_entity_init_context - Initialize the media entity context + * + * @entity: the media entity this context belongs to + * @ctx: the media entity context + * + * Initialize the media entity context by initializing the kref reference + * counter. The intended caller of this function are the video device context + * and subdevic context initialize functions. + */ +void media_entity_init_context(struct media_entity *entity, + struct media_entity_context *ctx); + +/** + * media_entity_cleanup_context - Cleanup the media entity context + * + * @ctx: the media entity context + * + * Cleanup the media entity context. The intended caller of this function are + * the video device and subdevice context cleanup functions. + */ +void media_entity_cleanup_context(struct media_entity_context *ctx); From patchwork Fri Sep 13 21:46:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13804111 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BDA941553A1 for ; Fri, 13 Sep 2024 21:47:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264040; cv=none; b=qCzq3zpYZQDCkyYyjK7uWNEeT/v3jPMX53VmCN6aTJ/RoiC7DUBzrVnaHBXExTFpvfYGW6e3Mrxa2wS4dymCDM03Lt6uIUeUKPeuUo0jO+a6KRD7bsyamMgupf6NuKIq20O6jQM+8fy8v5qzE82Hx7i65fC8GlBkC09tlvYWgCE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264040; c=relaxed/simple; bh=H4YgJl+ixXKsioffdNbdZUQ1TkUacypExs2wltDE3OU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aKTD4aURglKMlGMRnd7XEQILTD23EIXzYU0ou8CS3wxGm6wnw7u3saqHQBh8qK9ij6FPqkHBKqTW0AgPBaVIljHXl3cGoRVL5lXlwH4vVjL5foQO0kzJY7Bb/mW9PpLT+oYor/quw/bIo9i6J5b1eBGJOVOyZNSCXw4gMTD1xSw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=jbrHOSjS; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jbrHOSjS" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 06C781583; Fri, 13 Sep 2024 23:45:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263949; bh=H4YgJl+ixXKsioffdNbdZUQ1TkUacypExs2wltDE3OU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jbrHOSjSzzc/+CzcH0awOF7N0nPTEVHmjqQo0b5o0aK05xyyqZeOCGXmxKZgcOkQw KiGRerjE+uYzK8sWgKg0e/6TPWzwWDewMewlFjIeZGGz+eTq1Cnt8yaTnTBT8vPm4p ZUj3NdC2SFCe+v5ezbVSqiY9YKWtSHjzLt8iT9TQ= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 02/10] media: media-device: Introduce media device context Date: Fri, 13 Sep 2024 23:46:47 +0200 Message-ID: <20240913214657.1502838-3-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Introduce a new type in the media-fh.h header that represent a media device context. A media device context is allocated when the media device is open and released when the last reference to it is put. A new pair of media_device_ops is added to allow device drivers to allocate and release a media context. The media context groups together the media entity contexts that are associated with it to form an isolated execution context. Provide helpers in mc-device.c for drivers and for the v4l2-core to handle media device contexts and to bind/unbind entity contexts to it. Once an entity context has been bound to a media device context it is possible to retrieve it by using a pointer to the entity the device is represented by. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-device.c | 168 ++++++++++++++++++++++++++++ drivers/media/mc/mc-entity.c | 1 + include/media/media-device.h | 210 +++++++++++++++++++++++++++++++++++ include/media/media-fh.h | 4 + 4 files changed, 383 insertions(+) diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c index 6616757640ec..1b24d5e74ffb 100644 --- a/drivers/media/mc/mc-device.c +++ b/drivers/media/mc/mc-device.c @@ -12,7 +12,9 @@ #include #include #include +#include #include +#include #include #include #include @@ -49,11 +51,31 @@ static int media_device_open(struct media_devnode *devnode, struct file *filp) { struct media_device *mdev = to_media_device(devnode); struct media_device_fh *fh; + int ret; fh = kzalloc(sizeof(*fh), GFP_KERNEL); if (!fh) return -ENOMEM; + if (mdev->ops && mdev->ops->alloc_context) { + if (WARN_ON(!mdev->ops->destroy_context)) { + kfree(fh); + return -EINVAL; + } + + ret = mdev->ops->alloc_context(mdev, &fh->context); + if (ret) { + kfree(fh); + return ret; + } + + /* + * Make sure the driver implementing alloc_context has + * called media_device_init_context + */ + WARN_ON(!fh->context->initialized); + } + fh->fh.ref = devnode->ref; filp->private_data = &fh->fh; @@ -78,6 +100,8 @@ static int media_device_close(struct file *filp) spin_unlock_irq(&mdev->fh_list_lock); } + media_device_context_put(fh->context); + kfree(fh); return 0; @@ -885,6 +909,150 @@ void media_device_unregister(struct media_device *mdev) } EXPORT_SYMBOL_GPL(media_device_unregister); +/* ----------------------------------------------------------------------------- + * Context handling + */ + +static void media_device_release_context(struct kref *refcount) +{ + struct media_device_context *context = + container_of(refcount, struct media_device_context, refcount); + + /* + * All the associated entity contexts should have been released if we + * get here. + */ + WARN_ON(!list_empty(&context->contexts)); + + context->mdev->ops->destroy_context(context); +} + +struct media_device_context * +media_device_context_get(struct media_device_context *ctx) +{ + if (!ctx) + return ERR_PTR(-EINVAL); + + kref_get(&ctx->refcount); + + return ctx; +} +EXPORT_SYMBOL_GPL(media_device_context_get); + +void media_device_context_put(struct media_device_context *ctx) +{ + if (!ctx) + return; + + kref_put(&ctx->refcount, media_device_release_context); +} +EXPORT_SYMBOL_GPL(media_device_context_put); + +struct media_device_context *media_device_context_get_from_fd(unsigned int fd) +{ + struct media_device_context *ctx; + struct file *filp = fget(fd); + struct media_device_fh *fh; + + if (!filp) + return NULL; + + fh = media_device_fh(filp); + ctx = media_device_context_get(fh->context); + fput(filp); + + return ctx; +} +EXPORT_SYMBOL_GPL(media_device_context_get_from_fd); + +int media_device_init_context(struct media_device *mdev, + struct media_device_context *ctx) +{ + ctx->mdev = mdev; + INIT_LIST_HEAD(&ctx->contexts); + mutex_init(&ctx->lock); + kref_init(&ctx->refcount); + + ctx->initialized = true; + + return 0; +} +EXPORT_SYMBOL_GPL(media_device_init_context); + +void media_device_cleanup_context(struct media_device_context *ctx) +{ + mutex_destroy(&ctx->lock); + list_del_init(&ctx->contexts); +} +EXPORT_SYMBOL_GPL(media_device_cleanup_context); + +int media_device_bind_context(struct media_device_context *mdev_context, + struct media_entity_context *context) +{ + struct media_entity_context *entry; + + if (WARN_ON(!mdev_context || !context)) + return -EINVAL; + + guard(mutex)(&mdev_context->lock); + + /* Make sure the entity has not been bound already. */ + list_for_each_entry(entry, &mdev_context->contexts, list) { + if (entry == context) + return -EINVAL; + } + + list_add_tail(&context->list, &mdev_context->contexts); + context->mdev_context = media_device_context_get(mdev_context); + + return 0; +} +EXPORT_SYMBOL_GPL(media_device_bind_context); + +int media_device_unbind_context(struct media_entity_context *context) +{ + struct media_device_context *mdev_context = context->mdev_context; + struct media_entity_context *entry; + struct media_entity_context *tmp; + + if (WARN_ON(!mdev_context || !context)) + return -EINVAL; + + guard(mutex)(&mdev_context->lock); + list_for_each_entry_safe(entry, tmp, &mdev_context->contexts, list) { + if (entry != context) + continue; + + list_del(&entry->list); + media_device_context_put(mdev_context); + entry->mdev_context = NULL; + + return 0; + } + + WARN(true, "Media entity context is not bound to any media context\n"); + + return -EINVAL; +} +EXPORT_SYMBOL_GPL(media_device_unbind_context); + +struct media_entity_context * +media_device_get_entity_context(struct media_device_context *mdev_context, + struct media_entity *entity) +{ + struct media_entity_context *entry; + + guard(mutex)(&mdev_context->lock); + + list_for_each_entry(entry, &mdev_context->contexts, list) { + if (entry->entity == entity) + return media_entity_context_get(entry); + } + + return ERR_PTR(-EINVAL); +} +EXPORT_SYMBOL(media_device_get_entity_context); + #if IS_ENABLED(CONFIG_PCI) void media_device_pci_init(struct media_device *mdev, struct pci_dev *pci_dev, diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 8ce72d72bc1d..4108d3da4cb0 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -1704,5 +1704,6 @@ EXPORT_SYMBOL_GPL(media_entity_init_context); void media_entity_cleanup_context(struct media_entity_context *ctx) { + media_device_unbind_context(ctx); } EXPORT_SYMBOL_GPL(media_entity_cleanup_context); diff --git a/include/media/media-device.h b/include/media/media-device.h index dd937552daf7..a73f650f122f 100644 --- a/include/media/media-device.h +++ b/include/media/media-device.h @@ -18,10 +18,72 @@ #include #include +#include struct ida; struct media_device; +/** + * struct media_device_context - Media device context + * @mdev: The media device this context is associated with + * @refcount: The kref reference counter + * @lock: Protects the entities contexts list + * @contexts: List of entity contexts associated with this media device context + * @initialized: Flag set to true by media_device_init_context() + * + * A media device context is created every time the media device gets opened by + * userspace. It is then uniquely identified for applications by the numerical + * file descriptor returned by a successful call to open() and is associated + * with an instance of :c:type:`struct media_device_fh`. + * + * Media device contexts are ref-counted and thus freed once the last reference + * to them is released. + * + * A media device context groups together the media entity contexts registered + * on a video device or v4l2 subdevice that has been associated with a media + * device context. The association between a media entity context and media + * device context is called 'bounding', and the result of bounding is to create + * an 'execution context' independent from other execution contexts. + * + * An entity context is bound to a media device context by a call to the + * VIDIOC_BIND_CONTEXT ioctl on video devices and by a call to + * VIDIOC_SUBDEV_BIND_CONTEXT on subdevices by userspace. The bounding operation + * groups together entity contexts to the same media device context. As video + * devices and v4l2 subdevices devnodes can be opened multiple times, each file + * descriptor resulting from a successful open() call can be bound to a + * different media device context. + * + * Creating execution contexts by bounding video entity contexts to a media + * device context allows userspace to effectively multiplex the usage of a + * media graph and of the device nodes that are part of it. + * + * In order to create an execution context userspace should: + * 1) Open the media device to create a media device context identified by the + * file descriptor returned by a successful 'open()' call + * 2) Open the video device or v4l2 subdevice and bind the file descriptors to + * the media device context by calling the VIDIOC_BIND_CONTEXT and + * VIDIOC_SUBDEV_BIND_CONTEXT ioctls + * + * All devices bound to the same media device context are now part of the same + * execution context. From this point on all the operations performed on a file + * descriptor bound to a media device context are independent from operations + * performed on a file descriptor bound to a different execution context. + * + * Binding an entity context to a media device context increases the media + * device context reference count. This guarantees that references to media + * device context are valid as long as there are valid entity contexts that + * refers to it. Symmetrically, unbinding an entity context from a media + * device context decreases the media device context reference count. + */ +struct media_device_context { + struct media_device *mdev; + struct kref refcount; + /* Protects the 'contexts' list */ + struct mutex lock; + struct list_head contexts; + bool initialized; +}; + /** * struct media_entity_notify - Media Entity Notify * @@ -63,6 +125,13 @@ struct media_entity_notify { * And once a buffer is queued, then the driver can complete * or delete objects from the request before req_queue exits. * @release: Release the resources of the media device. + * @alloc_context: Allocate a media device context. The operation allows drivers to + * allocate a driver-specific structure that embeds a + * media_device_context instance as first member where to store + * driver-specific information that are global to all device + * contexts part of media device context. Returns 0 on success a + * negative error code otherwise. + * @destroy_context: Release a media device context. */ struct media_device_ops { int (*link_notify)(struct media_link *link, u32 flags, @@ -72,6 +141,9 @@ struct media_device_ops { int (*req_validate)(struct media_request *req); void (*req_queue)(struct media_request *req); void (*release)(struct media_device *mdev); + int (*alloc_context)(struct media_device *mdev, + struct media_device_context **ctx); + void (*destroy_context)(struct media_device_context *ctx); }; /** @@ -331,6 +403,144 @@ int __must_check __media_device_register(struct media_device *mdev, */ void media_device_unregister(struct media_device *mdev); +/* ----------------------------------------------------------------------------- + * media device context handling + */ + +/** + * media_device_context_get - Increase the media device context reference count + * and return a reference to it + * @ctx: The media device context + */ +struct media_device_context * +media_device_context_get(struct media_device_context *ctx); + +/** + * media_device_context_put - Decrease the media device context reference count + * @ctx: The media device context + */ +void media_device_context_put(struct media_device_context *ctx); + +/** + * media_device_context_get_from_fd - Get the media device context associated with a + * numerical file descriptor + * + * @fd: the numerical file descriptor + * + * A media device context is created whenever the media device devnode is opened + * by userspace. It is then associated uniquely with a numerical file descriptor + * which is unique in the userspace process context. + * + * This function allows to retrieve the media device associated with such + * numerical file descriptor and increases the media device context reference + * count to guarantee the returned reference stays valid at least until the + * caller does not call media_device_context_put(). + * + * Caller of this function are required to put the returned media device context + * once they are done with it. + * + * The intended caller of this function is the VIDIOC_BIND_CONTEXT ioctl handler + * which need to get the media device contexts associated to a numerical file + * descriptor. + */ +struct media_device_context *media_device_context_get_from_fd(unsigned int fd); + +/** + * media_device_init_context - Initialize the media device context + * + * @mdev: The media device this context belongs to + * @ctx: The media device context to initialize + * + * Initialize the fields of a media device context. Device drivers that support + * multi context operations shall call this function in their implementation of + * media_device_operations.alloc_context() + */ +int media_device_init_context(struct media_device *mdev, + struct media_device_context *ctx); + +/** + * media_device_cleanup_context - Cleanup the media device context + * + * @ctx: The media device context to clean up + * + * Cleanup a media device context. Device drivers that support multi context + * operations shall call this function in their implementation of + * media_device_operations.destroy_context() before releasing the memory allocated + * by media_device_operations.alloc_context(). + */ +void media_device_cleanup_context(struct media_device_context *ctx); + +/** + * media_device_bind_context - Bind an entity context to a media device context + * + * @mdev_context: pointer to struct &media_device_context + * @context: the entity context to bind + * + * This function creates a mapping entry in the media device context that + * associates an entity context to the media entity it belongs to and stores it + * in a linked list so that they can be retrieved later. + * + * Binding an entity context to a media device context increases the media + * device context refcount. + * + * The intended caller of this function is the VIDIOC_BIND_CONTEXT ioctl handler + * that binds a newly created context to a media device context. + */ +int media_device_bind_context(struct media_device_context *mdev_context, + struct media_entity_context *context); + +/** + * media_device_unbind_context - Unbind an entity context from a media device + * context + * + * @context: the entity context to unbind + * + * An entity context is unbound from a media device context when the file handle + * it is associated with gets closed. + * + * Unbinding an entity context from a media device context decreases the media + * device context refcount. + * + * Returns 0 if the context was bound to a media device context, -EINVAL + * otherwise. + */ +int media_device_unbind_context(struct media_entity_context *context); + +/** + * media_device_get_entity_context - Get the entity context associated with + * a media entity in a media device context + * + * @mdev_context: pointer to struct &media_device_context + * @entity: pointer to struct &media_entity that the entity context is + * associated with + * + * An entity context is uniquely associated with a media device context after it + * has been bound to it by a call to the VIDIOC_BIND_CONTEXT ioctl. This helper + * function retrieves the entity context associated with a media device context + * for a specific entity that represents a video device or a v4l2 subdevice. + * + * The reference count of the returned entity context is increased to guarantee + * the returned reference stays valid until the caller does not call + * media_entity_context_put(). + * + * Drivers are not expected to call this function directly but should instead + * use the helpers provided by the video_device and v4l2_subdevice layers, + * video_device_context_get() and v4l2_subdev_get_context() respectively. + * Drivers are always required to decrease the returned context reference count + * by calling video_device_context_put() and v4l2_subdev_put_context(). + * + * If no entity context has been associated with the media device context + * provided as first argument an error pointer is returned. Drivers are + * required to always check the value returned by this function. + */ +struct media_entity_context * +media_device_get_entity_context(struct media_device_context *mdev_context, + struct media_entity *entity); + +/*------------------------------------------------------------------------------ + * Media entity handling + */ + /** * media_device_register_entity() - registers a media entity inside a * previously registered media device. diff --git a/include/media/media-fh.h b/include/media/media-fh.h index 6f00744b81d6..70331d3e3470 100644 --- a/include/media/media-fh.h +++ b/include/media/media-fh.h @@ -13,6 +13,8 @@ #include +struct media_device_context; + /** * struct media_device_fh - File handle specific information on MC * @@ -22,6 +24,8 @@ struct media_device_fh { struct media_devnode_fh fh; struct list_head mdev_list; + + struct media_device_context *context; }; static inline struct media_device_fh *media_device_fh(struct file *filp) From patchwork Fri Sep 13 21:46:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13804112 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 897E66BFD4 for ; Fri, 13 Sep 2024 21:47:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264041; cv=none; b=IuSxOZjBKZctJH+7AGk4sG/u+2ZsNpPJN1AjZTWVZt9rSmlvocTeod9D9FizOU6Y2KhRgHJlGt/IfBK/fghSg9Gi5GzGX1IHdUH+OC4kLSzE8R/wxWt6DT6XmXUbkCYL0OCwT4ixcDURpLIpqJrLelZg6rGFCckiBdzxQ/vGNY4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264041; c=relaxed/simple; bh=0rfsX8AyCcPjFdwHSZwxO5xVQc289O+3Qd7RMdFkYL8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fa2UVYfvygsJjjq83EMk+IeVQrdOoqqD94tNCduDxIcXxa3JSbvn3ftJksLeuy7kG+cnd6YVDsnxwCuN1QDuPoZnXaTJZXiTAhH+4J0fffu0KN2b+X43Di3t7z5qSOmgBZA3iJcyZbmmH/sDqMzUM52SkPYEPwQQT7SQG67dxjQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=jkASImwi; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jkASImwi" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 510DD186C; Fri, 13 Sep 2024 23:45:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263949; bh=0rfsX8AyCcPjFdwHSZwxO5xVQc289O+3Qd7RMdFkYL8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jkASImwi9a7Fupb9X8Z79aSOKr4RRkRSfHmjJ/WglGdwHM7CFKNqQEP1LaknpauME 4O9KSvLyVB06w62iPyuJkkDrRXHWnUh/WmRTy7yOK73f0oJ0NOGHdALj2zHYj/g1c1 NCN4XucUGJeup6E+Jz+czAIS63FVNzKXxNIxCwN0= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 03/10] media: v4l2-dev: Introduce video device context Date: Fri, 13 Sep 2024 23:46:48 +0200 Message-ID: <20240913214657.1502838-4-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Introduce a new type in video-device that represents a video device context. It extends 'struct media_entity_context' to include video-device specific data and configuration. The new type is intended to be extended by drivers that can store driver-specific information in their video device derived types. The next patch will introduce the VIDIOC_BIND_CONTEXT ioctl that allows to create a video device context and uniquely associate it with a media context. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-dev.c | 43 ++++++++++ include/media/v4l2-dev.h | 126 +++++++++++++++++++++++++++++ include/media/v4l2-fh.h | 3 + 3 files changed, 172 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index b80010bf3ec3..0fd6ff3f00f6 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -1200,6 +1200,49 @@ struct media_pipeline *video_device_pipeline(struct video_device *vdev) } EXPORT_SYMBOL_GPL(video_device_pipeline); +struct video_device_context * +video_device_context_get(struct media_device_context *mdev_context, + struct video_device *vdev) +{ + struct media_entity *entity = &vdev->entity; + struct media_entity_context *ctx = + media_device_get_entity_context(mdev_context, entity); + + if (!ctx) + return NULL; + + return container_of(ctx, struct video_device_context, base); +} +EXPORT_SYMBOL_GPL(video_device_context_get); + +void video_device_context_put(struct video_device_context *ctx) +{ + if (!ctx) + return; + + media_entity_context_put(&ctx->base); +} +EXPORT_SYMBOL_GPL(video_device_context_put); + +int video_device_init_context(struct video_device *vdev, + struct video_device_context *ctx) +{ + media_entity_init_context(&vdev->entity, &ctx->base); + + ctx->vdev = vdev; + mutex_init(&ctx->queue_lock); + + return 0; +} +EXPORT_SYMBOL_GPL(video_device_init_context); + +void video_device_cleanup_context(struct video_device_context *ctx) +{ + mutex_destroy(&ctx->queue_lock); + media_entity_cleanup_context(&ctx->base); +} +EXPORT_SYMBOL_GPL(video_device_cleanup_context); + #endif /* CONFIG_MEDIA_CONTROLLER */ /* diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index e0a13505f88d..d538f0f32d6d 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -18,6 +18,7 @@ #include #include +#include #define VIDEO_MAJOR 81 @@ -639,6 +640,131 @@ __must_check int video_device_pipeline_alloc_start(struct video_device *vdev); */ struct media_pipeline *video_device_pipeline(struct video_device *vdev); +/** + * struct video_device_context - The video device context + * @base: The media entity context base class member + * @mdev_context: The media device context this context is associated with + * @vdev: The video device this context belongs to + * @queue_lock: Protects the vb2 queue + * @queue: The vb2 queue + * + * This structure represents an isolated execution context of a video device. + * This type 'derives' the base 'struct media_entity_context' type which + * implements refcounting on our behalf and allows instances of this type to be + * linked in the media_device_context contexts list. + * + * By storing the data and the configuration of a video device in a per-file + * handle context, userspace is allowed to multiplex the usage of a single video + * device devnode by opening it multiple times and by associating it with a + * media device context. This operation is called 'bounding' and is performed + * using the VIDIOC_BIND_CONTEXT ioctl. + * + * A video device context is created and stored in the v4l2-fh file handle + * associated with an open file descriptor when a video device is 'bound' to a + * media device context. The 'bounding' operation realizes a permanent + * association valid until the video device context is released. + * + * A video device can be bound to the same media device context once only. + * Trying to bind the same video device to the same media device context a + * second time, without releasing the already established context by closing the + * bound file descriptor first, will result in an error. + * + * To create a video device context userspace shall use the VIDIOC_BIND_CONTEXT + * ioctl that creates the video device context and uniquely associates it with a + * media device file descriptor. + * + * Once a video device file descriptor has been bound to a media device context, + * all the operations performed on the video device file descriptor will be + * directed on the just created video device context. This means, in example, + * that the video device format and the buffer queue are isolated from the ones + * associated with a different file descriptor obtained by opening again the + * same video device devnode but bound to a different media device context. + * + * Drivers that implement multiplexing support have to provide a valid + * implementation of the context-related operations in the + * media entity operations. + * + * Drivers are allowed to sub-class the video_device_context structure by + * defining a driver-specific type which embeds a struct video_device_context + * instance as first member, and allocate the driver-specific structure size in + * their implementation of the `alloc_context` operation. + * + * Video device contexts are ref-counted by embedding an instance of 'struct + * media_entity_context' and are freed once all the references to it are + * released. + * + * A video device context ref-count is increased when: + * - The context is created by bounding a video device to a media device context + * - The media pipeline starts streaming + * A video device context ref-count is decreased when: + * - The associated file handle is closed + * - The media pipeline stops streaming + * + * The ref-count is increased by a call to video_device_context_get() and is + * reponsibility of the caller to decrease the reference count with a call to + * video_device_context_put(). + */ +struct video_device_context { + struct media_entity_context base; + + struct video_device *vdev; + struct mutex queue_lock; + struct vb2_queue queue; +}; + +/** + * video_device_context_get - Helper to get a video device context from a + * media device context + * + * @mdev_context: The media device context + * @vdev: The video device the context refers to + * + * Helper function that wraps media_device_get_entity_context() and returns + * the video device context associated with a video device in a media device + * context. + * + * The reference count of the returned video device context is increased. + * Callers of this function are required to decrease the reference count of + * the context reference with a call to video_device_context_put(). + */ +struct video_device_context * +video_device_context_get(struct media_device_context *mdev_context, + struct video_device *vdev); + +/** + * video_device_context_put - Helper to decrease a video context reference + * count + * + * @ctx: The video context to release + */ +void video_device_context_put(struct video_device_context *ctx); + +/** + * video_device_init_context - Initialize the video device context + * + * @vdev: The video device this context belongs to + * @ctx: The context to initialize + * + * Initialize the video device context. The intended callers of this function + * are driver-specific implementations of the media_entity_ops.alloc_context() + * function that allocates their driver specific types that derive from + * struct video_device_context. + */ +int video_device_init_context(struct video_device *vdev, + struct video_device_context *ctx); + +/** + * video_device_cleanup_context - Cleanup the video device context + * + * @ctx: The context to cleanup. + * + * Cleanup the video device context. The intended callers of this function are + * driver specific implementation of the media_entity_ops.destroy_context() + * function before releasing the memory previously allocated by + * media_entity_ops.alloc_context(). + */ +void video_device_cleanup_context(struct video_device_context *ctx); + #endif /* CONFIG_MEDIA_CONTROLLER */ #endif /* _V4L2_DEV_H */ diff --git a/include/media/v4l2-fh.h b/include/media/v4l2-fh.h index b5b3e00c8e6a..a8de8613a026 100644 --- a/include/media/v4l2-fh.h +++ b/include/media/v4l2-fh.h @@ -20,6 +20,7 @@ struct video_device; struct v4l2_ctrl_handler; +struct video_device_context; /** * struct v4l2_fh - Describes a V4L2 file handler @@ -38,6 +39,7 @@ struct v4l2_ctrl_handler; * @sequence: event sequence number * * @m2m_ctx: pointer to &struct v4l2_m2m_ctx + * @context: The video device context */ struct v4l2_fh { struct list_head list; @@ -54,6 +56,7 @@ struct v4l2_fh { u32 sequence; struct v4l2_m2m_ctx *m2m_ctx; + struct video_device_context *context; }; /** From patchwork Fri Sep 13 21:46:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13804113 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C95346BFD4 for ; Fri, 13 Sep 2024 21:47:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264044; cv=none; b=aA+BZM02bHNGQVnBacU9Uab+HfGP8W/2V4053+giL4vpMPBKAng69s164aIyuYYHtaUWKU3UNt4JKJcVOrQf/rkuVgi6cWiapJInAzqqAXcEjkLf0RS8/Nad/iDo9Itg0u1gykH9xr1dZo25tQv/EHr7nL8Gh7dqO4OnT1eNrks= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264044; c=relaxed/simple; bh=C/tLRufzYECRe+GhTy0TGewSXXIqGyGCVTzqUTapgGA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=J2p/zbGk6Zv7M1uWK23I4wpo4qtjWlTEQyKJgfH8TlDpnOcepY1SPdZiJ8GflqdG3q03Oeir9la4v6CqKlDEmVzLDwd4oCqX2mhdYRahs/d+OEV+K6UXD6VHvc6ejBT4JGnYRYqqwjZHWxv7eHQYrvK+9MlEEmdPGFW3UoZULhg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=PZIymVBY; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PZIymVBY" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 989841961; Fri, 13 Sep 2024 23:45:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263949; bh=C/tLRufzYECRe+GhTy0TGewSXXIqGyGCVTzqUTapgGA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PZIymVBY0tIA3OpULT+ZeT9fear0PwP0kLVBu5GrxIv9zKNPHqGPYXuUqdJEqN1t4 wT9dP0Z1Mgg1pk3iK7Gn61PPA/dttUv9BfZQLWlSihWyhnlfWmXn/2mgP7uOYouwJb rE1JqqZVD/ld3KGInnCUiEDtEgHKWJ8gzpgtn/Vk= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 04/10] media: v4l2-ioctl: Introduce VIDIOC_BIND_CONTEXT Date: Fri, 13 Sep 2024 23:46:49 +0200 Message-ID: <20240913214657.1502838-5-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Introduce a new ioctl in V4L2 to allocate a video device context and associate it with a media device context. The ioctl is valid only if support for MEDIA_CONTROLLER is compiled in as it calls into the entity ops to let driver allocate a new context and binds the newly created context with the media context associated with the file descriptor provided by userspace as the new ioctl argument. The newly allocated video context is then stored in the v4l2-fh that represent the open file handle on which the ioctl has been called. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-dev.c | 10 +++++ drivers/media/v4l2-core/v4l2-fh.c | 1 + drivers/media/v4l2-core/v4l2-ioctl.c | 64 ++++++++++++++++++++++++++++ include/media/v4l2-ioctl.h | 5 +++ include/uapi/linux/videodev2.h | 11 +++++ 5 files changed, 91 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index 0fd6ff3f00f6..41719b009c1e 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -628,6 +628,10 @@ static void determine_valid_ioctls(struct video_device *vdev) __set_bit(_IOC_NR(VIDIOC_ENUM_FREQ_BANDS), valid_ioctls); if (is_vid) { +#ifdef CONFIG_MEDIA_CONTROLLER + __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls); +#endif + /* video specific ioctls */ if ((is_rx && (ops->vidioc_enum_fmt_vid_cap || ops->vidioc_enum_fmt_vid_overlay)) || @@ -681,12 +685,18 @@ static void determine_valid_ioctls(struct video_device *vdev) SET_VALID_IOCTL(ops, VIDIOC_G_FMT, vidioc_g_fmt_meta_cap); SET_VALID_IOCTL(ops, VIDIOC_S_FMT, vidioc_s_fmt_meta_cap); SET_VALID_IOCTL(ops, VIDIOC_TRY_FMT, vidioc_try_fmt_meta_cap); +#ifdef CONFIG_MEDIA_CONTROLLER + __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls); +#endif } else if (is_meta && is_tx) { /* metadata output specific ioctls */ SET_VALID_IOCTL(ops, VIDIOC_ENUM_FMT, vidioc_enum_fmt_meta_out); SET_VALID_IOCTL(ops, VIDIOC_G_FMT, vidioc_g_fmt_meta_out); SET_VALID_IOCTL(ops, VIDIOC_S_FMT, vidioc_s_fmt_meta_out); SET_VALID_IOCTL(ops, VIDIOC_TRY_FMT, vidioc_try_fmt_meta_out); +#ifdef CONFIG_MEDIA_CONTROLLER + __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls); +#endif } if (is_vbi) { /* vbi specific ioctls */ diff --git a/drivers/media/v4l2-core/v4l2-fh.c b/drivers/media/v4l2-core/v4l2-fh.c index 90eec79ee995..f7af444d2344 100644 --- a/drivers/media/v4l2-core/v4l2-fh.c +++ b/drivers/media/v4l2-core/v4l2-fh.c @@ -93,6 +93,7 @@ int v4l2_fh_release(struct file *filp) struct v4l2_fh *fh = filp->private_data; if (fh) { + video_device_context_put(fh->context); v4l2_fh_del(fh); v4l2_fh_exit(fh); kfree(fh); diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 4119b23bb954..0e37e777d21f 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -9,6 +9,7 @@ */ #include +#include #include #include #include @@ -349,6 +350,13 @@ static void v4l_print_format(const void *arg, bool write_only) } } +static void v4l_print_context(const void *arg, bool write_only) +{ + const struct v4l2_context *c = arg; + + pr_cont("context=%u\n", c->context_fd); +} + static void v4l_print_framebuffer(const void *arg, bool write_only) { const struct v4l2_framebuffer *p = arg; @@ -2110,6 +2118,61 @@ static int v4l_overlay(const struct v4l2_ioctl_ops *ops, return ops->vidioc_overlay(file, fh, *(unsigned int *)arg); } +static int v4l_bind_context(const struct v4l2_ioctl_ops *ops, + struct file *file, void *fh, void *arg) +{ + struct video_device *vdev = video_devdata(file); + struct media_device_context *mdev_context; + struct v4l2_fh *vfh = + test_bit(V4L2_FL_USES_V4L2_FH, &vdev->flags) ? fh : NULL; + struct v4l2_context *c = arg; + int ret; + + /* + * TODO: do not __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls) + * if V4L2_FL_USES_V4L2_FH isn't set or the driver does not implement + * alloc_context and destroy_context. + */ + if (!vfh) + return -ENOTTY; + + if (!vdev->entity.ops || !vdev->entity.ops->alloc_context || + !vdev->entity.ops->destroy_context) + return -ENOTTY; + + mdev_context = media_device_context_get_from_fd(c->context_fd); + if (!mdev_context) + return -EINVAL; + + /* Let the driver allocate the per-file handle context. */ + ret = vdev->entity.ops->alloc_context(&vdev->entity, + (struct media_entity_context **) + &vfh->context); + if (ret) + goto err_put_mdev_context; + + /* + * Bind the newly created video device context to the media device + * context identified by the file descriptor. + */ + ret = media_device_bind_context(mdev_context, + (struct media_entity_context *) + vfh->context); + if (ret) + goto err_put_context; + + media_device_context_put(mdev_context); + + return 0; + +err_put_context: + video_device_context_put(vfh->context); +err_put_mdev_context: + media_device_context_put(mdev_context); + + return ret; +} + static int v4l_reqbufs(const struct v4l2_ioctl_ops *ops, struct file *file, void *fh, void *arg) { @@ -2853,6 +2916,7 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = { IOCTL_INFO(VIDIOC_ENUM_FMT, v4l_enum_fmt, v4l_print_fmtdesc, 0), IOCTL_INFO(VIDIOC_G_FMT, v4l_g_fmt, v4l_print_format, 0), IOCTL_INFO(VIDIOC_S_FMT, v4l_s_fmt, v4l_print_format, INFO_FL_PRIO), + IOCTL_INFO(VIDIOC_BIND_CONTEXT, v4l_bind_context, v4l_print_context, 0), IOCTL_INFO(VIDIOC_REQBUFS, v4l_reqbufs, v4l_print_requestbuffers, INFO_FL_PRIO | INFO_FL_QUEUE), IOCTL_INFO(VIDIOC_QUERYBUF, v4l_querybuf, v4l_print_buffer, INFO_FL_QUEUE | INFO_FL_CLEAR(v4l2_buffer, length)), IOCTL_INFO(VIDIOC_G_FBUF, v4l_stub_g_fbuf, v4l_print_framebuffer, 0), diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h index edb733f21604..a70d2e446351 100644 --- a/include/media/v4l2-ioctl.h +++ b/include/media/v4l2-ioctl.h @@ -18,6 +18,7 @@ #include struct v4l2_fh; +struct video_device_context; /** * struct v4l2_ioctl_ops - describe operations for each V4L2 ioctl @@ -406,6 +407,10 @@ struct v4l2_ioctl_ops { int (*vidioc_try_fmt_meta_out)(struct file *file, void *fh, struct v4l2_format *f); + /* Context handlers */ + int (*vidioc_bind_context)(struct file *file, void *fh, + struct video_device_context *c); + /* Buffer handlers */ int (*vidioc_reqbufs)(struct file *file, void *fh, struct v4l2_requestbuffers *b); diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index f8157c9c9da6..0b7f12fd798e 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -1035,6 +1035,14 @@ struct v4l2_jpegcompression { * always use APP0 */ }; +/* + * V I D E O D E V I C E C O N T E X T + */ + +struct v4l2_context { + __u32 context_fd; +}; + /* * M E M O R Y - M A P P I N G B U F F E R S */ @@ -2758,6 +2766,9 @@ struct v4l2_create_buffers { #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl) +/* Context handling */ +#define VIDIOC_BIND_CONTEXT _IOW('V', 104, struct v4l2_context) + /* Reminder: when adding new ioctls please add support for them to drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */ From patchwork Fri Sep 13 21:46:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13804114 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4183B1494DC for ; Fri, 13 Sep 2024 21:47:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264044; cv=none; b=kjhnVz/Wo7bEJlQ5aSQzjsWM5J1OoI+6nOpjfKWfLMeE0eynlMYijuRsE0x4orA2UWYXMhgX1IjKLeckT1tyfhWWWact00UC2En8JKY/0KYAej5Ux99TJr4U3Jn5W3KdI1+4GIl6Hfz+qF+AKJPkKnQE/u4vMi2bz+HqmLK4suE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264044; c=relaxed/simple; bh=Mnk9/vCsnWJmWkDF7clpn0Msnwh3eeWCj3x+YSeI8Io=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=d1j9ml8le7hJC5e5REscQ7elTLnUdJml6MyVFvm8Usrz1mA36MB9oKTNB9KodDjKlZwhn3Ob5pHBCBuP4eNJBUUa8vujq2cHls/AR+nOCAbcp88zdWMVpIl2Q2X0ku/GOctrc9H3cXn43DKnSLLAjNMqRjjnWHHQ81ptWsx6oPw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=dHmNoi3d; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dHmNoi3d" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DF11A19BC; Fri, 13 Sep 2024 23:45:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263950; bh=Mnk9/vCsnWJmWkDF7clpn0Msnwh3eeWCj3x+YSeI8Io=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dHmNoi3dkn5wnWfSXyeQSK8csm/6SLjG0UZ+VboxYWYF0BWMxIacVFDGCfIIQN5we 8K7XWXI0pPYFp4pbDnDZF8p1R1ezZwXvxezbFPj+FRsNgZE6bG7faR8Q3YYLHVeK19 GRDmTuV6ID8g4b/kxwvEu9tr4ekpuUXZq+wBZ9No= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 05/10] media: Introduce default contexts Date: Fri, 13 Sep 2024 23:46:50 +0200 Message-ID: <20240913214657.1502838-6-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Introduce the media device and video device default contexts. Drivers ported to use multi-context support that used to work with a non-context aware userspace (which doesn't call VIDIOC_BIND_CONTEXT) shall continue to work even if they are context aware. Provide a default context in the media device and in the video device structures and let drivers allocate and release them with the newly introduced operations. Bind the video device default context to the default context of the media device associated with the v4l2_dev. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-device.c | 11 ++++++++ drivers/media/v4l2-core/v4l2-dev.c | 43 +++++++++++++++++++++++++++++- include/media/media-device.h | 5 ++++ include/media/v4l2-dev.h | 1 + 4 files changed, 59 insertions(+), 1 deletion(-) diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c index 1b24d5e74ffb..b4ebe30c21aa 100644 --- a/drivers/media/mc/mc-device.c +++ b/drivers/media/mc/mc-device.c @@ -828,6 +828,16 @@ void media_device_init(struct media_device *mdev) media_set_bus_info(mdev->bus_info, sizeof(mdev->bus_info), mdev->dev); + mdev->default_context = NULL; + if (mdev->ops && + mdev->ops->alloc_context && mdev->ops->destroy_context) { + if (mdev->ops->alloc_context(mdev, &mdev->default_context)) { + dev_err(mdev->dev, + "Failed to initialize media device default context\n"); + return; + } + } + dev_dbg(mdev->dev, "Media device initialized\n"); } EXPORT_SYMBOL_GPL(media_device_init); @@ -836,6 +846,7 @@ void media_device_cleanup(struct media_device *mdev) { WARN_ON(mdev->ops && mdev->ops->release); __media_device_release(mdev); + media_device_context_put(mdev->default_context); media_device_put(mdev); } EXPORT_SYMBOL_GPL(media_device_cleanup); diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index 41719b009c1e..73462191fd17 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -31,6 +31,8 @@ #include #include +#include + #define VIDEO_NUM_DEVICES 256 #define VIDEO_NAME "video4linux" @@ -221,6 +223,12 @@ static void v4l2_device_release(struct device *cd) media_device_put(mdev); #endif + /* Release default context. */ +#ifdef CONFIG_MEDIA_CONTROLLER + video_device_context_put(vdev->default_context); +#endif + vdev->default_context = NULL; + /* Release video_device and perform other cleanups as needed. */ vdev->release(vdev); @@ -1096,7 +1104,36 @@ int __video_register_device(struct video_device *vdev, /* Part 5: Register the entity. */ ret = video_register_media_controller(vdev); - /* Part 6: Activate this minor. The char device can now be used. */ + /* + * Part 6: Complete the video device registration by initializing the + * default context. The defaul context serves for context-aware driver + * to operate with a non-context-aware userspace that never creates + * new contexts. If the video device driver is not context aware, it + * will never implement 'context_ops' and will never use the default + * context. + */ + vdev->default_context = NULL; +#ifdef CONFIG_MEDIA_CONTROLLER + if (vdev->entity.ops && vdev->entity.ops->alloc_context && + vdev->entity.ops->destroy_context) { + ret = vdev->entity.ops->alloc_context(&vdev->entity, + (struct media_entity_context **) + &vdev->default_context); + if (ret) { + pr_err("%s: default context alloc failed\n", __func__); + goto cleanup; + } + + ret = media_device_bind_context(vdev->v4l2_dev->mdev->default_context, + &vdev->default_context->base); + if (ret) { + pr_err("%s: default context bind failed\n", __func__); + goto cleanup; + } + } +#endif + + /* Part 7: Activate this minor. The char device can now be used. */ set_bit(V4L2_FL_REGISTERED, &vdev->flags); mutex_unlock(&videodev_lock); @@ -1104,6 +1141,10 @@ int __video_register_device(struct video_device *vdev, cleanup: mutex_lock(&videodev_lock); +#ifdef CONFIG_MEDIA_CONTROLLER + video_device_context_put(vdev->default_context); +#endif + vdev->default_context = NULL; if (vdev->cdev) cdev_del(vdev->cdev); video_devices[vdev->minor] = NULL; diff --git a/include/media/media-device.h b/include/media/media-device.h index a73f650f122f..74c0a2e1f003 100644 --- a/include/media/media-device.h +++ b/include/media/media-device.h @@ -185,6 +185,9 @@ struct media_device_ops { * @fh_list: List of file handles in the media device * (struct media_device_fh.mdev_list). * @fh_list_lock: Serialise access to fh_list list. + * @default_context: The default video device context. Used by drivers that + * support multi-context operation when operated by a + * non-context aware userspace. * * This structure represents an abstract high-level media device. It allows easy * access to entities and provides basic media device-level support. The @@ -261,6 +264,8 @@ struct media_device { struct list_head fh_list; spinlock_t fh_list_lock; + + struct media_device_context *default_context; }; /* We don't need to include usb.h here */ diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index d538f0f32d6d..b79b14e36a2a 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -282,6 +282,7 @@ struct video_device { struct vb2_queue *queue; + struct video_device_context *default_context; struct v4l2_prio_state *prio; /* device info */ From patchwork Fri Sep 13 21:46:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13804115 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 327EA1527AC for ; Fri, 13 Sep 2024 21:47:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264047; cv=none; b=AV+8+oFHDt8bJU08TlUn1LKUshWigEfsHDodhxSA1zGOLKDqq7jugtdVIrmzv9NzhyHknkd118YYNwJd59vaYfexdHLUYDSZ/x8cej1CIlPftEpjlZZ7wDLv/gcJrebCz4lvdpujGvzcbWz832UgJG7MlHR6ecELzjjhEewn3L4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264047; c=relaxed/simple; bh=I9XkynIK6QYsUmsKZc85wfx8aP+41+aqdO1VT7BsRyg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YNUG8ERnfRXZfBkevA2cVoWAMyc173znPRKc+iisnCJIJvQkEch7ocff+s3aRdNQKIs5pHAN4sb1/v5IhdV8cmWAplw2Yt5z0fTxlzrPa3RZRhe0KYR1OnCc3m6cU0WBxA8wJbfzST4GY1AFc4umpTeiREJRjyAaT0hutOaj2i0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=UzsxQsWp; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UzsxQsWp" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 320311A6F; Fri, 13 Sep 2024 23:45:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263950; bh=I9XkynIK6QYsUmsKZc85wfx8aP+41+aqdO1VT7BsRyg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UzsxQsWpn/b3CDiEBYJkcPTwqgaAALbGpbP0BQQfnobwEROqwyWIPf7cnyBUcXo4f 2BoUl4aQLoF9BleLz8n5rxHPcZD0tNUoNy5l0aZB7zGaAv3fi5sJRPr4KGEqC0+f4X VQ028sNS/GUCb86JnCDW2T1vd2al7FNgXffs2Wfk= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 06/10] media: v4l2-dev: Add video_device_context_from_file() Date: Fri, 13 Sep 2024 23:46:51 +0200 Message-ID: <20240913214657.1502838-7-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Introduce an helper function to get a video_device_context from file. If no context has been bound to the file, return the video device default context. As a video device context lifetime is bound to the one of the file handle it is associated with, and the intended user of this helper are the v4l2_ioctl_ops handlers that operates on an open file handle, the context returned by this function is guaranteed to be valid for the whole function scope of the caller. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-dev.c | 15 +++++++++++++++ include/media/v4l2-dev.h | 30 ++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index 73462191fd17..d92989d76cf0 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -1181,6 +1181,21 @@ void video_unregister_device(struct video_device *vdev) } EXPORT_SYMBOL(video_unregister_device); +struct video_device_context * +video_device_context_from_file(struct file *filp, struct video_device *vfd) +{ + struct v4l2_fh *vfh = + test_bit(V4L2_FL_USES_V4L2_FH, &vfd->flags) ? filp->private_data + : NULL; + + /* If the file handle has been bound to a context return it. */ + if (vfh && vfh->context) + return vfh->context; + + return vfd->default_context; +} +EXPORT_SYMBOL_GPL(video_device_context_from_file); + #if defined(CONFIG_MEDIA_CONTROLLER) __must_check int video_device_pipeline_start(struct video_device *vdev, diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index b79b14e36a2a..32e61f9f9cb5 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -766,6 +766,36 @@ int video_device_init_context(struct video_device *vdev, */ void video_device_cleanup_context(struct video_device_context *ctx); +/** + * video_device_context_from_file - Get the video device context associated with + * an open file handle + * @filp: The file handle + * @vdev: The video device + * + * If a video device context has been bound to an open file handle by a call + * to the VIDIOC_BIND_CONTEXT ioctl this function returns it, otherwise returns + * the default video device context. + * + * The intended callers of this helper are the driver-specific v4l2_ioctl_ops + * handlers, which receive as first argument a file pointer. By using this + * helper drivers can get the context associated with such file, if any. + * Otherwise, if the video device has not been bound to any media device + * context, the default video device context is returned. + * + * As video device contexts are reference counted and their lifetime is + * guaranteed to be at least the one of the file handle they are associated + * with, callers of this function are guaranteed to always receive a valid + * context reference by this function. The reference will remain valid for the + * whole function scope of the caller that has received the open file handle as + * argument. Likewise, accessing the media context from the video device context + * returned by this function is always safe within the same function scope. + * + * This function does not increase the reference count of the returned video + * device context. + */ +struct video_device_context * +video_device_context_from_file(struct file *filp, struct video_device *vdev); + #endif /* CONFIG_MEDIA_CONTROLLER */ #endif /* _V4L2_DEV_H */ From patchwork Fri Sep 13 21:46:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13804116 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B3DB1494DC for ; Fri, 13 Sep 2024 21:47:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264047; cv=none; b=XvDTaacdK2H1lZRzJ104qrQc8odXcy6+Y/LIgB0wlQ87BNxMT/xAdaIpbqdHrQ5J/Qt46XmOpkj1jLKRT3rafIlBIU8e311mfAB2EClpipklfHZyMh6rGbx1ket3vf25U7p+KsPoNU3hFQvzkLwonMcleUs731m7dxTYM4G1kgI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264047; c=relaxed/simple; bh=xxUPyC7ZR+OTUVsrPaK7+U/zRU0HEWUUqm07wIEEjSo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fv3Omovj6i1Sj1G3ylpE6NB6FrEZICGgzSEZys5/HHS44tFnofqJJ/OiK3qT0Dfi9oyb2IfPJeKcka9NPf2VFwgowdX5WtVRJISKMlanBzuVXYyyYPKH81vMLKHz5CJqF08b9ArtCORBA9+V0iZirdhSnAsRh7tO8pxTmE6y7nM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=RyKLiwd3; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="RyKLiwd3" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 770221BC7; Fri, 13 Sep 2024 23:45:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263950; bh=xxUPyC7ZR+OTUVsrPaK7+U/zRU0HEWUUqm07wIEEjSo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RyKLiwd3Qplthlnz9Gqc3QcrJsk2BFKSUQ9WIQvqWzIMFqqASVupUkhDdhE8OMG6t ReQaafKK8LNSyCl6YqE/DSp1UQZPhQuyIKGY7eiwtOjGq6gCd71NzX/OlFW5TEKhig rMbA7I558dPpkiUcV8DairPSqlNaElATtOAvWG68= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 07/10] media: v4l2-dev: Add video_device_context_from_queue() Date: Fri, 13 Sep 2024 23:46:52 +0200 Message-ID: <20240913214657.1502838-8-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Introduce an helper function to get a video_device_context from a videobuf2 queue. Device drivers that support multi-context operations will receive in their vb2_ops callbacks implementation a vb2_queue argument that is embedded in a video device context, either a context created by binding the video device to a media device context or the video device default context. This helper allows those drivers to retrieve the context the vb2_queue is embedded with. As the intended callers of this helpers are vb2_ops callbacks implementation, called by the videobuf2 core in response to an operation on a file handle, the returned context is guaranteed to be valid for the whole duration of the callback. Signed-off-by: Jacopo Mondi --- include/media/v4l2-dev.h | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index 32e61f9f9cb5..2ef49d0aaf09 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -796,6 +796,42 @@ void video_device_cleanup_context(struct video_device_context *ctx); struct video_device_context * video_device_context_from_file(struct file *filp, struct video_device *vdev); +/** + * video_device_context_from_queue - Get a video device context associated with + * a vb2 queue + * + * @q: the videobuf2 queue + * + * Return the video device context this videobuf2 queue belongs to. + * + * Device drivers that support multi-context operations will always be provided + * with a device context to work with, either a context created by userspace + * by binding the video device to a media device context or the video device + * default context. The videobuf2 queue that context-aware drivers operates will + * always be contained in either one of those contexts. + * + * This function should be used by driver-specific callbacks implementation of + * the vb2_ops which receive a videobuf2 as argument. Being the vb2_ops callback + * called by the videobuf2 core in response to a userspace operation on an open + * file handle, callers of this function are guaranteed to always receive a + * valid context reference by this function. The reference will remain valid for + * the whole function scope of the vb2_ops callback that has received the + * videobuf2 queue as argument. Likewise, accessing the media device context + * from the video device context returned by this function is always safe within + * the same function scope. + * + * Drivers that do not support multi-context operations shall never use + * this function. + * + * This function does not increase the reference count of the returned video + * device context. + */ +static inline struct video_device_context * +video_device_context_from_queue(struct vb2_queue *q) +{ + return container_of(q, struct video_device_context, queue); +} + #endif /* CONFIG_MEDIA_CONTROLLER */ #endif /* _V4L2_DEV_H */ From patchwork Fri Sep 13 21:46:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13804117 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50E6A1494DC for ; Fri, 13 Sep 2024 21:47:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264051; cv=none; b=niSgTuNjQAimzTRvqQ/ocS+38nSu7FA71xmuqJeyFUuoOT+RxY3VDwMdeekj8WvpFm88GuplTYoJRD/KXdb+TXxjyhaLmdHYt4SrujKewVUupmgNMihCDaRDrVQ2fGLLCPXdCbrMNb9qFy05Ec61IEmdaKS0RN5bxBVMumcaIww= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264051; c=relaxed/simple; bh=JkL1mUCZoVEqCrviXrBtMAXA0r6wMGrZjDYOteOnxVk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=F9OCdJIT+HgRTvHgbjt58yOwEtAmtgDNWafsbCZupwSw2xMHxqOGZbUHMZ/V0WwB7lWnsSbzLwe2Fyq2d73sYCIn2hlyZ3XruMv6iLNeNIg9QQX6ploF8F7Me6l8lanqPrbWmTGh2aTlSQdt3PT6oDPl0d4NntI8sQJdw0rnRgU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=OYTnzKUZ; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="OYTnzKUZ" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BC7862590; Fri, 13 Sep 2024 23:45:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263950; bh=JkL1mUCZoVEqCrviXrBtMAXA0r6wMGrZjDYOteOnxVk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OYTnzKUZSYI7BxfCrL9jdhEHkJs0xV0gCAjWrCLdxZsHdKhN7nBdlv1QAEo22Dg1I IoT1teSLOgzNuzMcqp3YvUAeSjyZEwj7BOH7UCN3I9ExvPjfWECqSKw9ElX0c6gx2o dulyIgWliZFH4feMuqQv+D69M4WhCMKZLF0rZGKE= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 08/10] videobuf2-v4l2: Support vb2_queue embedded in a context Date: Fri, 13 Sep 2024 23:46:53 +0200 Message-ID: <20240913214657.1502838-9-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Support device drivers that implement multi-context operations in the videobuf2 core by providing an helper to retrieve the vb2_queue from the context associated with an open file handle. If no context is associated with a file handle, retrieve it from the video device default context, created by the core for multi-context aware drivers. Fall-back to use the vb2_queue from the video_device to support existing drivers which are not context aware. Signed-off-by: Jacopo Mondi --- .../media/common/videobuf2/videobuf2-v4l2.c | 129 +++++++++++------- 1 file changed, 81 insertions(+), 48 deletions(-) diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c index c7a54d82a55e..a221436718c2 100644 --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c @@ -976,27 +976,46 @@ EXPORT_SYMBOL_GPL(vb2_poll); * and so they simplify the driver code. */ +/* + * Helper to get the vb2 queue either from: + * 1) The video context bound to the open file handle + * 2) The default context for context-aware drivers if userspace has not bound + * a context to the file handle + * 3) From the video device for non-context aware drivers + */ +static struct vb2_queue *get_vb2_queue(struct file *file, + struct video_device *vdev) +{ + struct video_device_context *ctx = + video_device_context_from_file(file, vdev); + + return ctx ? &ctx->queue + : vdev->default_context ? &vdev->default_context->queue + : vdev->queue; +} + /* vb2 ioctl helpers */ int vb2_ioctl_reqbufs(struct file *file, void *priv, struct v4l2_requestbuffers *p) { struct video_device *vdev = video_devdata(file); - int res = vb2_verify_memory_type(vdev->queue, p->memory, p->type); + struct vb2_queue *q = get_vb2_queue(file, vdev); + int res = vb2_verify_memory_type(q, p->memory, p->type); u32 flags = p->flags; - fill_buf_caps(vdev->queue, &p->capabilities); - validate_memory_flags(vdev->queue, p->memory, &flags); + fill_buf_caps(q, &p->capabilities); + validate_memory_flags(q, p->memory, &flags); p->flags = flags; if (res) return res; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - res = vb2_core_reqbufs(vdev->queue, p->memory, p->flags, &p->count); + res = vb2_core_reqbufs(q, p->memory, p->flags, &p->count); /* If count == 0, then the owner has released all buffers and he is no longer owner of the queue. Otherwise we have a new owner. */ if (res == 0) - vdev->queue->owner = p->count ? file->private_data : NULL; + q->owner = p->count ? file->private_data : NULL; return res; } EXPORT_SYMBOL_GPL(vb2_ioctl_reqbufs); @@ -1005,12 +1024,13 @@ int vb2_ioctl_create_bufs(struct file *file, void *priv, struct v4l2_create_buffers *p) { struct video_device *vdev = video_devdata(file); - int res = vb2_verify_memory_type(vdev->queue, p->memory, + struct vb2_queue *q = get_vb2_queue(file, vdev); + int res = vb2_verify_memory_type(q, p->memory, p->format.type); - p->index = vdev->queue->num_buffers; - fill_buf_caps(vdev->queue, &p->capabilities); - validate_memory_flags(vdev->queue, p->memory, &p->flags); + p->index = q->num_buffers; + fill_buf_caps(q, &p->capabilities); + validate_memory_flags(q, p->memory, &p->flags); /* * If count == 0, then just check if memory and type are valid. * Any -EBUSY result from vb2_verify_memory_type can be mapped to 0. @@ -1019,12 +1039,12 @@ int vb2_ioctl_create_bufs(struct file *file, void *priv, return res != -EBUSY ? res : 0; if (res) return res; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - res = vb2_create_bufs(vdev->queue, p); + res = vb2_create_bufs(q, p); if (res == 0) - vdev->queue->owner = file->private_data; + q->owner = file->private_data; return res; } EXPORT_SYMBOL_GPL(vb2_ioctl_create_bufs); @@ -1033,69 +1053,76 @@ int vb2_ioctl_prepare_buf(struct file *file, void *priv, struct v4l2_buffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_prepare_buf(vdev->queue, vdev->v4l2_dev->mdev, p); + return vb2_prepare_buf(q, vdev->v4l2_dev->mdev, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_prepare_buf); int vb2_ioctl_querybuf(struct file *file, void *priv, struct v4l2_buffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); /* No need to call vb2_queue_is_busy(), anyone can query buffers. */ - return vb2_querybuf(vdev->queue, p); + return vb2_querybuf(q, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_querybuf); int vb2_ioctl_qbuf(struct file *file, void *priv, struct v4l2_buffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_qbuf(vdev->queue, vdev->v4l2_dev->mdev, p); + return vb2_qbuf(q, vdev->v4l2_dev->mdev, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_qbuf); int vb2_ioctl_dqbuf(struct file *file, void *priv, struct v4l2_buffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_dqbuf(vdev->queue, p, file->f_flags & O_NONBLOCK); + return vb2_dqbuf(q, p, file->f_flags & O_NONBLOCK); } EXPORT_SYMBOL_GPL(vb2_ioctl_dqbuf); int vb2_ioctl_streamon(struct file *file, void *priv, enum v4l2_buf_type i) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_streamon(vdev->queue, i); + return vb2_streamon(q, i); } EXPORT_SYMBOL_GPL(vb2_ioctl_streamon); int vb2_ioctl_streamoff(struct file *file, void *priv, enum v4l2_buf_type i) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_streamoff(vdev->queue, i); + return vb2_streamoff(q, i); } EXPORT_SYMBOL_GPL(vb2_ioctl_streamoff); int vb2_ioctl_expbuf(struct file *file, void *priv, struct v4l2_exportbuffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_expbuf(vdev->queue, p); + return vb2_expbuf(q, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf); @@ -1104,20 +1131,22 @@ EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf); int vb2_fop_mmap(struct file *file, struct vm_area_struct *vma) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - return vb2_mmap(vdev->queue, vma); + return vb2_mmap(q, vma); } EXPORT_SYMBOL_GPL(vb2_fop_mmap); int _vb2_fop_release(struct file *file, struct mutex *lock) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); if (lock) mutex_lock(lock); - if (file->private_data == vdev->queue->owner) { - vb2_queue_release(vdev->queue); - vdev->queue->owner = NULL; + if (file->private_data == q->owner) { + vb2_queue_release(q); + q->owner = NULL; } if (lock) mutex_unlock(lock); @@ -1128,7 +1157,8 @@ EXPORT_SYMBOL_GPL(_vb2_fop_release); int vb2_fop_release(struct file *file) { struct video_device *vdev = video_devdata(file); - struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock; + struct vb2_queue *q = get_vb2_queue(file, vdev); + struct mutex *lock = q->lock ? q->lock : vdev->lock; return _vb2_fop_release(file, lock); } @@ -1138,19 +1168,20 @@ ssize_t vb2_fop_write(struct file *file, const char __user *buf, size_t count, loff_t *ppos) { struct video_device *vdev = video_devdata(file); - struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock; + struct vb2_queue *q = get_vb2_queue(file, vdev); + struct mutex *lock = q->lock ? q->lock : vdev->lock; int err = -EBUSY; - if (!(vdev->queue->io_modes & VB2_WRITE)) + if (!(q->io_modes & VB2_WRITE)) return -EINVAL; if (lock && mutex_lock_interruptible(lock)) return -ERESTARTSYS; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) goto exit; - err = vb2_write(vdev->queue, buf, count, ppos, - file->f_flags & O_NONBLOCK); - if (vdev->queue->fileio) - vdev->queue->owner = file->private_data; + err = vb2_write(q, buf, count, ppos, + file->f_flags & O_NONBLOCK); + if (q->fileio) + q->owner = file->private_data; exit: if (lock) mutex_unlock(lock); @@ -1162,20 +1193,21 @@ ssize_t vb2_fop_read(struct file *file, char __user *buf, size_t count, loff_t *ppos) { struct video_device *vdev = video_devdata(file); - struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock; + struct vb2_queue *q = get_vb2_queue(file, vdev); + struct mutex *lock = q->lock ? q->lock : vdev->lock; int err = -EBUSY; - if (!(vdev->queue->io_modes & VB2_READ)) + if (!(q->io_modes & VB2_READ)) return -EINVAL; if (lock && mutex_lock_interruptible(lock)) return -ERESTARTSYS; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) goto exit; - vdev->queue->owner = file->private_data; - err = vb2_read(vdev->queue, buf, count, ppos, + q->owner = file->private_data; + err = vb2_read(q, buf, count, ppos, file->f_flags & O_NONBLOCK); - if (!vdev->queue->fileio) - vdev->queue->owner = NULL; + if (!q->fileio) + q->owner = NULL; exit: if (lock) mutex_unlock(lock); @@ -1186,7 +1218,7 @@ EXPORT_SYMBOL_GPL(vb2_fop_read); __poll_t vb2_fop_poll(struct file *file, poll_table *wait) { struct video_device *vdev = video_devdata(file); - struct vb2_queue *q = vdev->queue; + struct vb2_queue *q = get_vb2_queue(file, vdev); struct mutex *lock = q->lock ? q->lock : vdev->lock; __poll_t res; void *fileio; @@ -1202,7 +1234,7 @@ __poll_t vb2_fop_poll(struct file *file, poll_table *wait) fileio = q->fileio; - res = vb2_poll(vdev->queue, file, wait); + res = vb2_poll(q, file, wait); /* If fileio was started, then we have a new queue owner. */ if (!fileio && q->fileio) @@ -1218,8 +1250,9 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - return vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags); + return vb2_get_unmapped_area(q, addr, len, pgoff, flags); } EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area); #endif From patchwork Fri Sep 13 21:46:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13804118 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 872FD155321 for ; Fri, 13 Sep 2024 21:47:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264051; cv=none; b=PdII1cQIN2uGcpBn2/fnSXQsZc90/Hxdbr66wvVHYNH1aSI3Mg7zxdTLi3tRDifdbOvVU4Kz6cF70dgS4qyMTqAK83Y13Bz5x8JOlz3pHO5fhIENCImjAebrLcWWMGA95SstIgXbpSBPzCN/Mnu5OinzX2RNAOsb+Zm0hGpa550= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264051; c=relaxed/simple; bh=GUHZucC0wAGj+SaY2ua/nwT8wz9U+p/L2rERLawE1qE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hROIQzFoc9XmHvvJhQ/OUI0jBU4rpm8HxSEEOaA43VHXerLEx6oPU40LI1Uv95nQ2308q8nvroSqMRKLQLksruz3GrFsqVfTzPHYBFuFV6TE2C0Xr3AzQrOl4eBaYSKAjNAP1QqnD6jqCxlz+OdTxp9i8POhs36Uc9i1W1n8LMo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=ESxg6pjh; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ESxg6pjh" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0F17D2593; Fri, 13 Sep 2024 23:45:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263951; bh=GUHZucC0wAGj+SaY2ua/nwT8wz9U+p/L2rERLawE1qE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ESxg6pjhxMQ361CAXULysldLS4y/N5zK9qFERL3rV09WgiGb7ioPQIeOCtHwO2hU3 cP4GyYwSR1boJOdtgpCCCCmbACMyngMV5HlxqmhMF/OxXQKk5V0jF9Xo+vfNtJV84V 0yAgTztnWbbPz+Yzisofb2ywU94r6aUfvubvYxTY= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 09/10] media: media-entity: Validate context in pipeline start Date: Fri, 13 Sep 2024 23:46:54 +0200 Message-ID: <20240913214657.1502838-10-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add helpers to validate the an execution context when starting the media pipeline. Validating an execution context implies that when creating the streaming pipeline all the entities part of the pipeline shall have a valid context associated with the media context in use. When creating the media pipeline increase the device context use count to guarantee that during the streaming phase drivers will always operate with valid contexts. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-entity.c | 89 +++++++++++++++++++++++++++--- drivers/media/v4l2-core/v4l2-dev.c | 30 ++++++++++ include/media/media-entity.h | 42 +++++++++++++- include/media/v4l2-dev.h | 42 ++++++++++++++ 4 files changed, 194 insertions(+), 9 deletions(-) diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 4108d3da4cb0..b1932e3ddbdc 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -755,8 +755,51 @@ static int media_pipeline_populate(struct media_pipeline *pipe, return ret; } -__must_check int __media_pipeline_start(struct media_pad *pad, - struct media_pipeline *pipe) +static int +media_pipeline_validate_context(struct media_device_context *mdev_context, + struct media_entity *entity, + struct media_pipeline_pad *ppad) +{ + struct media_entity_context *context; + + if (!mdev_context) + return 0; + + /* + * It's not mandatory for all entities in the pipeline to support + * contexts. + */ + if (!entity->ops || !entity->ops->alloc_context || + !entity->ops->destroy_context) + return 0; + + /* + * But if they do they should be bound to the same media device context + * as all other entities. + */ + context = media_device_get_entity_context(mdev_context, entity); + + /* + * Fail validation if no context is associated with this media context. + */ + if (!context) + return -EINVAL; + + /* + * Increase the ref-counting of the context. Store a reference in the + * ppad for later decreasing the reference count when the pipeline is + * stopped. + */ + media_entity_context_get(context); + ppad->context = context; + + return 0; +} + +__must_check int +__media_pipeline_start_context(struct media_pad *pad, + struct media_pipeline *pipe, + struct media_device_context *mdev_context) { struct media_device *mdev = pad->graph_obj.mdev; struct media_pipeline_pad *err_ppad; @@ -816,7 +859,15 @@ __must_check int __media_pipeline_start(struct media_pad *pad, } /* - * 2. Validate all active links whose sink is the current pad. + * 2. If we have a media context, ensure the entity has a device + * context associated with it. + */ + ret = media_pipeline_validate_context(mdev_context, entity, ppad); + if (ret) + goto error; + + /* + * 3. Validate all active links whose sink is the current pad. * Validation of the source pads is performed in the context of * the connected sink pad to avoid duplicating checks. */ @@ -862,7 +913,7 @@ __must_check int __media_pipeline_start(struct media_pad *pad, } /* - * 3. If the pad has the MEDIA_PAD_FL_MUST_CONNECT flag set, + * 4. If the pad has the MEDIA_PAD_FL_MUST_CONNECT flag set, * ensure that it has either no link or an enabled link. */ if ((pad->flags & MEDIA_PAD_FL_MUST_CONNECT) && @@ -892,6 +943,9 @@ __must_check int __media_pipeline_start(struct media_pad *pad, if (err_ppad == ppad) break; + if (err_ppad->context) + media_entity_context_put(err_ppad->context); + err_ppad->pad->pipe = NULL; } @@ -899,19 +953,35 @@ __must_check int __media_pipeline_start(struct media_pad *pad, return ret; } +EXPORT_SYMBOL_GPL(__media_pipeline_start_context); + +__must_check int __media_pipeline_start(struct media_pad *pad, + struct media_pipeline *pipe) +{ + return __media_pipeline_start_context(pad, pipe, NULL); +} EXPORT_SYMBOL_GPL(__media_pipeline_start); -__must_check int media_pipeline_start(struct media_pad *pad, - struct media_pipeline *pipe) +__must_check int +media_pipeline_start_context(struct media_pad *pad, + struct media_pipeline *pipe, + struct media_device_context *context) { struct media_device *mdev = pad->graph_obj.mdev; int ret; mutex_lock(&mdev->graph_mutex); - ret = __media_pipeline_start(pad, pipe); + ret = __media_pipeline_start_context(pad, pipe, context); mutex_unlock(&mdev->graph_mutex); return ret; } +EXPORT_SYMBOL_GPL(media_pipeline_start_context); + +__must_check int media_pipeline_start(struct media_pad *pad, + struct media_pipeline *pipe) +{ + return media_pipeline_start_context(pad, pipe, NULL); +} EXPORT_SYMBOL_GPL(media_pipeline_start); void __media_pipeline_stop(struct media_pad *pad) @@ -929,8 +999,11 @@ void __media_pipeline_stop(struct media_pad *pad) if (--pipe->start_count) return; - list_for_each_entry(ppad, &pipe->pads, list) + list_for_each_entry(ppad, &pipe->pads, list) { + if (ppad->context) + media_entity_context_put(ppad->context); ppad->pad->pipe = NULL; + } media_pipeline_cleanup(pipe); diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index d92989d76cf0..791a32cd3fae 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -1222,6 +1222,36 @@ __must_check int __video_device_pipeline_start(struct video_device *vdev, } EXPORT_SYMBOL_GPL(__video_device_pipeline_start); +__must_check int +video_device_context_pipeline_start(struct video_device_context *context, + struct media_pipeline *pipe) +{ + struct video_device *vdev = context->vdev; + struct media_entity *entity = &vdev->entity; + + if (entity->num_pads != 1) + return -ENODEV; + + return media_pipeline_start_context(&entity->pads[0], pipe, + context->base.mdev_context); +} +EXPORT_SYMBOL(video_device_context_pipeline_start); + +__must_check int +__video_device_context_pipeline_start(struct video_device_context *context, + struct media_pipeline *pipe) +{ + struct video_device *vdev = context->vdev; + struct media_entity *entity = &vdev->entity; + + if (entity->num_pads != 1) + return -ENODEV; + + return __media_pipeline_start_context(&entity->pads[0], pipe, + context->base.mdev_context); +} +EXPORT_SYMBOL(__video_device_context_pipeline_start); + void video_device_pipeline_stop(struct video_device *vdev) { struct media_entity *entity = &vdev->entity; diff --git a/include/media/media-entity.h b/include/media/media-entity.h index 002a326de9b9..7161f8aea3fe 100644 --- a/include/media/media-entity.h +++ b/include/media/media-entity.h @@ -21,6 +21,7 @@ #include #include + /* Enums used internally at the media controller to represent graphs */ /** @@ -119,16 +120,20 @@ struct media_pipeline { * @list: Entry in the media_pad pads list * @pipe: The media_pipeline that the pad is part of * @pad: The media pad + * @context: Reference to a video device or subdevice context * * This structure associate a pad with a media pipeline. Instances of * media_pipeline_pad are created by media_pipeline_start() when it builds the * pipeline, and stored in the &media_pad.pads list. media_pipeline_stop() - * removes the entries from the list and deletes them. + * removes the entries from the list and deletes them. The context field is + * populated only if a valid context has been associated with the pad. */ +struct media_entity_context; struct media_pipeline_pad { struct list_head list; struct media_pipeline *pipe; struct media_pad *pad; + struct media_entity_context *context; }; /** @@ -1205,6 +1210,7 @@ struct media_entity *media_graph_walk_next(struct media_graph *graph); */ __must_check int media_pipeline_start(struct media_pad *pad, struct media_pipeline *pipe); + /** * __media_pipeline_start - Mark a pipeline as streaming * @@ -1216,6 +1222,40 @@ __must_check int media_pipeline_start(struct media_pad *pad, __must_check int __media_pipeline_start(struct media_pad *pad, struct media_pipeline *pipe); +/** + * media_pipeline_start_context - Mark a pipeline as streaming + * @pad: Starting pad + * @pipe: Media pipeline to be assigned to all pads in the pipeline. + * @context: The media device context the pipeline belongs to + * + * Mark all pads connected to a given pad through enabled links, either + * directly or indirectly, as streaming. The given pipeline object is assigned + * to every pad in the pipeline and stored in the media_pad pipe field. + * + * Calls to this function can be nested, in which case the same number of + * media_pipeline_stop() calls will be required to stop streaming. The + * pipeline pointer must be identical for all nested calls to + * media_pipeline_start(). + */ +__must_check int +media_pipeline_start_context(struct media_pad *pad, + struct media_pipeline *pipe, + struct media_device_context *context); + +/** + * __media_pipeline_start_context - Mark a pipeline as streaming + * + * @pad: Starting pad + * @pipe: Media pipeline to be assigned to all pads in the pipeline. + * @context: The media device context the pipeline belongs to + * + * ..note:: This is the non-locking version of media_pipeline_start() + */ +__must_check int +__media_pipeline_start_context(struct media_pad *pad, + struct media_pipeline *pipe, + struct media_device_context *mdev_context); + /** * media_pipeline_stop - Mark a pipeline as not streaming * @pad: Starting pad diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index 2ef49d0aaf09..1088bcac81a5 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -579,6 +579,48 @@ __must_check int video_device_pipeline_start(struct video_device *vdev, __must_check int __video_device_pipeline_start(struct video_device *vdev, struct media_pipeline *pipe); +/** + * video_device_context_pipeline_start - Mark a pipeline as streaming starting + * from a video device context + * @context: The video device context that starts the streaming + * @pipe: Media pipeline to be assigned to all entities in the pipeline. + * + * ..note:: This is the multi-context version of video_device_pipeline_start() + * Documentation of this function only describes specific aspects of + * this version. Refer to the video_device_pipeline_start() + * documentation for a complete reference. + * + * Validate that all entities connected to a video device through enabled links + * by ensuring that a context associated with the same media device context + * exists for them. Increase the reference counting of each of the contexts part + * of the pipeline to guarantee their lifetime is maintained as long as the + * pipeline is streaming. + * + * Context validation and refcounting of all entities that are part of a + * streaming pipeline ensures that device drivers can safely access device + * contexts in a media device context during streaming. References to contexts + * retried by a call to media_device_get_entity_context(), are guaranteed to be + * valid as long as the pipeline is streaming. Likewise, the media device + * context that contains the device contexts is guaranteed to be valid as long + * as the pipeline is streaming. + */ +__must_check int +video_device_context_pipeline_start(struct video_device_context *context, + struct media_pipeline *pipe); + +/** + * video_device_context_pipeline_start - Mark a pipeline as streaming starting + * from a video device context + * @context: The video device context that starts the streaming + * @pipe: Media pipeline to be assigned to all entities in the pipeline. + * + * ..note:: This is the non-locking version of + * __video_device_context_pipeline_start() + */ +__must_check int +__video_device_context_pipeline_start(struct video_device_context *context, + struct media_pipeline *pipe); + /** * video_device_pipeline_stop - Mark a pipeline as not streaming * @vdev: Starting video device From patchwork Fri Sep 13 21:46:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13804119 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE9FC155321 for ; Fri, 13 Sep 2024 21:47:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264055; cv=none; b=ZNJVkxErl4CD6elGQpCqohJpvtSK1n7lLX2yD2KxNvsYXVom10dng8jB7XtNLtYu7V99cSOp49KV2ncs7C5/afVEYue7Ir6Mf/PZp0iJHQ7Cihn6q2iFUnFsnp4MsDh+ZNa7AHhsf6t1xo3XnKdY2OTIKq7echtROPnb6hNNhR8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264055; c=relaxed/simple; bh=UjeQVJLvchVgUkD23gOYlJ7aYEUTepJcGMF+imJjm7A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dC7z6MX9/d7xd7v0AuuD66wP60mCM0n30SJjZUxPS7o42VrClY192c4DfxyfUZFfi5KPXe3XlwfqJjAa9uyEZTNEOP4qn3wohn3eYnXeHYlTlnOj2XO2OA9tuZD+XNx4at3Bl+afebZh9FpXnOMTTZVf2HZy4BGcALuBVYacu0w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=X0CmsT88; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="X0CmsT88" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5470C2598; Fri, 13 Sep 2024 23:45:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263951; bh=UjeQVJLvchVgUkD23gOYlJ7aYEUTepJcGMF+imJjm7A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=X0CmsT88crHPHV+70UFkLx4bfWissfunGauLCTF+nUBT1OLQrOzxXgl5v3MsBC5a+ TumUJtpP2+X/oet1RLBwwlgMm7IBOXloEGhVxi3x9YyUUTFQm7q1C7ZSn4AldS8szT pv1g83TAQuaZ17xrE4undLJqb34Gu3PwpNlxAG/E= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 10/10] media: pispbe: Add support for multi-context Date: Fri, 13 Sep 2024 23:46:55 +0200 Message-ID: <20240913214657.1502838-11-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add support for context multiplexing to the PiSP BackEnd driver. Move information that used to be per-node to the video context and global information to the media context. Implement media and video context allocation and release operations to allocate and initialize a context and initialize the video queue there contained. Remove the per-node video device format and the buffer queues and port the driver to use the ones in the video device context. The operations that used to work with a pispbe_node now operates on a pispbe_context. - v4l2 ioctl ops: receive a file pointer, retrieve the associated context from the open file handle - vb2 ops: receive a queue, retrieve the context from the queue - internal driver ops: given a media device contex retrieve the video context from the other video devices when assembling a job Signed-off-by: Jacopo Mondi --- .../platform/raspberrypi/pisp_be/pisp_be.c | 509 +++++++++++++----- 1 file changed, 370 insertions(+), 139 deletions(-) diff --git a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c index 555c28dddc4b..34fbe2b730b0 100644 --- a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c +++ b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c @@ -151,6 +151,70 @@ static const struct pispbe_node_description node_desc[PISPBE_NUM_NODES] = { * Structure to describe a single node /dev/video which represents a single * input or output queue to the PiSP Back End device. */ + +struct pispbe_media_context { + struct media_device_context mdev_context; + u32 streaming_map; +}; + +static struct pispbe_media_context * +pispbe_media_context(struct media_device_context *mdev_context) +{ + return container_of(mdev_context, struct pispbe_media_context, + mdev_context); +} + +struct pispbe_node; +struct pispbe_entity_context { + struct video_device_context vdev_context; + + struct v4l2_format format; + struct list_head ready_queue; + const struct pisp_be_format *pisp_format; + struct pispbe_dev *pispbe; + struct pispbe_node *node; +}; + +static struct pispbe_entity_context * +pispbe_entity_context(struct video_device_context *ctx) +{ + return container_of(ctx, struct pispbe_entity_context, vdev_context); +} + +static struct pispbe_entity_context * +pispbe_entity_context_from_queue(struct vb2_queue *queue) +{ + return pispbe_entity_context(video_device_context_from_queue(queue)); +} + +static struct pispbe_entity_context * +pispbe_entity_context_from_file(struct file *file, struct video_device *vfd) +{ + return pispbe_entity_context(video_device_context_from_file(file, vfd)); +} + +static struct pispbe_entity_context * +pispbe_get_dev_context(struct pispbe_media_context *pispbe_mdev_context, + struct video_device *vdev) +{ + struct video_device_context *ctx = + video_device_context_get(&pispbe_mdev_context->mdev_context, + vdev); + + return ctx ? pispbe_entity_context(ctx) : NULL; +} + +void pispbe_put_dev_context(struct pispbe_entity_context *ctx) +{ + video_device_context_put(&ctx->vdev_context); +} + +static struct pispbe_media_context * +pispbe_media_context_from_dev(struct pispbe_entity_context *context) +{ + return pispbe_media_context(context->vdev_context.base.mdev_context); +} + struct pispbe_node { unsigned int id; int vfl_dir; @@ -162,14 +226,13 @@ struct pispbe_node { struct pispbe_dev *pispbe; /* Video device lock */ struct mutex node_lock; - /* vb2_queue lock */ - struct mutex queue_lock; - struct list_head ready_queue; - struct vb2_queue queue; - struct v4l2_format format; - const struct pisp_be_format *pisp_format; }; +static struct pispbe_node *pispbe_node_from_vdev(struct video_device *vdev) +{ + return container_of(vdev, struct pispbe_node, vfd); +} + /* For logging only, use the entity name with "pispbe" and separator removed */ #define NODE_NAME(node) \ (node_desc[(node)->id].ent_name + sizeof(PISPBE_NAME)) @@ -216,15 +279,15 @@ struct pispbe_dev { struct pispbe_node node[PISPBE_NUM_NODES]; dma_addr_t config_dma_addr; unsigned int sequence; - u32 streaming_map; struct pispbe_job queued_job, running_job; - /* protects "hw_busy" flag, streaming_map and job_queue */ + /* protects "hw_busy" flag and job_queue */ spinlock_t hw_lock; bool hw_busy; /* non-zero if a job is queued or is being started */ struct list_head job_queue; int irq; u32 hw_version; u8 done, started; + struct media_pipeline pipe; }; static u32 pispbe_rd(struct pispbe_dev *pispbe, unsigned int offset) @@ -308,30 +371,31 @@ struct pispbe_buffer { }; static int pispbe_get_planes_addr(dma_addr_t addr[3], struct pispbe_buffer *buf, - struct pispbe_node *node) + struct pispbe_entity_context *context) { - unsigned int num_planes = node->format.fmt.pix_mp.num_planes; + struct v4l2_format *format = &context->format; + unsigned int num_planes = format->fmt.pix_mp.num_planes; unsigned int plane_factor = 0; unsigned int size; unsigned int p; - if (!buf || !node->pisp_format) + if (!buf || !context->pisp_format) return 0; /* * Determine the base plane size. This will not be the same - * as node->format.fmt.pix_mp.plane_fmt[0].sizeimage for a single + * as format->fmt.pix_mp.plane_fmt[0].sizeimage for a single * plane buffer in an mplane format. */ - size = node->format.fmt.pix_mp.plane_fmt[0].bytesperline * - node->format.fmt.pix_mp.height; + size = format->fmt.pix_mp.plane_fmt[0].bytesperline * + format->fmt.pix_mp.height; for (p = 0; p < num_planes && p < PISPBE_MAX_PLANES; p++) { addr[p] = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, p); - plane_factor += node->pisp_format->plane_factor[p]; + plane_factor += context->pisp_format->plane_factor[p]; } - for (; p < PISPBE_MAX_PLANES && node->pisp_format->plane_factor[p]; p++) { + for (; p < PISPBE_MAX_PLANES && context->pisp_format->plane_factor[p]; p++) { /* * Calculate the address offset of this plane as needed * by the hardware. This is specifically for non-mplane @@ -339,7 +403,7 @@ static int pispbe_get_planes_addr(dma_addr_t addr[3], struct pispbe_buffer *buf, * for the V4L2_PIX_FMT_YUV420 format. */ addr[p] = addr[0] + ((size * plane_factor) >> 3); - plane_factor += node->pisp_format->plane_factor[p]; + plane_factor += context->pisp_format->plane_factor[p]; } return num_planes; @@ -355,11 +419,13 @@ static dma_addr_t pispbe_get_addr(struct pispbe_buffer *buf) static void pispbe_xlate_addrs(struct pispbe_dev *pispbe, struct pispbe_job_descriptor *job, + struct pispbe_media_context *mdev_context, struct pispbe_buffer *buf[PISPBE_NUM_NODES]) { struct pispbe_hw_enables *hw_en = &job->hw_enables; struct pisp_be_tiles_config *config = job->config; dma_addr_t *addrs = job->hw_dma_addrs; + struct pispbe_entity_context *ctx; int ret; /* Take a copy of the "enable" bitmaps so we can modify them. */ @@ -370,8 +436,10 @@ static void pispbe_xlate_addrs(struct pispbe_dev *pispbe, * Main input first. There are 3 address pointers, corresponding to up * to 3 planes. */ - ret = pispbe_get_planes_addr(addrs, buf[MAIN_INPUT_NODE], - &pispbe->node[MAIN_INPUT_NODE]); + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[MAIN_INPUT_NODE].vfd); + ret = pispbe_get_planes_addr(addrs, buf[MAIN_INPUT_NODE], ctx); + pispbe_put_dev_context(ctx); if (ret <= 0) { /* Shouldn't happen, we have validated an input is available. */ dev_warn(pispbe->dev, "ISP-BE missing input\n"); @@ -427,9 +495,11 @@ static void pispbe_xlate_addrs(struct pispbe_dev *pispbe, /* Main image output channels. */ for (unsigned int i = 0; i < PISP_BACK_END_NUM_OUTPUTS; i++) { + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[OUTPUT0_NODE + i].vfd); ret = pispbe_get_planes_addr(addrs + 7 + 3 * i, - buf[OUTPUT0_NODE + i], - &pispbe->node[OUTPUT0_NODE + i]); + buf[OUTPUT0_NODE + i], ctx); + pispbe_put_dev_context(ctx); if (ret <= 0) hw_en->rgb_enables &= ~(PISP_BE_RGB_ENABLE_OUTPUT0 << i); } @@ -450,35 +520,46 @@ static void pispbe_xlate_addrs(struct pispbe_dev *pispbe, * * Returns 0 if a job has been successfully prepared, < 0 otherwise. */ -static int pispbe_prepare_job(struct pispbe_dev *pispbe) +static int pispbe_prepare_job(struct pispbe_dev *pispbe, + struct pispbe_entity_context *context) { + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); struct pispbe_buffer *buf[PISPBE_NUM_NODES] = {}; struct pispbe_job_descriptor *job; + struct pispbe_entity_context *ctx; unsigned int streaming_map; unsigned int config_index; - struct pispbe_node *node; scoped_guard(spinlock_irq, &pispbe->hw_lock) { static const u32 mask = BIT(CONFIG_NODE) | BIT(MAIN_INPUT_NODE); - if ((pispbe->streaming_map & mask) != mask) + if ((mdev_context->streaming_map & mask) != mask) return -ENODEV; /* * Take a copy of streaming_map: nodes activated after this * point are ignored when preparing this job. */ - streaming_map = pispbe->streaming_map; + streaming_map = mdev_context->streaming_map; } job = kzalloc(sizeof(*job), GFP_KERNEL); if (!job) return -ENOMEM; - node = &pispbe->node[CONFIG_NODE]; - buf[CONFIG_NODE] = list_first_entry_or_null(&node->ready_queue, + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[CONFIG_NODE].vfd); + if (!ctx) { + kfree(job); + return -ENODEV; + } + + buf[CONFIG_NODE] = list_first_entry_or_null(&ctx->ready_queue, struct pispbe_buffer, ready_list); + pispbe_put_dev_context(ctx); + if (!buf[CONFIG_NODE]) { kfree(job); return -ENODEV; @@ -530,12 +611,17 @@ static int pispbe_prepare_job(struct pispbe_dev *pispbe) ignore_buffers = true; } - node = &pispbe->node[i]; - /* Pull a buffer from each V4L2 queue to form the queued job */ - buf[i] = list_first_entry_or_null(&node->ready_queue, + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[i].vfd); + if (!ctx && !ignore_buffers) + goto err_return_buffers; + + buf[i] = list_first_entry_or_null(&ctx->ready_queue, struct pispbe_buffer, ready_list); + pispbe_put_dev_context(ctx); + if (buf[i]) { list_del(&buf[i]->ready_list); job->buffers[i] = buf[i]; @@ -546,7 +632,7 @@ static int pispbe_prepare_job(struct pispbe_dev *pispbe) } /* Convert buffers to DMA addresses for the hardware */ - pispbe_xlate_addrs(pispbe, job, buf); + pispbe_xlate_addrs(pispbe, job, mdev_context, buf); spin_lock(&pispbe->hw_lock); list_add_tail(&job->queue, &pispbe->job_queue); @@ -556,13 +642,17 @@ static int pispbe_prepare_job(struct pispbe_dev *pispbe) err_return_buffers: for (unsigned int i = 0; i < PISPBE_NUM_NODES; i++) { - struct pispbe_node *n = &pispbe->node[i]; - if (!buf[i]) continue; /* Return the buffer to the ready_list queue */ - list_add(&buf[i]->ready_list, &n->ready_queue); + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[i].vfd); + if (!ctx) + continue; + + list_add(&buf[i]->ready_list, &ctx->ready_queue); + pispbe_put_dev_context(ctx); } kfree(job); @@ -570,7 +660,9 @@ static int pispbe_prepare_job(struct pispbe_dev *pispbe) return -ENODEV; } -static void pispbe_schedule(struct pispbe_dev *pispbe, bool clear_hw_busy) +static void pispbe_schedule(struct pispbe_dev *pispbe, + struct pispbe_media_context *mdev_context, + bool clear_hw_busy) { struct pispbe_job_descriptor *job; @@ -674,18 +766,20 @@ static irqreturn_t pispbe_isr(int irq, void *dev) } /* check if there's more to do before going to sleep */ - pispbe_schedule(pispbe, can_queue_another); + pispbe_schedule(pispbe, NULL, can_queue_another); return IRQ_HANDLED; } static int pisp_be_validate_config(struct pispbe_dev *pispbe, + struct pispbe_media_context *mdev_context, struct pisp_be_tiles_config *config) { u32 bayer_enables = config->config.global.bayer_enables; u32 rgb_enables = config->config.global.rgb_enables; + struct pispbe_entity_context *context; + struct v4l2_pix_format_mplane *mp; struct device *dev = pispbe->dev; - struct v4l2_format *fmt; unsigned int bpl, size; if (!(bayer_enables & PISP_BE_BAYER_ENABLE_INPUT) == @@ -702,36 +796,50 @@ static int pisp_be_validate_config(struct pispbe_dev *pispbe, } /* Ensure output config strides and buffer sizes match the V4L2 formats. */ - fmt = &pispbe->node[TDN_OUTPUT_NODE].format; + context = pispbe_get_dev_context(mdev_context, + &pispbe->node[TDN_OUTPUT_NODE].vfd); + if (!context) + return -EINVAL; + + mp = &context->format.fmt.pix_mp; + pispbe_put_dev_context(context); + if (bayer_enables & PISP_BE_BAYER_ENABLE_TDN_OUTPUT) { bpl = config->config.tdn_output_format.stride; size = bpl * config->config.tdn_output_format.height; - if (fmt->fmt.pix_mp.plane_fmt[0].bytesperline < bpl) { + if (mp->plane_fmt[0].bytesperline < bpl) { dev_dbg(dev, "%s: bpl mismatch on tdn_output\n", __func__); return -EINVAL; } - if (fmt->fmt.pix_mp.plane_fmt[0].sizeimage < size) { + if (mp->plane_fmt[0].sizeimage < size) { dev_dbg(dev, "%s: size mismatch on tdn_output\n", __func__); return -EINVAL; } } - fmt = &pispbe->node[STITCH_OUTPUT_NODE].format; + context = pispbe_get_dev_context(mdev_context, + &pispbe->node[STITCH_OUTPUT_NODE].vfd); + if (!context) + return -EINVAL; + + mp = &context->format.fmt.pix_mp; + pispbe_put_dev_context(context); + if (bayer_enables & PISP_BE_BAYER_ENABLE_STITCH_OUTPUT) { bpl = config->config.stitch_output_format.stride; size = bpl * config->config.stitch_output_format.height; - if (fmt->fmt.pix_mp.plane_fmt[0].bytesperline < bpl) { + if (mp->plane_fmt[0].bytesperline < bpl) { dev_dbg(dev, "%s: bpl mismatch on stitch_output\n", __func__); return -EINVAL; } - if (fmt->fmt.pix_mp.plane_fmt[0].sizeimage < size) { + if (mp->plane_fmt[0].sizeimage < size) { dev_dbg(dev, "%s: size mismatch on stitch_output\n", __func__); return -EINVAL; @@ -746,8 +854,15 @@ static int pisp_be_validate_config(struct pispbe_dev *pispbe, PISP_IMAGE_FORMAT_WALLPAPER_ROLL) continue; /* TODO: Size checks for wallpaper formats */ - fmt = &pispbe->node[OUTPUT0_NODE + j].format; - for (unsigned int i = 0; i < fmt->fmt.pix_mp.num_planes; i++) { + context = pispbe_get_dev_context(mdev_context, + &pispbe->node[OUTPUT0_NODE + j].vfd); + if (!context) + return -EINVAL; + + mp = &context->format.fmt.pix_mp; + pispbe_put_dev_context(context); + + for (unsigned int i = 0; i < mp->num_planes; i++) { bpl = !i ? config->config.output_format[j].image.stride : config->config.output_format[j].image.stride2; size = bpl * config->config.output_format[j].image.height; @@ -756,13 +871,15 @@ static int pisp_be_validate_config(struct pispbe_dev *pispbe, PISP_IMAGE_FORMAT_SAMPLING_420) size >>= 1; - if (fmt->fmt.pix_mp.plane_fmt[i].bytesperline < bpl) { + if (mp->plane_fmt[i].bytesperline < bpl) { dev_dbg(dev, "%s: bpl mismatch on output %d\n", __func__, j); + dev_dbg(dev, "Expected %u, got %u\n", + bpl, mp->plane_fmt[i].bytesperline); return -EINVAL; } - if (fmt->fmt.pix_mp.plane_fmt[i].sizeimage < size) { + if (mp->plane_fmt[i].sizeimage < size) { dev_dbg(dev, "%s: size mismatch on output\n", __func__); return -EINVAL; @@ -777,10 +894,12 @@ static int pispbe_node_queue_setup(struct vb2_queue *q, unsigned int *nbuffers, unsigned int *nplanes, unsigned int sizes[], struct device *alloc_devs[]) { + struct pispbe_entity_context *context = pispbe_entity_context_from_queue(q); + struct v4l2_format *format = &context->format; struct pispbe_node *node = vb2_get_drv_priv(q); struct pispbe_dev *pispbe = node->pispbe; unsigned int num_planes = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.num_planes : 1; + format->fmt.pix_mp.num_planes : 1; if (*nplanes) { if (*nplanes != num_planes) @@ -788,8 +907,8 @@ static int pispbe_node_queue_setup(struct vb2_queue *q, unsigned int *nbuffers, for (unsigned int i = 0; i < *nplanes; i++) { unsigned int size = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.plane_fmt[i].sizeimage : - node->format.fmt.meta.buffersize; + format->fmt.pix_mp.plane_fmt[i].sizeimage : + format->fmt.meta.buffersize; if (sizes[i] < size) return -EINVAL; @@ -801,8 +920,8 @@ static int pispbe_node_queue_setup(struct vb2_queue *q, unsigned int *nbuffers, *nplanes = num_planes; for (unsigned int i = 0; i < *nplanes; i++) { unsigned int size = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.plane_fmt[i].sizeimage : - node->format.fmt.meta.buffersize; + format->fmt.pix_mp.plane_fmt[i].sizeimage : + format->fmt.meta.buffersize; sizes[i] = size; } @@ -815,15 +934,21 @@ static int pispbe_node_queue_setup(struct vb2_queue *q, unsigned int *nbuffers, static int pispbe_node_buffer_prepare(struct vb2_buffer *vb) { - struct pispbe_node *node = vb2_get_drv_priv(vb->vb2_queue); + struct vb2_queue *q = vb->vb2_queue; + struct pispbe_entity_context *context = + pispbe_entity_context_from_queue(q); + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); + struct v4l2_format *format = &context->format; + struct pispbe_node *node = context->node; struct pispbe_dev *pispbe = node->pispbe; unsigned int num_planes = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.num_planes : 1; + format->fmt.pix_mp.num_planes : 1; for (unsigned int i = 0; i < num_planes; i++) { unsigned long size = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.plane_fmt[i].sizeimage : - node->format.fmt.meta.buffersize; + format->fmt.pix_mp.plane_fmt[i].sizeimage : + format->fmt.meta.buffersize; if (vb2_plane_size(vb, i) < size) { dev_dbg(pispbe->dev, @@ -841,7 +966,7 @@ static int pispbe_node_buffer_prepare(struct vb2_buffer *vb) memcpy(dst, src, sizeof(struct pisp_be_tiles_config)); - return pisp_be_validate_config(pispbe, dst); + return pisp_be_validate_config(pispbe, mdev_context, dst); } return 0; @@ -849,53 +974,67 @@ static int pispbe_node_buffer_prepare(struct vb2_buffer *vb) static void pispbe_node_buffer_queue(struct vb2_buffer *buf) { - struct vb2_v4l2_buffer *vbuf = - container_of(buf, struct vb2_v4l2_buffer, vb2_buf); - struct pispbe_buffer *buffer = - container_of(vbuf, struct pispbe_buffer, vb); - struct pispbe_node *node = vb2_get_drv_priv(buf->vb2_queue); + struct vb2_v4l2_buffer *vbuf = container_of(buf, struct vb2_v4l2_buffer, + vb2_buf); + struct pispbe_buffer *buffer = container_of(vbuf, struct pispbe_buffer, + vb); + struct vb2_queue *q = buf->vb2_queue; + struct pispbe_entity_context *context = + pispbe_entity_context_from_queue(q); + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); + struct pispbe_node *node = context->node; struct pispbe_dev *pispbe = node->pispbe; dev_dbg(pispbe->dev, "%s: for node %s\n", __func__, NODE_NAME(node)); - list_add_tail(&buffer->ready_list, &node->ready_queue); + list_add_tail(&buffer->ready_list, &context->ready_queue); /* * Every time we add a buffer, check if there's now some work for the hw * to do. */ - if (!pispbe_prepare_job(pispbe)) - pispbe_schedule(pispbe, false); + if (!pispbe_prepare_job(pispbe, context)) + pispbe_schedule(pispbe, mdev_context, false); } static int pispbe_node_start_streaming(struct vb2_queue *q, unsigned int count) { - struct pispbe_node *node = vb2_get_drv_priv(q); + struct pispbe_entity_context *context = + pispbe_entity_context_from_queue(q); + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); + struct pispbe_node *node = context->node; struct pispbe_dev *pispbe = node->pispbe; struct pispbe_buffer *buf, *tmp; int ret; + ret = video_device_context_pipeline_start(&context->vdev_context, + &pispbe->pipe); + if (ret) + return ret; + ret = pm_runtime_resume_and_get(pispbe->dev); if (ret < 0) goto err_return_buffers; spin_lock_irq(&pispbe->hw_lock); - node->pispbe->streaming_map |= BIT(node->id); + mdev_context->streaming_map |= BIT(node->id); node->pispbe->sequence = 0; spin_unlock_irq(&pispbe->hw_lock); dev_dbg(pispbe->dev, "%s: for node %s (count %u)\n", __func__, NODE_NAME(node), count); dev_dbg(pispbe->dev, "Nodes streaming now 0x%x\n", - node->pispbe->streaming_map); + mdev_context->streaming_map); /* Maybe we're ready to run. */ - if (!pispbe_prepare_job(pispbe)) - pispbe_schedule(pispbe, false); + if (!pispbe_prepare_job(pispbe, context)) + pispbe_schedule(pispbe, mdev_context, false); return 0; err_return_buffers: - list_for_each_entry_safe(buf, tmp, &node->ready_queue, ready_list) { + list_for_each_entry_safe(buf, tmp, &context->ready_queue, ready_list) { list_del(&buf->ready_list); vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_QUEUED); } @@ -905,7 +1044,11 @@ static int pispbe_node_start_streaming(struct vb2_queue *q, unsigned int count) static void pispbe_node_stop_streaming(struct vb2_queue *q) { - struct pispbe_node *node = vb2_get_drv_priv(q); + struct pispbe_entity_context *context = + pispbe_entity_context_from_queue(q); + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); + struct pispbe_node *node = context->node; struct pispbe_dev *pispbe = node->pispbe; struct pispbe_buffer *buf; @@ -920,7 +1063,7 @@ static void pispbe_node_stop_streaming(struct vb2_queue *q) */ dev_dbg(pispbe->dev, "%s: for node %s\n", __func__, NODE_NAME(node)); do { - buf = list_first_entry_or_null(&node->ready_queue, + buf = list_first_entry_or_null(&context->ready_queue, struct pispbe_buffer, ready_list); if (buf) { @@ -929,13 +1072,13 @@ static void pispbe_node_stop_streaming(struct vb2_queue *q) } } while (buf); - vb2_wait_for_all_buffers(&node->queue); + vb2_wait_for_all_buffers(&context->vdev_context.queue); spin_lock_irq(&pispbe->hw_lock); - pispbe->streaming_map &= ~BIT(node->id); + mdev_context->streaming_map &= ~BIT(node->id); /* Release all jobs once all nodes have stopped streaming. */ - if (pispbe->streaming_map == 0) { + if (mdev_context->streaming_map == 0) { struct pispbe_job_descriptor *job, *temp; list_for_each_entry_safe(job, temp, &pispbe->job_queue, queue) { @@ -948,8 +1091,10 @@ static void pispbe_node_stop_streaming(struct vb2_queue *q) pm_runtime_mark_last_busy(pispbe->dev); pm_runtime_put_autosuspend(pispbe->dev); + video_device_pipeline_stop(context->vdev_context.vdev); + dev_dbg(pispbe->dev, "Nodes streaming now 0x%x\n", - pispbe->streaming_map); + mdev_context->streaming_map); } static const struct vb2_ops pispbe_node_queue_ops = { @@ -990,6 +1135,8 @@ static int pispbe_node_g_fmt_vid_cap(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_entity_context *context = + pispbe_entity_context_from_file(file, &node->vfd); if (!NODE_IS_CAPTURE(node) || NODE_IS_META(node)) { dev_dbg(pispbe->dev, @@ -998,7 +1145,7 @@ static int pispbe_node_g_fmt_vid_cap(struct file *file, void *priv, return -EINVAL; } - *f = node->format; + *f = context->format; dev_dbg(pispbe->dev, "Get capture format for node %s\n", NODE_NAME(node)); @@ -1010,6 +1157,8 @@ static int pispbe_node_g_fmt_vid_out(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_entity_context *context = + pispbe_entity_context_from_file(file, &node->vfd); if (NODE_IS_CAPTURE(node) || NODE_IS_META(node)) { dev_dbg(pispbe->dev, @@ -1018,7 +1167,7 @@ static int pispbe_node_g_fmt_vid_out(struct file *file, void *priv, return -EINVAL; } - *f = node->format; + *f = context->format; dev_dbg(pispbe->dev, "Get output format for node %s\n", NODE_NAME(node)); @@ -1030,6 +1179,8 @@ static int pispbe_node_g_fmt_meta_out(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_entity_context *context = + pispbe_entity_context_from_file(file, &node->vfd); if (!NODE_IS_META(node) || NODE_IS_CAPTURE(node)) { dev_dbg(pispbe->dev, @@ -1038,7 +1189,7 @@ static int pispbe_node_g_fmt_meta_out(struct file *file, void *priv, return -EINVAL; } - *f = node->format; + *f = context->format; dev_dbg(pispbe->dev, "Get output format for meta node %s\n", NODE_NAME(node)); @@ -1206,17 +1357,19 @@ static int pispbe_node_s_fmt_vid_cap(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_entity_context *context = + pispbe_entity_context_from_file(file, &node->vfd); int ret; ret = pispbe_node_try_fmt_vid_cap(file, priv, f); if (ret < 0) return ret; - if (vb2_is_busy(&node->queue)) + if (vb2_is_busy(&context->vdev_context.queue)) return -EBUSY; - node->format = *f; - node->pisp_format = pispbe_find_fmt(f->fmt.pix_mp.pixelformat); + context->format = *f; + context->pisp_format = pispbe_find_fmt(f->fmt.pix_mp.pixelformat); dev_dbg(pispbe->dev, "Set capture format for node %s to %p4cc\n", NODE_NAME(node), &f->fmt.pix_mp.pixelformat); @@ -1229,17 +1382,19 @@ static int pispbe_node_s_fmt_vid_out(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_entity_context *context = + pispbe_entity_context_from_file(file, &node->vfd); int ret; ret = pispbe_node_try_fmt_vid_out(file, priv, f); if (ret < 0) return ret; - if (vb2_is_busy(&node->queue)) + if (vb2_is_busy(&context->vdev_context.queue)) return -EBUSY; - node->format = *f; - node->pisp_format = pispbe_find_fmt(f->fmt.pix_mp.pixelformat); + context->format = *f; + context->pisp_format = pispbe_find_fmt(f->fmt.pix_mp.pixelformat); dev_dbg(pispbe->dev, "Set output format for node %s to %p4cc\n", NODE_NAME(node), &f->fmt.pix_mp.pixelformat); @@ -1252,17 +1407,19 @@ static int pispbe_node_s_fmt_meta_out(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_entity_context *context = + pispbe_entity_context_from_file(file, &node->vfd); int ret; ret = pispbe_node_try_fmt_meta_out(file, priv, f); if (ret < 0) return ret; - if (vb2_is_busy(&node->queue)) + if (vb2_is_busy(&context->vdev_context.queue)) return -EBUSY; - node->format = *f; - node->pisp_format = &meta_out_supported_formats[0]; + context->format = *f; + context->pisp_format = &meta_out_supported_formats[0]; dev_dbg(pispbe->dev, "Set output format for meta node %s to %p4cc\n", NODE_NAME(node), &f->fmt.meta.dataformat); @@ -1274,8 +1431,10 @@ static int pispbe_node_enum_fmt(struct file *file, void *priv, struct v4l2_fmtdesc *f) { struct pispbe_node *node = video_drvdata(file); + struct pispbe_entity_context *context = + pispbe_entity_context_from_file(file, &node->vfd); - if (f->type != node->queue.type) + if (f->type != context->vdev_context.queue.type) return -EINVAL; if (NODE_IS_META(node)) { @@ -1349,24 +1508,16 @@ static const struct v4l2_ioctl_ops pispbe_node_ioctl_ops = { .vidioc_streamoff = vb2_ioctl_streamoff, }; -static const struct video_device pispbe_videodev = { - .name = PISPBE_NAME, - .vfl_dir = VFL_DIR_M2M, /* gets overwritten */ - .fops = &pispbe_fops, - .ioctl_ops = &pispbe_node_ioctl_ops, - .minor = -1, - .release = video_device_release_empty, -}; - -static void pispbe_node_def_fmt(struct pispbe_node *node) +static void pispbe_node_def_fmt(struct pispbe_node *node, + struct pispbe_entity_context *context) { + struct v4l2_format *format = &context->format; + if (NODE_IS_META(node) && NODE_IS_OUTPUT(node)) { /* Config node */ - struct v4l2_format *f = &node->format; - - f->fmt.meta.dataformat = V4L2_META_FMT_RPI_BE_CFG; - f->fmt.meta.buffersize = sizeof(struct pisp_be_tiles_config); - f->type = node->buf_type; + format->fmt.meta.dataformat = V4L2_META_FMT_RPI_BE_CFG; + format->fmt.meta.buffersize = sizeof(struct pisp_be_tiles_config); + format->type = node->buf_type; } else { struct v4l2_format f = { .fmt.pix_mp.pixelformat = V4L2_PIX_FMT_YUV420, @@ -1375,36 +1526,39 @@ static void pispbe_node_def_fmt(struct pispbe_node *node) .type = node->buf_type, }; pispbe_try_format(&f, node); - node->format = f; + *format = f; } - node->pisp_format = pispbe_find_fmt(node->format.fmt.pix_mp.pixelformat); + context->pisp_format = pispbe_find_fmt(format->fmt.pix_mp.pixelformat); } -/* - * Initialise a struct pispbe_node and register it as /dev/video - * to represent one of the PiSP Back End's input or output streams. - */ -static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) +static int pispbe_alloc_entity_context(struct media_entity *entity, + struct media_entity_context **ctx) { - bool output = NODE_DESC_IS_OUTPUT(&node_desc[id]); - struct pispbe_node *node = &pispbe->node[id]; - struct media_entity *entity = &node->vfd.entity; - struct video_device *vdev = &node->vfd; - struct vb2_queue *q = &node->queue; + struct video_device *vdev = container_of(entity, struct video_device, + entity); + struct pispbe_node *node = pispbe_node_from_vdev(vdev); + struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_entity_context *context; + struct vb2_queue *q; int ret; - node->id = id; - node->pispbe = pispbe; - node->buf_type = node_desc[id].buf_type; + *ctx = kzalloc(sizeof(*context), GFP_KERNEL); + if (!*ctx) + return -ENOMEM; + context = (struct pispbe_entity_context *)*ctx; - mutex_init(&node->node_lock); - mutex_init(&node->queue_lock); - INIT_LIST_HEAD(&node->ready_queue); + video_device_init_context(vdev, &context->vdev_context); + + context->pispbe = pispbe; + context->node = node; + + INIT_LIST_HEAD(&context->ready_queue); - node->format.type = node->buf_type; - pispbe_node_def_fmt(node); + context->format.type = node->buf_type; + pispbe_node_def_fmt(node, context); + q = &context->vdev_context.queue; q->type = node->buf_type; q->io_modes = VB2_MMAP | VB2_DMABUF; q->mem_ops = &vb2_dma_contig_memops; @@ -1414,41 +1568,92 @@ static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; q->dev = pispbe->dev; /* get V4L2 to handle node->queue locking */ - q->lock = &node->queue_lock; + q->lock = &context->vdev_context.queue_lock; ret = vb2_queue_init(q); if (ret < 0) { dev_err(pispbe->dev, "vb2_queue_init failed\n"); - goto err_mutex_destroy; + goto err_cleanup; } + return 0; + +err_cleanup: + kfree(*ctx); + return ret; +} + +static void pispbe_free_entity_context(struct media_entity_context *ctx) +{ + struct pispbe_entity_context *context = + (struct pispbe_entity_context *)ctx; + + list_del_init(&context->ready_queue); + video_device_cleanup_context(&context->vdev_context); + kfree(context); +} + +static const struct media_entity_operations pispbe_media_entity_ops = { + .alloc_context = pispbe_alloc_entity_context, + .destroy_context = pispbe_free_entity_context, +}; + +static const struct video_device pispbe_videodev = { + .name = PISPBE_NAME, + .vfl_dir = VFL_DIR_M2M, /* gets overwritten */ + .fops = &pispbe_fops, + .ioctl_ops = &pispbe_node_ioctl_ops, + .minor = -1, + .release = video_device_release_empty, +}; + +/* + * Initialise a struct pispbe_node and register it as /dev/video + * to represent one of the PiSP Back End's input or output streams. + */ +static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) +{ + bool output = NODE_DESC_IS_OUTPUT(&node_desc[id]); + struct pispbe_node *node = &pispbe->node[id]; + struct media_entity *entity = &node->vfd.entity; + struct video_device *vdev = &node->vfd; + int ret; + + node->id = id; + node->pispbe = pispbe; + node->buf_type = node_desc[id].buf_type; + + mutex_init(&node->node_lock); + *vdev = pispbe_videodev; /* default initialization */ strscpy(vdev->name, node_desc[id].ent_name, sizeof(vdev->name)); vdev->v4l2_dev = &pispbe->v4l2_dev; vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX; /* get V4L2 to serialise our ioctls */ vdev->lock = &node->node_lock; - vdev->queue = &node->queue; vdev->device_caps = V4L2_CAP_STREAMING | node_desc[id].caps; + vdev->entity.ops = &pispbe_media_entity_ops; + node->pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK; ret = media_entity_pads_init(entity, 1, &node->pad); if (ret) { dev_err(pispbe->dev, "Failed to register media pads for %s device node\n", NODE_NAME(node)); - goto err_unregister_queue; + goto err_mutex_destroy; } + video_set_drvdata(vdev, node); + ret = video_register_device(vdev, VFL_TYPE_VIDEO, PISPBE_VIDEO_NODE_OFFSET); if (ret) { dev_err(pispbe->dev, "Failed to register video %s device node\n", NODE_NAME(node)); - goto err_unregister_queue; + goto err_mutex_destroy; } - video_set_drvdata(vdev, node); if (output) ret = media_create_pad_link(entity, 0, &pispbe->sd.entity, @@ -1468,11 +1673,8 @@ static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) err_unregister_video_dev: video_unregister_device(&node->vfd); -err_unregister_queue: - vb2_queue_release(&node->queue); err_mutex_destroy: mutex_destroy(&node->node_lock); - mutex_destroy(&node->queue_lock); return ret; } @@ -1484,6 +1686,38 @@ static const struct v4l2_subdev_ops pispbe_sd_ops = { .pad = &pispbe_pad_ops, }; +static int pispbe_media_alloc_context(struct media_device *mdev, + struct media_device_context **ctx) +{ + struct pispbe_media_context *context; + + *ctx = kzalloc(sizeof(*context), GFP_KERNEL); + if (!*ctx) + return -ENOMEM; + context = (struct pispbe_media_context *)*ctx; + + media_device_init_context(mdev, &context->mdev_context); + + context->streaming_map = 0; + + return 0; +} + +static void pispbe_media_destroy_context(struct media_device_context *ctx) +{ + struct pispbe_media_context *context = + (struct pispbe_media_context *)ctx; + + context->streaming_map = 0; + media_device_cleanup_context(&context->mdev_context); + kfree(context); +} + +static const struct media_device_ops pispbe_media_device_ops = { + .alloc_context = pispbe_media_alloc_context, + .destroy_context = pispbe_media_destroy_context, +}; + static int pispbe_init_subdev(struct pispbe_dev *pispbe) { struct v4l2_subdev *sd = &pispbe->sd; @@ -1527,6 +1761,7 @@ static int pispbe_init_devices(struct pispbe_dev *pispbe) mdev = &pispbe->mdev; mdev->hw_revision = pispbe->hw_version; mdev->dev = pispbe->dev; + mdev->ops = &pispbe_media_device_ops; strscpy(mdev->model, PISPBE_NAME, sizeof(mdev->model)); media_device_init(mdev); @@ -1570,10 +1805,8 @@ static int pispbe_init_devices(struct pispbe_dev *pispbe) err_unregister_mdev: media_device_unregister(mdev); err_unregister_nodes: - while (num_regist-- > 0) { + while (num_regist-- > 0) video_unregister_device(&pispbe->node[num_regist].vfd); - vb2_queue_release(&pispbe->node[num_regist].queue); - } v4l2_device_unregister_subdev(&pispbe->sd); media_entity_cleanup(&pispbe->sd.entity); err_unregister_v4l2: @@ -1601,9 +1834,7 @@ static void pispbe_destroy_devices(struct pispbe_dev *pispbe) for (int i = PISPBE_NUM_NODES - 1; i >= 0; i--) { video_unregister_device(&pispbe->node[i].vfd); - vb2_queue_release(&pispbe->node[i].queue); mutex_destroy(&pispbe->node[i].node_lock); - mutex_destroy(&pispbe->node[i].queue_lock); } media_device_cleanup(&pispbe->mdev);