From patchwork Fri Nov 26 00:16:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 12639991 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C4ADC433FE for ; Fri, 26 Nov 2021 00:18:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351757AbhKZAVj (ORCPT ); Thu, 25 Nov 2021 19:21:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237036AbhKZATj (ORCPT ); Thu, 25 Nov 2021 19:19:39 -0500 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F0D35C061746 for ; Thu, 25 Nov 2021 16:16:20 -0800 (PST) Received: by mail-wm1-x335.google.com with SMTP id p18so6788454wmq.5 for ; Thu, 25 Nov 2021 16:16:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=i0Izz8KH3j327RlKK4TwK19uTSmxVKmRVDkq2hLcddk=; b=ZPSP60X5v4XZgNoeDenlTFOqIk2FZwsr2EKfGw035cMwDVoQoWkJZL+UIA/75wkruZ 09gGEm0SkdikuBASxGcfW3VLRslcBgkc8IBc4lErQVpjZFUU6tGI+sfkZCsqf3j/Vsmv 47aVIZX8753gISMo3mXDjMxuxU60cjSLA1ReUfRAEp/IvVwDNcjueumZG+XFASgNYJEe sFhy+7RtZr/z/B2uJKRN6xtKrar294jF5l4cm9Wh/rkSt0T3rc1FPBdbUp4dDNJNF7EW lKEMLhjNsILxh6PL3lCu9rVMjE09ruMZcFbSMWG0W/a4pIKK6Y5QLgprgzSO30X6EES8 b9QA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=i0Izz8KH3j327RlKK4TwK19uTSmxVKmRVDkq2hLcddk=; b=XgJY2H9Prq619bgzZ+riUIHMklzg4AOrka+xFGjuPvh7mQfFmGvzvcKEdCoEblXcIR NETKjr+vEz2zEkhnZ80B2bx1n4SlvogptF/fgciIp172TJBS23KnHRf6+0RnasajPdo7 8JZRK8bEgj7T8+dv2K2RhdM/EOQGumx2ypxGK6U4kFn1QwHkNjXiMKTIXtMLIUVM48bf QWZWjyE5sShclIhe2d2BKjH+g6B8S8tq8TUttY+lvmZE48GhMCYvWdjd9VkCDiTwsO1+ oNEWrr1sMIVZ08FWJ9A8E3JsxmOT0Nq4HORc4GXOPDu9N3InvOaijRiRHxFj1pKXdEBo DAZQ== X-Gm-Message-State: AOAM5327aizhy0O/DMIM6aOnTxcUCB9zF3mbc1epxar/aJAFCbe/04jc aqmokyrubEGlZjgc/P/iXlj6vW22Zt4= X-Google-Smtp-Source: ABdhPJx5aXsJr4gK3HseSFEI8dNiA+HezXXYWrq1rlhVqYUKaTru0n8qBM0/JBb+Eo6VJu32w2+TcQ== X-Received: by 2002:a1c:98ca:: with SMTP id a193mr11875097wme.162.1637885779202; Thu, 25 Nov 2021 16:16:19 -0800 (PST) Received: from localhost.localdomain (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net. [86.13.91.161]) by smtp.gmail.com with ESMTPSA id p14sm8693728wms.29.2021.11.25.16.16.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Nov 2021 16:16:18 -0800 (PST) From: Daniel Scally To: linux-media@vger.kernel.org Cc: sakari.ailus@linux.intel.com, laurent.pinchart@ideasonboard.com, hanlinchen@chromium.org, tfiga@chromium.org, hdegoede@redhat.com, kieran.bingham@ideasonboard.com Subject: [RFC PATCH 1/2] media: entity: Add support for ancillary links Date: Fri, 26 Nov 2021 00:16:02 +0000 Message-Id: <20211126001603.41148-2-djrscally@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211126001603.41148-1-djrscally@gmail.com> References: <20211126001603.41148-1-djrscally@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org To describe in the kernel the connection between devices and their supporting peripherals (for example, a camera sensor and the vcm driving the focusing lens for it), add a new type of media link which connects two instances of struct media_entity. Signed-off-by: Daniel Scally --- I was tempted to 'fix' the spaces between # and define in include/uapi/linux/media.h but eventually decided they were probably deliberate but if that's not true I'd fix those too. drivers/media/mc/mc-entity.c | 30 ++++++++++++++++++++++++++++++ include/media/media-entity.h | 30 ++++++++++++++++++++++++++++++ include/uapi/linux/media.h | 1 + 3 files changed, 61 insertions(+) diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index f40f41977142..9c18b974e117 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -1032,3 +1032,33 @@ void media_remove_intf_links(struct media_interface *intf) mutex_unlock(&mdev->graph_mutex); } EXPORT_SYMBOL_GPL(media_remove_intf_links); + +struct media_link *media_create_ancillary_link(struct media_entity *primary, + struct media_entity *ancillary, + u32 flags) +{ + struct media_link *link; + + link = media_add_link(&primary->links); + if (!link) + return ERR_PTR(-ENOMEM); + + link->primary = primary; + link->ancillary = ancillary; + link->flags = flags | MEDIA_LNK_FL_ANCILLARY_LINK; + + /* Initialize graph object embedded at the new link */ + media_gobj_create(primary->graph_obj.mdev, MEDIA_GRAPH_LINK, + &link->graph_obj); + + return link; +} +EXPORT_SYMBOL_GPL(media_create_ancillary_link); + +void media_remove_ancillary_link(struct media_link *link) +{ + list_del(&link->list); + media_gobj_destroy(&link->graph_obj); + kfree(link); +} +EXPORT_SYMBOL_GPL(media_remove_ancillary_link); diff --git a/include/media/media-entity.h b/include/media/media-entity.h index fea489f03d57..400b864857ee 100644 --- a/include/media/media-entity.h +++ b/include/media/media-entity.h @@ -119,12 +119,16 @@ struct media_pipeline { * a pad. In that case, it represents the source pad. * @intf: Part of a union. Used only if the first object (gobj0) is * an interface. + * @primary: Part of a union. Used only if the first object (gobj0) is + * an entity and the link type is MEDIA_LNK_FL_ANCILLARY_LINK. * @gobj1: Part of a union. Used to get the pointer for the second * graph_object of the link. * @sink: Part of a union. Used only if the second object (gobj1) is * a pad. In that case, it represents the sink pad. * @entity: Part of a union. Used only if the second object (gobj1) is * an entity. + * @ancillary: Part of a union. Used only if the second object (gobj1) is + * an entity and the link type is MEDIA_LNK_FL_ANCILLARY_LINK. * @reverse: Pointer to the link for the reverse direction of a pad to pad * link. * @flags: Link flags, as defined in uapi/media.h (MEDIA_LNK_FL_*) @@ -137,11 +141,13 @@ struct media_link { struct media_gobj *gobj0; struct media_pad *source; struct media_interface *intf; + struct media_entity *primary; }; union { struct media_gobj *gobj1; struct media_pad *sink; struct media_entity *entity; + struct media_entity *ancillary; }; struct media_link *reverse; unsigned long flags; @@ -1104,6 +1110,30 @@ void media_remove_intf_links(struct media_interface *intf); * it will issue a call to @operation\(@entity, @args\). */ +/** + * media_create_ancillary_link() - creates a link between two entities + * + * @primary: pointer to the primary %media_entity + * @ancillary: pointer to the ancillary %media_entity + * @flags: Link flags, as defined in + * :ref:`include/uapi/linux/media.h ` + * ( seek for ``MEDIA_LNK_FL_*``) + * + * + * Valid values for flags: + * + * %MEDIA_LNK_FL_ENABLED + * Indicates that the two entities are connected pieces of hardware that form + * a single logical unit. + * + * A typical example is a camera lens being linked to the sensor that it is + * supporting. + */ +struct media_link * +media_create_ancillary_link(struct media_entity *primary, + struct media_entity *ancillary, + u32 flags); + #define media_entity_call(entity, operation, args...) \ (((entity)->ops && (entity)->ops->operation) ? \ (entity)->ops->operation((entity) , ##args) : -ENOIOCTLCMD) diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h index 200fa8462b90..afbae7213d35 100644 --- a/include/uapi/linux/media.h +++ b/include/uapi/linux/media.h @@ -226,6 +226,7 @@ struct media_pad_desc { #define MEDIA_LNK_FL_LINK_TYPE (0xf << 28) # define MEDIA_LNK_FL_DATA_LINK (0 << 28) # define MEDIA_LNK_FL_INTERFACE_LINK (1 << 28) +# define MEDIA_LNK_FL_ANCILLARY_LINK (2 << 28) struct media_link_desc { struct media_pad_desc source; From patchwork Fri Nov 26 00:16:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 12639993 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C56ECC4332F for ; Fri, 26 Nov 2021 00:18:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351768AbhKZAVj (ORCPT ); Thu, 25 Nov 2021 19:21:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351860AbhKZATj (ORCPT ); Thu, 25 Nov 2021 19:19:39 -0500 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B2BB1C061757 for ; Thu, 25 Nov 2021 16:16:21 -0800 (PST) Received: by mail-wm1-x32f.google.com with SMTP id p27-20020a05600c1d9b00b0033bf8532855so5698087wms.3 for ; Thu, 25 Nov 2021 16:16:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iFC0g7sHT2Cln7UNIyKY10hlNRb0uj3DbZGbBRfYld4=; b=jXflj76qY0SSY7oaxnvYd6crN/7i764AF2iW+UNOOtnxptmlLMyTIjuh6x/uFWuuPQ PSyX9XIf4hRPVYfrI3ZIj8/4p4fgTEFGzupRSe3V9MGsndnlkLD57/V22xon8U/WqB7u JteuZcO0AKuFksT6eGp8bAew7sL2CyLCTW5S7XKW9dQKjxxxt11ecivibJr0MvRyZVQk FYymY52eRwwvnFmqqjdDW6yncW4r4LXsuTSSdztXjdLzFVDyFFO7hRm2UlYLerEmy74u Ot9vZuR8UMeABevbdAPZw2jiBxcGeF+EVoqU66JGHs9EjIa9z0GMxBXNxRSH5aDn8ve0 fmeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iFC0g7sHT2Cln7UNIyKY10hlNRb0uj3DbZGbBRfYld4=; b=ux6d4T9jj8DaEh+vEovnLSW0Wl+UeAa66KNEqjhfQeGRsvCFh86aIlqtulvTtMv8Tr +FZkMu8sXMUSh+YvHTnz4nLXxUTW+RHKhkRFNJLSoRHTZdLFcjWC833e+/7IpdSEKl8O PlX6PY4SXFLBXeR33kIuNQFaOPQX86UKnFJt7UTnjEkVn7K/D/DrABqcsVstJxzMNdv1 lWd7ab0xz64AVVk4NSSzM/919b4+MzmA1sMKHuHAKV01WKP0+SY0zrql6vjG58saDS/2 t+HHq6xqHGlfoFVxMdjtZFeM4+WAdX9KGzZlQIuqUbI6TBl7ym26o6EPzIBX1pTAdKzu rEHw== X-Gm-Message-State: AOAM532+O5XjdiCjGev3PxMmC7rzzLx4TIizrAkmnskIR1dNg6ZAq+IK iM5Bqa7HAsvALWf4SNoDyq2AzlNAOsI= X-Google-Smtp-Source: ABdhPJxCG8FHmkAPNIEtPSViR1Py5jI8aSCEiHyqEb0euvXs1BfYKmHRjvVuEoJHLuQVv5Co8TfgIA== X-Received: by 2002:a05:600c:214f:: with SMTP id v15mr11688716wml.194.1637885780242; Thu, 25 Nov 2021 16:16:20 -0800 (PST) Received: from localhost.localdomain (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net. [86.13.91.161]) by smtp.gmail.com with ESMTPSA id p14sm8693728wms.29.2021.11.25.16.16.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Nov 2021 16:16:19 -0800 (PST) From: Daniel Scally To: linux-media@vger.kernel.org Cc: sakari.ailus@linux.intel.com, laurent.pinchart@ideasonboard.com, hanlinchen@chromium.org, tfiga@chromium.org, hdegoede@redhat.com, kieran.bingham@ideasonboard.com Subject: [RFC PATCH 2/2] media: v4l2-async: Create links during v4l2_async_match_notify() Date: Fri, 26 Nov 2021 00:16:03 +0000 Message-Id: <20211126001603.41148-3-djrscally@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211126001603.41148-1-djrscally@gmail.com> References: <20211126001603.41148-1-djrscally@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Upon an async fwnode match, there's some typical behaviour that the notifier and matching subdev will want to do. For example, a notifier representing a sensor matching to an async subdev representing its VCM will want to create an ancillary link to expose that relationship to userspace. To avoid lots of code in individual drivers, try to build these links within v4l2 core. Signed-off-by: Daniel Scally --- This is probably more complex than it needs to be at present, but the intent would be to expand it with __v4l2_async_create_data_link() to be called where the notifier entity's function was MEDIA_ENT_F_VID_IF_BRIDGE for example. drivers/media/v4l2-core/v4l2-async.c | 51 ++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-async.c b/drivers/media/v4l2-core/v4l2-async.c index 0404267f1ae4..6575b1cbe95f 100644 --- a/drivers/media/v4l2-core/v4l2-async.c +++ b/drivers/media/v4l2-core/v4l2-async.c @@ -275,6 +275,45 @@ v4l2_async_nf_try_complete(struct v4l2_async_notifier *notifier) static int v4l2_async_nf_try_all_subdevs(struct v4l2_async_notifier *notifier); +static int +__v4l2_async_create_ancillary_link(struct v4l2_async_notifier *notifier, + struct v4l2_subdev *sd) +{ + struct media_link *link; + + if (sd->entity.function != MEDIA_ENT_F_LENS && + sd->entity.function != MEDIA_ENT_F_FLASH) + return -EINVAL; + + link = media_create_ancillary_link(¬ifier->sd->entity, &sd->entity, + MEDIA_LNK_FL_ENABLED | + MEDIA_LNK_FL_IMMUTABLE); + + return IS_ERR(link) ? PTR_ERR(link) : 0; +} + +/* + * Setup links on behalf of the notifier and subdev, where it's obvious what + * should be done. At the moment, we only support cases where the notifier + * is a sensor and the subdev is a lens. + * + * TODO: Setup pad links if the notifier's function is MEDIA_ENT_F_VID_IF_BRIDGE + * and the subdev's is MEDIA_ENT_F_CAM_SENSOR + */ +static int v4l2_async_try_create_links(struct v4l2_async_notifier *notifier, + struct v4l2_subdev *sd) +{ + if (!notifier->sd) + return 0; + + switch (notifier->sd->entity.function) { + case MEDIA_ENT_F_CAM_SENSOR: + return __v4l2_async_create_ancillary_link(notifier, sd); + default: + return 0; + } +} + static int v4l2_async_match_notify(struct v4l2_async_notifier *notifier, struct v4l2_device *v4l2_dev, struct v4l2_subdev *sd, @@ -293,6 +332,18 @@ static int v4l2_async_match_notify(struct v4l2_async_notifier *notifier, return ret; } + /* + * Depending of the function of the entities involved, we may want to + * create links between them (for example between a sensor and its lens + * or between a sensor's source pad and the connected device's sink + * pad) + */ + ret = v4l2_async_try_create_links(notifier, sd); + if (ret) { + v4l2_device_unregister_subdev(sd); + return ret; + } + /* Remove from the waiting list */ list_del(&asd->list); sd->asd = asd;