From patchwork Thu Nov 25 02:28:10 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 355822 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter1.kernel.org (8.14.4/8.14.3) with ESMTP id oAP2cCkU026076 for ; Thu, 25 Nov 2010 02:38:12 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756672Ab0KYCiL (ORCPT ); Wed, 24 Nov 2010 21:38:11 -0500 Received: from perceval.ideasonboard.com ([95.142.166.194]:54586 "EHLO perceval.ideasonboard.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756194Ab0KYCh4 (ORCPT ); Wed, 24 Nov 2010 21:37:56 -0500 Received: from localhost.localdomain (unknown [91.178.68.191]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D344A35CB8; Thu, 25 Nov 2010 02:28:14 +0000 (UTC) From: Laurent Pinchart To: linux-media@vger.kernel.org, linux-omap@vger.kernel.org, linux-kernel@vger.kernel.org Cc: sakari.ailus@maxwell.research.nokia.com, broonie@opensource.wolfsonmicro.com, lennart@poettering.net Subject: [RFC/PATCH v6 03/12] media: Entities, pads and links Date: Thu, 25 Nov 2010 03:28:10 +0100 Message-Id: <1290652099-15102-4-git-send-email-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 1.7.2.2 In-Reply-To: <1290652099-15102-1-git-send-email-laurent.pinchart@ideasonboard.com> References: <1290652099-15102-1-git-send-email-laurent.pinchart@ideasonboard.com> Sender: linux-omap-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-omap@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.3 (demeter1.kernel.org [140.211.167.41]); Thu, 25 Nov 2010 02:38:13 +0000 (UTC) diff --git a/Documentation/DocBook/v4l/media-controller.xml b/Documentation/DocBook/v4l/media-controller.xml index 253ddb4..f89228d 100644 --- a/Documentation/DocBook/v4l/media-controller.xml +++ b/Documentation/DocBook/v4l/media-controller.xml @@ -53,4 +53,24 @@ implementing policies that belong to userspace. The media controller API aims at solving those problems. + +
+ Media device model + Discovering a device internal topology, and configuring it at runtime, + is one of the goals of the media controller API. To achieve this, hardware + devices are modelled as an oriented graph of building blocks called entities + connected through pads. + An entity is a basic media hardware or software building block. It can + correspond to a large variety of logical blocks such as physical hardware + devices (CMOS sensor for instance), logical hardware devices (a building + block in a System-on-Chip image processing pipeline), DMA channels or + physical connectors. + A pad is a connection endpoint through which an entity can interact + with other entities. Data (not restricted to video) produced by an entity + flows from the entity's output to one or more entity inputs. Pads should not + be confused with physical pins at chip boundaries. + A link is a point-to-point oriented connection between two pads, + either on the same entity or on different entities. Data flows from a source + pad to a sink pad. +
diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt index 84fa43a..0332162 100644 --- a/Documentation/media-framework.txt +++ b/Documentation/media-framework.txt @@ -13,6 +13,30 @@ Documentation/DocBook/v4l/media-controller.xml. This document will focus on the kernel-side implementation of the media framework. +Abstract media device model +--------------------------- + +Discovering a device internal topology, and configuring it at runtime, is one +of the goals of the media framework. To achieve this, hardware devices are +modeled as an oriented graph of building blocks called entities connected +through pads. + +An entity is a basic media hardware building block. It can correspond to +a large variety of logical blocks such as physical hardware devices +(CMOS sensor for instance), logical hardware devices (a building block +in a System-on-Chip image processing pipeline), DMA channels or physical +connectors. + +A pad is a connection endpoint through which an entity can interact with +other entities. Data (not restricted to video) produced by an entity +flows from the entity's output to one or more entity inputs. Pads should +not be confused with physical pins at chip boundaries. + +A link is a point-to-point oriented connection between two pads, either +on the same entity or on different entities. Data flows from a source +pad to a sink pad. + + Media device ------------ @@ -66,3 +90,128 @@ Drivers unregister media device instances by calling Unregistering a media device that hasn't been registered is *NOT* safe. + +Entities, pads and links +------------------------ + +- Entities + +Entities are represented by a struct media_entity instance, defined in +include/media/media-entity.h. The structure is usually embedded into a +higher-level structure, such as a v4l2_subdev or video_device instance, +although drivers can allocate entities directly. + +Drivers initialize entities by calling + + media_entity_init(struct media_entity *entity, u16 num_pads, + struct media_pad *pads, u16 extra_links); + +The media_entity name, type, flags, revision and group_id fields can be +initialized before or after calling media_entity_init. Entities embedded in +higher-level standard structures can have some of those fields set by the +higher-level framework. + +As the number of pads is known in advance, the pads array is not allocated +dynamically but is managed by the entity driver. Most drivers will embed the +pads array in a driver-specific structure, avoiding dynamic allocation. + +Drivers must set the direction of every pad in the pads array before calling +media_entity_init. The function will initialize the other pads fields. + +Unlike the number of pads, the total number of links isn't always known in +advance by the entity driver. As an initial estimate, media_entity_init +pre-allocates a number of links equal to the number of pads plus an optional +number of extra links. The links array will be reallocated if it grows beyond +the initial estimate. + +Drivers register entities with a media device by calling + + media_device_register_entity(struct media_device *mdev, + struct media_entity *entity); + +When registered the entity is assigned an ID. Entity IDs are positive integers +and are guaranteed to be unique in the context of the media device. The +framework doesn't guarantee that IDs will always be continuous. + +Drivers unregister entities by calling + + media_device_unregister_entity(struct media_entity *entity); + +Unregistering an entity will not change the IDs of the other entities, and the +ID will never be reused for a newly registered entity. + +When a media device is unregistered, all its entities are unregistered +automatically. No manual entities unregistration is then required. + +Drivers free resources associated with an entity by calling + + media_entity_cleanup(struct media_entity *entity); + +This function must be called during the cleanup phase after unregistering the +entity. Note that the media_entity instance itself must be freed explicitly by +the driver if required. + +Entities have flags that describe the entity capabilities and state. + + MEDIA_ENTITY_FLAG_DEFAULT indicates the default entity for a given + type. This can be used to report the default audio and video devices + or the default camera sensor. + +Logical entity groups can be defined by setting the group ID of all member +entities to the same non-zero value. An entity group serves no purpose in the +kernel, but is reported to userspace during entities enumeration. The group_id +field belongs to the media device driver and must not by touched by entity +drivers. + +Media device drivers should define groups if several entities are logically +bound together. Example usages include reporting + + - ALSA, VBI and video nodes that carry the same media stream + - lens and flash controllers associated with a sensor + +- Pads + +Pads are represented by a struct media_pad instance, defined in +include/media/media-entity.h. Each entity stores its pads in a pads array +managed by the entity driver. Drivers usually embed the array in a +driver-specific structure. + +Pads are identified by their entity and their 0-based index in the pads array. +Both information are stored in the media_pad structure, making the media_pad +pointer the canonical way to store and pass link references. + +Pads have flags that describe the pad capabilities and state. + + MEDIA_PAD_FLAG_INPUT indicates that the pad supports sinking data. + MEDIA_PAD_FLAG_OUTPUT indicates that the pad supports sourcing data. + +One and only one of MEDIA_PAD_FLAG_INPUT and MEDIA_PAD_FLAG_OUTPUT must be set +for each pad. + +- Links + +Links are represented by a struct media_link instance, defined in +include/media/media-entity.h. Each entity stores all links originating at or +targetting any of its pads in a links array. A given link is thus stored +twice, once in the source entity and once in the target entity. The array is +pre-allocated and grows dynamically as needed. + +Drivers create links by calling + + media_entity_create_link(struct media_entity *source, u16 source_pad, + struct media_entity *sink, u16 sink_pad, + u32 flags); + +An entry in the link array of each entity is allocated and stores pointers +to source and sink pads. + +Links have flags that describe the link capabilities and state. + + MEDIA_LINK_FLAG_ACTIVE indicates that the link is active and can be + used to transfer media data. When two or more links target a sink pad, + only one of them can be active at a time. + MEDIA_LINK_FLAG_IMMUTABLE indicates that the link active state can't + be modified at runtime. If MEDIA_LINK_FLAG_IMMUTABLE is set, then + MEDIA_LINK_FLAG_ACTIVE must also be set since an immutable link is + always active. + diff --git a/drivers/media/Makefile b/drivers/media/Makefile index 019d3e0..b890248 100644 --- a/drivers/media/Makefile +++ b/drivers/media/Makefile @@ -2,7 +2,7 @@ # Makefile for the kernel multimedia device drivers. # -media-objs := media-device.o media-devnode.o +media-objs := media-device.o media-devnode.o media-entity.o ifeq ($(CONFIG_MEDIA_CONTROLLER),y) obj-$(CONFIG_MEDIA_SUPPORT) += media.o diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c index 57a9c6b..2163610 100644 --- a/drivers/media/media-device.c +++ b/drivers/media/media-device.c @@ -25,6 +25,7 @@ #include #include +#include static const struct media_file_operations media_device_fops = { .owner = THIS_MODULE, @@ -69,6 +70,10 @@ int __must_check media_device_register(struct media_device *mdev) if (WARN_ON(mdev->dev == NULL || mdev->model[0] == 0)) return -EINVAL; + mdev->entity_id = 1; + INIT_LIST_HEAD(&mdev->entities); + spin_lock_init(&mdev->lock); + /* Register the device node. */ mdev->devnode.fops = &media_device_fops; mdev->devnode.parent = mdev->dev; @@ -94,7 +99,55 @@ EXPORT_SYMBOL_GPL(media_device_register); */ void media_device_unregister(struct media_device *mdev) { + struct media_entity *entity; + struct media_entity *next; + + list_for_each_entry_safe(entity, next, &mdev->entities, list) + media_device_unregister_entity(entity); + device_remove_file(&mdev->devnode.dev, &dev_attr_model); media_devnode_unregister(&mdev->devnode); } EXPORT_SYMBOL_GPL(media_device_unregister); + +/** + * media_device_register_entity - Register an entity with a media device + * @mdev: The media device + * @entity: The entity + */ +int __must_check media_device_register_entity(struct media_device *mdev, + struct media_entity *entity) +{ + /* Warn if we apparently re-register an entity */ + WARN_ON(entity->parent != NULL); + entity->parent = mdev; + + spin_lock(&mdev->lock); + entity->id = mdev->entity_id++; + list_add_tail(&entity->list, &mdev->entities); + spin_unlock(&mdev->lock); + + return 0; +} +EXPORT_SYMBOL_GPL(media_device_register_entity); + +/** + * media_device_unregister_entity - Unregister an entity + * @entity: The entity + * + * If the entity has never been registered this function will return + * immediately. + */ +void media_device_unregister_entity(struct media_entity *entity) +{ + struct media_device *mdev = entity->parent; + + if (mdev == NULL) + return; + + spin_lock(&mdev->lock); + list_del(&entity->list); + spin_unlock(&mdev->lock); + entity->parent = NULL; +} +EXPORT_SYMBOL_GPL(media_device_unregister_entity); diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c new file mode 100644 index 0000000..e4ba2bc --- /dev/null +++ b/drivers/media/media-entity.c @@ -0,0 +1,147 @@ +/* + * Media entity + * + * Copyright (C) 2010 Nokia Corporation + * + * Contacts: Laurent Pinchart + * Sakari Ailus + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include +#include +#include + +/** + * media_entity_init - Initialize a media entity + * + * @num_pads: Total number of input and output pads. + * @extra_links: Initial estimate of the number of extra links. + * @pads: Array of 'num_pads' pads. + * + * The total number of pads is an intrinsic property of entities known by the + * entity driver, while the total number of links depends on hardware design + * and is an extrinsic property unknown to the entity driver. However, in most + * use cases the entity driver can guess the number of links which can safely + * be assumed to be equal to or larger than the number of pads. + * + * For those reasons the links array can be preallocated based on the entity + * driver guess and will be reallocated later if extra links need to be + * created. + * + * This function allocates a links array with enough space to hold at least + * 'num_pads' + 'extra_links' elements. The media_entity::max_links field will + * be set to the number of allocated elements. + * + * The pads array is managed by the entity driver and passed to + * media_entity_init() where its pointer will be stored in the entity structure. + */ +int +media_entity_init(struct media_entity *entity, u16 num_pads, + struct media_pad *pads, u16 extra_links) +{ + struct media_link *links; + unsigned int max_links = num_pads + extra_links; + unsigned int i; + + links = kzalloc(max_links * sizeof(links[0]), GFP_KERNEL); + if (links == NULL) + return -ENOMEM; + + entity->group_id = 0; + entity->max_links = max_links; + entity->num_links = 0; + entity->num_backlinks = 0; + entity->num_pads = num_pads; + entity->pads = pads; + entity->links = links; + + for (i = 0; i < num_pads; i++) { + pads[i].entity = entity; + pads[i].index = i; + } + + return 0; +} +EXPORT_SYMBOL_GPL(media_entity_init); + +void +media_entity_cleanup(struct media_entity *entity) +{ + kfree(entity->links); +} +EXPORT_SYMBOL_GPL(media_entity_cleanup); + +static struct media_link *media_entity_add_link(struct media_entity *entity) +{ + if (entity->num_links >= entity->max_links) { + struct media_link *links = entity->links; + unsigned int max_links = entity->max_links + 2; + unsigned int i; + + links = krealloc(links, max_links * sizeof(*links), GFP_KERNEL); + if (links == NULL) + return NULL; + + for (i = 0; i < entity->num_links; i++) + links[i].reverse->reverse = &links[i]; + + entity->max_links = max_links; + entity->links = links; + } + + return &entity->links[entity->num_links++]; +} + +int +media_entity_create_link(struct media_entity *source, u16 source_pad, + struct media_entity *sink, u16 sink_pad, u32 flags) +{ + struct media_link *link; + struct media_link *backlink; + + BUG_ON(source == NULL || sink == NULL); + BUG_ON(source_pad >= source->num_pads); + BUG_ON(sink_pad >= sink->num_pads); + + link = media_entity_add_link(source); + if (link == NULL) + return -ENOMEM; + + link->source = &source->pads[source_pad]; + link->sink = &sink->pads[sink_pad]; + link->flags = flags; + + /* Create the backlink. Backlinks are used to help graph traversal and + * are not reported to userspace. + */ + backlink = media_entity_add_link(sink); + if (backlink == NULL) { + source->num_links--; + return -ENOMEM; + } + + backlink->source = &source->pads[source_pad]; + backlink->sink = &sink->pads[sink_pad]; + backlink->flags = flags; + + link->reverse = backlink; + backlink->reverse = link; + + sink->num_backlinks++; + + return 0; +} +EXPORT_SYMBOL_GPL(media_entity_create_link); diff --git a/include/media/media-device.h b/include/media/media-device.h index 930a17a..acd6a29 100644 --- a/include/media/media-device.h +++ b/include/media/media-device.h @@ -25,8 +25,10 @@ #include #include +#include #include +#include /** * struct media_device - Media device @@ -37,6 +39,9 @@ * @bus_info: Unique and stable device location identifier * @hw_revision: Hardware device revision * @driver_version: Device driver version + * @entity_id: ID of the next entity to be registered + * @entities: List of registered entities + * @lock: Entities list lock * * This structure represents an abstract high-level media device. It allows easy * access to entities and provides basic media device-level support. The @@ -58,9 +63,23 @@ struct media_device { u8 bus_info[32]; u32 hw_revision; u32 driver_version; + + u32 entity_id; + struct list_head entities; + + /* Protects the entities list */ + spinlock_t lock; }; int __must_check media_device_register(struct media_device *mdev); void media_device_unregister(struct media_device *mdev); +int __must_check media_device_register_entity(struct media_device *mdev, + struct media_entity *entity); +void media_device_unregister_entity(struct media_entity *entity); + +/* Iterate over all entities. */ +#define media_device_for_each_entity(entity, mdev) \ + list_for_each_entry(entity, &(mdev)->entities, list) + #endif diff --git a/include/media/media-entity.h b/include/media/media-entity.h new file mode 100644 index 0000000..60b4f09 --- /dev/null +++ b/include/media/media-entity.h @@ -0,0 +1,118 @@ +/* + * Media entity + * + * Copyright (C) 2010 Nokia Corporation + * + * Contacts: Laurent Pinchart + * Sakari Ailus + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _MEDIA_ENTITY_H +#define _MEDIA_ENTITY_H + +#include + +#define MEDIA_ENTITY_TYPE_SHIFT 16 +#define MEDIA_ENTITY_TYPE_MASK 0x00ff0000 +#define MEDIA_ENTITY_SUBTYPE_MASK 0x0000ffff + +#define MEDIA_ENTITY_TYPE_NODE (1 << MEDIA_ENTITY_TYPE_SHIFT) +#define MEDIA_ENTITY_TYPE_NODE_V4L (MEDIA_ENTITY_TYPE_NODE + 1) +#define MEDIA_ENTITY_TYPE_NODE_FB (MEDIA_ENTITY_TYPE_NODE + 2) +#define MEDIA_ENTITY_TYPE_NODE_ALSA (MEDIA_ENTITY_TYPE_NODE + 3) +#define MEDIA_ENTITY_TYPE_NODE_DVB (MEDIA_ENTITY_TYPE_NODE + 4) + +#define MEDIA_ENTITY_TYPE_SUBDEV (2 << MEDIA_ENTITY_TYPE_SHIFT) +#define MEDIA_ENTITY_TYPE_SUBDEV_SENSOR (MEDIA_ENTITY_TYPE_SUBDEV + 1) +#define MEDIA_ENTITY_TYPE_SUBDEV_FLASH (MEDIA_ENTITY_TYPE_SUBDEV + 2) +#define MEDIA_ENTITY_TYPE_SUBDEV_LENS (MEDIA_ENTITY_TYPE_SUBDEV + 3) + +#define MEDIA_ENTITY_FLAG_DEFAULT (1 << 0) + +#define MEDIA_LINK_FLAG_ACTIVE (1 << 0) +#define MEDIA_LINK_FLAG_IMMUTABLE (1 << 1) + +#define MEDIA_PAD_FLAG_INPUT (1 << 0) +#define MEDIA_PAD_FLAG_OUTPUT (1 << 1) + +struct media_link { + struct media_pad *source; /* Source pad */ + struct media_pad *sink; /* Sink pad */ + struct media_link *reverse; /* Link in the reverse direction */ + unsigned long flags; /* Link flags (MEDIA_LINK_FLAG_*) */ +}; + +struct media_pad { + struct media_entity *entity; /* Entity this pad belongs to */ + u16 index; /* Pad index in the entity pads array */ + unsigned long flags; /* Pad flags (MEDIA_PAD_FLAG_*) */ +}; + +struct media_entity { + struct list_head list; + struct media_device *parent; /* Media device this entity belongs to*/ + u32 id; /* Entity ID, unique in the parent media + * device context */ + const char *name; /* Entity name */ + u32 type; /* Entity type (MEDIA_ENTITY_TYPE_*) */ + u32 revision; /* Entity revision, driver specific */ + unsigned long flags; /* Entity flags (MEDIA_ENTITY_FLAG_*) */ + u32 group_id; /* Entity group ID */ + + u16 num_pads; /* Number of input and output pads */ + u16 num_links; /* Number of existing links, both active + * and inactive */ + u16 num_backlinks; /* Number of backlinks */ + u16 max_links; /* Maximum number of links */ + + struct media_pad *pads; /* Pads array (num_pads elements) */ + struct media_link *links; /* Links array (max_links elements)*/ + + union { + /* Node specifications */ + struct { + u32 major; + u32 minor; + } v4l; + struct { + u32 major; + u32 minor; + } fb; + int alsa; + int dvb; + + /* Sub-device specifications */ + /* Nothing needed yet */ + }; +}; + +static inline u32 media_entity_type(struct media_entity *entity) +{ + return entity->type & MEDIA_ENTITY_TYPE_MASK; +} + +static inline u32 media_entity_subtype(struct media_entity *entity) +{ + return entity->type & MEDIA_ENTITY_SUBTYPE_MASK; +} + +int media_entity_init(struct media_entity *entity, u16 num_pads, + struct media_pad *pads, u16 extra_links); +void media_entity_cleanup(struct media_entity *entity); +int media_entity_create_link(struct media_entity *source, u16 source_pad, + struct media_entity *sink, u16 sink_pad, u32 flags); + +#endif