| Message ID | 20230619112838.19797-3-jack.zhu@starfivetech.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Add StarFive Camera Subsystem driver | expand |
On 19/06/2023 12:28, Jack Zhu wrote: > Add starfive_camss.rst file that documents the Starfive Camera > Subsystem driver which is used for handing image sensor data. > > Signed-off-by: Jack Zhu <jack.zhu@starfivetech.com> > --- > .../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++ > .../media/starfive_camss_graph.dot | 16 ++++++ > .../admin-guide/media/v4l-drivers.rst | 1 + > MAINTAINERS | 1 + > 4 files changed, 75 insertions(+) > create mode 100644 Documentation/admin-guide/media/starfive_camss.rst > create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot > > diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst > new file mode 100644 > index 000000000000..a6378849384f > --- /dev/null > +++ b/Documentation/admin-guide/media/starfive_camss.rst > @@ -0,0 +1,57 @@ > +.. SPDX-License-Identifier: GPL-2.0 > + > +.. include:: <isonum.txt> > + > +================================ > +Starfive Camera Subsystem driver > +================================ > + > +Introduction > +------------ > + > +This file documents the driver for the Starfive Camera Subsystem found on > +Starfive JH7110 SoC. The driver is located under drivers/media/platform/ > +starfive. > + > +The driver implements V4L2, Media controller and v4l2_subdev interfaces. > +Camera sensor using V4L2 subdev interface in the kernel is supported. > + > +The driver has been successfully used on the Gstreamer 1.18.5 with > +v4l2src plugin. > + > + > +Starfive Camera Subsystem hardware > +---------------------------------- > + > +The Starfive Camera Subsystem hardware consists of: > + > +- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor. Feels like a terribe nit-pick but you have "mipi" and "MIPI" here. I'd be consistent with one - recommend MIPI throughout your documentation. > +- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol > + based camera sensor data stream. > +- ISP: handles the image data streams from the MIPI CSIRx Controller. Maybe you've done this elsewhere but, it would be worthwhile describing what the ISP does. Debayering ? 3As ? Just raw dumps ? > +- VIN(Video In): a top-level module, is responsible for controlling power > + and clocks to other modules, dumps the input data to memory or transfers the > + input data to ISP. > + > + > +Topology > +-------- > + > +The media controller pipeline graph is as follows: > + > +.. _starfive_camss_graph: > + > +.. kernel-figure:: starfive_camss_graph.dot > + :alt: starfive_camss_graph.dot > + :align: center > + > +The driver has 2 video devices: > + > +- stf_vin0_wr_video0: capture device for images directly from the VIN module. OK so you can get the raw images. > +- stf_vin0_isp0_video1: capture device for images without scaling. > + > +The driver has 3 subdevices: > + > +- stf_isp0: is responsible for all the isp operations. > +- stf_vin0_wr: used to dump RAW images to memory. > +- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device. But what is being output here ? RGB, jpeg, YUV ? It would be worth adding a few bits of text to describe that so you don't have to dive into code to understand it. --- bod
Hi Bryan, Thank you for your comments. On 2023/7/26 19:26, Bryan O'Donoghue wrote: > On 19/06/2023 12:28, Jack Zhu wrote: >> Add starfive_camss.rst file that documents the Starfive Camera >> Subsystem driver which is used for handing image sensor data. >> >> Signed-off-by: Jack Zhu <jack.zhu@starfivetech.com> >> --- >> .../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++ >> .../media/starfive_camss_graph.dot | 16 ++++++ >> .../admin-guide/media/v4l-drivers.rst | 1 + >> MAINTAINERS | 1 + >> 4 files changed, 75 insertions(+) >> create mode 100644 Documentation/admin-guide/media/starfive_camss.rst >> create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot >> >> diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst >> new file mode 100644 >> index 000000000000..a6378849384f >> --- /dev/null >> +++ b/Documentation/admin-guide/media/starfive_camss.rst >> @@ -0,0 +1,57 @@ >> +.. SPDX-License-Identifier: GPL-2.0 >> + >> +.. include:: <isonum.txt> >> + >> +================================ >> +Starfive Camera Subsystem driver >> +================================ >> + >> +Introduction >> +------------ >> + >> +This file documents the driver for the Starfive Camera Subsystem found on >> +Starfive JH7110 SoC. The driver is located under drivers/media/platform/ >> +starfive. >> + >> +The driver implements V4L2, Media controller and v4l2_subdev interfaces. >> +Camera sensor using V4L2 subdev interface in the kernel is supported. >> + >> +The driver has been successfully used on the Gstreamer 1.18.5 with >> +v4l2src plugin. >> + >> + >> +Starfive Camera Subsystem hardware >> +---------------------------------- >> + >> +The Starfive Camera Subsystem hardware consists of: >> + >> +- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor. > > Feels like a terribe nit-pick but you have "mipi" and "MIPI" here. I'd be consistent with one - recommend MIPI throughout your documentation. > Okay, I will keep a consistent description. >> +- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol >> + based camera sensor data stream. >> +- ISP: handles the image data streams from the MIPI CSIRx Controller. > > Maybe you've done this elsewhere but, it would be worthwhile describing what the ISP does. Debayering ? 3As ? Just raw dumps ? > Ok, I will add more descriptions in the next version. >> +- VIN(Video In): a top-level module, is responsible for controlling power >> + and clocks to other modules, dumps the input data to memory or transfers the >> + input data to ISP. >> + >> + >> +Topology >> +-------- >> + >> +The media controller pipeline graph is as follows: >> + >> +.. _starfive_camss_graph: >> + >> +.. kernel-figure:: starfive_camss_graph.dot >> + :alt: starfive_camss_graph.dot >> + :align: center >> + >> +The driver has 2 video devices: >> + >> +- stf_vin0_wr_video0: capture device for images directly from the VIN module. > > OK so you can get the raw images. > >> +- stf_vin0_isp0_video1: capture device for images without scaling. >> + >> +The driver has 3 subdevices: >> + >> +- stf_isp0: is responsible for all the isp operations. >> +- stf_vin0_wr: used to dump RAW images to memory. >> +- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device. > > But what is being output here ? RGB, jpeg, YUV ? > It would be worth adding a few bits of text to describe that so you don't have to dive into code to understand it. > Ok, I will add more descriptions. > --- > bod
On Wed, Jul 26, 2023 at 12:26:18PM +0100, Bryan O'Donoghue wrote: > On 19/06/2023 12:28, Jack Zhu wrote: > > Add starfive_camss.rst file that documents the Starfive Camera > > Subsystem driver which is used for handing image sensor data. > > > > Signed-off-by: Jack Zhu <jack.zhu@starfivetech.com> > > --- > > .../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++ > > .../media/starfive_camss_graph.dot | 16 ++++++ > > .../admin-guide/media/v4l-drivers.rst | 1 + > > MAINTAINERS | 1 + > > 4 files changed, 75 insertions(+) > > create mode 100644 Documentation/admin-guide/media/starfive_camss.rst > > create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot > > > > diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst > > new file mode 100644 > > index 000000000000..a6378849384f > > --- /dev/null > > +++ b/Documentation/admin-guide/media/starfive_camss.rst > > @@ -0,0 +1,57 @@ > > +.. SPDX-License-Identifier: GPL-2.0 > > + > > +.. include:: <isonum.txt> > > + > > +================================ > > +Starfive Camera Subsystem driver > > +================================ > > + > > +Introduction > > +------------ > > + > > +This file documents the driver for the Starfive Camera Subsystem found on > > +Starfive JH7110 SoC. The driver is located under drivers/media/platform/ > > +starfive. > > + > > +The driver implements V4L2, Media controller and v4l2_subdev interfaces. > > +Camera sensor using V4L2 subdev interface in the kernel is supported. > > + > > +The driver has been successfully used on the Gstreamer 1.18.5 with > > +v4l2src plugin. > > + > > + > > +Starfive Camera Subsystem hardware > > +---------------------------------- > > + > > +The Starfive Camera Subsystem hardware consists of: > > + > > +- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor. > > Feels like a terribe nit-pick but you have "mipi" and "MIPI" here. I'd > be consistent with one - recommend MIPI throughout your documentation. I'd drop the "mipi" and just write "data", not "MIPI data". > > +- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol It's "CSI-2", not "CSI2". > > + based camera sensor data stream. > > +- ISP: handles the image data streams from the MIPI CSIRx Controller. > > Maybe you've done this elsewhere but, it would be worthwhile describing > what the ISP does. Debayering ? 3As ? Just raw dumps ? > > > +- VIN(Video In): a top-level module, is responsible for controlling power > > + and clocks to other modules, dumps the input data to memory or transfers the > > + input data to ISP. A block diagram could be nice. Let me give it a try, based on my understanding: Starfive Camera Subsystem hardware ---------------------------------- The Starfive Camera Subsystem hardware consists of:: +---------------------------------------------+ | VIN | +--------+ | +-------+ +-------+ +-----+ | | Camera |----->| D-PHY |-->| CSI-2 |-+->| DMA | | | Sensor | | | RX | | RX | | +-----+ | +--------+ | +-------+ +-------+ | | | | +-----+ +-----+ | | \->| ISP |-->| DMA | | | +-----+ +-----+ | +---------------------------------------------+ - VIN (Video IN): The top-level module, responsible for controlling power and clocks, and routing data between modules. - D-PHY RX: The MIPI D-PHY receiver, receiving data from a MIPI CSI-2 camera sensor. - CSI-2 RX: The MIPI CSI-2 receiver controller, responsible for decoding the CSI-2 protocol. - ISP: The ISP, processing raw Bayer data from the CSI-2 receiver and producing YUV frames. I haven't tried compiling this, so please fix formatting if anything is wrong (and of course feel free to adapt the diagram and text). > > + > > + > > +Topology > > +-------- > > + > > +The media controller pipeline graph is as follows: > > + > > +.. _starfive_camss_graph: > > + > > +.. kernel-figure:: starfive_camss_graph.dot > > + :alt: starfive_camss_graph.dot > > + :align: center > > + > > +The driver has 2 video devices: > > + > > +- stf_vin0_wr_video0: capture device for images directly from the VIN module. > > OK so you can get the raw images. > > > +- stf_vin0_isp0_video1: capture device for images without scaling. > > + > > +The driver has 3 subdevices: > > + > > +- stf_isp0: is responsible for all the isp operations. > > +- stf_vin0_wr: used to dump RAW images to memory. > > +- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device. > > But what is being output here ? RGB, jpeg, YUV ? > It would be worth adding a few bits of text to describe that so you > don't have to dive into code to understand it.
Hi Laurent, Thank you for your comments. On 2023/7/27 18:23, Laurent Pinchart wrote: > On Wed, Jul 26, 2023 at 12:26:18PM +0100, Bryan O'Donoghue wrote: >> On 19/06/2023 12:28, Jack Zhu wrote: >> > Add starfive_camss.rst file that documents the Starfive Camera >> > Subsystem driver which is used for handing image sensor data. >> > >> > Signed-off-by: Jack Zhu <jack.zhu@starfivetech.com> >> > --- >> > .../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++ >> > .../media/starfive_camss_graph.dot | 16 ++++++ >> > .../admin-guide/media/v4l-drivers.rst | 1 + >> > MAINTAINERS | 1 + >> > 4 files changed, 75 insertions(+) >> > create mode 100644 Documentation/admin-guide/media/starfive_camss.rst >> > create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot >> > >> > diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst >> > new file mode 100644 >> > index 000000000000..a6378849384f >> > --- /dev/null >> > +++ b/Documentation/admin-guide/media/starfive_camss.rst >> > @@ -0,0 +1,57 @@ >> > +.. SPDX-License-Identifier: GPL-2.0 >> > + >> > +.. include:: <isonum.txt> >> > + >> > +================================ >> > +Starfive Camera Subsystem driver >> > +================================ >> > + >> > +Introduction >> > +------------ >> > + >> > +This file documents the driver for the Starfive Camera Subsystem found on >> > +Starfive JH7110 SoC. The driver is located under drivers/media/platform/ >> > +starfive. >> > + >> > +The driver implements V4L2, Media controller and v4l2_subdev interfaces. >> > +Camera sensor using V4L2 subdev interface in the kernel is supported. >> > + >> > +The driver has been successfully used on the Gstreamer 1.18.5 with >> > +v4l2src plugin. >> > + >> > + >> > +Starfive Camera Subsystem hardware >> > +---------------------------------- >> > + >> > +The Starfive Camera Subsystem hardware consists of: >> > + >> > +- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor. >> >> Feels like a terribe nit-pick but you have "mipi" and "MIPI" here. I'd >> be consistent with one - recommend MIPI throughout your documentation. > > I'd drop the "mipi" and just write "data", not "MIPI data". > Ok, I will modify it. >> > +- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol > > It's "CSI-2", not "CSI2". Ok, I will modify it. > >> > + based camera sensor data stream. >> > +- ISP: handles the image data streams from the MIPI CSIRx Controller. >> >> Maybe you've done this elsewhere but, it would be worthwhile describing >> what the ISP does. Debayering ? 3As ? Just raw dumps ? >> >> > +- VIN(Video In): a top-level module, is responsible for controlling power >> > + and clocks to other modules, dumps the input data to memory or transfers the >> > + input data to ISP. > > A block diagram could be nice. Let me give it a try, based on my > understanding: > > Starfive Camera Subsystem hardware > ---------------------------------- > > The Starfive Camera Subsystem hardware consists of:: > > +---------------------------------------------+ > | VIN | > +--------+ | +-------+ +-------+ +-----+ | > | Camera |----->| D-PHY |-->| CSI-2 |-+->| DMA | | > | Sensor | | | RX | | RX | | +-----+ | > +--------+ | +-------+ +-------+ | | > | | +-----+ +-----+ | > | \->| ISP |-->| DMA | | > | +-----+ +-----+ | > +---------------------------------------------+ > > - VIN (Video IN): The top-level module, responsible for controlling > power and clocks, and routing data between modules. > > - D-PHY RX: The MIPI D-PHY receiver, receiving data from a MIPI CSI-2 > camera sensor. > > - CSI-2 RX: The MIPI CSI-2 receiver controller, responsible for decoding > the CSI-2 protocol. > > - ISP: The ISP, processing raw Bayer data from the CSI-2 receiver and > producing YUV frames. > > > I haven't tried compiling this, so please fix formatting if anything is > wrong (and of course feel free to adapt the diagram and text). > Ok, I will add a block diagram. >> > + >> > + >> > +Topology >> > +-------- >> > + >> > +The media controller pipeline graph is as follows: >> > + >> > +.. _starfive_camss_graph: >> > + >> > +.. kernel-figure:: starfive_camss_graph.dot >> > + :alt: starfive_camss_graph.dot >> > + :align: center >> > + >> > +The driver has 2 video devices: >> > + >> > +- stf_vin0_wr_video0: capture device for images directly from the VIN module. >> >> OK so you can get the raw images. >> >> > +- stf_vin0_isp0_video1: capture device for images without scaling. >> > + >> > +The driver has 3 subdevices: >> > + >> > +- stf_isp0: is responsible for all the isp operations. >> > +- stf_vin0_wr: used to dump RAW images to memory. >> > +- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device. >> >> But what is being output here ? RGB, jpeg, YUV ? >> It would be worth adding a few bits of text to describe that so you >> don't have to dive into code to understand it. >
diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst new file mode 100644 index 000000000000..a6378849384f --- /dev/null +++ b/Documentation/admin-guide/media/starfive_camss.rst @@ -0,0 +1,57 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. include:: <isonum.txt> + +================================ +Starfive Camera Subsystem driver +================================ + +Introduction +------------ + +This file documents the driver for the Starfive Camera Subsystem found on +Starfive JH7110 SoC. The driver is located under drivers/media/platform/ +starfive. + +The driver implements V4L2, Media controller and v4l2_subdev interfaces. +Camera sensor using V4L2 subdev interface in the kernel is supported. + +The driver has been successfully used on the Gstreamer 1.18.5 with +v4l2src plugin. + + +Starfive Camera Subsystem hardware +---------------------------------- + +The Starfive Camera Subsystem hardware consists of: + +- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor. +- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol + based camera sensor data stream. +- ISP: handles the image data streams from the MIPI CSIRx Controller. +- VIN(Video In): a top-level module, is responsible for controlling power + and clocks to other modules, dumps the input data to memory or transfers the + input data to ISP. + + +Topology +-------- + +The media controller pipeline graph is as follows: + +.. _starfive_camss_graph: + +.. kernel-figure:: starfive_camss_graph.dot + :alt: starfive_camss_graph.dot + :align: center + +The driver has 2 video devices: + +- stf_vin0_wr_video0: capture device for images directly from the VIN module. +- stf_vin0_isp0_video1: capture device for images without scaling. + +The driver has 3 subdevices: + +- stf_isp0: is responsible for all the isp operations. +- stf_vin0_wr: used to dump RAW images to memory. +- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device. diff --git a/Documentation/admin-guide/media/starfive_camss_graph.dot b/Documentation/admin-guide/media/starfive_camss_graph.dot new file mode 100644 index 000000000000..275661b720dc --- /dev/null +++ b/Documentation/admin-guide/media/starfive_camss_graph.dot @@ -0,0 +1,16 @@ +digraph board { + rankdir=TB + n00000001 [label="{{<port0> 0} | stf_isp0\n/dev/v4l-subdev0 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] + n00000001:port1 -> n0000000d:port0 [style=dashed] + n00000004 [label="{{<port0> 0} | stf_vin0_wr\n/dev/v4l-subdev1 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] + n00000004:port1 -> n00000007 [style=bold] + n00000007 [label="stf_vin0_wr_video0\n/dev/video0", shape=box, style=filled, fillcolor=yellow] + n0000000d [label="{{<port0> 0} | stf_vin0_isp0\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] + n0000000d:port1 -> n00000010 [style=bold] + n00000010 [label="stf_vin0_isp0_video1\n/dev/video1", shape=box, style=filled, fillcolor=yellow] + n00000018 [label="{{<port0> 0} | cdns_csi2rx.19800000.csi-bridge\n | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green] + n00000018:port1 -> n00000004:port0 [style=dashed] + n00000018:port1 -> n00000001:port0 [style=dashed] + n00000022 [label="{{} | imx219 6-0010\n/dev/v4l-subdev3 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] + n00000022:port0 -> n00000018:port0 [style=bold] +} diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst index 1c41f87c3917..2274fd29c1d7 100644 --- a/Documentation/admin-guide/media/v4l-drivers.rst +++ b/Documentation/admin-guide/media/v4l-drivers.rst @@ -27,6 +27,7 @@ Video4Linux (V4L) driver-specific documentation si470x si4713 si476x + starfive_camss vimc visl vivid diff --git a/MAINTAINERS b/MAINTAINERS index 9ef5d0e5dff0..71291dc58671 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -20075,6 +20075,7 @@ M: Jack Zhu <jack.zhu@starfivetech.com> M: Changhuang Liang <changhuang.liang@starfivetech.com> L: linux-media@vger.kernel.org S: Maintained +F: Documentation/admin-guide/media/starfive_camss.rst F: Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml STARFIVE DEVICETREES
Add starfive_camss.rst file that documents the Starfive Camera Subsystem driver which is used for handing image sensor data. Signed-off-by: Jack Zhu <jack.zhu@starfivetech.com> --- .../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++ .../media/starfive_camss_graph.dot | 16 ++++++ .../admin-guide/media/v4l-drivers.rst | 1 + MAINTAINERS | 1 + 4 files changed, 75 insertions(+) create mode 100644 Documentation/admin-guide/media/starfive_camss.rst create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot