| Message ID | 20240720071606.27930-1-yunfei.dong@mediatek.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | media: mediatek: add driver to support secure video decoder | expand |
Hey Yunfei, On 20.07.2024 15:15, Yunfei Dong wrote: >The patch series used to enable secure video playback (SVP) on MediaTek >hardware in the Linux kernel. I will set this series as obsolete for now, please answer the open questions on your patches and then send a new series. Regards, Sebastian > >Memory Definitions: >secure memory - Memory allocated in the TEE (Trusted Execution >Environment) which is inaccessible in the REE (Rich Execution >Environment, i.e. linux kernel/user space). >secure handle - Integer value which acts as reference to 'secure >memory'. Used in communication between TEE and REE to reference >'secure memory'. >secure buffer - 'secure memory' that is used to store decrypted, >compressed video or for other general purposes in the TEE. >secure surface - 'secure memory' that is used to store graphic buffers. > >Memory Usage in SVP: >The overall flow of SVP starts with encrypted video coming in from an >outside source into the REE. The REE will then allocate a 'secure >buffer' and send the corresponding 'secure handle' along with the >encrypted, compressed video data to the TEE. The TEE will then decrypt >the video and store the result in the 'secure buffer'. The REE will >then allocate a 'secure surface'. The REE will pass the 'secure >handles' for both the 'secure buffer' and 'secure surface' into the >TEE for video decoding. The video decoder HW will then decode the >contents of the 'secure buffer' and place the result in the 'secure >surface'. The REE will then attach the 'secure surface' to the overlay >plane for rendering of the video. > >Everything relating to ensuring security of the actual contents of the >'secure buffer' and 'secure surface' is out of scope for the REE and >is the responsibility of the TEE. > >This patch series is consists of four parts. The first is from Jeffrey, >adding secure memory flag in v4l2 framework to support request secure >buffer. > >The second and third parts are from John and T.J, adding some heap >interfaces, then our kernel users could allocate buffer from special >heap. The patch v1 is inside below dmabuf link. >https://lore.kernel.org/linux-mediatek/20230911023038.30649-1-yong.wu@mediatek.com/ >To avoid confusing, move them into vcodec patch set since we use the >new interfaces directly. > >The last part is mediatek video decoder driver, adding tee interface and >decoder driver to support secure video playback. > >This patch set depends on "dma-buf: heaps: Add restricted heap"[1] > >[1] https://patchwork.kernel.org/project/linux-mediatek/list/?series=853380 >--- >Changed in v7: >- fix many reviewer's comments >- build optee driver to ko >- support h264 svp and non svp vsi > >Changed in v6: >- fix unreasonable logic for patch 2/3/23 >- add to support vp9 for patch 24 > >Changed in v5: >- fix merge conflict when rebase to latest media stage for patch 1/2 >- change allocate memory type to cma for patch 12 >- add to support av1 for patch 23 > >Changed in v4: >- change the driver according to maintainer advice for patch 1/2/3/4 >- replace secure with restricted for patch 1/2/3/4 >- fix svp decoder error for patch 21 >- add to support hevc for patch 22 > >Changed in v3: >- rewrite the cover-letter of this patch series >- disable irq for svp mode >- rebase the driver based on the latest media stage > >Changed in v2: >- remove setting decoder mode and getting secure handle from decode >- add Jeffrey's patch >- add John and T.J's patch >- getting secure flag with request buffer >- fix some comments from patch v1 >--- >Jeffrey Kardatzke (2): > v4l2: add restricted memory flags > v4l2: handle restricted memory flags in queue setup > >John Stultz (2): > dma-heap: Add proper kref handling on dma-buf heaps > dma-heap: Provide accessors so that in-kernel drivers can allocate > dmabufs from specific heaps > >T.J. Mercier (1): > dma-buf: heaps: Deduplicate docs and adopt common format > >Xiaoyong Lu (1): > media: mediatek: vcodec: support av1 svp decoder for mt8188 > >Yilong Zhou (1): > media: mediatek: vcodec: support vp9 svp decoder for mt8188 > >Yunfei Dong (21): > media: videobuf2: calculate restricted memory size > media: mediatek: vcodec: add tee client interface to communiate with > optee-os > media: mediatek: vcodec: build decoder OPTEE driver as module > media: mediatek: vcodec: allocate tee share memory > media: mediatek: vcodec: send share memory data to optee > media: mediatek: vcodec: initialize msg and vsi information > media: mediatek: vcodec: add interface to allocate/free secure memory > media: mediatek: vcodec: using shared memory as vsi address > media: mediatek: vcodec: add single allocation format > media: mediatek: vcodec: support single allocation format > media: mediatek: vcodec: support single allocation buffer > media: mediatek: vcodec: re-construct h264 driver to support svp mode > media: mediatek: vcodec: remove parse nal_info in kernel > media: mediatek: vcodec: disable wait interrupt for svp mode > media: mediatek: vcodec: support tee decoder > media: mediatek: vcodec: move vdec init interface to setup callback > media: mediatek: vcodec: support hevc svp for mt8188 > media: mediatek: vcodec: remove vsi data from common interface > media: mediatek: vcodec: rename vsi to extend vsi > media: mediatek: vcodec: adding non extend struct > media: mediatek: vcodec: support extend h264 driver > > .../userspace-api/media/v4l/buffer.rst | 10 +- > .../media/v4l/pixfmt-reserved.rst | 7 + > .../media/v4l/vidioc-reqbufs.rst | 6 + > drivers/dma-buf/dma-heap.c | 139 ++++- > .../media/common/videobuf2/videobuf2-core.c | 29 + > .../common/videobuf2/videobuf2-dma-contig.c | 34 +- > .../media/common/videobuf2/videobuf2-v4l2.c | 4 +- > .../media/platform/mediatek/vcodec/Kconfig | 13 + > .../mediatek/vcodec/common/mtk_vcodec_util.c | 117 +++- > .../mediatek/vcodec/common/mtk_vcodec_util.h | 8 +- > .../platform/mediatek/vcodec/decoder/Makefile | 4 + > .../mediatek/vcodec/decoder/mtk_vcodec_dec.c | 152 +++-- > .../vcodec/decoder/mtk_vcodec_dec_drv.c | 8 + > .../vcodec/decoder/mtk_vcodec_dec_drv.h | 11 + > .../vcodec/decoder/mtk_vcodec_dec_hw.c | 34 +- > .../vcodec/decoder/mtk_vcodec_dec_optee.c | 391 +++++++++++++ > .../vcodec/decoder/mtk_vcodec_dec_optee.h | 198 +++++++ > .../vcodec/decoder/mtk_vcodec_dec_pm.c | 6 +- > .../vcodec/decoder/mtk_vcodec_dec_stateless.c | 35 +- > .../vcodec/decoder/vdec/vdec_av1_req_lat_if.c | 104 ++-- > .../decoder/vdec/vdec_h264_req_common.c | 18 +- > .../decoder/vdec/vdec_h264_req_multi_if.c | 536 +++++++++++++++++- > .../decoder/vdec/vdec_hevc_req_multi_if.c | 88 +-- > .../vcodec/decoder/vdec/vdec_vp9_req_lat_if.c | 101 ++-- > .../mediatek/vcodec/decoder/vdec_drv_if.c | 4 +- > .../mediatek/vcodec/decoder/vdec_msg_queue.c | 9 +- > .../mediatek/vcodec/decoder/vdec_vpu_if.c | 51 +- > .../mediatek/vcodec/decoder/vdec_vpu_if.h | 4 + > drivers/media/v4l2-core/v4l2-common.c | 2 + > drivers/media/v4l2-core/v4l2-ioctl.c | 1 + > include/linux/dma-heap.h | 29 +- > include/media/videobuf2-core.h | 8 +- > include/uapi/linux/videodev2.h | 3 + > 33 files changed, 1868 insertions(+), 296 deletions(-) > create mode 100644 drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_optee.c > create mode 100644 drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_optee.h > >-- >2.18.0 > Sebastian Fricke Consultant Software Engineer Collabora Ltd Platinum Building, St John's Innovation Park, Cambridge CB4 0DS, UK Registered in England & Wales no 5513718.
The patch series used to enable secure video playback (SVP) on MediaTek hardware in the Linux kernel. Memory Definitions: secure memory - Memory allocated in the TEE (Trusted Execution Environment) which is inaccessible in the REE (Rich Execution Environment, i.e. linux kernel/user space). secure handle - Integer value which acts as reference to 'secure memory'. Used in communication between TEE and REE to reference 'secure memory'. secure buffer - 'secure memory' that is used to store decrypted, compressed video or for other general purposes in the TEE. secure surface - 'secure memory' that is used to store graphic buffers. Memory Usage in SVP: The overall flow of SVP starts with encrypted video coming in from an outside source into the REE. The REE will then allocate a 'secure buffer' and send the corresponding 'secure handle' along with the encrypted, compressed video data to the TEE. The TEE will then decrypt the video and store the result in the 'secure buffer'. The REE will then allocate a 'secure surface'. The REE will pass the 'secure handles' for both the 'secure buffer' and 'secure surface' into the TEE for video decoding. The video decoder HW will then decode the contents of the 'secure buffer' and place the result in the 'secure surface'. The REE will then attach the 'secure surface' to the overlay plane for rendering of the video. Everything relating to ensuring security of the actual contents of the 'secure buffer' and 'secure surface' is out of scope for the REE and is the responsibility of the TEE. This patch series is consists of four parts. The first is from Jeffrey, adding secure memory flag in v4l2 framework to support request secure buffer. The second and third parts are from John and T.J, adding some heap interfaces, then our kernel users could allocate buffer from special heap. The patch v1 is inside below dmabuf link. https://lore.kernel.org/linux-mediatek/20230911023038.30649-1-yong.wu@mediatek.com/ To avoid confusing, move them into vcodec patch set since we use the new interfaces directly. The last part is mediatek video decoder driver, adding tee interface and decoder driver to support secure video playback. This patch set depends on "dma-buf: heaps: Add restricted heap"[1] [1] https://patchwork.kernel.org/project/linux-mediatek/list/?series=853380 --- Changed in v7: - fix many reviewer's comments - build optee driver to ko - support h264 svp and non svp vsi Changed in v6: - fix unreasonable logic for patch 2/3/23 - add to support vp9 for patch 24 Changed in v5: - fix merge conflict when rebase to latest media stage for patch 1/2 - change allocate memory type to cma for patch 12 - add to support av1 for patch 23 Changed in v4: - change the driver according to maintainer advice for patch 1/2/3/4 - replace secure with restricted for patch 1/2/3/4 - fix svp decoder error for patch 21 - add to support hevc for patch 22 Changed in v3: - rewrite the cover-letter of this patch series - disable irq for svp mode - rebase the driver based on the latest media stage Changed in v2: - remove setting decoder mode and getting secure handle from decode - add Jeffrey's patch - add John and T.J's patch - getting secure flag with request buffer - fix some comments from patch v1 --- Jeffrey Kardatzke (2): v4l2: add restricted memory flags v4l2: handle restricted memory flags in queue setup John Stultz (2): dma-heap: Add proper kref handling on dma-buf heaps dma-heap: Provide accessors so that in-kernel drivers can allocate dmabufs from specific heaps T.J. Mercier (1): dma-buf: heaps: Deduplicate docs and adopt common format Xiaoyong Lu (1): media: mediatek: vcodec: support av1 svp decoder for mt8188 Yilong Zhou (1): media: mediatek: vcodec: support vp9 svp decoder for mt8188 Yunfei Dong (21): media: videobuf2: calculate restricted memory size media: mediatek: vcodec: add tee client interface to communiate with optee-os media: mediatek: vcodec: build decoder OPTEE driver as module media: mediatek: vcodec: allocate tee share memory media: mediatek: vcodec: send share memory data to optee media: mediatek: vcodec: initialize msg and vsi information media: mediatek: vcodec: add interface to allocate/free secure memory media: mediatek: vcodec: using shared memory as vsi address media: mediatek: vcodec: add single allocation format media: mediatek: vcodec: support single allocation format media: mediatek: vcodec: support single allocation buffer media: mediatek: vcodec: re-construct h264 driver to support svp mode media: mediatek: vcodec: remove parse nal_info in kernel media: mediatek: vcodec: disable wait interrupt for svp mode media: mediatek: vcodec: support tee decoder media: mediatek: vcodec: move vdec init interface to setup callback media: mediatek: vcodec: support hevc svp for mt8188 media: mediatek: vcodec: remove vsi data from common interface media: mediatek: vcodec: rename vsi to extend vsi media: mediatek: vcodec: adding non extend struct media: mediatek: vcodec: support extend h264 driver .../userspace-api/media/v4l/buffer.rst | 10 +- .../media/v4l/pixfmt-reserved.rst | 7 + .../media/v4l/vidioc-reqbufs.rst | 6 + drivers/dma-buf/dma-heap.c | 139 ++++- .../media/common/videobuf2/videobuf2-core.c | 29 + .../common/videobuf2/videobuf2-dma-contig.c | 34 +- .../media/common/videobuf2/videobuf2-v4l2.c | 4 +- .../media/platform/mediatek/vcodec/Kconfig | 13 + .../mediatek/vcodec/common/mtk_vcodec_util.c | 117 +++- .../mediatek/vcodec/common/mtk_vcodec_util.h | 8 +- .../platform/mediatek/vcodec/decoder/Makefile | 4 + .../mediatek/vcodec/decoder/mtk_vcodec_dec.c | 152 +++-- .../vcodec/decoder/mtk_vcodec_dec_drv.c | 8 + .../vcodec/decoder/mtk_vcodec_dec_drv.h | 11 + .../vcodec/decoder/mtk_vcodec_dec_hw.c | 34 +- .../vcodec/decoder/mtk_vcodec_dec_optee.c | 391 +++++++++++++ .../vcodec/decoder/mtk_vcodec_dec_optee.h | 198 +++++++ .../vcodec/decoder/mtk_vcodec_dec_pm.c | 6 +- .../vcodec/decoder/mtk_vcodec_dec_stateless.c | 35 +- .../vcodec/decoder/vdec/vdec_av1_req_lat_if.c | 104 ++-- .../decoder/vdec/vdec_h264_req_common.c | 18 +- .../decoder/vdec/vdec_h264_req_multi_if.c | 536 +++++++++++++++++- .../decoder/vdec/vdec_hevc_req_multi_if.c | 88 +-- .../vcodec/decoder/vdec/vdec_vp9_req_lat_if.c | 101 ++-- .../mediatek/vcodec/decoder/vdec_drv_if.c | 4 +- .../mediatek/vcodec/decoder/vdec_msg_queue.c | 9 +- .../mediatek/vcodec/decoder/vdec_vpu_if.c | 51 +- .../mediatek/vcodec/decoder/vdec_vpu_if.h | 4 + drivers/media/v4l2-core/v4l2-common.c | 2 + drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/linux/dma-heap.h | 29 +- include/media/videobuf2-core.h | 8 +- include/uapi/linux/videodev2.h | 3 + 33 files changed, 1868 insertions(+), 296 deletions(-) create mode 100644 drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_optee.c create mode 100644 drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_optee.h