From patchwork Thu Oct 18 16:08:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 10647571 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB92017DE for ; Thu, 18 Oct 2018 16:08:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 69E9C28DDC for ; Thu, 18 Oct 2018 16:08:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5DFCE28DE8; Thu, 18 Oct 2018 16:08:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D300428DDC for ; Thu, 18 Oct 2018 16:08:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728481AbeJSAKd (ORCPT ); Thu, 18 Oct 2018 20:10:33 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:55292 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727859AbeJSAKd (ORCPT ); Thu, 18 Oct 2018 20:10:33 -0400 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id 0EC3127DF97 From: Ezequiel Garcia To: linux-media@vger.kernel.org Cc: Hans Verkuil , kernel@collabora.com, Nicolas Dufresne , Ezequiel Garcia Subject: [PATCH 1/2] vicodec: Have decoder propagate changes to the CAPTURE queue Date: Thu, 18 Oct 2018 13:08:40 -0300 Message-Id: <20181018160841.17674-2-ezequiel@collabora.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181018160841.17674-1-ezequiel@collabora.com> References: <20181018160841.17674-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The decoder interface (not yet merged) specifies that width and height values set on the OUTPUT queue, must be propagated to the CAPTURE queue. This is not enough to comply with the specification, which would require to properly support stream resolution changes detection and notification. However, it's a relatively small change, which fixes behavior required by some applications such as gstreamer. With this change, it's possible to run a simple T(T⁻¹) pipeline: gst-launch-1.0 videotestsrc ! v4l2fwhtenc ! v4l2fwhtdec ! fakevideosink Signed-off-by: Ezequiel Garcia --- drivers/media/platform/vicodec/vicodec-core.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/media/platform/vicodec/vicodec-core.c b/drivers/media/platform/vicodec/vicodec-core.c index 1eb9132bfc85..a2c487b4b80d 100644 --- a/drivers/media/platform/vicodec/vicodec-core.c +++ b/drivers/media/platform/vicodec/vicodec-core.c @@ -673,6 +673,13 @@ static int vidioc_s_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f) q_data->width = pix->width; q_data->height = pix->height; q_data->sizeimage = pix->sizeimage; + + /* Propagate changes to CAPTURE queue */ + if (!ctx->is_enc && V4L2_TYPE_IS_OUTPUT(f->type)) { + ctx->q_data[V4L2_M2M_DST].width = pix->width; + ctx->q_data[V4L2_M2M_DST].height = pix->height; + ctx->q_data[V4L2_M2M_DST].sizeimage = pix->sizeimage; + } break; case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: @@ -693,6 +700,14 @@ static int vidioc_s_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f) q_data->width = pix_mp->width; q_data->height = pix_mp->height; q_data->sizeimage = pix_mp->plane_fmt[0].sizeimage; + + /* Propagate changes to CAPTURE queue */ + if (!ctx->is_enc && V4L2_TYPE_IS_OUTPUT(f->type)) { + ctx->q_data[V4L2_M2M_DST].width = pix_mp->width; + ctx->q_data[V4L2_M2M_DST].height = pix_mp->height; + ctx->q_data[V4L2_M2M_DST].sizeimage = + pix_mp->plane_fmt[0].sizeimage; + } break; default: return -EINVAL; From patchwork Thu Oct 18 16:08:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 10647573 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F3C7517DE for ; Thu, 18 Oct 2018 16:08:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E7E9528DE8 for ; Thu, 18 Oct 2018 16:08:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DC41928DEF; Thu, 18 Oct 2018 16:08:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5D3CC28DE8 for ; Thu, 18 Oct 2018 16:08:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728394AbeJSAKg (ORCPT ); Thu, 18 Oct 2018 20:10:36 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:55298 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727859AbeJSAKg (ORCPT ); Thu, 18 Oct 2018 20:10:36 -0400 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id 5F9172726CC From: Ezequiel Garcia To: linux-media@vger.kernel.org Cc: Hans Verkuil , kernel@collabora.com, Nicolas Dufresne , Ezequiel Garcia Subject: [PATCH 2/2] vicodec: Implement spec-compliant stop command Date: Thu, 18 Oct 2018 13:08:41 -0300 Message-Id: <20181018160841.17674-3-ezequiel@collabora.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181018160841.17674-1-ezequiel@collabora.com> References: <20181018160841.17674-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Set up a statically-allocated, dummy buffer to be used as flush buffer, which signals a encoding (or decoding) stop. When the flush buffer is queued to the OUTPUT queue, the driver will send an V4L2_EVENT_EOS event, and mark the CAPTURE buffer with V4L2_BUF_FLAG_LAST. With this change, it's possible to run a pipeline to completion: gst-launch-1.0 videotestsrc num-buffers=10 ! v4l2fwhtenc ! v4l2fwhtdec ! fakevideosink Signed-off-by: Ezequiel Garcia --- drivers/media/platform/vicodec/vicodec-core.c | 80 ++++++++++--------- 1 file changed, 44 insertions(+), 36 deletions(-) diff --git a/drivers/media/platform/vicodec/vicodec-core.c b/drivers/media/platform/vicodec/vicodec-core.c index a2c487b4b80d..4ed4dae10e30 100644 --- a/drivers/media/platform/vicodec/vicodec-core.c +++ b/drivers/media/platform/vicodec/vicodec-core.c @@ -113,7 +113,7 @@ struct vicodec_ctx { struct v4l2_ctrl_handler hdl; struct vb2_v4l2_buffer *last_src_buf; - struct vb2_v4l2_buffer *last_dst_buf; + struct vb2_v4l2_buffer flush_buf; /* Source and destination queue data */ struct vicodec_q_data q_data[2]; @@ -220,6 +220,7 @@ static void device_run(void *priv) struct vicodec_dev *dev = ctx->dev; struct vb2_v4l2_buffer *src_buf, *dst_buf; struct vicodec_q_data *q_out; + bool flushing; u32 state; src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx); @@ -227,26 +228,36 @@ static void device_run(void *priv) q_out = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT); state = VB2_BUF_STATE_DONE; - if (device_process(ctx, src_buf, dst_buf)) + + flushing = (src_buf == &ctx->flush_buf); + if (!flushing && device_process(ctx, src_buf, dst_buf)) state = VB2_BUF_STATE_ERROR; - ctx->last_dst_buf = dst_buf; spin_lock(ctx->lock); - if (!ctx->comp_has_next_frame && src_buf == ctx->last_src_buf) { + if (!flushing) { + if (!ctx->comp_has_next_frame && src_buf == ctx->last_src_buf) { + dst_buf->flags |= V4L2_BUF_FLAG_LAST; + v4l2_event_queue_fh(&ctx->fh, &eos_event); + } + + if (ctx->is_enc) { + src_buf->sequence = q_out->sequence++; + src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx); + v4l2_m2m_buf_done(src_buf, state); + } else if (vb2_get_plane_payload(&src_buf->vb2_buf, 0) + == ctx->cur_buf_offset) { + src_buf->sequence = q_out->sequence++; + src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx); + v4l2_m2m_buf_done(src_buf, state); + ctx->cur_buf_offset = 0; + ctx->comp_has_next_frame = false; + } + } else { + src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx); + vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0); dst_buf->flags |= V4L2_BUF_FLAG_LAST; v4l2_event_queue_fh(&ctx->fh, &eos_event); } - if (ctx->is_enc) { - src_buf->sequence = q_out->sequence++; - src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx); - v4l2_m2m_buf_done(src_buf, state); - } else if (vb2_get_plane_payload(&src_buf->vb2_buf, 0) == ctx->cur_buf_offset) { - src_buf->sequence = q_out->sequence++; - src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx); - v4l2_m2m_buf_done(src_buf, state); - ctx->cur_buf_offset = 0; - ctx->comp_has_next_frame = false; - } v4l2_m2m_buf_done(dst_buf, state); ctx->comp_size = 0; ctx->comp_magic_cnt = 0; @@ -293,6 +304,8 @@ static int job_ready(void *priv) src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx); if (!src_buf) return 0; + if (src_buf == &ctx->flush_buf) + return 1; p_out = vb2_plane_vaddr(&src_buf->vb2_buf, 0); sz = vb2_get_plane_payload(&src_buf->vb2_buf, 0); p = p_out + ctx->cur_buf_offset; @@ -770,21 +783,6 @@ static int vidioc_s_fmt_vid_out(struct file *file, void *priv, return ret; } -static void vicodec_mark_last_buf(struct vicodec_ctx *ctx) -{ - static const struct v4l2_event eos_event = { - .type = V4L2_EVENT_EOS - }; - - spin_lock(ctx->lock); - ctx->last_src_buf = v4l2_m2m_last_src_buf(ctx->fh.m2m_ctx); - if (!ctx->last_src_buf && ctx->last_dst_buf) { - ctx->last_dst_buf->flags |= V4L2_BUF_FLAG_LAST; - v4l2_event_queue_fh(&ctx->fh, &eos_event); - } - spin_unlock(ctx->lock); -} - static int vicodec_try_encoder_cmd(struct file *file, void *fh, struct v4l2_encoder_cmd *ec) { @@ -806,8 +804,8 @@ static int vicodec_encoder_cmd(struct file *file, void *fh, ret = vicodec_try_encoder_cmd(file, fh, ec); if (ret < 0) return ret; - - vicodec_mark_last_buf(ctx); + v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, &ctx->flush_buf); + v4l2_m2m_try_schedule(ctx->fh.m2m_ctx); return 0; } @@ -835,8 +833,8 @@ static int vicodec_decoder_cmd(struct file *file, void *fh, ret = vicodec_try_decoder_cmd(file, fh, dc); if (ret < 0) return ret; - - vicodec_mark_last_buf(ctx); + v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, &ctx->flush_buf); + v4l2_m2m_try_schedule(ctx->fh.m2m_ctx); return 0; } @@ -991,7 +989,7 @@ static void vicodec_return_bufs(struct vb2_queue *q, u32 state) vbuf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx); else vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx); - if (vbuf == NULL) + if (!vbuf || vbuf == &ctx->flush_buf) return; spin_lock(ctx->lock); v4l2_m2m_buf_done(vbuf, state); @@ -1031,7 +1029,6 @@ static int vicodec_start_streaming(struct vb2_queue *q, state->ref_frame.cb = state->ref_frame.luma + size; state->ref_frame.cr = state->ref_frame.cb + size / chroma_div; ctx->last_src_buf = NULL; - ctx->last_dst_buf = NULL; state->gop_cnt = 0; ctx->cur_buf_offset = 0; ctx->comp_size = 0; @@ -1158,6 +1155,7 @@ static int vicodec_open(struct file *file) struct vicodec_dev *dev = video_drvdata(file); struct vicodec_ctx *ctx = NULL; struct v4l2_ctrl_handler *hdl; + struct vb2_queue *vq; unsigned int size; int rc = 0; @@ -1234,6 +1232,16 @@ static int vicodec_open(struct file *file) v4l2_fh_add(&ctx->fh); + /* Setup a dummy flush buffer, used to signal + * encoding/decoding stop operation. When this buffer + * is queued to the OUTPUT queue, the driver will send + * V4L2_EVENT_EOS and send the last buffer to userspace. + */ + vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, multiplanar ? + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE : + V4L2_BUF_TYPE_VIDEO_OUTPUT); + ctx->flush_buf.vb2_buf.vb2_queue = vq; + open_unlock: mutex_unlock(vfd->lock); return rc;