From patchwork Mon Dec 9 08:38:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanimir Varbanov X-Patchwork-Id: 11278599 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 95A33112B for ; Mon, 9 Dec 2019 08:39:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 559F7207FD for ; Mon, 9 Dec 2019 08:39:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="Ei4mo98z" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727330AbfLIIjT (ORCPT ); Mon, 9 Dec 2019 03:39:19 -0500 Received: from mail-lf1-f67.google.com ([209.85.167.67]:39977 "EHLO mail-lf1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727299AbfLIIjS (ORCPT ); Mon, 9 Dec 2019 03:39:18 -0500 Received: by mail-lf1-f67.google.com with SMTP id y5so9995899lfy.7 for ; Mon, 09 Dec 2019 00:39:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=luQuoQiHyFLGPHFoUQ2T7meQcoSWdh7qwynN1/AVI+o=; b=Ei4mo98z/YBtmh4KvnpFOL1Yrazpxj+X+XpbMBwWfjgdiDdhJtLVCHcW6fJnW5iAvd H27DInbdx4eW2HNosf3UhYmyMxxYDypEe5aJK77Jh3c9yB09DAn/BvDm/Moep8Y7zpjI GuxGqggdUdJUYV2p1CxnNOawbe3bOUW8j/bo4jBvTx272ExNdGf6TPBM7jHmzQBGbAQP 64S6k3synHm5bi6GFEGLYPBG0njLq4sH6ncGJeCwmdPHNCgmQWQqfr1Bkt2DB214z/PL lls3Ycv3cZzOBx7Auiqn9535OhKMlHn2GmnLVuemjii/0agoDA3gdzzHAJJv8SdM0sT4 KdKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=luQuoQiHyFLGPHFoUQ2T7meQcoSWdh7qwynN1/AVI+o=; b=IdRLbufYw1adxpne9tUgDcU04xiXNLR0Wq8POtFepYuoPiqyIWX1jll+mh78xL56zx 3d9OD0ggeobHI3fFm2N3rpJA6qztnwCiLxdpZQIAEnd2am0lq3Sn9I0948X8UqfL/Gq2 DmOi7rB5AG2cjjIpFz4x2fKAMbw+wc1/7URrGU6erV75suMg9pB1PtUue2unoacnm3sc Y4IoAtm8Dy8wro0zefQmKY+X6Sr9CIBvIqYb57V7i5aSLkS+XD5nFJCTxA2lUCsMy53H sSgQ8jtNsDZvv223jaBI0rK+6kdevuQg9O1wOM2bSw74sddB21UXEZI5tHqfTIBetOmh ticA== X-Gm-Message-State: APjAAAX6LVmAZbs7ha+MYOVyL2TJtoh8GyTy+cwcYxuVGz2qnbS/ld21 HmPX+0O63bsWnQjADmJ3i0rSKASm9NU= X-Google-Smtp-Source: APXvYqzydkfOnplODtEK1VOBvjVKD5LzDO5B9Td01bPXy+bi/+B4ZcKE163pTGKpN63Fdbhc3iIlVg== X-Received: by 2002:a19:4b87:: with SMTP id y129mr8734703lfa.32.1575880753378; Mon, 09 Dec 2019 00:39:13 -0800 (PST) Received: from localhost.localdomain ([37.157.136.193]) by smtp.gmail.com with ESMTPSA id r26sm10438971lfm.82.2019.12.09.00.39.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2019 00:39:12 -0800 (PST) From: Stanimir Varbanov To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org Cc: Vikash Garodia , Tomasz Figa , Stanimir Varbanov Subject: [PATCH 1/6] venus: redesign clocks and pm domains control Date: Mon, 9 Dec 2019 10:38:19 +0200 Message-Id: <20191209083824.806-2-stanimir.varbanov@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191209083824.806-1-stanimir.varbanov@linaro.org> References: <20191209083824.806-1-stanimir.varbanov@linaro.org> Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Redesign core (vcodec) clock control to give the venus core more freedom to control them in order to make possible to use core selection feature on Venus IP v4. Move all clock and pmdomain functions in separate file and abstract power control with common operations per Venus IP version. Signed-off-by: Stanimir Varbanov --- drivers/media/platform/qcom/venus/Makefile | 2 +- drivers/media/platform/qcom/venus/core.c | 89 +- drivers/media/platform/qcom/venus/core.h | 5 + drivers/media/platform/qcom/venus/helpers.c | 321 +------ drivers/media/platform/qcom/venus/helpers.h | 3 - .../media/platform/qcom/venus/pm_helpers.c | 798 ++++++++++++++++++ .../media/platform/qcom/venus/pm_helpers.h | 39 + drivers/media/platform/qcom/venus/vdec.c | 69 +- drivers/media/platform/qcom/venus/venc.c | 65 +- 9 files changed, 911 insertions(+), 480 deletions(-) create mode 100644 drivers/media/platform/qcom/venus/pm_helpers.c create mode 100644 drivers/media/platform/qcom/venus/pm_helpers.h diff --git a/drivers/media/platform/qcom/venus/Makefile b/drivers/media/platform/qcom/venus/Makefile index b44b11b03e12..64af0bc1edae 100644 --- a/drivers/media/platform/qcom/venus/Makefile +++ b/drivers/media/platform/qcom/venus/Makefile @@ -3,7 +3,7 @@ venus-core-objs += core.o helpers.o firmware.o \ hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \ - hfi_parser.o + hfi_parser.o pm_helpers.o venus-dec-objs += vdec.o vdec_ctrls.o venus-enc-objs += venc.o venc_ctrls.o diff --git a/drivers/media/platform/qcom/venus/core.c b/drivers/media/platform/qcom/venus/core.c index 07312a2fab24..49d1e9c6481c 100644 --- a/drivers/media/platform/qcom/venus/core.c +++ b/drivers/media/platform/qcom/venus/core.c @@ -3,7 +3,6 @@ * Copyright (c) 2012-2016, The Linux Foundation. All rights reserved. * Copyright (C) 2017 Linaro Ltd. */ -#include #include #include #include @@ -19,9 +18,8 @@ #include #include "core.h" -#include "vdec.h" -#include "venc.h" #include "firmware.h" +#include "pm_helpers.h" static void venus_event_notify(struct venus_core *core, u32 event) { @@ -100,50 +98,6 @@ static void venus_sys_error_handler(struct work_struct *work) mutex_unlock(&core->lock); } -static int venus_clks_get(struct venus_core *core) -{ - const struct venus_resources *res = core->res; - struct device *dev = core->dev; - unsigned int i; - - for (i = 0; i < res->clks_num; i++) { - core->clks[i] = devm_clk_get(dev, res->clks[i]); - if (IS_ERR(core->clks[i])) - return PTR_ERR(core->clks[i]); - } - - return 0; -} - -static int venus_clks_enable(struct venus_core *core) -{ - const struct venus_resources *res = core->res; - unsigned int i; - int ret; - - for (i = 0; i < res->clks_num; i++) { - ret = clk_prepare_enable(core->clks[i]); - if (ret) - goto err; - } - - return 0; -err: - while (i--) - clk_disable_unprepare(core->clks[i]); - - return ret; -} - -static void venus_clks_disable(struct venus_core *core) -{ - const struct venus_resources *res = core->res; - unsigned int i = res->clks_num; - - while (i--) - clk_disable_unprepare(core->clks[i]); -} - static u32 to_v4l2_codec_type(u32 codec) { switch (codec) { @@ -256,9 +210,15 @@ static int venus_probe(struct platform_device *pdev) if (!core->res) return -ENODEV; - ret = venus_clks_get(core); - if (ret) - return ret; + core->pm_ops = venus_pm_get(core->res->hfi_version); + if (!core->pm_ops) + return -ENODEV; + + if (core->pm_ops->core_get) { + ret = core->pm_ops->core_get(dev); + if (ret) + return ret; + } ret = dma_set_mask_and_coherent(dev, core->res->dma_mask); if (ret) @@ -350,6 +310,7 @@ static int venus_probe(struct platform_device *pdev) static int venus_remove(struct platform_device *pdev) { struct venus_core *core = platform_get_drvdata(pdev); + const struct venus_pm_ops *pm_ops = core->pm_ops; struct device *dev = core->dev; int ret; @@ -368,6 +329,9 @@ static int venus_remove(struct platform_device *pdev) pm_runtime_put_sync(dev); pm_runtime_disable(dev); + if (pm_ops->core_put) + pm_ops->core_put(dev); + icc_put(core->video_path); icc_put(core->cpucfg_path); @@ -379,11 +343,15 @@ static int venus_remove(struct platform_device *pdev) static __maybe_unused int venus_runtime_suspend(struct device *dev) { struct venus_core *core = dev_get_drvdata(dev); + const struct venus_pm_ops *pm_ops = core->pm_ops; int ret; ret = hfi_core_suspend(core); + if (ret) + return ret; - venus_clks_disable(core); + if (pm_ops->core_power) + ret = pm_ops->core_power(dev, POWER_OFF); return ret; } @@ -391,21 +359,16 @@ static __maybe_unused int venus_runtime_suspend(struct device *dev) static __maybe_unused int venus_runtime_resume(struct device *dev) { struct venus_core *core = dev_get_drvdata(dev); + const struct venus_pm_ops *pm_ops = core->pm_ops; int ret; - ret = venus_clks_enable(core); - if (ret) - return ret; - - ret = hfi_core_resume(core, false); - if (ret) - goto err_clks_disable; - - return 0; + if (pm_ops->core_power) { + ret = pm_ops->core_power(dev, POWER_ON); + if (ret) + return ret; + } -err_clks_disable: - venus_clks_disable(core); - return ret; + return hfi_core_resume(core, false); } static const struct dev_pm_ops venus_pm_ops = { diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h index 11585fb3cae3..179ff861a618 100644 --- a/drivers/media/platform/qcom/venus/core.h +++ b/drivers/media/platform/qcom/venus/core.h @@ -138,6 +138,10 @@ struct venus_core { struct clk *core1_bus_clk; struct icc_path *video_path; struct icc_path *cpucfg_path; + struct device *pd_core; + struct device_link *pd_dl_venus; + struct device *pd_core0; + struct device *pd_core1; struct video_device *vdev_dec; struct video_device *vdev_enc; struct v4l2_device v4l2_dev; @@ -159,6 +163,7 @@ struct venus_core { unsigned int error; bool sys_error; const struct hfi_core_ops *core_ops; + const struct venus_pm_ops *pm_ops; unsigned long enc_codecs; unsigned long dec_codecs; unsigned int max_sessions_supported; diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c index a172f1ac0b35..cbeeaf6077e0 100644 --- a/drivers/media/platform/qcom/venus/helpers.c +++ b/drivers/media/platform/qcom/venus/helpers.c @@ -3,12 +3,8 @@ * Copyright (c) 2012-2016, The Linux Foundation. All rights reserved. * Copyright (C) 2017 Linaro Ltd. */ -#include -#include -#include #include #include -#include #include #include #include @@ -17,7 +13,7 @@ #include "core.h" #include "helpers.h" #include "hfi_helper.h" -#include "hfi_venus_io.h" +#include "pm_helpers.h" struct intbuf { struct list_head list; @@ -360,266 +356,6 @@ int venus_helper_intbufs_realloc(struct venus_inst *inst) } EXPORT_SYMBOL_GPL(venus_helper_intbufs_realloc); -static u32 load_per_instance(struct venus_inst *inst) -{ - u32 mbs; - - if (!inst || !(inst->state >= INST_INIT && inst->state < INST_STOP)) - return 0; - - mbs = (ALIGN(inst->width, 16) / 16) * (ALIGN(inst->height, 16) / 16); - - return mbs * inst->fps; -} - -static u32 load_per_type(struct venus_core *core, u32 session_type) -{ - struct venus_inst *inst = NULL; - u32 mbs_per_sec = 0; - - mutex_lock(&core->lock); - list_for_each_entry(inst, &core->instances, list) { - if (inst->session_type != session_type) - continue; - - mbs_per_sec += load_per_instance(inst); - } - mutex_unlock(&core->lock); - - return mbs_per_sec; -} - -static void mbs_to_bw(struct venus_inst *inst, u32 mbs, u32 *avg, u32 *peak) -{ - const struct venus_resources *res = inst->core->res; - const struct bw_tbl *bw_tbl; - unsigned int num_rows, i; - - *avg = 0; - *peak = 0; - - if (mbs == 0) - return; - - if (inst->session_type == VIDC_SESSION_TYPE_ENC) { - num_rows = res->bw_tbl_enc_size; - bw_tbl = res->bw_tbl_enc; - } else if (inst->session_type == VIDC_SESSION_TYPE_DEC) { - num_rows = res->bw_tbl_dec_size; - bw_tbl = res->bw_tbl_dec; - } else { - return; - } - - if (!bw_tbl || num_rows == 0) - return; - - for (i = 0; i < num_rows; i++) { - if (mbs > bw_tbl[i].mbs_per_sec) - break; - - if (inst->dpb_fmt & HFI_COLOR_FORMAT_10_BIT_BASE) { - *avg = bw_tbl[i].avg_10bit; - *peak = bw_tbl[i].peak_10bit; - } else { - *avg = bw_tbl[i].avg; - *peak = bw_tbl[i].peak; - } - } -} - -static int load_scale_bw(struct venus_core *core) -{ - struct venus_inst *inst = NULL; - u32 mbs_per_sec, avg, peak, total_avg = 0, total_peak = 0; - - mutex_lock(&core->lock); - list_for_each_entry(inst, &core->instances, list) { - mbs_per_sec = load_per_instance(inst); - mbs_to_bw(inst, mbs_per_sec, &avg, &peak); - total_avg += avg; - total_peak += peak; - } - mutex_unlock(&core->lock); - - dev_dbg(core->dev, "total: avg_bw: %u, peak_bw: %u\n", - total_avg, total_peak); - - return icc_set_bw(core->video_path, total_avg, total_peak); -} - -static int set_clk_freq(struct venus_core *core, unsigned long freq) -{ - struct clk *clk = core->clks[0]; - int ret; - - ret = clk_set_rate(clk, freq); - if (ret) - return ret; - - ret = clk_set_rate(core->core0_clk, freq); - if (ret) - return ret; - - ret = clk_set_rate(core->core1_clk, freq); - if (ret) - return ret; - - return 0; -} - -static int scale_clocks(struct venus_inst *inst) -{ - struct venus_core *core = inst->core; - const struct freq_tbl *table = core->res->freq_tbl; - unsigned int num_rows = core->res->freq_tbl_size; - unsigned long freq = table[0].freq; - struct device *dev = core->dev; - u32 mbs_per_sec; - unsigned int i; - int ret; - - mbs_per_sec = load_per_type(core, VIDC_SESSION_TYPE_ENC) + - load_per_type(core, VIDC_SESSION_TYPE_DEC); - - if (mbs_per_sec > core->res->max_load) - dev_warn(dev, "HW is overloaded, needed: %d max: %d\n", - mbs_per_sec, core->res->max_load); - - if (!mbs_per_sec && num_rows > 1) { - freq = table[num_rows - 1].freq; - goto set_freq; - } - - for (i = 0; i < num_rows; i++) { - if (mbs_per_sec > table[i].load) - break; - freq = table[i].freq; - } - -set_freq: - - ret = set_clk_freq(core, freq); - if (ret) { - dev_err(dev, "failed to set clock rate %lu (%d)\n", - freq, ret); - return ret; - } - - ret = load_scale_bw(core); - if (ret) { - dev_err(dev, "failed to set bandwidth (%d)\n", - ret); - return ret; - } - - return 0; -} - -static unsigned long calculate_inst_freq(struct venus_inst *inst, - unsigned long filled_len) -{ - unsigned long vpp_freq = 0, vsp_freq = 0; - u32 fps = (u32)inst->fps; - u32 mbs_per_sec; - - mbs_per_sec = load_per_instance(inst) / fps; - - vpp_freq = mbs_per_sec * inst->clk_data.codec_freq_data->vpp_freq; - /* 21 / 20 is overhead factor */ - vpp_freq += vpp_freq / 20; - vsp_freq = mbs_per_sec * inst->clk_data.codec_freq_data->vsp_freq; - - /* 10 / 7 is overhead factor */ - if (inst->session_type == VIDC_SESSION_TYPE_ENC) - vsp_freq += (inst->controls.enc.bitrate * 10) / 7; - else - vsp_freq += ((fps * filled_len * 8) * 10) / 7; - - return max(vpp_freq, vsp_freq); -} - -static int scale_clocks_v4(struct venus_inst *inst) -{ - struct venus_core *core = inst->core; - const struct freq_tbl *table = core->res->freq_tbl; - unsigned int num_rows = core->res->freq_tbl_size; - struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; - struct device *dev = core->dev; - unsigned long freq = 0, freq_core1 = 0, freq_core2 = 0; - unsigned long filled_len = 0; - struct venus_buffer *buf, *n; - struct vb2_buffer *vb; - int i, ret; - - v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buf, n) { - vb = &buf->vb.vb2_buf; - filled_len = max(filled_len, vb2_get_plane_payload(vb, 0)); - } - - if (inst->session_type == VIDC_SESSION_TYPE_DEC && !filled_len) - return 0; - - freq = calculate_inst_freq(inst, filled_len); - inst->clk_data.freq = freq; - - mutex_lock(&core->lock); - list_for_each_entry(inst, &core->instances, list) { - if (inst->clk_data.core_id == VIDC_CORE_ID_1) { - freq_core1 += inst->clk_data.freq; - } else if (inst->clk_data.core_id == VIDC_CORE_ID_2) { - freq_core2 += inst->clk_data.freq; - } else if (inst->clk_data.core_id == VIDC_CORE_ID_3) { - freq_core1 += inst->clk_data.freq; - freq_core2 += inst->clk_data.freq; - } - } - mutex_unlock(&core->lock); - - freq = max(freq_core1, freq_core2); - - if (freq >= table[0].freq) { - freq = table[0].freq; - dev_warn(dev, "HW is overloaded, needed: %lu max: %lu\n", - freq, table[0].freq); - goto set_freq; - } - - for (i = num_rows - 1 ; i >= 0; i--) { - if (freq <= table[i].freq) { - freq = table[i].freq; - break; - } - } - -set_freq: - - ret = set_clk_freq(core, freq); - if (ret) { - dev_err(dev, "failed to set clock rate %lu (%d)\n", - freq, ret); - return ret; - } - - ret = load_scale_bw(core); - if (ret) { - dev_err(dev, "failed to set bandwidth (%d)\n", - ret); - return ret; - } - - return 0; -} - -int venus_helper_load_scale_clocks(struct venus_inst *inst) -{ - if (IS_V4(inst->core)) - return scale_clocks_v4(inst); - - return scale_clocks(inst); -} -EXPORT_SYMBOL_GPL(venus_helper_load_scale_clocks); - static void fill_buffer_desc(const struct venus_buffer *buf, struct hfi_buffer_desc *bd, bool response) { @@ -723,7 +459,7 @@ session_process_buf(struct venus_inst *inst, struct vb2_v4l2_buffer *vbuf) if (inst->session_type == VIDC_SESSION_TYPE_DEC) put_ts_metadata(inst, vbuf); - venus_helper_load_scale_clocks(inst); + venus_pm_load_scale(inst); } else if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) { if (inst->session_type == VIDC_SESSION_TYPE_ENC) fdata.buffer_type = HFI_BUFFER_OUTPUT; @@ -1354,7 +1090,7 @@ void venus_helper_vb2_stop_streaming(struct vb2_queue *q) venus_helper_free_dpb_bufs(inst); - venus_helper_load_scale_clocks(inst); + venus_pm_load_scale(inst); INIT_LIST_HEAD(&inst->registeredbufs); } @@ -1417,7 +1153,7 @@ int venus_helper_vb2_start_streaming(struct venus_inst *inst) if (ret) goto err_bufs_free; - venus_helper_load_scale_clocks(inst); + venus_pm_load_scale(inst); ret = hfi_session_load_res(inst); if (ret) @@ -1542,52 +1278,3 @@ int venus_helper_get_out_fmts(struct venus_inst *inst, u32 v4l2_fmt, return -EINVAL; } EXPORT_SYMBOL_GPL(venus_helper_get_out_fmts); - -int venus_helper_power_enable(struct venus_core *core, u32 session_type, - bool enable) -{ - void __iomem *ctrl, *stat; - u32 val; - int ret; - - if (!IS_V3(core) && !IS_V4(core)) - return 0; - - if (IS_V3(core)) { - if (session_type == VIDC_SESSION_TYPE_DEC) - ctrl = core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL; - else - ctrl = core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL; - if (enable) - writel(0, ctrl); - else - writel(1, ctrl); - - return 0; - } - - if (session_type == VIDC_SESSION_TYPE_DEC) { - ctrl = core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL; - stat = core->base + WRAPPER_VCODEC0_MMCC_POWER_STATUS; - } else { - ctrl = core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL; - stat = core->base + WRAPPER_VCODEC1_MMCC_POWER_STATUS; - } - - if (enable) { - writel(0, ctrl); - - ret = readl_poll_timeout(stat, val, val & BIT(1), 1, 100); - if (ret) - return ret; - } else { - writel(1, ctrl); - - ret = readl_poll_timeout(stat, val, !(val & BIT(1)), 1, 100); - if (ret) - return ret; - } - - return 0; -} -EXPORT_SYMBOL_GPL(venus_helper_power_enable); diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h index 34dcd0c13f06..44b50914cd12 100644 --- a/drivers/media/platform/qcom/venus/helpers.h +++ b/drivers/media/platform/qcom/venus/helpers.h @@ -53,14 +53,11 @@ int venus_helper_get_out_fmts(struct venus_inst *inst, u32 fmt, u32 *out_fmt, u32 *out2_fmt, bool ubwc); int venus_helper_alloc_dpb_bufs(struct venus_inst *inst); int venus_helper_free_dpb_bufs(struct venus_inst *inst); -int venus_helper_power_enable(struct venus_core *core, u32 session_type, - bool enable); int venus_helper_intbufs_alloc(struct venus_inst *inst); int venus_helper_intbufs_free(struct venus_inst *inst); int venus_helper_intbufs_realloc(struct venus_inst *inst); int venus_helper_queue_dpb_bufs(struct venus_inst *inst); int venus_helper_unregister_bufs(struct venus_inst *inst); -int venus_helper_load_scale_clocks(struct venus_inst *inst); int venus_helper_process_initial_cap_bufs(struct venus_inst *inst); int venus_helper_process_initial_out_bufs(struct venus_inst *inst); void venus_helper_get_ts_metadata(struct venus_inst *inst, u64 timestamp_us, diff --git a/drivers/media/platform/qcom/venus/pm_helpers.c b/drivers/media/platform/qcom/venus/pm_helpers.c new file mode 100644 index 000000000000..4e4aace2d8c8 --- /dev/null +++ b/drivers/media/platform/qcom/venus/pm_helpers.c @@ -0,0 +1,798 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2019 Linaro Ltd. + * + * Author: Stanimir Varbanov + */ +#include +#include +#include +#include +#include +#include +#include +#include + +#include "core.h" +#include "hfi_venus_io.h" +#include "pm_helpers.h" + +static bool legacy_binding; + +static int core_clks_get(struct venus_core *core) +{ + const struct venus_resources *res = core->res; + struct device *dev = core->dev; + unsigned int i; + + for (i = 0; i < res->clks_num; i++) { + core->clks[i] = devm_clk_get(dev, res->clks[i]); + if (IS_ERR(core->clks[i])) + return PTR_ERR(core->clks[i]); + } + + return 0; +} + +static int core_clks_enable(struct venus_core *core) +{ + const struct venus_resources *res = core->res; + unsigned int i; + int ret; + + for (i = 0; i < res->clks_num; i++) { + ret = clk_prepare_enable(core->clks[i]); + if (ret) + goto err; + } + + return 0; +err: + while (i--) + clk_disable_unprepare(core->clks[i]); + + return ret; +} + +static void core_clks_disable(struct venus_core *core) +{ + const struct venus_resources *res = core->res; + unsigned int i = res->clks_num; + + while (i--) + clk_disable_unprepare(core->clks[i]); +} + +static int core_clks_set_rate(struct venus_core *core, unsigned long freq) +{ + struct clk *clk = core->clks[0]; + int ret; + + ret = clk_set_rate(clk, freq); + if (ret) + return ret; + + ret = clk_set_rate(core->core0_clk, freq); + if (ret) + return ret; + + ret = clk_set_rate(core->core1_clk, freq); + if (ret) + return ret; + + return 0; +} + +static u32 load_per_instance(struct venus_inst *inst) +{ + u32 mbs; + + if (!inst || !(inst->state >= INST_INIT && inst->state < INST_STOP)) + return 0; + + mbs = (ALIGN(inst->width, 16) / 16) * (ALIGN(inst->height, 16) / 16); + + return mbs * inst->fps; +} + +static u32 load_per_type(struct venus_core *core, u32 session_type) +{ + struct venus_inst *inst = NULL; + u32 mbs_per_sec = 0; + + mutex_lock(&core->lock); + list_for_each_entry(inst, &core->instances, list) { + if (inst->session_type != session_type) + continue; + + mbs_per_sec += load_per_instance(inst); + } + mutex_unlock(&core->lock); + + return mbs_per_sec; +} + +static void mbs_to_bw(struct venus_inst *inst, u32 mbs, u32 *avg, u32 *peak) +{ + const struct venus_resources *res = inst->core->res; + const struct bw_tbl *bw_tbl; + unsigned int num_rows, i; + + *avg = 0; + *peak = 0; + + if (mbs == 0) + return; + + if (inst->session_type == VIDC_SESSION_TYPE_ENC) { + num_rows = res->bw_tbl_enc_size; + bw_tbl = res->bw_tbl_enc; + } else if (inst->session_type == VIDC_SESSION_TYPE_DEC) { + num_rows = res->bw_tbl_dec_size; + bw_tbl = res->bw_tbl_dec; + } else { + return; + } + + if (!bw_tbl || num_rows == 0) + return; + + for (i = 0; i < num_rows; i++) { + if (mbs > bw_tbl[i].mbs_per_sec) + break; + + if (inst->dpb_fmt & HFI_COLOR_FORMAT_10_BIT_BASE) { + *avg = bw_tbl[i].avg_10bit; + *peak = bw_tbl[i].peak_10bit; + } else { + *avg = bw_tbl[i].avg; + *peak = bw_tbl[i].peak; + } + } +} + +static int load_scale_bw(struct venus_core *core) +{ + struct venus_inst *inst = NULL; + u32 mbs_per_sec, avg, peak, total_avg = 0, total_peak = 0; + + mutex_lock(&core->lock); + list_for_each_entry(inst, &core->instances, list) { + mbs_per_sec = load_per_instance(inst); + mbs_to_bw(inst, mbs_per_sec, &avg, &peak); + total_avg += avg; + total_peak += peak; + } + mutex_unlock(&core->lock); + + dev_dbg(core->dev, "total: avg_bw: %u, peak_bw: %u\n", + total_avg, total_peak); + + return icc_set_bw(core->video_path, total_avg, total_peak); +} + +static int load_scale_v1(struct venus_inst *inst) +{ + struct venus_core *core = inst->core; + const struct freq_tbl *table = core->res->freq_tbl; + unsigned int num_rows = core->res->freq_tbl_size; + unsigned long freq = table[0].freq; + struct device *dev = core->dev; + u32 mbs_per_sec; + unsigned int i; + int ret; + + mbs_per_sec = load_per_type(core, VIDC_SESSION_TYPE_ENC) + + load_per_type(core, VIDC_SESSION_TYPE_DEC); + + if (mbs_per_sec > core->res->max_load) + dev_warn(dev, "HW is overloaded, needed: %d max: %d\n", + mbs_per_sec, core->res->max_load); + + if (!mbs_per_sec && num_rows > 1) { + freq = table[num_rows - 1].freq; + goto set_freq; + } + + for (i = 0; i < num_rows; i++) { + if (mbs_per_sec > table[i].load) + break; + freq = table[i].freq; + } + +set_freq: + + ret = core_clks_set_rate(core, freq); + if (ret) { + dev_err(dev, "failed to set clock rate %lu (%d)\n", + freq, ret); + return ret; + } + + ret = load_scale_bw(core); + if (ret) { + dev_err(dev, "failed to set bandwidth (%d)\n", + ret); + return ret; + } + + return 0; +} + +static int core_get_v1(struct device *dev) +{ + struct venus_core *core = dev_get_drvdata(dev); + + return core_clks_get(core); +} + +static int core_power_v1(struct device *dev, int on) +{ + struct venus_core *core = dev_get_drvdata(dev); + int ret = 0; + + if (on == POWER_ON) + ret = core_clks_enable(core); + else + core_clks_disable(core); + + return ret; +} + +static const struct venus_pm_ops pm_ops_v1 = { + .core_get = core_get_v1, + .core_power = core_power_v1, + .load_scale = load_scale_v1, +}; + +static void +vcodec_control_v3(struct venus_core *core, u32 session_type, bool enable) +{ + void __iomem *ctrl; + + if (session_type == VIDC_SESSION_TYPE_DEC) + ctrl = core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL; + else + ctrl = core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL; + + if (enable) + writel(0, ctrl); + else + writel(1, ctrl); +} + +static int vdec_get_v3(struct device *dev) +{ + struct venus_core *core = dev_get_drvdata(dev); + + core->core0_clk = devm_clk_get(dev, "core"); + if (IS_ERR(core->core0_clk)) + return PTR_ERR(core->core0_clk); + + return 0; +} + +static int vdec_power_v3(struct device *dev, int on) +{ + struct venus_core *core = dev_get_drvdata(dev); + int ret = 0; + + vcodec_control_v3(core, VIDC_SESSION_TYPE_DEC, true); + + if (on == POWER_ON) + ret = clk_prepare_enable(core->core0_clk); + else + clk_disable_unprepare(core->core0_clk); + + vcodec_control_v3(core, VIDC_SESSION_TYPE_DEC, false); + + return ret; +} + +static int venc_get_v3(struct device *dev) +{ + struct venus_core *core = dev_get_drvdata(dev); + + core->core1_clk = devm_clk_get(dev, "core"); + if (IS_ERR(core->core1_clk)) + return PTR_ERR(core->core1_clk); + + return 0; +} + +static int venc_power_v3(struct device *dev, int on) +{ + struct venus_core *core = dev_get_drvdata(dev); + int ret = 0; + + vcodec_control_v3(core, VIDC_SESSION_TYPE_ENC, true); + + if (on == POWER_ON) + ret = clk_prepare_enable(core->core1_clk); + else + clk_disable_unprepare(core->core1_clk); + + vcodec_control_v3(core, VIDC_SESSION_TYPE_ENC, false); + + return ret; +} + +static const struct venus_pm_ops pm_ops_v3 = { + .core_get = core_get_v1, + .core_power = core_power_v1, + .vdec_get = vdec_get_v3, + .vdec_power = vdec_power_v3, + .venc_get = venc_get_v3, + .venc_power = venc_power_v3, + .load_scale = load_scale_v1, +}; + +static int vcodec_control_v4(struct venus_core *core, u32 coreid, bool enable) +{ + void __iomem *ctrl, *stat; + u32 val; + int ret; + + if (coreid == VIDC_CORE_ID_1) { + ctrl = core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL; + stat = core->base + WRAPPER_VCODEC0_MMCC_POWER_STATUS; + } else { + ctrl = core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL; + stat = core->base + WRAPPER_VCODEC1_MMCC_POWER_STATUS; + } + + if (enable) { + writel(0, ctrl); + + ret = readl_poll_timeout(stat, val, val & BIT(1), 1, 100); + if (ret) + return ret; + } else { + writel(1, ctrl); + + ret = readl_poll_timeout(stat, val, !(val & BIT(1)), 1, 100); + if (ret) + return ret; + } + + return 0; +} + +static int poweroff_coreid(struct venus_core *core, unsigned int coreid_mask) +{ + int ret; + + if (coreid_mask & VIDC_CORE_ID_1) { + ret = vcodec_control_v4(core, VIDC_CORE_ID_1, true); + if (ret) + return ret; + + clk_disable_unprepare(core->core0_bus_clk); + clk_disable_unprepare(core->core0_clk); + + ret = vcodec_control_v4(core, VIDC_CORE_ID_1, false); + if (ret) + return ret; + + ret = pm_runtime_put_sync(core->pd_core0); + if (ret < 0) + return ret; + } + + if (coreid_mask & VIDC_CORE_ID_2) { + ret = vcodec_control_v4(core, VIDC_CORE_ID_2, true); + if (ret) + return ret; + + clk_disable_unprepare(core->core1_bus_clk); + clk_disable_unprepare(core->core1_clk); + + ret = vcodec_control_v4(core, VIDC_CORE_ID_2, false); + if (ret) + return ret; + + ret = pm_runtime_put_sync(core->pd_core1); + if (ret < 0) + return ret; + } + + return 0; +} + +static int poweron_coreid(struct venus_core *core, unsigned int coreid_mask) +{ + int ret; + + if (coreid_mask & VIDC_CORE_ID_1) { + ret = pm_runtime_get_sync(core->pd_core0); + if (ret < 0) + return ret; + + ret = vcodec_control_v4(core, VIDC_CORE_ID_1, true); + if (ret) + return ret; + + ret = clk_prepare_enable(core->core0_clk); + if (ret) + return ret; + + ret = clk_prepare_enable(core->core0_bus_clk); + if (ret) + return ret; + + ret = vcodec_control_v4(core, VIDC_CORE_ID_1, false); + if (ret < 0) + return ret; + } + + if (coreid_mask & VIDC_CORE_ID_2) { + ret = pm_runtime_get_sync(core->pd_core1); + if (ret < 0) + return ret; + + ret = vcodec_control_v4(core, VIDC_CORE_ID_2, true); + if (ret) + return ret; + + ret = clk_prepare_enable(core->core1_clk); + if (ret) + return ret; + + ret = clk_prepare_enable(core->core1_bus_clk); + if (ret) + return ret; + + ret = vcodec_control_v4(core, VIDC_CORE_ID_2, false); + if (ret < 0) + return ret; + } + + return 0; +} + +static int vdec_get_v4(struct device *dev) +{ + struct venus_core *core = dev_get_drvdata(dev); + + if (!legacy_binding) + return 0; + + core->core0_clk = devm_clk_get(dev, "core"); + if (IS_ERR(core->core0_clk)) + return PTR_ERR(core->core0_clk); + + core->core0_bus_clk = devm_clk_get(dev, "bus"); + if (IS_ERR(core->core0_bus_clk)) + return PTR_ERR(core->core0_bus_clk); + + return 0; +} + +static void vdec_put_v4(struct device *dev) +{ + struct venus_core *core = dev_get_drvdata(dev); + + if (!legacy_binding) + return; + + core->core0_clk = NULL; + core->core0_bus_clk = NULL; +} + +static int vdec_power_v4(struct device *dev, int on) +{ + struct venus_core *core = dev_get_drvdata(dev); + int ret; + + if (!legacy_binding) + return 0; + + ret = vcodec_control_v4(core, VIDC_CORE_ID_1, true); + if (ret) + return ret; + + if (on == POWER_ON) { + ret = clk_prepare_enable(core->core0_clk); + if (ret) + goto disable; + + ret = clk_prepare_enable(core->core0_bus_clk); + } else { + clk_disable_unprepare(core->core0_bus_clk); + clk_disable_unprepare(core->core0_clk); + } + +disable: + vcodec_control_v4(core, VIDC_CORE_ID_1, false); + + return ret; +} + +static int venc_get_v4(struct device *dev) +{ + struct venus_core *core = dev_get_drvdata(dev); + + if (!legacy_binding) + return 0; + + core->core1_clk = devm_clk_get(dev, "core"); + if (IS_ERR(core->core1_clk)) + return PTR_ERR(core->core1_clk); + + core->core1_bus_clk = devm_clk_get(dev, "bus"); + if (IS_ERR(core->core1_bus_clk)) + return PTR_ERR(core->core1_bus_clk); + + return 0; +} + +static void venc_put_v4(struct device *dev) +{ + struct venus_core *core = dev_get_drvdata(dev); + + if (!legacy_binding) + return; + + core->core1_clk = NULL; + core->core1_bus_clk = NULL; +} + +static int venc_power_v4(struct device *dev, int on) +{ + struct venus_core *core = dev_get_drvdata(dev); + int ret; + + if (!legacy_binding) + return 0; + + ret = vcodec_control_v4(core, VIDC_CORE_ID_2, true); + if (ret) + return ret; + + if (on == POWER_ON) { + ret = clk_prepare_enable(core->core1_clk); + if (ret) + goto disable; + + ret = clk_prepare_enable(core->core1_bus_clk); + } else { + clk_disable_unprepare(core->core1_bus_clk); + clk_disable_unprepare(core->core1_clk); + } + +disable: + vcodec_control_v4(core, VIDC_CORE_ID_2, false); + + return ret; +} + +static int vcodec_clks_domains(struct device *dev) +{ + struct venus_core *core = dev_get_drvdata(dev); + + core->core0_clk = devm_clk_get(dev, "vcodec0_core"); + if (IS_ERR(core->core0_clk)) + return PTR_ERR(core->core0_clk); + + core->core0_bus_clk = devm_clk_get(dev, "vcodec0_bus"); + if (IS_ERR(core->core0_bus_clk)) + return PTR_ERR(core->core0_bus_clk); + + core->core1_clk = devm_clk_get(dev, "vcodec1_core"); + if (IS_ERR(core->core1_clk)) + return PTR_ERR(core->core1_clk); + + core->core1_bus_clk = devm_clk_get(dev, "vcodec1_bus"); + if (IS_ERR(core->core1_bus_clk)) + return PTR_ERR(core->core1_bus_clk); + + core->pd_core = dev_pm_domain_attach_by_name(dev, "venus"); + if (IS_ERR(core->pd_core)) + return PTR_ERR(core->pd_core); + + core->pd_core0 = dev_pm_domain_attach_by_name(dev, "vcodec0"); + if (IS_ERR(core->pd_core0)) + return PTR_ERR(core->pd_core0); + + core->pd_core1 = dev_pm_domain_attach_by_name(dev, "vcodec1"); + if (IS_ERR(core->pd_core1)) + return PTR_ERR(core->pd_core1); + + core->pd_dl_venus = device_link_add(dev, core->pd_core, + DL_FLAG_PM_RUNTIME | + DL_FLAG_STATELESS | + DL_FLAG_RPM_ACTIVE); + if (!core->pd_dl_venus) + return -ENODEV; + + return 0; +} + +static int core_get_v4(struct device *dev) +{ + struct venus_core *core = dev_get_drvdata(dev); + int ret; + + ret = core_clks_get(core); + if (ret) + return ret; + + ret = vcodec_clks_domains(dev); + if (ret == -EPROBE_DEFER) { + return ret; + } else if (ret) { + core->core0_clk = NULL; + core->core0_bus_clk = NULL; + core->core1_clk = NULL; + core->core1_bus_clk = NULL; + legacy_binding = true; + } + + return 0; +} + +static void core_put_v4(struct device *dev) +{ + struct venus_core *core = dev_get_drvdata(dev); + + if (legacy_binding) + return; + + if (core->pd_dl_venus) + device_link_del(core->pd_dl_venus); + + if (!IS_ERR_OR_NULL(core->pd_core)) + dev_pm_domain_detach(core->pd_core, true); + + if (!IS_ERR_OR_NULL(core->pd_core0)) + dev_pm_domain_detach(core->pd_core0, true); + + if (!IS_ERR_OR_NULL(core->pd_core1)) + dev_pm_domain_detach(core->pd_core1, true); +} + +static int core_power_v4(struct device *dev, int on) +{ + struct venus_core *core = dev_get_drvdata(dev); + const unsigned int coreid_mask = VIDC_CORE_ID_1 | VIDC_CORE_ID_2; + int ret = 0; + + if (on == POWER_ON) { + ret = core_clks_enable(core); + if (ret) + return ret; + + if (!legacy_binding) + ret = poweron_coreid(core, coreid_mask); + } else { + if (!legacy_binding) + ret = poweroff_coreid(core, coreid_mask); + + core_clks_disable(core); + } + + return ret; +} + +static unsigned long calculate_inst_freq(struct venus_inst *inst, + unsigned long filled_len) +{ + unsigned long vpp_freq = 0, vsp_freq = 0; + u32 fps = (u32)inst->fps; + u32 mbs_per_sec; + + mbs_per_sec = load_per_instance(inst) / fps; + + vpp_freq = mbs_per_sec * inst->clk_data.codec_freq_data->vpp_freq; + /* 21 / 20 is overhead factor */ + vpp_freq += vpp_freq / 20; + vsp_freq = mbs_per_sec * inst->clk_data.codec_freq_data->vsp_freq; + + /* 10 / 7 is overhead factor */ + if (inst->session_type == VIDC_SESSION_TYPE_ENC) + vsp_freq += (inst->controls.enc.bitrate * 10) / 7; + else + vsp_freq += ((fps * filled_len * 8) * 10) / 7; + + return max(vpp_freq, vsp_freq); +} + +static int load_scale_v4(struct venus_inst *inst) +{ + struct venus_core *core = inst->core; + const struct freq_tbl *table = core->res->freq_tbl; + unsigned int num_rows = core->res->freq_tbl_size; + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct device *dev = core->dev; + unsigned long freq = 0, freq_core1 = 0, freq_core2 = 0; + unsigned long filled_len = 0; + struct venus_buffer *buf, *n; + struct vb2_buffer *vb; + int i, ret; + + v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buf, n) { + vb = &buf->vb.vb2_buf; + filled_len = max(filled_len, vb2_get_plane_payload(vb, 0)); + } + + if (inst->session_type == VIDC_SESSION_TYPE_DEC && !filled_len) + return 0; + + freq = calculate_inst_freq(inst, filled_len); + inst->clk_data.freq = freq; + + mutex_lock(&core->lock); + list_for_each_entry(inst, &core->instances, list) { + if (inst->clk_data.core_id == VIDC_CORE_ID_1) { + freq_core1 += inst->clk_data.freq; + } else if (inst->clk_data.core_id == VIDC_CORE_ID_2) { + freq_core2 += inst->clk_data.freq; + } else if (inst->clk_data.core_id == VIDC_CORE_ID_3) { + freq_core1 += inst->clk_data.freq; + freq_core2 += inst->clk_data.freq; + } + } + mutex_unlock(&core->lock); + + freq = max(freq_core1, freq_core2); + + if (freq >= table[0].freq) { + freq = table[0].freq; + dev_warn(dev, "HW is overloaded, needed: %lu max: %lu\n", + freq, table[0].freq); + goto set_freq; + } + + for (i = num_rows - 1 ; i >= 0; i--) { + if (freq <= table[i].freq) { + freq = table[i].freq; + break; + } + } + +set_freq: + + ret = core_clks_set_rate(core, freq); + if (ret) { + dev_err(dev, "failed to set clock rate %lu (%d)\n", + freq, ret); + return ret; + } + + ret = load_scale_bw(core); + if (ret) { + dev_err(dev, "failed to set bandwidth (%d)\n", + ret); + return ret; + } + + return 0; +} + +static const struct venus_pm_ops pm_ops_v4 = { + .core_get = core_get_v4, + .core_put = core_put_v4, + .core_power = core_power_v4, + .vdec_get = vdec_get_v4, + .vdec_put = vdec_put_v4, + .vdec_power = vdec_power_v4, + .venc_get = venc_get_v4, + .venc_put = venc_put_v4, + .venc_power = venc_power_v4, + .load_scale = load_scale_v4, +}; + +const struct venus_pm_ops *venus_pm_get(enum hfi_version version) +{ + switch (version) { + case HFI_VERSION_1XX: + default: + return &pm_ops_v1; + case HFI_VERSION_3XX: + return &pm_ops_v3; + case HFI_VERSION_4XX: + return &pm_ops_v4; + } + + return NULL; +} diff --git a/drivers/media/platform/qcom/venus/pm_helpers.h b/drivers/media/platform/qcom/venus/pm_helpers.h new file mode 100644 index 000000000000..d98b6dc20740 --- /dev/null +++ b/drivers/media/platform/qcom/venus/pm_helpers.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright (C) 2019 Linaro Ltd. */ +#ifndef __VENUS_PM_HELPERS_H__ +#define __VENUS_PM_HELPERS_H__ + +struct device; + +#define POWER_ON 1 +#define POWER_OFF 0 + +struct venus_pm_ops { + int (*core_get)(struct device *dev); + void (*core_put)(struct device *dev); + int (*core_power)(struct device *dev, int on); + + int (*vdec_get)(struct device *dev); + void (*vdec_put)(struct device *dev); + int (*vdec_power)(struct device *dev, int on); + + int (*venc_get)(struct device *dev); + void (*venc_put)(struct device *dev); + int (*venc_power)(struct device *dev, int on); + + int (*load_scale)(struct venus_inst *inst); +}; + +const struct venus_pm_ops *venus_pm_get(enum hfi_version version); + +static inline int venus_pm_load_scale(struct venus_inst *inst) +{ + struct venus_core *core = inst->core; + + if (!core->pm_ops || !core->pm_ops->load_scale) + return 0; + + return core->pm_ops->load_scale(inst); +} + +#endif diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c index 8feaf5daece9..83ce30516591 100644 --- a/drivers/media/platform/qcom/venus/vdec.c +++ b/drivers/media/platform/qcom/venus/vdec.c @@ -20,6 +20,7 @@ #include "core.h" #include "helpers.h" #include "vdec.h" +#include "pm_helpers.h" /* * Three resons to keep MPLANE formats (despite that the number of planes @@ -868,7 +869,7 @@ static int vdec_start_capture(struct venus_inst *inst) if (ret) goto free_dpb_bufs; - venus_helper_load_scale_clocks(inst); + venus_pm_load_scale(inst); ret = hfi_session_continue(inst); if (ret) @@ -1076,7 +1077,7 @@ static void vdec_session_release(struct venus_inst *inst) hfi_session_abort(inst); venus_helper_free_dpb_bufs(inst); - venus_helper_load_scale_clocks(inst); + venus_pm_load_scale(inst); INIT_LIST_HEAD(&inst->registeredbufs); mutex_unlock(&inst->lock); @@ -1432,20 +1433,14 @@ static int vdec_probe(struct platform_device *pdev) if (!core) return -EPROBE_DEFER; - if (IS_V3(core) || IS_V4(core)) { - core->core0_clk = devm_clk_get(dev, "core"); - if (IS_ERR(core->core0_clk)) - return PTR_ERR(core->core0_clk); - } + platform_set_drvdata(pdev, core); - if (IS_V4(core)) { - core->core0_bus_clk = devm_clk_get(dev, "bus"); - if (IS_ERR(core->core0_bus_clk)) - return PTR_ERR(core->core0_bus_clk); + if (core->pm_ops->vdec_get) { + ret = core->pm_ops->vdec_get(dev); + if (ret) + return ret; } - platform_set_drvdata(pdev, core); - vdev = video_device_alloc(); if (!vdev) return -ENOMEM; @@ -1482,57 +1477,33 @@ static int vdec_remove(struct platform_device *pdev) video_unregister_device(core->vdev_dec); pm_runtime_disable(core->dev_dec); + if (core->pm_ops->vdec_put) + core->pm_ops->vdec_put(core->dev_dec); + return 0; } static __maybe_unused int vdec_runtime_suspend(struct device *dev) { struct venus_core *core = dev_get_drvdata(dev); - int ret; - - if (IS_V1(core)) - return 0; - - ret = venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, true); - if (ret) - return ret; - - if (IS_V4(core)) - clk_disable_unprepare(core->core0_bus_clk); + const struct venus_pm_ops *pm_ops = core->pm_ops; + int ret = 0; - clk_disable_unprepare(core->core0_clk); + if (pm_ops->vdec_power) + ret = pm_ops->vdec_power(dev, POWER_OFF); - return venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, false); + return ret; } static __maybe_unused int vdec_runtime_resume(struct device *dev) { struct venus_core *core = dev_get_drvdata(dev); - int ret; - - if (IS_V1(core)) - return 0; - - ret = venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, true); - if (ret) - return ret; - - ret = clk_prepare_enable(core->core0_clk); - if (ret) - goto err_power_disable; - - if (IS_V4(core)) - ret = clk_prepare_enable(core->core0_bus_clk); - - if (ret) - goto err_unprepare_core0; + const struct venus_pm_ops *pm_ops = core->pm_ops; + int ret = 0; - return venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, false); + if (pm_ops->vdec_power) + ret = pm_ops->vdec_power(dev, POWER_ON); -err_unprepare_core0: - clk_disable_unprepare(core->core0_clk); -err_power_disable: - venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, false); return ret; } diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c index 453edf966d4f..5fa1724cb537 100644 --- a/drivers/media/platform/qcom/venus/venc.c +++ b/drivers/media/platform/qcom/venus/venc.c @@ -20,6 +20,7 @@ #include "core.h" #include "helpers.h" #include "venc.h" +#include "pm_helpers.h" #define NUM_B_FRAMES_MAX 4 @@ -1255,20 +1256,14 @@ static int venc_probe(struct platform_device *pdev) if (!core) return -EPROBE_DEFER; - if (IS_V3(core) || IS_V4(core)) { - core->core1_clk = devm_clk_get(dev, "core"); - if (IS_ERR(core->core1_clk)) - return PTR_ERR(core->core1_clk); - } + platform_set_drvdata(pdev, core); - if (IS_V4(core)) { - core->core1_bus_clk = devm_clk_get(dev, "bus"); - if (IS_ERR(core->core1_bus_clk)) - return PTR_ERR(core->core1_bus_clk); + if (core->pm_ops->venc_get) { + ret = core->pm_ops->venc_get(dev); + if (ret) + return ret; } - platform_set_drvdata(pdev, core); - vdev = video_device_alloc(); if (!vdev) return -ENOMEM; @@ -1305,57 +1300,33 @@ static int venc_remove(struct platform_device *pdev) video_unregister_device(core->vdev_enc); pm_runtime_disable(core->dev_enc); + if (core->pm_ops->venc_put) + core->pm_ops->venc_put(core->dev_enc); + return 0; } static __maybe_unused int venc_runtime_suspend(struct device *dev) { struct venus_core *core = dev_get_drvdata(dev); - int ret; - - if (IS_V1(core)) - return 0; - - ret = venus_helper_power_enable(core, VIDC_SESSION_TYPE_ENC, true); - if (ret) - return ret; - - if (IS_V4(core)) - clk_disable_unprepare(core->core1_bus_clk); + const struct venus_pm_ops *pm_ops = core->pm_ops; + int ret = 0; - clk_disable_unprepare(core->core1_clk); + if (pm_ops->venc_power) + ret = pm_ops->venc_power(dev, POWER_OFF); - return venus_helper_power_enable(core, VIDC_SESSION_TYPE_ENC, false); + return ret; } static __maybe_unused int venc_runtime_resume(struct device *dev) { struct venus_core *core = dev_get_drvdata(dev); - int ret; - - if (IS_V1(core)) - return 0; - - ret = venus_helper_power_enable(core, VIDC_SESSION_TYPE_ENC, true); - if (ret) - return ret; - - ret = clk_prepare_enable(core->core1_clk); - if (ret) - goto err_power_disable; - - if (IS_V4(core)) - ret = clk_prepare_enable(core->core1_bus_clk); - - if (ret) - goto err_unprepare_core1; + const struct venus_pm_ops *pm_ops = core->pm_ops; + int ret = 0; - return venus_helper_power_enable(core, VIDC_SESSION_TYPE_ENC, false); + if (pm_ops->venc_power) + ret = pm_ops->venc_power(dev, POWER_ON); -err_unprepare_core1: - clk_disable_unprepare(core->core1_clk); -err_power_disable: - venus_helper_power_enable(core, VIDC_SESSION_TYPE_ENC, false); return ret; } From patchwork Mon Dec 9 08:38:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanimir Varbanov X-Patchwork-Id: 11278615 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D5F31112B for ; Mon, 9 Dec 2019 08:39:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A00222080D for ; Mon, 9 Dec 2019 08:39:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="n7x3zaPS" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727326AbfLIIjS (ORCPT ); Mon, 9 Dec 2019 03:39:18 -0500 Received: from mail-lf1-f67.google.com ([209.85.167.67]:38879 "EHLO mail-lf1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727313AbfLIIjS (ORCPT ); Mon, 9 Dec 2019 03:39:18 -0500 Received: by mail-lf1-f67.google.com with SMTP id r14so9993641lfm.5 for ; Mon, 09 Dec 2019 00:39:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=He6t0UxaZpAmu7uu3983tFNonS2D3TSq+7lw5wWZaw4=; b=n7x3zaPSm+SIgfnAzqehPd0Ye77W1oBMZ+fQegdK/G4C876Be+fF7to+xe+rruMqRe mMGdAd+oVM3Yazdrr/HiYHoXNCXkyeyLlDLcgzhiNadMHJXRlSGDrkQaMStVrgLikPx5 /ozr+FHlkpAHWtxCE5IjvPGqfIMwpXdi9nGmf6kuNuWHh/lSYIGUoQFesgyVJ/eJtfY+ ewrjENNWNO+0CxJXhA9himGXPEiADgXVqu0rIZiZ2no6k9D2erRztT2huyF1JYjowwXM O7kO1KcHE//XCOVUai1xc6zB2RplnzZB7oF9XNtn+vC0Rlbe7JYQhKTbZrkHO9hJk4xP 20YQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=He6t0UxaZpAmu7uu3983tFNonS2D3TSq+7lw5wWZaw4=; b=YIXQh4md957pviPVMRDeAK3FgA5htV4hHJcHZvSlXJlpgn26MV83RepxhECJ2LVMb1 MFGNX0Hv5XVRZs6Jz7w3f+qgvYuK8/nui8P36F6We/WnH9nYjRciOr5IoaobzVVDefcc MFxDv1H+qFdKMhgyhMyicU5cxfGAXc0TjzYZc8yCGxt92uJyToS3bnZJ+ctZhIKei36X 744v8HFB9Zwoc8CL5+R92MTg1a7rlkeRetk8BRXseh6EfoQDueenp2bJCwV7lMTMSXFv 2miGMXreaQsNDPXl4jhgM2l8iPORZsoCP/qNTbZk/ZzaK2iQcVewLQlZWpBHSY5B4Zx/ 7x8g== X-Gm-Message-State: APjAAAXfaGjf+WjmEGThLy2z6A4cB7e3k7f4Ni8KSWQzp+Udg5pX1pUS HvGmM0vwzQsrmfNabt7JR2dGSXW2kks= X-Google-Smtp-Source: APXvYqw6gCAmeY37IUgqCFSLI6hdINJc2uTYgcbpU0Rkqh28rA92VRdvnpBAh9VGyw4w5mrCv97zbQ== X-Received: by 2002:a19:c3cc:: with SMTP id t195mr6029067lff.144.1575880755416; Mon, 09 Dec 2019 00:39:15 -0800 (PST) Received: from localhost.localdomain ([37.157.136.193]) by smtp.gmail.com with ESMTPSA id r26sm10438971lfm.82.2019.12.09.00.39.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2019 00:39:14 -0800 (PST) From: Stanimir Varbanov To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org Cc: Vikash Garodia , Tomasz Figa , Aniket Masule , Stanimir Varbanov Subject: [PATCH 2/6] media: venus: introduce core selection Date: Mon, 9 Dec 2019 10:38:20 +0200 Message-Id: <20191209083824.806-3-stanimir.varbanov@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191209083824.806-1-stanimir.varbanov@linaro.org> References: <20191209083824.806-1-stanimir.varbanov@linaro.org> Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Aniket Masule Presently the core (vcodec pipelines) assignment is static. Here we introduce dynamic load balancing across the cores depending on the current session load. The load on earch core is calculated and core with minimum load is assigned to given instance. This will be applicable on Venus v4 with more than one vcodec cores. Signed-off-by: Aniket Masule Co-developed-by: Stanimir Varbanov Signed-off-by: Stanimir Varbanov --- drivers/media/platform/qcom/venus/core.h | 3 + drivers/media/platform/qcom/venus/helpers.c | 17 +- drivers/media/platform/qcom/venus/helpers.h | 1 - .../media/platform/qcom/venus/hfi_helper.h | 1 + .../media/platform/qcom/venus/hfi_parser.h | 5 + .../media/platform/qcom/venus/pm_helpers.c | 173 ++++++++++++++++-- .../media/platform/qcom/venus/pm_helpers.h | 26 +++ drivers/media/platform/qcom/venus/vdec.c | 11 +- drivers/media/platform/qcom/venus/venc.c | 10 +- 9 files changed, 211 insertions(+), 36 deletions(-) diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h index 179ff861a618..7efa5669dda3 100644 --- a/drivers/media/platform/qcom/venus/core.h +++ b/drivers/media/platform/qcom/venus/core.h @@ -177,6 +177,8 @@ struct venus_core { struct delayed_work work; struct venus_caps caps[MAX_CODEC_NUM]; unsigned int codecs_count; + unsigned int core0_usage_count; + unsigned int core1_usage_count; }; struct vdec_controls { @@ -375,6 +377,7 @@ struct venus_inst { const struct hfi_inst_ops *ops; u32 session_type; union hfi_get_property hprop; + unsigned int core_acquired: 1; }; #define IS_V1(core) ((core)->res->hfi_version == HFI_VERSION_1XX) diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c index cbeeaf6077e0..043e0d8705a8 100644 --- a/drivers/media/platform/qcom/venus/helpers.c +++ b/drivers/media/platform/qcom/venus/helpers.c @@ -723,21 +723,6 @@ int venus_helper_set_work_mode(struct venus_inst *inst, u32 mode) } EXPORT_SYMBOL_GPL(venus_helper_set_work_mode); -int venus_helper_set_core_usage(struct venus_inst *inst, u32 usage) -{ - const u32 ptype = HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE; - struct hfi_videocores_usage_type cu; - - inst->clk_data.core_id = usage; - if (!IS_V4(inst->core)) - return 0; - - cu.video_core_enable_mask = usage; - - return hfi_session_set_property(inst, ptype, &cu); -} -EXPORT_SYMBOL_GPL(venus_helper_set_core_usage); - int venus_helper_init_codec_freq_data(struct venus_inst *inst) { const struct codec_freq_data *data; @@ -1101,6 +1086,8 @@ void venus_helper_vb2_stop_streaming(struct vb2_queue *q) else inst->streamon_cap = 0; + venus_pm_release_core(inst); + mutex_unlock(&inst->lock); } EXPORT_SYMBOL_GPL(venus_helper_vb2_stop_streaming); diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h index 44b50914cd12..b64875564064 100644 --- a/drivers/media/platform/qcom/venus/helpers.h +++ b/drivers/media/platform/qcom/venus/helpers.h @@ -34,7 +34,6 @@ int venus_helper_set_output_resolution(struct venus_inst *inst, u32 buftype); int venus_helper_set_work_mode(struct venus_inst *inst, u32 mode); int venus_helper_init_codec_freq_data(struct venus_inst *inst); -int venus_helper_set_core_usage(struct venus_inst *inst, u32 usage); int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs, unsigned int output_bufs, unsigned int output2_bufs); diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h b/drivers/media/platform/qcom/venus/hfi_helper.h index b70551e296b7..2a47f6c0a07a 100644 --- a/drivers/media/platform/qcom/venus/hfi_helper.h +++ b/drivers/media/platform/qcom/venus/hfi_helper.h @@ -550,6 +550,7 @@ struct hfi_bitrate { #define HFI_CAPABILITY_LCU_SIZE 0x14 #define HFI_CAPABILITY_HIER_P_HYBRID_NUM_ENH_LAYERS 0x15 #define HFI_CAPABILITY_MBS_PER_SECOND_POWERSAVE 0x16 +#define HFI_CAPABILITY_MAX_VIDEOCORES 0x2b struct hfi_capability { u32 capability_type; diff --git a/drivers/media/platform/qcom/venus/hfi_parser.h b/drivers/media/platform/qcom/venus/hfi_parser.h index 3e931c747e19..264e6dd2415f 100644 --- a/drivers/media/platform/qcom/venus/hfi_parser.h +++ b/drivers/media/platform/qcom/venus/hfi_parser.h @@ -107,4 +107,9 @@ static inline u32 frate_step(struct venus_inst *inst) return cap_step(inst, HFI_CAPABILITY_FRAMERATE); } +static inline u32 core_num_max(struct venus_inst *inst) +{ + return cap_max(inst, HFI_CAPABILITY_MAX_VIDEOCORES); +} + #endif diff --git a/drivers/media/platform/qcom/venus/pm_helpers.c b/drivers/media/platform/qcom/venus/pm_helpers.c index 4e4aace2d8c8..794c9210ea46 100644 --- a/drivers/media/platform/qcom/venus/pm_helpers.c +++ b/drivers/media/platform/qcom/venus/pm_helpers.c @@ -14,6 +14,7 @@ #include #include "core.h" +#include "hfi_parser.h" #include "hfi_venus_io.h" #include "pm_helpers.h" @@ -450,6 +451,163 @@ static int poweron_coreid(struct venus_core *core, unsigned int coreid_mask) return 0; } +static void +min_loaded_core(struct venus_inst *inst, u32 *min_coreid, u32 *min_load) +{ + u32 mbs_per_sec, load, core1_load = 0, core2_load = 0; + u32 cores_max = core_num_max(inst); + struct venus_core *core = inst->core; + struct venus_inst *inst_pos; + + mutex_lock(&core->lock); + + list_for_each_entry(inst_pos, &core->instances, list) { + if (inst_pos == inst) + continue; + mbs_per_sec = load_per_instance(inst_pos); + load = mbs_per_sec * + inst_pos->clk_data.codec_freq_data->vpp_freq; + if (inst_pos->clk_data.core_id & VIDC_CORE_ID_3) { + core1_load += load / 2; + core2_load += load / 2; + } else if (inst_pos->clk_data.core_id & VIDC_CORE_ID_1) { + core1_load += load; + } else if (inst_pos->clk_data.core_id & VIDC_CORE_ID_2) { + core2_load += load; + } + } + + *min_coreid = core1_load <= core2_load ? + VIDC_CORE_ID_1 : VIDC_CORE_ID_2; + *min_load = min(core1_load, core2_load); + + if (cores_max < VIDC_CORE_ID_2) { + *min_coreid = VIDC_CORE_ID_1; + *min_load = core1_load; + } + + mutex_unlock(&core->lock); +} + +static int decide_core(struct venus_inst *inst) +{ + const u32 ptype = HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE; + struct venus_core *core = inst->core; + u32 min_coreid, min_load, inst_load; + struct hfi_videocores_usage_type cu; + unsigned long max_freq; + + if (legacy_binding) { + if (inst->session_type == VIDC_SESSION_TYPE_DEC) + cu.video_core_enable_mask = VIDC_CORE_ID_1; + else + cu.video_core_enable_mask = VIDC_CORE_ID_2; + + goto done; + } + + if (inst->clk_data.core_id != VIDC_CORE_ID_DEFAULT) + return 0; + + inst_load = load_per_instance(inst); + inst_load *= inst->clk_data.codec_freq_data->vpp_freq; + max_freq = core->res->freq_tbl[0].freq; + + min_loaded_core(inst, &min_coreid, &min_load); + + if ((inst_load + min_load) > max_freq) { + dev_warn(core->dev, "HW is overloaded, needed: %u max: %lu\n", + inst_load, max_freq); + return -EINVAL; + } + + inst->clk_data.core_id = min_coreid; + cu.video_core_enable_mask = min_coreid; + +done: + return hfi_session_set_property(inst, ptype, &cu); +} + +static int acquire_core(struct venus_inst *inst) +{ + struct venus_core *core = inst->core; + unsigned int coreid_mask; + + if (inst->core_acquired) + return 0; + + inst->core_acquired = true; + + if (inst->clk_data.core_id & VIDC_CORE_ID_1) { + if (core->core0_usage_count++) + return 0; + + coreid_mask = VIDC_CORE_ID_1; + } + + if (inst->clk_data.core_id & VIDC_CORE_ID_2) { + if (core->core1_usage_count++) + return 0; + + coreid_mask |= VIDC_CORE_ID_2; + } + + return poweron_coreid(core, coreid_mask); +} + +static int release_core(struct venus_inst *inst) +{ + struct venus_core *core = inst->core; + unsigned int coreid_mask; + int ret; + + if (!inst->core_acquired) + return 0; + + if (inst->clk_data.core_id & VIDC_CORE_ID_1) { + if (--core->core0_usage_count) + goto done; + + coreid_mask = VIDC_CORE_ID_1; + } + + if (inst->clk_data.core_id & VIDC_CORE_ID_2) { + if (--core->core1_usage_count) + goto done; + + coreid_mask |= VIDC_CORE_ID_2; + } + + ret = poweroff_coreid(core, coreid_mask); + if (ret) + return ret; + +done: + inst->clk_data.core_id = VIDC_CORE_ID_DEFAULT; + inst->core_acquired = false; + return 0; +} + +static int coreid_power_v4(struct venus_inst *inst, int on) +{ + int ret; + + if (legacy_binding) + return 0; + + if (on == POWER_ON) { + ret = decide_core(inst); + if (ret) + return ret; + + ret = acquire_core(inst); + } else { + ret = release_core(inst); + } + + return ret; +} + static int vdec_get_v4(struct device *dev) { struct venus_core *core = dev_get_drvdata(dev); @@ -654,22 +812,12 @@ static void core_put_v4(struct device *dev) static int core_power_v4(struct device *dev, int on) { struct venus_core *core = dev_get_drvdata(dev); - const unsigned int coreid_mask = VIDC_CORE_ID_1 | VIDC_CORE_ID_2; int ret = 0; - if (on == POWER_ON) { + if (on == POWER_ON) ret = core_clks_enable(core); - if (ret) - return ret; - - if (!legacy_binding) - ret = poweron_coreid(core, coreid_mask); - } else { - if (!legacy_binding) - ret = poweroff_coreid(core, coreid_mask); - + else core_clks_disable(core); - } return ret; } @@ -779,6 +927,7 @@ static const struct venus_pm_ops pm_ops_v4 = { .venc_get = venc_get_v4, .venc_put = venc_put_v4, .venc_power = venc_power_v4, + .coreid_power = coreid_power_v4, .load_scale = load_scale_v4, }; diff --git a/drivers/media/platform/qcom/venus/pm_helpers.h b/drivers/media/platform/qcom/venus/pm_helpers.h index d98b6dc20740..aa2f6afa2354 100644 --- a/drivers/media/platform/qcom/venus/pm_helpers.h +++ b/drivers/media/platform/qcom/venus/pm_helpers.h @@ -21,6 +21,8 @@ struct venus_pm_ops { void (*venc_put)(struct device *dev); int (*venc_power)(struct device *dev, int on); + int (*coreid_power)(struct venus_inst *inst, int on); + int (*load_scale)(struct venus_inst *inst); }; @@ -36,4 +38,28 @@ static inline int venus_pm_load_scale(struct venus_inst *inst) return core->pm_ops->load_scale(inst); } +static inline int venus_pm_acquire_core(struct venus_inst *inst) +{ + struct venus_core *core = inst->core; + const struct venus_pm_ops *pm_ops = core->pm_ops; + int ret = 0; + + if (pm_ops && pm_ops->coreid_power) + ret = pm_ops->coreid_power(inst, POWER_ON); + + return ret; +} + +static inline int venus_pm_release_core(struct venus_inst *inst) +{ + struct venus_core *core = inst->core; + const struct venus_pm_ops *pm_ops = core->pm_ops; + int ret = 0; + + if (pm_ops && pm_ops->coreid_power) + ret = pm_ops->coreid_power(inst, POWER_OFF); + + return ret; +} + #endif diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c index 83ce30516591..7b7a4f3377cf 100644 --- a/drivers/media/platform/qcom/venus/vdec.c +++ b/drivers/media/platform/qcom/venus/vdec.c @@ -579,10 +579,6 @@ static int vdec_output_conf(struct venus_inst *inst) if (ret) return ret; - ret = venus_helper_set_core_usage(inst, VIDC_CORE_ID_1); - if (ret) - return ret; - if (core->res->hfi_version == HFI_VERSION_1XX) { ptype = HFI_PROPERTY_PARAM_VDEC_CONTINUE_DATA_TRANSFER; ret = hfi_session_set_property(inst, ptype, &en); @@ -951,6 +947,10 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count) mutex_lock(&inst->lock); + ret = venus_pm_acquire_core(inst); + if (ret) + return ret; + if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) ret = vdec_start_capture(inst); else @@ -1078,6 +1078,7 @@ static void vdec_session_release(struct venus_inst *inst) venus_helper_free_dpb_bufs(inst); venus_pm_load_scale(inst); + venus_pm_release_core(inst); INIT_LIST_HEAD(&inst->registeredbufs); mutex_unlock(&inst->lock); @@ -1337,6 +1338,8 @@ static int vdec_open(struct file *file) inst->num_output_bufs = 1; inst->codec_state = VENUS_DEC_STATE_DEINIT; inst->buf_count = 0; + inst->clk_data.core_id = VIDC_CORE_ID_DEFAULT; + inst->core_acquired = false; init_waitqueue_head(&inst->reconf_wait); venus_helper_init_instance(inst); diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c index 5fa1724cb537..4f594fe4b661 100644 --- a/drivers/media/platform/qcom/venus/venc.c +++ b/drivers/media/platform/qcom/venus/venc.c @@ -656,10 +656,6 @@ static int venc_set_properties(struct venus_inst *inst) if (ret) return ret; - ret = venus_helper_set_core_usage(inst, VIDC_CORE_ID_2); - if (ret) - return ret; - ptype = HFI_PROPERTY_CONFIG_FRAME_RATE; frate.buffer_type = HFI_BUFFER_OUTPUT; frate.framerate = inst->fps * (1 << 16); @@ -992,6 +988,10 @@ static int venc_start_streaming(struct vb2_queue *q, unsigned int count) if (ret) goto bufs_done; + ret = venus_pm_acquire_core(inst); + if (ret) + goto deinit_sess; + ret = venc_set_properties(inst); if (ret) goto deinit_sess; @@ -1160,6 +1160,8 @@ static int venc_open(struct file *file) inst->core = core; inst->session_type = VIDC_SESSION_TYPE_ENC; + inst->clk_data.core_id = VIDC_CORE_ID_DEFAULT; + inst->core_acquired = false; venus_helper_init_instance(inst); From patchwork Mon Dec 9 08:38:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanimir Varbanov X-Patchwork-Id: 11278619 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 41C741864 for ; Mon, 9 Dec 2019 08:39:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 168022080D for ; Mon, 9 Dec 2019 08:39:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="v9gYohOs" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727200AbfLIIjr (ORCPT ); Mon, 9 Dec 2019 03:39:47 -0500 Received: from mail-lj1-f194.google.com ([209.85.208.194]:34602 "EHLO mail-lj1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727320AbfLIIjS (ORCPT ); Mon, 9 Dec 2019 03:39:18 -0500 Received: by mail-lj1-f194.google.com with SMTP id m6so14627539ljc.1 for ; Mon, 09 Dec 2019 00:39:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=v49ydhRHchofWzIVQxqpzPrTtsddtPryUTJT2t4dhHg=; b=v9gYohOsVaW5DeUzwcntIszGR11cpCwA1aGZQI/0YtSyva2+iY9Ho4nwPh1XkuEEcs ePzEjiMXfOWUfjkuL6au580P6z9q6aTIQ4BAFKeXFDBtnUhU+aVTh0lEAgY4C7HCpSV9 ORwIprAIN+8zsxiBuym7PqglNRmEVQycLevLaHhL1ebSrdLLpkdVn/rdkrrGJkyYQf8v c/52xy2F25iRJr8xYXD9Nv2jRKO5G/tG/EsYycAbXnJ/v4Fz6AxBSvRdmt06fg7z7xF1 U0uhvGFU/i/H8jRdR5icC6XyPExufkNMMEEbkYwB0qS9424ytO17ogMD6Dgb8ahdoFI9 ukiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=v49ydhRHchofWzIVQxqpzPrTtsddtPryUTJT2t4dhHg=; b=PKkTbxcMF2wsYX+gFsovH/ODhnS6R0ds1DWg/B+po3ZSw/HWnA4zRPVSfdfYy5CB/n NHBBQDeIj3CjRTPWDgTMGB21cKqedhc0WUr8NCSs31OIYANAgdU+DONpN6JCDh6v7Rc5 2gurO15fstO+v1F2/uvEN5vgslK6dH4cAAXB7TO0abyH4VpkOkiVqK5YcDdGKo2Weet/ C0/uGAcGojLz6mSFXoMOFWbtwyNkV4lJKLFB6chuEF873Und16C/Fw0TbVNjGK5k3JpH Nhr5HESD8/IYCxaHIKUSA95KAYssxIbUkoS7u6oVwpKnYQxcTbhk6+ypM7L7u/VjP2jB AU6w== X-Gm-Message-State: APjAAAXeA4LevWLjPZ5Beh/SRtmdf/wWg0q2cY5bEDHztV6O4522fEap oAuZYaqJDkx+2FwNEY9PCjRJwPiPpvM= X-Google-Smtp-Source: APXvYqw8SpOAGnK9Y6ldb09TK9z4GTkamwKgzyRGZM1ZhvBxsbDATkag4guNVGmYYMSeCsQjPFddtA== X-Received: by 2002:a2e:808a:: with SMTP id i10mr15268982ljg.151.1575880757072; Mon, 09 Dec 2019 00:39:17 -0800 (PST) Received: from localhost.localdomain ([37.157.136.193]) by smtp.gmail.com with ESMTPSA id r26sm10438971lfm.82.2019.12.09.00.39.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2019 00:39:16 -0800 (PST) From: Stanimir Varbanov To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org Cc: Vikash Garodia , Tomasz Figa , Stanimir Varbanov Subject: [PATCH 3/6] arm64: dts: sdm845: move venus clocks and pmdomains in parent Date: Mon, 9 Dec 2019 10:38:21 +0200 Message-Id: <20191209083824.806-4-stanimir.varbanov@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191209083824.806-1-stanimir.varbanov@linaro.org> References: <20191209083824.806-1-stanimir.varbanov@linaro.org> Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Move venus DT subnode resources like clocks and pmdomains in the parent Venus DT node. This will make possible to support dynamic core selection on Venus v4. Signed-off-by: Stanimir Varbanov --- arch/arm64/boot/dts/qcom/sdm845.dtsi | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/arch/arm64/boot/dts/qcom/sdm845.dtsi b/arch/arm64/boot/dts/qcom/sdm845.dtsi index f406a4340b05..540028fed3ae 100644 --- a/arch/arm64/boot/dts/qcom/sdm845.dtsi +++ b/arch/arm64/boot/dts/qcom/sdm845.dtsi @@ -2571,29 +2571,30 @@ compatible = "qcom,sdm845-venus"; reg = <0 0x0aa00000 0 0xff000>; interrupts = ; - power-domains = <&videocc VENUS_GDSC>; + power-domains = <&videocc VENUS_GDSC>, + <&videocc VCODEC0_GDSC>, + <&videocc VCODEC1_GDSC>; + power-domain-names = "venus", "vcodec0", "vcodec1"; clocks = <&videocc VIDEO_CC_VENUS_CTL_CORE_CLK>, <&videocc VIDEO_CC_VENUS_AHB_CLK>, - <&videocc VIDEO_CC_VENUS_CTL_AXI_CLK>; - clock-names = "core", "iface", "bus"; + <&videocc VIDEO_CC_VENUS_CTL_AXI_CLK>, + <&videocc VIDEO_CC_VCODEC0_CORE_CLK>, + <&videocc VIDEO_CC_VCODEC0_AXI_CLK>, + <&videocc VIDEO_CC_VCODEC1_CORE_CLK>, + <&videocc VIDEO_CC_VCODEC1_AXI_CLK>; + clock-names = "core", "iface", "bus", + "vcodec0_core", "vcodec0_bus", + "vcodec1_core", "vcodec1_bus"; iommus = <&apps_smmu 0x10a0 0x8>, <&apps_smmu 0x10b0 0x0>; memory-region = <&venus_mem>; video-core0 { compatible = "venus-decoder"; - clocks = <&videocc VIDEO_CC_VCODEC0_CORE_CLK>, - <&videocc VIDEO_CC_VCODEC0_AXI_CLK>; - clock-names = "core", "bus"; - power-domains = <&videocc VCODEC0_GDSC>; }; video-core1 { compatible = "venus-encoder"; - clocks = <&videocc VIDEO_CC_VCODEC1_CORE_CLK>, - <&videocc VIDEO_CC_VCODEC1_AXI_CLK>; - clock-names = "core", "bus"; - power-domains = <&videocc VCODEC1_GDSC>; }; }; From patchwork Mon Dec 9 08:38:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanimir Varbanov X-Patchwork-Id: 11278609 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 963FA112B for ; Mon, 9 Dec 2019 08:39:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 73D862075B for ; Mon, 9 Dec 2019 08:39:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="vFQKn9Bv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727354AbfLIIjV (ORCPT ); Mon, 9 Dec 2019 03:39:21 -0500 Received: from mail-lj1-f196.google.com ([209.85.208.196]:47064 "EHLO mail-lj1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727299AbfLIIjU (ORCPT ); Mon, 9 Dec 2019 03:39:20 -0500 Received: by mail-lj1-f196.google.com with SMTP id z17so14539113ljk.13 for ; Mon, 09 Dec 2019 00:39:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=M6xBk2t3JSq6Gm+btMkUw0HvLbeKjwldK9lPc/9p6IU=; b=vFQKn9BvVVUUQ4Y9uusSgffYFjQKPOk2bHHTZorOpYzW/4FB9kB6JzD1XbXDirsL66 NJhJcFSJBS9/9nXEbtC4h3cl0FQ9w3y0WHYi/zqUm06sd3JfJjSIfaZTPQfTVEvIL9Nx kwThT+MoV+y5uvtbP83Hoe1MPxjqQyz9E2MYnYL7N3U7Vt6onDSv+PlqO/P0Fxto37YB 9zAywrd1pHIk68ouG2NpoiLA4GEz04/VVpDXnRTcMbH9d5jo4uQBhJcUBy8qFLL1JgVM rD/gDDq6nMW+NtJi+oVMZx/sL53+sIDMBMcspigWp//UCstQBEPwEJqCo+xH8CoXmcCQ Z5UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=M6xBk2t3JSq6Gm+btMkUw0HvLbeKjwldK9lPc/9p6IU=; b=HwZm/3fZ8E/3igo0h1JwJp/rkk87c95+eJXjwqxo/SCWTr2CQfvvzyiK7lDchmslHy aQ1G0yiAJIQst1kCyvKQADVZ1UrpqLQELCaw78TCYE1qAfopSB7e17LUwYiB1OI4eAtr 1ZAw0I+G+irgc+Udq0XjlwCi8p4yZzz5eiIuKsPB6N7nsOf9b3ssq9QhYiwsnvVHoWkL 3M4XafaP+lBFZaVOwqaPFMBdrIcC6SMeq25Nn6FYSxnuNGy2o5bOScMon3HRnLyz660m AVmqhJAKKK00Mur/bIoYNKw5g483TF4+b0AdYw04IPsfwy5y4ZlOg8R6eKpEFisF9ePQ 5nQw== X-Gm-Message-State: APjAAAWVVmsNRaXxTaR27+jbZIjaeSojrM88+1yjhbY/xKw5arvQT/mi e81xNTfoV0z3SCALXS2jiHO2tJDTrTc= X-Google-Smtp-Source: APXvYqyd/Hl/dcqTOW4HAPtSVFyeUDEZNQ0Muro85srrZE45Au6soKwyk1tCfVT095Stl7QIjLEjvw== X-Received: by 2002:a2e:844e:: with SMTP id u14mr12502198ljh.183.1575880758682; Mon, 09 Dec 2019 00:39:18 -0800 (PST) Received: from localhost.localdomain ([37.157.136.193]) by smtp.gmail.com with ESMTPSA id r26sm10438971lfm.82.2019.12.09.00.39.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2019 00:39:18 -0800 (PST) From: Stanimir Varbanov To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org Cc: Vikash Garodia , Tomasz Figa , Stanimir Varbanov Subject: [PATCH 4/6] dt-bindings: media: venus: describe sdm845 with optional pmdomains Date: Mon, 9 Dec 2019 10:38:22 +0200 Message-Id: <20191209083824.806-5-stanimir.varbanov@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191209083824.806-1-stanimir.varbanov@linaro.org> References: <20191209083824.806-1-stanimir.varbanov@linaro.org> Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Describe an optional DT binding for sdm845 with power-domain-names and clocks in Venus DT node. After that the binding for sdm845 will have two options, the first will be to describe vcodec clocks and pmdomains in (encoder and decoder) subnodes and the second will be to describe vcodec clocks and pmdomains in the Venus DT node (the parent). Both option will be handled in driver, and the second one will give an opportunity to the driver to dynamically assign vcodec pipelines for every given instance depending on its load. Signed-off-by: Stanimir Varbanov --- .../devicetree/bindings/media/qcom,venus.txt | 22 ++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/Documentation/devicetree/bindings/media/qcom,venus.txt b/Documentation/devicetree/bindings/media/qcom,venus.txt index b602c4c025e7..69a104fcb8eb 100644 --- a/Documentation/devicetree/bindings/media/qcom,venus.txt +++ b/Documentation/devicetree/bindings/media/qcom,venus.txt @@ -35,6 +35,21 @@ - "iface" Video accelerator AHB clock - "bus" Video accelerator AXI clock - "mbus" Video MAXI clock +- clock-names: + Usage: required for sdm845 + Value type: + Definition: Should contain the following entries: + - "core" Core video accelerator clock + - "iface" Video accelerator AHB clock + - "bus" Video accelerator AXI clock +- clock-names: + Usage: optional for sdm845 + Value type: + Definition: Should contain the following entries: + - "vcodec0_core" Codec0 core clock + - "vcodec0_bus" Codec0 AXI clock + - "vcodec1_core" Codec1 core clock + - "vcoded1_bus" Codec1 AXI clock - power-domains: Usage: required Value type: @@ -72,7 +87,12 @@ Every of video-encoder or video-decoder subnode should have: Value type: Definition: Should contain the following entries: - "core" Subcore video accelerator clock - +- clock-names: + Usage: required for sdm845 + Value type: + Definition: Should contain the following entries: + - "core" Video codec accelerator clock + - "bus" Video codec AXI clock - power-domains: Usage: required for msm8996 Value type: From patchwork Mon Dec 9 08:38:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanimir Varbanov X-Patchwork-Id: 11278607 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AAD6B14B7 for ; Mon, 9 Dec 2019 08:39:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 88F162073D for ; Mon, 9 Dec 2019 08:39:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="pP/6yFBS" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727403AbfLIIje (ORCPT ); Mon, 9 Dec 2019 03:39:34 -0500 Received: from mail-lj1-f193.google.com ([209.85.208.193]:36538 "EHLO mail-lj1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727351AbfLIIjW (ORCPT ); Mon, 9 Dec 2019 03:39:22 -0500 Received: by mail-lj1-f193.google.com with SMTP id r19so14627375ljg.3 for ; Mon, 09 Dec 2019 00:39:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=fdnv5G54kjVyBGm0/gl9gu7nvtpvFPt8yg5pxAgrcmE=; b=pP/6yFBSAEaUiUH7CRhehuzaeVIWOjsGGhLeEuAGrVx2W2kXBMCABusQnG2xt+qaa3 9+6vOi7s+y77yQ6gFflSeY7pd9YAAoWss/Xve7ocv7TpCjMOqVCjJa9RjV9J9Ze9bLbv 6rEQo5qGbCne/g7FopoIkF7Q97iFLbFh3OyHWPHfzcCwaYpdCD2qfqGj0t2V3BeNv7DP g7e2kYRRKvOWqabt4DztTDYSMjopZgfBrN/hARbEtiv2E/bu05Jl4KRmTaaH4aNFlo65 3Y0yQei9g9/tTcqXsfk2+gm7OplpDCi+a7aim/o5HUzxNWdLWvHRyrJ2FDppvZ/WkOA0 /zoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=fdnv5G54kjVyBGm0/gl9gu7nvtpvFPt8yg5pxAgrcmE=; b=dyyfthzYD9isr/PrhHpmUb3MI0YlyzV2LTU1Dcy1udUpK99RmljQBUA1eGum6JcuW8 y1q9S+lWhad8T3zhbH1/5duZl2bzupJvsQDz3L2rRh0U3r5K39W0WzqkgsAtmU2PANqz fZnzQQNj8J5TG8s0IPu94cKzOZtU6aUNQF3ftSpkdFeZh42o4NvP+e7skd2+mokFii6f vrKm0+aaddl6eG7srnVBGxzZ6sbnKTPETZ+stervZVLiKciPFBh5y/4Mq7wxxjMmeahb /uPH5U//lnwZSj2F3CV2yfule7yJtpbPrndqgSzCz4XYrA9zYWmUzJKNxSphTRnbbWxL X3qg== X-Gm-Message-State: APjAAAXeAErTttFgeimYSuHU0znkF0aJoMjCLbJWrZOfOjVIKsg2WmQM UXFxFOtym1vBwqpKZqHTYh/KboTPHrg= X-Google-Smtp-Source: APXvYqynMar8YEFWbO4Rp+sphv+VAYBcOnLMWnfw760ddHNzTaxqy4s0hXFAhu9ZcxfqEEAKn9wL6Q== X-Received: by 2002:a2e:9708:: with SMTP id r8mr16116609lji.92.1575880760335; Mon, 09 Dec 2019 00:39:20 -0800 (PST) Received: from localhost.localdomain ([37.157.136.193]) by smtp.gmail.com with ESMTPSA id r26sm10438971lfm.82.2019.12.09.00.39.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2019 00:39:19 -0800 (PST) From: Stanimir Varbanov To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org Cc: Vikash Garodia , Tomasz Figa , Stanimir Varbanov Subject: [PATCH 5/6] v4l: Add source event change for bit-depth Date: Mon, 9 Dec 2019 10:38:23 +0200 Message-Id: <20191209083824.806-6-stanimir.varbanov@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191209083824.806-1-stanimir.varbanov@linaro.org> References: <20191209083824.806-1-stanimir.varbanov@linaro.org> Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This event indicate that the source color bit-depth is changed during run-time. The client must get the new format and re-allocate buffers for it. This can usually happens with video decoder (encoders) when the bit-stream color bit-depth is changed from 8 to 10bits or vice versa. Acked-by: Sakari Ailus Signed-off-by: Stanimir Varbanov --- Documentation/media/uapi/v4l/vidioc-dqevent.rst | 8 +++++++- Documentation/media/videodev2.h.rst.exceptions | 1 + include/uapi/linux/videodev2.h | 1 + 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/Documentation/media/uapi/v4l/vidioc-dqevent.rst b/Documentation/media/uapi/v4l/vidioc-dqevent.rst index 42659a3d1705..fad853d440cf 100644 --- a/Documentation/media/uapi/v4l/vidioc-dqevent.rst +++ b/Documentation/media/uapi/v4l/vidioc-dqevent.rst @@ -402,7 +402,13 @@ call. that many Video Capture devices are not able to recover from a temporary loss of signal and so restarting streaming I/O is required in order for the hardware to synchronize to the video signal. - + * - ``V4L2_EVENT_SRC_CH_COLOR_DEPTH`` + - 0x0002 + - This event gets triggered when color bit-depth change is detected + from a video decoder. Applications will have to query the new pixel + format and re-negotiate the queue. In most cases the streaming must be + stopped and restarted (:ref:`VIDIOC_STREAMOFF ` + followed by :ref:`VIDIOC_STREAMON `). Return Value ============ diff --git a/Documentation/media/videodev2.h.rst.exceptions b/Documentation/media/videodev2.h.rst.exceptions index cb6ccf91776e..209709114378 100644 --- a/Documentation/media/videodev2.h.rst.exceptions +++ b/Documentation/media/videodev2.h.rst.exceptions @@ -490,6 +490,7 @@ replace define V4L2_EVENT_CTRL_CH_FLAGS ctrl-changes-flags replace define V4L2_EVENT_CTRL_CH_RANGE ctrl-changes-flags replace define V4L2_EVENT_SRC_CH_RESOLUTION src-changes-flags +replace define V4L2_EVENT_SRC_CH_COLOR_DEPTH src-changes-flags replace define V4L2_EVENT_MD_FL_HAVE_FRAME_SEQ :c:type:`v4l2_event_motion_det` diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 04481c717fee..2a5138ac6d25 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -2307,6 +2307,7 @@ struct v4l2_event_frame_sync { }; #define V4L2_EVENT_SRC_CH_RESOLUTION (1 << 0) +#define V4L2_EVENT_SRC_CH_COLOR_DEPTH (1 << 1) struct v4l2_event_src_change { __u32 changes; From patchwork Mon Dec 9 08:38:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanimir Varbanov X-Patchwork-Id: 11278605 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2EC1914B7 for ; Mon, 9 Dec 2019 08:39:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EDCB32073D for ; Mon, 9 Dec 2019 08:39:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="T/Hcz0DH" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727370AbfLIIjc (ORCPT ); Mon, 9 Dec 2019 03:39:32 -0500 Received: from mail-lf1-f67.google.com ([209.85.167.67]:46849 "EHLO mail-lf1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727306AbfLIIjY (ORCPT ); Mon, 9 Dec 2019 03:39:24 -0500 Received: by mail-lf1-f67.google.com with SMTP id f15so9173831lfl.13 for ; Mon, 09 Dec 2019 00:39:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=E7OL9HCqlq6FTMnIkypwI+Orss8jA9UZv9drDoXCN7g=; b=T/Hcz0DHy/jPRqHKJPkvu53lgWdRoJhzWNiW+rwmc/+guJDxpMbmQ+DQs47qW+aNBK iw/VnFeS0pgtq2UHwHb/AuZPVzEDe90Fh5k9V3YE5+Azq/3duK9HcXs6EiRiSEBAZdOB tTPnO614bWAGhncbEsHNp2Dmupm/bYtCFlSGHlQZRFU6yfnNaCu7qWcXQ8BGSI9vBUT9 sgh5ndy+i28vfnryE8GOzD894SIJI+pXL0Lf5ngSjOlT9WXgiCtht/0HBtuWVUkZFJMR Pm2PkWKs+kc6WJJmY0lbpfF2kn3NNaGr0VBW9Nf4YiwOrpUUO4a9bi/WM24Ab/d7B1fV l0Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=E7OL9HCqlq6FTMnIkypwI+Orss8jA9UZv9drDoXCN7g=; b=Td2gvfTfDUjdHppzow2uvkKo3SfQmC4/QOLK9Frbr2SaZdr60UiSY8OXPd+itieMjd Ha5H3s6riNR2xbZTpDAxSvnMq/agfAI5IiXOXzqMPf5FPlHdQNGYzkU0PNU+fKCihKsA 8NVOAi8P2it7UYMr9Bg3CDzWHLEZdLEOqOYpQGiZbQ2tkt+J9qapAMguXwSF6D0LCpgQ nx+CKMiPAAlroaWoxPJ0AdYr27mDOw7WiOP07QCy7Pdvz1up7pdQoPAv4/VkKKRiqowz cZZrPoy8QIBf3mo3qMuAf0OXn/RXdkVc4sQwaPcWWI80Zwf139fp4UCTledWcDuFsoXl lc3Q== X-Gm-Message-State: APjAAAWrwLFkHec20wksfC6MBYltfOtj+CMJAYXR1Ur+3IBTJg7F3PsN vhdd+S4UAOu8WKvAa28oevcmpkqRjyg= X-Google-Smtp-Source: APXvYqz9xkSLnDb+4BqrI+ki6aeRrtCLVeW9AByrjOWC9meOxrhoxNU0ECrfOFl0eR5A7Y0iGbzQ2g== X-Received: by 2002:a19:5201:: with SMTP id m1mr13047865lfb.114.1575880762176; Mon, 09 Dec 2019 00:39:22 -0800 (PST) Received: from localhost.localdomain ([37.157.136.193]) by smtp.gmail.com with ESMTPSA id r26sm10438971lfm.82.2019.12.09.00.39.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2019 00:39:21 -0800 (PST) From: Stanimir Varbanov To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org Cc: Vikash Garodia , Tomasz Figa , Aniket Masule , Stanimir Varbanov Subject: [PATCH 6/6] media: venus: vdec: handle 10bit bitstreams Date: Mon, 9 Dec 2019 10:38:24 +0200 Message-Id: <20191209083824.806-7-stanimir.varbanov@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191209083824.806-1-stanimir.varbanov@linaro.org> References: <20191209083824.806-1-stanimir.varbanov@linaro.org> Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Aniket Masule Handle 10bit video streams in the decoder by using dithering, i.e the decoder output buffers will be in 8bit format. The runtime handling is implemented by sending v4l2 event to userspace application, then the application should stop the streaming on capture queue and initiate format negotiation, and start streaming again. Signed-off-by: Aniket Masule Co-developed-by: Stanimir Varbanov Signed-off-by: Stanimir Varbanov --- drivers/media/platform/qcom/venus/core.h | 1 + drivers/media/platform/qcom/venus/helpers.c | 97 +++++++++++++++++++ .../media/platform/qcom/venus/hfi_helper.h | 5 + drivers/media/platform/qcom/venus/vdec.c | 8 +- 4 files changed, 110 insertions(+), 1 deletion(-) diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h index 7efa5669dda3..8f71646df4c8 100644 --- a/drivers/media/platform/qcom/venus/core.h +++ b/drivers/media/platform/qcom/venus/core.h @@ -378,6 +378,7 @@ struct venus_inst { u32 session_type; union hfi_get_property hprop; unsigned int core_acquired: 1; + unsigned int bit_depth; }; #define IS_V1(core) ((core)->res->hfi_version == HFI_VERSION_1XX) diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c index 043e0d8705a8..a29662405e3a 100644 --- a/drivers/media/platform/qcom/venus/helpers.c +++ b/drivers/media/platform/qcom/venus/helpers.c @@ -626,6 +626,76 @@ static u32 get_framesize_raw_nv12_ubwc(u32 width, u32 height) max(extradata, y_stride * 48), SZ_4K); } +static u32 get_framesize_raw_p010(u32 width, u32 height) +{ + u32 y_plane, uv_plane, y_stride, uv_stride, y_sclines, uv_sclines; + + y_stride = ALIGN(width * 2, 256); + uv_stride = ALIGN(width * 2, 256); + y_sclines = ALIGN(height, 32); + uv_sclines = ALIGN((height + 1) >> 1, 16); + y_plane = y_stride * y_sclines; + uv_plane = uv_stride * uv_sclines; + + return ALIGN((y_plane + uv_plane), SZ_4K); +} + +static u32 get_framesize_raw_p010_ubwc(u32 width, u32 height) +{ + u32 y_stride, uv_stride, y_sclines, uv_sclines; + u32 y_ubwc_plane, uv_ubwc_plane; + u32 y_meta_stride, y_meta_scanlines; + u32 uv_meta_stride, uv_meta_scanlines; + u32 y_meta_plane, uv_meta_plane; + u32 size; + + y_stride = ALIGN(width * 2, 256); + uv_stride = ALIGN(width * 2, 256); + y_sclines = ALIGN(height, 16); + uv_sclines = ALIGN((height + 1) >> 1, 16); + + y_ubwc_plane = ALIGN(y_stride * y_sclines, SZ_4K); + uv_ubwc_plane = ALIGN(uv_stride * uv_sclines, SZ_4K); + y_meta_stride = ALIGN(DIV_ROUND_UP(width, 32), 64); + y_meta_scanlines = ALIGN(DIV_ROUND_UP(height, 4), 16); + y_meta_plane = ALIGN(y_meta_stride * y_meta_scanlines, SZ_4K); + uv_meta_stride = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 16), 64); + uv_meta_scanlines = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16); + uv_meta_plane = ALIGN(uv_meta_stride * uv_meta_scanlines, SZ_4K); + + size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane + uv_meta_plane; + + return ALIGN(size, SZ_4K); +} + +static u32 get_framesize_raw_yuv420_tp10_ubwc(u32 width, u32 height) +{ + u32 y_stride, uv_stride, y_sclines, uv_sclines; + u32 y_ubwc_plane, uv_ubwc_plane; + u32 y_meta_stride, y_meta_scanlines; + u32 uv_meta_stride, uv_meta_scanlines; + u32 y_meta_plane, uv_meta_plane; + u32 size; + + y_stride = ALIGN(ALIGN(width, 192) * 4 / 3, 256); + uv_stride = ALIGN(ALIGN(width, 192) * 4 / 3, 256); + y_sclines = ALIGN(height, 16); + uv_sclines = ALIGN((height + 1) >> 1, 16); + + y_ubwc_plane = ALIGN(y_stride * y_sclines, SZ_4K); + uv_ubwc_plane = ALIGN(uv_stride * uv_sclines, SZ_4K); + y_meta_stride = ALIGN(DIV_ROUND_UP(width, 48), 64); + y_meta_scanlines = ALIGN(DIV_ROUND_UP(height, 4), 16); + y_meta_plane = ALIGN(y_meta_stride * y_meta_scanlines, SZ_4K); + uv_meta_stride = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 24), 64); + uv_meta_scanlines = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16); + uv_meta_plane = ALIGN(uv_meta_stride * uv_meta_scanlines, SZ_4K); + + size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane + uv_meta_plane; + + return ALIGN(size, SZ_4K); +} + u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height) { switch (hfi_fmt) { @@ -634,6 +704,12 @@ u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height) return get_framesize_raw_nv12(width, height); case HFI_COLOR_FORMAT_NV12_UBWC: return get_framesize_raw_nv12_ubwc(width, height); + case HFI_COLOR_FORMAT_P010: + return get_framesize_raw_p010(width, height); + case HFI_COLOR_FORMAT_P010_UBWC: + return get_framesize_raw_p010_ubwc(width, height); + case HFI_COLOR_FORMAT_YUV420_TP10_UBWC: + return get_framesize_raw_yuv420_tp10_ubwc(width, height); default: return 0; } @@ -1235,6 +1311,27 @@ int venus_helper_get_out_fmts(struct venus_inst *inst, u32 v4l2_fmt, if (!caps) return -EINVAL; + if (inst->bit_depth == VIDC_BITDEPTH_10 && + inst->session_type == VIDC_SESSION_TYPE_DEC) { + found_ubwc = + find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT, + HFI_COLOR_FORMAT_YUV420_TP10_UBWC); + found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT2, + HFI_COLOR_FORMAT_NV12); + if (found_ubwc && found) { + /* + * Hard-code DPB buffers to be 10bit UBWC and decoder + * output buffers in 8bit NV12 until V4L2 is able to + * expose compressed/tiled formats to applications. + */ + *out_fmt = HFI_COLOR_FORMAT_YUV420_TP10_UBWC; + *out2_fmt = HFI_COLOR_FORMAT_NV12; + return 0; + } + + return -EINVAL; + } + if (ubwc) { ubwc_fmt = fmt | HFI_COLOR_FORMAT_UBWC_BASE; found_ubwc = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT, diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h b/drivers/media/platform/qcom/venus/hfi_helper.h index 2a47f6c0a07a..0b4597c73fb5 100644 --- a/drivers/media/platform/qcom/venus/hfi_helper.h +++ b/drivers/media/platform/qcom/venus/hfi_helper.h @@ -793,6 +793,9 @@ struct hfi_h264_vui_timing_info { u32 time_scale; }; +#define VIDC_BITDEPTH_8 0 +#define VIDC_BITDEPTH_10 1 + struct hfi_bit_depth { u32 buffer_type; u32 bit_depth; @@ -841,8 +844,10 @@ struct hfi_extradata_input_crop { #define HFI_COLOR_FORMAT_10_BIT_BASE 0x4000 #define HFI_COLOR_FORMAT_YUV420_TP10 0x4002 +#define HFI_COLOR_FORMAT_P010 0x4003 #define HFI_COLOR_FORMAT_NV12_UBWC 0x8002 #define HFI_COLOR_FORMAT_YUV420_TP10_UBWC 0xc002 +#define HFI_COLOR_FORMAT_P010_UBWC 0xc003 #define HFI_COLOR_FORMAT_RGBA8888_UBWC 0x8010 struct hfi_uncompressed_format_select { diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c index 7b7a4f3377cf..eeff0d8e46b2 100644 --- a/drivers/media/platform/qcom/venus/vdec.c +++ b/drivers/media/platform/qcom/venus/vdec.c @@ -1172,7 +1172,7 @@ static void vdec_buf_done(struct venus_inst *inst, unsigned int buf_type, static void vdec_event_change(struct venus_inst *inst, struct hfi_event_data *ev_data, bool sufficient) { - static const struct v4l2_event ev = { + struct v4l2_event ev = { .type = V4L2_EVENT_SOURCE_CHANGE, .u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION }; struct device *dev = inst->core->dev_dec; @@ -1193,6 +1193,11 @@ static void vdec_event_change(struct venus_inst *inst, inst->out_width = ev_data->width; inst->out_height = ev_data->height; + if (inst->bit_depth != ev_data->bit_depth) { + inst->bit_depth = ev_data->bit_depth; + ev.u.src_change.changes |= V4L2_EVENT_SRC_CH_COLOR_DEPTH; + } + dev_dbg(dev, "event %s sufficient resources (%ux%u)\n", sufficient ? "" : "not", ev_data->width, ev_data->height); @@ -1340,6 +1345,7 @@ static int vdec_open(struct file *file) inst->buf_count = 0; inst->clk_data.core_id = VIDC_CORE_ID_DEFAULT; inst->core_acquired = false; + inst->bit_depth = VIDC_BITDEPTH_8; init_waitqueue_head(&inst->reconf_wait); venus_helper_init_instance(inst);