From patchwork Wed Apr 20 11:34:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Qian X-Patchwork-Id: 12820118 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DA7D0C433F5 for ; Wed, 20 Apr 2022 11:52:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=9eiMO1HsztZsQNtNkzzItvmOTCreNdH5Lv6D9iZy05I=; b=gyri+kH53zkhZ8 yIzL8bP6+zgoTzTY3oa20T+oZ4HorwmWmevQEhZdTjSNFRmdmVUgIzGEq8fQxvzBfscW3zb7Qtx4K TLx7CVoRV6AiPW16wLxS40pJ2CIlX5p/9RZ4WC/K/JeOcjTH8pbJXhhFptsjoO9S0GU6ZhM2bJp8R 9aFDKHyjElrEAeXhsVFSs//2CPmabHdlCK0gg4kxRLYwbuDyzlnvMVuOhIHK6H9AjSSOaBQa4C0Ep J2GBBqWxvV75bY8WKAkosVFiD0bXQaOUaB/pDO7OXM1eYi8zWG5bSkKAlDdqrI6kii2ZNzx5MxO0O FTPoT/pY6zGWXJJ12vsA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nh8rp-008qnw-4g; Wed, 20 Apr 2022 11:51:29 +0000 Received: from mail-am6eur05on20610.outbound.protection.outlook.com ([2a01:111:f400:7e1b::610] helo=EUR05-AM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nh8rl-008qmG-N7 for linux-arm-kernel@lists.infradead.org; Wed, 20 Apr 2022 11:51:27 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VH3+DoOND4XBX/hBjskImz3t9rswCKP+rBDI40QVHn4is4SZe0JLbJU2mW4sGGfKrGNTYVDywaWcKgtD9jvnFPGwds2eH1N881ojz6e82tDEjcefvVIuB9SeWtEQu573omxvjD9LmduiCLwejbxwQJUAsOkx2aLc/Wzvi3MTjlVUmh2nv3UT7VochPxmLay63datCmw3itASBC+UwEXBhjwqWPcDXWs0BMmZRlBgSQnekmA2CEM8wMt5L4bcyodoKJ2eJpzBfkurqEB+IT3q/nWIcQ0Io+k6Bai/QH+X3KNlmyASIYiu7pvUjjT2DJTzs+zXBZyHlaQP1TF6kG+iKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=b+I9FEztKWd4Bwr+DteqvAJpkrWIAnN6WtfEbArW0aI=; b=f9By44c6py18v2BasvOClioVN50zexg6ie19gFx7JqKgE2F/f3oO9qzEeRIuDlKcDd7wY8LN9fQaB2tL8fhnUr1gYezoC5btAA+zFn9nZvPk8A8T4pAodcR611emgk1Ks/YbQV8tTjNvqFxztobMO8a+BgpZ6y60aLPAlqjlGx5DKIpZbHv6Uoqe0h7LOpcKio90QyUEgtDxlD3Xvf15G9QYTYZ8uUQbBu4AhwkaJy6hoL2iSgbsrtYQUUp9NmAB8iwn6uHSrgZadZHBFz6PFXSTqTbZ4XEEMPS7nLzviE9NviKvjYrpdUIQGvPhMaax2S789gG3B4TrplJDpbF4Iw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=b+I9FEztKWd4Bwr+DteqvAJpkrWIAnN6WtfEbArW0aI=; b=VpJHxJ39LghWcAtXdGSA2eEqQ5BC0aLn0QzhdHdajX91GjLFlSU452mpma++GI6x9z1tlLL/t+BiOT6A5chAWmyPdBnT/mL/qEvIFYda6OQhRv8ovWkgdZv0ORTtTgWGXUKSiy0e1Dt66eAurHn+ZUxvEqDPIHE/IChxfST8BxQ= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) by VI1PR04MB5376.eurprd04.prod.outlook.com (2603:10a6:803:cd::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.13; Wed, 20 Apr 2022 11:35:34 +0000 Received: from AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::d8b5:5914:5189:ffa5]) by AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::d8b5:5914:5189:ffa5%6]) with mapi id 15.20.5186.014; Wed, 20 Apr 2022 11:35:34 +0000 From: Ming Qian To: mchehab@kernel.org, shawnguo@kernel.org Cc: hverkuil-cisco@xs4all.nl, robh+dt@kernel.org, s.hauer@pengutronix.de, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, aisheng.dong@nxp.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 0/2] media: amphion: copy timestamp from output to Date: Wed, 20 Apr 2022 19:34:52 +0800 Message-Id: X-Mailer: git-send-email 2.35.1 X-ClientProxiedBy: SI2PR01CA0029.apcprd01.prod.exchangelabs.com (2603:1096:4:192::8) To AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 833b259f-148d-4c99-7f15-08da22c1e20c X-MS-TrafficTypeDiagnostic: VI1PR04MB5376:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: UOuuwn1p/7doBjef5FV9FNscYg3rY/ZuRGJyK3BeDy9vYZt67YqpdtyFSGyaaMaS12kLRGp3ZqitMo8JtPFF1aZwxOgF8GtatKS5O8KLxKKmWB5/cHWNREhuy58v4ztn4/Aa43SuQCxIj6wVtefUxZM+EVQ9YjLhG2Mm5xt7CM2/6PKe2GRhjoF4SCdU/mhPhWhYIw6l+SvaIcJeL8vHxFrCmtZUsqcA0l5TQ5nyFYQbHL/YQpyZBsz0E7Aa/ef2w5CAX8MtgGZj7R813p7KqE6ZrxTH/bOVauRov5yX/iLJPw9z9wAzAcMFBdMWrzac5zLVsh2pLLZKtUd3OEkSjXewz097G5EJItJHXR/G3pC/b4CfB6ql43gWaqqHBTjuT24ULj9FVJoVPhX45drwg+eK3OXvVoKZm48vfAeRwAEJXT+sCFpYZYzvU62mD/ldNeZx3pnzCDL5R8//4b+w/LKCuTT29loFWniZ8CPTNY06RRpMXKeDcLzYS55fnRRoqInTWXewuWA9MRkXAlXEqKEYDGniR8aX7oNrjz5meK+jbGeElWF9sw4gauXn2/pAcqVH1L5hrFgeTMHPGTSzmnqVqf9Dfqs4RZ58qEMilAyExtM3fTw4UjGjM17kXH3AnQib8bukYtLjTPREn5Agc9WcLkqzc1Ty3ilBTFnUobjfx8gNDB9h5ToUvzppm5S6tkZ1hA8RZu0cfilrOhaRUA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR04MB6341.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(52116002)(6506007)(6486002)(508600001)(86362001)(2906002)(6666004)(38350700002)(38100700002)(316002)(26005)(186003)(2616005)(36756003)(7416002)(5660300002)(66946007)(8676002)(4326008)(83380400001)(44832011)(66556008)(8936002)(6512007)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: oSc58la1Ii3OPrag4NjSRTVQWLZeGYU3C4AkBUAmVQ8VobTpFsCux+0HEiSvOX4ogTcZYmGPIWWJPPlqOIBStvYY12CDsdGUvTP+4gyG5FAQPWBgaMm/k1fhMDD06J1wmMOJiBy/Hgwak79iz4i2/b+UVsWov40PrSA6KO07pYxkhJgrK0FEl5aAcH9jn1P1uRp8wzZlMAZKP6KDxdmO+weI5pdxEbXH9Re6xkzynAwtBVnHhh8Td6nxojqNefU6DGIYQ7xNwcb33IMn0ePaPoq7w8XOjHItJ5ZxJwcdQItDEyarDGluHcWXIA+k1RJL89UJB8+zmCsD3SJ3wdp9MPe8QltfYc1vIe+gOFPB804lvxIafYG0nVZ5NAEOCOerIlFw12YQwjOEa67mmp2lnwp7KuKr2qXsCkfxFziWSi+RkYj61XSKNAz3PkbaQ80wUNBRnGUybrAmKfeH5UDx9wET41/UL8L7Zg+3pTtXDLv8rzzaRFmUmgyekM9eB7xW/+L+BWQKaeIsSZDodcjDM6sDdFu6D2GoZkA4TPBi/JvhcoQcPgx3ephoyAGCvpua/Vt/mTsVV/0HV+qB+Fpa1+IxXpse7EAiaAGL4uY4vVtC9mLhH9A3uUaOaPT2lBiBFhmiD86Tsak3j/8V3loV6M4HYef2k8YMOhupJ5ZiJMZIO87rrRq17hwBR8YbxofNuwjJWkqXvU4vsyIl1h390kFXAI52iNnp4syp0GzWdNWMPu355QuVHT9l5J+uEM14WJ24TKKWMJX8liggk1fL9Du0pGs4WaCzfGuoTo+Mim89lkq84pxO+X+r+IWumgsX58nrpZRbJ55EosFqNGbvweN09QEBiJ6sopKqKWRVR0K5WH6bBiha/vkGlvFPnLcMQv5CmaeYFCe9CMEroGY2qTZN22Qw9pD8gVpP4T3jTLiWiaim02SjQCsgl+zUhoRJEnZlwmleB8y1yXcGM3ZHIAy8+ccCw9ofnatHsc2N312lLJ7XKrol0NbvTBSfQABV4w0wFabfcBwdu7zZzs2Ja4jcCnlKiiGWsPA38fzYY6prOIB5adG8j1yWshEoMctJQ5Fah0TefYpZMjenwpdhtyF3HCP31ZyGBqveHf4ED3qZJEOWRjYEvt5Fu2UZ4VdhHvZDJ3pf6zYgOFrrVsn/vM/QtGvEpxlW+m8mYP0D2gzBaXxmNc4ZFbgacqUrFx/WGrCTFjeue+9iAlK8LKQZdDLeFxWRg5vx8MxVFAFY4VWfW54JOnpqV1ySNwH4RkoanI9yOepmnXElU7SU3jMWpEZkReFIALX5wos4kh/1qTPTgslegiaOd66NGXdu9s41KbBX/PGuuckBiydin7n/RGOoxVM73A1Gt6XpxGbjdyLg/LKgwQnDa6yhOQjN2D54JGSN5K8ABdBQEMFB9eVic2aYBu8eN+Dq0XFlc7WGglT7NYtZ4rEeadQPwdDOgGcXqx3SSoTVWfemFFLhGsA8czQaWobo8r6GVg7rZEyDAAVsqO+On9EGSvL7ef4m+m4KnL7zrCw/KR0imh4Ktm7G7zxOAUNA+ZVv3U108HqDtkYpVGzaIOQCpgk7CC41lLyGHWDouw0+HRx61P3cQYq5eJ3ZGLzD41/NgjgDB4EbEUGDPG48kF1JL3x8PKGeygEodQc+kPflTO1RxyxBODqFGhwx5YhdxEuzQ1JYRGMgLQtrykIfHn7wgoH2MVYk1gjFdhMRavQkWcc80Cu5Y++b3w== X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 833b259f-148d-4c99-7f15-08da22c1e20c X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB6341.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2022 11:35:34.4567 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: T4gMFZRu1ruBNCfN3T2gTJ4jdWD/GEN9tp/X4NPFyz6CW1Mu1w83jJTeH7DKqezkJKFp4Jf1r1Ga+SiXw21YnA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5376 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220420_045125_789358_BD276E6C X-CRM114-Status: GOOD ( 10.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org v3 - add helper function vpu_skip_frame() and vpu_next_src_buf() v2 - reduce the min buffers count There is a timestamp manager in firmware, so in previous, driver will pass the timestamp to firmware, and when a frame is decoded or encoded, firmware will pass the timestamp to driver, then driver set the timestamp to the capture vb2_buffer. But there is problem that the timestamp manager will ensure that the timestamps are monotonically increasing. So if there are reorder frames, the capture buffer's timestamp is inconsistent with the timestamp of the corresponding output buffer. It goes against the description of V4L2_BUF_FLAG_TIMESTAMP_COPY in the specification In order to comply with the specification, driver should copy timestamp from output buffer to capture buffer, and there is an helper function v4l2_m2m_buf_copy_metadata to handle it. Ming Qian (2): media: amphion: decoder copy timestamp from output to capture media: amphion: encoder copy timestamp from output to capture drivers/media/platform/amphion/vdec.c | 76 ++++++++------------- drivers/media/platform/amphion/venc.c | 47 +++++-------- drivers/media/platform/amphion/vpu_malone.c | 2 +- drivers/media/platform/amphion/vpu_v4l2.c | 56 +++++++++++++++ drivers/media/platform/amphion/vpu_v4l2.h | 3 + 5 files changed, 103 insertions(+), 81 deletions(-)