From patchwork Mon Feb 3 16:25:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Basavaraj Natikar X-Patchwork-Id: 13957777 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2051.outbound.protection.outlook.com [40.107.101.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6130820A5FD for ; Mon, 3 Feb 2025 16:25:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.101.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738599946; cv=fail; b=TKUPFNUU/5cBnGV3gFXj4sxsVxjtl3QVAXVmgxD6w+lnd+gQ7cEbwR4BQJTKw6la6Gk9JMb0cy34Shr6qnG3LII3b+qzyObcAB6po0LL67IaYbDdc1NZ70jd9rmV6QU0SaCL21KHcf4MfuVRMki2iZLoCRaEoOUjSIdjPujYBgs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738599946; c=relaxed/simple; bh=MGlVTOFtGkM7Jc6EVxVHh4cYHrzhXeyZ9FoxuWJ3Iww=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FG5tk/EAh2canDkSTmjPAHzIc+C0Nlv0zYqxuqlEL3PqcFdeWYncSmUKS+Na3Px8Pl+1Hd89iZ43POkK0Vu+ufeDKnLnyHhp8nDcTvLnKrwclQr6Z+Dh9YEJi6hCaVpxgVG3WxCPjcpDYQ09JTHkKfBK0VcPXTzH7TttBxOKoXw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=KCUJCCBm; arc=fail smtp.client-ip=40.107.101.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="KCUJCCBm" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=k3Evt5B5tHj7DkuTnxP3F4Pu+SsNlFsHsMrgauwu9yjv/Vt97tKcYW8dfidxaK5bxG2a+6ermFRUX863nYzq/s1TUnTBidSxuOkDTwOtZEakqn6UsU1FF8WxVX//yF7eaWdbv+4zB3osmQgimddaT3fpOmsS+/os4Hx274/RaqKLayxhMxd34ZNdvwuVTfIKEppnNNAql47EOcxbLLWGBRTRNtF8YO6DEZaCNHQ7CJOrFt8J0C3Qauu+rbru/ybOE7cV57tb4EHTzwitBDmsxvetvviL91DR09wpaCZ5o+8SbMCIqGIHqYPkV6vGmVZU0exrvFr/muRjD8tpIQU63Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KDWHaf6h0dphshqWg7GuQjVFt+0fz/6RZhRrdaJd0g8=; b=LAYC30TS4c60ANqOUPv/FTlMpoIOVIRDm9DzVdWVdGnv3JYpOtOUxea1/jCl1RRi123/dyPP+J5o26evGO0Gl7SBeLsGgu/SuyjqEwKme1PjlqDw+HtiGEfo7ZHh5BWaTYYDZ+/gqqmomDhnr8adjUHdGZnTb7TTXZwbnAn9bdW7zfsNCIoXZPhvvg3AjcNqIGTOmcy3sKCopBE1acFaiTSd3eU8AYsgKY6FeGHZ8bHJpucpIDRy7E1RsMseRqZEsT4nJSHmnYENo14qSYIUQuiGbjebVJMNbuaiDOmfFGRxfgc7OD4Z5IW6ssobXOUuP9wdWJ1ClkDhemBc4xmZaA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KDWHaf6h0dphshqWg7GuQjVFt+0fz/6RZhRrdaJd0g8=; b=KCUJCCBmCm2tIOHi1TgDPNWweipt8S4nKxi1/Gv8SGv/h3ncnO+eszVSsDOpTq89oOcpgvmMg4PDtjaJ4vFimLDpLiwXtes3CAE6fRSqEz+4+9wmjCsxP8G36kXlQOI6iJDwtt/uSnQm7Ke1zRuZRu+0d8S+htkkcLo/jzUGvOo= Received: from BYAPR07CA0021.namprd07.prod.outlook.com (2603:10b6:a02:bc::34) by PH8PR12MB6842.namprd12.prod.outlook.com (2603:10b6:510:1c9::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8398.25; Mon, 3 Feb 2025 16:25:39 +0000 Received: from MWH0EPF000A672F.namprd04.prod.outlook.com (2603:10b6:a02:bc:cafe::d6) by BYAPR07CA0021.outlook.office365.com (2603:10b6:a02:bc::34) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8398.22 via Frontend Transport; Mon, 3 Feb 2025 16:25:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MWH0EPF000A672F.mail.protection.outlook.com (10.167.249.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8398.14 via Frontend Transport; Mon, 3 Feb 2025 16:25:38 +0000 Received: from jatayu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 3 Feb 2025 10:25:36 -0600 From: Basavaraj Natikar To: , CC: Basavaraj Natikar Subject: [PATCH 3/3] dmaengine: ptdma: Utilize the AE4DMA engine's multi-queue functionality Date: Mon, 3 Feb 2025 21:55:11 +0530 Message-ID: <20250203162511.911946-4-Basavaraj.Natikar@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250203162511.911946-1-Basavaraj.Natikar@amd.com> References: <20250203162511.911946-1-Basavaraj.Natikar@amd.com> Precedence: bulk X-Mailing-List: dmaengine@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000A672F:EE_|PH8PR12MB6842:EE_ X-MS-Office365-Filtering-Correlation-Id: 8f63e8ce-a809-44a2-188e-08dd446f655d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|1800799024|376014; X-Microsoft-Antispam-Message-Info: =?utf-8?q?GJUMxHKyRck/2RRsbpBzKMlUiCMApJd?= =?utf-8?q?1nqcb+e/8yBSAZ/phDjZuk/bQ8/wgfNFN9HxfZR7iDbz8Ofq1bCVDRgsuZcxLeMDf?= =?utf-8?q?bF8IxXlQI1GXsJNGVNCQYU+e8n36gO72i/yb+k6e3EEZbB2JpUGYaDAHWs676hEIX?= =?utf-8?q?0oMMEVyD3QmqFi6xfyA/hOvW36Ex+Dnd8t6Jsbjo9YzS0xDFPoVth3Is+78+KwqYZ?= =?utf-8?q?WdCd5znY3zAI2gRBU5PLvPwxdWuVHBlQfvZ8KTp7tUO7gUo2op1B7vOLQoa9ldBop?= =?utf-8?q?/DSefqK134JMf7DNvVYcY8puPSxf/Bcul5Z2HVyDeaBfQeMvByvkGYSOaVCgcj7Ni?= =?utf-8?q?8z3De0/VP0o2wIqG/+OJL79hJyQI8aUzUd/pzwF2p0YXo9pNFU6qOSNlCXLx1akRf?= =?utf-8?q?ll/ObQYfxMq8d3rFRZSODp43dAsH1eZhY663Gdoec9j5hnZC4HssHZp3Z9PBuo4K8?= =?utf-8?q?ZZ79DC6xQfpRYGkEhBeLdPxOEMALwMTzKvMCkLllRVI+L9WtEvgByYTUKSo6daX6+?= =?utf-8?q?S11kWHCBZBLh2BTLaylfkws9uo32xMfoJw5oHq/i4rPyIDe1+YM975TKaRpgBIg2R?= =?utf-8?q?2a51zk1COtBnXn9RS7fySBt4OSf1Dn4PI3MbdfIFTxGVnlN7wPNOGHc0IETFtyk1F?= =?utf-8?q?HKvwh1xrIFSlx1Kn4oPZmbitzIhvg1x0BVicb8LPq9miuBObxj92DCLBfo/1Bqr7r?= =?utf-8?q?+zpGcvtoNSI1qZGUNLo5nO93zYW4lNgWY2PB9Er++60aM24spe35pijFSm29dslPN?= =?utf-8?q?voNd39lIpLwUgdm/06/iW74UdrdkwLXUF7ua72DofgjGe3hyhNB45TZ36YLHfaaJJ?= =?utf-8?q?oXCq7nYXepqGhPub8FTSHLBjn39vP0kConZvzbMA2Hnq+PWBtIY9sEWrr+fPzANGx?= =?utf-8?q?dEv5p04FhOZ1sY24/I7X2QmK1b+tdKPR4tgeEpJKCt36bK6T7RakZUgm6gY3DXREJ?= =?utf-8?q?zg/o/cFWPr7HFH6UAkZlwiqYaQoHhTKW34tyBugS1RT6C+nMz+MIP0Q7xQFBAlZxZ?= =?utf-8?q?W044IZuWa+dIPEgwFh3aBhciAx+vCrCU/DgNVgtwa+x9wXDejM1uUu8WUQVhc5w/9?= =?utf-8?q?BtfFvlozAszHRab3ZgxzuWYS63bPX/rQ+5nKcXMF7ULbWdv3bsIbJNiZee7bjPSpi?= =?utf-8?q?t//3wPTsHqAfWdaUkJ+unyzc13nKVNV5IXjOtJTWABYnFGi1Em++isOgZMZusgzm/?= =?utf-8?q?TtDmPglkl3AzSgasJNnCnMRxkTGnGjdJEYBGPH1zLw3QJvO0pgtjB/B+SBULR10sz?= =?utf-8?q?FF2CxYAi4yQkNCyGktsLAPhv2Kps870Yx2DOJ+XPBcGj3qPLcr7XJulHf4E2tR8WC?= =?utf-8?q?bK1V66WxAOkYhB6meq3SqJpA+4sXmSkoOwyUjROKOBgX6Ape2fOMxeOMNfC7/U7td?= =?utf-8?q?Ljl9NDOg5dP+Rwye0RbQ0cWaYVZKB1ZtrTAGbD5Pq6pQy0UHHdY2P0=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(1800799024)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Feb 2025 16:25:38.7574 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8f63e8ce-a809-44a2-188e-08dd446f655d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000A672F.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB6842 As AE4DMA offers multi-channel functionality compared to PTDMA’s single queue, utilize multi-queue, which supports higher speeds than PTDMA, to achieve higher performance using the AE4DMA workqueue based mechanism. Fixes: 69a47b16a51b ("dmaengine: ptdma: Extend ptdma to support multi-channel and version") Signed-off-by: Basavaraj Natikar --- drivers/dma/amd/ae4dma/ae4dma.h | 2 + drivers/dma/amd/ptdma/ptdma-dmaengine.c | 90 ++++++++++++++++++++++++- 2 files changed, 89 insertions(+), 3 deletions(-) diff --git a/drivers/dma/amd/ae4dma/ae4dma.h b/drivers/dma/amd/ae4dma/ae4dma.h index 265c5d436008..57f6048726bb 100644 --- a/drivers/dma/amd/ae4dma/ae4dma.h +++ b/drivers/dma/amd/ae4dma/ae4dma.h @@ -37,6 +37,8 @@ #define AE4_DMA_VERSION 4 #define CMD_AE4_DESC_DW0_VAL 2 +#define AE4_TIME_OUT 5000 + struct ae4_msix { int msix_count; struct msix_entry msix_entry[MAX_AE4_HW_QUEUES]; diff --git a/drivers/dma/amd/ptdma/ptdma-dmaengine.c b/drivers/dma/amd/ptdma/ptdma-dmaengine.c index 35c84ec9608b..715ac3ae067b 100644 --- a/drivers/dma/amd/ptdma/ptdma-dmaengine.c +++ b/drivers/dma/amd/ptdma/ptdma-dmaengine.c @@ -198,8 +198,10 @@ static struct pt_dma_desc *pt_handle_active_desc(struct pt_dma_chan *chan, { struct dma_async_tx_descriptor *tx_desc; struct virt_dma_desc *vd; + struct pt_device *pt; unsigned long flags; + pt = chan->pt; /* Loop over descriptors until one is found with commands */ do { if (desc) { @@ -217,7 +219,7 @@ static struct pt_dma_desc *pt_handle_active_desc(struct pt_dma_chan *chan, spin_lock_irqsave(&chan->vc.lock, flags); - if (desc) { + if (pt->ver != AE4_DMA_VERSION && desc) { if (desc->status != DMA_COMPLETE) { if (desc->status != DMA_ERROR) desc->status = DMA_COMPLETE; @@ -235,7 +237,7 @@ static struct pt_dma_desc *pt_handle_active_desc(struct pt_dma_chan *chan, spin_unlock_irqrestore(&chan->vc.lock, flags); - if (tx_desc) { + if (pt->ver != AE4_DMA_VERSION && tx_desc) { dmaengine_desc_get_callback_invoke(tx_desc, NULL); dma_run_dependencies(tx_desc); vchan_vdesc_fini(vd); @@ -245,11 +247,25 @@ static struct pt_dma_desc *pt_handle_active_desc(struct pt_dma_chan *chan, return NULL; } +static inline bool ae4_core_queue_full(struct pt_cmd_queue *cmd_q) +{ + u32 front_wi = readl(cmd_q->reg_control + AE4_WR_IDX_OFF); + u32 rear_ri = readl(cmd_q->reg_control + AE4_RD_IDX_OFF); + + if (((MAX_CMD_QLEN + front_wi - rear_ri) % MAX_CMD_QLEN) >= (MAX_CMD_QLEN - 1)) + return true; + + return false; +} + static void pt_cmd_callback(void *data, int err) { struct pt_dma_desc *desc = data; + struct ae4_cmd_queue *ae4cmd_q; struct dma_chan *dma_chan; struct pt_dma_chan *chan; + struct ae4_device *ae4; + struct pt_device *pt; int ret; if (err == -EINPROGRESS) @@ -257,11 +273,32 @@ static void pt_cmd_callback(void *data, int err) dma_chan = desc->vd.tx.chan; chan = to_pt_chan(dma_chan); + pt = chan->pt; if (err) desc->status = DMA_ERROR; while (true) { + if (pt->ver == AE4_DMA_VERSION) { + ae4 = container_of(pt, struct ae4_device, pt); + ae4cmd_q = &ae4->ae4cmd_q[chan->id]; + + if (ae4cmd_q->q_cmd_count >= (CMD_Q_LEN - 1) || + ae4_core_queue_full(&ae4cmd_q->cmd_q)) { + wake_up(&ae4cmd_q->q_w); + + if (wait_for_completion_timeout(&ae4cmd_q->cmp, + msecs_to_jiffies(AE4_TIME_OUT)) + == 0) { + dev_err(pt->dev, "TIMEOUT %d:\n", ae4cmd_q->id); + break; + } + + reinit_completion(&ae4cmd_q->cmp); + continue; + } + } + /* Check for DMA descriptor completion */ desc = pt_handle_active_desc(chan, desc); @@ -296,6 +333,49 @@ static struct pt_dma_desc *pt_alloc_dma_desc(struct pt_dma_chan *chan, return desc; } +static void pt_cmd_callback_work(void *data, int err) +{ + struct dma_async_tx_descriptor *tx_desc; + struct pt_dma_desc *desc = data; + struct dma_chan *dma_chan; + struct virt_dma_desc *vd; + struct pt_dma_chan *chan; + unsigned long flags; + + dma_chan = desc->vd.tx.chan; + chan = to_pt_chan(dma_chan); + + if (err == -EINPROGRESS) + return; + + tx_desc = &desc->vd.tx; + vd = &desc->vd; + + if (err) + desc->status = DMA_ERROR; + + spin_lock_irqsave(&chan->vc.lock, flags); + if (desc) { + if (desc->status != DMA_COMPLETE) { + if (desc->status != DMA_ERROR) + desc->status = DMA_COMPLETE; + + dma_cookie_complete(tx_desc); + dma_descriptor_unmap(tx_desc); + } else { + tx_desc = NULL; + } + } + spin_unlock_irqrestore(&chan->vc.lock, flags); + + if (tx_desc) { + dmaengine_desc_get_callback_invoke(tx_desc, NULL); + dma_run_dependencies(tx_desc); + list_del(&desc->vd.node); + vchan_vdesc_fini(vd); + } +} + static struct pt_dma_desc *pt_create_desc(struct dma_chan *dma_chan, dma_addr_t dst, dma_addr_t src, @@ -327,6 +407,7 @@ static struct pt_dma_desc *pt_create_desc(struct dma_chan *dma_chan, desc->len = len; if (pt->ver == AE4_DMA_VERSION) { + pt_cmd->pt_cmd_callback = pt_cmd_callback_work; ae4 = container_of(pt, struct ae4_device, pt); ae4cmd_q = &ae4->ae4cmd_q[chan->id]; mutex_lock(&ae4cmd_q->cmd_lock); @@ -367,13 +448,16 @@ static void pt_issue_pending(struct dma_chan *dma_chan) { struct pt_dma_chan *chan = to_pt_chan(dma_chan); struct pt_dma_desc *desc; + struct pt_device *pt; unsigned long flags; bool engine_is_idle = true; + pt = chan->pt; + spin_lock_irqsave(&chan->vc.lock, flags); desc = pt_next_dma_desc(chan); - if (desc) + if (desc && pt->ver != AE4_DMA_VERSION) engine_is_idle = false; vchan_issue_pending(&chan->vc);