From patchwork Thu Aug 1 10:45:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huan Yang X-Patchwork-Id: 13750174 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2AC5FC3DA64 for ; Thu, 1 Aug 2024 10:45:40 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 454B710E8F8; Thu, 1 Aug 2024 10:45:35 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=vivo.com header.i=@vivo.com header.b="VGGU+g+9"; dkim-atps=neutral Received: from APC01-PSA-obe.outbound.protection.outlook.com (mail-psaapc01on2043.outbound.protection.outlook.com [40.107.255.43]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1CC7F10E8F6 for ; Thu, 1 Aug 2024 10:45:34 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=LOqJsYIpbgd4tAxIhlM3/+mxWt06akfggV6v5wfWd+WOV2J5nNgGlM4Lk2Gu4TSfrSsY5XLVtEl4UcROy7Wjn6C5PXvyCo9iOzkKy6mnE7z8BgEyJ2IAkpq18s1t8fz+EKSRa0FjI4Sz75MRXaQ5tQshPlLoOsKFJc2c0wrOojcUFm+8hTcCpSdClZgSYyAEcxbT12yzZglxx/6gUm5F5nDErxDq+I/jg36x6VcCDQC9cHvU+1L5oWXNVaouwgXCtWjJk819zNj1uPEtVOs5vWsrttmpgWiE6LssRC9OmbMRusCGTszGjNfHAzPxpHmI3zkB7SNtygWdisEXdOSKpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XGNxavGBonMGVhAMEsLOUEdKXKTo0xNEi6YQPpzYGHg=; b=JK/TxzeLwHWmjmfZubCInhmRroTLYZslzX6NWDZHZ/nP1RFN8/nyC+yQdPKMG/gYVwGVb8OjyQvh25b1ekKz0geMTkspsfBMUV2V95kAwXYO/AVjytk/r4vVZgtNKgQwXvOYsuOPS1rdG+jynC+4zaknuQtL/U2zckGYrmyZ22+M8ycyxOvY340JMfRySm7bj6Jqgdnq03asHoIK1s/uwLA86I4h+DqoP5MRrlQ2M/niB1OxEoPsH5wJnZYpbkYAEJAcq/KWsgTBQEXFDOlHz5UlD+d5lgmPHAzjq4HwriqfGxu/GrzBCHymmO5GIgLf1jkV1rd++iiDFZupoYZTmw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XGNxavGBonMGVhAMEsLOUEdKXKTo0xNEi6YQPpzYGHg=; b=VGGU+g+9FesUybzCztGfDy+gdjd6yhSm1z1hVOFGHs8tNYZaVnHi1+T8Zlhjx79nZwxH/vFQ/liJIdi2MJdNfGATqQpw7ck/cagQfjEtuhAraA8TsgYtDb90n2IUarARwWIet2N0AoMlMaEMwx5PrGXff3BJFDJ/K5ypMPRRpbzMSQ4y0wkA0pf7tlBE2f2BDnEGNfkiQRE8GTkaUBCYR+zwJITrkNPqpHx7r6La3hwawVSHIoZUhF/LZeQuvr9+3Wdr/VhuTDgJcxkasu3A3nBogb+L2W8RNgPPT5tLw4xBpHfgaze7FspoYHCIrSKOhXEfBUO6e+0B8oM6h9JBTA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by SEZPR06MB6304.apcprd06.prod.outlook.com (2603:1096:101:10d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7807.28; Thu, 1 Aug 2024 10:45:25 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%4]) with mapi id 15.20.7828.021; Thu, 1 Aug 2024 10:45:25 +0000 From: Huan Yang To: Gerd Hoffmann , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= , dri-devel@lists.freedesktop.org, linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org Cc: opensource.kernel@vivo.com, Huan Yang Subject: [PATCH 1/5] udmabuf: cancel mmap page fault, direct map it Date: Thu, 1 Aug 2024 18:45:05 +0800 Message-ID: <20240801104512.4056860-2-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240801104512.4056860-1-link@vivo.com> References: <20240801104512.4056860-1-link@vivo.com> X-ClientProxiedBy: SG2PR06CA0209.apcprd06.prod.outlook.com (2603:1096:4:68::17) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|SEZPR06MB6304:EE_ X-MS-Office365-Filtering-Correlation-Id: d0386eab-960b-458b-c3b7-08dcb2170d26 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|52116014|366016|376014|38350700014; X-Microsoft-Antispam-Message-Info: /5svspaDkFltQhYXYqp7RLcVKThBmMv583X3nQq3JRH2Jp5ekPWXmDzpNVkn0Ks1McA4337l8nUZjrDr5pfKfg28uywONrRgtMakCpTMu5KFVGA9wLQB6E/gcid5Ld/ZHUWSAfEVz1bZmuC2r+vwLu26a26JVSn/+5Vqtvv0w4MKbLxCdLPQ2S+eJz2GQ70ZsTAL0XpB+fihLXi1gIhkLzqDIgd0oYt4l9GKxwh0ghSO7vXJ0cvbh37S9bezHs1DJ/iwrITFItTV4noVVLeu9y67gb5PFadBM4gID9kNPgBJ54MFJ0iCkqVgaR7Tr2Kpi8glmEV6Lhs34ni33+urfnlsCPCz9eaCpLFPdXzHylkCa/AreD8r2wjD1IGYSdnR6M0U7ojTWcQunsG5Y9hAuu4aLOpW2hEWjre8YulnKXT2rNfdwCam8IqlXQ98UHFbgL4Fdex3oAI5wh/7KMmmNDUBTJ1GIkZVjlzibxuH0/QifdtqbXLWW8Lll5YDxQxFNB0s2b9PI9kwiohi8eIJzBAZ4lTWWUm2q9Zy9p0kMeGhd0Q4rVbPW4QpgbY1kZQpAquoCXhh6VTCWaNZntHMYcYxxXUmyhIVmvU8RF2LgIJy3oPA5+KKAapHR500kaXb2st35suiJ/3sSECDZgXP+JWKmZxNK/rQkLtgVXPWYFW2l6Ht0jOZP6yPHj/GhYh8pVV+4ME0Ot78voz6s5WP+hGC19AmTZSeMzkacOKrNfitIp+6q+gYtDQubpsBPhtC7gUISvtRfATNHRfuA5z5w1pvNhW1ik9SCyH6LPj8mTCKlzpQp7xtIgZ1AGhfms/2P2Or9THrU8ikGUNwpvP4a8mPAVBUED8AqTjtm6cfkLJxVvIjmupzfkCEPflc5SM1JGeRr8gPLf7RMt3hKzYBObLFW+4GA3FsvWPe8SPOtADaAY2ECcJdxMDH8sMp4gUlaTBvJ67UuDS20QpLGjaeePv07vhTK457BJ6uzxduxpa1VrYYR/CeS/z1FpqezHvbBZLgE69OMj84uJIuBV04j7WjoWkXkP9xLjMg4G8zQRaQkVB0wnNKeQQraEkjmxjNfdApopNy+RwRbEuuqJ8PlBWfPTmPHWdCWo1NjJsNed6yM7KKcnxY+4dY2kVwXFjUCaDx6iclMX+09g7/gcKS87eHnkb9DdXi1rfVyUFqNY98ep9U8Nkiv9CJ5fBQ2rSyUmKKzT9Xap9/aov88unt+gx9YpZsCbksAHg+PQ57YikKmcwYDOrMJ+2zfKfDSNrLze3HkQKJozIyrWiUX83y0rmj7tcD6XhGuVh01Gfq6vHk+DKx/YiAJWaXl2scJf8y0h8gLbs0o6WlPAkYva7Ix4K0DQ/ar1glYrNcgLmlesEPCV5r1co5yyfw325IBp02JqYKzS+XD4ilIzDB0B4pFg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PUZPR06MB5676.apcprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(52116014)(366016)(376014)(38350700014); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8kqGCeGnIEIkdHDDO3gevumGfRzRMwCV2aiVGxLVsroI8E794pGePXZIxuTghzwtZgN1OXU9t/NbD6YLyRhK/L1CxpDiNWM4iryKYKdvO4Mv6/PWy8DFVwoKQX/vEwOMtV845TzqBCCTi1nt0Dg7omFboLqPip9UjI75mzhGs3ZhlUf1+ToM3ltMA1cJidCPxaBnhHST62OpDkKFNotCuEosro/mLf5CJ4Q+jV01CO5Hv3NE+3/+OMEMUQQfJaQgjg4qGPnCXEI7i/9t3kqy82XtmSeX8yewPQwNlh9sRj/VkxEgY7exYc++GqDFSkpM/gs+BMYg1+55U2hS4HwjZpCzOUhznqx155V/Vl8pr3DeMs0IwmDfDvuG84mSRcPj0vixyUe1rx2EzEiuJLwnbJJysE2Lxb+Zr3K8+1eGEpkkEWnMIqbflr29w/nivsGg/I3nX7Qvaj2WFY2diToO7SgVZpTiOvvlV/S7ZJLmPSWK9Js8HQebpVfz5vomiB7wP7f133UaL9kK6NlxFDLjiMlY8S0kufAXQJOthdvuLzxQeeHOwo3VQuGF8cVAsI8PdjBZHE+eyEVvqbxDZKdTmvqukByUOK0uG4UVOWicfVqHqMA3n0pGokuOPeo2f8klIrIQQ9DU4vZICCqEeqSuOZsEf+GPWpEwnEBCsizb+JmBVMygtJRv4Y/xMiHVfyNm67plmTVEGFKAG822c7p7uBqDcGJ3aPxvBw16U5qzUfKbt64ijG8Vukn/tRlTBgs3Z4m8T4xztoBkHtzwdLgdXOWB2jiGT6NlyQQXgvVR8f2VNjmTNuXL6m02vOWGQ9Kkk8Re25s4XD8iJ7HXbsI/y/rtV3fct1zXzT9yrZlYI+xAkMyExxY4jH4WeWxp0qvt05FifLo9ZAYkK9HfjI0GVfVR8uGojoT7MC2SAA2zQWzurzapnMV/rIomqoDmdHQBfDQLA4k3mrP2lUfjAO08TiFRyux+N39+E6Td+/kAO5TmKPBzUAdqNInzNHd899hGbsnhVnioBYD9oQuC602fUleXq04u3IbMEa+VAaJf9SkVTHErOzSVHclgScasPbsuOJZXRhi/g2K2RE6WLNLMclQ+rr5bZ64/BhpSg31HFkM9j9MlXEEGYarw3vW9RRv1gjBHM5f5DuYBrYRsDaJ+fDSLO8H59qYaTAwuaKE4fraZwbDNkcPTQmYdRB1MYVLLxlWrZfe+vTNmGU7zG51PbiUvdiMQBXVRaSJQcX2tK3iBp2jboD99YrCht4XZQMbVIWI8PQXuQ8LqOHl5OfiUO66uLTSmjScYFDuSJr91wa06zUbt9c6P8853mYw+uOHVXzxe1vyM7JekFM3ztvMPjBL3JxROFZsXwbCbBXLU6nLSy4fHZVulIzhFgA/+6C0ntHO8MwsuSwX4OVs/xYKhASGVauJlrEYGpNaG8Da5VXHFVRRPcTqMxQE+hmKjLwnHCO+zMyP6INQbYOEOCYSCQA32W5aBwryFpoeRYUK5/w7eIxNyc2m62HOJ/ecUHmSEJYMPd9FIQEc2zWR6W2k+zRPd7DcFNiH2CBdeePw6bhdugW1FImu+uVxoQhOhJUF1 X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: d0386eab-960b-458b-c3b7-08dcb2170d26 X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Aug 2024 10:45:25.6025 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: OAMxMoTh7hW0Q4mVpDeW45Uu/UyFDm0fobl7BIQvStaqUD0nwihqxZ/uQZ9u/YDqdAk8TYpzyRQeCsBBkt7KUg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEZPR06MB6304 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" The current udmabuf mmap uses a page fault mechanism to populate the vma. However, the current udmabuf has already obtained and pinned the folio upon completion of the creation.This means that the physical memory has already been acquired, rather than being accessed dynamically. The current page fault method only saves some page table memory. As a result, the page fault mechanism has lost its purpose as a demanding page. Due to the fact that page fault requires trapping into kernel mode and filling in when accessing the corresponding virtual address in mmap, this means that user mode access to virtual addresses needs to trap into kernel mode. Therefore, when creating a large size udmabuf, this represents a considerable overhead. Therefore, the current patch removes the page fault method of mmap and instead fills it directly when mmap is triggered. Signed-off-by: Huan Yang --- drivers/dma-buf/udmabuf.c | 70 ++++++++++++++++++++++----------------- 1 file changed, 39 insertions(+), 31 deletions(-) diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c index 047c3cd2ceff..d69aeada7367 100644 --- a/drivers/dma-buf/udmabuf.c +++ b/drivers/dma-buf/udmabuf.c @@ -38,36 +38,39 @@ struct udmabuf_folio { struct list_head list; }; -static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf) -{ - struct vm_area_struct *vma = vmf->vma; - struct udmabuf *ubuf = vma->vm_private_data; - pgoff_t pgoff = vmf->pgoff; - unsigned long pfn; - - if (pgoff >= ubuf->pagecount) - return VM_FAULT_SIGBUS; - - pfn = folio_pfn(ubuf->folios[pgoff]); - pfn += ubuf->offsets[pgoff] >> PAGE_SHIFT; - - return vmf_insert_pfn(vma, vmf->address, pfn); -} - -static const struct vm_operations_struct udmabuf_vm_ops = { - .fault = udmabuf_vm_fault, -}; +static struct sg_table *get_sg_table(struct device *dev, struct dma_buf *buf, + enum dma_data_direction direction); static int mmap_udmabuf(struct dma_buf *buf, struct vm_area_struct *vma) { struct udmabuf *ubuf = buf->priv; + struct sg_table *table = ubuf->sg; + unsigned long addr = vma->vm_start; + struct sg_page_iter piter; + int ret; if ((vma->vm_flags & (VM_SHARED | VM_MAYSHARE)) == 0) return -EINVAL; - vma->vm_ops = &udmabuf_vm_ops; - vma->vm_private_data = ubuf; - vm_flags_set(vma, VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP); + if (!table) { + table = get_sg_table(NULL, buf, 0); + if (IS_ERR(table)) + return PTR_ERR(table); + ubuf->sg = table; + } + + for_each_sgtable_page(table, &piter, vma->vm_pgoff) { + struct page *page = sg_page_iter_page(&piter); + + ret = remap_pfn_range(vma, addr, page_to_pfn(page), PAGE_SIZE, + vma->vm_page_prot); + if (ret) + return ret; + addr += PAGE_SIZE; + if (addr >= vma->vm_end) + return 0; + } + return 0; } @@ -126,6 +129,10 @@ static struct sg_table *get_sg_table(struct device *dev, struct dma_buf *buf, sg_set_folio(sgl, ubuf->folios[i], PAGE_SIZE, ubuf->offsets[i]); + // if dev is NULL, no need to sync. + if (!dev) + return sg; + ret = dma_map_sgtable(dev, sg, direction, 0); if (ret < 0) goto err_map; @@ -206,20 +213,21 @@ static int begin_cpu_udmabuf(struct dma_buf *buf, { struct udmabuf *ubuf = buf->priv; struct device *dev = ubuf->device->this_device; - int ret = 0; + struct sg_table *sg; - if (!ubuf->sg) { - ubuf->sg = get_sg_table(dev, buf, direction); - if (IS_ERR(ubuf->sg)) { - ret = PTR_ERR(ubuf->sg); - ubuf->sg = NULL; - } - } else { + if (ubuf->sg) { dma_sync_sg_for_cpu(dev, ubuf->sg->sgl, ubuf->sg->nents, direction); + return 0; } - return ret; + sg = get_sg_table(dev, buf, direction); + if (IS_ERR(sg)) + return PTR_ERR(sg); + + ubuf->sg = sg; + + return 0; } static int end_cpu_udmabuf(struct dma_buf *buf, From patchwork Thu Aug 1 10:45:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Huan Yang X-Patchwork-Id: 13750172 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 375E0C3DA4A for ; Thu, 1 Aug 2024 10:45:35 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 9A09F10E8F6; Thu, 1 Aug 2024 10:45:34 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=vivo.com header.i=@vivo.com header.b="dCcRqFoC"; dkim-atps=neutral Received: from APC01-PSA-obe.outbound.protection.outlook.com (mail-psaapc01on2043.outbound.protection.outlook.com [40.107.255.43]) by gabe.freedesktop.org (Postfix) with ESMTPS id 663B810E8F6 for ; Thu, 1 Aug 2024 10:45:33 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=B9/UHc2GtBuBgfslbsAm2GtE1JJZgfZPUc0JZ5CmHr/4LGke+J6+/NONFjGkXwZwqzSUHJuaWxl5sBApm9/iSr+/RUo7G6FsI2qCcb1NkoGgiC0if2dwWIch1XVPhH7WwBCCxEr5LDvSgEfV6eKn/2K7X7wOHmacU0ONHLLYagrND8CWJeRzHBgEdFN/6BWu4ux69mxVHwNQOFKDrEczR1NK7JKS4LTvonu/SQlGzUwGFzw5DE8eP1owbPTNfW+OZ7SNkJOQXRZl0UBYD5hNUEKZ81XFhik5g6difC2uUhPU0dIPVr1rxyle6VL7QdrwfKp+AGoQ+FNDf+8A19sMlA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Vi78hp5Aou3qDPxtxj+R4tHNMNYIlPV0XE+Goi9lKLk=; b=GYJWAcGZGSgBvOvguKy+3uig8XqWmSFCwgJjZ9BhygHpY0KPULxz2UPPOUqo/+K264nPCOIN+2sh3hW/kEhziUQEqOlHuzSESly+1Vspe2wX8tlIT9MoujFWTYt/nFFBF6fACcDmyZKgsjTIVAW7YZ3uMHjEQCKc14rEPSRwjf7lAJi5OhMZrOGP+GLk42H84cQ6OY1gT6rGdopdDJB0CdM/hTiGEB3SQyVV9MtyvE5sKg/5fmb+Be7Cnm9bg489xxVzpaTtrpyzLxr9yL+R00uYgqkkwtgGpK2++nSRZxkIwzZp1qUOkahSiCvDk87Z7KguQXpwCJyWUHXT+Nb+OA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Vi78hp5Aou3qDPxtxj+R4tHNMNYIlPV0XE+Goi9lKLk=; b=dCcRqFoCSjhkVB92MH+5KQSEzwGeHyjCAV17dDdSxjEDFkT3P0vziKEMgUr32LuN8xh4mRxz4ezlGx5Uivqpdw9zPDorVCyxZK6dLwwdaXMG8hetzeR7+ZbIgddukkNQBxKMdiThocQVlCLAKZ81pw8qARncmB/1PGzsPgVU+94QNQ15q9Z99wX4uK5h5pA9CLHS6L7EejfMvzEsLnDQHVutQP4NSKgFw2lC5YFC5mky0P0sEniikHFFXzYfoZEaN+8cohW0p71/toQDJl9WK1ke8i21iop1vposYM6L457+l03UyL2XdYIvYOJloM4gPf8U5oKWn4/OSKvHEpkpdA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by SEZPR06MB6304.apcprd06.prod.outlook.com (2603:1096:101:10d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7807.28; Thu, 1 Aug 2024 10:45:27 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%4]) with mapi id 15.20.7828.021; Thu, 1 Aug 2024 10:45:27 +0000 From: Huan Yang To: Gerd Hoffmann , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= , dri-devel@lists.freedesktop.org, linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org Cc: opensource.kernel@vivo.com, Huan Yang Subject: [PATCH 2/5] udmabuf: change folios array from kmalloc to kvmalloc Date: Thu, 1 Aug 2024 18:45:06 +0800 Message-ID: <20240801104512.4056860-3-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240801104512.4056860-1-link@vivo.com> References: <20240801104512.4056860-1-link@vivo.com> X-ClientProxiedBy: SG2PR06CA0209.apcprd06.prod.outlook.com (2603:1096:4:68::17) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|SEZPR06MB6304:EE_ X-MS-Office365-Filtering-Correlation-Id: b51f6e2d-040b-4fb2-f96b-08dcb2170e6f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|52116014|366016|376014|38350700014; X-Microsoft-Antispam-Message-Info: +KRtv17USUJ35sIlu5vVxkv6RdX302vZIcfGBd08a+k0TbjVeTtR7lX2hWw2JbAkCMxzUNc9yyhRGEpyxOgsr75GDlqV+ksFPiZ7KbIPgLkWpYWVasPMxOeA+eCQYIOAdlcbPMMf3HHP1T+yBRp2yuLxBc/vktML5uG5RT2F+9GuKGteGmnIj2Fb7LpM967/CtTc9vNlcbYvx2bktpRIGiCJaIY6TZyx8QI2/bYCskDg8dFtYjBtbXsPtMNeQSXfKy0BYFN/I3BkuwkXC9xJ+KZIAVYctey1Joj2K27sBmT2Fx2a5HFXtCZzF61Utqa2sde0EjrvU9IQOJHqBhaDKZqwDf3gof5z4Vjlcpb7XBFq7Giev7iH0q0MlJEkveRaDBBKZwTlZ6o41OXoLbmNS6/c9glMxKWKdhXiOaeSux2zQ0CIBTbgMGBp1KGXkr46+HwH4Dnr7Pu+1h/wLHuc7u24buzxW3yAI65+cAjqSv2kEQJX8KNRrg8+5NXH4WWxj9MVoo2jpqVPpV8tkMGb37RCIShed2htiL2L2rorf3EZruREOOM7PyZhiNxgDWJnWg+bjz8p+bCKvqsyRdwH3VXr6ZIBm8wXNILtZlOZFtD+59dxSMwuJmiAlRAS/ugp+15GnPIvylGrzgIrzXWpcsr3IeofHyftfhIM3rTWFndGcTFTO5frymIMC28qcMkoJCSmQODSi5X8pgCKYIAk8zXMANanqc/e/J92mO1djVXNoN8PSCJBjiPEDgicyS9sBcWsN063vQR9Vj+xb/SbGxFxbD/FaEbE3A08j6wssBKEGT0FuVAbbCqpEqoA5EG7vWMrmIVhOxcOMiZjLG2lZ3dRJfPDvfYGPQxmvEqUG48xaeH1XOrVgkmZj1nNb93YJHQMRYUQkUakDXpemdGIIBjkUUftFWy+7qq+7wnM1qbj9kFToMAd6C6rDHdfXXpwTHgAGBUSAfr4Y16Yr8fNpCp5ANy7Vhuvw2BWj2t2GeMVxQawB5UoOwVBpDmcNmX89g0kToe0CY768/JKcty+Kq0mGB4ndgccI8Qoy8By98K2CkjzAGQwiI/CLtl8wr9jDdWe5FGxd67B+WpukaU7+aPyk5b2XQSkOsJQIjh59omRuNkOvyrxL8YgBZ56r1gtX2px8+nqG9yD5aJTlZlRyd+XuStHBhXDhqU91NHumtb2UDiavnf7SBdSmO6yRNNEKdfvenr/nHop5gBwbThd/Ca8ofCb6mfyyyqXXQKTiv9BXyIv3plWvZWzRfpXl+gOt3PjsPfnBliAIaA9hg43xc//NR9uIj0PGUd/MUhqaMOJFLtKb2mtTaqEzBILrMf3mlki0vraKimR011IxfZT4oIMlB5oo79YB47HXRpEV+vnqQzL4nJciK+gdlCzZ5IVnsH8Ts9xPgEbH4WmPnpE5A== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PUZPR06MB5676.apcprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(52116014)(366016)(376014)(38350700014); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 6YyoEtaEmLvqUFiLgKtkV80TGsHKYs2BbVsbTmJUGba20cxJXHJ66KnCjr38uEyLRsoj3B3t9uBlFoPJjKsfd5KHyUAuyMemqEY/t7wi9quVMyfcpJ+f+p2ow9FLNxj56XXxaBh59UrWz+BaumjB6OAHPlh05Plmgm6p/RI1hznIuPcfVH/FqM1ZLn3O0lJ3nKaM0wJF5ArfakBpb7430OTmcBd+Ok7IH5iecQraRaeWthiCcsiiS0A3Ohm7e1QI3uEZKU4dg+XeYk2eQ2b+0uryc/6I2kmMJhMPuwkUzXYe03UrBC10In5PVkfjTOUAr+bhfFqosjnGzf9KDi1bq0VUxc90adq9VNKOwmRGjFk+FiaqO/3ohLyEKtds35TwILU+nqUeoi2XILTNh6Ha04co35Ld+K02zfkxyRu2oy86fs+xvzd8qf0uKe+d1uifpDbKFbEaLzgo/95YoljgANpG1G5/PYn8XRKIQ1fpKlXwe6QBGnE3hIbHm79hcx5mhrUlokodKolMdUUEtlZ+CIzT3LQESi18f0+yX7IBP2ccVQDnn4UWfCm/mMgGDtXEwxVSbLn5VgocZiYy0G6HawXET9kSQtfRjwWh9MDR+Oex9w1Zjaeimt7zGwfmjPnkKOgNC4kvnX+GDP9KHleKe63yAwlF/J5XNMJzxLMGsCcUNxqc3Y0MKAfHoYGDL3b4ZyXjlftdZ/X7Wj+u3fLZFoAqXLqbt8WOFEesjWwKvgWiHqesF/nDQfLvLWVMbdAzEM4u8dIXIbbJnx1QG19/rjQy+An4fOV4m1YqaUByPKIpLbMFd66vsnGkk1CvUExNnTGOeWQb7oW+1WLEsGuPNjSkxlYuIvp9jjlzAuY/jauOCYEJQey6DyAIs59aRkOtmERXtYMreWqiLprIbjZNlLoo/AgqCaOhHFMdq4DNEZmKL5S0g6mhY4hRr7A6TkoXuCu3XCyJ4TSyhlqnfoyCf0UPQOt9iSS41OtYomruIxyXwSTj6beSKoCWmK4K//3llpTfawxxj+lT2F+98DZd9jBKzBPLES1YnvxkgF4cAoBNUy63dx3S18IkWokXytibDaJJMGksiUIDKNmHc3vhvjXcoYByuEZLGltlmM5k2pU3oPAGZ2pCewHO3YQjGnhCVcY9bX1HCIZbtjMJMSVmPGTmX5WzN/ribH1xk6AxFxAbIMP1AMcVLLMZdwYYQl0TLvf/mYqqGKtWwPR16S3adC7FTqbCyOZ5Nqg5lp8OgpoPqfYqEFBrbfekwBwpudslJlTQqSlSjFivsIqP2ram1I47T5YFkK+yMBwyfn8KX5ojl5glO5vYcnIf0nB8c6xilJeobmeO8izoCKWYVHwJO3/ZB152LHjWmzXxMqbDBdGo/0JWpJXxGj3onNKm6YQ9kyn9acVpKxQxKoVR2LwfBVzxY66iTJs7BuJ1J5neU1FsSfZb3+v/M0rHzeTJ3hOzGy1T0V48GL64Dj1yoXvG69acO0TLP2fs2X86cPETz3LlXsQMi9qlRysjhlEKizDFk3Aha2/8Trfqs8YIxURGEjfTbU17s2uDBW4bqBlJo0rkL/IgOW2JKR/U34+otA9x X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: b51f6e2d-040b-4fb2-f96b-08dcb2170e6f X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Aug 2024 10:45:27.7704 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hMDEtKBb9woECbNBtFVqPjxuPg04GXV5e4jeOB6IpZ4mDoZDT3v2Vvb3dH2tLfpVuaqGKr6wn4Mkdpw+etYIsA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEZPR06MB6304 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" When PAGE_SIZE 4096, MAX_PAGE_ORDER 10, 64bit machine, page_alloc only support 4MB. If above this, trigger this warn and return NULL. udmabuf can change size limit, if change it to 3072(3GB), and then alloc 3GB udmabuf, will fail create. [ 4080.876581] ------------[ cut here ]------------ [ 4080.876843] WARNING: CPU: 3 PID: 2015 at mm/page_alloc.c:4556 __alloc_pages+0x2c8/0x350 [ 4080.878839] RIP: 0010:__alloc_pages+0x2c8/0x350 [ 4080.879470] Call Trace: [ 4080.879473] [ 4080.879473] ? __alloc_pages+0x2c8/0x350 [ 4080.879475] ? __warn.cold+0x8e/0xe8 [ 4080.880647] ? __alloc_pages+0x2c8/0x350 [ 4080.880909] ? report_bug+0xff/0x140 [ 4080.881175] ? handle_bug+0x3c/0x80 [ 4080.881556] ? exc_invalid_op+0x17/0x70 [ 4080.881559] ? asm_exc_invalid_op+0x1a/0x20 [ 4080.882077] ? udmabuf_create+0x131/0x400 Because MAX_PAGE_ORDER, kmalloc can max alloc 4096 * (1 << 10), 4MB memory, each array entry is pointer(8byte), so can save 524288 pages(2GB). Further more, costly order(order 3) may not be guaranteed that it can be applied for, due to fragmentation. This patch change udmabuf array use kvmalloc_array, this can fallback alloc into vmalloc, which can guarantee allocation for any size and does not affect the performance of kmalloc allocations. Signed-off-by: Huan Yang Acked-by: Christian König --- drivers/dma-buf/udmabuf.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c index d69aeada7367..a915714c5dce 100644 --- a/drivers/dma-buf/udmabuf.c +++ b/drivers/dma-buf/udmabuf.c @@ -83,7 +83,7 @@ static int vmap_udmabuf(struct dma_buf *buf, struct iosys_map *map) dma_resv_assert_held(buf->resv); - pages = kmalloc_array(ubuf->pagecount, sizeof(*pages), GFP_KERNEL); + pages = kvmalloc_array(ubuf->pagecount, sizeof(*pages), GFP_KERNEL); if (!pages) return -ENOMEM; @@ -91,7 +91,7 @@ static int vmap_udmabuf(struct dma_buf *buf, struct iosys_map *map) pages[pg] = &ubuf->folios[pg]->page; vaddr = vm_map_ram(pages, ubuf->pagecount, -1); - kfree(pages); + kvfree(pages); if (!vaddr) return -EINVAL; @@ -203,8 +203,8 @@ static void release_udmabuf(struct dma_buf *buf) put_sg_table(dev, ubuf->sg, DMA_BIDIRECTIONAL); unpin_all_folios(&ubuf->unpin_list); - kfree(ubuf->offsets); - kfree(ubuf->folios); + kvfree(ubuf->offsets); + kvfree(ubuf->folios); kfree(ubuf); } @@ -330,14 +330,14 @@ static long udmabuf_create(struct miscdevice *device, if (!ubuf->pagecount) goto err; - ubuf->folios = kmalloc_array(ubuf->pagecount, sizeof(*ubuf->folios), - GFP_KERNEL); + ubuf->folios = kvmalloc_array(ubuf->pagecount, sizeof(*ubuf->folios), + GFP_KERNEL); if (!ubuf->folios) { ret = -ENOMEM; goto err; } - ubuf->offsets = kcalloc(ubuf->pagecount, sizeof(*ubuf->offsets), - GFP_KERNEL); + ubuf->offsets = + kvcalloc(ubuf->pagecount, sizeof(*ubuf->offsets), GFP_KERNEL); if (!ubuf->offsets) { ret = -ENOMEM; goto err; @@ -351,7 +351,7 @@ static long udmabuf_create(struct miscdevice *device, goto err; pgcnt = list[i].size >> PAGE_SHIFT; - folios = kmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL); + folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL); if (!folios) { ret = -ENOMEM; goto err; @@ -361,7 +361,7 @@ static long udmabuf_create(struct miscdevice *device, ret = memfd_pin_folios(memfd, list[i].offset, end, folios, pgcnt, &pgoff); if (ret <= 0) { - kfree(folios); + kvfree(folios); if (!ret) ret = -EINVAL; goto err; @@ -390,7 +390,7 @@ static long udmabuf_create(struct miscdevice *device, } } - kfree(folios); + kvfree(folios); fput(memfd); memfd = NULL; } @@ -406,8 +406,8 @@ static long udmabuf_create(struct miscdevice *device, if (memfd) fput(memfd); unpin_all_folios(&ubuf->unpin_list); - kfree(ubuf->offsets); - kfree(ubuf->folios); + kvfree(ubuf->offsets); + kvfree(ubuf->folios); kfree(ubuf); return ret; } From patchwork Thu Aug 1 10:45:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huan Yang X-Patchwork-Id: 13750175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6141AC3DA7F for ; Thu, 1 Aug 2024 10:45:41 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4B08110E8FA; Thu, 1 Aug 2024 10:45:36 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=vivo.com header.i=@vivo.com header.b="DdOHWUFG"; dkim-atps=neutral Received: from APC01-PSA-obe.outbound.protection.outlook.com (mail-psaapc01on2043.outbound.protection.outlook.com [40.107.255.43]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5BD5310E8F6 for ; Thu, 1 Aug 2024 10:45:34 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=H+G3EyOEXo73JkPiG1EmoXUjznVxcuxHItWBS2soK8fN7vkc0A0Wt156uTQbaxaiyZYtcgnV8wD8HIHeVSJFxhtMS+8EVY75mkqOgMjf51cu9wySZAFUml8XOJ9fAlaBlbhgGARhQ9P/gGc24NSRVigjCaNwh+pTmv6adRvS/EfSPDvkDGsFw+GjgGFIybZPJEttFVOLOuaLAI0z9Mtreo3aryPFOPt/byIx47wquTui4JnbmEeOPO8SAMFPTEwfOA/9Eq4Bt6Pk6hC3BhFD7gXm8HxTmgaBIx8VsHQmSwwDgFY64/G+qlq47FJJh8e5pUZ7R4FuEeF2ImmZmXKzEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UWLnCdkp610v5/9iVGLy5wlG5gZLG9AmQrLM+9Xbh9Y=; b=UJzeqNpBlwWE0mWF21YI4p8pRKUvoiRDoUJtLf7macvMneHxUXS8Ov+uzs58OOWRSP0fKGY3oFVvHr+cRTv4VMNjTW1fhtOtcOCmk3f0ReSSRuE73jU29cJgx2f6ESYV1HOg82bgsrKtKIofHAcNNTMtWg3UPaMWtkDW3DspOCtCP63deBaApXb55hIMOovqz2AFOgPGeplqnnLAXQNZ3l8hRKEVv7vqggCqKWvrb65/QfCPTkf8+hOPOw6xcvKYiYPwc/kkMlJUBLm8PSBUYrG27DkFVXE/mBGKMJ9bAXPT48G3TYSIAv7zYETS1kGEFZnvDMt53G6duH0jU+iIRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UWLnCdkp610v5/9iVGLy5wlG5gZLG9AmQrLM+9Xbh9Y=; b=DdOHWUFGFVCKodCcWwtLXvRC9jNIy7TmjQaU/ssgEAYr3xc7IAfVJQuKNdm2LMAhauf82Lpr6a35v3fOieX+rOuF0xoK3wua7SFDK/bGU8qHSgpSAUdnhYC6ZXH/tc3FjgUPgClaCbd89m/F1qGFG2fNeFxI+Zgkotqn5rwvB6OjT6l9lnEjLwt5j8DR47O4ne0jWBecHCkEkSBcDRwQt2D7312tEwihhMc3h5r9aXFPHW6cZPB0/SL20gSmprBZ07zQVIxTXiaJTHSqLCb3UGQtyBS7GhqKcQ8Q/SoIqcaCuo6GF4QIOP2xgyFAZgAX5gHp+7Ovm4e8s5ewh08ByQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by SEZPR06MB6304.apcprd06.prod.outlook.com (2603:1096:101:10d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7807.28; Thu, 1 Aug 2024 10:45:29 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%4]) with mapi id 15.20.7828.021; Thu, 1 Aug 2024 10:45:29 +0000 From: Huan Yang To: Gerd Hoffmann , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= , dri-devel@lists.freedesktop.org, linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org Cc: opensource.kernel@vivo.com, Huan Yang Subject: [PATCH 3/5] udmabuf: fix vmap_udmabuf error page set Date: Thu, 1 Aug 2024 18:45:07 +0800 Message-ID: <20240801104512.4056860-4-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240801104512.4056860-1-link@vivo.com> References: <20240801104512.4056860-1-link@vivo.com> X-ClientProxiedBy: SG2PR06CA0209.apcprd06.prod.outlook.com (2603:1096:4:68::17) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|SEZPR06MB6304:EE_ X-MS-Office365-Filtering-Correlation-Id: 4719b95b-7ece-48df-94e2-08dcb2170fb6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|52116014|366016|376014|38350700014; X-Microsoft-Antispam-Message-Info: PMm5e6bHf66DsFkQJdP5uAMmAv5EhKxKPkW0s6tEmGaYHBqvjkJVrHrdPY5zy7d/czmrvW9AI9871xjCnpO8sYFQJQfzyR8eeKplur473qUtOOytO7xlHfBF+2wOyjkvEzlXRYoIBZZipp+xiK7pmIyENyYng4oBNwVErc4kEq/G0ZfBPe2bSKdUspapa7WrlcoS4wqVrqj9xryQxXHIMrok3oBzOZ95PtNHEIntMrAKr48VN2jjoQm5/yeNCowQ9S3PCiIUwK796Q6rYCSj6pPaSMZqx71UdR1Eds4Ya7F90QCuuF40NVmse6qWVEEgvqodSw3WssLRE2yclvG+sXTNqrf53BFMlwnA+0SURHYiWjIhzsgxIi8c20Vrl5lzj/B7g5QRQfRhjUP4RhUmMNTlt/kq+zO3cXFLXW8uh9tqMvML+i6EfQWPW33y1Ta4q95ZmoCUxk4Hp/YGeXZNr7nHUURV5WnOZVVVn6QxESWLWxseh3AapPE82X/LlmjbGJZEEzrro+4rxu9ESpaRUOqT0iPhQ3HwHHqcpr7sOfgIhfxw8eOwLAB8BVLkFNRn0VY2jLb8ueGY0oECVcxxTmGSBWFkrT+AJqpNa/WPmC7WwEOM9ssfA7YA80NssAZi3u1o77WVM7UYZzUDRZGtmxcFrjaHRhEZyFhVdst+4T6m4qVzGGToINuxIvx5fQkxC8gwXQd4Hx8N87M7LDx4NAFdAdzsWH4SPIVcK0RMHxisomj5Q39AciOSnAV+X1E0L44HOAF3wV9YgcCwH+r0G6g1OEZEPS5U1pMYYrPTbA02JWhOMxVmubaYkJuA0L0otO2eJgc8oQcO5wqgGDS9jeK8ATIngUfhfHB1ObTZHa4Yt0LP0+G3+jfX2PPx0uRjGYH4RNdRBv3neZkTJOx84Wt6n/bsTAVGOlpt2f6bHfbwBxWgK9HxNj9She+YxGvBB7W6MhLpik3DlT9pOShTuavjtY7t9Dl8jeI8pHk8dTFpq2uvE6am/AXkZwnw7Qvnb2xxlbfARk6O3APU0eMoJ3PQCV9GSaj9QwkZ3urC85zFWCSshx3ghwmNjgha0zqcDWN6kvc4MArc/Mb2emuG85+TRKQ82eerFX0T/qTrxrdg8dk8/yoS0dc+2abeDdwSrp3KLzuIUytn0cSgDHGHyrYmK5yGSxGUD3Vso5x0AiebmTeEHsgqgwi8QjnzRch00Ri07orfj6FkGB4F2TUeAZRR5Nt6aRVFrhHhx1LteFIQ0gWgFjgjnsMOnhZadZQ21eJai9jDnfSXSyy5XlmkeUxtvscSOgkKIAGLw0CtXcVm5120tszvFR40h7gVv14qiNejODet5JCO+Vgb3WtLffmko4iD68kZdV9fHwmW1n2EzBK679072WLsomheBChpT2aONHmgOCM8U6BlKaAxoQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PUZPR06MB5676.apcprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(52116014)(366016)(376014)(38350700014); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: utMyP9wT6KRPW0toEjXI93yth4JkGI6xr15Dw28av04eqtuRZc+VCiwXnhXp7lJmdQoxJZcaCt+ZCV1VVdANR6sYEEF4iIx/utEcAUGIlLxomxgR3O7gG4fYwv5I/ifIht8CywWwN4A4axm7Vt4hj7k306Y5YKuvAuA1N6MD50hma/T8Hj/qS5tp54R/ohBSq5eCCU7fKo6XDxL8MmoARLwV/7L5jZ3tVfWyDUKYG0HNPyv8KDvVmogOrm8tH7LL8QpvUAmaVGz6ymSjsMBvcxeUbkkRMtuhUaZ0KJwSjJmFGdSkaeNmQtF5lP7YONMRWBfvcr5po97nImy/eRKFgU3fndpqKeEmF4VuFlzC+UnJUFK+hE/n98h3cET9deGHXuyIvv62k6IJabQoGIWYlLvxbZAd+0YFaGaQxft6oO+YrgYgLhwrocVl+31rgOUJqrhm3lBybb+KsethNjpI5WYGzL67UHnMehH1qevOl2r1tKXsJMQuLeYoBf4uUB6FBhgYGwYiw706fBPwKyddbpB+6+fMWMiW5clQCsc1jKMLjDqOANLF6Hyx53NJ8yyLCDyDxrNtyJhrctkRyNGnEYl3mGrLzuHnCs1HHPNergSxu0gyYaYg6IfriWbZHjnM3gu50AYFc1+rjReBnCtcvvA4P7tZRI1fkIZICmnYJV/1NPVNNLBlk7PMq+2gLgYq6t+H1CfbL3fgWG72m2IvScn8ju4sVQ15EuQoDomBRootH/mhGk7NXeDuWsfA3BiU3XdhSXdtVH/XWLFxmu1VHlFv2AEKsOztpKJBbzvR0wegC6FkOe2dn57H9bhMDFU+foA0QVwNPQTQhOY0ZwUE7z+lUG3zp068U+PlLgVVOmB8b3l729F1KgeD9RD3HRkVHQcsOg9see1iu8FNy3vms3ydP2KD7T8SDGzrPahUMXdgxl8LuBhG9Gq6JR7JW7klEe2Mr6F3QW5C00IKKj5Z4LhIHeUGY7BJpoXqn89zEvFWRMK7YZDf+0lplMDvN0fNiN1qhmVotmK9y+aiZyyv/3H7BHRCN/hwiMoaFzv+FKErFA11rZ39U4x61NvoIux3B7HM9lB23TykJbMJg7DwGbmmnMoctDbXc8NbhHO3c46zt7oKZa3SBmA1UBsScpEVJmHVDaTVPAGhT76HZZK7eSSXj7Kri9wbW/8e8qLH3GlF+AKBZo6ns2O9lTHQajXpPc+KB+7BJrWf3MxLGQ9CflFzvIQhCk46h5+io/x/Xq8S+AqaiXpiZssQ5wZTXqk0dvIYiwB9Wn/JQrJV3mGYrOgCneuHaRXn3XNEy+J3QfXPEC2xkjB51w/vvKHtoh7CJt39CPSdEXoesRwi+ln0QHOv/w9KmTjVhByMFhC5upUn7/Q6eqPg/1ObZaSvkxcVWJAjfcHqZOvjXJ5rV5rEt5mS2uTswCzUdIrp9cyVh5sKw06ZshqpbqurHemH2yX3y3Cod33tF1KndUUyEsaHOPfMwxqaSu7P79K97l9PFp8bN8cXVtnB7dKdWxlNupCT1HB6B/xEgpbMfa16WukidURPCKmANBNN4aFp5tFjBjONPZAoLWjwX7mZxk3ga8CE X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4719b95b-7ece-48df-94e2-08dcb2170fb6 X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Aug 2024 10:45:29.8643 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9zr45C1J5ovY7QqR15omKBELEuOiRRn/bB7mKz0UmlWfWwpaLnIDuQkvE/AJqF02gI/bA7b15r4kSTDTLeQvOQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEZPR06MB6304 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Currently vmap_udmabuf set page's array by each folio. But, ubuf->folios is only contain's the folio's head page. That mean we repeatedly mapped the folio head page to the vmalloc area. This patch fix it, set each folio's page correct, so that pages array contains right page, and then map into vmalloc area Signed-off-by: Huan Yang --- drivers/dma-buf/udmabuf.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c index a915714c5dce..7ed532342d7f 100644 --- a/drivers/dma-buf/udmabuf.c +++ b/drivers/dma-buf/udmabuf.c @@ -77,18 +77,27 @@ static int mmap_udmabuf(struct dma_buf *buf, struct vm_area_struct *vma) static int vmap_udmabuf(struct dma_buf *buf, struct iosys_map *map) { struct udmabuf *ubuf = buf->priv; - struct page **pages; + struct page **pages, **tmp; + struct sg_table *sg = ubuf->sg; + struct sg_page_iter piter; void *vaddr; - pgoff_t pg; dma_resv_assert_held(buf->resv); + if (!sg) { + sg = get_sg_table(NULL, buf, 0); + if (IS_ERR(sg)) + return PTR_ERR(sg); + ubuf->sg = sg; + } + pages = kvmalloc_array(ubuf->pagecount, sizeof(*pages), GFP_KERNEL); if (!pages) return -ENOMEM; + tmp = pages; - for (pg = 0; pg < ubuf->pagecount; pg++) - pages[pg] = &ubuf->folios[pg]->page; + for_each_sgtable_page(sg, &piter, 0) + *tmp++ = sg_page_iter_page(&piter); vaddr = vm_map_ram(pages, ubuf->pagecount, -1); kvfree(pages); From patchwork Thu Aug 1 10:45:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huan Yang X-Patchwork-Id: 13750173 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2D00AC3DA4A for ; Thu, 1 Aug 2024 10:45:38 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4446810E8F7; Thu, 1 Aug 2024 10:45:35 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=vivo.com header.i=@vivo.com header.b="ci+x4WhZ"; dkim-atps=neutral Received: from APC01-PSA-obe.outbound.protection.outlook.com (mail-psaapc01on2043.outbound.protection.outlook.com [40.107.255.43]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9AA7410E8F7 for ; Thu, 1 Aug 2024 10:45:34 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dRKDy0tMOTZ7vqj9rwb+RFQGfxWAltBVURl3QyLcyTn6vc5QH55fZ8fI2hyvZODC+uRWo8masoGNZDA6qbn5ytW4bQsBvLc5OqXqSwg4O3lONOdnbtANSvxE8cJ8iUZ+uKb7zB4pbAAg3IYCXmQpb2W0nZDhake1LN2MNEbLf2weX5QrdM2Oey8/fK43070P2uqcrqPl/EKi0zRG5WlUSj/iqBC1KbsbIfiL81WAZjV2I/AHgXV8fmq0Yg2FwtA5RcpQXKhkvaMA6cHEMnR0wXtjtSkuK5vCaIuf2ITJNOgeGD3MRgrRfnBQhBu7ixtuAp3w1cL+O+gNCuNFVJmPzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NI3xBL6FLUipU1PFIttllNUVCDK4zSGmhIWGLdMmCss=; b=ErqAFazfZlhfa2scC1bujrXpJ1buA6jfYu0mBUwPL2PursO15EhByWVIkaHFGqpA3nNxYpgCRoIq2wP2rntWUMEl7IBvQL+5BA5yE6BrD8lO/TCGacCgSLhVIrmY1tpQ7zAPK7ZqDDvlKQJL8Xs0NsL6p4YzeDUDQoS96Vle0iPZUZmnbbJQPNa4+xr4zpFe7+wCzARZh/Z9E90Q+zdixXZFons92V6C85lR3NAWLxy8JLbeuK1K5WwYEPbM8id2wiUokbPMDHNQRw6Sk8wVFMZstOIMs/P6LaJB8a+ciFQz9+crFvkP1mBMMcpYJLjPmoHLC+yles5M7KwRjTvJvA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NI3xBL6FLUipU1PFIttllNUVCDK4zSGmhIWGLdMmCss=; b=ci+x4WhZbPQFBG/Bk17oPCfT3a4Tc2dYSx8bzr11hufX0ENl13qEalcePFvrAWzP1cS479CRe9/QAaAHqgSWScAJQGD1IBeSMfM0Md4xZ3hEIANe3qqy5pnfSL38dL1zfulfAuPw5lWfNit5MgMpA72IE+IX7Q5LDoEbMFY6OLwK4cDVPcjSrd0gR/kz7I/Z6o1wal4+DdJUEEbpX+dXPg8qPbcxqD+M3Zi9INvtXQOejodYLv9Gbgdjskdherahw7jQMq2rGLxYS9pII2RCqiYpHCT80ETpT2HEvXy87bEmltFQKkFPDrEgnmNrYDcfAKA10Aw97MPvBHy00ocZBQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by SEZPR06MB6304.apcprd06.prod.outlook.com (2603:1096:101:10d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7807.28; Thu, 1 Aug 2024 10:45:32 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%4]) with mapi id 15.20.7828.021; Thu, 1 Aug 2024 10:45:32 +0000 From: Huan Yang To: Gerd Hoffmann , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= , dri-devel@lists.freedesktop.org, linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org Cc: opensource.kernel@vivo.com, Huan Yang Subject: [PATCH 4/5] udmabuf: add get_sg_table helper function Date: Thu, 1 Aug 2024 18:45:08 +0800 Message-ID: <20240801104512.4056860-5-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240801104512.4056860-1-link@vivo.com> References: <20240801104512.4056860-1-link@vivo.com> X-ClientProxiedBy: SG2PR06CA0209.apcprd06.prod.outlook.com (2603:1096:4:68::17) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|SEZPR06MB6304:EE_ X-MS-Office365-Filtering-Correlation-Id: 66d7008f-dcf3-4e29-4275-08dcb21710fd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|52116014|366016|376014|38350700014; X-Microsoft-Antispam-Message-Info: RnN179dQMAZP+g9I1ygim76cdcaeWggR1Whm8P4rDd9QHKvS2WluM3sXtfpGPxypify8sCqPpL7s1etuueMUrfQqODPocUbWgHC12CxHLhJ2/9GmAjLscoEjtkTZPK/cj1AIemldrkWyAhVhzcCRuIGvJA7QzW0nMXWkgyQMiFSjioOKGOB7M2K6laMWZatJNP7ciiq8rtTfrX4Q8lN6kHCvddDC0aq14B89jUIK6JIY96VgkaooPvh1nIWiRx/SzjSox9Gr/YVt7VuFUma73auziXaepCTT/txiPeRTPNRllEeBN15yhy3+Pgckqdrv7OgXL40/Cdv7jt8At8sAOJ1iE0hP+6TKy2LYnWCniBPJrGSm+GQHdsJZXqgPgGmikohzA/tdRYK1Dujk6jDx8TJI71EKsLLYH2Bk+kIiIi5h/iPAf5hlLfJO2Id2rTHCcSeT1kXhbbU5lxV7/Y/aOlT/YqpX89VQsiq8mLdNCnzqdn7Vb4gawSiEUAlJo3w5WJ3QObQzTdcWt5HCrrccxvADEFSaPPxNdBZHvABbYD6zTh7FkKGvsaLfIn8K6VMaEiU092ivlniPV3zQ2zBx2tFtnzGYCUTUd6ILJM+epvNtmlkpoELLvEp+x7xuz03gdFwrzn3gyqWTlIMveAA8r3ltu8+8iCVrPWr9EiBAX+VpvUrWG7pj1JCOziKm7X6tB4W4oqd7PwvD56DfPH9c1p8BxQQ1D4RvVsfuRrkUgSK5IOIdd4RrpuBivgJGqLi5JqnCaYKe95blG32FXB3ceXA+6L91E2UeQlbZDJzAAQnq+3SJNQJlezbNjie9YGhgNa6ulRYCybnY2VpCpi8m54yHvxMZHKqTlOgOQML1EERBKgxWL+axCZ+d+sD/fVaPeHri6rCFcVI89k+CzmC9ISQ/LCi7v1BczkTvlwALWhc8PltecQLgksEPEbGYpQT8FcMVc970y8bNZDUFJ0poczL6XL6iAU/c3NX64S0uIGAZxc66zSkE5LAaf0GanmDgNkdRsUWKKpwPbPg7DScX8+M1QUnGoAbmUOSnBDuv25UVZrnWPile/T4zNMIFWs7K/UGzCrFPUiUQwDeSPMkLf2noGmPcyQXRs+ePm8dhROmFOBxQDKYjNdKQQLb8ix93vaoqbJbNkn3PPhaedy11UDUuWyuOW43OQecfETeCs1HUmWwOQ9uuqsv6/JGFikGhxsNhKEa0iqyVbFumYYcada9cnhGBJ1dat90nt1lFAY9GxVL1+mEX+CuoRwQA9ZVAGUKKatBqbLmYeFDnsb/EJrb+9uHR4C1hD0d6pmIHk4lx2JZCXqxv+hMSjR4bdlMjWIqALdnbysL0CnB+CuUDPoz0LvZNBAF8THoOiYq90vaGoQ1Y3C5RX9r0PfuWBYeuXC+F+12wgdGU3dZ0Kbd8RA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PUZPR06MB5676.apcprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(52116014)(366016)(376014)(38350700014); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 3RBCb3j+kxXrO3QrFa4M3QZCE84rmNisjxVovD8NFFASan4zocmIlH7db07I+tqqYr1Eum0ccuvTJ/nXnDjvvgco9lwTVq+An3MG/vs+RZPGzLoTNpRJD4lYpDQj2ZErmQ1D3C5yGBjBwWShD0IlhQV8hNL/2pWCqtmuMWYD7GkVR/AzqWbzM7WnV8ickXxgAr55IMC6kFzcVPA1+GZKpJaheYRO/eGuIh/3wRrpBXtWkn6FPxl26aU4+m2wv3jgLmctLok8pjJHVdj/feUtfUWRMZc25ayfSm7qT0aJdZM1Cj2uyy+h2N/fC2AFBTzL7U00O5suDgv3cLQsaXdNv3AW7XsIk2Kou+Xk2TBB53fNrxrrk8tV2iAZpFe+hanHty3DIrUFV7/b4K/HlSLqoXkLkjhZjRIDePDPsLfGsicwmRapd1L8O8K9g2elFJ6cKWZhMR/Mm7gUJkwY0/vjbmDF2tYQ8P/rqxanpBECnJgj1nogDgj0283ikxbEI2zXetwoNTu6EM5/mXZ3A7kwR+71+IMue+IE8F9q4eadKS7f/BeQBbyLfI2kVtQMM+o86bt2YnMuZH7ueEZWzMSxV81p4yraGWo5pziE5xZxeBg1/eCzXiNOW3J5whYI3y2KxnbjUBQv6Z3dvlgs5vu6PFGYe6JNzK7WGKrsrc8Xy56xbOIrgoOvJt0quYXVYfSsNicfVdkqy9JpBLzXQxo43e49K9vaLoQUl2YB475d1ZXrhbr3ub/Z5eRmlbP51dXJFpmRGJqcl6b8l3to1UTkLZZYdDY7V5O7nRTKjFQraQMNkoBg6OI/bgoWaE/f6MFbLfKhNEIZNVqOfdBzxh7duW13dea3/2t06YI2u9ckHFfDD2GUhs5u0T1DrJNRK3ANOIEr5r3DD7SX5WswGZjHkNtvLP9G48BpDxsQHVmTGSNagrwlhTX30aoYJ2p0wP3fLgkrHal1rxjYy47Br6W3cYnoaoOzpSy5lf9r5QYrQuedU1oNL8/xXY+Y75/U9Dmpmn7WjH3/YnMoWajHsS1aDe7rDDDLxYeHK3lyhnjbZcBlDtii8WtwmL+kSVx55F8TCGw5nk34lRUs+LIGUSJvKMN34uIFtXf+jqahR+5Ej3P9tqIXql1D6zMG7p4oCLJg9G24spM8SXuR7n8rvYwSbhnZsQbVBeu+unNeKGKyl3Je5hpyoCQET//9A6SYx/1SCDdSKzXQ89ddctoU7bYD9iOJeLKAkGzK/47cP1aempQCsW5TlGRdnw5VQO9iRwjiAbrtuAdlN5MXoX89IC7WKIqFFq2Qgzsa3xxh3UhcLBKIpyKXxPvwjTts3eI7ZExNsz4MBby0bJyeY8CcLm7jAYWIMVX/yF0hATsKnYiIe18KCJbPWR94jStPXN2uxfpASt0TqewL/lq6Fwe5XqVeTQ/PH4kwYQbVvoN4EInRkbgVjw9Gct1NzPZAc9cZAu2YyBUMKOG56MMsoaaw33qwDNVHcHOwqk+zMl5I+WVIeG7WH2dcFkOmoVtVgA0v2YGwzCiEtDf6Ma+expv5sG2p4vOeh4SaqrcIZeoAEICzdnz4JB6wwMbLdGNbgx9gW68l X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 66d7008f-dcf3-4e29-4275-08dcb21710fd X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Aug 2024 10:45:32.0430 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 4R0YrKQyTsVQKHJX49XWK5HSoKZ6OkYsVtTEyjo04e1XCDlEZfcvKYuAyjxXMf5z8idplXTpAl6FaO7EslBgZg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEZPR06MB6304 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Currently, there are three duplicate pieces of code that retrieve sg_table and update uduf->sg. Since the sgt is used to populate the page in both mmap and vmap.It is necessary to ensure that ubuf->sg is set correctly. This patch add a helper function, if ubuf->sg exist, just return it. Or else, try alloc a new sgt, and cmpxchg to set it. When the swap fails, it means that another process has set sg correctly. Therefore, we reuse the new sg. If trigger by device, need invoke map to sync it. Signed-off-by: Huan Yang --- drivers/dma-buf/udmabuf.c | 60 ++++++++++++++++++++++++++++----------- 1 file changed, 43 insertions(+), 17 deletions(-) diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c index 7ed532342d7f..677ebb2d462f 100644 --- a/drivers/dma-buf/udmabuf.c +++ b/drivers/dma-buf/udmabuf.c @@ -38,8 +38,9 @@ struct udmabuf_folio { struct list_head list; }; -static struct sg_table *get_sg_table(struct device *dev, struct dma_buf *buf, - enum dma_data_direction direction); +static struct sg_table *udmabuf_get_sg_table(struct device *dev, + struct dma_buf *buf, + enum dma_data_direction direction); static int mmap_udmabuf(struct dma_buf *buf, struct vm_area_struct *vma) { @@ -52,12 +53,9 @@ static int mmap_udmabuf(struct dma_buf *buf, struct vm_area_struct *vma) if ((vma->vm_flags & (VM_SHARED | VM_MAYSHARE)) == 0) return -EINVAL; - if (!table) { - table = get_sg_table(NULL, buf, 0); - if (IS_ERR(table)) - return PTR_ERR(table); - ubuf->sg = table; - } + table = udmabuf_get_sg_table(NULL, buf, 0); + if (IS_ERR(table)) + return PTR_ERR(table); for_each_sgtable_page(table, &piter, vma->vm_pgoff) { struct page *page = sg_page_iter_page(&piter); @@ -84,12 +82,9 @@ static int vmap_udmabuf(struct dma_buf *buf, struct iosys_map *map) dma_resv_assert_held(buf->resv); - if (!sg) { - sg = get_sg_table(NULL, buf, 0); - if (IS_ERR(sg)) - return PTR_ERR(sg); - ubuf->sg = sg; - } + sg = udmabuf_get_sg_table(NULL, buf, 0); + if (IS_ERR(sg)) + return PTR_ERR(sg); pages = kvmalloc_array(ubuf->pagecount, sizeof(*pages), GFP_KERNEL); if (!pages) @@ -154,6 +149,39 @@ static struct sg_table *get_sg_table(struct device *dev, struct dma_buf *buf, return ERR_PTR(ret); } +static struct sg_table *udmabuf_get_sg_table(struct device *dev, + struct dma_buf *buf, + enum dma_data_direction direction) +{ + struct udmabuf *ubuf = buf->priv; + struct sg_table *sg = READ_ONCE(ubuf->sg); + int ret = 0; + + if (sg) + return sg; + + sg = get_sg_table(dev, buf, direction); + if (IS_ERR(sg)) + return sg; + + // Success update ubuf's sg, just return. + if (!cmpxchg(&ubuf->sg, NULL, sg)) + return sg; + + // use the new sg table. + sg_free_table(sg); + kfree(sg); + sg = READ_ONCE(ubuf->sg); + + if (dev) + ret = dma_map_sgtable(dev, sg, direction, 0); + + if (ret < 0) + return ERR_PTR(ret); + + return sg; +} + static void put_sg_table(struct device *dev, struct sg_table *sg, enum dma_data_direction direction) { @@ -230,12 +258,10 @@ static int begin_cpu_udmabuf(struct dma_buf *buf, return 0; } - sg = get_sg_table(dev, buf, direction); + sg = udmabuf_get_sg_table(dev, buf, direction); if (IS_ERR(sg)) return PTR_ERR(sg); - ubuf->sg = sg; - return 0; } From patchwork Thu Aug 1 10:45:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huan Yang X-Patchwork-Id: 13750177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 38E3BC52D6D for ; Thu, 1 Aug 2024 10:45:43 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0308E10E8FB; Thu, 1 Aug 2024 10:45:38 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=vivo.com header.i=@vivo.com header.b="kb0Epgu8"; dkim-atps=neutral Received: from APC01-PSA-obe.outbound.protection.outlook.com (mail-psaapc01on2081.outbound.protection.outlook.com [40.107.255.81]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9588D10E8FB for ; Thu, 1 Aug 2024 10:45:36 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=lirBDIr6xwMWyvzTpS4u3DvoCoaAXekgRRO/dALDraUo8pCq3vmy+1vShVJW278BCudtslxLzC98/RRvekH6plUm9TYwywm8VTe+6tRNomO5mP8JOQMpxe+6Z4MTG66A3KZOrA8B1cLXXByMujM2oygiLlK4ya2URbSem5wc5pNaQ+cAUOe846rdTVYkUhiIRGs3jwnEu9EnjK2m8XUf8e9coORqvgbn4/b+405ODP2gQ54KsQb5fBkdDnN9NeDRSwFWPTZXwKESMvQ6pXAs/4Xuk10qsm0K7EQfVUovRM8rktCgSixqJJ53MkYH0Wejky5/bouAxiK7vrMgtcLilg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mI0gpVfBwy2PG4XtWWDhnXZeLqC3mjTemls4sGgVBLY=; b=P/aYQ8s31pYIRZqDgdV+SXkGKH9e7XiCOQM+E8c3RyfOJMBgbV+4B1rbXK7KQebatUBvRkka19noY1BdlhaMZ+4mCHiSZ4mXoaEks+rue4M+aw18xZgZAJlwMm7lBTlRXGsfppsS4jzyJopY2wIJe1kvE+k2T1SaBTpCFZxb9ucU/0nddsXHJIa+DQvZTzxc4sAJNhseXQc+P8SpVI6eE7LEMKaj0J7zmLlilLlQU8KTqReWmrUBKl/NwCiWECH5Z9ojG9j/teQVDonrFRCdB2Rppf5utS1FY9ihfsjGFTQCEyJEUMIytX7loS0xjPzPjhxT+tj5O+KOVZu8KpU79w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mI0gpVfBwy2PG4XtWWDhnXZeLqC3mjTemls4sGgVBLY=; b=kb0Epgu8W5NxEIPigM34o+j/ZF0nNQQDAGsEtxn9a7q03ybT4QUZXy5upj2CeEgXpXENy1eu/J8Xkn7e/cSFXI2J2bGeV3e14vJEZ3n4xHm54JHS3DyV1QoD82KZZSi+tsspq0RWtlj17aDonfCnbAQuWFRlWrUWNX9E02Qqhg21wknwwFrarofpLz84YdV1hTUhMXpbJCD+xTNl7h7lvd2AZfXhCj+4hYtppoinbRLtCovP5FBEz8ZsQ445v3/TLxOQRc9HWroQcTQWxuDg+W3192pCHUxdymloPaX7NLJpEwkEyx2FImxz0uvgbNwPIs8gX1CRC70hkr8F50R54g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by SEZPR06MB6304.apcprd06.prod.outlook.com (2603:1096:101:10d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7807.28; Thu, 1 Aug 2024 10:45:34 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%4]) with mapi id 15.20.7828.021; Thu, 1 Aug 2024 10:45:34 +0000 From: Huan Yang To: Gerd Hoffmann , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= , dri-devel@lists.freedesktop.org, linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org Cc: opensource.kernel@vivo.com, Huan Yang Subject: [PATCH 5/5] udmabuf: remove folio pin list Date: Thu, 1 Aug 2024 18:45:09 +0800 Message-ID: <20240801104512.4056860-6-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240801104512.4056860-1-link@vivo.com> References: <20240801104512.4056860-1-link@vivo.com> X-ClientProxiedBy: SG2PR06CA0209.apcprd06.prod.outlook.com (2603:1096:4:68::17) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|SEZPR06MB6304:EE_ X-MS-Office365-Filtering-Correlation-Id: 695854b1-ffc2-46cd-d71a-08dcb2171248 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|52116014|366016|376014|38350700014; X-Microsoft-Antispam-Message-Info: 9kUOPAsfPTn1uCTgCV4uhSX47XZ97GDF2hkh+fSEy6JVvU277BoB/ZkDwfRSsHeFjedHnAAgeE5+uw/jfVb2kJxkptwzVxjfR7oI5+QFCnge0yLQi5kdrUrnAx3w5lY4r3IdP5PmbMNp6s+eCnXEnsF1eTXmdXDnQDxil0+IUiMTyj0XA5dWGlvIv2vc7f6Zj+g3kSGMS/12JXKus/uR1YAAGHdXqhNuzBCeDPfi2NFYvb68qHp5lX9jUP/zENsSZ7TPSbEEmOo1FKXknnhI27xjQJ3TG2AAYCqpf801B2zGxqqNUkOUZSNRw8Ioy1DTEEVhlZhS1bL3UGAyph2racIVGjT66DvM63LSVsRsZk1nYdEJZkZAT/kfsyDJLhHSIgJku63wjNsLWD0YntIK+6nEhjA+I9kR9Yfg6HLOSE8Opf4pM4E/daA7kJj7IjkaxHGE34tRt43OoubRrD9vsacPMWQVHjCLndWSE+WlNdlUjKGnFZXiL0WCUoNUsCZ6Qxl7+dGH6TIYqiAtzQo2/iFE9V+HwZ5/804oTbkSCFhNkOhTRDln9mEjt4/wD5WY2ZZQBFuG1yIINSEXjEDIn9SWba5q3q23dNluNsGWfn7OJLHwxHr+gHDT4q2vsKk0NRu6+WpK31j3cpNgZBFG6R3dcmb4y+NWGQoduKwuRZCKz/r9ih2d0hPvflbfvauRDdpZYxm0/NI6uHVJvwpdWnr+Nskm1UrdZWSaomMQK3/GhG4u7PFq4uBVjmeoSo9f2DwuUHAHpIGjuo5hmrqnJpMHv170y5zIk0PXtkUlqRZZxMLKZs7Zbf2rwrKP723uS6ynmfV2cGtODc1EmIgvbssXhmoWsJjsaIg7w0mt+AFD+HBCZs1DSIVBHVVNvEgf2eHjF6O+JaGBmajtamXQ/aFqFSSVL49Xb4utViLqDVXdyPKIfcOBHBvAgTHXUfCX+OzVHsLCALAHfpG0mzx1W2CeJgj73tEb02scLKFXjJU4tS5ZgOq7cPwIc2D2AoTaR1Rrn0XTns6Vw+ogDLuBjqnfTaDrQTox0xjJ94x9U/g1hoAh76qiYH0hciS6h3fuKW48ecrZVv97kS+g+0Efw26D9Wwd0Ttfh/nhNCsk6aaOA22tbxvhKihbUji4BxhkHZSMmb0azZyEWWpPMLmsxUz+EBb64rifQZ1J1lZL1U6QROP0tBk+lj5b0ZcnP6hRasfn9t1BVDedV9Y2NW6w1RI6eFLNIBByvI6kJUR3+3hnPMB6/weZX9DpndQzMYGF0/to8Kn7ILx810jp70yon+KknjUWzq5J/nW7d1xnA2pFUrYuUxJz0IjT5zJrX9ZOPQl1esR20JcQraku60Bp0QKoxK4GDsrKxAoRKez/j19+EeSGHYsnWwgPOv1OcyAj6fbIZg23hbXQz6XHkngJFw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PUZPR06MB5676.apcprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(52116014)(366016)(376014)(38350700014); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 6dMfQAsa2Ri2zx0N8TLizqDWm60R9/lHx2nZuAk8Bqb4oEekIINS77R5p9oehstiiC2KPjz0IK54Tyo0I9E7bu9D+C4d9G1gADSuj1VeCx4tWTU0LjIKHMfEDs94rA8AffTTbVpqH6G9GHPuJzdKKYtHFkT6SqLDA8XZzr1L4fgg3Hgt7z/RSy5LYEzKKNwUGE4kPPeL8i53bf6PBoHqfOv2HIFo/5LpAkcUXogtI64n0Y9l8yKpDbHRgUHGhJD1KfVTLksajIVdgZ+4ogO/PsGBqvr3qKIMKyeybZGG6bPPDmCy58wh230BBHM4RKv+tczpe5oFHFR+Y4SjwfUEj7wpjDP3ssaKgpFoIoAUVYujVrZgmZxZimHqIHXJFc6RvDu04Kj1+qnazUy06vlupCwOH9tUbMKgh4qWNFJtB8m9cWhGaZOWHfREnocfu2Tr53WoN7r7NzDzBoG27CV8nN9R+2NU7IIgEKA5gWRnjKw6Bt8yKrZ/LtznTYgSAt0HzSdMe7uctL0W5SeReGvpToh22QThq/VJqqCAhJTaHR/R3C36iHbwEsfvgKBmnwEQ9wzjqubdOCj7cqXr4gbbs5V7TageigF7/vYWm3nY+ipvTTTIoHXTW8Z717AtxXFoWXasM14QSV4UGORHHMQx2SXCJBbGFce7rQvxpdiHUfCEiajRCseiT3jaw0IOawbLK4gKLyvP4fvMJj+wnLJ/14WtxXfVzRyZfDjRU3+iCHUbP8Z3M645iXikw3+5nc6+uT20pBh/h5n7Sokw5fea/GcvSJEalj52jqWnHHuhHVMXpF2DEoUUopqtvj3QGSG2dZcI52I7278ruS5+kJoxHj0dkze8S8YUh45Nj7JHWv1oVRcs8nxLYGof9OqQondsNLmKCs/NYQy/qCPX79zxiwAi9xQRSy2A0Oj4ARx//8jZ4CfOffYQHb1014SbyvQWRjUfJTtX6UnH9chu/q0igZaX2uxPM4JIBVof0q9Y3VWJ+9Jhloh/I3d9GpN39GryjU4ApSAb9fLFGfI+Ae+OPtUbC3WTWCrC0hOBPzqZ1ZwJpvhWlou0v6qf6dQBf8Ky9yb0T3/5gAqMdRS984DKL+cwsZpb4wQ1IMqeiWWXkUIztFuPXB91UOyN/oyJx9hw4/6PR2jVa9r1mtCdM663JgbP99qly7VgCOsEaXDWYqbKPQ0hiqC8X1aEZh1JS0vvbr+ZZKzSwOtJiGVY6zrMSBh/jAgQ3I3RNEIRwYGQTPie5i5GtWzuliP5vw2s9yIhYYYvIMJCq2XISnYT5CWEDXGLO+lpvjULG4V6wAJ62r+y+n2IkeexbVDH9WxCaEfFNrOyDyLNylf8x7lH4Ik3IanR44Jq8uzIogaCQNmMPGZbR3kEnnuxCcELkpUUN8PRNJUiSobGCeeq8wtGCm+hY9c+Sor76O7Hay9G8zz7me1axW3O1LNEmJKwXRGcPzRNvu1Vlp9MF07rGXjvB3K2jLUH4YeCuvYPUfEvVe64vtiSoG2W4wNrsRGeiNvpeRmlg0wDAgyz3WDUEH7N6/DR2UbrPc/fprtZT6bJuEZQVlLRyPbxub+f4YDpxFJd08W/ X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 695854b1-ffc2-46cd-d71a-08dcb2171248 X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Aug 2024 10:45:34.1825 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uS6Tan53xdYHYJi6M9CDMUS8wJNzXWev6Z04NfMsjqol0ijQ8VXYZVr6vIt8k3Fu03ji3c9S7xKxBFOOJ/SqoA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEZPR06MB6304 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Currently, udmabuf handles folio by creating an unpin list to record each folio obtained from the list and unpinning them when released. To maintain this approach, many data structures have been established. However, maintaining this type of data structure requires a significant amount of memory and traversing the list is a substantial overhead, which is not friendly to the CPU cache, TLB, and so on. Therefore, this patch removes the relationship between the folio and its offset in the linear address mapping. As an alternative, udmabuf only tracks all folio structures and splits them into individual pages when needed by traversing them in the required locations.(mmap/vmap, sg table.) So, udmabuf's folios_array only save the folio struct, add nr_folios to point how many folio saved in it. offset is removed, and add item's offset and size to replace, due to memfd create may have offset, we must set correctly page in folio. So, when setup sg_table, we must start correct offset in each item at begin, and then set each folio's page into sgtable. Both item's offset and size number just the create list number, so, memory size will not too large. By doing this, we can accept the overhead of the udmabuf_folio structure and the performance loss of traversing the list during unpinning. Signed-off-by: Huan Yang --- drivers/dma-buf/udmabuf.c | 149 +++++++++++++++++--------------------- 1 file changed, 66 insertions(+), 83 deletions(-) diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c index 677ebb2d462f..1106e0b1e746 100644 --- a/drivers/dma-buf/udmabuf.c +++ b/drivers/dma-buf/udmabuf.c @@ -25,17 +25,19 @@ module_param(size_limit_mb, int, 0644); MODULE_PARM_DESC(size_limit_mb, "Max size of a dmabuf, in megabytes. Default is 64."); struct udmabuf { + // all page's count, pagecount * PAGE_SIZE is the udmabuf's size pgoff_t pagecount; + + // folios array only point to each folio, do not duplicate set. struct folio **folios; + // folios array's number + pgoff_t nr_folios; + struct sg_table *sg; struct miscdevice *device; - pgoff_t *offsets; - struct list_head unpin_list; -}; -struct udmabuf_folio { - struct folio *folio; - struct list_head list; + pgoff_t *item_offset; + size_t *item_size; }; static struct sg_table *udmabuf_get_sg_table(struct device *dev, @@ -118,7 +120,10 @@ static struct sg_table *get_sg_table(struct device *dev, struct dma_buf *buf, struct udmabuf *ubuf = buf->priv; struct sg_table *sg; struct scatterlist *sgl; - unsigned int i = 0; + struct folio *folio = NULL; + size_t fsize, foffset; + unsigned int i = 0, item_idx = 0, findex = 0; + size_t cur_size, item_size; int ret; sg = kzalloc(sizeof(*sg), GFP_KERNEL); @@ -129,9 +134,33 @@ static struct sg_table *get_sg_table(struct device *dev, struct dma_buf *buf, if (ret < 0) goto err_alloc; - for_each_sg(sg->sgl, sgl, ubuf->pagecount, i) - sg_set_folio(sgl, ubuf->folios[i], PAGE_SIZE, - ubuf->offsets[i]); + cur_size = 0; + item_size = ubuf->item_size[0]; + foffset = ubuf->item_offset[0]; + folio = ubuf->folios[0]; + fsize = folio_size(folio); + + for_each_sg(sg->sgl, sgl, ubuf->pagecount, i) { + sg_set_folio(sgl, folio, PAGE_SIZE, foffset); + foffset += PAGE_SIZE; + cur_size += PAGE_SIZE; + + // move to next folio. + if (foffset == fsize) { + ++findex; + folio = ubuf->folios[findex]; + fsize = folio_size(folio); + foffset = 0; + } + + // if reach to next item, must check the start offset. + if (cur_size == item_size) { + ++item_idx; + foffset = ubuf->item_offset[item_idx]; + item_size = ubuf->item_size[item_idx]; + cur_size = 0; + } + } // if dev is NULL, no need to sync. if (!dev) @@ -203,34 +232,6 @@ static void unmap_udmabuf(struct dma_buf_attachment *at, return put_sg_table(at->dev, sg, direction); } -static void unpin_all_folios(struct list_head *unpin_list) -{ - struct udmabuf_folio *ubuf_folio; - - while (!list_empty(unpin_list)) { - ubuf_folio = list_first_entry(unpin_list, - struct udmabuf_folio, list); - unpin_folio(ubuf_folio->folio); - - list_del(&ubuf_folio->list); - kfree(ubuf_folio); - } -} - -static int add_to_unpin_list(struct list_head *unpin_list, - struct folio *folio) -{ - struct udmabuf_folio *ubuf_folio; - - ubuf_folio = kzalloc(sizeof(*ubuf_folio), GFP_KERNEL); - if (!ubuf_folio) - return -ENOMEM; - - ubuf_folio->folio = folio; - list_add_tail(&ubuf_folio->list, unpin_list); - return 0; -} - static void release_udmabuf(struct dma_buf *buf) { struct udmabuf *ubuf = buf->priv; @@ -239,8 +240,9 @@ static void release_udmabuf(struct dma_buf *buf) if (ubuf->sg) put_sg_table(dev, ubuf->sg, DMA_BIDIRECTIONAL); - unpin_all_folios(&ubuf->unpin_list); - kvfree(ubuf->offsets); + unpin_folios(ubuf->folios, ubuf->nr_folios); + kfree(ubuf->item_offset); + kfree(ubuf->item_size); kvfree(ubuf->folios); kfree(ubuf); } @@ -338,19 +340,18 @@ static long udmabuf_create(struct miscdevice *device, struct udmabuf_create_list *head, struct udmabuf_create_item *list) { - pgoff_t pgoff, pgcnt, pglimit, pgbuf = 0; + pgoff_t pgoff, pgcnt, pglimit; long nr_folios, ret = -EINVAL; struct file *memfd = NULL; struct folio **folios; struct udmabuf *ubuf; - u32 i, j, k, flags; + u32 i, flags; loff_t end; ubuf = kzalloc(sizeof(*ubuf), GFP_KERNEL); if (!ubuf) return -ENOMEM; - INIT_LIST_HEAD(&ubuf->unpin_list); pglimit = (size_limit_mb * 1024 * 1024) >> PAGE_SHIFT; for (i = 0; i < head->count; i++) { if (!IS_ALIGNED(list[i].offset, PAGE_SIZE)) @@ -365,20 +366,27 @@ static long udmabuf_create(struct miscdevice *device, if (!ubuf->pagecount) goto err; - ubuf->folios = kvmalloc_array(ubuf->pagecount, sizeof(*ubuf->folios), - GFP_KERNEL); - if (!ubuf->folios) { + ubuf->item_size = + kmalloc_array(head->count, sizeof(size_t), GFP_KERNEL); + if (!ubuf->item_size) + return -ENOMEM; + + ubuf->item_offset = + kmalloc_array(head->count, sizeof(pgoff_t), GFP_KERNEL); + if (!ubuf->item_offset) { ret = -ENOMEM; goto err; } - ubuf->offsets = - kvcalloc(ubuf->pagecount, sizeof(*ubuf->offsets), GFP_KERNEL); - if (!ubuf->offsets) { + + ubuf->folios = kvmalloc_array(ubuf->pagecount, sizeof(*ubuf->folios), + GFP_KERNEL); + if (!ubuf->folios) { ret = -ENOMEM; goto err; } + folios = ubuf->folios; - pgbuf = 0; + nr_folios = 0; for (i = 0; i < head->count; i++) { memfd = fget(list[i].memfd); ret = check_memfd_seals(memfd); @@ -386,49 +394,24 @@ static long udmabuf_create(struct miscdevice *device, goto err; pgcnt = list[i].size >> PAGE_SHIFT; - folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL); - if (!folios) { - ret = -ENOMEM; - goto err; - } end = list[i].offset + (pgcnt << PAGE_SHIFT) - 1; ret = memfd_pin_folios(memfd, list[i].offset, end, folios, pgcnt, &pgoff); if (ret <= 0) { - kvfree(folios); - if (!ret) - ret = -EINVAL; + ret = ret ?: -EINVAL; goto err; } + ubuf->item_size[i] = list[i].size; + ubuf->item_offset[i] = pgoff; - nr_folios = ret; - pgoff >>= PAGE_SHIFT; - for (j = 0, k = 0; j < pgcnt; j++) { - ubuf->folios[pgbuf] = folios[k]; - ubuf->offsets[pgbuf] = pgoff << PAGE_SHIFT; - - if (j == 0 || ubuf->folios[pgbuf-1] != folios[k]) { - ret = add_to_unpin_list(&ubuf->unpin_list, - folios[k]); - if (ret < 0) { - kfree(folios); - goto err; - } - } - - pgbuf++; - if (++pgoff == folio_nr_pages(folios[k])) { - pgoff = 0; - if (++k == nr_folios) - break; - } - } + nr_folios += ret; + folios += ret; - kvfree(folios); fput(memfd); memfd = NULL; } + ubuf->nr_folios = nr_folios; flags = head->flags & UDMABUF_FLAGS_CLOEXEC ? O_CLOEXEC : 0; ret = export_udmabuf(ubuf, device, flags); @@ -440,8 +423,8 @@ static long udmabuf_create(struct miscdevice *device, err: if (memfd) fput(memfd); - unpin_all_folios(&ubuf->unpin_list); - kvfree(ubuf->offsets); + kfree(ubuf->item_size); + kfree(ubuf->item_offset); kvfree(ubuf->folios); kfree(ubuf); return ret;