From patchwork Tue Jul 30 07:57:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huan Yang X-Patchwork-Id: 13746908 Received: from APC01-PSA-obe.outbound.protection.outlook.com (mail-psaapc01on2080.outbound.protection.outlook.com [40.107.255.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D63E819306A; Tue, 30 Jul 2024 07:58:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.255.80 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722326318; cv=fail; b=G9zIdnys4rola/ncKYCDvj7Eol6HsDePqHLVl4eN5BgqlkjymgtLOcoys/Bln22GdFdKspvp39bejkfrKb2Sp42CheL0lRqmjY2dRFBGQwRMEuB4jAO/ybgB6G2dL6/lWCaI/RTJH5XXvjk4Lr0Qj0vyonR1c2H7hCDDw6EOw+s= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722326318; c=relaxed/simple; bh=UJJMaih7si44eGopyrimd00ZFXBeKcT/oJCoSb2rlJM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=t+G4Squxvzp5reqjO8VJuclSM+7dhouvOhPdC2myinVMTYPe2rZQ/jsUMNiEaf3t0eePU43FKQZQ0Z3CJ6D7LDKfIqZZnN6usq8UHq7l8O6Fn1gRbE/qIhF7uD/F3b8OPV10oHXdnnaeDAWroN1wx/7ubvJYZExq93YeFNAgpPI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com; spf=pass smtp.mailfrom=vivo.com; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b=jKZHXZA3; arc=fail smtp.client-ip=40.107.255.80 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=vivo.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b="jKZHXZA3" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=eCflF0Fz6mGUnTN+P9OsE4thAiRbqw6pxSroIs/ogKohMByBARuVojvdr5oocYOrC0mihymrUzMhBdGfL9bTplcNelOXCuO2YpDeOjy6akuVC9PGE6BcJ9gNen8FbPo1zj7pbppeZDKJsRnNOpz7Eca5okCP9T4xsjhdjyvDNo4xeWCtVJ7iR846TOjSRUpCdFzCsMNz1yzSKXAO6F7IV4KuwfB3V2M+iWCFMKG+NWW94yRdGydK4j2S6ER0dOBIpP9G3BiCg9cRDEEasHbqFbkzenBoEssjI3hTUASa9t9e5UNkZbyuBx46HbngXK0f4qCDDtBWWhjPOgTZzJfQeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JHX/UiO7M2V7tZ1K2p2u8yZfB1+JJYYUuYVrD1PHkr0=; b=LB2MUVrYLiKCDrBDbLtgniqJynFncosIEGmZgvmL/MFaX1cUA9XGdwEG/Z/CdWVCF+bVWm6BPIX6MZS7t+wkhpTRsergS5c5Ubvoq1kPmc4A11/fhDm/x+OKMAy8mcMkf8Tyz1ezxkRPcGbSTwD8FaptM42ktL5YmM7S4RjKD7fDt1bF39aW5KgecaqBHIxIXfhVotnO0mp5SWSmEwCoXj66c939IaHC2iBTnZRuXoWo5iPBdN5G4wNgBNXqAto58siWxde4dPkM+atmeRlcfevJRbvN4TuzysUQx35xqTisJDgfiKIOqJyOQbsRrY5mKsqak+RzVUFIEh2l/QEoeg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JHX/UiO7M2V7tZ1K2p2u8yZfB1+JJYYUuYVrD1PHkr0=; b=jKZHXZA3TF4Ag+YDCbEIId7iH7QMeL84dr04WFOpCSHGeJsZWAswD5GjBaa+KAlF82ftfx1mp8XfJ2/h5Sq/L657YPR9JEPbYaDFVz/szjNBHq4t0K+C/1iOPD+AVWeG2P0ALgTh5YiWs/xpJUhtSnVLokyvfOeRJCdgWnNHRbP0ax7KORhvoKCsDZDLaelvdO9ibePGN9vDWwS+Jqj9+94ioy24D+CzQ7tqkHOTzbAjXTgE6SFfz/7YkZAfWfBs9OO/RAgYEmtt4kKRa4ythAkYS0qbz2gPkWfOE1oDgliGOnxsf0hMEKbg1JnHUT7FViLY+zaVQMWcImBQA7w7+g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by TYZPR06MB5123.apcprd06.prod.outlook.com (2603:1096:400:1c1::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7807.27; Tue, 30 Jul 2024 07:58:18 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%6]) with mapi id 15.20.7807.026; Tue, 30 Jul 2024 07:58:17 +0000 From: Huan Yang To: Sumit Semwal , Benjamin Gaignard , Brian Starkey , John Stultz , "T.J. Mercier" , =?utf-8?q?Christian_K=C3=B6nig?= , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org Cc: opensource.kernel@vivo.com, Huan Yang Subject: [PATCH v2 1/5] dma-buf: heaps: Introduce DMA_HEAP_ALLOC_AND_READ_FILE heap flag Date: Tue, 30 Jul 2024 15:57:45 +0800 Message-ID: <20240730075755.10941-2-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240730075755.10941-1-link@vivo.com> References: <20240730075755.10941-1-link@vivo.com> X-ClientProxiedBy: SI1PR02CA0008.apcprd02.prod.outlook.com (2603:1096:4:1f7::14) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|TYZPR06MB5123:EE_ X-MS-Office365-Filtering-Correlation-Id: 85174c0b-a3ec-4847-05df-08dcb06d5f42 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|52116014|366016|1800799024|38350700014|921020; X-Microsoft-Antispam-Message-Info: IXY1yecc9OTgHuiqN5ux2zJYvqolghZ+pukSO5ZPIHyjfTVh235alhizUICs4oXytsP6lHMTw+lhBzmEaFcN00GVg5l+sPOqUYx5M1XS7IresJluFKWkhlEkw0LB/2T9J4akoQKdqyCk79hrvdDjTlP0r7bXKghF+eLIH240SO1AWTIJnKspA41PYiIgDTYyfdN0XZyzBQDJJ3qAye58keLYE8RDVR9xWgbSHJz3sSWy046+ej0lvVoQD8PfPDu/epoKQvTOQShcSQPqh9mjlNoY/YdL1F8FumOeyA0p/7E71GLZViRND+GEr7IXa9N/mJfzUeA4RBjIsCDwHC0v5agTMUvpujmBgPWY5l5uVaSdBEcP8qH4S3oNp3bQiNQNxaXihQT57CxPa8oEqbiz3lRG/q0kU8qgDz2STgQ10yJg/8/wBNwLJlIpUo51UkwaazEMOQ6EwVSDGfniCI3hNC/b9z+naRbF9KdR6DQYozlpteY9ckAUivXKfG0TZszDx6r/8sqWPUL1+0Q1NlszOJ4W7KRL2NKiT2XWUN1ncaVvddkvHi5qUal4aNzu2Z2rHS81z5CPCIdTeUjPtx6bt15Js8w511RaFoI2xoOJTXD6Xm2+AutGHb3sAtzL5wnMkdonLO2AyQ/I+5H2FH21jVD8TzWXdPJWifJ4VkxLuhxKYp1d7SObiIJF7nnUlI5GY0UEyBXIosAmTBTI/kub97hi9Nt1GSaEBe0nh/Q1UElv5bsI9xER+Sh4UarTLmlHfwTQVn96tYQvrVs58ibKYa3tzGI1Bq1jBTwTMVSCcyjLraQd1RY96ZwWfM8IUfJL0M8muew2xadresPmHR9bLL0B6nSaZPYvLT+w61hgKds4EwAPMRsI/YLpKNrsQjNHJhLJBbVgnrsrfvUqR0HJ4F/2mFLZWyKfiC81kJ/+0G5cUgjbLotIGwNyn45VvNqBnBV3ZMY1HWbC7L1Yhfh5DxNIZGd7p/K0hVIDN2/rJ9Z/IPPMmnfHNEJcRpiInSiHCKLkW7Gq1Ye/oMBzHabzjouKHojifwnfGV5roRxAuPwgHhTnEHryVFhINh03hIfc7fIiPrRCsnRFK+DZ0aGLcUFv3lNEk+o1PPMzfs3Zj+9yP12OqSsUd/SGCkLvmNcs3sEkyeLWXTlVaaqsONxtOrfJa1VKhZVaSYJLvF9+gw3qQTa1u9PLOJkdWJ37uOGnApFFg4oNjYNPha4sroxBc97T3bW36vOStL4VZvdg54SLQjPz7rUSPuzklT8+3yslzP1+7vgWMG17+7M1rEpLmGInvO7I3C/B0eiGHtu3usne8XYJ70GfIoAhO9KnO5ZUzakU04lPstt0AU8C1Mgnyvc879Rm2BO94r9j4dPXBiYbnBmn1c/dj6mo5wZl2UV3lAsSOnq0ClWPOJVWG7r2G2A1X7fyfDXfWNQSjoRq2/A= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PUZPR06MB5676.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(52116014)(366016)(1800799024)(38350700014)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: SVvGDNwXFOu4S7kgXQ8jBObYBdWAdXXqYsoTMTD82A0Qq9OW3/oBqfhJZViOeaq8hli6MO1SvDfM3J/HRSbXSvXXEpKAjONTP8qx4JyhdcYLFxqsrFL1tVm9dRVmYxUhZh6KDiMbYJuESOCjJ2giugy7alIRLddebStEbp6GpFH2BrTSDPFyNUKPf2thAggd/RpEygjGDM76uBFRIkd/5ezVZaNEX+Y0MDzD3E+y1RwzNPwvHmP7RAKMG1SwRI7z9eMQ9mSXJ+s77nbl+htp974L1KAP15K9AAQHbZyRUTCRTGVMx0ypW5QCW4TkOZgTghAIBqTG9iUmJzTSdneb0w0VImFVkItesiiuCF1fBK9RJTfbfHiNPj33uZEexEnJKif7DbJf8ddewaPeU1h0tKNOppGdZIqYXZHgERAjtcCZ8Qxv+2Zllm80IkKCHqJyy7UyYZY0CO9ayC7ipnl99fGJtiUikBhEa/DZmVvaVD+s9XrgSyCS4BHOReqzhLjQdrofZaayfkZvqAiOkybkobOtuywf4kprdtj8BRAO+aw3kFZHLUGIiG2IpMJRlEYvepClDQ1OKzVoKJevUMxMAv83GdBteyEboP7uFnNd6AVL5f5ucM5JfRHaUeJopwf7ytUKVTKRFMCy0krzV2B8O2Y7N9ghIct8x+s+1OXnShurbVlhritpvrfjGC62GRlQulk2UTLpDd/aiyGqgoduaaYbb45rGmDtQofqz2tn2ulxiXA7m1XlNwg+ndSD5RFhFuksP4T3v2KIIVOq3saV8I0CQVX8bH/8bwLBWfQZPptP57zMVc1WdBMsLkuTclvT4zyzq9cWKzhMeIuaZqKBbkJpo6059Bk1XGbI/iXgJ9JrJwVzE1DFS/mTzK15aL7A78djS1NyB38SgES4KR98Qua30yGkEoFmbOTHhuOIeDe7S2MzUWyUVKD9I0GTHSGPEj3bqlW8qGYvVTWD9WMxF7Wl2xUpX9LzCVA4Y/AFq7XvWvIuW4HqMHUhyKPwPI2g4UbZ5RomLHnOZK5ODuoTJh6MnLd9FcR5AHmCeTT7Yi570NPZBv92H2A6bpR9/1/jdH72kB4Q64FtUTMnhpPMjlhRsvFb4b4COpgnxbiwIMLyWYzTuoo05w6+agdBJ7a7lck55JjSl/bMcqxA7Xwp0ejGenPa4VYGpiucA/CDCo5fJUyEG87qoWMdgCsSOw4B2zGYu+LnZc00gNYjoK8TZ7mEjFnYv0jR0ZXuqYGtCmx17RDakYM1VYxKKp6+AfHp/mHRlPHgc/FCCXm0ARvuUieY7sN9ki/9az6e/ftR70yZdAotsQU3boqGFFs+ufERqTXvWl0qDIrddP8lpH55NUylOF5Vqf8ulapwyLIey5q0gDJ8luikNvdVc80sRf0/XkUIE5ouqriI0Ikkwe46BAxt5EZJRR3OQsWzH79Y/0S5qQrWxHQP2/0o9seJLr6rKsJMBjr9GROx6ox7zlUzw1R3n3IaVaM6MhgPq5s8g8b8iqf4SnzQbY0f56Xh3/amEvz0sRUHFYw60fgtlpdKXrw5eHByihIxeINt3vpae2gMOLA3j+q1kwl5Jdg83Y6N X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 85174c0b-a3ec-4847-05df-08dcb06d5f42 X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jul 2024 07:58:17.7801 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /ihmnhvaeE9vVy4Qvj/6kz8SY16Dv476VRMaXTahYSNRwtd8K+zTSXx9kbBY2iEYvnuPW2xWtw45Pu1HpA3Q3Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYZPR06MB5123 Some user may need load file into dma-buf, current way is: 1. allocate a dma-buf, get dma-buf fd 2. mmap dma-buf fd into user vaddr 3. read(file_fd, vaddr, fsz) Due to dma-buf can't support direct I/O(can't pin, not pure page base), the file read must be buffer I/O. This means that during the process of reading the file into dma-buf, page cache needs to be generated, and the corresponding content needs to be first copied to the page cache before being copied to the dma-buf. This method worked well when reading relatively small files before, as the page cache can cache the file content, thus improving performance. However, there are new challenges currently, especially as AI models are becoming larger and need to be shared between DMA devices and the CPU via dma-buf. For example, the current 3B model file size is around 3.4GB. Using the previous method would mean generating a total of 3.4GB of page cache (even if it will be reclaimed), and also requiring the copying of 3.4GB of content between page cache and dma-buf. Due to the limited nature of system memory, files in the gigabyte range cannot persist in memory indefinitely, so this portion of page cache may not provide much assistance for subsequent reads. Additionally, the existence of page cache will consume additional system resources due to the extra copying required by the CPU. Therefore, it is necessary for dma-buf to support direct I/O. This patch provides a method to immediately read the file content after the dma-buf is allocated, and only returns the dma-buf file descriptor after the file is fully read. Since the dma-buf file descriptor is not returned, no other thread can access it except for the current thread, so we don't need to worry about race conditions. Map the dma-buf to the vmalloc area and initiate file reads in kernel space, supporting both buffer I/O and direct I/O. This patch adds the DMA_HEAP_ALLOC_AND_READ heap_flag for upper layers. When a user needs to allocate a dma-buf and read a file, they should pass this flag. As the size of the file being read is fixed, there is no need to pass the 'len' parameter. Instead, The file_fd needs to be passed to indicate to the kernel the file that needs to be read, and the file open flag determines the mode of file reading. But, please note that if direct I/O(O_DIRECT) is needed to read the file, the file size must be page aligned. Therefore, for the user, len and file_fd are mutually exclusive, and they are combined using a union. Once the user obtains the dma-buf fd, the dma-buf directly contains the file content. Signed-off-by: Huan Yang --- drivers/dma-buf/dma-heap.c | 127 +++++++++++++++++++++++++++++++++- include/uapi/linux/dma-heap.h | 11 ++- 2 files changed, 133 insertions(+), 5 deletions(-) diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c index 2298ca5e112e..f19b944d4eaa 100644 --- a/drivers/dma-buf/dma-heap.c +++ b/drivers/dma-buf/dma-heap.c @@ -43,12 +43,128 @@ struct dma_heap { struct cdev heap_cdev; }; +/** + * struct dma_heap_file - wrap the file, read task for dma_heap allocate use. + * @file: file to read from. + * @fsize: file size. + */ +struct dma_heap_file { + struct file *file; + size_t fsize; +}; + static LIST_HEAD(heap_list); static DEFINE_MUTEX(heap_list_lock); static dev_t dma_heap_devt; static struct class *dma_heap_class; static DEFINE_XARRAY_ALLOC(dma_heap_minors); +static int init_dma_heap_file(struct dma_heap_file *heap_file, int file_fd) +{ + struct file *file; + size_t fsz; + + file = fget(file_fd); + if (!file) + return -EINVAL; + + // Direct I/O only support PAGE_SIZE aligned files. + fsz = i_size_read(file_inode(file)); + if (file->f_flags & O_DIRECT && !PAGE_ALIGNED(fsz)) + return -EINVAL; + + heap_file->fsize = fsz; + heap_file->file = file; + + return 0; +} + +static void deinit_dma_heap_file(struct dma_heap_file *heap_file) +{ + fput(heap_file->file); +} + +/** + * Trigger sync file read, read into dma-buf. + * + * @dmabuf: which we done alloced and export. + * @heap_file: file info wrapper to read from. + * + * Whether to use buffer I/O or direct I/O depends on the mode when the + * file is opened. + * Remember, if use direct I/O, file must be page aligned. + * Since the buffer used for file reading is provided by dma-buf, when + * using direct I/O, the file content will be directly filled into + * dma-buf without the need for additional CPU copying. + * + * 0 on success, negative if anything wrong. + */ +static int dma_heap_read_file_sync(struct dma_buf *dmabuf, + struct dma_heap_file *heap_file) +{ + struct iosys_map map; + ssize_t bytes; + int ret; + + ret = dma_buf_vmap(dmabuf, &map); + if (ret) + return ret; + + /** + * The kernel_read_file function can handle file reading effectively, + * and if the return value does not match the file size, + * then it indicates an error. + */ + bytes = kernel_read_file(heap_file->file, 0, &map.vaddr, dmabuf->size, + &heap_file->fsize, READING_POLICY); + if (bytes != heap_file->fsize) + ret = -EIO; + + dma_buf_vunmap(dmabuf, &map); + + return ret; +} + +static int dma_heap_buffer_alloc_and_read(struct dma_heap *heap, int file_fd, + u32 fd_flags, u64 heap_flags) +{ + struct dma_heap_file heap_file; + struct dma_buf *dmabuf; + int ret, fd; + + ret = init_dma_heap_file(&heap_file, file_fd); + if (ret) + return ret; + + dmabuf = heap->ops->allocate(heap, heap_file.fsize, fd_flags, + heap_flags); + if (IS_ERR(dmabuf)) { + ret = PTR_ERR(dmabuf); + goto error_file; + } + + ret = dma_heap_read_file_sync(dmabuf, &heap_file); + if (ret) + goto error_put; + + ret = dma_buf_fd(dmabuf, fd_flags); + if (ret < 0) + goto error_put; + + fd = ret; + + deinit_dma_heap_file(&heap_file); + + return fd; + +error_put: + dma_buf_put(dmabuf); +error_file: + deinit_dma_heap_file(&heap_file); + + return ret; +} + static int dma_heap_buffer_alloc(struct dma_heap *heap, size_t len, u32 fd_flags, u64 heap_flags) @@ -108,9 +224,14 @@ static long dma_heap_ioctl_allocate(struct file *file, void *data) if (heap_allocation->heap_flags & ~DMA_HEAP_VALID_HEAP_FLAGS) return -EINVAL; - fd = dma_heap_buffer_alloc(heap, heap_allocation->len, - heap_allocation->fd_flags, - heap_allocation->heap_flags); + if (heap_allocation->heap_flags & DMA_HEAP_ALLOC_AND_READ_FILE) + fd = dma_heap_buffer_alloc_and_read( + heap, heap_allocation->file_fd, + heap_allocation->fd_flags, heap_allocation->heap_flags); + else + fd = dma_heap_buffer_alloc(heap, heap_allocation->len, + heap_allocation->fd_flags, + heap_allocation->heap_flags); if (fd < 0) return fd; diff --git a/include/uapi/linux/dma-heap.h b/include/uapi/linux/dma-heap.h index a4cf716a49fa..ef2fbd885825 100644 --- a/include/uapi/linux/dma-heap.h +++ b/include/uapi/linux/dma-heap.h @@ -18,13 +18,17 @@ /* Valid FD_FLAGS are O_CLOEXEC, O_RDONLY, O_WRONLY, O_RDWR */ #define DMA_HEAP_VALID_FD_FLAGS (O_CLOEXEC | O_ACCMODE) +/* Heap will read file after alloc done, len field change to file fd */ +#define DMA_HEAP_ALLOC_AND_READ_FILE 00000001 + /* Currently no heap flags */ -#define DMA_HEAP_VALID_HEAP_FLAGS (0ULL) +#define DMA_HEAP_VALID_HEAP_FLAGS (DMA_HEAP_ALLOC_AND_READ_FILE) /** * struct dma_heap_allocation_data - metadata passed from userspace for * allocations * @len: size of the allocation + * @file_fd: file descriptor to read the allocation from * @fd: will be populated with a fd which provides the * handle to the allocated dma-buf * @fd_flags: file descriptor flags used when allocating @@ -33,7 +37,10 @@ * Provided by userspace as an argument to the ioctl */ struct dma_heap_allocation_data { - __u64 len; + union { + __u64 len; + __u32 file_fd; + }; __u32 fd; __u32 fd_flags; __u64 heap_flags; From patchwork Tue Jul 30 07:57:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huan Yang X-Patchwork-Id: 13746904 Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2076.outbound.protection.outlook.com [40.107.117.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47002191F98; Tue, 30 Jul 2024 07:58:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.117.76 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722326307; cv=fail; b=cPGcVpjZrVLNCHaHG0hSlnqcy4/rNrkE+iFUeRdZ0NFw4vanJ01P6QF3MCyZ85Omjlmcb0pu9m/pgwnCSK7gHCR20wh+n743reLEUTYkQw8MP4lmLvB/qdGF7mXC7tQxl0gP8uq5jjg2Zgi0AjKosExUfO41RvPQxO0ILOfaAew= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722326307; c=relaxed/simple; bh=CZUjXj9bmoizkrbRqr8zfeLy+KnAMnRi95XjXJAnAqE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=m8eURSzmS40GEHRiNMKjw1LzbOKM3e3wWhuYXR2v93MnRgAugqptWJFGgihWY4k1P7wx22qIxBWMZbzIGCSaU6GY7oDeQHFwit8ADMrRIasS8ioWLlOG7a8uoUZEbmKlItFMeXbfhD46YgC1uZtCKOU8CBV+f79u3/orLWL8wM0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com; spf=pass smtp.mailfrom=vivo.com; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b=PLyEKIs8; arc=fail smtp.client-ip=40.107.117.76 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=vivo.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b="PLyEKIs8" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KmpLahEW/EIuXcsIqOONhE5KG9gbd2SbNsuYAZjsQWMmnEcpS12epFBQSDZXGVKSyE7YPcAZknRG2QCK3Vblrz06N5chrUOjlIbKZRbNZtMeXD9jKp4T8yy4WOB87T7hFbb09+YJW/THS8WdQ+LOOlyNYntc0iK9PY85fVgg9xYZfGqMXlpTFW2yfnNBZV3Ypk2seQPiwVHtXbjDJ7ZgZ/CwNLbjrwCq6CBu/poFVRR6lsfolDR2OQ90dq8mn/KECViCOztv4gusq05qx3tTC2AifNWhMijGqxKi6MoNXXhMqI3QstDrINnnz6l4w3JiNGTz7JErg8k8kse1fnFCgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DwkAl1b4MB+YSDpyXIavZO0sgvpDyxBgLGnVFVR2ayc=; b=mExzFPzaow7P/JDS7gFIZPx7IYGO/geRrJcRaXi+Gcj/7GkJ/phexSmKZomCGrLTQLfaojY/HO0PUMD+fPfHJXnTJiCa8SxRuc4/y0VmrKcCK9ZqIu+O2lXq2YyplxUNIaDtAsiVWi5cFc+AcXObcDwYPBJI6TTgGhbEF85/oyERxPt7DxN0WsMZiCwIJPanhc8G2VVTW1sJWolHsayGMDge1LPZVuNdAMfzyJGh2ctsk78xtCsnes1pYHwZ9YdF8ERYRNcuwLXX9s14zZ7lqC/4vOlmpCSBqIpT6B052PO6nB5bB1ufb1JTUKwA2Syvl8ZQ4dYmNDIAzzH+4sWS1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DwkAl1b4MB+YSDpyXIavZO0sgvpDyxBgLGnVFVR2ayc=; b=PLyEKIs8SFfrAGXe8KNCHrAV0K3C9XpgV89wrVSmrqRERiFVBOHwVcr1/VRdgTiVqmovg2aG4EnZ5nLqhyIPqXl89erITRYapnhhDM8JyqdY2rnRnxlQSHjxeuikUp90ExwlpbdKUxF4RJHIXx0pLmaixTlR7JbSGPzoomNbam0/XXhu42j+SS3RaCcsafFXi0K8J6NbzKQkzmbZr2Wts/n4jqN08JjZlo5jz7nQSA/G0rIat7oIt2eeFtfwlgvgX59iZC0E3YuNVcN+cYO4MmxKKeLjT1FPhx30PNfYXlpBBkx3S19xEBmX6hwYdHpAw/eUBdwQt8lWarPf9ONyBQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by SI2PR06MB5386.apcprd06.prod.outlook.com (2603:1096:4:1ed::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7807.27; Tue, 30 Jul 2024 07:58:21 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%6]) with mapi id 15.20.7807.026; Tue, 30 Jul 2024 07:58:20 +0000 From: Huan Yang To: Sumit Semwal , Benjamin Gaignard , Brian Starkey , John Stultz , "T.J. Mercier" , =?utf-8?q?Christian_K=C3=B6nig?= , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org Cc: opensource.kernel@vivo.com, Huan Yang Subject: [PATCH v2 2/5] dma-buf: heaps: Introduce async alloc read ops Date: Tue, 30 Jul 2024 15:57:46 +0800 Message-ID: <20240730075755.10941-3-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240730075755.10941-1-link@vivo.com> References: <20240730075755.10941-1-link@vivo.com> X-ClientProxiedBy: SI1PR02CA0008.apcprd02.prod.outlook.com (2603:1096:4:1f7::14) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|SI2PR06MB5386:EE_ X-MS-Office365-Filtering-Correlation-Id: 95d278e5-0b2c-4ee1-57b0-08dcb06d6103 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|52116014|366016|38350700014|921020; X-Microsoft-Antispam-Message-Info: IghyNLEf/EVSJfKqkqL4evAnqjReJLon7Zy0UmYAAJrf0hdab7ICriGWnXxiD0Fa/8mUE069sFHtwuymL8OTlx5Ot/3tB8F8b1OaxidrknAgVa+ULR6ynUYdEkrb6FkWoveZGAEhOA56iEwpIGiPecypi+mjNGElWNLyUBIXLWStTWcmiujkS9UNbmxCV2/8GqzdBoc95tApD5H8R3Nx5X3wEmgl/u5AMkrD/in3C2fSodh8UAOC3hDfuqa3KPQrFxXb7XQWhWcwqwiNwSGDahKjC91VZpkyeDJ389qDCH430LvX9jyNltV1LquvGU7OQ2iXcGaJq391RDMSluewIm5o/xnMekVx/grZqwlowBXlzdC8gDkcbLmYWD89sSEaT3MdpLGxcNf50w81rkUlPuKpLowcVYyYSD8rbEOxPqFJeSU/UdFzb+FyZsy/1qgUegtxvXUai3PYtDA3ZrK3E5zG35ooUQu2C7GKry8wVQrK2U6j4u51MV74eeJUNQfy734GPs9hDOTucy4yTnnOb3VIMqJOtjIFr7rpA//aDbCHyYH5QobcbcIpkt1Zom/1giDf+nFd2yReC0HSUq7L/fX0cpJOMaXjAR7UU6Mpj9IViejLh61mzkOvf41lZMnmkdRYp8DzsHJ6t3HemA1xVp3PbCFrZ9JTyNfzhlmUjUul3Cs9V8VxUFqPr3+BJABvDyvJky1P6GAGIlRXuwnLbA1Xs9MSD1wM06q5cMkcn4vTDBGJ9EkjwGmLEUrokjpnve3wLJ7262qLIzS5igC3b/SlAZFR2DfRZVD4piOCJwFx95cr3A7M3rWGh16/cjg3mo73WWpqg/sN/ElM3dm5MbAa4KlcwZ6pnUVMhq4EY2/h0Wli787vHp7LVlB0gOjw2Ve6wnkqaxLojDODDUTALvqs8coFfx44zuBXa92ocSao8YXsd2Cwy6aFtQ97/bfXrSFhVY68MYIg1/fbcz6se67gWFfuVD3NRz/ZvmovbFcuJykYTfMPmWAuMyeI/KkjrXK3oigyWv/sIa25CBx4KNsuoj151QvMHaGvqpsAVphdEP1xXAYA6DZe3r4hPucWr3ce7ctfrQYPVCkRO/eCvbBYT6qPA3Js86HIHE3s/MJbGmitHWRC4l8ebUONHPwTUCC9NdkYuBwz4dPgP54Vg8WA/RSNmhm76Au06hPnPD8273T/W1Y6YU2GIWuPHik+vGc3b0gjpWd8dM+4IDz2Nnw5I+NSd/g4bbIzfiYJNsKrlF9ihLfqysKIIJRfAIIe95e3G5iMc3x+SyPUkJxSx0diPAhntJS9NsRBp10VJQXnZv2PTFCOLRHh9Trea5rwj3zPRd5l0KYdCF6lGCX6g77RwNBMzdv9hJi+vPgpl/aUxs7gsePHkoVg8sH2pMla0gqKq82f/antp+A7XgnPiuPV4Dy16n1mkDzoGxfsvaQ= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PUZPR06MB5676.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(52116014)(366016)(38350700014)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: n4hNhpF3EqIOqqo85rGT3p2rjQHiCy7jvoOVXJkQDSfybSNLP2tPXWa51oLZZCUgNbsBvOGy+VLPG/PjzeG6Z8Sq2DIsY8UFmGNHzNDpEP7wi0SJkAQMied3ECm5rHw7WkwJO614+HlHcgwGTYHOL1SOJ4aWlpeBGZ7+qKvnBZd+yysRkkrvP5dHVkmf1RXKU7X0lCcbHTcvts4ZHEju6Vvczm2s44ku+BYv6oenTvcbaVCeXVY9aWBF/dKv2v1qtAzb0npD6WBfADvz1H/wIILN0PzbTbSDpMDO+iHJ/gq/MvX75jpYDblo9E3jUKQWO9X2Y/VbWhiArfmG+ax0cuj8sCqe9Wo7bt7nWPA1Roxy7rXRJ1Yh/2vwPenCvnlXjvUeCS7oyj92qVM3OsS7Gct8bc/hcggPlC1U9Ewi01RiWc5FcKvlerSs1SotSdUphC3vkMQ8QTxNbMGceWo5gSFX2J3HViCyQLMPX28ckYvU7eGZEDRKpU9JJwlCZcPCu6TfTRFQj6EXBdVoXbSMPPX2uabC7HCjN6xsXXQ0aWb6hIDwizdD/ug7vKsbdKL4ZFWsQsg9DTjnjc5WtNEeeJp6rRkLtpbsr7S5aQnnPSP40OZubqeuV2WProRu467DxZOLezAeSuLB/w7Sd7B37oWaaOEc81IoeI88UdZr0NL7uIm3pIlqgSFxG2KZuZpSp1UL/niQ4CzeemNBC1G0wUp5gI9L342B28TCdlV5G9CjdODtpx2d+JA/5mNfkxQI2vGAXbO1kMR35ucrnX5C/O+zUKRhTlawm86xFjXV9QBD2kYooVN40zfjKyUo6B2RIed1pHTiYQsxRd99WUrNjteUjnaN/A2CCqDZgGtFSaO6c7uT7AHVSgBww4719f3OYOasdvucZVKNCDOdUN0Qwo9AZFSsNPZYpuKJoeIVwj8cKHx6p8jFfbj2VviBq+AuDdAJN8BlDXpVBPqwOj1x1t7Lsi+bi6/UmfSHUtNxTK1cWNTZuNwtyfayFknqgXebs0MRpHHxaoKwql/3emjQhYDLcHTG8eCaOC/QI9uVQd/MPYT3XRlNVFlRKf1+Ugs6+RlUbn7JTUFq//Hip/L93l2BglGL4F9F5yylHtkripERrGMXL5/0Xtv+z6OFUFxGseNlWZENL6eq+dWBAYOHFgeSkdpQajmMbIqocqBx+vTYlOdVCiwehz3IgBHB9CnujHFfPvoJwMR6iXSEYe9K73iv+glQ5ni7K3Qf5BR5jElmsjShiEVGDhzjY7MWRBVvg+io5Ihlo//3MWNeF02NHSFPOl4NRhFII/XZsBdCayVuBIHnRoQVdjfIVeBXMoBXMh+UGfFU/NgFXWuKYQ+5fUOLpiubLyqH7I0+ubsSuoeU67O5NpTq967Jwd5fqFMx+bmfyGCfhyeYwZhZULLA6Si+RyXqrmUa/vN3Zqs1la447LseVHg34vdaatIFBlYxnmJWibr0LIhKCX/m2sGyatnUB80+Rym5cS79aKgrg5k0iCIsG5l4mSXXDwMdTDuGT5cHF5gaTG1NNiwQqgjUCllpg45RAu5ATv33zNdo+EgDoXB/zRno5Hljo+7I+9d2 X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 95d278e5-0b2c-4ee1-57b0-08dcb06d6103 X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jul 2024 07:58:20.6791 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RXzKR5JumcTh8xC6AN5T9SvgEz2OVpGNOvfNOE++LEHoZYwDfaxVmVGBZtflj8RzxXjFErV42NcLD3WEeQI1PA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SI2PR06MB5386 The DMA_HEAP_ALLOC_AND_READ_FILE heap flag patch enables us to synchronously read files using direct I/O. This approach helps to save CPU copying and avoid a certain degree of memory thrashing (page cache generation and reclamation) When dealing with large file sizes, the benefits of this approach become particularly significant. However, there are currently some methods that can improve performance, not just save system resources: Due to the large file size, for example, a AI 7B model of around 3.4GB, the time taken to allocate DMA-BUF memory will be relatively long. Waiting for the allocation to complete before reading the file will add to the overall time consumption. Therefore, the total time for DMA-BUF allocation and file read can be calculated using the formula T(total) = T(alloc) + T(I/O) However, if we change our approach, we don't necessarily need to wait for the DMA-BUF allocation to complete before initiating I/O. In fact, during the allocation process, we already hold a portion of the page, which means that waiting for subsequent page allocations to complete before carrying out file reads is actually unfair to the pages that have already been allocated. The allocation of pages is sequential, and the reading of the file is also sequential, with the content and size corresponding to the file. This means that the memory location for each page, which holds the content of a specific position in the file, can be determined at the time of allocation. However, to fully leverage I/O performance, it is best to wait and gather a certain number of pages before initiating batch processing. This patch only provides an allocate_async_read heap ops, without including other infrastructure for completing async reads and the corresponding heap implementation. When the allocate_async_read ops heap is not implemented, it will wait for the dma-buf to be allocated before reading the file (sync). Signed-off-by: Huan Yang --- drivers/dma-buf/dma-heap.c | 14 ++++++++++---- include/linux/dma-heap.h | 8 ++++++-- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c index f19b944d4eaa..91e241763ebc 100644 --- a/drivers/dma-buf/dma-heap.c +++ b/drivers/dma-buf/dma-heap.c @@ -131,21 +131,27 @@ static int dma_heap_buffer_alloc_and_read(struct dma_heap *heap, int file_fd, struct dma_heap_file heap_file; struct dma_buf *dmabuf; int ret, fd; + bool async_read = heap->ops->allocate_async_read ? true : false; ret = init_dma_heap_file(&heap_file, file_fd); if (ret) return ret; - dmabuf = heap->ops->allocate(heap, heap_file.fsize, fd_flags, - heap_flags); + if (async_read) + dmabuf = heap->ops->allocate_async_read(heap, &heap_file, + fd_flags, heap_flags); + else + dmabuf = heap->ops->allocate(heap, heap_file.fsize, fd_flags, + heap_flags); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf); goto error_file; } - ret = dma_heap_read_file_sync(dmabuf, &heap_file); - if (ret) + if (!async_read && dma_heap_read_file_sync(dmabuf, &heap_file)) { + ret = -EIO; goto error_put; + } ret = dma_buf_fd(dmabuf, fd_flags); if (ret < 0) diff --git a/include/linux/dma-heap.h b/include/linux/dma-heap.h index 064bad725061..824acbf5a1bc 100644 --- a/include/linux/dma-heap.h +++ b/include/linux/dma-heap.h @@ -13,11 +13,12 @@ #include struct dma_heap; +struct dma_heap_file; /** * struct dma_heap_ops - ops to operate on a given heap - * @allocate: allocate dmabuf and return struct dma_buf ptr - * + * @allocate: allocate dmabuf and return struct dma_buf ptr + * @allocate_async_read: allocate and async read file. * allocate returns dmabuf on success, ERR_PTR(-errno) on error. */ struct dma_heap_ops { @@ -25,6 +26,9 @@ struct dma_heap_ops { unsigned long len, u32 fd_flags, u64 heap_flags); + struct dma_buf *(*allocate_async_read)(struct dma_heap *heap, + struct dma_heap_file *heap_file, + u32 fd_flags, u64 heap_flags); }; /** From patchwork Tue Jul 30 07:57:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huan Yang X-Patchwork-Id: 13746905 Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2076.outbound.protection.outlook.com [40.107.117.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACEA21922DD; Tue, 30 Jul 2024 07:58:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.117.76 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722326309; cv=fail; b=J/nNafCYLh56MVFAFpdaxduRB4T0ru8ZO4Q5wYwFFW/IzrG+HDEFaVMQRvIcOu/6aLJ1ccfeZWj46cPFu3ROj36vNNgY2HrH/jy3OeDa9pfZ45HJSjIyXmcieEOCL10ss7qSIB8t59iW8pZM/REZmySAxfkejoYK5XcE7kGhgm4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722326309; c=relaxed/simple; bh=u30OrBk8yTIscAfnPWQdZ5W9p1d3R9SN3HkOeoZkonI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=YJid85AGgvXcUD6Bdb5zKzqQZvYj0EtKj+wzMtj6eWkyUCyIB1S1CwO0uZfVkZuAx170cO8O7Z7/N0EogJ+HbkfBt26HAtaWyYeRw6dI6usYjeRmn6KerA9wdO/IfIRLM/h8mJLICA/tsOY3EMRddTHoBZPsFR8Nqr7ksMvwR+U= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com; spf=pass smtp.mailfrom=vivo.com; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b=ejmV3vMR; arc=fail smtp.client-ip=40.107.117.76 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=vivo.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b="ejmV3vMR" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ZJMHxyQyFa9VgUuTP6JT+j8fZuh2RA8G+cQcb5F7QnmpHNv3n2tEu4w67jpTZqMABQLjrE9sw8pc5s328MASyz4LleRe34+Q1owDEC7l/SJNdo7p8KVwXjKjwwwjoniaMU+Nudqwi054ltBQD/OYDjLgsadMsKwMzOuzM1z9hYuZ+eVtH9QCFMTjb0ii0S72S2UINFWFjv/aAoDmI5EWagfwkd1cAz9BFehsv+gImmoC8h9+t1SA4sOChRv88ZyBLe3fVmPoUv3+hNVXYJ10pAOmq/R3flwaeb+r0J52O4iiqrQ3xyEZ3/5CzlnIE1BlsO16XVClSxLTSMtOoRu+Aw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jk9wIFl4MSkUn+JaziDh0bjJMIVOpKsmVkq+2U70zec=; b=WdcAinn2uUtZnMMWswcmxOwMB8bORPVLn/ZxDn1lfNGApoJIE+Pv+DwKgfhNFfwFJNWKsse82A3mG/niSo+0ziOK6wgDqH+iI1ti9V0/xMqCVkMPZDbuM1kkt2FR++wyQmGoOMp6a7jrCMUQ4FuuGK5BGA9JzpOadvosY/tBo3YRWL/3/9KlM8f7RZ9bWShb4JOtySYOT9Hhjg1Iat3CxK+IdNNW4KGsOkOf14/ig9JZNV3FBBIlNHknV81LbI1S2dyNnrCbmVs8KoNgiX1U+jepUNvFoqwgLpH1annV7QgZmkz4frbqNhasawxE/Jg0Y4TSkK5h3k0snwnZba6ibw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jk9wIFl4MSkUn+JaziDh0bjJMIVOpKsmVkq+2U70zec=; b=ejmV3vMR2Sr0suaBW66CY4i+mpFRYyAhVScoOVhMQtpUiXE8jslevudV00B8VXR3lJVfxigV/JfJhxLpG4UUlqgNucJAgZeFDolXMm0+GJvzOriyXTuQVIqiP6aScuBWwzSqILBEE6+e6P/ooc0nWebND1AJ14MMKO2NdFGs/jylzpCxDzjeULpEa8zRdrUhMeGTURj9ZUZHd96QmM0Uqfzd0fsqXZNH2iJ/maop2lxcc/Z1PdKAeGlAxIGkUtg86q4KG95sprwLX4ah//2X9hAru5dAhgjLbQho1rESI0TF9he+udCj/RSydTxZq51ppykkseFetGP/nTA0NU7ttg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by SI2PR06MB5386.apcprd06.prod.outlook.com (2603:1096:4:1ed::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7807.27; Tue, 30 Jul 2024 07:58:23 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%6]) with mapi id 15.20.7807.026; Tue, 30 Jul 2024 07:58:23 +0000 From: Huan Yang To: Sumit Semwal , Benjamin Gaignard , Brian Starkey , John Stultz , "T.J. Mercier" , =?utf-8?q?Christian_K=C3=B6nig?= , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org Cc: opensource.kernel@vivo.com, Huan Yang Subject: [PATCH v2 3/5] dma-buf: heaps: support alloc async read file Date: Tue, 30 Jul 2024 15:57:47 +0800 Message-ID: <20240730075755.10941-4-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240730075755.10941-1-link@vivo.com> References: <20240730075755.10941-1-link@vivo.com> X-ClientProxiedBy: SI1PR02CA0008.apcprd02.prod.outlook.com (2603:1096:4:1f7::14) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|SI2PR06MB5386:EE_ X-MS-Office365-Filtering-Correlation-Id: ee548953-10a7-49e2-1b1a-08dcb06d62d0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|52116014|366016|38350700014|921020; X-Microsoft-Antispam-Message-Info: 5s3wvcueDnpqAoehrNK9CR+MmJNkSSjmNJEXxuBLrzeDx8U98V06gies/lGBBASz1bGCNMLSkyU6AyT7gwI9UAqKiy5QxqbGtj5IERzxbD+g9tVgcHVPg1BinnbYjbNGkWQk7vv6QNjf9nPPp4FI55MC9dtXMhFsA+4PBRjQQnkCzqTlHZexRGGbgsER2AbqilzFmYxHxWg/q/7S38azxteG2Ouil7mjka/4uHy2RCeO9DEQ1Px+8dgmI5zJb6yHocCMfv+0CWio1WwMH0GaAthEfl9pzF8JYvqG4AB1Hv7gbEvqkxVppTWX69DNRZKzny96B1+3wVxBCC9+Z++q+YwaToCXQmIK6B1Lg2uKjwJu3oN/u41mLdhUbVYFG6LsJnd2pgsp+zcBezET9yB9UhNdGqNLC3RiC/NBf80GFiJuINejp5qv6f9VrFtP/Vui7vWOebO141+9veZmFTigV13tImUXa9BHkCEKKvnOdKoghnGaBjhd057Cvenv+JBJjexsFr9X2Ydqh9igoj6pP3DCiweq2C6Fnl0S1HD0vhU8ef4/IXU9dCoqrA9/fDJoBXJK3LQWQc6TAhgt/lq+G+e6Bi2ktbkegJxOsA/ABJcs/Bp4sdEWiBQwxTzoZrDN3AV14w/SkknHPQyRN1nUofWTC7Uu4mt1qKXUd6EMqk/t+artHNXlNXx0XyOVQv/Alupb5mFiJCup2GcpS5jkeLI01NHGZotKBRLWUcQ8cvr52CopDtPJY4RyUNiSbW88WLBLYzJIsNLeNWdgy9WIics/v9LR/5cA4GT0odlB9Q6HmezJcktWhTpUDiRFJ3R4YNz9/onf0ZNSs2UoMddKUaeAlz4ZNB8TNlaCb7lhgQTBH8sc7p9YdDImQ+GY0t7Ln28B1nLMwThpb6IzzlJBL7D4EBkIMi++eqtSxV101PIP8uU6mBZQOkSUyvG5GUSjv56uY7XvJnqp0EDrtOZLgQa6DYgofZrMrPXNN9KSpCbZAdphQwg+TADHuzAYnXoY3Yxne793ErTSciGLlSTio30Z9BIs/luhcPDuLadF1DKLiJH9oIOA+5rXaaF95/y0dZ7XQowvnhNVMYW8w2iuKF4OvAt2jZIfbEezTR+x7KwVleXM02QTG9Bq0ce2/86W2TGMSwkbNMbWyXiAKFawUxY3hqse09G1db/XFOIE5tWCeZnYrUx6mllgM3k8CGLqhhWgHCgozcNJxWpj9WWGJK/gc47+oGH31DaBwT3MSNmMS2KGoWu877AXzLgFB15rVj1pOKEHhys9qsRW7VQqS+c12Z33DIKkqOBeSormIB7e4FfcC9g+7JBZcvHDxK1IgBQEwFarc5Z0GOiDIOXCYdy8rHFTk9L2X9MR8h9lC0uRwTYDPFjWCyxpBppQVg2CGLSJJxdKl0Rw/4t2LyRGX8Mjk1HXmQ8eyk/Oi1lCnnI= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PUZPR06MB5676.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(52116014)(366016)(38350700014)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: kSLEgMTIDpvDq0pXEGzz2pIrYOKDgfiD98q3r7Tm+ClRsp8CMriMc7iG0ZNpq0v5nTFTApEQ4c6084KhDsZFmYdndnxcvidmwsXA8qLj82Z9jelA48SPor59RBDMd/tKqKDnLq0Mef1lmXn7akIwlXkt3jeg+fw4PXmxpKoRUyJj5O212xQz7AqILv5JVx546lESul8m5KwYnuv+2bed5UjCFPUPBfj9ZFy+IkjQc2LsHrI/1RyDKzLmHJWsG+J36WwijvD9slU1USetcK+qs2U8ElzGl3IYfVPD+JeBct49lYVR1QZw+EA1XZk1X7ekhMB6vChpfn5zB5WBzPEZGEeKSGMLhnjlEe8xw2YQeQsrRCqEZwr/rRy19ENcYFEGy9rQd8NuhNs9qjePSm+3rCsosbcpOJf0eENHJVlsBcXYru7y62cP3RvO9HkdYgsiNsMQONG59f6Hl/yGS6zQ6MuWahKj9m1E22SYdWGXbHBpcER6TP4TfT5zmSQJPNIt9RASDlQ1lQXGTqANpSytqP7K+ETJr9w+k8yxKzlbq1/kpvVYRpr63f8HyV6WxtL6yOsHrTaj+hocauCeVMBVmsZFP+22O1MaCy/zrjAtUY+1bZ828HM44a1vk0+233M2OSzHB3LNHVkBqculyI7FPK+oE2NMeyvojRuWwlGzaxyGr6CIgfhEca5dAzpEAXr2CUjWe2ElrETzl45tldKTbyaLcV+mnaETNF4i6ECLbDSuvXRbpH0Nuk8smqjHqh9aGUsoZam7G3xYa8RzdlvQduV1apajOxGLD1M/5aWqL2ZWv6tefmGg9jPkfVJTVtGg0pqjdImJaqbOzqy5EKV549YRLjtmPO8joBenIqc9XPJjOHdGugiZzOlqOMyTUnf68e19ic25hP+rcjXnvaU7sqbqW/v7huNWTdUqIqbEKsyKZ7WX4seOj9Ji8cicTBObMMbhYTI5abTOyptbrtRvhB8ioKHC5XjjzsfHd2n3syYSJRtLOibu3A7shZpjqhGdldZAsHkoswGBFYrrtME4WJr0RrVlXwzbLbDAcQj69AWu+6UOVlNCxv6qkTHwDVKTr0wBnnN7UT/FQDRnC0r4wGCV1bGK4kmPOHcIKXCcUhY4YpxhFJj2a3sGXq4M76OqAFam1g0afhPDP8MivO8FZPRfgutkLjCar5bdkoNECHb2Oexgsmua2BD0K0BRE+4W/RIV+a+GOgg5aSZaCwwbN+6NOkPPzXDGpeBIxZbI5CoHvX8so/zAez8Gz1CYC+iYfsJAqlWqG+QBspuwbx6oQ6Q2U9njg6yzxD2UTX8uwgKUTy4N/WnYyfEtCZ5IVoyjUxp/RCCySV5Qla7N5Kmw3U2SHNYR0oc5pNaNMtRrmv++Fj5709Jff1AZWk3WF4/psKm0M7khDowbZpdbJHjdfpyInfUQ59PHXHPdYwB5HWrIZ4ETBNx7Q5LvIUJoML73KVKMfUqlM2zdDKHnMErK4SafsGcOGUDq9yWwuCdFDovJ7Z0DdD2TjnBW3XwSnMym1zji5uBcuyWfKPNeC/oHMVL/SM3QhO8zH16MMMwCDpinHu7aHEw3OKe+Cywe9wzg X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: ee548953-10a7-49e2-1b1a-08dcb06d62d0 X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jul 2024 07:58:23.7483 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: BttZdFaMUMRVMKVO2V9gLpg8t62YhRuMeL3rOTuUo2CyLCFge7ydZoIx4LDFYvQXBUmk8fqBnetOmQwEItCb+A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SI2PR06MB5386 This patch completes the infrastructure for async read. It treats memory allocation as the producer and assigns file reading to the heap_fwork_t thread as the consumer. The heap needs to gather each allocated page and, when a certain amount (default 128MB) is gathered, it will package and pass it to heap_fwork_t to initiate file reading. This process is completed by the helper functions dma_heap_gather_file_page. Ever heap declare a task and pass each page, then wait file read done before return dma-buf. Because the memory allocation and file reading correspond to each other, the number of gathers during the prepare process and submit process can determine the offset in the file as well as the size to be read. When a gather page initiates a read, it is packaged into a work and passed to the heap_fwork_t thread, containing the offset and size of the file being read, the buffer obtained by mapping the gather page to vmalloc, and the credentials used during the read. The buffer for file reading is provided by mapping the gathered pages to vmalloc. This means that if direct I/O is used to read a file, the file content will be directly transferred to the corresponding memory of the dma-buf, without the need for additional CPU copying and intermediate buffers. Although direct I/O requires page aligned, this patch can automatically adapt to the file size and use buffer I/O to read the unaligned parts. Note that heap_fwork_t is a single-threaded process, which means that the file read work is executed serially. Considering that the default I/O amount initiated at a time is 128MB, which is already quite large, multiple threads will not help accelerate I/O performance. So, this is more suit for large size file read into dma-buf. Signed-off-by: Huan Yang --- drivers/dma-buf/dma-heap.c | 423 ++++++++++++++++++++++++++++++++++++- include/linux/dma-heap.h | 45 ++++ 2 files changed, 462 insertions(+), 6 deletions(-) diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c index 91e241763ebc..df1b2518f126 100644 --- a/drivers/dma-buf/dma-heap.c +++ b/drivers/dma-buf/dma-heap.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #define DEVNAME "dma_heap" @@ -46,42 +47,419 @@ struct dma_heap { /** * struct dma_heap_file - wrap the file, read task for dma_heap allocate use. * @file: file to read from. + * @cred: kthread use, user cred copy to use for the read. + * @glimit: The size limit for gathering. Whenever the page of the + * gather reaches the limit, file I/O is triggered. + * This is the maximum limit for the current ALLOC_AND_READ + * operation. * @fsize: file size. + * @direct: use direct IO? */ struct dma_heap_file { struct file *file; + struct cred *cred; + size_t glimit; size_t fsize; + bool direct; }; +/** + * struct dma_heap_file_work - represents a dma_heap file read real work. + * @vaddr: contigous virtual address alloc by vmap, file read need. + * + * @start_size: file read start offset, same to @dma_heap_file_task->roffset. + * + * @need_size: file read need size, same to @dma_heap_file_task->rsize. + * + * @heap_file: file wrapper. + * + * @list: child node of @dma_heap_file_control->works. + * + * @refp: same @dma_heap_file_task->ref, if end of read, put ref. + * + * @failp: if any work io failed, set it true, pointp @dma_heap_file_task->fail. + */ +struct dma_heap_file_work { + void *vaddr; + ssize_t start_size; + ssize_t need_size; + struct dma_heap_file *heap_file; + struct list_head list; + atomic_t *refp; + bool *failp; +}; + +/** + * struct dma_heap_file_task - represents a dma_heap file read process + * @ref: current file work counter, if zero, allocate and read + * done. + * + * @roffset: last read offset, current prepared work' begin file + * start offset. + * + * @rsize: current allocated page size use to read, if reach rbatch, + * trigger commit. + * + * @nr_gathered: current gathered page, Take the minimum value + * between the @glimit and the remaining allocation amount. + * + * @heap_file: current dma_heap_file + * + * @parray: used for vmap, size is @dma_heap_file's batch's number + * pages.(this is maximum). Due to single thread file read, + * one page array reuse in ftask prepare is OK. + * Each index in parray is PAGE_SIZE.(vmap need) + * + * @pindex: current allocated page filled in @parray's index. + * + * @fail: any work failed when file read? + * + * dma_heap_file_task is the production of file read, will prepare each work + * during allocate dma_buf pages, if match current batch, then trigger commit + * and prepare next work. After all batch queued, user going on prepare dma_buf + * and so on, but before return dma_buf fd, need to wait file read end and + * check read result. + */ +struct dma_heap_file_task { + atomic_t ref; + size_t roffset; + size_t rsize; + size_t nr_gathered; + struct dma_heap_file *heap_file; + struct page **parray; + unsigned int pindex; + bool fail; +}; + +/** + * struct dma_heap_file_control - global control of dma_heap file read. + * @works: @dma_heap_file_work's list head. + * + * @threadwq: wait queue for @work_thread, if commit work, @work_thread + * wakeup and read this work's file contains. + * + * @workwq: used for main thread wait for file read end, if allocation + * end before file read. @dma_heap_file_task ref effect this. + * + * @work_thread: file read kthread. the dma_heap_file_task work's consumer. + * + * @heap_fwork_cachep: @dma_heap_file_work's cachep, it's alloc/free frequently. + * + * @nr_work: global number of how many work committed. + */ +struct dma_heap_file_control { + struct list_head works; + spinlock_t lock; // only lock for @works. + wait_queue_head_t threadwq; + wait_queue_head_t workwq; + struct task_struct *work_thread; + struct kmem_cache *heap_fwork_cachep; + atomic_t nr_work; +}; + +static struct dma_heap_file_control *heap_fctl; static LIST_HEAD(heap_list); static DEFINE_MUTEX(heap_list_lock); static dev_t dma_heap_devt; static struct class *dma_heap_class; static DEFINE_XARRAY_ALLOC(dma_heap_minors); +static struct dma_heap_file_work * +init_file_work(struct dma_heap_file_task *heap_ftask) +{ + struct dma_heap_file_work *heap_fwork; + struct dma_heap_file *heap_file = heap_ftask->heap_file; + + if (READ_ONCE(heap_ftask->fail)) + return NULL; + + heap_fwork = kmem_cache_alloc(heap_fctl->heap_fwork_cachep, GFP_KERNEL); + if (unlikely(!heap_fwork)) + return NULL; + + /** + * Map the gathered page to the vmalloc area. + * So we get a continuous virtual address, even if the physical address + * is scatter, can use this to trigger file read, if use direct I/O, + * all content can direct read into dma-buf pages without extra copy. + * + * Now that we get vaddr page, cached pages can return to original user, so we + * will not effect dma-buf export even if file read not end. + */ + heap_fwork->vaddr = vmap(heap_ftask->parray, heap_ftask->pindex, VM_MAP, + PAGE_KERNEL); + if (unlikely(!heap_fwork->vaddr)) { + kmem_cache_free(heap_fctl->heap_fwork_cachep, heap_fwork); + return NULL; + } + + heap_fwork->heap_file = heap_file; + heap_fwork->start_size = heap_ftask->roffset; + heap_fwork->need_size = heap_ftask->rsize; + heap_fwork->refp = &heap_ftask->ref; + heap_fwork->failp = &heap_ftask->fail; + atomic_inc(&heap_ftask->ref); + return heap_fwork; +} + +static void deinit_file_work(struct dma_heap_file_work *heap_fwork) +{ + vunmap(heap_fwork->vaddr); + atomic_dec(heap_fwork->refp); + wake_up(&heap_fctl->workwq); + + kmem_cache_free(heap_fctl->heap_fwork_cachep, heap_fwork); +} + +/** + * dma_heap_submit_file_read - prepare collect enough memory, going to trigger IO + * @heap_ftask: info that current IO needs + * + * This will also check if reach to tail read. + * For direct I/O submissions, it is necessary to pay attention to file reads + * that are not page-aligned. For the unaligned portion of the read, buffer IO + * needs to be triggered. + * Returns: + * 0 if all right, negative if something wrong + */ +static int dma_heap_submit_file_read(struct dma_heap_file_task *heap_ftask) +{ + struct dma_heap_file_work *heap_fwork = init_file_work(heap_ftask); + struct page *last = NULL; + struct dma_heap_file *heap_file = heap_ftask->heap_file; + size_t start = heap_ftask->roffset; + struct file *file = heap_file->file; + size_t fsz = heap_file->fsize; + + if (unlikely(!heap_fwork)) + return -ENOMEM; + + /** + * If file size is not page aligned, direct io can't process the tail. + * So, if reach to tail, remain the last page use buffer read. + */ + if (heap_file->direct && start + heap_ftask->rsize > fsz) { + heap_fwork->need_size -= PAGE_SIZE; + last = heap_ftask->parray[heap_ftask->pindex - 1]; + } + + spin_lock(&heap_fctl->lock); + list_add_tail(&heap_fwork->list, &heap_fctl->works); + spin_unlock(&heap_fctl->lock); + atomic_inc(&heap_fctl->nr_work); + + wake_up(&heap_fctl->threadwq); + + if (last) { + char *buf, *pathp; + ssize_t err; + void *buffer; + + buf = kmalloc(PATH_MAX, GFP_KERNEL); + if (unlikely(!buf)) + return -ENOMEM; + + start = PAGE_ALIGN_DOWN(fsz); + + pathp = file_path(file, buf, PATH_MAX); + if (IS_ERR(pathp)) { + kfree(buf); + return PTR_ERR(pathp); + } + + // use page's kaddr as file read buffer. + buffer = kmap_local_page(last); + err = kernel_read_file_from_path(pathp, start, &buffer, + fsz - start, &fsz, + READING_POLICY); + kunmap_local(buffer); + kfree(buf); + if (err < 0) + return err; + } + + heap_ftask->roffset += heap_ftask->rsize; + heap_ftask->rsize = 0; + heap_ftask->pindex = 0; + heap_ftask->nr_gathered = min_t(size_t, + PAGE_ALIGN(fsz) - heap_ftask->roffset, + heap_ftask->nr_gathered); + return 0; +} + +int dma_heap_gather_file_page(struct dma_heap_file_task *heap_ftask, + struct page *page) +{ + struct page **array = heap_ftask->parray; + int index = heap_ftask->pindex; + int num = compound_nr(page), i; + unsigned long sz = page_size(page); + + heap_ftask->rsize += sz; + for (i = 0; i < num; ++i) + array[index++] = &page[i]; + heap_ftask->pindex = index; + + if (heap_ftask->rsize < heap_ftask->nr_gathered) + return 0; + + // already reach to limit, trigger file read. + return dma_heap_submit_file_read(heap_ftask); +} + +int dma_heap_wait_for_file_read(struct dma_heap_file_task *heap_ftask) +{ + wait_event_freezable(heap_fctl->workwq, + atomic_read(&heap_ftask->ref) == 0); + return heap_ftask->fail ? -EIO : 0; +} + +int dma_heap_end_file_read(struct dma_heap_file_task *heap_ftask) +{ + int ret; + + ret = dma_heap_wait_for_file_read(heap_ftask); + kvfree(heap_ftask->parray); + kfree(heap_ftask); + + return ret; +} + +struct dma_heap_file_task * +dma_heap_declare_file_read(struct dma_heap_file *heap_file) +{ + struct dma_heap_file_task *heap_ftask = + kzalloc(sizeof(*heap_ftask), GFP_KERNEL); + if (unlikely(!heap_ftask)) + return NULL; + + /** + * glimit is the maximum size which we prepare work will meet. + * So, direct alloc this number's page array is OK. + */ + heap_ftask->parray = kvmalloc_array(heap_file->glimit >> PAGE_SHIFT, + sizeof(struct page *), GFP_KERNEL); + if (unlikely(!heap_ftask->parray)) + goto put; + + heap_ftask->heap_file = heap_file; + heap_ftask->nr_gathered = heap_file->glimit; + return heap_ftask; + +put: + kfree(heap_ftask); + return NULL; +} + +static void __work_this_io(struct dma_heap_file_work *heap_fwork) +{ + struct dma_heap_file *heap_file = heap_fwork->heap_file; + struct file *file = heap_file->file; + ssize_t start = heap_fwork->start_size; + ssize_t size = heap_fwork->need_size; + void *buffer = heap_fwork->vaddr; + const struct cred *old_cred; + ssize_t err; + + // use real task's cred to read this file. + old_cred = override_creds(heap_file->cred); + err = kernel_read_file(file, start, &buffer, size, &heap_file->fsize, + READING_POLICY); + if (err < 0) + WRITE_ONCE(*heap_fwork->failp, true); + // recovery to my cred. + revert_creds(old_cred); +} + +static int dma_heap_file_work_thread(void *data) +{ + struct dma_heap_file_control *heap_fctl = + (struct dma_heap_file_control *)data; + struct dma_heap_file_work *worker, *tmp; + int nr_work; + + LIST_HEAD(pages); + LIST_HEAD(workers); + + while (true) { + wait_event_freezable(heap_fctl->threadwq, + atomic_read(&heap_fctl->nr_work) > 0); +recheck: + spin_lock(&heap_fctl->lock); + list_splice_init(&heap_fctl->works, &workers); + spin_unlock(&heap_fctl->lock); + + if (unlikely(kthread_should_stop())) { + list_for_each_entry_safe(worker, tmp, &workers, list) { + list_del(&worker->list); + deinit_file_work(worker); + } + break; + } + + nr_work = 0; + list_for_each_entry_safe(worker, tmp, &workers, list) { + ++nr_work; + list_del(&worker->list); + __work_this_io(worker); + + deinit_file_work(worker); + } + + if (atomic_sub_return(nr_work, &heap_fctl->nr_work) > 0) + goto recheck; + } + return 0; +} + +size_t dma_heap_file_size(struct dma_heap_file *heap_file) +{ + return heap_file->fsize; +} + static int init_dma_heap_file(struct dma_heap_file *heap_file, int file_fd) { struct file *file; size_t fsz; + int ret; file = fget(file_fd); if (!file) return -EINVAL; - // Direct I/O only support PAGE_SIZE aligned files. fsz = i_size_read(file_inode(file)); - if (file->f_flags & O_DIRECT && !PAGE_ALIGNED(fsz)) - return -EINVAL; - heap_file->fsize = fsz; + /** + * Selinux block our read, but actually we are reading the stand-in + * for this file. + * So save current's cred and when going to read, override mine, and + * end of read, revert. + */ + heap_file->cred = prepare_kernel_cred(current); + if (unlikely(!heap_file->cred)) { + ret = -ENOMEM; + goto err; + } + heap_file->file = file; +#define DEFAULT_DMA_BUF_HEAPS_GATHER_LIMIT (128 << 20) + heap_file->glimit = min_t(size_t, PAGE_ALIGN(fsz), + DEFAULT_DMA_BUF_HEAPS_GATHER_LIMIT); + heap_file->fsize = fsz; + + heap_file->direct = file->f_flags & O_DIRECT; return 0; + +err: + fput(file); + return ret; } static void deinit_dma_heap_file(struct dma_heap_file *heap_file) { fput(heap_file->file); + put_cred(heap_file->cred); } /** @@ -443,11 +821,44 @@ static int dma_heap_init(void) dma_heap_class = class_create(DEVNAME); if (IS_ERR(dma_heap_class)) { - unregister_chrdev_region(dma_heap_devt, NUM_HEAP_MINORS); - return PTR_ERR(dma_heap_class); + ret = PTR_ERR(dma_heap_class); + goto fail_class; } dma_heap_class->devnode = dma_heap_devnode; + heap_fctl = kzalloc(sizeof(*heap_fctl), GFP_KERNEL); + if (unlikely(!heap_fctl)) { + ret = -ENOMEM; + goto fail_alloc; + } + + INIT_LIST_HEAD(&heap_fctl->works); + init_waitqueue_head(&heap_fctl->threadwq); + init_waitqueue_head(&heap_fctl->workwq); + + heap_fctl->work_thread = kthread_run(dma_heap_file_work_thread, + heap_fctl, "heap_fwork_t"); + if (IS_ERR(heap_fctl->work_thread)) { + ret = -ENOMEM; + goto fail_thread; + } + + heap_fctl->heap_fwork_cachep = KMEM_CACHE(dma_heap_file_work, 0); + if (unlikely(!heap_fctl->heap_fwork_cachep)) { + ret = -ENOMEM; + goto fail_cache; + } + return 0; + +fail_cache: + kthread_stop(heap_fctl->work_thread); +fail_thread: + kfree(heap_fctl); +fail_alloc: + class_destroy(dma_heap_class); +fail_class: + unregister_chrdev_region(dma_heap_devt, NUM_HEAP_MINORS); + return ret; } subsys_initcall(dma_heap_init); diff --git a/include/linux/dma-heap.h b/include/linux/dma-heap.h index 824acbf5a1bc..3becbd08963a 100644 --- a/include/linux/dma-heap.h +++ b/include/linux/dma-heap.h @@ -14,6 +14,8 @@ struct dma_heap; struct dma_heap_file; +struct dma_heap_file_task; +struct dma_heap_file; /** * struct dma_heap_ops - ops to operate on a given heap @@ -69,4 +71,47 @@ const char *dma_heap_get_name(struct dma_heap *heap); */ struct dma_heap *dma_heap_add(const struct dma_heap_export_info *exp_info); +/** + * dma_heap_wait_for_file_read - waits for a file read to complete + * + * Some users need to call this function before destroying the page to ensure + * that all file work has been completed, in order to avoid UAF issues. + * Remember, this function does not destroy the data structure corresponding to + * the ftask. Before ending the actual processing, you need to call + * @dma_heap_end_file_read. + * + * 0 - success, -EIO - if any file work failed + */ +int dma_heap_wait_for_file_read(struct dma_heap_file_task *heap_ftask); + +/** + * dma_heap_end_file_read - waits for a file read to complete then destroy it + * 0 - success, -EIO - if any file work failed + */ +int dma_heap_end_file_read(struct dma_heap_file_task *heap_ftask); + +/** + * dma_heap_alloc_file_read - Declare a task to read file when allocate pages. + * @heap_file: target file to read + * + * Return NULL if failed, otherwise return a struct pointer. + */ +struct dma_heap_file_task * +dma_heap_declare_file_read(struct dma_heap_file *heap_file); + +/** + * dma_heap_gather_file_page - gather each allocated page. + * @heap_ftask: prepared and need to commit's work. + * @page: current allocated page. don't care which order. + * + * This function gather all allocated pages, automatically submit when the + * gathering reaches the limit. Submit will package pages, prepare the data + * required for reading file, then submit to async read thread. + * + * 0 - success, nagtive - failed. + */ +int dma_heap_gather_file_page(struct dma_heap_file_task *heap_ftask, + struct page *page); +size_t dma_heap_file_size(struct dma_heap_file *heap_file); + #endif /* _DMA_HEAPS_H */ From patchwork Tue Jul 30 07:57:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huan Yang X-Patchwork-Id: 13746906 Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2076.outbound.protection.outlook.com [40.107.117.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 457AC192B76; Tue, 30 Jul 2024 07:58:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.117.76 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722326312; cv=fail; b=or6KXebMBjpQFJtzSG29+pioafb3VIQ/XNUJ4c53SdBvfFTFEUePxnUxF3dzLpreK2WuT23zA0mBu3GTGS3rY7rcTUFZRrssBgGoq3nRCDZ6m5w2uXw3U0Ko935c3tVrWN7Zw2DzD2vkbrrSOB5XECghSihHrb/zNK2ssqxC9gQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722326312; c=relaxed/simple; bh=WLUeGvduSKmoh3+tNk9tPeun80yiHkwtZRy9dOCxmuo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=m0OqsC3T2bf9b07a6wnDzmrYt2KOzFMUsj9NXthKjXmPAGdHN5goPp2JIn5aKRfews71qfINTx8dwuE4bNo3d808yplTxvRTXozIQ+kjt6mpCiHU8/trBJ334d7Dts0qHhE9Y4lhuwcAzrkxl42RBMDu1cOWQC4O6NG+Dx1NToM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com; spf=pass smtp.mailfrom=vivo.com; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b=BFh6cPpW; arc=fail smtp.client-ip=40.107.117.76 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=vivo.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b="BFh6cPpW" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ibMIILcptAK/lRvWraNbzWECbj2y7tJFcaYT7717xxucO7IMJY6/ZKp+rAQpWxb863s1ti7IP7wEj5X9QdRtJ7y36jBtas4hMdxNPDkGLF+vyFrJHzLMqSutzwCeqdqybXd40VXmZrtocj5SJzs9NtomBF2hk3SJQKGivuZq6OFp/XkeaD9WR1DQgCvqWGe3jOtLt7ti3/SEl6eKcdmkx51wGCARjN8cJxiRNzYefPxhZh7ITbiSBsbK9YymE5ylXtxcN1Uu9ji9PzXIMhOZeWy2uBJgpZzNmxe/KyGliiAk7D3RxxO30RsT8MJSSKDYmWx8JM+s/ecTZcSx4fOTVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gGR4Lh2YwhY6zpo33Uuirwkrwfje1bok/xxCrDBZXJA=; b=HumtZLDjLsHxy7hAb6NHBM6eCZGcgg/GJntOsa80mBZykY0fphAHYfixMtRbkgNDTxiQjNOfjH+4t8DP8Uxazj4zXC/NrGG4V6smuZNH41bnKxaZd2EOKGkmOhanjgHySqqx/zXEDlcKWcmnE3HUZhIirKkauAPjAyoP1DyFQYnJ/1syqTXBel7RAurBaDWxpdcWyMewgvJVVYim8HSCiItPcb/CaDUZobcn85ZUZM4bE2gMot1NIoiBM+AQ5keM5aZK2GR3nSqj22tnlU9Ru9yJ49/yFqJEJNUIa5nHXheKDVA6J6s8NJpEaw6zMFFv3tA9oodhX3eYl/8FbPn5FQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gGR4Lh2YwhY6zpo33Uuirwkrwfje1bok/xxCrDBZXJA=; b=BFh6cPpWm2H1VZHiqRoY6g6qICFZLURZOX0lg+aU9RxWEXBa+5o3PNt4umtBofEiRbPVpm1V1Usck2W3TpozrNIpGcR7Wk8NFy43tL3sndQnGS6HH0JQB/9Ih0O8Ln0v+9XItL5W0shhpJdcWWsCg5IbDYBh/nJBTZOQztj8zVzrNfBRqubxQnVNZoOKlDu7MlFZAURVD8AldhrfTxOGj0AFtr3E4xvuLO8V331VgXzhUiu1DT2yAno0p8AMDGsMQ9nheo/NGzSdhZ6v5ueLh8bsbvdUqzJmjyBNY88k2LzyAt4UPrxuBpkTd5yKNy0jRNW2Chff8a2T/Vo+Nr6PKA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by SI2PR06MB5386.apcprd06.prod.outlook.com (2603:1096:4:1ed::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7807.27; Tue, 30 Jul 2024 07:58:26 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%6]) with mapi id 15.20.7807.026; Tue, 30 Jul 2024 07:58:26 +0000 From: Huan Yang To: Sumit Semwal , Benjamin Gaignard , Brian Starkey , John Stultz , "T.J. Mercier" , =?utf-8?q?Christian_K=C3=B6nig?= , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org Cc: opensource.kernel@vivo.com, Huan Yang Subject: [PATCH v2 4/5] dma-buf: heaps: system_heap alloc support async read Date: Tue, 30 Jul 2024 15:57:48 +0800 Message-ID: <20240730075755.10941-5-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240730075755.10941-1-link@vivo.com> References: <20240730075755.10941-1-link@vivo.com> X-ClientProxiedBy: SI1PR02CA0008.apcprd02.prod.outlook.com (2603:1096:4:1f7::14) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|SI2PR06MB5386:EE_ X-MS-Office365-Filtering-Correlation-Id: 523a96b6-36ba-426e-9eb9-08dcb06d6498 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|52116014|366016|38350700014|921020; X-Microsoft-Antispam-Message-Info: XQo43WgRewK5kpvSKfgdb4ECEQdfCWT1eD0ebSz1dwxSou8oVBEi49XDSJ5C7HWohD47tVB8mDv/G458diLaRVtZuqb26G9shvRiQKJoSpaxmQ1JpwzfEtK3Kx3D46wqsRFgmOoSo743sUlQBKP7fW/74W2orZQZ0NvZBCfVYMD+aJyzOvNvkWoUzlyKjqO2f0LDc+PBkzdhGLnV20R2HuL4wNRMQ9q734PX17BN9YULhIt3WeDThVRLTfomVEEyaCOjdNakHt64Rq7LZJ3eQCbFBZcwNmQAUSq4HykyrE1gSl6u7Wan1aYlJT4fOYMt7u0nvsjkJLOcZCtCOdtq7GS2TOpMJ/WK67OwPBc3Zd4kKU5v9rRAZhbSpdfzIxdcMy952i89tgHQnMOB4U9dIWGAIYZ49mHQ7XKJEDx0207561hu8Q9v5dtCIZQYs6YQwhOcnGhmfeP+T1ZNjlS1jypwhF02Bk9RFvCO7bjmbfRNXIB8RrbvKJPM2e60XmHnqsa+8tcnOLMxatlaL7VzSBssfntJ+Fo1ihe+FGffgWJ+GpCRalfq8flRKsjhvHD2KZVPe6u065OBV2gQPZ1m/RWzylK+djiCvMzJsB3oZ+Tru/3rkCIGIl1NKQNFdRWNun++M9p2nsqka5SiHL61IQHzn19iGK+afZwbEcLL8HJWvZrs6B649auL2iajuU+1Te7hhkdXZ5FW12Otwep37tqBrztlzqngt60nlXS8WiK1p+0L0sVyWIVcK2YIhSSmMsUtbF0JIMlL20mO9JU2u1nYudumFfN8ekfCm0ZSt3gQly2HQZD8veCHxMCtyI48qWDB4vQU35hd8Hr/M9vKOPGNTy7IBZFFR4Geo5Nvc9nntZN2ZoH7YDW2iAIqcuaYkEs8wM4C9TW2lgT8byZX2Czt2jQQw9Q1QsEvpxKAlkkCcCt5CMS1N6aNOIQtZtUuJ//7+EsCtyKtSrRhittBjbLSlIdkykPLXRoz8RrBRW2dy8IlrTZpRO9eC2BO9sBid4tQ6emfUQwtYWHSKNtPp+bXO4Wb1gE64SHijh6FEQ3Yhhuta39/0Emas/6Fxg6GPPSLOuIVAOV3S9kzH0qFB1TXokTNiYx7InifAKTc6RzQ0+Ny8+QEvYkxR1vxCzHCzPblIDAx4ZbB3gfEjN7dDejYywLAUzvB1T2kC3tc4/KxglQuexC9lf0JkBqAObb8tiLjNZ0eNp85rlNeahriM6/ahM6sSmzLP/0Qr2ek6qIpYOJ7jf9Zkz6wACjyCFRH9TlGbAom0XGM3bwE9OpYtept6ToHwGE53iLSmJYsUKACqLS2aDSpkxQ2vgi51ElK2R4kPlftMRWiHKKzP2Y1KmpX0WG1iZfD7HtwsJqI1mC1VlMzscztVuI9cVSoJn44aslOjz2sPJFf27jp2JX5I4I74tLiKU4TmZu0BXXVs78= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PUZPR06MB5676.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(52116014)(366016)(38350700014)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Rzs6RQHbab/xiuEBYw0QujHca8xB44dT6umiaA6GSD33EuRRyDCNo8W0OU57S2PcyK2lGn6Y5Vh3rZvw+DFquZJkOxZ3y/HxSm4ViFUNdsCXlU/vrFpm63iyxINl0gVcPHLrnzY2SRYb/owyt+bwrfHg6XUnQGtPBdhUn5ECa85KB5zOSglMqz+stVibBGnM51lr6NYDoMUO86dDNHnDAhsg7EBTyqKmVaZWXNgP6Enz8li0dPSk4OCBq7oBFax4UCS2lRJaXGltZ1IOGNJfW53bKkJ9Haax+i64pf1C6/pFgwDZFGaEAC6Wkt8Eu+Swiz3YA4KAcYIh24AGncEklgvMZN7I9l9rjcufIiIlVnBfvNoPJsLWh7XoLJtdLKQRXjx8btGqUmjLnCO+c0/2p16cD5gkOznq7XC8zMSEXWs5xLCJddd/3273LEj+LSA6lle+Ge7RvBdbsGvBASauD6AOCZVSu0MkHMl++zpM1MGHM3RKckR4jnRVV/oicnRVEhMsVgld9xLRjkuwhQqjSCsIaOX6carXIHDrhHn5A00RGESuPkoRvE+uysiXGnCq1F45ENJGelB6wQpw8k+d+80lVD2H1SPyZ1JEyyxHJ6olhlvVxZ9MoIxPTZRJh5+PJcLKkCo1jX0pzLIIABDVIxCm7iajsabok5JABKXEXaHentnceRHFzm6pTdwpvWlV7JFLOWPEOw2sxNEN/pklmttxs632sW4dXGLxQHYPCbE6FD10qVwkbA0wcE0EBiMxBIw5lkwfS9InroqpRu8CMUYLsRpSX393skXzlSkZKzwr6JFY8doSIYEB3KqZ+W8RNQdKQXEeCr1gq+XQieJhqvvAji1phM8dEXgl/U6s4XRmnRjtZpsBOx5suESQuzI+eDQHgJYRZgyiYhozVOE2uZyvqOHyEWI5R36De2Hp34ujtevSTYMig5I/lNmL2REn1jW5dFp8dEmZIJd+0LvWJSnmmKkExsRTb1FvJAB/CiwaDtOapLSbEQz1ENJnMSD9C69LRQlofqSUdo52apLT/lVOVck4OqnebzpPaERG5waOiLeNIbK/QwZdbEB0fejxP7HhxIUBPLHOqrFpti+u5UHvoRPI2C2N0ZUGCNhVlthb/JEnGW0LGvHJIgtPrlk9iZR17Gh2/Td22rLSxdDRc71AXMoiC8GTQJQAw9uhmBVD1UFvsC7TeWcRmSHSbqUDd2MUn7CRTdJZO/QdQiB/Em6QhpGeqPgWQD13KFPjq2xminLkoy0k4OLbFk+Kh4Vie6o3r8qvZ8R20wfngwycobEczDlFEJSBmBVc3ZQy6cA1rPsxzm90kWIKD1dS5UQnfrqQXGi8JV5CLT6sV84b20c/sFN98+UHRpg9yKMNv5LHCQND4uUcVnL6XIAqx50DrqsfIcyvpjeSAkvW4CLaQm9Mg7bQzq/U+C8BqY9bEmS23G02iWEaDt/Oq+ofQdF8lKPtyAWnO3gWIhGi/BpAGRDAxD0RUxiiyFQMm/NKHfvmkzbSOEJK1tXoiilRdItB8KgthGg9LNLgxdXU2LBC59W6N2fxsTMKF7/Qn4fQnAMwJjlKD73C4JB0t2fnQoTX X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 523a96b6-36ba-426e-9eb9-08dcb06d6498 X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jul 2024 07:58:26.6794 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nG7mh6LvnCbRz2V/JfUt2wJNVJlzC3Mr3fIG+QY3t4pkgglaAWaRZ7jBnTsoD6o/IKpiQ73zpdReDnS/aOPpLA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SI2PR06MB5386 The system heap cyclically allocates pages and then places these pages into a scatter_list, which is then managed by the dma-buf. This process can parallelize memory allocation and I/O read operations: Gather each allocated page and trigger a submit once the limit is reached. Once the memory allocation is complete, there is no need to wait immediately for the file read to finish. Instead, continue preparing the dma-buf until it is necessary to return the dma-buf, at which point waiting for the file content to be fully read is required. Note that the content of the page cannot be modified after it is allocated in the heap, as it may cause conflicts with accessing the page when reading from the file. There are currently no conflicts in the system_heap for this part. The formula for the time taken for system_heap buffer allocation and file reading through async_read is as follows: T(total) = T(first gather page) + Max(T(remain alloc), T(I/O)) Compared to the synchronous read: T(total) = T(alloc) + T(I/O) If the allocation time or I/O time is long, the time difference will be covered by the maximum value between the allocation and I/O. The other party will be concealed. Therefore, the larger the size of the file that needs to be read, the greater the corresponding benefits will be. Signed-off-by: Huan Yang --- drivers/dma-buf/heaps/system_heap.c | 70 +++++++++++++++++++++++++++-- 1 file changed, 66 insertions(+), 4 deletions(-) diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index d78cdb9d01e5..ba0c3d8ce090 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -331,10 +331,10 @@ static struct page *alloc_largest_available(unsigned long size, return NULL; } -static struct dma_buf *system_heap_allocate(struct dma_heap *heap, - unsigned long len, - u32 fd_flags, - u64 heap_flags) +static struct dma_buf *__system_heap_allocate(struct dma_heap *heap, + struct dma_heap_file *heap_file, + unsigned long len, u32 fd_flags, + u64 heap_flags) { struct system_heap_buffer *buffer; DEFINE_DMA_BUF_EXPORT_INFO(exp_info); @@ -346,6 +346,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, struct list_head pages; struct page *page, *tmp_page; int i, ret = -ENOMEM; + struct dma_heap_file_task *heap_ftask; buffer = kzalloc(sizeof(*buffer), GFP_KERNEL); if (!buffer) @@ -357,6 +358,15 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, buffer->len = len; INIT_LIST_HEAD(&pages); + + if (heap_file) { + heap_ftask = dma_heap_declare_file_read(heap_file); + if (!heap_ftask) { + kfree(buffer); + return ERR_PTR(-ENOMEM); + } + } + i = 0; while (size_remaining > 0) { /* @@ -372,6 +382,13 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, if (!page) goto free_buffer; + /** + * If alloc and read, gather each page to read task. + * If got error, free buffer and return error. + */ + if (heap_file && dma_heap_gather_file_page(heap_ftask, page)) + goto free_buffer; + list_add_tail(&page->lru, &pages); size_remaining -= page_size(page); max_order = compound_order(page); @@ -400,9 +417,29 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, ret = PTR_ERR(dmabuf); goto free_pages; } + + /** + * End of alloc, dma-buf export and anything we need, but maybe + * file read is still work, so, wait it. if done, destroy all file + * task. + * But maybe something wrong when read file, if so, abandon dma-buf + * return error. + */ + if (heap_file && dma_heap_end_file_read(heap_ftask)) { + dma_buf_put(dmabuf); + dmabuf = ERR_PTR(-EIO); + } + return dmabuf; free_pages: + /** + * maybe we already trigger file read, so, before release pages, + * wait for all running file read task done. + */ + if (heap_file) + dma_heap_wait_for_file_read(heap_ftask); + for_each_sgtable_sg(table, sg, i) { struct page *p = sg_page(sg); @@ -410,6 +447,13 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, } sg_free_table(table); free_buffer: + /** + * maybe we already trigger file read, so, before release pages and + * return, destroy file task, include running task. + */ + if (heap_file) + dma_heap_end_file_read(heap_ftask); + list_for_each_entry_safe(page, tmp_page, &pages, lru) __free_pages(page, compound_order(page)); kfree(buffer); @@ -417,8 +461,26 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, return ERR_PTR(ret); } +static struct dma_buf *system_heap_allocate(struct dma_heap *heap, + unsigned long len, u32 fd_flags, + u64 heap_flags) +{ + return __system_heap_allocate(heap, NULL, len, fd_flags, heap_flags); +} + +static struct dma_buf * +system_heap_allocate_async_read_file(struct dma_heap *heap, + struct dma_heap_file *heap_file, + u32 fd_flags, u64 heap_flags) +{ + return __system_heap_allocate(heap, heap_file, + PAGE_ALIGN(dma_heap_file_size(heap_file)), + fd_flags, heap_flags); +} + static const struct dma_heap_ops system_heap_ops = { .allocate = system_heap_allocate, + .allocate_async_read = system_heap_allocate_async_read_file, }; static int system_heap_create(void) From patchwork Tue Jul 30 07:57:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huan Yang X-Patchwork-Id: 13746907 Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2076.outbound.protection.outlook.com [40.107.117.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 94886192B9E; Tue, 30 Jul 2024 07:58:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.117.76 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722326314; cv=fail; b=DeW+YM/CU5qflBLZV0NdBfYBdhOpwMJSC6s/aKZ9nZJussanz4MQAKTciASUynQmjBu+svLlbzWPDMM/UVq+At6S2+4CPcqkBaaiUY+IJ6A5+gRIanSk+5sZ3KU7ru11/y4Zuh9rghFkF6p5gk/HmH1mBWSdne4vjEBzS7ePnKw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722326314; c=relaxed/simple; bh=/ulFSipo6OAt7vlHQA53ySXbJgChgaI+l7LsVciRZFQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=lONlpgjsG4m9HCKl48yysTT8IZr7TsVtxu2qTvbRAicDUEwiKsPjVVbPowawgTCIFr4mL5PQSM8/nCWDRDvOr5Tju2TLVkG7E+t5hklShyOX42Bbi5egUC6jo7SIaewLm/DroR5wjGa//TwZkbE4XAI8hZyb9eLnN8OERRNdhGk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com; spf=pass smtp.mailfrom=vivo.com; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b=Lj9XAYVu; arc=fail smtp.client-ip=40.107.117.76 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=vivo.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b="Lj9XAYVu" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Kf7S6uYNuv6kxyx30fnD7OGu0Cs7uZvPiJVPSC+h4A6d/szezgwVLZ9+VpP5tFW2qaipegBWlLS5/Ym11vUqRTOQR7bugk6S13GhPc0JOBTZaxUdkQXwdvdfihjkVjnQM0KJkshLfxLgDbMUDlNO/6ouLxDdVCPK9taGVRRpnA+sC/bTNOfEj4Sq41bzkD52vcidgl5Yb/nY1K3+wtjvk2CKfgamUxilfCvl8jkbbPELape+X+aeakYQ3X+zMo0Ive4UYBve7rz/3OqAQ6b/BqpwihCJJ7Qdv1YsrcIYc1chQVqRHTnoI9YmhdsSwsLCiCCM2ALEBeon5nGIZO1Dpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bMhwemUUoB/3WcjWcxAIFyG/JasZnHDkItVkTKGRmO0=; b=xgjc5uXUQXVGkYzbHX3w+50ObcgBsaEYgpuRyimwVBQWd7lM93WMqw/b6HT/wiZDlHPDm28I0jNK259Jr6OP/1DO2BrFt7htxJBI0LCS9pBGEcVTD8gFWbgUxEjJ7R3Af/poib88A7pNSO7qPJyVMU3d6f7vu96y9r7KSGH6Wp1c2SlWVsb++sq/1yvqDZkUet7duFKCndGUN7V3ShLSAj+uihjNq5VPycdZwjtOG+P+20g+LOE2oCJJ3e9eCg8hswO31P2LOyGfHyA7LKfw/M62kbKGqiZhWbU3rrzZBr/paM896MxywQ2S1Vzrvt3z3/1zFhOOrBbuIUvYR9WxMQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bMhwemUUoB/3WcjWcxAIFyG/JasZnHDkItVkTKGRmO0=; b=Lj9XAYVuZ8R6ZsK6zx5iUmEGPzgVqZztdiTqwePe0+mRi9tMrS3J/gxLYaYbNbcJpZg72WcOuYJIEfIku37IdXbVejsYfkTt8LxBdC//QpmCvG+ZVQSf6oH0OMCsU21vzZsCXP3XDDfGtrXTws2BQh3B2OucLK9oOHgFKkP98lCWqavu7cB5AtoZrBuhE+yB4MQ7ahIhG5cvhOc14EIQi2gmyGZFy/tV7kD0Dn3I9iI1c9fzF388pQQlDjDiZHJWisT6t8ZAA4BW263oU06sEF/rEcqOTbakSFt9MdXY4kU76a1ZvpFq6MeDSBoxRC4/hgbPnMttM8k49To89xVIig== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by SI2PR06MB5386.apcprd06.prod.outlook.com (2603:1096:4:1ed::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7807.27; Tue, 30 Jul 2024 07:58:29 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%6]) with mapi id 15.20.7807.026; Tue, 30 Jul 2024 07:58:29 +0000 From: Huan Yang To: Sumit Semwal , Benjamin Gaignard , Brian Starkey , John Stultz , "T.J. Mercier" , =?utf-8?q?Christian_K=C3=B6nig?= , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org Cc: opensource.kernel@vivo.com, Huan Yang Subject: [PATCH v2 5/5] dma-buf: heaps: configurable async read gather limit Date: Tue, 30 Jul 2024 15:57:49 +0800 Message-ID: <20240730075755.10941-6-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240730075755.10941-1-link@vivo.com> References: <20240730075755.10941-1-link@vivo.com> X-ClientProxiedBy: SI1PR02CA0008.apcprd02.prod.outlook.com (2603:1096:4:1f7::14) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|SI2PR06MB5386:EE_ X-MS-Office365-Filtering-Correlation-Id: 99971946-0b0a-4192-4e2b-08dcb06d6652 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|52116014|366016|38350700014|921020; X-Microsoft-Antispam-Message-Info: kqaTSyl0nuEklRGXrlP6kU9dvC8CrjW13PytQsT3RiGLtmBXfrTROZAvgF/8L9sMyvARcVpm25NcGgcqMnnTP87gtsuSD86DOL52pQM9V89Lx/YE9LeTdc5vw2joHuOekp9kp0VTOpKYtbclcSToLjZzG49cb35mBfflywL1pec4g1BH95tKZ31/AgwWt4H+RLnBV8ZX6VX8E4gYyTlex/waVjzS1gWZ9TG0tIFI5f7f9rkcQlxsBG/rnw8a1pDpM+ee8GpBglzZUYqu+H0edxV8WN2h0p7L821n0YzPt7bXHGxrx4rlzQ+TOL/apfoJ3Uju2SmhBrPx1wG+0YQQ2L4MEF3q8etDWAD6Pf/zX8wWtyr6LXE0TjWqUeIIeKko0BTxPNh3K6u4HIQi9CLG5QkMq7uuyQ0EIMFkwA2h4rIRo+ykxiAAbRy02jfZglpciln3coPT8UaT4q5PTj5nTDJXJdD3ckUG84Yla0Erei1MLo0Ij531Lj4elmxdQbJdxHto2C4Ii4zxyNMsJ2rVM6hcaBoZNp2MoAYSLo5T8YpDzzOIa+tX9bglA1moqDHsTpuGxJg5OWqUWxP707CQ/Ff4kWlsVZjOUA2z/vYqmiu+mLEtQjEHjzh3z9BRj6/EYK3qYQH+xIySPFUiDI/iBJd7u2JhuOSuH9xpQQHnm0vB+lVkCJxR18Tymdwqel0wTWj4giwcDWaaqtKsaLyvlVcbKgDcFjCxxXG6TDYlIF3edRK0vmiHtDk+LcYKOSvAFZd//Gyym3NghXSjciiTKj6UXN2uQTCZ6xL9H/9XnTjadTFirORAMWiVc+HZ87/yK5X5dTODH0Qqsn3lZU9P58b/4l4FyA0HYEo8TLfM/f2ytF5AtN8XRZaYldZX5J3wejSDL1PQ2/yb3Coc+qtPCqsg9p2XRY6EX0xD2zGjrxaTSZzImU+K4xt8ksGMC77Ye9f3RBqws4sVmkZWGghfe2S5Cg2H2DdkqUtpdD4LPT9koltHsthxL/lKewrJG3nXRiB8ruha+RRkT7qEXxlW/yyMr4pif/IXgFzSE7vVqiCwxkL8avxbSOZ7LgpHkk9MVTwu1UIAIlJlLGg6MliOjCXbf21wKMhV2MB+qJsxz9bi6GrtSKTsLHCCVY6OxHnHvK4jx0uZtlCJz7EUktqJEscAPhtxf57ont6sB9y7CU7TNl4Z9px5ABx7jrvsR9D+jM7p11ZMMvDlsyN6YMajIFsFT4nQgKsekfTtE0lUcNu5DYqhBD3Qnn7Crb1AR5jDkXyIG1RJvyWDTmgm/gMiSgqnuB35x8OPR+YrPgrnE201Dk++P5XaML8ytba2gmul4uVd8oStPwuD8xt/pzwzN0U5sivEzMCR+hlJ21/SqqZKm5LJ+VDo1+5JKOkqYLCpI8oc6X+o9KFtfSyGsve842q4V2A1l8R+BnypV7leHr0= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PUZPR06MB5676.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(52116014)(366016)(38350700014)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: fZqK6JJojZC9m15E9DrjVKd37EPpsoTZyk8yLyqMyq0/eYI3e+qiFCKNOrJGSAEJ5jpDBp5VP1f+OXIktT/RAy2SLXdnyG+4lnrEyToiuidh/UV/3PnpZpjz8mEx0VsCEGtLOOluo7WzcB+KhpM1zO2DueJiSnXDxVXXtoCWAO6wvVg0UY7RFV2oaxt2ND1FwFoHO/+YlV4yE4LAwmm2lAwRFJ/MA9ajA1ovcaAyVQftuogc7TNsYqbnx/65zkAVno5/SKGBJAZHtv/sbpMHpibxDlHAoa5pi5zgT1mjziS0FR8q1viriYIpvM5uPcr+Ci/J2dsjwbyBx4gHLQcMwr+0xVqBDA3AqEwSgbt9pUpBSY971O6fkhA3HY+PYrD6jVVWkSjdTJ/WIf6Pkq6gygpuR7M2g4L9z4bd7aZ810q6ozNe5ErsoJKG1mSXHC9fcMr0W0saqoBG4JBdr5K+udi/uLrdg0V0U58Olzn45wGfJ0vpaW3jG/SkESspxALZ3gosON7MDUo02FQ7doVV5YU2moOzlK7b8J0wnIXfTWkJnxQSqOr1OrgXS01dct79OpNeqJYcrEpUY/kwztHe8WBJblpfDTXZgtvbUeHoehfFCbdHFCoSYzDb3ZbQTYpeAYWKuc7+vnznrLb/SVBdiI1m9tZQ0TSpvCd+3rSpkwJJxvl/OoEnnL4VdB9fI+19NXYbuBXEGFWz720gq9xtzjJqMP31OQu7fvuKVLMtdxhTX41S/JmKebygjlwLM3PYG8o+cI1oil0VCAa1Ga6fDZhhMGNO8ueDcXb2ueCWgq106gxtg4S2fch/tc+eQW1h1PwSzFZpjwBB3WZZJQxfK+cNdQ65WSG2UpnL/9owXKNc3qBf/NyLsztQr/YNRW8YBppAa1LK9mqFVE9/PnlrZgoupDTsFBsNCx1lYbQQhHKuXBU5NwWGpkZnta1AVEs73Y73hgijbW+rN61SHG5sPXWOoNIvDPaeghjwLqtlTW870zViOPD7X5veemo5Joh8Gi5bPZkmiA3j4g/S+HpeUQxOYaCt6+HfcpV98Zh/Jcn8TsBzGR7DsYViBfAbLl+ksfggmkh1mxmlsSrHce0bU/igT3xTZ7qNweiIdSqRg/j3UQMetW1s4d/KW/O3df9jZSrmc84HRT+kJQXSIk+wGFo7Usj135i2NaViFlSE1QvwlIoK6EETWNTS808VUWqnaq94oMUPSskdlXRMni0lg5MXJ2V60HfCzbozrO0hXLxW+ncxb5vZglqfhuYBGb9Z9g6AfV+yHYJ2jNIyFPActMygdm6D3EvGIQ5Ge3Akh3TmKCnHjjYIMYXWY0GAORiIjOTQNrQasLziaS0cVyRCslJbyggYoHLkKoehpyiDV816XBvaPJsvJcwKOv1QDRR8R4vVqOsDfhHti6jLV/DUgHyeLAAxqWfXmsXHJ/ayggNdBzrVIumTUqFaDXmWnsFOgdlrIYQprS7G0t7/KzoHUc9ezlKmzR8CShWDmrw1MQMVP2y+5ZSt0MMdfPaW4lljsy2RAMBFvkMxIDhAycJ3dDjVyyVSwBzNIf9GmrlaZC4hFPOMciH2/5ni7yDn0hpI X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 99971946-0b0a-4192-4e2b-08dcb06d6652 X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jul 2024 07:58:29.5867 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 4CWGcnBzisIgCBGkZCAx3UWENPTN3tDJ2J4o0x2JGfS16XFkaKq0LMR/hkTiO2VsN117nOGTEzKXdrodMvDOEA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SI2PR06MB5386 The current limit default is 128MB, which is a good experience value for I/O reading. However, system administrators should be given a considerable degree of freedom to adjust based on the system's situation. This patch exports the limit to the corresponding area of the dma-heap. Signed-off-by: Huan Yang --- drivers/dma-buf/dma-heap.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c index df1b2518f126..2b69cf3ca570 100644 --- a/drivers/dma-buf/dma-heap.c +++ b/drivers/dma-buf/dma-heap.c @@ -417,6 +417,11 @@ size_t dma_heap_file_size(struct dma_heap_file *heap_file) return heap_file->fsize; } +#define DEFAULT_DMA_BUF_HEAPS_GATHER_LIMIT (128 << 20) +static int dma_buf_heaps_gather_limit = DEFAULT_DMA_BUF_HEAPS_GATHER_LIMIT; +module_param_named(gather_limit, dma_buf_heaps_gather_limit, int, 0644); +MODULE_PARM_DESC(gather_limit, "Asynchronous file reading, with a maximum limit on the amount to be gathered"); + static int init_dma_heap_file(struct dma_heap_file *heap_file, int file_fd) { struct file *file; @@ -442,9 +447,8 @@ static int init_dma_heap_file(struct dma_heap_file *heap_file, int file_fd) } heap_file->file = file; -#define DEFAULT_DMA_BUF_HEAPS_GATHER_LIMIT (128 << 20) heap_file->glimit = min_t(size_t, PAGE_ALIGN(fsz), - DEFAULT_DMA_BUF_HEAPS_GATHER_LIMIT); + PAGE_ALIGN(dma_buf_heaps_gather_limit)); heap_file->fsize = fsz; heap_file->direct = file->f_flags & O_DIRECT;