From patchwork Wed Jul 10 14:09:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lei Liu X-Patchwork-Id: 13729628 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E8553C3DA41 for ; Wed, 10 Jul 2024 20:10:22 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F070010E027; Wed, 10 Jul 2024 20:10:21 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=vivo.com header.i=@vivo.com header.b="Slec6BHt"; dkim-atps=neutral Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2070.outbound.protection.outlook.com [40.107.117.70]) by gabe.freedesktop.org (Postfix) with ESMTPS id A160210E7E7 for ; Wed, 10 Jul 2024 14:10:20 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gYuU/+Vr/j1FBoiakqKsyeoK/KlCf820pLltB2Wj1NlQXFe/DBIpRJ7Yxko5VMAG2Orui+hMgqVRcSeg92VGTRkpjrzSki4f7FNHf0uR+Or48+jZFRHFk2ajnoGe7gSpl8y/WtoGb0RFD5ShYhwRH9up72szwfdADbnxN8+KAvikGvuyED5VppGAZLMry8cYKmFIraire6kTzU/IokrHtrr94antfe2YIZI2vQjOH181p37a64MPMyJ5QOYmRCz6PWgzbvhcCsfYHsn8Fo5z6vyjELmGMNE6HR1CtL8/9brmGF0ABw1sBwkRfb2AiBDLTuk667XGmFnoCvAc9M3Zgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZGQLyyp4O5Uf6YpKS7Lwp4qMYi9ESWvBfGifXLA6dNc=; b=LKTfRrxJQ4BHyK0cSNGSw55ABvKRAcs6LSBkZ5zuB4gOSgkCJ80g4RE0wyw2JcrugKPPIzZ1eUHsd97x99T7NtHfs3hxPni8Wc+drgWCZHGOF7TmzMJXQ4xIZ1uSiGGEfToa8mvXi51nHDTqrf+P8nRo3rt2x1DKeb++E/+HwNOjLbza+6j4F2yOs9+oN6l8YN5qBQoIjXK+lTb3sztOPMrt1HATr/zdTGs/D7czLBeIa6I9Qzjo3MP2Yk5VQY/f+XH6u0Bo40gj8VYmEgrzKthPZUcEBORhMzrcZ6sZx2MbdE6kqRW31I9+GGgrL2vOgUwajxP6CSemRYPka1fpnA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZGQLyyp4O5Uf6YpKS7Lwp4qMYi9ESWvBfGifXLA6dNc=; b=Slec6BHtOm8BvZyT4FtWypfXyaH5Gj+65WWNm0+BnWXsZdxtg/2BB4R7aDkMLi5UyDexgWhEUYR2DzglnxjqJIlZ+X6H41iGr8s/FCkhqZz2sjAfWc4oWMXZbeKK/tbOFe2YR8m1nBB2uX+kMKCs279rjwZrnvtlxrdOY9kVcexG5thhKCKehsPttSLxOpQej6kFRNaRxlXllxRPFD3vEhQhIpsGkfMnFDlLXGp8V9SKUN4JsjFH0QioCeZ+eXUobnNBd0nUzZzLEMUS4095I8Qkl+qGL7qIlhbD4RCeMY9gQArwkUjJBDt9hD+/hpjx1c2kPB7qOxiOKV/pcQa1Zg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from SEZPR06MB5624.apcprd06.prod.outlook.com (2603:1096:101:c8::14) by SEZPR06MB7077.apcprd06.prod.outlook.com (2603:1096:101:1f4::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.35; Wed, 10 Jul 2024 14:10:14 +0000 Received: from SEZPR06MB5624.apcprd06.prod.outlook.com ([fe80::e837:10e3:818e:bdfd]) by SEZPR06MB5624.apcprd06.prod.outlook.com ([fe80::e837:10e3:818e:bdfd%5]) with mapi id 15.20.7741.033; Wed, 10 Jul 2024 14:10:14 +0000 From: Lei Liu To: Sumit Semwal , Benjamin Gaignard , Brian Starkey , John Stultz , "T.J. Mercier" , =?utf-8?q?Christian_K=C3=B6nig?= , Andrew Morton , David Hildenbrand , Matthew Wilcox , Muhammad Usama Anjum , Andrei Vagin , Ryan Roberts , Peter Xu , Kefeng Wang , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: opensource.kernel@vivo.com, Lei Liu Subject: [PATCH 1/2] mm: dmabuf_direct_io: Support direct_io for memory allocated by dmabuf Date: Wed, 10 Jul 2024 22:09:43 +0800 Message-Id: <20240710140948.25870-2-liulei.rjpt@vivo.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240710140948.25870-1-liulei.rjpt@vivo.com> References: <20240710140948.25870-1-liulei.rjpt@vivo.com> X-ClientProxiedBy: SI2PR02CA0031.apcprd02.prod.outlook.com (2603:1096:4:195::16) To SEZPR06MB5624.apcprd06.prod.outlook.com (2603:1096:101:c8::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SEZPR06MB5624:EE_|SEZPR06MB7077:EE_ X-MS-Office365-Filtering-Correlation-Id: 3cfefde5-2dad-47c5-0a0c-08dca0ea049d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|366016|376014|52116014|7416014|38350700014|921020; X-Microsoft-Antispam-Message-Info: u/aGUDZfojbwWYKcNYHVDFmiCyKFIraR2sPvEsUh9bPqEMVkTm4NsMpRBgIHFJgex0cMg/Wxu7n/zIw5B7kH2zn+ICj0vidb2crEgOjpFHaC+aMwcE9khh0ieo3+fHQE7Nh0VbgGh4wOpvMtZ/FNj/thBep1lSOsl1mBiVfklxrFWNK53T0UQ4FDmgc/pfEOD7yI9FN4aujyuh6jfH/KvNEEdNEk9KXxS7KLB7Tjq8w4FZQpKxLpHWsSAlihziPJpSkAf9QpJswImGVKmD65q7/Mxn8ZdPSGjZhhY49Vpv7pC3q5zsV8XyhnudLmL6Trbe0xIkmDtUXAiFpcreBGjEpPKIjZTFrIvPlD7M+j4/yfVCRf7eRUTvduRfA2F0VmaXk2mJ2JHKvcMUBz9wRNnP91PvM+8zVNmRA0WnQFEy/4uyqxfZ52GKcHuwAnsodcEWBUVKbNsWgfHxRowaN1hkDMHMSaJdwmZkjMG2bXvAMxEYdx5B5JydD/uufvOCnrCFGltUFHv7ftWb/Rs709CX3sWvuZlDDgJCEHL3bWs0C9Uggbj75cMhgDtprCB5NQDEXiNARcl1riCHZd+2BbKkq6qEhEuVqmxpRPsuhvXdxO3og7KJCNYF4b06heKto1NQzffb2vT6qO48D/k/BCO1x5X+z6GyqnmmGexd0CMBfggZHy7Z3kk9OgHawJdINSy5l/MR3ermHAw2CgcFnfixSx2/FGk53yZ9Exxtg7pCicSgyskddeCYzoUAA95dFeXgdoh2lDLaFfeYFvagTJ8mSLp46JPGZx0JFAgexWhikL0nxrqJYZUEhtpQUbDYnNc7yhQyzSf1nXRh+BqXPZ0jQT4bWwHwg+KJG7dyybirvthLC/qNbyo6vAelTIpIGztCnF1O62R9Hhjrn2vZB2+LbWp9QhySM7ihcsGBWiTMvE+9TBc7W194Ff/oEh8zDzb78//S34TvgpI8tyTw1RbToRhi/hTCOa/Uz/hoWQmruW0OhLixvOH/8PMy9Iq92qLL08FZ0YIi+Kr6TvpbCFS61ieJEmsWUR5DytFGniOIJcXPfD0m6EGx+r34jhLElbHmDoZOhBD0dPFj0r6PbV64p4LqCa2+3eldziTYcGpu1Vk+WcCssNRnGJxHYlYUfx7PtKR0vZM/38xGhKRmnVsL3Tk3kxwfK4jgHdhEbFV7iCVE2AriEeP9zhJmpTc7al7vHyj3KdqZppvQfs0aqH61IhA3PgihCO3s0LmRCLa2CzsaV/tNwIUebXDdprCpnt/VgBgrp29JAdWZFD565IlaXfH8QSxL0sY6cY6M/n0bPMFiQSYViigNyQyv+WBQIKZM9CtSpguseQCApZaqJVsexVICWLOLsaIQ0PA3dt3WwMiVhqeUH2cMBbIXZkvd2xz9jXNs3aVEkJUol8S6MYptV4ompIz7JWr0bRXivU3hg= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SEZPR06MB5624.apcprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(366016)(376014)(52116014)(7416014)(38350700014)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: FqdvLa6Suw/HRehljZh1WxQ4ZNEJp6x5N7gjltMqknWku/y/ggrBocwzW++O7297z7sXDkQpY9xP3aApFOKP2qARr60Zp07VkmZqhEGGP8GAv557mLw3fX8eoxNSAtnhj2L2gvC/uc/xxAS4ZvitCWI/xo+UjUOhdMc1jookolYN3zuw3kgy98AejJn/vh1hk0z/tYH3oIKTwxXbZ9mH0+vMd/hSSgqdUYvNbHtUELbGS4B5qaMRIx2H3lqSCWvIoygzv06+sm27+37OMOJfcfSS1msFcXdTPgwYDMp0Qc753vZSWrdGT1Fgj1+d7vZY9vGtjl0aFrxCNJNw8FX/APDSSgRowldg4frzPsUu5QwWnMQFyQ4jx8PdcJdK3tlTbW6a2oqm3ielGuuGb/KoUl2gYbmxggX6D6zm6NPjuHBHlKHTyYEKyooH3zoEQzlfYeuZoODMeeX1McS4U6C/acJkHC9PkoAiewCocecMQr/njHlZwXH/GnWfiJctIXNQOdNppeUwsFUef77mtTH2tDd6mIVcrikg2/1taWh7Sd/rQf5ykZUFtvLtIGmrK0O3LD873kYFf9jznuRuOGs2jhRF9caG/hZl/mCJvwOKsbCQP8cacr94oC30i+rtS5wUgts4lDradb5w/BLOxFLfCmUyGVW5crZ0mWoJQS7eMOqAQgS6xV/kMLJuAuOpqk6wkXf1UfCVRIL0hPM1mNkfPsHpLaY3KgTN3m0ctQI9q1c76krVuaYv8Ti9+mCsQCAY0gQQRoS/trQR/TXbvqEyM3pmK5M56iqu1mZjlDIZVILK58rARQbbmtgBEbSJ5S/jRm96W9e60yJAE6n4QnWxqR8iFoJJ5y1wCbiyunZg5zBYJiE18al1OEfXY1MezeEXXsM1jOtLo/UWBMlVqcPtEbuldE6+PEATXxhEd+mJEyP7vqtnx01Wm+IVvzIW+34Qvk4IR0WS8lIvqdzbqs1zfYWY19DxJmVA4kG97fYv75hZq1VCv5I+MN68MY3MgLnjdEKtTX66lRjb8HtxslnI9crBnH6qKZ8xsKpzeaA2SO4+KOUvDqFFDCjPybvEgOKqK5jQNUTVCYxRRtBs0Q8XT8j/u2V/hABNB+ysbktRcX6+fGHVSx2oBK/CmsxY3Ituv4Sv79LLREPvxQMbhlClgjzyR4P6N7+VXzf4ZHr5DBwoLbXoNalHcm6qcWBF41ALKBic/gLrPZF0WH3MlblzoRmbisk/d9gHCQ7U9S2xxy2z3sRVsw4edD5J/wwoLTShXVbl2nIFG1IpIDkGltyIzMrsKPGZ9I8dJmaqjtrFWxgVrZF2aqVLMRLA76KP5N8CnYihPuj2ALx3NJquaJjrBE9TuDJoSU2XXRYfZj5IBVV87i+APu/Lzat06mX7SuvFk+KiMXGmAY4MUOG/rZhLwyARmc+3z//2fiTLHc0t0rG0IqRXhEf28Rk87YU7w6QYo0h8+gmbLk7F2ZtGWRnHf4hYbCHvhAGMtpmLmy1S5+Q9RT8ntuYHQt4+7MHrtOVQfSh33ItJC3pTJ4co89ZfczHFXNH8aRZjlNsX1xgEz6ksuBy0ey8sB9QSyhFep90G X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3cfefde5-2dad-47c5-0a0c-08dca0ea049d X-MS-Exchange-CrossTenant-AuthSource: SEZPR06MB5624.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jul 2024 14:10:14.1583 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QGL90Q66LYoNFkc0DM+n5/187aGLq51wvJsu2b4Za5/kfeFoyTt8x30U4YS6JqGlJ9jkH2xpfpwfxgExyoxFUw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEZPR06MB7077 X-Mailman-Approved-At: Wed, 10 Jul 2024 20:10:21 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" 1.Effects and reasons for lack of support: Currently, memory allocated by dmabuf cannot be read from files using direct_io. With the increasing use of AI models in mobile applications, there is a growing need to load large model files occupying up to 3-4GB into mobile memory. Presently, the only way to read is through buffer_io, which limits performance. In low memory scenarios on 12GB RAM smartphones, buffer_io requires additional memory, leading to a 3-4 times degradation in read performance with significant fluctuations. The reason for the lack of support for direct_io reading is that the current system establishes mappings for memory allocated by dmabuf using remap_pfn_range, which includes the VM_PFN_MAP flag. When attempting direct_io reads, the get_user_page process intercepts the VM_PFN_MAP flag, preventing the page from being returned and resulting in read failures. 2.Proposed solution: (1) Establish mmap mappings for memory allocated by dmabuf using the vm_insert_page method to support direct_io read and write. 3.Advantages and benefits: (1) Faster and more stable reading speed. (2) Reduced pagecache memory usage. (3) Reduction in CPU data copying and unnecessary power consumption. 4.In a clean and stressapptest(a 16GB memory phone consumed 4GB of memory). A comparison of the time taken to read a 3.2GB large AI model file using buffer_io and direct_io. Read 3.21G AI large model file on mobilephone Memstress Rounds DIO-Time/ms BIO-Time/ms 01 1432 2034 Clean 02 1406 2225 03 1476 2097 average 1438 2118 Memstress Rounds DIO-Time/ms BIO-Time/ms 01 1585 4821 Eat 4GB 02 1560 4957 03 1519 4936 average 1554 4905 Signed-off-by: Lei Liu --- drivers/dma-buf/heaps/system_heap.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index 9076d47ed2ef..87547791f9e1 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -203,8 +203,7 @@ static int system_heap_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma) for_each_sgtable_page(table, &piter, vma->vm_pgoff) { struct page *page = sg_page_iter_page(&piter); - ret = remap_pfn_range(vma, addr, page_to_pfn(page), PAGE_SIZE, - vma->vm_page_prot); + ret = vm_insert_page(vma, addr, page); if (ret) return ret; addr += PAGE_SIZE; From patchwork Wed Jul 10 14:09:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lei Liu X-Patchwork-Id: 13729632 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7F648C3DA42 for ; Wed, 10 Jul 2024 20:10:30 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F0E1C10E820; Wed, 10 Jul 2024 20:10:27 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=vivo.com header.i=@vivo.com header.b="OI0LJb9l"; dkim-atps=neutral Received: from APC01-PSA-obe.outbound.protection.outlook.com (mail-psaapc01on2069.outbound.protection.outlook.com [40.107.255.69]) by gabe.freedesktop.org (Postfix) with ESMTPS id EA54410E7E7 for ; Wed, 10 Jul 2024 14:10:30 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UsjEARze45uaZglGPZ62dXp6H0KFYTxveYHIjdqpba9E2w0J44qPe1mCloOoy5NnNdWvBbHLZ8dpQYGQLiTDdIK6IcCaQEZK9x1HC9yICM/MTMW5kfOS2aLd9yO1ExB3SwkKObdS2A2SkgJaO9TsprS8PTq2SeLSENK2VB2edml2AJbeZS4+PYXtweRwkVKFGfz5uD2edqZ8qfmicP3yVshfKelKuNw9Wgqm8Pe/ClFu0qYkSvaYQ9E6iQQUvRn0LioYALZZs8ivZPfD0MKX/0sMzcdFHyrGdR2nAJl78xUPs58KLmln/5/PwyhBciMYr4LeBNE2JJclOJ/k3aeYJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=H0NvuY57QNH1vlcV06mPoVs4bwYYR229k9KLABjB1jo=; b=E8tB4W1EK5saeUfBYhr6nMTx310WcwccEOE0qTtarOkloja7v9slmNOOoU6fQYuscy+SKSo4akMU3FdWwFiYK7f5ehtgtycUoXLPXSTbrStFgesxvFOIoNlMtcBgz/BI3bmOLgYz7eJe/mTRM5JwuQ7EXTIKAswkF7hTfwaOXkhfDAAXjP4TSMfQLjzEmdYT3mHYMYFIi5oqtIRWoY0vGPjLu6HCUdcoFOvDZQ162yQfLdoBkXXzd/4Mblhd9qOSMGmQzQrXIKK1WqR/NZHBpo5XgHiE4mel5oP9PO+4WlqOhK968F8RI+OiRBLHsjprhxa0tPLFX4RL0jmC0tRL3g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H0NvuY57QNH1vlcV06mPoVs4bwYYR229k9KLABjB1jo=; b=OI0LJb9lTJiGzXx3GkPDsLW35Zaa+Iwn2oQljB8fk4jk1zXNvKEMg9edxPGrANEscAjpyLR+G3LBHkouhB1XuwKCQOHsjvIC3eEh4oDOzwmJIkiLRpwA92NZJWK7LLwoNJNVgUtQemqayOcNH7ZQ4guS/BsaL40cHHjz22ZmRp2H/E1K/YgeuBAO0Ao58B9V130Bqa32pUpPZ1tsJ4mWo1m+iTD2jkkFkiFFULNIiwhhhCS41QnoTGCg5ZUYIEqNdfROgUQ1auJZRZYSLYlCALb2fQFhD0kDAkgiO0jA/mG9NFrt0NMo5mlgmtKKTO9Ig8dqRoTU/CJKXqwvL0gvyA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from SEZPR06MB5624.apcprd06.prod.outlook.com (2603:1096:101:c8::14) by SEZPR06MB5479.apcprd06.prod.outlook.com (2603:1096:101:a2::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.35; Wed, 10 Jul 2024 14:10:24 +0000 Received: from SEZPR06MB5624.apcprd06.prod.outlook.com ([fe80::e837:10e3:818e:bdfd]) by SEZPR06MB5624.apcprd06.prod.outlook.com ([fe80::e837:10e3:818e:bdfd%5]) with mapi id 15.20.7741.033; Wed, 10 Jul 2024 14:10:24 +0000 From: Lei Liu To: Sumit Semwal , Benjamin Gaignard , Brian Starkey , John Stultz , "T.J. Mercier" , =?utf-8?q?Christian_K=C3=B6nig?= , Andrew Morton , David Hildenbrand , Matthew Wilcox , Muhammad Usama Anjum , Andrei Vagin , Ryan Roberts , Peter Xu , Kefeng Wang , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: opensource.kernel@vivo.com, Lei Liu Subject: [PATCH 2/2] mm: dmabuf_direct_io: Fix memory statistics error for dmabuf allocated memory with direct_io support Date: Wed, 10 Jul 2024 22:09:44 +0800 Message-Id: <20240710140948.25870-3-liulei.rjpt@vivo.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240710140948.25870-1-liulei.rjpt@vivo.com> References: <20240710140948.25870-1-liulei.rjpt@vivo.com> X-ClientProxiedBy: SI2PR02CA0031.apcprd02.prod.outlook.com (2603:1096:4:195::16) To SEZPR06MB5624.apcprd06.prod.outlook.com (2603:1096:101:c8::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SEZPR06MB5624:EE_|SEZPR06MB5479:EE_ X-MS-Office365-Filtering-Correlation-Id: 9def1156-fa98-4d64-daef-08dca0ea0ad5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|366016|1800799024|7416014|52116014|376014|38350700014|921020; X-Microsoft-Antispam-Message-Info: NYDex2UkDosOX/q4TRhyEtvXtzYkCcCCKvA0chv3N3S6pmpJuTgI6uiuvuoTh6dQ0Pu/xYF96s0NZ++5f6yrvRET661N5JcitNSbtiQ869DXR0zU8Ry9e9BXZmeBeN4+8g+8HGZuyp/T9GviuxHdcTjzP1vCTt4p+GhmFQhl59D14PteKj2h28/SVwqF3CeVQsuOe1lTLtc0VYa5cT/uJMAJt5QX/C6sG7eqS1bTUhmwoa8NOwsoX0H/fpAm36zUyAAyY6SFbUa6KHnQgAzqgmTyP07oUMyxQDne+leZou6Iz2i9XF3/PZIJwOEPABgWxOJU13xg++pTAfnnS93o/eUe1bHnVtUcxIkOr7D26VUFX7R18gCnOd1U3uXB+7bebL/23kr1jQx4WZ4l6HhoKrFO3TJM/jmVr8d5kw+/NNtfsfLFbwSmF+90spoQnlIWNxgehGEwu2ZejaLFc4TrKj5iYwg6e0JhpcgwNlsEhCg18t/6ov4vawm+bBcQdAcJo+n2OJSVEn0N/SPiVm6fz1KsJSLJLMBr/5LCFWey7NWU0dWO3p01kcsYdCFPCYXppbaE+8LWhWJfn/6J8NTFgP8abL56CBRgZYW/Iiq3iyQXYUXD8mSSObTifH5aqqBLtYsIAHXvW+Q9w4Te19bce3IHVgQ/cscwDlC9QhOhuhR0kMuvI9PtUtO7oRjxzfgd6z9WwmT0iwMV4HmBi30aiQEmeYeFeYKBn8OaiV+7DTjTjXz07Uz84PfPv7SrggU2T9Auq/Hjcp3+fRb/ZBXy8bUcHHStFRpenYTJvlizCwwlR3qPUDYOorDcxFhSktOjA5oJ+UB/tpC0wGBahZeic08Sx6pPrqWWHGN+iDSgDolqnwQXGi5S46rsDBSP17gK2b5gpZNkWP58N1QRj3zZNGdx8VoPUa2UAPb96eVeVKKz1RWzjaj5QbiOHvPi49nryEbDDBcFeyg3DooxoqV94jeLqGtDrbW4jSPPHHG1i6e8IQRb0sz9W++iimZioSfcuNoe5hw1YUeRN0NATgPm2ZzkPZD+joMyQxiURde8eO3qHAP+9y5seF+T7FtbOv73av1QIAO8A/QMszYWj2cQvDTQP8c76udVmrJiIPXVoQ/KRUFXabATTgJjfrV/gxxhMIL/WDvgDqwz+Y6lWQMpagWUF2n/HeL1yox+m1Mzg5Zm9okjkTS0JdqyuYzhcLly2WuLkNWUBM5dd0B+NyXy2OEoijgBKugwJGp/RS0CRglop1GQ3bzRl1ai+VCOwDg2e6haNsX8ogZX1L2xdmtqN+bGNUBW5jvn3RsYQ48CXDOU672VeNDNUhbtdZVCVPTX/7otx3Byo2XGbVy772HpheN553ralQ2Y40yCPH3PLp25tmhP6zlA2D0O75xU++JTp1L3Hill5EKEYLN5pZ+dEvqmcbo1yQ8NTHih4jwLGrI= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SEZPR06MB5624.apcprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(366016)(1800799024)(7416014)(52116014)(376014)(38350700014)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: zybxAaViPyR7DjW0mIpdI2J1a3L2IPRhANjnVWKF3jPFB7ZlSoWfxUri36f1S1O0F6jJsibvYz0dDp6bmUZCV7iNtj9aIj1xNvHPmMYS+0jCuDJG2JoI1wZ9hYsuZEdoLXFlr8Gk3PJMj27xsS5sm5fAtDybdsaZNGdc/xlHLM9veJTbdjij8JuLsnTV43VcU7J078Y6dDaSLNdeJ0eZuoCg7YbYulWY91gXCRb3bFK1UIjnHXotnjOj7p2zxLPgvP5jhi4QXJt59Z2JIcZTZu+jYTlwdwTnATp7wcMNiJRwhJAfofhIxBt1TihYeiCym/p+Q9UUr0hOuUHFJOU6ionYhKUi1SjTPG2cjwBBA5lkHrWSLyPk5vGOY6y5yZfF+CqvRVfLRONifndttCdHDz5qkdhC5WmrtSOmGE6I4tIEF5c3OIMe6dsai21KMIXt9Ccs77fjwlfcYvivpBAtecosNclmXIvEtxVgiyk/GYbCV8ZHl1Gu9oR/tVXXibBOQAGC0mv4wVCoJkKDxVpAN+FOogXbYl4TBgXgCiTVqdEGwHNYyDzQMEi1m4vzxKwwiUhm6Z1exy10/ndVISaaIOyV2K2mtGTl8p+Y9aWdfwbG4SVDDtnmEdWP7h9N1GOFoQqdexa6tMi21nuqwiqYhIbYzSl9bk/onne4DBJvgBsjAK8G/TIRwXAzB/A71sXGpniKUt/q+BQbQs/A7f8oRwbJJmneWSdVMOaM9lgkjGicLlK5mMnszUgS4pI5JZBvr49niLmzOEjNM0FepmC1Q/vmG0GKJZbw/4ukmWUH3cvZS6KzLe/ZbVcqrONPdgdnofJ6M/YiyJC386IVFSXDIX96VhWPEAOaVbaqlQEdglstNJEQirq8v/ed3WGNgwy3PDDnSYKyGhEhght0o4I78ZsdtAqsPDoBqV4v5QABvwlfEte9BoPqhevaMmklcXJLA7ZxqM+tLNf1Dbi/+NCfQ2tAmxlB0TwGW8rWwjHmdV1o5Km9vA+4AHjkH3/OLjpFIrsfrrceH3OdQ1BNztCC6D0TCQCrIRDv+a76xKRnVcfLiHUj7f49n/aeKIcxNQra5n4RAW+KKgG9BDWSrMMqVIBQb3/SYVjWTJUwazIvq1H+4627KOH5Fg1kLZ2/m1YyqD5Zo+DZ4iaB+Z7mMktaieNDLnXeKWc4ITH1dsKx1bLSW9UkKPgpWHFMzkpWTTxRj+ytL/CATAjT1c/yW0FUipC0owSjC6DAXimAlwe1+AKrt4vJOm29hCcFcjqqF3RQJ/ONfvfEp28z+BM/tsQWWmnqNJtghz/lEkiiqbLeOM1bUPwydreTKeQsI/aze6GCfUJ2ET7qCid5DNXA9X+fLpHUjbwIrdkD6rTFq0OCIzh31IccWswrjuYU3h7WYflYbzDkeiXGhbL6lSu5sXCbOtFdv3xuIwX0ebPpWKTy2HQ4QfHltDXQ53tzmZ6ZkhJj17lFB4vSOeqZ1bIIAm8wFq06UZz8rK8an7nsGrqf11R9IukX9/XOTLvxnYmMBh0BkWjpPJj8Nzo/v4f0IbNaTp2Heo9WD8JKzpitdvsmtgVIrElRsYfX2Oz1BvJEG/JE X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9def1156-fa98-4d64-daef-08dca0ea0ad5 X-MS-Exchange-CrossTenant-AuthSource: SEZPR06MB5624.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jul 2024 14:10:24.6045 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fYWVdod289ZC+lvKOI31ZQWidBybFPFT6YnVSdHgBfpjd0EoNvm3DGt8cYYBosD0UmXP6Fj7NWxfO/Dsw4Zfpg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEZPR06MB5479 X-Mailman-Approved-At: Wed, 10 Jul 2024 20:10:21 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" The method of establishing mmap mapping for memory allocated by dmabuf through vm_insert_page causes changes in the way dmabuf memory is accounted for, primarily in the following three aspects: (1) The memory usage of dmabuf is accounted for in mm->rss. (2) /proc/self/smaps will account for the memory usage of dmabuf. (3) Memory usage of dmabuf after mmap will be counted in Mapped in /proc/meminfo. By adding a VM_DMABUF_DIO_MAP flag, we address the memory accounting issues in the three aspects mentioned above, ensuring that the memory allocated by dmabuf with direct_io support does not undergo changes in its memory accounting method. Signed-off-by: Lei Liu --- drivers/dma-buf/heaps/system_heap.c | 2 ++ fs/proc/task_mmu.c | 8 +++++++- include/linux/mm.h | 1 + mm/memory.c | 15 ++++++++++----- mm/rmap.c | 9 +++++---- 5 files changed, 25 insertions(+), 10 deletions(-) diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index 87547791f9e1..1d6f08b1dc5b 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -200,6 +200,8 @@ static int system_heap_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma) struct sg_page_iter piter; int ret; + vm_flags_set(vma, VM_DMABUF_DIO); + for_each_sgtable_page(table, &piter, vma->vm_pgoff) { struct page *page = sg_page_iter_page(&piter); diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 71e5039d940d..8070fdd4ac7b 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -784,7 +784,13 @@ static void smap_gather_stats(struct vm_area_struct *vma, /* Invalid start */ if (start >= vma->vm_end) return; - + /* + * The memory of DMABUF needs to be mmaped using vm_insert_page in order to + * support direct_io. It will not with VM_PFNMAP flag, but it does have the + * VM_DMABUF_DIO flag memory will be counted in the process's RSS. + */ + if (vma->vm_flags & VM_DMABUF_DIO) + return; if (vma->vm_file && shmem_mapping(vma->vm_file->f_mapping)) { /* * For shared or readonly shmem mappings we know that all diff --git a/include/linux/mm.h b/include/linux/mm.h index eb7c96d24ac0..86d23f1a9717 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -283,6 +283,7 @@ extern unsigned int kobjsize(const void *objp); #define VM_UFFD_MISSING 0 #endif /* CONFIG_MMU */ #define VM_PFNMAP 0x00000400 /* Page-ranges managed without "struct page", just pure PFN */ +#define VM_DMABUF_DIO 0x00000800 /* Memory accounting for dmabuf support direct_io */ #define VM_UFFD_WP 0x00001000 /* wrprotect pages tracking */ #define VM_LOCKED 0x00002000 diff --git a/mm/memory.c b/mm/memory.c index d10e616d7389..8b126ce0f788 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1003,7 +1003,8 @@ copy_present_ptes(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma VM_WARN_ON_FOLIO(PageAnonExclusive(page), folio); } else { folio_dup_file_rmap_ptes(folio, page, nr); - rss[mm_counter_file(folio)] += nr; + if (likely(!(src_vma->vm_flags & VM_DMABUF_DIO))) + rss[mm_counter_file(folio)] += nr; } if (any_writable) pte = pte_mkwrite(pte, src_vma); @@ -1031,7 +1032,8 @@ copy_present_ptes(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma VM_WARN_ON_FOLIO(PageAnonExclusive(page), folio); } else { folio_dup_file_rmap_pte(folio, page); - rss[mm_counter_file(folio)]++; + if (likely(!(src_vma->vm_flags & VM_DMABUF_DIO))) + rss[mm_counter_file(folio)]++; } copy_pte: @@ -1488,7 +1490,8 @@ static __always_inline void zap_present_folio_ptes(struct mmu_gather *tlb, } if (pte_young(ptent) && likely(vma_has_recency(vma))) folio_mark_accessed(folio); - rss[mm_counter(folio)] -= nr; + if (likely(!(vma->vm_flags & VM_DMABUF_DIO))) + rss[mm_counter(folio)] -= nr; } else { /* We don't need up-to-date accessed/dirty bits. */ clear_full_ptes(mm, addr, pte, nr, tlb->fullmm); @@ -1997,7 +2000,8 @@ static int insert_page_into_pte_locked(struct vm_area_struct *vma, pte_t *pte, return -EBUSY; /* Ok, finally just insert the thing.. */ folio_get(folio); - inc_mm_counter(vma->vm_mm, mm_counter_file(folio)); + if (likely(!(vma->vm_flags & VM_DMABUF_DIO))) + inc_mm_counter(vma->vm_mm, mm_counter_file(folio)); folio_add_file_rmap_pte(folio, page, vma); set_pte_at(vma->vm_mm, addr, pte, mk_pte(page, prot)); return 0; @@ -4641,7 +4645,8 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page) if (write) entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); - add_mm_counter(vma->vm_mm, mm_counter_file(folio), HPAGE_PMD_NR); + if (likely(!(vma->vm_flags & VM_DMABUF_DIO))) + add_mm_counter(vma->vm_mm, mm_counter_file(folio), HPAGE_PMD_NR); folio_add_file_rmap_pmd(folio, page, vma); /* diff --git a/mm/rmap.c b/mm/rmap.c index e8fc5ecb59b2..17cab358acc1 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1441,10 +1441,10 @@ static __always_inline void __folio_add_file_rmap(struct folio *folio, VM_WARN_ON_FOLIO(folio_test_anon(folio), folio); nr = __folio_add_rmap(folio, page, nr_pages, level, &nr_pmdmapped); - if (nr_pmdmapped) + if (nr_pmdmapped && !(vma->vm_flags & VM_DMABUF_DIO)) __mod_node_page_state(pgdat, folio_test_swapbacked(folio) ? NR_SHMEM_PMDMAPPED : NR_FILE_PMDMAPPED, nr_pmdmapped); - if (nr) + if (nr && !(vma->vm_flags & VM_DMABUF_DIO)) __lruvec_stat_mod_folio(folio, NR_FILE_MAPPED, nr); /* See comments in folio_add_anon_rmap_*() */ @@ -1545,7 +1545,7 @@ static __always_inline void __folio_remove_rmap(struct folio *folio, /* NR_{FILE/SHMEM}_PMDMAPPED are not maintained per-memcg */ if (folio_test_anon(folio)) __lruvec_stat_mod_folio(folio, NR_ANON_THPS, -nr_pmdmapped); - else + else if (likely(!(vma->vm_flags & VM_DMABUF_DIO))) __mod_node_page_state(pgdat, folio_test_swapbacked(folio) ? NR_SHMEM_PMDMAPPED : NR_FILE_PMDMAPPED, @@ -1553,7 +1553,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio, } if (nr) { idx = folio_test_anon(folio) ? NR_ANON_MAPPED : NR_FILE_MAPPED; - __lruvec_stat_mod_folio(folio, idx, -nr); + if (likely(!(vma->vm_flags & VM_DMABUF_DIO))) + __lruvec_stat_mod_folio(folio, idx, -nr); /* * Queue anon large folio for deferred split if at least one