From patchwork Tue Jan 24 05:42:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13113537 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4043CC38142 for ; Tue, 24 Jan 2023 05:48:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC5F26B0089; Tue, 24 Jan 2023 00:48:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A4F276B008C; Tue, 24 Jan 2023 00:48:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C9E76B0092; Tue, 24 Jan 2023 00:48:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 72F1F6B0089 for ; Tue, 24 Jan 2023 00:48:55 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 578EE1A01D3 for ; Tue, 24 Jan 2023 05:48:55 +0000 (UTC) X-FDA: 80388613830.17.BC8B66C Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2051.outbound.protection.outlook.com [40.107.93.51]) by imf12.hostedemail.com (Postfix) with ESMTP id 879264000C for ; Tue, 24 Jan 2023 05:48:51 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=WoL9BiR2; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf12.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.51 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674539331; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+cwo0tfKFqlSjGojYIjz88S7pf4ONV6bklIA6oByAF4=; b=KDED2dQAegRnDNGEv/ADlGuLrUkCPgMOcO2g+6bOERtp+mmHa8xttVwlULYoid1Eb4zoIE Yj0KcEOSiyn5ZfGH+cLANPxUymZJ4iVVuCyuSGMp93L6vujTzdXP+P5Qvd1A+KlOu2O2vh fbwZqIM/5wKLPAAz1SvZGfjEIDKkLYk= ARC-Authentication-Results: i=2; imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=WoL9BiR2; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf12.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.51 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674539331; a=rsa-sha256; cv=pass; b=cZq+Ai+bJWo+8WeUa8+Wj0hcqojQm/UthUiDEwYHGLHSQs33nvodvk26JkVDN/9d7nzGL+ XKQYfPo+XL/jfv/wC0Yv4SW1F7sfiivK/hEE1Flsh35Oif6geo3UnHzTqT6aApNoR+/PPI T/FVceNQDh49pGmENkXHY/miJ8hY/As= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=g08ONDz8RF9X3Ehc4Y8ZmLl5NTpcsTw1TM/qBe4O2dcMfbXBzRMP+urpb69k9AME17Y+tW7+orDNWQLDqMCE5AjWF1hYiuVPrGBdWA8eC0mRPzZzVnsV8LQC5/aJwYADVqa7Op/f/VuN69I+oXcXXxfq/yLtRDkjNAUHcf3kZJ7CewzatflfRogjW58Jis7jtV6q9Gev2zfd/cM3OBZX3ugy3KcyEw3HTkzXUiuZHdGdlu0UEl4fEuHrYrY7P63XGDigBTSJxQERG2b1k/OBJPA1lwPLctXlvrgwGIE/ukMCz5KAXxBAdpnRC+2UQk4nZ0V6+Y0kvIBwFjdvHOxaug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+cwo0tfKFqlSjGojYIjz88S7pf4ONV6bklIA6oByAF4=; b=lsB35nKfttiyjjrkW6zSuBBAWPHzIX+qclbw6bgthp4n3j57c/QaBOo/UrObW3d89FGwSRNXy8NjC397VAmwQeICkHomFx52YZCxgmfKmg/akd8F2lJSDvCU29i6tXVhNvHJrr9t7bZ3MPfICRebhuMDNIL9jJNpSVIVVU1dqn0y2OcYzCXSlq85k5pWXHLvivEsKzz/O7bgqVzRcRylFuH7GIzfhc7AHSVU7gOkGYlUPYG2BX1qiSqeEwHRacFQ0AamUUt7OSu34A0ZCHK3PN4pmav+VBrDCmLVLLrX6+Ay8yy8jKyxDDURMJ0OEcwgL9PJuESylaBg4BXkaCx9Xg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+cwo0tfKFqlSjGojYIjz88S7pf4ONV6bklIA6oByAF4=; b=WoL9BiR2yLNy912yNr/5ywnnEnSpU2Z8QQ09z74/o96XY8GFpmvaleX/P6bu79Lz7wVe/onJp9mjlz/S51BLQhrttZ1wSTdMoUU6m6H8QsDbEEmCNKbbCdLoi6a6LGNgOX66QCRaZhXBDHv5oVPZZy2sp1b29um1b2kLvETMNLiYZf8GxEi8kfMFsbJCa4d18m0NTrEije59aOH3TSLCQtL1RnrOFau0EYJ2QlqS8gxnCLmlW0Mwlm1v9Zupyi1+0lmNi/4Q/eBjWZs3hhuBZfG5caU6vYTDTIS4e8pAY3yn3k3U6ij+9hT7K3ggAp2Pnr5L33ahHmBgdA2sFQqmVQ== Received: from BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) by PH7PR12MB7793.namprd12.prod.outlook.com (2603:10b6:510:270::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 05:48:49 +0000 Received: from BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::465a:6564:6198:2f4e]) by BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::465a:6564:6198:2f4e%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 05:48:49 +0000 From: Alistair Popple To: linux-mm@kvack.org, cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org, jgg@nvidia.com, jhubbard@nvidia.com, tjmercier@google.com, hannes@cmpxchg.org, surenb@google.com, mkoutny@suse.com, daniel@ffwll.ch, Alistair Popple Subject: [RFC PATCH 18/19] mm/mmap: Charge locked memory to pins cgroup Date: Tue, 24 Jan 2023 16:42:47 +1100 Message-Id: <3883f8fefba4fa039c8bb5ea1882aaf4585dbc07.1674538665.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: References: X-ClientProxiedBy: SY6PR01CA0086.ausprd01.prod.outlook.com (2603:10c6:10:110::19) To BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR12MB3176:EE_|PH7PR12MB7793:EE_ X-MS-Office365-Filtering-Correlation-Id: df3d31c9-199d-47a4-8ac5-08dafdceaac1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sVqAUNH3AQV7YF8agzDLryko40gLhCW9ldfty/HINCSttBJYg30cPg6g1cKMeJf9jQgavmvD7iUmOOVAFjsPg4bADZzGQmEf8SeBy+DiRi1+A7t2Vt/4aM5qSUSkEMAaD8AHxP9DzBfWa8goZZSO5NdNPvb1WigibWDSo7BW9hXbeHuOCFKcsaJ3oe64Lo5EuDP/TlFlAZG3H6ZQNID4I7VEjHxjfcCZLtUPwLj4X8CpuUXvomL09AghoV8s90bnOAAzzjFLiQos913e65Sb4gWxeypcbLclkfjviuewpisM8Tb1fdc3YgAApy+VjdvhL+W8mqjVHZkTWr7K+Wyfh2INzPkUSyichiYviFMqhI5c37jy4gYdPGihcos0WiRMRAp/3qm9Awwx1NqBa4meifcsvRgIvdtRpnKzGxWFTNQ1D25LlK2dkVVY3NOgozXCMOL6JUxvRiUbsW5oVWExvWAjHzd8baqnKHoZf+nJCiSgn9Jf2ORKw8R2466IYUZtrgbGLUc3sVKDuQwYYwm8blRN7IWWQ3GlZJATcj3EYoZABOUqolE44rjRuFb3z5809cPQbEFSevdi7n8TAMaHyIDH/FwlF6TUGnfm0tFI6tNwXJTOK97CuWLcSiq4N1P/GLsAHbSBvM+3JjPQypRQUg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB3176.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(346002)(136003)(396003)(366004)(376002)(39860400002)(451199015)(36756003)(316002)(4326008)(66556008)(66476007)(8676002)(86362001)(66946007)(186003)(26005)(6512007)(6506007)(107886003)(83380400001)(6486002)(478600001)(2616005)(5660300002)(8936002)(41300700001)(2906002)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 2xoZ+UxT1JWl6ziSaM2I9RqoUmuQbusQjqYIZo2umK8rHZCZnYP8sNTObxND5/PF2zW6gut2lXEpua5w4E9DItYZvXoK8iiar+382IPUBOLEA/0DifwpllrkkF/6aXN+WeyJjOoSj5U/j7sQf3BspFwFNzWYoci0CMsiMDGzfMe+hbjDDu9M3mdM09KY8l6kFqhzhC7K6iCohR/MXkC/x0nAXTAaDCQF6dROB2Isc8WrWj8A66Ul+0TF5409COjv7CskMIIbpkTSMb686MrMTTfOc+yqSi+VQi0Gftmv0Quhz54gK89BvEOE2INpYoXmk7WSb9F93f4soZyqIgw1dYYzPmaN1hRYrl4e8AJXeryzVGpvl4FhR0DIL4rBjhM40zMk2kQ/VrLGZBbgCS54D8Slym1d34bmQHuEhvpvNwfI7TKKKqOs5j1DHOIh8NA6c409Q5ibS/lYsj0aHKVixnwZjcVKbBezfWQpvqkqqWr0Um8zohwZtf4VFmm4gEfCjYTFmPoTQslzD4+8/SBXgEDegnsU3bGoiAHq2fcT2FSavrY2n/ZA/NvcEVfpETZ/YHRWLpLEjP/wcuDruNrM1mgn8fDk8tCLgsqj5jf5JeqGzud9U+uXgK6PbEE0vMs/8mXzO7qMIeSQOAgPe0jFgW5gzh1hRcQNN94bSNpOeqw/aU/SkPl+5Dn5DNu4jE6gbomvemwv7P2BAmayowX1EtXna4R/j0CrmIDzDMA09XQV/a4HXj7iM8oKHmunlpeWnZpZCobeWxNE3vLy6pcVG6ea8xNCEbEN5nUHl5vdmDgSDVNah8rE+bLM+cgDkbLFOcCBut9OyhjfgWQKltR7KqGDDrWKnzN97b1Er033SjvE7MtS28npx5u3w2LNSyJCdl3Zcyb/OHbZQBK3rKZ7sWorx3uH5JyN/Ar77GuVI/lLeMGtz0er7bkXiesVRaM2y7I+vvcW7Fy4V67Zc2tPNjrkUAa/CY8sRDdcdmJBpqbNYMmsj1yw0Ra2Su9Op/DVD1OTCO8e+buRr0c5zer1LJ/nocozG34zFSfpDOCDbC62lQpuJkjuWC8WEXSpsFBi/Dvi0jURyTc/G8iQePOJb3nzsEAXK+IgqhloR8akzgNsqdpL4Nq+09BtBD023OrmKZzS2Oo0fNdUi+BlrhL4Hw8lzi0nRZcDDEcXRQbI3+hMmTnaOvYaqJYAzSy1MOc1y4RBZ5+Oz9L9gg4xBOgZ4qAAloyCYuGO3AHcIbuAVQHWiD4GSH56mM8PbtUp9xvL1w+frFDWvfqC/udZkld4J9gRAGRhFIo+ooZBJDaFSuGPzmXQ6La41lkOl8uZIAYW5/p54C0mVYuyU+zOepxdIEBeq1kxaWxULkSh/J1/i6KW/5yLiuH4FI3rJFQmOTnHNG93cD0j/EgdmcsaJdTqJNz8SGbKzsQzYGVOMO9F1XPW34qwb7LclCAHHe1fNKM3zcQcGhJf0qvKjNHhoX5YlSxzFJ1RnlIbqxYCKWWrb3FZkV8HgDnrC33j77I5EkxYbRn1++X2YTV6DTC+Nmw1+FAETdBu0Knh+MZ/qFOtV3BKwufqp17BkAZvCic71s1C X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: df3d31c9-199d-47a4-8ac5-08dafdceaac1 X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 05:48:49.7966 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uIQPRqr7jcxV645b/6A2tvrxcxKneDs/uGpW3WazBaTYZOTgAc6dwKVOfj2nSafkEBSEsG7MOoXFdAM8kwdMzw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7793 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 879264000C X-Stat-Signature: e43jz91pnskgecpu7fkfmnsif3oio94m X-Rspam-User: X-HE-Tag: 1674539331-833007 X-HE-Meta: U2FsdGVkX18I8ZwQ6Joe24oeAXGxIP671GZJKHjxD+ZoX9pt8SU5PD/uWdR64yzOG/9JzH1RP4yImGDrPK6kMNoWAwOr7QaRvBj0ScYAOmiOJFbly2Vn3HYFTGNEmphl2KGSRtKnrBDAx5awn2lztALhTlrLVqBef+2FhlL6Miwc2p5qMaly315EcTyIkQsBJdtaFo3u5eEEqrLNLaFI3j9lSjw+7fPpwxe6JejPNMQEWFiRcp4iTpj8o9Bw+Ilm2HLOfeaDvAWSQPjK5UKCkm3aW9NqjE6lJlZrvj4VT76ltOTkGpPeo4f+svgZ4hxQKMN1d+xPYix3PzeeDoJ8r+x6Y14sBEdOpke9o22JYA1MsWIC1Uzs2yDOEgBKUW1G4x01wvW8tzjjOhi0UrkvJCn9JGG20iDgkj7+ypZj8nc5ug3aRQxIG7zut0nqpz8uXWeQ5GQR5bgl6XEuZcUgELfpJmSINQbB7WRiFnHYsPiJweyjnuZywsHteRggFgOgCySKpZpCfHLP1v7UxQn3DeMmyJ04Wgy22hRL2MD4S4+JtLaVJQnKohvu4rtCrmeNEDLd0yvNmNctyyc969Wy9g3N/I2H8ZvE5RG7I1mJ2zKbdr5hF4OZmSNnGxZuKBNbKK7/mHC7fGNSsYuYpxFUromvG45as689xeA0oqFmJsOIGEL4IoQeML4yzbPPaoqVrcvgXV8U8O73Vh14c4SAk8UTmLpkw9izZySzDxkkDReD2yhTHkoV074kx+LmmuDuAyMjJyu8JYsZuqfiV72Nrr609T4UDF3HJAjQD2gSvK/X9TJ4iNd+CxzVepvrh0b+lqjOfA44byK2r2DnVmfkmiZA1IDP19TE0LtcpEMTo0SXYaRIQ2B0VPPBUClu7dh4qKuLaSs9mfJns3+Pt7UyTZfSLupbi4qc6woQf+iq+dbgKpK9uv4Jt9Gx/I6NMTEr1J9aoFUCWnyU7Q2wLQz RsWVm3T3 9h+zhl4zeHhRzqNzwEuh6gHb5YhhXU2vXyfsm5UfaDNfccqjK+VHMYCN6cEdLUnzrVfFXDnvC8q+hWIjqJ0lbEJAerhrsnW2MMGHDAsBx/JcRPZ8meob3V5aMbjIx13QOKYkyazkJL6ZbWCHAjUlOmGTw0qCq+UVPA1MQd3J1NJ7DzHCDB8VS1z51tBZGl6nOidgy4eWM6dmaE74o7DbyN3hHfnfoWyxDY77kw3iECjezx7RHHP0wfj/LuRb9aDsiXsLY+3HnSRmlx8RoF492oVp5znAsYY3kcGimka3FF6F/nia4MMxQg4/IcwA5POGPo4WzJLkwPxYuaGK+pj63/CM19ukO+TiU+SKgTKNCylB0n3s= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: account_locked_vm() is used to account memory to mm->locked_vm. This adds accounting to the pins cgorup as it behaves similarly and should be accounted against the same global limit if set. This means memory must now be unaccounted for correctly, as the cgroup typically outlives both the mm and the task. It is assumed that callers of account_locked_vm() only do accounting against the current task. Callers that need to do accounting against remote tasks should use account_pinned_vm() and associated struct vm_account to hold references to the cgroup. Signed-off-by: Alistair Popple Cc: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org --- mm/util.c | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/mm/util.c b/mm/util.c index 78b060d..d6159e3 100644 --- a/mm/util.c +++ b/mm/util.c @@ -551,15 +551,21 @@ int __account_locked_vm(struct mm_struct *mm, unsigned long pages, struct task_struct *task, bool bypass_rlim) { unsigned long locked_vm, limit; + struct pins_cgroup *pins_cg = get_pins_cg(task); int ret = 0; mmap_assert_write_locked(mm); + if (pins_cg && pins_try_charge(pins_cg, pages)) + return -ENOMEM; + locked_vm = mm->locked_vm; if (!bypass_rlim) { limit = task_rlimit(task, RLIMIT_MEMLOCK) >> PAGE_SHIFT; - if (locked_vm + pages > limit) + if (locked_vm + pages > limit) { + pins_uncharge(pins_cg, pages); ret = -ENOMEM; + } } if (!ret) @@ -569,6 +575,12 @@ int __account_locked_vm(struct mm_struct *mm, unsigned long pages, (void *)_RET_IP_, pages << PAGE_SHIFT, locked_vm << PAGE_SHIFT, task_rlimit(task, RLIMIT_MEMLOCK), ret ? " - exceeded" : ""); + pr_debug("%s: [%d] caller %ps %lu %lu/%lu%s\n", __func__, task->pid, + (void *)_RET_IP_, pages << PAGE_SHIFT, locked_vm << PAGE_SHIFT, + task_rlimit(task, RLIMIT_MEMLOCK), ret ? " - exceeded" : ""); + + if (pins_cg) + put_pins_cg(pins_cg); return ret; } EXPORT_SYMBOL_GPL(__account_locked_vm); @@ -584,8 +596,18 @@ void __unaccount_locked_vm(struct mm_struct *mm, unsigned long pages) { unsigned long locked_vm = mm->locked_vm; + /* + * TODO: Convert book3s vio to use pinned vm to ensure + * unaccounting happens to the correct cgroup. + */ + struct pins_cgroup *pins_cg = get_pins_cg(current); + mmap_assert_write_locked(mm); WARN_ON_ONCE(pages > locked_vm); + if (pins_cg) { + pins_uncharge(pins_cg, pages); + put_pins_cg(pins_cg); + } mm->locked_vm = locked_vm - pages; } EXPORT_SYMBOL_GPL(__unaccount_locked_vm);