From patchwork Tue Jan 24 05:42:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13113497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07198C38142 for ; Tue, 24 Jan 2023 05:48:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A2B1B6B008A; Tue, 24 Jan 2023 00:48:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9DA056B0089; Tue, 24 Jan 2023 00:48:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 87BC76B008A; Tue, 24 Jan 2023 00:48:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 798626B0085 for ; Tue, 24 Jan 2023 00:48:31 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5D73AAAA1D for ; Tue, 24 Jan 2023 05:48:31 +0000 (UTC) X-FDA: 80388612822.04.C4B715B Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2055.outbound.protection.outlook.com [40.107.93.55]) by imf27.hostedemail.com (Postfix) with ESMTP id 9150540009 for ; Tue, 24 Jan 2023 05:48:28 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=rrFIO2dS; spf=pass (imf27.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.55 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674539308; a=rsa-sha256; cv=pass; b=Z6JxvgH8oD5V9hhdpCn1xiYx7CNNo6QuTsuK8FpwuXelBONmGzmX3BQJZevBQmjowjK7Un Ucf2ukdrPzgCArnC7wiOzoHvgYakm3XF4BRqeGwd4MPkdGq9KtMEJWnKHNAP8QI1q04k2Z PUxyrhYpr/kp6xYalrQYUddlnRaAmQE= ARC-Authentication-Results: i=2; imf27.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=rrFIO2dS; spf=pass (imf27.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.55 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674539308; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KKauSACOmjp+bD2erTq7Z7sPoUZPfQj7diDqarD8XaI=; b=te4NJCmrnWOpHox0HBQX7PHF14Ecm6VGxQd/aHp7Rb65fjJ/+ilWw+Wcobj1+I6O7hwd5Z ZccJTnjBPxDYJgiORvdweB8IxCEBCBc8/ajeeVUGkbWgv0QQOspaI2jveqzsssrHhRPSq/ jCSDwu9bxCgyKnLJKffA85rs7c57MZI= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Djnvy2h5fFFFHewpifc5gQWX/zWH/4WryCmRgfW+fGx/AIxqCXqo4ka+LlxfngFsT9zWVut1PWvwmShbw1A3CydL2V4sXJVeww4mtorCL2xtWXlNDzKluXe0w8zoDW6AjrlWshKitqsiOFt0+EOyyTg3yEEegANEi3GI+h/KVFXsK4xvzt+CHOSpXovVdOEJoesLsr8GjmUzXjDcSon4oitk0jbx9O8BfdpIQ1e3oMk3pdmns5W1Xi4aBPSD+meneIz4s6/EVQJkL9drVbn/mpB0LMqUAviZ8KgWjK+9T2lFLIxNBLyy18oWXsMcxThTwDxpJV6tdX0ZLtTU9BVPFg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KKauSACOmjp+bD2erTq7Z7sPoUZPfQj7diDqarD8XaI=; b=Y/RLQkN34TLLAkbOm4IEEmUYwogokuoJkJ448HsV6axmSseUK6ePM3giUVAnE7ipd/q13BGY975Szw8vvyNWcaWsEaPcbdgmbeWZN7byJ0CD7ro8tat1Ahhc+t1BckAIBKYk5R2J83FHauyLpH4enJxtM+HO04oun9FTm+E/A/tcbtWc4WWx2yQZ/wrekDbdZtY5QXwtkizpU7zQ96XFGpoCed2GjQmXiSVLuKMOhzUmCETf9hWcjh75OaGdtJ6HLh7L3m4GNfjioiFfOmhZtUvUzRQZguB4k3XysXvq5E1HDQKWJGx7nGKEPmH9e+BezBvzOiAQETSM7erTByy7Ag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KKauSACOmjp+bD2erTq7Z7sPoUZPfQj7diDqarD8XaI=; b=rrFIO2dS0S07TXuOCHnBa/xKtMzAh0p9Y/97yh58OAMl8kppEmSbpi+ehr6vt68Cru4vpnrl9UDqpLT/l/B7n/cMWZoLiEQlINqKV1v36BmetIPoK1TMi2aixQAKBeZlfAHbluielXfN/1hAIMbs9dcV7ME7Ixx10FKgk0x1ldVsYj18RJOtvtj+Uk1xQLUfRQawG6X0GV/+dM1FDRgG8yQoedt2IpadnqrskuIbWK0xrM//4ihzcBmrDQwCJD2BZ4cMkeDoP9VTrE+Z73WXW34ComVk+y9GcmOKp87FHhcTOGKQbl/XO+NuSqAJoiu7OVjKIsAONGbrX5BnuU7uXQ== Received: from BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) by PH7PR12MB7793.namprd12.prod.outlook.com (2603:10b6:510:270::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 05:48:26 +0000 Received: from BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::465a:6564:6198:2f4e]) by BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::465a:6564:6198:2f4e%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 05:48:26 +0000 From: Alistair Popple To: linux-mm@kvack.org, cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org, jgg@nvidia.com, jhubbard@nvidia.com, tjmercier@google.com, hannes@cmpxchg.org, surenb@google.com, mkoutny@suse.com, daniel@ffwll.ch, Alistair Popple Subject: [RFC PATCH 16/19] mm/util: Refactor account_locked_vm Date: Tue, 24 Jan 2023 16:42:45 +1100 Message-Id: X-Mailer: git-send-email 2.39.0 In-Reply-To: References: X-ClientProxiedBy: SY2PR01CA0006.ausprd01.prod.outlook.com (2603:10c6:1:14::18) To BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR12MB3176:EE_|PH7PR12MB7793:EE_ X-MS-Office365-Filtering-Correlation-Id: eb1ed208-4dce-44b8-cc7a-08dafdce9ca5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MUrLMQP6VYi2FshBWZPFDHh5/5FgB1z7nhp2c4Zk/uXkKXeIB29602GWxBFidb4r1xNk9BYI+0WWBhejAXYMsycA050IGUMKPyIL8V2TgCO2Oc07nQ8uMQsUrAVkziW7QA66Tplh3DvHRmp99glIZeFO8CaUgHUBR95BpKoFJQq5Om2ncQNn4cQaULkii2S5MVrvOrMviwaJ9UCD9qS+8tQP5B2KQ/6DV1Jy39/hx/zVRsTWAf8mq2/F8cFYf+X/LPCal2DYTtj0tJMFrO5h2omEPSXMgGoaCu5U6UGdUFCMuaXCYOcSf6lqxyQUTfC4ftz/7x33kvOdI0EleJfPKRqOW+GpbEgEIGcisIR+i/v1LuAoXj08sqS8gZyEy4+2bTny6Qovj8bZpREcveHQMjmwHKU6oN7aWtmy3cjq6VFYcbuTQOUb77XYCIgPNxgeZoljP0vFYpXICpbdYmfqQ8jnay6h+A1wsCDfkdUUozTrzT5QgoNvRPTSEPmYkBIae2dvdhxFZu87dxHnxB56U1+h//f46OHME37+RIqIwmxFtLg0ZYVfpvKybCgOuxD9um/u8LtrnsOO822z+Pgb7ranOqAKrZHH3GmYVwmtGfZWQWpnO+0EvOmlhmY0y7cpTgfgUFy1skHqhVANf5dRFw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB3176.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(346002)(136003)(396003)(366004)(376002)(39860400002)(451199015)(36756003)(316002)(4326008)(66556008)(66476007)(8676002)(86362001)(66946007)(186003)(26005)(6512007)(6506007)(6666004)(107886003)(83380400001)(6486002)(478600001)(2616005)(5660300002)(8936002)(41300700001)(2906002)(15650500001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ai++sjTIWs1atPu1MeV9PIHS2Xj47DNdNs+opiD2JENbcFfLRnGaIaBNWalrNYbNW5qlwSod5fDph7bITQducoMS8MIaJPmfQTKDteepo71LzWTkwJzNSyisxIS8+g38+NuCL4swIAUA5qGRcpxs3e0uE78OwA95pXOFNHp8h7RkcItbmVtBbSz1dstUzfzfyU9LPEEE7rk8iwMZm4R1T8YanCOZqs6lX0uMv5bxG5AUoRI3+qe/kXaubVeIbJDzaqU+K8kqrFVC2MOPQPJGVX+2DFPZUTgERl+dCwEEZC2c7NZPCipqnq9CcWc9LnV+4pV8kSMwk+IgRPy7iVXw0YSiGQRAz3ORFW3VbpanouHno1KZSCtIhP3zgtmYCh5PRKrCADLOl3WGLohBJuMI0bDngAut5gv4zy+Nyd1qG7aSLFqAsmTSyNfDyqU2/Q1meKYiadgzXlzocHj5qC4BFqRyszK9RExhowQDeLUQu5/2YKoba0ea5C0WGQ4uhzty9YXYr2rV8/9gPfVCXz7k1p52BV6lwUglvSnSpJAiT9QiXvfdLcF8yVZ0KnF0ZNyh11HcVbp3KaQs4y2skYp1r0k6HWYtSz9m3+uyk+CeB4QP1EeWLkHmPpm+X0xa9iL/6sCx+r8h/wY9eHZ2h31giHWFwMP7RdIk3onJ8nOLZhyvdMtZHT+Nudj9asqoK786sC1zEX9dwU2F7TJJiN8U0N2kBz7EV7YsocZlmvm/820bsWNg0kSVN7CXUo72L7C2h0YQ9XgorGGw+WLpeCjWQlSN3B7DS4349pFaxckNaTk61JYEjWF1e7gQ9DIUqqzy5YZ5A5oYmCyqOkVVE7ireZe7mkXNJYuZf8dGrUAPAJfDw0RU5optYGr8TgFS0XK8bjaHGR4HnpIVE5SNrMLjdAJMmj4gsWWSdzE3SbWiGuZItR9P7/ZaDCwNq6WCH/dcgvV6lroKdOKJvM8vlpgXGTO0tjWIBmTEqPSJcujMgjQ1KEcdQhnqGMqDadw9omUxLpo+b4MUSyiZeWBfz6yOOx+d8uyZO9JS659rwThMxwH9SV56CpSHiGacAe+C/XXdLEiiS6XvhoLE17oIsxSScBidL5zJRNa6+iBLW9AEG1sWv7wG/P7Geob8jp7KgfCD/V3iprV7Wq2UmiDgCaA0xakX11AcvApojzpmZuLh/LDnIaX8Zm3USeGzUcIdlzN0J7YXuQ2n1VTfzy0DDKaejPhID0LhyVei8cgbUS7dE7D6D1zEJH0QLIqp3nhJLukBs/ZFM/Rtb9W5nmOxve7EZOAwYlsjQFEMZCTm9kEp/sHNXjgCp4HJx1NSmizWJVaPcwEsYR0VaNlDKFpjmjA8LIN3q38fawGjQhzyf4Jg8aWe2NBkQW2/tJH4Kpz5nY4rcGg3FrR6tEcZDfGZpfVY0bRQQ8a2iseMPCCxnCWXwJX3f5jnZ4F8n2U+wH+HfuF5NOYfB7to0deIXnkvr1PRQz45x1uN8rePH5Mex1h/us9di+KmFVyQH7TbKVYW0r+ryPVlmPzOLIhWuMjy5VpzsziJa66kxaczNDMBK3iq/5GKes5xkxeXR8feziT4MFGd X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: eb1ed208-4dce-44b8-cc7a-08dafdce9ca5 X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 05:48:26.4704 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YX/Ay4g83eSbAYhQHqkCH48N0bksO9HCgAgprE2g3sLqJrFhcA94gFmb0QaiG+LL+lswkhmHNq1wwwj3DEtixg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7793 X-Rspam-User: X-Rspamd-Queue-Id: 9150540009 X-Rspamd-Server: rspam01 X-Stat-Signature: 1qmwf4rb4ty8hugonbusueei8fttykkj X-HE-Tag: 1674539308-478180 X-HE-Meta: U2FsdGVkX1/53AcJLtjb1FaN/mRgdI1zmLB3TIvuH5jTy/wMLMXCbVJk7FrXtn8UoLSd7O5P/HhRtxP9AkOjO5duOEvsrxYLPwGKJzZrntBEc0JiWPFyDFnyOW3Gk+R2Bca4/BhNn2AyWWBoAQ8+3tQ/GNcdqNVo25oQJrz+HP+WVsCxNf4Hu6aUlqXhx3Jf+uzz7Z/9X2SjxnzD8sDRaXhxM/eYs8Bd8BKzWLBtXkC4TjaknJYwga/lFUKno1tLalnztueEt1mMgRA5NlnxIQcJMXxwGvsnzvQMz8Z7c+dheOZJ4ZN9zKCy6Wj/vQ8iUmK0f6/OQJeQ+PfzjHn/FZvG+qwQ6KXDeusSUjXYEKAogyilMf05zYtdWR+aMgcR5NMNefWjnC5VdsH5smQ3Rnz93N44VB5cbfBPOmNAzQdXAKaetOA77MftwHvq7gvR0RhZUFzJVzi9Zt1nE5HmKRd1juHtKq6uChhM81MWPvtssXJoX+KubVx75/uGLaaPJu7Vgvwg3Ynv9RNJodngVbFlfGTyHIQVINh/ZkVAU5gJgospz6vjiEZuu9iwv2veV8eoIpeomSQeAuiGzRd5M6aL9g1uU5ze1BvmAO/BuUcnxm3BzdBz+jphqzO6/Wd6RObXVb1vKpFFmc82BY63vSrEUmC7eYyqCX3ZAbrNH2W1OvIQzbgzgdabyMDG3ASRvaJ8UhX8xHN9OEqlDiUJhVqSkEAvDxEG2hcwdzNL+kFpnx88bdGWJgdwvi+0V9KLnCP0VCnkCBIJ+u/apnBfPJHa8kusD3HuRxwG1ZzhlTgX9yyKZ5iSysgIPVp+EAjdYh6UE8uvWCoVq/oLfK8CLhQ+VxL1rjUfVS1Gm3mugQ9j/lSw+CHkpKs18ajjU1DIg8MwZLBaKIRx/eXyEfMOeiol6Bt9zNn1mj/RiTx9roPC+AQcIhDPmCfGXaaazHirSuYzXr6FSBJg6EDUxc8 4O4J4CTy wxJpeUFbWFJnD5cy+gjD6yMQvEJ2Llt7O7bsMb+ZuIg3w6OJCKFzM0oUIHUQdKHjNbBBCj0NaiGcvPZgrvKHv0LV4cy0UVVGILKkmWfseA8kaHyYOiYK/Ch+a/vksn+EMiJVfBCKX609X9JsRrEZzUUvk3VrwLU5r2EJ2RuF+h9iKxgzDXn0xJLMO2XLsuPfg7yudFUGcqJ9Ld+JkcF8tGR4D930VWzT02FvmqjZpYO/awyi3csdFIYa5/ndYri/gPoIYMsi1vlSmqJNhTQKIBJWVaBTCY1p2KluWabzvoM8h+DEjImKsQyHlvJ+Tb/ebHAHqYiRQ7/jHHjEn+bhuBdpzwOeyMzOQwpfYd0yWBn9ypfQhVuCKog4/J56yaUIhyIE+atpfozDzu/JQEUffnMcjpuLjGjX01H+MBcwsIrAXYWX+b3fWJGr1Pg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: account_locked_vm() takes a flag to indicate if pages are being accounted or unaccounted for. A flag is also provided to bypass rlimits. However unaccounting of pages always succeeds and the flag to ignore the limits is ignored. The flags make calling code harder to understand so refactor the accounting and unaccounting paths into separate functions. Signed-off-by: Alistair Popple Cc: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org --- include/linux/mm.h | 5 +-- mm/util.c | 73 +++++++++++++++++++++++++++++++++-------------- 2 files changed, 55 insertions(+), 23 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 8f85716..126b756 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2090,9 +2090,10 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int pin_user_pages_fast(unsigned long start, int nr_pages, unsigned int gup_flags, struct page **pages); -int account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc); -int __account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc, +int account_locked_vm(struct mm_struct *mm, unsigned long pages); +int __account_locked_vm(struct mm_struct *mm, unsigned long pages, struct task_struct *task, bool bypass_rlim); +void __unaccount_locked_vm(struct mm_struct *mm, unsigned long pages); struct kvec; int get_kernel_pages(const struct kvec *iov, int nr_pages, int write, diff --git a/mm/util.c b/mm/util.c index e5fb01a..78b060d 100644 --- a/mm/util.c +++ b/mm/util.c @@ -537,7 +537,6 @@ EXPORT_SYMBOL_GPL(vm_unaccount_pinned); * __account_locked_vm - account locked pages to an mm's locked_vm * @mm: mm to account against * @pages: number of pages to account - * @inc: %true if @pages should be considered positive, %false if not * @task: task used to check RLIMIT_MEMLOCK * @bypass_rlim: %true if checking RLIMIT_MEMLOCK should be skipped * @@ -548,7 +547,7 @@ EXPORT_SYMBOL_GPL(vm_unaccount_pinned); * * 0 on success * * -ENOMEM if RLIMIT_MEMLOCK would be exceeded. */ -int __account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc, +int __account_locked_vm(struct mm_struct *mm, unsigned long pages, struct task_struct *task, bool bypass_rlim) { unsigned long locked_vm, limit; @@ -557,33 +556,44 @@ int __account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc, mmap_assert_write_locked(mm); locked_vm = mm->locked_vm; - if (inc) { - if (!bypass_rlim) { - limit = task_rlimit(task, RLIMIT_MEMLOCK) >> PAGE_SHIFT; - if (locked_vm + pages > limit) - ret = -ENOMEM; - } - if (!ret) - mm->locked_vm = locked_vm + pages; - } else { - WARN_ON_ONCE(pages > locked_vm); - mm->locked_vm = locked_vm - pages; + if (!bypass_rlim) { + limit = task_rlimit(task, RLIMIT_MEMLOCK) >> PAGE_SHIFT; + if (locked_vm + pages > limit) + ret = -ENOMEM; } - pr_debug("%s: [%d] caller %ps %c%lu %lu/%lu%s\n", __func__, task->pid, - (void *)_RET_IP_, (inc) ? '+' : '-', pages << PAGE_SHIFT, - locked_vm << PAGE_SHIFT, task_rlimit(task, RLIMIT_MEMLOCK), - ret ? " - exceeded" : ""); + if (!ret) + mm->locked_vm = locked_vm + pages; + + pr_debug("%s: [%d] caller %ps %lu %lu/%lu%s\n", __func__, task->pid, + (void *)_RET_IP_, pages << PAGE_SHIFT, locked_vm << PAGE_SHIFT, + task_rlimit(task, RLIMIT_MEMLOCK), ret ? " - exceeded" : ""); return ret; } EXPORT_SYMBOL_GPL(__account_locked_vm); /** + * __unaccount_locked_vm - unaccount locked pages to an mm's locked_vm + * @mm: mm to account against + * @pages: number of pages to account + * + * Assumes @mm are valid and that mmap_lock is held as writer. + */ +void __unaccount_locked_vm(struct mm_struct *mm, unsigned long pages) +{ + unsigned long locked_vm = mm->locked_vm; + + mmap_assert_write_locked(mm); + WARN_ON_ONCE(pages > locked_vm); + mm->locked_vm = locked_vm - pages; +} +EXPORT_SYMBOL_GPL(__unaccount_locked_vm); + +/** * account_locked_vm - account locked pages to an mm's locked_vm * @mm: mm to account against, may be NULL * @pages: number of pages to account - * @inc: %true if @pages should be considered positive, %false if not * * Assumes a non-NULL @mm is valid (i.e. at least one reference on it). * @@ -591,7 +601,7 @@ EXPORT_SYMBOL_GPL(__account_locked_vm); * * 0 on success, or if mm is NULL * * -ENOMEM if RLIMIT_MEMLOCK would be exceeded. */ -int account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc) +int account_locked_vm(struct mm_struct *mm, unsigned long pages) { int ret; @@ -599,14 +609,35 @@ int account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc) return 0; mmap_write_lock(mm); - ret = __account_locked_vm(mm, pages, inc, current, - capable(CAP_IPC_LOCK)); + ret = __account_locked_vm(mm, pages, current, capable(CAP_IPC_LOCK)); mmap_write_unlock(mm); return ret; } EXPORT_SYMBOL_GPL(account_locked_vm); +/** + * unaccount_locked_vm - account locked pages to an mm's locked_vm + * @mm: mm to account against, may be NULL + * @pages: number of pages to account + * + * Assumes a non-NULL @mm is valid (i.e. at least one reference on it). + * + * Return: + * * 0 on success, or if mm is NULL + * * -ENOMEM if RLIMIT_MEMLOCK would be exceeded. + */ +void unaccount_locked_vm(struct mm_struct *mm, unsigned long pages) +{ + if (pages == 0 || !mm) + return; + + mmap_write_lock(mm); + __unaccount_locked_vm(mm, pages); + mmap_write_unlock(mm); +} +EXPORT_SYMBOL_GPL(unaccount_locked_vm); + unsigned long vm_mmap_pgoff(struct file *file, unsigned long addr, unsigned long len, unsigned long prot, unsigned long flag, unsigned long pgoff)