From patchwork Wed Oct 16 20:22:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 13838877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94CA7D2A559 for ; Wed, 16 Oct 2024 20:22:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 228F76B007B; Wed, 16 Oct 2024 16:22:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1B2BE6B0083; Wed, 16 Oct 2024 16:22:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 004B36B0088; Wed, 16 Oct 2024 16:22:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D29C96B007B for ; Wed, 16 Oct 2024 16:22:51 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id B4450A0B85 for ; Wed, 16 Oct 2024 20:22:32 +0000 (UTC) X-FDA: 82680588564.03.78F6D8A Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2072.outbound.protection.outlook.com [40.107.223.72]) by imf22.hostedemail.com (Postfix) with ESMTP id E5268C0002 for ; Wed, 16 Oct 2024 20:22:37 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=HZvHnBoK; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf22.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.223.72 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729110025; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=XfYFb1Zfu310SHf6hyaRs/ssZFI/tl+Loje/mHs5xWY=; b=IuPrHvGnowIdztUvggEM9GhAR7NysKyXVS5klPv3NG6wMfH651gqwWCufwCrCJDo/WpG5G 4Bv7S3bpx5tei8ugAdgAQDCRqxs5K64X9ATS212GLeqiHunqOSASJ4Mv8uPE3Q0qTNk6qT k7+tIaJudLMPOiJ/gGPENcTksmNFNWo= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1729110025; a=rsa-sha256; cv=pass; b=ZK5znTXybAC41MJKhxd5i18kj8HlDc9uSCKSK9duiSaYJorujIT1Y/fz5zGPd1/j3lK7hg bE6YbeOLtNLqPfQ0eotxdl5Zx00I/Id9En6jIfGUagQzdZwt+p8953MDAlFXZ0J3t1Wyxh H/77fahyugGHYxSaxpLMjJwYwvi5xos= ARC-Authentication-Results: i=2; imf22.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=HZvHnBoK; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf22.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.223.72 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=QkMyPVCSznKtCF8TgIy9XdwUdz9FA9/XPoBBUsmq2SC8ZpYZaMCbm5vFkAesiRmyEk05VZ7ljxc4gTbLxYAExzaxAk5EsLkGw7uNWGwZxqFEKoTtM7NLwkzNDqbN/AEmbbqAMvvUZ6zFsOGtu0Fd/BGEa/VOmH+y6CbGSi1ryI2iKCJ5Or8Fu3UKnPufmbi57izbxVlEcRIZoOFSFdPf4iQQqB/B6A5QdAD5v8tyOEbHp6iLEN1l0RNUn6nvn7BZW54ZiWiafl/Sv9wdIAOyTWV75UB9n6sTdGVuRvTbl3xhJZ+q9L5bXF/DGMEupRAPF6lxTM/7YHse+Nb7LwRDhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XfYFb1Zfu310SHf6hyaRs/ssZFI/tl+Loje/mHs5xWY=; b=VKE4s7swDWt6whwv4Q1DfGUHkyWoePP+nHuj7foWEEGHZsgyVCIuoLrLWR4N4yUEa9Qkqvpxe9Uys/m7HyWhsqyVaS1vbl9QWvOBMt+z1bEa4qPmnKzNhSkJ0r1TDLp3m4ZIFrGiuO6WGePla6thvquVW560vGQqDky1OHuSr6Ws6123oyCFfnaIrl/Wg2qGmONIOSvT5RdFeCIgVDtoX6cw2lSPvEd22r7CO5prwkMLxBvkVwwbT4hDDrDOUnsI18orS/bQVF2qPg7ztpTqju5ePPn4762cpy0nlIkhkfdxXfI1CTlLk0botjXgDdEdxOFwYuxAuG6bprS7ktUnLA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XfYFb1Zfu310SHf6hyaRs/ssZFI/tl+Loje/mHs5xWY=; b=HZvHnBoKjxBa34Zl+Tpjku+WvbeQNTjCtzzLfDsJ1Fa0s4u9ccHrd+cMw7Q6zvpdq6eU+5E1mKk8Z0BEwbaLcYxPAYTSPPcngPs4U3aF7rbLBty6vHVpc2i0DcoNs5CJSegBehD3EEr334e0ekggBECJYGTgbr98TvmXOw5YObi8d5s2MKSF1fOd24y2igSBtHb7aWJ2z/mowfmGLxRgDsmgEQaGRDyHer2zGoSKRk3l2V+XKzn47tQi/EvngMcSCp3DKNY/fkins2MCeR+AEwhoaceYgot0UrQFXyuLrqeqo6YuQ8Te73LnWwEr8pPwiZbeDyGCc03ekutMRkgUsw== Received: from SJ0PR12MB5469.namprd12.prod.outlook.com (2603:10b6:a03:37f::16) by SJ2PR12MB8941.namprd12.prod.outlook.com (2603:10b6:a03:542::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.21; Wed, 16 Oct 2024 20:22:45 +0000 Received: from SJ0PR12MB5469.namprd12.prod.outlook.com ([fe80::ff21:d180:55f2:d0c0]) by SJ0PR12MB5469.namprd12.prod.outlook.com ([fe80::ff21:d180:55f2:d0c0%6]) with mapi id 15.20.8069.016; Wed, 16 Oct 2024 20:22:44 +0000 From: John Hubbard To: Andrew Morton Cc: LKML , linux-mm@kvack.org, John Hubbard , Alistair Popple , Shigeru Yoshida , David Hildenbrand , Jason Gunthorpe , Minchan Kim , Pasha Tatashin Subject: [PATCH] mm/gup: stop leaking pinned pages in low memory conditions Date: Wed, 16 Oct 2024 13:22:42 -0700 Message-ID: <20241016202242.456953-1-jhubbard@nvidia.com> X-Mailer: git-send-email 2.47.0 X-NVConfidentiality: public X-ClientProxiedBy: SJ0PR05CA0044.namprd05.prod.outlook.com (2603:10b6:a03:33f::19) To SJ0PR12MB5469.namprd12.prod.outlook.com (2603:10b6:a03:37f::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR12MB5469:EE_|SJ2PR12MB8941:EE_ X-MS-Office365-Filtering-Correlation-Id: 9176744a-41b2-40fc-2f4f-08dcee204b32 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014; X-Microsoft-Antispam-Message-Info: yETHfsoPCa5FYtx+/He7Bo54DlADsGUH5mMSakNq1JE8qxkPdmBKJyg4GmlFprZYiu+i3UZ+nKu/Drxs0KpHcKHT+PMEq2lR1NLBRAviIlq6wDig6Y483mdcJqgGTJ3Uua6QlUZS/NILunrRqA/OGjaclsTMtrqz4nUzNKINk1ytDAHoZLsrsFLbZL/cENtsy8lUc98gA4kMhTFjVx4r8qMAd0vu5gx5Gy+jQKx7hWZD0Mbpo19fjPrwgdGE6/zP9DcS1i7Ut8UNVP/UdBtLlZ0PyXyGIk01r6TwqU+30PkzXSUT/n34CVp5uDFKkl1m/BqEjzkdQ6ZxSydID6Cy7v2jBD9ajAufYk4M/qiZY9xfeoo7f2ymhKhQeQmDBwkwXMCj4OJYc2C5auBSp7bf8dOOkjulVpkpZZcPnKLXKlkkSkBmjipM35epBJu4I1JKdoIt680E12IhulLE1LCQsIHMuWMr87Ya4/QgT4R6p0eZsBHUDoszgpH7eMfMgNitYLfRlNdnJb5BZihyx4KDJBxuaoT3AmcDXIxbvOBuqJs5JfZgxKBhDC98qgt1eYSzoVIwfI+zZgfhqUUYf/Aht+mLw5Uxm5m5E3X0HtPZTXw/gxx8ADFd7PoYUHR0W6uzGzDDMUCnqvo6TVlVVjpJRBpBVHfb55qa0rIvIKgvjv/WOfrJofn6TuJKUAMeDyH7mpyBzbVQzLOOwdyAQttOksO/QUvcQxIsb+NffpNYdSO/4KTGF7CL7f8Co54WdaLacrMKhNQvKCYxpjEnUZIMtsA3jOnCpprm/UckIKTP0sheF7BXnSWdUDMQhakvsh2BQregTahX4gPLpz3MZgW7HrNKYCTSJYSeEJUnRXTSaae4kWQ2NGKoa+w0nxzWkiqAxb0U21w+qpkWLpMVwbjgXF/j0h77TFcxYVsxyxQ8J5qhhXjtztywwYsD9bGArbJCaz9AY/s6HPgTlsF13wrL1S8dlnP+hCK/SgWPIHbGOOdRCSM/Y7hS/QrrMgv+uxSpjckQy6KuKxVwRj92EZSlGANRDib7AzDhuiTP8AkTEwqYNSJmQr0CnPh/fiAscUbdaDGa8mVaGBYpUFeFjLEdaU9LITkA6Ib7SwTtKCdwheKwuAMNPIFNCHegCj0X5/LDw2FxPgnQifHEfD9oe31uR+FYD70LOpM4b1O86H80+lxUYo/7YrnX3oTk9JhMP0GMzfyH5rDzGB3U7fiVUB8N5Itftde/iwXpmXrmjlWWtYPjEIvR1zI37YQ5Fg1lFkDE3bfgsLo2rxR1vDLqFLQgdZmWp8S3LEcIhs87oT7Zs2bGUMUhUzyuxpv9hve4+QI9 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR12MB5469.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Zfpgh6O4SctJoTgcyizu+8z/nLoY9OpT2/IpW76WTRi0MAoNWxxfpPxzZ7m5UoYO73LWiRljrqsS6rVxvm5eh/P52Cf6ArKcnAyT8FQO+6OD1BBCfpNOe02dV7zKgteALQWc98nT2Bhipm3zKRwNai6/xz4XqRt6un3SU6PX+jTDSKn6GuAvA8Sap3ENgqCZgGXPmnIwdZhSPRrNF+fgKtclDvoxEg3OG9ciuoR4fHdAz2oXb+MWMQdtYHHU3uL55oBpHO6gaI4r+xifvIH/HNQfeqMt+195DkZ2I2WG58m+GXmse+0Y5zaerLNuXM4BlL7RpZ+FaW0ftiHSToVU9hm18LFMBLKABrwo6RxnKUb+KplgtIaL7F3uOH9ZBjNcoujcXnU2eDgf9lpWmtWNFXKRHvPDXdz6QhFWk92Zfh4lGN3QEZsHY7xBSAIJ4tmlxIIyW7vwhl3F55SXLgoB79wMV//qCSHEHZErwxlY50M6dqL7wsRXtljPCDAiQN+ahdvgMVChO2FA9oDQgJEbltD6myd1q1NajSjbichlXRotjmPBkVzAPAYfDeQf46XZA4bd2b5TLSorvosR9WTMHTvc9Kc5xPTwhH+emVwBuBuEjRilc4xhTp1iF8JBvhkXsW8fkKkV+RRhlCxCUml+QA2f8SAfmpOgc2tPxvcZtc1/PCYwpCRjwW79T1XfdlSgT6/ZJ74iRI9+ImE392V6eKaP2yQYCp8WRrzvdXfYR0qggXta8pvDFTs6ZJFyHZc6eZZkByAz42gQb5RpixJkFBhaf+65Rgb6CoD/tcFAwGTX4qieCfj3vfehLgV2WlCtLU3VKpttefRZv42C9o3N7rYITpIRovRXtF/Mvzvp9vM3WGf8RFcV+2y68hJY6gru4WatlEzDxhi+YJYOJLdDmy4vG5NZ/W775jbn9IozuklsMQgxNGiU4UA2VRHn4F7ZHokrCr24N1vXqjLDC+5zMNXwBFLwzGS2ZRmUCNTtNoJeua0uwOExSH9GsrlXi9kIwfsnLP2UJwhp/I7+reFc26P6Xe0Ue0KLSV+pWAeXG1wFwtcIH9I3lEXSDEBnWLyaM6oU9NjjL+C5fXGZ8qWqondDFPesnPxP1iQs+9gF+lQdXmCwmxJM64XaiLdNmo7FMEyCxqRTFzeDi/rigF4D3EQhdpOh/VGv2OyzEaVRAOu2xxMdQqcBkD07SoIzxy++RPcY2hu11+wS/Nt+YuIW6lV56SWY7ATW/yoewvfDUi6e7742AueJBIYOTJdECHG1n34mYIrHsFlMsi0grB7KW3Wd8Y9VCR3eu5B2CiWfzIK9/BDWodNuA+4PZwGa0oHJpCmv8K68oCzgJvN7fk51Z4GPPzuat01QFugH8KaBz5S6XS2Jzkag23fzaTnBkf2e4fNMeiIRv3O8I0sDcQVrA9yHmSK4aE27BC/Nz9j8bPXHcQcpc8cOot9fdkEvU6z8jXUxcAHbqxREKOYLAUBUg4oamHEGD45Ar+55BWvqqoIqSGiNdqzpU9LT5JgdGX80aePwZ837aEyt1L7pFKyRUh8JuOHi1jsJSUivwWrV8TzAfPQHZ3YnlMUzdETd2Pb9 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9176744a-41b2-40fc-2f4f-08dcee204b32 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR12MB5469.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Oct 2024 20:22:44.8812 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: kCACIPfPiRADllwnrsOR2U1vZtxUYV/CJyQUJlJDwpoPUDfiu7fbeJIiOjR3UlQy6PbDghqj1RUgT0+JIudztg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8941 X-Rspamd-Queue-Id: E5268C0002 X-Stat-Signature: bqmgtd83ojyzm4pz66stcondmapdk1r5 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1729110157-52028 X-HE-Meta: U2FsdGVkX1/raxdLNlgyDJMWO7n9XlZGYmovhL2y6355fA8XEMNAEY7aRRJI6tI7V6OS71H8PjCVe3T0WPgJl9/WNWUY3ritXxRmRBw+Nvu0DS+wuUVx8E+qNX+NJyggpXerLK0lUFpgyzhVPmlSAFxkxksV3ZfYlFt3EyVh/hlFlpAA2mWO5okYsuHsh6osK4MEXc2ijSpZ2peYTmVP6G3UbrgU6vVbEMPhiIhTtyI9vWl+W2lHuxsdXZd8GXYIe/f0ymNHnOMZrZ8MBOcNYZjIUFtwvXris7gaMeobOxYGhDh7GtwqhkWH/Srf7hWXpzGP6pYtCXlBH2/GC0DwNXz1fY1Mh2zu7BdlwGWojxK2V+3gB47pWloFs+5EEUYSYBpSgQI++u3sBvmQ0dFttuBASiZNVu7XfqkdC8bHhEKZCzgFFR1SF1K8i5ZqCfjlt7edAEAx3mmjlwbxUz/zDDEM9nZQktBR1DqgdiQucoF4pXXI2J57pUug72CovtzckwJi82nXeysJkmugJDujf2P++XBE1IU6W7KbBmh2s7+w9ToxLRHnpf1z+Zo8jLJuZMWtedYAI2d+EPkZQzocbTJH+TwoZ486LLqJe3PGrXO0NerZXJjYc9O0gR2RGWT3yuav1uNE5ArjSx8svDd2srsal15Y4c+F+SpYxnyAgugLTZFUj021eBg4hJqnxhClEEilFA00BjBs15hq3OjuObLVrwFY9gbg70qPxAYOSvykjpBnQTK1j/72SonUPVEj/5NJhxtYuCwmEovkDP6IF9pfAvzVKHqC1ddUiakcMEo0DnxH0986dLVWwI0ZtMQ6EDbZGhE0jdi3uT4eVq9SB2y2qHEb4c2NI/aF0EuGvBgWKxBifjeIJl1QVsQ9vvvUSHk9/3X0DhIHFbmWaDkeRiYY34VcTO4XoxyJTSZYRHlyxgIpi49fFqppk/u9ZbO1MQuyC3E6BJ5nqiduADm Gyu0DxGw ZEyABGjFQgeo0ZJEmXqVo8L0wtDjKAnM74p/0R4UYeCK5muieE2rGvJGinAtLdSOdpVnOkcB2uSkmE/Xld7XiMtD6qi5Vxhtd9ptTZWqqbEQtiBKZiaIFefiuEWkYz2arcJEOPGao1kQt9cXbVYqZENWJHj04Mr/JwHG34BBH4X3FyLNjFt6Es1m4z8v/GDjlf7F97tMpgy+R2dbbr5wjP0xK2Va4GLaVHbepr9l/rUjotlfGASVwR48uLNjhUDF9oLhNSExK9G+dQrIHAEd7rNs8mSOCvsYWWHE3WFRq6MH3Cae70K8Mu8KGscKU+SdSvXWg9iXyLcLBCXE2tcAPNVznUhPWDGAMOlrAHrkguIaUCW4IbwUR/f6liMr6HugYjBWw4AOJ1QytwFU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If a driver tries to call any of the pin_user_pages*(FOLL_LONGTERM) family of functions, and requests "too many" pages, then the call will erroneously leave pages pinned. This is visible in user space as an actual memory leak. Repro is trivial: just make enough pin_user_pages(FOLL_LONGTERM) calls to exhaust memory. The root cause of the problem is this sequence, within __gup_longterm_locked(): __get_user_pages_locked() rc = check_and_migrate_movable_pages() ...which gets retried in a loop. The loop error handling is incomplete, clearly due to a somewhat unusual and complicated tri-state error API. But anyway, if -ENOMEM, or in fact, any unexpected error is returned from check_and_migrate_movable_pages(), then __gup_longterm_locked() happily returns the error, while leaving the pages pinned. In the failed case, which is an app that requests (via a device driver) 30720000000 bytes to be pinned, and then exits, I see this: $ grep foll /proc/vmstat nr_foll_pin_acquired 7502048 nr_foll_pin_released 2048 And after applying this patch, it returns to balanced pins: $ grep foll /proc/vmstat nr_foll_pin_acquired 7502048 nr_foll_pin_released 7502048 Fix this by unpinning the pages that __get_user_pages_locked() has pinned, in such error cases. Fixes: 24a95998e9ba ("mm/gup.c: simplify and fix check_and_migrate_movable_pages() return codes") Cc: Alistair Popple Cc: Shigeru Yoshida Cc: David Hildenbrand Cc: Jason Gunthorpe Cc: Minchan Kim Cc: Pasha Tatashin Signed-off-by: John Hubbard Reviewed-by: Alistair Popple Acked-by: David Hildenbrand --- mm/gup.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/mm/gup.c b/mm/gup.c index a82890b46a36..24acf53c8294 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2492,6 +2492,17 @@ static long __gup_longterm_locked(struct mm_struct *mm, /* FOLL_LONGTERM implies FOLL_PIN */ rc = check_and_migrate_movable_pages(nr_pinned_pages, pages); + + /* + * The __get_user_pages_locked() call happens before we know + * that whether it's possible to successfully complete the whole + * operation. To compensate for this, if we get an unexpected + * error (such as -ENOMEM) then we must unpin everything, before + * erroring out. + */ + if (rc != -EAGAIN && rc != 0) + unpin_user_pages(pages, nr_pinned_pages); + } while (rc == -EAGAIN); memalloc_pin_restore(flags); return rc ? rc : nr_pinned_pages;