From patchwork Tue Jan 24 20:34:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13114779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF26DC61D9D for ; Tue, 24 Jan 2023 20:34:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 638E26B007B; Tue, 24 Jan 2023 15:34:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5EA3C6B0078; Tue, 24 Jan 2023 15:34:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 413A76B007B; Tue, 24 Jan 2023 15:34:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 293906B0075 for ; Tue, 24 Jan 2023 15:34:46 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id E6AEEAB258 for ; Tue, 24 Jan 2023 20:34:45 +0000 (UTC) X-FDA: 80390846130.20.997464F Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by imf12.hostedemail.com (Postfix) with ESMTP id 312E74001C for ; Tue, 24 Jan 2023 20:34:43 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=HW3dAHkA; spf=pass (imf12.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674592483; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qi9NIlQIZAipa3MF9I93lS2N1oyVrOD46d4a19rPLic=; b=Z1mi8qiUaPYkgPg0rpuwhebC9kW+140rshYDX10hrjiEZyHc0o3YM26cUFBVCvlWNey4s8 m0tmnUtA7axXla3F9XjxMRJfa6cX5a54hRAcnvxlSFGUzWW3gpqGLqcjg5cYWAmc3vtGDS ZSQ7BpjC9bGngdkUfh8pLtqPede17/o= ARC-Authentication-Results: i=2; imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=HW3dAHkA; spf=pass (imf12.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674592483; a=rsa-sha256; cv=pass; b=mxVN9G7EtBEpcXU3y1rFPyVxaavnT9E5zAuz4mMkU7PQy98GVTt38a93GGh0CJb+0YtLm/ Oom7xuTfiZ59Mgaw2v5pSKpHid7bJ1zcCA6lWxsrUOLL/SJbdEaHYqHfq1Ug2eXCMNvaRy nVvp9WarZvyMIPnidoqGwYUCt09bTRw= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DC0TgyvvSml00y0ZkOfNYlU4pi6qukQas2DTiGBEDQCkCb3sHWf2v7vc7D8/5/8NyCBdRydWvo11Pso6xHeAFsKIMv2fjNGYB25A43zP0b1Xtj5MHncOKxh1H6HaxswpK9UwLRXZhN0QNE+h7VBu4z+3OkCo8J450Z9FSue/r4mkf21JOh9iSL+zQEI5w9Zfb/dv359NKcPdp4WkkgT+YQyAv7Zz9QGp8awALfPmrNtbbN4CBBvqqrLwP3WvD2ZSV60gp4v6Z9Lo2d41qP4VpAxD+9Zkvpv8kwpeoTyvOSTFH39wDIkgEPt/L8hUFW00gfsTArzkG8gK2+npwKnWEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qi9NIlQIZAipa3MF9I93lS2N1oyVrOD46d4a19rPLic=; b=fzbT3YSnHJgbL2K5/Os6qS5sdA1DcZ03kKpDVHeBAMMlFCKXeL4uZ+cYDzln03356p5eULNahQwZZkVu6v39TjQBDRr+DhLR/+6RcfTfP5NlvkmzxuQNb2tbLOvh8Hta7cqbm43O8ek4RcqgkCgQW0VquO/OiX8Zj5y8p5gTxCD4WaX0u4R5ClG00TGdeourYiQu3M2LlAzRbM33Q0GyaLziRlAHWt2zozA7Aqqlc9nCkbQYDE9JQN6HsHEspM1m7h5nW7exDltJRal5j2x9u01htVGR9i4VpqEDYTK3Zv4vSpM6WNzxC68TolB6EQ5FEgoAwIlGG0/Bm1ze+Zdv4w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qi9NIlQIZAipa3MF9I93lS2N1oyVrOD46d4a19rPLic=; b=HW3dAHkAJrq6lM5EO5ugt+dEYttwJ0JDqzLqR3tos1WJcCHCuMevfi/4299NFsWs9pZjIi3G8hj9UA29+LRkzvxMiZuOjFPL1Cwl+HblURkVk6/UzcryahP1fiWiGNb/Oc1EOZ5l1gv1dlD4l0Untitqtx60U7aeziXGJ3V41+X5iBzMtuzWCUs/OjwiIdz3RLBMhwOocjX9dfB8QioNAQ1AntZI6725AJpcPCu4nkQV9wyVITLSIa/835xKCxyQk7Y0tBnpS+8cDIO+95Jr4JVCRgtwNmVNwIVr498/gmtPYLBm6VoGxAj0dxivkHzMib5kYxAIKWdk3LVUTan7ig== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by MN2PR12MB4095.namprd12.prod.outlook.com (2603:10b6:208:1d1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 20:34:35 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 20:34:35 +0000 From: Jason Gunthorpe To: Cc: Alistair Popple , David Hildenbrand , David Howells , Christoph Hellwig , John Hubbard , linux-mm@kvack.org, "Mike Rapoport (IBM)" Subject: [PATCH v2 01/13] mm/gup: have internal functions get the mmap_read_lock() Date: Tue, 24 Jan 2023 16:34:22 -0400 Message-Id: <1-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> In-Reply-To: <0-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR02CA0022.namprd02.prod.outlook.com (2603:10b6:207:3c::35) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|MN2PR12MB4095:EE_ X-MS-Office365-Filtering-Correlation-Id: 91982323-2081-48c1-54db-08dafe4a67e8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5rN01T8oDvKM87tKbQby1V6B5ftUr4LQdFHsBjsVRiQfPdhg6oS7apx5a9kQyB9IaROx20J8snaAQI5gvWWbXetuUsoOujWoCHP7dnQ0yMKaAGRfzu2lsAzCxu66s3KomsfzyDAFQNs6etgnmAuLglxIdfeeUkMV7xmL8mr1jPatT4RGSEaQfo1edzKTJvEDaF+ex7SkX7PPMXphJE3bU4Nm7EfzJhZ0QCyG+q+BtdjqWGeVVh68IMTbjay2jRNdUNzhKdm2iNXzsDzPipocgslQUSCVQdnoL6ov/nYGTzat3/kTr0oSHFpi2tXCehnssJT7xGd8GgtxKy+3KexZNmffsD7p8fNuz8BxVeoK9+v4SWrwa1Nu5WgW+IF9/OJao3CxzFFn6auxxKZNXW5G6TE4HxEn0utZuEbEVFgPIHc1olE6/qhNG8robEYeGpfPI3t+PJwnzWQvQ33i7GNTErpip6xV5bUVhzqdvzuNDo3MhydNsMQ+0Tj6WN6iTUBjrKZJrWsfUvwPbF8atG3CVUBwIU9XuOLQr/cnWX2b2j8EO6PO2mGScmxt3xXavrSG1wWLZdCUHVmYllTLhybJPsuQDBuT/8jK4gBEeG+WEhi6qsow+ILNKHmPzrtmm/hg2hjCdhhF//WmNrQiCSsYRXk6D8Da1g2IUwxyDA7G/bK5e+DEYucS1d6rJ3ggugfDxFeEBuEfCaCVUFJFlX3FdFFMPD81nRr3Cw4BeMtS/IA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(396003)(39860400002)(376002)(346002)(136003)(451199018)(109986016)(38100700002)(83380400001)(5660300002)(41300700001)(86362001)(2906002)(26005)(4326008)(8936002)(316002)(6666004)(6512007)(6506007)(8676002)(186003)(66476007)(66556008)(54906003)(2616005)(478600001)(6486002)(36756003)(66946007)(4216001)(266003)(15583001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: jRuRVuIg4NOqnHoifhw+/DnNc7xvgwmo/UjZ9+3/r1HI5kStnoTvCq76AGnNqK4hV3ZW6GGbclIwYvLStI8fKecVjvXAPAe2Sz6ysuGVTI/q+BVCuB/iezorGvXwiXNALt0oRiQ8wIPv1+ruSSvzeL7Aw0pLn+UNLmq21OotDBgCSs76A8SxVnYOvLyzl+R+vLi7QfV1u/qRO62zjvlAq0cHYNUSP4TcEW4hSGhOb0wygTxVkjVfzuNj+M6iJuYF254+C5mCQhyykdCg74m0IX15eBVjGDaAJuH+Qi4tokDwHU9p3K1qe3ku8iKpj2xqPtzdxlPeGPy9t0eHlgKt3MIEfLyE1oWvUDNYrxMcTy/nlMXhSASU9nuYrchfs/rnIp4XROkYjsdCttdgvbIySND6jtML62mG9SNraqt8VS5B6o2i5gx1gEcGallVhTVbeC2eFgFuGSlmaBjQTb9j6f1ec4zMNn9t5pGkz60PlFnII4Hmr5IFI/+xlCAo0cDiasmAKEGfrgaOObgvOLx9y5ICEAakYFS57ovgOre0NLBFKRI21kp5jdIlJmF1hxaVI0v3U7zJE0cKwezJMixQLnDMWvPw8ISUcAuSc6gLQXTtwkUZZdB4w0yUVGt6T9I+6jXVCWmHxzbAti0pLgA1EzTMY6j+R4qsoNbQewxMoHCvQY2sY3c/nxlZ1jfR9hbl1yXPRtZ5AyIwx1ncAMcc8TZ3CxWzQxkKT1HQ+75PIHeVRnVHMWn0my7RT1ahqqjEpUeTp1YNMxG3buPJvHv4zIQSFE2OAWn7UJOmqP8wPykrgYaciToWu8I/zRDxFpq+nGoPT6DWIADBhiA56z4KWEofg2mZSwYnTTakQDlNiopj3T055wwwzVJo6RsL2G7r9iS84SHp7EKUS2YgEUTybrNAXafk3pDgab/uwScwxcQsy6mzIPQElMYZKQQy9Uwyh4Id+1I1oWp+MvxxVhRWPu9wEwfL6eYExBuRbstYarvI1pPqqUC2xaPbrpCikaSJ78V8lYyYExbtQxBJ3Hjum4hOw0LNCTFY7F1ukGHkGCPwNRAalpj0sU5hASBQuW0nSC0P7zoy62IQUfBLpXMDS9RIsx/RkqGNCoEzQLjk/kIZrNOd671NmQueqw+VKGFQIh9CI5LzZHcI/2CnUpEWo76VctokTwdJsZN4d5OPtDIvHvB5w9m840P5YD2sNc8XAnPNEhYehQOaPnjKNKnQgpjkIFAU/ZXWAbYsQxle5v3e1J2Bq1SWL/SvoHRsJIKiduryy1QAb1dx2UtWpra6NiSqTl+t/AveTH+hzGFee/+9endhgjCKJtbDu6b8BgDxd5P7HcLGy6BdACV0viyQltBjjpL4Iaw3MQ3dDFwuoNxX3a2UxBsOCaJj+Il7MqPAO3CUvp0ke/AQP0l+b8peytY12WD1RDAISb8g0GonlNFvVN0/4Z1BIiMcA5MK87AnREqFIjVHb7lbEpPzV10rAaq0q+tnUhwzfg7r5o7wS6Eik10Ub28wQ92uXdG9yrMzsH6oTAtH4GheFTIhYhMmCxg8U2apWgrmZuH5+Jwmb2tmVPx+WXzmKEISA0GohqzZ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 91982323-2081-48c1-54db-08dafe4a67e8 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 20:34:35.2983 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CpCYBkaQuobZch0yYFl6UISRS9cm4TTw4z86V9cRWASpPplPovtmvbKeoB8N/ogs X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4095 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 312E74001C X-Stat-Signature: em4wtqa93dde1ztubt1y3hm9kjwxdx53 X-HE-Tag: 1674592483-266739 X-HE-Meta: U2FsdGVkX1/+uGeI/2gjbAJoentx+IhQdryvPh4/dpx0Ikn8DQQe4y/QyC7S48CjrKwZdy9FjC3jFoCpU6wUSFP1FuVK5EypYYBQ5u+lPoCukDd7B4vkby2vSzmWP1Coep/dZE0ovZvQEOSrr2BE7YC8MQAB5jE2Z63+MeXhxE/9oJYkutSJIYUH7arFYSTt8Ff2UjHUcOCY/RviEN/Y71gYVRgJXD9rL1E2okNfgWF6xRRDp8vlp1RXhokLrkziyP6gyAx+pjM2WAkDvXkUMwQ1mbn/PdvSWM9KnEZhyDfxa09pPn5vYYVA0VuamS+SDa+6LpKoxct59P7HpwUC7mC0AX5wsRn1iy7AZsgiHHe7CycQRGBlqqhmTlPNKsXd+LO+JC/OWFR1RjuqH7w5nV45+Fg3AiJ25RI2U5aWoWWOi2dFQB1PtaCBiPnEluQDypgLS5haVz0FJNVD3l7v4MtHH/AtZQVx9HsPZMtF4iW0Vn8YFE7ecZRN+YKbpzhCmmZPoMUv3R7UY/Sf3AY23opAy25mQaX5sMVbDq/zHNKEtChZntFpAupJ21eFk2f+om6FUV7p3oYaKfLA6Qda71uTGxm05oLOUeuzDE8KHOCD7aajlp/gK9BpgT3Wzz3XO7xf7/2lnOaenWo08p2mtEr1EqLhkn0JvWR0rghJ42EuYXpdzDVRgvAxkhARJgRhXibSxjieqS0IiXR94ZP3buzsRtvFUrF+LK9vy0WZz8wbjumS13Aj9i58u+7B+rnuGRB1soFvkI2AUs054oaEi+f/T/ARbXZVaU5oC6l8iB+b0+PPEHmKOEA1Yab2+QvA8o5+VYyMs3lC2G99b1/iEOW/tUq0HOL8gXFwhvIljo37WrDia3C5siuM1CnJjTE6d6zN0f8tmSG0OWKWB/AeWhCJSUKiCXL0U+SgENOovpxIbw0/AUTI7M839IU6iRmDt08bmGBpr0QdqJ60whk R1eAaBR5 l0fVXmmM7wsAEXVXOktI+wPj0+N3GtoFhzoIiMw+h2cxFDrMJH6bc3JvrH83z1OAj7mGnyEHBc7Hf8MmIr1IsgRXW3zFcpv5kH1Ro+jNHqCU5sEyYYTSHFWqCk9ps8voqxFLG9H00lXq9GIWAo7Of7RsE04+PYLj9oYT3KKPohrf6+46TTAKG98McMD/g3DkxTocTp6S+ZLEdLww0cVoLis1NnkpN6OQNBSWIguy+KPi16yGdEECYWYbt0L4vC0qQpyRDQstDtoug7BEbAgD/Ji5fBqW1KXCmHOoRgLaNZ8opUGUrwgvU9pLzFxa3G/vjTaEq95Uq5ONwQwAQeaB8InnYsbLJrJsNPTEI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: __get_user_pages_locked() and __gup_longterm_locked() both require the mmap lock to be held. They have a slightly unusual locked parameter that is used to allow these functions to unlock and relock the mmap lock and convey that fact to the caller. Several places wrap these functions with a simple mmap_read_lock() just so they can follow the optimized locked protocol. Consolidate this internally to the functions. Allow internal callers to set locked = 0 to cause the functions to acquire and release the lock on their own. Reorganize __gup_longterm_locked() to use the autolocking in __get_user_pages_locked(). Replace all the places obtaining the mmap_read_lock() just to call __get_user_pages_locked() with the new mechanism. Replace all the internal callers of get_user_pages_unlocked() with direct calls to __gup_longterm_locked() using the new mechanism. A following patch will add assertions ensuring the external interface continues to always pass in locked = 1. Acked-by: Mike Rapoport (IBM) Signed-off-by: Jason Gunthorpe Reviewed-by: John Hubbard --- mm/gup.c | 113 ++++++++++++++++++++++++++++++++----------------------- 1 file changed, 65 insertions(+), 48 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 920ee4d85e70ba..7007b3afc4fda8 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1331,8 +1331,17 @@ static bool gup_signal_pending(unsigned int flags) } /* - * Please note that this function, unlike __get_user_pages will not - * return 0 for nr_pages > 0 without FOLL_NOWAIT + * Locking: (*locked == 1) means that the mmap_lock has already been acquired by + * the caller. This function may drop the mmap_lock. If it does so, then it will + * set (*locked = 0). + * + * (*locked == 0) means that the caller expects this function to acquire and + * drop the mmap_lock. Therefore, the value of *locked will still be zero when + * the function returns, even though it may have changed temporarily during + * function execution. + * + * Please note that this function, unlike __get_user_pages(), will not return 0 + * for nr_pages > 0, unless FOLL_NOWAIT is used. */ static __always_inline long __get_user_pages_locked(struct mm_struct *mm, unsigned long start, @@ -1343,13 +1352,22 @@ static __always_inline long __get_user_pages_locked(struct mm_struct *mm, unsigned int flags) { long ret, pages_done; - bool lock_dropped; + bool must_unlock = false; if (locked) { /* if VM_FAULT_RETRY can be returned, vmas become invalid */ BUG_ON(vmas); - /* check caller initialized locked */ - BUG_ON(*locked != 1); + } + + /* + * The internal caller expects GUP to manage the lock internally and the + * lock must be released when this returns. + */ + if (locked && !*locked) { + if (mmap_read_lock_killable(mm)) + return -EAGAIN; + must_unlock = true; + *locked = 1; } if (flags & FOLL_PIN) @@ -1368,7 +1386,6 @@ static __always_inline long __get_user_pages_locked(struct mm_struct *mm, flags |= FOLL_GET; pages_done = 0; - lock_dropped = false; for (;;) { ret = __get_user_pages(mm, start, nr_pages, flags, pages, vmas, locked); @@ -1404,7 +1421,9 @@ static __always_inline long __get_user_pages_locked(struct mm_struct *mm, if (likely(pages)) pages += ret; start += ret << PAGE_SHIFT; - lock_dropped = true; + + /* The lock was temporarily dropped, so we must unlock later */ + must_unlock = true; retry: /* @@ -1451,10 +1470,11 @@ static __always_inline long __get_user_pages_locked(struct mm_struct *mm, pages++; start += PAGE_SIZE; } - if (lock_dropped && *locked) { + if (must_unlock && *locked) { /* - * We must let the caller know we temporarily dropped the lock - * and so the critical section protected by it was lost. + * We either temporarily dropped the lock, or the caller + * requested that we both acquire and drop the lock. Either way, + * we must now unlock, and notify the caller of that state. */ mmap_read_unlock(mm); *locked = 0; @@ -1659,9 +1679,24 @@ static long __get_user_pages_locked(struct mm_struct *mm, unsigned long start, unsigned int foll_flags) { struct vm_area_struct *vma; + bool must_unlock = false; unsigned long vm_flags; long i; + if (!nr_pages) + return 0; + + /* + * The internal caller expects GUP to manage the lock internally and the + * lock must be released when this returns. + */ + if (locked && !*locked) { + if (mmap_read_lock_killable(mm)) + return -EAGAIN; + must_unlock = true; + *locked = 1; + } + /* calculate required read or write permissions. * If FOLL_FORCE is set, we only require the "MAY" flags. */ @@ -1673,12 +1708,12 @@ static long __get_user_pages_locked(struct mm_struct *mm, unsigned long start, for (i = 0; i < nr_pages; i++) { vma = find_vma(mm, start); if (!vma) - goto finish_or_fault; + break; /* protect what we can, including chardevs */ if ((vma->vm_flags & (VM_IO | VM_PFNMAP)) || !(vm_flags & vma->vm_flags)) - goto finish_or_fault; + break; if (pages) { pages[i] = virt_to_page((void *)start); @@ -1690,9 +1725,11 @@ static long __get_user_pages_locked(struct mm_struct *mm, unsigned long start, start = (start + PAGE_SIZE) & PAGE_MASK; } - return i; + if (must_unlock && *locked) { + mmap_read_unlock(mm); + *locked = 0; + } -finish_or_fault: return i ? : -EFAULT; } #endif /* !CONFIG_MMU */ @@ -1861,17 +1898,13 @@ EXPORT_SYMBOL(fault_in_readable); #ifdef CONFIG_ELF_CORE struct page *get_dump_page(unsigned long addr) { - struct mm_struct *mm = current->mm; struct page *page; - int locked = 1; + int locked = 0; int ret; - if (mmap_read_lock_killable(mm)) - return NULL; - ret = __get_user_pages_locked(mm, addr, 1, &page, NULL, &locked, + ret = __get_user_pages_locked(current->mm, addr, 1, &page, NULL, + &locked, FOLL_FORCE | FOLL_DUMP | FOLL_GET); - if (locked) - mmap_read_unlock(mm); return (ret == 1) ? page : NULL; } #endif /* CONFIG_ELF_CORE */ @@ -2047,13 +2080,9 @@ static long __gup_longterm_locked(struct mm_struct *mm, int *locked, unsigned int gup_flags) { - bool must_unlock = false; unsigned int flags; long rc, nr_pinned_pages; - if (locked && WARN_ON_ONCE(!*locked)) - return -EINVAL; - if (!(gup_flags & FOLL_LONGTERM)) return __get_user_pages_locked(mm, start, nr_pages, pages, vmas, locked, gup_flags); @@ -2070,11 +2099,6 @@ static long __gup_longterm_locked(struct mm_struct *mm, return -EINVAL; flags = memalloc_pin_save(); do { - if (locked && !*locked) { - mmap_read_lock(mm); - must_unlock = true; - *locked = 1; - } nr_pinned_pages = __get_user_pages_locked(mm, start, nr_pages, pages, vmas, locked, gup_flags); @@ -2085,11 +2109,6 @@ static long __gup_longterm_locked(struct mm_struct *mm, rc = check_and_migrate_movable_pages(nr_pinned_pages, pages); } while (rc == -EAGAIN); memalloc_pin_restore(flags); - - if (locked && *locked && must_unlock) { - mmap_read_unlock(mm); - *locked = 0; - } return rc ? rc : nr_pinned_pages; } @@ -2242,16 +2261,10 @@ EXPORT_SYMBOL(get_user_pages); long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages, struct page **pages, unsigned int gup_flags) { - struct mm_struct *mm = current->mm; - int locked = 1; - long ret; + int locked = 0; - mmap_read_lock(mm); - ret = __gup_longterm_locked(mm, start, nr_pages, pages, NULL, &locked, - gup_flags | FOLL_TOUCH); - if (locked) - mmap_read_unlock(mm); - return ret; + return __gup_longterm_locked(current->mm, start, nr_pages, pages, NULL, + &locked, gup_flags | FOLL_TOUCH); } EXPORT_SYMBOL(get_user_pages_unlocked); @@ -2904,6 +2917,7 @@ static int internal_get_user_pages_fast(unsigned long start, { unsigned long len, end; unsigned long nr_pinned; + int locked = 0; int ret; if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM | @@ -2932,8 +2946,9 @@ static int internal_get_user_pages_fast(unsigned long start, /* Slow path: try to get the remaining pages with get_user_pages */ start += nr_pinned << PAGE_SHIFT; pages += nr_pinned; - ret = get_user_pages_unlocked(start, nr_pages - nr_pinned, pages, - gup_flags); + ret = __gup_longterm_locked(current->mm, start, nr_pages - nr_pinned, + pages, NULL, &locked, + gup_flags | FOLL_TOUCH); if (ret < 0) { /* * The caller has to unpin the pages we already pinned so @@ -3183,11 +3198,13 @@ long pin_user_pages_unlocked(unsigned long start, unsigned long nr_pages, /* FOLL_GET and FOLL_PIN are mutually exclusive. */ if (WARN_ON_ONCE(gup_flags & FOLL_GET)) return -EINVAL; + int locked = 0; if (WARN_ON_ONCE(!pages)) return -EINVAL; - gup_flags |= FOLL_PIN; - return get_user_pages_unlocked(start, nr_pages, pages, gup_flags); + gup_flags |= FOLL_PIN | FOLL_TOUCH; + return __gup_longterm_locked(current->mm, start, nr_pages, pages, NULL, + &locked, gup_flags); } EXPORT_SYMBOL(pin_user_pages_unlocked); From patchwork Tue Jan 24 20:34:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13114786 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8FE3C61D97 for ; Tue, 24 Jan 2023 20:34:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E763E6B0081; Tue, 24 Jan 2023 15:34:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DB2AA6B0083; Tue, 24 Jan 2023 15:34:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BB5636B0087; Tue, 24 Jan 2023 15:34:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A12116B0081 for ; Tue, 24 Jan 2023 15:34:54 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7D699C0516 for ; Tue, 24 Jan 2023 20:34:54 +0000 (UTC) X-FDA: 80390846508.12.6C0451A Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by imf12.hostedemail.com (Postfix) with ESMTP id C4B2940005 for ; Tue, 24 Jan 2023 20:34:51 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=BnIa0pW4; spf=pass (imf12.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674592491; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LD6pgx3+mCBOiRvZl8kvDgLnMNfUxttkl+1KJn4Jg8g=; b=24A3fzGEE5BFqsCMnEBoEB1dwDtCRsSM3TWGXimBL/7M9ZGNrq6LfSMmMu5xuf5ni3Q15Y j6/zoL/44il+tpIBb9XhxMQwjS4hP8SKK64uBeT6KVvkIxyMV7HK1I2E6zdi/N4kUAYQVz q7v674pxrpU61dRm6A0WmWQvWJmKrw8= ARC-Authentication-Results: i=2; imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=BnIa0pW4; spf=pass (imf12.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674592491; a=rsa-sha256; cv=pass; b=IDCZjdO/QzNpiYYckyaJoeuB2aox+Lz1pq8qm9RZSdccY1Xhrp0o7LnZsZ9i15XPluEMS+ 4ckJzOTEuFayyctfm5ZnUaQWYb0/+7JaKSrHBZANVmUNs/FV2uoPihn4s+8ljFLPT3Rtgs NdwhbFxSKJphNxuI3CEDOUUw9fZ6818= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q1m8PX05PHFC05jRQvLjsNydDFgToFUkSbbjPnU1ok4qLw1JqhIf7m2ZAvoeTSohvY1F1mrR8g9FEse8H/HX8LXDj+3KdD3HQfdMxmrg71XbgUuKdI+Vuo/BUfkG8SuxlrTHRXwZmbQ5+0Va/cA4eRBkJ4yUzI/vyKI13vfuLcd5Kn6ETw9axl3yW1vH5f2JVFKLYfzKEFJ4SzG8ORV/NOcmxhy6/XEqDiaCMywEV5DO6xURFiJVG1iDQS1tvlfM5N3RWThAz8R091Ot+wMvPGArgejFqjraLMxCrquxwjkrRK0dpANyiKHenIWGXQq/V3ubeHyBxKv4pnz7YjIN9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LD6pgx3+mCBOiRvZl8kvDgLnMNfUxttkl+1KJn4Jg8g=; b=MH0+20eb5uImArPOmSCK0DD18i52ma8GqhKNogJIAUxyIo/0v1U7uWzQ09RZtCS2nP5IzrhvPF8P8Ic8qkxqCYRj0BBYCCEUk5sTCejX4SsEZwxujp0c/V/TmqPJISDKXSJddeG4awGZmv9D+pcwq9P5ISw/s6Um+SFEb0HZ+jnJ+JLf/aFGZ728HybJpCH/0+iifzAP+lnY+RICF/RcWafCmlJ/qCvPFHteLBokzt2qOxmxQrlS2YEhocdkvqjL3b7z0DfqeFMa1X4YUUhDbxcuqhmxDE4W4zGcMjBF6QHDzhMwrkTedkmeI4zaYlH3ddSe/IcmChxY/wy5KTWbMQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LD6pgx3+mCBOiRvZl8kvDgLnMNfUxttkl+1KJn4Jg8g=; b=BnIa0pW45Mw80MKVQSPA+I1lqp1Nj+1TA9+2jdCc2cwAVzScnAFjsP5+ZCh7jRTyr4wsmkJ+odm1eBqzAD7KhQ6NDPRjHTo4VW+/lr434oCsyp9/03OBmSKfyuq+gmtw63qdo3K5mPZ2Oyv6GyDhqOFIbjlfQMjm9JctuWe/hwEIMxmSGve9SkQJTDWvgbCkPicLF5kQrqiw/WeQVm67d4J5ArjphMaH5f7xJjoE/Z3Q9k3JY+CJ2JzdByjubbjYMxprqaTTNCuLEyZ+aK3d7ce8q8SaUauhkl1UZF3Myjp+tgvjTpyGl8o4L1N2uWZI1AnM0bLoCg4oXTZtl9kyFw== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by MN2PR12MB4095.namprd12.prod.outlook.com (2603:10b6:208:1d1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 20:34:39 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 20:34:39 +0000 From: Jason Gunthorpe To: Cc: Alistair Popple , David Hildenbrand , David Howells , Christoph Hellwig , John Hubbard , linux-mm@kvack.org, "Mike Rapoport (IBM)" Subject: [PATCH v2 02/13] mm/gup: remove obsolete FOLL_LONGTERM comment Date: Tue, 24 Jan 2023 16:34:23 -0400 Message-Id: <2-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> In-Reply-To: <0-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> References: X-ClientProxiedBy: BL1P221CA0008.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:2c5::28) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|MN2PR12MB4095:EE_ X-MS-Office365-Filtering-Correlation-Id: c3d70917-e8a7-4030-b412-08dafe4a68c5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cLH3bA9IwuTbdiA3PlJcEw3ZMhpX4DwPJ4TmcpBjZiKv50GrwzYDAho4kHfrhO6dBti3m9hWXkAlpzkUz6isT37uVK5PlRm3XiynWZ3OSrtYcc/kgsRUPB9xmF8ewpdGn12wnW3FxuEgxJEJHcygAHWUVw9lQPPPCBZVDxPhsDsl7W9+qX2/j6nRdcXYwIJz0WPPS1KHXHwsTvvd8EOf4swMOH02zXubXI0wM+JyqNleyyNnfbXwMb6qL9M9CUwsOBymlZS2xfdSZ6Gv3TBowDNOTzh/nzPuiqBFzi0YODkaSFx3gi2RKlFJmq0J71HVAtbBaauKjAwHTB9HwyJhFubqs0PXFaYaHB0km0LEIdhxaDBSfLM2uEfJ2S0BYw2PYFWtr/0R5nWR90EPMmSv2WioDWBnoyYuqmVo12lJ8afAFss3tyeW3HF4K74SbjOoQp+JEP4Zi/AYUEL1GYOaQ4RNL1tYdXkbprQqPIJXSumIrwsmNybO2luuQy7X2lhhtzn6Ch1EwGHXwQqHWxDK5pLTihSfgM7/gr2MIK8nbpV9RpmyJEIUO1MkN+ZoCu049weJTvUvqOQeH7oEGgxGhCtaGaP+VpYBJ96TDasGXEbUyFyfWCoBK43/xD/0+/97cR0v3bUnhPjJhXmJeqsyxfbV5YMUwmx/Eng1oOPWxmM/JpElvRyzahHzDs3dFkzwFM7f3ELHgbpiQzkXuO6AxA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(396003)(39860400002)(376002)(346002)(136003)(451199018)(109986016)(38100700002)(83380400001)(5660300002)(41300700001)(86362001)(2906002)(26005)(4326008)(8936002)(316002)(6666004)(6512007)(6506007)(8676002)(186003)(66476007)(66556008)(54906003)(2616005)(478600001)(6486002)(36756003)(66946007)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: +D+RvpVlwuHlwXFvHIn3zczcJKUVlLFwDEpv5TZl5fKN3ekauWQAcb3wNMdXgAchMBOVtWKFI335G1woUhVvhHJ4M6ol3KGtZQZE6+MSVro1WvfykU/0I/NzLOIsl8qC40nw2ssy4Bu3+N9KEaaTIjnl/sITvAQ77IpAIFTUfXN5REcTQq8lU5thVDx3DRhTL+36FooyBl9D9WBv6DhFdx2GIXuS3p3G3QNN9gLmhlkt/wtvkIEorA51bEcC2e85ZMz4YVun4Wo14ddgNpznQrhK4PmVmIQNcX43VABcfDCLDSR+oz61owBDT378+cHzWhQkJfi0bEED6MVW/vU3fTpzpc3SGdZWcFCzWGeyxvgGo3+EhmScSLeAlQ4i3IsWyut9oRUpGkePWV+GohQcCVX84RSShl8uFTnAe7+5qBeEnWtHUJmtqbC0vsKb1a6NI12MB8Xh+DePzhr+z8H3HBH4RVjSdvIs+parLs1XcyvEN/NrkUi/gne7nfF+zbMOOsOHDjmx5lUuf0BDPFM/4RphIRQ5GU35iI8LMB8QXiDYY/nSbSOiEM12hetPoBNb3noRYVXkdL6geQECtJuRiKT44EwD6jz2LYor8Ig0dHsR0LE7pzurlLH92V9VNsnnmTnW5fxnGcK/237b71nxaycf0GYZD0OwiLXxIcuzrnmpt20z+INv/LxEFi6E3wA+WS+Dzy+q8Cghw+T5p0oHdBzr71clm/WCffglviPDfq47tGp9+K1j/WMsjVTRkb5GbpQTIV7Y8lIFJrv/3CzbW5BlgawZWDfAgBxcsu4n9Coq0N85qR5n592nkQdaCMlXQAGlkXwSUYqtwhx39gPGl2snh1bWJmMW0WFRDtKOLxFKcO90HbB8zuY9tgGEoPsAHa1cvVsfbxlzuDbrjowMe8fbl8IG68zu8KcEUA7XZWK5O2cS/0WEM1sV34fFW/o9dW+Ju02v+JkgHLVMiaVT/4rh2HRyYcTj1CrtPIq8rvHLR6QoLonMvY8uSLBQuK6t8c8vHRYuvmknec3rn5UyvGeGimdHrnNZjQ/DSLj14HdBoMqwYUlXuJNBsQ8xLmhtEGdQd6WB9u6Acnb//BDFev8WyXp55WhUVac+wJcwcAZletfylwj647yprIg52imPnvhdIPz0trahP9xJ9FsPp9YZTisuLK/HYGy0s50jZjRDmAEPX/aHJ+ZQB5YD17OQ5YnOLo4vHab0OK3PMNKZRbkyWBDq7g3fg51KNvG5ZJvY9gPy7Mj4FwV/e0xHj+J109fOZ4w1QlYeYIeupUDuiVDQo5N3hiohgbrtfOlL7/JZYcM8DbZwLCSOQiFFtsfkJvBUUYG1kVormKjlJU/lEifPaGn8YyvkmnrnqIUerqrrf5cKwTWE11nMGtmq9r91127kBsW4f3Bls8to6IcPed0pIx/RBqDkDnz3E4dUajU2Yb5wx2hoaPPIqApmcjYFMw4RcFpukPmDs7rXMH58LeaCts7dpQG4niLA2xAfXs507jicuwM4xk3If6v+CWXAWb966IPGiXgX7LJXmHRfaT6huUP0YVeI2raFJHVMJnklB5BkF5eJ+gV7Wzh8Sh+m X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: c3d70917-e8a7-4030-b412-08dafe4a68c5 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 20:34:36.6587 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: thOBbpoXd8by5VM5/BtAqG4ag70pO+o7PpOGguMnGrCKp/NqyTM5Lf6/hMpwATxK X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4095 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: C4B2940005 X-Stat-Signature: wjwyyduygg6jztns6ym7g5113r9cdw7a X-HE-Tag: 1674592491-469119 X-HE-Meta: U2FsdGVkX183xXA4Kt/hQh/SRBFmAQZCgF8tp1O0ZmKqFWJlv9ckJaWZvRxSmo1RvjIaaaXkpBEYC7RreE0MWNRvmkDYAb5wwlICF6nxwQy5462bb+Cy/d8MA/Yg0O1OSZqyaZP8rip7U9aD4Vl6VZzV+EeHqn76vLp7aYUk4hpnicLd1v2hv0Z6OAw1nP99wi6OHKOCbdHaxoBDAmC+7Suh2Y8CQUoDqr+sVQjS8YP1JTtcPumEKwJXu4Spr3qKO+U6HLLojDPv4sI7VvFF9JSFDfnA8hA45Pas2ZPS8hmZoMpnNLsJn8K68XOhbAS6dS8QtxVMEskyMHlIrYtxeCwbMx5rIuiuOarueYqCF7Fuo//LyLuom6f1bYB+3EAHkkc4Oa42YkmTGITB4PA3lCVOs/J5v11axd8qESZvHWaMK4YtNGKcQXFjP3BZSVEdStqkLIYvmegbJLg2xh4X368vtTbOplrwr4oNKeBOa7boRCGqe/Mq3+Pr7RsO7sPEiDCwFqURBV/4lqnChE51OxKVVTBMi1KYhDXKkljDfR0gXJx4BWv8EmgtMxAPRqAX2PgX+fp5QlpoT/UCdYLm7eIeCyMW0Xzt1bS0jURD3pvpSlUXfjljRWdzfllJvur+1mP/Ov1tNzzJPRiB3aboG8iJblsSTmx/jf6rrPwPaq6gcyW79ujD3dwhBPRm/FNJP+aJFsUWox7KmJS+vjW6p3N5gfOZHmnq+JNTk6SDGAUvPKN3ZmZjJOQ4JgWZMT71DWzk9uxjjBpOyv330TiqHfwFPMd/aXGY3XL5WT1EhmZKXvRxlXCouDTSwanUUkI/OGGWIAW+tZ+2hCFQDKifr3OsTY5bpXJ4C213Eofo93aUfdxT+0wg9ol5+WRav7DY+Rp/WL2dzEK6nCboN6Ya9u8Ozp5G4NK6eQpx8PFRs/pxX10rPG+Lf9F+uhxCqD+tCENTuAR3pKjtFzr7wgw 1fQ8WLQc tJUWfrAozYsW+fiPYzdZ8L9gz3n3Y+Fzo8qBo4d49gOeWUKZgGMqm5eSDLIw6m8OYfEAAmEPjWVxsY1iTb47migWyZV03EMr0DDe0zKIBY5GcFkEVUO6wfr3j3glk4VHNb08qsQK0E32zcfZF7saSFZjQmqp5rEkxAbQHozdyg8DZ/YOwjxM/We16NxJxBCEH7rZygt1k/GWu6pfNkzP7I36BR/TE8ISKKgFhl2YE0iRXt7sPD5KtO/dHWCuBOfL/hdmG0wSqPF1OTJerQ1Kn83Oda63TbFdeH81J X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: These days FOLL_LONGTERM is not allowed at all on any get_user_pages*() functions, it must be only be used with pin_user_pages*(), plus it now has universal support for all the pin_user_pages*() functions. Signed-off-by: Jason Gunthorpe Reviewed-by: John Hubbard Reviewed-by: David Hildenbrand --- include/linux/mm_types.h | 6 ------ 1 file changed, 6 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index a689198caf7408..8971a40c120e38 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -1052,12 +1052,6 @@ typedef unsigned int __bitwise zap_flags_t; * specifically failed. Filesystem pages are still subject to bugs and use of * FOLL_LONGTERM should be avoided on those pages. * - * FIXME: Also NOTE that FOLL_LONGTERM is not supported in every GUP call. - * Currently only get_user_pages() and get_user_pages_fast() support this flag - * and calls to get_user_pages_[un]locked are specifically not allowed. This - * is due to an incompatibility with the FS DAX check and - * FAULT_FLAG_ALLOW_RETRY. - * * In the CMA case: long term pins in a CMA region would unnecessarily fragment * that region. And so, CMA attempts to migrate the page before pinning, when * FOLL_LONGTERM is specified. From patchwork Tue Jan 24 20:34:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13114781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57100C61D97 for ; Tue, 24 Jan 2023 20:34:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 63A536B0078; Tue, 24 Jan 2023 15:34:47 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5263E6B007D; Tue, 24 Jan 2023 15:34:47 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3781F6B007E; Tue, 24 Jan 2023 15:34:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 1AB0B6B0078 for ; Tue, 24 Jan 2023 15:34:47 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id CD4161C648A for ; Tue, 24 Jan 2023 20:34:46 +0000 (UTC) X-FDA: 80390846172.14.6FC0E0C Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2044.outbound.protection.outlook.com [40.107.243.44]) by imf04.hostedemail.com (Postfix) with ESMTP id 0E63C40021 for ; Tue, 24 Jan 2023 20:34:43 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="fF61EZo/"; spf=pass (imf04.hostedemail.com: domain of jgg@nvidia.com designates 40.107.243.44 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674592484; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6C1mPyrOdOAKQZxg0z4XyWJ1Fbd0rMj+97/SFfFOIz0=; b=qzoHiDTieqhcD6S0IxvmxfTmWNicu9BI1YPgxZPOGZswk6/ntS48KL8xwkPHiBkk5cEhrw +Nep1FAQUPSoAOoaon2oqDtWPWoxI0d3lASNNluIEDkQNMv3/3HBwhsasBK3L4jzwrOuPs MNbtZPe0v/xu657IqprzoTqWE2BQCpM= ARC-Authentication-Results: i=2; imf04.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="fF61EZo/"; spf=pass (imf04.hostedemail.com: domain of jgg@nvidia.com designates 40.107.243.44 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674592484; a=rsa-sha256; cv=pass; b=svH2IRPYJ70Q3dNP/AT84b6VDCsnLwt2E+X0ymCDi1AKbSJCL6MK1l8sf7SX0dYlErGOJt 34lORZtzAJNFjUu/tWpUZTuKpNJt21xxjkxDttCK7uLmxyOn77xT6mFFI2bwjybjgoptnf efSk9vLjQphkB9l3xNguV8TWDaujxi0= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OW9pUOObOCKnVnSxNFauQZHh7FRFVrqFRfPvCioapvJH8VJfKttMb+3WU4k+f96NiUfQHZt+lEyxQr4camS3JZaM/X7JprSTjBQaYEk6eDzsRND0JIfynX/dI6MyMxoHroHHbit1YyIUWOKFolNk83LxYGLCEIjrXCcN3vLCL/UZ3sOUp+EJjegh+cP6x+jCP75BVq3Vl9qcnSaUVEJVpNqP3Wa3RWqu7ogWBWc+FDyr/Qa1GqDp4/VFwZohwudV1AXDeKipKU8yfhsDpkJcPyW4WBFHmRn/RzSc/83g25RXzlwvX2hgFv4QjOzOE6zWOMZclq2Nhfx/8i4XNolNag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6C1mPyrOdOAKQZxg0z4XyWJ1Fbd0rMj+97/SFfFOIz0=; b=TwdbSwXCxPkdxqRieeHWgwt47B+p9SwDDKATPmbRdbitNjyPKWv0OBvXO0z/xKJBdOBeTvUxverYIN3F1mqdnJI3MVRlydKZUCQObzVLK2rUqiWoU5DFfSxDSwBIdx2GVDGlhWTsZpcOVDe6ejQ4cmPvxBMlHBdDnxL2Vk4IU3Vzq0kb7jtYs1ENO8LbxnaOXVDNfhXoPnD/NdPIFuUWJMJDBfimwk3aHauVHssbAA/k3Ebx1pZx4R9vvK8xVaGyobPGPV22Gdou9bvOG7a7PTyca830z/It2j6q+GTILmWAYCvDFClGJMQwrB6BvTus2sIRvsT8XKi/Q/ihXKahcw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6C1mPyrOdOAKQZxg0z4XyWJ1Fbd0rMj+97/SFfFOIz0=; b=fF61EZo/ogO2gcz7DNSgE2vWJ3eKoi04N7HsZusmjv/FqiskcctNoJDMlMqZhp6/azeeA1a97CCD4vZm2p2YsgwzyNyeityqLmk+XRnyF3/rV9tCFPZXrG2NKJlGMwlB0YasyVX4dUbPvRfuUbVKIOpN/WlCgOwjfRKlDTtzn+pWJ5JL/6pM1F5nF/ojMZuet2U1U3etWq5kCTtwy+tb+fmvPS75ALv3dbEe6LMkexdOy0PKze7DU4OHBlOhisBkp6y/kLHLwpndvT2j33yLy2dbSOtOsZYsMlK2T18zUw1qztDQ7JbHFKQUPBvgcw8NXua5ttlRgSSZem+Z7Yl/eg== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by DM6PR12MB4220.namprd12.prod.outlook.com (2603:10b6:5:21d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 20:34:42 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 20:34:41 +0000 From: Jason Gunthorpe To: Cc: Alistair Popple , David Hildenbrand , David Howells , Christoph Hellwig , John Hubbard , linux-mm@kvack.org, "Mike Rapoport (IBM)" Subject: [PATCH v2 03/13] mm/gup: don't call __gup_longterm_locked() if FOLL_LONGTERM cannot be set Date: Tue, 24 Jan 2023 16:34:24 -0400 Message-Id: <3-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> In-Reply-To: <0-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR02CA0083.namprd02.prod.outlook.com (2603:10b6:208:51::24) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|DM6PR12MB4220:EE_ X-MS-Office365-Filtering-Correlation-Id: 4d528839-b74e-4b3a-1ce5-08dafe4a6b91 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: i3eb+yND73QcALSRSkoFeHhUsoXuGyIs16vyXFOnLzFl0HU2yfskP1XbvdAWCga5zjmm2FN3F7vRg7w0IldgdZfCu5mwf2KHWyMYZ0oiU0u9cdFJ/KdN+q1LpJ/ixPiEreKWQc71QOAiqOHhX7BJNJJqACxWzGVDOrcfSLW60WHzmI1aMXFsexY/k+9LIJHALeOXgg0nrqB4i0y6t0pfBLvlFrDuQSaV5Kqv1I8v9YeN4zgSAAlz4YgsTO4VX+Wkf5jxazKn/9bt5ZUnqOAL20dFmUY+kLsl6ksZP260U87cqN9hPVPDP3n8lMDZxMonFJBTeVQiUQ/gMzuZf1hhqJWZdSIIxleLShAqATBKxkvxQ8A6o34tW1Nt5L+MHJUu17KtetJISQk6vnMD60HGfdvebH+Vs4gpcOmqESB4UCEQiEDERz3DFDr9sctTH2bQ7W4OAr8hTNL4ynk+FAVl5NO0i7rtxBk8ubwFunq+qPIJmb0clDj8FtaJ4umtomOWLjNbMuXUYya0AXWwa+vpaE3HYYh3hiMttUkT5WYe6oJZ7+xdN9iw4hFcbhU32IfD769vK3RFBxqq4fgP0ufIyWXivgiYFOmucqZUaN/UX5nWzCszmHQ35lBw8l7exuHR2dTpBzTsN4kk+nY8RGCZv0VYNNVxCUTYh72OSo1Y4TUJy35xLhD6VJRdHJVPdYvVQvJNcXIrYCd/y51yBc+yHg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(346002)(366004)(39860400002)(136003)(396003)(376002)(451199018)(109986016)(38100700002)(36756003)(66556008)(316002)(478600001)(66476007)(8676002)(6486002)(2906002)(4326008)(2616005)(54906003)(41300700001)(6506007)(86362001)(83380400001)(6666004)(6512007)(66946007)(26005)(186003)(8936002)(5660300002)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8kPWPq+zKmZgIS1+rPWInt9rPLFJpHrtUq6fwgbMxlg370DGjAkfGOCp41Ijz1PTVwYpECd3sqmgg9VzEv7HCDfKWU9kskMdNLICAPMJH7aj+NykRz5Vi4BvaKjzlZMqGybroaohN5+DYgu66QcGf9APOlOWspIDVVJ8L/JRgAyKWNIJPSRD5DyBdA0L5V+PxByU6h1Ex1YS+UTGOhOKD4G28m9fesyWwkj8rBRilGg2DQrNm48UAIJNcCZRzezRD2UcZyj7gr3MVTFFJfOENtiKSVlIxmq280IpBivCMFe3F8XdekWuTSs1j/LEMgGTrLEjoLQFZxRGWRUL+DPgWuImZ9tvjZ3Drq6tLxBumYJau4aY5eUKpmkjvXeQ/LYBFMjed4AjvhdFfvkdXG30BNojRHhyVOzVR16GQ5pgT7Kw7f1iphUsHzdJRn+iYJ8dNftqubqa7n21tyU+srpz+Ro7B5RE74pjHhVu5CbbjTdPDDWUYNY92IFwPGjpWI1lr8GNVa1UBew9nPNB2l+R41HDcAVe1dcITZ4MTZMOHnV9o8F9fD7dfRO+FFAs+HEe8v7tdCg6fpKygN4+tiP6k997pn9irPh8Tq5gqph9v3lagg40c+ybA+fsWpdHxgnkrUqQAi+tHZfGSTZg/ABrV3Yoblad5FiJQrgG26L7kDDh7qgtYBR0iezIOyVGR4zroDrjfj76RVlS40PzMgq2lpYFEHVSKX80J1vCb4/zfFeC5YqlIiZ/uCx/1N+UYBCpp/CFlD2L3zJc2NnczZd0lGrSEZvajrBXpfbcaiUUBjkknd08V6ZuUYaQiWw50t/0AG5S2f7Yj5Yb/ga89/sputBUBmSBqbiU8qr5nRVt6PF989x3CPHHZ0NKgUBSm9iX0iBbmaMoswCw5rG+kbUHxN4GI1wuEhjyRUFs/XBL+aJMCPB/OOEf3K82kZ/boqV+BUDij+IGflRIgvOCGIH6ZLsjqyDc+X/XyzMveae7IY6INNGfNV/ErFHv8qG/wWE8G1M9m+lNp51oDIUnnDvG1yueR56qm5ZM9qL2UEO+2zJ+5nwW9jNNRQHObAWrkA7rwWv6GEusBGAfG06kXVUJ3lWyBEyVouaNku0+DRDWH5M8Qu/qMTenBecS0/nw1NBTuaOn4WfJMy50Pwz4sUErll5xDVyeGnOwW0a80bP+wD2U/3+aQIznUdCsdlGaBhbWvzvaoX51+DDU8CR3n2KUJY1lInHAPfDVfax9w71e4NZlmMwq1MXqjHiS9IBrgVePReHXMTs8MZ9VUINUrNXyjaNrWy2+lKWe+wzLkpwCwVFDsxovlzQuhuFhLTWUgGz/PeFqruSsrwRXn5MS6oioHl1+0IaGuGzbGaYoA+vj2IGFvHJG9ZiKwvVsDAmT5oA10y6MWcv8Cte2VoS0pjgxiDJbrcxoBPF59R0yFTJWrvgwZ11tdjLSP6JzydUoaXVqxT04zZLb+Pggh5F7WBjmI93sYk/ftVPBnXI0YDPcrkkxrV8dBUJE912kOBNbO8deaYO786BVUyhhaxJcPO6zR3tqZbthD4+GLqLu12uyjprxrMps2j7v07BydOELStEF X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4d528839-b74e-4b3a-1ce5-08dafe4a6b91 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 20:34:41.3303 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6E2Qug/TBlAPF/tLnNbdo+6p7qB/YHt8mTDHOs6MbCciQQakt5wL87UjbeiBH/GX X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4220 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 0E63C40021 X-Stat-Signature: 5xinsoimcr9yq71oxefzeh7g1r98pn15 X-HE-Tag: 1674592483-182320 X-HE-Meta: U2FsdGVkX1/9jdxjqJhf0OXW8rfywPEYqfT57XLBgyDnyp7osgxA8V1f8qIDVjoEjNFBdSgp0LhWxotC74mmpY5yGnJRe81i32wbb8RSgliAhlTWbjU3K6Nx6YuMG5YYxwx+v2ihJGxry+q0649Mv26P/tGBKe0fmVayKaIyvI5mlsKF7foWwmKZCHY52foXsFfn/53TgXqdplcuChEYE54oX8Awfxcy62o6Opungdwfb62jeHG/13G/dbVWeCYiy/aUuVA1KZTqBs/LKdWTtp4/9FHKUT8chEeALQf9+C3mvp748fX1GKDj/c3ob6urdrlRncEDJ/RtKHdFsrqaG9SbkoKH4lH6Gos+fbe3Vf5JONckNqibautDKc9/gvs/GiPvoJLdX5JKPJcqMk7jvPyzRvs74z0VUrrD2GIhtaWO/l6a2yfyccZ1fEqWKNsLZoUB2qRon/Zg2Fx48kU/sKlXftL+RG2Wy6bFtZxCkbkZCGUUSOXmDSeAhKJOufXTlNNH/3lm5JhETE2Oxmw7O7I12MR4ZzCRpKdViensiGR8udGsGAdaVLBCvOoN0LoGs5lt8eOCKBoPlbUJBeeUzJ6i5G1r9SlFQUeGKqpPuDmpg3LPOEKwMljS4AXq/ByzTttNKcCSk3RxpXP3fIv6PbflogEiaiIk0LfcVCPOgE2Y2Zb/F0x75Ey7PDLpdZUqQzSbVolA9lzVkgf9Ffrsx2Vaqxuz6HUtJehhMNN1OJ8wltlMn3XfphM+sgTyXGx+GfkholFV0N3xF/XaYWhVgZ1X5pYZFXhfyCCH8cgiyCUVFmDPIXKyHnRYmDO8JtdlZT3EjFIAHtS7Ry1Ypb0HiSRt4CgNqYzMgcMFea3ghuqruOzzdyOZTKDeWCSxe/5na8xN0npIRazXdU33iljz/oMXh9u0JZraoPFvAJmhw4xIlmDChaj+zGB/KgatEbsCs+b8/tlNhx7/nHDoPSp xObQUwzP PzqCMsa+EBoS/rBBbIzUZn8RUU7b5PWsZLMHNMtIcLqIG+rYgQ4Ps54fHSxaQWoiQqQ4cSLXUqNq8QVSDuMLWrbXrICva+rSPWcW0ccwM3gz+NNDawzUmcMXfZlA4k1tBHr3bCRw2Og5mKcqyMKreIevl+Pb5MxnADE25fp+YQDVsZ1Q/vx065vhPBxuUld6CvB16TWjvs72Txf5yYZf/Qoyn4xC5/vbywx8cAxLWcF+aSQaHCp6fv+Tw22IlaRZp/LaCEEdQCBLBDo1+2L5JBH3VHPdTSu1q968HnYgfqR8Kh69obyyAuqBiLvfiTl9WThpA/nwFtrGdNTILMG5JIxnapA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: get_user_pages_remote(), get_user_pages_unlocked() and get_user_pages() are never called with FOLL_LONGTERM, so directly call __get_user_pages_locked() The next patch will add an assertion for this. Suggested-by: John Hubbard Reviewed-by: John Hubbard Acked-by: Mike Rapoport (IBM) Signed-off-by: Jason Gunthorpe Reviewed-by: David Hildenbrand --- mm/gup.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 7007b3afc4fda8..a6559d7243db92 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2200,8 +2200,8 @@ long get_user_pages_remote(struct mm_struct *mm, if (!is_valid_gup_flags(gup_flags)) return -EINVAL; - return __gup_longterm_locked(mm, start, nr_pages, pages, vmas, locked, - gup_flags | FOLL_TOUCH | FOLL_REMOTE); + return __get_user_pages_locked(mm, start, nr_pages, pages, vmas, locked, + gup_flags | FOLL_TOUCH | FOLL_REMOTE); } EXPORT_SYMBOL(get_user_pages_remote); @@ -2238,8 +2238,8 @@ long get_user_pages(unsigned long start, unsigned long nr_pages, if (!is_valid_gup_flags(gup_flags)) return -EINVAL; - return __gup_longterm_locked(current->mm, start, nr_pages, - pages, vmas, NULL, gup_flags | FOLL_TOUCH); + return __get_user_pages_locked(current->mm, start, nr_pages, pages, + vmas, NULL, gup_flags | FOLL_TOUCH); } EXPORT_SYMBOL(get_user_pages); @@ -2263,8 +2263,8 @@ long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages, { int locked = 0; - return __gup_longterm_locked(current->mm, start, nr_pages, pages, NULL, - &locked, gup_flags | FOLL_TOUCH); + return __get_user_pages_locked(current->mm, start, nr_pages, pages, + NULL, &locked, gup_flags | FOLL_TOUCH); } EXPORT_SYMBOL(get_user_pages_unlocked); From patchwork Tue Jan 24 20:34:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13114776 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2DD3C54E94 for ; Tue, 24 Jan 2023 20:34:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F11C16B0071; Tue, 24 Jan 2023 15:34:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EC1426B0072; Tue, 24 Jan 2023 15:34:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D13E26B0073; Tue, 24 Jan 2023 15:34:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id AF99E6B0071 for ; Tue, 24 Jan 2023 15:34:40 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8552D1605FE for ; Tue, 24 Jan 2023 20:34:40 +0000 (UTC) X-FDA: 80390845920.08.FE68A52 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by imf12.hostedemail.com (Postfix) with ESMTP id C3C3E4000B for ; Tue, 24 Jan 2023 20:34:37 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=iB7+N5S2; spf=pass (imf12.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674592477; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Meg0lJjxcm1jvtu7lAs5qCLEBZuPFTvTPYQpefKTDP4=; b=7892Z+g15FWJQD/JHW8C5vNQdejgJTJp/V1bhuTz+NIuhVWQEAwxu9orBetjIrU1uB8OHv rJZBeUdJWh2IHMaDow3PmZTiH1T2zBDWPppfZAquTUbDAJDPnpDHfsmKjSFrkexBKXscyu msve6FLwE6KnoNgWaMlboR5hhnDWBj8= ARC-Authentication-Results: i=2; imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=iB7+N5S2; spf=pass (imf12.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674592477; a=rsa-sha256; cv=pass; b=LyrYWPx713sUw3Vq4Nr4kTUjS6HtHtyW+OmuvWr0PCRK0ArdJ47TnyPVhc9QFRotHj3J1K 0JrT37ClU8LSGKCXsIfI0jNmkujG766KgdIiRaHkHz68ZwETgDNQytB7jQRLbLmW5KaX4w MWgj+0otBf72bZOXW7ohcHpgzNckxNg= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MqQez8qlvVZINRF8QI7GGHrTToXVXg8bo/kOX1Ff1oj0XguI8g747TEIyUw/aHWd9Oel8Co2xZ1tlUQHnORs91/2ROW1EXnmmOE0LW/ybNjmHimdIY7064MMtEZUjCqTHxNBtll4S2YiP9LkGpzEJGQh1geZDrYIb2Gp6FV0Ol10cfQnLzXGU64Z5e85WclET+gI8qxfKiIYoHVaN5H0GWxUhSdBmx41hIdiDeYA1HMN2sDR9gTERfyHqMMidXBJMHOm9CQcjSQSNfTGMzIc/it6pvN9vUipXeDt7b8mxJiIIyENhkxSDeD8QqlQH5gUuOvE+x8MRyjkqr7Qjup1AQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Meg0lJjxcm1jvtu7lAs5qCLEBZuPFTvTPYQpefKTDP4=; b=YLiPQ9/j/lpVGiCfRDc5bAWr6HJNmZfOU0ENHLqHIF8XZBvwrttmJYxeFv+rVquSUTBIFSGtCn96kh4sMTwyhR2XvhAPsQ++cNSDGj+dcI8GCzWRxkKD9llhOWgHw9gURIdrde+IVyki29T4/cq01Nf9duZV71f0YCqsBuvGCaxwpo5lJIv6c44DM/rUxC6ir7TKYvXBCEVKddLwxlbubJwK6NXyCMZQH7+PtPcwKW0BuEfYCvZBuQsxHwdQ7Wcz2m6Mxt7w736nFeK2zQTh1llvOj7V3YRpbPUqNNT2w3xXD9tjDl+GcGPu+iAMMeishmwJG3rStxS9D1gNV4FFaQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Meg0lJjxcm1jvtu7lAs5qCLEBZuPFTvTPYQpefKTDP4=; b=iB7+N5S2gBouyX1w2JsI35SXJREEvUzYbRXrjLRHZBtEn9pEPQXoCUi2roXpEko+8n1aFhpwoqQK32PVGOduYfT2HSOjvLZYyPEEnPPKHNR+5lFlf0KWEIP2b1iPqjzPSk2Kv62mXsQxjd1oxp72FcOY3uxNYqTBEQ8V8Sx3hiFqYIoaE2z8flm/Hms+qRL9slvV12EUVI4hVDONSQ+P96m+AhCxzqGBxCT4TXyDLDGkSEKIwUhpuOOAz9EsP0hsVp20oGe6K7Fcw2XUxMqxuH1gUMQ3sW/T5r7W3S8JDYNabVyUmdIFS5Ubl1pwaOLEZixC2XeBHQbfkFXI0IFYnA== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by MN2PR12MB4095.namprd12.prod.outlook.com (2603:10b6:208:1d1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 20:34:35 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 20:34:35 +0000 From: Jason Gunthorpe To: Cc: Alistair Popple , David Hildenbrand , David Howells , Christoph Hellwig , John Hubbard , linux-mm@kvack.org, "Mike Rapoport (IBM)" Subject: [PATCH v2 04/13] mm/gup: move try_grab_page() to mm/internal.h Date: Tue, 24 Jan 2023 16:34:25 -0400 Message-Id: <4-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> In-Reply-To: <0-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR02CA0003.namprd02.prod.outlook.com (2603:10b6:207:3c::16) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|MN2PR12MB4095:EE_ X-MS-Office365-Filtering-Correlation-Id: fd2a6b75-948b-4e06-a249-08dafe4a67f6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: g4t4XlXLUaaON/kUSJ9aPKnbGV9Wq3OYrslAPtDZIytisyX8NxefccaAmF9qbLbzDPqrDlmc49uhZrncSosPGK6vF+bf6MHfX0AfJxWBOMr8FCOmkNk9mfNZ/Rvz7Ghbnnf4yIZKzds0G0IZdH+bAHqNlIBpnM9ATwp1rwdU6HUb7+Z5GZV9FjUjyZtDTWRA5O3HqFPq9t3g3w35ilqxfeDZ4xyb6j41jGznR0cDCYxqKgqzF9kp8ynuBsf95+gR98EOiZnZzeBzkwSmwk0UOO76xR+ii9lDyNwWvL2jt9DRv3KKHCd0imRPavP6fTQ3vIEXw06fImz4ABsgFoZL72KybuZ4Y+m7ZPUSFHsgY+sXV1zq+pFFMR18oukBScHjZqr0JWsVTNwFbJPN6Ld5E8X5VfKrn2/MWA1T19NAa7RRuyGl+TP6PQfM4LRoC2Ou7KX/TlBa0RjKf2OjXUNRG4jIpDIyOi72ddZ36F5MWJLnuCqaOh8ef1bXwEaH8FGHI2hODmd/7BwQDiOeNDNCsO4RMx00HqanjRI5UAdBaJGibUrlXnQJTIKOmYfn8E6sHgdeSXAUEVAS2yf5sar9kmirXAHP+Asvw7dwq/Sl1c1uq8qTLrDB8tTHrJe194kVDEbi8868jTLEM/+sc+XbChkmE0eXAuO38O6ab3KhVF3d/7ja2aew6PiY0sSd1zFSaU2kvK9z+cyiojAvVZrb7A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(396003)(39860400002)(376002)(346002)(136003)(451199018)(109986016)(38100700002)(83380400001)(5660300002)(41300700001)(86362001)(2906002)(26005)(4326008)(8936002)(316002)(6666004)(6512007)(6506007)(8676002)(186003)(66476007)(66556008)(54906003)(2616005)(478600001)(6486002)(36756003)(66946007)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: m82yTrxOaCC/49WbSowqTKna6AcMzSraM/7fQ00RM4bzsjwARqUj+p+yzHwA2uOV2R9P+zRzXYeJMsm1jnXPyQ7na6SVDM2rviZ3vzXqznK7kD6Dw9wmATNFTPvYT3eNFDpCCIhCAOMn596FuXWBi7YPoBewvgF7JXcuXArl0JTTu2inEdSbLmSaC8zfKBywBFWr1G7n4QV0mlcw1pzuOVSp/YOsSJLCXrCTHnHU5oHSxpLeQqFz7dpn/eFI0sN05Z1KXyZ/4VYONWzCJcxgnLpuknUEmV7ed1YkqUsZhoma+47Srx2pxz27XdaYzwF+JZD2oLcPm/Hx1CRKMebqx3CnczCWfk5SZFMVmHvoShcFPcgMvoKtIw5Bz2Y2h3pDXOTt9wn0dT21fRFbsuVIXSSH9fvVK8OkDkQdNxCDWJ0f5g4kmOUoJYPYHc5e/3/GDcoIt90reHRb1FpmbHHdGriymoxbDRbeUQpkWx1zj/gBWn81I1lbQW/Mipyo1xsmAMDrTLOLQfPuHwcLYxfT65V0ETjZuWnp+YS6T0SZekphfX/yslKsUQ4iPs/X8UaZ6xCRP+VvYyn43ODHU/C/NHOiU/5Iz1ZFDrtXue2lv810zSj8iBtGsP9rwHjWYdvC/pTL0PgWWz/YKX7hYkzgX6aauZ7PkxcwH7irNfojYx+m1FFVO9rFpBzlccJqvHUuWbR0tI+Aihvow4OevNeIkeuvX864keZahYTWagiAGz+kNfYcW+VpQjXr+u/wZP6/zPMzyRk7BmwqbyFvEI8KAqERQbWYudnL72ZLtYWriminvxq+5en6faOsP0MM3VJq7/IZiODnRaWorprIZBQb//vZSQDUhrMBThIDNWglGwOQdYQehYMz8loMhtOsSGIiMAO0JML6WBWl0//jygL94lXWXqkbJuufyBS1XxsGl/AKJemhAvrUHcUJUAAH/uevjcfeYhezwt+ZFpaNes0lHdjoGRznGD9aDYd872GeR16wCDASsaqXApmpL5PgsuKmvmL/l1xoRUJhlZipfk63zkRdwJQYY9KM+CM+DdDukpa5NsSuPw6wXk3knp3/YynjaWw3+PLGbOLR0J2alGstuhxyHkCztTYcIRcaA0jKmYB8ziRyejNcgyDdYKyZDP3NGsDwgxJhhFKkv60VRL/+iZodAkeNSpKa04Zm8l1GaBzmAFmngeJHGaup5EpeG2wMlFBsdrocrlsRZYKbCWsvtZ2KBT1BdtS5cCkIAq4vh8JM4xwn7qpEdMYfawDAou7QSX8cEXCXIA/1uPt+IWXhvLQuViXb+57rrMvBGEwElwkog4EXL3K0+RZtQLU8c5vKFb+3rpQsSHWUFOnbYrVgFqLFlORxbP7QBwSvIcUOgHL5mlJHXbjluGWSa25IqZ9HzGXgJvUJVb4lzHwFcly5kGeQRPqJQWjhUbakHgf3iPdUvTkQBmOQsCUvcQQDkG1W5f1bC8Aleq0QSGR0Or2TzKDnDln9TUs1IYfy5plSaJBVpLVwDC0YpKrqMmBKBELecxGxDGk9Gxq7i7HqdJQ3qlVCGVwaDY1ImLa1Q7m1c2Ph7x+ed3rSFLtecMsZNtyQ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: fd2a6b75-948b-4e06-a249-08dafe4a67f6 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 20:34:35.2827 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nXhOu4Y9aaYh6oCZyu/7w3hPeh0jApcF8iwBPT/rybBrHiD/1zYQyo5fivhqxWXU X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4095 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: C3C3E4000B X-Stat-Signature: ockd4kcxxddeifbyehw88nf893fe9ga9 X-HE-Tag: 1674592477-167163 X-HE-Meta: U2FsdGVkX1+N4YZjhQ3/4fEpVXHBUC5+Wlml3tHg3UWUh2+uBxa/LxRdzqWyaTgmXyFaAHElJMgaNudPRH87r26P4mvgRkC24PepgM8kKOaib+QXTrOSfN6OzyP+qyFFNpa25hY9wqHY/pmEukjo/a/O5dH+iUWkatOy1q5zqD+fnk7gPISZhDuwV76gN9NNdRTVdJkNv+YtcwzFfbA7r4C4/DCNUl0aZtMgryWQrRCRZc7iIwFjp+Ckivfi6k5bxHdJa5tFIIDykyhNpb1rouwOT+3mM8NEBzWiEqKdmWa+t9fUd/JinEuvpncOKAG27RaMDtKH6iARzAjHI8x9D91pmbnE984uvLwBnJ0OAc1PlTjUwDr5LnpAt5OIteV0B6nmYUN8Du2JU4hPtcV2QRwD2JSGCfWSye1j1KVRuOcFUhiSUbspt/+Mo8CnNarXhOhV2pzSMfFsQ4LKiiku2LeSc7hbLe/kTUXp/0KABPoCh+BH/mGYzhHAVOvIffVgQaUtzegC6nJOTPu6MtGFzzFl2PUKlm7NgBkS2ozlgf4UUi6xkJuLDrhn21UpdajOdD3dg+Fl1Up+FYehNSHqtshCRNxtdRCVeR7KahcQQP/fSwwJndeO4/Ml9cFnIf0T+nODC6ftSInX765LteSMf3yEeeHJB1InAhLHrGhG9nb0Xp6ZWiPxNQTVTIPReS/yCH6Lb+1I8ansVkmuQy7vSkNV1fh5NgvYL1PGlAHHyO0Yuv3jkhAKVrWttYfxgUcW1rf/ZVpAgHE6F9yOesbAuzbKKuxBHv8GBWE2CMckWDosUIWGPKBxmhyd+9j4yUa7bNMqCBMFcvY0jckqkiJ7WpxDn8dVNU80aE5z9UjHfLv7U8jfe2LqtsbTboJCf5K5h0nFArge2rHu7WDhWL+4ukPEm/BthWr4rKBOv/CIBfDZiMeboAxdw9snR9lKK7W4zNfrJKZfwnz1kfuT/iI EEYsBZIg VURmEub64gop4/IWuBLdbB/gxwIg8VrbP2JHKgmXlXb36mKwNav+X4c4Drm/3lIgcKAA0xdUtoSxiPFpmKO/jRNoS4JRu5WvCIh92dLs4GiozOGLNerFW8Dug7/RrwUHaNGWUWWSdM7B7rldHo4dMMydirXUJGiNvWJaNECnxpA6veMDx+jFYTGYZaSxn7BVxruyvUvLNiUOv6IMOxE2jdCa9nevkXUzeEiTyA5grN6cc10nmHrabiuyVW40tFL856t5en7CiKnH5BHFvz3FWUKdQhrXVXeBt55/I X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is part of the internal function of gup.c and is only non-static so that the parts of gup.c in the huge_memory.c and hugetlb.c can call it. Put it in internal.h beside the similarly purposed try_grab_folio() Signed-off-by: Jason Gunthorpe Reviewed-by: John Hubbard Reviewed-by: David Hildenbrand --- include/linux/mm.h | 2 -- mm/internal.h | 1 + 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index c9db257f09b307..dfc2a88bc4a8ed 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1173,8 +1173,6 @@ static inline void get_page(struct page *page) folio_get(page_folio(page)); } -int __must_check try_grab_page(struct page *page, unsigned int flags); - static inline __must_check bool try_get_page(struct page *page) { page = compound_head(page); diff --git a/mm/internal.h b/mm/internal.h index ce462bf145b441..0f035bcaf133f5 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -852,6 +852,7 @@ int migrate_device_coherent_page(struct page *page); * mm/gup.c */ struct folio *try_grab_folio(struct page *page, int refs, unsigned int flags); +int __must_check try_grab_page(struct page *page, unsigned int flags); extern bool mirrored_kernelcore; From patchwork Tue Jan 24 20:34:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13114778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 732CCC54EED for ; Tue, 24 Jan 2023 20:34:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C79396B0072; Tue, 24 Jan 2023 15:34:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C02A46B0073; Tue, 24 Jan 2023 15:34:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A2E046B0075; Tue, 24 Jan 2023 15:34:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9290F6B0072 for ; Tue, 24 Jan 2023 15:34:43 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5005F140394 for ; Tue, 24 Jan 2023 20:34:43 +0000 (UTC) X-FDA: 80390846046.16.7770245 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by imf12.hostedemail.com (Postfix) with ESMTP id 7B5EE4000F for ; Tue, 24 Jan 2023 20:34:40 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=E3w2kYJe; spf=pass (imf12.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674592480; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7UTXihNrqSj3uJ04YCq9D4TPrSqn62F7/CDsp9sAGlw=; b=y0kXvozLeBYbXk72Zn2EqSm34y3r90eq7lmlZFGysflF4V3v+OPfFVyT5OF8XhJLDFCqvJ phwNsDKLQcSiAZzBB5FdC+QCCDQNPjuk6yqOUxDwKW5Tu/O5IX1VmyBihUff4i6j4VjNsi Rfldbvl63eLlr+DyHpbEkOHPZzHp5ss= ARC-Authentication-Results: i=2; imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=E3w2kYJe; spf=pass (imf12.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674592480; a=rsa-sha256; cv=pass; b=wmxu1TCrMOYCZngQS7BMKGJZTV71+2oRw9ww9w9da+ZUyxQF0jNG4ZXW2UZSII6PZbxwxR VU8GU2fvSFTVKRTXNraQRdwhlzCALkqXzyYLMp0qBm0PYpx+gFGzqtWsYVpgAc9oyI9Kc8 Rgfnc3RPnMLt0kIZ/QOjiUW5XrSYdgs= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SnSGVakYo9+JJWq3Bppbo9/8ToEMYpk4I7OXpBAeKdfH/EBGcduW2hB8njOlp7Wdgo7x6MBxNJcjSpYrc90cGPsoW/aVdznQKaz2xhp4y3FK1suOkkbkLyX2kXgRQaZ8D8GGLatqZQ1T6F9SEorZXs+xPQpP+GDha1HCDqwr+oCAoeB+qEh8JDwOwXu2ucVLlP6iKjwnLXK9znogn6CfM6bADEUB56T78Y0TbEy0flxarZ24oKWg4vAw+s9tHRbzsYD7zIbn4/NQ5om3ySqy3ciSz9VPWJCfsqZyOWhjuK1GA66DjUiDNaSCdAjwzg4VqDrzjHBfWmPwNXglB84tRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7UTXihNrqSj3uJ04YCq9D4TPrSqn62F7/CDsp9sAGlw=; b=cfbP+irtDlgJclhs4dtxhJ+alT743VQMxu6MaD+ukB3W+VtQGG2dbe2OcZGuKi8jvaTjJkiLzITp/4LnY7kPU2lY5FWgTw+jmeja8wrfl8YlL8cOghgnsVAhEQLwBNTiWWvchIMsI8LEBYVMaNyvrVten1lO/RIVuOI0D2tW2/zzompgsfVzN2cRfCcQ6N5hEa22ue5uS8ex1MxDzF1rzZ5rlHs6JjrYc+4a9yeYp/BVDwhD2M8Ln3sNlY/VOix5xZ7otg1yPbwxUi4LzYg5GMKpzsCJjo5AbplvtoeP/8TTLPuFFWgUizFSvhOl/AQG+OT/t/jUIycdyGmPG9kQ+g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7UTXihNrqSj3uJ04YCq9D4TPrSqn62F7/CDsp9sAGlw=; b=E3w2kYJe7THdGmDtExLlmElQpIJTSJR5g43mQfxxuo4mI8mfh1NmSQqs3LWy4qRjLV5M6NNtzUHGCaiBw3BhNzXWoggLr25pQibDWRf0MJz5855D/skgCLhndIxOTilqrA5f/LZBUMBNUAkRr5CH5QNM6D8pToDpewawyT+1DDENVtcAz/tb0WC+yWHvDSl3r4xs+Qmo6xlgFYvKd75ioYEpXysMTpCQSWKQOlR7b2H9ZQxl6TRt2/D4Wq0iZUAn2MviHjfKGJTpAN6i4jOjTpQWaUnoauUckKHumOU9DXAkh3IoGkYBBxqNbybPjBVYg3gOiBRsSBHbReUbffqQqg== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by MN2PR12MB4095.namprd12.prod.outlook.com (2603:10b6:208:1d1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 20:34:37 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 20:34:37 +0000 From: Jason Gunthorpe To: Cc: Alistair Popple , David Hildenbrand , David Howells , Christoph Hellwig , John Hubbard , linux-mm@kvack.org, "Mike Rapoport (IBM)" Subject: [PATCH v2 05/13] mm/gup: simplify the external interface functions and consolidate invariants Date: Tue, 24 Jan 2023 16:34:26 -0400 Message-Id: <5-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> In-Reply-To: <0-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR02CA0075.namprd02.prod.outlook.com (2603:10b6:208:51::16) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|MN2PR12MB4095:EE_ X-MS-Office365-Filtering-Correlation-Id: a2bc0992-1b88-47f8-7836-08dafe4a67fd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Ttszt4PGOPl5JBg5kFpm1KcBLmUWlMD0cmZi43sR3EUoUPQjb3gEOu+fxMPHMICHAunx3IZhlW3v499k/WwqrNoVQc9UWY37S52nFpl9bnRjrAiQxx4fFKynlD/MKeJShMC+p0UwG2RTZnEsyiD2Cs5s75rJyWKansVFFXL6P3Z8vvo4u/0w9fLMVn5OSP5kO2CD6iGwuuS0A3xA+YPxcg59BBDLbTHXuiUk56TfHV+g5lvpgLBGCy4UHQXq5zO7LOqb0ferzDC1Wj0WozdyCidacpsIuPS01Fsx4xdBeLh8roKv8xZShQxdQoUqjCZm8lwFzY/eP/9KkDOoCu5VNeng4XJKOYM4V1UNJZULtfEbaYDAmB8uhzTwqDhTxZNaqbZuTE03OJrI/ECcwFBu6wMasdNx84EOTCVlBEOVG3Fd4cVTfN5SPfVRSN8l76mSCE/YnGMEuqsibe4T/biZJLj5CGbWFSgXqSJiypZ+DvQn52JAXgOvHn75u9Sq8FJnR9MB7nYBBQ6a1yE9LMpHWeyDU3oM7mI9L/QmEiEP3KTcDqtZ60EAy0Czm9ABILlPpzURIIDf+kpUIrvjfgPSnbogY+PS1v1E79f0/a2z8bhi23IBXSus1wDiUG1rpwvV5Eiacne/bICDOb1+cm4NQpODD6UMf1ZojZy382mWNqchnZMogbxAk+M7VJzOTTwqJcwQIBm0KKePrlJRy7cquQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(396003)(39860400002)(376002)(346002)(136003)(451199018)(109986016)(38100700002)(83380400001)(5660300002)(41300700001)(86362001)(30864003)(2906002)(26005)(4326008)(8936002)(316002)(6666004)(6512007)(6506007)(8676002)(186003)(66476007)(66556008)(54906003)(2616005)(478600001)(6486002)(36756003)(66946007)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: kEnYp2gf4QU5e2XlnGk2PuRC8F6TVZjlNDRcCUsQEXvW8jZChdCzbOTmClsdnLCDdh2tgU3PJ94Y35GBM4SHvYYP8ofsvWP3XFwNhOBZDlLpVfoAcBaAKt9rVmUSxz79MjAHzqqnNUn2/g2GvD5/Sn6WQ5tD9UtNAFJNGtuaSo2v40KXjHoHLV1YCO8rVO09oUNIdjRyxJrU4fChlIFjqjeiP+jjcfuJ31SQ6KRHU5CW2RHYoa0D4TDhHRhQJrfV0w6Z8/M8B9HryhUc4xIwPY+nIJcb1gbdEOarkycTeqX9aphjuBNOM+b5oRYqoMHXnHDHYasE+1LiRxKkiXpzug4YMrFp3MZjWvghDmsk8xJVhHI5iD90Uc+GjuErtXr4Rp4sH3uDyI9vLBUFF0+XaaL6XWCvr4cxN3hlyHtrjeInRsgYR1ONRCIO7cMLnnIgJZV/lY9cEDqoUxoPVkcvyTiU4xreCf9YRW8NT9Yll3IPQiXsTuiOcJg69xr8I7rnZP/4oQvN+D4eta5HO+XJeAkkU5DIacwjiyIk4aJSODBW4z6QtfXGw0dOMvCZAczdRAE2pBOejBllKMQx57KeRB6NfsSQcZjcG573LyE9lhE3bGGVJsPKwdj50cxOZJJsEp6Id6F8XBS4HsXuLSjHTSvqtkL+8OisxgceHkQFHQHqClB33IBVpf2LZ2DiUtWEu9idXDZlU7AKAcbu0BR1MW9xXmOjv6OIYhxSewngRWeOj7g+aqvu2m3mZjFwJrHpKqLAMoLsR02h7GOINDo7f9k8bokkCYMru3OMH3HWAbbDXTek60jHZ+037JPZ/jrhf1qOiIGzHo9feSkM6Azv7D3CMu7qZD7P7gZptv1hcKMUW6MkoFqDeAJYMdIBPvtFUP0MbY69fCi6rKnMEct2GTXa/4MY8SGn1lp2SwqrP3oSoZD6P3te0ReorNMxdO8GYoLtJ1mN4/OwpJROeNtlqb2DlMegZbKAh2RkjGlvIfklDLrm66hoWyymxnY2w+PzWWU48pLYmfl8gkMD9gX47mzPX+arD72b3YYm+1DvKsO9NfvYvLCZXLREe056ntCudOHH3YSDBAEogRSXyvv0usQ2/zEjtH38SePkb0mSdzycpXpOHiVrRC3x5sV7cEqrTpwZ6A2QH28pdELTr1oUA5OYGrKGkTVrTCyMz7i9v8uHnmfN7tTDNDoh6ikajURludACRXU0jTmlN6H6IqNvoxZfBbb3FHyjPEOyUUuFQASwIn/e3wajHQnpxPVizKgHHFawFHxJTpXJe2KAxlafDNAWpySA3sOqrobb0t2wShRiIvPep/3YlpPuNNKdSZnGBFqkTU3ZsUtKbBaxxME+9XSdl11Bt6uloLLnMmY0hYzlsDdcqWxcZ1Evik6xp1cDs4122MJeqbuX+QKERcS9D6Ba6q6p7OTDOLXo0hHzfa5EjhD/PrjDDF2pwwalkOauufwrKBOFklSKmwGMeAVdGzgTBiYUHUoojOyk5IeEK00bUNJAKubebbHnYxD+MYsjhKpeOcP5W53CAH/Vn3x2E/2NTVP51A6ifDDYsQTNgLTc8fBYVsm/mMMAKkJAJpjo X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: a2bc0992-1b88-47f8-7836-08dafe4a67fd X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 20:34:35.3295 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Sn0Bs0DdCu+cVmYxfZEcpUg2Z2ZPtnyJXoKH3tm56r1E73S2/eBY2qg1fyAsMU5F X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4095 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7B5EE4000F X-Stat-Signature: w9xqcur391s9b1ztgtipoupcy1qkmm1k X-HE-Tag: 1674592480-90266 X-HE-Meta: U2FsdGVkX18uu7MCNf7rHCjIvCMeSlzBQpWUrOe0aBHuTdAjyFvYLyStL+U5HoqfP5VoHcQF2f6qi8eOkuLMpP5SGnacoUsqgxTudMr5JN3wC307nXWBMd/ZTx9M9QboOAzC0d8pkqM3Iz2ib317CGCO+qA4i2JwQFC7TQfrGa2zij+8b34+X5DwZYuP7GG6Cw7jFUbiFDgMKTmd0TH/5ve6gqvkwqVPwlVe1IADgBwLh+K7WMKPzkDDPyy/zqDkudkazfSXsVy4okeVHnEc+x9UCDtDKxmmIGtR+nL9aRcd3g6z7qCRgEAOjaZPNKt8cqNJhyfZgGthTdvwXWRqquWfMZqEPt2GiS2jfHmE7ZqhSYewocv55Z5rQk8gk7KLzRF7eKB31S2IeXHaqH8c7WXDf3XK9GqVPRnRrgoSIbOCrWyHGITMHpoVhNBjxiLNZMA85zwsjGtK+9BEGO0dn5n5Vc57KALnea5Fh8bYh0tp/k4fXW4QQ7Jj80IUP98KSsvZ6fSm1gthJnUL/M2iSROzxHZDfuS5y5S1PPndsqdomPLp+N2lIw4KM3wEcS4xOPHO426es4Gub9InTjbQriiWthv9JOJrLV0ktDYO6F2sPylQMyq9GNLdzTWnwvnJxMpoTBQt1JBDTJmyy3mUz6aKRGYGB4GimEiUKeLM3RMkEZkZ4T80iFAoymXfFGOOw51dvwn7ty6VkCZnyeRlPbidK3yruntDq2FetmwDyg+FLEf5RAxnDeszvBWe/vH29Z8caxlTfaPOW+TeYiluzDIRfR6qKi48mCjvB/p9C0caC18ZbbaXQ11SGMj0jd+Te9N3zZD5IgKwJ5F+EPR0AdP2NerAxiGv+VTqaFw+SkDTVbT0bU/uLEtRcI1yKuhCiFKdE/A6dBV/EnTMxNjz9r8DyOvFHVkxvG39ifVer+3JzodG20tVL9rqpVkx2YjtJt6qwPWu1lg4S/HTVD4 M3Rp2mmx IGhGy4i4dMkwdNoJSKcroE1IPN3BWrpVkzSXk6we/RKaazsWd4hYjngqcHPC9JBgj/5TqcUo00107W5/jNJecUi+Ne6W/1V3eCj2LiCSR//vW2SdXHLtQTqI5Gjq//f0QZ/joO4OJi4WuMw01zdR1WUrexwkQTWGcGZdk6x+9hfsLQU0OZ+PBco8JLx7RZIbhgD9V2tRO8b2r5lIkEzQ7BBM7iuQD0diVCUIxtXU3R0xKJXIXgLZ3P2hVYEkbc1JbHyc5y0xjjlyC3ltMsqjkHvhUTvSElZVELwRq9rXSV0cGW6U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The GUP family of functions have a complex, but fairly well defined, set of invariants for their arguments. Currently these are sprinkled about, sometimes in duplicate through many functions. Internally we don't follow all the invariants that the external interface has to follow, so place these checks directly at the exported interface. This ensures the internal functions never reach a violated invariant. Remove the duplicated invariant checks. The end result is to make these functions fully internal: __get_user_pages_locked() internal_get_user_pages_fast() __gup_longterm_locked() And all the other functions call directly into one of these. Suggested-by: John Hubbard Acked-by: Mike Rapoport (IBM) Signed-off-by: Jason Gunthorpe Reviewed-by: John Hubbard --- mm/gup.c | 153 +++++++++++++++++++++++------------------------ mm/huge_memory.c | 10 ---- 2 files changed, 75 insertions(+), 88 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index a6559d7243db92..4c236fb83dcd3e 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -215,7 +215,6 @@ int __must_check try_grab_page(struct page *page, unsigned int flags) { struct folio *folio = page_folio(page); - WARN_ON_ONCE((flags & (FOLL_GET | FOLL_PIN)) == (FOLL_GET | FOLL_PIN)); if (WARN_ON_ONCE(folio_ref_count(folio) <= 0)) return -ENOMEM; @@ -818,7 +817,7 @@ struct page *follow_page(struct vm_area_struct *vma, unsigned long address, if (vma_is_secretmem(vma)) return NULL; - if (foll_flags & FOLL_PIN) + if (WARN_ON_ONCE(foll_flags & FOLL_PIN)) return NULL; page = follow_page_mask(vma, address, foll_flags, &ctx); @@ -975,9 +974,6 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags) if ((gup_flags & FOLL_LONGTERM) && vma_is_fsdax(vma)) return -EOPNOTSUPP; - if ((gup_flags & FOLL_LONGTERM) && (gup_flags & FOLL_PCI_P2PDMA)) - return -EOPNOTSUPP; - if (vma_is_secretmem(vma)) return -EFAULT; @@ -1354,11 +1350,6 @@ static __always_inline long __get_user_pages_locked(struct mm_struct *mm, long ret, pages_done; bool must_unlock = false; - if (locked) { - /* if VM_FAULT_RETRY can be returned, vmas become invalid */ - BUG_ON(vmas); - } - /* * The internal caller expects GUP to manage the lock internally and the * lock must be released when this returns. @@ -2087,16 +2078,6 @@ static long __gup_longterm_locked(struct mm_struct *mm, return __get_user_pages_locked(mm, start, nr_pages, pages, vmas, locked, gup_flags); - /* - * If we get to this point then FOLL_LONGTERM is set, and FOLL_LONGTERM - * implies FOLL_PIN (although the reverse is not true). Therefore it is - * correct to unconditionally call check_and_migrate_movable_pages() - * which assumes pages have been pinned via FOLL_PIN. - * - * Enforce the above reasoning by asserting that FOLL_PIN is set. - */ - if (WARN_ON(!(gup_flags & FOLL_PIN))) - return -EINVAL; flags = memalloc_pin_save(); do { nr_pinned_pages = __get_user_pages_locked(mm, start, nr_pages, @@ -2106,28 +2087,66 @@ static long __gup_longterm_locked(struct mm_struct *mm, rc = nr_pinned_pages; break; } + + /* FOLL_LONGTERM implies FOLL_PIN */ rc = check_and_migrate_movable_pages(nr_pinned_pages, pages); } while (rc == -EAGAIN); memalloc_pin_restore(flags); return rc ? rc : nr_pinned_pages; } -static bool is_valid_gup_flags(unsigned int gup_flags) +/* + * Check that the given flags are valid for the exported gup/pup interface, and + * update them with the required flags that the caller must have set. + */ +static bool is_valid_gup_args(struct page **pages, struct vm_area_struct **vmas, + int *locked, unsigned int *gup_flags_p, + unsigned int to_set) { + unsigned int gup_flags = *gup_flags_p; + /* - * FOLL_PIN must only be set internally by the pin_user_pages*() APIs, - * never directly by the caller, so enforce that with an assertion: + * These flags not allowed to be specified externally to the gup + * interfaces: + * - FOLL_PIN/FOLL_TRIED/FOLL_FAST_ONLY are internal only + * - FOLL_REMOTE is internal only and used on follow_page() */ - if (WARN_ON_ONCE(gup_flags & FOLL_PIN)) + if (WARN_ON_ONCE(gup_flags & (FOLL_PIN | FOLL_TRIED | + FOLL_REMOTE | FOLL_FAST_ONLY))) + return false; + + gup_flags |= to_set; + + /* FOLL_GET and FOLL_PIN are mutually exclusive. */ + if (WARN_ON_ONCE((gup_flags & (FOLL_PIN | FOLL_GET)) == + (FOLL_PIN | FOLL_GET))) + return false; + + /* LONGTERM can only be specified when pinning */ + if (WARN_ON_ONCE(!(gup_flags & FOLL_PIN) && (gup_flags & FOLL_LONGTERM))) + return false; + + /* Pages input must be given if using GET/PIN */ + if (WARN_ON_ONCE((gup_flags & (FOLL_GET | FOLL_PIN)) && !pages)) return false; + + /* At the external interface locked must be set */ + if (WARN_ON_ONCE(locked && *locked != 1)) + return false; + + /* We want to allow the pgmap to be hot-unplugged at all times */ + if (WARN_ON_ONCE((gup_flags & FOLL_LONGTERM) && + (gup_flags & FOLL_PCI_P2PDMA))) + return false; + /* - * FOLL_PIN is a prerequisite to FOLL_LONGTERM. Another way of saying - * that is, FOLL_LONGTERM is a specific case, more restrictive case of - * FOLL_PIN. + * Can't use VMAs with locked, as locked allows GUP to unlock + * which invalidates the vmas array */ - if (WARN_ON_ONCE(gup_flags & FOLL_LONGTERM)) + if (WARN_ON_ONCE(vmas && locked)) return false; + *gup_flags_p = gup_flags; return true; } @@ -2197,11 +2216,12 @@ long get_user_pages_remote(struct mm_struct *mm, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas, int *locked) { - if (!is_valid_gup_flags(gup_flags)) + if (!is_valid_gup_args(pages, vmas, locked, &gup_flags, + FOLL_TOUCH | FOLL_REMOTE)) return -EINVAL; return __get_user_pages_locked(mm, start, nr_pages, pages, vmas, locked, - gup_flags | FOLL_TOUCH | FOLL_REMOTE); + gup_flags); } EXPORT_SYMBOL(get_user_pages_remote); @@ -2235,11 +2255,11 @@ long get_user_pages(unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas) { - if (!is_valid_gup_flags(gup_flags)) + if (!is_valid_gup_args(pages, vmas, NULL, &gup_flags, FOLL_TOUCH)) return -EINVAL; return __get_user_pages_locked(current->mm, start, nr_pages, pages, - vmas, NULL, gup_flags | FOLL_TOUCH); + vmas, NULL, gup_flags); } EXPORT_SYMBOL(get_user_pages); @@ -2263,8 +2283,11 @@ long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages, { int locked = 0; + if (!is_valid_gup_args(pages, NULL, NULL, &gup_flags, FOLL_TOUCH)) + return -EINVAL; + return __get_user_pages_locked(current->mm, start, nr_pages, pages, - NULL, &locked, gup_flags | FOLL_TOUCH); + NULL, &locked, gup_flags); } EXPORT_SYMBOL(get_user_pages_unlocked); @@ -2992,7 +3015,9 @@ int get_user_pages_fast_only(unsigned long start, int nr_pages, * FOLL_FAST_ONLY is required in order to match the API description of * this routine: no fall back to regular ("slow") GUP. */ - gup_flags |= FOLL_GET | FOLL_FAST_ONLY; + if (!is_valid_gup_args(pages, NULL, NULL, &gup_flags, + FOLL_GET | FOLL_FAST_ONLY)) + return -EINVAL; nr_pinned = internal_get_user_pages_fast(start, nr_pages, gup_flags, pages); @@ -3029,16 +3054,14 @@ EXPORT_SYMBOL_GPL(get_user_pages_fast_only); int get_user_pages_fast(unsigned long start, int nr_pages, unsigned int gup_flags, struct page **pages) { - if (!is_valid_gup_flags(gup_flags)) - return -EINVAL; - /* * The caller may or may not have explicitly set FOLL_GET; either way is * OK. However, internally (within mm/gup.c), gup fast variants must set * FOLL_GET, because gup fast is always a "pin with a +1 page refcount" * request. */ - gup_flags |= FOLL_GET; + if (!is_valid_gup_args(pages, NULL, NULL, &gup_flags, FOLL_GET)) + return -EINVAL; return internal_get_user_pages_fast(start, nr_pages, gup_flags, pages); } EXPORT_SYMBOL_GPL(get_user_pages_fast); @@ -3062,14 +3085,8 @@ EXPORT_SYMBOL_GPL(get_user_pages_fast); int pin_user_pages_fast(unsigned long start, int nr_pages, unsigned int gup_flags, struct page **pages) { - /* FOLL_GET and FOLL_PIN are mutually exclusive. */ - if (WARN_ON_ONCE(gup_flags & FOLL_GET)) + if (!is_valid_gup_args(pages, NULL, NULL, &gup_flags, FOLL_PIN)) return -EINVAL; - - if (WARN_ON_ONCE(!pages)) - return -EINVAL; - - gup_flags |= FOLL_PIN; return internal_get_user_pages_fast(start, nr_pages, gup_flags, pages); } EXPORT_SYMBOL_GPL(pin_user_pages_fast); @@ -3085,20 +3102,14 @@ int pin_user_pages_fast_only(unsigned long start, int nr_pages, { int nr_pinned; - /* - * FOLL_GET and FOLL_PIN are mutually exclusive. Note that the API - * rules require returning 0, rather than -errno: - */ - if (WARN_ON_ONCE(gup_flags & FOLL_GET)) - return 0; - - if (WARN_ON_ONCE(!pages)) - return 0; /* * FOLL_FAST_ONLY is required in order to match the API description of * this routine: no fall back to regular ("slow") GUP. */ - gup_flags |= (FOLL_PIN | FOLL_FAST_ONLY); + if (!is_valid_gup_args(pages, NULL, NULL, &gup_flags, + FOLL_PIN | FOLL_FAST_ONLY)) + return 0; + nr_pinned = internal_get_user_pages_fast(start, nr_pages, gup_flags, pages); /* @@ -3140,16 +3151,11 @@ long pin_user_pages_remote(struct mm_struct *mm, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas, int *locked) { - /* FOLL_GET and FOLL_PIN are mutually exclusive. */ - if (WARN_ON_ONCE(gup_flags & FOLL_GET)) - return -EINVAL; - - if (WARN_ON_ONCE(!pages)) - return -EINVAL; - + if (!is_valid_gup_args(pages, vmas, locked, &gup_flags, + FOLL_PIN | FOLL_TOUCH | FOLL_REMOTE)) + return 0; return __gup_longterm_locked(mm, start, nr_pages, pages, vmas, locked, - gup_flags | FOLL_PIN | FOLL_TOUCH | - FOLL_REMOTE); + gup_flags); } EXPORT_SYMBOL(pin_user_pages_remote); @@ -3174,14 +3180,8 @@ long pin_user_pages(unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas) { - /* FOLL_GET and FOLL_PIN are mutually exclusive. */ - if (WARN_ON_ONCE(gup_flags & FOLL_GET)) - return -EINVAL; - - if (WARN_ON_ONCE(!pages)) - return -EINVAL; - - gup_flags |= FOLL_PIN; + if (!is_valid_gup_args(pages, vmas, NULL, &gup_flags, FOLL_PIN)) + return 0; return __gup_longterm_locked(current->mm, start, nr_pages, pages, vmas, NULL, gup_flags); } @@ -3195,15 +3195,12 @@ EXPORT_SYMBOL(pin_user_pages); long pin_user_pages_unlocked(unsigned long start, unsigned long nr_pages, struct page **pages, unsigned int gup_flags) { - /* FOLL_GET and FOLL_PIN are mutually exclusive. */ - if (WARN_ON_ONCE(gup_flags & FOLL_GET)) - return -EINVAL; int locked = 0; - if (WARN_ON_ONCE(!pages)) - return -EINVAL; + if (!is_valid_gup_args(pages, NULL, NULL, &gup_flags, + FOLL_PIN | FOLL_TOUCH)) + return 0; - gup_flags |= FOLL_PIN | FOLL_TOUCH; return __gup_longterm_locked(current->mm, start, nr_pages, pages, NULL, &locked, gup_flags); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1d6977dc6b31ba..1343a7d88299be 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1042,11 +1042,6 @@ struct page *follow_devmap_pmd(struct vm_area_struct *vma, unsigned long addr, assert_spin_locked(pmd_lockptr(mm, pmd)); - /* FOLL_GET and FOLL_PIN are mutually exclusive. */ - if (WARN_ON_ONCE((flags & (FOLL_PIN | FOLL_GET)) == - (FOLL_PIN | FOLL_GET))) - return NULL; - if (flags & FOLL_WRITE && !pmd_write(*pmd)) return NULL; @@ -1205,11 +1200,6 @@ struct page *follow_devmap_pud(struct vm_area_struct *vma, unsigned long addr, if (flags & FOLL_WRITE && !pud_write(*pud)) return NULL; - /* FOLL_GET and FOLL_PIN are mutually exclusive. */ - if (WARN_ON_ONCE((flags & (FOLL_PIN | FOLL_GET)) == - (FOLL_PIN | FOLL_GET))) - return NULL; - if (pud_present(*pud) && pud_devmap(*pud)) /* pass */; else From patchwork Tue Jan 24 20:34:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13114780 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D674AC54E94 for ; Tue, 24 Jan 2023 20:34:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CF0E86B0075; Tue, 24 Jan 2023 15:34:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CA0736B0078; Tue, 24 Jan 2023 15:34:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACA1E6B007D; Tue, 24 Jan 2023 15:34:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8AFF86B0075 for ; Tue, 24 Jan 2023 15:34:46 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 5829AA06D8 for ; Tue, 24 Jan 2023 20:34:46 +0000 (UTC) X-FDA: 80390846172.18.850DC72 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2056.outbound.protection.outlook.com [40.107.100.56]) by imf28.hostedemail.com (Postfix) with ESMTP id 9F447C000B for ; Tue, 24 Jan 2023 20:34:43 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=oFK08f9Y; spf=pass (imf28.hostedemail.com: domain of jgg@nvidia.com designates 40.107.100.56 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674592483; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/sDDSmpcu61aiordYuKth43RrR76IP7M1Qa0wICOl20=; b=BId0vEe10cDfn1qbOesUEwXJtPiLBGdHwW3Wf9FV2hIOLbqsXnogN+dzxDM/XJcwNv2d9e fDuU6D4vFU1LDUg5B8nl4TL/pEA4WQi65AAjQsHfp2aG49iokdobpJvJ/RdxE++s6wCoC0 zcCVzWUy+c43QsVBrLsqnIB3chmzHUU= ARC-Authentication-Results: i=2; imf28.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=oFK08f9Y; spf=pass (imf28.hostedemail.com: domain of jgg@nvidia.com designates 40.107.100.56 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674592483; a=rsa-sha256; cv=pass; b=tfj1Timmcp1nnSbvpcCZ8NPRLClIeQjwCPvYWUmH7gTbtIADFD1rDWPn1HGX+yS/J7h7iI 3NZ+IemgmpC+RyZdEMwrUA9TSnoXwGfCZzxbbTVB7t8mGIq0WqcUMDOcXLSbDce5VM5gmv Eu5Dg9/Nn1/aboEb2MYcIQE46vxVhpY= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NuhmVXN+z76QJB4eR2mlpskFMr2u3vgJXB558SzmyucHLT9/gPJU4I5+F/kLXuZLJYEWSVc35I1xqsxAhx/8Ts1aIA9YJoD8UaRgGAvcfirqjT6CWqBWFPskgN9aO5iP2U239K16wZ70jnkiyzzWT68cqKGKffbsoLzKrgwY6H7atDw8YjVcvsImudMyIAM8kc1UcFY2oobOKDSWvkEdMtQV6s/56K30v8sOuChw5Nor9JOkO13AOJF8rbIpS5MXab/ihxzZSOir4+bnzOssXEnPgK+97oDKjD/xElk/gBCIsnfwz5jpfrh8opHqfEbQaazQOiJ/r2Lyv1Ih5jXXkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/sDDSmpcu61aiordYuKth43RrR76IP7M1Qa0wICOl20=; b=avBpzNwgbSRktnCN4VZ1fmr65x1UIWEypwxMI2PAsIb6z8V/vMRXTMA8djK0luJXvL1vhFwT1YzRi6+QMzVHyHF1gfFwNYxmrvI2m1YKXE1K5yG3z5SUkahVqNRMKY3IXDc0fjljca2ZDDxdqrNUuOPZno6/J7htLTxXvkWpMdCjRO7KZJ3WeSXdTZzceujZDDEt2gJ8jSd9mM+vwLiSWA/wTVKP3sVI14AY+v9vI5VVbnMkiMvOacsdxAKLjJYR9Q/lh4HDd+905i9xwmObfEXfUkaKPLqiX53ERHuzZaf+g1QGEBQVB9WXUy9qB5TBDI1W5jzb7VmL6B62Oo+hsw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/sDDSmpcu61aiordYuKth43RrR76IP7M1Qa0wICOl20=; b=oFK08f9YGnnIBqTUe44Nz7jGbjh90xB0kCtRT/ERNXcbseAf3BiX4KEbtaN6Yx6ErB9e0TCYlAnGninl8xcFVYF2smxW2HXlDM0rAU9oeRlEKRM7SkJKDHNlKh2HuZwRDqf2Gb0MlH+gE9rUF7tDlJW00Ng/Fes/crm/LhKOI/s0stMQ6h360HBWOpKZsfm2nHb7R+IXUWd5L5thMKj1MWdz66A5j7ldj+otOkqy31KK6kn5uBMyltOgW3ygj01GtTL1t0Uons697tmoCgQD3W6HC4yWTtcVPJCWGZqAk3FCtwWbHAnYI3zQSoPZsx4nDfwcw2oTZIYAT8SGUNJOyQ== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by MN2PR12MB4095.namprd12.prod.outlook.com (2603:10b6:208:1d1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 20:34:37 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 20:34:37 +0000 From: Jason Gunthorpe To: Cc: Alistair Popple , David Hildenbrand , David Howells , Christoph Hellwig , John Hubbard , linux-mm@kvack.org, "Mike Rapoport (IBM)" Subject: [PATCH v2 06/13] mm/gup: add an assertion that the mmap lock is locked Date: Tue, 24 Jan 2023 16:34:27 -0400 Message-Id: <6-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> In-Reply-To: <0-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR20CA0062.namprd20.prod.outlook.com (2603:10b6:208:235::31) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|MN2PR12MB4095:EE_ X-MS-Office365-Filtering-Correlation-Id: d91c0f31-1bea-40af-d667-08dafe4a683d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WpP4g0QoWTvAW1R7Fdy8h7Bxn2ajyqE6OjpLsulYLHBRgRVYQSCJAN0lFkUfPAgFj9DkDqYPYxuna2tvTrgJfDetpR70lknLK/KdtfGv2gLR303xxackF8jkjWdZaFKiWXCXKqTMqnCHW8XcqRFzx85OsfvmsDsuEsnTzHO9gHiboEBqNwD3B7jtFi+RPQOSxALXV9zInj/3xQVOz8C0BPSbbnVSPa9VOLuE89aF2hdMkRKinDN1eX85CPHaVU5xFBJ40sma61DrK6oW2ukf2JkRwhiLMgCsKLhCmOX/XBHpwV+z6rLCWcexI5n8TSSSYiPfWHNJu1umdx83jePH1U8lTPNybUHVRkRg5uC8ipK6c/v5FCI3n+PClv8V0rYaBC9F4VP1VCX0JzeQwINPP3o3Gko3jWaUsgAh5vQuKVhrqYlE2DAazqXt6uxQOvmvtt5Byr1J5PgXTrc9OPUYM39mKitwE/ryiZ5uquHWu53bvH08snd6f0nJlVcC61TvYK1ILYHIAX5Q1YRe/3LYDoKV2ZTbi4M1Wawv2K2SQkFBbnAS5F+Fvg/BdO652j+KS37z4/9wQU/peI3QWXfJFsvn06aAuqsCLgSx3DsfMF9evW6h4O4b2t1BLaVV796hT7pV+npM1y/o4j06gcOWKrwEvehX0rnrJRHTFDyN2VmTmhNUdMR5fbCCSBLbJfstElkeRpDTNlPoYbrJ2Y37BQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(396003)(39860400002)(376002)(346002)(136003)(451199018)(109986016)(38100700002)(83380400001)(5660300002)(41300700001)(86362001)(4744005)(2906002)(26005)(4326008)(8936002)(316002)(6666004)(6512007)(6506007)(8676002)(186003)(66476007)(66556008)(54906003)(2616005)(478600001)(6486002)(36756003)(66946007)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: fV9rKw+404HQIVhqt867L1WVwLRfz9wrS5quT8MLpfILnpu+LP8LfDMkwuCaW7sIodEK9wIR8uk8fsZng7RybJTmH6+lt5kJ2bpLVVj6iH5vFpRIxq1ym0DiRQszhC/NsK0IJyutcOhqnEoH6DtBUmOuhuC91x2Ifc15MSHwzYObqQ/g4D08wJBuqyOkU9U2rmLd90XB1IgvMwmv1S5Q5qUIj51NQnuV5qGl1nyfpBU6c+SbW77yjfa9RLwBRA6BuSsv8YQTp8aIdL70y8cdc6BWdgUj69RfNMqpKwLTvUgjwmyqrS9fw0bruUgPdmEGi0P1exEtzxgEddjhutGgcO7rT94LJkBPLtBbMr/LuaITrruuQ9XNWaAkHvHWLltAHhfLuIUrV8NApSzGymlNOdsUWL8xWZtJ4NaY143RFx5ZsFnPBKt4dAjjk2vN8UPGNdxHwBCvVsdzvdI8LWCmywpA0UncdlGj2MC4vuRYfNc21RUJJ+WMnd5Ei89ewnl5zZjlnSpIMkRqAmxq4EJHA/6eDwAeaWqMy5EYF0foikUHrut8OU+2hRacPRyX3ogObJOMXZpFnr/0OPoKjWcox3RMgNHg5T8Kfshp2eRzO3CMXeV8H7vU2WbUGaSN0lCyOoTt33XWziVJQmKD+8DfE8ahEsPjQPHVAqAaAjCqOdnJhD2y2V9S3vTkBboth83CbtdEdOTcLKclrvlqy/G+nzDzCSPM8Rl7ycKC3RJUuMquAjHUA6eJdUMhvGDzvpidok9oTftFRlej/ft9M9IeG5W4wVA2Nx3thwWzDBAGpJ+QDkhOY3wpvWoNtJQrHEUs2d7Opvcg8EcIaSMadDdb3aNMBmx7LWDmHfF8zoU+2k7M9GGGqW1bLinjMXKYxxPLxnJkYv+D/G+OuvfAfQqMo9z05buyr7wM9aeDY5x9uoKlEx7CTxQL5CGB4POEuDNjCn902YUgv68sZHbESpVH7l7u8yeJAWAdXjt2bwhDSFCs9isscexZLrG6KpouJEIXfo7YvbCcvzFWEseV5Vz+KdsojJOLcjIndz4SNq67dGi2viwwDbfeTnBnqR3+AbuCww0ZqFzGRc508na1lxbOFAznUGmkwaaeiza0+MUNYE6/84WZVEDv1RLZYr4ZEf11US8EG4kV+SkiWhDcPFM49LEflXM39l8//GgNb+msKzPx3+ypjOdak2OwwB894PGbMj3R1XFk6GYoL4D1oTsSoaeIj9FbzadmV5N8btObx1qjk+vE/isYDKu1hZ2NwBtBQ+a3RsbNg5/9BtF3K+LpzSDFIHOUSmVEduVRn8OujAhnEClmTLONv5AiQ3A1wIdYiXG6z24jprUhya/vyI+2MMmVsWuPNkzt7EB8V/GWekA91SBwIfkoF9WQdq4e9JHWlyDPAQd8KY+q8qC4DE9t3WPkjjmZvPtQ9iSig7kTb+353XcsnA3IvwOs/isNw8vvHu6Al+E+DjSqKI0Gr1yIC4gmQmvljI3zmrcp3Fyt8+9hAX0tT1vaKxFGZzrvy6BimrnGSquK+Ya7ZWhNq7G0h2iYZz1Mn+0+TyEl8A9HIPRgNH/jPovgY/hm5t1I3y2w X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: d91c0f31-1bea-40af-d667-08dafe4a683d X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 20:34:35.7513 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ClnXV9G2e3E9vWUSp07J4wBQK9o2IKq1I+FTkqKWZqD+BSUXZ/Fn19l3XE/HPYYY X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4095 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 9F447C000B X-Stat-Signature: 6kd7a85jc7bfzu1g3cyr8dkdtfx54ma6 X-HE-Tag: 1674592483-574454 X-HE-Meta: U2FsdGVkX190fhdhJq2dX6ICKk5L/qv4nfqC4vrzLrqJF3aQOQ88qA1rlE4sC7s/VS+lX/i37BzQyqztdwHlSolqAmwWLAD2cfmWpm/5D3KdPppLwocEiNrzfgSBEQ3ALEItxED1T/vtbfqqB4L5+KF2f2x/gSFCRxo29EXqPWgaBB7pdT/mp6O9DMKarZxpoPuQmmCyQRkShNe0DzDQKhe8JfoRYbXFVeGYp2WaUhJxwcFZoapKhvclzJ6OafEnwTlsrtsYPdEEST1SoiBQNh8vHmPSgz5J/fH7R6VcOsJrtX2DcVLiLbJ2WmjzaZLPmsAE8Nik35HrAZ8I9+gmhWmJzNGNDgR46p3A8BpIyAOtkQ6XaA+utZU+H3l0A3xXlWRkToYKMpdSrmduc/fsrtSotDE7H6YWaqk4zT0WPt6igiU2cYBhiaxrjz6kTD/RXB+y0EvTPf2bsVwyVUz9t1RcCjzljuKzcsg4e0NiTDbH/O0zo4kEL+oeiOZspeTQzoZoEon91Uf5Gtc/aWK8FYPSLtG5GJMdyUs5pO46p3MqiQL4kRimgBtlD0wNJtSP6I6XLL++3kcsCu4GYK0Rj1/rD5LTuJWBPPvKE20tvRs8lyF3ccH3lSYewwz1X1L77sDmH/2GQG+yekCy2YVuvPVpbJh00poRdbx3TmfJ/uSlVoEPKqscSnA9S66vdeneFhT/kpwpNr2bn5+rQPNFHCegwlhVpkMa1LkXEfW2hRKKUelhdf2y3hOZQAxX2b+gHtZnkQUkBfoAl6ERVvMJ2gWkwovGdEWGsCnl7jezxiLiJQC3dUwAHn9BF+tkANqxZdaqPn3gg76WEsC9qpWqllGbHq1gdQdte82MT0ONmfkDnixvtdV/oQvEQCLECucOkpypGdZLg22MAz/PQZnFRWpl2Wyhuk5NpKDKV226DHpb6LgCH+l1ceurR9uNtsQ4xWZSGmorMp+qhGfH3ww 7adLM548 KxQM48F7K9fHQXlXyPF2CJu9zQccBiC6WlR7wPolpB98cgAKeiTElNbgwAmQGeudc6cKvR0+tPAQieFlSO7fM4xTRJ1FPibDQ5yPjXSVtXvvAMQEvBFIocJbuV5LLXXdg3H1cE/SG7LRiaycfCBvZkSQl8xquBYalo5aPQVO95jzX2Lcn4kvZnt7N5KRNOkQ23+toqU4gwA4JQeDgodHFSXnrtN+6Kl1wo6D/wEGd5vTYtoopV35wPL12Ilhus0oPS6eYuBIy7HzdGAb2oYAJde6yUyQ1vsLnBnYG29Bne0oP3I9o3+jaFIerWg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000061, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since commit 5b78ed24e8ec ("mm/pagemap: add mmap_assert_locked() annotations to find_vma*()") we already have this assertion, it is just buried in find_vma(): __get_user_pages_locked() __get_user_pages() find_extend_vma() find_vma() Also check it at the top of __get_user_pages_locked() as a form of documentation. Signed-off-by: Jason Gunthorpe Reviewed-by: John Hubbard --- mm/gup.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/gup.c b/mm/gup.c index 4c236fb83dcd3e..de1a5c64fdfdcf 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1360,6 +1360,8 @@ static __always_inline long __get_user_pages_locked(struct mm_struct *mm, must_unlock = true; *locked = 1; } + else + mmap_assert_locked(mm); if (flags & FOLL_PIN) mm_set_has_pinned_flag(&mm->flags); From patchwork Tue Jan 24 20:34:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13114785 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA614C54EED for ; Tue, 24 Jan 2023 20:34:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 550DE6B0080; Tue, 24 Jan 2023 15:34:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4B93D6B0085; Tue, 24 Jan 2023 15:34:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 157056B0081; Tue, 24 Jan 2023 15:34:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 012366B0082 for ; Tue, 24 Jan 2023 15:34:51 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C28E640714 for ; Tue, 24 Jan 2023 20:34:51 +0000 (UTC) X-FDA: 80390846382.07.D762B25 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2056.outbound.protection.outlook.com [40.107.100.56]) by imf28.hostedemail.com (Postfix) with ESMTP id 1F4DAC0003 for ; Tue, 24 Jan 2023 20:34:48 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=pQRgju8p; spf=pass (imf28.hostedemail.com: domain of jgg@nvidia.com designates 40.107.100.56 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674592489; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Hqh32U3L0qcQ6RDnBFAgQsDim+Ys7n/Vd5L4Vngc468=; b=KLoAi7s4O1+vvILddAHFivPFigb2ShhOi4FMklX3FfyqX3Ol37xTknhDuO86HjF9Xo8z5J IkY4PZEghT2vWFN+hdh5/b/cCL+Eoz5DNRi/Nc0mPyOe9TssTep8W5q10+L8AuP/6GREgV JX8/2SYI69Vz0f42AzOyUkoKaLVHp2U= ARC-Authentication-Results: i=2; imf28.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=pQRgju8p; spf=pass (imf28.hostedemail.com: domain of jgg@nvidia.com designates 40.107.100.56 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674592489; a=rsa-sha256; cv=pass; b=Z/rL9Phw+XgdgvcvUUKabjgwGCAzq8zeYfhAUcKwVX8rFvCi1B5noN1P9/Y2Sjyw99m0R/ PEFFLCcCHjq1PKAD48O/akUqVLcV5M2nwd3s4+KNL4pG1ILO3QGweySp2emuCrNcLMsHNW 8jdmf1GE0+ka5lIVpL+K3gt5Wki/wYw= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YVyfG555qPlvnHeJETScOWckp4wPVvAjE33CFv0ujzleX7/HxC8emxMvIII0rtZNfHg1CI7Pub340y5EOnwVKMTAlWHngOT2P9A30YkDa7tZcYtzHqeCza3baixMwvOUVM9XybgpHuFjZzYvsFrzbuA73Z+CoTW28WsQEE/FBXVO0y8dLHP0K55FXhXGJwizSzwlEGjFXn6tH7c4gP9puwXr9sPSHu+vo/n3ALeVauzdhRqiiD8F6Rm5OQL6hDHKk32T6mg+POXZLs6hWv73Ax3jVddGP5Phe3MJo03bDhSG9dQuc01bla1XFP1k5hd3ZZveT9HgveZA5vj782NcjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Hqh32U3L0qcQ6RDnBFAgQsDim+Ys7n/Vd5L4Vngc468=; b=aHyMdnmVjr98pYkSnI2IhlwXEhhjRLLqExQXP9F8CYruVDD7NIhdpqPiGsYq3wn2R4luxWbgcFkyuBgcSNyBY7UCrh4dIBsxQVaGLetl46HPszp56ZpLqmqiTMzKXozHRGkB+WoA4N42k/PfZN5oh5NY5iv5Iuq04U0BUlhvr2n9qXSmphaJv6AQWqDX7F9RsuqSInNiD4igX59booQ1yqhdjTfbzH2nL9Y3TcUekzIwfjtpPbhTqvezvgKOjrQMhfBbb55jc+8Blzu5A68gFT2au8n3nk2yTnXbJkpy9aq8ciUY0oLckUYrYb0E9MhDm/QGamFltniN/lEZqhfufw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Hqh32U3L0qcQ6RDnBFAgQsDim+Ys7n/Vd5L4Vngc468=; b=pQRgju8pNVS+dnpQsCbMBM3EFATGMwdhnz/lt6dYOYJBPu47y42Zeq3RZFfPk4txtLkX22a2WwVzu9uAgfsaks2D/vqlfd0FUmEUs1/CT/0pPTw1eRETdlRCOHUX2uqbuJzBLA2jgR6RkDCS1S7RRT/gs1+XwLI3loDv9/F29gCq6BO4TJzdodIvNcYw3IVyhWIlhnk2Hdzq0hV4Mt0aW1WvdHtFlz/6jyOQgSIHxNVTa+3uB5i2Ao9yRJN+7qP3FoSB7A375qaTrxCoOXmdFHONxGTCFZLOomkatWEim6YBSGSBN+EF8Glmf3T4EVaCKpEVkau1MaelQeJ31ZluYg== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by MN2PR12MB4095.namprd12.prod.outlook.com (2603:10b6:208:1d1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 20:34:38 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 20:34:38 +0000 From: Jason Gunthorpe To: Cc: Alistair Popple , David Hildenbrand , David Howells , Christoph Hellwig , John Hubbard , linux-mm@kvack.org, "Mike Rapoport (IBM)" Subject: [PATCH v2 07/13] mm/gup: remove locked being NULL from faultin_vma_page_range() Date: Tue, 24 Jan 2023 16:34:28 -0400 Message-Id: <7-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> In-Reply-To: <0-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR02CA0081.namprd02.prod.outlook.com (2603:10b6:208:51::22) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|MN2PR12MB4095:EE_ X-MS-Office365-Filtering-Correlation-Id: 6f8a1292-e568-4413-81d7-08dafe4a688c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Cj08k9LkhP11QtOv96Zl/f5Hs9Um7LqfY3M9EtcS94/4lM06XoI4riLWq0oS4D4j+niC4FdTVJZmDPouJc8QD1z6mIPTtoFuLe9y3rdsfSUrUMYODHdLg2HZyrEJLxsuR6s6q3kWQEKC8fPnIhzMks/oTVWBP9Mjys3wQf0NV3YWR3ygQw1aDg8Xs/YpA2BP8u50yneX31SLG+TxAveZZwtNsZgXXGSlzC7pWrwKqGcVXPTZSQa+sbYpgyzuumiCui9irIy9sR2MoMyM8npedYa6njVkOAlFnBqKCT+mbDwuUCDdj2nqlI4qPcYBRbdDA6lAJOfabikKOswKvGAiaxPU7EzsPVmO9HoeT9N0grZ7bRIO84cynwu2X2kDaSQYoMGVxLykbd+544rHfN4kqeJrDVh/UTtZ2K5xtUt3HLwVmU7eceC8sT8HgIKoULQ8ytWLJMHHEbn5ZEnd6oldF21CBizRrA5BXuRmALqZ6bHpssR3fFCbZwOcrbyWHNCS38U6dg2FQbNzzW2RGIrbcAA+Zko13tIJpDHsL511d9BtMKFPYxL+B6+8iSs2aq5eWUmK5Xb9jWecjYpgXut6aqgMVEIMUBRTe9q/UgjT1pXnaGKUu4tVQE3JKFSnQeoCAHqkcMr58M9kahAsJeqHWadBgxx6U5b5iR33KqbKZqdeVoZeAWjmOJgxmrC/YzIBEtgfsQp/xbAUW46/4GoyPg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(396003)(39860400002)(376002)(346002)(136003)(451199018)(109986016)(38100700002)(83380400001)(5660300002)(41300700001)(86362001)(4744005)(2906002)(26005)(4326008)(8936002)(316002)(6666004)(6512007)(6506007)(8676002)(186003)(66476007)(66556008)(54906003)(2616005)(478600001)(6486002)(36756003)(66946007)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: GgeA4mhX7e98hbfGwZdRg2aZ1ZbzQVPFBPP38kOlBi1N57KfZWW7WW4v4hUfzv8ChXRKqci1bB8geQrt6gnE3D8jzpxi9z2XRzn5KK0Z3ChycuszWLoItgTt4FrHq/KtKKWGoR+L0sGp3mngdi+veSZ4dZBWZKx8siwUYS37IkYdGfUzlII684KRqbp2kPZ+rKP8oq5Szd/pWgM6tEFdmc6MLOENi995mQALF4gJqDElmJ/4nIyVo+ZfE8VDlVdFnkXOLSZlh3jzRTBn4Ueb6M/Jws8wIFYsbYQ/Gb7ZSHFdnLlW0/NmuR3T2VfcGuJZhB9+I77MPu2XXykhbNU1vnuh3xe3ufWLQh+NtNvY3GcHPIjlTqLWKQm4pDH3ZJ6IhqUDCf40REVPSWLAUwcbSOtCu29DlIfwgl20A7CIoc2PxM3ooy4UAV2WDV4sA9+T8aqkurZnpLRx0nIDGX43V+UXWYlDZXi3A2YKjD+/hIQ5/2KXwWci389bZjWejrSXegsLOlT5nxH7iVjkALsPEiCAwI+3BnzBinY1LAvCukGBf8OcLcV8OGzhq5Yks2jAVxWS+cEvI5t68k1Wpic7sRZdhEYA0U4OkNdCgK0bFKhmpVo6RPE2Osq8OqfrRDh6KY/IEx0A//NVy157paxvwKDeTjGdstgPYeb92vCZFQ6NzLQ6u+hkbeDVODssbwysjLeVZN+wcdlcp7fl/fzK/u0kBVggulPY8/T39TZZdtQWbkDv3oCwxzAt6BRQ0FNy5N8wRoMjiDlN48bjKLNLlFY8JYRw8MaKCH2N5xeASPo/IfrqLc1/ybR9q2J8ZKxKytLXSETjA8OpmGu4kBWnx7gfV7TfPg0p+h8Hle4qryIKE8I4AKmmzwYEOgu6/Na6YKmPx6FxLXEJIuZNHMhZAbOWNuoKMP11YfV49/Rr7LwJhzt4bdGF1178gE18xT88SI++PWwQG2XUE97Op5mcp5BgKA+V/P4uCl4Mzv8UDy49NXh+h36ks1AsJ2cgEd06Hq5xqlJZP8Dh0VGkV12gaZjlw8sBqyifL+X5XBGUYxZqt92s25oRJhUDc5B6qwXNFTeqHHvGe+OQktKJP7IuUnUyHjInMP/qNdD7zu1JNV9sLnepcZ7V4jnmsfXUiwpJXRD4fpasjpc+D40cRxgtsky4MOPDmhcb+eh4dMFGWmh0ZnzkyZGDr9Cu16QHetL5qbVo648I1F2/3KGChNz9Kh3bsufjdOluO2eTesQe17pwze/2mTgL2yZMTSxS0mCRWgmPNLkY4cC8Vh0BWsI5S0UEPbCdzF406mO2WilBs3ljlDda8rRLVLnJET8FfrhZymJJYy2smtIXK2J6wQst5esE6r6ZLRdq0Bo2MjYGPK1Weqb3yR5XU6TqZD5L6AYXBPZhuFHS1irpzVQLfQgbAd+R4oZPw5/jHHI+fglG9LY765yRDgixEdAhnyJZ09ZZGTPyQDClgnqLVZTTOodugV+3MqqRt6L9NmvyBCDS07qcGjsbPf6js2yzPMDXNSru3JoFmCaY5Vr5FBaLSAPw8q8+Q7EnyKuEkFy+0xPLumFbDUuuS7Rnw95HYX1f5fV7 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6f8a1292-e568-4413-81d7-08dafe4a688c X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 20:34:36.2512 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VIOiV/vhUDt3xOAP/AchK30ms/PkFeWaRLf8Ao3PaRON9MWHqdU+EU1g3l/p0cU4 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4095 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1F4DAC0003 X-Stat-Signature: oqak6wq8c33sn4r895qytgxn9jjpew7o X-HE-Tag: 1674592488-292439 X-HE-Meta: U2FsdGVkX18DvQkFa0QoQbZA8w3OTFkx6uQeIrHRmIk5N/PGU590/tLxj3KC0lxB1Cr16+fJjcOC/QNfEtDLbB1X/HXToQGMG46oYUxswE0UISJoWf5cSQ6cFW+KyiQrlHW0Ud0BR58ycNyzhno9FY1xDRMx/R+zbHrL6ydVideM+TMH6IJ4LAQ73882B6x62Yfytpf1G9czieBpBVUEC6nAVTnQNe4qHQHTO8o0lDYvH3311y2H81fnMD+U1100wcMmcJWglTkng1xe0XWbd1k4f6uWCUVChNDNBIMGhuRIifj4c5EPLor1RALhm0u0DO4jh+RUvUV8QOK0vHNh0jO7obPV4QFeCLv1+WaDZyXZ0xeizwpiQ0p6lfVht4bKwF23xbEJ7sjinGar38C0XkgK9sCYr9ZlbM8FVEb9GFqqe34jPnsLJ4VDWrVTBj6mLwETquaek6VDiRFir6LwmkN6p1wxt7eWfs+r7mChvCWMdNo1IfIRiqbC8846vIAMPLls8vfGimdcCU3l7U2WQSzWv+Zz7C2jjTp28HKpG/SHERAVsrYPzuJ39ldHicSzDqrR6AjnQEdSSJwYsasQgVavi/tkPrTIUB51XcVdpHK9+FMrR9edb9ZnHLGCLsExKILI11fSpnbYbYEb3BYvu2HJiAFv/ywsvZVTSg3943+HbSWTZUiMGFAcMnKEdorsx9DBdimMb29gofIRpX7ONznhzhn/VZ74yHsIuCNWeFuA1oPa/8nJjLTJTLPRXysy3c0HuHvJkf6pRT2sSRH8UECduRTo6JSXoH025FceejxUbnJ1kGF6ulyb3zZZ200zJKoq8GNgZ9LjIJVnO9+llyCnihQdeq4/u1CgTevxTXL1jsCq70EDuLFRA4UpqJV4Tu0A23h+VaO100WyweSqGc1cZ0ZRz+ZPjYcgC98s8fPvZwrxg7pSIYbmc/bKBuJPB3WmfNkvfLdinfmtzIZ BU4LDPVr Jp9LqY+LDxWfmaXPRTe6KzjIXj9mA06R+XwANA/Nnoi+3DnbImtJjcXTHdUN6SVb42CJjYVvd/iZGEpUnq5bHZpL1O/bFpXgyx1iHLCo5xUFN71DRg1KbMvh/bATLaA0/6QHKCWjfT+G0V4yzj5kNkMtKmIXkL+1mtXdYxsZCRt2WyPH3sjWL6whybLKy9TTWhLhddCkE7+wTAGqadr+MBpOTpz2O3j+U5Z1Jsj8xNSA7K25WgzHwU0vl0GcFsMlCDl46EdgxXk6LG0qm/1/wWF8ofEQwOdgLOQQ+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The only caller of this function always passes in a non-NULL locked, so just remove this obsolete comment. Signed-off-by: Jason Gunthorpe Reviewed-by: John Hubbard --- mm/gup.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index de1a5c64fdfdcf..dfb315b3f2950d 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1558,12 +1558,7 @@ long populate_vma_page_range(struct vm_area_struct *vma, * code on error (see __get_user_pages()). * * vma->vm_mm->mmap_lock must be held. The range must be page-aligned and - * covered by the VMA. - * - * If @locked is NULL, it may be held for read or write and will be unperturbed. - * - * If @locked is non-NULL, it must held for read only and may be released. If - * it's released, *@locked will be set to 0. + * covered by the VMA. If it's released, *@locked will be set to 0. */ long faultin_vma_page_range(struct vm_area_struct *vma, unsigned long start, unsigned long end, bool write, int *locked) From patchwork Tue Jan 24 20:34:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13114784 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 411A7C61D97 for ; Tue, 24 Jan 2023 20:34:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3059D6B0082; Tue, 24 Jan 2023 15:34:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 290E16B0080; Tue, 24 Jan 2023 15:34:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B8E06B0083; Tue, 24 Jan 2023 15:34:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E0DC26B0080 for ; Tue, 24 Jan 2023 15:34:51 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id AF248AB1BB for ; Tue, 24 Jan 2023 20:34:51 +0000 (UTC) X-FDA: 80390846382.07.5ED24C2 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by imf12.hostedemail.com (Postfix) with ESMTP id 169E04000B for ; Tue, 24 Jan 2023 20:34:48 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=pxEDXGZ2; spf=pass (imf12.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674592489; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eF0zU2ZoBV9oclS+2djXfPffPq07D2lWplMza8/TPgs=; b=PpK3zsNcAAUP7SBYDmKfTWeJzZyKz+TNnK32MHONp05+tH+PltG4vtRt96zwIn9XEZEJxq M0IKoeUDoWvpsSc++sbuuf4GffPTY6vpVqok9GKNJWBG+AxvuMIARISLLBoLcvsqn4lrMP ZgDgFCU+fVWPSszBk7PvMimpqQPujwU= ARC-Authentication-Results: i=2; imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=pxEDXGZ2; spf=pass (imf12.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674592489; a=rsa-sha256; cv=pass; b=8nypJiRwsAc3rTdrQGTrozuWiiamanpeHWiWG9dCuOD28nwZZGTNRIKRBXxv/RQ/UfEx0y +UfqlXCKMlKA2I1rS+r6Ha09MBSy1Hm0LgMHKr7vOW0IjlnpjRUv/TpuYft+R+iR/zaCLQ cTr+efrgEbRS8sS3FSxkTwCdt0Oa+qc= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PuS0mzJfwNDoY7wCYTXsHv+/JBRLhlDiAlPul+X5bshJN9fbPrPXU+BwOS4LYcf5dp+xnF+/MYMezq+yksOSqzNZt/qDhcj8uBP1/V7t9A64sr2QStl+d8bgAjBjkLvPkG0i09bsVNOHTpymUmvmMOB+z/dLr5Qgxkg/BCewK9EhdrNla9GnOlLRMYxoaIs8cwE5vjUI4ZRi8JuhUlntl7h0BXPXCYXKnDutSGoU//OFXqI7jidBHT7mLXKRVgH2TtgHfLzIDiouLr1km/J57zoURhUlAEAVaIyz0w/SCO/TZ8s9+1qBwvtnzF5IRjdZXYqeGrh3Qrs7VySv9HGpwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eF0zU2ZoBV9oclS+2djXfPffPq07D2lWplMza8/TPgs=; b=WiJoeTBAUnyGicbtKF/dJPOCz6eR+QBFijD+/dsVhloenAfW8lM7DkgqDeE6bTclRqQCztr+zdyrA2UVj+QiHNPLyiRZmrtgTr7jJK8jroc3oskQjK3yeTF7BQrY6ltcLlVUunYRpbrAoN/WCykeucPC/ZC6Wb8L919446ZA4AsO47Fv4boU0cOv/s79GM07iMfpiBGLCG/eUXBlIxL7JloU8fvEcL6L4fnSF73PktnBFe77kfqIxcUX74qwS/YUNvJPudTvj2o3c3L5y3xJWHkV8RrK2rDQ2aYCTbQlXAU5xrpOnCp0tX4HdPr8BVeJpIHN1TyBueahbvhWuRTADw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eF0zU2ZoBV9oclS+2djXfPffPq07D2lWplMza8/TPgs=; b=pxEDXGZ2NBEDtwLYRDKN1K5yI8Bp+W9uQnTKR0N3OGThtgFoLTuJ4/ugX5xdQfukyDpiMuf+FuNhzJu7+hnNDCJHzjG4wdwQ1nhTvyxj8JMAZLX7D8abKjcOtCBjnZ2eC1fINNSmp28+kPR1H5Px9zkCljb9QCP98trSXRWW5w6Jqx6VB3exlfUVlkLGjlAGWjjG0e6rFuXb8nIc7CiPPTRZwFwHuxHeaY92U4gjoDc+D2Ur7kWL+kw2ff5XtfHztvLMBfyV7ddw2CKJH6IINCfXPZsKyb7U1tD1BxEfwxJznElCZyQ8yucgZerLVYqqBsq/wk4pi62v9CKueLrTVg== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by MN2PR12MB4095.namprd12.prod.outlook.com (2603:10b6:208:1d1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 20:34:39 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 20:34:39 +0000 From: Jason Gunthorpe To: Cc: Alistair Popple , David Hildenbrand , David Howells , Christoph Hellwig , John Hubbard , linux-mm@kvack.org, "Mike Rapoport (IBM)" Subject: [PATCH v2 08/13] mm/gup: add FOLL_UNLOCKABLE Date: Tue, 24 Jan 2023 16:34:29 -0400 Message-Id: <8-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> In-Reply-To: <0-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR20CA0041.namprd20.prod.outlook.com (2603:10b6:208:235::10) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|MN2PR12MB4095:EE_ X-MS-Office365-Filtering-Correlation-Id: b3c3d6fa-0e19-436a-c8dd-08dafe4a68b9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6nPxDeZMWsl1ZwjH4t2PZgEphtRd3jRqGFr7jsuJ9sGnBAS4o2fpF0exD5VXJhOINZAoTeS0I6+wUUKabFxARulHvfIAk0068J9MNrHDlvicoLx3nrEalG24vtTiGzh+hlKwsRYx7xJZMhdM6ZPcRPst6YVPJvb+QzLXsMYMrO7i8m57Kj4rGr1AF9fboEo8fXbidMzz4SfPvLh8+ZZ03i7/8EwIrjt2KpIwfK+n5BzgQD2B1IVpLsQBs35e3r1Igek9C9v8NzfJH2RgyH1yzRNlxHykBDAB8dRIJpQ//lVj+0XfrNhnQqjcJbc0xK6lGgscE1XiOxr7OjmtBbbiOWlQFINbUxPfcStkh2hznoHLkMJK9NLQM9UPaNvH/qxVb9jUr05hspJbCwer+hQyDzey4GQIHxL/O6vBbs084wqTPO1DAScJKAFm+qqs9qPV2s5D1eYpRRRSgtFHok5bhMD1nUjRIjHcjtdz0pAqx/LRhxOYukUIBmp9z0MdoJYv6Urc/d2NuJtHMe1MQDkHKjvuFwOarrvrtmNGUMIoqjfSb1PGweUL21/Wz/Op+XeBHtDQBA7xbkIHN/wdtHBc8x7pQqI3PMOXg/+20IOiemiE8hUUS1L5NynOzkrcO4hO1SJPcX0Y5JVKrOpElI6kqXdPMoiu1jYKxK/p0oxFm7FezVwGA4m7ip2J1bkBvEhjx50rPz/p4exUVrfZ3yYMUg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(396003)(39860400002)(376002)(346002)(136003)(451199018)(109986016)(38100700002)(83380400001)(5660300002)(41300700001)(86362001)(2906002)(26005)(4326008)(8936002)(316002)(6666004)(6512007)(6506007)(8676002)(186003)(66476007)(66556008)(54906003)(2616005)(478600001)(6486002)(36756003)(66946007)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: HQCQJG9ektNR2vikEK2fS1AjFGBBcOrhAn8ryn5nBjwEuu3vhC/SqCiHtTMaIjvoTPwPY9G1RqUTe+83R3VRUL09tHeXt55keCHUAm0d+odaUL40REzPcKNBa0uYdOx8XLuPRNIQWYgfQf1ZrKlXgdvJnq1hzvk9W7bRyNO4BOkXeDcAVWnArFyA/FyK68rrQPC0XX69bCFn4GkHy60+aJXEGUlJxRvV39rfVn0lxmOSdlEUx5y2XKW71nz6vXmJ3nAWpwirRR/pIvZ6QzUFdk8SXDinM80dOIZYB9Rlkpw0JXF3JNt7ImzsgahgaoAf5jMnJ4mwaM5cfqKgcRrHJZd4Hbmrhn+yCW9VALrIExQFDiuFW/Pys5vnsYcV0b32OKC1sZlqZl7Ub1Byl7qspqjx9BJtXIkazdazNmUW59uLZSpZPtF6VSjX76o/aORtg2aCWHFRjIGdtAAq2K/puKpCOVxJP0HAz/e4y7duQlz63U4SgrkdKNv/mwDlltTlZn1E6fSBs6De4f91SC4ar76FC5dT2om5RQTOe5uuBI9sqhuq4IU+dz49lgb1HCBQcg1/dfZy5rDmYTEKo4lk81/zeULdawF08aVOvXsNCDdWRZs7SQcmLMViQbWK3vbbmqjkKFumW3yaphsaGEfsplS/Bft4rr37rYVaMYQji20JWsr+eLQ7bgpC1iokqXMkkinw3tpO4cflKvIXgLO8W4dSy6M26K/JJJVQaA2d+P2AytkOK59nRddnduO3B+lxhOGAc4j4+dIt6dZpK4fDUgRamCY48TPHEg6wNnPXuR2PMH2TsmnNYm4bqoCA3srgREFG3jVEcjddigWth2dHOBH1p0bgcdIhbknVnHk7vC40erLsKLuA2rOTc7sDmQvkPKCnYPq5I+JUqgCueMAEyk3dQmcuRqmHkawRfABENJE9jKtVB3XHdHU2qT/6ukjqn9xqNbNi9mJGyRmL/t8oexpTF5vyDcbjskxfe8rdo25/93tIvkt5fXH3GJUXSYsY+U9vw3sndlrXq0pog9s2aottJRBZoNF+7mQroK0jM7T6kXNrEw515PqzHpmMI2cwdPq+Q81BMYFl7B0cQiJnCAR2G1b83nBNYwsXE8yUfGTsBe4HgXOt+K7pMhr2vX4FHrMMUVKo8JHAtYKU2PXL1jQnnEIQkNw/ivkhS0MMUw9m1ssoYeR+4luJt1pEI9wldsrgDedbyvWNLnjoj1GUP1rd4HQCV/SnnSaOBYEY7+nkv5lVX+WL4utnuvqrNar+M4+euVbBWgbMugnlzfLgFgxj+pup8qUWHTfCnEC3d+tZYcLKzrzoLV0JLyPgKobSNQp5sW77OQXDDTac1xwbZjRnl5CBg6Zpk7v+cEoaSIFnQPS4K8SWfn7p8rt6V4aTVHRLkoW1LPDVoOxMA/984x2TJIk96esfypratm5/u0G7D4NzW5TbUKvaFQ3VqATsuAWHl4WS2hD+6C1NIglP370ztf7mI0QsnrAW4QHHXJTo2HVL7PyQ3XhIglq+WJcxe1rxf1VyTRe73+cYBeFDwgjN+7EBFB5ykArnqN/ubC89w8O0hY1+bDPOWkwvXO1K X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: b3c3d6fa-0e19-436a-c8dd-08dafe4a68b9 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 20:34:36.5481 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lRbUBT/tVOfIZUM07saLpPHZNQ09YWjfpj2DIHHa4noLcZ5SJOzv9zx/h8EaFBkC X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4095 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 169E04000B X-Stat-Signature: 35qrtkzy1oyarq99k9baqwpxf6xxengf X-HE-Tag: 1674592488-368798 X-HE-Meta: U2FsdGVkX1/paPs+ORjR9tyG1Q2C0J8lDxjW6NR+zZzwwxs0qFCmzELF3QK//W4dSWRJ/dbaDQUINvg9MPy1nkqcLOxUQUhy/U5cDIaJsg+Y9hK4LEml2k4vm2rQd8gOPl8kd8O1hu+UeGjohhz1tlNnzc71Lj1cGTC8Ztm/Hm6ePdMmJvKfY0BLtvY5d+Fq9vslg8/PU8tl4mus/iUDd7V490+L1JS7v4cJhBwKmkGzCE00D7UenWhxDo9eUYQxuMpYn6/onMvEoGK1aUOQSlYER6yETkmsPzFiniRqE/4l9iRFfDRacMFphCXiau3yrBdh6nnPE3FKQt5166Ab5jVT306KYSfsLKnV75EqiOGm+N6ECyt0L1A0aqXoqUupiKUuT6zehSwjfW9tqRQoyC8Re0iyky5UupenaiAA0RjGPKNqhLU7XmqUGmXdhqMKdRuYE9sMZaj+xXHkiHBk1k+n/Jh+FlFKt9glmQMmPMOjmONbFmbtJOQHjEyRjKexRNFQTR98ZkmjvVOTcydLnm/IezFkdochU76p+enEUcTgHt4pJOw9Kz7nkrKom3BDMxFTRjd0jpvAgWP86Htep4UVYmVf6DZpOr7EAv9Flue824aKFO3OamDrhZRgJTCMxHRmaJfjzUt0KXqrBRMg9StvHxVyp2ROtLbDRFIetZp5xK8wMPW4lMIpUbj2xYhYMEuoGHRK4oRIQiidglQE0Frb5FT/La8iyBk2g1tT1OGz0StR0UiiGY2tSzYLkRSqECrOft2IZVQnfqawJWVhbzLRfq+jmZ8VbUX4CNRuthMdqYyQSwEG0rQfufWCP1/HgsC00hUTTzyTGIFZHr802nTl3Fxs6FYvSxbvflUSHzetLKV9Hx4bGie8eeTRBu+O4sDRO7EFR5/8jf6utdVFCEZFIwAU8UR7hYPxMsvPZQvl0QTgQH+qQ4RY1v6heWeRf8qfRbW48nX95H5FFQ3 uJ/hyC6i 4ErkTmVE5QOkVdvJaquQFE4Ys/8HeQhDAKzrQYpB1UmwoqxM71kWrA7j1VAyWZz/jocXNadEXSgHfLk6TzYSI1M7oc2c2+AfnGXVZdt+WTSnpPzwluSOIX5h2+8xlQgUF/KdjlFJCyftcoLJeIj5+7QqXot86TW4PWu26tne/6uZJz3ZdflG4du9ZhVwGNvEQRcG+mP7Sbo6QmxHJGsLN1Uy2JMfpA1bl3l0an2FM8vCPG8l4Us1+i7At28Lf55uze5s4MgK4yNAwbFccdaJzKrKYp7wVvr1BG7zWo0jCX3851/iRfR17zkIHBadB+6cxayuGoj9k3Q1REx0ELvGJEfI5Ag== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Setting FOLL_UNLOCKABLE allows GUP to lock/unlock the mmap lock on its own. It is a more explicit replacement for locked != NULL. This clears the way for passing in locked = 1, without intending that the lock can be unlocked. Set the flag in all cases where it is used, eg locked is present in the external interface or locked is used internally with locked = 0. Acked-by: Mike Rapoport (IBM) Reviewed-by: John Hubbard Signed-off-by: Jason Gunthorpe --- include/linux/mm_types.h | 1 + mm/gup.c | 36 +++++++++++++++++++++++------------- 2 files changed, 24 insertions(+), 13 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 8971a40c120e38..518617431f8085 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -1103,5 +1103,6 @@ typedef unsigned int __bitwise zap_flags_t; #define FOLL_FAST_ONLY 0x80000 /* gup_fast: prevent fall-back to slow gup */ #define FOLL_PCI_P2PDMA 0x100000 /* allow returning PCI P2PDMA pages */ #define FOLL_INTERRUPTIBLE 0x200000 /* allow interrupts from generic signals */ +#define FOLL_UNLOCKABLE 0x400000 /* allow unlocking the mmap lock (internal only) */ #endif /* _LINUX_MM_TYPES_H */ diff --git a/mm/gup.c b/mm/gup.c index dfb315b3f2950d..37849fcd962e9a 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -896,7 +896,7 @@ static int faultin_page(struct vm_area_struct *vma, fault_flags |= FAULT_FLAG_WRITE; if (*flags & FOLL_REMOTE) fault_flags |= FAULT_FLAG_REMOTE; - if (locked) { + if (*flags & FOLL_UNLOCKABLE) { fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; /* * FAULT_FLAG_INTERRUPTIBLE is opt-in. GUP callers must set @@ -1382,9 +1382,11 @@ static __always_inline long __get_user_pages_locked(struct mm_struct *mm, for (;;) { ret = __get_user_pages(mm, start, nr_pages, flags, pages, vmas, locked); - if (!locked) + if (!(flags & FOLL_UNLOCKABLE)) { /* VM_FAULT_RETRY couldn't trigger, bypass */ - return ret; + pages_done = ret; + break; + } /* VM_FAULT_RETRY or VM_FAULT_COMPLETED cannot return errors */ if (!*locked) { @@ -1532,6 +1534,9 @@ long populate_vma_page_range(struct vm_area_struct *vma, if (vma_is_accessible(vma)) gup_flags |= FOLL_FORCE; + if (locked) + gup_flags |= FOLL_UNLOCKABLE; + /* * We made sure addr is within a VMA, so the following will * not result in a stack expansion that recurses back here. @@ -1583,7 +1588,7 @@ long faultin_vma_page_range(struct vm_area_struct *vma, unsigned long start, * a poisoned page. * !FOLL_FORCE: Require proper access permissions. */ - gup_flags = FOLL_TOUCH | FOLL_HWPOISON; + gup_flags = FOLL_TOUCH | FOLL_HWPOISON | FOLL_UNLOCKABLE; if (write) gup_flags |= FOLL_WRITE; @@ -2107,12 +2112,20 @@ static bool is_valid_gup_args(struct page **pages, struct vm_area_struct **vmas, * interfaces: * - FOLL_PIN/FOLL_TRIED/FOLL_FAST_ONLY are internal only * - FOLL_REMOTE is internal only and used on follow_page() + * - FOLL_UNLOCKABLE is internal only and used if locked is !NULL */ - if (WARN_ON_ONCE(gup_flags & (FOLL_PIN | FOLL_TRIED | + if (WARN_ON_ONCE(gup_flags & (FOLL_PIN | FOLL_TRIED | FOLL_UNLOCKABLE | FOLL_REMOTE | FOLL_FAST_ONLY))) return false; gup_flags |= to_set; + if (locked) { + /* At the external interface locked must be set */ + if (WARN_ON_ONCE(*locked != 1)) + return false; + + gup_flags |= FOLL_UNLOCKABLE; + } /* FOLL_GET and FOLL_PIN are mutually exclusive. */ if (WARN_ON_ONCE((gup_flags & (FOLL_PIN | FOLL_GET)) == @@ -2127,10 +2140,6 @@ static bool is_valid_gup_args(struct page **pages, struct vm_area_struct **vmas, if (WARN_ON_ONCE((gup_flags & (FOLL_GET | FOLL_PIN)) && !pages)) return false; - /* At the external interface locked must be set */ - if (WARN_ON_ONCE(locked && *locked != 1)) - return false; - /* We want to allow the pgmap to be hot-unplugged at all times */ if (WARN_ON_ONCE((gup_flags & FOLL_LONGTERM) && (gup_flags & FOLL_PCI_P2PDMA))) @@ -2140,7 +2149,7 @@ static bool is_valid_gup_args(struct page **pages, struct vm_area_struct **vmas, * Can't use VMAs with locked, as locked allows GUP to unlock * which invalidates the vmas array */ - if (WARN_ON_ONCE(vmas && locked)) + if (WARN_ON_ONCE(vmas && (gup_flags & FOLL_UNLOCKABLE))) return false; *gup_flags_p = gup_flags; @@ -2280,7 +2289,8 @@ long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages, { int locked = 0; - if (!is_valid_gup_args(pages, NULL, NULL, &gup_flags, FOLL_TOUCH)) + if (!is_valid_gup_args(pages, NULL, NULL, &gup_flags, + FOLL_TOUCH | FOLL_UNLOCKABLE)) return -EINVAL; return __get_user_pages_locked(current->mm, start, nr_pages, pages, @@ -2968,7 +2978,7 @@ static int internal_get_user_pages_fast(unsigned long start, pages += nr_pinned; ret = __gup_longterm_locked(current->mm, start, nr_pages - nr_pinned, pages, NULL, &locked, - gup_flags | FOLL_TOUCH); + gup_flags | FOLL_TOUCH | FOLL_UNLOCKABLE); if (ret < 0) { /* * The caller has to unpin the pages we already pinned so @@ -3195,7 +3205,7 @@ long pin_user_pages_unlocked(unsigned long start, unsigned long nr_pages, int locked = 0; if (!is_valid_gup_args(pages, NULL, NULL, &gup_flags, - FOLL_PIN | FOLL_TOUCH)) + FOLL_PIN | FOLL_TOUCH | FOLL_UNLOCKABLE)) return 0; return __gup_longterm_locked(current->mm, start, nr_pages, pages, NULL, From patchwork Tue Jan 24 20:34:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13114788 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9306C54EED for ; Tue, 24 Jan 2023 20:35:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A593C6B0085; Tue, 24 Jan 2023 15:34:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 942BE6B0087; Tue, 24 Jan 2023 15:34:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F9EF6B0089; Tue, 24 Jan 2023 15:34:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5E7F36B0087 for ; Tue, 24 Jan 2023 15:34:57 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 39463AB1D5 for ; Tue, 24 Jan 2023 20:34:57 +0000 (UTC) X-FDA: 80390846634.23.A131987 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by imf12.hostedemail.com (Postfix) with ESMTP id 82B4A40018 for ; Tue, 24 Jan 2023 20:34:54 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=DYKFYF0t; spf=pass (imf12.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674592494; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=S5ttauOJcEpPx3GZLCR6G/PephTpgeGFxt4/6gTcmLc=; b=313Ey4Mj3CvF3R2oTzNcsowQgFso5xgp+hnbJjG5/I1oY7OnawXqqVkXQ7Wd8EoefuH4jf h5pz/qHkklbXHZx79TDO2Yuo57f+IjeTzgeDsaqRxlR46UPxPemgRXMmnsRzyg1DNg1Alq XyH3ML8LLmhV2Mb8F6pKRP4awq/6mxQ= ARC-Authentication-Results: i=2; imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=DYKFYF0t; spf=pass (imf12.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674592494; a=rsa-sha256; cv=pass; b=xdXPjBGyrKsoIu26v30giTbvPYeoAhHX2hSq2CO/xL5L0IsBWzmjlNtC4gn0tZ2aboRaS3 EzAqL5ulD60ScxILJ1YefV0ahLepmF0htHYVaHNRiEN+hNJ3Nkn44lioxZvNOnpEn2/Ykh Kqk6UgHWVSzPwoVW8AIQq80OuTtbRF8= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ff3/cED01mXGv4Siw6k/9gbsGpiH5jAN6BXDZoZPJ0mrMFRyGkZEIyFedtr/sE5U7Rq5P9RSPUE+4emMgCGSnL6QylF5Q7BT6zECzRYIPZBSyDoNaQgNwAeAl39eOtKm4vOWmDvYjz28WWbx+Q4qIbGPPgZ1i74TJw2L0SFPwdzAeyR7+n3l7ZySLLcXqIRrUGkE3Jei5Q8cM2thkvj0zDArj6QV/ob1q1MNcgumrVFqyCqG+HWlPkg4PhAP+6HzlNDeztwicXCnnzusXDBzmZBGlDpIuXmUaNOhvkZaeUwV1MtdQv16q+nxY7BBgMLLLASAfXoNnC4z2oJgSb6roA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=S5ttauOJcEpPx3GZLCR6G/PephTpgeGFxt4/6gTcmLc=; b=bNg7jfk087d+rkjJAQh4svEsxz8ZpLDEQb68VJAy/BoSBdbz5m3tilggjFUB3D/h7HcQwVRMcADtmRKasbtomOSrM01uT81KdmPolqoISepu5K5+xZuczrC1uondKEtn59xaYp9Bl3mlX1B3SjobRgWBH/o9P/whhs8lTTmkTwjLBDSlAVeDAxXaH0aCmB/Qyxx1W8gaaDwYlzJmwswaR4uo6/r+6Q9vthYs4EvFkRWtd6vt648yFAZcoz9Lo7KEFhxWNPgPaE5qVcA0rFanvEYwk/2F4ByUtl270hVj5zsJHzfHlSPIFZWZFP4Oz+lBRU4YI1YkK9SzzrCVkISokQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S5ttauOJcEpPx3GZLCR6G/PephTpgeGFxt4/6gTcmLc=; b=DYKFYF0tEQg5HJiFEJQqNoDCUXwZ1H5SfrkMv3N50pLbMbErbLk/eMZbBFyjmsacJzd1egS52WWeofuEOFMnJ1OX9tyTGbmARKcTJ19boIfbcnEGqEIhC07WSk4GT8GJItTtLbQrScK9EQAwKeWO9DCruVxTFUA0Sz4UerCkjbIM7y/Du7RuE32RI+vi7wRUcg4cIU9r6WI6rA9o4jWJwEowUSF7boc/gMyT8vXJtHBWvDNa51TOaiS09zDCTGasMNOoy9p0SBiYDiWIw2qjr3JrPsi2wIsXLksMpE2WmJxKWSj6yux4s5n75jyCDKRD9U/b5LkCJT+IvuWSq5XvZg== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by MN2PR12MB4095.namprd12.prod.outlook.com (2603:10b6:208:1d1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 20:34:40 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 20:34:39 +0000 From: Jason Gunthorpe To: Cc: Alistair Popple , David Hildenbrand , David Howells , Christoph Hellwig , John Hubbard , linux-mm@kvack.org, "Mike Rapoport (IBM)" Subject: [PATCH v2 09/13] mm/gup: make locked never NULL in the internal GUP functions Date: Tue, 24 Jan 2023 16:34:30 -0400 Message-Id: <9-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> In-Reply-To: <0-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR02CA0093.namprd02.prod.outlook.com (2603:10b6:208:51::34) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|MN2PR12MB4095:EE_ X-MS-Office365-Filtering-Correlation-Id: 975c8d53-c552-44d7-7a63-08dafe4a6920 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qdBAHHxyOurPxow9DXjB32Im4IbemwDoaMKmLVZhvSvXU5nLtIBm8gWpgd/GWcuyLH9sbBGsLw1sl1hhaXQAnfyeLlF0Bb8WWZ3eSgqu6lbkudpLXBk3y5N/RxA/2joZd1qc+oxIyzZKKKqMcJ19Ts+Pw77oH2+ZQt/Xhd2GOK7UsC3FjkZKHVNpXx7H9GBsDghJbQSprk4pnKQURNDTzcxq4GX+CBIRGhqfOa5mU83S662afy8jIthx3a7vomA8qxzl5Vptosi1vW1IyjJyVJnl41zzV4i5K0Luz1ySibsEJBZzT+yCnLIOjERN0RwSwALhN2hWkWSFXjOxskYI3r5C2TaN0g0EyCZWsFAq8ZIY0uU7Lee8aU5Md3bJVJd+co5zgbFflh2ymplTtvJgO+Q3exO3lt48j4TJWBVqmhNgKCx350aonCPAn63CcWWMd4322vrYLu1BdbhSAUvrvSVpFugslqbAVpm4pndOngOnBNnyzjwSN3Iag5XywTgS47672BGXWDWmTIyiHZUtJRMb7ut124lUtzsNnUqMHleCGBYV/0rkip9HtzWuqYM2WVJ7eTJKV2QvulcymPlGE0dBZlPji+3GRywBPeOqv3VSRSrTn9fDQ9qKu8+312/T/7k38EwBYb0xyk8g18/Y/VLvEb5+jARPv/GNmezsQUFvC6ZjHYW+eJLITptNgGIfuVF0CPOzuyUMAdDkcr5GEg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(396003)(39860400002)(376002)(346002)(136003)(451199018)(109986016)(38100700002)(83380400001)(5660300002)(41300700001)(86362001)(2906002)(26005)(4326008)(8936002)(316002)(6666004)(6512007)(6506007)(8676002)(186003)(66476007)(66556008)(54906003)(2616005)(478600001)(6486002)(36756003)(66946007)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: MGHIy5CxrS/hK7gIHUQPqvy+DrZ/rqm2JBwrI3zIm0HtOnXP+z3xfMoT9hsj/VCI/bDCsojpnXtBhBdO9QZxmlbNfVhBKXSjBuEeIocY74Dq2GCjtcdiOZuf3BYWXQxxPhXLvJAm62jzAlSJ0DC1St2jaDdEaKjZtWQKpp/7IoT/8bkjLIud7gVgY/HgIqQekC4tXaHNEwbsQgWZSh6Dl5cpgR2X5HQS5f2CSdjnwb3NeLb8qEX39FTIBLoGF/B7Rs1KCifu1MHJY67OLHNWtfNAU0puHkzJ6HEfhZUt72/1qrPC8UxZNR1l0DuQJ6EP8DfouCEcrjtYCFvyRES4oMVmSoNf/TozpCfhhCTD/roHkQj0NYiUVl/Nif3F0E6mZ+mUWwYNeAraU/mg5KOxITllHROV3cluEPKty1JOMO3WayaQR2JUA47IaYgHZgOKufHkUpnBWdLWP5PwZ5nzyvPI2KQFfmRxOT864dLG9Df5MS4ytDTmKHcfSVsHmhP7IZgeRUt5AL+2Y3m3svpnTnQwX5gfwAbfYU8bSjoDTXPpEH8fu5PBRB+oA+ac9KGbGuZGFoWBDmQPmgBX+9PUzYKHBIH+9swrMobrdPP8t/wsiGgyrZE2jw7Oesrgsa3A1esCw0xt5SUYUR2abh5IESiBOUtAdvuNb7BQKLLck+JcIPE5d+CETwKi4IEM6HtboEEZUvWm7gJZb8uDmG6xjhR8p5jlCWQLIvsItna3kKKO/SFwGJtr1TWVIp15RaWN1fxw2v04NkdtDCWDUp9dAbVFjw+B31xZHCZJ1N8vXg71VYMTBDX4h7nlM7PMPxUrvwBhWumY92/wACfPxqPuv0GF0sC37noc+r1/YnfIEqWRA8QaCdIa5nubwUa7TRmIkHVW006eeSxTRp5TOYqXLrJbe0e8aYObWJp9Qx7B7kGY+HqHo7NbxqGqkrQ1MicszBtQL6gBS/NgN9zsTkwcHCOeGDbOIryywaekaWFQujUKoQvpuSUToqwKGV+gYYksIQbH0Ito/tjNoSg8V1wi29rfDQlkAOvPgjxPxLdKDclPfcHft+H8q4WOx/uo4KbByg+x38EMdrUMFuQN07ZZpbiAcHlkSGcaymnTQq8QcrzOHUOw0ZORNX3ANSfBMimBpWN2jv+rrR1g+9DJLjMHI91FWDcd2KdaC/Ja/bTZvfptg//HhQ0aEk6QekMQ1s3ppZa5oKiIZlN8T83GtJveJ1t9/+qYVGMim0yWszt6882E5jqO4nKg/NRxwSQPOCCJnAZSsNjJ3fA0qBgijYGGJrSMDo4QS/R55a/Kf9ch6cMaq8o+//0Q1T2//fBO/O2rfdMfqrNeEvVOzWgNeM4xK9z5UB809n8Of3CNLXwne0F6PfMSSkuG+vVyQq9IbbSFzt5NCtdTExeQpPGmLEsCnTRnzsz0qaxgY66QxhMi7XDpI1OEXgEnSoyUE7SUhXr0Y5NnyGEG//qX9kKSzpyQFORRHdscotsRC4WdlSvRI9mq5iOWWfJvso4whe2HZZ47HFuakrFw+RiXCd0thY+DAqAk1imiugCuVNxzr3kt49IYATlI3LWBS9LZWLhlCiyY X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 975c8d53-c552-44d7-7a63-08dafe4a6920 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 20:34:37.2523 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RKP4fhw+X12kgL3J1dlXUAlzqFvw4vVWvMKvA/YF4AlmSXE5c1b3Wg6D1R/1ygFw X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4095 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 82B4A40018 X-Stat-Signature: toyphu8zrsuetqwr5hgcjcnh59im1z3b X-HE-Tag: 1674592494-129174 X-HE-Meta: U2FsdGVkX1/kVsV/REqcfcBm1eEvVxG3qRV0WrJY5PAEYFFoIZEuWdF28Q+/MJTNoDo0DPnC75Ue8yBOuQ68f3STGoTgmzqx5MOTYtndd8EXKf+YIISzbNBhp6fg6t+XXgzQW1casrhDKO2m8vYbSlGcT6hVzWiuec9OULTjNXfD6Nyq6fzeyQVH57KH4d/VX1SKLWHm61aI337tXUiU8V8pE7fpsYWy3EX2B2vTa0Er6GK9Ep/gYNZhtrtdc5BYZJW/Syd4W8UCJwb5tUu8BGPGQlzB3yQ/Tl1VVbiKphzb5VAEksiFFEO3HQrACxssZ+JpiTEd3Cd7tOZVaDzXt9+LpzAxzm6C0kKiKs9DiMOEhDqpEwrlcT+N3JtZO09XNcRVRGNU18PbaKpparvU6WDFTVXQk4vpo5kI39NgGTxqf6HSZSK2wX4XZ+HNQluak5ZylsBYXKYzaLkM5ziIvuqqCqPkwa2Pmbbi36pELUrgq7ro20lDJTB2zOPs+62w8PefYSRpbuumv5KUN1Gf/oHlrgHk3g0JCNnvp+U1vNDAqRPF/B+LMXkN59Eu14Cnu023u0H7okie2pvaN0Sc2mN+gdpiwvAev87wp2K9CihDd9odYk367CwRB8AJoKaKhA/OHlq5AuZmJw6mDG/cKCHObEqCcA09KL06hEfozLueXVijLe6OQDJ3Cu1vy55R1fDeEO8DzeQDLQzOsADw+LYWj2Sm3qFvYgO3+5j0GGYRD6DM1UwQJZ8Y2vHT7WNGfrAiIArkxFQN6+7P8kefTqSHLYdW/+oZPUtMaqPvUJTMjDTk8e0HeeGubR2uhciRSeBrYINPZZJUBfKJdQ6rU0eHVcwHqWknMShicJpPF+XRjyBqvL2jLy+CIZ3W9b7QyUwLjR/X2fz2g2PpcznkTkRxRfp8/3OqjHHfzMUTIgKjyENmPws3XWZ2NSGTy6tKgK7Vlzs7f10lbo62wzI sTvWZaSN nokMbpuC4VRg/F1gjPDZcyAKw7/l5Tjzt6uuHsfR0aa5wGOk2bQfzxqr4EXHkcJ6N9X5CIwS9ivTu8bsq3W01BFYBgwYLvZ6AESQRAEl07HAAR3yaYSsm86hZOBKA91BbRoxulVEXf5dsFjizk2lO/wPE+wx9fZyjFZQaspgFbRZldmh46VLjP1Oqpq/AHBKBHaAqwB2EdWhWn8NoXsjrk4DIotiy5lKHg/HqXy7F5IYm6+TyMsxXwHjm5IeSxobRk8BgVRW9YhfjnpMvlNC4Mu6RBpMQNAdZdqDs12uTRo6YbSc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now that NULL locked doesn't have a special meaning we can just make it non-NULL in all cases and remove the special tests. get_user_pages() and pin_user_pages() can safely pass in a locked = 1 get_user_pages_remote) and pin_user_pages_remote() can swap in a local variable for locked if NULL is passed. Remove all the NULL checks. Acked-by: Mike Rapoport (IBM) Signed-off-by: Jason Gunthorpe Reviewed-by: John Hubbard --- mm/gup.c | 51 ++++++++++++++++++++++++++++++--------------------- 1 file changed, 30 insertions(+), 21 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 37849fcd962e9a..932a2339613c2f 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -879,9 +879,9 @@ static int get_gate_page(struct mm_struct *mm, unsigned long address, } /* - * mmap_lock must be held on entry. If @locked != NULL and *@flags - * does not include FOLL_NOWAIT, the mmap_lock may be released. If it - * is, *@locked will be set to 0 and -EBUSY returned. + * mmap_lock must be held on entry. If @flags has FOLL_UNLOCKABLE but not + * FOLL_NOWAIT, the mmap_lock may be released. If it is, *@locked will be set + * to 0 and -EBUSY returned. */ static int faultin_page(struct vm_area_struct *vma, unsigned long address, unsigned int *flags, bool unshare, @@ -930,8 +930,8 @@ static int faultin_page(struct vm_area_struct *vma, * mmap lock in the page fault handler. Sanity check this. */ WARN_ON_ONCE(fault_flags & FAULT_FLAG_RETRY_NOWAIT); - if (locked) - *locked = 0; + *locked = 0; + /* * We should do the same as VM_FAULT_RETRY, but let's not * return -EBUSY since that's not reflecting the reality of @@ -951,7 +951,7 @@ static int faultin_page(struct vm_area_struct *vma, } if (ret & VM_FAULT_RETRY) { - if (locked && !(fault_flags & FAULT_FLAG_RETRY_NOWAIT)) + if (!(fault_flags & FAULT_FLAG_RETRY_NOWAIT)) *locked = 0; return -EBUSY; } @@ -1062,14 +1062,12 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags) * appropriate) must be called after the page is finished with, and * before put_page is called. * - * If @locked != NULL, *@locked will be set to 0 when mmap_lock is - * released by an up_read(). That can happen if @gup_flags does not - * have FOLL_NOWAIT. + * If FOLL_UNLOCKABLE is set without FOLL_NOWAIT then the mmap_lock may + * be released. If this happens *@locked will be set to 0 on return. * - * A caller using such a combination of @locked and @gup_flags - * must therefore hold the mmap_lock for reading only, and recognize - * when it's been released. Otherwise, it must be held for either - * reading or writing and will not be released. + * A caller using such a combination of @gup_flags must therefore hold the + * mmap_lock for reading only, and recognize when it's been released. Otherwise, + * it must be held for either reading or writing and will not be released. * * In most cases, get_user_pages or get_user_pages_fast should be used * instead of __get_user_pages. __get_user_pages should be used only if @@ -1121,7 +1119,7 @@ static long __get_user_pages(struct mm_struct *mm, i = follow_hugetlb_page(mm, vma, pages, vmas, &start, &nr_pages, i, gup_flags, locked); - if (locked && *locked == 0) { + if (!*locked) { /* * We've got a VM_FAULT_RETRY * and we've lost mmap_lock. @@ -1354,7 +1352,7 @@ static __always_inline long __get_user_pages_locked(struct mm_struct *mm, * The internal caller expects GUP to manage the lock internally and the * lock must be released when this returns. */ - if (locked && !*locked) { + if (!*locked) { if (mmap_read_lock_killable(mm)) return -EAGAIN; must_unlock = true; @@ -1502,6 +1500,7 @@ long populate_vma_page_range(struct vm_area_struct *vma, { struct mm_struct *mm = vma->vm_mm; unsigned long nr_pages = (end - start) / PAGE_SIZE; + int local_locked = 1; int gup_flags; long ret; @@ -1542,7 +1541,7 @@ long populate_vma_page_range(struct vm_area_struct *vma, * not result in a stack expansion that recurses back here. */ ret = __get_user_pages(mm, start, nr_pages, gup_flags, - NULL, NULL, locked); + NULL, NULL, locked ? locked : &local_locked); lru_add_drain(); return ret; } @@ -1683,7 +1682,7 @@ static long __get_user_pages_locked(struct mm_struct *mm, unsigned long start, * The internal caller expects GUP to manage the lock internally and the * lock must be released when this returns. */ - if (locked && !*locked) { + if (!*locked) { if (mmap_read_lock_killable(mm)) return -EAGAIN; must_unlock = true; @@ -2222,11 +2221,14 @@ long get_user_pages_remote(struct mm_struct *mm, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas, int *locked) { + int local_locked = 1; + if (!is_valid_gup_args(pages, vmas, locked, &gup_flags, FOLL_TOUCH | FOLL_REMOTE)) return -EINVAL; - return __get_user_pages_locked(mm, start, nr_pages, pages, vmas, locked, + return __get_user_pages_locked(mm, start, nr_pages, pages, vmas, + locked ? locked : &local_locked, gup_flags); } EXPORT_SYMBOL(get_user_pages_remote); @@ -2261,11 +2263,13 @@ long get_user_pages(unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas) { + int locked = 1; + if (!is_valid_gup_args(pages, vmas, NULL, &gup_flags, FOLL_TOUCH)) return -EINVAL; return __get_user_pages_locked(current->mm, start, nr_pages, pages, - vmas, NULL, gup_flags); + vmas, &locked, gup_flags); } EXPORT_SYMBOL(get_user_pages); @@ -3158,10 +3162,13 @@ long pin_user_pages_remote(struct mm_struct *mm, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas, int *locked) { + int local_locked = 1; + if (!is_valid_gup_args(pages, vmas, locked, &gup_flags, FOLL_PIN | FOLL_TOUCH | FOLL_REMOTE)) return 0; - return __gup_longterm_locked(mm, start, nr_pages, pages, vmas, locked, + return __gup_longterm_locked(mm, start, nr_pages, pages, vmas, + locked ? locked : &local_locked, gup_flags); } EXPORT_SYMBOL(pin_user_pages_remote); @@ -3187,10 +3194,12 @@ long pin_user_pages(unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas) { + int locked = 1; + if (!is_valid_gup_args(pages, vmas, NULL, &gup_flags, FOLL_PIN)) return 0; return __gup_longterm_locked(current->mm, start, nr_pages, - pages, vmas, NULL, gup_flags); + pages, vmas, &locked, gup_flags); } EXPORT_SYMBOL(pin_user_pages); From patchwork Tue Jan 24 20:34:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13114782 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B426BC54EED for ; Tue, 24 Jan 2023 20:34:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6DAF26B007D; Tue, 24 Jan 2023 15:34:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 636D96B007E; Tue, 24 Jan 2023 15:34:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 43B436B0080; Tue, 24 Jan 2023 15:34:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 3550C6B007D for ; Tue, 24 Jan 2023 15:34:49 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 0F3A8AB275 for ; Tue, 24 Jan 2023 20:34:49 +0000 (UTC) X-FDA: 80390846298.26.F1496A0 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2056.outbound.protection.outlook.com [40.107.100.56]) by imf28.hostedemail.com (Postfix) with ESMTP id 5C2E3C0003 for ; Tue, 24 Jan 2023 20:34:46 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=KukUuHEZ; spf=pass (imf28.hostedemail.com: domain of jgg@nvidia.com designates 40.107.100.56 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674592486; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HnWdVLTP0vwT1IIIHq2HtqYFL5/FoBDjQ7USEOuU1xU=; b=aAbfGZBn/BKHvtr6rCpd8GrRROuUmHXIkdJkujvSGX38suchzisLdBDh1ny/Bv9qAekaK8 owxkhlUBwb+UYzDFGge/Rp4kG632A815T24FCSOm9gnF6ZXjUUTPyQSKbqDR6X2foyWxNo cU+gybz1/WPNjfcKM74iJ2j35LFoIm4= ARC-Authentication-Results: i=2; imf28.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=KukUuHEZ; spf=pass (imf28.hostedemail.com: domain of jgg@nvidia.com designates 40.107.100.56 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674592486; a=rsa-sha256; cv=pass; b=dj9rSDqd0on2pUltJXKAiqkJvQ27I+J1x+/UKzY0vu395kotI1ryNA3sOWBq0CtsD0vgAd UUw+Nv5zUPctssNGvy97dEEM49gWRYriW0BxVSH52ymJ+YhDnwlpr3zxidHO0G+U8shcXL G1c9Pv+v3PiXJc/VDwJhBoU0vB4Rujk= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nwC/CeDaKg3LeLXP67NLWiFmLTp0pqsISk7Jf7UYYMJ5JwyLXw/vFrvZZH5L0Pa/pgNitJyFdpWUA6uD69Kd+U6V5qMrIjrcmaGcMufwToya3dNXnTU+opSV3yaBeGrQ/EgRUBWm8V4emuCe4PX63ljPvVNSt2tkqYtogZ/RuWGxrM2x/tO5/0hloZKXEnRlfBhRwH7YtuKTexuo5Avt2szff3obnmRFkXNctgTJUGi1X3zc0Erpd2zKUpeWxqKYqWPVl5Axa3pPaVVIBJwNSsTTiGjm7nwoGJH7T1ZBpLGbZW0cvlRD7ujpxWz1UDi9pkqmHJqAbXQ4RLkaIKaZfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HnWdVLTP0vwT1IIIHq2HtqYFL5/FoBDjQ7USEOuU1xU=; b=Ou3qUipmI68hPFM2sQwhxTnFv1KQL+XoG/wsF3nEplzZUtoyio1Ehg1iXHFHpMJ3zz0onOhsriuHIa1uswvPgTB1xIMaxZEAfr3FNUz6XoBjurgr32B9qmsVL8SFFmEAt2JJm42QJnNBmfx03jUTp38L4zo4WscHmAWZPKImxR01vz6qu7ef4uFE5MKiMyCklAdFzszBl1ngunPoQ7dbsLkX9oPleuuDntAxc1WOaogUjgdY6TXEUTWjtJQGxphsqGdN5bSrm9VaSzTMmfjkzxU/7JLkEvjJ83KhVWsZoZ8G6kYHt8z0cY1shBFTlMJM29jlZCBuoH6Bv8YD9vYW6w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HnWdVLTP0vwT1IIIHq2HtqYFL5/FoBDjQ7USEOuU1xU=; b=KukUuHEZM2+149G7FdZBWWsyqhlGlCjYoNecanu7DbPZuZOHG6g2cvHNy8/ToIYr/SjjNK4RayHMqxL3EiiM8lXvwC7mkuRo4Ho/VnmqieBuIpZKkdmVBff1nLmFCAZ6cmexdiZel2dj/YQ6PSBUqqWpVr41cFMfJJiLNCBdZUFaCkt+nAo6VL83SCegPgwKqfBTqIuREwNHq8DuXVy9/cKF61NcBtuwsMOuQWgxTzDP+9Vc9Ai8rpja0MOM7Fabl8Gu9HrZEmTrb/fIcQTX0XcMcSVLt/FmVCDtIxCkEAF9bRELmp3xGXbKDr1bGFE9uDe0KjwrvDACjf2DNKrFlw== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by MN2PR12MB4095.namprd12.prod.outlook.com (2603:10b6:208:1d1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 20:34:38 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 20:34:38 +0000 From: Jason Gunthorpe To: Cc: Alistair Popple , David Hildenbrand , David Howells , Christoph Hellwig , John Hubbard , linux-mm@kvack.org, "Mike Rapoport (IBM)" Subject: [PATCH v2 10/13] mm/gup: remove pin_user_pages_fast_only() Date: Tue, 24 Jan 2023 16:34:31 -0400 Message-Id: <10-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> In-Reply-To: <0-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR20CA0049.namprd20.prod.outlook.com (2603:10b6:208:235::18) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|MN2PR12MB4095:EE_ X-MS-Office365-Filtering-Correlation-Id: 1cd5f929-71dd-4fa3-9343-08dafe4a6880 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7YzL88vfbB8yZh/n7MpgkgGc6tXNublCLEn4+43jEdwfDr+lOCkhg1o0B9U7WhNAHWsvSCV8gPXnPBzg5IGEwGW1TjELYpr/IqQDrd1BSJuGYI6OleTY00SHU/FMiQ36THVTwVeFy+wURLfH3PWmOiAC7Yo2oQFvmJHimmHTK6KO/6o9Iwc1mq9Kfj/c2IU4QM+OvWYL23AToOMYoYVpeT6vsU6RTvng5e9AHPTqnJGOjo9iVwLqQ3FtEByaleyzE1PeY+StxAGReNvbqUDgbKnMO3zs+G9dRwBzvcLtTwo9iVdAfRfyYmsx0592EBtBBYEOi189NmlCOn6WB0utM7EF54NbN6zCyqlLCcj5o+ZjCop0SZV5zaEJWwmVIUjTjMh/XO7JCPMlsxqFMMWUXW/e4MRooMhiC1Z3c0jPkWBGsymU29Kro18w4zOBk085DTb5rNs94qbKfJz+U5otOfpNLsVRUzzdN4w5I6grT+kBZ9aRWk7zisDdGApHXBY/jb8kPeUDxPeZz39+SxfPvop50jQC/dmDA69teIZFe3xMOUODyB2w87XNTOOj/jvb1mw3gZ5ksQ521BMWXqEDJ2FeTc84PMSuch+A0ZBbCBURiks5P84iJdka6r/0RkPaHQM70okK0S+8NemghwTZCziPmksbpAT7moY93KdEJvO46B0Mf68gHFFkn8cHlytm X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(396003)(39860400002)(376002)(346002)(136003)(451199018)(109986016)(38100700002)(83380400001)(5660300002)(41300700001)(86362001)(2906002)(26005)(4326008)(8936002)(316002)(6666004)(6512007)(6506007)(8676002)(186003)(66476007)(66556008)(54906003)(2616005)(478600001)(6486002)(36756003)(66946007)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: BnfNCtecuWBQfllsAcCg7Pp778sNe2WmbxDb+Hyao6bcxhFpisqCWD/CX2uDNVnovj8LDREQqTwq8J8VbCzXsOvVfqehPieSUGIpSSBgdxUkxPccbZ+qkyqgouTW2WaLioWhFxGSP2x0howYetUeHM18dC28gACg9CTFRkEBMpi/dO9yxgVA6dR5LTeD/G7z2RO0LNnWn01XIeLakZqziCQLoQXCN/cs0EORYF6go6tQBcdnjrupyVPtMlaUDXngNjf8qHEbQQXP6Pj2D3oDvUwWbjrIUajHh/frCwyvUykRB3LefsNBilCfTv+hMOWHTPJH9KOUP8sU1qjiP00QAnqJgCq/gehtjyZ/Mm825pbl7/1yoRAvYRNDCqw30ikcMKyKRRDD2/Mkxz5TTcmP5UJeLfGurKVMHCn4Nm70Bike8BUsQRksNNJehFYcAn6crmfFM52bzO9+X+IQYTHiwTOCCKxV8oi5fO2qMqKrqsXuNQY620iP+X5IB8fBCLdC12MqZ7IemtCORBGUs684MqlMnnKfHfGe4Be+nujUddkvJxfAuQKSjXf4Ly1inJPl3NLoaRYErPPXltGD9yjJt/dMbmig2tsyUQXRIpV4rcYnLHD17+Joc4T/YaJEsgDzU5mvoMPRSA/wsJJeZlRMUQChdZmzV7rHeFEwCIfu0AwYqGB4w0ZbPJkroiSmE4HiF5iG0jSXZGud21dTB4WPvRVQI/ktsWduermsF/yA/ZSYowcPwCfJYMFZLgdVtWP9L+/XOQ++eg5YIVR3fdc7xlpy94rpl3kz583UIdfCLenX/kOAc+NPxktUWDAEsIiGS8tyzE2719piN/PKYuycR5lrejr67rKDKKheBQLIMQuIhAETwVJtOpL+96Lc+FtntZeTCWKAOPCBGnQKODDru9K6dA/SSFo0rrYBTEB+MmRMuA9co8E2WtvcRt3sGd+QHgh3rpTP86GVEliBuuqqQlvs5tyFs4rPI0lxdOqpBj3OdzCVYG+1Vp5UkiRuZFaNYWx+FsCsT+wT43tfVmVuYN6iNz7upGTZWnf07rfGsZTSF5oDSPm3LBbR3bZpj9li8M/+kxKNoLklktBZvqIhC7uo9cVoMY5RZzYjXoy9P7QnpvBd+w6eNJ8+lqte448Gi7vH8eayuuUB/3OzxNwhxWOMHuf6HqrDwIPQEmTs6MdXHIWC0r0zXdM5oFCgGx9FWb+H/vAmZ78i+l2TYI8dU0prptISkVQXgfV/eOA3o71AyBcc2sGyx3dqykiP8T78G0TyUYMGPCEgw31EFrTn0mwWzMoKTt+gW+kXzNScjWe21Ancx/FGUYa7engc7d98ADmaweQtVXm0L7g/4ec3zfNenZZ2zydaeluFOCQwVG7QnzQZwVeg595xJf6/G2zdY/HvEpBrTZZLPb3zZtBpTjCO8sK9833PTz3b+v4eQ0auSqdEMNSJqAucmeYnEb32gkUfn+fsol6lX+GSe2PLzwK2zgZ0lGl4ZX2smABY6p4L3DMABQC4MtFX2W+kFaZC9p8KNbVXYyr1M2QVewAhcjscYB7GpyQPDZ9BiQJl2BXZCGi4peVfQe4b5M/h9oKM X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1cd5f929-71dd-4fa3-9343-08dafe4a6880 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 20:34:36.1731 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Q3icHdGwX7YBeKYvLcLoetY/bWouU+vXbswFdCQZUjX2IuivaX1V6l3cludU9u2t X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4095 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 5C2E3C0003 X-Stat-Signature: ubjzfsrqj9fqx9fyo8i134fto13f4m8s X-HE-Tag: 1674592486-463010 X-HE-Meta: U2FsdGVkX1/htQAuTpvazQySNc+mqxuYOC1kufNo3rA6qDWB543X4Y4HdpDzN3UwzCGy2Wo1g4DfwvFK0aabue6wX2AAbHTeaSDfs2pmUlFIleQN8WfSjqJZh2vesNF9o5/nYG8Arxcb53i+y1uYONQs42dm2Pu2Bkrs+kTheAcmSYgXN/JUcrbJus9hUxwjRS6K3+DvVZPBsPDa+YviLibxg5toYJo9pF+1+Rj3JCWpHfoiliKqOvO0GJIGaZm26DCt7Ri25d+7C0tL5G0b+Qf1OruR2Plio7YoOvWCUiE19sQcdUZeo/xUPodopdQpX4Lg2niz/k7b3YrnpLRTyHueByZc0/51B1WHfpFgygE+4rRVukaPGKbeXFRBBLdwtV68E2+Br1JdzgcZKRvFD49TrjvsU8G+FJpn6/Sk9dOgO2hp5j+dTGIxkEPQLDEib3oTM2f56190P30nVcCg7NKXLVHixSTKCUTMaSuCUTPrmu5aSbeFXpE5sbIEcP8aTZAuLsav/pML7vtuN52lKTimVs6pZkwjgcIYrtKs697nE8TaA9Y57EwXTnhQxXSsjWf1fvh9XqyhdTUfn9l1yFIpKLkGrxr/Cgcmc39xWvvie7H61FPzT5L5PQwOvZbKpD29OueR4Qw6CgNRWxnmaFRuw4biRC+izfQ5Y8EN26GML+058qs+I88ngkMF2fkFNrERHg84KE2YdYfuO1RJP2JpslNe+YSM9fZ7g3evUTW17kyHy/UxeJUJLOqpslRRHXF+NLUICWS+lAK6eulAlC7R2BsSN3a84ku6FYcHMrhnd8Z3u9AHFrHAXPhwjIJF4eEtEDnOYNpwqQtCKNGDghZ1T7xQwNkLNtAz8NUM+nldxDdzQwf+NR9HOc6qI+FnuF4fgqG7P3tfEVy3LFWYUdH78nW3ukIIwPYEhP+Wgve2IH+G+N52SNDasArv9BLoAB6/n6kwgoHtTIlJJJn cjPsWSLk l16VTfuIsiF+sIsC+Vfq4j/WNoUKXDQoJIjQMXj3pt6xNiFIPCdudgv97kzO8k5BXeJh0OzNZtdBI3mpJDeb5kIjCDscomH6SyOVuPK4fL/Wo1/mHusUk9ObeRBGj/4saHFyPWqk7zPCpd40ELeoHx1bicK5xK4mlBMd3tLYKdpwb1q16C0bLpSXmhZM1u5s7tfP247Ve2GT7OCcgH8Bnjt1Uuy9RDnORoST1FE+2v0T8C9L3g9WKemfCRXHuwjKjehDXVwPJmWZ9MbyNeFVigZLoRRpoIuahxEPjBzBgwP0Opum4Xdj4C5IOxgmnbLkgsrmy+q8Sp9VgXvJ3924z9JsVVZ9/uPd9CPFro2XYKdjNLr6kRtrkrtIVbA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Commit ed29c2691188 ("drm/i915: Fix userptr so we do not have to worry about obj->mm.lock, v7.") removed the only caller, remove this dead code too. Acked-by: Mike Rapoport (IBM) Reviewed-by: John Hubbard Reviewed-by: David Hildenbrand Signed-off-by: Jason Gunthorpe --- include/linux/mm.h | 2 -- mm/gup.c | 33 --------------------------------- 2 files changed, 35 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index dfc2a88bc4a8ed..a47a6e8a9c78be 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2166,8 +2166,6 @@ extern int mprotect_fixup(struct mmu_gather *tlb, struct vm_area_struct *vma, */ int get_user_pages_fast_only(unsigned long start, int nr_pages, unsigned int gup_flags, struct page **pages); -int pin_user_pages_fast_only(unsigned long start, int nr_pages, - unsigned int gup_flags, struct page **pages); static inline bool get_user_page_fast_only(unsigned long addr, unsigned int gup_flags, struct page **pagep) diff --git a/mm/gup.c b/mm/gup.c index 932a2339613c2f..8edfa66277fe20 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -3102,39 +3102,6 @@ int pin_user_pages_fast(unsigned long start, int nr_pages, } EXPORT_SYMBOL_GPL(pin_user_pages_fast); -/* - * This is the FOLL_PIN equivalent of get_user_pages_fast_only(). Behavior - * is the same, except that this one sets FOLL_PIN instead of FOLL_GET. - * - * The API rules are the same, too: no negative values may be returned. - */ -int pin_user_pages_fast_only(unsigned long start, int nr_pages, - unsigned int gup_flags, struct page **pages) -{ - int nr_pinned; - - /* - * FOLL_FAST_ONLY is required in order to match the API description of - * this routine: no fall back to regular ("slow") GUP. - */ - if (!is_valid_gup_args(pages, NULL, NULL, &gup_flags, - FOLL_PIN | FOLL_FAST_ONLY)) - return 0; - - nr_pinned = internal_get_user_pages_fast(start, nr_pages, gup_flags, - pages); - /* - * This routine is not allowed to return negative values. However, - * internal_get_user_pages_fast() *can* return -errno. Therefore, - * correct for that here: - */ - if (nr_pinned < 0) - nr_pinned = 0; - - return nr_pinned; -} -EXPORT_SYMBOL_GPL(pin_user_pages_fast_only); - /** * pin_user_pages_remote() - pin pages of a remote process * From patchwork Tue Jan 24 20:34:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13114783 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 791E1C54E94 for ; Tue, 24 Jan 2023 20:34:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B8F8C6B007E; Tue, 24 Jan 2023 15:34:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B1AA76B0080; Tue, 24 Jan 2023 15:34:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D8DB6B0082; Tue, 24 Jan 2023 15:34:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 642626B0080 for ; Tue, 24 Jan 2023 15:34:49 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 3159B120C59 for ; Tue, 24 Jan 2023 20:34:49 +0000 (UTC) X-FDA: 80390846298.25.8A53BA1 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by imf12.hostedemail.com (Postfix) with ESMTP id 49D964000F for ; Tue, 24 Jan 2023 20:34:46 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=tDVL8zKo; spf=pass (imf12.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674592486; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jJcEkWTumjbrcdOLZr40qLuiQxxPvJ1xACBtTcmfS1U=; b=xYSbGb9auZz65UtSM+JOh/OAcq1FiDx9omzPl+KesCHY95KMFJxEYKnWMejxYUV3OiME1v vnWadwrmZjmSRLhHlI6Bh1PtNEmN62A+Wg/TbhJDh+g3pmcGc8tjsdigtCF5yNKAQN3T5B VcvvDOCG3R9Uxlh3BLPVIiIrbfIg51Y= ARC-Authentication-Results: i=2; imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=tDVL8zKo; spf=pass (imf12.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674592486; a=rsa-sha256; cv=pass; b=EZWx0+H1YyX+tMY4FgjIJKunC0cC/ZJvMCmOgQlhPDECMNbF4goZfGxVNqPX6j+P4fuLRe M4yrCN+YEHyLy2GIVqt5zDenb2WdpMI6SwO2NzU6g+c0JnZAq7lbFE6AepZ0Eyt/Vz3b8o AAXtYdDbO6I9Qt+sCcYdx2b052pgm3o= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cHq/ro606SCfoTbN9DCcX1rhgE2PZKwC0eOVJlMYwYID/lRx9afGTAsvw9JxAlXsXZKOfmTQ4q5hmr2hhPzy4bjSGZjt+g4Iu2DU85oUJ234VPVkiv9ahOicp2FdcYOWn9QF4Uuv8LrL3lgM+ZHE131pfr/rX/M6oa1PPdMULZdssnFKqDK/lg6ZyoKAE1oZzdkWdxzC5Ifx4SX3NPbZKKu8Zr3WU3wpbnSWwCr+KlL6CykFtEBibmLUx86XaHZGXI9K/TvWFj0JYTC1VUUMYExNjDnsRc7eL9eqzB9UntHyScD8ribqL583boYv+gwOnZQvVkTw9aZwLtVGdiunWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jJcEkWTumjbrcdOLZr40qLuiQxxPvJ1xACBtTcmfS1U=; b=Pk9m5A1aGWArkuCCPG0iot38eeEc9p60u0XqmBUeLJnWOfQhqy5tAvulI8XXjrX6DavMopilbBs0FoTRsJkkP0tXcxp8JsQMXulWaV3a1jyMQSPjeu07incawT9WBz+wF6uUFSJiIReo3sILiPGw5T2QFXeREVvdMBeEALp30Tvi4amiAaeFSIiD4MkMsn6oLhb882pNHBebpHvwipHWA6NE60Y+jxMN9td+k18/nxTZ3CO13nwKr/l2TBnZbQlWfaboaKAFwBOTgZ4Lgig6FodmJFynwxobif8BBCjRv5o9w36JyvEBojYH3liJt7ZY+EKaqiGYwBh0tkWQ+SZ5/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jJcEkWTumjbrcdOLZr40qLuiQxxPvJ1xACBtTcmfS1U=; b=tDVL8zKo50/i9NBmhD1tNfwF9aGlhHYVdQ24OQJE4LmaUqlmuj0fXs8UkJQh06OFYpWzeEJbjZLXPjD3hx3SQ0WZj5tcYmhkBO/KmRQcOHPqhuLSNdH6gJoYmlqEBqq0VadM/kkgmeIO+j9HwSX5iGAxsMASxTx2r3RCA66viBWWjWasxYWEdFuO/2agoJGC4K7sl9qGKDzt/vxh2OVPtAXkQrMZOFrX0wQqwbDLT34HWfOPi0NfCy8Cle4rpg6x39UQ9Jj0OKB0Hrp+xXWxAzTaHIEGYg+P4eByYF+J2qcQFmev8tSw1gflgn2F7tqtPilEErUC/YpvvtloQya/KA== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by MN2PR12MB4095.namprd12.prod.outlook.com (2603:10b6:208:1d1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 20:34:37 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 20:34:37 +0000 From: Jason Gunthorpe To: Cc: Alistair Popple , David Hildenbrand , David Howells , Christoph Hellwig , John Hubbard , linux-mm@kvack.org, "Mike Rapoport (IBM)" Subject: [PATCH v2 11/13] mm/gup: make get_user_pages_fast_only() return the common return value Date: Tue, 24 Jan 2023 16:34:32 -0400 Message-Id: <11-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> In-Reply-To: <0-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR20CA0038.namprd20.prod.outlook.com (2603:10b6:208:235::7) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|MN2PR12MB4095:EE_ X-MS-Office365-Filtering-Correlation-Id: 39a4a4a9-330b-468f-9931-08dafe4a6842 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3AWv8qFAQVHdcJQ0FiRDQBRQ00aXW+YfblrGNsHzUensYtf9Hj83wsUJJr8y8kSLJlLeigIIYIDN1+IFp4hLfAX4DZThlsBHWuV7x3iFw/44YArvIo8T2VZ/2AEbM8MQjQu2pzurz5yV4UINWP+RpxwWO1jziQbju2xHQjyXX/1tvV27KA74doI99HYupgWDom7UxDh6Ii3qzWxfuEThzCkUmZruS+S77lJi/Pd1zOS/uSGhXSmkmZx3SHigfcaRLtiQ8vL+Dr4np2zLq5lrcuTO39Ei3smbOiiYa6PLe0D1wFHzimTvbFJ051db/+7BJNC4GNdiDITVQJ091r7vBj7UoGGAb6FEmwBh2aCJB2EG0XTWxL0H1aEwhnwzXkkOqr4ir+jlCALQGIqynWpte7GRTb4HxGFu/dszqONfO7b/sclVeP3Ae7tWyqYOcRB/ZSXr8etkgV72DML6rhdsDhpd1vc9DnAvPHtWkdzKlWpUT3RBQTiuvpoPlenNDPsOdEtnm+aJc1I+KiXNiHODSKblJ3N38Ilh6TuHGumnnBXLPZsTFUjCK7yveqbWf2geVYsYBzR/UUbSopesmTaFBsmbtMsfmLNxZ2S/uKIdtYpA8Ft/j3qtTQxS/C74a/oN0iLkj2ReQPvIo+ioqlhwvkJHrHP1OhQ92Spv3/N0WI5ERIyjQRlDK7Rhr49NFLaS X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(396003)(39860400002)(376002)(346002)(136003)(451199018)(109986016)(38100700002)(83380400001)(5660300002)(41300700001)(86362001)(2906002)(26005)(4326008)(8936002)(316002)(6666004)(6512007)(6506007)(8676002)(186003)(66476007)(66556008)(54906003)(2616005)(478600001)(6486002)(36756003)(66946007)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: TtzOCsPcDRMVPwhpbvq5I8f8kwXo6DSyGY+oxkSJpxuqkZ/Ct0JmeEwiOPdzNh97et21uh5jH/CtglYMxrjdKQKbyaUxuTaY1jKRc50izSwmHviaFlUN7KSCLEhmx2o09gAXr/jEFUcdFdVggOPETuQwTnG/G6AvCEi1iLG03fxnQW4Pn3BZdsG+VFUSNhM9arI4Xu5cozcblv0REvaNZLJ/fuVy8zsT2yvWt/O9/cWfOj/KE8thz4hfe5T/HhL/fgHsS7KigDo3ygxhTRJBQeQtwbX3R7+DgdfPgJUMg8QEIimu9mskfMoOZSRKtggElsJXEcEP2tz9uZqAIKRPByU3VIfCGX78YxxRyS2lBs129CbJfInrfcxI/FZbyyAxcXYKY/sr53ULpkxwiZ2QH9UtuoYQaD1XGjSYzs/JOs7b2JLY4MURn+WTnu02iEQctxRSvxM1IbI6d6VuUTBf5UH3hZpdf1U2X5plpVCQukz8iaIUNl1P7or+V1RymCuMAptTePc7QrB9z9/BlH6H9t1ZBjQYlDEHVuEkQRQ+FQbmUVEyrh01VvX3LjeeypmDjsc1KhscxAjn7dNp74uATIIN6LWT0BSRnhCAaQ7CcA3kdNeJplZ14JsXoV0LTaCAVLlWRk5r1tXzW3kDEWAr17Q6a1hLoPJ8JaRv2SLwdXbVh3bjEmx5ha4o50jZLXbbBHzWcQ0CftkOGYFcciJbPyA4YqnY5gE2wbb4fSiVF8nnNU5Z7Q8vV77cjvOFQbONAhs28hmyY4uPrhGmEs67dqrim1cgQ1fbE7mGfV7pQZCuWs+aEjox5KqqI57/fJdMgTgXNAzqNgMUvczJ27T+Ww8BXET+qz8WK2qxHuv2scQUk/3z6iqZNuR2ezP4gJKoBb7YC45JyWH6eXvieNw18VspE9PRHbnT5HIhTcwdrsi1vODhOjmgCTZSILAgtIC3f6rBt1/86/PkEcTpIkyt39YPxUWmJGJOKEUwJhwrXbHytwF40r+g8lSx69PF+qsmfvq8cbiJ6JQ6DqN1dGOL/SYk55F0/l4FOpoi5Sl2+N+5nzroaCFmW5nHTvr3drEvhGMd9mdfCjYRM/NBKHgMPJHIQ2yvYuhchQkaDMnGWIPSUD4p4qSqofaYdm6CbYhu91KuWHg7SJq9A1EYBkCLG+U3RZ4JfbeXFUw5Q80LdxKZkKFwwZ+TT0lWHA349hat3i+KakCnrOz9eiQKERLOGWhcQLxxQII2tZvcMyIe5fT1qLkWUdcVCeHfcyu56c6IXLVeJUogtcxa2zxol6wDWIiND3ABm5PY2al7xmrNYUlNaL/kGKySRk1fiHOBimCZ1BCaW46FoT7IbvrCV7T9VlLLwcj+euooBlhzOJIm2INLAL8BARSKjkbmGvdy+cMT5a5//jfCCIDWJExpB4lYIcmWUZRnzvWc/oXASN6GSb2JhKOsoipB51G+LQoJFSxn1kawRMgrfSZGbRVOeGJ79GNVyjKJbUHPZF+g4QZhYySLgidb1yRYik0TxbeIvIshOmBKM+XzRJTahPwmAVFGp1cRLwIEwcvk8p5IK23I+BfKp1osTnR6/lobEIq4nmW8 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 39a4a4a9-330b-468f-9931-08dafe4a6842 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 20:34:35.7671 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: MYcXvDnSh6fSjlmzkMqqCaR2o+gUQ8crPYU773OisfmbAw42ddX4vntBGM4Wzk2F X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4095 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 49D964000F X-Stat-Signature: gnuujyuninnkaqfiiipid3jprix31bwz X-HE-Tag: 1674592486-427522 X-HE-Meta: U2FsdGVkX1/BJYOJlbTqPFJuXFe6G53dVkMSCJvqm1tZbJDQkOnN1swv+40Prj5QBrU2LvOX3oGEJyA8s0t+QRFnQoLGloCPGaXqh1JI4Xenjk8QZkyTOEEoX8n3EzC0X2raIDDAu6X0IsIr7iIzdi8uM46yqiXF475yJ3VTFRhG5sLfksQ4kpKBBhUMG01m9Jsm3TAE4iEzLchelu94xhFakVUOCW9TB5oGtlMxEcLzX6OcZtM11QBV9v65iaAHvzBzAaCmTG6Em+chVbtO9h3CI4ejaEDdn0zbT2cy0lZHy0wMjs8OqawEBS21DyP2sOkX8xfQu7ajqc4OtFhiHyzGoe8yA884XQ9/d7MD9dCKpxMZ+UBYiyCr9Kpnr+tahLL8uhZEbpuq4SNIqSP0nfLslMvsRPfgADuIdHBDO1tevsaAiRSJd0kpMVj7iF4SyN6Rl0ki3HlfbdXXG/T5fBFyPDc8Ts99PJQ8ZuIfX3e8aqW1jYmwzFhSo2l7kR8Z8ROf3QTT/ddqy1m4UshccJqBbPjj6IpNdeKXLifXX1rCy+v3zZnkvt+tZQ9nnCmAiyLBsqyqE5hLRRhHkKLBLDwWiiKawxoirxXEgwsVNtU+DB5iMNyI/IZ+4wf44LbvTTuEtCrvhSnhLYuph751Ex0axx7gRwGqJJxwMuSgaHMgjonHovrtHSBpSEc6yUMwNNcgguh6Fm1ehyi5ADugLcGo6dGkvcjyESPLwYIXoIRcVfbSq6vc5J6KgsTIlPHRQdhGFj+f72cp7wQzN77D8GXhM33tcR5DsGuzkibCTRbgyeAG3VzoMUA43CtB6R+33zburn0q4h3rS9HrXnzDPbwrLVaxEb5dCeT91pNMaW4C1olwaDKAAwVgFKd/bR1j8pmDkJMV1UnRwT8gORTPG5cwxD/d94In0RC8TUnP3hxpch0IFKcDvz3HgvbsXjouGUP0n78AEQxlKHLkXSf MBB4e1QS eVlzhptNXvTB2jLZV/pLPDBo8Z67C0zINAgZzpaXBm78yICy5rDgVbdmuSwaeUWIVw5IXMNhW5rhX2kDJ35zObti5gVpwDtZRfnO3JXF4I2ANGqsbM14kBQIDMvMOlinRmcM+GZZcgvjaRZfNja9QODCd57jUtLrbeQen9wbPSmIbNhPJKKRQazb/zThadHHa+pLeWjKX56KHNQQCWTclRbB5K24R+/BtITYGmDZZq/0QEOijuLSfMj+Hk1DFu+Vpv3NVoyMVe5lWjuaS2JxyM2hTobvGDEBlKgOFV180RbnqaH1+PcvgRg5j380iGyaGhVshu2DnJrCFdMbuymUlVWURFyz0u7w8r/cdqj8RzKV+0OZ0QrAOSXL0nw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are only two callers, both can handle the common return code: - get_user_page_fast_only() checks == 1 - gfn_to_page_many_atomic() already returns -1, and the only caller checks for negative return values Remove the restriction against returning negative values. Acked-by: Mike Rapoport (IBM) Reviewed-by: John Hubbard Reviewed-by: David Hildenbrand Signed-off-by: Jason Gunthorpe --- mm/gup.c | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 8edfa66277fe20..05ca9b0a06c8c5 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -3005,8 +3005,6 @@ static int internal_get_user_pages_fast(unsigned long start, * * Like get_user_pages_fast() except it's IRQ-safe in that it won't fall back to * the regular GUP. - * Note a difference with get_user_pages_fast: this always returns the - * number of pages pinned, 0 if no pages were pinned. * * If the architecture does not support this function, simply return with no * pages pinned. @@ -3018,7 +3016,6 @@ static int internal_get_user_pages_fast(unsigned long start, int get_user_pages_fast_only(unsigned long start, int nr_pages, unsigned int gup_flags, struct page **pages) { - int nr_pinned; /* * Internally (within mm/gup.c), gup fast variants must set FOLL_GET, * because gup fast is always a "pin with a +1 page refcount" request. @@ -3030,19 +3027,7 @@ int get_user_pages_fast_only(unsigned long start, int nr_pages, FOLL_GET | FOLL_FAST_ONLY)) return -EINVAL; - nr_pinned = internal_get_user_pages_fast(start, nr_pages, gup_flags, - pages); - - /* - * As specified in the API description above, this routine is not - * allowed to return negative values. However, the common core - * routine internal_get_user_pages_fast() *can* return -errno. - * Therefore, correct for that here: - */ - if (nr_pinned < 0) - nr_pinned = 0; - - return nr_pinned; + return internal_get_user_pages_fast(start, nr_pages, gup_flags, pages); } EXPORT_SYMBOL_GPL(get_user_pages_fast_only); From patchwork Tue Jan 24 20:34:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13114789 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E778BC54E94 for ; Tue, 24 Jan 2023 20:35:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CCF346B0087; Tue, 24 Jan 2023 15:34:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BB5A36B008A; Tue, 24 Jan 2023 15:34:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 795E06B0088; Tue, 24 Jan 2023 15:34:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5CF926B0085 for ; Tue, 24 Jan 2023 15:34:57 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 463FD4037E for ; Tue, 24 Jan 2023 20:34:57 +0000 (UTC) X-FDA: 80390846634.19.93ADED7 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2056.outbound.protection.outlook.com [40.107.100.56]) by imf28.hostedemail.com (Postfix) with ESMTP id 96ED1C0003 for ; Tue, 24 Jan 2023 20:34:54 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=N3pjiIOR; spf=pass (imf28.hostedemail.com: domain of jgg@nvidia.com designates 40.107.100.56 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674592494; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TandkHoeiGygDdfgH7afNs02tnvU4eGp9W/Vp5Be4Fs=; b=njSrt7jhThKmCgBUUZozkDFBjIJRyLL4T1joNBs55v0LfQTZkwEiYsfAP5cKUObjNZV37J onLzRtxCdxG679olpxYUn/j6B9PN0F0qQouSiCNrrm1nJWhIzJS8hnzFQDdfElsWBH9GxZ FgwY0HTC0CYsG62PEeAK88NdWTM0F8M= ARC-Authentication-Results: i=2; imf28.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=N3pjiIOR; spf=pass (imf28.hostedemail.com: domain of jgg@nvidia.com designates 40.107.100.56 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674592494; a=rsa-sha256; cv=pass; b=iAaxvDs0vIJqRG3DqkRiFtOfzhe7w/cd5H1Ypp/OVDRXOGF0hvEMgqWyxSKRxt5dK1/0h/ NkOaKGpqUtPIYoJvjm/2GPddU3GarQ6LitmvMtlPmknkoLiQS0tfqwuD1Rc7Zuh46PjRnq PpRJUb1f5brvfMYBFdnDlSEHZdEaxCM= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n2uPHriNVRb7gWHxJGdRI/Q/SSD/ShvKHCdehDtJB9D6Ie9zX4BMYOdeXGPrgF7xkhnVM/i1t3/pV13SCsxVs0P1eTAZPg/uiM/RF2KHDGE5E3gzlxeVvKGurx4W+ax8hVt0cmYvmXE6UGaa9EdQ2GwEcw4dW6qLU28DRMIXG1vTKpALQlslBDrEtn+aZrPnP2M/rzyNJKPWymrMsb9mjuyldpgidtvYGMJP4Vszm/ZS3SkYMlFxt8wa9beTwIpWSCY3JFKvhv/Xaik07l0nnPxw0FTg9JzZlfYm0FsJJ14KdB8w/W7VxUhcpjzRennsqjREwxI3hIl1glw7oPvccg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TandkHoeiGygDdfgH7afNs02tnvU4eGp9W/Vp5Be4Fs=; b=dw6+CIuO3tzopDF+9RYFZTbCKlkHSgjrs3m44vB+aGkmorgDSbewBjlzJXi0DcRBqHGDvJC0ouLZblxi7LdwTyOxiVPwhvxNsi5zSetmE74SUflwHXVwP8rkYhfvKfRA5xORMtWJ+dYg9biCPSZECx4x13/Fv4Lf1lg1VxPjBDzpx1DtHOeAsiqB5wHev+BuQxh7zD9U90LM8/7Vk7NV6LXx1zMWBW/MMtwkoQNO7XghDoGWWQ+rJbycOHPSVlnTLDybg+lED66rR9N6QC4TolT/8iYL7DX+68yGZNQV7pi1sNApL9PF/41lJcu+at5yYFWsh0SJHyQwD4gzFfNFEw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TandkHoeiGygDdfgH7afNs02tnvU4eGp9W/Vp5Be4Fs=; b=N3pjiIORRdhbgOrcuJ5oyHIO80XJRKsuYlVrkymTR+tEmHgedh+eEVxEMu6OLWVonkVTU6O+SB5V9KfSjF5+6grF8DwHaVMhqv0SgcCyVilbeLCy3kEsosK6PMFABVopY8GFd9jJ1DoVOMni14wu9R/rgmyH71bCGbqJHVU/+PyjuKSVLQvnmv85Xcwk2veEaJ88spWkbGOHkDt76yAQsMKHdyVTNbQp2wKYKSDzMkEgM7bB3r+zgS0xDnKVROyDoA2AFUQ3ZuqcDsExKdGqN+KFEwkv38agJ8QjBaV6WUW9BUJ622fQ0PQ/zt/+oNEsPsfwGxY4k0/+c0aQNoN+xA== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by MN2PR12MB4095.namprd12.prod.outlook.com (2603:10b6:208:1d1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 20:34:40 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 20:34:40 +0000 From: Jason Gunthorpe To: Cc: Alistair Popple , David Hildenbrand , David Howells , Christoph Hellwig , John Hubbard , linux-mm@kvack.org, "Mike Rapoport (IBM)" Subject: [PATCH v2 12/13] mm/gup: move gup_must_unshare() to mm/internal.h Date: Tue, 24 Jan 2023 16:34:33 -0400 Message-Id: <12-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> In-Reply-To: <0-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR02CA0030.namprd02.prod.outlook.com (2603:10b6:207:3c::43) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|MN2PR12MB4095:EE_ X-MS-Office365-Filtering-Correlation-Id: 21fb699d-2fe1-4651-a0f6-08dafe4a6927 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xRtf3/eUAAXURlxh2txpZ9NElLurjm9RC/C7Lq2sKRkxppxVVBgBfYria+OZR2oPBxMe0HhsoBFiJNioo8kcjSkbwpBBY+bzK95bJl2LW+dEKON+zYwd/fbhRwK7zZOIfjI6P+VguaD1UT5Ygf8ju9SeBfVS5RagNnd8XbKIGNAV/SOKZxKuXrwuGmmk3Lhrls8jUulJ11WwqtI6dZwFpRzh/brUCjFk6VhICbUzSNr2h9/mBcm19Q7DjPsRaFP+zsrmSk1TsLhi0ZRjVJSm2lR9M/Y49xUBiR5ffileU6NHwDvMHqwLk6tiCnQM8P4FJrF4aWGEbkP/7CJwYgQFWXYL+uAoNrxND1DFV7LDiY6jOqkwjAXVYs8uGSDQeTZuCqzIyCaIjcZDs07O2r6mpyhocOKOn6LquP1eGxPE27VmBoPUdBTed65x2KCbdqT4K12xrR8Ha3VWkKCo461eAwCPVeokOcMfGBGUeT4h2gYq8mMbfkO58gkR2KRJXsH81ENNw3qxIh3ffNE4/QAGmukIrtWgfNHyWu1vgu76iMeMVlGdiPJe48vE1833NQTLgp4RjkbHU3rSHcFB1CqcjDcPCqwdRu8MnmLH19+wF/oI7g03NHBV9nfz/7mhUtaJ8Z+RgQ2axhkjdgvbi7ZYNo3yNYPCR0grP7Tb4nxXlqam+h51K6mOzg3CUYLWEAxD X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(396003)(39860400002)(376002)(346002)(136003)(451199018)(109986016)(38100700002)(83380400001)(5660300002)(41300700001)(86362001)(2906002)(26005)(4326008)(8936002)(316002)(6666004)(6512007)(6506007)(8676002)(186003)(66476007)(66556008)(54906003)(2616005)(478600001)(6486002)(36756003)(66946007)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: BXHFa6QrwJESZK6eiSEN08Wmm+Gsu/Omc3lFSgW02n5mDHHHamMQZwqG/0MWWCozlXxgSU9jM6UfwSdoTsuuDejdbNBycK0SZeja+G9OtxkwHZ6pPLFQh+4NHjew1SO3vYEzyHkvUKOzCoTzAsqGsGFoUl9qMKVuAy4xtDCO/Ca12I+eh9J6unZS3G3e601sZ9wV19sGvyf4DWJaNspQAxHacJy2a7VEb4/CbMDEFWL9CmD791sinwXkUG3k87xBkQ2K4jlS8WdfO7Q88emMCQQ8x66hLCNINe6KMEbUyVqJ5Dt2QqKMFGphGqyIyTg+P8q9zuol5ro9zjSnR7jLEUmXBjP4zTBjxlZoAka/vdk8GVel4Dy1DRQSHo+LaLp3bldq9DXCQWcRjqhPv8Dbxrfc3qQJ0pEGpMF4Is+AicH5/YIfBsFkwsj7GFDt/W11PzeaLu27D8m0QjvM6y7M7mC4aGQznwJH1WR4O+qT9GH4qxDZZo69+xRQ9fWspPV+9Kv9P5Q0x8lrP3bS/hT4Yhynzpcuz58aDaDB9/CoevMKQBM1VbrUic/GUqJYl9cZSbCE5XFflOtaEkoakswOfeJHR40e03uYwKKrpvSQ4ANwRaQyIc+aUB7pPTk4N6x4pf0p81G+YoDgn1AjC8iozG8jKuMqm13opSQ2lPm79T/ywlFfymtZZZYcm8N5EsmkuS1RXECfK/ZuQGXGCHpaezFBbs+6CFantSRSN47O8it1jm7AgTy0Uom3mDWM/YJKXedTMvaCaqsuvqT/cebeI9T8/zrZXmU4nG5AG1Yke8tnMbaFuS7+KdqISTlGt3Ekq/1giXUmUSJosPMrTDg4eksUo17BX53lV0RvgJU4+ihHq/ENQjsT3DvHHZUVs48ERHm/biRgV2xKL1XuIOo/yTUj2Sz3gxiJx5boaRxQiCJ0XjCZ/4+fnjuwI4fodcoMHwht1uhBs3Dh6uBEf6dR6CbGGwNdw6VfVpDiGHzcEOfPoTlXS4WiB/CV1iU7Bbb4q60IKOiMiyIR34LDTHycyRAYMPmlt633MW2YsmbmJG/4o6/tGdYyTnXvquTXINGAj+5557rqeSzvaOzk04j//AX2y23Wm2BRPi5OoBm++VLboMDQp4q4I2xMLhJMT5v4jeiWTHPodTymde5h+hG2bRpMgTe8d3AsRKTYtnIa75IaFwRF9XVKstuEqeFChN3OzZjBlKshQyAchW1EyJTaQAeWrOcoIthIsuCax7QrcwmeBuyjTREBj0KwRX8qHaiStk1CITLVhXtlLmWYDCjwFgdi304J9NXVT3sHONnaLFSgerAYdiJAIAnUdT+a/Zvx1pkR17ok310AboosmZ0XKAFShvBjK9JbBC5WA5yQDsZ/ljUnVqxSfX12KQoCEynWAlMLAKoEP5sJtAKUDmX96fXSqy7CZqQgUxrXJTi884SPffCN51L9TG1Lm+Cs0ucexRzrTZIPFnjcYF8smLJ8QlOA5p3hnsU5hKVxu8Gf4OX5hOTGr9PQRajL4/iyJwRg4OFpsw3QnY6JG0U1c4GpHbXtK98cC3KovDdPke7TV5ghX3ziIx+wTZ1rHoh6Pn2d X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 21fb699d-2fe1-4651-a0f6-08dafe4a6927 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 20:34:37.3148 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: vZefzCyIe8TGWmoHVjhk4q87wdzKuvSFhYdpUCnyXSYRwDYtI6PccZt7Oyk60P7b X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4095 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 96ED1C0003 X-Stat-Signature: ca97bnkw8ex56ps3o3kxn3981maoarai X-HE-Tag: 1674592494-797855 X-HE-Meta: U2FsdGVkX1+4gxfo0H2KaexYRtYirGHXzawQj6Sk0bD5F1ten00JEbR9cR8/NG3ezOGIRWA1/ao26SsT+QbyhR5CUWOEyCugIr0bIcEc2Xv6gq8fYLv8gcQSkuseRP3156PzRcelUOK1/hTyZY+BKbfDAtPqnr4HOWuYPnABBbnrEd9ONoFNU6PQbe+KSwsMAogk/sG8ZR6L1Cw/RcTSc+bILSD5AV7XWEQT4/KsLC08LX2CH9CHGDa+i5vEz/dhKHTIudPvtsBcfLQ7Om7x/RxS8Bd/5jrn8mFXg7c7aOXUUvhPhiuOVzFy8qBs0N0MbP0Sux0M6NP0F69ij3zywS/aHbf1nMlqUGVAv6qgGcBTWYQXbi0xOd0/PKj3Rr40R8PMhymFA0FfzEigZCJy/UqfNFztKg7Q+QKMcbHE4nMoVoNEPNCrgkekPHRZEMNxxYF0H/0VkVqnm8v1q+NuJ0OyMyijUYdRvpLXwrRxe6nW3sEtpSj5Dhc3edyXAtpC8VQDfx+Pn5x1YvLUeHds9aXTS+UPB2xO6TCsAXOafkq3O6KdI56KkEmQq3Q7Sq88yVg47Ljmr7bB60OM2NpwVaHowb5Szbwot1RY02TTQ0soESc3r+Qd+gEhZJ3U/rrnYNywc3wsDcV7X5KL6YHd1xrQCWGKdHKeJBUkCG3D87XiJ2dqjtZqaASer6aSOQV0dkEYvtdrjGGi1d6CIf8I+zxwK2XyD6kqQdjv9u95fvKXnl00fF/WbIte4Brxjh3TR9G5NmO4LXrD6bVhAcI4qAKMFUe3ACHfHq2C7+GuxV/lHCzZbB/0hBRZr73vMp6Wczvj94vZ2fOue8Wfmn3GeMT+rhVdeZFRs+39QMjmAME5QAyV+l6F/TF2UboylCZEOVSIjI1BjbSmJB6fDQlQAw3ju6JQSLzWnd4oHq117MjZwKMCeuMFvUv/XA0RuigNepCT6Pkyqew2SSUo8mV nMlkCzYo s7eQ/X9G2qKQG+TP5OT5za6Rf8qkuctM28WwuJ4zDz+fpTNa85unyffb2HxIfwXLZbKDV0BhCJ4vyrINM1DMwEB8zBJys+fdiOFuabtCwaEpVLdbbSbDNAykamT9u1tjScwrkkU4dpm2bE9butZz+NHh/fj+v4Mvhd24vWWWr6Kw2cIFqS7njdntNOPlcdsdGq7+suTiucQRxM2GXRNoYpJKnZSbnlAWhEpmh79hx9mjCZqyaVSIquVjCcrvkbA3+HYi8XZKw0QiUGu8I3cDXH2A1mYUu9j00DiMP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This function is only used in gup.c and closely related. It touches FOLL_PIN so it must be moved before the next patch. Signed-off-by: Jason Gunthorpe Reviewed-by: John Hubbard Reviewed-by: David Hildenbrand --- include/linux/mm.h | 65 ---------------------------------------------- mm/internal.h | 65 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+), 65 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index a47a6e8a9c78be..e0bacf9f2c5ebe 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3087,71 +3087,6 @@ static inline int vm_fault_to_errno(vm_fault_t vm_fault, int foll_flags) return 0; } -/* - * Indicates for which pages that are write-protected in the page table, - * whether GUP has to trigger unsharing via FAULT_FLAG_UNSHARE such that the - * GUP pin will remain consistent with the pages mapped into the page tables - * of the MM. - * - * Temporary unmapping of PageAnonExclusive() pages or clearing of - * PageAnonExclusive() has to protect against concurrent GUP: - * * Ordinary GUP: Using the PT lock - * * GUP-fast and fork(): mm->write_protect_seq - * * GUP-fast and KSM or temporary unmapping (swap, migration): see - * page_try_share_anon_rmap() - * - * Must be called with the (sub)page that's actually referenced via the - * page table entry, which might not necessarily be the head page for a - * PTE-mapped THP. - * - * If the vma is NULL, we're coming from the GUP-fast path and might have - * to fallback to the slow path just to lookup the vma. - */ -static inline bool gup_must_unshare(struct vm_area_struct *vma, - unsigned int flags, struct page *page) -{ - /* - * FOLL_WRITE is implicitly handled correctly as the page table entry - * has to be writable -- and if it references (part of) an anonymous - * folio, that part is required to be marked exclusive. - */ - if ((flags & (FOLL_WRITE | FOLL_PIN)) != FOLL_PIN) - return false; - /* - * Note: PageAnon(page) is stable until the page is actually getting - * freed. - */ - if (!PageAnon(page)) { - /* - * We only care about R/O long-term pining: R/O short-term - * pinning does not have the semantics to observe successive - * changes through the process page tables. - */ - if (!(flags & FOLL_LONGTERM)) - return false; - - /* We really need the vma ... */ - if (!vma) - return true; - - /* - * ... because we only care about writable private ("COW") - * mappings where we have to break COW early. - */ - return is_cow_mapping(vma->vm_flags); - } - - /* Paired with a memory barrier in page_try_share_anon_rmap(). */ - if (IS_ENABLED(CONFIG_HAVE_FAST_GUP)) - smp_rmb(); - - /* - * Note that PageKsm() pages cannot be exclusive, and consequently, - * cannot get pinned. - */ - return !PageAnonExclusive(page); -} - /* * Indicates whether GUP can follow a PROT_NONE mapped page, or whether * a (NUMA hinting) fault is required. diff --git a/mm/internal.h b/mm/internal.h index 0f035bcaf133f5..5c1310b98db64d 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -854,6 +854,71 @@ int migrate_device_coherent_page(struct page *page); struct folio *try_grab_folio(struct page *page, int refs, unsigned int flags); int __must_check try_grab_page(struct page *page, unsigned int flags); +/* + * Indicates for which pages that are write-protected in the page table, + * whether GUP has to trigger unsharing via FAULT_FLAG_UNSHARE such that the + * GUP pin will remain consistent with the pages mapped into the page tables + * of the MM. + * + * Temporary unmapping of PageAnonExclusive() pages or clearing of + * PageAnonExclusive() has to protect against concurrent GUP: + * * Ordinary GUP: Using the PT lock + * * GUP-fast and fork(): mm->write_protect_seq + * * GUP-fast and KSM or temporary unmapping (swap, migration): see + * page_try_share_anon_rmap() + * + * Must be called with the (sub)page that's actually referenced via the + * page table entry, which might not necessarily be the head page for a + * PTE-mapped THP. + * + * If the vma is NULL, we're coming from the GUP-fast path and might have + * to fallback to the slow path just to lookup the vma. + */ +static inline bool gup_must_unshare(struct vm_area_struct *vma, + unsigned int flags, struct page *page) +{ + /* + * FOLL_WRITE is implicitly handled correctly as the page table entry + * has to be writable -- and if it references (part of) an anonymous + * folio, that part is required to be marked exclusive. + */ + if ((flags & (FOLL_WRITE | FOLL_PIN)) != FOLL_PIN) + return false; + /* + * Note: PageAnon(page) is stable until the page is actually getting + * freed. + */ + if (!PageAnon(page)) { + /* + * We only care about R/O long-term pining: R/O short-term + * pinning does not have the semantics to observe successive + * changes through the process page tables. + */ + if (!(flags & FOLL_LONGTERM)) + return false; + + /* We really need the vma ... */ + if (!vma) + return true; + + /* + * ... because we only care about writable private ("COW") + * mappings where we have to break COW early. + */ + return is_cow_mapping(vma->vm_flags); + } + + /* Paired with a memory barrier in page_try_share_anon_rmap(). */ + if (IS_ENABLED(CONFIG_HAVE_FAST_GUP)) + smp_rmb(); + + /* + * Note that PageKsm() pages cannot be exclusive, and consequently, + * cannot get pinned. + */ + return !PageAnonExclusive(page); +} + extern bool mirrored_kernelcore; static inline bool vma_soft_dirty_enabled(struct vm_area_struct *vma) From patchwork Tue Jan 24 20:34:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13114787 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A632C54E94 for ; Tue, 24 Jan 2023 20:35:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B6F26B0083; Tue, 24 Jan 2023 15:34:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 099BF6B0088; Tue, 24 Jan 2023 15:34:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C50296B0085; Tue, 24 Jan 2023 15:34:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AE6AA6B0083 for ; Tue, 24 Jan 2023 15:34:54 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 86CF980366 for ; Tue, 24 Jan 2023 20:34:54 +0000 (UTC) X-FDA: 80390846508.10.EF751EE Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2056.outbound.protection.outlook.com [40.107.100.56]) by imf28.hostedemail.com (Postfix) with ESMTP id D4574C0003 for ; Tue, 24 Jan 2023 20:34:51 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=CI82frt0; spf=pass (imf28.hostedemail.com: domain of jgg@nvidia.com designates 40.107.100.56 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674592491; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=n9QOZ0CxPutgVlVGahnDR92oM8ovpkJvVl6oiKzGidk=; b=1CI43n5wPfcoq/M3p+Nfng50e+++Hp47WhH2kSLiGw2CRPcm+8GPdYavrUO7BxXu37MZul mRLSFjGlHY46JYX1d+bw3h5viA/lqHHNZVO6zZiHVzuKPi954duhJpYzA3JuqqzGCC8JfW lJ3w6+qi7z76tk5yp1nHnWU5STcS5+8= ARC-Authentication-Results: i=2; imf28.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=CI82frt0; spf=pass (imf28.hostedemail.com: domain of jgg@nvidia.com designates 40.107.100.56 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674592491; a=rsa-sha256; cv=pass; b=rF5bcnPylH3/MpqrN0cy4INX0c/PJE3xFXE40blGkIthzm2gO2R7Ep3CARKDcvqsSpmKuw wFw/2/QXGgx+/ZlAGbxQU8izVqrqjCj7N0zR0+pQ3xV1kEcXbPXnhL24ZQcMjtNKaIxOaK ww4hkoNsG1GNjqJjlQszcdVby+o5UkY= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SFnAFp8DpSHEvdZ3aWVy3r6MuMsM2DEccP0ftSnfamaoTfyRlIzCbv6JTMTWu/13hOD3vVh42YHlU3qWhuJr7ssWHDRXMLTKvYnxVSk2crFj9EgVyXPHTLcIdVs0kN2r8CrwQFoR8KU29QjZVwiXiflDyMKHLGSS++nzonIF+zekbr1jEcpObOrdlJNOnQei0GDKmq3Pnqb4RUwHuRgSALjkeFqnkb0drDIE3BHXlqY+mzyRb8WhmO5xCfr3/2rNLiE+z7qeAGQGBQpH98AgtQhn1JWqCgt8guxpwvHcZ7yG+ud7SPWxgxYRaC3XlqohsY1V2lIv/MQPXUbjYU5mRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=n9QOZ0CxPutgVlVGahnDR92oM8ovpkJvVl6oiKzGidk=; b=FUCs5djVdYodUUwHWafGV0HG5b2HjmFlreaTTj9u7AsNIEAGkd/Ey2Q8N3UauoT3uS5UbHQd7PUEcx2jMXjogOtbDqZFh7IFiKldWavXgfpGXEw03HY2snp1afZmHLxzrHD3B2W9SVoER1oxj5knSmR9/tkFOxrqayyPWEkyt8vyPYdhiH7SHa2qKTLtu8HOX5WbCYCalgLDzpxKf8XfPgNCMYAf8NHl/Fg9Dl/emvivTIKcUtVEVQb9WdIdhxun4VMxiEJ+j5ZeNqJiFUwQCaFZ2l1uZBni/m1nMUXlfV9YEZrpq/4BXhPn19n/8V90rGhVU9jifWwSk2S4d68gRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=n9QOZ0CxPutgVlVGahnDR92oM8ovpkJvVl6oiKzGidk=; b=CI82frt0aIIrREjfIOMdE1WBFlQdvvCbQTWy/OOt3nEv+q28nb0WZQ9R1NQ2qaRkLEanM2EOqVvyrMxM4/YZO2IpGiEDmRR4U7sh5HPm9d5FYq5JtezAKV6Pyh27m3hey28vtgYRe3LxUaVSoaO/kmA0xymZuvHEPxs4To7eTOa1sMws80BbzpWphT4mNHogVqH0tBTG+2GnF/W1YdxOnOURf7zn6rKwG3WwW+xvo+IBdtjDEXSI34gtHXUykOz2281xrH1mhwBXKJ7QsewDDLzF+hBKAjJxP77rp4vPi0pbYkOhqOI/u+PmQ8nnQvh2Z7ImH7A1LIUTZZ24ZbiNHQ== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by MN2PR12MB4095.namprd12.prod.outlook.com (2603:10b6:208:1d1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan 2023 20:34:38 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::3cb3:2fce:5c8f:82ee%4]) with mapi id 15.20.6002.033; Tue, 24 Jan 2023 20:34:38 +0000 From: Jason Gunthorpe To: Cc: Alistair Popple , David Hildenbrand , David Howells , Christoph Hellwig , John Hubbard , linux-mm@kvack.org, "Mike Rapoport (IBM)" Subject: [PATCH v2 13/13] mm/gup: move private gup FOLL_ flags to internal.h Date: Tue, 24 Jan 2023 16:34:34 -0400 Message-Id: <13-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> In-Reply-To: <0-v2-987e91b59705+36b-gup_tidy_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR20CA0037.namprd20.prod.outlook.com (2603:10b6:208:235::6) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|MN2PR12MB4095:EE_ X-MS-Office365-Filtering-Correlation-Id: 5d1125d9-04cb-45c5-ad4d-08dafe4a6855 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: D0MsqKPXI7k+eVZzgQfRev9+20Tdlp1skF3M5nawoWVOdt1zAbrfmNahq3eHcsqLc7sMwGzC9/gkRA3MQttq71DQZ5mFcDvV4zD2hb3qKmOVZNW+CQzOF7pHtXI3Wfdg76Ylq/AcINI2BO/OzeIe3x5G3McwqJfHSO9o4hP73L042CXE0Twu7ygIwKOvMxN92mkr3yq06lbA2PpUnEkitMVJ6iUNr+Y0qJQCBqeDmN1JKLchd4/aMMfIwJeILjPYjXRzrFElqHM+E4cTkfleq5ZVwZn+gjs71Xt0pFI3/xFIa1Vq5PQhCu1qdRqAM+fZGEZm0ZIGk5FpPS7/3pkFvBKSzgf5Ek6aeMtFFvo3VmPtB96j9iW65Z4ZK5EdT5PWykLOFGoUsas+/aqDo3//ittnGOIDA7XTH/oGR4JIOyMLu5nQGQxyxyHqLif3QDq+uPeK+0u9VZaX4vXlf6apwU8D1evjBzaba7lG7G6pNNhvDbb+i2pivbP8mN2PtaJl7K0oaKK1FlFKSbx279CegY0sJuQh2Uu7VXHSt1SN7XnYU/CSFSo44zSbqgGz2lu6TMMKQKOsYqxM2FBnmUWMg3v3lrM3Oimk8m65dRN+Y2w+/XneVObEKfSdMa+k9LWDrlriYT5GyKNs30qGXL64ltWNoO12a4mxRILi2DVcc2BP5+laUPOA5BZk9Ko2qYyv X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(396003)(39860400002)(376002)(346002)(136003)(451199018)(109986016)(38100700002)(83380400001)(5660300002)(41300700001)(86362001)(2906002)(26005)(4326008)(8936002)(316002)(6666004)(6512007)(6506007)(8676002)(186003)(66476007)(66556008)(54906003)(2616005)(478600001)(6486002)(36756003)(66946007)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: wuwcFc+0Jg7PnUBg5ZqJMm1IkXkZR04llmJX7scYPhE6gR8FR0NaGbuOJoK2aKkgBDW1DrPqDDLhSFvWmI342N+7byfSFy4ogiq4aG+WVAT9ddpl6EsEkwL8/r7wVNoJ1GJ8Z+V7zyTTGcBSxgEtsU37vdlWtDh2Hs5338g6sJsOhM4R3eus5CaRjhOjhTZhOLlgPr70MUNO4JVNw/6XG6wK2L/RHv6CjMeC7jzmphWYtxfbdYYQBxqzaXiFeiLwZB9k5DEyus1MXnAx1McIVkfHh/+nevIOE+uD9d1dd1iJJqfPxMAr/T90lkNRASm1RZ2CeG4AlU9JHJ6kh+90EQzWuGLkhWEwqUpGLgPg9UUY/5vyTegIgdTJD5syEheyOFlkCTms+lroYkMzCZnB8PWSI1NLho829c0aylrHEc7Qt0qMFdtN3La2bnuhPMibgZ0v2J+ny0NWlgCcQTYxz3KmcHn8jvS5C84lMYHeGS9PeicyTD39EZvMhFm88yI7ZAMYVtN/G62HEzZifWTAF+e0vpozIR2uQucc7rNryYHX/HscBOh9oTa6yQlGpIqit/RQvx5fq3PCLqvMLklWfFyQ3uPdC9qlxF1Sj31k/nrFaZACKSKFjVTGzT6NQvQJCXKLnrbtoT44GseeZqdywkq/KVRnw3jTscC9SvPuFXHv8ld//57iFGh5VPq1onA/tdtLQzYYHKF7lWySjh+pkXi6fuAhasr1fXdGdbPz8Fp1urH6llh59oCMTFoD25+c/E8SzZG35b9RhRHnS/xZrorATZKQ43qzWThngWsPs7o4cd4cXTZhkx6N/LffGOkIqSzdjRY9McRz9fRYW8yAb1DzgcCHu1gcL2rZjiToQDLmkuSV07lE1Ub8tXh7WFOUAqEUz5zqfInDy1tbxJd12tZfbat2xarf8qlrVGBiCDaj/6ZnwACn3So/6XBnXRvV8sXmx3zMwLsp9hnrxNiwOlv6K1UAq49iObQaZuD9I0Vof7YRzi32mQqnD9ew+dbLtAAjWWVC/K07RwaoLPso80m1DFez5cMLYJCEnLwpgSYTLBVbHrbef3JlDjUywzkFNoLaD80Dda+TJKHOEbIy49bOP/S7xLFlI70qJTrB6T6mJwH+unw+2b1QC4bF0VOf84QE9aj7PEs95fQQVSJs6JNPvVKkqPUhhULuBDfNayRO4FDNuEen1X7HYh9NBLIa7oGjz2scaGpUokR/Rht4KFTdEdrM8N/IKr8/5kg7tKPJL0JL8QaYWXtxtg1sedQxBJVq+oBG3mpGRg9UaocXSr2/jxviraMkf/PQCsGrzaIfAqFQYCmAlCSxOBea98p3IRghJmjNHHM+KTop0Mzna4YxVYLDY8j9nh9cKlLRU0m9gd0F84CE9zra31EKfelcY8tSbk7LYIbUXA/mA0JStWDElrE8mAa38PlmA6CtZlDsM5wg8ZLAYLTnvT53YJ4A4DKXEQdtPDbTvbDrjww/bHc/o0OwOophceszQ6do2G++nX/Iroyd/3ZctND8tHu3unzY/DJY9keRYymIc7U19L0gi/wXSWK2ripfW26yrVU2ojF9kHNVwxIIukgCfOIY X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5d1125d9-04cb-45c5-ad4d-08dafe4a6855 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 20:34:35.9388 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KO1HqhyektUW69SIVn3WYG+mT4yMNPwoXQiEOrDBvAvy7mgPvlAqpjHX9JPFbf93 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4095 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: D4574C0003 X-Stat-Signature: 96fo4yey9pxwmjxhcj6zcq6p31g3kju8 X-HE-Tag: 1674592491-24752 X-HE-Meta: U2FsdGVkX1+Y+mo7K0WOs3T6OgEGmRC/AgbAYeDcd9FWmoA0GG5rjznoBuNuq9v+ygmKvWz6/tHdW4h/NTB4UVmTmW1gT28/ydjrZi0uPTRj++SxTkfsXLPABpgrNsXJ4sMHEJ673d5cH2DHElzVbl59StUT5jW8SCpp4/+Jvucpql/qQ0t3c3iqnbpXWXgEPBxlWWLjPWwX0dQETehLnWQPc0TbTEJWc6hkmv0pVMCgks/T0v32NVie69zRs+Jix+SWgrF/N+Qxqgy+1u5QDQR6swg/CCMRFqNOg+fpNlDoEHZYzDaCZeqSEhEaZPvnuSwohLcCImuu+jIX/m5w81m/KVgiJkZ4TiqvAClawg+I9Yt2zpLnhhfzB0sHpZtho0tbR/viGsjuEkkD2OoX1tm9cbRKVDMu7dua3FzgZW6lz6oxVyseqY6Af5efNFemrCWXmwT3uXAbx9WasK9LKHyLqTEjqd9d8BNMuhUTFhVLJUBMoG6PmfhkODeM/gxciTH0YymD6GKLyKpB9Khkh6Klysp6bxJv6m19Z5oqMQIiiKxZd3DFe1+sMoV7FuwBw4H/pQlUicpzWhgrFDypef4ExMv13/mNn5ucSZmbhYw2LuBmGEusLx4H0hSvcF7xlpUZgDMbpyExWYB0eTRG3q+COjJfFOjoQ7YFriVbgeWj8KSPrs03SCsrgyRd6ssSFY6ZqItsdRQ6MlHGw7D1xFJkpTnlAAGse6wH3+xOp49/pzKmiVcQ9ib5M2h3VBPEitc6Nb511i/3ZseUq93UHnf7hSkTrPdnSZmQol7NZuNH1vLSowsAznPKabP0Co4/xtYuyY3ApHm0oFfRmrVedZXCWjHJd+nbY5vRSH0GOwh4u2Do6CSgWLR1G19y3h5fNyRKLlw6hK6P6zHQD46h+A6nN8ljtYarorP0TqYQq7SGhInhT83iADFfoKtdV7yaNwK7sJyJxBUMAZG0glA YFpbMSBu wM0G06QP/Oim9sNYn2CjBUJWHuoVa4d7sEfB3q2x7ryD8DEH6TVs49shmqylB67zp9oJ+1eyjwXcC4pfojtiNfBFuYo6haIpfW1ZoswOSiK3aVkD6U2Sb6JXN2OZToy8IFWF5qsoz4TysYBkVxAMm7Dd9w+bFdgffbSQFVu4+T/kPTc+pe43zGzN5F7KAnAZ2S/TnJ6C+vaNzP2OyBbIuh8hKmYPZmOPbbvJ6Iy4p45QsS2mSBbc3ZU/OLTDdfZaUbKbCqimI/EkrLEBTno00usViw3rfK+LQQV9HS+w5tYMsyFEtXGr8mY9+rZimp+cNvdeinzHzJI42yct4ce6ZMXWowY13ut/VJ2QdzZNA8eJalf2YiVHFIPVRCg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move the flags that should not/are not used outside gup.c and related into mm/internal.h to discourage driver abuse. To make this more maintainable going forward compact the two FOLL ranges with new bit numbers from 0 to 11 and 16 to 21, using shifts so it is explict. Switch to an enum so the whole thing is easier to read. Cc: David Howells Cc: Christoph Hellwig Signed-off-by: Jason Gunthorpe Reviewed-by: John Hubbard Acked-by: David Hildenbrand --- include/linux/mm_types.h | 57 ++++++++++++++++++++++++---------------- mm/internal.h | 15 +++++++++++ 2 files changed, 50 insertions(+), 22 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 518617431f8085..dbfc3a7e9559b4 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -1039,9 +1039,6 @@ typedef unsigned int __bitwise zap_flags_t; * FOLL_PIN and FOLL_LONGTERM may be used in various combinations with each * other. Here is what they mean, and how to use them: * - * FOLL_LONGTERM indicates that the page will be held for an indefinite time - * period _often_ under userspace control. This is in contrast to - * iov_iter_get_pages(), whose usages are transient. * * FIXME: For pages which are part of a filesystem, mappings are subject to the * lifetime enforced by the filesystem and we need guarantees that longterm @@ -1085,24 +1082,40 @@ typedef unsigned int __bitwise zap_flags_t; * Please see Documentation/core-api/pin_user_pages.rst for more information. */ -#define FOLL_WRITE 0x01 /* check pte is writable */ -#define FOLL_TOUCH 0x02 /* mark page accessed */ -#define FOLL_GET 0x04 /* do get_page on page */ -#define FOLL_DUMP 0x08 /* give error on hole if it would be zero */ -#define FOLL_FORCE 0x10 /* get_user_pages read/write w/o permission */ -#define FOLL_NOWAIT 0x20 /* if a disk transfer is needed, start the IO - * and return without waiting upon it */ -#define FOLL_NOFAULT 0x80 /* do not fault in pages */ -#define FOLL_HWPOISON 0x100 /* check page is hwpoisoned */ -#define FOLL_TRIED 0x800 /* a retry, previous pass started an IO */ -#define FOLL_REMOTE 0x2000 /* we are working on non-current tsk/mm */ -#define FOLL_ANON 0x8000 /* don't do file mappings */ -#define FOLL_LONGTERM 0x10000 /* mapping lifetime is indefinite: see below */ -#define FOLL_SPLIT_PMD 0x20000 /* split huge pmd before returning */ -#define FOLL_PIN 0x40000 /* pages must be released via unpin_user_page */ -#define FOLL_FAST_ONLY 0x80000 /* gup_fast: prevent fall-back to slow gup */ -#define FOLL_PCI_P2PDMA 0x100000 /* allow returning PCI P2PDMA pages */ -#define FOLL_INTERRUPTIBLE 0x200000 /* allow interrupts from generic signals */ -#define FOLL_UNLOCKABLE 0x400000 /* allow unlocking the mmap lock (internal only) */ +enum { + /* check pte is writable */ + FOLL_WRITE = 1 << 0, + /* do get_page on page */ + FOLL_GET = 1 << 1, + /* give error on hole if it would be zero */ + FOLL_DUMP = 1 << 2, + /* get_user_pages read/write w/o permission */ + FOLL_FORCE = 1 << 3, + /* + * if a disk transfer is needed, start the IO and return without waiting + * upon it + */ + FOLL_NOWAIT = 1 << 4, + /* do not fault in pages */ + FOLL_NOFAULT = 1 << 5, + /* check page is hwpoisoned */ + FOLL_HWPOISON = 1 << 6, + /* don't do file mappings */ + FOLL_ANON = 1 << 7, + /* + * FOLL_LONGTERM indicates that the page will be held for an indefinite + * time period _often_ under userspace control. This is in contrast to + * iov_iter_get_pages(), whose usages are transient. + */ + FOLL_LONGTERM = 1 << 8, + /* split huge pmd before returning */ + FOLL_SPLIT_PMD = 1 << 9, + /* allow returning PCI P2PDMA pages */ + FOLL_PCI_P2PDMA = 1 << 10, + /* allow interrupts from generic signals */ + FOLL_INTERRUPTIBLE = 1 << 11, + + /* See also internal only FOLL flags in mm/internal.h */ +}; #endif /* _LINUX_MM_TYPES_H */ diff --git a/mm/internal.h b/mm/internal.h index 5c1310b98db64d..673b4b1d5f4c79 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -854,6 +854,21 @@ int migrate_device_coherent_page(struct page *page); struct folio *try_grab_folio(struct page *page, int refs, unsigned int flags); int __must_check try_grab_page(struct page *page, unsigned int flags); +enum { + /* mark page accessed */ + FOLL_TOUCH = 1 << 16, + /* a retry, previous pass started an IO */ + FOLL_TRIED = 1 << 17, + /* we are working on non-current tsk/mm */ + FOLL_REMOTE = 1 << 18, + /* pages must be released via unpin_user_page */ + FOLL_PIN = 1 << 19, + /* gup_fast: prevent fall-back to slow gup */ + FOLL_FAST_ONLY = 1 << 20, + /* allow unlocking the mmap lock */ + FOLL_UNLOCKABLE = 1 << 21, +}; + /* * Indicates for which pages that are write-protected in the page table, * whether GUP has to trigger unsharing via FAULT_FLAG_UNSHARE such that the