From patchwork Thu Feb 3 09:32:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12733992 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C7E4C433FE for ; Thu, 3 Feb 2022 09:33:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 910408D013E; Thu, 3 Feb 2022 04:33:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8C5588D0124; Thu, 3 Feb 2022 04:33:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 721748D013E; Thu, 3 Feb 2022 04:33:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0081.hostedemail.com [216.40.44.81]) by kanga.kvack.org (Postfix) with ESMTP id 46B708D013C for ; Thu, 3 Feb 2022 04:33:06 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 070B299911 for ; Thu, 3 Feb 2022 09:33:06 +0000 (UTC) X-FDA: 79100954772.12.FD403ED Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2052.outbound.protection.outlook.com [40.107.243.52]) by imf29.hostedemail.com (Postfix) with ESMTP id 638CA120002 for ; Thu, 3 Feb 2022 09:33:05 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FPjUADvL5qUdWvu1Pv6h5JKe16lxPFXv0xaGAS9aopaKc7CfzaAektbgFYP0fxGoHX/KWH1kJsfrbTDbX8AQu/KnBGBMBC9OovsIgNDm69G9KBPVHFsMkxvBs7N+coxF2yKT5WVUuP06aRohTMV8SokO65Z7zzvqkXukdhPSPhVxjgY2GJyHRAMTDQ6bAaIi3mJ2Tv9HA7AsG9dnZhna1bv76qyZ3Wj4xN72tZcZBzBmLcUrYkcEeC/02agekeHPUzQ59dl0OWiUH+0MDwPGk9FyJNxmO0Twsoxi56J9T4PQtzuLYHFQuGsADLyllUZTDerzxr1xMO2lHjF1kWce/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kyMU6TmMl5wkPP3kDvsxlk2/mNsd6Nh//tpGH/3US8s=; b=VxIszikueq3ULkbIYCFg6gwbCrnUmbBfALpzMm3x0GaTPEN8dsD4a3gZFH/yt8e9TyKuRgtKu+BSvrvsF3uDWy0+9r7e7fPdB7WELQuKn4WQU1jWOaYgOKczPjoC3DxAsc2cVZONP5cAMImjo8TN4HTTcUBH6MW6SR18LgmBslAA0Z6LFtSSOCtyrr7yJHaJTcm39AuPbhlI5ev4hingzEcuc3ebYKLi/B1wbd86aH82LlTWRfJcMRA12vdNrlBCsFGxAaT0Bd6C0/ErwsNUURMAWCwscOKgMpvkWf+kUt27lWPeWXW/yOXK5xgdR1NjK9YRsN+X02UGnGOIYR/5FA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kyMU6TmMl5wkPP3kDvsxlk2/mNsd6Nh//tpGH/3US8s=; b=hEAyOSxSRzunJyR7iHLjJsctksAZPqRW2sCdQobeERYuJ41OkbcGX/1znH24fC5t3oZ8gXJBAcdMV/DJ8nLlpZXOhQhnZhx5RnBeEhgJMubNg01ot9/bGizsHIrcAM85Y3lGGlmGuLgwW90VOTj8hSvJFmyBHl7RN31w/F61stAqTFhJWXFlwxw4Culn3m/krZV06vWnK+ZhMvUPOZFNY1HzJy7j2PfGOm+uqIup43c2tiV1KjCRELUA+RBVxoa+Z9fvJxEIPlOpAmufh7sDVeQRzs8q+ByOiSlqYcyx5lv7ZP5ces+apXRW60Twuh4ko7VytkefEcemm3rpo40sHA== Received: from DM6PR17CA0003.namprd17.prod.outlook.com (2603:10b6:5:1b3::16) by MN2PR12MB3376.namprd12.prod.outlook.com (2603:10b6:208:c2::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.12; Thu, 3 Feb 2022 09:33:01 +0000 Received: from DM6NAM11FT013.eop-nam11.prod.protection.outlook.com (2603:10b6:5:1b3:cafe::3d) by DM6PR17CA0003.outlook.office365.com (2603:10b6:5:1b3::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.12 via Frontend Transport; Thu, 3 Feb 2022 09:33:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.238) by DM6NAM11FT013.mail.protection.outlook.com (10.13.173.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4951.12 via Frontend Transport; Thu, 3 Feb 2022 09:33:00 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Thu, 3 Feb 2022 09:33:00 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Thu, 3 Feb 2022 01:32:59 -0800 Received: from sandstorm.attlocal.net (10.127.8.12) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Thu, 3 Feb 2022 01:32:59 -0800 From: John Hubbard To: Andrew Morton , Peter Xu , Jason Gunthorpe CC: David Hildenbrand , Lukas Bulwahn , Jan Kara , Claudio Imbrenda , "Kirill A . Shutemov" , Alex Williamson , Andrea Arcangeli , LKML , , John Hubbard Subject: [PATCH v3 1/4] mm: Fix invalid page pointer returned with FOLL_PIN gups Date: Thu, 3 Feb 2022 01:32:29 -0800 Message-ID: <20220203093232.572380-2-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220203093232.572380-1-jhubbard@nvidia.com> References: <20220203093232.572380-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 10ab1d80-9787-4556-fd71-08d9e6f82bc9 X-MS-TrafficTypeDiagnostic: MN2PR12MB3376:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:264; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: DzCauIr0ABToq4YvJl6Q5PisKRQMjeOQm6zU0NXx3xYS1k6LCMvoMe+bZUIQuioaFtTeF/5fOWxklom7iF1w8qkaZ4WpvpcD/Qk3Kr7heohClVE2C2Ka17O8+Zpefp+H9XAqmz5Sq1pv70+BHW7IMhRzOAq+zknYk9TjMZ/Ysaq2WNW2uc/f0bEZ9IB2soTYejKyvHGOXFr9NSIpuXN1DGCTxZuXjB9gtmqSPmt8DojPREVIqnAvaYfXNJKFuBZGtf9BO9fnpeYpA/tftlTDXTgkzlzuwYoTcA+09I1+U34z+p3LxkzGgWqxE8RFGdXs1jawndpeXq1m56DaczeeqR153g9jgxxru6UVnpSiXh2SYGSONj68RLxmJuw+iu3OX0K+0u2EHEclPWaqL4QhlcT249YxiQLA8uJ7uxbT9RhuUXUHfUUYYxiI/D5CQwnuaO5QZAaf3HNz210C4uoD5KsAz6zxIGXqpn7k//JpBnJeKxkG4MP78u0XiA3e7d8PXKrO6QxmXFnWN90KuTCofXyvSuhSKC6unfFYOoyFMvqSGMa5+tECsLk2ZEhNpmbQQNNqpCCmDQ33rGXUtCdC+DRE1Ore9fGjUdy3LwUTp12svOG/aJi9fX7Ggd+B1LsII3eA5N6BE9aiqH3tGEWejUv7XBvBZezn09+L7/m6FvIwyGS9A1UApW6VUf8OlnWBm/zp4NCjzufKBrKRdU4wiQ== X-Forefront-Antispam-Report: CIP:12.22.5.238;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(54906003)(508600001)(426003)(336012)(36860700001)(86362001)(110136005)(316002)(47076005)(40460700003)(36756003)(5660300002)(8936002)(8676002)(6666004)(26005)(70206006)(70586007)(83380400001)(1076003)(186003)(82310400004)(2616005)(2906002)(7416002)(4326008)(81166007)(356005)(107886003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Feb 2022 09:33:00.9321 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 10ab1d80-9787-4556-fd71-08d9e6f82bc9 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.238];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT013.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3376 X-Stat-Signature: g1htrxc8da9y8f7ioc1kjdz63ops8wit X-Rspam-User: nil Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=hEAyOSxS; spf=none (imf29.hostedemail.com: domain of jhubbard@nvidia.com has no SPF policy when checking 40.107.243.52) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 638CA120002 X-HE-Tag: 1643880785-73512 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Peter Xu Alex reported invalid page pointer returned with pin_user_pages_remote() from vfio after upstream commit 4b6c33b32296 ("vfio/type1: Prepare for batched pinning with struct vfio_batch"). This problem breaks NVIDIA vfio mdev. It turns out that it's not the fault of the vfio commit; however after vfio switches to a full page buffer to store the page pointers it starts to expose the problem easier. The problem is for VM_PFNMAP vmas we should normally fail with an -EFAULT then vfio will carry on to handle the MMIO regions. However when the bug triggered, follow_page_mask() returned -EEXIST for such a page, which will jump over the current page, leaving that entry in **pages untouched. However the caller is not aware of it, hence the caller will reference the page as usual even if the pointer data can be anything. We had that -EEXIST logic since commit 1027e4436b6a ("mm: make GUP handle pfn mapping unless FOLL_GET is requested") which seems very reasonable. It could be that when we reworked GUP with FOLL_PIN we could have overlooked that special path in commit 3faa52c03f44 ("mm/gup: track FOLL_PIN pages"), even if that commit rightfully touched up follow_devmap_pud() on checking FOLL_PIN when it needs to return an -EEXIST. Attaching the Fixes to the FOLL_PIN rework commit, as it happened later than 1027e4436b6a. Cc: Andrea Arcangeli Cc: Andrew Morton Cc: Jan Kara Cc: Kirill A. Shutemov Fixes: 3faa52c03f44 ("mm/gup: track FOLL_PIN pages") Reviewed-by: John Hubbard Reported-by: Alex Williamson Debugged-by: Alex Williamson Tested-by: Alex Williamson Signed-off-by: Peter Xu Signed-off-by: John Hubbard Reviewed-by: Claudio Imbrenda --- mm/gup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/gup.c b/mm/gup.c index f0af462ac1e2..65575ae3602f 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -440,7 +440,7 @@ static int follow_pfn_pte(struct vm_area_struct *vma, unsigned long address, pte_t *pte, unsigned int flags) { /* No page to get reference */ - if (flags & FOLL_GET) + if (flags & (FOLL_GET | FOLL_PIN)) return -EFAULT; if (flags & FOLL_TOUCH) { From patchwork Thu Feb 3 09:32:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12733990 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27E2EC433FE for ; Thu, 3 Feb 2022 09:33:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2F64D8D013B; Thu, 3 Feb 2022 04:33:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C85C8D0124; Thu, 3 Feb 2022 04:33:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 169B18D013B; Thu, 3 Feb 2022 04:33:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0252.hostedemail.com [216.40.44.252]) by kanga.kvack.org (Postfix) with ESMTP id 06BBE8D0124 for ; Thu, 3 Feb 2022 04:33:05 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id BC67498C1A for ; Thu, 3 Feb 2022 09:33:04 +0000 (UTC) X-FDA: 79100954688.28.1CD0221 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2060.outbound.protection.outlook.com [40.107.223.60]) by imf31.hostedemail.com (Postfix) with ESMTP id 5803D20008 for ; Thu, 3 Feb 2022 09:33:04 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NKYuqJ2Ftc3oz4i1nPdw2UMPwKi/qVnT14ff5dQb8IvyM9FGLjEGY750m4euFGEfVQA1TzCp/xR8P2jskbnEo7mfP8Nx61oOfvcE2qzg4zylVux/5rew/+ChwBmEYgrUS8PH71FmPM2DL2Wm2xPgntnd5TBdSRLIEKN0Dq3Lte9gJmr4wV2W4s7VoSjAr16dCzizhKhJ5jHhA/rQmMwm0u3eARfrhdajLgfd95E0bTd/JHZ8d9F6SLvRvQR4jvVnomsz4DXwereo3Myz2nW4p0uWB/vXeRzH6s+7Uo1M2IvOegQ/3wpF9duL0VjkHq3AB/OS7iOmz3pMabWOS1g4Yg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=88HPXi7oVGrTV2UcC+4qj7aDts0BOBT5CVbZa9sKizY=; b=d8dbi7IPUI1eAifCgpgMo6ty3gJAVnI5sOCP0PWOmI5jmIHZAxG5zIJvH0BM8ikOWaocdJFOILeV72nNA2GGOE5e/QGdL/wX8RJOidf4kdIL8usHfozyZ/SY5vYGH74c/7fJkBTbSvTVTm/+eb19c600uoQ4tXE7/vUhRGOoxwMuZRG3Gzp313olLHFNsYrix0reVBpsVRNhwGNYnX2E31uYPLN4X8DQryWO8RWkoBdRMHSzlygnUbGnstpqmAjASB6/unbcgcTXB9oa+l58dxnNzm+TJzHEgCWX+ZIBujwXzV2twyPGHo9oWRcB+Pb+0cTI3EN/Y6YXAey9Vy3AqA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=88HPXi7oVGrTV2UcC+4qj7aDts0BOBT5CVbZa9sKizY=; b=SMZzu0Zll+Pcs06ljHzY1i2WI1tGVQWID59CCpEfFgQElz+bCwfQzAQvUEQFTI2MxdcT5eUSKLg4baorzIO3W8AdoAqo6rdB7uLuYnoj65mf/dcqvTGY5bk6en03xNUugw1Gl4iqKWF23FrXCD9ggTwpmw4M9w92nv79RNscKxvvEb7LYdZEOx3LRBlr2O7siK1saYYKysmwpNamKu97nbs9zZR2BtxSHzggBvQCAkLwiKDxAOdtrMRVtud2DdnMVAG/yxpckpa6B0t4j4YanB4vL8BvKLML7kA0l0iDB0mdsrKqNBamLoKNpcytYPYrbu3mxBLDj03D89aO3UqvSQ== Received: from DM6PR17CA0005.namprd17.prod.outlook.com (2603:10b6:5:1b3::18) by BL0PR12MB2385.namprd12.prod.outlook.com (2603:10b6:207:4d::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15; Thu, 3 Feb 2022 09:33:02 +0000 Received: from DM6NAM11FT013.eop-nam11.prod.protection.outlook.com (2603:10b6:5:1b3:cafe::6c) by DM6PR17CA0005.outlook.office365.com (2603:10b6:5:1b3::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.12 via Frontend Transport; Thu, 3 Feb 2022 09:33:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.238) by DM6NAM11FT013.mail.protection.outlook.com (10.13.173.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4951.12 via Frontend Transport; Thu, 3 Feb 2022 09:33:02 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Thu, 3 Feb 2022 09:33:01 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Thu, 3 Feb 2022 01:33:00 -0800 Received: from sandstorm.attlocal.net (10.127.8.12) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Thu, 3 Feb 2022 01:33:00 -0800 From: John Hubbard To: Andrew Morton , Peter Xu , Jason Gunthorpe CC: David Hildenbrand , Lukas Bulwahn , Jan Kara , Claudio Imbrenda , "Kirill A . Shutemov" , Alex Williamson , Andrea Arcangeli , LKML , , John Hubbard , Jason Gunthorpe Subject: [PATCH v3 2/4] mm/gup: clean up follow_pfn_pte() slightly Date: Thu, 3 Feb 2022 01:32:30 -0800 Message-ID: <20220203093232.572380-3-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220203093232.572380-1-jhubbard@nvidia.com> References: <20220203093232.572380-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a79b81bf-2afd-47bb-1aac-08d9e6f82c6c X-MS-TrafficTypeDiagnostic: BL0PR12MB2385:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lP3TGQ04xtPswJSl2EQnj9LnZaEBQr52PRAj3OQXnQaWrq8h6YK3hNhqAMPvPre+MXk8mWrpwyu26mWx9DbUKD06T5W8FNgJJL+ZW21AxV7MyHtSG00CexypCkEAIPoFk2FB7d29rrMGyBN8AU33lAbD5DhVxVsLkKsrxgpFeLfetpAcLG4cbDc4z+egFH1V4Y9klpIAkuEgVmh5al43pXhQTkDyJGZruECZNT7/ldKovg8j1NdOKakSylx+/px9TLCCF3g31AVAjnrQOiW6VP6vmNPD/V5Hh57JgOdzBvJyq4tp//V6RpeeUZZg8RV2fdoI/Hc0BJ3jNO2QxG6fk+ZB+ViUpAyg65tgFtYISslyrDoc9FIZttmZx0J7Q7PVLo1mfFgo64kCheqf/Dp8+tzu6Qaqx1Fq1Qhg2X8gFwYuJ2ISj8rxmw119Y+4OjB150DII/jV6wErKYO6GxQBXIA+ZnOYh1rp+xUFSmdz8L3OTclhoBSx2wE+fOiXVFJuw02TQQcIrHG3uUHyzVYZALP4QKT4npWqqnGKvn+VmwY48fyzpokEM7ojAiCEeDqFcAfYyd7InwUDNfc4AsmdaTYxeSmTn/15/vBsqYlFZuVKGnlZg2xKLL2V0NyO+sTruUlutArbaT2O0LHPnAqDg5S0Bnh6lgwR5mC9Xzz41PQdYPtF2t34uDmi/P9Q/noPQZMQAiMEAqVkjPgDY7oFwg== X-Forefront-Antispam-Report: CIP:12.22.5.238;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(8936002)(54906003)(4326008)(8676002)(36860700001)(40460700003)(508600001)(47076005)(36756003)(83380400001)(26005)(70206006)(86362001)(110136005)(70586007)(426003)(107886003)(1076003)(2616005)(336012)(186003)(356005)(82310400004)(81166007)(316002)(7416002)(6666004)(2906002)(5660300002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Feb 2022 09:33:02.0102 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a79b81bf-2afd-47bb-1aac-08d9e6f82c6c X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.238];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT013.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB2385 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 5803D20008 X-Rspam-User: nil Authentication-Results: imf31.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=SMZzu0Zl; dmarc=pass (policy=reject) header.from=nvidia.com; spf=none (imf31.hostedemail.com: domain of jhubbard@nvidia.com has no SPF policy when checking 40.107.223.60) smtp.mailfrom=jhubbard@nvidia.com X-Stat-Signature: 3465r4rfw3biz741wu3un4p1yc6kha8p X-HE-Tag: 1643880784-931529 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Regardless of any FOLL_* flags, get_user_pages() and its variants should handle PFN-only entries by stopping early, if the caller expected **pages to be filled in. This makes for a more reliable API, as compared to the previous approach of skipping over such entries (and thus leaving them silently unwritten). Cc: Peter Xu Cc: Lukas Bulwahn Suggested-by: Jason Gunthorpe Reviewed-by: Jason Gunthorpe Signed-off-by: John Hubbard --- mm/gup.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 65575ae3602f..cad3f28492e3 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -439,10 +439,6 @@ static struct page *no_page_table(struct vm_area_struct *vma, static int follow_pfn_pte(struct vm_area_struct *vma, unsigned long address, pte_t *pte, unsigned int flags) { - /* No page to get reference */ - if (flags & (FOLL_GET | FOLL_PIN)) - return -EFAULT; - if (flags & FOLL_TOUCH) { pte_t entry = *pte; @@ -1180,8 +1176,13 @@ static long __get_user_pages(struct mm_struct *mm, } else if (PTR_ERR(page) == -EEXIST) { /* * Proper page table entry exists, but no corresponding - * struct page. + * struct page. If the caller expects **pages to be + * filled in, bail out now, because that can't be done + * for this page. */ + if (pages) + goto out; + goto next_page; } else if (IS_ERR(page)) { ret = PTR_ERR(page); From patchwork Thu Feb 3 09:32:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12733993 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88105C433F5 for ; Thu, 3 Feb 2022 09:33:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E1F6A8D013C; Thu, 3 Feb 2022 04:33:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DA06F8D0124; Thu, 3 Feb 2022 04:33:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A46548D013F; Thu, 3 Feb 2022 04:33:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0204.hostedemail.com [216.40.44.204]) by kanga.kvack.org (Postfix) with ESMTP id 8D8028D013C for ; Thu, 3 Feb 2022 04:33:06 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 448C8180989F9 for ; Thu, 3 Feb 2022 09:33:06 +0000 (UTC) X-FDA: 79100954772.25.B684ADF Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2088.outbound.protection.outlook.com [40.107.102.88]) by imf16.hostedemail.com (Postfix) with ESMTP id AF00A180002 for ; Thu, 3 Feb 2022 09:33:05 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KgYHBqtM1TF8a0WuQIZN+XzvfmTLxAkjNu0bqsEtdJ0JQF3idbFmSo/AeAbJTO2ZQFaYA6oZj8yhEsv/KGb/V4ysnm/cBFcLOGWHIOpu90XzPxxghmJnjhjtdU6FDzScMh1LQlB5oo7nkpjT4SnwbR+KJ7rtVW+f0bCDdonU8tNtqGT3Xg+pGf42fccNs9dvccz3kgMlIsrQN5P06U98SRYWEopm/5WhoSQwCTu7kJ52SUpEJAgkaMPcG+/H+pOWGyjL3BAwD6SCgB8aC3fm1FUAnq5F/V/PSA2sipGtQ9B6y+ezAqN8EqGXbUoxhJ2ggYB+d26ISOKBhyxEs2gpYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8JERJax4eJmh0g9cB8bxy6cGi4SvTxqiobq6FqFcixg=; b=mIYI3pagBAoCc1JJ/FTqTWmPJER2/BNccPEZ7iYeycFfGf5wPUWi+mzPEmW89wx2uz4mwOhDeBuWOr4KMnLngpu1xqLwhDKkRgvOB1vVeIRq9MCfR+OtQfmFL/q7jSPtHnPBP4hfrLhEts7LqXPCWCpvrax6//zZdQyIPP5/+PMr8fLb+Qfwkr9EyyO10IZPi1EcMbS6ObQVVtT1tVXv7quq+o3fCB/LVYwr3bW8LO/rEYagosH/pS+sFGHQi3rAmIC7NrHOoF8a4dLrzFRuFlPvCRz5VS8ehhN6r3Jdptq3aAaUm8upzjMEeHd0ujEymcMCuiBMNjY64yrBOIzUnA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8JERJax4eJmh0g9cB8bxy6cGi4SvTxqiobq6FqFcixg=; b=nDRBTzXKfKf6omcKn7ez0zL3bp2Vp/ARj8cO0FVeywzEqs9WSA2eGCWPTpEIP8p+Ly/el5QsSFAir0BqpULRVIjfjzqhhpl9j/scQ4pmdvg2/tEoejvQcd78oMdcrYacoyS+qgJJ1TRvLVLF5+YKExDKC0cn+ExMgM08LEwtSNcbqvOE/AYv6yanypuR5RrNz0TT9gnLTkKzWalCR58HPiJz8pYMpny7X4ueYIhSTx+oFvMKru/4X7/k8y7AtiV6Fwgvd0Lx6fP6LmOEWZkVO+AfekGvnq/eNAXtgFGlADOF7T3lvcTe/qXTir8qtwxUymXUwe1Ew2rpCILnIwb5dg== Received: from MW4PR04CA0109.namprd04.prod.outlook.com (2603:10b6:303:83::24) by BN8PR12MB4786.namprd12.prod.outlook.com (2603:10b6:408:a9::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.12; Thu, 3 Feb 2022 09:33:02 +0000 Received: from CO1NAM11FT007.eop-nam11.prod.protection.outlook.com (2603:10b6:303:83:cafe::95) by MW4PR04CA0109.outlook.office365.com (2603:10b6:303:83::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.18 via Frontend Transport; Thu, 3 Feb 2022 09:33:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT007.mail.protection.outlook.com (10.13.174.131) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4951.12 via Frontend Transport; Thu, 3 Feb 2022 09:33:01 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Thu, 3 Feb 2022 09:33:01 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Thu, 3 Feb 2022 01:33:00 -0800 Received: from sandstorm.attlocal.net (10.127.8.12) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Thu, 3 Feb 2022 01:33:00 -0800 From: John Hubbard To: Andrew Morton , Peter Xu , Jason Gunthorpe CC: David Hildenbrand , Lukas Bulwahn , Jan Kara , Claudio Imbrenda , "Kirill A . Shutemov" , Alex Williamson , Andrea Arcangeli , LKML , , John Hubbard , Jason Gunthorpe , Christoph Hellwig Subject: [PATCH v3 3/4] mm/gup: remove unused pin_user_pages_locked() Date: Thu, 3 Feb 2022 01:32:31 -0800 Message-ID: <20220203093232.572380-4-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220203093232.572380-1-jhubbard@nvidia.com> References: <20220203093232.572380-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e807c420-61a8-417e-467f-08d9e6f82c38 X-MS-TrafficTypeDiagnostic: BN8PR12MB4786:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:972; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1Ua2V01+zbWWfQhMlb+gMrlugvgqEPSHMGOz23vTqEtW7qzuqNbTUkEJWBRVKu9tZh8lML0CFPNfAnX5RinwKcrCkmOijRPMHbzGxIIyGwjX8EG7EdS2ByscXtuPxF+4FIGPJEYJxKQcO+EhwYYCagIVHbhuxyMoXZlmDxtOhVZgV0+xdCpvDabT3NPmefiODi7iiF/GiHxAy9e46yJr3777m2HFXW4Lznf7VZSuo/e8fwAUFkKQEx8E8l7eKukK+3R/9Pk1db0YBM5bh+ihtFU8OlpjigIq9plzTjRQA+mxDN224TyYcwL3h+biDlzY6PXkeA82ZaUr086rjOZHFi4x5wrWvuo6yX4LbVott5X2YNhLZF4JbsrCjQS4QUyWw6iVqWkHWjId0E9r/DmL19AICfAde26ImneHXViovYNNKBh98wOn8n2MOyReS6954KuRsN8moPqiRiwFuNrWuWi3fVAdyi55DO4jMs8xIA6cdeatgs8+SwmqtHjnn0oJC+kPoTg5XncAHOx4pCbjNG2KDcFiGPI9QYlkkMpa/JTMQIDzeAl0PGvY1fX60D0otZ/UJfH3swtnJDdHpThYpdCOePAaPUSr2PUvOILjhnz1AFcg9ypfEsu6o7TbE4yCmdJnavO0frzhH3xXUwBEAfa2D5knAD10nvExagrhqPMTiQNJ96WLXfSWmQyyHFFBEbGGi/b4oVDN6q+xdQlKrw== X-Forefront-Antispam-Report: CIP:12.22.5.235;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(46966006)(36840700001)(40470700004)(81166007)(356005)(83380400001)(336012)(82310400004)(426003)(47076005)(36860700001)(86362001)(5660300002)(7416002)(40460700003)(70586007)(4326008)(8936002)(70206006)(8676002)(54906003)(110136005)(316002)(6666004)(36756003)(2906002)(186003)(26005)(2616005)(508600001)(1076003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Feb 2022 09:33:01.6705 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e807c420-61a8-417e-467f-08d9e6f82c38 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.235];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT007.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR12MB4786 X-Rspam-User: nil X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: AF00A180002 X-Stat-Signature: 6hwj56jn83mok6o9muczgy566c1ffc9d Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=nDRBTzXK; spf=none (imf16.hostedemail.com: domain of jhubbard@nvidia.com has no SPF policy when checking 40.107.102.88) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com X-HE-Tag: 1643880785-624537 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This routine was used for a short while, but then the calling code was refactored and the only caller was removed. Reviewed-by: David Hildenbrand Reviewed-by: Jason Gunthorpe Reviewed-by: Jan Kara Reviewed-by: Christoph Hellwig Signed-off-by: John Hubbard Reviewed-by: Claudio Imbrenda --- include/linux/mm.h | 2 -- mm/gup.c | 29 ----------------------------- 2 files changed, 31 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 213cc569b192..80c540c17d83 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1918,8 +1918,6 @@ long pin_user_pages(unsigned long start, unsigned long nr_pages, struct vm_area_struct **vmas); long get_user_pages_locked(unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, int *locked); -long pin_user_pages_locked(unsigned long start, unsigned long nr_pages, - unsigned int gup_flags, struct page **pages, int *locked); long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages, struct page **pages, unsigned int gup_flags); long pin_user_pages_unlocked(unsigned long start, unsigned long nr_pages, diff --git a/mm/gup.c b/mm/gup.c index cad3f28492e3..b0ecbfe03928 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -3119,32 +3119,3 @@ long pin_user_pages_unlocked(unsigned long start, unsigned long nr_pages, return get_user_pages_unlocked(start, nr_pages, pages, gup_flags); } EXPORT_SYMBOL(pin_user_pages_unlocked); - -/* - * pin_user_pages_locked() is the FOLL_PIN variant of get_user_pages_locked(). - * Behavior is the same, except that this one sets FOLL_PIN and rejects - * FOLL_GET. - */ -long pin_user_pages_locked(unsigned long start, unsigned long nr_pages, - unsigned int gup_flags, struct page **pages, - int *locked) -{ - /* - * FIXME: Current FOLL_LONGTERM behavior is incompatible with - * FAULT_FLAG_ALLOW_RETRY because of the FS DAX check requirement on - * vmas. As there are no users of this flag in this call we simply - * disallow this option for now. - */ - if (WARN_ON_ONCE(gup_flags & FOLL_LONGTERM)) - return -EINVAL; - - /* FOLL_GET and FOLL_PIN are mutually exclusive. */ - if (WARN_ON_ONCE(gup_flags & FOLL_GET)) - return -EINVAL; - - gup_flags |= FOLL_PIN; - return __get_user_pages_locked(current->mm, start, nr_pages, - pages, NULL, locked, - gup_flags | FOLL_TOUCH); -} -EXPORT_SYMBOL(pin_user_pages_locked); From patchwork Thu Feb 3 09:32:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12733991 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA6F1C433F5 for ; Thu, 3 Feb 2022 09:33:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4A9058D013D; Thu, 3 Feb 2022 04:33:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 316E58D0124; Thu, 3 Feb 2022 04:33:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1DEFA8D013C; Thu, 3 Feb 2022 04:33:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0172.hostedemail.com [216.40.44.172]) by kanga.kvack.org (Postfix) with ESMTP id 04BD08D0124 for ; Thu, 3 Feb 2022 04:33:06 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id BF02E98C1A for ; Thu, 3 Feb 2022 09:33:05 +0000 (UTC) X-FDA: 79100954730.28.4A1AA83 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2088.outbound.protection.outlook.com [40.107.92.88]) by imf11.hostedemail.com (Postfix) with ESMTP id 3F2A940006 for ; Thu, 3 Feb 2022 09:33:05 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SHnS3wCIVup09iv3oPMXf+wqVYKFwTTjjBlqkWwrIC7sseJFobSDxIK3SP6GoP5pVbOd/c6SDFqRo4haAKvHnCXpWImf3+VmSxutTmZi7N/qHj47BSZHZvptafpUOKqZNhxgaO9a+8uN+gjEqlaedhrwEshghVYZTaf8r71gBl770kMpYbPt5z8d5WZSf1gBGEYiHl7QvmC51OqUl11c/fxx3b3/y6QF819IAo86j1BJIZH43VBuoX40GJwa5LyjskaKJf+KcwOavsCHrmKWrCptNJzbx1AzZmKfKSIJ11iIZVVxVmbmkgl8oxAMIbEIQ6dRjvarLowKy+SHYc35gQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2ZOQlvAvDS4/SPLvLjBz/5Ap3tNIGRbwmHAPau7fsxo=; b=WirnxRda0noN1ISeRX6JA2FV/xPKiRJb9Hao1IQkaK9QtjbhGETGpGjlFGh8iGFqv6NRx0xm3eIItRNrO5WKpSu7nhss/tkp9Zm2P21slmQVh5DC+itbuj9eHeYxRpgNIDMZ57YsUBm3pSJBhR4oZCpDqrUtX4aNLowJ2qrCzRa76RAMhEgvEZuEgpotRvBEJXhAvUPoROPFFqP1/mEJRWD+/V5fQ8+DqRAKa9sT+DgS/QaM7mE+sEa2Iod2CTDm/mQl+ouAgY5x1zc11r0QKzsGATO17IFFAqeCfUAlDMVQmqeKWi7KmkDrkjpehddpm7BfMWqa4W5DxvYXmEXcnA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2ZOQlvAvDS4/SPLvLjBz/5Ap3tNIGRbwmHAPau7fsxo=; b=f4GOiChxOf+T4oDV64oYZ5nHk6j073bC6gj4hY8I/vWzXGQx/voaO3uI5OVl7dXpqi+1LOwyVEjDFSJg9G9cSQ/bhPJkOsApzXWrLxe6FPG7a8snBMOquZiKqx3aRwvvlDeJyHDMGVweQ5kPmFZ26+wBOQa2YXWp1Cd1QQ89BNeZ4hBcTi+rqN01fmlH8rYZPkj0j1K7rQYNzPE6INoBTmuRWkonQ97t7kJYqWl820J/6MGlM0diWcvGutkv7moVkMVeyjX2A/HPP+S2m6k6ptyw/xd2xlPHQe1/KHWw4BYhqDotZFsEHcEkzNU6sk4pjCJlLiS6ldBlkvGNlZtmeQ== Received: from MWHPR1701CA0019.namprd17.prod.outlook.com (2603:10b6:301:14::29) by BN6PR1201MB0004.namprd12.prod.outlook.com (2603:10b6:404:ac::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.12; Thu, 3 Feb 2022 09:33:03 +0000 Received: from CO1NAM11FT041.eop-nam11.prod.protection.outlook.com (2603:10b6:301:14:cafe::18) by MWHPR1701CA0019.outlook.office365.com (2603:10b6:301:14::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.13 via Frontend Transport; Thu, 3 Feb 2022 09:33:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.234) by CO1NAM11FT041.mail.protection.outlook.com (10.13.174.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4951.12 via Frontend Transport; Thu, 3 Feb 2022 09:33:02 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Thu, 3 Feb 2022 09:33:02 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Thu, 3 Feb 2022 01:33:01 -0800 Received: from sandstorm.attlocal.net (10.127.8.12) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Thu, 3 Feb 2022 01:33:01 -0800 From: John Hubbard To: Andrew Morton , Peter Xu , Jason Gunthorpe CC: David Hildenbrand , Lukas Bulwahn , Jan Kara , Claudio Imbrenda , "Kirill A . Shutemov" , Alex Williamson , Andrea Arcangeli , LKML , , John Hubbard , Jason Gunthorpe Subject: [PATCH v3 4/4] mm/gup: remove get_user_pages_locked() Date: Thu, 3 Feb 2022 01:32:32 -0800 Message-ID: <20220203093232.572380-5-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220203093232.572380-1-jhubbard@nvidia.com> References: <20220203093232.572380-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: acdca6a0-5072-47d9-e2f3-08d9e6f82ce3 X-MS-TrafficTypeDiagnostic: BN6PR1201MB0004:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1X6Krkv+Q8BDDCjh6cYyH+lMFUkgYIIGD1/a0gX8H+YcCIugs8G1BQ92EwfnCpPaRi8nb+JHQhAaFD/7g/uZuxwtdEm+dnFcEx3D37Zy3x4kU0JfdQrxsZMUdYVWCEaLe0Bns/5NjEwXH6jDLKHpTcMfeVoSbJY1xSVHrlCWJczU0iDUXOVYYuYJic1rz9Vq+NwahcsGM64wfHFPGAgWGtttc5o8o2vkF85TqGOpS7m8/R/ZmfPpU0B+NCAlRfmWQMQClzo4WRf0SQp02Tsn3300J0HxK6GyUkBQmyW9E5XW2eoJsLFkPcTT4St5ewR5L0qGhc7OVvtFf896jfd8sPy0rBl5QnWCS8S/+IB1a2X6hBBWNV2T28YSsIVwCBK3wxc4e2EdjcpxcKRhZ3TXeDn3ZCJ2srq+DcJSne/H1Ez0L50mSC4TnQBknDscTzUW6FB4DOKESj8v9gtmZ+g6031SgNpBDjmXRq69KF2gDgjE7J2GXC4Cwlk+IpMNxowbThrA4UXdQZ1bui0tgPtohjuvAvgIKb0eTMHtqRlyyuAgcUxjJderAQBrhVA8CBWPdmW7BdjI9xiJMAHjKbTMxQwS8TzxtldCHLgxAWit9SOUZJU57jQuU9Rx2CpGnpdAzcEyzP6Wn7H8bRuGLsKCF0nrwTepjM8XtmT/rt1JPtubYm1LNpRmEF2SWblMU6tc8jMCkYaM2kDQ30UlUzYd5Q== X-Forefront-Antispam-Report: CIP:12.22.5.234;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(46966006)(36840700001)(40470700004)(356005)(2906002)(110136005)(426003)(86362001)(40460700003)(186003)(336012)(26005)(36756003)(107886003)(47076005)(5660300002)(81166007)(7416002)(36860700001)(70586007)(508600001)(70206006)(8676002)(8936002)(4326008)(83380400001)(54906003)(82310400004)(316002)(1076003)(2616005)(6666004)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Feb 2022 09:33:02.7952 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: acdca6a0-5072-47d9-e2f3-08d9e6f82ce3 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.234];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT041.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1201MB0004 X-Stat-Signature: mki7cyxpcwb5nfrnz5c3kkyczr7rsq3d X-Rspam-User: nil Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=f4GOiChx; spf=none (imf11.hostedemail.com: domain of jhubbard@nvidia.com has no SPF policy when checking 40.107.92.88) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3F2A940006 X-HE-Tag: 1643880785-378451 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There is only one caller of get_user_pages_locked(). The purpose of get_user_pages_locked() is to allow for unlocking the mmap_lock when reading a page from the disk during a page fault (hidden behind VM_FAULT_RETRY). The idea is to reduce contention on the heavily-used mmap_lock. (Thanks to Jan Kara for clearly pointing that out, and in fact I've used some of his wording here.) However, it is unlikely for lookup_node() to take a page fault. With that in mind, change over to calling get_user_pages_fast(). This simplifies the code, runs a little faster in the expected case, and allows removing get_user_pages_locked() entirely. Reviewed-by: Jan Kara Reviewed-by: Jason Gunthorpe Signed-off-by: John Hubbard Reviewed-by: Claudio Imbrenda --- include/linux/mm.h | 2 -- mm/gup.c | 59 ---------------------------------------------- mm/mempolicy.c | 21 +++++++---------- 3 files changed, 9 insertions(+), 73 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 80c540c17d83..528ef1cb4f3a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1916,8 +1916,6 @@ long get_user_pages(unsigned long start, unsigned long nr_pages, long pin_user_pages(unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas); -long get_user_pages_locked(unsigned long start, unsigned long nr_pages, - unsigned int gup_flags, struct page **pages, int *locked); long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages, struct page **pages, unsigned int gup_flags); long pin_user_pages_unlocked(unsigned long start, unsigned long nr_pages, diff --git a/mm/gup.c b/mm/gup.c index b0ecbfe03928..7da49df59110 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2118,65 +2118,6 @@ long get_user_pages(unsigned long start, unsigned long nr_pages, } EXPORT_SYMBOL(get_user_pages); -/** - * get_user_pages_locked() - variant of get_user_pages() - * - * @start: starting user address - * @nr_pages: number of pages from start to pin - * @gup_flags: flags modifying lookup behaviour - * @pages: array that receives pointers to the pages pinned. - * Should be at least nr_pages long. Or NULL, if caller - * only intends to ensure the pages are faulted in. - * @locked: pointer to lock flag indicating whether lock is held and - * subsequently whether VM_FAULT_RETRY functionality can be - * utilised. Lock must initially be held. - * - * It is suitable to replace the form: - * - * mmap_read_lock(mm); - * do_something() - * get_user_pages(mm, ..., pages, NULL); - * mmap_read_unlock(mm); - * - * to: - * - * int locked = 1; - * mmap_read_lock(mm); - * do_something() - * get_user_pages_locked(mm, ..., pages, &locked); - * if (locked) - * mmap_read_unlock(mm); - * - * We can leverage the VM_FAULT_RETRY functionality in the page fault - * paths better by using either get_user_pages_locked() or - * get_user_pages_unlocked(). - * - */ -long get_user_pages_locked(unsigned long start, unsigned long nr_pages, - unsigned int gup_flags, struct page **pages, - int *locked) -{ - /* - * FIXME: Current FOLL_LONGTERM behavior is incompatible with - * FAULT_FLAG_ALLOW_RETRY because of the FS DAX check requirement on - * vmas. As there are no users of this flag in this call we simply - * disallow this option for now. - */ - if (WARN_ON_ONCE(gup_flags & FOLL_LONGTERM)) - return -EINVAL; - /* - * FOLL_PIN must only be set internally by the pin_user_pages*() APIs, - * never directly by the caller, so enforce that: - */ - if (WARN_ON_ONCE(gup_flags & FOLL_PIN)) - return -EINVAL; - - return __get_user_pages_locked(current->mm, start, nr_pages, - pages, NULL, locked, - gup_flags | FOLL_TOUCH); -} -EXPORT_SYMBOL(get_user_pages_locked); - /* * get_user_pages_unlocked() is suitable to replace the form: * diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 028e8dd82b44..3f8dc58da3e8 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -907,17 +907,14 @@ static void get_policy_nodemask(struct mempolicy *p, nodemask_t *nodes) static int lookup_node(struct mm_struct *mm, unsigned long addr) { struct page *p = NULL; - int err; + int ret; - int locked = 1; - err = get_user_pages_locked(addr & PAGE_MASK, 1, 0, &p, &locked); - if (err > 0) { - err = page_to_nid(p); + ret = get_user_pages_fast(addr & PAGE_MASK, 1, 0, &p); + if (ret > 0) { + ret = page_to_nid(p); put_page(p); } - if (locked) - mmap_read_unlock(mm); - return err; + return ret; } /* Retrieve NUMA policy */ @@ -968,14 +965,14 @@ static long do_get_mempolicy(int *policy, nodemask_t *nmask, if (flags & MPOL_F_NODE) { if (flags & MPOL_F_ADDR) { /* - * Take a refcount on the mpol, lookup_node() - * will drop the mmap_lock, so after calling - * lookup_node() only "pol" remains valid, "vma" - * is stale. + * Take a refcount on the mpol, because we are about to + * drop the mmap_lock, after which only "pol" remains + * valid, "vma" is stale. */ pol_refcount = pol; vma = NULL; mpol_get(pol); + mmap_read_unlock(mm); err = lookup_node(mm, addr); if (err < 0) goto out;