From patchwork Fri Feb 25 08:50:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12759916 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 743F0C43217 for ; Fri, 25 Feb 2022 08:50:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238646AbiBYIvO (ORCPT ); Fri, 25 Feb 2022 03:51:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238632AbiBYIvI (ORCPT ); Fri, 25 Feb 2022 03:51:08 -0500 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2044.outbound.protection.outlook.com [40.107.93.44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D8161768DA; Fri, 25 Feb 2022 00:50:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CrIlI6LmgXIJlvEhBKBeleuNbhB8S+2Sz/uE5ik1xiyvDseFkwUIy5/7mvxN+OBPn/E1eRjyNbCOUfZZhX7FDiNVxKlensBMGhEIl5lChPnfrkNyYRe9BVDuu6YDwPRbhgCIvaO7wZQosJQiWF1gMIUdGSzwxPQh5cHg93ZkaSzqWwe/XS6/OsyXvS0FpgFtmjnYlwwzrjEazbhePXzrroEDaNJI0AvvvsYGzk6bSrUA+OuIC0RPrFoI2BGdWdigjop8b76LRsK3NGgRZ4tXImTbmySPBw3cMiAhipUgscJp2hJzLpcjKzXpyn6MHEx9BkfxULnAxMrG8EHdnbVRHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Oa+H/fbjV0fLvDgOl8iwViOqTlta9iVUjSg5NEzHQXY=; b=gFhhWgLj5/DUcALqZDypwFt9pQDxot2CSnch6BQqv+XCd5V/6xVUJb9kwnhfqz1i3KV47l9zqTLMmNFzeRnj8F8iAGEg7yb40BwKyxPj0fB4UUoB3vAPxW534svUN0Yp4YJAQ623UD9PPUBXuHnK9B1YSIWQJnsRKFlFbhazhQb/umyxVkQDk1Wn/20vUE4j89+q4Os+4U1omyAJNKVPHkffCaiAKPCyPmwTfOti2TYoYM0UXBsM7F+Ke4yv7mXVL7nDnSIoHCWQGK3RaK1MxhsSp7Hs7abxfUaQyc1Of1XDlXc/cxSbrSVRGzYSGBQIZSJ6pkP7yKushmM0hl+00A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Oa+H/fbjV0fLvDgOl8iwViOqTlta9iVUjSg5NEzHQXY=; b=KDdhhE7hRTl5ZbZNSFRYP2njHiKEf/FTiJARgU/gnqkMXHs1jBqZGQtez1l2XWdtqXpizg2CGyrLUC1yNRikuISmGil2yPQPLhWt3OjrIDlyDit39H+Mv7QqZ+rCfbpVZmC0I6WfZAfju9feToMLROLfl29XVqNGNN1SaXIRNR6TmFmFh7Xo3cszEe+mg1Mq5NaDdDAIIS5Oeq3APGhsIpW+ppVLVIs7x/g0awLwlmqwIJYAunvW8egBbyp+dOwUQfY6cq8trHl91reIQC8GBlRkKjnyJYwmQpgbZW2bpVl6PuhgNyKv0NwSl03kgIElrRoTsAJcL7/3cZPVyVpy0Q== Received: from BN9PR03CA0790.namprd03.prod.outlook.com (2603:10b6:408:13f::15) by BL1PR12MB5094.namprd12.prod.outlook.com (2603:10b6:208:312::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.21; Fri, 25 Feb 2022 08:50:35 +0000 Received: from BN8NAM11FT030.eop-nam11.prod.protection.outlook.com (2603:10b6:408:13f:cafe::ab) by BN9PR03CA0790.outlook.office365.com (2603:10b6:408:13f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.21 via Frontend Transport; Fri, 25 Feb 2022 08:50:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by BN8NAM11FT030.mail.protection.outlook.com (10.13.177.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:34 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 25 Feb 2022 08:50:34 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 25 Feb 2022 00:50:33 -0800 Received: from sandstorm.attlocal.net (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 25 Feb 2022 00:50:31 -0800 From: John Hubbard To: Jens Axboe , Jan Kara , Christoph Hellwig , Dave Chinner , "Darrick J . Wong" , Theodore Ts'o , Alexander Viro , Miklos Szeredi , Andrew Morton , Chaitanya Kulkarni CC: , , , , LKML , John Hubbard Subject: [RFC PATCH 1/7] mm/gup: introduce pin_user_page() Date: Fri, 25 Feb 2022 00:50:19 -0800 Message-ID: <20220225085025.3052894-2-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220225085025.3052894-1-jhubbard@nvidia.com> References: <20220225085025.3052894-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0c8acfe7-76ca-4054-9073-08d9f83be34a X-MS-TrafficTypeDiagnostic: BL1PR12MB5094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Sl3jY+YV8dP/1Cx+ZRwSk+sHTf/28FG9t8fcujr/Gi5Rj8Ql2gwFA2W4AoOxzdcSQ8R/Y/uORHA7tKMQ9iiw6b3zppi1MCjfMtiDHypAbRd5dgJKGNvFwEGDKvkhFyesquuhmzrzmmGEtSbT8QkS9gGN+d8TcvwJWaW66TFgXHwCWF6E2uxEkbpPdxI596Ym16Y4KWEBbm+JQ0Xm1pbsmjkNSWJo/ZXfecQUc8vltUQaY4uOYilxYA3H0UOddatEBqvOqwWOHz3nr2kAd8EbtqWT+lNQRhwbmZrAG5LoJWRPR0vVPvtpPC0m5f70kzQNg3GVFclYIgsNZ5/qXvtD6gTjVCCvqiGNIlYUH39hewP0Dg+517dUjefgImEKKVA4hnS9At1voy9uvQCNqsJG1sY8pxYNFQK6kJ+nkWy1wXd/Pxc0I9ys77EyakvRnkO30UAtIdkS17FA7SxtY7Xi3eo+7agB9GSUgzi2F/Gb6qOqYLQiG0qbFfJq2nDOjcknpPt/fK+4Wm8RpILGsG3/G/4dmwO713NgrcHfXxu37FGdeAWYxIrg72ONywwx6rxBYZzQftFK5PT8AxINPEt5O9iJ+tjg7i8t7RBd4XcrQVBfLi7jA4co1zUHMiOVu8/w6AXJyU/NrP3NTXkxLsBE6KilA60Jz3y66FGlqdFo24XacqaRtb3pllrN2TjIKxxULkh/cg4xmZOSLNzOK5r0FVT4hSBUL4+R0IxVy979IJk= X-Forefront-Antispam-Report: CIP:12.22.5.235;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(46966006)(40470700004)(36840700001)(36756003)(8936002)(7416002)(356005)(921005)(81166007)(5660300002)(508600001)(86362001)(40460700003)(8676002)(70206006)(70586007)(47076005)(107886003)(1076003)(6636002)(2616005)(316002)(336012)(426003)(186003)(26005)(2906002)(6666004)(83380400001)(36860700001)(4326008)(82310400004)(54906003)(110136005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 08:50:34.7990 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0c8acfe7-76ca-4054-9073-08d9f83be34a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.235];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT030.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5094 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org pin_user_page() is an externally-usable version of try_grab_page(), but with semantics that match get_page(), so that it can act as a drop-in replacement for get_page(). Specifically, pin_user_page() has a void return type. pin_user_page() elevates a page's refcount is using FOLL_PIN rules. This means that the caller must release the page via unpin_user_page(). Signed-off-by: John Hubbard --- include/linux/mm.h | 1 + mm/gup.c | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 929488a47181..bb51f5487aef 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1914,6 +1914,7 @@ long pin_user_pages_remote(struct mm_struct *mm, long get_user_pages(unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas); +void pin_user_page(struct page *page); long pin_user_pages(unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas); diff --git a/mm/gup.c b/mm/gup.c index 5c3f6ede17eb..44446241c3a9 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -3034,6 +3034,40 @@ long pin_user_pages(unsigned long start, unsigned long nr_pages, } EXPORT_SYMBOL(pin_user_pages); +/** + * pin_user_page() - apply a FOLL_PIN reference to a page () + * + * @page: the page to be pinned. + * + * Similar to get_user_pages(), in that the page's refcount is elevated using + * FOLL_PIN rules. + * + * IMPORTANT: That means that the caller must release the page via + * unpin_user_page(). + * + */ +void pin_user_page(struct page *page) +{ + struct folio *folio = page_folio(page); + + WARN_ON_ONCE(folio_ref_count(folio) <= 0); + + /* + * Similar to try_grab_page(): be sure to *also* + * increment the normal page refcount field at least once, + * so that the page really is pinned. + */ + if (folio_test_large(folio)) { + folio_ref_add(folio, 1); + atomic_add(1, folio_pincount_ptr(folio)); + } else { + folio_ref_add(folio, GUP_PIN_COUNTING_BIAS); + } + + node_stat_mod_folio(folio, NR_FOLL_PIN_ACQUIRED, 1); +} +EXPORT_SYMBOL(pin_user_page); + /* * pin_user_pages_unlocked() is the FOLL_PIN variant of * get_user_pages_unlocked(). Behavior is the same, except that this one sets From patchwork Fri Feb 25 08:50:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12759917 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3C33C4167D for ; Fri, 25 Feb 2022 08:50:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238652AbiBYIvQ (ORCPT ); Fri, 25 Feb 2022 03:51:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45890 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238634AbiBYIvL (ORCPT ); Fri, 25 Feb 2022 03:51:11 -0500 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2043.outbound.protection.outlook.com [40.107.92.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A228177773; Fri, 25 Feb 2022 00:50:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XBlO4zIFxmhx3wt8IOztZcQMmeyv2m7n6X9tW0E4NR1miYig6L+KLPPbBZngfC9Pv4Z2+aTRvnq9KQUgQuU5fEUL8fYIWDc0CVOLwewTPB8Dkx+UMWJ/mmyJF2VF80WD7IH9znDb94cncFqYFrdPjppKoG8iVfW5lgIbnGK9A7Yn9g1ZpTo7SO0iBtas5ygeaeUXKh72PbnPJAMJ7b+WVkB/q5s6lqfYopSa4aomOl69rgGI5dZD0nFKxSbSOvjihXrDcrWvlaa7VSE7RgtRKV6Ad8PBheFYsg2+AdvfWs3D7ZzAeeFfokYFK90VGJb7KylRjjjWPEgpHfAHj2R9XA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=h1M9kqFuSLuRzAkMql4Y2/4feCJMUalRUwt7co+e448=; b=K4zp4D+IPCAhVT5oGaYBOyX7giQHw34lN8XNBSnMy8Eb714urAYLYVo9CRlmGMuWCCd3kLi6R0i4bMVwIVKkU/fE2q4IivbBRjmy4y+Gpv1cLxEmYS0f0Zagr7bzPfi8Cze5O6b4CtfxSmqAhck+j6pWCsShtcM/cqBfMLYGWIAQ28J/4Oaa0xtRphMHamdWjEKGfT1fEhmss5PvoS6vy0HE8BXHkneuRMG7nbac6z8gcw/ve08B07QQMXrHmQpYWfFEyiA+n9mxsaR02LnwLNjVegXKHV/i6g9+90Cgh0a3d+f5RKcJLfodNZ3GFNIvc9cmiUD5f5MuBqS7LN9DLw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h1M9kqFuSLuRzAkMql4Y2/4feCJMUalRUwt7co+e448=; b=GJ2qMS3jPCG1uGGHF19x46M3X0lH+EZNhz6bcbzgRloaM64kotkSCCinVqQ6bk7Uat4AASlYRw32YIFwa098IYmI8YwfhqjGpDlGVn/yQ7u0uEiQIQeTlGILoCdprmhgfhxkc5md12KCfyCZtZW80TP3m77y7PiAo5Ewk/0kQShRB2vaRrpcDktOfHd5Ex83H98oSwV9wmTWQfeAnkmwAwD6XTzZsIMk9YZcS7RvCsSjBIsJgbh0fcid94ld1tCM7Z6AUtgrSkyxcgYwZYBl8No66F+yvMzQisigDfGcMaF+p2YWDG46pO7s6L7cP2slBUJJ0T71xK9Qww4cWjVbbg== Received: from DM5PR18CA0073.namprd18.prod.outlook.com (2603:10b6:3:3::11) by BN6PR12MB1188.namprd12.prod.outlook.com (2603:10b6:404:19::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.22; Fri, 25 Feb 2022 08:50:37 +0000 Received: from DM6NAM11FT066.eop-nam11.prod.protection.outlook.com (2603:10b6:3:3:cafe::73) by DM5PR18CA0073.outlook.office365.com (2603:10b6:3:3::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.24 via Frontend Transport; Fri, 25 Feb 2022 08:50:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.234) by DM6NAM11FT066.mail.protection.outlook.com (10.13.173.179) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:36 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 25 Feb 2022 08:50:36 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 25 Feb 2022 00:50:35 -0800 Received: from sandstorm.attlocal.net (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 25 Feb 2022 00:50:33 -0800 From: John Hubbard To: Jens Axboe , Jan Kara , Christoph Hellwig , Dave Chinner , "Darrick J . Wong" , Theodore Ts'o , Alexander Viro , Miklos Szeredi , Andrew Morton , Chaitanya Kulkarni CC: , , , , LKML , John Hubbard Subject: [RFC PATCH 2/7] block: add dio_w_*() wrappers for pin, unpin user pages Date: Fri, 25 Feb 2022 00:50:20 -0800 Message-ID: <20220225085025.3052894-3-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220225085025.3052894-1-jhubbard@nvidia.com> References: <20220225085025.3052894-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 92112809-dd4f-48be-2f22-08d9f83be46b X-MS-TrafficTypeDiagnostic: BN6PR12MB1188:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rUy++dt94z8m32z8jyFAhYaKVBy1XC4rHK5kuji65Rk3Uvn96GsULMmTxXNowEHCpOS1fMzoG3tu+DoQje0a5F8acSVVXe+fUG7ajZfp2ExQHazAdbAPjp0cnmIjLQF55EqL2C12V61EjQASNzP2RpvNpYJ+Cj04sHsE3WZmTVWsHO5TFdNy6hTEhF1/AaSDubme6f+ca/0qQYfpmh1fP9Pga8lufmVju+7hl5wJLCV/66aYw98yQ0n9l06hbf25pK+KWCDWJU8lAp/arfv33dPNwbjQAQT8VukekZqFtDhKmvrdWMTQxvAKUxaNWGv38go3PC5ECWRaiI3R7NN02FyF0kHHEnYCHnjfAURC733IwcW16uqmLldyyOG57qwxmDGEBDG83o6zTabwFjm6G7IqLFpCx7TQuWOqK5I9pKcm5tWSP2i83sW6zXZ6Qz6w+lG9yA5Tt/rDDi56DPnjU8M92zafBg+SUSdDGnmqcUtqe8jYN5qIVRhi7GMKsKk1F0wbQyZHjEWZeAOrYuWVxmWGHyLSk4k30BSQvFTVlWlbL2RSWP3WbjQlIzQfXiLTfQIPTKti3ll3b2edQWMLM5l21DkqUthtsX9fO52zL/F6oQummgUmvIk/3NXmhAU2j4TXaZOfBELf8VQux/EFBQsXXMWYAPozftvRtaaiu8Xub4bfmayQscZCK10JbiA0AMfZpFruh4iKGgVfZ0ojhkqJFQzoQwpKCGTyiWmNbEw= X-Forefront-Antispam-Report: CIP:12.22.5.234;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(5660300002)(336012)(26005)(426003)(107886003)(36756003)(2616005)(7416002)(2906002)(6666004)(508600001)(186003)(8936002)(47076005)(1076003)(83380400001)(110136005)(81166007)(356005)(36860700001)(4326008)(70206006)(70586007)(8676002)(86362001)(921005)(6636002)(82310400004)(54906003)(40460700003)(316002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 08:50:36.7530 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 92112809-dd4f-48be-2f22-08d9f83be46b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.234];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT066.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1188 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Add a new config parameter, CONFIG_BLK_USE_PIN_USER_PAGES_FOR_DIO, and dio_w_*() wrapper functions. Together, these allow the developer to choose between these sets of routines, for Direct IO code paths: a) pin_user_pages_fast() pin_user_page() unpin_user_page() b) get_user_pages_fast() get_page() put_page() CONFIG_BLK_USE_PIN_USER_PAGES_FOR_DIO is a temporary setting, and will be deleted once the conversion is complete. In the meantime, developers can enable this in order to try out each filesystem. More information: The Direct IO part of the block infrastructure is being changed to use pin_user_page*() and unpin_user_page*() calls, in place of a mix of get_user_pages_fast(), get_page(), and put_page(). These have to be changed over all at the same time, for block, bio, and all filesystems. While that changeover is in progress (but disabled via this new CONFIG option), kernel developers need a way to test their changes. The steps are: a) Enable CONFIG_BLK_USE_PIN_USER_PAGES_FOR_DIO b) Monitor these /proc/vmstat items: nr_foll_pin_acquired nr_foll_pin_released ...to ensure that they remain equal, when "at rest". Signed-off-by: John Hubbard --- block/Kconfig | 25 +++++++++++++++++++++++++ include/linux/bvec.h | 11 +++++++++++ 2 files changed, 36 insertions(+) diff --git a/block/Kconfig b/block/Kconfig index 168b873eb666..f6ca5e9597e4 100644 --- a/block/Kconfig +++ b/block/Kconfig @@ -50,6 +50,31 @@ config BLK_DEV_BSG_COMMON config BLK_ICQ bool +config BLK_USE_PIN_USER_PAGES_FOR_DIO + bool "DEVELOPERS ONLY: Enable pin_user_pages() for Direct IO" if EXPERT + default n + help + For Direct IO code, retain the pages via calls to + pin_user_pages_fast(), instead of via get_user_pages_fast(). + Likewise, use pin_user_page() instead of get_page(). And then + release such pages via unpin_user_page(), instead of + put_page(). + + This is a temporary setting, which will be deleted once the + conversion is completed, reviewed, and tested. In the meantime, + developers can enable this in order to try out each filesystem. + For that, it's best to monitor these /proc/vmstat items: + + nr_foll_pin_acquired + nr_foll_pin_released + + ...to ensure that they remain equal, when "at rest". + + Say yes here ONLY if are actively developing or testing the + block layer or filesystems with pin_user_pages_fast(). + Otherwise, this is just a way to throw off the refcounting of + pages in the system. + config BLK_DEV_BSGLIB bool "Block layer SG support v4 helper lib" select BLK_DEV_BSG_COMMON diff --git a/include/linux/bvec.h b/include/linux/bvec.h index 35c25dff651a..a96a68c687f6 100644 --- a/include/linux/bvec.h +++ b/include/linux/bvec.h @@ -241,4 +241,15 @@ static inline void *bvec_virt(struct bio_vec *bvec) return page_address(bvec->bv_page) + bvec->bv_offset; } +#ifdef CONFIG_BLK_USE_PIN_USER_PAGES_FOR_DIO +#define dio_w_pin_user_pages_fast(s, n, p, f) pin_user_pages_fast(s, n, p, f) +#define dio_w_pin_user_page(p) pin_user_page(p) +#define dio_w_unpin_user_page(p) unpin_user_page(p) + +#else +#define dio_w_pin_user_pages_fast(s, n, p, f) get_user_pages_fast(s, n, p, f) +#define dio_w_pin_user_page(p) get_page(p) +#define dio_w_unpin_user_page(p) put_page(p) +#endif + #endif /* __LINUX_BVEC_H */ From patchwork Fri Feb 25 08:50:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12759918 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 641AFC43219 for ; Fri, 25 Feb 2022 08:50:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238664AbiBYIvT (ORCPT ); Fri, 25 Feb 2022 03:51:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45944 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238647AbiBYIvO (ORCPT ); Fri, 25 Feb 2022 03:51:14 -0500 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2072.outbound.protection.outlook.com [40.107.93.72]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DAF38177D3F; Fri, 25 Feb 2022 00:50:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WyACxF7fPkE2rMezm401crnu/DYgiYzX4NkfVZmgP+AqV4eae1UuvF2outqZRNrbfiQJzvBMofrQox3Sxt71gFCW5xToCaiN7RWQiGGPKL8My8pPYh/fU3iWZI7p4IP9pcxTPq0T/ggKWk0cGkuw7cxnKqlbIGXiafL/tkLALhYcXFALXLnoX5hohzYZj7qUdQ5x+i5s2T98fbvP+H9cuQfSZKdwX2svGsAH52T4lXVaC0k/g+WNaMUrHZAaENc+PhDVfvkRUN+p/H5WMnhWBM2+mMUpK4jRCdpozW/XZhcSA5wn5KsqiwVRBItiPt90I64JjlsE91rwGBbfm70qaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KCh3aDMkQYStRKBxVavUNPnaTNvm4hCnzAHey6l38Dw=; b=Lwkg2W3UPyscDCUYO3ipEP8AfT0b3i56zw2EEsrHTNrMm2azaItP3Rewz0hja+0AZ1+3FfeSsYEjsz09R17yKQj6lz9UdkDQbNGxm0WY61w6yuFwROMXxDrkd9VRzkY2D9rVNTXYqrsZES71crrTVAZp4dnObefkVK/SZy84X33mhYnho6M73B2WFtfl0TcHvyEFmw6GCaZIEiUPkQhtBQb73+lHb0vznZvjSm4UTg5SJcn1o6SiY5iuYMEON6XPT9krjPttEFgwYVPq8rtnuNOSLZcK8X2KTFNsTDFNUwqUZuXGLnnEoLXflmgzWZtxGnrXQmgTpz+SVbt6aeN2Rg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KCh3aDMkQYStRKBxVavUNPnaTNvm4hCnzAHey6l38Dw=; b=PY8TDTmGFToOfTDgVT1eFl9ditLQvPXUH6v0T+e6nKFDm0QYc2dEcPq2V7YPtr0n1ENen2cg++ij+2r11pxvhpoMr8k+tDL5GdQCfsym15aFAshHlrcKgv1+VpRQ03xjbD7h0qqkNJodqOaYdIEkocgAl9id9alvAhP/uySE0w/oh5UlRAvR8sY0YirWSe9Rim6GYw6FCt128/Lu+kyIllSp2mIDxpBk1GLFSpERnzdPSPxBXG9lIuR6TXGyAb9XjhWvHMlP6tNEQQVOzadS7aQPs6vlzwvCzk/DLwEaqgiG4W0Xh9+sGo6TpVu9Q5PxyYKXJOrOwLZVm1m2+V4XYw== Received: from MWHPR20CA0029.namprd20.prod.outlook.com (2603:10b6:300:ed::15) by SA0PR12MB4432.namprd12.prod.outlook.com (2603:10b6:806:98::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.21; Fri, 25 Feb 2022 08:50:40 +0000 Received: from CO1NAM11FT037.eop-nam11.prod.protection.outlook.com (2603:10b6:300:ed:cafe::26) by MWHPR20CA0029.outlook.office365.com (2603:10b6:300:ed::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.19 via Frontend Transport; Fri, 25 Feb 2022 08:50:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT037.mail.protection.outlook.com (10.13.174.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:39 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 25 Feb 2022 08:50:38 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 25 Feb 2022 00:50:37 -0800 Received: from sandstorm.attlocal.net (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 25 Feb 2022 00:50:35 -0800 From: John Hubbard To: Jens Axboe , Jan Kara , Christoph Hellwig , Dave Chinner , "Darrick J . Wong" , Theodore Ts'o , Alexander Viro , Miklos Szeredi , Andrew Morton , Chaitanya Kulkarni CC: , , , , LKML , John Hubbard Subject: [RFC PATCH 3/7] block, fs: assert that key paths use iovecs, and nothing else Date: Fri, 25 Feb 2022 00:50:21 -0800 Message-ID: <20220225085025.3052894-4-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220225085025.3052894-1-jhubbard@nvidia.com> References: <20220225085025.3052894-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 892e3132-73d9-41d6-34a5-08d9f83be61e X-MS-TrafficTypeDiagnostic: SA0PR12MB4432:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: El6t/1kSUiKOsLJMZfaTKBxsmM6FJwsDQiueoSdM8DfLi8Y3qpYgeopIjwbwE+LaikxDletUrU6wQlyJ7ss6d3kr80CdIYlFBiokgVjvCzgGHk3jGURchWTkTzT/0ekN6IRUHS+OgHG50KoY9JyYQCBQCNIZ+0y7fzN/sAz81cwZtFuO/GdmcCM12Vho8VgksEZKQLiDs+bqt6wopBqJ6vWIeju2UyJpHoikw5Wd1ICWNwKm13iQwShzGk4vRtfSrGgXOYXx/JAOUdUP8coyXnd5JF04x0N49gp1vyB42eaauFN5Qn+EeuYQJ4KhP91wx6dmcNaK+uV3mZR4cwKBvb48RAEMzulCvYcvklMp9J/a4lqC5ulaEwNzGd82NIaCz5g9Qi2aGR7eFPpnUujwt+RuQavzx7xlpniniVQQMBjAUkF72wLsI23Z5/guMZg09yxHo4DMWhquDTRoX/PniZCKevgrNlZeKaSjnqdNLeQ6Oux4MvJOWtVx4XjC6dktJEVqTtg/nfXyu0J97bg4L+WVcpKWDVLG46O41uhmAksIfN+dflvQgCvdfmlpBGDjQKkrJ5wlMPwPMd9UXmijetYC2+zBiapjctwBYmxu3nNWptQc/4pjlXePIOao1CYpwlbeLA7rPk1af3XbyJ9804KcHx9HdvxtiPpq4lHPAcgqmaKSf9QOii3/In+H2coHs4m2OnWR066AQNEsIyw+0ci2eDA4ilT4kAIi+wLLm2Q= X-Forefront-Antispam-Report: CIP:12.22.5.236;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(36756003)(6666004)(8936002)(107886003)(2616005)(356005)(40460700003)(7416002)(508600001)(336012)(426003)(26005)(2906002)(36860700001)(83380400001)(82310400004)(4326008)(8676002)(81166007)(110136005)(5660300002)(921005)(1076003)(86362001)(47076005)(54906003)(316002)(186003)(6636002)(70206006)(70586007)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 08:50:39.6040 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 892e3132-73d9-41d6-34a5-08d9f83be61e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.236];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT037.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4432 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Upcoming changes to Direct IO will change it from acquiring pages via get_user_pages_fast(), to calling pin_user_pages_fast() instead. Place a few assertions at key points, that the pages are IOVEC (user pages), to enforce the assumptions that there are no kernel or pipe or other odd variations being passed. Signed-off-by: John Hubbard --- block/bio.c | 4 ++++ fs/direct-io.c | 2 ++ 2 files changed, 6 insertions(+) diff --git a/block/bio.c b/block/bio.c index b15f5466ce08..4679d6539e2d 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1167,6 +1167,8 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) BUILD_BUG_ON(PAGE_PTRS_PER_BVEC < 2); pages += entries_left * (PAGE_PTRS_PER_BVEC - 1); + WARN_ON_ONCE(!iter_is_iovec(iter)); + size = iov_iter_get_pages(iter, pages, LONG_MAX, nr_pages, &offset); if (unlikely(size <= 0)) return size ? size : -EFAULT; @@ -1217,6 +1219,8 @@ static int __bio_iov_append_get_pages(struct bio *bio, struct iov_iter *iter) BUILD_BUG_ON(PAGE_PTRS_PER_BVEC < 2); pages += entries_left * (PAGE_PTRS_PER_BVEC - 1); + WARN_ON_ONCE(!iter_is_iovec(iter)); + size = iov_iter_get_pages(iter, pages, LONG_MAX, nr_pages, &offset); if (unlikely(size <= 0)) return size ? size : -EFAULT; diff --git a/fs/direct-io.c b/fs/direct-io.c index 38bca4980a1c..7dbbbfef300d 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -169,6 +169,8 @@ static inline int dio_refill_pages(struct dio *dio, struct dio_submit *sdio) { ssize_t ret; + WARN_ON_ONCE(!iter_is_iovec(sdio->iter)); + ret = iov_iter_get_pages(sdio->iter, dio->pages, LONG_MAX, DIO_PAGES, &sdio->from); From patchwork Fri Feb 25 08:50:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12759919 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B36F1C4321E for ; Fri, 25 Feb 2022 08:50:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238674AbiBYIvT (ORCPT ); Fri, 25 Feb 2022 03:51:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45948 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238648AbiBYIvO (ORCPT ); Fri, 25 Feb 2022 03:51:14 -0500 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2068.outbound.protection.outlook.com [40.107.220.68]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3912317AEC4; Fri, 25 Feb 2022 00:50:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WEcuxonxnpTUmRleoW6WM6DYYUTvp6aRGwo8+06mNJt5fp3R2W8edtdCLha1z9a3tooNu+1L69N1WLBqX4rzNC4Ko8T6ehrDWFDtBWalbjqQMYAvPacLicQLmeRnjs6gZDTo7QBV3CgFXi3pKnDcQABTs+arND0aOP+Mxy8WP7dqpZUqQl/UIfgBK4r9SwtWwQC2EldZWdOVGX+dL6+ctABelOV70rlLJcd56nSvSj3ameF2rAsz4IuGMQpruCCVsyb8dED5Uq+p94QCIifBn4USuPviK6lFcl7UL4gcx+6NgykwWbUGit4E7p+/d11cMz7KoRH6TaYoe5I6kwzkrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DpGZ4D4kOXbjY11gED52E3qdRR7ord22Md9fFluP4Sk=; b=J+tpkgUCmc3tS1ZEmjNWxGNhhAjbEBP3YIy+pY6iSuKQbtHM8xTATv5HB8FJRY8r8dCPvC42TG+PtJDm10qUoQ/JKfIKRUy61SgDuYtt7yLhZnQQoEl2L94ebDW0R+gRJd//jmRjp3tK9u2v7mbFqm/UXtiJKdXDV8/P1VX77vh1CES2sxdQqNsYNN016pQuO1W6gfliz/us5hRcp23DTDB4RFY5krihgn5IaQI018YxcVA90EQWMNbNR79bEjdhFBOfzwP0KHP7XKWqC/gHhAhZXydQdIy4Gg/vBPFiLfiGj7GlcGnzmtt20hW/FFVGXF1eqCpplBeYkkie6AIRYg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DpGZ4D4kOXbjY11gED52E3qdRR7ord22Md9fFluP4Sk=; b=DjTK0haWTYIr4n/145v8RajRmD3wYMIRMrd1znySIjlBD5ITNSvUOF59i2WIesFFDChCQMpCjrj9U8nuPnS1uuao9J2Sm8cmZlC4YTw5KWXDhZMjsX6zTV3U8P6GphcsPkiDdMEYwLCYy2ie0fU7WFiGev/gFbHwwu9gzmfto7/4NxS3Cov7Ay9Ri0JtwpB1ewne1Fd8tP6qyJYDJ8fmbSGecJyWqX/U1GdvFUWzosD38KVqQDhbZvbcDJHnI+FV4eOz/+Na+2hhpFvUnvDSUzfBQVTd9/SCXgaNJNsq/QAtTYxcsPOw3fx5LSYGd86pkmthF8lB+yvV3+M3DKNvkA== Received: from MWHPR1601CA0023.namprd16.prod.outlook.com (2603:10b6:300:da::33) by CO6PR12MB5457.namprd12.prod.outlook.com (2603:10b6:5:355::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.21; Fri, 25 Feb 2022 08:50:41 +0000 Received: from CO1NAM11FT018.eop-nam11.prod.protection.outlook.com (2603:10b6:300:da:cafe::7) by MWHPR1601CA0023.outlook.office365.com (2603:10b6:300:da::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.11 via Frontend Transport; Fri, 25 Feb 2022 08:50:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT018.mail.protection.outlook.com (10.13.175.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:40 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 25 Feb 2022 08:50:40 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 25 Feb 2022 00:50:39 -0800 Received: from sandstorm.attlocal.net (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 25 Feb 2022 00:50:37 -0800 From: John Hubbard To: Jens Axboe , Jan Kara , Christoph Hellwig , Dave Chinner , "Darrick J . Wong" , Theodore Ts'o , Alexander Viro , Miklos Szeredi , Andrew Morton , Chaitanya Kulkarni CC: , , , , LKML , John Hubbard Subject: [RFC PATCH 4/7] block, bio, fs: initial pin_user_pages_fast() changes Date: Fri, 25 Feb 2022 00:50:22 -0800 Message-ID: <20220225085025.3052894-5-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220225085025.3052894-1-jhubbard@nvidia.com> References: <20220225085025.3052894-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: da1004e2-f5ad-4439-b72a-08d9f83be6ef X-MS-TrafficTypeDiagnostic: CO6PR12MB5457:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: N4jpCwG3jip3+F70tpFrej0AYMVbTi05Y0CCyl4iIDdU6RHU2YkQ2PQg8wpVhcjKwI+ScOUWteHJ9f7CSoYle9B5V5DDj6BUV3WdtQ/6UT/aX/X6hevW3vSyjjEf53L34WpDFTQjKTHeBCt5TrApT2X2U7XM4xU5QKdk2kpQp0DVzEEhBEPE+rOpjTfgfGon5jw+BTbUvHD7GG19Z0hnoeboX7PuL7ahOfbqE1REwDyxWIA3lPtp6qnKtWxKi+OND4RTmwoJ3UIvDrJBr3l2u60FickaYm9Cr5p+UOnmE0Dd4iJS+SSL756VGlny8CnAQRiy8YLA2LRRBI46nW1oR/xbLna4//XfSiKAqMtpdEbJQC2Rg6v4o9M8imNCThR2KSwhYxsEqOTHqMitAS5DQKx1MeKY4TrPt0TQ2JM0XqV0WajwzZIYCdbURtJKJHFahlqSW6LOlFExP+AxSVHC8qgwQGwVCuvsdIAQNV5nlOWTB2N5a6IivN8Z4Z/IPICthmba7+BYHIU0yNuWJaAjhfa061WPbX9laqNzjfyvpskrg6I0Q9nt6HzlP+IZGgVrsxGGWv5UsLyBZpnA5wi5TfywqhfzGynxZaDjHRnvLe5/s7wNtHT+dp40wCewwlNsLptVy825VyNWxFhY31+O6LdwErxAv3Rd7XRrmv+MgSvSKXdbSlTNA2cZtrH9gVi4neA6BeGVqk6xUfmv8c3Mdm+BYeBozbCwaUKl/kC7/3s= X-Forefront-Antispam-Report: CIP:12.22.5.236;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(6636002)(2616005)(316002)(1076003)(426003)(336012)(186003)(26005)(2906002)(6666004)(107886003)(82310400004)(110136005)(54906003)(83380400001)(36860700001)(4326008)(81166007)(5660300002)(356005)(921005)(36756003)(8936002)(7416002)(40460700003)(8676002)(86362001)(70206006)(70586007)(47076005)(508600001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 08:50:40.9458 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: da1004e2-f5ad-4439-b72a-08d9f83be6ef X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.236];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT018.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5457 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Use dio_w_*() wrapper calls, in place of get_user_pages_fast(), get_page() and put_page(). This sets up core parts of the block, bio, and direct-io subsystems to use dio_*() wrapper calls. Signed-off-by: John Hubbard --- block/bio.c | 18 +++++++++--------- block/blk-map.c | 4 ++-- fs/direct-io.c | 24 ++++++++++++------------ fs/iomap/direct-io.c | 2 +- lib/iov_iter.c | 4 ++-- 5 files changed, 26 insertions(+), 26 deletions(-) diff --git a/block/bio.c b/block/bio.c index 4679d6539e2d..8541245c53bd 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1102,7 +1102,7 @@ void __bio_release_pages(struct bio *bio, bool mark_dirty) bio_for_each_segment_all(bvec, bio, iter_all) { if (mark_dirty && !PageCompound(bvec->bv_page)) set_page_dirty_lock(bvec->bv_page); - put_page(bvec->bv_page); + dio_w_unpin_user_page(bvec->bv_page); } } EXPORT_SYMBOL_GPL(__bio_release_pages); @@ -1133,7 +1133,7 @@ static void bio_put_pages(struct page **pages, size_t size, size_t off) size_t i, nr = DIV_ROUND_UP(size + (off & ~PAGE_MASK), PAGE_SIZE); for (i = 0; i < nr; i++) - put_page(pages[i]); + dio_w_unpin_user_page(pages[i]); } #define PAGE_PTRS_PER_BVEC (sizeof(struct bio_vec) / sizeof(struct page *)) @@ -1144,9 +1144,9 @@ static void bio_put_pages(struct page **pages, size_t size, size_t off) * @iter: iov iterator describing the region to be mapped * * Pins pages from *iter and appends them to @bio's bvec array. The - * pages will have to be released using put_page() when done. - * For multi-segment *iter, this function only adds pages from the - * next non-empty segment of the iov iterator. + * pages will have to be released using unpin_user_page() when done. For + * multi-segment *iter, this function only adds pages from the next non-empty + * segment of the iov iterator. */ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) { @@ -1180,7 +1180,7 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) if (__bio_try_merge_page(bio, page, len, offset, &same_page)) { if (same_page) - put_page(page); + dio_w_unpin_user_page(page); } else { if (WARN_ON_ONCE(bio_full(bio, len))) { bio_put_pages(pages + i, left, offset); @@ -1237,7 +1237,7 @@ static int __bio_iov_append_get_pages(struct bio *bio, struct iov_iter *iter) break; } if (same_page) - put_page(page); + dio_w_unpin_user_page(page); offset = 0; } @@ -1434,8 +1434,8 @@ void bio_set_pages_dirty(struct bio *bio) * the BIO and re-dirty the pages in process context. * * It is expected that bio_check_pages_dirty() will wholly own the BIO from - * here on. It will run one put_page() against each page and will run one - * bio_put() against the BIO. + * here on. It will run one unpin_user_page() against each page and will run + * one bio_put() against the BIO. */ static void bio_dirty_fn(struct work_struct *work); diff --git a/block/blk-map.c b/block/blk-map.c index c7f71d83eff1..5f0d04c5dd9d 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -275,7 +275,7 @@ static int bio_map_user_iov(struct request *rq, struct iov_iter *iter, if (!bio_add_hw_page(rq->q, bio, page, n, offs, max_sectors, &same_page)) { if (same_page) - put_page(page); + dio_w_unpin_user_page(page); break; } @@ -289,7 +289,7 @@ static int bio_map_user_iov(struct request *rq, struct iov_iter *iter, * release the pages we didn't map into the bio, if any */ while (j < npages) - put_page(pages[j++]); + dio_w_unpin_user_page(pages[j++]); kvfree(pages); /* couldn't stuff something into bio? */ if (bytes) diff --git a/fs/direct-io.c b/fs/direct-io.c index 7dbbbfef300d..0d7104aa40c3 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -183,7 +183,7 @@ static inline int dio_refill_pages(struct dio *dio, struct dio_submit *sdio) */ if (dio->page_errors == 0) dio->page_errors = ret; - get_page(page); + dio_w_pin_user_page(page); dio->pages[0] = page; sdio->head = 0; sdio->tail = 1; @@ -452,7 +452,7 @@ static inline void dio_bio_submit(struct dio *dio, struct dio_submit *sdio) static inline void dio_cleanup(struct dio *dio, struct dio_submit *sdio) { while (sdio->head < sdio->tail) - put_page(dio->pages[sdio->head++]); + dio_w_unpin_user_page(dio->pages[sdio->head++]); } /* @@ -717,7 +717,7 @@ static inline int dio_bio_add_page(struct dio_submit *sdio) */ if ((sdio->cur_page_len + sdio->cur_page_offset) == PAGE_SIZE) sdio->pages_in_io--; - get_page(sdio->cur_page); + dio_w_pin_user_page(sdio->cur_page); sdio->final_block_in_bio = sdio->cur_page_block + (sdio->cur_page_len >> sdio->blkbits); ret = 0; @@ -832,13 +832,13 @@ submit_page_section(struct dio *dio, struct dio_submit *sdio, struct page *page, */ if (sdio->cur_page) { ret = dio_send_cur_page(dio, sdio, map_bh); - put_page(sdio->cur_page); + dio_w_unpin_user_page(sdio->cur_page); sdio->cur_page = NULL; if (ret) return ret; } - get_page(page); /* It is in dio */ + dio_w_pin_user_page(page); /* It is in dio */ sdio->cur_page = page; sdio->cur_page_offset = offset; sdio->cur_page_len = len; @@ -853,7 +853,7 @@ submit_page_section(struct dio *dio, struct dio_submit *sdio, struct page *page, ret = dio_send_cur_page(dio, sdio, map_bh); if (sdio->bio) dio_bio_submit(dio, sdio); - put_page(sdio->cur_page); + dio_w_unpin_user_page(sdio->cur_page); sdio->cur_page = NULL; } return ret; @@ -953,7 +953,7 @@ static int do_direct_IO(struct dio *dio, struct dio_submit *sdio, ret = get_more_blocks(dio, sdio, map_bh); if (ret) { - put_page(page); + dio_w_unpin_user_page(page); goto out; } if (!buffer_mapped(map_bh)) @@ -998,7 +998,7 @@ static int do_direct_IO(struct dio *dio, struct dio_submit *sdio, /* AKPM: eargh, -ENOTBLK is a hack */ if (dio->op == REQ_OP_WRITE) { - put_page(page); + dio_w_unpin_user_page(page); return -ENOTBLK; } @@ -1011,7 +1011,7 @@ static int do_direct_IO(struct dio *dio, struct dio_submit *sdio, if (sdio->block_in_file >= i_size_aligned >> blkbits) { /* We hit eof */ - put_page(page); + dio_w_unpin_user_page(page); goto out; } zero_user(page, from, 1 << blkbits); @@ -1051,7 +1051,7 @@ static int do_direct_IO(struct dio *dio, struct dio_submit *sdio, sdio->next_block_for_io, map_bh); if (ret) { - put_page(page); + dio_w_unpin_user_page(page); goto out; } sdio->next_block_for_io += this_chunk_blocks; @@ -1067,7 +1067,7 @@ static int do_direct_IO(struct dio *dio, struct dio_submit *sdio, } /* Drop the ref which was taken in get_user_pages() */ - put_page(page); + dio_w_unpin_user_page(page); } out: return ret; @@ -1289,7 +1289,7 @@ do_blockdev_direct_IO(struct kiocb *iocb, struct inode *inode, ret2 = dio_send_cur_page(dio, &sdio, &map_bh); if (retval == 0) retval = ret2; - put_page(sdio.cur_page); + dio_w_unpin_user_page(sdio.cur_page); sdio.cur_page = NULL; } if (sdio.bio) diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index 67cf9c16f80c..4e818648a1aa 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -192,7 +192,7 @@ static void iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio, bio->bi_private = dio; bio->bi_end_io = iomap_dio_bio_end_io; - get_page(page); + dio_w_pin_user_page(page); __bio_add_page(bio, page, len, 0); iomap_dio_submit_bio(iter, dio, bio, pos); } diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 6dd5330f7a99..f15a3ef5a481 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -1538,7 +1538,7 @@ ssize_t iov_iter_get_pages(struct iov_iter *i, addr = first_iovec_segment(i, &len, start, maxsize, maxpages); n = DIV_ROUND_UP(len, PAGE_SIZE); - res = get_user_pages_fast(addr, n, gup_flags, pages); + res = dio_w_pin_user_pages_fast(addr, n, gup_flags, pages); if (unlikely(res <= 0)) return res; return (res == n ? len : res * PAGE_SIZE) - *start; @@ -1667,7 +1667,7 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i, p = get_pages_array(n); if (!p) return -ENOMEM; - res = get_user_pages_fast(addr, n, gup_flags, p); + res = dio_w_pin_user_pages_fast(addr, n, gup_flags, p); if (unlikely(res <= 0)) { kvfree(p); *pages = NULL; From patchwork Fri Feb 25 08:50:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12759920 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97B0FC4332F for ; Fri, 25 Feb 2022 08:50:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238677AbiBYIvU (ORCPT ); Fri, 25 Feb 2022 03:51:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238659AbiBYIvS (ORCPT ); Fri, 25 Feb 2022 03:51:18 -0500 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2045.outbound.protection.outlook.com [40.107.237.45]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1697117BC41; Fri, 25 Feb 2022 00:50:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GScl9BenBELXDXb+6CD6bKA7k6rOQ5epj//MZcZFU31oW6psUcZn9InZyemWqPiF5Srt2s3esktruIz24UqRggsxZjKN2rSwAnfNDx6C8fETCtKKGzJzNWZNE+WCZYhkyKmbvvmUt8fAMSbuC3G+nykKMPxwOkJOhY+T34/IRNHpBBtuwb+jON21edEFCm6/TTGpHpNjYbgv1tdsJnJnWvJqO3taxNNFll39bZCYJwCJxSIQ+ZyllkZQMzir5Qhca6WzxDVLpIrC8IN1FK7UqCtBUwnpKjSqH5SG58LPl/wgQleqJ18fM7WAbwuKgk3g+CU0ryvGN9ST1ib0+8q1vg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=P4QhKeT9zgdstYjOxfhw3Jd7XZ11uxPnfOza55tfrhs=; b=Dt8lBYaS7lwi78FUqI5h9nXg1i8Yfz8WTUVqtw/ISzTdT8UC05k67vlN+jU0p1HLqx0JTpvIZ5OdPF/G4VkHHkAIYK0N2fyP1ioUdQkWhb9Vry30L6pKR55Jd77EBzeW5aiz/RHEwQ1KAwRNh1IvJArhuxl6VN7aM0Bmd3Npnh0KR/fnkvcyi2TAAjxsxy8R9V0MhAuLLmsJDHBhHGF0DKyN57MU4icP9EFKWZSkuDUxp27Djxjhb4cTAWNSlsqzdyaufhIqA3KO0fpPclq63BRy1nf0A910p95Teebyjkm1c61GRQMbI2kouwG3LspQWAJ3cy+4GS8MsmYch2VUyQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=P4QhKeT9zgdstYjOxfhw3Jd7XZ11uxPnfOza55tfrhs=; b=myiqbymWylkCR/Nhklgo4NfPGEHFsvdGyFN3gJUgLjRC0YEFGqkkr5+ooobHeNqOKgiy8JAV0VXYjhuXouT4YkVvNtAqDv9SSRJTvLn/F/x5GkWTkB8gCqSSGIMYHciOHUPprDSm1ZhPpaFmZWC99OHmVCaBRyVDVG9VbZZ3DyeDaytNbkyYbe2HRqWqAAFaLnuDGDfFUgkw6KTeVqmm+j0gj7PEVI8uFHBg0AlS6m7jvYgBlQVnpNoA5wD10P3Jr9anHNuWuzoXHyiV4C2EqlLlTAZkKGJQ0hSQqabZTiwtyO8hkAwi6vwj0gbK/aqWRQc8KHTZFARil2SACZA4EA== Received: from DM5PR12CA0049.namprd12.prod.outlook.com (2603:10b6:3:103::11) by MWHPR1201MB0256.namprd12.prod.outlook.com (2603:10b6:301:53::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.24; Fri, 25 Feb 2022 08:50:43 +0000 Received: from DM6NAM11FT019.eop-nam11.prod.protection.outlook.com (2603:10b6:3:103:cafe::bd) by DM5PR12CA0049.outlook.office365.com (2603:10b6:3:103::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.23 via Frontend Transport; Fri, 25 Feb 2022 08:50:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.238) by DM6NAM11FT019.mail.protection.outlook.com (10.13.172.172) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:43 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 25 Feb 2022 08:50:42 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 25 Feb 2022 00:50:41 -0800 Received: from sandstorm.attlocal.net (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 25 Feb 2022 00:50:40 -0800 From: John Hubbard To: Jens Axboe , Jan Kara , Christoph Hellwig , Dave Chinner , "Darrick J . Wong" , Theodore Ts'o , Alexander Viro , Miklos Szeredi , Andrew Morton , Chaitanya Kulkarni CC: , , , , LKML , John Hubbard Subject: [RFC PATCH 5/7] NFS: direct-io: convert to FOLL_PIN pages Date: Fri, 25 Feb 2022 00:50:23 -0800 Message-ID: <20220225085025.3052894-6-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220225085025.3052894-1-jhubbard@nvidia.com> References: <20220225085025.3052894-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b7e90141-910f-495c-939a-08d9f83be833 X-MS-TrafficTypeDiagnostic: MWHPR1201MB0256:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zTqxQpF83G0gwsqmu6gw7wRmodhDJ3kf0EdhJvT31Oz2DxULS3d6Br9SjbF7i8VFPgCb55E165cuUegGM136z1uPz73oXXbz+/iXhweQY5tlk2uJgzp7X2Vbek80Kl7NRox5jlvsHPOQOyFeMDk50Sid0mxnjq/G5ICFkSuCPoouJ1rYZXDNG7n5gzz+Zh9FsilOAVuR+MBMV4Bg6gXJobUjm3iJSQY/7HL/6i1EuQybwTrt6LswCqxznOXm+7OfArf51w6GHNPplxzlGLmH35T2kuMKSiWUjBFnd6G8PHEj+jWln1Src6cHoRq2RmJ0Ulq0mzLSovqGmtAFTHUXQapD7FKJSd5gKVGtqgZy7tgVDkvGeTqp3vdF1Gp5xe9q/h7gcAY7dgq9Qox7oOnRRvEGZlWohK7+X6FnFtW/GSwoT14CHNIqknceVcoW4Yk3L9vj5A8KuAw/cjdax2S+V3qImP0egjQ6gtGRUlqKO+8on1vLykgmmhPqkbTQnyDIJj2/xjBdhb2yDP0uKxF1HgVwn1bDK54XIZScRJeJbxYMs/R3BB+SiaQpcVgrrXUuNJ6znRl0zxJCkQixsFy1rxiNyGiLZvN42OJmx0VJPEW4X0qDubB4qId5ce9L4rR9MP0f52/e+CUwFAlhQr35Jo29fSCaYeZhtS05IXCl6kCayGLltjjRxz5AntCZ8XEfuFhrGnp+jTMVj17B08giZ72xxc3h8Cf6ZcdJPSyanU8= X-Forefront-Antispam-Report: CIP:12.22.5.238;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(46966006)(36840700001)(40470700004)(4326008)(6636002)(2906002)(508600001)(316002)(47076005)(186003)(336012)(36860700001)(26005)(110136005)(83380400001)(54906003)(36756003)(426003)(107886003)(2616005)(921005)(82310400004)(7416002)(356005)(40460700003)(86362001)(5660300002)(6666004)(8936002)(70206006)(8676002)(81166007)(1076003)(70586007)(4744005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 08:50:43.0682 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b7e90141-910f-495c-939a-08d9f83be833 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.238];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT019.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB0256 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Now that Direct IO's core allocators invoke pin_user_pages_fast(), those pages must free released via unpin_user_page(), instead of put_page(). Signed-off-by: John Hubbard --- fs/nfs/direct.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index eabfdab543c8..2e0d399c5a5a 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -181,7 +181,7 @@ static void nfs_direct_release_pages(struct page **pages, unsigned int npages) { unsigned int i; for (i = 0; i < npages; i++) - put_page(pages[i]); + dio_w_unpin_user_page(pages[i]); } void nfs_init_cinfo_from_dreq(struct nfs_commit_info *cinfo, From patchwork Fri Feb 25 08:50:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12759922 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD954C433EF for ; Fri, 25 Feb 2022 08:50:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238696AbiBYIvZ (ORCPT ); Fri, 25 Feb 2022 03:51:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238663AbiBYIvT (ORCPT ); Fri, 25 Feb 2022 03:51:19 -0500 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam07on2074.outbound.protection.outlook.com [40.107.212.74]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 65FEF17BC4D; Fri, 25 Feb 2022 00:50:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=edu6bmnk/q3Pz7CSfnwZvK4X5kz9+qsV9hpYeyh3oaS879yoBxKmQPr+GfMU18JdK6yDWRPf9VB64KpEdm7RewrNheaqivHWU0/d2zCQmLzDcOpc3oUzQ6Pja8gBvykUrPW3pGhwpNOeZYLm477yWOD0kbOTpqfMTT6ip8mmGnkJp3UQRf/00y1qV+Ox/+Kpvsq7+7W/6dJe+Y1/pJ5VOWVsIZY6HaBJx+bhvJ6kEIh5wDuaMMFPW7C3Z6bQV9/g4qeD1Ts0Ll0KkgghR8C/ciQAS7MfjuKMYdLfvcFmtQzDakuejd3dlxYnbP4pDJMdbEEbaiKQHm8+XYCvMiuVCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tWclyEKq4CVkl7hdRiKCw5b/8BrSgghcSXNAqeSOe0M=; b=AwZ9tg/Hxrv5DsLNSqALjbpd0qtYfnjvkN31rDe70RCCLByUC1XWV20VZUUk1JufA+nHqwt9ErWI11brGMwwikSfqAnLpsjgrb2AzTl0WGpMJawkO+xYA7rl9QCU04AMQirVK4HLywEc4yozTdzZ83btEvYXWF9tB3zo9pDFyWmmovDjUqX4ryOElF5HCSGrQeUClvClnXuJubziaGBBMtQ3gKf3aIBu675gGCOcdZtYW6kvd63XakYKziLQd9Xrc1DzazwyjjeX0sC//nN1d4WroYqJVZ3tMhgNIL1Jphf9c1HVgfVram9Zkylkon9tSuIkbSqcottU0A9Yzy7QRw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tWclyEKq4CVkl7hdRiKCw5b/8BrSgghcSXNAqeSOe0M=; b=j+X9aDhPNh5/uMJybg0OIztzh0JteNtmcMpwgRWtoxPx9ekv8c0UgOuUZpT3SYpBmSrLtchssDwceUFhwDBtnegaUDe1GDE8eKMPdCIZILSbvbH8WapNj15QbNO4PTa1dHomFJFzodhYVrjRJJCT6079jM89DBIp31KMNYa/NXbOBRDN6Io0xU6UrCd1JlTuM3BEqA2Pij3UMva3blpq5Yk00TSG/VxcSDEKzAMakUURFVcX1LmVA55UgKjyKytsSI08KgfzzU5k64AjCOUM4x0v83c2nEyKliBhID6HqCSGGhwxmMIe0ks1KrD5Fu7NMoj14WLvV18kkfyXugEU/g== Received: from BN0PR04CA0139.namprd04.prod.outlook.com (2603:10b6:408:ed::24) by CH0PR12MB5074.namprd12.prod.outlook.com (2603:10b6:610:e1::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.21; Fri, 25 Feb 2022 08:50:45 +0000 Received: from BN8NAM11FT007.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ed:cafe::66) by BN0PR04CA0139.outlook.office365.com (2603:10b6:408:ed::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by BN8NAM11FT007.mail.protection.outlook.com (10.13.177.109) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:45 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 25 Feb 2022 08:50:44 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 25 Feb 2022 00:50:43 -0800 Received: from sandstorm.attlocal.net (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 25 Feb 2022 00:50:41 -0800 From: John Hubbard To: Jens Axboe , Jan Kara , Christoph Hellwig , Dave Chinner , "Darrick J . Wong" , Theodore Ts'o , Alexander Viro , Miklos Szeredi , Andrew Morton , Chaitanya Kulkarni CC: , , , , LKML , John Hubbard Subject: [RFC PATCH 6/7] fuse: convert direct IO paths to use FOLL_PIN Date: Fri, 25 Feb 2022 00:50:24 -0800 Message-ID: <20220225085025.3052894-7-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220225085025.3052894-1-jhubbard@nvidia.com> References: <20220225085025.3052894-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e14dc0c9-f000-4920-5c71-08d9f83be98e X-MS-TrafficTypeDiagnostic: CH0PR12MB5074:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: utN9OnhBYfjLaaMUa/ek04txJYqDYRj0suaZTme+Nk7I0fQNDmfMFgZxyUeymK5aCy6c/ophxUtBZYeMkFbULHxkvLrHY/Vxd1wHjhBdVrLJ/sPSWNZw44asH9+UEC2qL6ZOEvJk5UHVGXSZYd2P+idVZNRlkADx3fHfbwtMn519F6FGoWWUsIq0mmSFOloMMoywXNLCQF8j44k+kD+OmU04+jV5oL41tWdySwBvGjaefGKHbr5f1rKQ6OzjtEyQyQK09PY6kyh+gW/15Q76w9loX+xRMoewEj8TA5skT/4oWuNw1FjfHk9zdGzQwMV34CvER6shuj6Sst34qDSnArudSJvY5VvT5wIu81t/1WW51a0/p92lDsaiF7sti7sptewJayv8HHq1jI/B/dlNaEUHLvqvS5daeF6ITGGCtyYMK5KUrSaVFjQu94w7QQS2bINVpudxnTiuMVI5+hac3JfKs+gRvoL/DPimXjRgoJUdHDxy6hgvWvCBMLVe8Ec+OaWQOt4/aWBLcc/yQ2uAxkT0ZrahV1qHivooso1o2LNq0zrQd6r84pGiMEjp7g4VLzclFwy5NIWlvA24zNZLNyybHHxdS5hnBi1tMS2r1XDXbF8kaKeF0BEObdFmO8+UM795qD8Q/VuCdT+lkFUbms5eVLMmADj+VqdEHy752tdoJdmU6kj7+xFRCXdtL+TWobaWAGW+WBsRK/4crIlFi9StDq9uEndIxuJGL1Q6UpdY6WXrX7cKEEULivHZqdH5dWxCz3KaSenB2V/kaVrun+9AyvGxmEujIFY+YPPI8xgBZthMUSYKBIS/orJ6Sndd X-Forefront-Antispam-Report: CIP:12.22.5.235;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(70206006)(2906002)(40460700003)(86362001)(82310400004)(70586007)(426003)(7416002)(8936002)(107886003)(6666004)(186003)(2616005)(1076003)(26005)(83380400001)(508600001)(336012)(54906003)(966005)(921005)(356005)(316002)(6636002)(81166007)(110136005)(36860700001)(47076005)(8676002)(36756003)(4326008)(5660300002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 08:50:45.3097 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e14dc0c9-f000-4920-5c71-08d9f83be98e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.235];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT007.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB5074 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Convert the fuse filesystem to support the new iov_iter_get_pages() behavior. That routine now invokes pin_user_pages_fast(), which means that such pages must be released via unpin_user_page(), rather than via put_page(). This commit also removes any possibility of kernel pages being handled, in the fuse_get_user_pages() call. Although this may seem like a steep price to pay, Christoph Hellwig actually recommended it a few years ago for nearly the same situation [1]. [1] https://lore.kernel.org/kvm/20190724061750.GA19397@infradead.org/ Signed-off-by: John Hubbard --- fs/fuse/dev.c | 5 ++++- fs/fuse/file.c | 23 ++++++++--------------- 2 files changed, 12 insertions(+), 16 deletions(-) diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c index e1b4a846c90d..a93037c96b89 100644 --- a/fs/fuse/dev.c +++ b/fs/fuse/dev.c @@ -675,7 +675,10 @@ static void fuse_copy_finish(struct fuse_copy_state *cs) flush_dcache_page(cs->pg); set_page_dirty_lock(cs->pg); } - put_page(cs->pg); + if (cs->pipebufs) + put_page(cs->pg); + else + unpin_user_page(cs->pg); } cs->pg = NULL; } diff --git a/fs/fuse/file.c b/fs/fuse/file.c index 94747bac3489..395c2fb613fb 100644 --- a/fs/fuse/file.c +++ b/fs/fuse/file.c @@ -619,7 +619,7 @@ static void fuse_release_user_pages(struct fuse_args_pages *ap, for (i = 0; i < ap->num_pages; i++) { if (should_dirty) set_page_dirty_lock(ap->pages[i]); - put_page(ap->pages[i]); + unpin_user_page(ap->pages[i]); } } @@ -1382,20 +1382,13 @@ static int fuse_get_user_pages(struct fuse_args_pages *ap, struct iov_iter *ii, size_t nbytes = 0; /* # bytes already packed in req */ ssize_t ret = 0; - /* Special case for kernel I/O: can copy directly into the buffer */ - if (iov_iter_is_kvec(ii)) { - unsigned long user_addr = fuse_get_user_addr(ii); - size_t frag_size = fuse_get_frag_size(ii, *nbytesp); - - if (write) - ap->args.in_args[1].value = (void *) user_addr; - else - ap->args.out_args[0].value = (void *) user_addr; - - iov_iter_advance(ii, frag_size); - *nbytesp = frag_size; - return 0; - } + /* + * TODO: this warning can eventually be removed. It is just to help + * developers, during a transitional period, while direct IO code is + * moving over to FOLL_PIN pages. + */ + if (WARN_ON_ONCE(!iter_is_iovec(ii))) + return -EOPNOTSUPP; while (nbytes < *nbytesp && ap->num_pages < max_pages) { unsigned npages; From patchwork Fri Feb 25 08:50:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12759921 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C4A2C433F5 for ; Fri, 25 Feb 2022 08:50:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238669AbiBYIvW (ORCPT ); Fri, 25 Feb 2022 03:51:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238687AbiBYIvV (ORCPT ); Fri, 25 Feb 2022 03:51:21 -0500 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2052.outbound.protection.outlook.com [40.107.94.52]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CB60E177D3F; Fri, 25 Feb 2022 00:50:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EzODPZatyn9W0Fjj7PNYZPtgjou+HfkkWOMOk915RdLejpI39ZqHGopKxsWvfWFf7q6veCcuj8Apyg+4UQonViQSe/m4G7A+jImdlNIW/9moN4gGGT7EIe+yvI1Py37VD9nG6tt9wgi+ri3S0GBykg+Hrk0y4U/pn59YW+++Te/KKHAd3QQ6KuByYfbiu9lU0/oFWeQNlyVpm7zWxkFh+HPLUyHw/3oDOEX+SR5RBJ2yGBpCOzPeT/D2rEe7ws2NicDbrzXzKYOg+x6J8ub90dFMrYqtdCDWgKFsSsdllC6Djp4D9v3Rkwq8ob3BW7Hm0V6dfmrBy2aj1j00vbYkbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mVjachwLZOJaasM1HxAYAlh+fxfrxwqQnXama3fFC5A=; b=ZvvE3gAjRZD6lkDf6bp/VvilesY74JFtFJVK8lP4sJf5cHb1ISF/3bgejS8VwMNP2QeQa39wUl/ZK9maOvC+e3Psgpxk7T6I/6T5Bd5Fw6PxyypwH56xxoN1KKIB4j/e1L/2VDSlSgSYnPJXWAbr0r2NiaYjVT5aklWwUt+1C4wpylkPofOLcFwCHgpNmW+D6Z4j60vrWC17zvplbX62/lLIzkmRgEVuPgHYUNO2bg3Yf2orHzWKEhyvTszx5ouJv8uTSrKctIzgyryhH4RYorsrFMRz212az+eaG1zEsC5gpY8bZ7jlIhcOhnVhhk6w1+u03jlRJHwvlQrqZ4E3yQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mVjachwLZOJaasM1HxAYAlh+fxfrxwqQnXama3fFC5A=; b=Zve+2qknEQaTEYfdwjqqoB7LB8CZjjmFxrhuc3oz3egNKcdqFqHa7Wixckcz4FMkeKnTuihuK9Ow/bhYkFCWtp1csu+piBVAdlXPzIloOZmXuf5TptO/xUtIyJPj2yN9k/7j9DTrcyAM1ZoNgH0RQc4DErdFhbUN5lP/KknK1lPgVZZa2EZWDf6X0FAn9Qyl7ztQ2V3NvnpVWbaqN0JW92GKUXndT7PaMDhMDLk0Dq9+1rMjgf2qSQ+ODj5lBRpLnplNVEQFinAYoYfGCQXtDyWYzYh2QI0r2nIs+iKuAWNUHRz8zpbQM9IgDtD7XkTon4zN5ym55dQe6+6AjYqktg== Received: from BN9PR03CA0806.namprd03.prod.outlook.com (2603:10b6:408:13f::31) by BN6PR1201MB0180.namprd12.prod.outlook.com (2603:10b6:405:56::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.22; Fri, 25 Feb 2022 08:50:48 +0000 Received: from BN8NAM11FT054.eop-nam11.prod.protection.outlook.com (2603:10b6:408:13f:cafe::43) by BN9PR03CA0806.outlook.office365.com (2603:10b6:408:13f::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.24 via Frontend Transport; Fri, 25 Feb 2022 08:50:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by BN8NAM11FT054.mail.protection.outlook.com (10.13.177.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:47 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 25 Feb 2022 08:50:46 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 25 Feb 2022 00:50:45 -0800 Received: from sandstorm.attlocal.net (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 25 Feb 2022 00:50:44 -0800 From: John Hubbard To: Jens Axboe , Jan Kara , Christoph Hellwig , Dave Chinner , "Darrick J . Wong" , Theodore Ts'o , Alexander Viro , Miklos Szeredi , Andrew Morton , Chaitanya Kulkarni CC: , , , , LKML , John Hubbard Subject: [RFC PATCH 7/7] block, direct-io: flip the switch: use pin_user_pages_fast() Date: Fri, 25 Feb 2022 00:50:25 -0800 Message-ID: <20220225085025.3052894-8-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220225085025.3052894-1-jhubbard@nvidia.com> References: <20220225085025.3052894-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6480806c-cca4-4a7c-20ec-08d9f83beb12 X-MS-TrafficTypeDiagnostic: BN6PR1201MB0180:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Lv1rCmizZyWkMNp8GC3k5HyeJ/MMauJ+khLsMeeEYr3My1wa3QL0fdCQFQ23HHqE7WbShOdxgO3/+QGrSOZ+YdeRCRKuDx18PQpVt6QwRqvlYR8TIA82pe0sqJXXgoW49SMM190IM8YZlMTa0ue/j4PXBr7v5D6StzTIjIPHxf9u+PibqrvSkOOZm+QwKwdd/DKQs9mYPLoR4ao5fgz2treiO9ZYnCSEz7f4GokGtu17y0OGMtLAjba2JEncRfcxtwcr2txPuiCSK/XlDUs7C7e+Fa3BmiWwbZ2ZMwtdYFxFAUcRCCxUJglPlbbz+pj4YFnwibpAR1lktacjqpr/X+LPzHxqWteir+V4HAbHYufxqPMvBxyw9lBTvlqxOKQlMZI6e9NiWUkvEFhK1ibu0fwVEnEBPdoFxiWjRRZ0SF+clBeWOSgLhiXI5bbBM29IJTCuzm8BwDov1MO4oxicQa1Nd64HxuNMjGOtzwjZcuRxzzGyvZJGdqvY+oazJoVeizRu8HPSxXd/xqgjyKJgK2a3EvDw9c3YzTOPmAh5+9oXTGQSHpUS4194VBy24VALZO/CeGHuVD4S5uilFwZLjwH+xmuD0fZjIXXx7vSKgAxwA51n4KkinxTtqpxKbwelRdtbgOe+XvUkrm9AibGNBsvW/OFUGCg33qb9VIcyw8+iizU15/sdzHLefCKDp0RB/sSxiU4XngatORUXOErV89OB1FVyj7MhCmMiiPUs2RM= X-Forefront-Antispam-Report: CIP:12.22.5.235;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(83380400001)(82310400004)(336012)(6666004)(40460700003)(110136005)(6636002)(316002)(54906003)(36860700001)(2616005)(1076003)(426003)(47076005)(186003)(81166007)(7416002)(2906002)(36756003)(508600001)(70586007)(356005)(921005)(70206006)(8676002)(86362001)(4326008)(107886003)(26005)(8936002)(5660300002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 08:50:47.8353 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6480806c-cca4-4a7c-20ec-08d9f83beb12 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.235];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT054.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1201MB0180 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Remove CONFIG_BLK_USE_PIN_USER_PAGES_FOR_DIO, but leave the dio_w_*() wrapper functions in place, with the pin_user_pages_fast() defines as the only choice. A subsequent patch is now possible, to rename all dio_w_*() functions so as to remove the dio_w_ prefix. Signed-off-by: John Hubbard --- block/Kconfig | 25 ------------------------- include/linux/bvec.h | 7 ------- 2 files changed, 32 deletions(-) diff --git a/block/Kconfig b/block/Kconfig index f6ca5e9597e4..168b873eb666 100644 --- a/block/Kconfig +++ b/block/Kconfig @@ -50,31 +50,6 @@ config BLK_DEV_BSG_COMMON config BLK_ICQ bool -config BLK_USE_PIN_USER_PAGES_FOR_DIO - bool "DEVELOPERS ONLY: Enable pin_user_pages() for Direct IO" if EXPERT - default n - help - For Direct IO code, retain the pages via calls to - pin_user_pages_fast(), instead of via get_user_pages_fast(). - Likewise, use pin_user_page() instead of get_page(). And then - release such pages via unpin_user_page(), instead of - put_page(). - - This is a temporary setting, which will be deleted once the - conversion is completed, reviewed, and tested. In the meantime, - developers can enable this in order to try out each filesystem. - For that, it's best to monitor these /proc/vmstat items: - - nr_foll_pin_acquired - nr_foll_pin_released - - ...to ensure that they remain equal, when "at rest". - - Say yes here ONLY if are actively developing or testing the - block layer or filesystems with pin_user_pages_fast(). - Otherwise, this is just a way to throw off the refcounting of - pages in the system. - config BLK_DEV_BSGLIB bool "Block layer SG support v4 helper lib" select BLK_DEV_BSG_COMMON diff --git a/include/linux/bvec.h b/include/linux/bvec.h index a96a68c687f6..5bc98b334efe 100644 --- a/include/linux/bvec.h +++ b/include/linux/bvec.h @@ -241,15 +241,8 @@ static inline void *bvec_virt(struct bio_vec *bvec) return page_address(bvec->bv_page) + bvec->bv_offset; } -#ifdef CONFIG_BLK_USE_PIN_USER_PAGES_FOR_DIO #define dio_w_pin_user_pages_fast(s, n, p, f) pin_user_pages_fast(s, n, p, f) #define dio_w_pin_user_page(p) pin_user_page(p) #define dio_w_unpin_user_page(p) unpin_user_page(p) -#else -#define dio_w_pin_user_pages_fast(s, n, p, f) get_user_pages_fast(s, n, p, f) -#define dio_w_pin_user_page(p) get_page(p) -#define dio_w_unpin_user_page(p) put_page(p) -#endif - #endif /* __LINUX_BVEC_H */