From patchwork Fri Feb 25 08:50:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12759907 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2D6FC433F5 for ; Fri, 25 Feb 2022 08:50:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2EE258D0002; Fri, 25 Feb 2022 03:50:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2769D8D0001; Fri, 25 Feb 2022 03:50:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 117E78D0002; Fri, 25 Feb 2022 03:50:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0058.hostedemail.com [216.40.44.58]) by kanga.kvack.org (Postfix) with ESMTP id F24B58D0001 for ; Fri, 25 Feb 2022 03:50:37 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 9DFE0181CCCAA for ; Fri, 25 Feb 2022 08:50:37 +0000 (UTC) X-FDA: 79180681314.17.87613E1 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2055.outbound.protection.outlook.com [40.107.93.55]) by imf12.hostedemail.com (Postfix) with ESMTP id 1D17E40003 for ; Fri, 25 Feb 2022 08:50:36 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CrIlI6LmgXIJlvEhBKBeleuNbhB8S+2Sz/uE5ik1xiyvDseFkwUIy5/7mvxN+OBPn/E1eRjyNbCOUfZZhX7FDiNVxKlensBMGhEIl5lChPnfrkNyYRe9BVDuu6YDwPRbhgCIvaO7wZQosJQiWF1gMIUdGSzwxPQh5cHg93ZkaSzqWwe/XS6/OsyXvS0FpgFtmjnYlwwzrjEazbhePXzrroEDaNJI0AvvvsYGzk6bSrUA+OuIC0RPrFoI2BGdWdigjop8b76LRsK3NGgRZ4tXImTbmySPBw3cMiAhipUgscJp2hJzLpcjKzXpyn6MHEx9BkfxULnAxMrG8EHdnbVRHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Oa+H/fbjV0fLvDgOl8iwViOqTlta9iVUjSg5NEzHQXY=; b=gFhhWgLj5/DUcALqZDypwFt9pQDxot2CSnch6BQqv+XCd5V/6xVUJb9kwnhfqz1i3KV47l9zqTLMmNFzeRnj8F8iAGEg7yb40BwKyxPj0fB4UUoB3vAPxW534svUN0Yp4YJAQ623UD9PPUBXuHnK9B1YSIWQJnsRKFlFbhazhQb/umyxVkQDk1Wn/20vUE4j89+q4Os+4U1omyAJNKVPHkffCaiAKPCyPmwTfOti2TYoYM0UXBsM7F+Ke4yv7mXVL7nDnSIoHCWQGK3RaK1MxhsSp7Hs7abxfUaQyc1Of1XDlXc/cxSbrSVRGzYSGBQIZSJ6pkP7yKushmM0hl+00A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Oa+H/fbjV0fLvDgOl8iwViOqTlta9iVUjSg5NEzHQXY=; b=KDdhhE7hRTl5ZbZNSFRYP2njHiKEf/FTiJARgU/gnqkMXHs1jBqZGQtez1l2XWdtqXpizg2CGyrLUC1yNRikuISmGil2yPQPLhWt3OjrIDlyDit39H+Mv7QqZ+rCfbpVZmC0I6WfZAfju9feToMLROLfl29XVqNGNN1SaXIRNR6TmFmFh7Xo3cszEe+mg1Mq5NaDdDAIIS5Oeq3APGhsIpW+ppVLVIs7x/g0awLwlmqwIJYAunvW8egBbyp+dOwUQfY6cq8trHl91reIQC8GBlRkKjnyJYwmQpgbZW2bpVl6PuhgNyKv0NwSl03kgIElrRoTsAJcL7/3cZPVyVpy0Q== Received: from BN9PR03CA0790.namprd03.prod.outlook.com (2603:10b6:408:13f::15) by BL1PR12MB5094.namprd12.prod.outlook.com (2603:10b6:208:312::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.21; Fri, 25 Feb 2022 08:50:35 +0000 Received: from BN8NAM11FT030.eop-nam11.prod.protection.outlook.com (2603:10b6:408:13f:cafe::ab) by BN9PR03CA0790.outlook.office365.com (2603:10b6:408:13f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.21 via Frontend Transport; Fri, 25 Feb 2022 08:50:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by BN8NAM11FT030.mail.protection.outlook.com (10.13.177.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:34 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 25 Feb 2022 08:50:34 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 25 Feb 2022 00:50:33 -0800 Received: from sandstorm.attlocal.net (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 25 Feb 2022 00:50:31 -0800 From: John Hubbard To: Jens Axboe , Jan Kara , Christoph Hellwig , Dave Chinner , "Darrick J . Wong" , Theodore Ts'o , Alexander Viro , Miklos Szeredi , Andrew Morton , Chaitanya Kulkarni CC: , , , , LKML , John Hubbard Subject: [RFC PATCH 1/7] mm/gup: introduce pin_user_page() Date: Fri, 25 Feb 2022 00:50:19 -0800 Message-ID: <20220225085025.3052894-2-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220225085025.3052894-1-jhubbard@nvidia.com> References: <20220225085025.3052894-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0c8acfe7-76ca-4054-9073-08d9f83be34a X-MS-TrafficTypeDiagnostic: BL1PR12MB5094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Sl3jY+YV8dP/1Cx+ZRwSk+sHTf/28FG9t8fcujr/Gi5Rj8Ql2gwFA2W4AoOxzdcSQ8R/Y/uORHA7tKMQ9iiw6b3zppi1MCjfMtiDHypAbRd5dgJKGNvFwEGDKvkhFyesquuhmzrzmmGEtSbT8QkS9gGN+d8TcvwJWaW66TFgXHwCWF6E2uxEkbpPdxI596Ym16Y4KWEBbm+JQ0Xm1pbsmjkNSWJo/ZXfecQUc8vltUQaY4uOYilxYA3H0UOddatEBqvOqwWOHz3nr2kAd8EbtqWT+lNQRhwbmZrAG5LoJWRPR0vVPvtpPC0m5f70kzQNg3GVFclYIgsNZ5/qXvtD6gTjVCCvqiGNIlYUH39hewP0Dg+517dUjefgImEKKVA4hnS9At1voy9uvQCNqsJG1sY8pxYNFQK6kJ+nkWy1wXd/Pxc0I9ys77EyakvRnkO30UAtIdkS17FA7SxtY7Xi3eo+7agB9GSUgzi2F/Gb6qOqYLQiG0qbFfJq2nDOjcknpPt/fK+4Wm8RpILGsG3/G/4dmwO713NgrcHfXxu37FGdeAWYxIrg72ONywwx6rxBYZzQftFK5PT8AxINPEt5O9iJ+tjg7i8t7RBd4XcrQVBfLi7jA4co1zUHMiOVu8/w6AXJyU/NrP3NTXkxLsBE6KilA60Jz3y66FGlqdFo24XacqaRtb3pllrN2TjIKxxULkh/cg4xmZOSLNzOK5r0FVT4hSBUL4+R0IxVy979IJk= X-Forefront-Antispam-Report: CIP:12.22.5.235;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(46966006)(40470700004)(36840700001)(36756003)(8936002)(7416002)(356005)(921005)(81166007)(5660300002)(508600001)(86362001)(40460700003)(8676002)(70206006)(70586007)(47076005)(107886003)(1076003)(6636002)(2616005)(316002)(336012)(426003)(186003)(26005)(2906002)(6666004)(83380400001)(36860700001)(4326008)(82310400004)(54906003)(110136005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 08:50:34.7990 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0c8acfe7-76ca-4054-9073-08d9f83be34a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.235];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT030.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5094 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=KDdhhE7h; dmarc=pass (policy=reject) header.from=nvidia.com; spf=none (imf12.hostedemail.com: domain of jhubbard@nvidia.com has no SPF policy when checking 40.107.93.55) smtp.mailfrom=jhubbard@nvidia.com X-Rspam-User: X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1D17E40003 X-Stat-Signature: kp56g3fc4zkzu5xnbjikubq4r4y47k6i X-HE-Tag: 1645779036-887139 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: pin_user_page() is an externally-usable version of try_grab_page(), but with semantics that match get_page(), so that it can act as a drop-in replacement for get_page(). Specifically, pin_user_page() has a void return type. pin_user_page() elevates a page's refcount is using FOLL_PIN rules. This means that the caller must release the page via unpin_user_page(). Signed-off-by: John Hubbard --- include/linux/mm.h | 1 + mm/gup.c | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 929488a47181..bb51f5487aef 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1914,6 +1914,7 @@ long pin_user_pages_remote(struct mm_struct *mm, long get_user_pages(unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas); +void pin_user_page(struct page *page); long pin_user_pages(unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas); diff --git a/mm/gup.c b/mm/gup.c index 5c3f6ede17eb..44446241c3a9 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -3034,6 +3034,40 @@ long pin_user_pages(unsigned long start, unsigned long nr_pages, } EXPORT_SYMBOL(pin_user_pages); +/** + * pin_user_page() - apply a FOLL_PIN reference to a page () + * + * @page: the page to be pinned. + * + * Similar to get_user_pages(), in that the page's refcount is elevated using + * FOLL_PIN rules. + * + * IMPORTANT: That means that the caller must release the page via + * unpin_user_page(). + * + */ +void pin_user_page(struct page *page) +{ + struct folio *folio = page_folio(page); + + WARN_ON_ONCE(folio_ref_count(folio) <= 0); + + /* + * Similar to try_grab_page(): be sure to *also* + * increment the normal page refcount field at least once, + * so that the page really is pinned. + */ + if (folio_test_large(folio)) { + folio_ref_add(folio, 1); + atomic_add(1, folio_pincount_ptr(folio)); + } else { + folio_ref_add(folio, GUP_PIN_COUNTING_BIAS); + } + + node_stat_mod_folio(folio, NR_FOLL_PIN_ACQUIRED, 1); +} +EXPORT_SYMBOL(pin_user_page); + /* * pin_user_pages_unlocked() is the FOLL_PIN variant of * get_user_pages_unlocked(). Behavior is the same, except that this one sets From patchwork Fri Feb 25 08:50:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12759909 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BF75C433EF for ; Fri, 25 Feb 2022 08:50:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F27A18D0005; Fri, 25 Feb 2022 03:50:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E86A58D0001; Fri, 25 Feb 2022 03:50:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C152A8D0005; Fri, 25 Feb 2022 03:50:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0050.hostedemail.com [216.40.44.50]) by kanga.kvack.org (Postfix) with ESMTP id A50FD8D0001 for ; Fri, 25 Feb 2022 03:50:40 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 608B9181CCCAA for ; Fri, 25 Feb 2022 08:50:40 +0000 (UTC) X-FDA: 79180681440.20.083DF6D Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2055.outbound.protection.outlook.com [40.107.243.55]) by imf04.hostedemail.com (Postfix) with ESMTP id B43414000D for ; Fri, 25 Feb 2022 08:50:39 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XBlO4zIFxmhx3wt8IOztZcQMmeyv2m7n6X9tW0E4NR1miYig6L+KLPPbBZngfC9Pv4Z2+aTRvnq9KQUgQuU5fEUL8fYIWDc0CVOLwewTPB8Dkx+UMWJ/mmyJF2VF80WD7IH9znDb94cncFqYFrdPjppKoG8iVfW5lgIbnGK9A7Yn9g1ZpTo7SO0iBtas5ygeaeUXKh72PbnPJAMJ7b+WVkB/q5s6lqfYopSa4aomOl69rgGI5dZD0nFKxSbSOvjihXrDcrWvlaa7VSE7RgtRKV6Ad8PBheFYsg2+AdvfWs3D7ZzAeeFfokYFK90VGJb7KylRjjjWPEgpHfAHj2R9XA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=h1M9kqFuSLuRzAkMql4Y2/4feCJMUalRUwt7co+e448=; b=K4zp4D+IPCAhVT5oGaYBOyX7giQHw34lN8XNBSnMy8Eb714urAYLYVo9CRlmGMuWCCd3kLi6R0i4bMVwIVKkU/fE2q4IivbBRjmy4y+Gpv1cLxEmYS0f0Zagr7bzPfi8Cze5O6b4CtfxSmqAhck+j6pWCsShtcM/cqBfMLYGWIAQ28J/4Oaa0xtRphMHamdWjEKGfT1fEhmss5PvoS6vy0HE8BXHkneuRMG7nbac6z8gcw/ve08B07QQMXrHmQpYWfFEyiA+n9mxsaR02LnwLNjVegXKHV/i6g9+90Cgh0a3d+f5RKcJLfodNZ3GFNIvc9cmiUD5f5MuBqS7LN9DLw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h1M9kqFuSLuRzAkMql4Y2/4feCJMUalRUwt7co+e448=; b=GJ2qMS3jPCG1uGGHF19x46M3X0lH+EZNhz6bcbzgRloaM64kotkSCCinVqQ6bk7Uat4AASlYRw32YIFwa098IYmI8YwfhqjGpDlGVn/yQ7u0uEiQIQeTlGILoCdprmhgfhxkc5md12KCfyCZtZW80TP3m77y7PiAo5Ewk/0kQShRB2vaRrpcDktOfHd5Ex83H98oSwV9wmTWQfeAnkmwAwD6XTzZsIMk9YZcS7RvCsSjBIsJgbh0fcid94ld1tCM7Z6AUtgrSkyxcgYwZYBl8No66F+yvMzQisigDfGcMaF+p2YWDG46pO7s6L7cP2slBUJJ0T71xK9Qww4cWjVbbg== Received: from DM5PR18CA0073.namprd18.prod.outlook.com (2603:10b6:3:3::11) by BN6PR12MB1188.namprd12.prod.outlook.com (2603:10b6:404:19::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.22; Fri, 25 Feb 2022 08:50:37 +0000 Received: from DM6NAM11FT066.eop-nam11.prod.protection.outlook.com (2603:10b6:3:3:cafe::73) by DM5PR18CA0073.outlook.office365.com (2603:10b6:3:3::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.24 via Frontend Transport; Fri, 25 Feb 2022 08:50:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.234) by DM6NAM11FT066.mail.protection.outlook.com (10.13.173.179) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:36 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 25 Feb 2022 08:50:36 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 25 Feb 2022 00:50:35 -0800 Received: from sandstorm.attlocal.net (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 25 Feb 2022 00:50:33 -0800 From: John Hubbard To: Jens Axboe , Jan Kara , Christoph Hellwig , Dave Chinner , "Darrick J . Wong" , Theodore Ts'o , Alexander Viro , Miklos Szeredi , Andrew Morton , Chaitanya Kulkarni CC: , , , , LKML , John Hubbard Subject: [RFC PATCH 2/7] block: add dio_w_*() wrappers for pin, unpin user pages Date: Fri, 25 Feb 2022 00:50:20 -0800 Message-ID: <20220225085025.3052894-3-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220225085025.3052894-1-jhubbard@nvidia.com> References: <20220225085025.3052894-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 92112809-dd4f-48be-2f22-08d9f83be46b X-MS-TrafficTypeDiagnostic: BN6PR12MB1188:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rUy++dt94z8m32z8jyFAhYaKVBy1XC4rHK5kuji65Rk3Uvn96GsULMmTxXNowEHCpOS1fMzoG3tu+DoQje0a5F8acSVVXe+fUG7ajZfp2ExQHazAdbAPjp0cnmIjLQF55EqL2C12V61EjQASNzP2RpvNpYJ+Cj04sHsE3WZmTVWsHO5TFdNy6hTEhF1/AaSDubme6f+ca/0qQYfpmh1fP9Pga8lufmVju+7hl5wJLCV/66aYw98yQ0n9l06hbf25pK+KWCDWJU8lAp/arfv33dPNwbjQAQT8VukekZqFtDhKmvrdWMTQxvAKUxaNWGv38go3PC5ECWRaiI3R7NN02FyF0kHHEnYCHnjfAURC733IwcW16uqmLldyyOG57qwxmDGEBDG83o6zTabwFjm6G7IqLFpCx7TQuWOqK5I9pKcm5tWSP2i83sW6zXZ6Qz6w+lG9yA5Tt/rDDi56DPnjU8M92zafBg+SUSdDGnmqcUtqe8jYN5qIVRhi7GMKsKk1F0wbQyZHjEWZeAOrYuWVxmWGHyLSk4k30BSQvFTVlWlbL2RSWP3WbjQlIzQfXiLTfQIPTKti3ll3b2edQWMLM5l21DkqUthtsX9fO52zL/F6oQummgUmvIk/3NXmhAU2j4TXaZOfBELf8VQux/EFBQsXXMWYAPozftvRtaaiu8Xub4bfmayQscZCK10JbiA0AMfZpFruh4iKGgVfZ0ojhkqJFQzoQwpKCGTyiWmNbEw= X-Forefront-Antispam-Report: CIP:12.22.5.234;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(5660300002)(336012)(26005)(426003)(107886003)(36756003)(2616005)(7416002)(2906002)(6666004)(508600001)(186003)(8936002)(47076005)(1076003)(83380400001)(110136005)(81166007)(356005)(36860700001)(4326008)(70206006)(70586007)(8676002)(86362001)(921005)(6636002)(82310400004)(54906003)(40460700003)(316002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 08:50:36.7530 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 92112809-dd4f-48be-2f22-08d9f83be46b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.234];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT066.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1188 X-Rspam-User: Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=GJ2qMS3j; spf=none (imf04.hostedemail.com: domain of jhubbard@nvidia.com has no SPF policy when checking 40.107.243.55) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: B43414000D X-Stat-Signature: x7is9fbxpceephx1fmoam9zeeaizocp1 X-HE-Tag: 1645779039-157638 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add a new config parameter, CONFIG_BLK_USE_PIN_USER_PAGES_FOR_DIO, and dio_w_*() wrapper functions. Together, these allow the developer to choose between these sets of routines, for Direct IO code paths: a) pin_user_pages_fast() pin_user_page() unpin_user_page() b) get_user_pages_fast() get_page() put_page() CONFIG_BLK_USE_PIN_USER_PAGES_FOR_DIO is a temporary setting, and will be deleted once the conversion is complete. In the meantime, developers can enable this in order to try out each filesystem. More information: The Direct IO part of the block infrastructure is being changed to use pin_user_page*() and unpin_user_page*() calls, in place of a mix of get_user_pages_fast(), get_page(), and put_page(). These have to be changed over all at the same time, for block, bio, and all filesystems. While that changeover is in progress (but disabled via this new CONFIG option), kernel developers need a way to test their changes. The steps are: a) Enable CONFIG_BLK_USE_PIN_USER_PAGES_FOR_DIO b) Monitor these /proc/vmstat items: nr_foll_pin_acquired nr_foll_pin_released ...to ensure that they remain equal, when "at rest". Signed-off-by: John Hubbard --- block/Kconfig | 25 +++++++++++++++++++++++++ include/linux/bvec.h | 11 +++++++++++ 2 files changed, 36 insertions(+) diff --git a/block/Kconfig b/block/Kconfig index 168b873eb666..f6ca5e9597e4 100644 --- a/block/Kconfig +++ b/block/Kconfig @@ -50,6 +50,31 @@ config BLK_DEV_BSG_COMMON config BLK_ICQ bool +config BLK_USE_PIN_USER_PAGES_FOR_DIO + bool "DEVELOPERS ONLY: Enable pin_user_pages() for Direct IO" if EXPERT + default n + help + For Direct IO code, retain the pages via calls to + pin_user_pages_fast(), instead of via get_user_pages_fast(). + Likewise, use pin_user_page() instead of get_page(). And then + release such pages via unpin_user_page(), instead of + put_page(). + + This is a temporary setting, which will be deleted once the + conversion is completed, reviewed, and tested. In the meantime, + developers can enable this in order to try out each filesystem. + For that, it's best to monitor these /proc/vmstat items: + + nr_foll_pin_acquired + nr_foll_pin_released + + ...to ensure that they remain equal, when "at rest". + + Say yes here ONLY if are actively developing or testing the + block layer or filesystems with pin_user_pages_fast(). + Otherwise, this is just a way to throw off the refcounting of + pages in the system. + config BLK_DEV_BSGLIB bool "Block layer SG support v4 helper lib" select BLK_DEV_BSG_COMMON diff --git a/include/linux/bvec.h b/include/linux/bvec.h index 35c25dff651a..a96a68c687f6 100644 --- a/include/linux/bvec.h +++ b/include/linux/bvec.h @@ -241,4 +241,15 @@ static inline void *bvec_virt(struct bio_vec *bvec) return page_address(bvec->bv_page) + bvec->bv_offset; } +#ifdef CONFIG_BLK_USE_PIN_USER_PAGES_FOR_DIO +#define dio_w_pin_user_pages_fast(s, n, p, f) pin_user_pages_fast(s, n, p, f) +#define dio_w_pin_user_page(p) pin_user_page(p) +#define dio_w_unpin_user_page(p) unpin_user_page(p) + +#else +#define dio_w_pin_user_pages_fast(s, n, p, f) get_user_pages_fast(s, n, p, f) +#define dio_w_pin_user_page(p) get_page(p) +#define dio_w_unpin_user_page(p) put_page(p) +#endif + #endif /* __LINUX_BVEC_H */ From patchwork Fri Feb 25 08:50:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12759911 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60DA3C433FE for ; Fri, 25 Feb 2022 08:50:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0AAA08D0001; Fri, 25 Feb 2022 03:50:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F20EC8D0008; Fri, 25 Feb 2022 03:50:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C88BD8D0006; Fri, 25 Feb 2022 03:50:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0078.hostedemail.com [216.40.44.78]) by kanga.kvack.org (Postfix) with ESMTP id 9CD528D0001 for ; Fri, 25 Feb 2022 03:50:44 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 47A78181CA087 for ; Fri, 25 Feb 2022 08:50:44 +0000 (UTC) X-FDA: 79180681608.27.7A0BED5 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2040.outbound.protection.outlook.com [40.107.93.40]) by imf05.hostedemail.com (Postfix) with ESMTP id 63816100002 for ; Fri, 25 Feb 2022 08:50:43 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WyACxF7fPkE2rMezm401crnu/DYgiYzX4NkfVZmgP+AqV4eae1UuvF2outqZRNrbfiQJzvBMofrQox3Sxt71gFCW5xToCaiN7RWQiGGPKL8My8pPYh/fU3iWZI7p4IP9pcxTPq0T/ggKWk0cGkuw7cxnKqlbIGXiafL/tkLALhYcXFALXLnoX5hohzYZj7qUdQ5x+i5s2T98fbvP+H9cuQfSZKdwX2svGsAH52T4lXVaC0k/g+WNaMUrHZAaENc+PhDVfvkRUN+p/H5WMnhWBM2+mMUpK4jRCdpozW/XZhcSA5wn5KsqiwVRBItiPt90I64JjlsE91rwGBbfm70qaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KCh3aDMkQYStRKBxVavUNPnaTNvm4hCnzAHey6l38Dw=; b=Lwkg2W3UPyscDCUYO3ipEP8AfT0b3i56zw2EEsrHTNrMm2azaItP3Rewz0hja+0AZ1+3FfeSsYEjsz09R17yKQj6lz9UdkDQbNGxm0WY61w6yuFwROMXxDrkd9VRzkY2D9rVNTXYqrsZES71crrTVAZp4dnObefkVK/SZy84X33mhYnho6M73B2WFtfl0TcHvyEFmw6GCaZIEiUPkQhtBQb73+lHb0vznZvjSm4UTg5SJcn1o6SiY5iuYMEON6XPT9krjPttEFgwYVPq8rtnuNOSLZcK8X2KTFNsTDFNUwqUZuXGLnnEoLXflmgzWZtxGnrXQmgTpz+SVbt6aeN2Rg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KCh3aDMkQYStRKBxVavUNPnaTNvm4hCnzAHey6l38Dw=; b=PY8TDTmGFToOfTDgVT1eFl9ditLQvPXUH6v0T+e6nKFDm0QYc2dEcPq2V7YPtr0n1ENen2cg++ij+2r11pxvhpoMr8k+tDL5GdQCfsym15aFAshHlrcKgv1+VpRQ03xjbD7h0qqkNJodqOaYdIEkocgAl9id9alvAhP/uySE0w/oh5UlRAvR8sY0YirWSe9Rim6GYw6FCt128/Lu+kyIllSp2mIDxpBk1GLFSpERnzdPSPxBXG9lIuR6TXGyAb9XjhWvHMlP6tNEQQVOzadS7aQPs6vlzwvCzk/DLwEaqgiG4W0Xh9+sGo6TpVu9Q5PxyYKXJOrOwLZVm1m2+V4XYw== Received: from MWHPR20CA0029.namprd20.prod.outlook.com (2603:10b6:300:ed::15) by SA0PR12MB4432.namprd12.prod.outlook.com (2603:10b6:806:98::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.21; Fri, 25 Feb 2022 08:50:40 +0000 Received: from CO1NAM11FT037.eop-nam11.prod.protection.outlook.com (2603:10b6:300:ed:cafe::26) by MWHPR20CA0029.outlook.office365.com (2603:10b6:300:ed::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.19 via Frontend Transport; Fri, 25 Feb 2022 08:50:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT037.mail.protection.outlook.com (10.13.174.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:39 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 25 Feb 2022 08:50:38 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 25 Feb 2022 00:50:37 -0800 Received: from sandstorm.attlocal.net (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 25 Feb 2022 00:50:35 -0800 From: John Hubbard To: Jens Axboe , Jan Kara , Christoph Hellwig , Dave Chinner , "Darrick J . Wong" , Theodore Ts'o , Alexander Viro , Miklos Szeredi , Andrew Morton , Chaitanya Kulkarni CC: , , , , LKML , John Hubbard Subject: [RFC PATCH 3/7] block, fs: assert that key paths use iovecs, and nothing else Date: Fri, 25 Feb 2022 00:50:21 -0800 Message-ID: <20220225085025.3052894-4-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220225085025.3052894-1-jhubbard@nvidia.com> References: <20220225085025.3052894-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 892e3132-73d9-41d6-34a5-08d9f83be61e X-MS-TrafficTypeDiagnostic: SA0PR12MB4432:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: El6t/1kSUiKOsLJMZfaTKBxsmM6FJwsDQiueoSdM8DfLi8Y3qpYgeopIjwbwE+LaikxDletUrU6wQlyJ7ss6d3kr80CdIYlFBiokgVjvCzgGHk3jGURchWTkTzT/0ekN6IRUHS+OgHG50KoY9JyYQCBQCNIZ+0y7fzN/sAz81cwZtFuO/GdmcCM12Vho8VgksEZKQLiDs+bqt6wopBqJ6vWIeju2UyJpHoikw5Wd1ICWNwKm13iQwShzGk4vRtfSrGgXOYXx/JAOUdUP8coyXnd5JF04x0N49gp1vyB42eaauFN5Qn+EeuYQJ4KhP91wx6dmcNaK+uV3mZR4cwKBvb48RAEMzulCvYcvklMp9J/a4lqC5ulaEwNzGd82NIaCz5g9Qi2aGR7eFPpnUujwt+RuQavzx7xlpniniVQQMBjAUkF72wLsI23Z5/guMZg09yxHo4DMWhquDTRoX/PniZCKevgrNlZeKaSjnqdNLeQ6Oux4MvJOWtVx4XjC6dktJEVqTtg/nfXyu0J97bg4L+WVcpKWDVLG46O41uhmAksIfN+dflvQgCvdfmlpBGDjQKkrJ5wlMPwPMd9UXmijetYC2+zBiapjctwBYmxu3nNWptQc/4pjlXePIOao1CYpwlbeLA7rPk1af3XbyJ9804KcHx9HdvxtiPpq4lHPAcgqmaKSf9QOii3/In+H2coHs4m2OnWR066AQNEsIyw+0ci2eDA4ilT4kAIi+wLLm2Q= X-Forefront-Antispam-Report: CIP:12.22.5.236;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(36756003)(6666004)(8936002)(107886003)(2616005)(356005)(40460700003)(7416002)(508600001)(336012)(426003)(26005)(2906002)(36860700001)(83380400001)(82310400004)(4326008)(8676002)(81166007)(110136005)(5660300002)(921005)(1076003)(86362001)(47076005)(54906003)(316002)(186003)(6636002)(70206006)(70586007)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 08:50:39.6040 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 892e3132-73d9-41d6-34a5-08d9f83be61e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.236];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT037.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4432 X-Rspamd-Queue-Id: 63816100002 X-Stat-Signature: ew3ko8gcuuuum6ychypdum1qrmn7ydqo Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=PY8TDTmG; dmarc=pass (policy=reject) header.from=nvidia.com; spf=none (imf05.hostedemail.com: domain of jhubbard@nvidia.com has no SPF policy when checking 40.107.93.40) smtp.mailfrom=jhubbard@nvidia.com X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1645779043-88322 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Upcoming changes to Direct IO will change it from acquiring pages via get_user_pages_fast(), to calling pin_user_pages_fast() instead. Place a few assertions at key points, that the pages are IOVEC (user pages), to enforce the assumptions that there are no kernel or pipe or other odd variations being passed. Signed-off-by: John Hubbard --- block/bio.c | 4 ++++ fs/direct-io.c | 2 ++ 2 files changed, 6 insertions(+) diff --git a/block/bio.c b/block/bio.c index b15f5466ce08..4679d6539e2d 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1167,6 +1167,8 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) BUILD_BUG_ON(PAGE_PTRS_PER_BVEC < 2); pages += entries_left * (PAGE_PTRS_PER_BVEC - 1); + WARN_ON_ONCE(!iter_is_iovec(iter)); + size = iov_iter_get_pages(iter, pages, LONG_MAX, nr_pages, &offset); if (unlikely(size <= 0)) return size ? size : -EFAULT; @@ -1217,6 +1219,8 @@ static int __bio_iov_append_get_pages(struct bio *bio, struct iov_iter *iter) BUILD_BUG_ON(PAGE_PTRS_PER_BVEC < 2); pages += entries_left * (PAGE_PTRS_PER_BVEC - 1); + WARN_ON_ONCE(!iter_is_iovec(iter)); + size = iov_iter_get_pages(iter, pages, LONG_MAX, nr_pages, &offset); if (unlikely(size <= 0)) return size ? size : -EFAULT; diff --git a/fs/direct-io.c b/fs/direct-io.c index 38bca4980a1c..7dbbbfef300d 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -169,6 +169,8 @@ static inline int dio_refill_pages(struct dio *dio, struct dio_submit *sdio) { ssize_t ret; + WARN_ON_ONCE(!iter_is_iovec(sdio->iter)); + ret = iov_iter_get_pages(sdio->iter, dio->pages, LONG_MAX, DIO_PAGES, &sdio->from); From patchwork Fri Feb 25 08:50:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12759910 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54BCFC433EF for ; Fri, 25 Feb 2022 08:50:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E11D48D0007; Fri, 25 Feb 2022 03:50:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D74AD8D0001; Fri, 25 Feb 2022 03:50:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC7058D0007; Fri, 25 Feb 2022 03:50:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id A36B88D0006 for ; Fri, 25 Feb 2022 03:50:44 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 71EAB23E4F for ; Fri, 25 Feb 2022 08:50:44 +0000 (UTC) X-FDA: 79180681608.10.5BA1775 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2049.outbound.protection.outlook.com [40.107.93.49]) by imf08.hostedemail.com (Postfix) with ESMTP id A120216000F for ; Fri, 25 Feb 2022 08:50:43 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WEcuxonxnpTUmRleoW6WM6DYYUTvp6aRGwo8+06mNJt5fp3R2W8edtdCLha1z9a3tooNu+1L69N1WLBqX4rzNC4Ko8T6ehrDWFDtBWalbjqQMYAvPacLicQLmeRnjs6gZDTo7QBV3CgFXi3pKnDcQABTs+arND0aOP+Mxy8WP7dqpZUqQl/UIfgBK4r9SwtWwQC2EldZWdOVGX+dL6+ctABelOV70rlLJcd56nSvSj3ameF2rAsz4IuGMQpruCCVsyb8dED5Uq+p94QCIifBn4USuPviK6lFcl7UL4gcx+6NgykwWbUGit4E7p+/d11cMz7KoRH6TaYoe5I6kwzkrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DpGZ4D4kOXbjY11gED52E3qdRR7ord22Md9fFluP4Sk=; b=J+tpkgUCmc3tS1ZEmjNWxGNhhAjbEBP3YIy+pY6iSuKQbtHM8xTATv5HB8FJRY8r8dCPvC42TG+PtJDm10qUoQ/JKfIKRUy61SgDuYtt7yLhZnQQoEl2L94ebDW0R+gRJd//jmRjp3tK9u2v7mbFqm/UXtiJKdXDV8/P1VX77vh1CES2sxdQqNsYNN016pQuO1W6gfliz/us5hRcp23DTDB4RFY5krihgn5IaQI018YxcVA90EQWMNbNR79bEjdhFBOfzwP0KHP7XKWqC/gHhAhZXydQdIy4Gg/vBPFiLfiGj7GlcGnzmtt20hW/FFVGXF1eqCpplBeYkkie6AIRYg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DpGZ4D4kOXbjY11gED52E3qdRR7ord22Md9fFluP4Sk=; b=DjTK0haWTYIr4n/145v8RajRmD3wYMIRMrd1znySIjlBD5ITNSvUOF59i2WIesFFDChCQMpCjrj9U8nuPnS1uuao9J2Sm8cmZlC4YTw5KWXDhZMjsX6zTV3U8P6GphcsPkiDdMEYwLCYy2ie0fU7WFiGev/gFbHwwu9gzmfto7/4NxS3Cov7Ay9Ri0JtwpB1ewne1Fd8tP6qyJYDJ8fmbSGecJyWqX/U1GdvFUWzosD38KVqQDhbZvbcDJHnI+FV4eOz/+Na+2hhpFvUnvDSUzfBQVTd9/SCXgaNJNsq/QAtTYxcsPOw3fx5LSYGd86pkmthF8lB+yvV3+M3DKNvkA== Received: from MWHPR1601CA0023.namprd16.prod.outlook.com (2603:10b6:300:da::33) by CO6PR12MB5457.namprd12.prod.outlook.com (2603:10b6:5:355::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.21; Fri, 25 Feb 2022 08:50:41 +0000 Received: from CO1NAM11FT018.eop-nam11.prod.protection.outlook.com (2603:10b6:300:da:cafe::7) by MWHPR1601CA0023.outlook.office365.com (2603:10b6:300:da::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.11 via Frontend Transport; Fri, 25 Feb 2022 08:50:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT018.mail.protection.outlook.com (10.13.175.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:40 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 25 Feb 2022 08:50:40 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 25 Feb 2022 00:50:39 -0800 Received: from sandstorm.attlocal.net (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 25 Feb 2022 00:50:37 -0800 From: John Hubbard To: Jens Axboe , Jan Kara , Christoph Hellwig , Dave Chinner , "Darrick J . Wong" , Theodore Ts'o , Alexander Viro , Miklos Szeredi , Andrew Morton , Chaitanya Kulkarni CC: , , , , LKML , John Hubbard Subject: [RFC PATCH 4/7] block, bio, fs: initial pin_user_pages_fast() changes Date: Fri, 25 Feb 2022 00:50:22 -0800 Message-ID: <20220225085025.3052894-5-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220225085025.3052894-1-jhubbard@nvidia.com> References: <20220225085025.3052894-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: da1004e2-f5ad-4439-b72a-08d9f83be6ef X-MS-TrafficTypeDiagnostic: CO6PR12MB5457:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: N4jpCwG3jip3+F70tpFrej0AYMVbTi05Y0CCyl4iIDdU6RHU2YkQ2PQg8wpVhcjKwI+ScOUWteHJ9f7CSoYle9B5V5DDj6BUV3WdtQ/6UT/aX/X6hevW3vSyjjEf53L34WpDFTQjKTHeBCt5TrApT2X2U7XM4xU5QKdk2kpQp0DVzEEhBEPE+rOpjTfgfGon5jw+BTbUvHD7GG19Z0hnoeboX7PuL7ahOfbqE1REwDyxWIA3lPtp6qnKtWxKi+OND4RTmwoJ3UIvDrJBr3l2u60FickaYm9Cr5p+UOnmE0Dd4iJS+SSL756VGlny8CnAQRiy8YLA2LRRBI46nW1oR/xbLna4//XfSiKAqMtpdEbJQC2Rg6v4o9M8imNCThR2KSwhYxsEqOTHqMitAS5DQKx1MeKY4TrPt0TQ2JM0XqV0WajwzZIYCdbURtJKJHFahlqSW6LOlFExP+AxSVHC8qgwQGwVCuvsdIAQNV5nlOWTB2N5a6IivN8Z4Z/IPICthmba7+BYHIU0yNuWJaAjhfa061WPbX9laqNzjfyvpskrg6I0Q9nt6HzlP+IZGgVrsxGGWv5UsLyBZpnA5wi5TfywqhfzGynxZaDjHRnvLe5/s7wNtHT+dp40wCewwlNsLptVy825VyNWxFhY31+O6LdwErxAv3Rd7XRrmv+MgSvSKXdbSlTNA2cZtrH9gVi4neA6BeGVqk6xUfmv8c3Mdm+BYeBozbCwaUKl/kC7/3s= X-Forefront-Antispam-Report: CIP:12.22.5.236;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(6636002)(2616005)(316002)(1076003)(426003)(336012)(186003)(26005)(2906002)(6666004)(107886003)(82310400004)(110136005)(54906003)(83380400001)(36860700001)(4326008)(81166007)(5660300002)(356005)(921005)(36756003)(8936002)(7416002)(40460700003)(8676002)(86362001)(70206006)(70586007)(47076005)(508600001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 08:50:40.9458 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: da1004e2-f5ad-4439-b72a-08d9f83be6ef X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.236];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT018.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5457 X-Stat-Signature: tcupwgkchxeeq5rtcd1mm1x16zry3s66 X-Rspam-User: Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=DjTK0haW; spf=none (imf08.hostedemail.com: domain of jhubbard@nvidia.com has no SPF policy when checking 40.107.93.49) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: A120216000F X-HE-Tag: 1645779043-227738 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use dio_w_*() wrapper calls, in place of get_user_pages_fast(), get_page() and put_page(). This sets up core parts of the block, bio, and direct-io subsystems to use dio_*() wrapper calls. Signed-off-by: John Hubbard --- block/bio.c | 18 +++++++++--------- block/blk-map.c | 4 ++-- fs/direct-io.c | 24 ++++++++++++------------ fs/iomap/direct-io.c | 2 +- lib/iov_iter.c | 4 ++-- 5 files changed, 26 insertions(+), 26 deletions(-) diff --git a/block/bio.c b/block/bio.c index 4679d6539e2d..8541245c53bd 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1102,7 +1102,7 @@ void __bio_release_pages(struct bio *bio, bool mark_dirty) bio_for_each_segment_all(bvec, bio, iter_all) { if (mark_dirty && !PageCompound(bvec->bv_page)) set_page_dirty_lock(bvec->bv_page); - put_page(bvec->bv_page); + dio_w_unpin_user_page(bvec->bv_page); } } EXPORT_SYMBOL_GPL(__bio_release_pages); @@ -1133,7 +1133,7 @@ static void bio_put_pages(struct page **pages, size_t size, size_t off) size_t i, nr = DIV_ROUND_UP(size + (off & ~PAGE_MASK), PAGE_SIZE); for (i = 0; i < nr; i++) - put_page(pages[i]); + dio_w_unpin_user_page(pages[i]); } #define PAGE_PTRS_PER_BVEC (sizeof(struct bio_vec) / sizeof(struct page *)) @@ -1144,9 +1144,9 @@ static void bio_put_pages(struct page **pages, size_t size, size_t off) * @iter: iov iterator describing the region to be mapped * * Pins pages from *iter and appends them to @bio's bvec array. The - * pages will have to be released using put_page() when done. - * For multi-segment *iter, this function only adds pages from the - * next non-empty segment of the iov iterator. + * pages will have to be released using unpin_user_page() when done. For + * multi-segment *iter, this function only adds pages from the next non-empty + * segment of the iov iterator. */ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) { @@ -1180,7 +1180,7 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) if (__bio_try_merge_page(bio, page, len, offset, &same_page)) { if (same_page) - put_page(page); + dio_w_unpin_user_page(page); } else { if (WARN_ON_ONCE(bio_full(bio, len))) { bio_put_pages(pages + i, left, offset); @@ -1237,7 +1237,7 @@ static int __bio_iov_append_get_pages(struct bio *bio, struct iov_iter *iter) break; } if (same_page) - put_page(page); + dio_w_unpin_user_page(page); offset = 0; } @@ -1434,8 +1434,8 @@ void bio_set_pages_dirty(struct bio *bio) * the BIO and re-dirty the pages in process context. * * It is expected that bio_check_pages_dirty() will wholly own the BIO from - * here on. It will run one put_page() against each page and will run one - * bio_put() against the BIO. + * here on. It will run one unpin_user_page() against each page and will run + * one bio_put() against the BIO. */ static void bio_dirty_fn(struct work_struct *work); diff --git a/block/blk-map.c b/block/blk-map.c index c7f71d83eff1..5f0d04c5dd9d 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -275,7 +275,7 @@ static int bio_map_user_iov(struct request *rq, struct iov_iter *iter, if (!bio_add_hw_page(rq->q, bio, page, n, offs, max_sectors, &same_page)) { if (same_page) - put_page(page); + dio_w_unpin_user_page(page); break; } @@ -289,7 +289,7 @@ static int bio_map_user_iov(struct request *rq, struct iov_iter *iter, * release the pages we didn't map into the bio, if any */ while (j < npages) - put_page(pages[j++]); + dio_w_unpin_user_page(pages[j++]); kvfree(pages); /* couldn't stuff something into bio? */ if (bytes) diff --git a/fs/direct-io.c b/fs/direct-io.c index 7dbbbfef300d..0d7104aa40c3 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -183,7 +183,7 @@ static inline int dio_refill_pages(struct dio *dio, struct dio_submit *sdio) */ if (dio->page_errors == 0) dio->page_errors = ret; - get_page(page); + dio_w_pin_user_page(page); dio->pages[0] = page; sdio->head = 0; sdio->tail = 1; @@ -452,7 +452,7 @@ static inline void dio_bio_submit(struct dio *dio, struct dio_submit *sdio) static inline void dio_cleanup(struct dio *dio, struct dio_submit *sdio) { while (sdio->head < sdio->tail) - put_page(dio->pages[sdio->head++]); + dio_w_unpin_user_page(dio->pages[sdio->head++]); } /* @@ -717,7 +717,7 @@ static inline int dio_bio_add_page(struct dio_submit *sdio) */ if ((sdio->cur_page_len + sdio->cur_page_offset) == PAGE_SIZE) sdio->pages_in_io--; - get_page(sdio->cur_page); + dio_w_pin_user_page(sdio->cur_page); sdio->final_block_in_bio = sdio->cur_page_block + (sdio->cur_page_len >> sdio->blkbits); ret = 0; @@ -832,13 +832,13 @@ submit_page_section(struct dio *dio, struct dio_submit *sdio, struct page *page, */ if (sdio->cur_page) { ret = dio_send_cur_page(dio, sdio, map_bh); - put_page(sdio->cur_page); + dio_w_unpin_user_page(sdio->cur_page); sdio->cur_page = NULL; if (ret) return ret; } - get_page(page); /* It is in dio */ + dio_w_pin_user_page(page); /* It is in dio */ sdio->cur_page = page; sdio->cur_page_offset = offset; sdio->cur_page_len = len; @@ -853,7 +853,7 @@ submit_page_section(struct dio *dio, struct dio_submit *sdio, struct page *page, ret = dio_send_cur_page(dio, sdio, map_bh); if (sdio->bio) dio_bio_submit(dio, sdio); - put_page(sdio->cur_page); + dio_w_unpin_user_page(sdio->cur_page); sdio->cur_page = NULL; } return ret; @@ -953,7 +953,7 @@ static int do_direct_IO(struct dio *dio, struct dio_submit *sdio, ret = get_more_blocks(dio, sdio, map_bh); if (ret) { - put_page(page); + dio_w_unpin_user_page(page); goto out; } if (!buffer_mapped(map_bh)) @@ -998,7 +998,7 @@ static int do_direct_IO(struct dio *dio, struct dio_submit *sdio, /* AKPM: eargh, -ENOTBLK is a hack */ if (dio->op == REQ_OP_WRITE) { - put_page(page); + dio_w_unpin_user_page(page); return -ENOTBLK; } @@ -1011,7 +1011,7 @@ static int do_direct_IO(struct dio *dio, struct dio_submit *sdio, if (sdio->block_in_file >= i_size_aligned >> blkbits) { /* We hit eof */ - put_page(page); + dio_w_unpin_user_page(page); goto out; } zero_user(page, from, 1 << blkbits); @@ -1051,7 +1051,7 @@ static int do_direct_IO(struct dio *dio, struct dio_submit *sdio, sdio->next_block_for_io, map_bh); if (ret) { - put_page(page); + dio_w_unpin_user_page(page); goto out; } sdio->next_block_for_io += this_chunk_blocks; @@ -1067,7 +1067,7 @@ static int do_direct_IO(struct dio *dio, struct dio_submit *sdio, } /* Drop the ref which was taken in get_user_pages() */ - put_page(page); + dio_w_unpin_user_page(page); } out: return ret; @@ -1289,7 +1289,7 @@ do_blockdev_direct_IO(struct kiocb *iocb, struct inode *inode, ret2 = dio_send_cur_page(dio, &sdio, &map_bh); if (retval == 0) retval = ret2; - put_page(sdio.cur_page); + dio_w_unpin_user_page(sdio.cur_page); sdio.cur_page = NULL; } if (sdio.bio) diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index 67cf9c16f80c..4e818648a1aa 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -192,7 +192,7 @@ static void iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio, bio->bi_private = dio; bio->bi_end_io = iomap_dio_bio_end_io; - get_page(page); + dio_w_pin_user_page(page); __bio_add_page(bio, page, len, 0); iomap_dio_submit_bio(iter, dio, bio, pos); } diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 6dd5330f7a99..f15a3ef5a481 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -1538,7 +1538,7 @@ ssize_t iov_iter_get_pages(struct iov_iter *i, addr = first_iovec_segment(i, &len, start, maxsize, maxpages); n = DIV_ROUND_UP(len, PAGE_SIZE); - res = get_user_pages_fast(addr, n, gup_flags, pages); + res = dio_w_pin_user_pages_fast(addr, n, gup_flags, pages); if (unlikely(res <= 0)) return res; return (res == n ? len : res * PAGE_SIZE) - *start; @@ -1667,7 +1667,7 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i, p = get_pages_array(n); if (!p) return -ENOMEM; - res = get_user_pages_fast(addr, n, gup_flags, p); + res = dio_w_pin_user_pages_fast(addr, n, gup_flags, p); if (unlikely(res <= 0)) { kvfree(p); *pages = NULL; From patchwork Fri Feb 25 08:50:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12759912 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5671C43217 for ; Fri, 25 Feb 2022 08:50:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3DA2F8D0008; Fri, 25 Feb 2022 03:50:47 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3615E8D0006; Fri, 25 Feb 2022 03:50:47 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 18D698D0008; Fri, 25 Feb 2022 03:50:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0095.hostedemail.com [216.40.44.95]) by kanga.kvack.org (Postfix) with ESMTP id E978B8D0006 for ; Fri, 25 Feb 2022 03:50:46 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 99DA68249980 for ; Fri, 25 Feb 2022 08:50:46 +0000 (UTC) X-FDA: 79180681692.23.EF26373 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2044.outbound.protection.outlook.com [40.107.236.44]) by imf03.hostedemail.com (Postfix) with ESMTP id ED54C2000C for ; Fri, 25 Feb 2022 08:50:45 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GScl9BenBELXDXb+6CD6bKA7k6rOQ5epj//MZcZFU31oW6psUcZn9InZyemWqPiF5Srt2s3esktruIz24UqRggsxZjKN2rSwAnfNDx6C8fETCtKKGzJzNWZNE+WCZYhkyKmbvvmUt8fAMSbuC3G+nykKMPxwOkJOhY+T34/IRNHpBBtuwb+jON21edEFCm6/TTGpHpNjYbgv1tdsJnJnWvJqO3taxNNFll39bZCYJwCJxSIQ+ZyllkZQMzir5Qhca6WzxDVLpIrC8IN1FK7UqCtBUwnpKjSqH5SG58LPl/wgQleqJ18fM7WAbwuKgk3g+CU0ryvGN9ST1ib0+8q1vg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=P4QhKeT9zgdstYjOxfhw3Jd7XZ11uxPnfOza55tfrhs=; b=Dt8lBYaS7lwi78FUqI5h9nXg1i8Yfz8WTUVqtw/ISzTdT8UC05k67vlN+jU0p1HLqx0JTpvIZ5OdPF/G4VkHHkAIYK0N2fyP1ioUdQkWhb9Vry30L6pKR55Jd77EBzeW5aiz/RHEwQ1KAwRNh1IvJArhuxl6VN7aM0Bmd3Npnh0KR/fnkvcyi2TAAjxsxy8R9V0MhAuLLmsJDHBhHGF0DKyN57MU4icP9EFKWZSkuDUxp27Djxjhb4cTAWNSlsqzdyaufhIqA3KO0fpPclq63BRy1nf0A910p95Teebyjkm1c61GRQMbI2kouwG3LspQWAJ3cy+4GS8MsmYch2VUyQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=P4QhKeT9zgdstYjOxfhw3Jd7XZ11uxPnfOza55tfrhs=; b=myiqbymWylkCR/Nhklgo4NfPGEHFsvdGyFN3gJUgLjRC0YEFGqkkr5+ooobHeNqOKgiy8JAV0VXYjhuXouT4YkVvNtAqDv9SSRJTvLn/F/x5GkWTkB8gCqSSGIMYHciOHUPprDSm1ZhPpaFmZWC99OHmVCaBRyVDVG9VbZZ3DyeDaytNbkyYbe2HRqWqAAFaLnuDGDfFUgkw6KTeVqmm+j0gj7PEVI8uFHBg0AlS6m7jvYgBlQVnpNoA5wD10P3Jr9anHNuWuzoXHyiV4C2EqlLlTAZkKGJQ0hSQqabZTiwtyO8hkAwi6vwj0gbK/aqWRQc8KHTZFARil2SACZA4EA== Received: from DM5PR12CA0049.namprd12.prod.outlook.com (2603:10b6:3:103::11) by MWHPR1201MB0256.namprd12.prod.outlook.com (2603:10b6:301:53::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.24; Fri, 25 Feb 2022 08:50:43 +0000 Received: from DM6NAM11FT019.eop-nam11.prod.protection.outlook.com (2603:10b6:3:103:cafe::bd) by DM5PR12CA0049.outlook.office365.com (2603:10b6:3:103::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.23 via Frontend Transport; Fri, 25 Feb 2022 08:50:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.238) by DM6NAM11FT019.mail.protection.outlook.com (10.13.172.172) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:43 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 25 Feb 2022 08:50:42 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 25 Feb 2022 00:50:41 -0800 Received: from sandstorm.attlocal.net (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 25 Feb 2022 00:50:40 -0800 From: John Hubbard To: Jens Axboe , Jan Kara , Christoph Hellwig , Dave Chinner , "Darrick J . Wong" , Theodore Ts'o , Alexander Viro , Miklos Szeredi , Andrew Morton , Chaitanya Kulkarni CC: , , , , LKML , John Hubbard Subject: [RFC PATCH 5/7] NFS: direct-io: convert to FOLL_PIN pages Date: Fri, 25 Feb 2022 00:50:23 -0800 Message-ID: <20220225085025.3052894-6-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220225085025.3052894-1-jhubbard@nvidia.com> References: <20220225085025.3052894-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b7e90141-910f-495c-939a-08d9f83be833 X-MS-TrafficTypeDiagnostic: MWHPR1201MB0256:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zTqxQpF83G0gwsqmu6gw7wRmodhDJ3kf0EdhJvT31Oz2DxULS3d6Br9SjbF7i8VFPgCb55E165cuUegGM136z1uPz73oXXbz+/iXhweQY5tlk2uJgzp7X2Vbek80Kl7NRox5jlvsHPOQOyFeMDk50Sid0mxnjq/G5ICFkSuCPoouJ1rYZXDNG7n5gzz+Zh9FsilOAVuR+MBMV4Bg6gXJobUjm3iJSQY/7HL/6i1EuQybwTrt6LswCqxznOXm+7OfArf51w6GHNPplxzlGLmH35T2kuMKSiWUjBFnd6G8PHEj+jWln1Src6cHoRq2RmJ0Ulq0mzLSovqGmtAFTHUXQapD7FKJSd5gKVGtqgZy7tgVDkvGeTqp3vdF1Gp5xe9q/h7gcAY7dgq9Qox7oOnRRvEGZlWohK7+X6FnFtW/GSwoT14CHNIqknceVcoW4Yk3L9vj5A8KuAw/cjdax2S+V3qImP0egjQ6gtGRUlqKO+8on1vLykgmmhPqkbTQnyDIJj2/xjBdhb2yDP0uKxF1HgVwn1bDK54XIZScRJeJbxYMs/R3BB+SiaQpcVgrrXUuNJ6znRl0zxJCkQixsFy1rxiNyGiLZvN42OJmx0VJPEW4X0qDubB4qId5ce9L4rR9MP0f52/e+CUwFAlhQr35Jo29fSCaYeZhtS05IXCl6kCayGLltjjRxz5AntCZ8XEfuFhrGnp+jTMVj17B08giZ72xxc3h8Cf6ZcdJPSyanU8= X-Forefront-Antispam-Report: CIP:12.22.5.238;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(46966006)(36840700001)(40470700004)(4326008)(6636002)(2906002)(508600001)(316002)(47076005)(186003)(336012)(36860700001)(26005)(110136005)(83380400001)(54906003)(36756003)(426003)(107886003)(2616005)(921005)(82310400004)(7416002)(356005)(40460700003)(86362001)(5660300002)(6666004)(8936002)(70206006)(8676002)(81166007)(1076003)(70586007)(4744005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 08:50:43.0682 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b7e90141-910f-495c-939a-08d9f83be833 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.238];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT019.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB0256 X-Rspamd-Queue-Id: ED54C2000C X-Stat-Signature: prg5yaxcid3ccmfn1aqdthe7ty4q4gzn Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=myiqbymW; dmarc=pass (policy=reject) header.from=nvidia.com; spf=none (imf03.hostedemail.com: domain of jhubbard@nvidia.com has no SPF policy when checking 40.107.236.44) smtp.mailfrom=jhubbard@nvidia.com X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1645779045-918944 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now that Direct IO's core allocators invoke pin_user_pages_fast(), those pages must free released via unpin_user_page(), instead of put_page(). Signed-off-by: John Hubbard --- fs/nfs/direct.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index eabfdab543c8..2e0d399c5a5a 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -181,7 +181,7 @@ static void nfs_direct_release_pages(struct page **pages, unsigned int npages) { unsigned int i; for (i = 0; i < npages; i++) - put_page(pages[i]); + dio_w_unpin_user_page(pages[i]); } void nfs_init_cinfo_from_dreq(struct nfs_commit_info *cinfo, From patchwork Fri Feb 25 08:50:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12759913 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E818C433F5 for ; Fri, 25 Feb 2022 08:50:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A67C98D0009; Fri, 25 Feb 2022 03:50:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9F1428D0006; Fri, 25 Feb 2022 03:50:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81F8A8D0009; Fri, 25 Feb 2022 03:50:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0090.hostedemail.com [216.40.44.90]) by kanga.kvack.org (Postfix) with ESMTP id 6ACC88D0006 for ; Fri, 25 Feb 2022 03:50:48 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 162A79D65B for ; Fri, 25 Feb 2022 08:50:48 +0000 (UTC) X-FDA: 79180681776.21.E25DC4C Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam07on2086.outbound.protection.outlook.com [40.107.212.86]) by imf29.hostedemail.com (Postfix) with ESMTP id 73E4D120006 for ; Fri, 25 Feb 2022 08:50:47 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=edu6bmnk/q3Pz7CSfnwZvK4X5kz9+qsV9hpYeyh3oaS879yoBxKmQPr+GfMU18JdK6yDWRPf9VB64KpEdm7RewrNheaqivHWU0/d2zCQmLzDcOpc3oUzQ6Pja8gBvykUrPW3pGhwpNOeZYLm477yWOD0kbOTpqfMTT6ip8mmGnkJp3UQRf/00y1qV+Ox/+Kpvsq7+7W/6dJe+Y1/pJ5VOWVsIZY6HaBJx+bhvJ6kEIh5wDuaMMFPW7C3Z6bQV9/g4qeD1Ts0Ll0KkgghR8C/ciQAS7MfjuKMYdLfvcFmtQzDakuejd3dlxYnbP4pDJMdbEEbaiKQHm8+XYCvMiuVCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tWclyEKq4CVkl7hdRiKCw5b/8BrSgghcSXNAqeSOe0M=; b=AwZ9tg/Hxrv5DsLNSqALjbpd0qtYfnjvkN31rDe70RCCLByUC1XWV20VZUUk1JufA+nHqwt9ErWI11brGMwwikSfqAnLpsjgrb2AzTl0WGpMJawkO+xYA7rl9QCU04AMQirVK4HLywEc4yozTdzZ83btEvYXWF9tB3zo9pDFyWmmovDjUqX4ryOElF5HCSGrQeUClvClnXuJubziaGBBMtQ3gKf3aIBu675gGCOcdZtYW6kvd63XakYKziLQd9Xrc1DzazwyjjeX0sC//nN1d4WroYqJVZ3tMhgNIL1Jphf9c1HVgfVram9Zkylkon9tSuIkbSqcottU0A9Yzy7QRw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tWclyEKq4CVkl7hdRiKCw5b/8BrSgghcSXNAqeSOe0M=; b=j+X9aDhPNh5/uMJybg0OIztzh0JteNtmcMpwgRWtoxPx9ekv8c0UgOuUZpT3SYpBmSrLtchssDwceUFhwDBtnegaUDe1GDE8eKMPdCIZILSbvbH8WapNj15QbNO4PTa1dHomFJFzodhYVrjRJJCT6079jM89DBIp31KMNYa/NXbOBRDN6Io0xU6UrCd1JlTuM3BEqA2Pij3UMva3blpq5Yk00TSG/VxcSDEKzAMakUURFVcX1LmVA55UgKjyKytsSI08KgfzzU5k64AjCOUM4x0v83c2nEyKliBhID6HqCSGGhwxmMIe0ks1KrD5Fu7NMoj14WLvV18kkfyXugEU/g== Received: from BN0PR04CA0139.namprd04.prod.outlook.com (2603:10b6:408:ed::24) by CH0PR12MB5074.namprd12.prod.outlook.com (2603:10b6:610:e1::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.21; Fri, 25 Feb 2022 08:50:45 +0000 Received: from BN8NAM11FT007.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ed:cafe::66) by BN0PR04CA0139.outlook.office365.com (2603:10b6:408:ed::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by BN8NAM11FT007.mail.protection.outlook.com (10.13.177.109) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:45 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 25 Feb 2022 08:50:44 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 25 Feb 2022 00:50:43 -0800 Received: from sandstorm.attlocal.net (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 25 Feb 2022 00:50:41 -0800 From: John Hubbard To: Jens Axboe , Jan Kara , Christoph Hellwig , Dave Chinner , "Darrick J . Wong" , Theodore Ts'o , Alexander Viro , Miklos Szeredi , Andrew Morton , Chaitanya Kulkarni CC: , , , , LKML , John Hubbard Subject: [RFC PATCH 6/7] fuse: convert direct IO paths to use FOLL_PIN Date: Fri, 25 Feb 2022 00:50:24 -0800 Message-ID: <20220225085025.3052894-7-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220225085025.3052894-1-jhubbard@nvidia.com> References: <20220225085025.3052894-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e14dc0c9-f000-4920-5c71-08d9f83be98e X-MS-TrafficTypeDiagnostic: CH0PR12MB5074:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: utN9OnhBYfjLaaMUa/ek04txJYqDYRj0suaZTme+Nk7I0fQNDmfMFgZxyUeymK5aCy6c/ophxUtBZYeMkFbULHxkvLrHY/Vxd1wHjhBdVrLJ/sPSWNZw44asH9+UEC2qL6ZOEvJk5UHVGXSZYd2P+idVZNRlkADx3fHfbwtMn519F6FGoWWUsIq0mmSFOloMMoywXNLCQF8j44k+kD+OmU04+jV5oL41tWdySwBvGjaefGKHbr5f1rKQ6OzjtEyQyQK09PY6kyh+gW/15Q76w9loX+xRMoewEj8TA5skT/4oWuNw1FjfHk9zdGzQwMV34CvER6shuj6Sst34qDSnArudSJvY5VvT5wIu81t/1WW51a0/p92lDsaiF7sti7sptewJayv8HHq1jI/B/dlNaEUHLvqvS5daeF6ITGGCtyYMK5KUrSaVFjQu94w7QQS2bINVpudxnTiuMVI5+hac3JfKs+gRvoL/DPimXjRgoJUdHDxy6hgvWvCBMLVe8Ec+OaWQOt4/aWBLcc/yQ2uAxkT0ZrahV1qHivooso1o2LNq0zrQd6r84pGiMEjp7g4VLzclFwy5NIWlvA24zNZLNyybHHxdS5hnBi1tMS2r1XDXbF8kaKeF0BEObdFmO8+UM795qD8Q/VuCdT+lkFUbms5eVLMmADj+VqdEHy752tdoJdmU6kj7+xFRCXdtL+TWobaWAGW+WBsRK/4crIlFi9StDq9uEndIxuJGL1Q6UpdY6WXrX7cKEEULivHZqdH5dWxCz3KaSenB2V/kaVrun+9AyvGxmEujIFY+YPPI8xgBZthMUSYKBIS/orJ6Sndd X-Forefront-Antispam-Report: CIP:12.22.5.235;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(70206006)(2906002)(40460700003)(86362001)(82310400004)(70586007)(426003)(7416002)(8936002)(107886003)(6666004)(186003)(2616005)(1076003)(26005)(83380400001)(508600001)(336012)(54906003)(966005)(921005)(356005)(316002)(6636002)(81166007)(110136005)(36860700001)(47076005)(8676002)(36756003)(4326008)(5660300002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 08:50:45.3097 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e14dc0c9-f000-4920-5c71-08d9f83be98e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.235];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT007.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB5074 Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=j+X9aDhP; dmarc=pass (policy=reject) header.from=nvidia.com; spf=none (imf29.hostedemail.com: domain of jhubbard@nvidia.com has no SPF policy when checking 40.107.212.86) smtp.mailfrom=jhubbard@nvidia.com X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 73E4D120006 X-Stat-Signature: hqtps3wgii5ys7564kmrbjuta5wbkeyk X-HE-Tag: 1645779047-912249 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert the fuse filesystem to support the new iov_iter_get_pages() behavior. That routine now invokes pin_user_pages_fast(), which means that such pages must be released via unpin_user_page(), rather than via put_page(). This commit also removes any possibility of kernel pages being handled, in the fuse_get_user_pages() call. Although this may seem like a steep price to pay, Christoph Hellwig actually recommended it a few years ago for nearly the same situation [1]. [1] https://lore.kernel.org/kvm/20190724061750.GA19397@infradead.org/ Signed-off-by: John Hubbard --- fs/fuse/dev.c | 5 ++++- fs/fuse/file.c | 23 ++++++++--------------- 2 files changed, 12 insertions(+), 16 deletions(-) diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c index e1b4a846c90d..a93037c96b89 100644 --- a/fs/fuse/dev.c +++ b/fs/fuse/dev.c @@ -675,7 +675,10 @@ static void fuse_copy_finish(struct fuse_copy_state *cs) flush_dcache_page(cs->pg); set_page_dirty_lock(cs->pg); } - put_page(cs->pg); + if (cs->pipebufs) + put_page(cs->pg); + else + unpin_user_page(cs->pg); } cs->pg = NULL; } diff --git a/fs/fuse/file.c b/fs/fuse/file.c index 94747bac3489..395c2fb613fb 100644 --- a/fs/fuse/file.c +++ b/fs/fuse/file.c @@ -619,7 +619,7 @@ static void fuse_release_user_pages(struct fuse_args_pages *ap, for (i = 0; i < ap->num_pages; i++) { if (should_dirty) set_page_dirty_lock(ap->pages[i]); - put_page(ap->pages[i]); + unpin_user_page(ap->pages[i]); } } @@ -1382,20 +1382,13 @@ static int fuse_get_user_pages(struct fuse_args_pages *ap, struct iov_iter *ii, size_t nbytes = 0; /* # bytes already packed in req */ ssize_t ret = 0; - /* Special case for kernel I/O: can copy directly into the buffer */ - if (iov_iter_is_kvec(ii)) { - unsigned long user_addr = fuse_get_user_addr(ii); - size_t frag_size = fuse_get_frag_size(ii, *nbytesp); - - if (write) - ap->args.in_args[1].value = (void *) user_addr; - else - ap->args.out_args[0].value = (void *) user_addr; - - iov_iter_advance(ii, frag_size); - *nbytesp = frag_size; - return 0; - } + /* + * TODO: this warning can eventually be removed. It is just to help + * developers, during a transitional period, while direct IO code is + * moving over to FOLL_PIN pages. + */ + if (WARN_ON_ONCE(!iter_is_iovec(ii))) + return -EOPNOTSUPP; while (nbytes < *nbytesp && ap->num_pages < max_pages) { unsigned npages; From patchwork Fri Feb 25 08:50:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12759914 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08090C433FE for ; Fri, 25 Feb 2022 08:50:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D01F08D000A; Fri, 25 Feb 2022 03:50:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C63F28D0006; Fri, 25 Feb 2022 03:50:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB45E8D000A; Fri, 25 Feb 2022 03:50:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.25]) by kanga.kvack.org (Postfix) with ESMTP id 94D5E8D0006 for ; Fri, 25 Feb 2022 03:50:51 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 6E0E2219C7 for ; Fri, 25 Feb 2022 08:50:51 +0000 (UTC) X-FDA: 79180681902.02.FCCB3DA Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2080.outbound.protection.outlook.com [40.107.94.80]) by imf14.hostedemail.com (Postfix) with ESMTP id 98A1410000F for ; Fri, 25 Feb 2022 08:50:50 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EzODPZatyn9W0Fjj7PNYZPtgjou+HfkkWOMOk915RdLejpI39ZqHGopKxsWvfWFf7q6veCcuj8Apyg+4UQonViQSe/m4G7A+jImdlNIW/9moN4gGGT7EIe+yvI1Py37VD9nG6tt9wgi+ri3S0GBykg+Hrk0y4U/pn59YW+++Te/KKHAd3QQ6KuByYfbiu9lU0/oFWeQNlyVpm7zWxkFh+HPLUyHw/3oDOEX+SR5RBJ2yGBpCOzPeT/D2rEe7ws2NicDbrzXzKYOg+x6J8ub90dFMrYqtdCDWgKFsSsdllC6Djp4D9v3Rkwq8ob3BW7Hm0V6dfmrBy2aj1j00vbYkbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mVjachwLZOJaasM1HxAYAlh+fxfrxwqQnXama3fFC5A=; b=ZvvE3gAjRZD6lkDf6bp/VvilesY74JFtFJVK8lP4sJf5cHb1ISF/3bgejS8VwMNP2QeQa39wUl/ZK9maOvC+e3Psgpxk7T6I/6T5Bd5Fw6PxyypwH56xxoN1KKIB4j/e1L/2VDSlSgSYnPJXWAbr0r2NiaYjVT5aklWwUt+1C4wpylkPofOLcFwCHgpNmW+D6Z4j60vrWC17zvplbX62/lLIzkmRgEVuPgHYUNO2bg3Yf2orHzWKEhyvTszx5ouJv8uTSrKctIzgyryhH4RYorsrFMRz212az+eaG1zEsC5gpY8bZ7jlIhcOhnVhhk6w1+u03jlRJHwvlQrqZ4E3yQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mVjachwLZOJaasM1HxAYAlh+fxfrxwqQnXama3fFC5A=; b=Zve+2qknEQaTEYfdwjqqoB7LB8CZjjmFxrhuc3oz3egNKcdqFqHa7Wixckcz4FMkeKnTuihuK9Ow/bhYkFCWtp1csu+piBVAdlXPzIloOZmXuf5TptO/xUtIyJPj2yN9k/7j9DTrcyAM1ZoNgH0RQc4DErdFhbUN5lP/KknK1lPgVZZa2EZWDf6X0FAn9Qyl7ztQ2V3NvnpVWbaqN0JW92GKUXndT7PaMDhMDLk0Dq9+1rMjgf2qSQ+ODj5lBRpLnplNVEQFinAYoYfGCQXtDyWYzYh2QI0r2nIs+iKuAWNUHRz8zpbQM9IgDtD7XkTon4zN5ym55dQe6+6AjYqktg== Received: from BN9PR03CA0806.namprd03.prod.outlook.com (2603:10b6:408:13f::31) by BN6PR1201MB0180.namprd12.prod.outlook.com (2603:10b6:405:56::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.22; Fri, 25 Feb 2022 08:50:48 +0000 Received: from BN8NAM11FT054.eop-nam11.prod.protection.outlook.com (2603:10b6:408:13f:cafe::43) by BN9PR03CA0806.outlook.office365.com (2603:10b6:408:13f::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.24 via Frontend Transport; Fri, 25 Feb 2022 08:50:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by BN8NAM11FT054.mail.protection.outlook.com (10.13.177.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Fri, 25 Feb 2022 08:50:47 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 25 Feb 2022 08:50:46 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 25 Feb 2022 00:50:45 -0800 Received: from sandstorm.attlocal.net (10.127.8.10) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 25 Feb 2022 00:50:44 -0800 From: John Hubbard To: Jens Axboe , Jan Kara , Christoph Hellwig , Dave Chinner , "Darrick J . Wong" , Theodore Ts'o , Alexander Viro , Miklos Szeredi , Andrew Morton , Chaitanya Kulkarni CC: , , , , LKML , John Hubbard Subject: [RFC PATCH 7/7] block, direct-io: flip the switch: use pin_user_pages_fast() Date: Fri, 25 Feb 2022 00:50:25 -0800 Message-ID: <20220225085025.3052894-8-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220225085025.3052894-1-jhubbard@nvidia.com> References: <20220225085025.3052894-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6480806c-cca4-4a7c-20ec-08d9f83beb12 X-MS-TrafficTypeDiagnostic: BN6PR1201MB0180:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Lv1rCmizZyWkMNp8GC3k5HyeJ/MMauJ+khLsMeeEYr3My1wa3QL0fdCQFQ23HHqE7WbShOdxgO3/+QGrSOZ+YdeRCRKuDx18PQpVt6QwRqvlYR8TIA82pe0sqJXXgoW49SMM190IM8YZlMTa0ue/j4PXBr7v5D6StzTIjIPHxf9u+PibqrvSkOOZm+QwKwdd/DKQs9mYPLoR4ao5fgz2treiO9ZYnCSEz7f4GokGtu17y0OGMtLAjba2JEncRfcxtwcr2txPuiCSK/XlDUs7C7e+Fa3BmiWwbZ2ZMwtdYFxFAUcRCCxUJglPlbbz+pj4YFnwibpAR1lktacjqpr/X+LPzHxqWteir+V4HAbHYufxqPMvBxyw9lBTvlqxOKQlMZI6e9NiWUkvEFhK1ibu0fwVEnEBPdoFxiWjRRZ0SF+clBeWOSgLhiXI5bbBM29IJTCuzm8BwDov1MO4oxicQa1Nd64HxuNMjGOtzwjZcuRxzzGyvZJGdqvY+oazJoVeizRu8HPSxXd/xqgjyKJgK2a3EvDw9c3YzTOPmAh5+9oXTGQSHpUS4194VBy24VALZO/CeGHuVD4S5uilFwZLjwH+xmuD0fZjIXXx7vSKgAxwA51n4KkinxTtqpxKbwelRdtbgOe+XvUkrm9AibGNBsvW/OFUGCg33qb9VIcyw8+iizU15/sdzHLefCKDp0RB/sSxiU4XngatORUXOErV89OB1FVyj7MhCmMiiPUs2RM= X-Forefront-Antispam-Report: CIP:12.22.5.235;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(83380400001)(82310400004)(336012)(6666004)(40460700003)(110136005)(6636002)(316002)(54906003)(36860700001)(2616005)(1076003)(426003)(47076005)(186003)(81166007)(7416002)(2906002)(36756003)(508600001)(70586007)(356005)(921005)(70206006)(8676002)(86362001)(4326008)(107886003)(26005)(8936002)(5660300002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 08:50:47.8353 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6480806c-cca4-4a7c-20ec-08d9f83beb12 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.235];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT054.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1201MB0180 X-Rspam-User: Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=Zve+2qkn; spf=none (imf14.hostedemail.com: domain of jhubbard@nvidia.com has no SPF policy when checking 40.107.94.80) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 98A1410000F X-Stat-Signature: nja4g4zf1u6a96tw9es136qtywfe8i9u X-HE-Tag: 1645779050-373543 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Remove CONFIG_BLK_USE_PIN_USER_PAGES_FOR_DIO, but leave the dio_w_*() wrapper functions in place, with the pin_user_pages_fast() defines as the only choice. A subsequent patch is now possible, to rename all dio_w_*() functions so as to remove the dio_w_ prefix. Signed-off-by: John Hubbard --- block/Kconfig | 25 ------------------------- include/linux/bvec.h | 7 ------- 2 files changed, 32 deletions(-) diff --git a/block/Kconfig b/block/Kconfig index f6ca5e9597e4..168b873eb666 100644 --- a/block/Kconfig +++ b/block/Kconfig @@ -50,31 +50,6 @@ config BLK_DEV_BSG_COMMON config BLK_ICQ bool -config BLK_USE_PIN_USER_PAGES_FOR_DIO - bool "DEVELOPERS ONLY: Enable pin_user_pages() for Direct IO" if EXPERT - default n - help - For Direct IO code, retain the pages via calls to - pin_user_pages_fast(), instead of via get_user_pages_fast(). - Likewise, use pin_user_page() instead of get_page(). And then - release such pages via unpin_user_page(), instead of - put_page(). - - This is a temporary setting, which will be deleted once the - conversion is completed, reviewed, and tested. In the meantime, - developers can enable this in order to try out each filesystem. - For that, it's best to monitor these /proc/vmstat items: - - nr_foll_pin_acquired - nr_foll_pin_released - - ...to ensure that they remain equal, when "at rest". - - Say yes here ONLY if are actively developing or testing the - block layer or filesystems with pin_user_pages_fast(). - Otherwise, this is just a way to throw off the refcounting of - pages in the system. - config BLK_DEV_BSGLIB bool "Block layer SG support v4 helper lib" select BLK_DEV_BSG_COMMON diff --git a/include/linux/bvec.h b/include/linux/bvec.h index a96a68c687f6..5bc98b334efe 100644 --- a/include/linux/bvec.h +++ b/include/linux/bvec.h @@ -241,15 +241,8 @@ static inline void *bvec_virt(struct bio_vec *bvec) return page_address(bvec->bv_page) + bvec->bv_offset; } -#ifdef CONFIG_BLK_USE_PIN_USER_PAGES_FOR_DIO #define dio_w_pin_user_pages_fast(s, n, p, f) pin_user_pages_fast(s, n, p, f) #define dio_w_pin_user_page(p) pin_user_page(p) #define dio_w_unpin_user_page(p) unpin_user_page(p) -#else -#define dio_w_pin_user_pages_fast(s, n, p, f) get_user_pages_fast(s, n, p, f) -#define dio_w_pin_user_page(p) get_page(p) -#define dio_w_unpin_user_page(p) put_page(p) -#endif - #endif /* __LINUX_BVEC_H */