From patchwork Fri Aug 13 04:41:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12434697 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E717EC4338F for ; Fri, 13 Aug 2021 04:41:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CC0AD61038 for ; Fri, 13 Aug 2021 04:41:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238702AbhHMEmJ (ORCPT ); Fri, 13 Aug 2021 00:42:09 -0400 Received: from mail-co1nam11on2058.outbound.protection.outlook.com ([40.107.220.58]:48353 "EHLO NAM11-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S238685AbhHMEmG (ORCPT ); Fri, 13 Aug 2021 00:42:06 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WXs8cPnlGKrT+HEtO+Q/zx0WdpxNjd8Qp4buEesJwNqh+jzWlgs27EpSIIkjGXCH6aklCRJQmT+kPn/duSEPudLuN9NGif7B/w9jYcZC33LsXuOAEyuZ8rTfCyWwXTciksdc+TT4GqLv1QlYmU3N+jqXa5FMYJA0o227TWOvaM/9R4tSS/jAKdLqMFgoo8seUwrJhzGVyjGlOFhxXNiqUjpDI7aqfFMIBBMtmt1XNEcUDtDEqkvXtiBCtOHS+gaNjvLqaadvBKua64Vtx3TSbcKkmQCUuX0gbMlPtnFyEuWNo87FXikvhbnQDGuvfZ7R6CpGh7sf/1BHfjNfDLbF/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T93KNdHJhW3WHWQJizJ9zl19NjWkpWWPLX17IPvFQpQ=; b=bzHmYo2h0KDNnK364aTJoDybAHsV7Wvc9L/i5aaxJhws3oNvY2UK2ovKJRrJHmsyRrLUkG9yxBxeEEhLm5YayAcBlrRGKhs+QVSmgjZKkKrp7DOnuWLX1C3x9efhvI+y5AhIXM4SGRvUeIPfBtp8ndMOQ+xRwjSjpylo4wuV5tboZbpXxlqFTFiR1YK17wDmaHLU/jgUttvpjhYnUP7FHk5IpnydYHCcRvKPjGJ8K5rRmiZ9PM3K6dype8/Z+gnq8rZTg/8Do27Gh2iWIGdzvJIOr/HZfMeXl+5FS7z4Ss7i8qGNO/tT9oh+bZy61Qa4gZyzPo7tT85HbsfN7llGdA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.36) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T93KNdHJhW3WHWQJizJ9zl19NjWkpWWPLX17IPvFQpQ=; b=VGMC4j9g6UHVFnhJVOQnVO1nLjfXNHNKA16+Ru9LpBeM5S295as8yKvU5FjrgWvOIJLYBG19PZ5ozxAIMpe2TjaofriYs9PDKs0DQIjcCskCZYQ0edkKguOzdYCxF1TJR3cKXpsp3974mzPdNjU1vapKHvwVHpGtTDmAxxURAEe7mt44ajCWrfxPM2O3PocVvIerMea2Hj7olJVI4p2wmMV6Dc1M1XszfhFwwSybWHk1MhYWrkwjd/a7VxnF8G3B4QjUHM89GNf7LOCdL6BNp1y8GYvZKaT/yJJJvuS4NkuaZafKgtf6Ig1zc+R1hU/dc+xVmFbPMYd7GODGVj6BjA== Received: from DS7PR03CA0049.namprd03.prod.outlook.com (2603:10b6:5:3b5::24) by DM5PR12MB1916.namprd12.prod.outlook.com (2603:10b6:3:112::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4394.21; Fri, 13 Aug 2021 04:41:36 +0000 Received: from DM6NAM11FT023.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3b5:cafe::58) by DS7PR03CA0049.outlook.office365.com (2603:10b6:5:3b5::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.15 via Frontend Transport; Fri, 13 Aug 2021 04:41:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.36) smtp.mailfrom=nvidia.com; vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.36 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.36; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.36) by DM6NAM11FT023.mail.protection.outlook.com (10.13.173.96) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4415.14 via Frontend Transport; Fri, 13 Aug 2021 04:41:35 +0000 Received: from HQMAIL107.nvidia.com (172.20.187.13) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 13 Aug 2021 04:41:35 +0000 Received: from sandstorm.attlocal.net (172.20.187.5) by mail.nvidia.com (172.20.187.13) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 13 Aug 2021 04:41:35 +0000 From: John Hubbard To: Andrew Morton CC: Matthew Wilcox , Christoph Hellwig , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , LKML , , , , John Hubbard Subject: [PATCH v3 1/3] mm/gup: documentation corrections for gup/pup Date: Thu, 12 Aug 2021 21:41:31 -0700 Message-ID: <20210813044133.1536842-2-jhubbard@nvidia.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210813044133.1536842-1-jhubbard@nvidia.com> References: <20210813044133.1536842-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 312a56dd-c3af-40d2-4183-08d95e14a1eb X-MS-TrafficTypeDiagnostic: DM5PR12MB1916: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: eHxYscZEFmS5PbL4bQxMivixKzdqYxRNzkq9bmBdKs3TZDA3rf4Ezi2ypg7dSuWmSY6EP8RbyMy1pVz50gB4CrfURg3DJ8prqNfIUInA+UvNthMXCDMOrzTfz6QxQEu8W0WjlAstBLXqWb0b8Z2R/Mjfcu8WrWH3qdAMbb542NE5Xt558mCarzK8+clW0rCPNfaLzrdStcdHVUvH9ThNHU1qjLAowOUzmydHyOPREevnaH8Btd96W3rCSW9y8WdAC4L3qiKPV48OcadUjB4IRJuUbt11gIb3ATqj85FEw1CM/uNEc8IoSSj1krEZmCb1DzzBrocdYCXRDPr/QIULHDDkR6Dd43+ETTzKhfBNHWNhUhqRuTh1wNcSFtjnMyKdZ/Q05YgjYJRAMES9nl12fz0UxxB2B00vCKMjDj9eAcyjUSeUjAAmbaFhF76m5IPNTd2NCOBa/bX23e7IDRIyt7cUY3LdDbp0bxiDapgxuxp0uw3u/X0QMnwyy5gmhUA4NL9T9KBLDpMkcFx1YzixVoPPUASGWTvCKhKfg9Mqd8hKZKR24I/FEhqtRk9IjR9vfcvLqSEZtHwXY7Kk6x7KloiIXv9/sBzYPwQj5QZjOAJ0ywtxwWNBLPTLZRIsivHsovw/01TCMARKXjXHHR2BXYKFgXg2arMUQMecKcbKaKUmqujcfJBEBcZQ6yrHHrRn4ewvijsaIEIjYCWJJ+4uOg== X-Forefront-Antispam-Report: CIP:216.228.112.36;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid05.nvidia.com;CAT:NONE;SFS:(4636009)(39860400002)(136003)(396003)(376002)(346002)(46966006)(36840700001)(36906005)(426003)(316002)(5660300002)(54906003)(6666004)(2616005)(478600001)(336012)(82310400003)(36860700001)(86362001)(356005)(47076005)(83380400001)(36756003)(186003)(4326008)(70206006)(107886003)(2906002)(26005)(70586007)(8936002)(8676002)(6916009)(7416002)(82740400003)(1076003)(7636003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2021 04:41:35.7482 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 312a56dd-c3af-40d2-4183-08d95e14a1eb X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.36];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT023.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1916 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The documentation for try_grab_compound_head() and try_grab_page() has fallen a little out of date. Update and clarify a few points. Also make it kerneldoc-correct, by adding @args documentation. Cc: Matthew Wilcox Cc: Christoph Hellwig Signed-off-by: John Hubbard --- mm/gup.c | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 2630ed1bb4f4..52f08e3177e9 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -92,10 +92,17 @@ static inline struct page *try_get_compound_head(struct page *page, int refs) return head; } -/* +/** * try_grab_compound_head() - attempt to elevate a page's refcount, by a * flags-dependent amount. * + * Even though the name includes "compound_head", this function is still + * appropriate for callers that have a non-compound @page to get. + * + * @page: pointer to page to be grabbed + * @refs: the value to (effectively) add to the page's refcount + * @flags: gup flags: these are the FOLL_* flag values. + * * "grab" names in this file mean, "look at flags to decide whether to use * FOLL_PIN or FOLL_GET behavior, when incrementing the page's refcount. * @@ -103,8 +110,14 @@ static inline struct page *try_get_compound_head(struct page *page, int refs) * same time. (That's true throughout the get_user_pages*() and * pin_user_pages*() APIs.) Cases: * - * FOLL_GET: page's refcount will be incremented by 1. - * FOLL_PIN: page's refcount will be incremented by GUP_PIN_COUNTING_BIAS. + * FOLL_GET: page's refcount will be incremented by @refs. + * + * FOLL_PIN on compound pages that are > two pages long: page's refcount will + * be incremented by @refs, and page[2].hpage_pinned_refcount will be + * incremented by @refs * GUP_PIN_COUNTING_BIAS. + * + * FOLL_PIN on normal pages, or compound pages that are two pages long: + * page's refcount will be incremented by @refs * GUP_PIN_COUNTING_BIAS. * * Return: head page (with refcount appropriately incremented) for success, or * NULL upon failure. If neither FOLL_GET nor FOLL_PIN was set, that's @@ -141,6 +154,8 @@ __maybe_unused struct page *try_grab_compound_head(struct page *page, * * However, be sure to *also* increment the normal page refcount * field at least once, so that the page really is pinned. + * That's why the refcount from the earlier + * try_get_compound_head() is left intact. */ if (hpage_pincount_available(page)) hpage_pincount_add(page, refs); @@ -184,10 +199,8 @@ static void put_compound_head(struct page *page, int refs, unsigned int flags) * @flags: gup flags: these are the FOLL_* flag values. * * Either FOLL_PIN or FOLL_GET (or neither) may be set, but not both at the same - * time. Cases: - * - * FOLL_GET: page's refcount will be incremented by 1. - * FOLL_PIN: page's refcount will be incremented by GUP_PIN_COUNTING_BIAS. + * time. Cases: please see the try_grab_compound_head() documentation, with + * "refs=1". * * Return: true for success, or if no action was required (if neither FOLL_PIN * nor FOLL_GET was set, nothing is done). False for failure: FOLL_GET or From patchwork Fri Aug 13 04:41:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12434693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A253C4338F for ; Fri, 13 Aug 2021 04:41:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1F758610CC for ; Fri, 13 Aug 2021 04:41:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238682AbhHMEmF (ORCPT ); Fri, 13 Aug 2021 00:42:05 -0400 Received: from mail-mw2nam08on2089.outbound.protection.outlook.com ([40.107.101.89]:64736 "EHLO NAM04-MW2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S238660AbhHMEmE (ORCPT ); Fri, 13 Aug 2021 00:42:04 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UA2jfN5J0PnmvaLM9C8g0MkGGNg53K0xp9JUbRuQYnwdMkS+7fWSeu/uH63z/YCZGHoLgo6LTO0/yf/T+XfN0Az5ZJ5qY+gY+HgSGB7PSqbVRnyjgyugQ2r/FIGK2RtYP1EV0+qYp/24FcrMPvzoG5NQotfZUwO7vR4OCffxa/CWZO06rO5g2I3yH0I2aa+KwUYg5idMcUUGYbEPwnibFCn7fTIziEHkbG00u6Co803qLAvRrs/i1Qx0Za1xuUe+l0go0DHhmomNoO2mvqkvdzGegNkCC8EGazLvOiUJ0t0a0XHcLNt+Mij5Gm//9wlT9kyOdpGPBAQ8JkjUP8spvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5xLf1m1g7cLMUBXRJJepv9cv1cwJzRrbSXYcSu7o0ZE=; b=cQIWQdAsYPOVZ0nt0EOT74Pt5YUdzpCOZdIK/nkgdiKa1nS9ab9h19/g6vohk90wv6EJkv3ypRQbfnHs+aUPRZ+I4shBv3APHshoXV3Xg9l/70tqtu7yogYNZXn7a2Zd3G8WIILdLE8nl7ogjmYk4iRf53H+m2e/f4Q0mSy5S+ApDvHMlb4O2UFq6d8O/+CbdlhINz4093GBgRs87gYN9GzW5uo/ObSIwIxFLrT7UYUXQjiQuOVMjOqoBxcVI/4vZxsWzJSHctgxN+qQm3Nd12I8LT6rP5L6eg4QX9wKNOsSO9yqh1b3UKkfBU0/HSsXdXZGVx8+8GhGY8/EQ+y5uA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=linux-foundation.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5xLf1m1g7cLMUBXRJJepv9cv1cwJzRrbSXYcSu7o0ZE=; b=CeM3JZJXMy9rkBREESRfsD1esPiH6UvCgMn4NSouu/pRZAB07utnYaUwRFrfBTbhQ99t8R2OQRFfocQACA8IfJaHVKO/6TqIMNI5U8PoeSuD8icqADjf0pcc2fMd/EajxfFxQbpkga2dBId0E9LMD8uiBOnFGZL8IHwHtkZLCUijOGxrA10cBTvvH6ajuDX4pYZNTZhyNRM5bUh8HMAqgceey8P1bpLjx0r6IwGBDwIu8ULl8YMj7dauXg4uZKk9WfAkoUrB+LwXWxOMrKbNXSzR9/ySzXrzY4GnAiZj0cl/RZ1lc0/6KDG7GXWyvlnoCgvysPQxSynKWcAJokMSBg== Received: from MWHPR18CA0036.namprd18.prod.outlook.com (2603:10b6:320:31::22) by MN2PR12MB4063.namprd12.prod.outlook.com (2603:10b6:208:1dc::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.14; Fri, 13 Aug 2021 04:41:37 +0000 Received: from CO1NAM11FT044.eop-nam11.prod.protection.outlook.com (2603:10b6:320:31:cafe::4c) by MWHPR18CA0036.outlook.office365.com (2603:10b6:320:31::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.14 via Frontend Transport; Fri, 13 Aug 2021 04:41:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; linux-foundation.org; dkim=none (message not signed) header.d=none;linux-foundation.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT044.mail.protection.outlook.com (10.13.175.188) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4415.16 via Frontend Transport; Fri, 13 Aug 2021 04:41:36 +0000 Received: from HQMAIL105.nvidia.com (172.20.187.12) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 13 Aug 2021 04:41:35 +0000 Received: from HQMAIL107.nvidia.com (172.20.187.13) by HQMAIL105.nvidia.com (172.20.187.12) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 13 Aug 2021 04:41:35 +0000 Received: from sandstorm.attlocal.net (172.20.187.5) by mail.nvidia.com (172.20.187.13) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 13 Aug 2021 04:41:35 +0000 From: John Hubbard To: Andrew Morton CC: Matthew Wilcox , Christoph Hellwig , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , LKML , , , , John Hubbard Subject: [PATCH v3 2/3] mm/gup: small refactoring: simplify try_grab_page() Date: Thu, 12 Aug 2021 21:41:32 -0700 Message-ID: <20210813044133.1536842-3-jhubbard@nvidia.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210813044133.1536842-1-jhubbard@nvidia.com> References: <20210813044133.1536842-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cf799d3e-8b0f-41a1-751d-08d95e14a226 X-MS-TrafficTypeDiagnostic: MN2PR12MB4063: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4714; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: RLp3l3hLR1KDQQrKW4vQMgxNgGO/hyHV1xrgRE+5JgymXMA6JvK1xmOXQbm8HM8dvp1bOeZDrBDYNeHJfl4u8tLCtMvEVzU/8ph4SrhKQFBFIOkfQy0C0helu1xPWsQNp987nv/d6dxuNiNo1nn1/lcKTXeB/9NRIAhy74Jp8NICXECuP1ek8d1MUjihDPpkFpZrgR7gGFgY0l2zq3zuqwggVNLTLHGKFMXLKlCp1n6qoQ94aaD8li2UJvpKpEsarwP0tXAjRxB4iVmM69hrb4RwUMw+YY1GgCNR7iLThxkN9NJ5kDMk5yEp4R11Xiv/ODM+WgyjElTg9gGKQ522+E6TYH4mXlaVtY+OoRzoKNrpah5tzdSOF7L4ydaULn4MgU2ex3LM6MqZ0+sBAhXvexDTCtREH4DXzqPtmleY0Djr3anx6zeY4ZmbWGDTmrnnMi4v7Aath7EieGO/PA6diAlLXI03sgPYqbLRTXmIF4IRdQqzU0X6AZUhNOgvXCp/9UWzG7CZuGlUl8W9FZVFZwsPpdYnwM/SAGGEFwj6kH9kYN4A3YkNUPNPThWtGs1AYAkLc0gmWSd0lRvTqycvCuT4fnQDK5WnxSwCUK25GRB5wnfPbI72Q5PUVkVq7DqP4weHdrLCD53adXZI+qFZaIQsk/HmLi7wrQfe9sQA/GeIvDvLh24E0c9M0k/uXJwuHZBPGOmwFdJ96LVAoBzIiw== X-Forefront-Antispam-Report: CIP:216.228.112.34;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid03.nvidia.com;CAT:NONE;SFS:(4636009)(39860400002)(396003)(346002)(136003)(376002)(36840700001)(46966006)(82310400003)(8676002)(2906002)(36906005)(82740400003)(316002)(5660300002)(7636003)(70206006)(70586007)(7416002)(6916009)(86362001)(36860700001)(356005)(83380400001)(6666004)(2616005)(54906003)(1076003)(426003)(4326008)(26005)(186003)(47076005)(107886003)(36756003)(336012)(478600001)(8936002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2021 04:41:36.1312 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cf799d3e-8b0f-41a1-751d-08d95e14a226 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.34];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT044.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4063 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org try_grab_page() does the same thing as try_grab_compound_head(..., refs=1, ...), just with a different API. So there is a lot of code duplication there. Change try_grab_page() to call try_grab_compound_head(), while keeping the API contract identical for callers. Also, now that try_grab_compound_head() always has a caller, remove the __maybe_unused annotation. Cc: Matthew Wilcox Reviewed-by: Christoph Hellwig Signed-off-by: John Hubbard --- include/linux/mm.h | 4 ++-- mm/gup.c | 35 +++++------------------------------ 2 files changed, 7 insertions(+), 32 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index ce8fc0fd6d6e..ba985eaf3f19 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1204,8 +1204,8 @@ static inline void get_page(struct page *page) } bool __must_check try_grab_page(struct page *page, unsigned int flags); -__maybe_unused struct page *try_grab_compound_head(struct page *page, int refs, - unsigned int flags); +struct page *try_grab_compound_head(struct page *page, int refs, + unsigned int flags); static inline __must_check bool try_get_page(struct page *page) diff --git a/mm/gup.c b/mm/gup.c index 52f08e3177e9..886d6148d3d0 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -124,8 +124,8 @@ static inline struct page *try_get_compound_head(struct page *page, int refs) * considered failure, and furthermore, a likely bug in the caller, so a warning * is also emitted. */ -__maybe_unused struct page *try_grab_compound_head(struct page *page, - int refs, unsigned int flags) +struct page *try_grab_compound_head(struct page *page, + int refs, unsigned int flags) { if (flags & FOLL_GET) return try_get_compound_head(page, refs); @@ -208,35 +208,10 @@ static void put_compound_head(struct page *page, int refs, unsigned int flags) */ bool __must_check try_grab_page(struct page *page, unsigned int flags) { - WARN_ON_ONCE((flags & (FOLL_GET | FOLL_PIN)) == (FOLL_GET | FOLL_PIN)); + if (!(flags & (FOLL_GET | FOLL_PIN))) + return true; - if (flags & FOLL_GET) - return try_get_page(page); - else if (flags & FOLL_PIN) { - int refs = 1; - - page = compound_head(page); - - if (WARN_ON_ONCE(page_ref_count(page) <= 0)) - return false; - - if (hpage_pincount_available(page)) - hpage_pincount_add(page, 1); - else - refs = GUP_PIN_COUNTING_BIAS; - - /* - * Similar to try_grab_compound_head(): even if using the - * hpage_pincount_add/_sub() routines, be sure to - * *also* increment the normal page refcount field at least - * once, so that the page really is pinned. - */ - page_ref_add(page, refs); - - mod_node_page_state(page_pgdat(page), NR_FOLL_PIN_ACQUIRED, 1); - } - - return true; + return try_grab_compound_head(page, 1, flags); } /** From patchwork Fri Aug 13 04:41:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12434695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C88DC43216 for ; Fri, 13 Aug 2021 04:41:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E7147610A5 for ; Fri, 13 Aug 2021 04:41:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238688AbhHMEmH (ORCPT ); Fri, 13 Aug 2021 00:42:07 -0400 Received: from mail-co1nam11on2075.outbound.protection.outlook.com ([40.107.220.75]:58113 "EHLO NAM11-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S238661AbhHMEmE (ORCPT ); Fri, 13 Aug 2021 00:42:04 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NaArGSSh2WgxIf/NZgayKZYGwFfluJPeE6JiZ6snT3hJAGY8MbXrmtzqik04s0X9buPLX2CFNbzW9eXzHCtEap35YnlWr48iyqnNdrXcvjBXpx09sLjHjqHl2swDfU1eLskzvUdLlrk2L9+eJasVt7eqguYuLynUFdo7fH3RQ7pkNi4X8ET8dJSj4VFQzt7m1uCUcTsJfx84TjDXc8yL5QQ29AP8/XyF5DkRMk3wYCjJvxkB+U+KspG8bSElEgrJnmFN4r/ZgipksQC2x9HUmyC8YrYkNnWOE23LrPQxtIAJzIr91Pm8LpsFmStSKOWQjng/1rq6XIzXYrHB8PC1Fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TBxDzvWl/c/IHWxMVFXMKwj27YOd/yVk3L+q5DJEkWs=; b=deMOq6BkjFnKFAkoBlkBU+rH6K9NzgZAma+q8yVtDDyMEY31V/SKA/4a7HlYrS2GbN26KhE/AuCPDFY0ZLrBrhU5FuJVK2XBRNaahstIIkJJPk81ibcWes5R8mihOk4Wvto8DhQBCTRWy0hrGOTzbTIRJA0CfwQyR5e42Gig2ntLF/SHjNMcAsgZdJPRG/HJEcX4zVjTtNv/K/XUf8+TomCxqOlPUwZcmV25Ew7W2QxlRsd7+rzTqHs8V1xLSrJOozGBV3Mn3qV5WbVgbvtJQY+ztLkdVN63A3y2ommsua2QWzYLi7ZiUCXGXa6F82jhVX9YMJjZxPAQHT8Rd3ei9g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.32) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TBxDzvWl/c/IHWxMVFXMKwj27YOd/yVk3L+q5DJEkWs=; b=OzzxVNTsqF0y9CM8m749T9kUaMpKp3SZbg3g1J/yskPHUIJ1LnmtaO12BtZyUiWdDnykcNKRvS8neUu9fCZCk5KyP6W7mbxWe80NiMhePqDrWrrLBoJxRCGUIb/1QX3ZT2GQcP02ouOH5jzuRPtbChULAHdOUmc1iu+Mrd6srq2mdJspHGe6q8FucDmuCjHzdGmA+Edfly6gwkCAJjXYf0BnKd/5CCcCeXjmznWSrsfu39Jbcijeqfu4FvyWCkdsEV6pBRmmNdVz8ylLmytLjbOpr4fyfkxGYUqYoM5jU+r5UaDA2aeq3f2J1ajjqpCj2RhHtcSwAchfcWq+xua6KA== Received: from BN9PR03CA0076.namprd03.prod.outlook.com (2603:10b6:408:fc::21) by BN9PR12MB5099.namprd12.prod.outlook.com (2603:10b6:408:118::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16; Fri, 13 Aug 2021 04:41:36 +0000 Received: from BN8NAM11FT016.eop-nam11.prod.protection.outlook.com (2603:10b6:408:fc:cafe::eb) by BN9PR03CA0076.outlook.office365.com (2603:10b6:408:fc::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.17 via Frontend Transport; Fri, 13 Aug 2021 04:41:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.32) smtp.mailfrom=nvidia.com; vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.32 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.32; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.32) by BN8NAM11FT016.mail.protection.outlook.com (10.13.176.97) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4415.14 via Frontend Transport; Fri, 13 Aug 2021 04:41:36 +0000 Received: from HQMAIL107.nvidia.com (172.20.187.13) by HQMAIL109.nvidia.com (172.20.187.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 12 Aug 2021 21:41:35 -0700 Received: from sandstorm.attlocal.net (172.20.187.5) by mail.nvidia.com (172.20.187.13) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 13 Aug 2021 04:41:35 +0000 From: John Hubbard To: Andrew Morton CC: Matthew Wilcox , Christoph Hellwig , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , LKML , , , , John Hubbard Subject: [PATCH v3 3/3] mm/gup: Remove try_get_page(), call try_get_compound_head() directly Date: Thu, 12 Aug 2021 21:41:33 -0700 Message-ID: <20210813044133.1536842-4-jhubbard@nvidia.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210813044133.1536842-1-jhubbard@nvidia.com> References: <20210813044133.1536842-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 76314af1-ad14-4db4-0d81-08d95e14a265 X-MS-TrafficTypeDiagnostic: BN9PR12MB5099: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6430; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PxwZtv+fXudGMB3gGfMvgCyAj29gbEhM+b4t3uuf8WF6NqHb++VuuJWqinNeqjjgk+NXh+lWcz9Z7DVBJ2h14dV0BDIFgHAk/XoiYtSHle7g1hNzp39WUyy3KZhTc+xv2gUhoWgEJ2qmL9SMnVLyiv5ijbLkeWwwOqtEMNpxaMqhcgVbf8UTKiPt2vSiO+7MIbaui5BdMd2xKbQZtdkf9Zgsk4WJ1zIMFaS3kxzQibtyz8LUOfVU+hsd6hBcOwiGgnjvOiu2nT++/8K9MU+QSHbE0RppnOm0Iz0gsQ+dhZBSRAIU3S1HyWwpONJ1T/pJW55DAlRxBJKSSaIdyiIDc9ig3ykbkOWxGo9lt2HugsAvVBAI0lLU+XzZSLEGtnpNwEAsJb9sjikqRL8miVoouU4s1HkcD20FuMJy5w8gGJ0DiYk2HmIy4DjlgvmUE0vf8OO/NcyyStZBJpwAAObzLdYQsW4G8tnXkgVm7oCI3OlHucoJw/MYgQwSJbmjq6rmWm4pfWF8d8SvJt0c9YLiB0xYmTSYg9ef5jbFYhRkiTVeR6qhuHe03orgslCztFSNq8zqwO1C6oumKhhBFP2lNLRjKu8X12SAW0AAc0wSAZYIMYN/mNL2wbqZUpDNqJ2Wn8GIiihnJ51RImXc0kt5MWAUSDdtc7n6KXAHeOfxVYbL6rJ5PfSPK8w41Im4alMcPKioOK/usiSqGy06VcSb8g== X-Forefront-Antispam-Report: CIP:216.228.112.32;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid01.nvidia.com;CAT:NONE;SFS:(4636009)(136003)(39860400002)(376002)(396003)(346002)(46966006)(36840700001)(82740400003)(70586007)(7636003)(186003)(7416002)(36860700001)(70206006)(86362001)(4326008)(426003)(82310400003)(336012)(2906002)(83380400001)(356005)(54906003)(2616005)(47076005)(5660300002)(316002)(8676002)(6916009)(1076003)(36756003)(26005)(8936002)(107886003)(478600001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2021 04:41:36.3875 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 76314af1-ad14-4db4-0d81-08d95e14a265 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.32];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT016.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5099 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org try_get_page() is very similar to try_get_compound_head(), and in fact try_get_page() has fallen a little behind in terms of maintenance: try_get_compound_head() handles speculative page references more thoroughly. There are only two try_get_page() callsites, so just call try_get_compound_head() directly from those, and remove try_get_page() entirely. Also, seeing as how this changes try_get_compound_head() into a non-static function, provide some kerneldoc documentation for it. Cc: Matthew Wilcox Reviewed-by: Christoph Hellwig Signed-off-by: John Hubbard --- arch/s390/mm/fault.c | 2 +- fs/pipe.c | 2 +- include/linux/mm.h | 10 +--------- mm/gup.c | 21 +++++++++++++++++---- 4 files changed, 20 insertions(+), 15 deletions(-) diff --git a/arch/s390/mm/fault.c b/arch/s390/mm/fault.c index 212632d57db9..a834e4672f72 100644 --- a/arch/s390/mm/fault.c +++ b/arch/s390/mm/fault.c @@ -822,7 +822,7 @@ void do_secure_storage_access(struct pt_regs *regs) break; case KERNEL_FAULT: page = phys_to_page(addr); - if (unlikely(!try_get_page(page))) + if (unlikely(!try_get_compound_head(page, 1))) break; rc = arch_make_page_accessible(page); put_page(page); diff --git a/fs/pipe.c b/fs/pipe.c index 8e6ef62aeb1c..4394b80dc84f 100644 --- a/fs/pipe.c +++ b/fs/pipe.c @@ -191,7 +191,7 @@ EXPORT_SYMBOL(generic_pipe_buf_try_steal); */ bool generic_pipe_buf_get(struct pipe_inode_info *pipe, struct pipe_buffer *buf) { - return try_get_page(buf->page); + return try_get_compound_head(buf->page, 1); } EXPORT_SYMBOL(generic_pipe_buf_get); diff --git a/include/linux/mm.h b/include/linux/mm.h index ba985eaf3f19..f5ab4fd6d48c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1207,15 +1207,7 @@ bool __must_check try_grab_page(struct page *page, unsigned int flags); struct page *try_grab_compound_head(struct page *page, int refs, unsigned int flags); - -static inline __must_check bool try_get_page(struct page *page) -{ - page = compound_head(page); - if (WARN_ON_ONCE(page_ref_count(page) <= 0)) - return false; - page_ref_inc(page); - return true; -} +struct page *try_get_compound_head(struct page *page, int refs); /** * folio_put - Decrement the reference count on a folio. diff --git a/mm/gup.c b/mm/gup.c index 886d6148d3d0..7a406d79bd2e 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -62,11 +62,24 @@ static void put_page_refs(struct page *page, int refs) put_page(page); } -/* - * Return the compound head page with ref appropriately incremented, - * or NULL if that failed. +/** + * try_get_compound_head() - return the compound head page with refcount + * appropriately incremented, or NULL if that failed. + * + * This handles potential refcount overflow correctly. It also works correclty + * for various lockless get_user_pages()-related callers, due to the use of + * page_cache_add_speculative(). + * + * Even though the name includes "compound_head", this function is still + * appropriate for callers that have a non-compound @page to get. + * + * @page: pointer to page to be gotten + * @refs: the value to add to the page's refcount + * + * Return: head page (with refcount appropriately incremented) for success, or + * NULL upon failure. */ -static inline struct page *try_get_compound_head(struct page *page, int refs) +struct page *try_get_compound_head(struct page *page, int refs) { struct page *head = compound_head(page);