From patchwork Mon Feb 7 06:22:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 12736918 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8820C433EF for ; Mon, 7 Feb 2022 06:30:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 60F026B007B; Mon, 7 Feb 2022 01:30:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E7446B007E; Mon, 7 Feb 2022 01:30:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 19ECD6B007D; Mon, 7 Feb 2022 01:30:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0157.hostedemail.com [216.40.44.157]) by kanga.kvack.org (Postfix) with ESMTP id 025DC6B007B for ; Mon, 7 Feb 2022 01:30:36 -0500 (EST) Received: from forelay.prod.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by fograve03.hostedemail.com (Postfix) with ESMTP id CB85C181EBF15 for ; Mon, 7 Feb 2022 06:22:44 +0000 (UTC) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id CF13A181E5176 for ; Mon, 7 Feb 2022 06:22:42 +0000 (UTC) X-FDA: 79114990164.08.F6220B1 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2086.outbound.protection.outlook.com [40.107.93.86]) by imf09.hostedemail.com (Postfix) with ESMTP id 4E28D140008 for ; Mon, 7 Feb 2022 06:22:42 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QvoNCWNZ2cggSDFZPLcpRrpp8Nn3I/yHTSTzTR4z34Y9nwJKNYhkmE7rIvslOADF5PCZscn8PjjGSIRZdneu4zvfQlxmswocwyKQfF37QOzKAXEKQRF1DCz6BdER1CwO66gL738BQUIDlem9XGiHwxnZQxxePF36RsXjf0qluYwpSzEQvXGlqQjBMTEbDSMKFCf2b1Y22Qp2xW2FJBfBGxewKcDvMpXkk7WnDIfp37IE0BTCngS5zLuYq3C27eyXQUBOi5Sv7B6GYm2afp10ytPCVPXGGv5Qo+jbowcpU57jVvKpN7qNWGuAzJ58BPti3Oi4TeNRYlpp8a51ipErIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DegTwE9NFEicYWZOblQZZlp9tH9DIXZWSkR7haX5kxY=; b=iZJ/EyUXau23ZjPhW3/Ud5BKYeC5fk/tAoZdlx7iNKcCn+9oQ0SA+IkE4Fkt0hEB5b2sfIWS9I9HG/7rgSEJecfokagbTFgzLNDA20Jzy2PfljgbZyY38c4pfV5s1XJCpkgMrWQbBsz3uMOYdBbDGmK77r0OqQOJAg2W1okXUT2BOYolMhtvgzxCBHU2OeuX9sgBwJTx7IugeMaXxpvALLuzR2tT25LLYJijkAomDtyA1KRdgxNZS0t3m4IrGihFL0JJV37iBEgwjNbdGIfHMwtVIeqQ4FW+sNtAejPmRHBQDB+iWEz5XAjs9ADgdzbL4ZPdmI9MfqQRsHHYU+ANuw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=gmail.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DegTwE9NFEicYWZOblQZZlp9tH9DIXZWSkR7haX5kxY=; b=IyP2X8Higir6GnJJq5liotvtAIV7HUE2JYJq5I2gMRmFzWTJYgb3IUTaipf1KZRXi/uOHqJhTdk9HApnHaj4Zq0hDi7rkGDwzWN7CTIM4thGw8ku+u3/Un9yTAyLq9495z7D8hMIoSE6WE0JGTDOY6eF68TnB6tQOE19wZzkcOWH3cUgfERy5dqS0P2XzuBxNOR4BouXCo5fDf4kfD68Tk0QdTOrgHWui6OGWEjVCyLwItvLBLBofaLCGofvWr9Gw+Fuvc0zjZeRc8ztHJ5971/ECIBH/+9IhesN6yn+PiffDVmPGtX3bWt0iHgPettRx82nPSSBgpDn0vJ9yIY8ZQ== Received: from MW4PR04CA0151.namprd04.prod.outlook.com (2603:10b6:303:85::6) by CH2PR12MB3990.namprd12.prod.outlook.com (2603:10b6:610:28::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.18; Mon, 7 Feb 2022 06:22:37 +0000 Received: from CO1NAM11FT020.eop-nam11.prod.protection.outlook.com (2603:10b6:303:85:cafe::4d) by MW4PR04CA0151.outlook.office365.com (2603:10b6:303:85::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.12 via Frontend Transport; Mon, 7 Feb 2022 06:22:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT020.mail.protection.outlook.com (10.13.174.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4951.12 via Frontend Transport; Mon, 7 Feb 2022 06:22:36 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 7 Feb 2022 06:22:36 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Sun, 6 Feb 2022 22:22:35 -0800 Received: from sandstorm.attlocal.net (10.127.8.11) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Sun, 6 Feb 2022 22:22:34 -0800 From: John Hubbard To: Andrew Morton , Peter Xu , Jason Gunthorpe CC: David Hildenbrand , Lukas Bulwahn , Claudio Imbrenda , Christoph Hellwig , Matthew Wilcox , Jan Kara , "Kirill A . Shutemov" , Alex Williamson , Andrea Arcangeli , LKML , , John Hubbard , Christoph Hellwig , Jason Gunthorpe Subject: [PATCH v5 4/5] mm: change lookup_node() to use get_user_pages_fast() Date: Sun, 6 Feb 2022 22:22:12 -0800 Message-ID: <20220207062213.235127-5-jhubbard@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220207062213.235127-1-jhubbard@nvidia.com> References: <20220207062213.235127-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 04adbb69-3679-4652-7eac-08d9ea023c07 X-MS-TrafficTypeDiagnostic: CH2PR12MB3990:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3826; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oX6KNGEXMvfcvZ0LNfd5jFRUGqKqSG1wL/5KqQk8Pk6xH14pa+HlT/cG5J+e4TzY/ennRnOzDTJZgbf5sROtOGKhdXjKDqCKHkd9ZOF3Qdi1SjTk3HkRUvk19tVCkHzSTPBJ3CyQAZkkWa8kLXUkwHY2HBZ8TDOI1dlFsj+dRtTapRXiU7QRX+LrzyxvhqOt7cAKHY6GOSoDEFW/amBi8MXAtif1HDmbx5Pv7EcQeJFyI+/0ttDfo3EIB5IaJEraaI5aRG9rJI54GVBuiz65VN9gs5xZnW3ugkQFKuC523Fy1hPq0CwOCOmzakyTuhOjhmFWnT/K1BEu8iz0mDjMN52V/8leGWnH9ZOhtUzclJlqM1eaMCBONH6gf21UIVtqDDt2yt0QXgJKplKysmcdzcGUTKUN9GYgxW2f7rRHX8+RLQXoC5R+ATPJeRtTzOVEbmm95iDu7LxntR+Vx2/VLKlatopRzH10GiwTPRx9XLip0rb639VaZomUh/olT4Y8gMU+DCj1GjeXkBocA9buGLwMMSwng9kFlDlntnWQMWgVlXLn9XeLSqPl1nYajK3z/lJfuyn17AgM1JOVhbyB/I9goto5R0IHdj891fctbJ70q2liWUFKN/uR8I6bK3r5HhxSkvXqFa1muX2UpXOjytmX81eppMTZHv6TwUZgQ5quis/A4JXsSGTIalwG9pRWe6pu6yhFacD43jhynRvGPA== X-Forefront-Antispam-Report: CIP:12.22.5.235;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(6666004)(8936002)(356005)(5660300002)(70206006)(70586007)(8676002)(81166007)(4326008)(36756003)(83380400001)(7416002)(426003)(336012)(26005)(40460700003)(110136005)(47076005)(1076003)(54906003)(36860700001)(86362001)(2906002)(508600001)(2616005)(107886003)(316002)(186003)(82310400004)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Feb 2022 06:22:36.5563 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 04adbb69-3679-4652-7eac-08d9ea023c07 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.235];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT020.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB3990 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 4E28D140008 X-Stat-Signature: nbs3men4dcnkwnkzxpf57z35cpezfuj6 X-Rspam-User: nil Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=IyP2X8Hi; spf=none (imf09.hostedemail.com: domain of jhubbard@nvidia.com has no SPF policy when checking 40.107.93.86) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com X-HE-Tag: 1644214962-316925 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The purpose of calling get_user_pages_locked() from lookup_node() was to allow for unlocking the mmap_lock when reading a page from the disk during a page fault (hidden behind VM_FAULT_RETRY). The idea was to reduce contention on the heavily-used mmap_lock. (Thanks to Jan Kara for clearly pointing that out, and in fact I've used some of his wording here.) However, it is unlikely for lookup_node() to take a page fault. With that in mind, change over to calling get_user_pages_fast(). This simplifies the code, runs a little faster in the expected case, and allows removing get_user_pages_locked() entirely, in a subsequent patch. Reviewed-by: Christoph Hellwig Reviewed-by: Jan Kara Reviewed-by: Jason Gunthorpe Reviewed-by: Claudio Imbrenda Signed-off-by: John Hubbard --- mm/mempolicy.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 028e8dd82b44..3f8dc58da3e8 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -907,17 +907,14 @@ static void get_policy_nodemask(struct mempolicy *p, nodemask_t *nodes) static int lookup_node(struct mm_struct *mm, unsigned long addr) { struct page *p = NULL; - int err; + int ret; - int locked = 1; - err = get_user_pages_locked(addr & PAGE_MASK, 1, 0, &p, &locked); - if (err > 0) { - err = page_to_nid(p); + ret = get_user_pages_fast(addr & PAGE_MASK, 1, 0, &p); + if (ret > 0) { + ret = page_to_nid(p); put_page(p); } - if (locked) - mmap_read_unlock(mm); - return err; + return ret; } /* Retrieve NUMA policy */ @@ -968,14 +965,14 @@ static long do_get_mempolicy(int *policy, nodemask_t *nmask, if (flags & MPOL_F_NODE) { if (flags & MPOL_F_ADDR) { /* - * Take a refcount on the mpol, lookup_node() - * will drop the mmap_lock, so after calling - * lookup_node() only "pol" remains valid, "vma" - * is stale. + * Take a refcount on the mpol, because we are about to + * drop the mmap_lock, after which only "pol" remains + * valid, "vma" is stale. */ pol_refcount = pol; vma = NULL; mpol_get(pol); + mmap_read_unlock(mm); err = lookup_node(mm, addr); if (err < 0) goto out;