From patchwork Fri Aug 24 19:25:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10575699 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 75D1F921 for ; Fri, 24 Aug 2018 19:25:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 668EF2BA85 for ; Fri, 24 Aug 2018 19:25:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5A9D92BAE7; Fri, 24 Aug 2018 19:25:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D3FC62BA85 for ; Fri, 24 Aug 2018 19:25:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 261816B312C; Fri, 24 Aug 2018 15:25:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2134A6B312E; Fri, 24 Aug 2018 15:25:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E95DD6B312C; Fri, 24 Aug 2018 15:25:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f197.google.com (mail-qt0-f197.google.com [209.85.216.197]) by kanga.kvack.org (Postfix) with ESMTP id A2D3F6B312D for ; Fri, 24 Aug 2018 15:25:53 -0400 (EDT) Received: by mail-qt0-f197.google.com with SMTP id u45-v6so8811978qte.12 for ; Fri, 24 Aug 2018 12:25:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=K3mw4RAhG2T3RfOVBkd57Axd4gtEfA0wB1aTKV08NHs=; b=sDHrLU4wM5f8MtwPDMdG1zFf7dAfAynKJbHh6svHfeN/+pyZ7gb68vBXhwX9WRTr+h V19qWBWSN7uB0rpAN76YkIXEpy5CE8tzwGHzD7ZCXpxsO0JVT+zwraorM66NDxBzg70U Tkg4wrei2MIyYNS8Lp8mBJ5e9wROFBdljSA9jqLbQ3HQne29ndVa9IyO0KvQXHtLk24X sfhKS+UZBGQRKLdhl6TMc8gVQral0+6z6b36CEvGYCUCeEPpCRWKZG8IZzuZNeL5DZxW cnyw5CWC4/UbuudcQ9hYOoeYldJt9AmYIgvMY2vUDKPyUpkOtVLvuqeZroKDnOEGx7Rp alpw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51Dg9ehdiejkVMyeO22D0rgdnmjkAZ44p/tXs286drxnkAW9Jf7O DtGIfcgd/a7H9/8QZEV8wjpARldKRfEo0dKVHDTEPTZ6JyrW36L8vFc41MDMMzbEdQ3rckA9H9e RCBlCfpkX/CPPVorHrGGKNFoj3Rv+2KZc0ALysI6I0fBp8QAWX54jq4qV3kFDnyf7oQ== X-Received: by 2002:ac8:690b:: with SMTP id e11-v6mr3337277qtr.214.1535138753419; Fri, 24 Aug 2018 12:25:53 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZVZGAkv29DbFvzcs1/JGco0Q/2Y5YKP0p2H+Zvud36bah2xkcWp2FFifvRL/TYrXE8Bv03 X-Received: by 2002:ac8:690b:: with SMTP id e11-v6mr3337244qtr.214.1535138752909; Fri, 24 Aug 2018 12:25:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535138752; cv=none; d=google.com; s=arc-20160816; b=Ot3wxGfj+1TI3Ap+QNnTo5vTlQowmQ1phuXRfiRNaLKuD7wUd4QfBVu0TdgiwFYzET nFtt6/zcWYsLo9ItiEL1A3niHVOi9DHGVaDBNhZ+ZQNsbfDYw0F3mfE6maiP1KyRm87e ltdbHThhrCiyRfSQhAM8/UMZtGM/wnWTkbCH+VEYbspO59D22uEaToAALYA0qYC47S03 47qF8+VLy0trkyxnS+/gEB0CFnYJj9cXb6IK3JmJMRoCjafsaEOZP4sh4Kf+mK3PbITr YBL8Ko+WCh3ui8BzddOyAT6S9dbtFtIujPHICpL5qdEDnGgv9znyZUTdZIdhV6LJyyrF cLXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=K3mw4RAhG2T3RfOVBkd57Axd4gtEfA0wB1aTKV08NHs=; b=eJVsWcuNkDyTFxk2pcKIf4YJ73iCyzyeGV3HXCkSuw23lCqMyQ1/pHANGKtk0fAoMK InFIKt+2WjGHOO9i68L7KhmpWfTTGFGk77Om+HMP5EJFpsouVGfUwkTg6ZdnMolMjj95 zy87ajQxecSj91zIQDgNJtWKRoz4nfuDSyM3WANWc5xhaZeVjwJFALs8gMOJKdFczFRH Tz7o0SE6yxnvcSrnjjjHgOCsI8MimvWaTD1YEZehcfSS0stRi9f4mgr0w935+MeqXNPy XIyIXKH+GzaVsgMKmP2l00mqmf+ZEmvyDib0HP65vbB1d1atmiKV89rBhUTylvzq9dl3 pQqQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id 6-v6si8295757qks.284.2018.08.24.12.25.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Aug 2018 12:25:52 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 97273402332F; Fri, 24 Aug 2018 19:25:52 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-125.rdu2.redhat.com [10.10.122.125]) by smtp.corp.redhat.com (Postfix) with ESMTP id 46B812026D6D; Fri, 24 Aug 2018 19:25:52 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= Subject: [PATCH 1/7] mm/hmm: fix utf8 ... Date: Fri, 24 Aug 2018 15:25:43 -0400 Message-Id: <20180824192549.30844-2-jglisse@redhat.com> In-Reply-To: <20180824192549.30844-1-jglisse@redhat.com> References: <20180824192549.30844-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Fri, 24 Aug 2018 19:25:52 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Fri, 24 Aug 2018 19:25:52 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jglisse@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Somehow utf=8 must have been broken. Signed-off-by: Jérôme Glisse Cc: Andrew Morton --- include/linux/hmm.h | 2 +- mm/hmm.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 4c92e3ba3e16..1ff4bae7ada7 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -11,7 +11,7 @@ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * - * Authors: Jérôme Glisse + * Authors: Jérôme Glisse */ /* * Heterogeneous Memory Management (HMM) diff --git a/mm/hmm.c b/mm/hmm.c index c968e49f7a0c..9a068a1da487 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -11,7 +11,7 @@ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * - * Authors: Jérôme Glisse + * Authors: Jérôme Glisse */ /* * Refer to include/linux/hmm.h for information about heterogeneous memory From patchwork Fri Aug 24 19:25:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10575701 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C7D61921 for ; Fri, 24 Aug 2018 19:26:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B7D592BA85 for ; Fri, 24 Aug 2018 19:26:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AC3C22BAE7; Fri, 24 Aug 2018 19:26:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 36FA12BA85 for ; Fri, 24 Aug 2018 19:26:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F3CE56B312D; Fri, 24 Aug 2018 15:25:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E70816B312E; Fri, 24 Aug 2018 15:25:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D12AE6B312F; Fri, 24 Aug 2018 15:25:54 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk0-f200.google.com (mail-qk0-f200.google.com [209.85.220.200]) by kanga.kvack.org (Postfix) with ESMTP id 9BE6D6B312D for ; Fri, 24 Aug 2018 15:25:54 -0400 (EDT) Received: by mail-qk0-f200.google.com with SMTP id u129-v6so8613359qkf.15 for ; Fri, 24 Aug 2018 12:25:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=vYaPmqhvzmuzdaIlTeNPHIcGuU+/Iamv9k0Tm9yI7k0=; b=rL2HWucrYFxLtI5TGOZEdTWjC5kqUGiZZTUIkv2udjq7RfpmItjKHH3LwRIH3k5pSG H5TIjPAk/bV2lXgQ7uQQPrwzNPm5WKGlFik9zrfQZfwLANBcLzD9nnq/gH1EhNEPZKvO nW7p9R9gbPxqco4Oho7oAG5uyAa5V/1rFq6aqcL7SUKqw+hiJDEr3DenH5EAR6MArS4E qp6Qho6GeU8b75Z+zCJYtoFuEMk27EUVvH9LmMDRlBZiNto809UJS38+k06RrDXrfhbL bnLdyq9SmgLyqR6PvtZmCGNU2NjFKvRXzJyW+E+43tW5l26q8yi/0w0ptlcVKOrk804c StUw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51DS7PnGvEwa2TM/D+7xet7r4n4pqa7r1dH+mVmDvubcazzYbhR4 kFNffSodM+lm5TUw1B0SYmb3KJG0KiJXdtifRUcaJ9o9yrAV1Zei99siq0HCPRYCI5hdTMFu5YP yeR10aULABUil+kdm8+Qzs/mKOnbLW9qYmmEwCPs8aM1jKUc5H1AIBrRkRdwnXFBaqA== X-Received: by 2002:ac8:36fa:: with SMTP id b55-v6mr3434768qtc.49.1535138754424; Fri, 24 Aug 2018 12:25:54 -0700 (PDT) X-Google-Smtp-Source: ANB0VdaAt8Z2crL9D0KMm7Td22a3SsCueXuLIZDIzF+fBU7jOJFvLnlk2RJs1JB9p9DbvqbgGHWY X-Received: by 2002:ac8:36fa:: with SMTP id b55-v6mr3434713qtc.49.1535138753721; Fri, 24 Aug 2018 12:25:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535138753; cv=none; d=google.com; s=arc-20160816; b=GUMWPcImb20vtDEjelMh4t7TPm7+2FpjAOHmOpyDITN3Lpdd7sZ8vEkyJf5pbpSn2l 4CNrjWSQYoQBsw144usVAofkkE0DGCXPp2g+BgBRMeZJ9qhwR+IHqW1KeAhr9GjPI57l R8wg5kH9PVTQX+REU3rKBiXKrianZ0OfJXjkyAizRU22GK4vJOw8Ugc/5QJSdvDkFmEB 6Z4d9YHis7o9rDGumBNlXoWyNV6mU3h6slJmwUe3b+JArQnIsDTMmgobFoTmqUHQ6u2P Dr0rblncsKbz3dRuudqtE7VYV7qU430fyPPX7TG7tGRDmPXFdSHmzk/7m81dGMCpDjbc LLlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=vYaPmqhvzmuzdaIlTeNPHIcGuU+/Iamv9k0Tm9yI7k0=; b=M2gYklnWuIaapA7GSYJRwQ4ctu+yN/hb5Sy8U3egtP62aHure0jW6pbgkdwkm4dNco 9A+p3Qacm1dc/qr9Q7UQIdFU8iSOu07Ook0xdSL6vKYdm3pcYtdy2vnWPPDJB06770j7 QV3kMoGMuoPU5dYNG9GcYADPGdjOA8qyIC0N9Dsfv/nIGT4GqBVxBk4bV12tq+z78MzQ cnOScQi5UWr6hhX4SDdalhuaoYPCHhXepE6nO7d4W8NHFBJA0gNxrw5alEjHSjZOqawZ Jas1L1dsKL5k2gDHCkfRnc9x00V6KJRp/O0DdA8r2Q4IciS93//OxAxQmyuSRx8dBMji P2iA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id o23-v6si1162440qtp.279.2018.08.24.12.25.53 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Aug 2018 12:25:53 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 62F80402332F; Fri, 24 Aug 2018 19:25:53 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-125.rdu2.redhat.com [10.10.122.125]) by smtp.corp.redhat.com (Postfix) with ESMTP id AC1AE2026D6D; Fri, 24 Aug 2018 19:25:52 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, Ralph Campbell , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , "Kirill A . Shutemov" , stable@vger.kernel.org Subject: [PATCH 2/7] mm/rmap: map_pte() was not handling private ZONE_DEVICE page properly Date: Fri, 24 Aug 2018 15:25:44 -0400 Message-Id: <20180824192549.30844-3-jglisse@redhat.com> In-Reply-To: <20180824192549.30844-1-jglisse@redhat.com> References: <20180824192549.30844-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Fri, 24 Aug 2018 19:25:53 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Fri, 24 Aug 2018 19:25:53 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jglisse@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Ralph Campbell Private ZONE_DEVICE pages use a special pte entry and thus are not present. Properly handle this case in map_pte(), it is already handled in check_pte(), the map_pte() part was lost in some rebase most probably. Without this patch the slow migration path can not migrate back private ZONE_DEVICE memory to regular memory. This was found after stress testing migration back to system memory. This ultimatly can lead the CPU to an infinite page fault loop on the special swap entry. Signed-off-by: Ralph Campbell Signed-off-by: Jérôme Glisse Cc: Andrew Morton Cc: Kirill A. Shutemov Cc: stable@vger.kernel.org --- mm/page_vma_mapped.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index ae3c2a35d61b..1cf5b9bfb559 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -21,6 +21,15 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw) if (!is_swap_pte(*pvmw->pte)) return false; } else { + if (is_swap_pte(*pvmw->pte)) { + swp_entry_t entry; + + /* Handle un-addressable ZONE_DEVICE memory */ + entry = pte_to_swp_entry(*pvmw->pte); + if (is_device_private_entry(entry)) + return true; + } + if (!pte_present(*pvmw->pte)) return false; } From patchwork Fri Aug 24 19:25:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10575703 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D4A03921 for ; Fri, 24 Aug 2018 19:26:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C4C612BA85 for ; Fri, 24 Aug 2018 19:26:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B89AE2BAE7; Fri, 24 Aug 2018 19:26:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4F15C2BA85 for ; Fri, 24 Aug 2018 19:26:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D10D6B312E; Fri, 24 Aug 2018 15:25:55 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 635476B312F; Fri, 24 Aug 2018 15:25:55 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 43A0C6B3131; Fri, 24 Aug 2018 15:25:55 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk0-f200.google.com (mail-qk0-f200.google.com [209.85.220.200]) by kanga.kvack.org (Postfix) with ESMTP id 0FE146B312F for ; Fri, 24 Aug 2018 15:25:55 -0400 (EDT) Received: by mail-qk0-f200.google.com with SMTP id t9-v6so8649515qkl.2 for ; Fri, 24 Aug 2018 12:25:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=i15p8svO4F0HhB2UonErervpfRK+VRH3fmW+ilgKuaQ=; b=SXBIe7YU6GWuzDLLcztx0wAWl1Cnmo19MiMEiEso+NBaet53EnP7iWbIfHdRGa2SId lwYyVNe97ECtdflt+snZWQBZKVE6VNb2ebU4cCB9fThwEm52BQZIhWT+DmbyKWDYyd00 NAA2mfzxgtQJnPp5jVn9OnDwB93tUnS3SesKFZw8bNWig+hgt0Eb0+51oET13xH02kO1 gWL8SBiCLKQdrDwCKPean3cnln+PNiktWTJXyGmy02TsryM0rLQXE0uu6249yKQ2i2FS Qw3t0qUe6CUFzjjdFikT7+faBl18piRj1MGAmTXyED3HV4euL2gVW1uuxsCyllbcqKhX Okag== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51DxNBNl5JjAatr/WjWWHuOQrVaEhvK0z/5TJKJsv5UwxDHnVo+Z Z+aFD7DrsQtaB41L6eTp/epeVjJJV9c4yVzEFSJkabxpNa7gq5DytxT3rQnScipF4UYRHmZvQmS Mo5+ab3GDSsrof8z+ZPplk3Ai9Jwf486hkPVcNgfGHCVNLy2ShJFSYASvMmVLIki6xQ== X-Received: by 2002:a37:1298:: with SMTP id 24-v6mr3192957qks.174.1535138754850; Fri, 24 Aug 2018 12:25:54 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbTrj2CBpUrTVPQFZpanuDETDbgBD2Cs4v0KNAXBmLEbHMdntLnakpwe+sxwLv4GCmMIbZl X-Received: by 2002:a37:1298:: with SMTP id 24-v6mr3192924qks.174.1535138754222; Fri, 24 Aug 2018 12:25:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535138754; cv=none; d=google.com; s=arc-20160816; b=ZhGHApdL5pzVnhcbWVPjcCqkM9yfMyFhwInNEagUx3InkDNZ9tSwV+nTVbFQlroVeS r7VIvs8to1gHKmeorimxJCXxFkpmsNbf2F4XQDsiQ/jaCE0i7Hn1+d0pX2HP+HaMwMqC MOZ3XY83HdiPbeuX9MzwGW+L7LU+00JCoCtdq4PLnAmkI4wJItdQw+PF2JvO3ABYZDKU PmJAJcR7D1gPICjhmjj1SWYWQZG2XxDQw+0ypvth8nbfERLLaREixWgX0RKlQOlKnUIw Gv3qPCRm/Tf9L5zG8qciW9qHb/szMEkjF8mn9xTxSbSDDHbxiFFgLM2o07byaLdAuXCQ TkMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=i15p8svO4F0HhB2UonErervpfRK+VRH3fmW+ilgKuaQ=; b=Nydn7VtIHnMnZEEeETXI30l9+Xw98jiG3jH1eKWITpLPXzJQfMkcQOZ2bzumbx2gwJ UjSc9teOnGaDUodLJXzm515BG5DHptiQuIglRXO+tbPdoEywqXhvtFdjsg/Zy9mkyYoa PQQmnyCPb+bnigSjTxURIRCKvMAAwilifpts5H0bkN9jZGwZcdOZy9LeXrQ9XRIJ9nhh 1CeSlKpAl6EGEDUfi31g92WdTizjGVDJHEXSemMdjtPWlGHdcV5ozHB9zvGOGx/auirB DDwgcrim370zoykLAHVCNhehVx/I0dHG/+XnOpXFHtd7vBuQy/0cvfiLTUeNtYPZd7Op d1BA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id b24-v6si4098549qtk.113.2018.08.24.12.25.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Aug 2018 12:25:54 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D56E440241C3; Fri, 24 Aug 2018 19:25:53 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-125.rdu2.redhat.com [10.10.122.125]) by smtp.corp.redhat.com (Postfix) with ESMTP id 784B32026D6D; Fri, 24 Aug 2018 19:25:53 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, Ralph Campbell , stable@vger.kernel.org Subject: [PATCH 3/7] mm/hmm: fix race between hmm_mirror_unregister() and mmu_notifier callback Date: Fri, 24 Aug 2018 15:25:45 -0400 Message-Id: <20180824192549.30844-4-jglisse@redhat.com> In-Reply-To: <20180824192549.30844-1-jglisse@redhat.com> References: <20180824192549.30844-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Fri, 24 Aug 2018 19:25:53 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Fri, 24 Aug 2018 19:25:53 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jglisse@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Ralph Campbell In hmm_mirror_unregister(), mm->hmm is set to NULL and then mmu_notifier_unregister_no_release() is called. That creates a small window where mmu_notifier can call mmu_notifier_ops with mm->hmm equal to NULL. Fix this by first unregistering mmu notifier callbacks and then setting mm->hmm to NULL. Similarly in hmm_register(), set mm->hmm before registering mmu_notifier callbacks so callback functions always see mm->hmm set. Signed-off-by: Ralph Campbell Reviewed-by: John Hubbard Reviewed-by: Jérôme Glisse Cc: Andrew Morton Cc: stable@vger.kernel.org Reviewed-by: Balbir Singh --- mm/hmm.c | 36 +++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 9a068a1da487..a16678d08127 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -91,16 +91,6 @@ static struct hmm *hmm_register(struct mm_struct *mm) spin_lock_init(&hmm->lock); hmm->mm = mm; - /* - * We should only get here if hold the mmap_sem in write mode ie on - * registration of first mirror through hmm_mirror_register() - */ - hmm->mmu_notifier.ops = &hmm_mmu_notifier_ops; - if (__mmu_notifier_register(&hmm->mmu_notifier, mm)) { - kfree(hmm); - return NULL; - } - spin_lock(&mm->page_table_lock); if (!mm->hmm) mm->hmm = hmm; @@ -108,12 +98,27 @@ static struct hmm *hmm_register(struct mm_struct *mm) cleanup = true; spin_unlock(&mm->page_table_lock); - if (cleanup) { - mmu_notifier_unregister(&hmm->mmu_notifier, mm); - kfree(hmm); - } + if (cleanup) + goto error; + + /* + * We should only get here if hold the mmap_sem in write mode ie on + * registration of first mirror through hmm_mirror_register() + */ + hmm->mmu_notifier.ops = &hmm_mmu_notifier_ops; + if (__mmu_notifier_register(&hmm->mmu_notifier, mm)) + goto error_mm; return mm->hmm; + +error_mm: + spin_lock(&mm->page_table_lock); + if (mm->hmm == hmm) + mm->hmm = NULL; + spin_unlock(&mm->page_table_lock); +error: + kfree(hmm); + return NULL; } void hmm_mm_destroy(struct mm_struct *mm) @@ -278,12 +283,13 @@ void hmm_mirror_unregister(struct hmm_mirror *mirror) if (!should_unregister || mm == NULL) return; + mmu_notifier_unregister_no_release(&hmm->mmu_notifier, mm); + spin_lock(&mm->page_table_lock); if (mm->hmm == hmm) mm->hmm = NULL; spin_unlock(&mm->page_table_lock); - mmu_notifier_unregister_no_release(&hmm->mmu_notifier, mm); kfree(hmm); } EXPORT_SYMBOL(hmm_mirror_unregister); From patchwork Fri Aug 24 19:25:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10575707 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 59D7D921 for ; Fri, 24 Aug 2018 19:26:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 45D592BA85 for ; Fri, 24 Aug 2018 19:26:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3A2A42BAE7; Fri, 24 Aug 2018 19:26:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B13DF2BA85 for ; Fri, 24 Aug 2018 19:26:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 76DC06B312F; Fri, 24 Aug 2018 15:25:56 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 68F1D6B3135; Fri, 24 Aug 2018 15:25:56 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E0E46B3131; Fri, 24 Aug 2018 15:25:56 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f197.google.com (mail-qt0-f197.google.com [209.85.216.197]) by kanga.kvack.org (Postfix) with ESMTP id 035C86B3131 for ; Fri, 24 Aug 2018 15:25:56 -0400 (EDT) Received: by mail-qt0-f197.google.com with SMTP id y54-v6so4678568qta.8 for ; Fri, 24 Aug 2018 12:25:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=LqOeKNuiVXhkTwrNaWxAc4QVuACWRzDUtG7dHR9dItA=; b=nt4BWy0yZD1isMVGErtBHLNg5mDJ0wU2jVNy9IXmUKkPCbT+6ndMDzM00/ttbPGI0S RrzG6J8lXijttRBPOYYvdx48U3ke5Q9whVDiRd058eIllUNcnJUS6O+ZAg5X906xOImi klG3EPFOQ9+VTCl2KfwXh0qIgNWK/lFalpDNnq+K4xccgamzdsRfBnGfAtKyC7/t/2CG FQtHu3JJBFA7AOJPBoWnhEEGFYo8cbweFOpH9vaYnYXpbUPDGmIYRpYwX4psr0gn2gdO 0SJyRTg/4gPAqKnbgYnZdJdWF3qYLeT672uHHtAVcPfeZ3gXxj2Z1RK4ha9RBzAmO0in UR9A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51C1HcDmRDnKkCZ9gov6XVZKFJ462HYLOeFIP3oZ3IIKBhf/L8Ve UI0mFsj0CCsZuDANW0QCIsS94cJV0t+5ttcJ3MgyzA+ZpyQ/YR0gjCGqQluVM98gUuVeVJpQ11k +rTerC4sHUCrGkxxZhvlaGu2ArHYHr2y0LFN+Zbr9keod/0Q5lYd91LPQwFAEczDbQQ== X-Received: by 2002:ac8:3492:: with SMTP id w18-v6mr3283543qtb.239.1535138755782; Fri, 24 Aug 2018 12:25:55 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbFAlwz3kS4L4mLDZeZu9mgrLL8d6QVDokSTlQ2MOC60CNzbQrzXdVfJaGM8J26MRASOO2G X-Received: by 2002:ac8:3492:: with SMTP id w18-v6mr3283475qtb.239.1535138754764; Fri, 24 Aug 2018 12:25:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535138754; cv=none; d=google.com; s=arc-20160816; b=gVHN3QTKnUTMKHZNIKnToxJvk/f1i9Zntehks9dsa4V03GrTdh+HZMDddFR+5MdYtq yzTFShtOMB/sQ/8ate0eQQC3/i6LtmZBjD6qrHJWLaV+XiSXAgZu9kMsHBlkUrILNUrF uLfMPBLdLuFlJ/+QzHLRaE1yPhWc/fYA4Rz6NWItDa0R8x9b7komnCaIHujOanp2+cRI 5floJW3PjJL18RWWsIG5/QR6maH3FKoXUIQubqeqI4HZ/WaecyeS12rdVj0nf2jGPwaP 0Faif6nqWB+YuRFpabWe2i1dw+LUAHAsZ0xbNyPesBHYbU8pcMoKBtae9OYv6gC+4K0h h7NQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=LqOeKNuiVXhkTwrNaWxAc4QVuACWRzDUtG7dHR9dItA=; b=SbvaSGyrdLu/Q0LCC/RWyFXwp0AAa5cHD5plZqywzEHaQUtg5skPKtZQz/qsz9TmhC CMcvkQEvA0ypsq78ps1hwC5RoOBAIoWViOz/Npj0mdPp6k3kZWWAny98m+tlyjNkb6er ARN4KA+r2jscix1bAp0gWoJgdP/WvX6i42ICcl/x6gJziOX07Yci95IHI6UXVS9MxlXE UcTL0yM/f0UA8tkdU9YI6z4V6bVRgPYL/6Tk/1oW2ihdiRdjw7O7DW6pHksN+36lr4db II7pFCgM3bOYoIpPv847+vEYSpda+OuhvbU30laTJoGLuCXRcmkbiZbA/oW0qTzqK9pr V+qw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id x3-v6si7043611qti.136.2018.08.24.12.25.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Aug 2018 12:25:54 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6C1407DAC2; Fri, 24 Aug 2018 19:25:54 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-125.rdu2.redhat.com [10.10.122.125]) by smtp.corp.redhat.com (Postfix) with ESMTP id EA35A2026D6D; Fri, 24 Aug 2018 19:25:53 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , "Aneesh Kumar K . V" , Ralph Campbell , John Hubbard Subject: [PATCH 4/7] mm/hmm: properly handle migration pmd Date: Fri, 24 Aug 2018 15:25:46 -0400 Message-Id: <20180824192549.30844-5-jglisse@redhat.com> In-Reply-To: <20180824192549.30844-1-jglisse@redhat.com> References: <20180824192549.30844-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Fri, 24 Aug 2018 19:25:54 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Fri, 24 Aug 2018 19:25:54 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jglisse@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Before this patch migration pmd entry (!pmd_present()) would have been treated as a bad entry (pmd_bad() returns true on migration pmd entry). The outcome was that device driver would believe that the range covered by the pmd was bad and would either SIGBUS or simply kill all the device's threads (each device driver decide how to react when the device tries to access poisonnous or invalid range of memory). This patch explicitly handle the case of migration pmd entry which are non present pmd entry and either wait for the migration to finish or report empty range (when device is just trying to pre- fill a range of virtual address and thus do not want to wait or trigger page fault). Signed-off-by: Aneesh Kumar K.V Signed-off-by: Jérôme Glisse Cc: Ralph Campbell Cc: John Hubbard Cc: Andrew Morton --- mm/hmm.c | 45 +++++++++++++++++++++++++++++++++++++++------ 1 file changed, 39 insertions(+), 6 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index a16678d08127..659efc9aada6 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -577,22 +577,47 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, { struct hmm_vma_walk *hmm_vma_walk = walk->private; struct hmm_range *range = hmm_vma_walk->range; + struct vm_area_struct *vma = walk->vma; uint64_t *pfns = range->pfns; unsigned long addr = start, i; pte_t *ptep; + pmd_t pmd; - i = (addr - range->start) >> PAGE_SHIFT; again: - if (pmd_none(*pmdp)) + pmd = READ_ONCE(*pmdp); + if (pmd_none(pmd)) return hmm_vma_walk_hole(start, end, walk); - if (pmd_huge(*pmdp) && (range->vma->vm_flags & VM_HUGETLB)) + if (pmd_huge(pmd) && (range->vma->vm_flags & VM_HUGETLB)) return hmm_pfns_bad(start, end, walk); - if (pmd_devmap(*pmdp) || pmd_trans_huge(*pmdp)) { - pmd_t pmd; + if (!pmd_present(pmd)) { + swp_entry_t entry = pmd_to_swp_entry(pmd); + + if (is_migration_entry(entry)) { + bool fault, write_fault; + unsigned long npages; + uint64_t *pfns; + + i = (addr - range->start) >> PAGE_SHIFT; + npages = (end - addr) >> PAGE_SHIFT; + pfns = &range->pfns[i]; + + hmm_range_need_fault(hmm_vma_walk, pfns, npages, + 0, &fault, &write_fault); + if (fault || write_fault) { + hmm_vma_walk->last = addr; + pmd_migration_entry_wait(vma->vm_mm, pmdp); + return -EAGAIN; + } + return 0; + } + + return hmm_pfns_bad(start, end, walk); + } + if (pmd_devmap(pmd) || pmd_trans_huge(pmd)) { /* * No need to take pmd_lock here, even if some other threads * is splitting the huge pmd we will get that event through @@ -607,13 +632,21 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, if (!pmd_devmap(pmd) && !pmd_trans_huge(pmd)) goto again; + i = (addr - range->start) >> PAGE_SHIFT; return hmm_vma_handle_pmd(walk, addr, end, &pfns[i], pmd); } - if (pmd_bad(*pmdp)) + /* + * We have handled all the valid case above ie either none, migration, + * huge or transparent huge. At this point either it is a valid pmd + * entry pointing to pte directory or it is a bad pmd that will not + * recover. + */ + if (pmd_bad(pmd)) return hmm_pfns_bad(start, end, walk); ptep = pte_offset_map(pmdp, addr); + i = (addr - range->start) >> PAGE_SHIFT; for (; addr < end; addr += PAGE_SIZE, ptep++, i++) { int r; From patchwork Fri Aug 24 19:25:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10575705 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 94F20921 for ; Fri, 24 Aug 2018 19:26:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 83DCB2BA85 for ; Fri, 24 Aug 2018 19:26:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 77F3E2BAE7; Fri, 24 Aug 2018 19:26:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E3A022BA85 for ; Fri, 24 Aug 2018 19:26:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D3366B3132; Fri, 24 Aug 2018 15:25:56 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 434E76B312F; Fri, 24 Aug 2018 15:25:56 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2DF6A6B3133; Fri, 24 Aug 2018 15:25:56 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk0-f199.google.com (mail-qk0-f199.google.com [209.85.220.199]) by kanga.kvack.org (Postfix) with ESMTP id EE96A6B312F for ; Fri, 24 Aug 2018 15:25:55 -0400 (EDT) Received: by mail-qk0-f199.google.com with SMTP id d194-v6so8459871qkb.12 for ; Fri, 24 Aug 2018 12:25:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=AAFnBEnM/0PgHx4HyJY2dSAuwG0Kwrv6FT26CubiiUk=; b=fbTLChLZKr2Jng9r4CfWWCrrOs5Zhwq7bMZfEjOAdv+fwyEY1Tu+e/cp45aV0o9S9V dZAQ22JouCM+MNRT//L2RC+586YgtPr+8P+HWq+uvgfTKRgdGv0bK3F9OEwJNW/CKsm9 +QeXp/2BnTKy19Yq4WxfLHpRHe+sa0vHO5k0F5WlF53K9jlcD1DXQt67YF7VKmOenRyA kVhnRfSjDUFoFo7I6DWGYtwPXr2evdPP97EuBp/aucDNFNao8KfdQcoOVVcU/DjIhsgp aSAid34fiiVXIzEvxNSJoqniGpIGWiVY2kJXHOylbsf3VMXcqdOf7cezIIJW4T6PdnEU Mj9g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51C4cptPFtDs6/4LgxwQqEReWuFThlONnkkFLCk1ExVyZFIjK47c mjJzI1bLdLGfxlxpUrh+e45NXhtWk6LokEtiQzU8TcVTLd2B7g5VaZG6ont/aS1KmSHVp/J0EBP e6/Aq9lWyobQ+yxw9pU3fRNmXUBlJt0IYZSTWEkwoWV4rTa4GvwX3gULSiuaWnGH1sA== X-Received: by 2002:a37:a38b:: with SMTP id m133-v6mr3180147qke.356.1535138755754; Fri, 24 Aug 2018 12:25:55 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZp1t8wGgTGR+JveLCPGIJhzKmc+1wPEHjT+1pGbX4ZwhFQ/oWkKDsEbvAd72LXnCkaJ5LI X-Received: by 2002:a37:a38b:: with SMTP id m133-v6mr3180123qke.356.1535138755281; Fri, 24 Aug 2018 12:25:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535138755; cv=none; d=google.com; s=arc-20160816; b=T+HVLuatV20MeaCiF6vWjWZ1AWOl3+/L6jKTKWIhNdQh7PB0DyzfgJ5RB2rbtdhJEx xdGw/79K3XXbM2tYpIAE0OXue22kGA3YYqYY0FwHF2S3qYpA7wWzpdTCo0gpWN53GFjh rT6BbFAD4409cudVDZ0bDkk3ML8P3xzGRv4oX77+smWRb29LXQRGKWPLMDM1fVRFi4yv Qx/1K/azg9zS5yNsbDW8YX9zoQA46QeOw8jA4YnUwcpeGYPGRLU48iBwZHyt77JRmlYG CEJh2Oty312k2PxUsKkk1G+a3ySkX1pFO8+d9S3kUhZIGe8kVEc67roToD64sfswDAtX SCBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=AAFnBEnM/0PgHx4HyJY2dSAuwG0Kwrv6FT26CubiiUk=; b=GoPRAl9fkHOVu/77ulQPNwcLLHbsF5juuCUbd3vLAy1p2HtFgtZ9NSDmIU1viDVf0+ /eCIf7FbLKkXP8Wzzfefl2sitNR+wpoIAFj0bXicX+lPt+bMYwywqAisI3spQj/xfnPy gVqsV7zeIzKry3kWF+XJdsQH/DoQOtENTIM9+MRtKrFjA+R1jC0yXMzilhMAX6XeJ09w gm7mZK53Mc0WobNTKZbkWQsc/zJYOrIv1kQECPfsnc7N5CsC1S1c6DQHoJ4VSIqztzlB eQaELyZASNxmkM7+55nV3juzKwg+uZuoUoKqNnw/kH7jcJ6a1gJsMOX94yAeEVw4b2iZ NLLg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id t28-v6si2188486qta.348.2018.08.24.12.25.55 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Aug 2018 12:25:55 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EBB8E804B9F4; Fri, 24 Aug 2018 19:25:54 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-125.rdu2.redhat.com [10.10.122.125]) by smtp.corp.redhat.com (Postfix) with ESMTP id 81A972026D6D; Fri, 24 Aug 2018 19:25:54 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Ralph Campbell , John Hubbard Subject: [PATCH 5/7] mm/hmm: use a structure for update callback parameters Date: Fri, 24 Aug 2018 15:25:47 -0400 Message-Id: <20180824192549.30844-6-jglisse@redhat.com> In-Reply-To: <20180824192549.30844-1-jglisse@redhat.com> References: <20180824192549.30844-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Fri, 24 Aug 2018 19:25:54 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Fri, 24 Aug 2018 19:25:54 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jglisse@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Use a structure to gather all the parameters for the update callback. This make it easier when adding new parameters by avoiding having to update all callback function signature. Signed-off-by: Jérôme Glisse Cc: Ralph Campbell Cc: John Hubbard Cc: Andrew Morton --- include/linux/hmm.h | 25 +++++++++++++++++-------- mm/hmm.c | 27 ++++++++++++++------------- 2 files changed, 31 insertions(+), 21 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 1ff4bae7ada7..a7f7600b6bb0 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -274,13 +274,26 @@ static inline uint64_t hmm_pfn_from_pfn(const struct hmm_range *range, struct hmm_mirror; /* - * enum hmm_update_type - type of update + * enum hmm_update_event - type of update * @HMM_UPDATE_INVALIDATE: invalidate range (no indication as to why) */ -enum hmm_update_type { +enum hmm_update_event { HMM_UPDATE_INVALIDATE, }; +/* + * struct hmm_update - HMM update informations for callback + * + * @start: virtual start address of the range to update + * @end: virtual end address of the range to update + * @event: event triggering the update (what is happening) + */ +struct hmm_update { + unsigned long start; + unsigned long end; + enum hmm_update_event event; +}; + /* * struct hmm_mirror_ops - HMM mirror device operations callback * @@ -300,9 +313,7 @@ struct hmm_mirror_ops { /* sync_cpu_device_pagetables() - synchronize page tables * * @mirror: pointer to struct hmm_mirror - * @update_type: type of update that occurred to the CPU page table - * @start: virtual start address of the range to update - * @end: virtual end address of the range to update + * @update: update informations (see struct hmm_update) * * This callback ultimately originates from mmu_notifiers when the CPU * page table is updated. The device driver must update its page table @@ -314,9 +325,7 @@ struct hmm_mirror_ops { * synchronous call. */ void (*sync_cpu_device_pagetables)(struct hmm_mirror *mirror, - enum hmm_update_type update_type, - unsigned long start, - unsigned long end); + const struct hmm_update *update); }; /* diff --git a/mm/hmm.c b/mm/hmm.c index 659efc9aada6..debd2f734ab5 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -127,9 +127,7 @@ void hmm_mm_destroy(struct mm_struct *mm) } static void hmm_invalidate_range(struct hmm *hmm, - enum hmm_update_type action, - unsigned long start, - unsigned long end) + const struct hmm_update *update) { struct hmm_mirror *mirror; struct hmm_range *range; @@ -138,21 +136,20 @@ static void hmm_invalidate_range(struct hmm *hmm, list_for_each_entry(range, &hmm->ranges, list) { unsigned long addr, idx, npages; - if (end < range->start || start >= range->end) + if (update->end < range->start || update->start >= range->end) continue; range->valid = false; - addr = max(start, range->start); + addr = max(update->start, range->start); idx = (addr - range->start) >> PAGE_SHIFT; - npages = (min(range->end, end) - addr) >> PAGE_SHIFT; + npages = (min(range->end, update->end) - addr) >> PAGE_SHIFT; memset(&range->pfns[idx], 0, sizeof(*range->pfns) * npages); } spin_unlock(&hmm->lock); down_read(&hmm->mirrors_sem); list_for_each_entry(mirror, &hmm->mirrors, list) - mirror->ops->sync_cpu_device_pagetables(mirror, action, - start, end); + mirror->ops->sync_cpu_device_pagetables(mirror, update); up_read(&hmm->mirrors_sem); } @@ -183,10 +180,10 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) } static int hmm_invalidate_range_start(struct mmu_notifier *mn, - struct mm_struct *mm, - unsigned long start, - unsigned long end, - bool blockable) + struct mm_struct *mm, + unsigned long start, + unsigned long end, + bool blockable) { struct hmm *hmm = mm->hmm; @@ -202,11 +199,15 @@ static void hmm_invalidate_range_end(struct mmu_notifier *mn, unsigned long start, unsigned long end) { + struct hmm_update update; struct hmm *hmm = mm->hmm; VM_BUG_ON(!hmm); - hmm_invalidate_range(mm->hmm, HMM_UPDATE_INVALIDATE, start, end); + update.start = start; + update.end = end; + update.event = HMM_UPDATE_INVALIDATE; + hmm_invalidate_range(hmm, &update); } static const struct mmu_notifier_ops hmm_mmu_notifier_ops = { From patchwork Fri Aug 24 19:25:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10575709 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E0AA5921 for ; Fri, 24 Aug 2018 19:26:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CF4B22BA85 for ; Fri, 24 Aug 2018 19:26:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C39192BAE7; Fri, 24 Aug 2018 19:26:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 408722BA85 for ; Fri, 24 Aug 2018 19:26:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 046AF6B3131; Fri, 24 Aug 2018 15:25:57 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EE66D6B3133; Fri, 24 Aug 2018 15:25:56 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC7596B3135; Fri, 24 Aug 2018 15:25:56 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f200.google.com (mail-qt0-f200.google.com [209.85.216.200]) by kanga.kvack.org (Postfix) with ESMTP id 9B6AB6B3131 for ; Fri, 24 Aug 2018 15:25:56 -0400 (EDT) Received: by mail-qt0-f200.google.com with SMTP id b5-v6so7855328qtk.4 for ; Fri, 24 Aug 2018 12:25:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=SK4WUpORe+//Dy7cTe6PZth2LRYQLWVWcWj9UEiAPZ4=; b=PolFRjXEQb0Eft9vtDBgbkHjc24IL7j+cHxKATgPKMkNYIQwBz0v9Q7p8lLE/uMT7B QnOZT0cY67bbBguhMGVAw/KSFN4uaA/acWS+9PNlhOsy6wP9HUuR/PkwvtxDk6fCbsxv BMVUf//UXpYbWEk5Ncwg+cNZD5eIbbQvp45qouKrTxBtumdtRKhSyHRi7NozUYf5sCo5 mhQdteL99RX/IbgBJG0PM5isWKGQJwDIlnBEMZ9KTfokyM0eE+LHGEMAw77JqHB7tBYg sMlK9pNYUlEXRsX1ivDCVWMey1338lYPfbXNPcXdvYX/sx8/GCzOKe0J5ZkMVp0Maekj w7UA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51CRPhYbqex/Q56XUCdpnJXnCdMppHm5NVsj15mV8N5ASo/YEVjr K3ZpsXSBLkqtGoe8+srNoL1DPJVDomgDhD2CAV9d/DPVvP0b6jDER4PNhfsxGlQjWqW20FQRrkc Wi4+KkG0o+qlUdok8Czj+UY7HF8CK6LDvJa1Pe7eVwfJD2TG0oVcuzAHlhnP9WWnqHA== X-Received: by 2002:a37:d282:: with SMTP id f124-v6mr3225050qkj.281.1535138756428; Fri, 24 Aug 2018 12:25:56 -0700 (PDT) X-Google-Smtp-Source: ANB0VdaAlo8JV1qLs8WlLIK2neuMIyMWIfv3imzvIMWlFSqx93qaVWNyGft/+IGUs2BcUwrBD1Ez X-Received: by 2002:a37:d282:: with SMTP id f124-v6mr3225006qkj.281.1535138755821; Fri, 24 Aug 2018 12:25:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535138755; cv=none; d=google.com; s=arc-20160816; b=pq7fldEVGX9MwGF9qTtDHG7yAR4DV79+yzaHul5XImqBfMQjyMBoltAHeNpSclK+FM n0mYjQDaDpqbdqp/9vnKobTGlIrB5UzQlWiDBYGiW9+U/toR86ndUKcN21EIdhYRD0+3 sr6YHpc/A2izpe1jYGRR/+VTmzW/AqBCLAFXaHzWCGD1VlbbPAGARI9P8St4/Gv+MWis hi+yfyog2v6c9AlcR6v2hmgaWvUk9zVmExzaUntye/6yvYFFDkiP8P3VDzcJddoXjH9D onpdZtXPgX8LdchdudvIiW09LUmZhfbb9eRMSdiPUuizuH2MAaMDhBiSHzdUSNq1f3gA vSlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=SK4WUpORe+//Dy7cTe6PZth2LRYQLWVWcWj9UEiAPZ4=; b=J57813wE/oiFjy98GzmwWlNipBlpqfXzDubi51WbyPnIkqu8lwfw7B3AjJLRo7/xaq 9U//yQjwDBF36WIvloltCV1mmJRudkW7CtWFX37TYw2Nf4z7zyiqss9zUMSNHWknlgYe lfTuPClgswepAJpGKMfLTMF3sZPcrBjOyhbLgy1nOo1P2sv01w1d9azUJh/BLVPQ4i9T KkfyzENgoUh3xl0GLLC8u17fVl5xIeVzjxLqDRLVj7ZWocg0eBxC7Hu1cAnIqFxALiTI hA2pbYrBnvQbEEur1s3FC1L6PARFBT0jOCmbjnaCjMnqEcPooM9nbdtUlwKzyZPLjBNi 0zIg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id n57-v6si7926198qtk.212.2018.08.24.12.25.55 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Aug 2018 12:25:55 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 76972402332F; Fri, 24 Aug 2018 19:25:55 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-125.rdu2.redhat.com [10.10.122.125]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0D8572026D6D; Fri, 24 Aug 2018 19:25:55 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Ralph Campbell , John Hubbard Subject: [PATCH 6/7] mm/hmm: invalidate device page table at start of invalidation Date: Fri, 24 Aug 2018 15:25:48 -0400 Message-Id: <20180824192549.30844-7-jglisse@redhat.com> In-Reply-To: <20180824192549.30844-1-jglisse@redhat.com> References: <20180824192549.30844-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Fri, 24 Aug 2018 19:25:55 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Fri, 24 Aug 2018 19:25:55 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jglisse@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Invalidate device page table at start of invalidation and invalidate in progress CPU page table snapshooting at both start and end of any invalidation. This is helpful when device need to dirty page because the device page table report the page as dirty. Dirtying page must happen in the start mmu notifier callback and not in the end one. Signed-off-by: Jérôme Glisse Cc: Ralph Campbell Cc: John Hubbard Cc: Andrew Morton --- include/linux/hmm.h | 2 +- mm/hmm.c | 21 ++++++++++++++------- 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index a7f7600b6bb0..064924bce75c 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -325,7 +325,7 @@ struct hmm_mirror_ops { * synchronous call. */ void (*sync_cpu_device_pagetables)(struct hmm_mirror *mirror, - const struct hmm_update *update); + const struct hmm_update *update); }; /* diff --git a/mm/hmm.c b/mm/hmm.c index debd2f734ab5..6fe31e2bfa1e 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -43,7 +43,6 @@ static const struct mmu_notifier_ops hmm_mmu_notifier_ops; * * @mm: mm struct this HMM struct is bound to * @lock: lock protecting ranges list - * @sequence: we track updates to the CPU page table with a sequence number * @ranges: list of range being snapshotted * @mirrors: list of mirrors for this mm * @mmu_notifier: mmu notifier to track updates to CPU page table @@ -52,7 +51,6 @@ static const struct mmu_notifier_ops hmm_mmu_notifier_ops; struct hmm { struct mm_struct *mm; spinlock_t lock; - atomic_t sequence; struct list_head ranges; struct list_head mirrors; struct mmu_notifier mmu_notifier; @@ -85,7 +83,6 @@ static struct hmm *hmm_register(struct mm_struct *mm) return NULL; INIT_LIST_HEAD(&hmm->mirrors); init_rwsem(&hmm->mirrors_sem); - atomic_set(&hmm->sequence, 0); hmm->mmu_notifier.ops = NULL; INIT_LIST_HEAD(&hmm->ranges); spin_lock_init(&hmm->lock); @@ -126,8 +123,8 @@ void hmm_mm_destroy(struct mm_struct *mm) kfree(mm->hmm); } -static void hmm_invalidate_range(struct hmm *hmm, - const struct hmm_update *update) +static void hmm_invalidate_range(struct hmm *hmm, bool device, + const struct hmm_update *update) { struct hmm_mirror *mirror; struct hmm_range *range; @@ -147,6 +144,9 @@ static void hmm_invalidate_range(struct hmm *hmm, } spin_unlock(&hmm->lock); + if (!device) + return; + down_read(&hmm->mirrors_sem); list_for_each_entry(mirror, &hmm->mirrors, list) mirror->ops->sync_cpu_device_pagetables(mirror, update); @@ -185,11 +185,18 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn, unsigned long end, bool blockable) { + struct hmm_update update; struct hmm *hmm = mm->hmm; + if (!blockable) + return -EAGAIN; + VM_BUG_ON(!hmm); - atomic_inc(&hmm->sequence); + update.start = start; + update.end = end; + update.event = HMM_UPDATE_INVALIDATE; + hmm_invalidate_range(hmm, true, &update); return 0; } @@ -207,7 +214,7 @@ static void hmm_invalidate_range_end(struct mmu_notifier *mn, update.start = start; update.end = end; update.event = HMM_UPDATE_INVALIDATE; - hmm_invalidate_range(hmm, &update); + hmm_invalidate_range(hmm, false, &update); } static const struct mmu_notifier_ops hmm_mmu_notifier_ops = { From patchwork Fri Aug 24 19:25:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10575711 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 82AE5921 for ; Fri, 24 Aug 2018 19:26:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 720B82BA85 for ; Fri, 24 Aug 2018 19:26:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6455D2BAE7; Fri, 24 Aug 2018 19:26:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C39112BA85 for ; Fri, 24 Aug 2018 19:26:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A842F6B3136; Fri, 24 Aug 2018 15:25:57 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 96D7A6B3135; Fri, 24 Aug 2018 15:25:57 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6ACC46B3136; Fri, 24 Aug 2018 15:25:57 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk0-f198.google.com (mail-qk0-f198.google.com [209.85.220.198]) by kanga.kvack.org (Postfix) with ESMTP id 2E0886B3133 for ; Fri, 24 Aug 2018 15:25:57 -0400 (EDT) Received: by mail-qk0-f198.google.com with SMTP id y130-v6so8635507qka.1 for ; Fri, 24 Aug 2018 12:25:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=VsPoJdPYZN3cAurM4nKwcE8vHkfFKNNaomFQl5BOPqE=; b=fUmbXdVsxtzhUS38kdpKZWWQNBQVSR62atvaoJPWTXQNSBQtmqoS4f6k/RA85poEhK PtpjiCDIaklkAfJOLmA0RRVEai9PinComO/jSNfORq4fZFkmYeLaIOkOf2x2uctsQ19t s/Pg1c9PVtYU5uDeu2/AvXg/HlySKjgQNw+iPOo7nu42X+epKX9PB6SCK4Y85mBm9wws fMjhdN3jdpGOmWYQXLxsVxk6pFBIAhe66Sfppb6lXZH68uSf3/IWzidQxy2bNVOUMfx0 SKIzLKhXmctJhjvxQBCQzkPdlK6w1dgwQzoG++rfgeZolpm5CDzvEKVgf+/+uv8Pq3RU wRcw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51CrMfMxeWXeAm5+RBR70Y9LYelq1iHT9Ut+QSpzhzV8sPOHYNBA 87MtY3YdzoK0RQUTgZm+BjINYKaQFcQRf3gN8iYdpdSdIOSZrYiRRQB2pjZCNQI/o79oHptvJvm TKyxHcyV4a2meYPAqhSbaCzZpbAHUDJaJ48ygDvjb1FdCqdTGdt+4AN+SKvMcnqJVJw== X-Received: by 2002:ac8:427:: with SMTP id v39-v6mr3226459qtg.120.1535138756952; Fri, 24 Aug 2018 12:25:56 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbYJg5tMOSTz9ULn4u/sNdIPEo1A64/7vlsl56vD0hWle0fRW4sIPAxwDF/SCrWolObsj+8 X-Received: by 2002:ac8:427:: with SMTP id v39-v6mr3226424qtg.120.1535138756368; Fri, 24 Aug 2018 12:25:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535138756; cv=none; d=google.com; s=arc-20160816; b=vohvGyQwky2GyxZXS+7A23htAzG50noEvnd6hqWBa7HWKMvqW7k5tqffVz7/h0aZmn KFrPGl7wNoHIRZb2iC2XXGslue/gAmBFrQtO5BL8PjaO8vOC0R0E0cLgzmBZ2zOHDO7F 5dWrLfKYuVxpbvBHS20lwHBirLprZCftfofDdEhqFFjrgcW0HB3DbBHGrr6G6HLSQxFg iq8NFZ6VCGf7COuP3ej5VWppYEgNkPlwt0iIrk+lt0MHTFSf/jJvHKyKUqYL9UFn8Bo1 r6Ex7FKGHPwKzWYqZLh5VRvjhGK+TtmUkrVynG/GyUAnRxw2SRC6cL8NyvVqeGMm+ykw M6zA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=VsPoJdPYZN3cAurM4nKwcE8vHkfFKNNaomFQl5BOPqE=; b=Jn/Ybp6QO8gnRMVdooSwX446ZZ2MgRYyUGOIKn6Vkwl7uqdpJZT/0dVbfsIxouAk/e m64RfJn5aut4MhaKJhsw7gbq6ynnTul4qbOQ/oM9i3+TsQ2pTAL4ATefD/nLTZPLyfYY 82IVqXKqyQ35cUD0qRUGFUDfTsgZgN0lL4buS8VZzCNGKINzMlQo1MmP3V7mHwpk04oz jxgchVivTluaIuftQMOoM/uZKhum7SHTBYDpkk8gsUkR19deQQcgYiWPp35QlsDZLEBI SzCXCJIbdG60yRbHmW71H5aaND/DloaRKH2ewiAwdNMqvx2KYU3p5CT8+Sz5Os0zSQZr JPIQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id j18-v6si4515277qtj.294.2018.08.24.12.25.56 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Aug 2018 12:25:56 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0B0564023475; Fri, 24 Aug 2018 19:25:56 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-125.rdu2.redhat.com [10.10.122.125]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8AF432026D6D; Fri, 24 Aug 2018 19:25:55 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Michal Hocko , Ralph Campbell , John Hubbard Subject: [PATCH 7/7] mm/hmm: proper support for blockable mmu_notifier Date: Fri, 24 Aug 2018 15:25:49 -0400 Message-Id: <20180824192549.30844-8-jglisse@redhat.com> In-Reply-To: <20180824192549.30844-1-jglisse@redhat.com> References: <20180824192549.30844-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Fri, 24 Aug 2018 19:25:56 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Fri, 24 Aug 2018 19:25:56 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jglisse@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse When mmu_notifier calls invalidate_range_start callback with blockable set to false we should not sleep. Properly propagate this to HMM users. Signed-off-by: Jérôme Glisse Cc: Michal Hocko Cc: Ralph Campbell Cc: John Hubbard Cc: Andrew Morton --- include/linux/hmm.h | 12 +++++++++--- mm/hmm.c | 39 ++++++++++++++++++++++++++++----------- 2 files changed, 37 insertions(+), 14 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 064924bce75c..c783916f8732 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -287,11 +287,13 @@ enum hmm_update_event { * @start: virtual start address of the range to update * @end: virtual end address of the range to update * @event: event triggering the update (what is happening) + * @blockable: can the callback block/sleep ? */ struct hmm_update { unsigned long start; unsigned long end; enum hmm_update_event event; + bool blockable; }; /* @@ -314,6 +316,8 @@ struct hmm_mirror_ops { * * @mirror: pointer to struct hmm_mirror * @update: update informations (see struct hmm_update) + * Returns: -EAGAIN if update.blockable false and callback need to + * block, 0 otherwise. * * This callback ultimately originates from mmu_notifiers when the CPU * page table is updated. The device driver must update its page table @@ -322,10 +326,12 @@ struct hmm_mirror_ops { * * The device driver must not return from this callback until the device * page tables are completely updated (TLBs flushed, etc); this is a - * synchronous call. + * synchronous call. If driver need to sleep and update->blockable is + * false then you need to abort (do not do anything that would sleep or + * block) and return -EAGAIN. */ - void (*sync_cpu_device_pagetables)(struct hmm_mirror *mirror, - const struct hmm_update *update); + int (*sync_cpu_device_pagetables)(struct hmm_mirror *mirror, + const struct hmm_update *update); }; /* diff --git a/mm/hmm.c b/mm/hmm.c index 6fe31e2bfa1e..1d8fcaa0606f 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -123,12 +123,18 @@ void hmm_mm_destroy(struct mm_struct *mm) kfree(mm->hmm); } -static void hmm_invalidate_range(struct hmm *hmm, bool device, - const struct hmm_update *update) +static int hmm_invalidate_range(struct hmm *hmm, bool device, + const struct hmm_update *update) { struct hmm_mirror *mirror; struct hmm_range *range; + /* + * It is fine to wait on lock here even if update->blockable is false + * as the hmm->lock is only held for short period of time (when adding + * or walking the ranges list). We could also convert the range list + * into a lru list and avoid the spinlock all together. + */ spin_lock(&hmm->lock); list_for_each_entry(range, &hmm->ranges, list) { unsigned long addr, idx, npages; @@ -145,12 +151,26 @@ static void hmm_invalidate_range(struct hmm *hmm, bool device, spin_unlock(&hmm->lock); if (!device) - return; + return 0; + /* + * It is fine to wait on mirrors_sem here even if update->blockable is + * false as this semaphore is only taken in write mode for short period + * when adding a new mirror to the list. + */ down_read(&hmm->mirrors_sem); - list_for_each_entry(mirror, &hmm->mirrors, list) - mirror->ops->sync_cpu_device_pagetables(mirror, update); + list_for_each_entry(mirror, &hmm->mirrors, list) { + int ret; + + ret = mirror->ops->sync_cpu_device_pagetables(mirror, update); + if (!update->blockable && ret == -EAGAIN) { + up_read(&hmm->mirrors_sem); + return -EAGAIN; + } + } up_read(&hmm->mirrors_sem); + + return 0; } static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) @@ -188,17 +208,13 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn, struct hmm_update update; struct hmm *hmm = mm->hmm; - if (!blockable) - return -EAGAIN; - VM_BUG_ON(!hmm); update.start = start; update.end = end; update.event = HMM_UPDATE_INVALIDATE; - hmm_invalidate_range(hmm, true, &update); - - return 0; + update.blockable = blockable; + return hmm_invalidate_range(hmm, true, &update); } static void hmm_invalidate_range_end(struct mmu_notifier *mn, @@ -214,6 +230,7 @@ static void hmm_invalidate_range_end(struct mmu_notifier *mn, update.start = start; update.end = end; update.event = HMM_UPDATE_INVALIDATE; + update.blockable = true; hmm_invalidate_range(hmm, false, &update); }