From patchwork Wed Jan 23 22:23:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10777971 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8DF491390 for ; Wed, 23 Jan 2019 22:23:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7FDCF2D368 for ; Wed, 23 Jan 2019 22:23:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 738D82D396; Wed, 23 Jan 2019 22:23:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E4E9C2D368 for ; Wed, 23 Jan 2019 22:23:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4699C8E004F; Wed, 23 Jan 2019 17:23:34 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 415DB8E0047; Wed, 23 Jan 2019 17:23:34 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3048B8E004F; Wed, 23 Jan 2019 17:23:34 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by kanga.kvack.org (Postfix) with ESMTP id 091588E0047 for ; Wed, 23 Jan 2019 17:23:34 -0500 (EST) Received: by mail-qt1-f198.google.com with SMTP id n50so4350077qtb.9 for ; Wed, 23 Jan 2019 14:23:34 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Ah2At7Z9WHVesdrMLiCK0xNMPj6Yxfh/2asilkzerWc=; b=Pb9RDlBtJPydFo6h/9MFX0r2ufEzL3MNK7iaWpgu8ob6F/IiGt4xk1pfDMDrQHnDVE 9pKKqHffvsuCH+eIo7P13KLDuWlAY3udkpP6ndR2kV4yGkzpe4M1Vmww1uYAmAiXtQXs XEkZxTkZWTEf33FJpKBNjiq0jF6b+iVKkCozagC3qt0CE+dIOB0TyfjjnCrYbY3NJU0n ojY9Bby0/tGbAdMrVj2o2nQIq5wSvq/ohGjuI1XOcLSB2VX/Otd3qZjNSQTrHFiY0s9D 1EIwIqoiGPKVEyGQPa1j1URXSCocTO3725ANXYzwr0BbYz7BclTDspjBvjeX68IhG2yK kfJQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukeztNmcS51rbc+6RoBxb/CuKmpXDXhNpCE6jjw2Pkn0/ri+YEhQ VnFNXT6rcxJ9oNtT2s4wFzOBD5gb4KC8qokL52RvJ25rat49mmOPnt02bpvFLqUBq6eWuqpEJf2 9GbJOIzILDEIuZ9HmKvW3a5y1SHlmp35np5mTXD7jvI5sc8DF32NUrFXh8RPxwQDOdg== X-Received: by 2002:a37:7cc5:: with SMTP id x188mr3308050qkc.307.1548282213748; Wed, 23 Jan 2019 14:23:33 -0800 (PST) X-Google-Smtp-Source: ALg8bN5E4gKbBW6irRY6OSWxxadsG8bMC5LBjRtVg8fGKfaceEMegsaJGpvKJvxzKySh5fe1DxRs X-Received: by 2002:a37:7cc5:: with SMTP id x188mr3308032qkc.307.1548282213248; Wed, 23 Jan 2019 14:23:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548282213; cv=none; d=google.com; s=arc-20160816; b=EFnEys3Lfl39CCbhpgZlgBnSaqziYb5OXTmgkak3AZNTF8kgmlgntCydlqIcdhrEfq zF7AMXcUNQOAUo4bqg+C5eWfFsk3FYvi3qejEryht6+5UXXKHMVH/0gUAhQGKkmUjwSY m4NSikmtVMfuJNrb6K9gbj9uRkTyiIs0+gE4a8s8oSpnlcoBSji/Mjm0RnoQtAgSFvF0 JRU/GeuDNV00tvgWwDFu7woDCq3RdsW6edWvPDRVXkOUXjrrhkqX+n3l9KHEob6iX4/s rFRqCIqahifXxQ6ZUERPQ8oFyQ8cHlIoEAZPVwA0/K4AZEkQ+R8vrn7nE7gHpv2QkH3m IP0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=Ah2At7Z9WHVesdrMLiCK0xNMPj6Yxfh/2asilkzerWc=; b=k+7ckw0Smrsh8dwKV19ruTKAC5yYtmM+Rdw0SjJ2bHZoFC0RYCCiD6bJq3u+/veACH dIfgeS30T7ALxNAQ77ZesSEUO6GSdzPkFvo7bX6XBF7L4Kl51Y/CltLnlYyhs0xXPm8U 02zZ4XCJaZPL6RA5rbUqUhNPDfheBf9vIRxwaH6iBoKEVPWIvvKBxs37v9Vn0+xNV7tx kQ1Yf+h6/CjSxkbpel918GDBUzp52bDhZhecDoPlUgfdbgz0whgUfvMPBpS9KsLmjJJI t2+6eYsRFyH+aBbzTzJ3QXeimv8aojTmaqR4rFFBrZE6Pi5Mw/SjN8unqOlOFjnqdphR 8etw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id r65si8550397qtd.301.2019.01.23.14.23.33 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jan 2019 14:23:33 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 20E67461FF; Wed, 23 Jan 2019 22:23:32 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-127.rdu2.redhat.com [10.10.120.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7FF865D964; Wed, 23 Jan 2019 22:23:29 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Jan Kara , Felix Kuehling , Jason Gunthorpe , Matthew Wilcox , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-fsdevel@vger.kernel.org, Arnd Bergmann Subject: [PATCH v4 1/9] mm/mmu_notifier: contextual information for event enums Date: Wed, 23 Jan 2019 17:23:07 -0500 Message-Id: <20190123222315.1122-2-jglisse@redhat.com> In-Reply-To: <20190123222315.1122-1-jglisse@redhat.com> References: <20190123222315.1122-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 23 Jan 2019 22:23:32 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse CPU page table update can happens for many reasons, not only as a result of a syscall (munmap(), mprotect(), mremap(), madvise(), ...) but also as a result of kernel activities (memory compression, reclaim, migration, ...). This patch introduce a set of enums that can be associated with each of the events triggering a mmu notifier. Latter patches take advantages of those enum values. - UNMAP: munmap() or mremap() - CLEAR: page table is cleared (migration, compaction, reclaim, ...) - PROTECTION_VMA: change in access protections for the range - PROTECTION_PAGE: change in access protections for page in the range - SOFT_DIRTY: soft dirtyness tracking Being able to identify munmap() and mremap() from other reasons why the page table is cleared is important to allow user of mmu notifier to update their own internal tracking structure accordingly (on munmap or mremap it is not longer needed to track range of virtual address as it becomes invalid). Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Jan Kara Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Andrew Morton Cc: Matthew Wilcox Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org Cc: Arnd Bergmann --- include/linux/mmu_notifier.h | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 4050ec1c3b45..abc9dbb7bcb6 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -10,6 +10,36 @@ struct mmu_notifier; struct mmu_notifier_ops; +/** + * enum mmu_notifier_event - reason for the mmu notifier callback + * @MMU_NOTIFY_UNMAP: either munmap() that unmap the range or a mremap() that + * move the range + * + * @MMU_NOTIFY_CLEAR: clear page table entry (many reasons for this like + * madvise() or replacing a page by another one, ...). + * + * @MMU_NOTIFY_PROTECTION_VMA: update is due to protection change for the range + * ie using the vma access permission (vm_page_prot) to update the whole range + * is enough no need to inspect changes to the CPU page table (mprotect() + * syscall) + * + * @MMU_NOTIFY_PROTECTION_PAGE: update is due to change in read/write flag for + * pages in the range so to mirror those changes the user must inspect the CPU + * page table (from the end callback). + * + * @MMU_NOTIFY_SOFT_DIRTY: soft dirty accounting (still same page and same + * access flags). User should soft dirty the page in the end callback to make + * sure that anyone relying on soft dirtyness catch pages that might be written + * through non CPU mappings. + */ +enum mmu_notifier_event { + MMU_NOTIFY_UNMAP = 0, + MMU_NOTIFY_CLEAR, + MMU_NOTIFY_PROTECTION_VMA, + MMU_NOTIFY_PROTECTION_PAGE, + MMU_NOTIFY_SOFT_DIRTY, +}; + #ifdef CONFIG_MMU_NOTIFIER /* From patchwork Wed Jan 23 22:23:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10777977 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 04E7A1515 for ; Wed, 23 Jan 2019 22:23:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EA8522D368 for ; Wed, 23 Jan 2019 22:23:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DAA412D396; Wed, 23 Jan 2019 22:23:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 24D6B2D396 for ; Wed, 23 Jan 2019 22:23:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D9FB8E0050; Wed, 23 Jan 2019 17:23:37 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 662D18E0047; Wed, 23 Jan 2019 17:23:37 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 578B68E0050; Wed, 23 Jan 2019 17:23:37 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id 1F90D8E0047 for ; Wed, 23 Jan 2019 17:23:37 -0500 (EST) Received: by mail-qk1-f197.google.com with SMTP id p79so3455816qki.15 for ; Wed, 23 Jan 2019 14:23:37 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=X3O+qlY+5uwmNqjKCr+nXDdC5u6h1pEcYFo+8J+p5Co=; b=NA+v2Zbuq/8nZQmpUR1Pu1M000N2O42hkOxc1rUMJuSEo9hMci4XSHdMMrlgXRvjX5 8jA1g8k6rpM38Kkwn1XRvKG3EABPWjxF05NxEt9+rrkc+nrGSo+FnQ4gn5vMFzrDqLd6 Lrahe0dJudYD2WHrOct0irUnKS47Lww4X5rLGp8ZlXrS7rJplhAipORbAG74jCY+Cou4 tfgR9Pg/SoQoK2HF6k/vOKK+Wz9htcMNrT4HN8rXcMGg8aU58ramaXY2FB5ibF2B0/Qo ihhclk6B2Jx5Qnr/XPdPBPN+WlzeI07kCE3cSakBrabmYpUg4dbF3YgYpOO6ilExPFYF bMpA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukcExhingo2CdeSzl0sHHavS5u8NT7H1NgR5x3BvtNA09O2vvJYh AssoLAZx5kZTREYtUTtS9WwlMJlWVeI83NOg0XljrJjHLn19Vj4PJn9/pYHNUYpamRUbTTOe7Lk LAtCUdo37gTsRlVEWlW39KHjKh3QONs4rEfcVyYbgw0YzM4BeCn8foV8mD/8Q6yUFoQ== X-Received: by 2002:ac8:34cb:: with SMTP id x11mr4289049qtb.115.1548282216861; Wed, 23 Jan 2019 14:23:36 -0800 (PST) X-Google-Smtp-Source: ALg8bN7OgvPEbl3kDFzcE22Ju2LbXfVoHSt4PJntl/AVtIQMya8Ydh6KVhNyIa9XJTWeeD8DIDkf X-Received: by 2002:ac8:34cb:: with SMTP id x11mr4289018qtb.115.1548282216048; Wed, 23 Jan 2019 14:23:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548282216; cv=none; d=google.com; s=arc-20160816; b=stz5eqY+3GqnLIjHLBu3WS1Ce2nU2nigfXvajOF8nhO0AW5gCRMoZ80FBwMovF6Snj On3NGmBZmtK3ptCWbjqcmcdpZhB5Ei03lT6uUm708nqwrZuhjpaTAnM76QnyePk36qx+ X31Hz99WR2/405KU4upv5aIb1nxfdhu1Hs0urfvUzgXVrh/pXXHucYTpwgye8qIfDx+E Bw/+ao2pQwao/8P806mEloQUU5eaSBUcp6FjNRlcENZpRSyw6vvPKzN2i3Ca1PaG7dyz 3kr/8qL7Ha9T/SHXyHEQVcEn+u1Zg1+V8eGknsE5SIZc0yNBoQagJyF3BCps1e8RKGZW XXBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=X3O+qlY+5uwmNqjKCr+nXDdC5u6h1pEcYFo+8J+p5Co=; b=NqT5sfFV5xRyDVGNEA27LfzazD8SsHeBgpZofdZW/DZV0EDkiSYpN/6SAtn+xKLCXZ p2sTPIZ7MQdlHTtDLGj+Xd43lcvyHBFEPH/o/2GN/9pH+sFWuvtWQPXmRa9EHW5iMlx3 XnDgVtkYRefFFpjO3CZGCE4W1coqL0BYfNWE4lEmoTruSkzNqrWJgnqc+ifOm0UpVCqw x1gMwhl9Uwn7DcGpfB0pGq7R9j/acr3P+aZxNOqefCgpRBL48n72lv71+wY1jgmwK+OB jWU8u4IL7qoRFmMag1oagg0p2eCjoz5YLRXydCQ5MgqB8/AlgPbusOMmnTApFF2CpS79 lGDA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id n15si1439233qtl.46.2019.01.23.14.23.35 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jan 2019 14:23:36 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 19010A7880; Wed, 23 Jan 2019 22:23:35 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-127.rdu2.redhat.com [10.10.120.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4DD495D965; Wed, 23 Jan 2019 22:23:32 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Jan Kara , Felix Kuehling , Jason Gunthorpe , Matthew Wilcox , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-fsdevel@vger.kernel.org, Arnd Bergmann Subject: [PATCH v4 2/9] mm/mmu_notifier: contextual information for event triggering invalidation Date: Wed, 23 Jan 2019 17:23:08 -0500 Message-Id: <20190123222315.1122-3-jglisse@redhat.com> In-Reply-To: <20190123222315.1122-1-jglisse@redhat.com> References: <20190123222315.1122-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 23 Jan 2019 22:23:35 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse CPU page table update can happens for many reasons, not only as a result of a syscall (munmap(), mprotect(), mremap(), madvise(), ...) but also as a result of kernel activities (memory compression, reclaim, migration, ...). Users of mmu notifier API track changes to the CPU page table and take specific action for them. While current API only provide range of virtual address affected by the change, not why the changes is happening. This patchset do the initial mechanical convertion of all the places that calls mmu_notifier_range_init to also provide the default MMU_NOTIFY_UNMAP event as well as the vma if it is know (most invalidation happens against a given vma). Passing down the vma allows the users of mmu notifier to inspect the new vma page protection. The MMU_NOTIFY_UNMAP is always the safe default as users of mmu notifier should assume that every for the range is going away when that event happens. A latter patch do convert mm call path to use a more appropriate events for each call. This is done as 2 patches so that no call site is forgotten especialy as it uses this following coccinelle patch: %<---------------------------------------------------------------------- @@ identifier I1, I2, I3, I4; @@ static inline void mmu_notifier_range_init(struct mmu_notifier_range *I1, +enum mmu_notifier_event event, +struct vm_area_struct *vma, struct mm_struct *I2, unsigned long I3, unsigned long I4) { ... } @@ @@ -#define mmu_notifier_range_init(range, mm, start, end) +#define mmu_notifier_range_init(range, event, vma, mm, start, end) @@ expression E1, E3, E4; identifier I1; @@ <... mmu_notifier_range_init(E1, +MMU_NOTIFY_UNMAP, I1, I1->vm_mm, E3, E4) ...> @@ expression E1, E2, E3, E4; identifier FN, VMA; @@ FN(..., struct vm_area_struct *VMA, ...) { <... mmu_notifier_range_init(E1, +MMU_NOTIFY_UNMAP, VMA, E2, E3, E4) ...> } @@ expression E1, E2, E3, E4; identifier FN, VMA; @@ FN(...) { struct vm_area_struct *VMA; <... mmu_notifier_range_init(E1, +MMU_NOTIFY_UNMAP, VMA, E2, E3, E4) ...> } @@ expression E1, E2, E3, E4; identifier FN; @@ FN(...) { <... mmu_notifier_range_init(E1, +MMU_NOTIFY_UNMAP, NULL, E2, E3, E4) ...> } ---------------------------------------------------------------------->% Applied with: spatch --all-includes --sp-file mmu-notifier.spatch fs/proc/task_mmu.c --in-place spatch --sp-file mmu-notifier.spatch --dir kernel/events/ --in-place spatch --sp-file mmu-notifier.spatch --dir mm --in-place Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Jan Kara Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Andrew Morton Cc: Matthew Wilcox Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org Cc: Arnd Bergmann --- fs/proc/task_mmu.c | 3 ++- include/linux/mmu_notifier.h | 4 +++- kernel/events/uprobes.c | 3 ++- mm/huge_memory.c | 12 ++++++++---- mm/hugetlb.c | 10 ++++++---- mm/khugepaged.c | 3 ++- mm/ksm.c | 6 ++++-- mm/madvise.c | 3 ++- mm/memory.c | 25 ++++++++++++++++--------- mm/migrate.c | 5 ++++- mm/mprotect.c | 3 ++- mm/mremap.c | 3 ++- mm/oom_kill.c | 3 ++- mm/rmap.c | 6 ++++-- 14 files changed, 59 insertions(+), 30 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index f0ec9edab2f3..57e7f98647d3 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1143,7 +1143,8 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf, break; } - mmu_notifier_range_init(&range, mm, 0, -1UL); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, + NULL, mm, 0, -1UL); mmu_notifier_invalidate_range_start(&range); } walk_page_range(0, mm->highest_vm_end, &clear_refs_walk); diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index abc9dbb7bcb6..a9808add4070 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -348,6 +348,8 @@ static inline void mmu_notifier_mm_destroy(struct mm_struct *mm) static inline void mmu_notifier_range_init(struct mmu_notifier_range *range, + enum mmu_notifier_event event, + struct vm_area_struct *vma, struct mm_struct *mm, unsigned long start, unsigned long end) @@ -482,7 +484,7 @@ static inline void _mmu_notifier_range_init(struct mmu_notifier_range *range, range->end = end; } -#define mmu_notifier_range_init(range, mm, start, end) \ +#define mmu_notifier_range_init(range,event,vma,mm,start,end) \ _mmu_notifier_range_init(range, start, end) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 8aef47ee7bfa..b67fe7e59621 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -174,7 +174,8 @@ static int __replace_page(struct vm_area_struct *vma, unsigned long addr, struct mmu_notifier_range range; struct mem_cgroup *memcg; - mmu_notifier_range_init(&range, mm, addr, addr + PAGE_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, addr, + addr + PAGE_SIZE); VM_BUG_ON_PAGE(PageTransHuge(old_page), old_page); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index faf357eaf0ce..b353e8b7876f 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1182,7 +1182,8 @@ static vm_fault_t do_huge_pmd_wp_page_fallback(struct vm_fault *vmf, cond_resched(); } - mmu_notifier_range_init(&range, vma->vm_mm, haddr, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + haddr, haddr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -1345,7 +1346,8 @@ vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd) vma, HPAGE_PMD_NR); __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, vma->vm_mm, haddr, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + haddr, haddr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -2023,7 +2025,8 @@ void __split_huge_pud(struct vm_area_struct *vma, pud_t *pud, spinlock_t *ptl; struct mmu_notifier_range range; - mmu_notifier_range_init(&range, vma->vm_mm, address & HPAGE_PUD_MASK, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + address & HPAGE_PUD_MASK, (address & HPAGE_PUD_MASK) + HPAGE_PUD_SIZE); mmu_notifier_invalidate_range_start(&range); ptl = pud_lock(vma->vm_mm, pud); @@ -2241,7 +2244,8 @@ void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, spinlock_t *ptl; struct mmu_notifier_range range; - mmu_notifier_range_init(&range, vma->vm_mm, address & HPAGE_PMD_MASK, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + address & HPAGE_PMD_MASK, (address & HPAGE_PMD_MASK) + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); ptl = pmd_lock(vma->vm_mm, pmd); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index df2e7dd5ff17..cbda46ad6a30 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3246,7 +3246,8 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, cow = (vma->vm_flags & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE; if (cow) { - mmu_notifier_range_init(&range, src, vma->vm_start, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, src, + vma->vm_start, vma->vm_end); mmu_notifier_invalidate_range_start(&range); } @@ -3358,7 +3359,7 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, /* * If sharing possible, alert mmu notifiers of worst case. */ - mmu_notifier_range_init(&range, mm, start, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, start, end); adjust_range_if_pmd_sharing_possible(vma, &range.start, &range.end); mmu_notifier_invalidate_range_start(&range); address = start; @@ -3626,7 +3627,8 @@ static vm_fault_t hugetlb_cow(struct mm_struct *mm, struct vm_area_struct *vma, __SetPageUptodate(new_page); set_page_huge_active(new_page); - mmu_notifier_range_init(&range, mm, haddr, haddr + huge_page_size(h)); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, haddr, + haddr + huge_page_size(h)); mmu_notifier_invalidate_range_start(&range); /* @@ -4346,7 +4348,7 @@ unsigned long hugetlb_change_protection(struct vm_area_struct *vma, * start/end. Set range.start/range.end to cover the maximum possible * range if PMD sharing is possible. */ - mmu_notifier_range_init(&range, mm, start, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, start, end); adjust_range_if_pmd_sharing_possible(vma, &range.start, &range.end); BUG_ON(address >= end); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 4f017339ddb2..f903acb1b0a6 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1016,7 +1016,8 @@ static void collapse_huge_page(struct mm_struct *mm, pte = pte_offset_map(pmd, address); pte_ptl = pte_lockptr(mm, pmd); - mmu_notifier_range_init(&range, mm, address, address + HPAGE_PMD_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, NULL, mm, address, + address + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); pmd_ptl = pmd_lock(mm, pmd); /* probably unnecessary */ /* diff --git a/mm/ksm.c b/mm/ksm.c index 6c48ad13b4c9..6b27c4f0fb1f 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1051,7 +1051,8 @@ static int write_protect_page(struct vm_area_struct *vma, struct page *page, BUG_ON(PageTransCompound(page)); - mmu_notifier_range_init(&range, mm, pvmw.address, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, + pvmw.address, pvmw.address + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -1139,7 +1140,8 @@ static int replace_page(struct vm_area_struct *vma, struct page *page, if (!pmd) goto out; - mmu_notifier_range_init(&range, mm, addr, addr + PAGE_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, addr, + addr + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); ptep = pte_offset_map_lock(mm, pmd, addr, &ptl); diff --git a/mm/madvise.c b/mm/madvise.c index 21a7881a2db4..04446dabba56 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -472,7 +472,8 @@ static int madvise_free_single_vma(struct vm_area_struct *vma, range.end = min(vma->vm_end, end_addr); if (range.end <= vma->vm_start) return -EINVAL; - mmu_notifier_range_init(&range, mm, range.start, range.end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, + range.start, range.end); lru_add_drain(); tlb_gather_mmu(&tlb, mm, range.start, range.end); diff --git a/mm/memory.c b/mm/memory.c index e11ca9dd823f..d9b7c935e812 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1009,7 +1009,8 @@ int copy_page_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, is_cow = is_cow_mapping(vma->vm_flags); if (is_cow) { - mmu_notifier_range_init(&range, src_mm, addr, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, src_mm, + addr, end); mmu_notifier_invalidate_range_start(&range); } @@ -1333,7 +1334,8 @@ void unmap_vmas(struct mmu_gather *tlb, { struct mmu_notifier_range range; - mmu_notifier_range_init(&range, vma->vm_mm, start_addr, end_addr); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + start_addr, end_addr); mmu_notifier_invalidate_range_start(&range); for ( ; vma && vma->vm_start < end_addr; vma = vma->vm_next) unmap_single_vma(tlb, vma, start_addr, end_addr, NULL); @@ -1355,7 +1357,8 @@ void zap_page_range(struct vm_area_struct *vma, unsigned long start, struct mmu_gather tlb; lru_add_drain(); - mmu_notifier_range_init(&range, vma->vm_mm, start, start + size); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + start, start + size); tlb_gather_mmu(&tlb, vma->vm_mm, start, range.end); update_hiwater_rss(vma->vm_mm); mmu_notifier_invalidate_range_start(&range); @@ -1381,7 +1384,8 @@ static void zap_page_range_single(struct vm_area_struct *vma, unsigned long addr struct mmu_gather tlb; lru_add_drain(); - mmu_notifier_range_init(&range, vma->vm_mm, address, address + size); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + address, address + size); tlb_gather_mmu(&tlb, vma->vm_mm, address, range.end); update_hiwater_rss(vma->vm_mm); mmu_notifier_invalidate_range_start(&range); @@ -2271,7 +2275,8 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, mm, vmf->address & PAGE_MASK, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, + vmf->address & PAGE_MASK, (vmf->address & PAGE_MASK) + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -4081,8 +4086,9 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, goto out; if (range) { - mmu_notifier_range_init(range, mm, address & PMD_MASK, - (address & PMD_MASK) + PMD_SIZE); + mmu_notifier_range_init(range, MMU_NOTIFY_UNMAP, NULL, + mm, address & PMD_MASK, + (address & PMD_MASK) + PMD_SIZE); mmu_notifier_invalidate_range_start(range); } *ptlp = pmd_lock(mm, pmd); @@ -4099,8 +4105,9 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, goto out; if (range) { - mmu_notifier_range_init(range, mm, address & PAGE_MASK, - (address & PAGE_MASK) + PAGE_SIZE); + mmu_notifier_range_init(range, MMU_NOTIFY_UNMAP, NULL, mm, + address & PAGE_MASK, + (address & PAGE_MASK) + PAGE_SIZE); mmu_notifier_invalidate_range_start(range); } ptep = pte_offset_map_lock(mm, pmd, address, ptlp); diff --git a/mm/migrate.c b/mm/migrate.c index a16b15090df3..385c59d5c28d 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2342,7 +2342,8 @@ static void migrate_vma_collect(struct migrate_vma *migrate) mm_walk.mm = migrate->vma->vm_mm; mm_walk.private = migrate; - mmu_notifier_range_init(&range, mm_walk.mm, migrate->start, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, NULL, mm_walk.mm, + migrate->start, migrate->end); mmu_notifier_invalidate_range_start(&range); walk_page_range(migrate->start, migrate->end, &mm_walk); @@ -2750,6 +2751,8 @@ static void migrate_vma_pages(struct migrate_vma *migrate) notified = true; mmu_notifier_range_init(&range, + MMU_NOTIFY_UNMAP, + NULL, migrate->vma->vm_mm, addr, migrate->end); mmu_notifier_invalidate_range_start(&range); diff --git a/mm/mprotect.c b/mm/mprotect.c index 36cb358db170..b22e660701f1 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -185,7 +185,8 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma, /* invoke the mmu notifier if the pmd is populated */ if (!range.start) { - mmu_notifier_range_init(&range, vma->vm_mm, addr, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, + vma->vm_mm, addr, end); mmu_notifier_invalidate_range_start(&range); } diff --git a/mm/mremap.c b/mm/mremap.c index 3320616ed93f..cac19c1e0af4 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -249,7 +249,8 @@ unsigned long move_page_tables(struct vm_area_struct *vma, old_end = old_addr + len; flush_cache_range(vma, old_addr, old_end); - mmu_notifier_range_init(&range, vma->vm_mm, old_addr, old_end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + old_addr, old_end); mmu_notifier_invalidate_range_start(&range); for (; old_addr < old_end; old_addr += extent, new_addr += extent) { diff --git a/mm/oom_kill.c b/mm/oom_kill.c index f0e8cd9edb1a..77289bc6a943 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -531,7 +531,8 @@ bool __oom_reap_task_mm(struct mm_struct *mm) struct mmu_notifier_range range; struct mmu_gather tlb; - mmu_notifier_range_init(&range, mm, vma->vm_start, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, + mm, vma->vm_start, vma->vm_end); tlb_gather_mmu(&tlb, mm, range.start, range.end); if (mmu_notifier_invalidate_range_start_nonblock(&range)) { diff --git a/mm/rmap.c b/mm/rmap.c index 0454ecc29537..49c75f0c6c33 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -896,7 +896,8 @@ static bool page_mkclean_one(struct page *page, struct vm_area_struct *vma, * We have to assume the worse case ie pmd for invalidation. Note that * the page can not be free from this function. */ - mmu_notifier_range_init(&range, vma->vm_mm, address, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + address, min(vma->vm_end, address + (PAGE_SIZE << compound_order(page)))); mmu_notifier_invalidate_range_start(&range); @@ -1371,7 +1372,8 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, * Note that the page can not be free in this function as call of * try_to_unmap() must hold a reference on the page. */ - mmu_notifier_range_init(&range, vma->vm_mm, address, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + address, min(vma->vm_end, address + (PAGE_SIZE << compound_order(page)))); if (PageHuge(page)) { From patchwork Wed Jan 23 22:23:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10777983 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 23B851399 for ; Wed, 23 Jan 2019 22:23:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 157BD2D368 for ; Wed, 23 Jan 2019 22:23:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 094E22D396; Wed, 23 Jan 2019 22:23:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 018772D368 for ; Wed, 23 Jan 2019 22:23:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E7238E0051; Wed, 23 Jan 2019 17:23:40 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 240D08E0047; Wed, 23 Jan 2019 17:23:40 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0BD478E0051; Wed, 23 Jan 2019 17:23:40 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id CEFB98E0047 for ; Wed, 23 Jan 2019 17:23:39 -0500 (EST) Received: by mail-qk1-f198.google.com with SMTP id y27so3418236qkj.21 for ; Wed, 23 Jan 2019 14:23:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=povx2BgWdKWZnT/gMz3YB3oo7NPWl/RpagLy1A16ZvY=; b=dFHIfZ9ZQm+UwLlIMIXWmfVx4ROvXGVOoIF520z2p5BSGkrrHasuGq4ifwOuwb1M5m hK45BLXUvlbj4cP7LrAQX0Xcp0n3TgeeGBs//C27dFqkZNf/JCGXPaTEunPNPtaIQpqm 8OhxzY8WoRrA8zM2mP5eStANJhFU+k3vOHo5MFKTgjfAtW+xpE5vGamqgdxbe3fFr0yY M0xlwEmakuiir1fGv12CZkNLS8FvHwzv3Dls7G6pV7UHEyPZho72JuTuB8FlxosLSQzO N8P21OJVMpjrdMkGHWnzzeUK/1EwCB2B6YeaHsIoSF5P5SMs16HGGVkBjsLkNPMXXqjJ jDvA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukclcC+R15QeadAJzherZs3U6vcOKzOa3ZC9eTNZRpKzs5FZS5N3 0vai7TuGT4n2LW/iDL68V9Y9ewwBbwwCVMH9G3+LpRT+7rED5iNXmitARLiXmqJIglq/pFi+9I8 3ompFkkHMs1tNbxdt9xO+fLZyCWHWlewD/lMeUgiGZh4HJKghH75Dc4T3axetqIX7Fw== X-Received: by 2002:ac8:2d53:: with SMTP id o19mr4132423qta.21.1548282219607; Wed, 23 Jan 2019 14:23:39 -0800 (PST) X-Google-Smtp-Source: ALg8bN5frA1T19MbK2mlHgaMCaoavCJjdzQCj6G79h6jfftJgrQ0Fs2tV5ceQwsQ469ibeJE9QXD X-Received: by 2002:ac8:2d53:: with SMTP id o19mr4132393qta.21.1548282218908; Wed, 23 Jan 2019 14:23:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548282218; cv=none; d=google.com; s=arc-20160816; b=Z2UV0SO/X96fu6ytdzvXloOnrvL85GZ6KiM4/6IDJyDFZilM2SkV2DTghq3jGGrXDx BUlhduEeBgHBUw2DJQvGFsSlsZkvfNp+X2u9MW6htmHSDV+Z+hPIeoed1mtGdR3OS0N8 lI845QON4hYAaPrNE681h5iZ9cEbf3Ne7WRvXZilA/s2zmg+4jFzNXvAoP8OBaMZkvMc Q8vuyw8eEz76ExOdfkszwSk//g9Xy0AGZUAM2yIk60LHkvHvDSjd4pMoFVqQhTWKLdEH lHL/+GMp4pulK7OffP4CB08xDIRN1KzaMyYgPfsTvgECzMHmML/dqLswvKR0833jvfLz nhtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=povx2BgWdKWZnT/gMz3YB3oo7NPWl/RpagLy1A16ZvY=; b=zSv5MmimKdcYDkeuMR1J9+o7bdkvijzEJmWyx8b420JWt1REI3tpZ3VRsBEyGhjPii wMtpOUXNIxFiO71y8HYVZrk+IBLq/9TjWBePTsrD48RwpkfH2erKcc0+B/bWFSR39qGc 2+zjfy/mEEtZWsECDTd/ObP93kLnLNIKY9I/f+8Yw/ogcOGSL0xcCH8LTTDiBZm1sM64 p6//0zMxNQHH+G/RDTcwPVP2bAd0gZxOimdoj8g13a+p4DLX60Wm0H2douGZmJSgvSrK hsw0cx+Z5jPzVIWGlxwEwKTeZ2Osc0d4LXhDPw3EZ7GPwvRJsd4GXplmIvFS1KVBhfW1 AbGw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id r15si790382qtj.56.2019.01.23.14.23.38 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jan 2019 14:23:38 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E81CFC0A1AE9; Wed, 23 Jan 2019 22:23:37 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-127.rdu2.redhat.com [10.10.120.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 47C655D964; Wed, 23 Jan 2019 22:23:35 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Jan Kara , Felix Kuehling , Jason Gunthorpe , Matthew Wilcox , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-fsdevel@vger.kernel.org, Arnd Bergmann Subject: [PATCH v4 3/9] mm/mmu_notifier: use correct mmu_notifier events for each invalidation Date: Wed, 23 Jan 2019 17:23:09 -0500 Message-Id: <20190123222315.1122-4-jglisse@redhat.com> In-Reply-To: <20190123222315.1122-1-jglisse@redhat.com> References: <20190123222315.1122-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 23 Jan 2019 22:23:38 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse This update each existing invalidation to use the correct mmu notifier event that represent what is happening to the CPU page table. See the patch which introduced the events to see the rational behind this. Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Jan Kara Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Andrew Morton Cc: Matthew Wilcox Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org Cc: Arnd Bergmann --- fs/proc/task_mmu.c | 2 +- kernel/events/uprobes.c | 2 +- mm/huge_memory.c | 14 ++++++-------- mm/hugetlb.c | 7 ++++--- mm/khugepaged.c | 2 +- mm/ksm.c | 4 ++-- mm/madvise.c | 2 +- mm/memory.c | 16 ++++++++-------- mm/migrate.c | 4 ++-- mm/mprotect.c | 5 +++-- mm/rmap.c | 6 +++--- 11 files changed, 32 insertions(+), 32 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 57e7f98647d3..cce226f3305f 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1143,7 +1143,7 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf, break; } - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, + mmu_notifier_range_init(&range, MMU_NOTIFY_SOFT_DIRTY, NULL, mm, 0, -1UL); mmu_notifier_invalidate_range_start(&range); } diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index b67fe7e59621..87e76a1dc758 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -174,7 +174,7 @@ static int __replace_page(struct vm_area_struct *vma, unsigned long addr, struct mmu_notifier_range range; struct mem_cgroup *memcg; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, addr, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, vma, mm, addr, addr + PAGE_SIZE); VM_BUG_ON_PAGE(PageTransHuge(old_page), old_page); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index b353e8b7876f..957d23754217 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1182,9 +1182,8 @@ static vm_fault_t do_huge_pmd_wp_page_fallback(struct vm_fault *vmf, cond_resched(); } - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, - haddr, - haddr + HPAGE_PMD_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, vma, vma->vm_mm, + haddr, haddr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd); @@ -1346,9 +1345,8 @@ vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd) vma, HPAGE_PMD_NR); __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, - haddr, - haddr + HPAGE_PMD_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, vma, vma->vm_mm, + haddr, haddr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); spin_lock(vmf->ptl); @@ -2025,7 +2023,7 @@ void __split_huge_pud(struct vm_area_struct *vma, pud_t *pud, spinlock_t *ptl; struct mmu_notifier_range range; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, vma, vma->vm_mm, address & HPAGE_PUD_MASK, (address & HPAGE_PUD_MASK) + HPAGE_PUD_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -2244,7 +2242,7 @@ void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, spinlock_t *ptl; struct mmu_notifier_range range; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, vma, vma->vm_mm, address & HPAGE_PMD_MASK, (address & HPAGE_PMD_MASK) + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index cbda46ad6a30..f691398ac6b6 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3246,7 +3246,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, cow = (vma->vm_flags & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE; if (cow) { - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, src, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, vma, src, vma->vm_start, vma->vm_end); mmu_notifier_invalidate_range_start(&range); @@ -3627,7 +3627,7 @@ static vm_fault_t hugetlb_cow(struct mm_struct *mm, struct vm_area_struct *vma, __SetPageUptodate(new_page); set_page_huge_active(new_page); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, haddr, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, vma, mm, haddr, haddr + huge_page_size(h)); mmu_notifier_invalidate_range_start(&range); @@ -4348,7 +4348,8 @@ unsigned long hugetlb_change_protection(struct vm_area_struct *vma, * start/end. Set range.start/range.end to cover the maximum possible * range if PMD sharing is possible. */ - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, start, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_PROTECTION_VMA, + vma, mm, start, end); adjust_range_if_pmd_sharing_possible(vma, &range.start, &range.end); BUG_ON(address >= end); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index f903acb1b0a6..d09d71f10f0f 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1016,7 +1016,7 @@ static void collapse_huge_page(struct mm_struct *mm, pte = pte_offset_map(pmd, address); pte_ptl = pte_lockptr(mm, pmd); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, NULL, mm, address, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, NULL, mm, address, address + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); pmd_ptl = pmd_lock(mm, pmd); /* probably unnecessary */ diff --git a/mm/ksm.c b/mm/ksm.c index 6b27c4f0fb1f..97757c5fa15f 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1051,7 +1051,7 @@ static int write_protect_page(struct vm_area_struct *vma, struct page *page, BUG_ON(PageTransCompound(page)); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, vma, mm, pvmw.address, pvmw.address + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -1140,7 +1140,7 @@ static int replace_page(struct vm_area_struct *vma, struct page *page, if (!pmd) goto out; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, addr, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, vma, mm, addr, addr + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); diff --git a/mm/madvise.c b/mm/madvise.c index 04446dabba56..a2f91bd286d9 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -472,7 +472,7 @@ static int madvise_free_single_vma(struct vm_area_struct *vma, range.end = min(vma->vm_end, end_addr); if (range.end <= vma->vm_start) return -EINVAL; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, vma, mm, range.start, range.end); lru_add_drain(); diff --git a/mm/memory.c b/mm/memory.c index d9b7c935e812..a8c6922526f6 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1009,8 +1009,8 @@ int copy_page_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, is_cow = is_cow_mapping(vma->vm_flags); if (is_cow) { - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, src_mm, - addr, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_PROTECTION_PAGE, + vma, src_mm, addr, end); mmu_notifier_invalidate_range_start(&range); } @@ -1334,7 +1334,7 @@ void unmap_vmas(struct mmu_gather *tlb, { struct mmu_notifier_range range; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, NULL, vma->vm_mm, start_addr, end_addr); mmu_notifier_invalidate_range_start(&range); for ( ; vma && vma->vm_start < end_addr; vma = vma->vm_next) @@ -1357,7 +1357,7 @@ void zap_page_range(struct vm_area_struct *vma, unsigned long start, struct mmu_gather tlb; lru_add_drain(); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, vma, vma->vm_mm, start, start + size); tlb_gather_mmu(&tlb, vma->vm_mm, start, range.end); update_hiwater_rss(vma->vm_mm); @@ -1384,7 +1384,7 @@ static void zap_page_range_single(struct vm_area_struct *vma, unsigned long addr struct mmu_gather tlb; lru_add_drain(); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, vma, vma->vm_mm, address, address + size); tlb_gather_mmu(&tlb, vma->vm_mm, address, range.end); update_hiwater_rss(vma->vm_mm); @@ -2275,7 +2275,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, vma, mm, vmf->address & PAGE_MASK, (vmf->address & PAGE_MASK) + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -4086,7 +4086,7 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, goto out; if (range) { - mmu_notifier_range_init(range, MMU_NOTIFY_UNMAP, NULL, + mmu_notifier_range_init(range, MMU_NOTIFY_CLEAR, NULL, mm, address & PMD_MASK, (address & PMD_MASK) + PMD_SIZE); mmu_notifier_invalidate_range_start(range); @@ -4105,7 +4105,7 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, goto out; if (range) { - mmu_notifier_range_init(range, MMU_NOTIFY_UNMAP, NULL, mm, + mmu_notifier_range_init(range, MMU_NOTIFY_CLEAR, NULL, mm, address & PAGE_MASK, (address & PAGE_MASK) + PAGE_SIZE); mmu_notifier_invalidate_range_start(range); diff --git a/mm/migrate.c b/mm/migrate.c index 385c59d5c28d..384cde91d886 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2342,7 +2342,7 @@ static void migrate_vma_collect(struct migrate_vma *migrate) mm_walk.mm = migrate->vma->vm_mm; mm_walk.private = migrate; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, NULL, mm_walk.mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, NULL, mm_walk.mm, migrate->start, migrate->end); mmu_notifier_invalidate_range_start(&range); @@ -2751,7 +2751,7 @@ static void migrate_vma_pages(struct migrate_vma *migrate) notified = true; mmu_notifier_range_init(&range, - MMU_NOTIFY_UNMAP, + MMU_NOTIFY_CLEAR, NULL, migrate->vma->vm_mm, addr, migrate->end); diff --git a/mm/mprotect.c b/mm/mprotect.c index b22e660701f1..c19e8bdc2648 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -185,8 +185,9 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma, /* invoke the mmu notifier if the pmd is populated */ if (!range.start) { - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, - vma->vm_mm, addr, end); + mmu_notifier_range_init(&range, + MMU_NOTIFY_PROTECTION_VMA, + vma, vma->vm_mm, addr, end); mmu_notifier_invalidate_range_start(&range); } diff --git a/mm/rmap.c b/mm/rmap.c index 49c75f0c6c33..81ea84ab511a 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -896,8 +896,8 @@ static bool page_mkclean_one(struct page *page, struct vm_area_struct *vma, * We have to assume the worse case ie pmd for invalidation. Note that * the page can not be free from this function. */ - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, - address, + mmu_notifier_range_init(&range, MMU_NOTIFY_PROTECTION_PAGE, vma, + vma->vm_mm, address, min(vma->vm_end, address + (PAGE_SIZE << compound_order(page)))); mmu_notifier_invalidate_range_start(&range); @@ -1372,7 +1372,7 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, * Note that the page can not be free in this function as call of * try_to_unmap() must hold a reference on the page. */ - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, vma, vma->vm_mm, address, min(vma->vm_end, address + (PAGE_SIZE << compound_order(page)))); From patchwork Wed Jan 23 22:23:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10777991 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9DFED1390 for ; Wed, 23 Jan 2019 22:23:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 908792D368 for ; Wed, 23 Jan 2019 22:23:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8479B2D396; Wed, 23 Jan 2019 22:23:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1857B2D3D7 for ; Wed, 23 Jan 2019 22:23:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5CAE38E0052; Wed, 23 Jan 2019 17:23:43 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 508D38E0047; Wed, 23 Jan 2019 17:23:43 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 21DD68E0052; Wed, 23 Jan 2019 17:23:43 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by kanga.kvack.org (Postfix) with ESMTP id E0BD98E0047 for ; Wed, 23 Jan 2019 17:23:42 -0500 (EST) Received: by mail-qt1-f198.google.com with SMTP id q33so4209476qte.23 for ; Wed, 23 Jan 2019 14:23:42 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fJRovSQTlFFDH/WUnYlgYbMcyCf5O04FUu8tM8xA5uc=; b=onTfLMDmO8unTL85CQIsjvxV/PNLq47G1QGZsjQhDMohRxifjOH4YvUJw51BwDTbN9 ZMz2ni7nuj1pQvcHf1ikiFww8dSRCVHsRlVOtAPNDFfRW3KyTrMxxBOYXeDcGiA4qKC0 3uEsI6sK/76cHagvL/3rQYqCbplnSiEE8MF8UgEk6VK6ZWVx1cLsFQ32U999bllYdTsZ L+F6sobXYe/35s1qYssWKhf7c/psf0MLuBwfcvme+xec0RMAoYnKhAYe1mjz4K3J+LYO qkLKlgMjgf5F55mioFrGTFruxRWtdN/8e9WNVQ47IgfLBVt1IPsOuF5zn7deEz4lhEej zspw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUuke8/eTG+mkPKGcxiSVGqAdpFrQaS2rp7yl//zaDaKhWlmqsOzQo /2P707Dk9bY1qmZpNwIVYFrzkJU0hynSB8SmCyR7LkbtB6Tb1s0vGYIGjefsqpbrMM12h1uiGzS Z0XR+qfqYfy0TWVX4TE7Fy/tkaQK7ERLQktykOwBAADR+uexYvF9YlyvE68idezVyHQ== X-Received: by 2002:a37:a904:: with SMTP id s4mr3564393qke.237.1548282222678; Wed, 23 Jan 2019 14:23:42 -0800 (PST) X-Google-Smtp-Source: ALg8bN6rnOe5lj8/yR8w560KP/Sli3DLBHRRMOYv7LSYyQB2b6EPXRDbvG/gtki6slHc+6cewGOB X-Received: by 2002:a37:a904:: with SMTP id s4mr3564356qke.237.1548282222121; Wed, 23 Jan 2019 14:23:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548282222; cv=none; d=google.com; s=arc-20160816; b=XdOUi2jh90ONK+0cU4Lx/EbJX2v/XPt6ZYN3BQsCOJC7OE+7PnOktXfQe9R1zUveCr Gz2R+zYg6oKX3InOglBAn8yiokIB35HTEOOGjsNz6eyQRAgQgPRzHUQ1Z5VChZJ0ys9O N7NhNgwAomK8zYJ5SsUenvXL925Pkl6VixEc4FBmAV3DauBKa+GOErk6pcnVGVtVRIt4 h/SI8dLf82d5+zG1B7YaKYXHYQXYKq73tti7sHi0Sb/E4Doedrwl0s6J/kVAraGfkXub Zly6Kl2WF0csqX5BOPUGgvmBFPiOE1N94t1qpG40QeOMgObr+33ENlVwV874iu5d0Xzi xxqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=fJRovSQTlFFDH/WUnYlgYbMcyCf5O04FUu8tM8xA5uc=; b=FFfR7fyHXBGpY343z9HKu+0SOpkZwPFwK6uSeCAdBw6jyUkhjycogO5bGrSfne0dN6 2kEJuEHJ2Jxthy9WAmrGLfpBlsaasIswx1FHNw7ahzqmAnLjcQVN52wror1FYd9tkviH AI9JUy1qcNdmCjCY/8tp2jnO5maXYeenjliCP1JgMTkLrB4/tBcJSbCmC3bztZSaTXf2 Q/ftjcwpI7XVEUR9uOGk9YratFRjIFrCHYNHc92ak281Z9A02LYyc7MLjtz0PhGVmPrb DDfiphflr8Jup72PYRpoH5WqCHaLRVxnwFvTJKIYnhuv/2/+KRp7cWLzRSrJj+iudH+j 0qyA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id k18si1459105qtb.401.2019.01.23.14.23.42 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jan 2019 14:23:42 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C9341D7D58; Wed, 23 Jan 2019 22:23:40 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-127.rdu2.redhat.com [10.10.120.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 21DE26A685; Wed, 23 Jan 2019 22:23:38 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Jan Kara , Felix Kuehling , Jason Gunthorpe , Matthew Wilcox , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-fsdevel@vger.kernel.org, Arnd Bergmann Subject: [PATCH v4 4/9] mm/mmu_notifier: pass down vma and reasons why mmu notifier is happening Date: Wed, 23 Jan 2019 17:23:10 -0500 Message-Id: <20190123222315.1122-5-jglisse@redhat.com> In-Reply-To: <20190123222315.1122-1-jglisse@redhat.com> References: <20190123222315.1122-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 23 Jan 2019 22:23:41 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse CPU page table update can happens for many reasons, not only as a result of a syscall (munmap(), mprotect(), mremap(), madvise(), ...) but also as a result of kernel activities (memory compression, reclaim, migration, ...). Users of mmu notifier API track changes to the CPU page table and take specific action for them. While current API only provide range of virtual address affected by the change, not why the changes is happening This patch is just passing down the new informations by adding it to the mmu_notifier_range structure. Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Jan Kara Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Andrew Morton Cc: Matthew Wilcox Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org Cc: Arnd Bergmann --- include/linux/mmu_notifier.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index a9808add4070..7514775817de 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -56,9 +56,11 @@ struct mmu_notifier_mm { }; struct mmu_notifier_range { + struct vm_area_struct *vma; struct mm_struct *mm; unsigned long start; unsigned long end; + enum mmu_notifier_event event; bool blockable; }; @@ -354,6 +356,8 @@ static inline void mmu_notifier_range_init(struct mmu_notifier_range *range, unsigned long start, unsigned long end) { + range->vma = vma; + range->event = event; range->mm = mm; range->start = start; range->end = end; From patchwork Wed Jan 23 22:23:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10777997 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9BB891399 for ; Wed, 23 Jan 2019 22:23:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5620B29207 for ; Wed, 23 Jan 2019 22:23:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4914F28DE2; Wed, 23 Jan 2019 22:23:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CA50F2D37E for ; Wed, 23 Jan 2019 22:23:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AD0D18E0053; Wed, 23 Jan 2019 17:23:45 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9E3798E0047; Wed, 23 Jan 2019 17:23:45 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 834FC8E0053; Wed, 23 Jan 2019 17:23:45 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by kanga.kvack.org (Postfix) with ESMTP id 5131B8E0047 for ; Wed, 23 Jan 2019 17:23:45 -0500 (EST) Received: by mail-qt1-f199.google.com with SMTP id k90so4384403qte.0 for ; Wed, 23 Jan 2019 14:23:45 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=9wi7YLiorG0zg+O0dNIqm0yoV34mSK8NVO5rYlqWX4Q=; b=nmFscXRvPv5prLx7uW8ekGejDLoTZI9VHhvIiHDhGmhiAk64rUTh6e/1ZBuce3954y Gyt7c9twmL/en+DUiX3xiHouO7DBQ4GhoyhJ0hpqRIrky3XdDWCjiReu73lhKaaoMbc9 nvXWT86T8oQEegwlMhUb7inu4XpJ8PUa026JYIgvcJceXVnBsCg3XTHagokcu+4BPlvB vnt9DDCzbWiKZ7vTHsfHCxSNMibMRyugntVLICk878BbCljbcXKobCuhaGFmh2kzTUfE mGvO0aLP9tg18jRr994J2UpOYGB6guqQNfUrIWaQECKe9o91SGC9ukQAACM6yw4cvSes lBFg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukeBf0oRL2EFknhM846spCrB9lFt3jkz5y/grT19VQfJ0fQqqjeb hda+QzUBOEi8vbUAgby8UlZsC8ijKOE1HcqaJU9EbU93eAyHCCct2qAIJhUt6jrvQgbUHDieyKh 7V4YhzCDfusYOTGaw87KAKoO9qBFi2rJtm2V2w3mrmjZfQECv6jm6q4C5N/f5uM7RdA== X-Received: by 2002:a37:2714:: with SMTP id n20mr3403667qkn.349.1548282225096; Wed, 23 Jan 2019 14:23:45 -0800 (PST) X-Google-Smtp-Source: ALg8bN6nER88iiIX+r/X5d4SCFI+KMh7aOvrdI+e+MIUxDtjI6vKm0jVEc0mLRXHxNmHRn+5HO5R X-Received: by 2002:a37:2714:: with SMTP id n20mr3403646qkn.349.1548282224664; Wed, 23 Jan 2019 14:23:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548282224; cv=none; d=google.com; s=arc-20160816; b=RGpLNRZQkK/Zg6YuFKpqjkyNZvx5tfqMmN5VFFVJCfQMqqKtUvUdEfGQ8K/OqlLUmC jRHoV5QTPBbNjEu8vdA+1/JRYVM4l1bMP4N2pDCMnmOnQm7Q2QK4OxDHQrNk6VATigTx 9BR940jpWIhxGNX62JkIpMfPkLy1VizVA80tD5pxKwVbBUFUXMNYc7soQaECRggxiTXl PaQ352CaCi5faICZRui5aARRsgcA0ZNQFx7bmsJscyCxQOCuMi1+AQ31MoxWcqISyhC+ lOQgFHSTyDksV3M2l1cJhNiH0mUQoUjJDb5K4TjujygF+uUHUse9seMx+zAaIvYefFzi 7WBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=9wi7YLiorG0zg+O0dNIqm0yoV34mSK8NVO5rYlqWX4Q=; b=D9tlP/TwMqLHObNFck0TKZxNEA8vPl9yzkHYH1DLdZCiSDQECR6vDpNOLyZr3LpJSO US6s5P3E4RB+yZ2qEo7s8HeYLNukvwv/RfZndlY43qZzbnkKcrC6nfW7hRr1cRFcIp4o Jsu7Iq2Pd9p0MgbfwJblIvLI4popQ6aqC05ZOu7FrNp52wGXnmcsENq/ZziyjBI2i8hX d4j+Hk0l36M1qBBhmsVIIX9vnqf9tXhV8Oc+GjBlT0FXpr5lk+JgJnX9RI8cgEFOTHei HF0xvo2uwZOIU/KIJ3PC4OpKKMsJhLnXw5NzvG78/2BWCYSgMNbv4mRbm/5idBsf5Rso UGSw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id h12si6136349qto.184.2019.01.23.14.23.44 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jan 2019 14:23:44 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 86234CC0A8; Wed, 23 Jan 2019 22:23:43 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-127.rdu2.redhat.com [10.10.120.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id E90475D964; Wed, 23 Jan 2019 22:23:40 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Jan Kara , Felix Kuehling , Jason Gunthorpe , Matthew Wilcox , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-fsdevel@vger.kernel.org, Arnd Bergmann Subject: [PATCH v4 5/9] mm/mmu_notifier: mmu_notifier_range_update_to_read_only() helper Date: Wed, 23 Jan 2019 17:23:11 -0500 Message-Id: <20190123222315.1122-6-jglisse@redhat.com> In-Reply-To: <20190123222315.1122-1-jglisse@redhat.com> References: <20190123222315.1122-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 23 Jan 2019 22:23:43 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Helper to test if a range is updated to read only (it is still valid to read from the range). This is useful for device driver or anyone who wish to optimize out update when they know that they already have the range map read only. Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Jan Kara Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Andrew Morton Cc: Matthew Wilcox Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org Cc: Arnd Bergmann --- include/linux/mmu_notifier.h | 4 ++++ mm/mmu_notifier.c | 10 ++++++++++ 2 files changed, 14 insertions(+) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 7514775817de..be873c431886 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -257,6 +257,8 @@ extern void __mmu_notifier_invalidate_range_end(struct mmu_notifier_range *r, bool only_end); extern void __mmu_notifier_invalidate_range(struct mm_struct *mm, unsigned long start, unsigned long end); +extern bool +mmu_notifier_range_update_to_read_only(const struct mmu_notifier_range *range); static inline void mmu_notifier_release(struct mm_struct *mm) { @@ -553,6 +555,8 @@ static inline void mmu_notifier_mm_destroy(struct mm_struct *mm) { } +#define mmu_notifier_range_update_to_read_only(r) false + #define ptep_clear_flush_young_notify ptep_clear_flush_young #define pmdp_clear_flush_young_notify pmdp_clear_flush_young #define ptep_clear_young_notify ptep_test_and_clear_young diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index 9c884abc7850..0b2f77715a08 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -395,3 +395,13 @@ void mmu_notifier_unregister_no_release(struct mmu_notifier *mn, mmdrop(mm); } EXPORT_SYMBOL_GPL(mmu_notifier_unregister_no_release); + +bool +mmu_notifier_range_update_to_read_only(const struct mmu_notifier_range *range) +{ + if (!range->vma || range->event != MMU_NOTIFY_PROTECTION_VMA) + return false; + /* Return true if the vma still have the read flag set. */ + return range->vma->vm_flags & VM_READ; +} +EXPORT_SYMBOL_GPL(mmu_notifier_range_update_to_read_only); From patchwork Wed Jan 23 22:23:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10778001 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3BDBF1515 for ; Wed, 23 Jan 2019 22:23:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2BF6E2C4E2 for ; Wed, 23 Jan 2019 22:23:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1EE502C4F8; Wed, 23 Jan 2019 22:23:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 908602C4E4 for ; Wed, 23 Jan 2019 22:23:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D2C48E0054; Wed, 23 Jan 2019 17:23:48 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 885A58E0047; Wed, 23 Jan 2019 17:23:48 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6409A8E0054; Wed, 23 Jan 2019 17:23:48 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id 220258E0047 for ; Wed, 23 Jan 2019 17:23:48 -0500 (EST) Received: by mail-qk1-f198.google.com with SMTP id y27so3418545qkj.21 for ; Wed, 23 Jan 2019 14:23:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3BKZ/v2XLYl6bJCscr3vEcuiItOFgg+U4duPRXYMcZs=; b=jJiYILb7XNTaIyZh3IO9G2b6e5IdaY0J5FV689LhKJXrU9HxRR7zAemCc+tpR1dhxM y+eGKt8H/SfxZjn4FrhZ26iisftVMV8KQxsBimTU7onVxvaiLlufTXUlWQscMd5dgqbh VMcrK2C0Wv5x0HFfmD3tMYAwq4xo7vgQPq2g2CWanlphbARG/uVcGTjj4D5EsFWM2xQC z3Jg9IxJ8TBGNG7E5lnF8zKdoqr9c8qlTjeFsGEe1MLJbZp1mAEhc5FnFF9EcBMdcdT/ tvgFgoBKBT8ghwYAy6Hj1LBR/blRiL37HsDPMiz4GAy4F/4iIbMdAk+lfkBTULvFxPNy IAXA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukdA2CrdJ+32hFx0xV8AYQkyGJy645NN9r1izEi3ygRml1wClu7L iagn2ow0IKJoU6X+dEGEGW74E/55SQJrpipbdHIW8Ep6/M43Xe11Wt+q+f+IjO2aF9ixXtWichw hlarbexDCT1U60lRCMIccq5vt/kmI7BDR0Wb5lCXFFCRJ6uoEuqoovCupUHroX/Ur/A== X-Received: by 2002:a0c:df12:: with SMTP id g18mr3594221qvl.208.1548282227915; Wed, 23 Jan 2019 14:23:47 -0800 (PST) X-Google-Smtp-Source: ALg8bN6UBInzbwTJYV+HJAMgFEB6g/5mMCWnOKz7i+QRJi/ezADdQyZUJXAcf77CPU5UEJPS0M2y X-Received: by 2002:a0c:df12:: with SMTP id g18mr3594201qvl.208.1548282227476; Wed, 23 Jan 2019 14:23:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548282227; cv=none; d=google.com; s=arc-20160816; b=T8J4cDwIDjCdNVmICh1E5/GK+9BNNDbpGaBfJ275OZ8Lri7wxDzsql9yuDVRmvHPDS +OfzAJvhmmD2BCGyLo5cqm++BL7r8KJ16jPfH+jPzYNrXQIYLp9aysC95ZvXKVANXuDr dyKLrSBkNaz1eTqhLz1zCb6DwQEfWhroIh6e6nVu4nviXOsWLFeKczVS0ruB8ayiuBg+ RS8b3g5yMU1T3/5rKlZvqtslRJE5/6Vxa1Wyh4yM7yjEsrAR46zsteOaso8bFLl7h+Ma WLJZ34oi6mReeu/PISPrGnKtE4BWghu/8g2mB65E7EuZ/0xduL7Y6NbPseY8NRKvKZOF Pm7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=3BKZ/v2XLYl6bJCscr3vEcuiItOFgg+U4duPRXYMcZs=; b=B2C2UZ5NpN3I9JNFqw28//N9s7ahE3ol+o7HHIJS2Mno86nC5CzRstlgJFKG2EFUCe gEjc5v9adFDeP1vlhR09Su04rek7ppvx5vCx6fhcu1vlJO3qwhBKKgtIw6i+QBdtdH4L zT1g2PrmbbFwMrwMwX5a2F2h9zAib3zOzDqFusWxpTKRwsKuAtEg+7d4S/E1hG/nrt1z YyHx9gtLmluDaNJ8/HVsNfwtGKfpwnYW5jIx/E3SNR79GErVzmmbvxtkRq0IidgytV39 m08lV7YIqsahT4D8Kcj8ZQS9v9UltwN88Cgu9JWnHxOjx0fEKI4jG2ro+pWIxXcS3o13 TTtA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id h10si8990634qtc.140.2019.01.23.14.23.47 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jan 2019 14:23:47 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 61CC618DF61; Wed, 23 Jan 2019 22:23:46 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-127.rdu2.redhat.com [10.10.120.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id B5EAC5D965; Wed, 23 Jan 2019 22:23:43 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Jan Kara , Felix Kuehling , Jason Gunthorpe , Matthew Wilcox , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-fsdevel@vger.kernel.org, Arnd Bergmann Subject: [PATCH v4 6/9] gpu/drm/radeon: optimize out the case when a range is updated to read only Date: Wed, 23 Jan 2019 17:23:12 -0500 Message-Id: <20190123222315.1122-7-jglisse@redhat.com> In-Reply-To: <20190123222315.1122-1-jglisse@redhat.com> References: <20190123222315.1122-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 23 Jan 2019 22:23:46 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse When range of virtual address is updated read only and corresponding user ptr object are already read only it is pointless to do anything. Optimize this case out. Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Jan Kara Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Andrew Morton Cc: Matthew Wilcox Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org Cc: Arnd Bergmann --- drivers/gpu/drm/radeon/radeon_mn.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/drivers/gpu/drm/radeon/radeon_mn.c b/drivers/gpu/drm/radeon/radeon_mn.c index b3019505065a..f77294f58e63 100644 --- a/drivers/gpu/drm/radeon/radeon_mn.c +++ b/drivers/gpu/drm/radeon/radeon_mn.c @@ -124,6 +124,7 @@ static int radeon_mn_invalidate_range_start(struct mmu_notifier *mn, struct radeon_mn *rmn = container_of(mn, struct radeon_mn, mn); struct ttm_operation_ctx ctx = { false, false }; struct interval_tree_node *it; + bool update_to_read_only; unsigned long end; int ret = 0; @@ -138,6 +139,8 @@ static int radeon_mn_invalidate_range_start(struct mmu_notifier *mn, else if (!mutex_trylock(&rmn->lock)) return -EAGAIN; + update_to_read_only = mmu_notifier_range_update_to_read_only(range); + it = interval_tree_iter_first(&rmn->objects, range->start, end); while (it) { struct radeon_mn_node *node; @@ -153,10 +156,20 @@ static int radeon_mn_invalidate_range_start(struct mmu_notifier *mn, it = interval_tree_iter_next(it, range->start, end); list_for_each_entry(bo, &node->bos, mn_list) { + bool read_only; if (!bo->tbo.ttm || bo->tbo.ttm->state != tt_bound) continue; + /* + * If it is already read only and we are updating to + * read only then we do not need to change anything. + * So save time and skip this one. + */ + read_only = radeon_ttm_tt_is_readonly(bo->tbo.ttm); + if (update_to_read_only && read_only) + continue; + r = radeon_bo_reserve(bo, true); if (r) { DRM_ERROR("(%ld) failed to reserve user bo\n", r); From patchwork Wed Jan 23 22:23:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10778005 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D341F1390 for ; Wed, 23 Jan 2019 22:23:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C39242C4E4 for ; Wed, 23 Jan 2019 22:23:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B79742C4FE; Wed, 23 Jan 2019 22:23:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 318482C4E4 for ; Wed, 23 Jan 2019 22:23:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 444F38E0055; Wed, 23 Jan 2019 17:23:51 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3545B8E0047; Wed, 23 Jan 2019 17:23:51 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F7628E0055; Wed, 23 Jan 2019 17:23:51 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by kanga.kvack.org (Postfix) with ESMTP id CEF748E0047 for ; Wed, 23 Jan 2019 17:23:50 -0500 (EST) Received: by mail-qt1-f200.google.com with SMTP id n95so4290103qte.16 for ; Wed, 23 Jan 2019 14:23:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Lzj9jwerKIllHI5/K1OIrzFaHxAO7K6V4cLHs1Jtprw=; b=Kx0k13kMk2Ll9HibXwR4I7ZjPAXRMEmfz6Q575OANxqxD9CvlaAOQvMbADtF+VrnAn nxa+i4UbokqrEE7inBfjejL41b7ZJM6b0SYF3CwnIZvy5OiEg720HSZoFckp0jcEh3H7 PYNgNuqvzx3xXcqnAv7bEfQ24hnrEx+d/SxFx8sb+tjgoLtoCfWIG2i+2mHu/mdCUKnS QHWHEfqx/DSFUPHjp4JzsVnRpZwRNuzNF5k2MXupGEjRMEdtOGnk13y7YRIihEUjr2Gx 3aze+ZnWxEoQ6KVvifZqJfCF54Q9ICowWPKoSVdGdkH6WInp8WP4+T2w0pVAZZRbiaH+ 93+g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukcxQZVwTyQ+ED14diyMLIKRb6jOTgziKaSHA+bWYiMVBZbcGUpM igWbqXMedWP2WOMcTQujkGOFAVyipV9hn9PSZReOzL6A/wbBKKAyCKF8w7VWJWZSkf+UJIZMqKc vlFYi22CRKfcctmXEK54HqSKv0NuhJ2DQ8iB1/SeUCSidL/tFR7QtNHD/w+RWl5KWyw== X-Received: by 2002:ac8:7493:: with SMTP id v19mr4227321qtq.309.1548282230635; Wed, 23 Jan 2019 14:23:50 -0800 (PST) X-Google-Smtp-Source: ALg8bN5COj3udJkKVpvIKUuKc3wOttWDNpbJyUU6vD0ACui4i7BhP2k6m3aaCp7U2C+gxTF31JBl X-Received: by 2002:ac8:7493:: with SMTP id v19mr4227303qtq.309.1548282230117; Wed, 23 Jan 2019 14:23:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548282230; cv=none; d=google.com; s=arc-20160816; b=XPZrBiunBiw9OD8nwN0nIwGDeguCBwUSfVk9TgGNCcAzOU+ekVzWImczvc4pbWtjfW HB0NRM44r4lUwxwuh2zWNmL+hju9MUyazbzgHDAEeY9j1gK5Thb8KEGDc+/Bq3pOnIXU pPEdAJXpcxx/18aKk74uioeBKvmcdTakQxIn+TeHNBNW1bws//K2puK5k06hGviFz4wN igMdSPOnoDsAAbi9wq6ABewSbacKezTjsNPh8cfZM9Sp2JxaFp1UNlXjGZ8MXML1Y49j enJXzqqoOjqFz+QF0+spdLWodjguPWYWgFYvJBV6z41kEs//ldYvZskgaSPLR0vRQ/b6 WQPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=Lzj9jwerKIllHI5/K1OIrzFaHxAO7K6V4cLHs1Jtprw=; b=zChagEjIePuLf97ZIRJUSti59+/QCGSGO9IYj5y5zfdkL3qG4nxJlTsSZGlk4qOZ0q AdlwAGbukw8jSWXOGu75M3PcIU9qDOOM2Nm752cpJOz/9QH7X+1ZzaLJwxRrVIu113vq FVQKQWQREtypvOyVLwXGLA6pjQG5Bp2jj9IwT7jEil1mZUoVs5Ytd/VnUZP81bvRCxci HAsd/suYMZDQZmE07hCU1URWwXfQw+DJkRFFY5LG7cAxYmQGsl4ltoArege627Y/ElWk 5RUrgATCMWn6LwfbRKcCkc7TgyHZZvLSrgpBxtXgGGyec3Ycm01yBEhvPfD6s9kUuG3p hF/Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id l2si2315120qtj.22.2019.01.23.14.23.49 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jan 2019 14:23:50 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2DA69432A0; Wed, 23 Jan 2019 22:23:49 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-127.rdu2.redhat.com [10.10.120.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 87EAF6A685; Wed, 23 Jan 2019 22:23:46 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Jan Kara , Felix Kuehling , Jason Gunthorpe , Matthew Wilcox , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-fsdevel@vger.kernel.org, Arnd Bergmann Subject: [PATCH v4 7/9] gpu/drm/amdgpu: optimize out the case when a range is updated to read only Date: Wed, 23 Jan 2019 17:23:13 -0500 Message-Id: <20190123222315.1122-8-jglisse@redhat.com> In-Reply-To: <20190123222315.1122-1-jglisse@redhat.com> References: <20190123222315.1122-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 23 Jan 2019 22:23:49 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse When range of virtual address is updated read only and corresponding user ptr object are already read only it is pointless to do anything. Optimize this case out. Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Jan Kara Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Andrew Morton Cc: Matthew Wilcox Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org Cc: Arnd Bergmann --- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c index 3e6823fdd939..7880eda064cd 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c @@ -294,6 +294,7 @@ static int amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, { struct amdgpu_mn *amn = container_of(mn, struct amdgpu_mn, mn); struct interval_tree_node *it; + bool update_to_read_only; unsigned long end; /* notification is exclusive, but interval is inclusive */ @@ -302,6 +303,8 @@ static int amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, if (amdgpu_mn_read_lock(amn, range->blockable)) return -EAGAIN; + update_to_read_only = mmu_notifier_range_update_to_read_only(range); + it = interval_tree_iter_first(&amn->objects, range->start, end); while (it) { struct amdgpu_mn_node *node; @@ -317,6 +320,16 @@ static int amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, list_for_each_entry(bo, &node->bos, mn_list) { struct kgd_mem *mem = bo->kfd_bo; + bool read_only; + + /* + * If it is already read only and we are updating to + * read only then we do not need to change anything. + * So save time and skip this one. + */ + read_only = amdgpu_ttm_tt_is_readonly(bo->tbo.ttm); + if (update_to_read_only && read_only) + continue; if (amdgpu_ttm_tt_affect_userptr(bo->tbo.ttm, range->start, From patchwork Wed Jan 23 22:23:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10778011 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C6511390 for ; Wed, 23 Jan 2019 22:23:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6E9512C4E4 for ; Wed, 23 Jan 2019 22:23:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 625462C4FE; Wed, 23 Jan 2019 22:23:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DD0922C4E4 for ; Wed, 23 Jan 2019 22:23:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1AC038E0056; Wed, 23 Jan 2019 17:23:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 132A68E0047; Wed, 23 Jan 2019 17:23:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F193E8E0056; Wed, 23 Jan 2019 17:23:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by kanga.kvack.org (Postfix) with ESMTP id B703A8E0047 for ; Wed, 23 Jan 2019 17:23:53 -0500 (EST) Received: by mail-qt1-f199.google.com with SMTP id n39so4243353qtn.18 for ; Wed, 23 Jan 2019 14:23:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=34YO05DWvZaBCpecq87OHHtqkWxKN3JMaZNkpjw+OvE=; b=DwH/48u4sQX62ClPyx+MUU8BX6GE26t2acAhWwlgVd9DISW9rC9mb92SDnTclZq73M aDbI7Q25MsnLXbHmPDUct1+q78qRI55XlzCuQVgqbXrPxA0dl24G485z55d/ZdG/6KDf v7OF9alq4K4BsncwArNtiNiIxY1JEmwQIuZHqUrTX7qVzMjdrHwWximyAkdOS1hWFUXD ZQt2w/oDcJJT0DbFWikCy/xKQYcChxg5L4uoJTVsetKVq9yDG4x/nyZdRsw7ozPa8O4f AMzSEK2FCoZlSax5d+IMesHrrpaqwaFtNmDFh6e2Y4alW3qTzc9MxylWps5dJ0CLHShh FefQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukegj5w36pISwRwDVpl2WofXsY79U6RaV+Pj2y2PgU1BNQ86UhBk 2IJflBcc6QuwfzQyDqsZf4Gx27po2dJQZqIjUUfcALjka+qReewPC0Wo2bJRWEhIMpw1iKniuoj AogoBv0Fhd0SEe0GDU3riLSrY+besE2SuiOIHvZvBOkof5/ZImPNeNElf1G5mbB1FSw== X-Received: by 2002:a37:9906:: with SMTP id b6mr3502733qke.208.1548282233509; Wed, 23 Jan 2019 14:23:53 -0800 (PST) X-Google-Smtp-Source: ALg8bN53tt8wz8lyZLBFP7iy4aUT1lmixNDy+L94+LwAU1gLaCV7lZ3ktmnUHJqStozMJHI8KpWP X-Received: by 2002:a37:9906:: with SMTP id b6mr3502719qke.208.1548282233043; Wed, 23 Jan 2019 14:23:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548282233; cv=none; d=google.com; s=arc-20160816; b=HS0a4nzM2WnvDdQ0j+3wbYiGhybXr/pxwfLq/OPd/vj0O1kbWzrLbG9JsGSjpmMb5q ooriEu3cWgwS/1w4Jm+ZlW40kRnz8R5Fvr7o0x1GWsa+HlQ9eUswW4RtBzbK7sHb/dg9 kTbp8yuLlFvr87zU7tXINYIQqEsiE0MVc9IlwhEJ1Q2gutTB5j98jtK7ujskU96KGd4Y BdwSY3f0qUhiFAnYtpiEUArCTvyGsZID3PcmsiJdoRku5hIRQHsF8KVm4JnTzf4JxTKJ kyWoD3yBwHqfnuzGEsmlPMtFCuF1LlwEaNYuSpm4mamHT41ZeRC80jskcRvIyRTYj322 HWsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=34YO05DWvZaBCpecq87OHHtqkWxKN3JMaZNkpjw+OvE=; b=rQfpShvetw755d0C2pQHoP6cLeWkU2y3UxrCCBorumvuWWE/OAsJm/eNdPWOXEPXh8 k0WRgiJNGPa/dnED/OHPolBrFImT24oalwESl+NJTtN5jVPQNaTVRKS5xb3QFd0enrkl iYDyLoDcgatk6oM6BJ7OEsrVnpl5Pyny+AK+T9C0pExrryIpXmXiZjNtE5rCu/4U0m0M oaq3K4AQwr3YmzAFri6KME37p+ECyzkAGuPUMVOs/rtL5mMNooYMxs/KG2iVlOJtJTaM hsb1gX/wElk8XjvreY8UJswfnjtQQjJudBjrYg8s2TOuAV7vBG0dpFeBysoZpIpAkoPZ Rx3g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id f20si323926qtm.242.2019.01.23.14.23.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jan 2019 14:23:53 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 067D8432C1; Wed, 23 Jan 2019 22:23:52 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-127.rdu2.redhat.com [10.10.120.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5CF0E5D965; Wed, 23 Jan 2019 22:23:49 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Jan Kara , Felix Kuehling , Jason Gunthorpe , Matthew Wilcox , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-fsdevel@vger.kernel.org, Arnd Bergmann Subject: [PATCH v4 8/9] gpu/drm/i915: optimize out the case when a range is updated to read only Date: Wed, 23 Jan 2019 17:23:14 -0500 Message-Id: <20190123222315.1122-9-jglisse@redhat.com> In-Reply-To: <20190123222315.1122-1-jglisse@redhat.com> References: <20190123222315.1122-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 23 Jan 2019 22:23:52 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse When range of virtual address is updated read only and corresponding user ptr object are already read only it is pointless to do anything. Optimize this case out. Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Jan Kara Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Andrew Morton Cc: Matthew Wilcox Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org Cc: Arnd Bergmann --- drivers/gpu/drm/i915/i915_gem_userptr.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c index 9558582c105e..23330ac3d7ea 100644 --- a/drivers/gpu/drm/i915/i915_gem_userptr.c +++ b/drivers/gpu/drm/i915/i915_gem_userptr.c @@ -59,6 +59,7 @@ struct i915_mmu_object { struct interval_tree_node it; struct list_head link; struct work_struct work; + bool read_only; bool attached; }; @@ -119,6 +120,7 @@ static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, container_of(_mn, struct i915_mmu_notifier, mn); struct i915_mmu_object *mo; struct interval_tree_node *it; + bool update_to_read_only; LIST_HEAD(cancelled); unsigned long end; @@ -128,6 +130,8 @@ static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, /* interval ranges are inclusive, but invalidate range is exclusive */ end = range->end - 1; + update_to_read_only = mmu_notifier_range_update_to_read_only(range); + spin_lock(&mn->lock); it = interval_tree_iter_first(&mn->objects, range->start, end); while (it) { @@ -145,6 +149,17 @@ static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, * object if it is not in the process of being destroyed. */ mo = container_of(it, struct i915_mmu_object, it); + + /* + * If it is already read only and we are updating to + * read only then we do not need to change anything. + * So save time and skip this one. + */ + if (update_to_read_only && mo->read_only) { + it = interval_tree_iter_next(it, range->start, end); + continue; + } + if (kref_get_unless_zero(&mo->obj->base.refcount)) queue_work(mn->wq, &mo->work); @@ -270,6 +285,7 @@ i915_gem_userptr_init__mmu_notifier(struct drm_i915_gem_object *obj, mo->mn = mn; mo->obj = obj; mo->it.start = obj->userptr.ptr; + mo->read_only = i915_gem_object_is_readonly(obj); mo->it.last = obj->userptr.ptr + obj->base.size - 1; INIT_WORK(&mo->work, cancel_userptr); From patchwork Wed Jan 23 22:23:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10778015 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BE1301399 for ; Wed, 23 Jan 2019 22:23:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AE43F2C4F8 for ; Wed, 23 Jan 2019 22:23:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A26532C4FF; Wed, 23 Jan 2019 22:23:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 090B92C4F8 for ; Wed, 23 Jan 2019 22:23:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D94638E0057; Wed, 23 Jan 2019 17:23:56 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C54758E0047; Wed, 23 Jan 2019 17:23:56 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B1BC38E0057; Wed, 23 Jan 2019 17:23:56 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by kanga.kvack.org (Postfix) with ESMTP id 792758E0047 for ; Wed, 23 Jan 2019 17:23:56 -0500 (EST) Received: by mail-qt1-f199.google.com with SMTP id p24so4368941qtl.2 for ; Wed, 23 Jan 2019 14:23:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=5ej6STBe/ObX0owf3bf30yvaziyxsISS35lOCKy3kXM=; b=KapB+W64wGTaaHgpdVgZtDJdTHZ/9asO3SME5kr869t1qt7PbYXqy6rWm373EgP0m7 kt3KckJ8lB5QNhMrraZIu1GBjs81e/tlgr21rCTPxkfVYMyrmRtZ6k2y6IJEyhUNoTQ+ YX8JQEaWLCCjCpRf9+GY21NXA4IIoOeAnjrUkm7Du8ckwSeZ39ta/qCKQ/nga/m1P8Ds bON7HvHFppjUwTF+FKDcvJDwg12/q+zNF9DX9Y4Yb00qDfkc2NqB6qDYXNEjwOq+jIKN DrYtkxTiCrgK4TVGIBb9s/Z5gIuRB0/5x6aZ/xT1bTkkiIirm7yuKdkOsSzuLNy92bqj UuWw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukfe6RCDH/MypI69wjVKDsiN2uhzsEQxnX0DFfa+sgb2T5Vc8ymn PLC5jwuSrJYFa7RYPAxEndry1H4LgBELCLt3gH+y/k+9acIFMpYBFZLINujmP03yO7jG/KrKtaN wfvUSC6c67tLDvWa98inWSb/Hq/kfHNUV1Iq4/V3a7g/1D25KyPfsezh3Ukb7quHAVw== X-Received: by 2002:ac8:2211:: with SMTP id o17mr4446899qto.170.1548282236279; Wed, 23 Jan 2019 14:23:56 -0800 (PST) X-Google-Smtp-Source: ALg8bN7SXfjG09j3PKcYYIGQy80cLksEH1ryt/Ce07wCvHKHepleMhsr8q+9mZBfnI/qDY1g1KvG X-Received: by 2002:ac8:2211:: with SMTP id o17mr4446878qto.170.1548282235824; Wed, 23 Jan 2019 14:23:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548282235; cv=none; d=google.com; s=arc-20160816; b=nSbG97p060HAw6cUmQuxbA4gI39/+E21W9BdfDrrlPc722GD+KNKnnsqgcEC4YgOS6 W6Q/2hVwQ380FOX8UbY3RXKCJc5gx3ljxOYY+e0yeiCYuv8Z4ZG7wApeXP3maj/3kWmT ErbcZjHHFvWSyE2sxGnifQ8ZudizP3lXJ349tzf/yhGmXNIxgWcL36GkLyWC9nrJ7gMq 5UqfrcE7vAEECaLdQkAC1JYfQi3MTXLKv/pTK5A0u+iBLAi6ciuBLbgC4shKUzzJJMnP zdJx3IhgfTBSvd8sGCCJSd5NQ6194lj09UXoINOiBKQrtLIFYWyohu5MiYshJPjYUbSn ja/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=5ej6STBe/ObX0owf3bf30yvaziyxsISS35lOCKy3kXM=; b=VCSclwf+J7IXrt+tJJQ1fPRBGbSDuk4zq7C7uynDR5s5FGrtIPxlMKhMxO4K4SVYFm BflYcfzA60Vr3SKXcKQaIR6YG9zZL0hXflk8kVkf4DommmT0N3vMfXiFETsgASAMupcu h9bdCvkVdj1GCGI/1u0W/eT3sJMKgDLbqG5htYlVYVimoaEubUbq639fU+iVmNVyNohF Rr5Roobmtx0+9mCzMIbvxpkpkqnOZYaWxm8Vesq70/gKPy1A3IG3EsD5S9CsUgd5uesM ytX1YutDO0Y5LZAjNUs2TlaJiaq5qu4u+AqodTEb46aAnbD4Sq8bfGNaaSaPdyqHGHrE UyDA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id c4si3776096qtj.64.2019.01.23.14.23.55 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jan 2019 14:23:55 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D387F432A0; Wed, 23 Jan 2019 22:23:54 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-127.rdu2.redhat.com [10.10.120.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3606D6A685; Wed, 23 Jan 2019 22:23:52 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Jan Kara , Felix Kuehling , Jason Gunthorpe , Matthew Wilcox , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-fsdevel@vger.kernel.org, Arnd Bergmann Subject: [PATCH v4 9/9] RDMA/umem_odp: optimize out the case when a range is updated to read only Date: Wed, 23 Jan 2019 17:23:15 -0500 Message-Id: <20190123222315.1122-10-jglisse@redhat.com> In-Reply-To: <20190123222315.1122-1-jglisse@redhat.com> References: <20190123222315.1122-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 23 Jan 2019 22:23:55 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse When range of virtual address is updated read only and corresponding user ptr object are already read only it is pointless to do anything. Optimize this case out. Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Jan Kara Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Andrew Morton Cc: Matthew Wilcox Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org Cc: Arnd Bergmann --- drivers/infiniband/core/umem_odp.c | 22 +++++++++++++++++++--- include/rdma/ib_umem_odp.h | 1 + 2 files changed, 20 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/core/umem_odp.c b/drivers/infiniband/core/umem_odp.c index a4ec43093cb3..fa4e7fdcabfc 100644 --- a/drivers/infiniband/core/umem_odp.c +++ b/drivers/infiniband/core/umem_odp.c @@ -140,8 +140,15 @@ static void ib_umem_notifier_release(struct mmu_notifier *mn, static int invalidate_range_start_trampoline(struct ib_umem_odp *item, u64 start, u64 end, void *cookie) { + bool update_to_read_only = *((bool *)cookie); + ib_umem_notifier_start_account(item); - item->umem.context->invalidate_range(item, start, end); + /* + * If it is already read only and we are updating to read only then we + * do not need to change anything. So save time and skip this one. + */ + if (!update_to_read_only || !item->read_only) + item->umem.context->invalidate_range(item, start, end); return 0; } @@ -150,6 +157,7 @@ static int ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn, { struct ib_ucontext_per_mm *per_mm = container_of(mn, struct ib_ucontext_per_mm, mn); + bool update_to_read_only; if (range->blockable) down_read(&per_mm->umem_rwsem); @@ -166,10 +174,13 @@ static int ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn, return 0; } + update_to_read_only = mmu_notifier_range_update_to_read_only(range); + return rbt_ib_umem_for_each_in_range(&per_mm->umem_tree, range->start, range->end, invalidate_range_start_trampoline, - range->blockable, NULL); + range->blockable, + &update_to_read_only); } static int invalidate_range_end_trampoline(struct ib_umem_odp *item, u64 start, @@ -363,6 +374,9 @@ struct ib_umem_odp *ib_alloc_odp_umem(struct ib_ucontext_per_mm *per_mm, goto out_odp_data; } + /* Assume read only at first, each time GUP is call this is updated. */ + odp_data->read_only = true; + odp_data->dma_list = vzalloc(array_size(pages, sizeof(*odp_data->dma_list))); if (!odp_data->dma_list) { @@ -619,8 +633,10 @@ int ib_umem_odp_map_dma_pages(struct ib_umem_odp *umem_odp, u64 user_virt, goto out_put_task; } - if (access_mask & ODP_WRITE_ALLOWED_BIT) + if (access_mask & ODP_WRITE_ALLOWED_BIT) { + umem_odp->read_only = false; flags |= FOLL_WRITE; + } start_idx = (user_virt - ib_umem_start(umem)) >> page_shift; k = start_idx; diff --git a/include/rdma/ib_umem_odp.h b/include/rdma/ib_umem_odp.h index 0b1446fe2fab..8256668c6170 100644 --- a/include/rdma/ib_umem_odp.h +++ b/include/rdma/ib_umem_odp.h @@ -76,6 +76,7 @@ struct ib_umem_odp { struct completion notifier_completion; int dying; struct work_struct work; + bool read_only; }; static inline struct ib_umem_odp *to_ib_umem_odp(struct ib_umem *umem)