From patchwork Tue Feb 19 20:04:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10820705 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B92D0184E for ; Tue, 19 Feb 2019 20:04:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A59162C8DC for ; Tue, 19 Feb 2019 20:04:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 995D02CA7B; Tue, 19 Feb 2019 20:04:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0C4A72CA7B for ; Tue, 19 Feb 2019 20:04:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7CAD78E0005; Tue, 19 Feb 2019 15:04:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 77D868E0002; Tue, 19 Feb 2019 15:04:55 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F30B8E0005; Tue, 19 Feb 2019 15:04:55 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by kanga.kvack.org (Postfix) with ESMTP id 34A598E0002 for ; Tue, 19 Feb 2019 15:04:55 -0500 (EST) Received: by mail-qt1-f199.google.com with SMTP id e31so20691255qtb.22 for ; Tue, 19 Feb 2019 12:04:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=vhT/JRh5Ora1ssBCghfNtXJPPBh/LCkkIILPHCIsWjs=; b=U7HwqYJMYPwhaRlK5t8+dxSWFn5bLeLe1a7Dmv4CPTqH6BhePz3YBktH+NKW2DXIMh LVom+4DrZcvLLWr9wh6JXxL77Tp/KJ4LgCRWIQR/zlX0DHNamemvypaqzDxKKMgAIjxP HhsMgNFM/03KPU7smaTA481dFJu5DdecYbxAMWpIhUn+Nqcv5Px1llsMQYVPyqcw1WUV uJHHp+grDj0MzJLduVP9yi8IOVGBM9n1iEEhj4UlcDGU3FeFAThdsMVJ7bEQeHKIC+wk Hdbh8PV65BavWI+yKfxbg5GqA2hqtTboQeaEQqBzKp6plOs7BqSSi2GxIesixfO4rSUA EZow== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AHQUAubonPKsp7cj7KXPJ9Xe2eOznYuRacoHWOvCMySv45u2aPdeA16N fenr3z3ppEVwI9UU2WmyQueXi0v6uK+samPRYB6UYFSdu0Km0SydcAWoN5f6PdIaQwRzAwVBGiJ zeVngCT5dHO0joFkg07XPtPuHg370JQKQn2e0TAB56LeOK9AyDBSOnHHA86h0367Pqg== X-Received: by 2002:a0c:e1c9:: with SMTP id v9mr23859754qvl.186.1550606695015; Tue, 19 Feb 2019 12:04:55 -0800 (PST) X-Google-Smtp-Source: AHgI3IZYt5kwL+Ql/RzVXU0yYNH5XUHO+EXh+v0tpWj417ywk0aCWR5s6/1gAhBfybko3IqH+oJa X-Received: by 2002:a0c:e1c9:: with SMTP id v9mr23859365qvl.186.1550606689708; Tue, 19 Feb 2019 12:04:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550606689; cv=none; d=google.com; s=arc-20160816; b=jgi0+pwTJFtzwuG1c03z37y7K4Yd23PGMFYxjUfOtOawpbrQ/SJNV4qVtW6c2BiwNG Q4TBrOKZdXBdnAFhaWvoFcUYck3aB8S6TY4XlEmLSUD7ABUEs/DdvysSuz2Eld87ap8x djPdkU7IzQbJOiUgx5QduplYmtwq4xz1wFEJHKXXyzpOjqGEuXHGRh7fj9aRVoYgDceU X//kIaoo4WsPPu1WOY8P2NgX/mcEf8VIrmdrQKybvEzgvXxRfQTGbjhKJNCvcTCxf+9i z4l6EB2+qR/nis/WWYsVvC5E+97V1Kmz1oxDqSP6TTbq9HN6LRZ0RchdTjpgursBxQIV TG6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=vhT/JRh5Ora1ssBCghfNtXJPPBh/LCkkIILPHCIsWjs=; b=QdnviGAJc8jzycK0hq3MTbn3XuowIXUvS2zXU9/WY7K65WdxPrh0NA2eWRorWNrh3o n8/Qg3HLzt3/hjV4GjfexVJKuzuOPGHwCRZJDYYeKU9HRdKWgtGx5+GnGshnYWRdltSN 9UREOTNL0B8MhEercSpvc/5MSxFC6kVQQ5TNN+zTxLLKi6e/qQ7jZrpKTesPLPYnVaB+ kT+vso3xGRr1+KbEMLrUgGSNSRSrmEA9DxIEdxy53ueL8wlidq2Hx7d25tkKsvyAAFBy fDQYC5oqRLgLJpd20bM6ql3x1xxSuFuSOUt2ko5b++iAtgOb53RoNlOFtM5239sjr8HW kZEQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id t53si1076265qtc.72.2019.02.19.12.04.49 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 19 Feb 2019 12:04:49 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6A42A369A0; Tue, 19 Feb 2019 20:04:48 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-134.rdu2.redhat.com [10.10.122.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5ABE86013D; Tue, 19 Feb 2019 20:04:45 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org, Andrew Morton Cc: linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-fsdevel@vger.kernel.org, Arnd Bergmann Subject: [PATCH v5 1/9] mm/mmu_notifier: helper to test if a range invalidation is blockable Date: Tue, 19 Feb 2019 15:04:22 -0500 Message-Id: <20190219200430.11130-2-jglisse@redhat.com> In-Reply-To: <20190219200430.11130-1-jglisse@redhat.com> References: <20190219200430.11130-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 19 Feb 2019 20:04:48 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Simple helpers to test if range invalidation is blockable. Latter patches use cocinnelle to convert all direct dereference of range-> blockable to use this function instead so that we can convert the blockable field to an unsigned for more flags. Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Andrew Morton Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org Cc: Arnd Bergmann Reviewed-by: Ralph Campbell --- include/linux/mmu_notifier.h | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 4050ec1c3b45..e630def131ce 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -226,6 +226,12 @@ extern void __mmu_notifier_invalidate_range_end(struct mmu_notifier_range *r, extern void __mmu_notifier_invalidate_range(struct mm_struct *mm, unsigned long start, unsigned long end); +static inline bool +mmu_notifier_range_blockable(const struct mmu_notifier_range *range) +{ + return range->blockable; +} + static inline void mmu_notifier_release(struct mm_struct *mm) { if (mm_has_notifiers(mm)) @@ -455,6 +461,11 @@ static inline void _mmu_notifier_range_init(struct mmu_notifier_range *range, #define mmu_notifier_range_init(range, mm, start, end) \ _mmu_notifier_range_init(range, start, end) +static inline bool +mmu_notifier_range_blockable(const struct mmu_notifier_range *range) +{ + return true; +} static inline int mm_has_notifiers(struct mm_struct *mm) { From patchwork Tue Feb 19 20:04:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10820713 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EAC4714E1 for ; Tue, 19 Feb 2019 20:05:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D67C92CA7B for ; Tue, 19 Feb 2019 20:05:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C95382C8BD; Tue, 19 Feb 2019 20:05:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0E2B82CA7B for ; Tue, 19 Feb 2019 20:05:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8B8BD8E0006; Tue, 19 Feb 2019 15:04:56 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 83EB88E0002; Tue, 19 Feb 2019 15:04:56 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 72CF58E0006; Tue, 19 Feb 2019 15:04:56 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by kanga.kvack.org (Postfix) with ESMTP id 43DC48E0002 for ; Tue, 19 Feb 2019 15:04:56 -0500 (EST) Received: by mail-qt1-f200.google.com with SMTP id 35so21129048qtq.5 for ; Tue, 19 Feb 2019 12:04:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ZuWLtcN1UYxcz3AFIn4HBVvwNrzebalGYVzIwEYwbhQ=; b=TX6hDk3CyJIff8TQvakd96YQYtUZ+6GaqyzDVT9KM5uNSyE5TxjFsCFpGYCB5OnJ1B S62/tiNQ5gSaOJ3D7N6l0Bbe9l6PYRH5lBv4TCAc31ik4k5jQX9zdk04EWymbHWHJigq VkWy03xzMb876igqIm0h8D9hV8plaQLxWha96a5fYQV1DWqfVrWpncQSmtdfvvIg9o4R fTfePS8v17LSYQ3NXpTn1zg7Jv1R7NRvlj2/bAwZZmvNS5nkxoEtX9oneZ6OWplgo8PQ QacZmPnzOCfevHvruxLrIuxN4kTJo7DbzrNYN7bBFzgLQfk9cz6Uv1dom78P9xGHDYlk xH5Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AHQUAuaWSo5tHWozpukdTS8VDRjw6uroucZBY2vdz5/WemlpzH7kNlad 5KcYhm5DYvigSnbvUqE3poq/EqISu4R7kHLFbLVVLcSkxgaKq1cX2Rpawgf7dcyXsgB4XgegGZQ ZaV46zqC6o7prRwmI7OTnFPY9ql1Z3wa9dfBR42CXJeYdRJY74K6/1uxCwLz2aeNuug== X-Received: by 2002:a37:c04a:: with SMTP id o71mr21521356qki.234.1550606696025; Tue, 19 Feb 2019 12:04:56 -0800 (PST) X-Google-Smtp-Source: AHgI3IaTX1AgDkMqMCRexySUllMAcXodBeQ+AD5IADzyE9H32BZa0XCcSm546qTzDcbiGbfcz8tH X-Received: by 2002:a37:c04a:: with SMTP id o71mr21521287qki.234.1550606694908; Tue, 19 Feb 2019 12:04:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550606694; cv=none; d=google.com; s=arc-20160816; b=fhOLhSBTR51Kjep+bC0uGs8mgP44jJ6615E/Lw8tKiyNnSjGW5qxOqNFj/8ZLOv5WW qc3pnloUi5AEvo5hLPsFVE71kOWdVcKvEnjZQWCdN9sOmBJ/hDmKjCAy8o9fpQA0Y8Lo zQT5jDFl1sQf7fg+XRyfI2NtO6Jem31joFLn3HG2nn6VyF1+QTXFU0YvIVr/XGiUMG8h TOGb30pbG/b8GkfqErC0X6kSBdUpAy87Xb3/x5J26sB+TOKkqCOM5Dh2Ysr6y9uYwSFy RfDZiC7CGFCzpAhTTYP7XMhRjol1AofQvj6SJxyc0SKU5HdypNrtYUp4mU8YuVQC4KvT aB+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=ZuWLtcN1UYxcz3AFIn4HBVvwNrzebalGYVzIwEYwbhQ=; b=d87+D6R1UJcYb5I95+hIBeTsIJuULS/WGwbyNqIHnHcK4nmZlw1Hs6HSnJOEweJkqh 6Dv8tP/71hbHdnHyFkWsAvvZmZzdjtc9Ax3LnhuyXBNZHlEQn39N1Hht1YJWGgSKZLAr ZRrpotsXgeMwF0YFoTSi2Svf7Z/n04Nl2Ca0j9YEcCRgl0wRehoVClBYguVhWgFweFS4 Pj2KV/xXQ4nEqlaBv7OvuSNBIqpEZKv7X+4A/70LuiZLiJQ/iFzQ2bAMEXwDR4Mv8Su6 q59sF38/OyQutQNM29OI9sQjo4CtQj4LZQxfnww4gKHM0PNbsf8gNaxIdQLr9uqlvqvd qXKg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id a49si1425133qva.62.2019.02.19.12.04.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 19 Feb 2019 12:04:54 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6C95F29A5D; Tue, 19 Feb 2019 20:04:53 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-134.rdu2.redhat.com [10.10.122.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 94AB86013C; Tue, 19 Feb 2019 20:04:48 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org, Andrew Morton Cc: linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v5 2/9] mm/mmu_notifier: convert user range->blockable to helper function Date: Tue, 19 Feb 2019 15:04:23 -0500 Message-Id: <20190219200430.11130-3-jglisse@redhat.com> In-Reply-To: <20190219200430.11130-1-jglisse@redhat.com> References: <20190219200430.11130-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 19 Feb 2019 20:04:54 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Use the mmu_notifier_range_blockable() helper function instead of directly dereferencing the range->blockable field. This is done to make it easier to change the mmu_notifier range field. This patch is the outcome of the following coccinelle patch: %<------------------------------------------------------------------- @@ identifier I1, FN; @@ FN(..., struct mmu_notifier_range *I1, ...) { <... -I1->blockable +mmu_notifier_range_blockable(I1) ...> } ------------------------------------------------------------------->% spatch --in-place --sp-file blockable.spatch --dir . Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann Reviewed-by: Ralph Campbell --- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 8 ++++---- drivers/gpu/drm/i915/i915_gem_userptr.c | 2 +- drivers/gpu/drm/radeon/radeon_mn.c | 4 ++-- drivers/infiniband/core/umem_odp.c | 5 +++-- drivers/xen/gntdev.c | 6 +++--- mm/hmm.c | 6 +++--- mm/mmu_notifier.c | 2 +- virt/kvm/kvm_main.c | 3 ++- 8 files changed, 19 insertions(+), 17 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c index 3e6823fdd939..58ed401c5996 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c @@ -256,14 +256,14 @@ static int amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, /* TODO we should be able to split locking for interval tree and * amdgpu_mn_invalidate_node */ - if (amdgpu_mn_read_lock(amn, range->blockable)) + if (amdgpu_mn_read_lock(amn, mmu_notifier_range_blockable(range))) return -EAGAIN; it = interval_tree_iter_first(&amn->objects, range->start, end); while (it) { struct amdgpu_mn_node *node; - if (!range->blockable) { + if (!mmu_notifier_range_blockable(range)) { amdgpu_mn_read_unlock(amn); return -EAGAIN; } @@ -299,7 +299,7 @@ static int amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, /* notification is exclusive, but interval is inclusive */ end = range->end - 1; - if (amdgpu_mn_read_lock(amn, range->blockable)) + if (amdgpu_mn_read_lock(amn, mmu_notifier_range_blockable(range))) return -EAGAIN; it = interval_tree_iter_first(&amn->objects, range->start, end); @@ -307,7 +307,7 @@ static int amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, struct amdgpu_mn_node *node; struct amdgpu_bo *bo; - if (!range->blockable) { + if (!mmu_notifier_range_blockable(range)) { amdgpu_mn_read_unlock(amn); return -EAGAIN; } diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c index 1d3f9a31ad61..777b3f8727e7 100644 --- a/drivers/gpu/drm/i915/i915_gem_userptr.c +++ b/drivers/gpu/drm/i915/i915_gem_userptr.c @@ -122,7 +122,7 @@ userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, while (it) { struct drm_i915_gem_object *obj; - if (!range->blockable) { + if (!mmu_notifier_range_blockable(range)) { ret = -EAGAIN; break; } diff --git a/drivers/gpu/drm/radeon/radeon_mn.c b/drivers/gpu/drm/radeon/radeon_mn.c index b3019505065a..c9bd1278f573 100644 --- a/drivers/gpu/drm/radeon/radeon_mn.c +++ b/drivers/gpu/drm/radeon/radeon_mn.c @@ -133,7 +133,7 @@ static int radeon_mn_invalidate_range_start(struct mmu_notifier *mn, /* TODO we should be able to split locking for interval tree and * the tear down. */ - if (range->blockable) + if (mmu_notifier_range_blockable(range)) mutex_lock(&rmn->lock); else if (!mutex_trylock(&rmn->lock)) return -EAGAIN; @@ -144,7 +144,7 @@ static int radeon_mn_invalidate_range_start(struct mmu_notifier *mn, struct radeon_bo *bo; long r; - if (!range->blockable) { + if (!mmu_notifier_range_blockable(range)) { ret = -EAGAIN; goto out_unlock; } diff --git a/drivers/infiniband/core/umem_odp.c b/drivers/infiniband/core/umem_odp.c index 012044f16d1c..3a3f1538d295 100644 --- a/drivers/infiniband/core/umem_odp.c +++ b/drivers/infiniband/core/umem_odp.c @@ -151,7 +151,7 @@ static int ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn, struct ib_ucontext_per_mm *per_mm = container_of(mn, struct ib_ucontext_per_mm, mn); - if (range->blockable) + if (mmu_notifier_range_blockable(range)) down_read(&per_mm->umem_rwsem); else if (!down_read_trylock(&per_mm->umem_rwsem)) return -EAGAIN; @@ -169,7 +169,8 @@ static int ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn, return rbt_ib_umem_for_each_in_range(&per_mm->umem_tree, range->start, range->end, invalidate_range_start_trampoline, - range->blockable, NULL); + mmu_notifier_range_blockable(range), + NULL); } static int invalidate_range_end_trampoline(struct ib_umem_odp *item, u64 start, diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index 5efc5eee9544..9da8f7192f46 100644 --- a/drivers/xen/gntdev.c +++ b/drivers/xen/gntdev.c @@ -526,20 +526,20 @@ static int mn_invl_range_start(struct mmu_notifier *mn, struct gntdev_grant_map *map; int ret = 0; - if (range->blockable) + if (mmu_notifier_range_blockable(range)) mutex_lock(&priv->lock); else if (!mutex_trylock(&priv->lock)) return -EAGAIN; list_for_each_entry(map, &priv->maps, next) { ret = unmap_if_in_range(map, range->start, range->end, - range->blockable); + mmu_notifier_range_blockable(range)); if (ret) goto out_unlock; } list_for_each_entry(map, &priv->freeable_maps, next) { ret = unmap_if_in_range(map, range->start, range->end, - range->blockable); + mmu_notifier_range_blockable(range)); if (ret) goto out_unlock; } diff --git a/mm/hmm.c b/mm/hmm.c index 3c9781037918..a03b5083d880 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -205,9 +205,9 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn, update.start = nrange->start; update.end = nrange->end; update.event = HMM_UPDATE_INVALIDATE; - update.blockable = nrange->blockable; + update.blockable = mmu_notifier_range_blockable(nrange); - if (nrange->blockable) + if (mmu_notifier_range_blockable(nrange)) mutex_lock(&hmm->lock); else if (!mutex_trylock(&hmm->lock)) { ret = -EAGAIN; @@ -222,7 +222,7 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn, } mutex_unlock(&hmm->lock); - if (nrange->blockable) + if (mmu_notifier_range_blockable(nrange)) down_read(&hmm->mirrors_sem); else if (!down_read_trylock(&hmm->mirrors_sem)) { ret = -EAGAIN; diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index 9c884abc7850..abd88c466eb2 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -180,7 +180,7 @@ int __mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) if (_ret) { pr_info("%pS callback failed with %d in %sblockable context.\n", mn->ops->invalidate_range_start, _ret, - !range->blockable ? "non-" : ""); + !mmu_notifier_range_blockable(range) ? "non-" : ""); ret = _ret; } } diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 38df17b7760e..629760c0fb95 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -386,7 +386,8 @@ static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, spin_unlock(&kvm->mmu_lock); ret = kvm_arch_mmu_notifier_invalidate_range(kvm, range->start, - range->end, range->blockable); + range->end, + mmu_notifier_range_blockable(range)); srcu_read_unlock(&kvm->srcu, idx); From patchwork Tue Feb 19 20:04:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10820715 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6EF0B1399 for ; Tue, 19 Feb 2019 20:05:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5CA782CAAF for ; Tue, 19 Feb 2019 20:05:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 500262CBC0; Tue, 19 Feb 2019 20:05:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 48DED2CAAF for ; Tue, 19 Feb 2019 20:05:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 582E18E0007; Tue, 19 Feb 2019 15:04:59 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 507F38E0002; Tue, 19 Feb 2019 15:04:59 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F7158E0007; Tue, 19 Feb 2019 15:04:59 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by kanga.kvack.org (Postfix) with ESMTP id 1911D8E0002 for ; Tue, 19 Feb 2019 15:04:59 -0500 (EST) Received: by mail-qt1-f199.google.com with SMTP id 43so20699347qtz.8 for ; Tue, 19 Feb 2019 12:04:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=y4cTz0AvI8I1jPok9kmJHggRtXPyxbatRSaKY8cn6S8=; b=CjxrCzXSKto9toEYi5qpXqq26qV0pQV/6emn8lbQqsJaAShqOxm3Qe8ZmLtMslx9Em pLXXnDGM1ahfyyhUHGPAh9IySkfK/zeIhxLp+5pNgiFi08mI2LzJegV2cOYDJZfL4oNI dykQxCDKRRPcoF5UueSNnM8Hhca7DiaA2DaahRCeX5LVvePOv5QCCp/MRcVGCjGt9E8H Jfw5RSMI18beaTa252VXpshtRjU8M2G3Vnu60t6ww2syghJHzocroNzho0BxvVSGGCJe uK5x/R28JG1Z0WKjh6moBw98l7KXmwn/AJ58plh8aqVg7f3LUG0tT2/qPNhzsCrqUnXV O4cw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AHQUAua9qMqWBBwuLAQvNhB9sUs4XNmQ8TcJo3J7f7nUVL2gW/rgiWRE NDj9/5Uz6XaaT/gaY6QWgcLZAneTpL8io8jg92myQWOoYLck0ZGXqtEQPo7zwpglj9hNoQYnnPh 2NbUH7/r81/lENkw8YNDQtZd8MqaGvH1Fv01JQl4WAFH4//l10q/ggtZl1htLYt9ceQ== X-Received: by 2002:ac8:3d51:: with SMTP id u17mr24443271qtf.127.1550606698885; Tue, 19 Feb 2019 12:04:58 -0800 (PST) X-Google-Smtp-Source: AHgI3IbJP7snB/dQg0koyC+po8j9DmRcoyH0dvCNTiVUpuDfYxKeh6M56c0yfQckJGBu1CiLtkKn X-Received: by 2002:ac8:3d51:: with SMTP id u17mr24443205qtf.127.1550606698084; Tue, 19 Feb 2019 12:04:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550606698; cv=none; d=google.com; s=arc-20160816; b=rL3lMdqmwf+iB9qMw+KlK4ybX//aL72zws55mK2ebexBR+4ZlVpPMhKClyuangmROx smzYugi7t8R4anSDDbRYD92byrDq6ehwsnWOe253fIU6AbDH+mLH03aI5CVG5r8BKSs7 FEOOcPAKn51KpL+UFuCM9u9EzRpse4v5rCzC9qzDKBB7fg1WwvNMI2ESzGprEQyiHsrM nqgmf9Ujo2FB3KyuUbkC/C5R8MHnQTNsZ7JTxLyADOlqDPMi/GGvYXTLuiIEzqug7ATs aV6LVrZ+anNCYLJkSGiV7eMw0pqlARqF6/iEBeEmgv/DGcUiWeWp+JAhv74U0zrMhdP/ zPAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=y4cTz0AvI8I1jPok9kmJHggRtXPyxbatRSaKY8cn6S8=; b=C+QhqxcHvMjMRI8382BN6+jIZn+nTyGV6t6ulUQwKu2Rsz9Rr+hQy/7IGUxXZ8QmSl QlRnpO9RkPBhxLNKH3Rd9pGLnwmhbDqzV83sZ7fYLjDVzgWu94rxd7gJOwONu9Eu8ZYi Rm5yP+HXbsclFJ6aKynH3Fe0j6VF8QGg9hyTPI28+G+QjeP/DTW4AtSnaLIRltCCMshB omTun2bUspjpqnuAlglC7l5EMg0HR3Z/Ut7XQMqooyM0nVaOsjMZhZdgfAtInE9jfVrA G1sxStJ2GqFO49p6on0+U6awhkruqJ+lisIp6wnvjMZsXVmOW7XHEUtu9vzlU4Bp4sEM koXQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id b54si2286355qvh.217.2019.02.19.12.04.57 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 19 Feb 2019 12:04:58 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 04C1DCA1FD; Tue, 19 Feb 2019 20:04:57 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-134.rdu2.redhat.com [10.10.122.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8FCED60141; Tue, 19 Feb 2019 20:04:53 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org, Andrew Morton Cc: linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v5 3/9] mm/mmu_notifier: convert mmu_notifier_range->blockable to a flags Date: Tue, 19 Feb 2019 15:04:24 -0500 Message-Id: <20190219200430.11130-4-jglisse@redhat.com> In-Reply-To: <20190219200430.11130-1-jglisse@redhat.com> References: <20190219200430.11130-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 19 Feb 2019 20:04:57 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Use an unsigned field for flags other than blockable and convert the blockable field to be one of those flags. Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Andrew Morton Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann Reviewed-by: Ralph Campbell --- include/linux/mmu_notifier.h | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index e630def131ce..c8672c366f67 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -25,11 +25,13 @@ struct mmu_notifier_mm { spinlock_t lock; }; +#define MMU_NOTIFIER_RANGE_BLOCKABLE (1 << 0) + struct mmu_notifier_range { struct mm_struct *mm; unsigned long start; unsigned long end; - bool blockable; + unsigned flags; }; struct mmu_notifier_ops { @@ -229,7 +231,7 @@ extern void __mmu_notifier_invalidate_range(struct mm_struct *mm, static inline bool mmu_notifier_range_blockable(const struct mmu_notifier_range *range) { - return range->blockable; + return (range->flags & MMU_NOTIFIER_RANGE_BLOCKABLE); } static inline void mmu_notifier_release(struct mm_struct *mm) @@ -275,7 +277,7 @@ static inline void mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) { if (mm_has_notifiers(range->mm)) { - range->blockable = true; + range->flags |= MMU_NOTIFIER_RANGE_BLOCKABLE; __mmu_notifier_invalidate_range_start(range); } } @@ -284,7 +286,7 @@ static inline int mmu_notifier_invalidate_range_start_nonblock(struct mmu_notifier_range *range) { if (mm_has_notifiers(range->mm)) { - range->blockable = false; + range->flags &= ~MMU_NOTIFIER_RANGE_BLOCKABLE; return __mmu_notifier_invalidate_range_start(range); } return 0; @@ -331,6 +333,7 @@ static inline void mmu_notifier_range_init(struct mmu_notifier_range *range, range->mm = mm; range->start = start; range->end = end; + range->flags = 0; } #define ptep_clear_flush_young_notify(__vma, __address, __ptep) \ From patchwork Tue Feb 19 20:04:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10820727 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3970014E1 for ; Tue, 19 Feb 2019 20:05:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2750E2CAAF for ; Tue, 19 Feb 2019 20:05:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1BB9A2CB8E; Tue, 19 Feb 2019 20:05:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7668C2CAAF for ; Tue, 19 Feb 2019 20:05:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D95628E0008; Tue, 19 Feb 2019 15:05:11 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D6B728E0002; Tue, 19 Feb 2019 15:05:11 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5BBD8E0008; Tue, 19 Feb 2019 15:05:11 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by kanga.kvack.org (Postfix) with ESMTP id 9E2EF8E0002 for ; Tue, 19 Feb 2019 15:05:11 -0500 (EST) Received: by mail-qk1-f199.google.com with SMTP id b187so620485qkf.3 for ; Tue, 19 Feb 2019 12:05:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=d3L8LEk8AB5D2JhmIyE9exlLcnknyD9XBU5141tFhtg=; b=ZPWqLWhIT+IN2zn1urX4b8MkeEj0V9qvXUfgCOqc4l89ablYCfVEjwEEK55krRKuZe 23t7XjzCRAT1N/32yV+Wk0/jW3YQpMXwnuB0d/Bxqf2t7sZN2FCTAqTVGtz/tHr8nkL6 X5jVRHVBRTstSNOshjjeCx+/mvLbvY7eRrdVsf9Hd2PufhyTZ07dXwnP2hhVzgzw/vM1 hYgjVud7mfr9Z2F3ly5N/2p9b4h5T/onWS1FW+F3OpsGRTG7Uesc3u5bKjTYr7MpBrW4 d0f2SsUtJzJgUUOhPv0zxi3nvQla7CSzBqgwIWkxfIOw0vEFShaoHv3/NOFP3bxfwv7G SICg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AHQUAuZJiRxfuf3s0Qb2QdlI4jedotKmvWfvQ3ftu+TjkHdHJS+Apx6K EXVvkD2sJe3epRwefWQbc0ZAhBUKQdStWmbNuPH8swOy3hv6YaGE7N8X3Y6DjpEh553tgq1T77l fpPNCj8MfUHkgGxdBcQmtBD+E7DAF8w2G5i1sTnVxb8Tc96CdQ4LWaZJ9uec886ihGA== X-Received: by 2002:a0c:b626:: with SMTP id f38mr22923248qve.166.1550606711340; Tue, 19 Feb 2019 12:05:11 -0800 (PST) X-Google-Smtp-Source: AHgI3IamSLLCTOYTovvsjQtsm2kJ0cte4BRI4zSwo1a6s09HBrE/iphhFvSf5IS/TqwWpPFFgppN X-Received: by 2002:a0c:b626:: with SMTP id f38mr22923167qve.166.1550606710339; Tue, 19 Feb 2019 12:05:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550606710; cv=none; d=google.com; s=arc-20160816; b=ultoW2sAyCrsPbyTgAJnTCDs7TX5XYZ6Mpw84FZuCqAgwWGnT4Y7U7GXvtneXcXqMT lXAW2wh5aIkdackllU0jZhqOhEuN6OzrOUr/8XCWgRn4Qo3HwaXdOhxYeXcxOV3yblgu 5ojT43MfVWYldee45+pmdYHtQ4CV9dPsn1CkNQD9zzZOjFmRnpB5fx1e3un9reKjn12J 7GYgGMqWDuUG0TlMciEOmMF8eFcrdV0rUP2t/9YxhIzRZP4I2PICP3cIlXVlj57WYdOa fUo0woIvrVlCyIXE7VSqru5aNz/6daWoownJodN/qpapB/+qxrRiRhj5UqgTm2Z8/LsJ Y4Eg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=d3L8LEk8AB5D2JhmIyE9exlLcnknyD9XBU5141tFhtg=; b=CateUk9GBW2UxpFJ0Kn00l1c937ZcOEggx1Y5tk1MxDEzvW2xYpKtdCZpL+zK9JxMy ho4TpJp1QHhm6D6eiyceW0uZXFNl6Su2xXY2N2yGfB97IstG6fCSPD0Ri+tXHwcRxC8c yBMEeNpVQLzZBYBsfXPckbTtzzqi/mZ/z56vsNWGlanvr+PdJpV1fd5hYON5YZ3vBMCi Po/NOIZkwMCCMxbjh/cD6DwpMefFaU3lHzVP4Qs+PeLy8ZS/Wai6Y3NgAYabWRWQatNX erGk1PmG/5bNpe30lN2i06qgjU7MDFKYdsYogqZw8ilf0xjBCDslXE+32V7pc844Itsb YIfw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id r46si2248581qta.43.2019.02.19.12.05.10 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 19 Feb 2019 12:05:10 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DB5ADC1047E2; Tue, 19 Feb 2019 20:05:08 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-134.rdu2.redhat.com [10.10.122.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 32198600C8; Tue, 19 Feb 2019 20:04:57 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org, Andrew Morton Cc: linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v5 4/9] mm/mmu_notifier: contextual information for event enums Date: Tue, 19 Feb 2019 15:04:25 -0500 Message-Id: <20190219200430.11130-5-jglisse@redhat.com> In-Reply-To: <20190219200430.11130-1-jglisse@redhat.com> References: <20190219200430.11130-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 19 Feb 2019 20:05:09 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse CPU page table update can happens for many reasons, not only as a result of a syscall (munmap(), mprotect(), mremap(), madvise(), ...) but also as a result of kernel activities (memory compression, reclaim, migration, ...). This patch introduce a set of enums that can be associated with each of the events triggering a mmu notifier. Latter patches take advantages of those enum values. - UNMAP: munmap() or mremap() - CLEAR: page table is cleared (migration, compaction, reclaim, ...) - PROTECTION_VMA: change in access protections for the range - PROTECTION_PAGE: change in access protections for page in the range - SOFT_DIRTY: soft dirtyness tracking Being able to identify munmap() and mremap() from other reasons why the page table is cleared is important to allow user of mmu notifier to update their own internal tracking structure accordingly (on munmap or mremap it is not longer needed to track range of virtual address as it becomes invalid). Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann --- include/linux/mmu_notifier.h | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index c8672c366f67..2386e71ac1b8 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -10,6 +10,36 @@ struct mmu_notifier; struct mmu_notifier_ops; +/** + * enum mmu_notifier_event - reason for the mmu notifier callback + * @MMU_NOTIFY_UNMAP: either munmap() that unmap the range or a mremap() that + * move the range + * + * @MMU_NOTIFY_CLEAR: clear page table entry (many reasons for this like + * madvise() or replacing a page by another one, ...). + * + * @MMU_NOTIFY_PROTECTION_VMA: update is due to protection change for the range + * ie using the vma access permission (vm_page_prot) to update the whole range + * is enough no need to inspect changes to the CPU page table (mprotect() + * syscall) + * + * @MMU_NOTIFY_PROTECTION_PAGE: update is due to change in read/write flag for + * pages in the range so to mirror those changes the user must inspect the CPU + * page table (from the end callback). + * + * @MMU_NOTIFY_SOFT_DIRTY: soft dirty accounting (still same page and same + * access flags). User should soft dirty the page in the end callback to make + * sure that anyone relying on soft dirtyness catch pages that might be written + * through non CPU mappings. + */ +enum mmu_notifier_event { + MMU_NOTIFY_UNMAP = 0, + MMU_NOTIFY_CLEAR, + MMU_NOTIFY_PROTECTION_VMA, + MMU_NOTIFY_PROTECTION_PAGE, + MMU_NOTIFY_SOFT_DIRTY, +}; + #ifdef CONFIG_MMU_NOTIFIER /* From patchwork Tue Feb 19 20:04:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10820743 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D2F21399 for ; Tue, 19 Feb 2019 20:05:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 71E542CC49 for ; Tue, 19 Feb 2019 20:05:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6577C2D394; Tue, 19 Feb 2019 20:05:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 089922CC49 for ; Tue, 19 Feb 2019 20:05:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A396A8E0009; Tue, 19 Feb 2019 15:05:24 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A118B8E0002; Tue, 19 Feb 2019 15:05:24 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8FF638E0009; Tue, 19 Feb 2019 15:05:24 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id 5CA938E0002 for ; Tue, 19 Feb 2019 15:05:24 -0500 (EST) Received: by mail-qk1-f197.google.com with SMTP id v67so564269qkl.22 for ; Tue, 19 Feb 2019 12:05:24 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=OqUNTN5veIAUfRPPOLpqOroSiessRltvoL9+FI9zSEI=; b=FDyVJeV4R4eq13T/Ugpl1+BR0gKG1y5zcQ2mvu1DC0rdcX7Wkr/UYL9Wi+kQvR7h30 EDRXmWaXadfzJmth4UsPqrT4R7l8L4O7+fPMOM2p7rx8ytpRGZ4o8r4CNqjWpTKmks56 rgwLWH54yO6Nfi7MG3rWJXrDrvauHNxC+4v1kh47w2xGwc+uLS5j59MeUEwKSB26bvkA NJPOTUVWsjmo2HaSSWM5Sxo0YMel1KGkd74H10lv481ckS548yVhpEN50YhNzEV1c8Nn VWOMXIpN+24yoj1FrCESH1+gBQDOK8IlE8AP0jQW9ZLjYexPJhvOtkeExRLDrhP7r5Ga LwoQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AHQUAuYRBxplWupGjrCYJewfXJnG+rjodwOsAFnWZvjoJ3EXe56UePsQ be1+gUoeTAqYL7U+NMKPT/4vClg3Px2g/jh7pNat0+NGELQiYeRELod1UwRBfPTfY86+TBNvBS6 6IAedCghIQJwJNfsJdp8BepdLrTnHJYDwhyUlS02odj6tVXx7WEm/RU7bUbfGnGc0BA== X-Received: by 2002:ac8:393a:: with SMTP id s55mr24206302qtb.70.1550606724040; Tue, 19 Feb 2019 12:05:24 -0800 (PST) X-Google-Smtp-Source: AHgI3IZWriy0TBUQmiksnxc6jfg6Hs+/HlnfXGf8UDtDPzXmbaP3KfG1ZTbRRJj3v/ha/Ao0UNKg X-Received: by 2002:ac8:393a:: with SMTP id s55mr24206224qtb.70.1550606722904; Tue, 19 Feb 2019 12:05:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550606722; cv=none; d=google.com; s=arc-20160816; b=nymbDPr2/YoBJznJuQifReuphz0hagwaxpEGGOA+xItdl7nYLftJKx/vssP1C4serL tSeNkgLb+gDwVuRTB4yYPWt/uCPM7/lU6qro7YkEilDkF7um/hayonJOhB/bmJSWhgNl 1XghegRzKQBUqju4eI/hAARKRzZtHS/BgmrLkNxWsr0WyKUhHcrgxVwk3xF0oOBvhaKa chReSkED4ZwlkD7lubaro4EwNRwpsJIdjLUt3mcndTp1g2CHghZaIdAqt2hpRUX3bTXX NGaJXx/Wh47n5NTFc/MrR3PfbID+rPmXtTNKBRlODAo1ALoBPS2N7966W7Unb3VGEub8 c6EA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=OqUNTN5veIAUfRPPOLpqOroSiessRltvoL9+FI9zSEI=; b=H7rakU13OV/ZaVTyfzEDpFtQ9kqak7mXcIdqDTzyvh01Ze8A0oRzws0wQMANSZkPve KuHR8FqA8RgqIO4zLPWZOj31LaQ3Iaj1/bh4aDoIrD8GvvtESS0+6QONcpSL8mZvfDK4 MT1Xww39V8oLdAKWBG8eOQn30qrRZ5o99fqNi+MgIak3we4PcoZg3G/ZzrInYWdD9z0w 7PgJArzK98WKNyd6XEiz0/Ad7lVRf6EBBaDrea/zhdWZN3GKTLPDw5H9Fs93Ju3872qq E7ZiYq2pGfbLttbl+UG/w4Ry6RggyircxtizuWQ2LB18cg52Pe4NxeBMez3vYev4wsYM e3uw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id r21si1743547qtj.379.2019.02.19.12.05.22 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 19 Feb 2019 12:05:22 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CC40659447; Tue, 19 Feb 2019 20:05:21 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-134.rdu2.redhat.com [10.10.122.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0012E6013C; Tue, 19 Feb 2019 20:05:08 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org, Andrew Morton Cc: linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v5 5/9] mm/mmu_notifier: contextual information for event triggering invalidation v2 Date: Tue, 19 Feb 2019 15:04:26 -0500 Message-Id: <20190219200430.11130-6-jglisse@redhat.com> In-Reply-To: <20190219200430.11130-1-jglisse@redhat.com> References: <20190219200430.11130-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 19 Feb 2019 20:05:22 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse CPU page table update can happens for many reasons, not only as a result of a syscall (munmap(), mprotect(), mremap(), madvise(), ...) but also as a result of kernel activities (memory compression, reclaim, migration, ...). Users of mmu notifier API track changes to the CPU page table and take specific action for them. While current API only provide range of virtual address affected by the change, not why the changes is happening. This patchset do the initial mechanical convertion of all the places that calls mmu_notifier_range_init to also provide the default MMU_NOTIFY_UNMAP event as well as the vma if it is know (most invalidation happens against a given vma). Passing down the vma allows the users of mmu notifier to inspect the new vma page protection. The MMU_NOTIFY_UNMAP is always the safe default as users of mmu notifier should assume that every for the range is going away when that event happens. A latter patch do convert mm call path to use a more appropriate events for each call. Changes since v1: - add the flags parameter to init range flags This is done as 2 patches so that no call site is forgotten especialy as it uses this following coccinelle patch: %<---------------------------------------------------------------------- @@ identifier I1, I2, I3, I4; @@ static inline void mmu_notifier_range_init(struct mmu_notifier_range *I1, +enum mmu_notifier_event event, +unsigned flags, +struct vm_area_struct *vma, struct mm_struct *I2, unsigned long I3, unsigned long I4) { ... } @@ @@ -#define mmu_notifier_range_init(range, mm, start, end) +#define mmu_notifier_range_init(range, event, flags, vma, mm, start, end) @@ expression E1, E3, E4; identifier I1; @@ <... mmu_notifier_range_init(E1, +MMU_NOTIFY_UNMAP, 0, I1, I1->vm_mm, E3, E4) ...> @@ expression E1, E2, E3, E4; identifier FN, VMA; @@ FN(..., struct vm_area_struct *VMA, ...) { <... mmu_notifier_range_init(E1, +MMU_NOTIFY_UNMAP, 0, VMA, E2, E3, E4) ...> } @@ expression E1, E2, E3, E4; identifier FN, VMA; @@ FN(...) { struct vm_area_struct *VMA; <... mmu_notifier_range_init(E1, +MMU_NOTIFY_UNMAP, 0, VMA, E2, E3, E4) ...> } @@ expression E1, E2, E3, E4; identifier FN; @@ FN(...) { <... mmu_notifier_range_init(E1, +MMU_NOTIFY_UNMAP, 0, NULL, E2, E3, E4) ...> } ---------------------------------------------------------------------->% Applied with: spatch --all-includes --sp-file mmu-notifier.spatch fs/proc/task_mmu.c --in-place spatch --sp-file mmu-notifier.spatch --dir kernel/events/ --in-place spatch --sp-file mmu-notifier.spatch --dir mm --in-place Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann Reviewed-by: Ralph Campbell --- fs/proc/task_mmu.c | 3 ++- include/linux/mmu_notifier.h | 5 ++++- kernel/events/uprobes.c | 3 ++- mm/huge_memory.c | 12 ++++++++---- mm/hugetlb.c | 12 ++++++++---- mm/khugepaged.c | 3 ++- mm/ksm.c | 6 ++++-- mm/madvise.c | 3 ++- mm/memory.c | 25 ++++++++++++++++--------- mm/migrate.c | 5 ++++- mm/mprotect.c | 3 ++- mm/mremap.c | 3 ++- mm/oom_kill.c | 3 ++- mm/rmap.c | 6 ++++-- 14 files changed, 62 insertions(+), 30 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 92a91e7816d8..fcbd0e574917 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1151,7 +1151,8 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf, break; } - mmu_notifier_range_init(&range, mm, 0, -1UL); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, + NULL, mm, 0, -1UL); mmu_notifier_invalidate_range_start(&range); } walk_page_range(0, mm->highest_vm_end, &clear_refs_walk); diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 2386e71ac1b8..62f94cd85455 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -356,6 +356,9 @@ static inline void mmu_notifier_mm_destroy(struct mm_struct *mm) static inline void mmu_notifier_range_init(struct mmu_notifier_range *range, + enum mmu_notifier_event event, + unsigned flags, + struct vm_area_struct *vma, struct mm_struct *mm, unsigned long start, unsigned long end) @@ -491,7 +494,7 @@ static inline void _mmu_notifier_range_init(struct mmu_notifier_range *range, range->end = end; } -#define mmu_notifier_range_init(range, mm, start, end) \ +#define mmu_notifier_range_init(range,event,flags,vma,mm,start,end) \ _mmu_notifier_range_init(range, start, end) static inline bool diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index affa830a198c..46f546bdba00 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -161,7 +161,8 @@ static int __replace_page(struct vm_area_struct *vma, unsigned long addr, struct mmu_notifier_range range; struct mem_cgroup *memcg; - mmu_notifier_range_init(&range, mm, addr, addr + PAGE_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, addr, + addr + PAGE_SIZE); VM_BUG_ON_PAGE(PageTransHuge(old_page), old_page); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d4847026d4b1..c9d638f1b34e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1184,7 +1184,8 @@ static vm_fault_t do_huge_pmd_wp_page_fallback(struct vm_fault *vmf, cond_resched(); } - mmu_notifier_range_init(&range, vma->vm_mm, haddr, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + haddr, haddr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -1348,7 +1349,8 @@ vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd) vma, HPAGE_PMD_NR); __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, vma->vm_mm, haddr, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + haddr, haddr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -2026,7 +2028,8 @@ void __split_huge_pud(struct vm_area_struct *vma, pud_t *pud, spinlock_t *ptl; struct mmu_notifier_range range; - mmu_notifier_range_init(&range, vma->vm_mm, address & HPAGE_PUD_MASK, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + address & HPAGE_PUD_MASK, (address & HPAGE_PUD_MASK) + HPAGE_PUD_SIZE); mmu_notifier_invalidate_range_start(&range); ptl = pud_lock(vma->vm_mm, pud); @@ -2244,7 +2247,8 @@ void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, spinlock_t *ptl; struct mmu_notifier_range range; - mmu_notifier_range_init(&range, vma->vm_mm, address & HPAGE_PMD_MASK, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + address & HPAGE_PMD_MASK, (address & HPAGE_PMD_MASK) + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); ptl = pmd_lock(vma->vm_mm, pmd); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 1c5219193b9e..d9e5c5a4c004 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3250,7 +3250,8 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, cow = (vma->vm_flags & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE; if (cow) { - mmu_notifier_range_init(&range, src, vma->vm_start, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, src, + vma->vm_start, vma->vm_end); mmu_notifier_invalidate_range_start(&range); } @@ -3362,7 +3363,8 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, /* * If sharing possible, alert mmu notifiers of worst case. */ - mmu_notifier_range_init(&range, mm, start, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, start, + end); adjust_range_if_pmd_sharing_possible(vma, &range.start, &range.end); mmu_notifier_invalidate_range_start(&range); address = start; @@ -3629,7 +3631,8 @@ static vm_fault_t hugetlb_cow(struct mm_struct *mm, struct vm_area_struct *vma, pages_per_huge_page(h)); __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, mm, haddr, haddr + huge_page_size(h)); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, haddr, + haddr + huge_page_size(h)); mmu_notifier_invalidate_range_start(&range); /* @@ -4354,7 +4357,8 @@ unsigned long hugetlb_change_protection(struct vm_area_struct *vma, * start/end. Set range.start/range.end to cover the maximum possible * range if PMD sharing is possible. */ - mmu_notifier_range_init(&range, mm, start, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, start, + end); adjust_range_if_pmd_sharing_possible(vma, &range.start, &range.end); BUG_ON(address >= end); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 449044378782..e7944f5e6258 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1016,7 +1016,8 @@ static void collapse_huge_page(struct mm_struct *mm, pte = pte_offset_map(pmd, address); pte_ptl = pte_lockptr(mm, pmd); - mmu_notifier_range_init(&range, mm, address, address + HPAGE_PMD_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, NULL, mm, + address, address + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); pmd_ptl = pmd_lock(mm, pmd); /* probably unnecessary */ /* diff --git a/mm/ksm.c b/mm/ksm.c index fa78626da9f0..2ea25fc0befb 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1066,7 +1066,8 @@ static int write_protect_page(struct vm_area_struct *vma, struct page *page, BUG_ON(PageTransCompound(page)); - mmu_notifier_range_init(&range, mm, pvmw.address, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, + pvmw.address, pvmw.address + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -1154,7 +1155,8 @@ static int replace_page(struct vm_area_struct *vma, struct page *page, if (!pmd) goto out; - mmu_notifier_range_init(&range, mm, addr, addr + PAGE_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, addr, + addr + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); ptep = pte_offset_map_lock(mm, pmd, addr, &ptl); diff --git a/mm/madvise.c b/mm/madvise.c index 21a7881a2db4..c617f53a9c09 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -472,7 +472,8 @@ static int madvise_free_single_vma(struct vm_area_struct *vma, range.end = min(vma->vm_end, end_addr); if (range.end <= vma->vm_start) return -EINVAL; - mmu_notifier_range_init(&range, mm, range.start, range.end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, + range.start, range.end); lru_add_drain(); tlb_gather_mmu(&tlb, mm, range.start, range.end); diff --git a/mm/memory.c b/mm/memory.c index 34ced1369883..4565f636cca3 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1010,7 +1010,8 @@ int copy_page_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, is_cow = is_cow_mapping(vma->vm_flags); if (is_cow) { - mmu_notifier_range_init(&range, src_mm, addr, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, + src_mm, addr, end); mmu_notifier_invalidate_range_start(&range); } @@ -1334,7 +1335,8 @@ void unmap_vmas(struct mmu_gather *tlb, { struct mmu_notifier_range range; - mmu_notifier_range_init(&range, vma->vm_mm, start_addr, end_addr); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + start_addr, end_addr); mmu_notifier_invalidate_range_start(&range); for ( ; vma && vma->vm_start < end_addr; vma = vma->vm_next) unmap_single_vma(tlb, vma, start_addr, end_addr, NULL); @@ -1356,7 +1358,8 @@ void zap_page_range(struct vm_area_struct *vma, unsigned long start, struct mmu_gather tlb; lru_add_drain(); - mmu_notifier_range_init(&range, vma->vm_mm, start, start + size); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + start, start + size); tlb_gather_mmu(&tlb, vma->vm_mm, start, range.end); update_hiwater_rss(vma->vm_mm); mmu_notifier_invalidate_range_start(&range); @@ -1382,7 +1385,8 @@ static void zap_page_range_single(struct vm_area_struct *vma, unsigned long addr struct mmu_gather tlb; lru_add_drain(); - mmu_notifier_range_init(&range, vma->vm_mm, address, address + size); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + address, address + size); tlb_gather_mmu(&tlb, vma->vm_mm, address, range.end); update_hiwater_rss(vma->vm_mm); mmu_notifier_invalidate_range_start(&range); @@ -2278,7 +2282,8 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, mm, vmf->address & PAGE_MASK, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, + vmf->address & PAGE_MASK, (vmf->address & PAGE_MASK) + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -4100,8 +4105,9 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, goto out; if (range) { - mmu_notifier_range_init(range, mm, address & PMD_MASK, - (address & PMD_MASK) + PMD_SIZE); + mmu_notifier_range_init(range, MMU_NOTIFY_UNMAP, 0, + NULL, mm, address & PMD_MASK, + (address & PMD_MASK) + PMD_SIZE); mmu_notifier_invalidate_range_start(range); } *ptlp = pmd_lock(mm, pmd); @@ -4118,8 +4124,9 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, goto out; if (range) { - mmu_notifier_range_init(range, mm, address & PAGE_MASK, - (address & PAGE_MASK) + PAGE_SIZE); + mmu_notifier_range_init(range, MMU_NOTIFY_UNMAP, 0, NULL, mm, + address & PAGE_MASK, + (address & PAGE_MASK) + PAGE_SIZE); mmu_notifier_invalidate_range_start(range); } ptep = pte_offset_map_lock(mm, pmd, address, ptlp); diff --git a/mm/migrate.c b/mm/migrate.c index 76517bf03621..81eb307b2b5b 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2340,7 +2340,8 @@ static void migrate_vma_collect(struct migrate_vma *migrate) mm_walk.mm = migrate->vma->vm_mm; mm_walk.private = migrate; - mmu_notifier_range_init(&range, mm_walk.mm, migrate->start, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, NULL, mm_walk.mm, + migrate->start, migrate->end); mmu_notifier_invalidate_range_start(&range); walk_page_range(migrate->start, migrate->end, &mm_walk); @@ -2748,6 +2749,8 @@ static void migrate_vma_pages(struct migrate_vma *migrate) notified = true; mmu_notifier_range_init(&range, + MMU_NOTIFY_UNMAP, 0, + NULL, migrate->vma->vm_mm, addr, migrate->end); mmu_notifier_invalidate_range_start(&range); diff --git a/mm/mprotect.c b/mm/mprotect.c index 028c724dcb1a..b10984052ae9 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -185,7 +185,8 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma, /* invoke the mmu notifier if the pmd is populated */ if (!range.start) { - mmu_notifier_range_init(&range, vma->vm_mm, addr, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, + vma, vma->vm_mm, addr, end); mmu_notifier_invalidate_range_start(&range); } diff --git a/mm/mremap.c b/mm/mremap.c index 3320616ed93f..364e79bcc1ff 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -249,7 +249,8 @@ unsigned long move_page_tables(struct vm_area_struct *vma, old_end = old_addr + len; flush_cache_range(vma, old_addr, old_end); - mmu_notifier_range_init(&range, vma->vm_mm, old_addr, old_end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + old_addr, old_end); mmu_notifier_invalidate_range_start(&range); for (; old_addr < old_end; old_addr += extent, new_addr += extent) { diff --git a/mm/oom_kill.c b/mm/oom_kill.c index 3a2484884cfd..539c91d0b26a 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -531,7 +531,8 @@ bool __oom_reap_task_mm(struct mm_struct *mm) struct mmu_notifier_range range; struct mmu_gather tlb; - mmu_notifier_range_init(&range, mm, vma->vm_start, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, + vma, mm, vma->vm_start, vma->vm_end); tlb_gather_mmu(&tlb, mm, range.start, range.end); if (mmu_notifier_invalidate_range_start_nonblock(&range)) { diff --git a/mm/rmap.c b/mm/rmap.c index 0454ecc29537..c6535a6ec850 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -896,7 +896,8 @@ static bool page_mkclean_one(struct page *page, struct vm_area_struct *vma, * We have to assume the worse case ie pmd for invalidation. Note that * the page can not be free from this function. */ - mmu_notifier_range_init(&range, vma->vm_mm, address, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + address, min(vma->vm_end, address + (PAGE_SIZE << compound_order(page)))); mmu_notifier_invalidate_range_start(&range); @@ -1371,7 +1372,8 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, * Note that the page can not be free in this function as call of * try_to_unmap() must hold a reference on the page. */ - mmu_notifier_range_init(&range, vma->vm_mm, address, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + address, min(vma->vm_end, address + (PAGE_SIZE << compound_order(page)))); if (PageHuge(page)) { From patchwork Tue Feb 19 20:04:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10820753 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 124601399 for ; Tue, 19 Feb 2019 20:05:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 002292CEBB for ; Tue, 19 Feb 2019 20:05:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E7E9C2D39B; Tue, 19 Feb 2019 20:05:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 916042CEBB for ; Tue, 19 Feb 2019 20:05:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E0B1E8E000A; Tue, 19 Feb 2019 15:05:33 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DE12D8E0002; Tue, 19 Feb 2019 15:05:33 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CCF378E000A; Tue, 19 Feb 2019 15:05:33 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id 9ABD78E0002 for ; Tue, 19 Feb 2019 15:05:33 -0500 (EST) Received: by mail-qk1-f198.google.com with SMTP id b6so618815qkg.4 for ; Tue, 19 Feb 2019 12:05:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=/MIZcvQ73oHvyizAkMy4bbNGmjZ0HL1PlOjIuvQh24c=; b=MOlMlKqPEOCsg3QUX+llkS59gfCa4hy0E6VVV8FxghP7nywISDZE0JnpUn/mrMsHyr 8lv6VU1ApPci3Qk4S07DxMAT6H0PP6HG0ZfCPtRHtBGQU2dbuMCGdaIvcbRBvM8AoqPh xpzp1YF8DyoAcY9lxqb/DL5XrPTRSb6z/WgYeMJq7K9kWtd5u/tSJUQgVHv6O+mhSzh+ p1g3pqGNJR0R1ijarJJzhBviZqdyBDqQPDIqNtCi7GgidrxOU3GB1Ot2PeU0KVMk8TUa fH1e/h/E6ici6t5OuDWOYR0j0MpN4aF0bymNv3vx1GXMHUbnXA5VA9j1YKGeAsuqhth/ 6yqw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AHQUAuaiFvIsO8C9e+bEwqDSfdWyN+UYxlKU5aCT3BQ5tKQgxH3dOpaK Semg7oXzOlRnozXFppCI49JJTadosfp2GNeGjWVAJwqXd6vav83fZ6F9VOKzBaePWVCOJ4hw9eU Z3PeFS9UEATQYrxpPTiMUxkGgFD78iKS6HjPXPJTr4yJLrRiFHW4hTs+oPoyPq8CELw== X-Received: by 2002:ac8:160d:: with SMTP id p13mr24306443qtj.20.1550606733363; Tue, 19 Feb 2019 12:05:33 -0800 (PST) X-Google-Smtp-Source: AHgI3IZ1ASiD79OnhW7aS9Yzcj/YL19108UPNpxc9o4ps1Dp4YHBCqPguIOJ52y62CtEiXmCclQN X-Received: by 2002:ac8:160d:: with SMTP id p13mr24306104qtj.20.1550606728695; Tue, 19 Feb 2019 12:05:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550606728; cv=none; d=google.com; s=arc-20160816; b=L0APHqSGga0xsOvsrkxEnpP9J2RjSeemtzUl38N77/NrXmMTGu4ZibkhUeHi3LzVz3 oa019aT4kKpC41q2S7GCBsPGHOQHYMHAbfHn0zVC4sGZ6AqnWv9sSKZW3b/6mvnjwdHX VYp2Ecale9UH6IaTtISk0pbbyBvn73BWsRZMlmfWzgqrzJVJtjssT+7kFwTbSFW1j/vL 7BErV1+H1WSJVQWJPnHmL/ubQsdvhms8ArvmsjkPdpRX4n61XaKqYNLn9ItIfF/Ic4vv 2eFglMB7EoCwd+ypfxxpW+Cc+XPHrDGkgMPplrj3JqAlhKqz8wGajeWTMJIiO8YXJVKM ym9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=/MIZcvQ73oHvyizAkMy4bbNGmjZ0HL1PlOjIuvQh24c=; b=HGLCQTQoftr9RDqvm5bMOfiXYk6WYLnMpHfHv8TOWNT6pso0jrQvKdDhh65hkHXA35 cPnLzSTelfyiyIo6ez+6qdcMfsrKL2tNA2SODzBzi4/fmmL5O5vqtRwfWg5uq1g3cXkr rRQBzNlknOmhfdVwbnCJw00imOwNFdzfNPjddn9zZTvI6HHUO1/auZfdxJ8ZpxbIKCmA 3SOiQFvIEQBzKRin4q7xkD+2TWYBYn0xouvAVgYFkI0oBkjQMokEH3iy5tcVjRtH5C2O BA22fwi4TmdS2yTGVx9w7qjgqcLvi4w9VEiRXPTg7y2eiAciJ9k5wDi11ao8xxs012mI X9vg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id 23si289299qtr.57.2019.02.19.12.05.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 19 Feb 2019 12:05:28 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A1534C074132; Tue, 19 Feb 2019 20:05:27 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-134.rdu2.redhat.com [10.10.122.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 05695600C8; Tue, 19 Feb 2019 20:05:21 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org, Andrew Morton Cc: linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v5 6/9] mm/mmu_notifier: use correct mmu_notifier events for each invalidation Date: Tue, 19 Feb 2019 15:04:27 -0500 Message-Id: <20190219200430.11130-7-jglisse@redhat.com> In-Reply-To: <20190219200430.11130-1-jglisse@redhat.com> References: <20190219200430.11130-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 19 Feb 2019 20:05:27 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse This update each existing invalidation to use the correct mmu notifier event that represent what is happening to the CPU page table. See the patch which introduced the events to see the rational behind this. Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann Reviewed-by: Ralph Campbell --- fs/proc/task_mmu.c | 4 ++-- kernel/events/uprobes.c | 2 +- mm/huge_memory.c | 14 ++++++-------- mm/hugetlb.c | 8 ++++---- mm/khugepaged.c | 2 +- mm/ksm.c | 4 ++-- mm/madvise.c | 2 +- mm/memory.c | 14 +++++++------- mm/migrate.c | 4 ++-- mm/mprotect.c | 5 +++-- mm/rmap.c | 6 +++--- 11 files changed, 32 insertions(+), 33 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index fcbd0e574917..3b93ce496dd4 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1151,8 +1151,8 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf, break; } - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, - NULL, mm, 0, -1UL); + mmu_notifier_range_init(&range, MMU_NOTIFY_SOFT_DIRTY, + 0, NULL, mm, 0, -1UL); mmu_notifier_invalidate_range_start(&range); } walk_page_range(0, mm->highest_vm_end, &clear_refs_walk); diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 46f546bdba00..8e8342080013 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -161,7 +161,7 @@ static int __replace_page(struct vm_area_struct *vma, unsigned long addr, struct mmu_notifier_range range; struct mem_cgroup *memcg; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, addr, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, addr, addr + PAGE_SIZE); VM_BUG_ON_PAGE(PageTransHuge(old_page), old_page); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index c9d638f1b34e..1da6ca0f0f6d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1184,9 +1184,8 @@ static vm_fault_t do_huge_pmd_wp_page_fallback(struct vm_fault *vmf, cond_resched(); } - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, - haddr, - haddr + HPAGE_PMD_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, + haddr, haddr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd); @@ -1349,9 +1348,8 @@ vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd) vma, HPAGE_PMD_NR); __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, - haddr, - haddr + HPAGE_PMD_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, + haddr, haddr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); spin_lock(vmf->ptl); @@ -2028,7 +2026,7 @@ void __split_huge_pud(struct vm_area_struct *vma, pud_t *pud, spinlock_t *ptl; struct mmu_notifier_range range; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, address & HPAGE_PUD_MASK, (address & HPAGE_PUD_MASK) + HPAGE_PUD_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -2247,7 +2245,7 @@ void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, spinlock_t *ptl; struct mmu_notifier_range range; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, address & HPAGE_PMD_MASK, (address & HPAGE_PMD_MASK) + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index d9e5c5a4c004..a58115c6b0a3 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3250,7 +3250,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, cow = (vma->vm_flags & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE; if (cow) { - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, src, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, src, vma->vm_start, vma->vm_end); mmu_notifier_invalidate_range_start(&range); @@ -3631,7 +3631,7 @@ static vm_fault_t hugetlb_cow(struct mm_struct *mm, struct vm_area_struct *vma, pages_per_huge_page(h)); __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, haddr, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, haddr, haddr + huge_page_size(h)); mmu_notifier_invalidate_range_start(&range); @@ -4357,8 +4357,8 @@ unsigned long hugetlb_change_protection(struct vm_area_struct *vma, * start/end. Set range.start/range.end to cover the maximum possible * range if PMD sharing is possible. */ - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, start, - end); + mmu_notifier_range_init(&range, MMU_NOTIFY_PROTECTION_VMA, + 0, vma, mm, start, end); adjust_range_if_pmd_sharing_possible(vma, &range.start, &range.end); BUG_ON(address >= end); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index e7944f5e6258..579699d2b347 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1016,7 +1016,7 @@ static void collapse_huge_page(struct mm_struct *mm, pte = pte_offset_map(pmd, address); pte_ptl = pte_lockptr(mm, pmd); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, NULL, mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, NULL, mm, address, address + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); pmd_ptl = pmd_lock(mm, pmd); /* probably unnecessary */ diff --git a/mm/ksm.c b/mm/ksm.c index 2ea25fc0befb..b782fadade8f 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1066,7 +1066,7 @@ static int write_protect_page(struct vm_area_struct *vma, struct page *page, BUG_ON(PageTransCompound(page)); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, pvmw.address, pvmw.address + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -1155,7 +1155,7 @@ static int replace_page(struct vm_area_struct *vma, struct page *page, if (!pmd) goto out; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, addr, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, addr, addr + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); diff --git a/mm/madvise.c b/mm/madvise.c index c617f53a9c09..a692d2a893b5 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -472,7 +472,7 @@ static int madvise_free_single_vma(struct vm_area_struct *vma, range.end = min(vma->vm_end, end_addr); if (range.end <= vma->vm_start) return -EINVAL; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, range.start, range.end); lru_add_drain(); diff --git a/mm/memory.c b/mm/memory.c index 4565f636cca3..45dbc174a88c 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1010,8 +1010,8 @@ int copy_page_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, is_cow = is_cow_mapping(vma->vm_flags); if (is_cow) { - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, - src_mm, addr, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_PROTECTION_PAGE, + 0, vma, src_mm, addr, end); mmu_notifier_invalidate_range_start(&range); } @@ -1358,7 +1358,7 @@ void zap_page_range(struct vm_area_struct *vma, unsigned long start, struct mmu_gather tlb; lru_add_drain(); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, start, start + size); tlb_gather_mmu(&tlb, vma->vm_mm, start, range.end); update_hiwater_rss(vma->vm_mm); @@ -1385,7 +1385,7 @@ static void zap_page_range_single(struct vm_area_struct *vma, unsigned long addr struct mmu_gather tlb; lru_add_drain(); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, address, address + size); tlb_gather_mmu(&tlb, vma->vm_mm, address, range.end); update_hiwater_rss(vma->vm_mm); @@ -2282,7 +2282,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, vmf->address & PAGE_MASK, (vmf->address & PAGE_MASK) + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -4105,7 +4105,7 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, goto out; if (range) { - mmu_notifier_range_init(range, MMU_NOTIFY_UNMAP, 0, + mmu_notifier_range_init(range, MMU_NOTIFY_CLEAR, 0, NULL, mm, address & PMD_MASK, (address & PMD_MASK) + PMD_SIZE); mmu_notifier_invalidate_range_start(range); @@ -4124,7 +4124,7 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, goto out; if (range) { - mmu_notifier_range_init(range, MMU_NOTIFY_UNMAP, 0, NULL, mm, + mmu_notifier_range_init(range, MMU_NOTIFY_CLEAR, 0, NULL, mm, address & PAGE_MASK, (address & PAGE_MASK) + PAGE_SIZE); mmu_notifier_invalidate_range_start(range); diff --git a/mm/migrate.c b/mm/migrate.c index 81eb307b2b5b..8e6d00541b3c 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2340,7 +2340,7 @@ static void migrate_vma_collect(struct migrate_vma *migrate) mm_walk.mm = migrate->vma->vm_mm; mm_walk.private = migrate; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, NULL, mm_walk.mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, NULL, mm_walk.mm, migrate->start, migrate->end); mmu_notifier_invalidate_range_start(&range); @@ -2749,7 +2749,7 @@ static void migrate_vma_pages(struct migrate_vma *migrate) notified = true; mmu_notifier_range_init(&range, - MMU_NOTIFY_UNMAP, 0, + MMU_NOTIFY_CLEAR, 0, NULL, migrate->vma->vm_mm, addr, migrate->end); diff --git a/mm/mprotect.c b/mm/mprotect.c index b10984052ae9..65242f1e4457 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -185,8 +185,9 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma, /* invoke the mmu notifier if the pmd is populated */ if (!range.start) { - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, - vma, vma->vm_mm, addr, end); + mmu_notifier_range_init(&range, + MMU_NOTIFY_PROTECTION_VMA, 0, + vma, vma->vm_mm, addr, end); mmu_notifier_invalidate_range_start(&range); } diff --git a/mm/rmap.c b/mm/rmap.c index c6535a6ec850..627b38ad5052 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -896,8 +896,8 @@ static bool page_mkclean_one(struct page *page, struct vm_area_struct *vma, * We have to assume the worse case ie pmd for invalidation. Note that * the page can not be free from this function. */ - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, - address, + mmu_notifier_range_init(&range, MMU_NOTIFY_PROTECTION_PAGE, + 0, vma, vma->vm_mm, address, min(vma->vm_end, address + (PAGE_SIZE << compound_order(page)))); mmu_notifier_invalidate_range_start(&range); @@ -1372,7 +1372,7 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, * Note that the page can not be free in this function as call of * try_to_unmap() must hold a reference on the page. */ - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, address, min(vma->vm_end, address + (PAGE_SIZE << compound_order(page)))); From patchwork Tue Feb 19 20:04:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10820755 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D2F01399 for ; Tue, 19 Feb 2019 20:05:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7800B2CBC0 for ; Tue, 19 Feb 2019 20:05:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6C7912D39B; Tue, 19 Feb 2019 20:05:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DF84E2D3B0 for ; Tue, 19 Feb 2019 20:05:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 46D368E000B; Tue, 19 Feb 2019 15:05:36 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 41DF08E0002; Tue, 19 Feb 2019 15:05:36 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E6068E000B; Tue, 19 Feb 2019 15:05:36 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id 04E7C8E0002 for ; Tue, 19 Feb 2019 15:05:36 -0500 (EST) Received: by mail-qk1-f197.google.com with SMTP id k66so635373qkf.1 for ; Tue, 19 Feb 2019 12:05:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=X8kdPUvC42ROC7+rczfKExqL8lE4yK5xysqqrdCsrac=; b=el/aSMUgaBKssnr1y0XI9YaNMcB8s14JqUEPf8q1WJ5edCVqSJ7xYwxPzcJ9M7zMuL JmScHonaNFAr4QGQoTdcCWWsOp2PoSRXvRuP5kped72dhRdHV6+Svj8dCgcO0WI2zS65 XTfcjuDqy/vQtZ3vDZWUw+Ktesib59lir12Zit97gXrkIZyAeyPRMo3CAU2OQme9a3Z0 xTC1FfwF6BFkxzqMbFP8bjEfDlC1MGPetDVniBdV0WQhxQ+dkJWxgK8O5KG4eMXvk7Cl +itxY106QsKWaJb6OhA3VvtQhv4Ai483oe7gIgdNlB8wwAAR+IgbJCOw155GEKuk8h1H gfpA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AHQUAuYAK/Y1FuNrwkCqHbAOSfyPfIt/lh8nBqRZu6NnlAUVR/f9cOcB 2SZAlnC8oTKjZaVNkZ0qBpGLE2XhSr9AmHh908vHo8X3h7uO5TPqtnHuyw9oRDAPAJ60kRPO2tZ bDuX2ctkbxzmwyrJRqb7GvMdJQOsm0p9G/extuLvy/SBXsGyOs9dUlqjDxidkvnwzZg== X-Received: by 2002:a37:2fc1:: with SMTP id v184mr2883735qkh.71.1550606735750; Tue, 19 Feb 2019 12:05:35 -0800 (PST) X-Google-Smtp-Source: AHgI3IYnWazzmcAi/IOtmEWULoOUL2oqzab11C9R2eafftHeW00Cwq1nlyn9VfLu1wb5ugxEO+ex X-Received: by 2002:a37:2fc1:: with SMTP id v184mr2883473qkh.71.1550606731930; Tue, 19 Feb 2019 12:05:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550606731; cv=none; d=google.com; s=arc-20160816; b=n5W0rSUenVweayvIbJs26ElE17dIqLA1roZkDc3c5wpPt1W1gcc8YDjyeRldVves8m OcMIOzT0xZkziG0oRWTadbqcU/3Lypob1Ia2elYni3To8+ENFbZKQH6jTGnqLA8qMa/u YYYY97S+DL65wbMSbsqCf4ovIm73GW5BlfrNFDYRqLLJY4kgEYPZbW0XyKyMMW2cAHdM C8oEHPRu5wY8ABWIB3/Kp0ogStra/RFahc6HT3yTOEHi0DHgHnH6cByhS7jjKd9ggtwY h9EbM+aIspzJWD0rEia14CmHLn+kP/jpPs5F4cVaapvRbwMwPMWR5mBcv7pIjWH2E8sE U4LQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=X8kdPUvC42ROC7+rczfKExqL8lE4yK5xysqqrdCsrac=; b=nExwS0sneL27pkYjhBwc0sVqNNiMX8NqzPZMHwJrckKBewsSm90IvQ36i+LeAxj9DY T8w2gZObZXMwotF2OIl+ib6M3Fg1vk6AZzelzn9UorYSkGw2delCIbwAoFilLgCLVWDi Ay8RKLO+qGqAmEyIIjk7bl3rMsY1Q+LXgq6QTtUYPD9QdVyzky336GqMI6E+gS4YKTto pO5KQlQNyjdw+QTJNpW/T+OnlEdYNr1RrcawATGWM9+Bw3RYWY5PX/UOqYmPBJ3rDZA9 LBsoBMBRn/RR2WKCcCHvp/t+WVNjIHQ5t97Li7FNilr5FKGk/ZCmklYJhdckBbt43t8a DZ+g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id k27si10334410qki.263.2019.02.19.12.05.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 19 Feb 2019 12:05:31 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B1E58C074132; Tue, 19 Feb 2019 20:05:30 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-134.rdu2.redhat.com [10.10.122.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id BA9FC6013C; Tue, 19 Feb 2019 20:05:27 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org, Andrew Morton Cc: linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v5 7/9] mm/mmu_notifier: pass down vma and reasons why mmu notifier is happening v2 Date: Tue, 19 Feb 2019 15:04:28 -0500 Message-Id: <20190219200430.11130-8-jglisse@redhat.com> In-Reply-To: <20190219200430.11130-1-jglisse@redhat.com> References: <20190219200430.11130-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 19 Feb 2019 20:05:31 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse CPU page table update can happens for many reasons, not only as a result of a syscall (munmap(), mprotect(), mremap(), madvise(), ...) but also as a result of kernel activities (memory compression, reclaim, migration, ...). Users of mmu notifier API track changes to the CPU page table and take specific action for them. While current API only provide range of virtual address affected by the change, not why the changes is happening This patch is just passing down the new informations by adding it to the mmu_notifier_range structure. Changes since v1: - Initialize flags field from mmu_notifier_range_init() arguments Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann Reviewed-by: Ralph Campbell --- include/linux/mmu_notifier.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 62f94cd85455..0379956fff23 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -58,10 +58,12 @@ struct mmu_notifier_mm { #define MMU_NOTIFIER_RANGE_BLOCKABLE (1 << 0) struct mmu_notifier_range { + struct vm_area_struct *vma; struct mm_struct *mm; unsigned long start; unsigned long end; unsigned flags; + enum mmu_notifier_event event; }; struct mmu_notifier_ops { @@ -363,10 +365,12 @@ static inline void mmu_notifier_range_init(struct mmu_notifier_range *range, unsigned long start, unsigned long end) { + range->vma = vma; + range->event = event; range->mm = mm; range->start = start; range->end = end; - range->flags = 0; + range->flags = flags; } #define ptep_clear_flush_young_notify(__vma, __address, __ptep) \ From patchwork Tue Feb 19 20:04:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10820771 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AADB91399 for ; Tue, 19 Feb 2019 20:05:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9BEBF2D4E2 for ; Tue, 19 Feb 2019 20:05:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 904AD2CC25; Tue, 19 Feb 2019 20:05:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 166E52D474 for ; Tue, 19 Feb 2019 20:05:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 075668E000C; Tue, 19 Feb 2019 15:05:57 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 024DC8E0002; Tue, 19 Feb 2019 15:05:56 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E7F608E000C; Tue, 19 Feb 2019 15:05:56 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by kanga.kvack.org (Postfix) with ESMTP id BFCF88E0002 for ; Tue, 19 Feb 2019 15:05:56 -0500 (EST) Received: by mail-qt1-f200.google.com with SMTP id k37so21085645qtb.20 for ; Tue, 19 Feb 2019 12:05:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=1rXUTed8CnGwTcmPq0oCfhnLKbGw3aW6rBbzJhdygy8=; b=OvinrxoLH6511Q1lB4R3vQOdYyQNLHNoCC+L8/RhkpARDfZmD7L5x0qaFc0yHZpW3j Wz0kkEp3ekfpIvspAikNPUOoj73Ci9z5fYC2ULMNuFZlWwUJkhJTeabfvl0ZyW/oeFBd mhToYWYkzz9Y8nKKvnQDskQOIQbbbvLRaMNt+vOvXJcoaUZbT7hDMooq1PN8S3xKryYl eP0BziNwD2qpYDo3Mtu0ernPk8T5UIHv9QrXkle5v6wgRTNicCyqj8F8dY+NS7jx5D4b +L+gVy5m3rk9HOKpt2nBA1FB1BjYToQ9KAeIasfAlT6ZN0FU4YhFWpGOLI7ePFOdeMCd BZXw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AHQUAub2ii0luO2O+lJfGRPSPufJulmJGCO1gfSeW+SmMgbcISjGiMKg vuNunTbRORBIHDC91bHxTTdAIuPf7t0+4JUSsJPfFlBZHUsg+xpOjXyKaAv/PfHL0KR/LxbpOFP WSglIjMguqvfCLFQDXYj60s8vFcMncecMarJwiL473OQ86kch6ie/NlsXpoQb5HvaBg== X-Received: by 2002:ae9:f712:: with SMTP id s18mr21289866qkg.83.1550606756572; Tue, 19 Feb 2019 12:05:56 -0800 (PST) X-Google-Smtp-Source: AHgI3IZTmRlwxO6eLrs5XFaoipVg3Usx61wFfpSW8neSkhankA3qBVcSfi/CYl40/27QjW9JYuPd X-Received: by 2002:ae9:f712:: with SMTP id s18mr21289643qkg.83.1550606753137; Tue, 19 Feb 2019 12:05:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550606753; cv=none; d=google.com; s=arc-20160816; b=YaewFunTXgzL4shZLZKm44noUWnwnHyWzsdBpwHyjBfnn+rwxY7m6lCLS+c4euDcvc vYpLz05f7eEt+3lwPTP7tpy13R4Wjbd8wme5uTW91U5OTp7yurh+C/G2GOZN2Nx3Pylw ce3EtABmaAn4VoSwn8tBK0QdNB4G12CyGJBzMR2V1jGstC5//wWGWGGviQP0BbaXlXNm euHeBC+XMecVgsF8sBVh8Tqu5AqV2YLIUIO+DHD3V3d/E85uES9w8aJcPC8tk4VY3KU0 jxob19nByyRdeku8a4FvjyzaYFhHXGmFifBFDO41bzYA5oTp2wWsvemQ9x/+80TnSDLQ uvUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=1rXUTed8CnGwTcmPq0oCfhnLKbGw3aW6rBbzJhdygy8=; b=th+JWHhY0ksHW64edDGzHY5J3lfFR7Jp6MiswbpE//4icwjMJ/Vrxb/U5gl1/P/DiM V4K+RV6E6Z8iG9bP0x4SX29byr515tjPvb/EUbB7o+URj4xq+SuhTQNE/LjO+99cVGqv C6owOXQ/OuraxT/3w8o8XGgG1JcFFb+gGGM9+2EL/J6qjtfxCAkqWRDA2cdrf15asg4X olDWwAvuGIJAUWQOkggDbOA+8xKP/BGZzp3bsdwEgrrtGRMEQmOvRY5nJPfoUcFBC82/ cVG+TjkBj2qASgf7XHWF6PNyVvyCjl720aggTgs8+T820Q/5tO4m/VHVp46ZyKeYDxo3 +hEQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id b21si1399457qvd.134.2019.02.19.12.05.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 19 Feb 2019 12:05:53 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0D706C074132; Tue, 19 Feb 2019 20:05:52 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-134.rdu2.redhat.com [10.10.122.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id C1DEA6013C; Tue, 19 Feb 2019 20:05:30 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org, Andrew Morton Cc: linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v5 8/9] mm/mmu_notifier: mmu_notifier_range_update_to_read_only() helper Date: Tue, 19 Feb 2019 15:04:29 -0500 Message-Id: <20190219200430.11130-9-jglisse@redhat.com> In-Reply-To: <20190219200430.11130-1-jglisse@redhat.com> References: <20190219200430.11130-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 19 Feb 2019 20:05:52 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Helper to test if a range is updated to read only (it is still valid to read from the range). This is useful for device driver or anyone who wish to optimize out update when they know that they already have the range map read only. Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann Reviewed-by: Ralph Campbell --- include/linux/mmu_notifier.h | 4 ++++ mm/mmu_notifier.c | 10 ++++++++++ 2 files changed, 14 insertions(+) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 0379956fff23..b6c004bd9f6a 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -259,6 +259,8 @@ extern void __mmu_notifier_invalidate_range_end(struct mmu_notifier_range *r, bool only_end); extern void __mmu_notifier_invalidate_range(struct mm_struct *mm, unsigned long start, unsigned long end); +extern bool +mmu_notifier_range_update_to_read_only(const struct mmu_notifier_range *range); static inline bool mmu_notifier_range_blockable(const struct mmu_notifier_range *range) @@ -568,6 +570,8 @@ static inline void mmu_notifier_mm_destroy(struct mm_struct *mm) { } +#define mmu_notifier_range_update_to_read_only(r) false + #define ptep_clear_flush_young_notify ptep_clear_flush_young #define pmdp_clear_flush_young_notify pmdp_clear_flush_young #define ptep_clear_young_notify ptep_test_and_clear_young diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index abd88c466eb2..ee36068077b6 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -395,3 +395,13 @@ void mmu_notifier_unregister_no_release(struct mmu_notifier *mn, mmdrop(mm); } EXPORT_SYMBOL_GPL(mmu_notifier_unregister_no_release); + +bool +mmu_notifier_range_update_to_read_only(const struct mmu_notifier_range *range) +{ + if (!range->vma || range->event != MMU_NOTIFY_PROTECTION_VMA) + return false; + /* Return true if the vma still have the read flag set. */ + return range->vma->vm_flags & VM_READ; +} +EXPORT_SYMBOL_GPL(mmu_notifier_range_update_to_read_only); From patchwork Tue Feb 19 20:04:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10820781 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 53768180E for ; Tue, 19 Feb 2019 20:06:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4574B2CD4A for ; Tue, 19 Feb 2019 20:06:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4368C2CDD9; Tue, 19 Feb 2019 20:06:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 98EEB2D4DD for ; Tue, 19 Feb 2019 20:06:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A38AA8E0003; Tue, 19 Feb 2019 15:06:09 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9E84B8E0002; Tue, 19 Feb 2019 15:06:09 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D8908E0003; Tue, 19 Feb 2019 15:06:09 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by kanga.kvack.org (Postfix) with ESMTP id 65F9C8E0002 for ; Tue, 19 Feb 2019 15:06:09 -0500 (EST) Received: by mail-qk1-f200.google.com with SMTP id b6so620398qkg.4 for ; Tue, 19 Feb 2019 12:06:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=7aZ3rlOSOpqeHvTkHcjlxdFQmNAzoKPyodvuVEMV8wI=; b=hX0ZqLMYXyi4ZDnVaAmsq/AxV8owcBNFD4R8uEoVdAWNE+dmKxm91fq8e6VPnztUux CKlYWvH8q0UNlGJpesPGES2uJwAGnCWpcffhz7JTcRZDoPTrX3lvt6GKjSsL4SEXFNiH NpzLpMllW1RbywFZcR63Em0HgsY9csAaQKNFN7CduqVFtGiMqNclQN5BSQIp4tHGBK3A L5y2fm+sXVOBAMbeBO4IH2DPkwkvSLcb5ElaHQH3NgKF9rj6Uhs2gmWqlYbe0h3d8ONZ iJ+24LDaxV+B/NbwLedPURL1eqXrEVuvu69VSyd0aJT2NCJRMQtEITqJq/vNHhqGJx28 EPHQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AHQUAuZVjw+otmSWknjPZwQEeMaF50IbWbkhP7F1AXPgj1IRPaPmDorM wmepD7GwUutYF4RcpeXaw+udIVOJRysDK/7gN9GHJAd8x0DD6+iCv3JCDgHxScrS4sp1pEQK/6N X8MTBNHT1Kx7VHh8TkZNINrSWoXS3l+4EwWJEgnv+/8ObSuUmxBmkr+LdLQX5othP2A== X-Received: by 2002:a37:884:: with SMTP id 126mr22559083qki.56.1550606769172; Tue, 19 Feb 2019 12:06:09 -0800 (PST) X-Google-Smtp-Source: AHgI3IbssxnD7/Ql4h4gTxERe3lmGDj8lD97l9MH38ttWWza8pK8jPHIKaXp5MIVkuRHfMH57U/K X-Received: by 2002:a37:884:: with SMTP id 126mr22558678qki.56.1550606763179; Tue, 19 Feb 2019 12:06:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550606763; cv=none; d=google.com; s=arc-20160816; b=Z8kOomHu7tCgjM47k9/I4yripPbhTD2slZzljY0fZ/dPK7qIuhG2hNHS+C3UVYla+H zP6nJ3E6+flAMdsWQDb3ASdwB/q+UuH6tTpGTLCcjHTZU+Ohh4tuOdXHbmnpUi3EVpvF CwvYPDSawbV0Pv02w1s5smJ0ZErT74dAoZtnRppG7kZ7aSiAS8f1toRTDyOLAbMhqc8i nzlQzLkTTywZXtI5PsY09hMcqKyd39hxxNPGDFtIP/YrsodLblSFeJeK903b1Qwy5tjK UMfKaPM9//EX1lDEYW69b2aKv2HOiTunKseU4pQVOZLYlteYvBvZGpxnSdR4kBzG40M0 +3BA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=7aZ3rlOSOpqeHvTkHcjlxdFQmNAzoKPyodvuVEMV8wI=; b=H1caZn/YnZCnAi+wgqjmItCfZR9dC7Q8pFTOeQDWDUuj/NOJb5bcxCYXTUaS2knLzj D7UC4/Z8taHYq7zwssfP1MoUxLsH8bEpyyxMXWhDxZwhHcsu8cC/teH6tXMdKBKWBgkH TCGd7zW9JVwo3B9uXlEgEuX+EcHIpZWk0qU1fBq3btYTP8HbsElT/dmQl9Ifn5OWEEP1 bpOL0lwh20eM/I/iRRnKtGUsdY018zPCSfOVLqOmcaxY4/01uYa5QamDlaHezVC30jvn Dlee0GmZ00al1Y8Bpzn4xmFWXR+d358ZnDGokxj9KSkB5ggjlNJB1auET1IB/57J29As bNPQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id m39si3264294qtf.226.2019.02.19.12.06.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 19 Feb 2019 12:06:03 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 30C07C074F05; Tue, 19 Feb 2019 20:06:02 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-134.rdu2.redhat.com [10.10.122.134]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3E7616013D; Tue, 19 Feb 2019 20:05:52 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org, Andrew Morton Cc: linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , =?utf-8?q?Christian_?= =?utf-8?q?K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v5 9/9] mm/mmu_notifier: set MMU_NOTIFIER_USE_CHANGE_PTE flag where appropriate v2 Date: Tue, 19 Feb 2019 15:04:30 -0500 Message-Id: <20190219200430.11130-10-jglisse@redhat.com> In-Reply-To: <20190219200430.11130-1-jglisse@redhat.com> References: <20190219200430.11130-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 19 Feb 2019 20:06:02 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse When notifying change for a range use MMU_NOTIFIER_USE_CHANGE_PTE flag for page table update that use set_pte_at_notify() and where the we are going either from read and write to read only with same pfn or read only to read and write with new pfn. Note that set_pte_at_notify() itself should only be use in rare cases ie we do not want to use it when we are updating a significant range of virtual addresses and thus a significant number of pte. Instead for those cases the event provided to mmu notifer invalidate_range_start() callback should be use for optimization. Changes since v1: - Use the new unsigned flags field in struct mmu_notifier_range - Use the new flags parameter to mmu_notifier_range_init() - Explicitly list all the patterns where we can use change_pte() Signed-off-by: Jérôme Glisse Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann Reviewed-by: Ralph Campbell --- include/linux/mmu_notifier.h | 34 ++++++++++++++++++++++++++++++++-- mm/ksm.c | 11 ++++++----- mm/memory.c | 5 +++-- 3 files changed, 41 insertions(+), 9 deletions(-) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index b6c004bd9f6a..0230a4b06b46 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -40,6 +40,26 @@ enum mmu_notifier_event { MMU_NOTIFY_SOFT_DIRTY, }; +/* + * @MMU_NOTIFIER_RANGE_BLOCKABLE: can the mmu notifier range_start/range_end + * callback block or not ? If set then the callback can block. + * + * @MMU_NOTIFIER_USE_CHANGE_PTE: only set when the page table it updated with + * the set_pte_at_notify() the valid patterns for this are: + * - pte read and write to read only same pfn + * - pte read only to read and write (pfn can change or stay the same) + * - pte read only to read only with different pfn + * It is illegal to set in any other circumstances. + * + * Note that set_pte_at_notify() should not be use outside of the above cases. + * When updating a range in batch (like write protecting a range) it is better + * to rely on invalidate_range_start() and struct mmu_notifier_range to infer + * the kind of update that is happening (as an example you can look at the + * mmu_notifier_range_update_to_read_only() function). + */ +#define MMU_NOTIFIER_RANGE_BLOCKABLE (1 << 0) +#define MMU_NOTIFIER_USE_CHANGE_PTE (1 << 1) + #ifdef CONFIG_MMU_NOTIFIER /* @@ -55,8 +75,6 @@ struct mmu_notifier_mm { spinlock_t lock; }; -#define MMU_NOTIFIER_RANGE_BLOCKABLE (1 << 0) - struct mmu_notifier_range { struct vm_area_struct *vma; struct mm_struct *mm; @@ -268,6 +286,12 @@ mmu_notifier_range_blockable(const struct mmu_notifier_range *range) return (range->flags & MMU_NOTIFIER_RANGE_BLOCKABLE); } +static inline bool +mmu_notifier_range_use_change_pte(const struct mmu_notifier_range *range) +{ + return (range->flags & MMU_NOTIFIER_USE_CHANGE_PTE); +} + static inline void mmu_notifier_release(struct mm_struct *mm) { if (mm_has_notifiers(mm)) @@ -509,6 +533,12 @@ mmu_notifier_range_blockable(const struct mmu_notifier_range *range) return true; } +static inline bool +mmu_notifier_range_use_change_pte(const struct mmu_notifier_range *range) +{ + return false; +} + static inline int mm_has_notifiers(struct mm_struct *mm) { return 0; diff --git a/mm/ksm.c b/mm/ksm.c index b782fadade8f..41e51882f999 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1066,9 +1066,9 @@ static int write_protect_page(struct vm_area_struct *vma, struct page *page, BUG_ON(PageTransCompound(page)); - mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, - pvmw.address, - pvmw.address + PAGE_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, + MMU_NOTIFIER_USE_CHANGE_PTE, vma, mm, + pvmw.address, pvmw.address + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); if (!page_vma_mapped_walk(&pvmw)) @@ -1155,8 +1155,9 @@ static int replace_page(struct vm_area_struct *vma, struct page *page, if (!pmd) goto out; - mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, addr, - addr + PAGE_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, + MMU_NOTIFIER_USE_CHANGE_PTE, + vma, mm, addr, addr + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); ptep = pte_offset_map_lock(mm, pmd, addr, &ptl); diff --git a/mm/memory.c b/mm/memory.c index 45dbc174a88c..cb71d3ff1b97 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2282,8 +2282,9 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, - vmf->address & PAGE_MASK, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, + MMU_NOTIFIER_USE_CHANGE_PTE, + vma, mm, vmf->address & PAGE_MASK, (vmf->address & PAGE_MASK) + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range);