From patchwork Tue Mar 26 16:47:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10871601 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6C500186E for ; Tue, 26 Mar 2019 16:48:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 556C328DD3 for ; Tue, 26 Mar 2019 16:48:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4860628DC6; Tue, 26 Mar 2019 16:48:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AE29228DCC for ; Tue, 26 Mar 2019 16:48:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B04BF6B0006; Tue, 26 Mar 2019 12:48:05 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AB4366B000D; Tue, 26 Mar 2019 12:48:05 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9CBA26B000E; Tue, 26 Mar 2019 12:48:05 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by kanga.kvack.org (Postfix) with ESMTP id 7E85C6B0006 for ; Tue, 26 Mar 2019 12:48:05 -0400 (EDT) Received: by mail-qt1-f198.google.com with SMTP id f89so14147165qtb.4 for ; Tue, 26 Mar 2019 09:48:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=6S96iBiVeL+UBOi95MsTRx8DtQV0M6xKTn3indFi6wY=; b=EWEHB2+SK4+tQI99CgTAs8ZeHjet2C5IgWaGOveg6xXjgIiirOGwHqwrzvNjmSIubn faLo7H2s1AUwg2BIfDqHj8GtUlVsE5oITXMbBqMCni+CYgfAbjGTJ6tkm1YzzAtkIeDz bCcrSfBt7bQhoQiHwHvzSq4xsBskIC9ZtQIPjBEcNIYP5i7ZRjHPP5wJNXUko+lNrc+P 2fjwG2e3RtHN7qkUwQtw34XXZmt1eLPC9FrAqAoN7MpCdAGI99u3DFsZaqyQiyeeW21N cA3MeV3sDxrLZDMdQFM+wovhpPR1v6b775f5pTpCi5cMhTVra5BW2ycanpnWi52tB1Cd ejHA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APjAAAU5xDwCMG3ZooDsSBuFe+xz7J6u9wle1uQqQ6bOAp7pTt+Qz9Eu 3OM3Up8GyQXx1GZyZ3sbT5J4fmxpCVxsCtOM/WCz8/T2vPPsanq0Tx255/nboG3FXjpmo4zAMqY VmK0Ln2dat2FFvoZQ6C/x9VswbfhxZ817eHj4GrmgTJGf0CUyo4zM0Gpa3akv8XI5mQ== X-Received: by 2002:a37:624e:: with SMTP id w75mr24216935qkb.11.1553618885241; Tue, 26 Mar 2019 09:48:05 -0700 (PDT) X-Google-Smtp-Source: APXvYqwO7cpNWCis0wtdRI9O7EKJOkK7NL1pinZ1nwU7Eu2nkiBgs4M0+z03xK4UqRD6Kk3BPsFM X-Received: by 2002:a37:624e:: with SMTP id w75mr24216900qkb.11.1553618884697; Tue, 26 Mar 2019 09:48:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553618884; cv=none; d=google.com; s=arc-20160816; b=IT1uzDCkTfgGi0mX7+l+2x4l1+yYjS2Jo5XGFmvo7DCrMBuqzJVTakWlLY27x/Jf1w jByykmckxvAVZF23QCozZcsp5gzTo6TohhG1m3kI63TPVZe/chyAGFUYgLAz9HwJOWyz oQjSHeOtK/PQabKknikhqMdIC6wVnk8fvmoGTeuoc2+dcrmgzUr/WLrY7zNhuL27/Ska J9iBtqGe4/pCfulz046CfyYF/fRx63aBzMtsc1a34yI5+LrjawsmUvXPYLb5emip3iyC cx2z5GqRwEtLEus6aU32gblLmw63rWS5i6zMTTVe4pTjhGdmNgmvr6RiPRmg9CUfUKBC O7hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=6S96iBiVeL+UBOi95MsTRx8DtQV0M6xKTn3indFi6wY=; b=WGF8FdDIUuvlt67XVFSAQ2cldWI0sYnQHaklDC97BL9v5XInu9vlHkhVv0A5BPr2Xn /a8tlZHlVpfFt2HLqh8dQrZia/NpLIRvfBaxMasTjZyakTkEzAVo3chimSdYUqjBJ0/P dfcVx6rg1M8DtKa9Zq8PSXrb/pc9OeWqx3mSELz5IuPeiL7hmDiKrtlT173kGdmjgtx4 5B+7M36lxfP4yRarKbXzRfecByndMyxgIY+lPNJ0EenfcoOznTk8QMzz+aG+E2y35a6+ M2gFkyjodH0W7aROs943LUpM/+sK2U8O1N97tFjt+2POz/gdK9invIEVvZEMv2xy4Dhn K0Ng== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id b3si3201314qta.369.2019.03.26.09.48.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Mar 2019 09:48:04 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A11473168906; Tue, 26 Mar 2019 16:48:03 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.20.6.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id 672F517595; Tue, 26 Mar 2019 16:48:01 +0000 (UTC) From: jglisse@redhat.com To: linux-kernel@vger.kernel.org Cc: =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Andrew Morton , linux-mm@kvack.org, =?utf-8?q?Ch?= =?utf-8?q?ristian_K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-fsdevel@vger.kernel.org, Arnd Bergmann Subject: [PATCH v6 1/8] mm/mmu_notifier: helper to test if a range invalidation is blockable Date: Tue, 26 Mar 2019 12:47:40 -0400 Message-Id: <20190326164747.24405-2-jglisse@redhat.com> In-Reply-To: <20190326164747.24405-1-jglisse@redhat.com> References: <20190326164747.24405-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Tue, 26 Mar 2019 16:48:03 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Simple helpers to test if range invalidation is blockable. Latter patches use cocinnelle to convert all direct dereference of range-> blockable to use this function instead so that we can convert the blockable field to an unsigned for more flags. Signed-off-by: Jérôme Glisse Cc: Andrew Morton Cc: linux-mm@kvack.org Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Andrew Morton Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org Cc: Arnd Bergmann --- include/linux/mmu_notifier.h | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 4050ec1c3b45..e630def131ce 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -226,6 +226,12 @@ extern void __mmu_notifier_invalidate_range_end(struct mmu_notifier_range *r, extern void __mmu_notifier_invalidate_range(struct mm_struct *mm, unsigned long start, unsigned long end); +static inline bool +mmu_notifier_range_blockable(const struct mmu_notifier_range *range) +{ + return range->blockable; +} + static inline void mmu_notifier_release(struct mm_struct *mm) { if (mm_has_notifiers(mm)) @@ -455,6 +461,11 @@ static inline void _mmu_notifier_range_init(struct mmu_notifier_range *range, #define mmu_notifier_range_init(range, mm, start, end) \ _mmu_notifier_range_init(range, start, end) +static inline bool +mmu_notifier_range_blockable(const struct mmu_notifier_range *range) +{ + return true; +} static inline int mm_has_notifiers(struct mm_struct *mm) { From patchwork Tue Mar 26 16:47:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10871611 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9B93C186D for ; Tue, 26 Mar 2019 16:48:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 80B4528D4B for ; Tue, 26 Mar 2019 16:48:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 73D1A28DC6; Tue, 26 Mar 2019 16:48:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 055C728D4B for ; Tue, 26 Mar 2019 16:48:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 253BD6B000D; Tue, 26 Mar 2019 12:48:09 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 205176B000E; Tue, 26 Mar 2019 12:48:09 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0D1206B028D; Tue, 26 Mar 2019 12:48:09 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id D10016B000D for ; Tue, 26 Mar 2019 12:48:08 -0400 (EDT) Received: by mail-qk1-f197.google.com with SMTP id k21so12085534qkg.19 for ; Tue, 26 Mar 2019 09:48:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=IoaIeKySqTbuYcbt1VrYg4qHpnesClR2CyNU6akPjSw=; b=ahAmFOaGoi2V+PX8Ldp6Ae1s8v0YpahV2iAJ7nktOiYoYd2/JlOcLE8NZFam6di6lJ 8djKOPWzXgA/0bi7gcTEfM/xqi/hQAzKgu9PlZKaDKisKTSiWRf2beDZ9T+6yup91BV2 9VuyspM1lF6uXsWGRQHVo1ksht8B46yJc+6RdNEkjC3y9inDcLlacz99zqKGo2bsoBK/ XUh9GafqbhOGFQCHQ6xvmfxjbMrKC7b+Zn864y9gr/B0FD6n7d5FmEbjqD6uI5GNiRfa 69WiGKQ5wlnHOAJLqO7DU8dpvF9wGPCVCOzKJX+hpM6mUs4mbMyMlb/wTkhniyieDp0y qHyQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APjAAAWKFQWsreuz0Cw8Z+/7IsfMNY2EOPjeBKNZX/ptvG7VFBHHz3ap hsCSYBT1wuiru3Kegjm1gC5Ice/stYGPCPD/p5Z55pUXgRWtmDUAuVtACKh+OrQIzBu1Bk3I8LW Tcva5rmdqEU4S2JrwJ0hU4TO89RYt9HpQhpb4oLWLYa3cO0tvT7TcCz9xUwDkkQ5JAg== X-Received: by 2002:a0c:8445:: with SMTP id l63mr26578500qva.187.1553618888111; Tue, 26 Mar 2019 09:48:08 -0700 (PDT) X-Google-Smtp-Source: APXvYqzsrMwC3ry8U9vm6VZ7IF8cmQr0Fl65fRM2blZ5NTp7xWWaBOUJL4K6ihlnS4TRYvFrSwF8 X-Received: by 2002:a0c:8445:: with SMTP id l63mr26578421qva.187.1553618886967; Tue, 26 Mar 2019 09:48:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553618886; cv=none; d=google.com; s=arc-20160816; b=0HcaAJMIOk5rb277GmrF8IB1mVeedq9W/ntbDhqy6yHG7tQDE654+0BenSRLiCnGnw nmbwccEq8C+hXb9nfvEXoK88NKxgCPmA90Cw0O7gswIu62GLywTMOoQnlIRw7jwf4yMT u8RhE0RJjx/aWBcohmAAznYfgEtRiO2ukKFskSoQ0msxS18fA4nmg+gHJPMzplI3rAIU kkdR2m4LAaduRlKRhmkwZ0IZvs6piUoXXVLGxQ8ie2eWMMFjxfX14n88kHuzAOeV04q9 DMb6nV4ZHkI7s7tAhEnKw1zefWLI0/JuzSYofv/I1Wba8eUISfio/bmFLyjEYblV9r0I GNMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=IoaIeKySqTbuYcbt1VrYg4qHpnesClR2CyNU6akPjSw=; b=f3UzUU/8+spZn0ovjW2pwxHxTDk705tlzNpO/ZDPNYJIwRRt1ZoPWChjWVvhrg5Z1h Itg6YJklfYhmfl88J01AzW3lsOBgw+8ksPrb+GZtz3aXJ9ZgEGOjZgWvWGQua9uwZM7o JvfoxKRBFYSr/BhKZ8gPsYMIaYW7RUrk1jkNfeeVmCvddKYvH32yWeBzQKbLUHRHYR01 1nruP9QjsqrUN5bDSVD3Fbq3qQKzLv3YYg24903lS95CPcyejBWTwqGWtZYvkrhsTSlZ mgQf6M6EoOk/NN9AZKuVCDYUPXAdVR5W6CinXuXcWTDLd/uM1OMF7O5170MIfbak1V/J ru3w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id r7si2788060qtj.61.2019.03.26.09.48.06 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Mar 2019 09:48:06 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E73168667D; Tue, 26 Mar 2019 16:48:05 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.20.6.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id C19FB62669; Tue, 26 Mar 2019 16:48:03 +0000 (UTC) From: jglisse@redhat.com To: linux-kernel@vger.kernel.org Cc: =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Andrew Morton , linux-mm@kvack.org, =?utf-8?q?Ch?= =?utf-8?q?ristian_K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v6 2/8] mm/mmu_notifier: convert user range->blockable to helper function Date: Tue, 26 Mar 2019 12:47:41 -0400 Message-Id: <20190326164747.24405-3-jglisse@redhat.com> In-Reply-To: <20190326164747.24405-1-jglisse@redhat.com> References: <20190326164747.24405-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 26 Mar 2019 16:48:06 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Use the mmu_notifier_range_blockable() helper function instead of directly dereferencing the range->blockable field. This is done to make it easier to change the mmu_notifier range field. This patch is the outcome of the following coccinelle patch: %<------------------------------------------------------------------- @@ identifier I1, FN; @@ FN(..., struct mmu_notifier_range *I1, ...) { <... -I1->blockable +mmu_notifier_range_blockable(I1) ...> } ------------------------------------------------------------------->% spatch --in-place --sp-file blockable.spatch --dir . Signed-off-by: Jérôme Glisse Cc: Andrew Morton Cc: linux-mm@kvack.org Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann --- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 8 ++++---- drivers/gpu/drm/i915/i915_gem_userptr.c | 2 +- drivers/gpu/drm/radeon/radeon_mn.c | 4 ++-- drivers/infiniband/core/umem_odp.c | 5 +++-- drivers/xen/gntdev.c | 6 +++--- mm/hmm.c | 6 +++--- mm/mmu_notifier.c | 2 +- virt/kvm/kvm_main.c | 3 ++- 8 files changed, 19 insertions(+), 17 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c index 3e6823fdd939..58ed401c5996 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c @@ -256,14 +256,14 @@ static int amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, /* TODO we should be able to split locking for interval tree and * amdgpu_mn_invalidate_node */ - if (amdgpu_mn_read_lock(amn, range->blockable)) + if (amdgpu_mn_read_lock(amn, mmu_notifier_range_blockable(range))) return -EAGAIN; it = interval_tree_iter_first(&amn->objects, range->start, end); while (it) { struct amdgpu_mn_node *node; - if (!range->blockable) { + if (!mmu_notifier_range_blockable(range)) { amdgpu_mn_read_unlock(amn); return -EAGAIN; } @@ -299,7 +299,7 @@ static int amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, /* notification is exclusive, but interval is inclusive */ end = range->end - 1; - if (amdgpu_mn_read_lock(amn, range->blockable)) + if (amdgpu_mn_read_lock(amn, mmu_notifier_range_blockable(range))) return -EAGAIN; it = interval_tree_iter_first(&amn->objects, range->start, end); @@ -307,7 +307,7 @@ static int amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, struct amdgpu_mn_node *node; struct amdgpu_bo *bo; - if (!range->blockable) { + if (!mmu_notifier_range_blockable(range)) { amdgpu_mn_read_unlock(amn); return -EAGAIN; } diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c index 1d3f9a31ad61..777b3f8727e7 100644 --- a/drivers/gpu/drm/i915/i915_gem_userptr.c +++ b/drivers/gpu/drm/i915/i915_gem_userptr.c @@ -122,7 +122,7 @@ userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, while (it) { struct drm_i915_gem_object *obj; - if (!range->blockable) { + if (!mmu_notifier_range_blockable(range)) { ret = -EAGAIN; break; } diff --git a/drivers/gpu/drm/radeon/radeon_mn.c b/drivers/gpu/drm/radeon/radeon_mn.c index b3019505065a..c9bd1278f573 100644 --- a/drivers/gpu/drm/radeon/radeon_mn.c +++ b/drivers/gpu/drm/radeon/radeon_mn.c @@ -133,7 +133,7 @@ static int radeon_mn_invalidate_range_start(struct mmu_notifier *mn, /* TODO we should be able to split locking for interval tree and * the tear down. */ - if (range->blockable) + if (mmu_notifier_range_blockable(range)) mutex_lock(&rmn->lock); else if (!mutex_trylock(&rmn->lock)) return -EAGAIN; @@ -144,7 +144,7 @@ static int radeon_mn_invalidate_range_start(struct mmu_notifier *mn, struct radeon_bo *bo; long r; - if (!range->blockable) { + if (!mmu_notifier_range_blockable(range)) { ret = -EAGAIN; goto out_unlock; } diff --git a/drivers/infiniband/core/umem_odp.c b/drivers/infiniband/core/umem_odp.c index e6ec79ad9cc8..59ef912fbe03 100644 --- a/drivers/infiniband/core/umem_odp.c +++ b/drivers/infiniband/core/umem_odp.c @@ -152,7 +152,7 @@ static int ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn, struct ib_ucontext_per_mm *per_mm = container_of(mn, struct ib_ucontext_per_mm, mn); - if (range->blockable) + if (mmu_notifier_range_blockable(range)) down_read(&per_mm->umem_rwsem); else if (!down_read_trylock(&per_mm->umem_rwsem)) return -EAGAIN; @@ -170,7 +170,8 @@ static int ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn, return rbt_ib_umem_for_each_in_range(&per_mm->umem_tree, range->start, range->end, invalidate_range_start_trampoline, - range->blockable, NULL); + mmu_notifier_range_blockable(range), + NULL); } static int invalidate_range_end_trampoline(struct ib_umem_odp *item, u64 start, diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index 7cf9c51318aa..8012ab7f5120 100644 --- a/drivers/xen/gntdev.c +++ b/drivers/xen/gntdev.c @@ -526,20 +526,20 @@ static int mn_invl_range_start(struct mmu_notifier *mn, struct gntdev_grant_map *map; int ret = 0; - if (range->blockable) + if (mmu_notifier_range_blockable(range)) mutex_lock(&priv->lock); else if (!mutex_trylock(&priv->lock)) return -EAGAIN; list_for_each_entry(map, &priv->maps, next) { ret = unmap_if_in_range(map, range->start, range->end, - range->blockable); + mmu_notifier_range_blockable(range)); if (ret) goto out_unlock; } list_for_each_entry(map, &priv->freeable_maps, next) { ret = unmap_if_in_range(map, range->start, range->end, - range->blockable); + mmu_notifier_range_blockable(range)); if (ret) goto out_unlock; } diff --git a/mm/hmm.c b/mm/hmm.c index fd143251b157..25f372da325a 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -201,9 +201,9 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn, update.start = nrange->start; update.end = nrange->end; update.event = HMM_UPDATE_INVALIDATE; - update.blockable = nrange->blockable; + update.blockable = mmu_notifier_range_blockable(nrange); - if (nrange->blockable) + if (mmu_notifier_range_blockable(nrange)) mutex_lock(&hmm->lock); else if (!mutex_trylock(&hmm->lock)) { ret = -EAGAIN; @@ -218,7 +218,7 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn, } mutex_unlock(&hmm->lock); - if (nrange->blockable) + if (mmu_notifier_range_blockable(nrange)) down_read(&hmm->mirrors_sem); else if (!down_read_trylock(&hmm->mirrors_sem)) { ret = -EAGAIN; diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index 9c884abc7850..abd88c466eb2 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -180,7 +180,7 @@ int __mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) if (_ret) { pr_info("%pS callback failed with %d in %sblockable context.\n", mn->ops->invalidate_range_start, _ret, - !range->blockable ? "non-" : ""); + !mmu_notifier_range_blockable(range) ? "non-" : ""); ret = _ret; } } diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index f25aa98a94df..16edfc3e0d1a 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -391,7 +391,8 @@ static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, spin_unlock(&kvm->mmu_lock); ret = kvm_arch_mmu_notifier_invalidate_range(kvm, range->start, - range->end, range->blockable); + range->end, + mmu_notifier_range_blockable(range)); srcu_read_unlock(&kvm->srcu, idx); From patchwork Tue Mar 26 16:47:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10871613 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA62B13B5 for ; Tue, 26 Mar 2019 16:48:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BAC9128D00 for ; Tue, 26 Mar 2019 16:48:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AEDB228DB8; Tue, 26 Mar 2019 16:48:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 33B9728D00 for ; Tue, 26 Mar 2019 16:48:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3B1866B000E; Tue, 26 Mar 2019 12:48:10 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3650A6B028D; Tue, 26 Mar 2019 12:48:10 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2080D6B0290; Tue, 26 Mar 2019 12:48:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by kanga.kvack.org (Postfix) with ESMTP id EF67B6B000E for ; Tue, 26 Mar 2019 12:48:09 -0400 (EDT) Received: by mail-qt1-f197.google.com with SMTP id t22so13646330qtc.13 for ; Tue, 26 Mar 2019 09:48:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=iK/wmVZ8p1b2GI2LEEXxGPJxVgFfciFqYn5pD/bZPGQ=; b=qQ3tXePLJZhDHgDyzGXViQMchDEf2gJD8diXt9fmpBnYjf0HzuMVMJB1x5qEbW1sRx gC2Iak9PTiK3nrMPNGfO+hU/rSog/IOCIVhFt0YQw0wN3L9xpdjo9GZgalkjbMsNKqZI 2RrG5X3IHonxxUl5RgwVWs3naQaR/aDwK9PROZ3YXcWeYBgyXZaR0XZIdf4GfiD6BHGt NkHOCjRDCUJPbJSDpk5i7Ccl/T9JPx+QRrEyafLFYwYQnSl/vdAtQsvxjnwpBn0SjIS3 4VEuZ4g8rkiaHHb8MosQqWCUqjHCJekOdwcvQ7FUW8uKA0nJqKhPGSaJ3XTBELggdlht qS3Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APjAAAUsHNUaKYjTnDcB/31qvQ5Cf50GhvRJXa2CuW7M1icx/UvF2jqK e4YQ+enIid6J5Lkw3JMuGMky5xAtilJ2ZlMMiwVxDw0u/iLys0OWpF65DAJ9hufJDYATiNviCGS XMw4cFJNHXZczvTh7+2FPvAj5pkQIx8W1rnLnurwuQywT/rFeNxUx3AETIPh95unvgA== X-Received: by 2002:a37:47cb:: with SMTP id u194mr21246754qka.358.1553618889772; Tue, 26 Mar 2019 09:48:09 -0700 (PDT) X-Google-Smtp-Source: APXvYqwXXwu6XoRAENNTbhTsKJ+pInu8HjSWBq8dOg0p3QJcmhpDtwenroUfazBizV1EGSWcSkoO X-Received: by 2002:a37:47cb:: with SMTP id u194mr21246718qka.358.1553618889177; Tue, 26 Mar 2019 09:48:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553618889; cv=none; d=google.com; s=arc-20160816; b=VBavY0Ru8Nbhrp6429rO+035Dt0yWfGx8zHgV7K3GxtRBalONVDFr1R7zIskAybYVN tQMSqTIn3eaj4mhcO4ueFskQOJ2uIKjDBXhaju9Rd+lejUlWt7L+AcKbcQrKfJZcz429 fdN2brTgmnDjFfMQTHLFRqWnWiFHumGfZ/nC7jBljoh1I5OSyBxD7tfH8w/Su0xRSqux U84PYJ0kCmGxaXdppziUhasOSuzVNlCDs05rWWPE/TOMXXRSNE5QI2vZCbI+ByI51rOO Ps/Sth95tXrAxxPNHVDxwGmS1rJ/Gd0cwPiiGv+VVwp0uo2gAq/y42B0ftMeFwW/OjHk daww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=iK/wmVZ8p1b2GI2LEEXxGPJxVgFfciFqYn5pD/bZPGQ=; b=iRL1m4/VHyPnHpw5AKukNAI6F7QHkR1wpcGkWRNf1QT11NEezngS3E3LrxILMZxQ9N ygmiPf+bxHpNVBQpVWwDjJO9yg9B/uUHijSD9zNwiYOp8NHeziY1B7VzlQ1K12CRcNcN BG/9Zxji86g+4UrlST6MasJtcB2mTrs4X/9xBrr6UXZRFExQR7tN1u4JXOJuE8L92iDu rPINbNYpF3muxZvFvV9+Is62LaSO9DIq7ESnmvpQodm4RPkv4VN21YteonFlxPc1+24G qzl5nyCRPGSfP/frYa0b8DHZZRr771lIMNYysuUEsQOeWUXQrFsonKDKUjLkNNpHM6b9 ECjA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id z4si723381qvz.104.2019.03.26.09.48.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Mar 2019 09:48:09 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3419485546; Tue, 26 Mar 2019 16:48:08 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.20.6.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id 10B7C17595; Tue, 26 Mar 2019 16:48:05 +0000 (UTC) From: jglisse@redhat.com To: linux-kernel@vger.kernel.org Cc: =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Andrew Morton , linux-mm@kvack.org, =?utf-8?q?Ch?= =?utf-8?q?ristian_K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v6 3/8] mm/mmu_notifier: convert mmu_notifier_range->blockable to a flags Date: Tue, 26 Mar 2019 12:47:42 -0400 Message-Id: <20190326164747.24405-4-jglisse@redhat.com> In-Reply-To: <20190326164747.24405-1-jglisse@redhat.com> References: <20190326164747.24405-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 26 Mar 2019 16:48:08 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Use an unsigned field for flags other than blockable and convert the blockable field to be one of those flags. Signed-off-by: Jérôme Glisse Cc: Andrew Morton Cc: linux-mm@kvack.org Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Andrew Morton Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann --- include/linux/mmu_notifier.h | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index e630def131ce..c8672c366f67 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -25,11 +25,13 @@ struct mmu_notifier_mm { spinlock_t lock; }; +#define MMU_NOTIFIER_RANGE_BLOCKABLE (1 << 0) + struct mmu_notifier_range { struct mm_struct *mm; unsigned long start; unsigned long end; - bool blockable; + unsigned flags; }; struct mmu_notifier_ops { @@ -229,7 +231,7 @@ extern void __mmu_notifier_invalidate_range(struct mm_struct *mm, static inline bool mmu_notifier_range_blockable(const struct mmu_notifier_range *range) { - return range->blockable; + return (range->flags & MMU_NOTIFIER_RANGE_BLOCKABLE); } static inline void mmu_notifier_release(struct mm_struct *mm) @@ -275,7 +277,7 @@ static inline void mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) { if (mm_has_notifiers(range->mm)) { - range->blockable = true; + range->flags |= MMU_NOTIFIER_RANGE_BLOCKABLE; __mmu_notifier_invalidate_range_start(range); } } @@ -284,7 +286,7 @@ static inline int mmu_notifier_invalidate_range_start_nonblock(struct mmu_notifier_range *range) { if (mm_has_notifiers(range->mm)) { - range->blockable = false; + range->flags &= ~MMU_NOTIFIER_RANGE_BLOCKABLE; return __mmu_notifier_invalidate_range_start(range); } return 0; @@ -331,6 +333,7 @@ static inline void mmu_notifier_range_init(struct mmu_notifier_range *range, range->mm = mm; range->start = start; range->end = end; + range->flags = 0; } #define ptep_clear_flush_young_notify(__vma, __address, __ptep) \ From patchwork Tue Mar 26 16:47:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10871617 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8C5C0139A for ; Tue, 26 Mar 2019 16:48:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 75B6828D00 for ; Tue, 26 Mar 2019 16:48:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 69BA228D5D; Tue, 26 Mar 2019 16:48:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD2F228D00 for ; Tue, 26 Mar 2019 16:48:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8E23C6B028D; Tue, 26 Mar 2019 12:48:12 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8910D6B0290; Tue, 26 Mar 2019 12:48:12 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70F136B0291; Tue, 26 Mar 2019 12:48:12 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by kanga.kvack.org (Postfix) with ESMTP id 4E4416B028D for ; Tue, 26 Mar 2019 12:48:12 -0400 (EDT) Received: by mail-qt1-f200.google.com with SMTP id x58so14159794qtc.1 for ; Tue, 26 Mar 2019 09:48:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=qd+j517MeUYcwakOP+kXZkhhiv8aT+TIOX0bOZ6xYI0=; b=kxZD11mOb4iRe6Py1bMZMcKOBl+gTiMflzs7sxcIpoHXagUvK0K4otr/vlIP3lqmgS Ii291Cagjrj1a6gRg2o9Jk5uZ+pBRWgFhfsEqMamAbxm8nUNdyvGRpQqRyJiVsl7BpJF bfTP8/qma8MqLNRd3zYEZrr2MoAf94PhMa0wNAlDSDd3UqRCecnt5vT0YGMgPd0HTELb shkfHDzlCOjXdwXg133yASfWhlziRRWAgYguoHOlY2cGTVU6H6gy3WkhfshBbbfPE7/i QGwKdR7ihqKHHyvmTyWj8TzaETFwsKpzlrC7DNR7vM3A+YpvjPoLeHjz0z90bN4nouu3 5nsQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APjAAAVi11TrePEu1AadhI/QT1zAU4GbNYSGflYlbwQ2dhG4Gy0LYUNV sv7v+fx9UKW+WPgrCvkCnaEZWbScC+n/RlGt8ympeHwk+kjm+KC9K1nkhMjLIIIcYOwGnspgsp1 0z9KKjMhSCBfW5/2JvsE0VMIeihLU98sWlC2UtRmQZWK5dnP0clq7O1lI7MRGHM43oQ== X-Received: by 2002:a0c:b00c:: with SMTP id k12mr26923791qvc.118.1553618892079; Tue, 26 Mar 2019 09:48:12 -0700 (PDT) X-Google-Smtp-Source: APXvYqweGED+Fpk4SZn7BTvuU2FSSM3nHji3fcEPiZfHyTjNAe/JGSID5nTY0QXv/Fmkk9RuhhJ2 X-Received: by 2002:a0c:b00c:: with SMTP id k12mr26923743qvc.118.1553618891440; Tue, 26 Mar 2019 09:48:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553618891; cv=none; d=google.com; s=arc-20160816; b=qQxk3IAUgi+DYvKQc5P8kZ5J8DmPbuCQQ4Vef88eGh9coIPCLNMoS1WwNwVOzFgtxC uDxKCPJ3lddMOQtM35WkpQ+VFanxjuW5lYb+RMDKt5Q2gfHZgl1gYuwcbu2XpvNbeFSg EULSuFiSFIAmWvmjvTCM46heOMWBUcX8CP/WVt7bDZUEESg64Ws5QAzUaRxZwAdcFqGe DMDZFZv9YXjtYbFHm74Xm59b7rGo3V71BVQbjPoiudT4LBYRqMBFhCmPfvSmRutzwPOZ Dd5u8i8XwhEUDk7hBCNkJ0KmZStFfx2c8W8nqabcRCQgruRVpEAbAAz2cQigP9YXT9OC keaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=qd+j517MeUYcwakOP+kXZkhhiv8aT+TIOX0bOZ6xYI0=; b=Uc1Rl4lUSNnwawnHmkTfsLjmOm/G1a5q0CDKqfmCpLReVUe7Lj5lP7S+jZjOzxBO8N 7rZy72VD/iX7nvcXqJzTvRcPXBVBzPrtUflcoh3lRnSjhBnNMYkghVvGckJ4i0YC0/Dq zhYghU+Nc7yeLshKElEJtbXMrp7fq0Wfg1FFOb8CVTqx+lSf4K8QQzREOUvxSA2Eu+fg s1lms/Rv3Cl3ILdZv3p2/HBBx4QqUIebiK9PiqsokydsX8uG2er6wQqvNOJmutbVTF+R WXD1hMcuyZ2uIR29Kd9Uy0Z3tySTuB0VEdprbGViAviFyDUZNVqhNVo/QFR4XsSDoOSI mM2Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id a66si3571927qkg.30.2019.03.26.09.48.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Mar 2019 09:48:11 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 761CCC04B2F6; Tue, 26 Mar 2019 16:48:10 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.20.6.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id 53C89176B1; Tue, 26 Mar 2019 16:48:08 +0000 (UTC) From: jglisse@redhat.com To: linux-kernel@vger.kernel.org Cc: =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Andrew Morton , linux-mm@kvack.org, =?utf-8?q?Ch?= =?utf-8?q?ristian_K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v6 4/8] mm/mmu_notifier: contextual information for event enums Date: Tue, 26 Mar 2019 12:47:43 -0400 Message-Id: <20190326164747.24405-5-jglisse@redhat.com> In-Reply-To: <20190326164747.24405-1-jglisse@redhat.com> References: <20190326164747.24405-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 26 Mar 2019 16:48:10 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse CPU page table update can happens for many reasons, not only as a result of a syscall (munmap(), mprotect(), mremap(), madvise(), ...) but also as a result of kernel activities (memory compression, reclaim, migration, ...). This patch introduce a set of enums that can be associated with each of the events triggering a mmu notifier. Latter patches take advantages of those enum values. - UNMAP: munmap() or mremap() - CLEAR: page table is cleared (migration, compaction, reclaim, ...) - PROTECTION_VMA: change in access protections for the range - PROTECTION_PAGE: change in access protections for page in the range - SOFT_DIRTY: soft dirtyness tracking Being able to identify munmap() and mremap() from other reasons why the page table is cleared is important to allow user of mmu notifier to update their own internal tracking structure accordingly (on munmap or mremap it is not longer needed to track range of virtual address as it becomes invalid). Signed-off-by: Jérôme Glisse Cc: Andrew Morton Cc: linux-mm@kvack.org Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann --- include/linux/mmu_notifier.h | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index c8672c366f67..2386e71ac1b8 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -10,6 +10,36 @@ struct mmu_notifier; struct mmu_notifier_ops; +/** + * enum mmu_notifier_event - reason for the mmu notifier callback + * @MMU_NOTIFY_UNMAP: either munmap() that unmap the range or a mremap() that + * move the range + * + * @MMU_NOTIFY_CLEAR: clear page table entry (many reasons for this like + * madvise() or replacing a page by another one, ...). + * + * @MMU_NOTIFY_PROTECTION_VMA: update is due to protection change for the range + * ie using the vma access permission (vm_page_prot) to update the whole range + * is enough no need to inspect changes to the CPU page table (mprotect() + * syscall) + * + * @MMU_NOTIFY_PROTECTION_PAGE: update is due to change in read/write flag for + * pages in the range so to mirror those changes the user must inspect the CPU + * page table (from the end callback). + * + * @MMU_NOTIFY_SOFT_DIRTY: soft dirty accounting (still same page and same + * access flags). User should soft dirty the page in the end callback to make + * sure that anyone relying on soft dirtyness catch pages that might be written + * through non CPU mappings. + */ +enum mmu_notifier_event { + MMU_NOTIFY_UNMAP = 0, + MMU_NOTIFY_CLEAR, + MMU_NOTIFY_PROTECTION_VMA, + MMU_NOTIFY_PROTECTION_PAGE, + MMU_NOTIFY_SOFT_DIRTY, +}; + #ifdef CONFIG_MMU_NOTIFIER /* From patchwork Tue Mar 26 16:47:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10871623 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D6AE513B5 for ; Tue, 26 Mar 2019 16:48:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C3D2E28D00 for ; Tue, 26 Mar 2019 16:48:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B7E6E28D4B; Tue, 26 Mar 2019 16:48:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6DA2D28D5D for ; Tue, 26 Mar 2019 16:48:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A3F066B0291; Tue, 26 Mar 2019 12:48:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9F79D6B0293; Tue, 26 Mar 2019 12:48:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8943C6B0292; Tue, 26 Mar 2019 12:48:15 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by kanga.kvack.org (Postfix) with ESMTP id 58D0A6B0290 for ; Tue, 26 Mar 2019 12:48:15 -0400 (EDT) Received: by mail-qt1-f200.google.com with SMTP id i3so14111352qtc.7 for ; Tue, 26 Mar 2019 09:48:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=APTlQjScOT37emZh6UYQOyFSJa80zn6lw9X4MvzbVM8=; b=Z++yYZWrp/Mg6fTAbOv/Gvk/B+m4jWvnsDs8uEcWUoHiPb0PKqKrLda/y1YcF/QZ5j alVUP/p2t7ROylN7ec/eGZ53Ulp0Yw5ovymcxI9scjBcQaRnIHwU6iU5UYs3ASqwD1Dx Tfnfz2PIMTkSYkZRvdBCfO1VWASlgkQ/9nJIczTo9KM0Lzgr6/mQ8hL3zsD4vYjum4sF cXYIzDNYgam0YRCek0lA3Kart4mkCnAVp708rJTmsQYsOi2svdno1d4qJbN3X3VJXRRR wM/gT+GFs81s5d3T4eCgwjpcc1U1r2zS+38yfzcoEpqYYKMccMu1YMdTp5bKZo/s55/C Npeg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APjAAAUQGuAXsaX9AvQ5rktfuenXM0PrZr0FP33CZQDP35GgcjmWhquW R38MO0Z7yxBSykWCvNobagBp3e5McP2jpNT+OJg2aW+9c3Zq9Lv6Yz/NCQ5ZXI+xJyrEUEvL6ZX 3uiFJiG3h0NqKnriTKMfn2hYdddrUx75uBhXNUHsjV6ZKlug16wy4vDZG+QpUZ7NIVg== X-Received: by 2002:a37:7ac7:: with SMTP id v190mr24713134qkc.275.1553618895067; Tue, 26 Mar 2019 09:48:15 -0700 (PDT) X-Google-Smtp-Source: APXvYqxIJBw/R0599BpEp2RtMflccR58FDd7eJMiB/Y3OYpsT3zHIBXBlBHzrZVQOHfOjY4uUlEG X-Received: by 2002:a37:7ac7:: with SMTP id v190mr24713063qkc.275.1553618893801; Tue, 26 Mar 2019 09:48:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553618893; cv=none; d=google.com; s=arc-20160816; b=bo4plQCsaKEVJJKeDRdyNSnDSCRyv9xFddBbzG1P8LUNe/gfgmMErHNbrK4Gu1P0yz a94QIxCFCCPIq03Z8xzFX/MKFGzKmbyBoYZGojWvjfa+t7LUbhUAAxuO3QoWWt1cv6vj KpbX2Ew9VATsYBHIwRR+2PE1iq1oEOB0y5ML3gPbH3VWRTJNmk4bYC+8+QClE7ec2F3m rSxnTMmvbJT6D5vTCOZuebrYpiK+jPwvjRcFSt9G+Ke7vsbnmtZD/geaEvIcm8Z6O39E HTad9HTKmLQfVdPXLi4DweTarEgkRVJ01CaK9Ttil7wgX1QMs6kRcRz886WoPTyOTdZV PatQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=APTlQjScOT37emZh6UYQOyFSJa80zn6lw9X4MvzbVM8=; b=PBPOyCBoe24E8W7Lvlgph7+uWwkCYg6747CJVITiq5jXtZhMJJnHOIvTZMJKIbspaF Xi88w8L9qn2LxKIe6DViYQr+Oy6RhqpBD2vFVPd6zXiKR37rrTqUXzWu6FW9gJvt3uK+ nAVpBN5iDF7l0l3fW8q8vxztqJiAAOhTsbsm62HM2RSk/mcbOhOQQ7Tbt7Zbu6GkJHW5 Rov0sMirp8cDk4sRItf1AX4vDZTN6gVkPAirL1F82DosRFL5sv76uTTuOMQh3h6dOSUM BHee82QUDZZIfPLc1wI10r0miHlwSqUaSGSNe5IgbOefeyC20gqhULVlI+mH4Gw0vW1v 8BOA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id o185si2242843qkb.48.2019.03.26.09.48.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Mar 2019 09:48:13 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CDD293092666; Tue, 26 Mar 2019 16:48:12 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.20.6.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9611762A87; Tue, 26 Mar 2019 16:48:10 +0000 (UTC) From: jglisse@redhat.com To: linux-kernel@vger.kernel.org Cc: =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Andrew Morton , linux-mm@kvack.org, =?utf-8?q?Ch?= =?utf-8?q?ristian_K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v6 5/8] mm/mmu_notifier: contextual information for event triggering invalidation v2 Date: Tue, 26 Mar 2019 12:47:44 -0400 Message-Id: <20190326164747.24405-6-jglisse@redhat.com> In-Reply-To: <20190326164747.24405-1-jglisse@redhat.com> References: <20190326164747.24405-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Tue, 26 Mar 2019 16:48:13 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse CPU page table update can happens for many reasons, not only as a result of a syscall (munmap(), mprotect(), mremap(), madvise(), ...) but also as a result of kernel activities (memory compression, reclaim, migration, ...). Users of mmu notifier API track changes to the CPU page table and take specific action for them. While current API only provide range of virtual address affected by the change, not why the changes is happening. This patchset do the initial mechanical convertion of all the places that calls mmu_notifier_range_init to also provide the default MMU_NOTIFY_UNMAP event as well as the vma if it is know (most invalidation happens against a given vma). Passing down the vma allows the users of mmu notifier to inspect the new vma page protection. The MMU_NOTIFY_UNMAP is always the safe default as users of mmu notifier should assume that every for the range is going away when that event happens. A latter patch do convert mm call path to use a more appropriate events for each call. Changes since v1: - add the flags parameter to init range flags This is done as 2 patches so that no call site is forgotten especialy as it uses this following coccinelle patch: %<---------------------------------------------------------------------- @@ identifier I1, I2, I3, I4; @@ static inline void mmu_notifier_range_init(struct mmu_notifier_range *I1, +enum mmu_notifier_event event, +unsigned flags, +struct vm_area_struct *vma, struct mm_struct *I2, unsigned long I3, unsigned long I4) { ... } @@ @@ -#define mmu_notifier_range_init(range, mm, start, end) +#define mmu_notifier_range_init(range, event, flags, vma, mm, start, end) @@ expression E1, E3, E4; identifier I1; @@ <... mmu_notifier_range_init(E1, +MMU_NOTIFY_UNMAP, 0, I1, I1->vm_mm, E3, E4) ...> @@ expression E1, E2, E3, E4; identifier FN, VMA; @@ FN(..., struct vm_area_struct *VMA, ...) { <... mmu_notifier_range_init(E1, +MMU_NOTIFY_UNMAP, 0, VMA, E2, E3, E4) ...> } @@ expression E1, E2, E3, E4; identifier FN, VMA; @@ FN(...) { struct vm_area_struct *VMA; <... mmu_notifier_range_init(E1, +MMU_NOTIFY_UNMAP, 0, VMA, E2, E3, E4) ...> } @@ expression E1, E2, E3, E4; identifier FN; @@ FN(...) { <... mmu_notifier_range_init(E1, +MMU_NOTIFY_UNMAP, 0, NULL, E2, E3, E4) ...> } ---------------------------------------------------------------------->% Applied with: spatch --all-includes --sp-file mmu-notifier.spatch fs/proc/task_mmu.c --in-place spatch --sp-file mmu-notifier.spatch --dir kernel/events/ --in-place spatch --sp-file mmu-notifier.spatch --dir mm --in-place Signed-off-by: Jérôme Glisse Cc: Andrew Morton Cc: linux-mm@kvack.org Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann --- fs/proc/task_mmu.c | 3 ++- include/linux/mmu_notifier.h | 5 ++++- kernel/events/uprobes.c | 3 ++- mm/huge_memory.c | 12 ++++++++---- mm/hugetlb.c | 12 ++++++++---- mm/khugepaged.c | 3 ++- mm/ksm.c | 6 ++++-- mm/madvise.c | 3 ++- mm/memory.c | 25 ++++++++++++++++--------- mm/migrate.c | 5 ++++- mm/mprotect.c | 3 ++- mm/mremap.c | 3 ++- mm/oom_kill.c | 3 ++- mm/rmap.c | 6 ++++-- 14 files changed, 62 insertions(+), 30 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 92a91e7816d8..fcbd0e574917 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1151,7 +1151,8 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf, break; } - mmu_notifier_range_init(&range, mm, 0, -1UL); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, + NULL, mm, 0, -1UL); mmu_notifier_invalidate_range_start(&range); } walk_page_range(0, mm->highest_vm_end, &clear_refs_walk); diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 2386e71ac1b8..62f94cd85455 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -356,6 +356,9 @@ static inline void mmu_notifier_mm_destroy(struct mm_struct *mm) static inline void mmu_notifier_range_init(struct mmu_notifier_range *range, + enum mmu_notifier_event event, + unsigned flags, + struct vm_area_struct *vma, struct mm_struct *mm, unsigned long start, unsigned long end) @@ -491,7 +494,7 @@ static inline void _mmu_notifier_range_init(struct mmu_notifier_range *range, range->end = end; } -#define mmu_notifier_range_init(range, mm, start, end) \ +#define mmu_notifier_range_init(range,event,flags,vma,mm,start,end) \ _mmu_notifier_range_init(range, start, end) static inline bool diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index c5cde87329c7..77c3f079c723 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -161,7 +161,8 @@ static int __replace_page(struct vm_area_struct *vma, unsigned long addr, struct mmu_notifier_range range; struct mem_cgroup *memcg; - mmu_notifier_range_init(&range, mm, addr, addr + PAGE_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, addr, + addr + PAGE_SIZE); VM_BUG_ON_PAGE(PageTransHuge(old_page), old_page); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 404acdcd0455..4309939be22d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1184,7 +1184,8 @@ static vm_fault_t do_huge_pmd_wp_page_fallback(struct vm_fault *vmf, cond_resched(); } - mmu_notifier_range_init(&range, vma->vm_mm, haddr, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + haddr, haddr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -1348,7 +1349,8 @@ vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd) vma, HPAGE_PMD_NR); __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, vma->vm_mm, haddr, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + haddr, haddr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -2024,7 +2026,8 @@ void __split_huge_pud(struct vm_area_struct *vma, pud_t *pud, spinlock_t *ptl; struct mmu_notifier_range range; - mmu_notifier_range_init(&range, vma->vm_mm, address & HPAGE_PUD_MASK, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + address & HPAGE_PUD_MASK, (address & HPAGE_PUD_MASK) + HPAGE_PUD_SIZE); mmu_notifier_invalidate_range_start(&range); ptl = pud_lock(vma->vm_mm, pud); @@ -2242,7 +2245,8 @@ void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, spinlock_t *ptl; struct mmu_notifier_range range; - mmu_notifier_range_init(&range, vma->vm_mm, address & HPAGE_PMD_MASK, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + address & HPAGE_PMD_MASK, (address & HPAGE_PMD_MASK) + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); ptl = pmd_lock(vma->vm_mm, pmd); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 97b1e0290c66..c20a8d2de3f3 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3247,7 +3247,8 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, cow = (vma->vm_flags & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE; if (cow) { - mmu_notifier_range_init(&range, src, vma->vm_start, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, src, + vma->vm_start, vma->vm_end); mmu_notifier_invalidate_range_start(&range); } @@ -3359,7 +3360,8 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, /* * If sharing possible, alert mmu notifiers of worst case. */ - mmu_notifier_range_init(&range, mm, start, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, start, + end); adjust_range_if_pmd_sharing_possible(vma, &range.start, &range.end); mmu_notifier_invalidate_range_start(&range); address = start; @@ -3626,7 +3628,8 @@ static vm_fault_t hugetlb_cow(struct mm_struct *mm, struct vm_area_struct *vma, pages_per_huge_page(h)); __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, mm, haddr, haddr + huge_page_size(h)); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, haddr, + haddr + huge_page_size(h)); mmu_notifier_invalidate_range_start(&range); /* @@ -4358,7 +4361,8 @@ unsigned long hugetlb_change_protection(struct vm_area_struct *vma, * start/end. Set range.start/range.end to cover the maximum possible * range if PMD sharing is possible. */ - mmu_notifier_range_init(&range, mm, start, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, start, + end); adjust_range_if_pmd_sharing_possible(vma, &range.start, &range.end); BUG_ON(address >= end); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 449044378782..e7944f5e6258 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1016,7 +1016,8 @@ static void collapse_huge_page(struct mm_struct *mm, pte = pte_offset_map(pmd, address); pte_ptl = pte_lockptr(mm, pmd); - mmu_notifier_range_init(&range, mm, address, address + HPAGE_PMD_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, NULL, mm, + address, address + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); pmd_ptl = pmd_lock(mm, pmd); /* probably unnecessary */ /* diff --git a/mm/ksm.c b/mm/ksm.c index fc64874dc6f4..01f5fe2c90cf 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1066,7 +1066,8 @@ static int write_protect_page(struct vm_area_struct *vma, struct page *page, BUG_ON(PageTransCompound(page)); - mmu_notifier_range_init(&range, mm, pvmw.address, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, + pvmw.address, pvmw.address + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -1154,7 +1155,8 @@ static int replace_page(struct vm_area_struct *vma, struct page *page, if (!pmd) goto out; - mmu_notifier_range_init(&range, mm, addr, addr + PAGE_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, addr, + addr + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); ptep = pte_offset_map_lock(mm, pmd, addr, &ptl); diff --git a/mm/madvise.c b/mm/madvise.c index 21a7881a2db4..c617f53a9c09 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -472,7 +472,8 @@ static int madvise_free_single_vma(struct vm_area_struct *vma, range.end = min(vma->vm_end, end_addr); if (range.end <= vma->vm_start) return -EINVAL; - mmu_notifier_range_init(&range, mm, range.start, range.end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, + range.start, range.end); lru_add_drain(); tlb_gather_mmu(&tlb, mm, range.start, range.end); diff --git a/mm/memory.c b/mm/memory.c index 47fe250307c7..ac6754bb30c8 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1010,7 +1010,8 @@ int copy_page_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, is_cow = is_cow_mapping(vma->vm_flags); if (is_cow) { - mmu_notifier_range_init(&range, src_mm, addr, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, + src_mm, addr, end); mmu_notifier_invalidate_range_start(&range); } @@ -1334,7 +1335,8 @@ void unmap_vmas(struct mmu_gather *tlb, { struct mmu_notifier_range range; - mmu_notifier_range_init(&range, vma->vm_mm, start_addr, end_addr); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + start_addr, end_addr); mmu_notifier_invalidate_range_start(&range); for ( ; vma && vma->vm_start < end_addr; vma = vma->vm_next) unmap_single_vma(tlb, vma, start_addr, end_addr, NULL); @@ -1356,7 +1358,8 @@ void zap_page_range(struct vm_area_struct *vma, unsigned long start, struct mmu_gather tlb; lru_add_drain(); - mmu_notifier_range_init(&range, vma->vm_mm, start, start + size); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + start, start + size); tlb_gather_mmu(&tlb, vma->vm_mm, start, range.end); update_hiwater_rss(vma->vm_mm); mmu_notifier_invalidate_range_start(&range); @@ -1382,7 +1385,8 @@ static void zap_page_range_single(struct vm_area_struct *vma, unsigned long addr struct mmu_gather tlb; lru_add_drain(); - mmu_notifier_range_init(&range, vma->vm_mm, address, address + size); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + address, address + size); tlb_gather_mmu(&tlb, vma->vm_mm, address, range.end); update_hiwater_rss(vma->vm_mm); mmu_notifier_invalidate_range_start(&range); @@ -2278,7 +2282,8 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, mm, vmf->address & PAGE_MASK, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, + vmf->address & PAGE_MASK, (vmf->address & PAGE_MASK) + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -4103,8 +4108,9 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, goto out; if (range) { - mmu_notifier_range_init(range, mm, address & PMD_MASK, - (address & PMD_MASK) + PMD_SIZE); + mmu_notifier_range_init(range, MMU_NOTIFY_UNMAP, 0, + NULL, mm, address & PMD_MASK, + (address & PMD_MASK) + PMD_SIZE); mmu_notifier_invalidate_range_start(range); } *ptlp = pmd_lock(mm, pmd); @@ -4121,8 +4127,9 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, goto out; if (range) { - mmu_notifier_range_init(range, mm, address & PAGE_MASK, - (address & PAGE_MASK) + PAGE_SIZE); + mmu_notifier_range_init(range, MMU_NOTIFY_UNMAP, 0, NULL, mm, + address & PAGE_MASK, + (address & PAGE_MASK) + PAGE_SIZE); mmu_notifier_invalidate_range_start(range); } ptep = pte_offset_map_lock(mm, pmd, address, ptlp); diff --git a/mm/migrate.c b/mm/migrate.c index ac6f4939bb59..8f6e4382f0ad 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2351,7 +2351,8 @@ static void migrate_vma_collect(struct migrate_vma *migrate) mm_walk.mm = migrate->vma->vm_mm; mm_walk.private = migrate; - mmu_notifier_range_init(&range, mm_walk.mm, migrate->start, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, NULL, mm_walk.mm, + migrate->start, migrate->end); mmu_notifier_invalidate_range_start(&range); walk_page_range(migrate->start, migrate->end, &mm_walk); @@ -2759,6 +2760,8 @@ static void migrate_vma_pages(struct migrate_vma *migrate) notified = true; mmu_notifier_range_init(&range, + MMU_NOTIFY_UNMAP, 0, + NULL, migrate->vma->vm_mm, addr, migrate->end); mmu_notifier_invalidate_range_start(&range); diff --git a/mm/mprotect.c b/mm/mprotect.c index 028c724dcb1a..b10984052ae9 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -185,7 +185,8 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma, /* invoke the mmu notifier if the pmd is populated */ if (!range.start) { - mmu_notifier_range_init(&range, vma->vm_mm, addr, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, + vma, vma->vm_mm, addr, end); mmu_notifier_invalidate_range_start(&range); } diff --git a/mm/mremap.c b/mm/mremap.c index e3edef6b7a12..fc241d23cd97 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -249,7 +249,8 @@ unsigned long move_page_tables(struct vm_area_struct *vma, old_end = old_addr + len; flush_cache_range(vma, old_addr, old_end); - mmu_notifier_range_init(&range, vma->vm_mm, old_addr, old_end); + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + old_addr, old_end); mmu_notifier_invalidate_range_start(&range); for (; old_addr < old_end; old_addr += extent, new_addr += extent) { diff --git a/mm/oom_kill.c b/mm/oom_kill.c index 3a2484884cfd..539c91d0b26a 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -531,7 +531,8 @@ bool __oom_reap_task_mm(struct mm_struct *mm) struct mmu_notifier_range range; struct mmu_gather tlb; - mmu_notifier_range_init(&range, mm, vma->vm_start, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, + vma, mm, vma->vm_start, vma->vm_end); tlb_gather_mmu(&tlb, mm, range.start, range.end); if (mmu_notifier_invalidate_range_start_nonblock(&range)) { diff --git a/mm/rmap.c b/mm/rmap.c index b30c7c71d1d9..dba724f83ddd 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -896,7 +896,8 @@ static bool page_mkclean_one(struct page *page, struct vm_area_struct *vma, * We have to assume the worse case ie pmd for invalidation. Note that * the page can not be free from this function. */ - mmu_notifier_range_init(&range, vma->vm_mm, address, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + address, min(vma->vm_end, address + (PAGE_SIZE << compound_order(page)))); mmu_notifier_invalidate_range_start(&range); @@ -1371,7 +1372,8 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, * Note that the page can not be free in this function as call of * try_to_unmap() must hold a reference on the page. */ - mmu_notifier_range_init(&range, vma->vm_mm, address, + mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + address, min(vma->vm_end, address + (PAGE_SIZE << compound_order(page)))); if (PageHuge(page)) { From patchwork Tue Mar 26 16:47:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10871629 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AC25B13B5 for ; Tue, 26 Mar 2019 16:48:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9090F28D00 for ; Tue, 26 Mar 2019 16:48:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 81B0228DCC; Tue, 26 Mar 2019 16:48:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5F91128D00 for ; Tue, 26 Mar 2019 16:48:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 55F9D6B0295; Tue, 26 Mar 2019 12:48:24 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 50DB26B0296; Tue, 26 Mar 2019 12:48:24 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3FDB36B0297; Tue, 26 Mar 2019 12:48:24 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by kanga.kvack.org (Postfix) with ESMTP id 2063A6B0295 for ; Tue, 26 Mar 2019 12:48:24 -0400 (EDT) Received: by mail-qt1-f197.google.com with SMTP id f89so14148032qtb.4 for ; Tue, 26 Mar 2019 09:48:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=5aXxvtn0O4+PNqIUkx9q2en4OaUhYXZSTYKnUsWACz8=; b=hcwBG1l7iEp6UK7pwZust7TVSyRhEnLjbG9zTnPM+aW3hzFBaukMpB+W1/jVvsnH6b SH4XgpljPX3Dh6KH49TuI9SWkwp6XoVAgFKUnYdatKyBy4qLWaArZTgnBBIcZMMMvuQB IsVOcy8utiTnk/PusG2Kribl/v7jyeiN3V5btiBnEe9sSHPg3GYXSj8/NkSOW8FsRxpn fk4yW+cTSoBC/xAwt25YNLginiX5q866CVqVjxGsfy5GgpxvPzNprZPDxzAr6GF2jriU +zkHM3eAZA+NUtxUkUuSDvU/o+3bny7t38bAt/FD0fR2f/fuG1FFlub1HeHZLcWg/azS +xNQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APjAAAWs7GhQybjnn42auvA6OArGwf254eYAZJDt0Iz/VZOAM63PM1TI nahz3FSMfMXcjI6Z2uaaH+s6bA0mPjL2Ek5Y/RJG+pcnVWhdjqd82AsEoFfGP3JtK37b7SyJUad XGu8cc6NuVZQGmLjBcunY5L8X7sPWoK4A06MYmnzSOuJTvnUBgk9GAfYfyO6W4REsTg== X-Received: by 2002:a0c:aee6:: with SMTP id n38mr25278125qvd.69.1553618903859; Tue, 26 Mar 2019 09:48:23 -0700 (PDT) X-Google-Smtp-Source: APXvYqydVyOUqgP3E9kLHtgUl/IsC99E/oOfk/dzw6AcfCFdWTtR/zEoxyGjhalo2eOa1p79FgrP X-Received: by 2002:a0c:aee6:: with SMTP id n38mr25278033qvd.69.1553618902574; Tue, 26 Mar 2019 09:48:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553618902; cv=none; d=google.com; s=arc-20160816; b=QCnFCHml1vprOwiXM70VjZ0pltn72ca6ufLH42fgdNZHlQU1OOTzvp1sitacCb7p+c A53ABgm3l8ClMUOSdtnvBdNPZU4XnXSupNQIZduXN2uJjeaPJlEi9Wk8ifs+9G0GBZCV W2N/Rn63oqeCqcI7U99M1RmHTqLAfDTiLaxJCwPve4AzRXLldhnUg5Vlm9kqtaoRu7Cz UXsN8Yw1y3eiGW7BCzTIbszzjpYR59lxL+9KrC9LmmfD5VO7N6n8p7ORRe+kbEhrDmCd 3pwee/N0uFfZh9rA2raiKJj/3sOlp5Q5+jaMqmtdRLWZYr6UC6gJhYIBJajJySB5gVtP MKkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=5aXxvtn0O4+PNqIUkx9q2en4OaUhYXZSTYKnUsWACz8=; b=L7ao2pPWG3ffbwLx/TvXybgtmUlHhLGC9HxU8aO44NzUMqjcDTaqfYBTdiD3E0cvbN o/E5/eQgKxS+fo9PjOi5dC1qBrsLrvzkO3rDqpS7rUx6vbnhtiuv0D0teSd24Pz1yABR 1qcxLK70xtNoV7GFtzn1YQH6qpaH7T5UvNqoHf1nzKc3/N8HCMbx0R0b3+c5od7M2tZ9 XASUFgRQLmeTFarS8mYr3TlHCRhOKsV0FH3TtzcOHQdY2XmJH9jd2knHYVpUS2hNelsx gOGa98gHuI4MdJ+ucro6RNS/0hwvOiKeBxSTrGyhG2gUsFE/8na/lL+MtU1H8XfmY32N fbWA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id h19si885738qto.168.2019.03.26.09.48.22 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Mar 2019 09:48:22 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 92EEC8667A; Tue, 26 Mar 2019 16:48:21 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.20.6.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id EA91C17C5F; Tue, 26 Mar 2019 16:48:12 +0000 (UTC) From: jglisse@redhat.com To: linux-kernel@vger.kernel.org Cc: =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Andrew Morton , linux-mm@kvack.org, =?utf-8?q?Ch?= =?utf-8?q?ristian_K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v6 6/8] mm/mmu_notifier: use correct mmu_notifier events for each invalidation Date: Tue, 26 Mar 2019 12:47:45 -0400 Message-Id: <20190326164747.24405-7-jglisse@redhat.com> In-Reply-To: <20190326164747.24405-1-jglisse@redhat.com> References: <20190326164747.24405-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 26 Mar 2019 16:48:21 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse This update each existing invalidation to use the correct mmu notifier event that represent what is happening to the CPU page table. See the patch which introduced the events to see the rational behind this. Signed-off-by: Jérôme Glisse Cc: Andrew Morton Cc: linux-mm@kvack.org Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann --- fs/proc/task_mmu.c | 4 ++-- kernel/events/uprobes.c | 2 +- mm/huge_memory.c | 14 ++++++-------- mm/hugetlb.c | 8 ++++---- mm/khugepaged.c | 2 +- mm/ksm.c | 4 ++-- mm/madvise.c | 2 +- mm/memory.c | 14 +++++++------- mm/migrate.c | 4 ++-- mm/mprotect.c | 5 +++-- mm/rmap.c | 6 +++--- 11 files changed, 32 insertions(+), 33 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index fcbd0e574917..3b93ce496dd4 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1151,8 +1151,8 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf, break; } - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, - NULL, mm, 0, -1UL); + mmu_notifier_range_init(&range, MMU_NOTIFY_SOFT_DIRTY, + 0, NULL, mm, 0, -1UL); mmu_notifier_invalidate_range_start(&range); } walk_page_range(0, mm->highest_vm_end, &clear_refs_walk); diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 77c3f079c723..79c84bb48ea9 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -161,7 +161,7 @@ static int __replace_page(struct vm_area_struct *vma, unsigned long addr, struct mmu_notifier_range range; struct mem_cgroup *memcg; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, addr, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, addr, addr + PAGE_SIZE); VM_BUG_ON_PAGE(PageTransHuge(old_page), old_page); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 4309939be22d..f0ad70c29500 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1184,9 +1184,8 @@ static vm_fault_t do_huge_pmd_wp_page_fallback(struct vm_fault *vmf, cond_resched(); } - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, - haddr, - haddr + HPAGE_PMD_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, + haddr, haddr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd); @@ -1349,9 +1348,8 @@ vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd) vma, HPAGE_PMD_NR); __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, - haddr, - haddr + HPAGE_PMD_SIZE); + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, + haddr, haddr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); spin_lock(vmf->ptl); @@ -2026,7 +2024,7 @@ void __split_huge_pud(struct vm_area_struct *vma, pud_t *pud, spinlock_t *ptl; struct mmu_notifier_range range; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, address & HPAGE_PUD_MASK, (address & HPAGE_PUD_MASK) + HPAGE_PUD_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -2245,7 +2243,7 @@ void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, spinlock_t *ptl; struct mmu_notifier_range range; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, address & HPAGE_PMD_MASK, (address & HPAGE_PMD_MASK) + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index c20a8d2de3f3..44fe3565ef37 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3247,7 +3247,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, cow = (vma->vm_flags & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE; if (cow) { - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, src, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, src, vma->vm_start, vma->vm_end); mmu_notifier_invalidate_range_start(&range); @@ -3628,7 +3628,7 @@ static vm_fault_t hugetlb_cow(struct mm_struct *mm, struct vm_area_struct *vma, pages_per_huge_page(h)); __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, haddr, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, haddr, haddr + huge_page_size(h)); mmu_notifier_invalidate_range_start(&range); @@ -4361,8 +4361,8 @@ unsigned long hugetlb_change_protection(struct vm_area_struct *vma, * start/end. Set range.start/range.end to cover the maximum possible * range if PMD sharing is possible. */ - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, start, - end); + mmu_notifier_range_init(&range, MMU_NOTIFY_PROTECTION_VMA, + 0, vma, mm, start, end); adjust_range_if_pmd_sharing_possible(vma, &range.start, &range.end); BUG_ON(address >= end); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index e7944f5e6258..579699d2b347 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1016,7 +1016,7 @@ static void collapse_huge_page(struct mm_struct *mm, pte = pte_offset_map(pmd, address); pte_ptl = pte_lockptr(mm, pmd); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, NULL, mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, NULL, mm, address, address + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); pmd_ptl = pmd_lock(mm, pmd); /* probably unnecessary */ diff --git a/mm/ksm.c b/mm/ksm.c index 01f5fe2c90cf..81c20ed57bf6 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1066,7 +1066,7 @@ static int write_protect_page(struct vm_area_struct *vma, struct page *page, BUG_ON(PageTransCompound(page)); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, pvmw.address, pvmw.address + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -1155,7 +1155,7 @@ static int replace_page(struct vm_area_struct *vma, struct page *page, if (!pmd) goto out; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, addr, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, addr, addr + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); diff --git a/mm/madvise.c b/mm/madvise.c index c617f53a9c09..a692d2a893b5 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -472,7 +472,7 @@ static int madvise_free_single_vma(struct vm_area_struct *vma, range.end = min(vma->vm_end, end_addr); if (range.end <= vma->vm_start) return -EINVAL; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, range.start, range.end); lru_add_drain(); diff --git a/mm/memory.c b/mm/memory.c index ac6754bb30c8..c24c5ffe950f 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1010,8 +1010,8 @@ int copy_page_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, is_cow = is_cow_mapping(vma->vm_flags); if (is_cow) { - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, - src_mm, addr, end); + mmu_notifier_range_init(&range, MMU_NOTIFY_PROTECTION_PAGE, + 0, vma, src_mm, addr, end); mmu_notifier_invalidate_range_start(&range); } @@ -1358,7 +1358,7 @@ void zap_page_range(struct vm_area_struct *vma, unsigned long start, struct mmu_gather tlb; lru_add_drain(); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, start, start + size); tlb_gather_mmu(&tlb, vma->vm_mm, start, range.end); update_hiwater_rss(vma->vm_mm); @@ -1385,7 +1385,7 @@ static void zap_page_range_single(struct vm_area_struct *vma, unsigned long addr struct mmu_gather tlb; lru_add_drain(); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, address, address + size); tlb_gather_mmu(&tlb, vma->vm_mm, address, range.end); update_hiwater_rss(vma->vm_mm); @@ -2282,7 +2282,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) __SetPageUptodate(new_page); - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, mm, vmf->address & PAGE_MASK, (vmf->address & PAGE_MASK) + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -4108,7 +4108,7 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, goto out; if (range) { - mmu_notifier_range_init(range, MMU_NOTIFY_UNMAP, 0, + mmu_notifier_range_init(range, MMU_NOTIFY_CLEAR, 0, NULL, mm, address & PMD_MASK, (address & PMD_MASK) + PMD_SIZE); mmu_notifier_invalidate_range_start(range); @@ -4127,7 +4127,7 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, goto out; if (range) { - mmu_notifier_range_init(range, MMU_NOTIFY_UNMAP, 0, NULL, mm, + mmu_notifier_range_init(range, MMU_NOTIFY_CLEAR, 0, NULL, mm, address & PAGE_MASK, (address & PAGE_MASK) + PAGE_SIZE); mmu_notifier_invalidate_range_start(range); diff --git a/mm/migrate.c b/mm/migrate.c index 8f6e4382f0ad..f37b1a4bf9c0 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2351,7 +2351,7 @@ static void migrate_vma_collect(struct migrate_vma *migrate) mm_walk.mm = migrate->vma->vm_mm; mm_walk.private = migrate; - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, NULL, mm_walk.mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, NULL, mm_walk.mm, migrate->start, migrate->end); mmu_notifier_invalidate_range_start(&range); @@ -2760,7 +2760,7 @@ static void migrate_vma_pages(struct migrate_vma *migrate) notified = true; mmu_notifier_range_init(&range, - MMU_NOTIFY_UNMAP, 0, + MMU_NOTIFY_CLEAR, 0, NULL, migrate->vma->vm_mm, addr, migrate->end); diff --git a/mm/mprotect.c b/mm/mprotect.c index b10984052ae9..65242f1e4457 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -185,8 +185,9 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma, /* invoke the mmu notifier if the pmd is populated */ if (!range.start) { - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, - vma, vma->vm_mm, addr, end); + mmu_notifier_range_init(&range, + MMU_NOTIFY_PROTECTION_VMA, 0, + vma, vma->vm_mm, addr, end); mmu_notifier_invalidate_range_start(&range); } diff --git a/mm/rmap.c b/mm/rmap.c index dba724f83ddd..c26becc3982c 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -896,8 +896,8 @@ static bool page_mkclean_one(struct page *page, struct vm_area_struct *vma, * We have to assume the worse case ie pmd for invalidation. Note that * the page can not be free from this function. */ - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, - address, + mmu_notifier_range_init(&range, MMU_NOTIFY_PROTECTION_PAGE, + 0, vma, vma->vm_mm, address, min(vma->vm_end, address + (PAGE_SIZE << compound_order(page)))); mmu_notifier_invalidate_range_start(&range); @@ -1372,7 +1372,7 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, * Note that the page can not be free in this function as call of * try_to_unmap() must hold a reference on the page. */ - mmu_notifier_range_init(&range, MMU_NOTIFY_UNMAP, 0, vma, vma->vm_mm, + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, address, min(vma->vm_end, address + (PAGE_SIZE << compound_order(page)))); From patchwork Tue Mar 26 16:47:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10871637 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 65379186D for ; Tue, 26 Mar 2019 16:48:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 54F3728D00 for ; Tue, 26 Mar 2019 16:48:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4955528D5D; Tue, 26 Mar 2019 16:48:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB6FF28D00 for ; Tue, 26 Mar 2019 16:48:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 364166B0296; Tue, 26 Mar 2019 12:48:26 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 312ED6B0297; Tue, 26 Mar 2019 12:48:26 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 205A56B0298; Tue, 26 Mar 2019 12:48:26 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by kanga.kvack.org (Postfix) with ESMTP id EF4826B0296 for ; Tue, 26 Mar 2019 12:48:25 -0400 (EDT) Received: by mail-qk1-f199.google.com with SMTP id d131so12036366qkc.18 for ; Tue, 26 Mar 2019 09:48:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=252zMKy7F4cZQRAlSYsRJSdfyv+AZ8onT930rac77Bs=; b=BSQLxBebrEI6HOcr8HXq0QsAP6nsHYU8TgHBu25NEb565Oi9XtC24USPwEyDklcbyr fcfRS5/JDD9QVIXGBDwnicmvzRVLGreD9SweNB0IRWn/FzHGCTOx/2Xq4ui/0tAGUyGg eioLJG8h9+fI7aycsfJ8h+4+eDSMaIEDIdBpuVwNqRS/B7TMHR8OUeor89NzkGWtTRAS 854L5sQPo66o/25+UIpofDj/2asE3WYp3Vgl7cxbpjFKpcr0SoFIisESEYZFcm2ZBysk PDViCFf8eOO5ZHeLIKJI67xd6e5Kfj5MWfGEqXbjzG6wee8+zBWeUAAmbOmRMRjD/5gv 5WLQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APjAAAXB/DX+u9caEWCuX/ts7QnX3jdGsrEPjjIlIPf9UwfIL/hZjsET Qp6/PtQY7eaCtEqLG6i5jYZrOWn+YgAUcdlqIL4N7nR1qe/eQm59Vro8egV/5fsNANvVfK5Q70j ybIkbqJLYiB8kLsSqIOBg39Yd09AlZJJ2ECIp3oeqGOBtC5clZP+cRAWi9PYr7B4nzA== X-Received: by 2002:ae9:eb4e:: with SMTP id b75mr25583247qkg.121.1553618905751; Tue, 26 Mar 2019 09:48:25 -0700 (PDT) X-Google-Smtp-Source: APXvYqxGaHnOycR9RHVbznvvUh2H4Re8kQ6ACceKyxBKHAsHqZ0HPUWwj4AEeGb2ZGYxY92/Bo/W X-Received: by 2002:ae9:eb4e:: with SMTP id b75mr25583194qkg.121.1553618905155; Tue, 26 Mar 2019 09:48:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553618905; cv=none; d=google.com; s=arc-20160816; b=gycXpCHtWtronHa6abd79UocJ5qPn+e2OPGr583i9IQMS6wojaqt6oTBBuPl0J74LE pKzIqWHIsPDTWjQEKk3/DXfs6SxbtBWi7IsN90WpyttiFjS29V9BPI8ZDRN3c722XumO ybd4TKzBUIQHUggpFDnECl49N3JIbK3On/IAx1c2srqp7hi2KIgL6pTmINcXZyWmM39p CeysV5wcidRyArhDRKQnyYMS4w2qt8pUZo0NIzy1f2nNM8IKTmM3uCVVSh6W33AYtFkp ye+K8GZEUuj77QkM7BNulZvBP8uPUyW2XrFbD5dKQIlTDmc9Oupg71MNUeU45S45o2r5 NDPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=252zMKy7F4cZQRAlSYsRJSdfyv+AZ8onT930rac77Bs=; b=piTE9E4WDZG9laRLfI/WlWHUOEWqbuHaGsOjDPx2bDXAkGs8qTIjY4gM2gjbRxGwRg BcBQ0h0g4KvdtoD/bE7UsXjFVNQMIedITRlUR/TcTDtr4phcVsl68Wxt8kcrHIxU+0X5 X0dwmd0hel3DLUC1VM6UdFtb7eo8C0NgWaIU0CzQQ19aExJ7xgch6Nfswk3bxrA9Tb64 rJJ2mNe3J/tP9D+ME4qx5HV6lYvMjIk/Q2Go+S5Nd9jg7f/mlyshga1N5j9g68L1FjEB yTbXKysE09qoEkGCE59MbvZCpOXD3nzNgcPlgOxQPluKY7kUSuyEohk3sGsWKzv1D+UZ RMxg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id n12si1363820qtc.99.2019.03.26.09.48.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Mar 2019 09:48:25 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 268F483F3D; Tue, 26 Mar 2019 16:48:24 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.20.6.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id B266717595; Tue, 26 Mar 2019 16:48:21 +0000 (UTC) From: jglisse@redhat.com To: linux-kernel@vger.kernel.org Cc: =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Andrew Morton , linux-mm@kvack.org, =?utf-8?q?Ch?= =?utf-8?q?ristian_K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v6 7/8] mm/mmu_notifier: pass down vma and reasons why mmu notifier is happening v2 Date: Tue, 26 Mar 2019 12:47:46 -0400 Message-Id: <20190326164747.24405-8-jglisse@redhat.com> In-Reply-To: <20190326164747.24405-1-jglisse@redhat.com> References: <20190326164747.24405-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 26 Mar 2019 16:48:24 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse CPU page table update can happens for many reasons, not only as a result of a syscall (munmap(), mprotect(), mremap(), madvise(), ...) but also as a result of kernel activities (memory compression, reclaim, migration, ...). Users of mmu notifier API track changes to the CPU page table and take specific action for them. While current API only provide range of virtual address affected by the change, not why the changes is happening This patch is just passing down the new informations by adding it to the mmu_notifier_range structure. Changes since v1: - Initialize flags field from mmu_notifier_range_init() arguments Signed-off-by: Jérôme Glisse Cc: Andrew Morton Cc: linux-mm@kvack.org Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann Reviewed-by: Ira Weiny --- include/linux/mmu_notifier.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 62f94cd85455..0379956fff23 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -58,10 +58,12 @@ struct mmu_notifier_mm { #define MMU_NOTIFIER_RANGE_BLOCKABLE (1 << 0) struct mmu_notifier_range { + struct vm_area_struct *vma; struct mm_struct *mm; unsigned long start; unsigned long end; unsigned flags; + enum mmu_notifier_event event; }; struct mmu_notifier_ops { @@ -363,10 +365,12 @@ static inline void mmu_notifier_range_init(struct mmu_notifier_range *range, unsigned long start, unsigned long end) { + range->vma = vma; + range->event = event; range->mm = mm; range->start = start; range->end = end; - range->flags = 0; + range->flags = flags; } #define ptep_clear_flush_young_notify(__vma, __address, __ptep) \ From patchwork Tue Mar 26 16:47:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10871643 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5BF33186D for ; Tue, 26 Mar 2019 16:48:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4BAD928D5D for ; Tue, 26 Mar 2019 16:48:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 401F428DCC; Tue, 26 Mar 2019 16:48:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B815628DB8 for ; Tue, 26 Mar 2019 16:48:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 86BB56B0297; Tue, 26 Mar 2019 12:48:28 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 81F826B0298; Tue, 26 Mar 2019 12:48:28 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70F156B0299; Tue, 26 Mar 2019 12:48:28 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by kanga.kvack.org (Postfix) with ESMTP id 391026B0297 for ; Tue, 26 Mar 2019 12:48:28 -0400 (EDT) Received: by mail-qt1-f200.google.com with SMTP id 35so14059494qty.12 for ; Tue, 26 Mar 2019 09:48:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=jnvlAV+GWfMx/7xQbFXaF0JgvpgKRF2qvBTJuIYTYOk=; b=D76z0XJ5uO/yoYIMbDnWQx63DTyNH0Bjs5Xdvya4ZGp6WgDfKjBpvptf5gBe6HtI1B ev5KHAVyZBQ/b2XOYXhAVdNFRKGJG8KNtSNI0lcrMg7m/TDA+KK7f8s327G38hgW5ZWD 9ee8fSH2qhdWQB9xQNzrKcR+b1rxB0LTaZ/nE7/SPXhEsW0ZhE38QDF7/+wo4+nYHC7H ZEABkAPdpAM0WHeyVY2D/FNNuoDGOItoBazUkAaRDNO1XKE+opJxF6zBt7U5x6to80dE BOqiOB3pnIqwl70rImJtWb/e332UO7uFQBDcvFK7QHl7EnczBC/oRcGAfGzQxgkejrXh TsGA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APjAAAWlFQGi5HIp8QHQWFhGbbbqpnJJiJY7ya1s0sPRDzkb9yqZDMj0 csxnVx9DaG5ff6+nh7oQ6nZSZTGXiqwHfLiVkESd0QlXLhGprm4bPJ8WAEwzHOWucu7+xo8GkOx DwQtmhPqmyCx9PycXUHvPEKKyhefhMbTxmB1S7HCeI78kISIRgQOYNAQ9lQJ+ieHyIg== X-Received: by 2002:a37:784:: with SMTP id 126mr24287211qkh.10.1553618908011; Tue, 26 Mar 2019 09:48:28 -0700 (PDT) X-Google-Smtp-Source: APXvYqx55GVNwmoFPjHG9+7g6vF5xHLfb9AH86ZrSYRkOI4V2NJmqECy3VkHTyg7UyHMjj9clmKi X-Received: by 2002:a37:784:: with SMTP id 126mr24287174qkh.10.1553618907430; Tue, 26 Mar 2019 09:48:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553618907; cv=none; d=google.com; s=arc-20160816; b=R1kozFGrKbJFishz2n3MdnZMZegfuNgoX1VF2KTCgy2jtNwKa4Qr+hLkRYr1eLm6bD wWEKAVX23RQJa4pR0xRc3J+kAZl7kVcSLv8yzm4DObdUCFvYw8ydfZ23FS5wouy2nTWq ZkPlOOzG4dcHkb28puQFGPjmmbMY2UA7lL5SWiKlu5WmbKbW4UrZxpJg15wL2oVSf4LA jJ99yCFApNhbTmlPYKYdIpyCi0+hgeB/iq2CP2cCB8mOkbe0Tajx3xWx29pBYay3c3rj kAav3BLt/5s1llm2G/VhscNv1dAWEu7d/QRU1I9G90azMDe+YOQhKR8E1j44WuNi2QDs Bnjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=jnvlAV+GWfMx/7xQbFXaF0JgvpgKRF2qvBTJuIYTYOk=; b=Dv6ITq7bnQ/gUW2hVcvcPI7ea6d1Y+j21nNveq57O7xMaPygOnBGJuuB+AW/J/61U7 /b8qNnyCYOf+DOYkTFM2//g4v6toJHvriFb8vIm1RSn5cuI/vbPs1Zy3ZIi8AGqGuRnw A0edqf2tNZxbOj9fjHNTvYLtdt1eS1tjNUi2yk0fJLicorzh35u2WNnvi+dPmb5agixK /U8CvpbGmssVtgbmJipUChBHEYRepTRlqLtx2myX/vEO1By5QVJ+GsbAfVIcD1/lTAGq pNR6LYSdaOD2/7vCHApiTz4FMlLZmX6LT8wY1DJ1Yu1qLWrs8uEalI/F7rtLMx9Y6HHg MSAQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id k26si11111636qve.94.2019.03.26.09.48.27 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Mar 2019 09:48:27 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7399A3086212; Tue, 26 Mar 2019 16:48:26 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.20.6.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id 485878428A; Tue, 26 Mar 2019 16:48:24 +0000 (UTC) From: jglisse@redhat.com To: linux-kernel@vger.kernel.org Cc: =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Andrew Morton , linux-mm@kvack.org, =?utf-8?q?Ch?= =?utf-8?q?ristian_K=C3=B6nig?= , Joonas Lahtinen , Jani Nikula , Rodrigo Vivi , Jan Kara , Andrea Arcangeli , Peter Xu , Felix Kuehling , Jason Gunthorpe , Ross Zwisler , Dan Williams , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Michal Hocko , Ralph Campbell , John Hubbard , kvm@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, Arnd Bergmann Subject: [PATCH v6 8/8] mm/mmu_notifier: mmu_notifier_range_update_to_read_only() helper Date: Tue, 26 Mar 2019 12:47:47 -0400 Message-Id: <20190326164747.24405-9-jglisse@redhat.com> In-Reply-To: <20190326164747.24405-1-jglisse@redhat.com> References: <20190326164747.24405-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Tue, 26 Mar 2019 16:48:26 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Helper to test if a range is updated to read only (it is still valid to read from the range). This is useful for device driver or anyone who wish to optimize out update when they know that they already have the range map read only. Signed-off-by: Jérôme Glisse Cc: Andrew Morton Cc: linux-mm@kvack.org Cc: Christian König Cc: Joonas Lahtinen Cc: Jani Nikula Cc: Rodrigo Vivi Cc: Jan Kara Cc: Andrea Arcangeli Cc: Peter Xu Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: Ross Zwisler Cc: Dan Williams Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Michal Hocko Cc: Christian Koenig Cc: Ralph Campbell Cc: John Hubbard Cc: kvm@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: Arnd Bergmann --- include/linux/mmu_notifier.h | 4 ++++ mm/mmu_notifier.c | 10 ++++++++++ 2 files changed, 14 insertions(+) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 0379956fff23..b6c004bd9f6a 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -259,6 +259,8 @@ extern void __mmu_notifier_invalidate_range_end(struct mmu_notifier_range *r, bool only_end); extern void __mmu_notifier_invalidate_range(struct mm_struct *mm, unsigned long start, unsigned long end); +extern bool +mmu_notifier_range_update_to_read_only(const struct mmu_notifier_range *range); static inline bool mmu_notifier_range_blockable(const struct mmu_notifier_range *range) @@ -568,6 +570,8 @@ static inline void mmu_notifier_mm_destroy(struct mm_struct *mm) { } +#define mmu_notifier_range_update_to_read_only(r) false + #define ptep_clear_flush_young_notify ptep_clear_flush_young #define pmdp_clear_flush_young_notify pmdp_clear_flush_young #define ptep_clear_young_notify ptep_test_and_clear_young diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index abd88c466eb2..ee36068077b6 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -395,3 +395,13 @@ void mmu_notifier_unregister_no_release(struct mmu_notifier *mn, mmdrop(mm); } EXPORT_SYMBOL_GPL(mmu_notifier_unregister_no_release); + +bool +mmu_notifier_range_update_to_read_only(const struct mmu_notifier_range *range) +{ + if (!range->vma || range->event != MMU_NOTIFY_PROTECTION_VMA) + return false; + /* Return true if the vma still have the read flag set. */ + return range->vma->vm_flags & VM_READ; +} +EXPORT_SYMBOL_GPL(mmu_notifier_range_update_to_read_only);