From patchwork Fri Oct 19 16:04:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10649727 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8E13617DE for ; Fri, 19 Oct 2018 16:05:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7B439283FE for ; Fri, 19 Oct 2018 16:05:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6F76F283C7; Fri, 19 Oct 2018 16:05:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC58E28329 for ; Fri, 19 Oct 2018 16:05:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 69DEE6B026B; Fri, 19 Oct 2018 12:04:56 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6268F6B026C; Fri, 19 Oct 2018 12:04:56 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A3C96B026D; Fri, 19 Oct 2018 12:04:56 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by kanga.kvack.org (Postfix) with ESMTP id 16C856B026C for ; Fri, 19 Oct 2018 12:04:56 -0400 (EDT) Received: by mail-qt1-f197.google.com with SMTP id x7-v6so37121754qtb.6 for ; Fri, 19 Oct 2018 09:04:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=i6KkAVyDd2RTg5dfzpklfysckMhBaeY8PLn3QDkDkDA=; b=pTWk2XtJfwa7do3b8YQRmMfiPN9Fy6cgaK7smk5lCFz5pdFDcD69mul6ZQ1CASHawq 7viJZ8F5shvdpOeKP8tY8lu41RONWHofHgdapYQOt6+ZB2vOZxqArl2tnDpy84qBSS4v XfXmbM/qdPHnWDmtGbOf90zyWCq9EiMtD7j20uB2bR8wnoKNc5rMW8zO+isq6q+QxlPD Tvsnmovi41IRrQKbJOCQgG8WveRaJVvWzHmBLEtObQV2mofWMwOuJhSWstLvgZOd0m2b Pi8SL0moH9C77+931bB479QtRWsF82A+aY1OpPqgFp8zHnah5j9/t7c2ONS7hqvQFJ/o 9OSQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: ABuFfohNIsdsfLJAGoCfV4XUcfnREqhZNDjJEtFrT0EPbLJmvofXe7F7 EXbfosLx4pycbzuM9doRGEPGCP6869ygk9KoMl8ec7kj6e2MIRHaxonUOfl2OD5yv2/eSrbdn4e mB5e1LIiBFJPG62+qTGmtM/HMhI/lfbDHRRk9/KtxYc9f4K2DqZqnvdZCI86K5UAzpA== X-Received: by 2002:a37:5ac5:: with SMTP id o188-v6mr3768860qkb.101.1539965095823; Fri, 19 Oct 2018 09:04:55 -0700 (PDT) X-Google-Smtp-Source: ACcGV635Tu9H4JKtkrKhLnYTigkHhbX0iCWVlwGce/uQ6hICxVrUvqYdwOCeLEW3k/sEueXvO/w8 X-Received: by 2002:a37:5ac5:: with SMTP id o188-v6mr3768815qkb.101.1539965095324; Fri, 19 Oct 2018 09:04:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539965095; cv=none; d=google.com; s=arc-20160816; b=0D6GZFbUnom5tXQ4/e4JutCp3Bqd2w3pbX0xGiMnZ6HP4ysVxYZ/j0GXjz+e8flPZJ jouyNQjDaxKHJD+vuTFc4hrByC/zJ1jPXLwHdhFTE8bzORgTcl70WBrmM/pkQfbM/B+Z XyFzjQwlLd0/FqasyPRAJYEACjXB3Al72rCLxrlmgKer2pc7uKL9n+s58fhCXDy5b++9 BQJJknkRixeCBEtut1Y5aR8ymkhe09rsADbaGCMXA7sQ1mGu1cBySde+7zvKmhC3BpCe TaR8tx6QrwJN6xMpQOMV60fZJqyMMVmnJ1gexW3VGUjMOVwSpBmBGm0VQpw7aX2+l2eu b3Gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=i6KkAVyDd2RTg5dfzpklfysckMhBaeY8PLn3QDkDkDA=; b=cDu1Ze7Qs4IAw4Hmn8I28ykDdAs7OO1QJNkhKWoe3Ke8bwx2oZTrQqWPsO8i2hwQz3 V9QKVXoEfmKqLsnvCbkHGibGymc76xOqCl2KTrSFSUrts75GQKakHlonpq9rvOdFvnPm EHedPA+ITy9okOWmS8Yx49dwTlWIEgZgWEoc657u677prtPjYvSvDBufe/HTOKPW1lK0 W3SbVhVWn3IFtOvztxACl/ihVjQgMMeQQq7ITswZKgiNEVCW5+Du6h47NIhbDFV2a+98 hlCuG2SsyLLbdFYLx+r07J8+ItYKjWjI9XaOysUhB5+BOqdyt6j1t8MjfFl5DXhz764k xCAw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id f4-v6si12636280qkg.151.2018.10.19.09.04.55 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 19 Oct 2018 09:04:55 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 82A9A30020AB; Fri, 19 Oct 2018 16:04:54 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-124-217.rdu2.redhat.com [10.10.124.217]) by smtp.corp.redhat.com (Postfix) with ESMTP id B073D65949; Fri, 19 Oct 2018 16:04:53 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Ralph Campbell , John Hubbard Subject: [PATCH 6/6] mm/hmm: invalidate device page table at start of invalidation Date: Fri, 19 Oct 2018 12:04:42 -0400 Message-Id: <20181019160442.18723-7-jglisse@redhat.com> In-Reply-To: <20181019160442.18723-1-jglisse@redhat.com> References: <20181019160442.18723-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Fri, 19 Oct 2018 16:04:54 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Invalidate device page table at start of invalidation and invalidate in progress CPU page table snapshooting at both start and end of any invalidation. This is helpful when device need to dirty page because the device page table report the page as dirty. Dirtying page must happen in the start mmu notifier callback and not in the end one. Signed-off-by: Jérôme Glisse Cc: Ralph Campbell Cc: John Hubbard Cc: Andrew Morton --- mm/hmm.c | 27 +++++++++++++++------------ 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 0eacf9627bc9..1aecf7c08cff 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -43,7 +43,6 @@ static const struct mmu_notifier_ops hmm_mmu_notifier_ops; * * @mm: mm struct this HMM struct is bound to * @lock: lock protecting ranges list - * @sequence: we track updates to the CPU page table with a sequence number * @ranges: list of range being snapshotted * @mirrors: list of mirrors for this mm * @mmu_notifier: mmu notifier to track updates to CPU page table @@ -52,7 +51,6 @@ static const struct mmu_notifier_ops hmm_mmu_notifier_ops; struct hmm { struct mm_struct *mm; spinlock_t lock; - atomic_t sequence; struct list_head ranges; struct list_head mirrors; struct mmu_notifier mmu_notifier; @@ -85,7 +83,6 @@ static struct hmm *hmm_register(struct mm_struct *mm) return NULL; INIT_LIST_HEAD(&hmm->mirrors); init_rwsem(&hmm->mirrors_sem); - atomic_set(&hmm->sequence, 0); hmm->mmu_notifier.ops = NULL; INIT_LIST_HEAD(&hmm->ranges); spin_lock_init(&hmm->lock); @@ -126,7 +123,7 @@ void hmm_mm_destroy(struct mm_struct *mm) kfree(mm->hmm); } -static int hmm_invalidate_range(struct hmm *hmm, +static int hmm_invalidate_range(struct hmm *hmm, bool device, const struct hmm_update *update) { struct hmm_mirror *mirror; @@ -147,6 +144,9 @@ static int hmm_invalidate_range(struct hmm *hmm, } spin_unlock(&hmm->lock); + if (!device) + return 0; + down_read(&hmm->mirrors_sem); list_for_each_entry(mirror, &hmm->mirrors, list) { int ret; @@ -189,18 +189,21 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) } static int hmm_invalidate_range_start(struct mmu_notifier *mn, - struct mm_struct *mm, - unsigned long start, - unsigned long end, - bool blockable) + struct mm_struct *mm, + unsigned long start, + unsigned long end, + bool blockable) { + struct hmm_update update; struct hmm *hmm = mm->hmm; VM_BUG_ON(!hmm); - atomic_inc(&hmm->sequence); - - return 0; + update.start = start; + update.end = end; + update.event = HMM_UPDATE_INVALIDATE; + update.blockable = blockable; + return hmm_invalidate_range(hmm, true, &update); } static void hmm_invalidate_range_end(struct mmu_notifier *mn, @@ -217,7 +220,7 @@ static void hmm_invalidate_range_end(struct mmu_notifier *mn, update.end = end; update.event = HMM_UPDATE_INVALIDATE; update.blockable = true; - hmm_invalidate_range(hmm, &update); + hmm_invalidate_range(hmm, false, &update); } static const struct mmu_notifier_ops hmm_mmu_notifier_ops = {