From patchwork Fri Oct 19 16:04:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10649717 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3000A109C for ; Fri, 19 Oct 2018 16:04:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1EAED2836F for ; Fri, 19 Oct 2018 16:04:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 12B75283B1; Fri, 19 Oct 2018 16:04:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ADF302836F for ; Fri, 19 Oct 2018 16:04:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D2FF36B000C; Fri, 19 Oct 2018 12:04:50 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CD7AF6B000D; Fri, 19 Oct 2018 12:04:50 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC5E16B000E; Fri, 19 Oct 2018 12:04:50 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id 94B3E6B000D for ; Fri, 19 Oct 2018 12:04:50 -0400 (EDT) Received: by mail-qk1-f198.google.com with SMTP id v198-v6so35784616qka.16 for ; Fri, 19 Oct 2018 09:04:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=vR1AiFB3wpjKWDM0g6zE06NjjRuo3VNVAAq0+o/CSlQ=; b=YFVOlSxypESEfdmMje1SFdlHbQ6SXDjZnTMMdXDgxt/MG/YHUewd5oBeYNsbjAOOi7 3/HkyUvqVH0L2cLtCqeq+WAVRFDIJBIC5Cz8jqL0Yj19Av8fRGvQsh7bj0ziXbF6AYq2 3+ZMvsWT9fTOcM0QY/QwcSDWj0hQvUItOVaJj03XMY5R8QWRJlNWwNvetN+k1cfXy3PT XSX1K5Q2td4tT+jg0f3jsFkbVo6mZB5BikuGdPkQTS30MYEl+9383foNtDn1/uYMcu8U gg0BaJBwDLNCWXGAjptGg3vRrzV7zuT5AiZte/RTri4CaMDtWNGATccrhnssdKapgK// cPGw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: ABuFfohhptxdCXa3V2a42G5i8yGR7vSSIAG18KoRIJH8NALkIOdmdOro 5U+4/b+WvEjQc1s1TVRg+SpCwp0jO+N9t1Dj8pPJS1BkXt51AoKiM9E85ld1p3666cRvesIcB3V pBYVkYy6N+Yn5IEjV3kXgOAadkrPRjXH1LjggNKgQmr80DunYfd42ZBCpJC329PVwbA== X-Received: by 2002:a37:5288:: with SMTP id g130-v6mr34547063qkb.124.1539965090383; Fri, 19 Oct 2018 09:04:50 -0700 (PDT) X-Google-Smtp-Source: ACcGV63V+XDpN1DGec45B+JFHTgcMafMlgWKxkq3s7najyrIqMn8Kt0oo3f2j2X+yG6Q4ghjZd4X X-Received: by 2002:a37:5288:: with SMTP id g130-v6mr34547015qkb.124.1539965089871; Fri, 19 Oct 2018 09:04:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539965089; cv=none; d=google.com; s=arc-20160816; b=rM3rhDIkUFlPwviLyfG/mPmfvKcZKEUz4opWpetW0wyAxIO0Ib0Fl6LhrZApsCgBMk fDyRIldDvnl+ShtgnSjDhudgS7TDjIumMpkLO4piSJoredh+qIxI4KoWD8wx3b/hseIN AebE9M37lnEfWAck2KAKi/g7LZWtLIH8XIAH7hh1fOtAp1zpbdkYbEbnN0TuQdyAsBjq 6gSXH4gfVxL1ApaEwAD6wwfH+7tb8makOgtD2rbZ3QObElT1Fvj1abOSrUXp8Eoi6rAJ Rbdxu4j1Ckv1iwqFZ9MrkotQkFvQVyjAfYp9+oFxUNjjqkazIlFhKKfUsjrM6BRJ3IFq YiDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=vR1AiFB3wpjKWDM0g6zE06NjjRuo3VNVAAq0+o/CSlQ=; b=IvzqpuJ3t0Pt8eaBU2kEvdZcqtObxMxjOb/mvBgSMHKHR5rkmj3cuPAQKpmDIEN+fl a0u+rxa27yDcmRjmve43BfwLtznbk9heCnQO0gpB/6AIrD5mVGf6017W3qhr29D5fnEN KX6hiih/s/3tOzmUQW6+XgD3iSpziEAOzAYePVM7HidrePuYZKQgy+oHut4iNaIxZQ8a J/3EqkU862H/rx9RykbJSmm93md2gaE0Fn2eNaENLOYOwEXKXjtYEDZs7Gj/40KFTmwG a5rRxdLWHp6Q4ZElYTYIA9+UmX/k5ReE6MoesSwpBdqiFhWZyf1Y+1w9MldnRXAXgZ4w 3+HA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id e27-v6si885598qtm.339.2018.10.19.09.04.49 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 19 Oct 2018 09:04:49 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1DAA63078AAF; Fri, 19 Oct 2018 16:04:49 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-124-217.rdu2.redhat.com [10.10.124.217]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7A66765949; Fri, 19 Oct 2018 16:04:48 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= Subject: [PATCH 1/6] mm/hmm: fix utf8 ... Date: Fri, 19 Oct 2018 12:04:37 -0400 Message-Id: <20181019160442.18723-2-jglisse@redhat.com> In-Reply-To: <20181019160442.18723-1-jglisse@redhat.com> References: <20181019160442.18723-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Fri, 19 Oct 2018 16:04:49 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Somehow utf=8 must have been broken. Signed-off-by: Jérôme Glisse Cc: Andrew Morton --- include/linux/hmm.h | 2 +- mm/hmm.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 4c92e3ba3e16..1ff4bae7ada7 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -11,7 +11,7 @@ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * - * Authors: Jérôme Glisse + * Authors: Jérôme Glisse */ /* * Heterogeneous Memory Management (HMM) diff --git a/mm/hmm.c b/mm/hmm.c index c968e49f7a0c..9a068a1da487 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -11,7 +11,7 @@ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * - * Authors: Jérôme Glisse + * Authors: Jérôme Glisse */ /* * Refer to include/linux/hmm.h for information about heterogeneous memory From patchwork Fri Oct 19 16:04:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10649719 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7765A17DE for ; Fri, 19 Oct 2018 16:04:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 636742836F for ; Fri, 19 Oct 2018 16:04:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 567DE283B1; Fri, 19 Oct 2018 16:04:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C6C5F2836F for ; Fri, 19 Oct 2018 16:04:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2CE766B000E; Fri, 19 Oct 2018 12:04:52 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 27B1A6B0010; Fri, 19 Oct 2018 12:04:52 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0CBEA6B0266; Fri, 19 Oct 2018 12:04:52 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by kanga.kvack.org (Postfix) with ESMTP id D00706B000E for ; Fri, 19 Oct 2018 12:04:51 -0400 (EDT) Received: by mail-qt1-f197.google.com with SMTP id t27-v6so37583057qta.1 for ; Fri, 19 Oct 2018 09:04:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=1ZELS7Lj9AUj6AzZeNE2DuRQXEYat7zCY16uj/3ZwWw=; b=N/HFtojmDXdRz6Hh34ywU1XiQ4QC9wQgluwgyDsVZXAdiTvLpN8X5hqHY+lNzdabNQ HmOmzfEti3Z+uZRR37Y3sOWDs2mUmY9P2vf/+Q2aKJM34/GMoYv89DjmpMsuPFgazyyV NlDhGzFyVMKpeI17zgx82xm2laK1DxBHVxTP13PQrdpc1kARb5Nkl6dyQYN6mYM7OMvU pDvdWRKuD2mhSmuAseV0M/oSmO4Ks9Yw5BRlFHoYu7Os1deYJj4P5yjwwQMAaFzqUr3i /nevJHtTJ5XnB5MnmLaSOUONiDLvfehV/5YQqLBDeMWskpT2joq/zfTvhwJOV9QMak/J NWmw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: ABuFfoiExfZNwizy82hXT+TLYzr94JhadbZTFZiQF5UB7JrlUv0rT6/Y Gq89OqY2FmfdJEwiJ6ZKVy2YLtTCYqw4gnUlz8e5KIU/7Gk/JdcKvT0M1gCSRL9mAnmZd2liJ3J hn1r6ZjkOlBix6rrDu3tF7T2SYNeWzEZzHGbpBG/nuypptU5pqUJtxlidPE0VT2d2tA== X-Received: by 2002:ac8:3d91:: with SMTP id v17-v6mr3045245qtf.60.1539965091536; Fri, 19 Oct 2018 09:04:51 -0700 (PDT) X-Google-Smtp-Source: ACcGV63V97puN1tEhCsU3FR2NMmXsu34Wd6/y/9yot3LErAkuvB2hJr6DIA7xyzfp68V23V+s+Q5 X-Received: by 2002:ac8:3d91:: with SMTP id v17-v6mr3045195qtf.60.1539965090985; Fri, 19 Oct 2018 09:04:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539965090; cv=none; d=google.com; s=arc-20160816; b=LvCfdIft9rskTBKXng3CC9BseN+iYV+5N/Pbh4FxyCWvQtyo2nCmKCXXK0h1XuAeW3 qCKvOhQytcJBxZpZv74NiCV39tRCpd6wgc+Pn6bS4KKHdTi8LR061SxxAQsDMGG8q8v4 RZi5sk8feLcMZ+9h6BjkOnjCRAlBGHkF9Zv49IclyVM/oTRQ32iZ8UqqHnOvdUYZa/FD 2bMuVNjQCfbNEFJW7n93nT+FiZArRJz4H0rhTEm45HhXo1TTLzLrUYbxOokFMnhDGGiQ qL3bwLdd55UfPFLK/BL3dVx0mOoUqayn/H24KUKs47RZLPL5E68zynB1jBLjkh9WF4mS M6lg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=1ZELS7Lj9AUj6AzZeNE2DuRQXEYat7zCY16uj/3ZwWw=; b=BMr2RLenzLTW74MhpU3DEqm9zBdpgZQG0wAvFt2u0IDwi6RgLRVUhdzPqNa1ecQghC cgqR09anA+/BUelDSnfWAwSMO3Og3fFSc+fRM5hfOX3dzxoxBCdnw9CaGld8dQpD7nnY W6QcBfM2FjkIHexBPpQh5435xCDnk9HNjVJ0cSCJnO4uVC9WpWsBvLj1KuC4vXdDkQ5E 87VjrgRw1JZChdYeG5KC0mpFQTjtm0rkJrqjZXK7+kULRiTzoADFluiKzNIjVUm97tag BKPhGVmbUYkbRavH1rPkDECK5ggCNsZI60g8FtcQ2fdEk83EkJheSYLXxHD0P7oam7fz pleg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id c7-v6si2047455qtp.300.2018.10.19.09.04.50 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 19 Oct 2018 09:04:50 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3C77B4E4C0; Fri, 19 Oct 2018 16:04:50 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-124-217.rdu2.redhat.com [10.10.124.217]) by smtp.corp.redhat.com (Postfix) with ESMTP id 45BA46594C; Fri, 19 Oct 2018 16:04:49 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, Ralph Campbell , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , "Kirill A . Shutemov" , stable@vger.kernel.org Subject: [PATCH 2/6] mm/rmap: map_pte() was not handling private ZONE_DEVICE page properly v3 Date: Fri, 19 Oct 2018 12:04:38 -0400 Message-Id: <20181019160442.18723-3-jglisse@redhat.com> In-Reply-To: <20181019160442.18723-1-jglisse@redhat.com> References: <20181019160442.18723-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 19 Oct 2018 16:04:50 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Ralph Campbell Private ZONE_DEVICE pages use a special pte entry and thus are not present. Properly handle this case in map_pte(), it is already handled in check_pte(), the map_pte() part was lost in some rebase most probably. Without this patch the slow migration path can not migrate back to any private ZONE_DEVICE memory to regular memory. This was found after stress testing migration back to system memory. This ultimatly can lead to the CPU constantly page fault looping on the special swap entry. Changes since v2: - add comments explaining what is going on Changes since v1: - properly lock pte directory in map_pte() Signed-off-by: Ralph Campbell Signed-off-by: Jérôme Glisse Reviewed-by: Balbir Singh Cc: Andrew Morton Cc: Kirill A. Shutemov Cc: stable@vger.kernel.org --- mm/page_vma_mapped.c | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index ae3c2a35d61b..11df03e71288 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -21,7 +21,29 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw) if (!is_swap_pte(*pvmw->pte)) return false; } else { - if (!pte_present(*pvmw->pte)) + /* + * We get here when we are trying to unmap a private + * device page from the process address space. Such + * page is not CPU accessible and thus is mapped as + * a special swap entry, nonetheless it still does + * count as a valid regular mapping for the page (and + * is accounted as such in page maps count). + * + * So handle this special case as if it was a normal + * page mapping ie lock CPU page table and returns + * true. + * + * For more details on device private memory see HMM + * (include/linux/hmm.h or mm/hmm.c). + */ + if (is_swap_pte(*pvmw->pte)) { + swp_entry_t entry; + + /* Handle un-addressable ZONE_DEVICE memory */ + entry = pte_to_swp_entry(*pvmw->pte); + if (!is_device_private_entry(entry)) + return false; + } else if (!pte_present(*pvmw->pte)) return false; } } From patchwork Fri Oct 19 16:04:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10649721 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2823517DE for ; Fri, 19 Oct 2018 16:05:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 158CB2836F for ; Fri, 19 Oct 2018 16:05:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 06FD7283B1; Fri, 19 Oct 2018 16:05:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF6432836F for ; Fri, 19 Oct 2018 16:05:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 083916B0010; Fri, 19 Oct 2018 12:04:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F26BA6B0269; Fri, 19 Oct 2018 12:04:53 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D78D36B026A; Fri, 19 Oct 2018 12:04:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id AB1596B0010 for ; Fri, 19 Oct 2018 12:04:53 -0400 (EDT) Received: by mail-qk1-f197.google.com with SMTP id m63-v6so36210214qkb.9 for ; Fri, 19 Oct 2018 09:04:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=J8XjcrNFcZSXts3aOYmIP4lmZnEClKb/MHvA1iKHQIs=; b=KXsbrtlEC46HE8ScQrit8ThgjdXH82ACA+xDJyNb3MUgyfr13RjMH8fGAV+E/m0uhb R/8gHNlPE66OVC999NgAnvk6H62cMxB7siM4ln1aTqt2cadURV5V6yKrcL0+ICI6HkxV DPhqaB3ZBpfamEiTVS9P1xa4kzkltjPDKcrihgAYoNOP0xejyNhjwVIURlELheYxB/Ar HSxhNeJ3QprW6niAL2K3QKR/QJIQBN7BPwRjQ1PTrWxuIYxvdnHyipQpFKDqnl6EFg1A ZumY29pBcqKJUVVHkCn6q7iKDGs2rdSJpJPj+v0+TB8fw8ANoxeL4m7SYf0LEvpJiv/p mn/g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: ABuFfog2DGtBxitNjOMd9DQ6BB45YX0r/k091p+wIZYmyFqLIpz9IDuG wzvpvoB5+EHHIYv/EQ6A1YW8VwvUn5hPE6UiNbUaEx5GeV35dsxFlF9dV3zCdDXJo76k2rDxXAQ ZhlJ+gAQpz7S3GrJjbzMMiz9MYjbWHTk/C7IYz6HFjVb2SIgdbNLrm5WRohIzQkJo2g== X-Received: by 2002:a37:634e:: with SMTP id x75-v6mr14897318qkb.122.1539965093384; Fri, 19 Oct 2018 09:04:53 -0700 (PDT) X-Google-Smtp-Source: ACcGV62fLm/Iaz8BZBo7yzTIW5BbC562spSsyCSL9IBE7gkIwQH4qw28+YxyanYB3s8EsGZfcACK X-Received: by 2002:a37:634e:: with SMTP id x75-v6mr14897205qkb.122.1539965091980; Fri, 19 Oct 2018 09:04:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539965091; cv=none; d=google.com; s=arc-20160816; b=uMQidLlNMSXXl02SEVEvi6IP+FLzCGdnKqu9WAuvYCiDdlssubZt8eJdsUmc1DDK0/ rNw+3NNQO9wWgd4A81vGu+K1wSpXdrXPljYMoQ/pWGGx76GRxuZ76JCGcU/9UzMyvawY H6DUTBTx8opyqAQkpKSSQ19XO+UI1XL31uCdYN1I20bHbUdzSEcdw2mOhXmOLSeigBh6 RbHtcinIwf0R2XhxeyBG6c6oqszZSscEhf43+zler77L4tm69Hu7Pn3Ppsk0e8lc8Do0 5J3ivslDGsNv6ktsFm8bBzvz4UauTI2draFBAkDOXTonWZKDI6dMoC9mzZC07JEB7Xxv HDTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=J8XjcrNFcZSXts3aOYmIP4lmZnEClKb/MHvA1iKHQIs=; b=AK4zXjb2yX/c9hY3HTrKpWvDoOgBpz4h7xgkEyJfvvjpmmeiFbH2GmKzy/ni/zxFuT sML6TGxGy73Wv1hxzwybdv571qy/4EG3kgBRTrV8Wd1/Bmrb8y8PDuRx95Erwxj0vet9 OMH8uL06ZsaFL2nugh81kjIhWgAINRVm921RAlKSUD9Rnk6hBI9xDetPGcpSdHh7XuFP JNtpEDbPmlBC5wTxffY9pxo+HuAx0JKuHqqpCC/z8kwr4Iwpr8zeku5YtBdoQgMl/iUm fv/AlPIxqtxXzJtu5aegpwxroHsUi7D9N9QLgZH37aUopBo2XSjEutgI6gZheBqWZ+/q arNQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id e2-v6si1167483qtd.355.2018.10.19.09.04.51 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 19 Oct 2018 09:04:51 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1AF9F32B660; Fri, 19 Oct 2018 16:04:51 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-124-217.rdu2.redhat.com [10.10.124.217]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6220165314; Fri, 19 Oct 2018 16:04:50 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, Ralph Campbell , stable@vger.kernel.org Subject: [PATCH 3/6] mm/hmm: fix race between hmm_mirror_unregister() and mmu_notifier callback Date: Fri, 19 Oct 2018 12:04:39 -0400 Message-Id: <20181019160442.18723-4-jglisse@redhat.com> In-Reply-To: <20181019160442.18723-1-jglisse@redhat.com> References: <20181019160442.18723-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 19 Oct 2018 16:04:51 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Ralph Campbell In hmm_mirror_unregister(), mm->hmm is set to NULL and then mmu_notifier_unregister_no_release() is called. That creates a small window where mmu_notifier can call mmu_notifier_ops with mm->hmm equal to NULL. Fix this by first unregistering mmu notifier callbacks and then setting mm->hmm to NULL. Similarly in hmm_register(), set mm->hmm before registering mmu_notifier callbacks so callback functions always see mm->hmm set. Signed-off-by: Ralph Campbell Reviewed-by: John Hubbard Reviewed-by: Jérôme Glisse Reviewed-by: Balbir Singh Cc: Andrew Morton Cc: stable@vger.kernel.org --- mm/hmm.c | 36 +++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 9a068a1da487..a16678d08127 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -91,16 +91,6 @@ static struct hmm *hmm_register(struct mm_struct *mm) spin_lock_init(&hmm->lock); hmm->mm = mm; - /* - * We should only get here if hold the mmap_sem in write mode ie on - * registration of first mirror through hmm_mirror_register() - */ - hmm->mmu_notifier.ops = &hmm_mmu_notifier_ops; - if (__mmu_notifier_register(&hmm->mmu_notifier, mm)) { - kfree(hmm); - return NULL; - } - spin_lock(&mm->page_table_lock); if (!mm->hmm) mm->hmm = hmm; @@ -108,12 +98,27 @@ static struct hmm *hmm_register(struct mm_struct *mm) cleanup = true; spin_unlock(&mm->page_table_lock); - if (cleanup) { - mmu_notifier_unregister(&hmm->mmu_notifier, mm); - kfree(hmm); - } + if (cleanup) + goto error; + + /* + * We should only get here if hold the mmap_sem in write mode ie on + * registration of first mirror through hmm_mirror_register() + */ + hmm->mmu_notifier.ops = &hmm_mmu_notifier_ops; + if (__mmu_notifier_register(&hmm->mmu_notifier, mm)) + goto error_mm; return mm->hmm; + +error_mm: + spin_lock(&mm->page_table_lock); + if (mm->hmm == hmm) + mm->hmm = NULL; + spin_unlock(&mm->page_table_lock); +error: + kfree(hmm); + return NULL; } void hmm_mm_destroy(struct mm_struct *mm) @@ -278,12 +283,13 @@ void hmm_mirror_unregister(struct hmm_mirror *mirror) if (!should_unregister || mm == NULL) return; + mmu_notifier_unregister_no_release(&hmm->mmu_notifier, mm); + spin_lock(&mm->page_table_lock); if (mm->hmm == hmm) mm->hmm = NULL; spin_unlock(&mm->page_table_lock); - mmu_notifier_unregister_no_release(&hmm->mmu_notifier, mm); kfree(hmm); } EXPORT_SYMBOL(hmm_mirror_unregister); From patchwork Fri Oct 19 16:04:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10649723 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C82CD109C for ; Fri, 19 Oct 2018 16:05:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B6A722836F for ; Fri, 19 Oct 2018 16:05:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AA78D283B1; Fri, 19 Oct 2018 16:05:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 83EE72836F for ; Fri, 19 Oct 2018 16:05:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C6C616B0266; Fri, 19 Oct 2018 12:04:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BF6CB6B0269; Fri, 19 Oct 2018 12:04:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9CC996B026A; Fri, 19 Oct 2018 12:04:54 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id 68CD96B0266 for ; Fri, 19 Oct 2018 12:04:54 -0400 (EDT) Received: by mail-qk1-f198.google.com with SMTP id n188-v6so35846022qke.6 for ; Fri, 19 Oct 2018 09:04:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=QKe3lWW26C2LcCIDif715BUiD8fEnIwUoTeMiLlseiI=; b=sBHJL8cCKL9Di1gtnWMzNPK1be9nawX/ZKfxmquxA/rdfLHSEVdi7jykx2qoG3GLEy zW1PEgoNYF1wAPad+k5+rOalCLvaXO9zyWCMzkL9qrgYAei4c0t9m+33T11+ayRYcFGc Z5Az6wPZ6vMkCg0/tMTZox6rS9mgVSwEhAUlwutYL+NRTsrYXdIGe4eOvWKmVbmH9YS3 i14NyXAsBMX6oKV/SqtgsKT1HehBQ8YIl9eNdsViYpTsRdXxNGIdh7ih+wJvXRr7uC1D z2CSs3lSqiujloRFUkbFfDgd3P0mRafk0brrJ5CBlHow86xgxVGfRM+kFVrxNz6vyvUw yeHA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: ABuFfogp2V6dnD6yzO86r7KPaNhAKC8RcaUYWGmmv2a7fCU9c6RRFDWa Gb+I3QB0Aeo23/mSpnTpgFNA9AjxD1OkFEDg2LXcyw8f+BkeJs6tXrMPAMod55AJv5mOOkjYxai oMPXzrpXwQxjBIJ+b1ay3TIUjTe7E/yf4qT5H10JJng34ZODgrwtx1pGNR7gZS+MAow== X-Received: by 2002:a37:ab12:: with SMTP id u18-v6mr32888839qke.305.1539965094205; Fri, 19 Oct 2018 09:04:54 -0700 (PDT) X-Google-Smtp-Source: ACcGV63s61mpcEkcR3Pf3HdJrPOGtEXpA/Pb2xi1WP5Xzl0/lvNhShfaNFF3lkoSkRDeGsPT0/8t X-Received: by 2002:a37:ab12:: with SMTP id u18-v6mr32888781qke.305.1539965093569; Fri, 19 Oct 2018 09:04:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539965093; cv=none; d=google.com; s=arc-20160816; b=RzWGY09Q+bhVF0CSr3YGF41pkHTGzizbP3az6U3jXWHjenoenakMN60Wv3xsG2K/O7 fo3vAoQUIbtIR1jqv4+wnoKrP/Ndk80Ck85DfVX49tgBJooXANMfUOfgvATNiRTc20qu gP22vYXXZBYgvYr7J1Uh5QyuACcPNtlclLzkQGm8ObAJevItK4Z3Bd8g5DEEo7FjPKzz k3N2SPg5cqGQAfXxvyG5ZTR5PbPIriPqlUbyc/niLuz/ewBVUIkzp4aCtUev7TTIv+zM YHXd2Q/X79vszb87980Vims31Gl235HHoleo25oXQ3zDJHtJYZxah4HbQRKPHVIvHt8O 7/EQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=QKe3lWW26C2LcCIDif715BUiD8fEnIwUoTeMiLlseiI=; b=i+FC7aGI6qbO8vtjelQH/vIqMJ+yaaFYUBp7e/vc+m6s9mHWtoheQt2UFvWQxRUr/u wsLeNtn9CMbHySxLu5V0FtkWzLzQJP6MUwZTlF3CRD6lfmkLM6Pxa32l1tYmpvG3+PLS 03uEBGeT86qIZTIBaX9plBVN6TS6li18NSeslWnv7RNVr8KG2iRZ9iXAuAOsFw3LC5Nb SNlrB5588lTIkZSaIixP7KVok5HqcXTR6vWNDLLCpvtDmBS6D2QIWkMKgwjwWvf9fMpr ZUaicB8EboNqJV12n3DIwzC4toVB5/KzHlQo+xNN92UcFVCmafrngmUzU6KCUGJ+FM/i zBAQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id n52-v6si8648903qtf.91.2018.10.19.09.04.53 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 19 Oct 2018 09:04:53 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8EF4B11533; Fri, 19 Oct 2018 16:04:52 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-124-217.rdu2.redhat.com [10.10.124.217]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43DCB65314; Fri, 19 Oct 2018 16:04:51 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , "Aneesh Kumar K . V" , Zi Yan , Michal Hocko , Ralph Campbell , John Hubbard Subject: [PATCH 4/6] mm/hmm: properly handle migration pmd v3 Date: Fri, 19 Oct 2018 12:04:40 -0400 Message-Id: <20181019160442.18723-5-jglisse@redhat.com> In-Reply-To: <20181019160442.18723-1-jglisse@redhat.com> References: <20181019160442.18723-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Fri, 19 Oct 2018 16:04:52 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Before this patch migration pmd entry (!pmd_present()) would have been treated as a bad entry (pmd_bad() returns true on migration pmd entry). The outcome was that device driver would believe that the range covered by the pmd was bad and would either SIGBUS or simply kill all the device's threads (each device driver decide how to react when the device tries to access poisonnous or invalid range of memory). This patch explicitly handle the case of migration pmd entry which are non present pmd entry and either wait for the migration to finish or report empty range (when device is just trying to pre- fill a range of virtual address and thus do not want to wait or trigger page fault). Changed since v1: - use is_pmd_migration_entry() instead of open coding the equivalent. Changed since v2: - protect is_pmd_migration_entry() with thp_migration_supported() Signed-off-by: Aneesh Kumar K.V Signed-off-by: Jérôme Glisse Cc: Zi Yan Cc: Michal Hocko Cc: Ralph Campbell Cc: John Hubbard Cc: Andrew Morton --- mm/hmm.c | 40 ++++++++++++++++++++++++++++++++++------ 1 file changed, 34 insertions(+), 6 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index a16678d08127..a7aff319bc5a 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -577,22 +577,42 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, { struct hmm_vma_walk *hmm_vma_walk = walk->private; struct hmm_range *range = hmm_vma_walk->range; + struct vm_area_struct *vma = walk->vma; uint64_t *pfns = range->pfns; unsigned long addr = start, i; pte_t *ptep; + pmd_t pmd; - i = (addr - range->start) >> PAGE_SHIFT; again: - if (pmd_none(*pmdp)) + pmd = READ_ONCE(*pmdp); + if (pmd_none(pmd)) return hmm_vma_walk_hole(start, end, walk); - if (pmd_huge(*pmdp) && (range->vma->vm_flags & VM_HUGETLB)) + if (pmd_huge(pmd) && (range->vma->vm_flags & VM_HUGETLB)) return hmm_pfns_bad(start, end, walk); - if (pmd_devmap(*pmdp) || pmd_trans_huge(*pmdp)) { - pmd_t pmd; + if (thp_migration_supported() && is_pmd_migration_entry(pmd)) { + bool fault, write_fault; + unsigned long npages; + uint64_t *pfns; + + i = (addr - range->start) >> PAGE_SHIFT; + npages = (end - addr) >> PAGE_SHIFT; + pfns = &range->pfns[i]; + + hmm_range_need_fault(hmm_vma_walk, pfns, npages, + 0, &fault, &write_fault); + if (fault || write_fault) { + hmm_vma_walk->last = addr; + pmd_migration_entry_wait(vma->vm_mm, pmdp); + return -EAGAIN; + } + return 0; + } else if (!pmd_present(pmd)) + return hmm_pfns_bad(start, end, walk); + if (pmd_devmap(pmd) || pmd_trans_huge(pmd)) { /* * No need to take pmd_lock here, even if some other threads * is splitting the huge pmd we will get that event through @@ -607,13 +627,21 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, if (!pmd_devmap(pmd) && !pmd_trans_huge(pmd)) goto again; + i = (addr - range->start) >> PAGE_SHIFT; return hmm_vma_handle_pmd(walk, addr, end, &pfns[i], pmd); } - if (pmd_bad(*pmdp)) + /* + * We have handled all the valid case above ie either none, migration, + * huge or transparent huge. At this point either it is a valid pmd + * entry pointing to pte directory or it is a bad pmd that will not + * recover. + */ + if (pmd_bad(pmd)) return hmm_pfns_bad(start, end, walk); ptep = pte_offset_map(pmdp, addr); + i = (addr - range->start) >> PAGE_SHIFT; for (; addr < end; addr += PAGE_SIZE, ptep++, i++) { int r; From patchwork Fri Oct 19 16:04:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10649725 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4449B17DE for ; Fri, 19 Oct 2018 16:05:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2DB1128329 for ; Fri, 19 Oct 2018 16:05:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1F1D3283C7; Fri, 19 Oct 2018 16:05:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4548F2836F for ; Fri, 19 Oct 2018 16:05:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0D07A6B026A; Fri, 19 Oct 2018 12:04:56 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 057516B026B; Fri, 19 Oct 2018 12:04:55 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E38C86B026C; Fri, 19 Oct 2018 12:04:55 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id A5D516B026A for ; Fri, 19 Oct 2018 12:04:55 -0400 (EDT) Received: by mail-qk1-f197.google.com with SMTP id s123-v6so35680361qkf.12 for ; Fri, 19 Oct 2018 09:04:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=KAVNlPILEZQ/0UbQ7sW5MDMCZKMqR8r4JD5zTgYimsQ=; b=fZkcq33GjA98nx/pCHbAKVbaHCuyjjnEionjTDh4BydO+0KTfhAfG8py4wkL4JdYB3 gntSN7BVY4rymdwWaUpPyMRQ7Eb1A+9YdfLaciky8f6yHs1Fwe5/Pf4+lQEaoVHc6cXE AfvsvHO4k1msui4FKEPkHvrD39BWuX5gtr6GjPdPUgpRKu6PLIUqPKYlrd2+W3nUnVB5 PnbT45AVNplIZ77Bd1kiov+KgZvI7PN+SBUeFWQNIFJ34u/TDjTrAubyq6SleXHP6TAt rjWUhWCQN7N4CE3V+6q8AD8D4X+BkxPvO3/wgT6yzj4BF9OaFJZCf2ujOfnZEmGXkPaD kX/Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: ABuFfoivuBaAvDTJSFB4Q49PSyA7CkkC8fcrOM2j8RMylh/Xlan5W1tH VbzWcUueO8RWoATy1Yg9jTu5c7dlK9LBMSmb7RW9z3vxsXzZkyk4tRdAZpYMPsoid59Y6IXeQOY AFXeFD5P1C+IVkxhhmNGA85zTEd7BWiWgQAXVVtDuznwwXkVsPsgA9ro7C+PNFjgv9w== X-Received: by 2002:a37:37d5:: with SMTP id e204-v6mr32957460qka.1.1539965095424; Fri, 19 Oct 2018 09:04:55 -0700 (PDT) X-Google-Smtp-Source: ACcGV60z4/1xtZjKHM1gs2NLA+/bwD9IxAhqyHpUniZXKmUzojSSFNNGgFwPWEl5ZMVhrkoHVVC/ X-Received: by 2002:a37:37d5:: with SMTP id e204-v6mr32957372qka.1.1539965094380; Fri, 19 Oct 2018 09:04:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539965094; cv=none; d=google.com; s=arc-20160816; b=WgPjIr4JDBrIH8iMO/E72FZcTXHIhozh5JOmve4H6wYGA7cdAphxUgk91eZkrav5gG GRttVuPMb/wVvu8YTNKMNuHnWnjVoEInAmBF6a/7BkFvExjC15+bgqMA2ehEHolzyPe+ IhG5IT+ZAneKFMBLAk8uTycB+Fv4QhN1C4+ZhY3ypP+UDLp7aGDm+kv7hLpONDXkmx/u pWC5wSqj6vsHgwXWGhlGwDnK3yfVlD+RK2wvcCkvXRDJzrNLpXIGiuVdwMd4CcFKrj0u nAqusrI6zFS2iwPV4oekTfT4R9Wx2D3QnUkdAlXRCQvqND+PSvW459mz7IclINDVWo+e vNFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=KAVNlPILEZQ/0UbQ7sW5MDMCZKMqR8r4JD5zTgYimsQ=; b=xRIF5EWs5Ri932GVYIfL+DQILe1/l3XNU1OLSCNnDXEmj0YTaJudI+jenb3iA/e0+j nCBtjJTVW/E+kktsYirDOhg7DHCilJL5JvLA2/LPPhwO/HZrPUwQpTrkt53VPSEA4Ghi fYyzVa4FPiaVd9wmuGIuL8iywUnZEE6k+yHIqb6elc7gE4xq2gecFf1VZtapoA5JUGYQ AbzdYU+ZH+OvenUW8SdFFRyBs3a0nnphb6TfSd5fS3zlfAPtD+LBJFzISmvmyPr75GuL hxTSnNOjZZ7JsZvmvheKRL2N1rb7lO3SCS6G04rC/Gfk0gh7/vsAdjkCLoVlImsCX2Q5 /Bnw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id w64-v6si840612qkb.193.2018.10.19.09.04.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 19 Oct 2018 09:04:54 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8D10E30014A1; Fri, 19 Oct 2018 16:04:53 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-124-217.rdu2.redhat.com [10.10.124.217]) by smtp.corp.redhat.com (Postfix) with ESMTP id B6D2465949; Fri, 19 Oct 2018 16:04:52 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Ralph Campbell , John Hubbard Subject: [PATCH 5/6] mm/hmm: use a structure for update callback parameters v2 Date: Fri, 19 Oct 2018 12:04:41 -0400 Message-Id: <20181019160442.18723-6-jglisse@redhat.com> In-Reply-To: <20181019160442.18723-1-jglisse@redhat.com> References: <20181019160442.18723-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Fri, 19 Oct 2018 16:04:53 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Use a structure to gather all the parameters for the update callback. This make it easier when adding new parameters by avoiding having to update all callback function signature. The hmm_update structure is always associated with a mmu_notifier callbacks so we are not planing on grouping multiple updates together. Nor do we care about page size for the range as range will over fully cover the page being invalidated (this is a mmu_notifier property). Changed since v1: - support for blockable mmu_notifier flags - improved commit log Signed-off-by: Jérôme Glisse Cc: Ralph Campbell Cc: John Hubbard Cc: Andrew Morton --- include/linux/hmm.h | 31 ++++++++++++++++++++++--------- mm/hmm.c | 33 ++++++++++++++++++++++----------- 2 files changed, 44 insertions(+), 20 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 1ff4bae7ada7..afc04dbbaf2f 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -274,13 +274,28 @@ static inline uint64_t hmm_pfn_from_pfn(const struct hmm_range *range, struct hmm_mirror; /* - * enum hmm_update_type - type of update + * enum hmm_update_event - type of update * @HMM_UPDATE_INVALIDATE: invalidate range (no indication as to why) */ -enum hmm_update_type { +enum hmm_update_event { HMM_UPDATE_INVALIDATE, }; +/* + * struct hmm_update - HMM update informations for callback + * + * @start: virtual start address of the range to update + * @end: virtual end address of the range to update + * @event: event triggering the update (what is happening) + * @blockable: can the callback block/sleep ? + */ +struct hmm_update { + unsigned long start; + unsigned long end; + enum hmm_update_event event; + bool blockable; +}; + /* * struct hmm_mirror_ops - HMM mirror device operations callback * @@ -300,9 +315,9 @@ struct hmm_mirror_ops { /* sync_cpu_device_pagetables() - synchronize page tables * * @mirror: pointer to struct hmm_mirror - * @update_type: type of update that occurred to the CPU page table - * @start: virtual start address of the range to update - * @end: virtual end address of the range to update + * @update: update informations (see struct hmm_update) + * Returns: -EAGAIN if update.blockable false and callback need to + * block, 0 otherwise. * * This callback ultimately originates from mmu_notifiers when the CPU * page table is updated. The device driver must update its page table @@ -313,10 +328,8 @@ struct hmm_mirror_ops { * page tables are completely updated (TLBs flushed, etc); this is a * synchronous call. */ - void (*sync_cpu_device_pagetables)(struct hmm_mirror *mirror, - enum hmm_update_type update_type, - unsigned long start, - unsigned long end); + int (*sync_cpu_device_pagetables)(struct hmm_mirror *mirror, + const struct hmm_update *update); }; /* diff --git a/mm/hmm.c b/mm/hmm.c index a7aff319bc5a..0eacf9627bc9 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -126,10 +126,8 @@ void hmm_mm_destroy(struct mm_struct *mm) kfree(mm->hmm); } -static void hmm_invalidate_range(struct hmm *hmm, - enum hmm_update_type action, - unsigned long start, - unsigned long end) +static int hmm_invalidate_range(struct hmm *hmm, + const struct hmm_update *update) { struct hmm_mirror *mirror; struct hmm_range *range; @@ -138,22 +136,30 @@ static void hmm_invalidate_range(struct hmm *hmm, list_for_each_entry(range, &hmm->ranges, list) { unsigned long addr, idx, npages; - if (end < range->start || start >= range->end) + if (update->end < range->start || update->start >= range->end) continue; range->valid = false; - addr = max(start, range->start); + addr = max(update->start, range->start); idx = (addr - range->start) >> PAGE_SHIFT; - npages = (min(range->end, end) - addr) >> PAGE_SHIFT; + npages = (min(range->end, update->end) - addr) >> PAGE_SHIFT; memset(&range->pfns[idx], 0, sizeof(*range->pfns) * npages); } spin_unlock(&hmm->lock); down_read(&hmm->mirrors_sem); - list_for_each_entry(mirror, &hmm->mirrors, list) - mirror->ops->sync_cpu_device_pagetables(mirror, action, - start, end); + list_for_each_entry(mirror, &hmm->mirrors, list) { + int ret; + + ret = mirror->ops->sync_cpu_device_pagetables(mirror, update); + if (!update->blockable && ret == -EAGAIN) { + up_read(&hmm->mirrors_sem); + return -EAGAIN; + } + } up_read(&hmm->mirrors_sem); + + return 0; } static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) @@ -202,11 +208,16 @@ static void hmm_invalidate_range_end(struct mmu_notifier *mn, unsigned long start, unsigned long end) { + struct hmm_update update; struct hmm *hmm = mm->hmm; VM_BUG_ON(!hmm); - hmm_invalidate_range(mm->hmm, HMM_UPDATE_INVALIDATE, start, end); + update.start = start; + update.end = end; + update.event = HMM_UPDATE_INVALIDATE; + update.blockable = true; + hmm_invalidate_range(hmm, &update); } static const struct mmu_notifier_ops hmm_mmu_notifier_ops = { From patchwork Fri Oct 19 16:04:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10649727 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8E13617DE for ; Fri, 19 Oct 2018 16:05:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7B439283FE for ; Fri, 19 Oct 2018 16:05:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6F76F283C7; Fri, 19 Oct 2018 16:05:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC58E28329 for ; Fri, 19 Oct 2018 16:05:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 69DEE6B026B; Fri, 19 Oct 2018 12:04:56 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6268F6B026C; Fri, 19 Oct 2018 12:04:56 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A3C96B026D; Fri, 19 Oct 2018 12:04:56 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by kanga.kvack.org (Postfix) with ESMTP id 16C856B026C for ; Fri, 19 Oct 2018 12:04:56 -0400 (EDT) Received: by mail-qt1-f197.google.com with SMTP id x7-v6so37121754qtb.6 for ; Fri, 19 Oct 2018 09:04:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=i6KkAVyDd2RTg5dfzpklfysckMhBaeY8PLn3QDkDkDA=; b=pTWk2XtJfwa7do3b8YQRmMfiPN9Fy6cgaK7smk5lCFz5pdFDcD69mul6ZQ1CASHawq 7viJZ8F5shvdpOeKP8tY8lu41RONWHofHgdapYQOt6+ZB2vOZxqArl2tnDpy84qBSS4v XfXmbM/qdPHnWDmtGbOf90zyWCq9EiMtD7j20uB2bR8wnoKNc5rMW8zO+isq6q+QxlPD Tvsnmovi41IRrQKbJOCQgG8WveRaJVvWzHmBLEtObQV2mofWMwOuJhSWstLvgZOd0m2b Pi8SL0moH9C77+931bB479QtRWsF82A+aY1OpPqgFp8zHnah5j9/t7c2ONS7hqvQFJ/o 9OSQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: ABuFfohNIsdsfLJAGoCfV4XUcfnREqhZNDjJEtFrT0EPbLJmvofXe7F7 EXbfosLx4pycbzuM9doRGEPGCP6869ygk9KoMl8ec7kj6e2MIRHaxonUOfl2OD5yv2/eSrbdn4e mB5e1LIiBFJPG62+qTGmtM/HMhI/lfbDHRRk9/KtxYc9f4K2DqZqnvdZCI86K5UAzpA== X-Received: by 2002:a37:5ac5:: with SMTP id o188-v6mr3768860qkb.101.1539965095823; Fri, 19 Oct 2018 09:04:55 -0700 (PDT) X-Google-Smtp-Source: ACcGV635Tu9H4JKtkrKhLnYTigkHhbX0iCWVlwGce/uQ6hICxVrUvqYdwOCeLEW3k/sEueXvO/w8 X-Received: by 2002:a37:5ac5:: with SMTP id o188-v6mr3768815qkb.101.1539965095324; Fri, 19 Oct 2018 09:04:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539965095; cv=none; d=google.com; s=arc-20160816; b=0D6GZFbUnom5tXQ4/e4JutCp3Bqd2w3pbX0xGiMnZ6HP4ysVxYZ/j0GXjz+e8flPZJ jouyNQjDaxKHJD+vuTFc4hrByC/zJ1jPXLwHdhFTE8bzORgTcl70WBrmM/pkQfbM/B+Z XyFzjQwlLd0/FqasyPRAJYEACjXB3Al72rCLxrlmgKer2pc7uKL9n+s58fhCXDy5b++9 BQJJknkRixeCBEtut1Y5aR8ymkhe09rsADbaGCMXA7sQ1mGu1cBySde+7zvKmhC3BpCe TaR8tx6QrwJN6xMpQOMV60fZJqyMMVmnJ1gexW3VGUjMOVwSpBmBGm0VQpw7aX2+l2eu b3Gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=i6KkAVyDd2RTg5dfzpklfysckMhBaeY8PLn3QDkDkDA=; b=cDu1Ze7Qs4IAw4Hmn8I28ykDdAs7OO1QJNkhKWoe3Ke8bwx2oZTrQqWPsO8i2hwQz3 V9QKVXoEfmKqLsnvCbkHGibGymc76xOqCl2KTrSFSUrts75GQKakHlonpq9rvOdFvnPm EHedPA+ITy9okOWmS8Yx49dwTlWIEgZgWEoc657u677prtPjYvSvDBufe/HTOKPW1lK0 W3SbVhVWn3IFtOvztxACl/ihVjQgMMeQQq7ITswZKgiNEVCW5+Du6h47NIhbDFV2a+98 hlCuG2SsyLLbdFYLx+r07J8+ItYKjWjI9XaOysUhB5+BOqdyt6j1t8MjfFl5DXhz764k xCAw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id f4-v6si12636280qkg.151.2018.10.19.09.04.55 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 19 Oct 2018 09:04:55 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 82A9A30020AB; Fri, 19 Oct 2018 16:04:54 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-124-217.rdu2.redhat.com [10.10.124.217]) by smtp.corp.redhat.com (Postfix) with ESMTP id B073D65949; Fri, 19 Oct 2018 16:04:53 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Ralph Campbell , John Hubbard Subject: [PATCH 6/6] mm/hmm: invalidate device page table at start of invalidation Date: Fri, 19 Oct 2018 12:04:42 -0400 Message-Id: <20181019160442.18723-7-jglisse@redhat.com> In-Reply-To: <20181019160442.18723-1-jglisse@redhat.com> References: <20181019160442.18723-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Fri, 19 Oct 2018 16:04:54 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Invalidate device page table at start of invalidation and invalidate in progress CPU page table snapshooting at both start and end of any invalidation. This is helpful when device need to dirty page because the device page table report the page as dirty. Dirtying page must happen in the start mmu notifier callback and not in the end one. Signed-off-by: Jérôme Glisse Cc: Ralph Campbell Cc: John Hubbard Cc: Andrew Morton --- mm/hmm.c | 27 +++++++++++++++------------ 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 0eacf9627bc9..1aecf7c08cff 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -43,7 +43,6 @@ static const struct mmu_notifier_ops hmm_mmu_notifier_ops; * * @mm: mm struct this HMM struct is bound to * @lock: lock protecting ranges list - * @sequence: we track updates to the CPU page table with a sequence number * @ranges: list of range being snapshotted * @mirrors: list of mirrors for this mm * @mmu_notifier: mmu notifier to track updates to CPU page table @@ -52,7 +51,6 @@ static const struct mmu_notifier_ops hmm_mmu_notifier_ops; struct hmm { struct mm_struct *mm; spinlock_t lock; - atomic_t sequence; struct list_head ranges; struct list_head mirrors; struct mmu_notifier mmu_notifier; @@ -85,7 +83,6 @@ static struct hmm *hmm_register(struct mm_struct *mm) return NULL; INIT_LIST_HEAD(&hmm->mirrors); init_rwsem(&hmm->mirrors_sem); - atomic_set(&hmm->sequence, 0); hmm->mmu_notifier.ops = NULL; INIT_LIST_HEAD(&hmm->ranges); spin_lock_init(&hmm->lock); @@ -126,7 +123,7 @@ void hmm_mm_destroy(struct mm_struct *mm) kfree(mm->hmm); } -static int hmm_invalidate_range(struct hmm *hmm, +static int hmm_invalidate_range(struct hmm *hmm, bool device, const struct hmm_update *update) { struct hmm_mirror *mirror; @@ -147,6 +144,9 @@ static int hmm_invalidate_range(struct hmm *hmm, } spin_unlock(&hmm->lock); + if (!device) + return 0; + down_read(&hmm->mirrors_sem); list_for_each_entry(mirror, &hmm->mirrors, list) { int ret; @@ -189,18 +189,21 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) } static int hmm_invalidate_range_start(struct mmu_notifier *mn, - struct mm_struct *mm, - unsigned long start, - unsigned long end, - bool blockable) + struct mm_struct *mm, + unsigned long start, + unsigned long end, + bool blockable) { + struct hmm_update update; struct hmm *hmm = mm->hmm; VM_BUG_ON(!hmm); - atomic_inc(&hmm->sequence); - - return 0; + update.start = start; + update.end = end; + update.event = HMM_UPDATE_INVALIDATE; + update.blockable = blockable; + return hmm_invalidate_range(hmm, true, &update); } static void hmm_invalidate_range_end(struct mmu_notifier *mn, @@ -217,7 +220,7 @@ static void hmm_invalidate_range_end(struct mmu_notifier *mn, update.end = end; update.event = HMM_UPDATE_INVALIDATE; update.blockable = true; - hmm_invalidate_range(hmm, &update); + hmm_invalidate_range(hmm, false, &update); } static const struct mmu_notifier_ops hmm_mmu_notifier_ops = {