From patchwork Wed Aug 29 17:17:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10580691 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 716715A4 for ; Wed, 29 Aug 2018 17:18:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 40F262ACF5 for ; Wed, 29 Aug 2018 17:18:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3419A2ADBF; Wed, 29 Aug 2018 17:18:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7CEE72ACF5 for ; Wed, 29 Aug 2018 17:18:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 886DF6B4CD0; Wed, 29 Aug 2018 13:18:01 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8355B6B4CD1; Wed, 29 Aug 2018 13:18:01 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 723756B4CD2; Wed, 29 Aug 2018 13:18:01 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f197.google.com (mail-qt0-f197.google.com [209.85.216.197]) by kanga.kvack.org (Postfix) with ESMTP id 496006B4CD0 for ; Wed, 29 Aug 2018 13:18:01 -0400 (EDT) Received: by mail-qt0-f197.google.com with SMTP id b5-v6so5252150qtk.4 for ; Wed, 29 Aug 2018 10:18:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:mime-version:in-reply-to :content-transfer-encoding; bh=BPWHH1t+Z1v1WovqYCeSSSvESua0KHf30y3KnaHUhWc=; b=XIXNHEgeMCKUwVdyJKXJjhw3K/EXtEyZGz8oK3FloDe3M1iPZcOb2qJzq8kkYOWxAN gHsRPvpoWC1EmYdcvGgPAosG7o6dbOwakEbzgi0vYMgCmmGNKQ1qgcAhNTMXz/lXSqa6 iYADk2AtZxTKAYnpCp7tDDNkGkqsoVGyeHPEBNubkbvNNsWgdhO58Es/zBoSu0+ut0DW b9Yd5tlgPjIs6Q/j0xLFZrWnXcWt5nIFXTRogAnteB+97cW7Lh9AxlsLFTO3oqusdzPI SQOrpVUxzbvthi1mrfVmuX5yDbYssHkokHrZizXT+ZP1jY7GHqb92P90wvFQCRnKdcyW 98IQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51CzZ7SlXvkfAw+r0KC/fMYxLcuUWqTFKR1jKEl/hzZ9HQ0+dZ88 tt5k0GYx9TQJ82xb39FBH0LpxmbXU21bdbb6J/xYZO179t2ZqMX6yDzs1kZJ0OZ6rLYtw8cXFUt HiyqwmfyHvQa90X1JweNWGuYnMjGNSbtC48+JngxtGllIuLKugymTorcohpRWgtCyqA== X-Received: by 2002:a37:48e:: with SMTP id 136-v6mr7468817qke.26.1535563081053; Wed, 29 Aug 2018 10:18:01 -0700 (PDT) X-Google-Smtp-Source: ANB0VdaTW2rOpL2vLg6MKE0nzhRCnWdYp6yS+ejSaCUeFaOtsZTKRY20W0vMHUn0CcR+nYiLWFWc X-Received: by 2002:a37:48e:: with SMTP id 136-v6mr7468768qke.26.1535563080378; Wed, 29 Aug 2018 10:18:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535563080; cv=none; d=google.com; s=arc-20160816; b=SK7blbE61tlSmn8g1G0UUt5eudUNNIO47LcRNsbQ5mwkgprCTOZh7et4LJ9+4SEuLg SKi0piikLikIgWqXXJtx+CrtklH2xwNd5d+LKYoiLgTpwLCBLOcF2gp/eW2ltRP3Bd7V caYdj/MfCdgtpT7atrHcekMykpHe39z5pi626DXh2RLVQc09te1M+NzBLRCL9Ft4o9YD VQiNeF29Qvh4M2wluEjeXVGergRCMufPAAjFUkisvqwykItMQzAQ+Y+ura3bChG9YK0X 5hg4xTndQqoYx9BfmG86PTahJ4dUDTu7SNpWzlU6AXyHoDgD0A/uwXRFs3BrRc2YQM5l hjQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:in-reply-to:mime-version:message-id:date :subject:cc:to:from:arc-authentication-results; bh=BPWHH1t+Z1v1WovqYCeSSSvESua0KHf30y3KnaHUhWc=; b=xFm78jAOyvxokFLRW/l7WNh2o8YkBrqyJKwhaWO0KMQjigLR9p6rpWyPaBs3UyBmf5 YOV6/XQ4P9c3DD6CXS+iqdbXL0bp0q3c5pupSrmB7U0DRGVXzmbtju4HkyanRheBgfIu ybAUnCy2CLT9rMTrchpa+N68oaGqI6thZ0sSiY8fACOOJRXhbEN2Pzsgse3s2aeqirIq +kxshYc53sJ7cfCO/bsdLqGAURora5Gra/QzanHVdHWblSlMpX3EFtOaVpvCf0/XJ8H8 M/A75EtAhkfeA9TVFe3NL7rif2BDdPNtNC1sLkXBBQaMiR3dhKc7LR9hLCnIVWMaOFus 2MXQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id k6-v6si1369104qke.96.2018.08.29.10.18.00 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 10:18:00 -0700 (PDT) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AB64E738F1; Wed, 29 Aug 2018 17:17:59 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-126-69.rdu2.redhat.com [10.10.126.69]) by smtp.corp.redhat.com (Postfix) with ESMTP id B7776B278D; Wed, 29 Aug 2018 17:17:58 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , "Aneesh Kumar K . V" , Zi Yan , Michal Hocko , Ralph Campbell , John Hubbard Subject: [PATCH 4/7] mm/hmm: properly handle migration pmd v2 Date: Wed, 29 Aug 2018 13:17:49 -0400 Message-Id: <20180829171749.9365-1-jglisse@redhat.com> MIME-Version: 1.0 In-Reply-To: <20180824192549.30844-5-jglisse@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 29 Aug 2018 17:17:59 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 29 Aug 2018 17:17:59 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'jglisse@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Before this patch migration pmd entry (!pmd_present()) would have been treated as a bad entry (pmd_bad() returns true on migration pmd entry). The outcome was that device driver would believe that the range covered by the pmd was bad and would either SIGBUS or simply kill all the device's threads (each device driver decide how to react when the device tries to access poisonnous or invalid range of memory). This patch explicitly handle the case of migration pmd entry which are non present pmd entry and either wait for the migration to finish or report empty range (when device is just trying to pre- fill a range of virtual address and thus do not want to wait or trigger page fault). Changed since v1: - use is_pmd_migration_entry() instead of open coding the equivalent. Signed-off-by: Aneesh Kumar K.V Signed-off-by: Jérôme Glisse Cc: Zi Yan Cc: Michal Hocko Cc: Ralph Campbell Cc: John Hubbard Cc: Andrew Morton --- mm/hmm.c | 42 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 36 insertions(+), 6 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index a16678d08127..fd3d19d98070 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -577,22 +577,44 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, { struct hmm_vma_walk *hmm_vma_walk = walk->private; struct hmm_range *range = hmm_vma_walk->range; + struct vm_area_struct *vma = walk->vma; uint64_t *pfns = range->pfns; unsigned long addr = start, i; pte_t *ptep; + pmd_t pmd; - i = (addr - range->start) >> PAGE_SHIFT; again: - if (pmd_none(*pmdp)) + pmd = READ_ONCE(*pmdp); + if (pmd_none(pmd)) return hmm_vma_walk_hole(start, end, walk); - if (pmd_huge(*pmdp) && (range->vma->vm_flags & VM_HUGETLB)) + if (pmd_huge(pmd) && (range->vma->vm_flags & VM_HUGETLB)) return hmm_pfns_bad(start, end, walk); - if (pmd_devmap(*pmdp) || pmd_trans_huge(*pmdp)) { - pmd_t pmd; + if (is_pmd_migration_entry(pmd)) { + swp_entry_t entry = pmd_to_swp_entry(pmd); + + bool fault, write_fault; + unsigned long npages; + uint64_t *pfns; + + i = (addr - range->start) >> PAGE_SHIFT; + npages = (end - addr) >> PAGE_SHIFT; + pfns = &range->pfns[i]; + hmm_range_need_fault(hmm_vma_walk, pfns, npages, + 0, &fault, &write_fault); + if (fault || write_fault) { + hmm_vma_walk->last = addr; + pmd_migration_entry_wait(vma->vm_mm, pmdp); + return -EAGAIN; + } + return 0; + } else if (!pmd_present(pmd)) + return hmm_pfns_bad(start, end, walk); + + if (pmd_devmap(pmd) || pmd_trans_huge(pmd)) { /* * No need to take pmd_lock here, even if some other threads * is splitting the huge pmd we will get that event through @@ -607,13 +629,21 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, if (!pmd_devmap(pmd) && !pmd_trans_huge(pmd)) goto again; + i = (addr - range->start) >> PAGE_SHIFT; return hmm_vma_handle_pmd(walk, addr, end, &pfns[i], pmd); } - if (pmd_bad(*pmdp)) + /* + * We have handled all the valid case above ie either none, migration, + * huge or transparent huge. At this point either it is a valid pmd + * entry pointing to pte directory or it is a bad pmd that will not + * recover. + */ + if (pmd_bad(pmd)) return hmm_pfns_bad(start, end, walk); ptep = pte_offset_map(pmdp, addr); + i = (addr - range->start) >> PAGE_SHIFT; for (; addr < end; addr += PAGE_SIZE, ptep++, i++) { int r;