From patchwork Tue Mar 24 01:14:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11454369 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 31350139A for ; Tue, 24 Mar 2020 01:15:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E6F932070A for ; Tue, 24 Mar 2020 01:15:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="TDaQ5NVT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E6F932070A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4E3C86B006E; Mon, 23 Mar 2020 21:15:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4B9EF6B0071; Mon, 23 Mar 2020 21:15:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A7D36B0073; Mon, 23 Mar 2020 21:15:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0170.hostedemail.com [216.40.44.170]) by kanga.kvack.org (Postfix) with ESMTP id 233ED6B006E for ; Mon, 23 Mar 2020 21:15:13 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id E9D75B2295 for ; Tue, 24 Mar 2020 01:15:12 +0000 (UTC) X-FDA: 76628487264.03.mice69_4a95116eb3f4f X-Spam-Summary: 2,0,0,d33cef92303f0c75,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1543:1711:1730:1747:1777:1792:2198:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3874:4117:4321:4605:5007:6119:6120:6261:6653:7576:7901:9036:9592:10004:11026:11658:11914:12043:12296:12297:12438:12517:12519:12555:12679:12683:12895:12986:13161:13229:13894:14181:14394:14721:14819:21080:21444:21451:21627:21990:30003:30054:30064:30070,0,RBL:209.85.160.193:@ziepe.ca:.lbl8.mailshell.net-66.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:28,LUA_SUMMARY:none X-HE-Tag: mice69_4a95116eb3f4f X-Filterd-Recvd-Size: 6926 Received: from mail-qt1-f193.google.com (mail-qt1-f193.google.com [209.85.160.193]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Tue, 24 Mar 2020 01:15:12 +0000 (UTC) Received: by mail-qt1-f193.google.com with SMTP id a5so6492164qtw.10 for ; Mon, 23 Mar 2020 18:15:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OW36Zq5O+nO4G7kBf9jtFzChA1mg5xP0SMOPJaW+Ny0=; b=TDaQ5NVTx16AjeAUiHW1dHB0M8wMkBgVSdm/NgVz28FL1JCJ2xpJScL7jgPwIOvPhb wIrSXyL/H/vSncDr5LSFkwUI3wPQuNi0dgonKvZTfr70EsJHxqQESjt6E5tq0ZGmSvTn x3mkjDvTpeJM3+/aBTbpmibwghhIIEiRvrfVGk+ftxskuavyEMOtwKi01NJzQzRMvFGV AxmWhcKB2gLyomOBTG3dof8s1USnL6cqGHocR/ZRFN2Ugw662kNLm0ujyQybygj5IOwg 7LYql1qmZK1FVNbv2/1sQVPuB+YnquB31v016ocidkS0z4AKNEbX4QhKde34pDYXCVC1 /Rfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OW36Zq5O+nO4G7kBf9jtFzChA1mg5xP0SMOPJaW+Ny0=; b=kHzfaUJ0+9jiMYBCYqTd7i5HCBCgJw0PV+lYokhvzI/JW2bz6JHkZEuB+EkEvpwCXF ZRvTfFx9cPdIn5GaTKp1VbbdRJ6dmWXzJ4ZqJF8RrYou3MFeUCQcYCXcJzcb3YXWlqOC sZYP24M3W+mh3/vcxdW3NJzyoNT8RtEGU7ShtdfEJZ2pmQ19jQJbO1s6nzQBWlLFlpRa Sv84jxxSrtjsAGK+YOr6UiLTSU1asQGNQmgcEAHZqEi/w9cK3ydyNOwz8sk6+nAySS0v NddyxdnNIoANHNEw5U3PjZJH2r/c1emIHgX1vgzgUX8josrcZmrN8U6nFXMI4RAZt/q3 /17A== X-Gm-Message-State: ANhLgQ1mqMJnKUHxqzRa7wm0tzGJvqWuu53kxYl/zsQD2F7VHKaSLl4u rBSPmABgkv40ownL+/qF3GYo7w== X-Google-Smtp-Source: ADFU+vurHQiZGzEk93HjPq+6N40heuGQe7NvOLr4e4QabkbD2CpMkG/mXFOgr7duxLwEVOjRMxTPeA== X-Received: by 2002:ac8:6c4e:: with SMTP id z14mr19233545qtu.138.1585012511661; Mon, 23 Mar 2020 18:15:11 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id j85sm12549468qke.20.2020.03.23.18.15.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Mar 2020 18:15:10 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jGY9u-0000qo-2Y; Mon, 23 Mar 2020 22:15:10 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 1/9] mm/hmm: remove pgmap checking for devmap pages Date: Mon, 23 Mar 2020 22:14:49 -0300 Message-Id: <20200324011457.2817-2-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200324011457.2817-1-jgg@ziepe.ca> References: <20200324011457.2817-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe The checking boils down to some racy check if the pagemap is still available or not. Instead of checking this, rely entirely on the notifiers, if a pagemap is destroyed then all pages that belong to it must be removed from the tables and the notifiers triggered. Reviewed-by: Ralph Campbell Reviewed-by: Christoph Hellwig Signed-off-by: Jason Gunthorpe --- mm/hmm.c | 50 ++------------------------------------------------ 1 file changed, 2 insertions(+), 48 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index a491d9aaafe45d..3a2610e0713329 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -28,7 +28,6 @@ struct hmm_vma_walk { struct hmm_range *range; - struct dev_pagemap *pgmap; unsigned long last; unsigned int flags; }; @@ -196,19 +195,8 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, return hmm_vma_fault(addr, end, fault, write_fault, walk); pfn = pmd_pfn(pmd) + ((addr & ~PMD_MASK) >> PAGE_SHIFT); - for (i = 0; addr < end; addr += PAGE_SIZE, i++, pfn++) { - if (pmd_devmap(pmd)) { - hmm_vma_walk->pgmap = get_dev_pagemap(pfn, - hmm_vma_walk->pgmap); - if (unlikely(!hmm_vma_walk->pgmap)) - return -EBUSY; - } + for (i = 0; addr < end; addr += PAGE_SIZE, i++, pfn++) pfns[i] = hmm_device_entry_from_pfn(range, pfn) | cpu_flags; - } - if (hmm_vma_walk->pgmap) { - put_dev_pagemap(hmm_vma_walk->pgmap); - hmm_vma_walk->pgmap = NULL; - } hmm_vma_walk->last = end; return 0; } @@ -300,15 +288,6 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, if (fault || write_fault) goto fault; - if (pte_devmap(pte)) { - hmm_vma_walk->pgmap = get_dev_pagemap(pte_pfn(pte), - hmm_vma_walk->pgmap); - if (unlikely(!hmm_vma_walk->pgmap)) { - pte_unmap(ptep); - return -EBUSY; - } - } - /* * Since each architecture defines a struct page for the zero page, just * fall through and treat it like a normal page. @@ -328,10 +307,6 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, return 0; fault: - if (hmm_vma_walk->pgmap) { - put_dev_pagemap(hmm_vma_walk->pgmap); - hmm_vma_walk->pgmap = NULL; - } pte_unmap(ptep); /* Fault any virtual address we were asked to fault */ return hmm_vma_fault(addr, end, fault, write_fault, walk); @@ -418,16 +393,6 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, return r; } } - if (hmm_vma_walk->pgmap) { - /* - * We do put_dev_pagemap() here and not in hmm_vma_handle_pte() - * so that we can leverage get_dev_pagemap() optimization which - * will not re-take a reference on a pgmap if we already have - * one. - */ - put_dev_pagemap(hmm_vma_walk->pgmap); - hmm_vma_walk->pgmap = NULL; - } pte_unmap(ptep - 1); hmm_vma_walk->last = addr; @@ -491,20 +456,9 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end, } pfn = pud_pfn(pud) + ((addr & ~PUD_MASK) >> PAGE_SHIFT); - for (i = 0; i < npages; ++i, ++pfn) { - hmm_vma_walk->pgmap = get_dev_pagemap(pfn, - hmm_vma_walk->pgmap); - if (unlikely(!hmm_vma_walk->pgmap)) { - ret = -EBUSY; - goto out_unlock; - } + for (i = 0; i < npages; ++i, ++pfn) pfns[i] = hmm_device_entry_from_pfn(range, pfn) | cpu_flags; - } - if (hmm_vma_walk->pgmap) { - put_dev_pagemap(hmm_vma_walk->pgmap); - hmm_vma_walk->pgmap = NULL; - } hmm_vma_walk->last = end; goto out_unlock; } From patchwork Tue Mar 24 01:14:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11454373 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 88A8815AB for ; Tue, 24 Mar 2020 01:15:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3CE2D2070A for ; Tue, 24 Mar 2020 01:15:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="VI/Jh9ch" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3CE2D2070A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 41F176B0073; Mon, 23 Mar 2020 21:15:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3F7166B0078; Mon, 23 Mar 2020 21:15:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2BDB76B007D; Mon, 23 Mar 2020 21:15:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0079.hostedemail.com [216.40.44.79]) by kanga.kvack.org (Postfix) with ESMTP id 158626B0073 for ; Mon, 23 Mar 2020 21:15:14 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id DE3FFBDE04 for ; Tue, 24 Mar 2020 01:15:13 +0000 (UTC) X-FDA: 76628487306.11.fang41_4abc15c20630d X-Spam-Summary: 2,0,0,4c1e4cc12f9182af,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:4:41:69:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1605:1730:1747:1777:1792:2198:2199:2393:2559:2562:2640:2898:2899:2901:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4250:4321:4605:5007:6117:6119:6120:6261:6653:7576:7875:7901:7903:9036:9592:10226:11026:11232:11233:11473:11658:11914:12043:12114:12291:12296:12297:12438:12517:12519:12555:12679:12683:12895:13894:14096:14110:14394:21080:21433:21444:21451:21611:21627:21796:21990:30003:30012:30036:30054:30070,0,RBL:209.85.219.65:@ziepe.ca:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: fang41_4abc15c20630d X-Filterd-Recvd-Size: 17430 Received: from mail-qv1-f65.google.com (mail-qv1-f65.google.com [209.85.219.65]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Tue, 24 Mar 2020 01:15:13 +0000 (UTC) Received: by mail-qv1-f65.google.com with SMTP id o18so8419414qvf.1 for ; Mon, 23 Mar 2020 18:15:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=o1MlItvq3/CcOwkWaXphhJAzDcJzLbhnUG4S07COKYc=; b=VI/Jh9chK4V7s3tjYT0khOqKd5olO2FygmNSG5UqEayDfWyPOkrNoyWSHNidyulAZo JOSbcu1CEzvBIX7PhHhGD9Ujd79JI7div1m2mxN+k4bBTDqOzGIo8V1j2J/TMwana/Pu HdlLwjf68bNMDHxnWaI1PzdUzJymnj0xPmF4fN8LvF8pdUIwaF7a3L0h1PHTiWCeZ3vB 0KWxIhGmHXv8G313UYOfn3gfSb0vw9AfjGfr2NFTJQbJTEqV5cGGQNygulDaj3Ivd947 ejvRJRoW2cs1iqz36kTlmkYsq0vW791hpeOuW7OWW6WytBfh8d0HZMyi3xMCFitBt96R 0eYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=o1MlItvq3/CcOwkWaXphhJAzDcJzLbhnUG4S07COKYc=; b=WWGYwcyQL5qLsfGUJRpfUl/EJ8c7vZ+afqfdUE8d1ekON4Q/WDTfM73sxMhDUEH+Iq vNThsDDKx4B6UiedwpD5cbEwOOvr98oipf92+7n+engYg0iZ3hV9n/1XkbFhXtVegaTO aJu2nTPezfIjjvDR1+ayiq49i84FMFNSzvwKuWtiBIofCbQ0EpNOgRaDlbl33V40n7jz iCPuZ28N5M6t4OKr/lTDZzej6S1lTEqOzfNGukmQRQTN/5ICN1Q88f5fpSarKRF7ccMw b7TKYRCKB4VZ3mtgvrPmXfJ3aIZaL5c1eF0p6LTtEgwbjvAtK6bUiIu+LXMuYdZtyMxh 3qlQ== X-Gm-Message-State: ANhLgQ3NcQNg7aXb5Zqy89xyiHiaTzHLwNXzCfCCoKiHBI/eBeyCXBuQ hPtvNJ8ukmIGxPPtZpeGQAzhjA== X-Google-Smtp-Source: ADFU+vtIogRzO3pADuND7By4iCRXCsa/DxPpoXlbRrrrm//K6PkufBoV5lk3+UXqPo6EJzrbd3NFqw== X-Received: by 2002:a0c:e886:: with SMTP id b6mr23233950qvo.31.1585012512542; Mon, 23 Mar 2020 18:15:12 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id u51sm14112250qth.46.2020.03.23.18.15.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Mar 2020 18:15:10 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jGY9u-0000qu-3v; Mon, 23 Mar 2020 22:15:10 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 2/9] mm/hmm: return the fault type from hmm_pte_need_fault() Date: Mon, 23 Mar 2020 22:14:50 -0300 Message-Id: <20200324011457.2817-3-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200324011457.2817-1-jgg@ziepe.ca> References: <20200324011457.2817-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe Using two bools instead of flags return is not necessary and leads to bugs. Returning a value is easier for the compiler to check and easier to pass around the code flow. Convert the two bools into flags and push the change to all callers. Signed-off-by: Jason Gunthorpe --- mm/hmm.c | 183 ++++++++++++++++++++++++------------------------------- 1 file changed, 81 insertions(+), 102 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 3a2610e0713329..2a0eda1534bcda 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -32,6 +32,12 @@ struct hmm_vma_walk { unsigned int flags; }; +enum { + HMM_NEED_FAULT = 1 << 0, + HMM_NEED_WRITE_FAULT = HMM_NEED_FAULT | (1 << 1), + HMM_NEED_ALL_BITS = HMM_NEED_FAULT | HMM_NEED_WRITE_FAULT, +}; + static int hmm_pfns_fill(unsigned long addr, unsigned long end, struct hmm_range *range, enum hmm_pfn_value_e value) { @@ -49,8 +55,7 @@ static int hmm_pfns_fill(unsigned long addr, unsigned long end, * hmm_vma_fault() - fault in a range lacking valid pmd or pte(s) * @addr: range virtual start address (inclusive) * @end: range virtual end address (exclusive) - * @fault: should we fault or not ? - * @write_fault: write fault ? + * @required_fault: HMM_NEED_* flags * @walk: mm_walk structure * Return: -EBUSY after page fault, or page fault error * @@ -58,8 +63,7 @@ static int hmm_pfns_fill(unsigned long addr, unsigned long end, * or whenever there is no page directory covering the virtual address range. */ static int hmm_vma_fault(unsigned long addr, unsigned long end, - bool fault, bool write_fault, - struct mm_walk *walk) + unsigned int required_fault, struct mm_walk *walk) { struct hmm_vma_walk *hmm_vma_walk = walk->private; struct hmm_range *range = hmm_vma_walk->range; @@ -68,13 +72,13 @@ static int hmm_vma_fault(unsigned long addr, unsigned long end, unsigned long i = (addr - range->start) >> PAGE_SHIFT; unsigned int fault_flags = FAULT_FLAG_REMOTE; - WARN_ON_ONCE(!fault && !write_fault); + WARN_ON_ONCE(!required_fault); hmm_vma_walk->last = addr; if (!vma) goto out_error; - if (write_fault) { + if ((required_fault & HMM_NEED_WRITE_FAULT) == HMM_NEED_WRITE_FAULT) { if (!(vma->vm_flags & VM_WRITE)) return -EPERM; fault_flags |= FAULT_FLAG_WRITE; @@ -91,14 +95,13 @@ static int hmm_vma_fault(unsigned long addr, unsigned long end, return -EFAULT; } -static inline void hmm_pte_need_fault(const struct hmm_vma_walk *hmm_vma_walk, - uint64_t pfns, uint64_t cpu_flags, - bool *fault, bool *write_fault) +static unsigned int hmm_pte_need_fault(const struct hmm_vma_walk *hmm_vma_walk, + uint64_t pfns, uint64_t cpu_flags) { struct hmm_range *range = hmm_vma_walk->range; if (hmm_vma_walk->flags & HMM_FAULT_SNAPSHOT) - return; + return 0; /* * So we not only consider the individual per page request we also @@ -114,37 +117,37 @@ static inline void hmm_pte_need_fault(const struct hmm_vma_walk *hmm_vma_walk, /* We aren't ask to do anything ... */ if (!(pfns & range->flags[HMM_PFN_VALID])) - return; + return 0; - /* If CPU page table is not valid then we need to fault */ - *fault = !(cpu_flags & range->flags[HMM_PFN_VALID]); /* Need to write fault ? */ if ((pfns & range->flags[HMM_PFN_WRITE]) && - !(cpu_flags & range->flags[HMM_PFN_WRITE])) { - *write_fault = true; - *fault = true; - } + !(cpu_flags & range->flags[HMM_PFN_WRITE])) + return HMM_NEED_WRITE_FAULT; + + /* If CPU page table is not valid then we need to fault */ + if (!(cpu_flags & range->flags[HMM_PFN_VALID])) + return HMM_NEED_FAULT; + return 0; } -static void hmm_range_need_fault(const struct hmm_vma_walk *hmm_vma_walk, - const uint64_t *pfns, unsigned long npages, - uint64_t cpu_flags, bool *fault, - bool *write_fault) +static unsigned int +hmm_range_need_fault(const struct hmm_vma_walk *hmm_vma_walk, + const uint64_t *pfns, unsigned long npages, + uint64_t cpu_flags) { + unsigned int required_fault = 0; unsigned long i; - if (hmm_vma_walk->flags & HMM_FAULT_SNAPSHOT) { - *fault = *write_fault = false; - return; - } + if (hmm_vma_walk->flags & HMM_FAULT_SNAPSHOT) + return 0; - *fault = *write_fault = false; for (i = 0; i < npages; ++i) { - hmm_pte_need_fault(hmm_vma_walk, pfns[i], cpu_flags, - fault, write_fault); - if ((*write_fault)) - return; + required_fault |= + hmm_pte_need_fault(hmm_vma_walk, pfns[i], cpu_flags); + if (required_fault == HMM_NEED_ALL_BITS) + return required_fault; } + return required_fault; } static int hmm_vma_walk_hole(unsigned long addr, unsigned long end, @@ -152,17 +155,16 @@ static int hmm_vma_walk_hole(unsigned long addr, unsigned long end, { struct hmm_vma_walk *hmm_vma_walk = walk->private; struct hmm_range *range = hmm_vma_walk->range; - bool fault, write_fault; + unsigned int required_fault; unsigned long i, npages; uint64_t *pfns; i = (addr - range->start) >> PAGE_SHIFT; npages = (end - addr) >> PAGE_SHIFT; pfns = &range->pfns[i]; - hmm_range_need_fault(hmm_vma_walk, pfns, npages, - 0, &fault, &write_fault); - if (fault || write_fault) - return hmm_vma_fault(addr, end, fault, write_fault, walk); + required_fault = hmm_range_need_fault(hmm_vma_walk, pfns, npages, 0); + if (required_fault) + return hmm_vma_fault(addr, end, required_fault, walk); hmm_vma_walk->last = addr; return hmm_pfns_fill(addr, end, range, HMM_PFN_NONE); } @@ -183,16 +185,15 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, struct hmm_vma_walk *hmm_vma_walk = walk->private; struct hmm_range *range = hmm_vma_walk->range; unsigned long pfn, npages, i; - bool fault, write_fault; + unsigned int required_fault; uint64_t cpu_flags; npages = (end - addr) >> PAGE_SHIFT; cpu_flags = pmd_to_hmm_pfn_flags(range, pmd); - hmm_range_need_fault(hmm_vma_walk, pfns, npages, cpu_flags, - &fault, &write_fault); - - if (fault || write_fault) - return hmm_vma_fault(addr, end, fault, write_fault, walk); + required_fault = + hmm_range_need_fault(hmm_vma_walk, pfns, npages, cpu_flags); + if (required_fault) + return hmm_vma_fault(addr, end, required_fault, walk); pfn = pmd_pfn(pmd) + ((addr & ~PMD_MASK) >> PAGE_SHIFT); for (i = 0; addr < end; addr += PAGE_SIZE, i++, pfn++) @@ -229,18 +230,15 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, { struct hmm_vma_walk *hmm_vma_walk = walk->private; struct hmm_range *range = hmm_vma_walk->range; - bool fault, write_fault; + unsigned int required_fault; uint64_t cpu_flags; pte_t pte = *ptep; uint64_t orig_pfn = *pfn; *pfn = range->values[HMM_PFN_NONE]; - fault = write_fault = false; - if (pte_none(pte)) { - hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0, - &fault, &write_fault); - if (fault || write_fault) + required_fault = hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0); + if (required_fault) goto fault; return 0; } @@ -261,9 +259,8 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, return 0; } - hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0, &fault, - &write_fault); - if (!fault && !write_fault) + required_fault = hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0); + if (!required_fault) return 0; if (!non_swap_entry(entry)) @@ -283,9 +280,8 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, } cpu_flags = pte_to_hmm_pfn_flags(range, pte); - hmm_pte_need_fault(hmm_vma_walk, orig_pfn, cpu_flags, &fault, - &write_fault); - if (fault || write_fault) + required_fault = hmm_pte_need_fault(hmm_vma_walk, orig_pfn, cpu_flags); + if (required_fault) goto fault; /* @@ -293,9 +289,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, * fall through and treat it like a normal page. */ if (pte_special(pte) && !is_zero_pfn(pte_pfn(pte))) { - hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0, &fault, - &write_fault); - if (fault || write_fault) { + if (hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0)) { pte_unmap(ptep); return -EFAULT; } @@ -309,7 +303,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, fault: pte_unmap(ptep); /* Fault any virtual address we were asked to fault */ - return hmm_vma_fault(addr, end, fault, write_fault, walk); + return hmm_vma_fault(addr, end, required_fault, walk); } static int hmm_vma_walk_pmd(pmd_t *pmdp, @@ -322,7 +316,6 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, uint64_t *pfns = &range->pfns[(start - range->start) >> PAGE_SHIFT]; unsigned long npages = (end - start) >> PAGE_SHIFT; unsigned long addr = start; - bool fault, write_fault; pte_t *ptep; pmd_t pmd; @@ -332,9 +325,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, return hmm_vma_walk_hole(start, end, -1, walk); if (thp_migration_supported() && is_pmd_migration_entry(pmd)) { - hmm_range_need_fault(hmm_vma_walk, pfns, npages, - 0, &fault, &write_fault); - if (fault || write_fault) { + if (hmm_range_need_fault(hmm_vma_walk, pfns, npages, 0)) { hmm_vma_walk->last = addr; pmd_migration_entry_wait(walk->mm, pmdp); return -EBUSY; @@ -343,9 +334,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, } if (!pmd_present(pmd)) { - hmm_range_need_fault(hmm_vma_walk, pfns, npages, 0, &fault, - &write_fault); - if (fault || write_fault) + if (hmm_range_need_fault(hmm_vma_walk, pfns, npages, 0)) return -EFAULT; return hmm_pfns_fill(start, end, range, HMM_PFN_ERROR); } @@ -375,9 +364,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, * recover. */ if (pmd_bad(pmd)) { - hmm_range_need_fault(hmm_vma_walk, pfns, npages, 0, &fault, - &write_fault); - if (fault || write_fault) + if (hmm_range_need_fault(hmm_vma_walk, pfns, npages, 0)) return -EFAULT; return hmm_pfns_fill(start, end, range, HMM_PFN_ERROR); } @@ -434,8 +421,8 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end, if (pud_huge(pud) && pud_devmap(pud)) { unsigned long i, npages, pfn; + unsigned int required_fault; uint64_t *pfns, cpu_flags; - bool fault, write_fault; if (!pud_present(pud)) { spin_unlock(ptl); @@ -447,12 +434,11 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end, pfns = &range->pfns[i]; cpu_flags = pud_to_hmm_pfn_flags(range, pud); - hmm_range_need_fault(hmm_vma_walk, pfns, npages, - cpu_flags, &fault, &write_fault); - if (fault || write_fault) { + required_fault = hmm_range_need_fault(hmm_vma_walk, pfns, + npages, cpu_flags); + if (required_fault) { spin_unlock(ptl); - return hmm_vma_fault(addr, end, fault, write_fault, - walk); + return hmm_vma_fault(addr, end, required_fault, walk); } pfn = pud_pfn(pud) + ((addr & ~PUD_MASK) >> PAGE_SHIFT); @@ -484,7 +470,7 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, struct hmm_range *range = hmm_vma_walk->range; struct vm_area_struct *vma = walk->vma; uint64_t orig_pfn, cpu_flags; - bool fault, write_fault; + unsigned int required_fault; spinlock_t *ptl; pte_t entry; @@ -495,12 +481,10 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, orig_pfn = range->pfns[i]; range->pfns[i] = range->values[HMM_PFN_NONE]; cpu_flags = pte_to_hmm_pfn_flags(range, entry); - fault = write_fault = false; - hmm_pte_need_fault(hmm_vma_walk, orig_pfn, cpu_flags, - &fault, &write_fault); - if (fault || write_fault) { + required_fault = hmm_pte_need_fault(hmm_vma_walk, orig_pfn, cpu_flags); + if (required_fault) { spin_unlock(ptl); - return hmm_vma_fault(addr, end, fault, write_fault, walk); + return hmm_vma_fault(addr, end, required_fault, walk); } pfn = pte_pfn(entry) + ((start & ~hmask) >> PAGE_SHIFT); @@ -522,37 +506,32 @@ static int hmm_vma_walk_test(unsigned long start, unsigned long end, struct hmm_range *range = hmm_vma_walk->range; struct vm_area_struct *vma = walk->vma; + if (!(vma->vm_flags & (VM_IO | VM_PFNMAP | VM_MIXEDMAP)) && + vma->vm_flags & VM_READ) + return 0; + /* - * Skip vma ranges that don't have struct page backing them or map I/O - * devices directly. + * vma ranges that don't have struct page backing them or map I/O + * devices directly cannot be handled by hmm_range_fault(). * * If the vma does not allow read access, then assume that it does not * allow write access either. HMM does not support architectures that * allow write without read. + * + * If a fault is requested for an unsupported range then it is a hard + * failure. */ - if ((vma->vm_flags & (VM_IO | VM_PFNMAP | VM_MIXEDMAP)) || - !(vma->vm_flags & VM_READ)) { - bool fault, write_fault; - - /* - * Check to see if a fault is requested for any page in the - * range. - */ - hmm_range_need_fault(hmm_vma_walk, range->pfns + - ((start - range->start) >> PAGE_SHIFT), - (end - start) >> PAGE_SHIFT, - 0, &fault, &write_fault); - if (fault || write_fault) - return -EFAULT; - - hmm_pfns_fill(start, end, range, HMM_PFN_ERROR); - hmm_vma_walk->last = end; + if (hmm_range_need_fault(hmm_vma_walk, + range->pfns + + ((start - range->start) >> PAGE_SHIFT), + (end - start) >> PAGE_SHIFT, 0)) + return -EFAULT; - /* Skip this vma and continue processing the next vma. */ - return 1; - } + hmm_pfns_fill(start, end, range, HMM_PFN_ERROR); + hmm_vma_walk->last = end; - return 0; + /* Skip this vma and continue processing the next vma. */ + return 1; } static const struct mm_walk_ops hmm_walk_ops = { From patchwork Tue Mar 24 01:14:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11454371 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DC017139A for ; Tue, 24 Mar 2020 01:15:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 910AD2070A for ; Tue, 24 Mar 2020 01:15:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="RoM/tzGF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 910AD2070A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E81506B0071; Mon, 23 Mar 2020 21:15:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DE41D6B0073; Mon, 23 Mar 2020 21:15:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD34C6B0078; Mon, 23 Mar 2020 21:15:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0177.hostedemail.com [216.40.44.177]) by kanga.kvack.org (Postfix) with ESMTP id B3C3B6B0071 for ; Mon, 23 Mar 2020 21:15:13 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 93E66BDE00 for ; Tue, 24 Mar 2020 01:15:13 +0000 (UTC) X-FDA: 76628487306.11.cakes36_4aa87ec932c54 X-Spam-Summary: 2,0,0,2800feb275130da5,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:1:2:41:69:152:355:379:541:800:960:973:982:988:989:1260:1277:1311:1313:1314:1345:1359:1437:1515:1516:1518:1593:1594:1605:1730:1747:1777:1792:2393:2538:2559:2562:2693:2895:2897:2901:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3873:3874:4051:4250:4321:4418:4605:5007:6117:6119:6120:6261:6653:7576:7875:7901:7903:9010:9036:9592:10004:10226:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12679:12683:12895:12986:13153:13228:13894:13972:14110:14394:14659:21080:21222:21324:21325:21433:21444:21451:21627:21990:30003:30012:30034:30045:30054:30067:30070:30079,0,RBL:209.85.160.175:@ziepe.ca:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:70,LUA_SUMMARY:none X-HE-Tag: cakes36_4aa87ec932c54 X-Filterd-Recvd-Size: 12033 Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Tue, 24 Mar 2020 01:15:12 +0000 (UTC) Received: by mail-qt1-f175.google.com with SMTP id c9so6716951qtw.7 for ; Mon, 23 Mar 2020 18:15:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=V1IgTDmslkV/D1CqLLp/7w94tK/gghqP4j2p3uIw/xU=; b=RoM/tzGFA5ED2yXIEDyLh3I5kBEf3ZNyQDZNe59jeBLgXfeXpmuHUHZBUZiLIZANVj aqi6GiciKWpBXzWNErSLcL/51cL+7wq/XbsjYzdgXh4NEgifRLSdRVCEv6IQ8ovcYsJQ VxJLcdrQAVlW8PcNm9SXoTd3sUs9Ylc4zdLVHxLohBHNze8v+Ngp2T15BlDb9jLJWM+c h/FefIU/pwV3xwp+D7ybRk4OMcxoDAx+kTj/DBXKyazLeumE/yJm66W0S3IuCt/98QOw Y1tITuHzZP06NxBTPitWYr7xfyJHtLSwb19ptMh79tElXPYLbe5PGd6XrA8eM/MwcNPR cmUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=V1IgTDmslkV/D1CqLLp/7w94tK/gghqP4j2p3uIw/xU=; b=cgcJMEr08Ct3v9T7n1ziBHWQYSxmXkAlh9RpmhN0kdTotksE8TPt71g0FyXCJlry00 4Ph9tFmdAxGWXnyNYgOxR8dHYc2G05A9QrBKIEi5jZvTnpGEZVXlBew+5u5Kz4M7ARwC 2BbvmngnWm/fK5kLx+0xHUOFj/ikC1jDNCAaIse2GidMYp39Sc2Xvn+T5nZvesfsIZuL H9l5ULUQr5ozwZaH0+w3v7PbthFBJ3rZZm85dv1KcBOGTkJECRtvL1bBmxuNX0ZrgtNA k6yc5MoRVYpV8M2O70Ww4h4GRwU/vXEuGJl/h7kgtdpwT6FQQTo/tB0/e67h6J83xQvp JuAQ== X-Gm-Message-State: ANhLgQ2ryQO3zPJJHKVKufE400UFvUS6GULxSapdl84RlB6lquWX0Tq/ 3qIAuPJPmaGwaB5Omq6cYWPoJw== X-Google-Smtp-Source: ADFU+vvVg6zpq1wS6Q/R9N/Ff9hbXPMDDRJM2yCjU/ohHi3AiuZa6ZEErM6AFdhzdmfmbvy2d3WTyA== X-Received: by 2002:aed:31c1:: with SMTP id 59mr18789590qth.370.1585012512050; Mon, 23 Mar 2020 18:15:12 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id s63sm3538860qkh.64.2020.03.23.18.15.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Mar 2020 18:15:10 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jGY9u-0000r0-5O; Mon, 23 Mar 2020 22:15:10 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 3/9] mm/hmm: remove unused code and tidy comments Date: Mon, 23 Mar 2020 22:14:51 -0300 Message-Id: <20200324011457.2817-4-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200324011457.2817-1-jgg@ziepe.ca> References: <20200324011457.2817-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe Delete several functions that are never called, fix some desync between comments and structure content, toss the now out of date top of file header, and move one function only used by hmm.c into hmm.c Signed-off-by: Jason Gunthorpe Reviewed-by: Christoph Hellwig --- include/linux/hmm.h | 104 +------------------------------------------- mm/hmm.c | 24 +++++++--- 2 files changed, 19 insertions(+), 109 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index bb6be4428633a8..daee6508a3f609 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -3,58 +3,8 @@ * Copyright 2013 Red Hat Inc. * * Authors: Jérôme Glisse - */ -/* - * Heterogeneous Memory Management (HMM) - * - * See Documentation/vm/hmm.rst for reasons and overview of what HMM is and it - * is for. Here we focus on the HMM API description, with some explanation of - * the underlying implementation. - * - * Short description: HMM provides a set of helpers to share a virtual address - * space between CPU and a device, so that the device can access any valid - * address of the process (while still obeying memory protection). HMM also - * provides helpers to migrate process memory to device memory, and back. Each - * set of functionality (address space mirroring, and migration to and from - * device memory) can be used independently of the other. - * - * - * HMM address space mirroring API: - * - * Use HMM address space mirroring if you want to mirror a range of the CPU - * page tables of a process into a device page table. Here, "mirror" means "keep - * synchronized". Prerequisites: the device must provide the ability to write- - * protect its page tables (at PAGE_SIZE granularity), and must be able to - * recover from the resulting potential page faults. * - * HMM guarantees that at any point in time, a given virtual address points to - * either the same memory in both CPU and device page tables (that is: CPU and - * device page tables each point to the same pages), or that one page table (CPU - * or device) points to no entry, while the other still points to the old page - * for the address. The latter case happens when the CPU page table update - * happens first, and then the update is mirrored over to the device page table. - * This does not cause any issue, because the CPU page table cannot start - * pointing to a new page until the device page table is invalidated. - * - * HMM uses mmu_notifiers to monitor the CPU page tables, and forwards any - * updates to each device driver that has registered a mirror. It also provides - * some API calls to help with taking a snapshot of the CPU page table, and to - * synchronize with any updates that might happen concurrently. - * - * - * HMM migration to and from device memory: - * - * HMM provides a set of helpers to hotplug device memory as ZONE_DEVICE, with - * a new MEMORY_DEVICE_PRIVATE type. This provides a struct page for each page - * of the device memory, and allows the device driver to manage its memory - * using those struct pages. Having struct pages for device memory makes - * migration easier. Because that memory is not addressable by the CPU it must - * never be pinned to the device; in other words, any CPU page fault can always - * cause the device memory to be migrated (copied/moved) back to regular memory. - * - * A new migrate helper (migrate_vma()) has been added (see mm/migrate.c) that - * allows use of a device DMA engine to perform the copy operation between - * regular system memory and device memory. + * See Documentation/vm/hmm.rst for reasons and overview of what HMM is. */ #ifndef LINUX_HMM_H #define LINUX_HMM_H @@ -120,9 +70,6 @@ enum hmm_pfn_value_e { * * @notifier: a mmu_interval_notifier that includes the start/end * @notifier_seq: result of mmu_interval_read_begin() - * @hmm: the core HMM structure this range is active against - * @vma: the vm area struct for the range - * @list: all range lock are on a list * @start: range virtual start address (inclusive) * @end: range virtual end address (exclusive) * @pfns: array of pfns (big enough for the range) @@ -130,8 +77,7 @@ enum hmm_pfn_value_e { * @values: pfn value for some special case (none, special, error, ...) * @default_flags: default flags for the range (write, read, ... see hmm doc) * @pfn_flags_mask: allows to mask pfn flags so that only default_flags matter - * @pfn_shifts: pfn shift value (should be <= PAGE_SHIFT) - * @valid: pfns array did not change since it has been fill by an HMM function + * @pfn_shift: pfn shift value (should be <= PAGE_SHIFT) * @dev_private_owner: owner of device private pages */ struct hmm_range { @@ -171,52 +117,6 @@ static inline struct page *hmm_device_entry_to_page(const struct hmm_range *rang return pfn_to_page(entry >> range->pfn_shift); } -/* - * hmm_device_entry_to_pfn() - return pfn value store in a device entry - * @range: range use to decode device entry value - * @entry: device entry to extract pfn from - * Return: pfn value if device entry is valid, -1UL otherwise - */ -static inline unsigned long -hmm_device_entry_to_pfn(const struct hmm_range *range, uint64_t pfn) -{ - if (pfn == range->values[HMM_PFN_NONE]) - return -1UL; - if (pfn == range->values[HMM_PFN_ERROR]) - return -1UL; - if (pfn == range->values[HMM_PFN_SPECIAL]) - return -1UL; - if (!(pfn & range->flags[HMM_PFN_VALID])) - return -1UL; - return (pfn >> range->pfn_shift); -} - -/* - * hmm_device_entry_from_page() - create a valid device entry for a page - * @range: range use to encode HMM pfn value - * @page: page for which to create the device entry - * Return: valid device entry for the page - */ -static inline uint64_t hmm_device_entry_from_page(const struct hmm_range *range, - struct page *page) -{ - return (page_to_pfn(page) << range->pfn_shift) | - range->flags[HMM_PFN_VALID]; -} - -/* - * hmm_device_entry_from_pfn() - create a valid device entry value from pfn - * @range: range use to encode HMM pfn value - * @pfn: pfn value for which to create the device entry - * Return: valid device entry for the pfn - */ -static inline uint64_t hmm_device_entry_from_pfn(const struct hmm_range *range, - unsigned long pfn) -{ - return (pfn << range->pfn_shift) | - range->flags[HMM_PFN_VALID]; -} - /* Don't fault in missing PTEs, just snapshot the current state. */ #define HMM_FAULT_SNAPSHOT (1 << 1) diff --git a/mm/hmm.c b/mm/hmm.c index 2a0eda1534bcda..c298c936469bbb 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -38,6 +38,18 @@ enum { HMM_NEED_ALL_BITS = HMM_NEED_FAULT | HMM_NEED_WRITE_FAULT, }; +/* + * hmm_device_entry_from_pfn() - create a valid device entry value from pfn + * @range: range use to encode HMM pfn value + * @pfn: pfn value for which to create the device entry + * Return: valid device entry for the pfn + */ +static uint64_t hmm_device_entry_from_pfn(const struct hmm_range *range, + unsigned long pfn) +{ + return (pfn << range->pfn_shift) | range->flags[HMM_PFN_VALID]; +} + static int hmm_pfns_fill(unsigned long addr, unsigned long end, struct hmm_range *range, enum hmm_pfn_value_e value) { @@ -544,7 +556,7 @@ static const struct mm_walk_ops hmm_walk_ops = { /** * hmm_range_fault - try to fault some address in a virtual address range - * @range: range being faulted + * @range: argument structure * @flags: HMM_FAULT_* flags * * Return: the number of valid pages in range->pfns[] (from range start @@ -558,13 +570,11 @@ static const struct mm_walk_ops hmm_walk_ops = { * only). * -EBUSY: The range has been invalidated and the caller needs to wait for * the invalidation to finish. - * -EFAULT: Invalid (i.e., either no valid vma or it is illegal to access - * that range) number of valid pages in range->pfns[] (from - * range start address). + * -EFAULT: A page was requested to be valid and could not be made valid + * ie it has no backing VMA or it is illegal to access * - * This is similar to a regular CPU page fault except that it will not trigger - * any memory migration if the memory being faulted is not accessible by CPUs - * and caller does not ask for migration. + * This is similar to get_user_pages(), except that it can read the page tables + * without mutating them (ie causing faults). * * On error, for one virtual address in the range, the function will mark the * corresponding HMM pfn entry with an error flag. From patchwork Tue Mar 24 01:14:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11454375 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6ED0615AB for ; Tue, 24 Mar 2020 01:15:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 22CE92070A for ; Tue, 24 Mar 2020 01:15:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="Nj1g0jZS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 22CE92070A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 900DB6B0078; Mon, 23 Mar 2020 21:15:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8B17E6B007D; Mon, 23 Mar 2020 21:15:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 72CFF6B007E; Mon, 23 Mar 2020 21:15:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0195.hostedemail.com [216.40.44.195]) by kanga.kvack.org (Postfix) with ESMTP id 581426B007D for ; Mon, 23 Mar 2020 21:15:14 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 2FEB3BD3D8 for ; Tue, 24 Mar 2020 01:15:14 +0000 (UTC) X-FDA: 76628487348.25.land65_4ac713871115f X-Spam-Summary: 2,0,0,067824493147453f,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:1:2:41:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1605:1730:1747:1777:1792:1801:1981:2194:2198:2199:2200:2393:2553:2559:2562:2693:2895:2901:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4049:4250:4321:4605:5007:6117:6119:6120:6261:6653:7576:7875:7901:7903:9036:9592:10004:11026:11232:11473:11657:11658:11914:12043:12295:12296:12297:12438:12517:12519:12555:12679:12895:12986:13149:13161:13229:13230:13894:14394:21063:21080:21324:21325:21433:21444:21451:21627:21990:30003:30012:30034:30045:30046:30054:30070:30079:30090,0,RBL:209.85.219.68:@ziepe.ca:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: land65_4ac713871115f X-Filterd-Recvd-Size: 10074 Received: from mail-qv1-f68.google.com (mail-qv1-f68.google.com [209.85.219.68]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Tue, 24 Mar 2020 01:15:13 +0000 (UTC) Received: by mail-qv1-f68.google.com with SMTP id cy12so8409784qvb.7 for ; Mon, 23 Mar 2020 18:15:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FMKo6LdkFYCdsPHJF0fYulYQYBiJ803wxyfJC3f+mbw=; b=Nj1g0jZSQ0AczJrww7hWZY8r8tpap5CLTIvsn2yaRNNdxslEPOg+842Vn6LrtGd1tg HtjjeNAsCQRHtcZtHJpW4DXurPnEed+4T4vGiYe3PnvJn4L94pHllJ37TuOIUV401H5D 94+36Td+rsulGU9kB7n1AAPV9NeNloGJsjC3xcK0nD7e6Y7t/TP6ON81j2BxPu1HyFcn EuaU5WtR5iSqz1oi81pzB0K/Iol09/AusYChyigO43dlEyf9VGWdYvMdcOvoznq37TSm ntW3K0awqVSSmHN4B+Ru6FQks+LZvBmMAGtITUgvEeLCQyOaQD9tzUdnK65nwuY5YgUu Qs3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FMKo6LdkFYCdsPHJF0fYulYQYBiJ803wxyfJC3f+mbw=; b=bZ85j0O03A2XghR+tehWw9lpqhL/Hkbtv5IF4PS0eOJDmOcRJsG/gW8TtaZO01Qenv 18aRnV7uLJLTeGuHuJLAqrQP63eTBiAgUlxANYaSNsV0BTHMQ5/1bC3UvSY9VzPgElJy VpHRDTLtJJcG8P4cZ8PZRhqw1d1F8vFwL0kUgmhf9smZzip1iG7QIHcfAewu5yjUOLuT z3r007AsljNsBGjXHunOC7Wy3rijx77v41Jjf9gY6XoafAtObXar0fS541hr2B9tJYYu yTNFLr0smAMOkkZ1udKhZmhShX05Xwf77qRdrVLGjAKd9AUMl4NkO11GLPvVGpEjCQH5 wKKw== X-Gm-Message-State: ANhLgQ0CONVubfo0LghMe7m9621fq2CvVjSyxWgL/WuZBRm6JrcAz4gs /YhgACkh36RJfDp9c2RlFAa4fw== X-Google-Smtp-Source: ADFU+vs1SPcubDDYu/nEfYMoed2J0S2bdq7cYi/3vm2e/mEXWMX5+97u67uWczDe8pro/1SWHnVy6A== X-Received: by 2002:ad4:4a89:: with SMTP id h9mr23410432qvx.168.1585012513012; Mon, 23 Mar 2020 18:15:13 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id n4sm12902832qti.55.2020.03.23.18.15.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Mar 2020 18:15:10 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jGY9u-0000r6-72; Mon, 23 Mar 2020 22:15:10 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 4/9] mm/hmm: remove HMM_FAULT_SNAPSHOT Date: Mon, 23 Mar 2020 22:14:52 -0300 Message-Id: <20200324011457.2817-5-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200324011457.2817-1-jgg@ziepe.ca> References: <20200324011457.2817-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe Now that flags are handled on a fine-grained per-page basis this global flag is redundant and has a confusing overlap with the pfn_flags_mask and default_flags. Normalize the HMM_FAULT_SNAPSHOT behavior into one place. Callers needing the SNAPSHOT behavior should set a pfn_flags_mask and default_flags that always results in a cleared HMM_PFN_VALID. Then no pages will be faulted, and HMM_FAULT_SNAPSHOT is not a special flow that overrides the masking mechanism. As this is the last flag, also remove the flags argument. If future flags are needed they can be part of the struct hmm_range function arguments. Signed-off-by: Jason Gunthorpe --- Documentation/vm/hmm.rst | 12 +++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 2 +- drivers/gpu/drm/nouveau/nouveau_svm.c | 2 +- include/linux/hmm.h | 5 +---- mm/hmm.c | 22 ++++++++++++++-------- 5 files changed, 22 insertions(+), 21 deletions(-) diff --git a/Documentation/vm/hmm.rst b/Documentation/vm/hmm.rst index 95fec596836262..4e3e9362afeb10 100644 --- a/Documentation/vm/hmm.rst +++ b/Documentation/vm/hmm.rst @@ -161,13 +161,11 @@ device must complete the update before the driver callback returns. When the device driver wants to populate a range of virtual addresses, it can use:: - long hmm_range_fault(struct hmm_range *range, unsigned int flags); + long hmm_range_fault(struct hmm_range *range); -With the HMM_RANGE_SNAPSHOT flag, it will only fetch present CPU page table -entries and will not trigger a page fault on missing or non-present entries. -Without that flag, it does trigger a page fault on missing or read-only entries -if write access is requested (see below). Page faults use the generic mm page -fault code path just like a CPU page fault. +It will trigger a page fault on missing or read-only entries if write access is +requested (see below). Page faults use the generic mm page fault code path just +like a CPU page fault. Both functions copy CPU page table entries into their pfns array argument. Each entry in that array corresponds to an address in the virtual range. HMM @@ -197,7 +195,7 @@ The usage pattern is:: again: range.notifier_seq = mmu_interval_read_begin(&interval_sub); down_read(&mm->mmap_sem); - ret = hmm_range_fault(&range, HMM_RANGE_SNAPSHOT); + ret = hmm_range_fault(&range); if (ret) { up_read(&mm->mmap_sem); if (ret == -EBUSY) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 90821ce5e6cad0..c520290709371b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -856,7 +856,7 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages) range->notifier_seq = mmu_interval_read_begin(&bo->notifier); down_read(&mm->mmap_sem); - r = hmm_range_fault(range, 0); + r = hmm_range_fault(range); up_read(&mm->mmap_sem); if (unlikely(r <= 0)) { /* diff --git a/drivers/gpu/drm/nouveau/nouveau_svm.c b/drivers/gpu/drm/nouveau/nouveau_svm.c index 39c731a99937c6..e3797b2d4d1759 100644 --- a/drivers/gpu/drm/nouveau/nouveau_svm.c +++ b/drivers/gpu/drm/nouveau/nouveau_svm.c @@ -540,7 +540,7 @@ static int nouveau_range_fault(struct nouveau_svmm *svmm, range.default_flags = 0; range.pfn_flags_mask = -1UL; down_read(&mm->mmap_sem); - ret = hmm_range_fault(&range, 0); + ret = hmm_range_fault(&range); up_read(&mm->mmap_sem); if (ret <= 0) { if (ret == 0 || ret == -EBUSY) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index daee6508a3f609..7475051100c782 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -117,13 +117,10 @@ static inline struct page *hmm_device_entry_to_page(const struct hmm_range *rang return pfn_to_page(entry >> range->pfn_shift); } -/* Don't fault in missing PTEs, just snapshot the current state. */ -#define HMM_FAULT_SNAPSHOT (1 << 1) - /* * Please see Documentation/vm/hmm.rst for how to use the range API. */ -long hmm_range_fault(struct hmm_range *range, unsigned int flags); +long hmm_range_fault(struct hmm_range *range); /* * HMM_RANGE_DEFAULT_TIMEOUT - default timeout (ms) when waiting for a range diff --git a/mm/hmm.c b/mm/hmm.c index c298c936469bbb..43d107a4d9dec6 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -29,7 +29,6 @@ struct hmm_vma_walk { struct hmm_range *range; unsigned long last; - unsigned int flags; }; enum { @@ -112,9 +111,6 @@ static unsigned int hmm_pte_need_fault(const struct hmm_vma_walk *hmm_vma_walk, { struct hmm_range *range = hmm_vma_walk->range; - if (hmm_vma_walk->flags & HMM_FAULT_SNAPSHOT) - return 0; - /* * So we not only consider the individual per page request we also * consider the default flags requested for the range. The API can @@ -142,15 +138,27 @@ static unsigned int hmm_pte_need_fault(const struct hmm_vma_walk *hmm_vma_walk, return 0; } +/* + * If the valid flag is masked off, and default_flags doesn't set valid, then + * hmm_pte_need_fault() always returns 0. + */ +static bool hmm_can_fault(struct hmm_range *range) +{ + return ((range->flags[HMM_PFN_VALID] & range->pfn_flags_mask) | + range->default_flags) & + range->flags[HMM_PFN_VALID]; +} + static unsigned int hmm_range_need_fault(const struct hmm_vma_walk *hmm_vma_walk, const uint64_t *pfns, unsigned long npages, uint64_t cpu_flags) { + struct hmm_range *range = hmm_vma_walk->range; unsigned int required_fault = 0; unsigned long i; - if (hmm_vma_walk->flags & HMM_FAULT_SNAPSHOT) + if (!hmm_can_fault(range)) return 0; for (i = 0; i < npages; ++i) { @@ -557,7 +565,6 @@ static const struct mm_walk_ops hmm_walk_ops = { /** * hmm_range_fault - try to fault some address in a virtual address range * @range: argument structure - * @flags: HMM_FAULT_* flags * * Return: the number of valid pages in range->pfns[] (from range start * address), which may be zero. On error one of the following status codes @@ -579,12 +586,11 @@ static const struct mm_walk_ops hmm_walk_ops = { * On error, for one virtual address in the range, the function will mark the * corresponding HMM pfn entry with an error flag. */ -long hmm_range_fault(struct hmm_range *range, unsigned int flags) +long hmm_range_fault(struct hmm_range *range) { struct hmm_vma_walk hmm_vma_walk = { .range = range, .last = range->start, - .flags = flags, }; struct mm_struct *mm = range->notifier->mm; int ret; From patchwork Tue Mar 24 01:14:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11454377 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0E3BE139A for ; Tue, 24 Mar 2020 01:15:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CCCB22070A for ; Tue, 24 Mar 2020 01:15:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="HhKFH+h9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CCCB22070A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 189AC6B007D; Mon, 23 Mar 2020 21:15:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0C5CB6B007E; Mon, 23 Mar 2020 21:15:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D714D6B0080; Mon, 23 Mar 2020 21:15:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0068.hostedemail.com [216.40.44.68]) by kanga.kvack.org (Postfix) with ESMTP id A7C8D6B007E for ; Mon, 23 Mar 2020 21:15:14 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 90D0618C9340D for ; Tue, 24 Mar 2020 01:15:14 +0000 (UTC) X-FDA: 76628487348.14.waste06_4ad7e5aebf84a X-Spam-Summary: 2,0,0,25018da2ac54239a,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:41:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2198:2199:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3866:3867:3871:4321:4419:5007:6120:6261:6653:7576:7808:7901:10004:11026:11473:11658:11914:12043:12114:12295:12296:12297:12438:12517:12519:12555:12679:12895:13069:13161:13229:13311:13357:13894:14181:14384:14394:14721:21080:21444:21627:21990:30012:30054:30064,0,RBL:209.85.160.196:@ziepe.ca:.lbl8.mailshell.net-66.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:27,LUA_SUMMARY:none X-HE-Tag: waste06_4ad7e5aebf84a X-Filterd-Recvd-Size: 4733 Received: from mail-qt1-f196.google.com (mail-qt1-f196.google.com [209.85.160.196]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Tue, 24 Mar 2020 01:15:14 +0000 (UTC) Received: by mail-qt1-f196.google.com with SMTP id t9so9740871qto.9 for ; Mon, 23 Mar 2020 18:15:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8z+Ak8LJ4cusnv639Xm98KV9Wso4IbiTcHYELSH/G24=; b=HhKFH+h9dsFIf1c+y5ZAjNcHJkLfL/7TByr8KqLLVhFohTYKwWhXx2XiETLziarPtW HmC89Ua/ZfOQ53NE5bjqh2YkzI3VjXOn9sjDenilAjM2tgLtIs+v5bMY3AnTgGPB6sBY b1oed859g52WVKLkm7EBX27mr9X3LBNTceG4ymOTF3BxLJp8wDGWq+wXPMW1orpJA3y1 o4TLZlvcbSHFVCu7RdAGl/h2z1hO8vJNXGZX6OIQ3uwn+EpAbkt+7b1ZXGgMMqMoOzpy FqJ5FKyR0qY7mwkSGi6gaRmlHxT/E4dqGkJiaugNvgjpT+6iFBJR1xYGeMw1Je2MPBe6 BQXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8z+Ak8LJ4cusnv639Xm98KV9Wso4IbiTcHYELSH/G24=; b=IiGs6vLUmQVzEN9KTYagsHBCFkEdxelmD+0sJLGGp9feF29sIDJtmvZusWwWQif/6P ObW3xHM78LBKNFE0Nzh9MG0jLbztwjGK6vGtjeMYcV/AFPWVwp7vnJspASC+lnQtfcmI 45boA6K9XaaAiA+wqaBbSDUkIyBOD1TgfH4F62S0KSy5l8KxlOXXNnyDo25za5APeAhU 77mcrvap6wge+LLE6FhPMh0mWfqt23NNLT5amKqFUtbwCNUW+RY5ELV5SHLoPeO4eNn2 tzpUMFHr0vmXZ1n+ckptreeGOuS2LItfesGrJ+WkRyx46S3dDL+w1sFweFM5Z63ScGkG TdUA== X-Gm-Message-State: ANhLgQ0zBnemLM/8jY+7xpmfDjv7vp1zHnQ8ewrMzVzqNGx+uFtFj5fI K0WhwW2BlBMsh16PY6TajWr/Rg== X-Google-Smtp-Source: ADFU+vtZbGElVq6NnluuIoEaJ5hmkuMgLimt1kzM3fgd1S4xdMrXicEqcrH3LtFkdjgiw9dY0n+5gA== X-Received: by 2002:ac8:3105:: with SMTP id g5mr3662259qtb.358.1585012513624; Mon, 23 Mar 2020 18:15:13 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id l13sm12025440qke.116.2020.03.23.18.15.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Mar 2020 18:15:11 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jGY9u-0000rC-9H; Mon, 23 Mar 2020 22:15:10 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 5/9] mm/hmm: remove the CONFIG_TRANSPARENT_HUGEPAGE #ifdef Date: Mon, 23 Mar 2020 22:14:53 -0300 Message-Id: <20200324011457.2817-6-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200324011457.2817-1-jgg@ziepe.ca> References: <20200324011457.2817-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe This code can be compiled when CONFIG_TRANSPARENT_HUGEPAGE is off, so remove the ifdef. The function is only ever called under if (pmd_devmap(pmd) || pmd_trans_huge(pmd)) Which is statically false if !CONFIG_TRANSPARENT_HUGEPAGE, so the compiler reliably eliminates all of this code. Reviewed-by: Ralph Campbell Signed-off-by: Jason Gunthorpe Reviewed-by: Christoph Hellwig --- mm/hmm.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 43d107a4d9dec6..f59e59fb303e95 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -198,7 +198,6 @@ static inline uint64_t pmd_to_hmm_pfn_flags(struct hmm_range *range, pmd_t pmd) range->flags[HMM_PFN_VALID]; } -#ifdef CONFIG_TRANSPARENT_HUGEPAGE static int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, unsigned long end, uint64_t *pfns, pmd_t pmd) { @@ -221,11 +220,6 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, hmm_vma_walk->last = end; return 0; } -#else /* CONFIG_TRANSPARENT_HUGEPAGE */ -/* stub to allow the code below to compile */ -int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, - unsigned long end, uint64_t *pfns, pmd_t pmd); -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ static inline bool hmm_is_device_private_entry(struct hmm_range *range, swp_entry_t entry) From patchwork Tue Mar 24 01:14:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11454379 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C4C71139A for ; Tue, 24 Mar 2020 01:15:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 93EF92070A for ; Tue, 24 Mar 2020 01:15:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="j2Fj9mGG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 93EF92070A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 706556B007E; Mon, 23 Mar 2020 21:15:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6E0D56B0080; Mon, 23 Mar 2020 21:15:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 559DD6B0081; Mon, 23 Mar 2020 21:15:15 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0143.hostedemail.com [216.40.44.143]) by kanga.kvack.org (Postfix) with ESMTP id 290786B0080 for ; Mon, 23 Mar 2020 21:15:15 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 070C71851CC9E for ; Tue, 24 Mar 2020 01:15:15 +0000 (UTC) X-FDA: 76628487390.17.hook59_4ae703656810c X-Spam-Summary: 2,0,0,d1df591239a5aac6,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1540:1711:1714:1730:1747:1777:1792:2393:2559:2562:2892:3138:3139:3140:3141:3142:3350:3865:3866:3867:3870:4321:5007:6119:6120:6261:6653:7576:7901:10004:11026:11473:11658:11914:12043:12114:12296:12297:12438:12517:12519:12555:12679:12895:13069:13161:13229:13311:13357:13894:14181:14384:14394:14721:21080:21444:21627:30054:30064:30075,0,RBL:209.85.222.193:@ziepe.ca:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: hook59_4ae703656810c X-Filterd-Recvd-Size: 4206 Received: from mail-qk1-f193.google.com (mail-qk1-f193.google.com [209.85.222.193]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Tue, 24 Mar 2020 01:15:14 +0000 (UTC) Received: by mail-qk1-f193.google.com with SMTP id k13so5984711qki.2 for ; Mon, 23 Mar 2020 18:15:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BtjvXIRl7JvjxGCCwhwKty2zmKTxYCK8uhb2/phXd0E=; b=j2Fj9mGGXkb0s5jfx+PaOyQglxSKvi9DzQ14cmVcTgbF31/1q+wroCRgzrwp6o0lQ/ PMrHTYwMUGlLwsiNJF2XuxvJX4Aj6pKrULOtMy1PyM1vypcNPVUYzBmwCBJNESpWOsyS JOkIyj3fbVfemR9nxyftNMco7DJxkZ6NYasYiV3jSGP2jWoKOaqJekhfAVS0HndDoMRY sEKvbkkAiciWAwNov/v7Qg1GYL8sIHnYstAwrJO22Txcxgv2IyW8RiB5s9/XVFJogf6L DJNCkbc11+nvZK7Tq18ykvSovfN4GbCtjxmOr3RGIWx/7vmJWd3LSencrwufu6ElenGz CxPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BtjvXIRl7JvjxGCCwhwKty2zmKTxYCK8uhb2/phXd0E=; b=mNt8oPgSxxnMJUqoIzpNBdixuoAQQ049Hzp5qpXYgMzbOSuzgzP129SURMIkn9BFuA JoOWNlqUEJbDmiNEZO0r0YAGO7lQOm2M45OIhUdXgTwEZIElUSv1VJbxgePyKWYFoYO5 1iWW+TnM4YCaOK850Ttyi+va58u8EQYBT8yRGpseq8UTRNPHbrMpIH6G6BmLaJ7JnoWt vZsgJMIzwB5orZqwCHFZRbNRXfInG7hgKSBxMbcDyXFyjlz10fNUARE+fhadkZ0c4Vgf I/aBybaqkDgiuRTqFKn20orEan3wH0kJOHMZ/cQue3559vbzqV6qEzvvor8+lBp5x3pX yojQ== X-Gm-Message-State: ANhLgQ2N3KipoFCPKRig1eOoTAfOL6uU4NTM2f7FoNKuVjToeQXSnF3B aMESeBamoaf/emrP3cWqpuANRA== X-Google-Smtp-Source: ADFU+vuqtjILT193yy1E1b2Co21btJLxUdSeIXs67XJc39y6GefW1zq10kzN67rE1jTtVo2ZUkW7VA== X-Received: by 2002:a37:9a81:: with SMTP id c123mr18495488qke.176.1585012513991; Mon, 23 Mar 2020 18:15:13 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id 199sm12325170qkm.7.2020.03.23.18.15.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Mar 2020 18:15:11 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jGY9u-0000rI-Aa; Mon, 23 Mar 2020 22:15:10 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 6/9] mm/hmm: use device_private_entry_to_pfn() Date: Mon, 23 Mar 2020 22:14:54 -0300 Message-Id: <20200324011457.2817-7-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200324011457.2817-1-jgg@ziepe.ca> References: <20200324011457.2817-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe swp_offset() should not be called directly, the wrappers are supposed to abstract away the encoding of the device_private specific information in the swap entry. Reviewed-by: Ralph Campbell Signed-off-by: Jason Gunthorpe Reviewed-by: Christoph Hellwig --- mm/hmm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/hmm.c b/mm/hmm.c index f59e59fb303e95..e114110ad498a2 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -266,7 +266,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, */ if (hmm_is_device_private_entry(range, entry)) { *pfn = hmm_device_entry_from_pfn(range, - swp_offset(entry)); + device_private_entry_to_pfn(entry)); *pfn |= range->flags[HMM_PFN_VALID]; if (is_write_device_private_entry(entry)) *pfn |= range->flags[HMM_PFN_WRITE]; From patchwork Tue Mar 24 01:14:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11454381 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 44AF815AB for ; Tue, 24 Mar 2020 01:15:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0A2922070A for ; Tue, 24 Mar 2020 01:15:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="JOsrAyd1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0A2922070A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B9A1E6B0080; Mon, 23 Mar 2020 21:15:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B4BFB6B0082; Mon, 23 Mar 2020 21:15:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9781F6B0080; Mon, 23 Mar 2020 21:15:15 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0018.hostedemail.com [216.40.44.18]) by kanga.kvack.org (Postfix) with ESMTP id 760016B0081 for ; Mon, 23 Mar 2020 21:15:15 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 667B4B61D0 for ; Tue, 24 Mar 2020 01:15:15 +0000 (UTC) X-FDA: 76628487390.25.brass00_4af4ec6fac462 X-Spam-Summary: 2,0,0,40e7861cfac1fdb5,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:41:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1541:1711:1730:1747:1777:1792:2393:2559:2562:2693:3138:3139:3140:3141:3142:3352:3865:3867:3868:3870:3874:4419:5007:6117:6119:6120:6261:6653:7576:7875:7901:7903:9036:10004:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12679:12895:13069:13161:13229:13311:13357:13894:14181:14384:14394:14721:21080:21444:21451:21627:21990:30054:30070,0,RBL:209.85.160.194:@ziepe.ca:.lbl8.mailshell.net-66.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: brass00_4af4ec6fac462 X-Filterd-Recvd-Size: 5159 Received: from mail-qt1-f194.google.com (mail-qt1-f194.google.com [209.85.160.194]) by imf17.hostedemail.com (Postfix) with ESMTP for ; Tue, 24 Mar 2020 01:15:14 +0000 (UTC) Received: by mail-qt1-f194.google.com with SMTP id 10so13715435qtp.1 for ; Mon, 23 Mar 2020 18:15:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RCEeJYfl6outct1qYnu7zwwXyFPg9MEiDmZ4l7AVE1o=; b=JOsrAyd16cYncuUmAnJofPxGEHPSMyXzVFRHTfUgaUtKQnMKQ9ynOt7d3lrwavlJp/ /IN0JyTxmAZ607ioCuNBWUktkLbgo/lpEm8DpreW8jOHxDlm1mxZZt9goSjN+0fKXnmg wJNKZ76KzknJPUVUZ3PaCndHijOTKfg69UiiJ5Jve2aEBIX3xThDU/BiqKbNtE0i32HM ZOzVclmIMhiAtmspcaRtkpSQqwwxoiL6rN/ZNJp3Ck415ZyIO+L5IlYTSO9QrYFRfnad +HYRZALtdAgJp8Hes3XgCqTNovtZFIIuSgA4CFH47W7eqX09xN2f+1uA1soJ8AcGpGnq 9Lkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RCEeJYfl6outct1qYnu7zwwXyFPg9MEiDmZ4l7AVE1o=; b=uFmJgTODTtM3TbYi3xD5OPa6I+FPFII/OJrOZ66sLkJpNXNIXkergNxWEY/zrGubkD kBL+oXIdlxZpwbD+J4OjRAZhCEQJi0cyu277cHc13G/aG1PdEzBkFqpi8R4oYz+5y+39 L+tGJC7CJLT8bVMbezBCJ/ZPETINqmdYZ6oCKXJ2h1gJZHqb5kKxyal/VyasCjaGwVjM Z0scjlg6ox0Bi+IKWQkt8o48EBoQSN/QOaT0zNbxAhKRwD3zlpZduJvLPRIeaGdGJ8CH GGQHGfACP+60TZVsEyRxyhj+1Jq+dRf+JowM+71SXRKUTnHjmIl8V9XzwW4W02pwDByy xGzw== X-Gm-Message-State: ANhLgQ0uif6/m0lCtJK1XAuFakJISyxs6KNikZOvPm01m4wQ7KGUFE70 BNWdMSna3a+8UH+CnbCJV2p8MA== X-Google-Smtp-Source: ADFU+vvXCEgg5WFWwZVfX5BvFcxgytFNWvUrqpjFKiAB7q1rUxOi4zWEQxeos003SLALiWs6BazZwA== X-Received: by 2002:ac8:4f03:: with SMTP id b3mr18213749qte.282.1585012514361; Mon, 23 Mar 2020 18:15:14 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id x203sm12013242qkb.44.2020.03.23.18.15.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Mar 2020 18:15:13 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jGY9u-0000rO-Cg; Mon, 23 Mar 2020 22:15:10 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 7/9] mm/hmm: do not unconditionally set pfns when returning EBUSY Date: Mon, 23 Mar 2020 22:14:55 -0300 Message-Id: <20200324011457.2817-8-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200324011457.2817-1-jgg@ziepe.ca> References: <20200324011457.2817-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe In hmm_vma_handle_pte() and hmm_vma_walk_hugetlb_entry() if fault happens then -EBUSY will be returned and the pfns input flags will have been destroyed. For hmm_vma_handle_pte() set HMM_PFN_NONE only on the success returns that don't otherwise store to pfns. For hmm_vma_walk_hugetlb_entry() all exit paths already set pfns, so remove the redundant store. Fixes: 2aee09d8c116 ("mm/hmm: change hmm_vma_fault() to allow write fault on page basis") Signed-off-by: Jason Gunthorpe Reviewed-by: Christoph Hellwig --- mm/hmm.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index e114110ad498a2..bf77b852f12d3a 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -249,11 +249,11 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, pte_t pte = *ptep; uint64_t orig_pfn = *pfn; - *pfn = range->values[HMM_PFN_NONE]; if (pte_none(pte)) { required_fault = hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0); if (required_fault) goto fault; + *pfn = range->values[HMM_PFN_NONE]; return 0; } @@ -274,8 +274,10 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, } required_fault = hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0); - if (!required_fault) + if (!required_fault) { + *pfn = range->values[HMM_PFN_NONE]; return 0; + } if (!non_swap_entry(entry)) goto fault; @@ -493,7 +495,6 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, i = (start - range->start) >> PAGE_SHIFT; orig_pfn = range->pfns[i]; - range->pfns[i] = range->values[HMM_PFN_NONE]; cpu_flags = pte_to_hmm_pfn_flags(range, entry); required_fault = hmm_pte_need_fault(hmm_vma_walk, orig_pfn, cpu_flags); if (required_fault) { From patchwork Tue Mar 24 01:14:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11454383 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 99B5C15AB for ; Tue, 24 Mar 2020 01:15:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 688112070A for ; Tue, 24 Mar 2020 01:15:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="hvs5QGf0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 688112070A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0EA8E6B0081; Mon, 23 Mar 2020 21:15:16 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 09C196B0082; Mon, 23 Mar 2020 21:15:16 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ECCD26B0083; Mon, 23 Mar 2020 21:15:15 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0175.hostedemail.com [216.40.44.175]) by kanga.kvack.org (Postfix) with ESMTP id D43EB6B0081 for ; Mon, 23 Mar 2020 21:15:15 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id AA98882DB5 for ; Tue, 24 Mar 2020 01:15:15 +0000 (UTC) X-FDA: 76628487390.26.gate64_4b00f91380758 X-Spam-Summary: 2,0,0,c2215c0a02c1bec7,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:41:69:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1542:1711:1730:1747:1777:1792:2198:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3354:3865:3866:3867:3870:3871:3872:3874:4250:4321:5007:6117:6119:6120:6261:6653:7576:7875:7901:7903:9036:9592:10004:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12679:12895:13894:14181:14394:14721:21080:21433:21444:21627:21990:30003:30012:30054,0,RBL:209.85.222.196:@ziepe.ca:.lbl8.mailshell.net-66.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: gate64_4b00f91380758 X-Filterd-Recvd-Size: 5911 Received: from mail-qk1-f196.google.com (mail-qk1-f196.google.com [209.85.222.196]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Tue, 24 Mar 2020 01:15:15 +0000 (UTC) Received: by mail-qk1-f196.google.com with SMTP id l25so12845054qki.7 for ; Mon, 23 Mar 2020 18:15:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6ikMiGopONSmGx8OLjWFTB0Uq9GGdoKVpRzgxdsKjDE=; b=hvs5QGf0jmRvSVP0uvKT4F36pRHcuVBcLvvrAOORF3tJWFJ/r8EMh9If+rsHZjGMej 0oiNAOLYvFCk8yhmmAKzYjQv/NpIX8VrgfadBPjJzMLn4KYdijiXiD8RoFvh/T9MmHYc j5Xnlsz2vE2JdTSl9/yvYIAD+XWQMBcSs5W4JKPD4c40Vah9bMdT4/dLp1geVnNKQG+N W6j1ga+NPsIulo4zBm0Kj+sUcQ3tC+A1zKFgIBCwxjgF/8aqEcjdOoZlPNz+QVGMYus5 GgbR7H+yU2BR84S3OORNa0BPpKMS53JJLPnxshumgsSbIzqWJONPwlMR8yTUxe0xayER 2xlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6ikMiGopONSmGx8OLjWFTB0Uq9GGdoKVpRzgxdsKjDE=; b=Wgt2WYQRZQWRxlyDWWGY2gi6BKVrZ59nRcFSYpJiEjdqmfA+S8fI051LO6sROy4wxk tHcJaEZAZjNv6z0OytQ1ErkgDJF+oR2J+NCKWqVnMBYlI+KmlO8BsEQoyCM1TgpfablT yiBM3rYJo0GP905t/FbiLnVe9Jle1MV7g1lXopBrcJloHcJmob3oEwvfSVIHSm9cma4U hpBDpO1LVaNBk5QY7BecQvwdpRrt23SkEjS86v3fYhY3lJFVWgwVb/u05LLAVkDEUcT2 DGDEZDXHytO/TRREac4rGyQiNodEtqeRCOj1S3RYNIhb8BaIv3prc1kRFXl/mo4LSNlz ubYQ== X-Gm-Message-State: ANhLgQ0UxH0XTbxaS+vJUn2z+ZDZVp1SE8QhvJatDzlDoi1ZWzeeNA9p xpmbDR7bklYSwTDI7Dgx/sfjqQ== X-Google-Smtp-Source: ADFU+vuZpepMcJKEuA2+c4r701ctyS8+lSo0DrK9q4pQxbmg58MGtjdzJn9NH5w8Jxb5T8n5goA8lw== X-Received: by 2002:a37:8101:: with SMTP id c1mr22947805qkd.236.1585012514707; Mon, 23 Mar 2020 18:15:14 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id u77sm12241810qka.134.2020.03.23.18.15.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Mar 2020 18:15:13 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jGY9u-0000rU-E3; Mon, 23 Mar 2020 22:15:10 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 8/9] mm/hmm: do not set pfns when returning an error code Date: Mon, 23 Mar 2020 22:14:56 -0300 Message-Id: <20200324011457.2817-9-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200324011457.2817-1-jgg@ziepe.ca> References: <20200324011457.2817-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe Most places that return an error code, like -EFAULT, do not set HMM_PFN_ERROR, only two places do this. Resolve this inconsistency by never setting the pfns on an error exit. This doesn't seem like a worthwhile thing to do anyhow. If for some reason it becomes important, it makes more sense to directly return the address of the failing page rather than have the caller scan for the HMM_PFN_ERROR. No caller inspects the pnfs output array if hmm_range_fault() fails. Signed-off-by: Jason Gunthorpe Reviewed-by: Christoph Hellwig --- mm/hmm.c | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index bf77b852f12d3a..14c33e1225866c 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -77,17 +77,14 @@ static int hmm_vma_fault(unsigned long addr, unsigned long end, unsigned int required_fault, struct mm_walk *walk) { struct hmm_vma_walk *hmm_vma_walk = walk->private; - struct hmm_range *range = hmm_vma_walk->range; struct vm_area_struct *vma = walk->vma; - uint64_t *pfns = range->pfns; - unsigned long i = (addr - range->start) >> PAGE_SHIFT; unsigned int fault_flags = FAULT_FLAG_REMOTE; WARN_ON_ONCE(!required_fault); hmm_vma_walk->last = addr; if (!vma) - goto out_error; + return -EFAULT; if ((required_fault & HMM_NEED_WRITE_FAULT) == HMM_NEED_WRITE_FAULT) { if (!(vma->vm_flags & VM_WRITE)) @@ -95,15 +92,10 @@ static int hmm_vma_fault(unsigned long addr, unsigned long end, fault_flags |= FAULT_FLAG_WRITE; } - for (; addr < end; addr += PAGE_SIZE, i++) + for (; addr < end; addr += PAGE_SIZE) if (handle_mm_fault(vma, addr, fault_flags) & VM_FAULT_ERROR) - goto out_error; - + return -EFAULT; return -EBUSY; - -out_error: - pfns[i] = range->values[HMM_PFN_ERROR]; - return -EFAULT; } static unsigned int hmm_pte_need_fault(const struct hmm_vma_walk *hmm_vma_walk, @@ -291,7 +283,6 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, /* Report error for everything else */ pte_unmap(ptep); - *pfn = range->values[HMM_PFN_ERROR]; return -EFAULT; } @@ -577,9 +568,6 @@ static const struct mm_walk_ops hmm_walk_ops = { * * This is similar to get_user_pages(), except that it can read the page tables * without mutating them (ie causing faults). - * - * On error, for one virtual address in the range, the function will mark the - * corresponding HMM pfn entry with an error flag. */ long hmm_range_fault(struct hmm_range *range) { From patchwork Tue Mar 24 01:14:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11454385 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3F4B7139A for ; Tue, 24 Mar 2020 01:15:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 03E9A2070A for ; Tue, 24 Mar 2020 01:15:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="Ff0xojsd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 03E9A2070A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9767F6B0082; Mon, 23 Mar 2020 21:15:16 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 928256B0083; Mon, 23 Mar 2020 21:15:16 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 816AA6B0088; Mon, 23 Mar 2020 21:15:16 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0131.hostedemail.com [216.40.44.131]) by kanga.kvack.org (Postfix) with ESMTP id 619DA6B0082 for ; Mon, 23 Mar 2020 21:15:16 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 3DEAD186CB8E8 for ; Tue, 24 Mar 2020 01:15:16 +0000 (UTC) X-FDA: 76628487432.12.day06_4b17559c6495a X-Spam-Summary: 2,0,0,bf348a1b48a8afea,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2198:2199:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3867:3871:3872:3874:4250:4321:5007:6117:6120:6261:6653:7576:7875:7901:8660:9036:9592:10004:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12679:12895:13069:13148:13161:13229:13230:13311:13357:13894:14181:14384:14394:14721:21080:21325:21444:21627:21990:30054:30070,0,RBL:209.85.160.193:@ziepe.ca:.lbl8.mailshell.net-66.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: day06_4b17559c6495a X-Filterd-Recvd-Size: 4727 Received: from mail-qt1-f193.google.com (mail-qt1-f193.google.com [209.85.160.193]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Tue, 24 Mar 2020 01:15:15 +0000 (UTC) Received: by mail-qt1-f193.google.com with SMTP id 10so13715455qtp.1 for ; Mon, 23 Mar 2020 18:15:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7cn9RgeyUV+TJ0C+dZY3/xabajaX98Ex7kztQKlCq4U=; b=Ff0xojsd3cq1dGVAoIHiPmcyh54SxIN+pENdLICcphcOvOXKSYBGWi2Lteb40L9Fdb 0uGdZf/FNkjfyoGYDj7B2NMBVFvfdBCh04XT/0MbUWrs3JMGezoHddekjb0DbZA05DF2 MT2M/znLe/J9vrIEnYQoXAvCC+BlbhZLDymJWUW1b9UzLcBKDP3pzI9IA8i5FCNAq/zn e2yz5bVTvnX3es5DF+kAJ1XlXnh1j2im/pX3/bAQG+6Kz6y3oPSV5hudjoC8i5BZ6fPB yXL+ZujImYZTgg9ip8MWOZoP08JFOiha+6xXv2t1tUiHWNW1u2NHn6mu85+aoEe3fmI1 9mwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7cn9RgeyUV+TJ0C+dZY3/xabajaX98Ex7kztQKlCq4U=; b=c7lEP3Kmkla54/BP0E1Ke32CwgBZvcOrxAHkwxuKyfmRcxZd/COUKpMXkb5ToNiabb 5BuJVVES1d1RnsTaxAuuzUAb6l56DdT/MK4/niB2VO9gW+bBuJdi6ZY5Ztcoh3jJn07D kfPGWxWRUg9gfy8Cm5iajaKGoRywCA6D5QDvF6cyBomCS5XtxAhyK4p6gcpJlSiQ300U 5w/N17078yTE/UUAntFVxTridIkcQ9J05SnDQLflcaOLbdeuCxJWtSiMp0gLbzrIXoSt spzFe+AgeukD4O+UnfR3JztYCpa1zq4WK0cRDJSsT4OU+TyYGFdZ9PcFkHpgwIjCNbof 0Zag== X-Gm-Message-State: ANhLgQ3dhY5wIDabXuJ7iUklL7w2vsPLfeTCeFu/deseDt5Uhg3sGkBu 6azeh9gb2yGtynJ4qCtSkC0n7lrB/o32jw== X-Google-Smtp-Source: ADFU+vtPNzJGTGzAwJ8tfsuitTGBjOWdY4RK0mrS2DUXzOKGkcTcLMcpw9n2NbdPeT5u90/Amd8p6A== X-Received: by 2002:ac8:1196:: with SMTP id d22mr24536367qtj.344.1585012515289; Mon, 23 Mar 2020 18:15:15 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id o67sm11728381qka.114.2020.03.23.18.15.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Mar 2020 18:15:13 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jGY9u-0000ra-FL; Mon, 23 Mar 2020 22:15:10 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 9/9] mm/hmm: return error for non-vma snapshots Date: Mon, 23 Mar 2020 22:14:57 -0300 Message-Id: <20200324011457.2817-10-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200324011457.2817-1-jgg@ziepe.ca> References: <20200324011457.2817-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe The pagewalker does not call most ops with NULL vma, those are all routed to pte_hole instead. Thus hmm_vma_fault() is only called with a NULL vma from hmm_vma_walk_hole(), so hoist the check to there. Now it is clear that snapshotting with no vma is a HMM_PFN_ERROR as without a vma we have no path to call hmm_vma_fault(). Signed-off-by: Jason Gunthorpe Reviewed-by: Christoph Hellwig --- mm/hmm.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 14c33e1225866c..df0574061b37d3 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -83,9 +83,6 @@ static int hmm_vma_fault(unsigned long addr, unsigned long end, WARN_ON_ONCE(!required_fault); hmm_vma_walk->last = addr; - if (!vma) - return -EFAULT; - if ((required_fault & HMM_NEED_WRITE_FAULT) == HMM_NEED_WRITE_FAULT) { if (!(vma->vm_flags & VM_WRITE)) return -EPERM; @@ -175,6 +172,11 @@ static int hmm_vma_walk_hole(unsigned long addr, unsigned long end, npages = (end - addr) >> PAGE_SHIFT; pfns = &range->pfns[i]; required_fault = hmm_range_need_fault(hmm_vma_walk, pfns, npages, 0); + if (!walk->vma) { + if (required_fault) + return -EFAULT; + return hmm_pfns_fill(addr, end, range, HMM_PFN_ERROR); + } if (required_fault) return hmm_vma_fault(addr, end, required_fault, walk); hmm_vma_walk->last = addr;