From patchwork Fri Mar 27 20:00:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11463045 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C75571667 for ; Fri, 27 Mar 2020 20:00:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7D79220714 for ; Fri, 27 Mar 2020 20:00:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="B8VIl8Cs" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7D79220714 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2F5BD6B0032; Fri, 27 Mar 2020 16:00:30 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2A61E6B0036; Fri, 27 Mar 2020 16:00:30 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0F7AC6B0037; Fri, 27 Mar 2020 16:00:30 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0202.hostedemail.com [216.40.44.202]) by kanga.kvack.org (Postfix) with ESMTP id ED80D6B0036 for ; Fri, 27 Mar 2020 16:00:29 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9B734824805A for ; Fri, 27 Mar 2020 20:00:29 +0000 (UTC) X-FDA: 76642209420.10.tray62_2169234c8d350 X-Spam-Summary: 2,0,0,d8ae632558a0edb2,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1543:1711:1730:1747:1777:1792:2198:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3874:4117:4321:4605:5007:6119:6120:6261:6653:7576:7901:9036:9592:10004:11026:11658:11914:12043:12296:12297:12438:12517:12519:12555:12679:12683:12895:12986:13161:13229:13894:14181:14394:14721:14819:21080:21444:21451:21627:21990:30003:30054:30064:30070,0,RBL:209.85.222.193:@ziepe.ca:.lbl8.mailshell.net-66.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:31,LUA_SUMMARY:none X-HE-Tag: tray62_2169234c8d350 X-Filterd-Recvd-Size: 6977 Received: from mail-qk1-f193.google.com (mail-qk1-f193.google.com [209.85.222.193]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Fri, 27 Mar 2020 20:00:29 +0000 (UTC) Received: by mail-qk1-f193.google.com with SMTP id j4so12143863qkc.11 for ; Fri, 27 Mar 2020 13:00:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=l6FSIJA11GUWp0iQEXEKmA47YraWhMsGW7mGBig6TKg=; b=B8VIl8CsE0i+yz5ktETkDbP6oCOwB9jFF8uCJ5FsWSqFmwjzIX+ImPVf858PHxOngZ ar1UKDakP5k672+PK0UTM50jgjlop3Up/AAzRr6TcXWAf6PymK/aYrFCVX+YRsLSHbWm BWMdTTu4lzuUPsSXRrv1YIzdXO6RBckMWW+1LFkz/8bclf6GpGihBt3R+5ir6kcwRwT4 waTAzWqFFB7r/KeILUCDs+8sdRcwDLOUFMQx+DnVqZcxVqd475UoFYlunnDoT7OUmUWp 4FXzV+aeAYANqWe0j4RUopm83PCyP3jcmVpMUqI/YhPVOy6WF4QHlKb3XvRoKGBXj7kS E7sQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l6FSIJA11GUWp0iQEXEKmA47YraWhMsGW7mGBig6TKg=; b=fhRd4YCqSOhJeX0FuFiqzj6crzysIEqqrJh2Vrdg0dybUo2prFgPVFIuIRc/MGQ5hP DEG/u3xRUdPhz6+WKhmSv1uwltt7E2Wyn7RwpALBENQBTt42PcQRRJK4/2d+czSkupDl +4Zej6GsTuh94pUwjm6qF97HK0c0dRvRHC3dd3JSERrMe6qjhdAPlpZxp6b784VkO1kT YKuviQBcHS97CO+98xQ7XZnkT8U4f2QfNHfeqLdbs116l8ZM3PAOK5B8fKirNASb+2I3 B+LT1cEPTk5Qav4jz89rUytHc9OnqkDoqgaSJCcxLpapX1oIpNqQL4Buvk4f/3uwu/Yo 8mWQ== X-Gm-Message-State: ANhLgQ3BEythdPqedifTdYQP/rhQDXTW+2OpsRSyAVnvNbOAz6NcNsD0 ID/H7QwykLwlk9dee0np+WfmaA== X-Google-Smtp-Source: ADFU+vvf5lpmfE6Vv1KUN24OJu+znyEDbu+4t7Aq/PDfqWm3EIcAZ1pXP806ufmPt/PEbjgcavPE0g== X-Received: by 2002:a37:aec2:: with SMTP id x185mr1076168qke.45.1585339228406; Fri, 27 Mar 2020 13:00:28 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id j6sm4681752qti.25.2020.03.27.13.00.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 27 Mar 2020 13:00:25 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jHv9U-0007fT-UC; Fri, 27 Mar 2020 17:00:24 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 1/9] mm/hmm: remove pgmap checking for devmap pages Date: Fri, 27 Mar 2020 17:00:13 -0300 Message-Id: <20200327200021.29372-2-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200327200021.29372-1-jgg@ziepe.ca> References: <20200327200021.29372-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe The checking boils down to some racy check if the pagemap is still available or not. Instead of checking this, rely entirely on the notifiers, if a pagemap is destroyed then all pages that belong to it must be removed from the tables and the notifiers triggered. Reviewed-by: Ralph Campbell Reviewed-by: Christoph Hellwig Tested-by: Ralph Campbell Signed-off-by: Jason Gunthorpe --- mm/hmm.c | 50 ++------------------------------------------------ 1 file changed, 2 insertions(+), 48 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index a491d9aaafe45d..3a2610e0713329 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -28,7 +28,6 @@ struct hmm_vma_walk { struct hmm_range *range; - struct dev_pagemap *pgmap; unsigned long last; unsigned int flags; }; @@ -196,19 +195,8 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, return hmm_vma_fault(addr, end, fault, write_fault, walk); pfn = pmd_pfn(pmd) + ((addr & ~PMD_MASK) >> PAGE_SHIFT); - for (i = 0; addr < end; addr += PAGE_SIZE, i++, pfn++) { - if (pmd_devmap(pmd)) { - hmm_vma_walk->pgmap = get_dev_pagemap(pfn, - hmm_vma_walk->pgmap); - if (unlikely(!hmm_vma_walk->pgmap)) - return -EBUSY; - } + for (i = 0; addr < end; addr += PAGE_SIZE, i++, pfn++) pfns[i] = hmm_device_entry_from_pfn(range, pfn) | cpu_flags; - } - if (hmm_vma_walk->pgmap) { - put_dev_pagemap(hmm_vma_walk->pgmap); - hmm_vma_walk->pgmap = NULL; - } hmm_vma_walk->last = end; return 0; } @@ -300,15 +288,6 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, if (fault || write_fault) goto fault; - if (pte_devmap(pte)) { - hmm_vma_walk->pgmap = get_dev_pagemap(pte_pfn(pte), - hmm_vma_walk->pgmap); - if (unlikely(!hmm_vma_walk->pgmap)) { - pte_unmap(ptep); - return -EBUSY; - } - } - /* * Since each architecture defines a struct page for the zero page, just * fall through and treat it like a normal page. @@ -328,10 +307,6 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, return 0; fault: - if (hmm_vma_walk->pgmap) { - put_dev_pagemap(hmm_vma_walk->pgmap); - hmm_vma_walk->pgmap = NULL; - } pte_unmap(ptep); /* Fault any virtual address we were asked to fault */ return hmm_vma_fault(addr, end, fault, write_fault, walk); @@ -418,16 +393,6 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, return r; } } - if (hmm_vma_walk->pgmap) { - /* - * We do put_dev_pagemap() here and not in hmm_vma_handle_pte() - * so that we can leverage get_dev_pagemap() optimization which - * will not re-take a reference on a pgmap if we already have - * one. - */ - put_dev_pagemap(hmm_vma_walk->pgmap); - hmm_vma_walk->pgmap = NULL; - } pte_unmap(ptep - 1); hmm_vma_walk->last = addr; @@ -491,20 +456,9 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end, } pfn = pud_pfn(pud) + ((addr & ~PUD_MASK) >> PAGE_SHIFT); - for (i = 0; i < npages; ++i, ++pfn) { - hmm_vma_walk->pgmap = get_dev_pagemap(pfn, - hmm_vma_walk->pgmap); - if (unlikely(!hmm_vma_walk->pgmap)) { - ret = -EBUSY; - goto out_unlock; - } + for (i = 0; i < npages; ++i, ++pfn) pfns[i] = hmm_device_entry_from_pfn(range, pfn) | cpu_flags; - } - if (hmm_vma_walk->pgmap) { - put_dev_pagemap(hmm_vma_walk->pgmap); - hmm_vma_walk->pgmap = NULL; - } hmm_vma_walk->last = end; goto out_unlock; } From patchwork Fri Mar 27 20:00:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11463043 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 13E7C81 for ; Fri, 27 Mar 2020 20:00:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BAD9F2071B for ; Fri, 27 Mar 2020 20:00:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="H1W0eveT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BAD9F2071B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D55BB6B0010; Fri, 27 Mar 2020 16:00:29 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D06B86B0032; Fri, 27 Mar 2020 16:00:29 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BCEA56B0036; Fri, 27 Mar 2020 16:00:29 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0154.hostedemail.com [216.40.44.154]) by kanga.kvack.org (Postfix) with ESMTP id A6E0E6B0010 for ; Fri, 27 Mar 2020 16:00:29 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C8C3C6110 for ; Fri, 27 Mar 2020 20:00:29 +0000 (UTC) X-FDA: 76642209378.26.note27_214d709dea357 X-Spam-Summary: 2,0,0,55631fab62805067,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:4:41:69:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1605:1730:1747:1777:1792:2198:2199:2393:2559:2562:2640:2898:2899:2901:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4250:4321:4605:5007:6117:6119:6120:6261:6653:7576:7875:7901:7903:9036:9592:10226:11026:11232:11233:11473:11658:11914:12043:12114:12291:12296:12297:12438:12517:12519:12555:12679:12683:12895:13894:14096:14110:14394:21080:21433:21444:21451:21611:21627:21796:21990:30003:30012:30036:30054:30070,0,RBL:209.85.160.194:@ziepe.ca:.lbl8.mailshell.net-62.14.0.100 66.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: note27_214d709dea357 X-Filterd-Recvd-Size: 17407 Received: from mail-qt1-f194.google.com (mail-qt1-f194.google.com [209.85.160.194]) by imf17.hostedemail.com (Postfix) with ESMTP for ; Fri, 27 Mar 2020 20:00:28 +0000 (UTC) Received: by mail-qt1-f194.google.com with SMTP id c9so9671580qtw.7 for ; Fri, 27 Mar 2020 13:00:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=q0OsabNYqQz9XezfUAjPoyuspZFWQcrY4Mhi6eg0HzY=; b=H1W0eveTDYFp9Me6A7/gA4rO/fywX4Lkj9lBPMfHlCm+QsCrBoc31mD4j7V3cQcAXh QOlQIvesfCTUyHo4J/1oAR4ksHK6O8H3SiXtz4NlIC/TQGNktGUOdjWIBn5JmLTTYgoN eTRYLQJRFW7bBXZ7IZ9KtyUBg60ZAYHc/jJn2QTkGvwjGIJQCfiBpnHRcbfyCeuXBqnx L+HxsN697K3iOPQWULz54HPHfQrAnVGG2YSoD1aJBkGpYxGqMJvLAJXPbTyYQGQFPobh J4uQhJF9Kk+QVN+wrbqTw0cj/TtOGyv71hmd15wH5Evbq7IdTyhDfza2i1rCJ/aRLHZv 1GTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=q0OsabNYqQz9XezfUAjPoyuspZFWQcrY4Mhi6eg0HzY=; b=Wxkm2Z12ZZg4Txs8rVR6VXMCYdBUD2ZPR1AHzN6f6lvnerYfe7H5+Au84+m9eI4wq5 FTikmcWkyi4W2P+5VeFsSxajC2lOjwVlfbytGcKXv/rMIN1XJ1qRfnymmOxvQGmc0I7r 3nEAmFt8uHGcJju8MnTa3vGbxUu1Hklo6vSlFmjVZ6II57BVRZpX/g+nxN7ChfkQybQB +V79PSccHHWIHtvrU7f0WPr0Lq5HZ4jxb0AusBRCqlRrlFubMkC3IPQdNuy2I72ln59+ IvyyDbPik6vF+PirCeJQeTDEP2IDqhxGpwh5cf4pyyYXrhVhmZPYNsLrgbRH1tbkYfy9 owdQ== X-Gm-Message-State: ANhLgQ08i2ohKJM//4vgmdeP6kQvc4lO0smO6gLw8JLUDIdLmSNNMJzC XrKa7x0XN+vT1roJn2yT5ssq0g== X-Google-Smtp-Source: ADFU+vv1KEJUNoOTFPYi7PUdKmoRKZ930ogjB3Gxy2qwLSxuhnLwJkV54eLdQAnPUsAEMzUP3D5Ziw== X-Received: by 2002:ac8:4d06:: with SMTP id w6mr956299qtv.287.1585339227895; Fri, 27 Mar 2020 13:00:27 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id m67sm4357086qke.101.2020.03.27.13.00.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 27 Mar 2020 13:00:25 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jHv9V-0007fZ-0I; Fri, 27 Mar 2020 17:00:25 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 2/9] mm/hmm: return the fault type from hmm_pte_need_fault() Date: Fri, 27 Mar 2020 17:00:14 -0300 Message-Id: <20200327200021.29372-3-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200327200021.29372-1-jgg@ziepe.ca> References: <20200327200021.29372-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe Using two bools instead of flags return is not necessary and leads to bugs. Returning a value is easier for the compiler to check and easier to pass around the code flow. Convert the two bools into flags and push the change to all callers. Signed-off-by: Jason Gunthorpe Reviewed-by: Christoph Hellwig --- mm/hmm.c | 183 ++++++++++++++++++++++++------------------------------- 1 file changed, 81 insertions(+), 102 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 3a2610e0713329..d208ddd351066f 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -32,6 +32,12 @@ struct hmm_vma_walk { unsigned int flags; }; +enum { + HMM_NEED_FAULT = 1 << 0, + HMM_NEED_WRITE_FAULT = 1 << 1, + HMM_NEED_ALL_BITS = HMM_NEED_FAULT | HMM_NEED_WRITE_FAULT, +}; + static int hmm_pfns_fill(unsigned long addr, unsigned long end, struct hmm_range *range, enum hmm_pfn_value_e value) { @@ -49,8 +55,7 @@ static int hmm_pfns_fill(unsigned long addr, unsigned long end, * hmm_vma_fault() - fault in a range lacking valid pmd or pte(s) * @addr: range virtual start address (inclusive) * @end: range virtual end address (exclusive) - * @fault: should we fault or not ? - * @write_fault: write fault ? + * @required_fault: HMM_NEED_* flags * @walk: mm_walk structure * Return: -EBUSY after page fault, or page fault error * @@ -58,8 +63,7 @@ static int hmm_pfns_fill(unsigned long addr, unsigned long end, * or whenever there is no page directory covering the virtual address range. */ static int hmm_vma_fault(unsigned long addr, unsigned long end, - bool fault, bool write_fault, - struct mm_walk *walk) + unsigned int required_fault, struct mm_walk *walk) { struct hmm_vma_walk *hmm_vma_walk = walk->private; struct hmm_range *range = hmm_vma_walk->range; @@ -68,13 +72,13 @@ static int hmm_vma_fault(unsigned long addr, unsigned long end, unsigned long i = (addr - range->start) >> PAGE_SHIFT; unsigned int fault_flags = FAULT_FLAG_REMOTE; - WARN_ON_ONCE(!fault && !write_fault); + WARN_ON_ONCE(!required_fault); hmm_vma_walk->last = addr; if (!vma) goto out_error; - if (write_fault) { + if (required_fault & HMM_NEED_WRITE_FAULT) { if (!(vma->vm_flags & VM_WRITE)) return -EPERM; fault_flags |= FAULT_FLAG_WRITE; @@ -91,14 +95,13 @@ static int hmm_vma_fault(unsigned long addr, unsigned long end, return -EFAULT; } -static inline void hmm_pte_need_fault(const struct hmm_vma_walk *hmm_vma_walk, - uint64_t pfns, uint64_t cpu_flags, - bool *fault, bool *write_fault) +static unsigned int hmm_pte_need_fault(const struct hmm_vma_walk *hmm_vma_walk, + uint64_t pfns, uint64_t cpu_flags) { struct hmm_range *range = hmm_vma_walk->range; if (hmm_vma_walk->flags & HMM_FAULT_SNAPSHOT) - return; + return 0; /* * So we not only consider the individual per page request we also @@ -114,37 +117,37 @@ static inline void hmm_pte_need_fault(const struct hmm_vma_walk *hmm_vma_walk, /* We aren't ask to do anything ... */ if (!(pfns & range->flags[HMM_PFN_VALID])) - return; + return 0; - /* If CPU page table is not valid then we need to fault */ - *fault = !(cpu_flags & range->flags[HMM_PFN_VALID]); /* Need to write fault ? */ if ((pfns & range->flags[HMM_PFN_WRITE]) && - !(cpu_flags & range->flags[HMM_PFN_WRITE])) { - *write_fault = true; - *fault = true; - } + !(cpu_flags & range->flags[HMM_PFN_WRITE])) + return HMM_NEED_FAULT | HMM_NEED_WRITE_FAULT; + + /* If CPU page table is not valid then we need to fault */ + if (!(cpu_flags & range->flags[HMM_PFN_VALID])) + return HMM_NEED_FAULT; + return 0; } -static void hmm_range_need_fault(const struct hmm_vma_walk *hmm_vma_walk, - const uint64_t *pfns, unsigned long npages, - uint64_t cpu_flags, bool *fault, - bool *write_fault) +static unsigned int +hmm_range_need_fault(const struct hmm_vma_walk *hmm_vma_walk, + const uint64_t *pfns, unsigned long npages, + uint64_t cpu_flags) { + unsigned int required_fault = 0; unsigned long i; - if (hmm_vma_walk->flags & HMM_FAULT_SNAPSHOT) { - *fault = *write_fault = false; - return; - } + if (hmm_vma_walk->flags & HMM_FAULT_SNAPSHOT) + return 0; - *fault = *write_fault = false; for (i = 0; i < npages; ++i) { - hmm_pte_need_fault(hmm_vma_walk, pfns[i], cpu_flags, - fault, write_fault); - if ((*write_fault)) - return; + required_fault |= + hmm_pte_need_fault(hmm_vma_walk, pfns[i], cpu_flags); + if (required_fault == HMM_NEED_ALL_BITS) + return required_fault; } + return required_fault; } static int hmm_vma_walk_hole(unsigned long addr, unsigned long end, @@ -152,17 +155,16 @@ static int hmm_vma_walk_hole(unsigned long addr, unsigned long end, { struct hmm_vma_walk *hmm_vma_walk = walk->private; struct hmm_range *range = hmm_vma_walk->range; - bool fault, write_fault; + unsigned int required_fault; unsigned long i, npages; uint64_t *pfns; i = (addr - range->start) >> PAGE_SHIFT; npages = (end - addr) >> PAGE_SHIFT; pfns = &range->pfns[i]; - hmm_range_need_fault(hmm_vma_walk, pfns, npages, - 0, &fault, &write_fault); - if (fault || write_fault) - return hmm_vma_fault(addr, end, fault, write_fault, walk); + required_fault = hmm_range_need_fault(hmm_vma_walk, pfns, npages, 0); + if (required_fault) + return hmm_vma_fault(addr, end, required_fault, walk); hmm_vma_walk->last = addr; return hmm_pfns_fill(addr, end, range, HMM_PFN_NONE); } @@ -183,16 +185,15 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, struct hmm_vma_walk *hmm_vma_walk = walk->private; struct hmm_range *range = hmm_vma_walk->range; unsigned long pfn, npages, i; - bool fault, write_fault; + unsigned int required_fault; uint64_t cpu_flags; npages = (end - addr) >> PAGE_SHIFT; cpu_flags = pmd_to_hmm_pfn_flags(range, pmd); - hmm_range_need_fault(hmm_vma_walk, pfns, npages, cpu_flags, - &fault, &write_fault); - - if (fault || write_fault) - return hmm_vma_fault(addr, end, fault, write_fault, walk); + required_fault = + hmm_range_need_fault(hmm_vma_walk, pfns, npages, cpu_flags); + if (required_fault) + return hmm_vma_fault(addr, end, required_fault, walk); pfn = pmd_pfn(pmd) + ((addr & ~PMD_MASK) >> PAGE_SHIFT); for (i = 0; addr < end; addr += PAGE_SIZE, i++, pfn++) @@ -229,18 +230,15 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, { struct hmm_vma_walk *hmm_vma_walk = walk->private; struct hmm_range *range = hmm_vma_walk->range; - bool fault, write_fault; + unsigned int required_fault; uint64_t cpu_flags; pte_t pte = *ptep; uint64_t orig_pfn = *pfn; *pfn = range->values[HMM_PFN_NONE]; - fault = write_fault = false; - if (pte_none(pte)) { - hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0, - &fault, &write_fault); - if (fault || write_fault) + required_fault = hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0); + if (required_fault) goto fault; return 0; } @@ -261,9 +259,8 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, return 0; } - hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0, &fault, - &write_fault); - if (!fault && !write_fault) + required_fault = hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0); + if (!required_fault) return 0; if (!non_swap_entry(entry)) @@ -283,9 +280,8 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, } cpu_flags = pte_to_hmm_pfn_flags(range, pte); - hmm_pte_need_fault(hmm_vma_walk, orig_pfn, cpu_flags, &fault, - &write_fault); - if (fault || write_fault) + required_fault = hmm_pte_need_fault(hmm_vma_walk, orig_pfn, cpu_flags); + if (required_fault) goto fault; /* @@ -293,9 +289,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, * fall through and treat it like a normal page. */ if (pte_special(pte) && !is_zero_pfn(pte_pfn(pte))) { - hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0, &fault, - &write_fault); - if (fault || write_fault) { + if (hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0)) { pte_unmap(ptep); return -EFAULT; } @@ -309,7 +303,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, fault: pte_unmap(ptep); /* Fault any virtual address we were asked to fault */ - return hmm_vma_fault(addr, end, fault, write_fault, walk); + return hmm_vma_fault(addr, end, required_fault, walk); } static int hmm_vma_walk_pmd(pmd_t *pmdp, @@ -322,7 +316,6 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, uint64_t *pfns = &range->pfns[(start - range->start) >> PAGE_SHIFT]; unsigned long npages = (end - start) >> PAGE_SHIFT; unsigned long addr = start; - bool fault, write_fault; pte_t *ptep; pmd_t pmd; @@ -332,9 +325,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, return hmm_vma_walk_hole(start, end, -1, walk); if (thp_migration_supported() && is_pmd_migration_entry(pmd)) { - hmm_range_need_fault(hmm_vma_walk, pfns, npages, - 0, &fault, &write_fault); - if (fault || write_fault) { + if (hmm_range_need_fault(hmm_vma_walk, pfns, npages, 0)) { hmm_vma_walk->last = addr; pmd_migration_entry_wait(walk->mm, pmdp); return -EBUSY; @@ -343,9 +334,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, } if (!pmd_present(pmd)) { - hmm_range_need_fault(hmm_vma_walk, pfns, npages, 0, &fault, - &write_fault); - if (fault || write_fault) + if (hmm_range_need_fault(hmm_vma_walk, pfns, npages, 0)) return -EFAULT; return hmm_pfns_fill(start, end, range, HMM_PFN_ERROR); } @@ -375,9 +364,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, * recover. */ if (pmd_bad(pmd)) { - hmm_range_need_fault(hmm_vma_walk, pfns, npages, 0, &fault, - &write_fault); - if (fault || write_fault) + if (hmm_range_need_fault(hmm_vma_walk, pfns, npages, 0)) return -EFAULT; return hmm_pfns_fill(start, end, range, HMM_PFN_ERROR); } @@ -434,8 +421,8 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end, if (pud_huge(pud) && pud_devmap(pud)) { unsigned long i, npages, pfn; + unsigned int required_fault; uint64_t *pfns, cpu_flags; - bool fault, write_fault; if (!pud_present(pud)) { spin_unlock(ptl); @@ -447,12 +434,11 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end, pfns = &range->pfns[i]; cpu_flags = pud_to_hmm_pfn_flags(range, pud); - hmm_range_need_fault(hmm_vma_walk, pfns, npages, - cpu_flags, &fault, &write_fault); - if (fault || write_fault) { + required_fault = hmm_range_need_fault(hmm_vma_walk, pfns, + npages, cpu_flags); + if (required_fault) { spin_unlock(ptl); - return hmm_vma_fault(addr, end, fault, write_fault, - walk); + return hmm_vma_fault(addr, end, required_fault, walk); } pfn = pud_pfn(pud) + ((addr & ~PUD_MASK) >> PAGE_SHIFT); @@ -484,7 +470,7 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, struct hmm_range *range = hmm_vma_walk->range; struct vm_area_struct *vma = walk->vma; uint64_t orig_pfn, cpu_flags; - bool fault, write_fault; + unsigned int required_fault; spinlock_t *ptl; pte_t entry; @@ -495,12 +481,10 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, orig_pfn = range->pfns[i]; range->pfns[i] = range->values[HMM_PFN_NONE]; cpu_flags = pte_to_hmm_pfn_flags(range, entry); - fault = write_fault = false; - hmm_pte_need_fault(hmm_vma_walk, orig_pfn, cpu_flags, - &fault, &write_fault); - if (fault || write_fault) { + required_fault = hmm_pte_need_fault(hmm_vma_walk, orig_pfn, cpu_flags); + if (required_fault) { spin_unlock(ptl); - return hmm_vma_fault(addr, end, fault, write_fault, walk); + return hmm_vma_fault(addr, end, required_fault, walk); } pfn = pte_pfn(entry) + ((start & ~hmask) >> PAGE_SHIFT); @@ -522,37 +506,32 @@ static int hmm_vma_walk_test(unsigned long start, unsigned long end, struct hmm_range *range = hmm_vma_walk->range; struct vm_area_struct *vma = walk->vma; + if (!(vma->vm_flags & (VM_IO | VM_PFNMAP | VM_MIXEDMAP)) && + vma->vm_flags & VM_READ) + return 0; + /* - * Skip vma ranges that don't have struct page backing them or map I/O - * devices directly. + * vma ranges that don't have struct page backing them or map I/O + * devices directly cannot be handled by hmm_range_fault(). * * If the vma does not allow read access, then assume that it does not * allow write access either. HMM does not support architectures that * allow write without read. + * + * If a fault is requested for an unsupported range then it is a hard + * failure. */ - if ((vma->vm_flags & (VM_IO | VM_PFNMAP | VM_MIXEDMAP)) || - !(vma->vm_flags & VM_READ)) { - bool fault, write_fault; - - /* - * Check to see if a fault is requested for any page in the - * range. - */ - hmm_range_need_fault(hmm_vma_walk, range->pfns + - ((start - range->start) >> PAGE_SHIFT), - (end - start) >> PAGE_SHIFT, - 0, &fault, &write_fault); - if (fault || write_fault) - return -EFAULT; - - hmm_pfns_fill(start, end, range, HMM_PFN_ERROR); - hmm_vma_walk->last = end; + if (hmm_range_need_fault(hmm_vma_walk, + range->pfns + + ((start - range->start) >> PAGE_SHIFT), + (end - start) >> PAGE_SHIFT, 0)) + return -EFAULT; - /* Skip this vma and continue processing the next vma. */ - return 1; - } + hmm_pfns_fill(start, end, range, HMM_PFN_ERROR); + hmm_vma_walk->last = end; - return 0; + /* Skip this vma and continue processing the next vma. */ + return 1; } static const struct mm_walk_ops hmm_walk_ops = { From patchwork Fri Mar 27 20:00:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11463051 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 343191667 for ; Fri, 27 Mar 2020 20:00:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D79BB20714 for ; Fri, 27 Mar 2020 20:00:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="gvMUeP/x" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D79BB20714 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A49796B006E; Fri, 27 Mar 2020 16:00:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9D2116B0070; Fri, 27 Mar 2020 16:00:33 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 89A0E6B0071; Fri, 27 Mar 2020 16:00:33 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0251.hostedemail.com [216.40.44.251]) by kanga.kvack.org (Postfix) with ESMTP id 721CF6B006E for ; Fri, 27 Mar 2020 16:00:33 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 3412153B4 for ; Fri, 27 Mar 2020 20:00:33 +0000 (UTC) X-FDA: 76642209546.05.farm38_21e109141ce25 X-Spam-Summary: 2,0,0,30c2b898542a12d5,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:1:2:41:69:152:355:379:541:800:960:973:982:988:989:1260:1277:1311:1313:1314:1345:1359:1437:1515:1516:1518:1593:1594:1605:1730:1747:1777:1792:2393:2538:2559:2562:2693:2895:2897:2901:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3873:3874:4051:4250:4321:4418:4605:5007:6117:6119:6120:6261:6653:7576:7875:7901:7903:9010:9036:9592:10004:10226:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12679:12683:12895:12986:13153:13228:13894:13972:14110:14394:14659:21080:21222:21324:21325:21433:21444:21451:21627:21990:30003:30012:30034:30045:30054:30067:30070:30079,0,RBL:209.85.219.66:@ziepe.ca:.lbl8.mailshell.net-66.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: farm38_21e109141ce25 X-Filterd-Recvd-Size: 12075 Received: from mail-qv1-f66.google.com (mail-qv1-f66.google.com [209.85.219.66]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Fri, 27 Mar 2020 20:00:32 +0000 (UTC) Received: by mail-qv1-f66.google.com with SMTP id v38so5551147qvf.6 for ; Fri, 27 Mar 2020 13:00:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eqAE2/msh/qEqKYZ/hz4L584EHRPwSmY4keaNvv7V8U=; b=gvMUeP/xzHcWI8xCXcJ8boM5h9AB4d2j3PspbUWXR62KrmAw+NhWi/gp5YLNjD22Qd yfqKPcMI7EppCOphzADKs5KkHqfyYF2UqexMdlfvS+NwnixLC9XNpBeKzTbQpBRIWRcF s4bDcNKS0LmwuWSuakp+PhkiwN/q8ON21XZ6RLOo4XnSayxKiHpHAWdm377ZqBzX1eHA gKI4jeg6tNRnXBemm7R422WZDpETH0fWUPEoKZOKvW36s2YR9r1lz0nzMS/RBRPcOeZb UIfO0VUflSIOQb1XXeuPUSj77Txxup221fHKsdZtLMdQ0ICsn9KL6rBJETYQagpia66u PE6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eqAE2/msh/qEqKYZ/hz4L584EHRPwSmY4keaNvv7V8U=; b=oHyVRrfZXK8ZC6VzmkGFKAZ3PM5h9JJcyEip/0DeoRqVDO5lKHhNh6CmpM7sj2dUP5 aNyUUG1v4Biz5CZjTKp4XcVDvW7/KZ/103LUVw2DxpLoGComI7L0yBakPgknpcmrbRvO dIwuDuoJS/5aRgGPscrJWlx+PLvGZIZslGi5I6Yl7Fszr8iHmXJ4XGj18rlNdEtXQPg1 DDHaYiehNT/ntApcjHy6ZMCdiYYDwaSZS9d/GdNGPQHTMZXmaPh4QNV+B7luPn0vCkBO VjcEuDtTR4IbxviHiUIkre9TPd+dXM6WPbAqgH6X6AwAYRcjg19RYOAfd3I1BnzE77E5 t0Ig== X-Gm-Message-State: ANhLgQ0ZzDO2jCgCJFsnfH8xcul0nNJhERVQDwFl+LwnvKK32vSnqRea brgU5M7hPwUXgCRjC6wIfkN4UA== X-Google-Smtp-Source: ADFU+vtnROEARIJwDxp6WQO0XL7PTsfDOJP4gnhVeWRF8Ec6Gka/86+B1uYUbuiN5vC2U3jmsnbTpg== X-Received: by 2002:a0c:fe04:: with SMTP id x4mr933306qvr.69.1585339231994; Fri, 27 Mar 2020 13:00:31 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id r29sm4804265qtj.76.2020.03.27.13.00.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 27 Mar 2020 13:00:25 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jHv9V-0007ff-1c; Fri, 27 Mar 2020 17:00:25 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 3/9] mm/hmm: remove unused code and tidy comments Date: Fri, 27 Mar 2020 17:00:15 -0300 Message-Id: <20200327200021.29372-4-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200327200021.29372-1-jgg@ziepe.ca> References: <20200327200021.29372-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe Delete several functions that are never called, fix some desync between comments and structure content, toss the now out of date top of file header, and move one function only used by hmm.c into hmm.c Reviewed-by: Christoph Hellwig Signed-off-by: Jason Gunthorpe --- include/linux/hmm.h | 104 +------------------------------------------- mm/hmm.c | 24 +++++++--- 2 files changed, 19 insertions(+), 109 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index bb6be4428633a8..daee6508a3f609 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -3,58 +3,8 @@ * Copyright 2013 Red Hat Inc. * * Authors: Jérôme Glisse - */ -/* - * Heterogeneous Memory Management (HMM) - * - * See Documentation/vm/hmm.rst for reasons and overview of what HMM is and it - * is for. Here we focus on the HMM API description, with some explanation of - * the underlying implementation. - * - * Short description: HMM provides a set of helpers to share a virtual address - * space between CPU and a device, so that the device can access any valid - * address of the process (while still obeying memory protection). HMM also - * provides helpers to migrate process memory to device memory, and back. Each - * set of functionality (address space mirroring, and migration to and from - * device memory) can be used independently of the other. - * - * - * HMM address space mirroring API: - * - * Use HMM address space mirroring if you want to mirror a range of the CPU - * page tables of a process into a device page table. Here, "mirror" means "keep - * synchronized". Prerequisites: the device must provide the ability to write- - * protect its page tables (at PAGE_SIZE granularity), and must be able to - * recover from the resulting potential page faults. * - * HMM guarantees that at any point in time, a given virtual address points to - * either the same memory in both CPU and device page tables (that is: CPU and - * device page tables each point to the same pages), or that one page table (CPU - * or device) points to no entry, while the other still points to the old page - * for the address. The latter case happens when the CPU page table update - * happens first, and then the update is mirrored over to the device page table. - * This does not cause any issue, because the CPU page table cannot start - * pointing to a new page until the device page table is invalidated. - * - * HMM uses mmu_notifiers to monitor the CPU page tables, and forwards any - * updates to each device driver that has registered a mirror. It also provides - * some API calls to help with taking a snapshot of the CPU page table, and to - * synchronize with any updates that might happen concurrently. - * - * - * HMM migration to and from device memory: - * - * HMM provides a set of helpers to hotplug device memory as ZONE_DEVICE, with - * a new MEMORY_DEVICE_PRIVATE type. This provides a struct page for each page - * of the device memory, and allows the device driver to manage its memory - * using those struct pages. Having struct pages for device memory makes - * migration easier. Because that memory is not addressable by the CPU it must - * never be pinned to the device; in other words, any CPU page fault can always - * cause the device memory to be migrated (copied/moved) back to regular memory. - * - * A new migrate helper (migrate_vma()) has been added (see mm/migrate.c) that - * allows use of a device DMA engine to perform the copy operation between - * regular system memory and device memory. + * See Documentation/vm/hmm.rst for reasons and overview of what HMM is. */ #ifndef LINUX_HMM_H #define LINUX_HMM_H @@ -120,9 +70,6 @@ enum hmm_pfn_value_e { * * @notifier: a mmu_interval_notifier that includes the start/end * @notifier_seq: result of mmu_interval_read_begin() - * @hmm: the core HMM structure this range is active against - * @vma: the vm area struct for the range - * @list: all range lock are on a list * @start: range virtual start address (inclusive) * @end: range virtual end address (exclusive) * @pfns: array of pfns (big enough for the range) @@ -130,8 +77,7 @@ enum hmm_pfn_value_e { * @values: pfn value for some special case (none, special, error, ...) * @default_flags: default flags for the range (write, read, ... see hmm doc) * @pfn_flags_mask: allows to mask pfn flags so that only default_flags matter - * @pfn_shifts: pfn shift value (should be <= PAGE_SHIFT) - * @valid: pfns array did not change since it has been fill by an HMM function + * @pfn_shift: pfn shift value (should be <= PAGE_SHIFT) * @dev_private_owner: owner of device private pages */ struct hmm_range { @@ -171,52 +117,6 @@ static inline struct page *hmm_device_entry_to_page(const struct hmm_range *rang return pfn_to_page(entry >> range->pfn_shift); } -/* - * hmm_device_entry_to_pfn() - return pfn value store in a device entry - * @range: range use to decode device entry value - * @entry: device entry to extract pfn from - * Return: pfn value if device entry is valid, -1UL otherwise - */ -static inline unsigned long -hmm_device_entry_to_pfn(const struct hmm_range *range, uint64_t pfn) -{ - if (pfn == range->values[HMM_PFN_NONE]) - return -1UL; - if (pfn == range->values[HMM_PFN_ERROR]) - return -1UL; - if (pfn == range->values[HMM_PFN_SPECIAL]) - return -1UL; - if (!(pfn & range->flags[HMM_PFN_VALID])) - return -1UL; - return (pfn >> range->pfn_shift); -} - -/* - * hmm_device_entry_from_page() - create a valid device entry for a page - * @range: range use to encode HMM pfn value - * @page: page for which to create the device entry - * Return: valid device entry for the page - */ -static inline uint64_t hmm_device_entry_from_page(const struct hmm_range *range, - struct page *page) -{ - return (page_to_pfn(page) << range->pfn_shift) | - range->flags[HMM_PFN_VALID]; -} - -/* - * hmm_device_entry_from_pfn() - create a valid device entry value from pfn - * @range: range use to encode HMM pfn value - * @pfn: pfn value for which to create the device entry - * Return: valid device entry for the pfn - */ -static inline uint64_t hmm_device_entry_from_pfn(const struct hmm_range *range, - unsigned long pfn) -{ - return (pfn << range->pfn_shift) | - range->flags[HMM_PFN_VALID]; -} - /* Don't fault in missing PTEs, just snapshot the current state. */ #define HMM_FAULT_SNAPSHOT (1 << 1) diff --git a/mm/hmm.c b/mm/hmm.c index d208ddd351066f..136de474221d77 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -38,6 +38,18 @@ enum { HMM_NEED_ALL_BITS = HMM_NEED_FAULT | HMM_NEED_WRITE_FAULT, }; +/* + * hmm_device_entry_from_pfn() - create a valid device entry value from pfn + * @range: range use to encode HMM pfn value + * @pfn: pfn value for which to create the device entry + * Return: valid device entry for the pfn + */ +static uint64_t hmm_device_entry_from_pfn(const struct hmm_range *range, + unsigned long pfn) +{ + return (pfn << range->pfn_shift) | range->flags[HMM_PFN_VALID]; +} + static int hmm_pfns_fill(unsigned long addr, unsigned long end, struct hmm_range *range, enum hmm_pfn_value_e value) { @@ -544,7 +556,7 @@ static const struct mm_walk_ops hmm_walk_ops = { /** * hmm_range_fault - try to fault some address in a virtual address range - * @range: range being faulted + * @range: argument structure * @flags: HMM_FAULT_* flags * * Return: the number of valid pages in range->pfns[] (from range start @@ -558,13 +570,11 @@ static const struct mm_walk_ops hmm_walk_ops = { * only). * -EBUSY: The range has been invalidated and the caller needs to wait for * the invalidation to finish. - * -EFAULT: Invalid (i.e., either no valid vma or it is illegal to access - * that range) number of valid pages in range->pfns[] (from - * range start address). + * -EFAULT: A page was requested to be valid and could not be made valid + * ie it has no backing VMA or it is illegal to access * - * This is similar to a regular CPU page fault except that it will not trigger - * any memory migration if the memory being faulted is not accessible by CPUs - * and caller does not ask for migration. + * This is similar to get_user_pages(), except that it can read the page tables + * without mutating them (ie causing faults). * * On error, for one virtual address in the range, the function will mark the * corresponding HMM pfn entry with an error flag. From patchwork Fri Mar 27 20:00:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11463047 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D9FE11667 for ; Fri, 27 Mar 2020 20:00:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 99C9F20714 for ; Fri, 27 Mar 2020 20:00:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="mnfkrC9j" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 99C9F20714 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D57C56B0036; Fri, 27 Mar 2020 16:00:30 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C6BB66B0037; Fri, 27 Mar 2020 16:00:30 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B329F6B006C; Fri, 27 Mar 2020 16:00:30 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0139.hostedemail.com [216.40.44.139]) by kanga.kvack.org (Postfix) with ESMTP id 973206B0036 for ; Fri, 27 Mar 2020 16:00:30 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id BAB3C4DD6 for ; Fri, 27 Mar 2020 20:00:30 +0000 (UTC) X-FDA: 76642209420.02.error88_217313795a44e X-Spam-Summary: 2,0,0,66d29656b1e1c730,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:2:41:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1605:1730:1747:1777:1792:1801:1981:2194:2198:2199:2200:2393:2553:2559:2562:2693:2895:2901:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4049:4120:4250:4321:4605:5007:6117:6119:6120:6261:6653:7576:7875:7901:7903:9036:9592:10004:11026:11232:11473:11657:11658:11914:12043:12295:12296:12297:12438:12517:12519:12555:12679:12895:12986:13149:13161:13229:13230:13894:14394:21063:21080:21324:21325:21433:21444:21451:21627:21990:30003:30012:30034:30045:30046:30054:30070:30079:30090,0,RBL:209.85.219.68:@ziepe.ca:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:28,LUA_SUMMARY:none X-HE-Tag: error88_217313795a44e X-Filterd-Recvd-Size: 9873 Received: from mail-qv1-f68.google.com (mail-qv1-f68.google.com [209.85.219.68]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Fri, 27 Mar 2020 20:00:29 +0000 (UTC) Received: by mail-qv1-f68.google.com with SMTP id t4so1353812qvz.8 for ; Fri, 27 Mar 2020 13:00:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0YwvFUjpXmbDvkZKNjbfujorytQMVs2tZT8OA9dQSF8=; b=mnfkrC9jyg4WaIQ779ig/zpj/klGfASY58WPazK78dL6/L9j0dM8LoArrmhGY6hNCW lPnRXcLdic1Pg+h7moGy18icvpcDp3Vdm0s1rq5HeRIJN9EcdUO/cu+co11JpoLrN4qL bBHI8AGZyTfnUTwrsVFc8uctNW7ix9eQCONaIedTp9BNTFhNh1EZyXYoel4HYfcRY8fw LQnPSoYQG3lmTytyBJMIt2zxl6duN/WkpaQwOOPuKfTg+pln0nPKRe46xyyeOCl2jNsw F64wdb95HRMI5lsN23ls+CRwYXLCXqbVVixTp0hUCTtYyIqfegAdfesSO2XZetjZvnNA JZFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0YwvFUjpXmbDvkZKNjbfujorytQMVs2tZT8OA9dQSF8=; b=O3uc0rnA+h9bZRAIHu8cfjpRWct1kkD1jk5jyRTEgBAEIwh+YTqM7/aCHThVX+3/pU E5V9Gb1Z0ZKiUVv20V5wuV9eYpeCZXuXOxECoWaBT+M/hHD0V69V24dnOi/2bj5Vv8oY ITI/U01hjWsNzD3/sNPdLqkptz5FutLZnOegAb7AL+IH9SCJ9DaEoDuBsOA+zdaVlh38 cD/gYJs9GZ9btq5YerDBeBDnMjFGy7f0QQLe5XAiHEvwofJu1bCbtl7lx4HwqDLWzyZC uPf8DI25wKoqC7um67O6zobBnutDwe+VvH3zEWKR1F0iy5WxpW0hBDGqD2xZnnKqfeb8 ya+g== X-Gm-Message-State: ANhLgQ2kRcQgbntFOcJ3kV2xCCM9sDfCLlXqnF3mNXFPT7XKvma7ntHD e/nwwWyHpOy0OQAik5Bk4UjF2w== X-Google-Smtp-Source: ADFU+vt3s0+lgv+iGRoTEkFt1ejwwiJRwUACnIjhsRxuQ+gcOw0zEySY+f1/M+LS9aENWkWVffV11A== X-Received: by 2002:ad4:4431:: with SMTP id e17mr992254qvt.64.1585339229040; Fri, 27 Mar 2020 13:00:29 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id s36sm4794732qtb.28.2020.03.27.13.00.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 27 Mar 2020 13:00:25 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jHv9V-0007fl-32; Fri, 27 Mar 2020 17:00:25 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 4/9] mm/hmm: remove HMM_FAULT_SNAPSHOT Date: Fri, 27 Mar 2020 17:00:16 -0300 Message-Id: <20200327200021.29372-5-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200327200021.29372-1-jgg@ziepe.ca> References: <20200327200021.29372-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe Now that flags are handled on a fine-grained per-page basis this global flag is redundant and has a confusing overlap with the pfn_flags_mask and default_flags. Normalize the HMM_FAULT_SNAPSHOT behavior into one place. Callers needing the SNAPSHOT behavior should set a pfn_flags_mask and default_flags that always results in a cleared HMM_PFN_VALID. Then no pages will be faulted, and HMM_FAULT_SNAPSHOT is not a special flow that overrides the masking mechanism. As this is the last flag, also remove the flags argument. If future flags are needed they can be part of the struct hmm_range function arguments. Signed-off-by: Jason Gunthorpe Reviewed-by: Christoph Hellwig --- Documentation/vm/hmm.rst | 12 +++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 2 +- drivers/gpu/drm/nouveau/nouveau_svm.c | 2 +- include/linux/hmm.h | 5 +---- mm/hmm.c | 17 +++++++++-------- 5 files changed, 17 insertions(+), 21 deletions(-) diff --git a/Documentation/vm/hmm.rst b/Documentation/vm/hmm.rst index 95fec596836262..4e3e9362afeb10 100644 --- a/Documentation/vm/hmm.rst +++ b/Documentation/vm/hmm.rst @@ -161,13 +161,11 @@ device must complete the update before the driver callback returns. When the device driver wants to populate a range of virtual addresses, it can use:: - long hmm_range_fault(struct hmm_range *range, unsigned int flags); + long hmm_range_fault(struct hmm_range *range); -With the HMM_RANGE_SNAPSHOT flag, it will only fetch present CPU page table -entries and will not trigger a page fault on missing or non-present entries. -Without that flag, it does trigger a page fault on missing or read-only entries -if write access is requested (see below). Page faults use the generic mm page -fault code path just like a CPU page fault. +It will trigger a page fault on missing or read-only entries if write access is +requested (see below). Page faults use the generic mm page fault code path just +like a CPU page fault. Both functions copy CPU page table entries into their pfns array argument. Each entry in that array corresponds to an address in the virtual range. HMM @@ -197,7 +195,7 @@ The usage pattern is:: again: range.notifier_seq = mmu_interval_read_begin(&interval_sub); down_read(&mm->mmap_sem); - ret = hmm_range_fault(&range, HMM_RANGE_SNAPSHOT); + ret = hmm_range_fault(&range); if (ret) { up_read(&mm->mmap_sem); if (ret == -EBUSY) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 90821ce5e6cad0..c520290709371b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -856,7 +856,7 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages) range->notifier_seq = mmu_interval_read_begin(&bo->notifier); down_read(&mm->mmap_sem); - r = hmm_range_fault(range, 0); + r = hmm_range_fault(range); up_read(&mm->mmap_sem); if (unlikely(r <= 0)) { /* diff --git a/drivers/gpu/drm/nouveau/nouveau_svm.c b/drivers/gpu/drm/nouveau/nouveau_svm.c index 39c731a99937c6..e3797b2d4d1759 100644 --- a/drivers/gpu/drm/nouveau/nouveau_svm.c +++ b/drivers/gpu/drm/nouveau/nouveau_svm.c @@ -540,7 +540,7 @@ static int nouveau_range_fault(struct nouveau_svmm *svmm, range.default_flags = 0; range.pfn_flags_mask = -1UL; down_read(&mm->mmap_sem); - ret = hmm_range_fault(&range, 0); + ret = hmm_range_fault(&range); up_read(&mm->mmap_sem); if (ret <= 0) { if (ret == 0 || ret == -EBUSY) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index daee6508a3f609..7475051100c782 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -117,13 +117,10 @@ static inline struct page *hmm_device_entry_to_page(const struct hmm_range *rang return pfn_to_page(entry >> range->pfn_shift); } -/* Don't fault in missing PTEs, just snapshot the current state. */ -#define HMM_FAULT_SNAPSHOT (1 << 1) - /* * Please see Documentation/vm/hmm.rst for how to use the range API. */ -long hmm_range_fault(struct hmm_range *range, unsigned int flags); +long hmm_range_fault(struct hmm_range *range); /* * HMM_RANGE_DEFAULT_TIMEOUT - default timeout (ms) when waiting for a range diff --git a/mm/hmm.c b/mm/hmm.c index 136de474221d77..8dbd9e1d0308b4 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -29,7 +29,6 @@ struct hmm_vma_walk { struct hmm_range *range; unsigned long last; - unsigned int flags; }; enum { @@ -112,9 +111,6 @@ static unsigned int hmm_pte_need_fault(const struct hmm_vma_walk *hmm_vma_walk, { struct hmm_range *range = hmm_vma_walk->range; - if (hmm_vma_walk->flags & HMM_FAULT_SNAPSHOT) - return 0; - /* * So we not only consider the individual per page request we also * consider the default flags requested for the range. The API can @@ -147,10 +143,17 @@ hmm_range_need_fault(const struct hmm_vma_walk *hmm_vma_walk, const uint64_t *pfns, unsigned long npages, uint64_t cpu_flags) { + struct hmm_range *range = hmm_vma_walk->range; unsigned int required_fault = 0; unsigned long i; - if (hmm_vma_walk->flags & HMM_FAULT_SNAPSHOT) + /* + * If the default flags do not request to fault pages, and the mask does + * not allow for individual pages to be faulted, then + * hmm_pte_need_fault() will always return 0. + */ + if (!((range->default_flags | range->pfn_flags_mask) & + range->flags[HMM_PFN_VALID])) return 0; for (i = 0; i < npages; ++i) { @@ -557,7 +560,6 @@ static const struct mm_walk_ops hmm_walk_ops = { /** * hmm_range_fault - try to fault some address in a virtual address range * @range: argument structure - * @flags: HMM_FAULT_* flags * * Return: the number of valid pages in range->pfns[] (from range start * address), which may be zero. On error one of the following status codes @@ -579,12 +581,11 @@ static const struct mm_walk_ops hmm_walk_ops = { * On error, for one virtual address in the range, the function will mark the * corresponding HMM pfn entry with an error flag. */ -long hmm_range_fault(struct hmm_range *range, unsigned int flags) +long hmm_range_fault(struct hmm_range *range) { struct hmm_vma_walk hmm_vma_walk = { .range = range, .last = range->start, - .flags = flags, }; struct mm_struct *mm = range->notifier->mm; int ret; From patchwork Fri Mar 27 20:00:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11463057 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EE3BA81 for ; Fri, 27 Mar 2020 20:00:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BB6BB20714 for ; Fri, 27 Mar 2020 20:00:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="WuL3sUI3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BB6BB20714 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B0FD76B0072; Fri, 27 Mar 2020 16:00:35 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A951E6B0073; Fri, 27 Mar 2020 16:00:35 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 983CB6B0074; Fri, 27 Mar 2020 16:00:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0222.hostedemail.com [216.40.44.222]) by kanga.kvack.org (Postfix) with ESMTP id 7F0616B0072 for ; Fri, 27 Mar 2020 16:00:35 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 3C745824805A for ; Fri, 27 Mar 2020 20:00:35 +0000 (UTC) X-FDA: 76642209630.04.limit41_2230bfb29b230 X-Spam-Summary: 2,0,0,08bdca79139bc454,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:41:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2198:2199:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3866:3867:3871:4419:5007:6120:6261:6653:7576:7808:7901:10004:11026:11473:11658:11914:12043:12114:12295:12296:12297:12438:12517:12519:12555:12679:12895:13069:13161:13229:13311:13357:13894:14181:14384:14394:14721:21080:21444:21451:21627:21990:30012:30054:30064,0,RBL:209.85.219.65:@ziepe.ca:.lbl8.mailshell.net-66.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: limit41_2230bfb29b230 X-Filterd-Recvd-Size: 4823 Received: from mail-qv1-f65.google.com (mail-qv1-f65.google.com [209.85.219.65]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Fri, 27 Mar 2020 20:00:34 +0000 (UTC) Received: by mail-qv1-f65.google.com with SMTP id t4so1353952qvz.8 for ; Fri, 27 Mar 2020 13:00:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9QoBL2elXCgBQYfLUofWPZ+WlaBKVcEFC/RukhDLmrw=; b=WuL3sUI3tbw/U0XG7NWMDd1WUeDaBUsogCrq7+FDuRkAbsrKV82rHP4vcJwfLkF2gg 7RFowYXUucT+ro5VSC9QSk0FrDwJabuexGQSPqng1BwqiZzp2un/0xDQ2mk5Knu6XEkh LbUoPopLvqKJ+0wumG9i0lhmsK01rb8n+mS19r2NwNP7c0uB5g2vLSXjwzZjvwxNWWUo kSLlfGN3HEmGqe58GdPkfi1QlPKBCxRJRBDQmZ1S+Q0BjI1a4SOjbGTjECQO+gcGk6mH lXiJMw3Qwp+Kt00yXEuxFp2hlnG7TKcI78dnI10wbTpfjf8nVZVas3N7BcXJPe63KZTC fcGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9QoBL2elXCgBQYfLUofWPZ+WlaBKVcEFC/RukhDLmrw=; b=KJx5W5qzF5d3IXNGmxja5OQHA4gHv3TM69MwT3sMwqBrABcOOKs6oaPjZCtKK5nAP0 Uom4OsTb9GIBhsRt5PUlRqen+m0vBZuTKxcpTJo3dZBqUZVKpH/4sll63/XO+TIWflQe zceoLJMrAvjtBuVPt6A5BauQttkX+C/pbz7m+ASMtwulJGYEhbRkyhsKvP36EkdW/3I6 09GNSRRA/aFvRewUomTI71X/9xPP/EL/svtOCg2Jmnc8/xAkjojRVTDDHQiy+yu+IEgi +eEj7A9qd000/I6sL0/PnpbUZW0VFT3wDAR4HzFaoNn3JygprkdQrJ+NyTE9iNhoxA4W ++SQ== X-Gm-Message-State: ANhLgQ0DIPEJrnVwUrs3RM6OTr8M4tS3Cx+Z5Vju4nACJb+p1cmSQerm mFbJf1sShQsmYzDt5B8Yzdza9w== X-Google-Smtp-Source: ADFU+vsJyJRZmZ5TMEYNXYMMzAdCRdRiK4gIWxEwhTOBgyVY+Pj2x550WvJcZ6da79IK6DKEnEHegw== X-Received: by 2002:ad4:43c1:: with SMTP id o1mr963932qvs.56.1585339234373; Fri, 27 Mar 2020 13:00:34 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id o81sm4515430qke.24.2020.03.27.13.00.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 27 Mar 2020 13:00:27 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jHv9V-0007fs-5I; Fri, 27 Mar 2020 17:00:25 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 5/9] mm/hmm: remove the CONFIG_TRANSPARENT_HUGEPAGE #ifdef Date: Fri, 27 Mar 2020 17:00:17 -0300 Message-Id: <20200327200021.29372-6-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200327200021.29372-1-jgg@ziepe.ca> References: <20200327200021.29372-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe This code can be compiled when CONFIG_TRANSPARENT_HUGEPAGE is off, so remove the ifdef. The function is only ever called under if (pmd_devmap(pmd) || pmd_trans_huge(pmd)) Which is statically false if !CONFIG_TRANSPARENT_HUGEPAGE, so the compiler reliably eliminates all of this code. Reviewed-by: Ralph Campbell Reviewed-by: Christoph Hellwig Tested-by: Ralph Campbell Signed-off-by: Jason Gunthorpe --- mm/hmm.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 8dbd9e1d0308b4..3a71ec72b0a42f 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -193,7 +193,6 @@ static inline uint64_t pmd_to_hmm_pfn_flags(struct hmm_range *range, pmd_t pmd) range->flags[HMM_PFN_VALID]; } -#ifdef CONFIG_TRANSPARENT_HUGEPAGE static int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, unsigned long end, uint64_t *pfns, pmd_t pmd) { @@ -216,11 +215,6 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, hmm_vma_walk->last = end; return 0; } -#else /* CONFIG_TRANSPARENT_HUGEPAGE */ -/* stub to allow the code below to compile */ -int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, - unsigned long end, uint64_t *pfns, pmd_t pmd); -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ static inline bool hmm_is_device_private_entry(struct hmm_range *range, swp_entry_t entry) From patchwork Fri Mar 27 20:00:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11463055 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E26AB81 for ; Fri, 27 Mar 2020 20:00:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A66872071B for ; Fri, 27 Mar 2020 20:00:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="noYJUaxT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A66872071B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 519956B0071; Fri, 27 Mar 2020 16:00:35 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4A39E6B0072; Fri, 27 Mar 2020 16:00:35 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3B7EB6B0073; Fri, 27 Mar 2020 16:00:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0032.hostedemail.com [216.40.44.32]) by kanga.kvack.org (Postfix) with ESMTP id 249EF6B0071 for ; Fri, 27 Mar 2020 16:00:35 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id CCA45824805A for ; Fri, 27 Mar 2020 20:00:34 +0000 (UTC) X-FDA: 76642209630.16.dad47_221a02006ad5e X-Spam-Summary: 2,0,0,7fb53806d67aa1f2,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1714:1730:1747:1777:1792:2393:2559:2562:2892:3138:3139:3140:3141:3142:3351:3865:3866:3867:3870:4321:5007:6119:6120:6261:6653:7576:7901:10004:11026:11473:11658:11914:12043:12114:12296:12297:12438:12517:12519:12555:12679:12895:13069:13161:13229:13311:13357:13894:14181:14384:14394:14721:21080:21444:21451:21627:30054:30064:30075,0,RBL:209.85.222.196:@ziepe.ca:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: dad47_221a02006ad5e X-Filterd-Recvd-Size: 4306 Received: from mail-qk1-f196.google.com (mail-qk1-f196.google.com [209.85.222.196]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Fri, 27 Mar 2020 20:00:34 +0000 (UTC) Received: by mail-qk1-f196.google.com with SMTP id 139so2306091qkd.9 for ; Fri, 27 Mar 2020 13:00:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TyPTTIrgr5jFkg31+AHSLRGpSHCvL1+EWcgijYv1QPM=; b=noYJUaxTqgZy/wHPJ7HGyMyohSauQN1H/BBlkuBWc8iQCW/ZppJ+LwXNIRokuECBml u6KEyLpSSemzeGjYlvPG3hJinN1mPYzfpw2TKQpLGrRm3QkSvLfqx+7sdqisc85aoNbF dAE0zPXSTZ5VOOUE4p3EJNV+nlIxlCbdWMU7jIWo61XJz2jsxynfZ8Jsv8TM8jKjjwLg W/OnoLB7/2LjBbA/03Wp1larZqxPw94OAM/d0G0vnQYlKbXBzvfptKdfRp2LDnmVwPiJ NmoLTGimt357fCiaJDyo9V+hfs+zQXXNlC4UR7IdT0Vh9odOWaoD5d7y5ym3WEq75ble uW1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TyPTTIrgr5jFkg31+AHSLRGpSHCvL1+EWcgijYv1QPM=; b=GWqHoQwzr/X478cVWxJmTgQ8KXhvDFCk1bmIX9sKHDWEktTs2Bl8T2YUrgNeavmJzf KwD1fFYSS7NMimS96V5qYjQeBNs4x8JgL0cBNYAWwBnGnIAEDCtP+fQu0rccQ9m7Z2DC O/rehB+KTrd7OXeU7DzCFY5w6vil4x8xWukvAbI1U5h/aLlWvYahKio04iqOOoBuH7Vr SDfdHBYH55C7/W/VDW2S3w8kfXrvW40LCY63jUQjEq+5HFlpdZ/IcpO4gVWIDBMQZ7H+ PKVCIl7gmVtfDJykaCWFvv3WpM8bfeYjFoiMPqcFpSG7VXDhPH6+7KRQPFVXAeYItpFS 3hCg== X-Gm-Message-State: ANhLgQ38Vk19aGAuVlLqwNyUh1DO+xBRsgNCi3gbKBykNTtS4Ppdd9pD 56qK7sWua5AhmqWVGwy6QgTI8eK9KNj5Gg== X-Google-Smtp-Source: ADFU+vusyRe6ojBp5llZXnMxLYofwoFjLl3UfJlP4qaMzbvBMyNTtVgrylTbiL3g/n+IqSKDUu4X9Q== X-Received: by 2002:a37:30f:: with SMTP id 15mr1101270qkd.44.1585339233822; Fri, 27 Mar 2020 13:00:33 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id f1sm3946514qkl.91.2020.03.27.13.00.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 27 Mar 2020 13:00:27 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jHv9V-0007fy-6Y; Fri, 27 Mar 2020 17:00:25 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 6/9] mm/hmm: use device_private_entry_to_pfn() Date: Fri, 27 Mar 2020 17:00:18 -0300 Message-Id: <20200327200021.29372-7-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200327200021.29372-1-jgg@ziepe.ca> References: <20200327200021.29372-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe swp_offset() should not be called directly, the wrappers are supposed to abstract away the encoding of the device_private specific information in the swap entry. Reviewed-by: Ralph Campbell Reviewed-by: Christoph Hellwig Tested-by: Ralph Campbell Signed-off-by: Jason Gunthorpe --- mm/hmm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/hmm.c b/mm/hmm.c index 3a71ec72b0a42f..3303686bf16d24 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -261,7 +261,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, */ if (hmm_is_device_private_entry(range, entry)) { *pfn = hmm_device_entry_from_pfn(range, - swp_offset(entry)); + device_private_entry_to_pfn(entry)); *pfn |= range->flags[HMM_PFN_VALID]; if (is_write_device_private_entry(entry)) *pfn |= range->flags[HMM_PFN_WRITE]; From patchwork Fri Mar 27 20:00:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11463053 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5DCD51667 for ; Fri, 27 Mar 2020 20:00:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2A65E20714 for ; Fri, 27 Mar 2020 20:00:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="LO8u6Sf1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2A65E20714 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EDE2A6B0070; Fri, 27 Mar 2020 16:00:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E3ECA6B0071; Fri, 27 Mar 2020 16:00:33 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2D5F6B0072; Fri, 27 Mar 2020 16:00:33 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0029.hostedemail.com [216.40.44.29]) by kanga.kvack.org (Postfix) with ESMTP id B3A066B0071 for ; Fri, 27 Mar 2020 16:00:33 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id D8FA86110 for ; Fri, 27 Mar 2020 20:00:33 +0000 (UTC) X-FDA: 76642209546.08.hour45_21ec5c6b9bc1e X-Spam-Summary: 2,0,0,9651002a8074b790,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:41:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1541:1711:1730:1747:1777:1792:2393:2559:2562:2693:3138:3139:3140:3141:3142:3353:3865:3867:3868:3870:3874:4321:4419:5007:6117:6119:6120:6261:6653:7576:7875:7901:7903:9036:10004:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12679:12895:12986:13069:13161:13229:13311:13357:13894:14181:14384:14394:14721:21080:21444:21451:21627:21990:30054:30070,0,RBL:209.85.222.194:@ziepe.ca:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: hour45_21ec5c6b9bc1e X-Filterd-Recvd-Size: 5207 Received: from mail-qk1-f194.google.com (mail-qk1-f194.google.com [209.85.222.194]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Fri, 27 Mar 2020 20:00:32 +0000 (UTC) Received: by mail-qk1-f194.google.com with SMTP id h14so12212328qke.5 for ; Fri, 27 Mar 2020 13:00:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=w7HT384dLquXkifrMSaPZlakZIt6uG6pSy8/vMIPtTg=; b=LO8u6Sf1tFMYHfFgs/mRngqSax2KnXlZcJO3p0Dc+bnFdWQ3mFI2oTFey6thDTmmaF 6vQSVW9hkX0MF2gHtADWhz2PG6//81Sy+koWSRmlUtdE4JkEDlbi+Hl7e9/fJkqBDQ1W wZGR9Wj/y9vVjDnX6QH2oWytEcLS1b2oPoz2PvXkTnw7cdsqRkzHAarbUERdJ7ThoQ/X cjfSJj6agzko8pFrT99pB+ejxChHQOTwCS/jnyQVX3vj5kkIgnEKBBVdjZ6Uyg+MCpb7 McioaeQ1NszNr6YE8lMUcEsAudEty68BJWZfarn6H3IuArhauChB9pgs+awHkyiMwPkH VdsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=w7HT384dLquXkifrMSaPZlakZIt6uG6pSy8/vMIPtTg=; b=Ey7JDETLWt4+3MD8DjkB39ujO9KX4YxBqnDuLx6XidL6U+AvbDlcNjuegrgS5N9Zi7 aIQrvpXTo7LlkfNyEkJmEd52s1Iqm6q71VRzhmHVLlYXUuR3veES/99LoHNHsXsKbpAX qFwyX/sFGXyv0Omjwm3cZ0gRqy1oowQc5APQCCVGTDvCDo3DnpddzM1+k78ZO6fJiTEy ntlwElJbb6unYEemvOu4IvuSKnlQFPMLtY2fD82zlA54VgwUO7vV1R/ptkGVAfcaKEXe yZAbrOQ8TYsCpa4tmSAfFDh6RRgApbNaOAj8yV67tGeKK/380ZMVA4vqx5vpj9KSbxTF 37OA== X-Gm-Message-State: ANhLgQ2hr2g+nnjNpaFY11+NyrmxKNdRt6WZMq7jeYH9obdts4BFeJq0 ZmoZInpKJSQKABWLafZt1KJtjQ== X-Google-Smtp-Source: ADFU+vsV4R08VNgi3U0EcEDSpq36V0bb1XOE53WhiumJ5wLNMq9YlKH0bNPjhSJsWggFm8HqFwZeRw== X-Received: by 2002:a37:86c4:: with SMTP id i187mr1090284qkd.110.1585339232485; Fri, 27 Mar 2020 13:00:32 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id n64sm4367174qkf.23.2020.03.27.13.00.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 27 Mar 2020 13:00:25 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jHv9V-0007g4-98; Fri, 27 Mar 2020 17:00:25 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 7/9] mm/hmm: do not unconditionally set pfns when returning EBUSY Date: Fri, 27 Mar 2020 17:00:19 -0300 Message-Id: <20200327200021.29372-8-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200327200021.29372-1-jgg@ziepe.ca> References: <20200327200021.29372-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe In hmm_vma_handle_pte() and hmm_vma_walk_hugetlb_entry() if fault happens then -EBUSY will be returned and the pfns input flags will have been destroyed. For hmm_vma_handle_pte() set HMM_PFN_NONE only on the success returns that don't otherwise store to pfns. For hmm_vma_walk_hugetlb_entry() all exit paths already set pfns, so remove the redundant store. Fixes: 2aee09d8c116 ("mm/hmm: change hmm_vma_fault() to allow write fault on page basis") Reviewed-by: Christoph Hellwig Signed-off-by: Jason Gunthorpe --- mm/hmm.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 3303686bf16d24..dc826898979bc5 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -244,11 +244,11 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, pte_t pte = *ptep; uint64_t orig_pfn = *pfn; - *pfn = range->values[HMM_PFN_NONE]; if (pte_none(pte)) { required_fault = hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0); if (required_fault) goto fault; + *pfn = range->values[HMM_PFN_NONE]; return 0; } @@ -269,8 +269,10 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, } required_fault = hmm_pte_need_fault(hmm_vma_walk, orig_pfn, 0); - if (!required_fault) + if (!required_fault) { + *pfn = range->values[HMM_PFN_NONE]; return 0; + } if (!non_swap_entry(entry)) goto fault; @@ -488,7 +490,6 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, i = (start - range->start) >> PAGE_SHIFT; orig_pfn = range->pfns[i]; - range->pfns[i] = range->values[HMM_PFN_NONE]; cpu_flags = pte_to_hmm_pfn_flags(range, entry); required_fault = hmm_pte_need_fault(hmm_vma_walk, orig_pfn, cpu_flags); if (required_fault) { From patchwork Fri Mar 27 20:00:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11463059 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C3CA4913 for ; Fri, 27 Mar 2020 20:00:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8735620714 for ; Fri, 27 Mar 2020 20:00:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="CBwtBrf0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8735620714 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 763886B0073; Fri, 27 Mar 2020 16:00:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6A12C6B0074; Fri, 27 Mar 2020 16:00:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 569A36B0075; Fri, 27 Mar 2020 16:00:36 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0249.hostedemail.com [216.40.44.249]) by kanga.kvack.org (Postfix) with ESMTP id 304256B0073 for ; Fri, 27 Mar 2020 16:00:36 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id D7E0C824805A for ; Fri, 27 Mar 2020 20:00:35 +0000 (UTC) X-FDA: 76642209630.22.flock75_2248ee45fe50b X-Spam-Summary: 2,0,0,7bcd319b3f8145a2,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:41:69:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1542:1711:1730:1747:1777:1792:2198:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3354:3865:3866:3867:3870:3871:3872:3874:4250:4321:5007:6117:6119:6120:6261:6653:7576:7875:7901:7903:9036:9592:10004:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12679:12895:12986:13894:14181:14394:14721:21080:21433:21444:21451:21627:21990:30003:30012:30054,0,RBL:209.85.222.193:@ziepe.ca:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: flock75_2248ee45fe50b X-Filterd-Recvd-Size: 5931 Received: from mail-qk1-f193.google.com (mail-qk1-f193.google.com [209.85.222.193]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Fri, 27 Mar 2020 20:00:35 +0000 (UTC) Received: by mail-qk1-f193.google.com with SMTP id j4so12144236qkc.11 for ; Fri, 27 Mar 2020 13:00:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TDrjscj5VUbtTUg81Gj8EA5DAPIuadUuhV+FJXt/EPU=; b=CBwtBrf0j+KndtyNs7QkByd1ivDhH+r1q5a3WPT7JV3TLYd84eFuXNa2pjKTL8rR01 fA3p/7kp415/LAwD4p4mKcIE3bas9ZhxZsIJyoLJH+hfkHIcJdPy+u+exZl/5J8jDK0m oDQomhjKgAoTF1Yp1ZL0iWj90lQAGleiyozXgaTgCoPdHdnpi3iFynvjUkd/iN5cdCLs opnE1kFN8Bgi3em7/ZY3yGrQbvWohI5okMJe51Qfr4N93myf/IcQCqmPms9ivBuiGyDH g+Rcmc1/kxA/RlbxIu3x8QQJQmnwm+/FkuKvDPbBSX5n058qaQBKousHs+ZmXxlOaPtP FDMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TDrjscj5VUbtTUg81Gj8EA5DAPIuadUuhV+FJXt/EPU=; b=ppyrIPwuMwAThupQR3WVfQTg+9wATSQCAK5jeUWwRMSpqS2LiBqwpXm2JJWXMmIso6 ZBkANLRlRo6cNcbBICVtdc/GkOHubVaoa73CuAdFw7SWM/1jWM+wrkkHLgnAUReeoRKf tEn0UKo1P/E6wLckjK6FYSI3T6lgx6RdsFSh9tyIqy/H2+VR1JXvqhaQ0Po39JNczyo/ 80bVuhOngLWT+En0hisazOBOHhHsAElWg86rjBUfKyBF/6jgMujwzvN2ABRjs4K4Jkvh MPyS+kUXsEzDpXLHwlLGbwUJlV5AVm0wgMmuaus2U37sGYEligbt57Zi5xceBxyenIrv JCHQ== X-Gm-Message-State: ANhLgQ1oKRk0nb+5gjiz+MN15o7nM8DGNz1jf+ddBDCSQjGFxxTvYLk3 j2k1q47j5A7sMZ/uruiH2blWoA== X-Google-Smtp-Source: ADFU+vtbEMaje0l8Yq/7uiD0ikZWgvPtViAmPYH3kZWMXZUn8JFdaevM0Ixe5LKqy7XPslfAtSA1jw== X-Received: by 2002:a37:e40d:: with SMTP id y13mr1045203qkf.39.1585339234848; Fri, 27 Mar 2020 13:00:34 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id y127sm4512371qkb.76.2020.03.27.13.00.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 27 Mar 2020 13:00:28 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jHv9V-0007gA-B2; Fri, 27 Mar 2020 17:00:25 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 8/9] mm/hmm: do not set pfns when returning an error code Date: Fri, 27 Mar 2020 17:00:20 -0300 Message-Id: <20200327200021.29372-9-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200327200021.29372-1-jgg@ziepe.ca> References: <20200327200021.29372-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe Most places that return an error code, like -EFAULT, do not set HMM_PFN_ERROR, only two places do this. Resolve this inconsistency by never setting the pfns on an error exit. This doesn't seem like a worthwhile thing to do anyhow. If for some reason it becomes important, it makes more sense to directly return the address of the failing page rather than have the caller scan for the HMM_PFN_ERROR. No caller inspects the pnfs output array if hmm_range_fault() fails. Reviewed-by: Christoph Hellwig Signed-off-by: Jason Gunthorpe --- mm/hmm.c | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index dc826898979bc5..e875d9ef0968fd 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -77,17 +77,14 @@ static int hmm_vma_fault(unsigned long addr, unsigned long end, unsigned int required_fault, struct mm_walk *walk) { struct hmm_vma_walk *hmm_vma_walk = walk->private; - struct hmm_range *range = hmm_vma_walk->range; struct vm_area_struct *vma = walk->vma; - uint64_t *pfns = range->pfns; - unsigned long i = (addr - range->start) >> PAGE_SHIFT; unsigned int fault_flags = FAULT_FLAG_REMOTE; WARN_ON_ONCE(!required_fault); hmm_vma_walk->last = addr; if (!vma) - goto out_error; + return -EFAULT; if (required_fault & HMM_NEED_WRITE_FAULT) { if (!(vma->vm_flags & VM_WRITE)) @@ -95,15 +92,10 @@ static int hmm_vma_fault(unsigned long addr, unsigned long end, fault_flags |= FAULT_FLAG_WRITE; } - for (; addr < end; addr += PAGE_SIZE, i++) + for (; addr < end; addr += PAGE_SIZE) if (handle_mm_fault(vma, addr, fault_flags) & VM_FAULT_ERROR) - goto out_error; - + return -EFAULT; return -EBUSY; - -out_error: - pfns[i] = range->values[HMM_PFN_ERROR]; - return -EFAULT; } static unsigned int hmm_pte_need_fault(const struct hmm_vma_walk *hmm_vma_walk, @@ -286,7 +278,6 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, /* Report error for everything else */ pte_unmap(ptep); - *pfn = range->values[HMM_PFN_ERROR]; return -EFAULT; } @@ -572,9 +563,6 @@ static const struct mm_walk_ops hmm_walk_ops = { * * This is similar to get_user_pages(), except that it can read the page tables * without mutating them (ie causing faults). - * - * On error, for one virtual address in the range, the function will mark the - * corresponding HMM pfn entry with an error flag. */ long hmm_range_fault(struct hmm_range *range) { From patchwork Fri Mar 27 20:00:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 11463061 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BA13681 for ; Fri, 27 Mar 2020 20:01:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 77DC320714 for ; Fri, 27 Mar 2020 20:01:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="Wu2ZwNFX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 77DC320714 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 05E996B0074; Fri, 27 Mar 2020 16:00:37 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 010786B0075; Fri, 27 Mar 2020 16:00:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E3F866B0078; Fri, 27 Mar 2020 16:00:36 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0247.hostedemail.com [216.40.44.247]) by kanga.kvack.org (Postfix) with ESMTP id B27276B0074 for ; Fri, 27 Mar 2020 16:00:36 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 7EE991260 for ; Fri, 27 Mar 2020 20:00:36 +0000 (UTC) X-FDA: 76642209672.11.rake37_225946edd4463 X-Spam-Summary: 2,0,0,587f6f01d530df02,d41d8cd98f00b204,jgg@ziepe.ca,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2198:2199:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3867:3871:3872:3874:4250:5007:6117:6120:6261:6653:7576:7875:7901:8660:9036:9592:10004:11026:11473:11658:11914:12043:12297:12438:12517:12519:12555:12679:12895:13069:13148:13161:13229:13230:13311:13357:13894:14181:14384:14394:14721:21080:21325:21444:21451:21627:21990:30054:30070,0,RBL:209.85.222.196:@ziepe.ca:.lbl8.mailshell.net-66.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:31,LUA_SUMMARY:none X-HE-Tag: rake37_225946edd4463 X-Filterd-Recvd-Size: 4783 Received: from mail-qk1-f196.google.com (mail-qk1-f196.google.com [209.85.222.196]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Fri, 27 Mar 2020 20:00:35 +0000 (UTC) Received: by mail-qk1-f196.google.com with SMTP id c145so12149671qke.12 for ; Fri, 27 Mar 2020 13:00:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JnmlXREPuYpdVgA5s4jhAhUS2rhGunirwIYiSIKrMtg=; b=Wu2ZwNFXHpdCBgvdA8u8Cz/oBGw1KkX1ncVdRnShcf4GElMjWiKn1EFoelD3SC8Pdu 5oSRc44vjt9HjDQIdnAjLiB+iqkIomFaK7HFZJFCv1kfCNeuzJVsRL7yLgCUG9Yr40/y WgrivYlj6PLrkUegOnWHJZ1F8UqAmw2TZvMLx7W8RoATRVA6AcfKj/tg/ZM++QQ2FInD OgMKezyjEZ1p8onZcLsVkCB8VKeSaGKmAahK36UK9mn1vHsesFmJ7K8NrAs9qVS1rjff SOFbQ2JU3d1BCXnSk9YKyqsHoaV8jbjKTUqEGLI2hP1Db4PBe3lavzYaTW19q+NqJqN0 9c+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JnmlXREPuYpdVgA5s4jhAhUS2rhGunirwIYiSIKrMtg=; b=c6gtCBldQd9DVWI+u4uOFsWQof5eGoxVBQjhrbLqVPsGy/K3kD2LXjR36NTqoEIbO8 gaGIUEGm+Sd7ODXgNQQw0JPilJfPYAX622WKFy6Dz+mPQ/cXB1xtIGEqFJDjnZAwiMaT PR3/EyAq88d0ABShtTJrWzQH6RN3y9iSJYQYC2h/wp0pwDljT8nHPfIbLNRHW2NBNIJb eW9mhDiPiDkiIp8fSxo5JjRn5ES5m4A3KRWQ1yhUehq1tK26UXflYcdNXPPFmUUtbLOY F5gomYMKasOQ21SMKGIcwgQMtg3UpC3bGRQkN9xwaM8Hct6ri4OUfkmCjxKHJ7aoJMAQ kNOg== X-Gm-Message-State: ANhLgQ0PdgTdkmf/CHfz3Zc5FBzIgwp+jkAdvTRkDzCmN7SLlythC6B0 OI9U8py/OjUm63R4U01J4ohOtQ== X-Google-Smtp-Source: ADFU+vtP+nRCig+6NY7kgxbUHqtF0cMlEgYlNWpvv3AXm5n10mHCqPWrSiPAvbRUdrMjQ+WdapHq0w== X-Received: by 2002:a05:620a:12fc:: with SMTP id f28mr1059601qkl.31.1585339235366; Fri, 27 Mar 2020 13:00:35 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id 207sm4452408qkj.41.2020.03.27.13.00.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 27 Mar 2020 13:00:29 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jHv9V-0007gG-CJ; Fri, 27 Mar 2020 17:00:25 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , Felix.Kuehling@amd.com Cc: linux-mm@kvack.org, John Hubbard , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christoph Hellwig , Philip Yang , Jason Gunthorpe Subject: [PATCH v2 hmm 9/9] mm/hmm: return error for non-vma snapshots Date: Fri, 27 Mar 2020 17:00:21 -0300 Message-Id: <20200327200021.29372-10-jgg@ziepe.ca> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200327200021.29372-1-jgg@ziepe.ca> References: <20200327200021.29372-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe The pagewalker does not call most ops with NULL vma, those are all routed to hmm_vma_walk_hole() via ops->pte_hole instead. Thus hmm_vma_fault() is only called with a NULL vma from hmm_vma_walk_hole(), so hoist the NULL vma check to there. Now it is clear that snapshotting with no vma is a HMM_PFN_ERROR as without a vma we have no path to call hmm_vma_fault(). Reviewed-by: Christoph Hellwig Signed-off-by: Jason Gunthorpe --- mm/hmm.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index e875d9ef0968fd..31d0f68689c32b 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -83,9 +83,6 @@ static int hmm_vma_fault(unsigned long addr, unsigned long end, WARN_ON_ONCE(!required_fault); hmm_vma_walk->last = addr; - if (!vma) - return -EFAULT; - if (required_fault & HMM_NEED_WRITE_FAULT) { if (!(vma->vm_flags & VM_WRITE)) return -EPERM; @@ -170,6 +167,11 @@ static int hmm_vma_walk_hole(unsigned long addr, unsigned long end, npages = (end - addr) >> PAGE_SHIFT; pfns = &range->pfns[i]; required_fault = hmm_range_need_fault(hmm_vma_walk, pfns, npages, 0); + if (!walk->vma) { + if (required_fault) + return -EFAULT; + return hmm_pfns_fill(addr, end, range, HMM_PFN_ERROR); + } if (required_fault) return hmm_vma_fault(addr, end, required_fault, walk); hmm_vma_walk->last = addr;