From patchwork Fri Feb 28 11:32:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 11412045 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 09F8013A4 for ; Fri, 28 Feb 2020 11:34:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CA8A8246A8 for ; Fri, 28 Feb 2020 11:34:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="k/JZQ00I" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CA8A8246A8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1947B6B0006; Fri, 28 Feb 2020 06:34:58 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 16B116B0007; Fri, 28 Feb 2020 06:34:58 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 059C76B0008; Fri, 28 Feb 2020 06:34:58 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0199.hostedemail.com [216.40.44.199]) by kanga.kvack.org (Postfix) with ESMTP id E2E1E6B0006 for ; Fri, 28 Feb 2020 06:34:57 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 9EE4B181AEF23 for ; Fri, 28 Feb 2020 11:34:57 +0000 (UTC) X-FDA: 76539329034.03.watch10_657c6604a063f X-Spam-Summary: 2,0,0,621baf663dcae07b,d41d8cd98f00b204,kernelfans@gmail.com,,RULES_HIT:41:69:355:379:541:800:960:973:988:989:1260:1345:1359:1431:1437:1535:1542:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:4605:5007:6119:6261:6653:6742:7514:7901:7903:8634:9413:10004:11026:11473:11658:11914:12043:12114:12296:12297:12438:12517:12519:12555:12895:13846:14096:14181:14394:14687:14721:21080:21444:21451:21627:21666:21990:30012:30045:30054:30064,0,RBL:209.85.215.196:@gmail.com:.lbl8.mailshell.net-66.100.201.100 62.50.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: watch10_657c6604a063f X-Filterd-Recvd-Size: 5348 Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Fri, 28 Feb 2020 11:34:57 +0000 (UTC) Received: by mail-pg1-f196.google.com with SMTP id u12so1366351pgb.10 for ; Fri, 28 Feb 2020 03:34:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=HJEEF+ZoXF3+Nd4diWecS36bLcljIlgCVbyfxnITy0g=; b=k/JZQ00IhfViLeK6G0njVXZ+whnThIG8lZANGkpgh80P/Y9DAoOPXtOjwaXFEvZlho oz9AlqAiMyxygoKt0ftGvwJtLrK7uIWroi+5Gq5I/wxXz3PzeGj1/EaDjmKcvABvVPcg MaBY6borR8Pn/mMueS/cgTmeyhF63ywAsnH963GgDtDlOD3UwHCtkYb6yTCWMbvd+GZg h0CPoJTu9VlPuuAVr0IKrhtCX4r5dLBgzUnHWkZRljK4KM+SYf4I55KsyP+iy92yTxcr 4rbE5aGfhTpQmaVltG2anr8eA8bM40vH4JqnfetYkYVtmCJ3w8+SBU7m6iKhkHqw4iM7 m1Yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=HJEEF+ZoXF3+Nd4diWecS36bLcljIlgCVbyfxnITy0g=; b=MFLEYO0sulkMSHgws7aa9P22owEcTmv0saHOc1l5DegcMMqlPACIDavOTs+C+dym7Q FVwsD9i6QjHNVsPZPet8P7sLI1V151u/N/w0PZdJHgzKPeaYAYF/V64CvYBAXOn2dDEL f3jHz9On3guvS54/cZ80PsUvZNi6njgCIYMwcR7hdLFukponh+F+bx9PfS9Jc7PVKYg/ FK1vwcCMa1Mf3T1qlB9C7VSv88ts9pJ8+r3U6MXYjq9n/5rhOVdVNJK02hoWxInARGAd JmYchiXBOMFWMdNTv7VzxmGUP47xTjVRSnrn4lUV5C3qa66k/nwMCWgPef4FGZpcIjR1 axng== X-Gm-Message-State: APjAAAU/GaEkayHQXb0GDb4D0a0HW6TKafzXlSC57gdcYB9ZeLd0pSBI ol7JvOA0m2nIqgCUdTt0zb+1Z+tfxQ== X-Google-Smtp-Source: APXvYqzjxd/Sq/oiTAGLmX1fGiq0C3UXJLMhVbet4cdnBwIB+rQQLDWwYP/Y2VJjq2wMVZgl16uJlA== X-Received: by 2002:a05:6a00:2ba:: with SMTP id q26mr2742501pfs.249.1582889696147; Fri, 28 Feb 2020 03:34:56 -0800 (PST) Received: from mylaptop.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id d14sm11402168pfq.117.2020.02.28.03.34.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 28 Feb 2020 03:34:55 -0800 (PST) From: Pingfan Liu To: linux-mm@kvack.org Cc: Pingfan Liu , Ira Weiny , Andrew Morton , Mike Rapoport , Dan Williams , Matthew Wilcox , John Hubbard , "Aneesh Kumar K.V" , Keith Busch , Christoph Hellwig , Shuah Khan , linux-kernel@vger.kernel.org Subject: [PATCHv5 1/3] mm/gup: rename nr as nr_pinned in internal_get_user_pages_fast() Date: Fri, 28 Feb 2020 19:32:28 +0800 Message-Id: <1582889550-9101-2-git-send-email-kernelfans@gmail.com> X-Mailer: git-send-email 2.7.5 In-Reply-To: <1582889550-9101-1-git-send-email-kernelfans@gmail.com> References: <1582889550-9101-1-git-send-email-kernelfans@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To better reflect the held state of pages and make code self-explaining, rename nr as nr_pinned. Signed-off-by: Pingfan Liu Cc: Ira Weiny Cc: Andrew Morton Cc: Mike Rapoport Cc: Dan Williams Cc: Matthew Wilcox Cc: John Hubbard Cc: "Aneesh Kumar K.V" Cc: Keith Busch Cc: Christoph Hellwig Cc: Shuah Khan To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org --- mm/gup.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 1b521e0..cd8075e 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2432,7 +2432,7 @@ static int internal_get_user_pages_fast(unsigned long start, int nr_pages, struct page **pages) { unsigned long addr, len, end; - int nr = 0, ret = 0; + int nr_pinned = 0, ret = 0; if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM | FOLL_FORCE | FOLL_PIN))) @@ -2451,25 +2451,25 @@ static int internal_get_user_pages_fast(unsigned long start, int nr_pages, if (IS_ENABLED(CONFIG_HAVE_FAST_GUP) && gup_fast_permitted(start, end)) { local_irq_disable(); - gup_pgd_range(addr, end, gup_flags, pages, &nr); + gup_pgd_range(addr, end, gup_flags, pages, &nr_pinned); local_irq_enable(); - ret = nr; + ret = nr_pinned; } - if (nr < nr_pages) { + if (nr_pinned < nr_pages) { /* Try to get the remaining pages with get_user_pages */ - start += nr << PAGE_SHIFT; - pages += nr; + start += nr_pinned << PAGE_SHIFT; + pages += nr_pinned; - ret = __gup_longterm_unlocked(start, nr_pages - nr, + ret = __gup_longterm_unlocked(start, nr_pages - nr_pinned, gup_flags, pages); /* Have to be a bit careful with return values */ - if (nr > 0) { + if (nr_pinned > 0) { if (ret < 0) - ret = nr; + ret = nr_pinned; else - ret += nr; + ret += nr_pinned; } } From patchwork Fri Feb 28 11:32:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 11412047 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 480F514BC for ; Fri, 28 Feb 2020 11:35:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 07326246BF for ; Fri, 28 Feb 2020 11:35:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="c0HuMjCc" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 07326246BF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 479EF6B0007; Fri, 28 Feb 2020 06:35:04 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4530B6B0008; Fri, 28 Feb 2020 06:35:04 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 340126B000A; Fri, 28 Feb 2020 06:35:04 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0201.hostedemail.com [216.40.44.201]) by kanga.kvack.org (Postfix) with ESMTP id 178C86B0007 for ; Fri, 28 Feb 2020 06:35:04 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id C8C57181AEF23 for ; Fri, 28 Feb 2020 11:35:03 +0000 (UTC) X-FDA: 76539329286.09.wheel53_665da3b6bfc57 X-Spam-Summary: 2,0,0,42d619e9577b4ec0,d41d8cd98f00b204,kernelfans@gmail.com,,RULES_HIT:41:355:379:541:800:960:966:973:988:989:1260:1345:1359:1431:1437:1535:1543:1711:1730:1747:1777:1792:2196:2199:2393:2559:2562:3138:3139:3140:3141:3142:3355:3608:3865:3866:3867:3868:3870:3871:3872:3874:4118:4250:4321:4385:5007:6261:6653:6742:7514:8634:9036:9413:10004:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:12986:13161:13229:13846:14181:14394:14687:14721:21080:21251:21433:21444:21451:21627:21666:21990:30054:30064:30070,0,RBL:209.85.215.195:@gmail.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: wheel53_665da3b6bfc57 X-Filterd-Recvd-Size: 7131 Received: from mail-pg1-f195.google.com (mail-pg1-f195.google.com [209.85.215.195]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Fri, 28 Feb 2020 11:35:03 +0000 (UTC) Received: by mail-pg1-f195.google.com with SMTP id u12so1366442pgb.10 for ; Fri, 28 Feb 2020 03:35:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=WaTuQUSD3FMKmtCCA0ipNCi0y5rbOoaZk9gisNac/9g=; b=c0HuMjCc+aTrgTZ7ztGz5UY2xFznt+qwmoB6AYlPqgnGS6AgQww4emfDwoa7uoDtVK IHINFeakoVuNaa8KmBiReuI8Q6/c9kNyJvZkJXgxkvytNtv193Whi3I/ftWKNionoNnP xOYiZUiLuIMeuwdPiC47pPYVhs+GC1OQLHamYcl6LE7D7X1fMfIym14wwo6dGQMxHw1S Ao+p2ou4rN8gMDZVrs5CQ6Em0Rfs7PvMeThMhWkaUySm3HmUh8shLALPc8yAoD1EyO2q dSEbrXe0ZM0Mz/CHx8GnzJH+95XfypzauvI2x3rjPtgTW8oqATxismqGx2DPtBgcl9dV 0t1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=WaTuQUSD3FMKmtCCA0ipNCi0y5rbOoaZk9gisNac/9g=; b=sG/3JDm/f98Ms+V0fdhgmRG2G+nRTLMXrLhPiMto762lW9M3IwTPNBriiMniayEZC5 Qd3zDbGSBCNitRlnAtZ0RdyZS6KaHvlt1vcdjYZz2wUYep9COtLSXREpG4ElTB+z5Ny3 dvdzMN5ulJFg4d9cBmXFyZ+/CWIuesxtwotOHb9RjA7m2q2iTIvTPWH8kyrK5m6MZo1V sXlZe6rubGfP392s63w+DItlZYUTyyTHfHYrWlTixcsGdwVq9O0qnTiR2PrFGRDjqpCR ynpzwmlzTWbUNCFqOt7EfWNHcAJpRg2s0DijLADxM46PEFVsidBg7UK0jUqbHkObmeU1 XQTA== X-Gm-Message-State: APjAAAVObs1HfVkvRvw9L8+jIWzGpH+ClwIoSVawFe5BQERquGaUk/RE FR3mQnon8nBYbDO4/soKl7jCG97pZQ== X-Google-Smtp-Source: APXvYqyMqYfx+Gl5paJ1UpKaYGQTelDIg9hDLU17UT8ZUVgANMbncsehzNg7hFkBoLvpxUMcMxDuJA== X-Received: by 2002:a63:e20d:: with SMTP id q13mr4096480pgh.6.1582889702220; Fri, 28 Feb 2020 03:35:02 -0800 (PST) Received: from mylaptop.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id d14sm11402168pfq.117.2020.02.28.03.34.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 28 Feb 2020 03:35:01 -0800 (PST) From: Pingfan Liu To: linux-mm@kvack.org Cc: Pingfan Liu , Ira Weiny , Andrew Morton , Mike Rapoport , Dan Williams , Matthew Wilcox , John Hubbard , "Aneesh Kumar K.V" , Keith Busch , Christoph Hellwig , Shuah Khan , linux-kernel@vger.kernel.org Subject: [PATCHv5 2/3] mm/gup: fix omission of check on FOLL_LONGTERM in gup fast path Date: Fri, 28 Feb 2020 19:32:29 +0800 Message-Id: <1582889550-9101-3-git-send-email-kernelfans@gmail.com> X-Mailer: git-send-email 2.7.5 In-Reply-To: <1582889550-9101-1-git-send-email-kernelfans@gmail.com> References: <1582889550-9101-1-git-send-email-kernelfans@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: FOLL_LONGTERM suggests a pin which is going to be given to hardware and can't move. It would truncate CMA permanently and should be excluded. FOLL_LONGTERM has already been checked in the slow path, but not checked in the fast path, which means a possible leak of CMA page to longterm pinned requirement through this crack. Place a check in try_get_compound_head() in the fast path. Some note about the check: Huge page's subpages have the same migrate type due to either allocation from a free_list[] or alloc_contig_range() with param MIGRATE_MOVABLE. So it is enough to check on a single subpage by is_migrate_cma_page(subpage) Signed-off-by: Pingfan Liu Cc: Ira Weiny Cc: Andrew Morton Cc: Mike Rapoport Cc: Dan Williams Cc: Matthew Wilcox Cc: John Hubbard Cc: "Aneesh Kumar K.V" Cc: Keith Busch Cc: Christoph Hellwig Cc: Shuah Khan To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org --- mm/gup.c | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index cd8075e..f0d6804 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -33,9 +33,21 @@ struct follow_page_context { * Return the compound head page with ref appropriately incremented, * or NULL if that failed. */ -static inline struct page *try_get_compound_head(struct page *page, int refs) +static inline struct page *try_get_compound_head(struct page *page, int refs, + unsigned int flags) { - struct page *head = compound_head(page); + struct page *head; + + /* + * Huge page's subpages have the same migrate type due to either + * allocation from a free_list[] or alloc_contig_range() with param + * MIGRATE_MOVABLE. So it is enough to check on a single subpage. + */ + if (unlikely(flags & FOLL_LONGTERM) && + is_migrate_cma_page(page)) + return NULL; + + head = compound_head(page); if (WARN_ON_ONCE(page_ref_count(head) < 0)) return NULL; @@ -1908,7 +1920,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, VM_BUG_ON(!pfn_valid(pte_pfn(pte))); page = pte_page(pte); - head = try_get_compound_head(page, 1); + head = try_get_compound_head(page, 1, flags); if (!head) goto pte_unmap; @@ -2083,7 +2095,7 @@ static int gup_hugepte(pte_t *ptep, unsigned long sz, unsigned long addr, page = head + ((addr & (sz-1)) >> PAGE_SHIFT); refs = record_subpages(page, addr, end, pages + *nr); - head = try_get_compound_head(head, refs); + head = try_get_compound_head(head, refs, flags); if (!head) return 0; @@ -2142,7 +2154,7 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, unsigned long addr, page = pmd_page(orig) + ((addr & ~PMD_MASK) >> PAGE_SHIFT); refs = record_subpages(page, addr, end, pages + *nr); - head = try_get_compound_head(pmd_page(orig), refs); + head = try_get_compound_head(pmd_page(orig), refs, flags); if (!head) return 0; @@ -2174,7 +2186,7 @@ static int gup_huge_pud(pud_t orig, pud_t *pudp, unsigned long addr, page = pud_page(orig) + ((addr & ~PUD_MASK) >> PAGE_SHIFT); refs = record_subpages(page, addr, end, pages + *nr); - head = try_get_compound_head(pud_page(orig), refs); + head = try_get_compound_head(pud_page(orig), refs, flags); if (!head) return 0; @@ -2203,7 +2215,7 @@ static int gup_huge_pgd(pgd_t orig, pgd_t *pgdp, unsigned long addr, page = pgd_page(orig) + ((addr & ~PGDIR_MASK) >> PAGE_SHIFT); refs = record_subpages(page, addr, end, pages + *nr); - head = try_get_compound_head(pgd_page(orig), refs); + head = try_get_compound_head(pgd_page(orig), refs, flags); if (!head) return 0; From patchwork Fri Feb 28 11:32:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 11412049 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 94CA713A4 for ; Fri, 28 Feb 2020 11:35:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 53827246A8 for ; Fri, 28 Feb 2020 11:35:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IQb3ZBk1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 53827246A8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8E6C26B0008; Fri, 28 Feb 2020 06:35:11 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8BDB86B000A; Fri, 28 Feb 2020 06:35:11 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 75E446B000C; Fri, 28 Feb 2020 06:35:11 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0029.hostedemail.com [216.40.44.29]) by kanga.kvack.org (Postfix) with ESMTP id 5EEEC6B0008 for ; Fri, 28 Feb 2020 06:35:11 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 27DE4180AD817 for ; Fri, 28 Feb 2020 11:35:11 +0000 (UTC) X-FDA: 76539329622.30.blood77_6769f0e91f004 X-Spam-Summary: 2,0,0,c43e56780cb0790e,d41d8cd98f00b204,kernelfans@gmail.com,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1345:1359:1431:1437:1535:1543:1711:1730:1747:1777:1792:1801:2393:2559:2562:3138:3139:3140:3141:3142:3354:3865:3867:3870:3871:4117:4321:4605:5007:6119:6261:6653:6742:7514:8603:8634:9413:10004:11026:11473:11657:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:12986:13846:14096:14181:14394:14687:14721:21080:21251:21444:21451:21627:21666:30054:30056:30064:30070,0,RBL:209.85.215.193:@gmail.com:.lbl8.mailshell.net-62.50.0.100 66.100.201.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: blood77_6769f0e91f004 X-Filterd-Recvd-Size: 6259 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Fri, 28 Feb 2020 11:35:10 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id d6so1380845pgn.5 for ; Fri, 28 Feb 2020 03:35:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ouTGWuIUR4ceNSkskbl9Csp7Y8Pui/2+vsjeiQPYtso=; b=IQb3ZBk1vuHvPHxRxGSKJCYTi8d30EqOzK2cHF2dZg+4pwg4qVI45YOAN+3U/mimRQ GAu6KBfoDzJhMJAZsOjhi1sj+qocX8+zCwiHI3UlNzB0BEu7FB0VIQxssualMf3nON+S FzjmVy498QDo9y2ei1T2ZRZhhYiHYEUzXKoY7DklI2j7uFgw2psu+hFQaHRkZ6WK2hWN jyeEamqcwPOpi7fTiZdt+iOGED7GWqk8YuWQeo+7bRU3R+4g96avicc2ZacjMD6vmKdL odfmN5kbK+1fvy1uNTYymPjRDNUU68fY2cFyIxwpN0z1NJPndEC0x7tKRRwtJmM+qpjH VyLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ouTGWuIUR4ceNSkskbl9Csp7Y8Pui/2+vsjeiQPYtso=; b=L/ERm8QshACRUZ9Op5T66pBReN3f04NhS2/HRHw/V5BDJ9Zt+acGHrInbnJnrJFuE3 MxvEBEgYv1B56oUuJcuGUQKSP3s8yFh7g7xoJ8GteyOPH6mOaW8ohGZunVRkPX2SMyG/ zwtNSrovsAXK4StG1GddZrsYjrGb9P3AB5vGj2jBcvijqPuU5aKi6P27YZy9kn6bEWq3 pzivWsBFTwleNq149O5eZ4s77uV+6L+LG4JEQuJj8y9NPLlKLj8vlFzR29b5PrkM4YXX JvBwhHGHFpj2MgKCbmH3Y62MgpmPn9JMV61+ClCpGdT+pwzZfx+qOJNxtSBjWXn/fEaL Qjxw== X-Gm-Message-State: APjAAAVBNPR6/cdY8dQXDq6jb34uKoVpLflu/9o23+IzBn2dgq9KBdSs l0FA0okSOt1on2ZQUW8E/JkLXPfGHA== X-Google-Smtp-Source: APXvYqwT4s9eUrkPHKnI8fgZNrAbSk5HCS7MHbN9UD+OioNjos79T6t0N5IxmNUiH9pBXZYSIbQvuA== X-Received: by 2002:a62:7d16:: with SMTP id y22mr3962132pfc.220.1582889709431; Fri, 28 Feb 2020 03:35:09 -0800 (PST) Received: from mylaptop.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id d14sm11402168pfq.117.2020.02.28.03.35.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 28 Feb 2020 03:35:08 -0800 (PST) From: Pingfan Liu To: linux-mm@kvack.org Cc: Pingfan Liu , Ira Weiny , Andrew Morton , Mike Rapoport , Dan Williams , Matthew Wilcox , John Hubbard , "Aneesh Kumar K.V" , Keith Busch , Christoph Hellwig , Shuah Khan , linux-kernel@vger.kernel.org Subject: [PATCHv5 3/3] mm/gup_benchemark: add LONGTERM_BENCHMARK test in gup fast path Date: Fri, 28 Feb 2020 19:32:30 +0800 Message-Id: <1582889550-9101-4-git-send-email-kernelfans@gmail.com> X-Mailer: git-send-email 2.7.5 In-Reply-To: <1582889550-9101-1-git-send-email-kernelfans@gmail.com> References: <1582889550-9101-1-git-send-email-kernelfans@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce a GUP_LONGTERM_BENCHMARK ioctl to test longterm pin in gup fast path. Signed-off-by: Pingfan Liu Cc: Ira Weiny Cc: Andrew Morton Cc: Mike Rapoport Cc: Dan Williams Cc: Matthew Wilcox Cc: John Hubbard Cc: "Aneesh Kumar K.V" Cc: Keith Busch Cc: Christoph Hellwig Cc: Shuah Khan To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org --- mm/gup_benchmark.c | 7 +++++++ tools/testing/selftests/vm/gup_benchmark.c | 6 +++++- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/mm/gup_benchmark.c b/mm/gup_benchmark.c index 8dba38e..bf61e7a 100644 --- a/mm/gup_benchmark.c +++ b/mm/gup_benchmark.c @@ -8,6 +8,7 @@ #define GUP_FAST_BENCHMARK _IOWR('g', 1, struct gup_benchmark) #define GUP_LONGTERM_BENCHMARK _IOWR('g', 2, struct gup_benchmark) #define GUP_BENCHMARK _IOWR('g', 3, struct gup_benchmark) +#define GUP_FAST_LONGTERM_BENCHMARK _IOWR('g', 4, struct gup_benchmark) struct gup_benchmark { __u64 get_delta_usec; @@ -57,6 +58,11 @@ static int __gup_benchmark_ioctl(unsigned int cmd, nr = get_user_pages_fast(addr, nr, gup->flags, pages + i); break; + case GUP_FAST_LONGTERM_BENCHMARK: + nr = get_user_pages_fast(addr, nr, + (gup->flags & 1) | FOLL_LONGTERM, + pages + i); + break; case GUP_LONGTERM_BENCHMARK: nr = get_user_pages(addr, nr, gup->flags | FOLL_LONGTERM, @@ -103,6 +109,7 @@ static long gup_benchmark_ioctl(struct file *filep, unsigned int cmd, switch (cmd) { case GUP_FAST_BENCHMARK: + case GUP_FAST_LONGTERM_BENCHMARK: case GUP_LONGTERM_BENCHMARK: case GUP_BENCHMARK: break; diff --git a/tools/testing/selftests/vm/gup_benchmark.c b/tools/testing/selftests/vm/gup_benchmark.c index 389327e..5a01c538 100644 --- a/tools/testing/selftests/vm/gup_benchmark.c +++ b/tools/testing/selftests/vm/gup_benchmark.c @@ -17,6 +17,7 @@ #define GUP_FAST_BENCHMARK _IOWR('g', 1, struct gup_benchmark) #define GUP_LONGTERM_BENCHMARK _IOWR('g', 2, struct gup_benchmark) #define GUP_BENCHMARK _IOWR('g', 3, struct gup_benchmark) +#define GUP_FAST_LONGTERM_BENCHMARK _IOWR('g', 4, struct gup_benchmark) /* Just the flags we need, copied from mm.h: */ #define FOLL_WRITE 0x01 /* check pte is writable */ @@ -40,7 +41,7 @@ int main(int argc, char **argv) char *file = "/dev/zero"; char *p; - while ((opt = getopt(argc, argv, "m:r:n:f:tTLUwSH")) != -1) { + while ((opt = getopt(argc, argv, "m:r:n:f:tTlLUwSH")) != -1) { switch (opt) { case 'm': size = atoi(optarg) * MB; @@ -57,6 +58,9 @@ int main(int argc, char **argv) case 'T': thp = 0; break; + case 'l': + cmd = GUP_FAST_LONGTERM_BENCHMARK; + break; case 'L': cmd = GUP_LONGTERM_BENCHMARK; break;