From patchwork Mon Sep 23 04:25:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11156035 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4659014E5 for ; Mon, 23 Sep 2019 04:26:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0FCF1208C2 for ; Mon, 23 Sep 2019 04:26:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0FCF1208C2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 443C16B0275; Mon, 23 Sep 2019 00:26:42 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4198F6B0276; Mon, 23 Sep 2019 00:26:42 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 308506B0277; Mon, 23 Sep 2019 00:26:42 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0116.hostedemail.com [216.40.44.116]) by kanga.kvack.org (Postfix) with ESMTP id 0F6F36B0275 for ; Mon, 23 Sep 2019 00:26:42 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id B8893181AC9AE for ; Mon, 23 Sep 2019 04:26:41 +0000 (UTC) X-FDA: 75964899402.25.rake37_74971f4b25c03 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:mcfadden8@llnl.gov:shli@fb.com:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30054:30070,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: rake37_74971f4b25c03 X-Filterd-Recvd-Size: 6201 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf16.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 04:26:41 +0000 (UTC) Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3F04B58569 for ; Mon, 23 Sep 2019 04:26:40 +0000 (UTC) Received: by mail-pg1-f199.google.com with SMTP id b12so8620729pgm.14 for ; Sun, 22 Sep 2019 21:26:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AKkEfjdoHGbVoKbr4xUEMS07lQumUdJTKR+cWS6/cLc=; b=g4zO3I3zFqtI5m8pbOhw7iOSRfXd+AadZzBFSU+uQDkrAa/rhc2A7zwPO8JdGT+Wdh 5UI9MxTT1lsdJluSQ6igjSDVFgj2qGTbskIkujX/EUoiUvYnIDfxKzvH+eUP77zfFfxR MCtpECqWmGJgDhMwrgldrgUZ494fzqTfNaR6cqIiaT6njKjRC/EaBX99yWxMzWU4fH/K tyYw6ZB+SoOl0nh2PYxHuPRh4EI8BMhFdAlhqbRwG7QdeqW/I2I2qmR8u3mt3yjzQMsm 3QF2C5XszdLlJ/cZNRJivDh9KVL3yG6UhA1QzfuZ8Fq5oaPgkGfHjcR1BYgORaWy3YK2 eyGw== X-Gm-Message-State: APjAAAW2mSskrqyw+khT2k4suUFWhnN7u/Q5QUAW5RlCbJEud45fq/Va rcQlvi2hT8u+rCUgjcV84DDlB8yuMgt+SsB7DSfTBxIw5IYpiZHDtW2HIeVOt1sGsgJVKfcGTH0 XJX2SOtEguVA= X-Received: by 2002:a63:f745:: with SMTP id f5mr12778398pgk.175.1569212798978; Sun, 22 Sep 2019 21:26:38 -0700 (PDT) X-Google-Smtp-Source: APXvYqzCmcEfrc8QuXpBvHO5rswwqPBwY/o99XMn56GJV9UHiDMAd1lbEY/N2Jk/9XaUVCdbc3jZ2w== X-Received: by 2002:a63:f745:: with SMTP id f5mr12778387pgk.175.1569212798740; Sun, 22 Sep 2019 21:26:38 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id x20sm11781867pfp.120.2019.09.22.21.26.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Sep 2019 21:26:38 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Marty McFadden , Shaohua Li , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v4 08/10] mm/gup: Allow VM_FAULT_RETRY for multiple times Date: Mon, 23 Sep 2019 12:25:21 +0800 Message-Id: <20190923042523.10027-9-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190923042523.10027-1-peterx@redhat.com> References: <20190923042523.10027-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is the gup counterpart of the change that allows the VM_FAULT_RETRY to happen for more than once. One thing to mention is that we must check the fatal signal here before retry because the GUP can be interrupted by that, otherwise we can loop forever. Signed-off-by: Peter Xu --- mm/gup.c | 27 +++++++++++++++++++++------ mm/hugetlb.c | 6 ++++-- 2 files changed, 25 insertions(+), 8 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index e60d32f1674d..d2811bb15a25 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -644,7 +644,10 @@ static int faultin_page(struct task_struct *tsk, struct vm_area_struct *vma, if (*flags & FOLL_NOWAIT) fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_RETRY_NOWAIT; if (*flags & FOLL_TRIED) { - VM_WARN_ON_ONCE(fault_flags & FAULT_FLAG_ALLOW_RETRY); + /* + * Note: FAULT_FLAG_ALLOW_RETRY and FAULT_FLAG_TRIED + * can co-exist + */ fault_flags |= FAULT_FLAG_TRIED; } @@ -994,7 +997,6 @@ int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm, down_read(&mm->mmap_sem); if (!(fault_flags & FAULT_FLAG_TRIED)) { *unlocked = true; - fault_flags &= ~FAULT_FLAG_ALLOW_RETRY; fault_flags |= FAULT_FLAG_TRIED; goto retry; } @@ -1069,17 +1071,30 @@ static __always_inline long __get_user_pages_locked(struct task_struct *tsk, if (likely(pages)) pages += ret; start += ret << PAGE_SHIFT; + lock_dropped = true; +retry: /* * Repeat on the address that fired VM_FAULT_RETRY - * without FAULT_FLAG_ALLOW_RETRY but with - * FAULT_FLAG_TRIED. + * with both FAULT_FLAG_ALLOW_RETRY and + * FAULT_FLAG_TRIED. Note that GUP can be interrupted + * by fatal signals, so we need to check it before we + * start trying again otherwise it can loop forever. */ + + if (fatal_signal_pending(current)) + break; + *locked = 1; - lock_dropped = true; down_read(&mm->mmap_sem); + ret = __get_user_pages(tsk, mm, start, 1, flags | FOLL_TRIED, - pages, NULL, NULL); + pages, NULL, locked); + if (!*locked) { + /* Continue to retry until we succeeded */ + BUG_ON(ret != 0); + goto retry; + } if (ret != 1) { BUG_ON(ret > 1); if (!pages_done) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 31c2a6275023..d0c98cff5b0f 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -4347,8 +4347,10 @@ long follow_hugetlb_page(struct mm_struct *mm, struct vm_area_struct *vma, fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_RETRY_NOWAIT; if (flags & FOLL_TRIED) { - VM_WARN_ON_ONCE(fault_flags & - FAULT_FLAG_ALLOW_RETRY); + /* + * Note: FAULT_FLAG_ALLOW_RETRY and + * FAULT_FLAG_TRIED can co-exist + */ fault_flags |= FAULT_FLAG_TRIED; } ret = hugetlb_fault(mm, vma, vaddr, fault_flags);