From patchwork Thu Feb 20 19:53:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11394889 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7086192A for ; Thu, 20 Feb 2020 19:54:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 33A01206F4 for ; Thu, 20 Feb 2020 19:54:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BpdBUtq8" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 33A01206F4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 700E36B0008; Thu, 20 Feb 2020 14:54:04 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6D76A6B000A; Thu, 20 Feb 2020 14:54:04 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5ED3C6B000C; Thu, 20 Feb 2020 14:54:04 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0187.hostedemail.com [216.40.44.187]) by kanga.kvack.org (Postfix) with ESMTP id 443F26B0008 for ; Thu, 20 Feb 2020 14:54:04 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 0B138180AD822 for ; Thu, 20 Feb 2020 19:54:04 +0000 (UTC) X-FDA: 76511556408.18.wrist31_408ae70d69422 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,,RULES_HIT:30054:30070,0,RBL:207.211.31.120:@redhat.com:.lbl8.mailshell.net-62.18.0.100 66.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:27,LUA_SUMMARY:none X-HE-Tag: wrist31_408ae70d69422 X-Filterd-Recvd-Size: 7316 Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Thu, 20 Feb 2020 19:54:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582228442; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=snp10lQxapAu2E6369t9Wb8KDKBuJc75vFCmG4PcxNA=; b=BpdBUtq8snlvnbjBKHS6LGg3rs5OHpps+HHXh29Hl3FdUczrGGuFMEegeecS63NnjJx/aR /muQndMiX1vdeZXrg/WWa0n8hYqSQcwhLxuAmdBUGvmyknw6AO2HWA9v17LjbEoFL3udHj zl9ikS4tL3eTzuaJnWgocHa7Jn/VfSA= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-315-zJvP9c5iPDa2NNQtesqbPA-1; Thu, 20 Feb 2020 14:54:00 -0500 Received: by mail-qv1-f70.google.com with SMTP id v19so3316364qvk.16 for ; Thu, 20 Feb 2020 11:54:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Fez1YLYoRTejKdl3XovFZRqHL8yeL2JwUep3jNimnQM=; b=b/dqmb3DzboFFyfRn/nBuRl+5nDnzb8TedBX8T1Ats4TkyQe7vn6j5wm2UPyCTvdfq 7kZb1Ei/rgKUgaLYmo4VhUJuCPENaDygzB77SNdNrH2T6JDt8Xsr1KD4MffH49QjmvgZ id21774yMH7VvTbbtwnkllTOc7yJfcM3gOgg0yQDdaIr/HYssswBzsm1qbEsXhiD2zZw jnUPS4vI/mCDIRE6BMOJLuO2OYxdQFztoACMj+L5QsWgBoR8wCvg796v7rsHSMCH5kRg wLTyZcW0QRX3OeWAGLrZdacyQLgAfsLbcT2URNWYkL+sh/qs4ekL2w2UT/0DRmVpHZgS bCtQ== X-Gm-Message-State: APjAAAXXtmvwKqhoEpXHg029SQPF0o2clB8fh/HINfhGnabX4PMkqDeS kcmqWCLsXNgTELpPnRVQXRksHkYpkK9xyaMITQJld6tISBc2izbUvLZ9qgS+mk2qB6eCo1iBP6Z S/7ml4gjs4D4= X-Received: by 2002:a05:6214:1750:: with SMTP id dc16mr27322415qvb.47.1582228439903; Thu, 20 Feb 2020 11:53:59 -0800 (PST) X-Google-Smtp-Source: APXvYqwEIhUPaSeXK2C2o3Bv58EPeWBkwTGMcKqMaRd3x8T+f6afhWOfRCq+hk0nYhl6E3hyCskSsQ== X-Received: by 2002:a05:6214:1750:: with SMTP id dc16mr27322394qvb.47.1582228439636; Thu, 20 Feb 2020 11:53:59 -0800 (PST) Received: from xz-x1.redhat.com ([104.156.64.75]) by smtp.gmail.com with ESMTPSA id a192sm332776qkb.53.2020.02.20.11.53.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Feb 2020 11:53:59 -0800 (PST) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Peter Xu , Martin Cracauer , Mike Rapoport , Hugh Dickins , Jerome Glisse , "Kirill A . Shutemov" , Matthew Wilcox , Pavel Emelyanov , Brian Geffon , Maya Gokhale , Denis Plotnikov , Andrea Arcangeli , Johannes Weiner , "Dr . David Alan Gilbert" , Linus Torvalds , Mike Kravetz , Marty McFadden , David Hildenbrand , Bobby Powers , Mel Gorman Subject: [PATCH RESEND v6 14/16] mm/gup: Allow VM_FAULT_RETRY for multiple times Date: Thu, 20 Feb 2020 14:53:57 -0500 Message-Id: <20200220195357.16371-1-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200220155353.8676-1-peterx@redhat.com> References: MIME-Version: 1.0 X-MC-Unique: zJvP9c5iPDa2NNQtesqbPA-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is the gup counterpart of the change that allows the VM_FAULT_RETRY to happen for more than once. One thing to mention is that we must check the fatal signal here before retry because the GUP can be interrupted by that, otherwise we can loop forever. Signed-off-by: Peter Xu --- mm/gup.c | 27 +++++++++++++++++++++------ mm/hugetlb.c | 6 ++++-- 2 files changed, 25 insertions(+), 8 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 76cb420c0fb7..ec2b76f44a01 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -652,7 +652,10 @@ static int faultin_page(struct task_struct *tsk, struct vm_area_struct *vma, if (*flags & FOLL_NOWAIT) fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_RETRY_NOWAIT; if (*flags & FOLL_TRIED) { - VM_WARN_ON_ONCE(fault_flags & FAULT_FLAG_ALLOW_RETRY); + /* + * Note: FAULT_FLAG_ALLOW_RETRY and FAULT_FLAG_TRIED + * can co-exist + */ fault_flags |= FAULT_FLAG_TRIED; } @@ -1012,7 +1015,6 @@ int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm, down_read(&mm->mmap_sem); if (!(fault_flags & FAULT_FLAG_TRIED)) { *unlocked = true; - fault_flags &= ~FAULT_FLAG_ALLOW_RETRY; fault_flags |= FAULT_FLAG_TRIED; goto retry; } @@ -1096,17 +1098,30 @@ static __always_inline long __get_user_pages_locked(struct task_struct *tsk, if (likely(pages)) pages += ret; start += ret << PAGE_SHIFT; + lock_dropped = true; +retry: /* * Repeat on the address that fired VM_FAULT_RETRY - * without FAULT_FLAG_ALLOW_RETRY but with - * FAULT_FLAG_TRIED. + * with both FAULT_FLAG_ALLOW_RETRY and + * FAULT_FLAG_TRIED. Note that GUP can be interrupted + * by fatal signals, so we need to check it before we + * start trying again otherwise it can loop forever. */ + + if (fatal_signal_pending(current)) + break; + *locked = 1; - lock_dropped = true; down_read(&mm->mmap_sem); + ret = __get_user_pages(tsk, mm, start, 1, flags | FOLL_TRIED, - pages, NULL, NULL); + pages, NULL, locked); + if (!*locked) { + /* Continue to retry until we succeeded */ + BUG_ON(ret != 0); + goto retry; + } if (ret != 1) { BUG_ON(ret > 1); if (!pages_done) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index c84f721db020..ac9a28d51674 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -4343,8 +4343,10 @@ long follow_hugetlb_page(struct mm_struct *mm, struct vm_area_struct *vma, fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_RETRY_NOWAIT; if (flags & FOLL_TRIED) { - VM_WARN_ON_ONCE(fault_flags & - FAULT_FLAG_ALLOW_RETRY); + /* + * Note: FAULT_FLAG_ALLOW_RETRY and + * FAULT_FLAG_TRIED can co-exist + */ fault_flags |= FAULT_FLAG_TRIED; } ret = hugetlb_fault(mm, vma, vaddr, fault_flags);