From patchwork Thu Sep 26 09:38:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162011 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C241A76 for ; Thu, 26 Sep 2019 09:39:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7E5EA222BE for ; Thu, 26 Sep 2019 09:39:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7E5EA222BE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AFB826B0008; Thu, 26 Sep 2019 05:39:30 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AABEF6B000A; Thu, 26 Sep 2019 05:39:30 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 999A56B000C; Thu, 26 Sep 2019 05:39:30 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0135.hostedemail.com [216.40.44.135]) by kanga.kvack.org (Postfix) with ESMTP id 774326B0008 for ; Thu, 26 Sep 2019 05:39:30 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id EAF68180AD803 for ; Thu, 26 Sep 2019 09:39:29 +0000 (UTC) X-FDA: 75976574058.20.hate42_6ff7684735135 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30003:30029:30034:30054:30070:30090,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: hate42_6ff7684735135 X-Filterd-Recvd-Size: 11250 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:39:29 +0000 (UTC) Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6D51A7FDF4 for ; Thu, 26 Sep 2019 09:39:28 +0000 (UTC) Received: by mail-pl1-f199.google.com with SMTP id j9so1155076plk.21 for ; Thu, 26 Sep 2019 02:39:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oIX66W7ZMb/5JfByU6MkGyWo2hWcSJBtpdabai9VliU=; b=NBOd3ytaoZoIxInemRJAC3/RuIeKZfvJDnVex3aH1hxARsD1IEE4LeyNrivxvEbGpv d/vtY4sdh0hxOVDKuso12QnKMjdwh1bCVPYfP1fZ1UQFnf7dtwZwOo04hy+ekgbCd+3P bfHA4tIcCemAMGq4pYYI3G0haAnLcV3GuW79TbgBGqJ72yGJIp0MXxroqqXyyBNm3BSi IfuwufjzIeAM3qnHBEyaXNwRi2VqIh2BZCvkoqM0A5FkcGRM2spRW2oM+VBdi3BtmKGq hoygL2LiKeRISqVN0jsxRjw9xoJfyhh9MnfnBi09sZqsF0eYKeWap5Ol0zfgNWxY0n0P rkEA== X-Gm-Message-State: APjAAAW6sC5gDao1sNt0y1mo4p/zGmTqnTfEM3Iy7JSphjnFi4mLlRum 2QTuoR1rhEWJZERkRim8GupgqEb2cVBdHEvxc30ERz1raEiuYPRQHpENSVgRBfxTO0tu2T5ng8W cfoiODvlM0u8= X-Received: by 2002:a17:90a:cb98:: with SMTP id a24mr2551193pju.124.1569490767471; Thu, 26 Sep 2019 02:39:27 -0700 (PDT) X-Google-Smtp-Source: APXvYqy8aT4CcThSGdj+5Mz3Q1ObKDEtmk9mXr3d+5VGKGsJuH5RpN1jp3FV1+mPDUvH/nesBEObkw== X-Received: by 2002:a17:90a:cb98:: with SMTP id a24mr2551160pju.124.1569490767173; Thu, 26 Sep 2019 02:39:27 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.39.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:39:26 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 01/16] mm/gup: Rename "nonblocking" to "locked" where proper Date: Thu, 26 Sep 2019 17:38:49 +0800 Message-Id: <20190926093904.5090-2-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There's plenty of places around __get_user_pages() that has a parameter "nonblocking" which does not really mean that "it won't block" (because it can really block) but instead it shows whether the mmap_sem is released by up_read() during the page fault handling mostly when VM_FAULT_RETRY is returned. We have the correct naming in e.g. get_user_pages_locked() or get_user_pages_remote() as "locked", however there're still many places that are using the "nonblocking" as name. Renaming the places to "locked" where proper to better suite the functionality of the variable. While at it, fixing up some of the comments accordingly. Reviewed-by: Mike Rapoport Reviewed-by: Jerome Glisse Reviewed-by: David Hildenbrand Signed-off-by: Peter Xu --- mm/gup.c | 44 +++++++++++++++++++++----------------------- mm/hugetlb.c | 8 ++++---- 2 files changed, 25 insertions(+), 27 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 98f13ab37bac..eddbb95dcb8f 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -622,12 +622,12 @@ static int get_gate_page(struct mm_struct *mm, unsigned long address, } /* - * mmap_sem must be held on entry. If @nonblocking != NULL and - * *@flags does not include FOLL_NOWAIT, the mmap_sem may be released. - * If it is, *@nonblocking will be set to 0 and -EBUSY returned. + * mmap_sem must be held on entry. If @locked != NULL and *@flags + * does not include FOLL_NOWAIT, the mmap_sem may be released. If it + * is, *@locked will be set to 0 and -EBUSY returned. */ static int faultin_page(struct task_struct *tsk, struct vm_area_struct *vma, - unsigned long address, unsigned int *flags, int *nonblocking) + unsigned long address, unsigned int *flags, int *locked) { unsigned int fault_flags = 0; vm_fault_t ret; @@ -639,7 +639,7 @@ static int faultin_page(struct task_struct *tsk, struct vm_area_struct *vma, fault_flags |= FAULT_FLAG_WRITE; if (*flags & FOLL_REMOTE) fault_flags |= FAULT_FLAG_REMOTE; - if (nonblocking) + if (locked) fault_flags |= FAULT_FLAG_ALLOW_RETRY; if (*flags & FOLL_NOWAIT) fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_RETRY_NOWAIT; @@ -665,8 +665,8 @@ static int faultin_page(struct task_struct *tsk, struct vm_area_struct *vma, } if (ret & VM_FAULT_RETRY) { - if (nonblocking && !(fault_flags & FAULT_FLAG_RETRY_NOWAIT)) - *nonblocking = 0; + if (locked && !(fault_flags & FAULT_FLAG_RETRY_NOWAIT)) + *locked = 0; return -EBUSY; } @@ -743,7 +743,7 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags) * only intends to ensure the pages are faulted in. * @vmas: array of pointers to vmas corresponding to each page. * Or NULL if the caller does not require them. - * @nonblocking: whether waiting for disk IO or mmap_sem contention + * @locked: whether we're still with the mmap_sem held * * Returns number of pages pinned. This may be fewer than the number * requested. If nr_pages is 0 or negative, returns 0. If no pages @@ -772,13 +772,11 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags) * appropriate) must be called after the page is finished with, and * before put_page is called. * - * If @nonblocking != NULL, __get_user_pages will not wait for disk IO - * or mmap_sem contention, and if waiting is needed to pin all pages, - * *@nonblocking will be set to 0. Further, if @gup_flags does not - * include FOLL_NOWAIT, the mmap_sem will be released via up_read() in - * this case. + * If @locked != NULL, *@locked will be set to 0 when mmap_sem is + * released by an up_read(). That can happen if @gup_flags does not + * have FOLL_NOWAIT. * - * A caller using such a combination of @nonblocking and @gup_flags + * A caller using such a combination of @locked and @gup_flags * must therefore hold the mmap_sem for reading only, and recognize * when it's been released. Otherwise, it must be held for either * reading or writing and will not be released. @@ -790,7 +788,7 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags) static long __get_user_pages(struct task_struct *tsk, struct mm_struct *mm, unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, - struct vm_area_struct **vmas, int *nonblocking) + struct vm_area_struct **vmas, int *locked) { long ret = 0, i = 0; struct vm_area_struct *vma = NULL; @@ -834,7 +832,7 @@ static long __get_user_pages(struct task_struct *tsk, struct mm_struct *mm, if (is_vm_hugetlb_page(vma)) { i = follow_hugetlb_page(mm, vma, pages, vmas, &start, &nr_pages, i, - gup_flags, nonblocking); + gup_flags, locked); continue; } } @@ -852,7 +850,7 @@ static long __get_user_pages(struct task_struct *tsk, struct mm_struct *mm, page = follow_page_mask(vma, start, foll_flags, &ctx); if (!page) { ret = faultin_page(tsk, vma, start, &foll_flags, - nonblocking); + locked); switch (ret) { case 0: goto retry; @@ -1178,7 +1176,7 @@ EXPORT_SYMBOL(get_user_pages_remote); * @vma: target vma * @start: start address * @end: end address - * @nonblocking: + * @locked: whether the mmap_sem is still held * * This takes care of mlocking the pages too if VM_LOCKED is set. * @@ -1186,14 +1184,14 @@ EXPORT_SYMBOL(get_user_pages_remote); * * vma->vm_mm->mmap_sem must be held. * - * If @nonblocking is NULL, it may be held for read or write and will + * If @locked is NULL, it may be held for read or write and will * be unperturbed. * - * If @nonblocking is non-NULL, it must held for read only and may be - * released. If it's released, *@nonblocking will be set to 0. + * If @locked is non-NULL, it must held for read only and may be + * released. If it's released, *@locked will be set to 0. */ long populate_vma_page_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end, int *nonblocking) + unsigned long start, unsigned long end, int *locked) { struct mm_struct *mm = vma->vm_mm; unsigned long nr_pages = (end - start) / PAGE_SIZE; @@ -1228,7 +1226,7 @@ long populate_vma_page_range(struct vm_area_struct *vma, * not result in a stack expansion that recurses back here. */ return __get_user_pages(current, mm, start, nr_pages, gup_flags, - NULL, NULL, nonblocking); + NULL, NULL, locked); } /* diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 6d7296dd11b8..31c2a6275023 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -4270,7 +4270,7 @@ int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, long follow_hugetlb_page(struct mm_struct *mm, struct vm_area_struct *vma, struct page **pages, struct vm_area_struct **vmas, unsigned long *position, unsigned long *nr_pages, - long i, unsigned int flags, int *nonblocking) + long i, unsigned int flags, int *locked) { unsigned long pfn_offset; unsigned long vaddr = *position; @@ -4341,7 +4341,7 @@ long follow_hugetlb_page(struct mm_struct *mm, struct vm_area_struct *vma, spin_unlock(ptl); if (flags & FOLL_WRITE) fault_flags |= FAULT_FLAG_WRITE; - if (nonblocking) + if (locked) fault_flags |= FAULT_FLAG_ALLOW_RETRY; if (flags & FOLL_NOWAIT) fault_flags |= FAULT_FLAG_ALLOW_RETRY | @@ -4358,9 +4358,9 @@ long follow_hugetlb_page(struct mm_struct *mm, struct vm_area_struct *vma, break; } if (ret & VM_FAULT_RETRY) { - if (nonblocking && + if (locked && !(fault_flags & FAULT_FLAG_RETRY_NOWAIT)) - *nonblocking = 0; + *locked = 0; *nr_pages = 0; /* * VM_FAULT_RETRY must not return an From patchwork Thu Sep 26 09:38:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162013 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A118576 for ; Thu, 26 Sep 2019 09:39:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6D54D222C5 for ; Thu, 26 Sep 2019 09:39:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6D54D222C5 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AD38D6B000A; Thu, 26 Sep 2019 05:39:37 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A845D6B000D; Thu, 26 Sep 2019 05:39:37 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A25D6B000A; Thu, 26 Sep 2019 05:39:37 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0099.hostedemail.com [216.40.44.99]) by kanga.kvack.org (Postfix) with ESMTP id 7739F6B000A for ; Thu, 26 Sep 2019 05:39:37 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 2700C1A4B4 for ; Thu, 26 Sep 2019 09:39:37 +0000 (UTC) X-FDA: 75976574394.16.son45_70fe8c249042c X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30054:30070,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:38,LUA_SUMMARY:none X-HE-Tag: son45_70fe8c249042c X-Filterd-Recvd-Size: 4269 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:39:36 +0000 (UTC) Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9A25BC06512C for ; Thu, 26 Sep 2019 09:39:35 +0000 (UTC) Received: by mail-pg1-f197.google.com with SMTP id o32so1059242pgm.18 for ; Thu, 26 Sep 2019 02:39:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nL20lZzBbwkIGL5V1u5lEN6f2V609pKVyYfWwZ4NMZQ=; b=SB1XH4zhd5QtAXOI7sSlmy5d79nmr7cjZ4GbZKhXmqkQeSjug7xq8QKjNIyyN6ii4k gwN0PfxvB/w9KO9GVy2T+66rBQmsNxKG3bBgDux5jDG/UyuMdjxe1jpUKKjjthWxuqeE ZislyMPRBVcwpWBSRyAyqQe/6olwTAyGceZB2nkJj8BEVLfTwEjYkBc1OzXxDn7NDcVQ LWdxO8O8iorRVzJ7zk1XsshIRjKnslq4C9z2J0sJDHf1Y6oq1PIH5yM+j7VV7oPL89CU Vlk3B5PaKJpU2vqKZGdBw2hQ4PT1aICPXM9gHObrS7qePwHpWsiqelw1+t+qMzrB8Nw+ 1G1Q== X-Gm-Message-State: APjAAAU3DEAtfa+YkoG24UHzWJJEDrDaJ4DQN6qTfKXoq04y0lgmp957 fQxRkgpLfw6q1sOkyuJpTU6ONoXlz7WLcrJZjP5c+enbOqKOanp51t190CuBmH4W7ZwVYZqSGH1 59wbsvVO4n8E= X-Received: by 2002:a63:5807:: with SMTP id m7mr2441415pgb.371.1569490774573; Thu, 26 Sep 2019 02:39:34 -0700 (PDT) X-Google-Smtp-Source: APXvYqygg2Fz4n5y2GLmYVjuAatu7ERh1+nfGR8vHRiW6krMcMD+JGfGkv1cX87Z1ZdlzHDovLsjPg== X-Received: by 2002:a63:5807:: with SMTP id m7mr2441382pgb.371.1569490774350; Thu, 26 Sep 2019 02:39:34 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.39.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:39:33 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 02/16] mm/gup: Fix __get_user_pages() on fault retry of hugetlb Date: Thu, 26 Sep 2019 17:38:50 +0800 Message-Id: <20190926093904.5090-3-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When follow_hugetlb_page() returns with *locked==0, it means we've got a VM_FAULT_RETRY within the fauling process and we've released the mmap_sem. When that happens, we should stop and bail out. Signed-off-by: Peter Xu --- mm/gup.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/mm/gup.c b/mm/gup.c index eddbb95dcb8f..e60d32f1674d 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -833,6 +833,16 @@ static long __get_user_pages(struct task_struct *tsk, struct mm_struct *mm, i = follow_hugetlb_page(mm, vma, pages, vmas, &start, &nr_pages, i, gup_flags, locked); + if (locked && *locked == 0) { + /* + * We've got a VM_FAULT_RETRY + * and we've lost mmap_sem. + * We must stop here. + */ + BUG_ON(gup_flags & FOLL_NOWAIT); + BUG_ON(ret != 0); + goto out; + } continue; } } From patchwork Thu Sep 26 09:38:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162015 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4AACC924 for ; Thu, 26 Sep 2019 09:39:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 073F1222BE for ; Thu, 26 Sep 2019 09:39:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 073F1222BE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 249BA6B000C; Thu, 26 Sep 2019 05:39:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 21E976B000D; Thu, 26 Sep 2019 05:39:46 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 10EE96B000E; Thu, 26 Sep 2019 05:39:46 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0252.hostedemail.com [216.40.44.252]) by kanga.kvack.org (Postfix) with ESMTP id DE9C96B000C for ; Thu, 26 Sep 2019 05:39:45 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 97B0F2EC5F for ; Thu, 26 Sep 2019 09:39:45 +0000 (UTC) X-FDA: 75976574730.06.crown86_7241540401f55 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30003:30012:30045:30054:30070:30091,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:0,LUA_SUMMARY:none X-HE-Tag: crown86_7241540401f55 X-Filterd-Recvd-Size: 14665 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:39:45 +0000 (UTC) Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3C5DC56F9 for ; Thu, 26 Sep 2019 09:39:44 +0000 (UTC) Received: by mail-pf1-f197.google.com with SMTP id m25so1336029pfa.23 for ; Thu, 26 Sep 2019 02:39:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8Xf2mVS9qjlemJ5iahoMhXHUO104cWay/UtduMM59II=; b=b6JOZbApZ/ZKg7dZEAvp77D8lSWv0+Aer7/1SWtpwTAp5VmYr3fWiiVxlMb8gKSJY8 q1kcB7sXqa9cHAEyrLP5kcdVngrKEQOndZNsz7gpgbfJT7xxhYfdL1jQKa0lITjLlIsA JMeF/jtKOsYG/QJg7ED6xcs8yX12uT1Fct8uEdc/TjfNZNDaDF9vZmfT0Agnj+1BOgdi 41RdIa/mQn4YeEwEYJKVw3bwsVHUgAdzlIoPCMIE5JKOyiOVnHNsezCIWVgksT3jbQcV dFwuI5FZw2wq9iyKbH2uSb0nOKgPnzcaLXrNCN2AENiXWHK0fozmnBeq7zVGu4VOtYNZ IzPg== X-Gm-Message-State: APjAAAWb5Xs/wB5mDP9N5ZIM8JZs18dmerKf/BIK7eCCHU3facGFcHv1 M3SOWQixbyoYZqjfhJmaWPY9nKlZQjV1+nDh2zAB50+2TKsFfUe2EGEVH6z3XJuk0ZcIR2oBm6h 0PiL7+X8gZvs= X-Received: by 2002:a17:902:7615:: with SMTP id k21mr3121425pll.116.1569490782953; Thu, 26 Sep 2019 02:39:42 -0700 (PDT) X-Google-Smtp-Source: APXvYqxHW0fwK+IoUchcWFPvMLpv6AvUXTp+y8D8ocp6GpWly4yG9DEolwsrBfK/u0buOfL3Ie3qTg== X-Received: by 2002:a17:902:7615:: with SMTP id k21mr3121390pll.116.1569490782632; Thu, 26 Sep 2019 02:39:42 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.39.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:39:41 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 03/16] mm: Introduce fault_signal_pending() Date: Thu, 26 Sep 2019 17:38:51 +0800 Message-Id: <20190926093904.5090-4-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For most architectures, we've got a quick path to detect fatal signal after a handle_mm_fault(). Introduce a helper for that quick path. It cleans the current codes a bit so we don't need to duplicate the same check across archs. More importantly, this will be an unified place that we handle the signal immediately right after an interrupted page fault, so it'll be much easier for us if we want to change the behavior of handling signals later on for all the archs. Note that currently only part of the archs are using this new helper, because some archs have their own way to handle signals. In the follow up patches, we'll try to apply this helper to all the rest of archs. Another note is that the "regs" parameter in the new helper is not used yet. It'll be used very soon. Now we kept it in this patch only to avoid touching all the archs again in the follow up patches. Signed-off-by: Peter Xu --- arch/alpha/mm/fault.c | 2 +- arch/arm/mm/fault.c | 2 +- arch/hexagon/mm/vm_fault.c | 2 +- arch/ia64/mm/fault.c | 2 +- arch/m68k/mm/fault.c | 2 +- arch/microblaze/mm/fault.c | 2 +- arch/mips/mm/fault.c | 2 +- arch/nds32/mm/fault.c | 2 +- arch/nios2/mm/fault.c | 2 +- arch/openrisc/mm/fault.c | 2 +- arch/parisc/mm/fault.c | 2 +- arch/riscv/mm/fault.c | 2 +- arch/s390/mm/fault.c | 3 +-- arch/sparc/mm/fault_32.c | 2 +- arch/sparc/mm/fault_64.c | 2 +- arch/unicore32/mm/fault.c | 2 +- arch/xtensa/mm/fault.c | 2 +- include/linux/sched/signal.h | 13 +++++++++++++ 18 files changed, 30 insertions(+), 18 deletions(-) diff --git a/arch/alpha/mm/fault.c b/arch/alpha/mm/fault.c index 741e61ef9d3f..aea33b599037 100644 --- a/arch/alpha/mm/fault.c +++ b/arch/alpha/mm/fault.c @@ -150,7 +150,7 @@ do_page_fault(unsigned long address, unsigned long mmcsr, the fault. */ fault = handle_mm_fault(vma, address, flags); - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) + if (fault_signal_pending(fault, regs)) return; if (unlikely(fault & VM_FAULT_ERROR)) { diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index 890eeaac3cbb..c8f2950fa733 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -295,7 +295,7 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) * signal first. We do not need to release the mmap_sem because * it would already be released in __lock_page_or_retry in * mm/filemap.c. */ - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) { + if (fault_signal_pending(fault, regs)) { if (!user_mode(regs)) goto no_context; return 0; diff --git a/arch/hexagon/mm/vm_fault.c b/arch/hexagon/mm/vm_fault.c index b3bc71680ae4..d19beaf11b4c 100644 --- a/arch/hexagon/mm/vm_fault.c +++ b/arch/hexagon/mm/vm_fault.c @@ -91,7 +91,7 @@ void do_page_fault(unsigned long address, long cause, struct pt_regs *regs) fault = handle_mm_fault(vma, address, flags); - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) + if (fault_signal_pending(fault, regs)) return; /* The most common case -- we are done. */ diff --git a/arch/ia64/mm/fault.c b/arch/ia64/mm/fault.c index c2f299fe9e04..211b4f439384 100644 --- a/arch/ia64/mm/fault.c +++ b/arch/ia64/mm/fault.c @@ -141,7 +141,7 @@ ia64_do_page_fault (unsigned long address, unsigned long isr, struct pt_regs *re */ fault = handle_mm_fault(vma, address, flags); - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) + if (fault_signal_pending(fault, regs)) return; if (unlikely(fault & VM_FAULT_ERROR)) { diff --git a/arch/m68k/mm/fault.c b/arch/m68k/mm/fault.c index e9b1d7585b43..a455e202691b 100644 --- a/arch/m68k/mm/fault.c +++ b/arch/m68k/mm/fault.c @@ -138,7 +138,7 @@ int do_page_fault(struct pt_regs *regs, unsigned long address, fault = handle_mm_fault(vma, address, flags); pr_debug("handle_mm_fault returns %x\n", fault); - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) + if (fault_signal_pending(fault, regs)) return 0; if (unlikely(fault & VM_FAULT_ERROR)) { diff --git a/arch/microblaze/mm/fault.c b/arch/microblaze/mm/fault.c index e6a810b0c7ad..cdde01dcdfc3 100644 --- a/arch/microblaze/mm/fault.c +++ b/arch/microblaze/mm/fault.c @@ -217,7 +217,7 @@ void do_page_fault(struct pt_regs *regs, unsigned long address, */ fault = handle_mm_fault(vma, address, flags); - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) + if (fault_signal_pending(fault, regs)) return; if (unlikely(fault & VM_FAULT_ERROR)) { diff --git a/arch/mips/mm/fault.c b/arch/mips/mm/fault.c index f589aa8f47d9..31fda59199eb 100644 --- a/arch/mips/mm/fault.c +++ b/arch/mips/mm/fault.c @@ -154,7 +154,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, unsigned long write, */ fault = handle_mm_fault(vma, address, flags); - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) + if (fault_signal_pending(regs)) return; perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address); diff --git a/arch/nds32/mm/fault.c b/arch/nds32/mm/fault.c index 064ae5d2159d..c8f32b4784bd 100644 --- a/arch/nds32/mm/fault.c +++ b/arch/nds32/mm/fault.c @@ -210,7 +210,7 @@ void do_page_fault(unsigned long entry, unsigned long addr, * signal first. We do not need to release the mmap_sem because it * would already be released in __lock_page_or_retry in mm/filemap.c. */ - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) { + if (fault_signal_pending(fault, regs)) { if (!user_mode(regs)) goto no_context; return; diff --git a/arch/nios2/mm/fault.c b/arch/nios2/mm/fault.c index 6a2e716b959f..704ace8ca0ee 100644 --- a/arch/nios2/mm/fault.c +++ b/arch/nios2/mm/fault.c @@ -133,7 +133,7 @@ asmlinkage void do_page_fault(struct pt_regs *regs, unsigned long cause, */ fault = handle_mm_fault(vma, address, flags); - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) + if (fault_signal_pending(fault, regs)) return; if (unlikely(fault & VM_FAULT_ERROR)) { diff --git a/arch/openrisc/mm/fault.c b/arch/openrisc/mm/fault.c index 5d4d3a9691d0..85c7eb0c0186 100644 --- a/arch/openrisc/mm/fault.c +++ b/arch/openrisc/mm/fault.c @@ -161,7 +161,7 @@ asmlinkage void do_page_fault(struct pt_regs *regs, unsigned long address, fault = handle_mm_fault(vma, address, flags); - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) + if (fault_signal_pending(fault, regs)) return; if (unlikely(fault & VM_FAULT_ERROR)) { diff --git a/arch/parisc/mm/fault.c b/arch/parisc/mm/fault.c index adbd5e2144a3..f9be1d1cb43f 100644 --- a/arch/parisc/mm/fault.c +++ b/arch/parisc/mm/fault.c @@ -304,7 +304,7 @@ void do_page_fault(struct pt_regs *regs, unsigned long code, fault = handle_mm_fault(vma, address, flags); - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) + if (fault_signal_pending(fault, regs)) return; if (unlikely(fault & VM_FAULT_ERROR)) { diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c index 96add1427a75..48a60e128b3f 100644 --- a/arch/riscv/mm/fault.c +++ b/arch/riscv/mm/fault.c @@ -115,7 +115,7 @@ asmlinkage void do_page_fault(struct pt_regs *regs) * signal first. We do not need to release the mmap_sem because it * would already be released in __lock_page_or_retry in mm/filemap.c. */ - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(tsk)) + if (fault_signal_pending(fault, regs)) return; if (unlikely(fault & VM_FAULT_ERROR)) { diff --git a/arch/s390/mm/fault.c b/arch/s390/mm/fault.c index 7b0bb475c166..179cf92a56e5 100644 --- a/arch/s390/mm/fault.c +++ b/arch/s390/mm/fault.c @@ -480,8 +480,7 @@ static inline vm_fault_t do_exception(struct pt_regs *regs, int access) * the fault. */ fault = handle_mm_fault(vma, address, flags); - /* No reason to continue if interrupted by SIGKILL. */ - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) { + if (fault_signal_pending(fault, regs)) { fault = VM_FAULT_SIGNAL; if (flags & FAULT_FLAG_RETRY_NOWAIT) goto out_up; diff --git a/arch/sparc/mm/fault_32.c b/arch/sparc/mm/fault_32.c index 8d69de111470..7a62edf7f68b 100644 --- a/arch/sparc/mm/fault_32.c +++ b/arch/sparc/mm/fault_32.c @@ -237,7 +237,7 @@ asmlinkage void do_sparc_fault(struct pt_regs *regs, int text_fault, int write, */ fault = handle_mm_fault(vma, address, flags); - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) + if (fault_signal_pending(fault, regs)) return; if (unlikely(fault & VM_FAULT_ERROR)) { diff --git a/arch/sparc/mm/fault_64.c b/arch/sparc/mm/fault_64.c index 2371fb6b97e4..d90bafb63e17 100644 --- a/arch/sparc/mm/fault_64.c +++ b/arch/sparc/mm/fault_64.c @@ -421,7 +421,7 @@ asmlinkage void __kprobes do_sparc64_fault(struct pt_regs *regs) fault = handle_mm_fault(vma, address, flags); - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) + if (fault_signal_pending(fault, regs)) goto exit_exception; if (unlikely(fault & VM_FAULT_ERROR)) { diff --git a/arch/unicore32/mm/fault.c b/arch/unicore32/mm/fault.c index 76342de9cf8c..59d0e6ec2cfc 100644 --- a/arch/unicore32/mm/fault.c +++ b/arch/unicore32/mm/fault.c @@ -250,7 +250,7 @@ static int do_pf(unsigned long addr, unsigned int fsr, struct pt_regs *regs) * signal first. We do not need to release the mmap_sem because * it would already be released in __lock_page_or_retry in * mm/filemap.c. */ - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) + if (fault_signal_pending(fault, regs)) return 0; if (!(fault & VM_FAULT_ERROR) && (flags & FAULT_FLAG_ALLOW_RETRY)) { diff --git a/arch/xtensa/mm/fault.c b/arch/xtensa/mm/fault.c index f81b1478da61..8651520de460 100644 --- a/arch/xtensa/mm/fault.c +++ b/arch/xtensa/mm/fault.c @@ -110,7 +110,7 @@ void do_page_fault(struct pt_regs *regs) */ fault = handle_mm_fault(vma, address, flags); - if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current)) + if (fault_signal_pending(fault, regs)) return; if (unlikely(fault & VM_FAULT_ERROR)) { diff --git a/include/linux/sched/signal.h b/include/linux/sched/signal.h index efd8ce7675ed..46429192733b 100644 --- a/include/linux/sched/signal.h +++ b/include/linux/sched/signal.h @@ -377,6 +377,19 @@ static inline int signal_pending_state(long state, struct task_struct *p) return (state & TASK_INTERRUPTIBLE) || __fatal_signal_pending(p); } +/* + * This should only be used in fault handlers to decide whether we + * should stop the current fault routine to handle the signals + * instead, especially with the case where we've got interrupted with + * a VM_FAULT_RETRY. + */ +static inline bool fault_signal_pending(unsigned int fault_flags, + struct pt_regs *regs) +{ + return unlikely((fault_flags & VM_FAULT_RETRY) && + fatal_signal_pending(current)); +} + /* * Reevaluate whether the task has signals pending delivery. * Wake the task if so. From patchwork Thu Sep 26 09:38:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162017 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E26776 for ; Thu, 26 Sep 2019 09:39:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E97F9222BE for ; Thu, 26 Sep 2019 09:39:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E97F9222BE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 332556B000D; Thu, 26 Sep 2019 05:39:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2E3A16B000E; Thu, 26 Sep 2019 05:39:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1AB2D6B0010; Thu, 26 Sep 2019 05:39:54 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0188.hostedemail.com [216.40.44.188]) by kanga.kvack.org (Postfix) with ESMTP id F12286B000D for ; Thu, 26 Sep 2019 05:39:53 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 71A20181AC9CC for ; Thu, 26 Sep 2019 09:39:53 +0000 (UTC) X-FDA: 75976575066.28.spoon07_736a9b417f147 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30003:30012:30051:30054:30070,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: spoon07_736a9b417f147 X-Filterd-Recvd-Size: 5063 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:39:53 +0000 (UTC) Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 15C8350F4D for ; Thu, 26 Sep 2019 09:39:52 +0000 (UTC) Received: by mail-pl1-f197.google.com with SMTP id k9so1165753pls.13 for ; Thu, 26 Sep 2019 02:39:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nj2m7MUZQj6IXXRy4zbweijjogaCLtcKQ4EIZak4N0g=; b=q/tKaJH8GnE7Vg1UR1cd3aFF5FsgsDsOIqCmG6sdIQtxDMxUecVkbuatDAI/ntWFBJ s7TIIK2LMJud9zpqaqUqSusicw/j/NTbOaJsQXktBnE46PhcHMzKVNfRYQej1WWUxYpk UHBFOV16nVlghGmnzBHIzMcw583cSvx1LX9KpwKOv6cyyGgjzcAPt+C9r6gSczhAgl8k LSov1Avywk5DxjT12Ls6xhlObsnKFrnKf3ByRsnboycNPwN2hO9tbEDYmYDmqr23fTtB sKxESd938lHm0n6xfHakCVphs7FQmjy84Js0uLzxTti0ONq0LSZBT7jt/gUF2gRRJqnD +Vpw== X-Gm-Message-State: APjAAAXbxhqgBqoQpT7kqCbxMU1Thhtp0ION8wH66Dx4xQvtoAKgk3nF JC80wrpJtNUPhW3X3CxcernOWTCG8vd0NeIgMqqfTJPQR42yA+5LaEA2BSdjaOrv98RgY2Y1LDH gUgrkVjo6xkQ= X-Received: by 2002:a63:9742:: with SMTP id d2mr2391544pgo.356.1569490791005; Thu, 26 Sep 2019 02:39:51 -0700 (PDT) X-Google-Smtp-Source: APXvYqw3zgXy3wRQcC9cw9sy7SwV86a9/+fNTVOhrGLjp87vBhGEjJORyCXZ82QFiBleXBcoc7oI5w== X-Received: by 2002:a63:9742:: with SMTP id d2mr2391518pgo.356.1569490790717; Thu, 26 Sep 2019 02:39:50 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.39.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:39:50 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 04/16] x86/mm: Use helper fault_signal_pending() Date: Thu, 26 Sep 2019 17:38:52 +0800 Message-Id: <20190926093904.5090-5-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let's move the fatal signal check even earlier so that we can directly use the new fault_signal_pending() in x86 mm code. Signed-off-by: Peter Xu --- arch/x86/mm/fault.c | 28 +++++++++++++--------------- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 9ceacd1156db..059331797827 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -1441,27 +1441,25 @@ void do_user_addr_fault(struct pt_regs *regs, fault = handle_mm_fault(vma, address, flags); major |= fault & VM_FAULT_MAJOR; + /* Quick path to respond to signals */ + if (fault_signal_pending(fault, regs)) { + if (!user_mode(regs)) + no_context(regs, hw_error_code, address, SIGBUS, + BUS_ADRERR); + return; + } + /* * If we need to retry the mmap_sem has already been released, * and if there is a fatal signal pending there is no guarantee * that we made any progress. Handle this case first. */ - if (unlikely(fault & VM_FAULT_RETRY)) { + if (unlikely((fault & VM_FAULT_RETRY) && + (flags & FAULT_FLAG_ALLOW_RETRY))) { /* Retry at most once */ - if (flags & FAULT_FLAG_ALLOW_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; - flags |= FAULT_FLAG_TRIED; - if (!fatal_signal_pending(tsk)) - goto retry; - } - - /* User mode? Just return to handle the fatal exception */ - if (flags & FAULT_FLAG_USER) - return; - - /* Not returning to user mode? Handle exceptions or die: */ - no_context(regs, hw_error_code, address, SIGBUS, BUS_ADRERR); - return; + flags &= ~FAULT_FLAG_ALLOW_RETRY; + flags |= FAULT_FLAG_TRIED; + goto retry; } up_read(&mm->mmap_sem); From patchwork Thu Sep 26 09:38:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162019 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A1A7E924 for ; Thu, 26 Sep 2019 09:40:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 76EF6222C3 for ; Thu, 26 Sep 2019 09:40:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 76EF6222C3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B21AB6B000E; Thu, 26 Sep 2019 05:40:02 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AD4A66B0010; Thu, 26 Sep 2019 05:40:02 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 99A466B0266; Thu, 26 Sep 2019 05:40:02 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0019.hostedemail.com [216.40.44.19]) by kanga.kvack.org (Postfix) with ESMTP id 7A8786B000E for ; Thu, 26 Sep 2019 05:40:02 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 27161180AD802 for ; Thu, 26 Sep 2019 09:40:02 +0000 (UTC) X-FDA: 75976575444.04.blood26_749b0a883a533 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30003:30012:30045:30051:30054:30070,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: blood26_749b0a883a533 X-Filterd-Recvd-Size: 4954 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:40:01 +0000 (UTC) Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4846EC06512C for ; Thu, 26 Sep 2019 09:40:00 +0000 (UTC) Received: by mail-pl1-f200.google.com with SMTP id q5so1184245pls.0 for ; Thu, 26 Sep 2019 02:40:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Jbuek0IBtNMleGLem0b6G1JP1yfyDp1ywawNI2SggWk=; b=Mvm6kXKSXjQJhrl1P6xQoQxhZ/txOpfAboFmsPAuaWnBQdMUqgvOZKfqQzdeezpycx Dqo1LYnlX/HQ06N8z9lSn6uZaUoY9QXhfHg87OzwpV3isMCNHCSYtuN5L4KqYAZYqY7+ EuGNq+Z/lAidEGYzmb5jpi4e7OUnykhJuL1IUStXDE8GhgQfXI41O47jW3gYOf2yeOOh rnV1G3MAWguzVrH1lG7SJt4GSU3N8WCFe4uyEhr7UUK216vIYaYYJ9oFzdi3b6yxufAn 5K+ckRrAh2xIvSM5rZ6giiVQZXUHeRNjtMEJ9EgthjXGfrsRmvaCyWZmS6Ph8YkibZfp dC7g== X-Gm-Message-State: APjAAAUBHUNgVdgbn6jMbYuBPhK6ekd433Oxv+IbEM5dDbiIMcLTvKyB covhCzFmfJx3Jh2NKln1ksXzKwtcnZxNnDYU5I9Yq3SQ6I07Ts87iPUELWy5+nmT8G4rDVduVLk y/4RFS+AoAFs= X-Received: by 2002:a17:902:202:: with SMTP id 2mr3091346plc.96.1569490799214; Thu, 26 Sep 2019 02:39:59 -0700 (PDT) X-Google-Smtp-Source: APXvYqxOsZ3BVghSeIFfGQEyW6lVgkhzRLJGhChjYo8ExyTTO9l78osOjDKhtCVViyXliYW7TRe8jw== X-Received: by 2002:a17:902:202:: with SMTP id 2mr3091314plc.96.1569490799031; Thu, 26 Sep 2019 02:39:59 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.39.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:39:58 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 05/16] arc/mm: Use helper fault_signal_pending() Date: Thu, 26 Sep 2019 17:38:53 +0800 Message-Id: <20190926093904.5090-6-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let ARC to use the new helper fault_signal_pending() by moving the signal check out of the retry logic as standalone. This should also helps to simplify the code a bit. Signed-off-by: Peter Xu --- arch/arc/mm/fault.c | 34 +++++++++++++--------------------- 1 file changed, 13 insertions(+), 21 deletions(-) diff --git a/arch/arc/mm/fault.c b/arch/arc/mm/fault.c index 3861543b66a0..ee3ba7c7b891 100644 --- a/arch/arc/mm/fault.c +++ b/arch/arc/mm/fault.c @@ -127,29 +127,21 @@ void do_page_fault(unsigned long address, struct pt_regs *regs) fault = handle_mm_fault(vma, address, flags); + /* Quick path to respond to signals */ + if (fault_signal_pending(fault, regs)) { + if (!user_mode(regs)) + goto no_context; + return; + } + /* - * Fault retry nuances + * Fault retry nuances, mmap_sem already relinquished by core mm */ - if (unlikely(fault & VM_FAULT_RETRY)) { - - /* - * If fault needs to be retried, handle any pending signals - * first (by returning to user mode). - * mmap_sem already relinquished by core mm for RETRY case - */ - if (fatal_signal_pending(current)) { - if (!user_mode(regs)) - goto no_context; - return; - } - /* - * retry state machine - */ - if (flags & FAULT_FLAG_ALLOW_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; - flags |= FAULT_FLAG_TRIED; - goto retry; - } + if (unlikely((fault & VM_FAULT_RETRY) && + (flags & FAULT_FLAG_ALLOW_RETRY))) { + flags &= ~FAULT_FLAG_ALLOW_RETRY; + flags |= FAULT_FLAG_TRIED; + goto retry; } bad_area: From patchwork Thu Sep 26 09:38:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162021 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7716B924 for ; Thu, 26 Sep 2019 09:40:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4CF3C222C3 for ; Thu, 26 Sep 2019 09:40:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4CF3C222C3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8637C6B0010; Thu, 26 Sep 2019 05:40:10 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7EBCA6B0266; Thu, 26 Sep 2019 05:40:10 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6DB576B0269; Thu, 26 Sep 2019 05:40:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0126.hostedemail.com [216.40.44.126]) by kanga.kvack.org (Postfix) with ESMTP id 4FE506B0010 for ; Thu, 26 Sep 2019 05:40:10 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id E348C180AD803 for ; Thu, 26 Sep 2019 09:40:09 +0000 (UTC) X-FDA: 75976575738.23.crow24_75cceeac89c59 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30054,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: crow24_75cceeac89c59 X-Filterd-Recvd-Size: 4643 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:40:09 +0000 (UTC) Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A5AB64FCD9 for ; Thu, 26 Sep 2019 09:40:08 +0000 (UTC) Received: by mail-pf1-f200.google.com with SMTP id f2so1358436pfk.13 for ; Thu, 26 Sep 2019 02:40:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ppg1NSAShmiSu6checGzn4FvUz9QiJFKA275BDKmjpQ=; b=h69Lg+4zyOrbDS5ME+NRQFI9COGEquGOXL84jZgKvHBC9AquPrew8lWdkWrGB6HGW7 KddenHIE6Q9ME4FlMhdppXQFAHmwFpYC9qQ5jEcn3c3ZEPYKuavT+ygRZVwWAwrfLgpL k5lMa7i2a8ncsobWZxcVV1L3cwTUM0AF2utr6LqfFumXVMTbPtkVDdEPiqeNec54gf0P rOTwPKtw20ehD29SBnJ+84wmGOmyyYIQUbJ61NmnO+d0ItlLqz8LisWVHB428b6wT2v2 k2KmbwKtebPxGR6SZ0thBCnce5OGXImzFgHA/qu1NIebV+na3Rd3dTrsLcKppH02gHJS nigA== X-Gm-Message-State: APjAAAX2RyRhs0pCxN14FgY4+38SSLRwFlYs/wH1H3jDbb4R3YXtnmmq C9GWmLCHc7/Pav4mfuGhLExMoSN2eJizKnO5a9fnxq+t8HpKYkBDoo2CMMUyvQ0yDddkG6FFtVU wfTdifJ60bwk= X-Received: by 2002:a17:90a:8986:: with SMTP id v6mr2583671pjn.115.1569490807827; Thu, 26 Sep 2019 02:40:07 -0700 (PDT) X-Google-Smtp-Source: APXvYqwQD3YCFxyODT7vLmOJs1BI9wxQJcPhmEkojKoQ0r5Yub0wIQeWe0SUpPIZubTSCqM/BHQXwg== X-Received: by 2002:a17:90a:8986:: with SMTP id v6mr2583641pjn.115.1569490807619; Thu, 26 Sep 2019 02:40:07 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.39.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:40:06 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 06/16] arm64/mm: Use helper fault_signal_pending() Date: Thu, 26 Sep 2019 17:38:54 +0800 Message-Id: <20190926093904.5090-7-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let the arm64 fault handling to use the new fault_signal_pending() helper, by moving the signal handling out of the retry logic. Signed-off-by: Peter Xu --- arch/arm64/mm/fault.c | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index cfd65b63f36f..4a695a44fb05 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -477,19 +477,14 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, fault = __do_page_fault(mm, addr, mm_flags, vm_flags); major |= fault & VM_FAULT_MAJOR; - if (fault & VM_FAULT_RETRY) { - /* - * If we need to retry but a fatal signal is pending, - * handle the signal first. We do not need to release - * the mmap_sem because it would already be released - * in __lock_page_or_retry in mm/filemap.c. - */ - if (fatal_signal_pending(current)) { - if (!user_mode(regs)) - goto no_context; - return 0; - } + /* Quick path to respond to signals */ + if (fault_signal_pending(fault, regs)) { + if (!user_mode(regs)) + goto no_context; + return 0; + } + if (fault & VM_FAULT_RETRY) { /* * Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk of * starvation. From patchwork Thu Sep 26 09:38:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162023 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6395C76 for ; Thu, 26 Sep 2019 09:40:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3AB68222C3 for ; Thu, 26 Sep 2019 09:40:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3AB68222C3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 79E8E6B0266; Thu, 26 Sep 2019 05:40:19 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 74FF06B0269; Thu, 26 Sep 2019 05:40:19 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 63F176B026A; Thu, 26 Sep 2019 05:40:19 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0056.hostedemail.com [216.40.44.56]) by kanga.kvack.org (Postfix) with ESMTP id 4633E6B0266 for ; Thu, 26 Sep 2019 05:40:19 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 022052A4AE for ; Thu, 26 Sep 2019 09:40:19 +0000 (UTC) X-FDA: 75976576116.12.wound93_7718b520d0b21 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30003:30012:30051:30054,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: wound93_7718b520d0b21 X-Filterd-Recvd-Size: 4534 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:40:18 +0000 (UTC) Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6254388306 for ; Thu, 26 Sep 2019 09:40:17 +0000 (UTC) Received: by mail-pg1-f200.google.com with SMTP id w13so1069943pge.15 for ; Thu, 26 Sep 2019 02:40:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LpCSyY/6JZQuJ1z7pLkTHdeWJUVPhg/tAxJyi5b+KHU=; b=twSLJiVMh02O+HVHGOCXS0dt9GjZXfal3YzfEUbEULYKavNr/6ufDbs5c9d6atUI9z gka5kfVhymFudWL44v4BW7IkoatQ+GmvRIauuoBUfryOs8AURmu7svsXFTpXDe2x8A+6 CmU/TyQIJxWvbWEeaIR5p3mNii8tUptPpIXMWyjII3CGuedjl4POF4G5bxuTJW0b3Xps 9B9wwxD2tlu7G/a+GkHFms4V6/JRZ0kOCQe4qsEYBAy5x2OZAaWC97LZEVi+8GsXp2Y+ jPfoqaCCuAAk2cV57mwG1nYbk+Penao16ItIfV4Zh2PJlIDZL89krL6PIkMsWGb2mho7 QebA== X-Gm-Message-State: APjAAAUpmbfD9vU2apxYbYsD5DImsb1fhwTgBEweNT95Y0TW6v8vJXtc Plr8wvbkS6BjktKOAVtXXn7Pdy/JtFrrc1pKUIaLMaPXhB1Es1WSRI11pCFb7PQyk5ahSR4RAu/ 2xrWkQrbdCzQ= X-Received: by 2002:a63:a369:: with SMTP id v41mr2525765pgn.148.1569490816503; Thu, 26 Sep 2019 02:40:16 -0700 (PDT) X-Google-Smtp-Source: APXvYqxKACEquhcLFybN2y5JPp/3ZKltqjkLTVK4ufHCX/1hwGKZNzOpTcD/V3ar6+4lQuJZ+kgFxw== X-Received: by 2002:a63:a369:: with SMTP id v41mr2525726pgn.148.1569490816214; Thu, 26 Sep 2019 02:40:16 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.40.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:40:15 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 07/16] powerpc/mm: Use helper fault_signal_pending() Date: Thu, 26 Sep 2019 17:38:55 +0800 Message-Id: <20190926093904.5090-8-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let powerpc code to use the new helper, by moving the signal handling earlier before the retry logic. Signed-off-by: Peter Xu --- arch/powerpc/mm/fault.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index 8432c281de92..7dabd1243f1b 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -583,6 +583,9 @@ static int __do_page_fault(struct pt_regs *regs, unsigned long address, major |= fault & VM_FAULT_MAJOR; + if (fault_signal_pending(fault, regs)) + return user_mode(regs) ? 0 : SIGBUS; + /* * Handle the retry right now, the mmap_sem has been released in that * case. @@ -596,15 +599,8 @@ static int __do_page_fault(struct pt_regs *regs, unsigned long address, */ flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; - if (!fatal_signal_pending(current)) - goto retry; + goto retry; } - - /* - * User mode? Just return to handle the fatal exception otherwise - * return to bad_page_fault - */ - return is_user ? 0 : SIGBUS; } up_read(¤t->mm->mmap_sem); From patchwork Thu Sep 26 09:38:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162025 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EF88876 for ; Thu, 26 Sep 2019 09:40:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C612A222BE for ; Thu, 26 Sep 2019 09:40:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C612A222BE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0C19A6B0269; Thu, 26 Sep 2019 05:40:28 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0991E6B026A; Thu, 26 Sep 2019 05:40:28 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EA3596B026B; Thu, 26 Sep 2019 05:40:27 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0171.hostedemail.com [216.40.44.171]) by kanga.kvack.org (Postfix) with ESMTP id CA7676B0269 for ; Thu, 26 Sep 2019 05:40:27 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 4A8B52F264 for ; Thu, 26 Sep 2019 09:40:27 +0000 (UTC) X-FDA: 75976576494.16.skate18_785294653264f X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30003:30054:30070,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:6,LUA_SUMMARY:none X-HE-Tag: skate18_785294653264f X-Filterd-Recvd-Size: 4812 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:40:26 +0000 (UTC) Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 104757FDF5 for ; Thu, 26 Sep 2019 09:40:26 +0000 (UTC) Received: by mail-pg1-f200.google.com with SMTP id 135so1052507pgc.23 for ; Thu, 26 Sep 2019 02:40:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=K+W2E1LzRmGELKrtQEGc7UL31viDl3TRdbLGP+s2aPk=; b=pKne8p/6bHMXI88j3KTMrSmxPfuzXrwJaf2l2VqVw0OjBsUepMWs1EvZ/oj1uhua4Y DyauwNqnG3lOLiWpHMONZUOb4MFrBdF8q1KBzcMzlP3Z7b0PTSRXUEz8K7CFCqpicB7X vR+1OpSjgm+00hCymgbetggScbTkdKddXVbjEesLKxKN7UxBUj6kuRVJ4jXUBhqQsTPt OFtI/0uyZFeTZHYbIXV6RZwISpSqIm8jAmWC7QGgMkBQx5ncmlsN3/IiA7Fh3VNF+8Dn y5+fWx2BMClPV3DNddZg2jjefC72kXgEIEpQFppAHhP2HXh95IWphOL7nYI3f3KtYAKp SpSw== X-Gm-Message-State: APjAAAVTA2SZ49dDmzdpu+8k+MC/QT+XjWw9kw5mTGzuvFd1gj4XLOXq IsPjqwGW1yDO5RCPJStaQDMRivP/FysZTS9VkRGTEJ/fLR6YkvU0wckXLxyUhJtY+/HVVVjDlHh qjqXmtX4T434= X-Received: by 2002:a17:902:8d98:: with SMTP id v24mr2842080plo.265.1569490825032; Thu, 26 Sep 2019 02:40:25 -0700 (PDT) X-Google-Smtp-Source: APXvYqynSS4HR4dDuIQIw6IsL02LZzQrkS9X1jvtp6dAnxYs3miRg0CJh42FhqBbBwvL8J0ehgBiUQ== X-Received: by 2002:a17:902:8d98:: with SMTP id v24mr2842062plo.265.1569490824859; Thu, 26 Sep 2019 02:40:24 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.40.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:40:24 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 08/16] sh/mm: Use helper fault_signal_pending() Date: Thu, 26 Sep 2019 17:38:56 +0800 Message-Id: <20190926093904.5090-9-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let SH to use the new fault_signal_pending() helper. Here we'll need to move the up_read() out because that's actually needed as long as !RETRY cases. At the meantime we can drop all the rest of up_read()s now (which seems to be cleaner). Signed-off-by: Peter Xu --- arch/sh/mm/fault.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/sh/mm/fault.c b/arch/sh/mm/fault.c index 5f51456f4fc7..eb4048ad0b38 100644 --- a/arch/sh/mm/fault.c +++ b/arch/sh/mm/fault.c @@ -302,25 +302,25 @@ mm_fault_error(struct pt_regs *regs, unsigned long error_code, * Pagefault was interrupted by SIGKILL. We have no reason to * continue pagefault. */ - if (fatal_signal_pending(current)) { - if (!(fault & VM_FAULT_RETRY)) - up_read(¤t->mm->mmap_sem); + if (fault_signal_pending(fault, regs)) { if (!user_mode(regs)) no_context(regs, error_code, address); return 1; } + /* Release mmap_sem first if necessary */ + if (!(fault & VM_FAULT_RETRY)) + up_read(¤t->mm->mmap_sem); + if (!(fault & VM_FAULT_ERROR)) return 0; if (fault & VM_FAULT_OOM) { /* Kernel mode? Handle exceptions or die: */ if (!user_mode(regs)) { - up_read(¤t->mm->mmap_sem); no_context(regs, error_code, address); return 1; } - up_read(¤t->mm->mmap_sem); /* * We ran out of memory, call the OOM killer, and return the From patchwork Thu Sep 26 09:38:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162027 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 929E476 for ; Thu, 26 Sep 2019 09:40:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5FB19222BE for ; Thu, 26 Sep 2019 09:40:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5FB19222BE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 99AAE6B0008; Thu, 26 Sep 2019 05:40:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 94B146B026A; Thu, 26 Sep 2019 05:40:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 861926B026B; Thu, 26 Sep 2019 05:40:36 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0203.hostedemail.com [216.40.44.203]) by kanga.kvack.org (Postfix) with ESMTP id 683D66B0008 for ; Thu, 26 Sep 2019 05:40:36 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id E91D5180AD802 for ; Thu, 26 Sep 2019 09:40:35 +0000 (UTC) X-FDA: 75976576830.15.books71_79936eb4b071b X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30051:30054,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: books71_79936eb4b071b X-Filterd-Recvd-Size: 4879 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:40:35 +0000 (UTC) Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8A9217FDCA for ; Thu, 26 Sep 2019 09:40:34 +0000 (UTC) Received: by mail-pg1-f198.google.com with SMTP id f10so313073pgj.0 for ; Thu, 26 Sep 2019 02:40:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Zs0icSpFo5EckbI9NuVzMV23UUzKD43r4J1NyJ/klic=; b=Et7L4qx3Q3mBOTxknJPg1ayxLrzHB464yMHvpEGYyWzSaUh7v80sWR7ckTu/orU5jw ZZcbRCGDWj+i6PpsF8+oL5p3QQ1cRHBiib2rpwyMzT35y0gQDqa7R0TxEP+Xn6vRcIsa nyd7f5C2ubB3egomg1Z2/xLs7qjz7GsVnd8+u0AFURdfyWgYb6IXOFgxLoLJ2mc8lwuB KxJx7aUS85g+8+ydVRfq9yX4HcS96TisuvQYoYcEdtmgg5cT7yLstjTv5XJZJ6KivWPt PUT2CBaDOPpatFcLNmgWJPWVI8aTNX+cJu3nCDZ06glPcLezjGYOTsvYF1C1gcFwCdw4 ZtyA== X-Gm-Message-State: APjAAAVzwDtEZ3UcrEaTueTAEbwo7V+z9lXw7IE+cCM/R4ztj7jbTaUf DiHW0eX8PoPI64UGqJxF/2k+Mqk0+DmedvLkZApNQQe4FuIZ34Ho7m+wQqXsqTJCwM6IjM4i7s2 kyscu8ckW4aY= X-Received: by 2002:a17:902:8488:: with SMTP id c8mr2742260plo.164.1569490833546; Thu, 26 Sep 2019 02:40:33 -0700 (PDT) X-Google-Smtp-Source: APXvYqy78qElHwxU+VN7yYhyu4WKCD5RFszqHAErQMKeM6BohEAgdvKCeLfzyvCTuMFSeTddGiEejA== X-Received: by 2002:a17:902:8488:: with SMTP id c8mr2742227plo.164.1569490833256; Thu, 26 Sep 2019 02:40:33 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.40.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:40:32 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 09/16] mm: Return faster for non-fatal signals in user mode faults Date: Thu, 26 Sep 2019 17:38:57 +0800 Message-Id: <20190926093904.5090-10-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The idea comes from the upstream discussion between Linus and Andrea: https://lore.kernel.org/lkml/20171102193644.GB22686@redhat.com/ A summary to the issue: there was a special path in handle_userfault() in the past that we'll return a VM_FAULT_NOPAGE when we detected non-fatal signals when waiting for userfault handling. We did that by reacquiring the mmap_sem before returning. However that brings a risk in that the vmas might have changed when we retake the mmap_sem and even we could be holding an invalid vma structure. This patch is a preparation of removing that special path by allowing the page fault to return even faster if we were interrupted by a non-fatal signal during a user-mode page fault handling routine. Suggested-by: Linus Torvalds Suggested-by: Andrea Arcangeli Signed-off-by: Peter Xu --- include/linux/sched/signal.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/linux/sched/signal.h b/include/linux/sched/signal.h index 46429192733b..031af0a6505a 100644 --- a/include/linux/sched/signal.h +++ b/include/linux/sched/signal.h @@ -387,7 +387,8 @@ static inline bool fault_signal_pending(unsigned int fault_flags, struct pt_regs *regs) { return unlikely((fault_flags & VM_FAULT_RETRY) && - fatal_signal_pending(current)); + (fatal_signal_pending(current) || + (user_mode(regs) && signal_pending(current)))); } /* From patchwork Thu Sep 26 09:38:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162029 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 18E2D76 for ; Thu, 26 Sep 2019 09:40:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E37EA222BE for ; Thu, 26 Sep 2019 09:40:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E37EA222BE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 332816B026A; Thu, 26 Sep 2019 05:40:45 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2E2446B026B; Thu, 26 Sep 2019 05:40:45 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1D1F46B026C; Thu, 26 Sep 2019 05:40:45 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0101.hostedemail.com [216.40.44.101]) by kanga.kvack.org (Postfix) with ESMTP id F2FC16B026A for ; Thu, 26 Sep 2019 05:40:44 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 73152181AC9AE for ; Thu, 26 Sep 2019 09:40:44 +0000 (UTC) X-FDA: 75976577208.02.end88_7acac96ea6833 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30012:30054:30091,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: end88_7acac96ea6833 X-Filterd-Recvd-Size: 5193 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf16.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:40:43 +0000 (UTC) Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DB29B7FDF5 for ; Thu, 26 Sep 2019 09:40:42 +0000 (UTC) Received: by mail-pf1-f199.google.com with SMTP id f2so1359522pfk.13 for ; Thu, 26 Sep 2019 02:40:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mt5SnWXXSk9uI7dmB8jKwFtY/VxQIE33S8hNcu1UngY=; b=ho5sbr5ZVqWTiGcnwJPZBPE6Q//p09Ts7DnWyT6fWIh8Q9xg7rd1HH5GylRGySPA1W UFGNalYZhcmsaKerunULY51uvwGXSzJ2xzXE5V6lgp2/BNGQ4wWtnAtM60UObmNcR0sC yY85rgjnFRB5XmaR9OY8+i+NRs2BpeNL7bujfpxDqSknSRSZY77PTRRqGe9yeQf5yC6S Oy/3kmAwez15Y0N1DwTRqWROKIXgNsIOgpjypVhi+aj28lN/fP/9AOyTaPTlgDf0NhBS v6izY+XuI5lg7QI+e0Fh7dUzXeFff88XFawUiBjuG2KIy5EoQInEq0lPqTlVCp64ysjl rGlA== X-Gm-Message-State: APjAAAVx/TzM6RrTzjtJNr+lXceC1/Gw9Ko4WAHM49e4s/4R4g5bklcr 4Mvimq2FR1LYoAb94Vcu5XLZSahsHgXUo4+0tn8qf5TC7W48jUQu52dkq9Audq/XA2D/1dfuueJ PqhZ3JXaDRNs= X-Received: by 2002:a17:902:7615:: with SMTP id k21mr3125718pll.116.1569490841822; Thu, 26 Sep 2019 02:40:41 -0700 (PDT) X-Google-Smtp-Source: APXvYqy8FZvMzCaS6tP09HihPyaGx7bD6hMWJf1hsxnBDZsjGhkkZymRtSV9xjgZdU3TLT4rGPDI4Q== X-Received: by 2002:a17:902:7615:: with SMTP id k21mr3125689pll.116.1569490841636; Thu, 26 Sep 2019 02:40:41 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.40.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:40:40 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 10/16] userfaultfd: Don't retake mmap_sem to emulate NOPAGE Date: Thu, 26 Sep 2019 17:38:58 +0800 Message-Id: <20190926093904.5090-11-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch removes the risk path in handle_userfault() then we will be sure that the callers of handle_mm_fault() will know that the VMAs might have changed. Meanwhile with previous patch we don't lose responsiveness as well since the core mm code now can handle the nonfatal userspace signals even if we return VM_FAULT_RETRY. Suggested-by: Andrea Arcangeli Suggested-by: Linus Torvalds Reviewed-by: Jerome Glisse Signed-off-by: Peter Xu --- fs/userfaultfd.c | 24 ------------------------ 1 file changed, 24 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index fe6d804a38dc..d8777146bae7 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -524,30 +524,6 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason) __set_current_state(TASK_RUNNING); - if (return_to_userland) { - if (signal_pending(current) && - !fatal_signal_pending(current)) { - /* - * If we got a SIGSTOP or SIGCONT and this is - * a normal userland page fault, just let - * userland return so the signal will be - * handled and gdb debugging works. The page - * fault code immediately after we return from - * this function is going to release the - * mmap_sem and it's not depending on it - * (unlike gup would if we were not to return - * VM_FAULT_RETRY). - * - * If a fatal signal is pending we still take - * the streamlined VM_FAULT_RETRY failure path - * and there's no need to retake the mmap_sem - * in such case. - */ - down_read(&mm->mmap_sem); - ret = VM_FAULT_NOPAGE; - } - } - /* * Here we race with the list_del; list_add in * userfaultfd_ctx_read(), however because we don't ever run From patchwork Thu Sep 26 09:38:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162031 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B056576 for ; Thu, 26 Sep 2019 09:40:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6CF3E222CB for ; Thu, 26 Sep 2019 09:40:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6CF3E222CB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 99A5B6B026B; Thu, 26 Sep 2019 05:40:53 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 94BF66B026C; Thu, 26 Sep 2019 05:40:53 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8124F6B026D; Thu, 26 Sep 2019 05:40:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0204.hostedemail.com [216.40.44.204]) by kanga.kvack.org (Postfix) with ESMTP id 58A5F6B026B for ; Thu, 26 Sep 2019 05:40:53 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 110F12F269 for ; Thu, 26 Sep 2019 09:40:53 +0000 (UTC) X-FDA: 75976577586.03.cloth80_7c129800eb559 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30003:30012:30051:30054,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: cloth80_7c129800eb559 X-Filterd-Recvd-Size: 17224 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:40:52 +0000 (UTC) Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BD8D78D0F8 for ; Thu, 26 Sep 2019 09:40:51 +0000 (UTC) Received: by mail-pg1-f200.google.com with SMTP id a31so1057328pgm.20 for ; Thu, 26 Sep 2019 02:40:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XP6gr3eQdJ6u+sy/NHvjIjEoZ5vslpS8bcX23tyuCE4=; b=W2w/2SSVB13+YGaN3xtek+YJI0toEqHt7Wj8pb57UtEwAvHvIBr/pGieG7f4cp5p/4 X3/CGSjohMm7Id+AJvFal0vnj/M4SYJVBP+Jdfk+Nvb9/mLKNdU9UH+McO4t0d5naGqE ply1EBJL5HMByN76BR7tMbcB+Aar4hlHgh6RyVJ+9UMnGL47XXav86abqRSF80wkSnvK S+BPp4LLQFYWex0RAJl1Zw8lkF6JaT0KWDh8dZf08s4XyIlVb6iE82mGuQi7G5U149Sd PINc+S9XNTsmZnis8ChvqbVUfu0CkR4GjntF9tFFWIWEzVmugp2ipqg2hGqEUrHrFlam BhYg== X-Gm-Message-State: APjAAAUfFmYi6Rk3Tf2z/sbYfAofENCYUHZnm0ro+lYuSxs3hx05v3tc jb8H0TWL4aD0yWTGoq8zc5JGRbf/epfq7O9U+FNyuY2ITlrNHuVF7yH69KLc3Y4s9QqW0oIP+mF 7+nZQGS7tbOQ= X-Received: by 2002:a17:90a:cb18:: with SMTP id z24mr2531940pjt.108.1569490850676; Thu, 26 Sep 2019 02:40:50 -0700 (PDT) X-Google-Smtp-Source: APXvYqx7yR58nRgGu4SMpKi5ogu9u89i//blno2FCMu/YjHE1i9BEbFh647zHZqW2SOGSk22OQdVhg== X-Received: by 2002:a17:90a:cb18:: with SMTP id z24mr2531894pjt.108.1569490850207; Thu, 26 Sep 2019 02:40:50 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.40.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:40:49 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 11/16] mm: Introduce FAULT_FLAG_DEFAULT Date: Thu, 26 Sep 2019 17:38:59 +0800 Message-Id: <20190926093904.5090-12-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Although there're tons of arch-specific page fault handlers, most of them are still sharing the same initial value of the page fault flags. Say, merely all of the page fault handlers would allow the fault to be retried, and they also allow the fault to respond to SIGKILL. Let's define a default value for the fault flags to replace those initial page fault flags that were copied over. With this, it'll be far easier to introduce new fault flag that can be used by all the architectures instead of touching all the archs. Reviewed-by: David Hildenbrand Signed-off-by: Peter Xu --- arch/alpha/mm/fault.c | 2 +- arch/arc/mm/fault.c | 2 +- arch/arm/mm/fault.c | 2 +- arch/arm64/mm/fault.c | 2 +- arch/hexagon/mm/vm_fault.c | 2 +- arch/ia64/mm/fault.c | 2 +- arch/m68k/mm/fault.c | 2 +- arch/microblaze/mm/fault.c | 2 +- arch/mips/mm/fault.c | 2 +- arch/nds32/mm/fault.c | 2 +- arch/nios2/mm/fault.c | 2 +- arch/openrisc/mm/fault.c | 2 +- arch/parisc/mm/fault.c | 2 +- arch/powerpc/mm/fault.c | 2 +- arch/riscv/mm/fault.c | 2 +- arch/s390/mm/fault.c | 2 +- arch/sh/mm/fault.c | 2 +- arch/sparc/mm/fault_32.c | 2 +- arch/sparc/mm/fault_64.c | 2 +- arch/um/kernel/trap.c | 2 +- arch/unicore32/mm/fault.c | 2 +- arch/x86/mm/fault.c | 2 +- arch/xtensa/mm/fault.c | 2 +- include/linux/mm.h | 7 +++++++ 24 files changed, 30 insertions(+), 23 deletions(-) diff --git a/arch/alpha/mm/fault.c b/arch/alpha/mm/fault.c index aea33b599037..fcfa229cc1e7 100644 --- a/arch/alpha/mm/fault.c +++ b/arch/alpha/mm/fault.c @@ -89,7 +89,7 @@ do_page_fault(unsigned long address, unsigned long mmcsr, const struct exception_table_entry *fixup; int si_code = SEGV_MAPERR; vm_fault_t fault; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; /* As of EV6, a load into $31/$f31 is a prefetch, and never faults (or is suppressed by the PALcode). Support that for older CPUs diff --git a/arch/arc/mm/fault.c b/arch/arc/mm/fault.c index ee3ba7c7b891..33a609f1c43c 100644 --- a/arch/arc/mm/fault.c +++ b/arch/arc/mm/fault.c @@ -94,7 +94,7 @@ void do_page_fault(unsigned long address, struct pt_regs *regs) (regs->ecr_cause == ECR_C_PROTV_INST_FETCH)) exec = 1; - flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + flags = FAULT_FLAG_DEFAULT; if (user_mode(regs)) flags |= FAULT_FLAG_USER; if (write) diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index c8f2950fa733..101ae6698637 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -241,7 +241,7 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) struct mm_struct *mm; int sig, code; vm_fault_t fault; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; if (kprobe_page_fault(regs, fsr)) return 0; diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 4a695a44fb05..23e3feca0f02 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -410,7 +410,7 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, struct mm_struct *mm = current->mm; vm_fault_t fault, major = 0; unsigned long vm_flags = VM_READ | VM_WRITE; - unsigned int mm_flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int mm_flags = FAULT_FLAG_DEFAULT; if (kprobe_page_fault(regs, esr)) return 0; diff --git a/arch/hexagon/mm/vm_fault.c b/arch/hexagon/mm/vm_fault.c index d19beaf11b4c..d9e15d941bdb 100644 --- a/arch/hexagon/mm/vm_fault.c +++ b/arch/hexagon/mm/vm_fault.c @@ -41,7 +41,7 @@ void do_page_fault(unsigned long address, long cause, struct pt_regs *regs) int si_code = SEGV_MAPERR; vm_fault_t fault; const struct exception_table_entry *fixup; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; /* * If we're in an interrupt or have no user context, diff --git a/arch/ia64/mm/fault.c b/arch/ia64/mm/fault.c index 211b4f439384..b5aa4e80c762 100644 --- a/arch/ia64/mm/fault.c +++ b/arch/ia64/mm/fault.c @@ -65,7 +65,7 @@ ia64_do_page_fault (unsigned long address, unsigned long isr, struct pt_regs *re struct mm_struct *mm = current->mm; unsigned long mask; vm_fault_t fault; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; mask = ((((isr >> IA64_ISR_X_BIT) & 1UL) << VM_EXEC_BIT) | (((isr >> IA64_ISR_W_BIT) & 1UL) << VM_WRITE_BIT)); diff --git a/arch/m68k/mm/fault.c b/arch/m68k/mm/fault.c index a455e202691b..182799fd9987 100644 --- a/arch/m68k/mm/fault.c +++ b/arch/m68k/mm/fault.c @@ -71,7 +71,7 @@ int do_page_fault(struct pt_regs *regs, unsigned long address, struct mm_struct *mm = current->mm; struct vm_area_struct * vma; vm_fault_t fault; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; pr_debug("do page fault:\nregs->sr=%#x, regs->pc=%#lx, address=%#lx, %ld, %p\n", regs->sr, regs->pc, address, error_code, mm ? mm->pgd : NULL); diff --git a/arch/microblaze/mm/fault.c b/arch/microblaze/mm/fault.c index cdde01dcdfc3..32da02778a63 100644 --- a/arch/microblaze/mm/fault.c +++ b/arch/microblaze/mm/fault.c @@ -91,7 +91,7 @@ void do_page_fault(struct pt_regs *regs, unsigned long address, int code = SEGV_MAPERR; int is_write = error_code & ESR_S; vm_fault_t fault; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; regs->ear = address; regs->esr = error_code; diff --git a/arch/mips/mm/fault.c b/arch/mips/mm/fault.c index 31fda59199eb..f811ef5efb80 100644 --- a/arch/mips/mm/fault.c +++ b/arch/mips/mm/fault.c @@ -44,7 +44,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, unsigned long write, const int field = sizeof(unsigned long) * 2; int si_code; vm_fault_t fault; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; static DEFINE_RATELIMIT_STATE(ratelimit_state, 5 * HZ, 10); diff --git a/arch/nds32/mm/fault.c b/arch/nds32/mm/fault.c index c8f32b4784bd..4e402f2c88c3 100644 --- a/arch/nds32/mm/fault.c +++ b/arch/nds32/mm/fault.c @@ -76,7 +76,7 @@ void do_page_fault(unsigned long entry, unsigned long addr, int si_code; vm_fault_t fault; unsigned int mask = VM_READ | VM_WRITE | VM_EXEC; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; error_code = error_code & (ITYPE_mskINST | ITYPE_mskETYPE); tsk = current; diff --git a/arch/nios2/mm/fault.c b/arch/nios2/mm/fault.c index 704ace8ca0ee..c38bea4220fb 100644 --- a/arch/nios2/mm/fault.c +++ b/arch/nios2/mm/fault.c @@ -47,7 +47,7 @@ asmlinkage void do_page_fault(struct pt_regs *regs, unsigned long cause, struct mm_struct *mm = tsk->mm; int code = SEGV_MAPERR; vm_fault_t fault; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; cause >>= 2; diff --git a/arch/openrisc/mm/fault.c b/arch/openrisc/mm/fault.c index 85c7eb0c0186..30d5c51e9d40 100644 --- a/arch/openrisc/mm/fault.c +++ b/arch/openrisc/mm/fault.c @@ -50,7 +50,7 @@ asmlinkage void do_page_fault(struct pt_regs *regs, unsigned long address, struct vm_area_struct *vma; int si_code; vm_fault_t fault; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; tsk = current; diff --git a/arch/parisc/mm/fault.c b/arch/parisc/mm/fault.c index f9be1d1cb43f..8e88e5c5f26a 100644 --- a/arch/parisc/mm/fault.c +++ b/arch/parisc/mm/fault.c @@ -274,7 +274,7 @@ void do_page_fault(struct pt_regs *regs, unsigned long code, if (!mm) goto no_context; - flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + flags = FAULT_FLAG_DEFAULT; if (user_mode(regs)) flags |= FAULT_FLAG_USER; diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index 7dabd1243f1b..db156537a2fa 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -435,7 +435,7 @@ static int __do_page_fault(struct pt_regs *regs, unsigned long address, { struct vm_area_struct * vma; struct mm_struct *mm = current->mm; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; int is_exec = TRAP(regs) == 0x400; int is_user = user_mode(regs); int is_write = page_fault_is_write(error_code); diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c index 48a60e128b3f..1f77dd2c491c 100644 --- a/arch/riscv/mm/fault.c +++ b/arch/riscv/mm/fault.c @@ -28,7 +28,7 @@ asmlinkage void do_page_fault(struct pt_regs *regs) struct vm_area_struct *vma; struct mm_struct *mm; unsigned long addr, cause; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; int code = SEGV_MAPERR; vm_fault_t fault; diff --git a/arch/s390/mm/fault.c b/arch/s390/mm/fault.c index 179cf92a56e5..551ac311bd35 100644 --- a/arch/s390/mm/fault.c +++ b/arch/s390/mm/fault.c @@ -429,7 +429,7 @@ static inline vm_fault_t do_exception(struct pt_regs *regs, int access) address = trans_exc_code & __FAIL_ADDR_MASK; perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address); - flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + flags = FAULT_FLAG_DEFAULT; if (user_mode(regs)) flags |= FAULT_FLAG_USER; if (access == VM_WRITE || (trans_exc_code & store_indication) == 0x400) diff --git a/arch/sh/mm/fault.c b/arch/sh/mm/fault.c index eb4048ad0b38..d9c8f2d00a54 100644 --- a/arch/sh/mm/fault.c +++ b/arch/sh/mm/fault.c @@ -380,7 +380,7 @@ asmlinkage void __kprobes do_page_fault(struct pt_regs *regs, struct mm_struct *mm; struct vm_area_struct * vma; vm_fault_t fault; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; tsk = current; mm = tsk->mm; diff --git a/arch/sparc/mm/fault_32.c b/arch/sparc/mm/fault_32.c index 7a62edf7f68b..447f61c4e996 100644 --- a/arch/sparc/mm/fault_32.c +++ b/arch/sparc/mm/fault_32.c @@ -168,7 +168,7 @@ asmlinkage void do_sparc_fault(struct pt_regs *regs, int text_fault, int write, int from_user = !(regs->psr & PSR_PS); int code; vm_fault_t fault; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; if (text_fault) address = regs->pc; diff --git a/arch/sparc/mm/fault_64.c b/arch/sparc/mm/fault_64.c index d90bafb63e17..6807fba66331 100644 --- a/arch/sparc/mm/fault_64.c +++ b/arch/sparc/mm/fault_64.c @@ -267,7 +267,7 @@ asmlinkage void __kprobes do_sparc64_fault(struct pt_regs *regs) int si_code, fault_code; vm_fault_t fault; unsigned long address, mm_rss; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; fault_code = get_thread_fault_code(); diff --git a/arch/um/kernel/trap.c b/arch/um/kernel/trap.c index 58fe36856182..bc2756782d64 100644 --- a/arch/um/kernel/trap.c +++ b/arch/um/kernel/trap.c @@ -32,7 +32,7 @@ int handle_page_fault(unsigned long address, unsigned long ip, pmd_t *pmd; pte_t *pte; int err = -EFAULT; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; *code_out = SEGV_MAPERR; diff --git a/arch/unicore32/mm/fault.c b/arch/unicore32/mm/fault.c index 59d0e6ec2cfc..34a90453ca18 100644 --- a/arch/unicore32/mm/fault.c +++ b/arch/unicore32/mm/fault.c @@ -202,7 +202,7 @@ static int do_pf(unsigned long addr, unsigned int fsr, struct pt_regs *regs) struct mm_struct *mm; int sig, code; vm_fault_t fault; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; tsk = current; mm = tsk->mm; diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 059331797827..d211585cfffd 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -1287,7 +1287,7 @@ void do_user_addr_fault(struct pt_regs *regs, struct task_struct *tsk; struct mm_struct *mm; vm_fault_t fault, major = 0; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; tsk = current; mm = tsk->mm; diff --git a/arch/xtensa/mm/fault.c b/arch/xtensa/mm/fault.c index 8651520de460..6467e3fb9762 100644 --- a/arch/xtensa/mm/fault.c +++ b/arch/xtensa/mm/fault.c @@ -43,7 +43,7 @@ void do_page_fault(struct pt_regs *regs) int is_write, is_exec; vm_fault_t fault; - unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + unsigned int flags = FAULT_FLAG_DEFAULT; code = SEGV_MAPERR; diff --git a/include/linux/mm.h b/include/linux/mm.h index 0334ca97c584..57fb5c535f8e 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -393,6 +393,13 @@ extern pgprot_t protection_map[16]; #define FAULT_FLAG_REMOTE 0x80 /* faulting for non current tsk/mm */ #define FAULT_FLAG_INSTRUCTION 0x100 /* The fault was during an instruction fetch */ +/* + * The default fault flags that should be used by most of the + * arch-specific page fault handlers. + */ +#define FAULT_FLAG_DEFAULT (FAULT_FLAG_ALLOW_RETRY | \ + FAULT_FLAG_KILLABLE) + #define FAULT_FLAG_TRACE \ { FAULT_FLAG_WRITE, "WRITE" }, \ { FAULT_FLAG_MKWRITE, "MKWRITE" }, \ From patchwork Thu Sep 26 09:39:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162033 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AAE68924 for ; Thu, 26 Sep 2019 09:41:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 78C11222C6 for ; Thu, 26 Sep 2019 09:41:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 78C11222C6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AFC606B000D; Thu, 26 Sep 2019 05:41:01 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AAD466B026C; Thu, 26 Sep 2019 05:41:01 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E7BA6B026D; Thu, 26 Sep 2019 05:41:01 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0050.hostedemail.com [216.40.44.50]) by kanga.kvack.org (Postfix) with ESMTP id 7E2106B000D for ; Thu, 26 Sep 2019 05:41:01 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 0396C181AC9AE for ; Thu, 26 Sep 2019 09:41:01 +0000 (UTC) X-FDA: 75976577922.01.rail23_7d39a1718e04f X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30051:30054:30070,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: rail23_7d39a1718e04f X-Filterd-Recvd-Size: 7764 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:41:00 +0000 (UTC) Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B2D6489AC6 for ; Thu, 26 Sep 2019 09:40:59 +0000 (UTC) Received: by mail-pl1-f200.google.com with SMTP id k9so1167487pls.13 for ; Thu, 26 Sep 2019 02:40:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UEirX6W09H+Oy9OrZHOkhdVkpbnjvhUTE+ZWBlFtC7w=; b=YnnJgc7BNGwGNjm6522sTo3AqflxuR0fCyCT1IVq3w/4SFbvSPi1tt3ZuVyn97fWj7 kfve2mPC4aCIbDvW1ZshZP49bEdnixrJsQqueAcIzM3wiRUE2cDPThFT3i4NUevb/d9U AiQr8++9ISNQJg37+Q2ajKz72Cc1h6htQr6M5b659xzeNXOVfgfaPMxmjZSc8TKYjPMF r4L7MmMOXiEKpoFA52iHu4Ou4/e645zFE7n0McjwnWyx72HNaU54++3qiB1v/CYxSsrp I+9k4p+X9MYBukqGEH/v7zLDXJOGQD7DBGGDZTna4A9lX7BAKlCDFdtrQkBcOZs4sk1x 97tg== X-Gm-Message-State: APjAAAVc0ec4HUlb6KnyBwCCvpcXpm6TzNYoECo/WJDUx70aIaQqb5Mj 9qagpgOe238tkrnOyzJ1wcryBVVlQQW+xBWl6wdZkfr/bGXWYAbeDf5pg6KrwBCp9sGJP/DXCjt nvv44OLGPz1w= X-Received: by 2002:a62:2d3:: with SMTP id 202mr2656247pfc.141.1569490858762; Thu, 26 Sep 2019 02:40:58 -0700 (PDT) X-Google-Smtp-Source: APXvYqwx9cpUuoqIPyEG4ywrhXDy3ZMiLq/Jfm7c9EInxtKmK2nYnXyeWPAuexsbI6aD+tXF7a843Q== X-Received: by 2002:a62:2d3:: with SMTP id 202mr2656215pfc.141.1569490858520; Thu, 26 Sep 2019 02:40:58 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.40.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:40:57 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 12/16] mm: Introduce FAULT_FLAG_INTERRUPTIBLE Date: Thu, 26 Sep 2019 17:39:00 +0800 Message-Id: <20190926093904.5090-13-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: handle_userfaultfd() is currently the only one place in the kernel page fault procedures that can respond to non-fatal userspace signals. It was trying to detect such an allowance by checking against USER & KILLABLE flags, which was "un-official". In this patch, we introduced a new flag (FAULT_FLAG_INTERRUPTIBLE) to show that the fault handler allows the fault procedure to respond even to non-fatal signals. Meanwhile, add this new flag to the default fault flags so that all the page fault handlers can benefit from the new flag. With that, replacing the userfault check to this one. Since the line is getting even longer, clean up the fault flags a bit too to ease TTY users. Although we've got a new flag and applied it, we shouldn't have any functional change with this patch so far. Suggested-by: Linus Torvalds Reviewed-by: David Hildenbrand Signed-off-by: Peter Xu --- fs/userfaultfd.c | 4 +--- include/linux/mm.h | 39 ++++++++++++++++++++++++++++----------- 2 files changed, 29 insertions(+), 14 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index d8777146bae7..2b3b48e94ae4 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -462,9 +462,7 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason) uwq.ctx = ctx; uwq.waken = false; - return_to_userland = - (vmf->flags & (FAULT_FLAG_USER|FAULT_FLAG_KILLABLE)) == - (FAULT_FLAG_USER|FAULT_FLAG_KILLABLE); + return_to_userland = vmf->flags & FAULT_FLAG_INTERRUPTIBLE; blocking_state = return_to_userland ? TASK_INTERRUPTIBLE : TASK_KILLABLE; diff --git a/include/linux/mm.h b/include/linux/mm.h index 57fb5c535f8e..53ec7abb8472 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -383,22 +383,38 @@ extern unsigned int kobjsize(const void *objp); */ extern pgprot_t protection_map[16]; -#define FAULT_FLAG_WRITE 0x01 /* Fault was a write access */ -#define FAULT_FLAG_MKWRITE 0x02 /* Fault was mkwrite of existing pte */ -#define FAULT_FLAG_ALLOW_RETRY 0x04 /* Retry fault if blocking */ -#define FAULT_FLAG_RETRY_NOWAIT 0x08 /* Don't drop mmap_sem and wait when retrying */ -#define FAULT_FLAG_KILLABLE 0x10 /* The fault task is in SIGKILL killable region */ -#define FAULT_FLAG_TRIED 0x20 /* Second try */ -#define FAULT_FLAG_USER 0x40 /* The fault originated in userspace */ -#define FAULT_FLAG_REMOTE 0x80 /* faulting for non current tsk/mm */ -#define FAULT_FLAG_INSTRUCTION 0x100 /* The fault was during an instruction fetch */ +/** + * Fault flag definitions. + * + * @FAULT_FLAG_WRITE: Fault was a write fault. + * @FAULT_FLAG_MKWRITE: Fault was mkwrite of existing PTE. + * @FAULT_FLAG_ALLOW_RETRY: Allow to retry the fault if blocked. + * @FAULT_FLAG_RETRY_NOWAIT: Don't drop mmap_sem and wait when retrying. + * @FAULT_FLAG_KILLABLE: The fault task is in SIGKILL killable region. + * @FAULT_FLAG_TRIED: The fault has been tried once. + * @FAULT_FLAG_USER: The fault originated in userspace. + * @FAULT_FLAG_REMOTE: The fault is not for current task/mm. + * @FAULT_FLAG_INSTRUCTION: The fault was during an instruction fetch. + * @FAULT_FLAG_INTERRUPTIBLE: The fault can be interrupted by non-fatal signals. + */ +#define FAULT_FLAG_WRITE 0x01 +#define FAULT_FLAG_MKWRITE 0x02 +#define FAULT_FLAG_ALLOW_RETRY 0x04 +#define FAULT_FLAG_RETRY_NOWAIT 0x08 +#define FAULT_FLAG_KILLABLE 0x10 +#define FAULT_FLAG_TRIED 0x20 +#define FAULT_FLAG_USER 0x40 +#define FAULT_FLAG_REMOTE 0x80 +#define FAULT_FLAG_INSTRUCTION 0x100 +#define FAULT_FLAG_INTERRUPTIBLE 0x200 /* * The default fault flags that should be used by most of the * arch-specific page fault handlers. */ #define FAULT_FLAG_DEFAULT (FAULT_FLAG_ALLOW_RETRY | \ - FAULT_FLAG_KILLABLE) + FAULT_FLAG_KILLABLE | \ + FAULT_FLAG_INTERRUPTIBLE) #define FAULT_FLAG_TRACE \ { FAULT_FLAG_WRITE, "WRITE" }, \ @@ -409,7 +425,8 @@ extern pgprot_t protection_map[16]; { FAULT_FLAG_TRIED, "TRIED" }, \ { FAULT_FLAG_USER, "USER" }, \ { FAULT_FLAG_REMOTE, "REMOTE" }, \ - { FAULT_FLAG_INSTRUCTION, "INSTRUCTION" } + { FAULT_FLAG_INSTRUCTION, "INSTRUCTION" }, \ + { FAULT_FLAG_INTERRUPTIBLE, "INTERRUPTIBLE" } /* * vm_fault is filled by the the pagefault handler and passed to the vma's From patchwork Thu Sep 26 09:39:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162035 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E50B31709 for ; Thu, 26 Sep 2019 09:41:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 928DB222C4 for ; Thu, 26 Sep 2019 09:41:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 928DB222C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AD25A6B026D; Thu, 26 Sep 2019 05:41:10 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A365F6B026E; Thu, 26 Sep 2019 05:41:10 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8FDD56B026F; Thu, 26 Sep 2019 05:41:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0049.hostedemail.com [216.40.44.49]) by kanga.kvack.org (Postfix) with ESMTP id 5F4676B026D for ; Thu, 26 Sep 2019 05:41:10 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 08ED1181AC9AE for ; Thu, 26 Sep 2019 09:41:10 +0000 (UTC) X-FDA: 75976578300.25.fight87_7e87210305c24 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30003:30012:30054:30070:30075,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: fight87_7e87210305c24 X-Filterd-Recvd-Size: 23347 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:41:09 +0000 (UTC) Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8E727C00A17B for ; Thu, 26 Sep 2019 09:41:08 +0000 (UTC) Received: by mail-pg1-f199.google.com with SMTP id h36so1090112pgb.3 for ; Thu, 26 Sep 2019 02:41:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9xvVlxRGxtjhVhVH4rXxp2l9DbwjrleHIi56X/lq4MQ=; b=f+M5HfXCRQV153pPvBlbkvcBQ/q4G+mn/H9L6NIFKS5gjiCtS16vJZvbeAVtD+SXCx jWa4P5nxKPFSpisHUB6qtcoh2NX65u4/L3Sg7+WzxzHoMGj+16iECdMmKe9pJ1dTlyjt f7IL5M9iVLMnmcONjmUgtIOltB1/PFhZ+O90vWAIAASdEl0bunx1HP2UPAVGy78PPmEc kJ6y+ARYS90ZF7FIhbWNX4t3iva/7s4ZOjM4tSuy2fZAeN11iaN7OClRqf9qqD6uYUyz xqzUldFPHEbGY664sWTE585JTXDnAPT6VQTKzkZ2sw9BNeyG1BNJ+RIzxR2bchTBjHY+ 0QQA== X-Gm-Message-State: APjAAAW4W/4bE6uCR5jDcIp/GRKCcORB1z4QBl/8N0GBW/CENTjFE0ye kRRhB9MQEYuxXGTz1hqfIdWJprSJPQr974LV18hGB74EpMHAfX94OqAl3REgRJHYJgw1Hvn1Ss2 upmXpr0QBoA8= X-Received: by 2002:a17:902:9f97:: with SMTP id g23mr542330plq.114.1569490867591; Thu, 26 Sep 2019 02:41:07 -0700 (PDT) X-Google-Smtp-Source: APXvYqwu8cRNuLpOyk6hFmjcNajbI8jzJJ3EsNfJf0Ih1wO4YbiOHoMq+4Q3/xAbiAdls5BijuMYxg== X-Received: by 2002:a17:902:9f97:: with SMTP id g23mr542269plq.114.1569490866981; Thu, 26 Sep 2019 02:41:06 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.40.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:41:06 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 13/16] mm: Allow VM_FAULT_RETRY for multiple times Date: Thu, 26 Sep 2019 17:39:01 +0800 Message-Id: <20190926093904.5090-14-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The idea comes from a discussion between Linus and Andrea [1]. Before this patch we only allow a page fault to retry once. We achieved this by clearing the FAULT_FLAG_ALLOW_RETRY flag when doing handle_mm_fault() the second time. This was majorly used to avoid unexpected starvation of the system by looping over forever to handle the page fault on a single page. However that should hardly happen, and after all for each code path to return a VM_FAULT_RETRY we'll first wait for a condition (during which time we should possibly yield the cpu) to happen before VM_FAULT_RETRY is really returned. This patch removes the restriction by keeping the FAULT_FLAG_ALLOW_RETRY flag when we receive VM_FAULT_RETRY. It means that the page fault handler now can retry the page fault for multiple times if necessary without the need to generate another page fault event. Meanwhile we still keep the FAULT_FLAG_TRIED flag so page fault handler can still identify whether a page fault is the first attempt or not. Then we'll have these combinations of fault flags (only considering ALLOW_RETRY flag and TRIED flag): - ALLOW_RETRY and !TRIED: this means the page fault allows to retry, and this is the first try - ALLOW_RETRY and TRIED: this means the page fault allows to retry, and this is not the first try - !ALLOW_RETRY and !TRIED: this means the page fault does not allow to retry at all - !ALLOW_RETRY and TRIED: this is forbidden and should never be used In existing code we have multiple places that has taken special care of the first condition above by checking against (fault_flags & FAULT_FLAG_ALLOW_RETRY). This patch introduces a simple helper to detect the first retry of a page fault by checking against both (fault_flags & FAULT_FLAG_ALLOW_RETRY) and !(fault_flag & FAULT_FLAG_TRIED) because now even the 2nd try will have the ALLOW_RETRY set, then use that helper in all existing special paths. One example is in __lock_page_or_retry(), now we'll drop the mmap_sem only in the first attempt of page fault and we'll keep it in follow up retries, so old locking behavior will be retained. This will be a nice enhancement for current code [2] at the same time a supporting material for the future userfaultfd-writeprotect work, since in that work there will always be an explicit userfault writeprotect retry for protected pages, and if that cannot resolve the page fault (e.g., when userfaultfd-writeprotect is used in conjunction with swapped pages) then we'll possibly need a 3rd retry of the page fault. It might also benefit other potential users who will have similar requirement like userfault write-protection. GUP code is not touched yet and will be covered in follow up patch. Please read the thread below for more information. [1] https://lore.kernel.org/lkml/20171102193644.GB22686@redhat.com/ [2] https://lore.kernel.org/lkml/20181230154648.GB9832@redhat.com/ Suggested-by: Linus Torvalds Suggested-by: Andrea Arcangeli Reviewed-by: Jerome Glisse Signed-off-by: Peter Xu --- arch/alpha/mm/fault.c | 2 +- arch/arc/mm/fault.c | 1 - arch/arm/mm/fault.c | 3 --- arch/arm64/mm/fault.c | 5 ----- arch/hexagon/mm/vm_fault.c | 1 - arch/ia64/mm/fault.c | 1 - arch/m68k/mm/fault.c | 3 --- arch/microblaze/mm/fault.c | 1 - arch/mips/mm/fault.c | 1 - arch/nds32/mm/fault.c | 1 - arch/nios2/mm/fault.c | 3 --- arch/openrisc/mm/fault.c | 1 - arch/parisc/mm/fault.c | 4 +--- arch/powerpc/mm/fault.c | 6 ------ arch/riscv/mm/fault.c | 5 ----- arch/s390/mm/fault.c | 5 +---- arch/sh/mm/fault.c | 1 - arch/sparc/mm/fault_32.c | 1 - arch/sparc/mm/fault_64.c | 1 - arch/um/kernel/trap.c | 1 - arch/unicore32/mm/fault.c | 4 +--- arch/x86/mm/fault.c | 2 -- arch/xtensa/mm/fault.c | 1 - drivers/gpu/drm/ttm/ttm_bo_vm.c | 12 ++++++++--- include/linux/mm.h | 37 +++++++++++++++++++++++++++++++++ mm/filemap.c | 2 +- mm/shmem.c | 2 +- 27 files changed, 52 insertions(+), 55 deletions(-) diff --git a/arch/alpha/mm/fault.c b/arch/alpha/mm/fault.c index fcfa229cc1e7..c2d7b6d7bac7 100644 --- a/arch/alpha/mm/fault.c +++ b/arch/alpha/mm/fault.c @@ -169,7 +169,7 @@ do_page_fault(unsigned long address, unsigned long mmcsr, else current->min_flt++; if (fault & VM_FAULT_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; + flags |= FAULT_FLAG_TRIED; /* No need to up_read(&mm->mmap_sem) as we would * have already released it in __lock_page_or_retry diff --git a/arch/arc/mm/fault.c b/arch/arc/mm/fault.c index 33a609f1c43c..fdf09db1581d 100644 --- a/arch/arc/mm/fault.c +++ b/arch/arc/mm/fault.c @@ -139,7 +139,6 @@ void do_page_fault(unsigned long address, struct pt_regs *regs) */ if (unlikely((fault & VM_FAULT_RETRY) && (flags & FAULT_FLAG_ALLOW_RETRY))) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; goto retry; } diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index 101ae6698637..952bd9d7487d 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -319,9 +319,6 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) regs, addr); } if (fault & VM_FAULT_RETRY) { - /* Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk - * of starvation. */ - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; goto retry; } diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 23e3feca0f02..b6badba3a677 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -485,12 +485,7 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, } if (fault & VM_FAULT_RETRY) { - /* - * Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk of - * starvation. - */ if (mm_flags & FAULT_FLAG_ALLOW_RETRY) { - mm_flags &= ~FAULT_FLAG_ALLOW_RETRY; mm_flags |= FAULT_FLAG_TRIED; goto retry; } diff --git a/arch/hexagon/mm/vm_fault.c b/arch/hexagon/mm/vm_fault.c index d9e15d941bdb..72334b26317a 100644 --- a/arch/hexagon/mm/vm_fault.c +++ b/arch/hexagon/mm/vm_fault.c @@ -102,7 +102,6 @@ void do_page_fault(unsigned long address, long cause, struct pt_regs *regs) else current->min_flt++; if (fault & VM_FAULT_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; goto retry; } diff --git a/arch/ia64/mm/fault.c b/arch/ia64/mm/fault.c index b5aa4e80c762..30d0c1fca99e 100644 --- a/arch/ia64/mm/fault.c +++ b/arch/ia64/mm/fault.c @@ -167,7 +167,6 @@ ia64_do_page_fault (unsigned long address, unsigned long isr, struct pt_regs *re else current->min_flt++; if (fault & VM_FAULT_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; /* No need to up_read(&mm->mmap_sem) as we would diff --git a/arch/m68k/mm/fault.c b/arch/m68k/mm/fault.c index 182799fd9987..f7afb9897966 100644 --- a/arch/m68k/mm/fault.c +++ b/arch/m68k/mm/fault.c @@ -162,9 +162,6 @@ int do_page_fault(struct pt_regs *regs, unsigned long address, else current->min_flt++; if (fault & VM_FAULT_RETRY) { - /* Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk - * of starvation. */ - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; /* diff --git a/arch/microblaze/mm/fault.c b/arch/microblaze/mm/fault.c index 32da02778a63..3248141f8ed5 100644 --- a/arch/microblaze/mm/fault.c +++ b/arch/microblaze/mm/fault.c @@ -236,7 +236,6 @@ void do_page_fault(struct pt_regs *regs, unsigned long address, else current->min_flt++; if (fault & VM_FAULT_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; /* diff --git a/arch/mips/mm/fault.c b/arch/mips/mm/fault.c index f811ef5efb80..02ae0c87301f 100644 --- a/arch/mips/mm/fault.c +++ b/arch/mips/mm/fault.c @@ -178,7 +178,6 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, unsigned long write, tsk->min_flt++; } if (fault & VM_FAULT_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; /* diff --git a/arch/nds32/mm/fault.c b/arch/nds32/mm/fault.c index 4e402f2c88c3..52097d5f63c6 100644 --- a/arch/nds32/mm/fault.c +++ b/arch/nds32/mm/fault.c @@ -242,7 +242,6 @@ void do_page_fault(unsigned long entry, unsigned long addr, 1, regs, addr); } if (fault & VM_FAULT_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; /* No need to up_read(&mm->mmap_sem) as we would diff --git a/arch/nios2/mm/fault.c b/arch/nios2/mm/fault.c index c38bea4220fb..ec9d8a9c426f 100644 --- a/arch/nios2/mm/fault.c +++ b/arch/nios2/mm/fault.c @@ -157,9 +157,6 @@ asmlinkage void do_page_fault(struct pt_regs *regs, unsigned long cause, else current->min_flt++; if (fault & VM_FAULT_RETRY) { - /* Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk - * of starvation. */ - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; /* diff --git a/arch/openrisc/mm/fault.c b/arch/openrisc/mm/fault.c index 30d5c51e9d40..8af1cc78c4fb 100644 --- a/arch/openrisc/mm/fault.c +++ b/arch/openrisc/mm/fault.c @@ -181,7 +181,6 @@ asmlinkage void do_page_fault(struct pt_regs *regs, unsigned long address, else tsk->min_flt++; if (fault & VM_FAULT_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; /* No need to up_read(&mm->mmap_sem) as we would diff --git a/arch/parisc/mm/fault.c b/arch/parisc/mm/fault.c index 8e88e5c5f26a..86e8c848f3d7 100644 --- a/arch/parisc/mm/fault.c +++ b/arch/parisc/mm/fault.c @@ -328,14 +328,12 @@ void do_page_fault(struct pt_regs *regs, unsigned long code, else current->min_flt++; if (fault & VM_FAULT_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; - /* * No need to up_read(&mm->mmap_sem) as we would * have already released it in __lock_page_or_retry * in mm/filemap.c. */ - + flags |= FAULT_FLAG_TRIED; goto retry; } } diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index db156537a2fa..65cc735045bb 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -591,13 +591,7 @@ static int __do_page_fault(struct pt_regs *regs, unsigned long address, * case. */ if (unlikely(fault & VM_FAULT_RETRY)) { - /* We retry only once */ if (flags & FAULT_FLAG_ALLOW_RETRY) { - /* - * Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk - * of starvation. - */ - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; goto retry; } diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c index 1f77dd2c491c..12954f02cf02 100644 --- a/arch/riscv/mm/fault.c +++ b/arch/riscv/mm/fault.c @@ -142,11 +142,6 @@ asmlinkage void do_page_fault(struct pt_regs *regs) 1, regs, addr); } if (fault & VM_FAULT_RETRY) { - /* - * Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk - * of starvation. - */ - flags &= ~(FAULT_FLAG_ALLOW_RETRY); flags |= FAULT_FLAG_TRIED; /* diff --git a/arch/s390/mm/fault.c b/arch/s390/mm/fault.c index 551ac311bd35..aeccdb30899a 100644 --- a/arch/s390/mm/fault.c +++ b/arch/s390/mm/fault.c @@ -513,10 +513,7 @@ static inline vm_fault_t do_exception(struct pt_regs *regs, int access) fault = VM_FAULT_PFAULT; goto out_up; } - /* Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk - * of starvation. */ - flags &= ~(FAULT_FLAG_ALLOW_RETRY | - FAULT_FLAG_RETRY_NOWAIT); + flags &= ~FAULT_FLAG_RETRY_NOWAIT; flags |= FAULT_FLAG_TRIED; down_read(&mm->mmap_sem); goto retry; diff --git a/arch/sh/mm/fault.c b/arch/sh/mm/fault.c index d9c8f2d00a54..13ee4d20e622 100644 --- a/arch/sh/mm/fault.c +++ b/arch/sh/mm/fault.c @@ -481,7 +481,6 @@ asmlinkage void __kprobes do_page_fault(struct pt_regs *regs, regs, address); } if (fault & VM_FAULT_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; /* diff --git a/arch/sparc/mm/fault_32.c b/arch/sparc/mm/fault_32.c index 447f61c4e996..a8c8cb3b891f 100644 --- a/arch/sparc/mm/fault_32.c +++ b/arch/sparc/mm/fault_32.c @@ -261,7 +261,6 @@ asmlinkage void do_sparc_fault(struct pt_regs *regs, int text_fault, int write, 1, regs, address); } if (fault & VM_FAULT_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; /* No need to up_read(&mm->mmap_sem) as we would diff --git a/arch/sparc/mm/fault_64.c b/arch/sparc/mm/fault_64.c index 6807fba66331..eb4cfd5634e2 100644 --- a/arch/sparc/mm/fault_64.c +++ b/arch/sparc/mm/fault_64.c @@ -445,7 +445,6 @@ asmlinkage void __kprobes do_sparc64_fault(struct pt_regs *regs) 1, regs, address); } if (fault & VM_FAULT_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; /* No need to up_read(&mm->mmap_sem) as we would diff --git a/arch/um/kernel/trap.c b/arch/um/kernel/trap.c index bc2756782d64..4bcf5873e931 100644 --- a/arch/um/kernel/trap.c +++ b/arch/um/kernel/trap.c @@ -96,7 +96,6 @@ int handle_page_fault(unsigned long address, unsigned long ip, else current->min_flt++; if (fault & VM_FAULT_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; goto retry; diff --git a/arch/unicore32/mm/fault.c b/arch/unicore32/mm/fault.c index 34a90453ca18..a9bd08fbe588 100644 --- a/arch/unicore32/mm/fault.c +++ b/arch/unicore32/mm/fault.c @@ -259,9 +259,7 @@ static int do_pf(unsigned long addr, unsigned int fsr, struct pt_regs *regs) else tsk->min_flt++; if (fault & VM_FAULT_RETRY) { - /* Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk - * of starvation. */ - flags &= ~FAULT_FLAG_ALLOW_RETRY; + flags |= FAULT_FLAG_TRIED; goto retry; } } diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index d211585cfffd..55e749af8a61 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -1456,8 +1456,6 @@ void do_user_addr_fault(struct pt_regs *regs, */ if (unlikely((fault & VM_FAULT_RETRY) && (flags & FAULT_FLAG_ALLOW_RETRY))) { - /* Retry at most once */ - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; goto retry; } diff --git a/arch/xtensa/mm/fault.c b/arch/xtensa/mm/fault.c index 6467e3fb9762..c97c2c84d14a 100644 --- a/arch/xtensa/mm/fault.c +++ b/arch/xtensa/mm/fault.c @@ -128,7 +128,6 @@ void do_page_fault(struct pt_regs *regs) else current->min_flt++; if (fault & VM_FAULT_RETRY) { - flags &= ~FAULT_FLAG_ALLOW_RETRY; flags |= FAULT_FLAG_TRIED; /* No need to up_read(&mm->mmap_sem) as we would diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index 6dacff49c1cc..8f2f9ee6effa 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -61,9 +61,10 @@ static vm_fault_t ttm_bo_vm_fault_idle(struct ttm_buffer_object *bo, /* * If possible, avoid waiting for GPU with mmap_sem - * held. + * held. We only do this if the fault allows retry and this + * is the first attempt. */ - if (vmf->flags & FAULT_FLAG_ALLOW_RETRY) { + if (fault_flag_allow_retry_first(vmf->flags)) { ret = VM_FAULT_RETRY; if (vmf->flags & FAULT_FLAG_RETRY_NOWAIT) goto out_unlock; @@ -132,7 +133,12 @@ static vm_fault_t ttm_bo_vm_fault(struct vm_fault *vmf) * for the buffer to become unreserved. */ if (unlikely(!reservation_object_trylock(bo->resv))) { - if (vmf->flags & FAULT_FLAG_ALLOW_RETRY) { + /* + * If the fault allows retry and this is the first + * fault attempt, we try to release the mmap_sem + * before waiting + */ + if (fault_flag_allow_retry_first(vmf->flags)) { if (!(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) { ttm_bo_get(bo); up_read(&vmf->vma->vm_mm->mmap_sem); diff --git a/include/linux/mm.h b/include/linux/mm.h index 53ec7abb8472..0fdbdcb257d6 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -396,6 +396,25 @@ extern pgprot_t protection_map[16]; * @FAULT_FLAG_REMOTE: The fault is not for current task/mm. * @FAULT_FLAG_INSTRUCTION: The fault was during an instruction fetch. * @FAULT_FLAG_INTERRUPTIBLE: The fault can be interrupted by non-fatal signals. + * + * About @FAULT_FLAG_ALLOW_RETRY and @FAULT_FLAG_TRIED: we can specify + * whether we would allow page faults to retry by specifying these two + * fault flags correctly. Currently there can be three legal combinations: + * + * (a) ALLOW_RETRY and !TRIED: this means the page fault allows retry, and + * this is the first try + * + * (b) ALLOW_RETRY and TRIED: this means the page fault allows retry, and + * we've already tried at least once + * + * (c) !ALLOW_RETRY and !TRIED: this means the page fault does not allow retry + * + * The unlisted combination (!ALLOW_RETRY && TRIED) is illegal and should never + * be used. Note that page faults can be allowed to retry for multiple times, + * in which case we'll have an initial fault with flags (a) then later on + * continuous faults with flags (b). We should always try to detect pending + * signals before a retry to make sure the continuous page faults can still be + * interrupted if necessary. */ #define FAULT_FLAG_WRITE 0x01 #define FAULT_FLAG_MKWRITE 0x02 @@ -416,6 +435,24 @@ extern pgprot_t protection_map[16]; FAULT_FLAG_KILLABLE | \ FAULT_FLAG_INTERRUPTIBLE) +/** + * fault_flag_allow_retry_first - check ALLOW_RETRY the first time + * + * This is mostly used for places where we want to try to avoid taking + * the mmap_sem for too long a time when waiting for another condition + * to change, in which case we can try to be polite to release the + * mmap_sem in the first round to avoid potential starvation of other + * processes that would also want the mmap_sem. + * + * Return: true if the page fault allows retry and this is the first + * attempt of the fault handling; false otherwise. + */ +static inline bool fault_flag_allow_retry_first(unsigned int flags) +{ + return (flags & FAULT_FLAG_ALLOW_RETRY) && + (!(flags & FAULT_FLAG_TRIED)); +} + #define FAULT_FLAG_TRACE \ { FAULT_FLAG_WRITE, "WRITE" }, \ { FAULT_FLAG_MKWRITE, "MKWRITE" }, \ diff --git a/mm/filemap.c b/mm/filemap.c index d0cf700bf201..543404617f5a 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1399,7 +1399,7 @@ EXPORT_SYMBOL_GPL(__lock_page_killable); int __lock_page_or_retry(struct page *page, struct mm_struct *mm, unsigned int flags) { - if (flags & FAULT_FLAG_ALLOW_RETRY) { + if (fault_flag_allow_retry_first(flags)) { /* * CAUTION! In this case, mmap_sem is not released * even though return 0. diff --git a/mm/shmem.c b/mm/shmem.c index 2bed4761f279..1af8d8e60231 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2011,7 +2011,7 @@ static vm_fault_t shmem_fault(struct vm_fault *vmf) DEFINE_WAIT_FUNC(shmem_fault_wait, synchronous_wake_function); ret = VM_FAULT_NOPAGE; - if ((vmf->flags & FAULT_FLAG_ALLOW_RETRY) && + if (fault_flag_allow_retry_first(vmf->flags) && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) { /* It's polite to up mmap_sem if we can */ up_read(&vma->vm_mm->mmap_sem); From patchwork Thu Sep 26 09:39:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162037 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9957F76 for ; Thu, 26 Sep 2019 09:41:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5993F222C4 for ; Thu, 26 Sep 2019 09:41:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5993F222C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 862A16B026F; Thu, 26 Sep 2019 05:41:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7EB9E6B0270; Thu, 26 Sep 2019 05:41:18 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D9B06B0271; Thu, 26 Sep 2019 05:41:18 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0114.hostedemail.com [216.40.44.114]) by kanga.kvack.org (Postfix) with ESMTP id 45BAE6B026F for ; Thu, 26 Sep 2019 05:41:18 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 04368180AD802 for ; Thu, 26 Sep 2019 09:41:18 +0000 (UTC) X-FDA: 75976578636.19.face70_7fb6439e20139 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30054:30070,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: face70_7fb6439e20139 X-Filterd-Recvd-Size: 6239 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:41:17 +0000 (UTC) Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 673B4C057F31 for ; Thu, 26 Sep 2019 09:41:16 +0000 (UTC) Received: by mail-pf1-f197.google.com with SMTP id w126so1338997pfd.22 for ; Thu, 26 Sep 2019 02:41:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AKkEfjdoHGbVoKbr4xUEMS07lQumUdJTKR+cWS6/cLc=; b=ifdlMv+/QF8BgMYONdlLur3vgEL97iwmDzRZUwp8ZWJQS1/KKQcLEvK1O03pJaEbAn M9KjD0yKlqoTJSZjGlAeK9vHZ3g/k534WjyHlxk5r9iLZXNTuGxglvLX05hCO8R5bbRl e5HIwOzH4LyZQ2U0BtHd4BJjTqgO0pAvNS5GD3pfy7IuYucOCDjzzCndx7t+RciOVKPW TkajeddVl8HQyMjaEdoQ3KKauLUPynCWCC5kIH6HbYMmLJnL9A7ytqfdbw5XcSw//ZhP 4vDKpS9Wf+7Tov31hIcKyRf0F98TbuiBj4riDVLJvMXXkV9eRI5RBQBomuILssIYq6F5 oeOA== X-Gm-Message-State: APjAAAXcZHsGAuClXckYEgWgSQRyHd15LB3wI7RLbvOv4rXfJ/vLoU0z sXViW1kZ5WfdV2H29mNg6Z3mYWlt4Wqv50jMWzLxyulNEEwbFq9GJAP7ctBW6wVlTDaFRrZCBos rynBp4NtvCfg= X-Received: by 2002:a63:550a:: with SMTP id j10mr2421065pgb.369.1569490875459; Thu, 26 Sep 2019 02:41:15 -0700 (PDT) X-Google-Smtp-Source: APXvYqxqpoKkAjxWiSXeTQqhUit0kZdkveOCToqkIiL/LTZ+4SbaywQVnIYZ9dztItKxDR6cACgk0A== X-Received: by 2002:a63:550a:: with SMTP id j10mr2421036pgb.369.1569490875206; Thu, 26 Sep 2019 02:41:15 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.41.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:41:14 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 14/16] mm/gup: Allow VM_FAULT_RETRY for multiple times Date: Thu, 26 Sep 2019 17:39:02 +0800 Message-Id: <20190926093904.5090-15-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is the gup counterpart of the change that allows the VM_FAULT_RETRY to happen for more than once. One thing to mention is that we must check the fatal signal here before retry because the GUP can be interrupted by that, otherwise we can loop forever. Signed-off-by: Peter Xu --- mm/gup.c | 27 +++++++++++++++++++++------ mm/hugetlb.c | 6 ++++-- 2 files changed, 25 insertions(+), 8 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index e60d32f1674d..d2811bb15a25 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -644,7 +644,10 @@ static int faultin_page(struct task_struct *tsk, struct vm_area_struct *vma, if (*flags & FOLL_NOWAIT) fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_RETRY_NOWAIT; if (*flags & FOLL_TRIED) { - VM_WARN_ON_ONCE(fault_flags & FAULT_FLAG_ALLOW_RETRY); + /* + * Note: FAULT_FLAG_ALLOW_RETRY and FAULT_FLAG_TRIED + * can co-exist + */ fault_flags |= FAULT_FLAG_TRIED; } @@ -994,7 +997,6 @@ int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm, down_read(&mm->mmap_sem); if (!(fault_flags & FAULT_FLAG_TRIED)) { *unlocked = true; - fault_flags &= ~FAULT_FLAG_ALLOW_RETRY; fault_flags |= FAULT_FLAG_TRIED; goto retry; } @@ -1069,17 +1071,30 @@ static __always_inline long __get_user_pages_locked(struct task_struct *tsk, if (likely(pages)) pages += ret; start += ret << PAGE_SHIFT; + lock_dropped = true; +retry: /* * Repeat on the address that fired VM_FAULT_RETRY - * without FAULT_FLAG_ALLOW_RETRY but with - * FAULT_FLAG_TRIED. + * with both FAULT_FLAG_ALLOW_RETRY and + * FAULT_FLAG_TRIED. Note that GUP can be interrupted + * by fatal signals, so we need to check it before we + * start trying again otherwise it can loop forever. */ + + if (fatal_signal_pending(current)) + break; + *locked = 1; - lock_dropped = true; down_read(&mm->mmap_sem); + ret = __get_user_pages(tsk, mm, start, 1, flags | FOLL_TRIED, - pages, NULL, NULL); + pages, NULL, locked); + if (!*locked) { + /* Continue to retry until we succeeded */ + BUG_ON(ret != 0); + goto retry; + } if (ret != 1) { BUG_ON(ret > 1); if (!pages_done) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 31c2a6275023..d0c98cff5b0f 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -4347,8 +4347,10 @@ long follow_hugetlb_page(struct mm_struct *mm, struct vm_area_struct *vma, fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_RETRY_NOWAIT; if (flags & FOLL_TRIED) { - VM_WARN_ON_ONCE(fault_flags & - FAULT_FLAG_ALLOW_RETRY); + /* + * Note: FAULT_FLAG_ALLOW_RETRY and + * FAULT_FLAG_TRIED can co-exist + */ fault_flags |= FAULT_FLAG_TRIED; } ret = hugetlb_fault(mm, vma, vaddr, fault_flags); From patchwork Thu Sep 26 09:39:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162039 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 42B35924 for ; Thu, 26 Sep 2019 09:41:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 10078222C3 for ; Thu, 26 Sep 2019 09:41:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 10078222C3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 506318E0001; Thu, 26 Sep 2019 05:41:27 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4909E6B0272; Thu, 26 Sep 2019 05:41:27 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 357368E0001; Thu, 26 Sep 2019 05:41:27 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0178.hostedemail.com [216.40.44.178]) by kanga.kvack.org (Postfix) with ESMTP id 0E0806B0271 for ; Thu, 26 Sep 2019 05:41:27 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id B783D256EE for ; Thu, 26 Sep 2019 09:41:26 +0000 (UTC) X-FDA: 75976578972.27.ants35_80fa8bb4dff63 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30003:30012:30036:30054,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:1,LUA_SUMMARY:none X-HE-Tag: ants35_80fa8bb4dff63 X-Filterd-Recvd-Size: 5838 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:41:26 +0000 (UTC) Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 87B06C057F88 for ; Thu, 26 Sep 2019 09:41:25 +0000 (UTC) Received: by mail-pg1-f198.google.com with SMTP id w13so1071512pge.15 for ; Thu, 26 Sep 2019 02:41:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Zt8EwZyFUvlwT1VYMNYV6I/TnTem7aMxIFLElv6YJ/M=; b=myw+xMugXwmSOvdMOhN+AxLjQ8W0ZdBZ9xQ5Sq670EHHowuOAefGYOmIu/49UKBAZn aI0NhQ+Wq8zbkOvaFMmxzDzNF7APds3ij0LpzEt4kn98P5ncRbYl6o5NosVK0t7tb95Q 43oJyHEeaT6JYwMF5r4vEYDKk9Bx1Nm033TB7019dFLoQpQyuAiXDQlz/Pbhq49WZobm XHRQcgel+ui6nSUa2bnQoGHpwp9xNMLrPQ1gAJOKJ0Z4uOgxcHrlY98sNp0Eizw34iHP loM966nEUw8wM9Ocp+xrFN3fv56jPXS8WybDXXzL2fTXJGA6EeXCB1T5yJoJFGTdeSuN FDvg== X-Gm-Message-State: APjAAAVtefd113uzVO5wa7otxaWUanVoi345+jn/MIVmG1YKt7WhAyAT 5wOWLTa9MUdicA8S+MSHJjnF+iVfer/+J+Qmox1fabm/k2iG7jGbwEhmBvxt2ExSV+L7gd7OCgZ Qjw1Slanrpjo= X-Received: by 2002:a17:90a:356d:: with SMTP id q100mr2591438pjb.53.1569490884000; Thu, 26 Sep 2019 02:41:24 -0700 (PDT) X-Google-Smtp-Source: APXvYqwUAI+/pkWVPqCwKNAGhjD7zSseN3FtCOt463W8scMo/2YSN4J0PKUhijZxIeZhAEgd+EwMDQ== X-Received: by 2002:a17:90a:356d:: with SMTP id q100mr2591405pjb.53.1569490883699; Thu, 26 Sep 2019 02:41:23 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.41.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:41:23 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 15/16] mm/gup: Allow to react to fatal signals Date: Thu, 26 Sep 2019 17:39:03 +0800 Message-Id: <20190926093904.5090-16-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The existing gup code does not react to the fatal signals in many code paths. For example, in one retry path of gup we're still using down_read() rather than down_read_killable(). Also, when doing page faults we don't pass in FAULT_FLAG_KILLABLE as well, which means that within the faulting process we'll wait in non-killable way as well. These were spotted by Linus during the code review of some other patches. Let's allow the gup code to react to fatal signals to improve the responsiveness of threads when during gup and being killed. Signed-off-by: Peter Xu --- mm/gup.c | 12 +++++++++--- mm/hugetlb.c | 3 ++- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index d2811bb15a25..4c638473db83 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -640,7 +640,7 @@ static int faultin_page(struct task_struct *tsk, struct vm_area_struct *vma, if (*flags & FOLL_REMOTE) fault_flags |= FAULT_FLAG_REMOTE; if (locked) - fault_flags |= FAULT_FLAG_ALLOW_RETRY; + fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; if (*flags & FOLL_NOWAIT) fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_RETRY_NOWAIT; if (*flags & FOLL_TRIED) { @@ -973,7 +973,7 @@ int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm, vm_fault_t ret, major = 0; if (unlocked) - fault_flags |= FAULT_FLAG_ALLOW_RETRY; + fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; retry: vma = find_extend_vma(mm, address); @@ -1086,7 +1086,13 @@ static __always_inline long __get_user_pages_locked(struct task_struct *tsk, break; *locked = 1; - down_read(&mm->mmap_sem); + ret = down_read_killable(&mm->mmap_sem); + if (ret) { + BUG_ON(ret > 0); + if (!pages_done) + pages_done = ret; + break; + } ret = __get_user_pages(tsk, mm, start, 1, flags | FOLL_TRIED, pages, NULL, locked); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index d0c98cff5b0f..84034154d50e 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -4342,7 +4342,8 @@ long follow_hugetlb_page(struct mm_struct *mm, struct vm_area_struct *vma, if (flags & FOLL_WRITE) fault_flags |= FAULT_FLAG_WRITE; if (locked) - fault_flags |= FAULT_FLAG_ALLOW_RETRY; + fault_flags |= FAULT_FLAG_ALLOW_RETRY | + FAULT_FLAG_KILLABLE; if (flags & FOLL_NOWAIT) fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_RETRY_NOWAIT; From patchwork Thu Sep 26 09:39:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11162041 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 86DA976 for ; Thu, 26 Sep 2019 09:41:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 46102222BE for ; Thu, 26 Sep 2019 09:41:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 46102222BE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6DB208E0006; Thu, 26 Sep 2019 05:41:35 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 665448E0005; Thu, 26 Sep 2019 05:41:35 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 504AF8E0006; Thu, 26 Sep 2019 05:41:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 29C328E0005 for ; Thu, 26 Sep 2019 05:41:35 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id D341729227 for ; Thu, 26 Sep 2019 09:41:34 +0000 (UTC) X-FDA: 75976579308.02.foot06_8225b3cc81b0e X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,peterx@redhat.com,::linux-kernel@vger.kernel.org:david@redhat.com:hughd@google.com:gokhale2@llnl.gov:jglisse@redhat.com:xemul@virtuozzo.com:hannes@cmpxchg.org:peterx@redhat.com:cracauer@cons.org:willy@infradead.org:shli@fb.com:mcfadden8@llnl.gov:aarcange@redhat.com:mike.kravetz@oracle.com:dplotnikov@virtuozzo.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:mgorman@suse.de:kirill@shutemov.name:dgilbert@redhat.com,RULES_HIT:30012:30054:30070,0,RBL:209.132.183.28:@redhat.com:.lbl8.mailshell.net-62.18.0.0 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: foot06_8225b3cc81b0e X-Filterd-Recvd-Size: 6606 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:41:34 +0000 (UTC) Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7C394C05683F for ; Thu, 26 Sep 2019 09:41:33 +0000 (UTC) Received: by mail-pl1-f200.google.com with SMTP id m8so1164252plt.14 for ; Thu, 26 Sep 2019 02:41:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=N4oZOlVdiq1iUNu2ipKdKrBjjBpOUecRmSvPVqDDXZc=; b=csTBdIeuc8BK6UXWcpclyohD5cVf284wKOS0o3b1BwyO/0LXWSwjK2ZRLvG8QrVLQn EIHEzkUTkCdorco5HTuS5Z0eH/Gq9hI50wx+T93hY7lqf1WvMYtGsta9y/qSL+cA+sGk yeOymtSXCvEUS3RGPsfNqbm9g0gF4sdbhavNyoIJ7aoQL53xqOTCcbWdHKqAxlTFhU9o mdKlUrnJCftXYf/5B+h6rwXAcml1DAOmTo8LVbHRDdkYRNeW5XmEFRckE4r9hN9/4/o+ udtIfgeXKp+EMRDGxC9di5Kw9Vz7BsRe+IbId10eOdH4wzKQ6WXG6Ss6oZ7NTUd/9MZQ Qo8g== X-Gm-Message-State: APjAAAX8wQ7B1VQ95ZR82NTuh8OKgl4bFv4qvBNVfBv9wjIccqT1QZpk aXeRnonjGu2OIr5k6kj06EY9lK0PD4O5pZNhIWdJbeULEJDYj1MdtcXgv3o4bJ0iMZmxgFWjEAU pjP6r+9wquWk= X-Received: by 2002:a63:f74a:: with SMTP id f10mr2441667pgk.171.1569490892245; Thu, 26 Sep 2019 02:41:32 -0700 (PDT) X-Google-Smtp-Source: APXvYqzXg+JR19ww+nQnL8fkRXYx+C2hv4fDrwyyYRZ7ETfndG7uYUVVmTKADTP0SGN/MJh50aMeMQ== X-Received: by 2002:a63:f74a:: with SMTP id f10mr2441638pgk.171.1569490892002; Thu, 26 Sep 2019 02:41:32 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.41.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:41:31 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 16/16] mm/userfaultfd: Honor FAULT_FLAG_KILLABLE in fault path Date: Thu, 26 Sep 2019 17:39:04 +0800 Message-Id: <20190926093904.5090-17-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Userfaultfd fault path was by default killable even if the caller does not have FAULT_FLAG_KILLABLE. That makes sense before in that when with gup we don't have FAULT_FLAG_KILLABLE properly set before. Now after previous patch we've got FAULT_FLAG_KILLABLE applied even for gup code so it should also make sense to let userfaultfd to honor the FAULT_FLAG_KILLABLE. Because we're unconditionally setting FAULT_FLAG_KILLABLE in gup code right now, this patch should have no functional change. It also cleaned the code a little bit by introducing some helpers. Signed-off-by: Peter Xu --- fs/userfaultfd.c | 36 ++++++++++++++++++++++++++++-------- 1 file changed, 28 insertions(+), 8 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 2b3b48e94ae4..8c5863ccbf0e 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -334,6 +334,30 @@ static inline bool userfaultfd_must_wait(struct userfaultfd_ctx *ctx, return ret; } +/* Should pair with userfaultfd_signal_pending() */ +static inline long userfaultfd_get_blocking_state(unsigned int flags) +{ + if (flags & FAULT_FLAG_INTERRUPTIBLE) + return TASK_INTERRUPTIBLE; + + if (flags & FAULT_FLAG_KILLABLE) + return TASK_KILLABLE; + + return TASK_UNINTERRUPTIBLE; +} + +/* Should pair with userfaultfd_get_blocking_state() */ +static inline bool userfaultfd_signal_pending(unsigned int flags) +{ + if (flags & FAULT_FLAG_INTERRUPTIBLE) + return signal_pending(current); + + if (flags & FAULT_FLAG_KILLABLE) + return fatal_signal_pending(current); + + return false; +} + /* * The locking rules involved in returning VM_FAULT_RETRY depending on * FAULT_FLAG_ALLOW_RETRY, FAULT_FLAG_RETRY_NOWAIT and @@ -355,7 +379,7 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason) struct userfaultfd_ctx *ctx; struct userfaultfd_wait_queue uwq; vm_fault_t ret = VM_FAULT_SIGBUS; - bool must_wait, return_to_userland; + bool must_wait; long blocking_state; /* @@ -462,9 +486,7 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason) uwq.ctx = ctx; uwq.waken = false; - return_to_userland = vmf->flags & FAULT_FLAG_INTERRUPTIBLE; - blocking_state = return_to_userland ? TASK_INTERRUPTIBLE : - TASK_KILLABLE; + blocking_state = userfaultfd_get_blocking_state(vmf->flags); spin_lock_irq(&ctx->fault_pending_wqh.lock); /* @@ -490,8 +512,7 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason) up_read(&mm->mmap_sem); if (likely(must_wait && !READ_ONCE(ctx->released) && - (return_to_userland ? !signal_pending(current) : - !fatal_signal_pending(current)))) { + userfaultfd_signal_pending(vmf->flags))) { wake_up_poll(&ctx->fd_wqh, EPOLLIN); schedule(); ret |= VM_FAULT_MAJOR; @@ -513,8 +534,7 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason) set_current_state(blocking_state); if (READ_ONCE(uwq.waken) || READ_ONCE(ctx->released) || - (return_to_userland ? signal_pending(current) : - fatal_signal_pending(current))) + userfaultfd_signal_pending(vmf->flags)) break; schedule(); }