From patchwork Tue Jul 7 18:54:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 11649741 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B911713B6 for ; Tue, 7 Jul 2020 18:54:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8D5F520771 for ; Tue, 7 Jul 2020 18:54:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8D5F520771 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 95C408D000F; Tue, 7 Jul 2020 14:54:45 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 90D2F8D0003; Tue, 7 Jul 2020 14:54:45 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8222F8D000F; Tue, 7 Jul 2020 14:54:45 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0174.hostedemail.com [216.40.44.174]) by kanga.kvack.org (Postfix) with ESMTP id 692708D0003 for ; Tue, 7 Jul 2020 14:54:45 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id C4C9D8248047 for ; Tue, 7 Jul 2020 18:54:44 +0000 (UTC) X-FDA: 77012181288.10.fear02_241794226eb6 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id 9935016A0AB for ; Tue, 7 Jul 2020 18:54:44 +0000 (UTC) X-Spam-Summary: 1,0,0,12bec01348156e2a,d41d8cd98f00b204,yang.shi@linux.alibaba.com,,RULES_HIT:41:355:379:541:960:973:988:989:1260:1261:1345:1437:1534:1542:1711:1730:1747:1777:1792:2194:2199:2393:2553:2559:2562:2895:2901:3138:3139:3140:3141:3142:3354:3865:3867:3868:3870:3871:3872:3874:4250:4321:5007:6119:6120:6261:7901:7903:8603:10004:11026:11232:11473:11658:11914:12043:12048:12296:12297:12438:12555:12663:12895:12986:13053:13161:13221:13229:14096:14181:14721:21060:21080:21324:21627:21987:21990:30003:30054:30070:30090,0,RBL:115.124.30.45:@linux.alibaba.com:.lbl8.mailshell.net-64.201.201.201 62.20.2.100;04y8zsk6mmgmta4i4178zmji9owuhypqgu8rqg4swzznyz9t4ggc1sjd3ho3sjt.wzsruizfbbkhmfih77epwfzpwc5a3rzamr7encsii5rfaezbn5tefdkpa1x3b4s.y-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: fear02_241794226eb6 X-Filterd-Recvd-Size: 3614 Received: from out30-45.freemail.mail.aliyun.com (out30-45.freemail.mail.aliyun.com [115.124.30.45]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Tue, 7 Jul 2020 18:54:43 +0000 (UTC) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R411e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04426;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=9;SR=0;TI=SMTPD_---0U23IaJm_1594148072; Received: from localhost(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0U23IaJm_1594148072) by smtp.aliyun-inc.com(127.0.0.1); Wed, 08 Jul 2020 02:54:39 +0800 From: Yang Shi To: hannes@cmpxchg.org, catalin.marinas@arm.com, will.deacon@arm.com, akpm@linux-foundation.org Cc: yang.shi@linux.alibaba.com, xuyu@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [RFC PATCH] mm: avoid access flag update TLB flush for retried page fault Date: Wed, 8 Jul 2020 02:54:32 +0800 Message-Id: <1594148072-91273-1-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 X-Rspamd-Queue-Id: 9935016A0AB X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Recently we found regression when running will_it_scale/page_fault3 test on ARM64. Over 70% down for the multi processes cases and over 20% down for the multi threads cases. It turns out the regression is caused by commit 89b15332af7c0312a41e50846819ca6613b58b4c ("mm: drop mmap_sem before calling balance_dirty_pages() in write fault"). The test mmaps a memory size file then write to the mapping, this would make all memory dirty and trigger dirty pages throttle, that upstream commit would release mmap_sem then retry the page fault. The retried page fault would see correct PTEs installed by the first try then update access flags and flush TLBs. The regression is caused by the excessive TLB flush. It is fine on x86 since x86 doesn't need flush TLB for access flag update. The page fault would be retried due to: 1. Waiting for page readahead 2. Waiting for page swapped in 3. Waiting for dirty pages throttling The first two cases don't have PTEs set up at all, so the retried page fault would install the PTEs, so they don't reach there. But the #3 case usually has PTEs installed, the retried page fault would reach the access flag update. But it seems not necessary to update access flags for #3 since retried page fault is not real "second access", so it sounds safe to skip access flag update for retried page fault. With this fix the test result get back to normal. Reported-by: Xu Yu Debugged-by: Xu Yu Tested-by: Xu Yu Signed-off-by: Yang Shi --- I'm not sure if this is safe for non-x86 machines, we did some tests on arm64, but there may be still corner cases not covered. mm/memory.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index 87ec87c..3d4e671 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4241,8 +4241,13 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) if (vmf->flags & FAULT_FLAG_WRITE) { if (!pte_write(entry)) return do_wp_page(vmf); - entry = pte_mkdirty(entry); } + + if ((vmf->flags & FAULT_FLAG_WRITE) && !(vmf->flags & FAULT_FLAG_TRIED)) + entry = pte_mkdirty(entry); + else if (vmf->flags & FAULT_FLAG_TRIED) + goto unlock; + entry = pte_mkyoung(entry); if (ptep_set_access_flags(vmf->vma, vmf->address, vmf->pte, entry, vmf->flags & FAULT_FLAG_WRITE)) {