From patchwork Tue Sep 22 13:56:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11792605 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A64AA112C for ; Tue, 22 Sep 2020 13:57:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 85AAB20719 for ; Tue, 22 Sep 2020 13:57:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 85AAB20719 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 64C086B00B0; Tue, 22 Sep 2020 09:57:08 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4A98A6B00B5; Tue, 22 Sep 2020 09:57:08 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B91E900063; Tue, 22 Sep 2020 09:57:08 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0003.hostedemail.com [216.40.44.3]) by kanga.kvack.org (Postfix) with ESMTP id 0152F6B00B2 for ; Tue, 22 Sep 2020 09:57:07 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id B12A71EF1 for ; Tue, 22 Sep 2020 13:57:07 +0000 (UTC) X-FDA: 77290848894.19.brush22_2f10a342714e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id 85A351AD1B9 for ; Tue, 22 Sep 2020 13:57:07 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30012:30046:30054:30064:30070:30079,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201;04yr4pughkqdk8cpao5oawftx6ozqypdwtax7s6513bn5zct4giabxmcwiy7qu9.7g9nxpg1asmna3u3narj9kpr4qa88hk3oqa5asuzueand8e6h8ja8atirrjm1qb.e-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:28,LUA_SUMMARY:none X-HE-Tag: brush22_2f10a342714e X-Filterd-Recvd-Size: 2137 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 13:57:06 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 646F5AB5C; Tue, 22 Sep 2020 13:57:42 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v7 01/14] mm,hwpoison: cleanup unused PageHuge() check Date: Tue, 22 Sep 2020 15:56:37 +0200 Message-Id: <20200922135650.1634-2-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200922135650.1634-1-osalvador@suse.de> References: <20200922135650.1634-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Drop the PageHuge check, which is dead code since memory_failure() forks into memory_failure_hugetlb() for hugetlb pages. memory_failure() and memory_failure_hugetlb() shares some functions like hwpoison_user_mappings() and identify_page_state(), so they should properly handle 4kB page, thp, and hugetlb. Signed-off-by: Naoya Horiguchi Signed-off-by: Oscar Salvador Reviewed-by: Mike Kravetz --- mm/memory-failure.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 166fad745918..b0eabc312381 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1381,10 +1381,7 @@ int memory_failure(unsigned long pfn, int flags) * page_remove_rmap() in try_to_unmap_one(). So to determine page status * correctly, we save a copy of the page flags at this time. */ - if (PageHuge(p)) - page_flags = hpage->flags; - else - page_flags = p->flags; + page_flags = p->flags; /* * unpoison always clear PG_hwpoison inside page lock From patchwork Tue Sep 22 13:56:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11792603 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6097F112C for ; Tue, 22 Sep 2020 13:57:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1A6E62395B for ; Tue, 22 Sep 2020 13:57:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1A6E62395B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3996D6B00B3; Tue, 22 Sep 2020 09:57:08 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 34A476B00B0; Tue, 22 Sep 2020 09:57:08 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 19CA76B00B5; Tue, 22 Sep 2020 09:57:08 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0139.hostedemail.com [216.40.44.139]) by kanga.kvack.org (Postfix) with ESMTP id 008066B00B0 for ; Tue, 22 Sep 2020 09:57:07 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 95A4E3629 for ; Tue, 22 Sep 2020 13:57:07 +0000 (UTC) X-FDA: 77290848894.08.bomb65_460023e2714e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin08.hostedemail.com (Postfix) with ESMTP id 72E0D1819E773 for ; Tue, 22 Sep 2020 13:57:07 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30051:30054:30064:30070,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201;04yfsnrxhoy6rifysntabzscjfjh8yc1pdub3i5o6xkgyafhhgcf7ay41qnigjy.nzjcrhsexp1cwpipwfdudxqg36a1hesim171io578p3s76m8s9b36gwdhsp1w6w.c-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: bomb65_460023e2714e X-Filterd-Recvd-Size: 2317 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 13:57:06 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 6FC94ACE5; Tue, 22 Sep 2020 13:57:42 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v7 02/14] mm, hwpoison: remove recalculating hpage Date: Tue, 22 Sep 2020 15:56:38 +0200 Message-Id: <20200922135650.1634-3-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200922135650.1634-1-osalvador@suse.de> References: <20200922135650.1634-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi hpage is never used after try_to_split_thp_page() in memory_failure(), so we don't have to update hpage. So let's not recalculate/use hpage. Signed-off-by: Naoya Horiguchi Signed-off-by: Oscar Salvador Suggested-by: "Aneesh Kumar K.V" Reviewed-by: Mike Kravetz --- mm/memory-failure.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index b0eabc312381..0e6edffa91a9 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1341,7 +1341,6 @@ int memory_failure(unsigned long pfn, int flags) } unlock_page(p); VM_BUG_ON_PAGE(!page_count(p), p); - hpage = compound_head(p); } /* @@ -1413,11 +1412,8 @@ int memory_failure(unsigned long pfn, int flags) /* * Now take care of user space mappings. * Abort on fail: __delete_from_page_cache() assumes unmapped page. - * - * When the raw error page is thp tail page, hpage points to the raw - * page after thp split. */ - if (!hwpoison_user_mappings(p, pfn, flags, &hpage)) { + if (!hwpoison_user_mappings(p, pfn, flags, &p)) { action_result(pfn, MF_MSG_UNMAP_FAILED, MF_IGNORED); res = -EBUSY; goto out; From patchwork Tue Sep 22 13:56:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11792609 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F3D15112E for ; Tue, 22 Sep 2020 13:57:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AA2B620719 for ; Tue, 22 Sep 2020 13:57:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AA2B620719 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EBF1D6B00B4; Tue, 22 Sep 2020 09:57:08 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9434A900063; Tue, 22 Sep 2020 09:57:08 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 791096B00B6; Tue, 22 Sep 2020 09:57:08 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0049.hostedemail.com [216.40.44.49]) by kanga.kvack.org (Postfix) with ESMTP id 585106B00B7 for ; Tue, 22 Sep 2020 09:57:08 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 05374180AD806 for ; Tue, 22 Sep 2020 13:57:08 +0000 (UTC) X-FDA: 77290848936.01.mom37_2709c572714e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id D07071004A75E for ; Tue, 22 Sep 2020 13:57:07 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30054:30070,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-64.100.201.201 62.2.6.2;04yrbn4gshmiibrhrpr1wt45didwoycjegyb73bce1m89uyxaqmkwyog5gs6rjo.siykxgt343d1xbor5hegareok1faq9tueberpw5ixuritegs3mz4aamgjszp9c6.h-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: mom37_2709c572714e X-Filterd-Recvd-Size: 2803 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 13:57:07 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 0299EAE79; Tue, 22 Sep 2020 13:57:43 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v7 03/14] mm,hwpoison-inject: don't pin for hwpoison_filter Date: Tue, 22 Sep 2020 15:56:39 +0200 Message-Id: <20200922135650.1634-4-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200922135650.1634-1-osalvador@suse.de> References: <20200922135650.1634-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Another memory error injection interface debugfs:hwpoison/corrupt-pfn also takes bogus refcount for hwpoison_filter(). It's justified because this does a coarse filter, expecting that memory_failure() redoes the check for sure. Signed-off-by: Naoya Horiguchi Signed-off-by: Oscar Salvador --- mm/hwpoison-inject.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/mm/hwpoison-inject.c b/mm/hwpoison-inject.c index e488876b168a..1ae1ebc2b9b1 100644 --- a/mm/hwpoison-inject.c +++ b/mm/hwpoison-inject.c @@ -26,11 +26,6 @@ static int hwpoison_inject(void *data, u64 val) p = pfn_to_page(pfn); hpage = compound_head(p); - /* - * This implies unable to support free buddy pages. - */ - if (!get_hwpoison_page(p)) - return 0; if (!hwpoison_filter_enable) goto inject; @@ -40,23 +35,20 @@ static int hwpoison_inject(void *data, u64 val) * This implies unable to support non-LRU pages. */ if (!PageLRU(hpage) && !PageHuge(p)) - goto put_out; + return 0; /* - * do a racy check with elevated page count, to make sure PG_hwpoison - * will only be set for the targeted owner (or on a free page). + * do a racy check to make sure PG_hwpoison will only be set for + * the targeted owner (or on a free page). * memory_failure() will redo the check reliably inside page lock. */ err = hwpoison_filter(hpage); if (err) - goto put_out; + return 0; inject: pr_info("Injecting memory failure at pfn %#lx\n", pfn); - return memory_failure(pfn, MF_COUNT_INCREASED); -put_out: - put_hwpoison_page(p); - return 0; + return memory_failure(pfn, 0); } static int hwpoison_unpoison(void *data, u64 val) From patchwork Tue Sep 22 13:56:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11792611 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5269F112C for ; Tue, 22 Sep 2020 13:57:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0D63820719 for ; Tue, 22 Sep 2020 13:57:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0D63820719 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 830536B00B6; Tue, 22 Sep 2020 09:57:09 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7487E6B00BA; Tue, 22 Sep 2020 09:57:09 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 549C26B00B9; Tue, 22 Sep 2020 09:57:09 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0186.hostedemail.com [216.40.44.186]) by kanga.kvack.org (Postfix) with ESMTP id 36F166B00B5 for ; Tue, 22 Sep 2020 09:57:09 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id E36B0181AEF09 for ; Tue, 22 Sep 2020 13:57:08 +0000 (UTC) X-FDA: 77290848936.21.flame73_3b08d782714e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 9A727180442C3 for ; Tue, 22 Sep 2020 13:57:08 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30012:30051:30054,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-64.100.201.201 62.2.6.2;04y8e8hchemqw65ees47jzb6tnx5zyc3ptxzqwqpir67xxmqrnbkuf1tufufbgr.r8wif78fzzfzrmoh4fek8nbetwxd51s6m79m7mf3zwwq7437tfadyp79fkmewx4.k-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: flame73_3b08d782714e X-Filterd-Recvd-Size: 2537 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 13:57:08 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 8F4B4AF26; Tue, 22 Sep 2020 13:57:43 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v7 04/14] mm,hwpoison: unexport get_hwpoison_page and make it static Date: Tue, 22 Sep 2020 15:56:40 +0200 Message-Id: <20200922135650.1634-5-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200922135650.1634-1-osalvador@suse.de> References: <20200922135650.1634-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since get_hwpoison_page is only used in memory-failure code now, let us un-export it and make it private to that code. Signed-off-by: Oscar Salvador Acked-by: Naoya Horiguchi --- include/linux/mm.h | 1 - mm/memory-failure.c | 3 +-- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 185fcef0deb6..20ac1653fb81 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3015,7 +3015,6 @@ extern int memory_failure(unsigned long pfn, int flags); extern void memory_failure_queue(unsigned long pfn, int flags); extern void memory_failure_queue_kick(int cpu); extern int unpoison_memory(unsigned long pfn); -extern int get_hwpoison_page(struct page *page); #define put_hwpoison_page(page) put_page(page) extern int sysctl_memory_failure_early_kill; extern int sysctl_memory_failure_recovery; diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 0e6edffa91a9..f8cff387451e 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -924,7 +924,7 @@ static int page_action(struct page_state *ps, struct page *p, * Return: return 0 if failed to grab the refcount, otherwise true (some * non-zero value.) */ -int get_hwpoison_page(struct page *page) +static int get_hwpoison_page(struct page *page) { struct page *head = compound_head(page); @@ -953,7 +953,6 @@ int get_hwpoison_page(struct page *page) return 0; } -EXPORT_SYMBOL_GPL(get_hwpoison_page); /* * Do all that is necessary to remove user space mappings. Unmap From patchwork Tue Sep 22 13:56:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11792613 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B184C112C for ; Tue, 22 Sep 2020 13:57:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8353420719 for ; Tue, 22 Sep 2020 13:57:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8353420719 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A720E6B00B5; Tue, 22 Sep 2020 09:57:09 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A217C6B00B9; Tue, 22 Sep 2020 09:57:09 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 82D4F6B00B5; Tue, 22 Sep 2020 09:57:09 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0227.hostedemail.com [216.40.44.227]) by kanga.kvack.org (Postfix) with ESMTP id 49F376B00B6 for ; Tue, 22 Sep 2020 09:57:09 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id EDB09824999B for ; Tue, 22 Sep 2020 13:57:08 +0000 (UTC) X-FDA: 77290848936.01.pigs38_341193d2714e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id C928B1004A75E for ; Tue, 22 Sep 2020 13:57:08 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30003:30054:30070,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201;04yfkpa3puzqbtcujp36kyisdo3a6opbpq4xpx14pk1ab1xd6akrtborehrfynm.d4wiw6e98b71ikqg7g6qkq7zrifaweosro1ehbnfx6pyombk91tq7gfsspktj5k.6-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: pigs38_341193d2714e X-Filterd-Recvd-Size: 3013 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 13:57:08 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 252DCAFBC; Tue, 22 Sep 2020 13:57:44 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v7 05/14] mm,hwpoison: refactor madvise_inject_error Date: Tue, 22 Sep 2020 15:56:41 +0200 Message-Id: <20200922135650.1634-6-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200922135650.1634-1-osalvador@suse.de> References: <20200922135650.1634-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make a proper if-else condition for {hard,soft}-offline. [akpm: refactor comment] Signed-off-by: Oscar Salvador Acked-by: Naoya Horiguchi --- mm/madvise.c | 30 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 16 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 9b065d412e5f..f3a86d547ceb 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -872,7 +872,6 @@ static long madvise_remove(struct vm_area_struct *vma, static int madvise_inject_error(int behavior, unsigned long start, unsigned long end) { - struct page *page; struct zone *zone; unsigned long size; @@ -881,6 +880,7 @@ static int madvise_inject_error(int behavior, for (; start < end; start += size) { + struct page *page; unsigned long pfn; int ret; @@ -903,25 +903,23 @@ static int madvise_inject_error(int behavior, if (behavior == MADV_SOFT_OFFLINE) { pr_info("Soft offlining pfn %#lx at process virtual address %#lx\n", - pfn, start); + pfn, start); ret = soft_offline_page(pfn, MF_COUNT_INCREASED); - if (ret) - return ret; - continue; + } else { + pr_info("Injecting memory failure for pfn %#lx at process virtual address %#lx\n", + pfn, start); + /* + * Drop the page reference taken by + * get_user_pages_fast(). In the absence of + * MF_COUNT_INCREASED the memory_failure() routine is + * responsible for pinning the page to prevent it + * from being released back to the page allocator. + */ + put_page(page); + ret = memory_failure(pfn, 0); } - pr_info("Injecting memory failure for pfn %#lx at process virtual address %#lx\n", - pfn, start); - - /* - * Drop the page reference taken by get_user_pages_fast(). In - * the absence of MF_COUNT_INCREASED the memory_failure() - * routine is responsible for pinning the page to prevent it - * from being released back to the page allocator. - */ - put_page(page); - ret = memory_failure(pfn, 0); if (ret) return ret; } From patchwork Tue Sep 22 13:56:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11792615 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 236A0112E for ; Tue, 22 Sep 2020 13:57:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E940620719 for ; Tue, 22 Sep 2020 13:57:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E940620719 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 43B1E6B00B9; Tue, 22 Sep 2020 09:57:10 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3C6286B00BA; Tue, 22 Sep 2020 09:57:10 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 28A6F900063; Tue, 22 Sep 2020 09:57:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0187.hostedemail.com [216.40.44.187]) by kanga.kvack.org (Postfix) with ESMTP id 090426B00B9 for ; Tue, 22 Sep 2020 09:57:10 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id BFE16180AD806 for ; Tue, 22 Sep 2020 13:57:09 +0000 (UTC) X-FDA: 77290848978.25.crush30_1b13e132714e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id 90D8B1804E3A1 for ; Tue, 22 Sep 2020 13:57:09 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30054,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201;04yg5kwqs9ms534ywjt53kji1m5bboczs6ucjrtnq3g8or79pi9pu8iym4h88db.e3pz1t68d8nzjxiu3u5rh3j5eep88bi4jnyo34s4yj7aiiejgt6y4obgya4xain.1-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:22,LUA_SUMMARY:none X-HE-Tag: crush30_1b13e132714e X-Filterd-Recvd-Size: 6127 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 13:57:08 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id B4068AFC9; Tue, 22 Sep 2020 13:57:44 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v7 06/14] mm,hwpoison: kill put_hwpoison_page Date: Tue, 22 Sep 2020 15:56:42 +0200 Message-Id: <20200922135650.1634-7-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200922135650.1634-1-osalvador@suse.de> References: <20200922135650.1634-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: After commit 4e41a30c6d50 ("mm: hwpoison: adjust for new thp refcounting"), put_hwpoison_page got reduced to a put_page. Let us just use put_page instead. Signed-off-by: Oscar Salvador Acked-by: Naoya Horiguchi --- include/linux/mm.h | 1 - mm/memory-failure.c | 30 +++++++++++++++--------------- 2 files changed, 15 insertions(+), 16 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 20ac1653fb81..61a80384afd3 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3015,7 +3015,6 @@ extern int memory_failure(unsigned long pfn, int flags); extern void memory_failure_queue(unsigned long pfn, int flags); extern void memory_failure_queue_kick(int cpu); extern int unpoison_memory(unsigned long pfn); -#define put_hwpoison_page(page) put_page(page) extern int sysctl_memory_failure_early_kill; extern int sysctl_memory_failure_recovery; extern void shake_page(struct page *p, int access); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index f8cff387451e..ed54c27ad668 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1143,7 +1143,7 @@ static int memory_failure_hugetlb(unsigned long pfn, int flags) pr_err("Memory failure: %#lx: just unpoisoned\n", pfn); num_poisoned_pages_dec(); unlock_page(head); - put_hwpoison_page(head); + put_page(head); return 0; } @@ -1335,7 +1335,7 @@ int memory_failure(unsigned long pfn, int flags) pfn); if (TestClearPageHWPoison(p)) num_poisoned_pages_dec(); - put_hwpoison_page(p); + put_page(p); return -EBUSY; } unlock_page(p); @@ -1388,14 +1388,14 @@ int memory_failure(unsigned long pfn, int flags) pr_err("Memory failure: %#lx: just unpoisoned\n", pfn); num_poisoned_pages_dec(); unlock_page(p); - put_hwpoison_page(p); + put_page(p); return 0; } if (hwpoison_filter(p)) { if (TestClearPageHWPoison(p)) num_poisoned_pages_dec(); unlock_page(p); - put_hwpoison_page(p); + put_page(p); return 0; } @@ -1629,9 +1629,9 @@ int unpoison_memory(unsigned long pfn) } unlock_page(page); - put_hwpoison_page(page); + put_page(page); if (freeit && !(pfn == my_zero_pfn(0) && page_count(p) == 1)) - put_hwpoison_page(page); + put_page(page); return 0; } @@ -1692,7 +1692,7 @@ static int get_any_page(struct page *page, unsigned long pfn, int flags) /* * Try to free it. */ - put_hwpoison_page(page); + put_page(page); shake_page(page, 1); /* @@ -1701,7 +1701,7 @@ static int get_any_page(struct page *page, unsigned long pfn, int flags) ret = __get_any_page(page, pfn, 0); if (ret == 1 && !PageLRU(page)) { /* Drop page reference which is from __get_any_page() */ - put_hwpoison_page(page); + put_page(page); pr_info("soft_offline: %#lx: unknown non LRU page type %lx (%pGp)\n", pfn, page->flags, &page->flags); return -EIO; @@ -1724,7 +1724,7 @@ static int soft_offline_huge_page(struct page *page, int flags) lock_page(hpage); if (PageHWPoison(hpage)) { unlock_page(hpage); - put_hwpoison_page(hpage); + put_page(hpage); pr_info("soft offline: %#lx hugepage already poisoned\n", pfn); return -EBUSY; } @@ -1735,7 +1735,7 @@ static int soft_offline_huge_page(struct page *page, int flags) * get_any_page() and isolate_huge_page() takes a refcount each, * so need to drop one here. */ - put_hwpoison_page(hpage); + put_page(hpage); if (!ret) { pr_info("soft offline: %#lx hugepage failed to isolate\n", pfn); return -EBUSY; @@ -1784,7 +1784,7 @@ static int __soft_offline_page(struct page *page, int flags) wait_on_page_writeback(page); if (PageHWPoison(page)) { unlock_page(page); - put_hwpoison_page(page); + put_page(page); pr_info("soft offline: %#lx page already poisoned\n", pfn); return -EBUSY; } @@ -1799,7 +1799,7 @@ static int __soft_offline_page(struct page *page, int flags) * would need to fix isolation locking first. */ if (ret == 1) { - put_hwpoison_page(page); + put_page(page); pr_info("soft_offline: %#lx: invalidated\n", pfn); SetPageHWPoison(page); num_poisoned_pages_inc(); @@ -1819,7 +1819,7 @@ static int __soft_offline_page(struct page *page, int flags) * Drop page reference which is came from get_any_page() * successful isolate_lru_page() already took another one. */ - put_hwpoison_page(page); + put_page(page); if (!ret) { LIST_HEAD(pagelist); /* @@ -1863,7 +1863,7 @@ static int soft_offline_in_use_page(struct page *page, int flags) pr_info("soft offline: %#lx: non anonymous thp\n", page_to_pfn(page)); else pr_info("soft offline: %#lx: thp split failed\n", page_to_pfn(page)); - put_hwpoison_page(page); + put_page(page); return -EBUSY; } unlock_page(page); @@ -1936,7 +1936,7 @@ int soft_offline_page(unsigned long pfn, int flags) if (PageHWPoison(page)) { pr_info("soft offline: %#lx page already poisoned\n", pfn); if (flags & MF_COUNT_INCREASED) - put_hwpoison_page(page); + put_page(page); return -EBUSY; } From patchwork Tue Sep 22 13:56:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11792617 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6BE1C112E for ; Tue, 22 Sep 2020 13:57:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3F87F20719 for ; Tue, 22 Sep 2020 13:57:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3F87F20719 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C4FC86B00BA; Tue, 22 Sep 2020 09:57:10 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BB097900063; Tue, 22 Sep 2020 09:57:10 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A29E26B00BC; Tue, 22 Sep 2020 09:57:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0151.hostedemail.com [216.40.44.151]) by kanga.kvack.org (Postfix) with ESMTP id 74FC76B00BB for ; Tue, 22 Sep 2020 09:57:10 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 36F4B824999B for ; Tue, 22 Sep 2020 13:57:10 +0000 (UTC) X-FDA: 77290849020.23.van97_4c021702714e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id 2426537606 for ; Tue, 22 Sep 2020 13:57:10 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30012:30054,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-64.100.201.201 62.2.6.2;04yg8r1iuiwnifkej6osrthjzeuigopjp7aaed7gnpnp3ghmwbn4a1ze9gbgazg.dkzw8py6htz4yw5r8trt1qcz3xs65mrp988kxsac37ind4ghk9buprksi75jci1.s-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: van97_4c021702714e X-Filterd-Recvd-Size: 3530 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 13:57:09 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 48C58AFCF; Tue, 22 Sep 2020 13:57:45 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v7 07/14] mm,hwpoison: unify THP handling for hard and soft offline Date: Tue, 22 Sep 2020 15:56:43 +0200 Message-Id: <20200922135650.1634-8-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200922135650.1634-1-osalvador@suse.de> References: <20200922135650.1634-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Place the THP's page handling in a helper and use it from both hard and soft-offline machinery, so we get rid of some duplicated code. Signed-off-by: Oscar Salvador Acked-by: Naoya Horiguchi --- mm/memory-failure.c | 48 +++++++++++++++++++++------------------------ 1 file changed, 22 insertions(+), 26 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index ed54c27ad668..9d80beb841a3 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1102,6 +1102,25 @@ static int identify_page_state(unsigned long pfn, struct page *p, return page_action(ps, p, pfn); } +static int try_to_split_thp_page(struct page *page, const char *msg) +{ + lock_page(page); + if (!PageAnon(page) || unlikely(split_huge_page(page))) { + unsigned long pfn = page_to_pfn(page); + + unlock_page(page); + if (!PageAnon(page)) + pr_info("%s: %#lx: non anonymous thp\n", msg, pfn); + else + pr_info("%s: %#lx: thp split failed\n", msg, pfn); + put_page(page); + return -EBUSY; + } + unlock_page(page); + + return 0; +} + static int memory_failure_hugetlb(unsigned long pfn, int flags) { struct page *p = pfn_to_page(pfn); @@ -1324,21 +1343,8 @@ int memory_failure(unsigned long pfn, int flags) } if (PageTransHuge(hpage)) { - lock_page(p); - if (!PageAnon(p) || unlikely(split_huge_page(p))) { - unlock_page(p); - if (!PageAnon(p)) - pr_err("Memory failure: %#lx: non anonymous thp\n", - pfn); - else - pr_err("Memory failure: %#lx: thp split failed\n", - pfn); - if (TestClearPageHWPoison(p)) - num_poisoned_pages_dec(); - put_page(p); + if (try_to_split_thp_page(p, "Memory Failure") < 0) return -EBUSY; - } - unlock_page(p); VM_BUG_ON_PAGE(!page_count(p), p); } @@ -1855,19 +1861,9 @@ static int soft_offline_in_use_page(struct page *page, int flags) int mt; struct page *hpage = compound_head(page); - if (!PageHuge(page) && PageTransHuge(hpage)) { - lock_page(page); - if (!PageAnon(page) || unlikely(split_huge_page(page))) { - unlock_page(page); - if (!PageAnon(page)) - pr_info("soft offline: %#lx: non anonymous thp\n", page_to_pfn(page)); - else - pr_info("soft offline: %#lx: thp split failed\n", page_to_pfn(page)); - put_page(page); + if (!PageHuge(page) && PageTransHuge(hpage)) + if (try_to_split_thp_page(page, "soft offline") < 0) return -EBUSY; - } - unlock_page(page); - } /* * Setting MIGRATE_ISOLATE here ensures that the page will be linked From patchwork Tue Sep 22 13:56:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11792619 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E69A3112C for ; Tue, 22 Sep 2020 13:57:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B152620936 for ; Tue, 22 Sep 2020 13:57:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B152620936 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3EB9D6B00BB; Tue, 22 Sep 2020 09:57:11 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 34F5E6B00BC; Tue, 22 Sep 2020 09:57:11 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1518D6B00BD; Tue, 22 Sep 2020 09:57:11 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0098.hostedemail.com [216.40.44.98]) by kanga.kvack.org (Postfix) with ESMTP id E63866B00BB for ; Tue, 22 Sep 2020 09:57:10 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A52DB1EF1 for ; Tue, 22 Sep 2020 13:57:10 +0000 (UTC) X-FDA: 77290849020.01.women78_25133d92714e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id 8C1271004AFBE for ; Tue, 22 Sep 2020 13:57:10 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30054:30070,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201;04y8m61oyn6f536swgnsdkqof7x4zoc7gfhjbaip4idunog1yyyfma5xycktu33.j793agadceudt3dr8y6etrwg5r9rjz4crrk8744nmcfe9exzntohhpx8qwnh9bt.s-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: women78_25133d92714e X-Filterd-Recvd-Size: 5535 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 13:57:09 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id CF90FAFD2; Tue, 22 Sep 2020 13:57:45 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v7 08/14] mm,hwpoison: rework soft offline for free pages Date: Tue, 22 Sep 2020 15:56:44 +0200 Message-Id: <20200922135650.1634-9-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200922135650.1634-1-osalvador@suse.de> References: <20200922135650.1634-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When trying to soft-offline a free page, we need to first take it off the buddy allocator. Once we know is out of reach, we can safely flag it as poisoned. take_page_off_buddy will be used to take a page meant to be poisoned off the buddy allocator. take_page_off_buddy calls break_down_buddy_pages, which splits a higher-order page in case our page belongs to one. Once the page is under our control, we call page_handle_poison to set it as poisoned and grab a refcount on it. Signed-off-by: Oscar Salvador Acked-by: Naoya Horiguchi --- include/linux/page-flags.h | 1 + mm/memory-failure.c | 18 ++++++---- mm/page_alloc.c | 68 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 81 insertions(+), 6 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 7182103583d2..d0fdb59794d8 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -429,6 +429,7 @@ PAGEFLAG(HWPoison, hwpoison, PF_ANY) TESTSCFLAG(HWPoison, hwpoison, PF_ANY) #define __PG_HWPOISON (1UL << PG_hwpoison) extern bool set_hwpoison_free_buddy_page(struct page *page); +extern bool take_page_off_buddy(struct page *page); #else PAGEFLAG_FALSE(HWPoison) static inline bool set_hwpoison_free_buddy_page(struct page *page) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 9d80beb841a3..de274356f8c7 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -65,6 +65,13 @@ int sysctl_memory_failure_recovery __read_mostly = 1; atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); +static void page_handle_poison(struct page *page) +{ + SetPageHWPoison(page); + page_ref_inc(page); + num_poisoned_pages_inc(); +} + #if defined(CONFIG_HWPOISON_INJECT) || defined(CONFIG_HWPOISON_INJECT_MODULE) u32 hwpoison_filter_enable = 0; @@ -1884,14 +1891,13 @@ static int soft_offline_in_use_page(struct page *page, int flags) static int soft_offline_free_page(struct page *page) { - int rc = dissolve_free_huge_page(page); + int rc = -EBUSY; - if (!rc) { - if (set_hwpoison_free_buddy_page(page)) - num_poisoned_pages_inc(); - else - rc = -EBUSY; + if (!dissolve_free_huge_page(page) && take_page_off_buddy(page)) { + page_handle_poison(page); + rc = 0; } + return rc; } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3c30be2e8e76..e02da891d8a9 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -8758,6 +8758,74 @@ bool is_free_buddy_page(struct page *page) } #ifdef CONFIG_MEMORY_FAILURE +/* + * Break down a higher-order page in sub-pages, and keep our target out of + * buddy allocator. + */ +static void break_down_buddy_pages(struct zone *zone, struct page *page, + struct page *target, int low, int high, + int migratetype) +{ + unsigned long size = 1 << high; + struct page *current_buddy, *next_page; + + while (high > low) { + high--; + size >>= 1; + + if (target >= &page[size]) { + next_page = page + size; + current_buddy = page; + } else { + next_page = page; + current_buddy = page + size; + } + + if (set_page_guard(zone, current_buddy, high, migratetype)) + continue; + + if (current_buddy != target) { + add_to_free_list(current_buddy, zone, high, migratetype); + set_page_order(current_buddy, high); + page = next_page; + } + } +} + +/* + * Take a page that will be marked as poisoned off the buddy allocator. + */ +bool take_page_off_buddy(struct page *page) +{ + struct zone *zone = page_zone(page); + unsigned long pfn = page_to_pfn(page); + unsigned long flags; + unsigned int order; + bool ret = false; + + spin_lock_irqsave(&zone->lock, flags); + for (order = 0; order < MAX_ORDER; order++) { + struct page *page_head = page - (pfn & ((1 << order) - 1)); + int buddy_order = page_order(page_head); + + if (PageBuddy(page_head) && buddy_order >= order) { + unsigned long pfn_head = page_to_pfn(page_head); + int migratetype = get_pfnblock_migratetype(page_head, + pfn_head); + + del_page_from_free_list(page_head, zone, buddy_order); + break_down_buddy_pages(zone, page_head, page, 0, + buddy_order, migratetype); + ret = true; + break; + } + if (page_count(page_head) > 0) + break; + } + spin_unlock_irqrestore(&zone->lock, flags); + return ret; +} + /* * Set PG_hwpoison flag if a given page is confirmed to be a free page. This * test is performed under the zone lock to prevent a race against page From patchwork Tue Sep 22 13:56:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11792621 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CB6F1112E for ; Tue, 22 Sep 2020 13:57:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 900CB20719 for ; Tue, 22 Sep 2020 13:57:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 900CB20719 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 067B66B00BD; Tue, 22 Sep 2020 09:57:12 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F0FCB6B00BE; Tue, 22 Sep 2020 09:57:11 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C72636B00BF; Tue, 22 Sep 2020 09:57:11 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0111.hostedemail.com [216.40.44.111]) by kanga.kvack.org (Postfix) with ESMTP id A91EB6B00BD for ; Tue, 22 Sep 2020 09:57:11 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 6B048181AEF09 for ; Tue, 22 Sep 2020 13:57:11 +0000 (UTC) X-FDA: 77290849062.16.join37_44057052714e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 3CF851010A158 for ; Tue, 22 Sep 2020 13:57:11 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30012:30045:30051:30054:30070:30075:30083:30091,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-64.100.201.201 62.2.6.2;04yfskkyhtakoqosm7fhe7otk8byrop59k34j3nwoe64wi44a5yzzwstxho5cit.fo3xgzn1mpnx45sdndrjqaakodsgpa3bz41fx8zktmyfe51rsrcm518az7u5ehs.6-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: join37_44057052714e X-Filterd-Recvd-Size: 10473 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 13:57:10 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 77CF9AE79; Tue, 22 Sep 2020 13:57:46 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v7 09/14] mm,hwpoison: rework soft offline for in-use pages Date: Tue, 22 Sep 2020 15:56:45 +0200 Message-Id: <20200922135650.1634-10-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200922135650.1634-1-osalvador@suse.de> References: <20200922135650.1634-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch changes the way we set and handle in-use poisoned pages. Until now, poisoned pages were released to the buddy allocator, trusting that the checks that take place at allocation time would act as a safe net and would skip that page. This has proved to be wrong, as we got some pfn walkers out there, like compaction, that all they care is the page to be in a buddy freelist. Although this might not be the only user, having poisoned pages in the buddy allocator seems a bad idea as we should only have free pages that are ready and meant to be used as such. Before explaining the taken approach, let us break down the kind of pages we can soft offline. - Anonymous THP (after the split, they end up being 4K pages) - Hugetlb - Order-0 pages (that can be either migrated or invalited) * Normal pages (order-0 and anon-THP) - If they are clean and unmapped page cache pages, we invalidate then by means of invalidate_inode_page(). - If they are mapped/dirty, we do the isolate-and-migrate dance. Either way, do not call put_page directly from those paths. Instead, we keep the page and send it to page_handle_poison to perform the right handling. page_handle_poison sets the HWPoison flag and does the last put_page. Down the chain, we placed a check for HWPoison page in free_pages_prepare, that just skips any poisoned page, so those pages do not end up in any pcplist/freelist. After that, we set the refcount on the page to 1 and we increment the poisoned pages counter. If we see that the check in free_pages_prepare creates trouble, we can always do what we do for free pages: - wait until the page hits buddy's freelists - take it off, and flag it The downside of the above approach is that we could race with an allocation, so by the time we want to take the page off the buddy, the page has been already allocated so we cannot soft offline it. But the user could always retry it. * Hugetlb pages - We isolate-and-migrate them After the migration has been successful, we call dissolve_free_huge_page, and we set HWPoison on the page if we succeed. Hugetlb has a slightly different handling though. While for non-hugetlb pages we cared about closing the race with an allocation, doing so for hugetlb pages requires quite some additional and intrusive code (we would need to hook in free_huge_page and some other places). So I decided to not make the code overly complicated and just fail normally if the page we allocated in the meantime. We can always build on top of this. As a bonus, because of the way we handle now in-use pages, we no longer need the put-as-isolation-migratetype dance, that was guarding for poisoned pages to end up in pcplists. Signed-off-by: Oscar Salvador Acked-by: Naoya Horiguchi --- include/linux/page-flags.h | 5 ----- mm/memory-failure.c | 43 +++++++++++++------------------------- mm/migrate.c | 11 +++------- mm/page_alloc.c | 39 ++++++++++------------------------ 4 files changed, 28 insertions(+), 70 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index d0fdb59794d8..fbbb841a9346 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -428,14 +428,9 @@ PAGEFLAG_FALSE(Uncached) PAGEFLAG(HWPoison, hwpoison, PF_ANY) TESTSCFLAG(HWPoison, hwpoison, PF_ANY) #define __PG_HWPOISON (1UL << PG_hwpoison) -extern bool set_hwpoison_free_buddy_page(struct page *page); extern bool take_page_off_buddy(struct page *page); #else PAGEFLAG_FALSE(HWPoison) -static inline bool set_hwpoison_free_buddy_page(struct page *page) -{ - return 0; -} #define __PG_HWPOISON 0 #endif diff --git a/mm/memory-failure.c b/mm/memory-failure.c index de274356f8c7..cc8757282860 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -65,9 +65,11 @@ int sysctl_memory_failure_recovery __read_mostly = 1; atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); -static void page_handle_poison(struct page *page) +static void page_handle_poison(struct page *page, bool release) { SetPageHWPoison(page); + if (release) + put_page(page); page_ref_inc(page); num_poisoned_pages_inc(); } @@ -1765,19 +1767,13 @@ static int soft_offline_huge_page(struct page *page, int flags) ret = -EIO; } else { /* - * We set PG_hwpoison only when the migration source hugepage - * was successfully dissolved, because otherwise hwpoisoned - * hugepage remains on free hugepage list, then userspace will - * find it as SIGBUS by allocation failure. That's not expected - * in soft-offlining. + * We set PG_hwpoison only when we were able to take the page + * off the buddy. */ - ret = dissolve_free_huge_page(page); - if (!ret) { - if (set_hwpoison_free_buddy_page(page)) - num_poisoned_pages_inc(); - else - ret = -EBUSY; - } + if (!dissolve_free_huge_page(page) && take_page_off_buddy(page)) + page_handle_poison(page, false); + else + ret = -EBUSY; } return ret; } @@ -1812,10 +1808,8 @@ static int __soft_offline_page(struct page *page, int flags) * would need to fix isolation locking first. */ if (ret == 1) { - put_page(page); pr_info("soft_offline: %#lx: invalidated\n", pfn); - SetPageHWPoison(page); - num_poisoned_pages_inc(); + page_handle_poison(page, true); return 0; } @@ -1846,7 +1840,9 @@ static int __soft_offline_page(struct page *page, int flags) list_add(&page->lru, &pagelist); ret = migrate_pages(&pagelist, new_page, NULL, MPOL_MF_MOVE_ALL, MIGRATE_SYNC, MR_MEMORY_FAILURE); - if (ret) { + if (!ret) { + page_handle_poison(page, true); + } else { if (!list_empty(&pagelist)) putback_movable_pages(&pagelist); @@ -1865,27 +1861,16 @@ static int __soft_offline_page(struct page *page, int flags) static int soft_offline_in_use_page(struct page *page, int flags) { int ret; - int mt; struct page *hpage = compound_head(page); if (!PageHuge(page) && PageTransHuge(hpage)) if (try_to_split_thp_page(page, "soft offline") < 0) return -EBUSY; - /* - * Setting MIGRATE_ISOLATE here ensures that the page will be linked - * to free list immediately (not via pcplist) when released after - * successful page migration. Otherwise we can't guarantee that the - * page is really free after put_page() returns, so - * set_hwpoison_free_buddy_page() highly likely fails. - */ - mt = get_pageblock_migratetype(page); - set_pageblock_migratetype(page, MIGRATE_ISOLATE); if (PageHuge(page)) ret = soft_offline_huge_page(page, flags); else ret = __soft_offline_page(page, flags); - set_pageblock_migratetype(page, mt); return ret; } @@ -1894,7 +1879,7 @@ static int soft_offline_free_page(struct page *page) int rc = -EBUSY; if (!dissolve_free_huge_page(page) && take_page_off_buddy(page)) { - page_handle_poison(page); + page_handle_poison(page, false); rc = 0; } diff --git a/mm/migrate.c b/mm/migrate.c index 7eb82af9f49a..64750ef021c6 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1223,16 +1223,11 @@ static int unmap_and_move(new_page_t get_new_page, * we want to retry. */ if (rc == MIGRATEPAGE_SUCCESS) { - put_page(page); - if (reason == MR_MEMORY_FAILURE) { + if (reason != MR_MEMORY_FAILURE) /* - * Set PG_HWPoison on just freed page - * intentionally. Although it's rather weird, - * it's how HWPoison flag works at the moment. + * We release the page in page_handle_poison. */ - if (set_hwpoison_free_buddy_page(page)) - num_poisoned_pages_inc(); - } + put_page(page); } else { if (rc != -EAGAIN) { if (likely(!__PageMovable(page))) { diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e02da891d8a9..9a1d1e3cc099 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1173,6 +1173,17 @@ static __always_inline bool free_pages_prepare(struct page *page, trace_mm_page_free(page, order); + if (unlikely(PageHWPoison(page)) && !order) { + /* + * Do not let hwpoison pages hit pcplists/buddy + * Untie memcg state and reset page's owner + */ + if (memcg_kmem_enabled() && PageKmemcg(page)) + __memcg_kmem_uncharge_page(page, order); + reset_page_owner(page, order); + return false; + } + /* * Check tail pages before head page information is cleared to * avoid checking PageCompound for order-0 pages. @@ -8825,32 +8836,4 @@ bool take_page_off_buddy(struct page *page) spin_unlock_irqrestore(&zone->lock, flags); return ret; } - -/* - * Set PG_hwpoison flag if a given page is confirmed to be a free page. This - * test is performed under the zone lock to prevent a race against page - * allocation. - */ -bool set_hwpoison_free_buddy_page(struct page *page) -{ - struct zone *zone = page_zone(page); - unsigned long pfn = page_to_pfn(page); - unsigned long flags; - unsigned int order; - bool hwpoisoned = false; - - spin_lock_irqsave(&zone->lock, flags); - for (order = 0; order < MAX_ORDER; order++) { - struct page *page_head = page - (pfn & ((1 << order) - 1)); - - if (PageBuddy(page_head) && page_order(page_head) >= order) { - if (!TestSetPageHWPoison(page)) - hwpoisoned = true; - break; - } - } - spin_unlock_irqrestore(&zone->lock, flags); - - return hwpoisoned; -} #endif From patchwork Tue Sep 22 13:56:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11792623 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 06048112E for ; Tue, 22 Sep 2020 13:57:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C021D20719 for ; Tue, 22 Sep 2020 13:57:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C021D20719 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9E1966B00BE; Tue, 22 Sep 2020 09:57:12 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 87DFF6B00BF; Tue, 22 Sep 2020 09:57:12 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 71CDC6B00C0; Tue, 22 Sep 2020 09:57:12 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0211.hostedemail.com [216.40.44.211]) by kanga.kvack.org (Postfix) with ESMTP id 525B96B00BE for ; Tue, 22 Sep 2020 09:57:12 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 100503629 for ; Tue, 22 Sep 2020 13:57:12 +0000 (UTC) X-FDA: 77290849104.19.bomb40_230710b2714e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id D2ED81AD1B7 for ; Tue, 22 Sep 2020 13:57:11 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30012:30034:30054:30070,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-64.100.201.201 62.2.6.2;04ygnps9nmazotopzam3dmmdn8rxooc7swdffwjs8yipsd9fbgxqnku5xf88ndy.nzdd3u5d8wpm39du7dpp4zjrpshczk3eqs5kwser7pwpahz5mwxtsgrkdgrhg1b.e-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: bomb40_230710b2714e X-Filterd-Recvd-Size: 9785 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 13:57:11 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 000A1AFE5; Tue, 22 Sep 2020 13:57:46 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v7 10/14] mm,hwpoison: refactor soft_offline_huge_page and __soft_offline_page Date: Tue, 22 Sep 2020 15:56:46 +0200 Message-Id: <20200922135650.1634-11-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200922135650.1634-1-osalvador@suse.de> References: <20200922135650.1634-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Merging soft_offline_huge_page and __soft_offline_page let us get rid of quite some duplicated code, and makes the code much easier to follow. Now, __soft_offline_page will handle both normal and hugetlb pages. Signed-off-by: Oscar Salvador Acked-by: Naoya Horiguchi --- mm/memory-failure.c | 182 ++++++++++++++++++++------------------------ 1 file changed, 82 insertions(+), 100 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index cc8757282860..7c122cca9f31 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -65,13 +65,31 @@ int sysctl_memory_failure_recovery __read_mostly = 1; atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); -static void page_handle_poison(struct page *page, bool release) +static bool page_handle_poison(struct page *page, bool hugepage_or_freepage, bool release) { + if (hugepage_or_freepage) { + /* + * Doing this check for free pages is also fine since dissolve_free_huge_page + * returns 0 for non-hugetlb pages as well. + */ + if (dissolve_free_huge_page(page) || !take_page_off_buddy(page)) + /* + * We could fail to take off the target page from buddy + * for example due to racy page allocaiton, but that's + * acceptable because soft-offlined page is not broken + * and if someone really want to use it, they should + * take it. + */ + return false; + } + SetPageHWPoison(page); if (release) put_page(page); page_ref_inc(page); num_poisoned_pages_inc(); + + return true; } #if defined(CONFIG_HWPOISON_INJECT) || defined(CONFIG_HWPOISON_INJECT_MODULE) @@ -1725,63 +1743,51 @@ static int get_any_page(struct page *page, unsigned long pfn, int flags) return ret; } -static int soft_offline_huge_page(struct page *page, int flags) +static bool isolate_page(struct page *page, struct list_head *pagelist) { - int ret; - unsigned long pfn = page_to_pfn(page); - struct page *hpage = compound_head(page); - LIST_HEAD(pagelist); + bool isolated = false; + bool lru = PageLRU(page); - /* - * This double-check of PageHWPoison is to avoid the race with - * memory_failure(). See also comment in __soft_offline_page(). - */ - lock_page(hpage); - if (PageHWPoison(hpage)) { - unlock_page(hpage); - put_page(hpage); - pr_info("soft offline: %#lx hugepage already poisoned\n", pfn); - return -EBUSY; + if (PageHuge(page)) { + isolated = isolate_huge_page(page, pagelist); + } else { + if (lru) + isolated = !isolate_lru_page(page); + else + isolated = !isolate_movable_page(page, ISOLATE_UNEVICTABLE); + + if (isolated) + list_add(&page->lru, pagelist); } - unlock_page(hpage); - ret = isolate_huge_page(hpage, &pagelist); + if (isolated && lru) + inc_node_page_state(page, NR_ISOLATED_ANON + + page_is_file_lru(page)); + /* - * get_any_page() and isolate_huge_page() takes a refcount each, - * so need to drop one here. + * If we succeed to isolate the page, we grabbed another refcount on + * the page, so we can safely drop the one we got from get_any_pages(). + * If we failed to isolate the page, it means that we cannot go further + * and we will return an error, so drop the reference we got from + * get_any_pages() as well. */ - put_page(hpage); - if (!ret) { - pr_info("soft offline: %#lx hugepage failed to isolate\n", pfn); - return -EBUSY; - } - - ret = migrate_pages(&pagelist, new_page, NULL, MPOL_MF_MOVE_ALL, - MIGRATE_SYNC, MR_MEMORY_FAILURE); - if (ret) { - pr_info("soft offline: %#lx: hugepage migration failed %d, type %lx (%pGp)\n", - pfn, ret, page->flags, &page->flags); - if (!list_empty(&pagelist)) - putback_movable_pages(&pagelist); - if (ret > 0) - ret = -EIO; - } else { - /* - * We set PG_hwpoison only when we were able to take the page - * off the buddy. - */ - if (!dissolve_free_huge_page(page) && take_page_off_buddy(page)) - page_handle_poison(page, false); - else - ret = -EBUSY; - } - return ret; + put_page(page); + return isolated; } -static int __soft_offline_page(struct page *page, int flags) +/* + * __soft_offline_page handles hugetlb-pages and non-hugetlb pages. + * If the page is a non-dirty unmapped page-cache page, it simply invalidates. + * If the page is mapped, it migrates the contents over. + */ +static int __soft_offline_page(struct page *page) { - int ret; + int ret = 0; unsigned long pfn = page_to_pfn(page); + struct page *hpage = compound_head(page); + char const *msg_page[] = {"page", "hugepage"}; + bool huge = PageHuge(page); + LIST_HEAD(pagelist); /* * Check PageHWPoison again inside page lock because PageHWPoison @@ -1790,98 +1796,74 @@ static int __soft_offline_page(struct page *page, int flags) * so there's no race between soft_offline_page() and memory_failure(). */ lock_page(page); - wait_on_page_writeback(page); + if (!PageHuge(page)) + wait_on_page_writeback(page); if (PageHWPoison(page)) { unlock_page(page); put_page(page); pr_info("soft offline: %#lx page already poisoned\n", pfn); return -EBUSY; } - /* - * Try to invalidate first. This should work for - * non dirty unmapped page cache pages. - */ - ret = invalidate_inode_page(page); + + if (!PageHuge(page)) + /* + * Try to invalidate first. This should work for + * non dirty unmapped page cache pages. + */ + ret = invalidate_inode_page(page); unlock_page(page); + /* * RED-PEN would be better to keep it isolated here, but we * would need to fix isolation locking first. */ - if (ret == 1) { + if (ret) { pr_info("soft_offline: %#lx: invalidated\n", pfn); - page_handle_poison(page, true); + page_handle_poison(page, false, true); return 0; } - /* - * Simple invalidation didn't work. - * Try to migrate to a new page instead. migrate.c - * handles a large number of cases for us. - */ - if (PageLRU(page)) - ret = isolate_lru_page(page); - else - ret = isolate_movable_page(page, ISOLATE_UNEVICTABLE); - /* - * Drop page reference which is came from get_any_page() - * successful isolate_lru_page() already took another one. - */ - put_page(page); - if (!ret) { - LIST_HEAD(pagelist); - /* - * After isolated lru page, the PageLRU will be cleared, - * so use !__PageMovable instead for LRU page's mapping - * cannot have PAGE_MAPPING_MOVABLE. - */ - if (!__PageMovable(page)) - inc_node_page_state(page, NR_ISOLATED_ANON + - page_is_file_lru(page)); - list_add(&page->lru, &pagelist); + if (isolate_page(hpage, &pagelist)) { ret = migrate_pages(&pagelist, new_page, NULL, MPOL_MF_MOVE_ALL, MIGRATE_SYNC, MR_MEMORY_FAILURE); if (!ret) { - page_handle_poison(page, true); + bool release = !huge; + + if (!page_handle_poison(page, huge, release)) + ret = -EBUSY; } else { if (!list_empty(&pagelist)) putback_movable_pages(&pagelist); - pr_info("soft offline: %#lx: migration failed %d, type %lx (%pGp)\n", - pfn, ret, page->flags, &page->flags); + pr_info("soft offline: %#lx: %s migration failed %d, type %lx (%pGp)\n", + pfn, msg_page[huge], ret, page->flags, &page->flags); if (ret > 0) ret = -EIO; } } else { - pr_info("soft offline: %#lx: isolation failed: %d, page count %d, type %lx (%pGp)\n", - pfn, ret, page_count(page), page->flags, &page->flags); + pr_info("soft offline: %#lx: %s isolation failed: %d, page count %d, type %lx (%pGp)\n", + pfn, msg_page[huge], ret, page_count(page), page->flags, &page->flags); + ret = -EBUSY; } return ret; } -static int soft_offline_in_use_page(struct page *page, int flags) +static int soft_offline_in_use_page(struct page *page) { - int ret; struct page *hpage = compound_head(page); if (!PageHuge(page) && PageTransHuge(hpage)) if (try_to_split_thp_page(page, "soft offline") < 0) return -EBUSY; - - if (PageHuge(page)) - ret = soft_offline_huge_page(page, flags); - else - ret = __soft_offline_page(page, flags); - return ret; + return __soft_offline_page(page); } static int soft_offline_free_page(struct page *page) { - int rc = -EBUSY; + int rc = 0; - if (!dissolve_free_huge_page(page) && take_page_off_buddy(page)) { - page_handle_poison(page, false); - rc = 0; - } + if (!page_handle_poison(page, true, false)) + rc = -EBUSY; return rc; } @@ -1932,7 +1914,7 @@ int soft_offline_page(unsigned long pfn, int flags) put_online_mems(); if (ret > 0) - ret = soft_offline_in_use_page(page, flags); + ret = soft_offline_in_use_page(page); else if (ret == 0) ret = soft_offline_free_page(page); From patchwork Tue Sep 22 13:56:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11792625 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3FDDF112C for ; Tue, 22 Sep 2020 13:57:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0773B20719 for ; Tue, 22 Sep 2020 13:57:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0773B20719 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 01BFA6B00BF; Tue, 22 Sep 2020 09:57:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E7616900063; Tue, 22 Sep 2020 09:57:12 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B8CC36B00C5; Tue, 22 Sep 2020 09:57:12 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0170.hostedemail.com [216.40.44.170]) by kanga.kvack.org (Postfix) with ESMTP id 8C63C6B00C0 for ; Tue, 22 Sep 2020 09:57:12 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 3FB7A8249980 for ; Tue, 22 Sep 2020 13:57:12 +0000 (UTC) X-FDA: 77290849104.07.star01_4001ada2714e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id 2C8751803F9A8 for ; Tue, 22 Sep 2020 13:57:12 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30003:30054:30070,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-64.100.201.201 62.2.6.2;04yfeb17rotukr59js6h744yon1sdypwufk344gfhy4j4bjs844jr4kqnec8cez.bak6o5sp7yeao8tzxy7iu8gt5j3sn1xwq5tkbkfaahxkh34amcj6ipsg7pe5qr8.n-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: star01_4001ada2714e X-Filterd-Recvd-Size: 3185 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 13:57:11 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 8D086AFF7; Tue, 22 Sep 2020 13:57:47 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v7 11/14] mm,hwpoison: return 0 if the page is already poisoned in soft-offline Date: Tue, 22 Sep 2020 15:56:47 +0200 Message-Id: <20200922135650.1634-12-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200922135650.1634-1-osalvador@suse.de> References: <20200922135650.1634-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, there is an inconsistency when calling soft-offline from different paths on a page that is already poisoned. 1) madvise: madvise_inject_error skips any poisoned page and continues the loop. If that was the only page to madvise, it returns 0. 2) /sys/devices/system/memory/: When calling soft_offline_page_store()->soft_offline_page(), we return -EBUSY in case the page is already poisoned. This is inconsistent with a) the above example and b) memory_failure, where we return 0 if the page was poisoned. Fix this by dropping the PageHWPoison() check in madvise_inject_error, and let soft_offline_page return 0 if it finds the page already poisoned. Please, note that this represents a user-api change, since now the return error when calling soft_offline_page_store()->soft_offline_page() will be different. Signed-off-by: Oscar Salvador Acked-by: Naoya Horiguchi --- mm/madvise.c | 5 ----- mm/memory-failure.c | 4 ++-- 2 files changed, 2 insertions(+), 7 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index f3a86d547ceb..5ab978b828f0 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -896,11 +896,6 @@ static int madvise_inject_error(int behavior, */ size = page_size(compound_head(page)); - if (PageHWPoison(page)) { - put_page(page); - continue; - } - if (behavior == MADV_SOFT_OFFLINE) { pr_info("Soft offlining pfn %#lx at process virtual address %#lx\n", pfn, start); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 7c122cca9f31..087f070b06b5 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1802,7 +1802,7 @@ static int __soft_offline_page(struct page *page) unlock_page(page); put_page(page); pr_info("soft offline: %#lx page already poisoned\n", pfn); - return -EBUSY; + return 0; } if (!PageHuge(page)) @@ -1906,7 +1906,7 @@ int soft_offline_page(unsigned long pfn, int flags) pr_info("soft offline: %#lx page already poisoned\n", pfn); if (flags & MF_COUNT_INCREASED) put_page(page); - return -EBUSY; + return 0; } get_online_mems(); From patchwork Tue Sep 22 13:56:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11792627 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8CCE0112E for ; Tue, 22 Sep 2020 13:57:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5F47C20719 for ; Tue, 22 Sep 2020 13:57:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5F47C20719 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7BA1B6B00C0; Tue, 22 Sep 2020 09:57:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 74753900063; Tue, 22 Sep 2020 09:57:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60E2E6B00C5; Tue, 22 Sep 2020 09:57:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0157.hostedemail.com [216.40.44.157]) by kanga.kvack.org (Postfix) with ESMTP id 44FB96B00C0 for ; Tue, 22 Sep 2020 09:57:13 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 01B503629 for ; Tue, 22 Sep 2020 13:57:13 +0000 (UTC) X-FDA: 77290849146.19.sack54_0a0ef8b2714e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id CE3351AD1B7 for ; Tue, 22 Sep 2020 13:57:12 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30054,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-64.100.201.201 62.2.6.2;04y8k1bziiprdsxk96mu4udzgsgsgocggownx1zrsp6pkq4n8t5mx4h9f1dz6ao.9eobar8knuoxrwr9hinthmbe6chngi7psea6gu1t5ky8b1yhsmr4q339y8u9xdn.y-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: sack54_0a0ef8b2714e X-Filterd-Recvd-Size: 3592 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 13:57:12 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 20B9AB12B; Tue, 22 Sep 2020 13:57:48 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v7 12/14] mm,hwpoison: introduce MF_MSG_UNSPLIT_THP Date: Tue, 22 Sep 2020 15:56:48 +0200 Message-Id: <20200922135650.1634-13-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200922135650.1634-1-osalvador@suse.de> References: <20200922135650.1634-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi memory_failure() is supposed to call action_result() when it handles a memory error event, but there's one missing case. So let's add it. I find that include/ras/ras_event.h has some other MF_MSG_* undefined, so this patch also adds them. Signed-off-by: Naoya Horiguchi Signed-off-by: Oscar Salvador --- include/linux/mm.h | 1 + include/ras/ras_event.h | 3 +++ mm/memory-failure.c | 5 ++++- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 61a80384afd3..2c09b3f2d912 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3054,6 +3054,7 @@ enum mf_action_page_type { MF_MSG_BUDDY, MF_MSG_BUDDY_2ND, MF_MSG_DAX, + MF_MSG_UNSPLIT_THP, MF_MSG_UNKNOWN, }; diff --git a/include/ras/ras_event.h b/include/ras/ras_event.h index 36c5c5e38c1d..0bdbc0d17d2f 100644 --- a/include/ras/ras_event.h +++ b/include/ras/ras_event.h @@ -361,6 +361,7 @@ TRACE_EVENT(aer_event, EM ( MF_MSG_POISONED_HUGE, "huge page already hardware poisoned" ) \ EM ( MF_MSG_HUGE, "huge page" ) \ EM ( MF_MSG_FREE_HUGE, "free huge page" ) \ + EM ( MF_MSG_NON_PMD_HUGE, "non-pmd-sized huge page" ) \ EM ( MF_MSG_UNMAP_FAILED, "unmapping failed page" ) \ EM ( MF_MSG_DIRTY_SWAPCACHE, "dirty swapcache page" ) \ EM ( MF_MSG_CLEAN_SWAPCACHE, "clean swapcache page" ) \ @@ -373,6 +374,8 @@ TRACE_EVENT(aer_event, EM ( MF_MSG_TRUNCATED_LRU, "already truncated LRU page" ) \ EM ( MF_MSG_BUDDY, "free buddy page" ) \ EM ( MF_MSG_BUDDY_2ND, "free buddy page (2nd try)" ) \ + EM ( MF_MSG_DAX, "dax page" ) \ + EM ( MF_MSG_UNSPLIT_THP, "unsplit thp" ) \ EMe ( MF_MSG_UNKNOWN, "unknown page" ) /* diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 087f070b06b5..963fd9af23ab 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -582,6 +582,7 @@ static const char * const action_page_types[] = { [MF_MSG_BUDDY] = "free buddy page", [MF_MSG_BUDDY_2ND] = "free buddy page (2nd try)", [MF_MSG_DAX] = "dax page", + [MF_MSG_UNSPLIT_THP] = "unsplit thp", [MF_MSG_UNKNOWN] = "unknown page", }; @@ -1370,8 +1371,10 @@ int memory_failure(unsigned long pfn, int flags) } if (PageTransHuge(hpage)) { - if (try_to_split_thp_page(p, "Memory Failure") < 0) + if (try_to_split_thp_page(p, "Memory Failure") < 0) { + action_result(pfn, MF_MSG_UNSPLIT_THP, MF_IGNORED); return -EBUSY; + } VM_BUG_ON_PAGE(!page_count(p), p); } From patchwork Tue Sep 22 13:56:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11792629 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 023D6112C for ; Tue, 22 Sep 2020 13:57:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BBA7120719 for ; Tue, 22 Sep 2020 13:57:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BBA7120719 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F38896B00C3; Tue, 22 Sep 2020 09:57:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E98836B00C5; Tue, 22 Sep 2020 09:57:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D385F6B00C7; Tue, 22 Sep 2020 09:57:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0102.hostedemail.com [216.40.44.102]) by kanga.kvack.org (Postfix) with ESMTP id B3DE86B00C3 for ; Tue, 22 Sep 2020 09:57:13 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 7748D1EF1 for ; Tue, 22 Sep 2020 13:57:13 +0000 (UTC) X-FDA: 77290849146.05.water63_221188e2714e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 430B318021BB1 for ; Tue, 22 Sep 2020 13:57:13 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30054:30070,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201;04yf6njx34opwqi8tm14hfxm9wtgeypc6f8qed1kphqg6644wkx3zqu6engxgb3.nh5jnqpzc17n7sx56k9skittgx16pnuij4kbptpsrf95dk3fw9tdak1z31smdbu.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: water63_221188e2714e X-Filterd-Recvd-Size: 2500 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 13:57:12 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id A2FBBAFED; Tue, 22 Sep 2020 13:57:48 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v7 13/14] mm,hwpoison: double-check page count in __get_any_page() Date: Tue, 22 Sep 2020 15:56:49 +0200 Message-Id: <20200922135650.1634-14-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200922135650.1634-1-osalvador@suse.de> References: <20200922135650.1634-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Soft offlining could fail with EIO due to the race condition with hugepage migration. This issuse became visible due to the change by previous patch that makes soft offline handler take page refcount by its own. We have no way to directly pin zero refcount page, and the page considered as a zero refcount page could be allocated just after the first check. This patch adds the second check to find the race and gives us chance to handle it more reliably. Signed-off-by: Naoya Horiguchi Reported-by: Qian Cai --- mm/memory-failure.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 963fd9af23ab..46b1821d2817 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1707,6 +1707,9 @@ static int __get_any_page(struct page *p, unsigned long pfn, int flags) } else if (is_free_buddy_page(p)) { pr_info("%s: %#lx free buddy page\n", __func__, pfn); ret = 0; + } else if (page_count(p)) { + /* raced with allocation */ + ret = -EBUSY; } else { pr_info("%s: %#lx: unknown zero refcount page type %lx\n", __func__, pfn, p->flags); @@ -1723,6 +1726,9 @@ static int get_any_page(struct page *page, unsigned long pfn, int flags) { int ret = __get_any_page(page, pfn, flags); + if (ret == -EBUSY) + ret = __get_any_page(page, pfn, flags); + if (ret == 1 && !PageHuge(page) && !PageLRU(page) && !__PageMovable(page)) { /* From patchwork Tue Sep 22 13:56:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11792631 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2E046112E for ; Tue, 22 Sep 2020 13:57:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EB32220719 for ; Tue, 22 Sep 2020 13:57:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EB32220719 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A2C956B00C5; Tue, 22 Sep 2020 09:57:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9B52A6B00C7; Tue, 22 Sep 2020 09:57:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A1376B00C8; Tue, 22 Sep 2020 09:57:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0026.hostedemail.com [216.40.44.26]) by kanga.kvack.org (Postfix) with ESMTP id 6DF7E6B00C5 for ; Tue, 22 Sep 2020 09:57:14 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 336733629 for ; Tue, 22 Sep 2020 13:57:14 +0000 (UTC) X-FDA: 77290849188.27.point22_5f171032714e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id 09BFD3D663 for ; Tue, 22 Sep 2020 13:57:14 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30054,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-64.100.201.201 62.2.6.2;04ygzixsoor4zs9fet6wfwg7sjetjyc7n47jk6jx3w6n55umzgj654mizrgbdg4.wyoax19apzybncqj1rdkfammkqa4sb8jxyzdfwzeofwa1wnkzr19grxbba6qc8n.1-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: point22_5f171032714e X-Filterd-Recvd-Size: 2497 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 13:57:13 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 39B4BAFBC; Tue, 22 Sep 2020 13:57:49 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v7 14/14] mm,hwpoison: Try to narrow window race for free pages Date: Tue, 22 Sep 2020 15:56:50 +0200 Message-Id: <20200922135650.1634-15-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200922135650.1634-1-osalvador@suse.de> References: <20200922135650.1634-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Aristeu Rozanski reported that a customer test case started to report -EBUSY after the hwpoison rework patchset. There is a race window between spotting a free page and taking it off its buddy freelist, so it might be that by the time we try to take it off, the page has been already allocated. This patch tries to handle such race window by trying to handle the new type of page again if the page was allocated under us. Signed-off-by: Oscar Salvador Reported-by: Aristeu Rozanski Tested-by: Aristeu Rozanski Acked-by: Naoya Horiguchi --- mm/memory-failure.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 46b1821d2817..8f23d3c7a0a2 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1903,6 +1903,7 @@ int soft_offline_page(unsigned long pfn, int flags) { int ret; struct page *page; + bool try_again = true; if (!pfn_valid(pfn)) return -ENXIO; @@ -1918,6 +1919,7 @@ int soft_offline_page(unsigned long pfn, int flags) return 0; } +retry: get_online_mems(); ret = get_any_page(page, pfn, flags); put_online_mems(); @@ -1925,7 +1927,10 @@ int soft_offline_page(unsigned long pfn, int flags) if (ret > 0) ret = soft_offline_in_use_page(page); else if (ret == 0) - ret = soft_offline_free_page(page); + if (soft_offline_free_page(page) && try_again) { + try_again = false; + goto retry; + } return ret; }