From patchwork Thu Sep 17 08:10:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11781967 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2F55E618 for ; Thu, 17 Sep 2020 08:27:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CD75D21973 for ; Thu, 17 Sep 2020 08:27:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CD75D21973 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BD7046B005C; Thu, 17 Sep 2020 04:27:57 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B62366B005D; Thu, 17 Sep 2020 04:27:57 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9DA9D6B0062; Thu, 17 Sep 2020 04:27:57 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0223.hostedemail.com [216.40.44.223]) by kanga.kvack.org (Postfix) with ESMTP id 83BC26B005C for ; Thu, 17 Sep 2020 04:27:57 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 0BD2B181AEF2A for ; Thu, 17 Sep 2020 08:27:57 +0000 (UTC) X-FDA: 77271875394.24.part79_1f0c26327121 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id CD63C1A4A0 for ; Thu, 17 Sep 2020 08:27:56 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30029:30054:30056:30070:30090,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-64.100.201.201 62.2.6.2;04yrpquoehjgu7qtr965e4ano7oa4op7yhyhj1htc5qwmem3iy5b9dni7jo8o3d.43ne6cikyoi4oo1h51xa4iyn5367xpjdohg645995xemgjgbcq3m365ofu5qwuf.o-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: part79_1f0c26327121 X-Filterd-Recvd-Size: 5715 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Thu, 17 Sep 2020 08:27:56 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 7AEF0B55C; Thu, 17 Sep 2020 08:28:26 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v4 1/7] mm,hwpoison: take free pages off the buddy freelists Date: Thu, 17 Sep 2020 10:10:43 +0200 Message-Id: <20200917081049.27428-2-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200917081049.27428-1-osalvador@suse.de> References: <20200917081049.27428-1-osalvador@suse.de> X-Rspamd-Queue-Id: CD63C1A4A0 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The crux of the matter is that historically we left poisoned pages in the buddy system because we have some checks in place when allocating a page that a gatekeeper for poisoned pages. Unfortunately, we do have other users (e.g: compaction [1]) that scan buddy freelists and try to get a page from there without checking whether the page is HWPoison. As I stated already, I think it is fundamentally wrong to keep HWPoison pages within the buddy systems, checks in place or not. Let us fix this we same way we did for soft_offline [2], and take the page off the buddy freelist, so it is completely unreachable. Note that this is fairly simple to trigger, as we only need to poison free buddy pages (madvise MADV_HWPOISON) and then we need to run some sort of memory stress system. Just for a matter of reference, I put a dump_page in compaction_alloc to trigger for HWPoison patches: kernel: page:0000000012b2982b refcount:1 mapcount:0 mapping:0000000000000000 index:0x1 pfn:0x1d5db kernel: flags: 0xfffffc0800000(hwpoison) kernel: raw: 000fffffc0800000 ffffea00007573c8 ffffc90000857de0 0000000000000000 kernel: raw: 0000000000000001 0000000000000000 00000001ffffffff 0000000000000000 kernel: page dumped because: compaction_alloc kernel: CPU: 4 PID: 123 Comm: kcompactd0 Tainted: G E 5.9.0-rc2-mm1-1-default+ #5 kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014 kernel: Call Trace: kernel: dump_stack+0x6d/0x8b kernel: compaction_alloc+0xb2/0xc0 kernel: migrate_pages+0x2a6/0x12a0 kernel: ? isolate_freepages+0xc80/0xc80 kernel: ? __ClearPageMovable+0xb0/0xb0 kernel: compact_zone+0x5eb/0x11c0 kernel: ? finish_task_switch+0x74/0x300 kernel: ? lock_timer_base+0xa8/0x170 kernel: proactive_compact_node+0x89/0xf0 kernel: ? kcompactd+0x2d0/0x3a0 kernel: kcompactd+0x2d0/0x3a0 kernel: ? finish_wait+0x80/0x80 kernel: ? kcompactd_do_work+0x350/0x350 kernel: kthread+0x118/0x130 kernel: ? kthread_associate_blkcg+0xa0/0xa0 kernel: ret_from_fork+0x22/0x30 After that, if e.g: someone faults in the page, that someone will get killed unexpectedly. While we are at it, I also changed the action result for such cases. I think that MF_DELAYED is a bit misleading, because in case we could contain the page and take it off the buddy, such a page is not to be used again unless it is unpoisoned, so we fixed the situation. So unless I am missing something, I strongly think that we should report MF_RECOVERED. [1] https://lore.kernel.org/linux-mm/20190826104144.GA7849@linux/T/#u [2] https://patchwork.kernel.org/patch/11694847/ Signed-off-by: Oscar Salvador Acked-by: Naoya Horiguchi --- mm/memory-failure.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 4eb3c42ffe35..74a53881f94b 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1323,8 +1323,9 @@ int memory_failure(unsigned long pfn, int flags) struct page *hpage; struct page *orig_head; struct dev_pagemap *pgmap; - int res; unsigned long page_flags; + int res = 0; + bool retry = true; if (!sysctl_memory_failure_recovery) panic("Memory failure on page %lx", pfn); @@ -1364,10 +1365,21 @@ int memory_failure(unsigned long pfn, int flags) * In fact it's dangerous to directly bump up page count from 0, * that may make page_ref_freeze()/page_ref_unfreeze() mismatch. */ +try_again: if (!(flags & MF_COUNT_INCREASED) && !get_hwpoison_page(p)) { if (is_free_buddy_page(p)) { - action_result(pfn, MF_MSG_BUDDY, MF_DELAYED); - return 0; + if (take_page_off_buddy(p)) { + action_result(pfn, MF_MSG_BUDDY, MF_RECOVERED); + return 0; + } else { + /* We lost the race, try again */ + if (retry) { + retry = false; + goto try_again; + } + action_result(pfn, MF_MSG_BUDDY, MF_FAILED); + return -EBUSY; + } } else { action_result(pfn, MF_MSG_KERNEL_HIGH_ORDER, MF_IGNORED); return -EBUSY; @@ -1393,11 +1405,15 @@ int memory_failure(unsigned long pfn, int flags) shake_page(p, 0); /* shake_page could have turned it free. */ if (!PageLRU(p) && is_free_buddy_page(p)) { + if (!take_page_off_buddy(p)) + res = -EBUSY; + if (flags & MF_COUNT_INCREASED) - action_result(pfn, MF_MSG_BUDDY, MF_DELAYED); + action_result(pfn, MF_MSG_BUDDY, res ? MF_FAILED : MF_RECOVERED); else - action_result(pfn, MF_MSG_BUDDY_2ND, MF_DELAYED); - return 0; + action_result(pfn, MF_MSG_BUDDY_2ND, res ? MF_FAILED : MF_RECOVERED); + + return res; } lock_page(p); From patchwork Thu Sep 17 08:10:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11781973 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 33365112E for ; Thu, 17 Sep 2020 08:28:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 06A0E20715 for ; Thu, 17 Sep 2020 08:28:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 06A0E20715 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 01E186B0068; Thu, 17 Sep 2020 04:28:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EE3B56B006C; Thu, 17 Sep 2020 04:28:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF83E6B006E; Thu, 17 Sep 2020 04:28:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0191.hostedemail.com [216.40.44.191]) by kanga.kvack.org (Postfix) with ESMTP id CAAD06B0068 for ; Thu, 17 Sep 2020 04:28:14 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 968C72C84 for ; Thu, 17 Sep 2020 08:28:14 +0000 (UTC) X-FDA: 77271876108.12.plane92_1f0d49327121 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin12.hostedemail.com (Postfix) with ESMTP id 6ED8018017AC7 for ; Thu, 17 Sep 2020 08:28:14 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30054,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-64.100.201.201 62.2.6.2;04y8xnhx63hcx5943x5zzkrzdmq67oc4pq8ao9ggp6wch6ramj5cqt91gefuh4r.1c3dhnbki379x4di58fxg5m3wzxz779pr4y9485tzzf3d9ibphnokdzezasxmq5.q-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: plane92_1f0d49327121 X-Filterd-Recvd-Size: 2050 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Thu, 17 Sep 2020 08:28:13 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 78355B244; Thu, 17 Sep 2020 08:28:45 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v4 2/7] mm,hwpoison: Do not set hugepage_or_freepage unconditionally Date: Thu, 17 Sep 2020 10:10:44 +0200 Message-Id: <20200917081049.27428-3-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200917081049.27428-1-osalvador@suse.de> References: <20200917081049.27428-1-osalvador@suse.de> X-Rspamd-Queue-Id: 6ED8018017AC7 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Aristeu Rozanski reported that some hwpoison tests were returning -EBUSY while before the rework they succeed [1] [2]. The root case is that during a rebase, the call to page_handle_poison was setting hugepage_or_freepage = true unconditionally from __soft_offline_page. Aristeu said that after this fix, everything works. [1] https://patchwork.kernel.org/comment/23617301/ [2] https://patchwork.kernel.org/comment/23619535/ Signed-off-by: Oscar Salvador Reported-by: Aristeu Rozanski Tested-by: Aristeu Rozanski --- mm/memory-failure.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 74a53881f94b..db61bdee9734 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1852,7 +1852,7 @@ static int __soft_offline_page(struct page *page) if (!ret) { bool release = !huge; - if (!page_handle_poison(page, true, release)) + if (!page_handle_poison(page, huge, release)) ret = -EBUSY; } else { if (!list_empty(&pagelist)) From patchwork Thu Sep 17 08:10:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11781963 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D456E618 for ; Thu, 17 Sep 2020 08:27:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8993E2083B for ; Thu, 17 Sep 2020 08:27:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8993E2083B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 76D986B0055; Thu, 17 Sep 2020 04:27:47 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6F2BF6B005A; Thu, 17 Sep 2020 04:27:47 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E1E36B005C; Thu, 17 Sep 2020 04:27:47 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0038.hostedemail.com [216.40.44.38]) by kanga.kvack.org (Postfix) with ESMTP id 449D86B0055 for ; Thu, 17 Sep 2020 04:27:47 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id EEFD78249980 for ; Thu, 17 Sep 2020 08:27:46 +0000 (UTC) X-FDA: 77271874932.02.cough26_2807aaf27121 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin02.hostedemail.com (Postfix) with ESMTP id BF08F100BEE39 for ; Thu, 17 Sep 2020 08:27:46 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30054,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201;04yfxbqaqkburu56fmn44kqkh7n8iyc7n47jk6jx7wncd9rop5o99ybwrzwhq5q.q6q3eu9apzybncqj1rdkfammkqhtx5mtg71dfwzeofwa1y8h4tkigx9bba6qc8n.1-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: cough26_2807aaf27121 X-Filterd-Recvd-Size: 2515 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Thu, 17 Sep 2020 08:27:46 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 2A772B134; Thu, 17 Sep 2020 08:28:18 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v4 3/7] mm,hwpoison: Try to narrow window race for free pages Date: Thu, 17 Sep 2020 10:10:45 +0200 Message-Id: <20200917081049.27428-4-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200917081049.27428-1-osalvador@suse.de> References: <20200917081049.27428-1-osalvador@suse.de> X-Rspamd-Queue-Id: BF08F100BEE39 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Aristeu Rozanski reported that a customer test case started to report -EBUSY after the hwpoison report patchset. There is a race window between spotting a free page and taking it off its buddy freelist, so it might be that by the time we try to take it off, the page has been already allocated. This patch tries to handle such race window by trying to handle the new type of page again if the page was allocated under us. After this patch, Aristeu said the test cases work properly. Signed-off-by: Oscar Salvador Reported-by: Aristeu Rozanski Tested-by: Aristeu Rozanski --- mm/memory-failure.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index db61bdee9734..a2ccd3ba4015 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1917,6 +1917,7 @@ int soft_offline_page(unsigned long pfn, int flags) { int ret; struct page *page; + bool try_again = true; if (!pfn_valid(pfn)) return -ENXIO; @@ -1932,6 +1933,7 @@ int soft_offline_page(unsigned long pfn, int flags) return 0; } +retry: get_online_mems(); ret = get_any_page(page, pfn, flags); put_online_mems(); @@ -1939,7 +1941,10 @@ int soft_offline_page(unsigned long pfn, int flags) if (ret > 0) ret = soft_offline_in_use_page(page); else if (ret == 0) - ret = soft_offline_free_page(page); + if (soft_offline_free_page(page) && try_again) { + try_again = false; + goto retry; + } return ret; } From patchwork Thu Sep 17 08:10:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11781977 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 45E93618 for ; Thu, 17 Sep 2020 08:28:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0C3DB20715 for ; Thu, 17 Sep 2020 08:28:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0C3DB20715 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2DA7F6B006E; Thu, 17 Sep 2020 04:28:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 285426B0070; Thu, 17 Sep 2020 04:28:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14E266B0071; Thu, 17 Sep 2020 04:28:36 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0069.hostedemail.com [216.40.44.69]) by kanga.kvack.org (Postfix) with ESMTP id F37D16B006E for ; Thu, 17 Sep 2020 04:28:35 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id BDA262C98 for ; Thu, 17 Sep 2020 08:28:35 +0000 (UTC) X-FDA: 77271876990.14.home33_3d0d8f227121 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin14.hostedemail.com (Postfix) with ESMTP id 9034118229818 for ; Thu, 17 Sep 2020 08:28:35 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30003:30054:30070,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201;04yfz8fk554bssdfjgtqq5ix7bxnhypr9uzfrqbppk1ab1xd6akrtborehrgkit.h4qztuma4ncuqnk97bie4htopqiuifn18i9yj7gb81649j9sw44tduiw1ebigjt.h-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: home33_3d0d8f227121 X-Filterd-Recvd-Size: 3161 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Thu, 17 Sep 2020 08:28:34 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id A0572B053; Thu, 17 Sep 2020 08:29:05 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador , Oscar Salvador Subject: [PATCH v4 4/7] mm,hwpoison: refactor madvise_inject_error Date: Thu, 17 Sep 2020 10:10:46 +0200 Message-Id: <20200917081049.27428-5-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200917081049.27428-1-osalvador@suse.de> References: <20200917081049.27428-1-osalvador@suse.de> X-Rspamd-Queue-Id: 9034118229818 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make a proper if-else condition for {hard,soft}-offline. [akpm: remove zone variable and refactor comment] Signed-off-by: Oscar Salvador --- mm/madvise.c | 32 ++++++++++++++------------------ 1 file changed, 14 insertions(+), 18 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 4d87b31a1576..4a48f7215195 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -869,8 +869,6 @@ static long madvise_remove(struct vm_area_struct *vma, static int madvise_inject_error(int behavior, unsigned long start, unsigned long end) { - struct page *page; - struct zone *zone; unsigned long size; if (!capable(CAP_SYS_ADMIN)) @@ -879,6 +877,7 @@ static int madvise_inject_error(int behavior, for (; start < end; start += size) { unsigned long pfn; + struct page *page; int ret; ret = get_user_pages_fast(start, 1, 0, &page); @@ -895,25 +894,22 @@ static int madvise_inject_error(int behavior, if (behavior == MADV_SOFT_OFFLINE) { pr_info("Soft offlining pfn %#lx at process virtual address %#lx\n", - pfn, start); - + pfn, start); ret = soft_offline_page(pfn, MF_COUNT_INCREASED); - if (ret) - return ret; - continue; + } else { + pr_info("Injecting memory failure for pfn %#lx at process virtual address %#lx\n", + pfn, start); + /* + * Drop the page reference taken by + * get_user_pages_fast(). In the absence of + * MF_COUNT_INCREASED the memory_failure() routine is + * responsible for pinning the page to prevent it + * from being released back to the page allocator. + */ + put_page(page); + ret = memory_failure(pfn, 0); } - pr_info("Injecting memory failure for pfn %#lx at process virtual address %#lx\n", - pfn, start); - - /* - * Drop the page reference taken by get_user_pages_fast(). In - * the absence of MF_COUNT_INCREASED the memory_failure() - * routine is responsible for pinning the page to prevent it - * from being released back to the page allocator. - */ - put_page(page); - ret = memory_failure(pfn, 0); if (ret) return ret; } From patchwork Thu Sep 17 08:10:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11781975 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 75618112E for ; Thu, 17 Sep 2020 08:28:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 45E4720715 for ; Thu, 17 Sep 2020 08:28:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 45E4720715 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7DC856B006C; Thu, 17 Sep 2020 04:28:32 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7664C6B006E; Thu, 17 Sep 2020 04:28:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 656916B0070; Thu, 17 Sep 2020 04:28:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0046.hostedemail.com [216.40.44.46]) by kanga.kvack.org (Postfix) with ESMTP id 4C4E86B006C for ; Thu, 17 Sep 2020 04:28:32 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id F276B180AD804 for ; Thu, 17 Sep 2020 08:28:31 +0000 (UTC) X-FDA: 77271876864.10.pet14_150219d27121 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id C2E3816A4A9 for ; Thu, 17 Sep 2020 08:28:31 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30051:30054:30070:30083,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201;04y8ptqs1kdbpurzjbsb933f8qjruoczx6xiuchmms4ttiekf1bs5m35isjgxzu.dih9k68hh1ij13juez4i5xtf6ejqnq6k4m8q4k4rxdqx9zy9wz9zc64rmbs4fbc.h-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: pet14_150219d27121 X-Filterd-Recvd-Size: 2935 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Thu, 17 Sep 2020 08:28:31 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id CE6BCB240; Thu, 17 Sep 2020 08:29:01 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v4 5/7] mm,hwpoison: drain pcplists before bailing out for non-buddy zero-refcount page Date: Thu, 17 Sep 2020 10:10:47 +0200 Message-Id: <20200917081049.27428-6-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200917081049.27428-1-osalvador@suse.de> References: <20200917081049.27428-1-osalvador@suse.de> X-Rspamd-Queue-Id: C2E3816A4A9 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A page with 0-refcount and !PageBuddy could perfectly be a pcppage. Currently, we bail out with an error if we encounter such a page, meaning that we do not handle pcppages neither from hard-offline nor from soft-offline path. Fix this by draining pcplists whenever we find this kind of page and retry the check again. It might be that pcplists have been spilled into the buddy allocator and so we can handle it. Signed-off-by: Oscar Salvador Acked-by: Naoya Horiguchi --- mm/memory-failure.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index a2ccd3ba4015..7fba4ba201d5 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -950,13 +950,13 @@ static int page_action(struct page_state *ps, struct page *p, } /** - * get_hwpoison_page() - Get refcount for memory error handling: + * __get_hwpoison_page() - Get refcount for memory error handling: * @page: raw error page (hit by memory error) * * Return: return 0 if failed to grab the refcount, otherwise true (some * non-zero value.) */ -static int get_hwpoison_page(struct page *page) +static int __get_hwpoison_page(struct page *page) { struct page *head = compound_head(page); @@ -986,6 +986,26 @@ static int get_hwpoison_page(struct page *page) return 0; } +static int get_hwpoison_page(struct page *p) +{ + int ret; + bool drained = false; + +retry: + ret = __get_hwpoison_page(p); + if (!ret && !is_free_buddy_page(p) && !page_count(p) && !drained) { + /* + * The page might be in a pcplist, so try to drain those + * and see if we are lucky. + */ + drain_all_pages(page_zone(p)); + drained = true; + goto retry; + } + + return ret; +} + /* * Do all that is necessary to remove user space mappings. Unmap * the pages and send SIGBUS to the processes if the data was dirty. From patchwork Thu Sep 17 08:10:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11781971 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DA03F618 for ; Thu, 17 Sep 2020 08:28:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B368920770 for ; Thu, 17 Sep 2020 08:28:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B368920770 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B39846B0062; Thu, 17 Sep 2020 04:28:12 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AC19A6B0068; Thu, 17 Sep 2020 04:28:12 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9B32D6B006C; Thu, 17 Sep 2020 04:28:12 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0026.hostedemail.com [216.40.44.26]) by kanga.kvack.org (Postfix) with ESMTP id 843196B0062 for ; Thu, 17 Sep 2020 04:28:12 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 4FACD33CD for ; Thu, 17 Sep 2020 08:28:12 +0000 (UTC) X-FDA: 77271876024.24.wire05_2702e8927121 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 2B03F1A4A0 for ; Thu, 17 Sep 2020 08:28:12 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30054,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-64.100.201.201 62.2.6.2;04yr5p35ashzzsidrtjxis854typ6oc5seeg7te93f7aqpnpu85z38yamoyzwzy.6363wihjqxm33eqixss7bddsfq4z4xr8herhub5utr59d4gtkm53rwrobeqo3fm.c-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: wire05_2702e8927121 X-Filterd-Recvd-Size: 1899 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Thu, 17 Sep 2020 08:28:11 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id C1A91B001; Thu, 17 Sep 2020 08:28:43 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v4 6/7] mm,hwpoison: drop unneeded pcplist draining Date: Thu, 17 Sep 2020 10:10:48 +0200 Message-Id: <20200917081049.27428-7-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200917081049.27428-1-osalvador@suse.de> References: <20200917081049.27428-1-osalvador@suse.de> X-Rspamd-Queue-Id: 2B03F1A4A0 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: memory_failure and soft_offline_path paths now drain pcplists by calling get_hwpoison_page. memory_failure flags the page as HWPoison before, so that page cannot longer go into a pcplist, and soft_offline_page only flags a page as HWPoison if 1) we took the page off a buddy freelist 2) the page was in-use and we migrated it 3) was a clean pagecache. Because of that, a page cannot longer be poisoned and be in a pcplist. Signed-off-by: Oscar Salvador --- mm/madvise.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 4a48f7215195..302f3a84d17c 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -914,10 +914,6 @@ static int madvise_inject_error(int behavior, return ret; } - /* Ensure that all poisoned pages are removed from per-cpu lists */ - for_each_populated_zone(zone) - drain_all_pages(zone); - return 0; } #endif From patchwork Thu Sep 17 08:10:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11781969 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 10171618 for ; Thu, 17 Sep 2020 08:28:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BEB9120770 for ; Thu, 17 Sep 2020 08:28:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BEB9120770 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 016806B005D; Thu, 17 Sep 2020 04:28:10 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EE1256B0062; Thu, 17 Sep 2020 04:28:09 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD05F6B0068; Thu, 17 Sep 2020 04:28:09 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0206.hostedemail.com [216.40.44.206]) by kanga.kvack.org (Postfix) with ESMTP id C3C086B005D for ; Thu, 17 Sep 2020 04:28:09 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 7FD618249980 for ; Thu, 17 Sep 2020 08:28:09 +0000 (UTC) X-FDA: 77271875898.29.eyes19_4906f0527121 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id 63900180868EE for ; Thu, 17 Sep 2020 08:28:09 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,,RULES_HIT:30012:30054:30070,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-64.100.201.201 62.2.6.2;04yrimipabzq5ca444adzxxxerr7ryceppofnf5ohqpp4p51e39cux9ywa3jnep.cf735d89fkekqtj5u1exkih3ysp3edtt4hkr9ykettdbdrsf56f6181f5bt9rc4.6-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: eyes19_4906f0527121 X-Filterd-Recvd-Size: 2163 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Thu, 17 Sep 2020 08:28:08 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id BAF28B146; Thu, 17 Sep 2020 08:28:40 +0000 (UTC) From: Oscar Salvador To: akpm@linux-foundation.org Cc: aris@ruivo.org, naoya.horiguchi@nec.com, mhocko@kernel.org, tony.luck@intel.com, cai@lca.pw, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Oscar Salvador Subject: [PATCH v4 7/7] mm,hwpoison: remove stale code Date: Thu, 17 Sep 2020 10:10:49 +0200 Message-Id: <20200917081049.27428-8-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20200917081049.27428-1-osalvador@suse.de> References: <20200917081049.27428-1-osalvador@suse.de> X-Rspamd-Queue-Id: 63900180868EE X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently we call shake_page and then check whether the page is Buddy because shake_page calls drain_all_pages, which sends pcp-pages back to the buddy freelists, so we could have a chance to handle free pages. get_hwpoison_page already calls drain_all_pages, and we do call get_hwpoison_page right before coming here, so we should be on the safe side. Signed-off-by: Oscar Salvador --- mm/memory-failure.c | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 7fba4ba201d5..f68cb5e3b320 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1423,18 +1423,6 @@ int memory_failure(unsigned long pfn, int flags) * walked by the page reclaim code, however that's not a big loss. */ shake_page(p, 0); - /* shake_page could have turned it free. */ - if (!PageLRU(p) && is_free_buddy_page(p)) { - if (!take_page_off_buddy(p)) - res = -EBUSY; - - if (flags & MF_COUNT_INCREASED) - action_result(pfn, MF_MSG_BUDDY, res ? MF_FAILED : MF_RECOVERED); - else - action_result(pfn, MF_MSG_BUDDY_2ND, res ? MF_FAILED : MF_RECOVERED); - - return res; - } lock_page(p);