From patchwork Mon Oct 25 23:05:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12583463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9990CC433F5 for ; Mon, 25 Oct 2021 23:06:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 373766112D for ; Mon, 25 Oct 2021 23:06:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 373766112D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.dev Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id C731A80008; Mon, 25 Oct 2021 19:06:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C2340940007; Mon, 25 Oct 2021 19:06:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AC43A80008; Mon, 25 Oct 2021 19:06:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0156.hostedemail.com [216.40.44.156]) by kanga.kvack.org (Postfix) with ESMTP id 9B4DE940007 for ; Mon, 25 Oct 2021 19:06:31 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5A39D267F0 for ; Mon, 25 Oct 2021 23:06:31 +0000 (UTC) X-FDA: 78736495782.05.64E0DEA Received: from out1.migadu.com (out1.migadu.com [91.121.223.63]) by imf22.hostedemail.com (Postfix) with ESMTP id E46071900 for ; Mon, 25 Oct 2021 23:06:30 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1635203189; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RR7imi14Z6SUKcAiAGApew2NGLLdTeu4YcmU7ch/wOs=; b=JSZSt4rEezuk5+bDKBhgCteVmui5+uPUamj0ljbeldA6sgTX2nqdcD3aOpqO9LxJvKTHde Eg0dz1OiBSx238wJ5lpjtX5FvYgLx5DYVQBv9GD7+8uFLXzpVZb8xOg+wNXGKc/td+1//i 23tAvpVQbVcL/5PXtU4a9rGzSCFzG3I= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Oscar Salvador , Michal Hocko , Ding Hui , Tony Luck , "Aneesh Kumar K.V" , Miaohe Lin , Yang Shi , Peter Xu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 1/4] mm/hwpoison: mf_mutex for soft offline and unpoison Date: Tue, 26 Oct 2021 08:05:00 +0900 Message-Id: <20211025230503.2650970-2-naoya.horiguchi@linux.dev> In-Reply-To: <20211025230503.2650970-1-naoya.horiguchi@linux.dev> References: <20211025230503.2650970-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: naoya.horiguchi@linux.dev X-Stat-Signature: 6kun9qbd7dhuoa8c79uuu6nqi4wgoumg X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: E46071900 Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=JSZSt4rE; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf22.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 91.121.223.63 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev X-HE-Tag: 1635203190-847893 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Originally mf_mutex is introduced to serialize multiple MCE events, but it's also helpful to exclude races among soft_offline_page() and unpoison_memory(). So apply mf_mutex to them. Signed-off-by: Naoya Horiguchi --- ChangeLog v2: - add mutex_unlock() in "page already poisoned" path in soft_offline_page(). (Thanks to Ding Hui) --- mm/memory-failure.c | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index fa9dda95a2a2..97297edfbd8e 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1628,6 +1628,8 @@ static int memory_failure_dev_pagemap(unsigned long pfn, int flags, return rc; } +static DEFINE_MUTEX(mf_mutex); + /** * memory_failure - Handle memory failure of a page. * @pfn: Page Number of the corrupted page @@ -1654,7 +1656,6 @@ int memory_failure(unsigned long pfn, int flags) int res = 0; unsigned long page_flags; bool retry = true; - static DEFINE_MUTEX(mf_mutex); if (!sysctl_memory_failure_recovery) panic("Memory failure on page %lx", pfn); @@ -1978,6 +1979,7 @@ int unpoison_memory(unsigned long pfn) struct page *page; struct page *p; int freeit = 0; + int ret = 0; unsigned long flags = 0; static DEFINE_RATELIMIT_STATE(unpoison_rs, DEFAULT_RATELIMIT_INTERVAL, DEFAULT_RATELIMIT_BURST); @@ -1988,28 +1990,30 @@ int unpoison_memory(unsigned long pfn) p = pfn_to_page(pfn); page = compound_head(p); + mutex_lock(&mf_mutex); + if (!PageHWPoison(p)) { unpoison_pr_info("Unpoison: Page was already unpoisoned %#lx\n", pfn, &unpoison_rs); - return 0; + goto unlock_mutex; } if (page_count(page) > 1) { unpoison_pr_info("Unpoison: Someone grabs the hwpoison page %#lx\n", pfn, &unpoison_rs); - return 0; + goto unlock_mutex; } if (page_mapped(page)) { unpoison_pr_info("Unpoison: Someone maps the hwpoison page %#lx\n", pfn, &unpoison_rs); - return 0; + goto unlock_mutex; } if (page_mapping(page)) { unpoison_pr_info("Unpoison: the hwpoison page has non-NULL mapping %#lx\n", pfn, &unpoison_rs); - return 0; + goto unlock_mutex; } /* @@ -2020,7 +2024,7 @@ int unpoison_memory(unsigned long pfn) if (!PageHuge(page) && PageTransHuge(page)) { unpoison_pr_info("Unpoison: Memory failure is now running on %#lx\n", pfn, &unpoison_rs); - return 0; + goto unlock_mutex; } if (!get_hwpoison_page(p, flags)) { @@ -2028,7 +2032,7 @@ int unpoison_memory(unsigned long pfn) num_poisoned_pages_dec(); unpoison_pr_info("Unpoison: Software-unpoisoned free page %#lx\n", pfn, &unpoison_rs); - return 0; + goto unlock_mutex; } lock_page(page); @@ -2050,7 +2054,9 @@ int unpoison_memory(unsigned long pfn) if (freeit && !(pfn == my_zero_pfn(0) && page_count(p) == 1)) put_page(page); - return 0; +unlock_mutex: + mutex_unlock(&mf_mutex); + return ret; } EXPORT_SYMBOL(unpoison_memory); @@ -2231,9 +2237,12 @@ int soft_offline_page(unsigned long pfn, int flags) return -EIO; } + mutex_lock(&mf_mutex); + if (PageHWPoison(page)) { pr_info("%s: %#lx page already poisoned\n", __func__, pfn); put_ref_page(ref_page); + mutex_unlock(&mf_mutex); return 0; } @@ -2251,5 +2260,7 @@ int soft_offline_page(unsigned long pfn, int flags) } } + mutex_unlock(&mf_mutex); + return ret; } From patchwork Mon Oct 25 23:05:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12583465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40C52C433EF for ; Mon, 25 Oct 2021 23:06:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E955561073 for ; Mon, 25 Oct 2021 23:06:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E955561073 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.dev Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 87CF480009; Mon, 25 Oct 2021 19:06:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 82B1A940007; Mon, 25 Oct 2021 19:06:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6CCA880009; Mon, 25 Oct 2021 19:06:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0121.hostedemail.com [216.40.44.121]) by kanga.kvack.org (Postfix) with ESMTP id 5A41F940007 for ; Mon, 25 Oct 2021 19:06:36 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 09F678249980 for ; Mon, 25 Oct 2021 23:06:36 +0000 (UTC) X-FDA: 78736495992.08.B3C6A7A Received: from out1.migadu.com (out1.migadu.com [91.121.223.63]) by imf21.hostedemail.com (Postfix) with ESMTP id AD928D0425FE for ; Mon, 25 Oct 2021 23:06:32 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1635203194; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jOfu9yayDhzvLAsYLRzMlhcWXOr3HMUMOxXzk71LJi4=; b=JKBY3MYo8v0gSQav25vpBtSiuxqoEwKJxKTWGqMLui4F+75+cQQNtI180Ey1Qr7LdBNVS5 p/eaMrrkla/PTz6zj2sHXRZVgy43DZoFbB5I+WYQDtkHNmjW3EGc5jaZfX8gw2QKr1LB1s NOZzRsppZTWxKU76PalPJRmFIZUeb7U= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Oscar Salvador , Michal Hocko , Ding Hui , Tony Luck , "Aneesh Kumar K.V" , Miaohe Lin , Yang Shi , Peter Xu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 2/4] mm/hwpoison: remove race consideration Date: Tue, 26 Oct 2021 08:05:01 +0900 Message-Id: <20211025230503.2650970-3-naoya.horiguchi@linux.dev> In-Reply-To: <20211025230503.2650970-1-naoya.horiguchi@linux.dev> References: <20211025230503.2650970-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: naoya.horiguchi@linux.dev X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: AD928D0425FE X-Stat-Signature: tcp93utt9yojzttser5xqgffkx7q3a34 Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=JKBY3MYo; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf21.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 91.121.223.63 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev X-HE-Tag: 1635203192-888257 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Now memory_failure() and unpoison_memory() are protected by mf_mutex, so no need to explicitly check races between them. So remove them. Signed-off-by: Naoya Horiguchi --- mm/memory-failure.c | 37 ------------------------------------- 1 file changed, 37 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 97297edfbd8e..a47b741ca04b 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1507,14 +1507,6 @@ static int memory_failure_hugetlb(unsigned long pfn, int flags) lock_page(head); page_flags = head->flags; - if (!PageHWPoison(head)) { - pr_err("Memory failure: %#lx: just unpoisoned\n", pfn); - num_poisoned_pages_dec(); - unlock_page(head); - put_page(head); - return 0; - } - /* * TODO: hwpoison for pud-sized hugetlb doesn't work right now, so * simply disable it. In order to make it work properly, we need @@ -1789,16 +1781,6 @@ int memory_failure(unsigned long pfn, int flags) */ page_flags = p->flags; - /* - * unpoison always clear PG_hwpoison inside page lock - */ - if (!PageHWPoison(p)) { - pr_err("Memory failure: %#lx: just unpoisoned\n", pfn); - num_poisoned_pages_dec(); - unlock_page(p); - put_page(p); - goto unlock_mutex; - } if (hwpoison_filter(p)) { if (TestClearPageHWPoison(p)) num_poisoned_pages_dec(); @@ -2016,17 +1998,6 @@ int unpoison_memory(unsigned long pfn) goto unlock_mutex; } - /* - * unpoison_memory() can encounter thp only when the thp is being - * worked by memory_failure() and the page lock is not held yet. - * In such case, we yield to memory_failure() and make unpoison fail. - */ - if (!PageHuge(page) && PageTransHuge(page)) { - unpoison_pr_info("Unpoison: Memory failure is now running on %#lx\n", - pfn, &unpoison_rs); - goto unlock_mutex; - } - if (!get_hwpoison_page(p, flags)) { if (TestClearPageHWPoison(p)) num_poisoned_pages_dec(); @@ -2035,20 +2006,12 @@ int unpoison_memory(unsigned long pfn) goto unlock_mutex; } - lock_page(page); - /* - * This test is racy because PG_hwpoison is set outside of page lock. - * That's acceptable because that won't trigger kernel panic. Instead, - * the PG_hwpoison page will be caught and isolated on the entrance to - * the free buddy page pool. - */ if (TestClearPageHWPoison(page)) { unpoison_pr_info("Unpoison: Software-unpoisoned page %#lx\n", pfn, &unpoison_rs); num_poisoned_pages_dec(); freeit = 1; } - unlock_page(page); put_page(page); if (freeit && !(pfn == my_zero_pfn(0) && page_count(p) == 1)) From patchwork Mon Oct 25 23:14:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12583477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EAECC433F5 for ; Mon, 25 Oct 2021 23:14:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BA45A61076 for ; Mon, 25 Oct 2021 23:14:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BA45A61076 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 453D4940008; Mon, 25 Oct 2021 19:14:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E83C940007; Mon, 25 Oct 2021 19:14:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2A3B9940008; Mon, 25 Oct 2021 19:14:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0199.hostedemail.com [216.40.44.199]) by kanga.kvack.org (Postfix) with ESMTP id 16A30940007 for ; Mon, 25 Oct 2021 19:14:20 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id B6BED8249980 for ; Mon, 25 Oct 2021 23:14:19 +0000 (UTC) X-FDA: 78736515438.22.6248E07 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf19.hostedemail.com (Postfix) with ESMTP id 11507B0000A3 for ; Mon, 25 Oct 2021 23:14:14 +0000 (UTC) Received: by mail-pj1-f47.google.com with SMTP id lx5-20020a17090b4b0500b001a262880e99so150133pjb.5 for ; Mon, 25 Oct 2021 16:14:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:date:to:cc:subject:message-id:mime-version:content-disposition :in-reply-to; bh=Lh3py47gA8jg6Gb0rOjwQD82lb53Sm8VrPET+7wqweU=; b=oC7s9lfcXvHb8GRvOWEgcGhIyRfEicCKCTCuu+2KC7c3BABC0NHhIBow7E6xLZIUyC 0xsSBuSoyShsveED9PlDgizbLrEAyiULB3490fdPvkS3iDgv1aEHxsl25nxTCwvBbpPA jMjqJo93kja70FwMf6zXL663O2Lh3CdcM70xFd4xNpBxDLVt41lVBxdRU0BEB6LkwD6/ Ll5Bjm1ZtwoD3lpGAcur5UaX9jaAKjAwuJgrE3rcEBd8dAIlY6q28kBLFagPNN6UTO5w w4ruzR2/Afvvyva2nkbLvm3B6gTOojkma9/APlNGsmvOlgZa86E6fzq6Uyg/Ipb9kXx5 i7tQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:date:to:cc:subject:message-id:mime-version :content-disposition:in-reply-to; bh=Lh3py47gA8jg6Gb0rOjwQD82lb53Sm8VrPET+7wqweU=; b=QlUGi2vwAoFctuEJ3Xus3BrJUpJOVEFSR2UXoVmFl3vcH69ZKRgNFI2QrUIZY2vtmj CuZ2E1VhfjyNYIzSuGNxVLRpYinV3WamM5YnUAMVOp6PtWv7Gwi7lkvx7E6nsGljzpZn oM2WZITJBZNv2hFBAs5MMSck47g5C84BVXBd6wH4kiA/xRvi+PuoByx1flcmsDI1HC39 H06/GiJ8bQqMNhY/k8mZXOD99OI4Z4GeuBoFv8RR+UU5BUC6i6wRIjulZmTBunUoumHN FadyD9X1vMkxQqBQ/wgWUXARw9pIBxWplt9hNwRdC2W0nrqHxLlCJLi6m7Kq3ZenR4t0 a6ew== X-Gm-Message-State: AOAM532+4rlblGTxC6aWnrrVoX9B+d+wacixA6ZILI4HRaScxMgBWG8f Xpp4CwXdiwTYMNMC+tMwYLQGToDRsBFC X-Google-Smtp-Source: ABdhPJwQNFIGfOE2VPjqLATShIXQYysjolvs2G1MnC0uCysSRth3EfS/RS0y3Jzr9s2LlVT31xGGyw== X-Received: by 2002:a17:90b:4a07:: with SMTP id kk7mr24128073pjb.37.1635203657849; Mon, 25 Oct 2021 16:14:17 -0700 (PDT) Received: from u2004 ([2407:c800:3f11:740:8ea1:3c5:6c2d:e5c9]) by smtp.gmail.com with ESMTPSA id g5sm12041028pfc.65.2021.10.25.16.14.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Oct 2021 16:14:17 -0700 (PDT) From: Naoya Horiguchi X-Google-Original-From: Naoya Horiguchi Date: Tue, 26 Oct 2021 08:14:13 +0900 To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Oscar Salvador , Michal Hocko , Ding Hui , Tony Luck , "Aneesh Kumar K.V" , Miaohe Lin , Yang Shi , Peter Xu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH 3/4] mm/hwpoison: remove MF_MSG_BUDDY_2ND and MF_MSG_POISONED_HUGE Message-ID: <20211025231120.GA2651146@u2004> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20211025230503.2650970-1-naoya.horiguchi@linux.dev> X-Mutt-References: <20211025230503.2650970-1-naoya.horiguchi@linux.dev> X-Mutt-Fcc: =Sent X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 11507B0000A3 X-Stat-Signature: erdtqcbqrzm4s9wgsjm5opuy35f67onp Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=oC7s9lfc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf19.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com X-HE-Tag: 1635203654-339032 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: (I failed to send patch 3/4 and 4/4 due to the ratelimit of linux.dev, so I switched mail server...) From: Naoya Horiguchi These action_page_types are no longer used, so remove them. Signed-off-by: Naoya Horiguchi Acked-by: Yang Shi --- include/linux/mm.h | 2 -- include/ras/ras_event.h | 2 -- mm/memory-failure.c | 2 -- 3 files changed, 6 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index a3229f609856..71d886470d71 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3246,7 +3246,6 @@ enum mf_action_page_type { MF_MSG_KERNEL_HIGH_ORDER, MF_MSG_SLAB, MF_MSG_DIFFERENT_COMPOUND, - MF_MSG_POISONED_HUGE, MF_MSG_HUGE, MF_MSG_FREE_HUGE, MF_MSG_NON_PMD_HUGE, @@ -3261,7 +3260,6 @@ enum mf_action_page_type { MF_MSG_CLEAN_LRU, MF_MSG_TRUNCATED_LRU, MF_MSG_BUDDY, - MF_MSG_BUDDY_2ND, MF_MSG_DAX, MF_MSG_UNSPLIT_THP, MF_MSG_UNKNOWN, diff --git a/include/ras/ras_event.h b/include/ras/ras_event.h index 0bdbc0d17d2f..d0337a41141c 100644 --- a/include/ras/ras_event.h +++ b/include/ras/ras_event.h @@ -358,7 +358,6 @@ TRACE_EVENT(aer_event, EM ( MF_MSG_KERNEL_HIGH_ORDER, "high-order kernel page" ) \ EM ( MF_MSG_SLAB, "kernel slab page" ) \ EM ( MF_MSG_DIFFERENT_COMPOUND, "different compound page after locking" ) \ - EM ( MF_MSG_POISONED_HUGE, "huge page already hardware poisoned" ) \ EM ( MF_MSG_HUGE, "huge page" ) \ EM ( MF_MSG_FREE_HUGE, "free huge page" ) \ EM ( MF_MSG_NON_PMD_HUGE, "non-pmd-sized huge page" ) \ @@ -373,7 +372,6 @@ TRACE_EVENT(aer_event, EM ( MF_MSG_CLEAN_LRU, "clean LRU page" ) \ EM ( MF_MSG_TRUNCATED_LRU, "already truncated LRU page" ) \ EM ( MF_MSG_BUDDY, "free buddy page" ) \ - EM ( MF_MSG_BUDDY_2ND, "free buddy page (2nd try)" ) \ EM ( MF_MSG_DAX, "dax page" ) \ EM ( MF_MSG_UNSPLIT_THP, "unsplit thp" ) \ EMe ( MF_MSG_UNKNOWN, "unknown page" ) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index a47b741ca04b..09f079987928 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -723,7 +723,6 @@ static const char * const action_page_types[] = { [MF_MSG_KERNEL_HIGH_ORDER] = "high-order kernel page", [MF_MSG_SLAB] = "kernel slab page", [MF_MSG_DIFFERENT_COMPOUND] = "different compound page after locking", - [MF_MSG_POISONED_HUGE] = "huge page already hardware poisoned", [MF_MSG_HUGE] = "huge page", [MF_MSG_FREE_HUGE] = "free huge page", [MF_MSG_NON_PMD_HUGE] = "non-pmd-sized huge page", @@ -738,7 +737,6 @@ static const char * const action_page_types[] = { [MF_MSG_CLEAN_LRU] = "clean LRU page", [MF_MSG_TRUNCATED_LRU] = "already truncated LRU page", [MF_MSG_BUDDY] = "free buddy page", - [MF_MSG_BUDDY_2ND] = "free buddy page (2nd try)", [MF_MSG_DAX] = "dax page", [MF_MSG_UNSPLIT_THP] = "unsplit thp", [MF_MSG_UNKNOWN] = "unknown page", From patchwork Mon Oct 25 23:16:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12583479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D40EC433F5 for ; Mon, 25 Oct 2021 23:16:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2562661076 for ; Mon, 25 Oct 2021 23:16:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2562661076 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id A6B1B8000A; Mon, 25 Oct 2021 19:16:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A1A1C940007; Mon, 25 Oct 2021 19:16:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 909928000A; Mon, 25 Oct 2021 19:16:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0254.hostedemail.com [216.40.44.254]) by kanga.kvack.org (Postfix) with ESMTP id 7D958940007 for ; Mon, 25 Oct 2021 19:16:24 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 35B102D397 for ; Mon, 25 Oct 2021 23:16:24 +0000 (UTC) X-FDA: 78736520688.16.22F7685 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by imf02.hostedemail.com (Postfix) with ESMTP id 946D07001A08 for ; Mon, 25 Oct 2021 23:16:20 +0000 (UTC) Received: by mail-pl1-f174.google.com with SMTP id n11so8976554plf.4 for ; Mon, 25 Oct 2021 16:16:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:date:to:cc:subject:message-id:mime-version:content-disposition :in-reply-to; bh=/bKrmeIchLWd2UwSF/qUuONddfUmW97o7hXpLTBF4jA=; b=XDb8wea8TT9IXEqxNxrm+O7D4EEBiO38CiHzTeqMegRj+XFxSWCGNx/nDZoHlhgu7W ZeDWHXrMjcydLfD7r/uij4fpb20MqCzYElEaM8Xl6GWRNbd5tcK0vmRFdsqNOW71EXB5 Va483TTEW/ER29LCnMcm9rMJUvgnw6xP+qAhMxXY6/tbBROIthV8lPHcp50gmkWjkESZ uBeLBWn1y7vBbs2NYGhhxpQYSes86oJSYt4pedBo1XvGXxjuU/lL9P4xXBRB+4XfPy6X 8yBykxy6lTSahjvHX/g53f7S5QAaZeCoIBlH5I0Td6pXLaoWRzDmOKI/oapdvO3yqYAo 69Mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:date:to:cc:subject:message-id:mime-version :content-disposition:in-reply-to; bh=/bKrmeIchLWd2UwSF/qUuONddfUmW97o7hXpLTBF4jA=; b=tf+tguKN535xVvt4Dmj71mM48JAjByjpcqt02kVKgIqn+pfjVKVcy2fZ6W3vPTuqW3 yeD+lnWBPEcGi5QOIWuJGfLLOkFCnxGeYRcMuAGvvLrr2xUE59cp52Hxlgm6vBLW2Req dmReAbomG+moxsy0UnPMOU7udd5k/9iXUH0ukb1HcDn01aIzYfP2MtJMMD3JGecvXcsm O780+lHMW8eU88ryAUNaDF5SvdH5ZjtFLsaMlwvnGVwfdMVjasF5FUsgft041sDO0Lcq R80skoY5p/y3mURtU6ZDmcsov9yVhRIsfU7A7DMYGCLrjIDuzsdIaliN6q8JdcQ45k1Y 8cHQ== X-Gm-Message-State: AOAM530T5ZF2a+QfkCGU+iVQsyzz1XObwcXgSsKCfGIkDJ1OPUXu5rMd rB/BrIhtTJEw+yzu7JMFHceC2bbSP6KP X-Google-Smtp-Source: ABdhPJzRMlpAXV8LLTGGWR9u86qn4lewfa5UEy1w+YcnMmMTZXj6r/lUs7pjikWTR0chAovLk7o48A== X-Received: by 2002:a17:90a:4283:: with SMTP id p3mr18813750pjg.23.1635203782587; Mon, 25 Oct 2021 16:16:22 -0700 (PDT) Received: from u2004 ([2407:c800:3f11:740:8ea1:3c5:6c2d:e5c9]) by smtp.gmail.com with ESMTPSA id gq15sm8069524pjb.42.2021.10.25.16.16.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Oct 2021 16:16:22 -0700 (PDT) From: Naoya Horiguchi X-Google-Original-From: Naoya Horiguchi Date: Tue, 26 Oct 2021 08:16:18 +0900 To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Oscar Salvador , Michal Hocko , Ding Hui , Tony Luck , "Aneesh Kumar K.V" , Miaohe Lin , Yang Shi , Peter Xu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 4/4] mm/hwpoison: fix unpoison_memory() Message-ID: <20211025231618.GA2651507@u2004> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20211025230503.2650970-1-naoya.horiguchi@linux.dev> X-Mutt-References: <20211025230503.2650970-1-naoya.horiguchi@linux.dev> X-Mutt-Fcc: =Sent X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 946D07001A08 X-Stat-Signature: dxjzr6cwjih6spt4zs6twgtm4wxu3fju Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=XDb8wea8; spf=pass (imf02.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.214.174 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1635203780-172903 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi After recent soft-offline rework, error pages can be taken off from buddy allocator, but the existing unpoison_memory() does not properly undo the operation. Moreover, due to the recent change on __get_hwpoison_page(), get_page_unless_zero() is hardly called for hwpoisoned pages. So __get_hwpoison_page() highly likely returns zero (meaning to fail to grab page refcount) and unpoison just clears PG_hwpoison without releasing a refcount. That does not lead to a critical issue like kernel panic, but unpoisoned pages never get back to buddy (leaked permanently), which is not good. To (partially) fix this, we need to identify "taken off" pages from other types of hwpoisoned pages. We can't use refcount or page flags for this purpose, so a pseudo flag is defined by hacking ->private field. Someone might think that put_page() is enough to cancel taken-off pages, but the normal free path contains some operations not suitable for the current purpose, and can fire VM_BUG_ON(). Note that unpoison_memory() is now supposed to be cancel hwpoison events injected only by madvise() or /sys/devices/system/memory/{hard,soft}_offline_page, not by MCE injection, so please don't try to use unpoison when testing with MCE injection. Signed-off-by: Naoya Horiguchi --- ChangeLog v2: - unpoison_memory() returns as commented - explicitly avoids unpoisoning slab pages - separates internal pinning function into __get_unpoison_page() --- include/linux/mm.h | 1 + include/linux/page-flags.h | 4 ++ mm/memory-failure.c | 104 ++++++++++++++++++++++++++++++------- mm/page_alloc.c | 23 ++++++++ 4 files changed, 113 insertions(+), 19 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 71d886470d71..c7ad3fdfee7c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3219,6 +3219,7 @@ enum mf_flags { MF_ACTION_REQUIRED = 1 << 1, MF_MUST_KILL = 1 << 2, MF_SOFT_OFFLINE = 1 << 3, + MF_UNPOISON = 1 << 4, }; extern int memory_failure(unsigned long pfn, int flags); extern void memory_failure_queue(unsigned long pfn, int flags); diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index b78f137acc62..8add006535f6 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -522,7 +522,11 @@ PAGEFLAG_FALSE(Uncached, uncached) PAGEFLAG(HWPoison, hwpoison, PF_ANY) TESTSCFLAG(HWPoison, hwpoison, PF_ANY) #define __PG_HWPOISON (1UL << PG_hwpoison) +#define MAGIC_HWPOISON 0x4857504f49534f4e +extern void SetPageHWPoisonTakenOff(struct page *page); +extern void ClearPageHWPoisonTakenOff(struct page *page); extern bool take_page_off_buddy(struct page *page); +extern bool take_page_back_buddy(struct page *page); #else PAGEFLAG_FALSE(HWPoison, hwpoison) #define __PG_HWPOISON 0 diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 09f079987928..a6f80a670012 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1160,6 +1160,22 @@ static int page_action(struct page_state *ps, struct page *p, return (result == MF_RECOVERED || result == MF_DELAYED) ? 0 : -EBUSY; } +static inline bool PageHWPoisonTakenOff(struct page *page) +{ + return PageHWPoison(page) && page_private(page) == MAGIC_HWPOISON; +} + +void SetPageHWPoisonTakenOff(struct page *page) +{ + set_page_private(page, MAGIC_HWPOISON); +} + +void ClearPageHWPoisonTakenOff(struct page *page) +{ + if (PageHWPoison(page)) + set_page_private(page, 0); +} + /* * Return true if a page type of a given page is supported by hwpoison * mechanism (while handling could fail), otherwise false. This function @@ -1262,6 +1278,27 @@ static int get_any_page(struct page *p, unsigned long flags) return ret; } +static int __get_unpoison_page(struct page *page) +{ + struct page *head = compound_head(page); + int ret = 0; + bool hugetlb = false; + + ret = get_hwpoison_huge_page(head, &hugetlb); + if (hugetlb) + return ret; + + /* + * PageHWPoisonTakenOff pages are not only marked as PG_hwpoison, + * but also isolated from buddy freelist, so need to identify the + * state and have to cancel both operations to unpoison. + */ + if (PageHWPoisonTakenOff(head)) + return -EHWPOISON; + + return get_page_unless_zero(head) ? 1 : 0; +} + /** * get_hwpoison_page() - Get refcount for memory error handling * @p: Raw error page (hit by memory error) @@ -1278,18 +1315,26 @@ static int get_any_page(struct page *p, unsigned long flags) * extra care for the error page's state (as done in __get_hwpoison_page()), * and has some retry logic in get_any_page(). * + * When called from unpoison_memory(), the caller should already ensure that + * the given page has PG_hwpoison. So it's never reused for other page + * allocations, and __get_unpoison_page() never races with them. + * * Return: 0 on failure, * 1 on success for in-use pages in a well-defined state, * -EIO for pages on which we can not handle memory errors, * -EBUSY when get_hwpoison_page() has raced with page lifecycle - * operations like allocation and free. + * operations like allocation and free, + * -EHWPOISON when the page is hwpoisoned and taken off from buddy. */ static int get_hwpoison_page(struct page *p, unsigned long flags) { int ret; zone_pcp_disable(page_zone(p)); - ret = get_any_page(p, flags); + if (flags & MF_UNPOISON) + ret = __get_unpoison_page(p); + else + ret = get_any_page(p, flags); zone_pcp_enable(page_zone(p)); return ret; @@ -1942,6 +1987,26 @@ core_initcall(memory_failure_init); pr_info(fmt, pfn); \ }) +static inline int clear_page_hwpoison(struct ratelimit_state *rs, struct page *p) +{ + if (TestClearPageHWPoison(p)) { + unpoison_pr_info("Unpoison: Software-unpoisoned page %#lx\n", + page_to_pfn(p), rs); + num_poisoned_pages_dec(); + return 0; + } + return -EBUSY; +} + +static inline int unpoison_taken_off_page(struct ratelimit_state *rs, + struct page *p) +{ + if (take_page_back_buddy(p) && !clear_page_hwpoison(rs, p)) + return 0; + else + return -EBUSY; +} + /** * unpoison_memory - Unpoison a previously poisoned page * @pfn: Page number of the to be unpoisoned page @@ -1958,9 +2023,7 @@ int unpoison_memory(unsigned long pfn) { struct page *page; struct page *p; - int freeit = 0; - int ret = 0; - unsigned long flags = 0; + int ret = -EBUSY; static DEFINE_RATELIMIT_STATE(unpoison_rs, DEFAULT_RATELIMIT_INTERVAL, DEFAULT_RATELIMIT_BURST); @@ -1996,24 +2059,27 @@ int unpoison_memory(unsigned long pfn) goto unlock_mutex; } - if (!get_hwpoison_page(p, flags)) { - if (TestClearPageHWPoison(p)) - num_poisoned_pages_dec(); - unpoison_pr_info("Unpoison: Software-unpoisoned free page %#lx\n", - pfn, &unpoison_rs); + if (PageSlab(page)) goto unlock_mutex; - } - if (TestClearPageHWPoison(page)) { - unpoison_pr_info("Unpoison: Software-unpoisoned page %#lx\n", - pfn, &unpoison_rs); - num_poisoned_pages_dec(); - freeit = 1; - } + ret = get_hwpoison_page(p, MF_UNPOISON); + if (!ret) { + ret = clear_page_hwpoison(&unpoison_rs, p); + } else if (ret < 0) { + if (ret == -EHWPOISON) { + ret = unpoison_taken_off_page(&unpoison_rs, p); + } else + unpoison_pr_info("Unpoison: failed to grab page %#lx\n", + pfn, &unpoison_rs); + } else { + int freeit = clear_page_hwpoison(&unpoison_rs, p); - put_page(page); - if (freeit && !(pfn == my_zero_pfn(0) && page_count(p) == 1)) put_page(page); + if (freeit && !(pfn == my_zero_pfn(0) && page_count(p) == 1)) { + put_page(page); + ret = 0; + } + } unlock_mutex: mutex_unlock(&mf_mutex); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 4ea590646f89..b6e4cbb44c54 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -9466,6 +9466,7 @@ bool take_page_off_buddy(struct page *page) del_page_from_free_list(page_head, zone, page_order); break_down_buddy_pages(zone, page_head, page, 0, page_order, migratetype); + SetPageHWPoisonTakenOff(page); if (!is_migrate_isolate(migratetype)) __mod_zone_freepage_state(zone, -1, migratetype); ret = true; @@ -9477,4 +9478,26 @@ bool take_page_off_buddy(struct page *page) spin_unlock_irqrestore(&zone->lock, flags); return ret; } + +/* + * Cancel takeoff done by take_page_off_buddy(). + */ +bool take_page_back_buddy(struct page *page) +{ + struct zone *zone = page_zone(page); + unsigned long pfn = page_to_pfn(page); + unsigned long flags; + int migratetype = get_pfnblock_migratetype(page, pfn); + bool ret = false; + + spin_lock_irqsave(&zone->lock, flags); + if (put_page_testzero(page)) { + ClearPageHWPoisonTakenOff(page); + __free_one_page(page, pfn, zone, 0, migratetype, FPI_NONE); + ret = true; + } + spin_unlock_irqrestore(&zone->lock, flags); + + return ret; +} #endif