From patchwork Fri Jul 8 05:36:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12910583 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C3BCC433EF for ; Fri, 8 Jul 2022 05:37:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 926A06B0073; Fri, 8 Jul 2022 01:37:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8AD726B0074; Fri, 8 Jul 2022 01:37:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 74EB96B0075; Fri, 8 Jul 2022 01:37:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 624A96B0073 for ; Fri, 8 Jul 2022 01:37:40 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0C5CC20F63 for ; Fri, 8 Jul 2022 05:37:40 +0000 (UTC) X-FDA: 79662825480.01.1572A3D Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by imf12.hostedemail.com (Postfix) with ESMTP id A11D840027 for ; Fri, 8 Jul 2022 05:37:39 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1657258658; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/78PY3uv4dMYi2rgsK50O1qa1/bVeZOW9OatOK9YQlA=; b=USG5qXuejk9c625UJ+AOO6x6ZUGQNRdXqP9XY15lxUlFeVZU/Elll6MVQOM8ij7G9r1HML xl9GdvUSa9Qkoay9aStnwx+ApYy48fppVIlNnxlj542vvAHorR7eknz0oosdqSyqscJSlD GivdmWJgTrU1Cqhswfad9lOApa+EX7Y= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [mm-unstable PATCH v5 1/8] mm/hugetlb: check gigantic_page_runtime_supported() in return_unused_surplus_pages() Date: Fri, 8 Jul 2022 14:36:46 +0900 Message-Id: <20220708053653.964464-2-naoya.horiguchi@linux.dev> In-Reply-To: <20220708053653.964464-1-naoya.horiguchi@linux.dev> References: <20220708053653.964464-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657258659; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/78PY3uv4dMYi2rgsK50O1qa1/bVeZOW9OatOK9YQlA=; b=EY68oxJj8vm3v+zOMAB+vgZhYThpMYVmSwUfwMSnF7cI15OC/7Vb9G2FeSMHAsN8QXe8bG 49n7LaF5G0+RCWe4qZME2zjV3w5aYHFOf/5wQ/y5KWxO2w4G/IR3DUUXr34BrDo6leSl/b BqxVidegCLtKsUdu587Omsyy8pUBOkU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657258659; a=rsa-sha256; cv=none; b=knBqZ8xezrcIM/YiYMbvQVVA3MYXeJ9IwTsj97UaiNw5NpwZXOG4Z+V/RSvgIeKsAcHsFe uy/4d6RMHTAVL9EuX3lOihUYu87ayf+MIujS2UlGU1oifncd/lNA4zLtclsGh1UnAm184G opp3ax1iiXhcGoXceMK1gXefaEKyfY8= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=USG5qXue; spf=pass (imf12.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Rspamd-Queue-Id: A11D840027 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=USG5qXue; spf=pass (imf12.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Rspam-User: X-Rspamd-Server: rspam06 X-Stat-Signature: mbb86fp8umq3sd57weeqsfpmz74kuhx3 X-HE-Tag: 1657258659-124578 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi I found a weird state of 1GB hugepage pool, caused by the following procedure: - run a process reserving all free 1GB hugepages, - shrink free 1GB hugepage pool to zero (i.e. writing 0 to /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages), then - kill the reserving process. , then all the hugepages are free *and* surplus at the same time. $ cat /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages 3 $ cat /sys/kernel/mm/hugepages/hugepages-1048576kB/free_hugepages 3 $ cat /sys/kernel/mm/hugepages/hugepages-1048576kB/resv_hugepages 0 $ cat /sys/kernel/mm/hugepages/hugepages-1048576kB/surplus_hugepages 3 This state is resolved by reserving and allocating the pages then freeing them again, so this seems not to result in serious problem. But it's a little surprising (shrinking pool suddenly fails). This behavior is caused by hstate_is_gigantic() check in return_unused_surplus_pages(). This was introduced so long ago in 2008 by commit aa888a74977a ("hugetlb: support larger than MAX_ORDER"), and at that time the gigantic pages were not supposed to be allocated/freed at run-time. Now kernel can support runtime allocation/free, so let's check gigantic_page_runtime_supported() together. Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin --- v4 -> v5: - drop additional gigantic_page_runtime_supported() checks. v2 -> v3: - Fixed typo in patch description, - add !gigantic_page_runtime_supported() check instead of removing hstate_is_gigantic() check (suggested by Miaohe and Muchun) - add a few more !gigantic_page_runtime_supported() check in set_max_huge_pages() (by Mike). --- mm/hugetlb.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a4506ed1f1db..cf8ccee7654c 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2432,8 +2432,7 @@ static void return_unused_surplus_pages(struct hstate *h, /* Uncommit the reservation */ h->resv_huge_pages -= unused_resv_pages; - /* Cannot return gigantic pages currently */ - if (hstate_is_gigantic(h)) + if (hstate_is_gigantic(h) && !gigantic_page_runtime_supported()) goto out; /* From patchwork Fri Jul 8 05:36:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12910584 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90443C43334 for ; Fri, 8 Jul 2022 05:37:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2BDA76B0074; Fri, 8 Jul 2022 01:37:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 244AE6B0075; Fri, 8 Jul 2022 01:37:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0BE5A900002; Fri, 8 Jul 2022 01:37:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id EF3076B0074 for ; Fri, 8 Jul 2022 01:37:44 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id BDF8734C99 for ; Fri, 8 Jul 2022 05:37:44 +0000 (UTC) X-FDA: 79662825648.12.1FDBDCD Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by imf25.hostedemail.com (Postfix) with ESMTP id 1C3CCA0031 for ; Fri, 8 Jul 2022 05:37:43 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1657258662; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uxYP30zRX3+087a7ADNvtSYGr7OYdBOK6+qjv5apzug=; b=QmNYSzR8WxPgvEHooLZcizvFyZgAoinmGHv6QYOIerCWy+kH2jeZvEKTg+JreBmUeK+29J qa49GAc/9y1cLVpURCTvHcjQxSsn0b5f1rC/YVXJv5L9INjsnoKfgmZT/iJT4k3nkFrAL9 bLYw8aaYAFzt9LBjxNgBsejCWUl8lGA= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [mm-unstable PATCH v5 2/8] mm/hugetlb: make pud_huge() and follow_huge_pud() aware of non-present pud entry Date: Fri, 8 Jul 2022 14:36:47 +0900 Message-Id: <20220708053653.964464-3-naoya.horiguchi@linux.dev> In-Reply-To: <20220708053653.964464-1-naoya.horiguchi@linux.dev> References: <20220708053653.964464-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657258664; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uxYP30zRX3+087a7ADNvtSYGr7OYdBOK6+qjv5apzug=; b=SA4aA/CB8eB0/F9Bj9qrdnQQYI7D0/whkZx7aE1R0MO1dbE6FdrDryPuNvfgKQqFJeCIDk OR+99ij/hSNKM9vVT/t/Y8EEShF6dtSTxzpZyGoILsQQQFAlAyaePFCrZok0Gqea3WbE4Z VpK13D1W6WVt1JXNN29TMm4ghklFzYo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657258664; a=rsa-sha256; cv=none; b=kb3jP8EVPcDB1KkYYY7PuuuzgFrZwlNcKjJSL243ET4dYU5t/h+SncIew1ICNSYGbVKYQV shxOHQZv2YCq9Anabsj7HfU3Vl3wy6+kFnaEQ0tCeRKhZgoE2EL0FjiREMl9UJbNX32cO7 Bn8K6U4aQTlc/ISC7sgHnHZ7g5nyEY4= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=QmNYSzR8; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf25.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=QmNYSzR8; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf25.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 1C3CCA0031 X-Stat-Signature: n3khwfa3gshxtbzhg5yrourpyawfqqhg X-Rspam-User: X-HE-Tag: 1657258663-305146 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi follow_pud_mask() does not support non-present pud entry now. As long as I tested on x86_64 server, follow_pud_mask() still simply returns no_page_table() for non-present_pud_entry() due to pud_bad(), so no severe user-visible effect should happen. But generally we should call follow_huge_pud() for non-present pud entry for 1GB hugetlb page. Update pud_huge() and follow_huge_pud() to handle non-present pud entries. The changes are similar to previous works for pud entries commit e66f17ff7177 ("mm/hugetlb: take page table lock in follow_huge_pmd()") and commit cbef8478bee5 ("mm/hugetlb: pmd_huge() returns true for non-present hugepage"). Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin Reviewed-by: Mike Kravetz --- v2 -> v3: - fixed typos in subject and description, - added comment on pud_huge(), - added comment about fallback for hwpoisoned entry, - updated initial check about FOLL_{PIN,GET} flags. --- arch/x86/mm/hugetlbpage.c | 8 +++++++- mm/hugetlb.c | 32 ++++++++++++++++++++++++++++++-- 2 files changed, 37 insertions(+), 3 deletions(-) diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c index 509408da0da1..6b3033845c6d 100644 --- a/arch/x86/mm/hugetlbpage.c +++ b/arch/x86/mm/hugetlbpage.c @@ -30,9 +30,15 @@ int pmd_huge(pmd_t pmd) (pmd_val(pmd) & (_PAGE_PRESENT|_PAGE_PSE)) != _PAGE_PRESENT; } +/* + * pud_huge() returns 1 if @pud is hugetlb related entry, that is normal + * hugetlb entry or non-present (migration or hwpoisoned) hugetlb entry. + * Otherwise, returns 0. + */ int pud_huge(pud_t pud) { - return !!(pud_val(pud) & _PAGE_PSE); + return !pud_none(pud) && + (pud_val(pud) & (_PAGE_PRESENT|_PAGE_PSE)) != _PAGE_PRESENT; } #ifdef CONFIG_HUGETLB_PAGE diff --git a/mm/hugetlb.c b/mm/hugetlb.c index cf8ccee7654c..77119d93a0f9 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6978,10 +6978,38 @@ struct page * __weak follow_huge_pud(struct mm_struct *mm, unsigned long address, pud_t *pud, int flags) { - if (flags & (FOLL_GET | FOLL_PIN)) + struct page *page = NULL; + spinlock_t *ptl; + pte_t pte; + + if (WARN_ON_ONCE(flags & FOLL_PIN)) return NULL; - return pte_page(*(pte_t *)pud) + ((address & ~PUD_MASK) >> PAGE_SHIFT); +retry: + ptl = huge_pte_lock(hstate_sizelog(PUD_SHIFT), mm, (pte_t *)pud); + if (!pud_huge(*pud)) + goto out; + pte = huge_ptep_get((pte_t *)pud); + if (pte_present(pte)) { + page = pud_page(*pud) + ((address & ~PUD_MASK) >> PAGE_SHIFT); + if (WARN_ON_ONCE(!try_grab_page(page, flags))) { + page = NULL; + goto out; + } + } else { + if (is_hugetlb_entry_migration(pte)) { + spin_unlock(ptl); + __migration_entry_wait(mm, (pte_t *)pud, ptl); + goto retry; + } + /* + * hwpoisoned entry is treated as no_page_table in + * follow_page_mask(). + */ + } +out: + spin_unlock(ptl); + return page; } struct page * __weak From patchwork Fri Jul 8 05:36:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12910585 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66FD6C43334 for ; Fri, 8 Jul 2022 05:37:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F3F9F6B0075; Fri, 8 Jul 2022 01:37:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EC8126B0078; Fri, 8 Jul 2022 01:37:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D69E8900002; Fri, 8 Jul 2022 01:37:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BEFE56B0075 for ; Fri, 8 Jul 2022 01:37:48 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 988E7120EA3 for ; Fri, 8 Jul 2022 05:37:48 +0000 (UTC) X-FDA: 79662825816.25.CCC6934 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by imf20.hostedemail.com (Postfix) with ESMTP id D19731C000D for ; Fri, 8 Jul 2022 05:37:47 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1657258666; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NJ+mDOaakngpz/4x65+36aEKJU8vDwXxJrIKxF7JY2E=; b=hVfsXFwZX9sY5iXMt0UCzhNj3goVBQplaROQXV/2VWwHnVSzu071sWwpGbNhtojUfxLPnT n3l7ceCLd5XRJBD5KaWIyiXmYz3tBQZTOnHlNvKcWRP+rsswHGNzMnYpbFQXkvsg4oydh0 FsmpUz220wtc9yRCOdkPhyjIHlUrdrE= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [mm-unstable PATCH v5 3/8] mm, hwpoison, hugetlb: support saving mechanism of raw error pages Date: Fri, 8 Jul 2022 14:36:48 +0900 Message-Id: <20220708053653.964464-4-naoya.horiguchi@linux.dev> In-Reply-To: <20220708053653.964464-1-naoya.horiguchi@linux.dev> References: <20220708053653.964464-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657258668; a=rsa-sha256; cv=none; b=cY2kckdozd8Ry8C0HwVKVErqdt12oOA8o0QKm5wUtu+ZVr4P+r3Ih7DIvS0X3u8VfMqzm2 2v4xqpIAZbB6L/fP0fJfO5rOXTxGAq+zG2Eh+jb3EWfG+0DcL+RKTwhyYrVc8SZHvQ5adw hWomMkqbmR+NEj2rzyE0ig9gQl69LTM= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=hVfsXFwZ; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf20.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657258668; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NJ+mDOaakngpz/4x65+36aEKJU8vDwXxJrIKxF7JY2E=; b=ak4mn4doqTASmGHmU0u/aHA6WgGeR6E+/sxZvoJix8WI6R6EHFOgIF0ARd1YLHqRC1oqVB Pd5K26zQ8x4BSYhSPz5zrXbBlhC1LwJ3YMmw9DX0Y6u2AU154VvFuFHslaufvsB16OoliL VHZSZWfDswJLhsL957Vnjo/tpJsvF/E= X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: D19731C000D Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=hVfsXFwZ; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf20.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev X-Stat-Signature: o166opnbm781aw5gsrayy69fnwg6kjce X-Rspam-User: X-HE-Tag: 1657258667-199422 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi When handling memory error on a hugetlb page, the error handler tries to dissolve and turn it into 4kB pages. If it's successfully dissolved, PageHWPoison flag is moved to the raw error page, so that's all right. However, dissolve sometimes fails, then the error page is left as hwpoisoned hugepage. It's useful if we can retry to dissolve it to save healthy pages, but that's not possible now because the information about where the raw error pages is lost. Use the private field of a few tail pages to keep that information. The code path of shrinking hugepage pool uses this info to try delayed dissolve. In order to remember multiple errors in a hugepage, a singly-linked list originated from SUBPAGE_INDEX_HWPOISON-th tail page is constructed. Only simple operations (adding an entry or clearing all) are required and the list is assumed not to be very long, so this simple data structure should be enough. If we failed to save raw error info, the hwpoison hugepage has errors on unknown subpage, then this new saving mechanism does not work any more, so disable saving new raw error info and freeing hwpoison hugepages. Signed-off-by: Naoya Horiguchi Reported-by: kernel test robot --- v4 -> v5: - fixed build error (reported by kernel test robot). - do not try to undo remove_hugetlb_page() when HPageRawHwpUnreliable is true, - check HPageRawHwpUnreliable() before hugetlb_vmemmap_restore(), - call num_poisoned_pages_inc() in hugetlb_set_page_hwpoison() when kalloc succeeds, - remove "inline" in the definition of hugetlb_clear_page_hwpoison(). v3 -> v4: - resolve conflict with "mm: hugetlb_vmemmap: improve hugetlb_vmemmap code readability", use hugetlb_vmemmap_restore() instead of hugetlb_vmemmap_alloc(). v2 -> v3: - remove duplicate "return ret" lines, - use GFP_ATOMIC instead of GFP_KERNEL, - introduce HPageRawHwpUnreliable pseudo flag (suggested by Muchun), - hugetlb_clear_page_hwpoison removes raw_hwp_page list even if HPageRawHwpUnreliable is true, (by Miaohe) v1 -> v2: - support hwpoison hugepage with multiple errors, - moved the new interface functions to mm/memory-failure.c, - define additional subpage index SUBPAGE_INDEX_HWPOISON_UNRELIABLE, - stop freeing/dissolving hwpoison hugepages with unreliable raw error info, - drop hugetlb_clear_page_hwpoison() in dissolve_free_huge_page() because that's done in update_and_free_page(), - move setting/clearing PG_hwpoison flag to the new interfaces, - checking already hwpoisoned or not on a subpage basis. ChangeLog since previous post on 4/27: - fixed typo in patch description (by Miaohe) - fixed config value in #ifdef statement (by Miaohe) - added sentences about "multiple hwpoison pages" scenario in patch description --- include/linux/hugetlb.h | 18 +++++++++- mm/hugetlb.c | 32 ++++++++++++----- mm/memory-failure.c | 79 +++++++++++++++++++++++++++++++++++++++-- 3 files changed, 116 insertions(+), 13 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 6d0620edf0a6..6fd128b80d57 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -42,6 +42,9 @@ enum { SUBPAGE_INDEX_CGROUP, /* reuse page->private */ SUBPAGE_INDEX_CGROUP_RSVD, /* reuse page->private */ __MAX_CGROUP_SUBPAGE_INDEX = SUBPAGE_INDEX_CGROUP_RSVD, +#endif +#ifdef CONFIG_MEMORY_FAILURE + SUBPAGE_INDEX_HWPOISON, #endif __NR_USED_SUBPAGE, }; @@ -551,7 +554,7 @@ generic_hugetlb_get_unmapped_area(struct file *file, unsigned long addr, * Synchronization: Initially set after new page allocation with no * locking. When examined and modified during migration processing * (isolate, migrate, putback) the hugetlb_lock is held. - * HPG_temporary - - Set on a page that is temporarily allocated from the buddy + * HPG_temporary - Set on a page that is temporarily allocated from the buddy * allocator. Typically used for migration target pages when no pages * are available in the pool. The hugetlb free page path will * immediately free pages with this flag set to the buddy allocator. @@ -561,6 +564,8 @@ generic_hugetlb_get_unmapped_area(struct file *file, unsigned long addr, * HPG_freed - Set when page is on the free lists. * Synchronization: hugetlb_lock held for examination and modification. * HPG_vmemmap_optimized - Set when the vmemmap pages of the page are freed. + * HPG_raw_hwp_unreliable - Set when the hugetlb page has a hwpoison sub-page + * that is not tracked by raw_hwp_page list. */ enum hugetlb_page_flags { HPG_restore_reserve = 0, @@ -568,6 +573,7 @@ enum hugetlb_page_flags { HPG_temporary, HPG_freed, HPG_vmemmap_optimized, + HPG_raw_hwp_unreliable, __NR_HPAGEFLAGS, }; @@ -614,6 +620,7 @@ HPAGEFLAG(Migratable, migratable) HPAGEFLAG(Temporary, temporary) HPAGEFLAG(Freed, freed) HPAGEFLAG(VmemmapOptimized, vmemmap_optimized) +HPAGEFLAG(RawHwpUnreliable, raw_hwp_unreliable) #ifdef CONFIG_HUGETLB_PAGE @@ -796,6 +803,15 @@ extern int dissolve_free_huge_page(struct page *page); extern int dissolve_free_huge_pages(unsigned long start_pfn, unsigned long end_pfn); +#ifdef CONFIG_MEMORY_FAILURE +extern int hugetlb_clear_page_hwpoison(struct page *hpage); +#else +static inline int hugetlb_clear_page_hwpoison(struct page *hpage) +{ + return 0; +} +#endif + #ifdef CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION #ifndef arch_hugetlb_migration_supported static inline bool arch_hugetlb_migration_supported(struct hstate *h) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 77119d93a0f9..3956494cc5fb 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1442,6 +1442,15 @@ static void __remove_hugetlb_page(struct hstate *h, struct page *page, h->surplus_huge_pages_node[nid]--; } + /* + * This leaves HPageRawHwpUnreliable pages as leaked hugepages, not + * as leaked generic-compound pages. Otherwise page_mapped() or + * folio_mapped() gets slow because for-loop for each subpage is + * called. + */ + if (HPageRawHwpUnreliable(page)) + return; + /* * Very subtle * @@ -1535,6 +1544,13 @@ static void __update_and_free_page(struct hstate *h, struct page *page) if (hstate_is_gigantic(h) && !gigantic_page_runtime_supported()) return; + /* + * If we don't know which subpages are hwpoisoned, we can't free + * the hugepage, so it's leaked intentionally. + */ + if (HPageRawHwpUnreliable(page)) + return; + if (hugetlb_vmemmap_restore(h, page)) { spin_lock_irq(&hugetlb_lock); /* @@ -1547,6 +1563,13 @@ static void __update_and_free_page(struct hstate *h, struct page *page) return; } + /* + * Move PageHWPoison flag from head page to the raw error pages, + * which makes any healthy subpages reusable. + */ + if (unlikely(PageHWPoison(page))) + hugetlb_clear_page_hwpoison(page); + for (i = 0; i < pages_per_huge_page(h); i++, subpage = mem_map_next(subpage, page, i)) { subpage->flags &= ~(1 << PG_locked | 1 << PG_error | @@ -2109,15 +2132,6 @@ int dissolve_free_huge_page(struct page *page) */ rc = hugetlb_vmemmap_restore(h, head); if (!rc) { - /* - * Move PageHWPoison flag from head page to the raw - * error page, which makes any subpages rather than - * the error page reusable. - */ - if (PageHWPoison(head) && page != head) { - SetPageHWPoison(page); - ClearPageHWPoison(head); - } update_and_free_page(h, head, false); } else { spin_lock_irq(&hugetlb_lock); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index c9931c676335..6833c5e4b410 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1664,6 +1664,80 @@ int mf_dax_kill_procs(struct address_space *mapping, pgoff_t index, EXPORT_SYMBOL_GPL(mf_dax_kill_procs); #endif /* CONFIG_FS_DAX */ +#ifdef CONFIG_HUGETLB_PAGE +/* + * Struct raw_hwp_page represents information about "raw error page", + * constructing singly linked list originated from ->private field of + * SUBPAGE_INDEX_HWPOISON-th tail page. + */ +struct raw_hwp_page { + struct llist_node node; + struct page *page; +}; + +static inline struct llist_head *raw_hwp_list_head(struct page *hpage) +{ + return (struct llist_head *)&page_private(hpage + SUBPAGE_INDEX_HWPOISON); +} + +static int hugetlb_set_page_hwpoison(struct page *hpage, struct page *page) +{ + struct llist_head *head; + struct raw_hwp_page *raw_hwp; + struct llist_node *t, *tnode; + int ret = TestSetPageHWPoison(hpage) ? -EHWPOISON : 0; + + /* + * Once the hwpoison hugepage has lost reliable raw error info, + * there is little meaning to keep additional error info precisely, + * so skip to add additional raw error info. + */ + if (HPageRawHwpUnreliable(hpage)) + return -EHWPOISON; + head = raw_hwp_list_head(hpage); + llist_for_each_safe(tnode, t, head->first) { + struct raw_hwp_page *p = container_of(tnode, struct raw_hwp_page, node); + + if (p->page == page) + return -EHWPOISON; + } + + raw_hwp = kmalloc(sizeof(struct raw_hwp_page), GFP_ATOMIC); + if (raw_hwp) { + raw_hwp->page = page; + llist_add(&raw_hwp->node, head); + /* the first error event will be counted in action_result(). */ + if (ret) + num_poisoned_pages_inc(); + } else { + /* + * Failed to save raw error info. We no longer trace all + * hwpoisoned subpages, and we need refuse to free/dissolve + * this hwpoisoned hugepage. + */ + SetHPageRawHwpUnreliable(hpage); + } + return ret; +} + +int hugetlb_clear_page_hwpoison(struct page *hpage) +{ + struct llist_head *head; + struct llist_node *t, *tnode; + + if (!HPageRawHwpUnreliable(hpage)) + ClearPageHWPoison(hpage); + head = raw_hwp_list_head(hpage); + llist_for_each_safe(tnode, t, head->first) { + struct raw_hwp_page *p = container_of(tnode, struct raw_hwp_page, node); + + SetPageHWPoison(p->page); + kfree(p); + } + llist_del_all(head); + return 0; +} + /* * Called from hugetlb code with hugetlb_lock held. * @@ -1698,7 +1772,7 @@ int __get_huge_page_for_hwpoison(unsigned long pfn, int flags) goto out; } - if (TestSetPageHWPoison(head)) { + if (hugetlb_set_page_hwpoison(head, page)) { ret = -EHWPOISON; goto out; } @@ -1710,7 +1784,6 @@ int __get_huge_page_for_hwpoison(unsigned long pfn, int flags) return ret; } -#ifdef CONFIG_HUGETLB_PAGE /* * Taking refcount of hugetlb pages needs extra care about race conditions * with basic operations like hugepage allocation/free/demotion. @@ -1751,7 +1824,7 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb lock_page(head); if (hwpoison_filter(p)) { - ClearPageHWPoison(head); + hugetlb_clear_page_hwpoison(head); res = -EOPNOTSUPP; goto out; } From patchwork Fri Jul 8 05:36:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12910586 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2880C433EF for ; Fri, 8 Jul 2022 05:37:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0728D6B0078; Fri, 8 Jul 2022 01:37:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F3D2E6B007B; Fri, 8 Jul 2022 01:37:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DDFA0900002; Fri, 8 Jul 2022 01:37:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C81B26B0078 for ; Fri, 8 Jul 2022 01:37:52 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 78E4334FDC for ; Fri, 8 Jul 2022 05:37:52 +0000 (UTC) X-FDA: 79662825984.06.D4F5397 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by imf19.hostedemail.com (Postfix) with ESMTP id DDB181A004B for ; Fri, 8 Jul 2022 05:37:51 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1657258670; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ARjZZkl6xjfRyo2hLbxFydMj+YgR/edWuX0ZkMftdV0=; b=PeVBC76276E6xDTbUK/9u53cROu7KuO9S9OhUwjLJBd39t4ONyAuiVM0wp8+qWSEz7Vg7m 7mi02nPTyc6IXEjONRjw8kmEmiTyunyBP+ZZPv6tAWg5xJ5MT4njVprcwkw+uC4EqzD+pW Hk7JGR8mSHswLx+yBbGbQmAALn7LBxk= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [mm-unstable PATCH v5 4/8] mm, hwpoison: make unpoison aware of raw error info in hwpoisoned hugepage Date: Fri, 8 Jul 2022 14:36:49 +0900 Message-Id: <20220708053653.964464-5-naoya.horiguchi@linux.dev> In-Reply-To: <20220708053653.964464-1-naoya.horiguchi@linux.dev> References: <20220708053653.964464-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=PeVBC762; spf=pass (imf19.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657258672; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ARjZZkl6xjfRyo2hLbxFydMj+YgR/edWuX0ZkMftdV0=; b=T8oym6H1N4nwZtBggi0cfY1J+i4JiphZegZE9A8PjHpQTyQ70+H/VwnWRKUrqXbRcB4HCq 8oLnh5Upb4zmoXsHJu6MQmZAy80nC9G19/JtGJgWTzBlb6i9CagN+wsVnEkz44iCQXlM7y 3HZUrM7DGqqDuioXwJq8AXH10nMmgwI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657258672; a=rsa-sha256; cv=none; b=yycAlbU+xvtp3oU9tBBcwiU6+nkVHoxY8wonI3A6ccnN3XGy8wKkcGIhNZ0rQPF3OnQ7w9 +YsvyhinyiFpLqV0C8XIzqHI++lCgC6PK1tv6/vCjz1HE4L/W1bwkZQ5hQzjpar2W4UhNZ JJ/4ksODcAy3UsZVZ7HK0ozZGpoOKX0= X-Rspam-User: X-Stat-Signature: zqthb54m6ci8b683aib7dibxew1zefjm X-Rspamd-Queue-Id: DDB181A004B Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=PeVBC762; spf=pass (imf19.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Rspamd-Server: rspam11 X-HE-Tag: 1657258671-213048 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Raw error info list needs to be removed when hwpoisoned hugetlb is unpoisoned. And unpoison handler needs to know how many errors there are in the target hugepage. So add them. HPageVmemmapOptimized(hpage) and HPageRawHwpUnreliable(hpage)) can't be unpoisoned, so let's skip them. Signed-off-by: Naoya Horiguchi Reported-by: kernel test robot --- v4 -> v5: - fix type of return value of free_raw_hwp_pages() (found by kernel test robot), - prevent unpoison for HPageVmemmapOptimized and HPageRawHwpUnreliable. --- include/linux/swapops.h | 9 ++++++++ mm/memory-failure.c | 50 ++++++++++++++++++++++++++++++++++++----- 2 files changed, 53 insertions(+), 6 deletions(-) diff --git a/include/linux/swapops.h b/include/linux/swapops.h index a01aeb3fcc0b..ddc98f96ad2c 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -498,6 +498,11 @@ static inline void num_poisoned_pages_dec(void) atomic_long_dec(&num_poisoned_pages); } +static inline void num_poisoned_pages_sub(long i) +{ + atomic_long_sub(i, &num_poisoned_pages); +} + #else static inline swp_entry_t make_hwpoison_entry(struct page *page) @@ -518,6 +523,10 @@ static inline struct page *hwpoison_entry_to_page(swp_entry_t entry) static inline void num_poisoned_pages_inc(void) { } + +static inline void num_poisoned_pages_sub(long i) +{ +} #endif static inline int non_swap_entry(swp_entry_t entry) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 6833c5e4b410..89e74ec8a95f 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1720,22 +1720,41 @@ static int hugetlb_set_page_hwpoison(struct page *hpage, struct page *page) return ret; } -int hugetlb_clear_page_hwpoison(struct page *hpage) +static long free_raw_hwp_pages(struct page *hpage, bool move_flag) { struct llist_head *head; struct llist_node *t, *tnode; + long count = 0; - if (!HPageRawHwpUnreliable(hpage)) - ClearPageHWPoison(hpage); + /* + * HPageVmemmapOptimized hugepages can't be unpoisoned because + * struct pages for tail pages are required to free hwpoisoned + * hugepages. HPageRawHwpUnreliable hugepages shouldn't be + * unpoisoned by definition. + */ + if (HPageVmemmapOptimized(hpage) || HPageRawHwpUnreliable(hpage)) + return 0; head = raw_hwp_list_head(hpage); llist_for_each_safe(tnode, t, head->first) { struct raw_hwp_page *p = container_of(tnode, struct raw_hwp_page, node); - SetPageHWPoison(p->page); + if (move_flag) + SetPageHWPoison(p->page); kfree(p); + count++; } llist_del_all(head); - return 0; + return count; +} + +int hugetlb_clear_page_hwpoison(struct page *hpage) +{ + int ret = -EBUSY; + + if (!HPageRawHwpUnreliable(hpage)) + ret = !TestClearPageHWPoison(hpage); + free_raw_hwp_pages(hpage, true); + return ret; } /* @@ -1879,6 +1898,10 @@ static inline int try_memory_failure_hugetlb(unsigned long pfn, int flags, int * return 0; } +static inline long free_raw_hwp_pages(struct page *hpage, bool move_flag) +{ + return 0; +} #endif /* CONFIG_HUGETLB_PAGE */ static int memory_failure_dev_pagemap(unsigned long pfn, int flags, @@ -2284,6 +2307,7 @@ int unpoison_memory(unsigned long pfn) struct page *p; int ret = -EBUSY; int freeit = 0; + long count = 1; static DEFINE_RATELIMIT_STATE(unpoison_rs, DEFAULT_RATELIMIT_INTERVAL, DEFAULT_RATELIMIT_BURST); @@ -2331,6 +2355,13 @@ int unpoison_memory(unsigned long pfn) ret = get_hwpoison_page(p, MF_UNPOISON); if (!ret) { + if (PageHuge(p)) { + count = free_raw_hwp_pages(page, false); + if (count == 0) { + ret = -EBUSY; + goto unlock_mutex; + } + } ret = TestClearPageHWPoison(page) ? 0 : -EBUSY; } else if (ret < 0) { if (ret == -EHWPOISON) { @@ -2339,6 +2370,13 @@ int unpoison_memory(unsigned long pfn) unpoison_pr_info("Unpoison: failed to grab page %#lx\n", pfn, &unpoison_rs); } else { + if (PageHuge(p)) { + count = free_raw_hwp_pages(page, false); + if (count == 0) { + ret = -EBUSY; + goto unlock_mutex; + } + } freeit = !!TestClearPageHWPoison(p); put_page(page); @@ -2351,7 +2389,7 @@ int unpoison_memory(unsigned long pfn) unlock_mutex: mutex_unlock(&mf_mutex); if (!ret || freeit) { - num_poisoned_pages_dec(); + num_poisoned_pages_sub(count); unpoison_pr_info("Unpoison: Software-unpoisoned page %#lx\n", page_to_pfn(p), &unpoison_rs); } From patchwork Fri Jul 8 05:36:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12910587 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 477B7C43334 for ; Fri, 8 Jul 2022 05:37:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BBE7D6B007B; Fri, 8 Jul 2022 01:37:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B4531900002; Fri, 8 Jul 2022 01:37:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9EAF96B007E; Fri, 8 Jul 2022 01:37:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 8BD536B007B for ; Fri, 8 Jul 2022 01:37:56 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6299320EEB for ; Fri, 8 Jul 2022 05:37:56 +0000 (UTC) X-FDA: 79662826152.26.3B95DEB Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by imf23.hostedemail.com (Postfix) with ESMTP id DBB98140025 for ; Fri, 8 Jul 2022 05:37:55 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1657258674; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y1K2uin893nuStX9RfB9QNHVZqNKzDhIgV7Ky15eqk8=; b=qvs0B2ecElKBLEbIGtVDL+K1w/riJSZ4KEUpvqvGyT1Gly/nBOXNQeQbpaRCFbcjwBC6x5 NA7/R28H5zHkYPYrm7n/hhBMoKOMrF+b8I72cuS0x0R1wyL+3ear+42NWkT42pZzxnWsd1 lB3EVQQUUANID1JlzYMm6lfGBKjZA4A= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [mm-unstable PATCH v5 5/8] mm, hwpoison: set PG_hwpoison for busy hugetlb pages Date: Fri, 8 Jul 2022 14:36:50 +0900 Message-Id: <20220708053653.964464-6-naoya.horiguchi@linux.dev> In-Reply-To: <20220708053653.964464-1-naoya.horiguchi@linux.dev> References: <20220708053653.964464-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657258676; a=rsa-sha256; cv=none; b=iEKXx+7qppi5bPwMIh2Kf/bHeSpXChOnmBSgJ6Uip7wKNsTwLLbzAIT2Z9jsNRvKXFm41m g1ODs2EdOl9teUsgXhUcPtTGxetNDDn7pcyhmDKf2X8rkECDvSp0Uiu3jnbi3qJRDLNq1S 7BIgFQrWqtCZWmf9QpFv/8mFWmC0TiE= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=qvs0B2ec; spf=pass (imf23.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657258676; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Y1K2uin893nuStX9RfB9QNHVZqNKzDhIgV7Ky15eqk8=; b=xueerofhoB3wstF4HEBeT3mC9glsvdKdVEcQNrUUQA9gIRnnA4UQeXvZKAzNn4drFHsbdT TCGrYCPDBswHhYo4WLuaWU/apPWQltPkk1PcakMHqzRJaBBTl2ihkoOFGyM/bGcRcnWN6P eVA5CII9uWduL/IxMAl/+xv9bvxjztU= X-Stat-Signature: okw4uduyg1xpgg6kkdr5xrshj47dir6t X-Rspamd-Queue-Id: DBB98140025 Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=qvs0B2ec; spf=pass (imf23.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1657258675-817728 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi If memory_failure() fails to grab page refcount on a hugetlb page because it's busy, it returns without setting PG_hwpoison on it. This not only loses a chance of error containment, but breaks the rule that action_result() should be called only when memory_failure() do any of handling work (even if that's just setting PG_hwpoison). This inconsistency could harm code maintainability. So set PG_hwpoison and call hugetlb_set_page_hwpoison() for such a case. Fixes: 405ce051236c ("mm/hwpoison: fix race between hugetlb free/demotion and memory_failure_hugetlb()") Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin --- include/linux/mm.h | 1 + mm/memory-failure.c | 8 ++++---- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 4287bec50c28..7668831c919f 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3188,6 +3188,7 @@ enum mf_flags { MF_SOFT_OFFLINE = 1 << 3, MF_UNPOISON = 1 << 4, MF_SW_SIMULATED = 1 << 5, + MF_NO_RETRY = 1 << 6, }; int mf_dax_kill_procs(struct address_space *mapping, pgoff_t index, unsigned long count, int mf_flags); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 89e74ec8a95f..f17f0475db93 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1788,7 +1788,8 @@ int __get_huge_page_for_hwpoison(unsigned long pfn, int flags) count_increased = true; } else { ret = -EBUSY; - goto out; + if (!(flags & MF_NO_RETRY)) + goto out; } if (hugetlb_set_page_hwpoison(head, page)) { @@ -1815,7 +1816,6 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb struct page *p = pfn_to_page(pfn); struct page *head; unsigned long page_flags; - bool retry = true; *hugetlb = 1; retry: @@ -1831,8 +1831,8 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb } return res; } else if (res == -EBUSY) { - if (retry) { - retry = false; + if (!(flags & MF_NO_RETRY)) { + flags |= MF_NO_RETRY; goto retry; } action_result(pfn, MF_MSG_UNKNOWN, MF_IGNORED); From patchwork Fri Jul 8 05:36:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12910588 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22D35C433EF for ; Fri, 8 Jul 2022 05:38:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A5DD36B007D; Fri, 8 Jul 2022 01:38:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9E6FD6B007E; Fri, 8 Jul 2022 01:38:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 887456B0080; Fri, 8 Jul 2022 01:38:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7459F6B007D for ; Fri, 8 Jul 2022 01:38:00 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 51C0F120EA3 for ; Fri, 8 Jul 2022 05:38:00 +0000 (UTC) X-FDA: 79662826320.03.08480B0 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by imf05.hostedemail.com (Postfix) with ESMTP id E56FF10001D for ; Fri, 8 Jul 2022 05:37:59 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1657258678; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a5cfdoF+Q3PicJ4hUmNvih68gRFF+QsIiLkN3fSd50o=; b=dGZnVxCkoUtL6sNLsB3dTwycJsCSweuvR53NKdprQWpUOF7aAitjtx8LO8PaN2o98Fy0Iz U92RPkDv9w4zWN06Pq4UhoK1d12SJknNswFbK97MQc7h6K44GKn4u0LA6yzlKK1TQsa4Ym hSaN806fuk++YW//Bo8IG36oV2501gc= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [mm-unstable PATCH v5 6/8] mm, hwpoison: make __page_handle_poison returns int Date: Fri, 8 Jul 2022 14:36:51 +0900 Message-Id: <20220708053653.964464-7-naoya.horiguchi@linux.dev> In-Reply-To: <20220708053653.964464-1-naoya.horiguchi@linux.dev> References: <20220708053653.964464-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657258680; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=a5cfdoF+Q3PicJ4hUmNvih68gRFF+QsIiLkN3fSd50o=; b=B/d3TyZIYrbtUV/66aK1cgEMSr7W0F/kVIK9RT8jOj26U/FnpJpdpIrwxnd4PTK1Y6CzdB NgCrf2fSoCsIP1buMiEnZ5oCW2sbfp0f++9/ep/HrXPKm+oTCBsRwDCiV3AemIRiBTAZrL 1Mjc0GqvJpzFutd27vSsXmjMwYFEnEs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657258680; a=rsa-sha256; cv=none; b=k+2KY8MhUGi9hCt9wyrlFWResS5m/cGcPNSUYhN4aGpeo8WN9+rJ0h9IQ7gc3u8zErMVDR n2VcAIsAIeN0GYDTJPclN1BryY3ScpFCQ3b96rha6R+j5Eg/2f5o03tBOPdIGxOx60vT3W PWW81m97JDwWLjymcbOP41trwWLywv0= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=dGZnVxCk; spf=pass (imf05.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Rspamd-Queue-Id: E56FF10001D Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=dGZnVxCk; spf=pass (imf05.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Rspam-User: X-Rspamd-Server: rspam06 X-Stat-Signature: unpgaei9qcg7bmxosgyx6zuyzcdbdog5 X-HE-Tag: 1657258679-508206 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi __page_handle_poison() returns bool that shows whether take_page_off_buddy() has passed or not now. But we will want to distinguish another case of "dissolve has passed but taking off failed" by its return value. So change the type of the return value. No functional change. Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin --- v2 -> v3: - move deleting "res = MF_FAILED" to the later patch. (by Miaohe) --- mm/memory-failure.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index f17f0475db93..e38a94007ffe 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -71,7 +71,13 @@ atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); static bool hw_memory_failure __read_mostly = false; -static bool __page_handle_poison(struct page *page) +/* + * Return values: + * 1: the page is dissolved (if needed) and taken off from buddy, + * 0: the page is dissolved (if needed) and not taken off from buddy, + * < 0: failed to dissolve. + */ +static int __page_handle_poison(struct page *page) { int ret; @@ -81,7 +87,7 @@ static bool __page_handle_poison(struct page *page) ret = take_page_off_buddy(page); zone_pcp_enable(page_zone(page)); - return ret > 0; + return ret; } static bool page_handle_poison(struct page *page, bool hugepage_or_freepage, bool release) @@ -91,7 +97,7 @@ static bool page_handle_poison(struct page *page, bool hugepage_or_freepage, boo * Doing this check for free pages is also fine since dissolve_free_huge_page * returns 0 for non-hugetlb pages as well. */ - if (!__page_handle_poison(page)) + if (__page_handle_poison(page) <= 0) /* * We could fail to take off the target page from buddy * for example due to racy page allocation, but that's @@ -1086,7 +1092,7 @@ static int me_huge_page(struct page_state *ps, struct page *p) * subpages. */ put_page(hpage); - if (__page_handle_poison(p)) { + if (__page_handle_poison(p) > 0) { page_ref_inc(p); res = MF_RECOVERED; } @@ -1855,7 +1861,7 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb if (res == 0) { unlock_page(head); res = MF_FAILED; - if (__page_handle_poison(p)) { + if (__page_handle_poison(p) > 0) { page_ref_inc(p); res = MF_RECOVERED; } From patchwork Fri Jul 8 05:36:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12910589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56595C433EF for ; Fri, 8 Jul 2022 05:38:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E54EB6B007E; Fri, 8 Jul 2022 01:38:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E04436B0080; Fri, 8 Jul 2022 01:38:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA6BA6B0081; Fri, 8 Jul 2022 01:38:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B7B936B007E for ; Fri, 8 Jul 2022 01:38:04 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 949EC120E8D for ; Fri, 8 Jul 2022 05:38:04 +0000 (UTC) X-FDA: 79662826488.09.4E61330 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by imf05.hostedemail.com (Postfix) with ESMTP id E091C10001D for ; Fri, 8 Jul 2022 05:38:03 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1657258682; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vOmKEsP5qHLQsoJSDPiON+o/eMH2bk607wMScNN4r/I=; b=sM2ptqqmr90GKbmlteE88gbqhMsdouCnpfczZuEQd0cSpbWUr3S6Vqwtdci2TBB40cVSWM qstknsxn//EHkKpCWkMk+BeNkNsTs4B31iyaZCfJVYoBVxsIw96s78oVZyhPOycCPLBz2A tIGf4MA0iAEtUbrJFRzU9hw3dpkzVBw= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [mm-unstable PATCH v5 7/8] mm, hwpoison: skip raw hwpoison page in freeing 1GB hugepage Date: Fri, 8 Jul 2022 14:36:52 +0900 Message-Id: <20220708053653.964464-8-naoya.horiguchi@linux.dev> In-Reply-To: <20220708053653.964464-1-naoya.horiguchi@linux.dev> References: <20220708053653.964464-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657258684; a=rsa-sha256; cv=none; b=v4zNuykhc33ggWObq7TyQKFcK7ycRQsFYVrxQjOBzYKbwSfUG2+Fm6p55V5mrxNc3rK0+o +smTWaem5swen8rok6fj/DjYRkI4Losmuc+EPW3OR19Za1D97Rs/xkUAGfci7J3cTzI3aE MYk7TwIo+FWXKqiEYiHoWwP790bBGuY= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=sM2ptqqm; spf=pass (imf05.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657258684; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vOmKEsP5qHLQsoJSDPiON+o/eMH2bk607wMScNN4r/I=; b=NjFPNXSR7Tyr6gq5x4roCsqCd3pFnpkOa46uMQlHiJujAy7H6X4uiu3QP4YsZ+NUTvAPph f1SPC3us+Rqu39b4Y59EOl7pQvWeQMEFtL/1qet5F/jpCnq2Zp/D+778S8hdMnPWDtjDlG Tlj7U5kNn73CTMGTXimNjRndYSPRsn0= X-Rspamd-Server: rspam08 X-Rspam-User: Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=sM2ptqqm; spf=pass (imf05.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Stat-Signature: n8hb5ong5npy5nwx1i5uaws9dpgpzcur X-Rspamd-Queue-Id: E091C10001D X-HE-Tag: 1657258683-123908 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Currently if memory_failure() (modified to remove blocking code with subsequent patch) is called on a page in some 1GB hugepage, memory error handling fails and the raw error page gets into leaked state. The impact is small in production systems (just leaked single 4kB page), but this limits the testability because unpoison doesn't work for it. We can no longer create 1GB hugepage on the 1GB physical address range with such leaked pages, that's not useful when testing on small systems. When a hwpoison page in a 1GB hugepage is handled, it's caught by the PageHWPoison check in free_pages_prepare() because the 1GB hugepage is broken down into raw error pages before coming to this point: if (unlikely(PageHWPoison(page)) && !order) { ... return false; } Then, the page is not sent to buddy and the page refcount is left 0. Originally this check is supposed to work when the error page is freed from page_handle_poison() (that is called from soft-offline), but now we are opening another path to call it, so the callers of __page_handle_poison() need to handle the case by considering the return value 0 as success. Then page refcount for hwpoison is properly incremented so unpoison works. Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin --- v2 -> v3: - remove "res = MF_FAILED" in try_memory_failure_hugetlb (by Miaohe) --- mm/memory-failure.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index e38a94007ffe..cdf266aaabdf 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1084,7 +1084,6 @@ static int me_huge_page(struct page_state *ps, struct page *p) res = truncate_error_page(hpage, page_to_pfn(p), mapping); unlock_page(hpage); } else { - res = MF_FAILED; unlock_page(hpage); /* * migration entry prevents later access on error hugepage, @@ -1092,9 +1091,11 @@ static int me_huge_page(struct page_state *ps, struct page *p) * subpages. */ put_page(hpage); - if (__page_handle_poison(p) > 0) { + if (__page_handle_poison(p) >= 0) { page_ref_inc(p); res = MF_RECOVERED; + } else { + res = MF_FAILED; } } @@ -1860,10 +1861,11 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb */ if (res == 0) { unlock_page(head); - res = MF_FAILED; - if (__page_handle_poison(p) > 0) { + if (__page_handle_poison(p) >= 0) { page_ref_inc(p); res = MF_RECOVERED; + } else { + res = MF_FAILED; } action_result(pfn, MF_MSG_FREE_HUGE, res); return res == MF_RECOVERED ? 0 : -EBUSY; From patchwork Fri Jul 8 05:36:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12910590 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86B3EC433EF for ; Fri, 8 Jul 2022 05:38:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 21F606B0082; Fri, 8 Jul 2022 01:38:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A97D6B0081; Fri, 8 Jul 2022 01:38:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0490A900002; Fri, 8 Jul 2022 01:38:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id E68E36B0080 for ; Fri, 8 Jul 2022 01:38:08 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BCB0420FA1 for ; Fri, 8 Jul 2022 05:38:08 +0000 (UTC) X-FDA: 79662826656.06.2AFC562 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by imf12.hostedemail.com (Postfix) with ESMTP id 2EB044002C for ; Fri, 8 Jul 2022 05:38:08 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1657258686; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ss61C/7eiat90GPlQTY7SZczV4odc8E10C4i+gZumvA=; b=pzd7nBpyFvj7x4I5lpt//Q/S05L53I3ag266sMGwSfkZBEI3LaDLxe1oDt1UfCSxgpnFqh AqQJgfu2jq7UbTtYwCR1H5T44/kVR0tcRNhm4Yw9RbkwkkjEnOUPUOyi/zcbeiFOzsRvw7 F3qaP4/iNzJLGvUBeAKFubWKxn1wnko= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [mm-unstable PATCH v5 8/8] mm, hwpoison: enable memory error handling on 1GB hugepage Date: Fri, 8 Jul 2022 14:36:53 +0900 Message-Id: <20220708053653.964464-9-naoya.horiguchi@linux.dev> In-Reply-To: <20220708053653.964464-1-naoya.horiguchi@linux.dev> References: <20220708053653.964464-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657258688; a=rsa-sha256; cv=none; b=uxRJFVsFIrZMSJ8Qwo2XqC+u5+X4Q+VJDMy57VV97PqO5AQSpA1QMbo0f+ARLwoleRIAm+ 3JIBqJBApVnOaxhHgVPO54xShOAVR/y9VS9P36Ddh62xYM+/q6QDxvgYbdaFxJ6b2qH/Cw kWVUlJ7UYgQNIcudgMsGXZwWt1+U/8M= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=pzd7nBpy; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf12.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657258688; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ss61C/7eiat90GPlQTY7SZczV4odc8E10C4i+gZumvA=; b=42sYEH2fHEOsSwiMBjo8TgtV+590iiuSmq1p+T6q2rXIUgXiVA2mrgEcBAkq1bvcTdHG/u v9DOc315Mb7jpebtuD3L1lkLOU5W6jTW9lDcpnddvzDei5yUg/GzusLa0L83CUpaIVQhe4 /14EZheULdrlYYBy9CFAXFGYeHwNtKE= X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 2EB044002C Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=pzd7nBpy; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf12.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev X-Stat-Signature: wj94uyk8n3apnyzg8uy1b1368wncejzc X-Rspam-User: X-HE-Tag: 1657258688-392279 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Now error handling code is prepared, so remove the blocking code and enable memory error handling on 1GB hugepage. Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin --- include/linux/mm.h | 1 - include/ras/ras_event.h | 1 - mm/memory-failure.c | 16 ---------------- 3 files changed, 18 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 7668831c919f..b0e83835184e 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3241,7 +3241,6 @@ enum mf_action_page_type { MF_MSG_DIFFERENT_COMPOUND, MF_MSG_HUGE, MF_MSG_FREE_HUGE, - MF_MSG_NON_PMD_HUGE, MF_MSG_UNMAP_FAILED, MF_MSG_DIRTY_SWAPCACHE, MF_MSG_CLEAN_SWAPCACHE, diff --git a/include/ras/ras_event.h b/include/ras/ras_event.h index d0337a41141c..cbd3ddd7c33d 100644 --- a/include/ras/ras_event.h +++ b/include/ras/ras_event.h @@ -360,7 +360,6 @@ TRACE_EVENT(aer_event, EM ( MF_MSG_DIFFERENT_COMPOUND, "different compound page after locking" ) \ EM ( MF_MSG_HUGE, "huge page" ) \ EM ( MF_MSG_FREE_HUGE, "free huge page" ) \ - EM ( MF_MSG_NON_PMD_HUGE, "non-pmd-sized huge page" ) \ EM ( MF_MSG_UNMAP_FAILED, "unmapping failed page" ) \ EM ( MF_MSG_DIRTY_SWAPCACHE, "dirty swapcache page" ) \ EM ( MF_MSG_CLEAN_SWAPCACHE, "clean swapcache page" ) \ diff --git a/mm/memory-failure.c b/mm/memory-failure.c index cdf266aaabdf..3640bbe5a3a2 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -765,7 +765,6 @@ static const char * const action_page_types[] = { [MF_MSG_DIFFERENT_COMPOUND] = "different compound page after locking", [MF_MSG_HUGE] = "huge page", [MF_MSG_FREE_HUGE] = "free huge page", - [MF_MSG_NON_PMD_HUGE] = "non-pmd-sized huge page", [MF_MSG_UNMAP_FAILED] = "unmapping failed page", [MF_MSG_DIRTY_SWAPCACHE] = "dirty swapcache page", [MF_MSG_CLEAN_SWAPCACHE] = "clean swapcache page", @@ -1873,21 +1872,6 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb page_flags = head->flags; - /* - * TODO: hwpoison for pud-sized hugetlb doesn't work right now, so - * simply disable it. In order to make it work properly, we need - * make sure that: - * - conversion of a pud that maps an error hugetlb into hwpoison - * entry properly works, and - * - other mm code walking over page table is aware of pud-aligned - * hwpoison entries. - */ - if (huge_page_size(page_hstate(head)) > PMD_SIZE) { - action_result(pfn, MF_MSG_NON_PMD_HUGE, MF_IGNORED); - res = -EBUSY; - goto out; - } - if (!hwpoison_user_mappings(p, pfn, flags, head)) { action_result(pfn, MF_MSG_UNMAP_FAILED, MF_IGNORED); res = -EBUSY;