From patchwork Tue Jul 17 05:32:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 10528127 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id CE2226020A for ; Tue, 17 Jul 2018 05:32:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB085289BA for ; Tue, 17 Jul 2018 05:32:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AECAF28A49; Tue, 17 Jul 2018 05:32:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE, T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 215B528A7F for ; Tue, 17 Jul 2018 05:32:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 717346B0006; Tue, 17 Jul 2018 01:32:50 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6746F6B0007; Tue, 17 Jul 2018 01:32:50 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 452B76B000C; Tue, 17 Jul 2018 01:32:50 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by kanga.kvack.org (Postfix) with ESMTP id F11A66B0006 for ; Tue, 17 Jul 2018 01:32:49 -0400 (EDT) Received: by mail-pg1-f198.google.com with SMTP id n5-v6so4352018pgp.20 for ; Mon, 16 Jul 2018 22:32:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:sender:from:to:cc:subject:date :message-id:in-reply-to:references; bh=A7nWFKVutHwD3dCodLh1vnhiiLBP95OUDJsJ6qnyAzM=; b=TxB90/CspG+/UdrTd9InDUHBcG4p9wl2+YMLJThWwrHLHr9DpEIUQSqtJAZsNcGJk7 QR5dOpGCrPsKxgNd0sdc5QD0MFi0Rhtem5EELCpN0oxHCRhK8+SG6oWU7Qy+xux2oXHG aWxLhepBPMcKQ8VaQtA6g71VCKkUaaHORv0cIJlEZDI+zhev+SEX1wBOZzeqyOvzHqi/ /VcUBQnb51GvjVhuGgxInJcI24N/RDFLAb/yQs8MCEgHf3hMbMhsYr/Fs2MgGp+8cBtf OvOQRwKC3psPBMnkIzPCn9NAu3LixDfGgZk9iXa8w3GXgsNDAjhuuuEsuSwSQ7R9yGA1 o80A== X-Gm-Message-State: AOUpUlHvXU/xQOdtgAtAmtLI5Izls+AprAhvyN6zxUZz5+xewkC4VS3f tmJUBh7CPDdMrbHX4Cx49qcfgT7LH1CseANmlu220wE4M3ZnysjVMGdVPWga8M8xfSwBTCzczU5 vngY+/pS0q2AU5UVTjdPbtffMGEZZUop6fgskibIn+Vh7NyPjGbEOGOmvOPd7KkTiPpdugbsoUR PaYiZHsRZYt+LmCZInd6bcxJf2uWq1llwx/cAeggAkuXjX0k4QRUAJ3JnQf37GjQyBQzHyK/FK8 T1tJA9HM7v+GPhmCA3iff+Eft1AJrexP4/gKfl/bqk+GEIwbhRwdnCV2jmz30oPCY3Zx6I8X/cZ zTFKhPI73OLJrE0W+AOKC3xhGOjHbzj+IKSNheWul1TiI+jQAJQpbtwxSkBq93t7yunfnFIqHw= = X-Received: by 2002:a62:b917:: with SMTP id z23-v6mr190938pfe.131.1531805569663; Mon, 16 Jul 2018 22:32:49 -0700 (PDT) X-Received: by 2002:a62:b917:: with SMTP id z23-v6mr190897pfe.131.1531805568714; Mon, 16 Jul 2018 22:32:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1531805568; cv=none; d=google.com; s=arc-20160816; b=WhE4u/rb3VxQWkGWkP8Jg0VdBVX6DHatPy3gTWbJW9x49IbERVG8QNNX6GalLGb4Xd XeGMlm5IM7HAk6jcTK5kb5fCSkmtabL1b6EUqwPSwReGlrNqa6s9fVv/Gaq1ZOIKgtGP cUc1l+Wzke1eNT2gJ2DE2wvOml0Avw1lVjybo1d8yRWd40Z7HfTLT6DeOC5Jz8Za4eye RHeW9ud2DQHehfnIamOuUSy0mvJnosr6HGBLjaPnriQoEIiY0mEfXckKGwUV0slmFcdB ognvUtXVBrAdA+eDkqkPbnGpxnHuLHD4igV2nEg847ufFA50uvLnzmZtDhs0bAk/39mm 7qRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature:arc-authentication-results; bh=A7nWFKVutHwD3dCodLh1vnhiiLBP95OUDJsJ6qnyAzM=; b=jiVhYDmTlfCi7mplHqhy80JibCswShCojV3qIJOZrQBfJqkMiPzjDw/8buOJdk76v+ HADkU5FXVIA8T9yJ6tffMpfTB1miWiHrjQeOaI47OJWgMy5wu9qgGYoEuFe2CXU2Rn4V tY92OIVt2LeZrpGTIle+BtQcvgypWSnoJH2wKLYY5V/6CTidFfcS0gw6KcVKE/u7swxs 6wmBNfkqNMGRjDDh23TBYkOctdZ3kd13OygZTB8rnIuEl5PLiamjrVQWIUIiUmLvRmeo VtaiMJENuGi/E6s5B6lknRVaAphxVJGqc18hoNW2+tLe/YvYi/m1xpH7TppIuNiqmiKf kPlg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=hjIahVon; spf=pass (google.com: domain of nao.horiguchi@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id g11-v6sor19451pgg.151.2018.07.16.22.32.48 for (Google Transport Security); Mon, 16 Jul 2018 22:32:48 -0700 (PDT) Received-SPF: pass (google.com: domain of nao.horiguchi@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=hjIahVon; spf=pass (google.com: domain of nao.horiguchi@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=A7nWFKVutHwD3dCodLh1vnhiiLBP95OUDJsJ6qnyAzM=; b=hjIahVonJgkPhqgktFN76RASQn8z1wkil86un5uMN5EL4VS64m/ibVWZyJtMwizrey 9xsrihPee964CtebxrGks2puXCxKyqeMGp1er2QnWDPfZR2NOVYsNsgqDD0aDGTtUwiX qqTCuAh9xHGvIg+c0FK2WxdoqCReQGjii/PFNln0n+SV1GDcic+t4lVfc1AOpfUqzj2j ddmKabtOFI6yDQOPw0HqP0ij5EG2CqD5eUxkTGXsgzVgvjvYwU/pyKM29TwUCmS8ntIS ai5OOwrQaPxf4UanMy1h8qA1Rtgn6El+gdvri4jHkQW8W83y2LXckCfQQr9WO3r3hOG2 rk8Q== X-Google-Smtp-Source: AAOMgpcgmpgHem3+VWmZPecXt92/x0fK5PujA+WHRs158LCMdSgGtdu1g4z7ToYI6hUEDelri64fNA== X-Received: by 2002:a63:fc44:: with SMTP id r4-v6mr173305pgk.169.1531805567991; Mon, 16 Jul 2018 22:32:47 -0700 (PDT) Received: from www9186uo.sakura.ne.jp (www9186uo.sakura.ne.jp. [153.121.56.200]) by smtp.gmail.com with ESMTPSA id y85-v6sm156220pfa.170.2018.07.16.22.32.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 16 Jul 2018 22:32:46 -0700 (PDT) From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Michal Hocko , Andrew Morton , xishi.qiuxishi@alibaba-inc.com, zy.zhengyi@alibaba-inc.com, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/2] mm: fix race on soft-offlining free huge pages Date: Tue, 17 Jul 2018 14:32:31 +0900 Message-Id: <1531805552-19547-2-git-send-email-n-horiguchi@ah.jp.nec.com> X-Mailer: git-send-email 2.7.0 In-Reply-To: <1531805552-19547-1-git-send-email-n-horiguchi@ah.jp.nec.com> References: <1531805552-19547-1-git-send-email-n-horiguchi@ah.jp.nec.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP There's a race condition between soft offline and hugetlb_fault which causes unexpected process killing and/or hugetlb allocation failure. The process killing is caused by the following flow: CPU 0 CPU 1 CPU 2 soft offline get_any_page // find the hugetlb is free mmap a hugetlb file page fault ... hugetlb_fault hugetlb_no_page alloc_huge_page // succeed soft_offline_free_page // set hwpoison flag mmap the hugetlb file page fault ... hugetlb_fault hugetlb_no_page find_lock_page return VM_FAULT_HWPOISON mm_fault_error do_sigbus // kill the process The hugetlb allocation failure comes from the following flow: CPU 0 CPU 1 mmap a hugetlb file // reserve all free page but don't fault-in soft offline get_any_page // find the hugetlb is free soft_offline_free_page // set hwpoison flag dissolve_free_huge_page // fail because all free hugepages are reserved page fault ... hugetlb_fault hugetlb_no_page alloc_huge_page ... dequeue_huge_page_node_exact // ignore hwpoisoned hugepage // and finally fail due to no-mem The root cause of this is that current soft-offline code is written based on an assumption that PageHWPoison flag should beset at first to avoid accessing the corrupted data. This makes sense for memory_failure() or hard offline, but does not for soft offline because soft offline is about corrected (not uncorrected) error and is safe from data lost. This patch changes soft offline semantics where it sets PageHWPoison flag only after containment of the error page completes successfully. Reported-by: Xishi Qiu Suggested-by: Xishi Qiu Signed-off-by: Naoya Horiguchi --- changelog v1->v2: - don't use set_hwpoison_free_buddy_page() (not defined yet) - updated comment in soft_offline_huge_page() --- mm/hugetlb.c | 11 +++++------ mm/memory-failure.c | 24 ++++++++++++++++++------ mm/migrate.c | 2 -- 3 files changed, 23 insertions(+), 14 deletions(-) diff --git v4.18-rc4-mmotm-2018-07-10-16-50/mm/hugetlb.c v4.18-rc4-mmotm-2018-07-10-16-50_patched/mm/hugetlb.c index 430be42..937c142 100644 --- v4.18-rc4-mmotm-2018-07-10-16-50/mm/hugetlb.c +++ v4.18-rc4-mmotm-2018-07-10-16-50_patched/mm/hugetlb.c @@ -1479,22 +1479,20 @@ static int free_pool_huge_page(struct hstate *h, nodemask_t *nodes_allowed, /* * Dissolve a given free hugepage into free buddy pages. This function does * nothing for in-use (including surplus) hugepages. Returns -EBUSY if the - * number of free hugepages would be reduced below the number of reserved - * hugepages. + * dissolution fails because a give page is not a free hugepage, or because + * free hugepages are fully reserved. */ int dissolve_free_huge_page(struct page *page) { - int rc = 0; + int rc = -EBUSY; spin_lock(&hugetlb_lock); if (PageHuge(page) && !page_count(page)) { struct page *head = compound_head(page); struct hstate *h = page_hstate(head); int nid = page_to_nid(head); - if (h->free_huge_pages - h->resv_huge_pages == 0) { - rc = -EBUSY; + if (h->free_huge_pages - h->resv_huge_pages == 0) goto out; - } /* * Move PageHWPoison flag from head page to the raw error page, * which makes any subpages rather than the error page reusable. @@ -1508,6 +1506,7 @@ int dissolve_free_huge_page(struct page *page) h->free_huge_pages_node[nid]--; h->max_huge_pages--; update_and_free_page(h, head); + rc = 0; } out: spin_unlock(&hugetlb_lock); diff --git v4.18-rc4-mmotm-2018-07-10-16-50/mm/memory-failure.c v4.18-rc4-mmotm-2018-07-10-16-50_patched/mm/memory-failure.c index 9d142b9..9b77f85 100644 --- v4.18-rc4-mmotm-2018-07-10-16-50/mm/memory-failure.c +++ v4.18-rc4-mmotm-2018-07-10-16-50_patched/mm/memory-failure.c @@ -1598,8 +1598,20 @@ static int soft_offline_huge_page(struct page *page, int flags) if (ret > 0) ret = -EIO; } else { - if (PageHuge(page)) - dissolve_free_huge_page(page); + /* + * We set PG_hwpoison only when the migration source hugepage + * was successfully dissolved, because otherwise hwpoisoned + * hugepage remains on free hugepage list. The allocator ignores + * such a hwpoisoned page so it's never allocated, but it could + * kill a process because of no-memory rather than hwpoison. + * Soft-offline never impacts the userspace, so this is + * undesired. + */ + ret = dissolve_free_huge_page(page); + if (!ret) { + if (!TestSetPageHWPoison(page)) + num_poisoned_pages_inc(); + } } return ret; } @@ -1715,13 +1727,13 @@ static int soft_offline_in_use_page(struct page *page, int flags) static void soft_offline_free_page(struct page *page) { + int rc = 0; struct page *head = compound_head(page); - if (!TestSetPageHWPoison(head)) { + if (PageHuge(head)) + rc = dissolve_free_huge_page(page); + if (!rc && !TestSetPageHWPoison(page)) num_poisoned_pages_inc(); - if (PageHuge(head)) - dissolve_free_huge_page(page); - } } /** diff --git v4.18-rc4-mmotm-2018-07-10-16-50/mm/migrate.c v4.18-rc4-mmotm-2018-07-10-16-50_patched/mm/migrate.c index 198af42..3ae213b 100644 --- v4.18-rc4-mmotm-2018-07-10-16-50/mm/migrate.c +++ v4.18-rc4-mmotm-2018-07-10-16-50_patched/mm/migrate.c @@ -1318,8 +1318,6 @@ static int unmap_and_move_huge_page(new_page_t get_new_page, out: if (rc != -EAGAIN) putback_active_hugepage(hpage); - if (reason == MR_MEMORY_FAILURE && !test_set_page_hwpoison(hpage)) - num_poisoned_pages_inc(); /* * If migration was not successful and there's a freeing callback, use