From patchwork Fri Jul 7 20:19:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaqi Yan X-Patchwork-Id: 13305251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 319C5C001B0 for ; Fri, 7 Jul 2023 20:19:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BE3016B0078; Fri, 7 Jul 2023 16:19:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B1A916B007B; Fri, 7 Jul 2023 16:19:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 96FA88D0001; Fri, 7 Jul 2023 16:19:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 82DCE6B0078 for ; Fri, 7 Jul 2023 16:19:19 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 4DB8514011D for ; Fri, 7 Jul 2023 20:19:19 +0000 (UTC) X-FDA: 80985930438.02.9E62333 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf05.hostedemail.com (Postfix) with ESMTP id 81213100019 for ; Fri, 7 Jul 2023 20:19:17 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=VqyQmTQk; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of 3RHOoZAgKCPcihZphxZmfnnfkd.bnlkhmtw-lljuZbj.nqf@flex--jiaqiyan.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3RHOoZAgKCPcihZphxZmfnnfkd.bnlkhmtw-lljuZbj.nqf@flex--jiaqiyan.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688761157; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8FNetjBnnWs4aWcoESXEPv5EQNod9M4H8ra/lwVlWng=; b=42Z1q8+KUgpzC/JBoIxVzPilMrWV0YFx6M78vhXStxYDdwXP95saHwEMl4gOIpgUObFC9W GVPxKlNVkGZD14LCoTZ9E1T30wIriiXegvLpJTS96+p9omVRU+ij5El+HVJlhJQ9qN/irE LhKd+nmDUDsNfUrhtgzo6jzk8nV0r1g= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=VqyQmTQk; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of 3RHOoZAgKCPcihZphxZmfnnfkd.bnlkhmtw-lljuZbj.nqf@flex--jiaqiyan.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3RHOoZAgKCPcihZphxZmfnnfkd.bnlkhmtw-lljuZbj.nqf@flex--jiaqiyan.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688761157; a=rsa-sha256; cv=none; b=LeLBOOZWniwDXBTyKFNd/N9R9YvjURhtTlhzGfTUSPUwbFXZl/EUP4XbtDCr+JEPuFVEGG z4vuwJTI/ATm02KBEzcACSXjawW2ReitSMBFRe8sdM3VvOjWwmr7sV2X8YeVTGm2blEpRe S6RDODto3PIrCtX7aw/31Fk4x5TexwI= Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1b801e6cebaso31199325ad.0 for ; Fri, 07 Jul 2023 13:19:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688761156; x=1691353156; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=8FNetjBnnWs4aWcoESXEPv5EQNod9M4H8ra/lwVlWng=; b=VqyQmTQkCzL2+2WReICnzA6BIm0pcsZl2UxgOC96YeMxOiWjK9NhMMsd6sh9vgNsyp l48aVL+YSen8q6QehXxVJmNvqqI169EdhdJF1wP18IrISpvNJpDmE8qvdh8iCozeWtb6 JHgOCmY5zsqIie6UwoBTIJKIq84brRDTo5Ou4o4qfUOH/nsVpBgCFa70jcSCS8JQjRGc YPkr1zj/Al/4tkKYO5OxobtQsE4XPWag4pb/fMTB6fXhhvDh+IoKStTY4VGEtEBkAolL +GRm79XEsc5P3ls1k+A19D7pSp5BkV+9M69nyu7mjSupo4UAMKBhqh3ZJivJ7t7YjP3Z tU8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688761156; x=1691353156; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8FNetjBnnWs4aWcoESXEPv5EQNod9M4H8ra/lwVlWng=; b=CwbFJBmyBEz0kfyoxTM9gydOI06Ys1nfDmg+jqk4IaHax2VdN0GpXHFRfrrqnY49SW 2iOYGTSUuBFXVogK0gc0HAv7Z+4NjpEftmD4froYvbRF1S558OoKGuQlATKcUH/EPjE7 a5BT/dTe/p3d5eI3Qp0+sa+KgwkYKuybFb1Kz1cwyhkA2psVN5xNz4YSS79Br3Y2t2Io UahthGty1tqMheLtnR5wL6AgWepL6+bq2GfiDVKXpGwNNX0pguxAdQUS+bkqk9eIMr0M xNpwKjU3LTTuU2ePmQV0OYhFMdi5SQTG8QwE6/GiV0POuz95uFy9ttMEzgnMpNjsyeVX BfxQ== X-Gm-Message-State: ABy/qLY6QJmID6++KPfMC8NRmfwepN/C8/VqJR7iE145TP7IVqZCoidI lfArffFlR9/ih5/ZmAcYVKSNbr+CW7KndQ== X-Google-Smtp-Source: APBJJlF/mGcdbnw0xXdoe2sEjVsVM5DocIm81TfZezvsw+ScgL/1ytfRgEc6ZH2UhLuK+hvQLPJDpV1K7/EAPA== X-Received: from yjq3.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:272f]) (user=jiaqiyan job=sendgmr) by 2002:a17:902:ec88:b0:1b7:fa05:e0c7 with SMTP id x8-20020a170902ec8800b001b7fa05e0c7mr5745441plg.13.1688761156325; Fri, 07 Jul 2023 13:19:16 -0700 (PDT) Date: Fri, 7 Jul 2023 20:19:01 +0000 In-Reply-To: <20230707201904.953262-1-jiaqiyan@google.com> Mime-Version: 1.0 References: <20230707201904.953262-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230707201904.953262-2-jiaqiyan@google.com> Subject: [PATCH v3 1/4] mm/hwpoison: delete all entries before traversal in __folio_free_raw_hwp From: Jiaqi Yan To: akpm@linux-foundation.org, mike.kravetz@oracle.com, naoya.horiguchi@nec.com Cc: songmuchun@bytedance.com, shy828301@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, duenwen@google.com, axelrasmussen@google.com, jthoughton@google.com, Jiaqi Yan X-Rspamd-Queue-Id: 81213100019 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: higm45kds5xg3d9y373jmw5c6iep5x8z X-HE-Tag: 1688761157-781286 X-HE-Meta: U2FsdGVkX1/wiXmecZwMlDf3EEs60aDnHI7CaW+BlQhdh98sPCRu2xB319zQm3qjnm8Q6cnTUN2o3IX9H/fQBx/Su/426ZvcfXeTQ2jBcEHDOjyBA+1KG0WUe4mVOZXsA1TX77zxYvM7M0iPlObDEZvvJw+fbqGQlg2fdV6bjdFX3IAXopX9lW1MZRj7wqvbEehyA0Enw88tq8aZGbAIuQ7+VhYiIji7v9Ugl4AHusa3xX7rybZMgFUi+FI+5bUZRscV155s+mSHOrPyK/0Rm9oe7TdZskhdeR2bXmLMpYXVym58VCambFBuL08SN5uFvlza9b++ATfvTfQNRzDzLMFP+S8sFur1cuNt3pFkCHk2DtDhjjGmSVAMNLXKYPPql6JPQXzvU5JvkIcpLhg3YRhfM8B4eF4t3j88MunKSpbXpw1tFXIa84cIIN9IlrJuEoWj0X7koKgeHfwJfExepm73eJj3kAPNiwRW54LKCT0/sXnjvOmutE3BrFvYEV9nFXFOBGBYxqtWtgLsPmXVyz7qcrIoSQBPwjGMmJDqVxK+VpLSb6WvZy8ZDfSbaH0ThUNzh/x7x2nvvf6iPXm3sBOka0fBk/bcEbVsn7HnwyuHdiqp2ycvk+n2jDR+6aEi1BgmBVNrL7Qsh0Yr9rdhJHnPs8/Z/WjgrRCH0ffp4+o8XDbWj4MurKJ4KXN21ijeKQ4oGbq+RieDj0p8sVOORkR+fLw9QtHY4QtBmLKiEbWSaia968JooHEtL59dKVuiiq+8dAWJ3YL6agIaETCwFjwNiYvIISuvaWxBzOrSJ0agon2roruyklU14G5NLeYN75Pz/3FxygwcN475Rrtlftbg3jJCn/S/FhGowcDUifX9TvX5OBqDhdbIZn7sv/d7YhYbGFpxdY8O4wlsuo80mg2goW88OZzXgivW7qAEA2JY8lCldWxn49TikSulvVq2VPyCg5nrIeAWEc3y4jQ EWQTyn9P 3yMSmoV41T2NJ1/+5Xe/+n+vEyoPUHvwhumoH7nIWlMp5IArLE1J2tVsthg/4Dc85GEjB6sCWcBzYPz9LlbPQQxRMwoTSZFeFwzkLWEbVhUvm+YGgWt1FGlzYWIgxlXPd4RKNBuyH08iJ5O0+3oK6Eo1AaXFHV94Z1SX4O5pmEoQT+VBQeR5tl+yWl3jACHfnVuhbMro7SrMCeX/MlTnpCks75WVMbpzs8YHPC4QMopN6n3l1INf5bOx9Tmido16oSMyrKoXxz1fSH8yCoFggiAbveRKgjgN5etAPa4CfG13JobhefkNJP4tGQuV2f/Cb8WEa578pJpOLI1/kRQ1pGHGtIXwSTLN5Dlc9t5yBgTYG5Orow5eIGi4qV0mRAP6JhzuMd+JQRRIZmSObwUnTqK/+9fUYlZ2KCJKNenoBbsCTyCgoHB/JVDLy/JcgZ1kYEyPTQq8zo67jE2gDGt1s79WlAi+9/5EKXttII92g2Pmep8b30NapVdlPt/GNb52FkEVY5/UA+XFkuuWy01hIZ5EtjLjBpg+2BhcPyLWwBVWPSbUECOG6T6OBr+7hMili8o5oU8rVryVnVgwPNHhdUPV4Css9r4zTcOgN2/Dhnr7gxHHbdm2DFXoFVIAwVd9Kx28L/kfxP2/D564KhliNg5r0+bCVtLizvCNDTsh/wGElfIXr+x2juq2LCiocMqhiCk4JR6a85utSQPatLni4V+xqKw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.002003, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Traversal on llist (e.g. llist_for_each_safe) is only safe AFTER entries are deleted from the llist. Correct the way __folio_free_raw_hwp deletes and frees raw_hwp_page entries in raw_hwp_list: first llist_del_all, then kfree within llist_for_each_safe. As of today, concurrent adding, deleting, and traversal on raw_hwp_list from hugetlb.c and/or memory-failure.c are fine with each other. Note this is guaranteed partly by the lock-free nature of llist, and partly by holding hugetlb_lock and/or mf_mutex. For example, as llist_del_all is lock-free with itself, folio_clear_hugetlb_hwpoison()s from __update_and_free_hugetlb_folio and memory_failure won't need explicit locking when freeing the raw_hwp_list. New code that manipulates raw_hwp_list must be careful to ensure the concurrency correctness. Acked-by: Mike Kravetz Acked-by: Naoya Horiguchi Signed-off-by: Jiaqi Yan Reviewed-by: Miaohe Lin --- mm/memory-failure.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index e245191e6b04..a08677dcf953 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1829,12 +1829,11 @@ static inline struct llist_head *raw_hwp_list_head(struct folio *folio) static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag) { - struct llist_head *head; - struct llist_node *t, *tnode; + struct llist_node *t, *tnode, *head; unsigned long count = 0; - head = raw_hwp_list_head(folio); - llist_for_each_safe(tnode, t, head->first) { + head = llist_del_all(raw_hwp_list_head(folio)); + llist_for_each_safe(tnode, t, head) { struct raw_hwp_page *p = container_of(tnode, struct raw_hwp_page, node); if (move_flag) @@ -1844,7 +1843,6 @@ static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag) kfree(p); count++; } - llist_del_all(head); return count; } From patchwork Fri Jul 7 20:19:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaqi Yan X-Patchwork-Id: 13305252 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4B3DEB64DA for ; Fri, 7 Jul 2023 20:19:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4B4486B007B; Fri, 7 Jul 2023 16:19:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 464408D0001; Fri, 7 Jul 2023 16:19:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2427E6B007E; Fri, 7 Jul 2023 16:19:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 155F06B007B for ; Fri, 7 Jul 2023 16:19:21 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id DBFCCAEBD8 for ; Fri, 7 Jul 2023 20:19:20 +0000 (UTC) X-FDA: 80985930480.18.2E057BE Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) by imf09.hostedemail.com (Postfix) with ESMTP id 02D7B140012 for ; Fri, 7 Jul 2023 20:19:18 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=Lm821Q9A; spf=pass (imf09.hostedemail.com: domain of 3RXOoZAgKCPgonfvn3fslttlqj.htrqnsz2-rrp0fhp.twl@flex--jiaqiyan.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3RXOoZAgKCPgonfvn3fslttlqj.htrqnsz2-rrp0fhp.twl@flex--jiaqiyan.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688761159; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lhHFjAXtuMYeiDPTb9enUa1sagPmUht2lF56qHA5nAY=; b=OeEfjD9di87XrkLbIIVCh2gm5t0mDsSgXeEC8KsmJFaIjLYvYmY2Hdrj7sYglmdSKrtMFJ pDUgBrlX6g6gtyAcU8sJ8MDbT8bwIbaG4NQI41jbsm3+wpC4xHPXEOv2FOm7/ePbhRz1tU HMoYGdw4ND/4OMPDW0yq80b7TAFbkdY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688761159; a=rsa-sha256; cv=none; b=Bv4USIkwt/eNAn6ny18I8qDseyJ3OTv15U9xr04gRKVr5jkFcrmdDsFh69dqM4T+Mikxi4 PSzNBDT6wwna7TxaQ3x07NhlT8erl2Uw8lKI6HP1bjxmZOMItwbzVKHAURQYp6+B04knNH 16aF9f+aUVlr+/D6G5iMwoG+3wy6A78= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=Lm821Q9A; spf=pass (imf09.hostedemail.com: domain of 3RXOoZAgKCPgonfvn3fslttlqj.htrqnsz2-rrp0fhp.twl@flex--jiaqiyan.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3RXOoZAgKCPgonfvn3fslttlqj.htrqnsz2-rrp0fhp.twl@flex--jiaqiyan.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-55bf2bf1cdeso3172003a12.3 for ; Fri, 07 Jul 2023 13:19:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688761158; x=1691353158; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=lhHFjAXtuMYeiDPTb9enUa1sagPmUht2lF56qHA5nAY=; b=Lm821Q9A8z9/jwUhSfYSBmBg3a1CJgB8uC9S29RR8iamr7rpAYSfO0Eeq787JeWICj aSBaqXte1TU+6+8Sh0EwU0DomQ2dskj2yaXJhdXbMBNjJ8QUuqx8PFueYsGG0LTCGMQM AG5oIBFVs6KD00JEaeesIKk4hiB7jSUeRJ5U9ZVeA4Xg20DZQrowBV2mwu3oDdpyaXg2 VSPJ+oq7NOFgZNlDG3dULstPjbk7FgX8TBv4efKwHSj4dG2SHkzR24FKkM89NmGdxa+V ORZx2FMfNdL9uzR1qedaRjJLe7tuSJorO9cCwuwQmen2ZaI5XXih1iy8y9odfFxhoErc 4nvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688761158; x=1691353158; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=lhHFjAXtuMYeiDPTb9enUa1sagPmUht2lF56qHA5nAY=; b=g8IiFpL/cQ/SdAKbTjF0ilhM8WRHT7g3KNR9oUgttBz14Wu+rRQCy+6u+OXsV/3V2v b2nIVLHbMKEJt3ogbYnfJ/tMfhOT2li6Z71+DZ8taxQgji6uSSb4pQf5yjkHAS3O/Igl xPru/+QGAeRqYz8yF58UKqsHO6X+F9rE2vmleiG6QH/am3bukCTn6RasUTPCIMvd6Ol7 6Gs2BhZSRFp8KwsUexIKe1T39WU7+fEYJ3/MSF/K6CG6nswtXvSLFuWUoBQNxBQSDG7/ zF6CYXvVNpn1cRPTq16VbZt2jjy0cnCgoF719lgsEvU+EKLbzsOeSg2symZkUIBk2j+T 2cuQ== X-Gm-Message-State: ABy/qLbrRfqLmvtGdo7hr7aCIV5t6blT9Oq6Y88PTV9rUI+YOwUOTRQz cbuJtSlUbjMU3mUr1SasPqAijxPidH2sKw== X-Google-Smtp-Source: APBJJlFoY61HjNHb0rFEYpqM/xY1hMlpJ1lQz10syaUR+Qp6bGufuof79IX3dfbrgyCAU0fML5WpoZTLRKI8TQ== X-Received: from yjq3.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:272f]) (user=jiaqiyan job=sendgmr) by 2002:a63:7f51:0:b0:557:5649:381 with SMTP id p17-20020a637f51000000b0055756490381mr4012081pgn.3.1688761157717; Fri, 07 Jul 2023 13:19:17 -0700 (PDT) Date: Fri, 7 Jul 2023 20:19:02 +0000 In-Reply-To: <20230707201904.953262-1-jiaqiyan@google.com> Mime-Version: 1.0 References: <20230707201904.953262-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230707201904.953262-3-jiaqiyan@google.com> Subject: [PATCH v3 2/4] mm/hwpoison: check if a subpage of a hugetlb folio is raw HWPOISON From: Jiaqi Yan To: akpm@linux-foundation.org, mike.kravetz@oracle.com, naoya.horiguchi@nec.com Cc: songmuchun@bytedance.com, shy828301@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, duenwen@google.com, axelrasmussen@google.com, jthoughton@google.com, Jiaqi Yan X-Rspamd-Queue-Id: 02D7B140012 X-Rspam-User: X-Stat-Signature: qrsybqkfqmmkn7joemcy4my1n7toff9x X-Rspamd-Server: rspam03 X-HE-Tag: 1688761158-276183 X-HE-Meta: U2FsdGVkX18fAtlur9ul9QZA/GigGDEkqaF8aMddJhbAk1GtnREfEIVxagfRexOBY0IC6UDHURZud4Dj4QU2U7W97aB8npgyZr3b/OpVEtNSIMITLoik18Ct/OpXbldnyxY47yw/XSJYoL+iemfgYESW8YdObVn3pqCcasNRsZ4HcV6hL8mWuIYtcTFZjlNbJXATHJc1UbGD67oHqUMfUyD/Wv3h/CfM0czOBBO/m2S8uGq1JqhJZHVmF4mnSZQxZ1MaFxYidOOrCb4eWjIu47PVw4XADiMiaDd5TutwoaoOkQ/RevLJRpglLTmPfLi12P6UX3jgYxEYeU8lC2oprhPbs0zKslLTHh48uVPCeuuELApPWZb9HXH1NvBPfXEabj4rVZMtQ9yk7gOpBSHgGpgk7i0mxjMZmSemHsTOsZHuRjCh8lKIdVcDsrFNAfKlOHnaJTl+kfbtmkfrb0TAGlrBqu0H20TWpSk4IKdvU7XjCDcvXooW55XZAPsPnM8dLWpky+2F4QDcnQAjJZODVInMZE8m4eA9JjWhVx4bsKgAL56ry6Mbj0cVKY16SLvW2AaYRQNw8I1UJ8JKnMzEI6p8ExzGufAdM3yLATMM3w8LknfnQTAYtWujMhxvO3Pbfp3nqi41kBirPpX8mncRbNUPjJMnQBZ77nIqm0vo06qojvO4KNFaqaGE6Fp0IfXlFelH+B+u3fEnFtoe7HI6p7ugFkgilOtxULh1Tp3wdebLXvdQi3+XgobuaJMAXmGzREDnsSQam6Yv64ZyPGA1buAS08JZJP+eyM9Gs7rnTUb/V9vOurqPI47JjPe/Z8Q+dqoqTv3svf/lIdhP5+eRe49KCRP/bLgHdmp0TenLIHJGwSNVbEc9ege9xP1p86sjP7anNgnAhwCklQWZ2v2UBtTTkGE85UfdIqcdhNqBBYefWOLOP08TfUR1py9Ae59vFnOvn7RvXETkZhavAMQ jw+yzjdS wdoHlVoo2pWM6312AgrnmRiuOIdkuLvy5sx7VcSQNXwQkZ31a9XMxL9EosPFoxOTTKG7SrONHmUe/PQkMT1WJvHeZh3d3W+FCG5oD07STTs3wWm3a0YG48MKHo1w97mZlhhdjOf4EDesEYTjrET2uZysuZ2RipevoG2K8hEr4qVeOqyZ3tjir5d6Vx0anIzbR5q5s5FwR5mwWsaSZem8CG3qblSXWYAKUkTYoghmGv2cikAFRiI725fZzfQWS/mfz0XmV606qyNajeCkR+03k6S/i876MxdD/Fm0DuC5vgatk/eHELE4NN66Eb/68SUNWwwC2o9O5usIPKwH2EekLBFhgwfM2IdQCuKkAicZWPI6XzqiA2BBWvuBohyAhblnC/6Ht03VaPZXnuFOv/Ju/8mwsPftpYERS0rxyMXFq6HMCL3X8vRQRw0UFs6WaK/aq6no/bV6NGe4QQIpGkIsocSm5A8smZ26yBQn/V514uRp3UGNLIuUrlv3HKrAo74R6LhybbxVsQUbnSpP7+aDPAD1H2kA/a915IWq3Pr1jpcTNuZoRAAxn+azhTrSmfnHI9mWPJxVRaGbRUwa6JZWhhjt4co2x/NiDd2MZWvf36rOC0pLKzFXcAq/KkS7QgpAwEclaI/gR0WVELnqv5AuMZEPiCfNasyo4mOrhcrWZ1Amen+o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add the functionality, is_raw_hwp_subpage, to tell if a subpage of a hugetlb folio is a raw HWPOISON page. This functionality relies on RawHwpUnreliable to be not set; otherwise hugepage's raw HWPOISON list becomes meaningless. is_raw_hwp_subpage needs to hold hugetlb_lock in order to synchronize with __get_huge_page_for_hwpoison, who iterates and inserts an entry to raw_hwp_list. llist itself doesn't ensure insertion is synchornized with the iterating used by __is_raw_hwp_list. Caller can minimize the overhead of lock cycles by first checking if folio / head page's HWPOISON flag is set. Exports this functionality to be immediately used in the read operation for hugetlbfs. Reviewed-by: Mike Kravetz Reviewed-by: Naoya Horiguchi Signed-off-by: Jiaqi Yan Reviewed-by: Miaohe Lin --- include/linux/hugetlb.h | 19 +++++++++++++++++++ include/linux/mm.h | 7 +++++++ mm/hugetlb.c | 10 ++++++++++ mm/memory-failure.c | 34 ++++++++++++++++++++++++---------- 4 files changed, 60 insertions(+), 10 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index ca3c8e10f24a..4a745af98525 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1007,6 +1007,25 @@ void hugetlb_register_node(struct node *node); void hugetlb_unregister_node(struct node *node); #endif +/* + * Struct raw_hwp_page represents information about "raw error page", + * constructing singly linked list from ->_hugetlb_hwpoison field of folio. + */ +struct raw_hwp_page { + struct llist_node node; + struct page *page; +}; + +static inline struct llist_head *raw_hwp_list_head(struct folio *folio) +{ + return (struct llist_head *)&folio->_hugetlb_hwpoison; +} + +/* + * Check if a given raw @subpage in a hugepage @folio is HWPOISON. + */ +bool is_raw_hwp_subpage(struct folio *folio, struct page *subpage); + #else /* CONFIG_HUGETLB_PAGE */ struct hstate {}; diff --git a/include/linux/mm.h b/include/linux/mm.h index 74f1be743ba2..edaa18b6f731 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3680,6 +3680,7 @@ extern const struct attribute_group memory_failure_attr_group; extern void memory_failure_queue(unsigned long pfn, int flags); extern int __get_huge_page_for_hwpoison(unsigned long pfn, int flags, bool *migratable_cleared); +extern bool __is_raw_hwp_subpage(struct folio *folio, struct page *subpage); void num_poisoned_pages_inc(unsigned long pfn); void num_poisoned_pages_sub(unsigned long pfn, long i); struct task_struct *task_early_kill(struct task_struct *tsk, int force_early); @@ -3694,6 +3695,12 @@ static inline int __get_huge_page_for_hwpoison(unsigned long pfn, int flags, return 0; } +static inline bool __is_raw_hwp_subpage(struct folio *folio, + struct page *subpage) +{ + return false; +} + static inline void num_poisoned_pages_inc(unsigned long pfn) { } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index bce28cca73a1..9c608d2f6630 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -7373,6 +7373,16 @@ int get_huge_page_for_hwpoison(unsigned long pfn, int flags, return ret; } +bool is_raw_hwp_subpage(struct folio *folio, struct page *subpage) +{ + bool ret; + + spin_lock_irq(&hugetlb_lock); + ret = __is_raw_hwp_subpage(folio, subpage); + spin_unlock_irq(&hugetlb_lock); + return ret; +} + void folio_putback_active_hugetlb(struct folio *folio) { spin_lock_irq(&hugetlb_lock); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index a08677dcf953..5b6c8ceb13c0 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1813,18 +1813,32 @@ EXPORT_SYMBOL_GPL(mf_dax_kill_procs); #endif /* CONFIG_FS_DAX */ #ifdef CONFIG_HUGETLB_PAGE -/* - * Struct raw_hwp_page represents information about "raw error page", - * constructing singly linked list from ->_hugetlb_hwpoison field of folio. - */ -struct raw_hwp_page { - struct llist_node node; - struct page *page; -}; -static inline struct llist_head *raw_hwp_list_head(struct folio *folio) +bool __is_raw_hwp_subpage(struct folio *folio, struct page *subpage) { - return (struct llist_head *)&folio->_hugetlb_hwpoison; + struct llist_head *raw_hwp_head; + struct raw_hwp_page *p, *tmp; + bool ret = false; + + if (!folio_test_hwpoison(folio)) + return false; + + /* + * When RawHwpUnreliable is set, kernel lost track of which subpages + * are HWPOISON. So return as if ALL subpages are HWPOISONed. + */ + if (folio_test_hugetlb_raw_hwp_unreliable(folio)) + return true; + + raw_hwp_head = raw_hwp_list_head(folio); + llist_for_each_entry_safe(p, tmp, raw_hwp_head->first, node) { + if (subpage == p->page) { + ret = true; + break; + } + } + + return ret; } static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag) From patchwork Fri Jul 7 20:19:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaqi Yan X-Patchwork-Id: 13305253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2898FEB64D9 for ; Fri, 7 Jul 2023 20:19:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B1E746B007D; Fri, 7 Jul 2023 16:19:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A09CE6B007E; Fri, 7 Jul 2023 16:19:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D1E86B0080; Fri, 7 Jul 2023 16:19:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7F2646B007D for ; Fri, 7 Jul 2023 16:19:22 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 461D3C0108 for ; Fri, 7 Jul 2023 20:19:22 +0000 (UTC) X-FDA: 80985930564.10.8E77495 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf10.hostedemail.com (Postfix) with ESMTP id 693D1C0016 for ; Fri, 7 Jul 2023 20:19:20 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=TldvMxXO; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf10.hostedemail.com: domain of 3R3OoZAgKCPolkcsk0cpiqqing.eqonkpwz-oomxcem.qti@flex--jiaqiyan.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3R3OoZAgKCPolkcsk0cpiqqing.eqonkpwz-oomxcem.qti@flex--jiaqiyan.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688761160; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=A+DZafON7JsoL7Ay0CKBI9HDnuWj0nfz7I/atmY5O6E=; b=f/ysoRPH0NsLjWulSSQvvY0fphXU9Fj8mv2zaHzXX5J1oDAjEd0JT1kBotn0x1yHcrrLSx a0/7rWrq5IDsWd+StBEGaaPor2McY8s2q7bCnv76rjk335K7pj4WluAZY22XSbKK9AweJ0 Cbc4k9f2NZ+Is77195Ua+t97gmla95g= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=TldvMxXO; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf10.hostedemail.com: domain of 3R3OoZAgKCPolkcsk0cpiqqing.eqonkpwz-oomxcem.qti@flex--jiaqiyan.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3R3OoZAgKCPolkcsk0cpiqqing.eqonkpwz-oomxcem.qti@flex--jiaqiyan.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688761160; a=rsa-sha256; cv=none; b=u0b/seMto0mkZh72hns8bXAFC0Qt04LNz5ThDldGxPmkoyoQiq1s5bGBtyuenRWsGOsumw eXybaDvz4DYIpwFP+9hGx7RWP5MqVqTVQhPaGZvxXPxH27x8GJxK2px4pv+H8ove3yh1aY MBm+Bf+5jo5cVWs7DuKadLqHUw/O/x4= Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-262e2cb725eso3744419a91.0 for ; Fri, 07 Jul 2023 13:19:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688761159; x=1691353159; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=A+DZafON7JsoL7Ay0CKBI9HDnuWj0nfz7I/atmY5O6E=; b=TldvMxXOlAitzanCYBdJPKTFsceaBUtZtfHE+NCMC7uRLSuEMdNFphYxZysQPlOWyt 6IIkCSmbZ0QhAj6BdA2S0ZbkwbhR7x3BX+nwYBGgaOxeyqB+HsmP8eFpNEMX/EjaKoHw Osqpj/mJ1MqPaQbsakEeDVTjMuhGWE9GnXwVc8F/i6rZzYOS2/oL2zNYYKzVVfG8aSiy zLaZC2KHe+RKfCbvOVNYPOcE1YOJUR4cbXfL9Nm1YDmuYA43h0rfivcjthaU3URwFYSK 96WU5M8uzwqIi+5+ywPkqjzgjsM0fGa0w0GzRv5r4kHftPEO8J5DvOvCOoUqVSHiN5aw 6/8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688761159; x=1691353159; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=A+DZafON7JsoL7Ay0CKBI9HDnuWj0nfz7I/atmY5O6E=; b=TvH1D5XCEWnzXMUcp4HsQus8cMMihba6/M1cWrnpo6GWauJExxpNCHuGvW07/8TJ/2 ipaTIWVOSUujSjFC4388YJLRP/rUQbX/upMDQFHJA6A6E4d4s+Ti7CrB0HDZji1QCCoG ZC/8rX14SG2WuOotX8JIRbdqVw/q0aey6RyOthv5KGjvO8fAFjI2aDUHHMe2MxdTz24x zW3FJA6J+ACcYLNUEHB4fL2CKj6GzKPz0HTfFzKt1RHU3bIXXZxzaWUMNBGs1rFOlzuJ zGtluNKTmpcCrzHcnGJPqGeo7MAvJ1FxflqRnLwL3unaEsAOqmteRPIQnzuSDkyeoGVa I+/A== X-Gm-Message-State: ABy/qLY2eQYJ5WDPClpIkYTj99npL+4Utjo9ivl4QODz3WHtgGk+s/Pv 6XcX1YWnYYDKIT1nN+pS7bLWTbujc+DeXw== X-Google-Smtp-Source: APBJJlFlQ7RKBImKNNS3u3h3d6OO1HQ+Hd+V+iBkTossPJ4vugefj5v7Rlyl3uxPne+VgqOiM11xG9BXaRtiDQ== X-Received: from yjq3.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:272f]) (user=jiaqiyan job=sendgmr) by 2002:a17:90a:ce18:b0:263:49d3:8024 with SMTP id f24-20020a17090ace1800b0026349d38024mr4778324pju.1.1688761159291; Fri, 07 Jul 2023 13:19:19 -0700 (PDT) Date: Fri, 7 Jul 2023 20:19:03 +0000 In-Reply-To: <20230707201904.953262-1-jiaqiyan@google.com> Mime-Version: 1.0 References: <20230707201904.953262-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230707201904.953262-4-jiaqiyan@google.com> Subject: [PATCH v3 3/4] hugetlbfs: improve read HWPOISON hugepage From: Jiaqi Yan To: akpm@linux-foundation.org, mike.kravetz@oracle.com, naoya.horiguchi@nec.com Cc: songmuchun@bytedance.com, shy828301@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, duenwen@google.com, axelrasmussen@google.com, jthoughton@google.com, Jiaqi Yan X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 693D1C0016 X-Stat-Signature: cs8gn377atgbqecg3qkmpo7bz7jcnkgp X-HE-Tag: 1688761160-209529 X-HE-Meta: U2FsdGVkX1+8u7NR5YsJ9XKLF6secW/8ZXqvKsv0ldBFoGmdrhWlgr2Tr8KFfVvARC9D5dsvtXZ3VOjsof3rGpxopGJ+Gqjjp4KKGPV3YcNok6ZIQGaWrZQu7Trs/q31oS/oRlR1XQ6JNp/mwMax0Bfdtt5+oVioqjClaPAIWHDQA46fk/aHLjocj6dc9ilpI+0A2v2+/dOxp5bQa1LgNpbeIrvHDaRQcVBpvRPELa+a01nK9un0W2eiwoNzOqp0SR5GeysuEyfct7/ha3kMY/qvR+E08UpQA6Fz9SIR2tH+gJzTMX4B/WPlA3ID4+07YToEJe8pBRQqbQDNCHHiO5gUZizZzddLidpbyQX7ydModaveuKTGaSehAT/hwB1GJ2LpML2nWy6+KNb6+OZCTbzmC2CPS+T9P3UJxLZkADqgBcHt2uKSX5jOdq0kp3dJtMul4k6QDm9LSESBuM66xeO6CDrKBUEFvbvXMmWQtc22ZBgrgkPrGV7/xJTZDiIgImHtPcVv4kBuIXH/AQnEPbO9SWUkhb0mpg9yO1wf1HmoTkCnJqzTnZ/hVZugpXIv5CbOojctJkc/cHcoWGtLtTWpTcUKCRfjP/2Xt1UhRg4jhvW0ZhsbPlCmF8bnuU7MuOWFnbQepmXPlSw/O+CB3XE3ca1xxCFmVvd0TdC94821mvOZxHksAry2u4COz1vrJqpH3ChjfEkbloiCMap5PdpgXZbs8eoydsSrM98KyxN+xtoDyDdbTw2X0arwey5cz/6+3HUgfHRdndzHmnpKhvXykAKT6UDqcORKW6NLNnxKO49CqVMpSWwXikr5tch85t8FxDwmVGQrM9osO6wYHNZQotvnxZJkECSajZhCDN4Wxr3IpgXgrLFomD8oTcDvW4lsonQ7MsbJpXY2vbiJk/VpK2y2ttqTPvLH08wV7d88rRpeYtjf2b7mjproUJTMeS9wC56pRFNmkVmHJbL Lsh2Qi+j 5cKodTW/FYotcI+b7dwzO2D7HITx+f+9xmne5Unjr7bjBQf4tb19YqkBFN6QpzNwmo1HuYoU10k3oPMN6WNcn7TlMeZhke3D6IIu1maO6Aix4+JksM10IvvgTzwYBrXkS2YniDt2YI8Hh7E9RYRLMQzVxTRszBbxCmk81cj/gd1kQ2nMRGHqX3zZAN0338LxZk4erfDbBYZRFHl8NemNC7uQqxmy1dVzjqU3UbAm9eAoWfCDNi2uxfAOGiBHNNmM/iDDfxRry5oKGYubIdPTD04xh5fk/1zWsDsB4dE3Hfp64upDxod+LDB5T0bc5/+0T7ll1cjVYZC9QEizVQZNzianMxhZPtb5F8YU+kazAlEWrqg53UMZE1104agBGPpAfL14U5NSqR/xPkcM/upb8t+1LMOS34nPPqNNoSlUeh1VNxqPxyhCR/W/uZcDpB2dtWzLWn6ue4oG42AlGoixYhk2EG09jPSvDyB9NQ97VfRiDBJ8DDmyZLhax55Io6+IamC60KeuUj28GqyBZ0rWBUZgIWVIATQw6gD52ctGU5eJ4Yxc5adAc1J9eUDZY0sZGokalhgtbtPY5lXASSVl1pGDzstVjOaLUyTVI8+9OGW2FPVTgxV8PU3PVvgpdg0+t39RviqT4en8Gy7AZgA6ogoZctiF3YHr1nhGB9bRU9XTYLls= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When a hugepage contains HWPOISON pages, read() fails to read any byte of the hugepage and returns -EIO, although many bytes in the HWPOISON hugepage are readable. Improve this by allowing hugetlbfs_read_iter returns as many bytes as possible. For a requested range [offset, offset + len) that contains HWPOISON page, return [offset, first HWPOISON page addr); the next read attempt will fail and return -EIO. Reviewed-by: Mike Kravetz Reviewed-by: Naoya Horiguchi Signed-off-by: Jiaqi Yan --- fs/hugetlbfs/inode.c | 58 +++++++++++++++++++++++++++++++++++++++----- 1 file changed, 52 insertions(+), 6 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 7b17ccfa039d..c2b807d37f85 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -282,6 +282,42 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr, } #endif +/* + * Someone wants to read @bytes from a HWPOISON hugetlb @page from @offset. + * Returns the maximum number of bytes one can read without touching the 1st raw + * HWPOISON subpage. + * + * The implementation borrows the iteration logic from copy_page_to_iter*. + */ +static size_t adjust_range_hwpoison(struct page *page, size_t offset, size_t bytes) +{ + size_t n = 0; + size_t res = 0; + struct folio *folio = page_folio(page); + + /* First subpage to start the loop. */ + page += offset / PAGE_SIZE; + offset %= PAGE_SIZE; + while (1) { + if (is_raw_hwp_subpage(folio, page)) + break; + + /* Safe to read n bytes without touching HWPOISON subpage. */ + n = min(bytes, (size_t)PAGE_SIZE - offset); + res += n; + bytes -= n; + if (!bytes || !n) + break; + offset += n; + if (offset == PAGE_SIZE) { + page++; + offset = 0; + } + } + + return res; +} + /* * Support for read() - Find the page attached to f_mapping and copy out the * data. This provides functionality similar to filemap_read(). @@ -300,7 +336,7 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) while (iov_iter_count(to)) { struct page *page; - size_t nr, copied; + size_t nr, copied, want; /* nr is the maximum number of bytes to copy from this page */ nr = huge_page_size(h); @@ -328,16 +364,26 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) } else { unlock_page(page); - if (PageHWPoison(page)) { - put_page(page); - retval = -EIO; - break; + if (!PageHWPoison(page)) + want = nr; + else { + /* + * Adjust how many bytes safe to read without + * touching the 1st raw HWPOISON subpage after + * offset. + */ + want = adjust_range_hwpoison(page, offset, nr); + if (want == 0) { + put_page(page); + retval = -EIO; + break; + } } /* * We have the page, copy it to user space buffer. */ - copied = copy_page_to_iter(page, offset, nr, to); + copied = copy_page_to_iter(page, offset, want, to); put_page(page); } offset += copied; From patchwork Fri Jul 7 20:19:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaqi Yan X-Patchwork-Id: 13305254 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5E77C001DC for ; Fri, 7 Jul 2023 20:19:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 32F9C6B007E; Fri, 7 Jul 2023 16:19:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2DE526B0080; Fri, 7 Jul 2023 16:19:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1315B8D0001; Fri, 7 Jul 2023 16:19:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 0503B6B007E for ; Fri, 7 Jul 2023 16:19:24 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C8231A0104 for ; Fri, 7 Jul 2023 20:19:23 +0000 (UTC) X-FDA: 80985930606.29.F1CD169 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) by imf28.hostedemail.com (Postfix) with ESMTP id C9E25C000D for ; Fri, 7 Jul 2023 20:19:21 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=cgDoU3tD; spf=pass (imf28.hostedemail.com: domain of 3SHOoZAgKCPsmldtl1dqjrrjoh.frpolqx0-ppnydfn.ruj@flex--jiaqiyan.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3SHOoZAgKCPsmldtl1dqjrrjoh.frpolqx0-ppnydfn.ruj@flex--jiaqiyan.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688761161; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dmSXGhNgjtW0Yb8+yIyK70NmvAjyR1vjV+RIYl0ULq4=; b=aKtKpWhgxJu0iS6t5UYSRLcDjFfgkEsobqSliyCC7xTsxmsTtH+vazBDpBuKePKofQTv/n gSR2kV8yD5Mi4Y5vyDRFWNfV8gv5iIMxmKmDXQhzlnWA/hpYcwCmbG+KI7LiwxJFDRZzqd M4YkY+RHxjlrKpVPjO9ZXhmVnyr9mu0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688761161; a=rsa-sha256; cv=none; b=Gh87O18xz3Tus+o7BZUSEXw6hmS8cnc11IfLfUxnU+PIwAmHnrN3Tsk3q/Xh0qH7Fm1giX WXViyVt18CSfIgPkjZPehmzaqYeSw9K4SGs2vXmDP6zk6niauoxPvoHPc+CkEq8J+VJYrQ r0RpTi2PAlVSdYheFynwnFydPtMHZR8= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=cgDoU3tD; spf=pass (imf28.hostedemail.com: domain of 3SHOoZAgKCPsmldtl1dqjrrjoh.frpolqx0-ppnydfn.ruj@flex--jiaqiyan.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3SHOoZAgKCPsmldtl1dqjrrjoh.frpolqx0-ppnydfn.ruj@flex--jiaqiyan.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-55b9ed8275aso2823226a12.1 for ; Fri, 07 Jul 2023 13:19:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688761160; x=1691353160; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=dmSXGhNgjtW0Yb8+yIyK70NmvAjyR1vjV+RIYl0ULq4=; b=cgDoU3tDkKP7JTQ+F0CzAiBgvzjb4qUPJbX80bmEdZNC2kBFPhSxB/2KWqZSIhGnu5 DNnicuDpK4RRjfvI8Od0feJbKV0/XU2e63+LYPuJBKE3rThjUkLPqnYx6RjItC6itmd6 E8oybw7gnCOvuqj/lkQyUEa8CGnU/e60XfcGhlrIjKezX1hZaq+Fbq1wQe5hiMJLfSxY 04YtCgIm9GcwTklBS9EFbPw5TLQgXGtx3bb42RDjvnpfonqnq7jboBzGk/LZy1us4JKo w+8/QeI5pWZgUe4X5h5uG4tRJUsMzcCNRRNm03kEX6WEvTDa0+st3K25FuKUsfWelq3J yj+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688761160; x=1691353160; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dmSXGhNgjtW0Yb8+yIyK70NmvAjyR1vjV+RIYl0ULq4=; b=IXarUKc1oF4qB0Yb/sgKdo3bP8LgsJ0FvWEiKNv2f7ueOzSDVs9RD4kA2MTo/AsqlW tunNGVP/Zry30RNUi/tjRvZEZFfcS3pyZbyAv/0KFGjE5B44uV+MeSqA7E8cNKscPTD4 mB+n2VDrcfyQljkHfGu1UsxrASgTsXCKapFsLCr4sQBtvwp6aCZfKEndpWTJaYcxNB3Z 0s4zdRIzk2iw42Nvu6M0g9QwiihW3fd9ZZKaNLov4owcHRQZUV7o7aHL1HawtjXf2TAg oxRLEwwXFVLPjBXLJhvCct3WY56h99Um4s7OehYjVNGqsSlWE6CE5q85pk+36YSiN7LY 5RRg== X-Gm-Message-State: ABy/qLb+jQqA4+xLiIgBxgfl4hV6Ai+0mV+W1L6tL20/yguBzF9VzLja LpR4Ihu3chQwsGRjN659T4LT6rwis/ia9A== X-Google-Smtp-Source: APBJJlH7Cq0eDn70cDDe92uconHq1/8/0rJBg2JnBViw/2FmoEOJgxc84w8NuFGWdffzBFiFeO6rajwXkfI/1A== X-Received: from yjq3.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:272f]) (user=jiaqiyan job=sendgmr) by 2002:a63:9316:0:b0:53f:f32b:1f20 with SMTP id b22-20020a639316000000b0053ff32b1f20mr4028211pge.2.1688761160596; Fri, 07 Jul 2023 13:19:20 -0700 (PDT) Date: Fri, 7 Jul 2023 20:19:04 +0000 In-Reply-To: <20230707201904.953262-1-jiaqiyan@google.com> Mime-Version: 1.0 References: <20230707201904.953262-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230707201904.953262-5-jiaqiyan@google.com> Subject: [PATCH v3 4/4] selftests/mm: add tests for HWPOISON hugetlbfs read From: Jiaqi Yan To: akpm@linux-foundation.org, mike.kravetz@oracle.com, naoya.horiguchi@nec.com Cc: songmuchun@bytedance.com, shy828301@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, duenwen@google.com, axelrasmussen@google.com, jthoughton@google.com, Jiaqi Yan X-Rspamd-Queue-Id: C9E25C000D X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: y45gktamchr7a4d457ueztz5d9m4pwtk X-HE-Tag: 1688761161-356070 X-HE-Meta: U2FsdGVkX1/6Ky8XpAk5YYGXSx2r26tSimcGfexE5kUhymHF4IgKwkZCYm1cbFJy2WokFIs9+KZYb3QHeckOd0kc50e8OWsbzdA2XRG2uzhQ/8TITZAa6lF7zWNjo042VI5nRB3lRneWMCs3NEUTlyaEXqgDT42IPMGig5LSHUiFBP/XgCbJSGmxSiPv8vxVldsigvZ78xY7FjTjZJtmNwWQgRxC5XSZ7GMu4KexTg2Vhhm2K4SxLso4PXu56cNXx8SoVE0iMqRYa9tEN+Ahm9HftZt7fpVlXsrg1tb2Q4rGPx92S4Ocx6VTyuEDG1anxX9yZiGFjv8Ds4egraqvIOe6AGpUTQwHIvfGal7z07PoQqJMBW8XF2TnLpzUXtQKjme0QmdCMx/yLFoct0ExcmFEPHz/5pNVirgbZeaQmR5MOoLKIUzTh6uhhxVvrZAbmUaYIC23wJ338cT3QHoeeqDMW6GhoJ0wxyO9ulwyxSHmyvc7OaO9qcQ6a5dZ9qhWPhtJ8VdENlscgGU6Eq1NbIC+R2AlkpxC8OjeQMjBg+ZFGlFS21qErHnj6fQnhKNyDK7pe9A8MyUT2EYca3Bco9OoSrKF5U0b3LcxNKHgn0uIHDwSpIBS6OXlqoEI5YOoiRBamwwMQigLQnOsPvyX/f+0v5d/s4lNONPaBwVpZ4xYQbdKBoXoqA+Evn/4355zMelt9JT3FmzQsKiMoVAdRJMBDpTsjKoZ2ghmyCNmgEBDijH6WfKd1gAuIWes38mXv7UKWEuVyGF/7i2HlVGnmLAvFMe206T2ymmCbrPyT4MakCEa2du0BoaQYyh3mZuqFTlz0s3Af3enOBXO+SedsZsIx5i5NqLOikuF+309Smctce85gqj7QPzkZ39I9p5FpzmO1FdbCFYzkoLmEtufTheH7j1NDIiWwNCpWnP4uDTScpF+DAiAucuNFqC3gKMinCoDV2nFpdK3NvC4UmB 2Q4MBF/1 92a8Nm28L91i+7fdxnoCfNn/ut11odzQB6JmK7J79MtSVtjJkk/EtRrh9NnshuuiS91nvyC1A+AMoTCE3KK6Ebu9kYS6rSZG63yu5qtoTDx8TciU3vh0Bzv3bf8EPgSWGaZlwVC846s65SwNyIRfr5+YjQRaJzk8sEyrgB+8yPUeixcZQzEsF0GhFQC+owTB+NlRw6h+nKYiVoiLncPqcMDVMjLUll+7zGZqEZI66OR9JWCV0x1CmNGvPM1W1eUsYQBgcnESKanerbngwxwDIL2IsW5JOmfkUIjEZ6tFngm+sU9y0CbYzYBZC38++fBZjR+70o61n7RDG+I6TzjRv8tlOCbI1roCt6dUZg+w7WakOGPQy3az9b9LoTAxNnoyetn78+9yrD0KLNCZ+VNd/fJ5av8uR/QspnX/ApBWKJ2hnVp0ITF56nmdpqlJByyWhHfiIiTmQdN6ZZammkyZNnJ0hl5ND8arJv4ok7BTCBgweWG2VyVj7c8XC/xPMg/3A2kVsULX/7obXh7riXdKJvVmQ7y8/u+HXVa9NuUMpARvI15ZR4EwLVUoitt0RyLwZK8Kibl4qSiaL34xZd7wlwiVfrc/9fkbWFYSr3/Qom/UP+uUADKmfIOckEoqJLu0RpdWsF1PGqsgQTtkdNfy3OZW4WA1/kE5fxrxuTDtqr1yM9Bg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add tests for the improvement made to read operation on HWPOISON hugetlb page with different read granularities. For each chunk size, three read scenarios are tested: 1. Simple regression test on read without HWPOISON. 2. Sequential read page by page should succeed until encounters the 1st raw HWPOISON subpage. 3. After skip a raw HWPOISON subpage by lseek, read()s always succeed. Acked-by: Mike Kravetz Reviewed-by: Naoya Horiguchi Signed-off-by: Jiaqi Yan --- tools/testing/selftests/mm/.gitignore | 1 + tools/testing/selftests/mm/Makefile | 1 + .../selftests/mm/hugetlb-read-hwpoison.c | 322 ++++++++++++++++++ 3 files changed, 324 insertions(+) create mode 100644 tools/testing/selftests/mm/hugetlb-read-hwpoison.c diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore index 7e2a982383c0..cdc9ce4426b9 100644 --- a/tools/testing/selftests/mm/.gitignore +++ b/tools/testing/selftests/mm/.gitignore @@ -5,6 +5,7 @@ hugepage-mremap hugepage-shm hugepage-vmemmap hugetlb-madvise +hugetlb-read-hwpoison khugepaged map_hugetlb map_populate diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile index 66d7c07dc177..b7fce9073279 100644 --- a/tools/testing/selftests/mm/Makefile +++ b/tools/testing/selftests/mm/Makefile @@ -41,6 +41,7 @@ TEST_GEN_PROGS += gup_longterm TEST_GEN_PROGS += gup_test TEST_GEN_PROGS += hmm-tests TEST_GEN_PROGS += hugetlb-madvise +TEST_GEN_PROGS += hugetlb-read-hwpoison TEST_GEN_PROGS += hugepage-mmap TEST_GEN_PROGS += hugepage-mremap TEST_GEN_PROGS += hugepage-shm diff --git a/tools/testing/selftests/mm/hugetlb-read-hwpoison.c b/tools/testing/selftests/mm/hugetlb-read-hwpoison.c new file mode 100644 index 000000000000..ba6cc6f9cabc --- /dev/null +++ b/tools/testing/selftests/mm/hugetlb-read-hwpoison.c @@ -0,0 +1,322 @@ +// SPDX-License-Identifier: GPL-2.0 + +#define _GNU_SOURCE +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "../kselftest.h" + +#define PREFIX " ... " +#define ERROR_PREFIX " !!! " + +#define MAX_WRITE_READ_CHUNK_SIZE (getpagesize() * 16) +#define MAX(a, b) (((a) > (b)) ? (a) : (b)) + +enum test_status { + TEST_PASSED = 0, + TEST_FAILED = 1, + TEST_SKIPPED = 2, +}; + +static char *status_to_str(enum test_status status) +{ + switch (status) { + case TEST_PASSED: + return "TEST_PASSED"; + case TEST_FAILED: + return "TEST_FAILED"; + case TEST_SKIPPED: + return "TEST_SKIPPED"; + default: + return "TEST_???"; + } +} + +static int setup_filemap(char *filemap, size_t len, size_t wr_chunk_size) +{ + char iter = 0; + + for (size_t offset = 0; offset < len; + offset += wr_chunk_size) { + iter++; + memset(filemap + offset, iter, wr_chunk_size); + } + + return 0; +} + +static bool verify_chunk(char *buf, size_t len, char val) +{ + size_t i; + + for (i = 0; i < len; ++i) { + if (buf[i] != val) { + printf(PREFIX ERROR_PREFIX "check fail: buf[%lu] = %u != %u\n", + i, buf[i], val); + return false; + } + } + + return true; +} + +static bool seek_read_hugepage_filemap(int fd, size_t len, size_t wr_chunk_size, + off_t offset, size_t expected) +{ + char buf[MAX_WRITE_READ_CHUNK_SIZE]; + ssize_t ret_count = 0; + ssize_t total_ret_count = 0; + char val = offset / wr_chunk_size + offset % wr_chunk_size; + + printf(PREFIX PREFIX "init val=%u with offset=0x%lx\n", val, offset); + printf(PREFIX PREFIX "expect to read 0x%lx bytes of data in total\n", + expected); + if (lseek(fd, offset, SEEK_SET) < 0) { + perror(PREFIX ERROR_PREFIX "seek failed"); + return false; + } + + while (offset + total_ret_count < len) { + ret_count = read(fd, buf, wr_chunk_size); + if (ret_count == 0) { + printf(PREFIX PREFIX "read reach end of the file\n"); + break; + } else if (ret_count < 0) { + perror(PREFIX ERROR_PREFIX "read failed"); + break; + } + ++val; + if (!verify_chunk(buf, ret_count, val)) + return false; + + total_ret_count += ret_count; + } + printf(PREFIX PREFIX "actually read 0x%lx bytes of data in total\n", + total_ret_count); + + return total_ret_count == expected; +} + +static bool read_hugepage_filemap(int fd, size_t len, + size_t wr_chunk_size, size_t expected) +{ + char buf[MAX_WRITE_READ_CHUNK_SIZE]; + ssize_t ret_count = 0; + ssize_t total_ret_count = 0; + char val = 0; + + printf(PREFIX PREFIX "expect to read 0x%lx bytes of data in total\n", + expected); + while (total_ret_count < len) { + ret_count = read(fd, buf, wr_chunk_size); + if (ret_count == 0) { + printf(PREFIX PREFIX "read reach end of the file\n"); + break; + } else if (ret_count < 0) { + perror(PREFIX ERROR_PREFIX "read failed"); + break; + } + ++val; + if (!verify_chunk(buf, ret_count, val)) + return false; + + total_ret_count += ret_count; + } + printf(PREFIX PREFIX "actually read 0x%lx bytes of data in total\n", + total_ret_count); + + return total_ret_count == expected; +} + +static enum test_status +test_hugetlb_read(int fd, size_t len, size_t wr_chunk_size) +{ + enum test_status status = TEST_SKIPPED; + char *filemap = NULL; + + if (ftruncate(fd, len) < 0) { + perror(PREFIX ERROR_PREFIX "ftruncate failed"); + return status; + } + + filemap = mmap(NULL, len, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_POPULATE, fd, 0); + if (filemap == MAP_FAILED) { + perror(PREFIX ERROR_PREFIX "mmap for primary mapping failed"); + goto done; + } + + setup_filemap(filemap, len, wr_chunk_size); + status = TEST_FAILED; + + if (read_hugepage_filemap(fd, len, wr_chunk_size, len)) + status = TEST_PASSED; + + munmap(filemap, len); +done: + if (ftruncate(fd, 0) < 0) { + perror(PREFIX ERROR_PREFIX "ftruncate back to 0 failed"); + status = TEST_FAILED; + } + + return status; +} + +static enum test_status +test_hugetlb_read_hwpoison(int fd, size_t len, size_t wr_chunk_size, + bool skip_hwpoison_page) +{ + enum test_status status = TEST_SKIPPED; + char *filemap = NULL; + char *hwp_addr = NULL; + const unsigned long pagesize = getpagesize(); + + if (ftruncate(fd, len) < 0) { + perror(PREFIX ERROR_PREFIX "ftruncate failed"); + return status; + } + + filemap = mmap(NULL, len, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_POPULATE, fd, 0); + if (filemap == MAP_FAILED) { + perror(PREFIX ERROR_PREFIX "mmap for primary mapping failed"); + goto done; + } + + setup_filemap(filemap, len, wr_chunk_size); + status = TEST_FAILED; + + /* + * Poisoned hugetlb page layout (assume hugepagesize=2MB): + * |<---------------------- 1MB ---------------------->| + * |<---- healthy page ---->|<---- HWPOISON page ----->| + * |<------------------- (1MB - 8KB) ----------------->| + */ + hwp_addr = filemap + len / 2 + pagesize; + if (madvise(hwp_addr, pagesize, MADV_HWPOISON) < 0) { + perror(PREFIX ERROR_PREFIX "MADV_HWPOISON failed"); + goto unmap; + } + + if (!skip_hwpoison_page) { + /* + * Userspace should be able to read (1MB + 1 page) from + * the beginning of the HWPOISONed hugepage. + */ + if (read_hugepage_filemap(fd, len, wr_chunk_size, + len / 2 + pagesize)) + status = TEST_PASSED; + } else { + /* + * Userspace should be able to read (1MB - 2 pages) from + * HWPOISONed hugepage. + */ + if (seek_read_hugepage_filemap(fd, len, wr_chunk_size, + len / 2 + MAX(2 * pagesize, wr_chunk_size), + len / 2 - MAX(2 * pagesize, wr_chunk_size))) + status = TEST_PASSED; + } + +unmap: + munmap(filemap, len); +done: + if (ftruncate(fd, 0) < 0) { + perror(PREFIX ERROR_PREFIX "ftruncate back to 0 failed"); + status = TEST_FAILED; + } + + return status; +} + +static int create_hugetlbfs_file(struct statfs *file_stat) +{ + int fd; + + fd = memfd_create("hugetlb_tmp", MFD_HUGETLB); + if (fd < 0) { + perror(PREFIX ERROR_PREFIX "could not open hugetlbfs file"); + return -1; + } + + memset(file_stat, 0, sizeof(*file_stat)); + if (fstatfs(fd, file_stat)) { + perror(PREFIX ERROR_PREFIX "fstatfs failed"); + goto close; + } + if (file_stat->f_type != HUGETLBFS_MAGIC) { + printf(PREFIX ERROR_PREFIX "not hugetlbfs file\n"); + goto close; + } + + return fd; +close: + close(fd); + return -1; +} + +int main(void) +{ + int fd; + struct statfs file_stat; + enum test_status status; + /* Test read() in different granularity. */ + size_t wr_chunk_sizes[] = { + getpagesize() / 2, getpagesize(), + getpagesize() * 2, getpagesize() * 4 + }; + size_t i; + + for (i = 0; i < ARRAY_SIZE(wr_chunk_sizes); ++i) { + printf("Write/read chunk size=0x%lx\n", + wr_chunk_sizes[i]); + + fd = create_hugetlbfs_file(&file_stat); + if (fd < 0) + goto create_failure; + printf(PREFIX "HugeTLB read regression test...\n"); + status = test_hugetlb_read(fd, file_stat.f_bsize, + wr_chunk_sizes[i]); + printf(PREFIX "HugeTLB read regression test...%s\n", + status_to_str(status)); + close(fd); + if (status == TEST_FAILED) + return -1; + + fd = create_hugetlbfs_file(&file_stat); + if (fd < 0) + goto create_failure; + printf(PREFIX "HugeTLB read HWPOISON test...\n"); + status = test_hugetlb_read_hwpoison(fd, file_stat.f_bsize, + wr_chunk_sizes[i], false); + printf(PREFIX "HugeTLB read HWPOISON test...%s\n", + status_to_str(status)); + close(fd); + if (status == TEST_FAILED) + return -1; + + fd = create_hugetlbfs_file(&file_stat); + if (fd < 0) + goto create_failure; + printf(PREFIX "HugeTLB seek then read HWPOISON test...\n"); + status = test_hugetlb_read_hwpoison(fd, file_stat.f_bsize, + wr_chunk_sizes[i], true); + printf(PREFIX "HugeTLB seek then read HWPOISON test...%s\n", + status_to_str(status)); + close(fd); + if (status == TEST_FAILED) + return -1; + } + + return 0; + +create_failure: + printf(ERROR_PREFIX "Abort test: failed to create hugetlbfs file\n"); + return -1; +}