From patchwork Mon Jun 3 09:24:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13683598 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7E58C25B75 for ; Mon, 3 Jun 2024 10:46:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 042FA6B008A; Mon, 3 Jun 2024 06:46:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F0DD06B0092; Mon, 3 Jun 2024 06:46:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D95326B008A; Mon, 3 Jun 2024 06:46:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id ACEFB6B0092 for ; Mon, 3 Jun 2024 06:46:06 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4CB1DA1FD4 for ; Mon, 3 Jun 2024 10:46:06 +0000 (UTC) X-FDA: 82189247532.17.14F4AC1 Received: from szxga05-in.huawei.com (szxga05-in.huawei.com [45.249.212.191]) by imf01.hostedemail.com (Postfix) with ESMTP id 360274001A; Mon, 3 Jun 2024 10:46:02 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf01.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717411564; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=C/nwspZXk749iQv0YMsEX+ziQeCcT9+0XfjX55Ig9+c=; b=JZ/IZLM1ot8qo2/PxTAe3935PVtROMZFJMucCY9Pbt/VXs2jldeex5Ah0hhNH3ACVe1xG3 /fH8wsWrX3J4XE/QeUi5R3t2B/s0Cowfm159Xo69KymDQt9aQJ113txlV9gGYW88eJTAKp Q9m2A3QCtEK23PtA1arzHoW0rcZyWUo= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf01.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717411564; a=rsa-sha256; cv=none; b=bPw1EE/2qMWcoNji9R2Fm5+uPPj2qI87XqoSiXV6GtOGeeEMN/qCyF7R04c0C3PS9Dc1qP 4/50qpn+CCyjB9y7r2Dt0zq/Q89MRzIp4f2EGRoZ3QwcHTZlLtbfZ4ZoDjr0xTvg1m/CcM BuctXtuY665tZhkfmit6Thb9Qqj6r/o= Received: from mail.maildlp.com (unknown [172.19.162.112]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4Vt9K76jq4z3554w; Mon, 3 Jun 2024 18:42:19 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id 5EB36140136; Mon, 3 Jun 2024 18:45:59 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 3 Jun 2024 18:45:58 +0800 From: Kefeng Wang To: , CC: Tony Luck , Miaohe Lin , , Matthew Wilcox , David Hildenbrand , Muchun Song , Benjamin LaHaise , , Jiaqi Yan , Hugh Dickins , Vishal Moola , Alistair Popple , Jane Chu , Oscar Salvador , Lance Yang , Kefeng Wang Subject: [PATCH v4 1/6] mm: move memory_failure_queue() into copy_mc_[user]_highpage() Date: Mon, 3 Jun 2024 17:24:34 +0800 Message-ID: <20240603092439.3360652-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240603092439.3360652-1-wangkefeng.wang@huawei.com> References: <20240603092439.3360652-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-Stat-Signature: 3zim7387k14mjidks7f1yjdsmd7xonwh X-Rspamd-Queue-Id: 360274001A X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1717411562-21943 X-HE-Meta: U2FsdGVkX18Rg3PNbygmxX2GbVNr4htsb9EPJlQuhVM2wWYvhTGGINIuGp7ICvHrmRzs7yUxY6ETeTRR3/fGQVODSx/6fv5eAoEI68hxTeQQUpB9RV5upehcrK51TSK5y4mJZvl5lnvEha52YiJdJKIofAondeEJK8zbe+7vl0SFNaIbmxQ1o99Y5ShUwVowV/TeriuOiREI5ucyDVQTnv7+q+0HCZRV4xKkEL2oEnjemDegqiVbMhPMBDdyCI21dd7ISH4x+O23DyhuxX6XraPrXXp6ciRl7Ur2HxzgUSbFAZDj0DlSfjUYJmxPq9x/PpSeUQeGZEzsQs0oUAC7pi5VG5qtG5oddsAcJwN86dIQSecsBFX5wFURauQXJHU9DudjvuIS1vOAWL0Uhqjj5b22nYI3jJ1CHqlmo2eP6nhYL8Jk9kcIAcCOUZvYJAO8Kql5ywigw4rRIe6AT1+RCJpIwtmXk4w5y+b/91bhPbgrtktlPfScu7JuFEKdEvzQCFer8P6ITLLlxDX0y2G7kqDPY0vKXej8hkx5HlqFzqr9I702Vr6HgQP5+7Nb1MXAqstDrg4N0nPST8kAUI0BE8Oa+MUR+B2esifqaw9aoe30SH5jzeu3qoksnz/Nr+KIjatAN6LXr1fiGjmgGgtuBCAmlemEr3iRFlo2Rg8bjsE/tA9fn3i3sebKkgxxFQRab73yrFE3W5Zf8B2VK73dXBcVU6e2tUH9O5prvndEdtdqhu9PcRbW/kJzbJvjifuWqj43rnje7lSFeZ63botD/mUNvKi5hdOjUgnssdsAxoRwz+qWPxmBDgCPCXjovf3Uet0yWj8YMToCAx3XCm1Ggi49hB6f6lgF8c282dYJ1P8bzo12U6J9D98u46u+IIzFVcp9BIfdeaupPWixQvlPmgul+mWiM/7vjsu4OTpJOykM3YF0XdlnCc429Ff19KuUT+HfGTLAxgOyVK4IBin bpZayRzX 5QdttiK9pDZybX7ZSordyGopvGqne7w/eZoV2YKoAmKrenQdc+Twj0Jwqv+S1ETUoQpSDPPQqY9WQmQJXMEuFL4rPz2t127HKdg3+9q1awK6hNy0PC3gT5HVXjSLpKxRSOhuYhxrJVGRx0BSQS/NsYvTxBrUaSknPfa0UtJra4l7phZy1JxZwepbteXrYGLCfJlFbQPLqDXG+AhEgawbk581q6iBx1Io85313ytcK4CMA4STCGQxXQyocrLYPkwwrOKuO+fsPMcGzkqs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There is a memory_failure_queue() call after copy_mc_[user]_highpage(), see callers, eg, CoW/KSM page copy, it is used to mark the source page as h/w poisoned and unmap it from other tasks, and the upcomming poison recover from migrate folio will do the similar thing, so let's move the memory_failure_queue() into the copy_mc_[user]_highpage() instead of adding it into each user, this should also enhance the handling of poisoned page in khugepaged. Signed-off-by: Kefeng Wang Reviewed-by: Jane Chu Reviewed-by: Miaohe Lin --- include/linux/highmem.h | 6 ++++++ mm/ksm.c | 1 - mm/memory.c | 12 +++--------- 3 files changed, 9 insertions(+), 10 deletions(-) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index 00341b56d291..6b0d6f3c8580 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -352,6 +352,9 @@ static inline int copy_mc_user_highpage(struct page *to, struct page *from, kunmap_local(vto); kunmap_local(vfrom); + if (ret) + memory_failure_queue(page_to_pfn(from), 0); + return ret; } @@ -368,6 +371,9 @@ static inline int copy_mc_highpage(struct page *to, struct page *from) kunmap_local(vto); kunmap_local(vfrom); + if (ret) + memory_failure_queue(page_to_pfn(from), 0); + return ret; } #else diff --git a/mm/ksm.c b/mm/ksm.c index 452ac8346e6e..3d95e5a9f301 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -3091,7 +3091,6 @@ struct folio *ksm_might_need_to_copy(struct folio *folio, if (copy_mc_user_highpage(folio_page(new_folio, 0), page, addr, vma)) { folio_put(new_folio); - memory_failure_queue(folio_pfn(folio), 0); return ERR_PTR(-EHWPOISON); } folio_set_dirty(new_folio); diff --git a/mm/memory.c b/mm/memory.c index 63f9f98b47bd..e06de844eaba 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3034,10 +3034,8 @@ static inline int __wp_page_copy_user(struct page *dst, struct page *src, unsigned long addr = vmf->address; if (likely(src)) { - if (copy_mc_user_highpage(dst, src, addr, vma)) { - memory_failure_queue(page_to_pfn(src), 0); + if (copy_mc_user_highpage(dst, src, addr, vma)) return -EHWPOISON; - } return 0; } @@ -6417,10 +6415,8 @@ static int copy_user_gigantic_page(struct folio *dst, struct folio *src, cond_resched(); if (copy_mc_user_highpage(dst_page, src_page, - addr + i*PAGE_SIZE, vma)) { - memory_failure_queue(page_to_pfn(src_page), 0); + addr + i*PAGE_SIZE, vma)) return -EHWPOISON; - } } return 0; } @@ -6437,10 +6433,8 @@ static int copy_subpage(unsigned long addr, int idx, void *arg) struct page *dst = nth_page(copy_arg->dst, idx); struct page *src = nth_page(copy_arg->src, idx); - if (copy_mc_user_highpage(dst, src, addr, copy_arg->vma)) { - memory_failure_queue(page_to_pfn(src), 0); + if (copy_mc_user_highpage(dst, src, addr, copy_arg->vma)) return -EHWPOISON; - } return 0; } From patchwork Mon Jun 3 09:24:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13683601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09E0AC25B76 for ; Mon, 3 Jun 2024 10:46:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7078A6B009C; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5F0756B00A0; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 328076B009A; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 128006B0095 for ; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 9C4491C15B5 for ; Mon, 3 Jun 2024 10:46:08 +0000 (UTC) X-FDA: 82189247616.15.1CCB6E6 Received: from szxga05-in.huawei.com (szxga05-in.huawei.com [45.249.212.191]) by imf08.hostedemail.com (Postfix) with ESMTP id CE500160005; Mon, 3 Jun 2024 10:46:05 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717411566; a=rsa-sha256; cv=none; b=nunD6SWzeg9vNi66WNQdnFOzCLQ3rsNlrPxRrkKZTCxdxTj4dvVj/kDeUONq/4QTdCFzDD 1Yu5P+9o37UD+l9WlPmISaIZZAAAilkQdZmyv3XvZTmbfKr6GYcPVjzUtMqjkF24nIq6W1 Qaa96cA1RsuyTLbOfWI+8HFgkp+AOY8= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717411566; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zb1QdBy78P5/D4Zv5f0vFADJA8vdcgU0jUJCMqW60/A=; b=2PWyW94WNjIbrd0DnrjL1F/p+6bv+2ljgVlkhCE14mIXn0vnOstgqYNLvIdnOScGRymjEs 4jBLEuRa+a0WDdgquAj75sOTUQXFVmhOofALv0t36t81wiel7Ti/bFI3jcpCODfoAyezX9 UEG+g0USUVkm7Z+IWwxSag4Z92BKp1s= Received: from mail.maildlp.com (unknown [172.19.88.214]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4Vt9K852LPz355Z4; Mon, 3 Jun 2024 18:42:20 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id 259391A016C; Mon, 3 Jun 2024 18:46:00 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 3 Jun 2024 18:45:59 +0800 From: Kefeng Wang To: , CC: Tony Luck , Miaohe Lin , , Matthew Wilcox , David Hildenbrand , Muchun Song , Benjamin LaHaise , , Jiaqi Yan , Hugh Dickins , Vishal Moola , Alistair Popple , Jane Chu , Oscar Salvador , Lance Yang , Kefeng Wang Subject: [PATCH v4 2/6] mm: add folio_mc_copy() Date: Mon, 3 Jun 2024 17:24:35 +0800 Message-ID: <20240603092439.3360652-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240603092439.3360652-1-wangkefeng.wang@huawei.com> References: <20240603092439.3360652-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspamd-Queue-Id: CE500160005 X-Rspam-User: X-Rspamd-Server: rspam12 X-Stat-Signature: ia3iwd8prn83xmrsbfnxjkacuaj896ye X-HE-Tag: 1717411565-294039 X-HE-Meta: U2FsdGVkX19E+8KMM6bo9EcF5N4YPDV7jrl3Oc63aLN7WnA3fn3iGSzyP8MurkmbSS36d/ghQy4ZglmqXm5FaerwFC+y4BlqBTO026od4n2XLjZ+kVPZROHZgx3cGDkt/IIP0QETOLEuTZIzJVfxupReDSwLpfwcYeUFyP1uDBcAi1CAEq2SQXifZSvsaih5sw7YWr0ppQH0AuZXzRu4qPmX66S+a/VDEU1GkYVE6zPR6CBTecbKxxorlCS5NBA75i2OZr1U33UJ4LPyBq5xbAuGsirUGM5hP3fYT3blPtLX6/l/90KcskZqwlJqRiVTAk1TuVwOnNPRtViTKM4emOpTFVNTt6KIOANKziXBC6bIxxpahCj2kcnWe+xzOlhl8LdHFr+9EMURshOkGrbtvG4y2keXJby56lPj7iTNhVh2BMrTOBsMIji7Q9dEVi3eqBQK9yAmjwcnqgKaNVeZnnQTi+HQULyxNkyjAQzfAV5376LjhBmqQqv4N2NgRGlcYpwOcK8SqJcvwsETRoC99EgPvi18NklPGs4MK+W78jg4rrlFPeD1dW6AsiCcBV7o02PumVwdziDGY1FOmBT0Yd1q7tV4UhuH17WXWVK7UDvVJxIOAKNTneDST8fG/SVlcjChTIEu6RvGLulW+GTfO4sA0Jgk+3+dLotkzD97ApoOFeOj6q9Z8+ebyF4KSWYeGLczZF15dZJ9Zvy3Di0YFYTIYfUrs+Xl9OAvNOKwAV5wwRFDdJtqz05WcJTMJHAwFcjle9V6vAGICWorIj2p7CrBdtg2sH4P5alugQOE16VB/9qOrq6VMTtXwML7w7J6Cl9jwp6+yUtmFXkYK+jwn61DuBm00YI9Oa/+Y16BAf5Zu1wxtyDgWRbg8GOrGO5r0uaoha/SBjMZFgFFC+JeprZfVPIo5Okh2MabO9QLa+1urER4KiKXs6r2lmQHn4GONOGPNa/fISqXF0EkG5r 7POt+Sda LqLgHpQFD6EqIJnDVh/JsEEQ2AE+Y5jkkEHF6nOhr5vmpQgYPvH7xG7JW6NESYHcj6jAIO2fiObRZuARaJQmszz/CxOSezjGD5jfFeV1H2qBUrrg8mvutLbUdqtXJBemFJdBnpfi5j7fBdOFevjaCshgsrFkv/YLhi1I2TQc2wMchVzfR2YuwNq0cnUgoFE6BvQ1TkmEPqhF8WQWTMmhz87ntRs6XLnmoqsrbHhqaTsxdh1K8mJmsJg7dhjltnWA5kesS9PbKACAE74o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a #MC variant of folio_copy() which uses copy_mc_highpage() to support #MC handled during folio copy, it will be used in folio migration soon. Signed-off-by: Kefeng Wang Reviewed-by: Jane Chu Reviewed-by: Miaohe Lin --- include/linux/mm.h | 1 + mm/util.c | 17 +++++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index c499d5adb748..cc21b2f0cdf8 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1321,6 +1321,7 @@ void put_pages_list(struct list_head *pages); void split_page(struct page *page, unsigned int order); void folio_copy(struct folio *dst, struct folio *src); +int folio_mc_copy(struct folio *dst, struct folio *src); unsigned long nr_free_buffer_pages(void); diff --git a/mm/util.c b/mm/util.c index c9e519e6811f..90ea0c0f10d3 100644 --- a/mm/util.c +++ b/mm/util.c @@ -828,6 +828,23 @@ void folio_copy(struct folio *dst, struct folio *src) } EXPORT_SYMBOL(folio_copy); +int folio_mc_copy(struct folio *dst, struct folio *src) +{ + long nr = folio_nr_pages(src); + long i = 0; + + for (;;) { + if (copy_mc_highpage(folio_page(dst, i), folio_page(src, i))) + return -EHWPOISON; + if (++i == nr) + break; + cond_resched(); + } + + return 0; +} +EXPORT_SYMBOL(folio_mc_copy); + int sysctl_overcommit_memory __read_mostly = OVERCOMMIT_GUESS; int sysctl_overcommit_ratio __read_mostly = 50; unsigned long sysctl_overcommit_kbytes __read_mostly; From patchwork Mon Jun 3 09:24:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13683603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2129C27C44 for ; Mon, 3 Jun 2024 10:46:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E75DB6B009A; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D62DA6B0095; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 995F36B009E; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 4E2696B009B for ; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 103A8C074C for ; Mon, 3 Jun 2024 10:46:09 +0000 (UTC) X-FDA: 82189247658.26.BD7965F Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf18.hostedemail.com (Postfix) with ESMTP id 12AA61C0014; Mon, 3 Jun 2024 10:46:05 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf18.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717411567; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7kqqQZlbr5ucIRBRFQ3mAQ/H/Bv2Ug3/mnfWhZttYO4=; b=d7ugxnVoQnrGZbiRuuKbEpdZ8dKeM3+z0oq/2xvT8EqqE//Ain4GPdszH4YSuv52FHc/ZQ nax6gS1CE+EyXd9cGTP196BMW9lrVBqvm0oNi6YzEcY8fhJkjHa8UuF72cRPVSs5Pw9vfr DIBFzovHBDh1pJxHxNK1L1Ku3LKGzh8= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf18.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717411567; a=rsa-sha256; cv=none; b=iqVcDRxTSC86n2EqWpK9h66NvKa0npWJ7/c7vYxpL3Ny/kIS74i7ZseW75ritidvqAEs4l BkoUSeAela4Qn3CdlYEIdwczNhIgED4gpBePShpSmNtnO7GDez43zd9R59zXisLE4T9tkY xROg6dGA+lY1hILSlF9y8EPkJV2Be9U= Received: from mail.maildlp.com (unknown [172.19.88.194]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4Vt9JB5WhvzmXXT; Mon, 3 Jun 2024 18:41:30 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id EA024140109; Mon, 3 Jun 2024 18:46:00 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 3 Jun 2024 18:46:00 +0800 From: Kefeng Wang To: , CC: Tony Luck , Miaohe Lin , , Matthew Wilcox , David Hildenbrand , Muchun Song , Benjamin LaHaise , , Jiaqi Yan , Hugh Dickins , Vishal Moola , Alistair Popple , Jane Chu , Oscar Salvador , Lance Yang , Kefeng Wang Subject: [PATCH v4 3/6] mm: migrate: split folio_migrate_mapping() Date: Mon, 3 Jun 2024 17:24:36 +0800 Message-ID: <20240603092439.3360652-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240603092439.3360652-1-wangkefeng.wang@huawei.com> References: <20240603092439.3360652-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 12AA61C0014 X-Stat-Signature: nktbt984ah65kzfyyz8k4f5giyngsz3i X-HE-Tag: 1717411565-83264 X-HE-Meta: U2FsdGVkX1/KmQUbxnTQRPpOjHtcwSN7vaSG2gSSWvw2mHHb/ZHjYO68WXIvQtwefzoXPF+90aQYzukRUPgbi49OF2mg3H33daRIzJr6JVyCEARBvOZG/sB/9NoGjacnBZoNploMf6ELBOK2qKaCVeTBZm6vSzdaYpHAscMwHxOlalO9p8rlah0MuaTv4dpCEqFISJqmfzzi00/DhCGjRXeEEu/G6baFTwUJy06s0cUSiw31V/grae5ZkIbLWAOXXobs+7+ij7H6fJZ14kgtLWGY47ImxVaFvALJFdtuUFpFzJgzQPoDm+xSseUFepBGdFt9aZIOkp0n/7rochdeEQXmLLWtB1QBKWUABQ1LAXm50F8YHK1CWb6xMBCBT4Q5s7nQQ12ICUsjd56VKdhaDl1GrttdNodkBmhnv1oJGZD0aTqQDJqkikjn4SvY/2tBP6QYc+wad6viAWumddgGrlgaqiifAdTjJ5fXcKmNHI4ACysbdf8iGDL5za1V/5xzp9Gez/3iHmmBDBkKsEu8vfPcG5GO9sJG1IJxw8q0+Vf1XFTjTGVw2KZ85Inem9gqAAIyU98RJ7maQTUGAZHTlctzMGb+UEKSwtqaKGJj/Zu9KvZQ85Jf6zVP/PuCEVwHxZYZmKo2GUKfj78QVGUqMgnqVy7E2npDaHLronMyk2PUztPf7LFzTDajdHDBBHUzX4xDXsZOREFuLooupi8PUhRf2Hwe7gKoWCYxqaMTbCvc0HfyPymPd6X3d6HsXDcqAyDWsybCGZIO/M0kxKBGsLriuxmG7b4w0spuclf7M0FBPLUnNDDp2UfsRMxRFNYYjpUzPwoALJSNbY96ZfPwC809EAm9KU5vUgxIViqY81p7Q19ftl/FnNOTyA4u7qj0kWdtDq8SB+YcPLU3gCCoi9UInleTVVXxDBG92N39z6xU/DZ8SsSWe/rv+Kl7l7U04x6M5IU4ft2aed8NQD3 4IM5K1jK O3+KyWRvEsn7b5CIolnCPya1GSa/24yZAGLGNXqxA1e8Bf5A8sO4IAI+38rWkr+iFnm4IOdt5akMubSMOBoSY79yueowPJV30PKjXDP7VwBLzkG8ijAVZtcDdjNG1EE2yj94QnxzOSiN4AeP8k2iHuipBlDW4w+hTGw252vFJvporVpNzlH6JHeAb3QRClIydJKBbKCmiKDMdcbC5g/22bEdtVFF7eApZ5quUawgPNzZF8STukv9T+jyU8QedO2ACswBBgwxTAQmB3v0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The folio refcount check for !mapping and folio_ref_freeze() for mapping are moved out of the original folio_migrate_mapping(), and there is no turning back for the new __folio_migrate_mapping(), also update comment from page to folio. Note, the folio_ref_freeze() is moved out of xas_lock_irq(), Since the folio is already isolated and locked during migration, so suppose that there is no functional change. Signed-off-by: Kefeng Wang Reviewed-by: Jane Chu --- mm/migrate.c | 63 ++++++++++++++++++++++++++-------------------------- 1 file changed, 32 insertions(+), 31 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index e04b451c4289..e930376c261a 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -393,50 +393,36 @@ static int folio_expected_refs(struct address_space *mapping, } /* - * Replace the page in the mapping. + * Replace the folio in the mapping. * * The number of remaining references must be: - * 1 for anonymous pages without a mapping - * 2 for pages with a mapping - * 3 for pages with a mapping and PagePrivate/PagePrivate2 set. + * 1 for anonymous folios without a mapping + * 2 for folios with a mapping + * 3 for folios with a mapping and PagePrivate/PagePrivate2 set. */ -int folio_migrate_mapping(struct address_space *mapping, - struct folio *newfolio, struct folio *folio, int extra_count) +static void __folio_migrate_mapping(struct address_space *mapping, + struct folio *newfolio, struct folio *folio, int expected_cnt) { XA_STATE(xas, &mapping->i_pages, folio_index(folio)); struct zone *oldzone, *newzone; - int dirty; - int expected_count = folio_expected_refs(mapping, folio) + extra_count; long nr = folio_nr_pages(folio); long entries, i; + int dirty; if (!mapping) { - /* Anonymous page without mapping */ - if (folio_ref_count(folio) != expected_count) - return -EAGAIN; - - /* No turning back from here */ + /* Anonymous folio without mapping */ newfolio->index = folio->index; newfolio->mapping = folio->mapping; if (folio_test_swapbacked(folio)) __folio_set_swapbacked(newfolio); - - return MIGRATEPAGE_SUCCESS; + return; } oldzone = folio_zone(folio); newzone = folio_zone(newfolio); xas_lock_irq(&xas); - if (!folio_ref_freeze(folio, expected_count)) { - xas_unlock_irq(&xas); - return -EAGAIN; - } - - /* - * Now we know that no one else is looking at the folio: - * no turning back from here. - */ + /* Now we know that no one else is looking at the folio */ newfolio->index = folio->index; newfolio->mapping = folio->mapping; folio_ref_add(newfolio, nr); /* add cache reference */ @@ -452,7 +438,7 @@ int folio_migrate_mapping(struct address_space *mapping, entries = 1; } - /* Move dirty while page refs frozen and newpage not yet exposed */ + /* Move dirty while folio refs frozen and newfolio not yet exposed */ dirty = folio_test_dirty(folio); if (dirty) { folio_clear_dirty(folio); @@ -466,22 +452,22 @@ int folio_migrate_mapping(struct address_space *mapping, } /* - * Drop cache reference from old page by unfreezing - * to one less reference. + * Since old folio's refcount freezed, now drop cache reference from + * old folio by unfreezing to one less reference. * We know this isn't the last reference. */ - folio_ref_unfreeze(folio, expected_count - nr); + folio_ref_unfreeze(folio, expected_cnt - nr); xas_unlock(&xas); /* Leave irq disabled to prevent preemption while updating stats */ /* * If moved to a different zone then also account - * the page for that zone. Other VM counters will be + * the folio for that zone. Other VM counters will be * taken care of when we establish references to the - * new page and drop references to the old page. + * new folio and drop references to the old folio. * - * Note that anonymous pages are accounted for + * Note that anonymous folios are accounted for * via NR_FILE_PAGES and NR_ANON_MAPPED if they * are mapped to swap space. */ @@ -518,7 +504,22 @@ int folio_migrate_mapping(struct address_space *mapping, } } local_irq_enable(); +} + +int folio_migrate_mapping(struct address_space *mapping, struct folio *newfolio, + struct folio *folio, int extra_count) +{ + int expected_cnt = folio_expected_refs(mapping, folio) + extra_count; + + if (!mapping) { + if (folio_ref_count(folio) != expected_cnt) + return -EAGAIN; + } else { + if (!folio_ref_freeze(folio, expected_cnt)) + return -EAGAIN; + } + __folio_migrate_mapping(mapping, newfolio, folio, expected_cnt); return MIGRATEPAGE_SUCCESS; } EXPORT_SYMBOL(folio_migrate_mapping); From patchwork Mon Jun 3 09:24:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13683600 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3A17C25B75 for ; Mon, 3 Jun 2024 10:46:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C516C6B0092; Mon, 3 Jun 2024 06:46:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD9C56B0093; Mon, 3 Jun 2024 06:46:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A05306B0095; Mon, 3 Jun 2024 06:46:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 751266B0092 for ; Mon, 3 Jun 2024 06:46:08 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 1BD381607DA for ; Mon, 3 Jun 2024 10:46:08 +0000 (UTC) X-FDA: 82189247616.15.76436A0 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf25.hostedemail.com (Postfix) with ESMTP id 3DA28A000E; Mon, 3 Jun 2024 10:46:04 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717411566; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=evrR0enGu4CVldhF9V8cpmPyJ0bXapKDS3IF6y1p7A8=; b=RBYEwjbZUxHoQ9TsVwVoLaAu17XdaB9vVR4QBZtspijocAlK86Kc2+Wg/k//M+2kj8c59W jBc7OUcpNj1FgXHe4GsFbqm++zPgt05Lw64Uj5ZwUqYYSIAtaCg1RKw+uFMLI2hTRWuEz1 qs0VqhWDWUWxkJNNdHo+xQK3B4FSQCk= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717411566; a=rsa-sha256; cv=none; b=VhEC0alVip4ADtaTlcJiqYS4RWPsW3/QLHV2+pPcqlFfZFh0Aguq1athiJ/Artmh0Y4qn0 /yVuktmP+Vi2m4iEI0gmemsPKBQ21Rqkb/uwFCuhYw4W2xSA3LjRdNb+HMqrT1JPQ2Q/mm HpOUk8zGoFIGalN552qG51gh2NOgRWA= Received: from mail.maildlp.com (unknown [172.19.163.252]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4Vt9Jv1Pm1zxNdS; Mon, 3 Jun 2024 18:42:07 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id BE39E180A9C; Mon, 3 Jun 2024 18:46:01 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 3 Jun 2024 18:46:00 +0800 From: Kefeng Wang To: , CC: Tony Luck , Miaohe Lin , , Matthew Wilcox , David Hildenbrand , Muchun Song , Benjamin LaHaise , , Jiaqi Yan , Hugh Dickins , Vishal Moola , Alistair Popple , Jane Chu , Oscar Salvador , Lance Yang , Kefeng Wang Subject: [PATCH v4 4/6] mm: migrate: support poisoned recover from migrate folio Date: Mon, 3 Jun 2024 17:24:37 +0800 Message-ID: <20240603092439.3360652-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240603092439.3360652-1-wangkefeng.wang@huawei.com> References: <20240603092439.3360652-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 3DA28A000E X-Stat-Signature: njbpqj84ycsu88tamh68iyer7bp99kty X-HE-Tag: 1717411564-368138 X-HE-Meta: U2FsdGVkX18egE1RYzXkTeG2FEN2sa03DUXmBnEogi09rbqTTr2fdelVWnh6K5kdwmtVS6sl8mNE8XcuKsyMkjjq5m6QAMYnX0WXO/UoxMqi+HDaHW4ByumdWDVwX0THL/YFiTyWjI2kQLnsj85wgaT46r2xAlCRGVo/g6XOa0UpZOpdWtJBiNV9FdvX19ga1zvkWn+SBmFk3C5GthSueTnysUM3lGDwIXUfu4pY4G2bj23kKoKl+lFD8to3VFbMx/UFv0/87pOOwQLCV28KNpB+atER04eZ8JaSS2QcnIsVq0v214FLpxOBgRopxMoQzQHtKiURn5GNxsx4c3gdQATV20Tp3HWk5rCx7hhPatHaUlEpANg0w/FGTeF9Jt9+BEI+en4lSR5xCGPI0ihNe4j19u20szURPGzuy4TEhmDPlKg/8YS1K3dr0sttvSG8MDG+Op0PZlYhx0tIADbPP/xBukU8fCwHOuBjBa17nBXfxuB5tdLUgBYwyw0FAsaB272eZViwnaf3jaRB3OzxvcZG+PU51juwBoJXnfWD2NOKQJmmDYasEB7jD2jz2NVwT/9jxYw4D1U7gWWB3sTlT0/6dfG0U9vEmesysBI6j3W/1vz41yQEPrKxAyA/DxE7vfTEq81SzOp30uoXB3Md4h7UsEtwDz/l4Fzfn9A0/oPtzpH3Sgt/OLC3ywCma9QsjrOvr7SDPGvMm6yQ4sh7+po5XqpE7ckbGVCtD6kQI/MQ7weS8IvhXf7eEgHYFBzehnvqBeAsdDUZHTcG2XPZR68xjZK5SNR83UAnLhmDT9Wbccb9ufm8HOVEouR0omTQ5BsqlZQvuuRi9ly/nxgP91/S9XdUvttau/wOqF8/zrb86X7n/pc1hkOh5ci/Ro48XwWMnrF0J+B3xAqhbZifGV+iAuHljauarhH8L72YigO59WV9r5K/a0RxGnIQIoP55wDZB9pceuF2uvktEfy LNt/e/4D 0I6Lq3+uMkogzLf+3xRnC2cIRyK0uYVMNDRtV+/Usj+SVpxcs4c+I4PZr4OgnqdtR+6r1lmgQCBPgf0V8ZaaeoAAhSS/Q2gYQpRaHsYV0ZXvdK4geqJTGB0qGzv0K5YvKto9aPbHUNAVO6mcRV+vuC11YctyTfkUrUFfxYSjFNBsaR0PjQtNrfsN9Z3lclfamBmd66+EbzcT5moKMYIlntwI12bf1SUirCQXc X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The folio migration is widely used in kernel, memory compaction, memory hotplug, soft offline page, numa balance, memory demote/promotion, etc, but once access a poisoned source folio when migrating, the kerenl will panic. There is a mechanism in the kernel to recover from uncorrectable memory errors, ARCH_HAS_COPY_MC, which is already used in other core-mm paths, eg, CoW, khugepaged, coredump, ksm copy, see copy_mc_to_{user,kernel}, copy_mc_{user_}highpage callers. In order to support poisoned folio copy recover from migrate folio, we chose to make folio migration tolerant of memory failures and return error for folio migration, because folio migration is no guarantee of success, this could avoid the similar panic shown below. CPU: 1 PID: 88343 Comm: test_softofflin Kdump: loaded Not tainted 6.6.0 pc : copy_page+0x10/0xc0 lr : copy_highpage+0x38/0x50 ... Call trace: copy_page+0x10/0xc0 folio_copy+0x78/0x90 migrate_folio_extra+0x54/0xa0 move_to_new_folio+0xd8/0x1f0 migrate_folio_move+0xb8/0x300 migrate_pages_batch+0x528/0x788 migrate_pages_sync+0x8c/0x258 migrate_pages+0x440/0x528 soft_offline_in_use_page+0x2ec/0x3c0 soft_offline_page+0x238/0x310 soft_offline_page_store+0x6c/0xc0 dev_attr_store+0x20/0x40 sysfs_kf_write+0x4c/0x68 kernfs_fop_write_iter+0x130/0x1c8 new_sync_write+0xa4/0x138 vfs_write+0x238/0x2d8 ksys_write+0x74/0x110 Signed-off-by: Kefeng Wang Reviewed-by: Jane Chu --- mm/migrate.c | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index e930376c261a..28aa9da95781 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -663,16 +663,29 @@ static int __migrate_folio(struct address_space *mapping, struct folio *dst, struct folio *src, void *src_private, enum migrate_mode mode) { - int rc; + int ret, expected_cnt = folio_expected_refs(mapping, src); - rc = folio_migrate_mapping(mapping, dst, src, 0); - if (rc != MIGRATEPAGE_SUCCESS) - return rc; + if (!mapping) { + if (folio_ref_count(src) != expected_cnt) + return -EAGAIN; + } else { + if (!folio_ref_freeze(src, expected_cnt)) + return -EAGAIN; + } + + ret = folio_mc_copy(dst, src); + if (unlikely(ret)) { + if (mapping) + folio_ref_unfreeze(src, expected_cnt); + return ret; + } + + __folio_migrate_mapping(mapping, dst, src, expected_cnt); if (src_private) folio_attach_private(dst, folio_detach_private(src)); - folio_migrate_copy(dst, src); + folio_migrate_flags(dst, src); return MIGRATEPAGE_SUCCESS; } From patchwork Mon Jun 3 09:24:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13683604 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26BD3C25B75 for ; Mon, 3 Jun 2024 10:46:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1A3626B0095; Mon, 3 Jun 2024 06:46:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E77F36B009B; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C7DE96B00A0; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 587856B0098 for ; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D905E1C0DC2 for ; Mon, 3 Jun 2024 10:46:08 +0000 (UTC) X-FDA: 82189247616.16.1F5DB41 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf14.hostedemail.com (Postfix) with ESMTP id EA163100014; Mon, 3 Jun 2024 10:46:05 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717411566; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gwSPhmkkIOKfH+OEGxsgxRPTgKSYi8a9uVM17S8Pvu4=; b=rm0nedq0C9Cn12Jmd9Ql4+RhmpULaUKap+lDTc1Q4IIVyC30qn4cZjjJGps5jAT/JZisOX Th0saBVTwqnhzIdVnaPipCLBKsGM8w+0IKJDlt9LcjYKBHykK/qeA1kD6Fj44IqFHMQ7dJ e0kkUM1H64xSwoc1Ve+zpqWnVYngAE8= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717411566; a=rsa-sha256; cv=none; b=iPrd7jSw6yr9Xi9VB+bvcxZr6lli/nvduunoRMLWZcMjbh8e+nlqM78BYzhRCJy/BbksNk T2nWKZ7i2nGKEqHGjK7JIxE5BoPAaCN2rTW0RxQFaEs4cH/h69AnIM2KchIpnewSDraFrQ Gy1HryAvDctwF6NsKCuC7H4YaGpARSY= Received: from mail.maildlp.com (unknown [172.19.163.174]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4Vt9Jv6pLRzxPtp; Mon, 3 Jun 2024 18:42:07 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id 8692D140417; Mon, 3 Jun 2024 18:46:02 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 3 Jun 2024 18:46:01 +0800 From: Kefeng Wang To: , CC: Tony Luck , Miaohe Lin , , Matthew Wilcox , David Hildenbrand , Muchun Song , Benjamin LaHaise , , Jiaqi Yan , Hugh Dickins , Vishal Moola , Alistair Popple , Jane Chu , Oscar Salvador , Lance Yang , Kefeng Wang Subject: [PATCH v4 5/6] fs: hugetlbfs: support poison recover from hugetlbfs_migrate_folio() Date: Mon, 3 Jun 2024 17:24:38 +0800 Message-ID: <20240603092439.3360652-6-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240603092439.3360652-1-wangkefeng.wang@huawei.com> References: <20240603092439.3360652-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: EA163100014 X-Stat-Signature: ryku4rwa5a1pg91d7wnqmkeosud9fox3 X-HE-Tag: 1717411565-662172 X-HE-Meta: U2FsdGVkX18CfqA7dadh8h74HDgdq45I3ckyJoteb8Idk+JWREoGYO2Y/iU1Kfo2FvyR2RrcmbuM+RP6OBLPoAusdtZ8Fw3G5nlPxCCzFfBeJL/YV4yAb745PJDLW8Zgc0zhd4gUZC87Wc6HsW8D246etDapBl/MW00ghTuS5hehSUo0G7bq2zdU/A1rQtNlntZkPCdaZureTjwZuTrnPRpvG+XCnZvuAgZ0242noQUlj79WMoEgUuCtWcUaSRV8STeFl87pk5STiSoQ+ZCtzuhClAP6RPUT+Ic8HYf1KJIIAz/7jQKSOSthOCVIvBertPKpzKDEbr4iK3loG4EWXG3o5/fXN+5jVrB6iDwy6RaYPCp/I2DnniOtnQOU2CJ83TaxA8lwQ3vxaO0xnv72IsOX7pGWxeg5zivrm9rt2ynTXlGD3eVBwjO7wV1P9vaXmOkyqh8aXaaoX6b52aImdt5XG2O+BJ031ctE4K/i4HL0M4dwNVskZJkuO3fNclv75ph6Gf/mpQj4WZIM7HerqHf+hnHF+V74mHQDVar9A40ecD/Wi6T+QmHex9ogfMnkgQ7rkWvfJVRSjBjeZIAfhGQfy8Mt+xv+5CZslCzCrkxSEETzkHFp4yTOztCIaiIhwZiU0IVxY0cNsAkfoGn8PPz0lsZjCvWs9A9bt0gtUVKQPI63vXiAj8y5+xXXVR5kdfsjSQdea3PDICs2ZSkhxB1a9f5EpQbAdJtEkGjffXRLzch75UeOmPw2Zj9BEsG8gqHZrRmVA69dHYY3x8MbvtI4CYLxwqvRyfKLQ9YaSRtZ89ztgs4khbxRzyU9CfczcYlMovwzIe/r0Nqde7OLBLhaLaIzHCGDXi7JmCb+uSt+xg44gAG77nLBob5svd3lBbLzlMOn1Y32vYv7myyj3myi0CKWna+QKdwM78iRmwdzAec+Q/fFg0GflIW+VQj0S4zdojrDe/Ni1GwBqQI n06PDNeP 11+nc9A9uL/gjF74/k21Z3BQ8ulO2TiyMbBiMDsXyivJU77/s7PbyDPkej9wmoRRYoy6bOviGYQtqCL4xyNyuQNO5WQ3xktdl5CGx6FuhqAHp56Q94nBt+q7ZdhYl54PQ1Co8qs9BN4cEo7r/LtrJd5EXBXWmabqnpBjx+lnxny/1Y/bIZMUpm5eLo5yeszUISXDaW1+7y49HV/Q9pZCBR4e+Z9U7PYUQxRNlCIsfATqjqJpTOuWM8ECW8eVoekb9ralkqkl+UOyuYn0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is similar to __migrate_folio(), use folio_mc_copy() in HugeTLB folio migration to avoid panic when copy from poisoned folio. Signed-off-by: Kefeng Wang Reviewed-by:  Jane Chu --- fs/hugetlbfs/inode.c | 2 +- mm/migrate.c | 14 +++++++++----- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 6df794ed4066..1107e5aa8343 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -1128,7 +1128,7 @@ static int hugetlbfs_migrate_folio(struct address_space *mapping, hugetlb_set_folio_subpool(src, NULL); } - folio_migrate_copy(dst, src); + folio_migrate_flags(dst, src); return MIGRATEPAGE_SUCCESS; } diff --git a/mm/migrate.c b/mm/migrate.c index 28aa9da95781..e9b52a86f539 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -532,15 +532,19 @@ int migrate_huge_page_move_mapping(struct address_space *mapping, struct folio *dst, struct folio *src) { XA_STATE(xas, &mapping->i_pages, folio_index(src)); - int expected_count; + int ret, expected_count = folio_expected_refs(mapping, src); - xas_lock_irq(&xas); - expected_count = folio_expected_refs(mapping, src); - if (!folio_ref_freeze(src, expected_count)) { - xas_unlock_irq(&xas); + if (!folio_ref_freeze(src, expected_count)) return -EAGAIN; + + ret = folio_mc_copy(dst, src); + if (unlikely(ret)) { + folio_ref_unfreeze(src, expected_count); + return ret; } + xas_lock_irq(&xas); + dst->index = src->index; dst->mapping = src->mapping; From patchwork Mon Jun 3 09:24:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13683602 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FD07C25B75 for ; Mon, 3 Jun 2024 10:46:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF1196B0099; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A55A56B0095; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6FCAD6B0095; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 45E116B0099 for ; Mon, 3 Jun 2024 06:46:09 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 0A5A41C128E for ; Mon, 3 Jun 2024 10:46:09 +0000 (UTC) X-FDA: 82189247658.22.64B8481 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf25.hostedemail.com (Postfix) with ESMTP id 51FC6A0004; Mon, 3 Jun 2024 10:46:05 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717411567; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RwVzLGUCkaxnY+8tSjDUTzVT3nPZf3ETs1o/YpP6JBc=; b=fSsVJG1bp+rSGHOBJNAGKPLWvJ/Dcci6dfs+82GfSJcqd0Y5gtuDS4xrs90KNpdvcMLTNT XRkc46IDNPlVadWNwvitlit94JEjsehWLy5CO5wt+gTYH1eWFvrr2BrYdOk1heJ1DlZl4B pg0ZWyaenu3GSbvj7sX8ZBhq6dxkR+w= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717411567; a=rsa-sha256; cv=none; b=vDLMEkF1Yhrofeuc3/9g4P/uf8XoqJwIC0k6fXPb6g7nrRv5ExTtX7etUpVisTYKjusBuY B0FV+w5w/J8mPYzbkakTv1lEFj3lsYM3zl12UtExmwMOw95coFTMOKu4FhYs0zwLwnpT82 n+fI+AtckzkPDvPSGMTmiFUi+Ji9xXM= Received: from mail.maildlp.com (unknown [172.19.88.194]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4Vt9Mr1JZvzclBm; Mon, 3 Jun 2024 18:44:40 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id 5CF99140109; Mon, 3 Jun 2024 18:46:03 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 3 Jun 2024 18:46:02 +0800 From: Kefeng Wang To: , CC: Tony Luck , Miaohe Lin , , Matthew Wilcox , David Hildenbrand , Muchun Song , Benjamin LaHaise , , Jiaqi Yan , Hugh Dickins , Vishal Moola , Alistair Popple , Jane Chu , Oscar Salvador , Lance Yang , Kefeng Wang Subject: [PATCH v4 6/6] mm: migrate: remove folio_migrate_copy() Date: Mon, 3 Jun 2024 17:24:39 +0800 Message-ID: <20240603092439.3360652-7-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240603092439.3360652-1-wangkefeng.wang@huawei.com> References: <20240603092439.3360652-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 51FC6A0004 X-Stat-Signature: y5ciyn43kiiaqndph5u4jwn16qggiaw7 X-HE-Tag: 1717411565-160486 X-HE-Meta: U2FsdGVkX18N9ftrwxNnIu6P3GjLbaaP7/KsDv3i6dPfyxmZ6HjQK61xbIZM14G0+R8uc/HTG/TfDoFNMwDjl/iaBnMS8Q/ZK0/Syds9e/djhJGwJpsQcwj0vKYAXsdW2JQzryh5ofxHq9uufGztPU3RibVMS39TXhfyzqQX5spS/5ELPNKG+oJLg1Et8PweMFOKnI9oMRkW6HrLHMaxHMa06P/yuE7x78ME8pVSpiT+FAhrXXz9B8YJqR8wlYjTn+BheqAWJ/IQ+cRszwvnRTY3cofmseJAsIcg/bO6jo+yzUy+Oaomr4so6Zm/rgvfDiafdKT6M2TTlO4/saA9LYr3wlVHbQk6UQ+CweKcN0FwARHLRlbc+VpyAbspIGybmGPXDUFvPBmHZ8TWgCtGV3Mvl1gByxk87xyh010FwFsQ+9DoGYJcaQRUslk63vp/YjC58faCa3pjvz5feeRE1xx+S7hFpglUJkuqV+5d3Z+zYKg6TBwR5bjnWIfOjqmLzvZbxoer2dhYXI+M5H8hJgTA1U23/u/zeS0JymuuK/2gbAxi7qJ8zE46Yn7ZhGDtIAdlP6lxr/b94Jm/i2usqNn5AjPcERFapvYrvOuAINkCkw2irVJSmSSiOZ+NMrnh025a07RBbJCZy4GpcrLybHYWy1u0gFc+JGQrEL5qQ7EpGx1fdSp8uQhdsYnUnCKIW5HmTLX7eDIkGHfiimll1PsswQocyIQc2lzKirFlshcRfnQnWY/KPUJ4tI0rkplmKGRIMQE6ssI+kArQ8YFnkcGOu8RvC1ooCke9BdPKzVXe5qFoTpOz00TF2etaM3ncJSfJhfaRVA/XlrfoZdPmgP3AO3HkU0QuLqv35hXXMzbDMAw5/NXDDhn4Bq3LWnKLMbt4I+CCANJayVp7p525sq5z2dLKxmOMPaPfNgqJj6kdUfyzB5ZzfL1MzRjbOfvTfWNqdgVK/oCxIsqFMLc RDh4a+tn mX96IZduSScAuH2ZjzC0LlNCzrNh2n93D4aihGg21F62xcxs6LQv8nVu6T/DIHix0w9A2KOvXnKNmD2EaySlJ6AthhCwds0NoJdUEODOROb3Ye257YuTjg+cKeX2h7/lx5iSimqvcRDEhV0xqL4xlGaienpOgUkLRShhgUHufblFedWoKK+MzoIV1apZA4dSXsr5BFt85BfgdvQsHzeAIrK4HUx3Ue7ncyv0V/ru1VpyfbI1MQyzQCWwqyyDWMXvPd8M45YJV/LGN4rA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The folio_migrate_copy() is just a wrapper of folio_copy() and folio_migrate_flags(), it is simple and only aio use it for now, unfold it and remove folio_migrate_copy(). Signed-off-by: Kefeng Wang Reviewed-by:  Jane Chu --- fs/aio.c | 3 ++- include/linux/migrate.h | 1 - mm/migrate.c | 7 ------- 3 files changed, 2 insertions(+), 9 deletions(-) diff --git a/fs/aio.c b/fs/aio.c index 07ff8bbdcd2a..bcee11fcb08b 100644 --- a/fs/aio.c +++ b/fs/aio.c @@ -455,7 +455,8 @@ static int aio_migrate_folio(struct address_space *mapping, struct folio *dst, * events from being lost. */ spin_lock_irqsave(&ctx->completion_lock, flags); - folio_migrate_copy(dst, src); + folio_copy(dst, src); + folio_migrate_flags(dst, src); BUG_ON(ctx->ring_folios[idx] != src); ctx->ring_folios[idx] = dst; spin_unlock_irqrestore(&ctx->completion_lock, flags); diff --git a/include/linux/migrate.h b/include/linux/migrate.h index 517f70b70620..f9d92482d117 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -76,7 +76,6 @@ int migrate_huge_page_move_mapping(struct address_space *mapping, void migration_entry_wait_on_locked(swp_entry_t entry, spinlock_t *ptl) __releases(ptl); void folio_migrate_flags(struct folio *newfolio, struct folio *folio); -void folio_migrate_copy(struct folio *newfolio, struct folio *folio); int folio_migrate_mapping(struct address_space *mapping, struct folio *newfolio, struct folio *folio, int extra_count); diff --git a/mm/migrate.c b/mm/migrate.c index e9b52a86f539..fe6ea3fb896e 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -652,13 +652,6 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio) } EXPORT_SYMBOL(folio_migrate_flags); -void folio_migrate_copy(struct folio *newfolio, struct folio *folio) -{ - folio_copy(newfolio, folio); - folio_migrate_flags(newfolio, folio); -} -EXPORT_SYMBOL(folio_migrate_copy); - /************************************************************ * Migration functions ***********************************************************/